[WARNING  | transformer_engine.pytorch.dot_product_attention.utils]: Supported flash-attn versions are >= 2.1.1, <= 2.8.0.post2. Found flash-attn 2.8.3.
[transformers] `torch_dtype` is deprecated! Use `dtype` instead!
Loading tokenizer...
Loading model BF16 on ROCm...

Loading weights:   0%|          | 0/398 [00:00<?, ?it/s]
Loading weights:   0%|          | 1/398 [00:00<03:05,  2.14it/s]
Loading weights:   4%|▍         | 15/398 [00:00<00:11, 34.08it/s]
Loading weights:   8%|▊         | 33/398 [00:00<00:05, 69.83it/s]
Loading weights:  12%|█▏        | 48/398 [00:00<00:03, 89.45it/s]
Loading weights:  17%|█▋        | 66/398 [00:00<00:02, 112.45it/s]
Loading weights:  20%|██        | 81/398 [00:00<00:02, 120.64it/s]
Loading weights:  25%|██▍       | 99/398 [00:01<00:02, 135.06it/s]
Loading weights:  29%|██▉       | 115/398 [00:01<00:02, 132.86it/s]
Loading weights:  34%|███▍      | 135/398 [00:01<00:01, 148.37it/s]
Loading weights:  38%|███▊      | 151/398 [00:01<00:01, 146.30it/s]
Loading weights:  42%|████▏     | 169/398 [00:01<00:01, 148.89it/s]
Loading weights:  47%|████▋     | 187/398 [00:01<00:01, 155.02it/s]
Loading weights:  51%|█████     | 203/398 [00:01<00:01, 145.79it/s]
Loading weights:  56%|█████▌    | 223/398 [00:01<00:01, 157.72it/s]
Loading weights:  60%|██████    | 240/398 [00:02<00:01, 155.45it/s]
Loading weights:  65%|██████▍   | 257/398 [00:02<00:00, 153.30it/s]
Loading weights:  69%|██████▉   | 275/398 [00:02<00:00, 159.04it/s]
Loading weights:  73%|███████▎  | 292/398 [00:02<00:00, 151.90it/s]
Loading weights:  78%|███████▊  | 309/398 [00:02<00:00, 153.89it/s]
Loading weights:  82%|████████▏ | 325/398 [00:02<00:00, 153.24it/s]
Loading weights:  86%|████████▌ | 342/398 [00:02<00:00, 155.38it/s]
Loading weights:  90%|████████▉ | 358/398 [00:02<00:00, 150.56it/s]
Loading weights:  95%|█████████▍| 377/398 [00:02<00:00, 159.75it/s]
Loading weights:  99%|█████████▉| 394/398 [00:03<00:00, 153.15it/s]
Loading weights: 100%|██████████| 398/398 [00:03<00:00, 131.76it/s]
[transformers] warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
Attaching LoRA...
trainable params: 132,120,576 || all params: 4,154,588,672 || trainable%: 3.1801
Loaded 1,807,941 conversations from /workspace/datasets/cybersec/cybersec_master.jsonl
Prepared 1,807,941 training texts

Adding EOS to train dataset:   0%|          | 0/1807941 [00:00<?, ? examples/s]
Adding EOS to train dataset:   0%|          | 5000/1807941 [00:00<00:36, 49293.37 examples/s]
Adding EOS to train dataset:   1%|          | 10000/1807941 [00:00<00:36, 49563.51 examples/s]
Adding EOS to train dataset:   1%|          | 15060/1807941 [00:00<00:35, 50033.91 examples/s]
Adding EOS to train dataset:   1%|          | 20174/1807941 [00:00<00:35, 50467.16 examples/s]
Adding EOS to train dataset:   2%|▏         | 27278/1807941 [00:00<00:36, 49042.57 examples/s]
Adding EOS to train dataset:   2%|▏         | 32287/1807941 [00:00<00:35, 49357.55 examples/s]
Adding EOS to train dataset:   2%|▏         | 39671/1807941 [00:00<00:35, 49305.03 examples/s]
Adding EOS to train dataset:   2%|▏         | 44622/1807941 [00:00<00:35, 49359.98 examples/s]
Adding EOS to train dataset:   3%|▎         | 51088/1807941 [00:01<00:37, 47046.62 examples/s]
Adding EOS to train dataset:   3%|▎         | 56116/1807941 [00:01<00:36, 47886.22 examples/s]
Adding EOS to train dataset:   3%|▎         | 61000/1807941 [00:01<00:36, 48126.46 examples/s]
Adding EOS to train dataset:   4%|▎         | 66000/1807941 [00:01<00:35, 48573.00 examples/s]
Adding EOS to train dataset:   4%|▍         | 70992/1807941 [00:01<00:35, 48955.52 examples/s]
Adding EOS to train dataset:   4%|▍         | 76000/1807941 [00:01<00:35, 49006.43 examples/s]
Adding EOS to train dataset:   4%|▍         | 81000/1807941 [00:01<00:35, 49049.58 examples/s]
Adding EOS to train dataset:   5%|▍         | 86000/1807941 [00:01<00:35, 49080.35 examples/s]
Adding EOS to train dataset:   5%|▌         | 92547/1807941 [00:01<00:38, 45023.59 examples/s]
Adding EOS to train dataset:   5%|▌         | 97586/1807941 [00:02<00:37, 46219.96 examples/s]
Adding EOS to train dataset:   6%|▌         | 102595/1807941 [00:02<00:36, 47260.33 examples/s]
Adding EOS to train dataset:   6%|▌         | 107682/1807941 [00:02<00:35, 48261.68 examples/s]
Adding EOS to train dataset:   6%|▌         | 112612/1807941 [00:02<00:34, 48553.38 examples/s]
Adding EOS to train dataset:   7%|▋         | 119922/1807941 [00:02<00:34, 48233.85 examples/s]
Adding EOS to train dataset:   7%|▋         | 127133/1807941 [00:02<00:34, 48176.22 examples/s]
Adding EOS to train dataset:   7%|▋         | 132000/1807941 [00:02<00:34, 47983.07 examples/s]
Adding EOS to train dataset:   8%|▊         | 137000/1807941 [00:02<00:34, 48427.94 examples/s]
Adding EOS to train dataset:   8%|▊         | 144149/1807941 [00:02<00:34, 48150.88 examples/s]
Adding EOS to train dataset:   8%|▊         | 149001/1807941 [00:03<00:34, 48239.12 examples/s]
Adding EOS to train dataset:   9%|▊         | 153920/1807941 [00:03<00:34, 48492.70 examples/s]
Adding EOS to train dataset:   9%|▉         | 158862/1807941 [00:03<00:33, 48747.09 examples/s]
Adding EOS to train dataset:   9%|▉         | 163834/1807941 [00:03<00:33, 49022.06 examples/s]
Adding EOS to train dataset:   9%|▉         | 171029/1807941 [00:03<00:33, 48621.29 examples/s]
Adding EOS to train dataset:  10%|▉         | 177488/1807941 [00:03<00:38, 41983.78 examples/s]
Adding EOS to train dataset:  10%|█         | 182629/1807941 [00:03<00:36, 43980.27 examples/s]
Adding EOS to train dataset:  10%|█         | 187438/1807941 [00:03<00:36, 44834.87 examples/s]
Adding EOS to train dataset:  11%|█         | 194552/1807941 [00:04<00:35, 45726.04 examples/s]
Adding EOS to train dataset:  11%|█         | 199609/1807941 [00:04<00:34, 46871.81 examples/s]
Adding EOS to train dataset:  11%|█▏        | 204513/1807941 [00:04<00:33, 47434.64 examples/s]
Adding EOS to train dataset:  12%|█▏        | 209554/1807941 [00:04<00:33, 48005.11 examples/s]
Adding EOS to train dataset:  12%|█▏        | 214553/1807941 [00:04<00:32, 48510.29 examples/s]
Adding EOS to train dataset:  12%|█▏        | 219599/1807941 [00:04<00:32, 48981.69 examples/s]
Adding EOS to train dataset:  12%|█▏        | 224683/1807941 [00:04<00:31, 49515.65 examples/s]
Adding EOS to train dataset:  13%|█▎        | 229729/1807941 [00:04<00:31, 49788.84 examples/s]
Adding EOS to train dataset:  13%|█▎        | 235050/1807941 [00:04<00:30, 50792.72 examples/s]
Adding EOS to train dataset:  13%|█▎        | 240723/1807941 [00:04<00:29, 52547.03 examples/s]
Adding EOS to train dataset:  14%|█▎        | 246000/1807941 [00:05<00:29, 52548.15 examples/s]
Adding EOS to train dataset:  14%|█▍        | 251489/1807941 [00:05<00:29, 53243.65 examples/s]
Adding EOS to train dataset:  14%|█▍        | 257000/1807941 [00:05<00:29, 53472.25 examples/s]
Adding EOS to train dataset:  15%|█▍        | 262538/1807941 [00:05<00:28, 54039.33 examples/s]
Adding EOS to train dataset:  15%|█▍        | 270340/1807941 [00:05<00:28, 53246.09 examples/s]
Adding EOS to train dataset:  15%|█▌        | 277903/1807941 [00:05<00:29, 52231.67 examples/s]
Adding EOS to train dataset:  16%|█▌        | 285183/1807941 [00:05<00:29, 50977.29 examples/s]
Adding EOS to train dataset:  16%|█▌        | 292696/1807941 [00:05<00:29, 50683.01 examples/s]
Adding EOS to train dataset:  16%|█▋        | 297824/1807941 [00:06<00:29, 50823.23 examples/s]
Adding EOS to train dataset:  17%|█▋        | 305229/1807941 [00:06<00:29, 50332.03 examples/s]
Adding EOS to train dataset:  17%|█▋        | 310679/1807941 [00:06<00:29, 51342.64 examples/s]
Adding EOS to train dataset:  17%|█▋        | 316000/1807941 [00:06<00:28, 51819.55 examples/s]
Adding EOS to train dataset:  18%|█▊        | 321455/1807941 [00:06<00:28, 52549.60 examples/s]
Adding EOS to train dataset:  18%|█▊        | 326959/1807941 [00:06<00:27, 53239.88 examples/s]
Adding EOS to train dataset:  18%|█▊        | 332366/1807941 [00:06<00:27, 53472.80 examples/s]
Adding EOS to train dataset:  19%|█▊        | 337934/1807941 [00:06<00:27, 54107.33 examples/s]
Adding EOS to train dataset:  19%|█▉        | 346000/1807941 [00:06<00:27, 53808.01 examples/s]
Adding EOS to train dataset:  19%|█▉        | 351451/1807941 [00:07<00:26, 53991.23 examples/s]
Adding EOS to train dataset:  20%|█▉        | 359446/1807941 [00:07<00:35, 41027.20 examples/s]
Adding EOS to train dataset:  20%|██        | 364622/1807941 [00:07<00:33, 43103.87 examples/s]
Adding EOS to train dataset:  20%|██        | 369512/1807941 [00:07<00:32, 44403.30 examples/s]
Adding EOS to train dataset:  21%|██        | 374608/1807941 [00:07<00:31, 45738.24 examples/s]
Adding EOS to train dataset:  21%|██        | 379526/1807941 [00:07<00:30, 46628.64 examples/s]
Adding EOS to train dataset:  21%|██▏       | 384761/1807941 [00:07<00:29, 48075.31 examples/s]
Adding EOS to train dataset:  22%|██▏       | 389951/1807941 [00:07<00:28, 49134.72 examples/s]
Adding EOS to train dataset:  22%|██▏       | 397279/1807941 [00:08<00:28, 49025.39 examples/s]
Adding EOS to train dataset:  22%|██▏       | 402318/1807941 [00:08<00:28, 49381.36 examples/s]
Adding EOS to train dataset:  23%|██▎       | 409982/1807941 [00:08<00:27, 49996.31 examples/s]
Adding EOS to train dataset:  23%|██▎       | 415089/1807941 [00:08<00:27, 50267.91 examples/s]
Adding EOS to train dataset:  23%|██▎       | 420540/1807941 [00:08<00:26, 51398.31 examples/s]
Adding EOS to train dataset:  24%|██▎       | 425883/1807941 [00:08<00:26, 51957.36 examples/s]
Adding EOS to train dataset:  24%|██▍       | 431191/1807941 [00:08<00:26, 52271.99 examples/s]
Adding EOS to train dataset:  24%|██▍       | 436867/1807941 [00:08<00:25, 53231.57 examples/s]
Adding EOS to train dataset:  25%|██▍       | 444981/1807941 [00:09<00:25, 53554.85 examples/s]
Adding EOS to train dataset:  25%|██▌       | 453000/1807941 [00:09<00:25, 53192.63 examples/s]
Adding EOS to train dataset:  25%|██▌       | 458475/1807941 [00:09<00:25, 53580.48 examples/s]
Adding EOS to train dataset:  26%|██▌       | 466456/1807941 [00:09<00:25, 53448.29 examples/s]
Adding EOS to train dataset:  26%|██▌       | 474110/1807941 [00:09<00:25, 52640.89 examples/s]
Adding EOS to train dataset:  27%|██▋       | 481550/1807941 [00:09<00:25, 51659.36 examples/s]
Adding EOS to train dataset:  27%|██▋       | 489184/1807941 [00:09<00:25, 51415.53 examples/s]
Adding EOS to train dataset:  27%|██▋       | 494411/1807941 [00:09<00:25, 51610.28 examples/s]
Adding EOS to train dataset:  28%|██▊       | 501941/1807941 [00:10<00:25, 51140.14 examples/s]
Adding EOS to train dataset:  28%|██▊       | 509396/1807941 [00:10<00:25, 50675.82 examples/s]
Adding EOS to train dataset:  29%|██▊       | 517031/1807941 [00:10<00:25, 50742.58 examples/s]
Adding EOS to train dataset:  29%|██▉       | 523119/1807941 [00:10<00:24, 53077.29 examples/s]
Adding EOS to train dataset:  29%|██▉       | 529257/1807941 [00:10<00:23, 55132.50 examples/s]
Adding EOS to train dataset:  30%|██▉       | 535151/1807941 [00:10<00:22, 56124.00 examples/s]
Adding EOS to train dataset:  30%|██▉       | 541084/1807941 [00:10<00:22, 56993.73 examples/s]
Adding EOS to train dataset:  30%|███       | 547192/1807941 [00:10<00:21, 58133.82 examples/s]
Adding EOS to train dataset:  31%|███       | 553302/1807941 [00:11<00:21, 58977.78 examples/s]
Adding EOS to train dataset:  31%|███       | 559585/1807941 [00:11<00:20, 60091.70 examples/s]
Adding EOS to train dataset:  31%|███▏      | 565810/1807941 [00:11<00:20, 60721.32 examples/s]
Adding EOS to train dataset:  32%|███▏      | 574788/1807941 [00:11<00:20, 60381.75 examples/s]
Adding EOS to train dataset:  32%|███▏      | 583714/1807941 [00:11<00:20, 60067.83 examples/s]
Adding EOS to train dataset:  33%|███▎      | 592977/1807941 [00:11<00:20, 60631.25 examples/s]
Adding EOS to train dataset:  33%|███▎      | 599535/1807941 [00:11<00:19, 61829.44 examples/s]
Adding EOS to train dataset:  34%|███▎      | 605980/1807941 [00:11<00:19, 62500.81 examples/s]
Adding EOS to train dataset:  34%|███▍      | 614459/1807941 [00:12<00:19, 60338.47 examples/s]
Adding EOS to train dataset:  34%|███▍      | 622817/1807941 [00:12<00:20, 58778.83 examples/s]
Adding EOS to train dataset:  35%|███▍      | 631453/1807941 [00:12<00:20, 58385.08 examples/s]
Adding EOS to train dataset:  35%|███▌      | 640105/1807941 [00:12<00:20, 58159.55 examples/s]
Adding EOS to train dataset:  36%|███▌      | 648849/1807941 [00:12<00:19, 58199.07 examples/s]
Adding EOS to train dataset:  36%|███▋      | 657423/1807941 [00:12<00:19, 57877.80 examples/s]
Adding EOS to train dataset:  37%|███▋      | 666137/1807941 [00:12<00:19, 57940.61 examples/s]
Adding EOS to train dataset:  37%|███▋      | 672105/1807941 [00:13<00:19, 58330.38 examples/s]
Adding EOS to train dataset:  38%|███▊      | 678196/1807941 [00:13<00:19, 58955.54 examples/s]
Adding EOS to train dataset:  38%|███▊      | 686754/1807941 [00:13<00:19, 58298.68 examples/s]
Adding EOS to train dataset:  38%|███▊      | 694708/1807941 [00:13<00:19, 56561.55 examples/s]
Adding EOS to train dataset:  39%|███▉      | 702807/1807941 [00:13<00:19, 55738.66 examples/s]
Adding EOS to train dataset:  39%|███▉      | 710671/1807941 [00:13<00:20, 54697.07 examples/s]
Adding EOS to train dataset:  40%|███▉      | 718403/1807941 [00:13<00:20, 53718.94 examples/s]
Adding EOS to train dataset:  40%|████      | 724000/1807941 [00:13<00:19, 54197.46 examples/s]
Adding EOS to train dataset:  40%|████      | 729737/1807941 [00:14<00:19, 54973.35 examples/s]
Adding EOS to train dataset:  41%|████      | 735757/1807941 [00:14<00:19, 56326.64 examples/s]
Adding EOS to train dataset:  41%|████      | 741819/1807941 [00:14<00:18, 57485.75 examples/s]
Adding EOS to train dataset:  41%|████▏     | 747991/1807941 [00:14<00:18, 58662.49 examples/s]
Adding EOS to train dataset:  42%|████▏     | 753916/1807941 [00:14<00:17, 58683.66 examples/s]
Adding EOS to train dataset:  42%|████▏     | 760011/1807941 [00:14<00:17, 59332.90 examples/s]
Adding EOS to train dataset:  42%|████▏     | 766205/1807941 [00:14<00:17, 60092.26 examples/s]
Adding EOS to train dataset:  43%|████▎     | 774000/1807941 [00:14<00:18, 56204.80 examples/s]
Adding EOS to train dataset:  43%|████▎     | 781235/1807941 [00:14<00:19, 53398.50 examples/s]
Adding EOS to train dataset:  44%|████▎     | 787814/1807941 [00:15<00:43, 23528.19 examples/s]
Adding EOS to train dataset:  44%|████▍     | 793979/1807941 [00:15<00:35, 28477.81 examples/s]
Adding EOS to train dataset:  44%|████▍     | 800402/1807941 [00:15<00:29, 34071.74 examples/s]
Adding EOS to train dataset:  45%|████▍     | 806783/1807941 [00:15<00:25, 39503.81 examples/s]
Adding EOS to train dataset:  45%|████▍     | 813235/1807941 [00:16<00:22, 44644.53 examples/s]
Adding EOS to train dataset:  45%|████▌     | 819458/1807941 [00:16<00:20, 48652.95 examples/s]
Adding EOS to train dataset:  46%|████▌     | 825614/1807941 [00:16<00:18, 51823.12 examples/s]
Adding EOS to train dataset:  46%|████▌     | 833903/1807941 [00:16<00:18, 53005.14 examples/s]
Adding EOS to train dataset:  47%|████▋     | 842000/1807941 [00:16<00:18, 53260.46 examples/s]
Adding EOS to train dataset:  47%|████▋     | 850579/1807941 [00:16<00:17, 54516.67 examples/s]
Adding EOS to train dataset:  47%|████▋     | 856426/1807941 [00:16<00:17, 55437.55 examples/s]
Adding EOS to train dataset:  48%|████▊     | 862235/1807941 [00:16<00:16, 56099.64 examples/s]
Adding EOS to train dataset:  48%|████▊     | 868027/1807941 [00:16<00:16, 56576.59 examples/s]
Adding EOS to train dataset:  48%|████▊     | 873855/1807941 [00:17<00:16, 57040.69 examples/s]
Adding EOS to train dataset:  49%|████▊     | 879714/1807941 [00:17<00:16, 57472.27 examples/s]
Adding EOS to train dataset:  49%|████▉     | 885586/1807941 [00:17<00:15, 57826.13 examples/s]
Adding EOS to train dataset:  49%|████▉     | 891422/1807941 [00:17<00:15, 57978.46 examples/s]
Adding EOS to train dataset:  50%|████▉     | 897368/1807941 [00:17<00:15, 58412.17 examples/s]
Adding EOS to train dataset:  50%|████▉     | 903357/1807941 [00:17<00:15, 58846.45 examples/s]
Adding EOS to train dataset:  50%|█████     | 909265/1807941 [00:17<00:15, 58913.11 examples/s]
Adding EOS to train dataset:  51%|█████     | 915202/1807941 [00:17<00:15, 59043.81 examples/s]
Adding EOS to train dataset:  51%|█████     | 921175/1807941 [00:17<00:14, 59244.34 examples/s]
Adding EOS to train dataset:  51%|█████▏    | 930072/1807941 [00:18<00:14, 59268.47 examples/s]
Adding EOS to train dataset:  52%|█████▏    | 936072/1807941 [00:18<00:14, 59466.31 examples/s]
Adding EOS to train dataset:  52%|█████▏    | 942094/1807941 [00:18<00:14, 59674.29 examples/s]
Adding EOS to train dataset:  53%|█████▎    | 951000/1807941 [00:18<00:14, 59472.26 examples/s]
Adding EOS to train dataset:  53%|█████▎    | 957000/1807941 [00:18<00:14, 59460.88 examples/s]
Adding EOS to train dataset:  53%|█████▎    | 963000/1807941 [00:18<00:14, 59302.74 examples/s]
Adding EOS to train dataset:  54%|█████▍    | 971790/1807941 [00:18<00:14, 59040.91 examples/s]
Adding EOS to train dataset:  54%|█████▍    | 977841/1807941 [00:18<00:13, 59418.12 examples/s]
Adding EOS to train dataset:  55%|█████▍    | 986620/1807941 [00:19<00:13, 59096.27 examples/s]
Adding EOS to train dataset:  55%|█████▍    | 992555/1807941 [00:19<00:13, 59159.76 examples/s]
Adding EOS to train dataset:  55%|█████▌    | 1001379/1807941 [00:19<00:13, 59038.93 examples/s]
Adding EOS to train dataset:  56%|█████▌    | 1009939/1807941 [00:19<00:13, 58375.01 examples/s]
Adding EOS to train dataset:  56%|█████▋    | 1018390/1807941 [00:19<00:13, 57713.78 examples/s]
Adding EOS to train dataset:  57%|█████▋    | 1024447/1807941 [00:19<00:13, 58385.22 examples/s]
Adding EOS to train dataset:  57%|█████▋    | 1030533/1807941 [00:19<00:13, 59006.91 examples/s]
Adding EOS to train dataset:  57%|█████▋    | 1036477/1807941 [00:19<00:13, 59120.58 examples/s]
Adding EOS to train dataset:  58%|█████▊    | 1042442/1807941 [00:19<00:12, 59264.89 examples/s]
Adding EOS to train dataset:  58%|█████▊    | 1048430/1807941 [00:20<00:12, 59436.01 examples/s]
Adding EOS to train dataset:  58%|█████▊    | 1054410/1807941 [00:20<00:12, 59539.15 examples/s]
Adding EOS to train dataset:  59%|█████▊    | 1060466/1807941 [00:20<00:12, 59832.86 examples/s]
Adding EOS to train dataset:  59%|█████▉    | 1066470/1807941 [00:20<00:12, 59890.92 examples/s]
Adding EOS to train dataset:  59%|█████▉    | 1072529/1807941 [00:20<00:12, 60093.41 examples/s]
Adding EOS to train dataset:  60%|█████▉    | 1078568/1807941 [00:20<00:12, 60177.72 examples/s]
Adding EOS to train dataset:  60%|█████▉    | 1084619/1807941 [00:20<00:12, 60272.67 examples/s]
Adding EOS to train dataset:  60%|██████    | 1090662/1807941 [00:20<00:11, 60317.18 examples/s]
Adding EOS to train dataset:  61%|██████    | 1099698/1807941 [00:20<00:11, 60282.73 examples/s]
Adding EOS to train dataset:  61%|██████    | 1105833/1807941 [00:21<00:11, 60570.17 examples/s]
Adding EOS to train dataset:  62%|██████▏   | 1111896/1807941 [00:21<00:11, 60584.34 examples/s]
Adding EOS to train dataset:  62%|██████▏   | 1120958/1807941 [00:21<00:11, 60517.65 examples/s]
Adding EOS to train dataset:  62%|██████▏   | 1127093/1807941 [00:21<00:11, 60621.73 examples/s]
Adding EOS to train dataset:  63%|██████▎   | 1133183/1807941 [00:21<00:11, 60694.30 examples/s]
Adding EOS to train dataset:  63%|██████▎   | 1139315/1807941 [00:21<00:10, 60868.17 examples/s]
Adding EOS to train dataset:  63%|██████▎   | 1145619/1807941 [00:21<00:10, 61485.94 examples/s]
Adding EOS to train dataset:  64%|██████▍   | 1154743/1807941 [00:21<00:10, 61230.40 examples/s]
Adding EOS to train dataset:  64%|██████▍   | 1160917/1807941 [00:21<00:10, 61365.09 examples/s]
Adding EOS to train dataset:  65%|██████▍   | 1170000/1807941 [00:22<00:10, 60855.46 examples/s]
Adding EOS to train dataset:  65%|██████▌   | 1179066/1807941 [00:22<00:10, 60633.21 examples/s]
Adding EOS to train dataset:  66%|██████▌   | 1185310/1807941 [00:22<00:10, 61071.65 examples/s]
Adding EOS to train dataset:  66%|██████▌   | 1191774/1807941 [00:22<00:09, 61995.00 examples/s]
Adding EOS to train dataset:  66%|██████▋   | 1198268/1807941 [00:22<00:09, 62673.29 examples/s]
Adding EOS to train dataset:  67%|██████▋   | 1204832/1807941 [00:22<00:09, 63496.83 examples/s]
Adding EOS to train dataset:  67%|██████▋   | 1211298/1807941 [00:22<00:09, 63824.07 examples/s]
Adding EOS to train dataset:  67%|██████▋   | 1217781/1807941 [00:22<00:09, 64110.49 examples/s]
Adding EOS to train dataset:  68%|██████▊   | 1224281/1807941 [00:22<00:09, 64302.55 examples/s]
Adding EOS to train dataset:  68%|██████▊   | 1231226/1807941 [00:23<00:10, 57374.40 examples/s]
Adding EOS to train dataset:  69%|██████▊   | 1239105/1807941 [00:23<00:10, 55398.58 examples/s]
Adding EOS to train dataset:  69%|██████▉   | 1244964/1807941 [00:23<00:10, 56188.66 examples/s]
Adding EOS to train dataset:  69%|██████▉   | 1250724/1807941 [00:23<00:09, 56559.45 examples/s]
Adding EOS to train dataset:  69%|██████▉   | 1256512/1807941 [00:23<00:09, 56919.48 examples/s]
Adding EOS to train dataset:  70%|██████▉   | 1265000/1807941 [00:23<00:09, 56341.45 examples/s]
Adding EOS to train dataset:  70%|███████   | 1272779/1807941 [00:23<00:09, 54176.21 examples/s]
Adding EOS to train dataset:  71%|███████   | 1278572/1807941 [00:23<00:09, 55093.94 examples/s]
Adding EOS to train dataset:  71%|███████   | 1284608/1807941 [00:24<00:09, 56454.38 examples/s]
Adding EOS to train dataset:  71%|███████▏  | 1290854/1807941 [00:24<00:08, 58073.13 examples/s]
Adding EOS to train dataset:  72%|███████▏  | 1296958/1807941 [00:24<00:08, 58893.58 examples/s]
Adding EOS to train dataset:  72%|███████▏  | 1303045/1807941 [00:24<00:08, 59451.71 examples/s]
Adding EOS to train dataset:  72%|███████▏  | 1309425/1807941 [00:24<00:08, 60706.63 examples/s]
Adding EOS to train dataset:  73%|███████▎  | 1315614/1807941 [00:24<00:08, 61048.12 examples/s]
Adding EOS to train dataset:  73%|███████▎  | 1321986/1807941 [00:24<00:07, 61832.98 examples/s]
Adding EOS to train dataset:  73%|███████▎  | 1328281/1807941 [00:24<00:07, 62162.83 examples/s]
Adding EOS to train dataset:  74%|███████▍  | 1334654/1807941 [00:24<00:07, 62627.55 examples/s]
Adding EOS to train dataset:  74%|███████▍  | 1341000/1807941 [00:24<00:07, 62819.09 examples/s]
Adding EOS to train dataset:  75%|███████▍  | 1350370/1807941 [00:25<00:07, 62676.82 examples/s]
Adding EOS to train dataset:  75%|███████▌  | 1359802/1807941 [00:25<00:07, 62745.34 examples/s]
Adding EOS to train dataset:  76%|███████▌  | 1366119/1807941 [00:25<00:07, 62850.86 examples/s]
Adding EOS to train dataset:  76%|███████▌  | 1372484/1807941 [00:25<00:06, 63059.55 examples/s]
Adding EOS to train dataset:  76%|███████▋  | 1381861/1807941 [00:25<00:06, 62859.64 examples/s]
Adding EOS to train dataset:  77%|███████▋  | 1388250/1807941 [00:25<00:06, 62994.51 examples/s]
Adding EOS to train dataset:  77%|███████▋  | 1394801/1807941 [00:25<00:06, 63667.25 examples/s]
Adding EOS to train dataset:  78%|███████▊  | 1401300/1807941 [00:25<00:06, 64022.73 examples/s]
Adding EOS to train dataset:  78%|███████▊  | 1407832/1807941 [00:25<00:06, 64388.65 examples/s]
Adding EOS to train dataset:  78%|███████▊  | 1414392/1807941 [00:26<00:06, 64736.66 examples/s]
Adding EOS to train dataset:  79%|███████▊  | 1421000/1807941 [00:26<00:05, 64991.45 examples/s]
Adding EOS to train dataset:  79%|███████▉  | 1427514/1807941 [00:26<00:05, 65033.68 examples/s]
Adding EOS to train dataset:  79%|███████▉  | 1434036/1807941 [00:26<00:05, 65086.97 examples/s]
Adding EOS to train dataset:  80%|███████▉  | 1440608/1807941 [00:26<00:05, 65271.12 examples/s]
Adding EOS to train dataset:  80%|████████  | 1447188/1807941 [00:26<00:05, 65426.68 examples/s]
Adding EOS to train dataset:  80%|████████  | 1453770/1807941 [00:26<00:05, 65541.94 examples/s]
Adding EOS to train dataset:  81%|████████  | 1463526/1807941 [00:26<00:05, 65342.01 examples/s]
Adding EOS to train dataset:  81%|████████▏ | 1473365/1807941 [00:26<00:05, 65426.71 examples/s]
Adding EOS to train dataset:  82%|████████▏ | 1482983/1807941 [00:27<00:05, 64981.39 examples/s]
Adding EOS to train dataset:  83%|████████▎ | 1492591/1807941 [00:27<00:04, 64674.96 examples/s]
Adding EOS to train dataset:  83%|████████▎ | 1502261/1807941 [00:27<00:04, 64605.67 examples/s]
Adding EOS to train dataset:  83%|████████▎ | 1508771/1807941 [00:27<00:04, 64718.64 examples/s]
Adding EOS to train dataset:  84%|████████▍ | 1515277/1807941 [00:27<00:04, 64775.34 examples/s]
Adding EOS to train dataset:  84%|████████▍ | 1521777/1807941 [00:27<00:04, 64833.73 examples/s]
Adding EOS to train dataset:  85%|████████▍ | 1531501/1807941 [00:27<00:04, 64710.10 examples/s]
Adding EOS to train dataset:  85%|████████▌ | 1541082/1807941 [00:28<00:04, 64425.17 examples/s]
Adding EOS to train dataset:  86%|████████▌ | 1550794/1807941 [00:28<00:03, 64527.53 examples/s]
Adding EOS to train dataset:  86%|████████▋ | 1560525/1807941 [00:28<00:03, 64446.25 examples/s]
Adding EOS to train dataset:  87%|████████▋ | 1567003/1807941 [00:28<00:03, 64519.77 examples/s]
Adding EOS to train dataset:  87%|████████▋ | 1575000/1807941 [00:28<00:04, 52460.81 examples/s]
Adding EOS to train dataset:  87%|████████▋ | 1581000/1807941 [00:28<00:04, 47351.58 examples/s]
Adding EOS to train dataset:  88%|████████▊ | 1588000/1807941 [00:28<00:04, 47008.63 examples/s]
Adding EOS to train dataset:  88%|████████▊ | 1594878/1807941 [00:29<00:04, 46672.36 examples/s]
Adding EOS to train dataset:  89%|████████▊ | 1601374/1807941 [00:29<00:04, 45288.58 examples/s]
Adding EOS to train dataset:  89%|████████▉ | 1606000/1807941 [00:29<00:04, 44813.12 examples/s]
Adding EOS to train dataset:  89%|████████▉ | 1610531/1807941 [00:29<00:04, 44926.03 examples/s]
Adding EOS to train dataset:  89%|████████▉ | 1616872/1807941 [00:29<00:04, 44038.33 examples/s]
Adding EOS to train dataset:  90%|████████▉ | 1623010/1807941 [00:29<00:04, 42418.45 examples/s]
Adding EOS to train dataset:  90%|█████████ | 1629342/1807941 [00:29<00:04, 42301.12 examples/s]
Adding EOS to train dataset:  90%|█████████ | 1635722/1807941 [00:30<00:04, 42369.85 examples/s]
Adding EOS to train dataset:  91%|█████████ | 1641894/1807941 [00:30<00:03, 41993.49 examples/s]
Adding EOS to train dataset:  91%|█████████ | 1647767/1807941 [00:30<00:03, 41127.63 examples/s]
Adding EOS to train dataset:  91%|█████████▏| 1653786/1807941 [00:30<00:03, 40822.58 examples/s]
Adding EOS to train dataset:  92%|█████████▏| 1658311/1807941 [00:30<00:03, 41703.29 examples/s]
Adding EOS to train dataset:  92%|█████████▏| 1663000/1807941 [00:30<00:03, 42885.00 examples/s]
Adding EOS to train dataset:  92%|█████████▏| 1668222/1807941 [00:30<00:03, 45275.98 examples/s]
Adding EOS to train dataset:  93%|█████████▎| 1673271/1807941 [00:30<00:02, 46671.03 examples/s]
Adding EOS to train dataset:  93%|█████████▎| 1678468/1807941 [00:31<00:02, 48137.66 examples/s]
Adding EOS to train dataset:  93%|█████████▎| 1683653/1807941 [00:31<00:02, 49149.39 examples/s]
Adding EOS to train dataset:  93%|█████████▎| 1688937/1807941 [00:31<00:02, 50211.70 examples/s]
Adding EOS to train dataset:  94%|█████████▎| 1694000/1807941 [00:31<00:02, 50184.34 examples/s]
Adding EOS to train dataset:  94%|█████████▍| 1699278/1807941 [00:31<00:02, 50942.98 examples/s]
Adding EOS to train dataset:  94%|█████████▍| 1704401/1807941 [00:31<00:02, 51023.38 examples/s]
Adding EOS to train dataset:  95%|█████████▍| 1709741/1807941 [00:31<00:01, 51369.77 examples/s]
Adding EOS to train dataset:  95%|█████████▍| 1716860/1807941 [00:32<00:07, 12906.86 examples/s]
Adding EOS to train dataset:  95%|█████████▌| 1722773/1807941 [00:32<00:05, 16933.19 examples/s]
Adding EOS to train dataset:  96%|█████████▌| 1728781/1807941 [00:33<00:03, 21707.54 examples/s]
Adding EOS to train dataset:  96%|█████████▌| 1734768/1807941 [00:33<00:02, 26917.01 examples/s]
Adding EOS to train dataset:  96%|█████████▋| 1740800/1807941 [00:33<00:02, 32366.87 examples/s]
Adding EOS to train dataset:  97%|█████████▋| 1746766/1807941 [00:33<00:01, 37525.11 examples/s]
Adding EOS to train dataset:  97%|█████████▋| 1755000/1807941 [00:33<00:01, 42518.78 examples/s]
Adding EOS to train dataset:  98%|█████████▊| 1763052/1807941 [00:33<00:00, 45772.29 examples/s]
Adding EOS to train dataset:  98%|█████████▊| 1770660/1807941 [00:33<00:00, 47224.45 examples/s]
Adding EOS to train dataset:  98%|█████████▊| 1777837/1807941 [00:33<00:00, 47234.81 examples/s]
Adding EOS to train dataset:  99%|█████████▊| 1784594/1807941 [00:34<00:00, 46344.74 examples/s]
Adding EOS to train dataset:  99%|█████████▉| 1791361/1807941 [00:34<00:00, 45976.53 examples/s]
Adding EOS to train dataset:  99%|█████████▉| 1796369/1807941 [00:34<00:00, 46878.02 examples/s]
Adding EOS to train dataset: 100%|█████████▉| 1801970/1807941 [00:34<00:00, 49046.49 examples/s]
Adding EOS to train dataset: 100%|█████████▉| 1807286/1807941 [00:34<00:00, 50092.08 examples/s]
Adding EOS to train dataset: 100%|██████████| 1807941/1807941 [00:34<00:00, 52218.39 examples/s]

Tokenizing train dataset:   0%|          | 0/1807941 [00:00<?, ? examples/s]
Tokenizing train dataset:   0%|          | 106/1807941 [00:00<28:42, 1049.57 examples/s]
Tokenizing train dataset:   0%|          | 219/1807941 [00:00<27:44, 1086.25 examples/s]
Tokenizing train dataset:   0%|          | 336/1807941 [00:00<26:56, 1118.07 examples/s]
Tokenizing train dataset:   0%|          | 454/1807941 [00:00<26:27, 1138.82 examples/s]
Tokenizing train dataset:   0%|          | 626/1807941 [00:00<26:28, 1137.83 examples/s]
Tokenizing train dataset:   0%|          | 743/1807941 [00:00<26:20, 1143.58 examples/s]
Tokenizing train dataset:   0%|          | 861/1807941 [00:00<26:06, 1153.78 examples/s]
Tokenizing train dataset:   0%|          | 981/1807941 [00:00<25:49, 1166.32 examples/s]
Tokenizing train dataset:   0%|          | 1104/1807941 [00:01<31:56, 942.66 examples/s]
Tokenizing train dataset:   0%|          | 1216/1807941 [00:01<30:34, 984.92 examples/s]
Tokenizing train dataset:   0%|          | 1335/1807941 [00:01<29:02, 1037.02 examples/s]
Tokenizing train dataset:   0%|          | 1449/1807941 [00:01<28:17, 1063.92 examples/s]
Tokenizing train dataset:   0%|          | 1561/1807941 [00:01<27:56, 1077.65 examples/s]
Tokenizing train dataset:   0%|          | 1674/1807941 [00:01<27:39, 1088.18 examples/s]
Tokenizing train dataset:   0%|          | 1792/1807941 [00:01<27:05, 1110.87 examples/s]
Tokenizing train dataset:   0%|          | 1908/1807941 [00:01<26:50, 1121.35 examples/s]
Tokenizing train dataset:   0%|          | 2057/1807941 [00:01<28:15, 1065.40 examples/s]
Tokenizing train dataset:   0%|          | 2176/1807941 [00:02<27:26, 1096.68 examples/s]
Tokenizing train dataset:   0%|          | 2296/1807941 [00:02<26:49, 1121.65 examples/s]
Tokenizing train dataset:   0%|          | 2423/1807941 [00:02<25:55, 1160.68 examples/s]
Tokenizing train dataset:   0%|          | 2547/1807941 [00:02<25:28, 1180.87 examples/s]
Tokenizing train dataset:   0%|          | 2676/1807941 [00:02<24:54, 1208.00 examples/s]
Tokenizing train dataset:   0%|          | 2803/1807941 [00:02<24:36, 1222.89 examples/s]
Tokenizing train dataset:   0%|          | 2928/1807941 [00:02<24:30, 1227.54 examples/s]
Tokenizing train dataset:   0%|          | 3084/1807941 [00:02<26:09, 1150.20 examples/s]
Tokenizing train dataset:   0%|          | 3202/1807941 [00:02<26:00, 1156.56 examples/s]
Tokenizing train dataset:   0%|          | 3326/1807941 [00:02<25:33, 1176.44 examples/s]
Tokenizing train dataset:   0%|          | 3455/1807941 [00:03<24:56, 1206.07 examples/s]
Tokenizing train dataset:   0%|          | 3582/1807941 [00:03<24:37, 1221.29 examples/s]
Tokenizing train dataset:   0%|          | 3706/1807941 [00:03<24:33, 1224.36 examples/s]
Tokenizing train dataset:   0%|          | 3893/1807941 [00:03<24:29, 1227.37 examples/s]
Tokenizing train dataset:   0%|          | 4062/1807941 [00:03<25:49, 1164.39 examples/s]
Tokenizing train dataset:   0%|          | 4182/1807941 [00:03<25:38, 1172.21 examples/s]
Tokenizing train dataset:   0%|          | 4314/1807941 [00:03<24:54, 1206.63 examples/s]
Tokenizing train dataset:   0%|          | 4492/1807941 [00:03<25:08, 1195.31 examples/s]
Tokenizing train dataset:   0%|          | 4618/1807941 [00:04<24:52, 1208.60 examples/s]
Tokenizing train dataset:   0%|          | 4746/1807941 [00:04<24:32, 1224.91 examples/s]
Tokenizing train dataset:   0%|          | 4928/1807941 [00:04<24:39, 1218.38 examples/s]
Tokenizing train dataset:   0%|          | 5079/1807941 [00:04<26:17, 1142.58 examples/s]
Tokenizing train dataset:   0%|          | 5258/1807941 [00:04<25:58, 1156.41 examples/s]
Tokenizing train dataset:   0%|          | 5379/1807941 [00:04<25:43, 1167.79 examples/s]
Tokenizing train dataset:   0%|          | 5554/1807941 [00:04<25:50, 1162.48 examples/s]
Tokenizing train dataset:   0%|          | 5679/1807941 [00:04<25:29, 1178.24 examples/s]
Tokenizing train dataset:   0%|          | 5859/1807941 [00:05<25:24, 1181.77 examples/s]
Tokenizing train dataset:   0%|          | 5986/1807941 [00:05<25:01, 1200.17 examples/s]
Tokenizing train dataset:   0%|          | 6138/1807941 [00:05<26:29, 1133.84 examples/s]
Tokenizing train dataset:   0%|          | 6256/1807941 [00:05<26:13, 1144.68 examples/s]
Tokenizing train dataset:   0%|          | 6374/1807941 [00:05<26:02, 1152.68 examples/s]
Tokenizing train dataset:   0%|          | 6540/1807941 [00:05<26:28, 1133.76 examples/s]
Tokenizing train dataset:   0%|          | 6664/1807941 [00:05<25:54, 1158.63 examples/s]
Tokenizing train dataset:   0%|          | 6792/1807941 [00:05<25:18, 1186.16 examples/s]
Tokenizing train dataset:   0%|          | 6915/1807941 [00:05<25:08, 1193.95 examples/s]
Tokenizing train dataset:   0%|          | 7062/1807941 [00:06<26:59, 1111.85 examples/s]
Tokenizing train dataset:   0%|          | 7229/1807941 [00:06<27:07, 1106.18 examples/s]
Tokenizing train dataset:   0%|          | 7347/1807941 [00:06<26:47, 1120.19 examples/s]
Tokenizing train dataset:   0%|          | 7469/1807941 [00:06<26:18, 1140.96 examples/s]
Tokenizing train dataset:   0%|          | 7587/1807941 [00:06<26:08, 1147.94 examples/s]
Tokenizing train dataset:   0%|          | 7761/1807941 [00:06<26:09, 1146.75 examples/s]
Tokenizing train dataset:   0%|          | 7877/1807941 [00:06<26:10, 1146.23 examples/s]
Tokenizing train dataset:   0%|          | 8000/1807941 [00:06<27:13, 1101.74 examples/s]
Tokenizing train dataset:   0%|          | 8122/1807941 [00:07<26:31, 1131.25 examples/s]
Tokenizing train dataset:   0%|          | 8251/1807941 [00:07<25:33, 1173.29 examples/s]
Tokenizing train dataset:   0%|          | 8429/1807941 [00:07<25:28, 1177.24 examples/s]
Tokenizing train dataset:   0%|          | 8558/1807941 [00:07<24:52, 1205.70 examples/s]
Tokenizing train dataset:   0%|          | 8680/1807941 [00:07<24:51, 1206.27 examples/s]
Tokenizing train dataset:   0%|          | 8846/1807941 [00:07<25:46, 1163.46 examples/s]
Tokenizing train dataset:   0%|          | 8970/1807941 [00:07<25:23, 1180.86 examples/s]
Tokenizing train dataset:   1%|          | 9105/1807941 [00:07<27:51, 1076.41 examples/s]
Tokenizing train dataset:   1%|          | 9217/1807941 [00:08<27:41, 1082.57 examples/s]
Tokenizing train dataset:   1%|          | 9333/1807941 [00:08<27:15, 1100.01 examples/s]
Tokenizing train dataset:   1%|          | 9492/1807941 [00:08<27:43, 1081.30 examples/s]
Tokenizing train dataset:   1%|          | 9602/1807941 [00:08<27:38, 1084.19 examples/s]
Tokenizing train dataset:   1%|          | 9774/1807941 [00:08<27:10, 1102.58 examples/s]
Tokenizing train dataset:   1%|          | 9897/1807941 [00:08<26:28, 1132.03 examples/s]
Tokenizing train dataset:   1%|          | 10067/1807941 [00:08<26:41, 1122.73 examples/s]
Tokenizing train dataset:   1%|          | 10199/1807941 [00:08<25:38, 1168.20 examples/s]
Tokenizing train dataset:   1%|          | 10324/1807941 [00:09<25:15, 1186.38 examples/s]
Tokenizing train dataset:   1%|          | 10471/1807941 [00:09<27:00, 1108.92 examples/s]
Tokenizing train dataset:   1%|          | 10636/1807941 [00:09<27:09, 1103.19 examples/s]
Tokenizing train dataset:   1%|          | 10759/1807941 [00:09<26:28, 1131.12 examples/s]
Tokenizing train dataset:   1%|          | 10886/1807941 [00:09<25:43, 1164.64 examples/s]
Tokenizing train dataset:   1%|          | 11074/1807941 [00:09<25:16, 1185.21 examples/s]
Tokenizing train dataset:   1%|          | 11252/1807941 [00:09<25:22, 1180.44 examples/s]
Tokenizing train dataset:   1%|          | 11372/1807941 [00:09<25:18, 1182.75 examples/s]
Tokenizing train dataset:   1%|          | 11499/1807941 [00:10<24:53, 1203.08 examples/s]
Tokenizing train dataset:   1%|          | 11634/1807941 [00:10<24:07, 1241.35 examples/s]
Tokenizing train dataset:   1%|          | 11814/1807941 [00:10<24:28, 1222.76 examples/s]
Tokenizing train dataset:   1%|          | 11988/1807941 [00:10<24:59, 1197.83 examples/s]
Tokenizing train dataset:   1%|          | 12131/1807941 [00:10<26:52, 1113.35 examples/s]
Tokenizing train dataset:   1%|          | 12247/1807941 [00:10<26:40, 1121.62 examples/s]
Tokenizing train dataset:   1%|          | 12373/1807941 [00:10<25:56, 1153.67 examples/s]
Tokenizing train dataset:   1%|          | 12550/1807941 [00:10<25:49, 1158.80 examples/s]
Tokenizing train dataset:   1%|          | 12725/1807941 [00:11<25:52, 1156.06 examples/s]
Tokenizing train dataset:   1%|          | 12848/1807941 [00:11<25:30, 1172.51 examples/s]
Tokenizing train dataset:   1%|          | 13000/1807941 [00:11<27:02, 1106.58 examples/s]
Tokenizing train dataset:   1%|          | 13151/1807941 [00:11<27:53, 1072.32 examples/s]
Tokenizing train dataset:   1%|          | 13280/1807941 [00:11<26:44, 1118.67 examples/s]
Tokenizing train dataset:   1%|          | 13394/1807941 [00:11<26:40, 1121.17 examples/s]
Tokenizing train dataset:   1%|          | 13508/1807941 [00:11<26:37, 1123.42 examples/s]
Tokenizing train dataset:   1%|          | 13630/1807941 [00:11<26:06, 1145.72 examples/s]
Tokenizing train dataset:   1%|          | 13807/1807941 [00:12<25:54, 1154.40 examples/s]
Tokenizing train dataset:   1%|          | 13926/1807941 [00:12<25:46, 1159.94 examples/s]
Tokenizing train dataset:   1%|          | 14079/1807941 [00:12<27:03, 1105.17 examples/s]
Tokenizing train dataset:   1%|          | 14199/1807941 [00:12<26:32, 1126.55 examples/s]
Tokenizing train dataset:   1%|          | 14373/1807941 [00:12<26:21, 1134.35 examples/s]
Tokenizing train dataset:   1%|          | 14498/1807941 [00:12<25:43, 1161.79 examples/s]
Tokenizing train dataset:   1%|          | 14624/1807941 [00:12<25:14, 1184.40 examples/s]
Tokenizing train dataset:   1%|          | 14745/1807941 [00:12<25:06, 1190.40 examples/s]
Tokenizing train dataset:   1%|          | 14871/1807941 [00:12<24:48, 1205.01 examples/s]
Tokenizing train dataset:   1%|          | 15021/1807941 [00:13<26:34, 1124.17 examples/s]
Tokenizing train dataset:   1%|          | 15139/1807941 [00:13<26:16, 1137.24 examples/s]
Tokenizing train dataset:   1%|          | 15300/1807941 [00:13<26:54, 1110.51 examples/s]
Tokenizing train dataset:   1%|          | 15456/1807941 [00:13<27:37, 1081.64 examples/s]
Tokenizing train dataset:   1%|          | 15592/1807941 [00:13<29:14, 1021.52 examples/s]
Tokenizing train dataset:   1%|          | 15751/1807941 [00:13<29:00, 1029.40 examples/s]
Tokenizing train dataset:   1%|          | 15863/1807941 [00:13<28:28, 1048.92 examples/s]
Tokenizing train dataset:   1%|          | 16000/1807941 [00:14<30:46, 970.40 examples/s] 
Tokenizing train dataset:   1%|          | 16160/1807941 [00:14<29:57, 996.75 examples/s]
Tokenizing train dataset:   1%|          | 16271/1807941 [00:14<29:17, 1019.40 examples/s]
Tokenizing train dataset:   1%|          | 16430/1807941 [00:14<29:00, 1029.22 examples/s]
Tokenizing train dataset:   1%|          | 16546/1807941 [00:14<28:14, 1057.18 examples/s]
Tokenizing train dataset:   1%|          | 16656/1807941 [00:14<27:58, 1067.01 examples/s]
Tokenizing train dataset:   1%|          | 16817/1807941 [00:14<28:01, 1064.95 examples/s]
Tokenizing train dataset:   1%|          | 16940/1807941 [00:14<27:02, 1104.18 examples/s]
Tokenizing train dataset:   1%|          | 17053/1807941 [00:15<28:34, 1044.63 examples/s]
Tokenizing train dataset:   1%|          | 17227/1807941 [00:15<27:34, 1082.25 examples/s]
Tokenizing train dataset:   1%|          | 17349/1807941 [00:15<26:48, 1113.50 examples/s]
Tokenizing train dataset:   1%|          | 17465/1807941 [00:15<26:32, 1124.63 examples/s]
Tokenizing train dataset:   1%|          | 17582/1807941 [00:15<26:16, 1135.68 examples/s]
Tokenizing train dataset:   1%|          | 17747/1807941 [00:15<26:36, 1121.04 examples/s]
Tokenizing train dataset:   1%|          | 17861/1807941 [00:15<26:33, 1123.26 examples/s]
Tokenizing train dataset:   1%|          | 17979/1807941 [00:15<26:13, 1137.24 examples/s]
Tokenizing train dataset:   1%|          | 18135/1807941 [00:16<27:12, 1096.20 examples/s]
Tokenizing train dataset:   1%|          | 18309/1807941 [00:16<26:43, 1116.21 examples/s]
Tokenizing train dataset:   1%|          | 18436/1807941 [00:16<25:52, 1152.31 examples/s]
Tokenizing train dataset:   1%|          | 18562/1807941 [00:16<25:18, 1178.41 examples/s]
Tokenizing train dataset:   1%|          | 18683/1807941 [00:16<25:13, 1182.32 examples/s]
Tokenizing train dataset:   1%|          | 18816/1807941 [00:16<24:25, 1221.17 examples/s]
Tokenizing train dataset:   1%|          | 18997/1807941 [00:16<24:37, 1210.48 examples/s]
Tokenizing train dataset:   1%|          | 19129/1807941 [00:16<27:17, 1092.11 examples/s]
Tokenizing train dataset:   1%|          | 19256/1807941 [00:16<26:15, 1135.06 examples/s]
Tokenizing train dataset:   1%|          | 19383/1807941 [00:17<25:33, 1166.34 examples/s]
Tokenizing train dataset:   1%|          | 19557/1807941 [00:17<25:37, 1162.94 examples/s]
Tokenizing train dataset:   1%|          | 19688/1807941 [00:17<24:52, 1198.42 examples/s]
Tokenizing train dataset:   1%|          | 19857/1807941 [00:17<25:26, 1171.30 examples/s]
Tokenizing train dataset:   1%|          | 19977/1807941 [00:17<25:20, 1176.27 examples/s]
Tokenizing train dataset:   1%|          | 20115/1807941 [00:17<27:28, 1084.68 examples/s]
Tokenizing train dataset:   1%|          | 20229/1807941 [00:17<27:09, 1096.98 examples/s]
Tokenizing train dataset:   1%|          | 20341/1807941 [00:17<27:01, 1102.37 examples/s]
Tokenizing train dataset:   1%|          | 20491/1807941 [00:18<28:05, 1060.28 examples/s]
Tokenizing train dataset:   1%|          | 20635/1807941 [00:18<29:09, 1021.54 examples/s]
Tokenizing train dataset:   1%|          | 20745/1807941 [00:18<28:41, 1038.37 examples/s]
Tokenizing train dataset:   1%|          | 20904/1807941 [00:18<28:31, 1043.91 examples/s]
Tokenizing train dataset:   1%|          | 21051/1807941 [00:18<30:52, 964.52 examples/s] 
Tokenizing train dataset:   1%|          | 21168/1807941 [00:18<29:29, 1009.59 examples/s]
Tokenizing train dataset:   1%|          | 21276/1807941 [00:18<29:02, 1025.39 examples/s]
Tokenizing train dataset:   1%|          | 21389/1807941 [00:18<28:20, 1050.75 examples/s]
Tokenizing train dataset:   1%|          | 21510/1807941 [00:19<27:14, 1093.07 examples/s]
Tokenizing train dataset:   1%|          | 21626/1807941 [00:19<26:51, 1108.78 examples/s]
Tokenizing train dataset:   1%|          | 21744/1807941 [00:19<26:24, 1127.13 examples/s]
Tokenizing train dataset:   1%|          | 21872/1807941 [00:19<25:29, 1168.06 examples/s]
Tokenizing train dataset:   1%|          | 22000/1807941 [00:19<28:34, 1041.74 examples/s]
Tokenizing train dataset:   1%|          | 22117/1807941 [00:19<27:45, 1072.54 examples/s]
Tokenizing train dataset:   1%|          | 22231/1807941 [00:19<27:24, 1085.97 examples/s]
Tokenizing train dataset:   1%|          | 22343/1807941 [00:19<27:15, 1091.45 examples/s]
Tokenizing train dataset:   1%|          | 22460/1807941 [00:19<26:49, 1109.34 examples/s]
Tokenizing train dataset:   1%|          | 22581/1807941 [00:20<26:10, 1136.91 examples/s]
Tokenizing train dataset:   1%|▏         | 22750/1807941 [00:20<26:18, 1131.02 examples/s]
Tokenizing train dataset:   1%|▏         | 22904/1807941 [00:20<27:14, 1091.98 examples/s]
Tokenizing train dataset:   1%|▏         | 23048/1807941 [00:20<28:38, 1038.57 examples/s]
Tokenizing train dataset:   1%|▏         | 23190/1807941 [00:20<29:35, 1005.19 examples/s]
Tokenizing train dataset:   1%|▏         | 23305/1807941 [00:20<28:40, 1037.36 examples/s]
Tokenizing train dataset:   1%|▏         | 23425/1807941 [00:20<27:39, 1075.39 examples/s]
Tokenizing train dataset:   1%|▏         | 23549/1807941 [00:20<26:40, 1115.05 examples/s]
Tokenizing train dataset:   1%|▏         | 23674/1807941 [00:21<25:54, 1147.70 examples/s]
Tokenizing train dataset:   1%|▏         | 23837/1807941 [00:21<26:31, 1121.12 examples/s]
Tokenizing train dataset:   1%|▏         | 23952/1807941 [00:21<26:23, 1126.85 examples/s]
Tokenizing train dataset:   1%|▏         | 24110/1807941 [00:21<28:23, 1047.17 examples/s]
Tokenizing train dataset:   1%|▏         | 24230/1807941 [00:21<27:28, 1082.07 examples/s]
Tokenizing train dataset:   1%|▏         | 24346/1807941 [00:21<26:58, 1101.89 examples/s]
Tokenizing train dataset:   1%|▏         | 24460/1807941 [00:21<26:45, 1110.61 examples/s]
Tokenizing train dataset:   1%|▏         | 24579/1807941 [00:21<26:19, 1129.13 examples/s]
Tokenizing train dataset:   1%|▏         | 24749/1807941 [00:22<26:25, 1124.36 examples/s]
Tokenizing train dataset:   1%|▏         | 24890/1807941 [00:22<28:04, 1058.19 examples/s]
Tokenizing train dataset:   1%|▏         | 25000/1807941 [00:22<29:13, 1016.59 examples/s]
Tokenizing train dataset:   1%|▏         | 25136/1807941 [00:22<30:28, 975.22 examples/s] 
Tokenizing train dataset:   1%|▏         | 25293/1807941 [00:22<29:49, 996.03 examples/s]
Tokenizing train dataset:   1%|▏         | 25409/1807941 [00:22<28:46, 1032.42 examples/s]
Tokenizing train dataset:   1%|▏         | 25532/1807941 [00:22<27:32, 1078.32 examples/s]
Tokenizing train dataset:   1%|▏         | 25649/1807941 [00:22<27:00, 1099.83 examples/s]
Tokenizing train dataset:   1%|▏         | 25810/1807941 [00:23<27:17, 1088.06 examples/s]
Tokenizing train dataset:   1%|▏         | 25946/1807941 [00:23<25:43, 1154.28 examples/s]
Tokenizing train dataset:   1%|▏         | 26079/1807941 [00:23<26:05, 1138.49 examples/s]
Tokenizing train dataset:   1%|▏         | 26214/1807941 [00:23<24:53, 1192.71 examples/s]
Tokenizing train dataset:   1%|▏         | 26384/1807941 [00:23<25:26, 1167.29 examples/s]
Tokenizing train dataset:   1%|▏         | 26560/1807941 [00:23<25:28, 1165.46 examples/s]
Tokenizing train dataset:   1%|▏         | 26686/1807941 [00:23<25:00, 1187.22 examples/s]
Tokenizing train dataset:   1%|▏         | 26809/1807941 [00:23<24:49, 1196.17 examples/s]
Tokenizing train dataset:   1%|▏         | 26983/1807941 [00:24<25:10, 1178.92 examples/s]
Tokenizing train dataset:   2%|▏         | 27130/1807941 [00:24<26:46, 1108.22 examples/s]
Tokenizing train dataset:   2%|▏         | 27246/1807941 [00:24<26:31, 1119.11 examples/s]
Tokenizing train dataset:   2%|▏         | 27361/1807941 [00:24<26:26, 1122.53 examples/s]
Tokenizing train dataset:   2%|▏         | 27478/1807941 [00:24<26:13, 1131.32 examples/s]
Tokenizing train dataset:   2%|▏         | 27604/1807941 [00:24<25:30, 1162.97 examples/s]
Tokenizing train dataset:   2%|▏         | 27726/1807941 [00:24<25:13, 1175.85 examples/s]
Tokenizing train dataset:   2%|▏         | 27860/1807941 [00:24<24:19, 1219.75 examples/s]
Tokenizing train dataset:   2%|▏         | 27983/1807941 [00:24<24:18, 1220.29 examples/s]
Tokenizing train dataset:   2%|▏         | 28134/1807941 [00:25<26:06, 1136.50 examples/s]
Tokenizing train dataset:   2%|▏         | 28263/1807941 [00:25<25:16, 1173.86 examples/s]
Tokenizing train dataset:   2%|▏         | 28450/1807941 [00:25<24:48, 1195.56 examples/s]
Tokenizing train dataset:   2%|▏         | 28615/1807941 [00:25<25:34, 1159.83 examples/s]
Tokenizing train dataset:   2%|▏         | 28756/1807941 [00:25<24:21, 1217.41 examples/s]
Tokenizing train dataset:   2%|▏         | 28890/1807941 [00:25<23:46, 1246.86 examples/s]
Tokenizing train dataset:   2%|▏         | 29067/1807941 [00:25<24:22, 1216.54 examples/s]
Tokenizing train dataset:   2%|▏         | 29205/1807941 [00:25<23:37, 1254.43 examples/s]
Tokenizing train dataset:   2%|▏         | 29334/1807941 [00:25<23:30, 1260.55 examples/s]
Tokenizing train dataset:   2%|▏         | 29496/1807941 [00:26<21:50, 1356.59 examples/s]
Tokenizing train dataset:   2%|▏         | 29671/1807941 [00:26<23:07, 1281.21 examples/s]
Tokenizing train dataset:   2%|▏         | 29831/1807941 [00:26<24:41, 1200.13 examples/s]
Tokenizing train dataset:   2%|▏         | 29995/1807941 [00:26<25:35, 1158.12 examples/s]
Tokenizing train dataset:   2%|▏         | 30138/1807941 [00:26<27:09, 1091.06 examples/s]
Tokenizing train dataset:   2%|▏         | 30284/1807941 [00:26<25:15, 1172.88 examples/s]
Tokenizing train dataset:   2%|▏         | 30442/1807941 [00:26<23:18, 1271.20 examples/s]
Tokenizing train dataset:   2%|▏         | 30577/1807941 [00:26<23:00, 1287.72 examples/s]
Tokenizing train dataset:   2%|▏         | 30754/1807941 [00:27<23:47, 1244.55 examples/s]
Tokenizing train dataset:   2%|▏         | 30918/1807941 [00:27<24:53, 1189.86 examples/s]
Tokenizing train dataset:   2%|▏         | 31059/1807941 [00:27<26:46, 1106.05 examples/s]
Tokenizing train dataset:   2%|▏         | 31173/1807941 [00:27<26:38, 1111.79 examples/s]
Tokenizing train dataset:   2%|▏         | 31288/1807941 [00:27<26:25, 1120.58 examples/s]
Tokenizing train dataset:   2%|▏         | 31404/1807941 [00:27<26:11, 1130.27 examples/s]
Tokenizing train dataset:   2%|▏         | 31521/1807941 [00:27<26:02, 1136.84 examples/s]
Tokenizing train dataset:   2%|▏         | 31646/1807941 [00:27<25:22, 1166.69 examples/s]
Tokenizing train dataset:   2%|▏         | 31765/1807941 [00:28<25:18, 1169.63 examples/s]
Tokenizing train dataset:   2%|▏         | 31937/1807941 [00:28<25:32, 1158.81 examples/s]
Tokenizing train dataset:   2%|▏         | 32085/1807941 [00:28<27:01, 1095.31 examples/s]
Tokenizing train dataset:   2%|▏         | 32205/1807941 [00:28<26:24, 1120.49 examples/s]
Tokenizing train dataset:   2%|▏         | 32338/1807941 [00:28<25:15, 1171.65 examples/s]
Tokenizing train dataset:   2%|▏         | 32462/1807941 [00:28<24:55, 1186.99 examples/s]
Tokenizing train dataset:   2%|▏         | 32584/1807941 [00:28<24:45, 1194.87 examples/s]
Tokenizing train dataset:   2%|▏         | 32710/1807941 [00:28<24:27, 1209.34 examples/s]
Tokenizing train dataset:   2%|▏         | 32837/1807941 [00:28<24:09, 1224.65 examples/s]
Tokenizing train dataset:   2%|▏         | 33000/1807941 [00:29<25:28, 1161.33 examples/s]
Tokenizing train dataset:   2%|▏         | 33126/1807941 [00:29<24:57, 1185.08 examples/s]
Tokenizing train dataset:   2%|▏         | 33300/1807941 [00:29<25:14, 1171.91 examples/s]
Tokenizing train dataset:   2%|▏         | 33484/1807941 [00:29<24:53, 1188.11 examples/s]
Tokenizing train dataset:   2%|▏         | 33605/1807941 [00:29<24:52, 1189.08 examples/s]
Tokenizing train dataset:   2%|▏         | 33731/1807941 [00:29<24:33, 1203.68 examples/s]
Tokenizing train dataset:   2%|▏         | 33861/1807941 [00:29<24:05, 1227.24 examples/s]
Tokenizing train dataset:   2%|▏         | 33990/1807941 [00:29<23:45, 1244.56 examples/s]
Tokenizing train dataset:   2%|▏         | 34155/1807941 [00:30<24:54, 1186.56 examples/s]
Tokenizing train dataset:   2%|▏         | 34296/1807941 [00:30<23:47, 1242.49 examples/s]
Tokenizing train dataset:   2%|▏         | 34426/1807941 [00:30<23:30, 1257.61 examples/s]
Tokenizing train dataset:   2%|▏         | 34556/1807941 [00:30<23:20, 1266.10 examples/s]
Tokenizing train dataset:   2%|▏         | 34692/1807941 [00:30<22:56, 1288.15 examples/s]
Tokenizing train dataset:   2%|▏         | 34865/1807941 [00:30<24:01, 1229.75 examples/s]
Tokenizing train dataset:   2%|▏         | 35000/1807941 [00:30<27:55, 1058.46 examples/s]
Tokenizing train dataset:   2%|▏         | 35138/1807941 [00:30<29:12, 1011.67 examples/s]
Tokenizing train dataset:   2%|▏         | 35303/1807941 [00:31<28:27, 1038.08 examples/s]
Tokenizing train dataset:   2%|▏         | 35413/1807941 [00:31<28:05, 1051.55 examples/s]
Tokenizing train dataset:   2%|▏         | 35528/1807941 [00:31<27:30, 1074.16 examples/s]
Tokenizing train dataset:   2%|▏         | 35638/1807941 [00:31<27:26, 1076.68 examples/s]
Tokenizing train dataset:   2%|▏         | 35755/1807941 [00:31<26:51, 1099.68 examples/s]
Tokenizing train dataset:   2%|▏         | 35868/1807941 [00:31<26:41, 1106.84 examples/s]
Tokenizing train dataset:   2%|▏         | 35980/1807941 [00:31<26:40, 1107.24 examples/s]
Tokenizing train dataset:   2%|▏         | 36132/1807941 [00:31<27:38, 1068.21 examples/s]
Tokenizing train dataset:   2%|▏         | 36297/1807941 [00:32<27:29, 1074.35 examples/s]
Tokenizing train dataset:   2%|▏         | 36437/1807941 [00:32<28:48, 1024.90 examples/s]
Tokenizing train dataset:   2%|▏         | 36569/1807941 [00:32<26:59, 1093.79 examples/s]
Tokenizing train dataset:   2%|▏         | 36688/1807941 [00:32<26:29, 1114.03 examples/s]
Tokenizing train dataset:   2%|▏         | 36807/1807941 [00:32<26:09, 1128.51 examples/s]
Tokenizing train dataset:   2%|▏         | 36923/1807941 [00:32<26:02, 1133.77 examples/s]
Tokenizing train dataset:   2%|▏         | 37068/1807941 [00:32<27:37, 1068.40 examples/s]
Tokenizing train dataset:   2%|▏         | 37178/1807941 [00:32<27:29, 1073.25 examples/s]
Tokenizing train dataset:   2%|▏         | 37299/1807941 [00:32<26:35, 1109.57 examples/s]
Tokenizing train dataset:   2%|▏         | 37416/1807941 [00:33<26:13, 1125.35 examples/s]
Tokenizing train dataset:   2%|▏         | 37591/1807941 [00:33<25:56, 1137.38 examples/s]
Tokenizing train dataset:   2%|▏         | 37762/1807941 [00:33<25:59, 1134.87 examples/s]
Tokenizing train dataset:   2%|▏         | 37889/1807941 [00:33<25:19, 1164.56 examples/s]
Tokenizing train dataset:   2%|▏         | 38077/1807941 [00:33<25:13, 1169.50 examples/s]
Tokenizing train dataset:   2%|▏         | 38217/1807941 [00:33<24:04, 1224.73 examples/s]
Tokenizing train dataset:   2%|▏         | 38390/1807941 [00:33<24:37, 1197.36 examples/s]
Tokenizing train dataset:   2%|▏         | 38519/1807941 [00:33<24:11, 1219.08 examples/s]
Tokenizing train dataset:   2%|▏         | 38646/1807941 [00:34<23:57, 1230.96 examples/s]
Tokenizing train dataset:   2%|▏         | 38826/1807941 [00:34<24:16, 1214.32 examples/s]
Tokenizing train dataset:   2%|▏         | 39000/1807941 [00:34<26:00, 1133.27 examples/s]
Tokenizing train dataset:   2%|▏         | 39121/1807941 [00:34<25:37, 1150.10 examples/s]
Tokenizing train dataset:   2%|▏         | 39244/1807941 [00:34<25:16, 1166.61 examples/s]
Tokenizing train dataset:   2%|▏         | 39415/1807941 [00:34<25:36, 1151.13 examples/s]
Tokenizing train dataset:   2%|▏         | 39539/1807941 [00:34<25:11, 1170.20 examples/s]
Tokenizing train dataset:   2%|▏         | 39704/1807941 [00:34<25:47, 1142.52 examples/s]
Tokenizing train dataset:   2%|▏         | 39833/1807941 [00:35<25:01, 1177.43 examples/s]
Tokenizing train dataset:   2%|▏         | 40000/1807941 [00:35<26:58, 1092.29 examples/s]
Tokenizing train dataset:   2%|▏         | 40113/1807941 [00:35<26:47, 1099.57 examples/s]
Tokenizing train dataset:   2%|▏         | 40268/1807941 [00:35<27:27, 1073.14 examples/s]
Tokenizing train dataset:   2%|▏         | 40385/1807941 [00:35<26:53, 1095.20 examples/s]
Tokenizing train dataset:   2%|▏         | 40511/1807941 [00:35<25:55, 1135.97 examples/s]
Tokenizing train dataset:   2%|▏         | 40632/1807941 [00:35<25:32, 1153.52 examples/s]
Tokenizing train dataset:   2%|▏         | 40753/1807941 [00:35<25:15, 1165.82 examples/s]
Tokenizing train dataset:   2%|▏         | 40871/1807941 [00:36<25:11, 1168.93 examples/s]
Tokenizing train dataset:   2%|▏         | 41015/1807941 [00:36<27:04, 1087.37 examples/s]
Tokenizing train dataset:   2%|▏         | 41138/1807941 [00:36<26:17, 1119.70 examples/s]
Tokenizing train dataset:   2%|▏         | 41254/1807941 [00:36<26:08, 1126.63 examples/s]
Tokenizing train dataset:   2%|▏         | 41417/1807941 [00:36<26:35, 1107.35 examples/s]
Tokenizing train dataset:   2%|▏         | 41541/1807941 [00:36<25:50, 1138.92 examples/s]
Tokenizing train dataset:   2%|▏         | 41687/1807941 [00:36<27:19, 1077.45 examples/s]
Tokenizing train dataset:   2%|▏         | 41815/1807941 [00:36<26:07, 1126.50 examples/s]
Tokenizing train dataset:   2%|▏         | 41944/1807941 [00:36<25:12, 1167.79 examples/s]
Tokenizing train dataset:   2%|▏         | 42120/1807941 [00:37<28:15, 1041.35 examples/s]
Tokenizing train dataset:   2%|▏         | 42242/1807941 [00:37<27:14, 1080.52 examples/s]
Tokenizing train dataset:   2%|▏         | 42375/1807941 [00:37<25:49, 1139.73 examples/s]
Tokenizing train dataset:   2%|▏         | 42538/1807941 [00:37<23:15, 1264.79 examples/s]
Tokenizing train dataset:   2%|▏         | 42715/1807941 [00:37<23:56, 1229.23 examples/s]
Tokenizing train dataset:   2%|▏         | 42892/1807941 [00:37<24:19, 1209.66 examples/s]
Tokenizing train dataset:   2%|▏         | 43050/1807941 [00:37<26:18, 1117.76 examples/s]
Tokenizing train dataset:   2%|▏         | 43217/1807941 [00:38<26:23, 1114.22 examples/s]
Tokenizing train dataset:   2%|▏         | 43335/1807941 [00:38<26:05, 1126.98 examples/s]
Tokenizing train dataset:   2%|▏         | 43456/1807941 [00:38<25:39, 1146.38 examples/s]
Tokenizing train dataset:   2%|▏         | 43573/1807941 [00:38<25:32, 1151.66 examples/s]
Tokenizing train dataset:   2%|▏         | 43692/1807941 [00:38<25:23, 1158.11 examples/s]
Tokenizing train dataset:   2%|▏         | 43864/1807941 [00:38<25:32, 1151.38 examples/s]
Tokenizing train dataset:   2%|▏         | 43992/1807941 [00:38<24:50, 1183.10 examples/s]
Tokenizing train dataset:   2%|▏         | 44137/1807941 [00:38<26:38, 1103.29 examples/s]
Tokenizing train dataset:   2%|▏         | 44275/1807941 [00:38<25:03, 1173.04 examples/s]
Tokenizing train dataset:   2%|▏         | 44447/1807941 [00:39<25:18, 1161.08 examples/s]
Tokenizing train dataset:   2%|▏         | 44566/1807941 [00:39<25:13, 1165.01 examples/s]
Tokenizing train dataset:   2%|▏         | 44742/1807941 [00:39<25:14, 1164.23 examples/s]
Tokenizing train dataset:   2%|▏         | 44865/1807941 [00:39<24:55, 1179.17 examples/s]
Tokenizing train dataset:   2%|▏         | 45001/1807941 [00:39<27:07, 1083.37 examples/s]
Tokenizing train dataset:   2%|▏         | 45173/1807941 [00:39<26:38, 1102.60 examples/s]
Tokenizing train dataset:   3%|▎         | 45293/1807941 [00:39<26:09, 1122.85 examples/s]
Tokenizing train dataset:   3%|▎         | 45411/1807941 [00:39<25:53, 1134.77 examples/s]
Tokenizing train dataset:   3%|▎         | 45541/1807941 [00:40<24:57, 1176.66 examples/s]
Tokenizing train dataset:   3%|▎         | 45660/1807941 [00:40<24:55, 1178.67 examples/s]
Tokenizing train dataset:   3%|▎         | 45809/1807941 [00:40<26:30, 1108.13 examples/s]
Tokenizing train dataset:   3%|▎         | 45964/1807941 [00:40<27:15, 1077.33 examples/s]
Tokenizing train dataset:   3%|▎         | 46130/1807941 [00:40<27:35, 1064.35 examples/s]
Tokenizing train dataset:   3%|▎         | 46251/1807941 [00:40<26:48, 1095.49 examples/s]
Tokenizing train dataset:   3%|▎         | 46378/1807941 [00:40<25:48, 1137.87 examples/s]
Tokenizing train dataset:   3%|▎         | 46511/1807941 [00:40<24:44, 1186.65 examples/s]
Tokenizing train dataset:   3%|▎         | 46647/1807941 [00:41<23:48, 1232.99 examples/s]
Tokenizing train dataset:   3%|▎         | 46773/1807941 [00:41<23:42, 1238.51 examples/s]
Tokenizing train dataset:   3%|▎         | 46951/1807941 [00:41<24:10, 1213.81 examples/s]
Tokenizing train dataset:   3%|▎         | 47091/1807941 [00:41<26:23, 1111.99 examples/s]
Tokenizing train dataset:   3%|▎         | 47219/1807941 [00:41<25:29, 1151.12 examples/s]
Tokenizing train dataset:   3%|▎         | 47350/1807941 [00:41<24:38, 1190.45 examples/s]
Tokenizing train dataset:   3%|▎         | 47478/1807941 [00:41<24:13, 1211.18 examples/s]
Tokenizing train dataset:   3%|▎         | 47607/1807941 [00:41<23:51, 1229.90 examples/s]
Tokenizing train dataset:   3%|▎         | 47738/1807941 [00:41<23:28, 1249.94 examples/s]
Tokenizing train dataset:   3%|▎         | 47906/1807941 [00:42<21:26, 1367.61 examples/s]
Tokenizing train dataset:   3%|▎         | 48077/1807941 [00:42<22:59, 1275.59 examples/s]
Tokenizing train dataset:   3%|▎         | 48213/1807941 [00:42<22:37, 1296.10 examples/s]
Tokenizing train dataset:   3%|▎         | 48360/1807941 [00:42<21:53, 1339.56 examples/s]
Tokenizing train dataset:   3%|▎         | 48499/1807941 [00:42<21:43, 1350.21 examples/s]
Tokenizing train dataset:   3%|▎         | 48700/1807941 [00:42<21:48, 1344.81 examples/s]
Tokenizing train dataset:   3%|▎         | 48839/1807941 [00:42<21:40, 1352.44 examples/s]
Tokenizing train dataset:   3%|▎         | 48981/1807941 [00:42<21:25, 1368.53 examples/s]
Tokenizing train dataset:   3%|▎         | 49150/1807941 [00:43<22:55, 1278.86 examples/s]
Tokenizing train dataset:   3%|▎         | 49288/1807941 [00:43<22:32, 1300.17 examples/s]
Tokenizing train dataset:   3%|▎         | 49472/1807941 [00:43<20:17, 1444.82 examples/s]
Tokenizing train dataset:   3%|▎         | 49668/1807941 [00:43<18:28, 1585.72 examples/s]
Tokenizing train dataset:   3%|▎         | 49895/1807941 [00:43<18:51, 1553.52 examples/s]
Tokenizing train dataset:   3%|▎         | 50066/1807941 [00:43<20:52, 1403.70 examples/s]
Tokenizing train dataset:   3%|▎         | 50214/1807941 [00:43<23:11, 1263.63 examples/s]
Tokenizing train dataset:   3%|▎         | 50385/1807941 [00:43<23:58, 1221.53 examples/s]
Tokenizing train dataset:   3%|▎         | 50517/1807941 [00:44<23:36, 1240.73 examples/s]
Tokenizing train dataset:   3%|▎         | 50695/1807941 [00:44<24:01, 1219.32 examples/s]
Tokenizing train dataset:   3%|▎         | 50828/1807941 [00:44<23:33, 1243.06 examples/s]
Tokenizing train dataset:   3%|▎         | 51000/1807941 [00:44<25:08, 1164.49 examples/s]
Tokenizing train dataset:   3%|▎         | 51182/1807941 [00:44<24:52, 1176.78 examples/s]
Tokenizing train dataset:   3%|▎         | 51367/1807941 [00:44<24:32, 1192.85 examples/s]
Tokenizing train dataset:   3%|▎         | 51517/1807941 [00:44<23:12, 1261.34 examples/s]
Tokenizing train dataset:   3%|▎         | 51694/1807941 [00:45<23:43, 1233.87 examples/s]
Tokenizing train dataset:   3%|▎         | 51844/1807941 [00:45<25:21, 1153.95 examples/s]
Tokenizing train dataset:   3%|▎         | 52000/1807941 [00:45<27:13, 1074.98 examples/s]
Tokenizing train dataset:   3%|▎         | 52132/1807941 [00:45<25:59, 1125.66 examples/s]
Tokenizing train dataset:   3%|▎         | 52262/1807941 [00:45<25:06, 1165.62 examples/s]
Tokenizing train dataset:   3%|▎         | 52394/1807941 [00:45<24:20, 1202.12 examples/s]
Tokenizing train dataset:   3%|▎         | 52565/1807941 [00:45<24:52, 1175.91 examples/s]
Tokenizing train dataset:   3%|▎         | 52691/1807941 [00:45<24:31, 1193.00 examples/s]
Tokenizing train dataset:   3%|▎         | 52813/1807941 [00:45<24:25, 1197.70 examples/s]
Tokenizing train dataset:   3%|▎         | 52938/1807941 [00:46<24:10, 1209.73 examples/s]
Tokenizing train dataset:   3%|▎         | 53081/1807941 [00:46<26:21, 1109.95 examples/s]
Tokenizing train dataset:   3%|▎         | 53225/1807941 [00:46<24:30, 1193.09 examples/s]
Tokenizing train dataset:   3%|▎         | 53352/1807941 [00:46<24:08, 1211.12 examples/s]
Tokenizing train dataset:   3%|▎         | 53527/1807941 [00:46<24:32, 1191.28 examples/s]
Tokenizing train dataset:   3%|▎         | 53664/1807941 [00:46<23:40, 1234.63 examples/s]
Tokenizing train dataset:   3%|▎         | 53805/1807941 [00:46<22:53, 1277.06 examples/s]
Tokenizing train dataset:   3%|▎         | 53943/1807941 [00:46<22:25, 1303.48 examples/s]
Tokenizing train dataset:   3%|▎         | 54119/1807941 [00:47<23:53, 1223.54 examples/s]
Tokenizing train dataset:   3%|▎         | 54257/1807941 [00:47<23:08, 1263.04 examples/s]
Tokenizing train dataset:   3%|▎         | 54392/1807941 [00:47<22:43, 1285.91 examples/s]
Tokenizing train dataset:   3%|▎         | 54544/1807941 [00:47<21:40, 1347.90 examples/s]
Tokenizing train dataset:   3%|▎         | 54696/1807941 [00:47<20:57, 1394.65 examples/s]
Tokenizing train dataset:   3%|▎         | 54839/1807941 [00:47<20:51, 1400.73 examples/s]
Tokenizing train dataset:   3%|▎         | 55000/1807941 [00:47<23:04, 1266.50 examples/s]
Tokenizing train dataset:   3%|▎         | 55131/1807941 [00:47<22:53, 1275.86 examples/s]
Tokenizing train dataset:   3%|▎         | 55322/1807941 [00:47<23:01, 1269.03 examples/s]
Tokenizing train dataset:   3%|▎         | 55511/1807941 [00:48<23:08, 1262.00 examples/s]
Tokenizing train dataset:   3%|▎         | 55699/1807941 [00:48<23:16, 1254.37 examples/s]
Tokenizing train dataset:   3%|▎         | 55888/1807941 [00:48<23:17, 1253.54 examples/s]
Tokenizing train dataset:   3%|▎         | 56069/1807941 [00:48<23:51, 1223.72 examples/s]
Tokenizing train dataset:   3%|▎         | 56194/1807941 [00:48<23:50, 1224.28 examples/s]
Tokenizing train dataset:   3%|▎         | 56326/1807941 [00:48<23:25, 1246.11 examples/s]
Tokenizing train dataset:   3%|▎         | 56460/1807941 [00:48<23:00, 1268.35 examples/s]
Tokenizing train dataset:   3%|▎         | 56595/1807941 [00:48<22:39, 1288.31 examples/s]
Tokenizing train dataset:   3%|▎         | 56726/1807941 [00:49<22:36, 1290.87 examples/s]
Tokenizing train dataset:   3%|▎         | 56918/1807941 [00:49<22:47, 1280.82 examples/s]
Tokenizing train dataset:   3%|▎         | 57094/1807941 [00:49<23:30, 1240.87 examples/s]
Tokenizing train dataset:   3%|▎         | 57227/1807941 [00:49<23:09, 1260.39 examples/s]
Tokenizing train dataset:   3%|▎         | 57362/1807941 [00:49<22:45, 1281.87 examples/s]
Tokenizing train dataset:   3%|▎         | 57495/1807941 [00:49<22:35, 1291.30 examples/s]
Tokenizing train dataset:   3%|▎         | 57669/1807941 [00:49<23:31, 1239.71 examples/s]
Tokenizing train dataset:   3%|▎         | 57851/1807941 [00:49<23:47, 1226.14 examples/s]
Tokenizing train dataset:   3%|▎         | 57981/1807941 [00:50<23:29, 1241.18 examples/s]
Tokenizing train dataset:   3%|▎         | 58137/1807941 [00:50<24:59, 1166.93 examples/s]
Tokenizing train dataset:   3%|▎         | 58292/1807941 [00:50<26:05, 1117.84 examples/s]
Tokenizing train dataset:   3%|▎         | 58415/1807941 [00:50<25:30, 1143.07 examples/s]
Tokenizing train dataset:   3%|▎         | 58542/1807941 [00:50<24:51, 1173.05 examples/s]
Tokenizing train dataset:   3%|▎         | 58667/1807941 [00:50<24:29, 1190.29 examples/s]
Tokenizing train dataset:   3%|▎         | 58793/1807941 [00:50<24:09, 1206.77 examples/s]
Tokenizing train dataset:   3%|▎         | 58915/1807941 [00:50<24:07, 1207.99 examples/s]
Tokenizing train dataset:   3%|▎         | 59070/1807941 [00:51<25:37, 1137.65 examples/s]
Tokenizing train dataset:   3%|▎         | 59240/1807941 [00:51<25:43, 1132.86 examples/s]
Tokenizing train dataset:   3%|▎         | 59360/1807941 [00:51<25:23, 1147.60 examples/s]
Tokenizing train dataset:   3%|▎         | 59487/1807941 [00:51<24:43, 1178.64 examples/s]
Tokenizing train dataset:   3%|▎         | 59607/1807941 [00:51<24:39, 1181.58 examples/s]
Tokenizing train dataset:   3%|▎         | 59733/1807941 [00:51<24:18, 1198.25 examples/s]
Tokenizing train dataset:   3%|▎         | 59856/1807941 [00:51<24:11, 1204.10 examples/s]
Tokenizing train dataset:   3%|▎         | 59984/1807941 [00:51<23:48, 1223.25 examples/s]
Tokenizing train dataset:   3%|▎         | 60129/1807941 [00:51<25:56, 1122.96 examples/s]
Tokenizing train dataset:   3%|▎         | 60302/1807941 [00:52<25:48, 1128.54 examples/s]
Tokenizing train dataset:   3%|▎         | 60471/1807941 [00:52<25:51, 1126.49 examples/s]
Tokenizing train dataset:   3%|▎         | 60597/1807941 [00:52<25:13, 1154.69 examples/s]
Tokenizing train dataset:   3%|▎         | 60724/1807941 [00:52<24:38, 1181.69 examples/s]
Tokenizing train dataset:   3%|▎         | 60844/1807941 [00:52<24:35, 1183.76 examples/s]
Tokenizing train dataset:   3%|▎         | 60967/1807941 [00:52<24:23, 1193.48 examples/s]
Tokenizing train dataset:   3%|▎         | 61128/1807941 [00:52<25:26, 1144.07 examples/s]
Tokenizing train dataset:   3%|▎         | 61253/1807941 [00:52<24:52, 1170.08 examples/s]
Tokenizing train dataset:   3%|▎         | 61389/1807941 [00:53<23:53, 1218.30 examples/s]
Tokenizing train dataset:   3%|▎         | 61573/1807941 [00:53<23:53, 1218.09 examples/s]
Tokenizing train dataset:   3%|▎         | 61712/1807941 [00:53<23:05, 1260.29 examples/s]
Tokenizing train dataset:   3%|▎         | 61903/1807941 [00:53<23:01, 1263.51 examples/s]
Tokenizing train dataset:   3%|▎         | 62069/1807941 [00:53<24:04, 1208.98 examples/s]
Tokenizing train dataset:   3%|▎         | 62207/1807941 [00:53<23:21, 1245.29 examples/s]
Tokenizing train dataset:   3%|▎         | 62341/1807941 [00:53<22:57, 1267.47 examples/s]
Tokenizing train dataset:   3%|▎         | 62476/1807941 [00:53<22:36, 1286.62 examples/s]
Tokenizing train dataset:   3%|▎         | 62615/1807941 [00:53<22:10, 1312.03 examples/s]
Tokenizing train dataset:   3%|▎         | 62811/1807941 [00:54<22:17, 1304.45 examples/s]
Tokenizing train dataset:   3%|▎         | 63000/1807941 [00:54<23:40, 1228.48 examples/s]
Tokenizing train dataset:   3%|▎         | 63179/1807941 [00:54<23:58, 1213.27 examples/s]
Tokenizing train dataset:   4%|▎         | 63303/1807941 [00:54<23:55, 1215.26 examples/s]
Tokenizing train dataset:   4%|▎         | 63487/1807941 [00:54<23:54, 1216.03 examples/s]
Tokenizing train dataset:   4%|▎         | 63628/1807941 [00:54<23:03, 1260.94 examples/s]
Tokenizing train dataset:   4%|▎         | 63797/1807941 [00:54<23:59, 1212.04 examples/s]
Tokenizing train dataset:   4%|▎         | 63932/1807941 [00:55<23:24, 1241.60 examples/s]
Tokenizing train dataset:   4%|▎         | 64089/1807941 [00:55<24:51, 1169.03 examples/s]
Tokenizing train dataset:   4%|▎         | 64260/1807941 [00:55<25:05, 1158.21 examples/s]
Tokenizing train dataset:   4%|▎         | 64434/1807941 [00:55<25:09, 1155.12 examples/s]
Tokenizing train dataset:   4%|▎         | 64589/1807941 [00:55<26:08, 1111.66 examples/s]
Tokenizing train dataset:   4%|▎         | 64735/1807941 [00:55<27:15, 1066.06 examples/s]
Tokenizing train dataset:   4%|▎         | 64866/1807941 [00:55<25:56, 1119.88 examples/s]
Tokenizing train dataset:   4%|▎         | 65000/1807941 [00:56<26:06, 1112.33 examples/s]
Tokenizing train dataset:   4%|▎         | 65123/1807941 [00:56<25:28, 1140.00 examples/s]
Tokenizing train dataset:   4%|▎         | 65257/1807941 [00:56<24:24, 1189.74 examples/s]
Tokenizing train dataset:   4%|▎         | 65380/1807941 [00:56<24:14, 1198.15 examples/s]
Tokenizing train dataset:   4%|▎         | 65506/1807941 [00:56<23:55, 1213.44 examples/s]
Tokenizing train dataset:   4%|▎         | 65687/1807941 [00:56<24:04, 1206.28 examples/s]
Tokenizing train dataset:   4%|▎         | 65812/1807941 [00:56<23:54, 1214.68 examples/s]
Tokenizing train dataset:   4%|▎         | 65956/1807941 [00:56<22:47, 1273.74 examples/s]
Tokenizing train dataset:   4%|▎         | 66134/1807941 [00:56<23:56, 1212.85 examples/s]
Tokenizing train dataset:   4%|▎         | 66269/1807941 [00:57<23:17, 1246.52 examples/s]
Tokenizing train dataset:   4%|▎         | 66405/1807941 [00:57<22:49, 1272.11 examples/s]
Tokenizing train dataset:   4%|▎         | 66544/1807941 [00:57<22:16, 1302.58 examples/s]
Tokenizing train dataset:   4%|▎         | 66685/1807941 [00:57<21:50, 1328.96 examples/s]
Tokenizing train dataset:   4%|▎         | 66828/1807941 [00:57<21:25, 1354.59 examples/s]
Tokenizing train dataset:   4%|▎         | 67000/1807941 [00:57<22:51, 1269.01 examples/s]
Tokenizing train dataset:   4%|▎         | 67194/1807941 [00:57<22:44, 1275.71 examples/s]
Tokenizing train dataset:   4%|▎         | 67329/1807941 [00:57<22:28, 1290.78 examples/s]
Tokenizing train dataset:   4%|▎         | 67460/1807941 [00:57<22:27, 1291.35 examples/s]
Tokenizing train dataset:   4%|▎         | 67646/1807941 [00:58<22:51, 1269.22 examples/s]
Tokenizing train dataset:   4%|▍         | 67829/1807941 [00:58<23:14, 1247.71 examples/s]
Tokenizing train dataset:   4%|▍         | 68000/1807941 [00:58<25:07, 1153.83 examples/s]
Tokenizing train dataset:   4%|▍         | 68171/1807941 [00:58<25:17, 1146.53 examples/s]
Tokenizing train dataset:   4%|▍         | 68307/1807941 [00:58<24:19, 1191.54 examples/s]
Tokenizing train dataset:   4%|▍         | 68497/1807941 [00:58<23:53, 1213.84 examples/s]
Tokenizing train dataset:   4%|▍         | 68626/1807941 [00:58<23:34, 1229.98 examples/s]
Tokenizing train dataset:   4%|▍         | 68755/1807941 [00:59<23:21, 1241.01 examples/s]
Tokenizing train dataset:   4%|▍         | 68881/1807941 [00:59<23:16, 1245.00 examples/s]
Tokenizing train dataset:   4%|▍         | 69066/1807941 [00:59<24:52, 1165.43 examples/s]
Tokenizing train dataset:   4%|▍         | 69188/1807941 [00:59<24:39, 1175.50 examples/s]
Tokenizing train dataset:   4%|▍         | 69317/1807941 [00:59<24:04, 1203.30 examples/s]
Tokenizing train dataset:   4%|▍         | 69442/1807941 [00:59<23:54, 1211.89 examples/s]
Tokenizing train dataset:   4%|▍         | 69574/1807941 [00:59<23:21, 1240.13 examples/s]
Tokenizing train dataset:   4%|▍         | 69700/1807941 [00:59<23:17, 1244.26 examples/s]
Tokenizing train dataset:   4%|▍         | 69912/1807941 [00:59<22:12, 1304.25 examples/s]
Tokenizing train dataset:   4%|▍         | 70074/1807941 [01:00<23:41, 1222.68 examples/s]
Tokenizing train dataset:   4%|▍         | 70206/1807941 [01:00<23:16, 1244.34 examples/s]
Tokenizing train dataset:   4%|▍         | 70362/1807941 [01:00<24:46, 1168.89 examples/s]
Tokenizing train dataset:   4%|▍         | 70497/1807941 [01:00<23:54, 1210.81 examples/s]
Tokenizing train dataset:   4%|▍         | 70631/1807941 [01:00<23:19, 1241.15 examples/s]
Tokenizing train dataset:   4%|▍         | 70762/1807941 [01:00<23:03, 1255.88 examples/s]
Tokenizing train dataset:   4%|▍         | 70893/1807941 [01:00<22:47, 1270.04 examples/s]
Tokenizing train dataset:   4%|▍         | 71050/1807941 [01:00<24:32, 1179.93 examples/s]
Tokenizing train dataset:   4%|▍         | 71214/1807941 [01:01<25:16, 1144.87 examples/s]
Tokenizing train dataset:   4%|▍         | 71346/1807941 [01:01<24:23, 1186.21 examples/s]
Tokenizing train dataset:   4%|▍         | 71480/1807941 [01:01<23:39, 1223.63 examples/s]
Tokenizing train dataset:   4%|▍         | 71612/1807941 [01:01<23:12, 1246.69 examples/s]
Tokenizing train dataset:   4%|▍         | 71749/1807941 [01:01<22:36, 1280.16 examples/s]
Tokenizing train dataset:   4%|▍         | 71893/1807941 [01:01<21:52, 1322.35 examples/s]
Tokenizing train dataset:   4%|▍         | 72070/1807941 [01:01<22:59, 1258.34 examples/s]
Tokenizing train dataset:   4%|▍         | 72209/1807941 [01:01<22:25, 1289.77 examples/s]
Tokenizing train dataset:   4%|▍         | 72400/1807941 [01:01<22:37, 1278.63 examples/s]
Tokenizing train dataset:   4%|▍         | 72587/1807941 [01:02<22:51, 1265.06 examples/s]
Tokenizing train dataset:   4%|▍         | 72715/1807941 [01:02<22:52, 1264.56 examples/s]
Tokenizing train dataset:   4%|▍         | 72903/1807941 [01:02<23:00, 1256.63 examples/s]
Tokenizing train dataset:   4%|▍         | 73050/1807941 [01:02<25:21, 1140.52 examples/s]
Tokenizing train dataset:   4%|▍         | 73223/1807941 [01:02<25:19, 1141.58 examples/s]
Tokenizing train dataset:   4%|▍         | 73347/1807941 [01:02<24:52, 1162.04 examples/s]
Tokenizing train dataset:   4%|▍         | 73482/1807941 [01:02<23:58, 1205.86 examples/s]
Tokenizing train dataset:   4%|▍         | 73607/1807941 [01:03<23:46, 1215.52 examples/s]
Tokenizing train dataset:   4%|▍         | 73789/1807941 [01:03<23:49, 1212.94 examples/s]
Tokenizing train dataset:   4%|▍         | 73976/1807941 [01:03<23:42, 1218.78 examples/s]
Tokenizing train dataset:   4%|▍         | 74167/1807941 [01:03<23:25, 1233.44 examples/s]
Tokenizing train dataset:   4%|▍         | 74316/1807941 [01:03<22:23, 1290.36 examples/s]
Tokenizing train dataset:   4%|▍         | 74498/1807941 [01:03<22:56, 1259.38 examples/s]
Tokenizing train dataset:   4%|▍         | 74627/1807941 [01:03<22:52, 1262.86 examples/s]
Tokenizing train dataset:   4%|▍         | 74812/1807941 [01:03<23:06, 1249.64 examples/s]
Tokenizing train dataset:   4%|▍         | 74939/1807941 [01:04<23:02, 1253.36 examples/s]
Tokenizing train dataset:   4%|▍         | 75089/1807941 [01:04<24:47, 1164.98 examples/s]
Tokenizing train dataset:   4%|▍         | 75220/1807941 [01:04<24:04, 1199.82 examples/s]
Tokenizing train dataset:   4%|▍         | 75351/1807941 [01:04<23:34, 1224.89 examples/s]
Tokenizing train dataset:   4%|▍         | 75485/1807941 [01:04<23:04, 1251.72 examples/s]
Tokenizing train dataset:   4%|▍         | 75673/1807941 [01:04<23:11, 1245.30 examples/s]
Tokenizing train dataset:   4%|▍         | 75803/1807941 [01:04<22:58, 1256.49 examples/s]
Tokenizing train dataset:   4%|▍         | 75986/1807941 [01:04<23:19, 1237.50 examples/s]
Tokenizing train dataset:   4%|▍         | 76143/1807941 [01:05<24:39, 1170.46 examples/s]
Tokenizing train dataset:   4%|▍         | 76319/1807941 [01:05<24:40, 1169.31 examples/s]
Tokenizing train dataset:   4%|▍         | 76494/1807941 [01:05<24:45, 1165.73 examples/s]
Tokenizing train dataset:   4%|▍         | 76661/1807941 [01:05<22:32, 1280.05 examples/s]
Tokenizing train dataset:   4%|▍         | 76811/1807941 [01:05<21:38, 1332.92 examples/s]
Tokenizing train dataset:   4%|▍         | 76982/1807941 [01:05<22:49, 1264.14 examples/s]
Tokenizing train dataset:   4%|▍         | 77126/1807941 [01:05<24:50, 1161.55 examples/s]
Tokenizing train dataset:   4%|▍         | 77292/1807941 [01:06<25:18, 1140.02 examples/s]
Tokenizing train dataset:   4%|▍         | 77409/1807941 [01:06<25:12, 1144.40 examples/s]
Tokenizing train dataset:   4%|▍         | 77532/1807941 [01:06<24:46, 1163.95 examples/s]
Tokenizing train dataset:   4%|▍         | 77654/1807941 [01:06<24:31, 1175.48 examples/s]
Tokenizing train dataset:   4%|▍         | 77823/1807941 [01:06<25:03, 1150.51 examples/s]
Tokenizing train dataset:   4%|▍         | 77985/1807941 [01:06<25:42, 1121.31 examples/s]
Tokenizing train dataset:   4%|▍         | 78143/1807941 [01:06<26:16, 1096.90 examples/s]
Tokenizing train dataset:   4%|▍         | 78314/1807941 [01:06<26:01, 1107.96 examples/s]
Tokenizing train dataset:   4%|▍         | 78437/1807941 [01:07<25:23, 1135.30 examples/s]
Tokenizing train dataset:   4%|▍         | 78568/1807941 [01:07<24:28, 1177.76 examples/s]
Tokenizing train dataset:   4%|▍         | 78694/1807941 [01:07<24:03, 1197.79 examples/s]
Tokenizing train dataset:   4%|▍         | 78827/1807941 [01:07<23:25, 1230.57 examples/s]
Tokenizing train dataset:   4%|▍         | 78952/1807941 [01:07<23:22, 1232.56 examples/s]
Tokenizing train dataset:   4%|▍         | 79082/1807941 [01:07<23:42, 1215.61 examples/s]
Tokenizing train dataset:   4%|▍         | 79207/1807941 [01:07<23:33, 1222.83 examples/s]
Tokenizing train dataset:   4%|▍         | 79334/1807941 [01:07<23:20, 1234.56 examples/s]
Tokenizing train dataset:   4%|▍         | 79460/1807941 [01:07<23:13, 1240.65 examples/s]
Tokenizing train dataset:   4%|▍         | 79596/1807941 [01:07<22:38, 1272.07 examples/s]
Tokenizing train dataset:   4%|▍         | 79724/1807941 [01:08<22:40, 1270.40 examples/s]
Tokenizing train dataset:   4%|▍         | 79901/1807941 [01:08<23:23, 1231.65 examples/s]
Tokenizing train dataset:   4%|▍         | 80065/1807941 [01:08<24:24, 1179.65 examples/s]
Tokenizing train dataset:   4%|▍         | 80190/1807941 [01:08<24:05, 1194.98 examples/s]
Tokenizing train dataset:   4%|▍         | 80367/1807941 [01:08<24:18, 1184.60 examples/s]
Tokenizing train dataset:   4%|▍         | 80498/1807941 [01:08<23:43, 1213.91 examples/s]
Tokenizing train dataset:   4%|▍         | 80631/1807941 [01:08<23:10, 1242.58 examples/s]
Tokenizing train dataset:   4%|▍         | 80759/1807941 [01:08<23:01, 1250.60 examples/s]
Tokenizing train dataset:   4%|▍         | 80886/1807941 [01:09<22:58, 1253.23 examples/s]
Tokenizing train dataset:   4%|▍         | 81068/1807941 [01:09<24:09, 1191.62 examples/s]
Tokenizing train dataset:   4%|▍         | 81219/1807941 [01:09<22:38, 1270.59 examples/s]
Tokenizing train dataset:   5%|▍         | 81422/1807941 [01:09<22:11, 1296.40 examples/s]
Tokenizing train dataset:   5%|▍         | 81581/1807941 [01:09<21:03, 1366.24 examples/s]
Tokenizing train dataset:   5%|▍         | 81723/1807941 [01:09<20:51, 1379.68 examples/s]
Tokenizing train dataset:   5%|▍         | 81921/1807941 [01:09<21:13, 1354.85 examples/s]
Tokenizing train dataset:   5%|▍         | 82077/1807941 [01:09<23:08, 1243.23 examples/s]
Tokenizing train dataset:   5%|▍         | 82256/1807941 [01:10<23:29, 1224.66 examples/s]
Tokenizing train dataset:   5%|▍         | 82382/1807941 [01:10<23:21, 1231.57 examples/s]
Tokenizing train dataset:   5%|▍         | 82564/1807941 [01:10<23:34, 1220.12 examples/s]
Tokenizing train dataset:   5%|▍         | 82740/1807941 [01:10<23:53, 1203.34 examples/s]
Tokenizing train dataset:   5%|▍         | 82869/1807941 [01:10<23:35, 1218.55 examples/s]
Tokenizing train dataset:   5%|▍         | 83018/1807941 [01:10<25:12, 1140.13 examples/s]
Tokenizing train dataset:   5%|▍         | 83187/1807941 [01:10<25:25, 1130.83 examples/s]
Tokenizing train dataset:   5%|▍         | 83321/1807941 [01:11<24:23, 1178.54 examples/s]
Tokenizing train dataset:   5%|▍         | 83446/1807941 [01:11<24:07, 1191.39 examples/s]
Tokenizing train dataset:   5%|▍         | 83624/1807941 [01:11<24:13, 1186.37 examples/s]
Tokenizing train dataset:   5%|▍         | 83806/1807941 [01:11<24:07, 1190.90 examples/s]
Tokenizing train dataset:   5%|▍         | 83940/1807941 [01:11<23:26, 1225.67 examples/s]
Tokenizing train dataset:   5%|▍         | 84111/1807941 [01:11<28:22, 1012.82 examples/s]
Tokenizing train dataset:   5%|▍         | 84224/1807941 [01:11<27:41, 1037.38 examples/s]
Tokenizing train dataset:   5%|▍         | 84386/1807941 [01:11<27:26, 1046.78 examples/s]
Tokenizing train dataset:   5%|▍         | 84502/1807941 [01:12<26:50, 1070.17 examples/s]
Tokenizing train dataset:   5%|▍         | 84624/1807941 [01:12<26:00, 1104.00 examples/s]
Tokenizing train dataset:   5%|▍         | 84745/1807941 [01:12<25:24, 1130.29 examples/s]
Tokenizing train dataset:   5%|▍         | 84916/1807941 [01:12<25:28, 1127.51 examples/s]
Tokenizing train dataset:   5%|▍         | 85063/1807941 [01:12<26:46, 1072.13 examples/s]
Tokenizing train dataset:   5%|▍         | 85178/1807941 [01:12<26:20, 1089.80 examples/s]
Tokenizing train dataset:   5%|▍         | 85339/1807941 [01:12<26:33, 1081.25 examples/s]
Tokenizing train dataset:   5%|▍         | 85465/1807941 [01:12<25:35, 1121.93 examples/s]
Tokenizing train dataset:   5%|▍         | 85593/1807941 [01:13<24:43, 1161.22 examples/s]
Tokenizing train dataset:   5%|▍         | 85712/1807941 [01:13<24:35, 1166.95 examples/s]
Tokenizing train dataset:   5%|▍         | 85890/1807941 [01:13<24:32, 1169.16 examples/s]
Tokenizing train dataset:   5%|▍         | 86012/1807941 [01:13<27:30, 1043.23 examples/s]
Tokenizing train dataset:   5%|▍         | 86164/1807941 [01:13<27:52, 1029.57 examples/s]
Tokenizing train dataset:   5%|▍         | 86289/1807941 [01:13<26:37, 1077.90 examples/s]
Tokenizing train dataset:   5%|▍         | 86417/1807941 [01:13<25:29, 1125.36 examples/s]
Tokenizing train dataset:   5%|▍         | 86543/1807941 [01:13<24:45, 1158.46 examples/s]
Tokenizing train dataset:   5%|▍         | 86666/1807941 [01:14<24:22, 1176.55 examples/s]
Tokenizing train dataset:   5%|▍         | 86788/1807941 [01:14<24:10, 1186.42 examples/s]
Tokenizing train dataset:   5%|▍         | 86912/1807941 [01:14<23:55, 1198.91 examples/s]
Tokenizing train dataset:   5%|▍         | 87064/1807941 [01:14<25:27, 1126.72 examples/s]
Tokenizing train dataset:   5%|▍         | 87196/1807941 [01:14<24:25, 1174.24 examples/s]
Tokenizing train dataset:   5%|▍         | 87332/1807941 [01:14<23:27, 1222.54 examples/s]
Tokenizing train dataset:   5%|▍         | 87463/1807941 [01:14<23:01, 1245.50 examples/s]
Tokenizing train dataset:   5%|▍         | 87598/1807941 [01:14<22:29, 1274.76 examples/s]
Tokenizing train dataset:   5%|▍         | 87732/1807941 [01:14<22:12, 1291.40 examples/s]
Tokenizing train dataset:   5%|▍         | 87873/1807941 [01:14<21:39, 1323.85 examples/s]
Tokenizing train dataset:   5%|▍         | 88090/1807941 [01:15<21:57, 1305.81 examples/s]
Tokenizing train dataset:   5%|▍         | 88222/1807941 [01:15<21:55, 1307.02 examples/s]
Tokenizing train dataset:   5%|▍         | 88420/1807941 [01:15<21:53, 1308.63 examples/s]
Tokenizing train dataset:   5%|▍         | 88562/1807941 [01:15<21:30, 1332.31 examples/s]
Tokenizing train dataset:   5%|▍         | 88746/1807941 [01:15<22:12, 1289.75 examples/s]
Tokenizing train dataset:   5%|▍         | 88939/1807941 [01:15<22:15, 1287.05 examples/s]
Tokenizing train dataset:   5%|▍         | 89123/1807941 [01:15<23:22, 1225.50 examples/s]
Tokenizing train dataset:   5%|▍         | 89257/1807941 [01:16<22:55, 1249.87 examples/s]
Tokenizing train dataset:   5%|▍         | 89387/1807941 [01:16<22:44, 1259.35 examples/s]
Tokenizing train dataset:   5%|▍         | 89576/1807941 [01:16<22:49, 1254.99 examples/s]
Tokenizing train dataset:   5%|▍         | 89761/1807941 [01:16<23:01, 1243.62 examples/s]
Tokenizing train dataset:   5%|▍         | 89945/1807941 [01:16<23:11, 1234.40 examples/s]
Tokenizing train dataset:   5%|▍         | 90117/1807941 [01:16<24:31, 1167.08 examples/s]
Tokenizing train dataset:   5%|▍         | 90238/1807941 [01:16<24:24, 1172.90 examples/s]
Tokenizing train dataset:   5%|▌         | 90411/1807941 [01:17<24:37, 1162.40 examples/s]
Tokenizing train dataset:   5%|▌         | 90530/1807941 [01:17<24:35, 1164.24 examples/s]
Tokenizing train dataset:   5%|▌         | 90653/1807941 [01:17<24:16, 1178.73 examples/s]
Tokenizing train dataset:   5%|▌         | 90782/1807941 [01:17<23:48, 1202.22 examples/s]
Tokenizing train dataset:   5%|▌         | 90908/1807941 [01:17<23:34, 1214.08 examples/s]
Tokenizing train dataset:   5%|▌         | 91067/1807941 [01:17<24:47, 1154.17 examples/s]
Tokenizing train dataset:   5%|▌         | 91201/1807941 [01:17<23:48, 1201.47 examples/s]
Tokenizing train dataset:   5%|▌         | 91329/1807941 [01:17<23:27, 1219.78 examples/s]
Tokenizing train dataset:   5%|▌         | 91462/1807941 [01:17<22:56, 1246.78 examples/s]
Tokenizing train dataset:   5%|▌         | 91652/1807941 [01:18<22:51, 1251.16 examples/s]
Tokenizing train dataset:   5%|▌         | 91784/1807941 [01:18<22:33, 1268.35 examples/s]
Tokenizing train dataset:   5%|▌         | 91915/1807941 [01:18<22:22, 1278.46 examples/s]
Tokenizing train dataset:   5%|▌         | 92080/1807941 [01:18<23:38, 1209.43 examples/s]
Tokenizing train dataset:   5%|▌         | 92266/1807941 [01:18<23:30, 1216.01 examples/s]
Tokenizing train dataset:   5%|▌         | 92397/1807941 [01:18<23:08, 1235.42 examples/s]
Tokenizing train dataset:   5%|▌         | 92589/1807941 [01:18<22:55, 1247.25 examples/s]
Tokenizing train dataset:   5%|▌         | 92768/1807941 [01:18<23:20, 1224.48 examples/s]
Tokenizing train dataset:   5%|▌         | 92898/1807941 [01:19<23:03, 1239.56 examples/s]
Tokenizing train dataset:   5%|▌         | 93062/1807941 [01:19<24:15, 1178.21 examples/s]
Tokenizing train dataset:   5%|▌         | 93192/1807941 [01:19<23:43, 1204.65 examples/s]
Tokenizing train dataset:   5%|▌         | 93371/1807941 [01:19<23:51, 1197.49 examples/s]
Tokenizing train dataset:   5%|▌         | 93492/1807941 [01:19<23:50, 1198.67 examples/s]
Tokenizing train dataset:   5%|▌         | 93659/1807941 [01:19<24:29, 1166.59 examples/s]
Tokenizing train dataset:   5%|▌         | 93778/1807941 [01:19<24:25, 1169.75 examples/s]
Tokenizing train dataset:   5%|▌         | 93900/1807941 [01:19<24:11, 1181.24 examples/s]
Tokenizing train dataset:   5%|▌         | 94064/1807941 [01:20<25:21, 1126.44 examples/s]
Tokenizing train dataset:   5%|▌         | 94181/1807941 [01:20<25:08, 1135.87 examples/s]
Tokenizing train dataset:   5%|▌         | 94314/1807941 [01:20<24:06, 1184.55 examples/s]
Tokenizing train dataset:   5%|▌         | 94443/1807941 [01:20<23:34, 1211.01 examples/s]
Tokenizing train dataset:   5%|▌         | 94575/1807941 [01:20<23:05, 1236.42 examples/s]
Tokenizing train dataset:   5%|▌         | 94751/1807941 [01:20<23:34, 1211.25 examples/s]
Tokenizing train dataset:   5%|▌         | 94925/1807941 [01:20<24:05, 1185.29 examples/s]
Tokenizing train dataset:   5%|▌         | 95090/1807941 [01:20<24:45, 1153.08 examples/s]
Tokenizing train dataset:   5%|▌         | 95220/1807941 [01:21<24:04, 1185.38 examples/s]
Tokenizing train dataset:   5%|▌         | 95345/1807941 [01:21<23:46, 1200.74 examples/s]
Tokenizing train dataset:   5%|▌         | 95489/1807941 [01:21<22:36, 1262.14 examples/s]
Tokenizing train dataset:   5%|▌         | 95626/1807941 [01:21<22:09, 1287.87 examples/s]
Tokenizing train dataset:   5%|▌         | 95775/1807941 [01:21<21:17, 1340.23 examples/s]
Tokenizing train dataset:   5%|▌         | 95976/1807941 [01:21<21:20, 1336.48 examples/s]
Tokenizing train dataset:   5%|▌         | 96161/1807941 [01:21<22:00, 1296.25 examples/s]
Tokenizing train dataset:   5%|▌         | 96293/1807941 [01:21<21:56, 1299.68 examples/s]
Tokenizing train dataset:   5%|▌         | 96428/1807941 [01:21<21:45, 1310.98 examples/s]
Tokenizing train dataset:   5%|▌         | 96565/1807941 [01:22<21:32, 1324.54 examples/s]
Tokenizing train dataset:   5%|▌         | 96704/1807941 [01:22<21:16, 1340.57 examples/s]
Tokenizing train dataset:   5%|▌         | 96903/1807941 [01:22<21:24, 1332.56 examples/s]
Tokenizing train dataset:   5%|▌         | 97080/1807941 [01:22<22:18, 1277.95 examples/s]
Tokenizing train dataset:   5%|▌         | 97212/1807941 [01:22<22:10, 1285.82 examples/s]
Tokenizing train dataset:   5%|▌         | 97354/1807941 [01:22<21:37, 1318.02 examples/s]
Tokenizing train dataset:   5%|▌         | 97538/1807941 [01:22<22:13, 1282.76 examples/s]
Tokenizing train dataset:   5%|▌         | 97728/1807941 [01:22<22:23, 1273.27 examples/s]
Tokenizing train dataset:   5%|▌         | 97956/1807941 [01:23<21:06, 1349.95 examples/s]
Tokenizing train dataset:   5%|▌         | 98102/1807941 [01:23<20:43, 1374.55 examples/s]
Tokenizing train dataset:   5%|▌         | 98299/1807941 [01:23<18:44, 1520.52 examples/s]
Tokenizing train dataset:   5%|▌         | 98490/1807941 [01:23<17:34, 1621.11 examples/s]
Tokenizing train dataset:   5%|▌         | 98687/1807941 [01:23<18:57, 1502.93 examples/s]
Tokenizing train dataset:   5%|▌         | 98864/1807941 [01:23<20:30, 1389.43 examples/s]
Tokenizing train dataset:   5%|▌         | 99023/1807941 [01:23<22:15, 1279.58 examples/s]
Tokenizing train dataset:   5%|▌         | 99161/1807941 [01:23<21:54, 1300.03 examples/s]
Tokenizing train dataset:   5%|▌         | 99296/1807941 [01:24<21:45, 1309.02 examples/s]
Tokenizing train dataset:   6%|▌         | 99495/1807941 [01:24<21:44, 1309.19 examples/s]
Tokenizing train dataset:   6%|▌         | 99639/1807941 [01:24<21:14, 1339.96 examples/s]
Tokenizing train dataset:   6%|▌         | 99839/1807941 [01:24<21:20, 1333.94 examples/s]
Tokenizing train dataset:   6%|▌         | 100008/1807941 [01:24<22:33, 1262.28 examples/s]
Tokenizing train dataset:   6%|▌         | 100143/1807941 [01:24<22:14, 1279.91 examples/s]
Tokenizing train dataset:   6%|▌         | 100337/1807941 [01:24<22:13, 1280.31 examples/s]
Tokenizing train dataset:   6%|▌         | 100475/1807941 [01:24<21:49, 1304.21 examples/s]
Tokenizing train dataset:   6%|▌         | 100607/1807941 [01:25<21:50, 1302.62 examples/s]
Tokenizing train dataset:   6%|▌         | 100798/1807941 [01:25<22:05, 1288.40 examples/s]
Tokenizing train dataset:   6%|▌         | 100940/1807941 [01:25<21:35, 1317.74 examples/s]
Tokenizing train dataset:   6%|▌         | 101120/1807941 [01:25<23:22, 1217.16 examples/s]
Tokenizing train dataset:   6%|▌         | 101248/1807941 [01:25<23:06, 1230.84 examples/s]
Tokenizing train dataset:   6%|▌         | 101428/1807941 [01:25<23:22, 1216.81 examples/s]
Tokenizing train dataset:   6%|▌         | 101565/1807941 [01:25<22:44, 1250.50 examples/s]
Tokenizing train dataset:   6%|▌         | 101701/1807941 [01:25<22:16, 1276.53 examples/s]
Tokenizing train dataset:   6%|▌         | 101878/1807941 [01:26<22:59, 1236.69 examples/s]
Tokenizing train dataset:   6%|▌         | 102065/1807941 [01:26<23:58, 1185.98 examples/s]
Tokenizing train dataset:   6%|▌         | 102201/1807941 [01:26<23:14, 1223.57 examples/s]
Tokenizing train dataset:   6%|▌         | 102377/1807941 [01:26<23:35, 1205.11 examples/s]
Tokenizing train dataset:   6%|▌         | 102561/1807941 [01:26<23:32, 1207.43 examples/s]
Tokenizing train dataset:   6%|▌         | 102685/1807941 [01:26<23:26, 1212.01 examples/s]
Tokenizing train dataset:   6%|▌         | 102866/1807941 [01:26<23:31, 1207.84 examples/s]
Tokenizing train dataset:   6%|▌         | 102994/1807941 [01:27<23:14, 1222.51 examples/s]
Tokenizing train dataset:   6%|▌         | 103145/1807941 [01:27<24:49, 1144.49 examples/s]
Tokenizing train dataset:   6%|▌         | 103265/1807941 [01:27<24:33, 1157.09 examples/s]
Tokenizing train dataset:   6%|▌         | 103392/1807941 [01:27<23:59, 1183.99 examples/s]
Tokenizing train dataset:   6%|▌         | 103577/1807941 [01:27<23:41, 1198.80 examples/s]
Tokenizing train dataset:   6%|▌         | 103705/1807941 [01:27<23:22, 1215.06 examples/s]
Tokenizing train dataset:   6%|▌         | 103831/1807941 [01:27<23:09, 1226.12 examples/s]
Tokenizing train dataset:   6%|▌         | 103967/1807941 [01:27<22:33, 1258.89 examples/s]
Tokenizing train dataset:   6%|▌         | 104127/1807941 [01:27<24:01, 1182.37 examples/s]
Tokenizing train dataset:   6%|▌         | 104311/1807941 [01:28<23:45, 1195.11 examples/s]
Tokenizing train dataset:   6%|▌         | 104439/1807941 [01:28<23:21, 1215.82 examples/s]
Tokenizing train dataset:   6%|▌         | 104573/1807941 [01:28<22:46, 1246.55 examples/s]
Tokenizing train dataset:   6%|▌         | 104716/1807941 [01:28<21:56, 1294.01 examples/s]
Tokenizing train dataset:   6%|▌         | 104855/1807941 [01:28<21:29, 1320.37 examples/s]
Tokenizing train dataset:   6%|▌         | 104991/1807941 [01:28<21:20, 1330.34 examples/s]
Tokenizing train dataset:   6%|▌         | 105153/1807941 [01:28<23:00, 1233.16 examples/s]
Tokenizing train dataset:   6%|▌         | 105288/1807941 [01:28<22:29, 1261.56 examples/s]
Tokenizing train dataset:   6%|▌         | 105427/1807941 [01:28<21:55, 1294.08 examples/s]
Tokenizing train dataset:   6%|▌         | 105563/1807941 [01:29<21:40, 1309.05 examples/s]
Tokenizing train dataset:   6%|▌         | 105705/1807941 [01:29<21:13, 1336.54 examples/s]
Tokenizing train dataset:   6%|▌         | 105884/1807941 [01:29<22:10, 1279.29 examples/s]
Tokenizing train dataset:   6%|▌         | 106062/1807941 [01:29<23:07, 1226.40 examples/s]
Tokenizing train dataset:   6%|▌         | 106239/1807941 [01:29<23:27, 1209.06 examples/s]
Tokenizing train dataset:   6%|▌         | 106366/1807941 [01:29<23:11, 1223.06 examples/s]
Tokenizing train dataset:   6%|▌         | 106551/1807941 [01:29<23:10, 1223.36 examples/s]
Tokenizing train dataset:   6%|▌         | 106682/1807941 [01:29<22:51, 1240.32 examples/s]
Tokenizing train dataset:   6%|▌         | 106810/1807941 [01:30<22:43, 1247.56 examples/s]
Tokenizing train dataset:   6%|▌         | 106942/1807941 [01:30<22:25, 1264.00 examples/s]
Tokenizing train dataset:   6%|▌         | 107125/1807941 [01:30<24:00, 1180.39 examples/s]
Tokenizing train dataset:   6%|▌         | 107252/1807941 [01:30<23:37, 1200.14 examples/s]
Tokenizing train dataset:   6%|▌         | 107379/1807941 [01:30<23:19, 1214.79 examples/s]
Tokenizing train dataset:   6%|▌         | 107504/1807941 [01:30<23:10, 1222.70 examples/s]
Tokenizing train dataset:   6%|▌         | 107635/1807941 [01:30<22:45, 1245.53 examples/s]
Tokenizing train dataset:   6%|▌         | 107817/1807941 [01:30<23:04, 1227.67 examples/s]
Tokenizing train dataset:   6%|▌         | 107951/1807941 [01:31<22:35, 1254.29 examples/s]
Tokenizing train dataset:   6%|▌         | 108129/1807941 [01:31<23:41, 1195.90 examples/s]
Tokenizing train dataset:   6%|▌         | 108256/1807941 [01:31<23:23, 1210.86 examples/s]
Tokenizing train dataset:   6%|▌         | 108420/1807941 [01:31<24:18, 1165.14 examples/s]
Tokenizing train dataset:   6%|▌         | 108542/1807941 [01:31<24:06, 1175.02 examples/s]
Tokenizing train dataset:   6%|▌         | 108663/1807941 [01:31<23:59, 1180.78 examples/s]
Tokenizing train dataset:   6%|▌         | 108794/1807941 [01:31<23:19, 1214.08 examples/s]
Tokenizing train dataset:   6%|▌         | 108975/1807941 [01:31<23:26, 1208.03 examples/s]
Tokenizing train dataset:   6%|▌         | 109135/1807941 [01:32<24:29, 1155.77 examples/s]
Tokenizing train dataset:   6%|▌         | 109270/1807941 [01:32<23:33, 1201.39 examples/s]
Tokenizing train dataset:   6%|▌         | 109397/1807941 [01:32<23:18, 1214.59 examples/s]
Tokenizing train dataset:   6%|▌         | 109535/1807941 [01:32<22:32, 1255.64 examples/s]
Tokenizing train dataset:   6%|▌         | 109665/1807941 [01:32<22:22, 1264.66 examples/s]
Tokenizing train dataset:   6%|▌         | 109793/1807941 [01:32<22:20, 1267.24 examples/s]
Tokenizing train dataset:   6%|▌         | 109924/1807941 [01:32<22:07, 1278.71 examples/s]
Tokenizing train dataset:   6%|▌         | 110081/1807941 [01:32<23:52, 1185.56 examples/s]
Tokenizing train dataset:   6%|▌         | 110257/1807941 [01:32<24:03, 1175.80 examples/s]
Tokenizing train dataset:   6%|▌         | 110380/1807941 [01:33<23:49, 1187.79 examples/s]
Tokenizing train dataset:   6%|▌         | 110509/1807941 [01:33<23:20, 1211.87 examples/s]
Tokenizing train dataset:   6%|▌         | 110681/1807941 [01:33<23:51, 1185.89 examples/s]
Tokenizing train dataset:   6%|▌         | 110802/1807941 [01:33<23:47, 1188.52 examples/s]
Tokenizing train dataset:   6%|▌         | 110927/1807941 [01:33<23:30, 1202.86 examples/s]
Tokenizing train dataset:   6%|▌         | 111079/1807941 [01:33<25:03, 1128.46 examples/s]
Tokenizing train dataset:   6%|▌         | 111245/1807941 [01:33<25:17, 1117.83 examples/s]
Tokenizing train dataset:   6%|▌         | 111418/1807941 [01:33<25:06, 1126.15 examples/s]
Tokenizing train dataset:   6%|▌         | 111568/1807941 [01:34<26:05, 1083.37 examples/s]
Tokenizing train dataset:   6%|▌         | 111687/1807941 [01:34<25:34, 1105.37 examples/s]
Tokenizing train dataset:   6%|▌         | 111828/1807941 [01:34<27:06, 1042.51 examples/s]
Tokenizing train dataset:   6%|▌         | 111936/1807941 [01:34<30:22, 930.56 examples/s] 
Tokenizing train dataset:   6%|▌         | 112062/1807941 [01:34<31:23, 900.32 examples/s]
Tokenizing train dataset:   6%|▌         | 112170/1807941 [01:34<30:04, 939.65 examples/s]
Tokenizing train dataset:   6%|▌         | 112276/1807941 [01:34<29:18, 964.53 examples/s]
Tokenizing train dataset:   6%|▌         | 112388/1807941 [01:34<28:10, 1002.79 examples/s]
Tokenizing train dataset:   6%|▌         | 112500/1807941 [01:35<31:02, 910.15 examples/s] 
Tokenizing train dataset:   6%|▌         | 112600/1807941 [01:35<30:21, 930.56 examples/s]
Tokenizing train dataset:   6%|▌         | 112720/1807941 [01:35<28:12, 1001.44 examples/s]
Tokenizing train dataset:   6%|▌         | 112886/1807941 [01:35<27:18, 1034.59 examples/s]
Tokenizing train dataset:   6%|▋         | 113000/1807941 [01:35<27:23, 1031.08 examples/s]
Tokenizing train dataset:   6%|▋         | 113165/1807941 [01:35<27:00, 1045.93 examples/s]
Tokenizing train dataset:   6%|▋         | 113286/1807941 [01:35<26:06, 1081.62 examples/s]
Tokenizing train dataset:   6%|▋         | 113398/1807941 [01:35<25:59, 1086.79 examples/s]
Tokenizing train dataset:   6%|▋         | 113511/1807941 [01:36<25:46, 1095.64 examples/s]
Tokenizing train dataset:   6%|▋         | 113642/1807941 [01:36<24:28, 1153.90 examples/s]
Tokenizing train dataset:   6%|▋         | 113797/1807941 [01:36<25:35, 1103.59 examples/s]
Tokenizing train dataset:   6%|▋         | 113952/1807941 [01:36<26:13, 1076.52 examples/s]
Tokenizing train dataset:   6%|▋         | 114116/1807941 [01:36<27:08, 1040.14 examples/s]
Tokenizing train dataset:   6%|▋         | 114228/1807941 [01:36<26:44, 1055.78 examples/s]
Tokenizing train dataset:   6%|▋         | 114339/1807941 [01:36<26:28, 1066.14 examples/s]
Tokenizing train dataset:   6%|▋         | 114459/1807941 [01:36<25:42, 1097.86 examples/s]
Tokenizing train dataset:   6%|▋         | 114598/1807941 [01:37<27:20, 1032.37 examples/s]
Tokenizing train dataset:   6%|▋         | 114704/1807941 [01:37<27:15, 1035.15 examples/s]
Tokenizing train dataset:   6%|▋         | 114836/1807941 [01:37<25:28, 1107.47 examples/s]
Tokenizing train dataset:   6%|▋         | 114958/1807941 [01:37<24:49, 1136.51 examples/s]
Tokenizing train dataset:   6%|▋         | 115133/1807941 [01:37<25:15, 1116.78 examples/s]
Tokenizing train dataset:   6%|▋         | 115258/1807941 [01:37<24:33, 1148.92 examples/s]
Tokenizing train dataset:   6%|▋         | 115383/1807941 [01:37<24:02, 1173.43 examples/s]
Tokenizing train dataset:   6%|▋         | 115528/1807941 [01:37<25:45, 1095.35 examples/s]
Tokenizing train dataset:   6%|▋         | 115650/1807941 [01:37<25:05, 1123.96 examples/s]
Tokenizing train dataset:   6%|▋         | 115792/1807941 [01:38<29:59, 940.25 examples/s] 
Tokenizing train dataset:   6%|▋         | 115912/1807941 [01:38<31:30, 895.02 examples/s]
Tokenizing train dataset:   6%|▋         | 116065/1807941 [01:38<30:51, 913.84 examples/s]
Tokenizing train dataset:   6%|▋         | 116190/1807941 [01:38<28:35, 985.88 examples/s]
Tokenizing train dataset:   6%|▋         | 116316/1807941 [01:38<26:51, 1049.47 examples/s]
Tokenizing train dataset:   6%|▋         | 116426/1807941 [01:38<26:35, 1060.17 examples/s]
Tokenizing train dataset:   6%|▋         | 116546/1807941 [01:38<25:43, 1096.01 examples/s]
Tokenizing train dataset:   6%|▋         | 116659/1807941 [01:39<25:32, 1103.78 examples/s]
Tokenizing train dataset:   6%|▋         | 116777/1807941 [01:39<25:07, 1122.18 examples/s]
Tokenizing train dataset:   6%|▋         | 116929/1807941 [01:39<26:10, 1076.64 examples/s]
Tokenizing train dataset:   6%|▋         | 117078/1807941 [01:39<27:02, 1042.23 examples/s]
Tokenizing train dataset:   6%|▋         | 117201/1807941 [01:39<25:54, 1087.47 examples/s]
Tokenizing train dataset:   6%|▋         | 117323/1807941 [01:39<25:10, 1119.58 examples/s]
Tokenizing train dataset:   6%|▋         | 117479/1807941 [01:39<25:51, 1089.73 examples/s]
Tokenizing train dataset:   7%|▋         | 117592/1807941 [01:39<25:41, 1096.60 examples/s]
Tokenizing train dataset:   7%|▋         | 117717/1807941 [01:40<31:46, 886.78 examples/s] 
Tokenizing train dataset:   7%|▋         | 117844/1807941 [01:40<28:57, 972.67 examples/s]
Tokenizing train dataset:   7%|▋         | 117976/1807941 [01:40<26:42, 1054.74 examples/s]
Tokenizing train dataset:   7%|▋         | 118109/1807941 [01:40<28:27, 989.39 examples/s] 
Tokenizing train dataset:   7%|▋         | 118224/1807941 [01:40<27:25, 1026.81 examples/s]
Tokenizing train dataset:   7%|▋         | 118344/1807941 [01:40<26:21, 1068.24 examples/s]
Tokenizing train dataset:   7%|▋         | 118466/1807941 [01:40<25:24, 1108.53 examples/s]
Tokenizing train dataset:   7%|▋         | 118603/1807941 [01:40<27:15, 1032.67 examples/s]
Tokenizing train dataset:   7%|▋         | 118734/1807941 [01:40<25:32, 1102.12 examples/s]
Tokenizing train dataset:   7%|▋         | 118854/1807941 [01:41<28:30, 987.59 examples/s] 
Tokenizing train dataset:   7%|▋         | 118982/1807941 [01:41<30:02, 937.09 examples/s]
Tokenizing train dataset:   7%|▋         | 119127/1807941 [01:41<29:51, 942.46 examples/s]
Tokenizing train dataset:   7%|▋         | 119232/1807941 [01:41<29:07, 966.23 examples/s]
Tokenizing train dataset:   7%|▋         | 119341/1807941 [01:41<28:21, 992.66 examples/s]
Tokenizing train dataset:   7%|▋         | 119457/1807941 [01:41<27:11, 1034.74 examples/s]
Tokenizing train dataset:   7%|▋         | 119566/1807941 [01:41<26:51, 1047.64 examples/s]
Tokenizing train dataset:   7%|▋         | 119721/1807941 [01:42<27:03, 1039.62 examples/s]
Tokenizing train dataset:   7%|▋         | 119833/1807941 [01:42<26:35, 1058.36 examples/s]
Tokenizing train dataset:   7%|▋         | 119950/1807941 [01:42<25:53, 1086.87 examples/s]
Tokenizing train dataset:   7%|▋         | 120099/1807941 [01:42<27:45, 1013.20 examples/s]
Tokenizing train dataset:   7%|▋         | 120227/1807941 [01:42<26:06, 1077.68 examples/s]
Tokenizing train dataset:   7%|▋         | 120364/1807941 [01:42<24:23, 1153.34 examples/s]
Tokenizing train dataset:   7%|▋         | 120486/1807941 [01:42<24:05, 1167.39 examples/s]
Tokenizing train dataset:   7%|▋         | 120672/1807941 [01:42<23:37, 1190.70 examples/s]
Tokenizing train dataset:   7%|▋         | 120812/1807941 [01:42<22:36, 1243.54 examples/s]
Tokenizing train dataset:   7%|▋         | 120939/1807941 [01:43<22:33, 1246.72 examples/s]
Tokenizing train dataset:   7%|▋         | 121066/1807941 [01:43<23:39, 1188.40 examples/s]
Tokenizing train dataset:   7%|▋         | 121208/1807941 [01:43<22:31, 1248.21 examples/s]
Tokenizing train dataset:   7%|▋         | 121340/1807941 [01:43<22:11, 1267.09 examples/s]
Tokenizing train dataset:   7%|▋         | 121469/1807941 [01:43<22:06, 1271.16 examples/s]
Tokenizing train dataset:   7%|▋         | 121607/1807941 [01:43<21:37, 1299.77 examples/s]
Tokenizing train dataset:   7%|▋         | 121781/1807941 [01:43<22:41, 1238.12 examples/s]
Tokenizing train dataset:   7%|▋         | 121934/1807941 [01:43<24:14, 1159.44 examples/s]
Tokenizing train dataset:   7%|▋         | 122054/1807941 [01:44<30:09, 931.59 examples/s] 
Tokenizing train dataset:   7%|▋         | 122167/1807941 [01:44<32:03, 876.45 examples/s]
Tokenizing train dataset:   7%|▋         | 122298/1807941 [01:44<35:44, 785.99 examples/s]
Tokenizing train dataset:   7%|▋         | 122390/1807941 [01:44<38:20, 732.65 examples/s]
Tokenizing train dataset:   7%|▋         | 122483/1807941 [01:44<40:27, 694.29 examples/s]
Tokenizing train dataset:   7%|▋         | 122566/1807941 [01:44<38:55, 721.60 examples/s]
Tokenizing train dataset:   7%|▋         | 122653/1807941 [01:44<37:18, 752.83 examples/s]
Tokenizing train dataset:   7%|▋         | 122777/1807941 [01:45<32:09, 873.18 examples/s]
Tokenizing train dataset:   7%|▋         | 122898/1807941 [01:45<29:18, 958.31 examples/s]
Tokenizing train dataset:   7%|▋         | 123000/1807941 [01:45<33:21, 841.72 examples/s]
Tokenizing train dataset:   7%|▋         | 123130/1807941 [01:45<29:21, 956.23 examples/s]
Tokenizing train dataset:   7%|▋         | 123269/1807941 [01:45<26:18, 1067.13 examples/s]
Tokenizing train dataset:   7%|▋         | 123383/1807941 [01:45<29:40, 946.19 examples/s] 
Tokenizing train dataset:   7%|▋         | 123517/1807941 [01:45<26:52, 1044.33 examples/s]
Tokenizing train dataset:   7%|▋         | 123645/1807941 [01:45<25:22, 1106.19 examples/s]
Tokenizing train dataset:   7%|▋         | 123789/1807941 [01:45<23:30, 1194.30 examples/s]
Tokenizing train dataset:   7%|▋         | 123917/1807941 [01:46<23:04, 1216.74 examples/s]
Tokenizing train dataset:   7%|▋         | 124067/1807941 [01:46<24:48, 1131.15 examples/s]
Tokenizing train dataset:   7%|▋         | 124201/1807941 [01:46<23:42, 1183.47 examples/s]
Tokenizing train dataset:   7%|▋         | 124336/1807941 [01:46<22:50, 1228.03 examples/s]
Tokenizing train dataset:   7%|▋         | 124462/1807941 [01:46<22:43, 1234.43 examples/s]
Tokenizing train dataset:   7%|▋         | 124588/1807941 [01:46<22:37, 1239.66 examples/s]
Tokenizing train dataset:   7%|▋         | 124782/1807941 [01:46<22:19, 1256.45 examples/s]
Tokenizing train dataset:   7%|▋         | 124911/1807941 [01:46<22:13, 1261.83 examples/s]
Tokenizing train dataset:   7%|▋         | 125084/1807941 [01:47<23:00, 1218.92 examples/s]
Tokenizing train dataset:   7%|▋         | 125213/1807941 [01:47<22:43, 1233.94 examples/s]
Tokenizing train dataset:   7%|▋         | 125392/1807941 [01:47<23:05, 1214.38 examples/s]
Tokenizing train dataset:   7%|▋         | 125532/1807941 [01:47<28:15, 992.32 examples/s] 
Tokenizing train dataset:   7%|▋         | 125656/1807941 [01:47<29:52, 938.27 examples/s]
Tokenizing train dataset:   7%|▋         | 125764/1807941 [01:47<28:57, 967.98 examples/s]
Tokenizing train dataset:   7%|▋         | 125869/1807941 [01:47<28:25, 986.54 examples/s]
Tokenizing train dataset:   7%|▋         | 125992/1807941 [01:47<26:50, 1044.46 examples/s]
Tokenizing train dataset:   7%|▋         | 126123/1807941 [01:48<28:32, 982.03 examples/s] 
Tokenizing train dataset:   7%|▋         | 126262/1807941 [01:48<25:55, 1081.20 examples/s]
Tokenizing train dataset:   7%|▋         | 126391/1807941 [01:48<24:43, 1133.81 examples/s]
Tokenizing train dataset:   7%|▋         | 126520/1807941 [01:48<23:52, 1173.70 examples/s]
Tokenizing train dataset:   7%|▋         | 126648/1807941 [01:48<23:20, 1200.91 examples/s]
Tokenizing train dataset:   7%|▋         | 126829/1807941 [01:48<23:21, 1199.46 examples/s]
Tokenizing train dataset:   7%|▋         | 126953/1807941 [01:48<23:10, 1208.65 examples/s]
Tokenizing train dataset:   7%|▋         | 127147/1807941 [01:48<22:53, 1223.32 examples/s]
Tokenizing train dataset:   7%|▋         | 127279/1807941 [01:48<22:28, 1246.27 examples/s]
Tokenizing train dataset:   7%|▋         | 127446/1807941 [01:49<23:29, 1191.95 examples/s]
Tokenizing train dataset:   7%|▋         | 127581/1807941 [01:49<28:34, 980.08 examples/s] 
Tokenizing train dataset:   7%|▋         | 127722/1807941 [01:49<26:08, 1071.21 examples/s]
Tokenizing train dataset:   7%|▋         | 127884/1807941 [01:49<29:05, 962.57 examples/s] 
Tokenizing train dataset:   7%|▋         | 127998/1807941 [01:49<28:02, 998.60 examples/s]
Tokenizing train dataset:   7%|▋         | 128132/1807941 [01:49<29:01, 964.49 examples/s]
Tokenizing train dataset:   7%|▋         | 128257/1807941 [01:49<27:13, 1028.06 examples/s]
Tokenizing train dataset:   7%|▋         | 128386/1807941 [01:50<25:39, 1090.81 examples/s]
Tokenizing train dataset:   7%|▋         | 128510/1807941 [01:50<24:51, 1125.84 examples/s]
Tokenizing train dataset:   7%|▋         | 128633/1807941 [01:50<24:18, 1151.55 examples/s]
Tokenizing train dataset:   7%|▋         | 128778/1807941 [01:50<26:02, 1074.63 examples/s]
Tokenizing train dataset:   7%|▋         | 128943/1807941 [01:50<25:56, 1078.76 examples/s]
Tokenizing train dataset:   7%|▋         | 129057/1807941 [01:50<28:52, 969.17 examples/s] 
Tokenizing train dataset:   7%|▋         | 129179/1807941 [01:50<27:15, 1026.20 examples/s]
Tokenizing train dataset:   7%|▋         | 129310/1807941 [01:50<25:32, 1095.05 examples/s]
Tokenizing train dataset:   7%|▋         | 129435/1807941 [01:51<24:40, 1133.40 examples/s]
Tokenizing train dataset:   7%|▋         | 129569/1807941 [01:51<23:34, 1186.90 examples/s]
Tokenizing train dataset:   7%|▋         | 129705/1807941 [01:51<22:41, 1232.28 examples/s]
Tokenizing train dataset:   7%|▋         | 129870/1807941 [01:51<23:42, 1179.58 examples/s]
Tokenizing train dataset:   7%|▋         | 129993/1807941 [01:51<23:29, 1190.70 examples/s]
Tokenizing train dataset:   7%|▋         | 130150/1807941 [01:51<24:36, 1136.66 examples/s]
Tokenizing train dataset:   7%|▋         | 130284/1807941 [01:51<23:34, 1186.26 examples/s]
Tokenizing train dataset:   7%|▋         | 130415/1807941 [01:51<22:59, 1216.03 examples/s]
Tokenizing train dataset:   7%|▋         | 130545/1807941 [01:51<22:35, 1237.50 examples/s]
Tokenizing train dataset:   7%|▋         | 130689/1807941 [01:52<24:45, 1129.08 examples/s]
Tokenizing train dataset:   7%|▋         | 130845/1807941 [01:52<25:30, 1095.54 examples/s]
Tokenizing train dataset:   7%|▋         | 130983/1807941 [01:52<24:01, 1162.95 examples/s]
Tokenizing train dataset:   7%|▋         | 131121/1807941 [01:52<29:07, 959.77 examples/s] 
Tokenizing train dataset:   7%|▋         | 131253/1807941 [01:52<29:55, 933.81 examples/s]
Tokenizing train dataset:   7%|▋         | 131388/1807941 [01:52<27:14, 1025.94 examples/s]
Tokenizing train dataset:   7%|▋         | 131556/1807941 [01:52<26:36, 1050.13 examples/s]
Tokenizing train dataset:   7%|▋         | 131671/1807941 [01:53<26:04, 1071.53 examples/s]
Tokenizing train dataset:   7%|▋         | 131791/1807941 [01:53<25:23, 1100.48 examples/s]
Tokenizing train dataset:   7%|▋         | 131908/1807941 [01:53<25:00, 1117.09 examples/s]
Tokenizing train dataset:   7%|▋         | 132057/1807941 [01:53<26:17, 1062.50 examples/s]
Tokenizing train dataset:   7%|▋         | 132166/1807941 [01:53<29:28, 947.35 examples/s] 
Tokenizing train dataset:   7%|▋         | 132278/1807941 [01:53<28:16, 987.90 examples/s]
Tokenizing train dataset:   7%|▋         | 132402/1807941 [01:53<26:36, 1049.72 examples/s]
Tokenizing train dataset:   7%|▋         | 132512/1807941 [01:53<26:17, 1061.93 examples/s]
Tokenizing train dataset:   7%|▋         | 132642/1807941 [01:53<24:51, 1123.33 examples/s]
Tokenizing train dataset:   7%|▋         | 132757/1807941 [01:54<24:42, 1129.90 examples/s]
Tokenizing train dataset:   7%|▋         | 132911/1807941 [01:54<25:46, 1083.28 examples/s]
Tokenizing train dataset:   7%|▋         | 133070/1807941 [01:54<26:04, 1070.56 examples/s]
Tokenizing train dataset:   7%|▋         | 133197/1807941 [01:54<24:56, 1119.24 examples/s]
Tokenizing train dataset:   7%|▋         | 133347/1807941 [01:54<29:05, 959.52 examples/s] 
Tokenizing train dataset:   7%|▋         | 133479/1807941 [01:54<26:51, 1039.23 examples/s]
Tokenizing train dataset:   7%|▋         | 133599/1807941 [01:54<25:53, 1077.48 examples/s]
Tokenizing train dataset:   7%|▋         | 133719/1807941 [01:54<25:12, 1106.68 examples/s]
Tokenizing train dataset:   7%|▋         | 133847/1807941 [01:55<24:14, 1150.81 examples/s]
Tokenizing train dataset:   7%|▋         | 134000/1807941 [01:55<26:42, 1044.80 examples/s]
Tokenizing train dataset:   7%|▋         | 134124/1807941 [01:55<25:35, 1089.94 examples/s]
Tokenizing train dataset:   7%|▋         | 134243/1807941 [01:55<25:02, 1113.85 examples/s]
Tokenizing train dataset:   7%|▋         | 134362/1807941 [01:55<24:36, 1133.60 examples/s]
Tokenizing train dataset:   7%|▋         | 134490/1807941 [01:55<23:46, 1173.07 examples/s]
Tokenizing train dataset:   7%|▋         | 134631/1807941 [01:55<22:30, 1239.37 examples/s]
Tokenizing train dataset:   7%|▋         | 134825/1807941 [01:55<22:14, 1254.16 examples/s]
Tokenizing train dataset:   7%|▋         | 134960/1807941 [01:56<21:50, 1276.94 examples/s]
Tokenizing train dataset:   7%|▋         | 135118/1807941 [01:56<23:39, 1178.71 examples/s]
Tokenizing train dataset:   7%|▋         | 135305/1807941 [01:56<23:15, 1198.43 examples/s]
Tokenizing train dataset:   7%|▋         | 135433/1807941 [01:56<22:55, 1216.32 examples/s]
Tokenizing train dataset:   8%|▊         | 135618/1807941 [01:56<22:49, 1220.85 examples/s]
Tokenizing train dataset:   8%|▊         | 135753/1807941 [01:56<22:16, 1251.16 examples/s]
Tokenizing train dataset:   8%|▊         | 135897/1807941 [01:56<21:30, 1296.07 examples/s]
Tokenizing train dataset:   8%|▊         | 136074/1807941 [01:57<25:49, 1079.14 examples/s]
Tokenizing train dataset:   8%|▊         | 136218/1807941 [01:57<29:37, 940.60 examples/s] 
Tokenizing train dataset:   8%|▊         | 136340/1807941 [01:57<30:50, 903.25 examples/s]
Tokenizing train dataset:   8%|▊         | 136478/1807941 [01:57<30:55, 900.60 examples/s]
Tokenizing train dataset:   8%|▊         | 136588/1807941 [01:57<29:36, 940.95 examples/s]
Tokenizing train dataset:   8%|▊         | 136714/1807941 [01:57<27:29, 1013.08 examples/s]
Tokenizing train dataset:   8%|▊         | 136853/1807941 [01:57<25:11, 1105.63 examples/s]
Tokenizing train dataset:   8%|▊         | 136985/1807941 [01:57<23:59, 1160.90 examples/s]
Tokenizing train dataset:   8%|▊         | 137108/1807941 [01:58<27:04, 1028.70 examples/s]
Tokenizing train dataset:   8%|▊         | 137235/1807941 [01:58<25:36, 1087.57 examples/s]
Tokenizing train dataset:   8%|▊         | 137357/1807941 [01:58<24:49, 1121.20 examples/s]
Tokenizing train dataset:   8%|▊         | 137502/1807941 [01:58<23:04, 1206.76 examples/s]
Tokenizing train dataset:   8%|▊         | 137632/1807941 [01:58<22:38, 1229.65 examples/s]
Tokenizing train dataset:   8%|▊         | 137768/1807941 [01:58<22:01, 1263.92 examples/s]
Tokenizing train dataset:   8%|▊         | 137905/1807941 [01:58<24:42, 1126.77 examples/s]
Tokenizing train dataset:   8%|▊         | 138050/1807941 [01:58<26:15, 1060.07 examples/s]
Tokenizing train dataset:   8%|▊         | 138200/1807941 [01:59<26:48, 1038.17 examples/s]
Tokenizing train dataset:   8%|▊         | 138318/1807941 [01:59<26:03, 1068.08 examples/s]
Tokenizing train dataset:   8%|▊         | 138432/1807941 [01:59<25:39, 1084.61 examples/s]
Tokenizing train dataset:   8%|▊         | 138575/1807941 [01:59<23:41, 1174.70 examples/s]
Tokenizing train dataset:   8%|▊         | 138706/1807941 [01:59<23:03, 1206.34 examples/s]
Tokenizing train dataset:   8%|▊         | 138865/1807941 [01:59<21:11, 1312.57 examples/s]
Tokenizing train dataset:   8%|▊         | 139000/1807941 [01:59<22:05, 1258.88 examples/s]
Tokenizing train dataset:   8%|▊         | 139150/1807941 [01:59<21:01, 1322.70 examples/s]
Tokenizing train dataset:   8%|▊         | 139338/1807941 [01:59<21:32, 1290.98 examples/s]
Tokenizing train dataset:   8%|▊         | 139486/1807941 [02:00<20:47, 1337.96 examples/s]
Tokenizing train dataset:   8%|▊         | 139664/1807941 [02:00<21:44, 1279.34 examples/s]
Tokenizing train dataset:   8%|▊         | 139812/1807941 [02:00<20:55, 1328.86 examples/s]
Tokenizing train dataset:   8%|▊         | 139992/1807941 [02:00<21:43, 1279.95 examples/s]
Tokenizing train dataset:   8%|▊         | 140162/1807941 [02:00<22:37, 1228.26 examples/s]
Tokenizing train dataset:   8%|▊         | 140290/1807941 [02:00<22:26, 1238.63 examples/s]
Tokenizing train dataset:   8%|▊         | 140472/1807941 [02:00<22:37, 1228.03 examples/s]
Tokenizing train dataset:   8%|▊         | 140651/1807941 [02:00<22:52, 1214.38 examples/s]
Tokenizing train dataset:   8%|▊         | 140835/1807941 [02:01<22:53, 1213.76 examples/s]
Tokenizing train dataset:   8%|▊         | 140958/1807941 [02:01<22:55, 1212.17 examples/s]
Tokenizing train dataset:   8%|▊         | 141133/1807941 [02:01<23:43, 1171.08 examples/s]
Tokenizing train dataset:   8%|▊         | 141303/1807941 [02:01<23:59, 1157.75 examples/s]
Tokenizing train dataset:   8%|▊         | 141430/1807941 [02:01<23:31, 1180.60 examples/s]
Tokenizing train dataset:   8%|▊         | 141557/1807941 [02:01<23:10, 1198.29 examples/s]
Tokenizing train dataset:   8%|▊         | 141683/1807941 [02:01<22:56, 1210.73 examples/s]
Tokenizing train dataset:   8%|▊         | 141821/1807941 [02:01<22:07, 1254.86 examples/s]
Tokenizing train dataset:   8%|▊         | 141968/1807941 [02:02<21:09, 1312.08 examples/s]
Tokenizing train dataset:   8%|▊         | 142136/1807941 [02:02<22:25, 1238.16 examples/s]
Tokenizing train dataset:   8%|▊         | 142285/1807941 [02:02<21:18, 1302.35 examples/s]
Tokenizing train dataset:   8%|▊         | 142428/1807941 [02:02<20:46, 1336.59 examples/s]
Tokenizing train dataset:   8%|▊         | 142618/1807941 [02:02<21:14, 1307.13 examples/s]
Tokenizing train dataset:   8%|▊         | 142767/1807941 [02:02<20:30, 1353.23 examples/s]
Tokenizing train dataset:   8%|▊         | 142905/1807941 [02:02<20:25, 1358.14 examples/s]
Tokenizing train dataset:   8%|▊         | 143093/1807941 [02:02<21:03, 1317.27 examples/s]
Tokenizing train dataset:   8%|▊         | 143228/1807941 [02:02<20:58, 1322.67 examples/s]
Tokenizing train dataset:   8%|▊         | 143430/1807941 [02:03<20:54, 1327.28 examples/s]
Tokenizing train dataset:   8%|▊         | 143564/1807941 [02:03<20:52, 1328.76 examples/s]
Tokenizing train dataset:   8%|▊         | 143732/1807941 [02:03<22:12, 1249.00 examples/s]
Tokenizing train dataset:   8%|▊         | 143881/1807941 [02:03<29:24, 943.11 examples/s] 
Tokenizing train dataset:   8%|▊         | 143989/1807941 [02:03<31:37, 876.74 examples/s]
Tokenizing train dataset:   8%|▊         | 144118/1807941 [02:03<32:04, 864.63 examples/s]
Tokenizing train dataset:   8%|▊         | 144254/1807941 [02:04<28:38, 968.14 examples/s]
Tokenizing train dataset:   8%|▊         | 144411/1807941 [02:04<25:07, 1103.82 examples/s]
Tokenizing train dataset:   8%|▊         | 144555/1807941 [02:04<23:22, 1186.08 examples/s]
Tokenizing train dataset:   8%|▊         | 144730/1807941 [02:04<23:34, 1176.08 examples/s]
Tokenizing train dataset:   8%|▊         | 144926/1807941 [02:04<22:47, 1215.70 examples/s]
Tokenizing train dataset:   8%|▊         | 145107/1807941 [02:04<22:52, 1211.53 examples/s]
Tokenizing train dataset:   8%|▊         | 145278/1807941 [02:04<23:20, 1187.23 examples/s]
Tokenizing train dataset:   8%|▊         | 145404/1807941 [02:04<23:01, 1203.18 examples/s]
Tokenizing train dataset:   8%|▊         | 145544/1807941 [02:05<22:13, 1246.93 examples/s]
Tokenizing train dataset:   8%|▊         | 145683/1807941 [02:05<24:29, 1131.54 examples/s]
Tokenizing train dataset:   8%|▊         | 145820/1807941 [02:05<23:23, 1184.04 examples/s]
Tokenizing train dataset:   8%|▊         | 145955/1807941 [02:05<22:36, 1225.03 examples/s]
Tokenizing train dataset:   8%|▊         | 146127/1807941 [02:05<23:25, 1182.30 examples/s]
Tokenizing train dataset:   8%|▊         | 146271/1807941 [02:05<25:07, 1101.99 examples/s]
Tokenizing train dataset:   8%|▊         | 146412/1807941 [02:05<23:38, 1171.74 examples/s]
Tokenizing train dataset:   8%|▊         | 146558/1807941 [02:05<22:15, 1243.87 examples/s]
Tokenizing train dataset:   8%|▊         | 146692/1807941 [02:06<21:50, 1267.33 examples/s]
Tokenizing train dataset:   8%|▊         | 146861/1807941 [02:06<25:44, 1075.35 examples/s]
Tokenizing train dataset:   8%|▊         | 146981/1807941 [02:06<25:07, 1101.78 examples/s]
Tokenizing train dataset:   8%|▊         | 147135/1807941 [02:06<25:45, 1074.37 examples/s]
Tokenizing train dataset:   8%|▊         | 147272/1807941 [02:06<24:13, 1142.23 examples/s]
Tokenizing train dataset:   8%|▊         | 147392/1807941 [02:06<23:58, 1154.21 examples/s]
Tokenizing train dataset:   8%|▊         | 147533/1807941 [02:06<22:42, 1218.27 examples/s]
Tokenizing train dataset:   8%|▊         | 147673/1807941 [02:06<21:54, 1263.03 examples/s]
Tokenizing train dataset:   8%|▊         | 147819/1807941 [02:06<20:59, 1317.79 examples/s]
Tokenizing train dataset:   8%|▊         | 147960/1807941 [02:07<20:36, 1342.45 examples/s]
Tokenizing train dataset:   8%|▊         | 148124/1807941 [02:07<22:15, 1242.75 examples/s]
Tokenizing train dataset:   8%|▊         | 148290/1807941 [02:07<23:17, 1187.94 examples/s]
Tokenizing train dataset:   8%|▊         | 148418/1807941 [02:07<22:51, 1209.74 examples/s]
Tokenizing train dataset:   8%|▊         | 148556/1807941 [02:07<22:04, 1252.81 examples/s]
Tokenizing train dataset:   8%|▊         | 148752/1807941 [02:07<21:49, 1267.46 examples/s]
Tokenizing train dataset:   8%|▊         | 148893/1807941 [02:07<21:15, 1300.58 examples/s]
Tokenizing train dataset:   8%|▊         | 149075/1807941 [02:08<22:55, 1205.64 examples/s]
Tokenizing train dataset:   8%|▊         | 149204/1807941 [02:08<22:36, 1222.80 examples/s]
Tokenizing train dataset:   8%|▊         | 149329/1807941 [02:08<22:30, 1228.45 examples/s]
Tokenizing train dataset:   8%|▊         | 149457/1807941 [02:08<22:19, 1238.10 examples/s]
Tokenizing train dataset:   8%|▊         | 149590/1807941 [02:08<21:53, 1262.25 examples/s]
Tokenizing train dataset:   8%|▊         | 149780/1807941 [02:08<21:57, 1259.01 examples/s]
Tokenizing train dataset:   8%|▊         | 149907/1807941 [02:08<21:56, 1259.43 examples/s]
Tokenizing train dataset:   8%|▊         | 150056/1807941 [02:08<23:48, 1160.32 examples/s]
Tokenizing train dataset:   8%|▊         | 150230/1807941 [02:08<23:52, 1156.98 examples/s]
Tokenizing train dataset:   8%|▊         | 150355/1807941 [02:09<23:26, 1178.59 examples/s]
Tokenizing train dataset:   8%|▊         | 150480/1807941 [02:09<23:06, 1195.64 examples/s]
Tokenizing train dataset:   8%|▊         | 150619/1807941 [02:09<22:10, 1245.49 examples/s]
Tokenizing train dataset:   8%|▊         | 150774/1807941 [02:09<23:47, 1160.67 examples/s]
Tokenizing train dataset:   8%|▊         | 150907/1807941 [02:09<29:05, 949.13 examples/s] 
Tokenizing train dataset:   8%|▊         | 151039/1807941 [02:09<34:14, 806.58 examples/s]
Tokenizing train dataset:   8%|▊         | 151139/1807941 [02:10<36:12, 762.58 examples/s]
Tokenizing train dataset:   8%|▊         | 151249/1807941 [02:10<36:40, 753.04 examples/s]
Tokenizing train dataset:   8%|▊         | 151333/1807941 [02:10<35:51, 769.84 examples/s]
Tokenizing train dataset:   8%|▊         | 151439/1807941 [02:10<36:56, 747.36 examples/s]
Tokenizing train dataset:   8%|▊         | 151543/1807941 [02:10<34:05, 809.93 examples/s]
Tokenizing train dataset:   8%|▊         | 151629/1807941 [02:10<37:54, 728.22 examples/s]
Tokenizing train dataset:   8%|▊         | 151735/1807941 [02:10<34:16, 805.27 examples/s]
Tokenizing train dataset:   8%|▊         | 151833/1807941 [02:10<32:31, 848.69 examples/s]
Tokenizing train dataset:   8%|▊         | 151959/1807941 [02:10<28:55, 954.40 examples/s]
Tokenizing train dataset:   8%|▊         | 152065/1807941 [02:11<29:23, 938.90 examples/s]
Tokenizing train dataset:   8%|▊         | 152197/1807941 [02:11<26:33, 1039.10 examples/s]
Tokenizing train dataset:   8%|▊         | 152327/1807941 [02:11<24:55, 1107.18 examples/s]
Tokenizing train dataset:   8%|▊         | 152457/1807941 [02:11<23:50, 1157.61 examples/s]
Tokenizing train dataset:   8%|▊         | 152619/1807941 [02:11<24:31, 1125.05 examples/s]
Tokenizing train dataset:   8%|▊         | 152786/1807941 [02:11<24:41, 1117.31 examples/s]
Tokenizing train dataset:   8%|▊         | 152911/1807941 [02:11<23:59, 1149.87 examples/s]
Tokenizing train dataset:   8%|▊         | 153064/1807941 [02:11<25:10, 1095.64 examples/s]
Tokenizing train dataset:   8%|▊         | 153185/1807941 [02:12<24:34, 1122.23 examples/s]
Tokenizing train dataset:   8%|▊         | 153326/1807941 [02:12<26:16, 1049.64 examples/s]
Tokenizing train dataset:   8%|▊         | 153447/1807941 [02:12<25:24, 1085.06 examples/s]
Tokenizing train dataset:   8%|▊         | 153579/1807941 [02:12<24:05, 1144.31 examples/s]
Tokenizing train dataset:   9%|▊         | 153699/1807941 [02:12<23:50, 1156.35 examples/s]
Tokenizing train dataset:   9%|▊         | 153873/1807941 [02:12<23:52, 1154.65 examples/s]
Tokenizing train dataset:   9%|▊         | 153996/1807941 [02:12<23:32, 1171.20 examples/s]
Tokenizing train dataset:   9%|▊         | 154142/1807941 [02:12<25:06, 1097.47 examples/s]
Tokenizing train dataset:   9%|▊         | 154261/1807941 [02:13<24:39, 1117.44 examples/s]
Tokenizing train dataset:   9%|▊         | 154392/1807941 [02:13<23:38, 1165.93 examples/s]
Tokenizing train dataset:   9%|▊         | 154515/1807941 [02:13<23:22, 1179.26 examples/s]
Tokenizing train dataset:   9%|▊         | 154641/1807941 [02:13<22:58, 1199.11 examples/s]
Tokenizing train dataset:   9%|▊         | 154768/1807941 [02:13<22:37, 1217.60 examples/s]
Tokenizing train dataset:   9%|▊         | 154915/1807941 [02:13<24:34, 1121.35 examples/s]
Tokenizing train dataset:   9%|▊         | 155056/1807941 [02:13<29:18, 939.98 examples/s] 
Tokenizing train dataset:   9%|▊         | 155190/1807941 [02:13<26:43, 1030.63 examples/s]
Tokenizing train dataset:   9%|▊         | 155332/1807941 [02:13<24:30, 1124.05 examples/s]
Tokenizing train dataset:   9%|▊         | 155464/1807941 [02:14<23:28, 1172.97 examples/s]
Tokenizing train dataset:   9%|▊         | 155609/1807941 [02:14<22:06, 1245.83 examples/s]
Tokenizing train dataset:   9%|▊         | 155750/1807941 [02:14<21:22, 1287.95 examples/s]
Tokenizing train dataset:   9%|▊         | 155923/1807941 [02:14<25:10, 1093.44 examples/s]
Tokenizing train dataset:   9%|▊         | 156081/1807941 [02:14<25:35, 1075.62 examples/s]
Tokenizing train dataset:   9%|▊         | 156223/1807941 [02:14<23:50, 1154.88 examples/s]
Tokenizing train dataset:   9%|▊         | 156362/1807941 [02:14<22:43, 1211.20 examples/s]
Tokenizing train dataset:   9%|▊         | 156492/1807941 [02:14<22:21, 1230.83 examples/s]
Tokenizing train dataset:   9%|▊         | 156627/1807941 [02:15<21:51, 1258.94 examples/s]
Tokenizing train dataset:   9%|▊         | 156808/1807941 [02:15<22:15, 1236.33 examples/s]
Tokenizing train dataset:   9%|▊         | 156946/1807941 [02:15<21:39, 1270.41 examples/s]
Tokenizing train dataset:   9%|▊         | 157129/1807941 [02:15<22:58, 1197.24 examples/s]
Tokenizing train dataset:   9%|▊         | 157313/1807941 [02:15<22:51, 1203.32 examples/s]
Tokenizing train dataset:   9%|▊         | 157436/1807941 [02:15<22:48, 1206.10 examples/s]
Tokenizing train dataset:   9%|▊         | 157562/1807941 [02:15<22:38, 1214.71 examples/s]
Tokenizing train dataset:   9%|▊         | 157731/1807941 [02:15<23:17, 1180.88 examples/s]
Tokenizing train dataset:   9%|▊         | 157875/1807941 [02:16<24:55, 1103.70 examples/s]
Tokenizing train dataset:   9%|▊         | 158000/1807941 [02:16<30:10, 911.37 examples/s] 
Tokenizing train dataset:   9%|▊         | 158122/1807941 [02:16<28:11, 975.52 examples/s]
Tokenizing train dataset:   9%|▉         | 158271/1807941 [02:16<25:06, 1095.15 examples/s]
Tokenizing train dataset:   9%|▉         | 158394/1807941 [02:16<24:24, 1126.25 examples/s]
Tokenizing train dataset:   9%|▉         | 158523/1807941 [02:16<23:33, 1167.24 examples/s]
Tokenizing train dataset:   9%|▉         | 158659/1807941 [02:16<22:33, 1218.24 examples/s]
Tokenizing train dataset:   9%|▉         | 158793/1807941 [02:16<21:59, 1250.27 examples/s]
Tokenizing train dataset:   9%|▉         | 158935/1807941 [02:17<21:11, 1297.00 examples/s]
Tokenizing train dataset:   9%|▉         | 159123/1807941 [02:17<22:38, 1213.42 examples/s]
Tokenizing train dataset:   9%|▉         | 159262/1807941 [02:17<21:53, 1255.63 examples/s]
Tokenizing train dataset:   9%|▉         | 159401/1807941 [02:17<24:18, 1130.33 examples/s]
Tokenizing train dataset:   9%|▉         | 159546/1807941 [02:17<28:44, 955.98 examples/s] 
Tokenizing train dataset:   9%|▉         | 159703/1807941 [02:17<28:01, 980.07 examples/s]
Tokenizing train dataset:   9%|▉         | 159836/1807941 [02:18<31:54, 860.81 examples/s]
Tokenizing train dataset:   9%|▉         | 159960/1807941 [02:18<29:20, 936.00 examples/s]
Tokenizing train dataset:   9%|▉         | 160068/1807941 [02:18<29:31, 930.02 examples/s]
Tokenizing train dataset:   9%|▉         | 160196/1807941 [02:18<27:12, 1009.46 examples/s]
Tokenizing train dataset:   9%|▉         | 160330/1807941 [02:18<25:09, 1091.73 examples/s]
Tokenizing train dataset:   9%|▉         | 160464/1807941 [02:18<27:00, 1016.72 examples/s]
Tokenizing train dataset:   9%|▉         | 160576/1807941 [02:18<26:23, 1040.03 examples/s]
Tokenizing train dataset:   9%|▉         | 160735/1807941 [02:18<26:18, 1043.24 examples/s]
Tokenizing train dataset:   9%|▉         | 160866/1807941 [02:18<24:54, 1102.33 examples/s]
Tokenizing train dataset:   9%|▉         | 160997/1807941 [02:19<23:48, 1153.24 examples/s]
Tokenizing train dataset:   9%|▉         | 161162/1807941 [02:19<24:13, 1132.96 examples/s]
Tokenizing train dataset:   9%|▉         | 161284/1807941 [02:19<23:46, 1154.03 examples/s]
Tokenizing train dataset:   9%|▉         | 161418/1807941 [02:19<22:51, 1200.65 examples/s]
Tokenizing train dataset:   9%|▉         | 161551/1807941 [02:19<22:13, 1235.02 examples/s]
Tokenizing train dataset:   9%|▉         | 161681/1807941 [02:19<21:57, 1249.35 examples/s]
Tokenizing train dataset:   9%|▉         | 161809/1807941 [02:19<21:51, 1255.46 examples/s]
Tokenizing train dataset:   9%|▉         | 161952/1807941 [02:19<21:03, 1303.13 examples/s]
Tokenizing train dataset:   9%|▉         | 162118/1807941 [02:20<25:43, 1066.22 examples/s]
Tokenizing train dataset:   9%|▉         | 162248/1807941 [02:20<24:26, 1121.88 examples/s]
Tokenizing train dataset:   9%|▉         | 162376/1807941 [02:20<23:37, 1161.20 examples/s]
Tokenizing train dataset:   9%|▉         | 162509/1807941 [02:20<22:49, 1201.54 examples/s]
Tokenizing train dataset:   9%|▉         | 162695/1807941 [02:20<22:39, 1210.35 examples/s]
Tokenizing train dataset:   9%|▉         | 162855/1807941 [02:20<21:00, 1305.59 examples/s]
Tokenizing train dataset:   9%|▉         | 162994/1807941 [02:20<20:41, 1324.96 examples/s]
Tokenizing train dataset:   9%|▉         | 163171/1807941 [02:20<30:08, 909.49 examples/s] 
Tokenizing train dataset:   9%|▉         | 163285/1807941 [02:21<28:47, 952.09 examples/s]
Tokenizing train dataset:   9%|▉         | 163399/1807941 [02:21<27:39, 991.28 examples/s]
Tokenizing train dataset:   9%|▉         | 163512/1807941 [02:21<26:46, 1023.45 examples/s]
Tokenizing train dataset:   9%|▉         | 163635/1807941 [02:21<25:32, 1073.24 examples/s]
Tokenizing train dataset:   9%|▉         | 163766/1807941 [02:21<24:10, 1133.71 examples/s]
Tokenizing train dataset:   9%|▉         | 163891/1807941 [02:21<23:37, 1160.10 examples/s]
Tokenizing train dataset:   9%|▉         | 164058/1807941 [02:21<25:21, 1080.72 examples/s]
Tokenizing train dataset:   9%|▉         | 164221/1807941 [02:21<25:20, 1080.74 examples/s]
Tokenizing train dataset:   9%|▉         | 164346/1807941 [02:22<24:26, 1120.59 examples/s]
Tokenizing train dataset:   9%|▉         | 164479/1807941 [02:22<23:21, 1172.48 examples/s]
Tokenizing train dataset:   9%|▉         | 164605/1807941 [02:22<22:57, 1193.20 examples/s]
Tokenizing train dataset:   9%|▉         | 164790/1807941 [02:22<22:43, 1205.15 examples/s]
Tokenizing train dataset:   9%|▉         | 164968/1807941 [02:22<22:55, 1194.64 examples/s]
Tokenizing train dataset:   9%|▉         | 165092/1807941 [02:22<25:44, 1064.01 examples/s]
Tokenizing train dataset:   9%|▉         | 165213/1807941 [02:22<31:09, 878.80 examples/s] 
Tokenizing train dataset:   9%|▉         | 165333/1807941 [02:23<35:14, 776.96 examples/s]
Tokenizing train dataset:   9%|▉         | 165437/1807941 [02:23<36:25, 751.61 examples/s]
Tokenizing train dataset:   9%|▉         | 165549/1807941 [02:23<33:08, 825.87 examples/s]
Tokenizing train dataset:   9%|▉         | 165678/1807941 [02:23<29:28, 928.79 examples/s]
Tokenizing train dataset:   9%|▉         | 165800/1807941 [02:23<27:25, 997.86 examples/s]
Tokenizing train dataset:   9%|▉         | 165922/1807941 [02:23<25:59, 1053.04 examples/s]
Tokenizing train dataset:   9%|▉         | 166058/1807941 [02:23<27:25, 997.75 examples/s] 
Tokenizing train dataset:   9%|▉         | 166169/1807941 [02:23<26:42, 1024.20 examples/s]
Tokenizing train dataset:   9%|▉         | 166285/1807941 [02:23<25:50, 1058.69 examples/s]
Tokenizing train dataset:   9%|▉         | 166408/1807941 [02:24<24:50, 1101.42 examples/s]
Tokenizing train dataset:   9%|▉         | 166522/1807941 [02:24<24:39, 1109.52 examples/s]
Tokenizing train dataset:   9%|▉         | 166645/1807941 [02:24<23:58, 1140.77 examples/s]
Tokenizing train dataset:   9%|▉         | 166779/1807941 [02:24<26:13, 1043.31 examples/s]
Tokenizing train dataset:   9%|▉         | 166926/1807941 [02:24<26:59, 1013.15 examples/s]
Tokenizing train dataset:   9%|▉         | 167056/1807941 [02:24<32:07, 851.21 examples/s] 
Tokenizing train dataset:   9%|▉         | 167195/1807941 [02:24<28:16, 967.16 examples/s]
Tokenizing train dataset:   9%|▉         | 167305/1807941 [02:25<27:26, 996.45 examples/s]
Tokenizing train dataset:   9%|▉         | 167414/1807941 [02:25<26:55, 1015.61 examples/s]
Tokenizing train dataset:   9%|▉         | 167537/1807941 [02:25<25:32, 1070.43 examples/s]
Tokenizing train dataset:   9%|▉         | 167668/1807941 [02:25<24:05, 1134.43 examples/s]
Tokenizing train dataset:   9%|▉         | 167818/1807941 [02:25<25:16, 1081.56 examples/s]
Tokenizing train dataset:   9%|▉         | 167983/1807941 [02:25<28:11, 969.81 examples/s] 
Tokenizing train dataset:   9%|▉         | 168119/1807941 [02:25<28:52, 946.60 examples/s]
Tokenizing train dataset:   9%|▉         | 168246/1807941 [02:25<26:52, 1017.17 examples/s]
Tokenizing train dataset:   9%|▉         | 168375/1807941 [02:26<25:17, 1080.74 examples/s]
Tokenizing train dataset:   9%|▉         | 168489/1807941 [02:26<24:58, 1093.84 examples/s]
Tokenizing train dataset:   9%|▉         | 168625/1807941 [02:26<23:30, 1162.47 examples/s]
Tokenizing train dataset:   9%|▉         | 168750/1807941 [02:26<23:02, 1185.59 examples/s]
Tokenizing train dataset:   9%|▉         | 168875/1807941 [02:26<22:48, 1197.84 examples/s]
Tokenizing train dataset:   9%|▉         | 169000/1807941 [02:26<24:03, 1135.33 examples/s]
Tokenizing train dataset:   9%|▉         | 169164/1807941 [02:26<24:29, 1115.25 examples/s]
Tokenizing train dataset:   9%|▉         | 169282/1807941 [02:26<24:08, 1131.50 examples/s]
Tokenizing train dataset:   9%|▉         | 169402/1807941 [02:26<23:48, 1147.01 examples/s]
Tokenizing train dataset:   9%|▉         | 169532/1807941 [02:27<29:46, 917.07 examples/s] 
Tokenizing train dataset:   9%|▉         | 169689/1807941 [02:27<28:34, 955.29 examples/s]
Tokenizing train dataset:   9%|▉         | 169834/1807941 [02:27<28:31, 957.19 examples/s]
Tokenizing train dataset:   9%|▉         | 169962/1807941 [02:27<26:32, 1028.83 examples/s]
Tokenizing train dataset:   9%|▉         | 170124/1807941 [02:27<26:53, 1015.04 examples/s]
Tokenizing train dataset:   9%|▉         | 170252/1807941 [02:27<25:25, 1073.24 examples/s]
Tokenizing train dataset:   9%|▉         | 170376/1807941 [02:27<24:32, 1112.17 examples/s]
Tokenizing train dataset:   9%|▉         | 170505/1807941 [02:27<23:34, 1157.37 examples/s]
Tokenizing train dataset:   9%|▉         | 170640/1807941 [02:28<22:35, 1208.16 examples/s]
Tokenizing train dataset:   9%|▉         | 170772/1807941 [02:28<22:03, 1237.14 examples/s]
Tokenizing train dataset:   9%|▉         | 170956/1807941 [02:28<22:11, 1229.65 examples/s]
Tokenizing train dataset:   9%|▉         | 171122/1807941 [02:28<23:22, 1166.80 examples/s]
Tokenizing train dataset:   9%|▉         | 171258/1807941 [02:28<22:29, 1212.97 examples/s]
Tokenizing train dataset:   9%|▉         | 171391/1807941 [02:28<21:59, 1240.23 examples/s]
Tokenizing train dataset:   9%|▉         | 171576/1807941 [02:28<22:04, 1235.45 examples/s]
Tokenizing train dataset:  10%|▉         | 171764/1807941 [02:28<22:02, 1237.60 examples/s]
Tokenizing train dataset:  10%|▉         | 171909/1807941 [02:29<23:47, 1146.05 examples/s]
Tokenizing train dataset:  10%|▉         | 172045/1807941 [02:29<25:35, 1065.15 examples/s]
Tokenizing train dataset:  10%|▉         | 172196/1807941 [02:29<26:13, 1039.61 examples/s]
Tokenizing train dataset:  10%|▉         | 172340/1807941 [02:29<24:11, 1126.58 examples/s]
Tokenizing train dataset:  10%|▉         | 172466/1807941 [02:29<23:34, 1156.53 examples/s]
Tokenizing train dataset:  10%|▉         | 172612/1807941 [02:29<25:00, 1089.68 examples/s]
Tokenizing train dataset:  10%|▉         | 172743/1807941 [02:29<23:52, 1141.70 examples/s]
Tokenizing train dataset:  10%|▉         | 172875/1807941 [02:29<22:59, 1185.48 examples/s]
Tokenizing train dataset:  10%|▉         | 173000/1807941 [02:30<23:36, 1154.41 examples/s]
Tokenizing train dataset:  10%|▉         | 173131/1807941 [02:30<22:50, 1192.84 examples/s]
Tokenizing train dataset:  10%|▉         | 173259/1807941 [02:30<22:27, 1213.21 examples/s]
Tokenizing train dataset:  10%|▉         | 173382/1807941 [02:30<22:23, 1216.63 examples/s]
Tokenizing train dataset:  10%|▉         | 173511/1807941 [02:30<22:03, 1234.47 examples/s]
Tokenizing train dataset:  10%|▉         | 173639/1807941 [02:30<21:52, 1244.99 examples/s]
Tokenizing train dataset:  10%|▉         | 173779/1807941 [02:30<21:10, 1285.83 examples/s]
Tokenizing train dataset:  10%|▉         | 173944/1807941 [02:30<22:43, 1198.51 examples/s]
Tokenizing train dataset:  10%|▉         | 174084/1807941 [02:31<24:40, 1103.64 examples/s]
Tokenizing train dataset:  10%|▉         | 174226/1807941 [02:31<23:04, 1179.68 examples/s]
Tokenizing train dataset:  10%|▉         | 174370/1807941 [02:31<24:42, 1101.70 examples/s]
Tokenizing train dataset:  10%|▉         | 174491/1807941 [02:31<24:10, 1126.26 examples/s]
Tokenizing train dataset:  10%|▉         | 174626/1807941 [02:31<22:59, 1184.15 examples/s]
Tokenizing train dataset:  10%|▉         | 174756/1807941 [02:31<22:27, 1211.94 examples/s]
Tokenizing train dataset:  10%|▉         | 174883/1807941 [02:31<22:13, 1224.87 examples/s]
Tokenizing train dataset:  10%|▉         | 175065/1807941 [02:31<26:00, 1046.47 examples/s]
Tokenizing train dataset:  10%|▉         | 175185/1807941 [02:31<25:11, 1079.90 examples/s]
Tokenizing train dataset:  10%|▉         | 175318/1807941 [02:32<30:17, 898.19 examples/s] 
Tokenizing train dataset:  10%|▉         | 175469/1807941 [02:32<32:23, 840.17 examples/s]
Tokenizing train dataset:  10%|▉         | 175582/1807941 [02:32<30:19, 897.26 examples/s]
Tokenizing train dataset:  10%|▉         | 175760/1807941 [02:32<27:44, 980.55 examples/s]
Tokenizing train dataset:  10%|▉         | 175911/1807941 [02:32<27:34, 986.50 examples/s]
Tokenizing train dataset:  10%|▉         | 176058/1807941 [02:33<30:35, 888.92 examples/s]
Tokenizing train dataset:  10%|▉         | 176185/1807941 [02:33<31:14, 870.43 examples/s]
Tokenizing train dataset:  10%|▉         | 176278/1807941 [02:33<30:58, 877.94 examples/s]
Tokenizing train dataset:  10%|▉         | 176376/1807941 [02:33<30:25, 893.95 examples/s]
Tokenizing train dataset:  10%|▉         | 176506/1807941 [02:33<27:25, 991.62 examples/s]
Tokenizing train dataset:  10%|▉         | 176639/1807941 [02:33<25:13, 1077.82 examples/s]
Tokenizing train dataset:  10%|▉         | 176782/1807941 [02:33<23:14, 1169.62 examples/s]
Tokenizing train dataset:  10%|▉         | 176936/1807941 [02:33<24:25, 1112.83 examples/s]
Tokenizing train dataset:  10%|▉         | 177055/1807941 [02:33<25:37, 1060.43 examples/s]
Tokenizing train dataset:  10%|▉         | 177180/1807941 [02:34<24:33, 1106.53 examples/s]
Tokenizing train dataset:  10%|▉         | 177319/1807941 [02:34<23:04, 1177.91 examples/s]
Tokenizing train dataset:  10%|▉         | 177445/1807941 [02:34<22:41, 1197.55 examples/s]
Tokenizing train dataset:  10%|▉         | 177618/1807941 [02:34<23:17, 1166.69 examples/s]
Tokenizing train dataset:  10%|▉         | 177747/1807941 [02:34<28:36, 949.71 examples/s] 
Tokenizing train dataset:  10%|▉         | 177901/1807941 [02:34<28:02, 968.82 examples/s]
Tokenizing train dataset:  10%|▉         | 178004/1807941 [02:34<27:43, 980.02 examples/s]
Tokenizing train dataset:  10%|▉         | 178126/1807941 [02:34<26:15, 1034.42 examples/s]
Tokenizing train dataset:  10%|▉         | 178259/1807941 [02:35<30:55, 878.16 examples/s] 
Tokenizing train dataset:  10%|▉         | 178374/1807941 [02:35<29:00, 936.09 examples/s]
Tokenizing train dataset:  10%|▉         | 178495/1807941 [02:35<27:08, 1000.53 examples/s]
Tokenizing train dataset:  10%|▉         | 178618/1807941 [02:35<32:18, 840.53 examples/s] 
Tokenizing train dataset:  10%|▉         | 178738/1807941 [02:35<29:31, 919.47 examples/s]
Tokenizing train dataset:  10%|▉         | 178858/1807941 [02:35<27:31, 986.36 examples/s]
Tokenizing train dataset:  10%|▉         | 178992/1807941 [02:35<25:14, 1075.65 examples/s]
Tokenizing train dataset:  10%|▉         | 179139/1807941 [02:36<26:10, 1037.22 examples/s]
Tokenizing train dataset:  10%|▉         | 179274/1807941 [02:36<24:24, 1112.42 examples/s]
Tokenizing train dataset:  10%|▉         | 179409/1807941 [02:36<23:10, 1171.58 examples/s]
Tokenizing train dataset:  10%|▉         | 179551/1807941 [02:36<21:55, 1238.16 examples/s]
Tokenizing train dataset:  10%|▉         | 179699/1807941 [02:36<23:57, 1132.94 examples/s]
Tokenizing train dataset:  10%|▉         | 179821/1807941 [02:36<23:35, 1150.01 examples/s]
Tokenizing train dataset:  10%|▉         | 179994/1807941 [02:36<23:38, 1147.57 examples/s]
Tokenizing train dataset:  10%|▉         | 180137/1807941 [02:36<25:07, 1079.50 examples/s]
Tokenizing train dataset:  10%|▉         | 180280/1807941 [02:37<26:12, 1035.16 examples/s]
Tokenizing train dataset:  10%|▉         | 180401/1807941 [02:37<25:15, 1074.18 examples/s]
Tokenizing train dataset:  10%|▉         | 180551/1807941 [02:37<23:00, 1178.44 examples/s]
Tokenizing train dataset:  10%|▉         | 180683/1807941 [02:37<22:22, 1211.96 examples/s]
Tokenizing train dataset:  10%|█         | 180827/1807941 [02:37<21:20, 1270.55 examples/s]
Tokenizing train dataset:  10%|█         | 180970/1807941 [02:37<20:38, 1313.24 examples/s]
Tokenizing train dataset:  10%|█         | 181115/1807941 [02:37<23:00, 1178.63 examples/s]
Tokenizing train dataset:  10%|█         | 181281/1807941 [02:37<23:33, 1150.62 examples/s]
Tokenizing train dataset:  10%|█         | 181469/1807941 [02:38<22:56, 1181.78 examples/s]
Tokenizing train dataset:  10%|█         | 181607/1807941 [02:38<22:06, 1225.92 examples/s]
Tokenizing train dataset:  10%|█         | 181746/1807941 [02:38<21:25, 1264.64 examples/s]
Tokenizing train dataset:  10%|█         | 181888/1807941 [02:38<20:47, 1303.83 examples/s]
Tokenizing train dataset:  10%|█         | 182068/1807941 [02:38<21:25, 1264.71 examples/s]
Tokenizing train dataset:  10%|█         | 182220/1807941 [02:38<20:24, 1327.22 examples/s]
Tokenizing train dataset:  10%|█         | 182420/1807941 [02:38<20:27, 1324.25 examples/s]
Tokenizing train dataset:  10%|█         | 182560/1807941 [02:38<20:10, 1342.38 examples/s]
Tokenizing train dataset:  10%|█         | 182698/1807941 [02:38<20:04, 1348.98 examples/s]
Tokenizing train dataset:  10%|█         | 182842/1807941 [02:39<19:45, 1370.41 examples/s]
Tokenizing train dataset:  10%|█         | 183000/1807941 [02:39<22:23, 1209.08 examples/s]
Tokenizing train dataset:  10%|█         | 183173/1807941 [02:39<22:47, 1188.44 examples/s]
Tokenizing train dataset:  10%|█         | 183309/1807941 [02:39<22:03, 1227.97 examples/s]
Tokenizing train dataset:  10%|█         | 183443/1807941 [02:39<21:36, 1252.84 examples/s]
Tokenizing train dataset:  10%|█         | 183577/1807941 [02:39<21:13, 1275.11 examples/s]
Tokenizing train dataset:  10%|█         | 183713/1807941 [02:39<20:54, 1294.78 examples/s]
Tokenizing train dataset:  10%|█         | 183850/1807941 [02:39<20:36, 1313.98 examples/s]
Tokenizing train dataset:  10%|█         | 184007/1807941 [02:39<22:23, 1208.62 examples/s]
Tokenizing train dataset:  10%|█         | 184145/1807941 [02:40<21:36, 1252.21 examples/s]
Tokenizing train dataset:  10%|█         | 184274/1807941 [02:40<21:29, 1258.77 examples/s]
Tokenizing train dataset:  10%|█         | 184412/1807941 [02:40<21:01, 1286.65 examples/s]
Tokenizing train dataset:  10%|█         | 184556/1807941 [02:40<20:25, 1324.74 examples/s]
Tokenizing train dataset:  10%|█         | 184691/1807941 [02:40<20:21, 1328.77 examples/s]
Tokenizing train dataset:  10%|█         | 184832/1807941 [02:40<20:03, 1349.17 examples/s]
Tokenizing train dataset:  10%|█         | 184982/1807941 [02:40<22:22, 1209.02 examples/s]
Tokenizing train dataset:  10%|█         | 185145/1807941 [02:40<23:13, 1164.51 examples/s]
Tokenizing train dataset:  10%|█         | 185283/1807941 [02:41<25:02, 1080.13 examples/s]
Tokenizing train dataset:  10%|█         | 185410/1807941 [02:41<24:02, 1124.58 examples/s]
Tokenizing train dataset:  10%|█         | 185552/1807941 [02:41<25:34, 1057.32 examples/s]
Tokenizing train dataset:  10%|█         | 185696/1807941 [02:41<26:26, 1022.69 examples/s]
Tokenizing train dataset:  10%|█         | 185857/1807941 [02:41<23:18, 1159.53 examples/s]
Tokenizing train dataset:  10%|█         | 186000/1807941 [02:41<23:26, 1153.12 examples/s]
Tokenizing train dataset:  10%|█         | 186137/1807941 [02:41<22:25, 1205.47 examples/s]
Tokenizing train dataset:  10%|█         | 186283/1807941 [02:41<21:15, 1271.67 examples/s]
Tokenizing train dataset:  10%|█         | 186478/1807941 [02:42<21:08, 1278.37 examples/s]
Tokenizing train dataset:  10%|█         | 186666/1807941 [02:42<21:19, 1266.65 examples/s]
Tokenizing train dataset:  10%|█         | 186856/1807941 [02:42<21:23, 1262.72 examples/s]
Tokenizing train dataset:  10%|█         | 187007/1807941 [02:42<22:54, 1179.72 examples/s]
Tokenizing train dataset:  10%|█         | 187134/1807941 [02:42<22:40, 1191.52 examples/s]
Tokenizing train dataset:  10%|█         | 187275/1807941 [02:42<27:15, 990.65 examples/s] 
Tokenizing train dataset:  10%|█         | 187441/1807941 [02:42<26:24, 1022.68 examples/s]
Tokenizing train dataset:  10%|█         | 187573/1807941 [02:43<24:52, 1085.34 examples/s]
Tokenizing train dataset:  10%|█         | 187717/1807941 [02:43<23:06, 1168.17 examples/s]
Tokenizing train dataset:  10%|█         | 187840/1807941 [02:43<22:50, 1182.26 examples/s]
Tokenizing train dataset:  10%|█         | 187966/1807941 [02:43<22:28, 1201.27 examples/s]
Tokenizing train dataset:  10%|█         | 188105/1807941 [02:43<24:41, 1093.73 examples/s]
Tokenizing train dataset:  10%|█         | 188230/1807941 [02:43<23:50, 1132.36 examples/s]
Tokenizing train dataset:  10%|█         | 188350/1807941 [02:43<26:45, 1008.94 examples/s]
Tokenizing train dataset:  10%|█         | 188460/1807941 [02:43<29:39, 910.20 examples/s] 
Tokenizing train dataset:  10%|█         | 188557/1807941 [02:44<32:50, 821.72 examples/s]
Tokenizing train dataset:  10%|█         | 188651/1807941 [02:44<31:48, 848.28 examples/s]
Tokenizing train dataset:  10%|█         | 188770/1807941 [02:44<28:58, 931.50 examples/s]
Tokenizing train dataset:  10%|█         | 188882/1807941 [02:44<27:35, 978.06 examples/s]
Tokenizing train dataset:  10%|█         | 189000/1807941 [02:44<28:04, 961.26 examples/s]
Tokenizing train dataset:  10%|█         | 189124/1807941 [02:44<26:14, 1028.31 examples/s]
Tokenizing train dataset:  10%|█         | 189250/1807941 [02:44<31:43, 850.48 examples/s] 
Tokenizing train dataset:  10%|█         | 189381/1807941 [02:44<34:49, 774.52 examples/s]
Tokenizing train dataset:  10%|█         | 189511/1807941 [02:45<30:31, 883.49 examples/s]
Tokenizing train dataset:  10%|█         | 189650/1807941 [02:45<26:59, 999.49 examples/s]
Tokenizing train dataset:  10%|█         | 189780/1807941 [02:45<25:09, 1071.64 examples/s]
Tokenizing train dataset:  11%|█         | 189957/1807941 [02:45<21:33, 1250.38 examples/s]
Tokenizing train dataset:  11%|█         | 190174/1807941 [02:45<20:28, 1317.11 examples/s]
Tokenizing train dataset:  11%|█         | 190347/1807941 [02:45<18:58, 1420.54 examples/s]
Tokenizing train dataset:  11%|█         | 190529/1807941 [02:45<20:06, 1340.86 examples/s]
Tokenizing train dataset:  11%|█         | 190713/1807941 [02:45<20:44, 1299.19 examples/s]
Tokenizing train dataset:  11%|█         | 190906/1807941 [02:46<20:52, 1291.15 examples/s]
Tokenizing train dataset:  11%|█         | 191070/1807941 [02:46<21:57, 1226.88 examples/s]
Tokenizing train dataset:  11%|█         | 191255/1807941 [02:46<21:58, 1226.61 examples/s]
Tokenizing train dataset:  11%|█         | 191422/1807941 [02:46<22:42, 1186.35 examples/s]
Tokenizing train dataset:  11%|█         | 191576/1807941 [02:46<26:16, 1025.01 examples/s]
Tokenizing train dataset:  11%|█         | 191704/1807941 [02:46<25:02, 1075.42 examples/s]
Tokenizing train dataset:  11%|█         | 191835/1807941 [02:46<23:55, 1125.63 examples/s]
Tokenizing train dataset:  11%|█         | 191994/1807941 [02:47<24:27, 1101.00 examples/s]
Tokenizing train dataset:  11%|█         | 192142/1807941 [02:47<25:20, 1062.66 examples/s]
Tokenizing train dataset:  11%|█         | 192272/1807941 [02:47<24:07, 1116.05 examples/s]
Tokenizing train dataset:  11%|█         | 192391/1807941 [02:47<23:46, 1132.23 examples/s]
Tokenizing train dataset:  11%|█         | 192512/1807941 [02:47<23:25, 1149.45 examples/s]
Tokenizing train dataset:  11%|█         | 192631/1807941 [02:47<23:13, 1159.21 examples/s]
Tokenizing train dataset:  11%|█         | 192793/1807941 [02:47<23:54, 1125.68 examples/s]
Tokenizing train dataset:  11%|█         | 192944/1807941 [02:47<22:00, 1223.47 examples/s]
Tokenizing train dataset:  11%|█         | 193139/1807941 [02:48<22:04, 1219.05 examples/s]
Tokenizing train dataset:  11%|█         | 193265/1807941 [02:48<21:56, 1226.78 examples/s]
Tokenizing train dataset:  11%|█         | 193444/1807941 [02:48<22:16, 1208.27 examples/s]
Tokenizing train dataset:  11%|█         | 193589/1807941 [02:48<23:54, 1125.47 examples/s]
Tokenizing train dataset:  11%|█         | 193717/1807941 [02:48<23:11, 1159.71 examples/s]
Tokenizing train dataset:  11%|█         | 193888/1807941 [02:48<23:27, 1147.03 examples/s]
Tokenizing train dataset:  11%|█         | 194061/1807941 [02:48<24:01, 1119.91 examples/s]
Tokenizing train dataset:  11%|█         | 194188/1807941 [02:48<23:18, 1153.81 examples/s]
Tokenizing train dataset:  11%|█         | 194344/1807941 [02:49<24:08, 1113.63 examples/s]
Tokenizing train dataset:  11%|█         | 194472/1807941 [02:49<23:20, 1151.92 examples/s]
Tokenizing train dataset:  11%|█         | 194607/1807941 [02:49<22:25, 1199.26 examples/s]
Tokenizing train dataset:  11%|█         | 194731/1807941 [02:49<22:15, 1207.52 examples/s]
Tokenizing train dataset:  11%|█         | 194862/1807941 [02:49<24:58, 1076.58 examples/s]
Tokenizing train dataset:  11%|█         | 195000/1807941 [02:49<26:36, 1010.19 examples/s]
Tokenizing train dataset:  11%|█         | 195120/1807941 [02:49<25:28, 1055.26 examples/s]
Tokenizing train dataset:  11%|█         | 195236/1807941 [02:49<24:51, 1081.47 examples/s]
Tokenizing train dataset:  11%|█         | 195366/1807941 [02:50<23:39, 1136.22 examples/s]
Tokenizing train dataset:  11%|█         | 195492/1807941 [02:50<23:00, 1167.69 examples/s]
Tokenizing train dataset:  11%|█         | 195615/1807941 [02:50<22:43, 1182.79 examples/s]
Tokenizing train dataset:  11%|█         | 195747/1807941 [02:50<21:59, 1221.86 examples/s]
Tokenizing train dataset:  11%|█         | 195883/1807941 [02:50<21:19, 1259.99 examples/s]
Tokenizing train dataset:  11%|█         | 196065/1807941 [02:50<22:01, 1219.66 examples/s]
Tokenizing train dataset:  11%|█         | 196189/1807941 [02:50<21:56, 1224.61 examples/s]
Tokenizing train dataset:  11%|█         | 196320/1807941 [02:50<21:32, 1246.73 examples/s]
Tokenizing train dataset:  11%|█         | 196464/1807941 [02:51<26:43, 1005.09 examples/s]
Tokenizing train dataset:  11%|█         | 196590/1807941 [02:51<25:13, 1064.69 examples/s]
Tokenizing train dataset:  11%|█         | 196715/1807941 [02:51<24:12, 1109.17 examples/s]
Tokenizing train dataset:  11%|█         | 196872/1807941 [02:51<24:44, 1084.97 examples/s]
Tokenizing train dataset:  11%|█         | 196993/1807941 [02:51<24:04, 1115.02 examples/s]
Tokenizing train dataset:  11%|█         | 197162/1807941 [02:51<24:01, 1117.13 examples/s]
Tokenizing train dataset:  11%|█         | 197340/1807941 [02:51<23:36, 1137.13 examples/s]
Tokenizing train dataset:  11%|█         | 197479/1807941 [02:51<22:27, 1195.24 examples/s]
Tokenizing train dataset:  11%|█         | 197661/1807941 [02:52<22:23, 1198.29 examples/s]
Tokenizing train dataset:  11%|█         | 197837/1807941 [02:52<22:36, 1186.66 examples/s]
Tokenizing train dataset:  11%|█         | 198000/1807941 [02:52<23:54, 1122.45 examples/s]
Tokenizing train dataset:  11%|█         | 198154/1807941 [02:52<24:37, 1089.69 examples/s]
Tokenizing train dataset:  11%|█         | 198273/1807941 [02:52<24:10, 1109.81 examples/s]
Tokenizing train dataset:  11%|█         | 198387/1807941 [02:52<24:06, 1112.75 examples/s]
Tokenizing train dataset:  11%|█         | 198511/1807941 [02:52<23:27, 1143.30 examples/s]
Tokenizing train dataset:  11%|█         | 198648/1807941 [02:52<22:19, 1201.00 examples/s]
Tokenizing train dataset:  11%|█         | 198775/1807941 [02:53<22:02, 1217.17 examples/s]
Tokenizing train dataset:  11%|█         | 198951/1807941 [02:53<22:31, 1190.88 examples/s]
Tokenizing train dataset:  11%|█         | 199135/1807941 [02:53<22:53, 1171.66 examples/s]
Tokenizing train dataset:  11%|█         | 199259/1807941 [02:53<22:36, 1185.74 examples/s]
Tokenizing train dataset:  11%|█         | 199425/1807941 [02:53<23:12, 1154.72 examples/s]
Tokenizing train dataset:  11%|█         | 199600/1807941 [02:53<23:13, 1153.88 examples/s]
Tokenizing train dataset:  11%|█         | 199726/1807941 [02:53<22:45, 1177.98 examples/s]
Tokenizing train dataset:  11%|█         | 199868/1807941 [02:53<21:40, 1236.81 examples/s]
Tokenizing train dataset:  11%|█         | 200000/1807941 [02:54<22:10, 1208.48 examples/s]
Tokenizing train dataset:  11%|█         | 200124/1807941 [02:54<22:03, 1215.14 examples/s]
Tokenizing train dataset:  11%|█         | 200258/1807941 [02:54<21:28, 1247.62 examples/s]
Tokenizing train dataset:  11%|█         | 200445/1807941 [02:54<21:32, 1243.89 examples/s]
Tokenizing train dataset:  11%|█         | 200587/1807941 [02:54<20:50, 1285.25 examples/s]
Tokenizing train dataset:  11%|█         | 200781/1807941 [02:54<20:49, 1286.42 examples/s]
Tokenizing train dataset:  11%|█         | 200920/1807941 [02:54<20:25, 1310.82 examples/s]
Tokenizing train dataset:  11%|█         | 201083/1807941 [02:54<21:48, 1228.03 examples/s]
Tokenizing train dataset:  11%|█         | 201214/1807941 [02:55<21:29, 1246.46 examples/s]
Tokenizing train dataset:  11%|█         | 201389/1807941 [02:55<22:03, 1213.73 examples/s]
Tokenizing train dataset:  11%|█         | 201569/1807941 [02:55<22:12, 1205.78 examples/s]
Tokenizing train dataset:  11%|█         | 201747/1807941 [02:55<22:22, 1196.25 examples/s]
Tokenizing train dataset:  11%|█         | 201928/1807941 [02:55<22:21, 1197.05 examples/s]
Tokenizing train dataset:  11%|█         | 202061/1807941 [02:55<24:23, 1096.97 examples/s]
Tokenizing train dataset:  11%|█         | 202181/1807941 [02:55<23:56, 1118.06 examples/s]
Tokenizing train dataset:  11%|█         | 202346/1807941 [02:56<24:07, 1109.35 examples/s]
Tokenizing train dataset:  11%|█         | 202474/1807941 [02:56<23:18, 1148.02 examples/s]
Tokenizing train dataset:  11%|█         | 202658/1807941 [02:56<22:50, 1170.92 examples/s]
Tokenizing train dataset:  11%|█         | 202810/1807941 [02:56<26:43, 1001.19 examples/s]
Tokenizing train dataset:  11%|█         | 202953/1807941 [02:56<24:29, 1091.97 examples/s]
Tokenizing train dataset:  11%|█         | 203118/1807941 [02:56<24:51, 1075.66 examples/s]
Tokenizing train dataset:  11%|█         | 203236/1807941 [02:56<24:20, 1099.04 examples/s]
Tokenizing train dataset:  11%|█▏        | 203402/1807941 [02:56<24:21, 1097.84 examples/s]
Tokenizing train dataset:  11%|█▏        | 203525/1807941 [02:57<23:42, 1128.19 examples/s]
Tokenizing train dataset:  11%|█▏        | 203652/1807941 [02:57<23:02, 1160.39 examples/s]
Tokenizing train dataset:  11%|█▏        | 203778/1807941 [02:57<22:37, 1181.85 examples/s]
Tokenizing train dataset:  11%|█▏        | 203912/1807941 [02:57<21:54, 1219.90 examples/s]
Tokenizing train dataset:  11%|█▏        | 204071/1807941 [02:57<23:05, 1157.31 examples/s]
Tokenizing train dataset:  11%|█▏        | 204199/1807941 [02:57<22:30, 1187.52 examples/s]
Tokenizing train dataset:  11%|█▏        | 204334/1807941 [02:57<21:44, 1229.51 examples/s]
Tokenizing train dataset:  11%|█▏        | 204467/1807941 [02:57<21:18, 1254.29 examples/s]
Tokenizing train dataset:  11%|█▏        | 204622/1807941 [02:57<22:53, 1167.52 examples/s]
Tokenizing train dataset:  11%|█▏        | 204742/1807941 [02:58<22:45, 1173.71 examples/s]
Tokenizing train dataset:  11%|█▏        | 204880/1807941 [02:58<21:46, 1226.67 examples/s]
Tokenizing train dataset:  11%|█▏        | 205060/1807941 [02:58<22:27, 1189.36 examples/s]
Tokenizing train dataset:  11%|█▏        | 205187/1807941 [02:58<22:09, 1205.86 examples/s]
Tokenizing train dataset:  11%|█▏        | 205317/1807941 [02:58<21:45, 1227.53 examples/s]
Tokenizing train dataset:  11%|█▏        | 205457/1807941 [02:58<20:59, 1272.45 examples/s]
Tokenizing train dataset:  11%|█▏        | 205601/1807941 [02:58<20:18, 1315.21 examples/s]
Tokenizing train dataset:  11%|█▏        | 205735/1807941 [02:58<20:12, 1321.10 examples/s]
Tokenizing train dataset:  11%|█▏        | 205870/1807941 [02:58<20:05, 1328.64 examples/s]
Tokenizing train dataset:  11%|█▏        | 206008/1807941 [02:59<22:51, 1168.23 examples/s]
Tokenizing train dataset:  11%|█▏        | 206129/1807941 [02:59<22:39, 1178.07 examples/s]
Tokenizing train dataset:  11%|█▏        | 206299/1807941 [02:59<23:01, 1159.75 examples/s]
Tokenizing train dataset:  11%|█▏        | 206422/1807941 [02:59<22:43, 1174.97 examples/s]
Tokenizing train dataset:  11%|█▏        | 206549/1807941 [02:59<22:16, 1197.87 examples/s]
Tokenizing train dataset:  11%|█▏        | 206693/1807941 [02:59<21:08, 1262.43 examples/s]
Tokenizing train dataset:  11%|█▏        | 206843/1807941 [02:59<20:06, 1327.04 examples/s]
Tokenizing train dataset:  11%|█▏        | 207011/1807941 [02:59<21:25, 1245.32 examples/s]
Tokenizing train dataset:  11%|█▏        | 207194/1807941 [03:00<21:38, 1232.71 examples/s]
Tokenizing train dataset:  11%|█▏        | 207368/1807941 [03:00<22:09, 1204.23 examples/s]
Tokenizing train dataset:  11%|█▏        | 207544/1807941 [03:00<22:25, 1189.68 examples/s]
Tokenizing train dataset:  11%|█▏        | 207667/1807941 [03:00<22:16, 1197.29 examples/s]
Tokenizing train dataset:  11%|█▏        | 207825/1807941 [03:00<23:16, 1145.51 examples/s]
Tokenizing train dataset:  12%|█▏        | 207992/1807941 [03:00<23:36, 1129.31 examples/s]
Tokenizing train dataset:  12%|█▏        | 208153/1807941 [03:00<26:36, 1001.90 examples/s]
Tokenizing train dataset:  12%|█▏        | 208273/1807941 [03:01<25:35, 1041.46 examples/s]
Tokenizing train dataset:  12%|█▏        | 208384/1807941 [03:01<25:14, 1056.48 examples/s]
Tokenizing train dataset:  12%|█▏        | 208528/1807941 [03:01<29:06, 915.85 examples/s] 
Tokenizing train dataset:  12%|█▏        | 208639/1807941 [03:01<27:52, 956.21 examples/s]
Tokenizing train dataset:  12%|█▏        | 208761/1807941 [03:01<26:14, 1015.60 examples/s]
Tokenizing train dataset:  12%|█▏        | 208881/1807941 [03:01<25:07, 1060.89 examples/s]
Tokenizing train dataset:  12%|█▏        | 209000/1807941 [03:01<25:23, 1049.82 examples/s]
Tokenizing train dataset:  12%|█▏        | 209121/1807941 [03:01<24:24, 1091.60 examples/s]
Tokenizing train dataset:  12%|█▏        | 209253/1807941 [03:01<23:07, 1152.33 examples/s]
Tokenizing train dataset:  12%|█▏        | 209382/1807941 [03:02<22:27, 1186.36 examples/s]
Tokenizing train dataset:  12%|█▏        | 209542/1807941 [03:02<26:22, 1009.83 examples/s]
Tokenizing train dataset:  12%|█▏        | 209662/1807941 [03:02<25:14, 1054.98 examples/s]
Tokenizing train dataset:  12%|█▏        | 209792/1807941 [03:02<23:51, 1116.54 examples/s]
Tokenizing train dataset:  12%|█▏        | 209932/1807941 [03:02<22:23, 1189.03 examples/s]
Tokenizing train dataset:  12%|█▏        | 210063/1807941 [03:02<23:20, 1141.02 examples/s]
Tokenizing train dataset:  12%|█▏        | 210189/1807941 [03:02<22:47, 1168.79 examples/s]
Tokenizing train dataset:  12%|█▏        | 210317/1807941 [03:02<22:15, 1195.95 examples/s]
Tokenizing train dataset:  12%|█▏        | 210446/1807941 [03:03<21:50, 1218.87 examples/s]
Tokenizing train dataset:  12%|█▏        | 210632/1807941 [03:03<21:43, 1225.51 examples/s]
Tokenizing train dataset:  12%|█▏        | 210765/1807941 [03:03<21:18, 1248.84 examples/s]
Tokenizing train dataset:  12%|█▏        | 210892/1807941 [03:03<21:16, 1251.24 examples/s]
Tokenizing train dataset:  12%|█▏        | 211061/1807941 [03:03<22:32, 1180.34 examples/s]
Tokenizing train dataset:  12%|█▏        | 211197/1807941 [03:03<21:46, 1221.98 examples/s]
Tokenizing train dataset:  12%|█▏        | 211329/1807941 [03:03<21:21, 1245.44 examples/s]
Tokenizing train dataset:  12%|█▏        | 211459/1807941 [03:03<21:07, 1259.86 examples/s]
Tokenizing train dataset:  12%|█▏        | 211592/1807941 [03:03<20:51, 1275.58 examples/s]
Tokenizing train dataset:  12%|█▏        | 211724/1807941 [03:04<20:42, 1284.47 examples/s]
Tokenizing train dataset:  12%|█▏        | 211907/1807941 [03:04<21:12, 1254.67 examples/s]
Tokenizing train dataset:  12%|█▏        | 212068/1807941 [03:04<22:23, 1187.98 examples/s]
Tokenizing train dataset:  12%|█▏        | 212200/1807941 [03:04<21:50, 1217.82 examples/s]
Tokenizing train dataset:  12%|█▏        | 212330/1807941 [03:04<21:29, 1237.13 examples/s]
Tokenizing train dataset:  12%|█▏        | 212461/1807941 [03:04<21:10, 1255.82 examples/s]
Tokenizing train dataset:  12%|█▏        | 212588/1807941 [03:04<21:09, 1256.95 examples/s]
Tokenizing train dataset:  12%|█▏        | 212722/1807941 [03:04<20:50, 1275.33 examples/s]
Tokenizing train dataset:  12%|█▏        | 212914/1807941 [03:05<20:52, 1273.79 examples/s]
Tokenizing train dataset:  12%|█▏        | 213076/1807941 [03:05<22:06, 1202.32 examples/s]
Tokenizing train dataset:  12%|█▏        | 213212/1807941 [03:05<21:26, 1239.58 examples/s]
Tokenizing train dataset:  12%|█▏        | 213348/1807941 [03:05<20:59, 1266.45 examples/s]
Tokenizing train dataset:  12%|█▏        | 213490/1807941 [03:05<20:21, 1305.80 examples/s]
Tokenizing train dataset:  12%|█▏        | 213624/1807941 [03:05<20:16, 1311.00 examples/s]
Tokenizing train dataset:  12%|█▏        | 213819/1807941 [03:05<20:23, 1303.27 examples/s]
Tokenizing train dataset:  12%|█▏        | 214000/1807941 [03:05<21:46, 1220.45 examples/s]
Tokenizing train dataset:  12%|█▏        | 214179/1807941 [03:06<21:59, 1208.30 examples/s]
Tokenizing train dataset:  12%|█▏        | 214302/1807941 [03:06<21:58, 1208.70 examples/s]
Tokenizing train dataset:  12%|█▏        | 214431/1807941 [03:06<21:39, 1226.10 examples/s]
Tokenizing train dataset:  12%|█▏        | 214557/1807941 [03:06<21:32, 1232.75 examples/s]
Tokenizing train dataset:  12%|█▏        | 214686/1807941 [03:06<21:18, 1246.35 examples/s]
Tokenizing train dataset:  12%|█▏        | 214872/1807941 [03:06<21:25, 1239.50 examples/s]
Tokenizing train dataset:  12%|█▏        | 215000/1807941 [03:06<22:31, 1178.99 examples/s]
Tokenizing train dataset:  12%|█▏        | 215133/1807941 [03:06<21:49, 1216.16 examples/s]
Tokenizing train dataset:  12%|█▏        | 215267/1807941 [03:06<21:15, 1248.74 examples/s]
Tokenizing train dataset:  12%|█▏        | 215396/1807941 [03:07<21:08, 1255.83 examples/s]
Tokenizing train dataset:  12%|█▏        | 215582/1807941 [03:07<21:17, 1246.33 examples/s]
Tokenizing train dataset:  12%|█▏        | 215709/1807941 [03:07<21:12, 1251.03 examples/s]
Tokenizing train dataset:  12%|█▏        | 215847/1807941 [03:07<20:40, 1283.37 examples/s]
Tokenizing train dataset:  12%|█▏        | 216007/1807941 [03:07<22:07, 1199.15 examples/s]
Tokenizing train dataset:  12%|█▏        | 216142/1807941 [03:07<21:28, 1235.42 examples/s]
Tokenizing train dataset:  12%|█▏        | 216271/1807941 [03:07<21:16, 1246.76 examples/s]
Tokenizing train dataset:  12%|█▏        | 216398/1807941 [03:07<21:12, 1250.97 examples/s]
Tokenizing train dataset:  12%|█▏        | 216531/1807941 [03:07<20:53, 1269.65 examples/s]
Tokenizing train dataset:  12%|█▏        | 216664/1807941 [03:08<20:37, 1285.73 examples/s]
Tokenizing train dataset:  12%|█▏        | 216803/1807941 [03:08<20:11, 1313.64 examples/s]
Tokenizing train dataset:  12%|█▏        | 217000/1807941 [03:08<21:14, 1248.02 examples/s]
Tokenizing train dataset:  12%|█▏        | 217134/1807941 [03:08<20:53, 1269.23 examples/s]
Tokenizing train dataset:  12%|█▏        | 217268/1807941 [03:08<20:36, 1286.85 examples/s]
Tokenizing train dataset:  12%|█▏        | 217401/1807941 [03:08<20:26, 1296.61 examples/s]
Tokenizing train dataset:  12%|█▏        | 217534/1807941 [03:08<20:20, 1303.07 examples/s]
Tokenizing train dataset:  12%|█▏        | 217722/1807941 [03:08<20:45, 1277.01 examples/s]
Tokenizing train dataset:  12%|█▏        | 217856/1807941 [03:08<20:32, 1289.72 examples/s]
Tokenizing train dataset:  12%|█▏        | 217995/1807941 [03:09<20:10, 1313.68 examples/s]
Tokenizing train dataset:  12%|█▏        | 218154/1807941 [03:09<21:48, 1215.36 examples/s]
Tokenizing train dataset:  12%|█▏        | 218293/1807941 [03:09<21:05, 1256.40 examples/s]
Tokenizing train dataset:  12%|█▏        | 218426/1807941 [03:09<20:45, 1275.84 examples/s]
Tokenizing train dataset:  12%|█▏        | 218559/1807941 [03:09<20:32, 1289.32 examples/s]
Tokenizing train dataset:  12%|█▏        | 218695/1807941 [03:09<20:16, 1306.71 examples/s]
Tokenizing train dataset:  12%|█▏        | 218833/1807941 [03:09<19:59, 1324.93 examples/s]
Tokenizing train dataset:  12%|█▏        | 218980/1807941 [03:09<19:23, 1365.35 examples/s]
Tokenizing train dataset:  12%|█▏        | 219168/1807941 [03:09<20:04, 1319.07 examples/s]
Tokenizing train dataset:  12%|█▏        | 219309/1807941 [03:10<19:43, 1342.20 examples/s]
Tokenizing train dataset:  12%|█▏        | 219449/1807941 [03:10<19:31, 1356.40 examples/s]
Tokenizing train dataset:  12%|█▏        | 219597/1807941 [03:10<19:04, 1388.15 examples/s]
Tokenizing train dataset:  12%|█▏        | 219739/1807941 [03:10<18:58, 1394.66 examples/s]
Tokenizing train dataset:  12%|█▏        | 219950/1807941 [03:10<19:00, 1392.90 examples/s]
Tokenizing train dataset:  12%|█▏        | 220145/1807941 [03:10<19:34, 1351.75 examples/s]
Tokenizing train dataset:  12%|█▏        | 220294/1807941 [03:10<19:08, 1382.50 examples/s]
Tokenizing train dataset:  12%|█▏        | 220438/1807941 [03:10<18:57, 1395.22 examples/s]
Tokenizing train dataset:  12%|█▏        | 220581/1807941 [03:10<18:54, 1399.52 examples/s]
Tokenizing train dataset:  12%|█▏        | 220728/1807941 [03:11<18:41, 1414.97 examples/s]
Tokenizing train dataset:  12%|█▏        | 220874/1807941 [03:11<18:34, 1424.28 examples/s]
Tokenizing train dataset:  12%|█▏        | 221073/1807941 [03:11<19:18, 1369.25 examples/s]
Tokenizing train dataset:  12%|█▏        | 221217/1807941 [03:11<19:05, 1385.55 examples/s]
Tokenizing train dataset:  12%|█▏        | 221370/1807941 [03:11<18:34, 1424.04 examples/s]
Tokenizing train dataset:  12%|█▏        | 221521/1807941 [03:11<18:17, 1445.97 examples/s]
Tokenizing train dataset:  12%|█▏        | 221730/1807941 [03:11<18:34, 1422.93 examples/s]
Tokenizing train dataset:  12%|█▏        | 221879/1807941 [03:11<18:24, 1436.63 examples/s]
Tokenizing train dataset:  12%|█▏        | 222071/1807941 [03:12<19:09, 1379.15 examples/s]
Tokenizing train dataset:  12%|█▏        | 222219/1807941 [03:12<18:48, 1404.58 examples/s]
Tokenizing train dataset:  12%|█▏        | 222365/1807941 [03:12<18:38, 1417.14 examples/s]
Tokenizing train dataset:  12%|█▏        | 222517/1807941 [03:12<18:19, 1442.54 examples/s]
Tokenizing train dataset:  12%|█▏        | 222667/1807941 [03:12<18:09, 1455.54 examples/s]
Tokenizing train dataset:  12%|█▏        | 222814/1807941 [03:12<18:06, 1458.50 examples/s]
Tokenizing train dataset:  12%|█▏        | 222970/1807941 [03:12<17:46, 1486.05 examples/s]
Tokenizing train dataset:  12%|█▏        | 223163/1807941 [03:12<18:47, 1405.21 examples/s]
Tokenizing train dataset:  12%|█▏        | 223309/1807941 [03:12<18:38, 1417.34 examples/s]
Tokenizing train dataset:  12%|█▏        | 223457/1807941 [03:12<18:27, 1431.30 examples/s]
Tokenizing train dataset:  12%|█▏        | 223602/1807941 [03:13<18:26, 1432.42 examples/s]
Tokenizing train dataset:  12%|█▏        | 223811/1807941 [03:13<18:40, 1413.69 examples/s]
Tokenizing train dataset:  12%|█▏        | 224000/1807941 [03:13<20:13, 1305.74 examples/s]
Tokenizing train dataset:  12%|█▏        | 224188/1807941 [03:13<20:32, 1284.86 examples/s]
Tokenizing train dataset:  12%|█▏        | 224379/1807941 [03:13<20:39, 1277.18 examples/s]
Tokenizing train dataset:  12%|█▏        | 224513/1807941 [03:13<20:28, 1289.09 examples/s]
Tokenizing train dataset:  12%|█▏        | 224650/1807941 [03:13<20:11, 1306.72 examples/s]
Tokenizing train dataset:  12%|█▏        | 224788/1807941 [03:14<19:56, 1323.16 examples/s]
Tokenizing train dataset:  12%|█▏        | 224976/1807941 [03:14<20:23, 1294.16 examples/s]
Tokenizing train dataset:  12%|█▏        | 225139/1807941 [03:14<21:36, 1220.53 examples/s]
Tokenizing train dataset:  12%|█▏        | 225278/1807941 [03:14<20:57, 1258.37 examples/s]
Tokenizing train dataset:  12%|█▏        | 225408/1807941 [03:14<20:50, 1265.79 examples/s]
Tokenizing train dataset:  12%|█▏        | 225544/1807941 [03:14<20:27, 1289.49 examples/s]
Tokenizing train dataset:  12%|█▏        | 225681/1807941 [03:14<20:08, 1309.28 examples/s]
Tokenizing train dataset:  12%|█▏        | 225820/1807941 [03:14<19:48, 1330.70 examples/s]
Tokenizing train dataset:  12%|█▏        | 225956/1807941 [03:14<19:44, 1335.35 examples/s]
Tokenizing train dataset:  13%|█▎        | 226126/1807941 [03:15<21:26, 1229.42 examples/s]
Tokenizing train dataset:  13%|█▎        | 226261/1807941 [03:15<20:56, 1258.81 examples/s]
Tokenizing train dataset:  13%|█▎        | 226401/1807941 [03:15<20:23, 1292.98 examples/s]
Tokenizing train dataset:  13%|█▎        | 226549/1807941 [03:15<19:37, 1342.46 examples/s]
Tokenizing train dataset:  13%|█▎        | 226689/1807941 [03:15<19:26, 1355.45 examples/s]
Tokenizing train dataset:  13%|█▎        | 226836/1807941 [03:15<19:00, 1385.92 examples/s]
Tokenizing train dataset:  13%|█▎        | 226982/1807941 [03:15<18:45, 1404.79 examples/s]
Tokenizing train dataset:  13%|█▎        | 227157/1807941 [03:15<20:05, 1310.80 examples/s]
Tokenizing train dataset:  13%|█▎        | 227298/1807941 [03:15<19:44, 1334.98 examples/s]
Tokenizing train dataset:  13%|█▎        | 227440/1807941 [03:16<19:26, 1354.55 examples/s]
Tokenizing train dataset:  13%|█▎        | 227581/1807941 [03:16<19:17, 1365.76 examples/s]
Tokenizing train dataset:  13%|█▎        | 227721/1807941 [03:16<19:11, 1372.46 examples/s]
Tokenizing train dataset:  13%|█▎        | 227866/1807941 [03:16<18:52, 1394.83 examples/s]
Tokenizing train dataset:  13%|█▎        | 228066/1807941 [03:16<20:16, 1298.86 examples/s]
Tokenizing train dataset:  13%|█▎        | 228262/1807941 [03:16<20:18, 1296.66 examples/s]
Tokenizing train dataset:  13%|█▎        | 228394/1807941 [03:16<20:14, 1300.88 examples/s]
Tokenizing train dataset:  13%|█▎        | 228597/1807941 [03:16<19:59, 1316.20 examples/s]
Tokenizing train dataset:  13%|█▎        | 228798/1807941 [03:17<19:55, 1320.90 examples/s]
Tokenizing train dataset:  13%|█▎        | 228935/1807941 [03:17<19:45, 1331.38 examples/s]
Tokenizing train dataset:  13%|█▎        | 229123/1807941 [03:17<21:05, 1247.73 examples/s]
Tokenizing train dataset:  13%|█▎        | 229316/1807941 [03:17<20:55, 1257.35 examples/s]
Tokenizing train dataset:  13%|█▎        | 229450/1807941 [03:17<20:39, 1273.78 examples/s]
Tokenizing train dataset:  13%|█▎        | 229584/1807941 [03:17<20:28, 1285.12 examples/s]
Tokenizing train dataset:  13%|█▎        | 229721/1807941 [03:17<20:10, 1303.64 examples/s]
Tokenizing train dataset:  13%|█▎        | 229854/1807941 [03:17<20:06, 1307.90 examples/s]
Tokenizing train dataset:  13%|█▎        | 229987/1807941 [03:17<20:03, 1310.70 examples/s]
Tokenizing train dataset:  13%|█▎        | 230160/1807941 [03:18<21:03, 1248.60 examples/s]
Tokenizing train dataset:  13%|█▎        | 230292/1807941 [03:18<20:49, 1262.39 examples/s]
Tokenizing train dataset:  13%|█▎        | 230429/1807941 [03:18<20:23, 1288.98 examples/s]
Tokenizing train dataset:  13%|█▎        | 230567/1807941 [03:18<20:03, 1310.85 examples/s]
Tokenizing train dataset:  13%|█▎        | 230703/1807941 [03:18<19:54, 1320.31 examples/s]
Tokenizing train dataset:  13%|█▎        | 230900/1807941 [03:18<19:58, 1315.82 examples/s]
Tokenizing train dataset:  13%|█▎        | 231077/1807941 [03:18<20:47, 1263.86 examples/s]
Tokenizing train dataset:  13%|█▎        | 231268/1807941 [03:18<20:49, 1262.21 examples/s]
Tokenizing train dataset:  13%|█▎        | 231401/1807941 [03:19<20:36, 1275.40 examples/s]
Tokenizing train dataset:  13%|█▎        | 231592/1807941 [03:19<20:38, 1272.69 examples/s]
Tokenizing train dataset:  13%|█▎        | 231786/1807941 [03:19<20:33, 1277.71 examples/s]
Tokenizing train dataset:  13%|█▎        | 231920/1807941 [03:19<20:23, 1288.64 examples/s]
Tokenizing train dataset:  13%|█▎        | 232077/1807941 [03:19<21:45, 1207.14 examples/s]
Tokenizing train dataset:  13%|█▎        | 232207/1807941 [03:19<21:22, 1228.40 examples/s]
Tokenizing train dataset:  13%|█▎        | 232337/1807941 [03:19<21:06, 1243.87 examples/s]
Tokenizing train dataset:  13%|█▎        | 232470/1807941 [03:19<20:45, 1264.44 examples/s]
Tokenizing train dataset:  13%|█▎        | 232602/1807941 [03:20<20:33, 1276.84 examples/s]
Tokenizing train dataset:  13%|█▎        | 232732/1807941 [03:20<20:31, 1279.35 examples/s]
Tokenizing train dataset:  13%|█▎        | 232863/1807941 [03:20<20:23, 1287.10 examples/s]
Tokenizing train dataset:  13%|█▎        | 232997/1807941 [03:20<20:11, 1299.69 examples/s]
Tokenizing train dataset:  13%|█▎        | 233161/1807941 [03:20<21:34, 1216.42 examples/s]
Tokenizing train dataset:  13%|█▎        | 233291/1807941 [03:20<21:12, 1237.36 examples/s]
Tokenizing train dataset:  13%|█▎        | 233422/1807941 [03:20<20:55, 1253.65 examples/s]
Tokenizing train dataset:  13%|█▎        | 233556/1807941 [03:20<20:34, 1275.04 examples/s]
Tokenizing train dataset:  13%|█▎        | 233687/1807941 [03:20<20:28, 1281.70 examples/s]
Tokenizing train dataset:  13%|█▎        | 233819/1807941 [03:21<20:18, 1291.66 examples/s]
Tokenizing train dataset:  13%|█▎        | 233952/1807941 [03:21<20:11, 1299.18 examples/s]
Tokenizing train dataset:  13%|█▎        | 234131/1807941 [03:21<21:16, 1233.05 examples/s]
Tokenizing train dataset:  13%|█▎        | 234266/1807941 [03:21<20:48, 1260.46 examples/s]
Tokenizing train dataset:  13%|█▎        | 234398/1807941 [03:21<20:34, 1274.41 examples/s]
Tokenizing train dataset:  13%|█▎        | 234528/1807941 [03:21<20:33, 1275.21 examples/s]
Tokenizing train dataset:  13%|█▎        | 234659/1807941 [03:21<20:25, 1284.09 examples/s]
Tokenizing train dataset:  13%|█▎        | 234793/1807941 [03:21<20:13, 1296.83 examples/s]
Tokenizing train dataset:  13%|█▎        | 234929/1807941 [03:21<20:01, 1309.56 examples/s]
Tokenizing train dataset:  13%|█▎        | 235104/1807941 [03:22<20:55, 1252.79 examples/s]
Tokenizing train dataset:  13%|█▎        | 235236/1807941 [03:22<20:38, 1269.36 examples/s]
Tokenizing train dataset:  13%|█▎        | 235368/1807941 [03:22<20:29, 1278.97 examples/s]
Tokenizing train dataset:  13%|█▎        | 235500/1807941 [03:22<20:21, 1287.49 examples/s]
Tokenizing train dataset:  13%|█▎        | 235691/1807941 [03:22<20:30, 1277.29 examples/s]
Tokenizing train dataset:  13%|█▎        | 235823/1807941 [03:22<20:24, 1284.18 examples/s]
Tokenizing train dataset:  13%|█▎        | 236000/1807941 [03:22<21:25, 1223.04 examples/s]
Tokenizing train dataset:  13%|█▎        | 236133/1807941 [03:22<21:00, 1246.70 examples/s]
Tokenizing train dataset:  13%|█▎        | 236263/1807941 [03:22<20:48, 1259.11 examples/s]
Tokenizing train dataset:  13%|█▎        | 236400/1807941 [03:23<20:22, 1285.47 examples/s]
Tokenizing train dataset:  13%|█▎        | 236538/1807941 [03:23<19:59, 1310.06 examples/s]
Tokenizing train dataset:  13%|█▎        | 236670/1807941 [03:23<19:58, 1311.13 examples/s]
Tokenizing train dataset:  13%|█▎        | 236862/1807941 [03:23<20:11, 1296.54 examples/s]
Tokenizing train dataset:  13%|█▎        | 237000/1807941 [03:23<20:59, 1246.86 examples/s]
Tokenizing train dataset:  13%|█▎        | 237186/1807941 [03:23<21:05, 1240.79 examples/s]
Tokenizing train dataset:  13%|█▎        | 237318/1807941 [03:23<20:48, 1258.08 examples/s]
Tokenizing train dataset:  13%|█▎        | 237452/1807941 [03:23<20:30, 1275.96 examples/s]
Tokenizing train dataset:  13%|█▎        | 237646/1807941 [03:24<20:27, 1279.07 examples/s]
Tokenizing train dataset:  13%|█▎        | 237776/1807941 [03:24<20:23, 1283.80 examples/s]
Tokenizing train dataset:  13%|█▎        | 237963/1807941 [03:24<20:39, 1266.31 examples/s]
Tokenizing train dataset:  13%|█▎        | 238124/1807941 [03:24<21:57, 1191.48 examples/s]
Tokenizing train dataset:  13%|█▎        | 238250/1807941 [03:24<21:40, 1206.88 examples/s]
Tokenizing train dataset:  13%|█▎        | 238380/1807941 [03:24<21:17, 1228.99 examples/s]
Tokenizing train dataset:  13%|█▎        | 238505/1807941 [03:24<21:12, 1233.52 examples/s]
Tokenizing train dataset:  13%|█▎        | 238634/1807941 [03:24<21:00, 1245.05 examples/s]
Tokenizing train dataset:  13%|█▎        | 238766/1807941 [03:24<20:42, 1263.38 examples/s]
Tokenizing train dataset:  13%|█▎        | 238901/1807941 [03:25<20:21, 1284.99 examples/s]
Tokenizing train dataset:  13%|█▎        | 239054/1807941 [03:25<22:12, 1177.73 examples/s]
Tokenizing train dataset:  13%|█▎        | 239187/1807941 [03:25<21:30, 1215.27 examples/s]
Tokenizing train dataset:  13%|█▎        | 239322/1807941 [03:25<20:56, 1248.59 examples/s]
Tokenizing train dataset:  13%|█▎        | 239502/1807941 [03:25<21:17, 1227.70 examples/s]
Tokenizing train dataset:  13%|█▎        | 239687/1807941 [03:25<21:19, 1225.31 examples/s]
Tokenizing train dataset:  13%|█▎        | 239819/1807941 [03:25<20:56, 1247.85 examples/s]
Tokenizing train dataset:  13%|█▎        | 239954/1807941 [03:25<20:32, 1272.52 examples/s]
Tokenizing train dataset:  13%|█▎        | 240125/1807941 [03:26<21:53, 1193.62 examples/s]
Tokenizing train dataset:  13%|█▎        | 240265/1807941 [03:26<21:01, 1242.98 examples/s]
Tokenizing train dataset:  13%|█▎        | 240393/1807941 [03:26<20:52, 1251.37 examples/s]
Tokenizing train dataset:  13%|█▎        | 240527/1807941 [03:26<20:32, 1271.30 examples/s]
Tokenizing train dataset:  13%|█▎        | 240663/1807941 [03:26<20:13, 1291.76 examples/s]
Tokenizing train dataset:  13%|█▎        | 240862/1807941 [03:26<20:02, 1302.76 examples/s]
Tokenizing train dataset:  13%|█▎        | 240993/1807941 [03:26<20:03, 1302.42 examples/s]
Tokenizing train dataset:  13%|█▎        | 241164/1807941 [03:26<21:00, 1242.58 examples/s]
Tokenizing train dataset:  13%|█▎        | 241353/1807941 [03:27<20:57, 1245.46 examples/s]
Tokenizing train dataset:  13%|█▎        | 241536/1807941 [03:27<21:09, 1233.85 examples/s]
Tokenizing train dataset:  13%|█▎        | 241664/1807941 [03:27<21:01, 1241.89 examples/s]
Tokenizing train dataset:  13%|█▎        | 241845/1807941 [03:27<21:15, 1228.13 examples/s]
Tokenizing train dataset:  13%|█▎        | 241969/1807941 [03:27<21:14, 1228.70 examples/s]
Tokenizing train dataset:  13%|█▎        | 242139/1807941 [03:27<21:51, 1193.53 examples/s]
Tokenizing train dataset:  13%|█▎        | 242264/1807941 [03:27<21:42, 1202.05 examples/s]
Tokenizing train dataset:  13%|█▎        | 242391/1807941 [03:27<21:26, 1216.56 examples/s]
Tokenizing train dataset:  13%|█▎        | 242526/1807941 [03:27<20:52, 1249.59 examples/s]
Tokenizing train dataset:  13%|█▎        | 242660/1807941 [03:28<20:28, 1273.65 examples/s]
Tokenizing train dataset:  13%|█▎        | 242854/1807941 [03:28<20:25, 1277.54 examples/s]
Tokenizing train dataset:  13%|█▎        | 242988/1807941 [03:28<20:11, 1291.83 examples/s]
Tokenizing train dataset:  13%|█▎        | 243156/1807941 [03:28<21:15, 1226.55 examples/s]
Tokenizing train dataset:  13%|█▎        | 243294/1807941 [03:28<20:40, 1261.79 examples/s]
Tokenizing train dataset:  13%|█▎        | 243427/1807941 [03:28<20:26, 1275.80 examples/s]
Tokenizing train dataset:  13%|█▎        | 243626/1807941 [03:28<20:10, 1292.61 examples/s]
Tokenizing train dataset:  13%|█▎        | 243771/1807941 [03:28<19:33, 1332.43 examples/s]
Tokenizing train dataset:  13%|█▎        | 243907/1807941 [03:29<19:28, 1338.36 examples/s]
Tokenizing train dataset:  14%|█▎        | 244086/1807941 [03:29<20:18, 1283.23 examples/s]
Tokenizing train dataset:  14%|█▎        | 244225/1807941 [03:29<19:55, 1308.27 examples/s]
Tokenizing train dataset:  14%|█▎        | 244374/1807941 [03:29<19:15, 1352.99 examples/s]
Tokenizing train dataset:  14%|█▎        | 244515/1807941 [03:29<19:03, 1366.94 examples/s]
Tokenizing train dataset:  14%|█▎        | 244721/1807941 [03:29<19:06, 1363.18 examples/s]
Tokenizing train dataset:  14%|█▎        | 244860/1807941 [03:29<19:03, 1367.09 examples/s]
Tokenizing train dataset:  14%|█▎        | 245034/1807941 [03:29<20:12, 1289.25 examples/s]
Tokenizing train dataset:  14%|█▎        | 245227/1807941 [03:30<20:15, 1285.26 examples/s]
Tokenizing train dataset:  14%|█▎        | 245366/1807941 [03:30<19:55, 1307.29 examples/s]
Tokenizing train dataset:  14%|█▎        | 245505/1807941 [03:30<19:38, 1325.81 examples/s]
Tokenizing train dataset:  14%|█▎        | 245652/1807941 [03:30<19:06, 1362.61 examples/s]
Tokenizing train dataset:  14%|█▎        | 245792/1807941 [03:30<19:00, 1369.86 examples/s]
Tokenizing train dataset:  14%|█▎        | 245997/1807941 [03:30<19:02, 1366.66 examples/s]
Tokenizing train dataset:  14%|█▎        | 246150/1807941 [03:30<20:55, 1243.76 examples/s]
Tokenizing train dataset:  14%|█▎        | 246340/1807941 [03:30<20:53, 1246.26 examples/s]
Tokenizing train dataset:  14%|█▎        | 246475/1807941 [03:30<20:30, 1268.56 examples/s]
Tokenizing train dataset:  14%|█▎        | 246608/1807941 [03:31<20:16, 1283.98 examples/s]
Tokenizing train dataset:  14%|█▎        | 246789/1807941 [03:31<20:44, 1254.20 examples/s]
Tokenizing train dataset:  14%|█▎        | 246918/1807941 [03:31<20:36, 1262.11 examples/s]
Tokenizing train dataset:  14%|█▎        | 247068/1807941 [03:31<22:17, 1167.26 examples/s]
Tokenizing train dataset:  14%|█▎        | 247191/1807941 [03:31<22:03, 1178.94 examples/s]
Tokenizing train dataset:  14%|█▎        | 247318/1807941 [03:31<21:39, 1201.01 examples/s]
Tokenizing train dataset:  14%|█▎        | 247441/1807941 [03:31<21:34, 1205.25 examples/s]
Tokenizing train dataset:  14%|█▎        | 247571/1807941 [03:31<21:11, 1227.18 examples/s]
Tokenizing train dataset:  14%|█▎        | 247701/1807941 [03:31<20:54, 1244.17 examples/s]
Tokenizing train dataset:  14%|█▎        | 247829/1807941 [03:32<20:44, 1253.62 examples/s]
Tokenizing train dataset:  14%|█▎        | 247955/1807941 [03:32<20:48, 1249.14 examples/s]
Tokenizing train dataset:  14%|█▎        | 248137/1807941 [03:32<21:33, 1205.98 examples/s]
Tokenizing train dataset:  14%|█▎        | 248274/1807941 [03:32<20:52, 1245.37 examples/s]
Tokenizing train dataset:  14%|█▎        | 248412/1807941 [03:32<20:20, 1277.71 examples/s]
Tokenizing train dataset:  14%|█▎        | 248548/1807941 [03:32<20:01, 1297.75 examples/s]
Tokenizing train dataset:  14%|█▍        | 248731/1807941 [03:32<20:32, 1265.12 examples/s]
Tokenizing train dataset:  14%|█▍        | 248863/1807941 [03:32<20:19, 1278.36 examples/s]
Tokenizing train dataset:  14%|█▍        | 249023/1807941 [03:33<21:39, 1199.41 examples/s]
Tokenizing train dataset:  14%|█▍        | 249157/1807941 [03:33<21:04, 1232.98 examples/s]
Tokenizing train dataset:  14%|█▍        | 249287/1807941 [03:33<20:47, 1249.41 examples/s]
Tokenizing train dataset:  14%|█▍        | 249425/1807941 [03:33<20:15, 1282.61 examples/s]
Tokenizing train dataset:  14%|█▍        | 249559/1807941 [03:33<20:02, 1296.36 examples/s]
Tokenizing train dataset:  14%|█▍        | 249691/1807941 [03:33<19:57, 1301.11 examples/s]
Tokenizing train dataset:  14%|█▍        | 249824/1807941 [03:33<19:52, 1306.79 examples/s]
Tokenizing train dataset:  14%|█▍        | 250000/1807941 [03:33<21:02, 1233.96 examples/s]
Tokenizing train dataset:  14%|█▍        | 250174/1807941 [03:33<21:33, 1204.25 examples/s]
Tokenizing train dataset:  14%|█▍        | 250344/1807941 [03:34<22:01, 1178.40 examples/s]
Tokenizing train dataset:  14%|█▍        | 250464/1807941 [03:34<21:57, 1181.90 examples/s]
Tokenizing train dataset:  14%|█▍        | 250594/1807941 [03:34<21:27, 1210.02 examples/s]
Tokenizing train dataset:  14%|█▍        | 250772/1807941 [03:34<21:40, 1197.02 examples/s]
Tokenizing train dataset:  14%|█▍        | 250947/1807941 [03:34<21:54, 1184.18 examples/s]
Tokenizing train dataset:  14%|█▍        | 251121/1807941 [03:34<22:57, 1129.92 examples/s]
Tokenizing train dataset:  14%|█▍        | 251242/1807941 [03:34<22:37, 1147.07 examples/s]
Tokenizing train dataset:  14%|█▍        | 251367/1807941 [03:34<22:10, 1170.05 examples/s]
Tokenizing train dataset:  14%|█▍        | 251488/1807941 [03:35<21:59, 1179.99 examples/s]
Tokenizing train dataset:  14%|█▍        | 251617/1807941 [03:35<21:29, 1207.03 examples/s]
Tokenizing train dataset:  14%|█▍        | 251753/1807941 [03:35<20:46, 1248.47 examples/s]
Tokenizing train dataset:  14%|█▍        | 251883/1807941 [03:35<20:37, 1257.84 examples/s]
Tokenizing train dataset:  14%|█▍        | 252059/1807941 [03:35<21:46, 1190.57 examples/s]
Tokenizing train dataset:  14%|█▍        | 252189/1807941 [03:35<21:19, 1215.60 examples/s]
Tokenizing train dataset:  14%|█▍        | 252315/1807941 [03:35<21:08, 1225.90 examples/s]
Tokenizing train dataset:  14%|█▍        | 252442/1807941 [03:35<20:57, 1236.70 examples/s]
Tokenizing train dataset:  14%|█▍        | 252576/1807941 [03:35<20:29, 1264.53 examples/s]
Tokenizing train dataset:  14%|█▍        | 252764/1807941 [03:36<20:38, 1256.15 examples/s]
Tokenizing train dataset:  14%|█▍        | 252941/1807941 [03:36<21:07, 1226.77 examples/s]
Tokenizing train dataset:  14%|█▍        | 253122/1807941 [03:36<22:09, 1169.09 examples/s]
Tokenizing train dataset:  14%|█▍        | 253252/1807941 [03:36<21:37, 1198.63 examples/s]
Tokenizing train dataset:  14%|█▍        | 253424/1807941 [03:36<21:57, 1180.04 examples/s]
Tokenizing train dataset:  14%|█▍        | 253549/1807941 [03:36<21:41, 1193.99 examples/s]
Tokenizing train dataset:  14%|█▍        | 253679/1807941 [03:36<21:16, 1217.82 examples/s]
Tokenizing train dataset:  14%|█▍        | 253807/1807941 [03:36<21:00, 1233.24 examples/s]
Tokenizing train dataset:  14%|█▍        | 253939/1807941 [03:37<20:39, 1254.23 examples/s]
Tokenizing train dataset:  14%|█▍        | 254102/1807941 [03:37<21:46, 1188.90 examples/s]
Tokenizing train dataset:  14%|█▍        | 254223/1807941 [03:37<21:45, 1190.26 examples/s]
Tokenizing train dataset:  14%|█▍        | 254397/1807941 [03:37<21:59, 1177.55 examples/s]
Tokenizing train dataset:  14%|█▍        | 254521/1807941 [03:37<21:44, 1190.64 examples/s]
Tokenizing train dataset:  14%|█▍        | 254650/1807941 [03:37<21:18, 1214.65 examples/s]
Tokenizing train dataset:  14%|█▍        | 254775/1807941 [03:37<21:10, 1222.12 examples/s]
Tokenizing train dataset:  14%|█▍        | 254903/1807941 [03:37<20:54, 1237.76 examples/s]
Tokenizing train dataset:  14%|█▍        | 255064/1807941 [03:38<22:06, 1170.55 examples/s]
Tokenizing train dataset:  14%|█▍        | 255187/1807941 [03:38<21:52, 1182.64 examples/s]
Tokenizing train dataset:  14%|█▍        | 255312/1807941 [03:38<21:36, 1197.74 examples/s]
Tokenizing train dataset:  14%|█▍        | 255488/1807941 [03:38<21:50, 1184.36 examples/s]
Tokenizing train dataset:  14%|█▍        | 255617/1807941 [03:38<21:24, 1208.80 examples/s]
Tokenizing train dataset:  14%|█▍        | 255741/1807941 [03:38<21:18, 1214.10 examples/s]
Tokenizing train dataset:  14%|█▍        | 255866/1807941 [03:38<21:10, 1221.58 examples/s]
Tokenizing train dataset:  14%|█▍        | 255990/1807941 [03:38<21:08, 1223.03 examples/s]
Tokenizing train dataset:  14%|█▍        | 256136/1807941 [03:38<23:01, 1123.63 examples/s]
Tokenizing train dataset:  14%|█▍        | 256258/1807941 [03:39<22:30, 1148.89 examples/s]
Tokenizing train dataset:  14%|█▍        | 256377/1807941 [03:39<22:19, 1158.10 examples/s]
Tokenizing train dataset:  14%|█▍        | 256557/1807941 [03:39<22:05, 1170.07 examples/s]
Tokenizing train dataset:  14%|█▍        | 256727/1807941 [03:39<22:23, 1154.87 examples/s]
Tokenizing train dataset:  14%|█▍        | 256848/1807941 [03:39<22:09, 1166.59 examples/s]
Tokenizing train dataset:  14%|█▍        | 257000/1807941 [03:39<23:19, 1108.19 examples/s]
Tokenizing train dataset:  14%|█▍        | 257121/1807941 [03:39<22:51, 1130.95 examples/s]
Tokenizing train dataset:  14%|█▍        | 257237/1807941 [03:39<22:44, 1136.13 examples/s]
Tokenizing train dataset:  14%|█▍        | 257357/1807941 [03:40<22:28, 1149.79 examples/s]
Tokenizing train dataset:  14%|█▍        | 257525/1807941 [03:40<22:46, 1134.53 examples/s]
Tokenizing train dataset:  14%|█▍        | 257699/1807941 [03:40<22:41, 1138.63 examples/s]
Tokenizing train dataset:  14%|█▍        | 257866/1807941 [03:40<22:53, 1128.46 examples/s]
Tokenizing train dataset:  14%|█▍        | 257991/1807941 [03:40<22:20, 1156.47 examples/s]
Tokenizing train dataset:  14%|█▍        | 258143/1807941 [03:40<23:20, 1106.79 examples/s]
Tokenizing train dataset:  14%|█▍        | 258264/1807941 [03:40<22:51, 1130.11 examples/s]
Tokenizing train dataset:  14%|█▍        | 258394/1807941 [03:40<22:03, 1170.65 examples/s]
Tokenizing train dataset:  14%|█▍        | 258573/1807941 [03:41<21:56, 1176.81 examples/s]
Tokenizing train dataset:  14%|█▍        | 258701/1807941 [03:41<21:30, 1200.67 examples/s]
Tokenizing train dataset:  14%|█▍        | 258887/1807941 [03:41<21:20, 1209.68 examples/s]
Tokenizing train dataset:  14%|█▍        | 259054/1807941 [03:41<22:59, 1122.58 examples/s]
Tokenizing train dataset:  14%|█▍        | 259225/1807941 [03:41<22:55, 1125.91 examples/s]
Tokenizing train dataset:  14%|█▍        | 259354/1807941 [03:41<22:13, 1160.96 examples/s]
Tokenizing train dataset:  14%|█▍        | 259528/1807941 [03:41<22:17, 1157.38 examples/s]
Tokenizing train dataset:  14%|█▍        | 259650/1807941 [03:41<22:04, 1168.77 examples/s]
Tokenizing train dataset:  14%|█▍        | 259772/1807941 [03:42<21:56, 1176.30 examples/s]
Tokenizing train dataset:  14%|█▍        | 259945/1807941 [03:42<22:08, 1165.66 examples/s]
Tokenizing train dataset:  14%|█▍        | 260118/1807941 [03:42<23:04, 1117.60 examples/s]
Tokenizing train dataset:  14%|█▍        | 260236/1807941 [03:42<22:47, 1131.89 examples/s]
Tokenizing train dataset:  14%|█▍        | 260395/1807941 [03:42<23:23, 1102.68 examples/s]
Tokenizing train dataset:  14%|█▍        | 260568/1807941 [03:42<23:03, 1118.13 examples/s]
Tokenizing train dataset:  14%|█▍        | 260687/1807941 [03:42<22:45, 1133.35 examples/s]
Tokenizing train dataset:  14%|█▍        | 260806/1807941 [03:43<22:28, 1146.94 examples/s]
Tokenizing train dataset:  14%|█▍        | 260926/1807941 [03:43<22:15, 1158.61 examples/s]
Tokenizing train dataset:  14%|█▍        | 261074/1807941 [03:43<23:33, 1094.54 examples/s]
Tokenizing train dataset:  14%|█▍        | 261190/1807941 [03:43<23:13, 1110.07 examples/s]
Tokenizing train dataset:  14%|█▍        | 261328/1807941 [03:43<21:50, 1180.33 examples/s]
Tokenizing train dataset:  14%|█▍        | 261451/1807941 [03:43<21:37, 1191.96 examples/s]
Tokenizing train dataset:  14%|█▍        | 261583/1807941 [03:43<21:01, 1226.16 examples/s]
Tokenizing train dataset:  14%|█▍        | 261748/1807941 [03:43<21:55, 1175.12 examples/s]
Tokenizing train dataset:  14%|█▍        | 261926/1807941 [03:43<21:55, 1175.22 examples/s]
Tokenizing train dataset:  14%|█▍        | 262076/1807941 [03:44<23:08, 1113.42 examples/s]
Tokenizing train dataset:  15%|█▍        | 262227/1807941 [03:44<23:54, 1077.19 examples/s]
Tokenizing train dataset:  15%|█▍        | 262383/1807941 [03:44<24:14, 1062.85 examples/s]
Tokenizing train dataset:  15%|█▍        | 262505/1807941 [03:44<23:28, 1097.50 examples/s]
Tokenizing train dataset:  15%|█▍        | 262621/1807941 [03:44<23:11, 1110.18 examples/s]
Tokenizing train dataset:  15%|█▍        | 262739/1807941 [03:44<22:54, 1124.07 examples/s]
Tokenizing train dataset:  15%|█▍        | 262859/1807941 [03:44<22:35, 1139.59 examples/s]
Tokenizing train dataset:  15%|█▍        | 262974/1807941 [03:44<22:33, 1141.33 examples/s]
Tokenizing train dataset:  15%|█▍        | 263113/1807941 [03:45<24:17, 1059.93 examples/s]
Tokenizing train dataset:  15%|█▍        | 263226/1807941 [03:45<23:54, 1076.64 examples/s]
Tokenizing train dataset:  15%|█▍        | 263341/1807941 [03:45<23:32, 1093.45 examples/s]
Tokenizing train dataset:  15%|█▍        | 263459/1807941 [03:45<23:04, 1115.57 examples/s]
Tokenizing train dataset:  15%|█▍        | 263577/1807941 [03:45<22:46, 1129.80 examples/s]
Tokenizing train dataset:  15%|█▍        | 263695/1807941 [03:45<22:33, 1141.09 examples/s]
Tokenizing train dataset:  15%|█▍        | 263810/1807941 [03:45<22:32, 1141.75 examples/s]
Tokenizing train dataset:  15%|█▍        | 263927/1807941 [03:45<22:25, 1147.49 examples/s]
Tokenizing train dataset:  15%|█▍        | 264068/1807941 [03:45<24:10, 1064.73 examples/s]
Tokenizing train dataset:  15%|█▍        | 264182/1807941 [03:46<23:44, 1083.60 examples/s]
Tokenizing train dataset:  15%|█▍        | 264297/1807941 [03:46<23:24, 1099.29 examples/s]
Tokenizing train dataset:  15%|█▍        | 264412/1807941 [03:46<23:08, 1111.90 examples/s]
Tokenizing train dataset:  15%|█▍        | 264525/1807941 [03:46<23:03, 1115.45 examples/s]
Tokenizing train dataset:  15%|█▍        | 264638/1807941 [03:46<23:00, 1117.70 examples/s]
Tokenizing train dataset:  15%|█▍        | 264800/1807941 [03:46<23:23, 1099.66 examples/s]
Tokenizing train dataset:  15%|█▍        | 264921/1807941 [03:46<22:49, 1126.39 examples/s]
Tokenizing train dataset:  15%|█▍        | 265069/1807941 [03:46<23:58, 1072.39 examples/s]
Tokenizing train dataset:  15%|█▍        | 265199/1807941 [03:46<22:46, 1128.67 examples/s]
Tokenizing train dataset:  15%|█▍        | 265325/1807941 [03:47<22:08, 1161.21 examples/s]
Tokenizing train dataset:  15%|█▍        | 265455/1807941 [03:47<21:29, 1196.09 examples/s]
Tokenizing train dataset:  15%|█▍        | 265584/1807941 [03:47<21:04, 1219.91 examples/s]
Tokenizing train dataset:  15%|█▍        | 265713/1807941 [03:47<20:46, 1237.13 examples/s]
Tokenizing train dataset:  15%|█▍        | 265903/1807941 [03:47<20:38, 1245.25 examples/s]
Tokenizing train dataset:  15%|█▍        | 266058/1807941 [03:47<22:01, 1166.84 examples/s]
Tokenizing train dataset:  15%|█▍        | 266188/1807941 [03:47<21:28, 1196.65 examples/s]
Tokenizing train dataset:  15%|█▍        | 266320/1807941 [03:47<20:55, 1228.12 examples/s]
Tokenizing train dataset:  15%|█▍        | 266450/1807941 [03:47<20:38, 1245.00 examples/s]
Tokenizing train dataset:  15%|█▍        | 266578/1807941 [03:48<20:31, 1251.78 examples/s]
Tokenizing train dataset:  15%|█▍        | 266755/1807941 [03:48<21:03, 1219.47 examples/s]
Tokenizing train dataset:  15%|█▍        | 266924/1807941 [03:48<21:44, 1181.53 examples/s]
Tokenizing train dataset:  15%|█▍        | 267077/1807941 [03:48<22:50, 1124.18 examples/s]
Tokenizing train dataset:  15%|█▍        | 267202/1807941 [03:48<22:18, 1151.06 examples/s]
Tokenizing train dataset:  15%|█▍        | 267320/1807941 [03:48<22:12, 1156.17 examples/s]
Tokenizing train dataset:  15%|█▍        | 267446/1807941 [03:48<21:42, 1182.72 examples/s]
Tokenizing train dataset:  15%|█▍        | 267567/1807941 [03:48<21:35, 1188.67 examples/s]
Tokenizing train dataset:  15%|█▍        | 267696/1807941 [03:49<21:08, 1214.16 examples/s]
Tokenizing train dataset:  15%|█▍        | 267819/1807941 [03:49<21:05, 1217.13 examples/s]
Tokenizing train dataset:  15%|█▍        | 267942/1807941 [03:49<21:03, 1218.51 examples/s]
Tokenizing train dataset:  15%|█▍        | 268123/1807941 [03:49<22:27, 1142.61 examples/s]
Tokenizing train dataset:  15%|█▍        | 268253/1807941 [03:49<21:43, 1181.06 examples/s]
Tokenizing train dataset:  15%|█▍        | 268384/1807941 [03:49<21:09, 1212.84 examples/s]
Tokenizing train dataset:  15%|█▍        | 268512/1807941 [03:49<20:52, 1229.48 examples/s]
Tokenizing train dataset:  15%|█▍        | 268644/1807941 [03:49<20:28, 1252.53 examples/s]
Tokenizing train dataset:  15%|█▍        | 268779/1807941 [03:49<20:05, 1277.28 examples/s]
Tokenizing train dataset:  15%|█▍        | 268913/1807941 [03:49<19:51, 1291.52 examples/s]
Tokenizing train dataset:  15%|█▍        | 269080/1807941 [03:50<21:01, 1219.96 examples/s]
Tokenizing train dataset:  15%|█▍        | 269212/1807941 [03:50<20:36, 1244.11 examples/s]
Tokenizing train dataset:  15%|█▍        | 269339/1807941 [03:50<20:32, 1248.10 examples/s]
Tokenizing train dataset:  15%|█▍        | 269471/1807941 [03:50<20:15, 1265.84 examples/s]
Tokenizing train dataset:  15%|█▍        | 269617/1807941 [03:50<19:25, 1319.34 examples/s]
Tokenizing train dataset:  15%|█▍        | 269763/1807941 [03:50<18:51, 1358.89 examples/s]
Tokenizing train dataset:  15%|█▍        | 269964/1807941 [03:50<18:59, 1349.60 examples/s]
Tokenizing train dataset:  15%|█▍        | 270141/1807941 [03:50<19:56, 1285.21 examples/s]
Tokenizing train dataset:  15%|█▍        | 270286/1807941 [03:51<19:21, 1323.30 examples/s]
Tokenizing train dataset:  15%|█▍        | 270423/1807941 [03:51<19:13, 1333.42 examples/s]
Tokenizing train dataset:  15%|█▍        | 270559/1807941 [03:51<19:10, 1336.51 examples/s]
Tokenizing train dataset:  15%|█▍        | 270753/1807941 [03:51<19:28, 1315.37 examples/s]
Tokenizing train dataset:  15%|█▍        | 270935/1807941 [03:51<20:04, 1275.56 examples/s]
Tokenizing train dataset:  15%|█▍        | 271085/1807941 [03:51<21:38, 1183.97 examples/s]
Tokenizing train dataset:  15%|█▌        | 271218/1807941 [03:51<21:05, 1214.62 examples/s]
Tokenizing train dataset:  15%|█▌        | 271349/1807941 [03:51<20:42, 1236.84 examples/s]
Tokenizing train dataset:  15%|█▌        | 271482/1807941 [03:52<20:20, 1258.51 examples/s]
Tokenizing train dataset:  15%|█▌        | 271615/1807941 [03:52<20:05, 1274.52 examples/s]
Tokenizing train dataset:  15%|█▌        | 271808/1807941 [03:52<20:02, 1276.95 examples/s]
Tokenizing train dataset:  15%|█▌        | 271994/1807941 [03:52<20:18, 1260.50 examples/s]
Tokenizing train dataset:  15%|█▌        | 272140/1807941 [03:52<22:03, 1160.40 examples/s]
Tokenizing train dataset:  15%|█▌        | 272261/1807941 [03:52<21:52, 1170.15 examples/s]
Tokenizing train dataset:  15%|█▌        | 272385/1807941 [03:52<21:34, 1186.41 examples/s]
Tokenizing train dataset:  15%|█▌        | 272521/1807941 [03:52<20:49, 1229.15 examples/s]
Tokenizing train dataset:  15%|█▌        | 272655/1807941 [03:52<20:22, 1255.48 examples/s]
Tokenizing train dataset:  15%|█▌        | 272785/1807941 [03:53<20:13, 1264.61 examples/s]
Tokenizing train dataset:  15%|█▌        | 272918/1807941 [03:53<20:02, 1276.75 examples/s]
Tokenizing train dataset:  15%|█▌        | 273091/1807941 [03:53<20:52, 1225.29 examples/s]
Tokenizing train dataset:  15%|█▌        | 273233/1807941 [03:53<20:04, 1273.75 examples/s]
Tokenizing train dataset:  15%|█▌        | 273379/1807941 [03:53<19:20, 1322.29 examples/s]
Tokenizing train dataset:  15%|█▌        | 273520/1807941 [03:53<19:01, 1343.94 examples/s]
Tokenizing train dataset:  15%|█▌        | 273657/1807941 [03:53<19:00, 1345.45 examples/s]
Tokenizing train dataset:  15%|█▌        | 273796/1807941 [03:53<18:51, 1355.42 examples/s]
Tokenizing train dataset:  15%|█▌        | 273938/1807941 [03:53<18:39, 1370.58 examples/s]
Tokenizing train dataset:  15%|█▌        | 274103/1807941 [03:54<20:15, 1261.79 examples/s]
Tokenizing train dataset:  15%|█▌        | 274237/1807941 [03:54<19:56, 1281.56 examples/s]
Tokenizing train dataset:  15%|█▌        | 274376/1807941 [03:54<19:32, 1307.59 examples/s]
Tokenizing train dataset:  15%|█▌        | 274576/1807941 [03:54<19:28, 1312.10 examples/s]
Tokenizing train dataset:  15%|█▌        | 274769/1807941 [03:54<19:38, 1301.29 examples/s]
Tokenizing train dataset:  15%|█▌        | 274952/1807941 [03:54<20:06, 1270.50 examples/s]
Tokenizing train dataset:  15%|█▌        | 275119/1807941 [03:54<21:34, 1184.18 examples/s]
Tokenizing train dataset:  15%|█▌        | 275241/1807941 [03:55<21:25, 1192.07 examples/s]
Tokenizing train dataset:  15%|█▌        | 275363/1807941 [03:55<21:20, 1196.96 examples/s]
Tokenizing train dataset:  15%|█▌        | 275485/1807941 [03:55<21:16, 1200.10 examples/s]
Tokenizing train dataset:  15%|█▌        | 275608/1807941 [03:55<21:12, 1204.54 examples/s]
Tokenizing train dataset:  15%|█▌        | 275731/1807941 [03:55<21:08, 1207.95 examples/s]
Tokenizing train dataset:  15%|█▌        | 275855/1807941 [03:55<21:03, 1212.25 examples/s]
Tokenizing train dataset:  15%|█▌        | 276003/1807941 [03:55<22:45, 1122.23 examples/s]
Tokenizing train dataset:  15%|█▌        | 276120/1807941 [03:55<22:32, 1132.32 examples/s]
Tokenizing train dataset:  15%|█▌        | 276242/1807941 [03:55<22:05, 1155.65 examples/s]
Tokenizing train dataset:  15%|█▌        | 276365/1807941 [03:55<21:43, 1174.66 examples/s]
Tokenizing train dataset:  15%|█▌        | 276531/1807941 [03:56<22:13, 1148.18 examples/s]
Tokenizing train dataset:  15%|█▌        | 276700/1807941 [03:56<22:25, 1137.72 examples/s]
Tokenizing train dataset:  15%|█▌        | 276869/1807941 [03:56<22:37, 1127.92 examples/s]
Tokenizing train dataset:  15%|█▌        | 277008/1807941 [03:56<24:02, 1061.38 examples/s]
Tokenizing train dataset:  15%|█▌        | 277117/1807941 [03:56<23:54, 1067.35 examples/s]
Tokenizing train dataset:  15%|█▌        | 277232/1807941 [03:56<23:30, 1085.22 examples/s]
Tokenizing train dataset:  15%|█▌        | 277352/1807941 [03:56<22:57, 1110.88 examples/s]
Tokenizing train dataset:  15%|█▌        | 277466/1807941 [03:56<22:52, 1115.08 examples/s]
Tokenizing train dataset:  15%|█▌        | 277587/1807941 [03:57<22:24, 1138.01 examples/s]
Tokenizing train dataset:  15%|█▌        | 277705/1807941 [03:57<22:15, 1145.80 examples/s]
Tokenizing train dataset:  15%|█▌        | 277827/1807941 [03:57<21:53, 1164.70 examples/s]
Tokenizing train dataset:  15%|█▌        | 277944/1807941 [03:57<21:55, 1163.23 examples/s]
Tokenizing train dataset:  15%|█▌        | 278111/1807941 [03:57<23:49, 1070.13 examples/s]
Tokenizing train dataset:  15%|█▌        | 278244/1807941 [03:57<22:29, 1133.44 examples/s]
Tokenizing train dataset:  15%|█▌        | 278409/1807941 [03:57<22:51, 1115.51 examples/s]
Tokenizing train dataset:  15%|█▌        | 278524/1807941 [03:57<22:44, 1120.92 examples/s]
Tokenizing train dataset:  15%|█▌        | 278646/1807941 [03:58<22:19, 1141.89 examples/s]
Tokenizing train dataset:  15%|█▌        | 278764/1807941 [03:58<22:12, 1147.92 examples/s]
Tokenizing train dataset:  15%|█▌        | 278881/1807941 [03:58<22:07, 1151.51 examples/s]
Tokenizing train dataset:  15%|█▌        | 279015/1807941 [03:58<24:13, 1052.20 examples/s]
Tokenizing train dataset:  15%|█▌        | 279128/1807941 [03:58<23:47, 1070.76 examples/s]
Tokenizing train dataset:  15%|█▌        | 279244/1807941 [03:58<23:17, 1093.50 examples/s]
Tokenizing train dataset:  15%|█▌        | 279363/1807941 [03:58<26:03, 977.52 examples/s] 
Tokenizing train dataset:  15%|█▌        | 279469/1807941 [03:58<25:35, 995.63 examples/s]
Tokenizing train dataset:  15%|█▌        | 279584/1807941 [03:58<24:37, 1034.70 examples/s]
Tokenizing train dataset:  15%|█▌        | 279697/1807941 [03:59<24:03, 1058.75 examples/s]
Tokenizing train dataset:  15%|█▌        | 279807/1807941 [03:59<23:53, 1066.36 examples/s]
Tokenizing train dataset:  15%|█▌        | 279918/1807941 [03:59<23:39, 1076.71 examples/s]
Tokenizing train dataset:  15%|█▌        | 280058/1807941 [03:59<24:57, 1020.26 examples/s]
Tokenizing train dataset:  15%|█▌        | 280174/1807941 [03:59<24:07, 1055.43 examples/s]
Tokenizing train dataset:  16%|█▌        | 280283/1807941 [03:59<23:58, 1061.67 examples/s]
Tokenizing train dataset:  16%|█▌        | 280395/1807941 [03:59<23:40, 1075.03 examples/s]
Tokenizing train dataset:  16%|█▌        | 280509/1807941 [03:59<23:19, 1091.27 examples/s]
Tokenizing train dataset:  16%|█▌        | 280675/1807941 [03:59<23:21, 1089.63 examples/s]
Tokenizing train dataset:  16%|█▌        | 280831/1807941 [04:00<23:50, 1067.49 examples/s]
Tokenizing train dataset:  16%|█▌        | 280940/1807941 [04:00<23:43, 1072.34 examples/s]
Tokenizing train dataset:  16%|█▌        | 281083/1807941 [04:00<24:48, 1026.11 examples/s]
Tokenizing train dataset:  16%|█▌        | 281190/1807941 [04:00<24:33, 1035.87 examples/s]
Tokenizing train dataset:  16%|█▌        | 281299/1807941 [04:00<24:15, 1048.87 examples/s]
Tokenizing train dataset:  16%|█▌        | 281409/1807941 [04:00<24:00, 1060.00 examples/s]
Tokenizing train dataset:  16%|█▌        | 281521/1807941 [04:00<23:42, 1073.26 examples/s]
Tokenizing train dataset:  16%|█▌        | 281635/1807941 [04:00<23:21, 1088.82 examples/s]
Tokenizing train dataset:  16%|█▌        | 281752/1807941 [04:00<22:55, 1109.63 examples/s]
Tokenizing train dataset:  16%|█▌        | 281871/1807941 [04:01<22:31, 1128.92 examples/s]
Tokenizing train dataset:  16%|█▌        | 282000/1807941 [04:01<25:22, 1002.52 examples/s]
Tokenizing train dataset:  16%|█▌        | 282139/1807941 [04:01<26:08, 972.98 examples/s] 
Tokenizing train dataset:  16%|█▌        | 282255/1807941 [04:01<24:59, 1017.37 examples/s]
Tokenizing train dataset:  16%|█▌        | 282365/1807941 [04:01<24:32, 1035.94 examples/s]
Tokenizing train dataset:  16%|█▌        | 282473/1807941 [04:01<24:18, 1046.04 examples/s]
Tokenizing train dataset:  16%|█▌        | 282589/1807941 [04:01<23:40, 1073.72 examples/s]
Tokenizing train dataset:  16%|█▌        | 282699/1807941 [04:01<23:34, 1078.53 examples/s]
Tokenizing train dataset:  16%|█▌        | 282815/1807941 [04:01<23:08, 1098.53 examples/s]
Tokenizing train dataset:  16%|█▌        | 282930/1807941 [04:02<22:51, 1111.58 examples/s]
Tokenizing train dataset:  16%|█▌        | 283071/1807941 [04:02<24:20, 1044.09 examples/s]
Tokenizing train dataset:  16%|█▌        | 283184/1807941 [04:02<23:54, 1063.08 examples/s]
Tokenizing train dataset:  16%|█▌        | 283330/1807941 [04:02<24:44, 1026.90 examples/s]
Tokenizing train dataset:  16%|█▌        | 283443/1807941 [04:02<24:12, 1049.59 examples/s]
Tokenizing train dataset:  16%|█▌        | 283559/1807941 [04:02<23:33, 1078.56 examples/s]
Tokenizing train dataset:  16%|█▌        | 283675/1807941 [04:02<23:09, 1096.60 examples/s]
Tokenizing train dataset:  16%|█▌        | 283789/1807941 [04:02<22:58, 1106.05 examples/s]
Tokenizing train dataset:  16%|█▌        | 283901/1807941 [04:02<22:59, 1104.98 examples/s]
Tokenizing train dataset:  16%|█▌        | 284055/1807941 [04:03<24:38, 1031.03 examples/s]
Tokenizing train dataset:  16%|█▌        | 284163/1807941 [04:03<24:27, 1038.49 examples/s]
Tokenizing train dataset:  16%|█▌        | 284273/1807941 [04:03<24:05, 1054.12 examples/s]
Tokenizing train dataset:  16%|█▌        | 284387/1807941 [04:03<23:36, 1075.30 examples/s]
Tokenizing train dataset:  16%|█▌        | 284511/1807941 [04:03<22:42, 1117.80 examples/s]
Tokenizing train dataset:  16%|█▌        | 284625/1807941 [04:03<22:35, 1124.11 examples/s]
Tokenizing train dataset:  16%|█▌        | 284742/1807941 [04:03<22:21, 1135.28 examples/s]
Tokenizing train dataset:  16%|█▌        | 284864/1807941 [04:03<21:53, 1159.33 examples/s]
Tokenizing train dataset:  16%|█▌        | 284995/1807941 [04:03<24:15, 1046.41 examples/s]
Tokenizing train dataset:  16%|█▌        | 285129/1807941 [04:04<25:35, 991.64 examples/s] 
Tokenizing train dataset:  16%|█▌        | 285243/1807941 [04:04<24:40, 1028.83 examples/s]
Tokenizing train dataset:  16%|█▌        | 285358/1807941 [04:04<23:57, 1059.08 examples/s]
Tokenizing train dataset:  16%|█▌        | 285475/1807941 [04:04<23:20, 1087.06 examples/s]
Tokenizing train dataset:  16%|█▌        | 285589/1807941 [04:04<23:08, 1096.51 examples/s]
Tokenizing train dataset:  16%|█▌        | 285707/1807941 [04:04<22:41, 1117.65 examples/s]
Tokenizing train dataset:  16%|█▌        | 285825/1807941 [04:04<22:23, 1133.30 examples/s]
Tokenizing train dataset:  16%|█▌        | 285946/1807941 [04:04<21:58, 1154.74 examples/s]
Tokenizing train dataset:  16%|█▌        | 286124/1807941 [04:05<22:55, 1106.47 examples/s]
Tokenizing train dataset:  16%|█▌        | 286258/1807941 [04:05<21:47, 1163.92 examples/s]
Tokenizing train dataset:  16%|█▌        | 286386/1807941 [04:05<21:18, 1190.23 examples/s]
Tokenizing train dataset:  16%|█▌        | 286518/1807941 [04:05<20:42, 1224.09 examples/s]
Tokenizing train dataset:  16%|█▌        | 286701/1807941 [04:05<20:48, 1218.15 examples/s]
Tokenizing train dataset:  16%|█▌        | 286828/1807941 [04:05<20:37, 1228.81 examples/s]
Tokenizing train dataset:  16%|█▌        | 286993/1807941 [04:05<21:28, 1180.71 examples/s]
Tokenizing train dataset:  16%|█▌        | 287152/1807941 [04:05<24:51, 1019.72 examples/s]
Tokenizing train dataset:  16%|█▌        | 287262/1807941 [04:06<24:29, 1034.80 examples/s]
Tokenizing train dataset:  16%|█▌        | 287384/1807941 [04:06<23:29, 1078.77 examples/s]
Tokenizing train dataset:  16%|█▌        | 287507/1807941 [04:06<22:45, 1113.64 examples/s]
Tokenizing train dataset:  16%|█▌        | 287627/1807941 [04:06<22:21, 1133.71 examples/s]
Tokenizing train dataset:  16%|█▌        | 287752/1807941 [04:06<21:48, 1162.12 examples/s]
Tokenizing train dataset:  16%|█▌        | 287871/1807941 [04:06<21:42, 1166.85 examples/s]
Tokenizing train dataset:  16%|█▌        | 288000/1807941 [04:06<22:34, 1122.31 examples/s]
Tokenizing train dataset:  16%|█▌        | 288131/1807941 [04:06<21:36, 1171.95 examples/s]
Tokenizing train dataset:  16%|█▌        | 288260/1807941 [04:06<21:04, 1201.95 examples/s]
Tokenizing train dataset:  16%|█▌        | 288393/1807941 [04:06<20:28, 1236.65 examples/s]
Tokenizing train dataset:  16%|█▌        | 288536/1807941 [04:07<19:37, 1290.22 examples/s]
Tokenizing train dataset:  16%|█▌        | 288679/1807941 [04:07<19:05, 1326.74 examples/s]
Tokenizing train dataset:  16%|█▌        | 288882/1807941 [04:07<19:00, 1331.81 examples/s]
Tokenizing train dataset:  16%|█▌        | 289074/1807941 [04:07<19:31, 1296.45 examples/s]
Tokenizing train dataset:  16%|█▌        | 289254/1807941 [04:07<20:06, 1258.38 examples/s]
Tokenizing train dataset:  16%|█▌        | 289384/1807941 [04:07<20:00, 1265.36 examples/s]
Tokenizing train dataset:  16%|█▌        | 289512/1807941 [04:07<20:00, 1264.34 examples/s]
Tokenizing train dataset:  16%|█▌        | 289652/1807941 [04:07<19:31, 1296.49 examples/s]
Tokenizing train dataset:  16%|█▌        | 289789/1807941 [04:08<19:14, 1314.80 examples/s]
Tokenizing train dataset:  16%|█▌        | 289932/1807941 [04:08<18:48, 1344.62 examples/s]
Tokenizing train dataset:  16%|█▌        | 290087/1807941 [04:08<20:40, 1223.57 examples/s]
Tokenizing train dataset:  16%|█▌        | 290213/1807941 [04:08<20:32, 1231.88 examples/s]
Tokenizing train dataset:  16%|█▌        | 290360/1807941 [04:08<19:32, 1294.75 examples/s]
Tokenizing train dataset:  16%|█▌        | 290498/1807941 [04:08<19:12, 1317.12 examples/s]
Tokenizing train dataset:  16%|█▌        | 290646/1807941 [04:08<18:36, 1359.23 examples/s]
Tokenizing train dataset:  16%|█▌        | 290829/1807941 [04:08<19:25, 1302.00 examples/s]
Tokenizing train dataset:  16%|█▌        | 291000/1807941 [04:08<21:19, 1185.50 examples/s]
Tokenizing train dataset:  16%|█▌        | 291129/1807941 [04:09<23:27, 1077.51 examples/s]
Tokenizing train dataset:  16%|█▌        | 291290/1807941 [04:09<23:34, 1072.03 examples/s]
Tokenizing train dataset:  16%|█▌        | 291412/1807941 [04:09<22:54, 1102.99 examples/s]
Tokenizing train dataset:  16%|█▌        | 291528/1807941 [04:09<22:41, 1113.95 examples/s]
Tokenizing train dataset:  16%|█▌        | 291653/1807941 [04:09<22:04, 1144.68 examples/s]
Tokenizing train dataset:  16%|█▌        | 291808/1807941 [04:09<20:11, 1251.42 examples/s]
Tokenizing train dataset:  16%|█▌        | 291969/1807941 [04:09<18:44, 1348.27 examples/s]
Tokenizing train dataset:  16%|█▌        | 292169/1807941 [04:09<18:52, 1338.63 examples/s]
Tokenizing train dataset:  16%|█▌        | 292325/1807941 [04:10<18:06, 1395.52 examples/s]
Tokenizing train dataset:  16%|█▌        | 292487/1807941 [04:10<17:21, 1454.87 examples/s]
Tokenizing train dataset:  16%|█▌        | 292651/1807941 [04:10<16:46, 1504.81 examples/s]
Tokenizing train dataset:  16%|█▌        | 292820/1807941 [04:10<16:16, 1552.18 examples/s]
Tokenizing train dataset:  16%|█▌        | 292995/1807941 [04:10<15:44, 1604.24 examples/s]
Tokenizing train dataset:  16%|█▌        | 293223/1807941 [04:10<16:04, 1570.26 examples/s]
Tokenizing train dataset:  16%|█▌        | 293398/1807941 [04:10<15:37, 1615.62 examples/s]
Tokenizing train dataset:  16%|█▌        | 293571/1807941 [04:10<15:21, 1642.66 examples/s]
Tokenizing train dataset:  16%|█▌        | 293742/1807941 [04:10<15:12, 1659.38 examples/s]
Tokenizing train dataset:  16%|█▋        | 293910/1807941 [04:11<15:10, 1662.81 examples/s]
Tokenizing train dataset:  16%|█▋        | 294138/1807941 [04:11<15:42, 1605.57 examples/s]
Tokenizing train dataset:  16%|█▋        | 294310/1807941 [04:11<15:27, 1632.47 examples/s]
Tokenizing train dataset:  16%|█▋        | 294486/1807941 [04:11<15:10, 1663.04 examples/s]
Tokenizing train dataset:  16%|█▋        | 294660/1807941 [04:11<15:00, 1680.79 examples/s]
Tokenizing train dataset:  16%|█▋        | 294875/1807941 [04:11<15:56, 1582.22 examples/s]
Tokenizing train dataset:  16%|█▋        | 295069/1807941 [04:11<17:34, 1434.58 examples/s]
Tokenizing train dataset:  16%|█▋        | 295257/1807941 [04:11<18:22, 1372.44 examples/s]
Tokenizing train dataset:  16%|█▋        | 295439/1807941 [04:12<19:07, 1317.67 examples/s]
Tokenizing train dataset:  16%|█▋        | 295626/1807941 [04:12<19:30, 1292.30 examples/s]
Tokenizing train dataset:  16%|█▋        | 295811/1807941 [04:12<19:48, 1272.47 examples/s]
Tokenizing train dataset:  16%|█▋        | 296000/1807941 [04:12<20:49, 1210.20 examples/s]
Tokenizing train dataset:  16%|█▋        | 296159/1807941 [04:12<21:40, 1162.18 examples/s]
Tokenizing train dataset:  16%|█▋        | 296278/1807941 [04:12<21:37, 1164.96 examples/s]
Tokenizing train dataset:  16%|█▋        | 296397/1807941 [04:12<21:35, 1166.75 examples/s]
Tokenizing train dataset:  16%|█▋        | 296516/1807941 [04:13<21:31, 1170.27 examples/s]
Tokenizing train dataset:  16%|█▋        | 296685/1807941 [04:13<21:53, 1150.75 examples/s]
Tokenizing train dataset:  16%|█▋        | 296816/1807941 [04:13<21:11, 1188.31 examples/s]
Tokenizing train dataset:  16%|█▋        | 296991/1807941 [04:13<21:23, 1177.65 examples/s]
Tokenizing train dataset:  16%|█▋        | 297126/1807941 [04:13<23:18, 1080.30 examples/s]
Tokenizing train dataset:  16%|█▋        | 297254/1807941 [04:13<22:21, 1126.53 examples/s]
Tokenizing train dataset:  16%|█▋        | 297419/1807941 [04:13<22:35, 1114.51 examples/s]
Tokenizing train dataset:  16%|█▋        | 297541/1807941 [04:13<22:07, 1138.10 examples/s]
Tokenizing train dataset:  16%|█▋        | 297665/1807941 [04:14<21:38, 1163.14 examples/s]
Tokenizing train dataset:  16%|█▋        | 297794/1807941 [04:14<21:04, 1194.20 examples/s]
Tokenizing train dataset:  16%|█▋        | 297967/1807941 [04:14<21:22, 1177.65 examples/s]
Tokenizing train dataset:  16%|█▋        | 298110/1807941 [04:14<22:58, 1095.29 examples/s]
Tokenizing train dataset:  16%|█▋        | 298225/1807941 [04:14<22:45, 1105.71 examples/s]
Tokenizing train dataset:  17%|█▋        | 298350/1807941 [04:14<24:56, 1008.83 examples/s]
Tokenizing train dataset:  17%|█▋        | 298495/1807941 [04:14<25:23, 990.94 examples/s] 
Tokenizing train dataset:  17%|█▋        | 298616/1807941 [04:14<24:10, 1040.51 examples/s]
Tokenizing train dataset:  17%|█▋        | 298735/1807941 [04:15<23:22, 1076.26 examples/s]
Tokenizing train dataset:  17%|█▋        | 298870/1807941 [04:15<21:59, 1143.90 examples/s]
Tokenizing train dataset:  17%|█▋        | 299000/1807941 [04:15<22:10, 1133.79 examples/s]
Tokenizing train dataset:  17%|█▋        | 299119/1807941 [04:15<21:57, 1145.11 examples/s]
Tokenizing train dataset:  17%|█▋        | 299243/1807941 [04:15<21:28, 1170.82 examples/s]
Tokenizing train dataset:  17%|█▋        | 299367/1807941 [04:15<21:09, 1187.94 examples/s]
Tokenizing train dataset:  17%|█▋        | 299543/1807941 [04:15<21:21, 1176.73 examples/s]
Tokenizing train dataset:  17%|█▋        | 299720/1807941 [04:15<21:26, 1172.67 examples/s]
Tokenizing train dataset:  17%|█▋        | 299838/1807941 [04:15<21:26, 1172.63 examples/s]
Tokenizing train dataset:  17%|█▋        | 300000/1807941 [04:16<23:01, 1091.72 examples/s]
Tokenizing train dataset:  17%|█▋        | 300130/1807941 [04:16<22:02, 1139.94 examples/s]
Tokenizing train dataset:  17%|█▋        | 300300/1807941 [04:16<22:07, 1135.68 examples/s]
Tokenizing train dataset:  17%|█▋        | 300475/1807941 [04:16<21:59, 1142.83 examples/s]
Tokenizing train dataset:  17%|█▋        | 300593/1807941 [04:16<21:51, 1149.09 examples/s]
Tokenizing train dataset:  17%|█▋        | 300761/1807941 [04:16<22:10, 1133.01 examples/s]
Tokenizing train dataset:  17%|█▋        | 300937/1807941 [04:16<22:01, 1140.23 examples/s]
Tokenizing train dataset:  17%|█▋        | 301092/1807941 [04:17<22:45, 1103.43 examples/s]
Tokenizing train dataset:  17%|█▋        | 301258/1807941 [04:17<22:47, 1102.00 examples/s]
Tokenizing train dataset:  17%|█▋        | 301382/1807941 [04:17<22:09, 1133.09 examples/s]
Tokenizing train dataset:  17%|█▋        | 301522/1807941 [04:17<23:34, 1064.79 examples/s]
Tokenizing train dataset:  17%|█▋        | 301694/1807941 [04:17<23:03, 1088.71 examples/s]
Tokenizing train dataset:  17%|█▋        | 301826/1807941 [04:17<22:00, 1140.26 examples/s]
Tokenizing train dataset:  17%|█▋        | 301953/1807941 [04:17<21:26, 1170.17 examples/s]
Tokenizing train dataset:  17%|█▋        | 302129/1807941 [04:18<22:11, 1130.66 examples/s]
Tokenizing train dataset:  17%|█▋        | 302307/1807941 [04:18<21:55, 1144.66 examples/s]
Tokenizing train dataset:  17%|█▋        | 302431/1807941 [04:18<21:31, 1165.39 examples/s]
Tokenizing train dataset:  17%|█▋        | 302598/1807941 [04:18<21:56, 1143.69 examples/s]
Tokenizing train dataset:  17%|█▋        | 302725/1807941 [04:18<21:23, 1172.91 examples/s]
Tokenizing train dataset:  17%|█▋        | 302851/1807941 [04:18<21:01, 1193.15 examples/s]
Tokenizing train dataset:  17%|█▋        | 303000/1807941 [04:18<22:53, 1095.91 examples/s]
Tokenizing train dataset:  17%|█▋        | 303160/1807941 [04:18<23:11, 1081.59 examples/s]
Tokenizing train dataset:  17%|█▋        | 303282/1807941 [04:19<22:31, 1113.06 examples/s]
Tokenizing train dataset:  17%|█▋        | 303410/1807941 [04:19<21:49, 1149.37 examples/s]
Tokenizing train dataset:  17%|█▋        | 303579/1807941 [04:19<22:01, 1138.31 examples/s]
Tokenizing train dataset:  17%|█▋        | 303755/1807941 [04:19<21:52, 1146.08 examples/s]
Tokenizing train dataset:  17%|█▋        | 303882/1807941 [04:19<21:19, 1175.39 examples/s]
Tokenizing train dataset:  17%|█▋        | 304024/1807941 [04:19<22:54, 1094.50 examples/s]
Tokenizing train dataset:  17%|█▋        | 304200/1807941 [04:19<22:25, 1117.95 examples/s]
Tokenizing train dataset:  17%|█▋        | 304333/1807941 [04:19<21:30, 1165.17 examples/s]
Tokenizing train dataset:  17%|█▋        | 304466/1807941 [04:20<20:48, 1204.13 examples/s]
Tokenizing train dataset:  17%|█▋        | 304603/1807941 [04:20<20:06, 1246.46 examples/s]
Tokenizing train dataset:  17%|█▋        | 304765/1807941 [04:20<18:37, 1345.54 examples/s]
Tokenizing train dataset:  17%|█▋        | 304962/1807941 [04:20<18:49, 1331.08 examples/s]
Tokenizing train dataset:  17%|█▋        | 305110/1807941 [04:20<20:52, 1199.75 examples/s]
Tokenizing train dataset:  17%|█▋        | 305286/1807941 [04:20<21:02, 1190.04 examples/s]
Tokenizing train dataset:  17%|█▋        | 305422/1807941 [04:20<22:52, 1094.51 examples/s]
Tokenizing train dataset:  17%|█▋        | 305573/1807941 [04:20<23:29, 1065.63 examples/s]
Tokenizing train dataset:  17%|█▋        | 305694/1807941 [04:21<22:48, 1097.43 examples/s]
Tokenizing train dataset:  17%|█▋        | 305828/1807941 [04:21<21:43, 1152.73 examples/s]
Tokenizing train dataset:  17%|█▋        | 305953/1807941 [04:21<21:19, 1173.95 examples/s]
Tokenizing train dataset:  17%|█▋        | 306123/1807941 [04:21<22:15, 1124.79 examples/s]
Tokenizing train dataset:  17%|█▋        | 306249/1807941 [04:21<21:38, 1156.20 examples/s]
Tokenizing train dataset:  17%|█▋        | 306374/1807941 [04:21<21:14, 1178.54 examples/s]
Tokenizing train dataset:  17%|█▋        | 306507/1807941 [04:21<20:33, 1217.14 examples/s]
Tokenizing train dataset:  17%|█▋        | 306666/1807941 [04:21<21:44, 1150.71 examples/s]
Tokenizing train dataset:  17%|█▋        | 306805/1807941 [04:22<23:25, 1068.41 examples/s]
Tokenizing train dataset:  17%|█▋        | 306945/1807941 [04:22<21:47, 1147.95 examples/s]
Tokenizing train dataset:  17%|█▋        | 307120/1807941 [04:22<22:33, 1108.46 examples/s]
Tokenizing train dataset:  17%|█▋        | 307253/1807941 [04:22<21:36, 1157.24 examples/s]
Tokenizing train dataset:  17%|█▋        | 307382/1807941 [04:22<21:05, 1185.39 examples/s]
Tokenizing train dataset:  17%|█▋        | 307505/1807941 [04:22<20:56, 1193.91 examples/s]
Tokenizing train dataset:  17%|█▋        | 307693/1807941 [04:22<20:39, 1209.95 examples/s]
Tokenizing train dataset:  17%|█▋        | 307865/1807941 [04:22<21:05, 1185.12 examples/s]
Tokenizing train dataset:  17%|█▋        | 308003/1807941 [04:23<22:47, 1096.46 examples/s]
Tokenizing train dataset:  17%|█▋        | 308121/1807941 [04:23<22:24, 1115.49 examples/s]
Tokenizing train dataset:  17%|█▋        | 308288/1807941 [04:23<19:56, 1253.15 examples/s]
Tokenizing train dataset:  17%|█▋        | 308457/1807941 [04:23<18:17, 1366.58 examples/s]
Tokenizing train dataset:  17%|█▋        | 308616/1807941 [04:23<17:31, 1426.05 examples/s]
Tokenizing train dataset:  17%|█▋        | 308780/1807941 [04:23<16:49, 1484.67 examples/s]
Tokenizing train dataset:  17%|█▋        | 308938/1807941 [04:23<16:34, 1507.07 examples/s]
Tokenizing train dataset:  17%|█▋        | 309123/1807941 [04:23<18:00, 1386.97 examples/s]
Tokenizing train dataset:  17%|█▋        | 309303/1807941 [04:24<18:57, 1317.98 examples/s]
Tokenizing train dataset:  17%|█▋        | 309484/1807941 [04:24<19:33, 1277.25 examples/s]
Tokenizing train dataset:  17%|█▋        | 309655/1807941 [04:24<20:17, 1230.84 examples/s]
Tokenizing train dataset:  17%|█▋        | 309843/1807941 [04:24<20:14, 1233.82 examples/s]
Tokenizing train dataset:  17%|█▋        | 309972/1807941 [04:24<20:01, 1246.39 examples/s]
Tokenizing train dataset:  17%|█▋        | 310143/1807941 [04:24<20:40, 1206.99 examples/s]
Tokenizing train dataset:  17%|█▋        | 310318/1807941 [04:24<20:58, 1189.60 examples/s]
Tokenizing train dataset:  17%|█▋        | 310453/1807941 [04:24<20:22, 1224.67 examples/s]
Tokenizing train dataset:  17%|█▋        | 310626/1807941 [04:25<20:50, 1196.98 examples/s]
Tokenizing train dataset:  17%|█▋        | 310785/1807941 [04:25<21:42, 1149.48 examples/s]
Tokenizing train dataset:  17%|█▋        | 310927/1807941 [04:25<23:01, 1083.31 examples/s]
Tokenizing train dataset:  17%|█▋        | 311074/1807941 [04:25<23:48, 1047.75 examples/s]
Tokenizing train dataset:  17%|█▋        | 311226/1807941 [04:25<24:07, 1033.84 examples/s]
Tokenizing train dataset:  17%|█▋        | 311348/1807941 [04:25<23:12, 1074.72 examples/s]
Tokenizing train dataset:  17%|█▋        | 311462/1807941 [04:25<22:53, 1089.21 examples/s]
Tokenizing train dataset:  17%|█▋        | 311586/1807941 [04:26<22:10, 1124.72 examples/s]
Tokenizing train dataset:  17%|█▋        | 311704/1807941 [04:26<21:54, 1138.63 examples/s]
Tokenizing train dataset:  17%|█▋        | 311830/1807941 [04:26<21:18, 1170.31 examples/s]
Tokenizing train dataset:  17%|█▋        | 311992/1807941 [04:26<22:00, 1132.90 examples/s]
Tokenizing train dataset:  17%|█▋        | 312143/1807941 [04:26<22:58, 1084.99 examples/s]
Tokenizing train dataset:  17%|█▋        | 312260/1807941 [04:26<22:33, 1105.02 examples/s]
Tokenizing train dataset:  17%|█▋        | 312380/1807941 [04:26<22:05, 1127.94 examples/s]
Tokenizing train dataset:  17%|█▋        | 312499/1807941 [04:26<21:50, 1140.90 examples/s]
Tokenizing train dataset:  17%|█▋        | 312620/1807941 [04:26<21:32, 1156.83 examples/s]
Tokenizing train dataset:  17%|█▋        | 312746/1807941 [04:27<21:04, 1182.05 examples/s]
Tokenizing train dataset:  17%|█▋        | 312869/1807941 [04:27<20:53, 1193.12 examples/s]
Tokenizing train dataset:  17%|█▋        | 312998/1807941 [04:27<20:29, 1216.37 examples/s]
Tokenizing train dataset:  17%|█▋        | 313145/1807941 [04:27<22:14, 1120.17 examples/s]
Tokenizing train dataset:  17%|█▋        | 313264/1807941 [04:27<21:56, 1135.32 examples/s]
Tokenizing train dataset:  17%|█▋        | 313384/1807941 [04:27<21:38, 1151.29 examples/s]
Tokenizing train dataset:  17%|█▋        | 313506/1807941 [04:27<21:19, 1168.14 examples/s]
Tokenizing train dataset:  17%|█▋        | 313630/1807941 [04:27<21:00, 1185.82 examples/s]
Tokenizing train dataset:  17%|█▋        | 313752/1807941 [04:27<20:53, 1191.72 examples/s]
Tokenizing train dataset:  17%|█▋        | 313884/1807941 [04:27<20:16, 1228.33 examples/s]
Tokenizing train dataset:  17%|█▋        | 314026/1807941 [04:28<22:17, 1116.93 examples/s]
Tokenizing train dataset:  17%|█▋        | 314154/1807941 [04:28<21:28, 1159.28 examples/s]
Tokenizing train dataset:  17%|█▋        | 314278/1807941 [04:28<21:06, 1178.99 examples/s]
Tokenizing train dataset:  17%|█▋        | 314403/1807941 [04:28<20:51, 1193.67 examples/s]
Tokenizing train dataset:  17%|█▋        | 314532/1807941 [04:28<20:23, 1220.16 examples/s]
Tokenizing train dataset:  17%|█▋        | 314668/1807941 [04:28<19:49, 1255.38 examples/s]
Tokenizing train dataset:  17%|█▋        | 314797/1807941 [04:28<19:43, 1261.25 examples/s]
Tokenizing train dataset:  17%|█▋        | 314929/1807941 [04:28<19:31, 1274.24 examples/s]
Tokenizing train dataset:  17%|█▋        | 315088/1807941 [04:29<20:58, 1186.43 examples/s]
Tokenizing train dataset:  17%|█▋        | 315270/1807941 [04:29<20:52, 1191.99 examples/s]
Tokenizing train dataset:  17%|█▋        | 315393/1807941 [04:29<20:42, 1201.26 examples/s]
Tokenizing train dataset:  17%|█▋        | 315523/1807941 [04:29<20:17, 1225.40 examples/s]
Tokenizing train dataset:  17%|█▋        | 315647/1807941 [04:29<20:18, 1224.45 examples/s]
Tokenizing train dataset:  17%|█▋        | 315777/1807941 [04:29<22:52, 1087.47 examples/s]
Tokenizing train dataset:  17%|█▋        | 315927/1807941 [04:29<23:36, 1053.67 examples/s]
Tokenizing train dataset:  17%|█▋        | 316078/1807941 [04:29<23:59, 1036.40 examples/s]
Tokenizing train dataset:  17%|█▋        | 316186/1807941 [04:30<23:47, 1045.06 examples/s]
Tokenizing train dataset:  17%|█▋        | 316306/1807941 [04:30<23:00, 1080.39 examples/s]
Tokenizing train dataset:  18%|█▊        | 316421/1807941 [04:30<22:40, 1096.35 examples/s]
Tokenizing train dataset:  18%|█▊        | 316594/1807941 [04:30<22:21, 1111.56 examples/s]
Tokenizing train dataset:  18%|█▊        | 316713/1807941 [04:30<22:00, 1129.54 examples/s]
Tokenizing train dataset:  18%|█▊        | 316840/1807941 [04:30<21:20, 1164.31 examples/s]
Tokenizing train dataset:  18%|█▊        | 317000/1807941 [04:30<22:57, 1082.67 examples/s]
Tokenizing train dataset:  18%|█▊        | 317165/1807941 [04:30<22:53, 1085.49 examples/s]
Tokenizing train dataset:  18%|█▊        | 317334/1807941 [04:31<22:38, 1097.08 examples/s]
Tokenizing train dataset:  18%|█▊        | 317451/1807941 [04:31<22:20, 1112.26 examples/s]
Tokenizing train dataset:  18%|█▊        | 317568/1807941 [04:31<22:05, 1124.50 examples/s]
Tokenizing train dataset:  18%|█▊        | 317692/1807941 [04:31<21:32, 1152.57 examples/s]
Tokenizing train dataset:  18%|█▊        | 317866/1807941 [04:31<21:31, 1153.46 examples/s]
Tokenizing train dataset:  18%|█▊        | 317989/1807941 [04:31<21:14, 1169.31 examples/s]
Tokenizing train dataset:  18%|█▊        | 318129/1807941 [04:31<22:55, 1083.11 examples/s]
Tokenizing train dataset:  18%|█▊        | 318256/1807941 [04:31<22:00, 1127.99 examples/s]
Tokenizing train dataset:  18%|█▊        | 318426/1807941 [04:31<22:04, 1124.72 examples/s]
Tokenizing train dataset:  18%|█▊        | 318549/1807941 [04:32<21:37, 1148.26 examples/s]
Tokenizing train dataset:  18%|█▊        | 318723/1807941 [04:32<21:34, 1150.22 examples/s]
Tokenizing train dataset:  18%|█▊        | 318843/1807941 [04:32<21:22, 1161.14 examples/s]
Tokenizing train dataset:  18%|█▊        | 319000/1807941 [04:32<22:43, 1091.77 examples/s]
Tokenizing train dataset:  18%|█▊        | 319160/1807941 [04:32<22:57, 1080.81 examples/s]
Tokenizing train dataset:  18%|█▊        | 319280/1807941 [04:32<22:24, 1107.30 examples/s]
Tokenizing train dataset:  18%|█▊        | 319440/1807941 [04:32<22:45, 1090.23 examples/s]
Tokenizing train dataset:  18%|█▊        | 319551/1807941 [04:33<22:43, 1091.24 examples/s]
Tokenizing train dataset:  18%|█▊        | 319685/1807941 [04:33<24:21, 1018.09 examples/s]
Tokenizing train dataset:  18%|█▊        | 319799/1807941 [04:33<23:44, 1044.34 examples/s]
Tokenizing train dataset:  18%|█▊        | 319914/1807941 [04:33<23:11, 1069.45 examples/s]
Tokenizing train dataset:  18%|█▊        | 320055/1807941 [04:33<24:21, 1018.22 examples/s]
Tokenizing train dataset:  18%|█▊        | 320175/1807941 [04:33<23:18, 1063.84 examples/s]
Tokenizing train dataset:  18%|█▊        | 320294/1807941 [04:33<22:41, 1092.93 examples/s]
Tokenizing train dataset:  18%|█▊        | 320411/1807941 [04:33<22:21, 1109.01 examples/s]
Tokenizing train dataset:  18%|█▊        | 320531/1807941 [04:33<21:51, 1133.75 examples/s]
Tokenizing train dataset:  18%|█▊        | 320646/1807941 [04:34<21:49, 1135.51 examples/s]
Tokenizing train dataset:  18%|█▊        | 320767/1807941 [04:34<21:29, 1152.95 examples/s]
Tokenizing train dataset:  18%|█▊        | 320895/1807941 [04:34<20:52, 1186.81 examples/s]
Tokenizing train dataset:  18%|█▊        | 321060/1807941 [04:34<21:58, 1127.77 examples/s]
Tokenizing train dataset:  18%|█▊        | 321235/1807941 [04:34<21:44, 1139.99 examples/s]
Tokenizing train dataset:  18%|█▊        | 321352/1807941 [04:34<21:39, 1143.79 examples/s]
Tokenizing train dataset:  18%|█▊        | 321478/1807941 [04:34<21:09, 1170.81 examples/s]
Tokenizing train dataset:  18%|█▊        | 321598/1807941 [04:34<21:05, 1174.48 examples/s]
Tokenizing train dataset:  18%|█▊        | 321758/1807941 [04:34<21:51, 1133.41 examples/s]
Tokenizing train dataset:  18%|█▊        | 321876/1807941 [04:35<21:39, 1143.25 examples/s]
Tokenizing train dataset:  18%|█▊        | 321998/1807941 [04:35<21:19, 1161.44 examples/s]
Tokenizing train dataset:  18%|█▊        | 322145/1807941 [04:35<22:43, 1089.71 examples/s]
Tokenizing train dataset:  18%|█▊        | 322269/1807941 [04:35<21:59, 1125.54 examples/s]
Tokenizing train dataset:  18%|█▊        | 322390/1807941 [04:35<21:36, 1145.90 examples/s]
Tokenizing train dataset:  18%|█▊        | 322510/1807941 [04:35<21:22, 1158.38 examples/s]
Tokenizing train dataset:  18%|█▊        | 322636/1807941 [04:35<20:54, 1183.90 examples/s]
Tokenizing train dataset:  18%|█▊        | 322760/1807941 [04:35<20:38, 1198.72 examples/s]
Tokenizing train dataset:  18%|█▊        | 322883/1807941 [04:35<20:33, 1203.61 examples/s]
Tokenizing train dataset:  18%|█▊        | 323051/1807941 [04:36<21:59, 1125.65 examples/s]
Tokenizing train dataset:  18%|█▊        | 323189/1807941 [04:36<23:34, 1049.55 examples/s]
Tokenizing train dataset:  18%|█▊        | 323298/1807941 [04:36<23:27, 1055.10 examples/s]
Tokenizing train dataset:  18%|█▊        | 323409/1807941 [04:36<23:11, 1066.90 examples/s]
Tokenizing train dataset:  18%|█▊        | 323542/1807941 [04:36<21:47, 1135.42 examples/s]
Tokenizing train dataset:  18%|█▊        | 323698/1807941 [04:36<19:46, 1251.18 examples/s]
Tokenizing train dataset:  18%|█▊        | 323840/1807941 [04:36<19:03, 1297.38 examples/s]
Tokenizing train dataset:  18%|█▊        | 324000/1807941 [04:36<21:02, 1175.09 examples/s]
Tokenizing train dataset:  18%|█▊        | 324122/1807941 [04:37<20:54, 1182.94 examples/s]
Tokenizing train dataset:  18%|█▊        | 324300/1807941 [04:37<20:56, 1180.67 examples/s]
Tokenizing train dataset:  18%|█▊        | 324484/1807941 [04:37<20:42, 1194.01 examples/s]
Tokenizing train dataset:  18%|█▊        | 324609/1807941 [04:37<20:33, 1202.76 examples/s]
Tokenizing train dataset:  18%|█▊        | 324784/1807941 [04:37<20:51, 1185.33 examples/s]
Tokenizing train dataset:  18%|█▊        | 324956/1807941 [04:37<21:09, 1168.31 examples/s]
Tokenizing train dataset:  18%|█▊        | 325146/1807941 [04:37<21:00, 1175.98 examples/s]
Tokenizing train dataset:  18%|█▊        | 325310/1807941 [04:38<19:18, 1280.24 examples/s]
Tokenizing train dataset:  18%|█▊        | 325464/1807941 [04:38<18:23, 1342.96 examples/s]
Tokenizing train dataset:  18%|█▊        | 325630/1807941 [04:38<19:38, 1257.92 examples/s]
Tokenizing train dataset:  18%|█▊        | 325801/1807941 [04:38<20:17, 1217.25 examples/s]
Tokenizing train dataset:  18%|█▊        | 325979/1807941 [04:38<20:32, 1202.31 examples/s]
Tokenizing train dataset:  18%|█▊        | 326136/1807941 [04:38<21:28, 1149.87 examples/s]
Tokenizing train dataset:  18%|█▊        | 326257/1807941 [04:38<21:15, 1161.46 examples/s]
Tokenizing train dataset:  18%|█▊        | 326439/1807941 [04:38<21:02, 1173.86 examples/s]
Tokenizing train dataset:  18%|█▊        | 326565/1807941 [04:39<20:43, 1191.30 examples/s]
Tokenizing train dataset:  18%|█▊        | 326695/1807941 [04:39<20:18, 1215.87 examples/s]
Tokenizing train dataset:  18%|█▊        | 326835/1807941 [04:39<19:32, 1262.88 examples/s]
Tokenizing train dataset:  18%|█▊        | 326969/1807941 [04:39<19:14, 1282.35 examples/s]
Tokenizing train dataset:  18%|█▊        | 327128/1807941 [04:39<20:40, 1194.10 examples/s]
Tokenizing train dataset:  18%|█▊        | 327299/1807941 [04:39<21:01, 1173.95 examples/s]
Tokenizing train dataset:  18%|█▊        | 327427/1807941 [04:39<20:35, 1197.97 examples/s]
Tokenizing train dataset:  18%|█▊        | 327551/1807941 [04:39<20:29, 1204.14 examples/s]
Tokenizing train dataset:  18%|█▊        | 327679/1807941 [04:39<20:15, 1217.94 examples/s]
Tokenizing train dataset:  18%|█▊        | 327827/1807941 [04:40<21:51, 1128.54 examples/s]
Tokenizing train dataset:  18%|█▊        | 327995/1807941 [04:40<21:59, 1121.47 examples/s]
Tokenizing train dataset:  18%|█▊        | 328142/1807941 [04:40<23:01, 1071.14 examples/s]
Tokenizing train dataset:  18%|█▊        | 328298/1807941 [04:40<23:17, 1058.83 examples/s]
Tokenizing train dataset:  18%|█▊        | 328419/1807941 [04:40<22:34, 1092.12 examples/s]
Tokenizing train dataset:  18%|█▊        | 328536/1807941 [04:40<22:14, 1108.58 examples/s]
Tokenizing train dataset:  18%|█▊        | 328689/1807941 [04:40<23:00, 1071.91 examples/s]
Tokenizing train dataset:  18%|█▊        | 328800/1807941 [04:41<22:50, 1078.93 examples/s]
Tokenizing train dataset:  18%|█▊        | 328920/1807941 [04:41<22:14, 1108.55 examples/s]
Tokenizing train dataset:  18%|█▊        | 329055/1807941 [04:41<23:56, 1029.60 examples/s]
Tokenizing train dataset:  18%|█▊        | 329179/1807941 [04:41<22:49, 1079.46 examples/s]
Tokenizing train dataset:  18%|█▊        | 329346/1807941 [04:41<22:42, 1085.03 examples/s]
Tokenizing train dataset:  18%|█▊        | 329462/1807941 [04:41<22:23, 1100.68 examples/s]
Tokenizing train dataset:  18%|█▊        | 329578/1807941 [04:41<22:08, 1112.61 examples/s]
Tokenizing train dataset:  18%|█▊        | 329721/1807941 [04:41<20:36, 1195.08 examples/s]
Tokenizing train dataset:  18%|█▊        | 329853/1807941 [04:41<20:05, 1226.14 examples/s]
Tokenizing train dataset:  18%|█▊        | 329979/1807941 [04:42<20:00, 1231.40 examples/s]
Tokenizing train dataset:  18%|█▊        | 330105/1807941 [04:42<52:35, 468.36 examples/s] 
Tokenizing train dataset:  18%|█▊        | 330212/1807941 [04:42<44:52, 548.75 examples/s]
Tokenizing train dataset:  18%|█▊        | 330331/1807941 [04:42<37:47, 651.74 examples/s]
Tokenizing train dataset:  18%|█▊        | 330453/1807941 [04:43<32:30, 757.36 examples/s]
Tokenizing train dataset:  18%|█▊        | 330582/1807941 [04:43<28:20, 869.01 examples/s]
Tokenizing train dataset:  18%|█▊        | 330716/1807941 [04:43<25:12, 976.95 examples/s]
Tokenizing train dataset:  18%|█▊        | 330842/1807941 [04:43<23:32, 1045.93 examples/s]
Tokenizing train dataset:  18%|█▊        | 330965/1807941 [04:43<22:34, 1090.33 examples/s]
Tokenizing train dataset:  18%|█▊        | 331118/1807941 [04:43<23:17, 1056.70 examples/s]
Tokenizing train dataset:  18%|█▊        | 331237/1807941 [04:43<22:37, 1088.10 examples/s]
Tokenizing train dataset:  18%|█▊        | 331408/1807941 [04:43<22:20, 1101.21 examples/s]
Tokenizing train dataset:  18%|█▊        | 331528/1807941 [04:43<21:52, 1124.54 examples/s]
Tokenizing train dataset:  18%|█▊        | 331701/1807941 [04:44<21:44, 1131.27 examples/s]
Tokenizing train dataset:  18%|█▊        | 331862/1807941 [04:44<22:11, 1108.58 examples/s]
Tokenizing train dataset:  18%|█▊        | 331995/1807941 [04:44<23:46, 1034.77 examples/s]
Tokenizing train dataset:  18%|█▊        | 332119/1807941 [04:44<25:27, 966.38 examples/s] 
Tokenizing train dataset:  18%|█▊        | 332241/1807941 [04:44<24:01, 1023.78 examples/s]
Tokenizing train dataset:  18%|█▊        | 332362/1807941 [04:44<23:02, 1067.57 examples/s]
Tokenizing train dataset:  18%|█▊        | 332499/1807941 [04:44<21:27, 1145.76 examples/s]
Tokenizing train dataset:  18%|█▊        | 332633/1807941 [04:44<20:35, 1193.88 examples/s]
Tokenizing train dataset:  18%|█▊        | 332758/1807941 [04:45<20:21, 1207.68 examples/s]
Tokenizing train dataset:  18%|█▊        | 332937/1807941 [04:45<20:32, 1196.97 examples/s]
Tokenizing train dataset:  18%|█▊        | 333064/1807941 [04:45<21:29, 1143.45 examples/s]
Tokenizing train dataset:  18%|█▊        | 333199/1807941 [04:45<20:33, 1195.41 examples/s]
Tokenizing train dataset:  18%|█▊        | 333362/1807941 [04:45<21:19, 1152.36 examples/s]
Tokenizing train dataset:  18%|█▊        | 333484/1807941 [04:45<21:03, 1167.02 examples/s]
Tokenizing train dataset:  18%|█▊        | 333617/1807941 [04:45<20:21, 1207.33 examples/s]
Tokenizing train dataset:  18%|█▊        | 333750/1807941 [04:45<19:50, 1238.59 examples/s]
Tokenizing train dataset:  18%|█▊        | 333892/1807941 [04:45<19:04, 1288.17 examples/s]
Tokenizing train dataset:  18%|█▊        | 334069/1807941 [04:46<19:46, 1241.87 examples/s]
Tokenizing train dataset:  18%|█▊        | 334209/1807941 [04:46<19:10, 1280.40 examples/s]
Tokenizing train dataset:  18%|█▊        | 334401/1807941 [04:46<19:13, 1277.68 examples/s]
Tokenizing train dataset:  19%|█▊        | 334583/1807941 [04:46<19:34, 1253.97 examples/s]
Tokenizing train dataset:  19%|█▊        | 334772/1807941 [04:46<19:35, 1253.20 examples/s]
Tokenizing train dataset:  19%|█▊        | 334953/1807941 [04:46<19:51, 1236.43 examples/s]
Tokenizing train dataset:  19%|█▊        | 335119/1807941 [04:46<21:22, 1148.29 examples/s]
Tokenizing train dataset:  19%|█▊        | 335295/1807941 [04:47<21:17, 1152.81 examples/s]
Tokenizing train dataset:  19%|█▊        | 335414/1807941 [04:47<21:11, 1158.47 examples/s]
Tokenizing train dataset:  19%|█▊        | 335531/1807941 [04:47<21:12, 1157.14 examples/s]
Tokenizing train dataset:  19%|█▊        | 335656/1807941 [04:47<20:48, 1179.25 examples/s]
Tokenizing train dataset:  19%|█▊        | 335790/1807941 [04:47<20:06, 1220.09 examples/s]
Tokenizing train dataset:  19%|█▊        | 335976/1807941 [04:47<20:04, 1222.42 examples/s]
Tokenizing train dataset:  19%|█▊        | 336110/1807941 [04:47<22:11, 1105.31 examples/s]
Tokenizing train dataset:  19%|█▊        | 336241/1807941 [04:47<21:17, 1151.83 examples/s]
Tokenizing train dataset:  19%|█▊        | 336417/1807941 [04:48<21:18, 1150.91 examples/s]
Tokenizing train dataset:  19%|█▊        | 336541/1807941 [04:48<20:57, 1170.29 examples/s]
Tokenizing train dataset:  19%|█▊        | 336662/1807941 [04:48<20:47, 1178.95 examples/s]
Tokenizing train dataset:  19%|█▊        | 336805/1807941 [04:48<22:24, 1093.80 examples/s]
Tokenizing train dataset:  19%|█▊        | 336960/1807941 [04:48<22:56, 1068.32 examples/s]
Tokenizing train dataset:  19%|█▊        | 337113/1807941 [04:48<23:47, 1030.71 examples/s]
Tokenizing train dataset:  19%|█▊        | 337228/1807941 [04:48<23:10, 1057.64 examples/s]
Tokenizing train dataset:  19%|█▊        | 337371/1807941 [04:48<21:21, 1147.14 examples/s]
Tokenizing train dataset:  19%|█▊        | 337497/1807941 [04:49<20:51, 1175.04 examples/s]
Tokenizing train dataset:  19%|█▊        | 337627/1807941 [04:49<20:20, 1205.14 examples/s]
Tokenizing train dataset:  19%|█▊        | 337758/1807941 [04:49<19:53, 1231.48 examples/s]
Tokenizing train dataset:  19%|█▊        | 337890/1807941 [04:49<19:31, 1254.38 examples/s]
Tokenizing train dataset:  19%|█▊        | 338050/1807941 [04:49<21:50, 1121.61 examples/s]
Tokenizing train dataset:  19%|█▊        | 338171/1807941 [04:49<21:26, 1142.42 examples/s]
Tokenizing train dataset:  19%|█▊        | 338292/1807941 [04:49<21:11, 1156.10 examples/s]
Tokenizing train dataset:  19%|█▊        | 338414/1807941 [04:49<20:55, 1170.79 examples/s]
Tokenizing train dataset:  19%|█▊        | 338587/1807941 [04:50<21:03, 1163.24 examples/s]
Tokenizing train dataset:  19%|█▊        | 338749/1807941 [04:50<21:37, 1132.09 examples/s]
Tokenizing train dataset:  19%|█▊        | 338864/1807941 [04:50<21:36, 1133.52 examples/s]
Tokenizing train dataset:  19%|█▉        | 339004/1807941 [04:50<23:03, 1061.60 examples/s]
Tokenizing train dataset:  19%|█▉        | 339131/1807941 [04:50<22:03, 1109.90 examples/s]
Tokenizing train dataset:  19%|█▉        | 339251/1807941 [04:50<21:41, 1128.75 examples/s]
Tokenizing train dataset:  19%|█▉        | 339424/1807941 [04:50<21:36, 1132.64 examples/s]
Tokenizing train dataset:  19%|█▉        | 339544/1807941 [04:50<21:19, 1147.50 examples/s]
Tokenizing train dataset:  19%|█▉        | 339663/1807941 [04:50<21:09, 1156.72 examples/s]
Tokenizing train dataset:  19%|█▉        | 339783/1807941 [04:51<20:58, 1166.80 examples/s]
Tokenizing train dataset:  19%|█▉        | 339945/1807941 [04:51<21:40, 1128.96 examples/s]
Tokenizing train dataset:  19%|█▉        | 340118/1807941 [04:51<22:39, 1079.74 examples/s]
Tokenizing train dataset:  19%|█▉        | 340274/1807941 [04:51<22:58, 1064.96 examples/s]
Tokenizing train dataset:  19%|█▉        | 340403/1807941 [04:51<21:55, 1115.67 examples/s]
Tokenizing train dataset:  19%|█▉        | 340526/1807941 [04:51<21:26, 1140.96 examples/s]
Tokenizing train dataset:  19%|█▉        | 340704/1807941 [04:51<21:11, 1153.57 examples/s]
Tokenizing train dataset:  19%|█▉        | 340830/1807941 [04:51<20:45, 1177.61 examples/s]
Tokenizing train dataset:  19%|█▉        | 340965/1807941 [04:52<20:02, 1219.99 examples/s]
Tokenizing train dataset:  19%|█▉        | 341165/1807941 [04:52<19:26, 1257.39 examples/s]
Tokenizing train dataset:  19%|█▉        | 341342/1807941 [04:52<19:55, 1226.92 examples/s]
Tokenizing train dataset:  19%|█▉        | 341466/1807941 [04:52<19:56, 1225.74 examples/s]
Tokenizing train dataset:  19%|█▉        | 341592/1807941 [04:52<19:49, 1233.25 examples/s]
Tokenizing train dataset:  19%|█▉        | 341747/1807941 [04:52<21:03, 1160.87 examples/s]
Tokenizing train dataset:  19%|█▉        | 341896/1807941 [04:52<22:12, 1100.14 examples/s]
Tokenizing train dataset:  19%|█▉        | 342057/1807941 [04:53<23:21, 1046.01 examples/s]
Tokenizing train dataset:  19%|█▉        | 342197/1807941 [04:53<21:45, 1123.07 examples/s]
Tokenizing train dataset:  19%|█▉        | 342322/1807941 [04:53<21:14, 1150.16 examples/s]
Tokenizing train dataset:  19%|█▉        | 342442/1807941 [04:53<21:01, 1161.33 examples/s]
Tokenizing train dataset:  19%|█▉        | 342564/1807941 [04:53<20:46, 1175.33 examples/s]
Tokenizing train dataset:  19%|█▉        | 342685/1807941 [04:53<20:36, 1184.53 examples/s]
Tokenizing train dataset:  19%|█▉        | 342860/1807941 [04:53<20:47, 1174.67 examples/s]
Tokenizing train dataset:  19%|█▉        | 342982/1807941 [04:53<20:39, 1181.49 examples/s]
Tokenizing train dataset:  19%|█▉        | 343125/1807941 [04:53<22:15, 1097.22 examples/s]
Tokenizing train dataset:  19%|█▉        | 343239/1807941 [04:54<22:03, 1106.45 examples/s]
Tokenizing train dataset:  19%|█▉        | 343352/1807941 [04:54<21:56, 1112.57 examples/s]
Tokenizing train dataset:  19%|█▉        | 343519/1807941 [04:54<21:58, 1110.35 examples/s]
Tokenizing train dataset:  19%|█▉        | 343641/1807941 [04:54<21:27, 1137.54 examples/s]
Tokenizing train dataset:  19%|█▉        | 343760/1807941 [04:54<21:12, 1150.33 examples/s]
Tokenizing train dataset:  19%|█▉        | 343881/1807941 [04:54<20:56, 1165.59 examples/s]
Tokenizing train dataset:  19%|█▉        | 344000/1807941 [04:54<22:23, 1089.80 examples/s]
Tokenizing train dataset:  19%|█▉        | 344112/1807941 [04:54<22:14, 1096.75 examples/s]
Tokenizing train dataset:  19%|█▉        | 344238/1807941 [04:54<21:26, 1137.69 examples/s]
Tokenizing train dataset:  19%|█▉        | 344356/1807941 [04:55<21:16, 1146.80 examples/s]
Tokenizing train dataset:  19%|█▉        | 344473/1807941 [04:55<21:14, 1148.59 examples/s]
Tokenizing train dataset:  19%|█▉        | 344594/1807941 [04:55<20:59, 1162.27 examples/s]
Tokenizing train dataset:  19%|█▉        | 344756/1807941 [04:55<21:40, 1125.51 examples/s]
Tokenizing train dataset:  19%|█▉        | 344901/1807941 [04:55<22:53, 1065.35 examples/s]
Tokenizing train dataset:  19%|█▉        | 345053/1807941 [04:55<24:54, 978.73 examples/s] 
Tokenizing train dataset:  19%|█▉        | 345200/1807941 [04:55<24:57, 976.85 examples/s]
Tokenizing train dataset:  19%|█▉        | 345308/1807941 [04:55<24:24, 998.91 examples/s]
Tokenizing train dataset:  19%|█▉        | 345411/1807941 [04:56<24:13, 1006.09 examples/s]
Tokenizing train dataset:  19%|█▉        | 345515/1807941 [04:56<24:03, 1012.85 examples/s]
Tokenizing train dataset:  19%|█▉        | 345620/1807941 [04:56<23:54, 1019.19 examples/s]
Tokenizing train dataset:  19%|█▉        | 345725/1807941 [04:56<23:49, 1023.14 examples/s]
Tokenizing train dataset:  19%|█▉        | 345836/1807941 [04:56<23:18, 1045.39 examples/s]
Tokenizing train dataset:  19%|█▉        | 345955/1807941 [04:56<22:30, 1082.59 examples/s]
Tokenizing train dataset:  19%|█▉        | 346112/1807941 [04:56<23:55, 1018.09 examples/s]
Tokenizing train dataset:  19%|█▉        | 346228/1807941 [04:56<23:09, 1051.91 examples/s]
Tokenizing train dataset:  19%|█▉        | 346344/1807941 [04:56<22:40, 1074.66 examples/s]
Tokenizing train dataset:  19%|█▉        | 346464/1807941 [04:57<21:59, 1107.44 examples/s]
Tokenizing train dataset:  19%|█▉        | 346639/1807941 [04:57<21:36, 1127.27 examples/s]
Tokenizing train dataset:  19%|█▉        | 346754/1807941 [04:57<21:31, 1130.97 examples/s]
Tokenizing train dataset:  19%|█▉        | 346873/1807941 [04:57<21:15, 1145.40 examples/s]
Tokenizing train dataset:  19%|█▉        | 346989/1807941 [04:57<21:14, 1146.31 examples/s]
Tokenizing train dataset:  19%|█▉        | 347132/1807941 [04:57<22:42, 1072.15 examples/s]
Tokenizing train dataset:  19%|█▉        | 347256/1807941 [04:57<21:49, 1115.27 examples/s]
Tokenizing train dataset:  19%|█▉        | 347433/1807941 [04:57<21:27, 1134.77 examples/s]
Tokenizing train dataset:  19%|█▉        | 347609/1807941 [04:58<21:17, 1143.06 examples/s]
Tokenizing train dataset:  19%|█▉        | 347780/1807941 [04:58<21:23, 1137.33 examples/s]
Tokenizing train dataset:  19%|█▉        | 347896/1807941 [04:58<21:23, 1137.53 examples/s]
Tokenizing train dataset:  19%|█▉        | 348060/1807941 [04:58<24:11, 1006.07 examples/s]
Tokenizing train dataset:  19%|█▉        | 348168/1807941 [04:58<23:50, 1020.48 examples/s]
Tokenizing train dataset:  19%|█▉        | 348280/1807941 [04:58<23:18, 1043.85 examples/s]
Tokenizing train dataset:  19%|█▉        | 348398/1807941 [04:58<22:38, 1074.15 examples/s]
Tokenizing train dataset:  19%|█▉        | 348515/1807941 [04:58<22:07, 1099.33 examples/s]
Tokenizing train dataset:  19%|█▉        | 348631/1807941 [04:59<21:50, 1113.62 examples/s]
Tokenizing train dataset:  19%|█▉        | 348756/1807941 [04:59<21:10, 1148.31 examples/s]
Tokenizing train dataset:  19%|█▉        | 348914/1807941 [04:59<21:53, 1110.47 examples/s]
Tokenizing train dataset:  19%|█▉        | 349053/1807941 [04:59<23:30, 1034.62 examples/s]
Tokenizing train dataset:  19%|█▉        | 349170/1807941 [04:59<22:50, 1064.68 examples/s]
Tokenizing train dataset:  19%|█▉        | 349288/1807941 [04:59<22:15, 1092.06 examples/s]
Tokenizing train dataset:  19%|█▉        | 349448/1807941 [04:59<22:34, 1076.63 examples/s]
Tokenizing train dataset:  19%|█▉        | 349607/1807941 [04:59<22:44, 1068.90 examples/s]
Tokenizing train dataset:  19%|█▉        | 349725/1807941 [05:00<22:12, 1094.51 examples/s]
Tokenizing train dataset:  19%|█▉        | 349846/1807941 [05:00<21:42, 1119.14 examples/s]
Tokenizing train dataset:  19%|█▉        | 350000/1807941 [05:00<23:39, 1026.77 examples/s]
Tokenizing train dataset:  19%|█▉        | 350111/1807941 [05:00<23:19, 1041.99 examples/s]
Tokenizing train dataset:  19%|█▉        | 350229/1807941 [05:00<22:37, 1074.10 examples/s]
Tokenizing train dataset:  19%|█▉        | 350343/1807941 [05:00<22:15, 1091.08 examples/s]
Tokenizing train dataset:  19%|█▉        | 350454/1807941 [05:00<22:11, 1094.37 examples/s]
Tokenizing train dataset:  19%|█▉        | 350607/1807941 [05:00<22:50, 1063.03 examples/s]
Tokenizing train dataset:  19%|█▉        | 350754/1807941 [05:01<23:32, 1031.67 examples/s]
Tokenizing train dataset:  19%|█▉        | 350875/1807941 [05:01<22:35, 1074.67 examples/s]
Tokenizing train dataset:  19%|█▉        | 350996/1807941 [05:01<21:52, 1109.67 examples/s]
Tokenizing train dataset:  19%|█▉        | 351133/1807941 [05:01<23:24, 1037.19 examples/s]
Tokenizing train dataset:  19%|█▉        | 351265/1807941 [05:01<21:56, 1106.09 examples/s]
Tokenizing train dataset:  19%|█▉        | 351398/1807941 [05:01<20:50, 1164.35 examples/s]
Tokenizing train dataset:  19%|█▉        | 351522/1807941 [05:01<20:31, 1183.03 examples/s]
Tokenizing train dataset:  19%|█▉        | 351700/1807941 [05:01<20:34, 1179.40 examples/s]
Tokenizing train dataset:  19%|█▉        | 351876/1807941 [05:01<20:40, 1173.45 examples/s]
Tokenizing train dataset:  19%|█▉        | 352017/1807941 [05:02<22:09, 1095.20 examples/s]
Tokenizing train dataset:  19%|█▉        | 352178/1807941 [05:02<22:20, 1086.18 examples/s]
Tokenizing train dataset:  19%|█▉        | 352295/1807941 [05:02<21:58, 1104.22 examples/s]
Tokenizing train dataset:  19%|█▉        | 352412/1807941 [05:02<21:41, 1118.63 examples/s]
Tokenizing train dataset:  19%|█▉        | 352529/1807941 [05:02<21:26, 1131.06 examples/s]
Tokenizing train dataset:  20%|█▉        | 352648/1807941 [05:02<21:10, 1145.83 examples/s]
Tokenizing train dataset:  20%|█▉        | 352767/1807941 [05:02<20:59, 1155.00 examples/s]
Tokenizing train dataset:  20%|█▉        | 352946/1807941 [05:02<20:49, 1164.29 examples/s]
Tokenizing train dataset:  20%|█▉        | 353066/1807941 [05:03<21:24, 1132.54 examples/s]
Tokenizing train dataset:  20%|█▉        | 353188/1807941 [05:03<21:00, 1153.89 examples/s]
Tokenizing train dataset:  20%|█▉        | 353332/1807941 [05:03<19:42, 1230.09 examples/s]
Tokenizing train dataset:  20%|█▉        | 353501/1807941 [05:03<20:21, 1190.98 examples/s]
Tokenizing train dataset:  20%|█▉        | 353671/1807941 [05:03<20:43, 1169.57 examples/s]
Tokenizing train dataset:  20%|█▉        | 353853/1807941 [05:03<20:30, 1181.62 examples/s]
Tokenizing train dataset:  20%|█▉        | 353996/1807941 [05:03<19:34, 1238.39 examples/s]
Tokenizing train dataset:  20%|█▉        | 354150/1807941 [05:03<20:50, 1162.97 examples/s]
Tokenizing train dataset:  20%|█▉        | 354274/1807941 [05:04<20:33, 1178.77 examples/s]
Tokenizing train dataset:  20%|█▉        | 354404/1807941 [05:04<20:03, 1207.39 examples/s]
Tokenizing train dataset:  20%|█▉        | 354534/1807941 [05:04<19:42, 1229.16 examples/s]
Tokenizing train dataset:  20%|█▉        | 354660/1807941 [05:04<19:36, 1234.75 examples/s]
Tokenizing train dataset:  20%|█▉        | 354785/1807941 [05:04<19:34, 1236.90 examples/s]
Tokenizing train dataset:  20%|█▉        | 354913/1807941 [05:04<19:26, 1245.21 examples/s]
Tokenizing train dataset:  20%|█▉        | 355092/1807941 [05:04<19:51, 1219.58 examples/s]
Tokenizing train dataset:  20%|█▉        | 355216/1807941 [05:04<19:48, 1222.21 examples/s]
Tokenizing train dataset:  20%|█▉        | 355347/1807941 [05:04<19:31, 1240.13 examples/s]
Tokenizing train dataset:  20%|█▉        | 355488/1807941 [05:05<18:50, 1284.23 examples/s]
Tokenizing train dataset:  20%|█▉        | 355620/1807941 [05:05<18:44, 1291.93 examples/s]
Tokenizing train dataset:  20%|█▉        | 355807/1807941 [05:05<19:03, 1270.05 examples/s]
Tokenizing train dataset:  20%|█▉        | 355998/1807941 [05:05<19:04, 1269.07 examples/s]
Tokenizing train dataset:  20%|█▉        | 356139/1807941 [05:05<20:54, 1157.07 examples/s]
Tokenizing train dataset:  20%|█▉        | 356321/1807941 [05:05<20:39, 1171.09 examples/s]
Tokenizing train dataset:  20%|█▉        | 356444/1807941 [05:05<20:27, 1182.67 examples/s]
Tokenizing train dataset:  20%|█▉        | 356575/1807941 [05:05<19:56, 1213.29 examples/s]
Tokenizing train dataset:  20%|█▉        | 356727/1807941 [05:06<21:12, 1140.28 examples/s]
Tokenizing train dataset:  20%|█▉        | 356879/1807941 [05:06<22:06, 1093.99 examples/s]
Tokenizing train dataset:  20%|█▉        | 356994/1807941 [05:06<21:54, 1103.92 examples/s]
Tokenizing train dataset:  20%|█▉        | 357127/1807941 [05:06<23:32, 1027.14 examples/s]
Tokenizing train dataset:  20%|█▉        | 357240/1807941 [05:06<23:00, 1050.72 examples/s]
Tokenizing train dataset:  20%|█▉        | 357359/1807941 [05:06<22:18, 1083.83 examples/s]
Tokenizing train dataset:  20%|█▉        | 357473/1807941 [05:06<22:01, 1097.88 examples/s]
Tokenizing train dataset:  20%|█▉        | 357617/1807941 [05:06<23:11, 1041.94 examples/s]
Tokenizing train dataset:  20%|█▉        | 357739/1807941 [05:07<22:16, 1084.89 examples/s]
Tokenizing train dataset:  20%|█▉        | 357863/1807941 [05:07<21:31, 1123.18 examples/s]
Tokenizing train dataset:  20%|█▉        | 357994/1807941 [05:07<20:37, 1171.43 examples/s]
Tokenizing train dataset:  20%|█▉        | 358144/1807941 [05:07<21:50, 1106.02 examples/s]
Tokenizing train dataset:  20%|█▉        | 358266/1807941 [05:07<21:18, 1133.62 examples/s]
Tokenizing train dataset:  20%|█▉        | 358384/1807941 [05:07<21:08, 1143.00 examples/s]
Tokenizing train dataset:  20%|█▉        | 358517/1807941 [05:07<20:13, 1193.99 examples/s]
Tokenizing train dataset:  20%|█▉        | 358666/1807941 [05:07<18:57, 1273.56 examples/s]
Tokenizing train dataset:  20%|█▉        | 358795/1807941 [05:07<18:57, 1274.47 examples/s]
Tokenizing train dataset:  20%|█▉        | 358925/1807941 [05:08<18:50, 1281.20 examples/s]
Tokenizing train dataset:  20%|█▉        | 359083/1807941 [05:08<20:16, 1190.95 examples/s]
Tokenizing train dataset:  20%|█▉        | 359212/1807941 [05:08<19:52, 1214.79 examples/s]
Tokenizing train dataset:  20%|█▉        | 359342/1807941 [05:08<19:33, 1234.09 examples/s]
Tokenizing train dataset:  20%|█▉        | 359520/1807941 [05:08<19:53, 1213.43 examples/s]
Tokenizing train dataset:  20%|█▉        | 359655/1807941 [05:08<19:22, 1245.36 examples/s]
Tokenizing train dataset:  20%|█▉        | 359827/1807941 [05:08<20:01, 1204.87 examples/s]
Tokenizing train dataset:  20%|█▉        | 359988/1807941 [05:08<20:51, 1156.82 examples/s]
Tokenizing train dataset:  20%|█▉        | 360130/1807941 [05:09<22:13, 1085.77 examples/s]
Tokenizing train dataset:  20%|█▉        | 360243/1807941 [05:09<22:02, 1094.98 examples/s]
Tokenizing train dataset:  20%|█▉        | 360355/1807941 [05:09<21:56, 1099.67 examples/s]
Tokenizing train dataset:  20%|█▉        | 360475/1807941 [05:09<21:28, 1123.11 examples/s]
Tokenizing train dataset:  20%|█▉        | 360644/1807941 [05:09<21:30, 1121.07 examples/s]
Tokenizing train dataset:  20%|█▉        | 360808/1807941 [05:09<21:46, 1108.00 examples/s]
Tokenizing train dataset:  20%|█▉        | 360938/1807941 [05:09<20:56, 1151.83 examples/s]
Tokenizing train dataset:  20%|█▉        | 361060/1807941 [05:09<21:44, 1109.56 examples/s]
Tokenizing train dataset:  20%|█▉        | 361184/1807941 [05:09<21:05, 1143.23 examples/s]
Tokenizing train dataset:  20%|█▉        | 361336/1807941 [05:10<19:23, 1242.92 examples/s]
Tokenizing train dataset:  20%|█▉        | 361465/1807941 [05:10<19:13, 1254.38 examples/s]
Tokenizing train dataset:  20%|██        | 361603/1807941 [05:10<18:43, 1286.86 examples/s]
Tokenizing train dataset:  20%|██        | 361737/1807941 [05:10<18:32, 1299.62 examples/s]
Tokenizing train dataset:  20%|██        | 361871/1807941 [05:10<18:25, 1308.24 examples/s]
Tokenizing train dataset:  20%|██        | 362063/1807941 [05:10<19:35, 1229.58 examples/s]
Tokenizing train dataset:  20%|██        | 362193/1807941 [05:10<19:22, 1243.42 examples/s]
Tokenizing train dataset:  20%|██        | 362328/1807941 [05:10<18:58, 1269.65 examples/s]
Tokenizing train dataset:  20%|██        | 362514/1807941 [05:11<19:12, 1253.77 examples/s]
Tokenizing train dataset:  20%|██        | 362704/1807941 [05:11<19:13, 1252.73 examples/s]
Tokenizing train dataset:  20%|██        | 362880/1807941 [05:11<19:42, 1221.53 examples/s]
Tokenizing train dataset:  20%|██        | 363055/1807941 [05:11<20:48, 1157.61 examples/s]
Tokenizing train dataset:  20%|██        | 363175/1807941 [05:11<20:41, 1163.56 examples/s]
Tokenizing train dataset:  20%|██        | 363338/1807941 [05:11<21:11, 1135.86 examples/s]
Tokenizing train dataset:  20%|██        | 363497/1807941 [05:11<21:41, 1109.77 examples/s]
Tokenizing train dataset:  20%|██        | 363609/1807941 [05:12<21:43, 1108.35 examples/s]
Tokenizing train dataset:  20%|██        | 363726/1807941 [05:12<21:27, 1121.30 examples/s]
Tokenizing train dataset:  20%|██        | 363847/1807941 [05:12<21:06, 1140.37 examples/s]
Tokenizing train dataset:  20%|██        | 363963/1807941 [05:12<21:03, 1143.13 examples/s]
Tokenizing train dataset:  20%|██        | 364108/1807941 [05:12<23:01, 1044.75 examples/s]
Tokenizing train dataset:  20%|██        | 364224/1807941 [05:12<22:25, 1072.88 examples/s]
Tokenizing train dataset:  20%|██        | 364337/1807941 [05:12<22:08, 1086.73 examples/s]
Tokenizing train dataset:  20%|██        | 364455/1807941 [05:12<21:42, 1108.03 examples/s]
Tokenizing train dataset:  20%|██        | 364591/1807941 [05:12<20:26, 1176.94 examples/s]
Tokenizing train dataset:  20%|██        | 364766/1807941 [05:13<20:31, 1171.53 examples/s]
Tokenizing train dataset:  20%|██        | 364894/1807941 [05:13<20:04, 1198.10 examples/s]
Tokenizing train dataset:  20%|██        | 365055/1807941 [05:13<21:35, 1113.58 examples/s]
Tokenizing train dataset:  20%|██        | 365174/1807941 [05:13<21:18, 1128.77 examples/s]
Tokenizing train dataset:  20%|██        | 365292/1807941 [05:13<21:03, 1141.45 examples/s]
Tokenizing train dataset:  20%|██        | 365412/1807941 [05:13<20:47, 1156.79 examples/s]
Tokenizing train dataset:  20%|██        | 365566/1807941 [05:13<19:02, 1262.95 examples/s]
Tokenizing train dataset:  20%|██        | 365729/1807941 [05:13<17:35, 1365.75 examples/s]
Tokenizing train dataset:  20%|██        | 365912/1807941 [05:13<18:23, 1306.97 examples/s]
Tokenizing train dataset:  20%|██        | 366101/1807941 [05:14<21:05, 1139.77 examples/s]
Tokenizing train dataset:  20%|██        | 366286/1807941 [05:14<20:38, 1163.86 examples/s]
Tokenizing train dataset:  20%|██        | 366463/1807941 [05:14<20:38, 1163.66 examples/s]
Tokenizing train dataset:  20%|██        | 366587/1807941 [05:14<20:25, 1176.23 examples/s]
Tokenizing train dataset:  20%|██        | 366774/1807941 [05:14<20:05, 1195.56 examples/s]
Tokenizing train dataset:  20%|██        | 366905/1807941 [05:14<19:41, 1219.31 examples/s]
Tokenizing train dataset:  20%|██        | 367045/1807941 [05:14<22:06, 1086.39 examples/s]
Tokenizing train dataset:  20%|██        | 367201/1807941 [05:15<22:29, 1067.51 examples/s]
Tokenizing train dataset:  20%|██        | 367322/1807941 [05:15<21:52, 1097.67 examples/s]
Tokenizing train dataset:  20%|██        | 367441/1807941 [05:15<21:28, 1117.81 examples/s]
Tokenizing train dataset:  20%|██        | 367561/1807941 [05:15<21:08, 1135.29 examples/s]
Tokenizing train dataset:  20%|██        | 367738/1807941 [05:15<20:56, 1146.41 examples/s]
Tokenizing train dataset:  20%|██        | 367910/1807941 [05:15<20:58, 1144.36 examples/s]
Tokenizing train dataset:  20%|██        | 368052/1807941 [05:15<22:29, 1067.36 examples/s]
Tokenizing train dataset:  20%|██        | 368177/1807941 [05:15<21:39, 1108.31 examples/s]
Tokenizing train dataset:  20%|██        | 368300/1807941 [05:16<21:04, 1138.18 examples/s]
Tokenizing train dataset:  20%|██        | 368466/1807941 [05:16<21:20, 1124.56 examples/s]
Tokenizing train dataset:  20%|██        | 368585/1807941 [05:16<21:03, 1138.96 examples/s]
Tokenizing train dataset:  20%|██        | 368704/1807941 [05:16<20:51, 1150.19 examples/s]
Tokenizing train dataset:  20%|██        | 368829/1807941 [05:16<20:25, 1174.36 examples/s]
Tokenizing train dataset:  20%|██        | 369000/1807941 [05:16<21:52, 1096.48 examples/s]
Tokenizing train dataset:  20%|██        | 369118/1807941 [05:16<21:28, 1116.79 examples/s]
Tokenizing train dataset:  20%|██        | 369232/1807941 [05:16<21:24, 1120.19 examples/s]
Tokenizing train dataset:  20%|██        | 369351/1807941 [05:17<21:07, 1134.79 examples/s]
Tokenizing train dataset:  20%|██        | 369473/1807941 [05:17<20:46, 1154.10 examples/s]
Tokenizing train dataset:  20%|██        | 369598/1807941 [05:17<20:18, 1180.31 examples/s]
Tokenizing train dataset:  20%|██        | 369717/1807941 [05:17<20:19, 1179.46 examples/s]
Tokenizing train dataset:  20%|██        | 369843/1807941 [05:17<20:00, 1197.75 examples/s]
Tokenizing train dataset:  20%|██        | 369974/1807941 [05:17<19:29, 1229.99 examples/s]
Tokenizing train dataset:  20%|██        | 370116/1807941 [05:17<21:28, 1115.62 examples/s]
Tokenizing train dataset:  20%|██        | 370239/1807941 [05:17<20:56, 1144.56 examples/s]
Tokenizing train dataset:  20%|██        | 370358/1807941 [05:17<20:45, 1154.51 examples/s]
Tokenizing train dataset:  20%|██        | 370483/1807941 [05:17<20:21, 1177.01 examples/s]
Tokenizing train dataset:  21%|██        | 370656/1807941 [05:18<20:35, 1163.40 examples/s]
Tokenizing train dataset:  21%|██        | 370834/1807941 [05:18<20:29, 1169.02 examples/s]
Tokenizing train dataset:  21%|██        | 370993/1807941 [05:18<21:12, 1128.94 examples/s]
Tokenizing train dataset:  21%|██        | 371140/1807941 [05:18<22:11, 1079.29 examples/s]
Tokenizing train dataset:  21%|██        | 371261/1807941 [05:18<21:37, 1107.44 examples/s]
Tokenizing train dataset:  21%|██        | 371380/1807941 [05:18<21:15, 1126.03 examples/s]
Tokenizing train dataset:  21%|██        | 371500/1807941 [05:18<20:57, 1142.20 examples/s]
Tokenizing train dataset:  21%|██        | 371621/1807941 [05:18<20:39, 1158.99 examples/s]
Tokenizing train dataset:  21%|██        | 371742/1807941 [05:19<20:25, 1172.09 examples/s]
Tokenizing train dataset:  21%|██        | 371920/1807941 [05:19<20:23, 1173.92 examples/s]
Tokenizing train dataset:  21%|██        | 372077/1807941 [05:19<21:13, 1127.61 examples/s]
Tokenizing train dataset:  21%|██        | 372195/1807941 [05:19<21:02, 1137.19 examples/s]
Tokenizing train dataset:  21%|██        | 372313/1807941 [05:19<20:52, 1146.23 examples/s]
Tokenizing train dataset:  21%|██        | 372443/1807941 [05:19<20:10, 1186.14 examples/s]
Tokenizing train dataset:  21%|██        | 372625/1807941 [05:19<20:03, 1192.75 examples/s]
Tokenizing train dataset:  21%|██        | 372746/1807941 [05:19<20:00, 1195.28 examples/s]
Tokenizing train dataset:  21%|██        | 372870/1807941 [05:20<19:51, 1204.91 examples/s]
Tokenizing train dataset:  21%|██        | 373014/1807941 [05:20<21:29, 1112.63 examples/s]
Tokenizing train dataset:  21%|██        | 373136/1807941 [05:20<21:00, 1138.44 examples/s]
Tokenizing train dataset:  21%|██        | 373260/1807941 [05:20<20:35, 1161.61 examples/s]
Tokenizing train dataset:  21%|██        | 373385/1807941 [05:20<20:11, 1184.54 examples/s]
Tokenizing train dataset:  21%|██        | 373511/1807941 [05:20<19:53, 1201.75 examples/s]
Tokenizing train dataset:  21%|██        | 373642/1807941 [05:20<19:26, 1229.44 examples/s]
Tokenizing train dataset:  21%|██        | 373821/1807941 [05:20<19:45, 1209.52 examples/s]
Tokenizing train dataset:  21%|██        | 373945/1807941 [05:20<19:39, 1216.09 examples/s]
Tokenizing train dataset:  21%|██        | 374120/1807941 [05:21<20:45, 1150.93 examples/s]
Tokenizing train dataset:  21%|██        | 374245/1807941 [05:21<20:23, 1172.02 examples/s]
Tokenizing train dataset:  21%|██        | 374374/1807941 [05:21<19:53, 1201.45 examples/s]
Tokenizing train dataset:  21%|██        | 374496/1807941 [05:21<19:50, 1203.84 examples/s]
Tokenizing train dataset:  21%|██        | 374623/1807941 [05:21<19:34, 1220.36 examples/s]
Tokenizing train dataset:  21%|██        | 374789/1807941 [05:21<20:20, 1174.35 examples/s]
Tokenizing train dataset:  21%|██        | 374911/1807941 [05:21<20:12, 1182.27 examples/s]
Tokenizing train dataset:  21%|██        | 375073/1807941 [05:21<20:51, 1144.77 examples/s]
Tokenizing train dataset:  21%|██        | 375204/1807941 [05:22<20:12, 1181.74 examples/s]
Tokenizing train dataset:  21%|██        | 375332/1807941 [05:22<19:48, 1205.17 examples/s]
Tokenizing train dataset:  21%|██        | 375459/1807941 [05:22<19:33, 1220.35 examples/s]
Tokenizing train dataset:  21%|██        | 375596/1807941 [05:22<18:55, 1261.26 examples/s]
Tokenizing train dataset:  21%|██        | 375725/1807941 [05:22<18:50, 1266.46 examples/s]
Tokenizing train dataset:  21%|██        | 375860/1807941 [05:22<18:34, 1285.53 examples/s]
Tokenizing train dataset:  21%|██        | 376000/1807941 [05:22<19:23, 1230.64 examples/s]
Tokenizing train dataset:  21%|██        | 376131/1807941 [05:22<19:04, 1251.42 examples/s]
Tokenizing train dataset:  21%|██        | 376263/1807941 [05:22<18:49, 1267.53 examples/s]
Tokenizing train dataset:  21%|██        | 376401/1807941 [05:22<18:23, 1297.39 examples/s]
Tokenizing train dataset:  21%|██        | 376592/1807941 [05:23<18:35, 1283.21 examples/s]
Tokenizing train dataset:  21%|██        | 376767/1807941 [05:23<19:16, 1237.90 examples/s]
Tokenizing train dataset:  21%|██        | 376938/1807941 [05:23<19:51, 1201.38 examples/s]
Tokenizing train dataset:  21%|██        | 377077/1807941 [05:23<21:31, 1108.29 examples/s]
Tokenizing train dataset:  21%|██        | 377198/1807941 [05:23<21:05, 1130.68 examples/s]
Tokenizing train dataset:  21%|██        | 377318/1807941 [05:23<20:49, 1145.32 examples/s]
Tokenizing train dataset:  21%|██        | 377490/1807941 [05:23<20:53, 1140.80 examples/s]
Tokenizing train dataset:  21%|██        | 377611/1807941 [05:24<20:35, 1157.25 examples/s]
Tokenizing train dataset:  21%|██        | 377792/1807941 [05:24<20:21, 1171.24 examples/s]
Tokenizing train dataset:  21%|██        | 377915/1807941 [05:24<20:07, 1184.54 examples/s]
Tokenizing train dataset:  21%|██        | 378073/1807941 [05:24<20:58, 1136.57 examples/s]
Tokenizing train dataset:  21%|██        | 378194/1807941 [05:24<20:42, 1151.00 examples/s]
Tokenizing train dataset:  21%|██        | 378367/1807941 [05:24<20:43, 1149.46 examples/s]
Tokenizing train dataset:  21%|██        | 378488/1807941 [05:24<20:29, 1162.31 examples/s]
Tokenizing train dataset:  21%|██        | 378608/1807941 [05:24<20:22, 1169.00 examples/s]
Tokenizing train dataset:  21%|██        | 378762/1807941 [05:25<21:20, 1115.70 examples/s]
Tokenizing train dataset:  21%|██        | 378883/1807941 [05:25<20:55, 1138.12 examples/s]
Tokenizing train dataset:  21%|██        | 379000/1807941 [05:25<22:05, 1077.81 examples/s]
Tokenizing train dataset:  21%|██        | 379114/1807941 [05:25<21:50, 1090.60 examples/s]
Tokenizing train dataset:  21%|██        | 379242/1807941 [05:25<20:53, 1140.00 examples/s]
Tokenizing train dataset:  21%|██        | 379364/1807941 [05:25<20:31, 1160.42 examples/s]
Tokenizing train dataset:  21%|██        | 379483/1807941 [05:25<20:27, 1164.15 examples/s]
Tokenizing train dataset:  21%|██        | 379664/1807941 [05:25<20:15, 1174.80 examples/s]
Tokenizing train dataset:  21%|██        | 379787/1807941 [05:25<20:03, 1186.28 examples/s]
Tokenizing train dataset:  21%|██        | 379955/1807941 [05:26<20:32, 1158.26 examples/s]
Tokenizing train dataset:  21%|██        | 380119/1807941 [05:26<21:31, 1105.13 examples/s]
Tokenizing train dataset:  21%|██        | 380250/1807941 [05:26<20:41, 1150.10 examples/s]
Tokenizing train dataset:  21%|██        | 380376/1807941 [05:26<20:14, 1175.70 examples/s]
Tokenizing train dataset:  21%|██        | 380499/1807941 [05:26<20:02, 1187.33 examples/s]
Tokenizing train dataset:  21%|██        | 380658/1807941 [05:26<18:21, 1296.30 examples/s]
Tokenizing train dataset:  21%|██        | 380793/1807941 [05:26<18:10, 1309.12 examples/s]
Tokenizing train dataset:  21%|██        | 380986/1807941 [05:26<18:22, 1294.73 examples/s]
Tokenizing train dataset:  21%|██        | 381142/1807941 [05:27<19:45, 1203.35 examples/s]
Tokenizing train dataset:  21%|██        | 381271/1807941 [05:27<19:27, 1222.06 examples/s]
Tokenizing train dataset:  21%|██        | 381408/1807941 [05:27<18:54, 1257.14 examples/s]
Tokenizing train dataset:  21%|██        | 381603/1807941 [05:27<18:42, 1270.31 examples/s]
Tokenizing train dataset:  21%|██        | 381795/1807941 [05:27<18:42, 1270.05 examples/s]
Tokenizing train dataset:  21%|██        | 381986/1807941 [05:27<18:46, 1266.16 examples/s]
Tokenizing train dataset:  21%|██        | 382164/1807941 [05:27<19:13, 1236.22 examples/s]
Tokenizing train dataset:  21%|██        | 382324/1807941 [05:27<18:02, 1316.87 examples/s]
Tokenizing train dataset:  21%|██        | 382519/1807941 [05:28<18:12, 1304.64 examples/s]
Tokenizing train dataset:  21%|██        | 382703/1807941 [05:28<18:35, 1277.83 examples/s]
Tokenizing train dataset:  21%|██        | 382836/1807941 [05:28<18:27, 1286.75 examples/s]
Tokenizing train dataset:  21%|██        | 383000/1807941 [05:28<20:12, 1175.49 examples/s]
Tokenizing train dataset:  21%|██        | 383169/1807941 [05:28<20:33, 1154.82 examples/s]
Tokenizing train dataset:  21%|██        | 383337/1807941 [05:28<20:46, 1142.55 examples/s]
Tokenizing train dataset:  21%|██        | 383514/1807941 [05:28<20:36, 1151.56 examples/s]
Tokenizing train dataset:  21%|██        | 383639/1807941 [05:29<20:14, 1172.50 examples/s]
Tokenizing train dataset:  21%|██        | 383811/1807941 [05:29<20:25, 1161.92 examples/s]
Tokenizing train dataset:  21%|██        | 383931/1807941 [05:29<20:18, 1168.65 examples/s]
Tokenizing train dataset:  21%|██        | 384072/1807941 [05:29<21:53, 1084.13 examples/s]
Tokenizing train dataset:  21%|██        | 384186/1807941 [05:29<21:40, 1094.94 examples/s]
Tokenizing train dataset:  21%|██▏       | 384304/1807941 [05:29<21:20, 1112.11 examples/s]
Tokenizing train dataset:  21%|██▏       | 384418/1807941 [05:29<21:15, 1115.67 examples/s]
Tokenizing train dataset:  21%|██▏       | 384540/1807941 [05:29<20:47, 1141.24 examples/s]
Tokenizing train dataset:  21%|██▏       | 384658/1807941 [05:29<20:37, 1149.91 examples/s]
Tokenizing train dataset:  21%|██▏       | 384782/1807941 [05:30<20:13, 1172.96 examples/s]
Tokenizing train dataset:  21%|██▏       | 384922/1807941 [05:30<19:10, 1237.02 examples/s]
Tokenizing train dataset:  21%|██▏       | 385076/1807941 [05:30<20:34, 1152.89 examples/s]
Tokenizing train dataset:  21%|██▏       | 385208/1807941 [05:30<19:49, 1196.44 examples/s]
Tokenizing train dataset:  21%|██▏       | 385340/1807941 [05:30<19:18, 1227.58 examples/s]
Tokenizing train dataset:  21%|██▏       | 385525/1807941 [05:30<19:21, 1224.93 examples/s]
Tokenizing train dataset:  21%|██▏       | 385651/1807941 [05:30<19:13, 1232.86 examples/s]
Tokenizing train dataset:  21%|██▏       | 385825/1807941 [05:30<19:43, 1201.20 examples/s]
Tokenizing train dataset:  21%|██▏       | 386000/1807941 [05:31<20:49, 1137.91 examples/s]
Tokenizing train dataset:  21%|██▏       | 386164/1807941 [05:31<21:08, 1121.23 examples/s]
Tokenizing train dataset:  21%|██▏       | 386296/1807941 [05:31<20:20, 1164.45 examples/s]
Tokenizing train dataset:  21%|██▏       | 386475/1807941 [05:31<20:17, 1167.88 examples/s]
Tokenizing train dataset:  21%|██▏       | 386602/1807941 [05:31<19:53, 1190.89 examples/s]
Tokenizing train dataset:  21%|██▏       | 386770/1807941 [05:31<20:20, 1164.67 examples/s]
Tokenizing train dataset:  21%|██▏       | 386939/1807941 [05:31<20:35, 1149.94 examples/s]
Tokenizing train dataset:  21%|██▏       | 387076/1807941 [05:32<22:04, 1072.72 examples/s]
Tokenizing train dataset:  21%|██▏       | 387240/1807941 [05:32<21:59, 1077.01 examples/s]
Tokenizing train dataset:  21%|██▏       | 387362/1807941 [05:32<21:24, 1106.04 examples/s]
Tokenizing train dataset:  21%|██▏       | 387533/1807941 [05:32<21:13, 1115.10 examples/s]
Tokenizing train dataset:  21%|██▏       | 387651/1807941 [05:32<20:58, 1128.22 examples/s]
Tokenizing train dataset:  21%|██▏       | 387810/1807941 [05:32<21:28, 1101.85 examples/s]
Tokenizing train dataset:  21%|██▏       | 387972/1807941 [05:32<21:39, 1092.42 examples/s]
Tokenizing train dataset:  21%|██▏       | 388119/1807941 [05:33<22:28, 1053.20 examples/s]
Tokenizing train dataset:  21%|██▏       | 388239/1807941 [05:33<21:51, 1082.56 examples/s]
Tokenizing train dataset:  21%|██▏       | 388358/1807941 [05:33<21:20, 1108.55 examples/s]
Tokenizing train dataset:  21%|██▏       | 388489/1807941 [05:33<20:25, 1158.29 examples/s]
Tokenizing train dataset:  21%|██▏       | 388629/1807941 [05:33<19:22, 1221.03 examples/s]
Tokenizing train dataset:  22%|██▏       | 388755/1807941 [05:33<19:13, 1230.01 examples/s]
Tokenizing train dataset:  22%|██▏       | 388882/1807941 [05:33<19:05, 1238.56 examples/s]
Tokenizing train dataset:  22%|██▏       | 389080/1807941 [05:33<19:22, 1220.18 examples/s]
Tokenizing train dataset:  22%|██▏       | 389215/1807941 [05:33<18:52, 1252.68 examples/s]
Tokenizing train dataset:  22%|██▏       | 389348/1807941 [05:33<18:36, 1271.01 examples/s]
Tokenizing train dataset:  22%|██▏       | 389536/1807941 [05:34<18:44, 1261.67 examples/s]
Tokenizing train dataset:  22%|██▏       | 389720/1807941 [05:34<18:59, 1244.87 examples/s]
Tokenizing train dataset:  22%|██▏       | 389849/1807941 [05:34<18:52, 1252.63 examples/s]
Tokenizing train dataset:  22%|██▏       | 390006/1807941 [05:34<20:04, 1177.30 examples/s]
Tokenizing train dataset:  22%|██▏       | 390176/1807941 [05:34<20:22, 1159.40 examples/s]
Tokenizing train dataset:  22%|██▏       | 390347/1807941 [05:34<20:31, 1151.19 examples/s]
Tokenizing train dataset:  22%|██▏       | 390518/1807941 [05:34<20:38, 1144.60 examples/s]
Tokenizing train dataset:  22%|██▏       | 390688/1807941 [05:35<20:45, 1137.77 examples/s]
Tokenizing train dataset:  22%|██▏       | 390809/1807941 [05:35<20:31, 1150.42 examples/s]
Tokenizing train dataset:  22%|██▏       | 390934/1807941 [05:35<20:11, 1169.97 examples/s]
Tokenizing train dataset:  22%|██▏       | 391086/1807941 [05:35<21:13, 1112.98 examples/s]
Tokenizing train dataset:  22%|██▏       | 391253/1807941 [05:35<21:17, 1108.91 examples/s]
Tokenizing train dataset:  22%|██▏       | 391375/1807941 [05:35<20:51, 1132.25 examples/s]
Tokenizing train dataset:  22%|██▏       | 391495/1807941 [05:35<20:34, 1147.43 examples/s]
Tokenizing train dataset:  22%|██▏       | 391618/1807941 [05:35<20:14, 1166.35 examples/s]
Tokenizing train dataset:  22%|██▏       | 391740/1807941 [05:36<20:02, 1177.51 examples/s]
Tokenizing train dataset:  22%|██▏       | 391914/1807941 [05:36<20:12, 1168.17 examples/s]
Tokenizing train dataset:  22%|██▏       | 392062/1807941 [05:36<21:30, 1097.51 examples/s]
Tokenizing train dataset:  22%|██▏       | 392181/1807941 [05:36<21:09, 1115.65 examples/s]
Tokenizing train dataset:  22%|██▏       | 392298/1807941 [05:36<20:57, 1126.08 examples/s]
Tokenizing train dataset:  22%|██▏       | 392416/1807941 [05:36<20:43, 1138.14 examples/s]
Tokenizing train dataset:  22%|██▏       | 392587/1807941 [05:36<20:46, 1135.43 examples/s]
Tokenizing train dataset:  22%|██▏       | 392756/1807941 [05:36<20:53, 1128.59 examples/s]
Tokenizing train dataset:  22%|██▏       | 392877/1807941 [05:37<20:37, 1143.41 examples/s]
Tokenizing train dataset:  22%|██▏       | 393000/1807941 [05:37<21:29, 1097.52 examples/s]
Tokenizing train dataset:  22%|██▏       | 393120/1807941 [05:37<21:04, 1118.90 examples/s]
Tokenizing train dataset:  22%|██▏       | 393281/1807941 [05:37<21:27, 1098.72 examples/s]
Tokenizing train dataset:  22%|██▏       | 393405/1807941 [05:37<20:50, 1131.13 examples/s]
Tokenizing train dataset:  22%|██▏       | 393524/1807941 [05:37<20:34, 1145.59 examples/s]
Tokenizing train dataset:  22%|██▏       | 393640/1807941 [05:37<20:35, 1144.80 examples/s]
Tokenizing train dataset:  22%|██▏       | 393758/1807941 [05:37<20:27, 1152.31 examples/s]
Tokenizing train dataset:  22%|██▏       | 393881/1807941 [05:37<20:06, 1172.38 examples/s]
Tokenizing train dataset:  22%|██▏       | 394022/1807941 [05:38<21:46, 1081.95 examples/s]
Tokenizing train dataset:  22%|██▏       | 394149/1807941 [05:38<20:53, 1127.99 examples/s]
Tokenizing train dataset:  22%|██▏       | 394264/1807941 [05:38<20:50, 1130.28 examples/s]
Tokenizing train dataset:  22%|██▏       | 394382/1807941 [05:38<20:38, 1141.79 examples/s]
Tokenizing train dataset:  22%|██▏       | 394556/1807941 [05:38<20:32, 1146.95 examples/s]
Tokenizing train dataset:  22%|██▏       | 394676/1807941 [05:38<20:19, 1158.80 examples/s]
Tokenizing train dataset:  22%|██▏       | 394797/1807941 [05:38<20:07, 1170.48 examples/s]
Tokenizing train dataset:  22%|██▏       | 394976/1807941 [05:38<20:02, 1175.34 examples/s]
Tokenizing train dataset:  22%|██▏       | 395145/1807941 [05:39<20:22, 1155.54 examples/s]
Tokenizing train dataset:  22%|██▏       | 395342/1807941 [05:39<17:23, 1353.65 examples/s]
Tokenizing train dataset:  22%|██▏       | 395534/1807941 [05:39<15:42, 1499.18 examples/s]
Tokenizing train dataset:  22%|██▏       | 395721/1807941 [05:39<16:45, 1404.66 examples/s]
Tokenizing train dataset:  22%|██▏       | 395911/1807941 [05:39<17:25, 1350.83 examples/s]
Tokenizing train dataset:  22%|██▏       | 396081/1807941 [05:39<18:27, 1275.35 examples/s]
Tokenizing train dataset:  22%|██▏       | 396215/1807941 [05:39<18:14, 1289.69 examples/s]
Tokenizing train dataset:  22%|██▏       | 396351/1807941 [05:39<18:02, 1304.24 examples/s]
Tokenizing train dataset:  22%|██▏       | 396538/1807941 [05:40<18:21, 1280.84 examples/s]
Tokenizing train dataset:  22%|██▏       | 396671/1807941 [05:40<18:14, 1289.76 examples/s]
Tokenizing train dataset:  22%|██▏       | 396802/1807941 [05:40<18:15, 1288.30 examples/s]
Tokenizing train dataset:  22%|██▏       | 396983/1807941 [05:40<18:43, 1255.76 examples/s]
Tokenizing train dataset:  22%|██▏       | 397116/1807941 [05:40<20:50, 1127.94 examples/s]
Tokenizing train dataset:  22%|██▏       | 397280/1807941 [05:40<21:06, 1114.03 examples/s]
Tokenizing train dataset:  22%|██▏       | 397402/1807941 [05:40<20:38, 1138.63 examples/s]
Tokenizing train dataset:  22%|██▏       | 397521/1807941 [05:40<20:26, 1149.65 examples/s]
Tokenizing train dataset:  22%|██▏       | 397641/1807941 [05:41<20:14, 1161.41 examples/s]
Tokenizing train dataset:  22%|██▏       | 397767/1807941 [05:41<19:50, 1184.34 examples/s]
Tokenizing train dataset:  22%|██▏       | 397944/1807941 [05:41<19:56, 1178.92 examples/s]
Tokenizing train dataset:  22%|██▏       | 398113/1807941 [05:41<21:17, 1103.74 examples/s]
Tokenizing train dataset:  22%|██▏       | 398233/1807941 [05:41<20:52, 1125.46 examples/s]
Tokenizing train dataset:  22%|██▏       | 398349/1807941 [05:41<20:45, 1131.38 examples/s]
Tokenizing train dataset:  22%|██▏       | 398469/1807941 [05:41<20:29, 1146.22 examples/s]
Tokenizing train dataset:  22%|██▏       | 398638/1807941 [05:41<20:40, 1135.63 examples/s]
Tokenizing train dataset:  22%|██▏       | 398754/1807941 [05:42<20:34, 1141.05 examples/s]
Tokenizing train dataset:  22%|██▏       | 398886/1807941 [05:42<19:46, 1187.64 examples/s]
Tokenizing train dataset:  22%|██▏       | 399058/1807941 [05:42<20:47, 1129.25 examples/s]
Tokenizing train dataset:  22%|██▏       | 399178/1807941 [05:42<20:28, 1146.42 examples/s]
Tokenizing train dataset:  22%|██▏       | 399303/1807941 [05:42<20:04, 1169.46 examples/s]
Tokenizing train dataset:  22%|██▏       | 399429/1807941 [05:42<19:41, 1192.36 examples/s]
Tokenizing train dataset:  22%|██▏       | 399612/1807941 [05:42<19:34, 1198.94 examples/s]
Tokenizing train dataset:  22%|██▏       | 399734/1807941 [05:42<19:31, 1202.43 examples/s]
Tokenizing train dataset:  22%|██▏       | 399858/1807941 [05:42<19:25, 1208.30 examples/s]
Tokenizing train dataset:  22%|██▏       | 400004/1807941 [05:43<20:56, 1120.61 examples/s]
Tokenizing train dataset:  22%|██▏       | 400120/1807941 [05:43<20:47, 1128.25 examples/s]
Tokenizing train dataset:  22%|██▏       | 400250/1807941 [05:43<20:02, 1170.76 examples/s]
Tokenizing train dataset:  22%|██▏       | 400371/1807941 [05:43<19:52, 1180.65 examples/s]
Tokenizing train dataset:  22%|██▏       | 400506/1807941 [05:43<19:08, 1225.05 examples/s]
Tokenizing train dataset:  22%|██▏       | 400642/1807941 [05:43<18:36, 1260.71 examples/s]
Tokenizing train dataset:  22%|██▏       | 400829/1807941 [05:43<18:46, 1248.72 examples/s]
Tokenizing train dataset:  22%|██▏       | 400961/1807941 [05:43<18:32, 1264.42 examples/s]
Tokenizing train dataset:  22%|██▏       | 401125/1807941 [05:43<19:40, 1191.88 examples/s]
Tokenizing train dataset:  22%|██▏       | 401249/1807941 [05:44<19:29, 1202.39 examples/s]
Tokenizing train dataset:  22%|██▏       | 401372/1807941 [05:44<19:26, 1205.89 examples/s]
Tokenizing train dataset:  22%|██▏       | 401500/1807941 [05:44<19:09, 1223.41 examples/s]
Tokenizing train dataset:  22%|██▏       | 401669/1807941 [05:44<19:49, 1181.82 examples/s]
Tokenizing train dataset:  22%|██▏       | 401852/1807941 [05:44<19:40, 1190.98 examples/s]
Tokenizing train dataset:  22%|██▏       | 401973/1807941 [05:44<19:39, 1192.12 examples/s]
Tokenizing train dataset:  22%|██▏       | 402127/1807941 [05:44<20:40, 1132.86 examples/s]
Tokenizing train dataset:  22%|██▏       | 402258/1807941 [05:44<19:57, 1174.20 examples/s]
Tokenizing train dataset:  22%|██▏       | 402384/1807941 [05:45<19:37, 1193.61 examples/s]
Tokenizing train dataset:  22%|██▏       | 402509/1807941 [05:45<19:25, 1205.97 examples/s]
Tokenizing train dataset:  22%|██▏       | 402639/1807941 [05:45<19:02, 1230.11 examples/s]
Tokenizing train dataset:  22%|██▏       | 402773/1807941 [05:45<18:38, 1256.61 examples/s]
Tokenizing train dataset:  22%|██▏       | 402962/1807941 [05:45<18:40, 1253.36 examples/s]
Tokenizing train dataset:  22%|██▏       | 403117/1807941 [05:45<19:59, 1171.01 examples/s]
Tokenizing train dataset:  22%|██▏       | 403238/1807941 [05:45<19:52, 1177.77 examples/s]
Tokenizing train dataset:  22%|██▏       | 403361/1807941 [05:45<19:43, 1186.84 examples/s]
Tokenizing train dataset:  22%|██▏       | 403482/1807941 [05:45<19:40, 1190.06 examples/s]
Tokenizing train dataset:  22%|██▏       | 403660/1807941 [05:46<19:46, 1183.82 examples/s]
Tokenizing train dataset:  22%|██▏       | 403819/1807941 [05:46<20:33, 1138.42 examples/s]
Tokenizing train dataset:  22%|██▏       | 403976/1807941 [05:46<21:10, 1105.25 examples/s]
Tokenizing train dataset:  22%|██▏       | 404108/1807941 [05:46<25:58, 900.59 examples/s] 
Tokenizing train dataset:  22%|██▏       | 404213/1807941 [05:46<25:07, 930.89 examples/s]
Tokenizing train dataset:  22%|██▏       | 404318/1807941 [05:46<24:29, 955.39 examples/s]
Tokenizing train dataset:  22%|██▏       | 404421/1807941 [05:46<24:08, 969.02 examples/s]
Tokenizing train dataset:  22%|██▏       | 404559/1807941 [05:47<24:40, 947.59 examples/s]
Tokenizing train dataset:  22%|██▏       | 404661/1807941 [05:47<24:21, 960.40 examples/s]
Tokenizing train dataset:  22%|██▏       | 404778/1807941 [05:47<23:04, 1013.20 examples/s]
Tokenizing train dataset:  22%|██▏       | 404892/1807941 [05:47<22:22, 1045.34 examples/s]
Tokenizing train dataset:  22%|██▏       | 405000/1807941 [05:47<23:05, 1012.87 examples/s]
Tokenizing train dataset:  22%|██▏       | 405105/1807941 [05:47<22:52, 1022.02 examples/s]
Tokenizing train dataset:  22%|██▏       | 405219/1807941 [05:47<22:10, 1054.55 examples/s]
Tokenizing train dataset:  22%|██▏       | 405367/1807941 [05:47<22:47, 1025.96 examples/s]
Tokenizing train dataset:  22%|██▏       | 405488/1807941 [05:47<21:46, 1073.63 examples/s]
Tokenizing train dataset:  22%|██▏       | 405599/1807941 [05:48<21:38, 1079.59 examples/s]
Tokenizing train dataset:  22%|██▏       | 405713/1807941 [05:48<21:22, 1093.20 examples/s]
Tokenizing train dataset:  22%|██▏       | 405858/1807941 [05:48<22:25, 1042.20 examples/s]
Tokenizing train dataset:  22%|██▏       | 405968/1807941 [05:48<22:11, 1053.23 examples/s]
Tokenizing train dataset:  22%|██▏       | 406102/1807941 [05:48<23:59, 973.81 examples/s] 
Tokenizing train dataset:  22%|██▏       | 406218/1807941 [05:48<22:56, 1018.16 examples/s]
Tokenizing train dataset:  22%|██▏       | 406337/1807941 [05:48<22:00, 1061.49 examples/s]
Tokenizing train dataset:  22%|██▏       | 406452/1807941 [05:48<21:33, 1083.70 examples/s]
Tokenizing train dataset:  22%|██▏       | 406593/1807941 [05:49<22:47, 1024.58 examples/s]
Tokenizing train dataset:  22%|██▏       | 406737/1807941 [05:49<26:26, 883.45 examples/s] 
Tokenizing train dataset:  23%|██▎       | 406888/1807941 [05:49<25:24, 918.97 examples/s]
Tokenizing train dataset:  23%|██▎       | 407000/1807941 [05:49<25:28, 916.73 examples/s]
Tokenizing train dataset:  23%|██▎       | 407117/1807941 [05:49<23:56, 975.08 examples/s]
Tokenizing train dataset:  23%|██▎       | 407279/1807941 [05:49<23:10, 1007.12 examples/s]
Tokenizing train dataset:  23%|██▎       | 407445/1807941 [05:49<22:31, 1035.95 examples/s]
Tokenizing train dataset:  23%|██▎       | 407551/1807941 [05:50<22:28, 1038.74 examples/s]
Tokenizing train dataset:  23%|██▎       | 407676/1807941 [05:50<21:25, 1089.16 examples/s]
Tokenizing train dataset:  23%|██▎       | 407836/1807941 [05:50<21:38, 1078.63 examples/s]
Tokenizing train dataset:  23%|██▎       | 408000/1807941 [05:50<25:09, 927.57 examples/s] 
Tokenizing train dataset:  23%|██▎       | 408125/1807941 [05:50<23:26, 995.06 examples/s]
Tokenizing train dataset:  23%|██▎       | 408236/1807941 [05:50<22:51, 1020.34 examples/s]
Tokenizing train dataset:  23%|██▎       | 408359/1807941 [05:50<21:49, 1068.80 examples/s]
Tokenizing train dataset:  23%|██▎       | 408483/1807941 [05:50<21:00, 1110.67 examples/s]
Tokenizing train dataset:  23%|██▎       | 408654/1807941 [05:51<20:52, 1116.77 examples/s]
Tokenizing train dataset:  23%|██▎       | 408771/1807941 [05:51<23:17, 1001.37 examples/s]
Tokenizing train dataset:  23%|██▎       | 408895/1807941 [05:51<22:04, 1056.44 examples/s]
Tokenizing train dataset:  23%|██▎       | 409051/1807941 [05:51<23:40, 984.95 examples/s] 
Tokenizing train dataset:  23%|██▎       | 409158/1807941 [05:51<23:18, 1000.48 examples/s]
Tokenizing train dataset:  23%|██▎       | 409311/1807941 [05:51<23:14, 1003.10 examples/s]
Tokenizing train dataset:  23%|██▎       | 409467/1807941 [05:51<22:59, 1013.72 examples/s]
Tokenizing train dataset:  23%|██▎       | 409593/1807941 [05:51<21:46, 1070.31 examples/s]
Tokenizing train dataset:  23%|██▎       | 409763/1807941 [05:52<21:23, 1089.18 examples/s]
Tokenizing train dataset:  23%|██▎       | 409877/1807941 [05:52<21:11, 1099.41 examples/s]
Tokenizing train dataset:  23%|██▎       | 409996/1807941 [05:52<20:47, 1120.55 examples/s]
Tokenizing train dataset:  23%|██▎       | 410132/1807941 [05:52<22:19, 1043.16 examples/s]
Tokenizing train dataset:  23%|██▎       | 410247/1807941 [05:52<21:49, 1067.71 examples/s]
Tokenizing train dataset:  23%|██▎       | 410366/1807941 [05:52<21:15, 1095.78 examples/s]
Tokenizing train dataset:  23%|██▎       | 410493/1807941 [05:52<20:24, 1141.66 examples/s]
Tokenizing train dataset:  23%|██▎       | 410638/1807941 [05:52<21:40, 1074.21 examples/s]
Tokenizing train dataset:  23%|██▎       | 410773/1807941 [05:53<25:50, 901.35 examples/s] 
Tokenizing train dataset:  23%|██▎       | 410878/1807941 [05:53<27:48, 837.51 examples/s]
Tokenizing train dataset:  23%|██▎       | 410981/1807941 [05:53<29:37, 785.87 examples/s]
Tokenizing train dataset:  23%|██▎       | 411065/1807941 [05:53<33:24, 696.70 examples/s]
Tokenizing train dataset:  23%|██▎       | 411152/1807941 [05:53<31:51, 730.83 examples/s]
Tokenizing train dataset:  23%|██▎       | 411270/1807941 [05:53<27:51, 835.59 examples/s]
Tokenizing train dataset:  23%|██▎       | 411406/1807941 [05:53<27:13, 855.03 examples/s]
Tokenizing train dataset:  23%|██▎       | 411520/1807941 [05:54<25:15, 921.63 examples/s]
Tokenizing train dataset:  23%|██▎       | 411657/1807941 [05:54<25:25, 915.02 examples/s]
Tokenizing train dataset:  23%|██▎       | 411783/1807941 [05:54<23:18, 998.00 examples/s]
Tokenizing train dataset:  23%|██▎       | 411903/1807941 [05:54<22:12, 1047.71 examples/s]
Tokenizing train dataset:  23%|██▎       | 412049/1807941 [05:54<23:39, 983.30 examples/s] 
Tokenizing train dataset:  23%|██▎       | 412161/1807941 [05:54<22:54, 1015.68 examples/s]
Tokenizing train dataset:  23%|██▎       | 412282/1807941 [05:54<21:51, 1063.80 examples/s]
Tokenizing train dataset:  23%|██▎       | 412404/1807941 [05:54<21:04, 1103.97 examples/s]
Tokenizing train dataset:  23%|██▎       | 412524/1807941 [05:54<20:37, 1127.78 examples/s]
Tokenizing train dataset:  23%|██▎       | 412646/1807941 [05:55<20:14, 1149.10 examples/s]
Tokenizing train dataset:  23%|██▎       | 412766/1807941 [05:55<19:59, 1163.02 examples/s]
Tokenizing train dataset:  23%|██▎       | 412887/1807941 [05:55<19:49, 1172.74 examples/s]
Tokenizing train dataset:  23%|██▎       | 413030/1807941 [05:55<24:17, 956.92 examples/s] 
Tokenizing train dataset:  23%|██▎       | 413192/1807941 [05:55<25:57, 895.22 examples/s]
Tokenizing train dataset:  23%|██▎       | 413289/1807941 [05:55<25:34, 908.57 examples/s]
Tokenizing train dataset:  23%|██▎       | 413387/1807941 [05:55<25:12, 922.07 examples/s]
Tokenizing train dataset:  23%|██▎       | 413511/1807941 [05:56<23:15, 999.42 examples/s]
Tokenizing train dataset:  23%|██▎       | 413631/1807941 [05:56<22:08, 1049.38 examples/s]
Tokenizing train dataset:  23%|██▎       | 413751/1807941 [05:56<21:21, 1088.22 examples/s]
Tokenizing train dataset:  23%|██▎       | 413866/1807941 [05:56<21:02, 1104.15 examples/s]
Tokenizing train dataset:  23%|██▎       | 413983/1807941 [05:56<20:45, 1119.25 examples/s]
Tokenizing train dataset:  23%|██▎       | 414142/1807941 [05:56<21:14, 1093.96 examples/s]
Tokenizing train dataset:  23%|██▎       | 414255/1807941 [05:56<21:04, 1102.01 examples/s]
Tokenizing train dataset:  23%|██▎       | 414418/1807941 [05:56<23:47, 976.00 examples/s] 
Tokenizing train dataset:  23%|██▎       | 414530/1807941 [05:56<23:04, 1006.59 examples/s]
Tokenizing train dataset:  23%|██▎       | 414659/1807941 [05:57<24:24, 951.15 examples/s] 
Tokenizing train dataset:  23%|██▎       | 414781/1807941 [05:57<22:53, 1014.61 examples/s]
Tokenizing train dataset:  23%|██▎       | 414910/1807941 [05:57<21:26, 1082.84 examples/s]
Tokenizing train dataset:  23%|██▎       | 415063/1807941 [05:57<22:01, 1054.05 examples/s]
Tokenizing train dataset:  23%|██▎       | 415179/1807941 [05:57<21:31, 1078.49 examples/s]
Tokenizing train dataset:  23%|██▎       | 415327/1807941 [05:57<22:25, 1035.39 examples/s]
Tokenizing train dataset:  23%|██▎       | 415493/1807941 [05:57<22:02, 1053.28 examples/s]
Tokenizing train dataset:  23%|██▎       | 415622/1807941 [05:57<20:56, 1108.42 examples/s]
Tokenizing train dataset:  23%|██▎       | 415750/1807941 [05:58<20:12, 1148.63 examples/s]
Tokenizing train dataset:  23%|██▎       | 415871/1807941 [05:58<20:00, 1159.89 examples/s]
Tokenizing train dataset:  23%|██▎       | 416008/1807941 [05:58<21:43, 1067.44 examples/s]
Tokenizing train dataset:  23%|██▎       | 416130/1807941 [05:58<21:01, 1103.39 examples/s]
Tokenizing train dataset:  23%|██▎       | 416258/1807941 [05:58<20:11, 1149.00 examples/s]
Tokenizing train dataset:  23%|██▎       | 416384/1807941 [05:58<19:41, 1177.90 examples/s]
Tokenizing train dataset:  23%|██▎       | 416514/1807941 [05:58<22:04, 1050.77 examples/s]
Tokenizing train dataset:  23%|██▎       | 416634/1807941 [05:58<21:20, 1086.13 examples/s]
Tokenizing train dataset:  23%|██▎       | 416749/1807941 [05:58<21:02, 1102.03 examples/s]
Tokenizing train dataset:  23%|██▎       | 416867/1807941 [05:59<23:42, 977.64 examples/s] 
Tokenizing train dataset:  23%|██▎       | 416988/1807941 [05:59<22:22, 1036.06 examples/s]
Tokenizing train dataset:  23%|██▎       | 417102/1807941 [05:59<24:47, 935.11 examples/s] 
Tokenizing train dataset:  23%|██▎       | 417215/1807941 [05:59<23:34, 983.07 examples/s]
Tokenizing train dataset:  23%|██▎       | 417338/1807941 [05:59<22:08, 1047.13 examples/s]
Tokenizing train dataset:  23%|██▎       | 417451/1807941 [05:59<21:42, 1067.44 examples/s]
Tokenizing train dataset:  23%|██▎       | 417590/1807941 [05:59<22:52, 1013.00 examples/s]
Tokenizing train dataset:  23%|██▎       | 417714/1807941 [05:59<21:40, 1068.75 examples/s]
Tokenizing train dataset:  23%|██▎       | 417839/1807941 [06:00<20:48, 1113.86 examples/s]
Tokenizing train dataset:  23%|██▎       | 418000/1807941 [06:00<22:46, 1016.93 examples/s]
Tokenizing train dataset:  23%|██▎       | 418122/1807941 [06:00<21:46, 1063.95 examples/s]
Tokenizing train dataset:  23%|██▎       | 418234/1807941 [06:00<21:30, 1076.74 examples/s]
Tokenizing train dataset:  23%|██▎       | 418355/1807941 [06:00<23:43, 976.40 examples/s] 
Tokenizing train dataset:  23%|██▎       | 418479/1807941 [06:00<22:12, 1042.46 examples/s]
Tokenizing train dataset:  23%|██▎       | 418595/1807941 [06:00<21:39, 1069.54 examples/s]
Tokenizing train dataset:  23%|██▎       | 418707/1807941 [06:00<21:28, 1078.48 examples/s]
Tokenizing train dataset:  23%|██▎       | 418835/1807941 [06:00<20:27, 1131.80 examples/s]
Tokenizing train dataset:  23%|██▎       | 418964/1807941 [06:01<19:43, 1173.94 examples/s]
Tokenizing train dataset:  23%|██▎       | 419131/1807941 [06:01<20:22, 1135.94 examples/s]
Tokenizing train dataset:  23%|██▎       | 419267/1807941 [06:01<19:24, 1192.08 examples/s]
Tokenizing train dataset:  23%|██▎       | 419401/1807941 [06:01<18:48, 1230.47 examples/s]
Tokenizing train dataset:  23%|██▎       | 419578/1807941 [06:01<19:08, 1209.14 examples/s]
Tokenizing train dataset:  23%|██▎       | 419759/1807941 [06:01<19:12, 1204.89 examples/s]
Tokenizing train dataset:  23%|██▎       | 419885/1807941 [06:01<19:01, 1215.92 examples/s]
Tokenizing train dataset:  23%|██▎       | 420064/1807941 [06:02<19:32, 1183.88 examples/s]
Tokenizing train dataset:  23%|██▎       | 420191/1807941 [06:02<24:04, 960.39 examples/s] 
Tokenizing train dataset:  23%|██▎       | 420301/1807941 [06:02<25:57, 891.22 examples/s]
Tokenizing train dataset:  23%|██▎       | 420419/1807941 [06:02<24:17, 952.16 examples/s]
Tokenizing train dataset:  23%|██▎       | 420543/1807941 [06:02<22:41, 1018.92 examples/s]
Tokenizing train dataset:  23%|██▎       | 420673/1807941 [06:02<21:13, 1089.47 examples/s]
Tokenizing train dataset:  23%|██▎       | 420804/1807941 [06:02<20:16, 1140.66 examples/s]
Tokenizing train dataset:  23%|██▎       | 420933/1807941 [06:02<19:35, 1179.69 examples/s]
Tokenizing train dataset:  23%|██▎       | 421067/1807941 [06:03<20:35, 1122.43 examples/s]
Tokenizing train dataset:  23%|██▎       | 421205/1807941 [06:03<19:25, 1189.76 examples/s]
Tokenizing train dataset:  23%|██▎       | 421339/1807941 [06:03<18:48, 1228.57 examples/s]
Tokenizing train dataset:  23%|██▎       | 421485/1807941 [06:03<20:33, 1123.62 examples/s]
Tokenizing train dataset:  23%|██▎       | 421633/1807941 [06:03<21:31, 1073.10 examples/s]
Tokenizing train dataset:  23%|██▎       | 421786/1807941 [06:03<21:57, 1052.44 examples/s]
Tokenizing train dataset:  23%|██▎       | 421900/1807941 [06:03<21:35, 1070.00 examples/s]
Tokenizing train dataset:  23%|██▎       | 422058/1807941 [06:03<21:59, 1050.41 examples/s]
Tokenizing train dataset:  23%|██▎       | 422191/1807941 [06:04<20:43, 1114.60 examples/s]
Tokenizing train dataset:  23%|██▎       | 422310/1807941 [06:04<20:24, 1131.60 examples/s]
Tokenizing train dataset:  23%|██▎       | 422428/1807941 [06:04<20:13, 1141.34 examples/s]
Tokenizing train dataset:  23%|██▎       | 422590/1807941 [06:04<20:42, 1115.26 examples/s]
Tokenizing train dataset:  23%|██▎       | 422713/1807941 [06:04<20:13, 1141.10 examples/s]
Tokenizing train dataset:  23%|██▎       | 422873/1807941 [06:04<20:46, 1110.97 examples/s]
Tokenizing train dataset:  23%|██▎       | 422986/1807941 [06:04<20:42, 1114.47 examples/s]
Tokenizing train dataset:  23%|██▎       | 423115/1807941 [06:04<22:33, 1023.49 examples/s]
Tokenizing train dataset:  23%|██▎       | 423230/1807941 [06:04<21:54, 1053.52 examples/s]
Tokenizing train dataset:  23%|██▎       | 423345/1807941 [06:05<21:25, 1077.45 examples/s]
Tokenizing train dataset:  23%|██▎       | 423469/1807941 [06:05<20:39, 1116.53 examples/s]
Tokenizing train dataset:  23%|██▎       | 423632/1807941 [06:05<20:54, 1103.53 examples/s]
Tokenizing train dataset:  23%|██▎       | 423746/1807941 [06:05<20:46, 1110.05 examples/s]
Tokenizing train dataset:  23%|██▎       | 423864/1807941 [06:05<20:27, 1127.45 examples/s]
Tokenizing train dataset:  23%|██▎       | 423987/1807941 [06:05<20:01, 1151.76 examples/s]
Tokenizing train dataset:  23%|██▎       | 424147/1807941 [06:05<20:41, 1114.38 examples/s]
Tokenizing train dataset:  23%|██▎       | 424271/1807941 [06:05<20:08, 1144.68 examples/s]
Tokenizing train dataset:  23%|██▎       | 424395/1807941 [06:05<19:44, 1167.96 examples/s]
Tokenizing train dataset:  23%|██▎       | 424518/1807941 [06:06<19:29, 1183.17 examples/s]
Tokenizing train dataset:  23%|██▎       | 424653/1807941 [06:06<18:46, 1228.37 examples/s]
Tokenizing train dataset:  23%|██▎       | 424779/1807941 [06:06<18:40, 1234.93 examples/s]
Tokenizing train dataset:  24%|██▎       | 424907/1807941 [06:06<18:32, 1243.11 examples/s]
Tokenizing train dataset:  24%|██▎       | 425056/1807941 [06:06<20:09, 1143.53 examples/s]
Tokenizing train dataset:  24%|██▎       | 425222/1807941 [06:06<20:28, 1125.10 examples/s]
Tokenizing train dataset:  24%|██▎       | 425352/1807941 [06:06<24:48, 928.91 examples/s] 
Tokenizing train dataset:  24%|██▎       | 425482/1807941 [06:07<28:12, 816.67 examples/s]
Tokenizing train dataset:  24%|██▎       | 425606/1807941 [06:07<25:32, 902.28 examples/s]
Tokenizing train dataset:  24%|██▎       | 425746/1807941 [06:07<22:53, 1006.56 examples/s]
Tokenizing train dataset:  24%|██▎       | 425865/1807941 [06:07<21:58, 1048.07 examples/s]
Tokenizing train dataset:  24%|██▎       | 426000/1807941 [06:07<23:19, 987.54 examples/s] 
Tokenizing train dataset:  24%|██▎       | 426133/1807941 [06:07<21:34, 1067.49 examples/s]
Tokenizing train dataset:  24%|██▎       | 426299/1807941 [06:07<21:24, 1075.71 examples/s]
Tokenizing train dataset:  24%|██▎       | 426426/1807941 [06:07<20:30, 1122.77 examples/s]
Tokenizing train dataset:  24%|██▎       | 426571/1807941 [06:08<21:37, 1064.64 examples/s]
Tokenizing train dataset:  24%|██▎       | 426700/1807941 [06:08<20:36, 1116.63 examples/s]
Tokenizing train dataset:  24%|██▎       | 426842/1807941 [06:08<19:18, 1192.38 examples/s]
Tokenizing train dataset:  24%|██▎       | 427000/1807941 [06:08<21:10, 1086.97 examples/s]
Tokenizing train dataset:  24%|██▎       | 427134/1807941 [06:08<20:04, 1146.40 examples/s]
Tokenizing train dataset:  24%|██▎       | 427271/1807941 [06:08<19:09, 1201.37 examples/s]
Tokenizing train dataset:  24%|██▎       | 427416/1807941 [06:08<20:44, 1109.55 examples/s]
Tokenizing train dataset:  24%|██▎       | 427535/1807941 [06:08<20:23, 1128.25 examples/s]
Tokenizing train dataset:  24%|██▎       | 427663/1807941 [06:08<19:41, 1167.93 examples/s]
Tokenizing train dataset:  24%|██▎       | 427786/1807941 [06:09<19:27, 1182.12 examples/s]
Tokenizing train dataset:  24%|██▎       | 427914/1807941 [06:09<19:02, 1207.80 examples/s]
Tokenizing train dataset:  24%|██▎       | 428069/1807941 [06:09<20:11, 1138.74 examples/s]
Tokenizing train dataset:  24%|██▎       | 428236/1807941 [06:09<20:30, 1121.30 examples/s]
Tokenizing train dataset:  24%|██▎       | 428407/1807941 [06:09<20:25, 1125.62 examples/s]
Tokenizing train dataset:  24%|██▎       | 428524/1807941 [06:09<20:15, 1134.93 examples/s]
Tokenizing train dataset:  24%|██▎       | 428654/1807941 [06:09<19:34, 1174.19 examples/s]
Tokenizing train dataset:  24%|██▎       | 428778/1807941 [06:09<19:21, 1187.74 examples/s]
Tokenizing train dataset:  24%|██▎       | 428903/1807941 [06:10<19:07, 1201.28 examples/s]
Tokenizing train dataset:  24%|██▎       | 429061/1807941 [06:10<20:24, 1126.25 examples/s]
Tokenizing train dataset:  24%|██▎       | 429192/1807941 [06:10<19:36, 1172.12 examples/s]
Tokenizing train dataset:  24%|██▎       | 429316/1807941 [06:10<19:21, 1187.10 examples/s]
Tokenizing train dataset:  24%|██▍       | 429488/1807941 [06:10<19:37, 1170.21 examples/s]
Tokenizing train dataset:  24%|██▍       | 429608/1807941 [06:10<19:33, 1174.62 examples/s]
Tokenizing train dataset:  24%|██▍       | 429729/1807941 [06:10<19:25, 1182.41 examples/s]
Tokenizing train dataset:  24%|██▍       | 429879/1807941 [06:10<20:42, 1109.30 examples/s]
Tokenizing train dataset:  24%|██▍       | 430000/1807941 [06:11<26:26, 868.35 examples/s] 
Tokenizing train dataset:  24%|██▍       | 430131/1807941 [06:11<29:04, 789.60 examples/s]
Tokenizing train dataset:  24%|██▍       | 430243/1807941 [06:11<29:44, 771.93 examples/s]
Tokenizing train dataset:  24%|██▍       | 430338/1807941 [06:11<28:32, 804.24 examples/s]
Tokenizing train dataset:  24%|██▍       | 430427/1807941 [06:11<31:21, 732.24 examples/s]
Tokenizing train dataset:  24%|██▍       | 430523/1807941 [06:11<29:29, 778.39 examples/s]
Tokenizing train dataset:  24%|██▍       | 430646/1807941 [06:11<25:55, 885.46 examples/s]
Tokenizing train dataset:  24%|██▍       | 430784/1807941 [06:12<22:42, 1010.63 examples/s]
Tokenizing train dataset:  24%|██▍       | 430907/1807941 [06:12<21:31, 1066.19 examples/s]
Tokenizing train dataset:  24%|██▍       | 431055/1807941 [06:12<22:20, 1027.15 examples/s]
Tokenizing train dataset:  24%|██▍       | 431208/1807941 [06:12<22:29, 1020.39 examples/s]
Tokenizing train dataset:  24%|██▍       | 431329/1807941 [06:12<21:34, 1063.36 examples/s]
Tokenizing train dataset:  24%|██▍       | 431454/1807941 [06:12<20:42, 1108.02 examples/s]
Tokenizing train dataset:  24%|██▍       | 431610/1807941 [06:12<21:12, 1081.29 examples/s]
Tokenizing train dataset:  24%|██▍       | 431736/1807941 [06:12<20:25, 1122.70 examples/s]
Tokenizing train dataset:  24%|██▍       | 431855/1807941 [06:13<20:08, 1138.80 examples/s]
Tokenizing train dataset:  24%|██▍       | 431972/1807941 [06:13<20:02, 1144.44 examples/s]
Tokenizing train dataset:  24%|██▍       | 432121/1807941 [06:13<21:08, 1085.03 examples/s]
Tokenizing train dataset:  24%|██▍       | 432248/1807941 [06:13<20:15, 1131.80 examples/s]
Tokenizing train dataset:  24%|██▍       | 432376/1807941 [06:13<19:36, 1168.98 examples/s]
Tokenizing train dataset:  24%|██▍       | 432519/1807941 [06:13<18:30, 1238.10 examples/s]
Tokenizing train dataset:  24%|██▍       | 432661/1807941 [06:13<20:33, 1114.87 examples/s]
Tokenizing train dataset:  24%|██▍       | 432840/1807941 [06:13<20:07, 1138.81 examples/s]
Tokenizing train dataset:  24%|██▍       | 432975/1807941 [06:13<19:17, 1188.15 examples/s]
Tokenizing train dataset:  24%|██▍       | 433145/1807941 [06:14<19:41, 1163.66 examples/s]
Tokenizing train dataset:  24%|██▍       | 433296/1807941 [06:14<20:40, 1108.27 examples/s]
Tokenizing train dataset:  24%|██▍       | 433424/1807941 [06:14<19:57, 1147.37 examples/s]
Tokenizing train dataset:  24%|██▍       | 433563/1807941 [06:14<19:01, 1203.86 examples/s]
Tokenizing train dataset:  24%|██▍       | 433703/1807941 [06:14<18:17, 1251.95 examples/s]
Tokenizing train dataset:  24%|██▍       | 433834/1807941 [06:14<18:06, 1264.45 examples/s]
Tokenizing train dataset:  24%|██▍       | 433969/1807941 [06:14<17:50, 1283.02 examples/s]
Tokenizing train dataset:  24%|██▍       | 434141/1807941 [06:14<18:36, 1230.11 examples/s]
Tokenizing train dataset:  24%|██▍       | 434278/1807941 [06:15<18:05, 1265.14 examples/s]
Tokenizing train dataset:  24%|██▍       | 434410/1807941 [06:15<17:54, 1278.02 examples/s]
Tokenizing train dataset:  24%|██▍       | 434582/1807941 [06:15<18:41, 1225.00 examples/s]
Tokenizing train dataset:  24%|██▍       | 434752/1807941 [06:15<21:31, 1063.04 examples/s]
Tokenizing train dataset:  24%|██▍       | 434886/1807941 [06:15<20:21, 1124.49 examples/s]
Tokenizing train dataset:  24%|██▍       | 435019/1807941 [06:15<21:57, 1042.12 examples/s]
Tokenizing train dataset:  24%|██▍       | 435138/1807941 [06:15<21:18, 1073.79 examples/s]
Tokenizing train dataset:  24%|██▍       | 435270/1807941 [06:15<20:10, 1134.15 examples/s]
Tokenizing train dataset:  24%|██▍       | 435390/1807941 [06:16<19:54, 1148.84 examples/s]
Tokenizing train dataset:  24%|██▍       | 435510/1807941 [06:16<19:40, 1162.11 examples/s]
Tokenizing train dataset:  24%|██▍       | 435635/1807941 [06:16<22:03, 1037.14 examples/s]
Tokenizing train dataset:  24%|██▍       | 435789/1807941 [06:16<24:46, 922.81 examples/s] 
Tokenizing train dataset:  24%|██▍       | 435935/1807941 [06:16<27:02, 845.74 examples/s]
Tokenizing train dataset:  24%|██▍       | 436090/1807941 [06:16<25:28, 897.27 examples/s]
Tokenizing train dataset:  24%|██▍       | 436229/1807941 [06:16<22:51, 1000.25 examples/s]
Tokenizing train dataset:  24%|██▍       | 436344/1807941 [06:17<22:11, 1030.30 examples/s]
Tokenizing train dataset:  24%|██▍       | 436454/1807941 [06:17<21:54, 1043.08 examples/s]
Tokenizing train dataset:  24%|██▍       | 436587/1807941 [06:17<20:29, 1115.80 examples/s]
Tokenizing train dataset:  24%|██▍       | 436721/1807941 [06:17<19:26, 1175.58 examples/s]
Tokenizing train dataset:  24%|██▍       | 436858/1807941 [06:17<18:36, 1228.14 examples/s]
Tokenizing train dataset:  24%|██▍       | 436992/1807941 [06:17<18:08, 1259.04 examples/s]
Tokenizing train dataset:  24%|██▍       | 437144/1807941 [06:17<19:38, 1163.07 examples/s]
Tokenizing train dataset:  24%|██▍       | 437273/1807941 [06:17<19:07, 1194.45 examples/s]
Tokenizing train dataset:  24%|██▍       | 437426/1807941 [06:17<20:16, 1126.30 examples/s]
Tokenizing train dataset:  24%|██▍       | 437555/1807941 [06:18<19:36, 1164.87 examples/s]
Tokenizing train dataset:  24%|██▍       | 437688/1807941 [06:18<18:57, 1204.70 examples/s]
Tokenizing train dataset:  24%|██▍       | 437821/1807941 [06:18<18:26, 1238.35 examples/s]
Tokenizing train dataset:  24%|██▍       | 437975/1807941 [06:18<17:17, 1320.85 examples/s]
Tokenizing train dataset:  24%|██▍       | 438133/1807941 [06:18<18:45, 1217.07 examples/s]
Tokenizing train dataset:  24%|██▍       | 438262/1807941 [06:18<18:29, 1234.17 examples/s]
Tokenizing train dataset:  24%|██▍       | 438390/1807941 [06:18<18:21, 1243.48 examples/s]
Tokenizing train dataset:  24%|██▍       | 438539/1807941 [06:18<17:25, 1310.03 examples/s]
Tokenizing train dataset:  24%|██▍       | 438734/1807941 [06:18<17:30, 1302.97 examples/s]
Tokenizing train dataset:  24%|██▍       | 438870/1807941 [06:19<17:22, 1313.68 examples/s]
Tokenizing train dataset:  24%|██▍       | 439070/1807941 [06:19<17:33, 1299.30 examples/s]
Tokenizing train dataset:  24%|██▍       | 439270/1807941 [06:19<17:27, 1306.67 examples/s]
Tokenizing train dataset:  24%|██▍       | 439430/1807941 [06:19<20:42, 1100.99 examples/s]
Tokenizing train dataset:  24%|██▍       | 439605/1807941 [06:19<22:23, 1018.31 examples/s]
Tokenizing train dataset:  24%|██▍       | 439742/1807941 [06:19<20:56, 1088.95 examples/s]
Tokenizing train dataset:  24%|██▍       | 439876/1807941 [06:19<19:58, 1141.07 examples/s]
Tokenizing train dataset:  24%|██▍       | 440000/1807941 [06:20<20:50, 1094.16 examples/s]
Tokenizing train dataset:  24%|██▍       | 440136/1807941 [06:20<19:42, 1156.73 examples/s]
Tokenizing train dataset:  24%|██▍       | 440273/1807941 [06:20<18:50, 1209.80 examples/s]
Tokenizing train dataset:  24%|██▍       | 440431/1807941 [06:20<19:58, 1141.00 examples/s]
Tokenizing train dataset:  24%|██▍       | 440553/1807941 [06:20<19:44, 1154.26 examples/s]
Tokenizing train dataset:  24%|██▍       | 440702/1807941 [06:20<20:52, 1092.01 examples/s]
Tokenizing train dataset:  24%|██▍       | 440826/1807941 [06:20<20:13, 1126.66 examples/s]
Tokenizing train dataset:  24%|██▍       | 440959/1807941 [06:20<19:21, 1177.17 examples/s]
Tokenizing train dataset:  24%|██▍       | 441099/1807941 [06:21<21:32, 1057.89 examples/s]
Tokenizing train dataset:  24%|██▍       | 441213/1807941 [06:21<23:50, 955.51 examples/s] 
Tokenizing train dataset:  24%|██▍       | 441326/1807941 [06:21<22:53, 995.15 examples/s]
Tokenizing train dataset:  24%|██▍       | 441438/1807941 [06:21<22:14, 1023.67 examples/s]
Tokenizing train dataset:  24%|██▍       | 441558/1807941 [06:21<21:19, 1067.88 examples/s]
Tokenizing train dataset:  24%|██▍       | 441678/1807941 [06:21<20:38, 1103.02 examples/s]
Tokenizing train dataset:  24%|██▍       | 441798/1807941 [06:21<20:12, 1126.83 examples/s]
Tokenizing train dataset:  24%|██▍       | 441915/1807941 [06:21<20:03, 1135.49 examples/s]
Tokenizing train dataset:  24%|██▍       | 442062/1807941 [06:22<21:11, 1074.02 examples/s]
Tokenizing train dataset:  24%|██▍       | 442178/1807941 [06:22<23:36, 964.03 examples/s] 
Tokenizing train dataset:  24%|██▍       | 442289/1807941 [06:22<22:46, 999.17 examples/s]
Tokenizing train dataset:  24%|██▍       | 442392/1807941 [06:22<22:37, 1006.02 examples/s]
Tokenizing train dataset:  24%|██▍       | 442516/1807941 [06:22<21:20, 1066.73 examples/s]
Tokenizing train dataset:  24%|██▍       | 442639/1807941 [06:22<20:29, 1110.77 examples/s]
Tokenizing train dataset:  24%|██▍       | 442757/1807941 [06:22<20:08, 1130.03 examples/s]
Tokenizing train dataset:  24%|██▍       | 442876/1807941 [06:22<19:50, 1146.41 examples/s]
Tokenizing train dataset:  25%|██▍       | 443000/1807941 [06:22<20:41, 1099.44 examples/s]
Tokenizing train dataset:  25%|██▍       | 443113/1807941 [06:22<20:33, 1106.25 examples/s]
Tokenizing train dataset:  25%|██▍       | 443241/1807941 [06:23<19:44, 1152.60 examples/s]
Tokenizing train dataset:  25%|██▍       | 443373/1807941 [06:23<18:59, 1197.28 examples/s]
Tokenizing train dataset:  25%|██▍       | 443498/1807941 [06:23<18:49, 1208.47 examples/s]
Tokenizing train dataset:  25%|██▍       | 443679/1807941 [06:23<18:58, 1198.80 examples/s]
Tokenizing train dataset:  25%|██▍       | 443840/1807941 [06:23<19:50, 1145.76 examples/s]
Tokenizing train dataset:  25%|██▍       | 444000/1807941 [06:23<21:21, 1064.44 examples/s]
Tokenizing train dataset:  25%|██▍       | 444120/1807941 [06:23<20:46, 1094.47 examples/s]
Tokenizing train dataset:  25%|██▍       | 444266/1807941 [06:24<21:38, 1050.52 examples/s]
Tokenizing train dataset:  25%|██▍       | 444384/1807941 [06:24<21:04, 1078.56 examples/s]
Tokenizing train dataset:  25%|██▍       | 444507/1807941 [06:24<20:21, 1115.84 examples/s]
Tokenizing train dataset:  25%|██▍       | 444631/1807941 [06:24<19:49, 1146.42 examples/s]
Tokenizing train dataset:  25%|██▍       | 444749/1807941 [06:24<19:46, 1148.81 examples/s]
Tokenizing train dataset:  25%|██▍       | 444870/1807941 [06:24<19:33, 1161.83 examples/s]
Tokenizing train dataset:  25%|██▍       | 444993/1807941 [06:24<19:15, 1179.55 examples/s]
Tokenizing train dataset:  25%|██▍       | 445126/1807941 [06:24<21:19, 1064.73 examples/s]
Tokenizing train dataset:  25%|██▍       | 445250/1807941 [06:24<20:28, 1109.18 examples/s]
Tokenizing train dataset:  25%|██▍       | 445402/1807941 [06:25<21:10, 1072.14 examples/s]
Tokenizing train dataset:  25%|██▍       | 445525/1807941 [06:25<20:25, 1111.62 examples/s]
Tokenizing train dataset:  25%|██▍       | 445647/1807941 [06:25<19:57, 1137.93 examples/s]
Tokenizing train dataset:  25%|██▍       | 445803/1807941 [06:25<20:39, 1099.05 examples/s]
Tokenizing train dataset:  25%|██▍       | 445946/1807941 [06:25<21:47, 1041.37 examples/s]
Tokenizing train dataset:  25%|██▍       | 446057/1807941 [06:25<28:13, 804.19 examples/s] 
Tokenizing train dataset:  25%|██▍       | 446164/1807941 [06:25<26:27, 858.06 examples/s]
Tokenizing train dataset:  25%|██▍       | 446311/1807941 [06:26<25:21, 894.65 examples/s]
Tokenizing train dataset:  25%|██▍       | 446437/1807941 [06:26<25:54, 876.07 examples/s]
Tokenizing train dataset:  25%|██▍       | 446538/1807941 [06:26<25:09, 902.13 examples/s]
Tokenizing train dataset:  25%|██▍       | 446662/1807941 [06:26<25:55, 875.21 examples/s]
Tokenizing train dataset:  25%|██▍       | 446773/1807941 [06:26<24:24, 929.46 examples/s]
Tokenizing train dataset:  25%|██▍       | 446903/1807941 [06:26<22:12, 1021.46 examples/s]
Tokenizing train dataset:  25%|██▍       | 447057/1807941 [06:26<23:34, 962.20 examples/s] 
Tokenizing train dataset:  25%|██▍       | 447173/1807941 [06:26<22:28, 1008.75 examples/s]
Tokenizing train dataset:  25%|██▍       | 447300/1807941 [06:27<21:10, 1070.55 examples/s]
Tokenizing train dataset:  25%|██▍       | 447417/1807941 [06:27<20:42, 1095.03 examples/s]
Tokenizing train dataset:  25%|██▍       | 447550/1807941 [06:27<22:24, 1011.98 examples/s]
Tokenizing train dataset:  25%|██▍       | 447679/1807941 [06:27<23:43, 955.74 examples/s] 
Tokenizing train dataset:  25%|██▍       | 447793/1807941 [06:27<22:44, 997.10 examples/s]
Tokenizing train dataset:  25%|██▍       | 447936/1807941 [06:27<25:54, 874.74 examples/s]
Tokenizing train dataset:  25%|██▍       | 448065/1807941 [06:27<26:15, 863.27 examples/s]
Tokenizing train dataset:  25%|██▍       | 448173/1807941 [06:28<27:44, 816.79 examples/s]
Tokenizing train dataset:  25%|██▍       | 448291/1807941 [06:28<25:18, 895.10 examples/s]
Tokenizing train dataset:  25%|██▍       | 448413/1807941 [06:28<23:18, 972.36 examples/s]
Tokenizing train dataset:  25%|██▍       | 448528/1807941 [06:28<22:17, 1016.28 examples/s]
Tokenizing train dataset:  25%|██▍       | 448658/1807941 [06:28<20:47, 1090.02 examples/s]
Tokenizing train dataset:  25%|██▍       | 448783/1807941 [06:28<20:00, 1131.80 examples/s]
Tokenizing train dataset:  25%|██▍       | 448935/1807941 [06:28<20:51, 1086.14 examples/s]
Tokenizing train dataset:  25%|██▍       | 449058/1807941 [06:28<22:48, 992.63 examples/s] 
Tokenizing train dataset:  25%|██▍       | 449168/1807941 [06:28<22:14, 1017.81 examples/s]
Tokenizing train dataset:  25%|██▍       | 449323/1807941 [06:29<22:13, 1018.50 examples/s]
Tokenizing train dataset:  25%|██▍       | 449449/1807941 [06:29<21:01, 1076.61 examples/s]
Tokenizing train dataset:  25%|██▍       | 449569/1807941 [06:29<20:27, 1106.60 examples/s]
Tokenizing train dataset:  25%|██▍       | 449692/1807941 [06:29<19:53, 1137.61 examples/s]
Tokenizing train dataset:  25%|██▍       | 449836/1807941 [06:29<21:10, 1068.98 examples/s]
Tokenizing train dataset:  25%|██▍       | 449959/1807941 [06:29<20:25, 1107.98 examples/s]
Tokenizing train dataset:  25%|██▍       | 450122/1807941 [06:29<21:47, 1038.47 examples/s]
Tokenizing train dataset:  25%|██▍       | 450241/1807941 [06:29<21:05, 1072.59 examples/s]
Tokenizing train dataset:  25%|██▍       | 450357/1807941 [06:30<20:42, 1092.23 examples/s]
Tokenizing train dataset:  25%|██▍       | 450472/1807941 [06:30<20:27, 1105.98 examples/s]
Tokenizing train dataset:  25%|██▍       | 450597/1807941 [06:30<19:48, 1142.10 examples/s]
Tokenizing train dataset:  25%|██▍       | 450716/1807941 [06:30<19:37, 1152.18 examples/s]
Tokenizing train dataset:  25%|██▍       | 450893/1807941 [06:30<19:30, 1159.59 examples/s]
Tokenizing train dataset:  25%|██▍       | 451034/1807941 [06:30<21:50, 1035.28 examples/s]
Tokenizing train dataset:  25%|██▍       | 451191/1807941 [06:30<21:51, 1034.84 examples/s]
Tokenizing train dataset:  25%|██▍       | 451309/1807941 [06:30<21:11, 1067.08 examples/s]
Tokenizing train dataset:  25%|██▍       | 451426/1807941 [06:30<20:44, 1090.26 examples/s]
Tokenizing train dataset:  25%|██▍       | 451595/1807941 [06:31<20:36, 1097.14 examples/s]
Tokenizing train dataset:  25%|██▍       | 451715/1807941 [06:31<20:09, 1121.03 examples/s]
Tokenizing train dataset:  25%|██▍       | 451839/1807941 [06:31<19:38, 1150.31 examples/s]
Tokenizing train dataset:  25%|██▍       | 451963/1807941 [06:31<19:15, 1173.26 examples/s]
Tokenizing train dataset:  25%|██▌       | 452122/1807941 [06:31<20:34, 1097.84 examples/s]
Tokenizing train dataset:  25%|██▌       | 452246/1807941 [06:31<19:59, 1130.23 examples/s]
Tokenizing train dataset:  25%|██▌       | 452403/1807941 [06:31<20:38, 1094.33 examples/s]
Tokenizing train dataset:  25%|██▌       | 452546/1807941 [06:32<21:42, 1040.53 examples/s]
Tokenizing train dataset:  25%|██▌       | 452667/1807941 [06:32<20:55, 1079.24 examples/s]
Tokenizing train dataset:  25%|██▌       | 452786/1807941 [06:32<23:02, 979.96 examples/s] 
Tokenizing train dataset:  25%|██▌       | 452931/1807941 [06:32<20:39, 1092.98 examples/s]
Tokenizing train dataset:  25%|██▌       | 453094/1807941 [06:32<20:45, 1088.23 examples/s]
Tokenizing train dataset:  25%|██▌       | 453231/1807941 [06:32<19:34, 1153.63 examples/s]
Tokenizing train dataset:  25%|██▌       | 453365/1807941 [06:32<18:50, 1197.85 examples/s]
Tokenizing train dataset:  25%|██▌       | 453492/1807941 [06:32<18:34, 1215.51 examples/s]
Tokenizing train dataset:  25%|██▌       | 453626/1807941 [06:32<18:05, 1248.01 examples/s]
Tokenizing train dataset:  25%|██▌       | 453779/1807941 [06:33<19:30, 1157.23 examples/s]
Tokenizing train dataset:  25%|██▌       | 453939/1807941 [06:33<20:08, 1120.71 examples/s]
Tokenizing train dataset:  25%|██▌       | 454078/1807941 [06:33<20:07, 1121.08 examples/s]
Tokenizing train dataset:  25%|██▌       | 454211/1807941 [06:33<19:16, 1170.54 examples/s]
Tokenizing train dataset:  25%|██▌       | 454341/1807941 [06:33<18:48, 1199.85 examples/s]
Tokenizing train dataset:  25%|██▌       | 454471/1807941 [06:33<18:25, 1224.46 examples/s]
Tokenizing train dataset:  25%|██▌       | 454598/1807941 [06:33<21:00, 1073.90 examples/s]
Tokenizing train dataset:  25%|██▌       | 454769/1807941 [06:34<23:02, 978.85 examples/s] 
Tokenizing train dataset:  25%|██▌       | 454892/1807941 [06:34<21:48, 1034.03 examples/s]
Tokenizing train dataset:  25%|██▌       | 455064/1807941 [06:34<21:48, 1033.66 examples/s]
Tokenizing train dataset:  25%|██▌       | 455179/1807941 [06:34<26:19, 856.41 examples/s] 
Tokenizing train dataset:  25%|██▌       | 455328/1807941 [06:34<25:13, 893.97 examples/s]
Tokenizing train dataset:  25%|██▌       | 455456/1807941 [06:34<23:07, 974.53 examples/s]
Tokenizing train dataset:  25%|██▌       | 455613/1807941 [06:34<20:17, 1111.08 examples/s]
Tokenizing train dataset:  25%|██▌       | 455792/1807941 [06:34<17:37, 1278.60 examples/s]
Tokenizing train dataset:  25%|██▌       | 455930/1807941 [06:35<17:16, 1304.15 examples/s]
Tokenizing train dataset:  25%|██▌       | 456079/1807941 [06:35<18:56, 1189.54 examples/s]
Tokenizing train dataset:  25%|██▌       | 456252/1807941 [06:35<19:11, 1173.47 examples/s]
Tokenizing train dataset:  25%|██▌       | 456421/1807941 [06:35<19:30, 1154.69 examples/s]
Tokenizing train dataset:  25%|██▌       | 456541/1807941 [06:35<19:24, 1160.54 examples/s]
Tokenizing train dataset:  25%|██▌       | 456678/1807941 [06:35<20:58, 1073.39 examples/s]
Tokenizing train dataset:  25%|██▌       | 456814/1807941 [06:35<19:45, 1139.48 examples/s]
Tokenizing train dataset:  25%|██▌       | 456982/1807941 [06:35<19:54, 1131.04 examples/s]
Tokenizing train dataset:  25%|██▌       | 457134/1807941 [06:36<20:39, 1089.55 examples/s]
Tokenizing train dataset:  25%|██▌       | 457246/1807941 [06:36<20:33, 1094.97 examples/s]
Tokenizing train dataset:  25%|██▌       | 457376/1807941 [06:36<19:42, 1142.55 examples/s]
Tokenizing train dataset:  25%|██▌       | 457555/1807941 [06:36<19:24, 1159.31 examples/s]
Tokenizing train dataset:  25%|██▌       | 457701/1807941 [06:36<18:15, 1232.33 examples/s]
Tokenizing train dataset:  25%|██▌       | 457835/1807941 [06:36<17:52, 1259.31 examples/s]
Tokenizing train dataset:  25%|██▌       | 457996/1807941 [06:36<19:02, 1181.87 examples/s]
Tokenizing train dataset:  25%|██▌       | 458140/1807941 [06:36<20:23, 1103.56 examples/s]
Tokenizing train dataset:  25%|██▌       | 458263/1807941 [06:37<19:51, 1132.88 examples/s]
Tokenizing train dataset:  25%|██▌       | 458386/1807941 [06:37<19:27, 1156.06 examples/s]
Tokenizing train dataset:  25%|██▌       | 458557/1807941 [06:37<19:36, 1146.99 examples/s]
Tokenizing train dataset:  25%|██▌       | 458690/1807941 [06:37<18:53, 1189.91 examples/s]
Tokenizing train dataset:  25%|██▌       | 458818/1807941 [06:37<18:33, 1211.33 examples/s]
Tokenizing train dataset:  25%|██▌       | 458973/1807941 [06:37<19:38, 1144.93 examples/s]
Tokenizing train dataset:  25%|██▌       | 459125/1807941 [06:37<20:28, 1097.67 examples/s]
Tokenizing train dataset:  25%|██▌       | 459247/1807941 [06:37<19:57, 1126.17 examples/s]
Tokenizing train dataset:  25%|██▌       | 459379/1807941 [06:38<19:09, 1172.95 examples/s]
Tokenizing train dataset:  25%|██▌       | 459515/1807941 [06:38<18:23, 1222.31 examples/s]
Tokenizing train dataset:  25%|██▌       | 459653/1807941 [06:38<17:47, 1262.48 examples/s]
Tokenizing train dataset:  25%|██▌       | 459845/1807941 [06:38<17:46, 1264.51 examples/s]
Tokenizing train dataset:  25%|██▌       | 460000/1807941 [06:38<22:04, 1017.93 examples/s]
Tokenizing train dataset:  25%|██▌       | 460113/1807941 [06:38<21:34, 1041.08 examples/s]
Tokenizing train dataset:  25%|██▌       | 460263/1807941 [06:38<21:54, 1024.88 examples/s]
Tokenizing train dataset:  25%|██▌       | 460386/1807941 [06:38<20:57, 1071.91 examples/s]
Tokenizing train dataset:  25%|██▌       | 460566/1807941 [06:39<20:16, 1107.18 examples/s]
Tokenizing train dataset:  25%|██▌       | 460686/1807941 [06:39<19:54, 1127.66 examples/s]
Tokenizing train dataset:  25%|██▌       | 460855/1807941 [06:39<19:55, 1126.35 examples/s]
Tokenizing train dataset:  25%|██▌       | 461000/1807941 [06:39<21:12, 1058.36 examples/s]
Tokenizing train dataset:  26%|██▌       | 461145/1807941 [06:39<21:51, 1026.60 examples/s]
Tokenizing train dataset:  26%|██▌       | 461264/1807941 [06:39<21:07, 1062.31 examples/s]
Tokenizing train dataset:  26%|██▌       | 461380/1807941 [06:39<20:43, 1082.58 examples/s]
Tokenizing train dataset:  26%|██▌       | 461505/1807941 [06:39<19:57, 1124.02 examples/s]
Tokenizing train dataset:  26%|██▌       | 461669/1807941 [06:40<20:09, 1112.71 examples/s]
Tokenizing train dataset:  26%|██▌       | 461805/1807941 [06:40<19:06, 1174.47 examples/s]
Tokenizing train dataset:  26%|██▌       | 461927/1807941 [06:40<18:58, 1182.15 examples/s]
Tokenizing train dataset:  26%|██▌       | 462066/1807941 [06:40<20:40, 1085.07 examples/s]
Tokenizing train dataset:  26%|██▌       | 462232/1807941 [06:40<20:35, 1089.26 examples/s]
Tokenizing train dataset:  26%|██▌       | 462353/1807941 [06:40<20:05, 1115.90 examples/s]
Tokenizing train dataset:  26%|██▌       | 462477/1807941 [06:40<19:33, 1146.78 examples/s]
Tokenizing train dataset:  26%|██▌       | 462596/1807941 [06:40<19:25, 1153.98 examples/s]
Tokenizing train dataset:  26%|██▌       | 462766/1807941 [06:41<19:39, 1140.59 examples/s]
Tokenizing train dataset:  26%|██▌       | 462943/1807941 [06:41<19:28, 1151.32 examples/s]
Tokenizing train dataset:  26%|██▌       | 463114/1807941 [06:41<20:23, 1098.99 examples/s]
Tokenizing train dataset:  26%|██▌       | 463235/1807941 [06:41<19:59, 1120.92 examples/s]
Tokenizing train dataset:  26%|██▌       | 463385/1807941 [06:41<20:48, 1077.26 examples/s]
Tokenizing train dataset:  26%|██▌       | 463501/1807941 [06:41<20:27, 1095.69 examples/s]
Tokenizing train dataset:  26%|██▌       | 463666/1807941 [06:41<20:29, 1092.91 examples/s]
Tokenizing train dataset:  26%|██▌       | 463830/1807941 [06:42<20:33, 1089.52 examples/s]
Tokenizing train dataset:  26%|██▌       | 463941/1807941 [06:42<20:32, 1090.54 examples/s]
Tokenizing train dataset:  26%|██▌       | 464056/1807941 [06:42<21:33, 1038.75 examples/s]
Tokenizing train dataset:  26%|██▌       | 464169/1807941 [06:42<21:11, 1057.07 examples/s]
Tokenizing train dataset:  26%|██▌       | 464285/1807941 [06:42<23:36, 948.45 examples/s] 
Tokenizing train dataset:  26%|██▌       | 464420/1807941 [06:42<21:23, 1047.01 examples/s]
Tokenizing train dataset:  26%|██▌       | 464532/1807941 [06:42<21:01, 1064.94 examples/s]
Tokenizing train dataset:  26%|██▌       | 464689/1807941 [06:42<21:13, 1055.18 examples/s]
Tokenizing train dataset:  26%|██▌       | 464799/1807941 [06:43<21:02, 1064.28 examples/s]
Tokenizing train dataset:  26%|██▌       | 464914/1807941 [06:43<20:40, 1082.53 examples/s]
Tokenizing train dataset:  26%|██▌       | 465051/1807941 [06:43<22:00, 1016.91 examples/s]
Tokenizing train dataset:  26%|██▌       | 465157/1807941 [06:43<21:57, 1018.94 examples/s]
Tokenizing train dataset:  26%|██▌       | 465278/1807941 [06:43<20:56, 1068.22 examples/s]
Tokenizing train dataset:  26%|██▌       | 465397/1807941 [06:43<20:22, 1098.50 examples/s]
Tokenizing train dataset:  26%|██▌       | 465545/1807941 [06:43<21:15, 1052.30 examples/s]
Tokenizing train dataset:  26%|██▌       | 465672/1807941 [06:43<20:14, 1105.25 examples/s]
Tokenizing train dataset:  26%|██▌       | 465792/1807941 [06:43<19:49, 1128.39 examples/s]
Tokenizing train dataset:  26%|██▌       | 465915/1807941 [06:44<19:23, 1152.98 examples/s]
Tokenizing train dataset:  26%|██▌       | 466064/1807941 [06:44<20:37, 1084.15 examples/s]
Tokenizing train dataset:  26%|██▌       | 466187/1807941 [06:44<19:59, 1118.57 examples/s]
Tokenizing train dataset:  26%|██▌       | 466354/1807941 [06:44<20:05, 1113.31 examples/s]
Tokenizing train dataset:  26%|██▌       | 466467/1807941 [06:44<20:03, 1114.28 examples/s]
Tokenizing train dataset:  26%|██▌       | 466590/1807941 [06:44<19:34, 1142.03 examples/s]
Tokenizing train dataset:  26%|██▌       | 466709/1807941 [06:44<19:24, 1152.12 examples/s]
Tokenizing train dataset:  26%|██▌       | 466835/1807941 [06:44<18:57, 1178.74 examples/s]
Tokenizing train dataset:  26%|██▌       | 466975/1807941 [06:44<18:02, 1238.36 examples/s]
Tokenizing train dataset:  26%|██▌       | 467113/1807941 [06:45<20:01, 1115.70 examples/s]
Tokenizing train dataset:  26%|██▌       | 467279/1807941 [06:45<20:08, 1109.82 examples/s]
Tokenizing train dataset:  26%|██▌       | 467446/1807941 [06:45<20:08, 1109.39 examples/s]
Tokenizing train dataset:  26%|██▌       | 467563/1807941 [06:45<19:57, 1119.22 examples/s]
Tokenizing train dataset:  26%|██▌       | 467708/1807941 [06:45<20:59, 1064.24 examples/s]
Tokenizing train dataset:  26%|██▌       | 467845/1807941 [06:45<22:07, 1009.32 examples/s]
Tokenizing train dataset:  26%|██▌       | 467961/1807941 [06:45<21:24, 1043.31 examples/s]
Tokenizing train dataset:  26%|██▌       | 468077/1807941 [06:46<24:37, 906.81 examples/s] 
Tokenizing train dataset:  26%|██▌       | 468224/1807941 [06:46<24:09, 924.36 examples/s]
Tokenizing train dataset:  26%|██▌       | 468340/1807941 [06:46<22:53, 975.62 examples/s]
Tokenizing train dataset:  26%|██▌       | 468459/1807941 [06:46<21:43, 1027.32 examples/s]
Tokenizing train dataset:  26%|██▌       | 468577/1807941 [06:46<20:58, 1064.00 examples/s]
Tokenizing train dataset:  26%|██▌       | 468697/1807941 [06:46<20:19, 1097.88 examples/s]
Tokenizing train dataset:  26%|██▌       | 468819/1807941 [06:46<22:35, 988.02 examples/s] 
Tokenizing train dataset:  26%|██▌       | 468935/1807941 [06:46<24:34, 908.10 examples/s]
Tokenizing train dataset:  26%|██▌       | 469045/1807941 [06:47<33:52, 658.64 examples/s]
Tokenizing train dataset:  26%|██▌       | 469150/1807941 [06:47<36:26, 612.19 examples/s]
Tokenizing train dataset:  26%|██▌       | 469241/1807941 [06:47<33:28, 666.52 examples/s]
Tokenizing train dataset:  26%|██▌       | 469339/1807941 [06:47<30:39, 727.86 examples/s]
Tokenizing train dataset:  26%|██▌       | 469431/1807941 [06:47<28:55, 771.30 examples/s]
Tokenizing train dataset:  26%|██▌       | 469535/1807941 [06:47<26:44, 834.38 examples/s]
Tokenizing train dataset:  26%|██▌       | 469641/1807941 [06:47<25:03, 890.07 examples/s]
Tokenizing train dataset:  26%|██▌       | 469752/1807941 [06:48<23:31, 947.83 examples/s]
Tokenizing train dataset:  26%|██▌       | 469854/1807941 [06:48<23:09, 962.94 examples/s]
Tokenizing train dataset:  26%|██▌       | 469966/1807941 [06:48<22:11, 1005.02 examples/s]
Tokenizing train dataset:  26%|██▌       | 470111/1807941 [06:48<23:21, 954.66 examples/s] 
Tokenizing train dataset:  26%|██▌       | 470220/1807941 [06:48<22:36, 986.42 examples/s]
Tokenizing train dataset:  26%|██▌       | 470335/1807941 [06:48<21:41, 1028.03 examples/s]
Tokenizing train dataset:  26%|██▌       | 470491/1807941 [06:48<21:40, 1028.13 examples/s]
Tokenizing train dataset:  26%|██▌       | 470655/1807941 [06:48<21:18, 1046.18 examples/s]
Tokenizing train dataset:  26%|██▌       | 470775/1807941 [06:49<25:53, 860.75 examples/s] 
Tokenizing train dataset:  26%|██▌       | 470888/1807941 [06:49<26:56, 827.26 examples/s]
Tokenizing train dataset:  26%|██▌       | 470982/1807941 [06:49<26:16, 847.83 examples/s]
Tokenizing train dataset:  26%|██▌       | 471094/1807941 [06:49<27:27, 811.49 examples/s]
Tokenizing train dataset:  26%|██▌       | 471212/1807941 [06:49<27:44, 802.91 examples/s]
Tokenizing train dataset:  26%|██▌       | 471319/1807941 [06:49<25:52, 861.10 examples/s]
Tokenizing train dataset:  26%|██▌       | 471411/1807941 [06:49<25:27, 875.17 examples/s]
Tokenizing train dataset:  26%|██▌       | 471508/1807941 [06:49<24:54, 894.51 examples/s]
Tokenizing train dataset:  26%|██▌       | 471650/1807941 [06:50<24:33, 906.59 examples/s]
Tokenizing train dataset:  26%|██▌       | 471770/1807941 [06:50<28:43, 775.11 examples/s]
Tokenizing train dataset:  26%|██▌       | 471863/1807941 [06:50<30:44, 724.40 examples/s]
Tokenizing train dataset:  26%|██▌       | 471965/1807941 [06:50<31:30, 706.70 examples/s]
Tokenizing train dataset:  26%|██▌       | 472076/1807941 [06:50<32:59, 674.92 examples/s]
Tokenizing train dataset:  26%|██▌       | 472164/1807941 [06:50<34:31, 644.83 examples/s]
Tokenizing train dataset:  26%|██▌       | 472230/1807941 [06:51<34:26, 646.28 examples/s]
Tokenizing train dataset:  26%|██▌       | 472326/1807941 [06:51<34:47, 639.87 examples/s]
Tokenizing train dataset:  26%|██▌       | 472419/1807941 [06:51<35:28, 627.42 examples/s]
Tokenizing train dataset:  26%|██▌       | 472509/1807941 [06:51<36:18, 613.06 examples/s]
Tokenizing train dataset:  26%|██▌       | 472588/1807941 [06:51<37:58, 586.15 examples/s]
Tokenizing train dataset:  26%|██▌       | 472676/1807941 [06:51<38:09, 583.12 examples/s]
Tokenizing train dataset:  26%|██▌       | 472754/1807941 [06:51<39:39, 561.16 examples/s]
Tokenizing train dataset:  26%|██▌       | 472817/1807941 [06:52<39:03, 569.72 examples/s]
Tokenizing train dataset:  26%|██▌       | 472881/1807941 [06:52<38:17, 580.98 examples/s]
Tokenizing train dataset:  26%|██▌       | 472959/1807941 [06:52<40:00, 556.20 examples/s]
Tokenizing train dataset:  26%|██▌       | 473022/1807941 [06:52<46:20, 480.17 examples/s]
Tokenizing train dataset:  26%|██▌       | 473080/1807941 [06:52<44:31, 499.61 examples/s]
Tokenizing train dataset:  26%|██▌       | 473137/1807941 [06:52<43:35, 510.27 examples/s]
Tokenizing train dataset:  26%|██▌       | 473206/1807941 [06:52<45:16, 491.42 examples/s]
Tokenizing train dataset:  26%|██▌       | 473278/1807941 [06:53<45:46, 485.92 examples/s]
Tokenizing train dataset:  26%|██▌       | 473336/1807941 [06:53<43:49, 507.54 examples/s]
Tokenizing train dataset:  26%|██▌       | 473414/1807941 [06:53<44:08, 503.93 examples/s]
Tokenizing train dataset:  26%|██▌       | 473487/1807941 [06:53<44:50, 496.07 examples/s]
Tokenizing train dataset:  26%|██▌       | 473544/1807941 [06:53<43:29, 511.45 examples/s]
Tokenizing train dataset:  26%|██▌       | 473604/1807941 [06:53<42:02, 528.89 examples/s]
Tokenizing train dataset:  26%|██▌       | 473679/1807941 [06:53<43:46, 507.91 examples/s]
Tokenizing train dataset:  26%|██▌       | 473743/1807941 [06:53<41:25, 536.84 examples/s]
Tokenizing train dataset:  26%|██▌       | 473822/1807941 [06:54<42:04, 528.51 examples/s]
Tokenizing train dataset:  26%|██▌       | 473905/1807941 [06:54<42:01, 529.05 examples/s]
Tokenizing train dataset:  26%|██▌       | 473968/1807941 [06:54<40:18, 551.57 examples/s]
Tokenizing train dataset:  26%|██▌       | 474032/1807941 [06:54<43:21, 512.78 examples/s]
Tokenizing train dataset:  26%|██▌       | 474095/1807941 [06:54<46:49, 474.68 examples/s]
Tokenizing train dataset:  26%|██▌       | 474150/1807941 [06:54<50:48, 437.48 examples/s]
Tokenizing train dataset:  26%|██▌       | 474213/1807941 [06:54<46:24, 479.05 examples/s]
Tokenizing train dataset:  26%|██▌       | 474285/1807941 [06:55<46:51, 474.34 examples/s]
Tokenizing train dataset:  26%|██▌       | 474340/1807941 [06:55<45:26, 489.16 examples/s]
Tokenizing train dataset:  26%|██▌       | 474405/1807941 [06:55<42:24, 524.08 examples/s]
Tokenizing train dataset:  26%|██▌       | 474473/1807941 [06:55<39:31, 562.33 examples/s]
Tokenizing train dataset:  26%|██▌       | 474538/1807941 [06:55<37:57, 585.41 examples/s]
Tokenizing train dataset:  26%|██▋       | 474609/1807941 [06:55<36:09, 614.51 examples/s]
Tokenizing train dataset:  26%|██▋       | 474681/1807941 [06:55<34:30, 643.92 examples/s]
Tokenizing train dataset:  26%|██▋       | 474783/1807941 [06:55<34:13, 649.06 examples/s]
Tokenizing train dataset:  26%|██▋       | 474870/1807941 [06:55<35:49, 620.11 examples/s]
Tokenizing train dataset:  26%|██▋       | 474953/1807941 [06:56<37:11, 597.32 examples/s]
Tokenizing train dataset:  26%|██▋       | 475028/1807941 [06:56<41:39, 533.37 examples/s]
Tokenizing train dataset:  26%|██▋       | 475095/1807941 [06:56<39:26, 563.25 examples/s]
Tokenizing train dataset:  26%|██▋       | 475160/1807941 [06:56<42:48, 518.90 examples/s]
Tokenizing train dataset:  26%|██▋       | 475219/1807941 [06:56<41:35, 534.06 examples/s]
Tokenizing train dataset:  26%|██▋       | 475297/1807941 [06:56<42:09, 526.90 examples/s]
Tokenizing train dataset:  26%|██▋       | 475369/1807941 [06:56<39:03, 568.52 examples/s]
Tokenizing train dataset:  26%|██▋       | 475437/1807941 [06:57<37:22, 594.19 examples/s]
Tokenizing train dataset:  26%|██▋       | 475502/1807941 [06:57<36:49, 603.14 examples/s]
Tokenizing train dataset:  26%|██▋       | 475584/1807941 [06:57<38:14, 580.71 examples/s]
Tokenizing train dataset:  26%|██▋       | 475652/1807941 [06:57<36:40, 605.37 examples/s]
Tokenizing train dataset:  26%|██▋       | 475729/1807941 [06:57<39:24, 563.46 examples/s]
Tokenizing train dataset:  26%|██▋       | 475812/1807941 [06:57<39:53, 556.46 examples/s]
Tokenizing train dataset:  26%|██▋       | 475870/1807941 [06:57<39:38, 560.05 examples/s]
Tokenizing train dataset:  26%|██▋       | 475942/1807941 [06:57<42:09, 526.52 examples/s]
Tokenizing train dataset:  26%|██▋       | 476000/1807941 [06:58<44:46, 495.80 examples/s]
Tokenizing train dataset:  26%|██▋       | 476065/1807941 [06:58<52:11, 425.34 examples/s]
Tokenizing train dataset:  26%|██▋       | 476129/1807941 [06:58<52:50, 420.06 examples/s]
Tokenizing train dataset:  26%|██▋       | 476191/1807941 [06:58<53:29, 414.90 examples/s]
Tokenizing train dataset:  26%|██▋       | 476237/1807941 [06:58<52:29, 422.83 examples/s]
Tokenizing train dataset:  26%|██▋       | 476286/1807941 [06:58<50:52, 436.29 examples/s]
Tokenizing train dataset:  26%|██▋       | 476339/1807941 [06:58<48:30, 457.51 examples/s]
Tokenizing train dataset:  26%|██▋       | 476389/1807941 [06:58<47:50, 463.94 examples/s]
Tokenizing train dataset:  26%|██▋       | 476459/1807941 [06:59<42:11, 525.86 examples/s]
Tokenizing train dataset:  26%|██▋       | 476515/1807941 [06:59<42:03, 527.67 examples/s]
Tokenizing train dataset:  26%|██▋       | 476591/1807941 [06:59<42:52, 517.61 examples/s]
Tokenizing train dataset:  26%|██▋       | 476651/1807941 [06:59<41:28, 535.00 examples/s]
Tokenizing train dataset:  26%|██▋       | 476711/1807941 [06:59<40:17, 550.69 examples/s]
Tokenizing train dataset:  26%|██▋       | 476783/1807941 [06:59<42:40, 519.84 examples/s]
Tokenizing train dataset:  26%|██▋       | 476852/1807941 [06:59<45:00, 492.91 examples/s]
Tokenizing train dataset:  26%|██▋       | 476920/1807941 [07:00<46:50, 473.66 examples/s]
Tokenizing train dataset:  26%|██▋       | 476985/1807941 [07:00<48:26, 457.87 examples/s]
Tokenizing train dataset:  26%|██▋       | 477044/1807941 [07:00<55:17, 401.13 examples/s]
Tokenizing train dataset:  26%|██▋       | 477094/1807941 [07:00<53:08, 417.34 examples/s]
Tokenizing train dataset:  26%|██▋       | 477154/1807941 [07:00<54:21, 408.09 examples/s]
Tokenizing train dataset:  26%|██▋       | 477204/1807941 [07:00<52:17, 424.13 examples/s]
Tokenizing train dataset:  26%|██▋       | 477261/1807941 [07:00<54:47, 404.77 examples/s]
Tokenizing train dataset:  26%|██▋       | 477310/1807941 [07:00<52:21, 423.61 examples/s]
Tokenizing train dataset:  26%|██▋       | 477354/1807941 [07:01<51:52, 427.48 examples/s]
Tokenizing train dataset:  26%|██▋       | 477423/1807941 [07:01<50:54, 435.57 examples/s]
Tokenizing train dataset:  26%|██▋       | 477488/1807941 [07:01<51:21, 431.75 examples/s]
Tokenizing train dataset:  26%|██▋       | 477538/1807941 [07:01<49:58, 443.63 examples/s]
Tokenizing train dataset:  26%|██▋       | 477585/1807941 [07:01<49:31, 447.66 examples/s]
Tokenizing train dataset:  26%|██▋       | 477642/1807941 [07:01<46:19, 478.54 examples/s]
Tokenizing train dataset:  26%|██▋       | 477698/1807941 [07:01<44:32, 497.82 examples/s]
Tokenizing train dataset:  26%|██▋       | 477760/1807941 [07:01<42:15, 524.62 examples/s]
Tokenizing train dataset:  26%|██▋       | 477814/1807941 [07:02<42:14, 524.74 examples/s]
Tokenizing train dataset:  26%|██▋       | 477874/1807941 [07:02<46:31, 476.51 examples/s]
Tokenizing train dataset:  26%|██▋       | 477946/1807941 [07:02<47:03, 471.06 examples/s]
Tokenizing train dataset:  26%|██▋       | 478000/1807941 [07:02<52:09, 424.95 examples/s]
Tokenizing train dataset:  26%|██▋       | 478103/1807941 [07:02<39:07, 566.41 examples/s]
Tokenizing train dataset:  26%|██▋       | 478184/1807941 [07:02<39:55, 555.06 examples/s]
Tokenizing train dataset:  26%|██▋       | 478249/1807941 [07:02<38:33, 574.81 examples/s]
Tokenizing train dataset:  26%|██▋       | 478317/1807941 [07:02<37:07, 597.03 examples/s]
Tokenizing train dataset:  26%|██▋       | 478403/1807941 [07:03<38:01, 582.74 examples/s]
Tokenizing train dataset:  26%|██▋       | 478471/1807941 [07:03<36:42, 603.55 examples/s]
Tokenizing train dataset:  26%|██▋       | 478567/1807941 [07:03<36:25, 608.29 examples/s]
Tokenizing train dataset:  26%|██▋       | 478633/1807941 [07:03<35:59, 615.48 examples/s]
Tokenizing train dataset:  26%|██▋       | 478712/1807941 [07:03<38:18, 578.33 examples/s]
Tokenizing train dataset:  26%|██▋       | 478780/1807941 [07:03<36:54, 600.33 examples/s]
Tokenizing train dataset:  26%|██▋       | 478848/1807941 [07:03<35:49, 618.31 examples/s]
Tokenizing train dataset:  26%|██▋       | 478916/1807941 [07:03<35:00, 632.84 examples/s]
Tokenizing train dataset:  26%|██▋       | 478987/1807941 [07:04<38:58, 568.21 examples/s]
Tokenizing train dataset:  26%|██▋       | 479061/1807941 [07:04<47:29, 466.28 examples/s]
Tokenizing train dataset:  27%|██▋       | 479139/1807941 [07:04<46:32, 475.79 examples/s]
Tokenizing train dataset:  27%|██▋       | 479203/1807941 [07:04<43:27, 509.50 examples/s]
Tokenizing train dataset:  27%|██▋       | 479283/1807941 [07:04<43:05, 513.89 examples/s]
Tokenizing train dataset:  27%|██▋       | 479345/1807941 [07:04<41:29, 533.65 examples/s]
Tokenizing train dataset:  27%|██▋       | 479406/1807941 [07:04<40:25, 547.78 examples/s]
Tokenizing train dataset:  27%|██▋       | 479471/1807941 [07:05<38:43, 571.79 examples/s]
Tokenizing train dataset:  27%|██▋       | 479530/1807941 [07:05<38:28, 575.32 examples/s]
Tokenizing train dataset:  27%|██▋       | 479602/1807941 [07:05<36:17, 609.95 examples/s]
Tokenizing train dataset:  27%|██▋       | 479666/1807941 [07:05<35:58, 615.50 examples/s]
Tokenizing train dataset:  27%|██▋       | 479733/1807941 [07:05<35:11, 628.95 examples/s]
Tokenizing train dataset:  27%|██▋       | 479808/1807941 [07:05<38:45, 571.01 examples/s]
Tokenizing train dataset:  27%|██▋       | 479895/1807941 [07:05<38:39, 572.62 examples/s]
Tokenizing train dataset:  27%|██▋       | 479959/1807941 [07:05<37:58, 582.80 examples/s]
Tokenizing train dataset:  27%|██▋       | 480022/1807941 [07:06<43:20, 510.64 examples/s]
Tokenizing train dataset:  27%|██▋       | 480093/1807941 [07:06<39:45, 556.68 examples/s]
Tokenizing train dataset:  27%|██▋       | 480181/1807941 [07:06<39:48, 555.92 examples/s]
Tokenizing train dataset:  27%|██▋       | 480269/1807941 [07:06<39:39, 557.94 examples/s]
Tokenizing train dataset:  27%|██▋       | 480356/1807941 [07:06<39:36, 558.74 examples/s]
Tokenizing train dataset:  27%|██▋       | 480441/1807941 [07:06<39:30, 560.08 examples/s]
Tokenizing train dataset:  27%|██▋       | 480505/1807941 [07:06<38:40, 572.03 examples/s]
Tokenizing train dataset:  27%|██▋       | 480581/1807941 [07:06<40:46, 542.47 examples/s]
Tokenizing train dataset:  27%|██▋       | 480641/1807941 [07:07<40:18, 548.74 examples/s]
Tokenizing train dataset:  27%|██▋       | 480725/1807941 [07:07<40:10, 550.51 examples/s]
Tokenizing train dataset:  27%|██▋       | 480799/1807941 [07:07<37:16, 593.32 examples/s]
Tokenizing train dataset:  27%|██▋       | 480860/1807941 [07:07<37:04, 596.52 examples/s]
Tokenizing train dataset:  27%|██▋       | 480947/1807941 [07:07<37:41, 586.72 examples/s]
Tokenizing train dataset:  27%|██▋       | 481030/1807941 [07:07<43:26, 509.11 examples/s]
Tokenizing train dataset:  27%|██▋       | 481090/1807941 [07:07<41:53, 527.79 examples/s]
Tokenizing train dataset:  27%|██▋       | 481171/1807941 [07:08<42:05, 525.32 examples/s]
Tokenizing train dataset:  27%|██▋       | 481237/1807941 [07:08<39:49, 555.20 examples/s]
Tokenizing train dataset:  27%|██▋       | 481325/1807941 [07:08<39:21, 561.88 examples/s]
Tokenizing train dataset:  27%|██▋       | 481392/1807941 [07:08<37:48, 584.70 examples/s]
Tokenizing train dataset:  27%|██▋       | 481458/1807941 [07:08<36:39, 603.00 examples/s]
Tokenizing train dataset:  27%|██▋       | 481527/1807941 [07:08<35:21, 625.36 examples/s]
Tokenizing train dataset:  27%|██▋       | 481597/1807941 [07:08<34:25, 642.27 examples/s]
Tokenizing train dataset:  27%|██▋       | 481688/1807941 [07:08<35:28, 622.95 examples/s]
Tokenizing train dataset:  27%|██▋       | 481794/1807941 [07:09<34:06, 648.02 examples/s]
Tokenizing train dataset:  27%|██▋       | 481882/1807941 [07:09<35:30, 622.44 examples/s]
Tokenizing train dataset:  27%|██▋       | 481950/1807941 [07:09<39:13, 563.38 examples/s]
Tokenizing train dataset:  27%|██▋       | 482023/1807941 [07:09<41:42, 529.85 examples/s]
Tokenizing train dataset:  27%|██▋       | 482084/1807941 [07:09<45:03, 490.38 examples/s]
Tokenizing train dataset:  27%|██▋       | 482159/1807941 [07:09<40:20, 547.69 examples/s]
Tokenizing train dataset:  27%|██▋       | 482252/1807941 [07:09<34:34, 638.93 examples/s]
Tokenizing train dataset:  27%|██▋       | 482337/1807941 [07:09<32:00, 690.27 examples/s]
Tokenizing train dataset:  27%|██▋       | 482421/1807941 [07:10<30:18, 728.98 examples/s]
Tokenizing train dataset:  27%|██▋       | 482510/1807941 [07:10<28:40, 770.24 examples/s]
Tokenizing train dataset:  27%|██▋       | 482599/1807941 [07:10<27:34, 800.95 examples/s]
Tokenizing train dataset:  27%|██▋       | 482682/1807941 [07:10<27:19, 808.29 examples/s]
Tokenizing train dataset:  27%|██▋       | 482796/1807941 [07:10<28:22, 778.20 examples/s]
Tokenizing train dataset:  27%|██▋       | 482886/1807941 [07:10<27:24, 805.51 examples/s]
Tokenizing train dataset:  27%|██▋       | 483000/1807941 [07:10<29:56, 737.57 examples/s]
Tokenizing train dataset:  27%|██▋       | 483107/1807941 [07:10<30:21, 727.18 examples/s]
Tokenizing train dataset:  27%|██▋       | 483189/1807941 [07:11<29:30, 748.26 examples/s]
Tokenizing train dataset:  27%|██▋       | 483271/1807941 [07:11<28:49, 765.79 examples/s]
Tokenizing train dataset:  27%|██▋       | 483384/1807941 [07:11<29:09, 756.94 examples/s]
Tokenizing train dataset:  27%|██▋       | 483498/1807941 [07:11<29:14, 755.00 examples/s]
Tokenizing train dataset:  27%|██▋       | 483602/1807941 [07:11<26:56, 819.10 examples/s]
Tokenizing train dataset:  27%|██▋       | 483694/1807941 [07:11<26:10, 843.24 examples/s]
Tokenizing train dataset:  27%|██▋       | 483816/1807941 [07:11<26:38, 828.52 examples/s]
Tokenizing train dataset:  27%|██▋       | 483934/1807941 [07:11<27:09, 812.45 examples/s]
Tokenizing train dataset:  27%|██▋       | 484047/1807941 [07:12<28:39, 769.75 examples/s]
Tokenizing train dataset:  27%|██▋       | 484132/1807941 [07:12<28:07, 784.59 examples/s]
Tokenizing train dataset:  27%|██▋       | 484221/1807941 [07:12<27:26, 804.03 examples/s]
Tokenizing train dataset:  27%|██▋       | 484319/1807941 [07:12<25:58, 849.21 examples/s]
Tokenizing train dataset:  27%|██▋       | 484440/1807941 [07:12<26:44, 825.09 examples/s]
Tokenizing train dataset:  27%|██▋       | 484568/1807941 [07:12<26:36, 829.13 examples/s]
Tokenizing train dataset:  27%|██▋       | 484664/1807941 [07:12<25:45, 856.21 examples/s]
Tokenizing train dataset:  27%|██▋       | 484757/1807941 [07:12<25:13, 874.28 examples/s]
Tokenizing train dataset:  27%|██▋       | 484869/1807941 [07:13<26:40, 826.86 examples/s]
Tokenizing train dataset:  27%|██▋       | 484994/1807941 [07:13<26:42, 825.74 examples/s]
Tokenizing train dataset:  27%|██▋       | 485095/1807941 [07:13<28:33, 772.07 examples/s]
Tokenizing train dataset:  27%|██▋       | 485179/1807941 [07:13<28:25, 775.52 examples/s]
Tokenizing train dataset:  27%|██▋       | 485313/1807941 [07:13<27:09, 811.89 examples/s]
Tokenizing train dataset:  27%|██▋       | 485435/1807941 [07:13<27:18, 807.34 examples/s]
Tokenizing train dataset:  27%|██▋       | 485521/1807941 [07:13<26:57, 817.33 examples/s]
Tokenizing train dataset:  27%|██▋       | 485608/1807941 [07:14<26:42, 825.10 examples/s]
Tokenizing train dataset:  27%|██▋       | 485701/1807941 [07:14<26:01, 846.67 examples/s]
Tokenizing train dataset:  27%|██▋       | 485816/1807941 [07:14<27:01, 815.55 examples/s]
Tokenizing train dataset:  27%|██▋       | 485909/1807941 [07:14<26:14, 839.66 examples/s]
Tokenizing train dataset:  27%|██▋       | 486003/1807941 [07:14<28:55, 761.78 examples/s]
Tokenizing train dataset:  27%|██▋       | 486114/1807941 [07:14<29:14, 753.47 examples/s]
Tokenizing train dataset:  27%|██▋       | 486215/1807941 [07:14<27:05, 813.33 examples/s]
Tokenizing train dataset:  27%|██▋       | 486326/1807941 [07:14<28:16, 779.21 examples/s]
Tokenizing train dataset:  27%|██▋       | 486411/1807941 [07:15<27:47, 792.73 examples/s]
Tokenizing train dataset:  27%|██▋       | 486493/1807941 [07:15<27:34, 798.90 examples/s]
Tokenizing train dataset:  27%|██▋       | 486584/1807941 [07:15<26:49, 820.79 examples/s]
Tokenizing train dataset:  27%|██▋       | 486670/1807941 [07:15<26:32, 829.90 examples/s]
Tokenizing train dataset:  27%|██▋       | 486785/1807941 [07:15<27:29, 800.93 examples/s]
Tokenizing train dataset:  27%|██▋       | 486904/1807941 [07:15<27:55, 788.36 examples/s]
Tokenizing train dataset:  27%|██▋       | 486992/1807941 [07:15<27:12, 809.35 examples/s]
Tokenizing train dataset:  27%|██▋       | 487107/1807941 [07:15<31:09, 706.68 examples/s]
Tokenizing train dataset:  27%|██▋       | 487192/1807941 [07:16<30:01, 733.32 examples/s]
Tokenizing train dataset:  27%|██▋       | 487309/1807941 [07:16<29:30, 745.71 examples/s]
Tokenizing train dataset:  27%|██▋       | 487417/1807941 [07:16<29:58, 734.26 examples/s]
Tokenizing train dataset:  27%|██▋       | 487525/1807941 [07:16<30:25, 723.37 examples/s]
Tokenizing train dataset:  27%|██▋       | 487608/1807941 [07:16<29:32, 745.02 examples/s]
Tokenizing train dataset:  27%|██▋       | 487687/1807941 [07:16<29:18, 750.80 examples/s]
Tokenizing train dataset:  27%|██▋       | 487799/1807941 [07:16<29:27, 747.09 examples/s]
Tokenizing train dataset:  27%|██▋       | 487876/1807941 [07:16<29:16, 751.59 examples/s]
Tokenizing train dataset:  27%|██▋       | 487999/1807941 [07:17<28:25, 774.13 examples/s]
Tokenizing train dataset:  27%|██▋       | 488080/1807941 [07:17<31:41, 693.95 examples/s]
Tokenizing train dataset:  27%|██▋       | 488180/1807941 [07:17<32:10, 683.62 examples/s]
Tokenizing train dataset:  27%|██▋       | 488265/1807941 [07:17<30:31, 720.59 examples/s]
Tokenizing train dataset:  27%|██▋       | 488364/1807941 [07:17<31:41, 693.85 examples/s]
Tokenizing train dataset:  27%|██▋       | 488471/1807941 [07:17<31:32, 697.18 examples/s]
Tokenizing train dataset:  27%|██▋       | 488576/1807941 [07:17<31:31, 697.54 examples/s]
Tokenizing train dataset:  27%|██▋       | 488656/1807941 [07:18<30:37, 718.12 examples/s]
Tokenizing train dataset:  27%|██▋       | 488738/1807941 [07:18<29:42, 740.08 examples/s]
Tokenizing train dataset:  27%|██▋       | 488822/1807941 [07:18<28:46, 764.19 examples/s]
Tokenizing train dataset:  27%|██▋       | 488902/1807941 [07:18<28:43, 765.47 examples/s]
Tokenizing train dataset:  27%|██▋       | 488987/1807941 [07:18<27:55, 787.08 examples/s]
Tokenizing train dataset:  27%|██▋       | 489076/1807941 [07:18<30:50, 712.80 examples/s]
Tokenizing train dataset:  27%|██▋       | 489182/1807941 [07:18<31:13, 703.89 examples/s]
Tokenizing train dataset:  27%|██▋       | 489269/1807941 [07:18<29:40, 740.75 examples/s]
Tokenizing train dataset:  27%|██▋       | 489360/1807941 [07:18<28:04, 782.67 examples/s]
Tokenizing train dataset:  27%|██▋       | 489461/1807941 [07:19<26:05, 842.43 examples/s]
Tokenizing train dataset:  27%|██▋       | 489558/1807941 [07:19<25:06, 875.18 examples/s]
Tokenizing train dataset:  27%|██▋       | 489661/1807941 [07:19<27:24, 801.46 examples/s]
Tokenizing train dataset:  27%|██▋       | 489751/1807941 [07:19<26:37, 825.09 examples/s]
Tokenizing train dataset:  27%|██▋       | 489853/1807941 [07:19<25:06, 875.15 examples/s]
Tokenizing train dataset:  27%|██▋       | 489944/1807941 [07:19<24:58, 879.39 examples/s]
Tokenizing train dataset:  27%|██▋       | 490050/1807941 [07:19<27:10, 808.29 examples/s]
Tokenizing train dataset:  27%|██▋       | 490136/1807941 [07:19<26:50, 818.11 examples/s]
Tokenizing train dataset:  27%|██▋       | 490230/1807941 [07:20<25:55, 847.30 examples/s]
Tokenizing train dataset:  27%|██▋       | 490344/1807941 [07:20<26:58, 814.19 examples/s]
Tokenizing train dataset:  27%|██▋       | 490428/1807941 [07:20<26:50, 818.28 examples/s]
Tokenizing train dataset:  27%|██▋       | 490512/1807941 [07:20<26:46, 819.90 examples/s]
Tokenizing train dataset:  27%|██▋       | 490636/1807941 [07:20<26:45, 820.63 examples/s]
Tokenizing train dataset:  27%|██▋       | 490725/1807941 [07:20<26:14, 836.43 examples/s]
Tokenizing train dataset:  27%|██▋       | 490843/1807941 [07:20<26:57, 814.41 examples/s]
Tokenizing train dataset:  27%|██▋       | 490950/1807941 [07:20<28:10, 778.94 examples/s]
Tokenizing train dataset:  27%|██▋       | 491036/1807941 [07:21<30:59, 708.30 examples/s]
Tokenizing train dataset:  27%|██▋       | 491114/1807941 [07:21<30:23, 722.07 examples/s]
Tokenizing train dataset:  27%|██▋       | 491225/1807941 [07:21<30:32, 718.61 examples/s]
Tokenizing train dataset:  27%|██▋       | 491337/1807941 [07:21<30:15, 725.13 examples/s]
Tokenizing train dataset:  27%|██▋       | 491438/1807941 [07:21<31:04, 706.10 examples/s]
Tokenizing train dataset:  27%|██▋       | 491521/1807941 [07:21<30:07, 728.27 examples/s]
Tokenizing train dataset:  27%|██▋       | 491601/1807941 [07:21<29:32, 742.77 examples/s]
Tokenizing train dataset:  27%|██▋       | 491679/1807941 [07:21<29:24, 745.85 examples/s]
Tokenizing train dataset:  27%|██▋       | 491764/1807941 [07:22<28:31, 769.22 examples/s]
Tokenizing train dataset:  27%|██▋       | 491889/1807941 [07:22<27:50, 788.02 examples/s]
Tokenizing train dataset:  27%|██▋       | 491981/1807941 [07:22<26:50, 817.04 examples/s]
Tokenizing train dataset:  27%|██▋       | 492086/1807941 [07:22<28:45, 762.45 examples/s]
Tokenizing train dataset:  27%|██▋       | 492171/1807941 [07:22<28:00, 782.84 examples/s]
Tokenizing train dataset:  27%|██▋       | 492252/1807941 [07:22<27:51, 787.24 examples/s]
Tokenizing train dataset:  27%|██▋       | 492371/1807941 [07:22<27:58, 783.96 examples/s]
Tokenizing train dataset:  27%|██▋       | 492493/1807941 [07:22<27:43, 790.60 examples/s]
Tokenizing train dataset:  27%|██▋       | 492592/1807941 [07:23<29:26, 744.44 examples/s]
Tokenizing train dataset:  27%|██▋       | 492670/1807941 [07:23<29:09, 751.98 examples/s]
Tokenizing train dataset:  27%|██▋       | 492755/1807941 [07:23<28:16, 775.19 examples/s]
Tokenizing train dataset:  27%|██▋       | 492840/1807941 [07:23<27:40, 792.21 examples/s]
Tokenizing train dataset:  27%|██▋       | 492924/1807941 [07:23<27:19, 802.25 examples/s]
Tokenizing train dataset:  27%|██▋       | 493006/1807941 [07:23<31:01, 706.49 examples/s]
Tokenizing train dataset:  27%|██▋       | 493103/1807941 [07:23<28:24, 771.35 examples/s]
Tokenizing train dataset:  27%|██▋       | 493208/1807941 [07:23<29:26, 744.09 examples/s]
Tokenizing train dataset:  27%|██▋       | 493296/1807941 [07:24<28:13, 776.43 examples/s]
Tokenizing train dataset:  27%|██▋       | 493386/1807941 [07:24<27:10, 806.25 examples/s]
Tokenizing train dataset:  27%|██▋       | 493480/1807941 [07:24<26:10, 837.18 examples/s]
Tokenizing train dataset:  27%|██▋       | 493608/1807941 [07:24<26:08, 837.78 examples/s]
Tokenizing train dataset:  27%|██▋       | 493700/1807941 [07:24<25:34, 856.74 examples/s]
Tokenizing train dataset:  27%|██▋       | 493809/1807941 [07:24<27:05, 808.66 examples/s]
Tokenizing train dataset:  27%|██▋       | 493892/1807941 [07:24<27:08, 806.67 examples/s]
Tokenizing train dataset:  27%|██▋       | 493982/1807941 [07:24<26:26, 828.08 examples/s]
Tokenizing train dataset:  27%|██▋       | 494074/1807941 [07:24<29:46, 735.32 examples/s]
Tokenizing train dataset:  27%|██▋       | 494160/1807941 [07:25<28:35, 765.83 examples/s]
Tokenizing train dataset:  27%|██▋       | 494270/1807941 [07:25<25:51, 846.76 examples/s]
Tokenizing train dataset:  27%|██▋       | 494360/1807941 [07:25<25:30, 858.06 examples/s]
Tokenizing train dataset:  27%|██▋       | 494450/1807941 [07:25<25:15, 866.95 examples/s]
Tokenizing train dataset:  27%|██▋       | 494564/1807941 [07:25<26:36, 822.53 examples/s]
Tokenizing train dataset:  27%|██▋       | 494673/1807941 [07:25<28:08, 777.93 examples/s]
Tokenizing train dataset:  27%|██▋       | 494753/1807941 [07:25<28:08, 777.52 examples/s]
Tokenizing train dataset:  27%|██▋       | 494846/1807941 [07:25<26:58, 811.31 examples/s]
Tokenizing train dataset:  27%|██▋       | 494940/1807941 [07:26<25:54, 844.58 examples/s]
Tokenizing train dataset:  27%|██▋       | 495053/1807941 [07:26<27:09, 805.77 examples/s]
Tokenizing train dataset:  27%|██▋       | 495141/1807941 [07:26<26:36, 822.42 examples/s]
Tokenizing train dataset:  27%|██▋       | 495229/1807941 [07:26<26:16, 832.56 examples/s]
Tokenizing train dataset:  27%|██▋       | 495358/1807941 [07:26<26:02, 840.04 examples/s]
Tokenizing train dataset:  27%|██▋       | 495479/1807941 [07:26<26:32, 824.22 examples/s]
Tokenizing train dataset:  27%|██▋       | 495592/1807941 [07:26<27:28, 796.01 examples/s]
Tokenizing train dataset:  27%|██▋       | 495717/1807941 [07:26<27:16, 802.07 examples/s]
Tokenizing train dataset:  27%|██▋       | 495803/1807941 [07:27<26:59, 810.29 examples/s]
Tokenizing train dataset:  27%|██▋       | 495928/1807941 [07:27<26:58, 810.48 examples/s]
Tokenizing train dataset:  27%|██▋       | 496031/1807941 [07:27<29:44, 735.09 examples/s]
Tokenizing train dataset:  27%|██▋       | 496113/1807941 [07:27<29:20, 745.24 examples/s]
Tokenizing train dataset:  27%|██▋       | 496229/1807941 [07:27<29:11, 749.00 examples/s]
Tokenizing train dataset:  27%|██▋       | 496316/1807941 [07:27<28:10, 775.99 examples/s]
Tokenizing train dataset:  27%|██▋       | 496401/1807941 [07:27<27:33, 793.16 examples/s]
Tokenizing train dataset:  27%|██▋       | 496488/1807941 [07:27<26:56, 811.07 examples/s]
Tokenizing train dataset:  27%|██▋       | 496613/1807941 [07:28<26:51, 813.87 examples/s]
Tokenizing train dataset:  27%|██▋       | 496727/1807941 [07:28<27:34, 792.48 examples/s]
Tokenizing train dataset:  27%|██▋       | 496845/1807941 [07:28<27:56, 782.26 examples/s]
Tokenizing train dataset:  27%|██▋       | 496927/1807941 [07:28<27:38, 790.53 examples/s]
Tokenizing train dataset:  27%|██▋       | 497041/1807941 [07:28<28:24, 768.97 examples/s]
Tokenizing train dataset:  27%|██▋       | 497131/1807941 [07:28<27:33, 792.72 examples/s]
Tokenizing train dataset:  28%|██▊       | 497212/1807941 [07:28<27:29, 794.48 examples/s]
Tokenizing train dataset:  28%|██▊       | 497303/1807941 [07:28<26:35, 821.51 examples/s]
Tokenizing train dataset:  28%|██▊       | 497396/1807941 [07:29<29:17, 745.81 examples/s]
Tokenizing train dataset:  28%|██▊       | 497488/1807941 [07:29<27:57, 781.33 examples/s]
Tokenizing train dataset:  28%|██▊       | 497580/1807941 [07:29<26:51, 813.35 examples/s]
Tokenizing train dataset:  28%|██▊       | 497703/1807941 [07:29<27:01, 808.08 examples/s]
Tokenizing train dataset:  28%|██▊       | 497824/1807941 [07:29<27:11, 803.13 examples/s]
Tokenizing train dataset:  28%|██▊       | 497942/1807941 [07:29<27:33, 792.22 examples/s]
Tokenizing train dataset:  28%|██▊       | 498052/1807941 [07:29<28:19, 770.93 examples/s]
Tokenizing train dataset:  28%|██▊       | 498134/1807941 [07:30<28:02, 778.37 examples/s]
Tokenizing train dataset:  28%|██▊       | 498225/1807941 [07:30<26:59, 808.66 examples/s]
Tokenizing train dataset:  28%|██▊       | 498344/1807941 [07:30<27:14, 801.40 examples/s]
Tokenizing train dataset:  28%|██▊       | 498433/1807941 [07:30<26:33, 821.56 examples/s]
Tokenizing train dataset:  28%|██▊       | 498543/1807941 [07:30<27:57, 780.66 examples/s]
Tokenizing train dataset:  28%|██▊       | 498629/1807941 [07:30<27:18, 799.21 examples/s]
Tokenizing train dataset:  28%|██▊       | 498719/1807941 [07:30<26:31, 822.88 examples/s]
Tokenizing train dataset:  28%|██▊       | 498806/1807941 [07:30<26:22, 827.38 examples/s]
Tokenizing train dataset:  28%|██▊       | 498904/1807941 [07:30<25:13, 864.84 examples/s]
Tokenizing train dataset:  28%|██▊       | 498999/1807941 [07:31<24:39, 884.88 examples/s]
Tokenizing train dataset:  28%|██▊       | 499100/1807941 [07:31<27:10, 802.63 examples/s]
Tokenizing train dataset:  28%|██▊       | 499190/1807941 [07:31<26:26, 824.84 examples/s]
Tokenizing train dataset:  28%|██▊       | 499304/1807941 [07:31<27:29, 793.51 examples/s]
Tokenizing train dataset:  28%|██▊       | 499434/1807941 [07:31<26:44, 815.37 examples/s]
Tokenizing train dataset:  28%|██▊       | 499519/1807941 [07:31<26:31, 822.26 examples/s]
Tokenizing train dataset:  28%|██▊       | 499634/1807941 [07:31<27:15, 800.06 examples/s]
Tokenizing train dataset:  28%|██▊       | 499757/1807941 [07:32<27:06, 804.52 examples/s]
Tokenizing train dataset:  28%|██▊       | 499841/1807941 [07:32<26:52, 811.18 examples/s]
Tokenizing train dataset:  28%|██▊       | 499926/1807941 [07:32<26:35, 820.07 examples/s]
Tokenizing train dataset:  28%|██▊       | 500041/1807941 [07:32<29:24, 741.38 examples/s]
Tokenizing train dataset:  28%|██▊       | 500122/1807941 [07:32<28:52, 754.87 examples/s]
Tokenizing train dataset:  28%|██▊       | 500201/1807941 [07:32<28:37, 761.55 examples/s]
Tokenizing train dataset:  28%|██▊       | 500313/1807941 [07:32<28:58, 752.05 examples/s]
Tokenizing train dataset:  28%|██▊       | 500392/1807941 [07:32<28:51, 755.06 examples/s]
Tokenizing train dataset:  28%|██▊       | 500491/1807941 [07:32<26:47, 813.52 examples/s]
Tokenizing train dataset:  28%|██▊       | 500605/1807941 [07:33<27:35, 789.69 examples/s]
Tokenizing train dataset:  28%|██▊       | 500730/1807941 [07:33<27:12, 800.59 examples/s]
Tokenizing train dataset:  28%|██▊       | 500847/1807941 [07:33<27:35, 789.46 examples/s]
Tokenizing train dataset:  28%|██▊       | 500938/1807941 [07:33<26:53, 810.15 examples/s]
Tokenizing train dataset:  28%|██▊       | 501047/1807941 [07:33<28:34, 762.17 examples/s]
Tokenizing train dataset:  28%|██▊       | 501126/1807941 [07:33<28:26, 765.99 examples/s]
Tokenizing train dataset:  28%|██▊       | 501232/1807941 [07:33<29:18, 743.07 examples/s]
Tokenizing train dataset:  28%|██▊       | 501334/1807941 [07:34<30:20, 717.57 examples/s]
Tokenizing train dataset:  28%|██▊       | 501418/1807941 [07:34<29:17, 743.27 examples/s]
Tokenizing train dataset:  28%|██▊       | 501530/1807941 [07:34<29:18, 742.84 examples/s]
Tokenizing train dataset:  28%|██▊       | 501633/1807941 [07:34<30:06, 723.03 examples/s]
Tokenizing train dataset:  28%|██▊       | 501736/1807941 [07:34<30:49, 706.22 examples/s]
Tokenizing train dataset:  28%|██▊       | 501808/1807941 [07:34<30:46, 707.38 examples/s]
Tokenizing train dataset:  28%|██▊       | 501885/1807941 [07:34<30:15, 719.57 examples/s]
Tokenizing train dataset:  28%|██▊       | 501966/1807941 [07:34<29:24, 740.03 examples/s]
Tokenizing train dataset:  28%|██▊       | 502072/1807941 [07:35<32:09, 676.93 examples/s]
Tokenizing train dataset:  28%|██▊       | 502145/1807941 [07:35<31:42, 686.42 examples/s]
Tokenizing train dataset:  28%|██▊       | 502220/1807941 [07:35<31:17, 695.62 examples/s]
Tokenizing train dataset:  28%|██▊       | 502301/1807941 [07:35<30:18, 717.78 examples/s]
Tokenizing train dataset:  28%|██▊       | 502389/1807941 [07:35<28:37, 760.09 examples/s]
Tokenizing train dataset:  28%|██▊       | 502494/1807941 [07:35<29:55, 726.90 examples/s]
Tokenizing train dataset:  28%|██▊       | 502605/1807941 [07:35<29:58, 725.97 examples/s]
Tokenizing train dataset:  28%|██▊       | 502679/1807941 [07:35<29:54, 727.52 examples/s]
Tokenizing train dataset:  28%|██▊       | 502785/1807941 [07:36<30:18, 717.64 examples/s]
Tokenizing train dataset:  28%|██▊       | 502878/1807941 [07:36<28:23, 765.97 examples/s]
Tokenizing train dataset:  28%|██▊       | 502964/1807941 [07:36<27:36, 788.00 examples/s]
Tokenizing train dataset:  28%|██▊       | 503074/1807941 [07:36<30:25, 714.62 examples/s]
Tokenizing train dataset:  28%|██▊       | 503157/1807941 [07:36<29:27, 738.02 examples/s]
Tokenizing train dataset:  28%|██▊       | 503234/1807941 [07:36<29:11, 744.89 examples/s]
Tokenizing train dataset:  28%|██▊       | 503321/1807941 [07:36<28:11, 771.19 examples/s]
Tokenizing train dataset:  28%|██▊       | 503407/1807941 [07:36<27:20, 795.27 examples/s]
Tokenizing train dataset:  28%|██▊       | 503512/1807941 [07:37<28:44, 756.55 examples/s]
Tokenizing train dataset:  28%|██▊       | 503612/1807941 [07:37<30:04, 722.74 examples/s]
Tokenizing train dataset:  28%|██▊       | 503703/1807941 [07:37<32:01, 678.61 examples/s]
Tokenizing train dataset:  28%|██▊       | 503797/1807941 [07:37<32:53, 660.81 examples/s]
Tokenizing train dataset:  28%|██▊       | 503872/1807941 [07:37<35:53, 605.46 examples/s]
Tokenizing train dataset:  28%|██▊       | 503951/1807941 [07:37<37:38, 577.40 examples/s]
Tokenizing train dataset:  28%|██▊       | 504039/1807941 [07:38<39:45, 546.59 examples/s]
Tokenizing train dataset:  28%|██▊       | 504095/1807941 [07:38<39:47, 546.01 examples/s]
Tokenizing train dataset:  28%|██▊       | 504173/1807941 [07:38<40:30, 536.33 examples/s]
Tokenizing train dataset:  28%|██▊       | 504253/1807941 [07:38<40:47, 532.66 examples/s]
Tokenizing train dataset:  28%|██▊       | 504326/1807941 [07:38<42:22, 512.83 examples/s]
Tokenizing train dataset:  28%|██▊       | 504401/1807941 [07:38<38:44, 560.84 examples/s]
Tokenizing train dataset:  28%|██▊       | 504485/1807941 [07:38<38:59, 557.16 examples/s]
Tokenizing train dataset:  28%|██▊       | 504551/1807941 [07:38<37:32, 578.64 examples/s]
Tokenizing train dataset:  28%|██▊       | 504616/1807941 [07:39<37:02, 586.47 examples/s]
Tokenizing train dataset:  28%|██▊       | 504683/1807941 [07:39<36:11, 600.29 examples/s]
Tokenizing train dataset:  28%|██▊       | 504754/1807941 [07:39<34:36, 627.54 examples/s]
Tokenizing train dataset:  28%|██▊       | 504819/1807941 [07:39<34:26, 630.65 examples/s]
Tokenizing train dataset:  28%|██▊       | 504905/1807941 [07:39<35:53, 604.96 examples/s]
Tokenizing train dataset:  28%|██▊       | 504998/1807941 [07:39<35:44, 607.52 examples/s]
Tokenizing train dataset:  28%|██▊       | 505074/1807941 [07:39<38:06, 569.76 examples/s]
Tokenizing train dataset:  28%|██▊       | 505135/1807941 [07:39<37:55, 572.56 examples/s]
Tokenizing train dataset:  28%|██▊       | 505221/1807941 [07:40<38:02, 570.71 examples/s]
Tokenizing train dataset:  28%|██▊       | 505280/1807941 [07:40<37:46, 574.68 examples/s]
Tokenizing train dataset:  28%|██▊       | 505344/1807941 [07:40<36:53, 588.54 examples/s]
Tokenizing train dataset:  28%|██▊       | 505415/1807941 [07:40<35:02, 619.53 examples/s]
Tokenizing train dataset:  28%|██▊       | 505479/1807941 [07:40<34:54, 621.81 examples/s]
Tokenizing train dataset:  28%|██▊       | 505555/1807941 [07:40<33:06, 655.61 examples/s]
Tokenizing train dataset:  28%|██▊       | 505643/1807941 [07:40<34:43, 624.99 examples/s]
Tokenizing train dataset:  28%|██▊       | 505716/1807941 [07:40<33:47, 642.38 examples/s]
Tokenizing train dataset:  28%|██▊       | 505794/1807941 [07:40<32:11, 674.31 examples/s]
Tokenizing train dataset:  28%|██▊       | 505874/1807941 [07:41<35:00, 619.81 examples/s]
Tokenizing train dataset:  28%|██▊       | 505946/1807941 [07:41<33:44, 643.10 examples/s]
Tokenizing train dataset:  28%|██▊       | 506025/1807941 [07:41<39:03, 555.52 examples/s]
Tokenizing train dataset:  28%|██▊       | 506104/1807941 [07:41<35:45, 606.82 examples/s]
Tokenizing train dataset:  28%|██▊       | 506182/1807941 [07:41<33:36, 645.52 examples/s]
Tokenizing train dataset:  28%|██▊       | 506256/1807941 [07:41<36:56, 587.31 examples/s]
Tokenizing train dataset:  28%|██▊       | 506320/1807941 [07:41<41:05, 527.97 examples/s]
Tokenizing train dataset:  28%|██▊       | 506392/1807941 [07:42<47:36, 455.58 examples/s]
Tokenizing train dataset:  28%|██▊       | 506460/1807941 [07:42<48:03, 451.29 examples/s]
Tokenizing train dataset:  28%|██▊       | 506529/1807941 [07:42<48:28, 447.45 examples/s]
Tokenizing train dataset:  28%|██▊       | 506579/1807941 [07:42<47:28, 456.93 examples/s]
Tokenizing train dataset:  28%|██▊       | 506633/1807941 [07:42<46:05, 470.57 examples/s]
Tokenizing train dataset:  28%|██▊       | 506698/1807941 [07:42<42:12, 513.84 examples/s]
Tokenizing train dataset:  28%|██▊       | 506776/1807941 [07:42<37:21, 580.39 examples/s]
Tokenizing train dataset:  28%|██▊       | 506847/1807941 [07:42<35:22, 613.06 examples/s]
Tokenizing train dataset:  28%|██▊       | 506912/1807941 [07:43<34:52, 621.81 examples/s]
Tokenizing train dataset:  28%|██▊       | 506976/1807941 [07:43<34:41, 624.93 examples/s]
Tokenizing train dataset:  28%|██▊       | 507041/1807941 [07:43<37:42, 574.95 examples/s]
Tokenizing train dataset:  28%|██▊       | 507120/1807941 [07:43<34:24, 630.21 examples/s]
Tokenizing train dataset:  28%|██▊       | 507190/1807941 [07:43<33:32, 646.36 examples/s]
Tokenizing train dataset:  28%|██▊       | 507297/1807941 [07:43<32:44, 662.02 examples/s]
Tokenizing train dataset:  28%|██▊       | 507375/1807941 [07:43<31:38, 684.96 examples/s]
Tokenizing train dataset:  28%|██▊       | 507459/1807941 [07:43<30:03, 721.02 examples/s]
Tokenizing train dataset:  28%|██▊       | 507561/1807941 [07:43<31:01, 698.58 examples/s]
Tokenizing train dataset:  28%|██▊       | 507662/1807941 [07:44<31:37, 685.29 examples/s]
Tokenizing train dataset:  28%|██▊       | 507746/1807941 [07:44<33:55, 638.67 examples/s]
Tokenizing train dataset:  28%|██▊       | 507811/1807941 [07:44<34:10, 634.04 examples/s]
Tokenizing train dataset:  28%|██▊       | 507894/1807941 [07:44<36:06, 600.18 examples/s]
Tokenizing train dataset:  28%|██▊       | 507966/1807941 [07:44<34:38, 625.32 examples/s]
Tokenizing train dataset:  28%|██▊       | 508034/1807941 [07:44<37:22, 579.67 examples/s]
Tokenizing train dataset:  28%|██▊       | 508105/1807941 [07:44<40:33, 534.13 examples/s]
Tokenizing train dataset:  28%|██▊       | 508175/1807941 [07:45<42:25, 510.57 examples/s]
Tokenizing train dataset:  28%|██▊       | 508229/1807941 [07:45<41:59, 515.78 examples/s]
Tokenizing train dataset:  28%|██▊       | 508285/1807941 [07:45<41:14, 525.23 examples/s]
Tokenizing train dataset:  28%|██▊       | 508361/1807941 [07:45<42:27, 510.13 examples/s]
Tokenizing train dataset:  28%|██▊       | 508441/1807941 [07:45<42:04, 514.70 examples/s]
Tokenizing train dataset:  28%|██▊       | 508496/1807941 [07:45<41:54, 516.73 examples/s]
Tokenizing train dataset:  28%|██▊       | 508558/1807941 [07:45<40:20, 536.88 examples/s]
Tokenizing train dataset:  28%|██▊       | 508614/1807941 [07:45<39:53, 542.85 examples/s]
Tokenizing train dataset:  28%|██▊       | 508673/1807941 [07:46<39:18, 550.97 examples/s]
Tokenizing train dataset:  28%|██▊       | 508751/1807941 [07:46<40:41, 532.07 examples/s]
Tokenizing train dataset:  28%|██▊       | 508809/1807941 [07:46<40:24, 535.80 examples/s]
Tokenizing train dataset:  28%|██▊       | 508866/1807941 [07:46<39:48, 543.79 examples/s]
Tokenizing train dataset:  28%|██▊       | 508944/1807941 [07:46<41:01, 527.76 examples/s]
Tokenizing train dataset:  28%|██▊       | 509000/1807941 [07:46<45:45, 473.05 examples/s]
Tokenizing train dataset:  28%|██▊       | 509050/1807941 [07:46<45:14, 478.50 examples/s]
Tokenizing train dataset:  28%|██▊       | 509100/1807941 [07:46<45:29, 475.86 examples/s]
Tokenizing train dataset:  28%|██▊       | 509173/1807941 [07:47<46:15, 468.01 examples/s]
Tokenizing train dataset:  28%|██▊       | 509226/1807941 [07:47<45:22, 477.05 examples/s]
Tokenizing train dataset:  28%|██▊       | 509295/1807941 [07:47<46:16, 467.65 examples/s]
Tokenizing train dataset:  28%|██▊       | 509360/1807941 [07:47<48:24, 447.04 examples/s]
Tokenizing train dataset:  28%|██▊       | 509411/1807941 [07:47<47:14, 458.12 examples/s]
Tokenizing train dataset:  28%|██▊       | 509467/1807941 [07:47<50:49, 425.74 examples/s]
Tokenizing train dataset:  28%|██▊       | 509511/1807941 [07:47<50:34, 427.82 examples/s]
Tokenizing train dataset:  28%|██▊       | 509561/1807941 [07:47<48:55, 442.34 examples/s]
Tokenizing train dataset:  28%|██▊       | 509611/1807941 [07:48<47:34, 454.82 examples/s]
Tokenizing train dataset:  28%|██▊       | 509659/1807941 [07:48<47:29, 455.67 examples/s]
Tokenizing train dataset:  28%|██▊       | 509725/1807941 [07:48<48:40, 444.58 examples/s]
Tokenizing train dataset:  28%|██▊       | 509771/1807941 [07:48<48:56, 442.08 examples/s]
Tokenizing train dataset:  28%|██▊       | 509822/1807941 [07:48<47:26, 456.00 examples/s]
Tokenizing train dataset:  28%|██▊       | 509888/1807941 [07:48<48:33, 445.54 examples/s]
Tokenizing train dataset:  28%|██▊       | 509938/1807941 [07:48<47:32, 455.01 examples/s]
Tokenizing train dataset:  28%|██▊       | 509996/1807941 [07:48<44:40, 484.24 examples/s]
Tokenizing train dataset:  28%|██▊       | 510051/1807941 [07:49<51:21, 421.16 examples/s]
Tokenizing train dataset:  28%|██▊       | 510111/1807941 [07:49<46:42, 463.14 examples/s]
Tokenizing train dataset:  28%|██▊       | 510165/1807941 [07:49<44:53, 481.74 examples/s]
Tokenizing train dataset:  28%|██▊       | 510215/1807941 [07:49<44:28, 486.35 examples/s]
Tokenizing train dataset:  28%|██▊       | 510287/1807941 [07:49<44:50, 482.23 examples/s]
Tokenizing train dataset:  28%|██▊       | 510359/1807941 [07:49<45:34, 474.52 examples/s]
Tokenizing train dataset:  28%|██▊       | 510429/1807941 [07:49<46:17, 467.21 examples/s]
Tokenizing train dataset:  28%|██▊       | 510488/1807941 [07:49<48:51, 442.53 examples/s]
Tokenizing train dataset:  28%|██▊       | 510535/1807941 [07:50<48:26, 446.35 examples/s]
Tokenizing train dataset:  28%|██▊       | 510590/1807941 [07:50<46:02, 469.58 examples/s]
Tokenizing train dataset:  28%|██▊       | 510653/1807941 [07:50<42:44, 505.91 examples/s]
Tokenizing train dataset:  28%|██▊       | 510707/1807941 [07:50<42:26, 509.35 examples/s]
Tokenizing train dataset:  28%|██▊       | 510759/1807941 [07:50<42:22, 510.16 examples/s]
Tokenizing train dataset:  28%|██▊       | 510827/1807941 [07:50<44:32, 485.35 examples/s]
Tokenizing train dataset:  28%|██▊       | 510883/1807941 [07:50<43:37, 495.45 examples/s]
Tokenizing train dataset:  28%|██▊       | 510942/1807941 [07:50<41:58, 514.91 examples/s]
Tokenizing train dataset:  28%|██▊       | 510996/1807941 [07:50<41:34, 519.97 examples/s]
Tokenizing train dataset:  28%|██▊       | 511057/1807941 [07:51<47:19, 456.66 examples/s]
Tokenizing train dataset:  28%|██▊       | 511108/1807941 [07:51<46:12, 467.80 examples/s]
Tokenizing train dataset:  28%|██▊       | 511163/1807941 [07:51<44:38, 484.15 examples/s]
Tokenizing train dataset:  28%|██▊       | 511219/1807941 [07:51<43:16, 499.38 examples/s]
Tokenizing train dataset:  28%|██▊       | 511282/1807941 [07:51<46:16, 466.95 examples/s]
Tokenizing train dataset:  28%|██▊       | 511330/1807941 [07:51<46:14, 467.31 examples/s]
Tokenizing train dataset:  28%|██▊       | 511407/1807941 [07:51<45:16, 477.24 examples/s]
Tokenizing train dataset:  28%|██▊       | 511482/1807941 [07:52<45:21, 476.31 examples/s]
Tokenizing train dataset:  28%|██▊       | 511533/1807941 [07:52<44:47, 482.43 examples/s]
Tokenizing train dataset:  28%|██▊       | 511603/1807941 [07:52<45:44, 472.39 examples/s]
Tokenizing train dataset:  28%|██▊       | 511663/1807941 [07:52<43:10, 500.44 examples/s]
Tokenizing train dataset:  28%|██▊       | 511721/1807941 [07:52<41:50, 516.41 examples/s]
Tokenizing train dataset:  28%|██▊       | 511789/1807941 [07:52<43:55, 491.72 examples/s]
Tokenizing train dataset:  28%|██▊       | 511861/1807941 [07:52<44:59, 480.10 examples/s]
Tokenizing train dataset:  28%|██▊       | 511926/1807941 [07:52<46:43, 462.27 examples/s]
Tokenizing train dataset:  28%|██▊       | 511987/1807941 [07:53<48:44, 443.14 examples/s]
Tokenizing train dataset:  28%|██▊       | 512051/1807941 [07:53<53:36, 402.91 examples/s]
Tokenizing train dataset:  28%|██▊       | 512094/1807941 [07:53<53:12, 405.93 examples/s]
Tokenizing train dataset:  28%|██▊       | 512146/1807941 [07:53<50:57, 423.87 examples/s]
Tokenizing train dataset:  28%|██▊       | 512195/1807941 [07:53<49:19, 437.89 examples/s]
Tokenizing train dataset:  28%|██▊       | 512249/1807941 [07:53<46:48, 461.41 examples/s]
Tokenizing train dataset:  28%|██▊       | 512300/1807941 [07:53<45:39, 472.98 examples/s]
Tokenizing train dataset:  28%|██▊       | 512370/1807941 [07:53<46:27, 464.75 examples/s]
Tokenizing train dataset:  28%|██▊       | 512426/1807941 [07:54<50:18, 429.19 examples/s]
Tokenizing train dataset:  28%|██▊       | 512493/1807941 [07:54<44:42, 482.85 examples/s]
Tokenizing train dataset:  28%|██▊       | 512576/1807941 [07:54<37:59, 568.16 examples/s]
Tokenizing train dataset:  28%|██▊       | 512652/1807941 [07:54<39:46, 542.74 examples/s]
Tokenizing train dataset:  28%|██▊       | 512726/1807941 [07:54<41:29, 520.31 examples/s]
Tokenizing train dataset:  28%|██▊       | 512780/1807941 [07:54<46:22, 465.45 examples/s]
Tokenizing train dataset:  28%|██▊       | 512841/1807941 [07:54<48:42, 443.12 examples/s]
Tokenizing train dataset:  28%|██▊       | 512893/1807941 [07:55<52:30, 411.08 examples/s]
Tokenizing train dataset:  28%|██▊       | 512944/1807941 [07:55<50:04, 430.99 examples/s]
Tokenizing train dataset:  28%|██▊       | 512994/1807941 [07:55<48:23, 445.95 examples/s]
Tokenizing train dataset:  28%|██▊       | 513041/1807941 [07:55<57:34, 374.80 examples/s]
Tokenizing train dataset:  28%|██▊       | 513092/1807941 [07:55<53:09, 405.94 examples/s]
Tokenizing train dataset:  28%|██▊       | 513140/1807941 [07:55<51:00, 423.06 examples/s]
Tokenizing train dataset:  28%|██▊       | 513207/1807941 [07:55<50:17, 429.02 examples/s]
Tokenizing train dataset:  28%|██▊       | 513252/1807941 [07:55<49:49, 433.01 examples/s]
Tokenizing train dataset:  28%|██▊       | 513304/1807941 [07:56<47:26, 454.75 examples/s]
Tokenizing train dataset:  28%|██▊       | 513353/1807941 [07:56<47:01, 458.78 examples/s]
Tokenizing train dataset:  28%|██▊       | 513422/1807941 [07:56<47:17, 456.14 examples/s]
Tokenizing train dataset:  28%|██▊       | 513473/1807941 [07:56<46:04, 468.29 examples/s]
Tokenizing train dataset:  28%|██▊       | 513549/1807941 [07:56<45:17, 476.40 examples/s]
Tokenizing train dataset:  28%|██▊       | 513620/1807941 [07:56<45:58, 469.13 examples/s]
Tokenizing train dataset:  28%|██▊       | 513700/1807941 [07:56<44:19, 486.66 examples/s]
Tokenizing train dataset:  28%|██▊       | 513755/1807941 [07:56<43:07, 500.15 examples/s]
Tokenizing train dataset:  28%|██▊       | 513823/1807941 [07:57<45:07, 478.00 examples/s]
Tokenizing train dataset:  28%|██▊       | 513875/1807941 [07:57<45:01, 479.00 examples/s]
Tokenizing train dataset:  28%|██▊       | 513927/1807941 [07:57<44:50, 480.97 examples/s]
Tokenizing train dataset:  28%|██▊       | 513979/1807941 [07:57<44:07, 488.71 examples/s]
Tokenizing train dataset:  28%|██▊       | 514046/1807941 [07:57<52:15, 412.64 examples/s]
Tokenizing train dataset:  28%|██▊       | 514105/1807941 [07:57<48:03, 448.74 examples/s]
Tokenizing train dataset:  28%|██▊       | 514153/1807941 [07:57<47:42, 451.92 examples/s]
Tokenizing train dataset:  28%|██▊       | 514203/1807941 [07:57<46:35, 462.71 examples/s]
Tokenizing train dataset:  28%|██▊       | 514262/1807941 [07:58<43:28, 495.85 examples/s]
Tokenizing train dataset:  28%|██▊       | 514321/1807941 [07:58<41:35, 518.47 examples/s]
Tokenizing train dataset:  28%|██▊       | 514398/1807941 [07:58<42:15, 510.24 examples/s]
Tokenizing train dataset:  28%|██▊       | 514454/1807941 [07:58<41:43, 516.66 examples/s]
Tokenizing train dataset:  28%|██▊       | 514516/1807941 [07:58<39:57, 539.59 examples/s]
Tokenizing train dataset:  28%|██▊       | 514589/1807941 [07:58<36:38, 588.35 examples/s]
Tokenizing train dataset:  28%|██▊       | 514665/1807941 [07:58<44:11, 487.76 examples/s]
Tokenizing train dataset:  28%|██▊       | 514734/1807941 [07:58<40:21, 534.07 examples/s]
Tokenizing train dataset:  28%|██▊       | 514802/1807941 [07:59<42:56, 501.87 examples/s]
Tokenizing train dataset:  28%|██▊       | 514931/1807941 [07:59<35:14, 611.59 examples/s]
Tokenizing train dataset:  28%|██▊       | 515150/1807941 [07:59<22:59, 936.92 examples/s]
Tokenizing train dataset:  29%|██▊       | 515463/1807941 [07:59<14:42, 1464.08 examples/s]
Tokenizing train dataset:  29%|██▊       | 515771/1807941 [07:59<11:28, 1875.45 examples/s]
Tokenizing train dataset:  29%|██▊       | 516056/1807941 [07:59<10:05, 2133.27 examples/s]
Tokenizing train dataset:  29%|██▊       | 516304/1807941 [07:59<12:23, 1737.17 examples/s]
Tokenizing train dataset:  29%|██▊       | 516511/1807941 [07:59<13:16, 1621.00 examples/s]
Tokenizing train dataset:  29%|██▊       | 516707/1807941 [08:00<14:09, 1520.55 examples/s]
Tokenizing train dataset:  29%|██▊       | 516925/1807941 [08:00<14:27, 1487.70 examples/s]
Tokenizing train dataset:  29%|██▊       | 517099/1807941 [08:00<15:31, 1385.74 examples/s]
Tokenizing train dataset:  29%|██▊       | 517249/1807941 [08:00<15:15, 1409.26 examples/s]
Tokenizing train dataset:  29%|██▊       | 517469/1807941 [08:00<15:06, 1423.76 examples/s]
Tokenizing train dataset:  29%|██▊       | 517628/1807941 [08:00<14:44, 1458.89 examples/s]
Tokenizing train dataset:  29%|██▊       | 517851/1807941 [08:00<14:39, 1466.50 examples/s]
Tokenizing train dataset:  29%|██▊       | 518000/1807941 [08:01<15:08, 1420.38 examples/s]
Tokenizing train dataset:  29%|██▊       | 518209/1807941 [08:01<15:15, 1408.29 examples/s]
Tokenizing train dataset:  29%|██▊       | 518373/1807941 [08:01<14:42, 1461.26 examples/s]
Tokenizing train dataset:  29%|██▊       | 518603/1807941 [08:01<14:29, 1482.45 examples/s]
Tokenizing train dataset:  29%|██▊       | 518830/1807941 [08:01<14:25, 1490.20 examples/s]
Tokenizing train dataset:  29%|██▊       | 518997/1807941 [08:01<14:02, 1530.36 examples/s]
Tokenizing train dataset:  29%|██▊       | 519219/1807941 [08:01<14:13, 1510.01 examples/s]
Tokenizing train dataset:  29%|██▊       | 519372/1807941 [08:01<14:11, 1513.14 examples/s]
Tokenizing train dataset:  29%|██▊       | 519579/1807941 [08:02<14:40, 1463.28 examples/s]
Tokenizing train dataset:  29%|██▉       | 519849/1807941 [08:02<12:10, 1762.91 examples/s]
Tokenizing train dataset:  29%|██▉       | 520148/1807941 [08:02<10:39, 2012.35 examples/s]
Tokenizing train dataset:  29%|██▉       | 520423/1807941 [08:02<09:43, 2205.95 examples/s]
Tokenizing train dataset:  29%|██▉       | 520715/1807941 [08:02<08:56, 2398.34 examples/s]
Tokenizing train dataset:  29%|██▉       | 521000/1807941 [08:02<08:44, 2454.63 examples/s]
Tokenizing train dataset:  29%|██▉       | 521285/1807941 [08:02<08:22, 2560.34 examples/s]
Tokenizing train dataset:  29%|██▉       | 521551/1807941 [08:02<08:17, 2587.71 examples/s]
Tokenizing train dataset:  29%|██▉       | 521947/1807941 [08:03<08:14, 2599.61 examples/s]
Tokenizing train dataset:  29%|██▉       | 522287/1807941 [08:03<08:38, 2481.29 examples/s]
Tokenizing train dataset:  29%|██▉       | 522591/1807941 [08:03<08:10, 2618.00 examples/s]
Tokenizing train dataset:  29%|██▉       | 522861/1807941 [08:03<08:07, 2636.34 examples/s]
Tokenizing train dataset:  29%|██▉       | 523198/1807941 [08:03<08:35, 2490.95 examples/s]
Tokenizing train dataset:  29%|██▉       | 523554/1807941 [08:03<08:44, 2447.25 examples/s]
Tokenizing train dataset:  29%|██▉       | 523958/1807941 [08:03<08:28, 2524.57 examples/s]
Tokenizing train dataset:  29%|██▉       | 524221/1807941 [08:03<08:23, 2548.20 examples/s]
Tokenizing train dataset:  29%|██▉       | 524525/1807941 [08:04<08:00, 2671.25 examples/s]
Tokenizing train dataset:  29%|██▉       | 524832/1807941 [08:04<07:42, 2774.80 examples/s]
Tokenizing train dataset:  29%|██▉       | 525249/1807941 [08:04<07:42, 2774.07 examples/s]
Tokenizing train dataset:  29%|██▉       | 525566/1807941 [08:04<07:26, 2873.17 examples/s]
Tokenizing train dataset:  29%|██▉       | 525891/1807941 [08:04<07:11, 2971.32 examples/s]
Tokenizing train dataset:  29%|██▉       | 526337/1807941 [08:04<07:11, 2969.96 examples/s]
Tokenizing train dataset:  29%|██▉       | 526687/1807941 [08:04<07:45, 2749.69 examples/s]
Tokenizing train dataset:  29%|██▉       | 527003/1807941 [08:04<08:25, 2534.84 examples/s]
Tokenizing train dataset:  29%|██▉       | 527349/1807941 [08:05<08:40, 2458.73 examples/s]
Tokenizing train dataset:  29%|██▉       | 527650/1807941 [08:05<09:14, 2307.13 examples/s]
Tokenizing train dataset:  29%|██▉       | 527966/1807941 [08:05<09:31, 2241.22 examples/s]
Tokenizing train dataset:  29%|██▉       | 528250/1807941 [08:05<10:01, 2128.71 examples/s]
Tokenizing train dataset:  29%|██▉       | 528475/1807941 [08:05<09:53, 2155.15 examples/s]
Tokenizing train dataset:  29%|██▉       | 528790/1807941 [08:05<09:59, 2134.49 examples/s]
Tokenizing train dataset:  29%|██▉       | 529056/1807941 [08:05<10:33, 2017.78 examples/s]
Tokenizing train dataset:  29%|██▉       | 529303/1807941 [08:06<11:12, 1901.27 examples/s]
Tokenizing train dataset:  29%|██▉       | 529540/1807941 [08:06<11:50, 1799.18 examples/s]
Tokenizing train dataset:  29%|██▉       | 529795/1807941 [08:06<12:03, 1766.65 examples/s]
Tokenizing train dataset:  29%|██▉       | 530029/1807941 [08:06<11:14, 1893.21 examples/s]
Tokenizing train dataset:  29%|██▉       | 530320/1807941 [08:06<09:58, 2134.57 examples/s]
Tokenizing train dataset:  29%|██▉       | 530600/1807941 [08:06<10:26, 2039.69 examples/s]
Tokenizing train dataset:  29%|██▉       | 530825/1807941 [08:06<12:44, 1670.62 examples/s]
Tokenizing train dataset:  29%|██▉       | 531019/1807941 [08:07<13:46, 1544.70 examples/s]
Tokenizing train dataset:  29%|██▉       | 531214/1807941 [08:07<13:03, 1630.46 examples/s]
Tokenizing train dataset:  29%|██▉       | 531394/1807941 [08:07<14:19, 1484.37 examples/s]
Tokenizing train dataset:  29%|██▉       | 531596/1807941 [08:07<16:19, 1302.44 examples/s]
Tokenizing train dataset:  29%|██▉       | 531774/1807941 [08:07<16:56, 1255.40 examples/s]
Tokenizing train dataset:  29%|██▉       | 531951/1807941 [08:07<17:17, 1230.31 examples/s]
Tokenizing train dataset:  29%|██▉       | 532119/1807941 [08:08<18:21, 1158.73 examples/s]
Tokenizing train dataset:  29%|██▉       | 532290/1807941 [08:08<18:30, 1148.49 examples/s]
Tokenizing train dataset:  29%|██▉       | 532450/1807941 [08:08<18:58, 1120.42 examples/s]
Tokenizing train dataset:  29%|██▉       | 532611/1807941 [08:08<19:19, 1100.31 examples/s]
Tokenizing train dataset:  29%|██▉       | 532753/1807941 [08:08<20:10, 1053.14 examples/s]
Tokenizing train dataset:  29%|██▉       | 532918/1807941 [08:08<17:58, 1181.76 examples/s]
Tokenizing train dataset:  29%|██▉       | 533111/1807941 [08:08<16:16, 1306.16 examples/s]
Tokenizing train dataset:  29%|██▉       | 533287/1807941 [08:08<16:54, 1256.25 examples/s]
Tokenizing train dataset:  30%|██▉       | 533422/1807941 [08:09<16:38, 1276.64 examples/s]
Tokenizing train dataset:  30%|██▉       | 533604/1807941 [08:09<16:56, 1253.45 examples/s]
Tokenizing train dataset:  30%|██▉       | 533785/1807941 [08:09<15:19, 1385.35 examples/s]
Tokenizing train dataset:  30%|██▉       | 533963/1807941 [08:09<16:10, 1313.06 examples/s]
Tokenizing train dataset:  30%|██▉       | 534168/1807941 [08:09<15:58, 1329.54 examples/s]
Tokenizing train dataset:  30%|██▉       | 534351/1807941 [08:09<16:24, 1294.19 examples/s]
Tokenizing train dataset:  30%|██▉       | 534567/1807941 [08:09<15:51, 1337.61 examples/s]
Tokenizing train dataset:  30%|██▉       | 534712/1807941 [08:10<15:38, 1357.13 examples/s]
Tokenizing train dataset:  30%|██▉       | 534884/1807941 [08:10<14:43, 1440.41 examples/s]
Tokenizing train dataset:  30%|██▉       | 535058/1807941 [08:10<16:15, 1304.63 examples/s]
Tokenizing train dataset:  30%|██▉       | 535193/1807941 [08:10<16:15, 1304.85 examples/s]
Tokenizing train dataset:  30%|██▉       | 535374/1807941 [08:10<16:45, 1265.93 examples/s]
Tokenizing train dataset:  30%|██▉       | 535513/1807941 [08:10<16:22, 1294.81 examples/s]
Tokenizing train dataset:  30%|██▉       | 535691/1807941 [08:10<16:57, 1250.18 examples/s]
Tokenizing train dataset:  30%|██▉       | 535885/1807941 [08:10<16:49, 1260.27 examples/s]
Tokenizing train dataset:  30%|██▉       | 536052/1807941 [08:11<18:06, 1170.32 examples/s]
Tokenizing train dataset:  30%|██▉       | 536179/1807941 [08:11<17:48, 1190.51 examples/s]
Tokenizing train dataset:  30%|██▉       | 536343/1807941 [08:11<16:22, 1293.65 examples/s]
Tokenizing train dataset:  30%|██▉       | 536484/1807941 [08:11<18:07, 1169.54 examples/s]
Tokenizing train dataset:  30%|██▉       | 536613/1807941 [08:11<17:41, 1197.42 examples/s]
Tokenizing train dataset:  30%|██▉       | 536761/1807941 [08:11<16:43, 1266.92 examples/s]
Tokenizing train dataset:  30%|██▉       | 536943/1807941 [08:11<17:06, 1238.31 examples/s]
Tokenizing train dataset:  30%|██▉       | 537100/1807941 [08:11<16:45, 1263.88 examples/s]
Tokenizing train dataset:  30%|██▉       | 537248/1807941 [08:12<16:13, 1305.36 examples/s]
Tokenizing train dataset:  30%|██▉       | 537423/1807941 [08:12<14:54, 1420.75 examples/s]
Tokenizing train dataset:  30%|██▉       | 537621/1807941 [08:12<15:21, 1378.89 examples/s]
Tokenizing train dataset:  30%|██▉       | 537851/1807941 [08:12<13:07, 1612.04 examples/s]
Tokenizing train dataset:  30%|██▉       | 538078/1807941 [08:12<11:50, 1786.72 examples/s]
Tokenizing train dataset:  30%|██▉       | 538294/1807941 [08:12<11:12, 1889.18 examples/s]
Tokenizing train dataset:  30%|██▉       | 538492/1807941 [08:12<11:04, 1910.70 examples/s]
Tokenizing train dataset:  30%|██▉       | 538763/1807941 [08:12<11:20, 1866.34 examples/s]
Tokenizing train dataset:  30%|██▉       | 539009/1807941 [08:13<11:51, 1782.57 examples/s]
Tokenizing train dataset:  30%|██▉       | 539289/1807941 [08:13<10:23, 2034.61 examples/s]
Tokenizing train dataset:  30%|██▉       | 539581/1807941 [08:13<10:33, 2001.72 examples/s]
Tokenizing train dataset:  30%|██▉       | 539834/1807941 [08:13<09:54, 2132.12 examples/s]
Tokenizing train dataset:  30%|██▉       | 540075/1807941 [08:13<09:35, 2202.96 examples/s]
Tokenizing train dataset:  30%|██▉       | 540321/1807941 [08:13<09:18, 2270.51 examples/s]
Tokenizing train dataset:  30%|██▉       | 540561/1807941 [08:13<09:10, 2304.28 examples/s]
Tokenizing train dataset:  30%|██▉       | 540796/1807941 [08:13<09:08, 2310.47 examples/s]
Tokenizing train dataset:  30%|██▉       | 541061/1807941 [08:13<10:02, 2101.01 examples/s]
Tokenizing train dataset:  30%|██▉       | 541376/1807941 [08:14<10:03, 2098.47 examples/s]
Tokenizing train dataset:  30%|██▉       | 541679/1807941 [08:14<10:11, 2069.83 examples/s]
Tokenizing train dataset:  30%|██▉       | 541937/1807941 [08:14<09:38, 2189.76 examples/s]
Tokenizing train dataset:  30%|██▉       | 542265/1807941 [08:14<09:38, 2186.98 examples/s]
Tokenizing train dataset:  30%|███       | 542493/1807941 [08:14<09:33, 2206.07 examples/s]
Tokenizing train dataset:  30%|███       | 542733/1807941 [08:14<09:21, 2254.70 examples/s]
Tokenizing train dataset:  30%|███       | 542972/1807941 [08:14<09:12, 2290.09 examples/s]
Tokenizing train dataset:  30%|███       | 543244/1807941 [08:14<09:58, 2112.26 examples/s]
Tokenizing train dataset:  30%|███       | 543530/1807941 [08:15<10:20, 2038.70 examples/s]
Tokenizing train dataset:  30%|███       | 543763/1807941 [08:15<09:59, 2107.13 examples/s]
Tokenizing train dataset:  30%|███       | 543980/1807941 [08:15<09:56, 2119.09 examples/s]
Tokenizing train dataset:  30%|███       | 544263/1807941 [08:15<10:21, 2032.63 examples/s]
Tokenizing train dataset:  30%|███       | 544473/1807941 [08:15<10:17, 2045.69 examples/s]
Tokenizing train dataset:  30%|███       | 544756/1807941 [08:15<11:50, 1777.25 examples/s]
Tokenizing train dataset:  30%|███       | 545000/1807941 [08:15<12:20, 1705.42 examples/s]
Tokenizing train dataset:  30%|███       | 545201/1807941 [08:15<11:52, 1771.08 examples/s]
Tokenizing train dataset:  30%|███       | 545399/1807941 [08:16<11:33, 1820.05 examples/s]
Tokenizing train dataset:  30%|███       | 545626/1807941 [08:16<10:53, 1932.43 examples/s]
Tokenizing train dataset:  30%|███       | 545840/1807941 [08:16<10:35, 1987.18 examples/s]
Tokenizing train dataset:  30%|███       | 546181/1807941 [08:16<10:04, 2088.17 examples/s]
Tokenizing train dataset:  30%|███       | 546393/1807941 [08:16<10:02, 2092.55 examples/s]
Tokenizing train dataset:  30%|███       | 546700/1807941 [08:16<08:56, 2352.56 examples/s]
Tokenizing train dataset:  30%|███       | 547000/1807941 [08:16<08:26, 2487.54 examples/s]
Tokenizing train dataset:  30%|███       | 547303/1807941 [08:16<07:57, 2637.76 examples/s]
Tokenizing train dataset:  30%|███       | 547590/1807941 [08:16<07:46, 2702.10 examples/s]
Tokenizing train dataset:  30%|███       | 547864/1807941 [08:17<07:45, 2708.29 examples/s]
Tokenizing train dataset:  30%|███       | 548137/1807941 [08:17<07:55, 2648.97 examples/s]
Tokenizing train dataset:  30%|███       | 548417/1807941 [08:17<07:48, 2690.05 examples/s]
Tokenizing train dataset:  30%|███       | 548694/1807941 [08:17<07:44, 2710.41 examples/s]
Tokenizing train dataset:  30%|███       | 548981/1807941 [08:17<07:36, 2754.90 examples/s]
Tokenizing train dataset:  30%|███       | 549359/1807941 [08:17<07:52, 2662.40 examples/s]
Tokenizing train dataset:  30%|███       | 549631/1807941 [08:17<07:50, 2676.41 examples/s]
Tokenizing train dataset:  30%|███       | 549910/1807941 [08:17<07:45, 2704.56 examples/s]
Tokenizing train dataset:  30%|███       | 550297/1807941 [08:17<07:54, 2652.16 examples/s]
Tokenizing train dataset:  30%|███       | 550572/1807941 [08:18<07:50, 2674.43 examples/s]
Tokenizing train dataset:  30%|███       | 550844/1807941 [08:18<07:48, 2683.42 examples/s]
Tokenizing train dataset:  30%|███       | 551243/1807941 [08:18<07:50, 2670.42 examples/s]
Tokenizing train dataset:  31%|███       | 551525/1807941 [08:18<07:44, 2707.05 examples/s]
Tokenizing train dataset:  31%|███       | 551800/1807941 [08:18<07:42, 2716.96 examples/s]
Tokenizing train dataset:  31%|███       | 552194/1807941 [08:18<07:48, 2681.50 examples/s]
Tokenizing train dataset:  31%|███       | 552471/1807941 [08:18<07:44, 2701.03 examples/s]
Tokenizing train dataset:  31%|███       | 552748/1807941 [08:18<07:42, 2714.61 examples/s]
Tokenizing train dataset:  31%|███       | 553126/1807941 [08:19<07:57, 2630.31 examples/s]
Tokenizing train dataset:  31%|███       | 553411/1807941 [08:19<07:47, 2684.69 examples/s]
Tokenizing train dataset:  31%|███       | 553712/1807941 [08:19<07:33, 2768.71 examples/s]
Tokenizing train dataset:  31%|███       | 554000/1807941 [08:19<07:38, 2733.37 examples/s]
Tokenizing train dataset:  31%|███       | 554281/1807941 [08:19<07:35, 2753.87 examples/s]
Tokenizing train dataset:  31%|███       | 554569/1807941 [08:19<07:29, 2786.18 examples/s]
Tokenizing train dataset:  31%|███       | 554861/1807941 [08:19<07:24, 2820.44 examples/s]
Tokenizing train dataset:  31%|███       | 555285/1807941 [08:19<07:35, 2747.24 examples/s]
Tokenizing train dataset:  31%|███       | 555577/1807941 [08:19<07:28, 2789.69 examples/s]
Tokenizing train dataset:  31%|███       | 555869/1807941 [08:19<07:23, 2822.89 examples/s]
Tokenizing train dataset:  31%|███       | 556305/1807941 [08:20<07:19, 2850.09 examples/s]
Tokenizing train dataset:  31%|███       | 556738/1807941 [08:20<07:17, 2859.43 examples/s]
Tokenizing train dataset:  31%|███       | 557177/1807941 [08:20<07:14, 2879.14 examples/s]
Tokenizing train dataset:  31%|███       | 557596/1807941 [08:20<07:18, 2850.04 examples/s]
Tokenizing train dataset:  31%|███       | 558000/1807941 [08:20<07:28, 2787.74 examples/s]
Tokenizing train dataset:  31%|███       | 558282/1807941 [08:20<07:27, 2793.02 examples/s]
Tokenizing train dataset:  31%|███       | 558583/1807941 [08:20<07:19, 2844.87 examples/s]
Tokenizing train dataset:  31%|███       | 558891/1807941 [08:21<07:10, 2904.15 examples/s]
Tokenizing train dataset:  31%|███       | 559308/1807941 [08:21<07:17, 2854.79 examples/s]
Tokenizing train dataset:  31%|███       | 559610/1807941 [08:21<07:11, 2893.81 examples/s]
Tokenizing train dataset:  31%|███       | 559915/1807941 [08:21<07:05, 2930.94 examples/s]
Tokenizing train dataset:  31%|███       | 560353/1807941 [08:21<07:06, 2922.21 examples/s]
Tokenizing train dataset:  31%|███       | 560669/1807941 [08:21<06:58, 2979.20 examples/s]
Tokenizing train dataset:  31%|███       | 561083/1807941 [08:21<07:10, 2899.22 examples/s]
Tokenizing train dataset:  31%|███       | 561378/1807941 [08:21<07:08, 2911.33 examples/s]
Tokenizing train dataset:  31%|███       | 561677/1807941 [08:22<07:05, 2930.81 examples/s]
Tokenizing train dataset:  31%|███       | 562000/1807941 [08:22<06:59, 2967.26 examples/s]
Tokenizing train dataset:  31%|███       | 562306/1807941 [08:22<06:56, 2990.21 examples/s]
Tokenizing train dataset:  31%|███       | 562733/1807941 [08:22<07:04, 2933.21 examples/s]
Tokenizing train dataset:  31%|███       | 563119/1807941 [08:22<07:23, 2803.82 examples/s]
Tokenizing train dataset:  31%|███       | 563410/1807941 [08:22<07:20, 2828.15 examples/s]
Tokenizing train dataset:  31%|███       | 563813/1807941 [08:22<07:28, 2774.58 examples/s]
Tokenizing train dataset:  31%|███       | 564171/1807941 [08:22<07:51, 2640.07 examples/s]
Tokenizing train dataset:  31%|███       | 564464/1807941 [08:23<07:39, 2705.55 examples/s]
Tokenizing train dataset:  31%|███       | 564771/1807941 [08:23<07:24, 2796.86 examples/s]
Tokenizing train dataset:  31%|███▏      | 565130/1807941 [08:23<07:52, 2631.14 examples/s]
Tokenizing train dataset:  31%|███▏      | 565407/1807941 [08:23<07:46, 2664.39 examples/s]
Tokenizing train dataset:  31%|███▏      | 565693/1807941 [08:23<07:38, 2711.58 examples/s]
Tokenizing train dataset:  31%|███▏      | 565970/1807941 [08:23<07:35, 2725.86 examples/s]
Tokenizing train dataset:  31%|███▏      | 566337/1807941 [08:23<07:53, 2620.52 examples/s]
Tokenizing train dataset:  31%|███▏      | 566704/1807941 [08:23<08:05, 2557.34 examples/s]
Tokenizing train dataset:  31%|███▏      | 567000/1807941 [08:23<07:52, 2627.22 examples/s]
Tokenizing train dataset:  31%|███▏      | 567275/1807941 [08:24<07:46, 2657.24 examples/s]
Tokenizing train dataset:  31%|███▏      | 567549/1807941 [08:24<07:43, 2676.26 examples/s]
Tokenizing train dataset:  31%|███▏      | 567851/1807941 [08:24<07:27, 2769.11 examples/s]
Tokenizing train dataset:  31%|███▏      | 568147/1807941 [08:24<07:30, 2754.41 examples/s]
Tokenizing train dataset:  31%|███▏      | 568441/1807941 [08:24<07:22, 2802.63 examples/s]
Tokenizing train dataset:  31%|███▏      | 568772/1807941 [08:24<11:06, 1859.67 examples/s]
Tokenizing train dataset:  31%|███▏      | 569062/1807941 [08:25<14:09, 1459.11 examples/s]
Tokenizing train dataset:  31%|███▏      | 569260/1807941 [08:25<14:30, 1423.03 examples/s]
Tokenizing train dataset:  32%|███▏      | 569600/1807941 [08:25<11:33, 1786.03 examples/s]
Tokenizing train dataset:  32%|███▏      | 569857/1807941 [08:25<10:36, 1943.64 examples/s]
Tokenizing train dataset:  32%|███▏      | 570211/1807941 [08:25<09:58, 2069.69 examples/s]
Tokenizing train dataset:  32%|███▏      | 570523/1807941 [08:25<08:56, 2305.86 examples/s]
Tokenizing train dataset:  32%|███▏      | 570804/1807941 [08:25<08:29, 2426.74 examples/s]
Tokenizing train dataset:  32%|███▏      | 571083/1807941 [08:25<08:11, 2518.63 examples/s]
Tokenizing train dataset:  32%|███▏      | 571351/1807941 [08:25<08:03, 2560.12 examples/s]
Tokenizing train dataset:  32%|███▏      | 571628/1807941 [08:26<07:52, 2614.73 examples/s]
Tokenizing train dataset:  32%|███▏      | 571921/1807941 [08:26<09:50, 2091.74 examples/s]
Tokenizing train dataset:  32%|███▏      | 572194/1807941 [08:26<13:46, 1494.49 examples/s]
Tokenizing train dataset:  32%|███▏      | 572402/1807941 [08:26<15:19, 1343.58 examples/s]
Tokenizing train dataset:  32%|███▏      | 572611/1807941 [08:27<16:32, 1244.85 examples/s]
Tokenizing train dataset:  32%|███▏      | 572809/1807941 [08:27<16:21, 1258.93 examples/s]
Tokenizing train dataset:  32%|███▏      | 572981/1807941 [08:27<16:50, 1221.60 examples/s]
Tokenizing train dataset:  32%|███▏      | 573143/1807941 [08:27<19:02, 1080.50 examples/s]
Tokenizing train dataset:  32%|███▏      | 573300/1807941 [08:27<19:15, 1068.71 examples/s]
Tokenizing train dataset:  32%|███▏      | 573443/1807941 [08:27<19:53, 1034.74 examples/s]
Tokenizing train dataset:  32%|███▏      | 573576/1807941 [08:27<20:47, 989.40 examples/s] 
Tokenizing train dataset:  32%|███▏      | 573679/1807941 [08:28<20:37, 997.28 examples/s]
Tokenizing train dataset:  32%|███▏      | 573813/1807941 [08:28<21:27, 958.91 examples/s]
Tokenizing train dataset:  32%|███▏      | 573931/1807941 [08:28<20:29, 1003.66 examples/s]
Tokenizing train dataset:  32%|███▏      | 574136/1807941 [08:28<16:48, 1223.79 examples/s]
Tokenizing train dataset:  32%|███▏      | 574360/1807941 [08:28<13:53, 1479.24 examples/s]
Tokenizing train dataset:  32%|███▏      | 574604/1807941 [08:28<11:52, 1729.81 examples/s]
Tokenizing train dataset:  32%|███▏      | 574855/1807941 [08:28<10:34, 1943.09 examples/s]
Tokenizing train dataset:  32%|███▏      | 575069/1807941 [08:28<10:19, 1990.34 examples/s]
Tokenizing train dataset:  32%|███▏      | 575316/1807941 [08:28<09:40, 2125.18 examples/s]
Tokenizing train dataset:  32%|███▏      | 575538/1807941 [08:29<09:33, 2148.61 examples/s]
Tokenizing train dataset:  32%|███▏      | 575774/1807941 [08:29<09:18, 2206.96 examples/s]
Tokenizing train dataset:  32%|███▏      | 576013/1807941 [08:29<09:05, 2258.86 examples/s]
Tokenizing train dataset:  32%|███▏      | 576278/1807941 [08:29<11:24, 1800.64 examples/s]
Tokenizing train dataset:  32%|███▏      | 576479/1807941 [08:29<13:48, 1486.23 examples/s]
Tokenizing train dataset:  32%|███▏      | 576678/1807941 [08:29<15:49, 1297.28 examples/s]
Tokenizing train dataset:  32%|███▏      | 576835/1807941 [08:30<18:20, 1118.93 examples/s]
Tokenizing train dataset:  32%|███▏      | 576981/1807941 [08:30<19:05, 1074.29 examples/s]
Tokenizing train dataset:  32%|███▏      | 577101/1807941 [08:30<20:33, 997.99 examples/s] 
Tokenizing train dataset:  32%|███▏      | 577224/1807941 [08:30<21:41, 945.26 examples/s]
Tokenizing train dataset:  32%|███▏      | 577345/1807941 [08:30<22:52, 896.33 examples/s]
Tokenizing train dataset:  32%|███▏      | 577479/1807941 [08:30<22:58, 892.71 examples/s]
Tokenizing train dataset:  32%|███▏      | 577572/1807941 [08:30<22:52, 896.66 examples/s]
Tokenizing train dataset:  32%|███▏      | 577692/1807941 [08:31<23:52, 858.58 examples/s]
Tokenizing train dataset:  32%|███▏      | 577815/1807941 [08:31<21:47, 941.02 examples/s]
Tokenizing train dataset:  32%|███▏      | 578010/1807941 [08:31<17:15, 1187.42 examples/s]
Tokenizing train dataset:  32%|███▏      | 578225/1807941 [08:31<14:16, 1434.98 examples/s]
Tokenizing train dataset:  32%|███▏      | 578477/1807941 [08:31<11:52, 1725.79 examples/s]
Tokenizing train dataset:  32%|███▏      | 578668/1807941 [08:31<16:28, 1243.01 examples/s]
Tokenizing train dataset:  32%|███▏      | 578823/1807941 [08:31<19:09, 1069.69 examples/s]
Tokenizing train dataset:  32%|███▏      | 578987/1807941 [08:32<19:08, 1069.96 examples/s]
Tokenizing train dataset:  32%|███▏      | 579171/1807941 [08:32<16:39, 1229.14 examples/s]
Tokenizing train dataset:  32%|███▏      | 579382/1807941 [08:32<14:23, 1422.09 examples/s]
Tokenizing train dataset:  32%|███▏      | 579568/1807941 [08:32<16:47, 1219.01 examples/s]
Tokenizing train dataset:  32%|███▏      | 579733/1807941 [08:32<17:20, 1180.59 examples/s]
Tokenizing train dataset:  32%|███▏      | 579873/1807941 [08:32<16:40, 1227.62 examples/s]
Tokenizing train dataset:  32%|███▏      | 580044/1807941 [08:32<17:04, 1198.05 examples/s]
Tokenizing train dataset:  32%|███▏      | 580216/1807941 [08:33<19:08, 1068.63 examples/s]
Tokenizing train dataset:  32%|███▏      | 580341/1807941 [08:33<18:31, 1104.16 examples/s]
Tokenizing train dataset:  32%|███▏      | 580533/1807941 [08:33<17:41, 1156.71 examples/s]
Tokenizing train dataset:  32%|███▏      | 580687/1807941 [08:33<18:26, 1109.19 examples/s]
Tokenizing train dataset:  32%|███▏      | 580828/1807941 [08:33<17:24, 1174.48 examples/s]
Tokenizing train dataset:  32%|███▏      | 580990/1807941 [08:33<18:02, 1133.42 examples/s]
Tokenizing train dataset:  32%|███▏      | 581134/1807941 [08:33<18:59, 1076.81 examples/s]
Tokenizing train dataset:  32%|███▏      | 581276/1807941 [08:34<17:44, 1152.52 examples/s]
Tokenizing train dataset:  32%|███▏      | 581425/1807941 [08:34<18:44, 1090.48 examples/s]
Tokenizing train dataset:  32%|███▏      | 581540/1807941 [08:34<20:44, 985.57 examples/s] 
Tokenizing train dataset:  32%|███▏      | 581645/1807941 [08:34<20:29, 997.05 examples/s]
Tokenizing train dataset:  32%|███▏      | 581796/1807941 [08:34<18:11, 1123.48 examples/s]
Tokenizing train dataset:  32%|███▏      | 581920/1807941 [08:34<20:06, 1016.17 examples/s]
Tokenizing train dataset:  32%|███▏      | 582085/1807941 [08:34<19:45, 1034.09 examples/s]
Tokenizing train dataset:  32%|███▏      | 582206/1807941 [08:34<21:24, 954.38 examples/s] 
Tokenizing train dataset:  32%|███▏      | 582342/1807941 [08:35<21:52, 933.54 examples/s]
Tokenizing train dataset:  32%|███▏      | 582470/1807941 [08:35<20:18, 1005.32 examples/s]
Tokenizing train dataset:  32%|███▏      | 582634/1807941 [08:35<19:48, 1031.34 examples/s]
Tokenizing train dataset:  32%|███▏      | 582892/1807941 [08:35<14:37, 1395.87 examples/s]
Tokenizing train dataset:  32%|███▏      | 583113/1807941 [08:35<13:01, 1566.80 examples/s]
Tokenizing train dataset:  32%|███▏      | 583348/1807941 [08:35<11:32, 1768.30 examples/s]
Tokenizing train dataset:  32%|███▏      | 583539/1807941 [08:35<14:31, 1405.68 examples/s]
Tokenizing train dataset:  32%|███▏      | 583705/1807941 [08:36<17:15, 1182.47 examples/s]
Tokenizing train dataset:  32%|███▏      | 583877/1807941 [08:36<19:06, 1067.78 examples/s]
Tokenizing train dataset:  32%|███▏      | 584016/1807941 [08:36<18:03, 1129.90 examples/s]
Tokenizing train dataset:  32%|███▏      | 584243/1807941 [08:36<14:46, 1380.16 examples/s]
Tokenizing train dataset:  32%|███▏      | 584436/1807941 [08:36<15:10, 1344.32 examples/s]
Tokenizing train dataset:  32%|███▏      | 584606/1807941 [08:36<15:58, 1275.83 examples/s]
Tokenizing train dataset:  32%|███▏      | 584778/1807941 [08:37<18:15, 1116.06 examples/s]
Tokenizing train dataset:  32%|███▏      | 584928/1807941 [08:37<20:48, 979.51 examples/s] 
Tokenizing train dataset:  32%|███▏      | 585042/1807941 [08:37<22:30, 905.49 examples/s]
Tokenizing train dataset:  32%|███▏      | 585139/1807941 [08:37<22:13, 916.90 examples/s]
Tokenizing train dataset:  32%|███▏      | 585250/1807941 [08:37<21:15, 958.52 examples/s]
Tokenizing train dataset:  32%|███▏      | 585397/1807941 [08:37<21:16, 958.02 examples/s]
Tokenizing train dataset:  32%|███▏      | 585529/1807941 [08:37<21:55, 929.59 examples/s]
Tokenizing train dataset:  32%|███▏      | 585673/1807941 [08:38<21:52, 931.36 examples/s]
Tokenizing train dataset:  32%|███▏      | 585771/1807941 [08:38<21:40, 939.96 examples/s]
Tokenizing train dataset:  32%|███▏      | 585876/1807941 [08:38<21:11, 961.10 examples/s]
Tokenizing train dataset:  32%|███▏      | 586058/1807941 [08:38<17:14, 1181.15 examples/s]
Tokenizing train dataset:  32%|███▏      | 586301/1807941 [08:38<13:25, 1515.70 examples/s]
Tokenizing train dataset:  32%|███▏      | 586526/1807941 [08:38<11:51, 1716.99 examples/s]
Tokenizing train dataset:  32%|███▏      | 586760/1807941 [08:38<10:45, 1891.25 examples/s]
Tokenizing train dataset:  32%|███▏      | 586960/1807941 [08:38<12:11, 1669.77 examples/s]
Tokenizing train dataset:  32%|███▏      | 587161/1807941 [08:39<14:47, 1375.07 examples/s]
Tokenizing train dataset:  32%|███▏      | 587339/1807941 [08:39<15:29, 1313.86 examples/s]
Tokenizing train dataset:  32%|███▏      | 587522/1807941 [08:39<14:14, 1427.95 examples/s]
Tokenizing train dataset:  33%|███▎      | 587734/1807941 [08:39<12:46, 1592.26 examples/s]
Tokenizing train dataset:  33%|███▎      | 587933/1807941 [08:39<15:11, 1339.11 examples/s]
Tokenizing train dataset:  33%|███▎      | 588126/1807941 [08:39<17:00, 1195.34 examples/s]
Tokenizing train dataset:  33%|███▎      | 588268/1807941 [08:39<18:06, 1122.07 examples/s]
Tokenizing train dataset:  33%|███▎      | 588466/1807941 [08:40<15:39, 1298.34 examples/s]
Tokenizing train dataset:  33%|███▎      | 588677/1807941 [08:40<13:41, 1483.67 examples/s]
Tokenizing train dataset:  33%|███▎      | 588862/1807941 [08:40<14:32, 1397.87 examples/s]
Tokenizing train dataset:  33%|███▎      | 589016/1807941 [08:40<16:04, 1264.00 examples/s]
Tokenizing train dataset:  33%|███▎      | 589253/1807941 [08:40<14:55, 1360.47 examples/s]
Tokenizing train dataset:  33%|███▎      | 589463/1807941 [08:40<13:17, 1528.32 examples/s]
Tokenizing train dataset:  33%|███▎      | 589689/1807941 [08:40<11:53, 1706.73 examples/s]
Tokenizing train dataset:  33%|███▎      | 589887/1807941 [08:40<11:27, 1772.66 examples/s]
Tokenizing train dataset:  33%|███▎      | 590134/1807941 [08:40<10:38, 1906.45 examples/s]
Tokenizing train dataset:  33%|███▎      | 590570/1807941 [08:41<07:53, 2573.65 examples/s]
Tokenizing train dataset:  33%|███▎      | 591212/1807941 [08:41<05:33, 3649.53 examples/s]
Tokenizing train dataset:  33%|███▎      | 591888/1807941 [08:41<04:28, 4536.89 examples/s]
Tokenizing train dataset:  33%|███▎      | 592523/1807941 [08:41<04:00, 5060.38 examples/s]
Tokenizing train dataset:  33%|███▎      | 593122/1807941 [08:41<03:47, 5331.00 examples/s]
Tokenizing train dataset:  33%|███▎      | 593762/1807941 [08:41<03:35, 5644.14 examples/s]
Tokenizing train dataset:  33%|███▎      | 594374/1807941 [08:41<03:29, 5784.04 examples/s]
Tokenizing train dataset:  33%|███▎      | 595020/1807941 [08:41<03:22, 5984.41 examples/s]
Tokenizing train dataset:  33%|███▎      | 595679/1807941 [08:41<03:16, 6162.74 examples/s]
Tokenizing train dataset:  33%|███▎      | 596327/1807941 [08:42<03:15, 6194.80 examples/s]
Tokenizing train dataset:  33%|███▎      | 596983/1807941 [08:42<03:12, 6302.68 examples/s]
Tokenizing train dataset:  33%|███▎      | 597941/1807941 [08:42<03:11, 6331.06 examples/s]
Tokenizing train dataset:  33%|███▎      | 598588/1807941 [08:42<03:09, 6365.81 examples/s]
Tokenizing train dataset:  33%|███▎      | 599232/1807941 [08:42<03:09, 6385.59 examples/s]
Tokenizing train dataset:  33%|███▎      | 599907/1807941 [08:42<03:06, 6486.41 examples/s]
Tokenizing train dataset:  33%|███▎      | 600866/1807941 [08:42<03:07, 6447.74 examples/s]
Tokenizing train dataset:  33%|███▎      | 601569/1807941 [08:42<03:27, 5821.75 examples/s]
Tokenizing train dataset:  33%|███▎      | 602187/1807941 [08:43<04:15, 4726.05 examples/s]
Tokenizing train dataset:  33%|███▎      | 602703/1807941 [08:43<04:37, 4338.08 examples/s]
Tokenizing train dataset:  33%|███▎      | 603208/1807941 [08:43<04:58, 4042.29 examples/s]
Tokenizing train dataset:  33%|███▎      | 603770/1807941 [08:43<05:04, 3949.10 examples/s]
Tokenizing train dataset:  33%|███▎      | 604278/1807941 [08:43<05:18, 3775.29 examples/s]
Tokenizing train dataset:  33%|███▎      | 604825/1807941 [08:43<05:22, 3732.54 examples/s]
Tokenizing train dataset:  33%|███▎      | 605350/1807941 [08:43<05:33, 3610.37 examples/s]
Tokenizing train dataset:  34%|███▎      | 605876/1807941 [08:44<05:36, 3576.21 examples/s]
Tokenizing train dataset:  34%|███▎      | 606344/1807941 [08:44<05:52, 3406.96 examples/s]
Tokenizing train dataset:  34%|███▎      | 606864/1807941 [08:44<05:51, 3420.92 examples/s]
Tokenizing train dataset:  34%|███▎      | 607334/1807941 [08:44<06:00, 3332.60 examples/s]
Tokenizing train dataset:  34%|███▎      | 607817/1807941 [08:44<06:03, 3297.66 examples/s]
Tokenizing train dataset:  34%|███▎      | 608300/1807941 [08:44<06:06, 3269.31 examples/s]
Tokenizing train dataset:  34%|███▎      | 608655/1807941 [08:44<06:00, 3325.59 examples/s]
Tokenizing train dataset:  34%|███▎      | 609002/1807941 [08:45<05:57, 3356.62 examples/s]
Tokenizing train dataset:  34%|███▎      | 609510/1807941 [08:45<05:56, 3365.47 examples/s]
Tokenizing train dataset:  34%|███▎      | 609849/1807941 [08:45<05:55, 3368.03 examples/s]
Tokenizing train dataset:  34%|███▍      | 610273/1807941 [08:45<06:16, 3177.27 examples/s]
Tokenizing train dataset:  34%|███▍      | 610731/1807941 [08:45<06:22, 3130.97 examples/s]
Tokenizing train dataset:  34%|███▍      | 611146/1807941 [08:45<06:37, 3010.52 examples/s]
Tokenizing train dataset:  34%|███▍      | 611458/1807941 [08:45<06:34, 3029.69 examples/s]
Tokenizing train dataset:  34%|███▍      | 611768/1807941 [08:45<06:32, 3045.84 examples/s]
Tokenizing train dataset:  34%|███▍      | 612222/1807941 [08:46<06:33, 3037.23 examples/s]
Tokenizing train dataset:  34%|███▍      | 612559/1807941 [08:46<06:23, 3113.65 examples/s]
Tokenizing train dataset:  34%|███▍      | 612890/1807941 [08:46<06:17, 3161.52 examples/s]
Tokenizing train dataset:  34%|███▍      | 613325/1807941 [08:46<06:29, 3063.52 examples/s]
Tokenizing train dataset:  34%|███▍      | 613656/1807941 [08:46<06:22, 3123.93 examples/s]
Tokenizing train dataset:  34%|███▍      | 614015/1807941 [08:46<06:57, 2860.97 examples/s]
Tokenizing train dataset:  34%|███▍      | 614329/1807941 [08:46<06:47, 2928.45 examples/s]
Tokenizing train dataset:  34%|███▍      | 614651/1807941 [08:46<06:37, 3001.26 examples/s]
Tokenizing train dataset:  34%|███▍      | 615073/1807941 [08:47<06:47, 2930.35 examples/s]
Tokenizing train dataset:  34%|███▍      | 615371/1807941 [08:47<06:45, 2941.41 examples/s]
Tokenizing train dataset:  34%|███▍      | 615707/1807941 [08:47<06:30, 3051.23 examples/s]
Tokenizing train dataset:  34%|███▍      | 616061/1807941 [08:47<07:07, 2790.56 examples/s]
Tokenizing train dataset:  34%|███▍      | 616507/1807941 [08:47<06:58, 2846.73 examples/s]
Tokenizing train dataset:  34%|███▍      | 616873/1807941 [08:47<06:31, 3043.73 examples/s]
Tokenizing train dataset:  34%|███▍      | 617226/1807941 [08:47<06:15, 3169.42 examples/s]
Tokenizing train dataset:  34%|███▍      | 617569/1807941 [08:47<06:07, 3236.93 examples/s]
Tokenizing train dataset:  34%|███▍      | 617974/1807941 [08:48<06:31, 3036.15 examples/s]
Tokenizing train dataset:  34%|███▍      | 618445/1807941 [08:48<06:27, 3071.13 examples/s]
Tokenizing train dataset:  34%|███▍      | 618799/1807941 [08:48<06:13, 3184.49 examples/s]
Tokenizing train dataset:  34%|███▍      | 619247/1807941 [08:48<06:22, 3110.64 examples/s]
Tokenizing train dataset:  34%|███▍      | 619588/1807941 [08:48<06:13, 3182.23 examples/s]
Tokenizing train dataset:  34%|███▍      | 619984/1807941 [08:48<06:37, 2989.08 examples/s]
Tokenizing train dataset:  34%|███▍      | 620414/1807941 [08:48<06:43, 2944.77 examples/s]
Tokenizing train dataset:  34%|███▍      | 620885/1807941 [08:48<06:35, 3004.94 examples/s]
Tokenizing train dataset:  34%|███▍      | 621209/1807941 [08:49<06:28, 3057.33 examples/s]
Tokenizing train dataset:  34%|███▍      | 621570/1807941 [08:49<06:58, 2837.47 examples/s]
Tokenizing train dataset:  34%|███▍      | 621890/1807941 [08:49<06:45, 2921.98 examples/s]
Tokenizing train dataset:  34%|███▍      | 622309/1807941 [08:49<06:52, 2873.74 examples/s]
Tokenizing train dataset:  34%|███▍      | 622602/1807941 [08:49<06:51, 2881.90 examples/s]
Tokenizing train dataset:  34%|███▍      | 622901/1807941 [08:49<06:47, 2907.79 examples/s]
Tokenizing train dataset:  34%|███▍      | 623269/1807941 [08:49<07:12, 2741.50 examples/s]
Tokenizing train dataset:  34%|███▍      | 623579/1807941 [08:49<06:58, 2829.87 examples/s]
Tokenizing train dataset:  35%|███▍      | 623927/1807941 [08:50<07:27, 2647.71 examples/s]
Tokenizing train dataset:  35%|███▍      | 624222/1807941 [08:50<07:15, 2720.63 examples/s]
Tokenizing train dataset:  35%|███▍      | 624621/1807941 [08:50<07:19, 2694.98 examples/s]
Tokenizing train dataset:  35%|███▍      | 624978/1807941 [08:50<06:46, 2910.00 examples/s]
Tokenizing train dataset:  35%|███▍      | 625292/1807941 [08:50<06:38, 2966.71 examples/s]
Tokenizing train dataset:  35%|███▍      | 625610/1807941 [08:50<06:31, 3023.37 examples/s]
Tokenizing train dataset:  35%|███▍      | 625991/1807941 [08:50<06:55, 2841.76 examples/s]
Tokenizing train dataset:  35%|███▍      | 626330/1807941 [08:50<07:28, 2636.69 examples/s]
Tokenizing train dataset:  35%|███▍      | 626678/1807941 [08:51<07:47, 2528.26 examples/s]
Tokenizing train dataset:  35%|███▍      | 627000/1807941 [08:51<08:16, 2376.49 examples/s]
Tokenizing train dataset:  35%|███▍      | 627340/1807941 [08:51<07:32, 2611.50 examples/s]
Tokenizing train dataset:  35%|███▍      | 627674/1807941 [08:51<07:03, 2788.58 examples/s]
Tokenizing train dataset:  35%|███▍      | 628059/1807941 [08:51<07:15, 2710.13 examples/s]
Tokenizing train dataset:  35%|███▍      | 628404/1807941 [08:51<07:38, 2570.21 examples/s]
Tokenizing train dataset:  35%|███▍      | 628731/1807941 [08:51<07:11, 2734.57 examples/s]
Tokenizing train dataset:  35%|███▍      | 629049/1807941 [08:52<07:47, 2522.15 examples/s]
Tokenizing train dataset:  35%|███▍      | 629352/1807941 [08:52<07:25, 2644.71 examples/s]
Tokenizing train dataset:  35%|███▍      | 629684/1807941 [08:52<07:53, 2490.47 examples/s]
Tokenizing train dataset:  35%|███▍      | 629999/1807941 [08:52<07:24, 2651.01 examples/s]
Tokenizing train dataset:  35%|███▍      | 630373/1807941 [08:52<07:34, 2589.78 examples/s]
Tokenizing train dataset:  35%|███▍      | 630679/1807941 [08:52<07:16, 2698.84 examples/s]
Tokenizing train dataset:  35%|███▍      | 630974/1807941 [08:52<07:06, 2760.49 examples/s]
Tokenizing train dataset:  35%|███▍      | 631283/1807941 [08:53<11:35, 1692.14 examples/s]
Tokenizing train dataset:  35%|███▍      | 631544/1807941 [08:53<14:29, 1353.53 examples/s]
Tokenizing train dataset:  35%|███▍      | 631730/1807941 [08:53<15:54, 1232.92 examples/s]
Tokenizing train dataset:  35%|███▍      | 631892/1807941 [08:53<16:26, 1192.73 examples/s]
Tokenizing train dataset:  35%|███▍      | 632062/1807941 [08:53<17:19, 1131.74 examples/s]
Tokenizing train dataset:  35%|███▍      | 632199/1807941 [08:54<18:14, 1074.14 examples/s]
Tokenizing train dataset:  35%|███▍      | 632338/1807941 [08:54<19:01, 1030.13 examples/s]
Tokenizing train dataset:  35%|███▍      | 632458/1807941 [08:54<20:22, 961.26 examples/s] 
Tokenizing train dataset:  35%|███▍      | 632573/1807941 [08:54<19:36, 998.67 examples/s]
Tokenizing train dataset:  35%|███▍      | 632699/1807941 [08:54<20:44, 944.24 examples/s]
Tokenizing train dataset:  35%|███▌      | 632832/1807941 [08:54<21:15, 921.38 examples/s]
Tokenizing train dataset:  35%|███▌      | 632931/1807941 [08:54<20:56, 935.47 examples/s]
Tokenizing train dataset:  35%|███▌      | 633038/1807941 [08:55<23:13, 843.05 examples/s]
Tokenizing train dataset:  35%|███▌      | 633140/1807941 [08:55<22:12, 881.34 examples/s]
Tokenizing train dataset:  35%|███▌      | 633404/1807941 [08:55<14:52, 1316.35 examples/s]
Tokenizing train dataset:  35%|███▌      | 633713/1807941 [08:55<11:00, 1778.78 examples/s]
Tokenizing train dataset:  35%|███▌      | 633922/1807941 [08:55<10:30, 1862.95 examples/s]
Tokenizing train dataset:  35%|███▌      | 634158/1807941 [08:55<09:47, 1999.38 examples/s]
Tokenizing train dataset:  35%|███▌      | 634404/1807941 [08:55<09:11, 2129.00 examples/s]
Tokenizing train dataset:  35%|███▌      | 634650/1807941 [08:55<08:47, 2222.19 examples/s]
Tokenizing train dataset:  35%|███▌      | 634903/1807941 [08:55<08:29, 2304.28 examples/s]
Tokenizing train dataset:  35%|███▌      | 635226/1807941 [08:55<08:42, 2244.39 examples/s]
Tokenizing train dataset:  35%|███▌      | 635606/1807941 [08:56<08:20, 2344.33 examples/s]
Tokenizing train dataset:  35%|███▌      | 635852/1807941 [08:56<08:14, 2370.82 examples/s]
Tokenizing train dataset:  35%|███▌      | 636189/1807941 [08:56<08:24, 2324.10 examples/s]
Tokenizing train dataset:  35%|███▌      | 636442/1807941 [08:56<08:13, 2373.08 examples/s]
Tokenizing train dataset:  35%|███▌      | 636795/1807941 [08:56<08:16, 2360.54 examples/s]
Tokenizing train dataset:  35%|███▌      | 637118/1807941 [08:56<08:31, 2289.02 examples/s]
Tokenizing train dataset:  35%|███▌      | 637350/1807941 [08:56<08:31, 2290.63 examples/s]
Tokenizing train dataset:  35%|███▌      | 637599/1807941 [08:56<08:20, 2337.89 examples/s]
Tokenizing train dataset:  35%|███▌      | 637837/1807941 [08:57<08:18, 2345.22 examples/s]
Tokenizing train dataset:  35%|███▌      | 638191/1807941 [08:57<08:17, 2349.19 examples/s]
Tokenizing train dataset:  35%|███▌      | 638432/1807941 [08:57<08:14, 2362.88 examples/s]
Tokenizing train dataset:  35%|███▌      | 638695/1807941 [08:57<08:00, 2433.09 examples/s]
Tokenizing train dataset:  35%|███▌      | 638942/1807941 [08:57<07:59, 2438.31 examples/s]
Tokenizing train dataset:  35%|███▌      | 639221/1807941 [08:57<12:07, 1605.45 examples/s]
Tokenizing train dataset:  35%|███▌      | 639451/1807941 [08:58<14:37, 1330.92 examples/s]
Tokenizing train dataset:  35%|███▌      | 639652/1807941 [08:58<17:15, 1128.53 examples/s]
Tokenizing train dataset:  35%|███▌      | 639792/1807941 [08:58<18:05, 1076.40 examples/s]
Tokenizing train dataset:  35%|███▌      | 639946/1807941 [08:58<18:21, 1060.38 examples/s]
Tokenizing train dataset:  35%|███▌      | 640076/1807941 [08:58<21:12, 917.57 examples/s] 
Tokenizing train dataset:  35%|███▌      | 640200/1807941 [08:59<21:52, 890.01 examples/s]
Tokenizing train dataset:  35%|███▌      | 640302/1807941 [08:59<21:16, 914.65 examples/s]
Tokenizing train dataset:  35%|███▌      | 640433/1807941 [08:59<21:46, 893.58 examples/s]
Tokenizing train dataset:  35%|███▌      | 640544/1807941 [08:59<23:03, 843.99 examples/s]
Tokenizing train dataset:  35%|███▌      | 640753/1807941 [08:59<19:23, 1003.32 examples/s]
Tokenizing train dataset:  35%|███▌      | 641000/1807941 [08:59<15:05, 1288.81 examples/s]
Tokenizing train dataset:  35%|███▌      | 641258/1807941 [08:59<12:14, 1589.27 examples/s]
Tokenizing train dataset:  35%|███▌      | 641507/1807941 [08:59<10:43, 1811.76 examples/s]
Tokenizing train dataset:  35%|███▌      | 641719/1807941 [08:59<10:17, 1889.62 examples/s]
Tokenizing train dataset:  36%|███▌      | 641962/1807941 [09:00<09:32, 2035.22 examples/s]
Tokenizing train dataset:  36%|███▌      | 642188/1807941 [09:00<09:16, 2095.07 examples/s]
Tokenizing train dataset:  36%|███▌      | 642426/1807941 [09:00<08:57, 2169.53 examples/s]
Tokenizing train dataset:  36%|███▌      | 642674/1807941 [09:00<11:10, 1736.79 examples/s]
Tokenizing train dataset:  36%|███▌      | 642892/1807941 [09:00<15:42, 1235.50 examples/s]
Tokenizing train dataset:  36%|███▌      | 643069/1807941 [09:01<17:20, 1119.02 examples/s]
Tokenizing train dataset:  36%|███▌      | 643267/1807941 [09:01<15:12, 1275.91 examples/s]
Tokenizing train dataset:  36%|███▌      | 643476/1807941 [09:01<13:27, 1442.22 examples/s]
Tokenizing train dataset:  36%|███▌      | 643701/1807941 [09:01<11:56, 1625.55 examples/s]
Tokenizing train dataset:  36%|███▌      | 643938/1807941 [09:01<10:43, 1807.76 examples/s]
Tokenizing train dataset:  36%|███▌      | 644150/1807941 [09:01<10:16, 1886.21 examples/s]
Tokenizing train dataset:  36%|███▌      | 644357/1807941 [09:01<10:02, 1932.49 examples/s]
Tokenizing train dataset:  36%|███▌      | 644579/1807941 [09:01<13:42, 1413.57 examples/s]
Tokenizing train dataset:  36%|███▌      | 644752/1807941 [09:01<13:11, 1469.56 examples/s]
Tokenizing train dataset:  36%|███▌      | 645000/1807941 [09:02<13:08, 1475.25 examples/s]
Tokenizing train dataset:  36%|███▌      | 645246/1807941 [09:02<12:44, 1521.02 examples/s]
Tokenizing train dataset:  36%|███▌      | 645441/1807941 [09:02<12:02, 1608.64 examples/s]
Tokenizing train dataset:  36%|███▌      | 645638/1807941 [09:02<12:50, 1508.39 examples/s]
Tokenizing train dataset:  36%|███▌      | 645901/1807941 [09:02<12:16, 1578.04 examples/s]
Tokenizing train dataset:  36%|███▌      | 646079/1807941 [09:02<13:21, 1450.15 examples/s]
Tokenizing train dataset:  36%|███▌      | 646268/1807941 [09:02<12:31, 1546.80 examples/s]
Tokenizing train dataset:  36%|███▌      | 646443/1807941 [09:03<12:09, 1592.99 examples/s]
Tokenizing train dataset:  36%|███▌      | 646609/1807941 [09:03<13:40, 1415.68 examples/s]
Tokenizing train dataset:  36%|███▌      | 646765/1807941 [09:03<13:21, 1449.45 examples/s]
Tokenizing train dataset:  36%|███▌      | 646975/1807941 [09:03<12:02, 1606.07 examples/s]
Tokenizing train dataset:  36%|███▌      | 647200/1807941 [09:03<13:54, 1391.44 examples/s]
Tokenizing train dataset:  36%|███▌      | 647373/1807941 [09:03<14:43, 1314.32 examples/s]
Tokenizing train dataset:  36%|███▌      | 647577/1807941 [09:03<13:05, 1476.86 examples/s]
Tokenizing train dataset:  36%|███▌      | 647758/1807941 [09:04<14:05, 1372.31 examples/s]
Tokenizing train dataset:  36%|███▌      | 647928/1807941 [09:04<16:37, 1162.78 examples/s]
Tokenizing train dataset:  36%|███▌      | 648115/1807941 [09:04<17:56, 1077.53 examples/s]
Tokenizing train dataset:  36%|███▌      | 648342/1807941 [09:04<14:41, 1314.86 examples/s]
Tokenizing train dataset:  36%|███▌      | 648524/1807941 [09:04<13:33, 1425.57 examples/s]
Tokenizing train dataset:  36%|███▌      | 648734/1807941 [09:04<15:08, 1276.23 examples/s]
Tokenizing train dataset:  36%|███▌      | 648953/1807941 [09:04<13:07, 1470.92 examples/s]
Tokenizing train dataset:  36%|███▌      | 649140/1807941 [09:05<13:55, 1386.42 examples/s]
Tokenizing train dataset:  36%|███▌      | 649320/1807941 [09:05<16:06, 1198.57 examples/s]
Tokenizing train dataset:  36%|███▌      | 649500/1807941 [09:05<14:41, 1314.87 examples/s]
Tokenizing train dataset:  36%|███▌      | 649651/1807941 [09:05<15:52, 1216.68 examples/s]
Tokenizing train dataset:  36%|███▌      | 649917/1807941 [09:05<12:30, 1542.55 examples/s]
Tokenizing train dataset:  36%|███▌      | 650140/1807941 [09:05<11:17, 1709.47 examples/s]
Tokenizing train dataset:  36%|███▌      | 650428/1807941 [09:05<09:35, 2011.29 examples/s]
Tokenizing train dataset:  36%|███▌      | 650702/1807941 [09:05<08:44, 2206.30 examples/s]
Tokenizing train dataset:  36%|███▌      | 650978/1807941 [09:06<08:10, 2358.41 examples/s]
Tokenizing train dataset:  36%|███▌      | 651239/1807941 [09:06<10:13, 1885.17 examples/s]
Tokenizing train dataset:  36%|███▌      | 651461/1807941 [09:06<13:22, 1440.33 examples/s]
Tokenizing train dataset:  36%|███▌      | 651640/1807941 [09:06<15:25, 1249.35 examples/s]
Tokenizing train dataset:  36%|███▌      | 651822/1807941 [09:06<14:11, 1357.60 examples/s]
Tokenizing train dataset:  36%|███▌      | 652003/1807941 [09:06<13:15, 1453.90 examples/s]
Tokenizing train dataset:  36%|███▌      | 652271/1807941 [09:06<11:05, 1737.80 examples/s]
Tokenizing train dataset:  36%|███▌      | 652503/1807941 [09:07<10:13, 1882.67 examples/s]
Tokenizing train dataset:  36%|███▌      | 652753/1807941 [09:07<09:25, 2044.43 examples/s]
Tokenizing train dataset:  36%|███▌      | 652992/1807941 [09:07<09:01, 2133.97 examples/s]
Tokenizing train dataset:  36%|███▌      | 653220/1807941 [09:07<08:52, 2169.86 examples/s]
Tokenizing train dataset:  36%|███▌      | 653493/1807941 [09:07<11:52, 1619.46 examples/s]
Tokenizing train dataset:  36%|███▌      | 653708/1807941 [09:07<14:50, 1296.62 examples/s]
Tokenizing train dataset:  36%|███▌      | 653921/1807941 [09:08<17:06, 1124.11 examples/s]
Tokenizing train dataset:  36%|███▌      | 654095/1807941 [09:08<18:53, 1017.85 examples/s]
Tokenizing train dataset:  36%|███▌      | 654263/1807941 [09:08<18:30, 1039.11 examples/s]
Tokenizing train dataset:  36%|███▌      | 654403/1807941 [09:08<19:02, 1009.90 examples/s]
Tokenizing train dataset:  36%|███▌      | 654534/1807941 [09:08<19:50, 968.55 examples/s] 
Tokenizing train dataset:  36%|███▌      | 654657/1807941 [09:08<20:45, 925.72 examples/s]
Tokenizing train dataset:  36%|███▌      | 654764/1807941 [09:09<20:14, 949.42 examples/s]
Tokenizing train dataset:  36%|███▌      | 654908/1807941 [09:09<20:18, 946.19 examples/s]
Tokenizing train dataset:  36%|███▌      | 655120/1807941 [09:09<16:33, 1160.70 examples/s]
Tokenizing train dataset:  36%|███▋      | 655382/1807941 [09:09<12:47, 1501.85 examples/s]
Tokenizing train dataset:  36%|███▋      | 655602/1807941 [09:09<11:27, 1675.42 examples/s]
Tokenizing train dataset:  36%|███▋      | 655824/1807941 [09:09<10:34, 1815.83 examples/s]
Tokenizing train dataset:  36%|███▋      | 656068/1807941 [09:09<09:40, 1985.64 examples/s]
Tokenizing train dataset:  36%|███▋      | 656295/1807941 [09:09<09:19, 2059.32 examples/s]
Tokenizing train dataset:  36%|███▋      | 656523/1807941 [09:09<09:03, 2119.07 examples/s]
Tokenizing train dataset:  36%|███▋      | 656767/1807941 [09:10<08:41, 2207.36 examples/s]
Tokenizing train dataset:  36%|███▋      | 656999/1807941 [09:10<08:35, 2234.79 examples/s]
Tokenizing train dataset:  36%|███▋      | 657292/1807941 [09:10<09:02, 2121.45 examples/s]
Tokenizing train dataset:  36%|███▋      | 657522/1807941 [09:10<08:51, 2164.67 examples/s]
Tokenizing train dataset:  36%|███▋      | 657866/1807941 [09:10<08:41, 2206.63 examples/s]
Tokenizing train dataset:  36%|███▋      | 658170/1807941 [09:10<08:57, 2138.32 examples/s]
Tokenizing train dataset:  36%|███▋      | 658500/1807941 [09:10<08:53, 2155.12 examples/s]
Tokenizing train dataset:  36%|███▋      | 658769/1807941 [09:11<10:26, 1835.41 examples/s]
Tokenizing train dataset:  36%|███▋      | 659000/1807941 [09:11<12:05, 1583.48 examples/s]
Tokenizing train dataset:  36%|███▋      | 659238/1807941 [09:11<12:05, 1582.92 examples/s]
Tokenizing train dataset:  36%|███▋      | 659413/1807941 [09:11<13:09, 1455.51 examples/s]
Tokenizing train dataset:  36%|███▋      | 659614/1807941 [09:11<12:10, 1571.24 examples/s]
Tokenizing train dataset:  36%|███▋      | 659861/1807941 [09:11<10:46, 1775.75 examples/s]
Tokenizing train dataset:  37%|███▋      | 660068/1807941 [09:11<10:22, 1845.37 examples/s]
Tokenizing train dataset:  37%|███▋      | 660268/1807941 [09:11<10:09, 1884.03 examples/s]
Tokenizing train dataset:  37%|███▋      | 660499/1807941 [09:12<12:12, 1567.49 examples/s]
Tokenizing train dataset:  37%|███▋      | 660691/1807941 [09:12<14:22, 1329.41 examples/s]
Tokenizing train dataset:  37%|███▋      | 660874/1807941 [09:12<14:46, 1293.70 examples/s]
Tokenizing train dataset:  37%|███▋      | 661106/1807941 [09:12<14:00, 1365.25 examples/s]
Tokenizing train dataset:  37%|███▋      | 661307/1807941 [09:12<12:43, 1501.26 examples/s]
Tokenizing train dataset:  37%|███▋      | 661511/1807941 [09:12<11:45, 1625.46 examples/s]
Tokenizing train dataset:  37%|███▋      | 661712/1807941 [09:12<11:05, 1721.89 examples/s]
Tokenizing train dataset:  37%|███▋      | 661896/1807941 [09:13<10:54, 1750.35 examples/s]
Tokenizing train dataset:  37%|███▋      | 662126/1807941 [09:13<11:26, 1668.88 examples/s]
Tokenizing train dataset:  37%|███▋      | 662326/1807941 [09:13<10:54, 1751.24 examples/s]
Tokenizing train dataset:  37%|███▋      | 662536/1807941 [09:13<10:22, 1840.78 examples/s]
Tokenizing train dataset:  37%|███▋      | 662740/1807941 [09:13<10:05, 1892.66 examples/s]
Tokenizing train dataset:  37%|███▋      | 662938/1807941 [09:13<09:58, 1913.23 examples/s]
Tokenizing train dataset:  37%|███▋      | 663178/1807941 [09:13<10:39, 1791.17 examples/s]
Tokenizing train dataset:  37%|███▋      | 663436/1807941 [09:13<10:49, 1762.70 examples/s]
Tokenizing train dataset:  37%|███▋      | 663688/1807941 [09:14<10:59, 1734.27 examples/s]
Tokenizing train dataset:  37%|███▋      | 663890/1807941 [09:14<10:35, 1800.75 examples/s]
Tokenizing train dataset:  37%|███▋      | 664155/1807941 [09:14<10:39, 1787.24 examples/s]
Tokenizing train dataset:  37%|███▋      | 664372/1807941 [09:14<10:08, 1878.54 examples/s]
Tokenizing train dataset:  37%|███▋      | 664595/1807941 [09:14<09:41, 1966.47 examples/s]
Tokenizing train dataset:  37%|███▋      | 664822/1807941 [09:14<09:18, 2046.00 examples/s]
Tokenizing train dataset:  37%|███▋      | 665135/1807941 [09:14<09:15, 2058.76 examples/s]
Tokenizing train dataset:  37%|███▋      | 665365/1807941 [09:14<08:59, 2118.82 examples/s]
Tokenizing train dataset:  37%|███▋      | 665594/1807941 [09:14<08:47, 2163.71 examples/s]
Tokenizing train dataset:  37%|███▋      | 665814/1807941 [09:15<08:46, 2169.78 examples/s]
Tokenizing train dataset:  37%|███▋      | 666121/1807941 [09:15<08:58, 2119.43 examples/s]
Tokenizing train dataset:  37%|███▋      | 666344/1807941 [09:15<08:51, 2146.50 examples/s]
Tokenizing train dataset:  37%|███▋      | 666570/1807941 [09:15<08:44, 2176.22 examples/s]
Tokenizing train dataset:  37%|███▋      | 666799/1807941 [09:15<08:37, 2204.82 examples/s]
Tokenizing train dataset:  37%|███▋      | 667109/1807941 [09:15<08:52, 2141.90 examples/s]
Tokenizing train dataset:  37%|███▋      | 667335/1807941 [09:15<08:45, 2172.02 examples/s]
Tokenizing train dataset:  37%|███▋      | 667561/1807941 [09:15<08:39, 2195.01 examples/s]
Tokenizing train dataset:  37%|███▋      | 667791/1807941 [09:16<08:32, 2223.74 examples/s]
Tokenizing train dataset:  37%|███▋      | 668111/1807941 [09:16<08:46, 2165.11 examples/s]
Tokenizing train dataset:  37%|███▋      | 668339/1807941 [09:16<08:40, 2191.25 examples/s]
Tokenizing train dataset:  37%|███▋      | 668570/1807941 [09:16<08:33, 2219.87 examples/s]
Tokenizing train dataset:  37%|███▋      | 668797/1807941 [09:16<08:30, 2231.83 examples/s]
Tokenizing train dataset:  37%|███▋      | 669112/1807941 [09:16<08:46, 2161.62 examples/s]
Tokenizing train dataset:  37%|███▋      | 669340/1807941 [09:16<08:40, 2188.07 examples/s]
Tokenizing train dataset:  37%|███▋      | 669566/1807941 [09:16<08:36, 2203.90 examples/s]
Tokenizing train dataset:  37%|███▋      | 669794/1807941 [09:16<08:31, 2224.15 examples/s]
Tokenizing train dataset:  37%|███▋      | 670116/1807941 [09:17<08:44, 2171.07 examples/s]
Tokenizing train dataset:  37%|███▋      | 670344/1807941 [09:17<08:38, 2195.00 examples/s]
Tokenizing train dataset:  37%|███▋      | 670575/1807941 [09:17<08:31, 2223.59 examples/s]
Tokenizing train dataset:  37%|███▋      | 670810/1807941 [09:17<08:24, 2255.46 examples/s]
Tokenizing train dataset:  37%|███▋      | 671119/1807941 [09:17<08:41, 2180.02 examples/s]
Tokenizing train dataset:  37%|███▋      | 671344/1807941 [09:17<08:37, 2194.39 examples/s]
Tokenizing train dataset:  37%|███▋      | 671569/1807941 [09:17<08:35, 2206.38 examples/s]
Tokenizing train dataset:  37%|███▋      | 671801/1807941 [09:17<08:28, 2234.02 examples/s]
Tokenizing train dataset:  37%|███▋      | 672115/1807941 [09:17<08:44, 2163.57 examples/s]
Tokenizing train dataset:  37%|███▋      | 672338/1807941 [09:18<08:41, 2178.50 examples/s]
Tokenizing train dataset:  37%|███▋      | 672560/1807941 [09:18<08:39, 2185.19 examples/s]
Tokenizing train dataset:  37%|███▋      | 672785/1807941 [09:18<08:36, 2199.22 examples/s]
Tokenizing train dataset:  37%|███▋      | 673117/1807941 [09:18<08:43, 2167.54 examples/s]
Tokenizing train dataset:  37%|███▋      | 673338/1807941 [09:18<08:41, 2175.85 examples/s]
Tokenizing train dataset:  37%|███▋      | 673568/1807941 [09:18<08:33, 2208.19 examples/s]
Tokenizing train dataset:  37%|███▋      | 673802/1807941 [09:18<08:26, 2240.53 examples/s]
Tokenizing train dataset:  37%|███▋      | 674126/1807941 [09:18<08:34, 2205.41 examples/s]
Tokenizing train dataset:  37%|███▋      | 674368/1807941 [09:18<08:21, 2259.19 examples/s]
Tokenizing train dataset:  37%|███▋      | 674610/1807941 [09:19<08:12, 2301.20 examples/s]
Tokenizing train dataset:  37%|███▋      | 674850/1807941 [09:19<08:07, 2326.17 examples/s]
Tokenizing train dataset:  37%|███▋      | 675186/1807941 [09:19<08:14, 2290.36 examples/s]
Tokenizing train dataset:  37%|███▋      | 675425/1807941 [09:19<08:09, 2312.32 examples/s]
Tokenizing train dataset:  37%|███▋      | 675679/1807941 [09:19<07:57, 2369.42 examples/s]
Tokenizing train dataset:  37%|███▋      | 675921/1807941 [09:19<07:55, 2382.37 examples/s]
Tokenizing train dataset:  37%|███▋      | 676261/1807941 [09:19<08:04, 2337.25 examples/s]
Tokenizing train dataset:  37%|███▋      | 676504/1807941 [09:19<07:59, 2360.59 examples/s]
Tokenizing train dataset:  37%|███▋      | 676748/1807941 [09:19<07:55, 2380.33 examples/s]
Tokenizing train dataset:  37%|███▋      | 676992/1807941 [09:20<07:52, 2395.32 examples/s]
Tokenizing train dataset:  37%|███▋      | 677329/1807941 [09:20<08:03, 2336.11 examples/s]
Tokenizing train dataset:  37%|███▋      | 677565/1807941 [09:20<08:02, 2341.73 examples/s]
Tokenizing train dataset:  37%|███▋      | 677869/1807941 [09:20<07:26, 2530.05 examples/s]
Tokenizing train dataset:  38%|███▊      | 678140/1807941 [09:20<07:26, 2531.04 examples/s]
Tokenizing train dataset:  38%|███▊      | 678428/1807941 [09:20<07:10, 2625.65 examples/s]
Tokenizing train dataset:  38%|███▊      | 678710/1807941 [09:20<07:01, 2677.48 examples/s]
Tokenizing train dataset:  38%|███▊      | 678998/1807941 [09:20<06:52, 2734.42 examples/s]
Tokenizing train dataset:  38%|███▊      | 679402/1807941 [09:21<06:55, 2714.29 examples/s]
Tokenizing train dataset:  38%|███▊      | 679693/1807941 [09:21<06:48, 2763.07 examples/s]
Tokenizing train dataset:  38%|███▊      | 679987/1807941 [09:21<06:41, 2808.69 examples/s]
Tokenizing train dataset:  38%|███▊      | 680391/1807941 [09:21<06:48, 2760.67 examples/s]
Tokenizing train dataset:  38%|███▊      | 680688/1807941 [09:21<06:40, 2813.49 examples/s]
Tokenizing train dataset:  38%|███▊      | 680981/1807941 [09:21<06:36, 2841.37 examples/s]
Tokenizing train dataset:  38%|███▊      | 681374/1807941 [09:21<06:48, 2755.11 examples/s]
Tokenizing train dataset:  38%|███▊      | 681758/1807941 [09:21<07:00, 2681.04 examples/s]
Tokenizing train dataset:  38%|███▊      | 682096/1807941 [09:22<08:13, 2283.39 examples/s]
Tokenizing train dataset:  38%|███▊      | 682337/1807941 [09:22<09:01, 2080.26 examples/s]
Tokenizing train dataset:  38%|███▊      | 682574/1807941 [09:22<09:43, 1927.01 examples/s]
Tokenizing train dataset:  38%|███▊      | 682774/1807941 [09:22<09:39, 1940.71 examples/s]
Tokenizing train dataset:  38%|███▊      | 683000/1807941 [09:22<09:32, 1964.95 examples/s]
Tokenizing train dataset:  38%|███▊      | 683320/1807941 [09:22<09:16, 2019.32 examples/s]
Tokenizing train dataset:  38%|███▊      | 683564/1807941 [09:22<08:50, 2119.97 examples/s]
Tokenizing train dataset:  38%|███▊      | 683810/1807941 [09:22<08:30, 2203.91 examples/s]
Tokenizing train dataset:  38%|███▊      | 684161/1807941 [09:23<08:19, 2248.07 examples/s]
Tokenizing train dataset:  38%|███▊      | 684411/1807941 [09:23<08:06, 2309.98 examples/s]
Tokenizing train dataset:  38%|███▊      | 684664/1807941 [09:23<07:54, 2365.30 examples/s]
Tokenizing train dataset:  38%|███▊      | 684918/1807941 [09:23<07:45, 2410.31 examples/s]
Tokenizing train dataset:  38%|███▊      | 685264/1807941 [09:23<07:53, 2368.70 examples/s]
Tokenizing train dataset:  38%|███▊      | 685514/1807941 [09:23<07:47, 2400.97 examples/s]
Tokenizing train dataset:  38%|███▊      | 685798/1807941 [09:23<11:34, 1616.59 examples/s]
Tokenizing train dataset:  38%|███▊      | 686000/1807941 [09:24<14:29, 1290.66 examples/s]
Tokenizing train dataset:  38%|███▊      | 686188/1807941 [09:24<15:50, 1180.76 examples/s]
Tokenizing train dataset:  38%|███▊      | 686355/1807941 [09:24<17:30, 1067.66 examples/s]
Tokenizing train dataset:  38%|███▊      | 686498/1807941 [09:24<18:03, 1035.06 examples/s]
Tokenizing train dataset:  38%|███▊      | 686624/1807941 [09:24<19:01, 982.02 examples/s] 
Tokenizing train dataset:  38%|███▊      | 686776/1807941 [09:25<18:59, 984.31 examples/s]
Tokenizing train dataset:  38%|███▊      | 686888/1807941 [09:25<20:26, 913.80 examples/s]
Tokenizing train dataset:  38%|███▊      | 686996/1807941 [09:25<19:55, 937.70 examples/s]
Tokenizing train dataset:  38%|███▊      | 687097/1807941 [09:25<21:54, 852.72 examples/s]
Tokenizing train dataset:  38%|███▊      | 687185/1807941 [09:25<21:57, 850.41 examples/s]
Tokenizing train dataset:  38%|███▊      | 687277/1807941 [09:25<21:33, 866.43 examples/s]
Tokenizing train dataset:  38%|███▊      | 687402/1807941 [09:25<22:07, 844.10 examples/s]
Tokenizing train dataset:  38%|███▊      | 687520/1807941 [09:25<22:47, 819.29 examples/s]
Tokenizing train dataset:  38%|███▊      | 687611/1807941 [09:26<22:43, 821.50 examples/s]
Tokenizing train dataset:  38%|███▊      | 687698/1807941 [09:26<22:39, 823.98 examples/s]
Tokenizing train dataset:  38%|███▊      | 687821/1807941 [09:26<22:51, 817.01 examples/s]
Tokenizing train dataset:  38%|███▊      | 687946/1807941 [09:26<22:51, 816.67 examples/s]
Tokenizing train dataset:  38%|███▊      | 688057/1807941 [09:26<24:08, 772.97 examples/s]
Tokenizing train dataset:  38%|███▊      | 688137/1807941 [09:26<24:05, 774.81 examples/s]
Tokenizing train dataset:  38%|███▊      | 688251/1807941 [09:26<24:56, 748.12 examples/s]
Tokenizing train dataset:  38%|███▊      | 688340/1807941 [09:27<24:00, 777.05 examples/s]
Tokenizing train dataset:  38%|███▊      | 688438/1807941 [09:27<22:47, 818.52 examples/s]
Tokenizing train dataset:  38%|███▊      | 688522/1807941 [09:27<22:40, 822.77 examples/s]
Tokenizing train dataset:  38%|███▊      | 688614/1807941 [09:27<22:01, 847.09 examples/s]
Tokenizing train dataset:  38%|███▊      | 688714/1807941 [09:27<21:06, 883.91 examples/s]
Tokenizing train dataset:  38%|███▊      | 688812/1807941 [09:27<20:31, 909.00 examples/s]
Tokenizing train dataset:  38%|███▊      | 688913/1807941 [09:27<20:02, 930.93 examples/s]
Tokenizing train dataset:  38%|███▊      | 689053/1807941 [09:27<23:47, 783.57 examples/s]
Tokenizing train dataset:  38%|███▊      | 689148/1807941 [09:28<25:28, 731.87 examples/s]
Tokenizing train dataset:  38%|███▊      | 689345/1807941 [09:28<18:23, 1013.90 examples/s]
Tokenizing train dataset:  38%|███▊      | 689635/1807941 [09:28<12:37, 1476.28 examples/s]
Tokenizing train dataset:  38%|███▊      | 689924/1807941 [09:28<10:07, 1840.98 examples/s]
Tokenizing train dataset:  38%|███▊      | 690153/1807941 [09:28<09:30, 1960.08 examples/s]
Tokenizing train dataset:  38%|███▊      | 690400/1807941 [09:28<08:52, 2098.16 examples/s]
Tokenizing train dataset:  38%|███▊      | 690653/1807941 [09:28<08:23, 2219.45 examples/s]
Tokenizing train dataset:  38%|███▊      | 690919/1807941 [09:28<07:56, 2343.49 examples/s]
Tokenizing train dataset:  38%|███▊      | 691281/1807941 [09:28<07:52, 2364.64 examples/s]
Tokenizing train dataset:  38%|███▊      | 691536/1807941 [09:28<07:43, 2410.78 examples/s]
Tokenizing train dataset:  38%|███▊      | 691790/1807941 [09:29<07:36, 2442.83 examples/s]
Tokenizing train dataset:  38%|███▊      | 692126/1807941 [09:29<07:51, 2365.46 examples/s]
Tokenizing train dataset:  38%|███▊      | 692370/1807941 [09:29<08:49, 2106.57 examples/s]
Tokenizing train dataset:  38%|███▊      | 692612/1807941 [09:29<08:31, 2182.31 examples/s]
Tokenizing train dataset:  38%|███▊      | 692882/1807941 [09:29<09:05, 2042.86 examples/s]
Tokenizing train dataset:  38%|███▊      | 693143/1807941 [09:29<09:35, 1935.68 examples/s]
Tokenizing train dataset:  38%|███▊      | 693389/1807941 [09:29<11:15, 1651.06 examples/s]
Tokenizing train dataset:  38%|███▊      | 693631/1807941 [09:30<11:23, 1630.06 examples/s]
Tokenizing train dataset:  38%|███▊      | 693875/1807941 [09:30<11:24, 1627.95 examples/s]
Tokenizing train dataset:  38%|███▊      | 694091/1807941 [09:30<12:12, 1521.64 examples/s]
Tokenizing train dataset:  38%|███▊      | 694250/1807941 [09:30<12:05, 1534.95 examples/s]
Tokenizing train dataset:  38%|███▊      | 694469/1807941 [09:30<12:18, 1507.94 examples/s]
Tokenizing train dataset:  38%|███▊      | 694692/1807941 [09:30<12:22, 1499.41 examples/s]
Tokenizing train dataset:  38%|███▊      | 694854/1807941 [09:30<12:09, 1525.18 examples/s]
Tokenizing train dataset:  38%|███▊      | 695066/1807941 [09:31<12:33, 1477.39 examples/s]
Tokenizing train dataset:  38%|███▊      | 695227/1807941 [09:31<12:18, 1506.52 examples/s]
Tokenizing train dataset:  38%|███▊      | 695443/1807941 [09:31<12:31, 1480.43 examples/s]
Tokenizing train dataset:  38%|███▊      | 695611/1807941 [09:31<12:08, 1526.66 examples/s]
Tokenizing train dataset:  38%|███▊      | 695851/1807941 [09:31<11:58, 1548.59 examples/s]
Tokenizing train dataset:  38%|███▊      | 696008/1807941 [09:31<13:25, 1380.69 examples/s]
Tokenizing train dataset:  39%|███▊      | 696173/1807941 [09:31<12:50, 1443.63 examples/s]
Tokenizing train dataset:  39%|███▊      | 696343/1807941 [09:31<12:18, 1505.42 examples/s]
Tokenizing train dataset:  39%|███▊      | 696515/1807941 [09:32<11:53, 1557.64 examples/s]
Tokenizing train dataset:  39%|███▊      | 696686/1807941 [09:32<11:37, 1593.57 examples/s]
Tokenizing train dataset:  39%|███▊      | 696923/1807941 [09:32<11:41, 1584.39 examples/s]
Tokenizing train dataset:  39%|███▊      | 697166/1807941 [09:32<11:54, 1554.27 examples/s]
Tokenizing train dataset:  39%|███▊      | 697329/1807941 [09:32<11:47, 1569.87 examples/s]
Tokenizing train dataset:  39%|███▊      | 697570/1807941 [09:32<11:44, 1575.45 examples/s]
Tokenizing train dataset:  39%|███▊      | 697730/1807941 [09:32<11:44, 1575.09 examples/s]
Tokenizing train dataset:  39%|███▊      | 697902/1807941 [09:32<11:31, 1606.20 examples/s]
Tokenizing train dataset:  39%|███▊      | 698111/1807941 [09:33<12:06, 1528.17 examples/s]
Tokenizing train dataset:  39%|███▊      | 698312/1807941 [09:33<11:14, 1644.15 examples/s]
Tokenizing train dataset:  39%|███▊      | 698513/1807941 [09:33<10:41, 1728.21 examples/s]
Tokenizing train dataset:  39%|███▊      | 698736/1807941 [09:33<09:57, 1855.92 examples/s]
Tokenizing train dataset:  39%|███▊      | 698939/1807941 [09:33<09:43, 1900.55 examples/s]
Tokenizing train dataset:  39%|███▊      | 699161/1807941 [09:33<10:36, 1740.96 examples/s]
Tokenizing train dataset:  39%|███▊      | 699420/1807941 [09:33<10:40, 1731.09 examples/s]
Tokenizing train dataset:  39%|███▊      | 699646/1807941 [09:33<11:14, 1643.91 examples/s]
Tokenizing train dataset:  39%|███▊      | 699817/1807941 [09:34<11:09, 1655.99 examples/s]
Tokenizing train dataset:  39%|███▊      | 699990/1807941 [09:34<11:03, 1670.39 examples/s]
Tokenizing train dataset:  39%|███▊      | 700190/1807941 [09:34<11:56, 1546.78 examples/s]
Tokenizing train dataset:  39%|███▊      | 700361/1807941 [09:34<11:38, 1584.53 examples/s]
Tokenizing train dataset:  39%|███▊      | 700525/1807941 [09:34<11:36, 1590.01 examples/s]
Tokenizing train dataset:  39%|███▉      | 700760/1807941 [09:34<11:42, 1576.67 examples/s]
Tokenizing train dataset:  39%|███▉      | 700973/1807941 [09:34<12:07, 1520.67 examples/s]
Tokenizing train dataset:  39%|███▉      | 701180/1807941 [09:34<12:33, 1468.24 examples/s]
Tokenizing train dataset:  39%|███▉      | 701337/1807941 [09:35<12:24, 1486.75 examples/s]
Tokenizing train dataset:  39%|███▉      | 701512/1807941 [09:35<11:53, 1549.86 examples/s]
Tokenizing train dataset:  39%|███▉      | 701677/1807941 [09:35<11:42, 1573.66 examples/s]
Tokenizing train dataset:  39%|███▉      | 701920/1807941 [09:35<11:36, 1588.89 examples/s]
Tokenizing train dataset:  39%|███▉      | 702156/1807941 [09:35<11:56, 1543.50 examples/s]
Tokenizing train dataset:  39%|███▉      | 702318/1807941 [09:35<11:48, 1559.90 examples/s]
Tokenizing train dataset:  39%|███▉      | 702552/1807941 [09:35<11:49, 1557.24 examples/s]
Tokenizing train dataset:  39%|███▉      | 702727/1807941 [09:35<11:32, 1596.84 examples/s]
Tokenizing train dataset:  39%|███▉      | 702960/1807941 [09:36<11:41, 1575.86 examples/s]
Tokenizing train dataset:  39%|███▉      | 703154/1807941 [09:36<12:33, 1466.19 examples/s]
Tokenizing train dataset:  39%|███▉      | 703330/1807941 [09:36<12:00, 1533.85 examples/s]
Tokenizing train dataset:  39%|███▉      | 703493/1807941 [09:36<11:50, 1554.32 examples/s]
Tokenizing train dataset:  39%|███▉      | 703666/1807941 [09:36<11:30, 1599.54 examples/s]
Tokenizing train dataset:  39%|███▉      | 703834/1807941 [09:36<11:21, 1619.15 examples/s]
Tokenizing train dataset:  39%|███▉      | 704000/1807941 [09:36<11:30, 1597.61 examples/s]
Tokenizing train dataset:  39%|███▉      | 704176/1807941 [09:36<11:13, 1639.39 examples/s]
Tokenizing train dataset:  39%|███▉      | 704363/1807941 [09:36<10:47, 1704.33 examples/s]
Tokenizing train dataset:  39%|███▉      | 704557/1807941 [09:37<10:23, 1770.54 examples/s]
Tokenizing train dataset:  39%|███▉      | 704816/1807941 [09:37<10:30, 1749.26 examples/s]
Tokenizing train dataset:  39%|███▉      | 704993/1807941 [09:37<10:28, 1753.90 examples/s]
Tokenizing train dataset:  39%|███▉      | 705253/1807941 [09:37<10:32, 1744.24 examples/s]
Tokenizing train dataset:  39%|███▉      | 705448/1807941 [09:37<11:44, 1565.68 examples/s]
Tokenizing train dataset:  39%|███▉      | 705611/1807941 [09:37<13:22, 1373.95 examples/s]
Tokenizing train dataset:  39%|███▉      | 705759/1807941 [09:37<14:48, 1239.81 examples/s]
Tokenizing train dataset:  39%|███▉      | 705900/1807941 [09:38<16:04, 1142.06 examples/s]
Tokenizing train dataset:  39%|███▉      | 706054/1807941 [09:38<17:30, 1049.10 examples/s]
Tokenizing train dataset:  39%|███▉      | 706186/1807941 [09:38<18:22, 998.94 examples/s] 
Tokenizing train dataset:  39%|███▉      | 706426/1807941 [09:38<14:06, 1300.90 examples/s]
Tokenizing train dataset:  39%|███▉      | 706843/1807941 [09:38<09:17, 1976.70 examples/s]
Tokenizing train dataset:  39%|███▉      | 707163/1807941 [09:38<08:02, 2281.19 examples/s]
Tokenizing train dataset:  39%|███▉      | 707514/1807941 [09:38<07:59, 2297.23 examples/s]
Tokenizing train dataset:  39%|███▉      | 707777/1807941 [09:39<08:40, 2112.28 examples/s]
Tokenizing train dataset:  39%|███▉      | 708026/1807941 [09:39<10:28, 1751.08 examples/s]
Tokenizing train dataset:  39%|███▉      | 708227/1807941 [09:39<11:16, 1625.18 examples/s]
Tokenizing train dataset:  39%|███▉      | 708447/1807941 [09:39<11:41, 1567.83 examples/s]
Tokenizing train dataset:  39%|███▉      | 708636/1807941 [09:39<12:24, 1476.28 examples/s]
Tokenizing train dataset:  39%|███▉      | 708807/1807941 [09:39<12:00, 1525.54 examples/s]
Tokenizing train dataset:  39%|███▉      | 709011/1807941 [09:39<12:27, 1469.17 examples/s]
Tokenizing train dataset:  39%|███▉      | 709163/1807941 [09:40<12:22, 1480.05 examples/s]
Tokenizing train dataset:  39%|███▉      | 709369/1807941 [09:40<12:41, 1442.73 examples/s]
Tokenizing train dataset:  39%|███▉      | 709517/1807941 [09:40<12:36, 1451.35 examples/s]
Tokenizing train dataset:  39%|███▉      | 709725/1807941 [09:40<12:49, 1427.72 examples/s]
Tokenizing train dataset:  39%|███▉      | 709939/1807941 [09:40<12:54, 1418.30 examples/s]
Tokenizing train dataset:  39%|███▉      | 710083/1807941 [09:40<12:54, 1416.65 examples/s]
Tokenizing train dataset:  39%|███▉      | 710295/1807941 [09:40<13:02, 1402.79 examples/s]
Tokenizing train dataset:  39%|███▉      | 710470/1807941 [09:40<13:50, 1321.15 examples/s]
Tokenizing train dataset:  39%|███▉      | 710605/1807941 [09:41<13:46, 1326.94 examples/s]
Tokenizing train dataset:  39%|███▉      | 710765/1807941 [09:41<13:08, 1391.13 examples/s]
Tokenizing train dataset:  39%|███▉      | 710912/1807941 [09:41<12:58, 1409.34 examples/s]
Tokenizing train dataset:  39%|███▉      | 711122/1807941 [09:41<13:07, 1392.85 examples/s]
Tokenizing train dataset:  39%|███▉      | 711266/1807941 [09:41<13:17, 1374.29 examples/s]
Tokenizing train dataset:  39%|███▉      | 711430/1807941 [09:41<12:42, 1437.48 examples/s]
Tokenizing train dataset:  39%|███▉      | 711598/1807941 [09:41<12:12, 1497.48 examples/s]
Tokenizing train dataset:  39%|███▉      | 711761/1807941 [09:41<12:02, 1517.31 examples/s]
Tokenizing train dataset:  39%|███▉      | 711988/1807941 [09:42<12:11, 1499.01 examples/s]
Tokenizing train dataset:  39%|███▉      | 712212/1807941 [09:42<12:13, 1493.83 examples/s]
Tokenizing train dataset:  39%|███▉      | 712433/1807941 [09:42<12:17, 1485.43 examples/s]
Tokenizing train dataset:  39%|███▉      | 712665/1807941 [09:42<12:18, 1483.98 examples/s]
Tokenizing train dataset:  39%|███▉      | 712817/1807941 [09:42<12:18, 1483.46 examples/s]
Tokenizing train dataset:  39%|███▉      | 713007/1807941 [09:42<12:58, 1407.15 examples/s]
Tokenizing train dataset:  39%|███▉      | 713203/1807941 [09:42<13:19, 1368.43 examples/s]
Tokenizing train dataset:  39%|███▉      | 713350/1807941 [09:42<13:06, 1391.10 examples/s]
Tokenizing train dataset:  39%|███▉      | 713528/1807941 [09:43<13:50, 1317.48 examples/s]
Tokenizing train dataset:  39%|███▉      | 713721/1807941 [09:43<14:04, 1295.34 examples/s]
Tokenizing train dataset:  39%|███▉      | 713876/1807941 [09:43<15:03, 1210.28 examples/s]
Tokenizing train dataset:  39%|███▉      | 714015/1807941 [09:43<16:16, 1120.25 examples/s]
Tokenizing train dataset:  40%|███▉      | 714151/1807941 [09:43<18:44, 972.52 examples/s] 
Tokenizing train dataset:  40%|███▉      | 714275/1807941 [09:43<17:44, 1027.19 examples/s]
Tokenizing train dataset:  40%|███▉      | 714393/1807941 [09:43<17:10, 1061.10 examples/s]
Tokenizing train dataset:  40%|███▉      | 714535/1807941 [09:44<17:52, 1019.26 examples/s]
Tokenizing train dataset:  40%|███▉      | 714651/1807941 [09:44<17:21, 1049.49 examples/s]
Tokenizing train dataset:  40%|███▉      | 714808/1807941 [09:44<17:26, 1044.32 examples/s]
Tokenizing train dataset:  40%|███▉      | 714934/1807941 [09:44<16:39, 1093.28 examples/s]
Tokenizing train dataset:  40%|███▉      | 715073/1807941 [09:44<17:39, 1031.86 examples/s]
Tokenizing train dataset:  40%|███▉      | 715212/1807941 [09:44<18:18, 994.48 examples/s] 
Tokenizing train dataset:  40%|███▉      | 715350/1807941 [09:44<18:46, 969.81 examples/s]
Tokenizing train dataset:  40%|███▉      | 715449/1807941 [09:45<18:43, 971.98 examples/s]
Tokenizing train dataset:  40%|███▉      | 715603/1807941 [09:45<18:26, 986.86 examples/s]
Tokenizing train dataset:  40%|███▉      | 715715/1807941 [09:45<17:54, 1016.48 examples/s]
Tokenizing train dataset:  40%|███▉      | 715834/1807941 [09:45<17:24, 1045.49 examples/s]
Tokenizing train dataset:  40%|███▉      | 715988/1807941 [09:45<17:35, 1034.39 examples/s]
Tokenizing train dataset:  40%|███▉      | 716110/1807941 [09:45<19:03, 954.59 examples/s] 
Tokenizing train dataset:  40%|███▉      | 716224/1807941 [09:45<18:24, 988.46 examples/s]
Tokenizing train dataset:  40%|███▉      | 716342/1807941 [09:45<17:43, 1026.45 examples/s]
Tokenizing train dataset:  40%|███▉      | 716467/1807941 [09:46<16:50, 1080.29 examples/s]
Tokenizing train dataset:  40%|███▉      | 716620/1807941 [09:46<17:27, 1042.17 examples/s]
Tokenizing train dataset:  40%|███▉      | 716772/1807941 [09:46<17:44, 1025.03 examples/s]
Tokenizing train dataset:  40%|███▉      | 716881/1807941 [09:46<17:33, 1035.48 examples/s]
Tokenizing train dataset:  40%|███▉      | 717012/1807941 [09:46<18:47, 967.85 examples/s] 
Tokenizing train dataset:  40%|███▉      | 717127/1807941 [09:46<18:01, 1008.47 examples/s]
Tokenizing train dataset:  40%|███▉      | 717271/1807941 [09:46<18:23, 988.59 examples/s] 
Tokenizing train dataset:  40%|███▉      | 717383/1807941 [09:46<17:49, 1019.49 examples/s]
Tokenizing train dataset:  40%|███▉      | 717521/1807941 [09:47<20:14, 897.63 examples/s] 
Tokenizing train dataset:  40%|███▉      | 717622/1807941 [09:47<19:53, 913.87 examples/s]
Tokenizing train dataset:  40%|███▉      | 717759/1807941 [09:47<20:10, 900.72 examples/s]
Tokenizing train dataset:  40%|███▉      | 717888/1807941 [09:47<18:20, 990.22 examples/s]
Tokenizing train dataset:  40%|███▉      | 718000/1807941 [09:47<18:45, 968.61 examples/s]
Tokenizing train dataset:  40%|███▉      | 718121/1807941 [09:47<17:39, 1028.81 examples/s]
Tokenizing train dataset:  40%|███▉      | 718357/1807941 [09:47<13:08, 1381.39 examples/s]
Tokenizing train dataset:  40%|███▉      | 718594/1807941 [09:47<11:00, 1650.46 examples/s]
Tokenizing train dataset:  40%|███▉      | 718826/1807941 [09:48<09:52, 1836.76 examples/s]
Tokenizing train dataset:  40%|███▉      | 719027/1807941 [09:48<09:38, 1883.08 examples/s]
Tokenizing train dataset:  40%|███▉      | 719266/1807941 [09:48<08:56, 2028.97 examples/s]
Tokenizing train dataset:  40%|███▉      | 719498/1807941 [09:48<08:35, 2111.67 examples/s]
Tokenizing train dataset:  40%|███▉      | 719729/1807941 [09:48<08:21, 2168.06 examples/s]
Tokenizing train dataset:  40%|███▉      | 719965/1807941 [09:48<08:09, 2221.66 examples/s]
Tokenizing train dataset:  40%|███▉      | 720281/1807941 [09:48<08:19, 2176.02 examples/s]
Tokenizing train dataset:  40%|███▉      | 720512/1807941 [09:48<08:12, 2208.51 examples/s]
Tokenizing train dataset:  40%|███▉      | 720742/1807941 [09:48<08:07, 2231.85 examples/s]
Tokenizing train dataset:  40%|███▉      | 720974/1807941 [09:48<08:02, 2253.83 examples/s]
Tokenizing train dataset:  40%|███▉      | 721291/1807941 [09:49<08:14, 2197.25 examples/s]
Tokenizing train dataset:  40%|███▉      | 721537/1807941 [09:49<07:59, 2264.98 examples/s]
Tokenizing train dataset:  40%|███▉      | 721781/1807941 [09:49<07:50, 2309.46 examples/s]
Tokenizing train dataset:  40%|███▉      | 722118/1807941 [09:49<08:00, 2260.22 examples/s]
Tokenizing train dataset:  40%|███▉      | 722354/1807941 [09:49<07:55, 2284.31 examples/s]
Tokenizing train dataset:  40%|███▉      | 722594/1807941 [09:49<07:48, 2314.30 examples/s]
Tokenizing train dataset:  40%|███▉      | 722942/1807941 [09:49<07:49, 2312.50 examples/s]
Tokenizing train dataset:  40%|████      | 723258/1807941 [09:49<08:04, 2239.64 examples/s]
Tokenizing train dataset:  40%|████      | 723495/1807941 [09:50<07:57, 2269.61 examples/s]
Tokenizing train dataset:  40%|████      | 723729/1807941 [09:50<07:54, 2284.81 examples/s]
Tokenizing train dataset:  40%|████      | 723962/1807941 [09:50<07:52, 2293.80 examples/s]
Tokenizing train dataset:  40%|████      | 724275/1807941 [09:50<08:09, 2214.83 examples/s]
Tokenizing train dataset:  40%|████      | 724507/1807941 [09:50<08:03, 2239.11 examples/s]
Tokenizing train dataset:  40%|████      | 724736/1807941 [09:50<08:01, 2248.46 examples/s]
Tokenizing train dataset:  40%|████      | 725000/1807941 [09:50<09:49, 1838.37 examples/s]
Tokenizing train dataset:  40%|████      | 725248/1807941 [09:50<10:09, 1777.00 examples/s]
Tokenizing train dataset:  40%|████      | 725454/1807941 [09:51<09:48, 1840.23 examples/s]
Tokenizing train dataset:  40%|████      | 725853/1807941 [09:51<07:36, 2372.32 examples/s]
Tokenizing train dataset:  40%|████      | 726214/1807941 [09:51<06:41, 2693.81 examples/s]
Tokenizing train dataset:  40%|████      | 726633/1807941 [09:51<05:48, 3099.07 examples/s]
Tokenizing train dataset:  40%|████      | 726962/1807941 [09:51<05:43, 3147.36 examples/s]
Tokenizing train dataset:  40%|████      | 727365/1807941 [09:51<07:35, 2372.89 examples/s]
Tokenizing train dataset:  40%|████      | 727656/1807941 [09:51<08:02, 2237.47 examples/s]
Tokenizing train dataset:  40%|████      | 727928/1807941 [09:51<07:41, 2340.10 examples/s]
Tokenizing train dataset:  40%|████      | 728279/1807941 [09:52<08:33, 2101.87 examples/s]
Tokenizing train dataset:  40%|████      | 728542/1807941 [09:52<09:01, 1995.15 examples/s]
Tokenizing train dataset:  40%|████      | 728769/1807941 [09:52<09:42, 1851.22 examples/s]
Tokenizing train dataset:  40%|████      | 728988/1807941 [09:52<10:22, 1732.57 examples/s]
Tokenizing train dataset:  40%|████      | 729255/1807941 [09:53<34:22, 522.95 examples/s] 
Tokenizing train dataset:  40%|████      | 729534/1807941 [09:54<25:41, 699.36 examples/s]
Tokenizing train dataset:  40%|████      | 729808/1807941 [09:54<19:53, 903.35 examples/s]
Tokenizing train dataset:  40%|████      | 730036/1807941 [09:54<16:43, 1074.54 examples/s]
Tokenizing train dataset:  40%|████      | 730284/1807941 [09:54<13:58, 1285.38 examples/s]
Tokenizing train dataset:  40%|████      | 730561/1807941 [09:54<11:36, 1547.13 examples/s]
Tokenizing train dataset:  40%|████      | 730910/1807941 [09:54<10:12, 1758.36 examples/s]
Tokenizing train dataset:  40%|████      | 731242/1807941 [09:54<08:37, 2080.24 examples/s]
Tokenizing train dataset:  40%|████      | 731654/1807941 [09:54<07:03, 2544.26 examples/s]
Tokenizing train dataset:  40%|████      | 732030/1807941 [09:54<06:18, 2842.46 examples/s]
Tokenizing train dataset:  41%|████      | 732430/1807941 [09:55<05:42, 3141.99 examples/s]
Tokenizing train dataset:  41%|████      | 732831/1807941 [09:55<05:18, 3374.07 examples/s]
Tokenizing train dataset:  41%|████      | 733197/1807941 [09:55<05:11, 3450.12 examples/s]
Tokenizing train dataset:  41%|████      | 733582/1807941 [09:55<05:01, 3560.62 examples/s]
Tokenizing train dataset:  41%|████      | 733964/1807941 [09:55<04:55, 3631.20 examples/s]
Tokenizing train dataset:  41%|████      | 734516/1807941 [09:55<04:54, 3645.91 examples/s]
Tokenizing train dataset:  41%|████      | 734919/1807941 [09:55<04:46, 3746.90 examples/s]
Tokenizing train dataset:  41%|████      | 735483/1807941 [09:55<04:46, 3748.07 examples/s]
Tokenizing train dataset:  41%|████      | 735869/1807941 [09:55<04:43, 3775.16 examples/s]
Tokenizing train dataset:  41%|████      | 736411/1807941 [09:56<04:48, 3715.08 examples/s]
Tokenizing train dataset:  41%|████      | 736795/1807941 [09:56<04:46, 3743.32 examples/s]
Tokenizing train dataset:  41%|████      | 737340/1807941 [09:56<04:49, 3699.08 examples/s]
Tokenizing train dataset:  41%|████      | 737726/1807941 [09:56<04:46, 3737.30 examples/s]
Tokenizing train dataset:  41%|████      | 738276/1807941 [09:56<04:48, 3708.59 examples/s]
Tokenizing train dataset:  41%|████      | 738839/1807941 [09:56<04:47, 3720.00 examples/s]
Tokenizing train dataset:  41%|████      | 739391/1807941 [09:56<04:48, 3702.49 examples/s]
Tokenizing train dataset:  41%|████      | 739770/1807941 [09:56<04:47, 3719.81 examples/s]
Tokenizing train dataset:  41%|████      | 740291/1807941 [09:57<04:53, 3633.19 examples/s]
Tokenizing train dataset:  41%|████      | 740666/1807941 [09:57<04:51, 3659.77 examples/s]
Tokenizing train dataset:  41%|████      | 741221/1807941 [09:57<04:50, 3669.40 examples/s]
Tokenizing train dataset:  41%|████      | 741627/1807941 [09:57<04:43, 3762.47 examples/s]
Tokenizing train dataset:  41%|████      | 742014/1807941 [09:57<04:41, 3787.84 examples/s]
Tokenizing train dataset:  41%|████      | 742428/1807941 [09:57<04:34, 3880.27 examples/s]
Tokenizing train dataset:  41%|████      | 742845/1807941 [09:57<04:29, 3957.88 examples/s]
Tokenizing train dataset:  41%|████      | 743250/1807941 [09:57<04:27, 3981.13 examples/s]
Tokenizing train dataset:  41%|████      | 743679/1807941 [09:57<04:22, 4061.54 examples/s]
Tokenizing train dataset:  41%|████      | 744277/1807941 [09:58<04:23, 4029.22 examples/s]
Tokenizing train dataset:  41%|████      | 744701/1807941 [09:58<04:20, 4083.48 examples/s]
Tokenizing train dataset:  41%|████      | 745116/1807941 [09:58<04:19, 4098.59 examples/s]
Tokenizing train dataset:  41%|████      | 745553/1807941 [09:58<04:14, 4172.50 examples/s]
Tokenizing train dataset:  41%|████▏     | 745989/1807941 [09:58<04:11, 4225.78 examples/s]
Tokenizing train dataset:  41%|████▏     | 746620/1807941 [09:58<04:11, 4213.62 examples/s]
Tokenizing train dataset:  41%|████▏     | 747232/1807941 [09:58<04:14, 4162.81 examples/s]
Tokenizing train dataset:  41%|████▏     | 747664/1807941 [09:58<04:12, 4200.07 examples/s]
Tokenizing train dataset:  41%|████▏     | 748275/1807941 [09:59<04:15, 4154.40 examples/s]
Tokenizing train dataset:  41%|████▏     | 748700/1807941 [09:59<04:13, 4174.06 examples/s]
Tokenizing train dataset:  41%|████▏     | 749322/1807941 [09:59<04:14, 4162.68 examples/s]
Tokenizing train dataset:  41%|████▏     | 749746/1807941 [09:59<04:13, 4181.06 examples/s]
Tokenizing train dataset:  42%|████▏     | 750370/1807941 [09:59<04:13, 4171.63 examples/s]
Tokenizing train dataset:  42%|████▏     | 750792/1807941 [09:59<05:49, 3024.26 examples/s]
Tokenizing train dataset:  42%|████▏     | 751152/1807941 [10:00<07:21, 2392.47 examples/s]
Tokenizing train dataset:  42%|████▏     | 751481/1807941 [10:00<08:09, 2157.22 examples/s]
Tokenizing train dataset:  42%|████▏     | 751760/1807941 [10:00<08:28, 2078.96 examples/s]
Tokenizing train dataset:  42%|████▏     | 752000/1807941 [10:00<09:04, 1940.86 examples/s]
Tokenizing train dataset:  42%|████▏     | 752257/1807941 [10:00<09:22, 1877.13 examples/s]
Tokenizing train dataset:  42%|████▏     | 752497/1807941 [10:00<10:08, 1735.74 examples/s]
Tokenizing train dataset:  42%|████▏     | 752765/1807941 [10:01<10:03, 1747.27 examples/s]
Tokenizing train dataset:  42%|████▏     | 752970/1807941 [10:01<09:43, 1809.26 examples/s]
Tokenizing train dataset:  42%|████▏     | 753215/1807941 [10:01<10:02, 1751.50 examples/s]
Tokenizing train dataset:  42%|████▏     | 753440/1807941 [10:01<10:34, 1662.65 examples/s]
Tokenizing train dataset:  42%|████▏     | 753770/1807941 [10:01<08:39, 2029.42 examples/s]
Tokenizing train dataset:  42%|████▏     | 754126/1807941 [10:01<07:19, 2398.24 examples/s]
Tokenizing train dataset:  42%|████▏     | 754533/1807941 [10:01<06:12, 2825.96 examples/s]
Tokenizing train dataset:  42%|████▏     | 754941/1807941 [10:01<05:33, 3161.40 examples/s]
Tokenizing train dataset:  42%|████▏     | 755323/1807941 [10:01<05:15, 3340.55 examples/s]
Tokenizing train dataset:  42%|████▏     | 755734/1807941 [10:02<04:55, 3555.80 examples/s]
Tokenizing train dataset:  42%|████▏     | 756117/1807941 [10:02<04:49, 3632.16 examples/s]
Tokenizing train dataset:  42%|████▏     | 756492/1807941 [10:02<04:47, 3661.69 examples/s]
Tokenizing train dataset:  42%|████▏     | 756868/1807941 [10:02<04:45, 3686.62 examples/s]
Tokenizing train dataset:  42%|████▏     | 757394/1807941 [10:02<04:50, 3615.35 examples/s]
Tokenizing train dataset:  42%|████▏     | 757937/1807941 [10:02<04:50, 3612.80 examples/s]
Tokenizing train dataset:  42%|████▏     | 758457/1807941 [10:02<04:54, 3559.99 examples/s]
Tokenizing train dataset:  42%|████▏     | 758816/1807941 [10:02<04:54, 3565.07 examples/s]
Tokenizing train dataset:  42%|████▏     | 759332/1807941 [10:03<04:58, 3518.27 examples/s]
Tokenizing train dataset:  42%|████▏     | 759695/1807941 [10:03<04:55, 3543.13 examples/s]
Tokenizing train dataset:  42%|████▏     | 760227/1807941 [10:03<04:55, 3542.04 examples/s]
Tokenizing train dataset:  42%|████▏     | 760597/1807941 [10:03<04:52, 3579.55 examples/s]
Tokenizing train dataset:  42%|████▏     | 760975/1807941 [10:03<04:48, 3629.45 examples/s]
Tokenizing train dataset:  42%|████▏     | 761505/1807941 [10:03<04:51, 3592.39 examples/s]
Tokenizing train dataset:  42%|████▏     | 761872/1807941 [10:03<04:49, 3609.07 examples/s]
Tokenizing train dataset:  42%|████▏     | 762397/1807941 [10:03<04:52, 3568.95 examples/s]
Tokenizing train dataset:  42%|████▏     | 762931/1807941 [10:04<04:53, 3562.33 examples/s]
Tokenizing train dataset:  42%|████▏     | 763437/1807941 [10:04<04:58, 3500.41 examples/s]
Tokenizing train dataset:  42%|████▏     | 763794/1807941 [10:04<04:57, 3513.05 examples/s]
Tokenizing train dataset:  42%|████▏     | 764304/1807941 [10:04<05:00, 3471.87 examples/s]
Tokenizing train dataset:  42%|████▏     | 764663/1807941 [10:04<04:58, 3497.69 examples/s]
Tokenizing train dataset:  42%|████▏     | 765186/1807941 [10:04<04:59, 3482.09 examples/s]
Tokenizing train dataset:  42%|████▏     | 765563/1807941 [10:04<04:53, 3551.15 examples/s]
Tokenizing train dataset:  42%|████▏     | 765930/1807941 [10:04<04:50, 3581.19 examples/s]
Tokenizing train dataset:  42%|████▏     | 766310/1807941 [10:05<06:43, 2583.98 examples/s]
Tokenizing train dataset:  42%|████▏     | 766677/1807941 [10:05<08:10, 2124.16 examples/s]
Tokenizing train dataset:  42%|████▏     | 766961/1807941 [10:05<09:11, 1887.92 examples/s]
Tokenizing train dataset:  42%|████▏     | 767182/1807941 [10:05<08:55, 1943.37 examples/s]
Tokenizing train dataset:  42%|████▏     | 767559/1807941 [10:05<07:28, 2321.14 examples/s]
Tokenizing train dataset:  42%|████▏     | 767904/1807941 [10:05<06:43, 2578.37 examples/s]
Tokenizing train dataset:  42%|████▏     | 768237/1807941 [10:07<22:37, 765.82 examples/s] 
Tokenizing train dataset:  43%|████▎     | 768472/1807941 [10:07<21:51, 792.62 examples/s]
Tokenizing train dataset:  43%|████▎     | 768660/1807941 [10:07<21:17, 813.52 examples/s]
Tokenizing train dataset:  43%|████▎     | 768813/1807941 [10:07<21:44, 796.33 examples/s]
Tokenizing train dataset:  43%|████▎     | 768951/1807941 [10:08<24:04, 719.18 examples/s]
Tokenizing train dataset:  43%|████▎     | 769068/1807941 [10:08<25:01, 692.10 examples/s]
Tokenizing train dataset:  43%|████▎     | 769184/1807941 [10:08<23:13, 745.61 examples/s]
Tokenizing train dataset:  43%|████▎     | 769320/1807941 [10:08<22:20, 774.69 examples/s]
Tokenizing train dataset:  43%|████▎     | 769419/1807941 [10:08<23:20, 741.31 examples/s]
Tokenizing train dataset:  43%|████▎     | 769518/1807941 [10:08<22:04, 784.03 examples/s]
Tokenizing train dataset:  43%|████▎     | 769628/1807941 [10:08<21:00, 823.65 examples/s]
Tokenizing train dataset:  43%|████▎     | 769753/1807941 [10:09<21:10, 817.23 examples/s]
Tokenizing train dataset:  43%|████▎     | 769848/1807941 [10:09<27:26, 630.62 examples/s]
Tokenizing train dataset:  43%|████▎     | 769940/1807941 [10:09<27:14, 635.00 examples/s]
Tokenizing train dataset:  43%|████▎     | 770054/1807941 [10:09<27:12, 635.71 examples/s]
Tokenizing train dataset:  43%|████▎     | 770125/1807941 [10:09<27:43, 623.87 examples/s]
Tokenizing train dataset:  43%|████▎     | 770208/1807941 [10:09<29:57, 577.44 examples/s]
Tokenizing train dataset:  43%|████▎     | 770270/1807941 [10:10<29:48, 580.29 examples/s]
Tokenizing train dataset:  43%|████▎     | 770333/1807941 [10:10<37:31, 460.86 examples/s]
Tokenizing train dataset:  43%|████▎     | 770396/1807941 [10:10<39:07, 441.94 examples/s]
Tokenizing train dataset:  43%|████▎     | 770479/1807941 [10:10<35:00, 494.00 examples/s]
Tokenizing train dataset:  43%|████▎     | 770536/1807941 [10:10<41:16, 418.96 examples/s]
Tokenizing train dataset:  43%|████▎     | 770608/1807941 [10:10<36:44, 470.59 examples/s]
Tokenizing train dataset:  43%|████▎     | 770676/1807941 [10:11<49:13, 351.17 examples/s]
Tokenizing train dataset:  43%|████▎     | 770759/1807941 [10:11<41:39, 414.99 examples/s]
Tokenizing train dataset:  43%|████▎     | 770840/1807941 [10:11<36:57, 467.62 examples/s]
Tokenizing train dataset:  43%|████▎     | 770896/1807941 [10:11<35:36, 485.30 examples/s]
Tokenizing train dataset:  43%|████▎     | 770992/1807941 [10:11<29:14, 590.97 examples/s]
Tokenizing train dataset:  43%|████▎     | 771105/1807941 [10:11<34:40, 498.46 examples/s]
Tokenizing train dataset:  43%|████▎     | 771183/1807941 [10:12<52:43, 327.77 examples/s]
Tokenizing train dataset:  43%|████▎     | 771230/1807941 [10:12<56:10, 307.55 examples/s]
Tokenizing train dataset:  43%|████▎     | 771287/1807941 [10:12<59:28, 290.48 examples/s]
Tokenizing train dataset:  43%|████▎     | 771340/1807941 [10:12<1:01:40, 280.12 examples/s]
Tokenizing train dataset:  43%|████▎     | 771389/1807941 [10:13<58:59, 292.87 examples/s]  
Tokenizing train dataset:  43%|████▎     | 771433/1807941 [10:13<55:19, 312.21 examples/s]
Tokenizing train dataset:  43%|████▎     | 771485/1807941 [10:13<57:59, 297.89 examples/s]
Tokenizing train dataset:  43%|████▎     | 771522/1807941 [10:13<57:17, 301.52 examples/s]
Tokenizing train dataset:  43%|████▎     | 771607/1807941 [10:13<42:54, 402.54 examples/s]
Tokenizing train dataset:  43%|████▎     | 771656/1807941 [10:13<55:09, 313.16 examples/s]
Tokenizing train dataset:  43%|████▎     | 771711/1807941 [10:14<1:04:00, 269.85 examples/s]
Tokenizing train dataset:  43%|████▎     | 771744/1807941 [10:14<1:01:55, 278.86 examples/s]
Tokenizing train dataset:  43%|████▎     | 771819/1807941 [10:14<48:33, 355.59 examples/s]  
Tokenizing train dataset:  43%|████▎     | 771868/1807941 [10:14<1:13:11, 235.94 examples/s]
Tokenizing train dataset:  43%|████▎     | 771938/1807941 [10:15<1:07:09, 257.08 examples/s]
Tokenizing train dataset:  43%|████▎     | 772000/1807941 [10:15<1:04:33, 267.46 examples/s]
Tokenizing train dataset:  43%|████▎     | 772071/1807941 [10:15<1:18:53, 218.84 examples/s]
Tokenizing train dataset:  43%|████▎     | 772171/1807941 [10:15<54:50, 314.82 examples/s]  
Tokenizing train dataset:  43%|████▎     | 772246/1807941 [10:16<52:27, 329.01 examples/s]
Tokenizing train dataset:  43%|████▎     | 772365/1807941 [10:16<37:11, 464.08 examples/s]
Tokenizing train dataset:  43%|████▎     | 772465/1807941 [10:16<53:16, 323.92 examples/s]
Tokenizing train dataset:  43%|████▎     | 772544/1807941 [10:16<47:59, 359.63 examples/s]
Tokenizing train dataset:  43%|████▎     | 772648/1807941 [10:17<50:39, 340.57 examples/s]
Tokenizing train dataset:  43%|████▎     | 772708/1807941 [10:17<49:40, 347.30 examples/s]
Tokenizing train dataset:  43%|████▎     | 772756/1807941 [10:17<1:19:35, 216.78 examples/s]
Tokenizing train dataset:  43%|████▎     | 772802/1807941 [10:18<2:08:06, 134.67 examples/s]
Tokenizing train dataset:  43%|████▎     | 772834/1807941 [10:18<1:55:40, 149.15 examples/s]
Tokenizing train dataset:  43%|████▎     | 772863/1807941 [10:18<1:54:18, 150.92 examples/s]
Tokenizing train dataset:  43%|████▎     | 772949/1807941 [10:19<1:23:54, 205.60 examples/s]
Tokenizing train dataset:  43%|████▎     | 772980/1807941 [10:19<1:26:43, 198.88 examples/s]
Tokenizing train dataset:  43%|████▎     | 773006/1807941 [10:19<1:55:22, 149.50 examples/s]
Tokenizing train dataset:  43%|████▎     | 773045/1807941 [10:19<1:54:02, 151.26 examples/s]
Tokenizing train dataset:  43%|████▎     | 773167/1807941 [10:20<1:02:00, 278.15 examples/s]
Tokenizing train dataset:  43%|████▎     | 773209/1807941 [10:20<1:40:51, 170.99 examples/s]
Tokenizing train dataset:  43%|████▎     | 773241/1807941 [10:20<1:39:05, 174.02 examples/s]
Tokenizing train dataset:  43%|████▎     | 773306/1807941 [10:21<1:20:48, 213.39 examples/s]
Tokenizing train dataset:  43%|████▎     | 773346/1807941 [10:21<1:57:42, 146.49 examples/s]
Tokenizing train dataset:  43%|████▎     | 773457/1807941 [10:21<1:08:11, 252.82 examples/s]
Tokenizing train dataset:  43%|████▎     | 773561/1807941 [10:21<49:44, 346.53 examples/s]  
Tokenizing train dataset:  43%|████▎     | 773649/1807941 [10:22<53:53, 319.88 examples/s]
Tokenizing train dataset:  43%|████▎     | 773707/1807941 [10:22<51:06, 337.22 examples/s]
Tokenizing train dataset:  43%|████▎     | 773754/1807941 [10:22<1:05:58, 261.23 examples/s]
Tokenizing train dataset:  43%|████▎     | 773863/1807941 [10:22<53:54, 319.71 examples/s]  
Tokenizing train dataset:  43%|████▎     | 773916/1807941 [10:23<1:26:22, 199.53 examples/s]
Tokenizing train dataset:  43%|████▎     | 773964/1807941 [10:23<1:17:13, 223.15 examples/s]
Tokenizing train dataset:  43%|████▎     | 774000/1807941 [10:23<1:14:03, 232.69 examples/s][transformers] Token indices sequence length is longer than the specified maximum sequence length for this model (153397 > 131072). Running this sequence through the model will result in indexing errors

Tokenizing train dataset:  43%|████▎     | 774050/1807941 [10:25<3:06:27, 92.42 examples/s] 
Tokenizing train dataset:  43%|████▎     | 774085/1807941 [10:25<2:38:31, 108.69 examples/s]
Tokenizing train dataset:  43%|████▎     | 774134/1807941 [10:25<2:01:58, 141.26 examples/s]
Tokenizing train dataset:  43%|████▎     | 774168/1807941 [10:25<2:07:20, 135.29 examples/s]
Tokenizing train dataset:  43%|████▎     | 774213/1807941 [10:25<1:41:01, 170.54 examples/s]
Tokenizing train dataset:  43%|████▎     | 774244/1807941 [10:25<1:46:19, 162.04 examples/s]
Tokenizing train dataset:  43%|████▎     | 774298/1807941 [10:26<1:22:04, 209.91 examples/s]
Tokenizing train dataset:  43%|████▎     | 774340/1807941 [10:26<1:25:15, 202.04 examples/s]
Tokenizing train dataset:  43%|████▎     | 774409/1807941 [10:26<1:03:46, 270.13 examples/s]
Tokenizing train dataset:  43%|████▎     | 774488/1807941 [10:26<1:13:50, 233.24 examples/s]
Tokenizing train dataset:  43%|████▎     | 774550/1807941 [10:27<1:09:39, 247.26 examples/s]
Tokenizing train dataset:  43%|████▎     | 774620/1807941 [10:27<58:13, 295.81 examples/s]  
Tokenizing train dataset:  43%|████▎     | 774715/1807941 [10:27<46:45, 368.24 examples/s]
Tokenizing train dataset:  43%|████▎     | 774816/1807941 [10:27<45:51, 375.44 examples/s]
Tokenizing train dataset:  43%|████▎     | 774941/1807941 [10:27<34:13, 503.09 examples/s]
Tokenizing train dataset:  43%|████▎     | 775050/1807941 [10:28<42:45, 402.67 examples/s]
Tokenizing train dataset:  43%|████▎     | 775103/1807941 [10:28<59:05, 291.29 examples/s]
Tokenizing train dataset:  43%|████▎     | 775172/1807941 [10:28<58:10, 295.92 examples/s]
Tokenizing train dataset:  43%|████▎     | 775209/1807941 [10:28<1:02:11, 276.75 examples/s]
Tokenizing train dataset:  43%|████▎     | 775247/1807941 [10:29<1:00:28, 284.64 examples/s]
Tokenizing train dataset:  43%|████▎     | 775292/1807941 [10:29<55:18, 311.21 examples/s]  
Tokenizing train dataset:  43%|████▎     | 775375/1807941 [10:29<1:02:03, 277.29 examples/s]
Tokenizing train dataset:  43%|████▎     | 775460/1807941 [10:29<49:33, 347.25 examples/s]  
Tokenizing train dataset:  43%|████▎     | 775525/1807941 [10:29<58:24, 294.60 examples/s]
Tokenizing train dataset:  43%|████▎     | 775609/1807941 [10:30<57:05, 301.38 examples/s]
Tokenizing train dataset:  43%|████▎     | 775660/1807941 [10:30<58:01, 296.46 examples/s]
Tokenizing train dataset:  43%|████▎     | 775720/1807941 [10:30<50:13, 342.58 examples/s]
Tokenizing train dataset:  43%|████▎     | 775821/1807941 [10:30<37:16, 461.53 examples/s]
Tokenizing train dataset:  43%|████▎     | 775892/1807941 [10:30<50:45, 338.91 examples/s]
Tokenizing train dataset:  43%|████▎     | 775947/1807941 [10:31<58:02, 296.31 examples/s]
Tokenizing train dataset:  43%|████▎     | 776000/1807941 [10:31<57:58, 296.65 examples/s]
Tokenizing train dataset:  43%|████▎     | 776088/1807941 [10:31<44:07, 389.69 examples/s]
Tokenizing train dataset:  43%|████▎     | 776140/1807941 [10:31<41:52, 410.68 examples/s]
Tokenizing train dataset:  43%|████▎     | 776252/1807941 [10:31<32:25, 530.34 examples/s]
Tokenizing train dataset:  43%|████▎     | 776326/1807941 [10:31<33:29, 513.25 examples/s]
Tokenizing train dataset:  43%|████▎     | 776413/1807941 [10:31<29:05, 591.08 examples/s]
Tokenizing train dataset:  43%|████▎     | 776531/1807941 [10:32<26:47, 641.46 examples/s]
Tokenizing train dataset:  43%|████▎     | 776625/1807941 [10:32<24:27, 702.89 examples/s]
Tokenizing train dataset:  43%|████▎     | 776739/1807941 [10:32<36:36, 469.43 examples/s]
Tokenizing train dataset:  43%|████▎     | 776828/1807941 [10:32<32:30, 528.56 examples/s]
Tokenizing train dataset:  43%|████▎     | 776903/1807941 [10:33<1:00:30, 283.99 examples/s]
Tokenizing train dataset:  43%|████▎     | 776967/1807941 [10:33<53:03, 323.89 examples/s]  
Tokenizing train dataset:  43%|████▎     | 777045/1807941 [10:33<48:14, 356.10 examples/s]
Tokenizing train dataset:  43%|████▎     | 777154/1807941 [10:33<36:21, 472.47 examples/s]
Tokenizing train dataset:  43%|████▎     | 777260/1807941 [10:34<46:03, 373.00 examples/s]
Tokenizing train dataset:  43%|████▎     | 777359/1807941 [10:34<40:01, 429.22 examples/s]
Tokenizing train dataset:  43%|████▎     | 777458/1807941 [10:34<34:26, 498.74 examples/s]
Tokenizing train dataset:  43%|████▎     | 777561/1807941 [10:34<34:16, 501.08 examples/s]
Tokenizing train dataset:  43%|████▎     | 777679/1807941 [10:34<29:43, 577.68 examples/s]
Tokenizing train dataset:  43%|████▎     | 777757/1807941 [10:34<28:00, 613.04 examples/s]
Tokenizing train dataset:  43%|████▎     | 777891/1807941 [10:35<42:37, 402.68 examples/s]
Tokenizing train dataset:  43%|████▎     | 778000/1807941 [10:35<42:26, 404.40 examples/s]
Tokenizing train dataset:  43%|████▎     | 778073/1807941 [10:35<39:43, 432.03 examples/s]
Tokenizing train dataset:  43%|████▎     | 778166/1807941 [10:35<36:12, 473.92 examples/s]
Tokenizing train dataset:  43%|████▎     | 778224/1807941 [10:36<36:44, 467.16 examples/s]
Tokenizing train dataset:  43%|████▎     | 778311/1807941 [10:36<31:41, 541.49 examples/s]
Tokenizing train dataset:  43%|████▎     | 778428/1807941 [10:36<34:19, 499.98 examples/s]
Tokenizing train dataset:  43%|████▎     | 778546/1807941 [10:36<30:09, 568.96 examples/s]
Tokenizing train dataset:  43%|████▎     | 778631/1807941 [10:36<32:26, 528.71 examples/s]
Tokenizing train dataset:  43%|████▎     | 778758/1807941 [10:36<25:41, 667.72 examples/s]
Tokenizing train dataset:  43%|████▎     | 778858/1807941 [10:37<26:47, 640.26 examples/s]
Tokenizing train dataset:  43%|████▎     | 778944/1807941 [10:37<43:10, 397.27 examples/s]
Tokenizing train dataset:  43%|████▎     | 779011/1807941 [10:38<1:28:04, 194.71 examples/s]
Tokenizing train dataset:  43%|████▎     | 779129/1807941 [10:38<1:02:28, 274.49 examples/s]
Tokenizing train dataset:  43%|████▎     | 779196/1807941 [10:38<1:00:34, 283.07 examples/s]
Tokenizing train dataset:  43%|████▎     | 779317/1807941 [10:38<48:53, 350.66 examples/s]  
Tokenizing train dataset:  43%|████▎     | 779376/1807941 [10:39<54:26, 314.91 examples/s]
Tokenizing train dataset:  43%|████▎     | 779446/1807941 [10:39<47:34, 360.25 examples/s]
Tokenizing train dataset:  43%|████▎     | 779559/1807941 [10:39<37:28, 457.36 examples/s]
Tokenizing train dataset:  43%|████▎     | 779619/1807941 [10:39<45:16, 378.54 examples/s]
Tokenizing train dataset:  43%|████▎     | 779727/1807941 [10:39<34:39, 494.56 examples/s]
Tokenizing train dataset:  43%|████▎     | 779803/1807941 [10:40<35:12, 486.59 examples/s]
Tokenizing train dataset:  43%|████▎     | 779962/1807941 [10:40<25:37, 668.49 examples/s]
Tokenizing train dataset:  43%|████▎     | 780091/1807941 [10:40<29:03, 589.46 examples/s]
Tokenizing train dataset:  43%|████▎     | 780243/1807941 [10:40<32:14, 531.33 examples/s]
Tokenizing train dataset:  43%|████▎     | 780331/1807941 [10:41<37:44, 453.83 examples/s]
Tokenizing train dataset:  43%|████▎     | 780446/1807941 [10:41<30:55, 553.80 examples/s]
Tokenizing train dataset:  43%|████▎     | 780547/1807941 [10:41<29:52, 573.03 examples/s]
Tokenizing train dataset:  43%|████▎     | 780699/1807941 [10:41<22:59, 744.92 examples/s]
Tokenizing train dataset:  43%|████▎     | 780805/1807941 [10:41<31:24, 545.09 examples/s]
Tokenizing train dataset:  43%|████▎     | 780955/1807941 [10:42<37:48, 452.75 examples/s]
Tokenizing train dataset:  43%|████▎     | 781032/1807941 [10:42<37:27, 456.93 examples/s]
Tokenizing train dataset:  43%|████▎     | 781103/1807941 [10:42<46:49, 365.43 examples/s]
Tokenizing train dataset:  43%|████▎     | 781224/1807941 [10:42<43:11, 396.13 examples/s]
Tokenizing train dataset:  43%|████▎     | 781281/1807941 [10:43<40:45, 419.80 examples/s]
Tokenizing train dataset:  43%|████▎     | 781374/1807941 [10:43<34:21, 498.02 examples/s]
Tokenizing train dataset:  43%|████▎     | 781455/1807941 [10:43<41:19, 413.93 examples/s]
Tokenizing train dataset:  43%|████▎     | 781564/1807941 [10:44<1:18:30, 217.88 examples/s]
Tokenizing train dataset:  43%|████▎     | 781621/1807941 [10:44<1:10:49, 241.51 examples/s]
Tokenizing train dataset:  43%|████▎     | 781707/1807941 [10:44<1:02:56, 271.76 examples/s]
Tokenizing train dataset:  43%|████▎     | 781778/1807941 [10:44<53:10, 321.59 examples/s]  
Tokenizing train dataset:  43%|████▎     | 781883/1807941 [10:45<53:00, 322.58 examples/s]
Tokenizing train dataset:  43%|████▎     | 781941/1807941 [10:45<48:03, 355.78 examples/s]
Tokenizing train dataset:  43%|████▎     | 782000/1807941 [10:45<47:54, 356.94 examples/s]
Tokenizing train dataset:  43%|████▎     | 782050/1807941 [10:45<1:01:06, 279.84 examples/s]
Tokenizing train dataset:  43%|████▎     | 782125/1807941 [10:45<48:43, 350.84 examples/s]  
Tokenizing train dataset:  43%|████▎     | 782194/1807941 [10:45<44:04, 387.93 examples/s]
Tokenizing train dataset:  43%|████▎     | 782261/1807941 [10:46<1:09:25, 246.21 examples/s]
Tokenizing train dataset:  43%|████▎     | 782316/1807941 [10:46<1:00:19, 283.36 examples/s]
Tokenizing train dataset:  43%|████▎     | 782381/1807941 [10:46<58:38, 291.50 examples/s]  
Tokenizing train dataset:  43%|████▎     | 782486/1807941 [10:46<44:02, 388.12 examples/s]
Tokenizing train dataset:  43%|████▎     | 782578/1807941 [10:47<43:07, 396.30 examples/s]
Tokenizing train dataset:  43%|████▎     | 782638/1807941 [10:47<1:08:17, 250.21 examples/s]
Tokenizing train dataset:  43%|████▎     | 782697/1807941 [10:47<1:00:37, 281.87 examples/s]
Tokenizing train dataset:  43%|████▎     | 782813/1807941 [10:47<43:02, 397.03 examples/s]  
Tokenizing train dataset:  43%|████▎     | 782869/1807941 [10:48<54:47, 311.82 examples/s]
Tokenizing train dataset:  43%|████▎     | 782927/1807941 [10:48<49:00, 348.62 examples/s]
Tokenizing train dataset:  43%|████▎     | 783000/1807941 [10:48<43:10, 395.70 examples/s]
Tokenizing train dataset:  43%|████▎     | 783066/1807941 [10:48<42:12, 404.62 examples/s]
Tokenizing train dataset:  43%|████▎     | 783166/1807941 [10:48<32:41, 522.56 examples/s]
Tokenizing train dataset:  43%|████▎     | 783230/1807941 [10:49<47:18, 360.97 examples/s]
Tokenizing train dataset:  43%|████▎     | 783281/1807941 [10:49<52:30, 325.20 examples/s]
Tokenizing train dataset:  43%|████▎     | 783356/1807941 [10:49<48:47, 350.00 examples/s]
Tokenizing train dataset:  43%|████▎     | 783448/1807941 [10:49<38:10, 447.38 examples/s]
Tokenizing train dataset:  43%|████▎     | 783510/1807941 [10:49<37:35, 454.13 examples/s]
Tokenizing train dataset:  43%|████▎     | 783628/1807941 [10:49<28:48, 592.46 examples/s]
Tokenizing train dataset:  43%|████▎     | 783701/1807941 [10:50<38:40, 441.46 examples/s]
Tokenizing train dataset:  43%|████▎     | 783769/1807941 [10:50<41:45, 408.73 examples/s]
Tokenizing train dataset:  43%|████▎     | 783881/1807941 [10:50<35:58, 474.47 examples/s]
Tokenizing train dataset:  43%|████▎     | 783940/1807941 [10:50<38:49, 439.61 examples/s]
Tokenizing train dataset:  43%|████▎     | 784000/1807941 [10:50<38:38, 441.65 examples/s]
Tokenizing train dataset:  43%|████▎     | 784098/1807941 [10:50<31:06, 548.41 examples/s]
Tokenizing train dataset:  43%|████▎     | 784203/1807941 [10:51<35:50, 476.05 examples/s]
Tokenizing train dataset:  43%|████▎     | 784270/1807941 [10:51<58:17, 292.69 examples/s]
Tokenizing train dataset:  43%|████▎     | 784319/1807941 [10:52<1:26:00, 198.37 examples/s]
Tokenizing train dataset:  43%|████▎     | 784356/1807941 [10:52<1:25:42, 199.03 examples/s]
Tokenizing train dataset:  43%|████▎     | 784390/1807941 [10:52<1:57:32, 145.14 examples/s]
Tokenizing train dataset:  43%|████▎     | 784418/1807941 [10:53<1:54:37, 148.83 examples/s]
Tokenizing train dataset:  43%|████▎     | 784452/1807941 [10:53<2:46:18, 102.57 examples/s]
Tokenizing train dataset:  43%|████▎     | 784485/1807941 [10:53<2:19:38, 122.15 examples/s]
Tokenizing train dataset:  43%|████▎     | 784525/1807941 [10:53<1:54:22, 149.12 examples/s]
Tokenizing train dataset:  43%|████▎     | 784572/1807941 [10:54<1:42:52, 165.80 examples/s]
Tokenizing train dataset:  43%|████▎     | 784647/1807941 [10:54<1:13:26, 232.24 examples/s]
Tokenizing train dataset:  43%|████▎     | 784711/1807941 [10:54<1:07:24, 252.98 examples/s]
Tokenizing train dataset:  43%|████▎     | 784777/1807941 [10:54<54:22, 313.62 examples/s]  
Tokenizing train dataset:  43%|████▎     | 784826/1807941 [10:54<52:21, 325.63 examples/s]
Tokenizing train dataset:  43%|████▎     | 784879/1807941 [10:54<51:34, 330.63 examples/s]
Tokenizing train dataset:  43%|████▎     | 784979/1807941 [10:55<37:08, 459.11 examples/s]
Tokenizing train dataset:  43%|████▎     | 785053/1807941 [10:55<1:03:47, 267.23 examples/s]
Tokenizing train dataset:  43%|████▎     | 785105/1807941 [10:55<1:04:15, 265.26 examples/s]
Tokenizing train dataset:  43%|████▎     | 785150/1807941 [10:56<1:16:59, 221.39 examples/s]
Tokenizing train dataset:  43%|████▎     | 785286/1807941 [10:56<45:06, 377.85 examples/s]  
Tokenizing train dataset:  43%|████▎     | 785400/1807941 [10:56<33:56, 502.20 examples/s]
Tokenizing train dataset:  43%|████▎     | 785512/1807941 [10:56<28:02, 607.53 examples/s]
Tokenizing train dataset:  43%|████▎     | 785609/1807941 [10:56<27:58, 608.93 examples/s]
Tokenizing train dataset:  43%|████▎     | 785710/1807941 [10:56<25:07, 677.94 examples/s]
Tokenizing train dataset:  43%|████▎     | 785805/1807941 [10:56<27:39, 615.91 examples/s]
Tokenizing train dataset:  43%|████▎     | 785924/1807941 [10:57<26:44, 636.83 examples/s]
Tokenizing train dataset:  43%|████▎     | 786000/1807941 [10:57<26:45, 636.71 examples/s]
Tokenizing train dataset:  43%|████▎     | 786118/1807941 [10:57<22:47, 747.02 examples/s]
Tokenizing train dataset:  43%|████▎     | 786203/1807941 [10:57<25:53, 657.64 examples/s]
Tokenizing train dataset:  43%|████▎     | 786337/1807941 [10:57<23:49, 714.46 examples/s]
Tokenizing train dataset:  44%|████▎     | 786502/1807941 [10:57<18:33, 917.48 examples/s]
Tokenizing train dataset:  44%|████▎     | 786617/1807941 [10:57<22:28, 757.46 examples/s]
Tokenizing train dataset:  44%|████▎     | 786775/1807941 [10:58<20:16, 839.75 examples/s]
Tokenizing train dataset:  44%|████▎     | 786877/1807941 [10:58<22:50, 745.01 examples/s]
Tokenizing train dataset:  44%|████▎     | 787000/1807941 [10:58<23:08, 735.15 examples/s]
Tokenizing train dataset:  44%|████▎     | 787097/1807941 [10:58<30:07, 564.76 examples/s]
Tokenizing train dataset:  44%|████▎     | 787213/1807941 [10:58<25:28, 667.65 examples/s]
Tokenizing train dataset:  44%|████▎     | 787312/1807941 [10:59<26:55, 631.60 examples/s]
Tokenizing train dataset:  44%|████▎     | 787469/1807941 [10:59<20:52, 815.02 examples/s]
Tokenizing train dataset:  44%|████▎     | 787568/1807941 [10:59<20:03, 847.49 examples/s]
Tokenizing train dataset:  44%|████▎     | 787711/1807941 [10:59<19:22, 877.91 examples/s]
Tokenizing train dataset:  44%|████▎     | 787824/1807941 [10:59<18:14, 931.88 examples/s]
Tokenizing train dataset:  44%|████▎     | 787949/1807941 [10:59<25:34, 664.55 examples/s]
Tokenizing train dataset:  44%|████▎     | 788036/1807941 [10:59<28:55, 587.76 examples/s]
Tokenizing train dataset:  44%|████▎     | 788131/1807941 [11:00<32:24, 524.51 examples/s]
Tokenizing train dataset:  44%|████▎     | 788225/1807941 [11:00<29:41, 572.43 examples/s]
Tokenizing train dataset:  44%|████▎     | 788306/1807941 [11:00<30:24, 558.82 examples/s]
Tokenizing train dataset:  44%|████▎     | 788368/1807941 [11:00<29:49, 569.63 examples/s]
Tokenizing train dataset:  44%|████▎     | 788458/1807941 [11:00<31:33, 538.38 examples/s]
Tokenizing train dataset:  44%|████▎     | 788522/1807941 [11:00<31:10, 545.00 examples/s]
Tokenizing train dataset:  44%|████▎     | 788618/1807941 [11:00<26:40, 637.04 examples/s]
Tokenizing train dataset:  44%|████▎     | 788770/1807941 [11:01<19:58, 850.69 examples/s]
Tokenizing train dataset:  44%|████▎     | 789000/1807941 [11:01<14:23, 1180.35 examples/s]
Tokenizing train dataset:  44%|████▎     | 789281/1807941 [11:01<10:34, 1606.58 examples/s]
Tokenizing train dataset:  44%|████▎     | 789563/1807941 [11:01<08:46, 1935.42 examples/s]
Tokenizing train dataset:  44%|████▎     | 789837/1807941 [11:01<07:52, 2156.89 examples/s]
Tokenizing train dataset:  44%|████▎     | 790088/1807941 [11:01<07:31, 2254.66 examples/s]
Tokenizing train dataset:  44%|████▎     | 790359/1807941 [11:01<07:06, 2384.11 examples/s]
Tokenizing train dataset:  44%|████▎     | 790625/1807941 [11:01<06:52, 2463.75 examples/s]
Tokenizing train dataset:  44%|████▎     | 790907/1807941 [11:01<06:36, 2565.42 examples/s]
Tokenizing train dataset:  44%|████▍     | 791297/1807941 [11:02<06:34, 2574.62 examples/s]
Tokenizing train dataset:  44%|████▍     | 791567/1807941 [11:02<06:30, 2605.31 examples/s]
Tokenizing train dataset:  44%|████▍     | 791855/1807941 [11:02<06:18, 2681.06 examples/s]
Tokenizing train dataset:  44%|████▍     | 792246/1807941 [11:02<06:23, 2648.51 examples/s]
Tokenizing train dataset:  44%|████▍     | 792640/1807941 [11:02<06:25, 2636.96 examples/s]
Tokenizing train dataset:  44%|████▍     | 792917/1807941 [11:02<06:20, 2665.40 examples/s]
Tokenizing train dataset:  44%|████▍     | 793284/1807941 [11:02<06:32, 2588.26 examples/s]
Tokenizing train dataset:  44%|████▍     | 793626/1807941 [11:02<06:49, 2479.61 examples/s]
Tokenizing train dataset:  44%|████▍     | 793947/1807941 [11:03<07:08, 2368.24 examples/s]
Tokenizing train dataset:  44%|████▍     | 794303/1807941 [11:03<07:08, 2367.63 examples/s]
Tokenizing train dataset:  44%|████▍     | 794575/1807941 [11:03<06:54, 2444.15 examples/s]
Tokenizing train dataset:  44%|████▍     | 794852/1807941 [11:03<06:41, 2523.16 examples/s]
Tokenizing train dataset:  44%|████▍     | 795246/1807941 [11:03<06:35, 2557.76 examples/s]
Tokenizing train dataset:  44%|████▍     | 795617/1807941 [11:03<06:40, 2525.93 examples/s]
Tokenizing train dataset:  44%|████▍     | 795898/1807941 [11:03<06:30, 2591.81 examples/s]
Tokenizing train dataset:  44%|████▍     | 796282/1807941 [11:04<06:32, 2578.56 examples/s]
Tokenizing train dataset:  44%|████▍     | 796586/1807941 [11:04<06:16, 2687.15 examples/s]
Tokenizing train dataset:  44%|████▍     | 796883/1807941 [11:04<06:06, 2756.53 examples/s]
Tokenizing train dataset:  44%|████▍     | 797269/1807941 [11:04<06:16, 2682.52 examples/s]
Tokenizing train dataset:  44%|████▍     | 797639/1807941 [11:04<06:27, 2606.67 examples/s]
Tokenizing train dataset:  44%|████▍     | 797937/1807941 [11:04<06:14, 2696.31 examples/s]
Tokenizing train dataset:  44%|████▍     | 798333/1807941 [11:04<06:17, 2673.57 examples/s]
Tokenizing train dataset:  44%|████▍     | 798607/1807941 [11:04<06:15, 2688.57 examples/s]
Tokenizing train dataset:  44%|████▍     | 798888/1807941 [11:04<06:11, 2718.42 examples/s]
Tokenizing train dataset:  44%|████▍     | 799236/1807941 [11:05<06:34, 2554.43 examples/s]
Tokenizing train dataset:  44%|████▍     | 799623/1807941 [11:05<06:33, 2561.22 examples/s]
Tokenizing train dataset:  44%|████▍     | 799900/1807941 [11:05<06:26, 2608.77 examples/s]
Tokenizing train dataset:  44%|████▍     | 800285/1807941 [11:05<06:28, 2591.72 examples/s]
Tokenizing train dataset:  44%|████▍     | 800573/1807941 [11:05<06:18, 2661.05 examples/s]
Tokenizing train dataset:  44%|████▍     | 800962/1807941 [11:05<06:22, 2632.01 examples/s]
Tokenizing train dataset:  44%|████▍     | 801330/1807941 [11:05<06:31, 2569.18 examples/s]
Tokenizing train dataset:  44%|████▍     | 801605/1807941 [11:06<06:25, 2608.70 examples/s]
Tokenizing train dataset:  44%|████▍     | 801881/1807941 [11:06<06:20, 2642.75 examples/s]
Tokenizing train dataset:  44%|████▍     | 802288/1807941 [11:06<06:19, 2650.26 examples/s]
Tokenizing train dataset:  44%|████▍     | 802575/1807941 [11:06<06:12, 2701.82 examples/s]
Tokenizing train dataset:  44%|████▍     | 802871/1807941 [11:06<06:51, 2442.47 examples/s]
Tokenizing train dataset:  44%|████▍     | 803127/1807941 [11:06<06:53, 2429.42 examples/s]
Tokenizing train dataset:  44%|████▍     | 803407/1807941 [11:06<06:38, 2522.14 examples/s]
Tokenizing train dataset:  44%|████▍     | 803679/1807941 [11:06<06:31, 2567.37 examples/s]
Tokenizing train dataset:  44%|████▍     | 804039/1807941 [11:06<06:40, 2503.76 examples/s]
Tokenizing train dataset:  44%|████▍     | 804311/1807941 [11:07<06:32, 2555.40 examples/s]
Tokenizing train dataset:  45%|████▍     | 804606/1807941 [11:07<06:17, 2657.07 examples/s]
Tokenizing train dataset:  45%|████▍     | 804881/1807941 [11:07<06:14, 2678.69 examples/s]
Tokenizing train dataset:  45%|████▍     | 805260/1807941 [11:07<06:25, 2600.94 examples/s]
Tokenizing train dataset:  45%|████▍     | 805534/1807941 [11:07<06:20, 2634.06 examples/s]
Tokenizing train dataset:  45%|████▍     | 805925/1807941 [11:07<06:22, 2622.03 examples/s]
Tokenizing train dataset:  45%|████▍     | 806192/1807941 [11:07<06:20, 2632.19 examples/s]
Tokenizing train dataset:  45%|████▍     | 806474/1807941 [11:07<06:13, 2680.19 examples/s]
Tokenizing train dataset:  45%|████▍     | 806757/1807941 [11:08<06:08, 2715.89 examples/s]
Tokenizing train dataset:  45%|████▍     | 807049/1807941 [11:08<06:01, 2772.01 examples/s]
Tokenizing train dataset:  45%|████▍     | 807398/1807941 [11:08<06:25, 2597.22 examples/s]
Tokenizing train dataset:  45%|████▍     | 807709/1807941 [11:08<06:06, 2731.00 examples/s]
Tokenizing train dataset:  45%|████▍     | 808017/1807941 [11:08<05:54, 2824.20 examples/s]
Tokenizing train dataset:  45%|████▍     | 808392/1807941 [11:08<06:09, 2703.34 examples/s]
Tokenizing train dataset:  45%|████▍     | 808801/1807941 [11:08<06:09, 2704.85 examples/s]
Tokenizing train dataset:  45%|████▍     | 809239/1807941 [11:08<06:00, 2774.15 examples/s]
Tokenizing train dataset:  45%|████▍     | 809530/1807941 [11:09<05:55, 2805.58 examples/s]
Tokenizing train dataset:  45%|████▍     | 809946/1807941 [11:09<05:57, 2790.43 examples/s]
Tokenizing train dataset:  45%|████▍     | 810318/1807941 [11:09<06:11, 2685.59 examples/s]
Tokenizing train dataset:  45%|████▍     | 810600/1807941 [11:09<06:07, 2714.22 examples/s]
Tokenizing train dataset:  45%|████▍     | 810901/1807941 [11:09<05:57, 2787.45 examples/s]
Tokenizing train dataset:  45%|████▍     | 811311/1807941 [11:09<06:00, 2763.55 examples/s]
Tokenizing train dataset:  45%|████▍     | 811725/1807941 [11:09<06:01, 2758.84 examples/s]
Tokenizing train dataset:  45%|████▍     | 812169/1807941 [11:09<05:53, 2817.46 examples/s]
Tokenizing train dataset:  45%|████▍     | 812469/1807941 [11:10<05:48, 2857.22 examples/s]
Tokenizing train dataset:  45%|████▍     | 812925/1807941 [11:10<05:41, 2913.40 examples/s]
Tokenizing train dataset:  45%|████▍     | 813241/1807941 [11:10<08:21, 1982.98 examples/s]
Tokenizing train dataset:  45%|████▍     | 813505/1807941 [11:10<10:10, 1629.93 examples/s]
Tokenizing train dataset:  45%|████▌     | 813819/1807941 [11:11<11:04, 1496.16 examples/s]
Tokenizing train dataset:  45%|████▌     | 814058/1807941 [11:11<10:06, 1639.97 examples/s]
Tokenizing train dataset:  45%|████▌     | 814349/1807941 [11:11<08:49, 1876.72 examples/s]
Tokenizing train dataset:  45%|████▌     | 814663/1807941 [11:11<08:33, 1935.82 examples/s]
Tokenizing train dataset:  45%|████▌     | 814905/1807941 [11:11<08:07, 2038.77 examples/s]
Tokenizing train dataset:  45%|████▌     | 815138/1807941 [11:11<08:00, 2067.93 examples/s]
Tokenizing train dataset:  45%|████▌     | 815426/1807941 [11:11<07:17, 2267.55 examples/s]
Tokenizing train dataset:  45%|████▌     | 815679/1807941 [11:11<07:05, 2329.49 examples/s]
Tokenizing train dataset:  45%|████▌     | 815971/1807941 [11:11<06:38, 2486.55 examples/s]
Tokenizing train dataset:  45%|████▌     | 816346/1807941 [11:12<06:39, 2484.65 examples/s]
Tokenizing train dataset:  45%|████▌     | 816695/1807941 [11:12<06:48, 2426.68 examples/s]
Tokenizing train dataset:  45%|████▌     | 816960/1807941 [11:12<06:39, 2478.02 examples/s]
Tokenizing train dataset:  45%|████▌     | 817241/1807941 [11:12<08:09, 2025.26 examples/s]
Tokenizing train dataset:  45%|████▌     | 817466/1807941 [11:12<10:41, 1544.29 examples/s]
Tokenizing train dataset:  45%|████▌     | 817671/1807941 [11:12<12:03, 1368.00 examples/s]
Tokenizing train dataset:  45%|████▌     | 817873/1807941 [11:13<13:13, 1247.41 examples/s]
Tokenizing train dataset:  45%|████▌     | 818065/1807941 [11:13<14:17, 1154.61 examples/s]
Tokenizing train dataset:  45%|████▌     | 818239/1807941 [11:13<14:17, 1154.69 examples/s]
Tokenizing train dataset:  45%|████▌     | 818400/1807941 [11:13<14:35, 1130.59 examples/s]
Tokenizing train dataset:  45%|████▌     | 818543/1807941 [11:13<15:17, 1078.13 examples/s]
Tokenizing train dataset:  45%|████▌     | 818717/1807941 [11:13<15:01, 1097.58 examples/s]
Tokenizing train dataset:  45%|████▌     | 818862/1807941 [11:14<15:39, 1052.59 examples/s]
Tokenizing train dataset:  45%|████▌     | 819000/1807941 [11:14<16:45, 983.67 examples/s] 
Tokenizing train dataset:  45%|████▌     | 819128/1807941 [11:14<17:28, 943.19 examples/s]
Tokenizing train dataset:  45%|████▌     | 819224/1807941 [11:14<17:26, 944.37 examples/s]
Tokenizing train dataset:  45%|████▌     | 819405/1807941 [11:14<14:29, 1136.91 examples/s]
Tokenizing train dataset:  45%|████▌     | 819680/1807941 [11:14<10:43, 1535.06 examples/s]
Tokenizing train dataset:  45%|████▌     | 819868/1807941 [11:14<10:08, 1624.56 examples/s]
Tokenizing train dataset:  45%|████▌     | 820073/1807941 [11:14<09:28, 1738.32 examples/s]
Tokenizing train dataset:  45%|████▌     | 820308/1807941 [11:15<08:37, 1907.96 examples/s]
Tokenizing train dataset:  45%|████▌     | 820579/1807941 [11:15<07:42, 2135.07 examples/s]
Tokenizing train dataset:  45%|████▌     | 820834/1807941 [11:15<07:18, 2253.40 examples/s]
Tokenizing train dataset:  45%|████▌     | 821182/1807941 [11:15<07:13, 2275.69 examples/s]
Tokenizing train dataset:  45%|████▌     | 821529/1807941 [11:15<07:11, 2285.24 examples/s]
Tokenizing train dataset:  45%|████▌     | 821780/1807941 [11:15<07:01, 2338.62 examples/s]
Tokenizing train dataset:  45%|████▌     | 822084/1807941 [11:15<07:24, 2215.56 examples/s]
Tokenizing train dataset:  45%|████▌     | 822334/1807941 [11:16<09:54, 1657.13 examples/s]
Tokenizing train dataset:  45%|████▌     | 822531/1807941 [11:16<11:33, 1421.43 examples/s]
Tokenizing train dataset:  46%|████▌     | 822699/1807941 [11:16<13:25, 1222.46 examples/s]
Tokenizing train dataset:  46%|████▌     | 822836/1807941 [11:16<14:24, 1139.17 examples/s]
Tokenizing train dataset:  46%|████▌     | 822985/1807941 [11:16<14:58, 1096.04 examples/s]
Tokenizing train dataset:  46%|████▌     | 823101/1807941 [11:16<16:24, 1000.44 examples/s]
Tokenizing train dataset:  46%|████▌     | 823213/1807941 [11:17<17:46, 923.56 examples/s] 
Tokenizing train dataset:  46%|████▌     | 823344/1807941 [11:17<18:11, 902.29 examples/s]
Tokenizing train dataset:  46%|████▌     | 823484/1807941 [11:17<18:05, 906.88 examples/s]
Tokenizing train dataset:  46%|████▌     | 823602/1807941 [11:17<18:50, 870.48 examples/s]
Tokenizing train dataset:  46%|████▌     | 823722/1807941 [11:17<19:25, 844.11 examples/s]
Tokenizing train dataset:  46%|████▌     | 823947/1807941 [11:17<14:16, 1148.75 examples/s]
Tokenizing train dataset:  46%|████▌     | 824122/1807941 [11:17<12:48, 1279.61 examples/s]
Tokenizing train dataset:  46%|████▌     | 824320/1807941 [11:17<11:16, 1453.46 examples/s]
Tokenizing train dataset:  46%|████▌     | 824540/1807941 [11:18<09:57, 1646.89 examples/s]
Tokenizing train dataset:  46%|████▌     | 824770/1807941 [11:18<08:59, 1822.74 examples/s]
Tokenizing train dataset:  46%|████▌     | 825000/1807941 [11:18<13:42, 1195.38 examples/s]
Tokenizing train dataset:  46%|████▌     | 825158/1807941 [11:18<15:32, 1053.86 examples/s]
Tokenizing train dataset:  46%|████▌     | 825344/1807941 [11:18<14:52, 1100.88 examples/s]
Tokenizing train dataset:  46%|████▌     | 825589/1807941 [11:18<11:58, 1366.63 examples/s]
Tokenizing train dataset:  46%|████▌     | 825799/1807941 [11:19<10:43, 1525.63 examples/s]
Tokenizing train dataset:  46%|████▌     | 825988/1807941 [11:19<12:40, 1291.46 examples/s]
Tokenizing train dataset:  46%|████▌     | 826188/1807941 [11:19<13:50, 1182.27 examples/s]
Tokenizing train dataset:  46%|████▌     | 826334/1807941 [11:19<13:14, 1235.48 examples/s]
Tokenizing train dataset:  46%|████▌     | 826482/1807941 [11:19<12:42, 1287.57 examples/s]
Tokenizing train dataset:  46%|████▌     | 826647/1807941 [11:19<13:26, 1217.41 examples/s]
Tokenizing train dataset:  46%|████▌     | 826833/1807941 [11:19<13:24, 1220.18 examples/s]
Tokenizing train dataset:  46%|████▌     | 827000/1807941 [11:20<13:46, 1186.25 examples/s]
Tokenizing train dataset:  46%|████▌     | 827151/1807941 [11:20<12:59, 1257.78 examples/s]
Tokenizing train dataset:  46%|████▌     | 827328/1807941 [11:20<13:20, 1225.23 examples/s]
Tokenizing train dataset:  46%|████▌     | 827483/1807941 [11:20<12:33, 1301.13 examples/s]
Tokenizing train dataset:  46%|████▌     | 827685/1807941 [11:20<12:25, 1315.77 examples/s]
Tokenizing train dataset:  46%|████▌     | 827863/1807941 [11:20<12:53, 1266.94 examples/s]
Tokenizing train dataset:  46%|████▌     | 828000/1807941 [11:20<14:45, 1107.12 examples/s]
Tokenizing train dataset:  46%|████▌     | 828131/1807941 [11:21<15:53, 1027.42 examples/s]
Tokenizing train dataset:  46%|████▌     | 828275/1807941 [11:21<16:15, 1003.97 examples/s]
Tokenizing train dataset:  46%|████▌     | 828414/1807941 [11:21<15:00, 1088.23 examples/s]
Tokenizing train dataset:  46%|████▌     | 828566/1807941 [11:21<15:28, 1054.27 examples/s]
Tokenizing train dataset:  46%|████▌     | 828726/1807941 [11:21<15:26, 1057.06 examples/s]
Tokenizing train dataset:  46%|████▌     | 828848/1807941 [11:21<15:03, 1083.92 examples/s]
Tokenizing train dataset:  46%|████▌     | 829000/1807941 [11:21<17:57, 908.79 examples/s] 
Tokenizing train dataset:  46%|████▌     | 829146/1807941 [11:22<15:57, 1022.22 examples/s]
Tokenizing train dataset:  46%|████▌     | 829276/1807941 [11:22<16:57, 961.61 examples/s] 
Tokenizing train dataset:  46%|████▌     | 829461/1807941 [11:22<14:03, 1159.71 examples/s]
Tokenizing train dataset:  46%|████▌     | 829708/1807941 [11:22<11:01, 1477.75 examples/s]
Tokenizing train dataset:  46%|████▌     | 829942/1807941 [11:22<09:35, 1698.08 examples/s]
Tokenizing train dataset:  46%|████▌     | 830140/1807941 [11:22<09:12, 1770.90 examples/s]
Tokenizing train dataset:  46%|████▌     | 830369/1807941 [11:22<12:10, 1338.83 examples/s]
Tokenizing train dataset:  46%|████▌     | 830529/1807941 [11:23<14:15, 1142.55 examples/s]
Tokenizing train dataset:  46%|████▌     | 830700/1807941 [11:23<15:38, 1040.83 examples/s]
Tokenizing train dataset:  46%|████▌     | 830890/1807941 [11:23<13:30, 1204.82 examples/s]
Tokenizing train dataset:  46%|████▌     | 831118/1807941 [11:23<11:52, 1370.99 examples/s]
Tokenizing train dataset:  46%|████▌     | 831352/1807941 [11:23<10:14, 1589.08 examples/s]
Tokenizing train dataset:  46%|████▌     | 831545/1807941 [11:23<12:13, 1331.14 examples/s]
Tokenizing train dataset:  46%|████▌     | 831701/1807941 [11:24<13:10, 1234.36 examples/s]
Tokenizing train dataset:  46%|████▌     | 831860/1807941 [11:24<15:10, 1072.14 examples/s]
Tokenizing train dataset:  46%|████▌     | 832000/1807941 [11:24<17:32, 927.31 examples/s] 
Tokenizing train dataset:  46%|████▌     | 832122/1807941 [11:24<18:14, 891.22 examples/s]
Tokenizing train dataset:  46%|████▌     | 832224/1807941 [11:24<17:50, 911.19 examples/s]
Tokenizing train dataset:  46%|████▌     | 832334/1807941 [11:24<17:05, 951.68 examples/s]
Tokenizing train dataset:  46%|████▌     | 832460/1807941 [11:24<15:57, 1018.40 examples/s]
Tokenizing train dataset:  46%|████▌     | 832601/1807941 [11:25<16:29, 985.55 examples/s] 
Tokenizing train dataset:  46%|████▌     | 832741/1807941 [11:25<16:51, 964.36 examples/s]
Tokenizing train dataset:  46%|████▌     | 832850/1807941 [11:25<16:23, 990.95 examples/s]
Tokenizing train dataset:  46%|████▌     | 832998/1807941 [11:25<16:26, 987.94 examples/s]
Tokenizing train dataset:  46%|████▌     | 833101/1807941 [11:25<16:17, 997.30 examples/s]
Tokenizing train dataset:  46%|████▌     | 833350/1807941 [11:25<11:47, 1377.88 examples/s]
Tokenizing train dataset:  46%|████▌     | 833592/1807941 [11:25<09:48, 1654.57 examples/s]
Tokenizing train dataset:  46%|████▌     | 833824/1807941 [11:25<08:50, 1836.36 examples/s]
Tokenizing train dataset:  46%|████▌     | 834027/1807941 [11:25<08:35, 1890.39 examples/s]
Tokenizing train dataset:  46%|████▌     | 834271/1807941 [11:26<07:55, 2046.46 examples/s]
Tokenizing train dataset:  46%|████▌     | 834491/1807941 [11:26<10:02, 1616.58 examples/s]
Tokenizing train dataset:  46%|████▌     | 834702/1807941 [11:26<10:29, 1546.67 examples/s]
Tokenizing train dataset:  46%|████▌     | 834880/1807941 [11:26<11:22, 1426.25 examples/s]
Tokenizing train dataset:  46%|████▌     | 835133/1807941 [11:26<09:59, 1622.50 examples/s]
Tokenizing train dataset:  46%|████▌     | 835331/1807941 [11:26<09:29, 1707.71 examples/s]
Tokenizing train dataset:  46%|████▌     | 835538/1807941 [11:26<11:22, 1425.71 examples/s]
Tokenizing train dataset:  46%|████▌     | 835712/1807941 [11:27<12:03, 1343.23 examples/s]
Tokenizing train dataset:  46%|████▌     | 835900/1807941 [11:27<12:19, 1314.84 examples/s]
Tokenizing train dataset:  46%|████▌     | 836113/1807941 [11:27<11:03, 1463.64 examples/s]
Tokenizing train dataset:  46%|████▋     | 836349/1807941 [11:27<09:39, 1676.12 examples/s]
Tokenizing train dataset:  46%|████▋     | 836546/1807941 [11:27<09:15, 1749.69 examples/s]
Tokenizing train dataset:  46%|████▋     | 836819/1807941 [11:27<08:03, 2006.59 examples/s]
Tokenizing train dataset:  46%|████▋     | 837114/1807941 [11:27<08:07, 1991.19 examples/s]
Tokenizing train dataset:  46%|████▋     | 837411/1807941 [11:27<07:14, 2235.62 examples/s]
Tokenizing train dataset:  46%|████▋     | 837713/1807941 [11:28<06:37, 2443.26 examples/s]
Tokenizing train dataset:  46%|████▋     | 838000/1807941 [11:28<08:10, 1976.96 examples/s]
Tokenizing train dataset:  46%|████▋     | 838270/1807941 [11:28<09:19, 1733.80 examples/s]
Tokenizing train dataset:  46%|████▋     | 838485/1807941 [11:28<10:51, 1488.30 examples/s]
Tokenizing train dataset:  46%|████▋     | 838711/1807941 [11:28<10:51, 1487.63 examples/s]
Tokenizing train dataset:  46%|████▋     | 838917/1807941 [11:28<11:06, 1453.98 examples/s]
Tokenizing train dataset:  46%|████▋     | 839085/1807941 [11:29<11:55, 1354.94 examples/s]
Tokenizing train dataset:  46%|████▋     | 839230/1807941 [11:29<11:49, 1366.14 examples/s]
Tokenizing train dataset:  46%|████▋     | 839383/1807941 [11:29<11:32, 1398.75 examples/s]
Tokenizing train dataset:  46%|████▋     | 839558/1807941 [11:29<10:52, 1483.74 examples/s]
Tokenizing train dataset:  46%|████▋     | 839754/1807941 [11:29<11:26, 1410.07 examples/s]
Tokenizing train dataset:  46%|████▋     | 839907/1807941 [11:29<11:14, 1434.54 examples/s]
Tokenizing train dataset:  46%|████▋     | 840129/1807941 [11:29<11:08, 1446.77 examples/s]
Tokenizing train dataset:  46%|████▋     | 840322/1807941 [11:29<10:18, 1564.77 examples/s]
Tokenizing train dataset:  46%|████▋     | 840511/1807941 [11:30<09:47, 1645.88 examples/s]
Tokenizing train dataset:  47%|████▋     | 840753/1807941 [11:30<09:53, 1630.25 examples/s]
Tokenizing train dataset:  47%|████▋     | 840995/1807941 [11:30<09:56, 1621.24 examples/s]
Tokenizing train dataset:  47%|████▋     | 841184/1807941 [11:30<10:43, 1501.31 examples/s]
Tokenizing train dataset:  47%|████▋     | 841403/1807941 [11:30<10:52, 1480.18 examples/s]
Tokenizing train dataset:  47%|████▋     | 841610/1807941 [11:30<11:09, 1442.33 examples/s]
Tokenizing train dataset:  47%|████▋     | 841757/1807941 [11:30<11:09, 1443.17 examples/s]
Tokenizing train dataset:  47%|████▋     | 841931/1807941 [11:31<13:17, 1210.61 examples/s]
Tokenizing train dataset:  47%|████▋     | 842067/1807941 [11:31<13:19, 1208.47 examples/s]
Tokenizing train dataset:  47%|████▋     | 842208/1807941 [11:31<12:53, 1248.21 examples/s]
Tokenizing train dataset:  47%|████▋     | 842386/1807941 [11:31<13:20, 1206.94 examples/s]
Tokenizing train dataset:  47%|████▋     | 842564/1807941 [11:31<13:27, 1196.02 examples/s]
Tokenizing train dataset:  47%|████▋     | 842700/1807941 [11:31<13:03, 1232.32 examples/s]
Tokenizing train dataset:  47%|████▋     | 842830/1807941 [11:31<12:54, 1246.91 examples/s]
Tokenizing train dataset:  47%|████▋     | 843000/1807941 [11:32<15:29, 1038.30 examples/s]
Tokenizing train dataset:  47%|████▋     | 843123/1807941 [11:32<18:35, 865.09 examples/s] 
Tokenizing train dataset:  47%|████▋     | 843227/1807941 [11:32<20:09, 797.45 examples/s]
Tokenizing train dataset:  47%|████▋     | 843337/1807941 [11:32<18:49, 853.76 examples/s]
Tokenizing train dataset:  47%|████▋     | 843470/1807941 [11:32<20:42, 776.20 examples/s]
Tokenizing train dataset:  47%|████▋     | 843593/1807941 [11:32<19:13, 836.23 examples/s]
Tokenizing train dataset:  47%|████▋     | 843729/1807941 [11:33<21:01, 764.35 examples/s]
Tokenizing train dataset:  47%|████▋     | 843822/1807941 [11:33<24:38, 652.28 examples/s]
Tokenizing train dataset:  47%|████▋     | 843928/1807941 [11:33<26:33, 605.03 examples/s]
Tokenizing train dataset:  47%|████▋     | 844000/1807941 [11:33<31:32, 509.30 examples/s]
Tokenizing train dataset:  47%|████▋     | 844078/1807941 [11:33<28:56, 555.00 examples/s]
Tokenizing train dataset:  47%|████▋     | 844164/1807941 [11:33<26:25, 607.80 examples/s]
Tokenizing train dataset:  47%|████▋     | 844258/1807941 [11:33<23:45, 675.88 examples/s]
Tokenizing train dataset:  47%|████▋     | 844341/1807941 [11:34<22:43, 706.63 examples/s]
Tokenizing train dataset:  47%|████▋     | 844438/1807941 [11:34<20:53, 768.93 examples/s]
Tokenizing train dataset:  47%|████▋     | 844564/1807941 [11:34<20:25, 785.90 examples/s]
Tokenizing train dataset:  47%|████▋     | 844690/1807941 [11:34<17:53, 897.54 examples/s]
Tokenizing train dataset:  47%|████▋     | 844818/1807941 [11:34<29:11, 550.03 examples/s]
Tokenizing train dataset:  47%|████▋     | 844906/1807941 [11:35<32:47, 489.54 examples/s]
Tokenizing train dataset:  47%|████▋     | 844983/1807941 [11:35<30:27, 527.03 examples/s]
Tokenizing train dataset:  47%|████▋     | 845081/1807941 [11:35<28:30, 562.98 examples/s]
Tokenizing train dataset:  47%|████▋     | 845589/1807941 [11:35<11:01, 1455.47 examples/s]
Tokenizing train dataset:  47%|████▋     | 846059/1807941 [11:35<07:23, 2168.79 examples/s]
Tokenizing train dataset:  47%|████▋     | 846562/1807941 [11:35<05:37, 2851.55 examples/s]
Tokenizing train dataset:  47%|████▋     | 847031/1807941 [11:35<04:49, 3318.09 examples/s]
Tokenizing train dataset:  47%|████▋     | 847534/1807941 [11:35<04:14, 3772.98 examples/s]
Tokenizing train dataset:  47%|████▋     | 848002/1807941 [11:35<03:58, 4020.36 examples/s]
Tokenizing train dataset:  47%|████▋     | 848498/1807941 [11:36<03:43, 4284.25 examples/s]
Tokenizing train dataset:  47%|████▋     | 848989/1807941 [11:36<03:34, 4461.25 examples/s]
Tokenizing train dataset:  47%|████▋     | 849696/1807941 [11:36<03:30, 4554.80 examples/s]
Tokenizing train dataset:  47%|████▋     | 850237/1807941 [11:36<03:47, 4215.02 examples/s]
Tokenizing train dataset:  47%|████▋     | 850750/1807941 [11:36<04:02, 3944.12 examples/s]
Tokenizing train dataset:  47%|████▋     | 851217/1807941 [11:36<04:20, 3673.23 examples/s]
Tokenizing train dataset:  47%|████▋     | 851674/1807941 [11:36<04:35, 3474.34 examples/s]
Tokenizing train dataset:  47%|████▋     | 852119/1807941 [11:37<04:48, 3314.52 examples/s]
Tokenizing train dataset:  47%|████▋     | 852601/1807941 [11:37<04:51, 3278.74 examples/s]
Tokenizing train dataset:  47%|████▋     | 853045/1807941 [11:37<05:00, 3178.47 examples/s]
Tokenizing train dataset:  47%|████▋     | 853517/1807941 [11:37<05:01, 3166.98 examples/s]
Tokenizing train dataset:  47%|████▋     | 853994/1807941 [11:37<05:01, 3167.51 examples/s]
Tokenizing train dataset:  47%|████▋     | 854444/1807941 [11:37<05:06, 3113.46 examples/s]
Tokenizing train dataset:  47%|████▋     | 854922/1807941 [11:37<05:04, 3134.24 examples/s]
Tokenizing train dataset:  47%|████▋     | 855364/1807941 [11:38<05:09, 3076.25 examples/s]
Tokenizing train dataset:  47%|████▋     | 855683/1807941 [11:38<05:07, 3099.42 examples/s]
Tokenizing train dataset:  47%|████▋     | 856000/1807941 [11:38<05:06, 3102.18 examples/s]
Tokenizing train dataset:  47%|████▋     | 856333/1807941 [11:38<05:01, 3157.71 examples/s]
Tokenizing train dataset:  47%|████▋     | 856661/1807941 [11:38<04:58, 3186.73 examples/s]
Tokenizing train dataset:  47%|████▋     | 857102/1807941 [11:38<05:07, 3094.43 examples/s]
Tokenizing train dataset:  47%|████▋     | 857426/1807941 [11:38<05:03, 3130.47 examples/s]
Tokenizing train dataset:  47%|████▋     | 857896/1807941 [11:38<05:03, 3128.70 examples/s]
Tokenizing train dataset:  47%|████▋     | 858336/1807941 [11:39<05:10, 3060.40 examples/s]
Tokenizing train dataset:  48%|████▊     | 858801/1807941 [11:39<05:10, 3060.61 examples/s]
Tokenizing train dataset:  48%|████▊     | 859223/1807941 [11:39<05:18, 2981.01 examples/s]
Tokenizing train dataset:  48%|████▊     | 859655/1807941 [11:39<05:21, 2946.11 examples/s]
Tokenizing train dataset:  48%|████▊     | 859959/1807941 [11:39<05:19, 2966.14 examples/s]
Tokenizing train dataset:  48%|████▊     | 860412/1807941 [11:39<05:17, 2979.78 examples/s]
Tokenizing train dataset:  48%|████▊     | 860722/1807941 [11:39<05:15, 3005.75 examples/s]
Tokenizing train dataset:  48%|████▊     | 861153/1807941 [11:40<05:24, 2920.60 examples/s]
Tokenizing train dataset:  48%|████▊     | 861461/1807941 [11:40<05:20, 2956.85 examples/s]
Tokenizing train dataset:  48%|████▊     | 861845/1807941 [11:40<05:35, 2818.82 examples/s]
Tokenizing train dataset:  48%|████▊     | 862255/1807941 [11:40<05:39, 2785.07 examples/s]
Tokenizing train dataset:  48%|████▊     | 862576/1807941 [11:40<05:27, 2885.39 examples/s]
Tokenizing train dataset:  48%|████▊     | 862897/1807941 [11:40<05:18, 2962.71 examples/s]
Tokenizing train dataset:  48%|████▊     | 863335/1807941 [11:40<05:20, 2943.31 examples/s]
Tokenizing train dataset:  48%|████▊     | 863676/1807941 [11:40<05:47, 2714.78 examples/s]
Tokenizing train dataset:  48%|████▊     | 863998/1807941 [11:41<05:33, 2834.24 examples/s]
Tokenizing train dataset:  48%|████▊     | 864422/1807941 [11:41<05:33, 2829.46 examples/s]
Tokenizing train dataset:  48%|████▊     | 864739/1807941 [11:41<05:24, 2909.94 examples/s]
Tokenizing train dataset:  48%|████▊     | 865157/1807941 [11:41<05:29, 2862.22 examples/s]
Tokenizing train dataset:  48%|████▊     | 865465/1807941 [11:41<05:23, 2912.36 examples/s]
Tokenizing train dataset:  48%|████▊     | 865791/1807941 [11:41<05:14, 2998.73 examples/s]
Tokenizing train dataset:  48%|████▊     | 866135/1807941 [11:41<06:24, 2446.38 examples/s]
Tokenizing train dataset:  48%|████▊     | 866442/1807941 [11:41<06:03, 2590.17 examples/s]
Tokenizing train dataset:  48%|████▊     | 866774/1807941 [11:42<05:39, 2769.42 examples/s]
Tokenizing train dataset:  48%|████▊     | 867124/1807941 [11:42<05:59, 2615.45 examples/s]
Tokenizing train dataset:  48%|████▊     | 867447/1807941 [11:42<05:39, 2766.53 examples/s]
Tokenizing train dataset:  48%|████▊     | 867778/1807941 [11:42<05:23, 2905.93 examples/s]
Tokenizing train dataset:  48%|████▊     | 868227/1807941 [11:42<05:20, 2933.26 examples/s]
Tokenizing train dataset:  48%|████▊     | 868538/1807941 [11:42<05:15, 2975.95 examples/s]
Tokenizing train dataset:  48%|████▊     | 868960/1807941 [11:42<05:22, 2907.34 examples/s]
Tokenizing train dataset:  48%|████▊     | 869365/1807941 [11:42<05:31, 2833.22 examples/s]
Tokenizing train dataset:  48%|████▊     | 869666/1807941 [11:43<05:26, 2873.97 examples/s]
Tokenizing train dataset:  48%|████▊     | 870000/1807941 [11:43<06:37, 2358.79 examples/s]
Tokenizing train dataset:  48%|████▊     | 870347/1807941 [11:43<05:59, 2604.72 examples/s]
Tokenizing train dataset:  48%|████▊     | 870713/1807941 [11:43<05:28, 2853.91 examples/s]
Tokenizing train dataset:  48%|████▊     | 871161/1807941 [11:43<05:23, 2896.94 examples/s]
Tokenizing train dataset:  48%|████▊     | 871485/1807941 [11:43<05:14, 2977.56 examples/s]
Tokenizing train dataset:  48%|████▊     | 871830/1807941 [11:43<05:02, 3098.70 examples/s]
Tokenizing train dataset:  48%|████▊     | 872213/1807941 [11:43<06:01, 2588.01 examples/s]
Tokenizing train dataset:  48%|████▊     | 872530/1807941 [11:44<05:43, 2720.42 examples/s]
Tokenizing train dataset:  48%|████▊     | 872870/1807941 [11:44<05:23, 2887.37 examples/s]
Tokenizing train dataset:  48%|████▊     | 873239/1807941 [11:44<05:41, 2733.63 examples/s]
Tokenizing train dataset:  48%|████▊     | 873672/1807941 [11:44<05:36, 2778.58 examples/s]
Tokenizing train dataset:  48%|████▊     | 874000/1807941 [11:44<05:26, 2859.17 examples/s]
Tokenizing train dataset:  48%|████▊     | 874414/1807941 [11:44<05:30, 2822.73 examples/s]
Tokenizing train dataset:  48%|████▊     | 874771/1807941 [11:44<05:10, 3000.61 examples/s]
Tokenizing train dataset:  48%|████▊     | 875173/1807941 [11:45<05:23, 2881.14 examples/s]
Tokenizing train dataset:  48%|████▊     | 875624/1807941 [11:45<05:54, 2629.37 examples/s]
Tokenizing train dataset:  48%|████▊     | 875907/1807941 [11:45<05:48, 2672.25 examples/s]
Tokenizing train dataset:  48%|████▊     | 876339/1807941 [11:45<05:40, 2735.12 examples/s]
Tokenizing train dataset:  48%|████▊     | 876654/1807941 [11:45<05:29, 2830.34 examples/s]
Tokenizing train dataset:  49%|████▊     | 876960/1807941 [11:45<05:22, 2883.93 examples/s]
Tokenizing train dataset:  49%|████▊     | 877402/1807941 [11:45<05:20, 2902.37 examples/s]
Tokenizing train dataset:  49%|████▊     | 877723/1807941 [11:45<05:12, 2977.31 examples/s]
Tokenizing train dataset:  49%|████▊     | 878179/1807941 [11:46<05:10, 2996.67 examples/s]
Tokenizing train dataset:  49%|████▊     | 878656/1807941 [11:46<05:04, 3053.00 examples/s]
Tokenizing train dataset:  49%|████▊     | 879000/1807941 [11:46<04:55, 3138.33 examples/s]
Tokenizing train dataset:  49%|████▊     | 879357/1807941 [11:46<04:46, 3246.06 examples/s]
Tokenizing train dataset:  49%|████▊     | 879799/1807941 [11:46<04:55, 3136.37 examples/s]
Tokenizing train dataset:  49%|████▊     | 880120/1807941 [11:46<04:54, 3152.88 examples/s]
Tokenizing train dataset:  49%|████▊     | 880467/1807941 [11:46<04:46, 3234.21 examples/s]
Tokenizing train dataset:  49%|████▊     | 880827/1807941 [11:46<04:38, 3332.68 examples/s]
Tokenizing train dataset:  49%|████▊     | 881354/1807941 [11:47<04:32, 3397.90 examples/s]
Tokenizing train dataset:  49%|████▉     | 881799/1807941 [11:47<04:45, 3246.71 examples/s]
Tokenizing train dataset:  49%|████▉     | 882160/1807941 [11:47<05:47, 2666.18 examples/s]
Tokenizing train dataset:  49%|████▉     | 882479/1807941 [11:47<06:09, 2503.07 examples/s]
Tokenizing train dataset:  49%|████▉     | 882775/1807941 [11:47<05:55, 2602.18 examples/s]
Tokenizing train dataset:  49%|████▉     | 883093/1807941 [11:47<05:38, 2735.32 examples/s]
Tokenizing train dataset:  49%|████▉     | 883402/1807941 [11:47<05:27, 2820.75 examples/s]
Tokenizing train dataset:  49%|████▉     | 883712/1807941 [11:47<05:19, 2891.39 examples/s]
Tokenizing train dataset:  49%|████▉     | 884051/1807941 [11:48<05:05, 3025.81 examples/s]
Tokenizing train dataset:  49%|████▉     | 884380/1807941 [11:48<04:58, 3095.79 examples/s]
Tokenizing train dataset:  49%|████▉     | 884735/1807941 [11:48<04:46, 3225.46 examples/s]
Tokenizing train dataset:  49%|████▉     | 885211/1807941 [11:48<04:48, 3202.07 examples/s]
Tokenizing train dataset:  49%|████▉     | 885573/1807941 [11:48<04:38, 3312.47 examples/s]
Tokenizing train dataset:  49%|████▉     | 886000/1807941 [11:48<04:57, 3095.00 examples/s]
Tokenizing train dataset:  49%|████▉     | 886340/1807941 [11:48<04:50, 3169.31 examples/s]
Tokenizing train dataset:  49%|████▉     | 886820/1807941 [11:48<04:50, 3172.75 examples/s]
Tokenizing train dataset:  49%|████▉     | 887266/1807941 [11:49<04:56, 3103.90 examples/s]
Tokenizing train dataset:  49%|████▉     | 887749/1807941 [11:49<04:53, 3136.26 examples/s]
Tokenizing train dataset:  49%|████▉     | 888208/1807941 [11:49<04:55, 3109.53 examples/s]
Tokenizing train dataset:  49%|████▉     | 888581/1807941 [11:49<04:42, 3250.23 examples/s]
Tokenizing train dataset:  49%|████▉     | 888920/1807941 [11:49<04:40, 3280.32 examples/s]
Tokenizing train dataset:  49%|████▉     | 889430/1807941 [11:49<04:36, 3316.77 examples/s]
Tokenizing train dataset:  49%|████▉     | 889901/1807941 [11:49<04:42, 3255.26 examples/s]
Tokenizing train dataset:  49%|████▉     | 890375/1807941 [11:49<04:44, 3221.10 examples/s]
Tokenizing train dataset:  49%|████▉     | 890838/1807941 [11:50<04:48, 3176.69 examples/s]
Tokenizing train dataset:  49%|████▉     | 891275/1807941 [11:50<04:56, 3091.52 examples/s]
Tokenizing train dataset:  49%|████▉     | 891599/1807941 [11:50<04:53, 3124.33 examples/s]
Tokenizing train dataset:  49%|████▉     | 891948/1807941 [11:50<04:45, 3208.96 examples/s]
Tokenizing train dataset:  49%|████▉     | 892313/1807941 [11:50<05:11, 2935.84 examples/s]
Tokenizing train dataset:  49%|████▉     | 892656/1807941 [11:50<05:36, 2718.76 examples/s]
Tokenizing train dataset:  49%|████▉     | 892973/1807941 [11:50<06:02, 2523.95 examples/s]
Tokenizing train dataset:  49%|████▉     | 893243/1807941 [11:51<05:57, 2561.03 examples/s]
Tokenizing train dataset:  49%|████▉     | 893588/1807941 [11:51<05:28, 2780.47 examples/s]
Tokenizing train dataset:  49%|████▉     | 894000/1807941 [11:51<05:30, 2762.13 examples/s]
Tokenizing train dataset:  49%|████▉     | 894334/1807941 [11:51<05:15, 2900.22 examples/s]
Tokenizing train dataset:  49%|████▉     | 894631/1807941 [11:51<05:13, 2915.61 examples/s]
Tokenizing train dataset:  49%|████▉     | 894929/1807941 [11:51<05:11, 2931.98 examples/s]
Tokenizing train dataset:  50%|████▉     | 895231/1807941 [11:51<05:08, 2953.97 examples/s]
Tokenizing train dataset:  50%|████▉     | 895553/1807941 [11:51<05:01, 3025.83 examples/s]
Tokenizing train dataset:  50%|████▉     | 895898/1807941 [11:51<04:49, 3147.54 examples/s]
Tokenizing train dataset:  50%|████▉     | 896354/1807941 [11:52<04:53, 3104.57 examples/s]
Tokenizing train dataset:  50%|████▉     | 896670/1807941 [11:52<04:52, 3116.17 examples/s]
Tokenizing train dataset:  50%|████▉     | 896984/1807941 [11:52<04:51, 3121.64 examples/s]
Tokenizing train dataset:  50%|████▉     | 897422/1807941 [11:52<04:59, 3043.71 examples/s]
Tokenizing train dataset:  50%|████▉     | 897887/1807941 [11:52<04:57, 3060.58 examples/s]
Tokenizing train dataset:  50%|████▉     | 898309/1807941 [11:52<05:05, 2976.80 examples/s]
Tokenizing train dataset:  50%|████▉     | 898753/1807941 [11:52<05:06, 2966.06 examples/s]
Tokenizing train dataset:  50%|████▉     | 899200/1807941 [11:52<05:06, 2967.62 examples/s]
Tokenizing train dataset:  50%|████▉     | 899499/1807941 [11:53<05:05, 2969.47 examples/s]
Tokenizing train dataset:  50%|████▉     | 899813/1807941 [11:53<05:01, 3007.10 examples/s]
Tokenizing train dataset:  50%|████▉     | 900117/1807941 [11:53<05:40, 2663.65 examples/s]
Tokenizing train dataset:  50%|████▉     | 900495/1807941 [11:53<05:47, 2613.77 examples/s]
Tokenizing train dataset:  50%|████▉     | 900766/1807941 [11:53<05:44, 2635.39 examples/s]
Tokenizing train dataset:  50%|████▉     | 901097/1807941 [11:53<06:04, 2485.14 examples/s]
Tokenizing train dataset:  50%|████▉     | 901358/1807941 [11:53<06:01, 2510.54 examples/s]
Tokenizing train dataset:  50%|████▉     | 901666/1807941 [11:53<05:41, 2655.39 examples/s]
Tokenizing train dataset:  50%|████▉     | 901989/1807941 [11:54<05:22, 2807.90 examples/s]
Tokenizing train dataset:  50%|████▉     | 902324/1807941 [11:54<05:06, 2955.21 examples/s]
Tokenizing train dataset:  50%|████▉     | 902682/1807941 [11:54<04:49, 3131.21 examples/s]
Tokenizing train dataset:  50%|████▉     | 903000/1807941 [11:54<04:54, 3069.99 examples/s]
Tokenizing train dataset:  50%|████▉     | 903407/1807941 [11:54<05:08, 2933.57 examples/s]
Tokenizing train dataset:  50%|████▉     | 903802/1807941 [11:54<05:20, 2823.39 examples/s]
Tokenizing train dataset:  50%|█████     | 904196/1807941 [11:54<05:28, 2755.04 examples/s]
Tokenizing train dataset:  50%|█████     | 904596/1807941 [11:54<05:31, 2721.60 examples/s]
Tokenizing train dataset:  50%|█████     | 904887/1807941 [11:55<05:26, 2763.58 examples/s]
Tokenizing train dataset:  50%|█████     | 905230/1807941 [11:55<05:47, 2599.14 examples/s]
Tokenizing train dataset:  50%|█████     | 905501/1807941 [11:55<05:43, 2624.52 examples/s]
Tokenizing train dataset:  50%|█████     | 905768/1807941 [11:55<05:42, 2634.86 examples/s]
Tokenizing train dataset:  50%|█████     | 906073/1807941 [11:55<05:28, 2743.94 examples/s]
Tokenizing train dataset:  50%|█████     | 906420/1807941 [11:55<05:06, 2944.40 examples/s]
Tokenizing train dataset:  50%|█████     | 906777/1807941 [11:55<04:48, 3118.89 examples/s]
Tokenizing train dataset:  50%|█████     | 907118/1807941 [11:55<04:41, 3199.22 examples/s]
Tokenizing train dataset:  50%|█████     | 907453/1807941 [11:55<04:37, 3239.97 examples/s]
Tokenizing train dataset:  50%|█████     | 907782/1807941 [11:56<04:36, 3251.84 examples/s]
Tokenizing train dataset:  50%|█████     | 908255/1807941 [11:56<04:40, 3209.81 examples/s]
Tokenizing train dataset:  50%|█████     | 908718/1807941 [11:56<04:44, 3159.73 examples/s]
Tokenizing train dataset:  50%|█████     | 909159/1807941 [11:56<04:51, 3083.30 examples/s]
Tokenizing train dataset:  50%|█████     | 909592/1807941 [11:56<04:57, 3016.99 examples/s]
Tokenizing train dataset:  50%|█████     | 909929/1807941 [11:56<04:49, 3098.43 examples/s]
Tokenizing train dataset:  50%|█████     | 910258/1807941 [11:56<05:22, 2787.14 examples/s]
Tokenizing train dataset:  50%|█████     | 910679/1807941 [11:57<05:21, 2790.09 examples/s]
Tokenizing train dataset:  50%|█████     | 911013/1807941 [11:57<05:44, 2607.34 examples/s]
Tokenizing train dataset:  50%|█████     | 911407/1807941 [11:57<05:43, 2610.80 examples/s]
Tokenizing train dataset:  50%|█████     | 911717/1807941 [11:57<05:29, 2719.70 examples/s]
Tokenizing train dataset:  50%|█████     | 912000/1807941 [11:57<05:26, 2743.32 examples/s]
Tokenizing train dataset:  50%|█████     | 912428/1807941 [11:57<05:22, 2778.83 examples/s]
Tokenizing train dataset:  50%|█████     | 912730/1807941 [11:57<05:15, 2837.36 examples/s]
Tokenizing train dataset:  51%|█████     | 913165/1807941 [11:57<05:14, 2849.32 examples/s]
Tokenizing train dataset:  51%|█████     | 913546/1807941 [11:58<05:25, 2744.75 examples/s]
Tokenizing train dataset:  51%|█████     | 913974/1807941 [11:58<05:21, 2777.71 examples/s]
Tokenizing train dataset:  51%|█████     | 914357/1807941 [11:58<05:30, 2703.80 examples/s]
Tokenizing train dataset:  51%|█████     | 914662/1807941 [11:58<05:21, 2782.18 examples/s]
Tokenizing train dataset:  51%|█████     | 914979/1807941 [11:58<05:10, 2876.12 examples/s]
Tokenizing train dataset:  51%|█████     | 915285/1807941 [11:58<05:05, 2922.68 examples/s]
Tokenizing train dataset:  51%|█████     | 915648/1807941 [11:58<04:46, 3113.76 examples/s]
Tokenizing train dataset:  51%|█████     | 915993/1807941 [11:58<04:38, 3204.69 examples/s]
Tokenizing train dataset:  51%|█████     | 916441/1807941 [11:59<04:45, 3119.05 examples/s]
Tokenizing train dataset:  51%|█████     | 916883/1807941 [11:59<04:51, 3054.10 examples/s]
Tokenizing train dataset:  51%|█████     | 917339/1807941 [11:59<04:52, 3045.20 examples/s]
Tokenizing train dataset:  51%|█████     | 917664/1807941 [11:59<04:47, 3093.25 examples/s]
Tokenizing train dataset:  51%|█████     | 917998/1807941 [11:59<04:42, 3151.76 examples/s]
Tokenizing train dataset:  51%|█████     | 918340/1807941 [11:59<05:13, 2840.29 examples/s]
Tokenizing train dataset:  51%|█████     | 918690/1807941 [11:59<04:56, 3003.46 examples/s]
Tokenizing train dataset:  51%|█████     | 919014/1807941 [11:59<04:50, 3063.89 examples/s]
Tokenizing train dataset:  51%|█████     | 919367/1807941 [11:59<04:38, 3190.83 examples/s]
Tokenizing train dataset:  51%|█████     | 919810/1807941 [12:00<04:46, 3099.92 examples/s]
Tokenizing train dataset:  51%|█████     | 920210/1807941 [12:00<05:01, 2948.77 examples/s]
Tokenizing train dataset:  51%|█████     | 920578/1807941 [12:00<04:43, 3126.56 examples/s]
Tokenizing train dataset:  51%|█████     | 920954/1807941 [12:00<04:29, 3289.76 examples/s]
Tokenizing train dataset:  51%|█████     | 921432/1807941 [12:00<04:32, 3248.44 examples/s]
Tokenizing train dataset:  51%|█████     | 921780/1807941 [12:00<04:28, 3304.40 examples/s]
Tokenizing train dataset:  51%|█████     | 922291/1807941 [12:00<04:25, 3334.65 examples/s]
Tokenizing train dataset:  51%|█████     | 922643/1807941 [12:00<04:21, 3379.29 examples/s]
Tokenizing train dataset:  51%|█████     | 923000/1807941 [12:01<04:22, 3371.02 examples/s]
Tokenizing train dataset:  51%|█████     | 923506/1807941 [12:01<04:22, 3368.25 examples/s]
Tokenizing train dataset:  51%|█████     | 923866/1807941 [12:01<04:18, 3426.17 examples/s]
Tokenizing train dataset:  51%|█████     | 924324/1807941 [12:01<04:28, 3289.63 examples/s]
Tokenizing train dataset:  51%|█████     | 924760/1807941 [12:01<04:40, 3151.11 examples/s]
Tokenizing train dataset:  51%|█████     | 925214/1807941 [12:01<04:44, 3106.23 examples/s]
Tokenizing train dataset:  51%|█████     | 925536/1807941 [12:02<06:26, 2283.98 examples/s]
Tokenizing train dataset:  51%|█████     | 925830/1807941 [12:02<08:34, 1713.67 examples/s]
Tokenizing train dataset:  51%|█████     | 926118/1807941 [12:02<10:18, 1425.08 examples/s]
Tokenizing train dataset:  51%|█████     | 926424/1807941 [12:02<08:47, 1670.44 examples/s]
Tokenizing train dataset:  51%|█████▏    | 926741/1807941 [12:02<07:35, 1935.02 examples/s]
Tokenizing train dataset:  51%|█████▏    | 927037/1807941 [12:02<06:51, 2139.73 examples/s]
Tokenizing train dataset:  51%|█████▏    | 927342/1807941 [12:03<06:16, 2341.94 examples/s]
Tokenizing train dataset:  51%|█████▏    | 927640/1807941 [12:03<05:52, 2494.51 examples/s]
Tokenizing train dataset:  51%|█████▏    | 927983/1807941 [12:03<06:03, 2419.59 examples/s]
Tokenizing train dataset:  51%|█████▏    | 928294/1807941 [12:03<06:22, 2302.19 examples/s]
Tokenizing train dataset:  51%|█████▏    | 928616/1807941 [12:03<05:49, 2518.82 examples/s]
Tokenizing train dataset:  51%|█████▏    | 928971/1807941 [12:03<05:56, 2463.55 examples/s]
Tokenizing train dataset:  51%|█████▏    | 929242/1807941 [12:03<05:48, 2521.16 examples/s]
Tokenizing train dataset:  51%|█████▏    | 929528/1807941 [12:03<05:36, 2607.33 examples/s]
Tokenizing train dataset:  51%|█████▏    | 929840/1807941 [12:04<05:20, 2740.34 examples/s]
Tokenizing train dataset:  51%|█████▏    | 930123/1807941 [12:04<05:17, 2762.92 examples/s]
Tokenizing train dataset:  51%|█████▏    | 930470/1807941 [12:04<04:56, 2958.40 examples/s]
Tokenizing train dataset:  51%|█████▏    | 930931/1807941 [12:04<04:52, 2993.66 examples/s]
Tokenizing train dataset:  52%|█████▏    | 931351/1807941 [12:04<04:59, 2924.83 examples/s]
Tokenizing train dataset:  52%|█████▏    | 931710/1807941 [12:04<04:43, 3088.68 examples/s]
Tokenizing train dataset:  52%|█████▏    | 932175/1807941 [12:04<04:43, 3089.10 examples/s]
Tokenizing train dataset:  52%|█████▏    | 932627/1807941 [12:04<04:46, 3057.80 examples/s]
Tokenizing train dataset:  52%|█████▏    | 932952/1807941 [12:05<04:42, 3100.83 examples/s]
Tokenizing train dataset:  52%|█████▏    | 933417/1807941 [12:05<04:42, 3097.76 examples/s]
Tokenizing train dataset:  52%|█████▏    | 933756/1807941 [12:05<04:36, 3167.12 examples/s]
Tokenizing train dataset:  52%|█████▏    | 934222/1807941 [12:05<04:38, 3141.47 examples/s]
Tokenizing train dataset:  52%|█████▏    | 934571/1807941 [12:05<04:30, 3226.72 examples/s]
Tokenizing train dataset:  52%|█████▏    | 934912/1807941 [12:05<04:26, 3273.23 examples/s]
Tokenizing train dataset:  52%|█████▏    | 935370/1807941 [12:05<04:33, 3191.02 examples/s]
Tokenizing train dataset:  52%|█████▏    | 935873/1807941 [12:05<04:28, 3242.78 examples/s]
Tokenizing train dataset:  52%|█████▏    | 936345/1807941 [12:06<04:33, 3185.92 examples/s]
Tokenizing train dataset:  52%|█████▏    | 936701/1807941 [12:06<04:26, 3271.46 examples/s]
Tokenizing train dataset:  52%|█████▏    | 937040/1807941 [12:06<04:23, 3299.76 examples/s]
Tokenizing train dataset:  52%|█████▏    | 937379/1807941 [12:06<04:22, 3317.88 examples/s]
Tokenizing train dataset:  52%|█████▏    | 937753/1807941 [12:06<04:13, 3432.96 examples/s]
Tokenizing train dataset:  52%|█████▏    | 938268/1807941 [12:06<04:13, 3429.43 examples/s]
Tokenizing train dataset:  52%|█████▏    | 938741/1807941 [12:06<04:20, 3331.78 examples/s]
Tokenizing train dataset:  52%|█████▏    | 939215/1807941 [12:06<04:25, 3270.76 examples/s]
Tokenizing train dataset:  52%|█████▏    | 939547/1807941 [12:07<04:24, 3278.80 examples/s]
Tokenizing train dataset:  52%|█████▏    | 939883/1807941 [12:07<04:23, 3298.67 examples/s]
Tokenizing train dataset:  52%|█████▏    | 940389/1807941 [12:07<04:21, 3320.22 examples/s]
Tokenizing train dataset:  52%|█████▏    | 940872/1807941 [12:07<04:24, 3284.01 examples/s]
Tokenizing train dataset:  52%|█████▏    | 941207/1807941 [12:07<04:22, 3296.66 examples/s]
Tokenizing train dataset:  52%|█████▏    | 941566/1807941 [12:07<04:17, 3368.49 examples/s]
Tokenizing train dataset:  52%|█████▏    | 941931/1807941 [12:07<04:11, 3441.38 examples/s]
Tokenizing train dataset:  52%|█████▏    | 942279/1807941 [12:07<04:11, 3448.39 examples/s]
Tokenizing train dataset:  52%|█████▏    | 942658/1807941 [12:07<04:04, 3542.38 examples/s]
Tokenizing train dataset:  52%|█████▏    | 943164/1807941 [12:08<04:08, 3473.38 examples/s]
Tokenizing train dataset:  52%|█████▏    | 943612/1807941 [12:08<04:22, 3298.70 examples/s]
Tokenizing train dataset:  52%|█████▏    | 944019/1807941 [12:08<04:38, 3099.66 examples/s]
Tokenizing train dataset:  52%|█████▏    | 944438/1807941 [12:08<04:48, 2997.73 examples/s]
Tokenizing train dataset:  52%|█████▏    | 944786/1807941 [12:08<05:10, 2779.41 examples/s]
Tokenizing train dataset:  52%|█████▏    | 945088/1807941 [12:08<05:04, 2833.38 examples/s]
Tokenizing train dataset:  52%|█████▏    | 945383/1807941 [12:08<05:01, 2860.28 examples/s]
Tokenizing train dataset:  52%|█████▏    | 945734/1807941 [12:08<04:44, 3027.34 examples/s]
Tokenizing train dataset:  52%|█████▏    | 946092/1807941 [12:09<05:08, 2793.31 examples/s]
Tokenizing train dataset:  52%|█████▏    | 946481/1807941 [12:09<05:16, 2724.17 examples/s]
Tokenizing train dataset:  52%|█████▏    | 946811/1807941 [12:09<05:00, 2863.00 examples/s]
Tokenizing train dataset:  52%|█████▏    | 947263/1807941 [12:09<04:55, 2911.36 examples/s]
Tokenizing train dataset:  52%|█████▏    | 947593/1807941 [12:09<04:46, 3005.99 examples/s]
Tokenizing train dataset:  52%|█████▏    | 947933/1807941 [12:09<04:36, 3107.41 examples/s]
Tokenizing train dataset:  52%|█████▏    | 948391/1807941 [12:09<04:38, 3085.78 examples/s]
Tokenizing train dataset:  52%|█████▏    | 948706/1807941 [12:09<04:37, 3101.19 examples/s]
Tokenizing train dataset:  52%|█████▏    | 949155/1807941 [12:10<04:40, 3061.79 examples/s]
Tokenizing train dataset:  53%|█████▎    | 949476/1807941 [12:10<04:37, 3096.78 examples/s]
Tokenizing train dataset:  53%|█████▎    | 949818/1807941 [12:10<04:29, 3181.70 examples/s]
Tokenizing train dataset:  53%|█████▎    | 950206/1807941 [12:10<04:49, 2962.26 examples/s]
Tokenizing train dataset:  53%|█████▎    | 950529/1807941 [12:10<04:43, 3027.43 examples/s]
Tokenizing train dataset:  53%|█████▎    | 950873/1807941 [12:10<04:33, 3134.09 examples/s]
Tokenizing train dataset:  53%|█████▎    | 951328/1807941 [12:10<04:36, 3095.17 examples/s]
Tokenizing train dataset:  53%|█████▎    | 951677/1807941 [12:10<04:27, 3195.18 examples/s]
Tokenizing train dataset:  53%|█████▎    | 952134/1807941 [12:11<04:32, 3136.58 examples/s]
Tokenizing train dataset:  53%|█████▎    | 952474/1807941 [12:11<04:27, 3200.52 examples/s]
Tokenizing train dataset:  53%|█████▎    | 952842/1807941 [12:11<04:17, 3325.42 examples/s]
Tokenizing train dataset:  53%|█████▎    | 953318/1807941 [12:11<04:24, 3236.97 examples/s]
Tokenizing train dataset:  53%|█████▎    | 953677/1807941 [12:11<04:16, 3324.58 examples/s]
Tokenizing train dataset:  53%|█████▎    | 954025/1807941 [12:11<04:13, 3364.23 examples/s]
Tokenizing train dataset:  53%|█████▎    | 954385/1807941 [12:11<04:09, 3424.90 examples/s]
Tokenizing train dataset:  53%|█████▎    | 954906/1807941 [12:11<04:08, 3439.02 examples/s]
Tokenizing train dataset:  53%|█████▎    | 955383/1807941 [12:12<04:14, 3344.72 examples/s]
Tokenizing train dataset:  53%|█████▎    | 955732/1807941 [12:12<04:12, 3368.90 examples/s]
Tokenizing train dataset:  53%|█████▎    | 956218/1807941 [12:12<04:16, 3315.91 examples/s]
Tokenizing train dataset:  53%|█████▎    | 956563/1807941 [12:12<04:14, 3346.06 examples/s]
Tokenizing train dataset:  53%|█████▎    | 957057/1807941 [12:12<04:15, 3325.52 examples/s]
Tokenizing train dataset:  53%|█████▎    | 957556/1807941 [12:12<04:15, 3323.20 examples/s]
Tokenizing train dataset:  53%|█████▎    | 957951/1807941 [12:12<04:05, 3468.79 examples/s]
Tokenizing train dataset:  53%|█████▎    | 958483/1807941 [12:12<04:03, 3490.96 examples/s]
Tokenizing train dataset:  53%|█████▎    | 958921/1807941 [12:13<04:17, 3301.25 examples/s]
Tokenizing train dataset:  53%|█████▎    | 959428/1807941 [12:13<04:15, 3323.82 examples/s]
Tokenizing train dataset:  53%|█████▎    | 959801/1807941 [12:13<04:08, 3417.43 examples/s]
Tokenizing train dataset:  53%|█████▎    | 960299/1807941 [12:13<04:10, 3381.15 examples/s]
Tokenizing train dataset:  53%|█████▎    | 960788/1807941 [12:13<04:13, 3338.20 examples/s]
Tokenizing train dataset:  53%|█████▎    | 961254/1807941 [12:13<04:19, 3258.89 examples/s]
Tokenizing train dataset:  53%|█████▎    | 961716/1807941 [12:13<04:24, 3201.70 examples/s]
Tokenizing train dataset:  53%|█████▎    | 962139/1807941 [12:14<04:34, 3080.87 examples/s]
Tokenizing train dataset:  53%|█████▎    | 962470/1807941 [12:14<04:30, 3129.63 examples/s]
Tokenizing train dataset:  53%|█████▎    | 962795/1807941 [12:14<04:27, 3155.16 examples/s]
Tokenizing train dataset:  53%|█████▎    | 963207/1807941 [12:14<04:40, 3013.00 examples/s]
Tokenizing train dataset:  53%|█████▎    | 963641/1807941 [12:14<04:44, 2970.67 examples/s]
Tokenizing train dataset:  53%|█████▎    | 964037/1807941 [12:14<04:54, 2861.39 examples/s]
Tokenizing train dataset:  53%|█████▎    | 964332/1807941 [12:14<04:53, 2879.05 examples/s]
Tokenizing train dataset:  53%|█████▎    | 964726/1807941 [12:15<05:02, 2791.28 examples/s]
Tokenizing train dataset:  53%|█████▎    | 965107/1807941 [12:15<05:11, 2706.58 examples/s]
Tokenizing train dataset:  53%|█████▎    | 965514/1807941 [12:15<05:11, 2704.18 examples/s]
Tokenizing train dataset:  53%|█████▎    | 965916/1807941 [12:15<05:12, 2694.47 examples/s]
Tokenizing train dataset:  53%|█████▎    | 966300/1807941 [12:15<05:17, 2648.70 examples/s]
Tokenizing train dataset:  53%|█████▎    | 966589/1807941 [12:15<05:11, 2699.25 examples/s]
Tokenizing train dataset:  53%|█████▎    | 966892/1807941 [12:15<05:03, 2775.24 examples/s]
Tokenizing train dataset:  54%|█████▎    | 967261/1807941 [12:15<05:16, 2654.51 examples/s]
Tokenizing train dataset:  54%|█████▎    | 967653/1807941 [12:16<05:18, 2637.22 examples/s]
Tokenizing train dataset:  54%|█████▎    | 968000/1807941 [12:16<05:31, 2532.11 examples/s]
Tokenizing train dataset:  54%|█████▎    | 968265/1807941 [12:16<05:28, 2557.51 examples/s]
Tokenizing train dataset:  54%|█████▎    | 968572/1807941 [12:16<05:52, 2383.16 examples/s]
Tokenizing train dataset:  54%|█████▎    | 968833/1807941 [12:16<05:44, 2433.64 examples/s]
Tokenizing train dataset:  54%|█████▎    | 969230/1807941 [12:16<05:35, 2502.72 examples/s]
Tokenizing train dataset:  54%|█████▎    | 969546/1807941 [12:16<05:15, 2661.55 examples/s]
Tokenizing train dataset:  54%|█████▎    | 969866/1807941 [12:16<04:59, 2797.99 examples/s]
Tokenizing train dataset:  54%|█████▎    | 970158/1807941 [12:17<04:57, 2815.17 examples/s]
Tokenizing train dataset:  54%|█████▎    | 970468/1807941 [12:17<04:49, 2893.69 examples/s]
Tokenizing train dataset:  54%|█████▎    | 970780/1807941 [12:17<04:43, 2954.92 examples/s]
Tokenizing train dataset:  54%|█████▎    | 971088/1807941 [12:17<04:39, 2990.50 examples/s]
Tokenizing train dataset:  54%|█████▎    | 971402/1807941 [12:17<04:35, 3031.44 examples/s]
Tokenizing train dataset:  54%|█████▎    | 971733/1807941 [12:17<04:28, 3111.15 examples/s]
Tokenizing train dataset:  54%|█████▍    | 972046/1807941 [12:17<04:28, 3114.40 examples/s]
Tokenizing train dataset:  54%|█████▍    | 972399/1807941 [12:17<04:18, 3235.86 examples/s]
Tokenizing train dataset:  54%|█████▍    | 972838/1807941 [12:17<04:28, 3110.18 examples/s]
Tokenizing train dataset:  54%|█████▍    | 973155/1807941 [12:18<04:33, 3054.94 examples/s]
Tokenizing train dataset:  54%|█████▍    | 973479/1807941 [12:18<04:28, 3103.79 examples/s]
Tokenizing train dataset:  54%|█████▍    | 973828/1807941 [12:18<04:19, 3208.99 examples/s]
Tokenizing train dataset:  54%|█████▍    | 974186/1807941 [12:18<04:16, 3247.26 examples/s]
Tokenizing train dataset:  54%|█████▍    | 974540/1807941 [12:18<04:10, 3328.21 examples/s]
Tokenizing train dataset:  54%|█████▍    | 974881/1807941 [12:18<04:08, 3350.16 examples/s]
Tokenizing train dataset:  54%|█████▍    | 975365/1807941 [12:18<04:12, 3301.46 examples/s]
Tokenizing train dataset:  54%|█████▍    | 975777/1807941 [12:18<03:56, 3517.03 examples/s]
Tokenizing train dataset:  54%|█████▍    | 976162/1807941 [12:18<04:22, 3168.93 examples/s]
Tokenizing train dataset:  54%|█████▍    | 976644/1807941 [12:19<04:21, 3180.84 examples/s]
Tokenizing train dataset:  54%|█████▍    | 977072/1807941 [12:19<04:30, 3068.83 examples/s]
Tokenizing train dataset:  54%|█████▍    | 977400/1807941 [12:19<04:26, 3115.25 examples/s]
Tokenizing train dataset:  54%|█████▍    | 977750/1807941 [12:19<04:18, 3210.36 examples/s]
Tokenizing train dataset:  54%|█████▍    | 978217/1807941 [12:19<04:21, 3173.89 examples/s]
Tokenizing train dataset:  54%|█████▍    | 978682/1807941 [12:19<04:23, 3147.17 examples/s]
Tokenizing train dataset:  54%|█████▍    | 979112/1807941 [12:19<04:31, 3053.82 examples/s]
Tokenizing train dataset:  54%|█████▍    | 979551/1807941 [12:20<04:35, 3008.69 examples/s]
Tokenizing train dataset:  54%|█████▍    | 979887/1807941 [12:20<04:28, 3086.98 examples/s]
Tokenizing train dataset:  54%|█████▍    | 980306/1807941 [12:20<04:37, 2984.27 examples/s]
Tokenizing train dataset:  54%|█████▍    | 980764/1807941 [12:20<04:35, 3005.64 examples/s]
Tokenizing train dataset:  54%|█████▍    | 981166/1807941 [12:20<04:45, 2900.12 examples/s]
Tokenizing train dataset:  54%|█████▍    | 981481/1807941 [12:20<04:39, 2955.72 examples/s]
Tokenizing train dataset:  54%|█████▍    | 981812/1807941 [12:20<04:31, 3041.24 examples/s]
Tokenizing train dataset:  54%|█████▍    | 982270/1807941 [12:20<04:31, 3042.28 examples/s]
Tokenizing train dataset:  54%|█████▍    | 982580/1807941 [12:21<04:30, 3053.36 examples/s]
Tokenizing train dataset:  54%|█████▍    | 983000/1807941 [12:21<04:42, 2919.03 examples/s]
Tokenizing train dataset:  54%|█████▍    | 983411/1807941 [12:21<04:48, 2858.70 examples/s]
Tokenizing train dataset:  54%|█████▍    | 983836/1807941 [12:21<04:49, 2847.33 examples/s]
Tokenizing train dataset:  54%|█████▍    | 984235/1807941 [12:21<04:55, 2786.40 examples/s]
Tokenizing train dataset:  54%|█████▍    | 984637/1807941 [12:21<04:59, 2750.87 examples/s]
Tokenizing train dataset:  54%|█████▍    | 985012/1807941 [12:21<05:07, 2673.19 examples/s]
Tokenizing train dataset:  54%|█████▍    | 985308/1807941 [12:22<05:00, 2736.81 examples/s]
Tokenizing train dataset:  55%|█████▍    | 985588/1807941 [12:22<04:59, 2749.94 examples/s]
Tokenizing train dataset:  55%|█████▍    | 985891/1807941 [12:22<04:51, 2819.10 examples/s]
Tokenizing train dataset:  55%|█████▍    | 986317/1807941 [12:22<04:51, 2822.37 examples/s]
Tokenizing train dataset:  55%|█████▍    | 986609/1807941 [12:22<04:48, 2843.55 examples/s]
Tokenizing train dataset:  55%|█████▍    | 986913/1807941 [12:22<04:43, 2891.94 examples/s]
Tokenizing train dataset:  55%|█████▍    | 987317/1807941 [12:22<04:51, 2816.40 examples/s]
Tokenizing train dataset:  55%|█████▍    | 987739/1807941 [12:22<04:51, 2811.08 examples/s]
Tokenizing train dataset:  55%|█████▍    | 988167/1807941 [12:23<04:50, 2820.02 examples/s]
Tokenizing train dataset:  55%|█████▍    | 988538/1807941 [12:23<05:02, 2706.09 examples/s]
Tokenizing train dataset:  55%|█████▍    | 988936/1807941 [12:23<04:33, 2999.95 examples/s]
Tokenizing train dataset:  55%|█████▍    | 989361/1807941 [12:23<04:38, 2941.50 examples/s]
Tokenizing train dataset:  55%|█████▍    | 989709/1807941 [12:23<04:26, 3069.37 examples/s]
Tokenizing train dataset:  55%|█████▍    | 990037/1807941 [12:23<04:22, 3120.16 examples/s]
Tokenizing train dataset:  55%|█████▍    | 990401/1807941 [12:23<04:10, 3258.82 examples/s]
Tokenizing train dataset:  55%|█████▍    | 990760/1807941 [12:23<04:04, 3347.27 examples/s]
Tokenizing train dataset:  55%|█████▍    | 991101/1807941 [12:23<04:02, 3362.26 examples/s]
Tokenizing train dataset:  55%|█████▍    | 991462/1807941 [12:24<03:58, 3428.89 examples/s]
Tokenizing train dataset:  55%|█████▍    | 991941/1807941 [12:24<04:04, 3335.50 examples/s]
Tokenizing train dataset:  55%|█████▍    | 992428/1807941 [12:24<04:07, 3301.28 examples/s]
Tokenizing train dataset:  55%|█████▍    | 992900/1807941 [12:24<04:11, 3241.99 examples/s]
Tokenizing train dataset:  55%|█████▍    | 993232/1807941 [12:24<04:10, 3257.00 examples/s]
Tokenizing train dataset:  55%|█████▍    | 993586/1807941 [12:24<04:04, 3327.30 examples/s]
Tokenizing train dataset:  55%|█████▍    | 993984/1807941 [12:24<03:52, 3499.81 examples/s]
Tokenizing train dataset:  55%|█████▌    | 994531/1807941 [12:24<03:49, 3548.03 examples/s]
Tokenizing train dataset:  55%|█████▌    | 994920/1807941 [12:25<03:43, 3631.62 examples/s]
Tokenizing train dataset:  55%|█████▌    | 995447/1807941 [12:25<03:46, 3586.59 examples/s]
Tokenizing train dataset:  55%|█████▌    | 995991/1807941 [12:25<03:45, 3595.93 examples/s]
Tokenizing train dataset:  55%|█████▌    | 996463/1807941 [12:25<03:55, 3446.21 examples/s]
Tokenizing train dataset:  55%|█████▌    | 996878/1807941 [12:25<04:11, 3229.64 examples/s]
Tokenizing train dataset:  55%|█████▌    | 997315/1807941 [12:25<04:19, 3128.09 examples/s]
Tokenizing train dataset:  55%|█████▌    | 997780/1807941 [12:25<04:19, 3116.05 examples/s]
Tokenizing train dataset:  55%|█████▌    | 998204/1807941 [12:26<04:27, 3025.41 examples/s]
Tokenizing train dataset:  55%|█████▌    | 998528/1807941 [12:26<04:23, 3072.13 examples/s]
Tokenizing train dataset:  55%|█████▌    | 998960/1807941 [12:26<04:29, 3006.33 examples/s]
Tokenizing train dataset:  55%|█████▌    | 999380/1807941 [12:26<04:35, 2936.41 examples/s]
Tokenizing train dataset:  55%|█████▌    | 999677/1807941 [12:26<04:34, 2941.98 examples/s]
Tokenizing train dataset:  55%|█████▌    | 999978/1807941 [12:26<04:33, 2958.03 examples/s]
Tokenizing train dataset:  55%|█████▌    | 1000383/1807941 [12:26<04:41, 2865.40 examples/s]
Tokenizing train dataset:  55%|█████▌    | 1000830/1807941 [12:27<04:38, 2900.37 examples/s]
Tokenizing train dataset:  55%|█████▌    | 1001219/1807941 [12:27<04:48, 2799.92 examples/s]
Tokenizing train dataset:  55%|█████▌    | 1001511/1807941 [12:27<04:45, 2824.86 examples/s]
Tokenizing train dataset:  55%|█████▌    | 1001929/1807941 [12:27<04:46, 2809.59 examples/s]
Tokenizing train dataset:  55%|█████▌    | 1002337/1807941 [12:27<04:50, 2775.97 examples/s]
Tokenizing train dataset:  55%|█████▌    | 1002629/1807941 [12:27<04:46, 2806.98 examples/s]
Tokenizing train dataset:  55%|█████▌    | 1003039/1807941 [12:27<04:49, 2779.92 examples/s]
Tokenizing train dataset:  55%|█████▌    | 1003375/1807941 [12:27<04:35, 2915.11 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1003749/1807941 [12:28<04:50, 2767.28 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1004031/1807941 [12:28<04:49, 2779.81 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1004335/1807941 [12:28<04:42, 2846.56 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1004762/1807941 [12:28<04:42, 2843.92 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1005055/1807941 [12:28<07:50, 1708.25 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1005292/1807941 [12:29<11:42, 1142.45 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1005559/1807941 [12:29<09:53, 1350.88 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1005964/1807941 [12:29<07:27, 1792.21 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1006271/1807941 [12:29<06:38, 2010.50 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1006573/1807941 [12:29<07:57, 1677.86 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1006819/1807941 [12:29<08:41, 1534.78 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1007023/1807941 [12:30<09:45, 1366.84 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1007218/1807941 [12:30<09:54, 1346.34 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1007385/1807941 [12:30<10:23, 1283.63 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1007583/1807941 [12:30<10:20, 1289.19 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1007810/1807941 [12:30<08:57, 1489.00 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1007997/1807941 [12:30<08:28, 1572.83 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1008188/1807941 [12:31<09:06, 1462.39 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1008421/1807941 [12:31<08:58, 1484.87 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1008582/1807941 [12:31<08:48, 1511.44 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1008792/1807941 [12:31<10:07, 1316.23 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1008987/1807941 [12:31<11:11, 1189.49 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1009117/1807941 [12:31<12:16, 1084.08 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1009272/1807941 [12:31<12:30, 1064.58 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1009397/1807941 [12:32<12:05, 1100.96 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1009518/1807941 [12:32<11:53, 1119.16 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1009668/1807941 [12:32<12:22, 1075.58 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1009786/1807941 [12:32<12:06, 1099.16 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1009913/1807941 [12:32<11:39, 1140.70 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1010065/1807941 [12:32<12:31, 1062.14 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1010191/1807941 [12:32<11:59, 1109.07 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1010347/1807941 [12:32<12:16, 1083.30 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1010467/1807941 [12:33<12:08, 1094.64 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1010596/1807941 [12:33<11:37, 1142.91 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1010758/1807941 [12:33<11:58, 1108.95 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1010885/1807941 [12:33<11:40, 1138.26 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1011029/1807941 [12:33<12:28, 1064.02 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1011197/1807941 [12:33<12:21, 1074.80 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1011346/1807941 [12:33<12:44, 1041.87 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1011452/1807941 [12:33<12:42, 1044.68 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1011574/1807941 [12:34<12:14, 1084.39 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1011737/1807941 [12:34<12:16, 1081.68 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1011894/1807941 [12:34<12:25, 1068.50 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1012054/1807941 [12:34<12:48, 1035.93 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1012207/1807941 [12:34<13:05, 1012.58 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1012343/1807941 [12:34<13:32, 979.12 examples/s] 
Tokenizing train dataset:  56%|█████▌    | 1012468/1807941 [12:34<12:51, 1031.17 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1012622/1807941 [12:35<13:03, 1015.37 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1012745/1807941 [12:35<12:26, 1065.10 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1012866/1807941 [12:35<12:06, 1094.12 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1012991/1807941 [12:35<11:44, 1128.82 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1013140/1807941 [12:35<12:18, 1076.26 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1013250/1807941 [12:35<12:17, 1077.10 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1013369/1807941 [12:35<12:01, 1101.99 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1013489/1807941 [12:35<11:45, 1126.68 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1013654/1807941 [12:36<11:54, 1110.90 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1013809/1807941 [12:36<12:17, 1077.09 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1013946/1807941 [12:36<11:31, 1147.92 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1014064/1807941 [12:36<13:08, 1006.80 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1014183/1807941 [12:36<12:43, 1040.07 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1014345/1807941 [12:36<12:39, 1045.04 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1014539/1807941 [12:36<10:28, 1262.57 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1014727/1807941 [12:36<09:18, 1420.57 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1014920/1807941 [12:37<08:29, 1555.88 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1015095/1807941 [12:37<08:24, 1572.84 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1015288/1807941 [12:37<07:54, 1669.40 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1015482/1807941 [12:37<07:34, 1744.05 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1015675/1807941 [12:37<07:20, 1796.91 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1015869/1807941 [12:37<07:11, 1836.32 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1016132/1807941 [12:37<07:19, 1801.61 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1016325/1807941 [12:37<07:11, 1833.23 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1016517/1807941 [12:37<07:06, 1856.51 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1016709/1807941 [12:37<07:02, 1873.88 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1016901/1807941 [12:38<06:59, 1883.63 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1017162/1807941 [12:38<07:13, 1823.33 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1017355/1807941 [12:38<07:07, 1847.37 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1017549/1807941 [12:38<07:02, 1870.67 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1017742/1807941 [12:38<06:59, 1884.32 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1017936/1807941 [12:38<06:56, 1898.31 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1018195/1807941 [12:38<07:11, 1829.53 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1018382/1807941 [12:38<07:09, 1837.25 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1018573/1807941 [12:38<07:05, 1855.44 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1018764/1807941 [12:39<07:02, 1866.71 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1018961/1807941 [12:39<06:56, 1893.72 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1019225/1807941 [12:39<07:08, 1838.77 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1019420/1807941 [12:39<07:02, 1866.51 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1019618/1807941 [12:39<06:56, 1894.59 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1019814/1807941 [12:39<06:52, 1909.62 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1020096/1807941 [12:39<07:04, 1856.81 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1020288/1807941 [12:39<07:00, 1871.18 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1020478/1807941 [12:39<06:59, 1876.53 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1020672/1807941 [12:40<06:56, 1891.24 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1020866/1807941 [12:40<06:53, 1901.17 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1021125/1807941 [12:40<07:09, 1831.25 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1021318/1807941 [12:40<07:04, 1852.14 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1021510/1807941 [12:40<07:01, 1866.66 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1021701/1807941 [12:40<06:59, 1875.51 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1021894/1807941 [12:40<06:56, 1887.40 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1022104/1807941 [12:40<06:59, 1873.49 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1022300/1807941 [12:40<06:54, 1897.58 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1022563/1807941 [12:41<07:07, 1837.95 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1022824/1807941 [12:41<07:16, 1797.97 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1023088/1807941 [12:41<07:33, 1730.79 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1023271/1807941 [12:41<07:27, 1751.61 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1023458/1807941 [12:41<07:21, 1777.43 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1023639/1807941 [12:41<07:20, 1780.48 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1023909/1807941 [12:41<07:19, 1784.51 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1024157/1807941 [12:42<07:31, 1736.32 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1024340/1807941 [12:42<07:26, 1754.10 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1024585/1807941 [12:42<07:38, 1708.66 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1024849/1807941 [12:42<07:34, 1721.64 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1025134/1807941 [12:42<07:20, 1775.17 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1025421/1807941 [12:42<06:25, 2027.53 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1025711/1807941 [12:42<06:32, 1992.77 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1025986/1807941 [12:42<06:43, 1937.05 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1026238/1807941 [12:43<07:01, 1853.91 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1026503/1807941 [12:43<06:23, 2036.33 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1026848/1807941 [12:43<05:27, 2384.15 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1027165/1807941 [12:43<05:05, 2555.99 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1027506/1807941 [12:43<04:40, 2784.71 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1027854/1807941 [12:43<04:22, 2969.94 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1028163/1807941 [12:43<04:19, 3000.63 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1028498/1807941 [12:43<04:11, 3099.61 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1028837/1807941 [12:43<04:05, 3179.83 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1029228/1807941 [12:44<04:57, 2619.13 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1029523/1807941 [12:44<05:24, 2401.05 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1029802/1807941 [12:44<05:49, 2223.31 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1030093/1807941 [12:44<06:50, 1897.02 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1030384/1807941 [12:44<06:47, 1907.10 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1030667/1807941 [12:44<06:49, 1899.29 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1030947/1807941 [12:45<06:51, 1887.64 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1031205/1807941 [12:45<07:03, 1833.53 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1031482/1807941 [12:45<07:03, 1833.91 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1031673/1807941 [12:45<06:59, 1849.86 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1032000/1807941 [12:45<06:43, 1923.38 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1032377/1807941 [12:45<05:30, 2346.53 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1032652/1807941 [12:45<05:56, 2172.78 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1032930/1807941 [12:46<06:15, 2066.33 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1033201/1807941 [12:46<06:30, 1983.57 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1033502/1807941 [12:46<06:29, 1988.17 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1033774/1807941 [12:46<06:41, 1930.48 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1034044/1807941 [12:46<06:49, 1888.18 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1034246/1807941 [12:46<06:43, 1915.42 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1034451/1807941 [12:46<06:37, 1944.62 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1034660/1807941 [12:46<06:30, 1978.47 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1034870/1807941 [12:47<06:25, 2006.81 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1035154/1807941 [12:47<06:34, 1961.29 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1035363/1807941 [12:47<06:28, 1990.84 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1035572/1807941 [12:47<06:23, 2014.09 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1035868/1807941 [12:47<06:26, 1995.38 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1036141/1807941 [12:47<06:39, 1930.42 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1036430/1807941 [12:47<06:40, 1926.83 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1036718/1807941 [12:48<06:41, 1922.22 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1037000/1807941 [12:48<06:51, 1874.54 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1037263/1807941 [12:48<06:59, 1835.21 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1037524/1807941 [12:48<07:07, 1802.44 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1037785/1807941 [12:48<07:12, 1779.24 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1038025/1807941 [12:48<07:27, 1721.66 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1038199/1807941 [12:48<07:26, 1722.98 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1038372/1807941 [12:48<07:26, 1723.49 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1038549/1807941 [12:49<07:24, 1731.80 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1038725/1807941 [12:49<07:23, 1735.53 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1038904/1807941 [12:49<07:20, 1746.82 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1039144/1807941 [12:49<07:35, 1687.31 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1039321/1807941 [12:49<07:31, 1703.76 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1039503/1807941 [12:49<07:23, 1733.05 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1039686/1807941 [12:49<07:17, 1757.27 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1039875/1807941 [12:49<07:08, 1792.65 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1040130/1807941 [12:49<07:17, 1753.25 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1040318/1807941 [12:50<07:10, 1783.61 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1040504/1807941 [12:50<07:05, 1802.83 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1040690/1807941 [12:50<07:02, 1817.99 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1040877/1807941 [12:50<06:58, 1832.30 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1041116/1807941 [12:50<07:21, 1736.67 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1041373/1807941 [12:50<07:24, 1724.33 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1041635/1807941 [12:50<07:23, 1727.31 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1041896/1807941 [12:51<07:23, 1727.52 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1042128/1807941 [12:51<07:39, 1667.45 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1042299/1807941 [12:51<07:36, 1675.68 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1042473/1807941 [12:51<07:33, 1687.86 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1042645/1807941 [12:51<07:31, 1695.37 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1042819/1807941 [12:51<07:28, 1705.45 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1042995/1807941 [12:51<07:25, 1715.75 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1043228/1807941 [12:51<07:42, 1652.41 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1043398/1807941 [12:51<07:40, 1661.55 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1043571/1807941 [12:52<07:35, 1676.29 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1043741/1807941 [12:52<07:35, 1677.73 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1043914/1807941 [12:52<07:32, 1689.67 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1044086/1807941 [12:52<07:47, 1635.36 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1044259/1807941 [12:52<07:40, 1659.29 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1044431/1807941 [12:52<07:36, 1673.91 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1044605/1807941 [12:52<07:31, 1691.92 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1044776/1807941 [12:52<07:29, 1695.96 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1044949/1807941 [12:52<07:28, 1702.27 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1045180/1807941 [12:52<07:45, 1638.68 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1045353/1807941 [12:53<07:39, 1661.01 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1045527/1807941 [12:53<07:33, 1681.42 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1045702/1807941 [12:53<07:28, 1699.51 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1045885/1807941 [12:53<07:18, 1736.01 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1046136/1807941 [12:53<07:25, 1708.90 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1046321/1807941 [12:53<07:16, 1744.88 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1046500/1807941 [12:53<07:13, 1756.10 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1046763/1807941 [12:53<07:14, 1750.28 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1047001/1807941 [12:54<07:30, 1690.30 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1047257/1807941 [12:54<07:29, 1691.47 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1047428/1807941 [12:54<07:28, 1694.91 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1047602/1807941 [12:54<07:26, 1702.40 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1047773/1807941 [12:54<07:26, 1703.27 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1047948/1807941 [12:54<07:23, 1713.02 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1048182/1807941 [12:54<07:40, 1651.14 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1048354/1807941 [12:54<07:35, 1666.02 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1048529/1807941 [12:54<07:30, 1686.13 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1048703/1807941 [12:55<07:26, 1699.38 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1048881/1807941 [12:55<07:21, 1717.93 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1049118/1807941 [12:55<07:37, 1660.40 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1049289/1807941 [12:55<07:33, 1671.07 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1049459/1807941 [12:55<07:32, 1676.22 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1049631/1807941 [12:55<07:30, 1684.99 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1049801/1807941 [12:55<07:30, 1684.53 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1049981/1807941 [12:55<07:21, 1714.86 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1050218/1807941 [12:55<07:37, 1657.97 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1050392/1807941 [12:56<07:32, 1675.70 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1050569/1807941 [12:56<07:25, 1698.40 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1050746/1807941 [12:56<07:21, 1715.18 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1050924/1807941 [12:56<07:17, 1728.46 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1051168/1807941 [12:56<07:36, 1658.67 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1051340/1807941 [12:56<07:32, 1672.91 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1051513/1807941 [12:56<07:28, 1686.53 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1051686/1807941 [12:56<07:26, 1695.17 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1051859/1807941 [12:56<07:23, 1702.99 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1052091/1807941 [12:57<07:41, 1638.98 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1052260/1807941 [12:57<07:37, 1652.22 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1052431/1807941 [12:57<07:34, 1663.58 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1052603/1807941 [12:57<07:30, 1675.25 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1052773/1807941 [12:57<07:29, 1680.69 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1052944/1807941 [12:57<07:27, 1687.80 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1053175/1807941 [12:57<07:43, 1627.34 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1053345/1807941 [12:57<07:38, 1645.19 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1053519/1807941 [12:57<07:32, 1667.11 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1053690/1807941 [12:58<07:29, 1677.83 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1053864/1807941 [12:58<07:25, 1692.49 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1054098/1807941 [12:58<07:40, 1638.47 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1054269/1807941 [12:58<07:35, 1653.35 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1054440/1807941 [12:58<07:32, 1665.09 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1054611/1807941 [12:58<07:29, 1676.60 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1054781/1807941 [12:58<07:27, 1682.00 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1054954/1807941 [12:58<07:24, 1693.11 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1055185/1807941 [12:58<07:41, 1629.91 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1055356/1807941 [12:59<07:36, 1647.83 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1055532/1807941 [12:59<07:28, 1677.48 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1055701/1807941 [12:59<07:27, 1679.57 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1055876/1807941 [12:59<07:23, 1695.37 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1056109/1807941 [12:59<07:39, 1637.57 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1056278/1807941 [12:59<07:35, 1648.61 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1056451/1807941 [12:59<07:30, 1667.20 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1056625/1807941 [12:59<07:26, 1682.73 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1056804/1807941 [12:59<07:19, 1710.62 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1056988/1807941 [12:59<07:10, 1742.53 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1057236/1807941 [13:00<07:20, 1704.73 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1057420/1807941 [13:00<07:11, 1738.26 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1057607/1807941 [13:00<07:03, 1771.89 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1057790/1807941 [13:00<07:00, 1785.74 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1057976/1807941 [13:00<06:55, 1804.51 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1058229/1807941 [13:00<07:07, 1755.40 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1058413/1807941 [13:00<07:01, 1776.24 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1058601/1807941 [13:00<06:55, 1804.48 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1058788/1807941 [13:00<06:51, 1818.42 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1058974/1807941 [13:01<06:49, 1828.91 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1059229/1807941 [13:01<07:01, 1775.06 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1059414/1807941 [13:01<06:57, 1792.60 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1059678/1807941 [13:01<07:00, 1777.76 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1059942/1807941 [13:01<07:03, 1766.99 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1060177/1807941 [13:01<07:19, 1700.00 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1060349/1807941 [13:01<07:19, 1702.31 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1060524/1807941 [13:01<07:17, 1709.92 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1060697/1807941 [13:02<07:15, 1714.05 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1060875/1807941 [13:02<07:12, 1728.04 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1061113/1807941 [13:02<07:26, 1673.54 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1061289/1807941 [13:02<07:21, 1691.51 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1061464/1807941 [13:02<07:17, 1705.52 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1061637/1807941 [13:02<07:16, 1709.61 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1061809/1807941 [13:02<07:16, 1708.49 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1061993/1807941 [13:02<07:08, 1742.77 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1062242/1807941 [13:02<07:16, 1708.56 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1062429/1807941 [13:03<07:06, 1748.91 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1062617/1807941 [13:03<06:58, 1782.24 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1062803/1807941 [13:03<06:53, 1803.19 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1062992/1807941 [13:03<06:48, 1824.70 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1063246/1807941 [13:03<07:00, 1770.85 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1063431/1807941 [13:03<06:55, 1791.41 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1063618/1807941 [13:03<06:51, 1810.68 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1063806/1807941 [13:03<06:47, 1827.27 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1063994/1807941 [13:03<06:44, 1840.58 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1064249/1807941 [13:04<06:56, 1783.52 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1064438/1807941 [13:04<06:51, 1808.72 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1064632/1807941 [13:04<06:43, 1840.30 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1064825/1807941 [13:04<06:38, 1862.79 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1065093/1807941 [13:04<06:49, 1813.32 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1065356/1807941 [13:04<06:55, 1788.71 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1065620/1807941 [13:04<06:57, 1776.66 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1065883/1807941 [13:05<07:00, 1766.79 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1066123/1807941 [13:05<07:13, 1711.22 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1066380/1807941 [13:05<07:13, 1710.45 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1066555/1807941 [13:05<07:11, 1716.70 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1066729/1807941 [13:05<07:11, 1718.64 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1066902/1807941 [13:05<07:11, 1718.98 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1067134/1807941 [13:05<07:27, 1654.80 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1067302/1807941 [13:05<07:26, 1659.07 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1067472/1807941 [13:05<07:23, 1667.74 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1067645/1807941 [13:06<07:20, 1681.17 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1067815/1807941 [13:06<07:19, 1682.54 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1067985/1807941 [13:06<07:19, 1684.98 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1068213/1807941 [13:06<07:37, 1616.12 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1068378/1807941 [13:06<07:35, 1623.56 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1068549/1807941 [13:06<07:29, 1645.48 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1068721/1807941 [13:06<07:24, 1662.03 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1068895/1807941 [13:06<07:19, 1682.58 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1069129/1807941 [13:06<07:32, 1633.99 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1069302/1807941 [13:07<07:25, 1657.47 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1069473/1807941 [13:07<07:22, 1669.57 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1069646/1807941 [13:07<07:18, 1683.24 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1069817/1807941 [13:07<07:17, 1687.71 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1069989/1807941 [13:07<07:15, 1694.07 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1070223/1807941 [13:07<07:29, 1640.13 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1070402/1807941 [13:07<07:19, 1677.15 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1070587/1807941 [13:07<07:08, 1719.34 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1070847/1807941 [13:07<07:08, 1721.20 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1071084/1807941 [13:08<07:22, 1664.42 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1071255/1807941 [13:08<07:20, 1672.54 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1071427/1807941 [13:08<07:17, 1681.61 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1071600/1807941 [13:08<07:15, 1692.33 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1071772/1807941 [13:08<07:13, 1696.27 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1071945/1807941 [13:08<07:12, 1703.12 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1072177/1807941 [13:08<07:28, 1638.78 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1072346/1807941 [13:08<07:25, 1651.47 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1072518/1807941 [13:08<07:21, 1667.52 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1072687/1807941 [13:09<07:19, 1672.26 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1072861/1807941 [13:09<07:15, 1687.14 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1073091/1807941 [13:09<07:32, 1624.68 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1073259/1807941 [13:09<07:29, 1635.39 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1073427/1807941 [13:09<07:26, 1644.53 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1073596/1807941 [13:09<07:23, 1656.58 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1073767/1807941 [13:09<07:20, 1665.62 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1073941/1807941 [13:09<07:16, 1683.07 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1074175/1807941 [13:09<07:29, 1633.83 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1074349/1807941 [13:10<07:21, 1660.31 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1074526/1807941 [13:10<07:14, 1687.29 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1074701/1807941 [13:10<07:11, 1700.23 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1074878/1807941 [13:10<07:07, 1716.57 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1075116/1807941 [13:10<07:21, 1661.73 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1075288/1807941 [13:10<07:17, 1674.03 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1075460/1807941 [13:10<07:15, 1682.59 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1075633/1807941 [13:10<07:12, 1694.17 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1075804/1807941 [13:10<07:11, 1696.68 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1075982/1807941 [13:11<07:05, 1718.51 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1076211/1807941 [13:11<07:25, 1640.67 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1076378/1807941 [13:11<07:24, 1645.87 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1076550/1807941 [13:11<07:19, 1664.42 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1076719/1807941 [13:11<07:18, 1667.06 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1076889/1807941 [13:11<07:16, 1674.10 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1077119/1807941 [13:11<07:31, 1617.02 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1077287/1807941 [13:11<07:27, 1632.05 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1077457/1807941 [13:11<07:23, 1647.31 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1077629/1807941 [13:12<07:18, 1666.99 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1077798/1807941 [13:12<07:16, 1671.85 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1077969/1807941 [13:12<07:14, 1681.82 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1078199/1807941 [13:12<07:30, 1621.42 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1078366/1807941 [13:12<07:26, 1633.20 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1078538/1807941 [13:12<07:20, 1655.85 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1078710/1807941 [13:12<07:16, 1671.90 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1078883/1807941 [13:12<07:12, 1684.63 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1079112/1807941 [13:12<07:29, 1620.06 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1079284/1807941 [13:13<07:23, 1643.07 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1079455/1807941 [13:13<07:18, 1659.91 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1079628/1807941 [13:13<07:14, 1675.47 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1079799/1807941 [13:13<07:13, 1680.56 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1079974/1807941 [13:13<07:09, 1695.78 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1080205/1807941 [13:13<07:25, 1633.22 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1080378/1807941 [13:13<07:19, 1655.17 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1080553/1807941 [13:13<07:13, 1678.30 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1080727/1807941 [13:13<07:09, 1694.87 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1080901/1807941 [13:14<07:06, 1704.96 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1081134/1807941 [13:14<07:21, 1645.17 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1081310/1807941 [13:14<07:14, 1672.79 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1081487/1807941 [13:14<07:07, 1698.54 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1081669/1807941 [13:14<06:59, 1729.28 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1081852/1807941 [13:14<06:53, 1753.95 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1082108/1807941 [13:14<06:58, 1733.60 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1082304/1807941 [13:14<06:45, 1789.97 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1082502/1807941 [13:14<06:34, 1839.46 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1082704/1807941 [13:15<06:24, 1886.97 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1082907/1807941 [13:15<06:16, 1923.32 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1083197/1807941 [13:15<06:24, 1884.84 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1083395/1807941 [13:15<06:19, 1907.04 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1083681/1807941 [13:15<06:20, 1904.02 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1083963/1807941 [13:15<06:22, 1893.13 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1084215/1807941 [13:15<06:37, 1820.75 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1084402/1807941 [13:15<06:35, 1828.34 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1084593/1807941 [13:16<06:31, 1846.22 examples/s]
Tokenizing train dataset:  60%|██████    | 1084782/1807941 [13:16<06:29, 1854.79 examples/s]
Tokenizing train dataset:  60%|██████    | 1084972/1807941 [13:16<06:27, 1864.69 examples/s]
Tokenizing train dataset:  60%|██████    | 1085208/1807941 [13:16<06:51, 1754.71 examples/s]
Tokenizing train dataset:  60%|██████    | 1085474/1807941 [13:16<06:51, 1757.82 examples/s]
Tokenizing train dataset:  60%|██████    | 1085734/1807941 [13:16<06:53, 1745.91 examples/s]
Tokenizing train dataset:  60%|██████    | 1085916/1807941 [13:16<06:49, 1761.34 examples/s]
Tokenizing train dataset:  60%|██████    | 1086180/1807941 [13:16<06:59, 1721.63 examples/s]
Tokenizing train dataset:  60%|██████    | 1086361/1807941 [13:17<06:54, 1742.11 examples/s]
Tokenizing train dataset:  60%|██████    | 1086546/1807941 [13:17<06:48, 1765.28 examples/s]
Tokenizing train dataset:  60%|██████    | 1086731/1807941 [13:17<06:44, 1784.84 examples/s]
Tokenizing train dataset:  60%|██████    | 1086917/1807941 [13:17<06:39, 1803.01 examples/s]
Tokenizing train dataset:  60%|██████    | 1087180/1807941 [13:17<06:52, 1746.12 examples/s]
Tokenizing train dataset:  60%|██████    | 1087362/1807941 [13:17<06:48, 1761.81 examples/s]
Tokenizing train dataset:  60%|██████    | 1087548/1807941 [13:17<06:43, 1784.90 examples/s]
Tokenizing train dataset:  60%|██████    | 1087733/1807941 [13:17<06:39, 1801.28 examples/s]
Tokenizing train dataset:  60%|██████    | 1087924/1807941 [13:17<06:33, 1829.37 examples/s]
Tokenizing train dataset:  60%|██████    | 1088192/1807941 [13:18<06:38, 1804.21 examples/s]
Tokenizing train dataset:  60%|██████    | 1088387/1807941 [13:18<06:31, 1839.79 examples/s]
Tokenizing train dataset:  60%|██████    | 1088582/1807941 [13:18<06:25, 1866.51 examples/s]
Tokenizing train dataset:  60%|██████    | 1088772/1807941 [13:18<06:23, 1872.88 examples/s]
Tokenizing train dataset:  60%|██████    | 1088965/1807941 [13:18<06:20, 1887.22 examples/s]
Tokenizing train dataset:  60%|██████    | 1089228/1807941 [13:18<06:32, 1832.53 examples/s]
Tokenizing train dataset:  60%|██████    | 1089503/1807941 [13:18<06:32, 1828.53 examples/s]
Tokenizing train dataset:  60%|██████    | 1089774/1807941 [13:18<06:35, 1814.94 examples/s]
Tokenizing train dataset:  60%|██████    | 1090020/1807941 [13:19<06:49, 1754.76 examples/s]
Tokenizing train dataset:  60%|██████    | 1090198/1807941 [13:19<06:48, 1756.63 examples/s]
Tokenizing train dataset:  60%|██████    | 1090380/1807941 [13:19<06:45, 1769.16 examples/s]
Tokenizing train dataset:  60%|██████    | 1090560/1807941 [13:19<06:44, 1775.35 examples/s]
Tokenizing train dataset:  60%|██████    | 1090740/1807941 [13:19<06:43, 1779.08 examples/s]
Tokenizing train dataset:  60%|██████    | 1090920/1807941 [13:19<06:42, 1781.35 examples/s]
Tokenizing train dataset:  60%|██████    | 1091174/1807941 [13:19<06:57, 1717.41 examples/s]
Tokenizing train dataset:  60%|██████    | 1091431/1807941 [13:19<06:57, 1714.52 examples/s]
Tokenizing train dataset:  60%|██████    | 1091605/1807941 [13:19<06:56, 1718.70 examples/s]
Tokenizing train dataset:  60%|██████    | 1091863/1807941 [13:20<06:57, 1715.45 examples/s]
Tokenizing train dataset:  60%|██████    | 1092100/1807941 [13:20<07:09, 1667.70 examples/s]
Tokenizing train dataset:  60%|██████    | 1092275/1807941 [13:20<07:04, 1684.68 examples/s]
Tokenizing train dataset:  60%|██████    | 1092448/1807941 [13:20<07:02, 1692.91 examples/s]
Tokenizing train dataset:  60%|██████    | 1092626/1807941 [13:20<06:57, 1712.80 examples/s]
Tokenizing train dataset:  60%|██████    | 1092799/1807941 [13:20<06:57, 1714.14 examples/s]
Tokenizing train dataset:  60%|██████    | 1092972/1807941 [13:20<06:56, 1717.18 examples/s]
Tokenizing train dataset:  60%|██████    | 1093207/1807941 [13:20<07:11, 1656.20 examples/s]
Tokenizing train dataset:  60%|██████    | 1093379/1807941 [13:21<07:07, 1671.25 examples/s]
Tokenizing train dataset:  60%|██████    | 1093553/1807941 [13:21<07:03, 1687.43 examples/s]
Tokenizing train dataset:  60%|██████    | 1093728/1807941 [13:21<06:58, 1704.79 examples/s]
Tokenizing train dataset:  61%|██████    | 1093903/1807941 [13:21<06:56, 1715.60 examples/s]
Tokenizing train dataset:  61%|██████    | 1094140/1807941 [13:21<07:09, 1660.44 examples/s]
Tokenizing train dataset:  61%|██████    | 1094321/1807941 [13:21<07:00, 1698.85 examples/s]
Tokenizing train dataset:  61%|██████    | 1094500/1807941 [13:21<06:54, 1721.94 examples/s]
Tokenizing train dataset:  61%|██████    | 1094686/1807941 [13:21<06:45, 1757.68 examples/s]
Tokenizing train dataset:  61%|██████    | 1094874/1807941 [13:21<06:38, 1789.68 examples/s]
Tokenizing train dataset:  61%|██████    | 1095118/1807941 [13:22<06:53, 1723.42 examples/s]
Tokenizing train dataset:  61%|██████    | 1095294/1807941 [13:22<06:51, 1730.86 examples/s]
Tokenizing train dataset:  61%|██████    | 1095558/1807941 [13:22<06:50, 1735.71 examples/s]
Tokenizing train dataset:  61%|██████    | 1095738/1807941 [13:22<06:47, 1747.91 examples/s]
Tokenizing train dataset:  61%|██████    | 1095918/1807941 [13:22<06:44, 1759.53 examples/s]
Tokenizing train dataset:  61%|██████    | 1096170/1807941 [13:22<07:01, 1689.86 examples/s]
Tokenizing train dataset:  61%|██████    | 1096345/1807941 [13:22<06:57, 1702.90 examples/s]
Tokenizing train dataset:  61%|██████    | 1096524/1807941 [13:22<06:52, 1722.91 examples/s]
Tokenizing train dataset:  61%|██████    | 1096785/1807941 [13:23<06:52, 1725.20 examples/s]
Tokenizing train dataset:  61%|██████    | 1096963/1807941 [13:23<06:49, 1737.21 examples/s]
Tokenizing train dataset:  61%|██████    | 1097202/1807941 [13:23<07:02, 1681.46 examples/s]
Tokenizing train dataset:  61%|██████    | 1097375/1807941 [13:23<07:00, 1691.70 examples/s]
Tokenizing train dataset:  61%|██████    | 1097553/1807941 [13:23<06:55, 1711.37 examples/s]
Tokenizing train dataset:  61%|██████    | 1097728/1807941 [13:23<06:52, 1719.70 examples/s]
Tokenizing train dataset:  61%|██████    | 1097906/1807941 [13:23<06:49, 1732.11 examples/s]
Tokenizing train dataset:  61%|██████    | 1098144/1807941 [13:23<07:04, 1673.17 examples/s]
Tokenizing train dataset:  61%|██████    | 1098320/1807941 [13:23<06:58, 1693.87 examples/s]
Tokenizing train dataset:  61%|██████    | 1098497/1807941 [13:24<06:54, 1710.20 examples/s]
Tokenizing train dataset:  61%|██████    | 1098679/1807941 [13:24<06:48, 1734.54 examples/s]
Tokenizing train dataset:  61%|██████    | 1098856/1807941 [13:24<06:46, 1742.82 examples/s]
Tokenizing train dataset:  61%|██████    | 1099097/1807941 [13:24<06:59, 1687.88 examples/s]
Tokenizing train dataset:  61%|██████    | 1099276/1807941 [13:24<06:53, 1712.47 examples/s]
Tokenizing train dataset:  61%|██████    | 1099453/1807941 [13:24<06:50, 1726.71 examples/s]
Tokenizing train dataset:  61%|██████    | 1099632/1807941 [13:24<06:46, 1741.04 examples/s]
Tokenizing train dataset:  61%|██████    | 1099809/1807941 [13:24<06:45, 1745.94 examples/s]
Tokenizing train dataset:  61%|██████    | 1099991/1807941 [13:24<06:41, 1765.27 examples/s]
Tokenizing train dataset:  61%|██████    | 1100230/1807941 [13:25<06:57, 1693.46 examples/s]
Tokenizing train dataset:  61%|██████    | 1100406/1807941 [13:25<06:53, 1709.67 examples/s]
Tokenizing train dataset:  61%|██████    | 1100587/1807941 [13:25<06:47, 1736.37 examples/s]
Tokenizing train dataset:  61%|██████    | 1100763/1807941 [13:25<06:46, 1738.68 examples/s]
Tokenizing train dataset:  61%|██████    | 1100938/1807941 [13:25<06:46, 1740.17 examples/s]
Tokenizing train dataset:  61%|██████    | 1101172/1807941 [13:25<07:04, 1666.85 examples/s]
Tokenizing train dataset:  61%|██████    | 1101349/1807941 [13:25<06:57, 1691.14 examples/s]
Tokenizing train dataset:  61%|██████    | 1101528/1807941 [13:25<06:52, 1714.25 examples/s]
Tokenizing train dataset:  61%|██████    | 1101705/1807941 [13:25<06:49, 1725.13 examples/s]
Tokenizing train dataset:  61%|██████    | 1101883/1807941 [13:25<06:46, 1737.68 examples/s]
Tokenizing train dataset:  61%|██████    | 1102124/1807941 [13:26<06:58, 1684.68 examples/s]
Tokenizing train dataset:  61%|██████    | 1102303/1807941 [13:26<06:52, 1709.98 examples/s]
Tokenizing train dataset:  61%|██████    | 1102482/1807941 [13:26<06:47, 1729.61 examples/s]
Tokenizing train dataset:  61%|██████    | 1102664/1807941 [13:26<06:43, 1749.69 examples/s]
Tokenizing train dataset:  61%|██████    | 1102842/1807941 [13:26<06:41, 1755.87 examples/s]
Tokenizing train dataset:  61%|██████    | 1103084/1807941 [13:26<06:57, 1688.60 examples/s]
Tokenizing train dataset:  61%|██████    | 1103340/1807941 [13:26<06:56, 1689.84 examples/s]
Tokenizing train dataset:  61%|██████    | 1103516/1807941 [13:26<06:53, 1703.75 examples/s]
Tokenizing train dataset:  61%|██████    | 1103773/1807941 [13:27<06:52, 1705.67 examples/s]
Tokenizing train dataset:  61%|██████    | 1103946/1807941 [13:27<06:51, 1710.24 examples/s]
Tokenizing train dataset:  61%|██████    | 1104184/1807941 [13:27<07:03, 1662.90 examples/s]
Tokenizing train dataset:  61%|██████    | 1104357/1807941 [13:27<06:59, 1677.46 examples/s]
Tokenizing train dataset:  61%|██████    | 1104534/1807941 [13:27<06:54, 1696.57 examples/s]
Tokenizing train dataset:  61%|██████    | 1104708/1807941 [13:27<06:52, 1706.30 examples/s]
Tokenizing train dataset:  61%|██████    | 1104884/1807941 [13:27<06:49, 1717.28 examples/s]
Tokenizing train dataset:  61%|██████    | 1105120/1807941 [13:27<07:03, 1659.02 examples/s]
Tokenizing train dataset:  61%|██████    | 1105292/1807941 [13:27<06:59, 1673.30 examples/s]
Tokenizing train dataset:  61%|██████    | 1105466/1807941 [13:28<06:56, 1687.59 examples/s]
Tokenizing train dataset:  61%|██████    | 1105642/1807941 [13:28<06:51, 1705.13 examples/s]
Tokenizing train dataset:  61%|██████    | 1105820/1807941 [13:28<06:47, 1724.09 examples/s]
Tokenizing train dataset:  61%|██████    | 1105995/1807941 [13:28<06:46, 1728.49 examples/s]
Tokenizing train dataset:  61%|██████    | 1106226/1807941 [13:28<07:04, 1653.03 examples/s]
Tokenizing train dataset:  61%|██████    | 1106398/1807941 [13:28<07:00, 1667.76 examples/s]
Tokenizing train dataset:  61%|██████    | 1106571/1807941 [13:28<06:57, 1681.74 examples/s]
Tokenizing train dataset:  61%|██████    | 1106745/1807941 [13:28<06:53, 1695.64 examples/s]
Tokenizing train dataset:  61%|██████    | 1106920/1807941 [13:28<06:50, 1706.27 examples/s]
Tokenizing train dataset:  61%|██████    | 1107170/1807941 [13:29<07:04, 1650.56 examples/s]
Tokenizing train dataset:  61%|██████    | 1107341/1807941 [13:29<07:01, 1662.30 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1107514/1807941 [13:29<06:57, 1677.73 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1107688/1807941 [13:29<06:54, 1691.23 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1107865/1807941 [13:29<06:49, 1711.46 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1108103/1807941 [13:29<07:01, 1658.76 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1108275/1807941 [13:29<06:58, 1673.68 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1108449/1807941 [13:29<06:54, 1688.06 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1108626/1807941 [13:29<06:49, 1707.23 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1108801/1807941 [13:30<06:47, 1716.43 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1108974/1807941 [13:30<06:47, 1717.31 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1109212/1807941 [13:30<06:59, 1664.99 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1109386/1807941 [13:30<06:55, 1680.92 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1109562/1807941 [13:30<06:50, 1700.07 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1109737/1807941 [13:30<06:48, 1710.53 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1109913/1807941 [13:30<06:45, 1720.35 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1110170/1807941 [13:30<07:00, 1657.74 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1110339/1807941 [13:30<06:59, 1664.87 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1110511/1807941 [13:31<06:55, 1677.32 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1110683/1807941 [13:31<06:53, 1686.47 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1110861/1807941 [13:31<06:47, 1708.61 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1111101/1807941 [13:31<06:58, 1663.67 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1111280/1807941 [13:31<06:51, 1694.27 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1111463/1807941 [13:31<06:43, 1727.97 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1111651/1807941 [13:31<06:33, 1767.42 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1111840/1807941 [13:31<06:26, 1799.32 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1112097/1807941 [13:31<06:34, 1765.27 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1112286/1807941 [13:32<06:27, 1795.32 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1112471/1807941 [13:32<06:24, 1809.29 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1112660/1807941 [13:32<06:19, 1830.66 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1112849/1807941 [13:32<06:17, 1842.89 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1113105/1807941 [13:32<06:28, 1787.38 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1113294/1807941 [13:32<06:23, 1812.87 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1113481/1807941 [13:32<06:20, 1824.29 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1113672/1807941 [13:32<06:16, 1844.67 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1113860/1807941 [13:32<06:14, 1851.72 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1114099/1807941 [13:33<06:36, 1750.49 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1114356/1807941 [13:33<06:39, 1734.32 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1114532/1807941 [13:33<06:38, 1738.97 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1114795/1807941 [13:33<06:38, 1741.46 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1114974/1807941 [13:33<06:36, 1748.80 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1115209/1807941 [13:33<06:51, 1682.21 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1115386/1807941 [13:33<06:47, 1701.14 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1115567/1807941 [13:33<06:40, 1727.94 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1115746/1807941 [13:34<06:37, 1742.62 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1115923/1807941 [13:34<06:35, 1748.40 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1116171/1807941 [13:34<06:52, 1678.75 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1116346/1807941 [13:34<06:47, 1695.21 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1116526/1807941 [13:34<06:41, 1722.25 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1116704/1807941 [13:34<06:38, 1736.01 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1116891/1807941 [13:34<06:30, 1771.30 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1117153/1807941 [13:34<06:33, 1756.83 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1117349/1807941 [13:34<06:22, 1806.23 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1117542/1807941 [13:35<06:15, 1838.58 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1117730/1807941 [13:35<06:13, 1846.37 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1117919/1807941 [13:35<06:12, 1854.67 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1118186/1807941 [13:35<06:23, 1799.48 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1118373/1807941 [13:35<06:19, 1816.22 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1118563/1807941 [13:35<06:15, 1836.83 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1118756/1807941 [13:35<06:10, 1858.77 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1118948/1807941 [13:35<06:07, 1874.89 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1119206/1807941 [13:35<06:20, 1812.41 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1119397/1807941 [13:36<06:14, 1836.95 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1119591/1807941 [13:36<06:09, 1862.90 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1119779/1807941 [13:36<06:09, 1864.15 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1119970/1807941 [13:36<06:07, 1874.56 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1120229/1807941 [13:36<06:19, 1812.24 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1120419/1807941 [13:36<06:15, 1832.46 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1120608/1807941 [13:36<06:11, 1847.69 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1120797/1807941 [13:36<06:09, 1857.82 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1120991/1807941 [13:36<06:05, 1877.92 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1121251/1807941 [13:37<06:17, 1820.25 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1121443/1807941 [13:37<06:11, 1845.97 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1121636/1807941 [13:37<06:07, 1867.29 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1121906/1807941 [13:37<06:12, 1841.60 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1122149/1807941 [13:37<06:29, 1760.55 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1122414/1807941 [13:37<06:29, 1760.58 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1122592/1807941 [13:37<06:29, 1761.40 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1122858/1807941 [13:37<06:28, 1762.39 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1123100/1807941 [13:38<06:40, 1711.95 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1123276/1807941 [13:38<06:38, 1720.23 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1123453/1807941 [13:38<06:35, 1729.41 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1123632/1807941 [13:38<06:32, 1742.87 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1123809/1807941 [13:38<06:31, 1746.78 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1123986/1807941 [13:38<06:30, 1751.77 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1124224/1807941 [13:38<06:45, 1687.01 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1124406/1807941 [13:38<06:37, 1720.07 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1124592/1807941 [13:38<06:29, 1755.50 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1124778/1807941 [13:39<06:23, 1782.15 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1124968/1807941 [13:39<06:16, 1811.74 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1125227/1807941 [13:39<06:24, 1774.85 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1125414/1807941 [13:39<06:19, 1798.06 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1125683/1807941 [13:39<06:20, 1792.05 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1125864/1807941 [13:39<06:20, 1793.71 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1126121/1807941 [13:39<06:26, 1762.25 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1126301/1807941 [13:39<06:25, 1769.60 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1126486/1807941 [13:40<06:21, 1787.60 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1126677/1807941 [13:40<06:14, 1817.87 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1126865/1807941 [13:40<06:11, 1833.74 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1127124/1807941 [13:40<06:20, 1791.57 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1127313/1807941 [13:40<06:15, 1812.42 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1127589/1807941 [13:40<06:13, 1819.74 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1127866/1807941 [13:40<06:12, 1825.87 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1128116/1807941 [13:40<06:23, 1771.73 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1128300/1807941 [13:41<06:20, 1786.82 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1128483/1807941 [13:41<06:18, 1795.50 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1128747/1807941 [13:41<06:21, 1778.61 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1129000/1807941 [13:41<06:34, 1719.09 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1129255/1807941 [13:41<06:36, 1710.60 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1129513/1807941 [13:41<06:36, 1710.55 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1129686/1807941 [13:41<06:36, 1711.44 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1129861/1807941 [13:41<06:34, 1718.78 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1130094/1807941 [13:42<06:48, 1658.06 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1130344/1807941 [13:42<06:48, 1657.24 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1130513/1807941 [13:42<06:47, 1663.98 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1130683/1807941 [13:42<06:45, 1671.48 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1130857/1807941 [13:42<06:41, 1687.53 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1131091/1807941 [13:42<06:53, 1637.56 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1131258/1807941 [13:42<06:51, 1643.59 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1131424/1807941 [13:42<06:51, 1644.85 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1131593/1807941 [13:43<06:48, 1656.16 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1131761/1807941 [13:43<06:47, 1660.50 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1131929/1807941 [13:43<06:45, 1665.39 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1132162/1807941 [13:43<07:03, 1595.77 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1132327/1807941 [13:43<07:00, 1608.55 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1132493/1807941 [13:43<06:56, 1620.37 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1132661/1807941 [13:43<06:53, 1633.58 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1132826/1807941 [13:43<06:52, 1636.14 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1133000/1807941 [13:43<06:57, 1615.67 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1133203/1807941 [13:44<07:25, 1515.70 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1133390/1807941 [13:44<07:56, 1415.19 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1133575/1807941 [13:44<08:19, 1349.37 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1133756/1807941 [13:44<08:39, 1298.40 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1133893/1807941 [13:44<08:33, 1311.62 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1134089/1807941 [13:44<08:42, 1290.67 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1134338/1807941 [13:44<07:07, 1574.58 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1134577/1807941 [13:44<06:18, 1779.47 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1134772/1807941 [13:45<06:09, 1823.62 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1135032/1807941 [13:45<06:16, 1788.21 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1135256/1807941 [13:45<05:53, 1901.63 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1135685/1807941 [13:45<04:24, 2539.62 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1136075/1807941 [13:45<03:50, 2914.13 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1136458/1807941 [13:45<03:31, 3171.32 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1136834/1807941 [13:45<03:21, 3338.77 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1137196/1807941 [13:45<03:16, 3420.02 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1137587/1807941 [13:45<03:08, 3562.01 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1137971/1807941 [13:46<03:04, 3636.98 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1138510/1807941 [13:46<03:05, 3616.40 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1138895/1807941 [13:46<03:01, 3677.58 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1139446/1807941 [13:46<03:01, 3673.95 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1139997/1807941 [13:46<03:02, 3668.74 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1140541/1807941 [13:46<03:02, 3652.61 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1140923/1807941 [13:46<03:01, 3685.17 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1141469/1807941 [13:46<03:01, 3666.12 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1141868/1807941 [13:47<02:58, 3741.44 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1142417/1807941 [13:47<02:59, 3712.93 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1142818/1807941 [13:47<02:55, 3784.37 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1143383/1807941 [13:47<02:57, 3735.35 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1143762/1807941 [13:47<02:57, 3747.98 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1144284/1807941 [13:47<03:01, 3652.47 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1144657/1807941 [13:47<03:00, 3669.40 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1145216/1807941 [13:47<02:59, 3684.57 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1145638/1807941 [13:48<02:53, 3815.85 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1146027/1807941 [13:48<02:52, 3830.27 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1146451/1807941 [13:48<02:47, 3939.66 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1146871/1807941 [13:48<02:44, 4011.32 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1147444/1807941 [13:48<02:47, 3936.74 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1148000/1807941 [13:48<02:52, 3825.28 examples/s]
Tokenizing train dataset:  64%|██████▎   | 1148553/1807941 [13:48<02:54, 3776.97 examples/s]
Tokenizing train dataset:  64%|██████▎   | 1149083/1807941 [13:48<02:58, 3696.44 examples/s]
Tokenizing train dataset:  64%|██████▎   | 1149479/1807941 [13:49<02:55, 3756.79 examples/s]
Tokenizing train dataset:  64%|██████▎   | 1149887/1807941 [13:49<02:51, 3834.35 examples/s]
Tokenizing train dataset:  64%|██████▎   | 1150282/1807941 [13:49<02:50, 3859.52 examples/s]
Tokenizing train dataset:  64%|██████▎   | 1150738/1807941 [13:49<02:42, 4047.38 examples/s]
Tokenizing train dataset:  64%|██████▎   | 1151179/1807941 [13:49<02:38, 4146.81 examples/s]
Tokenizing train dataset:  64%|██████▎   | 1151623/1807941 [13:49<02:35, 4227.43 examples/s]
Tokenizing train dataset:  64%|██████▎   | 1152066/1807941 [13:49<02:33, 4283.65 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1152687/1807941 [13:49<02:35, 4224.13 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1153266/1807941 [13:49<02:40, 4091.25 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1153863/1807941 [13:50<02:41, 4051.63 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1154444/1807941 [13:50<02:43, 3991.97 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1155000/1807941 [13:50<02:47, 3890.28 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1155557/1807941 [13:50<02:50, 3831.66 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1156074/1807941 [13:50<02:55, 3711.99 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1156451/1807941 [13:50<02:55, 3722.24 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1157000/1807941 [13:50<02:56, 3683.72 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1157374/1807941 [13:51<02:56, 3694.95 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1157756/1807941 [13:51<02:54, 3722.60 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1158304/1807941 [13:51<02:55, 3694.50 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1158679/1807941 [13:51<02:55, 3705.30 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1159215/1807941 [13:51<02:57, 3655.17 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1159649/1807941 [13:51<03:10, 3397.55 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1160012/1807941 [13:51<03:07, 3450.54 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1160388/1807941 [13:51<03:03, 3525.04 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1160803/1807941 [13:52<02:55, 3688.68 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1161365/1807941 [13:52<02:54, 3705.36 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1161760/1807941 [13:52<02:51, 3766.51 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1162323/1807941 [13:52<02:51, 3754.41 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1162711/1807941 [13:52<02:50, 3782.63 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1163281/1807941 [13:52<02:50, 3785.22 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1163679/1807941 [13:52<02:48, 3828.85 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1164244/1807941 [13:52<02:49, 3804.47 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1164628/1807941 [13:53<02:48, 3810.88 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1165011/1807941 [13:53<02:48, 3814.20 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1165399/1807941 [13:53<02:47, 3826.63 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1165830/1807941 [13:53<03:50, 2780.70 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1166295/1807941 [13:53<04:05, 2612.44 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1166607/1807941 [13:53<04:20, 2460.96 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1166937/1807941 [13:53<04:29, 2381.16 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1167284/1807941 [13:54<04:31, 2359.79 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1167531/1807941 [13:54<04:28, 2381.32 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1167849/1807941 [13:54<04:38, 2294.44 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1168166/1807941 [13:54<04:46, 2234.60 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1168464/1807941 [13:54<04:56, 2153.62 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1168690/1807941 [13:54<05:27, 1952.27 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1168923/1807941 [13:54<05:49, 1828.37 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1169150/1807941 [13:55<06:19, 1684.44 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1169379/1807941 [13:55<06:30, 1633.57 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1169614/1807941 [13:55<06:36, 1610.17 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1169841/1807941 [13:55<06:44, 1578.82 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1170040/1807941 [13:55<07:04, 1501.87 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1170260/1807941 [13:55<07:08, 1488.00 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1170485/1807941 [13:56<07:07, 1490.48 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1170639/1807941 [13:56<07:04, 1499.87 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1170799/1807941 [13:56<06:58, 1521.79 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1170978/1807941 [13:56<06:41, 1587.30 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1171216/1807941 [13:56<06:42, 1583.16 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1171379/1807941 [13:56<06:39, 1591.80 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1171544/1807941 [13:56<06:36, 1604.61 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1171713/1807941 [13:56<06:31, 1626.63 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1171895/1807941 [13:56<06:18, 1678.66 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1172132/1807941 [13:57<06:27, 1639.70 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1172308/1807941 [13:57<06:21, 1668.15 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1172486/1807941 [13:57<06:14, 1695.89 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1172672/1807941 [13:57<06:05, 1740.32 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1172858/1807941 [13:57<05:58, 1770.05 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1173109/1807941 [13:57<06:07, 1728.87 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1173293/1807941 [13:57<06:01, 1756.23 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1173476/1807941 [13:57<05:57, 1774.30 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1173734/1807941 [13:57<06:02, 1749.88 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1173993/1807941 [13:58<06:05, 1736.51 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1174223/1807941 [13:58<06:20, 1665.07 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1174474/1807941 [13:58<06:20, 1665.02 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1174642/1807941 [13:58<06:20, 1666.25 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1174810/1807941 [13:58<06:19, 1667.09 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1174979/1807941 [13:58<06:18, 1671.56 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1175201/1807941 [13:58<06:35, 1598.47 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1175366/1807941 [13:58<06:33, 1608.49 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1175608/1807941 [13:59<06:34, 1604.35 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1175851/1807941 [13:59<06:33, 1606.15 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1176085/1807941 [13:59<06:38, 1585.29 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1176263/1807941 [13:59<06:27, 1628.96 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1176440/1807941 [13:59<06:20, 1660.59 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1176622/1807941 [13:59<06:11, 1700.07 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1176796/1807941 [13:59<06:09, 1708.99 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1176979/1807941 [13:59<06:02, 1740.06 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1177202/1807941 [14:00<06:24, 1640.80 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1177449/1807941 [14:00<06:24, 1640.83 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1177618/1807941 [14:00<06:21, 1650.22 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1177790/1807941 [14:00<06:18, 1665.14 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1177959/1807941 [14:00<06:17, 1668.27 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1178191/1807941 [14:00<06:28, 1620.23 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1178363/1807941 [14:00<06:23, 1642.84 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1178537/1807941 [14:00<06:17, 1667.87 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1178710/1807941 [14:00<06:14, 1681.86 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1178884/1807941 [14:01<06:10, 1696.27 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1179116/1807941 [14:01<06:24, 1636.13 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1179287/1807941 [14:01<06:20, 1652.47 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1179460/1807941 [14:01<06:16, 1670.65 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1179631/1807941 [14:01<06:14, 1679.08 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1179800/1807941 [14:01<06:14, 1678.42 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1179971/1807941 [14:01<06:12, 1683.69 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1180202/1807941 [14:01<06:26, 1623.85 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1180374/1807941 [14:02<06:21, 1646.01 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1180548/1807941 [14:02<06:16, 1668.50 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1180801/1807941 [14:02<06:15, 1671.71 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1180973/1807941 [14:02<06:12, 1681.41 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1181199/1807941 [14:02<06:27, 1616.41 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1181443/1807941 [14:02<06:27, 1616.16 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1181609/1807941 [14:02<06:25, 1624.28 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1181855/1807941 [14:02<06:24, 1627.22 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1182081/1807941 [14:03<06:36, 1580.31 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1182244/1807941 [14:03<06:33, 1588.37 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1182407/1807941 [14:03<06:31, 1596.72 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1182574/1807941 [14:03<06:27, 1615.27 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1182744/1807941 [14:03<06:22, 1633.80 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1182918/1807941 [14:03<06:16, 1659.26 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1183167/1807941 [14:03<06:29, 1602.26 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1183338/1807941 [14:03<06:23, 1626.59 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1183509/1807941 [14:03<06:19, 1645.82 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1183679/1807941 [14:04<06:16, 1657.72 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1183849/1807941 [14:04<06:14, 1668.61 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1184085/1807941 [14:04<06:25, 1620.36 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1184262/1807941 [14:04<06:16, 1656.81 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1184437/1807941 [14:04<06:11, 1678.89 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1184686/1807941 [14:04<06:13, 1669.56 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1184938/1807941 [14:04<06:13, 1669.08 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1185164/1807941 [14:04<06:27, 1608.47 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1185332/1807941 [14:05<06:23, 1623.24 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1185501/1807941 [14:05<06:20, 1636.96 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1185672/1807941 [14:05<06:16, 1654.48 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1185844/1807941 [14:05<06:12, 1669.55 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1186082/1807941 [14:05<06:24, 1616.50 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1186251/1807941 [14:05<06:20, 1633.49 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1186419/1807941 [14:05<06:18, 1641.80 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1186594/1807941 [14:05<06:11, 1671.01 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1186782/1807941 [14:05<05:59, 1727.43 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1186972/1807941 [14:05<05:49, 1774.40 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1187223/1807941 [14:06<05:58, 1733.26 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1187410/1807941 [14:06<05:50, 1768.58 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1187598/1807941 [14:06<05:45, 1797.95 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1187782/1807941 [14:06<05:42, 1808.67 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1188027/1807941 [14:06<05:56, 1737.61 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1188282/1807941 [14:06<05:59, 1721.41 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1188534/1807941 [14:06<06:03, 1704.67 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1188786/1807941 [14:07<06:05, 1694.94 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1188957/1807941 [14:07<06:05, 1694.17 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1189179/1807941 [14:07<06:22, 1618.79 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1189422/1807941 [14:07<06:23, 1614.39 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1189590/1807941 [14:07<06:19, 1628.25 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1189759/1807941 [14:07<06:16, 1640.11 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1189931/1807941 [14:07<06:12, 1659.25 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1190169/1807941 [14:07<06:22, 1615.04 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1190341/1807941 [14:08<06:17, 1637.41 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1190515/1807941 [14:08<06:11, 1661.57 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1190688/1807941 [14:08<06:07, 1680.16 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1190863/1807941 [14:08<06:03, 1697.10 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1191097/1807941 [14:08<06:15, 1643.70 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1191269/1807941 [14:08<06:11, 1661.50 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1191442/1807941 [14:08<06:07, 1675.40 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1191615/1807941 [14:08<06:04, 1689.19 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1191786/1807941 [14:08<06:03, 1693.65 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1191961/1807941 [14:08<06:00, 1707.28 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1192193/1807941 [14:09<06:14, 1642.08 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1192365/1807941 [14:09<06:11, 1657.37 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1192539/1807941 [14:09<06:06, 1679.32 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1192713/1807941 [14:09<06:03, 1692.82 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1192888/1807941 [14:09<06:00, 1706.22 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1193121/1807941 [14:09<06:13, 1646.97 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1193294/1807941 [14:09<06:09, 1665.45 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1193466/1807941 [14:09<06:05, 1679.23 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1193641/1807941 [14:09<06:01, 1698.06 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1193812/1807941 [14:10<06:01, 1696.53 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1193984/1807941 [14:10<06:00, 1701.22 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1194214/1807941 [14:10<06:15, 1634.00 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1194383/1807941 [14:10<06:12, 1647.36 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1194558/1807941 [14:10<06:06, 1672.45 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1194728/1807941 [14:10<06:05, 1676.19 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1194898/1807941 [14:10<06:04, 1682.17 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1195124/1807941 [14:10<06:19, 1614.00 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1195298/1807941 [14:10<06:12, 1642.60 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1195478/1807941 [14:11<06:03, 1682.81 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1195661/1807941 [14:11<05:55, 1719.92 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1195842/1807941 [14:11<05:50, 1745.09 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1196091/1807941 [14:11<05:58, 1707.14 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1196274/1807941 [14:11<05:51, 1738.08 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1196458/1807941 [14:11<05:47, 1761.87 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1196647/1807941 [14:11<05:40, 1793.39 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1196836/1807941 [14:11<05:36, 1817.20 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1197096/1807941 [14:11<05:42, 1780.91 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1197286/1807941 [14:12<05:37, 1811.43 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1197472/1807941 [14:12<05:34, 1822.64 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1197660/1807941 [14:12<05:32, 1836.29 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1197924/1807941 [14:12<05:38, 1804.47 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1198170/1807941 [14:12<05:55, 1714.85 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1198427/1807941 [14:12<05:56, 1711.04 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1198603/1807941 [14:12<05:54, 1719.60 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1198777/1807941 [14:12<05:53, 1723.15 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1198956/1807941 [14:13<05:50, 1736.87 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1199194/1807941 [14:13<06:02, 1679.91 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1199371/1807941 [14:13<05:58, 1699.53 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1199548/1807941 [14:13<05:54, 1717.10 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1199722/1807941 [14:13<05:53, 1721.74 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1199901/1807941 [14:13<05:50, 1735.19 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1200142/1807941 [14:13<06:01, 1682.03 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1200326/1807941 [14:13<05:53, 1719.54 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1200514/1807941 [14:13<05:45, 1759.55 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1200703/1807941 [14:14<05:38, 1792.63 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1200893/1807941 [14:14<05:33, 1820.83 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1201147/1807941 [14:14<05:43, 1766.21 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1201333/1807941 [14:14<05:39, 1788.13 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1201593/1807941 [14:14<05:43, 1765.21 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1201851/1807941 [14:14<05:47, 1744.87 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1202083/1807941 [14:14<06:01, 1677.16 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1202337/1807941 [14:15<06:00, 1678.14 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1202508/1807941 [14:15<05:59, 1682.69 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1202681/1807941 [14:15<05:57, 1691.08 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1202858/1807941 [14:15<05:53, 1709.81 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1203095/1807941 [14:15<06:04, 1659.24 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1203267/1807941 [14:15<06:01, 1672.62 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1203436/1807941 [14:15<06:00, 1677.00 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1203609/1807941 [14:15<05:57, 1689.79 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1203780/1807941 [14:15<05:57, 1690.68 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1203958/1807941 [14:15<05:52, 1713.22 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1204202/1807941 [14:16<05:59, 1677.70 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1204387/1807941 [14:16<05:50, 1721.36 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1204576/1807941 [14:16<05:41, 1765.19 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1204764/1807941 [14:16<05:35, 1795.68 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1204957/1807941 [14:16<05:29, 1829.24 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1205213/1807941 [14:16<05:38, 1780.56 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1205400/1807941 [14:16<05:34, 1801.34 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1205588/1807941 [14:16<05:30, 1822.08 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1205776/1807941 [14:16<05:27, 1836.95 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1205968/1807941 [14:17<05:23, 1858.24 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1206230/1807941 [14:17<05:32, 1811.78 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1206422/1807941 [14:17<05:27, 1836.81 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1206614/1807941 [14:17<05:23, 1858.07 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1206808/1807941 [14:17<05:20, 1876.64 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1207000/1807941 [14:17<05:29, 1825.31 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1207263/1807941 [14:17<05:34, 1795.34 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1207527/1807941 [14:17<05:37, 1779.86 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1207792/1807941 [14:18<05:38, 1772.33 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1208033/1807941 [14:18<05:49, 1714.56 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1208208/1807941 [14:18<05:48, 1719.76 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1208384/1807941 [14:18<05:47, 1726.61 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1208561/1807941 [14:18<05:45, 1733.68 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1208822/1807941 [14:18<05:45, 1732.57 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1208998/1807941 [14:18<05:44, 1738.23 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1209233/1807941 [14:18<05:57, 1672.66 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1209408/1807941 [14:19<05:54, 1689.43 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1209583/1807941 [14:19<05:51, 1703.58 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1209758/1807941 [14:19<05:49, 1712.24 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1209934/1807941 [14:19<05:46, 1724.24 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1210171/1807941 [14:19<06:00, 1658.09 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1210344/1807941 [14:19<05:56, 1674.17 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1210517/1807941 [14:19<05:54, 1687.08 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1210691/1807941 [14:19<05:51, 1699.51 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1210867/1807941 [14:19<05:48, 1713.35 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1211103/1807941 [14:20<06:00, 1657.37 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1211274/1807941 [14:20<05:57, 1670.63 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1211447/1807941 [14:20<05:54, 1684.75 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1211621/1807941 [14:20<05:51, 1697.99 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1211792/1807941 [14:20<05:51, 1697.31 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1211966/1807941 [14:20<05:49, 1705.92 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1212192/1807941 [14:20<06:06, 1625.50 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1212363/1807941 [14:20<06:02, 1643.23 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1212537/1807941 [14:20<05:57, 1667.09 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1212709/1807941 [14:21<05:54, 1679.95 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1212884/1807941 [14:21<05:50, 1696.09 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1213118/1807941 [14:21<06:02, 1641.02 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1213291/1807941 [14:21<05:57, 1661.34 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1213465/1807941 [14:21<05:53, 1680.69 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1213639/1807941 [14:21<05:50, 1694.89 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1213810/1807941 [14:21<05:50, 1697.46 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1213987/1807941 [14:21<05:45, 1718.13 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1214224/1807941 [14:21<05:57, 1661.20 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1214399/1807941 [14:22<05:52, 1682.93 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1214576/1807941 [14:22<05:48, 1704.39 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1214752/1807941 [14:22<05:45, 1714.99 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1214928/1807941 [14:22<05:43, 1725.21 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1215171/1807941 [14:22<05:57, 1660.10 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1215344/1807941 [14:22<05:53, 1676.28 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1215519/1807941 [14:22<05:49, 1694.55 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1215693/1807941 [14:22<05:47, 1704.49 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1215870/1807941 [14:22<05:44, 1720.30 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1216109/1807941 [14:23<05:54, 1667.23 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1216285/1807941 [14:23<05:50, 1688.42 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1216461/1807941 [14:23<05:46, 1705.65 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1216639/1807941 [14:23<05:43, 1723.58 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1216814/1807941 [14:23<05:42, 1727.63 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1216991/1807941 [14:23<05:40, 1736.92 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1217227/1807941 [14:23<05:53, 1669.59 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1217413/1807941 [14:23<05:43, 1717.89 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1217603/1807941 [14:23<05:34, 1764.64 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1217794/1807941 [14:23<05:27, 1801.07 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1217986/1807941 [14:24<05:21, 1832.48 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1218241/1807941 [14:24<05:31, 1778.52 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1218509/1807941 [14:24<05:31, 1779.64 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1218776/1807941 [14:24<05:31, 1775.87 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1218955/1807941 [14:24<05:31, 1776.71 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1219193/1807941 [14:24<05:44, 1708.49 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1219368/1807941 [14:24<05:42, 1716.30 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1219545/1807941 [14:24<05:40, 1728.12 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1219724/1807941 [14:25<05:37, 1741.97 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1219914/1807941 [14:25<05:29, 1783.79 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1220185/1807941 [14:25<05:35, 1754.19 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1220371/1807941 [14:25<05:30, 1778.35 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1220560/1807941 [14:25<05:25, 1804.34 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1220744/1807941 [14:25<05:23, 1812.94 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1220934/1807941 [14:25<05:20, 1833.69 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1221191/1807941 [14:25<05:29, 1782.12 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1221379/1807941 [14:26<05:24, 1804.92 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1221567/1807941 [14:26<05:21, 1824.79 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1221757/1807941 [14:26<05:17, 1844.73 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1221946/1807941 [14:26<05:15, 1854.55 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1222205/1807941 [14:26<05:25, 1801.96 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1222474/1807941 [14:26<05:25, 1795.92 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1222743/1807941 [14:26<05:26, 1790.14 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1223000/1807941 [14:26<05:37, 1735.47 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1223263/1807941 [14:27<05:36, 1735.94 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1223439/1807941 [14:27<05:35, 1740.41 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1223624/1807941 [14:27<05:30, 1765.60 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1223811/1807941 [14:27<05:26, 1790.33 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1224047/1807941 [14:27<05:41, 1708.02 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1224283/1807941 [14:27<05:51, 1658.38 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1224550/1807941 [14:27<05:44, 1693.23 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1224784/1807941 [14:27<05:16, 1843.50 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1225006/1807941 [14:28<05:39, 1718.21 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1225202/1807941 [14:28<05:28, 1772.17 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1225394/1807941 [14:28<05:22, 1807.08 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1225604/1807941 [14:28<05:09, 1884.05 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1225876/1807941 [14:28<05:13, 1855.49 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1226120/1807941 [14:28<05:27, 1775.38 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1226372/1807941 [14:28<05:34, 1738.43 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1226617/1807941 [14:29<06:26, 1505.89 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1226806/1807941 [14:29<06:45, 1433.07 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1227000/1807941 [14:29<07:25, 1303.65 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1227166/1807941 [14:29<07:59, 1211.45 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1227299/1807941 [14:29<07:50, 1235.31 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1227464/1807941 [14:29<08:06, 1192.83 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1227641/1807941 [14:29<08:09, 1185.61 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1227810/1807941 [14:30<08:21, 1156.48 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1227941/1807941 [14:30<08:08, 1187.95 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1228069/1807941 [14:30<08:26, 1145.52 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1228245/1807941 [14:30<12:54, 748.72 examples/s] 
Tokenizing train dataset:  68%|██████▊   | 1228342/1807941 [14:30<13:21, 722.70 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1228475/1807941 [14:31<13:29, 716.24 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1228590/1807941 [14:31<12:09, 793.84 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1228686/1807941 [14:31<11:43, 823.40 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1228809/1807941 [14:31<11:55, 809.50 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1228902/1807941 [14:31<14:49, 650.76 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1228985/1807941 [14:31<15:40, 615.57 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229068/1807941 [14:32<27:34, 349.93 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229127/1807941 [14:32<31:18, 308.15 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229187/1807941 [14:33<59:44, 161.47 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229246/1807941 [14:34<1:18:48, 122.39 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229276/1807941 [14:34<1:15:28, 127.78 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229303/1807941 [14:34<1:09:23, 138.96 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229346/1807941 [14:34<58:50, 163.90 examples/s]  
Tokenizing train dataset:  68%|██████▊   | 1229376/1807941 [14:35<1:00:14, 160.07 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229406/1807941 [14:35<56:13, 171.52 examples/s]  
Tokenizing train dataset:  68%|██████▊   | 1229434/1807941 [14:35<52:07, 184.99 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229467/1807941 [14:35<52:15, 184.50 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229502/1807941 [14:35<1:06:58, 143.93 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229538/1807941 [14:35<57:04, 168.91 examples/s]  
Tokenizing train dataset:  68%|██████▊   | 1229566/1807941 [14:36<51:27, 187.34 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229590/1807941 [14:36<1:19:36, 121.07 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229618/1807941 [14:36<1:12:18, 133.30 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229636/1807941 [14:36<1:25:13, 113.10 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229691/1807941 [14:37<56:04, 171.86 examples/s]  
Tokenizing train dataset:  68%|██████▊   | 1229726/1807941 [14:37<53:59, 178.47 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229765/1807941 [14:37<56:55, 169.26 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229802/1807941 [14:37<55:15, 174.39 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229831/1807941 [14:37<1:04:45, 148.79 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229877/1807941 [14:38<1:19:19, 121.46 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229915/1807941 [14:38<1:27:58, 109.52 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229936/1807941 [14:39<1:22:06, 117.33 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229966/1807941 [14:39<1:11:29, 134.75 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229997/1807941 [14:39<1:02:37, 153.81 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230028/1807941 [14:39<1:34:07, 102.32 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230045/1807941 [14:39<1:35:04, 101.31 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230098/1807941 [14:40<1:00:26, 159.35 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230144/1807941 [14:40<49:56, 192.84 examples/s]  
Tokenizing train dataset:  68%|██████▊   | 1230177/1807941 [14:40<1:05:58, 145.97 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230203/1807941 [14:41<1:25:12, 113.01 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230220/1807941 [14:41<1:54:33, 84.05 examples/s] 
Tokenizing train dataset:  68%|██████▊   | 1230269/1807941 [14:41<1:16:16, 126.23 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230303/1807941 [14:41<1:02:24, 154.26 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230356/1807941 [14:41<50:44, 189.71 examples/s]  
Tokenizing train dataset:  68%|██████▊   | 1230387/1807941 [14:42<51:16, 187.73 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230427/1807941 [14:42<55:17, 174.08 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230449/1807941 [14:42<1:06:17, 145.18 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230481/1807941 [14:42<1:00:48, 158.27 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230519/1807941 [14:42<51:40, 186.26 examples/s]  
Tokenizing train dataset:  68%|██████▊   | 1230556/1807941 [14:43<56:06, 171.48 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230596/1807941 [14:43<1:05:57, 145.88 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230657/1807941 [14:43<45:36, 210.95 examples/s]  
Tokenizing train dataset:  68%|██████▊   | 1230704/1807941 [14:43<40:35, 237.01 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230759/1807941 [14:43<33:25, 287.85 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230839/1807941 [14:43<25:42, 374.06 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230930/1807941 [14:44<21:44, 442.19 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230980/1807941 [14:44<26:29, 363.05 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231035/1807941 [14:44<38:20, 250.76 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231069/1807941 [14:44<38:45, 248.04 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231114/1807941 [14:45<43:57, 218.70 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231187/1807941 [14:45<33:56, 283.15 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231228/1807941 [14:45<40:02, 240.00 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231280/1807941 [14:45<48:01, 200.09 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231346/1807941 [14:45<36:36, 262.48 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231425/1807941 [14:46<27:43, 346.50 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231483/1807941 [14:46<25:21, 378.79 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231542/1807941 [14:46<23:34, 407.60 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231608/1807941 [14:46<21:18, 450.80 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231701/1807941 [14:46<22:03, 435.38 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231799/1807941 [14:46<21:50, 439.80 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231869/1807941 [14:47<37:15, 257.67 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231920/1807941 [14:47<33:34, 285.97 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232000/1807941 [14:47<29:28, 325.61 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232062/1807941 [14:47<25:43, 373.14 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232119/1807941 [14:47<23:43, 404.44 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232192/1807941 [14:48<29:08, 329.19 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232273/1807941 [14:48<40:38, 236.07 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232314/1807941 [14:48<37:16, 257.32 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232375/1807941 [14:49<34:37, 276.99 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232425/1807941 [14:49<37:27, 256.12 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232478/1807941 [14:49<32:11, 297.98 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232536/1807941 [14:49<27:33, 348.02 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232597/1807941 [14:49<42:05, 227.83 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232658/1807941 [14:50<39:12, 244.58 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232704/1807941 [14:50<34:54, 274.66 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232778/1807941 [14:50<27:02, 354.58 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232848/1807941 [14:50<22:52, 418.91 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232913/1807941 [14:50<24:31, 390.83 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233000/1807941 [14:50<20:40, 463.61 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233069/1807941 [14:50<20:11, 474.63 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233137/1807941 [14:51<20:14, 473.25 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233211/1807941 [14:51<21:14, 451.01 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233275/1807941 [14:51<24:33, 389.94 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233326/1807941 [14:51<23:28, 408.00 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233395/1807941 [14:51<24:11, 395.92 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233454/1807941 [14:52<27:23, 349.49 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233496/1807941 [14:52<27:38, 346.27 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233580/1807941 [14:52<23:25, 408.61 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233670/1807941 [14:52<20:15, 472.60 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233720/1807941 [14:52<21:13, 451.03 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233799/1807941 [14:52<18:31, 516.61 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233893/1807941 [14:52<17:18, 552.70 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233963/1807941 [14:53<20:28, 467.33 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234050/1807941 [14:53<20:48, 459.79 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234149/1807941 [14:53<19:40, 486.21 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234202/1807941 [14:53<22:34, 423.50 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234247/1807941 [14:53<26:22, 362.63 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234315/1807941 [14:53<23:01, 415.17 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234382/1807941 [14:54<23:46, 402.07 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234432/1807941 [14:54<22:55, 417.03 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234524/1807941 [14:54<20:05, 475.70 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234589/1807941 [14:54<21:24, 446.39 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234640/1807941 [14:54<20:49, 458.86 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234717/1807941 [14:54<22:21, 427.22 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234783/1807941 [14:54<20:13, 472.35 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234871/1807941 [14:55<22:50, 418.11 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234935/1807941 [14:55<27:23, 348.71 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1235000/1807941 [14:55<32:18, 295.49 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1235066/1807941 [14:55<32:12, 296.39 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1235134/1807941 [14:56<27:15, 350.19 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1235187/1807941 [14:56<31:57, 298.64 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1235267/1807941 [14:56<25:32, 373.58 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1235325/1807941 [14:56<30:08, 316.64 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1235494/1807941 [14:56<17:15, 552.64 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1235833/1807941 [14:56<08:34, 1111.36 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1236164/1807941 [14:57<06:21, 1497.33 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1236484/1807941 [14:57<05:03, 1884.27 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1236811/1807941 [14:57<04:17, 2221.89 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1237112/1807941 [14:57<03:55, 2424.79 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1237439/1807941 [14:57<03:35, 2651.51 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1237766/1807941 [14:57<03:22, 2820.94 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1238202/1807941 [14:57<03:20, 2847.80 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1238509/1807941 [14:57<03:16, 2904.94 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1238948/1807941 [14:57<03:15, 2910.05 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1239303/1807941 [14:58<03:28, 2722.86 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1239721/1807941 [14:58<03:27, 2740.15 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1240029/1807941 [14:58<03:21, 2818.36 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1240337/1807941 [14:58<03:17, 2868.35 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1240637/1807941 [14:58<05:54, 1602.41 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1241000/1807941 [14:58<04:56, 1909.74 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1241291/1807941 [14:59<06:47, 1391.76 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1241617/1807941 [14:59<06:38, 1422.85 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1241844/1807941 [14:59<06:04, 1554.73 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1242119/1807941 [14:59<05:48, 1624.65 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1242339/1807941 [14:59<05:25, 1735.32 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1242541/1807941 [15:00<05:14, 1795.62 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1242754/1807941 [15:00<05:01, 1873.31 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1242974/1807941 [15:00<04:50, 1946.29 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1243183/1807941 [15:00<05:22, 1750.38 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1243409/1807941 [15:00<05:39, 1661.19 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1243637/1807941 [15:00<05:50, 1610.89 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1243859/1807941 [15:00<05:22, 1751.72 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1244143/1807941 [15:00<05:14, 1795.02 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1244334/1807941 [15:01<05:09, 1820.78 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1244541/1807941 [15:01<05:38, 1662.24 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1244777/1807941 [15:01<05:45, 1629.74 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1244962/1807941 [15:01<05:35, 1680.39 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1245160/1807941 [15:01<06:01, 1557.43 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1245353/1807941 [15:01<06:24, 1464.03 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1245564/1807941 [15:01<06:29, 1443.04 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1245741/1807941 [15:01<06:10, 1517.52 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1245911/1807941 [15:02<06:00, 1560.55 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1246125/1807941 [15:02<06:12, 1507.24 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1246301/1807941 [15:02<05:58, 1566.45 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1246496/1807941 [15:02<05:38, 1659.92 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1246682/1807941 [15:02<05:27, 1711.83 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1246903/1807941 [15:02<05:49, 1606.21 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1247139/1807941 [15:02<05:52, 1592.06 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1247359/1807941 [15:02<05:23, 1735.11 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1247630/1807941 [15:03<05:18, 1757.34 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1247877/1807941 [15:03<05:26, 1717.49 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1248065/1807941 [15:03<05:19, 1754.06 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1248259/1807941 [15:03<05:11, 1799.30 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1248481/1807941 [15:03<04:54, 1901.44 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1248683/1807941 [15:03<04:49, 1931.78 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1248922/1807941 [15:03<04:31, 2057.77 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1249139/1807941 [15:03<04:28, 2084.97 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1249403/1807941 [15:03<04:08, 2243.21 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1249656/1807941 [15:04<04:00, 2322.57 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1249917/1807941 [15:04<03:52, 2404.41 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1250264/1807941 [15:04<03:55, 2365.62 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1250522/1807941 [15:04<03:50, 2420.73 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1250768/1807941 [15:04<03:49, 2429.51 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1251100/1807941 [15:04<04:04, 2276.46 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1251440/1807941 [15:04<04:05, 2270.64 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1251671/1807941 [15:04<04:04, 2275.50 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1252000/1807941 [15:05<04:11, 2214.10 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1252341/1807941 [15:05<04:09, 2228.90 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1252568/1807941 [15:05<04:08, 2236.98 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1252800/1807941 [15:05<04:06, 2256.40 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1253123/1807941 [15:05<04:10, 2217.45 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1253349/1807941 [15:05<04:09, 2225.57 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1253585/1807941 [15:05<04:05, 2257.02 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1253884/1807941 [15:05<04:16, 2157.61 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1254103/1807941 [15:06<04:48, 1918.34 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1254348/1807941 [15:06<05:05, 1811.74 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1254578/1807941 [15:06<05:27, 1692.15 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1254806/1807941 [15:06<05:39, 1629.01 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1255000/1807941 [15:06<06:10, 1493.92 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1255212/1807941 [15:06<06:17, 1463.40 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1255440/1807941 [15:06<06:16, 1466.38 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1255595/1807941 [15:07<06:12, 1482.61 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1255773/1807941 [15:07<05:57, 1544.88 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1255951/1807941 [15:07<05:44, 1602.01 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1256156/1807941 [15:07<06:03, 1517.13 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1256408/1807941 [15:07<05:51, 1567.33 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1256583/1807941 [15:07<05:42, 1608.47 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1256791/1807941 [15:07<06:00, 1530.05 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1257000/1807941 [15:08<06:16, 1462.42 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1257215/1807941 [15:08<06:20, 1448.24 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1257411/1807941 [15:08<06:32, 1400.92 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1257563/1807941 [15:08<07:14, 1266.89 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1257761/1807941 [15:08<07:14, 1267.67 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1257951/1807941 [15:08<07:16, 1261.15 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1258120/1807941 [15:08<07:36, 1204.04 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1258247/1807941 [15:09<07:34, 1210.19 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1258450/1807941 [15:09<07:22, 1241.63 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1258586/1807941 [15:09<07:13, 1268.54 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1258719/1807941 [15:09<07:08, 1280.44 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1258852/1807941 [15:09<07:12, 1270.20 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1258990/1807941 [15:09<07:03, 1297.46 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1259205/1807941 [15:09<06:48, 1342.04 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1259353/1807941 [15:09<06:39, 1372.91 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1259551/1807941 [15:10<06:46, 1348.70 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1259727/1807941 [15:10<07:08, 1278.88 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1259909/1807941 [15:10<07:18, 1249.26 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1260052/1807941 [15:10<07:58, 1145.59 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1260182/1807941 [15:10<08:43, 1046.44 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1260294/1807941 [15:10<08:36, 1060.37 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1260412/1807941 [15:10<08:23, 1087.88 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1260558/1807941 [15:10<07:43, 1181.96 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1260738/1807941 [15:11<07:42, 1183.45 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1260878/1807941 [15:11<07:22, 1236.98 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1261035/1807941 [15:11<07:58, 1142.09 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1261180/1807941 [15:11<07:30, 1213.14 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1261327/1807941 [15:11<07:15, 1254.63 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1261506/1807941 [15:11<07:29, 1214.73 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1261661/1807941 [15:11<08:01, 1135.58 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1261792/1807941 [15:11<07:44, 1176.24 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1261915/1807941 [15:12<07:39, 1188.24 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1262049/1807941 [15:12<08:32, 1064.39 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1262211/1807941 [15:12<08:31, 1066.44 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1262370/1807941 [15:12<08:36, 1056.95 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1262481/1807941 [15:12<08:30, 1067.50 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1262593/1807941 [15:12<08:25, 1078.95 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1262703/1807941 [15:12<08:23, 1082.53 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1262832/1807941 [15:12<09:07, 996.39 examples/s] 
Tokenizing train dataset:  70%|██████▉   | 1262948/1807941 [15:13<08:46, 1035.11 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1263087/1807941 [15:13<09:37, 943.95 examples/s] 
Tokenizing train dataset:  70%|██████▉   | 1263241/1807941 [15:13<10:38, 852.76 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1263363/1807941 [15:13<10:52, 834.58 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1263485/1807941 [15:13<09:54, 916.00 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1263616/1807941 [15:13<09:02, 1004.26 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1263778/1807941 [15:13<08:50, 1026.52 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1263901/1807941 [15:14<08:26, 1073.62 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1264023/1807941 [15:14<09:13, 982.03 examples/s] 
Tokenizing train dataset:  70%|██████▉   | 1264139/1807941 [15:14<08:51, 1023.57 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1264253/1807941 [15:14<08:36, 1052.76 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1264407/1807941 [15:14<08:45, 1034.89 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1264518/1807941 [15:14<08:36, 1052.05 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1264646/1807941 [15:14<08:10, 1107.40 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1264804/1807941 [15:14<08:25, 1073.91 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1264973/1807941 [15:15<08:23, 1077.87 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1265128/1807941 [15:15<09:26, 957.95 examples/s] 
Tokenizing train dataset:  70%|██████▉   | 1265268/1807941 [15:15<09:37, 940.07 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1265365/1807941 [15:15<10:31, 859.42 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1265472/1807941 [15:15<10:00, 903.36 examples/s]
Tokenizing train dataset:  70%|███████   | 1265565/1807941 [15:15<09:56, 908.91 examples/s]
Tokenizing train dataset:  70%|███████   | 1265673/1807941 [15:15<09:29, 951.81 examples/s]
Tokenizing train dataset:  70%|███████   | 1265822/1807941 [15:16<09:23, 961.81 examples/s]
Tokenizing train dataset:  70%|███████   | 1265952/1807941 [15:16<09:50, 917.76 examples/s]
Tokenizing train dataset:  70%|███████   | 1266050/1807941 [15:16<10:52, 831.01 examples/s]
Tokenizing train dataset:  70%|███████   | 1266163/1807941 [15:16<11:31, 783.09 examples/s]
Tokenizing train dataset:  70%|███████   | 1266244/1807941 [15:16<11:28, 786.41 examples/s]
Tokenizing train dataset:  70%|███████   | 1266336/1807941 [15:16<11:07, 811.53 examples/s]
Tokenizing train dataset:  70%|███████   | 1266424/1807941 [15:16<10:55, 826.57 examples/s]
Tokenizing train dataset:  70%|███████   | 1266508/1807941 [15:16<10:53, 829.02 examples/s]
Tokenizing train dataset:  70%|███████   | 1266627/1807941 [15:17<11:05, 813.52 examples/s]
Tokenizing train dataset:  70%|███████   | 1266736/1807941 [15:17<11:39, 773.60 examples/s]
Tokenizing train dataset:  70%|███████   | 1266837/1807941 [15:17<12:19, 731.34 examples/s]
Tokenizing train dataset:  70%|███████   | 1266933/1807941 [15:17<12:54, 698.92 examples/s]
Tokenizing train dataset:  70%|███████   | 1267038/1807941 [15:17<13:43, 656.63 examples/s]
Tokenizing train dataset:  70%|███████   | 1267143/1807941 [15:17<13:37, 661.82 examples/s]
Tokenizing train dataset:  70%|███████   | 1267211/1807941 [15:17<13:42, 657.38 examples/s]
Tokenizing train dataset:  70%|███████   | 1267295/1807941 [15:18<12:57, 695.55 examples/s]
Tokenizing train dataset:  70%|███████   | 1267388/1807941 [15:18<13:39, 659.71 examples/s]
Tokenizing train dataset:  70%|███████   | 1267477/1807941 [15:18<12:40, 710.87 examples/s]
Tokenizing train dataset:  70%|███████   | 1267592/1807941 [15:18<12:26, 724.28 examples/s]
Tokenizing train dataset:  70%|███████   | 1267691/1807941 [15:18<12:57, 694.98 examples/s]
Tokenizing train dataset:  70%|███████   | 1267775/1807941 [15:18<12:25, 724.29 examples/s]
Tokenizing train dataset:  70%|███████   | 1267858/1807941 [15:18<13:37, 660.36 examples/s]
Tokenizing train dataset:  70%|███████   | 1267951/1807941 [15:19<14:17, 629.80 examples/s]
Tokenizing train dataset:  70%|███████   | 1268035/1807941 [15:19<15:15, 589.43 examples/s]
Tokenizing train dataset:  70%|███████   | 1268128/1807941 [15:19<15:05, 596.18 examples/s]
Tokenizing train dataset:  70%|███████   | 1268189/1807941 [15:19<15:07, 594.88 examples/s]
Tokenizing train dataset:  70%|███████   | 1268273/1807941 [15:19<15:28, 581.27 examples/s]
Tokenizing train dataset:  70%|███████   | 1268356/1807941 [15:19<16:02, 560.54 examples/s]
Tokenizing train dataset:  70%|███████   | 1268428/1807941 [15:19<15:06, 594.86 examples/s]
Tokenizing train dataset:  70%|███████   | 1268490/1807941 [15:20<14:59, 599.52 examples/s]
Tokenizing train dataset:  70%|███████   | 1268558/1807941 [15:20<14:32, 618.03 examples/s]
Tokenizing train dataset:  70%|███████   | 1268627/1807941 [15:20<14:11, 633.64 examples/s]
Tokenizing train dataset:  70%|███████   | 1268713/1807941 [15:20<14:49, 606.12 examples/s]
Tokenizing train dataset:  70%|███████   | 1268782/1807941 [15:20<14:29, 619.91 examples/s]
Tokenizing train dataset:  70%|███████   | 1268863/1807941 [15:20<15:14, 589.50 examples/s]
Tokenizing train dataset:  70%|███████   | 1268926/1807941 [15:20<15:05, 595.40 examples/s]
Tokenizing train dataset:  70%|███████   | 1268996/1807941 [15:20<14:28, 620.80 examples/s]
Tokenizing train dataset:  70%|███████   | 1269098/1807941 [15:21<15:41, 572.03 examples/s]
Tokenizing train dataset:  70%|███████   | 1269158/1807941 [15:21<15:32, 577.69 examples/s]
Tokenizing train dataset:  70%|███████   | 1269233/1807941 [15:21<14:31, 618.02 examples/s]
Tokenizing train dataset:  70%|███████   | 1269302/1807941 [15:21<14:15, 629.91 examples/s]
Tokenizing train dataset:  70%|███████   | 1269374/1807941 [15:21<13:46, 651.96 examples/s]
Tokenizing train dataset:  70%|███████   | 1269442/1807941 [15:21<13:44, 652.86 examples/s]
Tokenizing train dataset:  70%|███████   | 1269532/1807941 [15:21<14:18, 626.83 examples/s]
Tokenizing train dataset:  70%|███████   | 1269620/1807941 [15:21<14:50, 604.48 examples/s]
Tokenizing train dataset:  70%|███████   | 1269715/1807941 [15:22<14:46, 607.24 examples/s]
Tokenizing train dataset:  70%|███████   | 1269788/1807941 [15:22<14:09, 633.21 examples/s]
Tokenizing train dataset:  70%|███████   | 1269879/1807941 [15:22<14:56, 600.09 examples/s]
Tokenizing train dataset:  70%|███████   | 1269969/1807941 [15:22<15:11, 590.30 examples/s]
Tokenizing train dataset:  70%|███████   | 1270053/1807941 [15:22<17:08, 523.15 examples/s]
Tokenizing train dataset:  70%|███████   | 1270127/1807941 [15:22<17:39, 507.74 examples/s]
Tokenizing train dataset:  70%|███████   | 1270186/1807941 [15:22<17:19, 517.25 examples/s]
Tokenizing train dataset:  70%|███████   | 1270249/1807941 [15:23<16:43, 536.04 examples/s]
Tokenizing train dataset:  70%|███████   | 1270306/1807941 [15:23<16:30, 542.70 examples/s]
Tokenizing train dataset:  70%|███████   | 1270363/1807941 [15:23<16:33, 540.83 examples/s]
Tokenizing train dataset:  70%|███████   | 1270424/1807941 [15:23<16:04, 557.09 examples/s]
Tokenizing train dataset:  70%|███████   | 1270507/1807941 [15:23<16:14, 551.33 examples/s]
Tokenizing train dataset:  70%|███████   | 1270563/1807941 [15:23<16:41, 536.50 examples/s]
Tokenizing train dataset:  70%|███████   | 1270623/1807941 [15:23<16:18, 549.00 examples/s]
Tokenizing train dataset:  70%|███████   | 1270698/1807941 [15:23<17:06, 523.14 examples/s]
Tokenizing train dataset:  70%|███████   | 1270855/1807941 [15:23<11:24, 784.78 examples/s]
Tokenizing train dataset:  70%|███████   | 1271142/1807941 [15:24<07:21, 1216.02 examples/s]
Tokenizing train dataset:  70%|███████   | 1271456/1807941 [15:24<05:14, 1706.21 examples/s]
Tokenizing train dataset:  70%|███████   | 1271757/1807941 [15:24<04:21, 2052.49 examples/s]
Tokenizing train dataset:  70%|███████   | 1272038/1807941 [15:24<03:57, 2258.76 examples/s]
Tokenizing train dataset:  70%|███████   | 1272348/1807941 [15:24<03:34, 2494.46 examples/s]
Tokenizing train dataset:  70%|███████   | 1272661/1807941 [15:24<03:20, 2675.87 examples/s]
Tokenizing train dataset:  70%|███████   | 1272953/1807941 [15:24<03:14, 2743.56 examples/s]
Tokenizing train dataset:  70%|███████   | 1273322/1807941 [15:24<03:23, 2631.56 examples/s]
Tokenizing train dataset:  70%|███████   | 1273596/1807941 [15:24<03:21, 2656.20 examples/s]
Tokenizing train dataset:  70%|███████   | 1273895/1807941 [15:25<03:14, 2746.67 examples/s]
Tokenizing train dataset:  70%|███████   | 1274272/1807941 [15:25<03:21, 2653.41 examples/s]
Tokenizing train dataset:  70%|███████   | 1274581/1807941 [15:25<03:12, 2764.89 examples/s]
Tokenizing train dataset:  71%|███████   | 1274879/1807941 [15:25<03:09, 2817.97 examples/s]
Tokenizing train dataset:  71%|███████   | 1275305/1807941 [15:25<03:08, 2819.92 examples/s]
Tokenizing train dataset:  71%|███████   | 1275603/1807941 [15:25<03:06, 2859.25 examples/s]
Tokenizing train dataset:  71%|███████   | 1275899/1807941 [15:25<03:04, 2884.18 examples/s]
Tokenizing train dataset:  71%|███████   | 1276322/1807941 [15:25<03:06, 2856.17 examples/s]
Tokenizing train dataset:  71%|███████   | 1276633/1807941 [15:25<03:01, 2919.69 examples/s]
Tokenizing train dataset:  71%|███████   | 1276950/1807941 [15:26<02:57, 2984.81 examples/s]
Tokenizing train dataset:  71%|███████   | 1277411/1807941 [15:26<02:55, 3015.82 examples/s]
Tokenizing train dataset:  71%|███████   | 1277725/1807941 [15:26<02:54, 3044.71 examples/s]
Tokenizing train dataset:  71%|███████   | 1278049/1807941 [15:26<02:51, 3094.34 examples/s]
Tokenizing train dataset:  71%|███████   | 1278394/1807941 [15:26<02:46, 3189.39 examples/s]
Tokenizing train dataset:  71%|███████   | 1278735/1807941 [15:26<02:42, 3247.46 examples/s]
Tokenizing train dataset:  71%|███████   | 1279216/1807941 [15:26<02:43, 3228.00 examples/s]
Tokenizing train dataset:  71%|███████   | 1279545/1807941 [15:26<02:42, 3242.45 examples/s]
Tokenizing train dataset:  71%|███████   | 1280000/1807941 [15:27<02:47, 3159.34 examples/s]
Tokenizing train dataset:  71%|███████   | 1280340/1807941 [15:27<02:43, 3217.96 examples/s]
Tokenizing train dataset:  71%|███████   | 1280674/1807941 [15:27<02:42, 3249.59 examples/s]
Tokenizing train dataset:  71%|███████   | 1281163/1807941 [15:27<02:44, 3205.48 examples/s]
Tokenizing train dataset:  71%|███████   | 1281503/1807941 [15:27<02:41, 3252.12 examples/s]
Tokenizing train dataset:  71%|███████   | 1281844/1807941 [15:27<02:39, 3292.61 examples/s]
Tokenizing train dataset:  71%|███████   | 1282345/1807941 [15:27<02:40, 3272.24 examples/s]
Tokenizing train dataset:  71%|███████   | 1282688/1807941 [15:27<02:38, 3309.07 examples/s]
Tokenizing train dataset:  71%|███████   | 1283178/1807941 [15:28<02:39, 3291.55 examples/s]
Tokenizing train dataset:  71%|███████   | 1283678/1807941 [15:28<02:38, 3300.39 examples/s]
Tokenizing train dataset:  71%|███████   | 1284168/1807941 [15:28<02:40, 3254.28 examples/s]
Tokenizing train dataset:  71%|███████   | 1284499/1807941 [15:28<02:40, 3264.85 examples/s]
Tokenizing train dataset:  71%|███████   | 1284828/1807941 [15:28<02:40, 3268.30 examples/s]
Tokenizing train dataset:  71%|███████   | 1285303/1807941 [15:28<02:42, 3221.12 examples/s]
Tokenizing train dataset:  71%|███████   | 1285639/1807941 [15:28<02:40, 3254.10 examples/s]
Tokenizing train dataset:  71%|███████   | 1285968/1807941 [15:28<02:40, 3260.74 examples/s]
Tokenizing train dataset:  71%|███████   | 1286441/1807941 [15:29<02:42, 3216.76 examples/s]
Tokenizing train dataset:  71%|███████   | 1286779/1807941 [15:29<02:40, 3255.80 examples/s]
Tokenizing train dataset:  71%|███████   | 1287246/1807941 [15:29<02:42, 3200.50 examples/s]
Tokenizing train dataset:  71%|███████   | 1287731/1807941 [15:29<02:42, 3209.50 examples/s]
Tokenizing train dataset:  71%|███████▏  | 1288203/1807941 [15:29<02:43, 3186.38 examples/s]
Tokenizing train dataset:  71%|███████▏  | 1288538/1807941 [15:29<02:41, 3222.91 examples/s]
Tokenizing train dataset:  71%|███████▏  | 1288868/1807941 [15:29<02:40, 3240.79 examples/s]
Tokenizing train dataset:  71%|███████▏  | 1289352/1807941 [15:29<02:40, 3233.18 examples/s]
Tokenizing train dataset:  71%|███████▏  | 1289698/1807941 [15:30<02:37, 3286.87 examples/s]
Tokenizing train dataset:  71%|███████▏  | 1290201/1807941 [15:30<02:36, 3308.46 examples/s]
Tokenizing train dataset:  71%|███████▏  | 1290663/1807941 [15:30<02:40, 3230.48 examples/s]
Tokenizing train dataset:  71%|███████▏  | 1291089/1807941 [15:30<02:46, 3099.35 examples/s]
Tokenizing train dataset:  71%|███████▏  | 1291537/1807941 [15:30<02:48, 3061.03 examples/s]
Tokenizing train dataset:  71%|███████▏  | 1291978/1807941 [15:30<02:51, 3015.47 examples/s]
Tokenizing train dataset:  71%|███████▏  | 1292396/1807941 [15:30<02:55, 2940.77 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1292825/1807941 [15:31<02:56, 2913.74 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1293224/1807941 [15:31<03:01, 2833.35 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1293517/1807941 [15:31<03:00, 2852.22 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1293815/1807941 [15:31<02:58, 2880.98 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1294217/1807941 [15:31<03:03, 2806.85 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1294504/1807941 [15:31<03:02, 2818.63 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1294802/1807941 [15:31<02:59, 2856.97 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1295224/1807941 [15:31<03:00, 2835.60 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1295513/1807941 [15:32<03:00, 2844.26 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1295803/1807941 [15:32<02:59, 2856.60 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1296201/1807941 [15:32<03:04, 2780.96 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1296484/1807941 [15:32<03:03, 2790.62 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1296765/1807941 [15:32<03:03, 2792.24 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1297167/1807941 [15:32<03:05, 2748.28 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1297596/1807941 [15:32<03:03, 2777.12 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1297886/1807941 [15:32<03:02, 2801.76 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1298319/1807941 [15:33<03:00, 2824.95 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1298624/1807941 [15:33<02:57, 2876.99 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1298935/1807941 [15:33<02:53, 2934.28 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1299363/1807941 [15:33<02:55, 2904.08 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1299681/1807941 [15:33<02:50, 2972.43 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1299988/1807941 [15:33<02:49, 2995.11 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1300445/1807941 [15:33<02:48, 3010.01 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1300754/1807941 [15:33<02:47, 3028.68 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1301202/1807941 [15:33<02:48, 3009.21 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1301512/1807941 [15:34<02:47, 3031.46 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1301835/1807941 [15:34<02:44, 3080.68 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1302260/1807941 [15:34<02:49, 2987.61 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1302696/1807941 [15:34<02:51, 2950.37 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1303102/1807941 [15:34<02:56, 2865.56 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1303528/1807941 [15:34<02:56, 2853.03 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1303825/1807941 [15:34<02:55, 2876.21 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1304244/1807941 [15:35<02:56, 2846.17 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1304557/1807941 [15:35<02:52, 2911.95 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1304867/1807941 [15:35<02:50, 2956.29 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1305310/1807941 [15:35<02:50, 2942.31 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1305620/1807941 [15:35<02:48, 2978.70 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1305930/1807941 [15:35<02:46, 3008.28 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1306361/1807941 [15:35<02:49, 2955.80 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1306674/1807941 [15:35<02:47, 2998.74 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1306998/1807941 [15:35<02:43, 3060.19 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1307449/1807941 [15:36<02:44, 3037.79 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1307763/1807941 [15:36<02:43, 3060.92 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1308223/1807941 [15:36<02:43, 3060.73 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1308537/1807941 [15:36<02:42, 3077.01 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1308865/1807941 [15:36<02:39, 3126.86 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1309337/1807941 [15:36<02:39, 3129.61 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1309666/1807941 [15:36<02:37, 3169.14 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1310110/1807941 [15:36<02:41, 3087.94 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1310436/1807941 [15:37<02:38, 3130.10 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1310769/1807941 [15:37<02:36, 3180.57 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1311221/1807941 [15:37<02:39, 3115.78 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1311699/1807941 [15:37<02:38, 3133.68 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1312167/1807941 [15:37<02:38, 3120.49 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1312491/1807941 [15:37<02:37, 3148.15 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1312818/1807941 [15:37<02:35, 3177.98 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1313263/1807941 [15:37<02:39, 3100.20 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1313582/1807941 [15:38<02:38, 3120.24 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1313912/1807941 [15:38<02:36, 3161.13 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1314362/1807941 [15:38<02:39, 3100.88 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1314690/1807941 [15:38<02:36, 3145.79 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1315161/1807941 [15:38<02:38, 3117.30 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1315644/1807941 [15:38<02:36, 3148.33 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1315972/1807941 [15:38<02:34, 3177.52 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1316441/1807941 [15:38<02:35, 3155.70 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1316780/1807941 [15:39<02:32, 3210.47 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1317248/1807941 [15:39<02:34, 3176.72 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1317583/1807941 [15:39<02:32, 3216.78 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1317922/1807941 [15:39<02:30, 3260.94 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1318398/1807941 [15:39<02:31, 3226.87 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1318872/1807941 [15:39<02:32, 3199.68 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1319303/1807941 [15:39<02:38, 3082.80 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1319732/1807941 [15:40<02:42, 3010.63 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1320158/1807941 [15:40<02:45, 2955.60 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1320588/1807941 [15:40<02:46, 2927.21 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1320888/1807941 [15:40<02:45, 2941.87 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1321292/1807941 [15:40<02:50, 2855.36 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1321586/1807941 [15:40<02:49, 2873.99 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1321882/1807941 [15:40<02:48, 2890.20 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1322296/1807941 [15:40<02:51, 2838.95 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1322591/1807941 [15:41<02:49, 2863.67 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1322891/1807941 [15:41<02:47, 2898.00 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1323326/1807941 [15:41<02:47, 2896.35 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1323628/1807941 [15:41<02:45, 2924.02 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1323929/1807941 [15:41<02:44, 2941.68 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1324349/1807941 [15:41<02:47, 2886.38 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1324660/1807941 [15:41<02:44, 2940.83 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1324979/1807941 [15:41<02:40, 3003.43 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1325406/1807941 [15:41<02:44, 2939.71 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1325703/1807941 [15:42<02:43, 2943.67 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1326100/1807941 [15:42<02:50, 2831.40 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1326388/1807941 [15:42<02:49, 2841.10 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1326681/1807941 [15:42<02:48, 2861.38 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1326972/1807941 [15:42<02:47, 2872.63 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1327364/1807941 [15:42<02:53, 2773.49 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1327649/1807941 [15:42<02:52, 2790.58 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1327934/1807941 [15:42<02:51, 2802.87 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1328326/1807941 [15:43<02:55, 2730.98 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1328616/1807941 [15:43<02:52, 2770.99 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1328896/1807941 [15:43<02:52, 2776.66 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1329299/1807941 [15:43<02:54, 2741.35 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1329599/1807941 [15:43<02:50, 2805.56 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1329895/1807941 [15:43<02:48, 2844.61 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1330304/1807941 [15:43<02:50, 2798.55 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1330603/1807941 [15:43<02:47, 2844.74 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1330900/1807941 [15:43<02:45, 2873.92 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1331321/1807941 [15:44<02:47, 2845.85 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1331625/1807941 [15:44<02:44, 2894.08 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1331921/1807941 [15:44<02:43, 2908.61 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1332336/1807941 [15:44<02:46, 2852.20 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1332634/1807941 [15:44<02:44, 2882.49 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1332943/1807941 [15:44<02:41, 2936.12 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1333349/1807941 [15:44<02:46, 2846.61 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1333650/1807941 [15:44<02:44, 2884.38 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1333956/1807941 [15:44<02:41, 2926.34 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1334368/1807941 [15:45<02:45, 2857.43 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1334676/1807941 [15:45<02:42, 2912.16 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1335084/1807941 [15:45<02:46, 2841.33 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1335377/1807941 [15:45<02:45, 2861.47 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1335675/1807941 [15:45<02:43, 2891.04 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1335978/1807941 [15:45<02:41, 2927.38 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1336397/1807941 [15:45<02:44, 2872.87 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1336832/1807941 [15:45<02:43, 2879.30 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1337246/1807941 [15:46<02:46, 2834.43 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1337541/1807941 [15:46<02:44, 2859.72 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1337830/1807941 [15:46<02:44, 2863.92 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1338232/1807941 [15:46<02:48, 2795.77 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1338517/1807941 [15:46<02:47, 2807.56 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1338939/1807941 [15:46<02:47, 2803.75 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1339325/1807941 [15:46<02:51, 2724.60 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1339609/1807941 [15:46<02:50, 2749.34 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1339902/1807941 [15:47<02:47, 2791.54 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1340321/1807941 [15:47<02:47, 2789.60 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1340613/1807941 [15:47<02:45, 2820.09 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1340905/1807941 [15:47<02:44, 2844.16 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1341306/1807941 [15:47<02:47, 2780.58 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1341587/1807941 [15:47<02:47, 2786.33 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1341886/1807941 [15:47<02:44, 2839.98 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1342270/1807941 [15:47<02:50, 2732.17 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1342560/1807941 [15:48<02:47, 2772.60 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1342855/1807941 [15:48<02:45, 2817.10 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1343256/1807941 [15:48<02:48, 2761.92 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1343559/1807941 [15:48<02:44, 2828.53 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1343854/1807941 [15:48<02:42, 2858.47 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1344154/1807941 [15:48<02:43, 2836.77 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1344455/1807941 [15:48<02:40, 2882.60 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1344748/1807941 [15:48<02:40, 2887.73 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1345155/1807941 [15:48<02:44, 2815.90 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1345583/1807941 [15:49<02:43, 2824.06 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1345880/1807941 [15:49<02:41, 2858.55 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1346321/1807941 [15:49<02:40, 2883.72 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1346651/1807941 [15:49<02:34, 2984.37 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1346963/1807941 [15:49<02:32, 3015.87 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1347395/1807941 [15:49<02:35, 2965.10 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1347723/1807941 [15:49<02:31, 3042.02 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1348154/1807941 [15:49<02:34, 2978.97 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1348475/1807941 [15:50<02:31, 3035.07 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1348783/1807941 [15:50<02:30, 3044.46 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1349233/1807941 [15:50<02:31, 3027.84 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1349558/1807941 [15:50<02:28, 3077.48 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1349869/1807941 [15:50<02:28, 3082.94 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1350326/1807941 [15:50<02:29, 3069.05 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1350647/1807941 [15:50<02:27, 3102.46 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1350977/1807941 [15:50<02:25, 3149.87 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1351421/1807941 [15:51<02:28, 3078.54 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1351754/1807941 [15:51<02:25, 3140.77 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1352219/1807941 [15:51<02:25, 3124.67 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1352537/1807941 [15:51<02:25, 3136.65 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1352864/1807941 [15:51<02:23, 3169.40 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1353318/1807941 [15:51<02:26, 3111.27 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1353765/1807941 [15:51<02:28, 3061.24 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1354179/1807941 [15:51<02:33, 2958.38 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1354479/1807941 [15:52<02:32, 2966.31 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1354902/1807941 [15:52<02:35, 2911.82 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1355317/1807941 [15:52<02:38, 2860.70 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1355620/1807941 [15:52<02:36, 2897.20 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1356028/1807941 [15:52<02:39, 2835.54 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1356326/1807941 [15:52<02:37, 2868.52 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1356616/1807941 [15:52<02:36, 2874.92 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1356908/1807941 [15:52<02:36, 2884.83 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1357335/1807941 [15:53<02:37, 2866.74 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1357659/1807941 [15:53<02:32, 2961.25 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1357986/1807941 [15:53<02:27, 3041.18 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1358446/1807941 [15:53<02:27, 3047.32 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1358780/1807941 [15:53<02:24, 3118.45 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1359245/1807941 [15:53<02:24, 3107.57 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1359561/1807941 [15:53<02:23, 3119.19 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1359879/1807941 [15:53<02:23, 3132.31 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1360341/1807941 [15:53<02:24, 3106.18 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1360664/1807941 [15:54<02:22, 3134.62 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1360985/1807941 [15:54<02:21, 3152.44 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1361440/1807941 [15:54<02:23, 3104.54 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1361764/1807941 [15:54<02:22, 3136.72 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1362220/1807941 [15:54<02:23, 3099.59 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1362545/1807941 [15:54<02:22, 3135.10 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1362867/1807941 [15:54<02:21, 3153.47 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1363320/1807941 [15:54<02:23, 3104.16 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1363644/1807941 [15:55<02:21, 3135.77 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1363962/1807941 [15:55<02:21, 3144.41 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1364423/1807941 [15:55<02:22, 3114.02 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1364759/1807941 [15:55<02:19, 3172.88 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1365206/1807941 [15:55<02:22, 3099.31 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1365539/1807941 [15:55<02:20, 3154.96 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1365867/1807941 [15:55<02:18, 3186.81 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1366340/1807941 [15:55<02:19, 3170.25 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1366674/1807941 [15:55<02:17, 3210.51 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1367000/1807941 [15:56<02:18, 3194.46 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1367492/1807941 [15:56<02:16, 3222.65 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1367825/1807941 [15:56<02:15, 3246.26 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1368278/1807941 [15:56<02:19, 3161.90 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1368612/1807941 [15:56<02:17, 3203.82 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1368939/1807941 [15:56<02:16, 3217.87 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1369395/1807941 [15:56<02:19, 3151.46 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1369723/1807941 [15:56<02:17, 3182.54 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1370158/1807941 [15:57<02:22, 3078.27 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1370597/1807941 [15:57<02:24, 3025.75 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1371014/1807941 [15:57<02:28, 2944.42 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1371447/1807941 [15:57<02:29, 2924.05 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1371748/1807941 [15:57<02:28, 2941.87 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1372167/1807941 [15:57<02:30, 2887.23 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1372459/1807941 [15:57<02:30, 2893.82 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1372755/1807941 [15:57<02:29, 2908.31 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1373178/1807941 [15:58<02:31, 2869.81 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1373606/1807941 [15:58<02:31, 2860.03 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1373900/1807941 [15:58<02:30, 2877.81 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1374314/1807941 [15:58<02:33, 2830.65 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1374602/1807941 [15:58<02:32, 2839.00 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1374895/1807941 [15:58<02:31, 2862.16 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1375317/1807941 [15:58<02:32, 2841.37 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1375629/1807941 [15:59<02:28, 2908.20 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1375936/1807941 [15:59<02:26, 2947.19 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1376361/1807941 [15:59<02:28, 2901.08 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1376660/1807941 [15:59<02:27, 2921.58 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1376957/1807941 [15:59<02:26, 2931.98 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1377372/1807941 [15:59<02:30, 2870.27 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1377668/1807941 [15:59<02:28, 2892.95 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1377974/1807941 [15:59<02:26, 2936.34 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1378382/1807941 [15:59<02:30, 2854.75 examples/s]
Tokenizing train dataset:  76%|███████▋  | 1378694/1807941 [16:00<02:26, 2920.67 examples/s]
Tokenizing train dataset:  76%|███████▋  | 1378999/1807941 [16:00<02:25, 2954.36 examples/s]
Tokenizing train dataset:  76%|███████▋  | 1379428/1807941 [16:00<02:27, 2914.15 examples/s]
Tokenizing train dataset:  76%|███████▋  | 1379729/1807941 [16:00<02:25, 2934.81 examples/s]
Tokenizing train dataset:  76%|███████▋  | 1380162/1807941 [16:00<02:26, 2914.13 examples/s]
Tokenizing train dataset:  76%|███████▋  | 1380466/1807941 [16:00<02:25, 2944.45 examples/s]
Tokenizing train dataset:  76%|███████▋  | 1380767/1807941 [16:00<02:24, 2958.39 examples/s]
Tokenizing train dataset:  76%|███████▋  | 1381180/1807941 [16:00<02:28, 2881.25 examples/s]
Tokenizing train dataset:  76%|███████▋  | 1381477/1807941 [16:01<02:26, 2901.77 examples/s]
Tokenizing train dataset:  76%|███████▋  | 1381780/1807941 [16:01<02:25, 2934.61 examples/s]
Tokenizing train dataset:  76%|███████▋  | 1382202/1807941 [16:01<02:27, 2887.59 examples/s]
Tokenizing train dataset:  76%|███████▋  | 1382507/1807941 [16:01<02:25, 2926.26 examples/s]
Tokenizing train dataset:  76%|███████▋  | 1382942/1807941 [16:01<02:26, 2908.45 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1383344/1807941 [16:01<02:30, 2830.49 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1383633/1807941 [16:01<02:29, 2842.56 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1384032/1807941 [16:01<02:32, 2776.59 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1384327/1807941 [16:02<02:30, 2817.25 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1384621/1807941 [16:02<02:28, 2845.54 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1384916/1807941 [16:02<02:27, 2873.00 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1385308/1807941 [16:02<02:32, 2773.53 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1385605/1807941 [16:02<02:29, 2821.23 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1385913/1807941 [16:02<02:26, 2886.32 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1386337/1807941 [16:02<02:27, 2860.98 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1386646/1807941 [16:02<02:24, 2915.88 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1386946/1807941 [16:02<02:23, 2935.59 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1387362/1807941 [16:03<02:26, 2875.77 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1387660/1807941 [16:03<02:25, 2896.72 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1387968/1807941 [16:03<02:22, 2944.55 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1388386/1807941 [16:03<02:25, 2883.18 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1388679/1807941 [16:03<02:24, 2891.76 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1388976/1807941 [16:03<02:24, 2908.18 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1389390/1807941 [16:03<02:27, 2846.58 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1389692/1807941 [16:03<02:24, 2887.48 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1389996/1807941 [16:03<02:22, 2927.47 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1390416/1807941 [16:04<02:25, 2876.47 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1390724/1807941 [16:04<02:22, 2926.43 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1391153/1807941 [16:04<02:24, 2886.48 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1391445/1807941 [16:04<02:23, 2893.20 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1391744/1807941 [16:04<02:22, 2915.94 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1392176/1807941 [16:04<02:23, 2901.69 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1392481/1807941 [16:04<02:21, 2933.90 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1392786/1807941 [16:04<02:20, 2961.87 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1393194/1807941 [16:05<02:24, 2871.41 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1393493/1807941 [16:05<02:23, 2897.77 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1393788/1807941 [16:05<02:22, 2908.13 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1394213/1807941 [16:05<02:23, 2877.76 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1394506/1807941 [16:05<02:23, 2888.87 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1394809/1807941 [16:05<02:21, 2924.77 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1395235/1807941 [16:05<02:22, 2888.41 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1395526/1807941 [16:05<02:22, 2890.21 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1395823/1807941 [16:05<02:21, 2910.83 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1396231/1807941 [16:06<02:25, 2836.83 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1396654/1807941 [16:06<02:25, 2826.08 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1396965/1807941 [16:06<02:22, 2893.17 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1397353/1807941 [16:06<02:27, 2785.20 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1397649/1807941 [16:06<02:25, 2826.65 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1397948/1807941 [16:06<02:23, 2865.86 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1398348/1807941 [16:06<02:26, 2790.61 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1398645/1807941 [16:06<02:24, 2832.70 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1398939/1807941 [16:07<02:23, 2858.09 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1399340/1807941 [16:07<02:26, 2787.62 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1399639/1807941 [16:07<02:23, 2837.28 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1399933/1807941 [16:07<02:22, 2862.79 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1400359/1807941 [16:07<02:22, 2850.65 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1400658/1807941 [16:07<02:21, 2883.75 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1400956/1807941 [16:07<02:20, 2906.96 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1401351/1807941 [16:07<02:24, 2805.95 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1401654/1807941 [16:08<02:22, 2860.07 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1401965/1807941 [16:08<02:18, 2925.90 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1402395/1807941 [16:08<02:19, 2899.40 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1402721/1807941 [16:08<02:15, 2987.21 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1403179/1807941 [16:08<02:14, 3005.55 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1403499/1807941 [16:08<02:12, 3051.41 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1403820/1807941 [16:08<02:10, 3090.30 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1404276/1807941 [16:08<02:11, 3069.63 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1404599/1807941 [16:08<02:09, 3105.70 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1404927/1807941 [16:09<02:07, 3149.46 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1405388/1807941 [16:09<02:09, 3118.90 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1405709/1807941 [16:09<02:08, 3140.61 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1406147/1807941 [16:09<02:12, 3031.65 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1406602/1807941 [16:09<02:12, 3022.77 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1406907/1807941 [16:09<02:12, 3028.22 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1407352/1807941 [16:09<02:13, 3005.55 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1407656/1807941 [16:09<02:12, 3011.36 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1408052/1807941 [16:10<02:18, 2880.63 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1408358/1807941 [16:10<02:16, 2922.65 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1408659/1807941 [16:10<02:15, 2943.52 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1409072/1807941 [16:10<02:18, 2870.79 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1409373/1807941 [16:10<02:17, 2904.24 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1409686/1807941 [16:10<02:14, 2962.05 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1410000/1807941 [16:10<02:14, 2955.25 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1410325/1807941 [16:10<02:10, 3035.40 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1410662/1807941 [16:11<02:07, 3127.50 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1410995/1807941 [16:11<02:04, 3181.03 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1411461/1807941 [16:11<02:05, 3151.37 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1411798/1807941 [16:11<02:03, 3207.28 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1412257/1807941 [16:11<02:05, 3150.22 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1412593/1807941 [16:11<02:03, 3201.42 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1412923/1807941 [16:11<02:02, 3225.26 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1413396/1807941 [16:11<02:03, 3192.46 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1413883/1807941 [16:12<02:02, 3207.04 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1414355/1807941 [16:12<02:03, 3182.94 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1414695/1807941 [16:12<02:01, 3231.34 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1415163/1807941 [16:12<02:03, 3192.48 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1415500/1807941 [16:12<02:01, 3233.43 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1415863/1807941 [16:12<01:57, 3334.26 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1416350/1807941 [16:12<01:58, 3301.67 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1416706/1807941 [16:12<01:56, 3363.59 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1417202/1807941 [16:13<01:56, 3341.63 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1417568/1807941 [16:13<01:54, 3416.68 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1418058/1807941 [16:13<01:55, 3361.84 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1418567/1807941 [16:13<01:55, 3366.26 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1419036/1807941 [16:13<01:58, 3285.37 examples/s]
Tokenizing train dataset:  79%|███████▊  | 1419369/1807941 [16:13<01:57, 3294.35 examples/s]
Tokenizing train dataset:  79%|███████▊  | 1419702/1807941 [16:13<01:57, 3300.85 examples/s]
Tokenizing train dataset:  79%|███████▊  | 1420180/1807941 [16:13<01:58, 3258.58 examples/s]
Tokenizing train dataset:  79%|███████▊  | 1420512/1807941 [16:14<01:58, 3271.84 examples/s]
Tokenizing train dataset:  79%|███████▊  | 1420852/1807941 [16:14<01:57, 3303.25 examples/s]
Tokenizing train dataset:  79%|███████▊  | 1421333/1807941 [16:14<01:59, 3240.00 examples/s]
Tokenizing train dataset:  79%|███████▊  | 1421673/1807941 [16:14<01:57, 3278.65 examples/s]
Tokenizing train dataset:  79%|███████▊  | 1422166/1807941 [16:14<01:59, 3234.54 examples/s]
Tokenizing train dataset:  79%|███████▊  | 1422615/1807941 [16:14<02:02, 3152.05 examples/s]
Tokenizing train dataset:  79%|███████▊  | 1423046/1807941 [16:14<02:05, 3058.75 examples/s]
Tokenizing train dataset:  79%|███████▊  | 1423488/1807941 [16:14<02:07, 3021.40 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1423798/1807941 [16:15<02:06, 3039.15 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1424224/1807941 [16:15<02:09, 2969.32 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1424529/1807941 [16:15<02:08, 2987.98 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1424976/1807941 [16:15<02:08, 2983.34 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1425400/1807941 [16:15<02:10, 2930.30 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1425698/1807941 [16:15<02:09, 2940.55 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1426000/1807941 [16:15<02:11, 2915.35 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1426301/1807941 [16:15<02:10, 2934.69 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1426754/1807941 [16:16<02:08, 2960.41 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1427165/1807941 [16:16<02:12, 2882.39 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1427469/1807941 [16:16<02:10, 2916.14 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1427774/1807941 [16:16<02:09, 2946.83 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1428196/1807941 [16:16<02:11, 2895.96 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1428492/1807941 [16:16<02:10, 2911.43 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1428798/1807941 [16:16<02:08, 2950.26 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1429229/1807941 [16:16<02:09, 2920.49 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1429528/1807941 [16:17<02:08, 2936.54 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1429825/1807941 [16:17<02:08, 2942.30 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1430242/1807941 [16:17<02:11, 2880.69 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1430546/1807941 [16:17<02:09, 2921.13 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1430852/1807941 [16:17<02:07, 2957.22 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1431150/1807941 [16:17<02:09, 2901.99 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1431446/1807941 [16:17<02:09, 2916.24 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1431888/1807941 [16:17<02:08, 2921.88 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1432306/1807941 [16:18<02:10, 2871.77 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1432613/1807941 [16:18<02:08, 2917.91 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1432921/1807941 [16:18<02:06, 2958.79 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1433340/1807941 [16:18<02:09, 2889.58 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1433637/1807941 [16:18<02:08, 2908.72 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1433946/1807941 [16:18<02:06, 2954.67 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1434352/1807941 [16:18<02:10, 2862.26 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1434645/1807941 [16:18<02:09, 2877.11 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1434944/1807941 [16:18<02:08, 2904.97 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1435377/1807941 [16:19<02:08, 2896.46 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1435711/1807941 [16:19<02:03, 3008.08 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1436179/1807941 [16:19<02:02, 3041.41 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1436522/1807941 [16:19<01:58, 3135.89 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1436849/1807941 [16:19<01:57, 3169.36 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1437171/1807941 [16:19<01:57, 3154.89 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1437631/1807941 [16:19<01:58, 3119.21 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1438040/1807941 [16:19<02:04, 2982.25 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1438478/1807941 [16:20<02:04, 2959.00 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1438916/1807941 [16:20<02:05, 2943.33 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1439360/1807941 [16:20<02:05, 2944.48 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1439685/1807941 [16:20<02:02, 3012.69 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1440000/1807941 [16:20<02:02, 2991.82 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1440313/1807941 [16:20<02:01, 3022.05 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1440634/1807941 [16:20<01:59, 3070.93 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1440963/1807941 [16:20<01:57, 3131.09 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1441413/1807941 [16:21<01:59, 3075.66 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1441740/1807941 [16:21<01:57, 3123.77 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1442206/1807941 [16:21<01:57, 3112.98 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1442533/1807941 [16:21<01:56, 3147.87 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1442859/1807941 [16:21<01:55, 3170.63 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1443298/1807941 [16:21<02:00, 3023.89 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1443741/1807941 [16:21<02:01, 2996.39 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1444174/1807941 [16:21<02:02, 2957.63 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1444485/1807941 [16:22<02:01, 2992.88 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1444800/1807941 [16:22<01:59, 3029.88 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1445242/1807941 [16:22<02:01, 2996.93 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1445558/1807941 [16:22<01:59, 3033.61 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1445874/1807941 [16:22<01:58, 3064.41 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1446326/1807941 [16:22<01:58, 3041.49 examples/s]
Tokenizing train dataset:  80%|████████  | 1446633/1807941 [16:22<01:58, 3045.61 examples/s]
Tokenizing train dataset:  80%|████████  | 1446948/1807941 [16:22<01:57, 3070.62 examples/s]
Tokenizing train dataset:  80%|████████  | 1447378/1807941 [16:22<02:00, 2993.59 examples/s]
Tokenizing train dataset:  80%|████████  | 1447830/1807941 [16:23<02:00, 2997.26 examples/s]
Tokenizing train dataset:  80%|████████  | 1448246/1807941 [16:23<02:03, 2919.73 examples/s]
Tokenizing train dataset:  80%|████████  | 1448670/1807941 [16:23<02:04, 2886.37 examples/s]
Tokenizing train dataset:  80%|████████  | 1449075/1807941 [16:23<02:07, 2821.25 examples/s]
Tokenizing train dataset:  80%|████████  | 1449360/1807941 [16:23<02:06, 2825.64 examples/s]
Tokenizing train dataset:  80%|████████  | 1449644/1807941 [16:23<02:06, 2823.75 examples/s]
Tokenizing train dataset:  80%|████████  | 1449928/1807941 [16:23<02:06, 2826.16 examples/s]
Tokenizing train dataset:  80%|████████  | 1450348/1807941 [16:24<02:07, 2813.98 examples/s]
Tokenizing train dataset:  80%|████████  | 1450672/1807941 [16:24<02:02, 2917.27 examples/s]
Tokenizing train dataset:  80%|████████  | 1450989/1807941 [16:24<01:59, 2981.72 examples/s]
Tokenizing train dataset:  80%|████████  | 1451454/1807941 [16:24<01:58, 3019.06 examples/s]
Tokenizing train dataset:  80%|████████  | 1451763/1807941 [16:24<01:57, 3032.85 examples/s]
Tokenizing train dataset:  80%|████████  | 1452196/1807941 [16:24<01:59, 2975.85 examples/s]
Tokenizing train dataset:  80%|████████  | 1452502/1807941 [16:24<01:58, 2995.17 examples/s]
Tokenizing train dataset:  80%|████████  | 1452808/1807941 [16:24<01:58, 3007.57 examples/s]
Tokenizing train dataset:  80%|████████  | 1453232/1807941 [16:24<02:00, 2939.49 examples/s]
Tokenizing train dataset:  80%|████████  | 1453669/1807941 [16:25<02:00, 2928.51 examples/s]
Tokenizing train dataset:  80%|████████  | 1453976/1807941 [16:25<01:59, 2961.12 examples/s]
Tokenizing train dataset:  80%|████████  | 1454417/1807941 [16:25<01:59, 2949.34 examples/s]
Tokenizing train dataset:  80%|████████  | 1454732/1807941 [16:25<01:57, 2997.67 examples/s]
Tokenizing train dataset:  80%|████████  | 1455169/1807941 [16:25<01:58, 2965.15 examples/s]
Tokenizing train dataset:  81%|████████  | 1455498/1807941 [16:25<01:55, 3044.15 examples/s]
Tokenizing train dataset:  81%|████████  | 1455834/1807941 [16:25<01:52, 3125.23 examples/s]
Tokenizing train dataset:  81%|████████  | 1456177/1807941 [16:25<01:49, 3199.72 examples/s]
Tokenizing train dataset:  81%|████████  | 1456540/1807941 [16:26<01:45, 3315.87 examples/s]
Tokenizing train dataset:  81%|████████  | 1456884/1807941 [16:26<01:44, 3350.25 examples/s]
Tokenizing train dataset:  81%|████████  | 1457386/1807941 [16:26<01:44, 3344.57 examples/s]
Tokenizing train dataset:  81%|████████  | 1457740/1807941 [16:26<01:43, 3391.40 examples/s]
Tokenizing train dataset:  81%|████████  | 1458228/1807941 [16:26<01:44, 3339.27 examples/s]
Tokenizing train dataset:  81%|████████  | 1458565/1807941 [16:26<01:44, 3345.73 examples/s]
Tokenizing train dataset:  81%|████████  | 1459031/1807941 [16:26<01:47, 3257.51 examples/s]
Tokenizing train dataset:  81%|████████  | 1459495/1807941 [16:26<01:48, 3200.05 examples/s]
Tokenizing train dataset:  81%|████████  | 1459961/1807941 [16:27<01:49, 3165.52 examples/s]
Tokenizing train dataset:  81%|████████  | 1460390/1807941 [16:27<01:53, 3061.47 examples/s]
Tokenizing train dataset:  81%|████████  | 1460704/1807941 [16:27<01:52, 3076.15 examples/s]
Tokenizing train dataset:  81%|████████  | 1461150/1807941 [16:27<01:55, 3008.16 examples/s]
Tokenizing train dataset:  81%|████████  | 1461459/1807941 [16:27<01:54, 3026.64 examples/s]
Tokenizing train dataset:  81%|████████  | 1461769/1807941 [16:27<01:53, 3044.44 examples/s]
Tokenizing train dataset:  81%|████████  | 1462225/1807941 [16:27<01:53, 3038.81 examples/s]
Tokenizing train dataset:  81%|████████  | 1462556/1807941 [16:27<01:51, 3105.00 examples/s]
Tokenizing train dataset:  81%|████████  | 1462890/1807941 [16:28<01:49, 3163.97 examples/s]
Tokenizing train dataset:  81%|████████  | 1463371/1807941 [16:28<01:48, 3176.55 examples/s]
Tokenizing train dataset:  81%|████████  | 1463732/1807941 [16:28<01:44, 3284.10 examples/s]
Tokenizing train dataset:  81%|████████  | 1464229/1807941 [16:28<01:44, 3291.41 examples/s]
Tokenizing train dataset:  81%|████████  | 1464594/1807941 [16:28<01:41, 3377.16 examples/s]
Tokenizing train dataset:  81%|████████  | 1464942/1807941 [16:28<01:40, 3401.82 examples/s]
Tokenizing train dataset:  81%|████████  | 1465436/1807941 [16:28<01:41, 3359.01 examples/s]
Tokenizing train dataset:  81%|████████  | 1465790/1807941 [16:28<01:40, 3403.04 examples/s]
Tokenizing train dataset:  81%|████████  | 1466281/1807941 [16:29<01:41, 3354.82 examples/s]
Tokenizing train dataset:  81%|████████  | 1466634/1807941 [16:29<01:40, 3395.86 examples/s]
Tokenizing train dataset:  81%|████████  | 1466977/1807941 [16:29<01:40, 3402.75 examples/s]
Tokenizing train dataset:  81%|████████  | 1467460/1807941 [16:29<01:42, 3332.93 examples/s]
Tokenizing train dataset:  81%|████████  | 1467800/1807941 [16:29<01:41, 3347.86 examples/s]
Tokenizing train dataset:  81%|████████  | 1468297/1807941 [16:29<01:41, 3331.31 examples/s]
Tokenizing train dataset:  81%|████████  | 1468794/1807941 [16:29<01:42, 3322.48 examples/s]
Tokenizing train dataset:  81%|████████▏ | 1469299/1807941 [16:29<01:41, 3332.99 examples/s]
Tokenizing train dataset:  81%|████████▏ | 1469641/1807941 [16:30<01:40, 3350.41 examples/s]
Tokenizing train dataset:  81%|████████▏ | 1469993/1807941 [16:30<01:39, 3390.93 examples/s]
Tokenizing train dataset:  81%|████████▏ | 1470494/1807941 [16:30<01:40, 3370.13 examples/s]
Tokenizing train dataset:  81%|████████▏ | 1470842/1807941 [16:30<01:39, 3395.19 examples/s]
Tokenizing train dataset:  81%|████████▏ | 1471349/1807941 [16:30<01:40, 3355.78 examples/s]
Tokenizing train dataset:  81%|████████▏ | 1471696/1807941 [16:30<01:39, 3381.73 examples/s]
Tokenizing train dataset:  81%|████████▏ | 1472197/1807941 [16:30<01:39, 3365.04 examples/s]
Tokenizing train dataset:  81%|████████▏ | 1472557/1807941 [16:30<01:37, 3422.30 examples/s]
Tokenizing train dataset:  81%|████████▏ | 1472912/1807941 [16:31<01:36, 3454.38 examples/s]
Tokenizing train dataset:  81%|████████▏ | 1473372/1807941 [16:31<01:40, 3313.38 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1473833/1807941 [16:31<01:43, 3227.61 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1474272/1807941 [16:31<01:46, 3127.48 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1474730/1807941 [16:31<01:47, 3098.08 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1475149/1807941 [16:31<01:51, 2987.56 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1475596/1807941 [16:31<01:51, 2984.22 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1475976/1807941 [16:32<01:56, 2843.88 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1476310/1807941 [16:32<02:04, 2654.30 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1476680/1807941 [16:32<02:08, 2587.92 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1477032/1807941 [16:32<02:11, 2510.97 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1477413/1807941 [16:32<02:11, 2509.66 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1477684/1807941 [16:32<02:09, 2551.90 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1478014/1807941 [16:32<02:15, 2430.87 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1478371/1807941 [16:33<02:16, 2407.92 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1478620/1807941 [16:33<02:16, 2410.56 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1478868/1807941 [16:33<02:15, 2425.06 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1479196/1807941 [16:33<02:20, 2339.68 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1479578/1807941 [16:33<02:16, 2404.50 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1479945/1807941 [16:33<02:15, 2416.25 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1480272/1807941 [16:33<02:20, 2329.87 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1480510/1807941 [16:33<02:19, 2340.65 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1480867/1807941 [16:34<02:19, 2352.39 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1481191/1807941 [16:34<02:22, 2288.56 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1481543/1807941 [16:34<02:22, 2288.30 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1481779/1807941 [16:34<02:22, 2292.85 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1482079/1807941 [16:34<02:28, 2188.37 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1482404/1807941 [16:34<02:29, 2179.65 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1482626/1807941 [16:34<02:30, 2163.18 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1482879/1807941 [16:35<02:24, 2244.94 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1483184/1807941 [16:35<02:30, 2161.21 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1483414/1807941 [16:35<02:28, 2191.76 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1483745/1807941 [16:35<02:27, 2195.86 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1483981/1807941 [16:35<02:25, 2221.20 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1484260/1807941 [16:35<02:34, 2094.45 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1484568/1807941 [16:35<02:35, 2073.88 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1484865/1807941 [16:36<02:38, 2039.22 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1485129/1807941 [16:36<02:45, 1948.94 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1485419/1807941 [16:36<02:47, 1929.41 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1485706/1807941 [16:36<02:48, 1917.39 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1486000/1807941 [16:36<02:51, 1875.88 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1486275/1807941 [16:36<02:53, 1858.05 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1486485/1807941 [16:36<02:48, 1908.41 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1486680/1807941 [16:36<02:47, 1916.22 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1486965/1807941 [16:37<02:49, 1898.04 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1487249/1807941 [16:37<02:49, 1894.71 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1487522/1807941 [16:37<02:51, 1868.01 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1487714/1807941 [16:37<02:50, 1875.38 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1487922/1807941 [16:37<02:46, 1917.53 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1488199/1807941 [16:37<02:50, 1874.74 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1488394/1807941 [16:37<02:48, 1891.78 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1488704/1807941 [16:38<02:43, 1948.62 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1488938/1807941 [16:38<02:36, 2043.71 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1489186/1807941 [16:38<02:27, 2156.34 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1489412/1807941 [16:38<02:26, 2179.17 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1489671/1807941 [16:38<02:18, 2290.54 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1489928/1807941 [16:38<02:14, 2368.75 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1490270/1807941 [16:38<02:16, 2333.87 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1490542/1807941 [16:38<02:10, 2431.27 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1490898/1807941 [16:38<02:11, 2407.77 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1491224/1807941 [16:39<02:17, 2308.00 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1491464/1807941 [16:39<02:15, 2328.27 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1491816/1807941 [16:39<02:15, 2331.26 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1492106/1807941 [16:39<02:23, 2196.94 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1492333/1807941 [16:39<02:23, 2202.83 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1492556/1807941 [16:39<02:22, 2207.35 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1492882/1807941 [16:39<02:24, 2184.81 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1493158/1807941 [16:40<02:32, 2057.61 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1493387/1807941 [16:40<02:29, 2110.75 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1493610/1807941 [16:40<02:27, 2135.16 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1493937/1807941 [16:40<02:26, 2146.07 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1494174/1807941 [16:40<02:22, 2200.37 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1494423/1807941 [16:40<02:17, 2274.71 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1494671/1807941 [16:40<02:15, 2310.64 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1494992/1807941 [16:40<02:19, 2245.25 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1495338/1807941 [16:40<02:18, 2264.41 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1495675/1807941 [16:41<02:18, 2256.12 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1496000/1807941 [16:41<02:23, 2173.68 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1496253/1807941 [16:41<02:18, 2252.85 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1496497/1807941 [16:41<02:15, 2297.51 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1496734/1807941 [16:41<02:14, 2314.30 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1496999/1807941 [16:41<02:09, 2404.46 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1497340/1807941 [16:41<02:12, 2350.74 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1497585/1807941 [16:41<02:10, 2374.29 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1497855/1807941 [16:42<02:06, 2456.56 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1498213/1807941 [16:42<02:07, 2422.18 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1498479/1807941 [16:42<02:04, 2481.38 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1498878/1807941 [16:42<02:01, 2540.20 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1499133/1807941 [16:42<02:04, 2485.38 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1499517/1807941 [16:42<02:03, 2502.29 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1499906/1807941 [16:42<02:01, 2528.57 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1500281/1807941 [16:43<02:02, 2517.72 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1500555/1807941 [16:43<02:00, 2559.54 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1500843/1807941 [16:43<01:56, 2631.57 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1501198/1807941 [16:43<02:01, 2524.29 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1501573/1807941 [16:43<02:01, 2511.52 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1501860/1807941 [16:43<01:58, 2586.25 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1502191/1807941 [16:43<02:04, 2455.84 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1502484/1807941 [16:43<01:58, 2568.46 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1502801/1807941 [16:43<01:52, 2721.60 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1503179/1807941 [16:44<01:55, 2636.51 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1503462/1807941 [16:44<01:53, 2682.22 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1503858/1807941 [16:44<01:54, 2665.67 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1504231/1807941 [16:44<01:56, 2600.84 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1504618/1807941 [16:44<01:57, 2592.06 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1504991/1807941 [16:44<01:58, 2557.68 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1505333/1807941 [16:44<02:02, 2467.30 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1505589/1807941 [16:45<02:01, 2486.24 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1505853/1807941 [16:45<01:59, 2522.34 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1506170/1807941 [16:45<02:06, 2377.78 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1506421/1807941 [16:45<02:05, 2409.82 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1506757/1807941 [16:45<02:08, 2345.44 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1506995/1807941 [16:45<02:08, 2348.18 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1507337/1807941 [16:45<02:10, 2311.08 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1507587/1807941 [16:45<02:07, 2356.40 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1507828/1807941 [16:46<02:06, 2368.09 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1508173/1807941 [16:46<02:08, 2339.62 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1508428/1807941 [16:46<02:05, 2391.41 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1508688/1807941 [16:46<02:02, 2445.10 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1509017/1807941 [16:46<02:08, 2332.09 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1509357/1807941 [16:46<02:09, 2307.94 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1509620/1807941 [16:46<02:05, 2386.34 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1509881/1807941 [16:46<02:02, 2424.49 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1510140/1807941 [16:46<02:04, 2397.86 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1510415/1807941 [16:47<01:59, 2491.74 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1510721/1807941 [16:47<01:52, 2648.16 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1511000/1807941 [16:47<01:52, 2634.08 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1511266/1807941 [16:47<01:52, 2636.41 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1511544/1807941 [16:47<01:50, 2675.39 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1511831/1807941 [16:47<01:48, 2731.36 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1512244/1807941 [16:47<01:48, 2734.05 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1512659/1807941 [16:47<01:47, 2741.22 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1513000/1807941 [16:48<01:54, 2577.47 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1513377/1807941 [16:48<01:55, 2555.44 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1513756/1807941 [16:48<01:56, 2531.19 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1514133/1807941 [16:48<01:59, 2466.63 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1514511/1807941 [16:48<01:58, 2471.51 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1514766/1807941 [16:48<01:57, 2486.10 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1515090/1807941 [16:48<02:03, 2376.67 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1515441/1807941 [16:49<02:03, 2363.39 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1515803/1807941 [16:49<02:02, 2376.19 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1516107/1807941 [16:49<02:08, 2267.40 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1516348/1807941 [16:49<02:06, 2297.37 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1516595/1807941 [16:49<02:04, 2336.72 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1516831/1807941 [16:49<02:04, 2342.43 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1517179/1807941 [16:49<02:05, 2316.21 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1517442/1807941 [16:49<02:01, 2386.74 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1517709/1807941 [16:50<01:59, 2435.74 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1518041/1807941 [16:50<02:03, 2351.53 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1518303/1807941 [16:50<01:59, 2417.92 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1518576/1807941 [16:50<01:55, 2498.87 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1518925/1807941 [16:50<01:59, 2421.78 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1519283/1807941 [16:50<01:59, 2406.26 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1519664/1807941 [16:50<01:57, 2445.69 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1520000/1807941 [16:51<02:02, 2342.64 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1520258/1807941 [16:51<02:00, 2395.77 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1520527/1807941 [16:51<01:56, 2467.61 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1520784/1807941 [16:51<01:55, 2493.99 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1521134/1807941 [16:51<01:58, 2429.88 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1521410/1807941 [16:51<01:54, 2510.09 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1521682/1807941 [16:51<01:51, 2562.54 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1522030/1807941 [16:51<01:55, 2466.64 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1522283/1807941 [16:51<01:55, 2482.43 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1522546/1807941 [16:52<01:53, 2518.99 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1522912/1807941 [16:52<01:54, 2486.72 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1523265/1807941 [16:52<01:57, 2431.60 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1523627/1807941 [16:52<01:57, 2423.39 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1523906/1807941 [16:52<01:53, 2509.68 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1524227/1807941 [16:52<01:59, 2383.19 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1524589/1807941 [16:52<01:58, 2388.65 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1524913/1807941 [16:53<02:02, 2313.84 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1525230/1807941 [16:53<02:06, 2229.88 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1525475/1807941 [16:53<02:04, 2277.02 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1525721/1807941 [16:53<02:01, 2320.30 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1525971/1807941 [16:53<01:59, 2362.82 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1526296/1807941 [16:53<02:03, 2289.73 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1526622/1807941 [16:53<02:05, 2246.28 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1526916/1807941 [16:53<02:11, 2142.05 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1527256/1807941 [16:54<02:08, 2179.75 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1527498/1807941 [16:54<02:05, 2228.55 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1527823/1807941 [16:54<02:06, 2206.80 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1528047/1807941 [16:54<02:06, 2212.05 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1528297/1807941 [16:54<02:02, 2281.55 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1528648/1807941 [16:54<02:01, 2292.36 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1528880/1807941 [16:54<02:02, 2285.42 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1529130/1807941 [16:54<02:01, 2297.76 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1529463/1807941 [16:55<02:02, 2267.90 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1529790/1807941 [16:55<02:04, 2236.49 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1530031/1807941 [16:55<02:02, 2277.23 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1530376/1807941 [16:55<02:02, 2272.42 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1530696/1807941 [16:55<02:05, 2211.92 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1530935/1807941 [16:55<02:03, 2251.56 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1531263/1807941 [16:55<02:04, 2225.00 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1531584/1807941 [16:55<02:06, 2186.17 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1531880/1807941 [16:56<02:10, 2111.97 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1532165/1807941 [16:56<02:14, 2043.44 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1532383/1807941 [16:56<02:13, 2067.95 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1532598/1807941 [16:56<02:12, 2085.57 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1532907/1807941 [16:56<02:12, 2074.15 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1533236/1807941 [16:56<02:10, 2110.33 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1533459/1807941 [16:56<02:09, 2117.92 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1533692/1807941 [16:57<02:06, 2169.23 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1533929/1807941 [16:57<02:03, 2220.53 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1534258/1807941 [16:57<02:03, 2208.44 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1534494/1807941 [16:57<02:01, 2244.20 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1534812/1807941 [16:57<02:04, 2196.09 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1535064/1807941 [16:57<01:59, 2276.02 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1535322/1807941 [16:57<01:55, 2353.55 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1535604/1807941 [16:57<01:49, 2479.03 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1535903/1807941 [16:57<01:43, 2618.95 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1536177/1807941 [16:58<01:42, 2652.78 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1536470/1807941 [16:58<01:39, 2730.78 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1536785/1807941 [16:58<01:35, 2852.72 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1537208/1807941 [16:58<01:35, 2837.84 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1537506/1807941 [16:58<01:34, 2875.54 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1537803/1807941 [16:58<01:33, 2899.58 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1538179/1807941 [16:58<01:38, 2752.18 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1538493/1807941 [16:58<01:47, 2517.99 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1538814/1807941 [16:59<01:52, 2382.70 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1539122/1807941 [16:59<02:01, 2213.06 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1539476/1807941 [16:59<01:59, 2256.01 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1539726/1807941 [16:59<01:56, 2307.39 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1540022/1807941 [16:59<02:02, 2194.77 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1540246/1807941 [16:59<02:01, 2200.97 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1540498/1807941 [16:59<01:57, 2269.44 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1540736/1807941 [16:59<01:56, 2295.29 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1541052/1807941 [17:00<02:00, 2218.55 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1541371/1807941 [17:00<02:02, 2168.39 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1541603/1807941 [17:00<02:00, 2204.79 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1541850/1807941 [17:00<01:57, 2267.59 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1542171/1807941 [17:00<01:59, 2219.20 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1542467/1807941 [17:00<02:06, 2106.56 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1542816/1807941 [17:00<02:01, 2174.53 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1543112/1807941 [17:01<02:08, 2067.15 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1543322/1807941 [17:01<02:07, 2072.35 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1543540/1807941 [17:01<02:06, 2097.21 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1543844/1807941 [17:01<02:08, 2061.33 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1544155/1807941 [17:01<02:08, 2051.74 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1544455/1807941 [17:01<02:10, 2017.70 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1544748/1807941 [17:01<02:12, 1988.01 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1544982/1807941 [17:01<02:08, 2051.89 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1545282/1807941 [17:02<02:09, 2031.80 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1545497/1807941 [17:02<02:08, 2041.90 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1545735/1807941 [17:02<02:03, 2124.72 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1546026/1807941 [17:02<02:07, 2056.45 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1546256/1807941 [17:02<02:03, 2116.36 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1546532/1807941 [17:02<01:54, 2283.05 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1546800/1807941 [17:02<01:49, 2386.95 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1547173/1807941 [17:02<01:47, 2418.62 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1547458/1807941 [17:02<01:43, 2528.06 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1547742/1807941 [17:03<01:39, 2611.07 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1548136/1807941 [17:03<01:40, 2579.03 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1548414/1807941 [17:03<01:38, 2627.49 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1548680/1807941 [17:03<01:38, 2634.52 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1549016/1807941 [17:03<01:44, 2489.35 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1549353/1807941 [17:03<01:48, 2380.97 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1549691/1807941 [17:03<01:50, 2331.16 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1550010/1807941 [17:04<01:53, 2262.78 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1550312/1807941 [17:04<01:57, 2183.56 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1550549/1807941 [17:04<01:55, 2219.49 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1550797/1807941 [17:04<01:53, 2272.65 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1551063/1807941 [17:04<02:02, 2099.55 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1551283/1807941 [17:04<02:01, 2115.22 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1551596/1807941 [17:04<02:02, 2093.39 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1551873/1807941 [17:04<02:07, 2009.12 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1552146/1807941 [17:05<02:11, 1944.78 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1552428/1807941 [17:05<02:13, 1920.45 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1552648/1807941 [17:05<02:08, 1980.98 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1552924/1807941 [17:05<02:13, 1913.50 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1553212/1807941 [17:05<02:14, 1895.29 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1553415/1807941 [17:05<02:13, 1908.00 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1553632/1807941 [17:05<02:09, 1969.40 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1553832/1807941 [17:05<02:08, 1976.57 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1554170/1807941 [17:06<02:02, 2071.48 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1554425/1807941 [17:06<01:56, 2177.93 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1554699/1807941 [17:06<01:48, 2323.40 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1554966/1807941 [17:06<01:44, 2416.05 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1555222/1807941 [17:06<01:43, 2451.90 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1555477/1807941 [17:06<01:41, 2478.24 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1555760/1807941 [17:06<01:38, 2560.30 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1556124/1807941 [17:06<01:41, 2469.62 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1556394/1807941 [17:06<01:39, 2528.08 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1556666/1807941 [17:07<01:37, 2578.49 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1556935/1807941 [17:07<01:36, 2607.83 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1557299/1807941 [17:07<01:38, 2533.78 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1557562/1807941 [17:07<01:37, 2556.67 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1557821/1807941 [17:07<01:37, 2561.89 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1558190/1807941 [17:07<01:39, 2519.36 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1558461/1807941 [17:07<01:37, 2566.26 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1558836/1807941 [17:07<01:38, 2539.26 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1559175/1807941 [17:08<01:42, 2432.12 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1559535/1807941 [17:08<01:42, 2419.55 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1559791/1807941 [17:08<01:41, 2451.45 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1560124/1807941 [17:08<01:44, 2371.66 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1560390/1807941 [17:08<01:41, 2437.42 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1560674/1807941 [17:08<01:37, 2537.59 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1560963/1807941 [17:08<01:34, 2617.77 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1561363/1807941 [17:08<01:34, 2610.40 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1561652/1807941 [17:09<01:31, 2680.84 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1561954/1807941 [17:09<01:28, 2769.33 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1562350/1807941 [17:09<01:30, 2716.85 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1562763/1807941 [17:09<01:29, 2725.21 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1563142/1807941 [17:09<01:32, 2656.44 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1563541/1807941 [17:09<01:32, 2655.35 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1563814/1807941 [17:09<01:31, 2669.98 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1564182/1807941 [17:10<01:34, 2590.56 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1564456/1807941 [17:10<01:32, 2623.61 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1564846/1807941 [17:10<01:33, 2613.09 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1565214/1807941 [17:10<01:34, 2559.72 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1565487/1807941 [17:10<01:33, 2598.83 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1565861/1807941 [17:10<01:34, 2561.18 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1566211/1807941 [17:10<01:37, 2483.88 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1566464/1807941 [17:10<01:36, 2489.62 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1566816/1807941 [17:11<01:38, 2438.25 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1567136/1807941 [17:11<01:43, 2323.91 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1567484/1807941 [17:11<01:43, 2320.64 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1567831/1807941 [17:11<01:43, 2313.76 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1568136/1807941 [17:11<01:47, 2224.52 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1568361/1807941 [17:11<01:47, 2225.41 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1568605/1807941 [17:11<01:45, 2276.44 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1568837/1807941 [17:11<01:44, 2284.51 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1569143/1807941 [17:12<01:48, 2196.30 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1569382/1807941 [17:12<01:46, 2243.83 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1569716/1807941 [17:12<01:46, 2232.85 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1569946/1807941 [17:12<02:30, 1581.82 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570136/1807941 [17:13<08:11, 483.69 examples/s] 
Tokenizing train dataset:  87%|████████▋ | 1570272/1807941 [17:14<11:38, 340.05 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570375/1807941 [17:15<14:11, 279.04 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570455/1807941 [17:16<16:04, 246.30 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570511/1807941 [17:16<17:27, 226.72 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570558/1807941 [17:16<18:46, 210.71 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570598/1807941 [17:16<19:45, 200.15 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570632/1807941 [17:17<20:32, 192.52 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570658/1807941 [17:17<21:04, 187.66 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570682/1807941 [17:17<22:00, 179.65 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570706/1807941 [17:17<22:43, 173.93 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570730/1807941 [17:17<23:23, 169.01 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570754/1807941 [17:18<24:03, 164.28 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570778/1807941 [17:18<24:32, 161.01 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570802/1807941 [17:18<24:47, 159.40 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570826/1807941 [17:18<25:08, 157.14 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570842/1807941 [17:18<25:22, 155.76 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570858/1807941 [17:18<25:32, 154.68 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570874/1807941 [17:18<25:39, 154.01 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570890/1807941 [17:18<25:39, 154.00 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570906/1807941 [17:19<25:37, 154.13 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570924/1807941 [17:19<25:17, 156.24 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570942/1807941 [17:19<24:55, 158.45 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570959/1807941 [17:19<24:42, 159.86 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570977/1807941 [17:19<24:34, 160.73 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570994/1807941 [17:19<24:40, 160.02 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571016/1807941 [17:19<32:25, 121.79 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571033/1807941 [17:19<30:19, 130.19 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571051/1807941 [17:20<28:32, 138.36 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571069/1807941 [17:20<27:11, 145.23 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571087/1807941 [17:20<26:19, 149.97 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571105/1807941 [17:20<25:41, 153.62 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571123/1807941 [17:20<24:58, 158.00 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571141/1807941 [17:20<24:39, 160.05 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571158/1807941 [17:20<24:33, 160.71 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571176/1807941 [17:20<24:16, 162.59 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571194/1807941 [17:20<24:04, 163.91 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571212/1807941 [17:21<24:00, 164.32 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571230/1807941 [17:21<23:58, 164.55 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571248/1807941 [17:21<23:54, 165.01 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571266/1807941 [17:21<23:47, 165.81 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571284/1807941 [17:21<23:40, 166.57 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571302/1807941 [17:21<23:30, 167.77 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571320/1807941 [17:21<23:34, 167.33 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571338/1807941 [17:21<23:39, 166.72 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571355/1807941 [17:21<24:01, 164.16 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571373/1807941 [17:21<23:44, 166.12 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571391/1807941 [17:22<23:38, 166.82 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571409/1807941 [17:22<23:42, 166.27 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571427/1807941 [17:22<23:32, 167.50 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571445/1807941 [17:22<23:28, 167.89 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571463/1807941 [17:22<23:30, 167.70 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571481/1807941 [17:22<23:21, 168.71 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571499/1807941 [17:22<23:32, 167.43 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571523/1807941 [17:22<24:11, 162.88 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571547/1807941 [17:23<24:40, 159.70 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571572/1807941 [17:23<24:50, 158.54 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571588/1807941 [17:23<24:48, 158.83 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571605/1807941 [17:23<24:49, 158.62 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571621/1807941 [17:23<24:52, 158.36 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571637/1807941 [17:23<24:57, 157.84 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571653/1807941 [17:23<24:57, 157.75 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571669/1807941 [17:23<25:04, 156.99 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571692/1807941 [17:23<25:45, 152.85 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571708/1807941 [17:24<25:44, 152.96 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571724/1807941 [17:24<25:35, 153.85 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571740/1807941 [17:24<25:27, 154.67 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571756/1807941 [17:24<25:29, 154.45 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571772/1807941 [17:24<25:34, 153.90 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571788/1807941 [17:24<25:39, 153.37 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571804/1807941 [17:24<25:44, 152.88 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571820/1807941 [17:24<25:35, 153.75 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571836/1807941 [17:24<25:40, 153.28 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571852/1807941 [17:25<25:44, 152.81 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571868/1807941 [17:25<25:48, 152.48 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571884/1807941 [17:25<25:47, 152.56 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571900/1807941 [17:25<25:40, 153.27 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571916/1807941 [17:25<25:38, 153.45 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571932/1807941 [17:25<25:39, 153.31 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571948/1807941 [17:25<25:38, 153.44 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571964/1807941 [17:25<25:38, 153.37 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571980/1807941 [17:25<25:29, 154.26 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571996/1807941 [17:25<25:26, 154.54 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572016/1807941 [17:26<35:02, 112.22 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572032/1807941 [17:26<32:24, 121.34 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572048/1807941 [17:26<30:33, 128.65 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572064/1807941 [17:26<29:06, 135.07 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572080/1807941 [17:26<28:06, 139.81 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572096/1807941 [17:26<27:19, 143.83 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572112/1807941 [17:26<26:47, 146.67 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572128/1807941 [17:26<26:27, 148.57 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572144/1807941 [17:27<26:02, 150.94 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572160/1807941 [17:27<25:54, 151.68 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572176/1807941 [17:27<25:43, 152.73 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572192/1807941 [17:27<25:39, 153.18 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572208/1807941 [17:27<25:37, 153.28 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572224/1807941 [17:27<25:31, 153.90 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572240/1807941 [17:27<25:35, 153.50 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572256/1807941 [17:27<25:23, 154.72 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572272/1807941 [17:27<25:20, 154.97 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572289/1807941 [17:27<25:11, 155.90 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572306/1807941 [17:28<25:00, 157.08 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572324/1807941 [17:28<24:54, 157.66 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572340/1807941 [17:28<25:20, 154.93 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572356/1807941 [17:28<25:18, 155.15 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572372/1807941 [17:28<25:21, 154.80 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572388/1807941 [17:28<25:14, 155.51 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572404/1807941 [17:28<25:21, 154.78 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572420/1807941 [17:28<25:16, 155.26 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572436/1807941 [17:28<25:22, 154.73 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572452/1807941 [17:29<25:25, 154.36 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572468/1807941 [17:29<25:21, 154.74 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572484/1807941 [17:29<25:24, 154.44 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572500/1807941 [17:29<25:21, 154.71 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572516/1807941 [17:29<25:28, 154.04 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572532/1807941 [17:29<25:36, 153.21 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572548/1807941 [17:29<25:29, 153.86 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572565/1807941 [17:29<25:13, 155.52 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572581/1807941 [17:29<25:11, 155.68 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572597/1807941 [17:29<25:06, 156.27 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572613/1807941 [17:30<25:01, 156.78 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572629/1807941 [17:30<24:58, 157.02 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572645/1807941 [17:30<25:05, 156.34 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572664/1807941 [17:30<23:54, 164.00 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572686/1807941 [17:30<22:04, 177.66 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572709/1807941 [17:30<20:42, 189.33 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572730/1807941 [17:30<20:16, 193.42 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572753/1807941 [17:30<19:38, 199.64 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572776/1807941 [17:30<19:12, 204.11 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572797/1807941 [17:31<19:20, 202.70 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572818/1807941 [17:31<19:26, 201.64 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572840/1807941 [17:31<19:10, 204.36 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572863/1807941 [17:31<18:46, 208.64 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572887/1807941 [17:31<18:15, 214.61 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572910/1807941 [17:31<18:02, 217.06 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572934/1807941 [17:31<17:38, 222.04 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572957/1807941 [17:31<17:30, 223.73 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572982/1807941 [17:31<17:13, 227.32 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573012/1807941 [17:32<22:39, 172.86 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573034/1807941 [17:32<21:33, 181.59 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573058/1807941 [17:32<20:12, 193.71 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573085/1807941 [17:32<18:25, 212.52 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573108/1807941 [17:32<18:08, 215.77 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573142/1807941 [17:32<18:04, 216.60 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573169/1807941 [17:32<17:07, 228.40 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573198/1807941 [17:32<16:09, 242.17 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573228/1807941 [17:32<15:20, 254.92 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573257/1807941 [17:33<15:01, 260.32 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573292/1807941 [17:33<15:59, 244.47 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573318/1807941 [17:33<15:54, 245.78 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573347/1807941 [17:33<15:17, 255.81 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573378/1807941 [17:33<14:34, 268.26 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573412/1807941 [17:33<13:40, 285.88 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573444/1807941 [17:33<13:18, 293.80 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573480/1807941 [17:33<14:31, 268.90 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573513/1807941 [17:34<15:39, 249.45 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573547/1807941 [17:34<16:21, 238.85 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573573/1807941 [17:34<16:15, 240.25 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573609/1807941 [17:34<16:30, 236.59 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573634/1807941 [17:34<16:33, 235.90 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573658/1807941 [17:34<16:44, 233.19 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573683/1807941 [17:34<16:38, 234.57 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573707/1807941 [17:34<16:45, 233.00 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573731/1807941 [17:35<16:48, 232.17 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573756/1807941 [17:35<16:43, 233.42 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573781/1807941 [17:35<16:36, 234.98 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573806/1807941 [17:35<16:23, 237.96 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573832/1807941 [17:35<16:19, 238.94 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573856/1807941 [17:35<16:28, 236.76 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573881/1807941 [17:35<16:25, 237.47 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573905/1807941 [17:35<16:33, 235.67 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573930/1807941 [17:35<16:28, 236.85 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573963/1807941 [17:36<17:11, 226.78 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573993/1807941 [17:36<18:20, 212.62 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574021/1807941 [17:36<22:49, 170.76 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574041/1807941 [17:36<22:09, 175.89 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574062/1807941 [17:36<21:20, 182.70 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574097/1807941 [17:36<17:34, 221.69 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574128/1807941 [17:36<16:14, 239.81 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574156/1807941 [17:36<15:44, 247.55 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574184/1807941 [17:37<15:29, 251.37 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574210/1807941 [17:37<15:27, 252.11 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574238/1807941 [17:37<15:12, 256.24 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574265/1807941 [17:37<15:13, 255.89 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574294/1807941 [17:37<14:50, 262.28 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574325/1807941 [17:37<14:17, 272.58 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574360/1807941 [17:37<13:17, 292.90 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574393/1807941 [17:37<12:52, 302.44 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574427/1807941 [17:37<12:32, 310.48 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574461/1807941 [17:37<12:18, 316.23 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574502/1807941 [17:38<11:24, 341.26 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574542/1807941 [17:38<12:39, 307.33 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574581/1807941 [17:38<13:30, 287.92 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574620/1807941 [17:38<14:02, 276.84 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574660/1807941 [17:38<14:25, 269.52 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574690/1807941 [17:38<14:09, 274.57 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574721/1807941 [17:38<13:49, 281.06 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574754/1807941 [17:38<13:22, 290.63 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574787/1807941 [17:39<13:06, 296.57 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574829/1807941 [17:39<13:35, 285.81 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574860/1807941 [17:39<13:27, 288.68 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574893/1807941 [17:39<13:07, 296.03 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574938/1807941 [17:39<13:14, 293.44 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574982/1807941 [17:39<13:27, 288.37 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575013/1807941 [17:39<16:30, 235.10 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575038/1807941 [17:40<16:26, 236.03 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575074/1807941 [17:40<16:37, 233.56 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575099/1807941 [17:40<16:29, 235.38 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575126/1807941 [17:40<16:01, 242.18 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575163/1807941 [17:40<16:08, 240.39 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575189/1807941 [17:40<16:05, 241.13 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575216/1807941 [17:40<15:55, 243.50 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575242/1807941 [17:40<15:53, 244.05 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575278/1807941 [17:41<16:14, 238.83 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575304/1807941 [17:41<16:05, 240.99 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575330/1807941 [17:41<15:52, 244.09 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575356/1807941 [17:41<15:53, 243.88 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575384/1807941 [17:41<15:33, 249.25 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575412/1807941 [17:41<15:07, 256.27 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575438/1807941 [17:41<15:22, 252.08 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575475/1807941 [17:41<15:39, 247.46 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575500/1807941 [17:41<15:45, 245.81 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575525/1807941 [17:42<15:46, 245.62 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575553/1807941 [17:42<15:19, 252.75 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575579/1807941 [17:42<15:26, 250.76 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575607/1807941 [17:42<15:07, 256.02 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575638/1807941 [17:42<14:29, 267.31 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575667/1807941 [17:42<14:18, 270.68 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575707/1807941 [17:42<14:34, 265.71 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575734/1807941 [17:42<14:35, 265.33 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575772/1807941 [17:43<15:06, 256.07 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575810/1807941 [17:43<15:27, 250.36 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575845/1807941 [17:43<16:02, 241.05 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575881/1807941 [17:43<16:14, 238.03 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575910/1807941 [17:43<15:41, 246.47 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575948/1807941 [17:43<15:40, 246.64 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575976/1807941 [17:43<15:21, 251.62 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576012/1807941 [17:44<18:25, 209.78 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576039/1807941 [17:44<17:27, 221.29 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576063/1807941 [17:44<17:20, 222.84 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576096/1807941 [17:44<17:42, 218.24 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576130/1807941 [17:44<17:41, 218.41 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576153/1807941 [17:44<17:49, 216.75 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576183/1807941 [17:44<18:25, 209.72 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576205/1807941 [17:44<18:30, 208.62 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576237/1807941 [17:45<18:44, 206.03 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576258/1807941 [17:45<18:56, 203.89 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576280/1807941 [17:45<18:47, 205.45 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576302/1807941 [17:45<18:36, 207.41 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576325/1807941 [17:45<18:23, 209.88 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576356/1807941 [17:45<18:48, 205.24 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576377/1807941 [17:45<18:56, 203.76 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576400/1807941 [17:45<18:34, 207.77 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576423/1807941 [17:46<18:20, 210.41 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576447/1807941 [17:46<17:54, 215.35 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576470/1807941 [17:46<17:54, 215.33 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576492/1807941 [17:46<17:59, 214.45 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576516/1807941 [17:46<17:40, 218.26 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576548/1807941 [17:46<18:08, 212.49 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576572/1807941 [17:46<17:51, 215.92 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576595/1807941 [17:46<17:41, 217.98 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576619/1807941 [17:46<17:30, 220.15 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576653/1807941 [17:47<17:35, 219.14 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576687/1807941 [17:47<17:37, 218.65 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576716/1807941 [17:47<16:28, 233.97 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576743/1807941 [17:47<15:59, 241.07 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576770/1807941 [17:47<15:43, 245.12 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576796/1807941 [17:47<15:44, 244.75 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576824/1807941 [17:47<15:21, 250.75 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576851/1807941 [17:47<15:05, 255.11 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576889/1807941 [17:48<15:19, 251.19 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576926/1807941 [17:48<15:39, 245.86 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576955/1807941 [17:48<15:10, 253.73 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576983/1807941 [17:48<14:56, 257.51 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577011/1807941 [17:48<18:56, 203.20 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577035/1807941 [17:48<18:18, 210.29 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577061/1807941 [17:48<17:29, 219.91 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577087/1807941 [17:48<16:47, 229.06 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577114/1807941 [17:49<16:15, 236.60 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577142/1807941 [17:49<15:42, 244.88 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577180/1807941 [17:49<15:43, 244.62 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577205/1807941 [17:49<15:56, 241.19 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577230/1807941 [17:49<15:52, 242.31 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577256/1807941 [17:49<15:41, 245.10 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577281/1807941 [17:49<15:47, 243.41 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577317/1807941 [17:49<16:07, 238.46 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577343/1807941 [17:49<15:54, 241.60 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577370/1807941 [17:50<15:45, 243.86 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577396/1807941 [17:50<15:44, 244.15 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577433/1807941 [17:50<15:48, 242.93 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577470/1807941 [17:50<16:01, 239.70 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577507/1807941 [17:50<16:05, 238.67 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577532/1807941 [17:50<16:04, 238.86 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577558/1807941 [17:50<15:58, 240.46 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577585/1807941 [17:50<15:34, 246.54 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577612/1807941 [17:51<15:15, 251.59 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577641/1807941 [17:51<14:49, 259.02 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577670/1807941 [17:51<14:28, 265.18 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577698/1807941 [17:51<14:23, 266.75 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577726/1807941 [17:51<14:14, 269.49 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577754/1807941 [17:51<14:10, 270.76 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577782/1807941 [17:51<14:08, 271.21 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577810/1807941 [17:51<14:13, 269.67 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577838/1807941 [17:51<14:07, 271.64 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577868/1807941 [17:51<13:57, 274.64 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577896/1807941 [17:52<14:03, 272.76 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577934/1807941 [17:52<14:43, 260.40 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577973/1807941 [17:52<14:54, 257.13 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578000/1807941 [17:52<18:16, 209.68 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578023/1807941 [17:52<18:08, 211.26 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578047/1807941 [17:52<17:35, 217.90 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578073/1807941 [17:52<17:05, 224.14 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578099/1807941 [17:53<16:34, 231.04 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578128/1807941 [17:53<15:38, 244.86 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578155/1807941 [17:53<15:21, 249.35 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578192/1807941 [17:53<15:35, 245.69 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578218/1807941 [17:53<15:29, 247.17 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578243/1807941 [17:53<15:35, 245.43 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578279/1807941 [17:53<15:56, 240.02 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578304/1807941 [17:53<16:01, 238.80 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578329/1807941 [17:53<15:59, 239.31 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578366/1807941 [17:54<16:12, 236.14 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578391/1807941 [17:54<16:07, 237.20 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578415/1807941 [17:54<16:19, 234.22 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578450/1807941 [17:54<17:39, 216.63 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578476/1807941 [17:54<17:01, 224.71 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578504/1807941 [17:54<16:05, 237.56 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578529/1807941 [17:54<16:03, 238.12 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578554/1807941 [17:54<15:53, 240.53 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578590/1807941 [17:55<16:03, 238.05 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578616/1807941 [17:55<15:56, 239.80 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578653/1807941 [17:55<15:57, 239.50 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578678/1807941 [17:55<15:51, 240.94 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578703/1807941 [17:55<15:51, 240.84 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578738/1807941 [17:55<16:16, 234.75 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578762/1807941 [17:55<16:16, 234.72 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578787/1807941 [17:55<16:05, 237.24 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578814/1807941 [17:55<15:35, 245.05 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578846/1807941 [17:56<14:40, 260.08 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578885/1807941 [17:56<14:59, 254.64 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578911/1807941 [17:56<15:04, 253.28 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578941/1807941 [17:56<14:29, 263.34 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578970/1807941 [17:56<14:15, 267.80 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579000/1807941 [17:56<17:09, 222.29 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579028/1807941 [17:56<16:23, 232.85 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579054/1807941 [17:56<15:57, 238.94 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579084/1807941 [17:57<15:07, 252.15 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579110/1807941 [17:57<15:05, 252.62 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579150/1807941 [17:57<13:05, 291.32 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579181/1807941 [17:57<12:58, 294.02 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579223/1807941 [17:57<13:32, 281.54 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579252/1807941 [17:57<13:32, 281.41 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579281/1807941 [17:57<13:37, 279.64 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579319/1807941 [17:57<14:15, 267.27 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579354/1807941 [17:58<15:06, 252.14 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579390/1807941 [17:58<15:35, 244.24 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579424/1807941 [17:58<16:04, 236.91 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579449/1807941 [17:58<16:06, 236.31 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579474/1807941 [17:58<16:07, 236.14 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579511/1807941 [17:58<16:10, 235.36 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579538/1807941 [17:58<15:53, 239.64 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579569/1807941 [17:58<15:01, 253.25 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579596/1807941 [17:59<14:54, 255.15 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579631/1807941 [17:59<15:40, 242.72 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579665/1807941 [17:59<16:19, 233.04 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579698/1807941 [17:59<16:51, 225.76 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579732/1807941 [17:59<17:05, 222.46 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579755/1807941 [17:59<17:10, 221.42 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579788/1807941 [17:59<17:36, 215.90 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579811/1807941 [18:00<17:38, 215.51 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579833/1807941 [18:00<17:42, 214.67 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579856/1807941 [18:00<17:38, 215.53 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579878/1807941 [18:00<17:37, 215.66 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579900/1807941 [18:00<17:42, 214.63 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579933/1807941 [18:00<17:46, 213.80 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579956/1807941 [18:00<17:32, 216.52 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579978/1807941 [18:00<17:39, 215.07 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580000/1807941 [18:00<20:13, 187.80 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580049/1807941 [18:01<14:26, 263.10 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580101/1807941 [18:01<11:32, 329.13 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580152/1807941 [18:01<10:05, 376.10 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580203/1807941 [18:01<09:13, 411.25 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580255/1807941 [18:01<08:38, 439.49 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580306/1807941 [18:01<08:17, 457.80 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580358/1807941 [18:01<08:00, 473.60 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580410/1807941 [18:01<07:48, 485.72 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580462/1807941 [18:01<07:39, 494.89 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580516/1807941 [18:02<07:28, 507.07 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580571/1807941 [18:02<07:21, 515.34 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580624/1807941 [18:02<07:19, 517.28 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580677/1807941 [18:02<07:18, 518.48 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580755/1807941 [18:02<07:21, 514.88 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580811/1807941 [18:02<07:11, 525.80 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580865/1807941 [18:02<07:10, 527.24 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580919/1807941 [18:02<07:10, 527.24 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580976/1807941 [18:02<07:03, 536.42 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581049/1807941 [18:03<07:58, 474.52 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581101/1807941 [18:03<07:49, 483.53 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581159/1807941 [18:03<07:27, 507.25 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581212/1807941 [18:03<07:23, 511.02 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581289/1807941 [18:03<07:26, 507.94 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581341/1807941 [18:03<07:26, 508.07 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581394/1807941 [18:03<07:23, 510.54 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581472/1807941 [18:03<07:24, 509.50 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581526/1807941 [18:03<07:19, 514.75 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581581/1807941 [18:04<07:14, 520.95 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581634/1807941 [18:04<07:14, 520.97 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581714/1807941 [18:04<07:12, 523.60 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581791/1807941 [18:04<07:16, 517.67 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581843/1807941 [18:04<07:18, 515.38 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581920/1807941 [18:04<07:20, 513.12 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1581975/1807941 [18:04<07:14, 519.77 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582052/1807941 [18:05<07:52, 478.46 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582106/1807941 [18:05<07:41, 489.85 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582160/1807941 [18:05<07:32, 499.46 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582211/1807941 [18:05<07:31, 499.54 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582266/1807941 [18:05<07:22, 510.49 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582319/1807941 [18:05<07:19, 513.03 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582399/1807941 [18:05<07:16, 516.50 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582453/1807941 [18:05<07:13, 519.69 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582507/1807941 [18:05<07:09, 524.47 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582562/1807941 [18:05<07:06, 528.41 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582615/1807941 [18:06<07:08, 525.63 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582693/1807941 [18:06<07:12, 520.57 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582746/1807941 [18:06<07:12, 520.52 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582824/1807941 [18:06<07:14, 518.32 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582878/1807941 [18:06<07:10, 522.62 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582932/1807941 [18:06<07:10, 522.56 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582985/1807941 [18:06<07:09, 523.31 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583051/1807941 [18:06<08:03, 464.85 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583105/1807941 [18:07<07:46, 482.16 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583156/1807941 [18:07<07:40, 488.08 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583207/1807941 [18:07<07:36, 492.10 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583259/1807941 [18:07<07:31, 498.15 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583311/1807941 [18:07<07:29, 499.27 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583363/1807941 [18:07<07:27, 502.33 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583417/1807941 [18:07<07:20, 509.17 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583470/1807941 [18:07<07:16, 513.90 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583524/1807941 [18:07<07:11, 520.57 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583579/1807941 [18:08<07:06, 526.29 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583635/1807941 [18:08<07:00, 534.04 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583717/1807941 [18:08<06:58, 536.36 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583773/1807941 [18:08<06:55, 539.11 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583830/1807941 [18:08<06:51, 545.05 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583886/1807941 [18:08<06:49, 546.48 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583943/1807941 [18:08<06:46, 550.76 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583999/1807941 [18:08<06:47, 549.88 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584061/1807941 [18:08<07:34, 492.71 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584117/1807941 [18:09<07:21, 506.54 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584176/1807941 [18:09<07:05, 525.89 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584234/1807941 [18:09<06:54, 540.25 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584294/1807941 [18:09<06:43, 554.88 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584355/1807941 [18:09<06:34, 567.33 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584420/1807941 [18:09<06:20, 586.73 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584481/1807941 [18:09<06:17, 592.08 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584565/1807941 [18:09<06:26, 578.66 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584643/1807941 [18:09<06:43, 553.54 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584727/1807941 [18:10<06:43, 553.11 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584808/1807941 [18:10<06:47, 547.08 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584890/1807941 [18:10<06:49, 544.65 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584970/1807941 [18:10<06:53, 538.92 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585025/1807941 [18:10<07:40, 484.07 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585097/1807941 [18:10<07:43, 480.49 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585146/1807941 [18:10<07:43, 481.19 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585197/1807941 [18:11<07:37, 487.27 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585246/1807941 [18:11<07:39, 484.88 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585295/1807941 [18:11<07:38, 485.77 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585347/1807941 [18:11<07:32, 491.78 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585398/1807941 [18:11<07:31, 493.14 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585448/1807941 [18:11<07:31, 492.75 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585499/1807941 [18:11<07:30, 493.39 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585550/1807941 [18:11<07:29, 494.62 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585602/1807941 [18:11<07:26, 498.26 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585652/1807941 [18:11<07:28, 495.80 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585702/1807941 [18:12<07:28, 495.52 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585755/1807941 [18:12<07:22, 502.20 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585807/1807941 [18:12<07:19, 505.66 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585861/1807941 [18:12<07:12, 513.82 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585916/1807941 [18:12<07:04, 523.22 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585970/1807941 [18:12<07:03, 523.91 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586027/1807941 [18:12<07:56, 465.83 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586081/1807941 [18:12<07:37, 484.55 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586137/1807941 [18:12<07:21, 502.53 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586193/1807941 [18:13<07:09, 515.88 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586247/1807941 [18:13<07:05, 521.20 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586302/1807941 [18:13<07:00, 527.25 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586358/1807941 [18:13<06:55, 533.06 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586434/1807941 [18:13<07:07, 517.79 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586510/1807941 [18:13<07:12, 511.47 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586565/1807941 [18:13<07:05, 520.47 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586624/1807941 [18:13<06:53, 535.44 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586702/1807941 [18:14<06:59, 527.37 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586774/1807941 [18:14<07:14, 508.49 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586844/1807941 [18:14<07:31, 490.03 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586912/1807941 [18:14<07:45, 475.15 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586986/1807941 [18:14<07:42, 477.90 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587050/1807941 [18:14<08:21, 440.46 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587097/1807941 [18:14<08:15, 446.06 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587145/1807941 [18:14<08:10, 450.11 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587212/1807941 [18:15<08:15, 445.02 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587278/1807941 [18:15<08:19, 441.58 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587324/1807941 [18:15<08:16, 444.36 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587369/1807941 [18:15<08:15, 444.74 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587414/1807941 [18:15<08:16, 444.15 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587461/1807941 [18:15<08:12, 447.38 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587514/1807941 [18:15<07:50, 468.53 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587562/1807941 [18:15<07:50, 468.60 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587610/1807941 [18:16<07:47, 471.51 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587679/1807941 [18:16<07:56, 461.96 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587749/1807941 [18:16<07:58, 460.03 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587796/1807941 [18:16<07:59, 459.49 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587863/1807941 [18:16<08:07, 451.08 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587914/1807941 [18:16<07:54, 463.40 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587961/1807941 [18:16<07:56, 461.46 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588024/1807941 [18:16<08:43, 420.33 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588075/1807941 [18:17<08:21, 438.68 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588127/1807941 [18:17<08:03, 454.91 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588194/1807941 [18:17<08:08, 449.41 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588240/1807941 [18:17<08:07, 451.04 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588309/1807941 [18:17<08:07, 450.75 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588355/1807941 [18:17<08:07, 450.48 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588402/1807941 [18:17<08:04, 452.97 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588448/1807941 [18:17<08:06, 451.28 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588494/1807941 [18:17<08:04, 452.85 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588542/1807941 [18:18<08:00, 456.60 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588589/1807941 [18:18<08:00, 456.64 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588635/1807941 [18:18<08:00, 456.45 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588681/1807941 [18:18<08:02, 454.33 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588728/1807941 [18:18<08:01, 455.60 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588774/1807941 [18:18<08:03, 453.62 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588821/1807941 [18:18<08:01, 455.30 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588888/1807941 [18:18<08:09, 447.45 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588933/1807941 [18:18<08:11, 445.44 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588979/1807941 [18:19<08:12, 444.41 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589043/1807941 [18:19<09:11, 396.87 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589090/1807941 [18:19<08:51, 411.66 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589135/1807941 [18:19<08:42, 418.97 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589181/1807941 [18:19<08:32, 426.54 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589228/1807941 [18:19<08:22, 435.40 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589274/1807941 [18:19<08:17, 439.42 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589320/1807941 [18:19<08:11, 445.09 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589371/1807941 [18:19<07:55, 460.04 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589440/1807941 [18:20<08:00, 454.43 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589508/1807941 [18:20<08:04, 450.85 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589557/1807941 [18:20<07:57, 457.78 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589605/1807941 [18:20<07:51, 463.31 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589658/1807941 [18:20<07:37, 476.68 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589709/1807941 [18:20<07:32, 482.46 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589759/1807941 [18:20<07:29, 484.87 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589831/1807941 [18:20<07:34, 479.85 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589900/1807941 [18:21<07:45, 468.60 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589967/1807941 [18:21<07:58, 455.89 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590023/1807941 [18:21<08:45, 414.62 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590070/1807941 [18:21<08:31, 425.76 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590117/1807941 [18:21<08:19, 436.19 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590166/1807941 [18:21<08:08, 446.17 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590213/1807941 [18:21<08:05, 448.34 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590261/1807941 [18:21<07:57, 456.12 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590309/1807941 [18:22<07:55, 457.97 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590379/1807941 [18:22<07:53, 459.32 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590428/1807941 [18:22<07:50, 462.51 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590477/1807941 [18:22<07:45, 467.13 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590525/1807941 [18:22<07:44, 468.10 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590586/1807941 [18:22<08:15, 439.01 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590642/1807941 [18:22<08:48, 410.95 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590697/1807941 [18:22<09:13, 392.22 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590752/1807941 [18:23<09:28, 382.28 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590800/1807941 [18:23<08:57, 403.70 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590849/1807941 [18:23<08:31, 424.46 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590899/1807941 [18:23<08:11, 441.67 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590952/1807941 [18:23<07:47, 464.09 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591000/1807941 [18:23<08:38, 418.70 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591051/1807941 [18:23<08:13, 439.86 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591103/1807941 [18:23<07:51, 459.76 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591172/1807941 [18:24<07:58, 452.95 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591222/1807941 [18:24<07:48, 462.38 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591275/1807941 [18:24<07:34, 476.97 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591330/1807941 [18:24<07:19, 492.54 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591404/1807941 [18:24<07:22, 489.35 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591454/1807941 [18:24<07:21, 490.38 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591504/1807941 [18:24<07:19, 492.47 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591556/1807941 [18:24<07:15, 497.41 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591606/1807941 [18:24<07:16, 495.05 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591658/1807941 [18:24<07:13, 499.10 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591734/1807941 [18:25<07:16, 495.76 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591789/1807941 [18:25<07:06, 506.34 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591840/1807941 [18:25<07:07, 504.94 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591915/1807941 [18:25<07:13, 498.23 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591990/1807941 [18:25<07:17, 493.30 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592049/1807941 [18:25<08:05, 444.97 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592098/1807941 [18:25<07:57, 451.91 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592149/1807941 [18:26<07:45, 463.78 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592199/1807941 [18:26<07:39, 469.91 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592249/1807941 [18:26<07:35, 473.54 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592302/1807941 [18:26<07:23, 486.11 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592356/1807941 [18:26<07:13, 496.77 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592426/1807941 [18:26<07:25, 484.17 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592498/1807941 [18:26<07:28, 480.39 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592568/1807941 [18:26<07:34, 473.47 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592639/1807941 [18:27<07:37, 470.35 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592710/1807941 [18:27<07:40, 467.55 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592757/1807941 [18:27<07:40, 467.56 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592805/1807941 [18:27<07:40, 466.81 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592875/1807941 [18:27<07:43, 464.46 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592922/1807941 [18:27<07:43, 463.51 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592970/1807941 [18:27<07:41, 465.87 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593022/1807941 [18:27<08:33, 418.41 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593067/1807941 [18:28<08:26, 424.61 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593114/1807941 [18:28<08:16, 432.77 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593160/1807941 [18:28<08:10, 437.69 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593207/1807941 [18:28<08:05, 442.44 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593252/1807941 [18:28<08:04, 442.86 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593298/1807941 [18:28<08:00, 446.26 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593344/1807941 [18:28<07:58, 448.28 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593391/1807941 [18:28<07:52, 453.63 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593440/1807941 [18:28<07:45, 460.96 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593487/1807941 [18:28<07:44, 461.37 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593535/1807941 [18:29<07:43, 462.94 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593582/1807941 [18:29<07:45, 460.12 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593630/1807941 [18:29<07:43, 462.35 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593679/1807941 [18:29<07:40, 464.95 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593726/1807941 [18:29<07:40, 465.26 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593774/1807941 [18:29<07:39, 466.56 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593821/1807941 [18:29<07:38, 466.57 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593868/1807941 [18:29<07:41, 463.74 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593915/1807941 [18:29<07:44, 460.90 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593963/1807941 [18:29<07:42, 462.30 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594023/1807941 [18:30<08:27, 421.89 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594068/1807941 [18:30<08:18, 429.07 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594115/1807941 [18:30<08:07, 438.88 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594164/1807941 [18:30<07:54, 450.25 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594212/1807941 [18:30<07:48, 456.69 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594261/1807941 [18:30<07:42, 461.63 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594328/1807941 [18:30<07:52, 452.24 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594377/1807941 [18:30<07:45, 458.90 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594429/1807941 [18:30<07:33, 471.16 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594494/1807941 [18:31<07:49, 454.86 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594561/1807941 [18:31<07:56, 447.35 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594614/1807941 [18:31<07:38, 465.07 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594683/1807941 [18:31<07:42, 460.98 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594747/1807941 [18:31<07:57, 446.11 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594795/1807941 [18:31<07:50, 452.94 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594848/1807941 [18:31<07:32, 471.37 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594896/1807941 [18:32<07:33, 469.57 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594945/1807941 [18:32<07:31, 471.62 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594995/1807941 [18:32<07:27, 475.85 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595046/1807941 [18:32<08:33, 414.59 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595090/1807941 [18:32<08:30, 416.94 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595134/1807941 [18:32<08:24, 421.77 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595178/1807941 [18:32<08:22, 423.22 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595227/1807941 [18:32<08:05, 438.53 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595273/1807941 [18:32<08:03, 439.82 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595319/1807941 [18:32<07:58, 443.90 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595366/1807941 [18:33<07:52, 449.93 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595414/1807941 [18:33<07:44, 457.34 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595461/1807941 [18:33<07:43, 458.53 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595509/1807941 [18:33<07:38, 463.35 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595561/1807941 [18:33<07:24, 477.40 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595611/1807941 [18:33<07:22, 480.26 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595679/1807941 [18:33<07:35, 465.70 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595747/1807941 [18:33<07:43, 457.67 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595814/1807941 [18:34<07:50, 450.43 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595873/1807941 [18:34<08:15, 427.64 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595939/1807941 [18:34<08:17, 426.24 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596000/1807941 [18:34<09:18, 379.45 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596044/1807941 [18:34<09:02, 390.63 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596088/1807941 [18:34<08:47, 401.40 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596134/1807941 [18:34<08:32, 412.92 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596179/1807941 [18:34<08:21, 421.92 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596223/1807941 [18:35<08:16, 426.12 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596269/1807941 [18:35<08:10, 431.72 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596314/1807941 [18:35<08:05, 435.75 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596362/1807941 [18:35<07:56, 444.32 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596411/1807941 [18:35<07:43, 456.30 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596460/1807941 [18:35<07:34, 465.36 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596509/1807941 [18:35<07:28, 471.14 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596559/1807941 [18:35<07:23, 476.58 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596608/1807941 [18:35<07:22, 477.46 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596657/1807941 [18:35<07:21, 478.57 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596725/1807941 [18:36<07:38, 461.17 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596772/1807941 [18:36<07:38, 461.06 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596820/1807941 [18:36<07:37, 461.21 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596887/1807941 [18:36<07:47, 451.41 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596952/1807941 [18:36<07:56, 442.45 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596997/1807941 [18:36<07:55, 443.85 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597046/1807941 [18:36<08:45, 401.00 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597090/1807941 [18:37<08:38, 406.58 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597137/1807941 [18:37<08:21, 420.08 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597181/1807941 [18:37<08:19, 422.31 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597231/1807941 [18:37<07:57, 441.34 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597282/1807941 [18:37<07:41, 456.69 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597332/1807941 [18:37<07:32, 465.14 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597383/1807941 [18:37<07:23, 474.28 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597434/1807941 [18:37<07:15, 483.64 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597487/1807941 [18:37<07:05, 494.23 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597539/1807941 [18:37<07:00, 499.92 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597590/1807941 [18:38<06:59, 501.47 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597641/1807941 [18:38<06:58, 502.24 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597692/1807941 [18:38<06:57, 503.11 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597746/1807941 [18:38<06:52, 510.06 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597798/1807941 [18:38<06:51, 510.63 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597865/1807941 [18:38<07:17, 479.77 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597914/1807941 [18:38<07:16, 481.67 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597963/1807941 [18:38<07:14, 483.31 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598019/1807941 [18:38<08:26, 414.43 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598083/1807941 [18:39<08:26, 414.07 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598127/1807941 [18:39<08:23, 416.80 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598175/1807941 [18:39<08:07, 430.56 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598224/1807941 [18:39<07:52, 444.22 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598291/1807941 [18:39<07:55, 441.05 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598338/1807941 [18:39<07:50, 445.83 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598406/1807941 [18:39<07:48, 447.08 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598454/1807941 [18:39<07:44, 451.25 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598503/1807941 [18:40<07:35, 459.48 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598555/1807941 [18:40<07:22, 472.86 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598605/1807941 [18:40<07:18, 477.73 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598656/1807941 [18:40<07:12, 484.06 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598727/1807941 [18:40<07:20, 475.16 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598796/1807941 [18:40<07:28, 465.83 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598858/1807941 [18:40<07:47, 446.76 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598905/1807941 [18:40<07:44, 450.49 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598953/1807941 [18:41<07:39, 454.89 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599000/1807941 [18:41<08:09, 426.51 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599053/1807941 [18:41<07:43, 450.70 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599103/1807941 [18:41<07:34, 459.34 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599154/1807941 [18:41<07:22, 472.13 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599209/1807941 [18:41<07:05, 491.13 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599281/1807941 [18:41<07:12, 482.80 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599331/1807941 [18:41<07:11, 483.08 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599404/1807941 [18:41<07:15, 478.95 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599453/1807941 [18:42<07:14, 479.74 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599529/1807941 [18:42<07:07, 487.63 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599602/1807941 [18:42<07:10, 484.39 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599654/1807941 [18:42<07:05, 489.30 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599727/1807941 [18:42<07:10, 483.36 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599778/1807941 [18:42<07:06, 487.63 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599849/1807941 [18:42<07:14, 478.71 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599919/1807941 [18:43<07:19, 472.81 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599967/1807941 [18:43<07:18, 473.94 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1600023/1807941 [18:43<08:05, 428.03 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600070/1807941 [18:43<07:58, 434.78 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600117/1807941 [18:43<07:52, 439.54 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600162/1807941 [18:43<07:50, 441.50 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600209/1807941 [18:43<07:43, 447.77 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600255/1807941 [18:43<07:41, 450.33 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600301/1807941 [18:43<07:39, 451.89 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600368/1807941 [18:44<07:44, 447.14 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600414/1807941 [18:44<07:42, 448.55 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600460/1807941 [18:44<07:43, 447.76 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600505/1807941 [18:44<07:44, 446.21 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600550/1807941 [18:44<07:48, 442.72 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600596/1807941 [18:44<07:46, 444.33 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600645/1807941 [18:44<07:37, 452.85 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600692/1807941 [18:44<07:37, 452.76 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600739/1807941 [18:44<07:37, 452.85 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600785/1807941 [18:45<07:38, 452.28 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600831/1807941 [18:45<07:37, 453.17 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600877/1807941 [18:45<07:36, 454.05 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600924/1807941 [18:45<07:34, 455.63 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600972/1807941 [18:45<07:30, 459.67 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601024/1807941 [18:45<08:24, 410.14 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601070/1807941 [18:45<08:11, 420.90 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601117/1807941 [18:45<07:59, 431.44 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601162/1807941 [18:45<07:54, 435.67 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601208/1807941 [18:45<07:50, 439.27 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601255/1807941 [18:46<07:44, 445.11 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601302/1807941 [18:46<07:39, 449.94 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601371/1807941 [18:46<07:39, 449.73 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601417/1807941 [18:46<07:39, 449.18 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601485/1807941 [18:46<07:41, 446.89 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601531/1807941 [18:46<07:40, 447.88 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601578/1807941 [18:46<07:37, 451.41 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601628/1807941 [18:46<07:26, 462.40 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601676/1807941 [18:46<07:24, 464.10 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601736/1807941 [18:47<07:51, 437.70 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601799/1807941 [18:47<08:03, 426.53 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601843/1807941 [18:47<08:00, 429.00 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601889/1807941 [18:47<07:51, 436.60 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601952/1807941 [18:47<08:03, 426.35 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602000/1807941 [18:47<08:59, 381.86 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602045/1807941 [18:47<08:42, 394.37 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602092/1807941 [18:48<08:19, 412.34 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602147/1807941 [18:48<07:40, 446.47 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602200/1807941 [18:48<07:21, 465.63 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602252/1807941 [18:48<07:11, 477.09 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602303/1807941 [18:48<07:05, 483.05 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602353/1807941 [18:48<07:05, 482.72 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602402/1807941 [18:48<07:05, 482.73 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602473/1807941 [18:48<07:11, 476.08 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602543/1807941 [18:48<07:17, 469.53 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602591/1807941 [18:49<07:17, 469.08 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602664/1807941 [18:49<07:14, 472.65 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602717/1807941 [18:49<07:03, 484.78 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602767/1807941 [18:49<07:02, 485.84 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602818/1807941 [18:49<07:00, 488.28 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602892/1807941 [18:49<07:03, 483.87 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602944/1807941 [18:49<06:57, 490.98 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602995/1807941 [18:49<06:54, 494.09 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603049/1807941 [18:50<07:46, 439.50 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603099/1807941 [18:50<07:33, 452.12 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603149/1807941 [18:50<07:23, 462.12 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603204/1807941 [18:50<07:02, 484.82 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603276/1807941 [18:50<07:06, 480.20 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603326/1807941 [18:50<07:04, 481.63 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603396/1807941 [18:50<07:12, 473.24 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603468/1807941 [18:50<07:13, 471.95 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603516/1807941 [18:50<07:13, 471.57 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603566/1807941 [18:51<07:08, 476.65 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603614/1807941 [18:51<07:11, 474.06 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603662/1807941 [18:51<07:12, 472.11 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603730/1807941 [18:51<07:20, 463.39 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603777/1807941 [18:51<07:20, 463.66 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603848/1807941 [18:51<07:19, 464.50 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603896/1807941 [18:51<07:19, 463.99 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603947/1807941 [18:51<07:10, 473.73 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603995/1807941 [18:52<07:13, 470.88 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1604048/1807941 [18:52<08:03, 421.32 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1604094/1807941 [18:52<07:54, 429.47 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1604141/1807941 [18:52<07:45, 438.20 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1604189/1807941 [18:52<07:36, 445.94 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1604239/1807941 [18:52<07:24, 458.05 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1604289/1807941 [18:52<07:14, 468.88 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1604340/1807941 [18:52<07:09, 473.67 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1604388/1807941 [18:52<07:08, 474.54 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1604459/1807941 [18:53<07:13, 469.48 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1604509/1807941 [18:53<07:10, 473.04 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1604563/1807941 [18:53<06:55, 488.99 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1604614/1807941 [18:53<06:53, 491.83 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1604665/1807941 [18:53<06:52, 493.16 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1604716/1807941 [18:53<06:51, 494.04 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1604768/1807941 [18:53<06:47, 498.44 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1604820/1807941 [18:53<06:46, 499.12 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1604875/1807941 [18:53<06:39, 508.89 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1604927/1807941 [18:53<06:38, 509.40 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1604979/1807941 [18:54<06:38, 509.52 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605051/1807941 [18:54<07:18, 462.92 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605101/1807941 [18:54<07:11, 470.14 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605152/1807941 [18:54<07:03, 478.99 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605201/1807941 [18:54<07:02, 479.64 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605252/1807941 [18:54<06:56, 486.43 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605302/1807941 [18:54<06:58, 484.73 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605353/1807941 [18:54<06:53, 490.44 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605410/1807941 [18:54<06:36, 510.71 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605466/1807941 [18:55<06:26, 523.56 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605519/1807941 [18:55<06:26, 523.17 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605576/1807941 [18:55<06:18, 534.56 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605634/1807941 [18:55<06:10, 545.47 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605692/1807941 [18:55<06:06, 551.74 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605774/1807941 [18:55<06:10, 545.45 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605830/1807941 [18:55<06:11, 544.53 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605888/1807941 [18:55<06:07, 549.61 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605951/1807941 [18:55<05:54, 569.80 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606031/1807941 [18:56<06:13, 540.39 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606094/1807941 [18:56<05:59, 561.18 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606156/1807941 [18:56<05:51, 574.09 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606217/1807941 [18:56<05:46, 582.89 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606279/1807941 [18:56<05:40, 592.85 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606339/1807941 [18:56<05:40, 592.07 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606427/1807941 [18:56<05:45, 583.70 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606515/1807941 [18:56<05:46, 581.70 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606604/1807941 [18:57<05:45, 583.07 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606666/1807941 [18:57<05:42, 587.79 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606746/1807941 [18:57<05:56, 564.66 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606823/1807941 [18:57<06:08, 545.15 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606899/1807941 [18:57<06:18, 530.93 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606979/1807941 [18:57<06:18, 531.17 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607048/1807941 [18:57<07:04, 473.68 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607100/1807941 [18:58<06:56, 481.65 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607150/1807941 [18:58<06:54, 484.27 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607204/1807941 [18:58<06:46, 494.34 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607258/1807941 [18:58<06:38, 503.07 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607335/1807941 [18:58<06:38, 503.65 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607412/1807941 [18:58<06:38, 503.41 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607467/1807941 [18:58<06:30, 512.76 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607544/1807941 [18:58<06:33, 508.95 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607599/1807941 [18:59<06:27, 516.67 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607652/1807941 [18:59<06:25, 519.45 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607705/1807941 [18:59<06:27, 516.79 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607757/1807941 [18:59<06:28, 515.59 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607834/1807941 [18:59<06:30, 511.96 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607890/1807941 [18:59<06:23, 521.76 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607948/1807941 [18:59<06:12, 536.72 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608026/1807941 [18:59<06:48, 489.29 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608078/1807941 [18:59<06:42, 496.01 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608129/1807941 [19:00<06:42, 497.04 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608184/1807941 [19:00<06:32, 508.63 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608238/1807941 [19:00<06:28, 513.87 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608293/1807941 [19:00<06:23, 521.16 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608367/1807941 [19:00<06:33, 506.96 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608442/1807941 [19:00<06:39, 499.24 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608493/1807941 [19:00<06:40, 498.38 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608569/1807941 [19:00<06:42, 495.76 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608621/1807941 [19:01<06:39, 499.20 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608673/1807941 [19:01<06:36, 502.03 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608747/1807941 [19:01<06:41, 495.62 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608818/1807941 [19:01<06:51, 484.34 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608890/1807941 [19:01<06:53, 481.39 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608963/1807941 [19:01<06:56, 477.70 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609024/1807941 [19:01<07:27, 444.45 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609076/1807941 [19:02<07:14, 458.17 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609125/1807941 [19:02<07:07, 465.19 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609177/1807941 [19:02<06:56, 477.43 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609227/1807941 [19:02<06:54, 479.36 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609279/1807941 [19:02<06:47, 487.51 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609330/1807941 [19:02<06:43, 492.09 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609381/1807941 [19:02<06:41, 494.90 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609453/1807941 [19:02<06:49, 484.89 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609503/1807941 [19:02<06:48, 485.79 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609574/1807941 [19:03<06:55, 477.77 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609646/1807941 [19:03<06:58, 473.90 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609717/1807941 [19:03<07:01, 470.42 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609766/1807941 [19:03<07:00, 471.43 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609814/1807941 [19:03<06:59, 472.52 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609862/1807941 [19:03<06:59, 472.63 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609912/1807941 [19:03<06:55, 476.07 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609961/1807941 [19:03<06:55, 476.30 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610022/1807941 [19:04<07:45, 425.34 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610069/1807941 [19:04<07:35, 434.83 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610116/1807941 [19:04<07:28, 441.21 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610166/1807941 [19:04<07:15, 454.14 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610213/1807941 [19:04<07:12, 457.12 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610261/1807941 [19:04<07:07, 461.88 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610310/1807941 [19:04<07:03, 467.14 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610359/1807941 [19:04<07:02, 467.51 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610408/1807941 [19:04<06:59, 471.32 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610459/1807941 [19:04<06:53, 477.72 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610507/1807941 [19:05<06:56, 474.16 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610556/1807941 [19:05<06:55, 475.02 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610606/1807941 [19:05<06:53, 477.46 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610654/1807941 [19:05<06:55, 475.16 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610703/1807941 [19:05<06:57, 472.02 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610776/1807941 [19:05<06:56, 473.77 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610825/1807941 [19:05<06:56, 473.31 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610873/1807941 [19:05<06:56, 472.82 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610926/1807941 [19:05<06:46, 484.23 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610975/1807941 [19:06<06:46, 484.87 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611025/1807941 [19:06<07:46, 422.43 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611071/1807941 [19:06<07:36, 431.26 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611118/1807941 [19:06<07:28, 438.77 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611165/1807941 [19:06<07:21, 445.62 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611213/1807941 [19:06<07:16, 450.26 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611263/1807941 [19:06<07:05, 462.24 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611313/1807941 [19:06<06:58, 470.04 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611385/1807941 [19:06<06:58, 469.77 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611433/1807941 [19:07<06:58, 469.62 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611484/1807941 [19:07<06:52, 476.61 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611533/1807941 [19:07<06:50, 478.14 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611585/1807941 [19:07<06:42, 487.77 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611638/1807941 [19:07<06:36, 495.29 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611709/1807941 [19:07<06:48, 480.55 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611760/1807941 [19:07<06:43, 485.84 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611810/1807941 [19:07<06:42, 486.89 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611863/1807941 [19:07<06:34, 496.93 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611916/1807941 [19:08<06:30, 501.41 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611991/1807941 [19:08<06:34, 496.28 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612048/1807941 [19:08<07:24, 440.37 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612114/1807941 [19:08<07:27, 437.82 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612162/1807941 [19:08<07:19, 445.03 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612211/1807941 [19:08<07:12, 452.38 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612259/1807941 [19:08<07:07, 457.70 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612307/1807941 [19:08<07:04, 461.38 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612358/1807941 [19:09<06:56, 469.96 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612426/1807941 [19:09<07:02, 463.02 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612476/1807941 [19:09<06:56, 469.64 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612545/1807941 [19:09<07:01, 463.77 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612597/1807941 [19:09<06:50, 476.33 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612652/1807941 [19:09<06:34, 494.90 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612723/1807941 [19:09<06:43, 483.64 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612794/1807941 [19:09<06:49, 476.29 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612844/1807941 [19:10<06:47, 478.88 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612914/1807941 [19:10<06:54, 470.83 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612965/1807941 [19:10<06:47, 478.82 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613024/1807941 [19:10<07:35, 428.19 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613070/1807941 [19:10<07:29, 433.77 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613119/1807941 [19:10<07:18, 444.60 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613169/1807941 [19:10<07:05, 458.28 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613221/1807941 [19:10<06:53, 470.81 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613270/1807941 [19:10<06:51, 472.89 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613322/1807941 [19:11<06:42, 483.57 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613391/1807941 [19:11<06:53, 470.12 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613461/1807941 [19:11<06:57, 465.33 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613509/1807941 [19:11<06:57, 465.38 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613557/1807941 [19:11<06:57, 465.44 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613606/1807941 [19:11<06:53, 469.83 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613655/1807941 [19:11<06:52, 471.37 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613703/1807941 [19:11<06:53, 470.20 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613752/1807941 [19:11<06:50, 473.57 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613801/1807941 [19:12<06:47, 476.32 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613852/1807941 [19:12<06:40, 484.90 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613904/1807941 [19:12<06:34, 492.10 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613957/1807941 [19:12<06:27, 499.97 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614023/1807941 [19:12<07:18, 442.10 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614069/1807941 [19:12<07:15, 444.89 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614116/1807941 [19:12<07:11, 449.37 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614164/1807941 [19:12<07:06, 454.76 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614214/1807941 [19:12<06:57, 464.15 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614264/1807941 [19:13<06:49, 472.82 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614315/1807941 [19:13<06:43, 480.40 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614365/1807941 [19:13<06:42, 481.09 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614414/1807941 [19:13<06:42, 480.79 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614465/1807941 [19:13<06:38, 485.75 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614515/1807941 [19:13<06:36, 488.27 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614564/1807941 [19:13<06:38, 485.83 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614613/1807941 [19:13<06:38, 485.53 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614662/1807941 [19:13<06:39, 484.28 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614711/1807941 [19:14<06:41, 481.25 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614760/1807941 [19:14<06:40, 482.37 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614832/1807941 [19:14<06:43, 478.54 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614883/1807941 [19:14<06:38, 484.71 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614933/1807941 [19:14<06:35, 488.16 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614983/1807941 [19:14<06:36, 486.69 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615049/1807941 [19:14<07:19, 439.12 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615099/1807941 [19:14<07:09, 448.85 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615145/1807941 [19:14<07:08, 449.53 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615191/1807941 [19:15<07:07, 450.77 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615259/1807941 [19:15<07:08, 449.51 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615327/1807941 [19:15<07:10, 447.52 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615395/1807941 [19:15<07:12, 444.92 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615441/1807941 [19:15<07:11, 445.94 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615491/1807941 [19:15<07:01, 456.29 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615538/1807941 [19:15<07:00, 457.34 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615584/1807941 [19:15<07:01, 456.86 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615652/1807941 [19:16<07:06, 451.18 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615720/1807941 [19:16<07:10, 446.66 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615769/1807941 [19:16<07:02, 454.37 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615818/1807941 [19:16<06:57, 459.78 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615866/1807941 [19:16<06:55, 462.06 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615913/1807941 [19:16<06:54, 463.15 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615962/1807941 [19:16<06:49, 469.16 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616023/1807941 [19:16<07:43, 414.32 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616069/1807941 [19:17<07:33, 423.35 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616117/1807941 [19:17<07:20, 435.22 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616164/1807941 [19:17<07:13, 442.35 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616212/1807941 [19:17<07:07, 448.38 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616260/1807941 [19:17<07:01, 455.13 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616308/1807941 [19:17<06:58, 457.62 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616357/1807941 [19:17<06:53, 463.86 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616408/1807941 [19:17<06:43, 474.11 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616458/1807941 [19:17<06:39, 479.73 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616511/1807941 [19:17<06:29, 491.94 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616561/1807941 [19:18<06:30, 489.82 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616631/1807941 [19:18<06:40, 477.44 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616701/1807941 [19:18<06:46, 470.26 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616771/1807941 [19:18<06:51, 464.09 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616821/1807941 [19:18<06:45, 470.90 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616873/1807941 [19:18<06:38, 479.17 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616925/1807941 [19:18<06:31, 487.52 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616998/1807941 [19:18<06:34, 483.88 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617047/1807941 [19:19<07:27, 426.25 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617094/1807941 [19:19<07:18, 435.14 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617144/1807941 [19:19<07:05, 448.53 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617193/1807941 [19:19<06:58, 456.22 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617243/1807941 [19:19<06:50, 464.79 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617292/1807941 [19:19<06:45, 469.58 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617342/1807941 [19:19<06:42, 473.50 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617390/1807941 [19:19<06:41, 474.36 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617440/1807941 [19:19<06:37, 479.63 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617491/1807941 [19:20<06:31, 486.93 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617545/1807941 [19:20<06:22, 497.99 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617595/1807941 [19:20<06:25, 494.08 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617669/1807941 [19:20<06:29, 487.89 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617740/1807941 [19:20<06:35, 480.37 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617811/1807941 [19:20<06:42, 472.59 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617881/1807941 [19:20<06:46, 467.31 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617928/1807941 [19:20<06:48, 465.19 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617978/1807941 [19:21<06:42, 471.53 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1618026/1807941 [19:21<07:24, 426.91 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1618075/1807941 [19:21<07:11, 440.42 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618124/1807941 [19:21<07:00, 451.85 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618171/1807941 [19:21<06:57, 454.21 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618237/1807941 [19:21<07:05, 445.71 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618283/1807941 [19:21<07:02, 448.64 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618333/1807941 [19:21<06:52, 459.40 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618383/1807941 [19:21<06:45, 466.92 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618431/1807941 [19:22<06:44, 468.11 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618497/1807941 [19:22<06:57, 453.58 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618544/1807941 [19:22<06:56, 454.55 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618590/1807941 [19:22<06:57, 453.37 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618636/1807941 [19:22<06:58, 452.02 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618682/1807941 [19:22<06:57, 452.92 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618729/1807941 [19:22<06:56, 454.18 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618776/1807941 [19:22<06:55, 455.17 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618822/1807941 [19:22<06:55, 455.03 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618870/1807941 [19:23<06:51, 459.53 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618918/1807941 [19:23<06:50, 460.90 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618984/1807941 [19:23<07:01, 448.30 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619042/1807941 [19:23<08:03, 391.04 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619087/1807941 [19:23<07:48, 402.71 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619132/1807941 [19:23<07:38, 412.13 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619177/1807941 [19:23<07:31, 418.28 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619220/1807941 [19:23<07:28, 421.02 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619265/1807941 [19:24<07:21, 426.96 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619310/1807941 [19:24<07:17, 431.34 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619374/1807941 [19:24<07:23, 424.71 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619424/1807941 [19:24<07:06, 441.74 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619470/1807941 [19:24<07:03, 445.23 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619518/1807941 [19:24<06:58, 449.92 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619586/1807941 [19:24<07:01, 447.34 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619633/1807941 [19:24<06:57, 451.14 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619700/1807941 [19:24<07:02, 445.51 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619766/1807941 [19:25<07:05, 442.33 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619811/1807941 [19:25<07:06, 441.59 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619856/1807941 [19:25<07:06, 441.00 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619901/1807941 [19:25<07:08, 439.21 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619946/1807941 [19:25<07:05, 441.37 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619992/1807941 [19:25<07:02, 444.66 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620045/1807941 [19:25<07:53, 396.72 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620089/1807941 [19:25<07:40, 407.60 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620134/1807941 [19:26<07:31, 415.89 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620179/1807941 [19:26<07:23, 423.67 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620228/1807941 [19:26<07:06, 440.11 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620286/1807941 [19:26<06:32, 478.08 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620339/1807941 [19:26<06:23, 489.15 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620390/1807941 [19:26<06:21, 491.59 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620441/1807941 [19:26<06:18, 495.07 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620493/1807941 [19:26<06:16, 498.18 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620544/1807941 [19:26<06:15, 498.75 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620617/1807941 [19:26<06:22, 489.36 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620667/1807941 [19:27<06:22, 489.28 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620736/1807941 [19:27<06:33, 475.40 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620788/1807941 [19:27<06:25, 485.67 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620859/1807941 [19:27<06:30, 478.83 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620911/1807941 [19:27<06:24, 486.17 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620983/1807941 [19:27<06:27, 482.28 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621047/1807941 [19:27<07:12, 432.10 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621095/1807941 [19:28<07:04, 439.66 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621143/1807941 [19:28<06:58, 446.01 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621191/1807941 [19:28<06:51, 453.47 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621240/1807941 [19:28<06:46, 459.18 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621288/1807941 [19:28<06:43, 463.13 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621337/1807941 [19:28<06:38, 468.65 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621386/1807941 [19:28<06:34, 473.03 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621440/1807941 [19:28<06:20, 489.77 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621493/1807941 [19:28<06:13, 499.08 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621568/1807941 [19:28<06:16, 495.65 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621619/1807941 [19:29<06:14, 497.14 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621669/1807941 [19:29<06:16, 494.94 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621720/1807941 [19:29<06:14, 496.80 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621771/1807941 [19:29<06:13, 498.43 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621822/1807941 [19:29<06:14, 496.81 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621873/1807941 [19:29<06:12, 499.18 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621924/1807941 [19:29<06:11, 500.46 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621975/1807941 [19:29<06:11, 501.01 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622026/1807941 [19:29<06:51, 451.95 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622082/1807941 [19:30<06:27, 479.50 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622139/1807941 [19:30<06:10, 502.15 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622199/1807941 [19:30<05:51, 528.04 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622262/1807941 [19:30<05:36, 552.24 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622323/1807941 [19:30<05:28, 564.89 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622380/1807941 [19:30<05:30, 560.99 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622461/1807941 [19:30<05:37, 549.28 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622518/1807941 [19:30<05:35, 552.63 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622596/1807941 [19:30<05:43, 539.56 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622670/1807941 [19:31<05:57, 518.38 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622744/1807941 [19:31<06:05, 506.96 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622814/1807941 [19:31<06:15, 492.73 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622886/1807941 [19:31<06:22, 484.09 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622957/1807941 [19:31<06:27, 476.96 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623025/1807941 [19:31<06:54, 445.93 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623075/1807941 [19:32<06:45, 455.51 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623126/1807941 [19:32<06:36, 466.15 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623178/1807941 [19:32<06:27, 477.14 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623230/1807941 [19:32<06:20, 485.67 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623280/1807941 [19:32<06:19, 486.47 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623331/1807941 [19:32<06:17, 489.54 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623381/1807941 [19:32<06:17, 489.34 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623434/1807941 [19:32<06:09, 498.85 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623487/1807941 [19:32<06:04, 506.71 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623540/1807941 [19:32<06:00, 511.08 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623615/1807941 [19:33<06:05, 504.28 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623685/1807941 [19:33<06:17, 487.58 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623761/1807941 [19:33<06:15, 490.94 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623813/1807941 [19:33<06:10, 496.59 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623885/1807941 [19:33<06:16, 489.44 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623957/1807941 [19:33<06:20, 482.98 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624023/1807941 [19:33<07:06, 431.57 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624068/1807941 [19:34<07:03, 434.03 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624114/1807941 [19:34<06:57, 439.95 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624161/1807941 [19:34<06:52, 445.18 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624210/1807941 [19:34<06:43, 455.82 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624258/1807941 [19:34<06:40, 458.29 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624305/1807941 [19:34<06:41, 457.73 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624373/1807941 [19:34<06:45, 452.57 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624420/1807941 [19:34<06:42, 455.94 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624469/1807941 [19:34<06:37, 461.44 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624516/1807941 [19:35<06:36, 462.18 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624586/1807941 [19:35<06:37, 460.84 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624635/1807941 [19:35<06:32, 466.78 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624685/1807941 [19:35<06:26, 473.81 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624736/1807941 [19:35<06:21, 480.52 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624789/1807941 [19:35<06:14, 489.36 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624839/1807941 [19:35<06:13, 489.90 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624889/1807941 [19:35<06:13, 490.37 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624942/1807941 [19:35<06:05, 500.32 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625000/1807941 [19:36<07:28, 408.35 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625060/1807941 [19:36<07:34, 402.07 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625111/1807941 [19:36<07:10, 424.21 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625161/1807941 [19:36<06:55, 440.00 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625214/1807941 [19:36<06:36, 461.31 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625266/1807941 [19:36<06:25, 474.46 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625318/1807941 [19:36<06:17, 483.45 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625369/1807941 [19:36<06:14, 488.09 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625421/1807941 [19:36<06:09, 493.32 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625473/1807941 [19:37<06:06, 497.52 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625524/1807941 [19:37<06:06, 497.26 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625576/1807941 [19:37<06:03, 501.38 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625630/1807941 [19:37<05:56, 511.75 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625683/1807941 [19:37<05:54, 513.89 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625735/1807941 [19:37<05:55, 512.81 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625791/1807941 [19:37<05:48, 523.32 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625848/1807941 [19:37<05:41, 533.04 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625905/1807941 [19:37<05:36, 540.70 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625984/1807941 [19:38<05:40, 533.81 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626050/1807941 [19:38<06:28, 468.38 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626101/1807941 [19:38<06:22, 475.00 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626154/1807941 [19:38<06:14, 485.19 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626207/1807941 [19:38<06:08, 492.57 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626259/1807941 [19:38<06:06, 495.36 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626311/1807941 [19:38<06:04, 498.19 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626385/1807941 [19:38<06:07, 493.79 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626457/1807941 [19:39<06:13, 485.74 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626506/1807941 [19:39<06:14, 484.13 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626577/1807941 [19:39<06:19, 478.41 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626626/1807941 [19:39<06:18, 478.95 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626676/1807941 [19:39<06:17, 480.23 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626750/1807941 [19:39<06:16, 480.77 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626805/1807941 [19:39<06:05, 495.95 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626860/1807941 [19:39<05:56, 507.96 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626913/1807941 [19:39<05:55, 509.45 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626967/1807941 [19:40<05:52, 513.61 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1627024/1807941 [19:40<06:44, 447.73 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1627073/1807941 [19:40<06:37, 455.19 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1627126/1807941 [19:40<06:21, 473.46 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627195/1807941 [19:40<06:27, 465.85 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627267/1807941 [19:40<06:27, 466.56 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627315/1807941 [19:40<06:28, 464.77 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627366/1807941 [19:40<06:23, 470.78 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627436/1807941 [19:41<06:26, 467.26 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627505/1807941 [19:41<06:32, 459.19 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627553/1807941 [19:41<06:31, 460.63 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627601/1807941 [19:41<06:29, 462.74 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627671/1807941 [19:41<06:30, 461.50 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627739/1807941 [19:41<06:35, 455.68 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627787/1807941 [19:41<06:32, 459.11 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627837/1807941 [19:41<06:27, 465.15 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627884/1807941 [19:42<06:27, 464.58 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627932/1807941 [19:42<06:27, 465.01 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628000/1807941 [19:42<07:12, 416.38 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628045/1807941 [19:42<07:08, 420.31 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628089/1807941 [19:42<07:05, 422.58 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628135/1807941 [19:42<06:57, 430.24 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628181/1807941 [19:42<06:51, 436.54 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628228/1807941 [19:42<06:43, 444.92 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628274/1807941 [19:43<06:44, 444.45 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628322/1807941 [19:43<06:37, 452.09 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628369/1807941 [19:43<06:36, 453.37 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628417/1807941 [19:43<06:30, 459.20 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628465/1807941 [19:43<06:28, 461.83 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628515/1807941 [19:43<06:21, 470.09 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628585/1807941 [19:43<06:26, 463.54 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628633/1807941 [19:43<06:26, 463.75 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628704/1807941 [19:43<06:25, 465.35 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628752/1807941 [19:44<06:24, 465.79 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628799/1807941 [19:44<06:26, 463.86 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628847/1807941 [19:44<06:26, 463.39 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628894/1807941 [19:44<06:27, 462.36 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628963/1807941 [19:44<06:31, 457.49 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629021/1807941 [19:44<07:23, 403.16 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629065/1807941 [19:44<07:17, 409.02 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629112/1807941 [19:44<07:02, 422.96 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629159/1807941 [19:44<06:52, 433.88 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629206/1807941 [19:45<06:45, 440.69 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629256/1807941 [19:45<06:32, 455.04 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629303/1807941 [19:45<06:31, 456.37 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629370/1807941 [19:45<06:38, 448.39 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629418/1807941 [19:45<06:33, 453.27 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629464/1807941 [19:45<06:34, 452.02 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629516/1807941 [19:45<06:21, 467.35 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629567/1807941 [19:45<06:14, 476.92 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629619/1807941 [19:45<06:06, 486.68 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629692/1807941 [19:46<06:09, 482.30 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629746/1807941 [19:46<06:01, 493.58 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629818/1807941 [19:46<06:07, 484.98 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629886/1807941 [19:46<06:19, 468.57 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629935/1807941 [19:46<06:18, 470.55 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629984/1807941 [19:46<06:16, 472.89 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630048/1807941 [19:46<06:52, 431.45 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630097/1807941 [19:46<06:40, 444.12 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630148/1807941 [19:47<06:28, 458.04 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630203/1807941 [19:47<06:10, 479.84 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630257/1807941 [19:47<06:01, 491.82 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630308/1807941 [19:47<05:58, 495.18 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630383/1807941 [19:47<06:00, 492.87 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630435/1807941 [19:47<05:56, 498.43 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630509/1807941 [19:47<05:59, 493.14 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630562/1807941 [19:47<05:55, 499.13 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630633/1807941 [19:48<06:03, 487.41 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630704/1807941 [19:48<06:09, 479.55 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630773/1807941 [19:48<06:17, 469.75 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630843/1807941 [19:48<06:20, 465.71 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630890/1807941 [19:48<06:22, 463.02 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630937/1807941 [19:48<06:23, 461.89 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631000/1807941 [19:48<07:01, 419.75 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631046/1807941 [19:49<06:54, 427.10 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631095/1807941 [19:49<06:42, 439.57 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631143/1807941 [19:49<06:34, 447.88 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631191/1807941 [19:49<06:27, 456.18 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631242/1807941 [19:49<06:16, 469.48 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631295/1807941 [19:49<06:06, 482.28 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631348/1807941 [19:49<05:56, 494.89 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631400/1807941 [19:49<05:52, 501.28 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631453/1807941 [19:49<05:47, 507.82 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631504/1807941 [19:49<05:47, 507.30 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631556/1807941 [19:50<05:46, 508.91 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631607/1807941 [19:50<05:48, 506.17 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631658/1807941 [19:50<05:49, 504.94 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631712/1807941 [19:50<05:43, 512.44 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631767/1807941 [19:50<05:38, 520.11 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631842/1807941 [19:50<05:45, 509.51 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631918/1807941 [19:50<05:49, 504.05 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631969/1807941 [19:50<05:49, 502.84 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632026/1807941 [19:51<06:24, 457.30 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632078/1807941 [19:51<06:13, 470.91 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632128/1807941 [19:51<06:08, 477.47 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632180/1807941 [19:51<06:02, 485.43 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632233/1807941 [19:51<05:56, 492.93 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632285/1807941 [19:51<05:52, 498.84 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632336/1807941 [19:51<05:52, 497.58 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632387/1807941 [19:51<05:53, 497.25 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632443/1807941 [19:51<05:42, 513.06 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632495/1807941 [19:51<05:43, 511.16 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632572/1807941 [19:52<05:43, 510.58 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632627/1807941 [19:52<05:38, 518.14 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632681/1807941 [19:52<05:36, 521.26 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632735/1807941 [19:52<05:35, 522.90 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632789/1807941 [19:52<05:33, 525.29 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632843/1807941 [19:52<05:32, 526.79 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632898/1807941 [19:52<05:30, 529.28 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632954/1807941 [19:52<05:27, 533.72 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633026/1807941 [19:52<06:10, 472.55 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633078/1807941 [19:53<06:02, 481.87 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633131/1807941 [19:53<05:55, 492.31 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633185/1807941 [19:53<05:48, 501.47 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633237/1807941 [19:53<05:46, 503.58 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633313/1807941 [19:53<05:47, 502.78 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633388/1807941 [19:53<05:50, 498.28 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633440/1807941 [19:53<05:49, 499.98 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633491/1807941 [19:53<05:49, 499.80 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633543/1807941 [19:53<05:46, 503.12 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633596/1807941 [19:54<05:43, 507.94 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633649/1807941 [19:54<05:41, 510.12 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633726/1807941 [19:54<05:42, 509.24 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633778/1807941 [19:54<05:41, 510.45 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633830/1807941 [19:54<05:42, 508.66 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633882/1807941 [19:54<05:41, 509.51 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633959/1807941 [19:54<05:41, 508.87 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634024/1807941 [19:54<06:22, 455.03 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634074/1807941 [19:55<06:16, 462.29 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634124/1807941 [19:55<06:09, 470.36 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634176/1807941 [19:55<06:01, 480.93 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634228/1807941 [19:55<05:54, 490.13 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634280/1807941 [19:55<05:50, 495.88 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634331/1807941 [19:55<05:48, 497.94 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634382/1807941 [19:55<05:49, 496.92 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634433/1807941 [19:55<05:47, 499.41 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634509/1807941 [19:55<05:48, 498.22 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634560/1807941 [19:56<05:48, 498.11 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634611/1807941 [19:56<05:48, 497.82 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634661/1807941 [19:56<05:49, 495.36 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634735/1807941 [19:56<05:52, 491.15 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634809/1807941 [19:56<05:55, 487.05 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634858/1807941 [19:56<05:56, 485.71 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634908/1807941 [19:56<05:56, 485.16 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634958/1807941 [19:56<05:54, 487.54 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635024/1807941 [19:57<06:34, 438.61 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635075/1807941 [19:57<06:20, 453.79 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635124/1807941 [19:57<06:15, 459.73 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635174/1807941 [19:57<06:08, 468.79 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635226/1807941 [19:57<06:00, 479.44 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635276/1807941 [19:57<05:56, 483.74 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635326/1807941 [19:57<05:54, 486.71 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635375/1807941 [19:57<05:55, 484.94 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635425/1807941 [19:57<05:54, 487.11 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635477/1807941 [19:57<05:50, 491.35 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635528/1807941 [19:58<05:49, 493.46 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635578/1807941 [19:58<05:48, 493.94 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635628/1807941 [19:58<05:48, 494.37 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635680/1807941 [19:58<05:45, 497.91 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635733/1807941 [19:58<05:40, 505.63 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635786/1807941 [19:58<05:36, 511.68 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635838/1807941 [19:58<05:37, 509.67 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635913/1807941 [19:58<05:41, 503.28 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635966/1807941 [19:58<05:38, 507.30 examples/s]
Tokenizing train dataset:  90%|█████████ | 1636025/1807941 [19:59<06:12, 460.93 examples/s]
Tokenizing train dataset:  90%|█████████ | 1636075/1807941 [19:59<06:06, 468.54 examples/s]
Tokenizing train dataset:  90%|█████████ | 1636127/1807941 [19:59<05:58, 478.79 examples/s]
Tokenizing train dataset:  90%|█████████ | 1636178/1807941 [19:59<05:54, 484.75 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636229/1807941 [19:59<05:49, 491.52 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636280/1807941 [19:59<05:48, 492.16 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636332/1807941 [19:59<05:45, 496.73 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636385/1807941 [19:59<05:41, 502.02 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636437/1807941 [19:59<05:40, 503.09 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636488/1807941 [20:00<05:40, 503.07 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636540/1807941 [20:00<05:39, 505.21 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636592/1807941 [20:00<05:38, 506.70 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636644/1807941 [20:00<05:38, 505.32 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636721/1807941 [20:00<05:39, 504.35 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636773/1807941 [20:00<05:38, 505.75 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636824/1807941 [20:00<05:39, 504.35 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636875/1807941 [20:00<05:39, 503.18 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636926/1807941 [20:00<05:39, 503.09 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636978/1807941 [20:00<05:39, 503.54 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637051/1807941 [20:01<06:11, 459.48 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637101/1807941 [20:01<06:06, 466.70 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637151/1807941 [20:01<06:00, 473.33 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637202/1807941 [20:01<05:54, 481.05 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637253/1807941 [20:01<05:50, 486.63 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637304/1807941 [20:01<05:47, 491.13 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637378/1807941 [20:01<05:48, 489.93 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637428/1807941 [20:01<05:49, 488.24 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637479/1807941 [20:02<05:46, 491.44 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637531/1807941 [20:02<05:43, 495.95 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637582/1807941 [20:02<05:42, 498.08 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637633/1807941 [20:02<05:42, 497.11 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637700/1807941 [20:02<05:58, 474.70 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637760/1807941 [20:02<06:21, 446.50 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637820/1807941 [20:02<06:36, 429.29 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637882/1807941 [20:02<06:45, 419.80 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637944/1807941 [20:03<06:50, 413.76 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638000/1807941 [20:03<07:39, 369.78 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638040/1807941 [20:03<07:33, 374.47 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638079/1807941 [20:03<07:30, 376.66 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638120/1807941 [20:03<07:24, 381.63 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638159/1807941 [20:03<07:23, 382.48 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638199/1807941 [20:03<07:19, 385.85 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638241/1807941 [20:03<07:10, 393.87 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638283/1807941 [20:04<07:03, 400.16 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638327/1807941 [20:04<06:57, 406.37 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638369/1807941 [20:04<06:55, 407.87 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638413/1807941 [20:04<06:48, 414.76 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638457/1807941 [20:04<06:43, 419.85 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638501/1807941 [20:04<06:42, 421.29 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638545/1807941 [20:04<06:39, 423.64 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638589/1807941 [20:04<06:37, 425.81 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638635/1807941 [20:04<06:32, 430.84 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638679/1807941 [20:04<06:33, 430.01 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638723/1807941 [20:05<06:34, 429.15 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638767/1807941 [20:05<06:34, 428.88 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638811/1807941 [20:05<06:32, 430.49 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638857/1807941 [20:05<06:28, 435.47 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638901/1807941 [20:05<06:28, 434.87 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638964/1807941 [20:05<06:38, 424.33 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639020/1807941 [20:05<07:43, 364.08 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639061/1807941 [20:05<07:34, 371.84 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639102/1807941 [20:06<07:25, 379.38 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639142/1807941 [20:06<07:19, 384.23 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639183/1807941 [20:06<07:15, 387.60 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639224/1807941 [20:06<07:11, 391.38 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639264/1807941 [20:06<07:09, 392.70 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639305/1807941 [20:06<07:06, 395.36 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639365/1807941 [20:06<07:09, 392.05 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639406/1807941 [20:06<07:06, 395.45 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639446/1807941 [20:06<07:07, 394.38 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639486/1807941 [20:06<07:08, 393.16 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639528/1807941 [20:07<07:04, 396.78 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639570/1807941 [20:07<06:58, 402.04 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639614/1807941 [20:07<06:51, 408.68 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639657/1807941 [20:07<06:46, 414.29 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639699/1807941 [20:07<06:46, 413.75 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639742/1807941 [20:07<06:43, 416.97 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639786/1807941 [20:07<06:39, 420.42 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639829/1807941 [20:07<06:38, 422.09 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639873/1807941 [20:07<06:35, 424.72 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639918/1807941 [20:07<06:31, 429.28 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639962/1807941 [20:08<06:31, 428.65 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640021/1807941 [20:08<07:28, 374.33 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640066/1807941 [20:08<07:10, 390.31 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640112/1807941 [20:08<06:52, 406.39 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640159/1807941 [20:08<06:39, 419.56 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640205/1807941 [20:08<06:31, 428.96 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640252/1807941 [20:08<06:22, 438.34 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640300/1807941 [20:08<06:15, 446.68 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640347/1807941 [20:09<06:11, 450.66 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640394/1807941 [20:09<06:07, 456.09 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640443/1807941 [20:09<06:01, 463.13 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640493/1807941 [20:09<05:56, 469.42 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640563/1807941 [20:09<05:59, 466.02 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640630/1807941 [20:09<06:06, 456.76 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640696/1807941 [20:09<06:13, 448.08 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640742/1807941 [20:09<06:12, 449.27 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640788/1807941 [20:09<06:10, 450.92 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640834/1807941 [20:10<06:10, 451.25 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640902/1807941 [20:10<06:10, 450.81 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640949/1807941 [20:10<06:09, 452.55 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641000/1807941 [20:10<06:59, 398.17 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641043/1807941 [20:10<06:54, 402.85 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641086/1807941 [20:10<06:49, 407.56 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641130/1807941 [20:10<06:42, 414.12 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641173/1807941 [20:10<06:40, 415.97 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641234/1807941 [20:11<06:47, 409.51 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641276/1807941 [20:11<06:47, 408.62 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641338/1807941 [20:11<06:48, 408.01 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641380/1807941 [20:11<06:48, 408.09 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641421/1807941 [20:11<06:50, 405.90 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641463/1807941 [20:11<06:51, 405.01 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641505/1807941 [20:11<06:49, 406.47 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641546/1807941 [20:11<06:50, 405.33 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641588/1807941 [20:11<06:49, 406.40 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641649/1807941 [20:12<06:51, 403.66 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641710/1807941 [20:12<06:56, 399.44 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641771/1807941 [20:12<06:55, 399.55 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641813/1807941 [20:12<06:54, 401.02 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641855/1807941 [20:12<06:53, 401.58 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641897/1807941 [20:12<06:52, 402.82 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641939/1807941 [20:12<06:50, 404.39 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641980/1807941 [20:12<06:52, 402.04 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642040/1807941 [20:13<07:54, 349.69 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642081/1807941 [20:13<07:39, 361.16 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642121/1807941 [20:13<07:28, 369.58 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642162/1807941 [20:13<07:17, 378.94 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642203/1807941 [20:13<07:09, 386.24 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642245/1807941 [20:13<07:03, 390.89 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642287/1807941 [20:13<07:00, 394.25 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642329/1807941 [20:13<06:56, 397.34 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642370/1807941 [20:13<06:54, 399.16 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642412/1807941 [20:14<06:50, 402.79 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642453/1807941 [20:14<06:50, 403.31 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642496/1807941 [20:14<06:47, 406.40 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642538/1807941 [20:14<06:45, 407.87 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642580/1807941 [20:14<06:44, 408.41 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642621/1807941 [20:14<06:47, 405.49 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642663/1807941 [20:14<06:48, 404.26 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642704/1807941 [20:14<06:48, 404.01 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642746/1807941 [20:14<06:47, 405.08 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642787/1807941 [20:14<06:49, 403.42 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642828/1807941 [20:15<06:50, 402.41 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642870/1807941 [20:15<06:47, 404.78 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642912/1807941 [20:15<06:47, 404.61 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642953/1807941 [20:15<06:49, 403.21 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642994/1807941 [20:15<06:49, 402.98 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643040/1807941 [20:15<07:54, 347.55 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643081/1807941 [20:15<07:38, 359.52 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643122/1807941 [20:15<07:24, 370.87 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643163/1807941 [20:15<07:14, 379.11 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643204/1807941 [20:16<07:07, 385.80 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643245/1807941 [20:16<07:03, 389.14 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643287/1807941 [20:16<06:57, 394.75 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643328/1807941 [20:16<06:57, 394.58 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643368/1807941 [20:16<07:00, 391.58 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643410/1807941 [20:16<06:56, 394.80 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643450/1807941 [20:16<06:56, 394.47 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643491/1807941 [20:16<06:54, 397.16 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643532/1807941 [20:16<06:52, 399.00 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643574/1807941 [20:16<06:50, 400.56 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643616/1807941 [20:17<06:49, 401.74 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643658/1807941 [20:17<06:45, 404.94 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643699/1807941 [20:17<06:50, 400.29 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643741/1807941 [20:17<06:46, 403.77 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643782/1807941 [20:17<06:48, 402.22 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643824/1807941 [20:17<06:47, 403.06 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643866/1807941 [20:17<06:44, 405.24 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643908/1807941 [20:17<06:44, 405.15 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643969/1807941 [20:17<06:45, 404.28 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644019/1807941 [20:18<07:38, 357.69 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644060/1807941 [20:18<07:25, 368.26 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644102/1807941 [20:18<07:14, 377.20 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644143/1807941 [20:18<07:06, 383.62 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644184/1807941 [20:18<07:02, 387.72 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644225/1807941 [20:18<06:58, 390.85 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644266/1807941 [20:18<06:56, 393.34 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644308/1807941 [20:18<06:49, 399.36 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644349/1807941 [20:18<06:50, 398.43 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644391/1807941 [20:19<06:47, 401.36 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644433/1807941 [20:19<06:44, 404.58 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644475/1807941 [20:19<06:41, 406.83 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644519/1807941 [20:19<06:37, 410.95 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644561/1807941 [20:19<06:38, 409.96 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644622/1807941 [20:19<06:41, 407.06 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644663/1807941 [20:19<06:41, 406.65 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644724/1807941 [20:19<06:45, 402.71 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644766/1807941 [20:19<06:41, 405.95 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644808/1807941 [20:20<06:41, 405.81 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644849/1807941 [20:20<06:42, 405.11 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644891/1807941 [20:20<06:38, 408.84 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644932/1807941 [20:20<06:39, 407.58 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644973/1807941 [20:20<06:40, 407.24 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645020/1807941 [20:20<07:43, 351.62 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645060/1807941 [20:20<07:30, 361.43 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645102/1807941 [20:20<07:15, 373.63 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645144/1807941 [20:20<07:05, 382.85 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645186/1807941 [20:21<06:58, 389.33 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645227/1807941 [20:21<06:53, 393.36 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645271/1807941 [20:21<06:44, 402.09 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645314/1807941 [20:21<06:39, 407.48 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645357/1807941 [20:21<06:35, 411.36 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645399/1807941 [20:21<06:33, 412.97 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645443/1807941 [20:21<06:29, 417.04 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645486/1807941 [20:21<06:28, 418.20 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645530/1807941 [20:21<06:24, 422.62 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645574/1807941 [20:21<06:22, 424.70 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645619/1807941 [20:22<06:19, 427.33 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645663/1807941 [20:22<06:18, 429.30 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645706/1807941 [20:22<06:18, 428.33 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645749/1807941 [20:22<06:19, 427.32 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645793/1807941 [20:22<06:20, 426.70 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645838/1807941 [20:22<06:16, 430.77 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645883/1807941 [20:22<06:15, 431.40 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645928/1807941 [20:22<06:14, 432.79 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645973/1807941 [20:22<06:12, 434.89 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646020/1807941 [20:23<07:22, 365.92 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646060/1807941 [20:23<07:12, 373.91 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646101/1807941 [20:23<07:04, 381.62 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646141/1807941 [20:23<06:58, 386.51 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646182/1807941 [20:23<06:54, 390.33 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646223/1807941 [20:23<06:50, 393.69 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646264/1807941 [20:23<06:47, 396.90 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646305/1807941 [20:23<06:46, 397.91 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646365/1807941 [20:23<06:48, 396.00 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646407/1807941 [20:24<06:45, 398.72 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646448/1807941 [20:24<06:44, 399.40 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646489/1807941 [20:24<06:46, 397.57 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646530/1807941 [20:24<06:45, 398.25 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646571/1807941 [20:24<06:43, 400.28 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646613/1807941 [20:24<06:40, 403.17 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646656/1807941 [20:24<06:35, 407.92 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646699/1807941 [20:24<06:31, 412.15 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646741/1807941 [20:24<06:29, 414.11 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646784/1807941 [20:24<06:27, 415.54 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646828/1807941 [20:25<06:23, 420.16 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646872/1807941 [20:25<06:21, 422.58 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646917/1807941 [20:25<06:17, 426.89 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646960/1807941 [20:25<06:17, 426.71 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647021/1807941 [20:25<07:09, 374.53 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647064/1807941 [20:25<06:57, 385.40 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647107/1807941 [20:25<06:47, 394.77 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647151/1807941 [20:25<06:37, 404.83 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647195/1807941 [20:25<06:30, 412.12 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647239/1807941 [20:26<06:26, 416.21 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647283/1807941 [20:26<06:21, 421.25 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647345/1807941 [20:26<06:27, 414.55 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647387/1807941 [20:26<06:29, 411.99 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647448/1807941 [20:26<06:35, 405.86 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647508/1807941 [20:26<06:38, 402.71 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647549/1807941 [20:26<06:39, 401.48 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647610/1807941 [20:27<06:39, 401.39 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647651/1807941 [20:27<06:38, 402.19 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647692/1807941 [20:27<06:39, 401.21 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647733/1807941 [20:27<06:37, 403.04 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647774/1807941 [20:27<06:36, 403.70 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647816/1807941 [20:27<06:36, 403.40 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647858/1807941 [20:27<06:35, 404.38 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647902/1807941 [20:27<06:28, 411.45 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647946/1807941 [20:27<06:23, 417.06 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647990/1807941 [20:27<06:20, 420.34 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648043/1807941 [20:28<07:07, 374.40 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648086/1807941 [20:28<06:51, 388.23 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648130/1807941 [20:28<06:39, 400.48 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648174/1807941 [20:28<06:29, 410.34 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648216/1807941 [20:28<06:27, 411.79 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648258/1807941 [20:28<06:29, 410.10 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648300/1807941 [20:28<06:30, 408.97 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648361/1807941 [20:28<06:33, 405.86 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648402/1807941 [20:28<06:34, 404.67 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648443/1807941 [20:29<06:34, 404.72 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648485/1807941 [20:29<06:32, 406.01 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648527/1807941 [20:29<06:30, 408.22 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648569/1807941 [20:29<06:28, 410.28 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648611/1807941 [20:29<06:26, 412.63 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648653/1807941 [20:29<06:27, 411.09 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648695/1807941 [20:29<06:28, 410.21 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648737/1807941 [20:29<06:27, 411.09 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648780/1807941 [20:29<06:25, 413.16 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648822/1807941 [20:29<06:25, 412.96 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648864/1807941 [20:30<06:24, 414.19 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648906/1807941 [20:30<06:26, 411.99 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648967/1807941 [20:30<06:31, 406.42 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649019/1807941 [20:30<07:36, 348.36 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649059/1807941 [20:30<07:25, 356.49 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649099/1807941 [20:30<07:15, 364.61 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649139/1807941 [20:30<07:08, 370.56 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649180/1807941 [20:30<07:01, 376.82 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649220/1807941 [20:31<06:56, 381.47 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649261/1807941 [20:31<06:52, 384.77 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649302/1807941 [20:31<06:50, 386.65 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649342/1807941 [20:31<06:49, 386.95 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649382/1807941 [20:31<06:50, 386.66 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649422/1807941 [20:31<06:48, 388.31 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649462/1807941 [20:31<06:45, 391.17 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649503/1807941 [20:31<06:43, 393.01 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649543/1807941 [20:31<06:43, 392.37 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649583/1807941 [20:31<06:41, 394.09 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649624/1807941 [20:32<06:39, 395.90 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649665/1807941 [20:32<06:40, 395.57 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649705/1807941 [20:32<06:41, 394.14 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649746/1807941 [20:32<06:38, 396.63 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1649788/1807941 [20:32<06:37, 398.22 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1649828/1807941 [20:32<06:37, 397.70 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1649869/1807941 [20:32<06:36, 399.06 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1649909/1807941 [20:32<06:37, 397.38 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1649949/1807941 [20:32<06:38, 396.21 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1649989/1807941 [20:33<06:38, 396.47 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650038/1807941 [20:33<07:46, 338.76 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650077/1807941 [20:33<07:31, 349.62 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650117/1807941 [20:33<07:19, 359.07 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650158/1807941 [20:33<07:05, 370.65 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650200/1807941 [20:33<06:51, 383.37 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650242/1807941 [20:33<06:42, 392.07 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650283/1807941 [20:33<06:38, 395.40 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650325/1807941 [20:33<06:35, 398.55 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650386/1807941 [20:34<06:34, 398.92 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650428/1807941 [20:34<06:32, 401.44 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650469/1807941 [20:34<06:31, 402.49 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650512/1807941 [20:34<06:27, 406.03 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650554/1807941 [20:34<06:25, 408.08 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650596/1807941 [20:34<06:25, 407.76 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650657/1807941 [20:34<06:30, 402.75 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650698/1807941 [20:34<06:31, 401.57 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650740/1807941 [20:34<06:30, 403.07 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650781/1807941 [20:35<06:30, 401.99 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650823/1807941 [20:35<06:30, 402.33 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650864/1807941 [20:35<06:31, 401.60 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650906/1807941 [20:35<06:29, 403.26 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650948/1807941 [20:35<06:26, 406.36 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650989/1807941 [20:35<06:27, 405.00 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651041/1807941 [20:35<07:26, 351.66 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651081/1807941 [20:35<07:11, 363.20 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651122/1807941 [20:35<06:59, 373.78 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651164/1807941 [20:36<06:49, 382.38 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651206/1807941 [20:36<06:40, 391.41 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651247/1807941 [20:36<06:36, 394.81 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651288/1807941 [20:36<06:34, 397.03 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651330/1807941 [20:36<06:30, 401.13 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651390/1807941 [20:36<06:33, 397.86 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651431/1807941 [20:36<06:32, 398.44 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651473/1807941 [20:36<06:31, 399.55 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651514/1807941 [20:36<06:30, 400.47 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651555/1807941 [20:37<06:31, 399.63 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651597/1807941 [20:37<06:28, 402.64 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651639/1807941 [20:37<06:28, 402.51 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651699/1807941 [20:37<06:35, 394.96 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651740/1807941 [20:37<06:34, 395.93 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651781/1807941 [20:37<06:32, 397.79 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651823/1807941 [20:37<06:28, 401.96 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651864/1807941 [20:37<06:28, 401.30 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651906/1807941 [20:37<06:26, 403.22 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651948/1807941 [20:37<06:23, 406.90 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651992/1807941 [20:38<06:16, 414.13 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652044/1807941 [20:38<07:01, 370.26 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652086/1807941 [20:38<06:51, 378.99 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652127/1807941 [20:38<06:44, 385.51 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652169/1807941 [20:38<06:38, 390.89 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652210/1807941 [20:38<06:35, 393.29 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652252/1807941 [20:38<06:33, 396.11 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652292/1807941 [20:38<06:32, 396.69 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652333/1807941 [20:38<06:32, 396.95 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652392/1807941 [20:39<06:35, 392.84 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652433/1807941 [20:39<06:33, 395.01 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652473/1807941 [20:39<06:34, 394.40 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652513/1807941 [20:39<06:34, 394.36 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652555/1807941 [20:39<06:29, 399.28 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652597/1807941 [20:39<06:26, 401.96 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652640/1807941 [20:39<06:20, 407.68 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652684/1807941 [20:39<06:14, 414.32 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652729/1807941 [20:39<06:08, 420.72 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652776/1807941 [20:40<05:57, 434.06 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652823/1807941 [20:40<05:51, 440.99 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652870/1807941 [20:40<05:46, 448.07 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652915/1807941 [20:40<05:48, 444.31 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652980/1807941 [20:40<05:54, 437.05 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653039/1807941 [20:40<06:53, 374.43 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653097/1807941 [20:40<06:52, 375.20 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653138/1807941 [20:40<06:47, 379.85 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653196/1807941 [20:41<06:02, 427.06 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653265/1807941 [20:41<05:14, 492.08 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653334/1807941 [20:41<04:45, 541.78 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653402/1807941 [20:41<04:26, 578.84 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653467/1807941 [20:41<04:18, 596.62 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653537/1807941 [20:41<04:07, 624.15 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653605/1807941 [20:41<04:03, 634.25 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653674/1807941 [20:41<03:58, 647.99 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653742/1807941 [20:41<03:55, 655.61 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653817/1807941 [20:41<03:46, 679.28 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653893/1807941 [20:42<03:40, 700.09 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653969/1807941 [20:42<03:34, 716.92 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1654070/1807941 [20:42<03:58, 643.82 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1654145/1807941 [20:42<03:50, 667.24 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1654254/1807941 [20:42<03:44, 685.49 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1654325/1807941 [20:42<03:42, 690.75 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1654424/1807941 [20:42<03:46, 676.96 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1654494/1807941 [20:42<03:45, 681.31 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1654595/1807941 [20:43<03:47, 675.17 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1654679/1807941 [20:43<03:35, 712.42 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1654782/1807941 [20:43<03:39, 696.44 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1654857/1807941 [20:43<03:37, 705.09 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1654940/1807941 [20:43<03:28, 734.44 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1655036/1807941 [20:43<03:40, 692.00 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1655107/1807941 [20:43<03:40, 694.26 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1655180/1807941 [20:43<03:37, 701.88 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1655256/1807941 [20:44<03:33, 716.74 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1655332/1807941 [20:44<03:30, 726.37 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1655406/1807941 [20:44<03:29, 727.66 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1655515/1807941 [20:44<03:30, 722.94 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1655621/1807941 [20:44<03:33, 714.87 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1655729/1807941 [20:44<03:33, 713.80 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1655803/1807941 [20:44<03:31, 717.73 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1655877/1807941 [20:44<03:31, 720.53 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1655952/1807941 [20:45<03:29, 725.65 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1656048/1807941 [20:45<03:39, 691.26 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1656129/1807941 [20:45<03:30, 721.20 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1656211/1807941 [20:45<03:23, 745.16 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1656324/1807941 [20:45<03:23, 745.33 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1656403/1807941 [20:45<03:21, 752.97 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1656513/1807941 [20:45<03:23, 742.68 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1656588/1807941 [20:45<03:23, 743.81 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1656664/1807941 [20:45<03:23, 744.48 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1656773/1807941 [20:46<03:25, 734.05 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1656873/1807941 [20:46<03:33, 708.27 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1656979/1807941 [20:46<03:34, 704.09 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657064/1807941 [20:46<03:58, 632.36 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657136/1807941 [20:46<03:52, 648.83 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657209/1807941 [20:46<03:46, 665.36 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657279/1807941 [20:46<03:44, 672.07 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657348/1807941 [20:47<03:43, 673.89 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657417/1807941 [20:47<03:42, 675.38 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657507/1807941 [20:47<03:55, 639.73 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657591/1807941 [20:47<04:34, 546.85 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657654/1807941 [20:47<04:56, 506.39 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657717/1807941 [20:47<05:14, 477.65 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657782/1807941 [20:47<05:26, 459.68 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657845/1807941 [20:48<05:37, 444.11 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657905/1807941 [20:48<05:49, 429.13 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657965/1807941 [20:48<06:00, 416.59 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658033/1807941 [20:48<06:14, 400.16 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658099/1807941 [20:48<05:31, 452.24 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658166/1807941 [20:48<04:58, 501.10 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658231/1807941 [20:48<04:39, 535.73 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658315/1807941 [20:49<04:36, 540.62 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658379/1807941 [20:49<04:25, 563.60 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658466/1807941 [20:49<04:24, 565.17 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658535/1807941 [20:49<04:10, 595.27 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658626/1807941 [20:49<04:09, 597.66 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658710/1807941 [20:49<04:17, 580.11 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658772/1807941 [20:49<04:13, 587.66 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658843/1807941 [20:49<04:01, 616.95 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658937/1807941 [20:50<04:00, 618.68 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659007/1807941 [20:50<04:23, 564.56 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659092/1807941 [20:50<04:24, 562.88 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659152/1807941 [20:50<04:21, 569.37 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659213/1807941 [20:50<04:18, 576.14 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659282/1807941 [20:50<04:06, 604.30 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659370/1807941 [20:50<04:09, 594.43 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659433/1807941 [20:50<04:06, 601.64 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659497/1807941 [20:51<04:03, 608.67 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659583/1807941 [20:51<04:10, 591.23 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659644/1807941 [20:51<04:09, 594.68 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659721/1807941 [20:51<04:23, 562.15 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659784/1807941 [20:51<04:51, 508.68 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659842/1807941 [20:51<05:20, 462.34 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659893/1807941 [20:51<05:53, 418.79 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659946/1807941 [20:52<06:14, 394.70 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660000/1807941 [20:52<06:55, 356.04 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660037/1807941 [20:52<06:55, 355.70 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660075/1807941 [20:52<06:52, 358.68 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660113/1807941 [20:52<06:48, 362.31 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660151/1807941 [20:52<06:47, 363.07 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660189/1807941 [20:52<06:45, 364.07 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660227/1807941 [20:52<06:43, 366.36 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660266/1807941 [20:52<06:41, 368.08 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660305/1807941 [20:53<06:38, 370.04 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660344/1807941 [20:53<06:35, 372.88 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660385/1807941 [20:53<06:26, 381.30 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660427/1807941 [20:53<06:19, 389.03 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660469/1807941 [20:53<06:12, 395.94 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660509/1807941 [20:53<06:13, 394.34 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660550/1807941 [20:53<06:14, 393.32 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660619/1807941 [20:53<05:08, 477.80 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660685/1807941 [20:53<04:37, 530.55 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660790/1807941 [20:53<03:36, 680.07 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660879/1807941 [20:54<03:18, 740.16 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660961/1807941 [20:54<03:13, 761.42 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1661038/1807941 [20:54<03:29, 699.80 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1661126/1807941 [20:54<03:16, 746.56 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1661211/1807941 [20:54<03:09, 773.97 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1661316/1807941 [20:54<03:18, 737.05 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1661422/1807941 [20:54<03:23, 721.36 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1661534/1807941 [20:54<03:21, 725.66 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1661628/1807941 [20:55<03:32, 689.78 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1661738/1807941 [20:55<03:28, 700.77 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1661846/1807941 [20:55<03:27, 703.31 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1661955/1807941 [20:55<03:26, 708.45 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1662035/1807941 [20:55<03:44, 651.30 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1662115/1807941 [20:55<03:34, 680.25 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1662188/1807941 [20:55<03:31, 690.01 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1662300/1807941 [20:56<03:26, 704.79 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1662382/1807941 [20:56<03:20, 726.62 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1662466/1807941 [20:56<03:13, 750.12 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1662545/1807941 [20:56<03:11, 757.41 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1662652/1807941 [20:56<03:17, 736.61 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1662747/1807941 [20:56<03:27, 699.03 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1662829/1807941 [20:56<03:19, 727.20 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1662921/1807941 [20:56<03:07, 771.52 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1663007/1807941 [20:57<03:27, 699.75 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1663110/1807941 [20:57<03:29, 691.46 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1663205/1807941 [20:57<03:38, 662.27 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1663278/1807941 [20:57<03:33, 676.64 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1663378/1807941 [20:57<03:36, 666.95 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1663455/1807941 [20:57<03:29, 688.76 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1663534/1807941 [20:57<03:23, 711.10 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1663608/1807941 [20:57<03:24, 704.55 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1663680/1807941 [20:58<03:24, 705.45 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1663760/1807941 [20:58<03:17, 728.99 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1663836/1807941 [20:58<03:16, 732.59 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1663944/1807941 [20:58<03:19, 722.62 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1664039/1807941 [20:58<03:31, 680.34 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1664118/1807941 [20:58<03:24, 701.81 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1664199/1807941 [20:58<03:17, 726.94 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1664300/1807941 [20:58<03:25, 699.83 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1664382/1807941 [20:59<03:17, 727.29 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1664463/1807941 [20:59<03:12, 745.10 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1664543/1807941 [20:59<03:09, 756.31 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1664622/1807941 [20:59<03:07, 764.18 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1664733/1807941 [20:59<03:12, 744.69 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1664811/1807941 [20:59<03:10, 752.90 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1664891/1807941 [20:59<03:07, 763.14 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1664969/1807941 [20:59<03:07, 762.96 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1665073/1807941 [20:59<03:29, 682.99 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1665149/1807941 [21:00<03:25, 696.35 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1665264/1807941 [21:00<03:19, 715.14 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1665346/1807941 [21:00<03:13, 737.57 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1665422/1807941 [21:00<03:13, 736.70 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1665504/1807941 [21:00<03:08, 755.43 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1665625/1807941 [21:00<03:06, 762.20 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1665710/1807941 [21:00<03:02, 780.96 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1665833/1807941 [21:00<03:01, 784.49 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1665919/1807941 [21:01<02:58, 797.83 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1666000/1807941 [21:01<03:10, 743.72 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1666112/1807941 [21:01<03:11, 742.44 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1666226/1807941 [21:01<03:10, 745.73 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1666317/1807941 [21:01<03:01, 779.00 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1666409/1807941 [21:01<02:53, 813.87 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1666526/1807941 [21:01<02:57, 797.12 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1666641/1807941 [21:01<03:02, 776.28 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1666752/1807941 [21:02<03:05, 760.84 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1666854/1807941 [21:02<03:13, 729.12 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1666950/1807941 [21:02<03:21, 699.07 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1667043/1807941 [21:02<03:29, 672.94 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1667132/1807941 [21:02<03:15, 718.87 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1667213/1807941 [21:02<03:11, 736.51 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1667322/1807941 [21:02<03:12, 732.12 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1667397/1807941 [21:03<03:11, 734.07 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1667489/1807941 [21:03<03:00, 780.01 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1667606/1807941 [21:03<03:00, 775.45 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1667713/1807941 [21:03<03:10, 736.37 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1667795/1807941 [21:03<03:06, 750.24 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1667874/1807941 [21:03<03:04, 760.22 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1667976/1807941 [21:03<03:12, 727.61 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1668077/1807941 [21:03<03:27, 675.21 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1668176/1807941 [21:04<03:32, 657.58 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1668282/1807941 [21:04<03:28, 669.13 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1668362/1807941 [21:04<03:22, 689.55 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1668446/1807941 [21:04<03:13, 720.80 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1668521/1807941 [21:04<03:12, 726.02 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1668604/1807941 [21:04<03:06, 747.28 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1668721/1807941 [21:04<03:04, 753.50 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1668797/1807941 [21:04<03:05, 749.47 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1668884/1807941 [21:05<02:59, 776.25 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1669000/1807941 [21:05<03:09, 732.90 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1669087/1807941 [21:05<03:02, 762.23 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1669190/1807941 [21:05<03:10, 729.90 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1669270/1807941 [21:05<03:06, 743.67 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1669364/1807941 [21:05<03:18, 699.38 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1669446/1807941 [21:05<03:10, 726.15 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1669557/1807941 [21:06<03:12, 720.70 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1669639/1807941 [21:06<03:06, 739.99 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1669715/1807941 [21:06<03:06, 739.96 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1669832/1807941 [21:06<03:04, 748.97 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1669933/1807941 [21:06<03:11, 720.79 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1670036/1807941 [21:06<03:19, 691.07 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1670112/1807941 [21:06<03:15, 705.99 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1670190/1807941 [21:06<03:10, 722.25 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1670270/1807941 [21:06<03:06, 739.61 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1670356/1807941 [21:07<02:58, 769.69 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1670474/1807941 [21:07<02:58, 769.27 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1670559/1807941 [21:07<02:54, 787.41 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1670684/1807941 [21:07<02:51, 799.94 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1670775/1807941 [21:07<02:45, 826.30 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1670867/1807941 [21:07<02:41, 848.47 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1670956/1807941 [21:07<02:40, 853.65 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1671063/1807941 [21:07<02:52, 794.49 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1671179/1807941 [21:08<02:57, 768.96 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1671258/1807941 [21:08<02:57, 771.78 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1671358/1807941 [21:08<03:05, 734.58 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1671468/1807941 [21:08<03:07, 727.86 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1671544/1807941 [21:08<03:06, 731.15 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1671650/1807941 [21:08<03:09, 718.51 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1671753/1807941 [21:08<03:12, 706.20 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1671858/1807941 [21:09<03:14, 700.91 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1671931/1807941 [21:09<03:13, 703.60 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1672043/1807941 [21:09<03:14, 697.19 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1672123/1807941 [21:09<03:09, 716.34 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1672212/1807941 [21:09<03:01, 748.84 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1672288/1807941 [21:09<03:00, 750.59 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1672402/1807941 [21:09<03:01, 748.61 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1672487/1807941 [21:09<02:56, 768.29 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1672568/1807941 [21:10<02:53, 778.75 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1672649/1807941 [21:10<02:52, 785.93 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1672732/1807941 [21:10<02:50, 793.91 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1672844/1807941 [21:10<02:55, 769.92 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1672924/1807941 [21:10<02:55, 770.74 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1673044/1807941 [21:10<03:04, 732.85 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1673123/1807941 [21:10<03:01, 742.77 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1673203/1807941 [21:10<02:59, 751.58 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1673285/1807941 [21:10<02:57, 757.94 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1673410/1807941 [21:11<02:52, 779.02 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1673526/1807941 [21:11<02:53, 775.54 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1673616/1807941 [21:11<02:48, 797.91 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1673737/1807941 [21:11<02:48, 797.76 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1673846/1807941 [21:11<02:54, 768.71 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1673969/1807941 [21:11<02:51, 781.68 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1674075/1807941 [21:12<03:05, 722.62 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1674154/1807941 [21:12<03:01, 736.51 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1674231/1807941 [21:12<03:00, 740.81 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1674334/1807941 [21:12<03:05, 721.36 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1674422/1807941 [21:12<02:57, 752.32 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1674520/1807941 [21:12<02:45, 808.21 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1674605/1807941 [21:12<02:43, 814.88 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1674718/1807941 [21:12<02:49, 786.24 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1674833/1807941 [21:12<02:51, 774.77 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1674951/1807941 [21:13<02:50, 778.01 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1675054/1807941 [21:13<02:57, 747.15 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1675143/1807941 [21:13<02:50, 778.68 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1675257/1807941 [21:13<02:52, 771.33 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1675341/1807941 [21:13<02:48, 785.83 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1675426/1807941 [21:13<02:45, 800.59 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1675509/1807941 [21:13<02:44, 802.72 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1675599/1807941 [21:13<02:40, 824.33 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1675716/1807941 [21:14<02:45, 800.94 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1675835/1807941 [21:14<02:45, 796.47 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1675925/1807941 [21:14<02:41, 815.75 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1676009/1807941 [21:14<03:04, 714.58 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1676087/1807941 [21:14<03:01, 724.86 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1676178/1807941 [21:14<02:51, 770.52 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1676262/1807941 [21:14<02:47, 783.86 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1676383/1807941 [21:14<02:46, 791.45 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1676505/1807941 [21:15<02:46, 791.58 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1676608/1807941 [21:15<02:54, 751.56 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1676686/1807941 [21:15<02:54, 753.21 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1676766/1807941 [21:15<02:52, 761.35 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1676843/1807941 [21:15<02:51, 763.13 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1676925/1807941 [21:15<02:53, 755.67 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1677048/1807941 [21:15<02:58, 733.34 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1677155/1807941 [21:16<03:00, 723.60 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1677229/1807941 [21:16<03:00, 724.41 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1677303/1807941 [21:16<03:00, 725.23 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1677390/1807941 [21:16<02:52, 758.67 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1677497/1807941 [21:16<02:56, 741.00 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1677609/1807941 [21:16<02:56, 738.91 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1677684/1807941 [21:16<02:56, 738.19 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1677798/1807941 [21:16<02:55, 739.49 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1677875/1807941 [21:16<02:55, 741.35 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1677960/1807941 [21:17<02:49, 764.61 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1678046/1807941 [21:17<03:07, 693.62 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1678119/1807941 [21:17<03:05, 699.16 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1678227/1807941 [21:17<03:04, 702.54 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1678312/1807941 [21:17<02:56, 735.74 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1678392/1807941 [21:17<02:52, 749.25 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1678469/1807941 [21:17<02:52, 752.60 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1678572/1807941 [21:17<03:00, 715.15 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1678651/1807941 [21:18<02:57, 730.38 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1678741/1807941 [21:18<02:46, 774.89 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1678833/1807941 [21:18<02:38, 812.59 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1678918/1807941 [21:18<02:36, 822.44 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1679019/1807941 [21:18<02:49, 760.54 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1679123/1807941 [21:18<02:55, 734.25 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1679226/1807941 [21:18<02:59, 715.85 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1679305/1807941 [21:18<02:55, 732.46 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1679420/1807941 [21:19<02:53, 741.18 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1679526/1807941 [21:19<02:56, 727.99 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1679608/1807941 [21:19<02:52, 745.69 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1679690/1807941 [21:19<02:48, 760.05 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1679768/1807941 [21:19<02:48, 762.75 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1679875/1807941 [21:19<02:54, 735.72 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1679977/1807941 [21:19<02:59, 713.59 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1680065/1807941 [21:19<03:16, 650.94 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1680133/1807941 [21:20<03:14, 655.75 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1680208/1807941 [21:20<03:08, 677.42 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1680305/1807941 [21:20<02:50, 748.69 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1680392/1807941 [21:20<02:43, 779.26 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1680503/1807941 [21:20<02:47, 762.37 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1680587/1807941 [21:20<02:43, 777.28 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1680676/1807941 [21:20<02:38, 803.62 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1680798/1807941 [21:20<02:37, 804.73 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1680904/1807941 [21:21<02:45, 766.77 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1681000/1807941 [21:21<02:59, 708.89 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1681083/1807941 [21:21<02:53, 731.62 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1681190/1807941 [21:21<02:57, 714.14 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1681296/1807941 [21:21<02:58, 707.62 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1681405/1807941 [21:21<02:58, 709.98 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1681514/1807941 [21:21<02:57, 712.92 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1681586/1807941 [21:22<02:57, 712.56 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1681696/1807941 [21:22<02:56, 715.29 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1681775/1807941 [21:22<02:52, 731.31 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1681861/1807941 [21:22<02:45, 761.00 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1681946/1807941 [21:22<02:42, 777.35 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1682041/1807941 [21:22<02:55, 717.24 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1682128/1807941 [21:22<02:46, 754.05 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1682213/1807941 [21:22<02:41, 776.93 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1682296/1807941 [21:22<02:45, 761.43 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1682375/1807941 [21:23<02:44, 761.93 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1682457/1807941 [21:23<02:41, 775.87 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1682571/1807941 [21:23<02:43, 766.69 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1682662/1807941 [21:23<02:36, 799.76 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1682769/1807941 [21:23<02:43, 767.21 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1682872/1807941 [21:23<02:49, 738.06 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1682962/1807941 [21:23<02:41, 774.74 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1683078/1807941 [21:24<02:51, 730.03 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1683160/1807941 [21:24<02:47, 746.87 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1683260/1807941 [21:24<02:34, 807.54 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1683350/1807941 [21:24<02:30, 828.88 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1683465/1807941 [21:24<02:35, 800.81 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1683555/1807941 [21:24<02:31, 822.89 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1683642/1807941 [21:24<02:28, 834.81 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1683737/1807941 [21:24<02:23, 865.38 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1683835/1807941 [21:24<02:19, 891.66 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1683967/1807941 [21:25<02:20, 883.81 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1684066/1807941 [21:27<16:22, 126.13 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1684140/1807941 [21:27<13:11, 156.36 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1684210/1807941 [21:27<10:44, 191.98 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1684281/1807941 [21:27<08:43, 236.04 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1684356/1807941 [21:27<07:03, 292.00 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1684433/1807941 [21:27<05:46, 356.00 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1684512/1807941 [21:28<04:50, 424.91 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1684597/1807941 [21:28<04:04, 503.53 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1684677/1807941 [21:28<03:38, 564.16 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1684767/1807941 [21:28<03:49, 536.16 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1684847/1807941 [21:28<03:28, 591.36 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1684931/1807941 [21:28<03:09, 648.09 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1685030/1807941 [21:28<03:16, 623.94 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1685106/1807941 [21:28<03:07, 654.09 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1685179/1807941 [21:29<03:02, 671.49 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1685264/1807941 [21:29<02:51, 713.94 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1685348/1807941 [21:29<02:44, 745.28 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1685455/1807941 [21:29<02:47, 731.63 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1685553/1807941 [21:29<02:54, 699.88 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1685658/1807941 [21:29<02:55, 695.61 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1685731/1807941 [21:29<02:55, 697.88 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1685810/1807941 [21:29<02:50, 718.23 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1685895/1807941 [21:30<02:42, 751.01 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1685977/1807941 [21:30<02:39, 763.70 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1686090/1807941 [21:30<02:46, 732.28 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1686178/1807941 [21:30<02:38, 767.39 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1686296/1807941 [21:30<02:38, 767.60 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1686397/1807941 [21:30<02:46, 729.19 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1686473/1807941 [21:30<02:44, 736.22 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1686549/1807941 [21:30<02:44, 739.57 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1686631/1807941 [21:31<02:40, 758.12 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1686742/1807941 [21:31<02:42, 744.59 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1686833/1807941 [21:31<02:54, 692.15 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1686905/1807941 [21:31<02:54, 694.43 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1686977/1807941 [21:31<02:53, 698.29 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1687068/1807941 [21:31<03:05, 650.00 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1687135/1807941 [21:31<03:05, 652.84 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1687221/1807941 [21:31<02:51, 705.08 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1687308/1807941 [21:31<02:41, 745.93 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1687412/1807941 [21:32<02:47, 721.00 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1687493/1807941 [21:32<02:42, 740.28 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1687582/1807941 [21:32<02:35, 774.56 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1687681/1807941 [21:32<02:45, 727.99 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1687764/1807941 [21:32<02:39, 751.85 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1687841/1807941 [21:32<02:39, 754.65 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1687939/1807941 [21:32<02:50, 703.25 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1688036/1807941 [21:33<03:01, 660.53 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1688108/1807941 [21:33<03:00, 663.05 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1688180/1807941 [21:33<03:00, 662.13 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1688260/1807941 [21:33<02:52, 695.26 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1688349/1807941 [21:33<02:40, 743.02 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1688464/1807941 [21:33<02:39, 749.43 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1688569/1807941 [21:33<02:43, 729.34 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1688648/1807941 [21:33<02:40, 743.47 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1688764/1807941 [21:33<02:38, 750.84 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1688850/1807941 [21:34<02:34, 771.76 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1688961/1807941 [21:34<02:37, 754.11 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1689076/1807941 [21:34<02:54, 680.71 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1689158/1807941 [21:34<02:47, 709.32 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1689235/1807941 [21:34<02:44, 721.26 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1689338/1807941 [21:34<02:52, 688.95 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1689451/1807941 [21:34<02:47, 708.20 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1689531/1807941 [21:35<02:42, 728.25 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1689616/1807941 [21:35<02:36, 754.06 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1689698/1807941 [21:35<02:33, 768.18 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1689779/1807941 [21:35<02:31, 778.80 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1689870/1807941 [21:35<02:25, 810.30 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1689989/1807941 [21:35<02:28, 795.99 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1690071/1807941 [21:35<02:46, 706.45 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1690161/1807941 [21:35<02:36, 752.79 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1690270/1807941 [21:36<02:42, 725.97 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1690381/1807941 [21:36<02:41, 727.71 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1690464/1807941 [21:36<02:37, 746.38 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1690543/1807941 [21:36<02:35, 754.21 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1690640/1807941 [21:36<02:46, 706.38 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1690737/1807941 [21:36<02:52, 679.53 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1690824/1807941 [21:36<02:41, 722.99 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1690902/1807941 [21:36<02:39, 735.84 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1690993/1807941 [21:37<02:30, 778.71 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1691096/1807941 [21:37<02:41, 723.13 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1691176/1807941 [21:37<02:38, 737.01 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1691258/1807941 [21:37<02:34, 757.43 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1691345/1807941 [21:37<02:28, 785.39 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1691452/1807941 [21:37<02:34, 755.20 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1691547/1807941 [21:37<02:25, 800.40 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1691648/1807941 [21:37<02:16, 854.90 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1691771/1807941 [21:37<02:18, 837.03 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1691862/1807941 [21:38<02:15, 853.55 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1691977/1807941 [21:38<02:21, 818.84 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1692086/1807941 [21:38<02:27, 785.33 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1692178/1807941 [21:38<02:22, 811.96 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1692272/1807941 [21:38<02:17, 843.52 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1692385/1807941 [21:38<02:23, 805.45 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1692495/1807941 [21:38<02:27, 781.02 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1692576/1807941 [21:39<02:26, 786.27 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1692690/1807941 [21:39<02:29, 769.62 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1692809/1807941 [21:39<02:31, 759.24 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1692893/1807941 [21:39<02:28, 776.36 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1693000/1807941 [21:39<02:40, 714.92 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1693079/1807941 [21:39<02:37, 730.35 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1693161/1807941 [21:39<02:33, 749.01 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1693254/1807941 [21:39<02:47, 685.65 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1693329/1807941 [21:40<02:43, 700.07 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1693411/1807941 [21:40<02:36, 729.61 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1693490/1807941 [21:40<02:33, 743.54 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1693580/1807941 [21:40<02:26, 779.43 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1693661/1807941 [21:40<02:25, 783.58 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1693745/1807941 [21:40<02:23, 796.98 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1693834/1807941 [21:40<02:18, 822.60 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1693961/1807941 [21:40<02:18, 821.93 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1694086/1807941 [21:41<02:25, 780.58 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1694198/1807941 [21:41<02:29, 762.39 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1694289/1807941 [21:41<02:23, 794.59 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1694401/1807941 [21:41<02:27, 770.39 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1694491/1807941 [21:41<02:21, 800.04 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1694577/1807941 [21:41<02:20, 807.69 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1694701/1807941 [21:41<02:20, 804.73 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1694796/1807941 [21:41<02:15, 837.70 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1694882/1807941 [21:41<02:14, 842.55 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1695000/1807941 [21:42<02:21, 797.76 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1695090/1807941 [21:42<02:19, 809.77 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1695191/1807941 [21:42<02:11, 856.95 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1695313/1807941 [21:42<02:14, 839.72 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1695440/1807941 [21:42<02:14, 835.91 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1695558/1807941 [21:42<02:17, 817.10 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1695661/1807941 [21:42<02:26, 768.61 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1695776/1807941 [21:43<02:26, 765.66 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1695860/1807941 [21:43<02:23, 779.97 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1695975/1807941 [21:43<02:24, 773.62 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1696072/1807941 [21:43<02:36, 716.01 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1696154/1807941 [21:43<02:31, 735.79 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1696247/1807941 [21:43<02:23, 780.68 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1696356/1807941 [21:43<02:26, 761.16 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1696461/1807941 [21:44<02:31, 736.70 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1696574/1807941 [21:44<02:31, 735.09 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1696653/1807941 [21:44<02:28, 746.99 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1696751/1807941 [21:44<02:40, 691.49 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1696830/1807941 [21:44<02:36, 709.30 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1696926/1807941 [21:44<02:42, 684.43 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1696996/1807941 [21:44<02:48, 658.34 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1697081/1807941 [21:44<02:56, 626.40 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1697168/1807941 [21:45<02:42, 682.06 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1697251/1807941 [21:45<02:41, 687.32 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1697328/1807941 [21:45<02:36, 705.87 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1697402/1807941 [21:45<02:35, 711.53 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1697491/1807941 [21:45<02:25, 758.26 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1697573/1807941 [21:45<02:22, 773.34 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1697682/1807941 [21:45<02:26, 755.07 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1697773/1807941 [21:45<02:19, 792.51 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1697886/1807941 [21:46<02:21, 775.25 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1697968/1807941 [21:46<02:20, 784.80 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1698065/1807941 [21:46<02:36, 704.01 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1698169/1807941 [21:46<02:40, 684.89 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1698279/1807941 [21:46<02:37, 697.84 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1698362/1807941 [21:46<02:31, 725.46 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1698451/1807941 [21:46<02:42, 675.73 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1698529/1807941 [21:46<02:37, 696.55 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1698613/1807941 [21:47<02:31, 723.94 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1698707/1807941 [21:47<02:38, 687.50 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1698801/1807941 [21:47<02:45, 660.34 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1698871/1807941 [21:47<02:46, 656.74 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1698948/1807941 [21:47<02:39, 683.48 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1699052/1807941 [21:47<02:38, 685.22 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1699136/1807941 [21:47<02:31, 718.49 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1699241/1807941 [21:47<02:34, 705.81 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1699324/1807941 [21:48<02:27, 734.24 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1699407/1807941 [21:48<02:23, 756.49 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1699520/1807941 [21:48<02:24, 751.24 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1699630/1807941 [21:48<02:26, 740.08 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1699708/1807941 [21:48<02:25, 743.13 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1699789/1807941 [21:48<02:22, 758.28 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1699875/1807941 [21:48<02:17, 783.09 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1699981/1807941 [21:48<02:23, 751.70 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1700070/1807941 [21:49<02:37, 683.26 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1700183/1807941 [21:49<02:33, 700.85 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1700285/1807941 [21:49<02:35, 691.46 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1700375/1807941 [21:49<02:26, 734.83 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1700460/1807941 [21:49<02:21, 759.19 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1700563/1807941 [21:49<02:27, 728.16 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1700660/1807941 [21:49<02:34, 696.01 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1700765/1807941 [21:50<02:34, 694.38 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1700847/1807941 [21:50<02:28, 720.10 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1700925/1807941 [21:50<02:26, 732.75 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1701007/1807941 [21:50<02:42, 659.63 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1701086/1807941 [21:50<02:35, 689.26 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1701166/1807941 [21:50<02:29, 713.41 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1701241/1807941 [21:50<02:28, 719.65 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1701336/1807941 [21:50<02:35, 684.30 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1701410/1807941 [21:50<02:33, 696.16 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1701488/1807941 [21:51<02:28, 714.50 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1701566/1807941 [21:51<02:26, 728.35 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1701641/1807941 [21:51<02:25, 732.59 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1701716/1807941 [21:51<02:24, 733.52 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1701792/1807941 [21:51<02:23, 739.49 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1701888/1807941 [21:51<02:32, 695.57 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1701967/1807941 [21:51<02:27, 718.85 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1702048/1807941 [21:51<02:43, 649.46 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1702132/1807941 [21:52<02:32, 695.55 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1702241/1807941 [21:52<02:30, 703.63 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1702349/1807941 [21:52<02:29, 706.48 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1702427/1807941 [21:52<02:26, 721.99 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1702501/1807941 [21:52<02:25, 723.26 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1702610/1807941 [21:52<02:25, 722.34 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1702695/1807941 [21:52<02:20, 750.13 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1702810/1807941 [21:52<02:19, 752.01 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1702887/1807941 [21:53<02:19, 754.53 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1702970/1807941 [21:53<02:16, 771.39 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1703096/1807941 [21:53<02:15, 774.40 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1703183/1807941 [21:53<02:11, 795.93 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1703265/1807941 [21:53<02:11, 798.06 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1703372/1807941 [21:53<02:16, 764.32 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1703488/1807941 [21:53<02:17, 760.74 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1703575/1807941 [21:53<02:13, 783.89 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1703658/1807941 [21:53<02:11, 791.98 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1703766/1807941 [21:54<02:17, 758.67 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1703874/1807941 [21:54<02:20, 742.79 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1703974/1807941 [21:54<02:26, 710.88 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1704081/1807941 [21:54<02:32, 681.97 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1704153/1807941 [21:54<02:31, 686.52 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1704232/1807941 [21:54<02:26, 708.68 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1704305/1807941 [21:54<02:26, 709.67 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1704390/1807941 [21:55<02:19, 742.65 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1704472/1807941 [21:55<02:16, 757.96 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1704586/1807941 [21:55<02:17, 753.91 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1704669/1807941 [21:55<02:13, 771.99 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1704750/1807941 [21:55<02:12, 781.33 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1704861/1807941 [21:55<02:16, 757.02 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1704958/1807941 [21:55<02:08, 798.92 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1705045/1807941 [21:55<02:15, 759.18 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1705127/1807941 [21:55<02:13, 772.21 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1705229/1807941 [21:56<02:02, 836.62 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1705363/1807941 [21:56<02:00, 853.72 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1705496/1807941 [21:56<01:59, 857.14 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1705584/1807941 [21:56<01:59, 856.04 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1705698/1807941 [21:56<02:04, 819.42 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1705785/1807941 [21:56<02:03, 829.32 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1705900/1807941 [21:56<02:07, 803.35 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1705986/1807941 [21:56<02:05, 810.42 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1706076/1807941 [21:57<02:18, 735.97 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1706166/1807941 [21:57<02:11, 775.05 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1706280/1807941 [21:57<02:12, 766.26 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1706359/1807941 [21:57<02:12, 766.91 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1706481/1807941 [21:57<02:09, 780.77 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1706568/1807941 [21:57<02:06, 800.02 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1706656/1807941 [21:57<02:04, 811.65 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1706739/1807941 [21:57<02:04, 814.55 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1706863/1807941 [21:58<02:04, 813.97 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1706948/1807941 [21:58<02:02, 821.44 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1707051/1807941 [21:58<02:11, 767.59 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1707170/1807941 [21:58<02:10, 771.90 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1707259/1807941 [21:58<02:06, 796.22 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1707381/1807941 [21:58<02:05, 799.82 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1707462/1807941 [21:58<02:06, 797.10 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1707579/1807941 [21:59<02:07, 785.23 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1707699/1807941 [21:59<02:07, 783.16 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1707784/1807941 [21:59<02:05, 798.57 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1707874/1807941 [21:59<02:01, 823.81 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1707983/1807941 [21:59<02:08, 776.40 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1708082/1807941 [21:59<02:17, 726.48 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1708161/1807941 [21:59<02:15, 736.40 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1708275/1807941 [21:59<02:14, 739.68 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1708373/1807941 [22:00<02:20, 709.51 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1708484/1807941 [22:00<02:19, 714.52 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1708567/1807941 [22:00<02:14, 738.88 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1708649/1807941 [22:00<02:11, 757.82 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1708730/1807941 [22:00<02:08, 770.31 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1708835/1807941 [22:00<02:15, 731.81 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1708920/1807941 [22:00<02:10, 757.71 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1709013/1807941 [22:00<02:20, 704.51 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1709114/1807941 [22:01<02:22, 692.79 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1709218/1807941 [22:01<02:23, 686.06 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1709291/1807941 [22:01<02:22, 692.75 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1709392/1807941 [22:01<02:23, 685.08 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1709502/1807941 [22:01<02:20, 698.63 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1709582/1807941 [22:01<02:16, 720.19 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1709682/1807941 [22:01<02:20, 700.38 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1709764/1807941 [22:02<02:14, 727.79 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1709839/1807941 [22:02<02:14, 730.50 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1709921/1807941 [22:02<02:10, 750.43 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1709997/1807941 [22:02<02:11, 747.56 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1710076/1807941 [22:02<02:27, 663.44 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1710152/1807941 [22:02<02:23, 683.63 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1710231/1807941 [22:02<02:18, 706.16 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1710304/1807941 [22:02<02:19, 701.94 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1710378/1807941 [22:02<02:17, 709.27 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1710472/1807941 [22:03<02:24, 674.49 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1710550/1807941 [22:03<02:19, 699.71 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1710629/1807941 [22:03<02:14, 721.15 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1710732/1807941 [22:03<02:17, 705.43 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1710832/1807941 [22:03<02:21, 688.17 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1710904/1807941 [22:03<02:20, 692.57 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1710984/1807941 [22:03<02:15, 714.86 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1711074/1807941 [22:03<02:25, 664.31 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1711146/1807941 [22:04<02:23, 675.94 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1711247/1807941 [22:04<02:23, 672.17 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1711326/1807941 [22:04<02:18, 699.31 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1711433/1807941 [22:04<02:17, 702.49 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1711542/1807941 [22:04<02:16, 707.42 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1711618/1807941 [22:04<02:14, 718.42 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1711728/1807941 [22:04<02:13, 721.74 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1711828/1807941 [22:04<02:17, 698.83 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1711933/1807941 [22:05<02:17, 698.05 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1712036/1807941 [22:05<02:25, 657.68 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1712108/1807941 [22:05<02:23, 668.55 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1712185/1807941 [22:05<02:19, 685.05 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1712262/1807941 [22:05<02:16, 701.84 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1712342/1807941 [22:05<02:29, 638.16 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1712415/1807941 [22:05<02:25, 657.30 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1712496/1807941 [22:05<02:21, 676.41 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1712569/1807941 [22:06<02:18, 686.64 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1712670/1807941 [22:06<02:20, 676.52 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1712748/1807941 [22:06<02:16, 698.28 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1712829/1807941 [22:06<02:11, 725.13 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1712939/1807941 [22:06<02:11, 722.08 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1713033/1807941 [22:06<02:25, 650.31 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1713104/1807941 [22:06<02:23, 662.57 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1713172/1807941 [22:06<02:23, 660.46 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1713263/1807941 [22:07<02:29, 631.72 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1713331/1807941 [22:07<02:28, 635.26 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1713398/1807941 [22:07<02:27, 642.51 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1713495/1807941 [22:07<02:27, 639.33 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1713592/1807941 [22:07<02:27, 638.71 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1713660/1807941 [22:07<02:25, 647.38 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1713729/1807941 [22:07<02:23, 657.31 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1713797/1807941 [22:07<02:22, 658.54 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1713865/1807941 [22:08<02:21, 662.73 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1713939/1807941 [22:08<02:17, 681.91 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1714034/1807941 [22:08<02:32, 614.77 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1714103/1807941 [22:08<02:28, 631.21 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1714179/1807941 [22:08<02:21, 660.99 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1714277/1807941 [22:08<02:23, 654.79 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1714353/1807941 [22:08<02:17, 680.04 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1714427/1807941 [22:08<02:15, 692.05 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1714515/1807941 [22:09<02:06, 738.97 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1714622/1807941 [22:09<02:08, 727.23 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1714739/1807941 [22:09<02:05, 743.91 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1714817/1807941 [22:09<02:04, 749.48 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1714915/1807941 [22:09<02:10, 713.61 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715000/1807941 [22:09<02:30, 619.10 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715065/1807941 [22:09<02:45, 561.25 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715132/1807941 [22:10<02:39, 581.39 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715197/1807941 [22:10<02:35, 597.65 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715272/1807941 [22:10<02:26, 634.22 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715342/1807941 [22:10<02:22, 648.84 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715434/1807941 [22:10<02:26, 631.83 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715510/1807941 [22:10<02:20, 660.17 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715603/1807941 [22:10<02:23, 641.50 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715669/1807941 [22:10<02:24, 636.96 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715741/1807941 [22:10<02:20, 654.72 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715839/1807941 [22:11<02:21, 651.32 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715909/1807941 [22:11<02:19, 661.36 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715988/1807941 [22:11<02:14, 684.49 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1716086/1807941 [22:11<02:36, 588.54 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1716152/1807941 [22:11<02:32, 603.54 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1716217/1807941 [22:11<02:29, 613.61 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1716283/1807941 [22:11<02:26, 624.51 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1716358/1807941 [22:11<02:19, 656.14 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1716426/1807941 [22:12<02:19, 657.93 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1716506/1807941 [22:12<02:35, 586.31 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1716643/1807941 [22:12<01:56, 782.98 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1716773/1807941 [22:12<01:39, 919.57 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1716918/1807941 [22:12<01:25, 1062.21 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1717086/1807941 [22:12<01:23, 1081.61 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1717236/1807941 [22:12<01:16, 1189.17 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1717402/1807941 [22:12<01:08, 1314.47 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1717570/1807941 [22:12<01:03, 1414.72 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1717735/1807941 [22:13<01:00, 1479.82 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1717896/1807941 [22:13<00:59, 1513.73 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1718105/1807941 [22:13<01:01, 1462.02 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1718280/1807941 [22:13<00:58, 1537.48 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1718542/1807941 [22:13<00:48, 1833.77 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1718816/1807941 [22:13<00:42, 2088.17 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1719051/1807941 [22:13<00:47, 1868.77 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1719261/1807941 [22:13<00:58, 1522.43 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1719434/1807941 [22:14<01:03, 1402.60 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1719597/1807941 [22:14<01:07, 1302.86 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1719841/1807941 [22:14<01:03, 1397.53 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1720033/1807941 [22:14<00:58, 1511.71 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1720218/1807941 [22:14<00:55, 1591.89 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1720551/1807941 [22:14<00:42, 2035.15 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1720911/1807941 [22:14<00:35, 2451.38 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1721221/1807941 [22:14<00:37, 2305.70 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1721552/1807941 [22:15<00:38, 2266.94 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1721884/1807941 [22:15<00:38, 2245.19 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1722183/1807941 [22:15<00:39, 2162.60 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1722412/1807941 [22:15<00:39, 2191.70 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1722644/1807941 [22:15<00:38, 2220.25 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1722873/1807941 [22:15<00:38, 2234.41 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1723159/1807941 [22:15<00:45, 1880.37 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1723363/1807941 [22:16<00:49, 1714.32 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1723589/1807941 [22:16<00:56, 1483.06 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1723749/1807941 [22:16<01:20, 1047.16 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1723901/1807941 [22:16<01:40, 832.14 examples/s] 
Tokenizing train dataset:  95%|█████████▌| 1724072/1807941 [22:17<01:33, 896.67 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1724228/1807941 [22:17<01:23, 1008.39 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1724376/1807941 [22:17<01:15, 1099.61 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1724528/1807941 [22:17<01:10, 1188.48 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1724672/1807941 [22:17<01:06, 1246.26 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1724834/1807941 [22:17<01:02, 1339.58 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1724986/1807941 [22:17<00:59, 1383.12 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1725179/1807941 [22:17<01:01, 1344.78 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1725340/1807941 [22:17<00:58, 1409.06 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1725488/1807941 [22:17<00:58, 1416.78 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1725642/1807941 [22:18<00:56, 1449.47 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1725863/1807941 [22:18<00:56, 1452.31 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1726074/1807941 [22:18<00:58, 1411.19 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1726220/1807941 [22:18<00:57, 1421.29 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1726371/1807941 [22:18<00:56, 1443.04 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1726532/1807941 [22:18<00:54, 1485.08 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1726688/1807941 [22:19<01:32, 879.75 examples/s] 
Tokenizing train dataset:  96%|█████████▌| 1726829/1807941 [22:19<01:30, 892.83 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1726968/1807941 [22:19<01:21, 989.28 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1727136/1807941 [22:19<01:19, 1021.11 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1727271/1807941 [22:19<01:13, 1090.55 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1727406/1807941 [22:19<01:09, 1151.28 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1727539/1807941 [22:19<01:25, 944.01 examples/s] 
Tokenizing train dataset:  96%|█████████▌| 1727682/1807941 [22:20<01:33, 856.74 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1727822/1807941 [22:20<01:22, 967.14 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1727966/1807941 [22:20<01:14, 1072.74 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1728146/1807941 [22:20<01:11, 1112.58 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1728283/1807941 [22:20<01:07, 1171.67 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1728424/1807941 [22:20<01:04, 1228.67 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1728572/1807941 [22:20<01:01, 1294.33 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1728729/1807941 [22:20<00:57, 1367.27 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1728931/1807941 [22:20<00:58, 1355.64 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1729093/1807941 [22:21<01:02, 1256.55 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1729225/1807941 [22:21<01:01, 1271.96 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1729413/1807941 [22:21<01:02, 1261.63 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1729604/1807941 [22:21<01:02, 1263.27 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1729743/1807941 [22:21<01:00, 1291.45 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1729933/1807941 [22:21<01:01, 1278.20 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1730087/1807941 [22:21<01:05, 1193.94 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1730217/1807941 [22:22<01:03, 1215.44 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1730351/1807941 [22:22<01:02, 1245.78 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1730487/1807941 [22:22<01:18, 991.53 examples/s] 
Tokenizing train dataset:  96%|█████████▌| 1730608/1807941 [22:22<01:31, 843.03 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1730708/1807941 [22:22<01:57, 656.55 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1730813/1807941 [22:23<02:05, 615.43 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1730945/1807941 [22:23<01:43, 741.79 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1731063/1807941 [22:23<01:37, 786.17 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1731189/1807941 [22:23<01:26, 887.13 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1731318/1807941 [22:23<01:18, 980.30 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1731444/1807941 [22:23<01:12, 1049.71 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1731574/1807941 [22:23<01:08, 1112.82 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1731699/1807941 [22:23<01:06, 1147.81 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1731830/1807941 [22:23<01:03, 1189.52 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1731956/1807941 [22:23<01:02, 1206.56 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1732126/1807941 [22:24<01:05, 1152.98 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1732294/1807941 [22:24<01:06, 1138.63 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1732461/1807941 [22:24<01:06, 1127.39 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1732576/1807941 [22:24<01:06, 1130.77 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1732698/1807941 [22:24<01:05, 1150.72 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1732875/1807941 [22:24<01:04, 1156.31 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1733000/1807941 [22:24<01:07, 1111.43 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1733117/1807941 [22:24<01:06, 1122.27 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1733284/1807941 [22:25<01:06, 1114.81 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1733396/1807941 [22:25<01:06, 1113.81 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1733519/1807941 [22:25<01:05, 1138.97 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1733695/1807941 [22:25<01:04, 1146.28 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1733819/1807941 [22:25<01:03, 1167.16 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1734000/1807941 [22:25<01:06, 1119.53 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1734154/1807941 [22:25<01:07, 1086.74 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1734276/1807941 [22:26<01:06, 1115.62 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1734444/1807941 [22:26<01:05, 1114.83 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1734567/1807941 [22:26<01:04, 1142.09 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1734739/1807941 [22:26<01:04, 1140.41 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1734862/1807941 [22:26<01:02, 1160.60 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1735000/1807941 [22:26<01:08, 1069.80 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1735119/1807941 [22:26<01:06, 1097.42 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1735287/1807941 [22:26<01:05, 1102.46 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1735410/1807941 [22:27<01:04, 1132.10 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1735574/1807941 [22:27<01:04, 1116.98 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1735698/1807941 [22:27<01:03, 1146.24 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1735866/1807941 [22:27<01:03, 1133.77 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1735993/1807941 [22:27<01:01, 1165.93 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1736131/1807941 [22:27<01:06, 1078.32 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1736265/1807941 [22:27<01:02, 1141.41 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1736393/1807941 [22:27<01:08, 1037.13 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1736505/1807941 [22:28<01:42, 699.44 examples/s] 
Tokenizing train dataset:  96%|█████████▌| 1736634/1807941 [22:28<01:36, 738.43 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1736751/1807941 [22:28<01:26, 822.12 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1736872/1807941 [22:28<01:18, 906.20 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1737000/1807941 [22:28<01:17, 912.58 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1737120/1807941 [22:28<01:12, 979.68 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1737244/1807941 [22:28<01:07, 1043.42 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1737360/1807941 [22:29<01:05, 1071.45 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1737485/1807941 [22:29<01:03, 1117.12 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1737607/1807941 [22:29<01:01, 1144.66 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1737729/1807941 [22:29<01:00, 1161.49 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1737883/1807941 [22:29<01:11, 973.72 examples/s] 
Tokenizing train dataset:  96%|█████████▌| 1738000/1807941 [22:29<01:12, 964.84 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1738114/1807941 [22:29<01:09, 1007.15 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1738224/1807941 [22:29<01:08, 1022.68 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1738340/1807941 [22:29<01:05, 1056.04 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1738501/1807941 [22:30<01:05, 1058.49 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1738618/1807941 [22:30<01:03, 1085.08 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1738739/1807941 [22:30<01:01, 1116.68 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1738865/1807941 [22:30<00:59, 1153.91 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1739001/1807941 [22:30<01:05, 1049.07 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1739153/1807941 [22:30<01:06, 1030.93 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1739271/1807941 [22:30<01:04, 1066.09 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1739381/1807941 [22:30<01:03, 1071.96 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1739498/1807941 [22:31<01:02, 1093.93 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1739615/1807941 [22:31<01:01, 1112.18 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1739780/1807941 [22:31<01:01, 1106.40 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1739945/1807941 [22:31<01:01, 1100.32 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1740059/1807941 [22:31<01:04, 1059.95 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1740173/1807941 [22:31<01:02, 1076.87 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1740302/1807941 [22:31<00:59, 1131.52 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1740484/1807941 [22:31<00:58, 1160.07 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1740631/1807941 [22:32<01:01, 1096.92 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1740758/1807941 [22:32<00:59, 1136.20 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1740918/1807941 [22:32<01:00, 1109.54 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1741059/1807941 [22:32<01:03, 1050.83 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1741227/1807941 [22:32<01:02, 1070.25 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1741369/1807941 [22:32<01:04, 1026.50 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1741523/1807941 [22:32<01:05, 1020.96 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1741657/1807941 [22:33<01:08, 974.45 examples/s] 
Tokenizing train dataset:  96%|█████████▋| 1741786/1807941 [22:33<01:10, 938.55 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1741899/1807941 [22:33<01:08, 967.29 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1742001/1807941 [22:33<01:07, 979.49 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1742129/1807941 [22:33<01:10, 929.74 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1742258/1807941 [22:33<01:13, 898.21 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1742353/1807941 [22:33<01:12, 902.79 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1742485/1807941 [22:33<01:14, 883.61 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1742641/1807941 [22:34<01:10, 932.34 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1742751/1807941 [22:34<01:07, 969.58 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1742881/1807941 [22:34<01:01, 1049.66 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1743000/1807941 [22:34<01:02, 1039.50 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1743126/1807941 [22:34<00:59, 1094.81 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1743246/1807941 [22:34<00:57, 1120.21 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1743371/1807941 [22:34<00:56, 1151.95 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1743497/1807941 [22:34<00:54, 1178.88 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1743622/1807941 [22:34<00:53, 1197.87 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1743743/1807941 [22:35<00:53, 1198.91 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1743872/1807941 [22:35<00:52, 1223.07 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1744003/1807941 [22:35<00:59, 1081.12 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1744141/1807941 [22:35<01:17, 820.38 examples/s] 
Tokenizing train dataset:  96%|█████████▋| 1744276/1807941 [22:35<01:08, 931.24 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1744399/1807941 [22:35<01:03, 998.18 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1744529/1807941 [22:35<00:59, 1071.57 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1744652/1807941 [22:35<00:56, 1111.68 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1744774/1807941 [22:36<00:55, 1138.65 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1744944/1807941 [22:36<00:55, 1133.83 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1745114/1807941 [22:36<00:58, 1066.48 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1745276/1807941 [22:36<00:58, 1067.98 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1745387/1807941 [22:36<00:58, 1070.00 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1745504/1807941 [22:36<00:57, 1091.54 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1745635/1807941 [22:36<00:54, 1143.73 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1745811/1807941 [22:37<00:54, 1149.09 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1745977/1807941 [22:37<00:54, 1131.91 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1746111/1807941 [22:37<00:58, 1050.81 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1746224/1807941 [22:37<00:57, 1068.52 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1746334/1807941 [22:37<00:57, 1075.35 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1746443/1807941 [22:37<00:57, 1077.73 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1746598/1807941 [22:37<00:57, 1058.48 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1746716/1807941 [22:37<00:56, 1085.18 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1746849/1807941 [22:37<00:53, 1147.08 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1746970/1807941 [22:38<00:52, 1163.37 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1747116/1807941 [22:38<00:55, 1088.88 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1747231/1807941 [22:38<00:55, 1101.36 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1747350/1807941 [22:38<00:54, 1119.61 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1747464/1807941 [22:38<00:53, 1121.83 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1747640/1807941 [22:38<00:52, 1137.92 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1747767/1807941 [22:38<00:51, 1172.05 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1747943/1807941 [22:38<00:51, 1164.02 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1748100/1807941 [22:39<00:56, 1061.21 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1748213/1807941 [22:39<00:55, 1076.10 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1748324/1807941 [22:39<00:55, 1082.64 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1748436/1807941 [22:39<00:54, 1091.28 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1748604/1807941 [22:39<00:53, 1099.74 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1748762/1807941 [22:39<00:54, 1080.09 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1748887/1807941 [22:39<00:52, 1118.80 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1749054/1807941 [22:39<00:55, 1063.15 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1749210/1807941 [22:40<00:55, 1051.94 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1749359/1807941 [22:40<00:56, 1030.58 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1749473/1807941 [22:40<00:55, 1055.14 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1749584/1807941 [22:40<00:54, 1065.36 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1749694/1807941 [22:40<00:54, 1071.23 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1749804/1807941 [22:40<00:54, 1076.26 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1749967/1807941 [22:40<00:53, 1079.14 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1750085/1807941 [22:41<01:06, 868.68 examples/s] 
Tokenizing train dataset:  97%|█████████▋| 1750182/1807941 [22:41<01:12, 793.03 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1750266/1807941 [22:41<01:28, 652.43 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1750355/1807941 [22:41<01:31, 628.79 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1750422/1807941 [22:41<01:30, 635.37 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1750496/1807941 [22:41<01:27, 657.67 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1750614/1807941 [22:41<01:13, 784.18 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1750752/1807941 [22:41<01:01, 937.32 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1750888/1807941 [22:42<00:54, 1049.87 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1751000/1807941 [22:42<00:56, 1015.93 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1751129/1807941 [22:42<00:52, 1088.80 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1751258/1807941 [22:42<00:49, 1142.28 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1751392/1807941 [22:42<00:47, 1196.57 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1751526/1807941 [22:42<00:45, 1235.29 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1751656/1807941 [22:42<00:44, 1251.21 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1751791/1807941 [22:42<00:43, 1276.19 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1751927/1807941 [22:42<00:43, 1296.94 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1752089/1807941 [22:43<00:46, 1209.75 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1752214/1807941 [22:43<00:45, 1219.35 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1752340/1807941 [22:43<00:45, 1228.20 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1752468/1807941 [22:43<00:44, 1240.56 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1752599/1807941 [22:43<00:44, 1257.53 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1752727/1807941 [22:43<00:43, 1259.14 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1752911/1807941 [22:43<00:44, 1240.09 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1753068/1807941 [22:43<00:47, 1167.01 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1753191/1807941 [22:43<00:46, 1180.78 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1753319/1807941 [22:44<00:45, 1204.95 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1753499/1807941 [22:44<00:45, 1199.18 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1753628/1807941 [22:44<00:44, 1221.49 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1753816/1807941 [22:44<00:44, 1228.41 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1753942/1807941 [22:44<00:43, 1234.25 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1754116/1807941 [22:44<00:46, 1145.70 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1754295/1807941 [22:44<00:46, 1156.85 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1754421/1807941 [22:44<00:45, 1178.04 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1754549/1807941 [22:45<00:44, 1200.67 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1754675/1807941 [22:45<00:43, 1212.12 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1754801/1807941 [22:45<00:43, 1220.64 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1754927/1807941 [22:45<00:43, 1229.91 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1755075/1807941 [22:45<00:46, 1135.91 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1755195/1807941 [22:45<00:45, 1150.56 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1755317/1807941 [22:45<00:45, 1166.92 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1755439/1807941 [22:45<00:44, 1179.48 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1755566/1807941 [22:45<00:43, 1200.44 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1755695/1807941 [22:46<00:42, 1220.78 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1755878/1807941 [22:46<00:42, 1216.03 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1756027/1807941 [22:46<00:45, 1134.05 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1756150/1807941 [22:46<00:44, 1155.68 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1756270/1807941 [22:46<00:44, 1163.49 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1756404/1807941 [22:46<00:48, 1059.12 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1756528/1807941 [22:46<00:52, 976.03 examples/s] 
Tokenizing train dataset:  97%|█████████▋| 1756655/1807941 [22:47<00:55, 927.20 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1756780/1807941 [22:47<00:57, 888.53 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1756872/1807941 [22:47<00:57, 894.75 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1756975/1807941 [22:47<01:15, 671.15 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1757090/1807941 [22:47<01:18, 651.00 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1757191/1807941 [22:47<01:17, 654.19 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1757261/1807941 [22:47<01:16, 663.26 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1757371/1807941 [22:48<01:06, 755.63 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1757482/1807941 [22:48<01:00, 838.72 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1757612/1807941 [22:48<01:07, 750.12 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1757726/1807941 [22:48<01:06, 751.14 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1757810/1807941 [22:48<01:05, 770.60 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1757913/1807941 [22:48<01:07, 741.88 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1758000/1807941 [22:48<01:13, 676.63 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1758079/1807941 [22:49<01:19, 623.93 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1758177/1807941 [22:49<01:18, 630.81 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1758248/1807941 [22:49<01:16, 646.16 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1758360/1807941 [22:49<01:05, 757.63 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1758492/1807941 [22:49<00:55, 898.53 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1758627/1807941 [22:49<00:48, 1017.10 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1758760/1807941 [22:49<00:44, 1100.91 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1758895/1807941 [22:49<00:41, 1168.69 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1759064/1807941 [22:50<00:43, 1133.67 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1759190/1807941 [22:50<00:41, 1163.50 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1759312/1807941 [22:50<00:41, 1175.01 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1759448/1807941 [22:50<00:50, 956.27 examples/s] 
Tokenizing train dataset:  97%|█████████▋| 1759559/1807941 [22:50<00:54, 879.97 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1759701/1807941 [22:50<00:48, 996.18 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1759883/1807941 [22:50<00:45, 1064.11 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1760015/1807941 [22:50<00:47, 1002.69 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1760157/1807941 [22:51<00:43, 1098.07 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1760319/1807941 [22:51<00:43, 1089.06 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1760473/1807941 [22:51<00:44, 1067.22 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1760597/1807941 [22:51<00:42, 1105.42 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1760745/1807941 [22:51<00:54, 865.27 examples/s] 
Tokenizing train dataset:  97%|█████████▋| 1760865/1807941 [22:51<00:50, 928.98 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1760976/1807941 [22:52<01:05, 713.61 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1761061/1807941 [22:52<01:05, 713.41 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1761169/1807941 [22:52<00:59, 788.39 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1761258/1807941 [22:52<00:57, 808.87 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1761361/1807941 [22:52<00:54, 861.58 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1761504/1807941 [22:52<00:58, 791.31 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1761665/1807941 [22:52<00:47, 974.95 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1761844/1807941 [22:52<00:44, 1043.47 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1761965/1807941 [22:53<00:42, 1079.63 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1762115/1807941 [22:53<00:43, 1047.42 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1762260/1807941 [22:53<00:39, 1142.51 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1762434/1807941 [22:53<00:39, 1146.43 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1762556/1807941 [22:53<00:39, 1163.22 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1762683/1807941 [22:53<00:43, 1051.04 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1762812/1807941 [22:53<00:46, 976.39 examples/s] 
Tokenizing train dataset:  98%|█████████▊| 1762927/1807941 [22:54<00:55, 816.05 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1763034/1807941 [22:54<01:04, 692.08 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1763159/1807941 [22:54<00:55, 799.80 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1763281/1807941 [22:54<00:55, 800.00 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1763397/1807941 [22:54<01:02, 713.56 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1763491/1807941 [22:54<00:58, 756.06 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1763589/1807941 [22:54<00:55, 799.90 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1763691/1807941 [22:55<00:52, 849.16 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1763821/1807941 [22:55<00:51, 851.90 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1763923/1807941 [22:55<00:49, 891.03 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1764028/1807941 [22:55<00:54, 800.04 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1764158/1807941 [22:55<00:53, 818.37 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1764274/1807941 [22:55<00:48, 897.29 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1764369/1807941 [22:55<00:48, 907.47 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1764473/1807941 [22:56<00:58, 739.42 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1764560/1807941 [22:56<00:56, 767.36 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1764665/1807941 [22:56<00:58, 742.34 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1764767/1807941 [22:56<01:00, 719.32 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1764853/1807941 [22:56<00:57, 748.95 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1764943/1807941 [22:56<00:54, 783.47 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1765041/1807941 [22:56<00:59, 727.09 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1765122/1807941 [22:56<00:57, 744.73 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1765202/1807941 [22:57<01:11, 594.36 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1765294/1807941 [22:57<01:03, 666.63 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1765390/1807941 [22:57<00:58, 731.17 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1765474/1807941 [22:57<01:11, 594.89 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1765542/1807941 [22:57<01:09, 611.91 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1765692/1807941 [22:57<00:51, 823.70 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1765791/1807941 [22:57<00:48, 864.97 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1765900/1807941 [22:57<00:45, 922.03 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1765998/1807941 [22:58<00:45, 931.31 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1766120/1807941 [22:58<00:53, 785.22 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1766266/1807941 [22:58<00:49, 842.34 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1766391/1807941 [22:58<00:49, 836.11 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1766488/1807941 [22:58<00:54, 766.45 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1766624/1807941 [22:58<01:02, 664.93 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1766762/1807941 [22:59<00:51, 800.51 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1766874/1807941 [22:59<01:03, 649.57 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1766951/1807941 [22:59<01:01, 670.57 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1767037/1807941 [22:59<01:07, 608.65 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1767116/1807941 [22:59<01:03, 645.05 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1767215/1807941 [22:59<01:02, 646.49 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1767309/1807941 [23:00<01:03, 635.10 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1767444/1807941 [23:00<00:51, 792.73 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1767542/1807941 [23:00<00:48, 836.81 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1767656/1807941 [23:00<00:56, 717.87 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1767742/1807941 [23:00<01:06, 604.84 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1767816/1807941 [23:00<01:04, 625.67 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1767917/1807941 [23:00<01:02, 637.54 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1767988/1807941 [23:00<01:01, 652.33 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1768086/1807941 [23:01<01:08, 582.20 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1768165/1807941 [23:01<01:04, 618.95 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1768255/1807941 [23:01<01:05, 608.03 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1768354/1807941 [23:01<00:57, 690.13 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1768434/1807941 [23:01<01:08, 573.16 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1768504/1807941 [23:01<01:20, 489.59 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1768584/1807941 [23:02<01:11, 547.73 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1768651/1807941 [23:02<01:08, 570.78 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1768722/1807941 [23:02<01:05, 602.08 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1768806/1807941 [23:02<00:59, 661.39 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1768886/1807941 [23:02<00:56, 690.63 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1768980/1807941 [23:02<00:51, 753.60 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1769062/1807941 [23:02<00:58, 660.13 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1769144/1807941 [23:02<01:02, 620.83 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1769248/1807941 [23:03<01:00, 642.62 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1769347/1807941 [23:03<00:53, 721.29 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1769433/1807941 [23:03<01:04, 592.72 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1769505/1807941 [23:03<01:16, 499.62 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1769598/1807941 [23:03<01:12, 527.04 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1769691/1807941 [23:03<01:16, 500.19 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1769763/1807941 [23:04<01:10, 541.94 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1769846/1807941 [23:04<01:03, 598.13 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1769953/1807941 [23:04<00:53, 706.15 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1770035/1807941 [23:04<00:59, 638.45 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1770106/1807941 [23:04<00:57, 654.37 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1770181/1807941 [23:04<00:56, 672.86 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1770272/1807941 [23:04<00:58, 642.16 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1770351/1807941 [23:04<01:10, 534.40 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1770422/1807941 [23:05<01:21, 461.56 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1770489/1807941 [23:05<01:22, 454.18 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1770551/1807941 [23:05<01:17, 484.00 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1770635/1807941 [23:05<01:14, 503.98 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1770690/1807941 [23:05<01:13, 507.86 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1770763/1807941 [23:05<01:06, 558.96 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1770834/1807941 [23:05<01:02, 589.42 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1770915/1807941 [23:05<00:57, 643.80 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1770983/1807941 [23:06<00:56, 650.50 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1771051/1807941 [23:06<01:09, 534.10 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1771125/1807941 [23:06<01:10, 518.56 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1771182/1807941 [23:06<01:09, 526.14 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1771284/1807941 [23:06<00:56, 649.40 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1771421/1807941 [23:06<00:43, 836.99 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1771541/1807941 [23:06<00:38, 935.34 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1771678/1807941 [23:06<00:34, 1054.67 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1771796/1807941 [23:07<00:38, 942.97 examples/s] 
Tokenizing train dataset:  98%|█████████▊| 1771931/1807941 [23:07<00:43, 826.79 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1772029/1807941 [23:07<01:02, 570.36 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1772143/1807941 [23:07<01:03, 562.04 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1772265/1807941 [23:08<01:02, 572.02 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1772359/1807941 [23:08<01:01, 582.15 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1772438/1807941 [23:08<00:57, 616.82 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1772536/1807941 [23:08<00:51, 691.02 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1772628/1807941 [23:08<00:47, 737.77 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1772718/1807941 [23:08<00:45, 772.38 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1772815/1807941 [23:08<00:42, 822.06 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1772910/1807941 [23:08<00:47, 745.02 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773000/1807941 [23:09<00:57, 610.89 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773091/1807941 [23:09<00:57, 607.36 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773176/1807941 [23:09<00:52, 658.84 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773262/1807941 [23:09<01:01, 566.59 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773331/1807941 [23:09<01:12, 480.18 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773397/1807941 [23:09<01:21, 422.39 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773465/1807941 [23:10<01:20, 426.49 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773532/1807941 [23:10<01:20, 428.37 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773589/1807941 [23:10<01:15, 454.46 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773653/1807941 [23:10<01:09, 492.29 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773737/1807941 [23:10<00:59, 574.71 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773868/1807941 [23:10<00:44, 761.35 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773962/1807941 [23:10<00:47, 708.78 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774054/1807941 [23:11<00:58, 581.26 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774138/1807941 [23:11<00:59, 569.43 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774209/1807941 [23:11<01:03, 534.59 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774300/1807941 [23:11<00:54, 613.66 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774371/1807941 [23:11<00:52, 636.07 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774452/1807941 [23:11<01:02, 536.65 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774513/1807941 [23:11<01:14, 447.96 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774583/1807941 [23:12<01:07, 494.38 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774662/1807941 [23:12<01:06, 500.95 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774735/1807941 [23:12<01:07, 490.17 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774804/1807941 [23:12<01:09, 479.39 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774871/1807941 [23:12<01:04, 515.39 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774937/1807941 [23:12<01:00, 545.04 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775000/1807941 [23:12<01:05, 503.88 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775058/1807941 [23:12<01:03, 520.46 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775117/1807941 [23:13<01:01, 531.22 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775193/1807941 [23:13<01:02, 521.02 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775253/1807941 [23:13<01:00, 536.80 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775313/1807941 [23:13<00:59, 550.59 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775400/1807941 [23:13<00:58, 555.35 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775477/1807941 [23:13<01:00, 535.39 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775542/1807941 [23:13<00:57, 561.99 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775634/1807941 [23:13<00:49, 650.89 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775707/1807941 [23:14<00:54, 589.18 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775774/1807941 [23:14<01:06, 481.55 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775829/1807941 [23:14<01:19, 404.34 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775884/1807941 [23:14<01:22, 389.58 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775947/1807941 [23:14<01:20, 395.07 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775999/1807941 [23:14<01:24, 377.13 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776050/1807941 [23:15<01:29, 355.24 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776106/1807941 [23:15<01:19, 398.12 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776163/1807941 [23:15<01:13, 434.61 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776219/1807941 [23:15<01:18, 405.81 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776270/1807941 [23:15<01:14, 426.87 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776315/1807941 [23:15<01:13, 430.95 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776366/1807941 [23:15<01:11, 444.18 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776434/1807941 [23:15<01:11, 443.30 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776481/1807941 [23:16<01:10, 447.22 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776541/1807941 [23:16<01:04, 485.99 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776595/1807941 [23:16<01:11, 435.92 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776643/1807941 [23:16<01:10, 445.24 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776695/1807941 [23:16<01:07, 463.04 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776759/1807941 [23:16<01:01, 510.59 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776833/1807941 [23:16<01:02, 499.83 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776913/1807941 [23:16<01:01, 506.98 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776965/1807941 [23:17<01:01, 506.15 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777019/1807941 [23:17<01:08, 452.98 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777069/1807941 [23:17<01:06, 462.99 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777121/1807941 [23:17<01:04, 474.72 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777176/1807941 [23:17<01:02, 493.31 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777235/1807941 [23:17<00:59, 516.16 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777323/1807941 [23:17<00:49, 614.20 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777398/1807941 [23:17<00:53, 566.46 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777478/1807941 [23:18<00:55, 546.75 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777539/1807941 [23:18<01:01, 497.12 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777594/1807941 [23:18<01:00, 504.19 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777653/1807941 [23:18<00:57, 523.44 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777734/1807941 [23:18<00:57, 526.55 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777788/1807941 [23:18<00:57, 528.86 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777865/1807941 [23:18<00:57, 518.77 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777929/1807941 [23:18<01:02, 481.34 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777983/1807941 [23:19<01:00, 491.33 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778037/1807941 [23:19<01:03, 472.62 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778114/1807941 [23:19<00:54, 546.12 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778171/1807941 [23:19<00:54, 550.39 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778230/1807941 [23:19<00:53, 558.64 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778296/1807941 [23:19<01:05, 450.13 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778372/1807941 [23:19<01:03, 463.88 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778434/1807941 [23:20<01:14, 398.06 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778489/1807941 [23:20<01:23, 352.41 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778537/1807941 [23:20<01:18, 375.23 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778588/1807941 [23:20<01:21, 360.32 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778628/1807941 [23:20<01:29, 329.31 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778718/1807941 [23:20<01:04, 451.77 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778776/1807941 [23:20<01:07, 429.11 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778836/1807941 [23:21<01:09, 415.80 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778894/1807941 [23:21<01:11, 403.86 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778939/1807941 [23:21<01:10, 412.95 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778986/1807941 [23:21<01:08, 421.94 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779058/1807941 [23:21<01:23, 346.50 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779116/1807941 [23:21<01:21, 355.21 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779176/1807941 [23:22<01:18, 364.56 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779217/1807941 [23:22<01:17, 369.15 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779272/1807941 [23:22<01:18, 367.18 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779321/1807941 [23:22<01:21, 351.42 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779363/1807941 [23:22<01:18, 364.23 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779410/1807941 [23:22<01:14, 384.57 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779463/1807941 [23:22<01:17, 368.95 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779501/1807941 [23:22<01:16, 370.37 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779539/1807941 [23:23<01:16, 368.92 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779583/1807941 [23:23<01:14, 382.12 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779641/1807941 [23:23<01:15, 377.23 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779684/1807941 [23:23<01:13, 384.92 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779738/1807941 [23:23<01:15, 374.17 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779789/1807941 [23:23<01:09, 405.02 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779850/1807941 [23:23<01:09, 402.08 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779903/1807941 [23:23<01:13, 380.87 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779944/1807941 [23:24<01:12, 386.38 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780000/1807941 [23:24<01:24, 329.58 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780039/1807941 [23:24<01:22, 337.67 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780093/1807941 [23:24<01:21, 342.13 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780132/1807941 [23:24<01:19, 349.25 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780175/1807941 [23:24<01:15, 366.40 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780213/1807941 [23:24<01:16, 364.37 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780270/1807941 [23:25<01:15, 365.05 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780319/1807941 [23:25<01:18, 351.61 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780376/1807941 [23:25<01:17, 356.27 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780416/1807941 [23:25<01:15, 362.60 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780470/1807941 [23:25<01:16, 358.15 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780508/1807941 [23:25<01:15, 361.03 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780548/1807941 [23:25<01:14, 370.04 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780598/1807941 [23:25<01:08, 399.62 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780650/1807941 [23:26<01:12, 375.64 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780689/1807941 [23:26<01:13, 372.87 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780748/1807941 [23:26<01:12, 377.63 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780804/1807941 [23:26<01:13, 369.59 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1780855/1807941 [23:26<01:15, 357.91 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1780906/1807941 [23:26<01:17, 348.25 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1780972/1807941 [23:26<01:04, 416.53 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781021/1807941 [23:27<01:14, 361.97 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781068/1807941 [23:27<01:17, 344.92 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781120/1807941 [23:27<01:18, 341.65 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781173/1807941 [23:27<01:18, 342.21 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781225/1807941 [23:27<01:18, 340.43 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781261/1807941 [23:27<01:17, 342.44 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781302/1807941 [23:27<01:15, 355.17 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781351/1807941 [23:27<01:08, 388.01 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781398/1807941 [23:28<01:06, 400.14 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781439/1807941 [23:28<01:06, 398.94 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781497/1807941 [23:28<01:08, 387.92 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781553/1807941 [23:28<01:10, 376.92 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781607/1807941 [23:28<01:11, 367.80 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781658/1807941 [23:28<01:13, 357.19 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781711/1807941 [23:28<01:14, 353.96 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781771/1807941 [23:29<01:11, 363.77 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781824/1807941 [23:29<01:05, 398.75 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781939/1807941 [23:29<00:44, 578.53 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782027/1807941 [23:29<00:46, 558.51 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782164/1807941 [23:29<00:34, 749.07 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782259/1807941 [23:29<00:40, 633.96 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782334/1807941 [23:29<00:47, 533.88 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782409/1807941 [23:30<00:53, 476.11 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782466/1807941 [23:30<00:57, 442.29 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782529/1807941 [23:30<00:58, 433.19 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782609/1807941 [23:30<00:55, 459.68 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782673/1807941 [23:30<00:51, 494.79 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782730/1807941 [23:30<00:49, 510.16 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782788/1807941 [23:30<00:47, 524.46 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782852/1807941 [23:31<00:45, 551.83 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782938/1807941 [23:31<00:45, 554.39 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783000/1807941 [23:31<00:51, 481.62 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783058/1807941 [23:31<00:55, 445.57 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783110/1807941 [23:31<01:00, 409.56 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783198/1807941 [23:31<00:48, 513.40 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783287/1807941 [23:31<00:41, 596.42 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783381/1807941 [23:32<00:40, 602.80 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783465/1807941 [23:32<00:51, 475.42 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783525/1807941 [23:32<00:49, 494.92 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783581/1807941 [23:32<00:58, 413.10 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783630/1807941 [23:32<01:02, 386.44 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783679/1807941 [23:32<01:06, 364.36 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783746/1807941 [23:33<00:56, 426.64 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783802/1807941 [23:33<01:05, 367.26 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783858/1807941 [23:33<01:05, 366.43 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783914/1807941 [23:33<01:05, 366.69 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783968/1807941 [23:33<01:06, 361.23 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784018/1807941 [23:33<01:16, 311.79 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784069/1807941 [23:34<01:15, 316.56 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784104/1807941 [23:34<01:13, 322.82 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784154/1807941 [23:34<01:14, 320.62 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784217/1807941 [23:34<01:01, 387.05 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784261/1807941 [23:34<01:07, 350.55 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784314/1807941 [23:34<01:07, 348.39 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784369/1807941 [23:34<01:07, 348.95 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784409/1807941 [23:35<01:05, 358.83 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784448/1807941 [23:35<01:04, 364.49 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784486/1807941 [23:35<01:03, 366.99 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784533/1807941 [23:35<00:59, 391.52 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784584/1807941 [23:35<00:55, 419.81 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784634/1807941 [23:35<01:00, 382.12 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784683/1807941 [23:35<01:04, 361.75 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784736/1807941 [23:35<01:05, 355.19 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784782/1807941 [23:36<01:08, 335.67 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784830/1807941 [23:36<01:11, 323.73 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784879/1807941 [23:36<01:11, 321.72 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784926/1807941 [23:36<01:12, 318.22 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784961/1807941 [23:36<01:11, 322.77 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1785000/1807941 [23:36<01:18, 291.77 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1785050/1807941 [23:36<01:07, 337.12 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1785095/1807941 [23:37<01:11, 320.20 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1785142/1807941 [23:37<01:12, 314.98 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1785176/1807941 [23:37<01:11, 319.46 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1785209/1807941 [23:37<01:11, 319.13 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1785242/1807941 [23:37<01:11, 318.10 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1785278/1807941 [23:37<01:10, 323.29 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1785313/1807941 [23:37<01:09, 326.16 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785359/1807941 [23:37<01:11, 315.93 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785397/1807941 [23:37<01:08, 330.95 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785448/1807941 [23:38<00:59, 375.99 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785501/1807941 [23:38<01:02, 360.09 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785550/1807941 [23:38<01:05, 342.91 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785598/1807941 [23:38<01:07, 331.74 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785647/1807941 [23:38<01:08, 325.27 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785697/1807941 [23:38<01:08, 323.30 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785732/1807941 [23:38<01:08, 326.27 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785783/1807941 [23:39<01:08, 325.40 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785821/1807941 [23:39<01:05, 336.37 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785871/1807941 [23:39<01:06, 329.96 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785963/1807941 [23:39<00:52, 415.46 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786014/1807941 [23:39<01:02, 349.55 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786058/1807941 [23:39<01:06, 329.60 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786092/1807941 [23:40<01:06, 327.74 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786141/1807941 [23:40<01:07, 324.57 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786174/1807941 [23:40<01:07, 321.39 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786218/1807941 [23:40<01:10, 310.15 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786266/1807941 [23:40<01:10, 308.56 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786299/1807941 [23:40<01:09, 309.88 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786333/1807941 [23:40<01:08, 314.35 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786387/1807941 [23:40<01:05, 327.83 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786456/1807941 [23:41<00:51, 414.86 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786506/1807941 [23:41<00:55, 383.26 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786550/1807941 [23:41<01:00, 351.73 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786599/1807941 [23:41<01:02, 341.38 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786648/1807941 [23:41<01:04, 331.60 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786697/1807941 [23:41<01:05, 323.95 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786746/1807941 [23:41<01:06, 320.25 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786781/1807941 [23:42<01:05, 323.63 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786885/1807941 [23:42<00:43, 488.79 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787000/1807941 [23:42<00:36, 572.63 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787130/1807941 [23:42<00:31, 660.11 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787239/1807941 [23:42<00:27, 756.55 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787323/1807941 [23:42<00:30, 687.22 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787407/1807941 [23:43<00:39, 521.46 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787468/1807941 [23:43<00:46, 441.64 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787522/1807941 [23:43<00:49, 415.97 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787569/1807941 [23:43<00:52, 385.89 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787615/1807941 [23:43<00:56, 360.78 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787665/1807941 [23:43<00:57, 350.79 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787723/1807941 [23:44<00:56, 358.92 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787782/1807941 [23:44<00:49, 406.25 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787836/1807941 [23:44<00:46, 434.22 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787889/1807941 [23:44<00:43, 457.04 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787943/1807941 [23:44<00:41, 477.05 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787998/1807941 [23:44<00:40, 494.42 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1788054/1807941 [23:44<00:45, 439.35 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1788104/1807941 [23:44<00:43, 451.52 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1788165/1807941 [23:44<00:40, 492.64 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1788236/1807941 [23:44<00:36, 545.68 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1788306/1807941 [23:45<00:43, 454.57 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1788358/1807941 [23:45<00:41, 469.52 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1788435/1807941 [23:45<00:36, 540.50 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1788559/1807941 [23:45<00:26, 722.44 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1788671/1807941 [23:45<00:23, 828.98 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1788790/1807941 [23:45<00:20, 928.80 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1788914/1807941 [23:45<00:18, 1013.70 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1789048/1807941 [23:45<00:20, 924.59 examples/s] 
Tokenizing train dataset:  99%|█████████▉| 1789178/1807941 [23:46<00:23, 809.21 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1789279/1807941 [23:46<00:27, 690.75 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1789361/1807941 [23:46<00:31, 588.59 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1789449/1807941 [23:46<00:31, 586.44 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1789524/1807941 [23:46<00:36, 510.92 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1789583/1807941 [23:47<00:42, 433.86 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1789650/1807941 [23:47<00:38, 474.99 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1789730/1807941 [23:47<00:33, 537.86 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1789798/1807941 [23:47<00:43, 420.10 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1789866/1807941 [23:47<00:38, 469.65 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1789936/1807941 [23:47<00:34, 514.50 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1789999/1807941 [23:48<00:41, 434.28 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1790073/1807941 [23:48<00:40, 441.98 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1790127/1807941 [23:48<00:38, 458.13 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1790185/1807941 [23:48<00:40, 434.27 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1790274/1807941 [23:48<00:32, 536.68 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1790336/1807941 [23:48<00:35, 490.74 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1790404/1807941 [23:48<00:32, 533.56 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1790487/1807941 [23:48<00:28, 605.09 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1790593/1807941 [23:49<00:30, 568.65 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1790715/1807941 [23:49<00:24, 716.83 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1790830/1807941 [23:49<00:25, 659.04 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1790938/1807941 [23:49<00:22, 748.76 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1791052/1807941 [23:49<00:26, 627.49 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1791171/1807941 [23:49<00:22, 737.52 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1791260/1807941 [23:50<00:24, 683.72 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1791342/1807941 [23:50<00:25, 641.04 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1791437/1807941 [23:50<00:25, 636.24 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1791510/1807941 [23:50<00:25, 652.45 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1791599/1807941 [23:50<00:26, 624.65 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1791679/1807941 [23:50<00:27, 584.53 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1791761/1807941 [23:50<00:28, 572.06 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1791907/1807941 [23:50<00:20, 769.73 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1792010/1807941 [23:51<00:19, 831.00 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1792154/1807941 [23:51<00:16, 986.25 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1792311/1807941 [23:51<00:13, 1140.01 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1792459/1807941 [23:51<00:12, 1230.41 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1792642/1807941 [23:51<00:12, 1218.96 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1792822/1807941 [23:51<00:11, 1367.82 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1792999/1807941 [23:51<00:10, 1475.13 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1793224/1807941 [23:51<00:09, 1480.87 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1793406/1807941 [23:52<00:09, 1564.54 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1793580/1807941 [23:52<00:08, 1609.14 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1793768/1807941 [23:52<00:10, 1303.78 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1793915/1807941 [23:52<00:14, 982.95 examples/s] 
Tokenizing train dataset:  99%|█████████▉| 1794039/1807941 [23:52<00:17, 775.81 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1794149/1807941 [23:53<00:21, 656.26 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1794256/1807941 [23:53<00:23, 575.12 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1794351/1807941 [23:53<00:21, 632.93 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1794439/1807941 [23:53<00:25, 528.76 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1794503/1807941 [23:53<00:24, 545.33 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1794588/1807941 [23:53<00:22, 602.98 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1794672/1807941 [23:54<00:22, 582.71 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1794766/1807941 [23:54<00:24, 535.92 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1794849/1807941 [23:54<00:22, 592.98 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1794928/1807941 [23:54<00:20, 631.69 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1795000/1807941 [23:54<00:21, 607.73 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1795072/1807941 [23:54<00:25, 503.31 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1795144/1807941 [23:54<00:23, 548.72 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1795230/1807941 [23:55<00:20, 620.88 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1795316/1807941 [23:55<00:18, 675.55 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1795399/1807941 [23:55<00:22, 556.71 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1795491/1807941 [23:55<00:19, 636.89 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1795577/1807941 [23:55<00:17, 688.17 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1795675/1807941 [23:55<00:20, 598.89 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1795776/1807941 [23:55<00:19, 617.09 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1795868/1807941 [23:55<00:17, 682.62 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1795942/1807941 [23:56<00:19, 618.65 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1796042/1807941 [23:56<00:19, 601.99 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1796144/1807941 [23:56<00:17, 691.87 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1796222/1807941 [23:56<00:18, 628.36 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1796322/1807941 [23:56<00:16, 712.62 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1796399/1807941 [23:56<00:15, 725.53 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1796476/1807941 [23:56<00:15, 730.90 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1796570/1807941 [23:57<00:16, 692.46 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1796679/1807941 [23:57<00:17, 627.50 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1796812/1807941 [23:57<00:15, 699.58 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1796940/1807941 [23:57<00:13, 821.06 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1797081/1807941 [23:57<00:12, 903.07 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1797255/1807941 [23:57<00:09, 1100.79 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1797396/1807941 [23:57<00:08, 1177.28 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1797567/1807941 [23:57<00:07, 1318.26 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1797714/1807941 [23:58<00:10, 952.33 examples/s] 
Tokenizing train dataset:  99%|█████████▉| 1797838/1807941 [23:58<00:12, 828.69 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1797942/1807941 [23:58<00:11, 869.43 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1798080/1807941 [23:58<00:10, 955.42 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1798244/1807941 [23:58<00:08, 1116.06 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1798411/1807941 [23:58<00:07, 1254.84 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1798582/1807941 [23:58<00:06, 1373.44 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1798751/1807941 [23:58<00:06, 1458.39 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1798926/1807941 [23:59<00:05, 1539.22 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1799166/1807941 [23:59<00:05, 1486.78 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1799396/1807941 [23:59<00:05, 1499.73 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1799564/1807941 [23:59<00:05, 1540.77 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1799724/1807941 [23:59<00:05, 1553.77 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1799886/1807941 [23:59<00:05, 1567.82 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1800089/1807941 [23:59<00:05, 1485.61 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1800316/1807941 [24:00<00:05, 1490.54 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1800475/1807941 [24:00<00:04, 1511.08 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1800636/1807941 [24:00<00:04, 1535.29 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1800868/1807941 [24:00<00:04, 1535.36 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1801083/1807941 [24:00<00:04, 1496.29 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1801309/1807941 [24:00<00:04, 1495.95 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1801470/1807941 [24:00<00:04, 1520.86 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1801630/1807941 [24:00<00:04, 1539.54 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1801853/1807941 [24:01<00:04, 1515.33 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1802069/1807941 [24:01<00:04, 1455.61 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1802227/1807941 [24:01<00:03, 1482.74 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1802381/1807941 [24:01<00:03, 1492.82 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1802538/1807941 [24:01<00:03, 1511.74 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1802699/1807941 [24:01<00:03, 1535.18 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1802933/1807941 [24:01<00:03, 1523.82 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1803159/1807941 [24:01<00:03, 1468.85 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1803378/1807941 [24:02<00:03, 1461.96 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1803539/1807941 [24:02<00:02, 1496.18 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1803692/1807941 [24:02<00:02, 1501.98 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1803850/1807941 [24:02<00:02, 1521.14 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1804071/1807941 [24:02<00:02, 1422.04 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1804227/1807941 [24:02<00:02, 1452.71 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1804383/1807941 [24:02<00:02, 1476.29 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1804607/1807941 [24:02<00:02, 1478.04 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1804766/1807941 [24:02<00:02, 1505.94 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1804925/1807941 [24:03<00:01, 1527.67 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1805148/1807941 [24:03<00:01, 1431.46 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1805303/1807941 [24:03<00:01, 1458.31 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1805455/1807941 [24:03<00:01, 1472.24 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1805607/1807941 [24:03<00:02, 949.28 examples/s] 
Tokenizing train dataset: 100%|█████████▉| 1805735/1807941 [24:04<00:03, 665.63 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1805832/1807941 [24:04<00:03, 612.12 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1805921/1807941 [24:04<00:03, 655.18 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1806031/1807941 [24:04<00:03, 556.64 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1806125/1807941 [24:04<00:03, 569.51 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1806205/1807941 [24:05<00:03, 559.23 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1806268/1807941 [24:05<00:02, 570.96 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1806352/1807941 [24:05<00:02, 561.50 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1806421/1807941 [24:05<00:02, 585.83 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1806563/1807941 [24:05<00:01, 779.30 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1806662/1807941 [24:05<00:01, 827.35 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1806782/1807941 [24:05<00:01, 814.95 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1806932/1807941 [24:05<00:01, 983.17 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1807070/1807941 [24:05<00:00, 993.03 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1807216/1807941 [24:06<00:00, 1109.19 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1807359/1807941 [24:06<00:00, 1193.24 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1807500/1807941 [24:06<00:00, 1247.40 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1807646/1807941 [24:06<00:00, 1305.29 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1807790/1807941 [24:06<00:00, 1341.05 examples/s]
Tokenizing train dataset: 100%|██████████| 1807941/1807941 [24:06<00:00, 1219.98 examples/s]
Tokenizing train dataset: 100%|██████████| 1807941/1807941 [24:06<00:00, 1249.74 examples/s]
[transformers] The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.
Starting training...

  0%|          | 0/12000 [00:00<?, ?it/s]
  0%|          | 1/12000 [00:07<24:23:47,  7.32s/it]
                                                    

  0%|          | 1/12000 [00:07<24:23:47,  7.32s/it]
  0%|          | 2/12000 [00:09<13:21:47,  4.01s/it]
                                                    

  0%|          | 2/12000 [00:09<13:21:47,  4.01s/it]
  0%|          | 3/12000 [00:10<10:07:17,  3.04s/it]
                                                    

  0%|          | 3/12000 [00:10<10:07:17,  3.04s/it]
  0%|          | 4/12000 [00:12<8:35:04,  2.58s/it] 
                                                   

  0%|          | 4/12000 [00:12<8:35:04,  2.58s/it]
  0%|          | 5/12000 [00:14<7:36:05,  2.28s/it]
                                                   

  0%|          | 5/12000 [00:14<7:36:05,  2.28s/it]
  0%|          | 6/12000 [00:16<6:57:24,  2.09s/it]
                                                   

  0%|          | 6/12000 [00:16<6:57:24,  2.09s/it]
  0%|          | 7/12000 [00:17<6:15:10,  1.88s/it]
                                                   

  0%|          | 7/12000 [00:17<6:15:10,  1.88s/it]
  0%|          | 8/12000 [00:20<7:06:13,  2.13s/it]
                                                   

  0%|          | 8/12000 [00:20<7:06:13,  2.13s/it]
  0%|          | 9/12000 [00:21<6:26:28,  1.93s/it]
                                                   

  0%|          | 9/12000 [00:21<6:26:28,  1.93s/it]
  0%|          | 10/12000 [00:26<8:54:23,  2.67s/it]
                                                    

  0%|          | 10/12000 [00:26<8:54:23,  2.67s/it]
  0%|          | 11/12000 [00:27<7:42:58,  2.32s/it]
                                                    

  0%|          | 11/12000 [00:27<7:42:58,  2.32s/it]
  0%|          | 12/12000 [00:30<8:02:34,  2.42s/it]
                                                    

  0%|          | 12/12000 [00:30<8:02:34,  2.42s/it]
  0%|          | 13/12000 [00:32<7:44:14,  2.32s/it]
                                                    

  0%|          | 13/12000 [00:32<7:44:14,  2.32s/it]
  0%|          | 14/12000 [00:34<7:08:18,  2.14s/it]
                                                    

  0%|          | 14/12000 [00:34<7:08:18,  2.14s/it]
  0%|          | 15/12000 [00:35<6:37:20,  1.99s/it]
                                                    

  0%|          | 15/12000 [00:35<6:37:20,  1.99s/it]
  0%|          | 16/12000 [00:38<6:59:25,  2.10s/it]
                                                    

  0%|          | 16/12000 [00:38<6:59:25,  2.10s/it]
  0%|          | 17/12000 [00:39<6:26:57,  1.94s/it]
                                                    

  0%|          | 17/12000 [00:39<6:26:57,  1.94s/it]
  0%|          | 18/12000 [00:42<7:01:27,  2.11s/it]
                                                    

  0%|          | 18/12000 [00:42<7:01:27,  2.11s/it]
  0%|          | 19/12000 [00:44<7:04:34,  2.13s/it]
                                                    

  0%|          | 19/12000 [00:44<7:04:34,  2.13s/it]
  0%|          | 20/12000 [00:48<9:18:36,  2.80s/it]
                                                    

  0%|          | 20/12000 [00:48<9:18:36,  2.80s/it]
  0%|          | 21/12000 [00:50<8:06:41,  2.44s/it]
                                                    

  0%|          | 21/12000 [00:50<8:06:41,  2.44s/it]
  0%|          | 22/12000 [00:54<9:19:46,  2.80s/it]
                                                    

  0%|          | 22/12000 [00:54<9:19:46,  2.80s/it]
  0%|          | 23/12000 [00:55<8:09:30,  2.45s/it]
                                                    

  0%|          | 23/12000 [00:55<8:09:30,  2.45s/it]
  0%|          | 24/12000 [00:57<7:41:51,  2.31s/it]
                                                    

  0%|          | 24/12000 [00:57<7:41:51,  2.31s/it]
  0%|          | 25/12000 [00:59<6:52:54,  2.07s/it]
                                                    

  0%|          | 25/12000 [00:59<6:52:54,  2.07s/it]
  0%|          | 26/12000 [01:01<6:51:39,  2.06s/it]
                                                    

  0%|          | 26/12000 [01:01<6:51:39,  2.06s/it]
  0%|          | 27/12000 [01:03<7:19:43,  2.20s/it]
                                                    

  0%|          | 27/12000 [01:03<7:19:43,  2.20s/it]
  0%|          | 28/12000 [01:05<6:36:24,  1.99s/it]
                                                    

  0%|          | 28/12000 [01:05<6:36:24,  1.99s/it]
  0%|          | 29/12000 [01:06<6:21:16,  1.91s/it]
                                                    

  0%|          | 29/12000 [01:06<6:21:16,  1.91s/it]
  0%|          | 30/12000 [01:08<6:20:18,  1.91s/it]
                                                    

  0%|          | 30/12000 [01:08<6:20:18,  1.91s/it]
  0%|          | 31/12000 [01:10<6:12:56,  1.87s/it]
                                                    

  0%|          | 31/12000 [01:10<6:12:56,  1.87s/it]
  0%|          | 32/12000 [01:12<5:55:26,  1.78s/it]
                                                    

  0%|          | 32/12000 [01:12<5:55:26,  1.78s/it]
  0%|          | 33/12000 [01:13<5:44:18,  1.73s/it]
                                                    

  0%|          | 33/12000 [01:13<5:44:18,  1.73s/it]
  0%|          | 34/12000 [01:15<5:38:57,  1.70s/it]
                                                    

  0%|          | 34/12000 [01:15<5:38:57,  1.70s/it]
  0%|          | 35/12000 [01:17<5:39:01,  1.70s/it]
                                                    

  0%|          | 35/12000 [01:17<5:39:01,  1.70s/it]
  0%|          | 36/12000 [01:19<6:20:30,  1.91s/it]
                                                    

  0%|          | 36/12000 [01:19<6:20:30,  1.91s/it]
  0%|          | 37/12000 [01:21<6:49:29,  2.05s/it]
                                                    

  0%|          | 37/12000 [01:21<6:49:29,  2.05s/it]
  0%|          | 38/12000 [01:24<6:56:23,  2.09s/it]
                                                    

  0%|          | 38/12000 [01:24<6:56:23,  2.09s/it]
  0%|          | 39/12000 [01:25<6:30:46,  1.96s/it]
                                                    

  0%|          | 39/12000 [01:25<6:30:46,  1.96s/it]
  0%|          | 40/12000 [01:27<6:08:20,  1.85s/it]
                                                    

  0%|          | 40/12000 [01:27<6:08:20,  1.85s/it]
  0%|          | 41/12000 [01:28<5:51:28,  1.76s/it]
                                                    

  0%|          | 41/12000 [01:28<5:51:28,  1.76s/it]
  0%|          | 42/12000 [01:30<5:36:00,  1.69s/it]
                                                    

  0%|          | 42/12000 [01:30<5:36:00,  1.69s/it]
  0%|          | 43/12000 [01:32<5:50:46,  1.76s/it]
                                                    

  0%|          | 43/12000 [01:32<5:50:46,  1.76s/it]
  0%|          | 44/12000 [01:35<7:30:51,  2.26s/it]
                                                    

  0%|          | 44/12000 [01:35<7:30:51,  2.26s/it]
  0%|          | 45/12000 [01:37<7:08:13,  2.15s/it]
                                                    

  0%|          | 45/12000 [01:37<7:08:13,  2.15s/it]
  0%|          | 46/12000 [01:39<6:28:56,  1.95s/it]
                                                    

  0%|          | 46/12000 [01:39<6:28:56,  1.95s/it]
  0%|          | 47/12000 [01:40<6:03:57,  1.83s/it]
                                                    

  0%|          | 47/12000 [01:40<6:03:57,  1.83s/it]
  0%|          | 48/12000 [01:43<6:47:44,  2.05s/it]
                                                    

  0%|          | 48/12000 [01:43<6:47:44,  2.05s/it]
  0%|          | 49/12000 [01:45<6:46:51,  2.04s/it]
                                                    
{'loss': '3.205', 'grad_norm': '6.059', 'learning_rate': '0', 'entropy': '0.9085', 'num_tokens': '4939', 'mean_token_accuracy': '0.5347', 'epoch': '8.85e-06'}
{'loss': '3.145', 'grad_norm': '5.534', 'learning_rate': '3.333e-07', 'entropy': '0.9753', 'num_tokens': '9769', 'mean_token_accuracy': '0.5025', 'epoch': '1.77e-05'}
{'loss': '2.999', 'grad_norm': '6.69', 'learning_rate': '6.667e-07', 'entropy': '0.869', 'num_tokens': '1.429e+04', 'mean_token_accuracy': '0.5415', 'epoch': '2.655e-05'}
{'loss': '3.418', 'grad_norm': '6.557', 'learning_rate': '1e-06', 'entropy': '0.9978', 'num_tokens': '1.916e+04', 'mean_token_accuracy': '0.4883', 'epoch': '3.54e-05'}
{'loss': '3.601', 'grad_norm': '6.91', 'learning_rate': '1.333e-06', 'entropy': '1.041', 'num_tokens': '2.394e+04', 'mean_token_accuracy': '0.4931', 'epoch': '4.425e-05'}
{'loss': '3.327', 'grad_norm': '5.84', 'learning_rate': '1.667e-06', 'entropy': '0.9279', 'num_tokens': '2.835e+04', 'mean_token_accuracy': '0.523', 'epoch': '5.31e-05'}
{'loss': '3.24', 'grad_norm': '7.269', 'learning_rate': '2e-06', 'entropy': '0.9057', 'num_tokens': '3.218e+04', 'mean_token_accuracy': '0.5238', 'epoch': '6.195e-05'}
{'loss': '2.639', 'grad_norm': '4.997', 'learning_rate': '2.333e-06', 'entropy': '0.8669', 'num_tokens': '3.859e+04', 'mean_token_accuracy': '0.5414', 'epoch': '7.08e-05'}
{'loss': '3.267', 'grad_norm': '5.593', 'learning_rate': '2.667e-06', 'entropy': '0.9014', 'num_tokens': '4.314e+04', 'mean_token_accuracy': '0.5254', 'epoch': '7.965e-05'}
{'loss': '2.195', 'grad_norm': '4.681', 'learning_rate': '3e-06', 'entropy': '0.8373', 'num_tokens': '5.108e+04', 'mean_token_accuracy': '0.6024', 'epoch': '8.85e-05'}
{'loss': '3.389', 'grad_norm': '7.924', 'learning_rate': '3.333e-06', 'entropy': '1.055', 'num_tokens': '5.552e+04', 'mean_token_accuracy': '0.482', 'epoch': '9.735e-05'}
{'loss': '2.57', 'grad_norm': '4.376', 'learning_rate': '3.667e-06', 'entropy': '0.9588', 'num_tokens': '6.28e+04', 'mean_token_accuracy': '0.5517', 'epoch': '0.0001062'}
{'loss': '3.131', 'grad_norm': '6.556', 'learning_rate': '4e-06', 'entropy': '1.011', 'num_tokens': '6.86e+04', 'mean_token_accuracy': '0.5201', 'epoch': '0.000115'}
{'loss': '3.163', 'grad_norm': '6.193', 'learning_rate': '4.333e-06', 'entropy': '1.079', 'num_tokens': '7.386e+04', 'mean_token_accuracy': '0.5092', 'epoch': '0.0001239'}
{'loss': '3.51', 'grad_norm': '7.178', 'learning_rate': '4.667e-06', 'entropy': '1.282', 'num_tokens': '7.825e+04', 'mean_token_accuracy': '0.4408', 'epoch': '0.0001327'}
{'loss': '3.036', 'grad_norm': '4.833', 'learning_rate': '5e-06', 'entropy': '1.21', 'num_tokens': '8.414e+04', 'mean_token_accuracy': '0.4794', 'epoch': '0.0001416'}
{'loss': '2.85', 'grad_norm': '3.902', 'learning_rate': '5.333e-06', 'entropy': '1.168', 'num_tokens': '8.835e+04', 'mean_token_accuracy': '0.5207', 'epoch': '0.0001504'}
{'loss': '2.386', 'grad_norm': '2.445', 'learning_rate': '5.667e-06', 'entropy': '1.118', 'num_tokens': '9.598e+04', 'mean_token_accuracy': '0.5542', 'epoch': '0.0001593'}
{'loss': '2.969', 'grad_norm': '4.012', 'learning_rate': '6e-06', 'entropy': '1.174', 'num_tokens': '1.015e+05', 'mean_token_accuracy': '0.5074', 'epoch': '0.0001681'}
{'loss': '2.293', 'grad_norm': '2.027', 'learning_rate': '6.333e-06', 'entropy': '1.118', 'num_tokens': '1.109e+05', 'mean_token_accuracy': '0.5785', 'epoch': '0.000177'}
{'loss': '2.888', 'grad_norm': '3.434', 'learning_rate': '6.667e-06', 'entropy': '1.252', 'num_tokens': '1.155e+05', 'mean_token_accuracy': '0.4969', 'epoch': '0.0001858'}
{'loss': '2.704', 'grad_norm': '2.952', 'learning_rate': '7e-06', 'entropy': '1.177', 'num_tokens': '1.194e+05', 'mean_token_accuracy': '0.5436', 'epoch': '0.0001947'}
{'loss': '2.838', 'grad_norm': '2.822', 'learning_rate': '7.333e-06', 'entropy': '1.373', 'num_tokens': '1.237e+05', 'mean_token_accuracy': '0.5096', 'epoch': '0.0002035'}
{'loss': '2.706', 'grad_norm': '2.51', 'learning_rate': '7.667e-06', 'entropy': '1.368', 'num_tokens': '1.295e+05', 'mean_token_accuracy': '0.5028', 'epoch': '0.0002124'}
{'loss': '2.683', 'grad_norm': '2.562', 'learning_rate': '8e-06', 'entropy': '1.362', 'num_tokens': '1.332e+05', 'mean_token_accuracy': '0.526', 'epoch': '0.0002212'}
{'loss': '2.525', 'grad_norm': '2.009', 'learning_rate': '8.333e-06', 'entropy': '1.331', 'num_tokens': '1.401e+05', 'mean_token_accuracy': '0.5335', 'epoch': '0.0002301'}
{'loss': '2.418', 'grad_norm': '2.11', 'learning_rate': '8.667e-06', 'entropy': '1.271', 'num_tokens': '1.468e+05', 'mean_token_accuracy': '0.5535', 'epoch': '0.0002389'}
{'loss': '2.798', 'grad_norm': '2.31', 'learning_rate': '9e-06', 'entropy': '1.517', 'num_tokens': '1.511e+05', 'mean_token_accuracy': '0.4941', 'epoch': '0.0002478'}
{'loss': '2.458', 'grad_norm': '1.933', 'learning_rate': '9.333e-06', 'entropy': '1.386', 'num_tokens': '1.563e+05', 'mean_token_accuracy': '0.5511', 'epoch': '0.0002566'}
{'loss': '2.087', 'grad_norm': '1.47', 'learning_rate': '9.667e-06', 'entropy': '1.354', 'num_tokens': '1.62e+05', 'mean_token_accuracy': '0.5613', 'epoch': '0.0002655'}
{'loss': '2.456', 'grad_norm': '1.51', 'learning_rate': '1e-05', 'entropy': '1.637', 'num_tokens': '1.669e+05', 'mean_token_accuracy': '0.516', 'epoch': '0.0002743'}
{'loss': '2.244', 'grad_norm': '1.592', 'learning_rate': '1.033e-05', 'entropy': '1.485', 'num_tokens': '1.714e+05', 'mean_token_accuracy': '0.5591', 'epoch': '0.0002832'}
{'loss': '2.256', 'grad_norm': '1.465', 'learning_rate': '1.067e-05', 'entropy': '1.694', 'num_tokens': '1.759e+05', 'mean_token_accuracy': '0.514', 'epoch': '0.000292'}
{'loss': '2.638', 'grad_norm': '1.383', 'learning_rate': '1.1e-05', 'entropy': '2.01', 'num_tokens': '1.804e+05', 'mean_token_accuracy': '0.4569', 'epoch': '0.0003009'}
{'loss': '2.303', 'grad_norm': '1.218', 'learning_rate': '1.133e-05', 'entropy': '1.669', 'num_tokens': '1.86e+05', 'mean_token_accuracy': '0.5396', 'epoch': '0.0003097'}
{'loss': '1.875', 'grad_norm': '0.9926', 'learning_rate': '1.167e-05', 'entropy': '1.495', 'num_tokens': '1.922e+05', 'mean_token_accuracy': '0.5942', 'epoch': '0.0003186'}
{'loss': '2.475', 'grad_norm': '1.074', 'learning_rate': '1.2e-05', 'entropy': '1.912', 'num_tokens': '1.977e+05', 'mean_token_accuracy': '0.5077', 'epoch': '0.0003274'}
{'loss': '1.941', 'grad_norm': '0.9737', 'learning_rate': '1.233e-05', 'entropy': '1.622', 'num_tokens': '2.028e+05', 'mean_token_accuracy': '0.5698', 'epoch': '0.0003363'}
{'loss': '2.114', 'grad_norm': '0.9115', 'learning_rate': '1.267e-05', 'entropy': '1.824', 'num_tokens': '2.081e+05', 'mean_token_accuracy': '0.5411', 'epoch': '0.0003451'}
{'loss': '2.139', 'grad_norm': '0.9518', 'learning_rate': '1.3e-05', 'entropy': '1.804', 'num_tokens': '2.127e+05', 'mean_token_accuracy': '0.5607', 'epoch': '0.000354'}
{'loss': '2.016', 'grad_norm': '0.8904', 'learning_rate': '1.333e-05', 'entropy': '1.785', 'num_tokens': '2.177e+05', 'mean_token_accuracy': '0.5811', 'epoch': '0.0003628'}
{'loss': '1.83', 'grad_norm': '0.9005', 'learning_rate': '1.367e-05', 'entropy': '1.762', 'num_tokens': '2.224e+05', 'mean_token_accuracy': '0.5957', 'epoch': '0.0003717'}
{'loss': '1.494', 'grad_norm': '0.6743', 'learning_rate': '1.4e-05', 'entropy': '1.468', 'num_tokens': '2.279e+05', 'mean_token_accuracy': '0.6692', 'epoch': '0.0003805'}
{'loss': '2.126', 'grad_norm': '0.948', 'learning_rate': '1.433e-05', 'entropy': '2.087', 'num_tokens': '2.316e+05', 'mean_token_accuracy': '0.5607', 'epoch': '0.0003894'}
{'loss': '1.646', 'grad_norm': '0.7979', 'learning_rate': '1.467e-05', 'entropy': '1.722', 'num_tokens': '2.366e+05', 'mean_token_accuracy': '0.6374', 'epoch': '0.0003982'}
{'loss': '2.105', 'grad_norm': '0.8808', 'learning_rate': '1.5e-05', 'entropy': '2.224', 'num_tokens': '2.403e+05', 'mean_token_accuracy': '0.5527', 'epoch': '0.0004071'}
{'loss': '2.026', 'grad_norm': '0.7745', 'learning_rate': '1.533e-05', 'entropy': '2.109', 'num_tokens': '2.447e+05', 'mean_token_accuracy': '0.562', 'epoch': '0.0004159'}
{'loss': '1.927', 'grad_norm': '0.7237', 'learning_rate': '1.567e-05', 'entropy': '1.978', 'num_tokens': '2.505e+05', 'mean_token_accuracy': '0.5839', 'epoch': '0.0004248'}

  0%|          | 49/12000 [01:45<6:46:51,  2.04s/it]
  0%|          | 50/12000 [01:46<6:19:48,  1.91s/it]
                                                    

  0%|          | 50/12000 [01:46<6:19:48,  1.91s/it]
  0%|          | 51/12000 [01:49<7:18:12,  2.20s/it]
                                                    

  0%|          | 51/12000 [01:49<7:18:12,  2.20s/it]
  0%|          | 52/12000 [01:51<6:48:55,  2.05s/it]
                                                    

  0%|          | 52/12000 [01:51<6:48:55,  2.05s/it]
  0%|          | 53/12000 [01:52<6:18:15,  1.90s/it]
                                                    

  0%|          | 53/12000 [01:52<6:18:15,  1.90s/it]
  0%|          | 54/12000 [01:54<5:51:00,  1.76s/it]
                                                    

  0%|          | 54/12000 [01:54<5:51:00,  1.76s/it]
  0%|          | 55/12000 [01:56<6:12:56,  1.87s/it]
                                                    

  0%|          | 55/12000 [01:56<6:12:56,  1.87s/it]
  0%|          | 56/12000 [01:58<5:46:46,  1.74s/it]
                                                    

  0%|          | 56/12000 [01:58<5:46:46,  1.74s/it]
  0%|          | 57/12000 [01:59<5:36:58,  1.69s/it]
                                                    

  0%|          | 57/12000 [01:59<5:36:58,  1.69s/it]
  0%|          | 58/12000 [02:01<5:39:47,  1.71s/it]
                                                    

  0%|          | 58/12000 [02:01<5:39:47,  1.71s/it]
  0%|          | 59/12000 [02:02<5:28:48,  1.65s/it]
                                                    

  0%|          | 59/12000 [02:02<5:28:48,  1.65s/it]
  0%|          | 60/12000 [02:05<6:23:21,  1.93s/it]
                                                    

  0%|          | 60/12000 [02:05<6:23:21,  1.93s/it]
  1%|          | 61/12000 [02:06<5:59:51,  1.81s/it]
                                                    

  1%|          | 61/12000 [02:06<5:59:51,  1.81s/it]
  1%|          | 62/12000 [02:08<5:46:09,  1.74s/it]
                                                    

  1%|          | 62/12000 [02:08<5:46:09,  1.74s/it]
  1%|          | 63/12000 [02:10<5:37:30,  1.70s/it]
                                                    

  1%|          | 63/12000 [02:10<5:37:30,  1.70s/it]
  1%|          | 64/12000 [02:12<6:37:39,  2.00s/it]
                                                    

  1%|          | 64/12000 [02:12<6:37:39,  2.00s/it]
  1%|          | 65/12000 [02:14<6:14:23,  1.88s/it]
                                                    

  1%|          | 65/12000 [02:14<6:14:23,  1.88s/it]
  1%|          | 66/12000 [02:18<8:30:16,  2.57s/it]
                                                    

  1%|          | 66/12000 [02:18<8:30:16,  2.57s/it]
  1%|          | 67/12000 [02:20<7:38:55,  2.31s/it]
                                                    

  1%|          | 67/12000 [02:20<7:38:55,  2.31s/it]
  1%|          | 68/12000 [02:21<6:53:38,  2.08s/it]
                                                    

  1%|          | 68/12000 [02:21<6:53:38,  2.08s/it]
  1%|          | 69/12000 [02:23<6:14:08,  1.88s/it]
                                                    

  1%|          | 69/12000 [02:23<6:14:08,  1.88s/it]
  1%|          | 70/12000 [02:24<5:48:24,  1.75s/it]
                                                    

  1%|          | 70/12000 [02:24<5:48:24,  1.75s/it]
  1%|          | 71/12000 [02:27<6:24:06,  1.93s/it]
                                                    

  1%|          | 71/12000 [02:27<6:24:06,  1.93s/it]
  1%|          | 72/12000 [02:28<6:20:40,  1.91s/it]
                                                    

  1%|          | 72/12000 [02:28<6:20:40,  1.91s/it]
  1%|          | 73/12000 [02:31<7:18:51,  2.21s/it]
                                                    

  1%|          | 73/12000 [02:31<7:18:51,  2.21s/it]
  1%|          | 74/12000 [02:33<6:33:57,  1.98s/it]
                                                    

  1%|          | 74/12000 [02:33<6:33:57,  1.98s/it]
  1%|          | 75/12000 [02:36<7:41:13,  2.32s/it]
                                                    

  1%|          | 75/12000 [02:36<7:41:13,  2.32s/it]
  1%|          | 76/12000 [02:38<7:51:50,  2.37s/it]
                                                    

  1%|          | 76/12000 [02:38<7:51:50,  2.37s/it]
  1%|          | 77/12000 [02:40<7:05:08,  2.14s/it]
                                                    

  1%|          | 77/12000 [02:40<7:05:08,  2.14s/it]
  1%|          | 78/12000 [02:41<6:21:01,  1.92s/it]
                                                    

  1%|          | 78/12000 [02:41<6:21:01,  1.92s/it]
  1%|          | 79/12000 [02:44<7:13:31,  2.18s/it]
                                                    

  1%|          | 79/12000 [02:44<7:13:31,  2.18s/it]
  1%|          | 80/12000 [02:46<6:53:09,  2.08s/it]
                                                    

  1%|          | 80/12000 [02:46<6:53:09,  2.08s/it]
  1%|          | 81/12000 [02:48<6:19:07,  1.91s/it]
                                                    

  1%|          | 81/12000 [02:48<6:19:07,  1.91s/it]
  1%|          | 82/12000 [02:49<6:03:33,  1.83s/it]
                                                    

  1%|          | 82/12000 [02:49<6:03:33,  1.83s/it]
  1%|          | 83/12000 [02:51<6:03:58,  1.83s/it]
                                                    

  1%|          | 83/12000 [02:51<6:03:58,  1.83s/it]
  1%|          | 84/12000 [02:53<5:56:30,  1.80s/it]
                                                    

  1%|          | 84/12000 [02:53<5:56:30,  1.80s/it]
  1%|          | 85/12000 [02:54<5:32:30,  1.67s/it]
                                                    

  1%|          | 85/12000 [02:54<5:32:30,  1.67s/it]
  1%|          | 86/12000 [02:56<5:46:38,  1.75s/it]
                                                    

  1%|          | 86/12000 [02:56<5:46:38,  1.75s/it]
  1%|          | 87/12000 [02:58<5:45:17,  1.74s/it]
                                                    

  1%|          | 87/12000 [02:58<5:45:17,  1.74s/it]
  1%|          | 88/12000 [03:03<9:12:52,  2.78s/it]
                                                    

  1%|          | 88/12000 [03:03<9:12:52,  2.78s/it]
  1%|          | 89/12000 [03:05<8:00:14,  2.42s/it]
                                                    

  1%|          | 89/12000 [03:05<8:00:14,  2.42s/it]
  1%|          | 90/12000 [03:06<7:11:45,  2.18s/it]
                                                    

  1%|          | 90/12000 [03:06<7:11:45,  2.18s/it]
  1%|          | 91/12000 [03:08<6:49:56,  2.07s/it]
                                                    

  1%|          | 91/12000 [03:08<6:49:56,  2.07s/it]
  1%|          | 92/12000 [03:09<6:16:24,  1.90s/it]
                                                    

  1%|          | 92/12000 [03:09<6:16:24,  1.90s/it]
  1%|          | 93/12000 [03:11<6:00:29,  1.82s/it]
                                                    

  1%|          | 93/12000 [03:11<6:00:29,  1.82s/it]
  1%|          | 94/12000 [03:13<5:48:01,  1.75s/it]
                                                    

  1%|          | 94/12000 [03:13<5:48:01,  1.75s/it]
  1%|          | 95/12000 [03:16<7:22:57,  2.23s/it]
                                                    

  1%|          | 95/12000 [03:16<7:22:57,  2.23s/it]
  1%|          | 96/12000 [03:18<7:23:00,  2.23s/it]
                                                    
{'loss': '1.828', 'grad_norm': '0.8266', 'learning_rate': '1.6e-05', 'entropy': '1.937', 'num_tokens': '2.555e+05', 'mean_token_accuracy': '0.5937', 'epoch': '0.0004336'}
{'loss': '1.821', 'grad_norm': '0.7985', 'learning_rate': '1.633e-05', 'entropy': '1.92', 'num_tokens': '2.607e+05', 'mean_token_accuracy': '0.5919', 'epoch': '0.0004425'}
{'loss': '1.834', 'grad_norm': '0.8177', 'learning_rate': '1.667e-05', 'entropy': '1.999', 'num_tokens': '2.652e+05', 'mean_token_accuracy': '0.5869', 'epoch': '0.0004513'}
{'loss': '1.877', 'grad_norm': '0.7584', 'learning_rate': '1.7e-05', 'entropy': '1.927', 'num_tokens': '2.701e+05', 'mean_token_accuracy': '0.6019', 'epoch': '0.0004602'}
{'loss': '1.955', 'grad_norm': '0.796', 'learning_rate': '1.733e-05', 'entropy': '2.157', 'num_tokens': '2.748e+05', 'mean_token_accuracy': '0.5598', 'epoch': '0.000469'}
{'loss': '1.866', 'grad_norm': '0.7243', 'learning_rate': '1.767e-05', 'entropy': '2.005', 'num_tokens': '2.801e+05', 'mean_token_accuracy': '0.5939', 'epoch': '0.0004779'}
{'loss': '1.522', 'grad_norm': '0.6284', 'learning_rate': '1.8e-05', 'entropy': '2.02', 'num_tokens': '2.855e+05', 'mean_token_accuracy': '0.5876', 'epoch': '0.0004867'}
{'loss': '2.234', 'grad_norm': '0.9775', 'learning_rate': '1.833e-05', 'entropy': '2.32', 'num_tokens': '2.886e+05', 'mean_token_accuracy': '0.5323', 'epoch': '0.0004956'}
{'loss': '1.7', 'grad_norm': '0.7154', 'learning_rate': '1.867e-05', 'entropy': '1.77', 'num_tokens': '2.935e+05', 'mean_token_accuracy': '0.6207', 'epoch': '0.0005044'}
{'loss': '1.713', 'grad_norm': '0.6798', 'learning_rate': '1.9e-05', 'entropy': '1.81', 'num_tokens': '2.989e+05', 'mean_token_accuracy': '0.619', 'epoch': '0.0005133'}
{'loss': '1.85', 'grad_norm': '0.7373', 'learning_rate': '1.933e-05', 'entropy': '1.859', 'num_tokens': '3.037e+05', 'mean_token_accuracy': '0.6087', 'epoch': '0.0005221'}
{'loss': '1.641', 'grad_norm': '0.5867', 'learning_rate': '1.967e-05', 'entropy': '1.781', 'num_tokens': '3.108e+05', 'mean_token_accuracy': '0.6143', 'epoch': '0.000531'}
{'loss': '1.468', 'grad_norm': '0.6993', 'learning_rate': '2e-05', 'entropy': '1.57', 'num_tokens': '3.149e+05', 'mean_token_accuracy': '0.6737', 'epoch': '0.0005398'}
{'loss': '1.458', 'grad_norm': '0.8003', 'learning_rate': '2.033e-05', 'entropy': '1.523', 'num_tokens': '3.203e+05', 'mean_token_accuracy': '0.6745', 'epoch': '0.0005487'}
{'loss': '1.567', 'grad_norm': '0.7458', 'learning_rate': '2.067e-05', 'entropy': '1.635', 'num_tokens': '3.245e+05', 'mean_token_accuracy': '0.6406', 'epoch': '0.0005575'}
{'loss': '1.447', 'grad_norm': '0.8656', 'learning_rate': '2.1e-05', 'entropy': '1.532', 'num_tokens': '3.317e+05', 'mean_token_accuracy': '0.6512', 'epoch': '0.0005664'}
{'loss': '1.905', 'grad_norm': '0.8861', 'learning_rate': '2.133e-05', 'entropy': '1.997', 'num_tokens': '3.355e+05', 'mean_token_accuracy': '0.5744', 'epoch': '0.0005752'}
{'loss': '1.431', 'grad_norm': '0.5977', 'learning_rate': '2.167e-05', 'entropy': '1.462', 'num_tokens': '3.421e+05', 'mean_token_accuracy': '0.6708', 'epoch': '0.0005841'}
{'loss': '1.543', 'grad_norm': '0.8554', 'learning_rate': '2.2e-05', 'entropy': '1.647', 'num_tokens': '3.473e+05', 'mean_token_accuracy': '0.6522', 'epoch': '0.0005929'}
{'loss': '1.597', 'grad_norm': '0.7926', 'learning_rate': '2.233e-05', 'entropy': '1.732', 'num_tokens': '3.519e+05', 'mean_token_accuracy': '0.628', 'epoch': '0.0006018'}
{'loss': '1.698', 'grad_norm': '0.9055', 'learning_rate': '2.267e-05', 'entropy': '1.819', 'num_tokens': '3.557e+05', 'mean_token_accuracy': '0.6124', 'epoch': '0.0006106'}
{'loss': '1.636', 'grad_norm': '0.9434', 'learning_rate': '2.3e-05', 'entropy': '1.642', 'num_tokens': '3.6e+05', 'mean_token_accuracy': '0.6518', 'epoch': '0.0006195'}
{'loss': '1.839', 'grad_norm': '0.8037', 'learning_rate': '2.333e-05', 'entropy': '1.834', 'num_tokens': '3.658e+05', 'mean_token_accuracy': '0.5916', 'epoch': '0.0006283'}
{'loss': '1.802', 'grad_norm': '0.8794', 'learning_rate': '2.367e-05', 'entropy': '1.711', 'num_tokens': '3.698e+05', 'mean_token_accuracy': '0.6055', 'epoch': '0.0006372'}
{'loss': '1.356', 'grad_norm': '0.7213', 'learning_rate': '2.4e-05', 'entropy': '1.531', 'num_tokens': '3.761e+05', 'mean_token_accuracy': '0.6381', 'epoch': '0.000646'}
{'loss': '1.638', 'grad_norm': '1.061', 'learning_rate': '2.433e-05', 'entropy': '1.611', 'num_tokens': '3.804e+05', 'mean_token_accuracy': '0.6334', 'epoch': '0.0006549'}
{'loss': '1.359', 'grad_norm': '0.681', 'learning_rate': '2.467e-05', 'entropy': '1.453', 'num_tokens': '3.875e+05', 'mean_token_accuracy': '0.667', 'epoch': '0.0006637'}
{'loss': '1.631', 'grad_norm': '0.7619', 'learning_rate': '2.5e-05', 'entropy': '1.661', 'num_tokens': '3.929e+05', 'mean_token_accuracy': '0.6251', 'epoch': '0.0006726'}
{'loss': '1.437', 'grad_norm': '0.8248', 'learning_rate': '2.533e-05', 'entropy': '1.518', 'num_tokens': '3.971e+05', 'mean_token_accuracy': '0.6762', 'epoch': '0.0006814'}
{'loss': '1.51', 'grad_norm': '1.13', 'learning_rate': '2.567e-05', 'entropy': '1.645', 'num_tokens': '4.015e+05', 'mean_token_accuracy': '0.6259', 'epoch': '0.0006903'}
{'loss': '1.136', 'grad_norm': '0.6262', 'learning_rate': '2.6e-05', 'entropy': '1.214', 'num_tokens': '4.09e+05', 'mean_token_accuracy': '0.7147', 'epoch': '0.0006991'}
{'loss': '1.379', 'grad_norm': '1.001', 'learning_rate': '2.633e-05', 'entropy': '1.381', 'num_tokens': '4.147e+05', 'mean_token_accuracy': '0.6791', 'epoch': '0.000708'}
{'loss': '1.179', 'grad_norm': '0.9168', 'learning_rate': '2.667e-05', 'entropy': '1.25', 'num_tokens': '4.19e+05', 'mean_token_accuracy': '0.7098', 'epoch': '0.0007168'}
{'loss': '1.262', 'grad_norm': '0.8791', 'learning_rate': '2.7e-05', 'entropy': '1.301', 'num_tokens': '4.242e+05', 'mean_token_accuracy': '0.7072', 'epoch': '0.0007257'}
{'loss': '1.45', 'grad_norm': '0.9843', 'learning_rate': '2.733e-05', 'entropy': '1.439', 'num_tokens': '4.294e+05', 'mean_token_accuracy': '0.6597', 'epoch': '0.0007345'}
{'loss': '1.54', 'grad_norm': '1.041', 'learning_rate': '2.767e-05', 'entropy': '1.521', 'num_tokens': '4.336e+05', 'mean_token_accuracy': '0.6463', 'epoch': '0.0007434'}
{'loss': '1.424', 'grad_norm': '1.557', 'learning_rate': '2.8e-05', 'entropy': '1.493', 'num_tokens': '4.372e+05', 'mean_token_accuracy': '0.6604', 'epoch': '0.0007522'}
{'loss': '1.167', 'grad_norm': '0.7996', 'learning_rate': '2.833e-05', 'entropy': '1.187', 'num_tokens': '4.42e+05', 'mean_token_accuracy': '0.7239', 'epoch': '0.0007611'}
{'loss': '1.428', 'grad_norm': '0.7758', 'learning_rate': '2.867e-05', 'entropy': '1.326', 'num_tokens': '4.465e+05', 'mean_token_accuracy': '0.7073', 'epoch': '0.0007699'}
{'loss': '1.122', 'grad_norm': '0.6822', 'learning_rate': '2.9e-05', 'entropy': '1.21', 'num_tokens': '4.565e+05', 'mean_token_accuracy': '0.7243', 'epoch': '0.0007788'}
{'loss': '1.225', 'grad_norm': '1.194', 'learning_rate': '2.933e-05', 'entropy': '1.167', 'num_tokens': '4.613e+05', 'mean_token_accuracy': '0.7303', 'epoch': '0.0007876'}
{'loss': '1.373', 'grad_norm': '0.7126', 'learning_rate': '2.967e-05', 'entropy': '1.397', 'num_tokens': '4.667e+05', 'mean_token_accuracy': '0.6796', 'epoch': '0.0007965'}
{'loss': '1.304', 'grad_norm': '0.902', 'learning_rate': '3e-05', 'entropy': '1.315', 'num_tokens': '4.714e+05', 'mean_token_accuracy': '0.7008', 'epoch': '0.0008053'}
{'loss': '1.638', 'grad_norm': '0.9179', 'learning_rate': '3.033e-05', 'entropy': '1.636', 'num_tokens': '4.749e+05', 'mean_token_accuracy': '0.6465', 'epoch': '0.0008142'}
{'loss': '1.07', 'grad_norm': '0.7428', 'learning_rate': '3.067e-05', 'entropy': '1.151', 'num_tokens': '4.796e+05', 'mean_token_accuracy': '0.7478', 'epoch': '0.000823'}
{'loss': '1.287', 'grad_norm': '0.9772', 'learning_rate': '3.1e-05', 'entropy': '1.321', 'num_tokens': '4.841e+05', 'mean_token_accuracy': '0.7115', 'epoch': '0.0008319'}
{'loss': '1.514', 'grad_norm': '0.6023', 'learning_rate': '3.133e-05', 'entropy': '1.519', 'num_tokens': '4.927e+05', 'mean_token_accuracy': '0.6596', 'epoch': '0.0008407'}

  1%|          | 96/12000 [03:18<7:23:00,  2.23s/it]
  1%|          | 97/12000 [03:20<7:05:40,  2.15s/it]
                                                    

  1%|          | 97/12000 [03:20<7:05:40,  2.15s/it]
  1%|          | 98/12000 [03:23<8:06:07,  2.45s/it]
                                                    

  1%|          | 98/12000 [03:23<8:06:07,  2.45s/it]
  1%|          | 99/12000 [03:25<7:10:56,  2.17s/it]
                                                    

  1%|          | 99/12000 [03:25<7:10:56,  2.17s/it]
  1%|          | 100/12000 [03:26<6:34:32,  1.99s/it]
                                                     

  1%|          | 100/12000 [03:26<6:34:32,  1.99s/it]
  1%|          | 101/12000 [03:30<8:15:35,  2.50s/it]
                                                     

  1%|          | 101/12000 [03:30<8:15:35,  2.50s/it]
  1%|          | 102/12000 [03:33<8:13:47,  2.49s/it]
                                                     

  1%|          | 102/12000 [03:33<8:13:47,  2.49s/it]
  1%|          | 103/12000 [03:35<7:40:31,  2.32s/it]
                                                     

  1%|          | 103/12000 [03:35<7:40:31,  2.32s/it]
  1%|          | 104/12000 [03:36<7:03:14,  2.13s/it]
                                                     

  1%|          | 104/12000 [03:36<7:03:14,  2.13s/it]
  1%|          | 105/12000 [03:38<6:26:35,  1.95s/it]
                                                     

  1%|          | 105/12000 [03:38<6:26:35,  1.95s/it]
  1%|          | 106/12000 [03:40<7:05:24,  2.15s/it]
                                                     

  1%|          | 106/12000 [03:40<7:05:24,  2.15s/it]
  1%|          | 107/12000 [03:44<8:52:52,  2.69s/it]
                                                     

  1%|          | 107/12000 [03:44<8:52:52,  2.69s/it]
  1%|          | 108/12000 [03:47<8:45:33,  2.65s/it]
                                                     

  1%|          | 108/12000 [03:47<8:45:33,  2.65s/it]
  1%|          | 109/12000 [03:49<7:42:06,  2.33s/it]
                                                     

  1%|          | 109/12000 [03:49<7:42:06,  2.33s/it]
  1%|          | 110/12000 [03:53<9:22:18,  2.84s/it]
                                                     

  1%|          | 110/12000 [03:53<9:22:18,  2.84s/it]
  1%|          | 111/12000 [03:56<9:55:01,  3.00s/it]
                                                     

  1%|          | 111/12000 [03:56<9:55:01,  3.00s/it]
  1%|          | 112/12000 [03:59<9:54:58,  3.00s/it]
                                                     

  1%|          | 112/12000 [03:59<9:54:58,  3.00s/it]
  1%|          | 113/12000 [04:01<9:14:44,  2.80s/it]
                                                     

  1%|          | 113/12000 [04:01<9:14:44,  2.80s/it]
  1%|          | 114/12000 [04:04<9:41:20,  2.93s/it]
                                                     

  1%|          | 114/12000 [04:04<9:41:20,  2.93s/it]
  1%|          | 115/12000 [04:06<8:43:35,  2.64s/it]
                                                     

  1%|          | 115/12000 [04:06<8:43:35,  2.64s/it]
  1%|          | 116/12000 [04:08<7:49:19,  2.37s/it]
                                                     

  1%|          | 116/12000 [04:08<7:49:19,  2.37s/it]
  1%|          | 117/12000 [04:10<6:59:37,  2.12s/it]
                                                     

  1%|          | 117/12000 [04:10<6:59:37,  2.12s/it]
  1%|          | 118/12000 [04:12<6:52:28,  2.08s/it]
                                                     

  1%|          | 118/12000 [04:12<6:52:28,  2.08s/it]
  1%|          | 119/12000 [04:13<6:34:00,  1.99s/it]
                                                     

  1%|          | 119/12000 [04:13<6:34:00,  1.99s/it]
  1%|          | 120/12000 [04:16<7:24:05,  2.24s/it]
                                                     

  1%|          | 120/12000 [04:16<7:24:05,  2.24s/it]
  1%|          | 121/12000 [04:19<7:39:47,  2.32s/it]
                                                     

  1%|          | 121/12000 [04:19<7:39:47,  2.32s/it]
  1%|          | 122/12000 [04:20<6:54:24,  2.09s/it]
                                                     

  1%|          | 122/12000 [04:20<6:54:24,  2.09s/it]
  1%|          | 123/12000 [04:24<8:02:15,  2.44s/it]
                                                     

  1%|          | 123/12000 [04:24<8:02:15,  2.44s/it]
  1%|          | 124/12000 [04:26<8:26:16,  2.56s/it]
                                                     

  1%|          | 124/12000 [04:26<8:26:16,  2.56s/it]
  1%|          | 125/12000 [04:28<7:34:18,  2.30s/it]
                                                     

  1%|          | 125/12000 [04:28<7:34:18,  2.30s/it]
  1%|          | 126/12000 [04:30<6:47:32,  2.06s/it]
                                                     

  1%|          | 126/12000 [04:30<6:47:32,  2.06s/it]
  1%|          | 127/12000 [04:31<6:19:33,  1.92s/it]
                                                     

  1%|          | 127/12000 [04:31<6:19:33,  1.92s/it]
  1%|          | 128/12000 [04:33<6:31:11,  1.98s/it]
                                                     

  1%|          | 128/12000 [04:33<6:31:11,  1.98s/it]
  1%|          | 129/12000 [04:35<6:14:53,  1.89s/it]
                                                     

  1%|          | 129/12000 [04:35<6:14:53,  1.89s/it]
  1%|          | 130/12000 [04:38<7:05:49,  2.15s/it]
                                                     

  1%|          | 130/12000 [04:38<7:05:49,  2.15s/it]
  1%|          | 131/12000 [04:39<6:34:51,  2.00s/it]
                                                     

  1%|          | 131/12000 [04:39<6:34:51,  2.00s/it]
  1%|          | 132/12000 [04:43<7:50:21,  2.38s/it]
                                                     

  1%|          | 132/12000 [04:43<7:50:21,  2.38s/it]
  1%|          | 133/12000 [04:44<7:09:49,  2.17s/it]
                                                     

  1%|          | 133/12000 [04:44<7:09:49,  2.17s/it]
  1%|          | 134/12000 [04:47<7:32:03,  2.29s/it]
                                                     

  1%|          | 134/12000 [04:47<7:32:03,  2.29s/it]
  1%|          | 135/12000 [04:49<7:27:02,  2.26s/it]
                                                     

  1%|          | 135/12000 [04:49<7:27:02,  2.26s/it]
  1%|          | 136/12000 [04:53<8:43:47,  2.65s/it]
                                                     

  1%|          | 136/12000 [04:53<8:43:47,  2.65s/it]
  1%|          | 137/12000 [04:55<8:20:26,  2.53s/it]
                                                     

  1%|          | 137/12000 [04:55<8:20:26,  2.53s/it]
  1%|          | 138/12000 [04:57<7:36:17,  2.31s/it]
                                                     

  1%|          | 138/12000 [04:57<7:36:17,  2.31s/it]
  1%|          | 139/12000 [04:58<6:37:40,  2.01s/it]
                                                     

  1%|          | 139/12000 [04:58<6:37:40,  2.01s/it]
  1%|          | 140/12000 [04:59<6:02:05,  1.83s/it]
                                                     

  1%|          | 140/12000 [04:59<6:02:05,  1.83s/it]
  1%|          | 141/12000 [05:01<5:44:02,  1.74s/it]
                                                     

  1%|          | 141/12000 [05:01<5:44:02,  1.74s/it]
  1%|          | 142/12000 [05:02<5:27:55,  1.66s/it]
                                                     

  1%|          | 142/12000 [05:02<5:27:55,  1.66s/it]
  1%|          | 143/12000 [05:05<6:08:10,  1.86s/it]
                                                     
{'loss': '1.417', 'grad_norm': '0.6683', 'learning_rate': '3.167e-05', 'entropy': '1.39', 'num_tokens': '4.983e+05', 'mean_token_accuracy': '0.6961', 'epoch': '0.0008496'}
{'loss': '1.246', 'grad_norm': '0.7616', 'learning_rate': '3.2e-05', 'entropy': '1.325', 'num_tokens': '5.038e+05', 'mean_token_accuracy': '0.6988', 'epoch': '0.0008584'}
{'loss': '1.298', 'grad_norm': '0.5877', 'learning_rate': '3.233e-05', 'entropy': '1.313', 'num_tokens': '5.126e+05', 'mean_token_accuracy': '0.7138', 'epoch': '0.0008673'}
{'loss': '1.068', 'grad_norm': '0.948', 'learning_rate': '3.267e-05', 'entropy': '1.059', 'num_tokens': '5.17e+05', 'mean_token_accuracy': '0.754', 'epoch': '0.0008761'}
{'loss': '1.206', 'grad_norm': '0.7319', 'learning_rate': '3.3e-05', 'entropy': '1.321', 'num_tokens': '5.21e+05', 'mean_token_accuracy': '0.6973', 'epoch': '0.000885'}
{'loss': '1.364', 'grad_norm': '0.8003', 'learning_rate': '3.333e-05', 'entropy': '1.409', 'num_tokens': '5.258e+05', 'mean_token_accuracy': '0.6848', 'epoch': '0.0008938'}
{'loss': '1.174', 'grad_norm': '0.7529', 'learning_rate': '3.367e-05', 'entropy': '1.421', 'num_tokens': '5.316e+05', 'mean_token_accuracy': '0.6789', 'epoch': '0.0009027'}
{'loss': '1.191', 'grad_norm': '0.6978', 'learning_rate': '3.4e-05', 'entropy': '1.12', 'num_tokens': '5.38e+05', 'mean_token_accuracy': '0.7413', 'epoch': '0.0009115'}
{'loss': '1.45', 'grad_norm': '0.7195', 'learning_rate': '3.433e-05', 'entropy': '1.283', 'num_tokens': '5.43e+05', 'mean_token_accuracy': '0.6956', 'epoch': '0.0009204'}
{'loss': '1.329', 'grad_norm': '0.8253', 'learning_rate': '3.467e-05', 'entropy': '1.231', 'num_tokens': '5.471e+05', 'mean_token_accuracy': '0.7046', 'epoch': '0.0009292'}
{'loss': '1.158', 'grad_norm': '0.6075', 'learning_rate': '3.5e-05', 'entropy': '1.154', 'num_tokens': '5.531e+05', 'mean_token_accuracy': '0.7225', 'epoch': '0.0009381'}
{'loss': '1.036', 'grad_norm': '0.4938', 'learning_rate': '3.533e-05', 'entropy': '1.136', 'num_tokens': '5.609e+05', 'mean_token_accuracy': '0.7601', 'epoch': '0.0009469'}
{'loss': '1.399', 'grad_norm': '0.5517', 'learning_rate': '3.567e-05', 'entropy': '1.361', 'num_tokens': '5.693e+05', 'mean_token_accuracy': '0.6903', 'epoch': '0.0009558'}
{'loss': '1.028', 'grad_norm': '0.7146', 'learning_rate': '3.6e-05', 'entropy': '1.095', 'num_tokens': '5.736e+05', 'mean_token_accuracy': '0.7498', 'epoch': '0.0009646'}
{'loss': '1.486', 'grad_norm': '0.5535', 'learning_rate': '3.633e-05', 'entropy': '1.49', 'num_tokens': '5.791e+05', 'mean_token_accuracy': '0.6742', 'epoch': '0.0009735'}
{'loss': '0.8347', 'grad_norm': '0.5371', 'learning_rate': '3.667e-05', 'entropy': '0.8569', 'num_tokens': '5.88e+05', 'mean_token_accuracy': '0.8072', 'epoch': '0.0009823'}
{'loss': '1.011', 'grad_norm': '0.5153', 'learning_rate': '3.7e-05', 'entropy': '1.204', 'num_tokens': '5.943e+05', 'mean_token_accuracy': '0.7128', 'epoch': '0.0009912'}
{'loss': '1.263', 'grad_norm': '0.7608', 'learning_rate': '3.733e-05', 'entropy': '1.214', 'num_tokens': '6.016e+05', 'mean_token_accuracy': '0.72', 'epoch': '0.001'}
{'loss': '1.228', 'grad_norm': '0.4857', 'learning_rate': '3.767e-05', 'entropy': '1.326', 'num_tokens': '6.115e+05', 'mean_token_accuracy': '0.6929', 'epoch': '0.001009'}
{'loss': '1.143', 'grad_norm': '0.6256', 'learning_rate': '3.8e-05', 'entropy': '1.084', 'num_tokens': '6.174e+05', 'mean_token_accuracy': '0.7486', 'epoch': '0.001018'}
{'loss': '1.052', 'grad_norm': '0.5492', 'learning_rate': '3.833e-05', 'entropy': '1.048', 'num_tokens': '6.236e+05', 'mean_token_accuracy': '0.7575', 'epoch': '0.001027'}
{'loss': '1.258', 'grad_norm': '0.819', 'learning_rate': '3.867e-05', 'entropy': '1.311', 'num_tokens': '6.278e+05', 'mean_token_accuracy': '0.6964', 'epoch': '0.001035'}
{'loss': '1.131', 'grad_norm': '0.7076', 'learning_rate': '3.9e-05', 'entropy': '1.077', 'num_tokens': '6.336e+05', 'mean_token_accuracy': '0.7501', 'epoch': '0.001044'}
{'loss': '0.9425', 'grad_norm': '0.5485', 'learning_rate': '3.933e-05', 'entropy': '0.9499', 'num_tokens': '6.392e+05', 'mean_token_accuracy': '0.7861', 'epoch': '0.001053'}
{'loss': '0.9866', 'grad_norm': '0.6465', 'learning_rate': '3.967e-05', 'entropy': '0.9831', 'num_tokens': '6.461e+05', 'mean_token_accuracy': '0.7662', 'epoch': '0.001062'}
{'loss': '1.477', 'grad_norm': '0.6015', 'learning_rate': '4e-05', 'entropy': '1.369', 'num_tokens': '6.517e+05', 'mean_token_accuracy': '0.6767', 'epoch': '0.001071'}
{'loss': '0.9916', 'grad_norm': '0.6601', 'learning_rate': '4.033e-05', 'entropy': '0.9879', 'num_tokens': '6.566e+05', 'mean_token_accuracy': '0.7747', 'epoch': '0.00108'}
{'loss': '0.9363', 'grad_norm': '0.6207', 'learning_rate': '4.067e-05', 'entropy': '1.067', 'num_tokens': '6.632e+05', 'mean_token_accuracy': '0.7489', 'epoch': '0.001089'}
{'loss': '1.35', 'grad_norm': '0.4104', 'learning_rate': '4.1e-05', 'entropy': '1.211', 'num_tokens': '6.721e+05', 'mean_token_accuracy': '0.7036', 'epoch': '0.001097'}
{'loss': '1.175', 'grad_norm': '0.5925', 'learning_rate': '4.133e-05', 'entropy': '1.157', 'num_tokens': '6.773e+05', 'mean_token_accuracy': '0.7288', 'epoch': '0.001106'}
{'loss': '0.8331', 'grad_norm': '0.5534', 'learning_rate': '4.167e-05', 'entropy': '0.8555', 'num_tokens': '6.818e+05', 'mean_token_accuracy': '0.7942', 'epoch': '0.001115'}
{'loss': '1.198', 'grad_norm': '0.6242', 'learning_rate': '4.2e-05', 'entropy': '1.244', 'num_tokens': '6.868e+05', 'mean_token_accuracy': '0.7186', 'epoch': '0.001124'}
{'loss': '1.156', 'grad_norm': '0.5807', 'learning_rate': '4.233e-05', 'entropy': '1.034', 'num_tokens': '6.922e+05', 'mean_token_accuracy': '0.7789', 'epoch': '0.001133'}
{'loss': '1.566', 'grad_norm': '0.7099', 'learning_rate': '4.267e-05', 'entropy': '1.42', 'num_tokens': '6.963e+05', 'mean_token_accuracy': '0.6767', 'epoch': '0.001142'}
{'loss': '1.294', 'grad_norm': '0.715', 'learning_rate': '4.3e-05', 'entropy': '1.187', 'num_tokens': '7.027e+05', 'mean_token_accuracy': '0.7203', 'epoch': '0.00115'}
{'loss': '0.8444', 'grad_norm': '0.5499', 'learning_rate': '4.333e-05', 'entropy': '0.8561', 'num_tokens': '7.085e+05', 'mean_token_accuracy': '0.7908', 'epoch': '0.001159'}
{'loss': '1.027', 'grad_norm': '0.6774', 'learning_rate': '4.367e-05', 'entropy': '1.081', 'num_tokens': '7.126e+05', 'mean_token_accuracy': '0.7325', 'epoch': '0.001168'}
{'loss': '1.037', 'grad_norm': '0.74', 'learning_rate': '4.4e-05', 'entropy': '0.9862', 'num_tokens': '7.179e+05', 'mean_token_accuracy': '0.7679', 'epoch': '0.001177'}
{'loss': '1.029', 'grad_norm': '0.5495', 'learning_rate': '4.433e-05', 'entropy': '1.057', 'num_tokens': '7.246e+05', 'mean_token_accuracy': '0.7608', 'epoch': '0.001186'}
{'loss': '1.153', 'grad_norm': '0.6234', 'learning_rate': '4.467e-05', 'entropy': '1.144', 'num_tokens': '7.305e+05', 'mean_token_accuracy': '0.7351', 'epoch': '0.001195'}
{'loss': '0.9218', 'grad_norm': '0.5041', 'learning_rate': '4.5e-05', 'entropy': '0.9182', 'num_tokens': '7.38e+05', 'mean_token_accuracy': '0.774', 'epoch': '0.001204'}
{'loss': '1.103', 'grad_norm': '0.6247', 'learning_rate': '4.533e-05', 'entropy': '1.053', 'num_tokens': '7.438e+05', 'mean_token_accuracy': '0.7573', 'epoch': '0.001212'}
{'loss': '1.346', 'grad_norm': '0.7828', 'learning_rate': '4.567e-05', 'entropy': '1.294', 'num_tokens': '7.487e+05', 'mean_token_accuracy': '0.693', 'epoch': '0.001221'}
{'loss': '1.206', 'grad_norm': '0.7636', 'learning_rate': '4.6e-05', 'entropy': '1.291', 'num_tokens': '7.518e+05', 'mean_token_accuracy': '0.7023', 'epoch': '0.00123'}
{'loss': '1.101', 'grad_norm': '0.6584', 'learning_rate': '4.633e-05', 'entropy': '1.129', 'num_tokens': '7.557e+05', 'mean_token_accuracy': '0.7469', 'epoch': '0.001239'}
{'loss': '1.348', 'grad_norm': '0.6214', 'learning_rate': '4.667e-05', 'entropy': '1.354', 'num_tokens': '7.607e+05', 'mean_token_accuracy': '0.7006', 'epoch': '0.001248'}
{'loss': '0.9468', 'grad_norm': '0.5851', 'learning_rate': '4.7e-05', 'entropy': '1.057', 'num_tokens': '7.648e+05', 'mean_token_accuracy': '0.7647', 'epoch': '0.001257'}

  1%|          | 143/12000 [05:05<6:08:10,  1.86s/it]
  1%|          | 144/12000 [05:06<5:53:01,  1.79s/it]
                                                     

  1%|          | 144/12000 [05:06<5:53:01,  1.79s/it]
  1%|          | 145/12000 [05:09<6:11:00,  1.88s/it]
                                                     

  1%|          | 145/12000 [05:09<6:11:00,  1.88s/it]
  1%|          | 146/12000 [05:10<5:53:53,  1.79s/it]
                                                     

  1%|          | 146/12000 [05:10<5:53:53,  1.79s/it]
  1%|          | 147/12000 [05:14<8:03:31,  2.45s/it]
                                                     

  1%|          | 147/12000 [05:14<8:03:31,  2.45s/it]
  1%|          | 148/12000 [05:16<7:10:16,  2.18s/it]
                                                     

  1%|          | 148/12000 [05:16<7:10:16,  2.18s/it]
  1%|          | 149/12000 [05:17<6:29:05,  1.97s/it]
                                                     

  1%|          | 149/12000 [05:17<6:29:05,  1.97s/it]
  1%|▏         | 150/12000 [05:19<6:13:42,  1.89s/it]
                                                     

  1%|▏         | 150/12000 [05:19<6:13:42,  1.89s/it]
  1%|▏         | 151/12000 [05:22<7:07:47,  2.17s/it]
                                                     

  1%|▏         | 151/12000 [05:22<7:07:47,  2.17s/it]
  1%|▏         | 152/12000 [05:24<6:50:35,  2.08s/it]
                                                     

  1%|▏         | 152/12000 [05:24<6:50:35,  2.08s/it]
  1%|▏         | 153/12000 [05:25<6:16:01,  1.90s/it]
                                                     

  1%|▏         | 153/12000 [05:25<6:16:01,  1.90s/it]
  1%|▏         | 154/12000 [05:29<8:04:45,  2.46s/it]
                                                     

  1%|▏         | 154/12000 [05:29<8:04:45,  2.46s/it]
  1%|▏         | 155/12000 [05:33<9:22:46,  2.85s/it]
                                                     

  1%|▏         | 155/12000 [05:33<9:22:46,  2.85s/it]
  1%|▏         | 156/12000 [05:36<9:53:20,  3.01s/it]
                                                     

  1%|▏         | 156/12000 [05:36<9:53:20,  3.01s/it]
  1%|▏         | 157/12000 [05:38<8:44:41,  2.66s/it]
                                                     

  1%|▏         | 157/12000 [05:38<8:44:41,  2.66s/it]
  1%|▏         | 158/12000 [05:40<8:01:23,  2.44s/it]
                                                     

  1%|▏         | 158/12000 [05:40<8:01:23,  2.44s/it]
  1%|▏         | 159/12000 [05:41<7:17:05,  2.21s/it]
                                                     

  1%|▏         | 159/12000 [05:41<7:17:05,  2.21s/it]
  1%|▏         | 160/12000 [05:43<6:35:03,  2.00s/it]
                                                     

  1%|▏         | 160/12000 [05:43<6:35:03,  2.00s/it]
  1%|▏         | 161/12000 [05:46<7:17:05,  2.22s/it]
                                                     

  1%|▏         | 161/12000 [05:46<7:17:05,  2.22s/it]
  1%|▏         | 162/12000 [05:49<8:12:20,  2.50s/it]
                                                     

  1%|▏         | 162/12000 [05:49<8:12:20,  2.50s/it]
  1%|▏         | 163/12000 [05:51<8:17:20,  2.52s/it]
                                                     

  1%|▏         | 163/12000 [05:51<8:17:20,  2.52s/it]
  1%|▏         | 164/12000 [05:53<7:19:54,  2.23s/it]
                                                     

  1%|▏         | 164/12000 [05:53<7:19:54,  2.23s/it]
  1%|▏         | 165/12000 [05:55<7:15:55,  2.21s/it]
                                                     

  1%|▏         | 165/12000 [05:55<7:15:55,  2.21s/it]
  1%|▏         | 166/12000 [05:58<8:00:37,  2.44s/it]
                                                     

  1%|▏         | 166/12000 [05:58<8:00:37,  2.44s/it]
  1%|▏         | 167/12000 [06:00<7:30:58,  2.29s/it]
                                                     

  1%|▏         | 167/12000 [06:00<7:30:58,  2.29s/it]
  1%|▏         | 168/12000 [06:05<9:49:59,  2.99s/it]
                                                     

  1%|▏         | 168/12000 [06:05<9:49:59,  2.99s/it]
  1%|▏         | 169/12000 [06:06<8:30:32,  2.59s/it]
                                                     

  1%|▏         | 169/12000 [06:06<8:30:32,  2.59s/it]
  1%|▏         | 170/12000 [06:09<8:15:16,  2.51s/it]
                                                     

  1%|▏         | 170/12000 [06:09<8:15:16,  2.51s/it]
  1%|▏         | 171/12000 [06:10<7:31:15,  2.29s/it]
                                                     

  1%|▏         | 171/12000 [06:10<7:31:15,  2.29s/it]
  1%|▏         | 172/12000 [06:12<6:44:40,  2.05s/it]
                                                     

  1%|▏         | 172/12000 [06:12<6:44:40,  2.05s/it]
  1%|▏         | 173/12000 [06:15<7:28:04,  2.27s/it]
                                                     

  1%|▏         | 173/12000 [06:15<7:28:04,  2.27s/it]
  1%|▏         | 174/12000 [06:17<7:59:00,  2.43s/it]
                                                     

  1%|▏         | 174/12000 [06:17<7:59:00,  2.43s/it]
  1%|▏         | 175/12000 [06:19<7:18:20,  2.22s/it]
                                                     

  1%|▏         | 175/12000 [06:19<7:18:20,  2.22s/it]
  1%|▏         | 176/12000 [06:23<8:36:05,  2.62s/it]
                                                     

  1%|▏         | 176/12000 [06:23<8:36:05,  2.62s/it]
  1%|▏         | 177/12000 [06:24<7:35:38,  2.31s/it]
                                                     

  1%|▏         | 177/12000 [06:24<7:35:38,  2.31s/it]
  1%|▏         | 178/12000 [06:26<7:00:24,  2.13s/it]
                                                     

  1%|▏         | 178/12000 [06:26<7:00:24,  2.13s/it]
  1%|▏         | 179/12000 [06:31<9:21:15,  2.85s/it]
                                                     

  1%|▏         | 179/12000 [06:31<9:21:15,  2.85s/it]
  2%|▏         | 180/12000 [06:32<8:11:08,  2.49s/it]
                                                     

  2%|▏         | 180/12000 [06:32<8:11:08,  2.49s/it]
  2%|▏         | 181/12000 [06:34<7:16:44,  2.22s/it]
                                                     

  2%|▏         | 181/12000 [06:34<7:16:44,  2.22s/it]
  2%|▏         | 182/12000 [06:36<6:56:41,  2.12s/it]
                                                     

  2%|▏         | 182/12000 [06:36<6:56:41,  2.12s/it]
  2%|▏         | 183/12000 [06:37<6:17:15,  1.92s/it]
                                                     

  2%|▏         | 183/12000 [06:37<6:17:15,  1.92s/it]
  2%|▏         | 184/12000 [06:39<6:01:30,  1.84s/it]
                                                     

  2%|▏         | 184/12000 [06:39<6:01:30,  1.84s/it]
  2%|▏         | 185/12000 [06:41<6:18:12,  1.92s/it]
                                                     

  2%|▏         | 185/12000 [06:41<6:18:12,  1.92s/it]
  2%|▏         | 186/12000 [06:42<5:57:48,  1.82s/it]
                                                     

  2%|▏         | 186/12000 [06:42<5:57:48,  1.82s/it]
  2%|▏         | 187/12000 [06:44<5:58:38,  1.82s/it]
                                                     

  2%|▏         | 187/12000 [06:44<5:58:38,  1.82s/it]
  2%|▏         | 188/12000 [06:46<6:11:17,  1.89s/it]
                                                     

  2%|▏         | 188/12000 [06:46<6:11:17,  1.89s/it]
  2%|▏         | 189/12000 [06:48<6:25:22,  1.96s/it]
                                                     

  2%|▏         | 189/12000 [06:48<6:25:22,  1.96s/it]
  2%|▏         | 190/12000 [06:52<7:36:53,  2.32s/it]
                                                     
{'loss': '1.463', 'grad_norm': '0.6019', 'learning_rate': '4.733e-05', 'entropy': '1.359', 'num_tokens': '7.703e+05', 'mean_token_accuracy': '0.6847', 'epoch': '0.001266'}
{'loss': '1.095', 'grad_norm': '0.6553', 'learning_rate': '4.767e-05', 'entropy': '1.09', 'num_tokens': '7.754e+05', 'mean_token_accuracy': '0.7394', 'epoch': '0.001274'}
{'loss': '1.035', 'grad_norm': '0.6849', 'learning_rate': '4.8e-05', 'entropy': '0.9948', 'num_tokens': '7.81e+05', 'mean_token_accuracy': '0.765', 'epoch': '0.001283'}
{'loss': '1.134', 'grad_norm': '0.6743', 'learning_rate': '4.833e-05', 'entropy': '1.098', 'num_tokens': '7.851e+05', 'mean_token_accuracy': '0.7441', 'epoch': '0.001292'}
{'loss': '1.218', 'grad_norm': '0.4112', 'learning_rate': '4.867e-05', 'entropy': '1.305', 'num_tokens': '7.937e+05', 'mean_token_accuracy': '0.6862', 'epoch': '0.001301'}
{'loss': '1.018', 'grad_norm': '0.7283', 'learning_rate': '4.9e-05', 'entropy': '0.9944', 'num_tokens': '7.981e+05', 'mean_token_accuracy': '0.7796', 'epoch': '0.00131'}
{'loss': '1.184', 'grad_norm': '0.7883', 'learning_rate': '4.933e-05', 'entropy': '1.268', 'num_tokens': '8.018e+05', 'mean_token_accuracy': '0.7019', 'epoch': '0.001319'}
{'loss': '1.367', 'grad_norm': '0.6219', 'learning_rate': '4.967e-05', 'entropy': '1.295', 'num_tokens': '8.066e+05', 'mean_token_accuracy': '0.6877', 'epoch': '0.001327'}
{'loss': '1.004', 'grad_norm': '0.8819', 'learning_rate': '5e-05', 'entropy': '0.9769', 'num_tokens': '8.097e+05', 'mean_token_accuracy': '0.7631', 'epoch': '0.001336'}
{'loss': '0.8156', 'grad_norm': '0.5275', 'learning_rate': '5.033e-05', 'entropy': '0.8649', 'num_tokens': '8.151e+05', 'mean_token_accuracy': '0.79', 'epoch': '0.001345'}
{'loss': '1.254', 'grad_norm': '0.6128', 'learning_rate': '5.067e-05', 'entropy': '1.284', 'num_tokens': '8.194e+05', 'mean_token_accuracy': '0.7044', 'epoch': '0.001354'}
{'loss': '1.24', 'grad_norm': '0.5957', 'learning_rate': '5.1e-05', 'entropy': '1.203', 'num_tokens': '8.241e+05', 'mean_token_accuracy': '0.7052', 'epoch': '0.001363'}
{'loss': '0.8578', 'grad_norm': '0.5723', 'learning_rate': '5.133e-05', 'entropy': '0.9419', 'num_tokens': '8.317e+05', 'mean_token_accuracy': '0.7856', 'epoch': '0.001372'}
{'loss': '1.376', 'grad_norm': '0.5193', 'learning_rate': '5.167e-05', 'entropy': '1.261', 'num_tokens': '8.404e+05', 'mean_token_accuracy': '0.7086', 'epoch': '0.001381'}
{'loss': '1.231', 'grad_norm': '0.6629', 'learning_rate': '5.2e-05', 'entropy': '1.191', 'num_tokens': '8.457e+05', 'mean_token_accuracy': '0.7207', 'epoch': '0.001389'}
{'loss': '1.15', 'grad_norm': '0.8156', 'learning_rate': '5.233e-05', 'entropy': '1.255', 'num_tokens': '8.504e+05', 'mean_token_accuracy': '0.702', 'epoch': '0.001398'}
{'loss': '1.199', 'grad_norm': '0.662', 'learning_rate': '5.267e-05', 'entropy': '1.143', 'num_tokens': '8.549e+05', 'mean_token_accuracy': '0.7277', 'epoch': '0.001407'}
{'loss': '0.8939', 'grad_norm': '0.6847', 'learning_rate': '5.3e-05', 'entropy': '0.9351', 'num_tokens': '8.595e+05', 'mean_token_accuracy': '0.7834', 'epoch': '0.001416'}
{'loss': '0.8507', 'grad_norm': '0.4886', 'learning_rate': '5.333e-05', 'entropy': '0.8538', 'num_tokens': '8.658e+05', 'mean_token_accuracy': '0.8042', 'epoch': '0.001425'}
{'loss': '0.9396', 'grad_norm': '0.4433', 'learning_rate': '5.367e-05', 'entropy': '0.8869', 'num_tokens': '8.741e+05', 'mean_token_accuracy': '0.7992', 'epoch': '0.001434'}
{'loss': '1.463', 'grad_norm': '0.5367', 'learning_rate': '5.4e-05', 'entropy': '1.32', 'num_tokens': '8.806e+05', 'mean_token_accuracy': '0.6964', 'epoch': '0.001443'}
{'loss': '1.121', 'grad_norm': '0.91', 'learning_rate': '5.433e-05', 'entropy': '1.168', 'num_tokens': '8.85e+05', 'mean_token_accuracy': '0.7229', 'epoch': '0.001451'}
{'loss': '0.8387', 'grad_norm': '0.6853', 'learning_rate': '5.467e-05', 'entropy': '0.8467', 'num_tokens': '8.903e+05', 'mean_token_accuracy': '0.7978', 'epoch': '0.00146'}
{'loss': '1.291', 'grad_norm': '0.5399', 'learning_rate': '5.5e-05', 'entropy': '1.257', 'num_tokens': '8.971e+05', 'mean_token_accuracy': '0.7228', 'epoch': '0.001469'}
{'loss': '0.8848', 'grad_norm': '0.5638', 'learning_rate': '5.533e-05', 'entropy': '0.9889', 'num_tokens': '9.026e+05', 'mean_token_accuracy': '0.7729', 'epoch': '0.001478'}
{'loss': '1.644', 'grad_norm': '0.5104', 'learning_rate': '5.567e-05', 'entropy': '1.402', 'num_tokens': '9.094e+05', 'mean_token_accuracy': '0.7024', 'epoch': '0.001487'}
{'loss': '1.283', 'grad_norm': '0.6488', 'learning_rate': '5.6e-05', 'entropy': '1.214', 'num_tokens': '9.144e+05', 'mean_token_accuracy': '0.7077', 'epoch': '0.001496'}
{'loss': '1.429', 'grad_norm': '0.6209', 'learning_rate': '5.633e-05', 'entropy': '1.475', 'num_tokens': '9.206e+05', 'mean_token_accuracy': '0.6553', 'epoch': '0.001504'}
{'loss': '1.079', 'grad_norm': '0.5215', 'learning_rate': '5.667e-05', 'entropy': '1.122', 'num_tokens': '9.266e+05', 'mean_token_accuracy': '0.7467', 'epoch': '0.001513'}
{'loss': '0.888', 'grad_norm': '0.6493', 'learning_rate': '5.7e-05', 'entropy': '0.8793', 'num_tokens': '9.305e+05', 'mean_token_accuracy': '0.7872', 'epoch': '0.001522'}
{'loss': '1.351', 'grad_norm': '0.6239', 'learning_rate': '5.733e-05', 'entropy': '1.326', 'num_tokens': '9.366e+05', 'mean_token_accuracy': '0.7024', 'epoch': '0.001531'}
{'loss': '1.424', 'grad_norm': '0.6726', 'learning_rate': '5.767e-05', 'entropy': '1.443', 'num_tokens': '9.431e+05', 'mean_token_accuracy': '0.6807', 'epoch': '0.00154'}
{'loss': '1.061', 'grad_norm': '0.745', 'learning_rate': '5.8e-05', 'entropy': '1.027', 'num_tokens': '9.474e+05', 'mean_token_accuracy': '0.7529', 'epoch': '0.001549'}
{'loss': '1.086', 'grad_norm': '0.6316', 'learning_rate': '5.833e-05', 'entropy': '1.087', 'num_tokens': '9.517e+05', 'mean_token_accuracy': '0.7513', 'epoch': '0.001558'}
{'loss': '1.151', 'grad_norm': '0.587', 'learning_rate': '5.867e-05', 'entropy': '1.228', 'num_tokens': '9.559e+05', 'mean_token_accuracy': '0.7081', 'epoch': '0.001566'}
{'loss': '1.294', 'grad_norm': '0.6602', 'learning_rate': '5.9e-05', 'entropy': '1.305', 'num_tokens': '9.606e+05', 'mean_token_accuracy': '0.7044', 'epoch': '0.001575'}
{'loss': '0.9733', 'grad_norm': '0.5108', 'learning_rate': '5.933e-05', 'entropy': '1.007', 'num_tokens': '9.685e+05', 'mean_token_accuracy': '0.7525', 'epoch': '0.001584'}
{'loss': '1.009', 'grad_norm': '0.7543', 'learning_rate': '5.967e-05', 'entropy': '1.066', 'num_tokens': '9.724e+05', 'mean_token_accuracy': '0.7423', 'epoch': '0.001593'}
{'loss': '1.115', 'grad_norm': '0.5102', 'learning_rate': '6e-05', 'entropy': '1.052', 'num_tokens': '9.772e+05', 'mean_token_accuracy': '0.7413', 'epoch': '0.001602'}
{'loss': '1.091', 'grad_norm': '0.582', 'learning_rate': '6.033e-05', 'entropy': '1.035', 'num_tokens': '9.825e+05', 'mean_token_accuracy': '0.75', 'epoch': '0.001611'}
{'loss': '0.9807', 'grad_norm': '0.5885', 'learning_rate': '6.067e-05', 'entropy': '1.002', 'num_tokens': '9.869e+05', 'mean_token_accuracy': '0.7649', 'epoch': '0.00162'}
{'loss': '1.434', 'grad_norm': '0.5698', 'learning_rate': '6.1e-05', 'entropy': '1.491', 'num_tokens': '9.921e+05', 'mean_token_accuracy': '0.6697', 'epoch': '0.001628'}
{'loss': '1.388', 'grad_norm': '0.6094', 'learning_rate': '6.133e-05', 'entropy': '1.331', 'num_tokens': '9.977e+05', 'mean_token_accuracy': '0.6949', 'epoch': '0.001637'}
{'loss': '0.9918', 'grad_norm': '0.6149', 'learning_rate': '6.167e-05', 'entropy': '0.9487', 'num_tokens': '1.002e+06', 'mean_token_accuracy': '0.7636', 'epoch': '0.001646'}
{'loss': '1.068', 'grad_norm': '0.5493', 'learning_rate': '6.2e-05', 'entropy': '1.013', 'num_tokens': '1.008e+06', 'mean_token_accuracy': '0.7452', 'epoch': '0.001655'}
{'loss': '0.8692', 'grad_norm': '0.5055', 'learning_rate': '6.233e-05', 'entropy': '0.9109', 'num_tokens': '1.015e+06', 'mean_token_accuracy': '0.7796', 'epoch': '0.001664'}
{'loss': '1.091', 'grad_norm': '0.5036', 'learning_rate': '6.267e-05', 'entropy': '1.084', 'num_tokens': '1.021e+06', 'mean_token_accuracy': '0.7358', 'epoch': '0.001673'}

  2%|▏         | 190/12000 [06:52<7:36:53,  2.32s/it]
  2%|▏         | 191/12000 [06:53<6:56:07,  2.11s/it]
                                                     

  2%|▏         | 191/12000 [06:53<6:56:07,  2.11s/it]
  2%|▏         | 192/12000 [06:56<7:12:48,  2.20s/it]
                                                     

  2%|▏         | 192/12000 [06:56<7:12:48,  2.20s/it]
  2%|▏         | 193/12000 [06:58<7:12:13,  2.20s/it]
                                                     

  2%|▏         | 193/12000 [06:58<7:12:13,  2.20s/it]
  2%|▏         | 194/12000 [07:00<7:09:05,  2.18s/it]
                                                     

  2%|▏         | 194/12000 [07:00<7:09:05,  2.18s/it]
  2%|▏         | 195/12000 [07:02<6:44:08,  2.05s/it]
                                                     

  2%|▏         | 195/12000 [07:02<6:44:08,  2.05s/it]
  2%|▏         | 196/12000 [07:06<9:10:44,  2.80s/it]
                                                     

  2%|▏         | 196/12000 [07:06<9:10:44,  2.80s/it]
  2%|▏         | 197/12000 [07:08<7:59:45,  2.44s/it]
                                                     

  2%|▏         | 197/12000 [07:08<7:59:45,  2.44s/it]
  2%|▏         | 198/12000 [07:12<9:27:38,  2.89s/it]
                                                     

  2%|▏         | 198/12000 [07:12<9:27:38,  2.89s/it]
  2%|▏         | 199/12000 [07:14<8:41:10,  2.65s/it]
                                                     

  2%|▏         | 199/12000 [07:14<8:41:10,  2.65s/it]
  2%|▏         | 200/12000 [07:16<7:48:58,  2.38s/it]
                                                     

  2%|▏         | 200/12000 [07:16<7:48:58,  2.38s/it]
  2%|▏         | 201/12000 [07:19<8:37:51,  2.63s/it]
                                                     

  2%|▏         | 201/12000 [07:19<8:37:51,  2.63s/it]
  2%|▏         | 202/12000 [07:22<8:57:26,  2.73s/it]
                                                     

  2%|▏         | 202/12000 [07:22<8:57:26,  2.73s/it]
  2%|▏         | 203/12000 [07:25<9:37:26,  2.94s/it]
                                                     

  2%|▏         | 203/12000 [07:25<9:37:26,  2.94s/it]
  2%|▏         | 204/12000 [07:27<8:12:49,  2.51s/it]
                                                     

  2%|▏         | 204/12000 [07:27<8:12:49,  2.51s/it]
  2%|▏         | 205/12000 [07:28<7:13:09,  2.20s/it]
                                                     

  2%|▏         | 205/12000 [07:28<7:13:09,  2.20s/it]
  2%|▏         | 206/12000 [07:31<7:48:00,  2.38s/it]
                                                     

  2%|▏         | 206/12000 [07:31<7:48:00,  2.38s/it]
  2%|▏         | 207/12000 [07:33<7:12:39,  2.20s/it]
                                                     

  2%|▏         | 207/12000 [07:33<7:12:39,  2.20s/it]
  2%|▏         | 208/12000 [07:34<6:30:57,  1.99s/it]
                                                     

  2%|▏         | 208/12000 [07:34<6:30:57,  1.99s/it]
  2%|▏         | 209/12000 [07:36<6:02:31,  1.84s/it]
                                                     

  2%|▏         | 209/12000 [07:36<6:02:31,  1.84s/it]
  2%|▏         | 210/12000 [07:38<6:14:13,  1.90s/it]
                                                     

  2%|▏         | 210/12000 [07:38<6:14:13,  1.90s/it]
  2%|▏         | 211/12000 [07:39<5:57:48,  1.82s/it]
                                                     

  2%|▏         | 211/12000 [07:39<5:57:48,  1.82s/it]
  2%|▏         | 212/12000 [07:42<6:54:27,  2.11s/it]
                                                     

  2%|▏         | 212/12000 [07:42<6:54:27,  2.11s/it]
  2%|▏         | 213/12000 [07:44<6:42:58,  2.05s/it]
                                                     

  2%|▏         | 213/12000 [07:44<6:42:58,  2.05s/it]
  2%|▏         | 214/12000 [07:46<6:06:02,  1.86s/it]
                                                     

  2%|▏         | 214/12000 [07:46<6:06:02,  1.86s/it]
  2%|▏         | 215/12000 [07:49<7:20:12,  2.24s/it]
                                                     

  2%|▏         | 215/12000 [07:49<7:20:12,  2.24s/it]
  2%|▏         | 216/12000 [07:50<6:52:32,  2.10s/it]
                                                     

  2%|▏         | 216/12000 [07:50<6:52:32,  2.10s/it]
  2%|▏         | 217/12000 [07:52<6:21:25,  1.94s/it]
                                                     

  2%|▏         | 217/12000 [07:52<6:21:25,  1.94s/it]
  2%|▏         | 218/12000 [07:54<6:09:25,  1.88s/it]
                                                     

  2%|▏         | 218/12000 [07:54<6:09:25,  1.88s/it]
  2%|▏         | 219/12000 [07:55<5:43:20,  1.75s/it]
                                                     

  2%|▏         | 219/12000 [07:55<5:43:20,  1.75s/it]
  2%|▏         | 220/12000 [07:59<7:37:12,  2.33s/it]
                                                     

  2%|▏         | 220/12000 [07:59<7:37:12,  2.33s/it]
  2%|▏         | 221/12000 [08:00<6:46:47,  2.07s/it]
                                                     

  2%|▏         | 221/12000 [08:00<6:46:47,  2.07s/it]
  2%|▏         | 222/12000 [08:02<6:28:28,  1.98s/it]
                                                     

  2%|▏         | 222/12000 [08:02<6:28:28,  1.98s/it]
  2%|▏         | 223/12000 [08:07<9:12:17,  2.81s/it]
                                                     

  2%|▏         | 223/12000 [08:07<9:12:17,  2.81s/it]
  2%|▏         | 224/12000 [08:08<7:52:27,  2.41s/it]
                                                     

  2%|▏         | 224/12000 [08:08<7:52:27,  2.41s/it]
  2%|▏         | 225/12000 [08:11<7:49:09,  2.39s/it]
                                                     

  2%|▏         | 225/12000 [08:11<7:49:09,  2.39s/it]
  2%|▏         | 226/12000 [08:12<6:52:59,  2.10s/it]
                                                     

  2%|▏         | 226/12000 [08:12<6:52:59,  2.10s/it]
  2%|▏         | 227/12000 [08:14<6:43:34,  2.06s/it]
                                                     

  2%|▏         | 227/12000 [08:14<6:43:34,  2.06s/it]
  2%|▏         | 228/12000 [08:17<7:12:40,  2.21s/it]
                                                     

  2%|▏         | 228/12000 [08:17<7:12:40,  2.21s/it]
  2%|▏         | 229/12000 [08:21<9:25:53,  2.88s/it]
                                                     

  2%|▏         | 229/12000 [08:21<9:25:53,  2.88s/it]
  2%|▏         | 230/12000 [08:24<9:06:38,  2.79s/it]
                                                     

  2%|▏         | 230/12000 [08:24<9:06:38,  2.79s/it]
  2%|▏         | 231/12000 [08:26<8:18:43,  2.54s/it]
                                                     

  2%|▏         | 231/12000 [08:26<8:18:43,  2.54s/it]
  2%|▏         | 232/12000 [08:27<7:21:36,  2.25s/it]
                                                     

  2%|▏         | 232/12000 [08:27<7:21:36,  2.25s/it]
  2%|▏         | 233/12000 [08:29<7:08:00,  2.18s/it]
                                                     

  2%|▏         | 233/12000 [08:29<7:08:00,  2.18s/it]
  2%|▏         | 234/12000 [08:31<6:38:14,  2.03s/it]
                                                     

  2%|▏         | 234/12000 [08:31<6:38:14,  2.03s/it]
  2%|▏         | 235/12000 [08:33<6:35:39,  2.02s/it]
                                                     

  2%|▏         | 235/12000 [08:33<6:35:39,  2.02s/it]
  2%|▏         | 236/12000 [08:37<8:19:24,  2.55s/it]
                                                     

  2%|▏         | 236/12000 [08:37<8:19:24,  2.55s/it]
  2%|▏         | 237/12000 [08:38<7:30:32,  2.30s/it]
                                                     
{'loss': '1.21', 'grad_norm': '0.4998', 'learning_rate': '6.3e-05', 'entropy': '1.165', 'num_tokens': '1.029e+06', 'mean_token_accuracy': '0.7355', 'epoch': '0.001681'}
{'loss': '0.8022', 'grad_norm': '0.5741', 'learning_rate': '6.333e-05', 'entropy': '0.8369', 'num_tokens': '1.034e+06', 'mean_token_accuracy': '0.7939', 'epoch': '0.00169'}
{'loss': '1.154', 'grad_norm': '0.666', 'learning_rate': '6.367e-05', 'entropy': '1.168', 'num_tokens': '1.041e+06', 'mean_token_accuracy': '0.7268', 'epoch': '0.001699'}
{'loss': '0.8072', 'grad_norm': '0.5128', 'learning_rate': '6.4e-05', 'entropy': '0.8139', 'num_tokens': '1.046e+06', 'mean_token_accuracy': '0.7975', 'epoch': '0.001708'}
{'loss': '0.8648', 'grad_norm': '0.5587', 'learning_rate': '6.433e-05', 'entropy': '0.7982', 'num_tokens': '1.052e+06', 'mean_token_accuracy': '0.7974', 'epoch': '0.001717'}
{'loss': '0.7538', 'grad_norm': '0.5914', 'learning_rate': '6.467e-05', 'entropy': '0.8431', 'num_tokens': '1.056e+06', 'mean_token_accuracy': '0.7842', 'epoch': '0.001726'}
{'loss': '1.71', 'grad_norm': '0.518', 'learning_rate': '6.5e-05', 'entropy': '1.439', 'num_tokens': '1.064e+06', 'mean_token_accuracy': '0.6633', 'epoch': '0.001735'}
{'loss': '0.9665', 'grad_norm': '0.5971', 'learning_rate': '6.533e-05', 'entropy': '1.001', 'num_tokens': '1.068e+06', 'mean_token_accuracy': '0.7538', 'epoch': '0.001743'}
{'loss': '0.9921', 'grad_norm': '0.5396', 'learning_rate': '6.567e-05', 'entropy': '0.9799', 'num_tokens': '1.074e+06', 'mean_token_accuracy': '0.7639', 'epoch': '0.001752'}
{'loss': '1.091', 'grad_norm': '0.5079', 'learning_rate': '6.6e-05', 'entropy': '1.076', 'num_tokens': '1.08e+06', 'mean_token_accuracy': '0.7376', 'epoch': '0.001761'}
{'loss': '1.21', 'grad_norm': '0.7448', 'learning_rate': '6.633e-05', 'entropy': '1.201', 'num_tokens': '1.085e+06', 'mean_token_accuracy': '0.7279', 'epoch': '0.00177'}
{'loss': '1.247', 'grad_norm': '0.6323', 'learning_rate': '6.667e-05', 'entropy': '1.229', 'num_tokens': '1.089e+06', 'mean_token_accuracy': '0.7082', 'epoch': '0.001779'}
{'loss': '1.1', 'grad_norm': '0.6327', 'learning_rate': '6.7e-05', 'entropy': '1.033', 'num_tokens': '1.095e+06', 'mean_token_accuracy': '0.7464', 'epoch': '0.001788'}
{'loss': '0.9939', 'grad_norm': '0.4554', 'learning_rate': '6.733e-05', 'entropy': '0.867', 'num_tokens': '1.103e+06', 'mean_token_accuracy': '0.7794', 'epoch': '0.001797'}
{'loss': '0.8654', 'grad_norm': '0.6306', 'learning_rate': '6.767e-05', 'entropy': '0.9109', 'num_tokens': '1.108e+06', 'mean_token_accuracy': '0.7913', 'epoch': '0.001805'}
{'loss': '0.8602', 'grad_norm': '0.507', 'learning_rate': '6.8e-05', 'entropy': '0.9006', 'num_tokens': '1.113e+06', 'mean_token_accuracy': '0.8016', 'epoch': '0.001814'}
{'loss': '1.032', 'grad_norm': '0.4562', 'learning_rate': '6.833e-05', 'entropy': '1.018', 'num_tokens': '1.122e+06', 'mean_token_accuracy': '0.7538', 'epoch': '0.001823'}
{'loss': '1.076', 'grad_norm': '0.4574', 'learning_rate': '6.867e-05', 'entropy': '1.079', 'num_tokens': '1.128e+06', 'mean_token_accuracy': '0.7568', 'epoch': '0.001832'}
{'loss': '1.038', 'grad_norm': '0.6238', 'learning_rate': '6.9e-05', 'entropy': '1.135', 'num_tokens': '1.132e+06', 'mean_token_accuracy': '0.7418', 'epoch': '0.001841'}
{'loss': '1.215', 'grad_norm': '0.746', 'learning_rate': '6.933e-05', 'entropy': '1.258', 'num_tokens': '1.136e+06', 'mean_token_accuracy': '0.7115', 'epoch': '0.00185'}
{'loss': '0.8172', 'grad_norm': '0.5055', 'learning_rate': '6.967e-05', 'entropy': '0.8543', 'num_tokens': '1.142e+06', 'mean_token_accuracy': '0.8052', 'epoch': '0.001858'}
{'loss': '1.2', 'grad_norm': '0.6786', 'learning_rate': '7e-05', 'entropy': '1.204', 'num_tokens': '1.147e+06', 'mean_token_accuracy': '0.7362', 'epoch': '0.001867'}
{'loss': '1.03', 'grad_norm': '0.5228', 'learning_rate': '7.033e-05', 'entropy': '1.057', 'num_tokens': '1.153e+06', 'mean_token_accuracy': '0.7581', 'epoch': '0.001876'}
{'loss': '1.258', 'grad_norm': '0.6118', 'learning_rate': '7.067e-05', 'entropy': '1.288', 'num_tokens': '1.158e+06', 'mean_token_accuracy': '0.6985', 'epoch': '0.001885'}
{'loss': '1.088', 'grad_norm': '0.6599', 'learning_rate': '7.1e-05', 'entropy': '1.084', 'num_tokens': '1.163e+06', 'mean_token_accuracy': '0.7494', 'epoch': '0.001894'}
{'loss': '1.198', 'grad_norm': '0.5709', 'learning_rate': '7.133e-05', 'entropy': '1.118', 'num_tokens': '1.17e+06', 'mean_token_accuracy': '0.7358', 'epoch': '0.001903'}
{'loss': '1.216', 'grad_norm': '0.6094', 'learning_rate': '7.167e-05', 'entropy': '1.162', 'num_tokens': '1.174e+06', 'mean_token_accuracy': '0.7176', 'epoch': '0.001912'}
{'loss': '1.094', 'grad_norm': '0.5827', 'learning_rate': '7.2e-05', 'entropy': '1.07', 'num_tokens': '1.179e+06', 'mean_token_accuracy': '0.7522', 'epoch': '0.00192'}
{'loss': '1.068', 'grad_norm': '0.5543', 'learning_rate': '7.233e-05', 'entropy': '1.114', 'num_tokens': '1.184e+06', 'mean_token_accuracy': '0.7334', 'epoch': '0.001929'}
{'loss': '0.9231', 'grad_norm': '0.617', 'learning_rate': '7.267e-05', 'entropy': '1.111', 'num_tokens': '1.188e+06', 'mean_token_accuracy': '0.7257', 'epoch': '0.001938'}
{'loss': '0.8265', 'grad_norm': '0.5846', 'learning_rate': '7.3e-05', 'entropy': '0.9079', 'num_tokens': '1.192e+06', 'mean_token_accuracy': '0.7688', 'epoch': '0.001947'}
{'loss': '0.7706', 'grad_norm': '0.5714', 'learning_rate': '7.333e-05', 'entropy': '0.8313', 'num_tokens': '1.197e+06', 'mean_token_accuracy': '0.8032', 'epoch': '0.001956'}
{'loss': '0.7778', 'grad_norm': '0.5841', 'learning_rate': '7.367e-05', 'entropy': '0.8134', 'num_tokens': '1.202e+06', 'mean_token_accuracy': '0.8106', 'epoch': '0.001965'}
{'loss': '0.9116', 'grad_norm': '0.4206', 'learning_rate': '7.4e-05', 'entropy': '0.9313', 'num_tokens': '1.211e+06', 'mean_token_accuracy': '0.788', 'epoch': '0.001974'}
{'loss': '1.018', 'grad_norm': '0.6167', 'learning_rate': '7.433e-05', 'entropy': '1.055', 'num_tokens': '1.214e+06', 'mean_token_accuracy': '0.7495', 'epoch': '0.001982'}
{'loss': '1.126', 'grad_norm': '0.5853', 'learning_rate': '7.467e-05', 'entropy': '1.078', 'num_tokens': '1.22e+06', 'mean_token_accuracy': '0.735', 'epoch': '0.001991'}
{'loss': '0.9502', 'grad_norm': '0.9998', 'learning_rate': '7.5e-05', 'entropy': '1.009', 'num_tokens': '1.223e+06', 'mean_token_accuracy': '0.7466', 'epoch': '0.002'}
{'loss': '0.9365', 'grad_norm': '0.8', 'learning_rate': '7.533e-05', 'entropy': '0.9496', 'num_tokens': '1.229e+06', 'mean_token_accuracy': '0.7695', 'epoch': '0.002009'}
{'loss': '1.24', 'grad_norm': '0.43', 'learning_rate': '7.567e-05', 'entropy': '1.235', 'num_tokens': '1.236e+06', 'mean_token_accuracy': '0.7066', 'epoch': '0.002018'}
{'loss': '1.551', 'grad_norm': '0.4052', 'learning_rate': '7.6e-05', 'entropy': '1.502', 'num_tokens': '1.243e+06', 'mean_token_accuracy': '0.6392', 'epoch': '0.002027'}
{'loss': '1.223', 'grad_norm': '0.7479', 'learning_rate': '7.633e-05', 'entropy': '1.191', 'num_tokens': '1.249e+06', 'mean_token_accuracy': '0.7258', 'epoch': '0.002035'}
{'loss': '1.028', 'grad_norm': '0.5065', 'learning_rate': '7.667e-05', 'entropy': '1.004', 'num_tokens': '1.255e+06', 'mean_token_accuracy': '0.7565', 'epoch': '0.002044'}
{'loss': '1.108', 'grad_norm': '0.6363', 'learning_rate': '7.7e-05', 'entropy': '1.121', 'num_tokens': '1.259e+06', 'mean_token_accuracy': '0.734', 'epoch': '0.002053'}
{'loss': '0.9692', 'grad_norm': '0.612', 'learning_rate': '7.733e-05', 'entropy': '0.9133', 'num_tokens': '1.266e+06', 'mean_token_accuracy': '0.7614', 'epoch': '0.002062'}
{'loss': '1.124', 'grad_norm': '0.6718', 'learning_rate': '7.767e-05', 'entropy': '1.151', 'num_tokens': '1.27e+06', 'mean_token_accuracy': '0.7329', 'epoch': '0.002071'}
{'loss': '1.056', 'grad_norm': '0.5368', 'learning_rate': '7.8e-05', 'entropy': '0.997', 'num_tokens': '1.275e+06', 'mean_token_accuracy': '0.7619', 'epoch': '0.00208'}
{'loss': '0.9247', 'grad_norm': '0.3465', 'learning_rate': '7.833e-05', 'entropy': '0.8676', 'num_tokens': '1.286e+06', 'mean_token_accuracy': '0.7982', 'epoch': '0.002089'}

  2%|▏         | 237/12000 [08:38<7:30:32,  2.30s/it]
  2%|▏         | 238/12000 [08:40<7:00:32,  2.15s/it]
                                                     

  2%|▏         | 238/12000 [08:40<7:00:32,  2.15s/it]
  2%|▏         | 239/12000 [08:42<6:34:27,  2.01s/it]
                                                     

  2%|▏         | 239/12000 [08:42<6:34:27,  2.01s/it]
  2%|▏         | 240/12000 [08:44<6:19:53,  1.94s/it]
                                                     

  2%|▏         | 240/12000 [08:44<6:19:53,  1.94s/it]
  2%|▏         | 241/12000 [08:45<5:52:12,  1.80s/it]
                                                     

  2%|▏         | 241/12000 [08:45<5:52:12,  1.80s/it]
  2%|▏         | 242/12000 [08:49<7:47:46,  2.39s/it]
                                                     

  2%|▏         | 242/12000 [08:49<7:47:46,  2.39s/it]
  2%|▏         | 243/12000 [08:52<8:08:27,  2.49s/it]
                                                     

  2%|▏         | 243/12000 [08:52<8:08:27,  2.49s/it]
  2%|▏         | 244/12000 [08:54<7:54:33,  2.42s/it]
                                                     

  2%|▏         | 244/12000 [08:54<7:54:33,  2.42s/it]
  2%|▏         | 245/12000 [08:56<7:23:20,  2.26s/it]
                                                     

  2%|▏         | 245/12000 [08:56<7:23:20,  2.26s/it]
  2%|▏         | 246/12000 [08:57<6:36:22,  2.02s/it]
                                                     

  2%|▏         | 246/12000 [08:57<6:36:22,  2.02s/it]
  2%|▏         | 247/12000 [08:59<6:08:49,  1.88s/it]
                                                     

  2%|▏         | 247/12000 [08:59<6:08:49,  1.88s/it]
  2%|▏         | 248/12000 [09:01<6:49:20,  2.09s/it]
                                                     

  2%|▏         | 248/12000 [09:01<6:49:20,  2.09s/it]
  2%|▏         | 249/12000 [09:03<6:22:38,  1.95s/it]
                                                     

  2%|▏         | 249/12000 [09:03<6:22:38,  1.95s/it]
  2%|▏         | 250/12000 [09:05<6:08:05,  1.88s/it]
                                                     

  2%|▏         | 250/12000 [09:05<6:08:05,  1.88s/it]
  2%|▏         | 251/12000 [09:08<7:07:28,  2.18s/it]
                                                     

  2%|▏         | 251/12000 [09:08<7:07:28,  2.18s/it]
  2%|▏         | 252/12000 [09:09<6:27:39,  1.98s/it]
                                                     

  2%|▏         | 252/12000 [09:09<6:27:39,  1.98s/it]
  2%|▏         | 253/12000 [09:11<6:05:25,  1.87s/it]
                                                     

  2%|▏         | 253/12000 [09:11<6:05:25,  1.87s/it]
  2%|▏         | 254/12000 [09:12<5:44:17,  1.76s/it]
                                                     

  2%|▏         | 254/12000 [09:12<5:44:17,  1.76s/it]
  2%|▏         | 255/12000 [09:14<5:31:07,  1.69s/it]
                                                     

  2%|▏         | 255/12000 [09:14<5:31:07,  1.69s/it]
  2%|▏         | 256/12000 [09:16<5:56:16,  1.82s/it]
                                                     

  2%|▏         | 256/12000 [09:16<5:56:16,  1.82s/it]
  2%|▏         | 257/12000 [09:18<5:43:20,  1.75s/it]
                                                     

  2%|▏         | 257/12000 [09:18<5:43:20,  1.75s/it]
  2%|▏         | 258/12000 [09:19<5:37:09,  1.72s/it]
                                                     

  2%|▏         | 258/12000 [09:19<5:37:09,  1.72s/it]
  2%|▏         | 259/12000 [09:23<7:45:32,  2.38s/it]
                                                     

  2%|▏         | 259/12000 [09:23<7:45:32,  2.38s/it]
  2%|▏         | 260/12000 [09:25<7:16:49,  2.23s/it]
                                                     

  2%|▏         | 260/12000 [09:25<7:16:49,  2.23s/it]
  2%|▏         | 261/12000 [09:27<7:26:15,  2.28s/it]
                                                     

  2%|▏         | 261/12000 [09:27<7:26:15,  2.28s/it]
  2%|▏         | 262/12000 [09:30<7:20:20,  2.25s/it]
                                                     

  2%|▏         | 262/12000 [09:30<7:20:20,  2.25s/it]
  2%|▏         | 263/12000 [09:32<7:52:13,  2.41s/it]
                                                     

  2%|▏         | 263/12000 [09:32<7:52:13,  2.41s/it]
  2%|▏         | 264/12000 [09:38<11:14:03,  3.45s/it]
                                                      

  2%|▏         | 264/12000 [09:38<11:14:03,  3.45s/it]
  2%|▏         | 265/12000 [09:40<9:16:41,  2.85s/it] 
                                                     

  2%|▏         | 265/12000 [09:40<9:16:41,  2.85s/it]
  2%|▏         | 266/12000 [09:41<8:06:14,  2.49s/it]
                                                     

  2%|▏         | 266/12000 [09:41<8:06:14,  2.49s/it]
  2%|▏         | 267/12000 [09:43<7:17:51,  2.24s/it]
                                                     

  2%|▏         | 267/12000 [09:43<7:17:51,  2.24s/it]
  2%|▏         | 268/12000 [09:46<7:43:26,  2.37s/it]
                                                     

  2%|▏         | 268/12000 [09:46<7:43:26,  2.37s/it]
  2%|▏         | 269/12000 [09:48<7:52:26,  2.42s/it]
                                                     

  2%|▏         | 269/12000 [09:48<7:52:26,  2.42s/it]
  2%|▏         | 270/12000 [09:50<7:12:56,  2.21s/it]
                                                     

  2%|▏         | 270/12000 [09:50<7:12:56,  2.21s/it]
  2%|▏         | 271/12000 [09:51<6:32:26,  2.01s/it]
                                                     

  2%|▏         | 271/12000 [09:51<6:32:26,  2.01s/it]
  2%|▏         | 272/12000 [09:54<7:21:31,  2.26s/it]
                                                     

  2%|▏         | 272/12000 [09:54<7:21:31,  2.26s/it]
  2%|▏         | 273/12000 [09:56<6:36:49,  2.03s/it]
                                                     

  2%|▏         | 273/12000 [09:56<6:36:49,  2.03s/it]
  2%|▏         | 274/12000 [09:58<6:26:59,  1.98s/it]
                                                     

  2%|▏         | 274/12000 [09:58<6:26:59,  1.98s/it]
  2%|▏         | 275/12000 [10:00<6:29:07,  1.99s/it]
                                                     

  2%|▏         | 275/12000 [10:00<6:29:07,  1.99s/it]
  2%|▏         | 276/12000 [10:02<7:15:07,  2.23s/it]
                                                     

  2%|▏         | 276/12000 [10:02<7:15:07,  2.23s/it]
  2%|▏         | 277/12000 [10:04<6:34:11,  2.02s/it]
                                                     

  2%|▏         | 277/12000 [10:04<6:34:11,  2.02s/it]
  2%|▏         | 278/12000 [10:06<6:43:59,  2.07s/it]
                                                     

  2%|▏         | 278/12000 [10:06<6:43:59,  2.07s/it]
  2%|▏         | 279/12000 [10:08<6:20:54,  1.95s/it]
                                                     

  2%|▏         | 279/12000 [10:08<6:20:54,  1.95s/it]
  2%|▏         | 280/12000 [10:09<5:58:53,  1.84s/it]
                                                     

  2%|▏         | 280/12000 [10:09<5:58:53,  1.84s/it]
  2%|▏         | 281/12000 [10:11<6:02:19,  1.86s/it]
                                                     

  2%|▏         | 281/12000 [10:11<6:02:19,  1.86s/it]
  2%|▏         | 282/12000 [10:13<6:02:13,  1.85s/it]
                                                     

  2%|▏         | 282/12000 [10:13<6:02:13,  1.85s/it]
  2%|▏         | 283/12000 [10:15<5:43:12,  1.76s/it]
                                                     

  2%|▏         | 283/12000 [10:15<5:43:12,  1.76s/it]
  2%|▏         | 284/12000 [10:16<5:28:55,  1.68s/it]
                                                     
{'loss': '1.037', 'grad_norm': '0.5704', 'learning_rate': '7.867e-05', 'entropy': '0.9713', 'num_tokens': '1.291e+06', 'mean_token_accuracy': '0.7494', 'epoch': '0.002097'}
{'loss': '0.9107', 'grad_norm': '0.5559', 'learning_rate': '7.9e-05', 'entropy': '0.8741', 'num_tokens': '1.296e+06', 'mean_token_accuracy': '0.7868', 'epoch': '0.002106'}
{'loss': '0.9824', 'grad_norm': '0.5648', 'learning_rate': '7.933e-05', 'entropy': '1.057', 'num_tokens': '1.3e+06', 'mean_token_accuracy': '0.751', 'epoch': '0.002115'}
{'loss': '1.128', 'grad_norm': '0.5621', 'learning_rate': '7.967e-05', 'entropy': '1.141', 'num_tokens': '1.306e+06', 'mean_token_accuracy': '0.7348', 'epoch': '0.002124'}
{'loss': '0.9572', 'grad_norm': '0.6555', 'learning_rate': '8e-05', 'entropy': '1.048', 'num_tokens': '1.31e+06', 'mean_token_accuracy': '0.7513', 'epoch': '0.002133'}
{'loss': '1.644', 'grad_norm': '0.633', 'learning_rate': '8.033e-05', 'entropy': '1.591', 'num_tokens': '1.314e+06', 'mean_token_accuracy': '0.6273', 'epoch': '0.002142'}
{'loss': '0.9755', 'grad_norm': '0.5056', 'learning_rate': '8.067e-05', 'entropy': '0.9831', 'num_tokens': '1.32e+06', 'mean_token_accuracy': '0.7476', 'epoch': '0.002151'}
{'loss': '1.025', 'grad_norm': '0.5521', 'learning_rate': '8.1e-05', 'entropy': '0.9628', 'num_tokens': '1.327e+06', 'mean_token_accuracy': '0.7733', 'epoch': '0.002159'}
{'loss': '1.3', 'grad_norm': '0.5856', 'learning_rate': '8.133e-05', 'entropy': '1.285', 'num_tokens': '1.332e+06', 'mean_token_accuracy': '0.7068', 'epoch': '0.002168'}
{'loss': '1.026', 'grad_norm': '0.6755', 'learning_rate': '8.167e-05', 'entropy': '0.9974', 'num_tokens': '1.336e+06', 'mean_token_accuracy': '0.7472', 'epoch': '0.002177'}
{'loss': '1.136', 'grad_norm': '0.5506', 'learning_rate': '8.2e-05', 'entropy': '1.155', 'num_tokens': '1.342e+06', 'mean_token_accuracy': '0.7244', 'epoch': '0.002186'}
{'loss': '1.179', 'grad_norm': '0.6985', 'learning_rate': '8.233e-05', 'entropy': '1.18', 'num_tokens': '1.349e+06', 'mean_token_accuracy': '0.7217', 'epoch': '0.002195'}
{'loss': '0.9336', 'grad_norm': '0.4891', 'learning_rate': '8.267e-05', 'entropy': '0.9232', 'num_tokens': '1.354e+06', 'mean_token_accuracy': '0.7662', 'epoch': '0.002204'}
{'loss': '0.9976', 'grad_norm': '0.6663', 'learning_rate': '8.3e-05', 'entropy': '1.048', 'num_tokens': '1.359e+06', 'mean_token_accuracy': '0.7365', 'epoch': '0.002212'}
{'loss': '0.8649', 'grad_norm': '0.5834', 'learning_rate': '8.333e-05', 'entropy': '0.9258', 'num_tokens': '1.364e+06', 'mean_token_accuracy': '0.779', 'epoch': '0.002221'}
{'loss': '0.94', 'grad_norm': '0.6008', 'learning_rate': '8.367e-05', 'entropy': '0.9269', 'num_tokens': '1.368e+06', 'mean_token_accuracy': '0.7631', 'epoch': '0.00223'}
{'loss': '0.9752', 'grad_norm': '0.6756', 'learning_rate': '8.4e-05', 'entropy': '1.05', 'num_tokens': '1.371e+06', 'mean_token_accuracy': '0.7388', 'epoch': '0.002239'}
{'loss': '0.9386', 'grad_norm': '0.691', 'learning_rate': '8.433e-05', 'entropy': '0.9353', 'num_tokens': '1.375e+06', 'mean_token_accuracy': '0.7579', 'epoch': '0.002248'}
{'loss': '1.083', 'grad_norm': '0.5756', 'learning_rate': '8.467e-05', 'entropy': '1.059', 'num_tokens': '1.38e+06', 'mean_token_accuracy': '0.7417', 'epoch': '0.002257'}
{'loss': '1.049', 'grad_norm': '0.6024', 'learning_rate': '8.5e-05', 'entropy': '0.9926', 'num_tokens': '1.387e+06', 'mean_token_accuracy': '0.7696', 'epoch': '0.002266'}
{'loss': '1.042', 'grad_norm': '0.5877', 'learning_rate': '8.533e-05', 'entropy': '1.043', 'num_tokens': '1.391e+06', 'mean_token_accuracy': '0.7556', 'epoch': '0.002274'}
{'loss': '1.007', 'grad_norm': '0.5966', 'learning_rate': '8.567e-05', 'entropy': '1.045', 'num_tokens': '1.395e+06', 'mean_token_accuracy': '0.7436', 'epoch': '0.002283'}
{'loss': '0.9407', 'grad_norm': '0.4648', 'learning_rate': '8.6e-05', 'entropy': '0.9568', 'num_tokens': '1.403e+06', 'mean_token_accuracy': '0.7725', 'epoch': '0.002292'}
{'loss': '1.02', 'grad_norm': '0.6108', 'learning_rate': '8.633e-05', 'entropy': '0.9844', 'num_tokens': '1.408e+06', 'mean_token_accuracy': '0.7631', 'epoch': '0.002301'}
{'loss': '1.35', 'grad_norm': '0.5338', 'learning_rate': '8.667e-05', 'entropy': '1.297', 'num_tokens': '1.413e+06', 'mean_token_accuracy': '0.7077', 'epoch': '0.00231'}
{'loss': '1.204', 'grad_norm': '0.5196', 'learning_rate': '8.7e-05', 'entropy': '1.194', 'num_tokens': '1.419e+06', 'mean_token_accuracy': '0.7182', 'epoch': '0.002319'}
{'loss': '0.9921', 'grad_norm': '0.465', 'learning_rate': '8.733e-05', 'entropy': '1.131', 'num_tokens': '1.426e+06', 'mean_token_accuracy': '0.7309', 'epoch': '0.002328'}
{'loss': '0.8809', 'grad_norm': '0.3406', 'learning_rate': '8.767e-05', 'entropy': '0.9638', 'num_tokens': '1.434e+06', 'mean_token_accuracy': '0.7748', 'epoch': '0.002336'}
{'loss': '0.7848', 'grad_norm': '0.4991', 'learning_rate': '8.8e-05', 'entropy': '0.8252', 'num_tokens': '1.439e+06', 'mean_token_accuracy': '0.7969', 'epoch': '0.002345'}
{'loss': '1.211', 'grad_norm': '0.5473', 'learning_rate': '8.833e-05', 'entropy': '1.213', 'num_tokens': '1.443e+06', 'mean_token_accuracy': '0.7218', 'epoch': '0.002354'}
{'loss': '1.185', 'grad_norm': '0.5628', 'learning_rate': '8.867e-05', 'entropy': '1.156', 'num_tokens': '1.448e+06', 'mean_token_accuracy': '0.7293', 'epoch': '0.002363'}
{'loss': '1.386', 'grad_norm': '0.6021', 'learning_rate': '8.9e-05', 'entropy': '1.433', 'num_tokens': '1.454e+06', 'mean_token_accuracy': '0.6676', 'epoch': '0.002372'}
{'loss': '0.878', 'grad_norm': '1.341', 'learning_rate': '8.933e-05', 'entropy': '0.7995', 'num_tokens': '1.46e+06', 'mean_token_accuracy': '0.7961', 'epoch': '0.002381'}
{'loss': '1.158', 'grad_norm': '0.6938', 'learning_rate': '8.967e-05', 'entropy': '1.113', 'num_tokens': '1.465e+06', 'mean_token_accuracy': '0.721', 'epoch': '0.002389'}
{'loss': '1.191', 'grad_norm': '0.5224', 'learning_rate': '9e-05', 'entropy': '1.187', 'num_tokens': '1.47e+06', 'mean_token_accuracy': '0.7125', 'epoch': '0.002398'}
{'loss': '1.16', 'grad_norm': '0.4263', 'learning_rate': '9.033e-05', 'entropy': '1.088', 'num_tokens': '1.478e+06', 'mean_token_accuracy': '0.7381', 'epoch': '0.002407'}
{'loss': '0.9073', 'grad_norm': '0.4969', 'learning_rate': '9.067e-05', 'entropy': '0.9088', 'num_tokens': '1.483e+06', 'mean_token_accuracy': '0.787', 'epoch': '0.002416'}
{'loss': '1.262', 'grad_norm': '0.6185', 'learning_rate': '9.1e-05', 'entropy': '1.217', 'num_tokens': '1.487e+06', 'mean_token_accuracy': '0.7276', 'epoch': '0.002425'}
{'loss': '1.365', 'grad_norm': '0.5677', 'learning_rate': '9.133e-05', 'entropy': '1.291', 'num_tokens': '1.492e+06', 'mean_token_accuracy': '0.7143', 'epoch': '0.002434'}
{'loss': '0.9563', 'grad_norm': '0.4696', 'learning_rate': '9.167e-05', 'entropy': '0.872', 'num_tokens': '1.498e+06', 'mean_token_accuracy': '0.8025', 'epoch': '0.002443'}
{'loss': '1.002', 'grad_norm': '0.5442', 'learning_rate': '9.2e-05', 'entropy': '1.054', 'num_tokens': '1.503e+06', 'mean_token_accuracy': '0.7434', 'epoch': '0.002451'}
{'loss': '1.108', 'grad_norm': '0.4819', 'learning_rate': '9.233e-05', 'entropy': '1.155', 'num_tokens': '1.51e+06', 'mean_token_accuracy': '0.7201', 'epoch': '0.00246'}
{'loss': '1.28', 'grad_norm': '0.5643', 'learning_rate': '9.267e-05', 'entropy': '1.243', 'num_tokens': '1.515e+06', 'mean_token_accuracy': '0.7105', 'epoch': '0.002469'}
{'loss': '1.031', 'grad_norm': '0.5418', 'learning_rate': '9.3e-05', 'entropy': '1.079', 'num_tokens': '1.52e+06', 'mean_token_accuracy': '0.7329', 'epoch': '0.002478'}
{'loss': '1.295', 'grad_norm': '0.5479', 'learning_rate': '9.333e-05', 'entropy': '1.14', 'num_tokens': '1.525e+06', 'mean_token_accuracy': '0.7124', 'epoch': '0.002487'}
{'loss': '1.097', 'grad_norm': '0.5926', 'learning_rate': '9.367e-05', 'entropy': '1.073', 'num_tokens': '1.53e+06', 'mean_token_accuracy': '0.7363', 'epoch': '0.002496'}
{'loss': '1.003', 'grad_norm': '0.5315', 'learning_rate': '9.4e-05', 'entropy': '1.02', 'num_tokens': '1.535e+06', 'mean_token_accuracy': '0.7551', 'epoch': '0.002505'}

  2%|▏         | 284/12000 [10:16<5:28:55,  1.68s/it]
  2%|▏         | 285/12000 [10:19<6:32:36,  2.01s/it]
                                                     

  2%|▏         | 285/12000 [10:19<6:32:36,  2.01s/it]
  2%|▏         | 286/12000 [10:24<9:16:27,  2.85s/it]
                                                     

  2%|▏         | 286/12000 [10:24<9:16:27,  2.85s/it]
  2%|▏         | 287/12000 [10:26<8:13:05,  2.53s/it]
                                                     

  2%|▏         | 287/12000 [10:26<8:13:05,  2.53s/it]
  2%|▏         | 288/12000 [10:28<8:09:39,  2.51s/it]
                                                     

  2%|▏         | 288/12000 [10:28<8:09:39,  2.51s/it]
  2%|▏         | 289/12000 [10:30<7:19:26,  2.25s/it]
                                                     

  2%|▏         | 289/12000 [10:30<7:19:26,  2.25s/it]
  2%|▏         | 290/12000 [10:31<6:26:07,  1.98s/it]
                                                     

  2%|▏         | 290/12000 [10:31<6:26:07,  1.98s/it]
  2%|▏         | 291/12000 [10:37<10:12:38,  3.14s/it]
                                                      

  2%|▏         | 291/12000 [10:37<10:12:38,  3.14s/it]
  2%|▏         | 292/12000 [10:39<9:17:49,  2.86s/it] 
                                                     

  2%|▏         | 292/12000 [10:39<9:17:49,  2.86s/it]
  2%|▏         | 293/12000 [10:41<8:08:04,  2.50s/it]
                                                     

  2%|▏         | 293/12000 [10:41<8:08:04,  2.50s/it]
  2%|▏         | 294/12000 [10:42<7:18:06,  2.25s/it]
                                                     

  2%|▏         | 294/12000 [10:42<7:18:06,  2.25s/it]
  2%|▏         | 295/12000 [10:44<6:56:56,  2.14s/it]
                                                     

  2%|▏         | 295/12000 [10:44<6:56:56,  2.14s/it]
  2%|▏         | 296/12000 [10:46<6:26:49,  1.98s/it]
                                                     

  2%|▏         | 296/12000 [10:46<6:26:49,  1.98s/it]
  2%|▏         | 297/12000 [10:47<6:00:51,  1.85s/it]
                                                     

  2%|▏         | 297/12000 [10:47<6:00:51,  1.85s/it]
  2%|▏         | 298/12000 [10:50<6:25:44,  1.98s/it]
                                                     

  2%|▏         | 298/12000 [10:50<6:25:44,  1.98s/it]
  2%|▏         | 299/12000 [10:51<5:57:57,  1.84s/it]
                                                     

  2%|▏         | 299/12000 [10:51<5:57:57,  1.84s/it]
  2%|▎         | 300/12000 [10:53<6:02:59,  1.86s/it]
                                                     

  2%|▎         | 300/12000 [10:53<6:02:59,  1.86s/it]
  3%|▎         | 301/12000 [10:56<7:08:02,  2.20s/it]
                                                     

  3%|▎         | 301/12000 [10:56<7:08:02,  2.20s/it]
  3%|▎         | 302/12000 [10:58<7:05:30,  2.18s/it]
                                                     

  3%|▎         | 302/12000 [10:58<7:05:30,  2.18s/it]
  3%|▎         | 303/12000 [11:00<6:32:41,  2.01s/it]
                                                     

  3%|▎         | 303/12000 [11:00<6:32:41,  2.01s/it]
  3%|▎         | 304/12000 [11:02<6:26:13,  1.98s/it]
                                                     

  3%|▎         | 304/12000 [11:02<6:26:13,  1.98s/it]
  3%|▎         | 305/12000 [11:03<6:00:48,  1.85s/it]
                                                     

  3%|▎         | 305/12000 [11:03<6:00:48,  1.85s/it]
  3%|▎         | 306/12000 [11:06<7:02:50,  2.17s/it]
                                                     

  3%|▎         | 306/12000 [11:06<7:02:50,  2.17s/it]
  3%|▎         | 307/12000 [11:08<6:25:12,  1.98s/it]
                                                     

  3%|▎         | 307/12000 [11:08<6:25:12,  1.98s/it]
  3%|▎         | 308/12000 [11:12<8:54:41,  2.74s/it]
                                                     

  3%|▎         | 308/12000 [11:12<8:54:41,  2.74s/it]
  3%|▎         | 309/12000 [11:14<7:46:45,  2.40s/it]
                                                     

  3%|▎         | 309/12000 [11:14<7:46:45,  2.40s/it]
  3%|▎         | 310/12000 [11:16<7:52:46,  2.43s/it]
                                                     

  3%|▎         | 310/12000 [11:16<7:52:46,  2.43s/it]
  3%|▎         | 311/12000 [11:19<7:37:02,  2.35s/it]
                                                     

  3%|▎         | 311/12000 [11:19<7:37:02,  2.35s/it]
  3%|▎         | 312/12000 [11:21<7:35:41,  2.34s/it]
                                                     

  3%|▎         | 312/12000 [11:21<7:35:41,  2.34s/it]
  3%|▎         | 313/12000 [11:23<7:53:03,  2.43s/it]
                                                     

  3%|▎         | 313/12000 [11:23<7:53:03,  2.43s/it]
  3%|▎         | 314/12000 [11:25<6:54:26,  2.13s/it]
                                                     

  3%|▎         | 314/12000 [11:25<6:54:26,  2.13s/it]
  3%|▎         | 315/12000 [11:28<7:30:05,  2.31s/it]
                                                     

  3%|▎         | 315/12000 [11:28<7:30:05,  2.31s/it]
  3%|▎         | 316/12000 [11:31<8:42:26,  2.68s/it]
                                                     

  3%|▎         | 316/12000 [11:31<8:42:26,  2.68s/it]
  3%|▎         | 317/12000 [11:33<7:32:08,  2.32s/it]
                                                     

  3%|▎         | 317/12000 [11:33<7:32:08,  2.32s/it]
  3%|▎         | 318/12000 [11:34<6:52:32,  2.12s/it]
                                                     

  3%|▎         | 318/12000 [11:34<6:52:32,  2.12s/it]
  3%|▎         | 319/12000 [11:36<6:33:12,  2.02s/it]
                                                     

  3%|▎         | 319/12000 [11:36<6:33:12,  2.02s/it]
  3%|▎         | 320/12000 [11:38<6:03:00,  1.86s/it]
                                                     

  3%|▎         | 320/12000 [11:38<6:03:00,  1.86s/it]
  3%|▎         | 321/12000 [11:39<5:49:47,  1.80s/it]
                                                     

  3%|▎         | 321/12000 [11:39<5:49:47,  1.80s/it]
  3%|▎         | 322/12000 [11:42<7:01:34,  2.17s/it]
                                                     

  3%|▎         | 322/12000 [11:42<7:01:34,  2.17s/it]
  3%|▎         | 323/12000 [11:44<6:16:34,  1.93s/it]
                                                     

  3%|▎         | 323/12000 [11:44<6:16:34,  1.93s/it]
  3%|▎         | 324/12000 [11:46<6:29:35,  2.00s/it]
                                                     

  3%|▎         | 324/12000 [11:46<6:29:35,  2.00s/it]
  3%|▎         | 325/12000 [11:48<6:57:37,  2.15s/it]
                                                     

  3%|▎         | 325/12000 [11:48<6:57:37,  2.15s/it]
  3%|▎         | 326/12000 [11:50<6:20:26,  1.96s/it]
                                                     

  3%|▎         | 326/12000 [11:50<6:20:26,  1.96s/it]
  3%|▎         | 327/12000 [11:51<6:01:46,  1.86s/it]
                                                     

  3%|▎         | 327/12000 [11:51<6:01:46,  1.86s/it]
  3%|▎         | 328/12000 [11:53<5:57:17,  1.84s/it]
                                                     

  3%|▎         | 328/12000 [11:53<5:57:17,  1.84s/it]
  3%|▎         | 329/12000 [11:55<5:42:48,  1.76s/it]
                                                     

  3%|▎         | 329/12000 [11:55<5:42:48,  1.76s/it]
  3%|▎         | 330/12000 [12:00<8:49:03,  2.72s/it]
                                                     

  3%|▎         | 330/12000 [12:00<8:49:03,  2.72s/it]
  3%|▎         | 331/12000 [12:01<7:46:03,  2.40s/it]
                                                     
{'loss': '0.9704', 'grad_norm': '0.6061', 'learning_rate': '9.433e-05', 'entropy': '0.9922', 'num_tokens': '1.539e+06', 'mean_token_accuracy': '0.7616', 'epoch': '0.002513'}
{'loss': '1.724', 'grad_norm': '0.513', 'learning_rate': '9.467e-05', 'entropy': '1.475', 'num_tokens': '1.546e+06', 'mean_token_accuracy': '0.6828', 'epoch': '0.002522'}
{'loss': '0.9911', 'grad_norm': '0.7074', 'learning_rate': '9.5e-05', 'entropy': '1.024', 'num_tokens': '1.552e+06', 'mean_token_accuracy': '0.7585', 'epoch': '0.002531'}
{'loss': '0.9534', 'grad_norm': '0.4602', 'learning_rate': '9.533e-05', 'entropy': '0.9492', 'num_tokens': '1.558e+06', 'mean_token_accuracy': '0.7664', 'epoch': '0.00254'}
{'loss': '1.172', 'grad_norm': '0.5011', 'learning_rate': '9.567e-05', 'entropy': '1.145', 'num_tokens': '1.565e+06', 'mean_token_accuracy': '0.7302', 'epoch': '0.002549'}
{'loss': '0.9609', 'grad_norm': '0.4858', 'learning_rate': '9.6e-05', 'entropy': '0.9293', 'num_tokens': '1.569e+06', 'mean_token_accuracy': '0.7786', 'epoch': '0.002558'}
{'loss': '1.37', 'grad_norm': '0.5673', 'learning_rate': '9.633e-05', 'entropy': '1.377', 'num_tokens': '1.573e+06', 'mean_token_accuracy': '0.6815', 'epoch': '0.002566'}
{'loss': '1.224', 'grad_norm': '0.3962', 'learning_rate': '9.667e-05', 'entropy': '1.097', 'num_tokens': '1.584e+06', 'mean_token_accuracy': '0.7449', 'epoch': '0.002575'}
{'loss': '1.271', 'grad_norm': '0.4869', 'learning_rate': '9.7e-05', 'entropy': '1.197', 'num_tokens': '1.59e+06', 'mean_token_accuracy': '0.7123', 'epoch': '0.002584'}
{'loss': '0.858', 'grad_norm': '0.5411', 'learning_rate': '9.733e-05', 'entropy': '0.8442', 'num_tokens': '1.595e+06', 'mean_token_accuracy': '0.7947', 'epoch': '0.002593'}
{'loss': '0.781', 'grad_norm': '0.5816', 'learning_rate': '9.767e-05', 'entropy': '0.82', 'num_tokens': '1.6e+06', 'mean_token_accuracy': '0.8038', 'epoch': '0.002602'}
{'loss': '0.8481', 'grad_norm': '0.5097', 'learning_rate': '9.8e-05', 'entropy': '0.8832', 'num_tokens': '1.605e+06', 'mean_token_accuracy': '0.7863', 'epoch': '0.002611'}
{'loss': '0.7585', 'grad_norm': '0.4765', 'learning_rate': '9.833e-05', 'entropy': '0.7571', 'num_tokens': '1.611e+06', 'mean_token_accuracy': '0.7964', 'epoch': '0.00262'}
{'loss': '1.188', 'grad_norm': '0.5229', 'learning_rate': '9.867e-05', 'entropy': '1.224', 'num_tokens': '1.615e+06', 'mean_token_accuracy': '0.7291', 'epoch': '0.002628'}
{'loss': '0.962', 'grad_norm': '0.4576', 'learning_rate': '9.9e-05', 'entropy': '0.963', 'num_tokens': '1.622e+06', 'mean_token_accuracy': '0.766', 'epoch': '0.002637'}
{'loss': '0.9899', 'grad_norm': '0.6342', 'learning_rate': '9.933e-05', 'entropy': '0.9848', 'num_tokens': '1.625e+06', 'mean_token_accuracy': '0.7608', 'epoch': '0.002646'}
{'loss': '1.097', 'grad_norm': '0.4935', 'learning_rate': '9.967e-05', 'entropy': '1.072', 'num_tokens': '1.631e+06', 'mean_token_accuracy': '0.7382', 'epoch': '0.002655'}
{'loss': '1.053', 'grad_norm': '0.4297', 'learning_rate': '0.0001', 'entropy': '1.063', 'num_tokens': '1.635e+06', 'mean_token_accuracy': '0.7583', 'epoch': '0.002664'}
{'loss': '1.044', 'grad_norm': '0.5588', 'learning_rate': '0.0001003', 'entropy': '1.177', 'num_tokens': '1.641e+06', 'mean_token_accuracy': '0.7346', 'epoch': '0.002673'}
{'loss': '1.037', 'grad_norm': '0.5166', 'learning_rate': '0.0001007', 'entropy': '0.997', 'num_tokens': '1.646e+06', 'mean_token_accuracy': '0.7708', 'epoch': '0.002681'}
{'loss': '0.9309', 'grad_norm': '0.4853', 'learning_rate': '0.000101', 'entropy': '0.8704', 'num_tokens': '1.652e+06', 'mean_token_accuracy': '0.8093', 'epoch': '0.00269'}
{'loss': '1.129', 'grad_norm': '0.5011', 'learning_rate': '0.0001013', 'entropy': '1.122', 'num_tokens': '1.656e+06', 'mean_token_accuracy': '0.7327', 'epoch': '0.002699'}
{'loss': '1.049', 'grad_norm': '0.4837', 'learning_rate': '0.0001017', 'entropy': '1.03', 'num_tokens': '1.664e+06', 'mean_token_accuracy': '0.7494', 'epoch': '0.002708'}
{'loss': '1.09', 'grad_norm': '0.5863', 'learning_rate': '0.000102', 'entropy': '1.103', 'num_tokens': '1.668e+06', 'mean_token_accuracy': '0.7212', 'epoch': '0.002717'}
{'loss': '0.8201', 'grad_norm': '0.6096', 'learning_rate': '0.0001023', 'entropy': '0.8943', 'num_tokens': '1.673e+06', 'mean_token_accuracy': '0.782', 'epoch': '0.002726'}
{'loss': '0.8572', 'grad_norm': '0.6554', 'learning_rate': '0.0001027', 'entropy': '0.9786', 'num_tokens': '1.677e+06', 'mean_token_accuracy': '0.7573', 'epoch': '0.002735'}
{'loss': '1.08', 'grad_norm': '0.441', 'learning_rate': '0.000103', 'entropy': '1.136', 'num_tokens': '1.684e+06', 'mean_token_accuracy': '0.7335', 'epoch': '0.002743'}
{'loss': '1.511', 'grad_norm': '0.5039', 'learning_rate': '0.0001033', 'entropy': '1.449', 'num_tokens': '1.69e+06', 'mean_token_accuracy': '0.6712', 'epoch': '0.002752'}
{'loss': '0.8147', 'grad_norm': '0.4098', 'learning_rate': '0.0001037', 'entropy': '0.8168', 'num_tokens': '1.695e+06', 'mean_token_accuracy': '0.7974', 'epoch': '0.002761'}
{'loss': '1.168', 'grad_norm': '0.4495', 'learning_rate': '0.000104', 'entropy': '1.123', 'num_tokens': '1.702e+06', 'mean_token_accuracy': '0.736', 'epoch': '0.00277'}
{'loss': '0.9836', 'grad_norm': '0.5955', 'learning_rate': '0.0001043', 'entropy': '1.002', 'num_tokens': '1.706e+06', 'mean_token_accuracy': '0.7449', 'epoch': '0.002779'}
{'loss': '0.8581', 'grad_norm': '0.5703', 'learning_rate': '0.0001047', 'entropy': '0.9362', 'num_tokens': '1.711e+06', 'mean_token_accuracy': '0.7729', 'epoch': '0.002788'}
{'loss': '0.9629', 'grad_norm': '0.3577', 'learning_rate': '0.000105', 'entropy': '1.007', 'num_tokens': '1.721e+06', 'mean_token_accuracy': '0.7672', 'epoch': '0.002797'}
{'loss': '0.7604', 'grad_norm': '0.4534', 'learning_rate': '0.0001053', 'entropy': '0.8008', 'num_tokens': '1.726e+06', 'mean_token_accuracy': '0.8085', 'epoch': '0.002805'}
{'loss': '0.8296', 'grad_norm': '0.4901', 'learning_rate': '0.0001057', 'entropy': '0.8976', 'num_tokens': '1.73e+06', 'mean_token_accuracy': '0.7822', 'epoch': '0.002814'}
{'loss': '0.8059', 'grad_norm': '0.5199', 'learning_rate': '0.000106', 'entropy': '0.8679', 'num_tokens': '1.735e+06', 'mean_token_accuracy': '0.7945', 'epoch': '0.002823'}
{'loss': '1.048', 'grad_norm': '0.5564', 'learning_rate': '0.0001063', 'entropy': '1.15', 'num_tokens': '1.739e+06', 'mean_token_accuracy': '0.7219', 'epoch': '0.002832'}
{'loss': '1.368', 'grad_norm': '0.5566', 'learning_rate': '0.0001067', 'entropy': '1.422', 'num_tokens': '1.743e+06', 'mean_token_accuracy': '0.6697', 'epoch': '0.002841'}
{'loss': '0.9132', 'grad_norm': '0.4511', 'learning_rate': '0.000107', 'entropy': '0.9559', 'num_tokens': '1.751e+06', 'mean_token_accuracy': '0.762', 'epoch': '0.00285'}
{'loss': '1.293', 'grad_norm': '0.6207', 'learning_rate': '0.0001073', 'entropy': '1.358', 'num_tokens': '1.754e+06', 'mean_token_accuracy': '0.6754', 'epoch': '0.002858'}
{'loss': '1.241', 'grad_norm': '0.5283', 'learning_rate': '0.0001077', 'entropy': '1.17', 'num_tokens': '1.759e+06', 'mean_token_accuracy': '0.7139', 'epoch': '0.002867'}
{'loss': '1.22', 'grad_norm': '0.4659', 'learning_rate': '0.000108', 'entropy': '1.207', 'num_tokens': '1.766e+06', 'mean_token_accuracy': '0.7154', 'epoch': '0.002876'}
{'loss': '0.9625', 'grad_norm': '0.5402', 'learning_rate': '0.0001083', 'entropy': '1.052', 'num_tokens': '1.77e+06', 'mean_token_accuracy': '0.7478', 'epoch': '0.002885'}
{'loss': '1.066', 'grad_norm': '0.4608', 'learning_rate': '0.0001087', 'entropy': '1.103', 'num_tokens': '1.774e+06', 'mean_token_accuracy': '0.7432', 'epoch': '0.002894'}
{'loss': '0.9521', 'grad_norm': '0.5389', 'learning_rate': '0.000109', 'entropy': '0.9878', 'num_tokens': '1.779e+06', 'mean_token_accuracy': '0.7541', 'epoch': '0.002903'}
{'loss': '0.9463', 'grad_norm': '0.4996', 'learning_rate': '0.0001093', 'entropy': '1.065', 'num_tokens': '1.784e+06', 'mean_token_accuracy': '0.7455', 'epoch': '0.002912'}
{'loss': '1.364', 'grad_norm': '0.4317', 'learning_rate': '0.0001097', 'entropy': '1.382', 'num_tokens': '1.792e+06', 'mean_token_accuracy': '0.6912', 'epoch': '0.00292'}

  3%|▎         | 331/12000 [12:01<7:46:03,  2.40s/it]
  3%|▎         | 332/12000 [12:03<6:54:08,  2.13s/it]
                                                     

  3%|▎         | 332/12000 [12:03<6:54:08,  2.13s/it]
  3%|▎         | 333/12000 [12:05<6:45:17,  2.08s/it]
                                                     

  3%|▎         | 333/12000 [12:05<6:45:17,  2.08s/it]
  3%|▎         | 334/12000 [12:06<6:16:44,  1.94s/it]
                                                     

  3%|▎         | 334/12000 [12:06<6:16:44,  1.94s/it]
  3%|▎         | 335/12000 [12:08<6:07:48,  1.89s/it]
                                                     

  3%|▎         | 335/12000 [12:08<6:07:48,  1.89s/it]
  3%|▎         | 336/12000 [12:11<7:08:12,  2.20s/it]
                                                     

  3%|▎         | 336/12000 [12:11<7:08:12,  2.20s/it]
  3%|▎         | 337/12000 [12:14<7:27:02,  2.30s/it]
                                                     

  3%|▎         | 337/12000 [12:14<7:27:02,  2.30s/it]
  3%|▎         | 338/12000 [12:16<7:08:05,  2.20s/it]
                                                     

  3%|▎         | 338/12000 [12:16<7:08:05,  2.20s/it]
  3%|▎         | 339/12000 [12:17<6:30:53,  2.01s/it]
                                                     

  3%|▎         | 339/12000 [12:17<6:30:53,  2.01s/it]
  3%|▎         | 340/12000 [12:20<7:16:48,  2.25s/it]
                                                     

  3%|▎         | 340/12000 [12:20<7:16:48,  2.25s/it]
  3%|▎         | 341/12000 [12:22<6:37:29,  2.05s/it]
                                                     

  3%|▎         | 341/12000 [12:22<6:37:29,  2.05s/it]
  3%|▎         | 342/12000 [12:24<6:33:35,  2.03s/it]
                                                     

  3%|▎         | 342/12000 [12:24<6:33:35,  2.03s/it]
  3%|▎         | 343/12000 [12:25<5:58:21,  1.84s/it]
                                                     

  3%|▎         | 343/12000 [12:25<5:58:21,  1.84s/it]
  3%|▎         | 344/12000 [12:27<5:48:42,  1.79s/it]
                                                     

  3%|▎         | 344/12000 [12:27<5:48:42,  1.79s/it]
  3%|▎         | 345/12000 [12:28<5:27:23,  1.69s/it]
                                                     

  3%|▎         | 345/12000 [12:28<5:27:23,  1.69s/it]
  3%|▎         | 346/12000 [12:30<5:23:26,  1.67s/it]
                                                     

  3%|▎         | 346/12000 [12:30<5:23:26,  1.67s/it]
  3%|▎         | 347/12000 [12:31<5:22:59,  1.66s/it]
                                                     

  3%|▎         | 347/12000 [12:31<5:22:59,  1.66s/it]
  3%|▎         | 348/12000 [12:34<6:19:45,  1.96s/it]
                                                     

  3%|▎         | 348/12000 [12:34<6:19:45,  1.96s/it]
  3%|▎         | 349/12000 [12:36<6:03:51,  1.87s/it]
                                                     

  3%|▎         | 349/12000 [12:36<6:03:51,  1.87s/it]
  3%|▎         | 350/12000 [12:37<5:49:42,  1.80s/it]
                                                     

  3%|▎         | 350/12000 [12:37<5:49:42,  1.80s/it]
  3%|▎         | 351/12000 [12:42<8:09:03,  2.52s/it]
                                                     

  3%|▎         | 351/12000 [12:42<8:09:03,  2.52s/it]
  3%|▎         | 352/12000 [12:48<11:37:40,  3.59s/it]
                                                      

  3%|▎         | 352/12000 [12:48<11:37:40,  3.59s/it]
  3%|▎         | 353/12000 [12:49<9:31:47,  2.95s/it] 
                                                     

  3%|▎         | 353/12000 [12:49<9:31:47,  2.95s/it]
  3%|▎         | 354/12000 [12:51<8:21:19,  2.58s/it]
                                                     

  3%|▎         | 354/12000 [12:51<8:21:19,  2.58s/it]
  3%|▎         | 355/12000 [12:52<7:20:21,  2.27s/it]
                                                     

  3%|▎         | 355/12000 [12:52<7:20:21,  2.27s/it]
  3%|▎         | 356/12000 [12:54<6:56:27,  2.15s/it]
                                                     

  3%|▎         | 356/12000 [12:54<6:56:27,  2.15s/it]
  3%|▎         | 357/12000 [12:56<6:25:47,  1.99s/it]
                                                     

  3%|▎         | 357/12000 [12:56<6:25:47,  1.99s/it]
  3%|▎         | 358/12000 [12:58<6:24:46,  1.98s/it]
                                                     

  3%|▎         | 358/12000 [12:58<6:24:46,  1.98s/it]
  3%|▎         | 359/12000 [12:59<5:53:47,  1.82s/it]
                                                     

  3%|▎         | 359/12000 [12:59<5:53:47,  1.82s/it]
  3%|▎         | 360/12000 [13:01<5:29:42,  1.70s/it]
                                                     

  3%|▎         | 360/12000 [13:01<5:29:42,  1.70s/it]
  3%|▎         | 361/12000 [13:03<5:59:39,  1.85s/it]
                                                     

  3%|▎         | 361/12000 [13:03<5:59:39,  1.85s/it]
  3%|▎         | 362/12000 [13:05<6:08:20,  1.90s/it]
                                                     

  3%|▎         | 362/12000 [13:05<6:08:20,  1.90s/it]
  3%|▎         | 363/12000 [13:06<5:42:49,  1.77s/it]
                                                     

  3%|▎         | 363/12000 [13:06<5:42:49,  1.77s/it]
  3%|▎         | 364/12000 [13:08<5:29:22,  1.70s/it]
                                                     

  3%|▎         | 364/12000 [13:08<5:29:22,  1.70s/it]
  3%|▎         | 365/12000 [13:11<6:28:33,  2.00s/it]
                                                     

  3%|▎         | 365/12000 [13:11<6:28:33,  2.00s/it]
  3%|▎         | 366/12000 [13:13<6:28:15,  2.00s/it]
                                                     

  3%|▎         | 366/12000 [13:13<6:28:15,  2.00s/it]
  3%|▎         | 367/12000 [13:15<6:56:20,  2.15s/it]
                                                     

  3%|▎         | 367/12000 [13:15<6:56:20,  2.15s/it]
  3%|▎         | 368/12000 [13:17<6:14:34,  1.93s/it]
                                                     

  3%|▎         | 368/12000 [13:17<6:14:34,  1.93s/it]
  3%|▎         | 369/12000 [13:19<6:52:11,  2.13s/it]
                                                     

  3%|▎         | 369/12000 [13:19<6:52:11,  2.13s/it]
  3%|▎         | 370/12000 [13:21<6:49:42,  2.11s/it]
                                                     

  3%|▎         | 370/12000 [13:21<6:49:42,  2.11s/it]
  3%|▎         | 371/12000 [13:23<6:18:08,  1.95s/it]
                                                     

  3%|▎         | 371/12000 [13:23<6:18:08,  1.95s/it]
  3%|▎         | 372/12000 [13:25<6:10:13,  1.91s/it]
                                                     

  3%|▎         | 372/12000 [13:25<6:10:13,  1.91s/it]
  3%|▎         | 373/12000 [13:26<5:44:39,  1.78s/it]
                                                     

  3%|▎         | 373/12000 [13:26<5:44:39,  1.78s/it]
  3%|▎         | 374/12000 [13:30<7:33:28,  2.34s/it]
                                                     

  3%|▎         | 374/12000 [13:30<7:33:28,  2.34s/it]
  3%|▎         | 375/12000 [13:31<6:52:28,  2.13s/it]
                                                     

  3%|▎         | 375/12000 [13:31<6:52:28,  2.13s/it]
  3%|▎         | 376/12000 [13:33<6:26:03,  1.99s/it]
                                                     

  3%|▎         | 376/12000 [13:33<6:26:03,  1.99s/it]
  3%|▎         | 377/12000 [13:35<6:13:48,  1.93s/it]
                                                     

  3%|▎         | 377/12000 [13:35<6:13:48,  1.93s/it]
  3%|▎         | 378/12000 [13:37<6:14:14,  1.93s/it]
                                                     
{'loss': '1.014', 'grad_norm': '0.5866', 'learning_rate': '0.00011', 'entropy': '1.051', 'num_tokens': '1.796e+06', 'mean_token_accuracy': '0.7308', 'epoch': '0.002929'}
{'loss': '0.9473', 'grad_norm': '0.5664', 'learning_rate': '0.0001103', 'entropy': '0.9994', 'num_tokens': '1.801e+06', 'mean_token_accuracy': '0.7649', 'epoch': '0.002938'}
{'loss': '1.09', 'grad_norm': '0.4132', 'learning_rate': '0.0001107', 'entropy': '1.197', 'num_tokens': '1.807e+06', 'mean_token_accuracy': '0.7276', 'epoch': '0.002947'}
{'loss': '0.9566', 'grad_norm': '0.4405', 'learning_rate': '0.000111', 'entropy': '1.026', 'num_tokens': '1.812e+06', 'mean_token_accuracy': '0.7508', 'epoch': '0.002956'}
{'loss': '1.424', 'grad_norm': '0.4937', 'learning_rate': '0.0001113', 'entropy': '1.366', 'num_tokens': '1.816e+06', 'mean_token_accuracy': '0.6956', 'epoch': '0.002965'}
{'loss': '1.006', 'grad_norm': '0.4949', 'learning_rate': '0.0001117', 'entropy': '1.065', 'num_tokens': '1.823e+06', 'mean_token_accuracy': '0.7448', 'epoch': '0.002974'}
{'loss': '1.183', 'grad_norm': '0.5082', 'learning_rate': '0.000112', 'entropy': '1.07', 'num_tokens': '1.829e+06', 'mean_token_accuracy': '0.7358', 'epoch': '0.002982'}
{'loss': '1.326', 'grad_norm': '0.4228', 'learning_rate': '0.0001123', 'entropy': '1.281', 'num_tokens': '1.835e+06', 'mean_token_accuracy': '0.6885', 'epoch': '0.002991'}
{'loss': '0.9418', 'grad_norm': '0.49', 'learning_rate': '0.0001127', 'entropy': '0.9209', 'num_tokens': '1.84e+06', 'mean_token_accuracy': '0.7756', 'epoch': '0.003'}
{'loss': '1.045', 'grad_norm': '0.5655', 'learning_rate': '0.000113', 'entropy': '0.9691', 'num_tokens': '1.845e+06', 'mean_token_accuracy': '0.7499', 'epoch': '0.003009'}
{'loss': '0.8879', 'grad_norm': '0.482', 'learning_rate': '0.0001133', 'entropy': '0.9383', 'num_tokens': '1.849e+06', 'mean_token_accuracy': '0.7657', 'epoch': '0.003018'}
{'loss': '1.166', 'grad_norm': '0.4975', 'learning_rate': '0.0001137', 'entropy': '0.9778', 'num_tokens': '1.855e+06', 'mean_token_accuracy': '0.7574', 'epoch': '0.003027'}
{'loss': '0.9919', 'grad_norm': '0.5407', 'learning_rate': '0.000114', 'entropy': '0.9332', 'num_tokens': '1.858e+06', 'mean_token_accuracy': '0.7875', 'epoch': '0.003035'}
{'loss': '1.197', 'grad_norm': '0.4776', 'learning_rate': '0.0001143', 'entropy': '1.382', 'num_tokens': '1.864e+06', 'mean_token_accuracy': '0.6836', 'epoch': '0.003044'}
{'loss': '1.196', 'grad_norm': '0.5777', 'learning_rate': '0.0001147', 'entropy': '1.192', 'num_tokens': '1.868e+06', 'mean_token_accuracy': '0.7354', 'epoch': '0.003053'}
{'loss': '0.8889', 'grad_norm': '0.4093', 'learning_rate': '0.000115', 'entropy': '0.9142', 'num_tokens': '1.872e+06', 'mean_token_accuracy': '0.7767', 'epoch': '0.003062'}
{'loss': '1.37', 'grad_norm': '0.5271', 'learning_rate': '0.0001153', 'entropy': '1.381', 'num_tokens': '1.877e+06', 'mean_token_accuracy': '0.6781', 'epoch': '0.003071'}
{'loss': '0.9183', 'grad_norm': '0.3609', 'learning_rate': '0.0001157', 'entropy': '0.9715', 'num_tokens': '1.884e+06', 'mean_token_accuracy': '0.7704', 'epoch': '0.00308'}
{'loss': '1.067', 'grad_norm': '0.4331', 'learning_rate': '0.000116', 'entropy': '1.056', 'num_tokens': '1.889e+06', 'mean_token_accuracy': '0.7448', 'epoch': '0.003089'}
{'loss': '0.8852', 'grad_norm': '0.4229', 'learning_rate': '0.0001163', 'entropy': '0.9123', 'num_tokens': '1.895e+06', 'mean_token_accuracy': '0.7823', 'epoch': '0.003097'}
{'loss': '1.054', 'grad_norm': '0.3569', 'learning_rate': '0.0001167', 'entropy': '1.131', 'num_tokens': '1.902e+06', 'mean_token_accuracy': '0.7341', 'epoch': '0.003106'}
{'loss': '0.7206', 'grad_norm': '0.3654', 'learning_rate': '0.000117', 'entropy': '0.8354', 'num_tokens': '1.91e+06', 'mean_token_accuracy': '0.8023', 'epoch': '0.003115'}
{'loss': '1.064', 'grad_norm': '0.5763', 'learning_rate': '0.0001173', 'entropy': '1.175', 'num_tokens': '1.914e+06', 'mean_token_accuracy': '0.7169', 'epoch': '0.003124'}
{'loss': '1.369', 'grad_norm': '0.5092', 'learning_rate': '0.0001177', 'entropy': '1.343', 'num_tokens': '1.919e+06', 'mean_token_accuracy': '0.6867', 'epoch': '0.003133'}
{'loss': '1.205', 'grad_norm': '0.5841', 'learning_rate': '0.000118', 'entropy': '1.264', 'num_tokens': '1.923e+06', 'mean_token_accuracy': '0.7017', 'epoch': '0.003142'}
{'loss': '0.9092', 'grad_norm': '0.4383', 'learning_rate': '0.0001183', 'entropy': '0.9224', 'num_tokens': '1.929e+06', 'mean_token_accuracy': '0.7706', 'epoch': '0.003151'}
{'loss': '0.7237', 'grad_norm': '0.4778', 'learning_rate': '0.0001187', 'entropy': '0.7834', 'num_tokens': '1.934e+06', 'mean_token_accuracy': '0.7982', 'epoch': '0.003159'}
{'loss': '1.011', 'grad_norm': '0.3649', 'learning_rate': '0.000119', 'entropy': '1.009', 'num_tokens': '1.94e+06', 'mean_token_accuracy': '0.755', 'epoch': '0.003168'}
{'loss': '1.002', 'grad_norm': '0.66', 'learning_rate': '0.0001193', 'entropy': '1.014', 'num_tokens': '1.943e+06', 'mean_token_accuracy': '0.7501', 'epoch': '0.003177'}
{'loss': '1.386', 'grad_norm': '0.5979', 'learning_rate': '0.0001197', 'entropy': '1.293', 'num_tokens': '1.947e+06', 'mean_token_accuracy': '0.6957', 'epoch': '0.003186'}
{'loss': '0.9768', 'grad_norm': '0.4426', 'learning_rate': '0.00012', 'entropy': '0.9649', 'num_tokens': '1.953e+06', 'mean_token_accuracy': '0.7642', 'epoch': '0.003195'}
{'loss': '1.258', 'grad_norm': '0.4513', 'learning_rate': '0.0001203', 'entropy': '1.141', 'num_tokens': '1.958e+06', 'mean_token_accuracy': '0.7344', 'epoch': '0.003204'}
{'loss': '1.183', 'grad_norm': '0.5182', 'learning_rate': '0.0001207', 'entropy': '1.295', 'num_tokens': '1.962e+06', 'mean_token_accuracy': '0.6945', 'epoch': '0.003212'}
{'loss': '1.083', 'grad_norm': '0.443', 'learning_rate': '0.000121', 'entropy': '1.007', 'num_tokens': '1.967e+06', 'mean_token_accuracy': '0.765', 'epoch': '0.003221'}
{'loss': '1.025', 'grad_norm': '0.3785', 'learning_rate': '0.0001213', 'entropy': '1.002', 'num_tokens': '1.975e+06', 'mean_token_accuracy': '0.7678', 'epoch': '0.00323'}
{'loss': '1.295', 'grad_norm': '0.4739', 'learning_rate': '0.0001217', 'entropy': '1.228', 'num_tokens': '1.98e+06', 'mean_token_accuracy': '0.723', 'epoch': '0.003239'}
{'loss': '0.9326', 'grad_norm': '0.4213', 'learning_rate': '0.000122', 'entropy': '1.022', 'num_tokens': '1.986e+06', 'mean_token_accuracy': '0.7609', 'epoch': '0.003248'}
{'loss': '1.068', 'grad_norm': '0.5664', 'learning_rate': '0.0001223', 'entropy': '1.172', 'num_tokens': '1.99e+06', 'mean_token_accuracy': '0.7298', 'epoch': '0.003257'}
{'loss': '0.9402', 'grad_norm': '0.4347', 'learning_rate': '0.0001227', 'entropy': '0.9193', 'num_tokens': '1.997e+06', 'mean_token_accuracy': '0.78', 'epoch': '0.003266'}
{'loss': '1.249', 'grad_norm': '0.3904', 'learning_rate': '0.000123', 'entropy': '1.138', 'num_tokens': '2.002e+06', 'mean_token_accuracy': '0.7374', 'epoch': '0.003274'}
{'loss': '0.7814', 'grad_norm': '0.5179', 'learning_rate': '0.0001233', 'entropy': '0.823', 'num_tokens': '2.007e+06', 'mean_token_accuracy': '0.789', 'epoch': '0.003283'}
{'loss': '1.025', 'grad_norm': '0.4503', 'learning_rate': '0.0001237', 'entropy': '1.015', 'num_tokens': '2.012e+06', 'mean_token_accuracy': '0.7602', 'epoch': '0.003292'}
{'loss': '0.9472', 'grad_norm': '0.371', 'learning_rate': '0.000124', 'entropy': '0.9285', 'num_tokens': '2.017e+06', 'mean_token_accuracy': '0.7807', 'epoch': '0.003301'}
{'loss': '0.9238', 'grad_norm': '0.4583', 'learning_rate': '0.0001243', 'entropy': '0.9436', 'num_tokens': '2.022e+06', 'mean_token_accuracy': '0.7729', 'epoch': '0.00331'}
{'loss': '1.211', 'grad_norm': '0.4467', 'learning_rate': '0.0001247', 'entropy': '1.18', 'num_tokens': '2.027e+06', 'mean_token_accuracy': '0.7218', 'epoch': '0.003319'}
{'loss': '0.8753', 'grad_norm': '0.5251', 'learning_rate': '0.000125', 'entropy': '0.9093', 'num_tokens': '2.031e+06', 'mean_token_accuracy': '0.7724', 'epoch': '0.003328'}
{'loss': '0.8353', 'grad_norm': '0.4524', 'learning_rate': '0.0001253', 'entropy': '0.862', 'num_tokens': '2.037e+06', 'mean_token_accuracy': '0.7841', 'epoch': '0.003336'}

  3%|▎         | 378/12000 [13:37<6:14:14,  1.93s/it]
  3%|▎         | 379/12000 [13:38<5:54:14,  1.83s/it]
                                                     

  3%|▎         | 379/12000 [13:38<5:54:14,  1.83s/it]
  3%|▎         | 380/12000 [13:40<5:48:27,  1.80s/it]
                                                     

  3%|▎         | 380/12000 [13:40<5:48:27,  1.80s/it]
  3%|▎         | 381/12000 [13:41<5:22:29,  1.67s/it]
                                                     

  3%|▎         | 381/12000 [13:41<5:22:29,  1.67s/it]
  3%|▎         | 382/12000 [13:43<5:31:41,  1.71s/it]
                                                     

  3%|▎         | 382/12000 [13:43<5:31:41,  1.71s/it]
  3%|▎         | 383/12000 [13:46<6:03:58,  1.88s/it]
                                                     

  3%|▎         | 383/12000 [13:46<6:03:58,  1.88s/it]
  3%|▎         | 384/12000 [13:47<5:59:27,  1.86s/it]
                                                     

  3%|▎         | 384/12000 [13:47<5:59:27,  1.86s/it]
  3%|▎         | 385/12000 [13:51<7:55:47,  2.46s/it]
                                                     

  3%|▎         | 385/12000 [13:51<7:55:47,  2.46s/it]
  3%|▎         | 386/12000 [13:55<8:50:43,  2.74s/it]
                                                     

  3%|▎         | 386/12000 [13:55<8:50:43,  2.74s/it]
  3%|▎         | 387/12000 [13:56<7:47:42,  2.42s/it]
                                                     

  3%|▎         | 387/12000 [13:56<7:47:42,  2.42s/it]
  3%|▎         | 388/12000 [13:58<6:58:04,  2.16s/it]
                                                     

  3%|▎         | 388/12000 [13:58<6:58:04,  2.16s/it]
  3%|▎         | 389/12000 [14:00<6:38:25,  2.06s/it]
                                                     

  3%|▎         | 389/12000 [14:00<6:38:25,  2.06s/it]
  3%|▎         | 390/12000 [14:01<6:15:50,  1.94s/it]
                                                     

  3%|▎         | 390/12000 [14:01<6:15:50,  1.94s/it]
  3%|▎         | 391/12000 [14:03<5:58:09,  1.85s/it]
                                                     

  3%|▎         | 391/12000 [14:03<5:58:09,  1.85s/it]
  3%|▎         | 392/12000 [14:05<6:17:23,  1.95s/it]
                                                     

  3%|▎         | 392/12000 [14:05<6:17:23,  1.95s/it]
  3%|▎         | 393/12000 [14:08<7:12:19,  2.23s/it]
                                                     

  3%|▎         | 393/12000 [14:08<7:12:19,  2.23s/it]
  3%|▎         | 394/12000 [14:10<6:35:01,  2.04s/it]
                                                     

  3%|▎         | 394/12000 [14:10<6:35:01,  2.04s/it]
  3%|▎         | 395/12000 [14:11<6:20:28,  1.97s/it]
                                                     

  3%|▎         | 395/12000 [14:11<6:20:28,  1.97s/it]
  3%|▎         | 396/12000 [14:16<8:42:55,  2.70s/it]
                                                     

  3%|▎         | 396/12000 [14:16<8:42:55,  2.70s/it]
  3%|▎         | 397/12000 [14:19<8:59:58,  2.79s/it]
                                                     

  3%|▎         | 397/12000 [14:19<8:59:58,  2.79s/it]
  3%|▎         | 398/12000 [14:22<8:57:26,  2.78s/it]
                                                     

  3%|▎         | 398/12000 [14:22<8:57:26,  2.78s/it]
  3%|▎         | 399/12000 [14:23<7:39:47,  2.38s/it]
                                                     

  3%|▎         | 399/12000 [14:23<7:39:47,  2.38s/it]
  3%|▎         | 400/12000 [14:25<6:59:53,  2.17s/it]
                                                     

  3%|▎         | 400/12000 [14:25<6:59:53,  2.17s/it]
  3%|▎         | 401/12000 [14:29<8:45:05,  2.72s/it]
                                                     

  3%|▎         | 401/12000 [14:29<8:45:05,  2.72s/it]
  3%|▎         | 402/12000 [14:30<7:44:51,  2.40s/it]
                                                     

  3%|▎         | 402/12000 [14:30<7:44:51,  2.40s/it]
  3%|▎         | 403/12000 [14:32<6:53:38,  2.14s/it]
                                                     

  3%|▎         | 403/12000 [14:32<6:53:38,  2.14s/it]
  3%|▎         | 404/12000 [14:34<6:44:29,  2.09s/it]
                                                     

  3%|▎         | 404/12000 [14:34<6:44:29,  2.09s/it]
  3%|▎         | 405/12000 [14:37<7:25:24,  2.30s/it]
                                                     

  3%|▎         | 405/12000 [14:37<7:25:24,  2.30s/it]
  3%|▎         | 406/12000 [14:38<6:34:03,  2.04s/it]
                                                     

  3%|▎         | 406/12000 [14:38<6:34:03,  2.04s/it]
  3%|▎         | 407/12000 [14:40<6:11:59,  1.93s/it]
                                                     

  3%|▎         | 407/12000 [14:40<6:11:59,  1.93s/it]
  3%|▎         | 408/12000 [14:42<6:43:33,  2.09s/it]
                                                     

  3%|▎         | 408/12000 [14:42<6:43:33,  2.09s/it]
  3%|▎         | 409/12000 [14:44<6:44:26,  2.09s/it]
                                                     

  3%|▎         | 409/12000 [14:44<6:44:26,  2.09s/it]
  3%|▎         | 410/12000 [14:46<6:14:45,  1.94s/it]
                                                     

  3%|▎         | 410/12000 [14:46<6:14:45,  1.94s/it]
  3%|▎         | 411/12000 [14:48<6:47:12,  2.11s/it]
                                                     

  3%|▎         | 411/12000 [14:48<6:47:12,  2.11s/it]
  3%|▎         | 412/12000 [14:50<6:14:12,  1.94s/it]
                                                     

  3%|▎         | 412/12000 [14:50<6:14:12,  1.94s/it]
  3%|▎         | 413/12000 [14:51<5:49:49,  1.81s/it]
                                                     

  3%|▎         | 413/12000 [14:51<5:49:49,  1.81s/it]
  3%|▎         | 414/12000 [14:54<6:03:26,  1.88s/it]
                                                     

  3%|▎         | 414/12000 [14:54<6:03:26,  1.88s/it]
  3%|▎         | 415/12000 [14:55<5:59:20,  1.86s/it]
                                                     

  3%|▎         | 415/12000 [14:55<5:59:20,  1.86s/it]
  3%|▎         | 416/12000 [14:57<5:42:31,  1.77s/it]
                                                     

  3%|▎         | 416/12000 [14:57<5:42:31,  1.77s/it]
  3%|▎         | 417/12000 [15:00<6:41:12,  2.08s/it]
                                                     

  3%|▎         | 417/12000 [15:00<6:41:12,  2.08s/it]
  3%|▎         | 418/12000 [15:03<8:19:35,  2.59s/it]
                                                     

  3%|▎         | 418/12000 [15:03<8:19:35,  2.59s/it]
  3%|▎         | 419/12000 [15:05<7:25:54,  2.31s/it]
                                                     

  3%|▎         | 419/12000 [15:05<7:25:54,  2.31s/it]
  4%|▎         | 420/12000 [15:07<7:11:17,  2.23s/it]
                                                     

  4%|▎         | 420/12000 [15:07<7:11:17,  2.23s/it]
  4%|▎         | 421/12000 [15:09<6:53:57,  2.15s/it]
                                                     

  4%|▎         | 421/12000 [15:09<6:53:57,  2.15s/it]
  4%|▎         | 422/12000 [15:12<7:23:22,  2.30s/it]
                                                     

  4%|▎         | 422/12000 [15:12<7:23:22,  2.30s/it]
  4%|▎         | 423/12000 [15:14<6:59:52,  2.18s/it]
                                                     

  4%|▎         | 423/12000 [15:14<6:59:52,  2.18s/it]
  4%|▎         | 424/12000 [15:15<6:31:07,  2.03s/it]
                                                     

  4%|▎         | 424/12000 [15:15<6:31:07,  2.03s/it]
  4%|▎         | 425/12000 [15:17<6:20:19,  1.97s/it]
                                                     
{'loss': '0.9544', 'grad_norm': '0.442', 'learning_rate': '0.0001257', 'entropy': '0.9339', 'num_tokens': '2.043e+06', 'mean_token_accuracy': '0.767', 'epoch': '0.003345'}
{'loss': '0.8376', 'grad_norm': '0.4765', 'learning_rate': '0.000126', 'entropy': '0.8469', 'num_tokens': '2.048e+06', 'mean_token_accuracy': '0.7848', 'epoch': '0.003354'}
{'loss': '1.157', 'grad_norm': '0.5426', 'learning_rate': '0.0001263', 'entropy': '1.297', 'num_tokens': '2.053e+06', 'mean_token_accuracy': '0.6892', 'epoch': '0.003363'}
{'loss': '0.9618', 'grad_norm': '0.7097', 'learning_rate': '0.0001267', 'entropy': '1.051', 'num_tokens': '2.055e+06', 'mean_token_accuracy': '0.7483', 'epoch': '0.003372'}
{'loss': '1.099', 'grad_norm': '0.5252', 'learning_rate': '0.000127', 'entropy': '1.116', 'num_tokens': '2.061e+06', 'mean_token_accuracy': '0.7301', 'epoch': '0.003381'}
{'loss': '1.23', 'grad_norm': '0.4587', 'learning_rate': '0.0001273', 'entropy': '1.232', 'num_tokens': '2.068e+06', 'mean_token_accuracy': '0.7074', 'epoch': '0.003389'}
{'loss': '0.9044', 'grad_norm': '0.3971', 'learning_rate': '0.0001277', 'entropy': '0.9197', 'num_tokens': '2.073e+06', 'mean_token_accuracy': '0.786', 'epoch': '0.003398'}
{'loss': '0.9797', 'grad_norm': '0.355', 'learning_rate': '0.000128', 'entropy': '0.9272', 'num_tokens': '2.082e+06', 'mean_token_accuracy': '0.7702', 'epoch': '0.003407'}
{'loss': '1.307', 'grad_norm': '0.406', 'learning_rate': '0.0001283', 'entropy': '1.219', 'num_tokens': '2.089e+06', 'mean_token_accuracy': '0.7193', 'epoch': '0.003416'}
{'loss': '0.8559', 'grad_norm': '0.4753', 'learning_rate': '0.0001287', 'entropy': '0.8874', 'num_tokens': '2.094e+06', 'mean_token_accuracy': '0.7704', 'epoch': '0.003425'}
{'loss': '0.8689', 'grad_norm': '0.4084', 'learning_rate': '0.000129', 'entropy': '0.9528', 'num_tokens': '2.099e+06', 'mean_token_accuracy': '0.7705', 'epoch': '0.003434'}
{'loss': '1.103', 'grad_norm': '0.4465', 'learning_rate': '0.0001293', 'entropy': '1.026', 'num_tokens': '2.104e+06', 'mean_token_accuracy': '0.7318', 'epoch': '0.003443'}
{'loss': '0.9181', 'grad_norm': '0.446', 'learning_rate': '0.0001297', 'entropy': '0.9296', 'num_tokens': '2.11e+06', 'mean_token_accuracy': '0.7714', 'epoch': '0.003451'}
{'loss': '1.284', 'grad_norm': '0.4242', 'learning_rate': '0.00013', 'entropy': '1.425', 'num_tokens': '2.116e+06', 'mean_token_accuracy': '0.6873', 'epoch': '0.00346'}
{'loss': '1.155', 'grad_norm': '0.5083', 'learning_rate': '0.0001303', 'entropy': '1.132', 'num_tokens': '2.121e+06', 'mean_token_accuracy': '0.7381', 'epoch': '0.003469'}
{'loss': '1.063', 'grad_norm': '0.3766', 'learning_rate': '0.0001307', 'entropy': '1.073', 'num_tokens': '2.129e+06', 'mean_token_accuracy': '0.7486', 'epoch': '0.003478'}
{'loss': '1.073', 'grad_norm': '0.4773', 'learning_rate': '0.000131', 'entropy': '1.129', 'num_tokens': '2.133e+06', 'mean_token_accuracy': '0.7392', 'epoch': '0.003487'}
{'loss': '0.9577', 'grad_norm': '0.4176', 'learning_rate': '0.0001313', 'entropy': '0.9042', 'num_tokens': '2.139e+06', 'mean_token_accuracy': '0.7672', 'epoch': '0.003496'}
{'loss': '1.19', 'grad_norm': '0.4007', 'learning_rate': '0.0001317', 'entropy': '1.245', 'num_tokens': '2.146e+06', 'mean_token_accuracy': '0.7156', 'epoch': '0.003505'}
{'loss': '0.8', 'grad_norm': '0.3763', 'learning_rate': '0.000132', 'entropy': '0.9445', 'num_tokens': '2.153e+06', 'mean_token_accuracy': '0.7608', 'epoch': '0.003513'}
{'loss': '1.193', 'grad_norm': '0.4673', 'learning_rate': '0.0001323', 'entropy': '1.276', 'num_tokens': '2.159e+06', 'mean_token_accuracy': '0.6993', 'epoch': '0.003522'}
{'loss': '1.181', 'grad_norm': '0.4837', 'learning_rate': '0.0001327', 'entropy': '1.205', 'num_tokens': '2.163e+06', 'mean_token_accuracy': '0.7339', 'epoch': '0.003531'}
{'loss': '0.9178', 'grad_norm': '0.491', 'learning_rate': '0.000133', 'entropy': '0.9138', 'num_tokens': '2.168e+06', 'mean_token_accuracy': '0.7835', 'epoch': '0.00354'}
{'loss': '0.6601', 'grad_norm': '0.4998', 'learning_rate': '0.0001333', 'entropy': '0.6483', 'num_tokens': '2.174e+06', 'mean_token_accuracy': '0.8371', 'epoch': '0.003549'}
{'loss': '0.8588', 'grad_norm': '0.3667', 'learning_rate': '0.0001337', 'entropy': '0.865', 'num_tokens': '2.179e+06', 'mean_token_accuracy': '0.798', 'epoch': '0.003558'}
{'loss': '0.8942', 'grad_norm': '0.3636', 'learning_rate': '0.000134', 'entropy': '0.9064', 'num_tokens': '2.183e+06', 'mean_token_accuracy': '0.7735', 'epoch': '0.003566'}
{'loss': '0.7509', 'grad_norm': '1.139', 'learning_rate': '0.0001343', 'entropy': '0.8206', 'num_tokens': '2.19e+06', 'mean_token_accuracy': '0.7987', 'epoch': '0.003575'}
{'loss': '0.88', 'grad_norm': '0.3461', 'learning_rate': '0.0001347', 'entropy': '1.148', 'num_tokens': '2.196e+06', 'mean_token_accuracy': '0.7266', 'epoch': '0.003584'}
{'loss': '1.243', 'grad_norm': '0.4554', 'learning_rate': '0.000135', 'entropy': '1.315', 'num_tokens': '2.2e+06', 'mean_token_accuracy': '0.7184', 'epoch': '0.003593'}
{'loss': '0.8815', 'grad_norm': '0.4713', 'learning_rate': '0.0001353', 'entropy': '0.9012', 'num_tokens': '2.205e+06', 'mean_token_accuracy': '0.7739', 'epoch': '0.003602'}
{'loss': '1.12', 'grad_norm': '0.4557', 'learning_rate': '0.0001357', 'entropy': '1.079', 'num_tokens': '2.211e+06', 'mean_token_accuracy': '0.7507', 'epoch': '0.003611'}
{'loss': '1.077', 'grad_norm': '0.4462', 'learning_rate': '0.000136', 'entropy': '1.064', 'num_tokens': '2.216e+06', 'mean_token_accuracy': '0.7562', 'epoch': '0.00362'}
{'loss': '0.9153', 'grad_norm': '0.5529', 'learning_rate': '0.0001363', 'entropy': '0.9247', 'num_tokens': '2.219e+06', 'mean_token_accuracy': '0.7659', 'epoch': '0.003628'}
{'loss': '1.137', 'grad_norm': '0.4691', 'learning_rate': '0.0001367', 'entropy': '1.097', 'num_tokens': '2.225e+06', 'mean_token_accuracy': '0.7435', 'epoch': '0.003637'}
{'loss': '1.023', 'grad_norm': '0.6222', 'learning_rate': '0.000137', 'entropy': '0.9907', 'num_tokens': '2.228e+06', 'mean_token_accuracy': '0.7494', 'epoch': '0.003646'}
{'loss': '0.8555', 'grad_norm': '0.4755', 'learning_rate': '0.0001373', 'entropy': '0.8915', 'num_tokens': '2.232e+06', 'mean_token_accuracy': '0.7781', 'epoch': '0.003655'}
{'loss': '0.9386', 'grad_norm': '0.4535', 'learning_rate': '0.0001377', 'entropy': '0.9343', 'num_tokens': '2.238e+06', 'mean_token_accuracy': '0.7602', 'epoch': '0.003664'}
{'loss': '1.449', 'grad_norm': '0.4506', 'learning_rate': '0.000138', 'entropy': '1.341', 'num_tokens': '2.245e+06', 'mean_token_accuracy': '0.6688', 'epoch': '0.003673'}
{'loss': '1.104', 'grad_norm': '0.4896', 'learning_rate': '0.0001383', 'entropy': '1.147', 'num_tokens': '2.249e+06', 'mean_token_accuracy': '0.7105', 'epoch': '0.003682'}
{'loss': '1.007', 'grad_norm': '0.4453', 'learning_rate': '0.0001387', 'entropy': '0.9903', 'num_tokens': '2.255e+06', 'mean_token_accuracy': '0.7584', 'epoch': '0.00369'}
{'loss': '1.082', 'grad_norm': '0.4112', 'learning_rate': '0.000139', 'entropy': '1.061', 'num_tokens': '2.261e+06', 'mean_token_accuracy': '0.7497', 'epoch': '0.003699'}
{'loss': '0.7809', 'grad_norm': '0.3369', 'learning_rate': '0.0001393', 'entropy': '0.7922', 'num_tokens': '2.267e+06', 'mean_token_accuracy': '0.8096', 'epoch': '0.003708'}
{'loss': '1.162', 'grad_norm': '0.3826', 'learning_rate': '0.0001397', 'entropy': '1.181', 'num_tokens': '2.273e+06', 'mean_token_accuracy': '0.7342', 'epoch': '0.003717'}
{'loss': '0.8449', 'grad_norm': '0.4527', 'learning_rate': '0.00014', 'entropy': '0.9494', 'num_tokens': '2.279e+06', 'mean_token_accuracy': '0.7584', 'epoch': '0.003726'}
{'loss': '0.8185', 'grad_norm': '0.3873', 'learning_rate': '0.0001403', 'entropy': '0.8818', 'num_tokens': '2.286e+06', 'mean_token_accuracy': '0.8055', 'epoch': '0.003735'}
{'loss': '1.275', 'grad_norm': '0.4997', 'learning_rate': '0.0001407', 'entropy': '1.241', 'num_tokens': '2.291e+06', 'mean_token_accuracy': '0.7181', 'epoch': '0.003743'}
{'loss': '1.092', 'grad_norm': '0.3843', 'learning_rate': '0.000141', 'entropy': '1.019', 'num_tokens': '2.296e+06', 'mean_token_accuracy': '0.7507', 'epoch': '0.003752'}

  4%|▎         | 425/12000 [15:17<6:20:19,  1.97s/it]
  4%|▎         | 426/12000 [15:19<6:07:30,  1.91s/it]
                                                     

  4%|▎         | 426/12000 [15:19<6:07:30,  1.91s/it]
  4%|▎         | 427/12000 [15:22<6:49:39,  2.12s/it]
                                                     

  4%|▎         | 427/12000 [15:22<6:49:39,  2.12s/it]
  4%|▎         | 428/12000 [15:24<6:51:40,  2.13s/it]
                                                     

  4%|▎         | 428/12000 [15:24<6:51:40,  2.13s/it]
  4%|▎         | 429/12000 [15:25<6:12:25,  1.93s/it]
                                                     

  4%|▎         | 429/12000 [15:25<6:12:25,  1.93s/it]
  4%|▎         | 430/12000 [15:27<6:04:31,  1.89s/it]
                                                     

  4%|▎         | 430/12000 [15:27<6:04:31,  1.89s/it]
  4%|▎         | 431/12000 [15:29<5:45:00,  1.79s/it]
                                                     

  4%|▎         | 431/12000 [15:29<5:45:00,  1.79s/it]
  4%|▎         | 432/12000 [15:30<5:43:52,  1.78s/it]
                                                     

  4%|▎         | 432/12000 [15:30<5:43:52,  1.78s/it]
  4%|▎         | 433/12000 [15:32<5:57:05,  1.85s/it]
                                                     

  4%|▎         | 433/12000 [15:32<5:57:05,  1.85s/it]
  4%|▎         | 434/12000 [15:35<6:49:50,  2.13s/it]
                                                     

  4%|▎         | 434/12000 [15:35<6:49:50,  2.13s/it]
  4%|▎         | 435/12000 [15:37<6:57:14,  2.16s/it]
                                                     

  4%|▎         | 435/12000 [15:37<6:57:14,  2.16s/it]
  4%|▎         | 436/12000 [15:43<10:23:05,  3.23s/it]
                                                      

  4%|▎         | 436/12000 [15:43<10:23:05,  3.23s/it]
  4%|▎         | 437/12000 [15:45<8:40:34,  2.70s/it] 
                                                     

  4%|▎         | 437/12000 [15:45<8:40:34,  2.70s/it]
  4%|▎         | 438/12000 [15:46<7:34:26,  2.36s/it]
                                                     

  4%|▎         | 438/12000 [15:46<7:34:26,  2.36s/it]
  4%|▎         | 439/12000 [15:48<7:04:08,  2.20s/it]
                                                     

  4%|▎         | 439/12000 [15:48<7:04:08,  2.20s/it]
  4%|▎         | 440/12000 [15:52<8:45:41,  2.73s/it]
                                                     

  4%|▎         | 440/12000 [15:52<8:45:41,  2.73s/it]
  4%|▎         | 441/12000 [15:53<7:25:03,  2.31s/it]
                                                     

  4%|▎         | 441/12000 [15:53<7:25:03,  2.31s/it]
  4%|▎         | 442/12000 [15:56<7:30:28,  2.34s/it]
                                                     

  4%|▎         | 442/12000 [15:56<7:30:28,  2.34s/it]
  4%|▎         | 443/12000 [15:57<6:43:51,  2.10s/it]
                                                     

  4%|▎         | 443/12000 [15:57<6:43:51,  2.10s/it]
  4%|▎         | 444/12000 [15:59<6:43:19,  2.09s/it]
                                                     

  4%|▎         | 444/12000 [15:59<6:43:19,  2.09s/it]
  4%|▎         | 445/12000 [16:01<6:02:07,  1.88s/it]
                                                     

  4%|▎         | 445/12000 [16:01<6:02:07,  1.88s/it]
  4%|▎         | 446/12000 [16:02<5:59:52,  1.87s/it]
                                                     

  4%|▎         | 446/12000 [16:02<5:59:52,  1.87s/it]
  4%|▎         | 447/12000 [16:04<5:50:38,  1.82s/it]
                                                     

  4%|▎         | 447/12000 [16:04<5:50:38,  1.82s/it]
  4%|▎         | 448/12000 [16:06<5:27:22,  1.70s/it]
                                                     

  4%|▎         | 448/12000 [16:06<5:27:22,  1.70s/it]
  4%|▎         | 449/12000 [16:07<5:23:15,  1.68s/it]
                                                     

  4%|▎         | 449/12000 [16:07<5:23:15,  1.68s/it]
  4%|▍         | 450/12000 [16:09<5:48:03,  1.81s/it]
                                                     

  4%|▍         | 450/12000 [16:09<5:48:03,  1.81s/it]
  4%|▍         | 451/12000 [16:15<9:47:39,  3.05s/it]
                                                     

  4%|▍         | 451/12000 [16:15<9:47:39,  3.05s/it]
  4%|▍         | 452/12000 [16:18<9:52:40,  3.08s/it]
                                                     

  4%|▍         | 452/12000 [16:18<9:52:40,  3.08s/it]
  4%|▍         | 453/12000 [16:20<8:33:54,  2.67s/it]
                                                     

  4%|▍         | 453/12000 [16:20<8:33:54,  2.67s/it]
  4%|▍         | 454/12000 [16:22<8:05:00,  2.52s/it]
                                                     

  4%|▍         | 454/12000 [16:22<8:05:00,  2.52s/it]
  4%|▍         | 455/12000 [16:24<7:10:02,  2.23s/it]
                                                     

  4%|▍         | 455/12000 [16:24<7:10:02,  2.23s/it]
  4%|▍         | 456/12000 [16:25<6:31:10,  2.03s/it]
                                                     

  4%|▍         | 456/12000 [16:25<6:31:10,  2.03s/it]
  4%|▍         | 457/12000 [16:28<7:07:50,  2.22s/it]
                                                     

  4%|▍         | 457/12000 [16:28<7:07:50,  2.22s/it]
  4%|▍         | 458/12000 [16:35<11:14:29,  3.51s/it]
                                                      

  4%|▍         | 458/12000 [16:35<11:14:29,  3.51s/it]
  4%|▍         | 459/12000 [16:36<9:19:46,  2.91s/it] 
                                                     

  4%|▍         | 459/12000 [16:36<9:19:46,  2.91s/it]
  4%|▍         | 460/12000 [16:39<9:09:42,  2.86s/it]
                                                     

  4%|▍         | 460/12000 [16:39<9:09:42,  2.86s/it]
  4%|▍         | 461/12000 [16:40<7:58:41,  2.49s/it]
                                                     

  4%|▍         | 461/12000 [16:40<7:58:41,  2.49s/it]
  4%|▍         | 462/12000 [16:44<8:50:40,  2.76s/it]
                                                     

  4%|▍         | 462/12000 [16:44<8:50:40,  2.76s/it]
  4%|▍         | 463/12000 [16:45<7:29:21,  2.34s/it]
                                                     

  4%|▍         | 463/12000 [16:45<7:29:21,  2.34s/it]
  4%|▍         | 464/12000 [16:47<6:43:23,  2.10s/it]
                                                     

  4%|▍         | 464/12000 [16:47<6:43:23,  2.10s/it]
  4%|▍         | 465/12000 [16:49<6:29:51,  2.03s/it]
                                                     

  4%|▍         | 465/12000 [16:49<6:29:51,  2.03s/it]
  4%|▍         | 466/12000 [16:50<6:06:21,  1.91s/it]
                                                     

  4%|▍         | 466/12000 [16:50<6:06:21,  1.91s/it]
  4%|▍         | 467/12000 [16:54<7:56:56,  2.48s/it]
                                                     

  4%|▍         | 467/12000 [16:54<7:56:56,  2.48s/it]
  4%|▍         | 468/12000 [16:56<7:23:02,  2.31s/it]
                                                     

  4%|▍         | 468/12000 [16:56<7:23:02,  2.31s/it]
  4%|▍         | 469/12000 [16:58<6:45:00,  2.11s/it]
                                                     

  4%|▍         | 469/12000 [16:58<6:45:00,  2.11s/it]
  4%|▍         | 470/12000 [16:59<6:08:07,  1.92s/it]
                                                     

  4%|▍         | 470/12000 [16:59<6:08:07,  1.92s/it]
  4%|▍         | 471/12000 [17:02<6:56:13,  2.17s/it]
                                                     

  4%|▍         | 471/12000 [17:02<6:56:13,  2.17s/it]
  4%|▍         | 472/12000 [17:03<6:14:06,  1.95s/it]
                                                     
{'loss': '0.9861', 'grad_norm': '0.4105', 'learning_rate': '0.0001413', 'entropy': '0.9165', 'num_tokens': '2.301e+06', 'mean_token_accuracy': '0.763', 'epoch': '0.003761'}
{'loss': '1.402', 'grad_norm': '0.4353', 'learning_rate': '0.0001417', 'entropy': '1.362', 'num_tokens': '2.307e+06', 'mean_token_accuracy': '0.6733', 'epoch': '0.00377'}
{'loss': '0.7982', 'grad_norm': '0.3795', 'learning_rate': '0.000142', 'entropy': '0.7337', 'num_tokens': '2.315e+06', 'mean_token_accuracy': '0.8223', 'epoch': '0.003779'}
{'loss': '1.478', 'grad_norm': '0.5045', 'learning_rate': '0.0001423', 'entropy': '1.477', 'num_tokens': '2.32e+06', 'mean_token_accuracy': '0.6719', 'epoch': '0.003788'}
{'loss': '1.153', 'grad_norm': '0.5188', 'learning_rate': '0.0001427', 'entropy': '1.167', 'num_tokens': '2.323e+06', 'mean_token_accuracy': '0.7159', 'epoch': '0.003797'}
{'loss': '0.7294', 'grad_norm': '0.4508', 'learning_rate': '0.000143', 'entropy': '0.7868', 'num_tokens': '2.329e+06', 'mean_token_accuracy': '0.808', 'epoch': '0.003805'}
{'loss': '1.148', 'grad_norm': '0.4864', 'learning_rate': '0.0001433', 'entropy': '1.145', 'num_tokens': '2.333e+06', 'mean_token_accuracy': '0.7168', 'epoch': '0.003814'}
{'loss': '1.033', 'grad_norm': '0.3958', 'learning_rate': '0.0001437', 'entropy': '1.143', 'num_tokens': '2.339e+06', 'mean_token_accuracy': '0.7274', 'epoch': '0.003823'}
{'loss': '1.181', 'grad_norm': '0.5246', 'learning_rate': '0.000144', 'entropy': '1.166', 'num_tokens': '2.344e+06', 'mean_token_accuracy': '0.7205', 'epoch': '0.003832'}
{'loss': '1.094', 'grad_norm': '3.277', 'learning_rate': '0.0001443', 'entropy': '1.223', 'num_tokens': '2.351e+06', 'mean_token_accuracy': '0.7108', 'epoch': '0.003841'}
{'loss': '0.8868', 'grad_norm': '0.3757', 'learning_rate': '0.0001447', 'entropy': '0.9015', 'num_tokens': '2.356e+06', 'mean_token_accuracy': '0.7808', 'epoch': '0.00385'}
{'loss': '1.188', 'grad_norm': '0.3238', 'learning_rate': '0.000145', 'entropy': '1.037', 'num_tokens': '2.366e+06', 'mean_token_accuracy': '0.7687', 'epoch': '0.003859'}
{'loss': '0.8764', 'grad_norm': '0.3775', 'learning_rate': '0.0001453', 'entropy': '0.9092', 'num_tokens': '2.37e+06', 'mean_token_accuracy': '0.774', 'epoch': '0.003867'}
{'loss': '0.9369', 'grad_norm': '0.421', 'learning_rate': '0.0001457', 'entropy': '0.9111', 'num_tokens': '2.375e+06', 'mean_token_accuracy': '0.7757', 'epoch': '0.003876'}
{'loss': '0.7736', 'grad_norm': '0.4955', 'learning_rate': '0.000146', 'entropy': '0.8393', 'num_tokens': '2.38e+06', 'mean_token_accuracy': '0.791', 'epoch': '0.003885'}
{'loss': '1.161', 'grad_norm': '0.4166', 'learning_rate': '0.0001463', 'entropy': '1.246', 'num_tokens': '2.386e+06', 'mean_token_accuracy': '0.7104', 'epoch': '0.003894'}
{'loss': '1.164', 'grad_norm': '0.6098', 'learning_rate': '0.0001467', 'entropy': '1.191', 'num_tokens': '2.389e+06', 'mean_token_accuracy': '0.7076', 'epoch': '0.003903'}
{'loss': '1.201', 'grad_norm': '0.474', 'learning_rate': '0.000147', 'entropy': '1.15', 'num_tokens': '2.397e+06', 'mean_token_accuracy': '0.7108', 'epoch': '0.003912'}
{'loss': '0.9513', 'grad_norm': '0.4726', 'learning_rate': '0.0001473', 'entropy': '0.957', 'num_tokens': '2.401e+06', 'mean_token_accuracy': '0.7652', 'epoch': '0.00392'}
{'loss': '0.7728', 'grad_norm': '0.4272', 'learning_rate': '0.0001477', 'entropy': '0.7531', 'num_tokens': '2.407e+06', 'mean_token_accuracy': '0.8015', 'epoch': '0.003929'}
{'loss': '1.013', 'grad_norm': '0.4317', 'learning_rate': '0.000148', 'entropy': '1.074', 'num_tokens': '2.411e+06', 'mean_token_accuracy': '0.742', 'epoch': '0.003938'}
{'loss': '1.084', 'grad_norm': '0.4671', 'learning_rate': '0.0001483', 'entropy': '1.056', 'num_tokens': '2.416e+06', 'mean_token_accuracy': '0.748', 'epoch': '0.003947'}
{'loss': '1.308', 'grad_norm': '0.4026', 'learning_rate': '0.0001487', 'entropy': '1.296', 'num_tokens': '2.422e+06', 'mean_token_accuracy': '0.7003', 'epoch': '0.003956'}
{'loss': '0.9079', 'grad_norm': '0.4359', 'learning_rate': '0.000149', 'entropy': '0.9689', 'num_tokens': '2.426e+06', 'mean_token_accuracy': '0.7676', 'epoch': '0.003965'}
{'loss': '1.282', 'grad_norm': '0.4452', 'learning_rate': '0.0001493', 'entropy': '1.309', 'num_tokens': '2.43e+06', 'mean_token_accuracy': '0.6992', 'epoch': '0.003974'}
{'loss': '0.7994', 'grad_norm': '0.4021', 'learning_rate': '0.0001497', 'entropy': '0.8285', 'num_tokens': '2.435e+06', 'mean_token_accuracy': '0.7961', 'epoch': '0.003982'}
{'loss': '1.295', 'grad_norm': '0.3519', 'learning_rate': '0.00015', 'entropy': '1.116', 'num_tokens': '2.443e+06', 'mean_token_accuracy': '0.746', 'epoch': '0.003991'}
{'loss': '1.293', 'grad_norm': '0.419', 'learning_rate': '0.0001503', 'entropy': '1.376', 'num_tokens': '2.45e+06', 'mean_token_accuracy': '0.6879', 'epoch': '0.004'}
{'loss': '1.256', 'grad_norm': '0.483', 'learning_rate': '0.0001507', 'entropy': '1.283', 'num_tokens': '2.454e+06', 'mean_token_accuracy': '0.6838', 'epoch': '0.004009'}
{'loss': '1.086', 'grad_norm': '0.431', 'learning_rate': '0.000151', 'entropy': '0.9893', 'num_tokens': '2.46e+06', 'mean_token_accuracy': '0.7473', 'epoch': '0.004018'}
{'loss': '0.8733', 'grad_norm': '0.4684', 'learning_rate': '0.0001513', 'entropy': '0.8196', 'num_tokens': '2.465e+06', 'mean_token_accuracy': '0.7853', 'epoch': '0.004027'}
{'loss': '0.8652', 'grad_norm': '0.5174', 'learning_rate': '0.0001517', 'entropy': '0.8817', 'num_tokens': '2.469e+06', 'mean_token_accuracy': '0.7845', 'epoch': '0.004036'}
{'loss': '1.727', 'grad_norm': '0.4696', 'learning_rate': '0.000152', 'entropy': '1.33', 'num_tokens': '2.475e+06', 'mean_token_accuracy': '0.6916', 'epoch': '0.004044'}
{'loss': '1.028', 'grad_norm': '0.3476', 'learning_rate': '0.0001523', 'entropy': '0.9299', 'num_tokens': '2.487e+06', 'mean_token_accuracy': '0.7725', 'epoch': '0.004053'}
{'loss': '0.9449', 'grad_norm': '0.5619', 'learning_rate': '0.0001527', 'entropy': '0.9829', 'num_tokens': '2.491e+06', 'mean_token_accuracy': '0.7571', 'epoch': '0.004062'}
{'loss': '1.072', 'grad_norm': '0.3723', 'learning_rate': '0.000153', 'entropy': '1.014', 'num_tokens': '2.498e+06', 'mean_token_accuracy': '0.7635', 'epoch': '0.004071'}
{'loss': '0.8136', 'grad_norm': '0.4525', 'learning_rate': '0.0001533', 'entropy': '0.8187', 'num_tokens': '2.502e+06', 'mean_token_accuracy': '0.8062', 'epoch': '0.00408'}
{'loss': '1.101', 'grad_norm': '0.4951', 'learning_rate': '0.0001537', 'entropy': '1.156', 'num_tokens': '2.506e+06', 'mean_token_accuracy': '0.7296', 'epoch': '0.004089'}
{'loss': '0.8369', 'grad_norm': '0.3942', 'learning_rate': '0.000154', 'entropy': '0.915', 'num_tokens': '2.51e+06', 'mean_token_accuracy': '0.7872', 'epoch': '0.004097'}
{'loss': '1.054', 'grad_norm': '0.4659', 'learning_rate': '0.0001543', 'entropy': '1.103', 'num_tokens': '2.514e+06', 'mean_token_accuracy': '0.7318', 'epoch': '0.004106'}
{'loss': '0.9359', 'grad_norm': '0.4409', 'learning_rate': '0.0001547', 'entropy': '0.8858', 'num_tokens': '2.52e+06', 'mean_token_accuracy': '0.7747', 'epoch': '0.004115'}
{'loss': '0.8686', 'grad_norm': '0.4516', 'learning_rate': '0.000155', 'entropy': '0.9936', 'num_tokens': '2.524e+06', 'mean_token_accuracy': '0.7602', 'epoch': '0.004124'}
{'loss': '1.051', 'grad_norm': '0.3688', 'learning_rate': '0.0001553', 'entropy': '1.098', 'num_tokens': '2.532e+06', 'mean_token_accuracy': '0.7406', 'epoch': '0.004133'}
{'loss': '1.117', 'grad_norm': '0.453', 'learning_rate': '0.0001557', 'entropy': '1.167', 'num_tokens': '2.538e+06', 'mean_token_accuracy': '0.7112', 'epoch': '0.004142'}
{'loss': '1.077', 'grad_norm': '0.4578', 'learning_rate': '0.000156', 'entropy': '1.074', 'num_tokens': '2.543e+06', 'mean_token_accuracy': '0.7355', 'epoch': '0.004151'}
{'loss': '1.214', 'grad_norm': '0.4452', 'learning_rate': '0.0001563', 'entropy': '1.225', 'num_tokens': '2.547e+06', 'mean_token_accuracy': '0.6911', 'epoch': '0.004159'}
{'loss': '1.013', 'grad_norm': '0.3982', 'learning_rate': '0.0001567', 'entropy': '0.969', 'num_tokens': '2.555e+06', 'mean_token_accuracy': '0.7622', 'epoch': '0.004168'}

  4%|▍         | 472/12000 [17:03<6:14:06,  1.95s/it]
  4%|▍         | 473/12000 [17:05<5:45:33,  1.80s/it]
                                                     

  4%|▍         | 473/12000 [17:05<5:45:33,  1.80s/it]
  4%|▍         | 474/12000 [17:06<5:36:48,  1.75s/it]
                                                     

  4%|▍         | 474/12000 [17:06<5:36:48,  1.75s/it]
  4%|▍         | 475/12000 [17:09<6:29:36,  2.03s/it]
                                                     

  4%|▍         | 475/12000 [17:09<6:29:36,  2.03s/it]
  4%|▍         | 476/12000 [17:10<5:56:08,  1.85s/it]
                                                     

  4%|▍         | 476/12000 [17:10<5:56:08,  1.85s/it]
  4%|▍         | 477/12000 [17:12<5:55:01,  1.85s/it]
                                                     

  4%|▍         | 477/12000 [17:12<5:55:01,  1.85s/it]
  4%|▍         | 478/12000 [17:14<5:46:15,  1.80s/it]
                                                     

  4%|▍         | 478/12000 [17:14<5:46:15,  1.80s/it]
  4%|▍         | 479/12000 [17:16<5:42:59,  1.79s/it]
                                                     

  4%|▍         | 479/12000 [17:16<5:42:59,  1.79s/it]
  4%|▍         | 480/12000 [17:18<5:44:23,  1.79s/it]
                                                     

  4%|▍         | 480/12000 [17:18<5:44:23,  1.79s/it]
  4%|▍         | 481/12000 [17:20<6:29:58,  2.03s/it]
                                                     

  4%|▍         | 481/12000 [17:20<6:29:58,  2.03s/it]
  4%|▍         | 482/12000 [17:22<6:18:52,  1.97s/it]
                                                     

  4%|▍         | 482/12000 [17:22<6:18:52,  1.97s/it]
  4%|▍         | 483/12000 [17:24<5:59:26,  1.87s/it]
                                                     

  4%|▍         | 483/12000 [17:24<5:59:26,  1.87s/it]
  4%|▍         | 484/12000 [17:27<7:27:35,  2.33s/it]
                                                     

  4%|▍         | 484/12000 [17:27<7:27:35,  2.33s/it]
  4%|▍         | 485/12000 [17:29<7:05:58,  2.22s/it]
                                                     

  4%|▍         | 485/12000 [17:29<7:05:58,  2.22s/it]
  4%|▍         | 486/12000 [17:31<6:33:43,  2.05s/it]
                                                     

  4%|▍         | 486/12000 [17:31<6:33:43,  2.05s/it]
  4%|▍         | 487/12000 [17:33<6:30:24,  2.03s/it]
                                                     

  4%|▍         | 487/12000 [17:33<6:30:24,  2.03s/it]
  4%|▍         | 488/12000 [17:35<6:35:51,  2.06s/it]
                                                     

  4%|▍         | 488/12000 [17:35<6:35:51,  2.06s/it]
  4%|▍         | 489/12000 [17:36<6:01:10,  1.88s/it]
                                                     

  4%|▍         | 489/12000 [17:36<6:01:10,  1.88s/it]
  4%|▍         | 490/12000 [17:38<6:01:27,  1.88s/it]
                                                     

  4%|▍         | 490/12000 [17:38<6:01:27,  1.88s/it]
  4%|▍         | 491/12000 [17:40<6:02:10,  1.89s/it]
                                                     

  4%|▍         | 491/12000 [17:40<6:02:10,  1.89s/it]
  4%|▍         | 492/12000 [17:42<6:13:36,  1.95s/it]
                                                     

  4%|▍         | 492/12000 [17:42<6:13:36,  1.95s/it]
  4%|▍         | 493/12000 [17:44<6:25:14,  2.01s/it]
                                                     

  4%|▍         | 493/12000 [17:44<6:25:14,  2.01s/it]
  4%|▍         | 494/12000 [17:46<6:18:05,  1.97s/it]
                                                     

  4%|▍         | 494/12000 [17:46<6:18:05,  1.97s/it]
  4%|▍         | 495/12000 [17:48<5:50:57,  1.83s/it]
                                                     

  4%|▍         | 495/12000 [17:48<5:50:57,  1.83s/it]
  4%|▍         | 496/12000 [17:49<5:32:09,  1.73s/it]
                                                     

  4%|▍         | 496/12000 [17:49<5:32:09,  1.73s/it]
  4%|▍         | 497/12000 [17:54<8:32:10,  2.67s/it]
                                                     

  4%|▍         | 497/12000 [17:54<8:32:10,  2.67s/it]
  4%|▍         | 498/12000 [17:56<7:54:49,  2.48s/it]
                                                     

  4%|▍         | 498/12000 [17:56<7:54:49,  2.48s/it]
  4%|▍         | 499/12000 [17:58<7:08:37,  2.24s/it]
                                                     

  4%|▍         | 499/12000 [17:58<7:08:37,  2.24s/it]
  4%|▍         | 500/12000 [17:59<6:33:17,  2.05s/it]
                                                     

  4%|▍         | 500/12000 [17:59<6:33:17,  2.05s/it]
  4%|▍         | 501/12000 [18:03<7:56:01,  2.48s/it]
                                                     

  4%|▍         | 501/12000 [18:03<7:56:01,  2.48s/it]
  4%|▍         | 502/12000 [18:04<7:00:11,  2.19s/it]
                                                     

  4%|▍         | 502/12000 [18:04<7:00:11,  2.19s/it]
  4%|▍         | 503/12000 [18:06<6:30:44,  2.04s/it]
                                                     

  4%|▍         | 503/12000 [18:06<6:30:44,  2.04s/it]
  4%|▍         | 504/12000 [18:08<6:48:06,  2.13s/it]
                                                     

  4%|▍         | 504/12000 [18:08<6:48:06,  2.13s/it]
  4%|▍         | 505/12000 [18:10<6:12:48,  1.95s/it]
                                                     

  4%|▍         | 505/12000 [18:10<6:12:48,  1.95s/it]
  4%|▍         | 506/12000 [18:14<7:56:01,  2.48s/it]
                                                     

  4%|▍         | 506/12000 [18:14<7:56:01,  2.48s/it]
  4%|▍         | 507/12000 [18:15<7:11:28,  2.25s/it]
                                                     

  4%|▍         | 507/12000 [18:15<7:11:28,  2.25s/it]
  4%|▍         | 508/12000 [18:19<8:10:37,  2.56s/it]
                                                     

  4%|▍         | 508/12000 [18:19<8:10:37,  2.56s/it]
  4%|▍         | 509/12000 [18:20<7:09:05,  2.24s/it]
                                                     

  4%|▍         | 509/12000 [18:20<7:09:05,  2.24s/it]
  4%|▍         | 510/12000 [18:22<6:46:41,  2.12s/it]
                                                     

  4%|▍         | 510/12000 [18:22<6:46:41,  2.12s/it]
  4%|▍         | 511/12000 [18:25<7:20:05,  2.30s/it]
                                                     

  4%|▍         | 511/12000 [18:25<7:20:05,  2.30s/it]
  4%|▍         | 512/12000 [18:28<8:43:46,  2.74s/it]
                                                     

  4%|▍         | 512/12000 [18:28<8:43:46,  2.74s/it]
  4%|▍         | 513/12000 [18:30<7:50:21,  2.46s/it]
                                                     

  4%|▍         | 513/12000 [18:30<7:50:21,  2.46s/it]
  4%|▍         | 514/12000 [18:33<8:25:03,  2.64s/it]
                                                     

  4%|▍         | 514/12000 [18:33<8:25:03,  2.64s/it]
  4%|▍         | 515/12000 [18:39<10:56:18,  3.43s/it]
                                                      

  4%|▍         | 515/12000 [18:39<10:56:18,  3.43s/it]
  4%|▍         | 516/12000 [18:40<9:06:14,  2.85s/it] 
                                                     

  4%|▍         | 516/12000 [18:40<9:06:14,  2.85s/it]
  4%|▍         | 517/12000 [18:43<8:50:20,  2.77s/it]
                                                     

  4%|▍         | 517/12000 [18:43<8:50:20,  2.77s/it]
  4%|▍         | 518/12000 [18:44<7:41:02,  2.41s/it]
                                                     

  4%|▍         | 518/12000 [18:44<7:41:02,  2.41s/it]
  4%|▍         | 519/12000 [18:46<6:56:38,  2.18s/it]
                                                     
{'loss': '0.8377', 'grad_norm': '0.4823', 'learning_rate': '0.000157', 'entropy': '0.8607', 'num_tokens': '2.559e+06', 'mean_token_accuracy': '0.7896', 'epoch': '0.004177'}
{'loss': '1.034', 'grad_norm': '0.4585', 'learning_rate': '0.0001573', 'entropy': '1.032', 'num_tokens': '2.564e+06', 'mean_token_accuracy': '0.7407', 'epoch': '0.004186'}
{'loss': '0.8063', 'grad_norm': '0.4898', 'learning_rate': '0.0001577', 'entropy': '0.8086', 'num_tokens': '2.568e+06', 'mean_token_accuracy': '0.7984', 'epoch': '0.004195'}
{'loss': '1.084', 'grad_norm': '0.3397', 'learning_rate': '0.000158', 'entropy': '1.112', 'num_tokens': '2.575e+06', 'mean_token_accuracy': '0.7285', 'epoch': '0.004204'}
{'loss': '0.9259', 'grad_norm': '0.5863', 'learning_rate': '0.0001583', 'entropy': '0.9556', 'num_tokens': '2.579e+06', 'mean_token_accuracy': '0.7625', 'epoch': '0.004213'}
{'loss': '0.757', 'grad_norm': '0.4854', 'learning_rate': '0.0001587', 'entropy': '0.7353', 'num_tokens': '2.585e+06', 'mean_token_accuracy': '0.8003', 'epoch': '0.004221'}
{'loss': '1.248', 'grad_norm': '0.4264', 'learning_rate': '0.000159', 'entropy': '1.232', 'num_tokens': '2.59e+06', 'mean_token_accuracy': '0.7181', 'epoch': '0.00423'}
{'loss': '1.051', 'grad_norm': '0.4016', 'learning_rate': '0.0001593', 'entropy': '1.108', 'num_tokens': '2.595e+06', 'mean_token_accuracy': '0.7407', 'epoch': '0.004239'}
{'loss': '1.129', 'grad_norm': '0.4256', 'learning_rate': '0.0001597', 'entropy': '1.132', 'num_tokens': '2.601e+06', 'mean_token_accuracy': '0.7295', 'epoch': '0.004248'}
{'loss': '0.9937', 'grad_norm': '0.3721', 'learning_rate': '0.00016', 'entropy': '0.9616', 'num_tokens': '2.608e+06', 'mean_token_accuracy': '0.7702', 'epoch': '0.004257'}
{'loss': '0.9361', 'grad_norm': '0.3527', 'learning_rate': '0.0001603', 'entropy': '0.9669', 'num_tokens': '2.613e+06', 'mean_token_accuracy': '0.7648', 'epoch': '0.004266'}
{'loss': '0.9085', 'grad_norm': '0.396', 'learning_rate': '0.0001607', 'entropy': '0.9954', 'num_tokens': '2.618e+06', 'mean_token_accuracy': '0.7549', 'epoch': '0.004274'}
{'loss': '1.016', 'grad_norm': '0.4573', 'learning_rate': '0.000161', 'entropy': '1.039', 'num_tokens': '2.622e+06', 'mean_token_accuracy': '0.7453', 'epoch': '0.004283'}
{'loss': '1.243', 'grad_norm': '0.424', 'learning_rate': '0.0001613', 'entropy': '1.173', 'num_tokens': '2.628e+06', 'mean_token_accuracy': '0.7129', 'epoch': '0.004292'}
{'loss': '0.9154', 'grad_norm': '0.4257', 'learning_rate': '0.0001617', 'entropy': '0.968', 'num_tokens': '2.633e+06', 'mean_token_accuracy': '0.7707', 'epoch': '0.004301'}
{'loss': '0.798', 'grad_norm': '0.3963', 'learning_rate': '0.000162', 'entropy': '0.774', 'num_tokens': '2.639e+06', 'mean_token_accuracy': '0.8029', 'epoch': '0.00431'}
{'loss': '1.133', 'grad_norm': '0.4636', 'learning_rate': '0.0001623', 'entropy': '1.051', 'num_tokens': '2.646e+06', 'mean_token_accuracy': '0.7296', 'epoch': '0.004319'}
{'loss': '1.023', 'grad_norm': '0.4713', 'learning_rate': '0.0001627', 'entropy': '0.9578', 'num_tokens': '2.65e+06', 'mean_token_accuracy': '0.7682', 'epoch': '0.004328'}
{'loss': '0.8158', 'grad_norm': '0.4358', 'learning_rate': '0.000163', 'entropy': '0.7878', 'num_tokens': '2.656e+06', 'mean_token_accuracy': '0.8228', 'epoch': '0.004336'}
{'loss': '1.248', 'grad_norm': '0.4223', 'learning_rate': '0.0001633', 'entropy': '1.202', 'num_tokens': '2.662e+06', 'mean_token_accuracy': '0.721', 'epoch': '0.004345'}
{'loss': '1.087', 'grad_norm': '0.4692', 'learning_rate': '0.0001637', 'entropy': '1.079', 'num_tokens': '2.667e+06', 'mean_token_accuracy': '0.7318', 'epoch': '0.004354'}
{'loss': '1.02', 'grad_norm': '0.3852', 'learning_rate': '0.000164', 'entropy': '1.014', 'num_tokens': '2.673e+06', 'mean_token_accuracy': '0.7613', 'epoch': '0.004363'}
{'loss': '1.094', 'grad_norm': '0.4346', 'learning_rate': '0.0001643', 'entropy': '1.024', 'num_tokens': '2.679e+06', 'mean_token_accuracy': '0.7528', 'epoch': '0.004372'}
{'loss': '0.8498', 'grad_norm': '0.345', 'learning_rate': '0.0001647', 'entropy': '0.9171', 'num_tokens': '2.684e+06', 'mean_token_accuracy': '0.7712', 'epoch': '0.004381'}
{'loss': '0.6768', 'grad_norm': '0.3929', 'learning_rate': '0.000165', 'entropy': '0.7261', 'num_tokens': '2.689e+06', 'mean_token_accuracy': '0.8267', 'epoch': '0.00439'}
{'loss': '1.621', 'grad_norm': '0.4697', 'learning_rate': '0.0001653', 'entropy': '1.389', 'num_tokens': '2.697e+06', 'mean_token_accuracy': '0.6943', 'epoch': '0.004398'}
{'loss': '1.058', 'grad_norm': '0.3548', 'learning_rate': '0.0001657', 'entropy': '1.058', 'num_tokens': '2.702e+06', 'mean_token_accuracy': '0.7527', 'epoch': '0.004407'}
{'loss': '0.8705', 'grad_norm': '0.4396', 'learning_rate': '0.000166', 'entropy': '0.8832', 'num_tokens': '2.707e+06', 'mean_token_accuracy': '0.7808', 'epoch': '0.004416'}
{'loss': '1.204', 'grad_norm': '0.5521', 'learning_rate': '0.0001663', 'entropy': '1.26', 'num_tokens': '2.711e+06', 'mean_token_accuracy': '0.7089', 'epoch': '0.004425'}
{'loss': '0.947', 'grad_norm': '0.3761', 'learning_rate': '0.0001667', 'entropy': '0.9251', 'num_tokens': '2.719e+06', 'mean_token_accuracy': '0.7546', 'epoch': '0.004434'}
{'loss': '0.6812', 'grad_norm': '0.3662', 'learning_rate': '0.000167', 'entropy': '0.6579', 'num_tokens': '2.724e+06', 'mean_token_accuracy': '0.8245', 'epoch': '0.004443'}
{'loss': '1.07', 'grad_norm': '0.4893', 'learning_rate': '0.0001673', 'entropy': '1.06', 'num_tokens': '2.729e+06', 'mean_token_accuracy': '0.738', 'epoch': '0.004451'}
{'loss': '1.058', 'grad_norm': '0.34', 'learning_rate': '0.0001677', 'entropy': '1.031', 'num_tokens': '2.736e+06', 'mean_token_accuracy': '0.7523', 'epoch': '0.00446'}
{'loss': '0.9997', 'grad_norm': '0.4289', 'learning_rate': '0.000168', 'entropy': '0.9313', 'num_tokens': '2.741e+06', 'mean_token_accuracy': '0.758', 'epoch': '0.004469'}
{'loss': '0.7442', 'grad_norm': '0.4132', 'learning_rate': '0.0001683', 'entropy': '0.7138', 'num_tokens': '2.746e+06', 'mean_token_accuracy': '0.821', 'epoch': '0.004478'}
{'loss': '0.6991', 'grad_norm': '0.3987', 'learning_rate': '0.0001687', 'entropy': '0.6976', 'num_tokens': '2.751e+06', 'mean_token_accuracy': '0.8227', 'epoch': '0.004487'}
{'loss': '0.9326', 'grad_norm': '0.3597', 'learning_rate': '0.000169', 'entropy': '0.8556', 'num_tokens': '2.758e+06', 'mean_token_accuracy': '0.7888', 'epoch': '0.004496'}
{'loss': '0.9357', 'grad_norm': '0.4221', 'learning_rate': '0.0001693', 'entropy': '0.9893', 'num_tokens': '2.762e+06', 'mean_token_accuracy': '0.7482', 'epoch': '0.004505'}
{'loss': '0.9069', 'grad_norm': '0.3951', 'learning_rate': '0.0001697', 'entropy': '0.9705', 'num_tokens': '2.768e+06', 'mean_token_accuracy': '0.7657', 'epoch': '0.004513'}
{'loss': '1.283', 'grad_norm': '0.3737', 'learning_rate': '0.00017', 'entropy': '1.195', 'num_tokens': '2.775e+06', 'mean_token_accuracy': '0.7327', 'epoch': '0.004522'}
{'loss': '1.032', 'grad_norm': '0.361', 'learning_rate': '0.0001703', 'entropy': '0.9804', 'num_tokens': '2.782e+06', 'mean_token_accuracy': '0.7665', 'epoch': '0.004531'}
{'loss': '0.9991', 'grad_norm': '0.4567', 'learning_rate': '0.0001707', 'entropy': '1.048', 'num_tokens': '2.787e+06', 'mean_token_accuracy': '0.7397', 'epoch': '0.00454'}
{'loss': '0.8356', 'grad_norm': '0.3574', 'learning_rate': '0.000171', 'entropy': '0.8693', 'num_tokens': '2.794e+06', 'mean_token_accuracy': '0.7873', 'epoch': '0.004549'}
{'loss': '1.523', 'grad_norm': '0.2745', 'learning_rate': '0.0001713', 'entropy': '1.258', 'num_tokens': '2.805e+06', 'mean_token_accuracy': '0.714', 'epoch': '0.004558'}
{'loss': '0.8744', 'grad_norm': '0.3336', 'learning_rate': '0.0001717', 'entropy': '0.8957', 'num_tokens': '2.81e+06', 'mean_token_accuracy': '0.7929', 'epoch': '0.004567'}
{'loss': '0.8152', 'grad_norm': '0.4009', 'learning_rate': '0.000172', 'entropy': '1.082', 'num_tokens': '2.816e+06', 'mean_token_accuracy': '0.745', 'epoch': '0.004575'}
{'loss': '0.8429', 'grad_norm': '0.4009', 'learning_rate': '0.0001723', 'entropy': '0.9297', 'num_tokens': '2.821e+06', 'mean_token_accuracy': '0.7717', 'epoch': '0.004584'}

  4%|▍         | 519/12000 [18:46<6:56:38,  2.18s/it]
  4%|▍         | 520/12000 [18:47<6:16:58,  1.97s/it]
                                                     

  4%|▍         | 520/12000 [18:47<6:16:58,  1.97s/it]
  4%|▍         | 521/12000 [18:50<6:31:49,  2.05s/it]
                                                     

  4%|▍         | 521/12000 [18:50<6:31:49,  2.05s/it]
  4%|▍         | 522/12000 [18:51<6:16:43,  1.97s/it]
                                                     

  4%|▍         | 522/12000 [18:51<6:16:43,  1.97s/it]
  4%|▍         | 523/12000 [18:53<5:52:07,  1.84s/it]
                                                     

  4%|▍         | 523/12000 [18:53<5:52:07,  1.84s/it]
  4%|▍         | 524/12000 [18:55<5:40:47,  1.78s/it]
                                                     

  4%|▍         | 524/12000 [18:55<5:40:47,  1.78s/it]
  4%|▍         | 525/12000 [18:56<5:43:31,  1.80s/it]
                                                     

  4%|▍         | 525/12000 [18:56<5:43:31,  1.80s/it]
  4%|▍         | 526/12000 [18:59<6:06:17,  1.92s/it]
                                                     

  4%|▍         | 526/12000 [18:59<6:06:17,  1.92s/it]
  4%|▍         | 527/12000 [19:00<5:57:19,  1.87s/it]
                                                     

  4%|▍         | 527/12000 [19:00<5:57:19,  1.87s/it]
  4%|▍         | 528/12000 [19:04<7:54:04,  2.48s/it]
                                                     

  4%|▍         | 528/12000 [19:04<7:54:04,  2.48s/it]
  4%|▍         | 529/12000 [19:06<7:34:52,  2.38s/it]
                                                     

  4%|▍         | 529/12000 [19:06<7:34:52,  2.38s/it]
  4%|▍         | 530/12000 [19:09<7:26:05,  2.33s/it]
                                                     

  4%|▍         | 530/12000 [19:09<7:26:05,  2.33s/it]
  4%|▍         | 531/12000 [19:10<6:46:21,  2.13s/it]
                                                     

  4%|▍         | 531/12000 [19:10<6:46:21,  2.13s/it]
  4%|▍         | 532/12000 [19:12<6:25:52,  2.02s/it]
                                                     

  4%|▍         | 532/12000 [19:12<6:25:52,  2.02s/it]
  4%|▍         | 533/12000 [19:15<7:30:25,  2.36s/it]
                                                     

  4%|▍         | 533/12000 [19:15<7:30:25,  2.36s/it]
  4%|▍         | 534/12000 [19:17<6:51:43,  2.15s/it]
                                                     

  4%|▍         | 534/12000 [19:17<6:51:43,  2.15s/it]
  4%|▍         | 535/12000 [19:18<6:19:17,  1.98s/it]
                                                     

  4%|▍         | 535/12000 [19:18<6:19:17,  1.98s/it]
  4%|▍         | 536/12000 [19:21<6:40:03,  2.09s/it]
                                                     

  4%|▍         | 536/12000 [19:21<6:40:03,  2.09s/it]
  4%|▍         | 537/12000 [19:23<6:42:47,  2.11s/it]
                                                     

  4%|▍         | 537/12000 [19:23<6:42:47,  2.11s/it]
  4%|▍         | 538/12000 [19:26<7:22:43,  2.32s/it]
                                                     

  4%|▍         | 538/12000 [19:26<7:22:43,  2.32s/it]
  4%|▍         | 539/12000 [19:29<8:07:58,  2.55s/it]
                                                     

  4%|▍         | 539/12000 [19:29<8:07:58,  2.55s/it]
  4%|▍         | 540/12000 [19:30<7:04:14,  2.22s/it]
                                                     

  4%|▍         | 540/12000 [19:30<7:04:14,  2.22s/it]
  5%|▍         | 541/12000 [19:32<6:22:47,  2.00s/it]
                                                     

  5%|▍         | 541/12000 [19:32<6:22:47,  2.00s/it]
  5%|▍         | 542/12000 [19:34<6:06:32,  1.92s/it]
                                                     

  5%|▍         | 542/12000 [19:34<6:06:32,  1.92s/it]
  5%|▍         | 543/12000 [19:36<6:24:24,  2.01s/it]
                                                     

  5%|▍         | 543/12000 [19:36<6:24:24,  2.01s/it]
  5%|▍         | 544/12000 [19:37<6:03:22,  1.90s/it]
                                                     

  5%|▍         | 544/12000 [19:37<6:03:22,  1.90s/it]
  5%|▍         | 545/12000 [19:39<5:41:18,  1.79s/it]
                                                     

  5%|▍         | 545/12000 [19:39<5:41:18,  1.79s/it]
  5%|▍         | 546/12000 [19:40<5:22:50,  1.69s/it]
                                                     

  5%|▍         | 546/12000 [19:40<5:22:50,  1.69s/it]
  5%|▍         | 547/12000 [19:42<5:35:06,  1.76s/it]
                                                     

  5%|▍         | 547/12000 [19:42<5:35:06,  1.76s/it]
  5%|▍         | 548/12000 [19:44<5:13:29,  1.64s/it]
                                                     

  5%|▍         | 548/12000 [19:44<5:13:29,  1.64s/it]
  5%|▍         | 549/12000 [19:45<5:07:55,  1.61s/it]
                                                     

  5%|▍         | 549/12000 [19:45<5:07:55,  1.61s/it]
  5%|▍         | 550/12000 [19:51<8:54:21,  2.80s/it]
                                                     

  5%|▍         | 550/12000 [19:51<8:54:21,  2.80s/it]
  5%|▍         | 551/12000 [19:54<9:17:50,  2.92s/it]
                                                     

  5%|▍         | 551/12000 [19:54<9:17:50,  2.92s/it]
  5%|▍         | 552/12000 [19:56<8:07:42,  2.56s/it]
                                                     

  5%|▍         | 552/12000 [19:56<8:07:42,  2.56s/it]
  5%|▍         | 553/12000 [19:57<7:10:33,  2.26s/it]
                                                     

  5%|▍         | 553/12000 [19:57<7:10:33,  2.26s/it]
  5%|▍         | 554/12000 [19:59<6:31:36,  2.05s/it]
                                                     

  5%|▍         | 554/12000 [19:59<6:31:36,  2.05s/it]
  5%|▍         | 555/12000 [20:01<6:11:26,  1.95s/it]
                                                     

  5%|▍         | 555/12000 [20:01<6:11:26,  1.95s/it]
  5%|▍         | 556/12000 [20:02<6:08:09,  1.93s/it]
                                                     

  5%|▍         | 556/12000 [20:02<6:08:09,  1.93s/it]
  5%|▍         | 557/12000 [20:04<6:12:36,  1.95s/it]
                                                     

  5%|▍         | 557/12000 [20:04<6:12:36,  1.95s/it]
  5%|▍         | 558/12000 [20:07<6:28:50,  2.04s/it]
                                                     

  5%|▍         | 558/12000 [20:07<6:28:50,  2.04s/it]
  5%|▍         | 559/12000 [20:09<6:37:30,  2.08s/it]
                                                     

  5%|▍         | 559/12000 [20:09<6:37:30,  2.08s/it]
  5%|▍         | 560/12000 [20:12<7:19:34,  2.31s/it]
                                                     

  5%|▍         | 560/12000 [20:12<7:19:34,  2.31s/it]
  5%|▍         | 561/12000 [20:13<6:31:31,  2.05s/it]
                                                     

  5%|▍         | 561/12000 [20:13<6:31:31,  2.05s/it]
  5%|▍         | 562/12000 [20:15<6:10:06,  1.94s/it]
                                                     

  5%|▍         | 562/12000 [20:15<6:10:06,  1.94s/it]
  5%|▍         | 563/12000 [20:17<6:27:51,  2.03s/it]
                                                     

  5%|▍         | 563/12000 [20:17<6:27:51,  2.03s/it]
  5%|▍         | 564/12000 [20:19<6:17:41,  1.98s/it]
                                                     

  5%|▍         | 564/12000 [20:19<6:17:41,  1.98s/it]
  5%|▍         | 565/12000 [20:20<5:52:14,  1.85s/it]
                                                     

  5%|▍         | 565/12000 [20:20<5:52:14,  1.85s/it]
  5%|▍         | 566/12000 [20:23<6:12:03,  1.95s/it]
                                                     
{'loss': '0.9567', 'grad_norm': '0.4182', 'learning_rate': '0.0001727', 'entropy': '1.07', 'num_tokens': '2.825e+06', 'mean_token_accuracy': '0.7357', 'epoch': '0.004593'}
{'loss': '0.839', 'grad_norm': '0.3522', 'learning_rate': '0.000173', 'entropy': '0.9354', 'num_tokens': '2.829e+06', 'mean_token_accuracy': '0.7582', 'epoch': '0.004602'}
{'loss': '1.198', 'grad_norm': '0.3997', 'learning_rate': '0.0001733', 'entropy': '1.04', 'num_tokens': '2.836e+06', 'mean_token_accuracy': '0.7459', 'epoch': '0.004611'}
{'loss': '0.9118', 'grad_norm': '0.3633', 'learning_rate': '0.0001737', 'entropy': '0.9275', 'num_tokens': '2.841e+06', 'mean_token_accuracy': '0.774', 'epoch': '0.00462'}
{'loss': '0.7881', 'grad_norm': '0.4133', 'learning_rate': '0.000174', 'entropy': '0.7467', 'num_tokens': '2.846e+06', 'mean_token_accuracy': '0.8018', 'epoch': '0.004628'}
{'loss': '1.13', 'grad_norm': '0.388', 'learning_rate': '0.0001743', 'entropy': '1.083', 'num_tokens': '2.851e+06', 'mean_token_accuracy': '0.7388', 'epoch': '0.004637'}
{'loss': '1.139', 'grad_norm': '0.3995', 'learning_rate': '0.0001747', 'entropy': '1.185', 'num_tokens': '2.857e+06', 'mean_token_accuracy': '0.7347', 'epoch': '0.004646'}
{'loss': '1.201', 'grad_norm': '0.4105', 'learning_rate': '0.000175', 'entropy': '1.106', 'num_tokens': '2.863e+06', 'mean_token_accuracy': '0.7387', 'epoch': '0.004655'}
{'loss': '0.8798', 'grad_norm': '0.4089', 'learning_rate': '0.0001753', 'entropy': '0.9065', 'num_tokens': '2.868e+06', 'mean_token_accuracy': '0.7726', 'epoch': '0.004664'}
{'loss': '1.134', 'grad_norm': '0.3427', 'learning_rate': '0.0001757', 'entropy': '1.14', 'num_tokens': '2.874e+06', 'mean_token_accuracy': '0.717', 'epoch': '0.004673'}
{'loss': '1.018', 'grad_norm': '0.4217', 'learning_rate': '0.000176', 'entropy': '0.9927', 'num_tokens': '2.88e+06', 'mean_token_accuracy': '0.7566', 'epoch': '0.004682'}
{'loss': '1.049', 'grad_norm': '0.4003', 'learning_rate': '0.0001763', 'entropy': '0.9942', 'num_tokens': '2.887e+06', 'mean_token_accuracy': '0.7586', 'epoch': '0.00469'}
{'loss': '0.7031', 'grad_norm': '0.338', 'learning_rate': '0.0001767', 'entropy': '0.7808', 'num_tokens': '2.893e+06', 'mean_token_accuracy': '0.8104', 'epoch': '0.004699'}
{'loss': '0.9156', 'grad_norm': '0.3534', 'learning_rate': '0.000177', 'entropy': '1.033', 'num_tokens': '2.897e+06', 'mean_token_accuracy': '0.7438', 'epoch': '0.004708'}
{'loss': '0.6849', 'grad_norm': '0.3016', 'learning_rate': '0.0001773', 'entropy': '0.7802', 'num_tokens': '2.905e+06', 'mean_token_accuracy': '0.8073', 'epoch': '0.004717'}
{'loss': '1.282', 'grad_norm': '0.4597', 'learning_rate': '0.0001777', 'entropy': '1.214', 'num_tokens': '2.909e+06', 'mean_token_accuracy': '0.7057', 'epoch': '0.004726'}
{'loss': '1.039', 'grad_norm': '0.4709', 'learning_rate': '0.000178', 'entropy': '1.061', 'num_tokens': '2.914e+06', 'mean_token_accuracy': '0.7437', 'epoch': '0.004735'}
{'loss': '0.9573', 'grad_norm': '0.4167', 'learning_rate': '0.0001783', 'entropy': '0.807', 'num_tokens': '2.92e+06', 'mean_token_accuracy': '0.8014', 'epoch': '0.004744'}
{'loss': '0.8447', 'grad_norm': '0.4202', 'learning_rate': '0.0001787', 'entropy': '0.8481', 'num_tokens': '2.927e+06', 'mean_token_accuracy': '0.7896', 'epoch': '0.004752'}
{'loss': '1.006', 'grad_norm': '0.5225', 'learning_rate': '0.000179', 'entropy': '1.064', 'num_tokens': '2.932e+06', 'mean_token_accuracy': '0.7496', 'epoch': '0.004761'}
{'loss': '0.9646', 'grad_norm': '0.3214', 'learning_rate': '0.0001793', 'entropy': '0.9826', 'num_tokens': '2.94e+06', 'mean_token_accuracy': '0.7721', 'epoch': '0.00477'}
{'loss': '0.9999', 'grad_norm': '0.4465', 'learning_rate': '0.0001797', 'entropy': '1.058', 'num_tokens': '2.944e+06', 'mean_token_accuracy': '0.7491', 'epoch': '0.004779'}
{'loss': '0.874', 'grad_norm': '0.3761', 'learning_rate': '0.00018', 'entropy': '0.9229', 'num_tokens': '2.95e+06', 'mean_token_accuracy': '0.7815', 'epoch': '0.004788'}
{'loss': '0.7315', 'grad_norm': '0.402', 'learning_rate': '0.0001803', 'entropy': '0.7162', 'num_tokens': '2.955e+06', 'mean_token_accuracy': '0.8294', 'epoch': '0.004797'}
{'loss': '0.9076', 'grad_norm': '0.3568', 'learning_rate': '0.0001807', 'entropy': '0.9541', 'num_tokens': '2.961e+06', 'mean_token_accuracy': '0.7696', 'epoch': '0.004805'}
{'loss': '0.8986', 'grad_norm': '0.4403', 'learning_rate': '0.000181', 'entropy': '0.9561', 'num_tokens': '2.965e+06', 'mean_token_accuracy': '0.7569', 'epoch': '0.004814'}
{'loss': '1.269', 'grad_norm': '0.5296', 'learning_rate': '0.0001813', 'entropy': '1.235', 'num_tokens': '2.969e+06', 'mean_token_accuracy': '0.7136', 'epoch': '0.004823'}
{'loss': '0.7004', 'grad_norm': '0.3788', 'learning_rate': '0.0001817', 'entropy': '0.7759', 'num_tokens': '2.973e+06', 'mean_token_accuracy': '0.8104', 'epoch': '0.004832'}
{'loss': '0.9677', 'grad_norm': '0.389', 'learning_rate': '0.000182', 'entropy': '0.9881', 'num_tokens': '2.979e+06', 'mean_token_accuracy': '0.7675', 'epoch': '0.004841'}
{'loss': '1.077', 'grad_norm': '0.3581', 'learning_rate': '0.0001823', 'entropy': '1.126', 'num_tokens': '2.983e+06', 'mean_token_accuracy': '0.7435', 'epoch': '0.00485'}
{'loss': '0.6279', 'grad_norm': '0.5565', 'learning_rate': '0.0001827', 'entropy': '0.5925', 'num_tokens': '2.988e+06', 'mean_token_accuracy': '0.8507', 'epoch': '0.004859'}
{'loss': '0.7851', 'grad_norm': '0.2752', 'learning_rate': '0.000183', 'entropy': '0.8543', 'num_tokens': '2.997e+06', 'mean_token_accuracy': '0.7908', 'epoch': '0.004867'}
{'loss': '1.153', 'grad_norm': '0.3853', 'learning_rate': '0.0001833', 'entropy': '1.183', 'num_tokens': '3.002e+06', 'mean_token_accuracy': '0.724', 'epoch': '0.004876'}
{'loss': '1.087', 'grad_norm': '0.3944', 'learning_rate': '0.0001837', 'entropy': '1.153', 'num_tokens': '3.007e+06', 'mean_token_accuracy': '0.7382', 'epoch': '0.004885'}
{'loss': '1.134', 'grad_norm': '0.4377', 'learning_rate': '0.000184', 'entropy': '1.24', 'num_tokens': '3.012e+06', 'mean_token_accuracy': '0.7003', 'epoch': '0.004894'}
{'loss': '1.069', 'grad_norm': '2.273', 'learning_rate': '0.0001843', 'entropy': '1.139', 'num_tokens': '3.016e+06', 'mean_token_accuracy': '0.7265', 'epoch': '0.004903'}
{'loss': '1.118', 'grad_norm': '0.4716', 'learning_rate': '0.0001847', 'entropy': '1.039', 'num_tokens': '3.021e+06', 'mean_token_accuracy': '0.7288', 'epoch': '0.004912'}
{'loss': '1.188', 'grad_norm': '0.4704', 'learning_rate': '0.000185', 'entropy': '1.205', 'num_tokens': '3.025e+06', 'mean_token_accuracy': '0.7149', 'epoch': '0.004921'}
{'loss': '1.383', 'grad_norm': '0.406', 'learning_rate': '0.0001853', 'entropy': '1.239', 'num_tokens': '3.031e+06', 'mean_token_accuracy': '0.7039', 'epoch': '0.004929'}
{'loss': '1.219', 'grad_norm': '0.4035', 'learning_rate': '0.0001857', 'entropy': '1.271', 'num_tokens': '3.037e+06', 'mean_token_accuracy': '0.6876', 'epoch': '0.004938'}
{'loss': '1.009', 'grad_norm': '0.3628', 'learning_rate': '0.000186', 'entropy': '0.9734', 'num_tokens': '3.043e+06', 'mean_token_accuracy': '0.7643', 'epoch': '0.004947'}
{'loss': '1.075', 'grad_norm': '0.259', 'learning_rate': '0.0001863', 'entropy': '1.172', 'num_tokens': '3.052e+06', 'mean_token_accuracy': '0.7355', 'epoch': '0.004956'}
{'loss': '0.9611', 'grad_norm': '0.3505', 'learning_rate': '0.0001867', 'entropy': '0.9897', 'num_tokens': '3.056e+06', 'mean_token_accuracy': '0.7648', 'epoch': '0.004965'}
{'loss': '1.181', 'grad_norm': '0.3776', 'learning_rate': '0.000187', 'entropy': '1.114', 'num_tokens': '3.061e+06', 'mean_token_accuracy': '0.7357', 'epoch': '0.004974'}
{'loss': '1.105', 'grad_norm': '0.4154', 'learning_rate': '0.0001873', 'entropy': '1.191', 'num_tokens': '3.067e+06', 'mean_token_accuracy': '0.6982', 'epoch': '0.004982'}
{'loss': '0.8859', 'grad_norm': '0.4231', 'learning_rate': '0.0001877', 'entropy': '0.9409', 'num_tokens': '3.072e+06', 'mean_token_accuracy': '0.7813', 'epoch': '0.004991'}
{'loss': '1.091', 'grad_norm': '0.4526', 'learning_rate': '0.000188', 'entropy': '1.095', 'num_tokens': '3.076e+06', 'mean_token_accuracy': '0.7392', 'epoch': '0.005'}

  5%|▍         | 566/12000 [20:23<6:12:03,  1.95s/it]
  5%|▍         | 567/12000 [20:27<8:29:31,  2.67s/it]
                                                     

  5%|▍         | 567/12000 [20:27<8:29:31,  2.67s/it]
  5%|▍         | 568/12000 [20:30<8:39:51,  2.73s/it]
                                                     

  5%|▍         | 568/12000 [20:30<8:39:51,  2.73s/it]
  5%|▍         | 569/12000 [20:32<7:37:38,  2.40s/it]
                                                     

  5%|▍         | 569/12000 [20:32<7:37:38,  2.40s/it]
  5%|▍         | 570/12000 [20:33<7:13:20,  2.27s/it]
                                                     

  5%|▍         | 570/12000 [20:33<7:13:20,  2.27s/it]
  5%|▍         | 571/12000 [20:35<6:18:04,  1.98s/it]
                                                     

  5%|▍         | 571/12000 [20:35<6:18:04,  1.98s/it]
  5%|▍         | 572/12000 [20:38<7:47:04,  2.45s/it]
                                                     

  5%|▍         | 572/12000 [20:38<7:47:04,  2.45s/it]
  5%|▍         | 573/12000 [20:41<7:48:07,  2.46s/it]
                                                     

  5%|▍         | 573/12000 [20:41<7:48:07,  2.46s/it]
  5%|▍         | 574/12000 [20:42<6:54:00,  2.17s/it]
                                                     

  5%|▍         | 574/12000 [20:42<6:54:00,  2.17s/it]
  5%|▍         | 575/12000 [20:44<6:22:01,  2.01s/it]
                                                     

  5%|▍         | 575/12000 [20:44<6:22:01,  2.01s/it]
  5%|▍         | 576/12000 [20:46<5:59:16,  1.89s/it]
                                                     

  5%|▍         | 576/12000 [20:46<5:59:16,  1.89s/it]
  5%|▍         | 577/12000 [20:47<5:39:04,  1.78s/it]
                                                     

  5%|▍         | 577/12000 [20:47<5:39:04,  1.78s/it]
  5%|▍         | 578/12000 [20:49<5:44:02,  1.81s/it]
                                                     

  5%|▍         | 578/12000 [20:49<5:44:02,  1.81s/it]
  5%|▍         | 579/12000 [20:50<5:28:07,  1.72s/it]
                                                     

  5%|▍         | 579/12000 [20:50<5:28:07,  1.72s/it]
  5%|▍         | 580/12000 [20:53<6:11:08,  1.95s/it]
                                                     

  5%|▍         | 580/12000 [20:53<6:11:08,  1.95s/it]
  5%|▍         | 581/12000 [20:55<5:54:02,  1.86s/it]
                                                     

  5%|▍         | 581/12000 [20:55<5:54:02,  1.86s/it]
  5%|▍         | 582/12000 [20:56<5:36:01,  1.77s/it]
                                                     

  5%|▍         | 582/12000 [20:56<5:36:01,  1.77s/it]
  5%|▍         | 583/12000 [20:58<5:40:18,  1.79s/it]
                                                     

  5%|▍         | 583/12000 [20:58<5:40:18,  1.79s/it]
  5%|▍         | 584/12000 [20:59<5:23:29,  1.70s/it]
                                                     

  5%|▍         | 584/12000 [20:59<5:23:29,  1.70s/it]
  5%|▍         | 585/12000 [21:01<5:11:11,  1.64s/it]
                                                     

  5%|▍         | 585/12000 [21:01<5:11:11,  1.64s/it]
  5%|▍         | 586/12000 [21:03<5:19:41,  1.68s/it]
                                                     

  5%|▍         | 586/12000 [21:03<5:19:41,  1.68s/it]
  5%|▍         | 587/12000 [21:05<5:40:44,  1.79s/it]
                                                     

  5%|▍         | 587/12000 [21:05<5:40:44,  1.79s/it]
  5%|▍         | 588/12000 [21:06<5:24:53,  1.71s/it]
                                                     

  5%|▍         | 588/12000 [21:06<5:24:53,  1.71s/it]
  5%|▍         | 589/12000 [21:08<5:07:50,  1.62s/it]
                                                     

  5%|▍         | 589/12000 [21:08<5:07:50,  1.62s/it]
  5%|▍         | 590/12000 [21:10<6:12:19,  1.96s/it]
                                                     

  5%|▍         | 590/12000 [21:10<6:12:19,  1.96s/it]
  5%|▍         | 591/12000 [21:12<6:09:48,  1.94s/it]
                                                     

  5%|▍         | 591/12000 [21:12<6:09:48,  1.94s/it]
  5%|▍         | 592/12000 [21:14<5:47:17,  1.83s/it]
                                                     

  5%|▍         | 592/12000 [21:14<5:47:17,  1.83s/it]
  5%|▍         | 593/12000 [21:16<6:10:44,  1.95s/it]
                                                     

  5%|▍         | 593/12000 [21:16<6:10:44,  1.95s/it]
  5%|▍         | 594/12000 [21:20<7:34:56,  2.39s/it]
                                                     

  5%|▍         | 594/12000 [21:20<7:34:56,  2.39s/it]
  5%|▍         | 595/12000 [21:22<7:50:27,  2.48s/it]
                                                     

  5%|▍         | 595/12000 [21:22<7:50:27,  2.48s/it]
  5%|▍         | 596/12000 [21:24<6:55:39,  2.19s/it]
                                                     

  5%|▍         | 596/12000 [21:24<6:55:39,  2.19s/it]
  5%|▍         | 597/12000 [21:25<6:27:42,  2.04s/it]
                                                     

  5%|▍         | 597/12000 [21:25<6:27:42,  2.04s/it]
  5%|▍         | 598/12000 [21:27<6:04:01,  1.92s/it]
                                                     

  5%|▍         | 598/12000 [21:27<6:04:01,  1.92s/it]
  5%|▍         | 599/12000 [21:30<6:54:40,  2.18s/it]
                                                     

  5%|▍         | 599/12000 [21:30<6:54:40,  2.18s/it]
  5%|▌         | 600/12000 [21:39<13:20:22,  4.21s/it]
                                                      

  5%|▌         | 600/12000 [21:39<13:20:22,  4.21s/it]
  5%|▌         | 601/12000 [21:43<13:40:55,  4.32s/it]
                                                      

  5%|▌         | 601/12000 [21:43<13:40:55,  4.32s/it]
  5%|▌         | 602/12000 [21:45<11:11:47,  3.54s/it]
                                                      

  5%|▌         | 602/12000 [21:45<11:11:47,  3.54s/it]
  5%|▌         | 603/12000 [21:47<9:17:43,  2.94s/it] 
                                                     

  5%|▌         | 603/12000 [21:47<9:17:43,  2.94s/it]
  5%|▌         | 604/12000 [21:50<9:57:29,  3.15s/it]
                                                     

  5%|▌         | 604/12000 [21:50<9:57:29,  3.15s/it]
  5%|▌         | 605/12000 [21:52<8:24:03,  2.65s/it]
                                                     

  5%|▌         | 605/12000 [21:52<8:24:03,  2.65s/it]
  5%|▌         | 606/12000 [21:54<7:41:45,  2.43s/it]
                                                     

  5%|▌         | 606/12000 [21:54<7:41:45,  2.43s/it]
  5%|▌         | 607/12000 [21:55<6:54:10,  2.18s/it]
                                                     

  5%|▌         | 607/12000 [21:55<6:54:10,  2.18s/it]
  5%|▌         | 608/12000 [21:57<6:22:15,  2.01s/it]
                                                     

  5%|▌         | 608/12000 [21:57<6:22:15,  2.01s/it]
  5%|▌         | 609/12000 [21:59<6:05:52,  1.93s/it]
                                                     

  5%|▌         | 609/12000 [21:59<6:05:52,  1.93s/it]
  5%|▌         | 610/12000 [22:01<6:25:37,  2.03s/it]
                                                     

  5%|▌         | 610/12000 [22:01<6:25:37,  2.03s/it]
  5%|▌         | 611/12000 [22:02<5:52:50,  1.86s/it]
                                                     

  5%|▌         | 611/12000 [22:02<5:52:50,  1.86s/it]
  5%|▌         | 612/12000 [22:04<5:47:05,  1.83s/it]
                                                     

  5%|▌         | 612/12000 [22:04<5:47:05,  1.83s/it]
  5%|▌         | 613/12000 [22:06<5:47:50,  1.83s/it]
                                                     
{'loss': '1.264', 'grad_norm': '0.4508', 'learning_rate': '0.0001883', 'entropy': '1.18', 'num_tokens': '3.082e+06', 'mean_token_accuracy': '0.7156', 'epoch': '0.005009'}
{'loss': '0.8804', 'grad_norm': '0.2989', 'learning_rate': '0.0001887', 'entropy': '0.9433', 'num_tokens': '3.091e+06', 'mean_token_accuracy': '0.7758', 'epoch': '0.005018'}
{'loss': '1.033', 'grad_norm': '0.6177', 'learning_rate': '0.000189', 'entropy': '0.9815', 'num_tokens': '3.098e+06', 'mean_token_accuracy': '0.7493', 'epoch': '0.005027'}
{'loss': '0.7691', 'grad_norm': '0.3542', 'learning_rate': '0.0001893', 'entropy': '0.8169', 'num_tokens': '3.103e+06', 'mean_token_accuracy': '0.8005', 'epoch': '0.005036'}
{'loss': '0.8856', 'grad_norm': '0.3588', 'learning_rate': '0.0001897', 'entropy': '0.7818', 'num_tokens': '3.109e+06', 'mean_token_accuracy': '0.8057', 'epoch': '0.005044'}
{'loss': '1.194', 'grad_norm': '0.4419', 'learning_rate': '0.00019', 'entropy': '1.22', 'num_tokens': '3.112e+06', 'mean_token_accuracy': '0.7065', 'epoch': '0.005053'}
{'loss': '1.031', 'grad_norm': '0.436', 'learning_rate': '0.0001903', 'entropy': '1', 'num_tokens': '3.118e+06', 'mean_token_accuracy': '0.7575', 'epoch': '0.005062'}
{'loss': '1.279', 'grad_norm': '0.3727', 'learning_rate': '0.0001907', 'entropy': '1.269', 'num_tokens': '3.124e+06', 'mean_token_accuracy': '0.6924', 'epoch': '0.005071'}
{'loss': '1.032', 'grad_norm': '0.4747', 'learning_rate': '0.000191', 'entropy': '1.039', 'num_tokens': '3.129e+06', 'mean_token_accuracy': '0.7415', 'epoch': '0.00508'}
{'loss': '1.005', 'grad_norm': '0.4347', 'learning_rate': '0.0001913', 'entropy': '0.9994', 'num_tokens': '3.133e+06', 'mean_token_accuracy': '0.7601', 'epoch': '0.005089'}
{'loss': '0.8371', 'grad_norm': '0.8592', 'learning_rate': '0.0001917', 'entropy': '0.8747', 'num_tokens': '3.137e+06', 'mean_token_accuracy': '0.7958', 'epoch': '0.005098'}
{'loss': '1.047', 'grad_norm': '0.5244', 'learning_rate': '0.000192', 'entropy': '1.062', 'num_tokens': '3.141e+06', 'mean_token_accuracy': '0.7451', 'epoch': '0.005106'}
{'loss': '1.003', 'grad_norm': '0.5358', 'learning_rate': '0.0001923', 'entropy': '1.022', 'num_tokens': '3.145e+06', 'mean_token_accuracy': '0.7533', 'epoch': '0.005115'}
{'loss': '0.9625', 'grad_norm': '0.425', 'learning_rate': '0.0001927', 'entropy': '1.007', 'num_tokens': '3.149e+06', 'mean_token_accuracy': '0.7459', 'epoch': '0.005124'}
{'loss': '0.8567', 'grad_norm': '0.3452', 'learning_rate': '0.000193', 'entropy': '0.8078', 'num_tokens': '3.155e+06', 'mean_token_accuracy': '0.7901', 'epoch': '0.005133'}
{'loss': '0.8142', 'grad_norm': '0.425', 'learning_rate': '0.0001933', 'entropy': '0.8424', 'num_tokens': '3.159e+06', 'mean_token_accuracy': '0.7855', 'epoch': '0.005142'}
{'loss': '0.9781', 'grad_norm': '0.5229', 'learning_rate': '0.0001937', 'entropy': '0.9664', 'num_tokens': '3.164e+06', 'mean_token_accuracy': '0.7478', 'epoch': '0.005151'}
{'loss': '1.019', 'grad_norm': '0.4524', 'learning_rate': '0.000194', 'entropy': '1.024', 'num_tokens': '3.168e+06', 'mean_token_accuracy': '0.7571', 'epoch': '0.005159'}
{'loss': '1.137', 'grad_norm': '0.5113', 'learning_rate': '0.0001943', 'entropy': '1.09', 'num_tokens': '3.173e+06', 'mean_token_accuracy': '0.7336', 'epoch': '0.005168'}
{'loss': '1.126', 'grad_norm': '0.5154', 'learning_rate': '0.0001947', 'entropy': '1.166', 'num_tokens': '3.176e+06', 'mean_token_accuracy': '0.7178', 'epoch': '0.005177'}
{'loss': '0.8841', 'grad_norm': '0.3882', 'learning_rate': '0.000195', 'entropy': '0.8789', 'num_tokens': '3.181e+06', 'mean_token_accuracy': '0.7818', 'epoch': '0.005186'}
{'loss': '0.8419', 'grad_norm': '0.3534', 'learning_rate': '0.0001953', 'entropy': '0.8348', 'num_tokens': '3.187e+06', 'mean_token_accuracy': '0.8062', 'epoch': '0.005195'}
{'loss': '1.137', 'grad_norm': '0.4153', 'learning_rate': '0.0001957', 'entropy': '1.205', 'num_tokens': '3.191e+06', 'mean_token_accuracy': '0.728', 'epoch': '0.005204'}
{'loss': '1.066', 'grad_norm': '0.4613', 'learning_rate': '0.000196', 'entropy': '1.155', 'num_tokens': '3.195e+06', 'mean_token_accuracy': '0.7324', 'epoch': '0.005213'}
{'loss': '0.9376', 'grad_norm': '0.5063', 'learning_rate': '0.0001963', 'entropy': '1.038', 'num_tokens': '3.202e+06', 'mean_token_accuracy': '0.7518', 'epoch': '0.005221'}
{'loss': '0.9712', 'grad_norm': '0.3576', 'learning_rate': '0.0001967', 'entropy': '0.9857', 'num_tokens': '3.208e+06', 'mean_token_accuracy': '0.7668', 'epoch': '0.00523'}
{'loss': '0.8034', 'grad_norm': '0.419', 'learning_rate': '0.000197', 'entropy': '0.7536', 'num_tokens': '3.213e+06', 'mean_token_accuracy': '0.8029', 'epoch': '0.005239'}
{'loss': '0.9436', 'grad_norm': '0.3238', 'learning_rate': '0.0001973', 'entropy': '0.9221', 'num_tokens': '3.219e+06', 'mean_token_accuracy': '0.7706', 'epoch': '0.005248'}
{'loss': '1.572', 'grad_norm': '0.5633', 'learning_rate': '0.0001977', 'entropy': '1.493', 'num_tokens': '3.222e+06', 'mean_token_accuracy': '0.6709', 'epoch': '0.005257'}
{'loss': '1.276', 'grad_norm': '0.4819', 'learning_rate': '0.000198', 'entropy': '1.21', 'num_tokens': '3.228e+06', 'mean_token_accuracy': '0.6985', 'epoch': '0.005266'}
{'loss': '0.7989', 'grad_norm': '0.4409', 'learning_rate': '0.0001983', 'entropy': '0.766', 'num_tokens': '3.232e+06', 'mean_token_accuracy': '0.8037', 'epoch': '0.005274'}
{'loss': '1.154', 'grad_norm': '0.4047', 'learning_rate': '0.0001987', 'entropy': '1.161', 'num_tokens': '3.238e+06', 'mean_token_accuracy': '0.7218', 'epoch': '0.005283'}
{'loss': '0.8255', 'grad_norm': '0.5383', 'learning_rate': '0.000199', 'entropy': '0.8627', 'num_tokens': '3.243e+06', 'mean_token_accuracy': '0.7801', 'epoch': '0.005292'}
{'loss': '0.9569', 'grad_norm': '0.3746', 'learning_rate': '0.0001993', 'entropy': '0.9345', 'num_tokens': '3.25e+06', 'mean_token_accuracy': '0.7635', 'epoch': '0.005301'}
{'loss': '1.6', 'grad_norm': '0.2539', 'learning_rate': '0.0001997', 'entropy': '1.646', 'num_tokens': '3.265e+06', 'mean_token_accuracy': '0.6595', 'epoch': '0.00531'}
{'loss': '1.083', 'grad_norm': '0.352', 'learning_rate': '0.0002', 'entropy': '1.048', 'num_tokens': '3.272e+06', 'mean_token_accuracy': '0.7413', 'epoch': '0.005319'}
{'loss': '0.772', 'grad_norm': '0.3512', 'learning_rate': '0.0002', 'entropy': '0.7627', 'num_tokens': '3.277e+06', 'mean_token_accuracy': '0.8154', 'epoch': '0.005328'}
{'loss': '0.9447', 'grad_norm': '0.3969', 'learning_rate': '0.0002', 'entropy': '1.007', 'num_tokens': '3.281e+06', 'mean_token_accuracy': '0.759', 'epoch': '0.005336'}
{'loss': '1.452', 'grad_norm': '0.3241', 'learning_rate': '0.0002', 'entropy': '1.145', 'num_tokens': '3.29e+06', 'mean_token_accuracy': '0.7294', 'epoch': '0.005345'}
{'loss': '1.454', 'grad_norm': '0.454', 'learning_rate': '0.0002', 'entropy': '1.417', 'num_tokens': '3.294e+06', 'mean_token_accuracy': '0.6718', 'epoch': '0.005354'}
{'loss': '1.117', 'grad_norm': '0.3634', 'learning_rate': '0.0002', 'entropy': '1.01', 'num_tokens': '3.3e+06', 'mean_token_accuracy': '0.7623', 'epoch': '0.005363'}
{'loss': '0.8927', 'grad_norm': '0.3566', 'learning_rate': '0.0002', 'entropy': '0.9166', 'num_tokens': '3.305e+06', 'mean_token_accuracy': '0.766', 'epoch': '0.005372'}
{'loss': '0.9153', 'grad_norm': '0.66', 'learning_rate': '0.0002', 'entropy': '0.9595', 'num_tokens': '3.309e+06', 'mean_token_accuracy': '0.755', 'epoch': '0.005381'}
{'loss': '1.272', 'grad_norm': '0.4336', 'learning_rate': '0.0002', 'entropy': '1.262', 'num_tokens': '3.314e+06', 'mean_token_accuracy': '0.6978', 'epoch': '0.00539'}
{'loss': '0.8807', 'grad_norm': '0.3617', 'learning_rate': '0.0002', 'entropy': '0.913', 'num_tokens': '3.319e+06', 'mean_token_accuracy': '0.7781', 'epoch': '0.005398'}
{'loss': '1.094', 'grad_norm': '0.3681', 'learning_rate': '0.0002', 'entropy': '1.122', 'num_tokens': '3.324e+06', 'mean_token_accuracy': '0.7236', 'epoch': '0.005407'}
{'loss': '0.8676', 'grad_norm': '0.4008', 'learning_rate': '0.0002', 'entropy': '0.8363', 'num_tokens': '3.329e+06', 'mean_token_accuracy': '0.7935', 'epoch': '0.005416'}

  5%|▌         | 613/12000 [22:06<5:47:50,  1.83s/it]
  5%|▌         | 614/12000 [22:07<5:27:32,  1.73s/it]
                                                     

  5%|▌         | 614/12000 [22:07<5:27:32,  1.73s/it]
  5%|▌         | 615/12000 [22:11<7:35:34,  2.40s/it]
                                                     

  5%|▌         | 615/12000 [22:11<7:35:34,  2.40s/it]
  5%|▌         | 616/12000 [22:14<7:51:59,  2.49s/it]
                                                     

  5%|▌         | 616/12000 [22:14<7:51:59,  2.49s/it]
  5%|▌         | 617/12000 [22:16<7:37:21,  2.41s/it]
                                                     

  5%|▌         | 617/12000 [22:16<7:37:21,  2.41s/it]
  5%|▌         | 618/12000 [22:19<7:27:31,  2.36s/it]
                                                     

  5%|▌         | 618/12000 [22:19<7:27:31,  2.36s/it]
  5%|▌         | 619/12000 [22:20<6:43:59,  2.13s/it]
                                                     

  5%|▌         | 619/12000 [22:20<6:43:59,  2.13s/it]
  5%|▌         | 620/12000 [22:22<6:07:44,  1.94s/it]
                                                     

  5%|▌         | 620/12000 [22:22<6:07:44,  1.94s/it]
  5%|▌         | 621/12000 [22:24<6:47:10,  2.15s/it]
                                                     

  5%|▌         | 621/12000 [22:24<6:47:10,  2.15s/it]
  5%|▌         | 622/12000 [22:26<6:28:24,  2.05s/it]
                                                     

  5%|▌         | 622/12000 [22:26<6:28:24,  2.05s/it]
  5%|▌         | 623/12000 [22:28<6:12:33,  1.96s/it]
                                                     

  5%|▌         | 623/12000 [22:28<6:12:33,  1.96s/it]
  5%|▌         | 624/12000 [22:30<5:54:38,  1.87s/it]
                                                     

  5%|▌         | 624/12000 [22:30<5:54:38,  1.87s/it]
  5%|▌         | 625/12000 [22:31<5:39:05,  1.79s/it]
                                                     

  5%|▌         | 625/12000 [22:31<5:39:05,  1.79s/it]
  5%|▌         | 626/12000 [22:33<5:58:33,  1.89s/it]
                                                     

  5%|▌         | 626/12000 [22:33<5:58:33,  1.89s/it]
  5%|▌         | 627/12000 [22:36<6:32:33,  2.07s/it]
                                                     

  5%|▌         | 627/12000 [22:36<6:32:33,  2.07s/it]
  5%|▌         | 628/12000 [22:38<6:21:24,  2.01s/it]
                                                     

  5%|▌         | 628/12000 [22:38<6:21:24,  2.01s/it]
  5%|▌         | 629/12000 [22:39<5:56:23,  1.88s/it]
                                                     

  5%|▌         | 629/12000 [22:39<5:56:23,  1.88s/it]
  5%|▌         | 630/12000 [22:41<6:09:29,  1.95s/it]
                                                     

  5%|▌         | 630/12000 [22:41<6:09:29,  1.95s/it]
  5%|▌         | 631/12000 [22:43<6:15:53,  1.98s/it]
                                                     

  5%|▌         | 631/12000 [22:43<6:15:53,  1.98s/it]
  5%|▌         | 632/12000 [22:45<5:59:38,  1.90s/it]
                                                     

  5%|▌         | 632/12000 [22:45<5:59:38,  1.90s/it]
  5%|▌         | 633/12000 [22:47<5:49:54,  1.85s/it]
                                                     

  5%|▌         | 633/12000 [22:47<5:49:54,  1.85s/it]
  5%|▌         | 634/12000 [22:49<6:03:56,  1.92s/it]
                                                     

  5%|▌         | 634/12000 [22:49<6:03:56,  1.92s/it]
  5%|▌         | 635/12000 [22:50<5:42:53,  1.81s/it]
                                                     

  5%|▌         | 635/12000 [22:50<5:42:53,  1.81s/it]
  5%|▌         | 636/12000 [22:53<6:30:05,  2.06s/it]
                                                     

  5%|▌         | 636/12000 [22:53<6:30:05,  2.06s/it]
  5%|▌         | 637/12000 [22:57<8:29:26,  2.69s/it]
                                                     

  5%|▌         | 637/12000 [22:57<8:29:26,  2.69s/it]
  5%|▌         | 638/12000 [22:59<7:23:16,  2.34s/it]
                                                     

  5%|▌         | 638/12000 [22:59<7:23:16,  2.34s/it]
  5%|▌         | 639/12000 [23:00<6:44:50,  2.14s/it]
                                                     

  5%|▌         | 639/12000 [23:00<6:44:50,  2.14s/it]
  5%|▌         | 640/12000 [23:02<6:20:12,  2.01s/it]
                                                     

  5%|▌         | 640/12000 [23:02<6:20:12,  2.01s/it]
  5%|▌         | 641/12000 [23:04<5:50:05,  1.85s/it]
                                                     

  5%|▌         | 641/12000 [23:04<5:50:05,  1.85s/it]
  5%|▌         | 642/12000 [23:05<5:39:38,  1.79s/it]
                                                     

  5%|▌         | 642/12000 [23:05<5:39:38,  1.79s/it]
  5%|▌         | 643/12000 [23:07<5:22:34,  1.70s/it]
                                                     

  5%|▌         | 643/12000 [23:07<5:22:34,  1.70s/it]
  5%|▌         | 644/12000 [23:09<5:23:41,  1.71s/it]
                                                     

  5%|▌         | 644/12000 [23:09<5:23:41,  1.71s/it]
  5%|▌         | 645/12000 [23:12<6:36:46,  2.10s/it]
                                                     

  5%|▌         | 645/12000 [23:12<6:36:46,  2.10s/it]
  5%|▌         | 646/12000 [23:14<7:16:24,  2.31s/it]
                                                     

  5%|▌         | 646/12000 [23:14<7:16:24,  2.31s/it]
  5%|▌         | 647/12000 [23:16<6:38:18,  2.11s/it]
                                                     

  5%|▌         | 647/12000 [23:16<6:38:18,  2.11s/it]
  5%|▌         | 648/12000 [23:19<7:43:50,  2.45s/it]
                                                     

  5%|▌         | 648/12000 [23:19<7:43:50,  2.45s/it]
  5%|▌         | 649/12000 [23:25<11:06:59,  3.53s/it]
                                                      

  5%|▌         | 649/12000 [23:25<11:06:59,  3.53s/it]
  5%|▌         | 650/12000 [23:27<9:41:43,  3.08s/it] 
                                                     

  5%|▌         | 650/12000 [23:27<9:41:43,  3.08s/it]
  5%|▌         | 651/12000 [23:31<9:51:32,  3.13s/it]
                                                     

  5%|▌         | 651/12000 [23:31<9:51:32,  3.13s/it]
  5%|▌         | 652/12000 [23:32<8:31:43,  2.71s/it]
                                                     

  5%|▌         | 652/12000 [23:32<8:31:43,  2.71s/it]
  5%|▌         | 653/12000 [23:34<7:44:07,  2.45s/it]
                                                     

  5%|▌         | 653/12000 [23:34<7:44:07,  2.45s/it]
  5%|▌         | 654/12000 [23:36<6:58:51,  2.21s/it]
                                                     

  5%|▌         | 654/12000 [23:36<6:58:51,  2.21s/it]
  5%|▌         | 655/12000 [23:37<6:22:16,  2.02s/it]
                                                     

  5%|▌         | 655/12000 [23:37<6:22:16,  2.02s/it]
  5%|▌         | 656/12000 [23:39<6:05:01,  1.93s/it]
                                                     

  5%|▌         | 656/12000 [23:39<6:05:01,  1.93s/it]
  5%|▌         | 657/12000 [23:41<5:44:25,  1.82s/it]
                                                     

  5%|▌         | 657/12000 [23:41<5:44:25,  1.82s/it]
  5%|▌         | 658/12000 [23:43<6:04:43,  1.93s/it]
                                                     

  5%|▌         | 658/12000 [23:43<6:04:43,  1.93s/it]
  5%|▌         | 659/12000 [23:47<7:56:48,  2.52s/it]
                                                     

  5%|▌         | 659/12000 [23:47<7:56:48,  2.52s/it]
  6%|▌         | 660/12000 [23:50<8:12:40,  2.61s/it]
                                                     

  6%|▌         | 660/12000 [23:50<8:12:40,  2.61s/it]
  6%|▌         | 661/12000 [23:51<7:16:04,  2.31s/it]
                                                     
{'loss': '0.8809', 'grad_norm': '0.4791', 'learning_rate': '0.0002', 'entropy': '0.9423', 'num_tokens': '3.333e+06', 'mean_token_accuracy': '0.7612', 'epoch': '0.005425'}
{'loss': '0.994', 'grad_norm': '0.4141', 'learning_rate': '0.0002', 'entropy': '1.023', 'num_tokens': '3.338e+06', 'mean_token_accuracy': '0.7427', 'epoch': '0.005434'}
{'loss': '1.089', 'grad_norm': '0.4329', 'learning_rate': '0.0002', 'entropy': '1.084', 'num_tokens': '3.343e+06', 'mean_token_accuracy': '0.7406', 'epoch': '0.005443'}
{'loss': '0.8715', 'grad_norm': '0.3794', 'learning_rate': '0.0002', 'entropy': '0.8676', 'num_tokens': '3.351e+06', 'mean_token_accuracy': '0.7953', 'epoch': '0.005451'}
{'loss': '0.7808', 'grad_norm': '0.3112', 'learning_rate': '0.0002', 'entropy': '0.7746', 'num_tokens': '3.357e+06', 'mean_token_accuracy': '0.8134', 'epoch': '0.00546'}
{'loss': '0.8903', 'grad_norm': '0.3538', 'learning_rate': '0.0002', 'entropy': '0.8897', 'num_tokens': '3.364e+06', 'mean_token_accuracy': '0.7795', 'epoch': '0.005469'}
{'loss': '1.004', 'grad_norm': '0.4811', 'learning_rate': '0.0002', 'entropy': '1.021', 'num_tokens': '3.368e+06', 'mean_token_accuracy': '0.7525', 'epoch': '0.005478'}
{'loss': '0.8905', 'grad_norm': '0.4029', 'learning_rate': '0.0002', 'entropy': '0.8982', 'num_tokens': '3.372e+06', 'mean_token_accuracy': '0.7843', 'epoch': '0.005487'}
{'loss': '0.7338', 'grad_norm': '0.3756', 'learning_rate': '0.0002', 'entropy': '0.815', 'num_tokens': '3.378e+06', 'mean_token_accuracy': '0.8009', 'epoch': '0.005496'}
{'loss': '0.9036', 'grad_norm': '0.4125', 'learning_rate': '0.0002', 'entropy': '0.8821', 'num_tokens': '3.384e+06', 'mean_token_accuracy': '0.7718', 'epoch': '0.005505'}
{'loss': '1.107', 'grad_norm': '0.3859', 'learning_rate': '0.0002', 'entropy': '1.022', 'num_tokens': '3.389e+06', 'mean_token_accuracy': '0.7615', 'epoch': '0.005513'}
{'loss': '0.6301', 'grad_norm': '0.3616', 'learning_rate': '0.0002', 'entropy': '0.6908', 'num_tokens': '3.393e+06', 'mean_token_accuracy': '0.8249', 'epoch': '0.005522'}
{'loss': '0.797', 'grad_norm': '0.368', 'learning_rate': '0.0002', 'entropy': '0.8157', 'num_tokens': '3.397e+06', 'mean_token_accuracy': '0.7866', 'epoch': '0.005531'}
{'loss': '1.017', 'grad_norm': '0.3876', 'learning_rate': '0.0002', 'entropy': '1.019', 'num_tokens': '3.403e+06', 'mean_token_accuracy': '0.7434', 'epoch': '0.00554'}
{'loss': '0.9607', 'grad_norm': '0.4835', 'learning_rate': '0.0002', 'entropy': '0.933', 'num_tokens': '3.409e+06', 'mean_token_accuracy': '0.7704', 'epoch': '0.005549'}
{'loss': '0.8842', 'grad_norm': '0.3585', 'learning_rate': '0.0002', 'entropy': '0.8501', 'num_tokens': '3.415e+06', 'mean_token_accuracy': '0.7849', 'epoch': '0.005558'}
{'loss': '1.008', 'grad_norm': '0.418', 'learning_rate': '0.0002', 'entropy': '0.9625', 'num_tokens': '3.42e+06', 'mean_token_accuracy': '0.7607', 'epoch': '0.005567'}
{'loss': '0.7681', 'grad_norm': '0.3709', 'learning_rate': '0.0002', 'entropy': '0.7937', 'num_tokens': '3.426e+06', 'mean_token_accuracy': '0.8097', 'epoch': '0.005575'}
{'loss': '1.235', 'grad_norm': '0.3787', 'learning_rate': '0.0002', 'entropy': '1.306', 'num_tokens': '3.433e+06', 'mean_token_accuracy': '0.7107', 'epoch': '0.005584'}
{'loss': '0.8579', 'grad_norm': '0.3463', 'learning_rate': '0.0002', 'entropy': '0.9073', 'num_tokens': '3.439e+06', 'mean_token_accuracy': '0.7881', 'epoch': '0.005593'}
{'loss': '1.066', 'grad_norm': '0.3712', 'learning_rate': '0.0002', 'entropy': '1.06', 'num_tokens': '3.444e+06', 'mean_token_accuracy': '0.7539', 'epoch': '0.005602'}
{'loss': '1.178', 'grad_norm': '0.3685', 'learning_rate': '0.0002', 'entropy': '1.127', 'num_tokens': '3.449e+06', 'mean_token_accuracy': '0.7361', 'epoch': '0.005611'}
{'loss': '0.8292', 'grad_norm': '0.3474', 'learning_rate': '0.0002', 'entropy': '0.818', 'num_tokens': '3.454e+06', 'mean_token_accuracy': '0.7999', 'epoch': '0.00562'}
{'loss': '1.007', 'grad_norm': '0.3966', 'learning_rate': '0.0002', 'entropy': '0.9861', 'num_tokens': '3.461e+06', 'mean_token_accuracy': '0.7596', 'epoch': '0.005628'}
{'loss': '0.9727', 'grad_norm': '0.4334', 'learning_rate': '0.0002', 'entropy': '0.9023', 'num_tokens': '3.468e+06', 'mean_token_accuracy': '0.7693', 'epoch': '0.005637'}
{'loss': '0.9565', 'grad_norm': '0.4015', 'learning_rate': '0.0002', 'entropy': '0.9775', 'num_tokens': '3.473e+06', 'mean_token_accuracy': '0.7618', 'epoch': '0.005646'}
{'loss': '0.9252', 'grad_norm': '0.4159', 'learning_rate': '0.0002', 'entropy': '0.88', 'num_tokens': '3.477e+06', 'mean_token_accuracy': '0.7768', 'epoch': '0.005655'}
{'loss': '1.045', 'grad_norm': '0.3847', 'learning_rate': '0.0002', 'entropy': '1.022', 'num_tokens': '3.482e+06', 'mean_token_accuracy': '0.7551', 'epoch': '0.005664'}
{'loss': '1.027', 'grad_norm': '0.4276', 'learning_rate': '0.0002', 'entropy': '1.013', 'num_tokens': '3.486e+06', 'mean_token_accuracy': '0.7655', 'epoch': '0.005673'}
{'loss': '0.8863', 'grad_norm': '0.3595', 'learning_rate': '0.0002', 'entropy': '0.943', 'num_tokens': '3.492e+06', 'mean_token_accuracy': '0.7688', 'epoch': '0.005682'}
{'loss': '0.8749', 'grad_norm': '0.3553', 'learning_rate': '0.0002', 'entropy': '0.8895', 'num_tokens': '3.496e+06', 'mean_token_accuracy': '0.7836', 'epoch': '0.00569'}
{'loss': '1.13', 'grad_norm': '0.4689', 'learning_rate': '0.0002', 'entropy': '1.123', 'num_tokens': '3.501e+06', 'mean_token_accuracy': '0.7196', 'epoch': '0.005699'}
{'loss': '0.9402', 'grad_norm': '0.3501', 'learning_rate': '0.0002', 'entropy': '0.8294', 'num_tokens': '3.507e+06', 'mean_token_accuracy': '0.7933', 'epoch': '0.005708'}
{'loss': '0.9104', 'grad_norm': '0.3189', 'learning_rate': '0.0002', 'entropy': '1.079', 'num_tokens': '3.513e+06', 'mean_token_accuracy': '0.749', 'epoch': '0.005717'}
{'loss': '1.076', 'grad_norm': '0.3774', 'learning_rate': '0.0002', 'entropy': '1.104', 'num_tokens': '3.518e+06', 'mean_token_accuracy': '0.7305', 'epoch': '0.005726'}
{'loss': '0.8215', 'grad_norm': '0.3256', 'learning_rate': '0.0002', 'entropy': '0.8933', 'num_tokens': '3.527e+06', 'mean_token_accuracy': '0.8045', 'epoch': '0.005735'}
{'loss': '0.9588', 'grad_norm': '0.2648', 'learning_rate': '0.0002', 'entropy': '0.9484', 'num_tokens': '3.538e+06', 'mean_token_accuracy': '0.765', 'epoch': '0.005744'}
{'loss': '1.09', 'grad_norm': '0.377', 'learning_rate': '0.0002', 'entropy': '1.068', 'num_tokens': '3.543e+06', 'mean_token_accuracy': '0.7455', 'epoch': '0.005752'}
{'loss': '1.097', 'grad_norm': '0.4003', 'learning_rate': '0.0002', 'entropy': '1.141', 'num_tokens': '3.549e+06', 'mean_token_accuracy': '0.7506', 'epoch': '0.005761'}
{'loss': '0.8292', 'grad_norm': '0.3223', 'learning_rate': '0.0002', 'entropy': '0.884', 'num_tokens': '3.554e+06', 'mean_token_accuracy': '0.7908', 'epoch': '0.00577'}
{'loss': '0.8986', 'grad_norm': '0.4203', 'learning_rate': '0.0002', 'entropy': '0.9899', 'num_tokens': '3.558e+06', 'mean_token_accuracy': '0.7678', 'epoch': '0.005779'}
{'loss': '0.7576', 'grad_norm': '0.3626', 'learning_rate': '0.0002', 'entropy': '0.8081', 'num_tokens': '3.563e+06', 'mean_token_accuracy': '0.8056', 'epoch': '0.005788'}
{'loss': '1.195', 'grad_norm': '0.4299', 'learning_rate': '0.0002', 'entropy': '1.154', 'num_tokens': '3.568e+06', 'mean_token_accuracy': '0.7134', 'epoch': '0.005797'}
{'loss': '1.007', 'grad_norm': '0.4087', 'learning_rate': '0.0002', 'entropy': '1.044', 'num_tokens': '3.573e+06', 'mean_token_accuracy': '0.7459', 'epoch': '0.005805'}
{'loss': '1.034', 'grad_norm': '0.4406', 'learning_rate': '0.0002', 'entropy': '1.016', 'num_tokens': '3.578e+06', 'mean_token_accuracy': '0.7437', 'epoch': '0.005814'}
{'loss': '1.257', 'grad_norm': '0.5865', 'learning_rate': '0.0002', 'entropy': '1.218', 'num_tokens': '3.582e+06', 'mean_token_accuracy': '0.6977', 'epoch': '0.005823'}
{'loss': '0.8558', 'grad_norm': '0.4218', 'learning_rate': '0.0002', 'entropy': '0.9019', 'num_tokens': '3.588e+06', 'mean_token_accuracy': '0.7741', 'epoch': '0.005832'}
{'loss': '0.8888', 'grad_norm': '0.2978', 'learning_rate': '0.0002', 'entropy': '0.8786', 'num_tokens': '3.595e+06', 'mean_token_accuracy': '0.7881', 'epoch': '0.005841'}

  6%|▌         | 661/12000 [23:51<7:16:04,  2.31s/it]
  6%|▌         | 662/12000 [23:53<6:30:25,  2.07s/it]
                                                     

  6%|▌         | 662/12000 [23:53<6:30:25,  2.07s/it]
  6%|▌         | 663/12000 [23:54<5:59:42,  1.90s/it]
                                                     

  6%|▌         | 663/12000 [23:54<5:59:42,  1.90s/it]
  6%|▌         | 664/12000 [23:56<5:37:54,  1.79s/it]
                                                     

  6%|▌         | 664/12000 [23:56<5:37:54,  1.79s/it]
  6%|▌         | 665/12000 [24:00<8:04:57,  2.57s/it]
                                                     

  6%|▌         | 665/12000 [24:00<8:04:57,  2.57s/it]
  6%|▌         | 666/12000 [24:02<7:25:44,  2.36s/it]
                                                     

  6%|▌         | 666/12000 [24:02<7:25:44,  2.36s/it]
  6%|▌         | 667/12000 [24:04<6:54:36,  2.20s/it]
                                                     

  6%|▌         | 667/12000 [24:04<6:54:36,  2.20s/it]
  6%|▌         | 668/12000 [24:05<6:15:13,  1.99s/it]
                                                     

  6%|▌         | 668/12000 [24:05<6:15:13,  1.99s/it]
  6%|▌         | 669/12000 [24:07<6:04:51,  1.93s/it]
                                                     

  6%|▌         | 669/12000 [24:07<6:04:51,  1.93s/it]
  6%|▌         | 670/12000 [24:09<5:59:34,  1.90s/it]
                                                     

  6%|▌         | 670/12000 [24:09<5:59:34,  1.90s/it]
  6%|▌         | 671/12000 [24:10<5:31:46,  1.76s/it]
                                                     

  6%|▌         | 671/12000 [24:10<5:31:46,  1.76s/it]
  6%|▌         | 672/12000 [24:12<5:51:12,  1.86s/it]
                                                     

  6%|▌         | 672/12000 [24:12<5:51:12,  1.86s/it]
  6%|▌         | 673/12000 [24:14<6:02:22,  1.92s/it]
                                                     

  6%|▌         | 673/12000 [24:14<6:02:22,  1.92s/it]
  6%|▌         | 674/12000 [24:16<5:40:18,  1.80s/it]
                                                     

  6%|▌         | 674/12000 [24:16<5:40:18,  1.80s/it]
  6%|▌         | 675/12000 [24:18<5:42:29,  1.81s/it]
                                                     

  6%|▌         | 675/12000 [24:18<5:42:29,  1.81s/it]
  6%|▌         | 676/12000 [24:19<5:19:08,  1.69s/it]
                                                     

  6%|▌         | 676/12000 [24:19<5:19:08,  1.69s/it]
  6%|▌         | 677/12000 [24:21<5:18:26,  1.69s/it]
                                                     

  6%|▌         | 677/12000 [24:21<5:18:26,  1.69s/it]
  6%|▌         | 678/12000 [24:22<5:09:51,  1.64s/it]
                                                     

  6%|▌         | 678/12000 [24:22<5:09:51,  1.64s/it]
  6%|▌         | 679/12000 [24:24<5:23:27,  1.71s/it]
                                                     

  6%|▌         | 679/12000 [24:24<5:23:27,  1.71s/it]
  6%|▌         | 680/12000 [24:26<5:18:07,  1.69s/it]
                                                     

  6%|▌         | 680/12000 [24:26<5:18:07,  1.69s/it]
  6%|▌         | 681/12000 [24:29<6:56:52,  2.21s/it]
                                                     

  6%|▌         | 681/12000 [24:29<6:56:52,  2.21s/it]
  6%|▌         | 682/12000 [24:31<6:41:02,  2.13s/it]
                                                     

  6%|▌         | 682/12000 [24:31<6:41:02,  2.13s/it]
  6%|▌         | 683/12000 [24:34<7:30:36,  2.39s/it]
                                                     

  6%|▌         | 683/12000 [24:34<7:30:36,  2.39s/it]
  6%|▌         | 684/12000 [24:38<8:41:14,  2.76s/it]
                                                     

  6%|▌         | 684/12000 [24:38<8:41:14,  2.76s/it]
  6%|▌         | 685/12000 [24:40<7:39:57,  2.44s/it]
                                                     

  6%|▌         | 685/12000 [24:40<7:39:57,  2.44s/it]
  6%|▌         | 686/12000 [24:42<7:49:18,  2.49s/it]
                                                     

  6%|▌         | 686/12000 [24:42<7:49:18,  2.49s/it]
  6%|▌         | 687/12000 [24:44<7:07:59,  2.27s/it]
                                                     

  6%|▌         | 687/12000 [24:44<7:07:59,  2.27s/it]
  6%|▌         | 688/12000 [24:46<6:38:10,  2.11s/it]
                                                     

  6%|▌         | 688/12000 [24:46<6:38:10,  2.11s/it]
  6%|▌         | 689/12000 [24:48<6:17:38,  2.00s/it]
                                                     

  6%|▌         | 689/12000 [24:48<6:17:38,  2.00s/it]
  6%|▌         | 690/12000 [24:50<6:30:41,  2.07s/it]
                                                     

  6%|▌         | 690/12000 [24:50<6:30:41,  2.07s/it]
  6%|▌         | 691/12000 [24:51<6:03:53,  1.93s/it]
                                                     

  6%|▌         | 691/12000 [24:51<6:03:53,  1.93s/it]
  6%|▌         | 692/12000 [24:53<6:02:31,  1.92s/it]
                                                     

  6%|▌         | 692/12000 [24:53<6:02:31,  1.92s/it]
  6%|▌         | 693/12000 [24:55<5:38:09,  1.79s/it]
                                                     

  6%|▌         | 693/12000 [24:55<5:38:09,  1.79s/it]
  6%|▌         | 694/12000 [24:56<5:30:09,  1.75s/it]
                                                     

  6%|▌         | 694/12000 [24:56<5:30:09,  1.75s/it]
  6%|▌         | 695/12000 [24:58<5:17:14,  1.68s/it]
                                                     

  6%|▌         | 695/12000 [24:58<5:17:14,  1.68s/it]
  6%|▌         | 696/12000 [25:00<5:44:53,  1.83s/it]
                                                     

  6%|▌         | 696/12000 [25:00<5:44:53,  1.83s/it]
  6%|▌         | 697/12000 [25:03<7:01:21,  2.24s/it]
                                                     

  6%|▌         | 697/12000 [25:03<7:01:21,  2.24s/it]
  6%|▌         | 698/12000 [25:05<6:58:03,  2.22s/it]
                                                     

  6%|▌         | 698/12000 [25:05<6:58:03,  2.22s/it]
  6%|▌         | 699/12000 [25:07<6:19:23,  2.01s/it]
                                                     

  6%|▌         | 699/12000 [25:07<6:19:23,  2.01s/it]
  6%|▌         | 700/12000 [25:09<6:04:57,  1.94s/it]
                                                     

  6%|▌         | 700/12000 [25:09<6:04:57,  1.94s/it]
  6%|▌         | 701/12000 [25:13<7:57:07,  2.53s/it]
                                                     

  6%|▌         | 701/12000 [25:13<7:57:07,  2.53s/it]
  6%|▌         | 702/12000 [25:15<7:53:11,  2.51s/it]
                                                     

  6%|▌         | 702/12000 [25:15<7:53:11,  2.51s/it]
  6%|▌         | 703/12000 [25:19<8:53:55,  2.84s/it]
                                                     

  6%|▌         | 703/12000 [25:19<8:53:55,  2.84s/it]
  6%|▌         | 704/12000 [25:21<8:09:36,  2.60s/it]
                                                     

  6%|▌         | 704/12000 [25:21<8:09:36,  2.60s/it]
  6%|▌         | 705/12000 [25:23<8:09:13,  2.60s/it]
                                                     

  6%|▌         | 705/12000 [25:23<8:09:13,  2.60s/it]
  6%|▌         | 706/12000 [25:25<7:11:26,  2.29s/it]
                                                     

  6%|▌         | 706/12000 [25:25<7:11:26,  2.29s/it]
  6%|▌         | 707/12000 [25:26<6:23:13,  2.04s/it]
                                                     

  6%|▌         | 707/12000 [25:26<6:23:13,  2.04s/it]
  6%|▌         | 708/12000 [25:28<5:54:03,  1.88s/it]
                                                     

  6%|▌         | 708/12000 [25:28<5:54:03,  1.88s/it]
  6%|▌         | 709/12000 [25:29<5:30:14,  1.75s/it]
                                                     
{'loss': '0.836', 'grad_norm': '0.399', 'learning_rate': '0.0002', 'entropy': '0.8328', 'num_tokens': '3.6e+06', 'mean_token_accuracy': '0.7882', 'epoch': '0.00585'}
{'loss': '0.7807', 'grad_norm': '0.3541', 'learning_rate': '0.0002', 'entropy': '0.784', 'num_tokens': '3.604e+06', 'mean_token_accuracy': '0.8051', 'epoch': '0.005859'}
{'loss': '1.081', 'grad_norm': '0.4451', 'learning_rate': '0.0002', 'entropy': '1.067', 'num_tokens': '3.609e+06', 'mean_token_accuracy': '0.745', 'epoch': '0.005867'}
{'loss': '0.9244', 'grad_norm': '0.449', 'learning_rate': '0.0002', 'entropy': '0.9411', 'num_tokens': '3.612e+06', 'mean_token_accuracy': '0.7597', 'epoch': '0.005876'}
{'loss': '0.7689', 'grad_norm': '0.3207', 'learning_rate': '0.0002', 'entropy': '0.8036', 'num_tokens': '3.621e+06', 'mean_token_accuracy': '0.8112', 'epoch': '0.005885'}
{'loss': '0.9752', 'grad_norm': '0.4152', 'learning_rate': '0.0002', 'entropy': '1.018', 'num_tokens': '3.626e+06', 'mean_token_accuracy': '0.7597', 'epoch': '0.005894'}
{'loss': '1.462', 'grad_norm': '0.4369', 'learning_rate': '0.0002', 'entropy': '1.355', 'num_tokens': '3.63e+06', 'mean_token_accuracy': '0.6866', 'epoch': '0.005903'}
{'loss': '0.8989', 'grad_norm': '0.423', 'learning_rate': '0.0002', 'entropy': '0.9269', 'num_tokens': '3.634e+06', 'mean_token_accuracy': '0.7749', 'epoch': '0.005912'}
{'loss': '1.282', 'grad_norm': '0.41', 'learning_rate': '0.0002', 'entropy': '1.31', 'num_tokens': '3.639e+06', 'mean_token_accuracy': '0.6897', 'epoch': '0.005921'}
{'loss': '0.9209', 'grad_norm': '0.3532', 'learning_rate': '0.0002', 'entropy': '0.8538', 'num_tokens': '3.646e+06', 'mean_token_accuracy': '0.7838', 'epoch': '0.005929'}
{'loss': '0.8102', 'grad_norm': '0.3862', 'learning_rate': '0.0002', 'entropy': '0.7608', 'num_tokens': '3.65e+06', 'mean_token_accuracy': '0.8125', 'epoch': '0.005938'}
{'loss': '1.461', 'grad_norm': '0.3379', 'learning_rate': '0.0002', 'entropy': '1.488', 'num_tokens': '3.656e+06', 'mean_token_accuracy': '0.6464', 'epoch': '0.005947'}
{'loss': '0.81', 'grad_norm': '0.4198', 'learning_rate': '0.0002', 'entropy': '0.879', 'num_tokens': '3.661e+06', 'mean_token_accuracy': '0.777', 'epoch': '0.005956'}
{'loss': '1.068', 'grad_norm': '0.4414', 'learning_rate': '0.0002', 'entropy': '0.9985', 'num_tokens': '3.665e+06', 'mean_token_accuracy': '0.7657', 'epoch': '0.005965'}
{'loss': '1.089', 'grad_norm': '0.3426', 'learning_rate': '0.0002', 'entropy': '1.203', 'num_tokens': '3.67e+06', 'mean_token_accuracy': '0.7183', 'epoch': '0.005974'}
{'loss': '0.8922', 'grad_norm': '0.4672', 'learning_rate': '0.0002', 'entropy': '0.9075', 'num_tokens': '3.674e+06', 'mean_token_accuracy': '0.7806', 'epoch': '0.005982'}
{'loss': '0.9962', 'grad_norm': '0.3831', 'learning_rate': '0.0002', 'entropy': '1.014', 'num_tokens': '3.679e+06', 'mean_token_accuracy': '0.7626', 'epoch': '0.005991'}
{'loss': '0.9582', 'grad_norm': '0.4432', 'learning_rate': '0.0002', 'entropy': '0.9595', 'num_tokens': '3.683e+06', 'mean_token_accuracy': '0.7694', 'epoch': '0.006'}
{'loss': '0.6602', 'grad_norm': '0.2812', 'learning_rate': '0.0002', 'entropy': '0.8088', 'num_tokens': '3.689e+06', 'mean_token_accuracy': '0.8217', 'epoch': '0.006009'}
{'loss': '1.02', 'grad_norm': '0.3334', 'learning_rate': '0.0002', 'entropy': '1.126', 'num_tokens': '3.694e+06', 'mean_token_accuracy': '0.7348', 'epoch': '0.006018'}
{'loss': '0.7802', 'grad_norm': '0.4012', 'learning_rate': '0.0002', 'entropy': '0.8685', 'num_tokens': '3.697e+06', 'mean_token_accuracy': '0.7822', 'epoch': '0.006027'}
{'loss': '0.8845', 'grad_norm': '0.3724', 'learning_rate': '0.0002', 'entropy': '0.9264', 'num_tokens': '3.703e+06', 'mean_token_accuracy': '0.7619', 'epoch': '0.006036'}
{'loss': '0.9985', 'grad_norm': '0.3011', 'learning_rate': '0.0002', 'entropy': '1.051', 'num_tokens': '3.71e+06', 'mean_token_accuracy': '0.7322', 'epoch': '0.006044'}
{'loss': '0.8141', 'grad_norm': '0.2837', 'learning_rate': '0.0002', 'entropy': '0.9152', 'num_tokens': '3.718e+06', 'mean_token_accuracy': '0.764', 'epoch': '0.006053'}
{'loss': '1.229', 'grad_norm': '0.5192', 'learning_rate': '0.0002', 'entropy': '1.149', 'num_tokens': '3.722e+06', 'mean_token_accuracy': '0.7138', 'epoch': '0.006062'}
{'loss': '1.483', 'grad_norm': '0.4942', 'learning_rate': '0.0002', 'entropy': '1.311', 'num_tokens': '3.729e+06', 'mean_token_accuracy': '0.669', 'epoch': '0.006071'}
{'loss': '0.755', 'grad_norm': '0.3641', 'learning_rate': '0.0002', 'entropy': '0.7549', 'num_tokens': '3.735e+06', 'mean_token_accuracy': '0.8013', 'epoch': '0.00608'}
{'loss': '0.9588', 'grad_norm': '0.4191', 'learning_rate': '0.0002', 'entropy': '0.9705', 'num_tokens': '3.74e+06', 'mean_token_accuracy': '0.7545', 'epoch': '0.006089'}
{'loss': '1.215', 'grad_norm': '0.4332', 'learning_rate': '0.0002', 'entropy': '1.136', 'num_tokens': '3.745e+06', 'mean_token_accuracy': '0.7253', 'epoch': '0.006098'}
{'loss': '1.074', 'grad_norm': '0.418', 'learning_rate': '0.0002', 'entropy': '0.9814', 'num_tokens': '3.751e+06', 'mean_token_accuracy': '0.7683', 'epoch': '0.006106'}
{'loss': '0.8733', 'grad_norm': '0.4311', 'learning_rate': '0.0002', 'entropy': '0.9962', 'num_tokens': '3.756e+06', 'mean_token_accuracy': '0.7768', 'epoch': '0.006115'}
{'loss': '1.138', 'grad_norm': '0.4089', 'learning_rate': '0.0002', 'entropy': '1.153', 'num_tokens': '3.761e+06', 'mean_token_accuracy': '0.7292', 'epoch': '0.006124'}
{'loss': '0.9959', 'grad_norm': '0.4674', 'learning_rate': '0.0002', 'entropy': '1.055', 'num_tokens': '3.765e+06', 'mean_token_accuracy': '0.7533', 'epoch': '0.006133'}
{'loss': '1.334', 'grad_norm': '0.452', 'learning_rate': '0.0002', 'entropy': '1.3', 'num_tokens': '3.769e+06', 'mean_token_accuracy': '0.6982', 'epoch': '0.006142'}
{'loss': '0.8628', 'grad_norm': '0.3395', 'learning_rate': '0.0002', 'entropy': '0.8747', 'num_tokens': '3.774e+06', 'mean_token_accuracy': '0.779', 'epoch': '0.006151'}
{'loss': '0.8103', 'grad_norm': '0.3368', 'learning_rate': '0.0002', 'entropy': '0.8041', 'num_tokens': '3.781e+06', 'mean_token_accuracy': '0.7996', 'epoch': '0.006159'}
{'loss': '1.012', 'grad_norm': '0.3728', 'learning_rate': '0.0002', 'entropy': '0.9185', 'num_tokens': '3.788e+06', 'mean_token_accuracy': '0.7624', 'epoch': '0.006168'}
{'loss': '0.928', 'grad_norm': '0.3877', 'learning_rate': '0.0002', 'entropy': '1.004', 'num_tokens': '3.793e+06', 'mean_token_accuracy': '0.7551', 'epoch': '0.006177'}
{'loss': '0.8372', 'grad_norm': '0.5096', 'learning_rate': '0.0002', 'entropy': '0.8186', 'num_tokens': '3.797e+06', 'mean_token_accuracy': '0.7844', 'epoch': '0.006186'}
{'loss': '1.2', 'grad_norm': '0.4257', 'learning_rate': '0.0002', 'entropy': '1.136', 'num_tokens': '3.802e+06', 'mean_token_accuracy': '0.7338', 'epoch': '0.006195'}
{'loss': '1.239', 'grad_norm': '0.3441', 'learning_rate': '0.0002', 'entropy': '1.259', 'num_tokens': '3.809e+06', 'mean_token_accuracy': '0.713', 'epoch': '0.006204'}
{'loss': '0.6728', 'grad_norm': '0.312', 'learning_rate': '0.0002', 'entropy': '0.8262', 'num_tokens': '3.815e+06', 'mean_token_accuracy': '0.8115', 'epoch': '0.006213'}
{'loss': '0.9097', 'grad_norm': '0.422', 'learning_rate': '0.0002', 'entropy': '0.976', 'num_tokens': '3.819e+06', 'mean_token_accuracy': '0.7689', 'epoch': '0.006221'}
{'loss': '1.172', 'grad_norm': '0.3694', 'learning_rate': '0.0002', 'entropy': '1.197', 'num_tokens': '3.825e+06', 'mean_token_accuracy': '0.7253', 'epoch': '0.00623'}
{'loss': '0.7844', 'grad_norm': '0.4168', 'learning_rate': '0.0002', 'entropy': '0.8745', 'num_tokens': '3.831e+06', 'mean_token_accuracy': '0.7964', 'epoch': '0.006239'}
{'loss': '0.8128', 'grad_norm': '0.3333', 'learning_rate': '0.0002', 'entropy': '0.8788', 'num_tokens': '3.835e+06', 'mean_token_accuracy': '0.7832', 'epoch': '0.006248'}
{'loss': '0.6716', 'grad_norm': '0.3703', 'learning_rate': '0.0002', 'entropy': '0.7767', 'num_tokens': '3.839e+06', 'mean_token_accuracy': '0.809', 'epoch': '0.006257'}
{'loss': '0.7869', 'grad_norm': '0.4069', 'learning_rate': '0.0002', 'entropy': '0.8023', 'num_tokens': '3.843e+06', 'mean_token_accuracy': '0.7882', 'epoch': '0.006266'}

  6%|▌         | 709/12000 [25:29<5:30:14,  1.75s/it]
  6%|▌         | 710/12000 [25:31<5:21:09,  1.71s/it]
                                                     

  6%|▌         | 710/12000 [25:31<5:21:09,  1.71s/it]
  6%|▌         | 711/12000 [25:32<5:07:31,  1.63s/it]
                                                     

  6%|▌         | 711/12000 [25:32<5:07:31,  1.63s/it]
  6%|▌         | 712/12000 [25:34<5:03:08,  1.61s/it]
                                                     

  6%|▌         | 712/12000 [25:34<5:03:08,  1.61s/it]
  6%|▌         | 713/12000 [25:36<5:09:03,  1.64s/it]
                                                     

  6%|▌         | 713/12000 [25:36<5:09:03,  1.64s/it]
  6%|▌         | 714/12000 [25:37<4:56:14,  1.57s/it]
                                                     

  6%|▌         | 714/12000 [25:37<4:56:14,  1.57s/it]
  6%|▌         | 715/12000 [25:39<5:18:11,  1.69s/it]
                                                     

  6%|▌         | 715/12000 [25:39<5:18:11,  1.69s/it]
  6%|▌         | 716/12000 [25:41<5:09:31,  1.65s/it]
                                                     

  6%|▌         | 716/12000 [25:41<5:09:31,  1.65s/it]
  6%|▌         | 717/12000 [25:42<5:18:02,  1.69s/it]
                                                     

  6%|▌         | 717/12000 [25:42<5:18:02,  1.69s/it]
  6%|▌         | 718/12000 [25:44<5:07:04,  1.63s/it]
                                                     

  6%|▌         | 718/12000 [25:44<5:07:04,  1.63s/it]
  6%|▌         | 719/12000 [25:46<5:30:42,  1.76s/it]
                                                     

  6%|▌         | 719/12000 [25:46<5:30:42,  1.76s/it]
  6%|▌         | 720/12000 [25:47<5:14:16,  1.67s/it]
                                                     

  6%|▌         | 720/12000 [25:47<5:14:16,  1.67s/it]
  6%|▌         | 721/12000 [25:50<6:11:04,  1.97s/it]
                                                     

  6%|▌         | 721/12000 [25:50<6:11:04,  1.97s/it]
  6%|▌         | 722/12000 [25:52<5:46:38,  1.84s/it]
                                                     

  6%|▌         | 722/12000 [25:52<5:46:38,  1.84s/it]
  6%|▌         | 723/12000 [25:53<5:25:17,  1.73s/it]
                                                     

  6%|▌         | 723/12000 [25:53<5:25:17,  1.73s/it]
  6%|▌         | 724/12000 [25:55<5:13:59,  1.67s/it]
                                                     

  6%|▌         | 724/12000 [25:55<5:13:59,  1.67s/it]
  6%|▌         | 725/12000 [25:58<6:56:34,  2.22s/it]
                                                     

  6%|▌         | 725/12000 [25:58<6:56:34,  2.22s/it]
  6%|▌         | 726/12000 [26:00<6:46:35,  2.16s/it]
                                                     

  6%|▌         | 726/12000 [26:00<6:46:35,  2.16s/it]
  6%|▌         | 727/12000 [26:02<6:36:25,  2.11s/it]
                                                     

  6%|▌         | 727/12000 [26:02<6:36:25,  2.11s/it]
  6%|▌         | 728/12000 [26:04<6:01:38,  1.92s/it]
                                                     

  6%|▌         | 728/12000 [26:04<6:01:38,  1.92s/it]
  6%|▌         | 729/12000 [26:05<5:40:08,  1.81s/it]
                                                     

  6%|▌         | 729/12000 [26:05<5:40:08,  1.81s/it]
  6%|▌         | 730/12000 [26:07<5:33:49,  1.78s/it]
                                                     

  6%|▌         | 730/12000 [26:07<5:33:49,  1.78s/it]
  6%|▌         | 731/12000 [26:09<6:16:47,  2.01s/it]
                                                     

  6%|▌         | 731/12000 [26:09<6:16:47,  2.01s/it]
  6%|▌         | 732/12000 [26:12<6:21:49,  2.03s/it]
                                                     

  6%|▌         | 732/12000 [26:12<6:21:49,  2.03s/it]
  6%|▌         | 733/12000 [26:13<5:49:58,  1.86s/it]
                                                     

  6%|▌         | 733/12000 [26:13<5:49:58,  1.86s/it]
  6%|▌         | 734/12000 [26:15<5:38:56,  1.81s/it]
                                                     

  6%|▌         | 734/12000 [26:15<5:38:56,  1.81s/it]
  6%|▌         | 735/12000 [26:16<5:22:40,  1.72s/it]
                                                     

  6%|▌         | 735/12000 [26:16<5:22:40,  1.72s/it]
  6%|▌         | 736/12000 [26:18<5:30:23,  1.76s/it]
                                                     

  6%|▌         | 736/12000 [26:18<5:30:23,  1.76s/it]
  6%|▌         | 737/12000 [26:20<5:34:18,  1.78s/it]
                                                     

  6%|▌         | 737/12000 [26:20<5:34:18,  1.78s/it]
  6%|▌         | 738/12000 [26:21<5:15:59,  1.68s/it]
                                                     

  6%|▌         | 738/12000 [26:21<5:15:59,  1.68s/it]
  6%|▌         | 739/12000 [26:23<5:41:09,  1.82s/it]
                                                     

  6%|▌         | 739/12000 [26:23<5:41:09,  1.82s/it]
  6%|▌         | 740/12000 [26:27<7:15:25,  2.32s/it]
                                                     

  6%|▌         | 740/12000 [26:27<7:15:25,  2.32s/it]
  6%|▌         | 741/12000 [26:29<6:36:42,  2.11s/it]
                                                     

  6%|▌         | 741/12000 [26:29<6:36:42,  2.11s/it]
  6%|▌         | 742/12000 [26:31<7:13:33,  2.31s/it]
                                                     

  6%|▌         | 742/12000 [26:31<7:13:33,  2.31s/it]
  6%|▌         | 743/12000 [26:33<6:56:45,  2.22s/it]
                                                     

  6%|▌         | 743/12000 [26:33<6:56:45,  2.22s/it]
  6%|▌         | 744/12000 [26:35<6:21:59,  2.04s/it]
                                                     

  6%|▌         | 744/12000 [26:35<6:21:59,  2.04s/it]
  6%|▌         | 745/12000 [26:37<6:32:45,  2.09s/it]
                                                     

  6%|▌         | 745/12000 [26:37<6:32:45,  2.09s/it]
  6%|▌         | 746/12000 [26:39<6:09:50,  1.97s/it]
                                                     

  6%|▌         | 746/12000 [26:39<6:09:50,  1.97s/it]
  6%|▌         | 747/12000 [26:43<7:44:15,  2.48s/it]
                                                     

  6%|▌         | 747/12000 [26:43<7:44:15,  2.48s/it]
  6%|▌         | 748/12000 [26:44<6:46:44,  2.17s/it]
                                                     

  6%|▌         | 748/12000 [26:44<6:46:44,  2.17s/it]
  6%|▌         | 749/12000 [26:49<9:06:16,  2.91s/it]
                                                     

  6%|▌         | 749/12000 [26:49<9:06:16,  2.91s/it]
  6%|▋         | 750/12000 [26:51<8:09:46,  2.61s/it]
                                                     

  6%|▋         | 750/12000 [26:51<8:09:46,  2.61s/it]
  6%|▋         | 751/12000 [26:55<9:40:45,  3.10s/it]
                                                     

  6%|▋         | 751/12000 [26:55<9:40:45,  3.10s/it]
  6%|▋         | 752/12000 [26:57<8:52:21,  2.84s/it]
                                                     

  6%|▋         | 752/12000 [26:57<8:52:21,  2.84s/it]
  6%|▋         | 753/12000 [26:59<7:34:42,  2.43s/it]
                                                     

  6%|▋         | 753/12000 [26:59<7:34:42,  2.43s/it]
  6%|▋         | 754/12000 [27:00<7:06:26,  2.28s/it]
                                                     

  6%|▋         | 754/12000 [27:00<7:06:26,  2.28s/it]
  6%|▋         | 755/12000 [27:02<6:29:27,  2.08s/it]
                                                     

  6%|▋         | 755/12000 [27:02<6:29:27,  2.08s/it]
  6%|▋         | 756/12000 [27:04<6:05:09,  1.95s/it]
                                                     
{'loss': '0.8288', 'grad_norm': '0.325', 'learning_rate': '0.0002', 'entropy': '0.8331', 'num_tokens': '3.848e+06', 'mean_token_accuracy': '0.7927', 'epoch': '0.006275'}
{'loss': '0.8685', 'grad_norm': '0.3841', 'learning_rate': '0.0002', 'entropy': '0.8707', 'num_tokens': '3.853e+06', 'mean_token_accuracy': '0.7734', 'epoch': '0.006283'}
{'loss': '0.8788', 'grad_norm': '0.4049', 'learning_rate': '0.0002', 'entropy': '0.9785', 'num_tokens': '3.857e+06', 'mean_token_accuracy': '0.7535', 'epoch': '0.006292'}
{'loss': '0.9888', 'grad_norm': '0.4849', 'learning_rate': '0.0002', 'entropy': '0.975', 'num_tokens': '3.861e+06', 'mean_token_accuracy': '0.7544', 'epoch': '0.006301'}
{'loss': '1.112', 'grad_norm': '0.7507', 'learning_rate': '0.0002', 'entropy': '1.095', 'num_tokens': '3.865e+06', 'mean_token_accuracy': '0.7336', 'epoch': '0.00631'}
{'loss': '0.7888', 'grad_norm': '0.4161', 'learning_rate': '0.0002', 'entropy': '0.7842', 'num_tokens': '3.869e+06', 'mean_token_accuracy': '0.7897', 'epoch': '0.006319'}
{'loss': '1.07', 'grad_norm': '0.3891', 'learning_rate': '0.0002', 'entropy': '1.04', 'num_tokens': '3.875e+06', 'mean_token_accuracy': '0.7442', 'epoch': '0.006328'}
{'loss': '0.9201', 'grad_norm': '0.4565', 'learning_rate': '0.0001999', 'entropy': '0.9891', 'num_tokens': '3.88e+06', 'mean_token_accuracy': '0.7731', 'epoch': '0.006336'}
{'loss': '1.441', 'grad_norm': '0.4351', 'learning_rate': '0.0001999', 'entropy': '1.421', 'num_tokens': '3.885e+06', 'mean_token_accuracy': '0.6719', 'epoch': '0.006345'}
{'loss': '0.6691', 'grad_norm': '0.3091', 'learning_rate': '0.0001999', 'entropy': '0.7231', 'num_tokens': '3.89e+06', 'mean_token_accuracy': '0.8278', 'epoch': '0.006354'}
{'loss': '0.8415', 'grad_norm': '0.3886', 'learning_rate': '0.0001999', 'entropy': '0.9073', 'num_tokens': '3.895e+06', 'mean_token_accuracy': '0.7842', 'epoch': '0.006363'}
{'loss': '0.9042', 'grad_norm': '0.3765', 'learning_rate': '0.0001999', 'entropy': '0.929', 'num_tokens': '3.9e+06', 'mean_token_accuracy': '0.7725', 'epoch': '0.006372'}
{'loss': '0.8818', 'grad_norm': '0.3313', 'learning_rate': '0.0001999', 'entropy': '0.9114', 'num_tokens': '3.907e+06', 'mean_token_accuracy': '0.7854', 'epoch': '0.006381'}
{'loss': '0.7792', 'grad_norm': '0.3485', 'learning_rate': '0.0001999', 'entropy': '0.8207', 'num_tokens': '3.912e+06', 'mean_token_accuracy': '0.8061', 'epoch': '0.00639'}
{'loss': '1.32', 'grad_norm': '0.4887', 'learning_rate': '0.0001999', 'entropy': '1.27', 'num_tokens': '3.916e+06', 'mean_token_accuracy': '0.7002', 'epoch': '0.006398'}
{'loss': '0.7018', 'grad_norm': '0.3769', 'learning_rate': '0.0001999', 'entropy': '0.686', 'num_tokens': '3.92e+06', 'mean_token_accuracy': '0.8088', 'epoch': '0.006407'}
{'loss': '1.052', 'grad_norm': '0.3887', 'learning_rate': '0.0001999', 'entropy': '1.013', 'num_tokens': '3.925e+06', 'mean_token_accuracy': '0.7392', 'epoch': '0.006416'}
{'loss': '1.251', 'grad_norm': '0.3513', 'learning_rate': '0.0001999', 'entropy': '1.12', 'num_tokens': '3.931e+06', 'mean_token_accuracy': '0.7298', 'epoch': '0.006425'}
{'loss': '1.451', 'grad_norm': '0.4204', 'learning_rate': '0.0001999', 'entropy': '1.254', 'num_tokens': '3.936e+06', 'mean_token_accuracy': '0.6943', 'epoch': '0.006434'}
{'loss': '0.7938', 'grad_norm': '0.3221', 'learning_rate': '0.0001999', 'entropy': '0.8351', 'num_tokens': '3.941e+06', 'mean_token_accuracy': '0.7881', 'epoch': '0.006443'}
{'loss': '0.914', 'grad_norm': '0.4163', 'learning_rate': '0.0001999', 'entropy': '0.9033', 'num_tokens': '3.945e+06', 'mean_token_accuracy': '0.7802', 'epoch': '0.006452'}
{'loss': '1.025', 'grad_norm': '0.3632', 'learning_rate': '0.0001999', 'entropy': '1.013', 'num_tokens': '3.95e+06', 'mean_token_accuracy': '0.7659', 'epoch': '0.00646'}
{'loss': '1.074', 'grad_norm': '0.3406', 'learning_rate': '0.0001999', 'entropy': '1.115', 'num_tokens': '3.957e+06', 'mean_token_accuracy': '0.7378', 'epoch': '0.006469'}
{'loss': '0.8502', 'grad_norm': '0.3525', 'learning_rate': '0.0001999', 'entropy': '0.8339', 'num_tokens': '3.964e+06', 'mean_token_accuracy': '0.7999', 'epoch': '0.006478'}
{'loss': '1.089', 'grad_norm': '0.4706', 'learning_rate': '0.0001999', 'entropy': '1.161', 'num_tokens': '3.966e+06', 'mean_token_accuracy': '0.7226', 'epoch': '0.006487'}
{'loss': '0.7337', 'grad_norm': '0.3837', 'learning_rate': '0.0001999', 'entropy': '0.7349', 'num_tokens': '3.971e+06', 'mean_token_accuracy': '0.811', 'epoch': '0.006496'}
{'loss': '0.8375', 'grad_norm': '0.3755', 'learning_rate': '0.0001999', 'entropy': '0.9013', 'num_tokens': '3.975e+06', 'mean_token_accuracy': '0.7745', 'epoch': '0.006505'}
{'loss': '0.8492', 'grad_norm': '0.3998', 'learning_rate': '0.0001999', 'entropy': '0.8473', 'num_tokens': '3.98e+06', 'mean_token_accuracy': '0.7984', 'epoch': '0.006513'}
{'loss': '1.104', 'grad_norm': '0.4521', 'learning_rate': '0.0001999', 'entropy': '1.145', 'num_tokens': '3.985e+06', 'mean_token_accuracy': '0.7093', 'epoch': '0.006522'}
{'loss': '0.7821', 'grad_norm': '0.4367', 'learning_rate': '0.0001999', 'entropy': '0.8327', 'num_tokens': '3.989e+06', 'mean_token_accuracy': '0.7913', 'epoch': '0.006531'}
{'loss': '0.9749', 'grad_norm': '0.3858', 'learning_rate': '0.0001999', 'entropy': '0.9786', 'num_tokens': '3.995e+06', 'mean_token_accuracy': '0.7655', 'epoch': '0.00654'}
{'loss': '1.115', 'grad_norm': '0.3819', 'learning_rate': '0.0001999', 'entropy': '1.139', 'num_tokens': '4.002e+06', 'mean_token_accuracy': '0.728', 'epoch': '0.006549'}
{'loss': '1.28', 'grad_norm': '0.3962', 'learning_rate': '0.0001999', 'entropy': '1.221', 'num_tokens': '4.006e+06', 'mean_token_accuracy': '0.6913', 'epoch': '0.006558'}
{'loss': '0.7014', 'grad_norm': '0.3954', 'learning_rate': '0.0001999', 'entropy': '0.6532', 'num_tokens': '4.013e+06', 'mean_token_accuracy': '0.8278', 'epoch': '0.006567'}
{'loss': '1.047', 'grad_norm': '0.3461', 'learning_rate': '0.0001999', 'entropy': '0.9546', 'num_tokens': '4.018e+06', 'mean_token_accuracy': '0.7601', 'epoch': '0.006575'}
{'loss': '0.8489', 'grad_norm': '0.3471', 'learning_rate': '0.0001999', 'entropy': '0.8769', 'num_tokens': '4.023e+06', 'mean_token_accuracy': '0.7805', 'epoch': '0.006584'}
{'loss': '1.449', 'grad_norm': '0.4442', 'learning_rate': '0.0001999', 'entropy': '1.199', 'num_tokens': '4.029e+06', 'mean_token_accuracy': '0.695', 'epoch': '0.006593'}
{'loss': '0.825', 'grad_norm': '0.4166', 'learning_rate': '0.0001999', 'entropy': '0.7673', 'num_tokens': '4.033e+06', 'mean_token_accuracy': '0.8042', 'epoch': '0.006602'}
{'loss': '0.867', 'grad_norm': '0.3579', 'learning_rate': '0.0001999', 'entropy': '0.8795', 'num_tokens': '4.038e+06', 'mean_token_accuracy': '0.7894', 'epoch': '0.006611'}
{'loss': '0.861', 'grad_norm': '0.3259', 'learning_rate': '0.0001999', 'entropy': '0.9306', 'num_tokens': '4.043e+06', 'mean_token_accuracy': '0.7785', 'epoch': '0.00662'}
{'loss': '0.8924', 'grad_norm': '0.2674', 'learning_rate': '0.0001999', 'entropy': '0.9423', 'num_tokens': '4.05e+06', 'mean_token_accuracy': '0.7734', 'epoch': '0.006629'}
{'loss': '0.9506', 'grad_norm': '0.4175', 'learning_rate': '0.0001999', 'entropy': '1.005', 'num_tokens': '4.055e+06', 'mean_token_accuracy': '0.7569', 'epoch': '0.006637'}
{'loss': '0.8828', 'grad_norm': '0.45', 'learning_rate': '0.0001999', 'entropy': '0.9504', 'num_tokens': '4.061e+06', 'mean_token_accuracy': '0.7608', 'epoch': '0.006646'}
{'loss': '1.088', 'grad_norm': '0.302', 'learning_rate': '0.0001999', 'entropy': '1.052', 'num_tokens': '4.068e+06', 'mean_token_accuracy': '0.7576', 'epoch': '0.006655'}
{'loss': '0.9534', 'grad_norm': '0.3799', 'learning_rate': '0.0001999', 'entropy': '1.04', 'num_tokens': '4.071e+06', 'mean_token_accuracy': '0.7408', 'epoch': '0.006664'}
{'loss': '1.091', 'grad_norm': '0.3555', 'learning_rate': '0.0001999', 'entropy': '1.046', 'num_tokens': '4.078e+06', 'mean_token_accuracy': '0.7529', 'epoch': '0.006673'}
{'loss': '0.9333', 'grad_norm': '0.3922', 'learning_rate': '0.0001999', 'entropy': '0.991', 'num_tokens': '4.082e+06', 'mean_token_accuracy': '0.7577', 'epoch': '0.006682'}

  6%|▋         | 756/12000 [27:04<6:05:09,  1.95s/it]
  6%|▋         | 757/12000 [27:05<5:57:05,  1.91s/it]
                                                     

  6%|▋         | 757/12000 [27:05<5:57:05,  1.91s/it]
  6%|▋         | 758/12000 [27:07<5:37:29,  1.80s/it]
                                                     

  6%|▋         | 758/12000 [27:07<5:37:29,  1.80s/it]
  6%|▋         | 759/12000 [27:09<5:53:06,  1.88s/it]
                                                     

  6%|▋         | 759/12000 [27:09<5:53:06,  1.88s/it]
  6%|▋         | 760/12000 [27:12<6:50:01,  2.19s/it]
                                                     

  6%|▋         | 760/12000 [27:12<6:50:01,  2.19s/it]
  6%|▋         | 761/12000 [27:14<6:11:43,  1.98s/it]
                                                     

  6%|▋         | 761/12000 [27:14<6:11:43,  1.98s/it]
  6%|▋         | 762/12000 [27:15<5:48:05,  1.86s/it]
                                                     

  6%|▋         | 762/12000 [27:15<5:48:05,  1.86s/it]
  6%|▋         | 763/12000 [27:17<5:29:07,  1.76s/it]
                                                     

  6%|▋         | 763/12000 [27:17<5:29:07,  1.76s/it]
  6%|▋         | 764/12000 [27:20<6:51:24,  2.20s/it]
                                                     

  6%|▋         | 764/12000 [27:20<6:51:24,  2.20s/it]
  6%|▋         | 765/12000 [27:22<6:23:44,  2.05s/it]
                                                     

  6%|▋         | 765/12000 [27:22<6:23:44,  2.05s/it]
  6%|▋         | 766/12000 [27:23<6:12:33,  1.99s/it]
                                                     

  6%|▋         | 766/12000 [27:23<6:12:33,  1.99s/it]
  6%|▋         | 767/12000 [27:25<5:58:36,  1.92s/it]
                                                     

  6%|▋         | 767/12000 [27:25<5:58:36,  1.92s/it]
  6%|▋         | 768/12000 [27:27<6:02:45,  1.94s/it]
                                                     

  6%|▋         | 768/12000 [27:27<6:02:45,  1.94s/it]
  6%|▋         | 769/12000 [27:31<7:34:29,  2.43s/it]
                                                     

  6%|▋         | 769/12000 [27:31<7:34:29,  2.43s/it]
  6%|▋         | 770/12000 [27:32<6:35:10,  2.11s/it]
                                                     

  6%|▋         | 770/12000 [27:32<6:35:10,  2.11s/it]
  6%|▋         | 771/12000 [27:37<9:12:51,  2.95s/it]
                                                     

  6%|▋         | 771/12000 [27:37<9:12:51,  2.95s/it]
  6%|▋         | 772/12000 [27:38<7:50:29,  2.51s/it]
                                                     

  6%|▋         | 772/12000 [27:38<7:50:29,  2.51s/it]
  6%|▋         | 773/12000 [27:41<8:06:17,  2.60s/it]
                                                     

  6%|▋         | 773/12000 [27:41<8:06:17,  2.60s/it]
  6%|▋         | 774/12000 [27:43<7:32:30,  2.42s/it]
                                                     

  6%|▋         | 774/12000 [27:43<7:32:30,  2.42s/it]
  6%|▋         | 775/12000 [27:45<6:44:26,  2.16s/it]
                                                     

  6%|▋         | 775/12000 [27:45<6:44:26,  2.16s/it]
  6%|▋         | 776/12000 [27:46<6:11:17,  1.98s/it]
                                                     

  6%|▋         | 776/12000 [27:46<6:11:17,  1.98s/it]
  6%|▋         | 777/12000 [27:49<6:17:00,  2.02s/it]
                                                     

  6%|▋         | 777/12000 [27:49<6:17:00,  2.02s/it]
  6%|▋         | 778/12000 [27:51<6:31:26,  2.09s/it]
                                                     

  6%|▋         | 778/12000 [27:51<6:31:26,  2.09s/it]
  6%|▋         | 779/12000 [27:53<6:35:34,  2.12s/it]
                                                     

  6%|▋         | 779/12000 [27:53<6:35:34,  2.12s/it]
  6%|▋         | 780/12000 [27:55<6:04:19,  1.95s/it]
                                                     

  6%|▋         | 780/12000 [27:55<6:04:19,  1.95s/it]
  7%|▋         | 781/12000 [27:57<6:45:19,  2.17s/it]
                                                     

  7%|▋         | 781/12000 [27:57<6:45:19,  2.17s/it]
  7%|▋         | 782/12000 [27:59<6:07:43,  1.97s/it]
                                                     

  7%|▋         | 782/12000 [27:59<6:07:43,  1.97s/it]
  7%|▋         | 783/12000 [28:01<6:16:30,  2.01s/it]
                                                     

  7%|▋         | 783/12000 [28:01<6:16:30,  2.01s/it]
  7%|▋         | 784/12000 [28:02<5:56:30,  1.91s/it]
                                                     

  7%|▋         | 784/12000 [28:02<5:56:30,  1.91s/it]
  7%|▋         | 785/12000 [28:05<6:33:28,  2.11s/it]
                                                     

  7%|▋         | 785/12000 [28:05<6:33:28,  2.11s/it]
  7%|▋         | 786/12000 [28:07<5:58:10,  1.92s/it]
                                                     

  7%|▋         | 786/12000 [28:07<5:58:10,  1.92s/it]
  7%|▋         | 787/12000 [28:09<6:32:04,  2.10s/it]
                                                     

  7%|▋         | 787/12000 [28:09<6:32:04,  2.10s/it]
  7%|▋         | 788/12000 [28:12<7:02:27,  2.26s/it]
                                                     

  7%|▋         | 788/12000 [28:12<7:02:27,  2.26s/it]
  7%|▋         | 789/12000 [28:16<9:02:25,  2.90s/it]
                                                     

  7%|▋         | 789/12000 [28:16<9:02:25,  2.90s/it]
  7%|▋         | 790/12000 [28:18<8:01:16,  2.58s/it]
                                                     

  7%|▋         | 790/12000 [28:18<8:01:16,  2.58s/it]
  7%|▋         | 791/12000 [28:21<8:52:09,  2.85s/it]
                                                     

  7%|▋         | 791/12000 [28:21<8:52:09,  2.85s/it]
  7%|▋         | 792/12000 [28:23<7:59:39,  2.57s/it]
                                                     

  7%|▋         | 792/12000 [28:23<7:59:39,  2.57s/it]
  7%|▋         | 793/12000 [28:25<7:23:29,  2.37s/it]
                                                     

  7%|▋         | 793/12000 [28:25<7:23:29,  2.37s/it]
  7%|▋         | 794/12000 [28:27<6:40:00,  2.14s/it]
                                                     

  7%|▋         | 794/12000 [28:27<6:40:00,  2.14s/it]
  7%|▋         | 795/12000 [28:29<6:16:33,  2.02s/it]
                                                     

  7%|▋         | 795/12000 [28:29<6:16:33,  2.02s/it]
  7%|▋         | 796/12000 [28:32<7:10:10,  2.30s/it]
                                                     

  7%|▋         | 796/12000 [28:32<7:10:10,  2.30s/it]
  7%|▋         | 797/12000 [28:33<6:29:46,  2.09s/it]
                                                     

  7%|▋         | 797/12000 [28:33<6:29:46,  2.09s/it]
  7%|▋         | 798/12000 [28:35<5:53:37,  1.89s/it]
                                                     

  7%|▋         | 798/12000 [28:35<5:53:37,  1.89s/it]
  7%|▋         | 799/12000 [28:36<5:48:26,  1.87s/it]
                                                     

  7%|▋         | 799/12000 [28:36<5:48:26,  1.87s/it]
  7%|▋         | 800/12000 [28:38<5:42:20,  1.83s/it]
                                                     

  7%|▋         | 800/12000 [28:38<5:42:20,  1.83s/it]
  7%|▋         | 801/12000 [28:42<7:11:39,  2.31s/it]
                                                     

  7%|▋         | 801/12000 [28:42<7:11:39,  2.31s/it]
  7%|▋         | 802/12000 [28:43<6:46:13,  2.18s/it]
                                                     

  7%|▋         | 802/12000 [28:43<6:46:13,  2.18s/it]
  7%|▋         | 803/12000 [28:45<6:02:56,  1.94s/it]
                                                     
{'loss': '0.6754', 'grad_norm': '0.3744', 'learning_rate': '0.0001999', 'entropy': '0.7822', 'num_tokens': '4.087e+06', 'mean_token_accuracy': '0.8141', 'epoch': '0.00669'}
{'loss': '1.574', 'grad_norm': '0.3757', 'learning_rate': '0.0001999', 'entropy': '1.418', 'num_tokens': '4.092e+06', 'mean_token_accuracy': '0.6736', 'epoch': '0.006699'}
{'loss': '0.748', 'grad_norm': '0.3457', 'learning_rate': '0.0001999', 'entropy': '0.7753', 'num_tokens': '4.097e+06', 'mean_token_accuracy': '0.7931', 'epoch': '0.006708'}
{'loss': '1.322', 'grad_norm': '0.4129', 'learning_rate': '0.0001999', 'entropy': '1.17', 'num_tokens': '4.102e+06', 'mean_token_accuracy': '0.7223', 'epoch': '0.006717'}
{'loss': '1.539', 'grad_norm': '0.4134', 'learning_rate': '0.0001999', 'entropy': '1.453', 'num_tokens': '4.11e+06', 'mean_token_accuracy': '0.6639', 'epoch': '0.006726'}
{'loss': '0.8881', 'grad_norm': '0.3894', 'learning_rate': '0.0001999', 'entropy': '0.8758', 'num_tokens': '4.114e+06', 'mean_token_accuracy': '0.789', 'epoch': '0.006735'}
{'loss': '0.7968', 'grad_norm': '0.3082', 'learning_rate': '0.0001999', 'entropy': '0.8314', 'num_tokens': '4.119e+06', 'mean_token_accuracy': '0.7985', 'epoch': '0.006744'}
{'loss': '0.7846', 'grad_norm': '0.5051', 'learning_rate': '0.0001999', 'entropy': '0.8485', 'num_tokens': '4.123e+06', 'mean_token_accuracy': '0.8021', 'epoch': '0.006752'}
{'loss': '0.9705', 'grad_norm': '0.3116', 'learning_rate': '0.0001999', 'entropy': '1.069', 'num_tokens': '4.131e+06', 'mean_token_accuracy': '0.7596', 'epoch': '0.006761'}
{'loss': '1.051', 'grad_norm': '0.37', 'learning_rate': '0.0001999', 'entropy': '1.039', 'num_tokens': '4.137e+06', 'mean_token_accuracy': '0.7496', 'epoch': '0.00677'}
{'loss': '0.9394', 'grad_norm': '0.4797', 'learning_rate': '0.0001999', 'entropy': '0.9631', 'num_tokens': '4.142e+06', 'mean_token_accuracy': '0.7689', 'epoch': '0.006779'}
{'loss': '1.203', 'grad_norm': '0.4432', 'learning_rate': '0.0001999', 'entropy': '1.261', 'num_tokens': '4.147e+06', 'mean_token_accuracy': '0.7108', 'epoch': '0.006788'}
{'loss': '1.011', 'grad_norm': '0.3309', 'learning_rate': '0.0001999', 'entropy': '0.9756', 'num_tokens': '4.152e+06', 'mean_token_accuracy': '0.7461', 'epoch': '0.006797'}
{'loss': '0.9897', 'grad_norm': '0.4376', 'learning_rate': '0.0001999', 'entropy': '0.9562', 'num_tokens': '4.156e+06', 'mean_token_accuracy': '0.7753', 'epoch': '0.006806'}
{'loss': '0.907', 'grad_norm': '0.3521', 'learning_rate': '0.0001999', 'entropy': '0.9461', 'num_tokens': '4.161e+06', 'mean_token_accuracy': '0.7563', 'epoch': '0.006814'}
{'loss': '2.359', 'grad_norm': '0.4322', 'learning_rate': '0.0001999', 'entropy': '1.393', 'num_tokens': '4.17e+06', 'mean_token_accuracy': '0.6995', 'epoch': '0.006823'}
{'loss': '1.024', 'grad_norm': '0.3286', 'learning_rate': '0.0001999', 'entropy': '1.026', 'num_tokens': '4.175e+06', 'mean_token_accuracy': '0.7467', 'epoch': '0.006832'}
{'loss': '1.517', 'grad_norm': '0.4021', 'learning_rate': '0.0001999', 'entropy': '1.439', 'num_tokens': '4.18e+06', 'mean_token_accuracy': '0.6913', 'epoch': '0.006841'}
{'loss': '1.037', 'grad_norm': '0.4017', 'learning_rate': '0.0001999', 'entropy': '1.108', 'num_tokens': '4.185e+06', 'mean_token_accuracy': '0.7359', 'epoch': '0.00685'}
{'loss': '0.8835', 'grad_norm': '0.4205', 'learning_rate': '0.0001999', 'entropy': '0.9108', 'num_tokens': '4.189e+06', 'mean_token_accuracy': '0.7854', 'epoch': '0.006859'}
{'loss': '0.9591', 'grad_norm': '0.4394', 'learning_rate': '0.0001999', 'entropy': '1.001', 'num_tokens': '4.193e+06', 'mean_token_accuracy': '0.7732', 'epoch': '0.006867'}
{'loss': '0.8893', 'grad_norm': '0.3486', 'learning_rate': '0.0001999', 'entropy': '0.8722', 'num_tokens': '4.199e+06', 'mean_token_accuracy': '0.7954', 'epoch': '0.006876'}
{'loss': '0.9099', 'grad_norm': '0.328', 'learning_rate': '0.0001999', 'entropy': '0.8694', 'num_tokens': '4.205e+06', 'mean_token_accuracy': '0.7941', 'epoch': '0.006885'}
{'loss': '0.7416', 'grad_norm': '0.3626', 'learning_rate': '0.0001999', 'entropy': '0.8204', 'num_tokens': '4.211e+06', 'mean_token_accuracy': '0.7954', 'epoch': '0.006894'}
{'loss': '0.9731', 'grad_norm': '0.4566', 'learning_rate': '0.0001999', 'entropy': '0.9466', 'num_tokens': '4.216e+06', 'mean_token_accuracy': '0.7587', 'epoch': '0.006903'}
{'loss': '1.125', 'grad_norm': '0.3702', 'learning_rate': '0.0001999', 'entropy': '1.101', 'num_tokens': '4.222e+06', 'mean_token_accuracy': '0.728', 'epoch': '0.006912'}
{'loss': '0.9507', 'grad_norm': '0.347', 'learning_rate': '0.0001999', 'entropy': '0.923', 'num_tokens': '4.227e+06', 'mean_token_accuracy': '0.775', 'epoch': '0.006921'}
{'loss': '0.9055', 'grad_norm': '0.3291', 'learning_rate': '0.0001999', 'entropy': '0.8735', 'num_tokens': '4.233e+06', 'mean_token_accuracy': '0.79', 'epoch': '0.006929'}
{'loss': '1.081', 'grad_norm': '0.4797', 'learning_rate': '0.0001999', 'entropy': '1.089', 'num_tokens': '4.237e+06', 'mean_token_accuracy': '0.7235', 'epoch': '0.006938'}
{'loss': '0.9856', 'grad_norm': '0.3413', 'learning_rate': '0.0001999', 'entropy': '0.9333', 'num_tokens': '4.243e+06', 'mean_token_accuracy': '0.7687', 'epoch': '0.006947'}
{'loss': '0.7808', 'grad_norm': '0.3883', 'learning_rate': '0.0001999', 'entropy': '0.8462', 'num_tokens': '4.247e+06', 'mean_token_accuracy': '0.7985', 'epoch': '0.006956'}
{'loss': '1.02', 'grad_norm': '0.3706', 'learning_rate': '0.0001999', 'entropy': '1.004', 'num_tokens': '4.252e+06', 'mean_token_accuracy': '0.7573', 'epoch': '0.006965'}
{'loss': '0.7377', 'grad_norm': '0.2782', 'learning_rate': '0.0001999', 'entropy': '0.7926', 'num_tokens': '4.26e+06', 'mean_token_accuracy': '0.8099', 'epoch': '0.006974'}
{'loss': '0.6537', 'grad_norm': '0.282', 'learning_rate': '0.0001999', 'entropy': '0.7157', 'num_tokens': '4.269e+06', 'mean_token_accuracy': '0.8164', 'epoch': '0.006983'}
{'loss': '1.151', 'grad_norm': '0.3066', 'learning_rate': '0.0001999', 'entropy': '1.098', 'num_tokens': '4.275e+06', 'mean_token_accuracy': '0.7194', 'epoch': '0.006991'}
{'loss': '1.265', 'grad_norm': '0.3929', 'learning_rate': '0.0001999', 'entropy': '1.2', 'num_tokens': '4.279e+06', 'mean_token_accuracy': '0.7014', 'epoch': '0.007'}
{'loss': '0.9789', 'grad_norm': '0.4315', 'learning_rate': '0.0001999', 'entropy': '0.9429', 'num_tokens': '4.284e+06', 'mean_token_accuracy': '0.767', 'epoch': '0.007009'}
{'loss': '0.9777', 'grad_norm': '0.4433', 'learning_rate': '0.0001999', 'entropy': '0.9355', 'num_tokens': '4.289e+06', 'mean_token_accuracy': '0.752', 'epoch': '0.007018'}
{'loss': '0.8613', 'grad_norm': '0.4037', 'learning_rate': '0.0001999', 'entropy': '0.8811', 'num_tokens': '4.293e+06', 'mean_token_accuracy': '0.7732', 'epoch': '0.007027'}
{'loss': '0.8649', 'grad_norm': '0.3789', 'learning_rate': '0.0001999', 'entropy': '0.8858', 'num_tokens': '4.299e+06', 'mean_token_accuracy': '0.7776', 'epoch': '0.007036'}
{'loss': '0.9117', 'grad_norm': '0.3871', 'learning_rate': '0.0001999', 'entropy': '0.9332', 'num_tokens': '4.306e+06', 'mean_token_accuracy': '0.7727', 'epoch': '0.007044'}
{'loss': '0.7698', 'grad_norm': '0.3967', 'learning_rate': '0.0001999', 'entropy': '0.8239', 'num_tokens': '4.309e+06', 'mean_token_accuracy': '0.812', 'epoch': '0.007053'}
{'loss': '0.8375', 'grad_norm': '0.3707', 'learning_rate': '0.0001999', 'entropy': '0.8576', 'num_tokens': '4.314e+06', 'mean_token_accuracy': '0.7973', 'epoch': '0.007062'}
{'loss': '0.7134', 'grad_norm': '0.3464', 'learning_rate': '0.0001999', 'entropy': '0.7563', 'num_tokens': '4.319e+06', 'mean_token_accuracy': '0.8215', 'epoch': '0.007071'}
{'loss': '1.108', 'grad_norm': '0.3511', 'learning_rate': '0.0001998', 'entropy': '1.161', 'num_tokens': '4.324e+06', 'mean_token_accuracy': '0.7425', 'epoch': '0.00708'}
{'loss': '0.7323', 'grad_norm': '0.3712', 'learning_rate': '0.0001998', 'entropy': '0.7517', 'num_tokens': '4.331e+06', 'mean_token_accuracy': '0.8064', 'epoch': '0.007089'}
{'loss': '1.009', 'grad_norm': '0.3954', 'learning_rate': '0.0001998', 'entropy': '1.055', 'num_tokens': '4.336e+06', 'mean_token_accuracy': '0.7469', 'epoch': '0.007098'}

  7%|▋         | 803/12000 [28:45<6:02:56,  1.94s/it]
  7%|▋         | 804/12000 [28:47<6:12:35,  2.00s/it]
                                                     

  7%|▋         | 804/12000 [28:47<6:12:35,  2.00s/it]
  7%|▋         | 805/12000 [28:48<5:42:44,  1.84s/it]
                                                     

  7%|▋         | 805/12000 [28:48<5:42:44,  1.84s/it]
  7%|▋         | 806/12000 [28:50<5:32:38,  1.78s/it]
                                                     

  7%|▋         | 806/12000 [28:50<5:32:38,  1.78s/it]
  7%|▋         | 807/12000 [28:53<6:13:33,  2.00s/it]
                                                     

  7%|▋         | 807/12000 [28:53<6:13:33,  2.00s/it]
  7%|▋         | 808/12000 [28:55<6:36:22,  2.12s/it]
                                                     

  7%|▋         | 808/12000 [28:55<6:36:22,  2.12s/it]
  7%|▋         | 809/12000 [28:57<6:13:46,  2.00s/it]
                                                     

  7%|▋         | 809/12000 [28:57<6:13:46,  2.00s/it]
  7%|▋         | 810/12000 [28:59<6:29:54,  2.09s/it]
                                                     

  7%|▋         | 810/12000 [28:59<6:29:54,  2.09s/it]
  7%|▋         | 811/12000 [29:02<6:59:49,  2.25s/it]
                                                     

  7%|▋         | 811/12000 [29:02<6:59:49,  2.25s/it]
  7%|▋         | 812/12000 [29:07<9:36:01,  3.09s/it]
                                                     

  7%|▋         | 812/12000 [29:07<9:36:01,  3.09s/it]
  7%|▋         | 813/12000 [29:10<9:55:05,  3.19s/it]
                                                     

  7%|▋         | 813/12000 [29:10<9:55:05,  3.19s/it]
  7%|▋         | 814/12000 [29:11<8:11:14,  2.63s/it]
                                                     

  7%|▋         | 814/12000 [29:11<8:11:14,  2.63s/it]
  7%|▋         | 815/12000 [29:13<7:11:22,  2.31s/it]
                                                     

  7%|▋         | 815/12000 [29:13<7:11:22,  2.31s/it]
  7%|▋         | 816/12000 [29:15<6:27:57,  2.08s/it]
                                                     

  7%|▋         | 816/12000 [29:15<6:27:57,  2.08s/it]
  7%|▋         | 817/12000 [29:16<5:52:57,  1.89s/it]
                                                     

  7%|▋         | 817/12000 [29:16<5:52:57,  1.89s/it]
  7%|▋         | 818/12000 [29:18<5:43:23,  1.84s/it]
                                                     

  7%|▋         | 818/12000 [29:18<5:43:23,  1.84s/it]
  7%|▋         | 819/12000 [29:19<5:26:45,  1.75s/it]
                                                     

  7%|▋         | 819/12000 [29:19<5:26:45,  1.75s/it]
  7%|▋         | 820/12000 [29:21<5:06:32,  1.65s/it]
                                                     

  7%|▋         | 820/12000 [29:21<5:06:32,  1.65s/it]
  7%|▋         | 821/12000 [29:23<5:34:10,  1.79s/it]
                                                     

  7%|▋         | 821/12000 [29:23<5:34:10,  1.79s/it]
  7%|▋         | 822/12000 [29:27<8:11:37,  2.64s/it]
                                                     

  7%|▋         | 822/12000 [29:27<8:11:37,  2.64s/it]
  7%|▋         | 823/12000 [29:29<7:18:44,  2.36s/it]
                                                     

  7%|▋         | 823/12000 [29:29<7:18:44,  2.36s/it]
  7%|▋         | 824/12000 [29:31<6:36:26,  2.13s/it]
                                                     

  7%|▋         | 824/12000 [29:31<6:36:26,  2.13s/it]
  7%|▋         | 825/12000 [29:32<6:14:01,  2.01s/it]
                                                     

  7%|▋         | 825/12000 [29:32<6:14:01,  2.01s/it]
  7%|▋         | 826/12000 [29:34<6:14:06,  2.01s/it]
                                                     

  7%|▋         | 826/12000 [29:34<6:14:06,  2.01s/it]
  7%|▋         | 827/12000 [29:36<6:19:11,  2.04s/it]
                                                     

  7%|▋         | 827/12000 [29:36<6:19:11,  2.04s/it]
  7%|▋         | 828/12000 [29:38<5:49:37,  1.88s/it]
                                                     

  7%|▋         | 828/12000 [29:38<5:49:37,  1.88s/it]
  7%|▋         | 829/12000 [29:40<6:10:33,  1.99s/it]
                                                     

  7%|▋         | 829/12000 [29:40<6:10:33,  1.99s/it]
  7%|▋         | 830/12000 [29:42<5:42:21,  1.84s/it]
                                                     

  7%|▋         | 830/12000 [29:42<5:42:21,  1.84s/it]
  7%|▋         | 831/12000 [29:43<5:19:13,  1.71s/it]
                                                     

  7%|▋         | 831/12000 [29:43<5:19:13,  1.71s/it]
  7%|▋         | 832/12000 [29:45<5:42:22,  1.84s/it]
                                                     

  7%|▋         | 832/12000 [29:45<5:42:22,  1.84s/it]
  7%|▋         | 833/12000 [29:50<8:20:49,  2.69s/it]
                                                     

  7%|▋         | 833/12000 [29:50<8:20:49,  2.69s/it]
  7%|▋         | 834/12000 [29:51<7:14:42,  2.34s/it]
                                                     

  7%|▋         | 834/12000 [29:51<7:14:42,  2.34s/it]
  7%|▋         | 835/12000 [29:55<8:28:18,  2.73s/it]
                                                     

  7%|▋         | 835/12000 [29:55<8:28:18,  2.73s/it]
  7%|▋         | 836/12000 [29:57<8:04:04,  2.60s/it]
                                                     

  7%|▋         | 836/12000 [29:57<8:04:04,  2.60s/it]
  7%|▋         | 837/12000 [29:59<7:13:27,  2.33s/it]
                                                     

  7%|▋         | 837/12000 [29:59<7:13:27,  2.33s/it]
  7%|▋         | 838/12000 [30:01<6:40:19,  2.15s/it]
                                                     

  7%|▋         | 838/12000 [30:01<6:40:19,  2.15s/it]
  7%|▋         | 839/12000 [30:04<7:24:06,  2.39s/it]
                                                     

  7%|▋         | 839/12000 [30:04<7:24:06,  2.39s/it]
  7%|▋         | 840/12000 [30:05<6:40:10,  2.15s/it]
                                                     

  7%|▋         | 840/12000 [30:05<6:40:10,  2.15s/it]
  7%|▋         | 841/12000 [30:07<5:56:52,  1.92s/it]
                                                     

  7%|▋         | 841/12000 [30:07<5:56:52,  1.92s/it]
  7%|▋         | 842/12000 [30:09<5:46:58,  1.87s/it]
                                                     

  7%|▋         | 842/12000 [30:09<5:46:58,  1.87s/it]
  7%|▋         | 843/12000 [30:10<5:50:49,  1.89s/it]
                                                     

  7%|▋         | 843/12000 [30:10<5:50:49,  1.89s/it]
  7%|▋         | 844/12000 [30:12<5:44:35,  1.85s/it]
                                                     

  7%|▋         | 844/12000 [30:12<5:44:35,  1.85s/it]
  7%|▋         | 845/12000 [30:15<6:45:25,  2.18s/it]
                                                     

  7%|▋         | 845/12000 [30:15<6:45:25,  2.18s/it]
  7%|▋         | 846/12000 [30:17<6:29:01,  2.09s/it]
                                                     

  7%|▋         | 846/12000 [30:17<6:29:01,  2.09s/it]
  7%|▋         | 847/12000 [30:19<6:05:10,  1.96s/it]
                                                     

  7%|▋         | 847/12000 [30:19<6:05:10,  1.96s/it]
  7%|▋         | 848/12000 [30:21<6:14:16,  2.01s/it]
                                                     

  7%|▋         | 848/12000 [30:21<6:14:16,  2.01s/it]
  7%|▋         | 849/12000 [30:23<6:09:22,  1.99s/it]
                                                     

  7%|▋         | 849/12000 [30:23<6:09:22,  1.99s/it]
  7%|▋         | 850/12000 [30:24<5:35:07,  1.80s/it]
                                                     
{'loss': '0.9937', 'grad_norm': '0.4067', 'learning_rate': '0.0001998', 'entropy': '1.001', 'num_tokens': '4.34e+06', 'mean_token_accuracy': '0.756', 'epoch': '0.007106'}
{'loss': '1.097', 'grad_norm': '0.3751', 'learning_rate': '0.0001998', 'entropy': '1.038', 'num_tokens': '4.345e+06', 'mean_token_accuracy': '0.752', 'epoch': '0.007115'}
{'loss': '0.8962', 'grad_norm': '0.3842', 'learning_rate': '0.0001998', 'entropy': '0.8879', 'num_tokens': '4.349e+06', 'mean_token_accuracy': '0.7707', 'epoch': '0.007124'}
{'loss': '0.7999', 'grad_norm': '0.4222', 'learning_rate': '0.0001998', 'entropy': '0.8062', 'num_tokens': '4.353e+06', 'mean_token_accuracy': '0.7819', 'epoch': '0.007133'}
{'loss': '0.8652', 'grad_norm': '0.305', 'learning_rate': '0.0001998', 'entropy': '0.7651', 'num_tokens': '4.36e+06', 'mean_token_accuracy': '0.8025', 'epoch': '0.007142'}
{'loss': '1.066', 'grad_norm': '0.3501', 'learning_rate': '0.0001998', 'entropy': '1.094', 'num_tokens': '4.366e+06', 'mean_token_accuracy': '0.7301', 'epoch': '0.007151'}
{'loss': '0.954', 'grad_norm': '0.4022', 'learning_rate': '0.0001998', 'entropy': '0.9443', 'num_tokens': '4.37e+06', 'mean_token_accuracy': '0.7652', 'epoch': '0.00716'}
{'loss': '0.8164', 'grad_norm': '0.3826', 'learning_rate': '0.0001998', 'entropy': '0.7965', 'num_tokens': '4.376e+06', 'mean_token_accuracy': '0.7973', 'epoch': '0.007168'}
{'loss': '0.8771', 'grad_norm': '0.3431', 'learning_rate': '0.0001998', 'entropy': '0.8493', 'num_tokens': '4.383e+06', 'mean_token_accuracy': '0.7965', 'epoch': '0.007177'}
{'loss': '0.7377', 'grad_norm': '0.3927', 'learning_rate': '0.0001998', 'entropy': '0.9043', 'num_tokens': '4.392e+06', 'mean_token_accuracy': '0.7748', 'epoch': '0.007186'}
{'loss': '0.7738', 'grad_norm': '0.3269', 'learning_rate': '0.0001998', 'entropy': '0.8764', 'num_tokens': '4.396e+06', 'mean_token_accuracy': '0.7936', 'epoch': '0.007195'}
{'loss': '0.6631', 'grad_norm': '0.333', 'learning_rate': '0.0001998', 'entropy': '0.7406', 'num_tokens': '4.4e+06', 'mean_token_accuracy': '0.827', 'epoch': '0.007204'}
{'loss': '0.8375', 'grad_norm': '0.3575', 'learning_rate': '0.0001998', 'entropy': '0.8978', 'num_tokens': '4.405e+06', 'mean_token_accuracy': '0.779', 'epoch': '0.007213'}
{'loss': '0.8306', 'grad_norm': '0.3635', 'learning_rate': '0.0001998', 'entropy': '0.8302', 'num_tokens': '4.409e+06', 'mean_token_accuracy': '0.7917', 'epoch': '0.007221'}
{'loss': '0.874', 'grad_norm': '0.3524', 'learning_rate': '0.0001998', 'entropy': '0.951', 'num_tokens': '4.414e+06', 'mean_token_accuracy': '0.7681', 'epoch': '0.00723'}
{'loss': '1.005', 'grad_norm': '0.382', 'learning_rate': '0.0001998', 'entropy': '1.001', 'num_tokens': '4.418e+06', 'mean_token_accuracy': '0.7514', 'epoch': '0.007239'}
{'loss': '0.872', 'grad_norm': '0.3755', 'learning_rate': '0.0001998', 'entropy': '0.9525', 'num_tokens': '4.423e+06', 'mean_token_accuracy': '0.761', 'epoch': '0.007248'}
{'loss': '0.8634', 'grad_norm': '0.3862', 'learning_rate': '0.0001998', 'entropy': '0.8779', 'num_tokens': '4.426e+06', 'mean_token_accuracy': '0.7837', 'epoch': '0.007257'}
{'loss': '1.153', 'grad_norm': '0.3942', 'learning_rate': '0.0001998', 'entropy': '1.093', 'num_tokens': '4.433e+06', 'mean_token_accuracy': '0.7244', 'epoch': '0.007266'}
{'loss': '1.243', 'grad_norm': '0.3161', 'learning_rate': '0.0001998', 'entropy': '1.028', 'num_tokens': '4.441e+06', 'mean_token_accuracy': '0.7472', 'epoch': '0.007275'}
{'loss': '0.9555', 'grad_norm': '0.3593', 'learning_rate': '0.0001998', 'entropy': '0.9995', 'num_tokens': '4.446e+06', 'mean_token_accuracy': '0.7471', 'epoch': '0.007283'}
{'loss': '0.8014', 'grad_norm': '0.3907', 'learning_rate': '0.0001998', 'entropy': '0.8055', 'num_tokens': '4.45e+06', 'mean_token_accuracy': '0.7955', 'epoch': '0.007292'}
{'loss': '0.8988', 'grad_norm': '0.3828', 'learning_rate': '0.0001998', 'entropy': '0.8631', 'num_tokens': '4.454e+06', 'mean_token_accuracy': '0.7836', 'epoch': '0.007301'}
{'loss': '1.179', 'grad_norm': '0.3947', 'learning_rate': '0.0001998', 'entropy': '1.112', 'num_tokens': '4.46e+06', 'mean_token_accuracy': '0.729', 'epoch': '0.00731'}
{'loss': '1.077', 'grad_norm': '0.3256', 'learning_rate': '0.0001998', 'entropy': '1.022', 'num_tokens': '4.466e+06', 'mean_token_accuracy': '0.7583', 'epoch': '0.007319'}
{'loss': '0.9292', 'grad_norm': '0.3511', 'learning_rate': '0.0001998', 'entropy': '0.9107', 'num_tokens': '4.47e+06', 'mean_token_accuracy': '0.7815', 'epoch': '0.007328'}
{'loss': '0.8264', 'grad_norm': '0.3184', 'learning_rate': '0.0001998', 'entropy': '0.757', 'num_tokens': '4.477e+06', 'mean_token_accuracy': '0.8207', 'epoch': '0.007337'}
{'loss': '0.7798', 'grad_norm': '0.3862', 'learning_rate': '0.0001998', 'entropy': '0.7199', 'num_tokens': '4.481e+06', 'mean_token_accuracy': '0.8205', 'epoch': '0.007345'}
{'loss': '1.06', 'grad_norm': '0.2863', 'learning_rate': '0.0001998', 'entropy': '1.095', 'num_tokens': '4.485e+06', 'mean_token_accuracy': '0.7202', 'epoch': '0.007354'}
{'loss': '1.001', 'grad_norm': '0.4063', 'learning_rate': '0.0001998', 'entropy': '1.084', 'num_tokens': '4.49e+06', 'mean_token_accuracy': '0.7424', 'epoch': '0.007363'}
{'loss': '0.8584', 'grad_norm': '0.3069', 'learning_rate': '0.0001998', 'entropy': '0.8989', 'num_tokens': '4.499e+06', 'mean_token_accuracy': '0.7676', 'epoch': '0.007372'}
{'loss': '0.9108', 'grad_norm': '0.399', 'learning_rate': '0.0001998', 'entropy': '0.9027', 'num_tokens': '4.504e+06', 'mean_token_accuracy': '0.7738', 'epoch': '0.007381'}
{'loss': '1.215', 'grad_norm': '0.3596', 'learning_rate': '0.0001998', 'entropy': '1.216', 'num_tokens': '4.509e+06', 'mean_token_accuracy': '0.7108', 'epoch': '0.00739'}
{'loss': '1.083', 'grad_norm': '0.3559', 'learning_rate': '0.0001998', 'entropy': '1.061', 'num_tokens': '4.515e+06', 'mean_token_accuracy': '0.753', 'epoch': '0.007398'}
{'loss': '1.198', 'grad_norm': '0.3917', 'learning_rate': '0.0001998', 'entropy': '1.092', 'num_tokens': '4.52e+06', 'mean_token_accuracy': '0.7293', 'epoch': '0.007407'}
{'loss': '0.895', 'grad_norm': '0.4217', 'learning_rate': '0.0001998', 'entropy': '0.8825', 'num_tokens': '4.526e+06', 'mean_token_accuracy': '0.7702', 'epoch': '0.007416'}
{'loss': '1.322', 'grad_norm': '0.3859', 'learning_rate': '0.0001998', 'entropy': '1.326', 'num_tokens': '4.533e+06', 'mean_token_accuracy': '0.7066', 'epoch': '0.007425'}
{'loss': '0.7495', 'grad_norm': '0.3752', 'learning_rate': '0.0001998', 'entropy': '0.7742', 'num_tokens': '4.538e+06', 'mean_token_accuracy': '0.821', 'epoch': '0.007434'}
{'loss': '1.115', 'grad_norm': '0.4123', 'learning_rate': '0.0001998', 'entropy': '1.143', 'num_tokens': '4.542e+06', 'mean_token_accuracy': '0.7163', 'epoch': '0.007443'}
{'loss': '0.7989', 'grad_norm': '0.339', 'learning_rate': '0.0001998', 'entropy': '0.8037', 'num_tokens': '4.547e+06', 'mean_token_accuracy': '0.7958', 'epoch': '0.007452'}
{'loss': '1.12', 'grad_norm': '0.4236', 'learning_rate': '0.0001998', 'entropy': '1.025', 'num_tokens': '4.553e+06', 'mean_token_accuracy': '0.7516', 'epoch': '0.00746'}
{'loss': '1.164', 'grad_norm': '0.4419', 'learning_rate': '0.0001998', 'entropy': '1.142', 'num_tokens': '4.558e+06', 'mean_token_accuracy': '0.7197', 'epoch': '0.007469'}
{'loss': '0.9751', 'grad_norm': '0.2781', 'learning_rate': '0.0001998', 'entropy': '0.9483', 'num_tokens': '4.566e+06', 'mean_token_accuracy': '0.7725', 'epoch': '0.007478'}
{'loss': '0.9187', 'grad_norm': '0.4407', 'learning_rate': '0.0001998', 'entropy': '1.02', 'num_tokens': '4.571e+06', 'mean_token_accuracy': '0.7527', 'epoch': '0.007487'}
{'loss': '0.9067', 'grad_norm': '0.4261', 'learning_rate': '0.0001998', 'entropy': '0.9653', 'num_tokens': '4.576e+06', 'mean_token_accuracy': '0.7712', 'epoch': '0.007496'}
{'loss': '0.9135', 'grad_norm': '0.335', 'learning_rate': '0.0001998', 'entropy': '0.8966', 'num_tokens': '4.582e+06', 'mean_token_accuracy': '0.7784', 'epoch': '0.007505'}
{'loss': '0.8438', 'grad_norm': '0.3713', 'learning_rate': '0.0001998', 'entropy': '0.9594', 'num_tokens': '4.587e+06', 'mean_token_accuracy': '0.7644', 'epoch': '0.007514'}

  7%|▋         | 850/12000 [30:24<5:35:07,  1.80s/it]
  7%|▋         | 851/12000 [30:28<7:46:02,  2.51s/it]
                                                     

  7%|▋         | 851/12000 [30:28<7:46:02,  2.51s/it]
  7%|▋         | 852/12000 [30:31<7:30:48,  2.43s/it]
                                                     

  7%|▋         | 852/12000 [30:31<7:30:48,  2.43s/it]
  7%|▋         | 853/12000 [30:36<10:02:39,  3.24s/it]
                                                      

  7%|▋         | 853/12000 [30:36<10:02:39,  3.24s/it]
  7%|▋         | 854/12000 [30:38<9:02:59,  2.92s/it] 
                                                     

  7%|▋         | 854/12000 [30:38<9:02:59,  2.92s/it]
  7%|▋         | 855/12000 [30:39<7:44:20,  2.50s/it]
                                                     

  7%|▋         | 855/12000 [30:39<7:44:20,  2.50s/it]
  7%|▋         | 856/12000 [30:41<6:48:37,  2.20s/it]
                                                     

  7%|▋         | 856/12000 [30:41<6:48:37,  2.20s/it]
  7%|▋         | 857/12000 [30:45<8:14:36,  2.66s/it]
                                                     

  7%|▋         | 857/12000 [30:45<8:14:36,  2.66s/it]
  7%|▋         | 858/12000 [30:46<7:16:34,  2.35s/it]
                                                     

  7%|▋         | 858/12000 [30:46<7:16:34,  2.35s/it]
  7%|▋         | 859/12000 [30:49<7:18:21,  2.36s/it]
                                                     

  7%|▋         | 859/12000 [30:49<7:18:21,  2.36s/it]
  7%|▋         | 860/12000 [30:50<6:39:42,  2.15s/it]
                                                     

  7%|▋         | 860/12000 [30:50<6:39:42,  2.15s/it]
  7%|▋         | 861/12000 [30:52<6:08:45,  1.99s/it]
                                                     

  7%|▋         | 861/12000 [30:52<6:08:45,  1.99s/it]
  7%|▋         | 862/12000 [30:54<6:29:48,  2.10s/it]
                                                     

  7%|▋         | 862/12000 [30:54<6:29:48,  2.10s/it]
  7%|▋         | 863/12000 [30:57<7:23:01,  2.39s/it]
                                                     

  7%|▋         | 863/12000 [30:57<7:23:01,  2.39s/it]
  7%|▋         | 864/12000 [30:59<6:54:51,  2.24s/it]
                                                     

  7%|▋         | 864/12000 [30:59<6:54:51,  2.24s/it]
  7%|▋         | 865/12000 [31:01<6:09:45,  1.99s/it]
                                                     

  7%|▋         | 865/12000 [31:01<6:09:45,  1.99s/it]
  7%|▋         | 866/12000 [31:02<5:50:42,  1.89s/it]
                                                     

  7%|▋         | 866/12000 [31:02<5:50:42,  1.89s/it]
  7%|▋         | 867/12000 [31:04<5:42:46,  1.85s/it]
                                                     

  7%|▋         | 867/12000 [31:04<5:42:46,  1.85s/it]
  7%|▋         | 868/12000 [31:06<5:26:09,  1.76s/it]
                                                     

  7%|▋         | 868/12000 [31:06<5:26:09,  1.76s/it]
  7%|▋         | 869/12000 [31:08<5:45:25,  1.86s/it]
                                                     

  7%|▋         | 869/12000 [31:08<5:45:25,  1.86s/it]
  7%|▋         | 870/12000 [31:09<5:26:50,  1.76s/it]
                                                     

  7%|▋         | 870/12000 [31:09<5:26:50,  1.76s/it]
  7%|▋         | 871/12000 [31:12<6:24:02,  2.07s/it]
                                                     

  7%|▋         | 871/12000 [31:12<6:24:02,  2.07s/it]
  7%|▋         | 872/12000 [31:14<6:03:58,  1.96s/it]
                                                     

  7%|▋         | 872/12000 [31:14<6:03:58,  1.96s/it]
  7%|▋         | 873/12000 [31:15<5:34:40,  1.80s/it]
                                                     

  7%|▋         | 873/12000 [31:15<5:34:40,  1.80s/it]
  7%|▋         | 874/12000 [31:17<5:51:22,  1.89s/it]
                                                     

  7%|▋         | 874/12000 [31:17<5:51:22,  1.89s/it]
  7%|▋         | 875/12000 [31:19<5:34:37,  1.80s/it]
                                                     

  7%|▋         | 875/12000 [31:19<5:34:37,  1.80s/it]
  7%|▋         | 876/12000 [31:21<5:30:13,  1.78s/it]
                                                     

  7%|▋         | 876/12000 [31:21<5:30:13,  1.78s/it]
  7%|▋         | 877/12000 [31:22<5:22:24,  1.74s/it]
                                                     

  7%|▋         | 877/12000 [31:22<5:22:24,  1.74s/it]
  7%|▋         | 878/12000 [31:24<5:19:09,  1.72s/it]
                                                     

  7%|▋         | 878/12000 [31:24<5:19:09,  1.72s/it]
  7%|▋         | 879/12000 [31:28<7:38:07,  2.47s/it]
                                                     

  7%|▋         | 879/12000 [31:28<7:38:07,  2.47s/it]
  7%|▋         | 880/12000 [31:30<6:50:00,  2.21s/it]
                                                     

  7%|▋         | 880/12000 [31:30<6:50:00,  2.21s/it]
  7%|▋         | 881/12000 [31:31<6:08:31,  1.99s/it]
                                                     

  7%|▋         | 881/12000 [31:31<6:08:31,  1.99s/it]
  7%|▋         | 882/12000 [31:33<5:53:11,  1.91s/it]
                                                     

  7%|▋         | 882/12000 [31:33<5:53:11,  1.91s/it]
  7%|▋         | 883/12000 [31:35<5:51:29,  1.90s/it]
                                                     

  7%|▋         | 883/12000 [31:35<5:51:29,  1.90s/it]
  7%|▋         | 884/12000 [31:36<5:38:07,  1.83s/it]
                                                     

  7%|▋         | 884/12000 [31:36<5:38:07,  1.83s/it]
  7%|▋         | 885/12000 [31:38<5:27:32,  1.77s/it]
                                                     

  7%|▋         | 885/12000 [31:38<5:27:32,  1.77s/it]
  7%|▋         | 886/12000 [31:41<6:35:36,  2.14s/it]
                                                     

  7%|▋         | 886/12000 [31:41<6:35:36,  2.14s/it]
  7%|▋         | 887/12000 [31:43<6:01:36,  1.95s/it]
                                                     

  7%|▋         | 887/12000 [31:43<6:01:36,  1.95s/it]
  7%|▋         | 888/12000 [31:44<5:39:38,  1.83s/it]
                                                     

  7%|▋         | 888/12000 [31:44<5:39:38,  1.83s/it]
  7%|▋         | 889/12000 [31:46<5:25:06,  1.76s/it]
                                                     

  7%|▋         | 889/12000 [31:46<5:25:06,  1.76s/it]
  7%|▋         | 890/12000 [31:47<5:10:16,  1.68s/it]
                                                     

  7%|▋         | 890/12000 [31:47<5:10:16,  1.68s/it]
  7%|▋         | 891/12000 [31:49<4:58:11,  1.61s/it]
                                                     

  7%|▋         | 891/12000 [31:49<4:58:11,  1.61s/it]
  7%|▋         | 892/12000 [31:51<5:33:51,  1.80s/it]
                                                     

  7%|▋         | 892/12000 [31:51<5:33:51,  1.80s/it]
  7%|▋         | 893/12000 [31:53<6:10:53,  2.00s/it]
                                                     

  7%|▋         | 893/12000 [31:53<6:10:53,  2.00s/it]
  7%|▋         | 894/12000 [31:55<5:53:55,  1.91s/it]
                                                     

  7%|▋         | 894/12000 [31:55<5:53:55,  1.91s/it]
  7%|▋         | 895/12000 [31:58<6:49:35,  2.21s/it]
                                                     

  7%|▋         | 895/12000 [31:58<6:49:35,  2.21s/it]
  7%|▋         | 896/12000 [32:00<6:13:17,  2.02s/it]
                                                     

  7%|▋         | 896/12000 [32:00<6:13:17,  2.02s/it]
  7%|▋         | 897/12000 [32:01<5:38:53,  1.83s/it]
                                                     
{'loss': '1.068', 'grad_norm': '0.4302', 'learning_rate': '0.0001998', 'entropy': '1.112', 'num_tokens': '4.591e+06', 'mean_token_accuracy': '0.7212', 'epoch': '0.007522'}
{'loss': '0.7372', 'grad_norm': '0.3203', 'learning_rate': '0.0001998', 'entropy': '0.7408', 'num_tokens': '4.599e+06', 'mean_token_accuracy': '0.808', 'epoch': '0.007531'}
{'loss': '1.187', 'grad_norm': '0.3576', 'learning_rate': '0.0001998', 'entropy': '0.9783', 'num_tokens': '4.605e+06', 'mean_token_accuracy': '0.7787', 'epoch': '0.00754'}
{'loss': '1.462', 'grad_norm': '0.3909', 'learning_rate': '0.0001998', 'entropy': '1.163', 'num_tokens': '4.613e+06', 'mean_token_accuracy': '0.7337', 'epoch': '0.007549'}
{'loss': '0.8172', 'grad_norm': '0.3224', 'learning_rate': '0.0001998', 'entropy': '0.8606', 'num_tokens': '4.618e+06', 'mean_token_accuracy': '0.802', 'epoch': '0.007558'}
{'loss': '0.8066', 'grad_norm': '0.4013', 'learning_rate': '0.0001998', 'entropy': '0.7953', 'num_tokens': '4.622e+06', 'mean_token_accuracy': '0.8039', 'epoch': '0.007567'}
{'loss': '0.9284', 'grad_norm': '0.5017', 'learning_rate': '0.0001998', 'entropy': '0.9601', 'num_tokens': '4.625e+06', 'mean_token_accuracy': '0.7718', 'epoch': '0.007575'}
{'loss': '0.9603', 'grad_norm': '0.447', 'learning_rate': '0.0001998', 'entropy': '1.005', 'num_tokens': '4.63e+06', 'mean_token_accuracy': '0.7504', 'epoch': '0.007584'}
{'loss': '0.9643', 'grad_norm': '0.3968', 'learning_rate': '0.0001997', 'entropy': '0.991', 'num_tokens': '4.635e+06', 'mean_token_accuracy': '0.7516', 'epoch': '0.007593'}
{'loss': '0.9586', 'grad_norm': '0.2979', 'learning_rate': '0.0001997', 'entropy': '1.017', 'num_tokens': '4.643e+06', 'mean_token_accuracy': '0.7719', 'epoch': '0.007602'}
{'loss': '1.075', 'grad_norm': '0.3631', 'learning_rate': '0.0001997', 'entropy': '1.168', 'num_tokens': '4.648e+06', 'mean_token_accuracy': '0.7308', 'epoch': '0.007611'}
{'loss': '0.8748', 'grad_norm': '0.3423', 'learning_rate': '0.0001997', 'entropy': '0.8638', 'num_tokens': '4.652e+06', 'mean_token_accuracy': '0.802', 'epoch': '0.00762'}
{'loss': '0.7051', 'grad_norm': '0.3103', 'learning_rate': '0.0001997', 'entropy': '0.7448', 'num_tokens': '4.657e+06', 'mean_token_accuracy': '0.822', 'epoch': '0.007629'}
{'loss': '0.927', 'grad_norm': '0.3332', 'learning_rate': '0.0001997', 'entropy': '0.8174', 'num_tokens': '4.667e+06', 'mean_token_accuracy': '0.8032', 'epoch': '0.007637'}
{'loss': '1.134', 'grad_norm': '0.4594', 'learning_rate': '0.0001997', 'entropy': '1.154', 'num_tokens': '4.672e+06', 'mean_token_accuracy': '0.7277', 'epoch': '0.007646'}
{'loss': '0.8817', 'grad_norm': '0.3589', 'learning_rate': '0.0001997', 'entropy': '0.9237', 'num_tokens': '4.676e+06', 'mean_token_accuracy': '0.758', 'epoch': '0.007655'}
{'loss': '0.8468', 'grad_norm': '0.3291', 'learning_rate': '0.0001997', 'entropy': '0.8591', 'num_tokens': '4.68e+06', 'mean_token_accuracy': '0.7892', 'epoch': '0.007664'}
{'loss': '0.7456', 'grad_norm': '0.3667', 'learning_rate': '0.0001997', 'entropy': '0.7097', 'num_tokens': '4.685e+06', 'mean_token_accuracy': '0.8201', 'epoch': '0.007673'}
{'loss': '0.7142', 'grad_norm': '0.3338', 'learning_rate': '0.0001997', 'entropy': '0.7017', 'num_tokens': '4.69e+06', 'mean_token_accuracy': '0.8226', 'epoch': '0.007682'}
{'loss': '1.121', 'grad_norm': '0.3902', 'learning_rate': '0.0001997', 'entropy': '1.051', 'num_tokens': '4.695e+06', 'mean_token_accuracy': '0.7472', 'epoch': '0.007691'}
{'loss': '0.8576', 'grad_norm': '0.3595', 'learning_rate': '0.0001997', 'entropy': '0.821', 'num_tokens': '4.7e+06', 'mean_token_accuracy': '0.7909', 'epoch': '0.007699'}
{'loss': '0.7168', 'grad_norm': '0.3431', 'learning_rate': '0.0001997', 'entropy': '0.7458', 'num_tokens': '4.708e+06', 'mean_token_accuracy': '0.8103', 'epoch': '0.007708'}
{'loss': '1.368', 'grad_norm': '0.3553', 'learning_rate': '0.0001997', 'entropy': '1.261', 'num_tokens': '4.713e+06', 'mean_token_accuracy': '0.6956', 'epoch': '0.007717'}
{'loss': '1.034', 'grad_norm': '0.373', 'learning_rate': '0.0001997', 'entropy': '1.095', 'num_tokens': '4.717e+06', 'mean_token_accuracy': '0.7375', 'epoch': '0.007726'}
{'loss': '0.9215', 'grad_norm': '0.3535', 'learning_rate': '0.0001997', 'entropy': '0.9819', 'num_tokens': '4.723e+06', 'mean_token_accuracy': '0.7724', 'epoch': '0.007735'}
{'loss': '0.744', 'grad_norm': '0.3103', 'learning_rate': '0.0001997', 'entropy': '0.8039', 'num_tokens': '4.728e+06', 'mean_token_accuracy': '0.7971', 'epoch': '0.007744'}
{'loss': '0.7099', 'grad_norm': '0.3385', 'learning_rate': '0.0001997', 'entropy': '0.776', 'num_tokens': '4.732e+06', 'mean_token_accuracy': '0.8142', 'epoch': '0.007752'}
{'loss': '1.158', 'grad_norm': '0.4046', 'learning_rate': '0.0001997', 'entropy': '1.172', 'num_tokens': '4.737e+06', 'mean_token_accuracy': '0.7475', 'epoch': '0.007761'}
{'loss': '0.8901', 'grad_norm': '0.396', 'learning_rate': '0.0001997', 'entropy': '0.9289', 'num_tokens': '4.741e+06', 'mean_token_accuracy': '0.7829', 'epoch': '0.00777'}
{'loss': '1.236', 'grad_norm': '0.3297', 'learning_rate': '0.0001997', 'entropy': '1.11', 'num_tokens': '4.747e+06', 'mean_token_accuracy': '0.7374', 'epoch': '0.007779'}
{'loss': '0.7768', 'grad_norm': '0.3996', 'learning_rate': '0.0001997', 'entropy': '0.7544', 'num_tokens': '4.752e+06', 'mean_token_accuracy': '0.8019', 'epoch': '0.007788'}
{'loss': '0.8309', 'grad_norm': '0.4015', 'learning_rate': '0.0001997', 'entropy': '0.8006', 'num_tokens': '4.756e+06', 'mean_token_accuracy': '0.7896', 'epoch': '0.007797'}
{'loss': '1.133', 'grad_norm': '0.4069', 'learning_rate': '0.0001997', 'entropy': '1.045', 'num_tokens': '4.762e+06', 'mean_token_accuracy': '0.7467', 'epoch': '0.007806'}
{'loss': '1.08', 'grad_norm': '0.3532', 'learning_rate': '0.0001997', 'entropy': '0.9569', 'num_tokens': '4.767e+06', 'mean_token_accuracy': '0.7536', 'epoch': '0.007814'}
{'loss': '0.7299', 'grad_norm': '0.3678', 'learning_rate': '0.0001997', 'entropy': '0.7045', 'num_tokens': '4.772e+06', 'mean_token_accuracy': '0.8051', 'epoch': '0.007823'}
{'loss': '0.9025', 'grad_norm': '0.411', 'learning_rate': '0.0001997', 'entropy': '0.8417', 'num_tokens': '4.776e+06', 'mean_token_accuracy': '0.796', 'epoch': '0.007832'}
{'loss': '1.09', 'grad_norm': '0.313', 'learning_rate': '0.0001997', 'entropy': '1.109', 'num_tokens': '4.785e+06', 'mean_token_accuracy': '0.736', 'epoch': '0.007841'}
{'loss': '1.076', 'grad_norm': '0.3609', 'learning_rate': '0.0001997', 'entropy': '0.977', 'num_tokens': '4.79e+06', 'mean_token_accuracy': '0.7572', 'epoch': '0.00785'}
{'loss': '1.201', 'grad_norm': '0.3809', 'learning_rate': '0.0001997', 'entropy': '1.215', 'num_tokens': '4.794e+06', 'mean_token_accuracy': '0.71', 'epoch': '0.007859'}
{'loss': '0.8612', 'grad_norm': '0.3904', 'learning_rate': '0.0001997', 'entropy': '1.009', 'num_tokens': '4.799e+06', 'mean_token_accuracy': '0.7599', 'epoch': '0.007868'}
{'loss': '1.109', 'grad_norm': '0.3955', 'learning_rate': '0.0001997', 'entropy': '1.094', 'num_tokens': '4.803e+06', 'mean_token_accuracy': '0.7303', 'epoch': '0.007876'}
{'loss': '0.9493', 'grad_norm': '0.4057', 'learning_rate': '0.0001997', 'entropy': '1.038', 'num_tokens': '4.807e+06', 'mean_token_accuracy': '0.7531', 'epoch': '0.007885'}
{'loss': '1.136', 'grad_norm': '0.3924', 'learning_rate': '0.0001997', 'entropy': '1.233', 'num_tokens': '4.812e+06', 'mean_token_accuracy': '0.7234', 'epoch': '0.007894'}
{'loss': '0.7771', 'grad_norm': '0.3025', 'learning_rate': '0.0001997', 'entropy': '0.8632', 'num_tokens': '4.818e+06', 'mean_token_accuracy': '0.7962', 'epoch': '0.007903'}
{'loss': '1.142', 'grad_norm': '0.3813', 'learning_rate': '0.0001997', 'entropy': '1.214', 'num_tokens': '4.824e+06', 'mean_token_accuracy': '0.7192', 'epoch': '0.007912'}
{'loss': '0.8083', 'grad_norm': '0.3174', 'learning_rate': '0.0001997', 'entropy': '0.8692', 'num_tokens': '4.83e+06', 'mean_token_accuracy': '0.7877', 'epoch': '0.007921'}
{'loss': '0.8486', 'grad_norm': '0.3723', 'learning_rate': '0.0001997', 'entropy': '0.8365', 'num_tokens': '4.835e+06', 'mean_token_accuracy': '0.7847', 'epoch': '0.007929'}

  7%|▋         | 897/12000 [32:01<5:38:53,  1.83s/it]
  7%|▋         | 898/12000 [32:03<5:32:04,  1.79s/it]
                                                     

  7%|▋         | 898/12000 [32:03<5:32:04,  1.79s/it]
  7%|▋         | 899/12000 [32:05<5:45:29,  1.87s/it]
                                                     

  7%|▋         | 899/12000 [32:05<5:45:29,  1.87s/it]
  8%|▊         | 900/12000 [32:07<5:46:58,  1.88s/it]
                                                     

  8%|▊         | 900/12000 [32:07<5:46:58,  1.88s/it]
  8%|▊         | 901/12000 [32:12<8:44:58,  2.84s/it]
                                                     

  8%|▊         | 901/12000 [32:12<8:44:58,  2.84s/it]
  8%|▊         | 902/12000 [32:14<8:11:48,  2.66s/it]
                                                     

  8%|▊         | 902/12000 [32:14<8:11:48,  2.66s/it]
  8%|▊         | 903/12000 [32:16<7:12:26,  2.34s/it]
                                                     

  8%|▊         | 903/12000 [32:16<7:12:26,  2.34s/it]
  8%|▊         | 904/12000 [32:17<6:47:24,  2.20s/it]
                                                     

  8%|▊         | 904/12000 [32:17<6:47:24,  2.20s/it]
  8%|▊         | 905/12000 [32:19<6:13:40,  2.02s/it]
                                                     

  8%|▊         | 905/12000 [32:19<6:13:40,  2.02s/it]
  8%|▊         | 906/12000 [32:22<6:52:34,  2.23s/it]
                                                     

  8%|▊         | 906/12000 [32:22<6:52:34,  2.23s/it]
  8%|▊         | 907/12000 [32:23<6:08:06,  1.99s/it]
                                                     

  8%|▊         | 907/12000 [32:23<6:08:06,  1.99s/it]
  8%|▊         | 908/12000 [32:25<6:08:50,  2.00s/it]
                                                     

  8%|▊         | 908/12000 [32:25<6:08:50,  2.00s/it]
  8%|▊         | 909/12000 [32:28<6:43:15,  2.18s/it]
                                                     

  8%|▊         | 909/12000 [32:28<6:43:15,  2.18s/it]
  8%|▊         | 910/12000 [32:30<6:49:14,  2.21s/it]
                                                     

  8%|▊         | 910/12000 [32:30<6:49:14,  2.21s/it]
  8%|▊         | 911/12000 [32:32<6:13:06,  2.02s/it]
                                                     

  8%|▊         | 911/12000 [32:32<6:13:06,  2.02s/it]
  8%|▊         | 912/12000 [32:33<5:53:44,  1.91s/it]
                                                     

  8%|▊         | 912/12000 [32:33<5:53:44,  1.91s/it]
  8%|▊         | 913/12000 [32:35<5:36:58,  1.82s/it]
                                                     

  8%|▊         | 913/12000 [32:35<5:36:58,  1.82s/it]
  8%|▊         | 914/12000 [32:37<6:03:55,  1.97s/it]
                                                     

  8%|▊         | 914/12000 [32:37<6:03:55,  1.97s/it]
  8%|▊         | 915/12000 [32:39<5:45:10,  1.87s/it]
                                                     

  8%|▊         | 915/12000 [32:39<5:45:10,  1.87s/it]
  8%|▊         | 916/12000 [32:41<5:39:21,  1.84s/it]
                                                     

  8%|▊         | 916/12000 [32:41<5:39:21,  1.84s/it]
  8%|▊         | 917/12000 [32:43<5:55:29,  1.92s/it]
                                                     

  8%|▊         | 917/12000 [32:43<5:55:29,  1.92s/it]
  8%|▊         | 918/12000 [32:44<5:31:20,  1.79s/it]
                                                     

  8%|▊         | 918/12000 [32:44<5:31:20,  1.79s/it]
  8%|▊         | 919/12000 [32:46<5:23:31,  1.75s/it]
                                                     

  8%|▊         | 919/12000 [32:46<5:23:31,  1.75s/it]
  8%|▊         | 920/12000 [32:51<8:11:49,  2.66s/it]
                                                     

  8%|▊         | 920/12000 [32:51<8:11:49,  2.66s/it]
  8%|▊         | 921/12000 [32:52<7:16:53,  2.37s/it]
                                                     

  8%|▊         | 921/12000 [32:52<7:16:53,  2.37s/it]
  8%|▊         | 922/12000 [32:55<7:34:56,  2.46s/it]
                                                     

  8%|▊         | 922/12000 [32:55<7:34:56,  2.46s/it]
  8%|▊         | 923/12000 [33:00<9:27:10,  3.07s/it]
                                                     

  8%|▊         | 923/12000 [33:00<9:27:10,  3.07s/it]
  8%|▊         | 924/12000 [33:02<8:40:48,  2.82s/it]
                                                     

  8%|▊         | 924/12000 [33:02<8:40:48,  2.82s/it]
  8%|▊         | 925/12000 [33:04<7:46:49,  2.53s/it]
                                                     

  8%|▊         | 925/12000 [33:04<7:46:49,  2.53s/it]
  8%|▊         | 926/12000 [33:06<7:49:39,  2.54s/it]
                                                     

  8%|▊         | 926/12000 [33:06<7:49:39,  2.54s/it]
  8%|▊         | 927/12000 [33:08<7:05:21,  2.30s/it]
                                                     

  8%|▊         | 927/12000 [33:08<7:05:21,  2.30s/it]
  8%|▊         | 928/12000 [33:11<7:41:05,  2.50s/it]
                                                     

  8%|▊         | 928/12000 [33:11<7:41:05,  2.50s/it]
  8%|▊         | 929/12000 [33:12<6:44:23,  2.19s/it]
                                                     

  8%|▊         | 929/12000 [33:12<6:44:23,  2.19s/it]
  8%|▊         | 930/12000 [33:14<6:18:57,  2.05s/it]
                                                     

  8%|▊         | 930/12000 [33:14<6:18:57,  2.05s/it]
  8%|▊         | 931/12000 [33:16<5:52:20,  1.91s/it]
                                                     

  8%|▊         | 931/12000 [33:16<5:52:20,  1.91s/it]
  8%|▊         | 932/12000 [33:18<5:56:26,  1.93s/it]
                                                     

  8%|▊         | 932/12000 [33:18<5:56:26,  1.93s/it]
  8%|▊         | 933/12000 [33:19<5:39:22,  1.84s/it]
                                                     

  8%|▊         | 933/12000 [33:19<5:39:22,  1.84s/it]
  8%|▊         | 934/12000 [33:22<6:20:52,  2.07s/it]
                                                     

  8%|▊         | 934/12000 [33:22<6:20:52,  2.07s/it]
  8%|▊         | 935/12000 [33:23<5:51:54,  1.91s/it]
                                                     

  8%|▊         | 935/12000 [33:23<5:51:54,  1.91s/it]
  8%|▊         | 936/12000 [33:25<5:41:58,  1.85s/it]
                                                     

  8%|▊         | 936/12000 [33:25<5:41:58,  1.85s/it]
  8%|▊         | 937/12000 [33:27<5:40:42,  1.85s/it]
                                                     

  8%|▊         | 937/12000 [33:27<5:40:42,  1.85s/it]
  8%|▊         | 938/12000 [33:30<7:02:01,  2.29s/it]
                                                     

  8%|▊         | 938/12000 [33:30<7:02:01,  2.29s/it]
  8%|▊         | 939/12000 [33:32<6:48:01,  2.21s/it]
                                                     

  8%|▊         | 939/12000 [33:32<6:48:01,  2.21s/it]
  8%|▊         | 940/12000 [33:35<7:37:25,  2.48s/it]
                                                     

  8%|▊         | 940/12000 [33:35<7:37:25,  2.48s/it]
  8%|▊         | 941/12000 [33:38<7:14:55,  2.36s/it]
                                                     

  8%|▊         | 941/12000 [33:38<7:14:55,  2.36s/it]
  8%|▊         | 942/12000 [33:39<6:50:15,  2.23s/it]
                                                     

  8%|▊         | 942/12000 [33:39<6:50:15,  2.23s/it]
  8%|▊         | 943/12000 [33:41<6:39:09,  2.17s/it]
                                                     

  8%|▊         | 943/12000 [33:41<6:39:09,  2.17s/it]
  8%|▊         | 944/12000 [33:43<5:53:23,  1.92s/it]
                                                     
{'loss': '0.9595', 'grad_norm': '0.3733', 'learning_rate': '0.0001997', 'entropy': '0.922', 'num_tokens': '4.839e+06', 'mean_token_accuracy': '0.7692', 'epoch': '0.007938'}
{'loss': '0.7933', 'grad_norm': '0.3564', 'learning_rate': '0.0001997', 'entropy': '0.7602', 'num_tokens': '4.843e+06', 'mean_token_accuracy': '0.8034', 'epoch': '0.007947'}
{'loss': '1.179', 'grad_norm': '0.3449', 'learning_rate': '0.0001997', 'entropy': '1.131', 'num_tokens': '4.849e+06', 'mean_token_accuracy': '0.7267', 'epoch': '0.007956'}
{'loss': '0.9339', 'grad_norm': '0.4078', 'learning_rate': '0.0001997', 'entropy': '0.9542', 'num_tokens': '4.855e+06', 'mean_token_accuracy': '0.7663', 'epoch': '0.007965'}
{'loss': '1.021', 'grad_norm': '0.3898', 'learning_rate': '0.0001997', 'entropy': '1.012', 'num_tokens': '4.859e+06', 'mean_token_accuracy': '0.7363', 'epoch': '0.007974'}
{'loss': '0.7728', 'grad_norm': '0.3344', 'learning_rate': '0.0001997', 'entropy': '0.8577', 'num_tokens': '4.865e+06', 'mean_token_accuracy': '0.7887', 'epoch': '0.007983'}
{'loss': '0.7625', 'grad_norm': '0.3293', 'learning_rate': '0.0001997', 'entropy': '0.7982', 'num_tokens': '4.871e+06', 'mean_token_accuracy': '0.8074', 'epoch': '0.007991'}
{'loss': '1.142', 'grad_norm': '0.4075', 'learning_rate': '0.0001997', 'entropy': '1.139', 'num_tokens': '4.876e+06', 'mean_token_accuracy': '0.7343', 'epoch': '0.008'}
{'loss': '0.9292', 'grad_norm': '0.3898', 'learning_rate': '0.0001996', 'entropy': '0.8887', 'num_tokens': '4.88e+06', 'mean_token_accuracy': '0.7862', 'epoch': '0.008009'}
{'loss': '0.7518', 'grad_norm': '0.399', 'learning_rate': '0.0001996', 'entropy': '0.773', 'num_tokens': '4.886e+06', 'mean_token_accuracy': '0.8079', 'epoch': '0.008018'}
{'loss': '1.044', 'grad_norm': '0.368', 'learning_rate': '0.0001996', 'entropy': '1.074', 'num_tokens': '4.89e+06', 'mean_token_accuracy': '0.7338', 'epoch': '0.008027'}
{'loss': '0.6552', 'grad_norm': '0.2583', 'learning_rate': '0.0001996', 'entropy': '0.6611', 'num_tokens': '4.896e+06', 'mean_token_accuracy': '0.8324', 'epoch': '0.008036'}
{'loss': '0.5931', 'grad_norm': '0.29', 'learning_rate': '0.0001996', 'entropy': '0.6545', 'num_tokens': '4.902e+06', 'mean_token_accuracy': '0.8383', 'epoch': '0.008044'}
{'loss': '0.8411', 'grad_norm': '0.294', 'learning_rate': '0.0001996', 'entropy': '0.8307', 'num_tokens': '4.908e+06', 'mean_token_accuracy': '0.7925', 'epoch': '0.008053'}
{'loss': '1.422', 'grad_norm': '0.4722', 'learning_rate': '0.0001996', 'entropy': '1.266', 'num_tokens': '4.912e+06', 'mean_token_accuracy': '0.7089', 'epoch': '0.008062'}
{'loss': '0.7444', 'grad_norm': '0.3812', 'learning_rate': '0.0001996', 'entropy': '0.7063', 'num_tokens': '4.917e+06', 'mean_token_accuracy': '0.8067', 'epoch': '0.008071'}
{'loss': '0.8397', 'grad_norm': '0.3911', 'learning_rate': '0.0001996', 'entropy': '0.8624', 'num_tokens': '4.921e+06', 'mean_token_accuracy': '0.7828', 'epoch': '0.00808'}
{'loss': '1.059', 'grad_norm': '0.4149', 'learning_rate': '0.0001996', 'entropy': '0.9968', 'num_tokens': '4.926e+06', 'mean_token_accuracy': '0.7488', 'epoch': '0.008089'}
{'loss': '1.04', 'grad_norm': '0.3449', 'learning_rate': '0.0001996', 'entropy': '1.073', 'num_tokens': '4.931e+06', 'mean_token_accuracy': '0.743', 'epoch': '0.008098'}
{'loss': '0.7094', 'grad_norm': '0.3956', 'learning_rate': '0.0001996', 'entropy': '0.718', 'num_tokens': '4.937e+06', 'mean_token_accuracy': '0.8197', 'epoch': '0.008106'}
{'loss': '1.122', 'grad_norm': '0.368', 'learning_rate': '0.0001996', 'entropy': '1.087', 'num_tokens': '4.942e+06', 'mean_token_accuracy': '0.7362', 'epoch': '0.008115'}
{'loss': '0.9158', 'grad_norm': '0.293', 'learning_rate': '0.0001996', 'entropy': '0.9833', 'num_tokens': '4.948e+06', 'mean_token_accuracy': '0.7714', 'epoch': '0.008124'}
{'loss': '0.9111', 'grad_norm': '0.3995', 'learning_rate': '0.0001996', 'entropy': '0.9365', 'num_tokens': '4.952e+06', 'mean_token_accuracy': '0.7769', 'epoch': '0.008133'}
{'loss': '1.801', 'grad_norm': '0.349', 'learning_rate': '0.0001996', 'entropy': '1.304', 'num_tokens': '4.961e+06', 'mean_token_accuracy': '0.7254', 'epoch': '0.008142'}
{'loss': '0.781', 'grad_norm': '0.3511', 'learning_rate': '0.0001996', 'entropy': '0.8248', 'num_tokens': '4.966e+06', 'mean_token_accuracy': '0.8048', 'epoch': '0.008151'}
{'loss': '0.7346', 'grad_norm': '0.3289', 'learning_rate': '0.0001996', 'entropy': '0.8655', 'num_tokens': '4.973e+06', 'mean_token_accuracy': '0.7822', 'epoch': '0.00816'}
{'loss': '0.8658', 'grad_norm': '0.2878', 'learning_rate': '0.0001996', 'entropy': '0.9222', 'num_tokens': '4.98e+06', 'mean_token_accuracy': '0.7749', 'epoch': '0.008168'}
{'loss': '0.6937', 'grad_norm': '0.3145', 'learning_rate': '0.0001996', 'entropy': '0.7975', 'num_tokens': '4.985e+06', 'mean_token_accuracy': '0.7982', 'epoch': '0.008177'}
{'loss': '0.8423', 'grad_norm': '0.379', 'learning_rate': '0.0001996', 'entropy': '0.8126', 'num_tokens': '4.99e+06', 'mean_token_accuracy': '0.8012', 'epoch': '0.008186'}
{'loss': '0.7868', 'grad_norm': '0.2691', 'learning_rate': '0.0001996', 'entropy': '0.918', 'num_tokens': '4.997e+06', 'mean_token_accuracy': '0.7834', 'epoch': '0.008195'}
{'loss': '1.223', 'grad_norm': '0.39', 'learning_rate': '0.0001996', 'entropy': '1.221', 'num_tokens': '5.003e+06', 'mean_token_accuracy': '0.7042', 'epoch': '0.008204'}
{'loss': '0.9147', 'grad_norm': '0.3259', 'learning_rate': '0.0001996', 'entropy': '0.9303', 'num_tokens': '5.011e+06', 'mean_token_accuracy': '0.7742', 'epoch': '0.008213'}
{'loss': '0.9314', 'grad_norm': '0.3963', 'learning_rate': '0.0001996', 'entropy': '0.9837', 'num_tokens': '5.014e+06', 'mean_token_accuracy': '0.7576', 'epoch': '0.008221'}
{'loss': '0.7563', 'grad_norm': '0.3413', 'learning_rate': '0.0001996', 'entropy': '0.7267', 'num_tokens': '5.02e+06', 'mean_token_accuracy': '0.8067', 'epoch': '0.00823'}
{'loss': '0.8271', 'grad_norm': '0.3488', 'learning_rate': '0.0001996', 'entropy': '0.8103', 'num_tokens': '5.024e+06', 'mean_token_accuracy': '0.7944', 'epoch': '0.008239'}
{'loss': '0.822', 'grad_norm': '0.302', 'learning_rate': '0.0001996', 'entropy': '0.8394', 'num_tokens': '5.03e+06', 'mean_token_accuracy': '0.7936', 'epoch': '0.008248'}
{'loss': '1.191', 'grad_norm': '0.4408', 'learning_rate': '0.0001996', 'entropy': '1.194', 'num_tokens': '5.034e+06', 'mean_token_accuracy': '0.7123', 'epoch': '0.008257'}
{'loss': '0.976', 'grad_norm': '0.3952', 'learning_rate': '0.0001996', 'entropy': '0.9551', 'num_tokens': '5.042e+06', 'mean_token_accuracy': '0.7721', 'epoch': '0.008266'}
{'loss': '1.192', 'grad_norm': '0.3306', 'learning_rate': '0.0001996', 'entropy': '1.164', 'num_tokens': '5.047e+06', 'mean_token_accuracy': '0.731', 'epoch': '0.008275'}
{'loss': '0.7006', 'grad_norm': '0.3469', 'learning_rate': '0.0001996', 'entropy': '0.7491', 'num_tokens': '5.052e+06', 'mean_token_accuracy': '0.8177', 'epoch': '0.008283'}
{'loss': '0.9388', 'grad_norm': '0.3582', 'learning_rate': '0.0001996', 'entropy': '0.9422', 'num_tokens': '5.058e+06', 'mean_token_accuracy': '0.7695', 'epoch': '0.008292'}
{'loss': '0.9232', 'grad_norm': '0.2759', 'learning_rate': '0.0001996', 'entropy': '0.9659', 'num_tokens': '5.067e+06', 'mean_token_accuracy': '0.7773', 'epoch': '0.008301'}
{'loss': '1.078', 'grad_norm': '0.3926', 'learning_rate': '0.0001996', 'entropy': '0.9967', 'num_tokens': '5.072e+06', 'mean_token_accuracy': '0.7561', 'epoch': '0.00831'}
{'loss': '0.9712', 'grad_norm': '0.2917', 'learning_rate': '0.0001996', 'entropy': '0.9718', 'num_tokens': '5.08e+06', 'mean_token_accuracy': '0.7649', 'epoch': '0.008319'}
{'loss': '1.166', 'grad_norm': '0.4358', 'learning_rate': '0.0001996', 'entropy': '1.145', 'num_tokens': '5.085e+06', 'mean_token_accuracy': '0.7247', 'epoch': '0.008328'}
{'loss': '1.08', 'grad_norm': '0.4079', 'learning_rate': '0.0001996', 'entropy': '1.054', 'num_tokens': '5.089e+06', 'mean_token_accuracy': '0.7589', 'epoch': '0.008337'}
{'loss': '1.126', 'grad_norm': '0.359', 'learning_rate': '0.0001996', 'entropy': '1.062', 'num_tokens': '5.096e+06', 'mean_token_accuracy': '0.7303', 'epoch': '0.008345'}

  8%|▊         | 944/12000 [33:43<5:53:23,  1.92s/it]
  8%|▊         | 945/12000 [33:46<7:26:11,  2.42s/it]
                                                     

  8%|▊         | 945/12000 [33:46<7:26:11,  2.42s/it]
  8%|▊         | 946/12000 [33:48<6:38:50,  2.16s/it]
                                                     

  8%|▊         | 946/12000 [33:48<6:38:50,  2.16s/it]
  8%|▊         | 947/12000 [33:51<7:08:39,  2.33s/it]
                                                     

  8%|▊         | 947/12000 [33:51<7:08:39,  2.33s/it]
  8%|▊         | 948/12000 [33:52<6:34:22,  2.14s/it]
                                                     

  8%|▊         | 948/12000 [33:52<6:34:22,  2.14s/it]
  8%|▊         | 949/12000 [33:54<5:53:20,  1.92s/it]
                                                     

  8%|▊         | 949/12000 [33:54<5:53:20,  1.92s/it]
  8%|▊         | 950/12000 [33:56<5:54:21,  1.92s/it]
                                                     

  8%|▊         | 950/12000 [33:56<5:54:21,  1.92s/it]
  8%|▊         | 951/12000 [34:00<7:45:19,  2.53s/it]
                                                     

  8%|▊         | 951/12000 [34:00<7:45:19,  2.53s/it]
  8%|▊         | 952/12000 [34:02<7:13:57,  2.36s/it]
                                                     

  8%|▊         | 952/12000 [34:02<7:13:57,  2.36s/it]
  8%|▊         | 953/12000 [34:04<7:06:42,  2.32s/it]
                                                     

  8%|▊         | 953/12000 [34:04<7:06:42,  2.32s/it]
  8%|▊         | 954/12000 [34:05<6:24:41,  2.09s/it]
                                                     

  8%|▊         | 954/12000 [34:05<6:24:41,  2.09s/it]
  8%|▊         | 955/12000 [34:08<6:36:56,  2.16s/it]
                                                     

  8%|▊         | 955/12000 [34:08<6:36:56,  2.16s/it]
  8%|▊         | 956/12000 [34:10<6:36:31,  2.15s/it]
                                                     

  8%|▊         | 956/12000 [34:10<6:36:31,  2.15s/it]
  8%|▊         | 957/12000 [34:13<7:40:49,  2.50s/it]
                                                     

  8%|▊         | 957/12000 [34:13<7:40:49,  2.50s/it]
  8%|▊         | 958/12000 [34:16<8:12:37,  2.68s/it]
                                                     

  8%|▊         | 958/12000 [34:16<8:12:37,  2.68s/it]
  8%|▊         | 959/12000 [34:19<7:55:08,  2.58s/it]
                                                     

  8%|▊         | 959/12000 [34:19<7:55:08,  2.58s/it]
  8%|▊         | 960/12000 [34:21<7:21:50,  2.40s/it]
                                                     

  8%|▊         | 960/12000 [34:21<7:21:50,  2.40s/it]
  8%|▊         | 961/12000 [34:23<7:09:29,  2.33s/it]
                                                     

  8%|▊         | 961/12000 [34:23<7:09:29,  2.33s/it]
  8%|▊         | 962/12000 [34:24<6:18:58,  2.06s/it]
                                                     

  8%|▊         | 962/12000 [34:24<6:18:58,  2.06s/it]
  8%|▊         | 963/12000 [34:26<5:56:43,  1.94s/it]
                                                     

  8%|▊         | 963/12000 [34:26<5:56:43,  1.94s/it]
  8%|▊         | 964/12000 [34:28<6:19:13,  2.06s/it]
                                                     

  8%|▊         | 964/12000 [34:28<6:19:13,  2.06s/it]
  8%|▊         | 965/12000 [34:30<6:14:23,  2.04s/it]
                                                     

  8%|▊         | 965/12000 [34:30<6:14:23,  2.04s/it]
  8%|▊         | 966/12000 [34:32<5:47:54,  1.89s/it]
                                                     

  8%|▊         | 966/12000 [34:32<5:47:54,  1.89s/it]
  8%|▊         | 967/12000 [34:35<7:13:56,  2.36s/it]
                                                     

  8%|▊         | 967/12000 [34:35<7:13:56,  2.36s/it]
  8%|▊         | 968/12000 [34:37<6:24:09,  2.09s/it]
                                                     

  8%|▊         | 968/12000 [34:37<6:24:09,  2.09s/it]
  8%|▊         | 969/12000 [34:39<6:33:37,  2.14s/it]
                                                     

  8%|▊         | 969/12000 [34:39<6:33:37,  2.14s/it]
  8%|▊         | 970/12000 [34:42<7:26:47,  2.43s/it]
                                                     

  8%|▊         | 970/12000 [34:42<7:26:47,  2.43s/it]
  8%|▊         | 971/12000 [34:44<6:43:20,  2.19s/it]
                                                     

  8%|▊         | 971/12000 [34:44<6:43:20,  2.19s/it]
  8%|▊         | 972/12000 [34:48<9:00:11,  2.94s/it]
                                                     

  8%|▊         | 972/12000 [34:48<9:00:11,  2.94s/it]
  8%|▊         | 973/12000 [34:50<8:09:02,  2.66s/it]
                                                     

  8%|▊         | 973/12000 [34:50<8:09:02,  2.66s/it]
  8%|▊         | 974/12000 [34:52<7:02:18,  2.30s/it]
                                                     

  8%|▊         | 974/12000 [34:52<7:02:18,  2.30s/it]
  8%|▊         | 975/12000 [34:53<6:28:11,  2.11s/it]
                                                     

  8%|▊         | 975/12000 [34:53<6:28:11,  2.11s/it]
  8%|▊         | 976/12000 [34:55<6:19:52,  2.07s/it]
                                                     

  8%|▊         | 976/12000 [34:55<6:19:52,  2.07s/it]
  8%|▊         | 977/12000 [34:57<5:45:05,  1.88s/it]
                                                     

  8%|▊         | 977/12000 [34:57<5:45:05,  1.88s/it]
  8%|▊         | 978/12000 [35:01<7:42:14,  2.52s/it]
                                                     

  8%|▊         | 978/12000 [35:01<7:42:14,  2.52s/it]
  8%|▊         | 979/12000 [35:03<6:56:32,  2.27s/it]
                                                     

  8%|▊         | 979/12000 [35:03<6:56:32,  2.27s/it]
  8%|▊         | 980/12000 [35:06<7:55:25,  2.59s/it]
                                                     

  8%|▊         | 980/12000 [35:06<7:55:25,  2.59s/it]
  8%|▊         | 981/12000 [35:08<7:51:01,  2.56s/it]
                                                     

  8%|▊         | 981/12000 [35:08<7:51:01,  2.56s/it]
  8%|▊         | 982/12000 [35:10<7:05:08,  2.32s/it]
                                                     

  8%|▊         | 982/12000 [35:10<7:05:08,  2.32s/it]
  8%|▊         | 983/12000 [35:12<6:24:55,  2.10s/it]
                                                     

  8%|▊         | 983/12000 [35:12<6:24:55,  2.10s/it]
  8%|▊         | 984/12000 [35:13<6:04:04,  1.98s/it]
                                                     

  8%|▊         | 984/12000 [35:13<6:04:04,  1.98s/it]
  8%|▊         | 985/12000 [35:15<5:42:02,  1.86s/it]
                                                     

  8%|▊         | 985/12000 [35:15<5:42:02,  1.86s/it]
  8%|▊         | 986/12000 [35:17<6:05:43,  1.99s/it]
                                                     

  8%|▊         | 986/12000 [35:17<6:05:43,  1.99s/it]
  8%|▊         | 987/12000 [35:19<5:57:20,  1.95s/it]
                                                     

  8%|▊         | 987/12000 [35:19<5:57:20,  1.95s/it]
  8%|▊         | 988/12000 [35:21<5:41:40,  1.86s/it]
                                                     

  8%|▊         | 988/12000 [35:21<5:41:40,  1.86s/it]
  8%|▊         | 989/12000 [35:24<7:14:00,  2.36s/it]
                                                     

  8%|▊         | 989/12000 [35:24<7:14:00,  2.36s/it]
  8%|▊         | 990/12000 [35:26<6:29:14,  2.12s/it]
                                                     

  8%|▊         | 990/12000 [35:26<6:29:14,  2.12s/it]
  8%|▊         | 991/12000 [35:27<5:46:55,  1.89s/it]
                                                     
{'loss': '0.8895', 'grad_norm': '0.3422', 'learning_rate': '0.0001996', 'entropy': '0.9625', 'num_tokens': '5.099e+06', 'mean_token_accuracy': '0.7554', 'epoch': '0.008354'}
{'loss': '0.8438', 'grad_norm': '0.4073', 'learning_rate': '0.0001996', 'entropy': '0.8277', 'num_tokens': '5.104e+06', 'mean_token_accuracy': '0.7807', 'epoch': '0.008363'}
{'loss': '0.7414', 'grad_norm': '0.3558', 'learning_rate': '0.0001995', 'entropy': '0.7533', 'num_tokens': '5.108e+06', 'mean_token_accuracy': '0.8055', 'epoch': '0.008372'}
{'loss': '1.008', 'grad_norm': '0.3391', 'learning_rate': '0.0001995', 'entropy': '0.9929', 'num_tokens': '5.116e+06', 'mean_token_accuracy': '0.7601', 'epoch': '0.008381'}
{'loss': '0.9801', 'grad_norm': '0.3595', 'learning_rate': '0.0001995', 'entropy': '0.9357', 'num_tokens': '5.121e+06', 'mean_token_accuracy': '0.766', 'epoch': '0.00839'}
{'loss': '0.8456', 'grad_norm': '0.4258', 'learning_rate': '0.0001995', 'entropy': '0.8108', 'num_tokens': '5.125e+06', 'mean_token_accuracy': '0.7922', 'epoch': '0.008398'}
{'loss': '1.375', 'grad_norm': '0.3906', 'learning_rate': '0.0001995', 'entropy': '1.238', 'num_tokens': '5.129e+06', 'mean_token_accuracy': '0.7042', 'epoch': '0.008407'}
{'loss': '0.7326', 'grad_norm': '0.3643', 'learning_rate': '0.0001995', 'entropy': '0.9433', 'num_tokens': '5.135e+06', 'mean_token_accuracy': '0.7787', 'epoch': '0.008416'}
{'loss': '0.9968', 'grad_norm': '0.3424', 'learning_rate': '0.0001995', 'entropy': '0.9888', 'num_tokens': '5.14e+06', 'mean_token_accuracy': '0.7549', 'epoch': '0.008425'}
{'loss': '0.9808', 'grad_norm': '0.389', 'learning_rate': '0.0001995', 'entropy': '0.9542', 'num_tokens': '5.146e+06', 'mean_token_accuracy': '0.7851', 'epoch': '0.008434'}
{'loss': '1.007', 'grad_norm': '0.2838', 'learning_rate': '0.0001995', 'entropy': '0.9724', 'num_tokens': '5.15e+06', 'mean_token_accuracy': '0.7511', 'epoch': '0.008443'}
{'loss': '0.8736', 'grad_norm': '0.3943', 'learning_rate': '0.0001995', 'entropy': '0.8496', 'num_tokens': '5.156e+06', 'mean_token_accuracy': '0.7883', 'epoch': '0.008452'}
{'loss': '0.9699', 'grad_norm': '0.3146', 'learning_rate': '0.0001995', 'entropy': '0.9525', 'num_tokens': '5.162e+06', 'mean_token_accuracy': '0.7728', 'epoch': '0.00846'}
{'loss': '0.9254', 'grad_norm': '0.3535', 'learning_rate': '0.0001995', 'entropy': '0.8473', 'num_tokens': '5.17e+06', 'mean_token_accuracy': '0.7871', 'epoch': '0.008469'}
{'loss': '1.061', 'grad_norm': '0.3395', 'learning_rate': '0.0001995', 'entropy': '1.056', 'num_tokens': '5.178e+06', 'mean_token_accuracy': '0.7448', 'epoch': '0.008478'}
{'loss': '0.831', 'grad_norm': '0.3579', 'learning_rate': '0.0001995', 'entropy': '0.8893', 'num_tokens': '5.184e+06', 'mean_token_accuracy': '0.7719', 'epoch': '0.008487'}
{'loss': '0.9102', 'grad_norm': '0.335', 'learning_rate': '0.0001995', 'entropy': '0.9309', 'num_tokens': '5.189e+06', 'mean_token_accuracy': '0.7713', 'epoch': '0.008496'}
{'loss': '1.151', 'grad_norm': '0.5312', 'learning_rate': '0.0001995', 'entropy': '0.9555', 'num_tokens': '5.195e+06', 'mean_token_accuracy': '0.7606', 'epoch': '0.008505'}
{'loss': '0.97', 'grad_norm': '0.3592', 'learning_rate': '0.0001995', 'entropy': '1.024', 'num_tokens': '5.199e+06', 'mean_token_accuracy': '0.7494', 'epoch': '0.008514'}
{'loss': '0.7265', 'grad_norm': '2.738', 'learning_rate': '0.0001995', 'entropy': '0.8674', 'num_tokens': '5.204e+06', 'mean_token_accuracy': '0.7888', 'epoch': '0.008522'}
{'loss': '0.9767', 'grad_norm': '0.3009', 'learning_rate': '0.0001995', 'entropy': '1.15', 'num_tokens': '5.209e+06', 'mean_token_accuracy': '0.7337', 'epoch': '0.008531'}
{'loss': '1.22', 'grad_norm': '0.3382', 'learning_rate': '0.0001995', 'entropy': '1.168', 'num_tokens': '5.215e+06', 'mean_token_accuracy': '0.7171', 'epoch': '0.00854'}
{'loss': '1.089', 'grad_norm': '0.417', 'learning_rate': '0.0001995', 'entropy': '1.049', 'num_tokens': '5.218e+06', 'mean_token_accuracy': '0.7465', 'epoch': '0.008549'}
{'loss': '0.9622', 'grad_norm': '0.3915', 'learning_rate': '0.0001995', 'entropy': '0.9817', 'num_tokens': '5.223e+06', 'mean_token_accuracy': '0.767', 'epoch': '0.008558'}
{'loss': '1.352', 'grad_norm': '0.3998', 'learning_rate': '0.0001995', 'entropy': '1.436', 'num_tokens': '5.227e+06', 'mean_token_accuracy': '0.669', 'epoch': '0.008567'}
{'loss': '1.001', 'grad_norm': '0.4118', 'learning_rate': '0.0001995', 'entropy': '0.9899', 'num_tokens': '5.234e+06', 'mean_token_accuracy': '0.7676', 'epoch': '0.008575'}
{'loss': '0.7865', 'grad_norm': '0.3203', 'learning_rate': '0.0001995', 'entropy': '0.8555', 'num_tokens': '5.241e+06', 'mean_token_accuracy': '0.7921', 'epoch': '0.008584'}
{'loss': '0.7465', 'grad_norm': '0.3622', 'learning_rate': '0.0001995', 'entropy': '0.7723', 'num_tokens': '5.247e+06', 'mean_token_accuracy': '0.8113', 'epoch': '0.008593'}
{'loss': '1.706', 'grad_norm': '0.4032', 'learning_rate': '0.0001995', 'entropy': '1.472', 'num_tokens': '5.254e+06', 'mean_token_accuracy': '0.6686', 'epoch': '0.008602'}
{'loss': '0.7987', 'grad_norm': '0.3822', 'learning_rate': '0.0001995', 'entropy': '0.7842', 'num_tokens': '5.26e+06', 'mean_token_accuracy': '0.7884', 'epoch': '0.008611'}
{'loss': '1.254', 'grad_norm': '0.4586', 'learning_rate': '0.0001995', 'entropy': '1.168', 'num_tokens': '5.264e+06', 'mean_token_accuracy': '0.7122', 'epoch': '0.00862'}
{'loss': '0.6257', 'grad_norm': '0.3843', 'learning_rate': '0.0001995', 'entropy': '0.6314', 'num_tokens': '5.268e+06', 'mean_token_accuracy': '0.8391', 'epoch': '0.008629'}
{'loss': '1.417', 'grad_norm': '0.4649', 'learning_rate': '0.0001995', 'entropy': '1.298', 'num_tokens': '5.273e+06', 'mean_token_accuracy': '0.6825', 'epoch': '0.008637'}
{'loss': '0.8208', 'grad_norm': '0.4298', 'learning_rate': '0.0001995', 'entropy': '0.8168', 'num_tokens': '5.278e+06', 'mean_token_accuracy': '0.7967', 'epoch': '0.008646'}
{'loss': '1.299', 'grad_norm': '0.3319', 'learning_rate': '0.0001995', 'entropy': '1.403', 'num_tokens': '5.286e+06', 'mean_token_accuracy': '0.6823', 'epoch': '0.008655'}
{'loss': '1.137', 'grad_norm': '0.4531', 'learning_rate': '0.0001995', 'entropy': '1.141', 'num_tokens': '5.291e+06', 'mean_token_accuracy': '0.7185', 'epoch': '0.008664'}
{'loss': '1.392', 'grad_norm': '0.3525', 'learning_rate': '0.0001995', 'entropy': '1.118', 'num_tokens': '5.299e+06', 'mean_token_accuracy': '0.7381', 'epoch': '0.008673'}
{'loss': '0.9787', 'grad_norm': '0.3838', 'learning_rate': '0.0001995', 'entropy': '0.9715', 'num_tokens': '5.305e+06', 'mean_token_accuracy': '0.7749', 'epoch': '0.008682'}
{'loss': '1.088', 'grad_norm': '0.3412', 'learning_rate': '0.0001994', 'entropy': '1.115', 'num_tokens': '5.31e+06', 'mean_token_accuracy': '0.7278', 'epoch': '0.008691'}
{'loss': '0.8791', 'grad_norm': '0.3051', 'learning_rate': '0.0001994', 'entropy': '0.9337', 'num_tokens': '5.315e+06', 'mean_token_accuracy': '0.7717', 'epoch': '0.008699'}
{'loss': '1.079', 'grad_norm': '0.4482', 'learning_rate': '0.0001994', 'entropy': '1.043', 'num_tokens': '5.32e+06', 'mean_token_accuracy': '0.7822', 'epoch': '0.008708'}
{'loss': '0.8828', 'grad_norm': '0.4172', 'learning_rate': '0.0001994', 'entropy': '0.9189', 'num_tokens': '5.324e+06', 'mean_token_accuracy': '0.7712', 'epoch': '0.008717'}
{'loss': '1.253', 'grad_norm': '0.407', 'learning_rate': '0.0001994', 'entropy': '1.11', 'num_tokens': '5.329e+06', 'mean_token_accuracy': '0.7478', 'epoch': '0.008726'}
{'loss': '0.9078', 'grad_norm': '0.3104', 'learning_rate': '0.0001994', 'entropy': '0.9302', 'num_tokens': '5.335e+06', 'mean_token_accuracy': '0.7624', 'epoch': '0.008735'}
{'loss': '0.7988', 'grad_norm': '0.3403', 'learning_rate': '0.0001994', 'entropy': '0.9033', 'num_tokens': '5.34e+06', 'mean_token_accuracy': '0.7758', 'epoch': '0.008744'}
{'loss': '0.8923', 'grad_norm': '0.4413', 'learning_rate': '0.0001994', 'entropy': '0.9349', 'num_tokens': '5.344e+06', 'mean_token_accuracy': '0.7628', 'epoch': '0.008752'}
{'loss': '0.8916', 'grad_norm': '0.3966', 'learning_rate': '0.0001994', 'entropy': '0.9276', 'num_tokens': '5.349e+06', 'mean_token_accuracy': '0.7586', 'epoch': '0.008761'}

  8%|▊         | 991/12000 [35:27<5:46:55,  1.89s/it]
  8%|▊         | 992/12000 [35:29<5:34:03,  1.82s/it]
                                                     

  8%|▊         | 992/12000 [35:29<5:34:03,  1.82s/it]
  8%|▊         | 993/12000 [35:32<6:29:44,  2.12s/it]
                                                     

  8%|▊         | 993/12000 [35:32<6:29:44,  2.12s/it]
  8%|▊         | 994/12000 [35:34<6:38:10,  2.17s/it]
                                                     

  8%|▊         | 994/12000 [35:34<6:38:10,  2.17s/it]
  8%|▊         | 995/12000 [35:37<7:20:52,  2.40s/it]
                                                     

  8%|▊         | 995/12000 [35:37<7:20:52,  2.40s/it]
  8%|▊         | 996/12000 [35:39<6:43:43,  2.20s/it]
                                                     

  8%|▊         | 996/12000 [35:39<6:43:43,  2.20s/it]
  8%|▊         | 997/12000 [35:42<7:29:30,  2.45s/it]
                                                     

  8%|▊         | 997/12000 [35:42<7:29:30,  2.45s/it]
  8%|▊         | 998/12000 [35:44<7:13:13,  2.36s/it]
                                                     

  8%|▊         | 998/12000 [35:44<7:13:13,  2.36s/it]
  8%|▊         | 999/12000 [35:45<6:16:26,  2.05s/it]
                                                     

  8%|▊         | 999/12000 [35:45<6:16:26,  2.05s/it]
  8%|▊         | 1000/12000 [35:47<5:41:37,  1.86s/it]
                                                      

  8%|▊         | 1000/12000 [35:47<5:41:37,  1.86s/it]
  8%|▊         | 1001/12000 [35:51<7:40:47,  2.51s/it]
                                                      

  8%|▊         | 1001/12000 [35:51<7:40:47,  2.51s/it]
  8%|▊         | 1002/12000 [35:52<6:58:45,  2.28s/it]
                                                      

  8%|▊         | 1002/12000 [35:52<6:58:45,  2.28s/it]
  8%|▊         | 1003/12000 [35:55<6:49:00,  2.23s/it]
                                                      

  8%|▊         | 1003/12000 [35:55<6:49:00,  2.23s/it]
  8%|▊         | 1004/12000 [35:56<6:21:02,  2.08s/it]
                                                      

  8%|▊         | 1004/12000 [35:56<6:21:02,  2.08s/it]
  8%|▊         | 1005/12000 [35:58<6:04:50,  1.99s/it]
                                                      

  8%|▊         | 1005/12000 [35:58<6:04:50,  1.99s/it]
  8%|▊         | 1006/12000 [36:00<5:57:35,  1.95s/it]
                                                      

  8%|▊         | 1006/12000 [36:00<5:57:35,  1.95s/it]
  8%|▊         | 1007/12000 [36:03<6:55:10,  2.27s/it]
                                                      

  8%|▊         | 1007/12000 [36:03<6:55:10,  2.27s/it]
  8%|▊         | 1008/12000 [36:05<6:23:01,  2.09s/it]
                                                      

  8%|▊         | 1008/12000 [36:05<6:23:01,  2.09s/it]
  8%|▊         | 1009/12000 [36:07<6:13:57,  2.04s/it]
                                                      

  8%|▊         | 1009/12000 [36:07<6:13:57,  2.04s/it]
  8%|▊         | 1010/12000 [36:09<6:10:35,  2.02s/it]
                                                      

  8%|▊         | 1010/12000 [36:09<6:10:35,  2.02s/it]
  8%|▊         | 1011/12000 [36:12<7:26:07,  2.44s/it]
                                                      

  8%|▊         | 1011/12000 [36:12<7:26:07,  2.44s/it]
  8%|▊         | 1012/12000 [36:14<7:13:11,  2.37s/it]
                                                      

  8%|▊         | 1012/12000 [36:14<7:13:11,  2.37s/it]
  8%|▊         | 1013/12000 [36:16<6:21:36,  2.08s/it]
                                                      

  8%|▊         | 1013/12000 [36:16<6:21:36,  2.08s/it]
  8%|▊         | 1014/12000 [36:17<6:05:52,  2.00s/it]
                                                      

  8%|▊         | 1014/12000 [36:17<6:05:52,  2.00s/it]
  8%|▊         | 1015/12000 [36:19<5:36:43,  1.84s/it]
                                                      

  8%|▊         | 1015/12000 [36:19<5:36:43,  1.84s/it]
  8%|▊         | 1016/12000 [36:20<5:15:23,  1.72s/it]
                                                      

  8%|▊         | 1016/12000 [36:20<5:15:23,  1.72s/it]
  8%|▊         | 1017/12000 [36:25<7:42:45,  2.53s/it]
                                                      

  8%|▊         | 1017/12000 [36:25<7:42:45,  2.53s/it]
  8%|▊         | 1018/12000 [36:26<6:51:14,  2.25s/it]
                                                      

  8%|▊         | 1018/12000 [36:26<6:51:14,  2.25s/it]
  8%|▊         | 1019/12000 [36:28<6:20:52,  2.08s/it]
                                                      

  8%|▊         | 1019/12000 [36:28<6:20:52,  2.08s/it]
  8%|▊         | 1020/12000 [36:30<6:06:58,  2.01s/it]
                                                      

  8%|▊         | 1020/12000 [36:30<6:06:58,  2.01s/it]
  9%|▊         | 1021/12000 [36:31<5:42:28,  1.87s/it]
                                                      

  9%|▊         | 1021/12000 [36:31<5:42:28,  1.87s/it]
  9%|▊         | 1022/12000 [36:33<5:37:27,  1.84s/it]
                                                      

  9%|▊         | 1022/12000 [36:33<5:37:27,  1.84s/it]
  9%|▊         | 1023/12000 [36:36<6:39:11,  2.18s/it]
                                                      

  9%|▊         | 1023/12000 [36:36<6:39:11,  2.18s/it]
  9%|▊         | 1024/12000 [36:38<6:06:37,  2.00s/it]
                                                      

  9%|▊         | 1024/12000 [36:38<6:06:37,  2.00s/it]
  9%|▊         | 1025/12000 [36:40<5:59:15,  1.96s/it]
                                                      

  9%|▊         | 1025/12000 [36:40<5:59:15,  1.96s/it]
  9%|▊         | 1026/12000 [36:42<6:23:15,  2.10s/it]
                                                      

  9%|▊         | 1026/12000 [36:42<6:23:15,  2.10s/it]
  9%|▊         | 1027/12000 [36:45<6:50:47,  2.25s/it]
                                                      

  9%|▊         | 1027/12000 [36:45<6:50:47,  2.25s/it]
  9%|▊         | 1028/12000 [36:47<6:47:54,  2.23s/it]
                                                      

  9%|▊         | 1028/12000 [36:47<6:47:54,  2.23s/it]
  9%|▊         | 1029/12000 [36:50<7:50:25,  2.57s/it]
                                                      

  9%|▊         | 1029/12000 [36:50<7:50:25,  2.57s/it]
  9%|▊         | 1030/12000 [36:52<7:05:32,  2.33s/it]
                                                      

  9%|▊         | 1030/12000 [36:52<7:05:32,  2.33s/it]
  9%|▊         | 1031/12000 [36:54<6:35:14,  2.16s/it]
                                                      

  9%|▊         | 1031/12000 [36:54<6:35:14,  2.16s/it]
  9%|▊         | 1032/12000 [36:55<5:59:28,  1.97s/it]
                                                      

  9%|▊         | 1032/12000 [36:55<5:59:28,  1.97s/it]
  9%|▊         | 1033/12000 [36:59<7:42:29,  2.53s/it]
                                                      

  9%|▊         | 1033/12000 [36:59<7:42:29,  2.53s/it]
  9%|▊         | 1034/12000 [37:00<6:40:31,  2.19s/it]
                                                      

  9%|▊         | 1034/12000 [37:00<6:40:31,  2.19s/it]
  9%|▊         | 1035/12000 [37:02<6:01:59,  1.98s/it]
                                                      

  9%|▊         | 1035/12000 [37:02<6:01:59,  1.98s/it]
  9%|▊         | 1036/12000 [37:04<5:53:18,  1.93s/it]
                                                      

  9%|▊         | 1036/12000 [37:04<5:53:18,  1.93s/it]
  9%|▊         | 1037/12000 [37:06<5:54:30,  1.94s/it]
                                                      

  9%|▊         | 1037/12000 [37:06<5:54:30,  1.94s/it]
  9%|▊         | 1038/12000 [37:08<6:03:40,  1.99s/it]
                                                      
{'loss': '0.9713', 'grad_norm': '0.4777', 'learning_rate': '0.0001994', 'entropy': '0.9365', 'num_tokens': '5.352e+06', 'mean_token_accuracy': '0.7646', 'epoch': '0.00877'}
{'loss': '0.8548', 'grad_norm': '0.3281', 'learning_rate': '0.0001994', 'entropy': '0.861', 'num_tokens': '5.357e+06', 'mean_token_accuracy': '0.79', 'epoch': '0.008779'}
{'loss': '1.49', 'grad_norm': '0.3115', 'learning_rate': '0.0001994', 'entropy': '1.297', 'num_tokens': '5.364e+06', 'mean_token_accuracy': '0.6811', 'epoch': '0.008788'}
{'loss': '1.233', 'grad_norm': '0.4493', 'learning_rate': '0.0001994', 'entropy': '1.15', 'num_tokens': '5.369e+06', 'mean_token_accuracy': '0.712', 'epoch': '0.008797'}
{'loss': '1.018', 'grad_norm': '0.3024', 'learning_rate': '0.0001994', 'entropy': '1.022', 'num_tokens': '5.376e+06', 'mean_token_accuracy': '0.7553', 'epoch': '0.008806'}
{'loss': '1.224', 'grad_norm': '0.3358', 'learning_rate': '0.0001994', 'entropy': '1.28', 'num_tokens': '5.382e+06', 'mean_token_accuracy': '0.7046', 'epoch': '0.008814'}
{'loss': '1.168', 'grad_norm': '0.3637', 'learning_rate': '0.0001994', 'entropy': '1.195', 'num_tokens': '5.389e+06', 'mean_token_accuracy': '0.7317', 'epoch': '0.008823'}
{'loss': '1.084', 'grad_norm': '0.4958', 'learning_rate': '0.0001994', 'entropy': '1.037', 'num_tokens': '5.394e+06', 'mean_token_accuracy': '0.7471', 'epoch': '0.008832'}
{'loss': '0.7872', 'grad_norm': '0.5851', 'learning_rate': '0.0001994', 'entropy': '0.7986', 'num_tokens': '5.398e+06', 'mean_token_accuracy': '0.8024', 'epoch': '0.008841'}
{'loss': '0.9657', 'grad_norm': '0.4298', 'learning_rate': '0.0001994', 'entropy': '1.004', 'num_tokens': '5.401e+06', 'mean_token_accuracy': '0.7625', 'epoch': '0.00885'}
{'loss': '0.755', 'grad_norm': '0.4428', 'learning_rate': '0.0001994', 'entropy': '0.8631', 'num_tokens': '5.406e+06', 'mean_token_accuracy': '0.7895', 'epoch': '0.008859'}
{'loss': '1.418', 'grad_norm': '0.357', 'learning_rate': '0.0001994', 'entropy': '1.317', 'num_tokens': '5.41e+06', 'mean_token_accuracy': '0.7086', 'epoch': '0.008868'}
{'loss': '1.194', 'grad_norm': '0.3991', 'learning_rate': '0.0001994', 'entropy': '1.084', 'num_tokens': '5.416e+06', 'mean_token_accuracy': '0.7435', 'epoch': '0.008876'}
{'loss': '0.9964', 'grad_norm': '0.4404', 'learning_rate': '0.0001994', 'entropy': '0.993', 'num_tokens': '5.42e+06', 'mean_token_accuracy': '0.744', 'epoch': '0.008885'}
{'loss': '1.073', 'grad_norm': '0.4567', 'learning_rate': '0.0001994', 'entropy': '1.052', 'num_tokens': '5.426e+06', 'mean_token_accuracy': '0.7433', 'epoch': '0.008894'}
{'loss': '0.6347', 'grad_norm': '0.3413', 'learning_rate': '0.0001994', 'entropy': '0.7046', 'num_tokens': '5.431e+06', 'mean_token_accuracy': '0.8186', 'epoch': '0.008903'}
{'loss': '0.7619', 'grad_norm': '0.3794', 'learning_rate': '0.0001994', 'entropy': '0.7038', 'num_tokens': '5.438e+06', 'mean_token_accuracy': '0.8255', 'epoch': '0.008912'}
{'loss': '1.047', 'grad_norm': '0.4269', 'learning_rate': '0.0001994', 'entropy': '1.01', 'num_tokens': '5.443e+06', 'mean_token_accuracy': '0.7523', 'epoch': '0.008921'}
{'loss': '0.9703', 'grad_norm': '0.4128', 'learning_rate': '0.0001994', 'entropy': '0.9305', 'num_tokens': '5.449e+06', 'mean_token_accuracy': '0.7677', 'epoch': '0.008929'}
{'loss': '0.866', 'grad_norm': '15.68', 'learning_rate': '0.0001994', 'entropy': '0.775', 'num_tokens': '5.454e+06', 'mean_token_accuracy': '0.7976', 'epoch': '0.008938'}
{'loss': '1.031', 'grad_norm': '0.3384', 'learning_rate': '0.0001994', 'entropy': '1.082', 'num_tokens': '5.459e+06', 'mean_token_accuracy': '0.7392', 'epoch': '0.008947'}
{'loss': '0.8488', 'grad_norm': '0.3531', 'learning_rate': '0.0001994', 'entropy': '0.9499', 'num_tokens': '5.465e+06', 'mean_token_accuracy': '0.7593', 'epoch': '0.008956'}
{'loss': '0.7966', 'grad_norm': '0.6909', 'learning_rate': '0.0001994', 'entropy': '0.8098', 'num_tokens': '5.469e+06', 'mean_token_accuracy': '0.7955', 'epoch': '0.008965'}
{'loss': '1.125', 'grad_norm': '0.5036', 'learning_rate': '0.0001994', 'entropy': '1.092', 'num_tokens': '5.473e+06', 'mean_token_accuracy': '0.7266', 'epoch': '0.008974'}
{'loss': '0.595', 'grad_norm': '0.2913', 'learning_rate': '0.0001993', 'entropy': '0.6172', 'num_tokens': '5.477e+06', 'mean_token_accuracy': '0.8329', 'epoch': '0.008983'}
{'loss': '0.7718', 'grad_norm': '0.4103', 'learning_rate': '0.0001993', 'entropy': '0.7769', 'num_tokens': '5.482e+06', 'mean_token_accuracy': '0.7981', 'epoch': '0.008991'}
{'loss': '0.9238', 'grad_norm': '0.6902', 'learning_rate': '0.0001993', 'entropy': '0.8317', 'num_tokens': '5.491e+06', 'mean_token_accuracy': '0.7886', 'epoch': '0.009'}
{'loss': '0.8651', 'grad_norm': '0.4943', 'learning_rate': '0.0001993', 'entropy': '0.8379', 'num_tokens': '5.495e+06', 'mean_token_accuracy': '0.7901', 'epoch': '0.009009'}
{'loss': '0.7416', 'grad_norm': '0.3726', 'learning_rate': '0.0001993', 'entropy': '0.7834', 'num_tokens': '5.5e+06', 'mean_token_accuracy': '0.8043', 'epoch': '0.009018'}
{'loss': '0.9627', 'grad_norm': '0.4201', 'learning_rate': '0.0001993', 'entropy': '1.033', 'num_tokens': '5.504e+06', 'mean_token_accuracy': '0.7667', 'epoch': '0.009027'}
{'loss': '1.036', 'grad_norm': '0.4573', 'learning_rate': '0.0001993', 'entropy': '1.07', 'num_tokens': '5.508e+06', 'mean_token_accuracy': '0.7373', 'epoch': '0.009036'}
{'loss': '0.9343', 'grad_norm': '0.4952', 'learning_rate': '0.0001993', 'entropy': '0.944', 'num_tokens': '5.514e+06', 'mean_token_accuracy': '0.7675', 'epoch': '0.009045'}
{'loss': '0.9207', 'grad_norm': '0.3385', 'learning_rate': '0.0001993', 'entropy': '1.069', 'num_tokens': '5.52e+06', 'mean_token_accuracy': '0.7498', 'epoch': '0.009053'}
{'loss': '1.08', 'grad_norm': '0.4413', 'learning_rate': '0.0001993', 'entropy': '1.015', 'num_tokens': '5.524e+06', 'mean_token_accuracy': '0.7458', 'epoch': '0.009062'}
{'loss': '1.029', 'grad_norm': '0.4057', 'learning_rate': '0.0001993', 'entropy': '1.004', 'num_tokens': '5.529e+06', 'mean_token_accuracy': '0.755', 'epoch': '0.009071'}
{'loss': '1.273', 'grad_norm': '0.3634', 'learning_rate': '0.0001993', 'entropy': '1.244', 'num_tokens': '5.536e+06', 'mean_token_accuracy': '0.7258', 'epoch': '0.00908'}
{'loss': '1.119', 'grad_norm': '0.3396', 'learning_rate': '0.0001993', 'entropy': '1.039', 'num_tokens': '5.542e+06', 'mean_token_accuracy': '0.7516', 'epoch': '0.009089'}
{'loss': '0.98', 'grad_norm': '0.3922', 'learning_rate': '0.0001993', 'entropy': '0.9565', 'num_tokens': '5.548e+06', 'mean_token_accuracy': '0.7696', 'epoch': '0.009098'}
{'loss': '0.7296', 'grad_norm': '0.3441', 'learning_rate': '0.0001993', 'entropy': '0.8439', 'num_tokens': '5.555e+06', 'mean_token_accuracy': '0.7849', 'epoch': '0.009106'}
{'loss': '1.07', 'grad_norm': '0.4263', 'learning_rate': '0.0001993', 'entropy': '1.061', 'num_tokens': '5.56e+06', 'mean_token_accuracy': '0.745', 'epoch': '0.009115'}
{'loss': '0.8435', 'grad_norm': '0.3606', 'learning_rate': '0.0001993', 'entropy': '0.8122', 'num_tokens': '5.566e+06', 'mean_token_accuracy': '0.7955', 'epoch': '0.009124'}
{'loss': '0.7953', 'grad_norm': '0.3172', 'learning_rate': '0.0001993', 'entropy': '0.8155', 'num_tokens': '5.57e+06', 'mean_token_accuracy': '0.7745', 'epoch': '0.009133'}
{'loss': '0.8937', 'grad_norm': '0.3813', 'learning_rate': '0.0001993', 'entropy': '0.9308', 'num_tokens': '5.576e+06', 'mean_token_accuracy': '0.7637', 'epoch': '0.009142'}
{'loss': '0.9946', 'grad_norm': '0.4104', 'learning_rate': '0.0001993', 'entropy': '1.044', 'num_tokens': '5.579e+06', 'mean_token_accuracy': '0.7574', 'epoch': '0.009151'}
{'loss': '0.8251', 'grad_norm': '0.4454', 'learning_rate': '0.0001993', 'entropy': '0.7802', 'num_tokens': '5.583e+06', 'mean_token_accuracy': '0.7983', 'epoch': '0.00916'}
{'loss': '1.122', 'grad_norm': '0.3629', 'learning_rate': '0.0001993', 'entropy': '1.084', 'num_tokens': '5.588e+06', 'mean_token_accuracy': '0.7336', 'epoch': '0.009168'}
{'loss': '0.8527', 'grad_norm': '0.4201', 'learning_rate': '0.0001993', 'entropy': '0.9249', 'num_tokens': '5.593e+06', 'mean_token_accuracy': '0.7709', 'epoch': '0.009177'}

  9%|▊         | 1038/12000 [37:08<6:03:40,  1.99s/it]
  9%|▊         | 1039/12000 [37:10<6:24:14,  2.10s/it]
                                                      

  9%|▊         | 1039/12000 [37:10<6:24:14,  2.10s/it]
  9%|▊         | 1040/12000 [37:12<6:19:26,  2.08s/it]
                                                      

  9%|▊         | 1040/12000 [37:12<6:19:26,  2.08s/it]
  9%|▊         | 1041/12000 [37:14<6:07:12,  2.01s/it]
                                                      

  9%|▊         | 1041/12000 [37:14<6:07:12,  2.01s/it]
  9%|▊         | 1042/12000 [37:16<5:42:26,  1.88s/it]
                                                      

  9%|▊         | 1042/12000 [37:16<5:42:26,  1.88s/it]
  9%|▊         | 1043/12000 [37:17<5:14:46,  1.72s/it]
                                                      

  9%|▊         | 1043/12000 [37:17<5:14:46,  1.72s/it]
  9%|▊         | 1044/12000 [37:18<5:03:08,  1.66s/it]
                                                      

  9%|▊         | 1044/12000 [37:18<5:03:08,  1.66s/it]
  9%|▊         | 1045/12000 [37:20<5:01:12,  1.65s/it]
                                                      

  9%|▊         | 1045/12000 [37:20<5:01:12,  1.65s/it]
  9%|▊         | 1046/12000 [37:22<5:05:04,  1.67s/it]
                                                      

  9%|▊         | 1046/12000 [37:22<5:05:04,  1.67s/it]
  9%|▊         | 1047/12000 [37:23<5:06:51,  1.68s/it]
                                                      

  9%|▊         | 1047/12000 [37:23<5:06:51,  1.68s/it]
  9%|▊         | 1048/12000 [37:26<5:59:12,  1.97s/it]
                                                      

  9%|▊         | 1048/12000 [37:26<5:59:12,  1.97s/it]
  9%|▊         | 1049/12000 [37:29<6:56:44,  2.28s/it]
                                                      

  9%|▊         | 1049/12000 [37:29<6:56:44,  2.28s/it]
  9%|▉         | 1050/12000 [37:31<6:37:36,  2.18s/it]
                                                      

  9%|▉         | 1050/12000 [37:31<6:37:36,  2.18s/it]
  9%|▉         | 1051/12000 [37:34<7:13:30,  2.38s/it]
                                                      

  9%|▉         | 1051/12000 [37:34<7:13:30,  2.38s/it]
  9%|▉         | 1052/12000 [37:36<6:49:04,  2.24s/it]
                                                      

  9%|▉         | 1052/12000 [37:36<6:49:04,  2.24s/it]
  9%|▉         | 1053/12000 [37:37<6:14:56,  2.06s/it]
                                                      

  9%|▉         | 1053/12000 [37:37<6:14:56,  2.06s/it]
  9%|▉         | 1054/12000 [37:41<7:26:43,  2.45s/it]
                                                      

  9%|▉         | 1054/12000 [37:41<7:26:43,  2.45s/it]
  9%|▉         | 1055/12000 [37:43<7:18:09,  2.40s/it]
                                                      

  9%|▉         | 1055/12000 [37:43<7:18:09,  2.40s/it]
  9%|▉         | 1056/12000 [37:45<6:41:20,  2.20s/it]
                                                      

  9%|▉         | 1056/12000 [37:45<6:41:20,  2.20s/it]
  9%|▉         | 1057/12000 [37:46<6:00:28,  1.98s/it]
                                                      

  9%|▉         | 1057/12000 [37:46<6:00:28,  1.98s/it]
  9%|▉         | 1058/12000 [37:49<7:03:38,  2.32s/it]
                                                      

  9%|▉         | 1058/12000 [37:49<7:03:38,  2.32s/it]
  9%|▉         | 1059/12000 [37:55<10:15:11,  3.37s/it]
                                                       

  9%|▉         | 1059/12000 [37:55<10:15:11,  3.37s/it]
  9%|▉         | 1060/12000 [37:57<8:41:58,  2.86s/it] 
                                                      

  9%|▉         | 1060/12000 [37:57<8:41:58,  2.86s/it]
  9%|▉         | 1061/12000 [37:59<8:23:48,  2.76s/it]
                                                      

  9%|▉         | 1061/12000 [37:59<8:23:48,  2.76s/it]
  9%|▉         | 1062/12000 [38:02<8:21:01,  2.75s/it]
                                                      

  9%|▉         | 1062/12000 [38:02<8:21:01,  2.75s/it]
  9%|▉         | 1063/12000 [38:04<7:30:55,  2.47s/it]
                                                      

  9%|▉         | 1063/12000 [38:04<7:30:55,  2.47s/it]
  9%|▉         | 1064/12000 [38:07<7:39:10,  2.52s/it]
                                                      

  9%|▉         | 1064/12000 [38:07<7:39:10,  2.52s/it]
  9%|▉         | 1065/12000 [38:09<7:45:19,  2.55s/it]
                                                      

  9%|▉         | 1065/12000 [38:09<7:45:19,  2.55s/it]
  9%|▉         | 1066/12000 [38:11<6:45:34,  2.23s/it]
                                                      

  9%|▉         | 1066/12000 [38:11<6:45:34,  2.23s/it]
  9%|▉         | 1067/12000 [38:13<6:30:22,  2.14s/it]
                                                      

  9%|▉         | 1067/12000 [38:13<6:30:22,  2.14s/it]
  9%|▉         | 1068/12000 [38:16<7:13:16,  2.38s/it]
                                                      

  9%|▉         | 1068/12000 [38:16<7:13:16,  2.38s/it]
  9%|▉         | 1069/12000 [38:18<7:39:08,  2.52s/it]
                                                      

  9%|▉         | 1069/12000 [38:18<7:39:08,  2.52s/it]
  9%|▉         | 1070/12000 [38:20<6:59:26,  2.30s/it]
                                                      

  9%|▉         | 1070/12000 [38:20<6:59:26,  2.30s/it]
  9%|▉         | 1071/12000 [38:22<6:40:37,  2.20s/it]
                                                      

  9%|▉         | 1071/12000 [38:22<6:40:37,  2.20s/it]
  9%|▉         | 1072/12000 [38:24<6:00:33,  1.98s/it]
                                                      

  9%|▉         | 1072/12000 [38:24<6:00:33,  1.98s/it]
  9%|▉         | 1073/12000 [38:25<5:31:49,  1.82s/it]
                                                      

  9%|▉         | 1073/12000 [38:25<5:31:49,  1.82s/it]
  9%|▉         | 1074/12000 [38:27<5:15:59,  1.74s/it]
                                                      

  9%|▉         | 1074/12000 [38:27<5:15:59,  1.74s/it]
  9%|▉         | 1075/12000 [38:28<5:01:10,  1.65s/it]
                                                      

  9%|▉         | 1075/12000 [38:28<5:01:10,  1.65s/it]
  9%|▉         | 1076/12000 [38:30<4:51:07,  1.60s/it]
                                                      

  9%|▉         | 1076/12000 [38:30<4:51:07,  1.60s/it]
  9%|▉         | 1077/12000 [38:33<6:49:27,  2.25s/it]
                                                      

  9%|▉         | 1077/12000 [38:33<6:49:27,  2.25s/it]
  9%|▉         | 1078/12000 [38:35<6:07:42,  2.02s/it]
                                                      

  9%|▉         | 1078/12000 [38:35<6:07:42,  2.02s/it]
  9%|▉         | 1079/12000 [38:37<6:08:02,  2.02s/it]
                                                      

  9%|▉         | 1079/12000 [38:37<6:08:02,  2.02s/it]
  9%|▉         | 1080/12000 [38:39<6:23:49,  2.11s/it]
                                                      

  9%|▉         | 1080/12000 [38:39<6:23:49,  2.11s/it]
  9%|▉         | 1081/12000 [38:41<5:56:02,  1.96s/it]
                                                      

  9%|▉         | 1081/12000 [38:41<5:56:02,  1.96s/it]
  9%|▉         | 1082/12000 [38:46<8:28:21,  2.79s/it]
                                                      

  9%|▉         | 1082/12000 [38:46<8:28:21,  2.79s/it]
  9%|▉         | 1083/12000 [38:47<7:32:53,  2.49s/it]
                                                      

  9%|▉         | 1083/12000 [38:47<7:32:53,  2.49s/it]
  9%|▉         | 1084/12000 [38:49<6:46:44,  2.24s/it]
                                                      

  9%|▉         | 1084/12000 [38:49<6:46:44,  2.24s/it]
  9%|▉         | 1085/12000 [38:51<6:42:49,  2.21s/it]
                                                      
{'loss': '0.6042', 'grad_norm': '0.3609', 'learning_rate': '0.0001993', 'entropy': '0.7128', 'num_tokens': '5.599e+06', 'mean_token_accuracy': '0.8361', 'epoch': '0.009186'}
{'loss': '0.9964', 'grad_norm': '0.3664', 'learning_rate': '0.0001993', 'entropy': '0.9968', 'num_tokens': '5.606e+06', 'mean_token_accuracy': '0.7683', 'epoch': '0.009195'}
{'loss': '0.9858', 'grad_norm': '0.3698', 'learning_rate': '0.0001993', 'entropy': '0.9521', 'num_tokens': '5.611e+06', 'mean_token_accuracy': '0.7607', 'epoch': '0.009204'}
{'loss': '1.348', 'grad_norm': '0.4303', 'learning_rate': '0.0001993', 'entropy': '1.3', 'num_tokens': '5.616e+06', 'mean_token_accuracy': '0.7062', 'epoch': '0.009213'}
{'loss': '1.279', 'grad_norm': '0.4164', 'learning_rate': '0.0001993', 'entropy': '1.253', 'num_tokens': '5.621e+06', 'mean_token_accuracy': '0.7039', 'epoch': '0.009222'}
{'loss': '0.7385', 'grad_norm': '0.3226', 'learning_rate': '0.0001993', 'entropy': '0.8634', 'num_tokens': '5.625e+06', 'mean_token_accuracy': '0.8026', 'epoch': '0.00923'}
{'loss': '0.8016', 'grad_norm': '0.3585', 'learning_rate': '0.0001993', 'entropy': '0.8233', 'num_tokens': '5.63e+06', 'mean_token_accuracy': '0.7994', 'epoch': '0.009239'}
{'loss': '0.9049', 'grad_norm': '0.3502', 'learning_rate': '0.0001993', 'entropy': '0.8997', 'num_tokens': '5.635e+06', 'mean_token_accuracy': '0.7754', 'epoch': '0.009248'}
{'loss': '0.9892', 'grad_norm': '0.377', 'learning_rate': '0.0001992', 'entropy': '0.9766', 'num_tokens': '5.64e+06', 'mean_token_accuracy': '0.766', 'epoch': '0.009257'}
{'loss': '1.174', 'grad_norm': '0.4517', 'learning_rate': '0.0001992', 'entropy': '1.16', 'num_tokens': '5.646e+06', 'mean_token_accuracy': '0.7208', 'epoch': '0.009266'}
{'loss': '1.133', 'grad_norm': '0.3698', 'learning_rate': '0.0001992', 'entropy': '1.15', 'num_tokens': '5.653e+06', 'mean_token_accuracy': '0.7228', 'epoch': '0.009275'}
{'loss': '1.085', 'grad_norm': '0.3093', 'learning_rate': '0.0001992', 'entropy': '1.045', 'num_tokens': '5.661e+06', 'mean_token_accuracy': '0.7583', 'epoch': '0.009283'}
{'loss': '0.688', 'grad_norm': '0.3795', 'learning_rate': '0.0001992', 'entropy': '0.6773', 'num_tokens': '5.667e+06', 'mean_token_accuracy': '0.8294', 'epoch': '0.009292'}
{'loss': '1.172', 'grad_norm': '0.4156', 'learning_rate': '0.0001992', 'entropy': '1.169', 'num_tokens': '5.671e+06', 'mean_token_accuracy': '0.743', 'epoch': '0.009301'}
{'loss': '1.058', 'grad_norm': '0.4756', 'learning_rate': '0.0001992', 'entropy': '0.9146', 'num_tokens': '5.676e+06', 'mean_token_accuracy': '0.7825', 'epoch': '0.00931'}
{'loss': '0.6075', 'grad_norm': '0.291', 'learning_rate': '0.0001992', 'entropy': '0.6243', 'num_tokens': '5.682e+06', 'mean_token_accuracy': '0.8376', 'epoch': '0.009319'}
{'loss': '1.127', 'grad_norm': '0.3941', 'learning_rate': '0.0001992', 'entropy': '1.145', 'num_tokens': '5.686e+06', 'mean_token_accuracy': '0.7387', 'epoch': '0.009328'}
{'loss': '0.9664', 'grad_norm': '0.3403', 'learning_rate': '0.0001992', 'entropy': '0.9525', 'num_tokens': '5.693e+06', 'mean_token_accuracy': '0.7735', 'epoch': '0.009337'}
{'loss': '1.333', 'grad_norm': '0.3604', 'learning_rate': '0.0001992', 'entropy': '1.416', 'num_tokens': '5.698e+06', 'mean_token_accuracy': '0.6892', 'epoch': '0.009345'}
{'loss': '0.8302', 'grad_norm': '0.309', 'learning_rate': '0.0001992', 'entropy': '0.8393', 'num_tokens': '5.703e+06', 'mean_token_accuracy': '0.7919', 'epoch': '0.009354'}
{'loss': '1.121', 'grad_norm': '0.3456', 'learning_rate': '0.0001992', 'entropy': '1.224', 'num_tokens': '5.709e+06', 'mean_token_accuracy': '0.6997', 'epoch': '0.009363'}
{'loss': '1.264', 'grad_norm': '0.5677', 'learning_rate': '0.0001992', 'entropy': '1.095', 'num_tokens': '5.721e+06', 'mean_token_accuracy': '0.7423', 'epoch': '0.009372'}
{'loss': '0.8613', 'grad_norm': '0.3177', 'learning_rate': '0.0001992', 'entropy': '0.8795', 'num_tokens': '5.725e+06', 'mean_token_accuracy': '0.782', 'epoch': '0.009381'}
{'loss': '0.9706', 'grad_norm': '0.3585', 'learning_rate': '0.0001992', 'entropy': '0.9242', 'num_tokens': '5.732e+06', 'mean_token_accuracy': '0.7653', 'epoch': '0.00939'}
{'loss': '0.7856', 'grad_norm': '0.3514', 'learning_rate': '0.0001992', 'entropy': '0.7918', 'num_tokens': '5.739e+06', 'mean_token_accuracy': '0.7892', 'epoch': '0.009399'}
{'loss': '0.9923', 'grad_norm': '0.3857', 'learning_rate': '0.0001992', 'entropy': '0.8345', 'num_tokens': '5.745e+06', 'mean_token_accuracy': '0.7789', 'epoch': '0.009407'}
{'loss': '1.028', 'grad_norm': '0.3084', 'learning_rate': '0.0001992', 'entropy': '1.137', 'num_tokens': '5.751e+06', 'mean_token_accuracy': '0.721', 'epoch': '0.009416'}
{'loss': '1.37', 'grad_norm': '0.3362', 'learning_rate': '0.0001992', 'entropy': '1.359', 'num_tokens': '5.758e+06', 'mean_token_accuracy': '0.6744', 'epoch': '0.009425'}
{'loss': '0.7289', 'grad_norm': '0.3681', 'learning_rate': '0.0001992', 'entropy': '0.7254', 'num_tokens': '5.762e+06', 'mean_token_accuracy': '0.8135', 'epoch': '0.009434'}
{'loss': '1.088', 'grad_norm': '0.3916', 'learning_rate': '0.0001992', 'entropy': '1.096', 'num_tokens': '5.769e+06', 'mean_token_accuracy': '0.7495', 'epoch': '0.009443'}
{'loss': '1.114', 'grad_norm': '0.3429', 'learning_rate': '0.0001992', 'entropy': '1.077', 'num_tokens': '5.776e+06', 'mean_token_accuracy': '0.7459', 'epoch': '0.009452'}
{'loss': '1.381', 'grad_norm': '0.3716', 'learning_rate': '0.0001992', 'entropy': '1.345', 'num_tokens': '5.783e+06', 'mean_token_accuracy': '0.699', 'epoch': '0.00946'}
{'loss': '0.8227', 'grad_norm': '0.3514', 'learning_rate': '0.0001992', 'entropy': '0.8917', 'num_tokens': '5.789e+06', 'mean_token_accuracy': '0.779', 'epoch': '0.009469'}
{'loss': '0.8858', 'grad_norm': '0.3705', 'learning_rate': '0.0001992', 'entropy': '0.8536', 'num_tokens': '5.794e+06', 'mean_token_accuracy': '0.7859', 'epoch': '0.009478'}
{'loss': '0.6969', 'grad_norm': '0.3446', 'learning_rate': '0.0001992', 'entropy': '0.7617', 'num_tokens': '5.798e+06', 'mean_token_accuracy': '0.8152', 'epoch': '0.009487'}
{'loss': '1.138', 'grad_norm': '0.4214', 'learning_rate': '0.0001992', 'entropy': '1.125', 'num_tokens': '5.803e+06', 'mean_token_accuracy': '0.7401', 'epoch': '0.009496'}
{'loss': '0.7975', 'grad_norm': '0.3678', 'learning_rate': '0.0001992', 'entropy': '0.8013', 'num_tokens': '5.807e+06', 'mean_token_accuracy': '0.7964', 'epoch': '0.009505'}
{'loss': '0.7487', 'grad_norm': '0.3526', 'learning_rate': '0.0001991', 'entropy': '0.8344', 'num_tokens': '5.812e+06', 'mean_token_accuracy': '0.804', 'epoch': '0.009514'}
{'loss': '0.9398', 'grad_norm': '0.3246', 'learning_rate': '0.0001991', 'entropy': '1.007', 'num_tokens': '5.816e+06', 'mean_token_accuracy': '0.7545', 'epoch': '0.009522'}
{'loss': '0.85', 'grad_norm': '0.3528', 'learning_rate': '0.0001991', 'entropy': '0.8093', 'num_tokens': '5.821e+06', 'mean_token_accuracy': '0.8019', 'epoch': '0.009531'}
{'loss': '1.298', 'grad_norm': '0.4907', 'learning_rate': '0.0001991', 'entropy': '1.349', 'num_tokens': '5.825e+06', 'mean_token_accuracy': '0.6758', 'epoch': '0.00954'}
{'loss': '1.067', 'grad_norm': '0.3734', 'learning_rate': '0.0001991', 'entropy': '1.072', 'num_tokens': '5.83e+06', 'mean_token_accuracy': '0.7459', 'epoch': '0.009549'}
{'loss': '1.319', 'grad_norm': '0.4799', 'learning_rate': '0.0001991', 'entropy': '1.288', 'num_tokens': '5.835e+06', 'mean_token_accuracy': '0.7155', 'epoch': '0.009558'}
{'loss': '0.8488', 'grad_norm': '0.3192', 'learning_rate': '0.0001991', 'entropy': '0.8276', 'num_tokens': '5.84e+06', 'mean_token_accuracy': '0.7872', 'epoch': '0.009567'}
{'loss': '1.155', 'grad_norm': '0.3049', 'learning_rate': '0.0001991', 'entropy': '0.9967', 'num_tokens': '5.85e+06', 'mean_token_accuracy': '0.7823', 'epoch': '0.009576'}
{'loss': '1.09', 'grad_norm': '0.4372', 'learning_rate': '0.0001991', 'entropy': '0.9445', 'num_tokens': '5.855e+06', 'mean_token_accuracy': '0.7719', 'epoch': '0.009584'}
{'loss': '0.7505', 'grad_norm': '0.2812', 'learning_rate': '0.0001991', 'entropy': '0.7706', 'num_tokens': '5.86e+06', 'mean_token_accuracy': '0.8031', 'epoch': '0.009593'}

  9%|▉         | 1085/12000 [38:51<6:42:49,  2.21s/it]
  9%|▉         | 1086/12000 [38:53<6:46:48,  2.24s/it]
                                                      

  9%|▉         | 1086/12000 [38:53<6:46:48,  2.24s/it]
  9%|▉         | 1087/12000 [38:55<6:08:18,  2.03s/it]
                                                      

  9%|▉         | 1087/12000 [38:55<6:08:18,  2.03s/it]
  9%|▉         | 1088/12000 [38:57<5:42:05,  1.88s/it]
                                                      

  9%|▉         | 1088/12000 [38:57<5:42:05,  1.88s/it]
  9%|▉         | 1089/12000 [38:58<5:32:51,  1.83s/it]
                                                      

  9%|▉         | 1089/12000 [38:58<5:32:51,  1.83s/it]
  9%|▉         | 1090/12000 [39:00<5:39:10,  1.87s/it]
                                                      

  9%|▉         | 1090/12000 [39:00<5:39:10,  1.87s/it]
  9%|▉         | 1091/12000 [39:02<5:24:36,  1.79s/it]
                                                      

  9%|▉         | 1091/12000 [39:02<5:24:36,  1.79s/it]
  9%|▉         | 1092/12000 [39:03<5:12:24,  1.72s/it]
                                                      

  9%|▉         | 1092/12000 [39:03<5:12:24,  1.72s/it]
  9%|▉         | 1093/12000 [39:05<5:16:10,  1.74s/it]
                                                      

  9%|▉         | 1093/12000 [39:05<5:16:10,  1.74s/it]
  9%|▉         | 1094/12000 [39:07<5:20:38,  1.76s/it]
                                                      

  9%|▉         | 1094/12000 [39:07<5:20:38,  1.76s/it]
  9%|▉         | 1095/12000 [39:09<5:17:09,  1.75s/it]
                                                      

  9%|▉         | 1095/12000 [39:09<5:17:09,  1.75s/it]
  9%|▉         | 1096/12000 [39:11<5:46:13,  1.91s/it]
                                                      

  9%|▉         | 1096/12000 [39:11<5:46:13,  1.91s/it]
  9%|▉         | 1097/12000 [39:13<5:49:35,  1.92s/it]
                                                      

  9%|▉         | 1097/12000 [39:13<5:49:35,  1.92s/it]
  9%|▉         | 1098/12000 [39:14<5:25:32,  1.79s/it]
                                                      

  9%|▉         | 1098/12000 [39:14<5:25:32,  1.79s/it]
  9%|▉         | 1099/12000 [39:19<7:35:36,  2.51s/it]
                                                      

  9%|▉         | 1099/12000 [39:19<7:35:36,  2.51s/it]
  9%|▉         | 1100/12000 [39:23<9:35:56,  3.17s/it]
                                                      

  9%|▉         | 1100/12000 [39:23<9:35:56,  3.17s/it]
  9%|▉         | 1101/12000 [39:27<9:45:11,  3.22s/it]
                                                      

  9%|▉         | 1101/12000 [39:27<9:45:11,  3.22s/it]
  9%|▉         | 1102/12000 [39:28<8:22:46,  2.77s/it]
                                                      

  9%|▉         | 1102/12000 [39:28<8:22:46,  2.77s/it]
  9%|▉         | 1103/12000 [39:31<8:28:25,  2.80s/it]
                                                      

  9%|▉         | 1103/12000 [39:31<8:28:25,  2.80s/it]
  9%|▉         | 1104/12000 [39:33<7:12:53,  2.38s/it]
                                                      

  9%|▉         | 1104/12000 [39:33<7:12:53,  2.38s/it]
  9%|▉         | 1105/12000 [39:34<6:29:02,  2.14s/it]
                                                      

  9%|▉         | 1105/12000 [39:34<6:29:02,  2.14s/it]
  9%|▉         | 1106/12000 [39:37<7:26:26,  2.46s/it]
                                                      

  9%|▉         | 1106/12000 [39:37<7:26:26,  2.46s/it]
  9%|▉         | 1107/12000 [39:40<7:10:06,  2.37s/it]
                                                      

  9%|▉         | 1107/12000 [39:40<7:10:06,  2.37s/it]
  9%|▉         | 1108/12000 [39:41<6:20:52,  2.10s/it]
                                                      

  9%|▉         | 1108/12000 [39:41<6:20:52,  2.10s/it]
  9%|▉         | 1109/12000 [39:43<5:56:03,  1.96s/it]
                                                      

  9%|▉         | 1109/12000 [39:43<5:56:03,  1.96s/it]
  9%|▉         | 1110/12000 [39:44<5:41:54,  1.88s/it]
                                                      

  9%|▉         | 1110/12000 [39:44<5:41:54,  1.88s/it]
  9%|▉         | 1111/12000 [39:46<5:39:27,  1.87s/it]
                                                      

  9%|▉         | 1111/12000 [39:46<5:39:27,  1.87s/it]
  9%|▉         | 1112/12000 [39:51<8:21:54,  2.77s/it]
                                                      

  9%|▉         | 1112/12000 [39:51<8:21:54,  2.77s/it]
  9%|▉         | 1113/12000 [39:54<8:38:56,  2.86s/it]
                                                      

  9%|▉         | 1113/12000 [39:54<8:38:56,  2.86s/it]
  9%|▉         | 1114/12000 [39:56<7:22:24,  2.44s/it]
                                                      

  9%|▉         | 1114/12000 [39:56<7:22:24,  2.44s/it]
  9%|▉         | 1115/12000 [39:58<6:55:16,  2.29s/it]
                                                      

  9%|▉         | 1115/12000 [39:58<6:55:16,  2.29s/it]
  9%|▉         | 1116/12000 [40:00<7:26:16,  2.46s/it]
                                                      

  9%|▉         | 1116/12000 [40:00<7:26:16,  2.46s/it]
  9%|▉         | 1117/12000 [40:02<6:49:26,  2.26s/it]
                                                      

  9%|▉         | 1117/12000 [40:02<6:49:26,  2.26s/it]
  9%|▉         | 1118/12000 [40:04<6:41:30,  2.21s/it]
                                                      

  9%|▉         | 1118/12000 [40:04<6:41:30,  2.21s/it]
  9%|▉         | 1119/12000 [40:06<6:01:19,  1.99s/it]
                                                      

  9%|▉         | 1119/12000 [40:06<6:01:19,  1.99s/it]
  9%|▉         | 1120/12000 [40:10<7:37:26,  2.52s/it]
                                                      

  9%|▉         | 1120/12000 [40:10<7:37:26,  2.52s/it]
  9%|▉         | 1121/12000 [40:12<7:33:32,  2.50s/it]
                                                      

  9%|▉         | 1121/12000 [40:12<7:33:32,  2.50s/it]
  9%|▉         | 1122/12000 [40:15<7:47:31,  2.58s/it]
                                                      

  9%|▉         | 1122/12000 [40:15<7:47:31,  2.58s/it]
  9%|▉         | 1123/12000 [40:16<6:56:36,  2.30s/it]
                                                      

  9%|▉         | 1123/12000 [40:16<6:56:36,  2.30s/it]
  9%|▉         | 1124/12000 [40:18<6:15:00,  2.07s/it]
                                                      

  9%|▉         | 1124/12000 [40:18<6:15:00,  2.07s/it]
  9%|▉         | 1125/12000 [40:19<5:43:33,  1.90s/it]
                                                      

  9%|▉         | 1125/12000 [40:19<5:43:33,  1.90s/it]
  9%|▉         | 1126/12000 [40:21<5:26:08,  1.80s/it]
                                                      

  9%|▉         | 1126/12000 [40:21<5:26:08,  1.80s/it]
  9%|▉         | 1127/12000 [40:24<6:15:40,  2.07s/it]
                                                      

  9%|▉         | 1127/12000 [40:24<6:15:40,  2.07s/it]
  9%|▉         | 1128/12000 [40:25<5:42:39,  1.89s/it]
                                                      

  9%|▉         | 1128/12000 [40:25<5:42:39,  1.89s/it]
  9%|▉         | 1129/12000 [40:27<5:48:06,  1.92s/it]
                                                      

  9%|▉         | 1129/12000 [40:27<5:48:06,  1.92s/it]
  9%|▉         | 1130/12000 [40:29<5:22:07,  1.78s/it]
                                                      

  9%|▉         | 1130/12000 [40:29<5:22:07,  1.78s/it]
  9%|▉         | 1131/12000 [40:30<5:02:21,  1.67s/it]
                                                      

  9%|▉         | 1131/12000 [40:30<5:02:21,  1.67s/it]
  9%|▉         | 1132/12000 [40:32<5:28:50,  1.82s/it]
                                                      
{'loss': '1', 'grad_norm': '0.3814', 'learning_rate': '0.0001991', 'entropy': '0.8876', 'num_tokens': '5.865e+06', 'mean_token_accuracy': '0.7772', 'epoch': '0.009602'}
{'loss': '0.7562', 'grad_norm': '0.2921', 'learning_rate': '0.0001991', 'entropy': '0.752', 'num_tokens': '5.871e+06', 'mean_token_accuracy': '0.812', 'epoch': '0.009611'}
{'loss': '0.7269', 'grad_norm': '0.3488', 'learning_rate': '0.0001991', 'entropy': '0.7134', 'num_tokens': '5.876e+06', 'mean_token_accuracy': '0.8219', 'epoch': '0.00962'}
{'loss': '0.7206', 'grad_norm': '0.3266', 'learning_rate': '0.0001991', 'entropy': '0.721', 'num_tokens': '5.88e+06', 'mean_token_accuracy': '0.8063', 'epoch': '0.009629'}
{'loss': '0.7735', 'grad_norm': '0.4168', 'learning_rate': '0.0001991', 'entropy': '0.7787', 'num_tokens': '5.885e+06', 'mean_token_accuracy': '0.787', 'epoch': '0.009637'}
{'loss': '1.032', 'grad_norm': '0.3976', 'learning_rate': '0.0001991', 'entropy': '1.161', 'num_tokens': '5.89e+06', 'mean_token_accuracy': '0.725', 'epoch': '0.009646'}
{'loss': '0.9321', 'grad_norm': '0.3761', 'learning_rate': '0.0001991', 'entropy': '1.016', 'num_tokens': '5.895e+06', 'mean_token_accuracy': '0.7462', 'epoch': '0.009655'}
{'loss': '1.123', 'grad_norm': '0.421', 'learning_rate': '0.0001991', 'entropy': '1.094', 'num_tokens': '5.899e+06', 'mean_token_accuracy': '0.7345', 'epoch': '0.009664'}
{'loss': '0.8881', 'grad_norm': '0.3848', 'learning_rate': '0.0001991', 'entropy': '0.8998', 'num_tokens': '5.905e+06', 'mean_token_accuracy': '0.765', 'epoch': '0.009673'}
{'loss': '0.9107', 'grad_norm': '0.3359', 'learning_rate': '0.0001991', 'entropy': '0.8988', 'num_tokens': '5.91e+06', 'mean_token_accuracy': '0.7748', 'epoch': '0.009682'}
{'loss': '1.251', 'grad_norm': '0.363', 'learning_rate': '0.0001991', 'entropy': '1.226', 'num_tokens': '5.915e+06', 'mean_token_accuracy': '0.7148', 'epoch': '0.009691'}
{'loss': '0.7833', 'grad_norm': '0.2792', 'learning_rate': '0.0001991', 'entropy': '0.7849', 'num_tokens': '5.921e+06', 'mean_token_accuracy': '0.808', 'epoch': '0.009699'}
{'loss': '1.082', 'grad_norm': '0.3802', 'learning_rate': '0.0001991', 'entropy': '1.071', 'num_tokens': '5.928e+06', 'mean_token_accuracy': '0.7414', 'epoch': '0.009708'}
{'loss': '0.9183', 'grad_norm': '0.4018', 'learning_rate': '0.0001991', 'entropy': '1.036', 'num_tokens': '5.932e+06', 'mean_token_accuracy': '0.7529', 'epoch': '0.009717'}
{'loss': '0.9548', 'grad_norm': '0.5144', 'learning_rate': '0.0001991', 'entropy': '1.102', 'num_tokens': '5.937e+06', 'mean_token_accuracy': '0.7318', 'epoch': '0.009726'}
{'loss': '1.101', 'grad_norm': '0.3096', 'learning_rate': '0.0001991', 'entropy': '0.9356', 'num_tokens': '5.946e+06', 'mean_token_accuracy': '0.7708', 'epoch': '0.009735'}
{'loss': '0.8371', 'grad_norm': '0.3664', 'learning_rate': '0.0001991', 'entropy': '0.911', 'num_tokens': '5.952e+06', 'mean_token_accuracy': '0.7776', 'epoch': '0.009744'}
{'loss': '0.8704', 'grad_norm': '0.3328', 'learning_rate': '0.000199', 'entropy': '0.914', 'num_tokens': '5.957e+06', 'mean_token_accuracy': '0.781', 'epoch': '0.009753'}
{'loss': '1.122', 'grad_norm': '0.3121', 'learning_rate': '0.000199', 'entropy': '1.165', 'num_tokens': '5.965e+06', 'mean_token_accuracy': '0.7289', 'epoch': '0.009761'}
{'loss': '0.7404', 'grad_norm': '0.3001', 'learning_rate': '0.000199', 'entropy': '0.7968', 'num_tokens': '5.969e+06', 'mean_token_accuracy': '0.811', 'epoch': '0.00977'}
{'loss': '0.9567', 'grad_norm': '0.3553', 'learning_rate': '0.000199', 'entropy': '0.996', 'num_tokens': '5.974e+06', 'mean_token_accuracy': '0.7603', 'epoch': '0.009779'}
{'loss': '1.187', 'grad_norm': '0.3242', 'learning_rate': '0.000199', 'entropy': '1.029', 'num_tokens': '5.983e+06', 'mean_token_accuracy': '0.7494', 'epoch': '0.009788'}
{'loss': '0.8996', 'grad_norm': '0.3439', 'learning_rate': '0.000199', 'entropy': '0.9432', 'num_tokens': '5.989e+06', 'mean_token_accuracy': '0.7708', 'epoch': '0.009797'}
{'loss': '0.9466', 'grad_norm': '0.5617', 'learning_rate': '0.000199', 'entropy': '1.104', 'num_tokens': '5.992e+06', 'mean_token_accuracy': '0.7355', 'epoch': '0.009806'}
{'loss': '0.8595', 'grad_norm': '0.3253', 'learning_rate': '0.000199', 'entropy': '0.9018', 'num_tokens': '5.998e+06', 'mean_token_accuracy': '0.7855', 'epoch': '0.009814'}
{'loss': '0.9646', 'grad_norm': '0.3334', 'learning_rate': '0.000199', 'entropy': '0.903', 'num_tokens': '6.004e+06', 'mean_token_accuracy': '0.7693', 'epoch': '0.009823'}
{'loss': '1.006', 'grad_norm': '0.4029', 'learning_rate': '0.000199', 'entropy': '0.9466', 'num_tokens': '6.01e+06', 'mean_token_accuracy': '0.7638', 'epoch': '0.009832'}
{'loss': '0.5428', 'grad_norm': '0.2199', 'learning_rate': '0.000199', 'entropy': '0.7142', 'num_tokens': '6.019e+06', 'mean_token_accuracy': '0.8122', 'epoch': '0.009841'}
{'loss': '1.295', 'grad_norm': '0.428', 'learning_rate': '0.000199', 'entropy': '1.192', 'num_tokens': '6.026e+06', 'mean_token_accuracy': '0.7096', 'epoch': '0.00985'}
{'loss': '0.9352', 'grad_norm': '0.4248', 'learning_rate': '0.000199', 'entropy': '0.9334', 'num_tokens': '6.03e+06', 'mean_token_accuracy': '0.7671', 'epoch': '0.009859'}
{'loss': '0.9626', 'grad_norm': '0.3861', 'learning_rate': '0.000199', 'entropy': '0.9419', 'num_tokens': '6.034e+06', 'mean_token_accuracy': '0.7654', 'epoch': '0.009868'}
{'loss': '0.8187', 'grad_norm': '0.3613', 'learning_rate': '0.000199', 'entropy': '0.786', 'num_tokens': '6.041e+06', 'mean_token_accuracy': '0.8022', 'epoch': '0.009876'}
{'loss': '0.9896', 'grad_norm': '0.3491', 'learning_rate': '0.000199', 'entropy': '1.048', 'num_tokens': '6.047e+06', 'mean_token_accuracy': '0.7434', 'epoch': '0.009885'}
{'loss': '1.114', 'grad_norm': '0.4147', 'learning_rate': '0.000199', 'entropy': '1.102', 'num_tokens': '6.054e+06', 'mean_token_accuracy': '0.7351', 'epoch': '0.009894'}
{'loss': '0.8732', 'grad_norm': '0.3312', 'learning_rate': '0.000199', 'entropy': '0.8834', 'num_tokens': '6.059e+06', 'mean_token_accuracy': '0.7855', 'epoch': '0.009903'}
{'loss': '0.8027', 'grad_norm': '0.3817', 'learning_rate': '0.000199', 'entropy': '0.827', 'num_tokens': '6.064e+06', 'mean_token_accuracy': '0.799', 'epoch': '0.009912'}
{'loss': '1.571', 'grad_norm': '0.3731', 'learning_rate': '0.000199', 'entropy': '1.386', 'num_tokens': '6.071e+06', 'mean_token_accuracy': '0.6944', 'epoch': '0.009921'}
{'loss': '0.8604', 'grad_norm': '0.3266', 'learning_rate': '0.000199', 'entropy': '0.8539', 'num_tokens': '6.077e+06', 'mean_token_accuracy': '0.7727', 'epoch': '0.00993'}
{'loss': '1.014', 'grad_norm': '0.4068', 'learning_rate': '0.000199', 'entropy': '1.035', 'num_tokens': '6.081e+06', 'mean_token_accuracy': '0.735', 'epoch': '0.009938'}
{'loss': '0.9954', 'grad_norm': '0.383', 'learning_rate': '0.000199', 'entropy': '0.9888', 'num_tokens': '6.086e+06', 'mean_token_accuracy': '0.762', 'epoch': '0.009947'}
{'loss': '0.8017', 'grad_norm': '0.3403', 'learning_rate': '0.000199', 'entropy': '0.8528', 'num_tokens': '6.089e+06', 'mean_token_accuracy': '0.7879', 'epoch': '0.009956'}
{'loss': '1.606', 'grad_norm': '0.4266', 'learning_rate': '0.000199', 'entropy': '1.492', 'num_tokens': '6.093e+06', 'mean_token_accuracy': '0.6487', 'epoch': '0.009965'}
{'loss': '1.376', 'grad_norm': '0.3728', 'learning_rate': '0.000199', 'entropy': '1.131', 'num_tokens': '6.1e+06', 'mean_token_accuracy': '0.7419', 'epoch': '0.009974'}
{'loss': '0.5746', 'grad_norm': '0.3067', 'learning_rate': '0.0001989', 'entropy': '0.5679', 'num_tokens': '6.104e+06', 'mean_token_accuracy': '0.8522', 'epoch': '0.009983'}
{'loss': '0.759', 'grad_norm': '0.312', 'learning_rate': '0.0001989', 'entropy': '0.6895', 'num_tokens': '6.11e+06', 'mean_token_accuracy': '0.8207', 'epoch': '0.009991'}
{'loss': '0.7912', 'grad_norm': '0.308', 'learning_rate': '0.0001989', 'entropy': '0.8647', 'num_tokens': '6.114e+06', 'mean_token_accuracy': '0.7798', 'epoch': '0.01'}
{'loss': '0.7562', 'grad_norm': '0.345', 'learning_rate': '0.0001989', 'entropy': '0.799', 'num_tokens': '6.118e+06', 'mean_token_accuracy': '0.8113', 'epoch': '0.01001'}

  9%|▉         | 1132/12000 [40:32<5:28:50,  1.82s/it]
  9%|▉         | 1133/12000 [40:34<5:11:13,  1.72s/it]
                                                      

  9%|▉         | 1133/12000 [40:34<5:11:13,  1.72s/it]
  9%|▉         | 1134/12000 [40:35<5:18:00,  1.76s/it]
                                                      

  9%|▉         | 1134/12000 [40:35<5:18:00,  1.76s/it]
  9%|▉         | 1135/12000 [40:38<5:40:08,  1.88s/it]
                                                      

  9%|▉         | 1135/12000 [40:38<5:40:08,  1.88s/it]
  9%|▉         | 1136/12000 [40:39<5:20:21,  1.77s/it]
                                                      

  9%|▉         | 1136/12000 [40:39<5:20:21,  1.77s/it]
  9%|▉         | 1137/12000 [40:41<5:06:07,  1.69s/it]
                                                      

  9%|▉         | 1137/12000 [40:41<5:06:07,  1.69s/it]
  9%|▉         | 1138/12000 [40:43<5:47:24,  1.92s/it]
                                                      

  9%|▉         | 1138/12000 [40:43<5:47:24,  1.92s/it]
  9%|▉         | 1139/12000 [40:45<6:08:39,  2.04s/it]
                                                      

  9%|▉         | 1139/12000 [40:45<6:08:39,  2.04s/it]
 10%|▉         | 1140/12000 [40:49<7:10:55,  2.38s/it]
                                                      

 10%|▉         | 1140/12000 [40:49<7:10:55,  2.38s/it]
 10%|▉         | 1141/12000 [40:50<6:39:49,  2.21s/it]
                                                      

 10%|▉         | 1141/12000 [40:50<6:39:49,  2.21s/it]
 10%|▉         | 1142/12000 [40:55<8:30:25,  2.82s/it]
                                                      

 10%|▉         | 1142/12000 [40:55<8:30:25,  2.82s/it]
 10%|▉         | 1143/12000 [40:56<7:30:38,  2.49s/it]
                                                      

 10%|▉         | 1143/12000 [40:56<7:30:38,  2.49s/it]
 10%|▉         | 1144/12000 [40:58<6:57:58,  2.31s/it]
                                                      

 10%|▉         | 1144/12000 [40:58<6:57:58,  2.31s/it]
 10%|▉         | 1145/12000 [41:00<6:44:02,  2.23s/it]
                                                      

 10%|▉         | 1145/12000 [41:00<6:44:02,  2.23s/it]
 10%|▉         | 1146/12000 [41:02<6:13:00,  2.06s/it]
                                                      

 10%|▉         | 1146/12000 [41:02<6:13:00,  2.06s/it]
 10%|▉         | 1147/12000 [41:04<5:49:51,  1.93s/it]
                                                      

 10%|▉         | 1147/12000 [41:04<5:49:51,  1.93s/it]
 10%|▉         | 1148/12000 [41:05<5:22:27,  1.78s/it]
                                                      

 10%|▉         | 1148/12000 [41:05<5:22:27,  1.78s/it]
 10%|▉         | 1149/12000 [41:07<5:07:29,  1.70s/it]
                                                      

 10%|▉         | 1149/12000 [41:07<5:07:29,  1.70s/it]
 10%|▉         | 1150/12000 [41:08<5:04:02,  1.68s/it]
                                                      

 10%|▉         | 1150/12000 [41:08<5:04:02,  1.68s/it]
 10%|▉         | 1151/12000 [41:11<6:08:48,  2.04s/it]
                                                      

 10%|▉         | 1151/12000 [41:11<6:08:48,  2.04s/it]
 10%|▉         | 1152/12000 [41:13<5:42:38,  1.90s/it]
                                                      

 10%|▉         | 1152/12000 [41:13<5:42:38,  1.90s/it]
 10%|▉         | 1153/12000 [41:14<5:37:26,  1.87s/it]
                                                      

 10%|▉         | 1153/12000 [41:14<5:37:26,  1.87s/it]
 10%|▉         | 1154/12000 [41:16<5:12:40,  1.73s/it]
                                                      

 10%|▉         | 1154/12000 [41:16<5:12:40,  1.73s/it]
 10%|▉         | 1155/12000 [41:17<5:05:31,  1.69s/it]
                                                      

 10%|▉         | 1155/12000 [41:17<5:05:31,  1.69s/it]
 10%|▉         | 1156/12000 [41:20<5:36:34,  1.86s/it]
                                                      

 10%|▉         | 1156/12000 [41:20<5:36:34,  1.86s/it]
 10%|▉         | 1157/12000 [41:23<6:27:15,  2.14s/it]
                                                      

 10%|▉         | 1157/12000 [41:23<6:27:15,  2.14s/it]
 10%|▉         | 1158/12000 [41:25<6:27:16,  2.14s/it]
                                                      

 10%|▉         | 1158/12000 [41:25<6:27:16,  2.14s/it]
 10%|▉         | 1159/12000 [41:26<6:07:56,  2.04s/it]
                                                      

 10%|▉         | 1159/12000 [41:26<6:07:56,  2.04s/it]
 10%|▉         | 1160/12000 [41:29<6:22:47,  2.12s/it]
                                                      

 10%|▉         | 1160/12000 [41:29<6:22:47,  2.12s/it]
 10%|▉         | 1161/12000 [41:31<6:29:08,  2.15s/it]
                                                      

 10%|▉         | 1161/12000 [41:31<6:29:08,  2.15s/it]
 10%|▉         | 1162/12000 [41:33<5:57:02,  1.98s/it]
                                                      

 10%|▉         | 1162/12000 [41:33<5:57:02,  1.98s/it]
 10%|▉         | 1163/12000 [41:34<5:52:17,  1.95s/it]
                                                      

 10%|▉         | 1163/12000 [41:34<5:52:17,  1.95s/it]
 10%|▉         | 1164/12000 [41:38<7:40:35,  2.55s/it]
                                                      

 10%|▉         | 1164/12000 [41:38<7:40:35,  2.55s/it]
 10%|▉         | 1165/12000 [41:40<7:02:51,  2.34s/it]
                                                      

 10%|▉         | 1165/12000 [41:40<7:02:51,  2.34s/it]
 10%|▉         | 1166/12000 [41:43<7:19:06,  2.43s/it]
                                                      

 10%|▉         | 1166/12000 [41:43<7:19:06,  2.43s/it]
 10%|▉         | 1167/12000 [41:45<6:44:36,  2.24s/it]
                                                      

 10%|▉         | 1167/12000 [41:45<6:44:36,  2.24s/it]
 10%|▉         | 1168/12000 [41:46<6:09:18,  2.05s/it]
                                                      

 10%|▉         | 1168/12000 [41:46<6:09:18,  2.05s/it]
 10%|▉         | 1169/12000 [41:48<5:41:16,  1.89s/it]
                                                      

 10%|▉         | 1169/12000 [41:48<5:41:16,  1.89s/it]
 10%|▉         | 1170/12000 [41:49<5:10:57,  1.72s/it]
                                                      

 10%|▉         | 1170/12000 [41:49<5:10:57,  1.72s/it]
 10%|▉         | 1171/12000 [41:52<6:00:58,  2.00s/it]
                                                      

 10%|▉         | 1171/12000 [41:52<6:00:58,  2.00s/it]
 10%|▉         | 1172/12000 [41:54<5:48:04,  1.93s/it]
                                                      

 10%|▉         | 1172/12000 [41:54<5:48:04,  1.93s/it]
 10%|▉         | 1173/12000 [41:55<5:36:20,  1.86s/it]
                                                      

 10%|▉         | 1173/12000 [41:55<5:36:20,  1.86s/it]
 10%|▉         | 1174/12000 [41:58<6:19:43,  2.10s/it]
                                                      

 10%|▉         | 1174/12000 [41:58<6:19:43,  2.10s/it]
 10%|▉         | 1175/12000 [41:59<5:42:55,  1.90s/it]
                                                      

 10%|▉         | 1175/12000 [41:59<5:42:55,  1.90s/it]
 10%|▉         | 1176/12000 [42:01<5:22:39,  1.79s/it]
                                                      

 10%|▉         | 1176/12000 [42:01<5:22:39,  1.79s/it]
 10%|▉         | 1177/12000 [42:02<5:09:45,  1.72s/it]
                                                      

 10%|▉         | 1177/12000 [42:02<5:09:45,  1.72s/it]
 10%|▉         | 1178/12000 [42:04<4:56:22,  1.64s/it]
                                                      

 10%|▉         | 1178/12000 [42:04<4:56:22,  1.64s/it]
 10%|▉         | 1179/12000 [42:06<5:15:14,  1.75s/it]
                                                      
{'loss': '0.8237', 'grad_norm': '0.3747', 'learning_rate': '0.0001989', 'entropy': '0.778', 'num_tokens': '6.125e+06', 'mean_token_accuracy': '0.7976', 'epoch': '0.01002'}
{'loss': '0.8264', 'grad_norm': '0.3593', 'learning_rate': '0.0001989', 'entropy': '0.7914', 'num_tokens': '6.129e+06', 'mean_token_accuracy': '0.8045', 'epoch': '0.01003'}
{'loss': '0.7559', 'grad_norm': '0.3159', 'learning_rate': '0.0001989', 'entropy': '0.7716', 'num_tokens': '6.135e+06', 'mean_token_accuracy': '0.7958', 'epoch': '0.01004'}
{'loss': '0.7978', 'grad_norm': '0.3031', 'learning_rate': '0.0001989', 'entropy': '0.8862', 'num_tokens': '6.141e+06', 'mean_token_accuracy': '0.7828', 'epoch': '0.01004'}
{'loss': '0.8988', 'grad_norm': '0.3914', 'learning_rate': '0.0001989', 'entropy': '0.9379', 'num_tokens': '6.145e+06', 'mean_token_accuracy': '0.7591', 'epoch': '0.01005'}
{'loss': '0.697', 'grad_norm': '0.3052', 'learning_rate': '0.0001989', 'entropy': '0.7511', 'num_tokens': '6.15e+06', 'mean_token_accuracy': '0.8103', 'epoch': '0.01006'}
{'loss': '0.8167', 'grad_norm': '0.3331', 'learning_rate': '0.0001989', 'entropy': '0.9025', 'num_tokens': '6.156e+06', 'mean_token_accuracy': '0.791', 'epoch': '0.01007'}
{'loss': '0.9967', 'grad_norm': '0.3889', 'learning_rate': '0.0001989', 'entropy': '1.137', 'num_tokens': '6.162e+06', 'mean_token_accuracy': '0.7286', 'epoch': '0.01008'}
{'loss': '1.066', 'grad_norm': '0.313', 'learning_rate': '0.0001989', 'entropy': '1.048', 'num_tokens': '6.169e+06', 'mean_token_accuracy': '0.7455', 'epoch': '0.01009'}
{'loss': '0.78', 'grad_norm': '0.3923', 'learning_rate': '0.0001989', 'entropy': '0.8072', 'num_tokens': '6.175e+06', 'mean_token_accuracy': '0.7965', 'epoch': '0.0101'}
{'loss': '1.03', 'grad_norm': '0.3065', 'learning_rate': '0.0001989', 'entropy': '0.9182', 'num_tokens': '6.182e+06', 'mean_token_accuracy': '0.783', 'epoch': '0.01011'}
{'loss': '0.7994', 'grad_norm': '0.2911', 'learning_rate': '0.0001989', 'entropy': '0.8032', 'num_tokens': '6.188e+06', 'mean_token_accuracy': '0.7978', 'epoch': '0.01012'}
{'loss': '0.8368', 'grad_norm': '0.3699', 'learning_rate': '0.0001989', 'entropy': '0.79', 'num_tokens': '6.193e+06', 'mean_token_accuracy': '0.7919', 'epoch': '0.01012'}
{'loss': '1.051', 'grad_norm': '0.3418', 'learning_rate': '0.0001989', 'entropy': '1.034', 'num_tokens': '6.199e+06', 'mean_token_accuracy': '0.759', 'epoch': '0.01013'}
{'loss': '0.8103', 'grad_norm': '0.4653', 'learning_rate': '0.0001989', 'entropy': '0.776', 'num_tokens': '6.203e+06', 'mean_token_accuracy': '0.7822', 'epoch': '0.01014'}
{'loss': '1.125', 'grad_norm': '0.3852', 'learning_rate': '0.0001989', 'entropy': '1.09', 'num_tokens': '6.208e+06', 'mean_token_accuracy': '0.7452', 'epoch': '0.01015'}
{'loss': '1.14', 'grad_norm': '0.3705', 'learning_rate': '0.0001989', 'entropy': '1.26', 'num_tokens': '6.211e+06', 'mean_token_accuracy': '0.7176', 'epoch': '0.01016'}
{'loss': '0.6767', 'grad_norm': '0.3229', 'learning_rate': '0.0001989', 'entropy': '0.6871', 'num_tokens': '6.216e+06', 'mean_token_accuracy': '0.828', 'epoch': '0.01017'}
{'loss': '1.06', 'grad_norm': '0.4334', 'learning_rate': '0.0001989', 'entropy': '1.039', 'num_tokens': '6.22e+06', 'mean_token_accuracy': '0.7416', 'epoch': '0.01018'}
{'loss': '0.913', 'grad_norm': '0.4458', 'learning_rate': '0.0001989', 'entropy': '0.9243', 'num_tokens': '6.223e+06', 'mean_token_accuracy': '0.7687', 'epoch': '0.01019'}
{'loss': '1.177', 'grad_norm': '0.5658', 'learning_rate': '0.0001988', 'entropy': '1.098', 'num_tokens': '6.227e+06', 'mean_token_accuracy': '0.7287', 'epoch': '0.0102'}
{'loss': '1.179', 'grad_norm': '0.3652', 'learning_rate': '0.0001988', 'entropy': '1.229', 'num_tokens': '6.232e+06', 'mean_token_accuracy': '0.7142', 'epoch': '0.0102'}
{'loss': '0.7621', 'grad_norm': '0.3983', 'learning_rate': '0.0001988', 'entropy': '0.7854', 'num_tokens': '6.236e+06', 'mean_token_accuracy': '0.8058', 'epoch': '0.01021'}
{'loss': '0.9038', 'grad_norm': '0.4103', 'learning_rate': '0.0001988', 'entropy': '0.9854', 'num_tokens': '6.241e+06', 'mean_token_accuracy': '0.7569', 'epoch': '0.01022'}
{'loss': '1.032', 'grad_norm': '0.4202', 'learning_rate': '0.0001988', 'entropy': '0.981', 'num_tokens': '6.246e+06', 'mean_token_accuracy': '0.7715', 'epoch': '0.01023'}
{'loss': '0.8051', 'grad_norm': '0.3444', 'learning_rate': '0.0001988', 'entropy': '0.8255', 'num_tokens': '6.253e+06', 'mean_token_accuracy': '0.793', 'epoch': '0.01024'}
{'loss': '0.9261', 'grad_norm': '0.346', 'learning_rate': '0.0001988', 'entropy': '0.9397', 'num_tokens': '6.259e+06', 'mean_token_accuracy': '0.7769', 'epoch': '0.01025'}
{'loss': '1.055', 'grad_norm': '0.4157', 'learning_rate': '0.0001988', 'entropy': '1.095', 'num_tokens': '6.264e+06', 'mean_token_accuracy': '0.7174', 'epoch': '0.01026'}
{'loss': '1.125', 'grad_norm': '0.4545', 'learning_rate': '0.0001988', 'entropy': '1.038', 'num_tokens': '6.27e+06', 'mean_token_accuracy': '0.7484', 'epoch': '0.01027'}
{'loss': '0.7885', 'grad_norm': '0.3354', 'learning_rate': '0.0001988', 'entropy': '0.9835', 'num_tokens': '6.276e+06', 'mean_token_accuracy': '0.7677', 'epoch': '0.01027'}
{'loss': '0.8905', 'grad_norm': '0.3437', 'learning_rate': '0.0001988', 'entropy': '0.8262', 'num_tokens': '6.28e+06', 'mean_token_accuracy': '0.792', 'epoch': '0.01028'}
{'loss': '1.416', 'grad_norm': '0.4185', 'learning_rate': '0.0001988', 'entropy': '1.151', 'num_tokens': '6.285e+06', 'mean_token_accuracy': '0.7251', 'epoch': '0.01029'}
{'loss': '1.024', 'grad_norm': '0.4846', 'learning_rate': '0.0001988', 'entropy': '1.029', 'num_tokens': '6.29e+06', 'mean_token_accuracy': '0.7457', 'epoch': '0.0103'}
{'loss': '0.8525', 'grad_norm': '0.3916', 'learning_rate': '0.0001988', 'entropy': '0.8266', 'num_tokens': '6.294e+06', 'mean_token_accuracy': '0.8046', 'epoch': '0.01031'}
{'loss': '1.086', 'grad_norm': '0.3382', 'learning_rate': '0.0001988', 'entropy': '1.042', 'num_tokens': '6.302e+06', 'mean_token_accuracy': '0.762', 'epoch': '0.01032'}
{'loss': '1.06', 'grad_norm': '0.3974', 'learning_rate': '0.0001988', 'entropy': '1.09', 'num_tokens': '6.307e+06', 'mean_token_accuracy': '0.7437', 'epoch': '0.01033'}
{'loss': '0.9715', 'grad_norm': '0.3341', 'learning_rate': '0.0001988', 'entropy': '1.019', 'num_tokens': '6.311e+06', 'mean_token_accuracy': '0.7578', 'epoch': '0.01034'}
{'loss': '0.9169', 'grad_norm': '0.3605', 'learning_rate': '0.0001988', 'entropy': '0.8562', 'num_tokens': '6.316e+06', 'mean_token_accuracy': '0.7789', 'epoch': '0.01035'}
{'loss': '0.7572', 'grad_norm': '0.3321', 'learning_rate': '0.0001988', 'entropy': '0.7598', 'num_tokens': '6.32e+06', 'mean_token_accuracy': '0.7994', 'epoch': '0.01035'}
{'loss': '0.8964', 'grad_norm': '0.4211', 'learning_rate': '0.0001988', 'entropy': '0.8896', 'num_tokens': '6.325e+06', 'mean_token_accuracy': '0.7836', 'epoch': '0.01036'}
{'loss': '1.047', 'grad_norm': '0.3712', 'learning_rate': '0.0001988', 'entropy': '0.9976', 'num_tokens': '6.33e+06', 'mean_token_accuracy': '0.7588', 'epoch': '0.01037'}
{'loss': '1.041', 'grad_norm': '0.4179', 'learning_rate': '0.0001988', 'entropy': '1.064', 'num_tokens': '6.334e+06', 'mean_token_accuracy': '0.744', 'epoch': '0.01038'}
{'loss': '0.9299', 'grad_norm': '0.3176', 'learning_rate': '0.0001988', 'entropy': '0.9424', 'num_tokens': '6.341e+06', 'mean_token_accuracy': '0.7739', 'epoch': '0.01039'}
{'loss': '0.8632', 'grad_norm': '0.3513', 'learning_rate': '0.0001988', 'entropy': '0.8586', 'num_tokens': '6.346e+06', 'mean_token_accuracy': '0.791', 'epoch': '0.0104'}
{'loss': '0.7768', 'grad_norm': '0.3698', 'learning_rate': '0.0001987', 'entropy': '0.7581', 'num_tokens': '6.35e+06', 'mean_token_accuracy': '0.814', 'epoch': '0.01041'}
{'loss': '0.7419', 'grad_norm': '0.3888', 'learning_rate': '0.0001987', 'entropy': '0.832', 'num_tokens': '6.355e+06', 'mean_token_accuracy': '0.8024', 'epoch': '0.01042'}
{'loss': '0.7695', 'grad_norm': '0.3575', 'learning_rate': '0.0001987', 'entropy': '0.8147', 'num_tokens': '6.359e+06', 'mean_token_accuracy': '0.8045', 'epoch': '0.01043'}

 10%|▉         | 1179/12000 [42:06<5:15:14,  1.75s/it]
 10%|▉         | 1180/12000 [42:08<5:28:42,  1.82s/it]
                                                      

 10%|▉         | 1180/12000 [42:08<5:28:42,  1.82s/it]
 10%|▉         | 1181/12000 [42:11<6:20:19,  2.11s/it]
                                                      

 10%|▉         | 1181/12000 [42:11<6:20:19,  2.11s/it]
 10%|▉         | 1182/12000 [42:13<6:28:23,  2.15s/it]
                                                      

 10%|▉         | 1182/12000 [42:13<6:28:23,  2.15s/it]
 10%|▉         | 1183/12000 [42:15<6:09:56,  2.05s/it]
                                                      

 10%|▉         | 1183/12000 [42:15<6:09:56,  2.05s/it]
 10%|▉         | 1184/12000 [42:16<5:49:33,  1.94s/it]
                                                      

 10%|▉         | 1184/12000 [42:16<5:49:33,  1.94s/it]
 10%|▉         | 1185/12000 [42:18<5:55:52,  1.97s/it]
                                                      

 10%|▉         | 1185/12000 [42:18<5:55:52,  1.97s/it]
 10%|▉         | 1186/12000 [42:22<7:19:55,  2.44s/it]
                                                      

 10%|▉         | 1186/12000 [42:22<7:19:55,  2.44s/it]
 10%|▉         | 1187/12000 [42:24<7:00:47,  2.33s/it]
                                                      

 10%|▉         | 1187/12000 [42:24<7:00:47,  2.33s/it]
 10%|▉         | 1188/12000 [42:30<10:35:23,  3.53s/it]
                                                       

 10%|▉         | 1188/12000 [42:30<10:35:23,  3.53s/it]
 10%|▉         | 1189/12000 [42:34<10:16:52,  3.42s/it]
                                                       

 10%|▉         | 1189/12000 [42:34<10:16:52,  3.42s/it]
 10%|▉         | 1190/12000 [42:35<8:45:57,  2.92s/it] 
                                                      

 10%|▉         | 1190/12000 [42:35<8:45:57,  2.92s/it]
 10%|▉         | 1191/12000 [42:37<7:35:20,  2.53s/it]
                                                      

 10%|▉         | 1191/12000 [42:37<7:35:20,  2.53s/it]
 10%|▉         | 1192/12000 [42:39<6:59:06,  2.33s/it]
                                                      

 10%|▉         | 1192/12000 [42:39<6:59:06,  2.33s/it]
 10%|▉         | 1193/12000 [42:40<6:19:25,  2.11s/it]
                                                      

 10%|▉         | 1193/12000 [42:40<6:19:25,  2.11s/it]
 10%|▉         | 1194/12000 [42:42<5:43:22,  1.91s/it]
                                                      

 10%|▉         | 1194/12000 [42:42<5:43:22,  1.91s/it]
 10%|▉         | 1195/12000 [42:44<6:19:37,  2.11s/it]
                                                      

 10%|▉         | 1195/12000 [42:44<6:19:37,  2.11s/it]
 10%|▉         | 1196/12000 [42:47<6:28:14,  2.16s/it]
                                                      

 10%|▉         | 1196/12000 [42:47<6:28:14,  2.16s/it]
 10%|▉         | 1197/12000 [42:49<6:28:56,  2.16s/it]
                                                      

 10%|▉         | 1197/12000 [42:49<6:28:56,  2.16s/it]
 10%|▉         | 1198/12000 [42:51<6:30:12,  2.17s/it]
                                                      

 10%|▉         | 1198/12000 [42:51<6:30:12,  2.17s/it]
 10%|▉         | 1199/12000 [42:54<6:57:20,  2.32s/it]
                                                      

 10%|▉         | 1199/12000 [42:54<6:57:20,  2.32s/it]
 10%|█         | 1200/12000 [42:55<6:11:06,  2.06s/it]
                                                      

 10%|█         | 1200/12000 [42:55<6:11:06,  2.06s/it]
 10%|█         | 1201/12000 [42:58<7:11:27,  2.40s/it]
                                                      

 10%|█         | 1201/12000 [42:58<7:11:27,  2.40s/it]
 10%|█         | 1202/12000 [43:01<7:28:24,  2.49s/it]
                                                      

 10%|█         | 1202/12000 [43:01<7:28:24,  2.49s/it]
 10%|█         | 1203/12000 [43:03<6:45:50,  2.26s/it]
                                                      

 10%|█         | 1203/12000 [43:03<6:45:50,  2.26s/it]
 10%|█         | 1204/12000 [43:04<6:13:30,  2.08s/it]
                                                      

 10%|█         | 1204/12000 [43:04<6:13:30,  2.08s/it]
 10%|█         | 1205/12000 [43:06<6:10:04,  2.06s/it]
                                                      

 10%|█         | 1205/12000 [43:06<6:10:04,  2.06s/it]
 10%|█         | 1206/12000 [43:08<5:44:57,  1.92s/it]
                                                      

 10%|█         | 1206/12000 [43:08<5:44:57,  1.92s/it]
 10%|█         | 1207/12000 [43:09<5:12:52,  1.74s/it]
                                                      

 10%|█         | 1207/12000 [43:09<5:12:52,  1.74s/it]
 10%|█         | 1208/12000 [43:13<6:45:35,  2.25s/it]
                                                      

 10%|█         | 1208/12000 [43:13<6:45:35,  2.25s/it]
 10%|█         | 1209/12000 [43:15<6:39:00,  2.22s/it]
                                                      

 10%|█         | 1209/12000 [43:15<6:39:00,  2.22s/it]
 10%|█         | 1210/12000 [43:17<6:07:23,  2.04s/it]
                                                      

 10%|█         | 1210/12000 [43:17<6:07:23,  2.04s/it]
 10%|█         | 1211/12000 [43:22<9:09:49,  3.06s/it]
                                                      

 10%|█         | 1211/12000 [43:22<9:09:49,  3.06s/it]
 10%|█         | 1212/12000 [43:26<10:20:00,  3.45s/it]
                                                       

 10%|█         | 1212/12000 [43:26<10:20:00,  3.45s/it]
 10%|█         | 1213/12000 [43:28<8:43:32,  2.91s/it] 
                                                      

 10%|█         | 1213/12000 [43:28<8:43:32,  2.91s/it]
 10%|█         | 1214/12000 [43:30<7:36:24,  2.54s/it]
                                                      

 10%|█         | 1214/12000 [43:30<7:36:24,  2.54s/it]
 10%|█         | 1215/12000 [43:31<6:50:09,  2.28s/it]
                                                      

 10%|█         | 1215/12000 [43:31<6:50:09,  2.28s/it]
 10%|█         | 1216/12000 [43:33<6:10:31,  2.06s/it]
                                                      

 10%|█         | 1216/12000 [43:33<6:10:31,  2.06s/it]
 10%|█         | 1217/12000 [43:35<6:18:57,  2.11s/it]
                                                      

 10%|█         | 1217/12000 [43:35<6:18:57,  2.11s/it]
 10%|█         | 1218/12000 [43:37<5:46:03,  1.93s/it]
                                                      

 10%|█         | 1218/12000 [43:37<5:46:03,  1.93s/it]
 10%|█         | 1219/12000 [43:38<5:19:52,  1.78s/it]
                                                      

 10%|█         | 1219/12000 [43:38<5:19:52,  1.78s/it]
 10%|█         | 1220/12000 [43:40<5:11:09,  1.73s/it]
                                                      

 10%|█         | 1220/12000 [43:40<5:11:09,  1.73s/it]
 10%|█         | 1221/12000 [43:42<5:34:00,  1.86s/it]
                                                      

 10%|█         | 1221/12000 [43:42<5:34:00,  1.86s/it]
 10%|█         | 1222/12000 [43:44<6:02:42,  2.02s/it]
                                                      

 10%|█         | 1222/12000 [43:44<6:02:42,  2.02s/it]
 10%|█         | 1223/12000 [43:46<5:46:30,  1.93s/it]
                                                      

 10%|█         | 1223/12000 [43:46<5:46:30,  1.93s/it]
 10%|█         | 1224/12000 [43:48<5:28:01,  1.83s/it]
                                                      

 10%|█         | 1224/12000 [43:48<5:28:01,  1.83s/it]
 10%|█         | 1225/12000 [43:50<5:40:49,  1.90s/it]
                                                      

 10%|█         | 1225/12000 [43:50<5:40:49,  1.90s/it]
 10%|█         | 1226/12000 [43:51<5:38:55,  1.89s/it]
                                                      
{'loss': '1.072', 'grad_norm': '0.4576', 'learning_rate': '0.0001987', 'entropy': '1.045', 'num_tokens': '6.364e+06', 'mean_token_accuracy': '0.7502', 'epoch': '0.01043'}
{'loss': '1.01', 'grad_norm': '0.3641', 'learning_rate': '0.0001987', 'entropy': '1.05', 'num_tokens': '6.369e+06', 'mean_token_accuracy': '0.7497', 'epoch': '0.01044'}
{'loss': '1', 'grad_norm': '0.3402', 'learning_rate': '0.0001987', 'entropy': '0.8751', 'num_tokens': '6.378e+06', 'mean_token_accuracy': '0.7958', 'epoch': '0.01045'}
{'loss': '0.7646', 'grad_norm': '0.2977', 'learning_rate': '0.0001987', 'entropy': '0.8318', 'num_tokens': '6.384e+06', 'mean_token_accuracy': '0.7888', 'epoch': '0.01046'}
{'loss': '0.9292', 'grad_norm': '0.501', 'learning_rate': '0.0001987', 'entropy': '0.9375', 'num_tokens': '6.39e+06', 'mean_token_accuracy': '0.7857', 'epoch': '0.01047'}
{'loss': '1.078', 'grad_norm': '0.3573', 'learning_rate': '0.0001987', 'entropy': '1.068', 'num_tokens': '6.395e+06', 'mean_token_accuracy': '0.7437', 'epoch': '0.01048'}
{'loss': '1.102', 'grad_norm': '0.3896', 'learning_rate': '0.0001987', 'entropy': '0.9485', 'num_tokens': '6.401e+06', 'mean_token_accuracy': '0.7719', 'epoch': '0.01049'}
{'loss': '1.328', 'grad_norm': '0.4606', 'learning_rate': '0.0001987', 'entropy': '1.285', 'num_tokens': '6.405e+06', 'mean_token_accuracy': '0.7059', 'epoch': '0.0105'}
{'loss': '0.9431', 'grad_norm': '0.3604', 'learning_rate': '0.0001987', 'entropy': '0.8604', 'num_tokens': '6.411e+06', 'mean_token_accuracy': '0.7803', 'epoch': '0.0105'}
{'loss': '1.531', 'grad_norm': '0.3305', 'learning_rate': '0.0001987', 'entropy': '1.362', 'num_tokens': '6.422e+06', 'mean_token_accuracy': '0.6963', 'epoch': '0.01051'}
{'loss': '0.7443', 'grad_norm': '0.37', 'learning_rate': '0.0001987', 'entropy': '0.7121', 'num_tokens': '6.429e+06', 'mean_token_accuracy': '0.815', 'epoch': '0.01052'}
{'loss': '0.9246', 'grad_norm': '0.406', 'learning_rate': '0.0001987', 'entropy': '0.8779', 'num_tokens': '6.435e+06', 'mean_token_accuracy': '0.7841', 'epoch': '0.01053'}
{'loss': '0.7086', 'grad_norm': '0.438', 'learning_rate': '0.0001987', 'entropy': '0.7421', 'num_tokens': '6.438e+06', 'mean_token_accuracy': '0.8119', 'epoch': '0.01054'}
{'loss': '1.356', 'grad_norm': '0.4594', 'learning_rate': '0.0001987', 'entropy': '1.342', 'num_tokens': '6.444e+06', 'mean_token_accuracy': '0.6823', 'epoch': '0.01055'}
{'loss': '0.5387', 'grad_norm': '0.3214', 'learning_rate': '0.0001987', 'entropy': '0.6374', 'num_tokens': '6.449e+06', 'mean_token_accuracy': '0.8389', 'epoch': '0.01056'}
{'loss': '0.8672', 'grad_norm': '0.3478', 'learning_rate': '0.0001987', 'entropy': '0.9052', 'num_tokens': '6.453e+06', 'mean_token_accuracy': '0.7716', 'epoch': '0.01057'}
{'loss': '0.9784', 'grad_norm': '0.3285', 'learning_rate': '0.0001987', 'entropy': '1.019', 'num_tokens': '6.459e+06', 'mean_token_accuracy': '0.7546', 'epoch': '0.01058'}
{'loss': '0.9177', 'grad_norm': '0.3599', 'learning_rate': '0.0001987', 'entropy': '0.9556', 'num_tokens': '6.464e+06', 'mean_token_accuracy': '0.7805', 'epoch': '0.01058'}
{'loss': '0.9678', 'grad_norm': '0.3676', 'learning_rate': '0.0001987', 'entropy': '1.019', 'num_tokens': '6.47e+06', 'mean_token_accuracy': '0.7613', 'epoch': '0.01059'}
{'loss': '1.043', 'grad_norm': '0.3346', 'learning_rate': '0.0001986', 'entropy': '1.049', 'num_tokens': '6.476e+06', 'mean_token_accuracy': '0.7406', 'epoch': '0.0106'}
{'loss': '0.7382', 'grad_norm': '0.3037', 'learning_rate': '0.0001986', 'entropy': '0.7842', 'num_tokens': '6.483e+06', 'mean_token_accuracy': '0.8102', 'epoch': '0.01061'}
{'loss': '0.8871', 'grad_norm': '0.387', 'learning_rate': '0.0001986', 'entropy': '0.9389', 'num_tokens': '6.487e+06', 'mean_token_accuracy': '0.7744', 'epoch': '0.01062'}
{'loss': '0.9778', 'grad_norm': '0.332', 'learning_rate': '0.0001986', 'entropy': '1.04', 'num_tokens': '6.492e+06', 'mean_token_accuracy': '0.7444', 'epoch': '0.01063'}
{'loss': '0.9301', 'grad_norm': '0.3741', 'learning_rate': '0.0001986', 'entropy': '0.8928', 'num_tokens': '6.499e+06', 'mean_token_accuracy': '0.778', 'epoch': '0.01064'}
{'loss': '0.9269', 'grad_norm': '0.5116', 'learning_rate': '0.0001986', 'entropy': '0.9359', 'num_tokens': '6.503e+06', 'mean_token_accuracy': '0.7637', 'epoch': '0.01065'}
{'loss': '0.7983', 'grad_norm': '0.3728', 'learning_rate': '0.0001986', 'entropy': '0.9104', 'num_tokens': '6.508e+06', 'mean_token_accuracy': '0.779', 'epoch': '0.01066'}
{'loss': '1.544', 'grad_norm': '0.463', 'learning_rate': '0.0001986', 'entropy': '1.362', 'num_tokens': '6.513e+06', 'mean_token_accuracy': '0.6963', 'epoch': '0.01066'}
{'loss': '0.8532', 'grad_norm': '0.4356', 'learning_rate': '0.0001986', 'entropy': '0.8933', 'num_tokens': '6.517e+06', 'mean_token_accuracy': '0.7775', 'epoch': '0.01067'}
{'loss': '1.003', 'grad_norm': '0.4131', 'learning_rate': '0.0001986', 'entropy': '1.037', 'num_tokens': '6.52e+06', 'mean_token_accuracy': '0.7501', 'epoch': '0.01068'}
{'loss': '0.8779', 'grad_norm': '0.3294', 'learning_rate': '0.0001986', 'entropy': '0.8948', 'num_tokens': '6.524e+06', 'mean_token_accuracy': '0.7855', 'epoch': '0.01069'}
{'loss': '0.9052', 'grad_norm': '0.3908', 'learning_rate': '0.0001986', 'entropy': '0.7605', 'num_tokens': '6.531e+06', 'mean_token_accuracy': '0.8052', 'epoch': '0.0107'}
{'loss': '0.6508', 'grad_norm': '0.2909', 'learning_rate': '0.0001986', 'entropy': '0.6534', 'num_tokens': '6.536e+06', 'mean_token_accuracy': '0.8262', 'epoch': '0.01071'}
{'loss': '0.79', 'grad_norm': '0.2481', 'learning_rate': '0.0001986', 'entropy': '0.8678', 'num_tokens': '6.546e+06', 'mean_token_accuracy': '0.7828', 'epoch': '0.01072'}
{'loss': '0.9323', 'grad_norm': '0.4378', 'learning_rate': '0.0001986', 'entropy': '0.8416', 'num_tokens': '6.556e+06', 'mean_token_accuracy': '0.774', 'epoch': '0.01073'}
{'loss': '0.8772', 'grad_norm': '0.3543', 'learning_rate': '0.0001986', 'entropy': '0.9494', 'num_tokens': '6.561e+06', 'mean_token_accuracy': '0.755', 'epoch': '0.01073'}
{'loss': '0.9034', 'grad_norm': '0.3812', 'learning_rate': '0.0001986', 'entropy': '0.8615', 'num_tokens': '6.565e+06', 'mean_token_accuracy': '0.7771', 'epoch': '0.01074'}
{'loss': '1.122', 'grad_norm': '0.5779', 'learning_rate': '0.0001986', 'entropy': '1.153', 'num_tokens': '6.568e+06', 'mean_token_accuracy': '0.7261', 'epoch': '0.01075'}
{'loss': '0.8381', 'grad_norm': '0.3721', 'learning_rate': '0.0001986', 'entropy': '0.8627', 'num_tokens': '6.573e+06', 'mean_token_accuracy': '0.79', 'epoch': '0.01076'}
{'loss': '0.833', 'grad_norm': '0.2926', 'learning_rate': '0.0001986', 'entropy': '0.807', 'num_tokens': '6.58e+06', 'mean_token_accuracy': '0.7946', 'epoch': '0.01077'}
{'loss': '0.891', 'grad_norm': '0.3356', 'learning_rate': '0.0001986', 'entropy': '0.9537', 'num_tokens': '6.584e+06', 'mean_token_accuracy': '0.758', 'epoch': '0.01078'}
{'loss': '0.8277', 'grad_norm': '0.3639', 'learning_rate': '0.0001986', 'entropy': '0.8748', 'num_tokens': '6.588e+06', 'mean_token_accuracy': '0.7862', 'epoch': '0.01079'}
{'loss': '0.7116', 'grad_norm': '0.3431', 'learning_rate': '0.0001985', 'entropy': '0.749', 'num_tokens': '6.593e+06', 'mean_token_accuracy': '0.8077', 'epoch': '0.0108'}
{'loss': '0.9533', 'grad_norm': '0.3796', 'learning_rate': '0.0001985', 'entropy': '1.026', 'num_tokens': '6.598e+06', 'mean_token_accuracy': '0.7626', 'epoch': '0.01081'}
{'loss': '0.6227', 'grad_norm': '0.2732', 'learning_rate': '0.0001985', 'entropy': '0.7284', 'num_tokens': '6.603e+06', 'mean_token_accuracy': '0.809', 'epoch': '0.01081'}
{'loss': '0.9303', 'grad_norm': '0.3763', 'learning_rate': '0.0001985', 'entropy': '0.9085', 'num_tokens': '6.608e+06', 'mean_token_accuracy': '0.7664', 'epoch': '0.01082'}
{'loss': '1.099', 'grad_norm': '0.5383', 'learning_rate': '0.0001985', 'entropy': '1.055', 'num_tokens': '6.613e+06', 'mean_token_accuracy': '0.7294', 'epoch': '0.01083'}
{'loss': '1.079', 'grad_norm': '0.3842', 'learning_rate': '0.0001985', 'entropy': '1.014', 'num_tokens': '6.619e+06', 'mean_token_accuracy': '0.7387', 'epoch': '0.01084'}

 10%|█         | 1226/12000 [43:51<5:38:55,  1.89s/it]
 10%|█         | 1227/12000 [43:53<5:42:25,  1.91s/it]
                                                      

 10%|█         | 1227/12000 [43:53<5:42:25,  1.91s/it]
 10%|█         | 1228/12000 [43:55<5:16:53,  1.77s/it]
                                                      

 10%|█         | 1228/12000 [43:55<5:16:53,  1.77s/it]
 10%|█         | 1229/12000 [43:57<5:31:50,  1.85s/it]
                                                      

 10%|█         | 1229/12000 [43:57<5:31:50,  1.85s/it]
 10%|█         | 1230/12000 [44:01<7:12:47,  2.41s/it]
                                                      

 10%|█         | 1230/12000 [44:01<7:12:47,  2.41s/it]
 10%|█         | 1231/12000 [44:02<6:22:12,  2.13s/it]
                                                      

 10%|█         | 1231/12000 [44:02<6:22:12,  2.13s/it]
 10%|█         | 1232/12000 [44:05<7:26:47,  2.49s/it]
                                                      

 10%|█         | 1232/12000 [44:05<7:26:47,  2.49s/it]
 10%|█         | 1233/12000 [44:07<6:44:24,  2.25s/it]
                                                      

 10%|█         | 1233/12000 [44:07<6:44:24,  2.25s/it]
 10%|█         | 1234/12000 [44:11<7:59:58,  2.67s/it]
                                                      

 10%|█         | 1234/12000 [44:11<7:59:58,  2.67s/it]
 10%|█         | 1235/12000 [44:13<7:32:25,  2.52s/it]
                                                      

 10%|█         | 1235/12000 [44:13<7:32:25,  2.52s/it]
 10%|█         | 1236/12000 [44:15<7:05:53,  2.37s/it]
                                                      

 10%|█         | 1236/12000 [44:15<7:05:53,  2.37s/it]
 10%|█         | 1237/12000 [44:17<6:32:42,  2.19s/it]
                                                      

 10%|█         | 1237/12000 [44:17<6:32:42,  2.19s/it]
 10%|█         | 1238/12000 [44:18<6:06:11,  2.04s/it]
                                                      

 10%|█         | 1238/12000 [44:18<6:06:11,  2.04s/it]
 10%|█         | 1239/12000 [44:22<7:20:20,  2.46s/it]
                                                      

 10%|█         | 1239/12000 [44:22<7:20:20,  2.46s/it]
 10%|█         | 1240/12000 [44:25<7:51:53,  2.63s/it]
                                                      

 10%|█         | 1240/12000 [44:25<7:51:53,  2.63s/it]
 10%|█         | 1241/12000 [44:27<6:59:42,  2.34s/it]
                                                      

 10%|█         | 1241/12000 [44:27<6:59:42,  2.34s/it]
 10%|█         | 1242/12000 [44:28<6:18:41,  2.11s/it]
                                                      

 10%|█         | 1242/12000 [44:28<6:18:41,  2.11s/it]
 10%|█         | 1243/12000 [44:30<5:42:47,  1.91s/it]
                                                      

 10%|█         | 1243/12000 [44:30<5:42:47,  1.91s/it]
 10%|█         | 1244/12000 [44:32<5:44:45,  1.92s/it]
                                                      

 10%|█         | 1244/12000 [44:32<5:44:45,  1.92s/it]
 10%|█         | 1245/12000 [44:34<6:32:58,  2.19s/it]
                                                      

 10%|█         | 1245/12000 [44:34<6:32:58,  2.19s/it]
 10%|█         | 1246/12000 [44:36<5:47:51,  1.94s/it]
                                                      

 10%|█         | 1246/12000 [44:36<5:47:51,  1.94s/it]
 10%|█         | 1247/12000 [44:41<8:45:17,  2.93s/it]
                                                      

 10%|█         | 1247/12000 [44:41<8:45:17,  2.93s/it]
 10%|█         | 1248/12000 [44:42<7:20:41,  2.46s/it]
                                                      

 10%|█         | 1248/12000 [44:42<7:20:41,  2.46s/it]
 10%|█         | 1249/12000 [44:44<6:46:55,  2.27s/it]
                                                      

 10%|█         | 1249/12000 [44:44<6:46:55,  2.27s/it]
 10%|█         | 1250/12000 [44:46<6:45:15,  2.26s/it]
                                                      

 10%|█         | 1250/12000 [44:46<6:45:15,  2.26s/it]
 10%|█         | 1251/12000 [44:51<9:04:05,  3.04s/it]
                                                      

 10%|█         | 1251/12000 [44:51<9:04:05,  3.04s/it]
 10%|█         | 1252/12000 [44:56<10:20:59,  3.47s/it]
                                                       

 10%|█         | 1252/12000 [44:56<10:20:59,  3.47s/it]
 10%|█         | 1253/12000 [44:59<10:24:20,  3.49s/it]
                                                       

 10%|█         | 1253/12000 [44:59<10:24:20,  3.49s/it]
 10%|█         | 1254/12000 [45:02<9:55:06,  3.32s/it] 
                                                      

 10%|█         | 1254/12000 [45:02<9:55:06,  3.32s/it]
 10%|█         | 1255/12000 [45:04<8:42:12,  2.92s/it]
                                                      

 10%|█         | 1255/12000 [45:04<8:42:12,  2.92s/it]
 10%|█         | 1256/12000 [45:07<8:21:06,  2.80s/it]
                                                      

 10%|█         | 1256/12000 [45:07<8:21:06,  2.80s/it]
 10%|█         | 1257/12000 [45:08<7:17:27,  2.44s/it]
                                                      

 10%|█         | 1257/12000 [45:08<7:17:27,  2.44s/it]
 10%|█         | 1258/12000 [45:10<7:02:25,  2.36s/it]
                                                      

 10%|█         | 1258/12000 [45:10<7:02:25,  2.36s/it]
 10%|█         | 1259/12000 [45:13<6:53:23,  2.31s/it]
                                                      

 10%|█         | 1259/12000 [45:13<6:53:23,  2.31s/it]
 10%|█         | 1260/12000 [45:15<6:41:16,  2.24s/it]
                                                      

 10%|█         | 1260/12000 [45:15<6:41:16,  2.24s/it]
 11%|█         | 1261/12000 [45:16<6:03:20,  2.03s/it]
                                                      

 11%|█         | 1261/12000 [45:16<6:03:20,  2.03s/it]
 11%|█         | 1262/12000 [45:18<5:40:01,  1.90s/it]
                                                      

 11%|█         | 1262/12000 [45:18<5:40:01,  1.90s/it]
 11%|█         | 1263/12000 [45:19<5:20:50,  1.79s/it]
                                                      

 11%|█         | 1263/12000 [45:19<5:20:50,  1.79s/it]
 11%|█         | 1264/12000 [45:21<5:18:03,  1.78s/it]
                                                      

 11%|█         | 1264/12000 [45:21<5:18:03,  1.78s/it]
 11%|█         | 1265/12000 [45:23<5:03:23,  1.70s/it]
                                                      

 11%|█         | 1265/12000 [45:23<5:03:23,  1.70s/it]
 11%|█         | 1266/12000 [45:27<7:40:35,  2.57s/it]
                                                      

 11%|█         | 1266/12000 [45:27<7:40:35,  2.57s/it]
 11%|█         | 1267/12000 [45:29<7:00:42,  2.35s/it]
                                                      

 11%|█         | 1267/12000 [45:29<7:00:42,  2.35s/it]
 11%|█         | 1268/12000 [45:31<6:25:49,  2.16s/it]
                                                      

 11%|█         | 1268/12000 [45:31<6:25:49,  2.16s/it]
 11%|█         | 1269/12000 [45:32<5:52:56,  1.97s/it]
                                                      

 11%|█         | 1269/12000 [45:32<5:52:56,  1.97s/it]
 11%|█         | 1270/12000 [45:34<5:31:01,  1.85s/it]
                                                      

 11%|█         | 1270/12000 [45:34<5:31:01,  1.85s/it]
 11%|█         | 1271/12000 [45:36<5:54:16,  1.98s/it]
                                                      

 11%|█         | 1271/12000 [45:36<5:54:16,  1.98s/it]
 11%|█         | 1272/12000 [45:38<5:53:43,  1.98s/it]
                                                      

 11%|█         | 1272/12000 [45:38<5:53:43,  1.98s/it]
 11%|█         | 1273/12000 [45:40<5:50:07,  1.96s/it]
                                                      
{'loss': '0.7783', 'grad_norm': '0.3408', 'learning_rate': '0.0001985', 'entropy': '0.7577', 'num_tokens': '6.625e+06', 'mean_token_accuracy': '0.8021', 'epoch': '0.01085'}
{'loss': '0.879', 'grad_norm': '0.3656', 'learning_rate': '0.0001985', 'entropy': '0.8765', 'num_tokens': '6.63e+06', 'mean_token_accuracy': '0.786', 'epoch': '0.01086'}
{'loss': '1.01', 'grad_norm': '0.386', 'learning_rate': '0.0001985', 'entropy': '1.048', 'num_tokens': '6.634e+06', 'mean_token_accuracy': '0.7553', 'epoch': '0.01087'}
{'loss': '0.9498', 'grad_norm': '0.3625', 'learning_rate': '0.0001985', 'entropy': '0.9378', 'num_tokens': '6.641e+06', 'mean_token_accuracy': '0.7762', 'epoch': '0.01088'}
{'loss': '1.002', 'grad_norm': '0.3443', 'learning_rate': '0.0001985', 'entropy': '0.9013', 'num_tokens': '6.646e+06', 'mean_token_accuracy': '0.7873', 'epoch': '0.01089'}
{'loss': '0.8457', 'grad_norm': '0.3035', 'learning_rate': '0.0001985', 'entropy': '0.8316', 'num_tokens': '6.651e+06', 'mean_token_accuracy': '0.796', 'epoch': '0.01089'}
{'loss': '0.7878', 'grad_norm': '0.3514', 'learning_rate': '0.0001985', 'entropy': '0.8639', 'num_tokens': '6.658e+06', 'mean_token_accuracy': '0.7926', 'epoch': '0.0109'}
{'loss': '0.8988', 'grad_norm': '0.3693', 'learning_rate': '0.0001985', 'entropy': '1.047', 'num_tokens': '6.663e+06', 'mean_token_accuracy': '0.7415', 'epoch': '0.01091'}
{'loss': '1.012', 'grad_norm': '0.3404', 'learning_rate': '0.0001985', 'entropy': '0.9579', 'num_tokens': '6.67e+06', 'mean_token_accuracy': '0.7671', 'epoch': '0.01092'}
{'loss': '1.46', 'grad_norm': '0.3471', 'learning_rate': '0.0001985', 'entropy': '1.39', 'num_tokens': '6.677e+06', 'mean_token_accuracy': '0.6834', 'epoch': '0.01093'}
{'loss': '1.102', 'grad_norm': '0.5276', 'learning_rate': '0.0001985', 'entropy': '1.097', 'num_tokens': '6.682e+06', 'mean_token_accuracy': '0.7271', 'epoch': '0.01094'}
{'loss': '1.064', 'grad_norm': '0.3672', 'learning_rate': '0.0001985', 'entropy': '1.039', 'num_tokens': '6.688e+06', 'mean_token_accuracy': '0.755', 'epoch': '0.01095'}
{'loss': '0.8488', 'grad_norm': '0.3756', 'learning_rate': '0.0001985', 'entropy': '0.9555', 'num_tokens': '6.693e+06', 'mean_token_accuracy': '0.7731', 'epoch': '0.01096'}
{'loss': '1.13', 'grad_norm': '0.3082', 'learning_rate': '0.0001985', 'entropy': '0.9099', 'num_tokens': '6.7e+06', 'mean_token_accuracy': '0.7753', 'epoch': '0.01096'}
{'loss': '0.753', 'grad_norm': '0.338', 'learning_rate': '0.0001985', 'entropy': '0.72', 'num_tokens': '6.707e+06', 'mean_token_accuracy': '0.8105', 'epoch': '0.01097'}
{'loss': '1.184', 'grad_norm': '0.3627', 'learning_rate': '0.0001984', 'entropy': '1.142', 'num_tokens': '6.712e+06', 'mean_token_accuracy': '0.7195', 'epoch': '0.01098'}
{'loss': '1.05', 'grad_norm': '0.4017', 'learning_rate': '0.0001984', 'entropy': '1.196', 'num_tokens': '6.717e+06', 'mean_token_accuracy': '0.7249', 'epoch': '0.01099'}
{'loss': '1.069', 'grad_norm': '0.4337', 'learning_rate': '0.0001984', 'entropy': '1.134', 'num_tokens': '6.72e+06', 'mean_token_accuracy': '0.7227', 'epoch': '0.011'}
{'loss': '1.213', 'grad_norm': '0.3933', 'learning_rate': '0.0001984', 'entropy': '1.197', 'num_tokens': '6.726e+06', 'mean_token_accuracy': '0.6999', 'epoch': '0.01101'}
{'loss': '0.9527', 'grad_norm': '0.3593', 'learning_rate': '0.0001984', 'entropy': '0.94', 'num_tokens': '6.733e+06', 'mean_token_accuracy': '0.7671', 'epoch': '0.01102'}
{'loss': '1.054', 'grad_norm': '0.4278', 'learning_rate': '0.0001984', 'entropy': '1.097', 'num_tokens': '6.736e+06', 'mean_token_accuracy': '0.7468', 'epoch': '0.01103'}
{'loss': '0.7984', 'grad_norm': '0.2585', 'learning_rate': '0.0001984', 'entropy': '0.7805', 'num_tokens': '6.745e+06', 'mean_token_accuracy': '0.813', 'epoch': '0.01104'}
{'loss': '1.09', 'grad_norm': '0.4927', 'learning_rate': '0.0001984', 'entropy': '1.077', 'num_tokens': '6.749e+06', 'mean_token_accuracy': '0.7347', 'epoch': '0.01104'}
{'loss': '0.7939', 'grad_norm': '0.3624', 'learning_rate': '0.0001984', 'entropy': '0.775', 'num_tokens': '6.754e+06', 'mean_token_accuracy': '0.8032', 'epoch': '0.01105'}
{'loss': '0.9375', 'grad_norm': '0.3859', 'learning_rate': '0.0001984', 'entropy': '0.9044', 'num_tokens': '6.761e+06', 'mean_token_accuracy': '0.7755', 'epoch': '0.01106'}
{'loss': '0.882', 'grad_norm': '0.3267', 'learning_rate': '0.0001984', 'entropy': '0.9109', 'num_tokens': '6.768e+06', 'mean_token_accuracy': '0.77', 'epoch': '0.01107'}
{'loss': '0.8758', 'grad_norm': '0.5876', 'learning_rate': '0.0001984', 'entropy': '0.9302', 'num_tokens': '6.773e+06', 'mean_token_accuracy': '0.7733', 'epoch': '0.01108'}
{'loss': '0.8743', 'grad_norm': '0.316', 'learning_rate': '0.0001984', 'entropy': '1.002', 'num_tokens': '6.781e+06', 'mean_token_accuracy': '0.7683', 'epoch': '0.01109'}
{'loss': '0.9642', 'grad_norm': '0.3226', 'learning_rate': '0.0001984', 'entropy': '1.031', 'num_tokens': '6.788e+06', 'mean_token_accuracy': '0.7489', 'epoch': '0.0111'}
{'loss': '1.268', 'grad_norm': '0.4596', 'learning_rate': '0.0001984', 'entropy': '1.192', 'num_tokens': '6.792e+06', 'mean_token_accuracy': '0.709', 'epoch': '0.01111'}
{'loss': '1.077', 'grad_norm': '0.404', 'learning_rate': '0.0001984', 'entropy': '1.027', 'num_tokens': '6.799e+06', 'mean_token_accuracy': '0.7602', 'epoch': '0.01112'}
{'loss': '1.106', 'grad_norm': '0.3349', 'learning_rate': '0.0001984', 'entropy': '1.056', 'num_tokens': '6.804e+06', 'mean_token_accuracy': '0.7454', 'epoch': '0.01112'}
{'loss': '1.216', 'grad_norm': '0.412', 'learning_rate': '0.0001984', 'entropy': '1.12', 'num_tokens': '6.81e+06', 'mean_token_accuracy': '0.7298', 'epoch': '0.01113'}
{'loss': '1.247', 'grad_norm': '0.4842', 'learning_rate': '0.0001984', 'entropy': '1.111', 'num_tokens': '6.815e+06', 'mean_token_accuracy': '0.7404', 'epoch': '0.01114'}
{'loss': '1.499', 'grad_norm': '0.4265', 'learning_rate': '0.0001984', 'entropy': '1.431', 'num_tokens': '6.82e+06', 'mean_token_accuracy': '0.6641', 'epoch': '0.01115'}
{'loss': '1.022', 'grad_norm': '0.3374', 'learning_rate': '0.0001984', 'entropy': '0.9873', 'num_tokens': '6.824e+06', 'mean_token_accuracy': '0.76', 'epoch': '0.01116'}
{'loss': '0.8047', 'grad_norm': '0.34', 'learning_rate': '0.0001983', 'entropy': '0.8612', 'num_tokens': '6.829e+06', 'mean_token_accuracy': '0.786', 'epoch': '0.01117'}
{'loss': '1.027', 'grad_norm': '0.3775', 'learning_rate': '0.0001983', 'entropy': '1.073', 'num_tokens': '6.833e+06', 'mean_token_accuracy': '0.7516', 'epoch': '0.01118'}
{'loss': '0.8707', 'grad_norm': '0.4152', 'learning_rate': '0.0001983', 'entropy': '0.936', 'num_tokens': '6.838e+06', 'mean_token_accuracy': '0.7728', 'epoch': '0.01119'}
{'loss': '0.7509', 'grad_norm': '0.3983', 'learning_rate': '0.0001983', 'entropy': '0.8117', 'num_tokens': '6.842e+06', 'mean_token_accuracy': '0.8008', 'epoch': '0.0112'}
{'loss': '1.257', 'grad_norm': '0.3626', 'learning_rate': '0.0001983', 'entropy': '0.9954', 'num_tokens': '6.85e+06', 'mean_token_accuracy': '0.7716', 'epoch': '0.0112'}
{'loss': '1.491', 'grad_norm': '0.4769', 'learning_rate': '0.0001983', 'entropy': '1.258', 'num_tokens': '6.855e+06', 'mean_token_accuracy': '0.7108', 'epoch': '0.01121'}
{'loss': '0.9826', 'grad_norm': '0.4335', 'learning_rate': '0.0001983', 'entropy': '1.075', 'num_tokens': '6.859e+06', 'mean_token_accuracy': '0.7275', 'epoch': '0.01122'}
{'loss': '0.9167', 'grad_norm': '0.3625', 'learning_rate': '0.0001983', 'entropy': '0.992', 'num_tokens': '6.863e+06', 'mean_token_accuracy': '0.7586', 'epoch': '0.01123'}
{'loss': '0.8435', 'grad_norm': '0.4645', 'learning_rate': '0.0001983', 'entropy': '0.7634', 'num_tokens': '6.868e+06', 'mean_token_accuracy': '0.8016', 'epoch': '0.01124'}
{'loss': '1.192', 'grad_norm': '0.382', 'learning_rate': '0.0001983', 'entropy': '1.076', 'num_tokens': '6.873e+06', 'mean_token_accuracy': '0.7229', 'epoch': '0.01125'}
{'loss': '0.8016', 'grad_norm': '0.4447', 'learning_rate': '0.0001983', 'entropy': '0.8004', 'num_tokens': '6.879e+06', 'mean_token_accuracy': '0.7836', 'epoch': '0.01126'}

 11%|█         | 1273/12000 [45:40<5:50:07,  1.96s/it]
 11%|█         | 1274/12000 [45:45<8:09:42,  2.74s/it]
                                                      

 11%|█         | 1274/12000 [45:45<8:09:42,  2.74s/it]
 11%|█         | 1275/12000 [45:46<7:04:57,  2.38s/it]
                                                      

 11%|█         | 1275/12000 [45:46<7:04:57,  2.38s/it]
 11%|█         | 1276/12000 [45:49<7:20:36,  2.47s/it]
                                                      

 11%|█         | 1276/12000 [45:49<7:20:36,  2.47s/it]
 11%|█         | 1277/12000 [45:50<6:33:32,  2.20s/it]
                                                      

 11%|█         | 1277/12000 [45:50<6:33:32,  2.20s/it]
 11%|█         | 1278/12000 [45:56<9:30:06,  3.19s/it]
                                                      

 11%|█         | 1278/12000 [45:56<9:30:06,  3.19s/it]
 11%|█         | 1279/12000 [45:59<9:18:10,  3.12s/it]
                                                      

 11%|█         | 1279/12000 [45:59<9:18:10,  3.12s/it]
 11%|█         | 1280/12000 [46:00<7:55:23,  2.66s/it]
                                                      

 11%|█         | 1280/12000 [46:00<7:55:23,  2.66s/it]
 11%|█         | 1281/12000 [46:02<6:59:00,  2.35s/it]
                                                      

 11%|█         | 1281/12000 [46:02<6:59:00,  2.35s/it]
 11%|█         | 1282/12000 [46:04<6:37:07,  2.22s/it]
                                                      

 11%|█         | 1282/12000 [46:04<6:37:07,  2.22s/it]
 11%|█         | 1283/12000 [46:06<6:08:08,  2.06s/it]
                                                      

 11%|█         | 1283/12000 [46:06<6:08:08,  2.06s/it]
 11%|█         | 1284/12000 [46:08<6:23:24,  2.15s/it]
                                                      

 11%|█         | 1284/12000 [46:08<6:23:24,  2.15s/it]
 11%|█         | 1285/12000 [46:11<6:47:06,  2.28s/it]
                                                      

 11%|█         | 1285/12000 [46:11<6:47:06,  2.28s/it]
 11%|█         | 1286/12000 [46:13<6:28:38,  2.18s/it]
                                                      

 11%|█         | 1286/12000 [46:13<6:28:38,  2.18s/it]
 11%|█         | 1287/12000 [46:15<6:30:06,  2.18s/it]
                                                      

 11%|█         | 1287/12000 [46:15<6:30:06,  2.18s/it]
 11%|█         | 1288/12000 [46:17<6:07:19,  2.06s/it]
                                                      

 11%|█         | 1288/12000 [46:17<6:07:19,  2.06s/it]
 11%|█         | 1289/12000 [46:18<5:58:44,  2.01s/it]
                                                      

 11%|█         | 1289/12000 [46:18<5:58:44,  2.01s/it]
 11%|█         | 1290/12000 [46:20<5:38:12,  1.89s/it]
                                                      

 11%|█         | 1290/12000 [46:20<5:38:12,  1.89s/it]
 11%|█         | 1291/12000 [46:26<9:27:07,  3.18s/it]
                                                      

 11%|█         | 1291/12000 [46:26<9:27:07,  3.18s/it]
 11%|█         | 1292/12000 [46:28<8:12:37,  2.76s/it]
                                                      

 11%|█         | 1292/12000 [46:28<8:12:37,  2.76s/it]
 11%|█         | 1293/12000 [46:30<7:13:43,  2.43s/it]
                                                      

 11%|█         | 1293/12000 [46:30<7:13:43,  2.43s/it]
 11%|█         | 1294/12000 [46:32<7:22:12,  2.48s/it]
                                                      

 11%|█         | 1294/12000 [46:32<7:22:12,  2.48s/it]
 11%|█         | 1295/12000 [46:34<6:56:28,  2.33s/it]
                                                      

 11%|█         | 1295/12000 [46:34<6:56:28,  2.33s/it]
 11%|█         | 1296/12000 [46:41<10:51:51,  3.65s/it]
                                                       

 11%|█         | 1296/12000 [46:41<10:51:51,  3.65s/it]
 11%|█         | 1297/12000 [46:44<10:00:43,  3.37s/it]
                                                       

 11%|█         | 1297/12000 [46:44<10:00:43,  3.37s/it]
 11%|█         | 1298/12000 [46:45<8:22:12,  2.82s/it] 
                                                      

 11%|█         | 1298/12000 [46:45<8:22:12,  2.82s/it]
 11%|█         | 1299/12000 [46:47<7:15:27,  2.44s/it]
                                                      

 11%|█         | 1299/12000 [46:47<7:15:27,  2.44s/it]
 11%|█         | 1300/12000 [46:49<7:00:27,  2.36s/it]
                                                      

 11%|█         | 1300/12000 [46:49<7:00:27,  2.36s/it]
 11%|█         | 1301/12000 [46:54<9:05:18,  3.06s/it]
                                                      

 11%|█         | 1301/12000 [46:54<9:05:18,  3.06s/it]
 11%|█         | 1302/12000 [46:55<7:48:46,  2.63s/it]
                                                      

 11%|█         | 1302/12000 [46:55<7:48:46,  2.63s/it]
 11%|█         | 1303/12000 [46:57<6:49:28,  2.30s/it]
                                                      

 11%|█         | 1303/12000 [46:57<6:49:28,  2.30s/it]
 11%|█         | 1304/12000 [46:59<6:31:25,  2.20s/it]
                                                      

 11%|█         | 1304/12000 [46:59<6:31:25,  2.20s/it]
 11%|█         | 1305/12000 [47:00<6:01:28,  2.03s/it]
                                                      

 11%|█         | 1305/12000 [47:00<6:01:28,  2.03s/it]
 11%|█         | 1306/12000 [47:02<5:46:34,  1.94s/it]
                                                      

 11%|█         | 1306/12000 [47:02<5:46:34,  1.94s/it]
 11%|█         | 1307/12000 [47:04<5:17:41,  1.78s/it]
                                                      

 11%|█         | 1307/12000 [47:04<5:17:41,  1.78s/it]
 11%|█         | 1308/12000 [47:06<6:05:05,  2.05s/it]
                                                      

 11%|█         | 1308/12000 [47:06<6:05:05,  2.05s/it]
 11%|█         | 1309/12000 [47:08<5:37:50,  1.90s/it]
                                                      

 11%|█         | 1309/12000 [47:08<5:37:50,  1.90s/it]
 11%|█         | 1310/12000 [47:10<5:53:40,  1.99s/it]
                                                      

 11%|█         | 1310/12000 [47:10<5:53:40,  1.99s/it]
 11%|█         | 1311/12000 [47:12<6:11:32,  2.09s/it]
                                                      

 11%|█         | 1311/12000 [47:12<6:11:32,  2.09s/it]
 11%|█         | 1312/12000 [47:14<5:46:30,  1.95s/it]
                                                      

 11%|█         | 1312/12000 [47:14<5:46:30,  1.95s/it]
 11%|█         | 1313/12000 [47:15<5:18:55,  1.79s/it]
                                                      

 11%|█         | 1313/12000 [47:15<5:18:55,  1.79s/it]
 11%|█         | 1314/12000 [47:17<5:06:22,  1.72s/it]
                                                      

 11%|█         | 1314/12000 [47:17<5:06:22,  1.72s/it]
 11%|█         | 1315/12000 [47:21<7:36:56,  2.57s/it]
                                                      

 11%|█         | 1315/12000 [47:21<7:36:56,  2.57s/it]
 11%|█         | 1316/12000 [47:23<6:52:58,  2.32s/it]
                                                      

 11%|█         | 1316/12000 [47:23<6:52:58,  2.32s/it]
 11%|█         | 1317/12000 [47:25<6:14:58,  2.11s/it]
                                                      

 11%|█         | 1317/12000 [47:25<6:14:58,  2.11s/it]
 11%|█         | 1318/12000 [47:28<7:20:09,  2.47s/it]
                                                      

 11%|█         | 1318/12000 [47:28<7:20:09,  2.47s/it]
 11%|█         | 1319/12000 [47:30<7:04:19,  2.38s/it]
                                                      

 11%|█         | 1319/12000 [47:30<7:04:19,  2.38s/it]
 11%|█         | 1320/12000 [47:32<6:29:36,  2.19s/it]
                                                      
{'loss': '0.7598', 'grad_norm': '0.334', 'learning_rate': '0.0001983', 'entropy': '0.7697', 'num_tokens': '6.884e+06', 'mean_token_accuracy': '0.8215', 'epoch': '0.01127'}
{'loss': '0.7822', 'grad_norm': '0.3766', 'learning_rate': '0.0001983', 'entropy': '0.8556', 'num_tokens': '6.89e+06', 'mean_token_accuracy': '0.7912', 'epoch': '0.01127'}
{'loss': '0.7018', 'grad_norm': '0.3296', 'learning_rate': '0.0001983', 'entropy': '0.6798', 'num_tokens': '6.895e+06', 'mean_token_accuracy': '0.8212', 'epoch': '0.01128'}
{'loss': '1.216', 'grad_norm': '0.3665', 'learning_rate': '0.0001983', 'entropy': '1.072', 'num_tokens': '6.903e+06', 'mean_token_accuracy': '0.7382', 'epoch': '0.01129'}
{'loss': '1.076', 'grad_norm': '0.413', 'learning_rate': '0.0001983', 'entropy': '1.106', 'num_tokens': '6.907e+06', 'mean_token_accuracy': '0.7473', 'epoch': '0.0113'}
{'loss': '1.56', 'grad_norm': '0.3452', 'learning_rate': '0.0001983', 'entropy': '1.315', 'num_tokens': '6.917e+06', 'mean_token_accuracy': '0.7234', 'epoch': '0.01131'}
{'loss': '1.097', 'grad_norm': '0.3313', 'learning_rate': '0.0001983', 'entropy': '1.137', 'num_tokens': '6.925e+06', 'mean_token_accuracy': '0.7268', 'epoch': '0.01132'}
{'loss': '1.012', 'grad_norm': '0.3967', 'learning_rate': '0.0001983', 'entropy': '1.111', 'num_tokens': '6.929e+06', 'mean_token_accuracy': '0.7433', 'epoch': '0.01133'}
{'loss': '1.304', 'grad_norm': '0.4409', 'learning_rate': '0.0001982', 'entropy': '1.199', 'num_tokens': '6.934e+06', 'mean_token_accuracy': '0.7108', 'epoch': '0.01134'}
{'loss': '0.8677', 'grad_norm': '0.3933', 'learning_rate': '0.0001982', 'entropy': '0.8527', 'num_tokens': '6.939e+06', 'mean_token_accuracy': '0.8002', 'epoch': '0.01135'}
{'loss': '0.9322', 'grad_norm': '0.4077', 'learning_rate': '0.0001982', 'entropy': '0.9694', 'num_tokens': '6.943e+06', 'mean_token_accuracy': '0.757', 'epoch': '0.01135'}
{'loss': '0.83', 'grad_norm': '1.344', 'learning_rate': '0.0001982', 'entropy': '0.9259', 'num_tokens': '6.949e+06', 'mean_token_accuracy': '0.7785', 'epoch': '0.01136'}
{'loss': '1.091', 'grad_norm': '0.4195', 'learning_rate': '0.0001982', 'entropy': '0.9745', 'num_tokens': '6.957e+06', 'mean_token_accuracy': '0.7608', 'epoch': '0.01137'}
{'loss': '1.014', 'grad_norm': '0.3226', 'learning_rate': '0.0001982', 'entropy': '1.064', 'num_tokens': '6.963e+06', 'mean_token_accuracy': '0.7459', 'epoch': '0.01138'}
{'loss': '1.072', 'grad_norm': '0.3634', 'learning_rate': '0.0001982', 'entropy': '1.079', 'num_tokens': '6.968e+06', 'mean_token_accuracy': '0.7336', 'epoch': '0.01139'}
{'loss': '0.8184', 'grad_norm': '0.3309', 'learning_rate': '0.0001982', 'entropy': '0.8857', 'num_tokens': '6.974e+06', 'mean_token_accuracy': '0.7929', 'epoch': '0.0114'}
{'loss': '0.8807', 'grad_norm': '0.3436', 'learning_rate': '0.0001982', 'entropy': '0.919', 'num_tokens': '6.979e+06', 'mean_token_accuracy': '0.7853', 'epoch': '0.01141'}
{'loss': '0.8035', 'grad_norm': '0.4058', 'learning_rate': '0.0001982', 'entropy': '0.8045', 'num_tokens': '6.983e+06', 'mean_token_accuracy': '0.7844', 'epoch': '0.01142'}
{'loss': '0.8886', 'grad_norm': '0.2995', 'learning_rate': '0.0001982', 'entropy': '0.8551', 'num_tokens': '6.995e+06', 'mean_token_accuracy': '0.7906', 'epoch': '0.01143'}
{'loss': '0.762', 'grad_norm': '0.3745', 'learning_rate': '0.0001982', 'entropy': '0.81', 'num_tokens': '7.001e+06', 'mean_token_accuracy': '0.7947', 'epoch': '0.01143'}
{'loss': '0.6396', 'grad_norm': '0.3393', 'learning_rate': '0.0001982', 'entropy': '0.6898', 'num_tokens': '7.005e+06', 'mean_token_accuracy': '0.8208', 'epoch': '0.01144'}
{'loss': '0.8599', 'grad_norm': '0.3722', 'learning_rate': '0.0001982', 'entropy': '0.8516', 'num_tokens': '7.012e+06', 'mean_token_accuracy': '0.7997', 'epoch': '0.01145'}
{'loss': '0.8513', 'grad_norm': '0.3294', 'learning_rate': '0.0001982', 'entropy': '0.8707', 'num_tokens': '7.017e+06', 'mean_token_accuracy': '0.7939', 'epoch': '0.01146'}
{'loss': '0.7527', 'grad_norm': '0.2185', 'learning_rate': '0.0001982', 'entropy': '0.8338', 'num_tokens': '7.026e+06', 'mean_token_accuracy': '0.7926', 'epoch': '0.01147'}
{'loss': '0.6434', 'grad_norm': '0.2657', 'learning_rate': '0.0001982', 'entropy': '0.6276', 'num_tokens': '7.032e+06', 'mean_token_accuracy': '0.8442', 'epoch': '0.01148'}
{'loss': '0.8647', 'grad_norm': '0.4272', 'learning_rate': '0.0001982', 'entropy': '0.8638', 'num_tokens': '7.037e+06', 'mean_token_accuracy': '0.7915', 'epoch': '0.01149'}
{'loss': '0.8075', 'grad_norm': '0.3435', 'learning_rate': '0.0001982', 'entropy': '0.8366', 'num_tokens': '7.042e+06', 'mean_token_accuracy': '0.7969', 'epoch': '0.0115'}
{'loss': '1.14', 'grad_norm': '0.4518', 'learning_rate': '0.0001982', 'entropy': '1.121', 'num_tokens': '7.047e+06', 'mean_token_accuracy': '0.7409', 'epoch': '0.0115'}
{'loss': '0.8286', 'grad_norm': '0.3393', 'learning_rate': '0.0001981', 'entropy': '0.8741', 'num_tokens': '7.055e+06', 'mean_token_accuracy': '0.7845', 'epoch': '0.01151'}
{'loss': '0.7584', 'grad_norm': '0.3233', 'learning_rate': '0.0001981', 'entropy': '0.7426', 'num_tokens': '7.06e+06', 'mean_token_accuracy': '0.8125', 'epoch': '0.01152'}
{'loss': '0.7567', 'grad_norm': '0.341', 'learning_rate': '0.0001981', 'entropy': '0.738', 'num_tokens': '7.066e+06', 'mean_token_accuracy': '0.8183', 'epoch': '0.01153'}
{'loss': '1.356', 'grad_norm': '0.4197', 'learning_rate': '0.0001981', 'entropy': '1.232', 'num_tokens': '7.071e+06', 'mean_token_accuracy': '0.7085', 'epoch': '0.01154'}
{'loss': '0.7337', 'grad_norm': '0.345', 'learning_rate': '0.0001981', 'entropy': '0.7603', 'num_tokens': '7.076e+06', 'mean_token_accuracy': '0.7954', 'epoch': '0.01155'}
{'loss': '0.9429', 'grad_norm': '0.3843', 'learning_rate': '0.0001981', 'entropy': '0.9888', 'num_tokens': '7.081e+06', 'mean_token_accuracy': '0.755', 'epoch': '0.01156'}
{'loss': '0.9058', 'grad_norm': '0.457', 'learning_rate': '0.0001981', 'entropy': '0.9054', 'num_tokens': '7.084e+06', 'mean_token_accuracy': '0.7817', 'epoch': '0.01157'}
{'loss': '0.6951', 'grad_norm': '0.361', 'learning_rate': '0.0001981', 'entropy': '0.7222', 'num_tokens': '7.091e+06', 'mean_token_accuracy': '0.8101', 'epoch': '0.01158'}
{'loss': '0.586', 'grad_norm': '0.3101', 'learning_rate': '0.0001981', 'entropy': '0.5959', 'num_tokens': '7.096e+06', 'mean_token_accuracy': '0.8439', 'epoch': '0.01158'}
{'loss': '1.061', 'grad_norm': '0.3884', 'learning_rate': '0.0001981', 'entropy': '1.017', 'num_tokens': '7.102e+06', 'mean_token_accuracy': '0.7507', 'epoch': '0.01159'}
{'loss': '1.034', 'grad_norm': '0.387', 'learning_rate': '0.0001981', 'entropy': '1.037', 'num_tokens': '7.107e+06', 'mean_token_accuracy': '0.7489', 'epoch': '0.0116'}
{'loss': '0.8004', 'grad_norm': '0.3974', 'learning_rate': '0.0001981', 'entropy': '0.9066', 'num_tokens': '7.111e+06', 'mean_token_accuracy': '0.765', 'epoch': '0.01161'}
{'loss': '1.244', 'grad_norm': '0.4553', 'learning_rate': '0.0001981', 'entropy': '1.262', 'num_tokens': '7.115e+06', 'mean_token_accuracy': '0.6929', 'epoch': '0.01162'}
{'loss': '1.122', 'grad_norm': '0.4254', 'learning_rate': '0.0001981', 'entropy': '1.142', 'num_tokens': '7.119e+06', 'mean_token_accuracy': '0.7256', 'epoch': '0.01163'}
{'loss': '0.8131', 'grad_norm': '0.3935', 'learning_rate': '0.0001981', 'entropy': '0.7849', 'num_tokens': '7.128e+06', 'mean_token_accuracy': '0.8006', 'epoch': '0.01164'}
{'loss': '0.837', 'grad_norm': '0.3514', 'learning_rate': '0.0001981', 'entropy': '0.8525', 'num_tokens': '7.133e+06', 'mean_token_accuracy': '0.7799', 'epoch': '0.01165'}
{'loss': '0.917', 'grad_norm': '0.3551', 'learning_rate': '0.0001981', 'entropy': '0.8976', 'num_tokens': '7.138e+06', 'mean_token_accuracy': '0.7808', 'epoch': '0.01166'}
{'loss': '0.9765', 'grad_norm': '0.4983', 'learning_rate': '0.0001981', 'entropy': '0.9658', 'num_tokens': '7.141e+06', 'mean_token_accuracy': '0.7828', 'epoch': '0.01166'}
{'loss': '0.9399', 'grad_norm': '0.3936', 'learning_rate': '0.000198', 'entropy': '0.9294', 'num_tokens': '7.147e+06', 'mean_token_accuracy': '0.7792', 'epoch': '0.01167'}

 11%|█         | 1320/12000 [47:32<6:29:36,  2.19s/it]
 11%|█         | 1321/12000 [47:37<9:24:28,  3.17s/it]
                                                      

 11%|█         | 1321/12000 [47:37<9:24:28,  3.17s/it]
 11%|█         | 1322/12000 [47:39<8:04:08,  2.72s/it]
                                                      

 11%|█         | 1322/12000 [47:39<8:04:08,  2.72s/it]
 11%|█         | 1323/12000 [47:42<8:16:35,  2.79s/it]
                                                      

 11%|█         | 1323/12000 [47:42<8:16:35,  2.79s/it]
 11%|█         | 1324/12000 [47:45<7:57:43,  2.68s/it]
                                                      

 11%|█         | 1324/12000 [47:45<7:57:43,  2.68s/it]
 11%|█         | 1325/12000 [47:48<8:13:17,  2.77s/it]
                                                      

 11%|█         | 1325/12000 [47:48<8:13:17,  2.77s/it]
 11%|█         | 1326/12000 [47:49<7:20:50,  2.48s/it]
                                                      

 11%|█         | 1326/12000 [47:49<7:20:50,  2.48s/it]
 11%|█         | 1327/12000 [47:51<6:29:56,  2.19s/it]
                                                      

 11%|█         | 1327/12000 [47:51<6:29:56,  2.19s/it]
 11%|█         | 1328/12000 [47:53<6:05:40,  2.06s/it]
                                                      

 11%|█         | 1328/12000 [47:53<6:05:40,  2.06s/it]
 11%|█         | 1329/12000 [47:55<6:45:03,  2.28s/it]
                                                      

 11%|█         | 1329/12000 [47:55<6:45:03,  2.28s/it]
 11%|█         | 1330/12000 [47:57<6:08:12,  2.07s/it]
                                                      

 11%|█         | 1330/12000 [47:57<6:08:12,  2.07s/it]
 11%|█         | 1331/12000 [48:01<8:17:27,  2.80s/it]
                                                      

 11%|█         | 1331/12000 [48:01<8:17:27,  2.80s/it]
 11%|█         | 1332/12000 [48:03<7:13:00,  2.44s/it]
                                                      

 11%|█         | 1332/12000 [48:03<7:13:00,  2.44s/it]
 11%|█         | 1333/12000 [48:05<6:24:26,  2.16s/it]
                                                      

 11%|█         | 1333/12000 [48:05<6:24:26,  2.16s/it]
 11%|█         | 1334/12000 [48:08<7:07:19,  2.40s/it]
                                                      

 11%|█         | 1334/12000 [48:08<7:07:19,  2.40s/it]
 11%|█         | 1335/12000 [48:09<6:20:38,  2.14s/it]
                                                      

 11%|█         | 1335/12000 [48:09<6:20:38,  2.14s/it]
 11%|█         | 1336/12000 [48:11<6:05:34,  2.06s/it]
                                                      

 11%|█         | 1336/12000 [48:11<6:05:34,  2.06s/it]
 11%|█         | 1337/12000 [48:13<6:32:41,  2.21s/it]
                                                      

 11%|█         | 1337/12000 [48:13<6:32:41,  2.21s/it]
 11%|█         | 1338/12000 [48:15<6:11:22,  2.09s/it]
                                                      

 11%|█         | 1338/12000 [48:15<6:11:22,  2.09s/it]
 11%|█         | 1339/12000 [48:19<7:29:44,  2.53s/it]
                                                      

 11%|█         | 1339/12000 [48:19<7:29:44,  2.53s/it]
 11%|█         | 1340/12000 [48:20<6:36:36,  2.23s/it]
                                                      

 11%|█         | 1340/12000 [48:20<6:36:36,  2.23s/it]
 11%|█         | 1341/12000 [48:23<6:35:05,  2.22s/it]
                                                      

 11%|█         | 1341/12000 [48:23<6:35:05,  2.22s/it]
 11%|█         | 1342/12000 [48:25<6:25:23,  2.17s/it]
                                                      

 11%|█         | 1342/12000 [48:25<6:25:23,  2.17s/it]
 11%|█         | 1343/12000 [48:27<6:21:46,  2.15s/it]
                                                      

 11%|█         | 1343/12000 [48:27<6:21:46,  2.15s/it]
 11%|█         | 1344/12000 [48:31<7:53:29,  2.67s/it]
                                                      

 11%|█         | 1344/12000 [48:31<7:53:29,  2.67s/it]
 11%|█         | 1345/12000 [48:32<7:02:54,  2.38s/it]
                                                      

 11%|█         | 1345/12000 [48:32<7:02:54,  2.38s/it]
 11%|█         | 1346/12000 [48:34<6:10:43,  2.09s/it]
                                                      

 11%|█         | 1346/12000 [48:34<6:10:43,  2.09s/it]
 11%|█         | 1347/12000 [48:37<7:37:12,  2.58s/it]
                                                      

 11%|█         | 1347/12000 [48:37<7:37:12,  2.58s/it]
 11%|█         | 1348/12000 [48:39<7:01:47,  2.38s/it]
                                                      

 11%|█         | 1348/12000 [48:39<7:01:47,  2.38s/it]
 11%|█         | 1349/12000 [48:41<6:47:23,  2.29s/it]
                                                      

 11%|█         | 1349/12000 [48:41<6:47:23,  2.29s/it]
 11%|█▏        | 1350/12000 [48:43<6:19:58,  2.14s/it]
                                                      

 11%|█▏        | 1350/12000 [48:43<6:19:58,  2.14s/it]
 11%|█▏        | 1351/12000 [48:46<6:53:02,  2.33s/it]
                                                      

 11%|█▏        | 1351/12000 [48:46<6:53:02,  2.33s/it]
 11%|█▏        | 1352/12000 [48:48<6:22:08,  2.15s/it]
                                                      

 11%|█▏        | 1352/12000 [48:48<6:22:08,  2.15s/it]
 11%|█▏        | 1353/12000 [48:50<6:03:55,  2.05s/it]
                                                      

 11%|█▏        | 1353/12000 [48:50<6:03:55,  2.05s/it]
 11%|█▏        | 1354/12000 [48:51<5:38:14,  1.91s/it]
                                                      

 11%|█▏        | 1354/12000 [48:51<5:38:14,  1.91s/it]
 11%|█▏        | 1355/12000 [48:54<6:10:41,  2.09s/it]
                                                      

 11%|█▏        | 1355/12000 [48:54<6:10:41,  2.09s/it]
 11%|█▏        | 1356/12000 [48:55<5:37:23,  1.90s/it]
                                                      

 11%|█▏        | 1356/12000 [48:55<5:37:23,  1.90s/it]
 11%|█▏        | 1357/12000 [48:57<5:58:22,  2.02s/it]
                                                      

 11%|█▏        | 1357/12000 [48:57<5:58:22,  2.02s/it]
 11%|█▏        | 1358/12000 [48:59<5:46:22,  1.95s/it]
                                                      

 11%|█▏        | 1358/12000 [48:59<5:46:22,  1.95s/it]
 11%|█▏        | 1359/12000 [49:01<5:31:36,  1.87s/it]
                                                      

 11%|█▏        | 1359/12000 [49:01<5:31:36,  1.87s/it]
 11%|█▏        | 1360/12000 [49:02<5:07:11,  1.73s/it]
                                                      

 11%|█▏        | 1360/12000 [49:02<5:07:11,  1.73s/it]
 11%|█▏        | 1361/12000 [49:07<7:53:33,  2.67s/it]
                                                      

 11%|█▏        | 1361/12000 [49:07<7:53:33,  2.67s/it]
 11%|█▏        | 1362/12000 [49:10<8:07:36,  2.75s/it]
                                                      

 11%|█▏        | 1362/12000 [49:10<8:07:36,  2.75s/it]
 11%|█▏        | 1363/12000 [49:12<7:06:14,  2.40s/it]
                                                      

 11%|█▏        | 1363/12000 [49:12<7:06:14,  2.40s/it]
 11%|█▏        | 1364/12000 [49:13<6:29:47,  2.20s/it]
                                                      

 11%|█▏        | 1364/12000 [49:13<6:29:47,  2.20s/it]
 11%|█▏        | 1365/12000 [49:15<5:50:37,  1.98s/it]
                                                      

 11%|█▏        | 1365/12000 [49:15<5:50:37,  1.98s/it]
 11%|█▏        | 1366/12000 [49:17<5:57:43,  2.02s/it]
                                                      

 11%|█▏        | 1366/12000 [49:17<5:57:43,  2.02s/it]
 11%|█▏        | 1367/12000 [49:20<6:40:25,  2.26s/it]
                                                      
{'loss': '0.834', 'grad_norm': '0.3511', 'learning_rate': '0.000198', 'entropy': '0.8374', 'num_tokens': '7.153e+06', 'mean_token_accuracy': '0.7876', 'epoch': '0.01168'}
{'loss': '0.9339', 'grad_norm': '0.3048', 'learning_rate': '0.000198', 'entropy': '0.9409', 'num_tokens': '7.164e+06', 'mean_token_accuracy': '0.7689', 'epoch': '0.01169'}
{'loss': '0.8873', 'grad_norm': '0.4207', 'learning_rate': '0.000198', 'entropy': '0.9545', 'num_tokens': '7.168e+06', 'mean_token_accuracy': '0.7453', 'epoch': '0.0117'}
{'loss': '0.5959', 'grad_norm': '0.2837', 'learning_rate': '0.000198', 'entropy': '0.5745', 'num_tokens': '7.175e+06', 'mean_token_accuracy': '0.8476', 'epoch': '0.01171'}
{'loss': '0.9766', 'grad_norm': '0.2996', 'learning_rate': '0.000198', 'entropy': '1.007', 'num_tokens': '7.183e+06', 'mean_token_accuracy': '0.7498', 'epoch': '0.01172'}
{'loss': '1.062', 'grad_norm': '0.3319', 'learning_rate': '0.000198', 'entropy': '0.8735', 'num_tokens': '7.192e+06', 'mean_token_accuracy': '0.7765', 'epoch': '0.01173'}
{'loss': '0.9564', 'grad_norm': '0.386', 'learning_rate': '0.000198', 'entropy': '0.8744', 'num_tokens': '7.198e+06', 'mean_token_accuracy': '0.7748', 'epoch': '0.01173'}
{'loss': '0.9624', 'grad_norm': '0.4467', 'learning_rate': '0.000198', 'entropy': '1.005', 'num_tokens': '7.202e+06', 'mean_token_accuracy': '0.7522', 'epoch': '0.01174'}
{'loss': '0.859', 'grad_norm': '0.3566', 'learning_rate': '0.000198', 'entropy': '0.8418', 'num_tokens': '7.207e+06', 'mean_token_accuracy': '0.7772', 'epoch': '0.01175'}
{'loss': '0.9496', 'grad_norm': '0.3048', 'learning_rate': '0.000198', 'entropy': '0.924', 'num_tokens': '7.215e+06', 'mean_token_accuracy': '0.7689', 'epoch': '0.01176'}
{'loss': '0.9282', 'grad_norm': '0.3277', 'learning_rate': '0.000198', 'entropy': '0.9223', 'num_tokens': '7.22e+06', 'mean_token_accuracy': '0.7611', 'epoch': '0.01177'}
{'loss': '0.8009', 'grad_norm': '0.2797', 'learning_rate': '0.000198', 'entropy': '0.9538', 'num_tokens': '7.228e+06', 'mean_token_accuracy': '0.7719', 'epoch': '0.01178'}
{'loss': '1.119', 'grad_norm': '0.3948', 'learning_rate': '0.000198', 'entropy': '1.086', 'num_tokens': '7.233e+06', 'mean_token_accuracy': '0.7581', 'epoch': '0.01179'}
{'loss': '0.7123', 'grad_norm': '0.4068', 'learning_rate': '0.000198', 'entropy': '0.7586', 'num_tokens': '7.237e+06', 'mean_token_accuracy': '0.7956', 'epoch': '0.0118'}
{'loss': '0.6618', 'grad_norm': '0.2786', 'learning_rate': '0.000198', 'entropy': '0.7009', 'num_tokens': '7.245e+06', 'mean_token_accuracy': '0.8223', 'epoch': '0.01181'}
{'loss': '0.6983', 'grad_norm': '0.3688', 'learning_rate': '0.000198', 'entropy': '0.8316', 'num_tokens': '7.248e+06', 'mean_token_accuracy': '0.809', 'epoch': '0.01181'}
{'loss': '0.8175', 'grad_norm': '0.359', 'learning_rate': '0.000198', 'entropy': '0.7894', 'num_tokens': '7.254e+06', 'mean_token_accuracy': '0.7978', 'epoch': '0.01182'}
{'loss': '1.146', 'grad_norm': '0.3394', 'learning_rate': '0.000198', 'entropy': '1.069', 'num_tokens': '7.261e+06', 'mean_token_accuracy': '0.7471', 'epoch': '0.01183'}
{'loss': '0.8948', 'grad_norm': '0.378', 'learning_rate': '0.0001979', 'entropy': '0.9124', 'num_tokens': '7.267e+06', 'mean_token_accuracy': '0.7684', 'epoch': '0.01184'}
{'loss': '1.083', 'grad_norm': '0.3442', 'learning_rate': '0.0001979', 'entropy': '1.068', 'num_tokens': '7.272e+06', 'mean_token_accuracy': '0.7366', 'epoch': '0.01185'}
{'loss': '0.858', 'grad_norm': '0.3572', 'learning_rate': '0.0001979', 'entropy': '0.9672', 'num_tokens': '7.276e+06', 'mean_token_accuracy': '0.7698', 'epoch': '0.01186'}
{'loss': '1.047', 'grad_norm': '0.3755', 'learning_rate': '0.0001979', 'entropy': '0.9841', 'num_tokens': '7.281e+06', 'mean_token_accuracy': '0.7635', 'epoch': '0.01187'}
{'loss': '1.025', 'grad_norm': '0.4381', 'learning_rate': '0.0001979', 'entropy': '0.9388', 'num_tokens': '7.286e+06', 'mean_token_accuracy': '0.7661', 'epoch': '0.01188'}
{'loss': '0.9876', 'grad_norm': '0.3611', 'learning_rate': '0.0001979', 'entropy': '1.055', 'num_tokens': '7.292e+06', 'mean_token_accuracy': '0.7451', 'epoch': '0.01189'}
{'loss': '1.415', 'grad_norm': '0.4081', 'learning_rate': '0.0001979', 'entropy': '1.145', 'num_tokens': '7.3e+06', 'mean_token_accuracy': '0.7325', 'epoch': '0.01189'}
{'loss': '0.8315', 'grad_norm': '0.3899', 'learning_rate': '0.0001979', 'entropy': '0.7896', 'num_tokens': '7.305e+06', 'mean_token_accuracy': '0.7958', 'epoch': '0.0119'}
{'loss': '0.8368', 'grad_norm': '0.3757', 'learning_rate': '0.0001979', 'entropy': '0.8264', 'num_tokens': '7.309e+06', 'mean_token_accuracy': '0.8006', 'epoch': '0.01191'}
{'loss': '1.707', 'grad_norm': '0.3727', 'learning_rate': '0.0001979', 'entropy': '1.487', 'num_tokens': '7.317e+06', 'mean_token_accuracy': '0.6552', 'epoch': '0.01192'}
{'loss': '1.671', 'grad_norm': '0.4483', 'learning_rate': '0.0001979', 'entropy': '1.516', 'num_tokens': '7.322e+06', 'mean_token_accuracy': '0.647', 'epoch': '0.01193'}
{'loss': '1.334', 'grad_norm': '0.4582', 'learning_rate': '0.0001979', 'entropy': '1.271', 'num_tokens': '7.328e+06', 'mean_token_accuracy': '0.7044', 'epoch': '0.01194'}
{'loss': '1.234', 'grad_norm': '0.4344', 'learning_rate': '0.0001979', 'entropy': '1.157', 'num_tokens': '7.333e+06', 'mean_token_accuracy': '0.7187', 'epoch': '0.01195'}
{'loss': '0.914', 'grad_norm': '0.3971', 'learning_rate': '0.0001979', 'entropy': '0.8963', 'num_tokens': '7.337e+06', 'mean_token_accuracy': '0.783', 'epoch': '0.01196'}
{'loss': '0.9104', 'grad_norm': '0.414', 'learning_rate': '0.0001979', 'entropy': '0.9193', 'num_tokens': '7.342e+06', 'mean_token_accuracy': '0.7736', 'epoch': '0.01196'}
{'loss': '1.219', 'grad_norm': '0.4224', 'learning_rate': '0.0001979', 'entropy': '1.257', 'num_tokens': '7.347e+06', 'mean_token_accuracy': '0.7202', 'epoch': '0.01197'}
{'loss': '0.9213', 'grad_norm': '0.4327', 'learning_rate': '0.0001979', 'entropy': '0.9135', 'num_tokens': '7.351e+06', 'mean_token_accuracy': '0.7841', 'epoch': '0.01198'}
{'loss': '0.807', 'grad_norm': '0.3264', 'learning_rate': '0.0001978', 'entropy': '0.7604', 'num_tokens': '7.357e+06', 'mean_token_accuracy': '0.8054', 'epoch': '0.01199'}
{'loss': '0.9868', 'grad_norm': '0.3761', 'learning_rate': '0.0001978', 'entropy': '1.042', 'num_tokens': '7.361e+06', 'mean_token_accuracy': '0.7552', 'epoch': '0.012'}
{'loss': '1.165', 'grad_norm': '0.4197', 'learning_rate': '0.0001978', 'entropy': '1.082', 'num_tokens': '7.367e+06', 'mean_token_accuracy': '0.7594', 'epoch': '0.01201'}
{'loss': '0.7062', 'grad_norm': '0.3237', 'learning_rate': '0.0001978', 'entropy': '0.7372', 'num_tokens': '7.373e+06', 'mean_token_accuracy': '0.8135', 'epoch': '0.01202'}
{'loss': '1.225', 'grad_norm': '0.3646', 'learning_rate': '0.0001978', 'entropy': '1.143', 'num_tokens': '7.378e+06', 'mean_token_accuracy': '0.7438', 'epoch': '0.01203'}
{'loss': '0.7288', 'grad_norm': '0.351', 'learning_rate': '0.0001978', 'entropy': '0.7391', 'num_tokens': '7.382e+06', 'mean_token_accuracy': '0.8121', 'epoch': '0.01204'}
{'loss': '0.8585', 'grad_norm': '0.2744', 'learning_rate': '0.0001978', 'entropy': '0.9015', 'num_tokens': '7.39e+06', 'mean_token_accuracy': '0.7682', 'epoch': '0.01204'}
{'loss': '0.733', 'grad_norm': '0.3391', 'learning_rate': '0.0001978', 'entropy': '0.7795', 'num_tokens': '7.397e+06', 'mean_token_accuracy': '0.8023', 'epoch': '0.01205'}
{'loss': '0.8308', 'grad_norm': '0.4243', 'learning_rate': '0.0001978', 'entropy': '0.8257', 'num_tokens': '7.401e+06', 'mean_token_accuracy': '0.7874', 'epoch': '0.01206'}
{'loss': '1.182', 'grad_norm': '0.4309', 'learning_rate': '0.0001978', 'entropy': '1.128', 'num_tokens': '7.406e+06', 'mean_token_accuracy': '0.7225', 'epoch': '0.01207'}
{'loss': '0.9112', 'grad_norm': '0.4393', 'learning_rate': '0.0001978', 'entropy': '0.8733', 'num_tokens': '7.41e+06', 'mean_token_accuracy': '0.7789', 'epoch': '0.01208'}
{'loss': '0.8618', 'grad_norm': '0.3831', 'learning_rate': '0.0001978', 'entropy': '0.7694', 'num_tokens': '7.415e+06', 'mean_token_accuracy': '0.8005', 'epoch': '0.01209'}

 11%|█▏        | 1367/12000 [49:20<6:40:25,  2.26s/it]
 11%|█▏        | 1368/12000 [49:22<6:34:44,  2.23s/it]
                                                      

 11%|█▏        | 1368/12000 [49:22<6:34:44,  2.23s/it]
 11%|█▏        | 1369/12000 [49:24<6:19:50,  2.14s/it]
                                                      

 11%|█▏        | 1369/12000 [49:24<6:19:50,  2.14s/it]
 11%|█▏        | 1370/12000 [49:25<5:47:38,  1.96s/it]
                                                      

 11%|█▏        | 1370/12000 [49:25<5:47:38,  1.96s/it]
 11%|█▏        | 1371/12000 [49:27<5:48:09,  1.97s/it]
                                                      

 11%|█▏        | 1371/12000 [49:27<5:48:09,  1.97s/it]
 11%|█▏        | 1372/12000 [49:31<7:05:11,  2.40s/it]
                                                      

 11%|█▏        | 1372/12000 [49:31<7:05:11,  2.40s/it]
 11%|█▏        | 1373/12000 [49:33<6:56:51,  2.35s/it]
                                                      

 11%|█▏        | 1373/12000 [49:33<6:56:51,  2.35s/it]
 11%|█▏        | 1374/12000 [49:35<6:20:35,  2.15s/it]
                                                      

 11%|█▏        | 1374/12000 [49:35<6:20:35,  2.15s/it]
 11%|█▏        | 1375/12000 [49:37<6:01:23,  2.04s/it]
                                                      

 11%|█▏        | 1375/12000 [49:37<6:01:23,  2.04s/it]
 11%|█▏        | 1376/12000 [49:39<6:15:30,  2.12s/it]
                                                      

 11%|█▏        | 1376/12000 [49:39<6:15:30,  2.12s/it]
 11%|█▏        | 1377/12000 [49:40<5:46:52,  1.96s/it]
                                                      

 11%|█▏        | 1377/12000 [49:40<5:46:52,  1.96s/it]
 11%|█▏        | 1378/12000 [49:43<6:37:30,  2.25s/it]
                                                      

 11%|█▏        | 1378/12000 [49:43<6:37:30,  2.25s/it]
 11%|█▏        | 1379/12000 [49:46<7:23:08,  2.50s/it]
                                                      

 11%|█▏        | 1379/12000 [49:46<7:23:08,  2.50s/it]
 12%|█▏        | 1380/12000 [49:48<6:31:49,  2.21s/it]
                                                      

 12%|█▏        | 1380/12000 [49:48<6:31:49,  2.21s/it]
 12%|█▏        | 1381/12000 [49:51<6:51:59,  2.33s/it]
                                                      

 12%|█▏        | 1381/12000 [49:51<6:51:59,  2.33s/it]
 12%|█▏        | 1382/12000 [49:52<6:04:29,  2.06s/it]
                                                      

 12%|█▏        | 1382/12000 [49:52<6:04:29,  2.06s/it]
 12%|█▏        | 1383/12000 [49:53<5:31:57,  1.88s/it]
                                                      

 12%|█▏        | 1383/12000 [49:53<5:31:57,  1.88s/it]
 12%|█▏        | 1384/12000 [49:58<7:46:30,  2.64s/it]
                                                      

 12%|█▏        | 1384/12000 [49:58<7:46:30,  2.64s/it]
 12%|█▏        | 1385/12000 [50:01<8:17:27,  2.81s/it]
                                                      

 12%|█▏        | 1385/12000 [50:01<8:17:27,  2.81s/it]
 12%|█▏        | 1386/12000 [50:04<8:27:55,  2.87s/it]
                                                      

 12%|█▏        | 1386/12000 [50:04<8:27:55,  2.87s/it]
 12%|█▏        | 1387/12000 [50:06<7:24:40,  2.51s/it]
                                                      

 12%|█▏        | 1387/12000 [50:06<7:24:40,  2.51s/it]
 12%|█▏        | 1388/12000 [50:08<7:16:01,  2.47s/it]
                                                      

 12%|█▏        | 1388/12000 [50:08<7:16:01,  2.47s/it]
 12%|█▏        | 1389/12000 [50:10<7:03:28,  2.39s/it]
                                                      

 12%|█▏        | 1389/12000 [50:10<7:03:28,  2.39s/it]
 12%|█▏        | 1390/12000 [50:12<6:20:11,  2.15s/it]
                                                      

 12%|█▏        | 1390/12000 [50:12<6:20:11,  2.15s/it]
 12%|█▏        | 1391/12000 [50:13<5:47:10,  1.96s/it]
                                                      

 12%|█▏        | 1391/12000 [50:13<5:47:10,  1.96s/it]
 12%|█▏        | 1392/12000 [50:15<5:21:10,  1.82s/it]
                                                      

 12%|█▏        | 1392/12000 [50:15<5:21:10,  1.82s/it]
 12%|█▏        | 1393/12000 [50:17<5:34:29,  1.89s/it]
                                                      

 12%|█▏        | 1393/12000 [50:17<5:34:29,  1.89s/it]
 12%|█▏        | 1394/12000 [50:19<5:59:15,  2.03s/it]
                                                      

 12%|█▏        | 1394/12000 [50:19<5:59:15,  2.03s/it]
 12%|█▏        | 1395/12000 [50:21<5:34:12,  1.89s/it]
                                                      

 12%|█▏        | 1395/12000 [50:21<5:34:12,  1.89s/it]
 12%|█▏        | 1396/12000 [50:23<5:19:43,  1.81s/it]
                                                      

 12%|█▏        | 1396/12000 [50:23<5:19:43,  1.81s/it]
 12%|█▏        | 1397/12000 [50:24<5:08:27,  1.75s/it]
                                                      

 12%|█▏        | 1397/12000 [50:24<5:08:27,  1.75s/it]
 12%|█▏        | 1398/12000 [50:26<5:08:50,  1.75s/it]
                                                      

 12%|█▏        | 1398/12000 [50:26<5:08:50,  1.75s/it]
 12%|█▏        | 1399/12000 [50:28<5:41:42,  1.93s/it]
                                                      

 12%|█▏        | 1399/12000 [50:28<5:41:42,  1.93s/it]
 12%|█▏        | 1400/12000 [50:30<5:31:38,  1.88s/it]
                                                      

 12%|█▏        | 1400/12000 [50:30<5:31:38,  1.88s/it]
 12%|█▏        | 1401/12000 [50:33<6:45:17,  2.29s/it]
                                                      

 12%|█▏        | 1401/12000 [50:33<6:45:17,  2.29s/it]
 12%|█▏        | 1402/12000 [50:35<6:05:42,  2.07s/it]
                                                      

 12%|█▏        | 1402/12000 [50:35<6:05:42,  2.07s/it]
 12%|█▏        | 1403/12000 [50:36<5:35:37,  1.90s/it]
                                                      

 12%|█▏        | 1403/12000 [50:36<5:35:37,  1.90s/it]
 12%|█▏        | 1404/12000 [50:38<5:14:41,  1.78s/it]
                                                      

 12%|█▏        | 1404/12000 [50:38<5:14:41,  1.78s/it]
 12%|█▏        | 1405/12000 [50:41<6:52:30,  2.34s/it]
                                                      

 12%|█▏        | 1405/12000 [50:41<6:52:30,  2.34s/it]
 12%|█▏        | 1406/12000 [50:44<6:47:46,  2.31s/it]
                                                      

 12%|█▏        | 1406/12000 [50:44<6:47:46,  2.31s/it]
 12%|█▏        | 1407/12000 [50:46<6:51:14,  2.33s/it]
                                                      

 12%|█▏        | 1407/12000 [50:46<6:51:14,  2.33s/it]
 12%|█▏        | 1408/12000 [50:48<6:14:15,  2.12s/it]
                                                      

 12%|█▏        | 1408/12000 [50:48<6:14:15,  2.12s/it]
 12%|█▏        | 1409/12000 [50:49<5:52:36,  2.00s/it]
                                                      

 12%|█▏        | 1409/12000 [50:49<5:52:36,  2.00s/it]
 12%|█▏        | 1410/12000 [50:51<5:30:36,  1.87s/it]
                                                      

 12%|█▏        | 1410/12000 [50:51<5:30:36,  1.87s/it]
 12%|█▏        | 1411/12000 [50:52<5:10:40,  1.76s/it]
                                                      

 12%|█▏        | 1411/12000 [50:53<5:10:40,  1.76s/it]
 12%|█▏        | 1412/12000 [50:54<5:03:44,  1.72s/it]
                                                      

 12%|█▏        | 1412/12000 [50:54<5:03:44,  1.72s/it]
 12%|█▏        | 1413/12000 [50:56<5:01:12,  1.71s/it]
                                                      

 12%|█▏        | 1413/12000 [50:56<5:01:12,  1.71s/it]
 12%|█▏        | 1414/12000 [50:57<4:52:41,  1.66s/it]
                                                      
{'loss': '0.9335', 'grad_norm': '0.3055', 'learning_rate': '0.0001978', 'entropy': '1.036', 'num_tokens': '7.422e+06', 'mean_token_accuracy': '0.7576', 'epoch': '0.0121'}
{'loss': '0.8713', 'grad_norm': '0.3652', 'learning_rate': '0.0001978', 'entropy': '0.9258', 'num_tokens': '7.429e+06', 'mean_token_accuracy': '0.771', 'epoch': '0.01211'}
{'loss': '0.9926', 'grad_norm': '0.3872', 'learning_rate': '0.0001978', 'entropy': '1.071', 'num_tokens': '7.434e+06', 'mean_token_accuracy': '0.7453', 'epoch': '0.01212'}
{'loss': '0.6195', 'grad_norm': '0.2809', 'learning_rate': '0.0001978', 'entropy': '0.6131', 'num_tokens': '7.439e+06', 'mean_token_accuracy': '0.8444', 'epoch': '0.01212'}
{'loss': '1.095', 'grad_norm': '0.3424', 'learning_rate': '0.0001978', 'entropy': '1.042', 'num_tokens': '7.444e+06', 'mean_token_accuracy': '0.7452', 'epoch': '0.01213'}
{'loss': '0.9227', 'grad_norm': '0.4282', 'learning_rate': '0.0001978', 'entropy': '0.9905', 'num_tokens': '7.451e+06', 'mean_token_accuracy': '0.7612', 'epoch': '0.01214'}
{'loss': '0.7789', 'grad_norm': '0.397', 'learning_rate': '0.0001977', 'entropy': '0.6607', 'num_tokens': '7.458e+06', 'mean_token_accuracy': '0.8486', 'epoch': '0.01215'}
{'loss': '0.7154', 'grad_norm': '0.3423', 'learning_rate': '0.0001977', 'entropy': '0.7434', 'num_tokens': '7.463e+06', 'mean_token_accuracy': '0.8045', 'epoch': '0.01216'}
{'loss': '0.9408', 'grad_norm': '0.3573', 'learning_rate': '0.0001977', 'entropy': '0.9136', 'num_tokens': '7.468e+06', 'mean_token_accuracy': '0.7761', 'epoch': '0.01217'}
{'loss': '0.9051', 'grad_norm': '0.4773', 'learning_rate': '0.0001977', 'entropy': '0.9087', 'num_tokens': '7.474e+06', 'mean_token_accuracy': '0.7717', 'epoch': '0.01218'}
{'loss': '1.056', 'grad_norm': '0.3606', 'learning_rate': '0.0001977', 'entropy': '1.011', 'num_tokens': '7.479e+06', 'mean_token_accuracy': '0.7478', 'epoch': '0.01219'}
{'loss': '0.852', 'grad_norm': '0.3539', 'learning_rate': '0.0001977', 'entropy': '0.8243', 'num_tokens': '7.485e+06', 'mean_token_accuracy': '0.7939', 'epoch': '0.0122'}
{'loss': '1.024', 'grad_norm': '0.3526', 'learning_rate': '0.0001977', 'entropy': '1.034', 'num_tokens': '7.493e+06', 'mean_token_accuracy': '0.7476', 'epoch': '0.0122'}
{'loss': '1.095', 'grad_norm': '0.406', 'learning_rate': '0.0001977', 'entropy': '1.053', 'num_tokens': '7.498e+06', 'mean_token_accuracy': '0.7407', 'epoch': '0.01221'}
{'loss': '1.383', 'grad_norm': '0.4408', 'learning_rate': '0.0001977', 'entropy': '1.19', 'num_tokens': '7.504e+06', 'mean_token_accuracy': '0.7323', 'epoch': '0.01222'}
{'loss': '0.8912', 'grad_norm': '0.3721', 'learning_rate': '0.0001977', 'entropy': '1.009', 'num_tokens': '7.508e+06', 'mean_token_accuracy': '0.7576', 'epoch': '0.01223'}
{'loss': '0.7356', 'grad_norm': '0.3737', 'learning_rate': '0.0001977', 'entropy': '0.848', 'num_tokens': '7.512e+06', 'mean_token_accuracy': '0.8042', 'epoch': '0.01224'}
{'loss': '1.262', 'grad_norm': '0.4404', 'learning_rate': '0.0001977', 'entropy': '1.1', 'num_tokens': '7.52e+06', 'mean_token_accuracy': '0.753', 'epoch': '0.01225'}
{'loss': '1.159', 'grad_norm': '0.3628', 'learning_rate': '0.0001977', 'entropy': '1.079', 'num_tokens': '7.528e+06', 'mean_token_accuracy': '0.7481', 'epoch': '0.01226'}
{'loss': '1.117', 'grad_norm': '0.3856', 'learning_rate': '0.0001977', 'entropy': '0.9778', 'num_tokens': '7.536e+06', 'mean_token_accuracy': '0.7627', 'epoch': '0.01227'}
{'loss': '1.037', 'grad_norm': '0.3359', 'learning_rate': '0.0001977', 'entropy': '0.9622', 'num_tokens': '7.54e+06', 'mean_token_accuracy': '0.7672', 'epoch': '0.01227'}
{'loss': '0.7582', 'grad_norm': '0.3065', 'learning_rate': '0.0001977', 'entropy': '0.8779', 'num_tokens': '7.546e+06', 'mean_token_accuracy': '0.7844', 'epoch': '0.01228'}
{'loss': '0.8307', 'grad_norm': '0.3621', 'learning_rate': '0.0001977', 'entropy': '0.7751', 'num_tokens': '7.552e+06', 'mean_token_accuracy': '0.8049', 'epoch': '0.01229'}
{'loss': '1.104', 'grad_norm': '0.3902', 'learning_rate': '0.0001976', 'entropy': '1.14', 'num_tokens': '7.557e+06', 'mean_token_accuracy': '0.7266', 'epoch': '0.0123'}
{'loss': '1.111', 'grad_norm': '0.3881', 'learning_rate': '0.0001976', 'entropy': '1.144', 'num_tokens': '7.562e+06', 'mean_token_accuracy': '0.7102', 'epoch': '0.01231'}
{'loss': '0.8676', 'grad_norm': '0.3796', 'learning_rate': '0.0001976', 'entropy': '0.887', 'num_tokens': '7.566e+06', 'mean_token_accuracy': '0.7832', 'epoch': '0.01232'}
{'loss': '1.111', 'grad_norm': '0.4039', 'learning_rate': '0.0001976', 'entropy': '1.04', 'num_tokens': '7.57e+06', 'mean_token_accuracy': '0.7395', 'epoch': '0.01233'}
{'loss': '1.015', 'grad_norm': '0.3548', 'learning_rate': '0.0001976', 'entropy': '0.9678', 'num_tokens': '7.576e+06', 'mean_token_accuracy': '0.7643', 'epoch': '0.01234'}
{'loss': '0.9418', 'grad_norm': '0.4099', 'learning_rate': '0.0001976', 'entropy': '1.057', 'num_tokens': '7.58e+06', 'mean_token_accuracy': '0.7576', 'epoch': '0.01235'}
{'loss': '0.7607', 'grad_norm': '0.2954', 'learning_rate': '0.0001976', 'entropy': '0.788', 'num_tokens': '7.585e+06', 'mean_token_accuracy': '0.8047', 'epoch': '0.01235'}
{'loss': '0.8233', 'grad_norm': '0.3579', 'learning_rate': '0.0001976', 'entropy': '0.8639', 'num_tokens': '7.589e+06', 'mean_token_accuracy': '0.7846', 'epoch': '0.01236'}
{'loss': '0.8248', 'grad_norm': '0.3098', 'learning_rate': '0.0001976', 'entropy': '0.931', 'num_tokens': '7.594e+06', 'mean_token_accuracy': '0.7818', 'epoch': '0.01237'}
{'loss': '0.7709', 'grad_norm': '0.3309', 'learning_rate': '0.0001976', 'entropy': '0.9698', 'num_tokens': '7.6e+06', 'mean_token_accuracy': '0.7644', 'epoch': '0.01238'}
{'loss': '0.7853', 'grad_norm': '0.3599', 'learning_rate': '0.0001976', 'entropy': '0.8263', 'num_tokens': '7.604e+06', 'mean_token_accuracy': '0.8053', 'epoch': '0.01239'}
{'loss': '0.7974', 'grad_norm': '0.3922', 'learning_rate': '0.0001976', 'entropy': '0.8393', 'num_tokens': '7.61e+06', 'mean_token_accuracy': '0.8045', 'epoch': '0.0124'}
{'loss': '1.153', 'grad_norm': '0.4619', 'learning_rate': '0.0001976', 'entropy': '1.127', 'num_tokens': '7.614e+06', 'mean_token_accuracy': '0.7279', 'epoch': '0.01241'}
{'loss': '0.7731', 'grad_norm': '0.3817', 'learning_rate': '0.0001976', 'entropy': '0.7954', 'num_tokens': '7.617e+06', 'mean_token_accuracy': '0.7982', 'epoch': '0.01242'}
{'loss': '1.193', 'grad_norm': '0.3823', 'learning_rate': '0.0001976', 'entropy': '1.156', 'num_tokens': '7.621e+06', 'mean_token_accuracy': '0.7258', 'epoch': '0.01243'}
{'loss': '0.6296', 'grad_norm': '0.3278', 'learning_rate': '0.0001976', 'entropy': '0.6246', 'num_tokens': '7.626e+06', 'mean_token_accuracy': '0.8495', 'epoch': '0.01243'}
{'loss': '1.218', 'grad_norm': '0.3618', 'learning_rate': '0.0001975', 'entropy': '1.138', 'num_tokens': '7.634e+06', 'mean_token_accuracy': '0.7283', 'epoch': '0.01244'}
{'loss': '1.213', 'grad_norm': '0.4102', 'learning_rate': '0.0001975', 'entropy': '1.162', 'num_tokens': '7.64e+06', 'mean_token_accuracy': '0.709', 'epoch': '0.01245'}
{'loss': '0.9594', 'grad_norm': '0.4072', 'learning_rate': '0.0001975', 'entropy': '1.001', 'num_tokens': '7.645e+06', 'mean_token_accuracy': '0.7519', 'epoch': '0.01246'}
{'loss': '1.207', 'grad_norm': '0.3519', 'learning_rate': '0.0001975', 'entropy': '1.053', 'num_tokens': '7.651e+06', 'mean_token_accuracy': '0.7422', 'epoch': '0.01247'}
{'loss': '1.341', 'grad_norm': '0.4599', 'learning_rate': '0.0001975', 'entropy': '1.299', 'num_tokens': '7.654e+06', 'mean_token_accuracy': '0.7063', 'epoch': '0.01248'}
{'loss': '0.853', 'grad_norm': '0.402', 'learning_rate': '0.0001975', 'entropy': '0.9225', 'num_tokens': '7.658e+06', 'mean_token_accuracy': '0.7781', 'epoch': '0.01249'}
{'loss': '1.113', 'grad_norm': '0.4126', 'learning_rate': '0.0001975', 'entropy': '1.154', 'num_tokens': '7.662e+06', 'mean_token_accuracy': '0.7241', 'epoch': '0.0125'}
{'loss': '0.5602', 'grad_norm': '0.2401', 'learning_rate': '0.0001975', 'entropy': '0.6364', 'num_tokens': '7.668e+06', 'mean_token_accuracy': '0.8451', 'epoch': '0.0125'}

 12%|█▏        | 1414/12000 [50:57<4:52:41,  1.66s/it]
 12%|█▏        | 1415/12000 [51:00<5:46:21,  1.96s/it]
                                                      

 12%|█▏        | 1415/12000 [51:00<5:46:21,  1.96s/it]
 12%|█▏        | 1416/12000 [51:02<5:41:03,  1.93s/it]
                                                      

 12%|█▏        | 1416/12000 [51:02<5:41:03,  1.93s/it]
 12%|█▏        | 1417/12000 [51:04<5:53:08,  2.00s/it]
                                                      

 12%|█▏        | 1417/12000 [51:04<5:53:08,  2.00s/it]
 12%|█▏        | 1418/12000 [51:05<5:22:44,  1.83s/it]
                                                      

 12%|█▏        | 1418/12000 [51:05<5:22:44,  1.83s/it]
 12%|█▏        | 1419/12000 [51:07<5:29:34,  1.87s/it]
                                                      

 12%|█▏        | 1419/12000 [51:07<5:29:34,  1.87s/it]
 12%|█▏        | 1420/12000 [51:10<5:43:54,  1.95s/it]
                                                      

 12%|█▏        | 1420/12000 [51:10<5:43:54,  1.95s/it]
 12%|█▏        | 1421/12000 [51:11<5:22:09,  1.83s/it]
                                                      

 12%|█▏        | 1421/12000 [51:11<5:22:09,  1.83s/it]
 12%|█▏        | 1422/12000 [51:15<6:56:55,  2.36s/it]
                                                      

 12%|█▏        | 1422/12000 [51:15<6:56:55,  2.36s/it]
 12%|█▏        | 1423/12000 [51:16<6:18:00,  2.14s/it]
                                                      

 12%|█▏        | 1423/12000 [51:16<6:18:00,  2.14s/it]
 12%|█▏        | 1424/12000 [51:18<6:01:48,  2.05s/it]
                                                      

 12%|█▏        | 1424/12000 [51:18<6:01:48,  2.05s/it]
 12%|█▏        | 1425/12000 [51:23<8:35:54,  2.93s/it]
                                                      

 12%|█▏        | 1425/12000 [51:23<8:35:54,  2.93s/it]
 12%|█▏        | 1426/12000 [51:25<7:39:18,  2.61s/it]
                                                      

 12%|█▏        | 1426/12000 [51:25<7:39:18,  2.61s/it]
 12%|█▏        | 1427/12000 [51:29<8:27:50,  2.88s/it]
                                                      

 12%|█▏        | 1427/12000 [51:29<8:27:50,  2.88s/it]
 12%|█▏        | 1428/12000 [51:30<7:37:10,  2.59s/it]
                                                      

 12%|█▏        | 1428/12000 [51:30<7:37:10,  2.59s/it]
 12%|█▏        | 1429/12000 [51:33<7:33:20,  2.57s/it]
                                                      

 12%|█▏        | 1429/12000 [51:33<7:33:20,  2.57s/it]
 12%|█▏        | 1430/12000 [51:35<6:53:46,  2.35s/it]
                                                      

 12%|█▏        | 1430/12000 [51:35<6:53:46,  2.35s/it]
 12%|█▏        | 1431/12000 [51:38<7:27:01,  2.54s/it]
                                                      

 12%|█▏        | 1431/12000 [51:38<7:27:01,  2.54s/it]
 12%|█▏        | 1432/12000 [51:39<6:30:55,  2.22s/it]
                                                      

 12%|█▏        | 1432/12000 [51:39<6:30:55,  2.22s/it]
 12%|█▏        | 1433/12000 [51:41<6:26:16,  2.19s/it]
                                                      

 12%|█▏        | 1433/12000 [51:41<6:26:16,  2.19s/it]
 12%|█▏        | 1434/12000 [51:43<6:14:55,  2.13s/it]
                                                      

 12%|█▏        | 1434/12000 [51:43<6:14:55,  2.13s/it]
 12%|█▏        | 1435/12000 [51:45<5:49:35,  1.99s/it]
                                                      

 12%|█▏        | 1435/12000 [51:45<5:49:35,  1.99s/it]
 12%|█▏        | 1436/12000 [51:47<5:29:27,  1.87s/it]
                                                      

 12%|█▏        | 1436/12000 [51:47<5:29:27,  1.87s/it]
 12%|█▏        | 1437/12000 [51:48<5:15:38,  1.79s/it]
                                                      

 12%|█▏        | 1437/12000 [51:48<5:15:38,  1.79s/it]
 12%|█▏        | 1438/12000 [51:50<4:59:24,  1.70s/it]
                                                      

 12%|█▏        | 1438/12000 [51:50<4:59:24,  1.70s/it]
 12%|█▏        | 1439/12000 [51:52<5:20:43,  1.82s/it]
                                                      

 12%|█▏        | 1439/12000 [51:52<5:20:43,  1.82s/it]
 12%|█▏        | 1440/12000 [51:55<6:11:06,  2.11s/it]
                                                      

 12%|█▏        | 1440/12000 [51:55<6:11:06,  2.11s/it]
 12%|█▏        | 1441/12000 [51:56<5:56:46,  2.03s/it]
                                                      

 12%|█▏        | 1441/12000 [51:56<5:56:46,  2.03s/it]
 12%|█▏        | 1442/12000 [51:59<6:07:44,  2.09s/it]
                                                      

 12%|█▏        | 1442/12000 [51:59<6:07:44,  2.09s/it]
 12%|█▏        | 1443/12000 [52:00<5:44:50,  1.96s/it]
                                                      

 12%|█▏        | 1443/12000 [52:00<5:44:50,  1.96s/it]
 12%|█▏        | 1444/12000 [52:02<5:21:31,  1.83s/it]
                                                      

 12%|█▏        | 1444/12000 [52:02<5:21:31,  1.83s/it]
 12%|█▏        | 1445/12000 [52:04<5:12:26,  1.78s/it]
                                                      

 12%|█▏        | 1445/12000 [52:04<5:12:26,  1.78s/it]
 12%|█▏        | 1446/12000 [52:05<5:11:49,  1.77s/it]
                                                      

 12%|█▏        | 1446/12000 [52:05<5:11:49,  1.77s/it]
 12%|█▏        | 1447/12000 [52:07<5:00:29,  1.71s/it]
                                                      

 12%|█▏        | 1447/12000 [52:07<5:00:29,  1.71s/it]
 12%|█▏        | 1448/12000 [52:09<5:19:52,  1.82s/it]
                                                      

 12%|█▏        | 1448/12000 [52:09<5:19:52,  1.82s/it]
 12%|█▏        | 1449/12000 [52:13<7:08:26,  2.44s/it]
                                                      

 12%|█▏        | 1449/12000 [52:13<7:08:26,  2.44s/it]
 12%|█▏        | 1450/12000 [52:14<6:16:44,  2.14s/it]
                                                      

 12%|█▏        | 1450/12000 [52:14<6:16:44,  2.14s/it]
 12%|█▏        | 1451/12000 [52:17<6:51:30,  2.34s/it]
                                                      

 12%|█▏        | 1451/12000 [52:17<6:51:30,  2.34s/it]
 12%|█▏        | 1452/12000 [52:19<6:12:01,  2.12s/it]
                                                      

 12%|█▏        | 1452/12000 [52:19<6:12:01,  2.12s/it]
 12%|█▏        | 1453/12000 [52:21<6:04:52,  2.08s/it]
                                                      

 12%|█▏        | 1453/12000 [52:21<6:04:52,  2.08s/it]
 12%|█▏        | 1454/12000 [52:22<5:30:53,  1.88s/it]
                                                      

 12%|█▏        | 1454/12000 [52:22<5:30:53,  1.88s/it]
 12%|█▏        | 1455/12000 [52:24<5:23:36,  1.84s/it]
                                                      

 12%|█▏        | 1455/12000 [52:24<5:23:36,  1.84s/it]
 12%|█▏        | 1456/12000 [52:25<5:12:07,  1.78s/it]
                                                      

 12%|█▏        | 1456/12000 [52:25<5:12:07,  1.78s/it]
 12%|█▏        | 1457/12000 [52:27<5:20:21,  1.82s/it]
                                                      

 12%|█▏        | 1457/12000 [52:27<5:20:21,  1.82s/it]
 12%|█▏        | 1458/12000 [52:30<5:36:40,  1.92s/it]
                                                      

 12%|█▏        | 1458/12000 [52:30<5:36:40,  1.92s/it]
 12%|█▏        | 1459/12000 [52:31<5:22:13,  1.83s/it]
                                                      

 12%|█▏        | 1459/12000 [52:31<5:22:13,  1.83s/it]
 12%|█▏        | 1460/12000 [52:34<6:29:20,  2.22s/it]
                                                      

 12%|█▏        | 1460/12000 [52:34<6:29:20,  2.22s/it]
 12%|█▏        | 1461/12000 [52:36<5:59:12,  2.05s/it]
                                                      
{'loss': '0.9106', 'grad_norm': '0.295', 'learning_rate': '0.0001975', 'entropy': '0.9861', 'num_tokens': '7.673e+06', 'mean_token_accuracy': '0.7729', 'epoch': '0.01251'}
{'loss': '1.252', 'grad_norm': '0.408', 'learning_rate': '0.0001975', 'entropy': '1.266', 'num_tokens': '7.679e+06', 'mean_token_accuracy': '0.6955', 'epoch': '0.01252'}
{'loss': '1.17', 'grad_norm': '0.4101', 'learning_rate': '0.0001975', 'entropy': '1.21', 'num_tokens': '7.684e+06', 'mean_token_accuracy': '0.7082', 'epoch': '0.01253'}
{'loss': '0.8353', 'grad_norm': '0.3835', 'learning_rate': '0.0001975', 'entropy': '0.7844', 'num_tokens': '7.69e+06', 'mean_token_accuracy': '0.81', 'epoch': '0.01254'}
{'loss': '0.7017', 'grad_norm': '0.3402', 'learning_rate': '0.0001975', 'entropy': '0.8021', 'num_tokens': '7.694e+06', 'mean_token_accuracy': '0.7975', 'epoch': '0.01255'}
{'loss': '0.9107', 'grad_norm': '0.39', 'learning_rate': '0.0001975', 'entropy': '0.8487', 'num_tokens': '7.7e+06', 'mean_token_accuracy': '0.7816', 'epoch': '0.01256'}
{'loss': '1.143', 'grad_norm': '0.4111', 'learning_rate': '0.0001975', 'entropy': '1.128', 'num_tokens': '7.706e+06', 'mean_token_accuracy': '0.7266', 'epoch': '0.01257'}
{'loss': '1.005', 'grad_norm': '0.4151', 'learning_rate': '0.0001975', 'entropy': '0.9664', 'num_tokens': '7.71e+06', 'mean_token_accuracy': '0.7598', 'epoch': '0.01258'}
{'loss': '1.37', 'grad_norm': '0.3698', 'learning_rate': '0.0001975', 'entropy': '1.348', 'num_tokens': '7.718e+06', 'mean_token_accuracy': '0.6865', 'epoch': '0.01258'}
{'loss': '1.103', 'grad_norm': '0.3469', 'learning_rate': '0.0001974', 'entropy': '1.094', 'num_tokens': '7.722e+06', 'mean_token_accuracy': '0.728', 'epoch': '0.01259'}
{'loss': '1.064', 'grad_norm': '0.3564', 'learning_rate': '0.0001974', 'entropy': '1.066', 'num_tokens': '7.727e+06', 'mean_token_accuracy': '0.7487', 'epoch': '0.0126'}
{'loss': '1.075', 'grad_norm': '0.2963', 'learning_rate': '0.0001974', 'entropy': '1.157', 'num_tokens': '7.736e+06', 'mean_token_accuracy': '0.7248', 'epoch': '0.01261'}
{'loss': '0.725', 'grad_norm': '0.2655', 'learning_rate': '0.0001974', 'entropy': '0.7158', 'num_tokens': '7.742e+06', 'mean_token_accuracy': '0.8109', 'epoch': '0.01262'}
{'loss': '0.9034', 'grad_norm': '0.412', 'learning_rate': '0.0001974', 'entropy': '0.9673', 'num_tokens': '7.746e+06', 'mean_token_accuracy': '0.7659', 'epoch': '0.01263'}
{'loss': '1.283', 'grad_norm': '0.3792', 'learning_rate': '0.0001974', 'entropy': '1.235', 'num_tokens': '7.751e+06', 'mean_token_accuracy': '0.7041', 'epoch': '0.01264'}
{'loss': '1.2', 'grad_norm': '0.3217', 'learning_rate': '0.0001974', 'entropy': '1.177', 'num_tokens': '7.758e+06', 'mean_token_accuracy': '0.7224', 'epoch': '0.01265'}
{'loss': '1.222', 'grad_norm': '0.4322', 'learning_rate': '0.0001974', 'entropy': '1.22', 'num_tokens': '7.764e+06', 'mean_token_accuracy': '0.7166', 'epoch': '0.01266'}
{'loss': '0.8315', 'grad_norm': '0.2781', 'learning_rate': '0.0001974', 'entropy': '0.8394', 'num_tokens': '7.771e+06', 'mean_token_accuracy': '0.8023', 'epoch': '0.01266'}
{'loss': '0.7769', 'grad_norm': '0.3864', 'learning_rate': '0.0001974', 'entropy': '0.8342', 'num_tokens': '7.775e+06', 'mean_token_accuracy': '0.8039', 'epoch': '0.01267'}
{'loss': '1.107', 'grad_norm': '0.3437', 'learning_rate': '0.0001974', 'entropy': '1.133', 'num_tokens': '7.781e+06', 'mean_token_accuracy': '0.7361', 'epoch': '0.01268'}
{'loss': '0.648', 'grad_norm': '0.3024', 'learning_rate': '0.0001974', 'entropy': '0.6445', 'num_tokens': '7.786e+06', 'mean_token_accuracy': '0.837', 'epoch': '0.01269'}
{'loss': '0.9554', 'grad_norm': '0.3908', 'learning_rate': '0.0001974', 'entropy': '1', 'num_tokens': '7.79e+06', 'mean_token_accuracy': '0.7586', 'epoch': '0.0127'}
{'loss': '1.228', 'grad_norm': '0.3947', 'learning_rate': '0.0001974', 'entropy': '1.154', 'num_tokens': '7.794e+06', 'mean_token_accuracy': '0.7275', 'epoch': '0.01271'}
{'loss': '0.9868', 'grad_norm': '0.3593', 'learning_rate': '0.0001974', 'entropy': '0.9434', 'num_tokens': '7.798e+06', 'mean_token_accuracy': '0.7667', 'epoch': '0.01272'}
{'loss': '0.8521', 'grad_norm': '0.414', 'learning_rate': '0.0001974', 'entropy': '0.8508', 'num_tokens': '7.802e+06', 'mean_token_accuracy': '0.7878', 'epoch': '0.01273'}
{'loss': '1.197', 'grad_norm': '0.444', 'learning_rate': '0.0001973', 'entropy': '1.146', 'num_tokens': '7.808e+06', 'mean_token_accuracy': '0.7311', 'epoch': '0.01273'}
{'loss': '0.5872', 'grad_norm': '0.2926', 'learning_rate': '0.0001973', 'entropy': '0.71', 'num_tokens': '7.814e+06', 'mean_token_accuracy': '0.8177', 'epoch': '0.01274'}
{'loss': '1.099', 'grad_norm': '0.3904', 'learning_rate': '0.0001973', 'entropy': '1.062', 'num_tokens': '7.82e+06', 'mean_token_accuracy': '0.7495', 'epoch': '0.01275'}
{'loss': '1.131', 'grad_norm': '0.3764', 'learning_rate': '0.0001973', 'entropy': '1.161', 'num_tokens': '7.825e+06', 'mean_token_accuracy': '0.7218', 'epoch': '0.01276'}
{'loss': '1.216', 'grad_norm': '0.5341', 'learning_rate': '0.0001973', 'entropy': '1.224', 'num_tokens': '7.829e+06', 'mean_token_accuracy': '0.7089', 'epoch': '0.01277'}
{'loss': '0.6972', 'grad_norm': '0.3216', 'learning_rate': '0.0001973', 'entropy': '0.7118', 'num_tokens': '7.834e+06', 'mean_token_accuracy': '0.8183', 'epoch': '0.01278'}
{'loss': '0.8109', 'grad_norm': '0.3675', 'learning_rate': '0.0001973', 'entropy': '0.7997', 'num_tokens': '7.839e+06', 'mean_token_accuracy': '0.8095', 'epoch': '0.01279'}
{'loss': '0.8909', 'grad_norm': '0.3938', 'learning_rate': '0.0001973', 'entropy': '0.8628', 'num_tokens': '7.845e+06', 'mean_token_accuracy': '0.782', 'epoch': '0.0128'}
{'loss': '0.8681', 'grad_norm': '0.4589', 'learning_rate': '0.0001973', 'entropy': '0.8839', 'num_tokens': '7.848e+06', 'mean_token_accuracy': '0.7721', 'epoch': '0.01281'}
{'loss': '0.9377', 'grad_norm': '0.3834', 'learning_rate': '0.0001973', 'entropy': '0.9039', 'num_tokens': '7.854e+06', 'mean_token_accuracy': '0.7577', 'epoch': '0.01281'}
{'loss': '1.019', 'grad_norm': '0.3557', 'learning_rate': '0.0001973', 'entropy': '0.9794', 'num_tokens': '7.859e+06', 'mean_token_accuracy': '0.7619', 'epoch': '0.01282'}
{'loss': '1.072', 'grad_norm': '0.3392', 'learning_rate': '0.0001973', 'entropy': '1.092', 'num_tokens': '7.863e+06', 'mean_token_accuracy': '0.7284', 'epoch': '0.01283'}
{'loss': '0.5864', 'grad_norm': '0.2784', 'learning_rate': '0.0001973', 'entropy': '0.6372', 'num_tokens': '7.868e+06', 'mean_token_accuracy': '0.838', 'epoch': '0.01284'}
{'loss': '0.5698', 'grad_norm': '0.3013', 'learning_rate': '0.0001973', 'entropy': '0.588', 'num_tokens': '7.873e+06', 'mean_token_accuracy': '0.8486', 'epoch': '0.01285'}
{'loss': '1.043', 'grad_norm': '0.3876', 'learning_rate': '0.0001973', 'entropy': '1.086', 'num_tokens': '7.879e+06', 'mean_token_accuracy': '0.7369', 'epoch': '0.01286'}
{'loss': '1.007', 'grad_norm': '0.3907', 'learning_rate': '0.0001972', 'entropy': '1.104', 'num_tokens': '7.882e+06', 'mean_token_accuracy': '0.7499', 'epoch': '0.01287'}
{'loss': '0.8225', 'grad_norm': '0.3241', 'learning_rate': '0.0001972', 'entropy': '0.797', 'num_tokens': '7.888e+06', 'mean_token_accuracy': '0.8141', 'epoch': '0.01288'}
{'loss': '0.7448', 'grad_norm': '0.3444', 'learning_rate': '0.0001972', 'entropy': '0.7653', 'num_tokens': '7.893e+06', 'mean_token_accuracy': '0.8191', 'epoch': '0.01289'}
{'loss': '0.7846', 'grad_norm': '0.3047', 'learning_rate': '0.0001972', 'entropy': '0.8472', 'num_tokens': '7.899e+06', 'mean_token_accuracy': '0.7863', 'epoch': '0.01289'}
{'loss': '1.033', 'grad_norm': '0.3407', 'learning_rate': '0.0001972', 'entropy': '0.9499', 'num_tokens': '7.905e+06', 'mean_token_accuracy': '0.7688', 'epoch': '0.0129'}
{'loss': '1.327', 'grad_norm': '0.3882', 'learning_rate': '0.0001972', 'entropy': '1.351', 'num_tokens': '7.91e+06', 'mean_token_accuracy': '0.6856', 'epoch': '0.01291'}
{'loss': '0.7244', 'grad_norm': '0.2991', 'learning_rate': '0.0001972', 'entropy': '0.6972', 'num_tokens': '7.917e+06', 'mean_token_accuracy': '0.8104', 'epoch': '0.01292'}

 12%|█▏        | 1461/12000 [52:36<5:59:12,  2.05s/it]
 12%|█▏        | 1462/12000 [52:41<8:33:11,  2.92s/it]
                                                      

 12%|█▏        | 1462/12000 [52:41<8:33:11,  2.92s/it]
 12%|█▏        | 1463/12000 [52:43<7:26:48,  2.54s/it]
                                                      

 12%|█▏        | 1463/12000 [52:43<7:26:48,  2.54s/it]
 12%|█▏        | 1464/12000 [52:45<7:17:51,  2.49s/it]
                                                      

 12%|█▏        | 1464/12000 [52:45<7:17:51,  2.49s/it]
 12%|█▏        | 1465/12000 [52:46<6:26:14,  2.20s/it]
                                                      

 12%|█▏        | 1465/12000 [52:46<6:26:14,  2.20s/it]
 12%|█▏        | 1466/12000 [52:48<6:17:01,  2.15s/it]
                                                      

 12%|█▏        | 1466/12000 [52:48<6:17:01,  2.15s/it]
 12%|█▏        | 1467/12000 [52:50<5:37:09,  1.92s/it]
                                                      

 12%|█▏        | 1467/12000 [52:50<5:37:09,  1.92s/it]
 12%|█▏        | 1468/12000 [52:52<5:26:27,  1.86s/it]
                                                      

 12%|█▏        | 1468/12000 [52:52<5:26:27,  1.86s/it]
 12%|█▏        | 1469/12000 [52:53<5:08:59,  1.76s/it]
                                                      

 12%|█▏        | 1469/12000 [52:53<5:08:59,  1.76s/it]
 12%|█▏        | 1470/12000 [52:55<5:20:02,  1.82s/it]
                                                      

 12%|█▏        | 1470/12000 [52:55<5:20:02,  1.82s/it]
 12%|█▏        | 1471/12000 [52:59<6:50:27,  2.34s/it]
                                                      

 12%|█▏        | 1471/12000 [52:59<6:50:27,  2.34s/it]
 12%|█▏        | 1472/12000 [53:00<6:04:26,  2.08s/it]
                                                      

 12%|█▏        | 1472/12000 [53:00<6:04:26,  2.08s/it]
 12%|█▏        | 1473/12000 [53:02<5:56:54,  2.03s/it]
                                                      

 12%|█▏        | 1473/12000 [53:02<5:56:54,  2.03s/it]
 12%|█▏        | 1474/12000 [53:04<5:45:55,  1.97s/it]
                                                      

 12%|█▏        | 1474/12000 [53:04<5:45:55,  1.97s/it]
 12%|█▏        | 1475/12000 [53:06<5:39:52,  1.94s/it]
                                                      

 12%|█▏        | 1475/12000 [53:06<5:39:52,  1.94s/it]
 12%|█▏        | 1476/12000 [53:07<5:19:10,  1.82s/it]
                                                      

 12%|█▏        | 1476/12000 [53:07<5:19:10,  1.82s/it]
 12%|█▏        | 1477/12000 [53:09<5:23:46,  1.85s/it]
                                                      

 12%|█▏        | 1477/12000 [53:09<5:23:46,  1.85s/it]
 12%|█▏        | 1478/12000 [53:11<5:38:10,  1.93s/it]
                                                      

 12%|█▏        | 1478/12000 [53:11<5:38:10,  1.93s/it]
 12%|█▏        | 1479/12000 [53:13<5:39:01,  1.93s/it]
                                                      

 12%|█▏        | 1479/12000 [53:13<5:39:01,  1.93s/it]
 12%|█▏        | 1480/12000 [53:19<9:01:08,  3.09s/it]
                                                      

 12%|█▏        | 1480/12000 [53:19<9:01:08,  3.09s/it]
 12%|█▏        | 1481/12000 [53:20<7:35:25,  2.60s/it]
                                                      

 12%|█▏        | 1481/12000 [53:20<7:35:25,  2.60s/it]
 12%|█▏        | 1482/12000 [53:22<6:52:13,  2.35s/it]
                                                      

 12%|█▏        | 1482/12000 [53:22<6:52:13,  2.35s/it]
 12%|█▏        | 1483/12000 [53:25<7:03:41,  2.42s/it]
                                                      

 12%|█▏        | 1483/12000 [53:25<7:03:41,  2.42s/it]
 12%|█▏        | 1484/12000 [53:26<6:17:34,  2.15s/it]
                                                      

 12%|█▏        | 1484/12000 [53:26<6:17:34,  2.15s/it]
 12%|█▏        | 1485/12000 [53:30<7:35:04,  2.60s/it]
                                                      

 12%|█▏        | 1485/12000 [53:30<7:35:04,  2.60s/it]
 12%|█▏        | 1486/12000 [53:33<7:40:08,  2.63s/it]
                                                      

 12%|█▏        | 1486/12000 [53:33<7:40:08,  2.63s/it]
 12%|█▏        | 1487/12000 [53:35<7:36:15,  2.60s/it]
                                                      

 12%|█▏        | 1487/12000 [53:35<7:36:15,  2.60s/it]
 12%|█▏        | 1488/12000 [53:37<6:45:11,  2.31s/it]
                                                      

 12%|█▏        | 1488/12000 [53:37<6:45:11,  2.31s/it]
 12%|█▏        | 1489/12000 [53:39<6:52:54,  2.36s/it]
                                                      

 12%|█▏        | 1489/12000 [53:39<6:52:54,  2.36s/it]
 12%|█▏        | 1490/12000 [53:42<7:29:32,  2.57s/it]
                                                      

 12%|█▏        | 1490/12000 [53:42<7:29:32,  2.57s/it]
 12%|█▏        | 1491/12000 [53:44<6:36:22,  2.26s/it]
                                                      

 12%|█▏        | 1491/12000 [53:44<6:36:22,  2.26s/it]
 12%|█▏        | 1492/12000 [53:45<5:59:41,  2.05s/it]
                                                      

 12%|█▏        | 1492/12000 [53:45<5:59:41,  2.05s/it]
 12%|█▏        | 1493/12000 [53:50<7:59:26,  2.74s/it]
                                                      

 12%|█▏        | 1493/12000 [53:50<7:59:26,  2.74s/it]
 12%|█▏        | 1494/12000 [53:53<8:02:59,  2.76s/it]
                                                      

 12%|█▏        | 1494/12000 [53:53<8:02:59,  2.76s/it]
 12%|█▏        | 1495/12000 [53:54<6:59:12,  2.39s/it]
                                                      

 12%|█▏        | 1495/12000 [53:54<6:59:12,  2.39s/it]
 12%|█▏        | 1496/12000 [53:56<6:09:49,  2.11s/it]
                                                      

 12%|█▏        | 1496/12000 [53:56<6:09:49,  2.11s/it]
 12%|█▏        | 1497/12000 [53:58<6:03:48,  2.08s/it]
                                                      

 12%|█▏        | 1497/12000 [53:58<6:03:48,  2.08s/it]
 12%|█▏        | 1498/12000 [53:59<5:47:04,  1.98s/it]
                                                      

 12%|█▏        | 1498/12000 [53:59<5:47:04,  1.98s/it]
 12%|█▏        | 1499/12000 [54:03<6:54:24,  2.37s/it]
                                                      

 12%|█▏        | 1499/12000 [54:03<6:54:24,  2.37s/it]
 12%|█▎        | 1500/12000 [54:04<6:19:29,  2.17s/it]
                                                      

 12%|█▎        | 1500/12000 [54:04<6:19:29,  2.17s/it]
 13%|█▎        | 1501/12000 [54:07<6:46:23,  2.32s/it]
                                                      

 13%|█▎        | 1501/12000 [54:07<6:46:23,  2.32s/it]
 13%|█▎        | 1502/12000 [54:09<6:24:01,  2.19s/it]
                                                      

 13%|█▎        | 1502/12000 [54:09<6:24:01,  2.19s/it]
 13%|█▎        | 1503/12000 [54:11<6:04:09,  2.08s/it]
                                                      

 13%|█▎        | 1503/12000 [54:11<6:04:09,  2.08s/it]
 13%|█▎        | 1504/12000 [54:12<5:33:18,  1.91s/it]
                                                      

 13%|█▎        | 1504/12000 [54:12<5:33:18,  1.91s/it]
 13%|█▎        | 1505/12000 [54:14<5:15:34,  1.80s/it]
                                                      

 13%|█▎        | 1505/12000 [54:14<5:15:34,  1.80s/it]
 13%|█▎        | 1506/12000 [54:17<6:49:51,  2.34s/it]
                                                      

 13%|█▎        | 1506/12000 [54:17<6:49:51,  2.34s/it]
 13%|█▎        | 1507/12000 [54:19<6:16:25,  2.15s/it]
                                                      

 13%|█▎        | 1507/12000 [54:19<6:16:25,  2.15s/it]
 13%|█▎        | 1508/12000 [54:21<6:07:52,  2.10s/it]
                                                      
{'loss': '0.9239', 'grad_norm': '0.3818', 'learning_rate': '0.0001972', 'entropy': '0.9659', 'num_tokens': '7.921e+06', 'mean_token_accuracy': '0.7525', 'epoch': '0.01293'}
{'loss': '0.6584', 'grad_norm': '0.2615', 'learning_rate': '0.0001972', 'entropy': '0.8565', 'num_tokens': '7.931e+06', 'mean_token_accuracy': '0.7805', 'epoch': '0.01294'}
{'loss': '0.8478', 'grad_norm': '0.3728', 'learning_rate': '0.0001972', 'entropy': '0.8142', 'num_tokens': '7.935e+06', 'mean_token_accuracy': '0.7868', 'epoch': '0.01295'}
{'loss': '1.146', 'grad_norm': '0.3105', 'learning_rate': '0.0001972', 'entropy': '1.082', 'num_tokens': '7.943e+06', 'mean_token_accuracy': '0.7302', 'epoch': '0.01296'}
{'loss': '0.8422', 'grad_norm': '0.3147', 'learning_rate': '0.0001972', 'entropy': '0.849', 'num_tokens': '7.947e+06', 'mean_token_accuracy': '0.7861', 'epoch': '0.01297'}
{'loss': '0.8656', 'grad_norm': '0.3315', 'learning_rate': '0.0001972', 'entropy': '0.8465', 'num_tokens': '7.953e+06', 'mean_token_accuracy': '0.7963', 'epoch': '0.01297'}
{'loss': '0.7525', 'grad_norm': '0.3415', 'learning_rate': '0.0001972', 'entropy': '0.7739', 'num_tokens': '7.957e+06', 'mean_token_accuracy': '0.803', 'epoch': '0.01298'}
{'loss': '0.8804', 'grad_norm': '0.4229', 'learning_rate': '0.0001972', 'entropy': '0.8549', 'num_tokens': '7.961e+06', 'mean_token_accuracy': '0.794', 'epoch': '0.01299'}
{'loss': '0.8345', 'grad_norm': '0.4865', 'learning_rate': '0.0001972', 'entropy': '0.8134', 'num_tokens': '7.964e+06', 'mean_token_accuracy': '0.7997', 'epoch': '0.013'}
{'loss': '1.159', 'grad_norm': '0.3626', 'learning_rate': '0.0001971', 'entropy': '1.116', 'num_tokens': '7.969e+06', 'mean_token_accuracy': '0.7362', 'epoch': '0.01301'}
{'loss': '0.826', 'grad_norm': '0.3229', 'learning_rate': '0.0001971', 'entropy': '0.9003', 'num_tokens': '7.974e+06', 'mean_token_accuracy': '0.7797', 'epoch': '0.01302'}
{'loss': '0.9672', 'grad_norm': '0.3357', 'learning_rate': '0.0001971', 'entropy': '1.122', 'num_tokens': '7.979e+06', 'mean_token_accuracy': '0.7445', 'epoch': '0.01303'}
{'loss': '1.006', 'grad_norm': '0.3662', 'learning_rate': '0.0001971', 'entropy': '1.084', 'num_tokens': '7.984e+06', 'mean_token_accuracy': '0.7306', 'epoch': '0.01304'}
{'loss': '0.7037', 'grad_norm': '0.3003', 'learning_rate': '0.0001971', 'entropy': '0.788', 'num_tokens': '7.99e+06', 'mean_token_accuracy': '0.8154', 'epoch': '0.01304'}
{'loss': '0.8755', 'grad_norm': '0.3069', 'learning_rate': '0.0001971', 'entropy': '1.026', 'num_tokens': '7.997e+06', 'mean_token_accuracy': '0.7431', 'epoch': '0.01305'}
{'loss': '1.065', 'grad_norm': '0.3716', 'learning_rate': '0.0001971', 'entropy': '1.102', 'num_tokens': '8.001e+06', 'mean_token_accuracy': '0.7329', 'epoch': '0.01306'}
{'loss': '0.8399', 'grad_norm': '0.3208', 'learning_rate': '0.0001971', 'entropy': '0.9323', 'num_tokens': '8.007e+06', 'mean_token_accuracy': '0.7738', 'epoch': '0.01307'}
{'loss': '0.9211', 'grad_norm': '0.4065', 'learning_rate': '0.0001971', 'entropy': '0.9158', 'num_tokens': '8.012e+06', 'mean_token_accuracy': '0.782', 'epoch': '0.01308'}
{'loss': '1.036', 'grad_norm': '0.3194', 'learning_rate': '0.0001971', 'entropy': '0.967', 'num_tokens': '8.017e+06', 'mean_token_accuracy': '0.7756', 'epoch': '0.01309'}
{'loss': '1.544', 'grad_norm': '0.376', 'learning_rate': '0.0001971', 'entropy': '1.205', 'num_tokens': '8.027e+06', 'mean_token_accuracy': '0.7318', 'epoch': '0.0131'}
{'loss': '0.9789', 'grad_norm': '0.3746', 'learning_rate': '0.0001971', 'entropy': '0.9598', 'num_tokens': '8.031e+06', 'mean_token_accuracy': '0.7525', 'epoch': '0.01311'}
{'loss': '1.07', 'grad_norm': '0.4202', 'learning_rate': '0.0001971', 'entropy': '1.054', 'num_tokens': '8.036e+06', 'mean_token_accuracy': '0.7419', 'epoch': '0.01312'}
{'loss': '0.7788', 'grad_norm': '0.2886', 'learning_rate': '0.0001971', 'entropy': '0.7586', 'num_tokens': '8.044e+06', 'mean_token_accuracy': '0.8028', 'epoch': '0.01312'}
{'loss': '0.9084', 'grad_norm': '0.373', 'learning_rate': '0.0001971', 'entropy': '0.9', 'num_tokens': '8.048e+06', 'mean_token_accuracy': '0.7781', 'epoch': '0.01313'}
{'loss': '0.8532', 'grad_norm': '0.3008', 'learning_rate': '0.000197', 'entropy': '0.9816', 'num_tokens': '8.057e+06', 'mean_token_accuracy': '0.7737', 'epoch': '0.01314'}
{'loss': '0.7361', 'grad_norm': '0.249', 'learning_rate': '0.000197', 'entropy': '0.6847', 'num_tokens': '8.064e+06', 'mean_token_accuracy': '0.8239', 'epoch': '0.01315'}
{'loss': '0.9139', 'grad_norm': '0.2988', 'learning_rate': '0.000197', 'entropy': '0.8897', 'num_tokens': '8.07e+06', 'mean_token_accuracy': '0.7919', 'epoch': '0.01316'}
{'loss': '0.9171', 'grad_norm': '0.3737', 'learning_rate': '0.000197', 'entropy': '0.8979', 'num_tokens': '8.074e+06', 'mean_token_accuracy': '0.7807', 'epoch': '0.01317'}
{'loss': '0.986', 'grad_norm': '0.4645', 'learning_rate': '0.000197', 'entropy': '1.197', 'num_tokens': '8.08e+06', 'mean_token_accuracy': '0.7236', 'epoch': '0.01318'}
{'loss': '1.089', 'grad_norm': '0.3189', 'learning_rate': '0.000197', 'entropy': '1.05', 'num_tokens': '8.087e+06', 'mean_token_accuracy': '0.7719', 'epoch': '0.01319'}
{'loss': '1.101', 'grad_norm': '0.3852', 'learning_rate': '0.000197', 'entropy': '1.129', 'num_tokens': '8.092e+06', 'mean_token_accuracy': '0.7279', 'epoch': '0.0132'}
{'loss': '0.8237', 'grad_norm': '0.3395', 'learning_rate': '0.000197', 'entropy': '0.8429', 'num_tokens': '8.095e+06', 'mean_token_accuracy': '0.795', 'epoch': '0.0132'}
{'loss': '0.9833', 'grad_norm': '0.3305', 'learning_rate': '0.000197', 'entropy': '0.9871', 'num_tokens': '8.102e+06', 'mean_token_accuracy': '0.7615', 'epoch': '0.01321'}
{'loss': '0.9121', 'grad_norm': '0.3067', 'learning_rate': '0.000197', 'entropy': '0.9531', 'num_tokens': '8.109e+06', 'mean_token_accuracy': '0.7753', 'epoch': '0.01322'}
{'loss': '0.8603', 'grad_norm': '0.3455', 'learning_rate': '0.000197', 'entropy': '0.8388', 'num_tokens': '8.114e+06', 'mean_token_accuracy': '0.7879', 'epoch': '0.01323'}
{'loss': '0.9885', 'grad_norm': '0.4048', 'learning_rate': '0.000197', 'entropy': '1.01', 'num_tokens': '8.117e+06', 'mean_token_accuracy': '0.7699', 'epoch': '0.01324'}
{'loss': '1.219', 'grad_norm': '0.4046', 'learning_rate': '0.000197', 'entropy': '1.171', 'num_tokens': '8.123e+06', 'mean_token_accuracy': '0.7222', 'epoch': '0.01325'}
{'loss': '0.6369', 'grad_norm': '0.3215', 'learning_rate': '0.000197', 'entropy': '0.6028', 'num_tokens': '8.128e+06', 'mean_token_accuracy': '0.8393', 'epoch': '0.01326'}
{'loss': '0.886', 'grad_norm': '0.3508', 'learning_rate': '0.000197', 'entropy': '0.8913', 'num_tokens': '8.136e+06', 'mean_token_accuracy': '0.7788', 'epoch': '0.01327'}
{'loss': '0.6985', 'grad_norm': '0.4038', 'learning_rate': '0.0001969', 'entropy': '0.6665', 'num_tokens': '8.141e+06', 'mean_token_accuracy': '0.819', 'epoch': '0.01327'}
{'loss': '0.9369', 'grad_norm': '0.3369', 'learning_rate': '0.0001969', 'entropy': '0.8916', 'num_tokens': '8.145e+06', 'mean_token_accuracy': '0.7754', 'epoch': '0.01328'}
{'loss': '0.8959', 'grad_norm': '0.3663', 'learning_rate': '0.0001969', 'entropy': '0.8922', 'num_tokens': '8.151e+06', 'mean_token_accuracy': '0.7853', 'epoch': '0.01329'}
{'loss': '1.13', 'grad_norm': '0.3937', 'learning_rate': '0.0001969', 'entropy': '1.098', 'num_tokens': '8.156e+06', 'mean_token_accuracy': '0.7466', 'epoch': '0.0133'}
{'loss': '0.7398', 'grad_norm': '0.3147', 'learning_rate': '0.0001969', 'entropy': '0.783', 'num_tokens': '8.16e+06', 'mean_token_accuracy': '0.8023', 'epoch': '0.01331'}
{'loss': '0.9223', 'grad_norm': '0.3603', 'learning_rate': '0.0001969', 'entropy': '0.8876', 'num_tokens': '8.165e+06', 'mean_token_accuracy': '0.7719', 'epoch': '0.01332'}
{'loss': '0.9759', 'grad_norm': '0.7914', 'learning_rate': '0.0001969', 'entropy': '0.8921', 'num_tokens': '8.173e+06', 'mean_token_accuracy': '0.7741', 'epoch': '0.01333'}
{'loss': '0.8689', 'grad_norm': '0.3794', 'learning_rate': '0.0001969', 'entropy': '0.9338', 'num_tokens': '8.178e+06', 'mean_token_accuracy': '0.7665', 'epoch': '0.01334'}

 13%|█▎        | 1508/12000 [54:21<6:07:52,  2.10s/it]
 13%|█▎        | 1509/12000 [54:22<5:26:12,  1.87s/it]
                                                      

 13%|█▎        | 1509/12000 [54:22<5:26:12,  1.87s/it]
 13%|█▎        | 1510/12000 [54:24<5:15:18,  1.80s/it]
                                                      

 13%|█▎        | 1510/12000 [54:24<5:15:18,  1.80s/it]
 13%|█▎        | 1511/12000 [54:26<4:55:56,  1.69s/it]
                                                      

 13%|█▎        | 1511/12000 [54:26<4:55:56,  1.69s/it]
 13%|█▎        | 1512/12000 [54:27<4:49:26,  1.66s/it]
                                                      

 13%|█▎        | 1512/12000 [54:27<4:49:26,  1.66s/it]
 13%|█▎        | 1513/12000 [54:29<4:38:29,  1.59s/it]
                                                      

 13%|█▎        | 1513/12000 [54:29<4:38:29,  1.59s/it]
 13%|█▎        | 1514/12000 [54:30<4:38:17,  1.59s/it]
                                                      

 13%|█▎        | 1514/12000 [54:30<4:38:17,  1.59s/it]
 13%|█▎        | 1515/12000 [54:34<6:49:29,  2.34s/it]
                                                      

 13%|█▎        | 1515/12000 [54:34<6:49:29,  2.34s/it]
 13%|█▎        | 1516/12000 [54:37<6:47:35,  2.33s/it]
                                                      

 13%|█▎        | 1516/12000 [54:37<6:47:35,  2.33s/it]
 13%|█▎        | 1517/12000 [54:38<6:18:04,  2.16s/it]
                                                      

 13%|█▎        | 1517/12000 [54:38<6:18:04,  2.16s/it]
 13%|█▎        | 1518/12000 [54:41<6:40:10,  2.29s/it]
                                                      

 13%|█▎        | 1518/12000 [54:41<6:40:10,  2.29s/it]
 13%|█▎        | 1519/12000 [54:45<8:27:11,  2.90s/it]
                                                      

 13%|█▎        | 1519/12000 [54:45<8:27:11,  2.90s/it]
 13%|█▎        | 1520/12000 [54:47<7:25:40,  2.55s/it]
                                                      

 13%|█▎        | 1520/12000 [54:47<7:25:40,  2.55s/it]
 13%|█▎        | 1521/12000 [54:49<7:01:13,  2.41s/it]
                                                      

 13%|█▎        | 1521/12000 [54:49<7:01:13,  2.41s/it]
 13%|█▎        | 1522/12000 [54:51<6:52:04,  2.36s/it]
                                                      

 13%|█▎        | 1522/12000 [54:51<6:52:04,  2.36s/it]
 13%|█▎        | 1523/12000 [54:53<6:10:52,  2.12s/it]
                                                      

 13%|█▎        | 1523/12000 [54:53<6:10:52,  2.12s/it]
 13%|█▎        | 1524/12000 [54:54<5:39:08,  1.94s/it]
                                                      

 13%|█▎        | 1524/12000 [54:54<5:39:08,  1.94s/it]
 13%|█▎        | 1525/12000 [54:56<5:23:14,  1.85s/it]
                                                      

 13%|█▎        | 1525/12000 [54:56<5:23:14,  1.85s/it]
 13%|█▎        | 1526/12000 [54:59<6:12:00,  2.13s/it]
                                                      

 13%|█▎        | 1526/12000 [54:59<6:12:00,  2.13s/it]
 13%|█▎        | 1527/12000 [55:01<6:11:11,  2.13s/it]
                                                      

 13%|█▎        | 1527/12000 [55:01<6:11:11,  2.13s/it]
 13%|█▎        | 1528/12000 [55:03<5:48:49,  2.00s/it]
                                                      

 13%|█▎        | 1528/12000 [55:03<5:48:49,  2.00s/it]
 13%|█▎        | 1529/12000 [55:06<6:46:27,  2.33s/it]
                                                      

 13%|█▎        | 1529/12000 [55:06<6:46:27,  2.33s/it]
 13%|█▎        | 1530/12000 [55:07<6:00:14,  2.06s/it]
                                                      

 13%|█▎        | 1530/12000 [55:07<6:00:14,  2.06s/it]
 13%|█▎        | 1531/12000 [55:09<5:37:53,  1.94s/it]
                                                      

 13%|█▎        | 1531/12000 [55:09<5:37:53,  1.94s/it]
 13%|█▎        | 1532/12000 [55:10<5:16:59,  1.82s/it]
                                                      

 13%|█▎        | 1532/12000 [55:10<5:16:59,  1.82s/it]
 13%|█▎        | 1533/12000 [55:12<5:11:30,  1.79s/it]
                                                      

 13%|█▎        | 1533/12000 [55:12<5:11:30,  1.79s/it]
 13%|█▎        | 1534/12000 [55:14<5:11:11,  1.78s/it]
                                                      

 13%|█▎        | 1534/12000 [55:14<5:11:11,  1.78s/it]
 13%|█▎        | 1535/12000 [55:15<4:52:40,  1.68s/it]
                                                      

 13%|█▎        | 1535/12000 [55:15<4:52:40,  1.68s/it]
 13%|█▎        | 1536/12000 [55:17<4:47:10,  1.65s/it]
                                                      

 13%|█▎        | 1536/12000 [55:17<4:47:10,  1.65s/it]
 13%|█▎        | 1537/12000 [55:20<6:19:28,  2.18s/it]
                                                      

 13%|█▎        | 1537/12000 [55:20<6:19:28,  2.18s/it]
 13%|█▎        | 1538/12000 [55:22<5:42:10,  1.96s/it]
                                                      

 13%|█▎        | 1538/12000 [55:22<5:42:10,  1.96s/it]
 13%|█▎        | 1539/12000 [55:24<5:34:42,  1.92s/it]
                                                      

 13%|█▎        | 1539/12000 [55:24<5:34:42,  1.92s/it]
 13%|█▎        | 1540/12000 [55:25<5:26:53,  1.88s/it]
                                                      

 13%|█▎        | 1540/12000 [55:25<5:26:53,  1.88s/it]
 13%|█▎        | 1541/12000 [55:27<5:06:20,  1.76s/it]
                                                      

 13%|█▎        | 1541/12000 [55:27<5:06:20,  1.76s/it]
 13%|█▎        | 1542/12000 [55:29<5:34:39,  1.92s/it]
                                                      

 13%|█▎        | 1542/12000 [55:29<5:34:39,  1.92s/it]
 13%|█▎        | 1543/12000 [55:31<5:21:38,  1.85s/it]
                                                      

 13%|█▎        | 1543/12000 [55:31<5:21:38,  1.85s/it]
 13%|█▎        | 1544/12000 [55:32<5:11:38,  1.79s/it]
                                                      

 13%|█▎        | 1544/12000 [55:32<5:11:38,  1.79s/it]
 13%|█▎        | 1545/12000 [55:34<5:06:03,  1.76s/it]
                                                      

 13%|█▎        | 1545/12000 [55:34<5:06:03,  1.76s/it]
 13%|█▎        | 1546/12000 [55:36<5:28:52,  1.89s/it]
                                                      

 13%|█▎        | 1546/12000 [55:36<5:28:52,  1.89s/it]
 13%|█▎        | 1547/12000 [55:39<6:16:56,  2.16s/it]
                                                      

 13%|█▎        | 1547/12000 [55:39<6:16:56,  2.16s/it]
 13%|█▎        | 1548/12000 [55:41<5:38:58,  1.95s/it]
                                                      

 13%|█▎        | 1548/12000 [55:41<5:38:58,  1.95s/it]
 13%|█▎        | 1549/12000 [55:42<5:18:52,  1.83s/it]
                                                      

 13%|█▎        | 1549/12000 [55:42<5:18:52,  1.83s/it]
 13%|█▎        | 1550/12000 [55:44<5:16:26,  1.82s/it]
                                                      

 13%|█▎        | 1550/12000 [55:44<5:16:26,  1.82s/it]
 13%|█▎        | 1551/12000 [55:47<6:16:04,  2.16s/it]
                                                      

 13%|█▎        | 1551/12000 [55:47<6:16:04,  2.16s/it]
 13%|█▎        | 1552/12000 [55:48<5:44:25,  1.98s/it]
                                                      

 13%|█▎        | 1552/12000 [55:48<5:44:25,  1.98s/it]
 13%|█▎        | 1553/12000 [55:52<6:46:16,  2.33s/it]
                                                      

 13%|█▎        | 1553/12000 [55:52<6:46:16,  2.33s/it]
 13%|█▎        | 1554/12000 [55:53<6:04:57,  2.10s/it]
                                                      

 13%|█▎        | 1554/12000 [55:53<6:04:57,  2.10s/it]
 13%|█▎        | 1555/12000 [55:56<6:31:14,  2.25s/it]
                                                      
{'loss': '0.8096', 'grad_norm': '0.379', 'learning_rate': '0.0001969', 'entropy': '0.8268', 'num_tokens': '8.183e+06', 'mean_token_accuracy': '0.7768', 'epoch': '0.01335'}
{'loss': '0.7976', 'grad_norm': '0.3491', 'learning_rate': '0.0001969', 'entropy': '0.8517', 'num_tokens': '8.187e+06', 'mean_token_accuracy': '0.7845', 'epoch': '0.01335'}
{'loss': '0.8921', 'grad_norm': '0.3428', 'learning_rate': '0.0001969', 'entropy': '0.9743', 'num_tokens': '8.192e+06', 'mean_token_accuracy': '0.7616', 'epoch': '0.01336'}
{'loss': '0.9125', 'grad_norm': '0.4734', 'learning_rate': '0.0001969', 'entropy': '0.9182', 'num_tokens': '8.196e+06', 'mean_token_accuracy': '0.7776', 'epoch': '0.01337'}
{'loss': '0.8823', 'grad_norm': '0.3992', 'learning_rate': '0.0001969', 'entropy': '0.8243', 'num_tokens': '8.201e+06', 'mean_token_accuracy': '0.7941', 'epoch': '0.01338'}
{'loss': '1.08', 'grad_norm': '0.6178', 'learning_rate': '0.0001969', 'entropy': '1.067', 'num_tokens': '8.204e+06', 'mean_token_accuracy': '0.7324', 'epoch': '0.01339'}
{'loss': '0.9881', 'grad_norm': '0.4896', 'learning_rate': '0.0001969', 'entropy': '0.9465', 'num_tokens': '8.208e+06', 'mean_token_accuracy': '0.7641', 'epoch': '0.0134'}
{'loss': '0.6645', 'grad_norm': '0.3714', 'learning_rate': '0.0001968', 'entropy': '0.7059', 'num_tokens': '8.214e+06', 'mean_token_accuracy': '0.8167', 'epoch': '0.01341'}
{'loss': '1.076', 'grad_norm': '0.443', 'learning_rate': '0.0001968', 'entropy': '1.183', 'num_tokens': '8.219e+06', 'mean_token_accuracy': '0.7227', 'epoch': '0.01342'}
{'loss': '0.7634', 'grad_norm': '0.4005', 'learning_rate': '0.0001968', 'entropy': '0.7173', 'num_tokens': '8.225e+06', 'mean_token_accuracy': '0.8201', 'epoch': '0.01343'}
{'loss': '0.9828', 'grad_norm': '0.387', 'learning_rate': '0.0001968', 'entropy': '0.9571', 'num_tokens': '8.232e+06', 'mean_token_accuracy': '0.7665', 'epoch': '0.01343'}
{'loss': '0.9716', 'grad_norm': '0.3084', 'learning_rate': '0.0001968', 'entropy': '0.8537', 'num_tokens': '8.244e+06', 'mean_token_accuracy': '0.7867', 'epoch': '0.01344'}
{'loss': '0.7425', 'grad_norm': '0.3344', 'learning_rate': '0.0001968', 'entropy': '0.7346', 'num_tokens': '8.249e+06', 'mean_token_accuracy': '0.8183', 'epoch': '0.01345'}
{'loss': '1.155', 'grad_norm': '0.3845', 'learning_rate': '0.0001968', 'entropy': '1.229', 'num_tokens': '8.253e+06', 'mean_token_accuracy': '0.7123', 'epoch': '0.01346'}
{'loss': '1.017', 'grad_norm': '0.3337', 'learning_rate': '0.0001968', 'entropy': '1.052', 'num_tokens': '8.26e+06', 'mean_token_accuracy': '0.7512', 'epoch': '0.01347'}
{'loss': '0.8423', 'grad_norm': '0.4088', 'learning_rate': '0.0001968', 'entropy': '0.9065', 'num_tokens': '8.263e+06', 'mean_token_accuracy': '0.7829', 'epoch': '0.01348'}
{'loss': '0.9204', 'grad_norm': '0.3409', 'learning_rate': '0.0001968', 'entropy': '0.9338', 'num_tokens': '8.268e+06', 'mean_token_accuracy': '0.7612', 'epoch': '0.01349'}
{'loss': '0.6978', 'grad_norm': '0.3608', 'learning_rate': '0.0001968', 'entropy': '0.731', 'num_tokens': '8.272e+06', 'mean_token_accuracy': '0.8164', 'epoch': '0.0135'}
{'loss': '0.8923', 'grad_norm': '0.3107', 'learning_rate': '0.0001968', 'entropy': '1.045', 'num_tokens': '8.279e+06', 'mean_token_accuracy': '0.7551', 'epoch': '0.0135'}
{'loss': '1.033', 'grad_norm': '0.381', 'learning_rate': '0.0001968', 'entropy': '0.9995', 'num_tokens': '8.284e+06', 'mean_token_accuracy': '0.763', 'epoch': '0.01351'}
{'loss': '1.005', 'grad_norm': '0.3587', 'learning_rate': '0.0001968', 'entropy': '1.015', 'num_tokens': '8.289e+06', 'mean_token_accuracy': '0.7609', 'epoch': '0.01352'}
{'loss': '0.9492', 'grad_norm': '0.3822', 'learning_rate': '0.0001967', 'entropy': '0.9928', 'num_tokens': '8.297e+06', 'mean_token_accuracy': '0.7439', 'epoch': '0.01353'}
{'loss': '0.8682', 'grad_norm': '0.3123', 'learning_rate': '0.0001967', 'entropy': '0.9064', 'num_tokens': '8.301e+06', 'mean_token_accuracy': '0.7743', 'epoch': '0.01354'}
{'loss': '0.9641', 'grad_norm': '0.3266', 'learning_rate': '0.0001967', 'entropy': '0.9589', 'num_tokens': '8.306e+06', 'mean_token_accuracy': '0.7609', 'epoch': '0.01355'}
{'loss': '1.155', 'grad_norm': '0.3328', 'learning_rate': '0.0001967', 'entropy': '1.102', 'num_tokens': '8.311e+06', 'mean_token_accuracy': '0.7238', 'epoch': '0.01356'}
{'loss': '0.7123', 'grad_norm': '0.3887', 'learning_rate': '0.0001967', 'entropy': '0.7368', 'num_tokens': '8.315e+06', 'mean_token_accuracy': '0.8103', 'epoch': '0.01357'}
{'loss': '0.9451', 'grad_norm': '0.3151', 'learning_rate': '0.0001967', 'entropy': '0.9173', 'num_tokens': '8.321e+06', 'mean_token_accuracy': '0.7712', 'epoch': '0.01358'}
{'loss': '1.028', 'grad_norm': '0.4156', 'learning_rate': '0.0001967', 'entropy': '0.9869', 'num_tokens': '8.325e+06', 'mean_token_accuracy': '0.7533', 'epoch': '0.01358'}
{'loss': '0.8187', 'grad_norm': '0.3808', 'learning_rate': '0.0001967', 'entropy': '0.8658', 'num_tokens': '8.329e+06', 'mean_token_accuracy': '0.7848', 'epoch': '0.01359'}
{'loss': '0.9242', 'grad_norm': '0.4603', 'learning_rate': '0.0001967', 'entropy': '0.9264', 'num_tokens': '8.333e+06', 'mean_token_accuracy': '0.7752', 'epoch': '0.0136'}
{'loss': '1', 'grad_norm': '0.4348', 'learning_rate': '0.0001967', 'entropy': '0.9636', 'num_tokens': '8.337e+06', 'mean_token_accuracy': '0.7597', 'epoch': '0.01361'}
{'loss': '1.278', 'grad_norm': '0.4425', 'learning_rate': '0.0001967', 'entropy': '1.277', 'num_tokens': '8.342e+06', 'mean_token_accuracy': '0.6949', 'epoch': '0.01362'}
{'loss': '0.8875', 'grad_norm': '0.421', 'learning_rate': '0.0001967', 'entropy': '0.8647', 'num_tokens': '8.347e+06', 'mean_token_accuracy': '0.7758', 'epoch': '0.01363'}
{'loss': '1.046', 'grad_norm': '0.3998', 'learning_rate': '0.0001967', 'entropy': '1.043', 'num_tokens': '8.35e+06', 'mean_token_accuracy': '0.7458', 'epoch': '0.01364'}
{'loss': '0.9061', 'grad_norm': '0.303', 'learning_rate': '0.0001967', 'entropy': '0.9467', 'num_tokens': '8.356e+06', 'mean_token_accuracy': '0.7734', 'epoch': '0.01365'}
{'loss': '0.7681', 'grad_norm': '0.3078', 'learning_rate': '0.0001966', 'entropy': '0.7782', 'num_tokens': '8.362e+06', 'mean_token_accuracy': '0.7974', 'epoch': '0.01366'}
{'loss': '0.6473', 'grad_norm': '0.3817', 'learning_rate': '0.0001966', 'entropy': '0.6644', 'num_tokens': '8.366e+06', 'mean_token_accuracy': '0.8289', 'epoch': '0.01366'}
{'loss': '1.403', 'grad_norm': '0.5741', 'learning_rate': '0.0001966', 'entropy': '1.383', 'num_tokens': '8.372e+06', 'mean_token_accuracy': '0.6829', 'epoch': '0.01367'}
{'loss': '1.207', 'grad_norm': '0.4412', 'learning_rate': '0.0001966', 'entropy': '1.056', 'num_tokens': '8.377e+06', 'mean_token_accuracy': '0.7397', 'epoch': '0.01368'}
{'loss': '0.9023', 'grad_norm': '0.3573', 'learning_rate': '0.0001966', 'entropy': '0.876', 'num_tokens': '8.385e+06', 'mean_token_accuracy': '0.7754', 'epoch': '0.01369'}
{'loss': '0.7415', 'grad_norm': '0.3689', 'learning_rate': '0.0001966', 'entropy': '0.8549', 'num_tokens': '8.389e+06', 'mean_token_accuracy': '0.7939', 'epoch': '0.0137'}
{'loss': '0.9597', 'grad_norm': '0.3126', 'learning_rate': '0.0001966', 'entropy': '0.9934', 'num_tokens': '8.394e+06', 'mean_token_accuracy': '0.7694', 'epoch': '0.01371'}
{'loss': '1.054', 'grad_norm': '0.3901', 'learning_rate': '0.0001966', 'entropy': '1.084', 'num_tokens': '8.399e+06', 'mean_token_accuracy': '0.734', 'epoch': '0.01372'}
{'loss': '0.9867', 'grad_norm': '0.3971', 'learning_rate': '0.0001966', 'entropy': '1.019', 'num_tokens': '8.404e+06', 'mean_token_accuracy': '0.763', 'epoch': '0.01373'}
{'loss': '0.8405', 'grad_norm': '0.3019', 'learning_rate': '0.0001966', 'entropy': '0.8499', 'num_tokens': '8.409e+06', 'mean_token_accuracy': '0.7852', 'epoch': '0.01373'}
{'loss': '0.8258', 'grad_norm': '0.3132', 'learning_rate': '0.0001966', 'entropy': '0.9229', 'num_tokens': '8.415e+06', 'mean_token_accuracy': '0.78', 'epoch': '0.01374'}
{'loss': '0.7901', 'grad_norm': '0.3338', 'learning_rate': '0.0001966', 'entropy': '0.8037', 'num_tokens': '8.42e+06', 'mean_token_accuracy': '0.7978', 'epoch': '0.01375'}

 13%|█▎        | 1555/12000 [55:56<6:31:14,  2.25s/it]
 13%|█▎        | 1556/12000 [55:57<5:55:39,  2.04s/it]
                                                      

 13%|█▎        | 1556/12000 [55:57<5:55:39,  2.04s/it]
 13%|█▎        | 1557/12000 [55:59<5:36:59,  1.94s/it]
                                                      

 13%|█▎        | 1557/12000 [55:59<5:36:59,  1.94s/it]
 13%|█▎        | 1558/12000 [56:00<5:14:15,  1.81s/it]
                                                      

 13%|█▎        | 1558/12000 [56:00<5:14:15,  1.81s/it]
 13%|█▎        | 1559/12000 [56:06<8:19:09,  2.87s/it]
                                                      

 13%|█▎        | 1559/12000 [56:06<8:19:09,  2.87s/it]
 13%|█▎        | 1560/12000 [56:07<7:04:53,  2.44s/it]
                                                      

 13%|█▎        | 1560/12000 [56:07<7:04:53,  2.44s/it]
 13%|█▎        | 1561/12000 [56:10<7:10:25,  2.47s/it]
                                                      

 13%|█▎        | 1561/12000 [56:10<7:10:25,  2.47s/it]
 13%|█▎        | 1562/12000 [56:13<7:57:54,  2.75s/it]
                                                      

 13%|█▎        | 1562/12000 [56:13<7:57:54,  2.75s/it]
 13%|█▎        | 1563/12000 [56:16<7:53:12,  2.72s/it]
                                                      

 13%|█▎        | 1563/12000 [56:16<7:53:12,  2.72s/it]
 13%|█▎        | 1564/12000 [56:18<7:21:11,  2.54s/it]
                                                      

 13%|█▎        | 1564/12000 [56:18<7:21:11,  2.54s/it]
 13%|█▎        | 1565/12000 [56:19<6:24:33,  2.21s/it]
                                                      

 13%|█▎        | 1565/12000 [56:19<6:24:33,  2.21s/it]
 13%|█▎        | 1566/12000 [56:22<6:52:04,  2.37s/it]
                                                      

 13%|█▎        | 1566/12000 [56:22<6:52:04,  2.37s/it]
 13%|█▎        | 1567/12000 [56:24<6:50:44,  2.36s/it]
                                                      

 13%|█▎        | 1567/12000 [56:24<6:50:44,  2.36s/it]
 13%|█▎        | 1568/12000 [56:26<6:03:35,  2.09s/it]
                                                      

 13%|█▎        | 1568/12000 [56:26<6:03:35,  2.09s/it]
 13%|█▎        | 1569/12000 [56:27<5:30:34,  1.90s/it]
                                                      

 13%|█▎        | 1569/12000 [56:27<5:30:34,  1.90s/it]
 13%|█▎        | 1570/12000 [56:29<5:17:46,  1.83s/it]
                                                      

 13%|█▎        | 1570/12000 [56:29<5:17:46,  1.83s/it]
 13%|█▎        | 1571/12000 [56:31<5:07:53,  1.77s/it]
                                                      

 13%|█▎        | 1571/12000 [56:31<5:07:53,  1.77s/it]
 13%|█▎        | 1572/12000 [56:32<5:03:04,  1.74s/it]
                                                      

 13%|█▎        | 1572/12000 [56:32<5:03:04,  1.74s/it]
 13%|█▎        | 1573/12000 [56:34<4:57:19,  1.71s/it]
                                                      

 13%|█▎        | 1573/12000 [56:34<4:57:19,  1.71s/it]
 13%|█▎        | 1574/12000 [56:36<5:14:31,  1.81s/it]
                                                      

 13%|█▎        | 1574/12000 [56:36<5:14:31,  1.81s/it]
 13%|█▎        | 1575/12000 [56:38<5:43:53,  1.98s/it]
                                                      

 13%|█▎        | 1575/12000 [56:38<5:43:53,  1.98s/it]
 13%|█▎        | 1576/12000 [56:41<6:12:45,  2.15s/it]
                                                      

 13%|█▎        | 1576/12000 [56:41<6:12:45,  2.15s/it]
 13%|█▎        | 1577/12000 [56:46<8:46:41,  3.03s/it]
                                                      

 13%|█▎        | 1577/12000 [56:46<8:46:41,  3.03s/it]
 13%|█▎        | 1578/12000 [56:49<8:31:56,  2.95s/it]
                                                      

 13%|█▎        | 1578/12000 [56:49<8:31:56,  2.95s/it]
 13%|█▎        | 1579/12000 [56:50<7:17:38,  2.52s/it]
                                                      

 13%|█▎        | 1579/12000 [56:50<7:17:38,  2.52s/it]
 13%|█▎        | 1580/12000 [56:52<6:27:52,  2.23s/it]
                                                      

 13%|█▎        | 1580/12000 [56:52<6:27:52,  2.23s/it]
 13%|█▎        | 1581/12000 [56:56<8:26:53,  2.92s/it]
                                                      

 13%|█▎        | 1581/12000 [56:56<8:26:53,  2.92s/it]
 13%|█▎        | 1582/12000 [56:58<7:14:01,  2.50s/it]
                                                      

 13%|█▎        | 1582/12000 [56:58<7:14:01,  2.50s/it]
 13%|█▎        | 1583/12000 [57:00<6:43:32,  2.32s/it]
                                                      

 13%|█▎        | 1583/12000 [57:00<6:43:32,  2.32s/it]
 13%|█▎        | 1584/12000 [57:01<5:58:50,  2.07s/it]
                                                      

 13%|█▎        | 1584/12000 [57:01<5:58:50,  2.07s/it]
 13%|█▎        | 1585/12000 [57:03<5:31:19,  1.91s/it]
                                                      

 13%|█▎        | 1585/12000 [57:03<5:31:19,  1.91s/it]
 13%|█▎        | 1586/12000 [57:05<5:39:18,  1.95s/it]
                                                      

 13%|█▎        | 1586/12000 [57:05<5:39:18,  1.95s/it]
 13%|█▎        | 1587/12000 [57:07<5:31:35,  1.91s/it]
                                                      

 13%|█▎        | 1587/12000 [57:07<5:31:35,  1.91s/it]
 13%|█▎        | 1588/12000 [57:08<5:20:50,  1.85s/it]
                                                      

 13%|█▎        | 1588/12000 [57:08<5:20:50,  1.85s/it]
 13%|█▎        | 1589/12000 [57:11<5:34:37,  1.93s/it]
                                                      

 13%|█▎        | 1589/12000 [57:11<5:34:37,  1.93s/it]
 13%|█▎        | 1590/12000 [57:13<6:03:42,  2.10s/it]
                                                      

 13%|█▎        | 1590/12000 [57:13<6:03:42,  2.10s/it]
 13%|█▎        | 1591/12000 [57:15<6:13:53,  2.16s/it]
                                                      

 13%|█▎        | 1591/12000 [57:15<6:13:53,  2.16s/it]
 13%|█▎        | 1592/12000 [57:17<6:00:57,  2.08s/it]
                                                      

 13%|█▎        | 1592/12000 [57:17<6:00:57,  2.08s/it]
 13%|█▎        | 1593/12000 [57:19<5:23:45,  1.87s/it]
                                                      

 13%|█▎        | 1593/12000 [57:19<5:23:45,  1.87s/it]
 13%|█▎        | 1594/12000 [57:21<5:40:33,  1.96s/it]
                                                      

 13%|█▎        | 1594/12000 [57:21<5:40:33,  1.96s/it]
 13%|█▎        | 1595/12000 [57:22<5:13:07,  1.81s/it]
                                                      

 13%|█▎        | 1595/12000 [57:22<5:13:07,  1.81s/it]
 13%|█▎        | 1596/12000 [57:24<4:53:45,  1.69s/it]
                                                      

 13%|█▎        | 1596/12000 [57:24<4:53:45,  1.69s/it]
 13%|█▎        | 1597/12000 [57:28<7:12:36,  2.50s/it]
                                                      

 13%|█▎        | 1597/12000 [57:28<7:12:36,  2.50s/it]
 13%|█▎        | 1598/12000 [57:30<6:37:53,  2.30s/it]
                                                      

 13%|█▎        | 1598/12000 [57:30<6:37:53,  2.30s/it]
 13%|█▎        | 1599/12000 [57:31<5:54:16,  2.04s/it]
                                                      

 13%|█▎        | 1599/12000 [57:31<5:54:16,  2.04s/it]
 13%|█▎        | 1600/12000 [57:34<6:09:07,  2.13s/it]
                                                      

 13%|█▎        | 1600/12000 [57:34<6:09:07,  2.13s/it]
 13%|█▎        | 1601/12000 [57:38<7:40:45,  2.66s/it]
                                                      

 13%|█▎        | 1601/12000 [57:38<7:40:45,  2.66s/it]
 13%|█▎        | 1602/12000 [57:39<6:34:46,  2.28s/it]
                                                      
{'loss': '1.019', 'grad_norm': '0.3626', 'learning_rate': '0.0001966', 'entropy': '0.9306', 'num_tokens': '8.427e+06', 'mean_token_accuracy': '0.7634', 'epoch': '0.01376'}
{'loss': '0.9173', 'grad_norm': '0.3954', 'learning_rate': '0.0001966', 'entropy': '0.9676', 'num_tokens': '8.431e+06', 'mean_token_accuracy': '0.7509', 'epoch': '0.01377'}
{'loss': '0.7083', 'grad_norm': '0.3234', 'learning_rate': '0.0001965', 'entropy': '0.6842', 'num_tokens': '8.436e+06', 'mean_token_accuracy': '0.8174', 'epoch': '0.01378'}
{'loss': '0.6918', 'grad_norm': '0.3641', 'learning_rate': '0.0001965', 'entropy': '0.6549', 'num_tokens': '8.441e+06', 'mean_token_accuracy': '0.8213', 'epoch': '0.01379'}
{'loss': '0.8278', 'grad_norm': '0.3273', 'learning_rate': '0.0001965', 'entropy': '0.905', 'num_tokens': '8.447e+06', 'mean_token_accuracy': '0.7726', 'epoch': '0.0138'}
{'loss': '0.9259', 'grad_norm': '0.3884', 'learning_rate': '0.0001965', 'entropy': '0.9569', 'num_tokens': '8.451e+06', 'mean_token_accuracy': '0.7484', 'epoch': '0.01381'}
{'loss': '1.119', 'grad_norm': '0.4682', 'learning_rate': '0.0001965', 'entropy': '1.046', 'num_tokens': '8.457e+06', 'mean_token_accuracy': '0.7281', 'epoch': '0.01381'}
{'loss': '0.9417', 'grad_norm': '0.33', 'learning_rate': '0.0001965', 'entropy': '0.9139', 'num_tokens': '8.464e+06', 'mean_token_accuracy': '0.7634', 'epoch': '0.01382'}
{'loss': '0.9194', 'grad_norm': '0.3359', 'learning_rate': '0.0001965', 'entropy': '0.9781', 'num_tokens': '8.47e+06', 'mean_token_accuracy': '0.7512', 'epoch': '0.01383'}
{'loss': '1.053', 'grad_norm': '0.3222', 'learning_rate': '0.0001965', 'entropy': '0.9885', 'num_tokens': '8.477e+06', 'mean_token_accuracy': '0.756', 'epoch': '0.01384'}
{'loss': '0.9946', 'grad_norm': '0.3033', 'learning_rate': '0.0001965', 'entropy': '0.9887', 'num_tokens': '8.483e+06', 'mean_token_accuracy': '0.7618', 'epoch': '0.01385'}
{'loss': '0.6237', 'grad_norm': '0.3634', 'learning_rate': '0.0001965', 'entropy': '0.6751', 'num_tokens': '8.489e+06', 'mean_token_accuracy': '0.8259', 'epoch': '0.01386'}
{'loss': '0.9223', 'grad_norm': '0.3004', 'learning_rate': '0.0001965', 'entropy': '0.9895', 'num_tokens': '8.494e+06', 'mean_token_accuracy': '0.7723', 'epoch': '0.01387'}
{'loss': '0.8345', 'grad_norm': '0.3084', 'learning_rate': '0.0001965', 'entropy': '0.8862', 'num_tokens': '8.499e+06', 'mean_token_accuracy': '0.7921', 'epoch': '0.01388'}
{'loss': '0.9097', 'grad_norm': '0.3939', 'learning_rate': '0.0001965', 'entropy': '0.948', 'num_tokens': '8.503e+06', 'mean_token_accuracy': '0.7833', 'epoch': '0.01389'}
{'loss': '0.8848', 'grad_norm': '0.3342', 'learning_rate': '0.0001965', 'entropy': '0.983', 'num_tokens': '8.508e+06', 'mean_token_accuracy': '0.775', 'epoch': '0.01389'}
{'loss': '0.8939', 'grad_norm': '0.3784', 'learning_rate': '0.0001964', 'entropy': '0.8944', 'num_tokens': '8.512e+06', 'mean_token_accuracy': '0.7847', 'epoch': '0.0139'}
{'loss': '1.007', 'grad_norm': '0.4574', 'learning_rate': '0.0001964', 'entropy': '1.034', 'num_tokens': '8.517e+06', 'mean_token_accuracy': '0.7544', 'epoch': '0.01391'}
{'loss': '1.088', 'grad_norm': '0.5035', 'learning_rate': '0.0001964', 'entropy': '1.092', 'num_tokens': '8.521e+06', 'mean_token_accuracy': '0.7117', 'epoch': '0.01392'}
{'loss': '0.85', 'grad_norm': '0.3231', 'learning_rate': '0.0001964', 'entropy': '0.9465', 'num_tokens': '8.527e+06', 'mean_token_accuracy': '0.7761', 'epoch': '0.01393'}
{'loss': '1.125', 'grad_norm': '0.3188', 'learning_rate': '0.0001964', 'entropy': '1.043', 'num_tokens': '8.533e+06', 'mean_token_accuracy': '0.7379', 'epoch': '0.01394'}
{'loss': '0.8267', 'grad_norm': '0.3734', 'learning_rate': '0.0001964', 'entropy': '0.7914', 'num_tokens': '8.54e+06', 'mean_token_accuracy': '0.8035', 'epoch': '0.01395'}
{'loss': '1.033', 'grad_norm': '0.3085', 'learning_rate': '0.0001964', 'entropy': '0.956', 'num_tokens': '8.549e+06', 'mean_token_accuracy': '0.7634', 'epoch': '0.01396'}
{'loss': '0.6509', 'grad_norm': '0.3259', 'learning_rate': '0.0001964', 'entropy': '0.6337', 'num_tokens': '8.556e+06', 'mean_token_accuracy': '0.8297', 'epoch': '0.01397'}
{'loss': '1.084', 'grad_norm': '0.4095', 'learning_rate': '0.0001964', 'entropy': '1.068', 'num_tokens': '8.56e+06', 'mean_token_accuracy': '0.7437', 'epoch': '0.01397'}
{'loss': '0.8309', 'grad_norm': '0.3353', 'learning_rate': '0.0001964', 'entropy': '0.7976', 'num_tokens': '8.565e+06', 'mean_token_accuracy': '0.7985', 'epoch': '0.01398'}
{'loss': '0.7843', 'grad_norm': '0.3135', 'learning_rate': '0.0001964', 'entropy': '0.7458', 'num_tokens': '8.572e+06', 'mean_token_accuracy': '0.8114', 'epoch': '0.01399'}
{'loss': '0.8241', 'grad_norm': '0.4745', 'learning_rate': '0.0001964', 'entropy': '0.8979', 'num_tokens': '8.575e+06', 'mean_token_accuracy': '0.7796', 'epoch': '0.014'}
{'loss': '1.107', 'grad_norm': '0.3492', 'learning_rate': '0.0001964', 'entropy': '1.102', 'num_tokens': '8.581e+06', 'mean_token_accuracy': '0.7461', 'epoch': '0.01401'}
{'loss': '0.6222', 'grad_norm': '0.2808', 'learning_rate': '0.0001964', 'entropy': '0.6501', 'num_tokens': '8.586e+06', 'mean_token_accuracy': '0.8392', 'epoch': '0.01402'}
{'loss': '0.755', 'grad_norm': '0.335', 'learning_rate': '0.0001963', 'entropy': '0.8296', 'num_tokens': '8.591e+06', 'mean_token_accuracy': '0.7875', 'epoch': '0.01403'}
{'loss': '0.7712', 'grad_norm': '0.4292', 'learning_rate': '0.0001963', 'entropy': '0.7127', 'num_tokens': '8.597e+06', 'mean_token_accuracy': '0.8187', 'epoch': '0.01404'}
{'loss': '0.8712', 'grad_norm': '0.3705', 'learning_rate': '0.0001963', 'entropy': '0.8851', 'num_tokens': '8.602e+06', 'mean_token_accuracy': '0.7822', 'epoch': '0.01404'}
{'loss': '1.105', 'grad_norm': '0.35', 'learning_rate': '0.0001963', 'entropy': '0.9868', 'num_tokens': '8.607e+06', 'mean_token_accuracy': '0.7619', 'epoch': '0.01405'}
{'loss': '0.7633', 'grad_norm': '0.3118', 'learning_rate': '0.0001963', 'entropy': '0.7035', 'num_tokens': '8.613e+06', 'mean_token_accuracy': '0.8082', 'epoch': '0.01406'}
{'loss': '0.7527', 'grad_norm': '0.3034', 'learning_rate': '0.0001963', 'entropy': '0.7368', 'num_tokens': '8.619e+06', 'mean_token_accuracy': '0.8127', 'epoch': '0.01407'}
{'loss': '0.6629', 'grad_norm': '0.3428', 'learning_rate': '0.0001963', 'entropy': '0.6914', 'num_tokens': '8.625e+06', 'mean_token_accuracy': '0.8164', 'epoch': '0.01408'}
{'loss': '1.114', 'grad_norm': '0.3734', 'learning_rate': '0.0001963', 'entropy': '1.137', 'num_tokens': '8.63e+06', 'mean_token_accuracy': '0.7332', 'epoch': '0.01409'}
{'loss': '0.6737', 'grad_norm': '0.3961', 'learning_rate': '0.0001963', 'entropy': '0.7113', 'num_tokens': '8.633e+06', 'mean_token_accuracy': '0.8225', 'epoch': '0.0141'}
{'loss': '0.793', 'grad_norm': '0.3539', 'learning_rate': '0.0001963', 'entropy': '0.7557', 'num_tokens': '8.641e+06', 'mean_token_accuracy': '0.8162', 'epoch': '0.01411'}
{'loss': '0.8189', 'grad_norm': '0.3529', 'learning_rate': '0.0001963', 'entropy': '0.8567', 'num_tokens': '8.644e+06', 'mean_token_accuracy': '0.7873', 'epoch': '0.01412'}
{'loss': '1.131', 'grad_norm': '0.328', 'learning_rate': '0.0001963', 'entropy': '1.124', 'num_tokens': '8.648e+06', 'mean_token_accuracy': '0.7265', 'epoch': '0.01412'}
{'loss': '0.7703', 'grad_norm': '0.3203', 'learning_rate': '0.0001963', 'entropy': '0.8448', 'num_tokens': '8.657e+06', 'mean_token_accuracy': '0.7879', 'epoch': '0.01413'}
{'loss': '0.9571', 'grad_norm': '0.373', 'learning_rate': '0.0001962', 'entropy': '0.9471', 'num_tokens': '8.662e+06', 'mean_token_accuracy': '0.7632', 'epoch': '0.01414'}
{'loss': '1.055', 'grad_norm': '0.3584', 'learning_rate': '0.0001962', 'entropy': '1.103', 'num_tokens': '8.666e+06', 'mean_token_accuracy': '0.7274', 'epoch': '0.01415'}
{'loss': '0.9805', 'grad_norm': '0.3538', 'learning_rate': '0.0001962', 'entropy': '0.8976', 'num_tokens': '8.673e+06', 'mean_token_accuracy': '0.7774', 'epoch': '0.01416'}
{'loss': '1.371', 'grad_norm': '0.3933', 'learning_rate': '0.0001962', 'entropy': '1.298', 'num_tokens': '8.68e+06', 'mean_token_accuracy': '0.6864', 'epoch': '0.01417'}

 13%|█▎        | 1602/12000 [57:39<6:34:46,  2.28s/it]
 13%|█▎        | 1603/12000 [57:43<8:35:05,  2.97s/it]
                                                      

 13%|█▎        | 1603/12000 [57:43<8:35:05,  2.97s/it]
 13%|█▎        | 1604/12000 [57:45<7:29:40,  2.60s/it]
                                                      

 13%|█▎        | 1604/12000 [57:45<7:29:40,  2.60s/it]
 13%|█▎        | 1605/12000 [57:47<6:33:45,  2.27s/it]
                                                      

 13%|█▎        | 1605/12000 [57:47<6:33:45,  2.27s/it]
 13%|█▎        | 1606/12000 [57:48<5:58:04,  2.07s/it]
                                                      

 13%|█▎        | 1606/12000 [57:48<5:58:04,  2.07s/it]
 13%|█▎        | 1607/12000 [57:51<6:15:00,  2.16s/it]
                                                      

 13%|█▎        | 1607/12000 [57:51<6:15:00,  2.16s/it]
 13%|█▎        | 1608/12000 [57:53<6:22:37,  2.21s/it]
                                                      

 13%|█▎        | 1608/12000 [57:53<6:22:37,  2.21s/it]
 13%|█▎        | 1609/12000 [57:55<6:26:44,  2.23s/it]
                                                      

 13%|█▎        | 1609/12000 [57:55<6:26:44,  2.23s/it]
 13%|█▎        | 1610/12000 [57:57<5:47:17,  2.01s/it]
                                                      

 13%|█▎        | 1610/12000 [57:57<5:47:17,  2.01s/it]
 13%|█▎        | 1611/12000 [57:58<5:21:44,  1.86s/it]
                                                      

 13%|█▎        | 1611/12000 [57:58<5:21:44,  1.86s/it]
 13%|█▎        | 1612/12000 [58:01<5:44:43,  1.99s/it]
                                                      

 13%|█▎        | 1612/12000 [58:01<5:44:43,  1.99s/it]
 13%|█▎        | 1613/12000 [58:03<5:50:05,  2.02s/it]
                                                      

 13%|█▎        | 1613/12000 [58:03<5:50:05,  2.02s/it]
 13%|█▎        | 1614/12000 [58:05<5:44:49,  1.99s/it]
                                                      

 13%|█▎        | 1614/12000 [58:05<5:44:49,  1.99s/it]
 13%|█▎        | 1615/12000 [58:07<6:27:26,  2.24s/it]
                                                      

 13%|█▎        | 1615/12000 [58:07<6:27:26,  2.24s/it]
 13%|█▎        | 1616/12000 [58:11<7:24:57,  2.57s/it]
                                                      

 13%|█▎        | 1616/12000 [58:11<7:24:57,  2.57s/it]
 13%|█▎        | 1617/12000 [58:13<6:42:50,  2.33s/it]
                                                      

 13%|█▎        | 1617/12000 [58:13<6:42:50,  2.33s/it]
 13%|█▎        | 1618/12000 [58:15<6:38:56,  2.31s/it]
                                                      

 13%|█▎        | 1618/12000 [58:15<6:38:56,  2.31s/it]
 13%|█▎        | 1619/12000 [58:16<6:04:21,  2.11s/it]
                                                      

 13%|█▎        | 1619/12000 [58:16<6:04:21,  2.11s/it]
 14%|█▎        | 1620/12000 [58:20<7:02:57,  2.44s/it]
                                                      

 14%|█▎        | 1620/12000 [58:20<7:02:57,  2.44s/it]
 14%|█▎        | 1621/12000 [58:21<6:20:19,  2.20s/it]
                                                      

 14%|█▎        | 1621/12000 [58:21<6:20:19,  2.20s/it]
 14%|█▎        | 1622/12000 [58:23<6:05:05,  2.11s/it]
                                                      

 14%|█▎        | 1622/12000 [58:23<6:05:05,  2.11s/it]
 14%|█▎        | 1623/12000 [58:25<5:43:34,  1.99s/it]
                                                      

 14%|█▎        | 1623/12000 [58:25<5:43:34,  1.99s/it]
 14%|█▎        | 1624/12000 [58:30<8:17:40,  2.88s/it]
                                                      

 14%|█▎        | 1624/12000 [58:30<8:17:40,  2.88s/it]
 14%|█▎        | 1625/12000 [58:33<8:48:19,  3.06s/it]
                                                      

 14%|█▎        | 1625/12000 [58:33<8:48:19,  3.06s/it]
 14%|█▎        | 1626/12000 [58:36<8:10:58,  2.84s/it]
                                                      

 14%|█▎        | 1626/12000 [58:36<8:10:58,  2.84s/it]
 14%|█▎        | 1627/12000 [58:37<7:10:23,  2.49s/it]
                                                      

 14%|█▎        | 1627/12000 [58:37<7:10:23,  2.49s/it]
 14%|█▎        | 1628/12000 [58:39<6:20:31,  2.20s/it]
                                                      

 14%|█▎        | 1628/12000 [58:39<6:20:31,  2.20s/it]
 14%|█▎        | 1629/12000 [58:40<5:48:40,  2.02s/it]
                                                      

 14%|█▎        | 1629/12000 [58:40<5:48:40,  2.02s/it]
 14%|█▎        | 1630/12000 [58:42<5:16:47,  1.83s/it]
                                                      

 14%|█▎        | 1630/12000 [58:42<5:16:47,  1.83s/it]
 14%|█▎        | 1631/12000 [58:45<6:31:54,  2.27s/it]
                                                      

 14%|█▎        | 1631/12000 [58:45<6:31:54,  2.27s/it]
 14%|█▎        | 1632/12000 [58:48<7:12:43,  2.50s/it]
                                                      

 14%|█▎        | 1632/12000 [58:48<7:12:43,  2.50s/it]
 14%|█▎        | 1633/12000 [58:50<6:19:32,  2.20s/it]
                                                      

 14%|█▎        | 1633/12000 [58:50<6:19:32,  2.20s/it]
 14%|█▎        | 1634/12000 [58:51<5:37:18,  1.95s/it]
                                                      

 14%|█▎        | 1634/12000 [58:51<5:37:18,  1.95s/it]
 14%|█▎        | 1635/12000 [58:54<6:12:43,  2.16s/it]
                                                      

 14%|█▎        | 1635/12000 [58:54<6:12:43,  2.16s/it]
 14%|█▎        | 1636/12000 [58:55<5:42:18,  1.98s/it]
                                                      

 14%|█▎        | 1636/12000 [58:55<5:42:18,  1.98s/it]
 14%|█▎        | 1637/12000 [58:58<6:24:51,  2.23s/it]
                                                      

 14%|█▎        | 1637/12000 [58:58<6:24:51,  2.23s/it]
 14%|█▎        | 1638/12000 [59:00<5:45:24,  2.00s/it]
                                                      

 14%|█▎        | 1638/12000 [59:00<5:45:24,  2.00s/it]
 14%|█▎        | 1639/12000 [59:03<6:43:11,  2.33s/it]
                                                      

 14%|█▎        | 1639/12000 [59:03<6:43:11,  2.33s/it]
 14%|█▎        | 1640/12000 [59:04<6:08:10,  2.13s/it]
                                                      

 14%|█▎        | 1640/12000 [59:04<6:08:10,  2.13s/it]
 14%|█▎        | 1641/12000 [59:07<6:45:13,  2.35s/it]
                                                      

 14%|█▎        | 1641/12000 [59:07<6:45:13,  2.35s/it]
 14%|█▎        | 1642/12000 [59:09<6:12:05,  2.16s/it]
                                                      

 14%|█▎        | 1642/12000 [59:09<6:12:05,  2.16s/it]
 14%|█▎        | 1643/12000 [59:11<6:10:25,  2.15s/it]
                                                      

 14%|█▎        | 1643/12000 [59:11<6:10:25,  2.15s/it]
 14%|█▎        | 1644/12000 [59:13<5:49:11,  2.02s/it]
                                                      

 14%|█▎        | 1644/12000 [59:13<5:49:11,  2.02s/it]
 14%|█▎        | 1645/12000 [59:14<5:23:05,  1.87s/it]
                                                      

 14%|█▎        | 1645/12000 [59:14<5:23:05,  1.87s/it]
 14%|█▎        | 1646/12000 [59:16<5:06:22,  1.78s/it]
                                                      

 14%|█▎        | 1646/12000 [59:16<5:06:22,  1.78s/it]
 14%|█▎        | 1647/12000 [59:19<6:44:03,  2.34s/it]
                                                      

 14%|█▎        | 1647/12000 [59:19<6:44:03,  2.34s/it]
 14%|█▎        | 1648/12000 [59:22<6:50:33,  2.38s/it]
                                                      

 14%|█▎        | 1648/12000 [59:22<6:50:33,  2.38s/it]
 14%|█▎        | 1649/12000 [59:23<6:06:28,  2.12s/it]
                                                      
{'loss': '1.141', 'grad_norm': '0.453', 'learning_rate': '0.0001962', 'entropy': '1.186', 'num_tokens': '8.682e+06', 'mean_token_accuracy': '0.7127', 'epoch': '0.01418'}
{'loss': '0.9244', 'grad_norm': '0.4118', 'learning_rate': '0.0001962', 'entropy': '0.861', 'num_tokens': '8.689e+06', 'mean_token_accuracy': '0.7897', 'epoch': '0.01419'}
{'loss': '0.9584', 'grad_norm': '0.4182', 'learning_rate': '0.0001962', 'entropy': '0.8981', 'num_tokens': '8.694e+06', 'mean_token_accuracy': '0.7716', 'epoch': '0.0142'}
{'loss': '0.977', 'grad_norm': '0.3895', 'learning_rate': '0.0001962', 'entropy': '1.023', 'num_tokens': '8.698e+06', 'mean_token_accuracy': '0.7597', 'epoch': '0.0142'}
{'loss': '0.8779', 'grad_norm': '0.4734', 'learning_rate': '0.0001962', 'entropy': '0.8609', 'num_tokens': '8.703e+06', 'mean_token_accuracy': '0.7815', 'epoch': '0.01421'}
{'loss': '0.8732', 'grad_norm': '0.3048', 'learning_rate': '0.0001962', 'entropy': '0.9247', 'num_tokens': '8.71e+06', 'mean_token_accuracy': '0.7664', 'epoch': '0.01422'}
{'loss': '0.9636', 'grad_norm': '0.3327', 'learning_rate': '0.0001962', 'entropy': '0.9382', 'num_tokens': '8.718e+06', 'mean_token_accuracy': '0.7577', 'epoch': '0.01423'}
{'loss': '1.161', 'grad_norm': '0.4267', 'learning_rate': '0.0001962', 'entropy': '1.025', 'num_tokens': '8.724e+06', 'mean_token_accuracy': '0.7482', 'epoch': '0.01424'}
{'loss': '1.004', 'grad_norm': '0.3437', 'learning_rate': '0.0001962', 'entropy': '1.002', 'num_tokens': '8.728e+06', 'mean_token_accuracy': '0.7528', 'epoch': '0.01425'}
{'loss': '0.881', 'grad_norm': '0.3783', 'learning_rate': '0.0001962', 'entropy': '0.8242', 'num_tokens': '8.732e+06', 'mean_token_accuracy': '0.7963', 'epoch': '0.01426'}
{'loss': '0.9805', 'grad_norm': '0.4001', 'learning_rate': '0.0001961', 'entropy': '0.9429', 'num_tokens': '8.737e+06', 'mean_token_accuracy': '0.7571', 'epoch': '0.01427'}
{'loss': '1.439', 'grad_norm': '0.4264', 'learning_rate': '0.0001961', 'entropy': '1.307', 'num_tokens': '8.744e+06', 'mean_token_accuracy': '0.7003', 'epoch': '0.01427'}
{'loss': '1.058', 'grad_norm': '0.4113', 'learning_rate': '0.0001961', 'entropy': '1.001', 'num_tokens': '8.749e+06', 'mean_token_accuracy': '0.7565', 'epoch': '0.01428'}
{'loss': '0.807', 'grad_norm': '0.382', 'learning_rate': '0.0001961', 'entropy': '0.7906', 'num_tokens': '8.756e+06', 'mean_token_accuracy': '0.7817', 'epoch': '0.01429'}
{'loss': '1.182', 'grad_norm': '0.3309', 'learning_rate': '0.0001961', 'entropy': '1.256', 'num_tokens': '8.766e+06', 'mean_token_accuracy': '0.713', 'epoch': '0.0143'}
{'loss': '0.8628', 'grad_norm': '0.344', 'learning_rate': '0.0001961', 'entropy': '0.8545', 'num_tokens': '8.771e+06', 'mean_token_accuracy': '0.7973', 'epoch': '0.01431'}
{'loss': '1.23', 'grad_norm': '0.3551', 'learning_rate': '0.0001961', 'entropy': '1.067', 'num_tokens': '8.778e+06', 'mean_token_accuracy': '0.7491', 'epoch': '0.01432'}
{'loss': '0.7505', 'grad_norm': '0.3876', 'learning_rate': '0.0001961', 'entropy': '0.7641', 'num_tokens': '8.783e+06', 'mean_token_accuracy': '0.8165', 'epoch': '0.01433'}
{'loss': '1.04', 'grad_norm': '0.3356', 'learning_rate': '0.0001961', 'entropy': '1.09', 'num_tokens': '8.789e+06', 'mean_token_accuracy': '0.7497', 'epoch': '0.01434'}
{'loss': '0.9972', 'grad_norm': '0.4266', 'learning_rate': '0.0001961', 'entropy': '0.9912', 'num_tokens': '8.792e+06', 'mean_token_accuracy': '0.7564', 'epoch': '0.01435'}
{'loss': '0.7372', 'grad_norm': '0.3594', 'learning_rate': '0.0001961', 'entropy': '0.9145', 'num_tokens': '8.797e+06', 'mean_token_accuracy': '0.7882', 'epoch': '0.01435'}
{'loss': '0.9251', 'grad_norm': '0.4789', 'learning_rate': '0.0001961', 'entropy': '0.9779', 'num_tokens': '8.801e+06', 'mean_token_accuracy': '0.7638', 'epoch': '0.01436'}
{'loss': '1.32', 'grad_norm': '0.3346', 'learning_rate': '0.0001961', 'entropy': '1.146', 'num_tokens': '8.81e+06', 'mean_token_accuracy': '0.7361', 'epoch': '0.01437'}
{'loss': '0.7889', 'grad_norm': '0.3564', 'learning_rate': '0.000196', 'entropy': '0.7958', 'num_tokens': '8.814e+06', 'mean_token_accuracy': '0.8093', 'epoch': '0.01438'}
{'loss': '1.157', 'grad_norm': '0.3405', 'learning_rate': '0.000196', 'entropy': '1.03', 'num_tokens': '8.82e+06', 'mean_token_accuracy': '0.7377', 'epoch': '0.01439'}
{'loss': '0.6886', 'grad_norm': '0.317', 'learning_rate': '0.000196', 'entropy': '0.7816', 'num_tokens': '8.825e+06', 'mean_token_accuracy': '0.7889', 'epoch': '0.0144'}
{'loss': '0.9894', 'grad_norm': '0.4101', 'learning_rate': '0.000196', 'entropy': '0.9754', 'num_tokens': '8.828e+06', 'mean_token_accuracy': '0.7596', 'epoch': '0.01441'}
{'loss': '0.7436', 'grad_norm': '0.4252', 'learning_rate': '0.000196', 'entropy': '0.7049', 'num_tokens': '8.833e+06', 'mean_token_accuracy': '0.8102', 'epoch': '0.01442'}
{'loss': '0.9133', 'grad_norm': '0.4106', 'learning_rate': '0.000196', 'entropy': '0.8746', 'num_tokens': '8.836e+06', 'mean_token_accuracy': '0.7677', 'epoch': '0.01443'}
{'loss': '0.8426', 'grad_norm': '0.356', 'learning_rate': '0.000196', 'entropy': '0.8577', 'num_tokens': '8.843e+06', 'mean_token_accuracy': '0.781', 'epoch': '0.01443'}
{'loss': '0.5946', 'grad_norm': '0.2942', 'learning_rate': '0.000196', 'entropy': '0.7747', 'num_tokens': '8.849e+06', 'mean_token_accuracy': '0.805', 'epoch': '0.01444'}
{'loss': '0.7537', 'grad_norm': '0.4075', 'learning_rate': '0.000196', 'entropy': '0.7799', 'num_tokens': '8.853e+06', 'mean_token_accuracy': '0.7914', 'epoch': '0.01445'}
{'loss': '0.7634', 'grad_norm': '0.4142', 'learning_rate': '0.000196', 'entropy': '0.8717', 'num_tokens': '8.856e+06', 'mean_token_accuracy': '0.7798', 'epoch': '0.01446'}
{'loss': '0.9832', 'grad_norm': '0.416', 'learning_rate': '0.000196', 'entropy': '0.9305', 'num_tokens': '8.862e+06', 'mean_token_accuracy': '0.7715', 'epoch': '0.01447'}
{'loss': '0.9312', 'grad_norm': '0.3978', 'learning_rate': '0.000196', 'entropy': '0.9725', 'num_tokens': '8.867e+06', 'mean_token_accuracy': '0.748', 'epoch': '0.01448'}
{'loss': '0.8108', 'grad_norm': '0.3486', 'learning_rate': '0.000196', 'entropy': '0.8383', 'num_tokens': '8.873e+06', 'mean_token_accuracy': '0.8003', 'epoch': '0.01449'}
{'loss': '0.9038', 'grad_norm': '0.3461', 'learning_rate': '0.0001959', 'entropy': '0.9417', 'num_tokens': '8.877e+06', 'mean_token_accuracy': '0.7641', 'epoch': '0.0145'}
{'loss': '1.139', 'grad_norm': '0.3808', 'learning_rate': '0.0001959', 'entropy': '1.184', 'num_tokens': '8.885e+06', 'mean_token_accuracy': '0.7358', 'epoch': '0.0145'}
{'loss': '0.8118', 'grad_norm': '0.3628', 'learning_rate': '0.0001959', 'entropy': '0.9297', 'num_tokens': '8.889e+06', 'mean_token_accuracy': '0.779', 'epoch': '0.01451'}
{'loss': '1.023', 'grad_norm': '0.3933', 'learning_rate': '0.0001959', 'entropy': '1.033', 'num_tokens': '8.897e+06', 'mean_token_accuracy': '0.7627', 'epoch': '0.01452'}
{'loss': '0.967', 'grad_norm': '0.3832', 'learning_rate': '0.0001959', 'entropy': '1.037', 'num_tokens': '8.902e+06', 'mean_token_accuracy': '0.7561', 'epoch': '0.01453'}
{'loss': '1.181', 'grad_norm': '0.4218', 'learning_rate': '0.0001959', 'entropy': '1.106', 'num_tokens': '8.907e+06', 'mean_token_accuracy': '0.7219', 'epoch': '0.01454'}
{'loss': '0.902', 'grad_norm': '0.2911', 'learning_rate': '0.0001959', 'entropy': '0.923', 'num_tokens': '8.913e+06', 'mean_token_accuracy': '0.7738', 'epoch': '0.01455'}
{'loss': '0.7995', 'grad_norm': '0.3314', 'learning_rate': '0.0001959', 'entropy': '0.8066', 'num_tokens': '8.918e+06', 'mean_token_accuracy': '0.7924', 'epoch': '0.01456'}
{'loss': '0.6994', 'grad_norm': '0.3522', 'learning_rate': '0.0001959', 'entropy': '0.6896', 'num_tokens': '8.922e+06', 'mean_token_accuracy': '0.8219', 'epoch': '0.01457'}
{'loss': '0.9531', 'grad_norm': '0.4712', 'learning_rate': '0.0001959', 'entropy': '0.9109', 'num_tokens': '8.928e+06', 'mean_token_accuracy': '0.7679', 'epoch': '0.01458'}
{'loss': '0.9199', 'grad_norm': '0.3608', 'learning_rate': '0.0001959', 'entropy': '0.8764', 'num_tokens': '8.934e+06', 'mean_token_accuracy': '0.7854', 'epoch': '0.01458'}

 14%|█▎        | 1649/12000 [59:23<6:06:28,  2.12s/it]
 14%|█▍        | 1650/12000 [59:26<6:21:04,  2.21s/it]
                                                      

 14%|█▍        | 1650/12000 [59:26<6:21:04,  2.21s/it]
 14%|█▍        | 1651/12000 [59:29<7:30:35,  2.61s/it]
                                                      

 14%|█▍        | 1651/12000 [59:29<7:30:35,  2.61s/it]
 14%|█▍        | 1652/12000 [59:31<6:34:33,  2.29s/it]
                                                      

 14%|█▍        | 1652/12000 [59:31<6:34:33,  2.29s/it]
 14%|█▍        | 1653/12000 [59:33<6:09:19,  2.14s/it]
                                                      

 14%|█▍        | 1653/12000 [59:33<6:09:19,  2.14s/it]
 14%|█▍        | 1654/12000 [59:35<6:04:24,  2.11s/it]
                                                      

 14%|█▍        | 1654/12000 [59:35<6:04:24,  2.11s/it]
 14%|█▍        | 1655/12000 [59:37<6:20:52,  2.21s/it]
                                                      

 14%|█▍        | 1655/12000 [59:37<6:20:52,  2.21s/it]
 14%|█▍        | 1656/12000 [59:39<5:52:03,  2.04s/it]
                                                      

 14%|█▍        | 1656/12000 [59:39<5:52:03,  2.04s/it]
 14%|█▍        | 1657/12000 [59:40<5:21:44,  1.87s/it]
                                                      

 14%|█▍        | 1657/12000 [59:40<5:21:44,  1.87s/it]
 14%|█▍        | 1658/12000 [59:43<6:13:41,  2.17s/it]
                                                      

 14%|█▍        | 1658/12000 [59:43<6:13:41,  2.17s/it]
 14%|█▍        | 1659/12000 [59:46<6:21:54,  2.22s/it]
                                                      

 14%|█▍        | 1659/12000 [59:46<6:21:54,  2.22s/it]
 14%|█▍        | 1660/12000 [59:47<6:02:06,  2.10s/it]
                                                      

 14%|█▍        | 1660/12000 [59:47<6:02:06,  2.10s/it]
 14%|█▍        | 1661/12000 [59:49<6:00:23,  2.09s/it]
                                                      

 14%|█▍        | 1661/12000 [59:49<6:00:23,  2.09s/it]
 14%|█▍        | 1662/12000 [59:51<5:39:42,  1.97s/it]
                                                      

 14%|█▍        | 1662/12000 [59:51<5:39:42,  1.97s/it]
 14%|█▍        | 1663/12000 [59:55<7:05:42,  2.47s/it]
                                                      

 14%|█▍        | 1663/12000 [59:55<7:05:42,  2.47s/it]
 14%|█▍        | 1664/12000 [59:57<6:33:14,  2.28s/it]
                                                      

 14%|█▍        | 1664/12000 [59:57<6:33:14,  2.28s/it]
 14%|█▍        | 1665/12000 [59:58<5:52:50,  2.05s/it]
                                                      

 14%|█▍        | 1665/12000 [59:58<5:52:50,  2.05s/it]
 14%|█▍        | 1666/12000 [1:00:00<5:23:53,  1.88s/it]
                                                        

 14%|█▍        | 1666/12000 [1:00:00<5:23:53,  1.88s/it]
 14%|█▍        | 1667/12000 [1:00:01<5:03:36,  1.76s/it]
                                                        

 14%|█▍        | 1667/12000 [1:00:01<5:03:36,  1.76s/it]
 14%|█▍        | 1668/12000 [1:00:03<5:11:15,  1.81s/it]
                                                        

 14%|█▍        | 1668/12000 [1:00:03<5:11:15,  1.81s/it]
 14%|█▍        | 1669/12000 [1:00:08<7:36:05,  2.65s/it]
                                                        

 14%|█▍        | 1669/12000 [1:00:08<7:36:05,  2.65s/it]
 14%|█▍        | 1670/12000 [1:00:10<7:25:40,  2.59s/it]
                                                        

 14%|█▍        | 1670/12000 [1:00:10<7:25:40,  2.59s/it]
 14%|█▍        | 1671/12000 [1:00:12<6:29:00,  2.26s/it]
                                                        

 14%|█▍        | 1671/12000 [1:00:12<6:29:00,  2.26s/it]
 14%|█▍        | 1672/12000 [1:00:13<5:48:13,  2.02s/it]
                                                        

 14%|█▍        | 1672/12000 [1:00:13<5:48:13,  2.02s/it]
 14%|█▍        | 1673/12000 [1:00:16<6:38:31,  2.32s/it]
                                                        

 14%|█▍        | 1673/12000 [1:00:16<6:38:31,  2.32s/it]
 14%|█▍        | 1674/12000 [1:00:18<6:05:51,  2.13s/it]
                                                        

 14%|█▍        | 1674/12000 [1:00:18<6:05:51,  2.13s/it]
 14%|█▍        | 1675/12000 [1:00:19<5:46:25,  2.01s/it]
                                                        

 14%|█▍        | 1675/12000 [1:00:19<5:46:25,  2.01s/it]
 14%|█▍        | 1676/12000 [1:00:21<5:17:12,  1.84s/it]
                                                        

 14%|█▍        | 1676/12000 [1:00:21<5:17:12,  1.84s/it]
 14%|█▍        | 1677/12000 [1:00:23<5:53:18,  2.05s/it]
                                                        

 14%|█▍        | 1677/12000 [1:00:23<5:53:18,  2.05s/it]
 14%|█▍        | 1678/12000 [1:00:26<6:02:51,  2.11s/it]
                                                        

 14%|█▍        | 1678/12000 [1:00:26<6:02:51,  2.11s/it]
 14%|█▍        | 1679/12000 [1:00:27<5:43:47,  2.00s/it]
                                                        

 14%|█▍        | 1679/12000 [1:00:27<5:43:47,  2.00s/it]
 14%|█▍        | 1680/12000 [1:00:29<5:12:01,  1.81s/it]
                                                        

 14%|█▍        | 1680/12000 [1:00:29<5:12:01,  1.81s/it]
 14%|█▍        | 1681/12000 [1:00:30<4:56:55,  1.73s/it]
                                                        

 14%|█▍        | 1681/12000 [1:00:30<4:56:55,  1.73s/it]
 14%|█▍        | 1682/12000 [1:00:32<4:41:25,  1.64s/it]
                                                        

 14%|█▍        | 1682/12000 [1:00:32<4:41:25,  1.64s/it]
 14%|█▍        | 1683/12000 [1:00:33<4:46:36,  1.67s/it]
                                                        

 14%|█▍        | 1683/12000 [1:00:33<4:46:36,  1.67s/it]
 14%|█▍        | 1684/12000 [1:00:36<5:17:35,  1.85s/it]
                                                        

 14%|█▍        | 1684/12000 [1:00:36<5:17:35,  1.85s/it]
 14%|█▍        | 1685/12000 [1:00:40<7:07:40,  2.49s/it]
                                                        

 14%|█▍        | 1685/12000 [1:00:40<7:07:40,  2.49s/it]
 14%|█▍        | 1686/12000 [1:00:44<8:48:35,  3.08s/it]
                                                        

 14%|█▍        | 1686/12000 [1:00:44<8:48:35,  3.08s/it]
 14%|█▍        | 1687/12000 [1:00:46<7:26:53,  2.60s/it]
                                                        

 14%|█▍        | 1687/12000 [1:00:46<7:26:53,  2.60s/it]
 14%|█▍        | 1688/12000 [1:00:47<6:37:42,  2.31s/it]
                                                        

 14%|█▍        | 1688/12000 [1:00:47<6:37:42,  2.31s/it]
 14%|█▍        | 1689/12000 [1:00:50<6:38:04,  2.32s/it]
                                                        

 14%|█▍        | 1689/12000 [1:00:50<6:38:04,  2.32s/it]
 14%|█▍        | 1690/12000 [1:00:51<6:00:11,  2.10s/it]
                                                        

 14%|█▍        | 1690/12000 [1:00:51<6:00:11,  2.10s/it]
 14%|█▍        | 1691/12000 [1:00:55<7:30:40,  2.62s/it]
                                                        

 14%|█▍        | 1691/12000 [1:00:55<7:30:40,  2.62s/it]
 14%|█▍        | 1692/12000 [1:00:57<7:11:55,  2.51s/it]
                                                        

 14%|█▍        | 1692/12000 [1:00:57<7:11:55,  2.51s/it]
 14%|█▍        | 1693/12000 [1:00:59<6:22:44,  2.23s/it]
                                                        

 14%|█▍        | 1693/12000 [1:00:59<6:22:44,  2.23s/it]
 14%|█▍        | 1694/12000 [1:01:00<5:45:10,  2.01s/it]
                                                        

 14%|█▍        | 1694/12000 [1:01:00<5:45:10,  2.01s/it]
 14%|█▍        | 1695/12000 [1:01:02<5:44:23,  2.01s/it]
                                                        

 14%|█▍        | 1695/12000 [1:01:02<5:44:23,  2.01s/it]
 14%|█▍        | 1696/12000 [1:01:04<5:39:05,  1.97s/it]
                                                        
{'loss': '0.7121', 'grad_norm': '0.3169', 'learning_rate': '0.0001959', 'entropy': '0.6883', 'num_tokens': '8.94e+06', 'mean_token_accuracy': '0.8245', 'epoch': '0.01459'}
{'loss': '0.9301', 'grad_norm': '0.4134', 'learning_rate': '0.0001959', 'entropy': '0.8667', 'num_tokens': '8.947e+06', 'mean_token_accuracy': '0.7736', 'epoch': '0.0146'}
{'loss': '0.913', 'grad_norm': '0.3883', 'learning_rate': '0.0001958', 'entropy': '1.01', 'num_tokens': '8.953e+06', 'mean_token_accuracy': '0.7631', 'epoch': '0.01461'}
{'loss': '1.021', 'grad_norm': '0.3612', 'learning_rate': '0.0001958', 'entropy': '0.974', 'num_tokens': '8.958e+06', 'mean_token_accuracy': '0.7585', 'epoch': '0.01462'}
{'loss': '0.8675', 'grad_norm': '0.334', 'learning_rate': '0.0001958', 'entropy': '0.8488', 'num_tokens': '8.962e+06', 'mean_token_accuracy': '0.8015', 'epoch': '0.01463'}
{'loss': '1.176', 'grad_norm': '0.3606', 'learning_rate': '0.0001958', 'entropy': '1.202', 'num_tokens': '8.969e+06', 'mean_token_accuracy': '0.7151', 'epoch': '0.01464'}
{'loss': '0.7979', 'grad_norm': '0.3592', 'learning_rate': '0.0001958', 'entropy': '0.7542', 'num_tokens': '8.977e+06', 'mean_token_accuracy': '0.8063', 'epoch': '0.01465'}
{'loss': '0.8689', 'grad_norm': '0.3707', 'learning_rate': '0.0001958', 'entropy': '0.9836', 'num_tokens': '8.981e+06', 'mean_token_accuracy': '0.7738', 'epoch': '0.01466'}
{'loss': '0.7553', 'grad_norm': '0.3356', 'learning_rate': '0.0001958', 'entropy': '0.8687', 'num_tokens': '8.986e+06', 'mean_token_accuracy': '0.7986', 'epoch': '0.01466'}
{'loss': '1.229', 'grad_norm': '0.299', 'learning_rate': '0.0001958', 'entropy': '1.281', 'num_tokens': '8.995e+06', 'mean_token_accuracy': '0.721', 'epoch': '0.01467'}
{'loss': '1.067', 'grad_norm': '0.4236', 'learning_rate': '0.0001958', 'entropy': '1.093', 'num_tokens': '9.001e+06', 'mean_token_accuracy': '0.744', 'epoch': '0.01468'}
{'loss': '1.177', 'grad_norm': '0.4227', 'learning_rate': '0.0001958', 'entropy': '1.147', 'num_tokens': '9.005e+06', 'mean_token_accuracy': '0.719', 'epoch': '0.01469'}
{'loss': '0.8554', 'grad_norm': '0.3982', 'learning_rate': '0.0001958', 'entropy': '0.8645', 'num_tokens': '9.011e+06', 'mean_token_accuracy': '0.7848', 'epoch': '0.0147'}
{'loss': '1.02', 'grad_norm': '0.3757', 'learning_rate': '0.0001958', 'entropy': '1.023', 'num_tokens': '9.016e+06', 'mean_token_accuracy': '0.7732', 'epoch': '0.01471'}
{'loss': '0.8621', 'grad_norm': '0.3339', 'learning_rate': '0.0001957', 'entropy': '0.865', 'num_tokens': '9.024e+06', 'mean_token_accuracy': '0.7772', 'epoch': '0.01472'}
{'loss': '0.9896', 'grad_norm': '0.3266', 'learning_rate': '0.0001957', 'entropy': '0.9147', 'num_tokens': '9.03e+06', 'mean_token_accuracy': '0.7642', 'epoch': '0.01473'}
{'loss': '0.7334', 'grad_norm': '0.3256', 'learning_rate': '0.0001957', 'entropy': '0.7435', 'num_tokens': '9.034e+06', 'mean_token_accuracy': '0.8204', 'epoch': '0.01473'}
{'loss': '1.325', 'grad_norm': '0.5058', 'learning_rate': '0.0001957', 'entropy': '1.295', 'num_tokens': '9.038e+06', 'mean_token_accuracy': '0.6726', 'epoch': '0.01474'}
{'loss': '1.22', 'grad_norm': '0.554', 'learning_rate': '0.0001957', 'entropy': '1.201', 'num_tokens': '9.041e+06', 'mean_token_accuracy': '0.7195', 'epoch': '0.01475'}
{'loss': '0.9758', 'grad_norm': '0.3713', 'learning_rate': '0.0001957', 'entropy': '0.9956', 'num_tokens': '9.047e+06', 'mean_token_accuracy': '0.7625', 'epoch': '0.01476'}
{'loss': '0.7814', 'grad_norm': '0.2859', 'learning_rate': '0.0001957', 'entropy': '0.761', 'num_tokens': '9.055e+06', 'mean_token_accuracy': '0.8116', 'epoch': '0.01477'}
{'loss': '1.026', 'grad_norm': '0.3602', 'learning_rate': '0.0001957', 'entropy': '0.9773', 'num_tokens': '9.062e+06', 'mean_token_accuracy': '0.7377', 'epoch': '0.01478'}
{'loss': '0.7721', 'grad_norm': '0.372', 'learning_rate': '0.0001957', 'entropy': '0.8628', 'num_tokens': '9.066e+06', 'mean_token_accuracy': '0.7858', 'epoch': '0.01479'}
{'loss': '1.318', 'grad_norm': '0.3663', 'learning_rate': '0.0001957', 'entropy': '1.276', 'num_tokens': '9.07e+06', 'mean_token_accuracy': '0.7119', 'epoch': '0.0148'}
{'loss': '0.7841', 'grad_norm': '0.3017', 'learning_rate': '0.0001957', 'entropy': '0.8314', 'num_tokens': '9.076e+06', 'mean_token_accuracy': '0.7943', 'epoch': '0.01481'}
{'loss': '0.754', 'grad_norm': '0.3546', 'learning_rate': '0.0001957', 'entropy': '0.8005', 'num_tokens': '9.081e+06', 'mean_token_accuracy': '0.803', 'epoch': '0.01481'}
{'loss': '0.7926', 'grad_norm': '0.4696', 'learning_rate': '0.0001957', 'entropy': '0.7943', 'num_tokens': '9.085e+06', 'mean_token_accuracy': '0.7973', 'epoch': '0.01482'}
{'loss': '1.096', 'grad_norm': '0.4002', 'learning_rate': '0.0001956', 'entropy': '1.034', 'num_tokens': '9.089e+06', 'mean_token_accuracy': '0.7354', 'epoch': '0.01483'}
{'loss': '1.153', 'grad_norm': '0.4342', 'learning_rate': '0.0001956', 'entropy': '1.079', 'num_tokens': '9.096e+06', 'mean_token_accuracy': '0.7408', 'epoch': '0.01484'}
{'loss': '1.103', 'grad_norm': '0.4545', 'learning_rate': '0.0001956', 'entropy': '1.146', 'num_tokens': '9.1e+06', 'mean_token_accuracy': '0.7299', 'epoch': '0.01485'}
{'loss': '0.8288', 'grad_norm': '0.3921', 'learning_rate': '0.0001956', 'entropy': '0.8453', 'num_tokens': '9.106e+06', 'mean_token_accuracy': '0.7945', 'epoch': '0.01486'}
{'loss': '0.8289', 'grad_norm': '0.3806', 'learning_rate': '0.0001956', 'entropy': '0.9003', 'num_tokens': '9.109e+06', 'mean_token_accuracy': '0.7878', 'epoch': '0.01487'}
{'loss': '0.9096', 'grad_norm': '0.3563', 'learning_rate': '0.0001956', 'entropy': '0.8303', 'num_tokens': '9.114e+06', 'mean_token_accuracy': '0.7878', 'epoch': '0.01488'}
{'loss': '0.7198', 'grad_norm': '0.3347', 'learning_rate': '0.0001956', 'entropy': '0.7724', 'num_tokens': '9.118e+06', 'mean_token_accuracy': '0.8079', 'epoch': '0.01489'}
{'loss': '1.199', 'grad_norm': '0.377', 'learning_rate': '0.0001956', 'entropy': '1.194', 'num_tokens': '9.123e+06', 'mean_token_accuracy': '0.725', 'epoch': '0.01489'}
{'loss': '1.081', 'grad_norm': '0.4078', 'learning_rate': '0.0001956', 'entropy': '1.074', 'num_tokens': '9.13e+06', 'mean_token_accuracy': '0.7478', 'epoch': '0.0149'}
{'loss': '1.618', 'grad_norm': '0.4457', 'learning_rate': '0.0001956', 'entropy': '1.256', 'num_tokens': '9.137e+06', 'mean_token_accuracy': '0.7077', 'epoch': '0.01491'}
{'loss': '1.02', 'grad_norm': '0.302', 'learning_rate': '0.0001956', 'entropy': '0.9657', 'num_tokens': '9.147e+06', 'mean_token_accuracy': '0.7768', 'epoch': '0.01492'}
{'loss': '0.903', 'grad_norm': '0.3576', 'learning_rate': '0.0001956', 'entropy': '0.9136', 'num_tokens': '9.151e+06', 'mean_token_accuracy': '0.7653', 'epoch': '0.01493'}
{'loss': '0.6672', 'grad_norm': '0.3107', 'learning_rate': '0.0001955', 'entropy': '0.6674', 'num_tokens': '9.157e+06', 'mean_token_accuracy': '0.8179', 'epoch': '0.01494'}
{'loss': '0.9659', 'grad_norm': '0.3714', 'learning_rate': '0.0001955', 'entropy': '0.9992', 'num_tokens': '9.162e+06', 'mean_token_accuracy': '0.7559', 'epoch': '0.01495'}
{'loss': '1.08', 'grad_norm': '0.4477', 'learning_rate': '0.0001955', 'entropy': '1.145', 'num_tokens': '9.166e+06', 'mean_token_accuracy': '0.7223', 'epoch': '0.01496'}
{'loss': '1.139', 'grad_norm': '0.3766', 'learning_rate': '0.0001955', 'entropy': '1.169', 'num_tokens': '9.172e+06', 'mean_token_accuracy': '0.7273', 'epoch': '0.01497'}
{'loss': '0.7987', 'grad_norm': '0.4159', 'learning_rate': '0.0001955', 'entropy': '0.7804', 'num_tokens': '9.177e+06', 'mean_token_accuracy': '0.8039', 'epoch': '0.01497'}
{'loss': '0.9649', 'grad_norm': '0.366', 'learning_rate': '0.0001955', 'entropy': '0.9326', 'num_tokens': '9.182e+06', 'mean_token_accuracy': '0.7778', 'epoch': '0.01498'}
{'loss': '0.9069', 'grad_norm': '0.3699', 'learning_rate': '0.0001955', 'entropy': '0.9406', 'num_tokens': '9.187e+06', 'mean_token_accuracy': '0.7627', 'epoch': '0.01499'}
{'loss': '0.975', 'grad_norm': '0.5003', 'learning_rate': '0.0001955', 'entropy': '0.8696', 'num_tokens': '9.192e+06', 'mean_token_accuracy': '0.789', 'epoch': '0.015'}

 14%|█▍        | 1696/12000 [1:01:04<5:39:05,  1.97s/it]
 14%|█▍        | 1697/12000 [1:01:06<5:18:29,  1.85s/it]
                                                        

 14%|█▍        | 1697/12000 [1:01:06<5:18:29,  1.85s/it]
 14%|█▍        | 1698/12000 [1:01:10<6:57:41,  2.43s/it]
                                                        

 14%|█▍        | 1698/12000 [1:01:10<6:57:41,  2.43s/it]
 14%|█▍        | 1699/12000 [1:01:13<7:23:15,  2.58s/it]
                                                        

 14%|█▍        | 1699/12000 [1:01:13<7:23:15,  2.58s/it]
 14%|█▍        | 1700/12000 [1:01:15<7:38:57,  2.67s/it]
                                                        

 14%|█▍        | 1700/12000 [1:01:15<7:38:57,  2.67s/it]
 14%|█▍        | 1701/12000 [1:01:18<7:40:15,  2.68s/it]
                                                        

 14%|█▍        | 1701/12000 [1:01:18<7:40:15,  2.68s/it]
 14%|█▍        | 1702/12000 [1:01:20<6:41:46,  2.34s/it]
                                                        

 14%|█▍        | 1702/12000 [1:01:20<6:41:46,  2.34s/it]
 14%|█▍        | 1703/12000 [1:01:23<7:38:52,  2.67s/it]
                                                        

 14%|█▍        | 1703/12000 [1:01:23<7:38:52,  2.67s/it]
 14%|█▍        | 1704/12000 [1:01:25<7:21:06,  2.57s/it]
                                                        

 14%|█▍        | 1704/12000 [1:01:25<7:21:06,  2.57s/it]
 14%|█▍        | 1705/12000 [1:01:27<6:35:21,  2.30s/it]
                                                        

 14%|█▍        | 1705/12000 [1:01:27<6:35:21,  2.30s/it]
 14%|█▍        | 1706/12000 [1:01:30<7:14:03,  2.53s/it]
                                                        

 14%|█▍        | 1706/12000 [1:01:30<7:14:03,  2.53s/it]
 14%|█▍        | 1707/12000 [1:01:32<6:22:47,  2.23s/it]
                                                        

 14%|█▍        | 1707/12000 [1:01:32<6:22:47,  2.23s/it]
 14%|█▍        | 1708/12000 [1:01:34<5:59:27,  2.10s/it]
                                                        

 14%|█▍        | 1708/12000 [1:01:34<5:59:27,  2.10s/it]
 14%|█▍        | 1709/12000 [1:01:36<6:18:40,  2.21s/it]
                                                        

 14%|█▍        | 1709/12000 [1:01:36<6:18:40,  2.21s/it]
 14%|█▍        | 1710/12000 [1:01:38<5:42:09,  2.00s/it]
                                                        

 14%|█▍        | 1710/12000 [1:01:38<5:42:09,  2.00s/it]
 14%|█▍        | 1711/12000 [1:01:39<5:18:48,  1.86s/it]
                                                        

 14%|█▍        | 1711/12000 [1:01:39<5:18:48,  1.86s/it]
 14%|█▍        | 1712/12000 [1:01:41<5:04:56,  1.78s/it]
                                                        

 14%|█▍        | 1712/12000 [1:01:41<5:04:56,  1.78s/it]
 14%|█▍        | 1713/12000 [1:01:45<7:14:14,  2.53s/it]
                                                        

 14%|█▍        | 1713/12000 [1:01:45<7:14:14,  2.53s/it]
 14%|█▍        | 1714/12000 [1:01:47<7:13:28,  2.53s/it]
                                                        

 14%|█▍        | 1714/12000 [1:01:47<7:13:28,  2.53s/it]
 14%|█▍        | 1715/12000 [1:01:49<6:21:28,  2.23s/it]
                                                        

 14%|█▍        | 1715/12000 [1:01:49<6:21:28,  2.23s/it]
 14%|█▍        | 1716/12000 [1:01:51<5:48:32,  2.03s/it]
                                                        

 14%|█▍        | 1716/12000 [1:01:51<5:48:32,  2.03s/it]
 14%|█▍        | 1717/12000 [1:01:52<5:16:49,  1.85s/it]
                                                        

 14%|█▍        | 1717/12000 [1:01:52<5:16:49,  1.85s/it]
 14%|█▍        | 1718/12000 [1:01:54<5:07:20,  1.79s/it]
                                                        

 14%|█▍        | 1718/12000 [1:01:54<5:07:20,  1.79s/it]
 14%|█▍        | 1719/12000 [1:01:55<4:47:52,  1.68s/it]
                                                        

 14%|█▍        | 1719/12000 [1:01:55<4:47:52,  1.68s/it]
 14%|█▍        | 1720/12000 [1:01:57<4:54:31,  1.72s/it]
                                                        

 14%|█▍        | 1720/12000 [1:01:57<4:54:31,  1.72s/it]
 14%|█▍        | 1721/12000 [1:01:58<4:43:56,  1.66s/it]
                                                        

 14%|█▍        | 1721/12000 [1:01:58<4:43:56,  1.66s/it]
 14%|█▍        | 1722/12000 [1:02:00<4:36:10,  1.61s/it]
                                                        

 14%|█▍        | 1722/12000 [1:02:00<4:36:10,  1.61s/it]
 14%|█▍        | 1723/12000 [1:02:02<4:40:55,  1.64s/it]
                                                        

 14%|█▍        | 1723/12000 [1:02:02<4:40:55,  1.64s/it]
 14%|█▍        | 1724/12000 [1:02:03<4:30:01,  1.58s/it]
                                                        

 14%|█▍        | 1724/12000 [1:02:03<4:30:01,  1.58s/it]
 14%|█▍        | 1725/12000 [1:02:08<7:11:15,  2.52s/it]
                                                        

 14%|█▍        | 1725/12000 [1:02:08<7:11:15,  2.52s/it]
 14%|█▍        | 1726/12000 [1:02:10<6:35:28,  2.31s/it]
                                                        

 14%|█▍        | 1726/12000 [1:02:10<6:35:28,  2.31s/it]
 14%|█▍        | 1727/12000 [1:02:11<6:07:52,  2.15s/it]
                                                        

 14%|█▍        | 1727/12000 [1:02:11<6:07:52,  2.15s/it]
 14%|█▍        | 1728/12000 [1:02:14<6:30:38,  2.28s/it]
                                                        

 14%|█▍        | 1728/12000 [1:02:14<6:30:38,  2.28s/it]
 14%|█▍        | 1729/12000 [1:02:16<5:57:05,  2.09s/it]
                                                        

 14%|█▍        | 1729/12000 [1:02:16<5:57:05,  2.09s/it]
 14%|█▍        | 1730/12000 [1:02:17<5:31:37,  1.94s/it]
                                                        

 14%|█▍        | 1730/12000 [1:02:17<5:31:37,  1.94s/it]
 14%|█▍        | 1731/12000 [1:02:20<6:10:05,  2.16s/it]
                                                        

 14%|█▍        | 1731/12000 [1:02:20<6:10:05,  2.16s/it]
 14%|█▍        | 1732/12000 [1:02:21<5:39:50,  1.99s/it]
                                                        

 14%|█▍        | 1732/12000 [1:02:21<5:39:50,  1.99s/it]
 14%|█▍        | 1733/12000 [1:02:24<5:54:00,  2.07s/it]
                                                        

 14%|█▍        | 1733/12000 [1:02:24<5:54:00,  2.07s/it]
 14%|█▍        | 1734/12000 [1:02:26<5:54:26,  2.07s/it]
                                                        

 14%|█▍        | 1734/12000 [1:02:26<5:54:26,  2.07s/it]
 14%|█▍        | 1735/12000 [1:02:30<7:40:50,  2.69s/it]
                                                        

 14%|█▍        | 1735/12000 [1:02:30<7:40:50,  2.69s/it]
 14%|█▍        | 1736/12000 [1:02:31<6:44:06,  2.36s/it]
                                                        

 14%|█▍        | 1736/12000 [1:02:31<6:44:06,  2.36s/it]
 14%|█▍        | 1737/12000 [1:02:33<6:11:32,  2.17s/it]
                                                        

 14%|█▍        | 1737/12000 [1:02:33<6:11:32,  2.17s/it]
 14%|█▍        | 1738/12000 [1:02:36<6:19:14,  2.22s/it]
                                                        

 14%|█▍        | 1738/12000 [1:02:36<6:19:14,  2.22s/it]
 14%|█▍        | 1739/12000 [1:02:37<5:47:54,  2.03s/it]
                                                        

 14%|█▍        | 1739/12000 [1:02:37<5:47:54,  2.03s/it]
 14%|█▍        | 1740/12000 [1:02:41<7:01:03,  2.46s/it]
                                                        

 14%|█▍        | 1740/12000 [1:02:41<7:01:03,  2.46s/it]
 15%|█▍        | 1741/12000 [1:02:42<6:14:57,  2.19s/it]
                                                        

 15%|█▍        | 1741/12000 [1:02:42<6:14:57,  2.19s/it]
 15%|█▍        | 1742/12000 [1:02:44<5:55:03,  2.08s/it]
                                                        

 15%|█▍        | 1742/12000 [1:02:44<5:55:03,  2.08s/it]
 15%|█▍        | 1743/12000 [1:02:46<5:31:52,  1.94s/it]
                                                        
{'loss': '1.163', 'grad_norm': '0.4132', 'learning_rate': '0.0001955', 'entropy': '1.217', 'num_tokens': '9.197e+06', 'mean_token_accuracy': '0.6966', 'epoch': '0.01501'}
{'loss': '0.7783', 'grad_norm': '0.3705', 'learning_rate': '0.0001955', 'entropy': '0.8094', 'num_tokens': '9.202e+06', 'mean_token_accuracy': '0.8', 'epoch': '0.01502'}
{'loss': '0.59', 'grad_norm': '0.2986', 'learning_rate': '0.0001955', 'entropy': '0.6802', 'num_tokens': '9.21e+06', 'mean_token_accuracy': '0.8213', 'epoch': '0.01503'}
{'loss': '0.881', 'grad_norm': '0.3419', 'learning_rate': '0.0001955', 'entropy': '0.8799', 'num_tokens': '9.216e+06', 'mean_token_accuracy': '0.7893', 'epoch': '0.01504'}
{'loss': '0.7117', 'grad_norm': '0.3771', 'learning_rate': '0.0001954', 'entropy': '0.8546', 'num_tokens': '9.222e+06', 'mean_token_accuracy': '0.7908', 'epoch': '0.01504'}
{'loss': '0.9136', 'grad_norm': '0.4753', 'learning_rate': '0.0001954', 'entropy': '0.9167', 'num_tokens': '9.226e+06', 'mean_token_accuracy': '0.7816', 'epoch': '0.01505'}
{'loss': '0.9915', 'grad_norm': '0.3923', 'learning_rate': '0.0001954', 'entropy': '0.9711', 'num_tokens': '9.23e+06', 'mean_token_accuracy': '0.756', 'epoch': '0.01506'}
{'loss': '1.072', 'grad_norm': '0.3681', 'learning_rate': '0.0001954', 'entropy': '1.144', 'num_tokens': '9.238e+06', 'mean_token_accuracy': '0.728', 'epoch': '0.01507'}
{'loss': '1.135', 'grad_norm': '0.4214', 'learning_rate': '0.0001954', 'entropy': '1.039', 'num_tokens': '9.245e+06', 'mean_token_accuracy': '0.744', 'epoch': '0.01508'}
{'loss': '0.9382', 'grad_norm': '0.374', 'learning_rate': '0.0001954', 'entropy': '0.9822', 'num_tokens': '9.25e+06', 'mean_token_accuracy': '0.7519', 'epoch': '0.01509'}
{'loss': '0.7722', 'grad_norm': '0.3641', 'learning_rate': '0.0001954', 'entropy': '0.7531', 'num_tokens': '9.257e+06', 'mean_token_accuracy': '0.8236', 'epoch': '0.0151'}
{'loss': '1.115', 'grad_norm': '0.364', 'learning_rate': '0.0001954', 'entropy': '1.139', 'num_tokens': '9.261e+06', 'mean_token_accuracy': '0.7388', 'epoch': '0.01511'}
{'loss': '0.8248', 'grad_norm': '0.3267', 'learning_rate': '0.0001954', 'entropy': '0.8469', 'num_tokens': '9.267e+06', 'mean_token_accuracy': '0.7945', 'epoch': '0.01512'}
{'loss': '1.003', 'grad_norm': '0.366', 'learning_rate': '0.0001954', 'entropy': '0.9635', 'num_tokens': '9.273e+06', 'mean_token_accuracy': '0.7763', 'epoch': '0.01512'}
{'loss': '0.8541', 'grad_norm': '0.2819', 'learning_rate': '0.0001954', 'entropy': '0.8676', 'num_tokens': '9.278e+06', 'mean_token_accuracy': '0.784', 'epoch': '0.01513'}
{'loss': '0.9221', 'grad_norm': '0.3449', 'learning_rate': '0.0001954', 'entropy': '0.9287', 'num_tokens': '9.283e+06', 'mean_token_accuracy': '0.7682', 'epoch': '0.01514'}
{'loss': '0.6915', 'grad_norm': '0.3436', 'learning_rate': '0.0001953', 'entropy': '0.7866', 'num_tokens': '9.288e+06', 'mean_token_accuracy': '0.8046', 'epoch': '0.01515'}
{'loss': '1.144', 'grad_norm': '0.4152', 'learning_rate': '0.0001953', 'entropy': '0.9459', 'num_tokens': '9.294e+06', 'mean_token_accuracy': '0.7419', 'epoch': '0.01516'}
{'loss': '0.8422', 'grad_norm': '0.2874', 'learning_rate': '0.0001953', 'entropy': '0.8061', 'num_tokens': '9.301e+06', 'mean_token_accuracy': '0.7874', 'epoch': '0.01517'}
{'loss': '0.8333', 'grad_norm': '0.2836', 'learning_rate': '0.0001953', 'entropy': '0.8548', 'num_tokens': '9.306e+06', 'mean_token_accuracy': '0.7817', 'epoch': '0.01518'}
{'loss': '1.304', 'grad_norm': '0.4327', 'learning_rate': '0.0001953', 'entropy': '1.339', 'num_tokens': '9.31e+06', 'mean_token_accuracy': '0.6741', 'epoch': '0.01519'}
{'loss': '1.201', 'grad_norm': '0.4039', 'learning_rate': '0.0001953', 'entropy': '1.231', 'num_tokens': '9.314e+06', 'mean_token_accuracy': '0.6981', 'epoch': '0.0152'}
{'loss': '0.6073', 'grad_norm': '0.3531', 'learning_rate': '0.0001953', 'entropy': '0.6658', 'num_tokens': '9.319e+06', 'mean_token_accuracy': '0.8303', 'epoch': '0.0152'}
{'loss': '0.7909', 'grad_norm': '0.3119', 'learning_rate': '0.0001953', 'entropy': '0.8216', 'num_tokens': '9.324e+06', 'mean_token_accuracy': '0.7893', 'epoch': '0.01521'}
{'loss': '0.9354', 'grad_norm': '0.3263', 'learning_rate': '0.0001953', 'entropy': '1.005', 'num_tokens': '9.328e+06', 'mean_token_accuracy': '0.7634', 'epoch': '0.01522'}
{'loss': '0.9804', 'grad_norm': '0.3695', 'learning_rate': '0.0001953', 'entropy': '0.9837', 'num_tokens': '9.333e+06', 'mean_token_accuracy': '0.7532', 'epoch': '0.01523'}
{'loss': '0.8312', 'grad_norm': '0.3288', 'learning_rate': '0.0001953', 'entropy': '0.8437', 'num_tokens': '9.337e+06', 'mean_token_accuracy': '0.7879', 'epoch': '0.01524'}
{'loss': '0.8272', 'grad_norm': '0.4085', 'learning_rate': '0.0001953', 'entropy': '0.8245', 'num_tokens': '9.342e+06', 'mean_token_accuracy': '0.7813', 'epoch': '0.01525'}
{'loss': '0.7986', 'grad_norm': '0.2976', 'learning_rate': '0.0001952', 'entropy': '0.8266', 'num_tokens': '9.346e+06', 'mean_token_accuracy': '0.7855', 'epoch': '0.01526'}
{'loss': '1.454', 'grad_norm': '0.3562', 'learning_rate': '0.0001952', 'entropy': '1.121', 'num_tokens': '9.355e+06', 'mean_token_accuracy': '0.7289', 'epoch': '0.01527'}
{'loss': '0.7827', 'grad_norm': '0.4013', 'learning_rate': '0.0001952', 'entropy': '0.7679', 'num_tokens': '9.36e+06', 'mean_token_accuracy': '0.7973', 'epoch': '0.01527'}
{'loss': '0.9135', 'grad_norm': '0.4001', 'learning_rate': '0.0001952', 'entropy': '0.9229', 'num_tokens': '9.365e+06', 'mean_token_accuracy': '0.7681', 'epoch': '0.01528'}
{'loss': '0.9281', 'grad_norm': '0.3344', 'learning_rate': '0.0001952', 'entropy': '0.9213', 'num_tokens': '9.372e+06', 'mean_token_accuracy': '0.7615', 'epoch': '0.01529'}
{'loss': '0.8004', 'grad_norm': '0.369', 'learning_rate': '0.0001952', 'entropy': '0.8647', 'num_tokens': '9.376e+06', 'mean_token_accuracy': '0.7763', 'epoch': '0.0153'}
{'loss': '1.173', 'grad_norm': '0.3759', 'learning_rate': '0.0001952', 'entropy': '1.162', 'num_tokens': '9.381e+06', 'mean_token_accuracy': '0.7156', 'epoch': '0.01531'}
{'loss': '1.018', 'grad_norm': '0.3946', 'learning_rate': '0.0001952', 'entropy': '1.013', 'num_tokens': '9.388e+06', 'mean_token_accuracy': '0.757', 'epoch': '0.01532'}
{'loss': '0.7831', 'grad_norm': '0.3266', 'learning_rate': '0.0001952', 'entropy': '0.8219', 'num_tokens': '9.393e+06', 'mean_token_accuracy': '0.8044', 'epoch': '0.01533'}
{'loss': '0.8368', 'grad_norm': '0.3698', 'learning_rate': '0.0001952', 'entropy': '0.8107', 'num_tokens': '9.399e+06', 'mean_token_accuracy': '0.8089', 'epoch': '0.01534'}
{'loss': '1.106', 'grad_norm': '0.3267', 'learning_rate': '0.0001952', 'entropy': '0.9639', 'num_tokens': '9.405e+06', 'mean_token_accuracy': '0.7668', 'epoch': '0.01535'}
{'loss': '1.175', 'grad_norm': '0.5127', 'learning_rate': '0.0001952', 'entropy': '1.024', 'num_tokens': '9.41e+06', 'mean_token_accuracy': '0.7563', 'epoch': '0.01535'}
{'loss': '0.9682', 'grad_norm': '0.4615', 'learning_rate': '0.0001951', 'entropy': '1.015', 'num_tokens': '9.414e+06', 'mean_token_accuracy': '0.7411', 'epoch': '0.01536'}
{'loss': '1.472', 'grad_norm': '0.47', 'learning_rate': '0.0001951', 'entropy': '1.457', 'num_tokens': '9.418e+06', 'mean_token_accuracy': '0.68', 'epoch': '0.01537'}
{'loss': '0.6815', 'grad_norm': '0.265', 'learning_rate': '0.0001951', 'entropy': '0.6724', 'num_tokens': '9.423e+06', 'mean_token_accuracy': '0.8388', 'epoch': '0.01538'}
{'loss': '0.663', 'grad_norm': '0.33', 'learning_rate': '0.0001951', 'entropy': '0.6666', 'num_tokens': '9.428e+06', 'mean_token_accuracy': '0.8214', 'epoch': '0.01539'}
{'loss': '1.159', 'grad_norm': '0.3275', 'learning_rate': '0.0001951', 'entropy': '1.258', 'num_tokens': '9.436e+06', 'mean_token_accuracy': '0.6996', 'epoch': '0.0154'}
{'loss': '1.045', 'grad_norm': '0.4158', 'learning_rate': '0.0001951', 'entropy': '1.055', 'num_tokens': '9.44e+06', 'mean_token_accuracy': '0.739', 'epoch': '0.01541'}
{'loss': '0.7996', 'grad_norm': '0.3628', 'learning_rate': '0.0001951', 'entropy': '0.8277', 'num_tokens': '9.445e+06', 'mean_token_accuracy': '0.7938', 'epoch': '0.01542'}

 15%|█▍        | 1743/12000 [1:02:46<5:31:52,  1.94s/it]
 15%|█▍        | 1744/12000 [1:02:49<6:51:30,  2.41s/it]
                                                        

 15%|█▍        | 1744/12000 [1:02:49<6:51:30,  2.41s/it]
 15%|█▍        | 1745/12000 [1:02:53<8:03:39,  2.83s/it]
                                                        

 15%|█▍        | 1745/12000 [1:02:53<8:03:39,  2.83s/it]
 15%|█▍        | 1746/12000 [1:02:54<6:54:10,  2.42s/it]
                                                        

 15%|█▍        | 1746/12000 [1:02:54<6:54:10,  2.42s/it]
 15%|█▍        | 1747/12000 [1:02:56<6:17:44,  2.21s/it]
                                                        

 15%|█▍        | 1747/12000 [1:02:56<6:17:44,  2.21s/it]
 15%|█▍        | 1748/12000 [1:02:58<6:25:23,  2.26s/it]
                                                        

 15%|█▍        | 1748/12000 [1:02:58<6:25:23,  2.26s/it]
 15%|█▍        | 1749/12000 [1:03:00<6:04:22,  2.13s/it]
                                                        

 15%|█▍        | 1749/12000 [1:03:00<6:04:22,  2.13s/it]
 15%|█▍        | 1750/12000 [1:03:02<5:33:13,  1.95s/it]
                                                        

 15%|█▍        | 1750/12000 [1:03:02<5:33:13,  1.95s/it]
 15%|█▍        | 1751/12000 [1:03:05<6:40:09,  2.34s/it]
                                                        

 15%|█▍        | 1751/12000 [1:03:05<6:40:09,  2.34s/it]
 15%|█▍        | 1752/12000 [1:03:08<7:04:20,  2.48s/it]
                                                        

 15%|█▍        | 1752/12000 [1:03:08<7:04:20,  2.48s/it]
 15%|█▍        | 1753/12000 [1:03:09<6:08:04,  2.16s/it]
                                                        

 15%|█▍        | 1753/12000 [1:03:09<6:08:04,  2.16s/it]
 15%|█▍        | 1754/12000 [1:03:11<5:33:23,  1.95s/it]
                                                        

 15%|█▍        | 1754/12000 [1:03:11<5:33:23,  1.95s/it]
 15%|█▍        | 1755/12000 [1:03:12<5:19:26,  1.87s/it]
                                                        

 15%|█▍        | 1755/12000 [1:03:12<5:19:26,  1.87s/it]
 15%|█▍        | 1756/12000 [1:03:14<5:00:33,  1.76s/it]
                                                        

 15%|█▍        | 1756/12000 [1:03:14<5:00:33,  1.76s/it]
 15%|█▍        | 1757/12000 [1:03:17<6:22:04,  2.24s/it]
                                                        

 15%|█▍        | 1757/12000 [1:03:17<6:22:04,  2.24s/it]
 15%|█▍        | 1758/12000 [1:03:21<7:15:19,  2.55s/it]
                                                        

 15%|█▍        | 1758/12000 [1:03:21<7:15:19,  2.55s/it]
 15%|█▍        | 1759/12000 [1:03:22<6:17:46,  2.21s/it]
                                                        

 15%|█▍        | 1759/12000 [1:03:22<6:17:46,  2.21s/it]
 15%|█▍        | 1760/12000 [1:03:24<5:45:11,  2.02s/it]
                                                        

 15%|█▍        | 1760/12000 [1:03:24<5:45:11,  2.02s/it]
 15%|█▍        | 1761/12000 [1:03:25<5:33:24,  1.95s/it]
                                                        

 15%|█▍        | 1761/12000 [1:03:25<5:33:24,  1.95s/it]
 15%|█▍        | 1762/12000 [1:03:27<5:16:12,  1.85s/it]
                                                        

 15%|█▍        | 1762/12000 [1:03:27<5:16:12,  1.85s/it]
 15%|█▍        | 1763/12000 [1:03:29<5:42:09,  2.01s/it]
                                                        

 15%|█▍        | 1763/12000 [1:03:29<5:42:09,  2.01s/it]
 15%|█▍        | 1764/12000 [1:03:33<7:07:24,  2.51s/it]
                                                        

 15%|█▍        | 1764/12000 [1:03:33<7:07:24,  2.51s/it]
 15%|█▍        | 1765/12000 [1:03:36<7:08:08,  2.51s/it]
                                                        

 15%|█▍        | 1765/12000 [1:03:36<7:08:08,  2.51s/it]
 15%|█▍        | 1766/12000 [1:03:37<6:25:42,  2.26s/it]
                                                        

 15%|█▍        | 1766/12000 [1:03:37<6:25:42,  2.26s/it]
 15%|█▍        | 1767/12000 [1:03:39<6:16:20,  2.21s/it]
                                                        

 15%|█▍        | 1767/12000 [1:03:39<6:16:20,  2.21s/it]
 15%|█▍        | 1768/12000 [1:03:42<6:46:59,  2.39s/it]
                                                        

 15%|█▍        | 1768/12000 [1:03:42<6:46:59,  2.39s/it]
 15%|█▍        | 1769/12000 [1:03:46<7:42:44,  2.71s/it]
                                                        

 15%|█▍        | 1769/12000 [1:03:46<7:42:44,  2.71s/it]
 15%|█▍        | 1770/12000 [1:03:48<7:30:24,  2.64s/it]
                                                        

 15%|█▍        | 1770/12000 [1:03:48<7:30:24,  2.64s/it]
 15%|█▍        | 1771/12000 [1:03:56<11:55:12,  4.20s/it]
                                                         

 15%|█▍        | 1771/12000 [1:03:56<11:55:12,  4.20s/it]
 15%|█▍        | 1772/12000 [1:04:00<12:12:37,  4.30s/it]
                                                         

 15%|█▍        | 1772/12000 [1:04:00<12:12:37,  4.30s/it]
 15%|█▍        | 1773/12000 [1:04:02<9:58:00,  3.51s/it] 
                                                        

 15%|█▍        | 1773/12000 [1:04:02<9:58:00,  3.51s/it]
 15%|█▍        | 1774/12000 [1:04:04<8:14:55,  2.90s/it]
                                                        

 15%|█▍        | 1774/12000 [1:04:04<8:14:55,  2.90s/it]
 15%|█▍        | 1775/12000 [1:04:05<7:09:38,  2.52s/it]
                                                        

 15%|█▍        | 1775/12000 [1:04:05<7:09:38,  2.52s/it]
 15%|█▍        | 1776/12000 [1:04:07<6:31:07,  2.30s/it]
                                                        

 15%|█▍        | 1776/12000 [1:04:07<6:31:07,  2.30s/it]
 15%|█▍        | 1777/12000 [1:04:09<5:53:34,  2.08s/it]
                                                        

 15%|█▍        | 1777/12000 [1:04:09<5:53:34,  2.08s/it]
 15%|█▍        | 1778/12000 [1:04:11<6:18:56,  2.22s/it]
                                                        

 15%|█▍        | 1778/12000 [1:04:11<6:18:56,  2.22s/it]
 15%|█▍        | 1779/12000 [1:04:16<8:17:47,  2.92s/it]
                                                        

 15%|█▍        | 1779/12000 [1:04:16<8:17:47,  2.92s/it]
 15%|█▍        | 1780/12000 [1:04:17<6:57:33,  2.45s/it]
                                                        

 15%|█▍        | 1780/12000 [1:04:17<6:57:33,  2.45s/it]
 15%|█▍        | 1781/12000 [1:04:19<6:35:43,  2.32s/it]
                                                        

 15%|█▍        | 1781/12000 [1:04:19<6:35:43,  2.32s/it]
 15%|█▍        | 1782/12000 [1:04:21<6:12:29,  2.19s/it]
                                                        

 15%|█▍        | 1782/12000 [1:04:21<6:12:29,  2.19s/it]
 15%|█▍        | 1783/12000 [1:04:22<5:34:40,  1.97s/it]
                                                        

 15%|█▍        | 1783/12000 [1:04:22<5:34:40,  1.97s/it]
 15%|█▍        | 1784/12000 [1:04:24<5:00:29,  1.76s/it]
                                                        

 15%|█▍        | 1784/12000 [1:04:24<5:00:29,  1.76s/it]
 15%|█▍        | 1785/12000 [1:04:25<4:46:41,  1.68s/it]
                                                        

 15%|█▍        | 1785/12000 [1:04:25<4:46:41,  1.68s/it]
 15%|█▍        | 1786/12000 [1:04:27<4:45:51,  1.68s/it]
                                                        

 15%|█▍        | 1786/12000 [1:04:27<4:45:51,  1.68s/it]
 15%|█▍        | 1787/12000 [1:04:30<6:01:34,  2.12s/it]
                                                        

 15%|█▍        | 1787/12000 [1:04:30<6:01:34,  2.12s/it]
 15%|█▍        | 1788/12000 [1:04:33<6:30:08,  2.29s/it]
                                                        

 15%|█▍        | 1788/12000 [1:04:33<6:30:08,  2.29s/it]
 15%|█▍        | 1789/12000 [1:04:36<7:35:24,  2.68s/it]
                                                        

 15%|█▍        | 1789/12000 [1:04:36<7:35:24,  2.68s/it]
 15%|█▍        | 1790/12000 [1:04:38<6:41:47,  2.36s/it]
                                                        
{'loss': '0.9258', 'grad_norm': '0.4099', 'learning_rate': '0.0001951', 'entropy': '0.981', 'num_tokens': '9.45e+06', 'mean_token_accuracy': '0.7545', 'epoch': '0.01543'}
{'loss': '0.8197', 'grad_norm': '0.2981', 'learning_rate': '0.0001951', 'entropy': '0.8178', 'num_tokens': '9.458e+06', 'mean_token_accuracy': '0.8051', 'epoch': '0.01543'}
{'loss': '0.5766', 'grad_norm': '0.2411', 'learning_rate': '0.0001951', 'entropy': '0.5451', 'num_tokens': '9.465e+06', 'mean_token_accuracy': '0.8551', 'epoch': '0.01544'}
{'loss': '0.9504', 'grad_norm': '0.359', 'learning_rate': '0.0001951', 'entropy': '0.9256', 'num_tokens': '9.469e+06', 'mean_token_accuracy': '0.7736', 'epoch': '0.01545'}
{'loss': '0.7714', 'grad_norm': '0.3541', 'learning_rate': '0.0001951', 'entropy': '0.7726', 'num_tokens': '9.474e+06', 'mean_token_accuracy': '0.7972', 'epoch': '0.01546'}
{'loss': '0.4677', 'grad_norm': '0.2831', 'learning_rate': '0.000195', 'entropy': '0.5595', 'num_tokens': '9.48e+06', 'mean_token_accuracy': '0.8602', 'epoch': '0.01547'}
{'loss': '0.7939', 'grad_norm': '0.3798', 'learning_rate': '0.000195', 'entropy': '0.8517', 'num_tokens': '9.486e+06', 'mean_token_accuracy': '0.7889', 'epoch': '0.01548'}
{'loss': '1.009', 'grad_norm': '0.3941', 'learning_rate': '0.000195', 'entropy': '1.058', 'num_tokens': '9.491e+06', 'mean_token_accuracy': '0.7396', 'epoch': '0.01549'}
{'loss': '0.785', 'grad_norm': '0.3812', 'learning_rate': '0.000195', 'entropy': '0.7899', 'num_tokens': '9.497e+06', 'mean_token_accuracy': '0.8124', 'epoch': '0.0155'}
{'loss': '0.9474', 'grad_norm': '0.3632', 'learning_rate': '0.000195', 'entropy': '0.7917', 'num_tokens': '9.503e+06', 'mean_token_accuracy': '0.8066', 'epoch': '0.0155'}
{'loss': '0.9228', 'grad_norm': '0.4345', 'learning_rate': '0.000195', 'entropy': '0.9777', 'num_tokens': '9.507e+06', 'mean_token_accuracy': '0.767', 'epoch': '0.01551'}
{'loss': '0.7028', 'grad_norm': '0.3357', 'learning_rate': '0.000195', 'entropy': '0.7342', 'num_tokens': '9.511e+06', 'mean_token_accuracy': '0.8163', 'epoch': '0.01552'}
{'loss': '0.8549', 'grad_norm': '0.3461', 'learning_rate': '0.000195', 'entropy': '0.8263', 'num_tokens': '9.517e+06', 'mean_token_accuracy': '0.8002', 'epoch': '0.01553'}
{'loss': '0.9854', 'grad_norm': '0.4212', 'learning_rate': '0.000195', 'entropy': '1.059', 'num_tokens': '9.521e+06', 'mean_token_accuracy': '0.7424', 'epoch': '0.01554'}
{'loss': '0.6443', 'grad_norm': '0.3063', 'learning_rate': '0.000195', 'entropy': '0.7004', 'num_tokens': '9.524e+06', 'mean_token_accuracy': '0.8236', 'epoch': '0.01555'}
{'loss': '0.9335', 'grad_norm': '0.3394', 'learning_rate': '0.000195', 'entropy': '0.8911', 'num_tokens': '9.532e+06', 'mean_token_accuracy': '0.7886', 'epoch': '0.01556'}
{'loss': '0.9291', 'grad_norm': '0.4862', 'learning_rate': '0.000195', 'entropy': '0.9867', 'num_tokens': '9.537e+06', 'mean_token_accuracy': '0.7399', 'epoch': '0.01557'}
{'loss': '1.014', 'grad_norm': '0.4414', 'learning_rate': '0.0001949', 'entropy': '1.054', 'num_tokens': '9.541e+06', 'mean_token_accuracy': '0.7386', 'epoch': '0.01558'}
{'loss': '0.9534', 'grad_norm': '0.3454', 'learning_rate': '0.0001949', 'entropy': '0.9104', 'num_tokens': '9.545e+06', 'mean_token_accuracy': '0.7602', 'epoch': '0.01558'}
{'loss': '0.673', 'grad_norm': '0.3327', 'learning_rate': '0.0001949', 'entropy': '0.6564', 'num_tokens': '9.551e+06', 'mean_token_accuracy': '0.8184', 'epoch': '0.01559'}
{'loss': '1.02', 'grad_norm': '0.4037', 'learning_rate': '0.0001949', 'entropy': '1.051', 'num_tokens': '9.556e+06', 'mean_token_accuracy': '0.7481', 'epoch': '0.0156'}
{'loss': '0.8325', 'grad_norm': '0.2918', 'learning_rate': '0.0001949', 'entropy': '0.8347', 'num_tokens': '9.564e+06', 'mean_token_accuracy': '0.7845', 'epoch': '0.01561'}
{'loss': '1.008', 'grad_norm': '0.3375', 'learning_rate': '0.0001949', 'entropy': '1.016', 'num_tokens': '9.571e+06', 'mean_token_accuracy': '0.7347', 'epoch': '0.01562'}
{'loss': '0.8084', 'grad_norm': '0.4111', 'learning_rate': '0.0001949', 'entropy': '0.864', 'num_tokens': '9.577e+06', 'mean_token_accuracy': '0.7832', 'epoch': '0.01563'}
{'loss': '0.8587', 'grad_norm': '0.3542', 'learning_rate': '0.0001949', 'entropy': '0.9261', 'num_tokens': '9.582e+06', 'mean_token_accuracy': '0.7729', 'epoch': '0.01564'}
{'loss': '0.9315', 'grad_norm': '0.3168', 'learning_rate': '0.0001949', 'entropy': '1.051', 'num_tokens': '9.588e+06', 'mean_token_accuracy': '0.7357', 'epoch': '0.01565'}
{'loss': '0.7402', 'grad_norm': '0.2845', 'learning_rate': '0.0001949', 'entropy': '0.8261', 'num_tokens': '9.597e+06', 'mean_token_accuracy': '0.8069', 'epoch': '0.01566'}
{'loss': '0.8095', 'grad_norm': '0.3169', 'learning_rate': '0.0001949', 'entropy': '0.7979', 'num_tokens': '9.603e+06', 'mean_token_accuracy': '0.7978', 'epoch': '0.01566'}
{'loss': '1.764', 'grad_norm': '0.3344', 'learning_rate': '0.0001948', 'entropy': '1.509', 'num_tokens': '9.615e+06', 'mean_token_accuracy': '0.6862', 'epoch': '0.01567'}
{'loss': '1.021', 'grad_norm': '0.2921', 'learning_rate': '0.0001948', 'entropy': '0.9826', 'num_tokens': '9.625e+06', 'mean_token_accuracy': '0.7624', 'epoch': '0.01568'}
{'loss': '1.041', 'grad_norm': '0.3928', 'learning_rate': '0.0001948', 'entropy': '1.051', 'num_tokens': '9.629e+06', 'mean_token_accuracy': '0.7396', 'epoch': '0.01569'}
{'loss': '0.9097', 'grad_norm': '0.3616', 'learning_rate': '0.0001948', 'entropy': '0.9932', 'num_tokens': '9.634e+06', 'mean_token_accuracy': '0.7522', 'epoch': '0.0157'}
{'loss': '0.9565', 'grad_norm': '0.4189', 'learning_rate': '0.0001948', 'entropy': '0.8987', 'num_tokens': '9.639e+06', 'mean_token_accuracy': '0.7825', 'epoch': '0.01571'}
{'loss': '0.9709', 'grad_norm': '0.3497', 'learning_rate': '0.0001948', 'entropy': '1.015', 'num_tokens': '9.644e+06', 'mean_token_accuracy': '0.7514', 'epoch': '0.01572'}
{'loss': '0.8725', 'grad_norm': '0.3649', 'learning_rate': '0.0001948', 'entropy': '0.9048', 'num_tokens': '9.649e+06', 'mean_token_accuracy': '0.7805', 'epoch': '0.01573'}
{'loss': '1.41', 'grad_norm': '0.3484', 'learning_rate': '0.0001948', 'entropy': '1.199', 'num_tokens': '9.655e+06', 'mean_token_accuracy': '0.7199', 'epoch': '0.01574'}
{'loss': '1.046', 'grad_norm': '0.3662', 'learning_rate': '0.0001948', 'entropy': '0.9231', 'num_tokens': '9.663e+06', 'mean_token_accuracy': '0.7712', 'epoch': '0.01574'}
{'loss': '0.8174', 'grad_norm': '0.4262', 'learning_rate': '0.0001948', 'entropy': '0.8452', 'num_tokens': '9.666e+06', 'mean_token_accuracy': '0.7972', 'epoch': '0.01575'}
{'loss': '0.7482', 'grad_norm': '0.3327', 'learning_rate': '0.0001948', 'entropy': '0.7088', 'num_tokens': '9.672e+06', 'mean_token_accuracy': '0.8146', 'epoch': '0.01576'}
{'loss': '0.7742', 'grad_norm': '0.4309', 'learning_rate': '0.0001948', 'entropy': '0.8538', 'num_tokens': '9.676e+06', 'mean_token_accuracy': '0.7753', 'epoch': '0.01577'}
{'loss': '1.07', 'grad_norm': '0.3955', 'learning_rate': '0.0001947', 'entropy': '1.098', 'num_tokens': '9.681e+06', 'mean_token_accuracy': '0.7378', 'epoch': '0.01578'}
{'loss': '0.9057', 'grad_norm': '0.4015', 'learning_rate': '0.0001947', 'entropy': '0.8743', 'num_tokens': '9.684e+06', 'mean_token_accuracy': '0.7785', 'epoch': '0.01579'}
{'loss': '1.21', 'grad_norm': '0.4057', 'learning_rate': '0.0001947', 'entropy': '1.175', 'num_tokens': '9.688e+06', 'mean_token_accuracy': '0.7247', 'epoch': '0.0158'}
{'loss': '0.769', 'grad_norm': '0.4387', 'learning_rate': '0.0001947', 'entropy': '0.7853', 'num_tokens': '9.693e+06', 'mean_token_accuracy': '0.7986', 'epoch': '0.01581'}
{'loss': '0.9034', 'grad_norm': '0.2866', 'learning_rate': '0.0001947', 'entropy': '1.035', 'num_tokens': '9.699e+06', 'mean_token_accuracy': '0.7429', 'epoch': '0.01581'}
{'loss': '1.547', 'grad_norm': '0.4108', 'learning_rate': '0.0001947', 'entropy': '1.402', 'num_tokens': '9.706e+06', 'mean_token_accuracy': '0.682', 'epoch': '0.01582'}
{'loss': '0.9481', 'grad_norm': '0.2876', 'learning_rate': '0.0001947', 'entropy': '1.067', 'num_tokens': '9.714e+06', 'mean_token_accuracy': '0.7391', 'epoch': '0.01583'}

 15%|█▍        | 1790/12000 [1:04:38<6:41:47,  2.36s/it]
 15%|█▍        | 1791/12000 [1:04:41<7:13:08,  2.55s/it]
                                                        

 15%|█▍        | 1791/12000 [1:04:41<7:13:08,  2.55s/it]
 15%|█▍        | 1792/12000 [1:04:43<6:39:06,  2.35s/it]
                                                        

 15%|█▍        | 1792/12000 [1:04:43<6:39:06,  2.35s/it]
 15%|█▍        | 1793/12000 [1:04:44<6:07:33,  2.16s/it]
                                                        

 15%|█▍        | 1793/12000 [1:04:44<6:07:33,  2.16s/it]
 15%|█▍        | 1794/12000 [1:04:46<5:31:05,  1.95s/it]
                                                        

 15%|█▍        | 1794/12000 [1:04:46<5:31:05,  1.95s/it]
 15%|█▍        | 1795/12000 [1:04:48<5:42:12,  2.01s/it]
                                                        

 15%|█▍        | 1795/12000 [1:04:48<5:42:12,  2.01s/it]
 15%|█▍        | 1796/12000 [1:04:50<5:29:06,  1.94s/it]
                                                        

 15%|█▍        | 1796/12000 [1:04:50<5:29:06,  1.94s/it]
 15%|█▍        | 1797/12000 [1:04:51<5:05:56,  1.80s/it]
                                                        

 15%|█▍        | 1797/12000 [1:04:51<5:05:56,  1.80s/it]
 15%|█▍        | 1798/12000 [1:04:54<6:14:58,  2.21s/it]
                                                        

 15%|█▍        | 1798/12000 [1:04:54<6:14:58,  2.21s/it]
 15%|█▍        | 1799/12000 [1:04:56<5:48:45,  2.05s/it]
                                                        

 15%|█▍        | 1799/12000 [1:04:56<5:48:45,  2.05s/it]
 15%|█▌        | 1800/12000 [1:04:58<5:31:25,  1.95s/it]
                                                        

 15%|█▌        | 1800/12000 [1:04:58<5:31:25,  1.95s/it]
 15%|█▌        | 1801/12000 [1:05:03<8:31:38,  3.01s/it]
                                                        

 15%|█▌        | 1801/12000 [1:05:03<8:31:38,  3.01s/it]
 15%|█▌        | 1802/12000 [1:05:06<8:24:48,  2.97s/it]
                                                        

 15%|█▌        | 1802/12000 [1:05:06<8:24:48,  2.97s/it]
 15%|█▌        | 1803/12000 [1:05:08<7:15:47,  2.56s/it]
                                                        

 15%|█▌        | 1803/12000 [1:05:08<7:15:47,  2.56s/it]
 15%|█▌        | 1804/12000 [1:05:10<6:34:08,  2.32s/it]
                                                        

 15%|█▌        | 1804/12000 [1:05:10<6:34:08,  2.32s/it]
 15%|█▌        | 1805/12000 [1:05:15<9:12:14,  3.25s/it]
                                                        

 15%|█▌        | 1805/12000 [1:05:15<9:12:14,  3.25s/it]
 15%|█▌        | 1806/12000 [1:05:17<8:01:32,  2.83s/it]
                                                        

 15%|█▌        | 1806/12000 [1:05:17<8:01:32,  2.83s/it]
 15%|█▌        | 1807/12000 [1:05:19<7:03:24,  2.49s/it]
                                                        

 15%|█▌        | 1807/12000 [1:05:19<7:03:24,  2.49s/it]
 15%|█▌        | 1808/12000 [1:05:20<6:08:06,  2.17s/it]
                                                        

 15%|█▌        | 1808/12000 [1:05:20<6:08:06,  2.17s/it]
 15%|█▌        | 1809/12000 [1:05:23<6:53:31,  2.43s/it]
                                                        

 15%|█▌        | 1809/12000 [1:05:23<6:53:31,  2.43s/it]
 15%|█▌        | 1810/12000 [1:05:25<6:50:11,  2.42s/it]
                                                        

 15%|█▌        | 1810/12000 [1:05:25<6:50:11,  2.42s/it]
 15%|█▌        | 1811/12000 [1:05:27<6:04:08,  2.14s/it]
                                                        

 15%|█▌        | 1811/12000 [1:05:27<6:04:08,  2.14s/it]
 15%|█▌        | 1812/12000 [1:05:30<6:51:07,  2.42s/it]
                                                        

 15%|█▌        | 1812/12000 [1:05:30<6:51:07,  2.42s/it]
 15%|█▌        | 1813/12000 [1:05:32<6:18:36,  2.23s/it]
                                                        

 15%|█▌        | 1813/12000 [1:05:32<6:18:36,  2.23s/it]
 15%|█▌        | 1814/12000 [1:05:34<6:00:57,  2.13s/it]
                                                        

 15%|█▌        | 1814/12000 [1:05:34<6:00:57,  2.13s/it]
 15%|█▌        | 1815/12000 [1:05:36<6:18:55,  2.23s/it]
                                                        

 15%|█▌        | 1815/12000 [1:05:36<6:18:55,  2.23s/it]
 15%|█▌        | 1816/12000 [1:05:37<5:34:33,  1.97s/it]
                                                        

 15%|█▌        | 1816/12000 [1:05:37<5:34:33,  1.97s/it]
 15%|█▌        | 1817/12000 [1:05:40<6:13:45,  2.20s/it]
                                                        

 15%|█▌        | 1817/12000 [1:05:40<6:13:45,  2.20s/it]
 15%|█▌        | 1818/12000 [1:05:42<5:38:51,  2.00s/it]
                                                        

 15%|█▌        | 1818/12000 [1:05:42<5:38:51,  2.00s/it]
 15%|█▌        | 1819/12000 [1:05:45<6:43:32,  2.38s/it]
                                                        

 15%|█▌        | 1819/12000 [1:05:45<6:43:32,  2.38s/it]
 15%|█▌        | 1820/12000 [1:05:48<7:01:22,  2.48s/it]
                                                        

 15%|█▌        | 1820/12000 [1:05:48<7:01:22,  2.48s/it]
 15%|█▌        | 1821/12000 [1:05:49<6:05:35,  2.15s/it]
                                                        

 15%|█▌        | 1821/12000 [1:05:49<6:05:35,  2.15s/it]
 15%|█▌        | 1822/12000 [1:05:52<7:05:17,  2.51s/it]
                                                        

 15%|█▌        | 1822/12000 [1:05:52<7:05:17,  2.51s/it]
 15%|█▌        | 1823/12000 [1:05:56<7:51:46,  2.78s/it]
                                                        

 15%|█▌        | 1823/12000 [1:05:56<7:51:46,  2.78s/it]
 15%|█▌        | 1824/12000 [1:05:59<8:03:17,  2.85s/it]
                                                        

 15%|█▌        | 1824/12000 [1:05:59<8:03:17,  2.85s/it]
 15%|█▌        | 1825/12000 [1:06:01<7:06:01,  2.51s/it]
                                                        

 15%|█▌        | 1825/12000 [1:06:01<7:06:01,  2.51s/it]
 15%|█▌        | 1826/12000 [1:06:02<6:18:09,  2.23s/it]
                                                        

 15%|█▌        | 1826/12000 [1:06:02<6:18:09,  2.23s/it]
 15%|█▌        | 1827/12000 [1:06:04<5:58:34,  2.11s/it]
                                                        

 15%|█▌        | 1827/12000 [1:06:04<5:58:34,  2.11s/it]
 15%|█▌        | 1828/12000 [1:06:06<5:45:34,  2.04s/it]
                                                        

 15%|█▌        | 1828/12000 [1:06:06<5:45:34,  2.04s/it]
 15%|█▌        | 1829/12000 [1:06:08<5:43:36,  2.03s/it]
                                                        

 15%|█▌        | 1829/12000 [1:06:08<5:43:36,  2.03s/it]
 15%|█▌        | 1830/12000 [1:06:09<5:10:19,  1.83s/it]
                                                        

 15%|█▌        | 1830/12000 [1:06:09<5:10:19,  1.83s/it]
 15%|█▌        | 1831/12000 [1:06:11<5:24:33,  1.91s/it]
                                                        

 15%|█▌        | 1831/12000 [1:06:11<5:24:33,  1.91s/it]
 15%|█▌        | 1832/12000 [1:06:14<6:13:26,  2.20s/it]
                                                        

 15%|█▌        | 1832/12000 [1:06:14<6:13:26,  2.20s/it]
 15%|█▌        | 1833/12000 [1:06:16<6:06:48,  2.16s/it]
                                                        

 15%|█▌        | 1833/12000 [1:06:16<6:06:48,  2.16s/it]
 15%|█▌        | 1834/12000 [1:06:18<5:40:18,  2.01s/it]
                                                        

 15%|█▌        | 1834/12000 [1:06:18<5:40:18,  2.01s/it]
 15%|█▌        | 1835/12000 [1:06:21<6:45:29,  2.39s/it]
                                                        

 15%|█▌        | 1835/12000 [1:06:21<6:45:29,  2.39s/it]
 15%|█▌        | 1836/12000 [1:06:23<5:54:39,  2.09s/it]
                                                        

 15%|█▌        | 1836/12000 [1:06:23<5:54:39,  2.09s/it]
 15%|█▌        | 1837/12000 [1:06:24<5:35:38,  1.98s/it]
                                                        
{'loss': '0.9671', 'grad_norm': '0.3847', 'learning_rate': '0.0001947', 'entropy': '1.008', 'num_tokens': '9.718e+06', 'mean_token_accuracy': '0.7639', 'epoch': '0.01584'}
{'loss': '1.196', 'grad_norm': '0.3557', 'learning_rate': '0.0001947', 'entropy': '1.215', 'num_tokens': '9.725e+06', 'mean_token_accuracy': '0.7179', 'epoch': '0.01585'}
{'loss': '1.018', 'grad_norm': '0.3462', 'learning_rate': '0.0001947', 'entropy': '1.069', 'num_tokens': '9.731e+06', 'mean_token_accuracy': '0.7562', 'epoch': '0.01586'}
{'loss': '1.199', 'grad_norm': '0.4108', 'learning_rate': '0.0001947', 'entropy': '1.16', 'num_tokens': '9.737e+06', 'mean_token_accuracy': '0.717', 'epoch': '0.01587'}
{'loss': '1.076', 'grad_norm': '0.4114', 'learning_rate': '0.0001946', 'entropy': '1.118', 'num_tokens': '9.741e+06', 'mean_token_accuracy': '0.733', 'epoch': '0.01588'}
{'loss': '0.832', 'grad_norm': '0.3719', 'learning_rate': '0.0001946', 'entropy': '0.7931', 'num_tokens': '9.746e+06', 'mean_token_accuracy': '0.8016', 'epoch': '0.01589'}
{'loss': '0.8784', 'grad_norm': '0.3998', 'learning_rate': '0.0001946', 'entropy': '0.8349', 'num_tokens': '9.751e+06', 'mean_token_accuracy': '0.7777', 'epoch': '0.01589'}
{'loss': '0.8821', 'grad_norm': '0.3478', 'learning_rate': '0.0001946', 'entropy': '0.8469', 'num_tokens': '9.756e+06', 'mean_token_accuracy': '0.7808', 'epoch': '0.0159'}
{'loss': '0.8741', 'grad_norm': '0.3583', 'learning_rate': '0.0001946', 'entropy': '0.8638', 'num_tokens': '9.764e+06', 'mean_token_accuracy': '0.7759', 'epoch': '0.01591'}
{'loss': '0.8567', 'grad_norm': '0.4072', 'learning_rate': '0.0001946', 'entropy': '0.905', 'num_tokens': '9.768e+06', 'mean_token_accuracy': '0.7682', 'epoch': '0.01592'}
{'loss': '0.6711', 'grad_norm': '0.3269', 'learning_rate': '0.0001946', 'entropy': '0.6864', 'num_tokens': '9.773e+06', 'mean_token_accuracy': '0.8224', 'epoch': '0.01593'}
{'loss': '0.6505', 'grad_norm': '0.3372', 'learning_rate': '0.0001946', 'entropy': '0.6383', 'num_tokens': '9.779e+06', 'mean_token_accuracy': '0.8368', 'epoch': '0.01594'}
{'loss': '0.821', 'grad_norm': '0.321', 'learning_rate': '0.0001946', 'entropy': '0.8808', 'num_tokens': '9.786e+06', 'mean_token_accuracy': '0.774', 'epoch': '0.01595'}
{'loss': '0.7899', 'grad_norm': '0.318', 'learning_rate': '0.0001946', 'entropy': '0.8251', 'num_tokens': '9.791e+06', 'mean_token_accuracy': '0.7932', 'epoch': '0.01596'}
{'loss': '0.9421', 'grad_norm': '0.3336', 'learning_rate': '0.0001946', 'entropy': '0.9302', 'num_tokens': '9.796e+06', 'mean_token_accuracy': '0.7695', 'epoch': '0.01597'}
{'loss': '1.293', 'grad_norm': '0.3798', 'learning_rate': '0.0001945', 'entropy': '1.127', 'num_tokens': '9.806e+06', 'mean_token_accuracy': '0.7312', 'epoch': '0.01597'}
{'loss': '1.083', 'grad_norm': '0.3617', 'learning_rate': '0.0001945', 'entropy': '1.021', 'num_tokens': '9.811e+06', 'mean_token_accuracy': '0.7486', 'epoch': '0.01598'}
{'loss': '0.6894', 'grad_norm': '0.305', 'learning_rate': '0.0001945', 'entropy': '0.6888', 'num_tokens': '9.817e+06', 'mean_token_accuracy': '0.8345', 'epoch': '0.01599'}
{'loss': '1.134', 'grad_norm': '0.4101', 'learning_rate': '0.0001945', 'entropy': '1.207', 'num_tokens': '9.821e+06', 'mean_token_accuracy': '0.7238', 'epoch': '0.016'}
{'loss': '1.209', 'grad_norm': '0.4471', 'learning_rate': '0.0001945', 'entropy': '1.085', 'num_tokens': '9.828e+06', 'mean_token_accuracy': '0.7452', 'epoch': '0.01601'}
{'loss': '0.9214', 'grad_norm': '0.3565', 'learning_rate': '0.0001945', 'entropy': '0.994', 'num_tokens': '9.834e+06', 'mean_token_accuracy': '0.7568', 'epoch': '0.01602'}
{'loss': '1.151', 'grad_norm': '0.4389', 'learning_rate': '0.0001945', 'entropy': '1.159', 'num_tokens': '9.838e+06', 'mean_token_accuracy': '0.7544', 'epoch': '0.01603'}
{'loss': '0.7938', 'grad_norm': '0.2989', 'learning_rate': '0.0001945', 'entropy': '0.8416', 'num_tokens': '9.847e+06', 'mean_token_accuracy': '0.7944', 'epoch': '0.01604'}
{'loss': '0.8086', 'grad_norm': '0.4351', 'learning_rate': '0.0001945', 'entropy': '0.8496', 'num_tokens': '9.852e+06', 'mean_token_accuracy': '0.7873', 'epoch': '0.01604'}
{'loss': '0.8516', 'grad_norm': '0.4178', 'learning_rate': '0.0001945', 'entropy': '0.8142', 'num_tokens': '9.857e+06', 'mean_token_accuracy': '0.7947', 'epoch': '0.01605'}
{'loss': '1.095', 'grad_norm': '0.4054', 'learning_rate': '0.0001945', 'entropy': '1.12', 'num_tokens': '9.863e+06', 'mean_token_accuracy': '0.7431', 'epoch': '0.01606'}
{'loss': '0.8835', 'grad_norm': '0.38', 'learning_rate': '0.0001944', 'entropy': '0.8897', 'num_tokens': '9.866e+06', 'mean_token_accuracy': '0.7749', 'epoch': '0.01607'}
{'loss': '0.7747', 'grad_norm': '0.365', 'learning_rate': '0.0001944', 'entropy': '0.8619', 'num_tokens': '9.873e+06', 'mean_token_accuracy': '0.7876', 'epoch': '0.01608'}
{'loss': '0.6744', 'grad_norm': '0.402', 'learning_rate': '0.0001944', 'entropy': '0.7457', 'num_tokens': '9.877e+06', 'mean_token_accuracy': '0.8142', 'epoch': '0.01609'}
{'loss': '0.8951', 'grad_norm': '0.3338', 'learning_rate': '0.0001944', 'entropy': '0.8308', 'num_tokens': '9.885e+06', 'mean_token_accuracy': '0.7919', 'epoch': '0.0161'}
{'loss': '1.68', 'grad_norm': '0.4096', 'learning_rate': '0.0001944', 'entropy': '1.477', 'num_tokens': '9.891e+06', 'mean_token_accuracy': '0.6565', 'epoch': '0.01611'}
{'loss': '0.6319', 'grad_norm': '0.3115', 'learning_rate': '0.0001944', 'entropy': '0.6395', 'num_tokens': '9.895e+06', 'mean_token_accuracy': '0.8297', 'epoch': '0.01612'}
{'loss': '0.9508', 'grad_norm': '0.3647', 'learning_rate': '0.0001944', 'entropy': '0.952', 'num_tokens': '9.903e+06', 'mean_token_accuracy': '0.7678', 'epoch': '0.01612'}
{'loss': '1.026', 'grad_norm': '0.4819', 'learning_rate': '0.0001944', 'entropy': '1.033', 'num_tokens': '9.907e+06', 'mean_token_accuracy': '0.7407', 'epoch': '0.01613'}
{'loss': '1.208', 'grad_norm': '0.3356', 'learning_rate': '0.0001944', 'entropy': '1.222', 'num_tokens': '9.914e+06', 'mean_token_accuracy': '0.7146', 'epoch': '0.01614'}
{'loss': '0.7996', 'grad_norm': '0.4194', 'learning_rate': '0.0001944', 'entropy': '0.8779', 'num_tokens': '9.918e+06', 'mean_token_accuracy': '0.7947', 'epoch': '0.01615'}
{'loss': '0.9652', 'grad_norm': '0.3862', 'learning_rate': '0.0001944', 'entropy': '0.9983', 'num_tokens': '9.923e+06', 'mean_token_accuracy': '0.7691', 'epoch': '0.01616'}
{'loss': '0.9945', 'grad_norm': '0.3459', 'learning_rate': '0.0001943', 'entropy': '0.9964', 'num_tokens': '9.928e+06', 'mean_token_accuracy': '0.7626', 'epoch': '0.01617'}
{'loss': '1.27', 'grad_norm': '0.4126', 'learning_rate': '0.0001943', 'entropy': '1.191', 'num_tokens': '9.933e+06', 'mean_token_accuracy': '0.7191', 'epoch': '0.01618'}
{'loss': '0.9684', 'grad_norm': '0.388', 'learning_rate': '0.0001943', 'entropy': '1.028', 'num_tokens': '9.939e+06', 'mean_token_accuracy': '0.7627', 'epoch': '0.01619'}
{'loss': '0.9662', 'grad_norm': '0.3552', 'learning_rate': '0.0001943', 'entropy': '1.011', 'num_tokens': '9.942e+06', 'mean_token_accuracy': '0.7567', 'epoch': '0.0162'}
{'loss': '0.8068', 'grad_norm': '0.3266', 'learning_rate': '0.0001943', 'entropy': '0.7146', 'num_tokens': '9.948e+06', 'mean_token_accuracy': '0.8255', 'epoch': '0.0162'}
{'loss': '0.9062', 'grad_norm': '0.3302', 'learning_rate': '0.0001943', 'entropy': '1.029', 'num_tokens': '9.954e+06', 'mean_token_accuracy': '0.7388', 'epoch': '0.01621'}
{'loss': '1.082', 'grad_norm': '0.3663', 'learning_rate': '0.0001943', 'entropy': '1.174', 'num_tokens': '9.96e+06', 'mean_token_accuracy': '0.7267', 'epoch': '0.01622'}
{'loss': '0.5789', 'grad_norm': '0.3422', 'learning_rate': '0.0001943', 'entropy': '0.6018', 'num_tokens': '9.965e+06', 'mean_token_accuracy': '0.8425', 'epoch': '0.01623'}
{'loss': '0.9584', 'grad_norm': '0.2889', 'learning_rate': '0.0001943', 'entropy': '0.9439', 'num_tokens': '9.973e+06', 'mean_token_accuracy': '0.7635', 'epoch': '0.01624'}
{'loss': '1.045', 'grad_norm': '0.4137', 'learning_rate': '0.0001943', 'entropy': '0.986', 'num_tokens': '9.976e+06', 'mean_token_accuracy': '0.7529', 'epoch': '0.01625'}

 15%|█▌        | 1837/12000 [1:06:24<5:35:38,  1.98s/it]
 15%|█▌        | 1838/12000 [1:06:27<6:13:40,  2.21s/it]
                                                        

 15%|█▌        | 1838/12000 [1:06:27<6:13:40,  2.21s/it]
 15%|█▌        | 1839/12000 [1:06:30<7:03:15,  2.50s/it]
                                                        

 15%|█▌        | 1839/12000 [1:06:30<7:03:15,  2.50s/it]
 15%|█▌        | 1840/12000 [1:06:32<6:35:28,  2.34s/it]
                                                        

 15%|█▌        | 1840/12000 [1:06:32<6:35:28,  2.34s/it]
 15%|█▌        | 1841/12000 [1:06:34<6:19:20,  2.24s/it]
                                                        

 15%|█▌        | 1841/12000 [1:06:34<6:19:20,  2.24s/it]
 15%|█▌        | 1842/12000 [1:06:37<6:35:22,  2.34s/it]
                                                        

 15%|█▌        | 1842/12000 [1:06:37<6:35:22,  2.34s/it]
 15%|█▌        | 1843/12000 [1:06:39<6:08:57,  2.18s/it]
                                                        

 15%|█▌        | 1843/12000 [1:06:39<6:08:57,  2.18s/it]
 15%|█▌        | 1844/12000 [1:06:41<5:59:35,  2.12s/it]
                                                        

 15%|█▌        | 1844/12000 [1:06:41<5:59:35,  2.12s/it]
 15%|█▌        | 1845/12000 [1:06:44<7:10:17,  2.54s/it]
                                                        

 15%|█▌        | 1845/12000 [1:06:44<7:10:17,  2.54s/it]
 15%|█▌        | 1846/12000 [1:06:46<6:19:51,  2.24s/it]
                                                        

 15%|█▌        | 1846/12000 [1:06:46<6:19:51,  2.24s/it]
 15%|█▌        | 1847/12000 [1:06:47<5:36:24,  1.99s/it]
                                                        

 15%|█▌        | 1847/12000 [1:06:47<5:36:24,  1.99s/it]
 15%|█▌        | 1848/12000 [1:06:50<6:13:29,  2.21s/it]
                                                        

 15%|█▌        | 1848/12000 [1:06:50<6:13:29,  2.21s/it]
 15%|█▌        | 1849/12000 [1:06:51<5:41:26,  2.02s/it]
                                                        

 15%|█▌        | 1849/12000 [1:06:51<5:41:26,  2.02s/it]
 15%|█▌        | 1850/12000 [1:06:54<5:52:08,  2.08s/it]
                                                        

 15%|█▌        | 1850/12000 [1:06:54<5:52:08,  2.08s/it]
 15%|█▌        | 1851/12000 [1:06:56<6:31:39,  2.32s/it]
                                                        

 15%|█▌        | 1851/12000 [1:06:56<6:31:39,  2.32s/it]
 15%|█▌        | 1852/12000 [1:06:59<6:19:53,  2.25s/it]
                                                        

 15%|█▌        | 1852/12000 [1:06:59<6:19:53,  2.25s/it]
 15%|█▌        | 1853/12000 [1:07:00<5:48:54,  2.06s/it]
                                                        

 15%|█▌        | 1853/12000 [1:07:00<5:48:54,  2.06s/it]
 15%|█▌        | 1854/12000 [1:07:02<5:16:35,  1.87s/it]
                                                        

 15%|█▌        | 1854/12000 [1:07:02<5:16:35,  1.87s/it]
 15%|█▌        | 1855/12000 [1:07:06<7:38:16,  2.71s/it]
                                                        

 15%|█▌        | 1855/12000 [1:07:06<7:38:16,  2.71s/it]
 15%|█▌        | 1856/12000 [1:07:08<6:37:27,  2.35s/it]
                                                        

 15%|█▌        | 1856/12000 [1:07:08<6:37:27,  2.35s/it]
 15%|█▌        | 1857/12000 [1:07:10<6:04:38,  2.16s/it]
                                                        

 15%|█▌        | 1857/12000 [1:07:10<6:04:38,  2.16s/it]
 15%|█▌        | 1858/12000 [1:07:11<5:24:19,  1.92s/it]
                                                        

 15%|█▌        | 1858/12000 [1:07:11<5:24:19,  1.92s/it]
 15%|█▌        | 1859/12000 [1:07:13<5:14:19,  1.86s/it]
                                                        

 15%|█▌        | 1859/12000 [1:07:13<5:14:19,  1.86s/it]
 16%|█▌        | 1860/12000 [1:07:14<4:47:48,  1.70s/it]
                                                        

 16%|█▌        | 1860/12000 [1:07:14<4:47:48,  1.70s/it]
 16%|█▌        | 1861/12000 [1:07:15<4:36:58,  1.64s/it]
                                                        

 16%|█▌        | 1861/12000 [1:07:15<4:36:58,  1.64s/it]
 16%|█▌        | 1862/12000 [1:07:18<5:00:11,  1.78s/it]
                                                        

 16%|█▌        | 1862/12000 [1:07:18<5:00:11,  1.78s/it]
 16%|█▌        | 1863/12000 [1:07:19<4:41:51,  1.67s/it]
                                                        

 16%|█▌        | 1863/12000 [1:07:19<4:41:51,  1.67s/it]
 16%|█▌        | 1864/12000 [1:07:21<4:46:38,  1.70s/it]
                                                        

 16%|█▌        | 1864/12000 [1:07:21<4:46:38,  1.70s/it]
 16%|█▌        | 1865/12000 [1:07:22<4:36:32,  1.64s/it]
                                                        

 16%|█▌        | 1865/12000 [1:07:22<4:36:32,  1.64s/it]
 16%|█▌        | 1866/12000 [1:07:24<4:25:56,  1.57s/it]
                                                        

 16%|█▌        | 1866/12000 [1:07:24<4:25:56,  1.57s/it]
 16%|█▌        | 1867/12000 [1:07:27<6:15:08,  2.22s/it]
                                                        

 16%|█▌        | 1867/12000 [1:07:27<6:15:08,  2.22s/it]
 16%|█▌        | 1868/12000 [1:07:31<7:32:49,  2.68s/it]
                                                        

 16%|█▌        | 1868/12000 [1:07:31<7:32:49,  2.68s/it]
 16%|█▌        | 1869/12000 [1:07:33<6:38:54,  2.36s/it]
                                                        

 16%|█▌        | 1869/12000 [1:07:33<6:38:54,  2.36s/it]
 16%|█▌        | 1870/12000 [1:07:36<7:41:33,  2.73s/it]
                                                        

 16%|█▌        | 1870/12000 [1:07:36<7:41:33,  2.73s/it]
 16%|█▌        | 1871/12000 [1:07:39<7:39:47,  2.72s/it]
                                                        

 16%|█▌        | 1871/12000 [1:07:39<7:39:47,  2.72s/it]
 16%|█▌        | 1872/12000 [1:07:43<8:19:04,  2.96s/it]
                                                        

 16%|█▌        | 1872/12000 [1:07:43<8:19:04,  2.96s/it]
 16%|█▌        | 1873/12000 [1:07:44<7:09:16,  2.54s/it]
                                                        

 16%|█▌        | 1873/12000 [1:07:44<7:09:16,  2.54s/it]
 16%|█▌        | 1874/12000 [1:07:46<6:29:48,  2.31s/it]
                                                        

 16%|█▌        | 1874/12000 [1:07:46<6:29:48,  2.31s/it]
 16%|█▌        | 1875/12000 [1:07:47<5:44:09,  2.04s/it]
                                                        

 16%|█▌        | 1875/12000 [1:07:47<5:44:09,  2.04s/it]
 16%|█▌        | 1876/12000 [1:07:51<7:13:51,  2.57s/it]
                                                        

 16%|█▌        | 1876/12000 [1:07:51<7:13:51,  2.57s/it]
 16%|█▌        | 1877/12000 [1:07:53<6:23:07,  2.27s/it]
                                                        

 16%|█▌        | 1877/12000 [1:07:53<6:23:07,  2.27s/it]
 16%|█▌        | 1878/12000 [1:07:55<6:33:10,  2.33s/it]
                                                        

 16%|█▌        | 1878/12000 [1:07:55<6:33:10,  2.33s/it]
 16%|█▌        | 1879/12000 [1:07:57<6:10:56,  2.20s/it]
                                                        

 16%|█▌        | 1879/12000 [1:07:57<6:10:56,  2.20s/it]
 16%|█▌        | 1880/12000 [1:08:00<6:28:49,  2.31s/it]
                                                        

 16%|█▌        | 1880/12000 [1:08:00<6:28:49,  2.31s/it]
 16%|█▌        | 1881/12000 [1:08:01<5:46:36,  2.06s/it]
                                                        

 16%|█▌        | 1881/12000 [1:08:01<5:46:36,  2.06s/it]
 16%|█▌        | 1882/12000 [1:08:03<5:19:08,  1.89s/it]
                                                        

 16%|█▌        | 1882/12000 [1:08:03<5:19:08,  1.89s/it]
 16%|█▌        | 1883/12000 [1:08:07<7:27:25,  2.65s/it]
                                                        

 16%|█▌        | 1883/12000 [1:08:07<7:27:25,  2.65s/it]
 16%|█▌        | 1884/12000 [1:08:09<6:40:48,  2.38s/it]
                                                        
{'loss': '0.918', 'grad_norm': '0.3966', 'learning_rate': '0.0001943', 'entropy': '0.8783', 'num_tokens': '9.981e+06', 'mean_token_accuracy': '0.7803', 'epoch': '0.01626'}
{'loss': '0.8044', 'grad_norm': '0.3467', 'learning_rate': '0.0001942', 'entropy': '0.7623', 'num_tokens': '9.988e+06', 'mean_token_accuracy': '0.8039', 'epoch': '0.01627'}
{'loss': '0.9845', 'grad_norm': '0.3406', 'learning_rate': '0.0001942', 'entropy': '1.007', 'num_tokens': '9.996e+06', 'mean_token_accuracy': '0.7519', 'epoch': '0.01627'}
{'loss': '0.8584', 'grad_norm': '0.3803', 'learning_rate': '0.0001942', 'entropy': '0.8422', 'num_tokens': '1e+07', 'mean_token_accuracy': '0.7705', 'epoch': '0.01628'}
{'loss': '1.284', 'grad_norm': '0.4171', 'learning_rate': '0.0001942', 'entropy': '1.211', 'num_tokens': '1.001e+07', 'mean_token_accuracy': '0.7083', 'epoch': '0.01629'}
{'loss': '1.226', 'grad_norm': '0.4659', 'learning_rate': '0.0001942', 'entropy': '0.9853', 'num_tokens': '1.001e+07', 'mean_token_accuracy': '0.7472', 'epoch': '0.0163'}
{'loss': '1.033', 'grad_norm': '0.4387', 'learning_rate': '0.0001942', 'entropy': '1.04', 'num_tokens': '1.002e+07', 'mean_token_accuracy': '0.7529', 'epoch': '0.01631'}
{'loss': '1.432', 'grad_norm': '0.4458', 'learning_rate': '0.0001942', 'entropy': '1.305', 'num_tokens': '1.002e+07', 'mean_token_accuracy': '0.7186', 'epoch': '0.01632'}
{'loss': '0.7498', 'grad_norm': '0.367', 'learning_rate': '0.0001942', 'entropy': '0.7979', 'num_tokens': '1.003e+07', 'mean_token_accuracy': '0.802', 'epoch': '0.01633'}
{'loss': '0.8298', 'grad_norm': '0.4185', 'learning_rate': '0.0001942', 'entropy': '0.8956', 'num_tokens': '1.003e+07', 'mean_token_accuracy': '0.7806', 'epoch': '0.01634'}
{'loss': '0.8986', 'grad_norm': '0.3413', 'learning_rate': '0.0001942', 'entropy': '0.8959', 'num_tokens': '1.004e+07', 'mean_token_accuracy': '0.7771', 'epoch': '0.01635'}
{'loss': '1.054', 'grad_norm': '0.3687', 'learning_rate': '0.0001942', 'entropy': '1.12', 'num_tokens': '1.004e+07', 'mean_token_accuracy': '0.7377', 'epoch': '0.01635'}
{'loss': '1.056', 'grad_norm': '0.4444', 'learning_rate': '0.0001941', 'entropy': '1.086', 'num_tokens': '1.005e+07', 'mean_token_accuracy': '0.7406', 'epoch': '0.01636'}
{'loss': '0.9788', 'grad_norm': '0.4231', 'learning_rate': '0.0001941', 'entropy': '0.8753', 'num_tokens': '1.005e+07', 'mean_token_accuracy': '0.7831', 'epoch': '0.01637'}
{'loss': '0.8794', 'grad_norm': '0.415', 'learning_rate': '0.0001941', 'entropy': '0.8746', 'num_tokens': '1.006e+07', 'mean_token_accuracy': '0.7817', 'epoch': '0.01638'}
{'loss': '1.416', 'grad_norm': '0.3868', 'learning_rate': '0.0001941', 'entropy': '1.224', 'num_tokens': '1.006e+07', 'mean_token_accuracy': '0.7005', 'epoch': '0.01639'}
{'loss': '1.086', 'grad_norm': '0.3397', 'learning_rate': '0.0001941', 'entropy': '1.156', 'num_tokens': '1.007e+07', 'mean_token_accuracy': '0.712', 'epoch': '0.0164'}
{'loss': '0.9323', 'grad_norm': '0.3703', 'learning_rate': '0.0001941', 'entropy': '0.9668', 'num_tokens': '1.007e+07', 'mean_token_accuracy': '0.7689', 'epoch': '0.01641'}
{'loss': '1.183', 'grad_norm': '0.3685', 'learning_rate': '0.0001941', 'entropy': '0.9879', 'num_tokens': '1.008e+07', 'mean_token_accuracy': '0.7601', 'epoch': '0.01642'}
{'loss': '0.9073', 'grad_norm': '0.4745', 'learning_rate': '0.0001941', 'entropy': '0.9474', 'num_tokens': '1.008e+07', 'mean_token_accuracy': '0.7673', 'epoch': '0.01643'}
{'loss': '1.22', 'grad_norm': '0.3778', 'learning_rate': '0.0001941', 'entropy': '1.213', 'num_tokens': '1.009e+07', 'mean_token_accuracy': '0.7071', 'epoch': '0.01643'}
{'loss': '0.6835', 'grad_norm': '0.321', 'learning_rate': '0.0001941', 'entropy': '0.6943', 'num_tokens': '1.009e+07', 'mean_token_accuracy': '0.8249', 'epoch': '0.01644'}
{'loss': '0.9478', 'grad_norm': '0.4465', 'learning_rate': '0.0001941', 'entropy': '0.9174', 'num_tokens': '1.01e+07', 'mean_token_accuracy': '0.7835', 'epoch': '0.01645'}
{'loss': '0.8669', 'grad_norm': '0.3886', 'learning_rate': '0.000194', 'entropy': '0.8544', 'num_tokens': '1.01e+07', 'mean_token_accuracy': '0.7831', 'epoch': '0.01646'}
{'loss': '0.6296', 'grad_norm': '0.3508', 'learning_rate': '0.000194', 'entropy': '0.6429', 'num_tokens': '1.011e+07', 'mean_token_accuracy': '0.8336', 'epoch': '0.01647'}
{'loss': '0.8631', 'grad_norm': '0.3459', 'learning_rate': '0.000194', 'entropy': '0.8763', 'num_tokens': '1.011e+07', 'mean_token_accuracy': '0.7759', 'epoch': '0.01648'}
{'loss': '0.7341', 'grad_norm': '0.3224', 'learning_rate': '0.000194', 'entropy': '0.7667', 'num_tokens': '1.012e+07', 'mean_token_accuracy': '0.8111', 'epoch': '0.01649'}
{'loss': '0.8422', 'grad_norm': '0.4296', 'learning_rate': '0.000194', 'entropy': '0.8309', 'num_tokens': '1.012e+07', 'mean_token_accuracy': '0.7839', 'epoch': '0.0165'}
{'loss': '1.288', 'grad_norm': '0.4315', 'learning_rate': '0.000194', 'entropy': '1.252', 'num_tokens': '1.013e+07', 'mean_token_accuracy': '0.7036', 'epoch': '0.0165'}
{'loss': '0.7632', 'grad_norm': '0.289', 'learning_rate': '0.000194', 'entropy': '0.8423', 'num_tokens': '1.013e+07', 'mean_token_accuracy': '0.7853', 'epoch': '0.01651'}
{'loss': '1.092', 'grad_norm': '0.4571', 'learning_rate': '0.000194', 'entropy': '1.027', 'num_tokens': '1.014e+07', 'mean_token_accuracy': '0.7406', 'epoch': '0.01652'}
{'loss': '1.602', 'grad_norm': '1.175', 'learning_rate': '0.000194', 'entropy': '1.092', 'num_tokens': '1.014e+07', 'mean_token_accuracy': '0.7022', 'epoch': '0.01653'}
{'loss': '1.005', 'grad_norm': '0.3913', 'learning_rate': '0.000194', 'entropy': '1.083', 'num_tokens': '1.015e+07', 'mean_token_accuracy': '0.7478', 'epoch': '0.01654'}
{'loss': '0.8795', 'grad_norm': '0.3129', 'learning_rate': '0.0001939', 'entropy': '0.9491', 'num_tokens': '1.016e+07', 'mean_token_accuracy': '0.7785', 'epoch': '0.01655'}
{'loss': '0.8801', 'grad_norm': '0.3653', 'learning_rate': '0.0001939', 'entropy': '0.9209', 'num_tokens': '1.017e+07', 'mean_token_accuracy': '0.7681', 'epoch': '0.01656'}
{'loss': '1.104', 'grad_norm': '0.3536', 'learning_rate': '0.0001939', 'entropy': '1.046', 'num_tokens': '1.018e+07', 'mean_token_accuracy': '0.756', 'epoch': '0.01657'}
{'loss': '1.269', 'grad_norm': '0.4026', 'learning_rate': '0.0001939', 'entropy': '1.193', 'num_tokens': '1.018e+07', 'mean_token_accuracy': '0.7254', 'epoch': '0.01658'}
{'loss': '0.9136', 'grad_norm': '0.4014', 'learning_rate': '0.0001939', 'entropy': '0.9264', 'num_tokens': '1.018e+07', 'mean_token_accuracy': '0.7668', 'epoch': '0.01658'}
{'loss': '0.9556', 'grad_norm': '0.4465', 'learning_rate': '0.0001939', 'entropy': '1.056', 'num_tokens': '1.019e+07', 'mean_token_accuracy': '0.7473', 'epoch': '0.01659'}
{'loss': '1.136', 'grad_norm': '0.3075', 'learning_rate': '0.0001939', 'entropy': '1.197', 'num_tokens': '1.02e+07', 'mean_token_accuracy': '0.7217', 'epoch': '0.0166'}
{'loss': '0.5441', 'grad_norm': '0.3223', 'learning_rate': '0.0001939', 'entropy': '0.6225', 'num_tokens': '1.02e+07', 'mean_token_accuracy': '0.8342', 'epoch': '0.01661'}
{'loss': '0.8461', 'grad_norm': '0.3009', 'learning_rate': '0.0001939', 'entropy': '0.7895', 'num_tokens': '1.021e+07', 'mean_token_accuracy': '0.8165', 'epoch': '0.01662'}
{'loss': '0.7672', 'grad_norm': '0.4343', 'learning_rate': '0.0001939', 'entropy': '0.8062', 'num_tokens': '1.022e+07', 'mean_token_accuracy': '0.7842', 'epoch': '0.01663'}
{'loss': '1.035', 'grad_norm': '0.3671', 'learning_rate': '0.0001939', 'entropy': '1.049', 'num_tokens': '1.022e+07', 'mean_token_accuracy': '0.7514', 'epoch': '0.01664'}
{'loss': '0.4828', 'grad_norm': '0.2925', 'learning_rate': '0.0001938', 'entropy': '0.541', 'num_tokens': '1.023e+07', 'mean_token_accuracy': '0.8567', 'epoch': '0.01665'}
{'loss': '1.009', 'grad_norm': '0.4481', 'learning_rate': '0.0001938', 'entropy': '0.9978', 'num_tokens': '1.023e+07', 'mean_token_accuracy': '0.7422', 'epoch': '0.01666'}
{'loss': '1.137', 'grad_norm': '0.3448', 'learning_rate': '0.0001938', 'entropy': '0.9496', 'num_tokens': '1.024e+07', 'mean_token_accuracy': '0.7659', 'epoch': '0.01666'}

 16%|█▌        | 1884/12000 [1:08:09<6:40:48,  2.38s/it]
 16%|█▌        | 1885/12000 [1:08:10<6:08:58,  2.19s/it]
                                                        

 16%|█▌        | 1885/12000 [1:08:10<6:08:58,  2.19s/it]
 16%|█▌        | 1886/12000 [1:08:12<5:49:12,  2.07s/it]
                                                        

 16%|█▌        | 1886/12000 [1:08:12<5:49:12,  2.07s/it]
 16%|█▌        | 1887/12000 [1:08:17<8:18:59,  2.96s/it]
                                                        

 16%|█▌        | 1887/12000 [1:08:17<8:18:59,  2.96s/it]
 16%|█▌        | 1888/12000 [1:08:19<7:28:53,  2.66s/it]
                                                        

 16%|█▌        | 1888/12000 [1:08:19<7:28:53,  2.66s/it]
 16%|█▌        | 1889/12000 [1:08:23<8:19:40,  2.97s/it]
                                                        

 16%|█▌        | 1889/12000 [1:08:23<8:19:40,  2.97s/it]
 16%|█▌        | 1890/12000 [1:08:24<6:56:41,  2.47s/it]
                                                        

 16%|█▌        | 1890/12000 [1:08:24<6:56:41,  2.47s/it]
 16%|█▌        | 1891/12000 [1:08:26<6:00:26,  2.14s/it]
                                                        

 16%|█▌        | 1891/12000 [1:08:26<6:00:26,  2.14s/it]
 16%|█▌        | 1892/12000 [1:08:27<5:15:37,  1.87s/it]
                                                        

 16%|█▌        | 1892/12000 [1:08:27<5:15:37,  1.87s/it]
 16%|█▌        | 1893/12000 [1:08:28<4:58:03,  1.77s/it]
                                                        

 16%|█▌        | 1893/12000 [1:08:28<4:58:03,  1.77s/it]
 16%|█▌        | 1894/12000 [1:08:31<5:33:43,  1.98s/it]
                                                        

 16%|█▌        | 1894/12000 [1:08:31<5:33:43,  1.98s/it]
 16%|█▌        | 1895/12000 [1:08:34<6:14:58,  2.23s/it]
                                                        

 16%|█▌        | 1895/12000 [1:08:34<6:14:58,  2.23s/it]
 16%|█▌        | 1896/12000 [1:08:35<5:44:02,  2.04s/it]
                                                        

 16%|█▌        | 1896/12000 [1:08:35<5:44:02,  2.04s/it]
 16%|█▌        | 1897/12000 [1:08:38<6:37:05,  2.36s/it]
                                                        

 16%|█▌        | 1897/12000 [1:08:38<6:37:05,  2.36s/it]
 16%|█▌        | 1898/12000 [1:08:40<5:57:09,  2.12s/it]
                                                        

 16%|█▌        | 1898/12000 [1:08:40<5:57:09,  2.12s/it]
 16%|█▌        | 1899/12000 [1:08:43<6:46:48,  2.42s/it]
                                                        

 16%|█▌        | 1899/12000 [1:08:43<6:46:48,  2.42s/it]
 16%|█▌        | 1900/12000 [1:08:45<6:15:41,  2.23s/it]
                                                        

 16%|█▌        | 1900/12000 [1:08:45<6:15:41,  2.23s/it]
 16%|█▌        | 1901/12000 [1:08:48<6:44:01,  2.40s/it]
                                                        

 16%|█▌        | 1901/12000 [1:08:48<6:44:01,  2.40s/it]
 16%|█▌        | 1902/12000 [1:08:50<6:25:47,  2.29s/it]
                                                        

 16%|█▌        | 1902/12000 [1:08:50<6:25:47,  2.29s/it]
 16%|█▌        | 1903/12000 [1:08:51<5:41:19,  2.03s/it]
                                                        

 16%|█▌        | 1903/12000 [1:08:51<5:41:19,  2.03s/it]
 16%|█▌        | 1904/12000 [1:08:53<5:19:42,  1.90s/it]
                                                        

 16%|█▌        | 1904/12000 [1:08:53<5:19:42,  1.90s/it]
 16%|█▌        | 1905/12000 [1:08:55<5:40:50,  2.03s/it]
                                                        

 16%|█▌        | 1905/12000 [1:08:55<5:40:50,  2.03s/it]
 16%|█▌        | 1906/12000 [1:08:57<6:01:27,  2.15s/it]
                                                        

 16%|█▌        | 1906/12000 [1:08:57<6:01:27,  2.15s/it]
 16%|█▌        | 1907/12000 [1:09:00<6:15:07,  2.23s/it]
                                                        

 16%|█▌        | 1907/12000 [1:09:00<6:15:07,  2.23s/it]
 16%|█▌        | 1908/12000 [1:09:02<6:04:10,  2.17s/it]
                                                        

 16%|█▌        | 1908/12000 [1:09:02<6:04:10,  2.17s/it]
 16%|█▌        | 1909/12000 [1:09:03<5:28:46,  1.95s/it]
                                                        

 16%|█▌        | 1909/12000 [1:09:03<5:28:46,  1.95s/it]
 16%|█▌        | 1910/12000 [1:09:06<5:57:37,  2.13s/it]
                                                        

 16%|█▌        | 1910/12000 [1:09:06<5:57:37,  2.13s/it]
 16%|█▌        | 1911/12000 [1:09:10<7:51:46,  2.81s/it]
                                                        

 16%|█▌        | 1911/12000 [1:09:10<7:51:46,  2.81s/it]
 16%|█▌        | 1912/12000 [1:09:12<6:55:48,  2.47s/it]
                                                        

 16%|█▌        | 1912/12000 [1:09:12<6:55:48,  2.47s/it]
 16%|█▌        | 1913/12000 [1:09:15<7:16:39,  2.60s/it]
                                                        

 16%|█▌        | 1913/12000 [1:09:15<7:16:39,  2.60s/it]
 16%|█▌        | 1914/12000 [1:09:17<6:37:57,  2.37s/it]
                                                        

 16%|█▌        | 1914/12000 [1:09:17<6:37:57,  2.37s/it]
 16%|█▌        | 1915/12000 [1:09:19<6:23:44,  2.28s/it]
                                                        

 16%|█▌        | 1915/12000 [1:09:19<6:23:44,  2.28s/it]
 16%|█▌        | 1916/12000 [1:09:20<5:38:24,  2.01s/it]
                                                        

 16%|█▌        | 1916/12000 [1:09:20<5:38:24,  2.01s/it]
 16%|█▌        | 1917/12000 [1:09:22<5:29:55,  1.96s/it]
                                                        

 16%|█▌        | 1917/12000 [1:09:22<5:29:55,  1.96s/it]
 16%|█▌        | 1918/12000 [1:09:24<5:52:18,  2.10s/it]
                                                        

 16%|█▌        | 1918/12000 [1:09:24<5:52:18,  2.10s/it]
 16%|█▌        | 1919/12000 [1:09:26<5:30:05,  1.96s/it]
                                                        

 16%|█▌        | 1919/12000 [1:09:26<5:30:05,  1.96s/it]
 16%|█▌        | 1920/12000 [1:09:27<5:01:41,  1.80s/it]
                                                        

 16%|█▌        | 1920/12000 [1:09:27<5:01:41,  1.80s/it]
 16%|█▌        | 1921/12000 [1:09:29<5:01:00,  1.79s/it]
                                                        

 16%|█▌        | 1921/12000 [1:09:29<5:01:00,  1.79s/it]
 16%|█▌        | 1922/12000 [1:09:31<4:47:08,  1.71s/it]
                                                        

 16%|█▌        | 1922/12000 [1:09:31<4:47:08,  1.71s/it]
 16%|█▌        | 1923/12000 [1:09:32<4:30:27,  1.61s/it]
                                                        

 16%|█▌        | 1923/12000 [1:09:32<4:30:27,  1.61s/it]
 16%|█▌        | 1924/12000 [1:09:34<4:27:17,  1.59s/it]
                                                        

 16%|█▌        | 1924/12000 [1:09:34<4:27:17,  1.59s/it]
 16%|█▌        | 1925/12000 [1:09:35<4:31:04,  1.61s/it]
                                                        

 16%|█▌        | 1925/12000 [1:09:35<4:31:04,  1.61s/it]
 16%|█▌        | 1926/12000 [1:09:37<4:23:00,  1.57s/it]
                                                        

 16%|█▌        | 1926/12000 [1:09:37<4:23:00,  1.57s/it]
 16%|█▌        | 1927/12000 [1:09:39<4:30:21,  1.61s/it]
                                                        

 16%|█▌        | 1927/12000 [1:09:39<4:30:21,  1.61s/it]
 16%|█▌        | 1928/12000 [1:09:40<4:30:13,  1.61s/it]
                                                        

 16%|█▌        | 1928/12000 [1:09:40<4:30:13,  1.61s/it]
 16%|█▌        | 1929/12000 [1:09:43<5:25:56,  1.94s/it]
                                                        

 16%|█▌        | 1929/12000 [1:09:43<5:25:56,  1.94s/it]
 16%|█▌        | 1930/12000 [1:09:46<6:40:31,  2.39s/it]
                                                        

 16%|█▌        | 1930/12000 [1:09:46<6:40:31,  2.39s/it]
 16%|█▌        | 1931/12000 [1:09:48<5:52:20,  2.10s/it]
                                                        
{'loss': '1.234', 'grad_norm': '0.543', 'learning_rate': '0.0001938', 'entropy': '1.155', 'num_tokens': '1.024e+07', 'mean_token_accuracy': '0.7009', 'epoch': '0.01667'}
{'loss': '0.9295', 'grad_norm': '0.3171', 'learning_rate': '0.0001938', 'entropy': '0.7976', 'num_tokens': '1.025e+07', 'mean_token_accuracy': '0.7968', 'epoch': '0.01668'}
{'loss': '1.211', 'grad_norm': '0.3788', 'learning_rate': '0.0001938', 'entropy': '1.078', 'num_tokens': '1.026e+07', 'mean_token_accuracy': '0.7357', 'epoch': '0.01669'}
{'loss': '1.433', 'grad_norm': '0.3155', 'learning_rate': '0.0001938', 'entropy': '1.277', 'num_tokens': '1.027e+07', 'mean_token_accuracy': '0.6902', 'epoch': '0.0167'}
{'loss': '0.711', 'grad_norm': '0.2944', 'learning_rate': '0.0001938', 'entropy': '0.7533', 'num_tokens': '1.027e+07', 'mean_token_accuracy': '0.8076', 'epoch': '0.01671'}
{'loss': '1.138', 'grad_norm': '0.4335', 'learning_rate': '0.0001938', 'entropy': '1.171', 'num_tokens': '1.028e+07', 'mean_token_accuracy': '0.7188', 'epoch': '0.01672'}
{'loss': '0.7715', 'grad_norm': '0.4977', 'learning_rate': '0.0001938', 'entropy': '0.7521', 'num_tokens': '1.028e+07', 'mean_token_accuracy': '0.8054', 'epoch': '0.01673'}
{'loss': '0.8487', 'grad_norm': '0.3939', 'learning_rate': '0.0001937', 'entropy': '0.8416', 'num_tokens': '1.028e+07', 'mean_token_accuracy': '0.7974', 'epoch': '0.01674'}
{'loss': '0.9441', 'grad_norm': '0.4788', 'learning_rate': '0.0001937', 'entropy': '0.9874', 'num_tokens': '1.029e+07', 'mean_token_accuracy': '0.7711', 'epoch': '0.01674'}
{'loss': '1.082', 'grad_norm': '0.3553', 'learning_rate': '0.0001937', 'entropy': '1.087', 'num_tokens': '1.029e+07', 'mean_token_accuracy': '0.7403', 'epoch': '0.01675'}
{'loss': '1.311', 'grad_norm': '0.5092', 'learning_rate': '0.0001937', 'entropy': '1.123', 'num_tokens': '1.03e+07', 'mean_token_accuracy': '0.7445', 'epoch': '0.01676'}
{'loss': '0.9863', 'grad_norm': '0.39', 'learning_rate': '0.0001937', 'entropy': '1.04', 'num_tokens': '1.03e+07', 'mean_token_accuracy': '0.7444', 'epoch': '0.01677'}
{'loss': '0.6168', 'grad_norm': '0.2826', 'learning_rate': '0.0001937', 'entropy': '0.6804', 'num_tokens': '1.031e+07', 'mean_token_accuracy': '0.8309', 'epoch': '0.01678'}
{'loss': '1.177', 'grad_norm': '0.3255', 'learning_rate': '0.0001937', 'entropy': '1.384', 'num_tokens': '1.032e+07', 'mean_token_accuracy': '0.6957', 'epoch': '0.01679'}
{'loss': '0.8114', 'grad_norm': '0.405', 'learning_rate': '0.0001937', 'entropy': '0.8142', 'num_tokens': '1.032e+07', 'mean_token_accuracy': '0.7801', 'epoch': '0.0168'}
{'loss': '0.6852', 'grad_norm': '0.2864', 'learning_rate': '0.0001937', 'entropy': '0.717', 'num_tokens': '1.033e+07', 'mean_token_accuracy': '0.8259', 'epoch': '0.01681'}
{'loss': '0.7069', 'grad_norm': '0.3804', 'learning_rate': '0.0001937', 'entropy': '0.7121', 'num_tokens': '1.033e+07', 'mean_token_accuracy': '0.8099', 'epoch': '0.01681'}
{'loss': '0.7522', 'grad_norm': '0.4166', 'learning_rate': '0.0001937', 'entropy': '0.751', 'num_tokens': '1.034e+07', 'mean_token_accuracy': '0.8093', 'epoch': '0.01682'}
{'loss': '1.185', 'grad_norm': '0.6129', 'learning_rate': '0.0001936', 'entropy': '1.058', 'num_tokens': '1.034e+07', 'mean_token_accuracy': '0.7229', 'epoch': '0.01683'}
{'loss': '0.8971', 'grad_norm': '0.4619', 'learning_rate': '0.0001936', 'entropy': '0.8617', 'num_tokens': '1.035e+07', 'mean_token_accuracy': '0.7867', 'epoch': '0.01684'}
{'loss': '0.8881', 'grad_norm': '0.4091', 'learning_rate': '0.0001936', 'entropy': '0.8971', 'num_tokens': '1.035e+07', 'mean_token_accuracy': '0.7709', 'epoch': '0.01685'}
{'loss': '0.7086', 'grad_norm': '0.3411', 'learning_rate': '0.0001936', 'entropy': '0.6938', 'num_tokens': '1.036e+07', 'mean_token_accuracy': '0.822', 'epoch': '0.01686'}
{'loss': '0.9308', 'grad_norm': '0.4125', 'learning_rate': '0.0001936', 'entropy': '0.8579', 'num_tokens': '1.036e+07', 'mean_token_accuracy': '0.7831', 'epoch': '0.01687'}
{'loss': '1.096', 'grad_norm': '0.4124', 'learning_rate': '0.0001936', 'entropy': '0.966', 'num_tokens': '1.037e+07', 'mean_token_accuracy': '0.7558', 'epoch': '0.01688'}
{'loss': '0.7419', 'grad_norm': '0.3022', 'learning_rate': '0.0001936', 'entropy': '0.7422', 'num_tokens': '1.037e+07', 'mean_token_accuracy': '0.8168', 'epoch': '0.01689'}
{'loss': '0.723', 'grad_norm': '0.334', 'learning_rate': '0.0001936', 'entropy': '0.7433', 'num_tokens': '1.038e+07', 'mean_token_accuracy': '0.8241', 'epoch': '0.01689'}
{'loss': '0.8148', 'grad_norm': '0.3176', 'learning_rate': '0.0001936', 'entropy': '0.8176', 'num_tokens': '1.039e+07', 'mean_token_accuracy': '0.7825', 'epoch': '0.0169'}
{'loss': '1.167', 'grad_norm': '0.431', 'learning_rate': '0.0001936', 'entropy': '1.059', 'num_tokens': '1.039e+07', 'mean_token_accuracy': '0.7432', 'epoch': '0.01691'}
{'loss': '0.9353', 'grad_norm': '0.3707', 'learning_rate': '0.0001935', 'entropy': '0.924', 'num_tokens': '1.04e+07', 'mean_token_accuracy': '0.7803', 'epoch': '0.01692'}
{'loss': '0.8312', 'grad_norm': '0.4193', 'learning_rate': '0.0001935', 'entropy': '0.8483', 'num_tokens': '1.04e+07', 'mean_token_accuracy': '0.7952', 'epoch': '0.01693'}
{'loss': '0.9958', 'grad_norm': '0.3771', 'learning_rate': '0.0001935', 'entropy': '1.016', 'num_tokens': '1.041e+07', 'mean_token_accuracy': '0.7647', 'epoch': '0.01694'}
{'loss': '1.07', 'grad_norm': '0.3405', 'learning_rate': '0.0001935', 'entropy': '1.057', 'num_tokens': '1.042e+07', 'mean_token_accuracy': '0.7397', 'epoch': '0.01695'}
{'loss': '0.9456', 'grad_norm': '0.3463', 'learning_rate': '0.0001935', 'entropy': '1.015', 'num_tokens': '1.042e+07', 'mean_token_accuracy': '0.7568', 'epoch': '0.01696'}
{'loss': '0.8272', 'grad_norm': '0.3419', 'learning_rate': '0.0001935', 'entropy': '0.8663', 'num_tokens': '1.043e+07', 'mean_token_accuracy': '0.7882', 'epoch': '0.01697'}
{'loss': '0.9422', 'grad_norm': '0.5059', 'learning_rate': '0.0001935', 'entropy': '0.9437', 'num_tokens': '1.043e+07', 'mean_token_accuracy': '0.7611', 'epoch': '0.01697'}
{'loss': '0.8668', 'grad_norm': '0.3305', 'learning_rate': '0.0001935', 'entropy': '0.784', 'num_tokens': '1.044e+07', 'mean_token_accuracy': '0.7961', 'epoch': '0.01698'}
{'loss': '0.6551', 'grad_norm': '0.2905', 'learning_rate': '0.0001935', 'entropy': '0.726', 'num_tokens': '1.044e+07', 'mean_token_accuracy': '0.8075', 'epoch': '0.01699'}
{'loss': '1.13', 'grad_norm': '0.3496', 'learning_rate': '0.0001935', 'entropy': '1.088', 'num_tokens': '1.045e+07', 'mean_token_accuracy': '0.7223', 'epoch': '0.017'}
{'loss': '1.001', 'grad_norm': '0.3639', 'learning_rate': '0.0001934', 'entropy': '1.011', 'num_tokens': '1.045e+07', 'mean_token_accuracy': '0.7503', 'epoch': '0.01701'}
{'loss': '1.022', 'grad_norm': '0.4943', 'learning_rate': '0.0001934', 'entropy': '1.008', 'num_tokens': '1.046e+07', 'mean_token_accuracy': '0.7481', 'epoch': '0.01702'}
{'loss': '0.7323', 'grad_norm': '0.3757', 'learning_rate': '0.0001934', 'entropy': '0.7549', 'num_tokens': '1.046e+07', 'mean_token_accuracy': '0.8152', 'epoch': '0.01703'}
{'loss': '0.4882', 'grad_norm': '0.2359', 'learning_rate': '0.0001934', 'entropy': '0.5139', 'num_tokens': '1.047e+07', 'mean_token_accuracy': '0.8709', 'epoch': '0.01704'}
{'loss': '0.9819', 'grad_norm': '0.3365', 'learning_rate': '0.0001934', 'entropy': '0.9355', 'num_tokens': '1.047e+07', 'mean_token_accuracy': '0.7775', 'epoch': '0.01704'}
{'loss': '1.193', 'grad_norm': '0.3731', 'learning_rate': '0.0001934', 'entropy': '1.153', 'num_tokens': '1.048e+07', 'mean_token_accuracy': '0.7319', 'epoch': '0.01705'}
{'loss': '0.8547', 'grad_norm': '0.4734', 'learning_rate': '0.0001934', 'entropy': '0.8624', 'num_tokens': '1.048e+07', 'mean_token_accuracy': '0.7847', 'epoch': '0.01706'}
{'loss': '1.158', 'grad_norm': '0.4156', 'learning_rate': '0.0001934', 'entropy': '1.033', 'num_tokens': '1.049e+07', 'mean_token_accuracy': '0.7464', 'epoch': '0.01707'}
{'loss': '0.6932', 'grad_norm': '2.035', 'learning_rate': '0.0001934', 'entropy': '0.6844', 'num_tokens': '1.049e+07', 'mean_token_accuracy': '0.8281', 'epoch': '0.01708'}

 16%|█▌        | 1931/12000 [1:09:48<5:52:20,  2.10s/it]
 16%|█▌        | 1932/12000 [1:09:51<6:28:58,  2.32s/it]
                                                        

 16%|█▌        | 1932/12000 [1:09:51<6:28:58,  2.32s/it]
 16%|█▌        | 1933/12000 [1:09:55<7:51:29,  2.81s/it]
                                                        

 16%|█▌        | 1933/12000 [1:09:55<7:51:29,  2.81s/it]
 16%|█▌        | 1934/12000 [1:09:56<6:36:54,  2.37s/it]
                                                        

 16%|█▌        | 1934/12000 [1:09:56<6:36:54,  2.37s/it]
 16%|█▌        | 1935/12000 [1:09:57<5:59:10,  2.14s/it]
                                                        

 16%|█▌        | 1935/12000 [1:09:57<5:59:10,  2.14s/it]
 16%|█▌        | 1936/12000 [1:09:59<5:30:25,  1.97s/it]
                                                        

 16%|█▌        | 1936/12000 [1:09:59<5:30:25,  1.97s/it]
 16%|█▌        | 1937/12000 [1:10:01<5:39:28,  2.02s/it]
                                                        

 16%|█▌        | 1937/12000 [1:10:01<5:39:28,  2.02s/it]
 16%|█▌        | 1938/12000 [1:10:03<5:04:46,  1.82s/it]
                                                        

 16%|█▌        | 1938/12000 [1:10:03<5:04:46,  1.82s/it]
 16%|█▌        | 1939/12000 [1:10:05<5:21:00,  1.91s/it]
                                                        

 16%|█▌        | 1939/12000 [1:10:05<5:21:00,  1.91s/it]
 16%|█▌        | 1940/12000 [1:10:07<5:33:43,  1.99s/it]
                                                        

 16%|█▌        | 1940/12000 [1:10:07<5:33:43,  1.99s/it]
 16%|█▌        | 1941/12000 [1:10:09<5:19:21,  1.90s/it]
                                                        

 16%|█▌        | 1941/12000 [1:10:09<5:19:21,  1.90s/it]
 16%|█▌        | 1942/12000 [1:10:10<4:59:43,  1.79s/it]
                                                        

 16%|█▌        | 1942/12000 [1:10:10<4:59:43,  1.79s/it]
 16%|█▌        | 1943/12000 [1:10:11<4:42:59,  1.69s/it]
                                                        

 16%|█▌        | 1943/12000 [1:10:11<4:42:59,  1.69s/it]
 16%|█▌        | 1944/12000 [1:10:13<4:25:06,  1.58s/it]
                                                        

 16%|█▌        | 1944/12000 [1:10:13<4:25:06,  1.58s/it]
 16%|█▌        | 1945/12000 [1:10:15<5:11:20,  1.86s/it]
                                                        

 16%|█▌        | 1945/12000 [1:10:15<5:11:20,  1.86s/it]
 16%|█▌        | 1946/12000 [1:10:18<5:47:36,  2.07s/it]
                                                        

 16%|█▌        | 1946/12000 [1:10:18<5:47:36,  2.07s/it]
 16%|█▌        | 1947/12000 [1:10:19<5:15:15,  1.88s/it]
                                                        

 16%|█▌        | 1947/12000 [1:10:19<5:15:15,  1.88s/it]
 16%|█▌        | 1948/12000 [1:10:22<5:40:19,  2.03s/it]
                                                        

 16%|█▌        | 1948/12000 [1:10:22<5:40:19,  2.03s/it]
 16%|█▌        | 1949/12000 [1:10:24<5:30:37,  1.97s/it]
                                                        

 16%|█▌        | 1949/12000 [1:10:24<5:30:37,  1.97s/it]
 16%|█▋        | 1950/12000 [1:10:25<5:03:38,  1.81s/it]
                                                        

 16%|█▋        | 1950/12000 [1:10:25<5:03:38,  1.81s/it]
 16%|█▋        | 1951/12000 [1:10:28<6:00:26,  2.15s/it]
                                                        

 16%|█▋        | 1951/12000 [1:10:28<6:00:26,  2.15s/it]
 16%|█▋        | 1952/12000 [1:10:31<6:42:32,  2.40s/it]
                                                        

 16%|█▋        | 1952/12000 [1:10:31<6:42:32,  2.40s/it]
 16%|█▋        | 1953/12000 [1:10:32<5:53:36,  2.11s/it]
                                                        

 16%|█▋        | 1953/12000 [1:10:32<5:53:36,  2.11s/it]
 16%|█▋        | 1954/12000 [1:10:34<5:17:43,  1.90s/it]
                                                        

 16%|█▋        | 1954/12000 [1:10:34<5:17:43,  1.90s/it]
 16%|█▋        | 1955/12000 [1:10:39<8:03:53,  2.89s/it]
                                                        

 16%|█▋        | 1955/12000 [1:10:39<8:03:53,  2.89s/it]
 16%|█▋        | 1956/12000 [1:10:41<7:23:47,  2.65s/it]
                                                        

 16%|█▋        | 1956/12000 [1:10:41<7:23:47,  2.65s/it]
 16%|█▋        | 1957/12000 [1:10:43<7:10:28,  2.57s/it]
                                                        

 16%|█▋        | 1957/12000 [1:10:43<7:10:28,  2.57s/it]
 16%|█▋        | 1958/12000 [1:10:45<6:25:06,  2.30s/it]
                                                        

 16%|█▋        | 1958/12000 [1:10:45<6:25:06,  2.30s/it]
 16%|█▋        | 1959/12000 [1:10:47<5:56:57,  2.13s/it]
                                                        

 16%|█▋        | 1959/12000 [1:10:47<5:56:57,  2.13s/it]
 16%|█▋        | 1960/12000 [1:10:49<5:36:34,  2.01s/it]
                                                        

 16%|█▋        | 1960/12000 [1:10:49<5:36:34,  2.01s/it]
 16%|█▋        | 1961/12000 [1:10:50<5:19:46,  1.91s/it]
                                                        

 16%|█▋        | 1961/12000 [1:10:50<5:19:46,  1.91s/it]
 16%|█▋        | 1962/12000 [1:10:55<7:22:31,  2.65s/it]
                                                        

 16%|█▋        | 1962/12000 [1:10:55<7:22:31,  2.65s/it]
 16%|█▋        | 1963/12000 [1:10:56<6:35:36,  2.36s/it]
                                                        

 16%|█▋        | 1963/12000 [1:10:56<6:35:36,  2.36s/it]
 16%|█▋        | 1964/12000 [1:10:58<6:18:21,  2.26s/it]
                                                        

 16%|█▋        | 1964/12000 [1:10:58<6:18:21,  2.26s/it]
 16%|█▋        | 1965/12000 [1:11:00<5:53:31,  2.11s/it]
                                                        

 16%|█▋        | 1965/12000 [1:11:00<5:53:31,  2.11s/it]
 16%|█▋        | 1966/12000 [1:11:03<6:11:12,  2.22s/it]
                                                        

 16%|█▋        | 1966/12000 [1:11:03<6:11:12,  2.22s/it]
 16%|█▋        | 1967/12000 [1:11:04<5:41:15,  2.04s/it]
                                                        

 16%|█▋        | 1967/12000 [1:11:04<5:41:15,  2.04s/it]
 16%|█▋        | 1968/12000 [1:11:06<5:17:06,  1.90s/it]
                                                        

 16%|█▋        | 1968/12000 [1:11:06<5:17:06,  1.90s/it]
 16%|█▋        | 1969/12000 [1:11:07<5:05:39,  1.83s/it]
                                                        

 16%|█▋        | 1969/12000 [1:11:07<5:05:39,  1.83s/it]
 16%|█▋        | 1970/12000 [1:11:09<4:49:09,  1.73s/it]
                                                        

 16%|█▋        | 1970/12000 [1:11:09<4:49:09,  1.73s/it]
 16%|█▋        | 1971/12000 [1:11:12<5:39:44,  2.03s/it]
                                                        

 16%|█▋        | 1971/12000 [1:11:12<5:39:44,  2.03s/it]
 16%|█▋        | 1972/12000 [1:11:13<5:12:38,  1.87s/it]
                                                        

 16%|█▋        | 1972/12000 [1:11:13<5:12:38,  1.87s/it]
 16%|█▋        | 1973/12000 [1:11:16<6:02:45,  2.17s/it]
                                                        

 16%|█▋        | 1973/12000 [1:11:16<6:02:45,  2.17s/it]
 16%|█▋        | 1974/12000 [1:11:18<5:30:07,  1.98s/it]
                                                        

 16%|█▋        | 1974/12000 [1:11:18<5:30:07,  1.98s/it]
 16%|█▋        | 1975/12000 [1:11:20<5:44:05,  2.06s/it]
                                                        

 16%|█▋        | 1975/12000 [1:11:20<5:44:05,  2.06s/it]
 16%|█▋        | 1976/12000 [1:11:21<5:18:19,  1.91s/it]
                                                        

 16%|█▋        | 1976/12000 [1:11:21<5:18:19,  1.91s/it]
 16%|█▋        | 1977/12000 [1:11:25<7:06:04,  2.55s/it]
                                                        

 16%|█▋        | 1977/12000 [1:11:25<7:06:04,  2.55s/it]
 16%|█▋        | 1978/12000 [1:11:28<6:47:26,  2.44s/it]
                                                        
{'loss': '0.9211', 'grad_norm': '0.3116', 'learning_rate': '0.0001934', 'entropy': '0.9461', 'num_tokens': '1.05e+07', 'mean_token_accuracy': '0.7643', 'epoch': '0.01709'}
{'loss': '0.8827', 'grad_norm': '0.3717', 'learning_rate': '0.0001933', 'entropy': '0.9402', 'num_tokens': '1.05e+07', 'mean_token_accuracy': '0.7651', 'epoch': '0.0171'}
{'loss': '0.9136', 'grad_norm': '0.3413', 'learning_rate': '0.0001933', 'entropy': '0.9304', 'num_tokens': '1.051e+07', 'mean_token_accuracy': '0.7675', 'epoch': '0.01711'}
{'loss': '0.9093', 'grad_norm': '0.4016', 'learning_rate': '0.0001933', 'entropy': '0.9119', 'num_tokens': '1.051e+07', 'mean_token_accuracy': '0.7732', 'epoch': '0.01712'}
{'loss': '0.6873', 'grad_norm': '0.3599', 'learning_rate': '0.0001933', 'entropy': '0.6772', 'num_tokens': '1.052e+07', 'mean_token_accuracy': '0.8299', 'epoch': '0.01712'}
{'loss': '0.8862', 'grad_norm': '0.4118', 'learning_rate': '0.0001933', 'entropy': '0.9248', 'num_tokens': '1.052e+07', 'mean_token_accuracy': '0.7696', 'epoch': '0.01713'}
{'loss': '1.163', 'grad_norm': '0.4717', 'learning_rate': '0.0001933', 'entropy': '1.184', 'num_tokens': '1.053e+07', 'mean_token_accuracy': '0.7184', 'epoch': '0.01714'}
{'loss': '1.476', 'grad_norm': '0.454', 'learning_rate': '0.0001933', 'entropy': '1.448', 'num_tokens': '1.053e+07', 'mean_token_accuracy': '0.6528', 'epoch': '0.01715'}
{'loss': '0.9986', 'grad_norm': '0.4527', 'learning_rate': '0.0001933', 'entropy': '0.9051', 'num_tokens': '1.054e+07', 'mean_token_accuracy': '0.7777', 'epoch': '0.01716'}
{'loss': '0.9818', 'grad_norm': '0.3549', 'learning_rate': '0.0001933', 'entropy': '0.9208', 'num_tokens': '1.054e+07', 'mean_token_accuracy': '0.7805', 'epoch': '0.01717'}
{'loss': '1.188', 'grad_norm': '0.4215', 'learning_rate': '0.0001933', 'entropy': '1.135', 'num_tokens': '1.055e+07', 'mean_token_accuracy': '0.7172', 'epoch': '0.01718'}
{'loss': '0.9202', 'grad_norm': '0.3764', 'learning_rate': '0.0001932', 'entropy': '0.8995', 'num_tokens': '1.055e+07', 'mean_token_accuracy': '0.7776', 'epoch': '0.01719'}
{'loss': '1.021', 'grad_norm': '0.4193', 'learning_rate': '0.0001932', 'entropy': '1.061', 'num_tokens': '1.055e+07', 'mean_token_accuracy': '0.7433', 'epoch': '0.0172'}
{'loss': '0.5379', 'grad_norm': '0.2944', 'learning_rate': '0.0001932', 'entropy': '0.573', 'num_tokens': '1.056e+07', 'mean_token_accuracy': '0.8599', 'epoch': '0.0172'}
{'loss': '1.133', 'grad_norm': '0.4115', 'learning_rate': '0.0001932', 'entropy': '1.052', 'num_tokens': '1.057e+07', 'mean_token_accuracy': '0.75', 'epoch': '0.01721'}
{'loss': '0.8568', 'grad_norm': '0.3441', 'learning_rate': '0.0001932', 'entropy': '0.781', 'num_tokens': '1.057e+07', 'mean_token_accuracy': '0.8097', 'epoch': '0.01722'}
{'loss': '0.9256', 'grad_norm': '0.3698', 'learning_rate': '0.0001932', 'entropy': '0.953', 'num_tokens': '1.058e+07', 'mean_token_accuracy': '0.7693', 'epoch': '0.01723'}
{'loss': '1.086', 'grad_norm': '0.3844', 'learning_rate': '0.0001932', 'entropy': '1.06', 'num_tokens': '1.058e+07', 'mean_token_accuracy': '0.7417', 'epoch': '0.01724'}
{'loss': '0.8807', 'grad_norm': '0.3911', 'learning_rate': '0.0001932', 'entropy': '0.8779', 'num_tokens': '1.059e+07', 'mean_token_accuracy': '0.7759', 'epoch': '0.01725'}
{'loss': '0.7605', 'grad_norm': '0.3645', 'learning_rate': '0.0001932', 'entropy': '0.7477', 'num_tokens': '1.059e+07', 'mean_token_accuracy': '0.807', 'epoch': '0.01726'}
{'loss': '0.8827', 'grad_norm': '0.3155', 'learning_rate': '0.0001932', 'entropy': '0.9026', 'num_tokens': '1.06e+07', 'mean_token_accuracy': '0.7754', 'epoch': '0.01727'}
{'loss': '0.8866', 'grad_norm': '0.2955', 'learning_rate': '0.0001931', 'entropy': '0.982', 'num_tokens': '1.06e+07', 'mean_token_accuracy': '0.768', 'epoch': '0.01727'}
{'loss': '0.6261', 'grad_norm': '0.3098', 'learning_rate': '0.0001931', 'entropy': '0.7121', 'num_tokens': '1.061e+07', 'mean_token_accuracy': '0.8215', 'epoch': '0.01728'}
{'loss': '0.6893', 'grad_norm': '0.4176', 'learning_rate': '0.0001931', 'entropy': '0.7131', 'num_tokens': '1.061e+07', 'mean_token_accuracy': '0.8305', 'epoch': '0.01729'}
{'loss': '0.9825', 'grad_norm': '0.3638', 'learning_rate': '0.0001931', 'entropy': '0.8722', 'num_tokens': '1.062e+07', 'mean_token_accuracy': '0.7843', 'epoch': '0.0173'}
{'loss': '0.9872', 'grad_norm': '0.413', 'learning_rate': '0.0001931', 'entropy': '0.994', 'num_tokens': '1.063e+07', 'mean_token_accuracy': '0.7629', 'epoch': '0.01731'}
{'loss': '0.7347', 'grad_norm': '0.3407', 'learning_rate': '0.0001931', 'entropy': '0.8185', 'num_tokens': '1.063e+07', 'mean_token_accuracy': '0.7893', 'epoch': '0.01732'}
{'loss': '0.7683', 'grad_norm': '0.4345', 'learning_rate': '0.0001931', 'entropy': '0.7606', 'num_tokens': '1.064e+07', 'mean_token_accuracy': '0.7981', 'epoch': '0.01733'}
{'loss': '1.196', 'grad_norm': '0.4262', 'learning_rate': '0.0001931', 'entropy': '1.148', 'num_tokens': '1.064e+07', 'mean_token_accuracy': '0.7436', 'epoch': '0.01734'}
{'loss': '0.8195', 'grad_norm': '0.3253', 'learning_rate': '0.0001931', 'entropy': '0.8078', 'num_tokens': '1.065e+07', 'mean_token_accuracy': '0.8104', 'epoch': '0.01735'}
{'loss': '0.7656', 'grad_norm': '0.4622', 'learning_rate': '0.0001931', 'entropy': '0.7425', 'num_tokens': '1.065e+07', 'mean_token_accuracy': '0.796', 'epoch': '0.01735'}
{'loss': '1.067', 'grad_norm': '0.3675', 'learning_rate': '0.000193', 'entropy': '1.019', 'num_tokens': '1.066e+07', 'mean_token_accuracy': '0.7402', 'epoch': '0.01736'}
{'loss': '1.107', 'grad_norm': '0.348', 'learning_rate': '0.000193', 'entropy': '1.081', 'num_tokens': '1.067e+07', 'mean_token_accuracy': '0.7446', 'epoch': '0.01737'}
{'loss': '0.9775', 'grad_norm': '0.3038', 'learning_rate': '0.000193', 'entropy': '0.9817', 'num_tokens': '1.067e+07', 'mean_token_accuracy': '0.7621', 'epoch': '0.01738'}
{'loss': '1.21', 'grad_norm': '0.5099', 'learning_rate': '0.000193', 'entropy': '1.208', 'num_tokens': '1.068e+07', 'mean_token_accuracy': '0.7213', 'epoch': '0.01739'}
{'loss': '0.6412', 'grad_norm': '0.3596', 'learning_rate': '0.000193', 'entropy': '0.6156', 'num_tokens': '1.068e+07', 'mean_token_accuracy': '0.8566', 'epoch': '0.0174'}
{'loss': '0.9045', 'grad_norm': '0.3494', 'learning_rate': '0.000193', 'entropy': '0.9498', 'num_tokens': '1.069e+07', 'mean_token_accuracy': '0.7763', 'epoch': '0.01741'}
{'loss': '0.7513', 'grad_norm': '0.3475', 'learning_rate': '0.000193', 'entropy': '0.7942', 'num_tokens': '1.069e+07', 'mean_token_accuracy': '0.8131', 'epoch': '0.01742'}
{'loss': '0.9772', 'grad_norm': '0.4057', 'learning_rate': '0.000193', 'entropy': '1.019', 'num_tokens': '1.07e+07', 'mean_token_accuracy': '0.7592', 'epoch': '0.01743'}
{'loss': '0.9543', 'grad_norm': '0.3709', 'learning_rate': '0.000193', 'entropy': '0.9304', 'num_tokens': '1.07e+07', 'mean_token_accuracy': '0.7658', 'epoch': '0.01743'}
{'loss': '1.384', 'grad_norm': '0.4086', 'learning_rate': '0.000193', 'entropy': '0.9977', 'num_tokens': '1.071e+07', 'mean_token_accuracy': '0.7547', 'epoch': '0.01744'}
{'loss': '0.8271', 'grad_norm': '0.4435', 'learning_rate': '0.0001929', 'entropy': '0.8329', 'num_tokens': '1.071e+07', 'mean_token_accuracy': '0.7725', 'epoch': '0.01745'}
{'loss': '0.8699', 'grad_norm': '0.3642', 'learning_rate': '0.0001929', 'entropy': '0.8005', 'num_tokens': '1.072e+07', 'mean_token_accuracy': '0.7984', 'epoch': '0.01746'}
{'loss': '1.104', 'grad_norm': '0.4769', 'learning_rate': '0.0001929', 'entropy': '1.087', 'num_tokens': '1.072e+07', 'mean_token_accuracy': '0.725', 'epoch': '0.01747'}
{'loss': '1.126', 'grad_norm': '0.4534', 'learning_rate': '0.0001929', 'entropy': '1.062', 'num_tokens': '1.073e+07', 'mean_token_accuracy': '0.7479', 'epoch': '0.01748'}
{'loss': '0.8345', 'grad_norm': '0.3191', 'learning_rate': '0.0001929', 'entropy': '0.8224', 'num_tokens': '1.074e+07', 'mean_token_accuracy': '0.7912', 'epoch': '0.01749'}
{'loss': '0.747', 'grad_norm': '0.3728', 'learning_rate': '0.0001929', 'entropy': '0.7727', 'num_tokens': '1.074e+07', 'mean_token_accuracy': '0.8142', 'epoch': '0.0175'}

 16%|█▋        | 1978/12000 [1:11:28<6:47:26,  2.44s/it]
 16%|█▋        | 1979/12000 [1:11:30<6:37:22,  2.38s/it]
                                                        

 16%|█▋        | 1979/12000 [1:11:30<6:37:22,  2.38s/it]
 16%|█▋        | 1980/12000 [1:11:33<7:22:48,  2.65s/it]
                                                        

 16%|█▋        | 1980/12000 [1:11:33<7:22:48,  2.65s/it]
 17%|█▋        | 1981/12000 [1:11:35<6:59:58,  2.52s/it]
                                                        

 17%|█▋        | 1981/12000 [1:11:35<6:59:58,  2.52s/it]
 17%|█▋        | 1982/12000 [1:11:37<6:37:41,  2.38s/it]
                                                        

 17%|█▋        | 1982/12000 [1:11:37<6:37:41,  2.38s/it]
 17%|█▋        | 1983/12000 [1:11:39<6:23:31,  2.30s/it]
                                                        

 17%|█▋        | 1983/12000 [1:11:39<6:23:31,  2.30s/it]
 17%|█▋        | 1984/12000 [1:11:41<6:00:46,  2.16s/it]
                                                        

 17%|█▋        | 1984/12000 [1:11:41<6:00:46,  2.16s/it]
 17%|█▋        | 1985/12000 [1:11:43<5:31:17,  1.98s/it]
                                                        

 17%|█▋        | 1985/12000 [1:11:43<5:31:17,  1.98s/it]
 17%|█▋        | 1986/12000 [1:11:45<5:12:02,  1.87s/it]
                                                        

 17%|█▋        | 1986/12000 [1:11:45<5:12:02,  1.87s/it]
 17%|█▋        | 1987/12000 [1:11:46<4:49:45,  1.74s/it]
                                                        

 17%|█▋        | 1987/12000 [1:11:46<4:49:45,  1.74s/it]
 17%|█▋        | 1988/12000 [1:11:47<4:37:46,  1.66s/it]
                                                        

 17%|█▋        | 1988/12000 [1:11:47<4:37:46,  1.66s/it]
 17%|█▋        | 1989/12000 [1:11:51<6:02:49,  2.17s/it]
                                                        

 17%|█▋        | 1989/12000 [1:11:51<6:02:49,  2.17s/it]
 17%|█▋        | 1990/12000 [1:11:52<5:36:41,  2.02s/it]
                                                        

 17%|█▋        | 1990/12000 [1:11:52<5:36:41,  2.02s/it]
 17%|█▋        | 1991/12000 [1:11:55<5:51:44,  2.11s/it]
                                                        

 17%|█▋        | 1991/12000 [1:11:55<5:51:44,  2.11s/it]
 17%|█▋        | 1992/12000 [1:11:57<5:39:52,  2.04s/it]
                                                        

 17%|█▋        | 1992/12000 [1:11:57<5:39:52,  2.04s/it]
 17%|█▋        | 1993/12000 [1:11:58<5:18:11,  1.91s/it]
                                                        

 17%|█▋        | 1993/12000 [1:11:58<5:18:11,  1.91s/it]
 17%|█▋        | 1994/12000 [1:12:00<5:01:38,  1.81s/it]
                                                        

 17%|█▋        | 1994/12000 [1:12:00<5:01:38,  1.81s/it]
 17%|█▋        | 1995/12000 [1:12:02<5:23:43,  1.94s/it]
                                                        

 17%|█▋        | 1995/12000 [1:12:02<5:23:43,  1.94s/it]
 17%|█▋        | 1996/12000 [1:12:04<5:07:35,  1.84s/it]
                                                        

 17%|█▋        | 1996/12000 [1:12:04<5:07:35,  1.84s/it]
 17%|█▋        | 1997/12000 [1:12:06<5:41:42,  2.05s/it]
                                                        

 17%|█▋        | 1997/12000 [1:12:06<5:41:42,  2.05s/it]
 17%|█▋        | 1998/12000 [1:12:08<5:17:54,  1.91s/it]
                                                        

 17%|█▋        | 1998/12000 [1:12:08<5:17:54,  1.91s/it]
 17%|█▋        | 1999/12000 [1:12:11<6:39:57,  2.40s/it]
                                                        

 17%|█▋        | 1999/12000 [1:12:11<6:39:57,  2.40s/it]
 17%|█▋        | 2000/12000 [1:12:13<6:11:16,  2.23s/it]
                                                        

 17%|█▋        | 2000/12000 [1:12:13<6:11:16,  2.23s/it]
 17%|█▋        | 2001/12000 [1:12:16<6:37:55,  2.39s/it]
                                                        

 17%|█▋        | 2001/12000 [1:12:16<6:37:55,  2.39s/it]
 17%|█▋        | 2002/12000 [1:12:18<6:00:17,  2.16s/it]
                                                        

 17%|█▋        | 2002/12000 [1:12:18<6:00:17,  2.16s/it]
 17%|█▋        | 2003/12000 [1:12:20<6:00:48,  2.17s/it]
                                                        

 17%|█▋        | 2003/12000 [1:12:20<6:00:48,  2.17s/it]
 17%|█▋        | 2004/12000 [1:12:21<5:30:16,  1.98s/it]
                                                        

 17%|█▋        | 2004/12000 [1:12:21<5:30:16,  1.98s/it]
 17%|█▋        | 2005/12000 [1:12:23<5:11:40,  1.87s/it]
                                                        

 17%|█▋        | 2005/12000 [1:12:23<5:11:40,  1.87s/it]
 17%|█▋        | 2006/12000 [1:12:27<7:23:20,  2.66s/it]
                                                        

 17%|█▋        | 2006/12000 [1:12:27<7:23:20,  2.66s/it]
 17%|█▋        | 2007/12000 [1:12:29<6:50:22,  2.46s/it]
                                                        

 17%|█▋        | 2007/12000 [1:12:29<6:50:22,  2.46s/it]
 17%|█▋        | 2008/12000 [1:12:31<6:08:04,  2.21s/it]
                                                        

 17%|█▋        | 2008/12000 [1:12:31<6:08:04,  2.21s/it]
 17%|█▋        | 2009/12000 [1:12:36<8:10:50,  2.95s/it]
                                                        

 17%|█▋        | 2009/12000 [1:12:36<8:10:50,  2.95s/it]
 17%|█▋        | 2010/12000 [1:12:38<7:49:29,  2.82s/it]
                                                        

 17%|█▋        | 2010/12000 [1:12:38<7:49:29,  2.82s/it]
 17%|█▋        | 2011/12000 [1:12:40<7:16:48,  2.62s/it]
                                                        

 17%|█▋        | 2011/12000 [1:12:40<7:16:48,  2.62s/it]
 17%|█▋        | 2012/12000 [1:12:42<6:35:45,  2.38s/it]
                                                        

 17%|█▋        | 2012/12000 [1:12:42<6:35:45,  2.38s/it]
 17%|█▋        | 2013/12000 [1:12:45<6:45:08,  2.43s/it]
                                                        

 17%|█▋        | 2013/12000 [1:12:45<6:45:08,  2.43s/it]
 17%|█▋        | 2014/12000 [1:12:47<6:13:40,  2.25s/it]
                                                        

 17%|█▋        | 2014/12000 [1:12:47<6:13:40,  2.25s/it]
 17%|█▋        | 2015/12000 [1:12:49<6:45:30,  2.44s/it]
                                                        

 17%|█▋        | 2015/12000 [1:12:49<6:45:30,  2.44s/it]
 17%|█▋        | 2016/12000 [1:12:53<7:24:07,  2.67s/it]
                                                        

 17%|█▋        | 2016/12000 [1:12:53<7:24:07,  2.67s/it]
 17%|█▋        | 2017/12000 [1:12:54<6:35:59,  2.38s/it]
                                                        

 17%|█▋        | 2017/12000 [1:12:54<6:35:59,  2.38s/it]
 17%|█▋        | 2018/12000 [1:12:56<5:56:13,  2.14s/it]
                                                        

 17%|█▋        | 2018/12000 [1:12:56<5:56:13,  2.14s/it]
 17%|█▋        | 2019/12000 [1:12:58<6:02:12,  2.18s/it]
                                                        

 17%|█▋        | 2019/12000 [1:12:58<6:02:12,  2.18s/it]
 17%|█▋        | 2020/12000 [1:13:01<6:13:57,  2.25s/it]
                                                        

 17%|█▋        | 2020/12000 [1:13:01<6:13:57,  2.25s/it]
 17%|█▋        | 2021/12000 [1:13:04<7:20:09,  2.65s/it]
                                                        

 17%|█▋        | 2021/12000 [1:13:04<7:20:09,  2.65s/it]
 17%|█▋        | 2022/12000 [1:13:08<8:23:04,  3.03s/it]
                                                        

 17%|█▋        | 2022/12000 [1:13:08<8:23:04,  3.03s/it]
 17%|█▋        | 2023/12000 [1:13:10<7:19:01,  2.64s/it]
                                                        

 17%|█▋        | 2023/12000 [1:13:10<7:19:01,  2.64s/it]
 17%|█▋        | 2024/12000 [1:13:12<6:58:29,  2.52s/it]
                                                        

 17%|█▋        | 2024/12000 [1:13:12<6:58:29,  2.52s/it]
 17%|█▋        | 2025/12000 [1:13:13<6:02:27,  2.18s/it]
                                                        
{'loss': '1.261', 'grad_norm': '0.3806', 'learning_rate': '0.0001929', 'entropy': '1.283', 'num_tokens': '1.075e+07', 'mean_token_accuracy': '0.6823', 'epoch': '0.0175'}
{'loss': '1.236', 'grad_norm': '0.3854', 'learning_rate': '0.0001929', 'entropy': '1.223', 'num_tokens': '1.075e+07', 'mean_token_accuracy': '0.7145', 'epoch': '0.01751'}
{'loss': '1.249', 'grad_norm': '0.3535', 'learning_rate': '0.0001929', 'entropy': '1.242', 'num_tokens': '1.076e+07', 'mean_token_accuracy': '0.7052', 'epoch': '0.01752'}
{'loss': '0.9228', 'grad_norm': '0.3818', 'learning_rate': '0.0001929', 'entropy': '0.9109', 'num_tokens': '1.077e+07', 'mean_token_accuracy': '0.7797', 'epoch': '0.01753'}
{'loss': '0.9301', 'grad_norm': '0.3803', 'learning_rate': '0.0001928', 'entropy': '0.8798', 'num_tokens': '1.077e+07', 'mean_token_accuracy': '0.7772', 'epoch': '0.01754'}
{'loss': '1.342', 'grad_norm': '0.4673', 'learning_rate': '0.0001928', 'entropy': '1.254', 'num_tokens': '1.078e+07', 'mean_token_accuracy': '0.7002', 'epoch': '0.01755'}
{'loss': '1.004', 'grad_norm': '0.4375', 'learning_rate': '0.0001928', 'entropy': '1.002', 'num_tokens': '1.078e+07', 'mean_token_accuracy': '0.754', 'epoch': '0.01756'}
{'loss': '1.201', 'grad_norm': '0.4669', 'learning_rate': '0.0001928', 'entropy': '1.248', 'num_tokens': '1.079e+07', 'mean_token_accuracy': '0.7099', 'epoch': '0.01757'}
{'loss': '1.341', 'grad_norm': '0.4212', 'learning_rate': '0.0001928', 'entropy': '1.342', 'num_tokens': '1.079e+07', 'mean_token_accuracy': '0.7023', 'epoch': '0.01758'}
{'loss': '0.7976', 'grad_norm': '0.3774', 'learning_rate': '0.0001928', 'entropy': '0.809', 'num_tokens': '1.08e+07', 'mean_token_accuracy': '0.7946', 'epoch': '0.01758'}
{'loss': '0.6111', 'grad_norm': '0.2811', 'learning_rate': '0.0001928', 'entropy': '0.6014', 'num_tokens': '1.08e+07', 'mean_token_accuracy': '0.8412', 'epoch': '0.01759'}
{'loss': '1.004', 'grad_norm': '0.421', 'learning_rate': '0.0001928', 'entropy': '0.8991', 'num_tokens': '1.081e+07', 'mean_token_accuracy': '0.7803', 'epoch': '0.0176'}
{'loss': '1.237', 'grad_norm': '0.4184', 'learning_rate': '0.0001928', 'entropy': '1.15', 'num_tokens': '1.081e+07', 'mean_token_accuracy': '0.7225', 'epoch': '0.01761'}
{'loss': '0.922', 'grad_norm': '0.3579', 'learning_rate': '0.0001928', 'entropy': '0.9621', 'num_tokens': '1.082e+07', 'mean_token_accuracy': '0.7598', 'epoch': '0.01762'}
{'loss': '1.23', 'grad_norm': '0.4743', 'learning_rate': '0.0001927', 'entropy': '1.127', 'num_tokens': '1.082e+07', 'mean_token_accuracy': '0.7275', 'epoch': '0.01763'}
{'loss': '0.6784', 'grad_norm': '0.394', 'learning_rate': '0.0001927', 'entropy': '0.7377', 'num_tokens': '1.083e+07', 'mean_token_accuracy': '0.8147', 'epoch': '0.01764'}
{'loss': '0.7776', 'grad_norm': '0.3632', 'learning_rate': '0.0001927', 'entropy': '0.772', 'num_tokens': '1.083e+07', 'mean_token_accuracy': '0.8077', 'epoch': '0.01765'}
{'loss': '1.101', 'grad_norm': '0.4658', 'learning_rate': '0.0001927', 'entropy': '1.041', 'num_tokens': '1.084e+07', 'mean_token_accuracy': '0.7387', 'epoch': '0.01766'}
{'loss': '0.6424', 'grad_norm': '0.3516', 'learning_rate': '0.0001927', 'entropy': '0.6382', 'num_tokens': '1.084e+07', 'mean_token_accuracy': '0.829', 'epoch': '0.01766'}
{'loss': '1.094', 'grad_norm': '0.5057', 'learning_rate': '0.0001927', 'entropy': '0.9923', 'num_tokens': '1.085e+07', 'mean_token_accuracy': '0.7549', 'epoch': '0.01767'}
{'loss': '0.986', 'grad_norm': '0.4265', 'learning_rate': '0.0001927', 'entropy': '1.035', 'num_tokens': '1.085e+07', 'mean_token_accuracy': '0.777', 'epoch': '0.01768'}
{'loss': '0.9828', 'grad_norm': '0.3957', 'learning_rate': '0.0001927', 'entropy': '1.002', 'num_tokens': '1.086e+07', 'mean_token_accuracy': '0.7571', 'epoch': '0.01769'}
{'loss': '1.142', 'grad_norm': '0.4086', 'learning_rate': '0.0001927', 'entropy': '1.183', 'num_tokens': '1.086e+07', 'mean_token_accuracy': '0.7275', 'epoch': '0.0177'}
{'loss': '0.5632', 'grad_norm': '0.3076', 'learning_rate': '0.0001926', 'entropy': '0.6388', 'num_tokens': '1.087e+07', 'mean_token_accuracy': '0.832', 'epoch': '0.01771'}
{'loss': '0.8666', 'grad_norm': '0.418', 'learning_rate': '0.0001926', 'entropy': '0.9009', 'num_tokens': '1.087e+07', 'mean_token_accuracy': '0.777', 'epoch': '0.01772'}
{'loss': '1.021', 'grad_norm': '0.4665', 'learning_rate': '0.0001926', 'entropy': '1.108', 'num_tokens': '1.088e+07', 'mean_token_accuracy': '0.7208', 'epoch': '0.01773'}
{'loss': '0.7052', 'grad_norm': '0.3665', 'learning_rate': '0.0001926', 'entropy': '0.7086', 'num_tokens': '1.088e+07', 'mean_token_accuracy': '0.8208', 'epoch': '0.01774'}
{'loss': '0.85', 'grad_norm': '0.3492', 'learning_rate': '0.0001926', 'entropy': '0.8815', 'num_tokens': '1.089e+07', 'mean_token_accuracy': '0.7739', 'epoch': '0.01774'}
{'loss': '1.753', 'grad_norm': '1.193', 'learning_rate': '0.0001926', 'entropy': '1.23', 'num_tokens': '1.09e+07', 'mean_token_accuracy': '0.7213', 'epoch': '0.01775'}
{'loss': '1.054', 'grad_norm': '0.3987', 'learning_rate': '0.0001926', 'entropy': '1.004', 'num_tokens': '1.09e+07', 'mean_token_accuracy': '0.7488', 'epoch': '0.01776'}
{'loss': '1.02', 'grad_norm': '0.3767', 'learning_rate': '0.0001926', 'entropy': '0.956', 'num_tokens': '1.091e+07', 'mean_token_accuracy': '0.7548', 'epoch': '0.01777'}
{'loss': '1.38', 'grad_norm': '0.4012', 'learning_rate': '0.0001926', 'entropy': '1.217', 'num_tokens': '1.092e+07', 'mean_token_accuracy': '0.7128', 'epoch': '0.01778'}
{'loss': '1.103', 'grad_norm': '0.466', 'learning_rate': '0.0001926', 'entropy': '1.05', 'num_tokens': '1.092e+07', 'mean_token_accuracy': '0.7334', 'epoch': '0.01779'}
{'loss': '0.9184', 'grad_norm': '0.4036', 'learning_rate': '0.0001925', 'entropy': '0.9297', 'num_tokens': '1.093e+07', 'mean_token_accuracy': '0.765', 'epoch': '0.0178'}
{'loss': '1.191', 'grad_norm': '0.3866', 'learning_rate': '0.0001925', 'entropy': '1.147', 'num_tokens': '1.094e+07', 'mean_token_accuracy': '0.7271', 'epoch': '0.01781'}
{'loss': '0.991', 'grad_norm': '0.3738', 'learning_rate': '0.0001925', 'entropy': '1.257', 'num_tokens': '1.094e+07', 'mean_token_accuracy': '0.7', 'epoch': '0.01781'}
{'loss': '0.6867', 'grad_norm': '0.3047', 'learning_rate': '0.0001925', 'entropy': '0.7073', 'num_tokens': '1.095e+07', 'mean_token_accuracy': '0.8146', 'epoch': '0.01782'}
{'loss': '1.36', 'grad_norm': '0.3813', 'learning_rate': '0.0001925', 'entropy': '1.209', 'num_tokens': '1.095e+07', 'mean_token_accuracy': '0.7145', 'epoch': '0.01783'}
{'loss': '0.8911', 'grad_norm': '0.3592', 'learning_rate': '0.0001925', 'entropy': '0.9973', 'num_tokens': '1.096e+07', 'mean_token_accuracy': '0.7528', 'epoch': '0.01784'}
{'loss': '1.138', 'grad_norm': '0.3994', 'learning_rate': '0.0001925', 'entropy': '1.119', 'num_tokens': '1.096e+07', 'mean_token_accuracy': '0.7328', 'epoch': '0.01785'}
{'loss': '0.7946', 'grad_norm': '0.3466', 'learning_rate': '0.0001925', 'entropy': '0.7832', 'num_tokens': '1.097e+07', 'mean_token_accuracy': '0.809', 'epoch': '0.01786'}
{'loss': '1.409', 'grad_norm': '0.5274', 'learning_rate': '0.0001925', 'entropy': '1.378', 'num_tokens': '1.098e+07', 'mean_token_accuracy': '0.6692', 'epoch': '0.01787'}
{'loss': '0.8642', 'grad_norm': '0.5524', 'learning_rate': '0.0001925', 'entropy': '0.8554', 'num_tokens': '1.098e+07', 'mean_token_accuracy': '0.7995', 'epoch': '0.01788'}
{'loss': '0.7468', 'grad_norm': '0.3934', 'learning_rate': '0.0001924', 'entropy': '0.8203', 'num_tokens': '1.099e+07', 'mean_token_accuracy': '0.7958', 'epoch': '0.01789'}
{'loss': '0.5686', 'grad_norm': '0.3604', 'learning_rate': '0.0001924', 'entropy': '0.6964', 'num_tokens': '1.099e+07', 'mean_token_accuracy': '0.8158', 'epoch': '0.01789'}
{'loss': '1.19', 'grad_norm': '0.4265', 'learning_rate': '0.0001924', 'entropy': '1.183', 'num_tokens': '1.1e+07', 'mean_token_accuracy': '0.7086', 'epoch': '0.0179'}
{'loss': '0.8907', 'grad_norm': '0.4455', 'learning_rate': '0.0001924', 'entropy': '0.8569', 'num_tokens': '1.1e+07', 'mean_token_accuracy': '0.7886', 'epoch': '0.01791'}

 17%|█▋        | 2025/12000 [1:13:13<6:02:27,  2.18s/it]
 17%|█▋        | 2026/12000 [1:13:15<5:28:44,  1.98s/it]
                                                        

 17%|█▋        | 2026/12000 [1:13:15<5:28:44,  1.98s/it]
 17%|█▋        | 2027/12000 [1:13:17<5:55:13,  2.14s/it]
                                                        

 17%|█▋        | 2027/12000 [1:13:17<5:55:13,  2.14s/it]
 17%|█▋        | 2028/12000 [1:13:19<5:41:50,  2.06s/it]
                                                        

 17%|█▋        | 2028/12000 [1:13:19<5:41:50,  2.06s/it]
 17%|█▋        | 2029/12000 [1:13:21<5:09:52,  1.86s/it]
                                                        

 17%|█▋        | 2029/12000 [1:13:21<5:09:52,  1.86s/it]
 17%|█▋        | 2030/12000 [1:13:22<5:02:07,  1.82s/it]
                                                        

 17%|█▋        | 2030/12000 [1:13:22<5:02:07,  1.82s/it]
 17%|█▋        | 2031/12000 [1:13:24<4:39:44,  1.68s/it]
                                                        

 17%|█▋        | 2031/12000 [1:13:24<4:39:44,  1.68s/it]
 17%|█▋        | 2032/12000 [1:13:26<4:54:59,  1.78s/it]
                                                        

 17%|█▋        | 2032/12000 [1:13:26<4:54:59,  1.78s/it]
 17%|█▋        | 2033/12000 [1:13:28<5:03:45,  1.83s/it]
                                                        

 17%|█▋        | 2033/12000 [1:13:28<5:03:45,  1.83s/it]
 17%|█▋        | 2034/12000 [1:13:29<4:45:24,  1.72s/it]
                                                        

 17%|█▋        | 2034/12000 [1:13:29<4:45:24,  1.72s/it]
 17%|█▋        | 2035/12000 [1:13:31<4:45:08,  1.72s/it]
                                                        

 17%|█▋        | 2035/12000 [1:13:31<4:45:08,  1.72s/it]
 17%|█▋        | 2036/12000 [1:13:33<5:07:44,  1.85s/it]
                                                        

 17%|█▋        | 2036/12000 [1:13:33<5:07:44,  1.85s/it]
 17%|█▋        | 2037/12000 [1:13:35<4:59:32,  1.80s/it]
                                                        

 17%|█▋        | 2037/12000 [1:13:35<4:59:32,  1.80s/it]
 17%|█▋        | 2038/12000 [1:13:37<5:35:05,  2.02s/it]
                                                        

 17%|█▋        | 2038/12000 [1:13:37<5:35:05,  2.02s/it]
 17%|█▋        | 2039/12000 [1:13:39<5:08:06,  1.86s/it]
                                                        

 17%|█▋        | 2039/12000 [1:13:39<5:08:06,  1.86s/it]
 17%|█▋        | 2040/12000 [1:13:41<5:22:37,  1.94s/it]
                                                        

 17%|█▋        | 2040/12000 [1:13:41<5:22:37,  1.94s/it]
 17%|█▋        | 2041/12000 [1:13:43<5:41:13,  2.06s/it]
                                                        

 17%|█▋        | 2041/12000 [1:13:43<5:41:13,  2.06s/it]
 17%|█▋        | 2042/12000 [1:13:46<6:29:39,  2.35s/it]
                                                        

 17%|█▋        | 2042/12000 [1:13:46<6:29:39,  2.35s/it]
 17%|█▋        | 2043/12000 [1:13:50<7:48:37,  2.82s/it]
                                                        

 17%|█▋        | 2043/12000 [1:13:50<7:48:37,  2.82s/it]
 17%|█▋        | 2044/12000 [1:13:52<7:15:12,  2.62s/it]
                                                        

 17%|█▋        | 2044/12000 [1:13:52<7:15:12,  2.62s/it]
 17%|█▋        | 2045/12000 [1:13:55<7:06:19,  2.57s/it]
                                                        

 17%|█▋        | 2045/12000 [1:13:55<7:06:19,  2.57s/it]
 17%|█▋        | 2046/12000 [1:13:56<6:14:27,  2.26s/it]
                                                        

 17%|█▋        | 2046/12000 [1:13:56<6:14:27,  2.26s/it]
 17%|█▋        | 2047/12000 [1:13:58<6:05:19,  2.20s/it]
                                                        

 17%|█▋        | 2047/12000 [1:13:58<6:05:19,  2.20s/it]
 17%|█▋        | 2048/12000 [1:14:01<6:06:19,  2.21s/it]
                                                        

 17%|█▋        | 2048/12000 [1:14:01<6:06:19,  2.21s/it]
 17%|█▋        | 2049/12000 [1:14:02<5:34:03,  2.01s/it]
                                                        

 17%|█▋        | 2049/12000 [1:14:02<5:34:03,  2.01s/it]
 17%|█▋        | 2050/12000 [1:14:04<5:34:29,  2.02s/it]
                                                        

 17%|█▋        | 2050/12000 [1:14:04<5:34:29,  2.02s/it]
 17%|█▋        | 2051/12000 [1:14:08<6:57:58,  2.52s/it]
                                                        

 17%|█▋        | 2051/12000 [1:14:08<6:57:58,  2.52s/it]
 17%|█▋        | 2052/12000 [1:14:10<6:09:18,  2.23s/it]
                                                        

 17%|█▋        | 2052/12000 [1:14:10<6:09:18,  2.23s/it]
 17%|█▋        | 2053/12000 [1:14:11<5:25:52,  1.97s/it]
                                                        

 17%|█▋        | 2053/12000 [1:14:11<5:25:52,  1.97s/it]
 17%|█▋        | 2054/12000 [1:14:12<5:09:20,  1.87s/it]
                                                        

 17%|█▋        | 2054/12000 [1:14:12<5:09:20,  1.87s/it]
 17%|█▋        | 2055/12000 [1:14:14<4:55:49,  1.78s/it]
                                                        

 17%|█▋        | 2055/12000 [1:14:14<4:55:49,  1.78s/it]
 17%|█▋        | 2056/12000 [1:14:16<4:41:19,  1.70s/it]
                                                        

 17%|█▋        | 2056/12000 [1:14:16<4:41:19,  1.70s/it]
 17%|█▋        | 2057/12000 [1:14:18<5:08:50,  1.86s/it]
                                                        

 17%|█▋        | 2057/12000 [1:14:18<5:08:50,  1.86s/it]
 17%|█▋        | 2058/12000 [1:14:20<5:30:24,  1.99s/it]
                                                        

 17%|█▋        | 2058/12000 [1:14:20<5:30:24,  1.99s/it]
 17%|█▋        | 2059/12000 [1:14:22<5:04:31,  1.84s/it]
                                                        

 17%|█▋        | 2059/12000 [1:14:22<5:04:31,  1.84s/it]
 17%|█▋        | 2060/12000 [1:14:23<4:53:54,  1.77s/it]
                                                        

 17%|█▋        | 2060/12000 [1:14:23<4:53:54,  1.77s/it]
 17%|█▋        | 2061/12000 [1:14:25<4:44:10,  1.72s/it]
                                                        

 17%|█▋        | 2061/12000 [1:14:25<4:44:10,  1.72s/it]
 17%|█▋        | 2062/12000 [1:14:27<4:52:33,  1.77s/it]
                                                        

 17%|█▋        | 2062/12000 [1:14:27<4:52:33,  1.77s/it]
 17%|█▋        | 2063/12000 [1:14:30<6:04:56,  2.20s/it]
                                                        

 17%|█▋        | 2063/12000 [1:14:30<6:04:56,  2.20s/it]
 17%|█▋        | 2064/12000 [1:14:33<6:48:00,  2.46s/it]
                                                        

 17%|█▋        | 2064/12000 [1:14:33<6:48:00,  2.46s/it]
 17%|█▋        | 2065/12000 [1:14:37<7:45:35,  2.81s/it]
                                                        

 17%|█▋        | 2065/12000 [1:14:37<7:45:35,  2.81s/it]
 17%|█▋        | 2066/12000 [1:14:38<6:56:59,  2.52s/it]
                                                        

 17%|█▋        | 2066/12000 [1:14:38<6:56:59,  2.52s/it]
 17%|█▋        | 2067/12000 [1:14:40<5:59:34,  2.17s/it]
                                                        

 17%|█▋        | 2067/12000 [1:14:40<5:59:34,  2.17s/it]
 17%|█▋        | 2068/12000 [1:14:43<6:42:28,  2.43s/it]
                                                        

 17%|█▋        | 2068/12000 [1:14:43<6:42:28,  2.43s/it]
 17%|█▋        | 2069/12000 [1:14:46<7:23:13,  2.68s/it]
                                                        

 17%|█▋        | 2069/12000 [1:14:46<7:23:13,  2.68s/it]
 17%|█▋        | 2070/12000 [1:14:48<6:38:19,  2.41s/it]
                                                        

 17%|█▋        | 2070/12000 [1:14:48<6:38:19,  2.41s/it]
 17%|█▋        | 2071/12000 [1:14:51<7:06:27,  2.58s/it]
                                                        

 17%|█▋        | 2071/12000 [1:14:51<7:06:27,  2.58s/it]
 17%|█▋        | 2072/12000 [1:14:53<7:05:41,  2.57s/it]
                                                        
{'loss': '1.128', 'grad_norm': '0.3419', 'learning_rate': '0.0001924', 'entropy': '1.089', 'num_tokens': '1.101e+07', 'mean_token_accuracy': '0.7251', 'epoch': '0.01792'}
{'loss': '0.7822', 'grad_norm': '0.3508', 'learning_rate': '0.0001924', 'entropy': '0.7259', 'num_tokens': '1.101e+07', 'mean_token_accuracy': '0.8211', 'epoch': '0.01793'}
{'loss': '0.8132', 'grad_norm': '0.3385', 'learning_rate': '0.0001924', 'entropy': '0.9287', 'num_tokens': '1.102e+07', 'mean_token_accuracy': '0.7623', 'epoch': '0.01794'}
{'loss': '0.8146', 'grad_norm': '0.4115', 'learning_rate': '0.0001924', 'entropy': '0.7696', 'num_tokens': '1.103e+07', 'mean_token_accuracy': '0.7957', 'epoch': '0.01795'}
{'loss': '0.6367', 'grad_norm': '0.3526', 'learning_rate': '0.0001924', 'entropy': '0.7112', 'num_tokens': '1.103e+07', 'mean_token_accuracy': '0.8221', 'epoch': '0.01796'}
{'loss': '0.7676', 'grad_norm': '0.3162', 'learning_rate': '0.0001923', 'entropy': '0.72', 'num_tokens': '1.103e+07', 'mean_token_accuracy': '0.8175', 'epoch': '0.01797'}
{'loss': '1.065', 'grad_norm': '0.4557', 'learning_rate': '0.0001923', 'entropy': '1.097', 'num_tokens': '1.104e+07', 'mean_token_accuracy': '0.7433', 'epoch': '0.01797'}
{'loss': '1.137', 'grad_norm': '0.3638', 'learning_rate': '0.0001923', 'entropy': '1.112', 'num_tokens': '1.104e+07', 'mean_token_accuracy': '0.737', 'epoch': '0.01798'}
{'loss': '0.8019', 'grad_norm': '0.306', 'learning_rate': '0.0001923', 'entropy': '0.8807', 'num_tokens': '1.105e+07', 'mean_token_accuracy': '0.7792', 'epoch': '0.01799'}
{'loss': '0.73', 'grad_norm': '0.3117', 'learning_rate': '0.0001923', 'entropy': '0.7647', 'num_tokens': '1.105e+07', 'mean_token_accuracy': '0.8031', 'epoch': '0.018'}
{'loss': '0.5062', 'grad_norm': '0.2744', 'learning_rate': '0.0001923', 'entropy': '0.5366', 'num_tokens': '1.106e+07', 'mean_token_accuracy': '0.8545', 'epoch': '0.01801'}
{'loss': '0.8974', 'grad_norm': '0.3311', 'learning_rate': '0.0001923', 'entropy': '0.8361', 'num_tokens': '1.107e+07', 'mean_token_accuracy': '0.7791', 'epoch': '0.01802'}
{'loss': '1.053', 'grad_norm': '0.3916', 'learning_rate': '0.0001923', 'entropy': '1.106', 'num_tokens': '1.107e+07', 'mean_token_accuracy': '0.7351', 'epoch': '0.01803'}
{'loss': '1.068', 'grad_norm': '0.394', 'learning_rate': '0.0001923', 'entropy': '1.054', 'num_tokens': '1.108e+07', 'mean_token_accuracy': '0.7349', 'epoch': '0.01804'}
{'loss': '0.8721', 'grad_norm': '0.3819', 'learning_rate': '0.0001923', 'entropy': '0.8785', 'num_tokens': '1.108e+07', 'mean_token_accuracy': '0.7896', 'epoch': '0.01804'}
{'loss': '0.9863', 'grad_norm': '0.4027', 'learning_rate': '0.0001922', 'entropy': '0.9433', 'num_tokens': '1.109e+07', 'mean_token_accuracy': '0.776', 'epoch': '0.01805'}
{'loss': '1.036', 'grad_norm': '0.4116', 'learning_rate': '0.0001922', 'entropy': '1.011', 'num_tokens': '1.109e+07', 'mean_token_accuracy': '0.7448', 'epoch': '0.01806'}
{'loss': '0.8161', 'grad_norm': '0.33', 'learning_rate': '0.0001922', 'entropy': '0.7437', 'num_tokens': '1.11e+07', 'mean_token_accuracy': '0.8172', 'epoch': '0.01807'}
{'loss': '0.9132', 'grad_norm': '0.378', 'learning_rate': '0.0001922', 'entropy': '0.9023', 'num_tokens': '1.111e+07', 'mean_token_accuracy': '0.7633', 'epoch': '0.01808'}
{'loss': '0.906', 'grad_norm': '0.378', 'learning_rate': '0.0001922', 'entropy': '0.8763', 'num_tokens': '1.111e+07', 'mean_token_accuracy': '0.7638', 'epoch': '0.01809'}
{'loss': '0.9288', 'grad_norm': '0.327', 'learning_rate': '0.0001922', 'entropy': '0.9265', 'num_tokens': '1.112e+07', 'mean_token_accuracy': '0.7736', 'epoch': '0.0181'}
{'loss': '1.021', 'grad_norm': '0.497', 'learning_rate': '0.0001922', 'entropy': '1.012', 'num_tokens': '1.112e+07', 'mean_token_accuracy': '0.7668', 'epoch': '0.01811'}
{'loss': '0.8338', 'grad_norm': '0.3912', 'learning_rate': '0.0001922', 'entropy': '0.9529', 'num_tokens': '1.113e+07', 'mean_token_accuracy': '0.7639', 'epoch': '0.01812'}
{'loss': '0.8467', 'grad_norm': '0.3432', 'learning_rate': '0.0001922', 'entropy': '0.8797', 'num_tokens': '1.113e+07', 'mean_token_accuracy': '0.7851', 'epoch': '0.01812'}
{'loss': '0.9395', 'grad_norm': '0.3486', 'learning_rate': '0.0001921', 'entropy': '0.926', 'num_tokens': '1.114e+07', 'mean_token_accuracy': '0.7695', 'epoch': '0.01813'}
{'loss': '0.8094', 'grad_norm': '0.3678', 'learning_rate': '0.0001921', 'entropy': '0.8174', 'num_tokens': '1.114e+07', 'mean_token_accuracy': '0.7992', 'epoch': '0.01814'}
{'loss': '1.213', 'grad_norm': '0.436', 'learning_rate': '0.0001921', 'entropy': '1.065', 'num_tokens': '1.115e+07', 'mean_token_accuracy': '0.7462', 'epoch': '0.01815'}
{'loss': '0.9747', 'grad_norm': '0.4732', 'learning_rate': '0.0001921', 'entropy': '0.9233', 'num_tokens': '1.116e+07', 'mean_token_accuracy': '0.7689', 'epoch': '0.01816'}
{'loss': '0.8749', 'grad_norm': '0.4374', 'learning_rate': '0.0001921', 'entropy': '0.9453', 'num_tokens': '1.116e+07', 'mean_token_accuracy': '0.7737', 'epoch': '0.01817'}
{'loss': '1.006', 'grad_norm': '0.4285', 'learning_rate': '0.0001921', 'entropy': '1.07', 'num_tokens': '1.116e+07', 'mean_token_accuracy': '0.7413', 'epoch': '0.01818'}
{'loss': '0.8763', 'grad_norm': '0.444', 'learning_rate': '0.0001921', 'entropy': '1.045', 'num_tokens': '1.117e+07', 'mean_token_accuracy': '0.7461', 'epoch': '0.01819'}
{'loss': '0.7936', 'grad_norm': '0.4376', 'learning_rate': '0.0001921', 'entropy': '0.8049', 'num_tokens': '1.117e+07', 'mean_token_accuracy': '0.7824', 'epoch': '0.0182'}
{'loss': '0.9626', 'grad_norm': '0.4132', 'learning_rate': '0.0001921', 'entropy': '0.999', 'num_tokens': '1.118e+07', 'mean_token_accuracy': '0.7726', 'epoch': '0.0182'}
{'loss': '1.099', 'grad_norm': '0.4015', 'learning_rate': '0.000192', 'entropy': '1.107', 'num_tokens': '1.118e+07', 'mean_token_accuracy': '0.7389', 'epoch': '0.01821'}
{'loss': '0.7428', 'grad_norm': '0.2898', 'learning_rate': '0.000192', 'entropy': '0.7948', 'num_tokens': '1.119e+07', 'mean_token_accuracy': '0.8021', 'epoch': '0.01822'}
{'loss': '0.7437', 'grad_norm': '0.4233', 'learning_rate': '0.000192', 'entropy': '0.8134', 'num_tokens': '1.119e+07', 'mean_token_accuracy': '0.7872', 'epoch': '0.01823'}
{'loss': '0.7561', 'grad_norm': '0.3512', 'learning_rate': '0.000192', 'entropy': '0.7657', 'num_tokens': '1.12e+07', 'mean_token_accuracy': '0.8184', 'epoch': '0.01824'}
{'loss': '0.972', 'grad_norm': '0.3501', 'learning_rate': '0.000192', 'entropy': '1.019', 'num_tokens': '1.12e+07', 'mean_token_accuracy': '0.7685', 'epoch': '0.01825'}
{'loss': '0.7329', 'grad_norm': '1.746', 'learning_rate': '0.000192', 'entropy': '0.7173', 'num_tokens': '1.121e+07', 'mean_token_accuracy': '0.815', 'epoch': '0.01826'}
{'loss': '0.8492', 'grad_norm': '0.3577', 'learning_rate': '0.000192', 'entropy': '1.04', 'num_tokens': '1.122e+07', 'mean_token_accuracy': '0.7534', 'epoch': '0.01827'}
{'loss': '1.289', 'grad_norm': '0.493', 'learning_rate': '0.000192', 'entropy': '1.326', 'num_tokens': '1.122e+07', 'mean_token_accuracy': '0.7029', 'epoch': '0.01827'}
{'loss': '1.143', 'grad_norm': '0.3936', 'learning_rate': '0.000192', 'entropy': '1.139', 'num_tokens': '1.123e+07', 'mean_token_accuracy': '0.7278', 'epoch': '0.01828'}
{'loss': '1.077', 'grad_norm': '0.504', 'learning_rate': '0.0001919', 'entropy': '1.07', 'num_tokens': '1.123e+07', 'mean_token_accuracy': '0.7371', 'epoch': '0.01829'}
{'loss': '1.235', 'grad_norm': '0.8039', 'learning_rate': '0.0001919', 'entropy': '0.9321', 'num_tokens': '1.124e+07', 'mean_token_accuracy': '0.7556', 'epoch': '0.0183'}
{'loss': '1.009', 'grad_norm': '0.3349', 'learning_rate': '0.0001919', 'entropy': '1.004', 'num_tokens': '1.125e+07', 'mean_token_accuracy': '0.7654', 'epoch': '0.01831'}
{'loss': '0.6099', 'grad_norm': '0.2875', 'learning_rate': '0.0001919', 'entropy': '0.631', 'num_tokens': '1.125e+07', 'mean_token_accuracy': '0.8385', 'epoch': '0.01832'}
{'loss': '0.7311', 'grad_norm': '0.3773', 'learning_rate': '0.0001919', 'entropy': '0.7568', 'num_tokens': '1.126e+07', 'mean_token_accuracy': '0.8113', 'epoch': '0.01833'}

 17%|█▋        | 2072/12000 [1:14:53<7:05:41,  2.57s/it]
 17%|█▋        | 2073/12000 [1:14:55<6:27:15,  2.34s/it]
                                                        

 17%|█▋        | 2073/12000 [1:14:55<6:27:15,  2.34s/it]
 17%|█▋        | 2074/12000 [1:14:59<7:27:34,  2.71s/it]
                                                        

 17%|█▋        | 2074/12000 [1:14:59<7:27:34,  2.71s/it]
 17%|█▋        | 2075/12000 [1:15:01<6:39:20,  2.41s/it]
                                                        

 17%|█▋        | 2075/12000 [1:15:01<6:39:20,  2.41s/it]
 17%|█▋        | 2076/12000 [1:15:02<6:11:13,  2.24s/it]
                                                        

 17%|█▋        | 2076/12000 [1:15:02<6:11:13,  2.24s/it]
 17%|█▋        | 2077/12000 [1:15:05<6:09:32,  2.23s/it]
                                                        

 17%|█▋        | 2077/12000 [1:15:05<6:09:32,  2.23s/it]
 17%|█▋        | 2078/12000 [1:15:07<6:10:57,  2.24s/it]
                                                        

 17%|█▋        | 2078/12000 [1:15:07<6:10:57,  2.24s/it]
 17%|█▋        | 2079/12000 [1:15:09<6:14:50,  2.27s/it]
                                                        

 17%|█▋        | 2079/12000 [1:15:09<6:14:50,  2.27s/it]
 17%|█▋        | 2080/12000 [1:15:11<5:51:18,  2.12s/it]
                                                        

 17%|█▋        | 2080/12000 [1:15:11<5:51:18,  2.12s/it]
 17%|█▋        | 2081/12000 [1:15:13<5:37:54,  2.04s/it]
                                                        

 17%|█▋        | 2081/12000 [1:15:13<5:37:54,  2.04s/it]
 17%|█▋        | 2082/12000 [1:15:14<5:05:24,  1.85s/it]
                                                        

 17%|█▋        | 2082/12000 [1:15:14<5:05:24,  1.85s/it]
 17%|█▋        | 2083/12000 [1:15:18<6:41:36,  2.43s/it]
                                                        

 17%|█▋        | 2083/12000 [1:15:18<6:41:36,  2.43s/it]
 17%|█▋        | 2084/12000 [1:15:19<5:49:40,  2.12s/it]
                                                        

 17%|█▋        | 2084/12000 [1:15:19<5:49:40,  2.12s/it]
 17%|█▋        | 2085/12000 [1:15:21<5:25:32,  1.97s/it]
                                                        

 17%|█▋        | 2085/12000 [1:15:21<5:25:32,  1.97s/it]
 17%|█▋        | 2086/12000 [1:15:23<5:11:59,  1.89s/it]
                                                        

 17%|█▋        | 2086/12000 [1:15:23<5:11:59,  1.89s/it]
 17%|█▋        | 2087/12000 [1:15:26<6:33:32,  2.38s/it]
                                                        

 17%|█▋        | 2087/12000 [1:15:26<6:33:32,  2.38s/it]
 17%|█▋        | 2088/12000 [1:15:28<5:50:22,  2.12s/it]
                                                        

 17%|█▋        | 2088/12000 [1:15:28<5:50:22,  2.12s/it]
 17%|█▋        | 2089/12000 [1:15:30<5:35:09,  2.03s/it]
                                                        

 17%|█▋        | 2089/12000 [1:15:30<5:35:09,  2.03s/it]
 17%|█▋        | 2090/12000 [1:15:32<5:39:26,  2.06s/it]
                                                        

 17%|█▋        | 2090/12000 [1:15:32<5:39:26,  2.06s/it]
 17%|█▋        | 2091/12000 [1:15:33<5:13:23,  1.90s/it]
                                                        

 17%|█▋        | 2091/12000 [1:15:33<5:13:23,  1.90s/it]
 17%|█▋        | 2092/12000 [1:15:36<5:34:14,  2.02s/it]
                                                        

 17%|█▋        | 2092/12000 [1:15:36<5:34:14,  2.02s/it]
 17%|█▋        | 2093/12000 [1:15:37<5:19:58,  1.94s/it]
                                                        

 17%|█▋        | 2093/12000 [1:15:37<5:19:58,  1.94s/it]
 17%|█▋        | 2094/12000 [1:15:39<5:00:12,  1.82s/it]
                                                        

 17%|█▋        | 2094/12000 [1:15:39<5:00:12,  1.82s/it]
 17%|█▋        | 2095/12000 [1:15:42<5:58:09,  2.17s/it]
                                                        

 17%|█▋        | 2095/12000 [1:15:42<5:58:09,  2.17s/it]
 17%|█▋        | 2096/12000 [1:15:43<5:35:28,  2.03s/it]
                                                        

 17%|█▋        | 2096/12000 [1:15:43<5:35:28,  2.03s/it]
 17%|█▋        | 2097/12000 [1:15:45<5:04:39,  1.85s/it]
                                                        

 17%|█▋        | 2097/12000 [1:15:45<5:04:39,  1.85s/it]
 17%|█▋        | 2098/12000 [1:15:47<5:20:24,  1.94s/it]
                                                        

 17%|█▋        | 2098/12000 [1:15:47<5:20:24,  1.94s/it]
 17%|█▋        | 2099/12000 [1:15:49<4:57:51,  1.81s/it]
                                                        

 17%|█▋        | 2099/12000 [1:15:49<4:57:51,  1.81s/it]
 18%|█▊        | 2100/12000 [1:15:51<5:11:54,  1.89s/it]
                                                        

 18%|█▊        | 2100/12000 [1:15:51<5:11:54,  1.89s/it]
 18%|█▊        | 2101/12000 [1:15:55<6:53:41,  2.51s/it]
                                                        

 18%|█▊        | 2101/12000 [1:15:55<6:53:41,  2.51s/it]
 18%|█▊        | 2102/12000 [1:15:56<5:50:17,  2.12s/it]
                                                        

 18%|█▊        | 2102/12000 [1:15:56<5:50:17,  2.12s/it]
 18%|█▊        | 2103/12000 [1:16:00<7:15:11,  2.64s/it]
                                                        

 18%|█▊        | 2103/12000 [1:16:00<7:15:11,  2.64s/it]
 18%|█▊        | 2104/12000 [1:16:01<6:24:55,  2.33s/it]
                                                        

 18%|█▊        | 2104/12000 [1:16:01<6:24:55,  2.33s/it]
 18%|█▊        | 2105/12000 [1:16:05<7:34:08,  2.75s/it]
                                                        

 18%|█▊        | 2105/12000 [1:16:05<7:34:08,  2.75s/it]
 18%|█▊        | 2106/12000 [1:16:06<6:27:25,  2.35s/it]
                                                        

 18%|█▊        | 2106/12000 [1:16:06<6:27:25,  2.35s/it]
 18%|█▊        | 2107/12000 [1:16:08<6:01:21,  2.19s/it]
                                                        

 18%|█▊        | 2107/12000 [1:16:08<6:01:21,  2.19s/it]
 18%|█▊        | 2108/12000 [1:16:10<5:56:16,  2.16s/it]
                                                        

 18%|█▊        | 2108/12000 [1:16:10<5:56:16,  2.16s/it]
 18%|█▊        | 2109/12000 [1:16:15<7:48:57,  2.84s/it]
                                                        

 18%|█▊        | 2109/12000 [1:16:15<7:48:57,  2.84s/it]
 18%|█▊        | 2110/12000 [1:16:16<6:46:33,  2.47s/it]
                                                        

 18%|█▊        | 2110/12000 [1:16:16<6:46:33,  2.47s/it]
 18%|█▊        | 2111/12000 [1:16:18<5:49:11,  2.12s/it]
                                                        

 18%|█▊        | 2111/12000 [1:16:18<5:49:11,  2.12s/it]
 18%|█▊        | 2112/12000 [1:16:20<5:41:01,  2.07s/it]
                                                        

 18%|█▊        | 2112/12000 [1:16:20<5:41:01,  2.07s/it]
 18%|█▊        | 2113/12000 [1:16:21<5:19:28,  1.94s/it]
                                                        

 18%|█▊        | 2113/12000 [1:16:21<5:19:28,  1.94s/it]
 18%|█▊        | 2114/12000 [1:16:23<5:27:21,  1.99s/it]
                                                        

 18%|█▊        | 2114/12000 [1:16:23<5:27:21,  1.99s/it]
 18%|█▊        | 2115/12000 [1:16:25<5:03:57,  1.84s/it]
                                                        

 18%|█▊        | 2115/12000 [1:16:25<5:03:57,  1.84s/it]
 18%|█▊        | 2116/12000 [1:16:27<4:53:44,  1.78s/it]
                                                        

 18%|█▊        | 2116/12000 [1:16:27<4:53:44,  1.78s/it]
 18%|█▊        | 2117/12000 [1:16:29<5:04:53,  1.85s/it]
                                                        

 18%|█▊        | 2117/12000 [1:16:29<5:04:53,  1.85s/it]
 18%|█▊        | 2118/12000 [1:16:30<5:02:08,  1.83s/it]
                                                        

 18%|█▊        | 2118/12000 [1:16:30<5:02:08,  1.83s/it]
 18%|█▊        | 2119/12000 [1:16:32<4:48:22,  1.75s/it]
                                                        
{'loss': '0.6704', 'grad_norm': '0.3621', 'learning_rate': '0.0001919', 'entropy': '0.7542', 'num_tokens': '1.126e+07', 'mean_token_accuracy': '0.8165', 'epoch': '0.01834'}
{'loss': '1.328', 'grad_norm': '0.3945', 'learning_rate': '0.0001919', 'entropy': '1.257', 'num_tokens': '1.127e+07', 'mean_token_accuracy': '0.7078', 'epoch': '0.01835'}
{'loss': '1.082', 'grad_norm': '0.3412', 'learning_rate': '0.0001919', 'entropy': '1.051', 'num_tokens': '1.128e+07', 'mean_token_accuracy': '0.7387', 'epoch': '0.01835'}
{'loss': '0.8414', 'grad_norm': '0.3803', 'learning_rate': '0.0001919', 'entropy': '0.8389', 'num_tokens': '1.128e+07', 'mean_token_accuracy': '0.7944', 'epoch': '0.01836'}
{'loss': '1.304', 'grad_norm': '0.4346', 'learning_rate': '0.0001919', 'entropy': '1.221', 'num_tokens': '1.129e+07', 'mean_token_accuracy': '0.6999', 'epoch': '0.01837'}
{'loss': '1.344', 'grad_norm': '0.4291', 'learning_rate': '0.0001918', 'entropy': '1.243', 'num_tokens': '1.129e+07', 'mean_token_accuracy': '0.6906', 'epoch': '0.01838'}
{'loss': '1.436', 'grad_norm': '0.4626', 'learning_rate': '0.0001918', 'entropy': '1.107', 'num_tokens': '1.13e+07', 'mean_token_accuracy': '0.7304', 'epoch': '0.01839'}
{'loss': '1.123', 'grad_norm': '0.3982', 'learning_rate': '0.0001918', 'entropy': '1.072', 'num_tokens': '1.13e+07', 'mean_token_accuracy': '0.7461', 'epoch': '0.0184'}
{'loss': '0.8085', 'grad_norm': '0.3612', 'learning_rate': '0.0001918', 'entropy': '0.8316', 'num_tokens': '1.131e+07', 'mean_token_accuracy': '0.804', 'epoch': '0.01841'}
{'loss': '0.8517', 'grad_norm': '0.3916', 'learning_rate': '0.0001918', 'entropy': '0.87', 'num_tokens': '1.132e+07', 'mean_token_accuracy': '0.7776', 'epoch': '0.01842'}
{'loss': '0.7701', 'grad_norm': '0.3425', 'learning_rate': '0.0001918', 'entropy': '0.8632', 'num_tokens': '1.132e+07', 'mean_token_accuracy': '0.7886', 'epoch': '0.01843'}
{'loss': '0.8981', 'grad_norm': '0.339', 'learning_rate': '0.0001918', 'entropy': '0.923', 'num_tokens': '1.133e+07', 'mean_token_accuracy': '0.7729', 'epoch': '0.01843'}
{'loss': '0.7705', 'grad_norm': '0.2634', 'learning_rate': '0.0001918', 'entropy': '0.8314', 'num_tokens': '1.133e+07', 'mean_token_accuracy': '0.7962', 'epoch': '0.01844'}
{'loss': '0.6591', 'grad_norm': '0.3702', 'learning_rate': '0.0001918', 'entropy': '0.6777', 'num_tokens': '1.134e+07', 'mean_token_accuracy': '0.8203', 'epoch': '0.01845'}
{'loss': '0.9335', 'grad_norm': '0.3024', 'learning_rate': '0.0001917', 'entropy': '0.8683', 'num_tokens': '1.134e+07', 'mean_token_accuracy': '0.795', 'epoch': '0.01846'}
{'loss': '0.7097', 'grad_norm': '0.3239', 'learning_rate': '0.0001917', 'entropy': '0.7129', 'num_tokens': '1.135e+07', 'mean_token_accuracy': '0.816', 'epoch': '0.01847'}
{'loss': '0.7624', 'grad_norm': '0.3286', 'learning_rate': '0.0001917', 'entropy': '0.8111', 'num_tokens': '1.135e+07', 'mean_token_accuracy': '0.7962', 'epoch': '0.01848'}
{'loss': '0.8564', 'grad_norm': '0.4093', 'learning_rate': '0.0001917', 'entropy': '0.861', 'num_tokens': '1.136e+07', 'mean_token_accuracy': '0.778', 'epoch': '0.01849'}
{'loss': '1.065', 'grad_norm': '0.3538', 'learning_rate': '0.0001917', 'entropy': '0.8322', 'num_tokens': '1.136e+07', 'mean_token_accuracy': '0.7933', 'epoch': '0.0185'}
{'loss': '0.7604', 'grad_norm': '0.392', 'learning_rate': '0.0001917', 'entropy': '0.7671', 'num_tokens': '1.137e+07', 'mean_token_accuracy': '0.796', 'epoch': '0.0185'}
{'loss': '0.8829', 'grad_norm': '0.3619', 'learning_rate': '0.0001917', 'entropy': '0.8858', 'num_tokens': '1.137e+07', 'mean_token_accuracy': '0.7849', 'epoch': '0.01851'}
{'loss': '0.8576', 'grad_norm': '0.3989', 'learning_rate': '0.0001917', 'entropy': '0.8167', 'num_tokens': '1.138e+07', 'mean_token_accuracy': '0.7925', 'epoch': '0.01852'}
{'loss': '0.7051', 'grad_norm': '0.3397', 'learning_rate': '0.0001917', 'entropy': '0.6962', 'num_tokens': '1.138e+07', 'mean_token_accuracy': '0.8192', 'epoch': '0.01853'}
{'loss': '0.8027', 'grad_norm': '0.3513', 'learning_rate': '0.0001916', 'entropy': '0.7934', 'num_tokens': '1.139e+07', 'mean_token_accuracy': '0.8002', 'epoch': '0.01854'}
{'loss': '0.9949', 'grad_norm': '0.4809', 'learning_rate': '0.0001916', 'entropy': '0.9959', 'num_tokens': '1.14e+07', 'mean_token_accuracy': '0.7627', 'epoch': '0.01855'}
{'loss': '0.8686', 'grad_norm': '0.3235', 'learning_rate': '0.0001916', 'entropy': '0.8855', 'num_tokens': '1.14e+07', 'mean_token_accuracy': '0.787', 'epoch': '0.01856'}
{'loss': '0.8516', 'grad_norm': '0.3697', 'learning_rate': '0.0001916', 'entropy': '0.807', 'num_tokens': '1.141e+07', 'mean_token_accuracy': '0.7947', 'epoch': '0.01857'}
{'loss': '0.8215', 'grad_norm': '0.3449', 'learning_rate': '0.0001916', 'entropy': '0.8539', 'num_tokens': '1.141e+07', 'mean_token_accuracy': '0.7814', 'epoch': '0.01858'}
{'loss': '0.8782', 'grad_norm': '0.3361', 'learning_rate': '0.0001916', 'entropy': '0.7717', 'num_tokens': '1.142e+07', 'mean_token_accuracy': '0.8045', 'epoch': '0.01858'}
{'loss': '0.8172', 'grad_norm': '0.3529', 'learning_rate': '0.0001916', 'entropy': '0.9053', 'num_tokens': '1.142e+07', 'mean_token_accuracy': '0.7795', 'epoch': '0.01859'}
{'loss': '0.9978', 'grad_norm': '0.4591', 'learning_rate': '0.0001916', 'entropy': '1.034', 'num_tokens': '1.143e+07', 'mean_token_accuracy': '0.7389', 'epoch': '0.0186'}
{'loss': '1.352', 'grad_norm': '0.3726', 'learning_rate': '0.0001916', 'entropy': '1.328', 'num_tokens': '1.144e+07', 'mean_token_accuracy': '0.6759', 'epoch': '0.01861'}
{'loss': '0.9186', 'grad_norm': '0.383', 'learning_rate': '0.0001915', 'entropy': '0.9825', 'num_tokens': '1.144e+07', 'mean_token_accuracy': '0.755', 'epoch': '0.01862'}
{'loss': '1.095', 'grad_norm': '0.4712', 'learning_rate': '0.0001915', 'entropy': '1.007', 'num_tokens': '1.145e+07', 'mean_token_accuracy': '0.7677', 'epoch': '0.01863'}
{'loss': '0.852', 'grad_norm': '0.4668', 'learning_rate': '0.0001915', 'entropy': '0.873', 'num_tokens': '1.145e+07', 'mean_token_accuracy': '0.7854', 'epoch': '0.01864'}
{'loss': '0.7472', 'grad_norm': '0.3797', 'learning_rate': '0.0001915', 'entropy': '0.7908', 'num_tokens': '1.146e+07', 'mean_token_accuracy': '0.7956', 'epoch': '0.01865'}
{'loss': '0.8753', 'grad_norm': '0.3807', 'learning_rate': '0.0001915', 'entropy': '0.8993', 'num_tokens': '1.146e+07', 'mean_token_accuracy': '0.7768', 'epoch': '0.01866'}
{'loss': '1.055', 'grad_norm': '0.4203', 'learning_rate': '0.0001915', 'entropy': '1.043', 'num_tokens': '1.147e+07', 'mean_token_accuracy': '0.761', 'epoch': '0.01866'}
{'loss': '0.6749', 'grad_norm': '0.2967', 'learning_rate': '0.0001915', 'entropy': '0.6985', 'num_tokens': '1.147e+07', 'mean_token_accuracy': '0.8243', 'epoch': '0.01867'}
{'loss': '0.7125', 'grad_norm': '0.2856', 'learning_rate': '0.0001915', 'entropy': '0.7579', 'num_tokens': '1.148e+07', 'mean_token_accuracy': '0.7954', 'epoch': '0.01868'}
{'loss': '1', 'grad_norm': '0.5064', 'learning_rate': '0.0001915', 'entropy': '0.8851', 'num_tokens': '1.148e+07', 'mean_token_accuracy': '0.7872', 'epoch': '0.01869'}
{'loss': '0.8433', 'grad_norm': '0.3906', 'learning_rate': '0.0001914', 'entropy': '0.8727', 'num_tokens': '1.149e+07', 'mean_token_accuracy': '0.7973', 'epoch': '0.0187'}
{'loss': '0.9801', 'grad_norm': '0.3396', 'learning_rate': '0.0001914', 'entropy': '1.079', 'num_tokens': '1.149e+07', 'mean_token_accuracy': '0.7422', 'epoch': '0.01871'}
{'loss': '1.148', 'grad_norm': '0.4189', 'learning_rate': '0.0001914', 'entropy': '1.033', 'num_tokens': '1.15e+07', 'mean_token_accuracy': '0.7464', 'epoch': '0.01872'}
{'loss': '0.7775', 'grad_norm': '0.4255', 'learning_rate': '0.0001914', 'entropy': '0.8456', 'num_tokens': '1.15e+07', 'mean_token_accuracy': '0.7954', 'epoch': '0.01873'}
{'loss': '0.7942', 'grad_norm': '0.3712', 'learning_rate': '0.0001914', 'entropy': '0.7214', 'num_tokens': '1.151e+07', 'mean_token_accuracy': '0.8164', 'epoch': '0.01874'}
{'loss': '0.6772', 'grad_norm': '0.314', 'learning_rate': '0.0001914', 'entropy': '0.6415', 'num_tokens': '1.152e+07', 'mean_token_accuracy': '0.829', 'epoch': '0.01874'}

 18%|█▊        | 2119/12000 [1:16:32<4:48:22,  1.75s/it]
 18%|█▊        | 2120/12000 [1:16:34<5:24:47,  1.97s/it]
                                                        

 18%|█▊        | 2120/12000 [1:16:34<5:24:47,  1.97s/it]
 18%|█▊        | 2121/12000 [1:16:36<5:14:57,  1.91s/it]
                                                        

 18%|█▊        | 2121/12000 [1:16:36<5:14:57,  1.91s/it]
 18%|█▊        | 2122/12000 [1:16:38<5:04:48,  1.85s/it]
                                                        

 18%|█▊        | 2122/12000 [1:16:38<5:04:48,  1.85s/it]
 18%|█▊        | 2123/12000 [1:16:40<5:33:36,  2.03s/it]
                                                        

 18%|█▊        | 2123/12000 [1:16:40<5:33:36,  2.03s/it]
 18%|█▊        | 2124/12000 [1:16:42<5:01:37,  1.83s/it]
                                                        

 18%|█▊        | 2124/12000 [1:16:42<5:01:37,  1.83s/it]
 18%|█▊        | 2125/12000 [1:16:43<4:40:26,  1.70s/it]
                                                        

 18%|█▊        | 2125/12000 [1:16:43<4:40:26,  1.70s/it]
 18%|█▊        | 2126/12000 [1:16:45<5:05:48,  1.86s/it]
                                                        

 18%|█▊        | 2126/12000 [1:16:45<5:05:48,  1.86s/it]
 18%|█▊        | 2127/12000 [1:16:47<4:41:42,  1.71s/it]
                                                        

 18%|█▊        | 2127/12000 [1:16:47<4:41:42,  1.71s/it]
 18%|█▊        | 2128/12000 [1:16:48<4:35:50,  1.68s/it]
                                                        

 18%|█▊        | 2128/12000 [1:16:48<4:35:50,  1.68s/it]
 18%|█▊        | 2129/12000 [1:16:50<4:33:08,  1.66s/it]
                                                        

 18%|█▊        | 2129/12000 [1:16:50<4:33:08,  1.66s/it]
 18%|█▊        | 2130/12000 [1:16:54<6:14:15,  2.28s/it]
                                                        

 18%|█▊        | 2130/12000 [1:16:54<6:14:15,  2.28s/it]
 18%|█▊        | 2131/12000 [1:16:56<6:16:33,  2.29s/it]
                                                        

 18%|█▊        | 2131/12000 [1:16:56<6:16:33,  2.29s/it]
 18%|█▊        | 2132/12000 [1:16:59<6:44:35,  2.46s/it]
                                                        

 18%|█▊        | 2132/12000 [1:16:59<6:44:35,  2.46s/it]
 18%|█▊        | 2133/12000 [1:17:00<5:59:24,  2.19s/it]
                                                        

 18%|█▊        | 2133/12000 [1:17:00<5:59:24,  2.19s/it]
 18%|█▊        | 2134/12000 [1:17:02<5:33:04,  2.03s/it]
                                                        

 18%|█▊        | 2134/12000 [1:17:02<5:33:04,  2.03s/it]
 18%|█▊        | 2135/12000 [1:17:04<5:45:48,  2.10s/it]
                                                        

 18%|█▊        | 2135/12000 [1:17:04<5:45:48,  2.10s/it]
 18%|█▊        | 2136/12000 [1:17:07<6:06:10,  2.23s/it]
                                                        

 18%|█▊        | 2136/12000 [1:17:07<6:06:10,  2.23s/it]
 18%|█▊        | 2137/12000 [1:17:08<5:37:39,  2.05s/it]
                                                        

 18%|█▊        | 2137/12000 [1:17:08<5:37:39,  2.05s/it]
 18%|█▊        | 2138/12000 [1:17:11<5:48:01,  2.12s/it]
                                                        

 18%|█▊        | 2138/12000 [1:17:11<5:48:01,  2.12s/it]
 18%|█▊        | 2139/12000 [1:17:13<5:34:32,  2.04s/it]
                                                        

 18%|█▊        | 2139/12000 [1:17:13<5:34:32,  2.04s/it]
 18%|█▊        | 2140/12000 [1:17:16<6:31:33,  2.38s/it]
                                                        

 18%|█▊        | 2140/12000 [1:17:16<6:31:33,  2.38s/it]
 18%|█▊        | 2141/12000 [1:17:18<6:10:03,  2.25s/it]
                                                        

 18%|█▊        | 2141/12000 [1:17:18<6:10:03,  2.25s/it]
 18%|█▊        | 2142/12000 [1:17:19<5:39:25,  2.07s/it]
                                                        

 18%|█▊        | 2142/12000 [1:17:19<5:39:25,  2.07s/it]
 18%|█▊        | 2143/12000 [1:17:23<6:59:06,  2.55s/it]
                                                        

 18%|█▊        | 2143/12000 [1:17:23<6:59:06,  2.55s/it]
 18%|█▊        | 2144/12000 [1:17:25<6:19:41,  2.31s/it]
                                                        

 18%|█▊        | 2144/12000 [1:17:25<6:19:41,  2.31s/it]
 18%|█▊        | 2145/12000 [1:17:27<6:15:06,  2.28s/it]
                                                        

 18%|█▊        | 2145/12000 [1:17:27<6:15:06,  2.28s/it]
 18%|█▊        | 2146/12000 [1:17:29<5:58:13,  2.18s/it]
                                                        

 18%|█▊        | 2146/12000 [1:17:29<5:58:13,  2.18s/it]
 18%|█▊        | 2147/12000 [1:17:31<5:53:32,  2.15s/it]
                                                        

 18%|█▊        | 2147/12000 [1:17:31<5:53:32,  2.15s/it]
 18%|█▊        | 2148/12000 [1:17:33<5:46:56,  2.11s/it]
                                                        

 18%|█▊        | 2148/12000 [1:17:33<5:46:56,  2.11s/it]
 18%|█▊        | 2149/12000 [1:17:35<5:44:01,  2.10s/it]
                                                        

 18%|█▊        | 2149/12000 [1:17:35<5:44:01,  2.10s/it]
 18%|█▊        | 2150/12000 [1:17:37<5:50:58,  2.14s/it]
                                                        

 18%|█▊        | 2150/12000 [1:17:37<5:50:58,  2.14s/it]
 18%|█▊        | 2151/12000 [1:17:40<6:24:35,  2.34s/it]
                                                        

 18%|█▊        | 2151/12000 [1:17:40<6:24:35,  2.34s/it]
 18%|█▊        | 2152/12000 [1:17:43<7:15:15,  2.65s/it]
                                                        

 18%|█▊        | 2152/12000 [1:17:43<7:15:15,  2.65s/it]
 18%|█▊        | 2153/12000 [1:17:45<6:40:29,  2.44s/it]
                                                        

 18%|█▊        | 2153/12000 [1:17:45<6:40:29,  2.44s/it]
 18%|█▊        | 2154/12000 [1:17:49<7:13:46,  2.64s/it]
                                                        

 18%|█▊        | 2154/12000 [1:17:49<7:13:46,  2.64s/it]
 18%|█▊        | 2155/12000 [1:17:51<6:53:42,  2.52s/it]
                                                        

 18%|█▊        | 2155/12000 [1:17:51<6:53:42,  2.52s/it]
 18%|█▊        | 2156/12000 [1:17:53<7:02:29,  2.58s/it]
                                                        

 18%|█▊        | 2156/12000 [1:17:53<7:02:29,  2.58s/it]
 18%|█▊        | 2157/12000 [1:17:56<6:40:59,  2.44s/it]
                                                        

 18%|█▊        | 2157/12000 [1:17:56<6:40:59,  2.44s/it]
 18%|█▊        | 2158/12000 [1:17:57<6:02:36,  2.21s/it]
                                                        

 18%|█▊        | 2158/12000 [1:17:57<6:02:36,  2.21s/it]
 18%|█▊        | 2159/12000 [1:17:59<5:55:45,  2.17s/it]
                                                        

 18%|█▊        | 2159/12000 [1:17:59<5:55:45,  2.17s/it]
 18%|█▊        | 2160/12000 [1:18:01<5:23:00,  1.97s/it]
                                                        

 18%|█▊        | 2160/12000 [1:18:01<5:23:00,  1.97s/it]
 18%|█▊        | 2161/12000 [1:18:03<5:13:52,  1.91s/it]
                                                        

 18%|█▊        | 2161/12000 [1:18:03<5:13:52,  1.91s/it]
 18%|█▊        | 2162/12000 [1:18:04<4:52:40,  1.78s/it]
                                                        

 18%|█▊        | 2162/12000 [1:18:04<4:52:40,  1.78s/it]
 18%|█▊        | 2163/12000 [1:18:05<4:27:23,  1.63s/it]
                                                        

 18%|█▊        | 2163/12000 [1:18:05<4:27:23,  1.63s/it]
 18%|█▊        | 2164/12000 [1:18:07<4:23:40,  1.61s/it]
                                                        

 18%|█▊        | 2164/12000 [1:18:07<4:23:40,  1.61s/it]
 18%|█▊        | 2165/12000 [1:18:09<4:43:26,  1.73s/it]
                                                        

 18%|█▊        | 2165/12000 [1:18:09<4:43:26,  1.73s/it]
 18%|█▊        | 2166/12000 [1:18:11<4:40:11,  1.71s/it]
                                                        
{'loss': '0.6995', 'grad_norm': '0.3459', 'learning_rate': '0.0001914', 'entropy': '0.6796', 'num_tokens': '1.152e+07', 'mean_token_accuracy': '0.8265', 'epoch': '0.01875'}
{'loss': '0.9508', 'grad_norm': '0.3743', 'learning_rate': '0.0001914', 'entropy': '0.9896', 'num_tokens': '1.153e+07', 'mean_token_accuracy': '0.7512', 'epoch': '0.01876'}
{'loss': '0.8967', 'grad_norm': '0.3972', 'learning_rate': '0.0001914', 'entropy': '0.8902', 'num_tokens': '1.153e+07', 'mean_token_accuracy': '0.7896', 'epoch': '0.01877'}
{'loss': '0.8479', 'grad_norm': '0.4538', 'learning_rate': '0.0001913', 'entropy': '0.7948', 'num_tokens': '1.154e+07', 'mean_token_accuracy': '0.7874', 'epoch': '0.01878'}
{'loss': '0.9967', 'grad_norm': '0.4684', 'learning_rate': '0.0001913', 'entropy': '0.9239', 'num_tokens': '1.154e+07', 'mean_token_accuracy': '0.7682', 'epoch': '0.01879'}
{'loss': '0.8767', 'grad_norm': '0.4347', 'learning_rate': '0.0001913', 'entropy': '0.917', 'num_tokens': '1.155e+07', 'mean_token_accuracy': '0.7766', 'epoch': '0.0188'}
{'loss': '0.7552', 'grad_norm': '0.3538', 'learning_rate': '0.0001913', 'entropy': '0.7543', 'num_tokens': '1.155e+07', 'mean_token_accuracy': '0.8081', 'epoch': '0.01881'}
{'loss': '0.8127', 'grad_norm': '0.4497', 'learning_rate': '0.0001913', 'entropy': '0.7702', 'num_tokens': '1.156e+07', 'mean_token_accuracy': '0.7912', 'epoch': '0.01881'}
{'loss': '0.9417', 'grad_norm': '0.4523', 'learning_rate': '0.0001913', 'entropy': '1.04', 'num_tokens': '1.156e+07', 'mean_token_accuracy': '0.7488', 'epoch': '0.01882'}
{'loss': '0.8737', 'grad_norm': '0.4163', 'learning_rate': '0.0001913', 'entropy': '0.9364', 'num_tokens': '1.156e+07', 'mean_token_accuracy': '0.7719', 'epoch': '0.01883'}
{'loss': '0.7365', 'grad_norm': '0.3195', 'learning_rate': '0.0001913', 'entropy': '0.7527', 'num_tokens': '1.157e+07', 'mean_token_accuracy': '0.8156', 'epoch': '0.01884'}
{'loss': '0.8243', 'grad_norm': '0.3479', 'learning_rate': '0.0001913', 'entropy': '0.9228', 'num_tokens': '1.157e+07', 'mean_token_accuracy': '0.7785', 'epoch': '0.01885'}
{'loss': '0.9263', 'grad_norm': '0.3754', 'learning_rate': '0.0001912', 'entropy': '0.8953', 'num_tokens': '1.158e+07', 'mean_token_accuracy': '0.785', 'epoch': '0.01886'}
{'loss': '0.9172', 'grad_norm': '0.4737', 'learning_rate': '0.0001912', 'entropy': '0.9765', 'num_tokens': '1.159e+07', 'mean_token_accuracy': '0.7572', 'epoch': '0.01887'}
{'loss': '0.9101', 'grad_norm': '0.399', 'learning_rate': '0.0001912', 'entropy': '1.011', 'num_tokens': '1.159e+07', 'mean_token_accuracy': '0.7702', 'epoch': '0.01888'}
{'loss': '0.8669', 'grad_norm': '0.3673', 'learning_rate': '0.0001912', 'entropy': '0.9123', 'num_tokens': '1.16e+07', 'mean_token_accuracy': '0.7931', 'epoch': '0.01889'}
{'loss': '0.6207', 'grad_norm': '0.318', 'learning_rate': '0.0001912', 'entropy': '0.7303', 'num_tokens': '1.16e+07', 'mean_token_accuracy': '0.8283', 'epoch': '0.01889'}
{'loss': '0.8737', 'grad_norm': '0.3184', 'learning_rate': '0.0001912', 'entropy': '0.7925', 'num_tokens': '1.161e+07', 'mean_token_accuracy': '0.8045', 'epoch': '0.0189'}
{'loss': '1.036', 'grad_norm': '0.4115', 'learning_rate': '0.0001912', 'entropy': '1.049', 'num_tokens': '1.162e+07', 'mean_token_accuracy': '0.7586', 'epoch': '0.01891'}
{'loss': '1.379', 'grad_norm': '0.3874', 'learning_rate': '0.0001912', 'entropy': '1.15', 'num_tokens': '1.162e+07', 'mean_token_accuracy': '0.7215', 'epoch': '0.01892'}
{'loss': '0.7517', 'grad_norm': '0.3445', 'learning_rate': '0.0001912', 'entropy': '0.7541', 'num_tokens': '1.163e+07', 'mean_token_accuracy': '0.8088', 'epoch': '0.01893'}
{'loss': '1.048', 'grad_norm': '0.3422', 'learning_rate': '0.0001911', 'entropy': '1.047', 'num_tokens': '1.164e+07', 'mean_token_accuracy': '0.7293', 'epoch': '0.01894'}
{'loss': '0.855', 'grad_norm': '0.3705', 'learning_rate': '0.0001911', 'entropy': '0.8642', 'num_tokens': '1.164e+07', 'mean_token_accuracy': '0.7911', 'epoch': '0.01895'}
{'loss': '0.9307', 'grad_norm': '0.2888', 'learning_rate': '0.0001911', 'entropy': '0.798', 'num_tokens': '1.165e+07', 'mean_token_accuracy': '0.7979', 'epoch': '0.01896'}
{'loss': '0.7038', 'grad_norm': '0.3088', 'learning_rate': '0.0001911', 'entropy': '0.6917', 'num_tokens': '1.166e+07', 'mean_token_accuracy': '0.8103', 'epoch': '0.01897'}
{'loss': '1.215', 'grad_norm': '0.3825', 'learning_rate': '0.0001911', 'entropy': '1.074', 'num_tokens': '1.166e+07', 'mean_token_accuracy': '0.7517', 'epoch': '0.01897'}
{'loss': '0.9724', 'grad_norm': '0.3392', 'learning_rate': '0.0001911', 'entropy': '0.9522', 'num_tokens': '1.167e+07', 'mean_token_accuracy': '0.7748', 'epoch': '0.01898'}
{'loss': '0.9069', 'grad_norm': '0.4415', 'learning_rate': '0.0001911', 'entropy': '0.8819', 'num_tokens': '1.167e+07', 'mean_token_accuracy': '0.7851', 'epoch': '0.01899'}
{'loss': '0.9133', 'grad_norm': '0.3074', 'learning_rate': '0.0001911', 'entropy': '0.8902', 'num_tokens': '1.168e+07', 'mean_token_accuracy': '0.7876', 'epoch': '0.019'}
{'loss': '0.9285', 'grad_norm': '0.402', 'learning_rate': '0.000191', 'entropy': '0.9269', 'num_tokens': '1.168e+07', 'mean_token_accuracy': '0.7723', 'epoch': '0.01901'}
{'loss': '0.8265', 'grad_norm': '0.4116', 'learning_rate': '0.000191', 'entropy': '0.7734', 'num_tokens': '1.169e+07', 'mean_token_accuracy': '0.8155', 'epoch': '0.01902'}
{'loss': '1.262', 'grad_norm': '0.4024', 'learning_rate': '0.000191', 'entropy': '1.232', 'num_tokens': '1.17e+07', 'mean_token_accuracy': '0.7176', 'epoch': '0.01903'}
{'loss': '0.7207', 'grad_norm': '0.3858', 'learning_rate': '0.000191', 'entropy': '0.7932', 'num_tokens': '1.17e+07', 'mean_token_accuracy': '0.801', 'epoch': '0.01904'}
{'loss': '0.7383', 'grad_norm': '0.3633', 'learning_rate': '0.000191', 'entropy': '0.743', 'num_tokens': '1.17e+07', 'mean_token_accuracy': '0.8124', 'epoch': '0.01904'}
{'loss': '0.8778', 'grad_norm': '0.395', 'learning_rate': '0.000191', 'entropy': '0.9469', 'num_tokens': '1.171e+07', 'mean_token_accuracy': '0.7651', 'epoch': '0.01905'}
{'loss': '1.22', 'grad_norm': '0.4371', 'learning_rate': '0.000191', 'entropy': '1.124', 'num_tokens': '1.172e+07', 'mean_token_accuracy': '0.7323', 'epoch': '0.01906'}
{'loss': '1.118', 'grad_norm': '0.3207', 'learning_rate': '0.000191', 'entropy': '1.044', 'num_tokens': '1.172e+07', 'mean_token_accuracy': '0.7553', 'epoch': '0.01907'}
{'loss': '1.099', 'grad_norm': '0.463', 'learning_rate': '0.000191', 'entropy': '0.9845', 'num_tokens': '1.173e+07', 'mean_token_accuracy': '0.7659', 'epoch': '0.01908'}
{'loss': '0.9915', 'grad_norm': '0.3936', 'learning_rate': '0.0001909', 'entropy': '1.028', 'num_tokens': '1.173e+07', 'mean_token_accuracy': '0.7506', 'epoch': '0.01909'}
{'loss': '0.9213', 'grad_norm': '0.364', 'learning_rate': '0.0001909', 'entropy': '0.9459', 'num_tokens': '1.174e+07', 'mean_token_accuracy': '0.766', 'epoch': '0.0191'}
{'loss': '0.7094', 'grad_norm': '0.3168', 'learning_rate': '0.0001909', 'entropy': '0.7199', 'num_tokens': '1.175e+07', 'mean_token_accuracy': '0.8351', 'epoch': '0.01911'}
{'loss': '0.8939', 'grad_norm': '0.3356', 'learning_rate': '0.0001909', 'entropy': '0.9398', 'num_tokens': '1.175e+07', 'mean_token_accuracy': '0.77', 'epoch': '0.01912'}
{'loss': '1.129', 'grad_norm': '0.4111', 'learning_rate': '0.0001909', 'entropy': '1.095', 'num_tokens': '1.176e+07', 'mean_token_accuracy': '0.7353', 'epoch': '0.01912'}
{'loss': '0.8625', 'grad_norm': '0.3193', 'learning_rate': '0.0001909', 'entropy': '0.8725', 'num_tokens': '1.176e+07', 'mean_token_accuracy': '0.7869', 'epoch': '0.01913'}
{'loss': '0.779', 'grad_norm': '0.3848', 'learning_rate': '0.0001909', 'entropy': '0.8818', 'num_tokens': '1.176e+07', 'mean_token_accuracy': '0.7826', 'epoch': '0.01914'}
{'loss': '0.7114', 'grad_norm': '0.3796', 'learning_rate': '0.0001909', 'entropy': '0.7365', 'num_tokens': '1.177e+07', 'mean_token_accuracy': '0.8098', 'epoch': '0.01915'}
{'loss': '1.161', 'grad_norm': '0.4204', 'learning_rate': '0.0001909', 'entropy': '1.199', 'num_tokens': '1.177e+07', 'mean_token_accuracy': '0.7209', 'epoch': '0.01916'}

 18%|█▊        | 2166/12000 [1:18:11<4:40:11,  1.71s/it]
 18%|█▊        | 2167/12000 [1:18:12<4:29:03,  1.64s/it]
                                                        

 18%|█▊        | 2167/12000 [1:18:12<4:29:03,  1.64s/it]
 18%|█▊        | 2168/12000 [1:18:14<4:30:00,  1.65s/it]
                                                        

 18%|█▊        | 2168/12000 [1:18:14<4:30:00,  1.65s/it]
 18%|█▊        | 2169/12000 [1:18:16<4:53:53,  1.79s/it]
                                                        

 18%|█▊        | 2169/12000 [1:18:16<4:53:53,  1.79s/it]
 18%|█▊        | 2170/12000 [1:18:18<4:51:01,  1.78s/it]
                                                        

 18%|█▊        | 2170/12000 [1:18:18<4:51:01,  1.78s/it]
 18%|█▊        | 2171/12000 [1:18:19<4:32:47,  1.67s/it]
                                                        

 18%|█▊        | 2171/12000 [1:18:19<4:32:47,  1.67s/it]
 18%|█▊        | 2172/12000 [1:18:22<5:59:51,  2.20s/it]
                                                        

 18%|█▊        | 2172/12000 [1:18:22<5:59:51,  2.20s/it]
 18%|█▊        | 2173/12000 [1:18:25<5:55:50,  2.17s/it]
                                                        

 18%|█▊        | 2173/12000 [1:18:25<5:55:50,  2.17s/it]
 18%|█▊        | 2174/12000 [1:18:28<7:12:27,  2.64s/it]
                                                        

 18%|█▊        | 2174/12000 [1:18:28<7:12:27,  2.64s/it]
 18%|█▊        | 2175/12000 [1:18:30<6:25:35,  2.35s/it]
                                                        

 18%|█▊        | 2175/12000 [1:18:30<6:25:35,  2.35s/it]
 18%|█▊        | 2176/12000 [1:18:31<5:40:52,  2.08s/it]
                                                        

 18%|█▊        | 2176/12000 [1:18:31<5:40:52,  2.08s/it]
 18%|█▊        | 2177/12000 [1:18:34<5:49:31,  2.13s/it]
                                                        

 18%|█▊        | 2177/12000 [1:18:34<5:49:31,  2.13s/it]
 18%|█▊        | 2178/12000 [1:18:36<5:36:44,  2.06s/it]
                                                        

 18%|█▊        | 2178/12000 [1:18:36<5:36:44,  2.06s/it]
 18%|█▊        | 2179/12000 [1:18:37<5:06:18,  1.87s/it]
                                                        

 18%|█▊        | 2179/12000 [1:18:37<5:06:18,  1.87s/it]
 18%|█▊        | 2180/12000 [1:18:38<4:45:32,  1.74s/it]
                                                        

 18%|█▊        | 2180/12000 [1:18:38<4:45:32,  1.74s/it]
 18%|█▊        | 2181/12000 [1:18:40<4:44:04,  1.74s/it]
                                                        

 18%|█▊        | 2181/12000 [1:18:40<4:44:04,  1.74s/it]
 18%|█▊        | 2182/12000 [1:18:42<4:52:00,  1.78s/it]
                                                        

 18%|█▊        | 2182/12000 [1:18:42<4:52:00,  1.78s/it]
 18%|█▊        | 2183/12000 [1:18:44<4:47:52,  1.76s/it]
                                                        

 18%|█▊        | 2183/12000 [1:18:44<4:47:52,  1.76s/it]
 18%|█▊        | 2184/12000 [1:18:45<4:32:05,  1.66s/it]
                                                        

 18%|█▊        | 2184/12000 [1:18:45<4:32:05,  1.66s/it]
 18%|█▊        | 2185/12000 [1:18:47<4:37:24,  1.70s/it]
                                                        

 18%|█▊        | 2185/12000 [1:18:47<4:37:24,  1.70s/it]
 18%|█▊        | 2186/12000 [1:18:49<5:15:48,  1.93s/it]
                                                        

 18%|█▊        | 2186/12000 [1:18:49<5:15:48,  1.93s/it]
 18%|█▊        | 2187/12000 [1:18:52<5:35:06,  2.05s/it]
                                                        

 18%|█▊        | 2187/12000 [1:18:52<5:35:06,  2.05s/it]
 18%|█▊        | 2188/12000 [1:18:54<5:20:12,  1.96s/it]
                                                        

 18%|█▊        | 2188/12000 [1:18:54<5:20:12,  1.96s/it]
 18%|█▊        | 2189/12000 [1:18:55<5:03:26,  1.86s/it]
                                                        

 18%|█▊        | 2189/12000 [1:18:55<5:03:26,  1.86s/it]
 18%|█▊        | 2190/12000 [1:18:58<5:59:27,  2.20s/it]
                                                        

 18%|█▊        | 2190/12000 [1:18:58<5:59:27,  2.20s/it]
 18%|█▊        | 2191/12000 [1:19:00<5:22:48,  1.97s/it]
                                                        

 18%|█▊        | 2191/12000 [1:19:00<5:22:48,  1.97s/it]
 18%|█▊        | 2192/12000 [1:19:02<6:05:23,  2.24s/it]
                                                        

 18%|█▊        | 2192/12000 [1:19:02<6:05:23,  2.24s/it]
 18%|█▊        | 2193/12000 [1:19:04<5:30:51,  2.02s/it]
                                                        

 18%|█▊        | 2193/12000 [1:19:04<5:30:51,  2.02s/it]
 18%|█▊        | 2194/12000 [1:19:06<5:11:50,  1.91s/it]
                                                        

 18%|█▊        | 2194/12000 [1:19:06<5:11:50,  1.91s/it]
 18%|█▊        | 2195/12000 [1:19:07<4:52:30,  1.79s/it]
                                                        

 18%|█▊        | 2195/12000 [1:19:07<4:52:30,  1.79s/it]
 18%|█▊        | 2196/12000 [1:19:10<6:06:32,  2.24s/it]
                                                        

 18%|█▊        | 2196/12000 [1:19:10<6:06:32,  2.24s/it]
 18%|█▊        | 2197/12000 [1:19:12<5:44:17,  2.11s/it]
                                                        

 18%|█▊        | 2197/12000 [1:19:12<5:44:17,  2.11s/it]
 18%|█▊        | 2198/12000 [1:19:15<6:09:44,  2.26s/it]
                                                        

 18%|█▊        | 2198/12000 [1:19:15<6:09:44,  2.26s/it]
 18%|█▊        | 2199/12000 [1:19:16<5:36:18,  2.06s/it]
                                                        

 18%|█▊        | 2199/12000 [1:19:16<5:36:18,  2.06s/it]
 18%|█▊        | 2200/12000 [1:19:18<5:10:16,  1.90s/it]
                                                        

 18%|█▊        | 2200/12000 [1:19:18<5:10:16,  1.90s/it]
 18%|█▊        | 2201/12000 [1:19:21<5:56:37,  2.18s/it]
                                                        

 18%|█▊        | 2201/12000 [1:19:21<5:56:37,  2.18s/it]
 18%|█▊        | 2202/12000 [1:19:22<5:20:24,  1.96s/it]
                                                        

 18%|█▊        | 2202/12000 [1:19:22<5:20:24,  1.96s/it]
 18%|█▊        | 2203/12000 [1:19:24<4:58:39,  1.83s/it]
                                                        

 18%|█▊        | 2203/12000 [1:19:24<4:58:39,  1.83s/it]
 18%|█▊        | 2204/12000 [1:19:27<5:52:23,  2.16s/it]
                                                        

 18%|█▊        | 2204/12000 [1:19:27<5:52:23,  2.16s/it]
 18%|█▊        | 2205/12000 [1:19:32<8:12:55,  3.02s/it]
                                                        

 18%|█▊        | 2205/12000 [1:19:32<8:12:55,  3.02s/it]
 18%|█▊        | 2206/12000 [1:19:33<6:58:13,  2.56s/it]
                                                        

 18%|█▊        | 2206/12000 [1:19:33<6:58:13,  2.56s/it]
 18%|█▊        | 2207/12000 [1:19:35<6:02:00,  2.22s/it]
                                                        

 18%|█▊        | 2207/12000 [1:19:35<6:02:00,  2.22s/it]
 18%|█▊        | 2208/12000 [1:19:36<5:26:35,  2.00s/it]
                                                        

 18%|█▊        | 2208/12000 [1:19:36<5:26:35,  2.00s/it]
 18%|█▊        | 2209/12000 [1:19:38<5:01:36,  1.85s/it]
                                                        

 18%|█▊        | 2209/12000 [1:19:38<5:01:36,  1.85s/it]
 18%|█▊        | 2210/12000 [1:19:39<4:57:19,  1.82s/it]
                                                        

 18%|█▊        | 2210/12000 [1:19:39<4:57:19,  1.82s/it]
 18%|█▊        | 2211/12000 [1:19:41<4:43:18,  1.74s/it]
                                                        

 18%|█▊        | 2211/12000 [1:19:41<4:43:18,  1.74s/it]
 18%|█▊        | 2212/12000 [1:19:42<4:34:20,  1.68s/it]
                                                        

 18%|█▊        | 2212/12000 [1:19:42<4:34:20,  1.68s/it]
 18%|█▊        | 2213/12000 [1:19:45<4:54:53,  1.81s/it]
                                                        
{'loss': '0.7329', 'grad_norm': '0.3486', 'learning_rate': '0.0001908', 'entropy': '0.736', 'num_tokens': '1.178e+07', 'mean_token_accuracy': '0.8076', 'epoch': '0.01917'}
{'loss': '0.8296', 'grad_norm': '0.3375', 'learning_rate': '0.0001908', 'entropy': '0.9425', 'num_tokens': '1.179e+07', 'mean_token_accuracy': '0.7751', 'epoch': '0.01918'}
{'loss': '0.7697', 'grad_norm': '0.3884', 'learning_rate': '0.0001908', 'entropy': '0.7713', 'num_tokens': '1.179e+07', 'mean_token_accuracy': '0.8019', 'epoch': '0.01919'}
{'loss': '0.9465', 'grad_norm': '0.3164', 'learning_rate': '0.0001908', 'entropy': '1.074', 'num_tokens': '1.18e+07', 'mean_token_accuracy': '0.7407', 'epoch': '0.0192'}
{'loss': '0.9028', 'grad_norm': '0.4414', 'learning_rate': '0.0001908', 'entropy': '0.883', 'num_tokens': '1.18e+07', 'mean_token_accuracy': '0.7734', 'epoch': '0.0192'}
{'loss': '0.6992', 'grad_norm': '0.3185', 'learning_rate': '0.0001908', 'entropy': '0.7377', 'num_tokens': '1.18e+07', 'mean_token_accuracy': '0.8069', 'epoch': '0.01921'}
{'loss': '1.196', 'grad_norm': '0.3367', 'learning_rate': '0.0001908', 'entropy': '1.072', 'num_tokens': '1.181e+07', 'mean_token_accuracy': '0.7368', 'epoch': '0.01922'}
{'loss': '1.089', 'grad_norm': '0.3878', 'learning_rate': '0.0001908', 'entropy': '1.08', 'num_tokens': '1.182e+07', 'mean_token_accuracy': '0.7358', 'epoch': '0.01923'}
{'loss': '1.489', 'grad_norm': '0.4431', 'learning_rate': '0.0001908', 'entropy': '1.36', 'num_tokens': '1.182e+07', 'mean_token_accuracy': '0.695', 'epoch': '0.01924'}
{'loss': '0.733', 'grad_norm': '0.3792', 'learning_rate': '0.0001907', 'entropy': '0.7684', 'num_tokens': '1.183e+07', 'mean_token_accuracy': '0.8079', 'epoch': '0.01925'}
{'loss': '0.7352', 'grad_norm': '0.2631', 'learning_rate': '0.0001907', 'entropy': '0.805', 'num_tokens': '1.183e+07', 'mean_token_accuracy': '0.7927', 'epoch': '0.01926'}
{'loss': '1.058', 'grad_norm': '0.4285', 'learning_rate': '0.0001907', 'entropy': '1.028', 'num_tokens': '1.184e+07', 'mean_token_accuracy': '0.7638', 'epoch': '0.01927'}
{'loss': '0.6828', 'grad_norm': '0.2988', 'learning_rate': '0.0001907', 'entropy': '0.7036', 'num_tokens': '1.185e+07', 'mean_token_accuracy': '0.8256', 'epoch': '0.01927'}
{'loss': '0.7432', 'grad_norm': '0.3666', 'learning_rate': '0.0001907', 'entropy': '0.7461', 'num_tokens': '1.185e+07', 'mean_token_accuracy': '0.797', 'epoch': '0.01928'}
{'loss': '0.7487', 'grad_norm': '0.3002', 'learning_rate': '0.0001907', 'entropy': '0.7222', 'num_tokens': '1.185e+07', 'mean_token_accuracy': '0.8139', 'epoch': '0.01929'}
{'loss': '0.8008', 'grad_norm': '0.3476', 'learning_rate': '0.0001907', 'entropy': '0.7854', 'num_tokens': '1.186e+07', 'mean_token_accuracy': '0.8022', 'epoch': '0.0193'}
{'loss': '0.863', 'grad_norm': '0.3288', 'learning_rate': '0.0001907', 'entropy': '0.8249', 'num_tokens': '1.187e+07', 'mean_token_accuracy': '0.7899', 'epoch': '0.01931'}
{'loss': '0.8526', 'grad_norm': '0.3532', 'learning_rate': '0.0001906', 'entropy': '0.9034', 'num_tokens': '1.187e+07', 'mean_token_accuracy': '0.7822', 'epoch': '0.01932'}
{'loss': '1.179', 'grad_norm': '0.4255', 'learning_rate': '0.0001906', 'entropy': '1.326', 'num_tokens': '1.187e+07', 'mean_token_accuracy': '0.6926', 'epoch': '0.01933'}
{'loss': '0.9691', 'grad_norm': '0.3719', 'learning_rate': '0.0001906', 'entropy': '1.003', 'num_tokens': '1.188e+07', 'mean_token_accuracy': '0.7575', 'epoch': '0.01934'}
{'loss': '0.8333', 'grad_norm': '0.4059', 'learning_rate': '0.0001906', 'entropy': '0.9434', 'num_tokens': '1.189e+07', 'mean_token_accuracy': '0.7708', 'epoch': '0.01935'}
{'loss': '0.9879', 'grad_norm': '0.4037', 'learning_rate': '0.0001906', 'entropy': '1.037', 'num_tokens': '1.189e+07', 'mean_token_accuracy': '0.7604', 'epoch': '0.01935'}
{'loss': '0.8187', 'grad_norm': '0.448', 'learning_rate': '0.0001906', 'entropy': '0.9052', 'num_tokens': '1.19e+07', 'mean_token_accuracy': '0.7647', 'epoch': '0.01936'}
{'loss': '0.9937', 'grad_norm': '0.4093', 'learning_rate': '0.0001906', 'entropy': '1.004', 'num_tokens': '1.19e+07', 'mean_token_accuracy': '0.7565', 'epoch': '0.01937'}
{'loss': '1.531', 'grad_norm': '0.3331', 'learning_rate': '0.0001906', 'entropy': '1.261', 'num_tokens': '1.191e+07', 'mean_token_accuracy': '0.7062', 'epoch': '0.01938'}
{'loss': '0.6472', 'grad_norm': '0.3007', 'learning_rate': '0.0001906', 'entropy': '0.6448', 'num_tokens': '1.191e+07', 'mean_token_accuracy': '0.8223', 'epoch': '0.01939'}
{'loss': '0.732', 'grad_norm': '0.3018', 'learning_rate': '0.0001905', 'entropy': '0.8062', 'num_tokens': '1.192e+07', 'mean_token_accuracy': '0.7972', 'epoch': '0.0194'}
{'loss': '0.9678', 'grad_norm': '0.3806', 'learning_rate': '0.0001905', 'entropy': '0.9877', 'num_tokens': '1.192e+07', 'mean_token_accuracy': '0.7706', 'epoch': '0.01941'}
{'loss': '0.598', 'grad_norm': '0.2625', 'learning_rate': '0.0001905', 'entropy': '0.6447', 'num_tokens': '1.193e+07', 'mean_token_accuracy': '0.8461', 'epoch': '0.01942'}
{'loss': '0.8912', 'grad_norm': '0.3438', 'learning_rate': '0.0001905', 'entropy': '0.8458', 'num_tokens': '1.193e+07', 'mean_token_accuracy': '0.791', 'epoch': '0.01943'}
{'loss': '0.995', 'grad_norm': '0.4281', 'learning_rate': '0.0001905', 'entropy': '1.094', 'num_tokens': '1.194e+07', 'mean_token_accuracy': '0.7379', 'epoch': '0.01943'}
{'loss': '1.251', 'grad_norm': '0.4196', 'learning_rate': '0.0001905', 'entropy': '1.099', 'num_tokens': '1.194e+07', 'mean_token_accuracy': '0.7274', 'epoch': '0.01944'}
{'loss': '0.9301', 'grad_norm': '0.3643', 'learning_rate': '0.0001905', 'entropy': '0.958', 'num_tokens': '1.195e+07', 'mean_token_accuracy': '0.7628', 'epoch': '0.01945'}
{'loss': '0.8032', 'grad_norm': '0.3613', 'learning_rate': '0.0001905', 'entropy': '0.7882', 'num_tokens': '1.195e+07', 'mean_token_accuracy': '0.7963', 'epoch': '0.01946'}
{'loss': '1.006', 'grad_norm': '0.3652', 'learning_rate': '0.0001904', 'entropy': '1.061', 'num_tokens': '1.196e+07', 'mean_token_accuracy': '0.7424', 'epoch': '0.01947'}
{'loss': '1.002', 'grad_norm': '0.3666', 'learning_rate': '0.0001904', 'entropy': '0.9995', 'num_tokens': '1.196e+07', 'mean_token_accuracy': '0.7467', 'epoch': '0.01948'}
{'loss': '0.8478', 'grad_norm': '0.3604', 'learning_rate': '0.0001904', 'entropy': '0.8417', 'num_tokens': '1.197e+07', 'mean_token_accuracy': '0.7774', 'epoch': '0.01949'}
{'loss': '1.302', 'grad_norm': '0.4965', 'learning_rate': '0.0001904', 'entropy': '1.103', 'num_tokens': '1.197e+07', 'mean_token_accuracy': '0.7343', 'epoch': '0.0195'}
{'loss': '1.097', 'grad_norm': '0.3853', 'learning_rate': '0.0001904', 'entropy': '1.064', 'num_tokens': '1.198e+07', 'mean_token_accuracy': '0.7455', 'epoch': '0.01951'}
{'loss': '1.264', 'grad_norm': '0.3493', 'learning_rate': '0.0001904', 'entropy': '1.179', 'num_tokens': '1.199e+07', 'mean_token_accuracy': '0.7207', 'epoch': '0.01951'}
{'loss': '0.7695', 'grad_norm': '0.4067', 'learning_rate': '0.0001904', 'entropy': '0.8169', 'num_tokens': '1.199e+07', 'mean_token_accuracy': '0.8096', 'epoch': '0.01952'}
{'loss': '0.9955', 'grad_norm': '0.4092', 'learning_rate': '0.0001904', 'entropy': '0.9788', 'num_tokens': '1.2e+07', 'mean_token_accuracy': '0.7669', 'epoch': '0.01953'}
{'loss': '0.6621', 'grad_norm': '0.3008', 'learning_rate': '0.0001904', 'entropy': '0.7031', 'num_tokens': '1.2e+07', 'mean_token_accuracy': '0.8198', 'epoch': '0.01954'}
{'loss': '0.7517', 'grad_norm': '0.3427', 'learning_rate': '0.0001903', 'entropy': '0.8205', 'num_tokens': '1.2e+07', 'mean_token_accuracy': '0.8001', 'epoch': '0.01955'}
{'loss': '0.8226', 'grad_norm': '0.3603', 'learning_rate': '0.0001903', 'entropy': '0.9115', 'num_tokens': '1.201e+07', 'mean_token_accuracy': '0.7919', 'epoch': '0.01956'}
{'loss': '0.9444', 'grad_norm': '0.3459', 'learning_rate': '0.0001903', 'entropy': '0.9979', 'num_tokens': '1.201e+07', 'mean_token_accuracy': '0.7502', 'epoch': '0.01957'}
{'loss': '1.274', 'grad_norm': '0.4651', 'learning_rate': '0.0001903', 'entropy': '1.289', 'num_tokens': '1.202e+07', 'mean_token_accuracy': '0.6957', 'epoch': '0.01958'}

 18%|█▊        | 2213/12000 [1:19:45<4:54:53,  1.81s/it]
 18%|█▊        | 2214/12000 [1:19:46<4:31:06,  1.66s/it]
                                                        

 18%|█▊        | 2214/12000 [1:19:46<4:31:06,  1.66s/it]
 18%|█▊        | 2215/12000 [1:19:47<4:22:12,  1.61s/it]
                                                        

 18%|█▊        | 2215/12000 [1:19:47<4:22:12,  1.61s/it]
 18%|█▊        | 2216/12000 [1:19:49<4:12:35,  1.55s/it]
                                                        

 18%|█▊        | 2216/12000 [1:19:49<4:12:35,  1.55s/it]
 18%|█▊        | 2217/12000 [1:19:51<4:53:29,  1.80s/it]
                                                        

 18%|█▊        | 2217/12000 [1:19:51<4:53:29,  1.80s/it]
 18%|█▊        | 2218/12000 [1:19:55<6:51:48,  2.53s/it]
                                                        

 18%|█▊        | 2218/12000 [1:19:55<6:51:48,  2.53s/it]
 18%|█▊        | 2219/12000 [1:19:57<6:14:19,  2.30s/it]
                                                        

 18%|█▊        | 2219/12000 [1:19:57<6:14:19,  2.30s/it]
 18%|█▊        | 2220/12000 [1:20:00<6:26:31,  2.37s/it]
                                                        

 18%|█▊        | 2220/12000 [1:20:00<6:26:31,  2.37s/it]
 19%|█▊        | 2221/12000 [1:20:02<6:24:45,  2.36s/it]
                                                        

 19%|█▊        | 2221/12000 [1:20:02<6:24:45,  2.36s/it]
 19%|█▊        | 2222/12000 [1:20:04<5:58:46,  2.20s/it]
                                                        

 19%|█▊        | 2222/12000 [1:20:04<5:58:46,  2.20s/it]
 19%|█▊        | 2223/12000 [1:20:06<5:57:39,  2.19s/it]
                                                        

 19%|█▊        | 2223/12000 [1:20:06<5:57:39,  2.19s/it]
 19%|█▊        | 2224/12000 [1:20:08<5:27:46,  2.01s/it]
                                                        

 19%|█▊        | 2224/12000 [1:20:08<5:27:46,  2.01s/it]
 19%|█▊        | 2225/12000 [1:20:10<5:31:15,  2.03s/it]
                                                        

 19%|█▊        | 2225/12000 [1:20:10<5:31:15,  2.03s/it]
 19%|█▊        | 2226/12000 [1:20:14<7:35:27,  2.80s/it]
                                                        

 19%|█▊        | 2226/12000 [1:20:14<7:35:27,  2.80s/it]
 19%|█▊        | 2227/12000 [1:20:19<9:04:02,  3.34s/it]
                                                        

 19%|█▊        | 2227/12000 [1:20:19<9:04:02,  3.34s/it]
 19%|█▊        | 2228/12000 [1:20:21<8:01:53,  2.96s/it]
                                                        

 19%|█▊        | 2228/12000 [1:20:21<8:01:53,  2.96s/it]
 19%|█▊        | 2229/12000 [1:20:23<6:59:52,  2.58s/it]
                                                        

 19%|█▊        | 2229/12000 [1:20:23<6:59:52,  2.58s/it]
 19%|█▊        | 2230/12000 [1:20:26<7:25:13,  2.73s/it]
                                                        

 19%|█▊        | 2230/12000 [1:20:26<7:25:13,  2.73s/it]
 19%|█▊        | 2231/12000 [1:20:27<6:23:01,  2.35s/it]
                                                        

 19%|█▊        | 2231/12000 [1:20:27<6:23:01,  2.35s/it]
 19%|█▊        | 2232/12000 [1:20:29<6:07:48,  2.26s/it]
                                                        

 19%|█▊        | 2232/12000 [1:20:29<6:07:48,  2.26s/it]
 19%|█▊        | 2233/12000 [1:20:31<5:52:20,  2.16s/it]
                                                        

 19%|█▊        | 2233/12000 [1:20:31<5:52:20,  2.16s/it]
 19%|█▊        | 2234/12000 [1:20:36<7:41:10,  2.83s/it]
                                                        

 19%|█▊        | 2234/12000 [1:20:36<7:41:10,  2.83s/it]
 19%|█▊        | 2235/12000 [1:20:39<8:15:53,  3.05s/it]
                                                        

 19%|█▊        | 2235/12000 [1:20:39<8:15:53,  3.05s/it]
 19%|█▊        | 2236/12000 [1:20:41<7:08:37,  2.63s/it]
                                                        

 19%|█▊        | 2236/12000 [1:20:41<7:08:37,  2.63s/it]
 19%|█▊        | 2237/12000 [1:20:42<6:07:13,  2.26s/it]
                                                        

 19%|█▊        | 2237/12000 [1:20:42<6:07:13,  2.26s/it]
 19%|█▊        | 2238/12000 [1:20:44<5:57:32,  2.20s/it]
                                                        

 19%|█▊        | 2238/12000 [1:20:44<5:57:32,  2.20s/it]
 19%|█▊        | 2239/12000 [1:20:46<5:55:27,  2.19s/it]
                                                        

 19%|█▊        | 2239/12000 [1:20:46<5:55:27,  2.19s/it]
 19%|█▊        | 2240/12000 [1:20:50<7:18:30,  2.70s/it]
                                                        

 19%|█▊        | 2240/12000 [1:20:50<7:18:30,  2.70s/it]
 19%|█▊        | 2241/12000 [1:20:52<6:22:53,  2.35s/it]
                                                        

 19%|█▊        | 2241/12000 [1:20:52<6:22:53,  2.35s/it]
 19%|█▊        | 2242/12000 [1:20:53<5:37:30,  2.08s/it]
                                                        

 19%|█▊        | 2242/12000 [1:20:53<5:37:30,  2.08s/it]
 19%|█▊        | 2243/12000 [1:20:58<7:49:34,  2.89s/it]
                                                        

 19%|█▊        | 2243/12000 [1:20:58<7:49:34,  2.89s/it]
 19%|█▊        | 2244/12000 [1:21:00<6:38:55,  2.45s/it]
                                                        

 19%|█▊        | 2244/12000 [1:21:00<6:38:55,  2.45s/it]
 19%|█▊        | 2245/12000 [1:21:03<7:06:53,  2.63s/it]
                                                        

 19%|█▊        | 2245/12000 [1:21:03<7:06:53,  2.63s/it]
 19%|█▊        | 2246/12000 [1:21:05<7:04:43,  2.61s/it]
                                                        

 19%|█▊        | 2246/12000 [1:21:05<7:04:43,  2.61s/it]
 19%|█▊        | 2247/12000 [1:21:07<6:15:51,  2.31s/it]
                                                        

 19%|█▊        | 2247/12000 [1:21:07<6:15:51,  2.31s/it]
 19%|█▊        | 2248/12000 [1:21:09<6:17:53,  2.33s/it]
                                                        

 19%|█▊        | 2248/12000 [1:21:09<6:17:53,  2.33s/it]
 19%|█▊        | 2249/12000 [1:21:13<7:21:56,  2.72s/it]
                                                        

 19%|█▊        | 2249/12000 [1:21:13<7:21:56,  2.72s/it]
 19%|█▉        | 2250/12000 [1:21:18<9:05:41,  3.36s/it]
                                                        

 19%|█▉        | 2250/12000 [1:21:18<9:05:41,  3.36s/it]
 19%|█▉        | 2251/12000 [1:21:20<8:43:45,  3.22s/it]
                                                        

 19%|█▉        | 2251/12000 [1:21:20<8:43:45,  3.22s/it]
 19%|█▉        | 2252/12000 [1:21:23<8:32:19,  3.15s/it]
                                                        

 19%|█▉        | 2252/12000 [1:21:23<8:32:19,  3.15s/it]
 19%|█▉        | 2253/12000 [1:21:25<7:11:03,  2.65s/it]
                                                        

 19%|█▉        | 2253/12000 [1:21:25<7:11:03,  2.65s/it]
 19%|█▉        | 2254/12000 [1:21:27<6:34:27,  2.43s/it]
                                                        

 19%|█▉        | 2254/12000 [1:21:27<6:34:27,  2.43s/it]
 19%|█▉        | 2255/12000 [1:21:29<6:20:10,  2.34s/it]
                                                        

 19%|█▉        | 2255/12000 [1:21:29<6:20:10,  2.34s/it]
 19%|█▉        | 2256/12000 [1:21:30<5:35:21,  2.06s/it]
                                                        

 19%|█▉        | 2256/12000 [1:21:30<5:35:21,  2.06s/it]
 19%|█▉        | 2257/12000 [1:21:32<5:13:55,  1.93s/it]
                                                        

 19%|█▉        | 2257/12000 [1:21:32<5:13:55,  1.93s/it]
 19%|█▉        | 2258/12000 [1:21:34<5:32:28,  2.05s/it]
                                                        

 19%|█▉        | 2258/12000 [1:21:34<5:32:28,  2.05s/it]
 19%|█▉        | 2259/12000 [1:21:36<5:28:49,  2.03s/it]
                                                        

 19%|█▉        | 2259/12000 [1:21:36<5:28:49,  2.03s/it]
 19%|█▉        | 2260/12000 [1:21:38<5:07:55,  1.90s/it]
                                                        
{'loss': '0.7953', 'grad_norm': '0.3111', 'learning_rate': '0.0001903', 'entropy': '0.8244', 'num_tokens': '1.202e+07', 'mean_token_accuracy': '0.7998', 'epoch': '0.01958'}
{'loss': '0.8059', 'grad_norm': '0.3826', 'learning_rate': '0.0001903', 'entropy': '0.8532', 'num_tokens': '1.203e+07', 'mean_token_accuracy': '0.7719', 'epoch': '0.01959'}
{'loss': '0.9186', 'grad_norm': '0.3858', 'learning_rate': '0.0001903', 'entropy': '0.9293', 'num_tokens': '1.203e+07', 'mean_token_accuracy': '0.758', 'epoch': '0.0196'}
{'loss': '0.6819', 'grad_norm': '0.3519', 'learning_rate': '0.0001903', 'entropy': '0.6994', 'num_tokens': '1.203e+07', 'mean_token_accuracy': '0.8182', 'epoch': '0.01961'}
{'loss': '0.7103', 'grad_norm': '0.3132', 'learning_rate': '0.0001902', 'entropy': '0.6603', 'num_tokens': '1.204e+07', 'mean_token_accuracy': '0.8318', 'epoch': '0.01962'}
{'loss': '1.096', 'grad_norm': '0.425', 'learning_rate': '0.0001902', 'entropy': '1.12', 'num_tokens': '1.205e+07', 'mean_token_accuracy': '0.7364', 'epoch': '0.01963'}
{'loss': '0.596', 'grad_norm': '0.4063', 'learning_rate': '0.0001902', 'entropy': '0.6199', 'num_tokens': '1.205e+07', 'mean_token_accuracy': '0.8312', 'epoch': '0.01964'}
{'loss': '0.9714', 'grad_norm': '0.3863', 'learning_rate': '0.0001902', 'entropy': '1.105', 'num_tokens': '1.206e+07', 'mean_token_accuracy': '0.7409', 'epoch': '0.01965'}
{'loss': '0.67', 'grad_norm': '0.3367', 'learning_rate': '0.0001902', 'entropy': '0.7006', 'num_tokens': '1.206e+07', 'mean_token_accuracy': '0.8173', 'epoch': '0.01966'}
{'loss': '0.8809', 'grad_norm': '0.3349', 'learning_rate': '0.0001902', 'entropy': '0.9735', 'num_tokens': '1.207e+07', 'mean_token_accuracy': '0.7589', 'epoch': '0.01966'}
{'loss': '0.7299', 'grad_norm': '0.3091', 'learning_rate': '0.0001902', 'entropy': '0.7152', 'num_tokens': '1.207e+07', 'mean_token_accuracy': '0.824', 'epoch': '0.01967'}
{'loss': '0.6822', 'grad_norm': '0.4104', 'learning_rate': '0.0001902', 'entropy': '0.6999', 'num_tokens': '1.208e+07', 'mean_token_accuracy': '0.8263', 'epoch': '0.01968'}
{'loss': '0.8363', 'grad_norm': '0.2841', 'learning_rate': '0.0001902', 'entropy': '0.7534', 'num_tokens': '1.208e+07', 'mean_token_accuracy': '0.8034', 'epoch': '0.01969'}
{'loss': '0.7314', 'grad_norm': '0.2677', 'learning_rate': '0.0001901', 'entropy': '0.8849', 'num_tokens': '1.209e+07', 'mean_token_accuracy': '0.7733', 'epoch': '0.0197'}
{'loss': '1.088', 'grad_norm': '0.3483', 'learning_rate': '0.0001901', 'entropy': '0.9788', 'num_tokens': '1.21e+07', 'mean_token_accuracy': '0.7663', 'epoch': '0.01971'}
{'loss': '1.088', 'grad_norm': '0.3112', 'learning_rate': '0.0001901', 'entropy': '1.086', 'num_tokens': '1.211e+07', 'mean_token_accuracy': '0.7385', 'epoch': '0.01972'}
{'loss': '1.395', 'grad_norm': '0.3792', 'learning_rate': '0.0001901', 'entropy': '1.297', 'num_tokens': '1.211e+07', 'mean_token_accuracy': '0.6888', 'epoch': '0.01973'}
{'loss': '0.7048', 'grad_norm': '0.3128', 'learning_rate': '0.0001901', 'entropy': '0.6831', 'num_tokens': '1.212e+07', 'mean_token_accuracy': '0.8254', 'epoch': '0.01974'}
{'loss': '0.9624', 'grad_norm': '0.3482', 'learning_rate': '0.0001901', 'entropy': '0.9888', 'num_tokens': '1.212e+07', 'mean_token_accuracy': '0.7428', 'epoch': '0.01974'}
{'loss': '1.166', 'grad_norm': '0.4219', 'learning_rate': '0.0001901', 'entropy': '1.167', 'num_tokens': '1.213e+07', 'mean_token_accuracy': '0.7307', 'epoch': '0.01975'}
{'loss': '1.193', 'grad_norm': '0.4708', 'learning_rate': '0.0001901', 'entropy': '1.127', 'num_tokens': '1.213e+07', 'mean_token_accuracy': '0.7221', 'epoch': '0.01976'}
{'loss': '1.278', 'grad_norm': '0.3359', 'learning_rate': '0.00019', 'entropy': '1.11', 'num_tokens': '1.214e+07', 'mean_token_accuracy': '0.7491', 'epoch': '0.01977'}
{'loss': '1.391', 'grad_norm': '0.3819', 'learning_rate': '0.00019', 'entropy': '1.219', 'num_tokens': '1.215e+07', 'mean_token_accuracy': '0.7114', 'epoch': '0.01978'}
{'loss': '0.7845', 'grad_norm': '0.3605', 'learning_rate': '0.00019', 'entropy': '0.7641', 'num_tokens': '1.216e+07', 'mean_token_accuracy': '0.8062', 'epoch': '0.01979'}
{'loss': '1.073', 'grad_norm': '0.4905', 'learning_rate': '0.00019', 'entropy': '1.1', 'num_tokens': '1.216e+07', 'mean_token_accuracy': '0.7441', 'epoch': '0.0198'}
{'loss': '1.076', 'grad_norm': '0.4087', 'learning_rate': '0.00019', 'entropy': '0.9439', 'num_tokens': '1.217e+07', 'mean_token_accuracy': '0.7582', 'epoch': '0.01981'}
{'loss': '1.038', 'grad_norm': '0.4034', 'learning_rate': '0.00019', 'entropy': '1.044', 'num_tokens': '1.217e+07', 'mean_token_accuracy': '0.7557', 'epoch': '0.01981'}
{'loss': '1.053', 'grad_norm': '0.3286', 'learning_rate': '0.00019', 'entropy': '1.067', 'num_tokens': '1.218e+07', 'mean_token_accuracy': '0.7506', 'epoch': '0.01982'}
{'loss': '0.7875', 'grad_norm': '0.2852', 'learning_rate': '0.00019', 'entropy': '0.8438', 'num_tokens': '1.218e+07', 'mean_token_accuracy': '0.7879', 'epoch': '0.01983'}
{'loss': '0.7396', 'grad_norm': '0.3011', 'learning_rate': '0.0001899', 'entropy': '0.7568', 'num_tokens': '1.219e+07', 'mean_token_accuracy': '0.8072', 'epoch': '0.01984'}
{'loss': '1.685', 'grad_norm': '0.3809', 'learning_rate': '0.0001899', 'entropy': '1.155', 'num_tokens': '1.22e+07', 'mean_token_accuracy': '0.7314', 'epoch': '0.01985'}
{'loss': '0.7471', 'grad_norm': '0.3454', 'learning_rate': '0.0001899', 'entropy': '0.7852', 'num_tokens': '1.22e+07', 'mean_token_accuracy': '0.8023', 'epoch': '0.01986'}
{'loss': '0.951', 'grad_norm': '0.3361', 'learning_rate': '0.0001899', 'entropy': '1.047', 'num_tokens': '1.221e+07', 'mean_token_accuracy': '0.738', 'epoch': '0.01987'}
{'loss': '0.9704', 'grad_norm': '0.5565', 'learning_rate': '0.0001899', 'entropy': '0.8478', 'num_tokens': '1.221e+07', 'mean_token_accuracy': '0.7834', 'epoch': '0.01988'}
{'loss': '0.9665', 'grad_norm': '0.395', 'learning_rate': '0.0001899', 'entropy': '0.9255', 'num_tokens': '1.222e+07', 'mean_token_accuracy': '0.7606', 'epoch': '0.01989'}
{'loss': '0.8647', 'grad_norm': '0.3755', 'learning_rate': '0.0001899', 'entropy': '0.9044', 'num_tokens': '1.223e+07', 'mean_token_accuracy': '0.7734', 'epoch': '0.01989'}
{'loss': '0.5963', 'grad_norm': '0.3204', 'learning_rate': '0.0001899', 'entropy': '0.665', 'num_tokens': '1.223e+07', 'mean_token_accuracy': '0.8162', 'epoch': '0.0199'}
{'loss': '1.223', 'grad_norm': '0.3238', 'learning_rate': '0.0001899', 'entropy': '0.932', 'num_tokens': '1.224e+07', 'mean_token_accuracy': '0.7818', 'epoch': '0.01991'}
{'loss': '1.01', 'grad_norm': '0.3528', 'learning_rate': '0.0001898', 'entropy': '1.022', 'num_tokens': '1.225e+07', 'mean_token_accuracy': '0.7587', 'epoch': '0.01992'}
{'loss': '0.7654', 'grad_norm': '0.2812', 'learning_rate': '0.0001898', 'entropy': '0.8839', 'num_tokens': '1.225e+07', 'mean_token_accuracy': '0.7842', 'epoch': '0.01993'}
{'loss': '0.8443', 'grad_norm': '0.3751', 'learning_rate': '0.0001898', 'entropy': '0.8903', 'num_tokens': '1.226e+07', 'mean_token_accuracy': '0.7726', 'epoch': '0.01994'}
{'loss': '0.8299', 'grad_norm': '0.3206', 'learning_rate': '0.0001898', 'entropy': '0.793', 'num_tokens': '1.226e+07', 'mean_token_accuracy': '0.7994', 'epoch': '0.01995'}
{'loss': '0.9125', 'grad_norm': '0.3623', 'learning_rate': '0.0001898', 'entropy': '0.9261', 'num_tokens': '1.227e+07', 'mean_token_accuracy': '0.7714', 'epoch': '0.01996'}
{'loss': '0.9244', 'grad_norm': '0.3262', 'learning_rate': '0.0001898', 'entropy': '0.9348', 'num_tokens': '1.227e+07', 'mean_token_accuracy': '0.7786', 'epoch': '0.01997'}
{'loss': '0.9946', 'grad_norm': '0.4015', 'learning_rate': '0.0001898', 'entropy': '1.059', 'num_tokens': '1.228e+07', 'mean_token_accuracy': '0.7602', 'epoch': '0.01997'}
{'loss': '0.9638', 'grad_norm': '0.3287', 'learning_rate': '0.0001898', 'entropy': '0.9238', 'num_tokens': '1.229e+07', 'mean_token_accuracy': '0.783', 'epoch': '0.01998'}
{'loss': '0.9211', 'grad_norm': '0.3682', 'learning_rate': '0.0001897', 'entropy': '1.008', 'num_tokens': '1.229e+07', 'mean_token_accuracy': '0.769', 'epoch': '0.01999'}

 19%|█▉        | 2260/12000 [1:21:38<5:07:55,  1.90s/it]
 19%|█▉        | 2261/12000 [1:21:39<4:40:36,  1.73s/it]
                                                        

 19%|█▉        | 2261/12000 [1:21:39<4:40:36,  1.73s/it]
 19%|█▉        | 2262/12000 [1:21:45<8:04:49,  2.99s/it]
                                                        

 19%|█▉        | 2262/12000 [1:21:45<8:04:49,  2.99s/it]
 19%|█▉        | 2263/12000 [1:21:47<7:07:24,  2.63s/it]
                                                        

 19%|█▉        | 2263/12000 [1:21:47<7:07:24,  2.63s/it]
 19%|█▉        | 2264/12000 [1:21:50<7:10:57,  2.66s/it]
                                                        

 19%|█▉        | 2264/12000 [1:21:50<7:10:57,  2.66s/it]
 19%|█▉        | 2265/12000 [1:21:52<6:47:07,  2.51s/it]
                                                        

 19%|█▉        | 2265/12000 [1:21:52<6:47:07,  2.51s/it]
 19%|█▉        | 2266/12000 [1:21:55<7:17:19,  2.70s/it]
                                                        

 19%|█▉        | 2266/12000 [1:21:55<7:17:19,  2.70s/it]
 19%|█▉        | 2267/12000 [1:21:57<6:49:09,  2.52s/it]
                                                        

 19%|█▉        | 2267/12000 [1:21:57<6:49:09,  2.52s/it]
 19%|█▉        | 2268/12000 [1:22:00<7:05:36,  2.62s/it]
                                                        

 19%|█▉        | 2268/12000 [1:22:00<7:05:36,  2.62s/it]
 19%|█▉        | 2269/12000 [1:22:02<6:20:19,  2.35s/it]
                                                        

 19%|█▉        | 2269/12000 [1:22:02<6:20:19,  2.35s/it]
 19%|█▉        | 2270/12000 [1:22:05<6:58:53,  2.58s/it]
                                                        

 19%|█▉        | 2270/12000 [1:22:05<6:58:53,  2.58s/it]
 19%|█▉        | 2271/12000 [1:22:07<6:48:21,  2.52s/it]
                                                        

 19%|█▉        | 2271/12000 [1:22:07<6:48:21,  2.52s/it]
 19%|█▉        | 2272/12000 [1:22:09<6:36:40,  2.45s/it]
                                                        

 19%|█▉        | 2272/12000 [1:22:09<6:36:40,  2.45s/it]
 19%|█▉        | 2273/12000 [1:22:12<7:01:28,  2.60s/it]
                                                        

 19%|█▉        | 2273/12000 [1:22:12<7:01:28,  2.60s/it]
 19%|█▉        | 2274/12000 [1:22:14<6:10:03,  2.28s/it]
                                                        

 19%|█▉        | 2274/12000 [1:22:14<6:10:03,  2.28s/it]
 19%|█▉        | 2275/12000 [1:22:16<5:33:03,  2.05s/it]
                                                        

 19%|█▉        | 2275/12000 [1:22:16<5:33:03,  2.05s/it]
 19%|█▉        | 2276/12000 [1:22:17<4:59:07,  1.85s/it]
                                                        

 19%|█▉        | 2276/12000 [1:22:17<4:59:07,  1.85s/it]
 19%|█▉        | 2277/12000 [1:22:18<4:47:04,  1.77s/it]
                                                        

 19%|█▉        | 2277/12000 [1:22:18<4:47:04,  1.77s/it]
 19%|█▉        | 2278/12000 [1:22:20<4:41:40,  1.74s/it]
                                                        

 19%|█▉        | 2278/12000 [1:22:20<4:41:40,  1.74s/it]
 19%|█▉        | 2279/12000 [1:22:21<4:24:02,  1.63s/it]
                                                        

 19%|█▉        | 2279/12000 [1:22:21<4:24:02,  1.63s/it]
 19%|█▉        | 2280/12000 [1:22:23<4:13:42,  1.57s/it]
                                                        

 19%|█▉        | 2280/12000 [1:22:23<4:13:42,  1.57s/it]
 19%|█▉        | 2281/12000 [1:22:24<4:04:02,  1.51s/it]
                                                        

 19%|█▉        | 2281/12000 [1:22:24<4:04:02,  1.51s/it]
 19%|█▉        | 2282/12000 [1:22:26<3:59:41,  1.48s/it]
                                                        

 19%|█▉        | 2282/12000 [1:22:26<3:59:41,  1.48s/it]
 19%|█▉        | 2283/12000 [1:22:28<4:27:40,  1.65s/it]
                                                        

 19%|█▉        | 2283/12000 [1:22:28<4:27:40,  1.65s/it]
 19%|█▉        | 2284/12000 [1:22:33<7:00:47,  2.60s/it]
                                                        

 19%|█▉        | 2284/12000 [1:22:33<7:00:47,  2.60s/it]
 19%|█▉        | 2285/12000 [1:22:34<6:28:44,  2.40s/it]
                                                        

 19%|█▉        | 2285/12000 [1:22:34<6:28:44,  2.40s/it]
 19%|█▉        | 2286/12000 [1:22:36<5:49:54,  2.16s/it]
                                                        

 19%|█▉        | 2286/12000 [1:22:36<5:49:54,  2.16s/it]
 19%|█▉        | 2287/12000 [1:22:39<6:13:20,  2.31s/it]
                                                        

 19%|█▉        | 2287/12000 [1:22:39<6:13:20,  2.31s/it]
 19%|█▉        | 2288/12000 [1:22:40<5:44:00,  2.13s/it]
                                                        

 19%|█▉        | 2288/12000 [1:22:40<5:44:00,  2.13s/it]
 19%|█▉        | 2289/12000 [1:22:42<5:12:52,  1.93s/it]
                                                        

 19%|█▉        | 2289/12000 [1:22:42<5:12:52,  1.93s/it]
 19%|█▉        | 2290/12000 [1:22:43<4:53:02,  1.81s/it]
                                                        

 19%|█▉        | 2290/12000 [1:22:43<4:53:02,  1.81s/it]
 19%|█▉        | 2291/12000 [1:22:45<4:48:30,  1.78s/it]
                                                        

 19%|█▉        | 2291/12000 [1:22:45<4:48:30,  1.78s/it]
 19%|█▉        | 2292/12000 [1:22:47<4:32:39,  1.69s/it]
                                                        

 19%|█▉        | 2292/12000 [1:22:47<4:32:39,  1.69s/it]
 19%|█▉        | 2293/12000 [1:22:48<4:26:35,  1.65s/it]
                                                        

 19%|█▉        | 2293/12000 [1:22:48<4:26:35,  1.65s/it]
 19%|█▉        | 2294/12000 [1:22:51<5:13:42,  1.94s/it]
                                                        

 19%|█▉        | 2294/12000 [1:22:51<5:13:42,  1.94s/it]
 19%|█▉        | 2295/12000 [1:22:53<5:01:30,  1.86s/it]
                                                        

 19%|█▉        | 2295/12000 [1:22:53<5:01:30,  1.86s/it]
 19%|█▉        | 2296/12000 [1:22:55<5:19:00,  1.97s/it]
                                                        

 19%|█▉        | 2296/12000 [1:22:55<5:19:00,  1.97s/it]
 19%|█▉        | 2297/12000 [1:22:56<4:53:41,  1.82s/it]
                                                        

 19%|█▉        | 2297/12000 [1:22:56<4:53:41,  1.82s/it]
 19%|█▉        | 2298/12000 [1:22:58<4:35:10,  1.70s/it]
                                                        

 19%|█▉        | 2298/12000 [1:22:58<4:35:10,  1.70s/it]
 19%|█▉        | 2299/12000 [1:22:59<4:23:27,  1.63s/it]
                                                        

 19%|█▉        | 2299/12000 [1:22:59<4:23:27,  1.63s/it]
 19%|█▉        | 2300/12000 [1:23:01<4:29:58,  1.67s/it]
                                                        

 19%|█▉        | 2300/12000 [1:23:01<4:29:58,  1.67s/it]
 19%|█▉        | 2301/12000 [1:23:07<7:58:20,  2.96s/it]
                                                        

 19%|█▉        | 2301/12000 [1:23:07<7:58:20,  2.96s/it]
 19%|█▉        | 2302/12000 [1:23:09<7:21:53,  2.73s/it]
                                                        

 19%|█▉        | 2302/12000 [1:23:09<7:21:53,  2.73s/it]
 19%|█▉        | 2303/12000 [1:23:11<7:06:15,  2.64s/it]
                                                        

 19%|█▉        | 2303/12000 [1:23:11<7:06:15,  2.64s/it]
 19%|█▉        | 2304/12000 [1:23:13<6:16:37,  2.33s/it]
                                                        

 19%|█▉        | 2304/12000 [1:23:13<6:16:37,  2.33s/it]
 19%|█▉        | 2305/12000 [1:23:15<5:38:38,  2.10s/it]
                                                        

 19%|█▉        | 2305/12000 [1:23:15<5:38:38,  2.10s/it]
 19%|█▉        | 2306/12000 [1:23:19<7:49:40,  2.91s/it]
                                                        

 19%|█▉        | 2306/12000 [1:23:19<7:49:40,  2.91s/it]
 19%|█▉        | 2307/12000 [1:23:21<6:54:58,  2.57s/it]
                                                        
{'loss': '0.9909', 'grad_norm': '0.3753', 'learning_rate': '0.0001897', 'entropy': '0.9328', 'num_tokens': '1.23e+07', 'mean_token_accuracy': '0.7587', 'epoch': '0.02'}
{'loss': '1.147', 'grad_norm': '0.4563', 'learning_rate': '0.0001897', 'entropy': '1.174', 'num_tokens': '1.23e+07', 'mean_token_accuracy': '0.7255', 'epoch': '0.02001'}
{'loss': '0.9983', 'grad_norm': '0.2845', 'learning_rate': '0.0001897', 'entropy': '1.041', 'num_tokens': '1.231e+07', 'mean_token_accuracy': '0.7543', 'epoch': '0.02002'}
{'loss': '0.7411', 'grad_norm': '0.3624', 'learning_rate': '0.0001897', 'entropy': '0.7985', 'num_tokens': '1.232e+07', 'mean_token_accuracy': '0.7953', 'epoch': '0.02003'}
{'loss': '0.965', 'grad_norm': '0.3172', 'learning_rate': '0.0001897', 'entropy': '1.024', 'num_tokens': '1.232e+07', 'mean_token_accuracy': '0.752', 'epoch': '0.02004'}
{'loss': '0.8051', 'grad_norm': '0.37', 'learning_rate': '0.0001897', 'entropy': '0.7988', 'num_tokens': '1.233e+07', 'mean_token_accuracy': '0.7877', 'epoch': '0.02004'}
{'loss': '0.8409', 'grad_norm': '0.3462', 'learning_rate': '0.0001897', 'entropy': '0.9052', 'num_tokens': '1.233e+07', 'mean_token_accuracy': '0.7774', 'epoch': '0.02005'}
{'loss': '0.7702', 'grad_norm': '0.3127', 'learning_rate': '0.0001896', 'entropy': '0.724', 'num_tokens': '1.234e+07', 'mean_token_accuracy': '0.8125', 'epoch': '0.02006'}
{'loss': '0.8723', 'grad_norm': '0.3857', 'learning_rate': '0.0001896', 'entropy': '0.8144', 'num_tokens': '1.235e+07', 'mean_token_accuracy': '0.8051', 'epoch': '0.02007'}
{'loss': '0.8935', 'grad_norm': '0.4556', 'learning_rate': '0.0001896', 'entropy': '0.9253', 'num_tokens': '1.235e+07', 'mean_token_accuracy': '0.7614', 'epoch': '0.02008'}
{'loss': '0.7146', 'grad_norm': '0.2777', 'learning_rate': '0.0001896', 'entropy': '0.6543', 'num_tokens': '1.236e+07', 'mean_token_accuracy': '0.8252', 'epoch': '0.02009'}
{'loss': '0.9072', 'grad_norm': '0.3934', 'learning_rate': '0.0001896', 'entropy': '1', 'num_tokens': '1.236e+07', 'mean_token_accuracy': '0.7524', 'epoch': '0.0201'}
{'loss': '0.7219', 'grad_norm': '0.3662', 'learning_rate': '0.0001896', 'entropy': '0.7397', 'num_tokens': '1.237e+07', 'mean_token_accuracy': '0.8178', 'epoch': '0.02011'}
{'loss': '1.001', 'grad_norm': '0.3618', 'learning_rate': '0.0001896', 'entropy': '1.008', 'num_tokens': '1.238e+07', 'mean_token_accuracy': '0.7502', 'epoch': '0.02012'}
{'loss': '0.6946', 'grad_norm': '0.3595', 'learning_rate': '0.0001896', 'entropy': '0.6827', 'num_tokens': '1.238e+07', 'mean_token_accuracy': '0.8312', 'epoch': '0.02012'}
{'loss': '0.9164', 'grad_norm': '0.3909', 'learning_rate': '0.0001895', 'entropy': '0.9594', 'num_tokens': '1.239e+07', 'mean_token_accuracy': '0.7663', 'epoch': '0.02013'}
{'loss': '0.9235', 'grad_norm': '0.4159', 'learning_rate': '0.0001895', 'entropy': '0.9701', 'num_tokens': '1.239e+07', 'mean_token_accuracy': '0.7793', 'epoch': '0.02014'}
{'loss': '1.065', 'grad_norm': '0.4299', 'learning_rate': '0.0001895', 'entropy': '0.989', 'num_tokens': '1.24e+07', 'mean_token_accuracy': '0.7622', 'epoch': '0.02015'}
{'loss': '0.8366', 'grad_norm': '0.3583', 'learning_rate': '0.0001895', 'entropy': '0.9602', 'num_tokens': '1.24e+07', 'mean_token_accuracy': '0.7641', 'epoch': '0.02016'}
{'loss': '0.816', 'grad_norm': '0.3742', 'learning_rate': '0.0001895', 'entropy': '0.8536', 'num_tokens': '1.24e+07', 'mean_token_accuracy': '0.7913', 'epoch': '0.02017'}
{'loss': '0.6973', 'grad_norm': '0.3649', 'learning_rate': '0.0001895', 'entropy': '0.6957', 'num_tokens': '1.241e+07', 'mean_token_accuracy': '0.8195', 'epoch': '0.02018'}
{'loss': '1.019', 'grad_norm': '0.4207', 'learning_rate': '0.0001895', 'entropy': '1.065', 'num_tokens': '1.241e+07', 'mean_token_accuracy': '0.7421', 'epoch': '0.02019'}
{'loss': '0.635', 'grad_norm': '0.3513', 'learning_rate': '0.0001895', 'entropy': '0.6323', 'num_tokens': '1.242e+07', 'mean_token_accuracy': '0.8272', 'epoch': '0.0202'}
{'loss': '1.055', 'grad_norm': '0.3919', 'learning_rate': '0.0001894', 'entropy': '1.007', 'num_tokens': '1.242e+07', 'mean_token_accuracy': '0.7535', 'epoch': '0.0202'}
{'loss': '0.9381', 'grad_norm': '0.3789', 'learning_rate': '0.0001894', 'entropy': '0.9623', 'num_tokens': '1.243e+07', 'mean_token_accuracy': '0.7825', 'epoch': '0.02021'}
{'loss': '0.8611', 'grad_norm': '0.3683', 'learning_rate': '0.0001894', 'entropy': '0.8843', 'num_tokens': '1.243e+07', 'mean_token_accuracy': '0.7805', 'epoch': '0.02022'}
{'loss': '0.932', 'grad_norm': '0.3602', 'learning_rate': '0.0001894', 'entropy': '0.8749', 'num_tokens': '1.244e+07', 'mean_token_accuracy': '0.7783', 'epoch': '0.02023'}
{'loss': '0.9208', 'grad_norm': '0.4563', 'learning_rate': '0.0001894', 'entropy': '0.9767', 'num_tokens': '1.245e+07', 'mean_token_accuracy': '0.7685', 'epoch': '0.02024'}
{'loss': '0.6737', 'grad_norm': '0.3416', 'learning_rate': '0.0001894', 'entropy': '0.6955', 'num_tokens': '1.245e+07', 'mean_token_accuracy': '0.8211', 'epoch': '0.02025'}
{'loss': '0.9876', 'grad_norm': '0.3816', 'learning_rate': '0.0001894', 'entropy': '0.9393', 'num_tokens': '1.246e+07', 'mean_token_accuracy': '0.7505', 'epoch': '0.02026'}
{'loss': '0.8883', 'grad_norm': '0.355', 'learning_rate': '0.0001894', 'entropy': '0.8321', 'num_tokens': '1.246e+07', 'mean_token_accuracy': '0.7888', 'epoch': '0.02027'}
{'loss': '0.923', 'grad_norm': '0.3496', 'learning_rate': '0.0001893', 'entropy': '0.8991', 'num_tokens': '1.247e+07', 'mean_token_accuracy': '0.7765', 'epoch': '0.02027'}
{'loss': '0.7603', 'grad_norm': '0.2918', 'learning_rate': '0.0001893', 'entropy': '0.7601', 'num_tokens': '1.247e+07', 'mean_token_accuracy': '0.8018', 'epoch': '0.02028'}
{'loss': '0.7432', 'grad_norm': '0.3304', 'learning_rate': '0.0001893', 'entropy': '0.771', 'num_tokens': '1.247e+07', 'mean_token_accuracy': '0.8141', 'epoch': '0.02029'}
{'loss': '1.301', 'grad_norm': '0.4396', 'learning_rate': '0.0001893', 'entropy': '1.217', 'num_tokens': '1.248e+07', 'mean_token_accuracy': '0.7049', 'epoch': '0.0203'}
{'loss': '1.258', 'grad_norm': '0.4353', 'learning_rate': '0.0001893', 'entropy': '1.167', 'num_tokens': '1.249e+07', 'mean_token_accuracy': '0.7172', 'epoch': '0.02031'}
{'loss': '0.9723', 'grad_norm': '0.4407', 'learning_rate': '0.0001893', 'entropy': '0.854', 'num_tokens': '1.249e+07', 'mean_token_accuracy': '0.776', 'epoch': '0.02032'}
{'loss': '0.8023', 'grad_norm': '0.3931', 'learning_rate': '0.0001893', 'entropy': '0.7884', 'num_tokens': '1.25e+07', 'mean_token_accuracy': '0.7965', 'epoch': '0.02033'}
{'loss': '0.9137', 'grad_norm': '0.3582', 'learning_rate': '0.0001893', 'entropy': '0.961', 'num_tokens': '1.25e+07', 'mean_token_accuracy': '0.7745', 'epoch': '0.02034'}
{'loss': '1.125', 'grad_norm': '0.4607', 'learning_rate': '0.0001893', 'entropy': '1.103', 'num_tokens': '1.25e+07', 'mean_token_accuracy': '0.7233', 'epoch': '0.02035'}
{'loss': '1.128', 'grad_norm': '0.3344', 'learning_rate': '0.0001892', 'entropy': '1.105', 'num_tokens': '1.251e+07', 'mean_token_accuracy': '0.7346', 'epoch': '0.02035'}
{'loss': '0.4687', 'grad_norm': '0.206', 'learning_rate': '0.0001892', 'entropy': '0.5059', 'num_tokens': '1.252e+07', 'mean_token_accuracy': '0.8717', 'epoch': '0.02036'}
{'loss': '0.9444', 'grad_norm': '0.3298', 'learning_rate': '0.0001892', 'entropy': '0.9789', 'num_tokens': '1.252e+07', 'mean_token_accuracy': '0.7623', 'epoch': '0.02037'}
{'loss': '0.6449', 'grad_norm': '0.4079', 'learning_rate': '0.0001892', 'entropy': '0.6751', 'num_tokens': '1.253e+07', 'mean_token_accuracy': '0.8298', 'epoch': '0.02038'}
{'loss': '1.044', 'grad_norm': '0.4422', 'learning_rate': '0.0001892', 'entropy': '1.025', 'num_tokens': '1.254e+07', 'mean_token_accuracy': '0.7595', 'epoch': '0.02039'}
{'loss': '0.7928', 'grad_norm': '0.3177', 'learning_rate': '0.0001892', 'entropy': '0.741', 'num_tokens': '1.254e+07', 'mean_token_accuracy': '0.8182', 'epoch': '0.0204'}
{'loss': '0.9844', 'grad_norm': '0.3637', 'learning_rate': '0.0001892', 'entropy': '0.9957', 'num_tokens': '1.255e+07', 'mean_token_accuracy': '0.7599', 'epoch': '0.02041'}

 19%|█▉        | 2307/12000 [1:23:21<6:54:58,  2.57s/it]
 19%|█▉        | 2308/12000 [1:23:23<6:11:40,  2.30s/it]
                                                        

 19%|█▉        | 2308/12000 [1:23:23<6:11:40,  2.30s/it]
 19%|█▉        | 2309/12000 [1:23:24<5:29:49,  2.04s/it]
                                                        

 19%|█▉        | 2309/12000 [1:23:24<5:29:49,  2.04s/it]
 19%|█▉        | 2310/12000 [1:23:26<5:09:51,  1.92s/it]
                                                        

 19%|█▉        | 2310/12000 [1:23:26<5:09:51,  1.92s/it]
 19%|█▉        | 2311/12000 [1:23:28<4:53:56,  1.82s/it]
                                                        

 19%|█▉        | 2311/12000 [1:23:28<4:53:56,  1.82s/it]
 19%|█▉        | 2312/12000 [1:23:30<5:12:02,  1.93s/it]
                                                        

 19%|█▉        | 2312/12000 [1:23:30<5:12:02,  1.93s/it]
 19%|█▉        | 2313/12000 [1:23:31<4:55:36,  1.83s/it]
                                                        

 19%|█▉        | 2313/12000 [1:23:31<4:55:36,  1.83s/it]
 19%|█▉        | 2314/12000 [1:23:33<4:35:40,  1.71s/it]
                                                        

 19%|█▉        | 2314/12000 [1:23:33<4:35:40,  1.71s/it]
 19%|█▉        | 2315/12000 [1:23:34<4:25:16,  1.64s/it]
                                                        

 19%|█▉        | 2315/12000 [1:23:34<4:25:16,  1.64s/it]
 19%|█▉        | 2316/12000 [1:23:36<4:23:18,  1.63s/it]
                                                        

 19%|█▉        | 2316/12000 [1:23:36<4:23:18,  1.63s/it]
 19%|█▉        | 2317/12000 [1:23:37<4:19:31,  1.61s/it]
                                                        

 19%|█▉        | 2317/12000 [1:23:37<4:19:31,  1.61s/it]
 19%|█▉        | 2318/12000 [1:23:39<4:08:36,  1.54s/it]
                                                        

 19%|█▉        | 2318/12000 [1:23:39<4:08:36,  1.54s/it]
 19%|█▉        | 2319/12000 [1:23:42<5:13:04,  1.94s/it]
                                                        

 19%|█▉        | 2319/12000 [1:23:42<5:13:04,  1.94s/it]
 19%|█▉        | 2320/12000 [1:23:43<4:54:59,  1.83s/it]
                                                        

 19%|█▉        | 2320/12000 [1:23:43<4:54:59,  1.83s/it]
 19%|█▉        | 2321/12000 [1:23:45<4:48:06,  1.79s/it]
                                                        

 19%|█▉        | 2321/12000 [1:23:45<4:48:06,  1.79s/it]
 19%|█▉        | 2322/12000 [1:23:46<4:37:55,  1.72s/it]
                                                        

 19%|█▉        | 2322/12000 [1:23:46<4:37:55,  1.72s/it]
 19%|█▉        | 2323/12000 [1:23:48<4:30:57,  1.68s/it]
                                                        

 19%|█▉        | 2323/12000 [1:23:48<4:30:57,  1.68s/it]
 19%|█▉        | 2324/12000 [1:23:50<4:30:36,  1.68s/it]
                                                        

 19%|█▉        | 2324/12000 [1:23:50<4:30:36,  1.68s/it]
 19%|█▉        | 2325/12000 [1:23:51<4:28:45,  1.67s/it]
                                                        

 19%|█▉        | 2325/12000 [1:23:51<4:28:45,  1.67s/it]
 19%|█▉        | 2326/12000 [1:23:54<5:17:17,  1.97s/it]
                                                        

 19%|█▉        | 2326/12000 [1:23:54<5:17:17,  1.97s/it]
 19%|█▉        | 2327/12000 [1:23:56<4:54:25,  1.83s/it]
                                                        

 19%|█▉        | 2327/12000 [1:23:56<4:54:25,  1.83s/it]
 19%|█▉        | 2328/12000 [1:24:00<7:07:50,  2.65s/it]
                                                        

 19%|█▉        | 2328/12000 [1:24:00<7:07:50,  2.65s/it]
 19%|█▉        | 2329/12000 [1:24:02<6:12:16,  2.31s/it]
                                                        

 19%|█▉        | 2329/12000 [1:24:02<6:12:16,  2.31s/it]
 19%|█▉        | 2330/12000 [1:24:04<6:03:17,  2.25s/it]
                                                        

 19%|█▉        | 2330/12000 [1:24:04<6:03:17,  2.25s/it]
 19%|█▉        | 2331/12000 [1:24:05<5:36:22,  2.09s/it]
                                                        

 19%|█▉        | 2331/12000 [1:24:05<5:36:22,  2.09s/it]
 19%|█▉        | 2332/12000 [1:24:08<5:47:04,  2.15s/it]
                                                        

 19%|█▉        | 2332/12000 [1:24:08<5:47:04,  2.15s/it]
 19%|█▉        | 2333/12000 [1:24:09<5:24:18,  2.01s/it]
                                                        

 19%|█▉        | 2333/12000 [1:24:09<5:24:18,  2.01s/it]
 19%|█▉        | 2334/12000 [1:24:11<5:04:07,  1.89s/it]
                                                        

 19%|█▉        | 2334/12000 [1:24:11<5:04:07,  1.89s/it]
 19%|█▉        | 2335/12000 [1:24:13<4:46:36,  1.78s/it]
                                                        

 19%|█▉        | 2335/12000 [1:24:13<4:46:36,  1.78s/it]
 19%|█▉        | 2336/12000 [1:24:15<5:32:13,  2.06s/it]
                                                        

 19%|█▉        | 2336/12000 [1:24:15<5:32:13,  2.06s/it]
 19%|█▉        | 2337/12000 [1:24:19<6:33:17,  2.44s/it]
                                                        

 19%|█▉        | 2337/12000 [1:24:19<6:33:17,  2.44s/it]
 19%|█▉        | 2338/12000 [1:24:20<5:54:21,  2.20s/it]
                                                        

 19%|█▉        | 2338/12000 [1:24:20<5:54:21,  2.20s/it]
 19%|█▉        | 2339/12000 [1:24:22<5:40:30,  2.11s/it]
                                                        

 19%|█▉        | 2339/12000 [1:24:22<5:40:30,  2.11s/it]
 20%|█▉        | 2340/12000 [1:24:24<5:10:24,  1.93s/it]
                                                        

 20%|█▉        | 2340/12000 [1:24:24<5:10:24,  1.93s/it]
 20%|█▉        | 2341/12000 [1:24:26<5:20:07,  1.99s/it]
                                                        

 20%|█▉        | 2341/12000 [1:24:26<5:20:07,  1.99s/it]
 20%|█▉        | 2342/12000 [1:24:27<5:00:10,  1.86s/it]
                                                        

 20%|█▉        | 2342/12000 [1:24:27<5:00:10,  1.86s/it]
 20%|█▉        | 2343/12000 [1:24:29<4:54:00,  1.83s/it]
                                                        

 20%|█▉        | 2343/12000 [1:24:29<4:54:00,  1.83s/it]
 20%|█▉        | 2344/12000 [1:24:31<4:48:32,  1.79s/it]
                                                        

 20%|█▉        | 2344/12000 [1:24:31<4:48:32,  1.79s/it]
 20%|█▉        | 2345/12000 [1:24:32<4:37:42,  1.73s/it]
                                                        

 20%|█▉        | 2345/12000 [1:24:32<4:37:42,  1.73s/it]
 20%|█▉        | 2346/12000 [1:24:35<5:20:42,  1.99s/it]
                                                        

 20%|█▉        | 2346/12000 [1:24:35<5:20:42,  1.99s/it]
 20%|█▉        | 2347/12000 [1:24:37<5:20:46,  1.99s/it]
                                                        

 20%|█▉        | 2347/12000 [1:24:37<5:20:46,  1.99s/it]
 20%|█▉        | 2348/12000 [1:24:39<5:07:05,  1.91s/it]
                                                        

 20%|█▉        | 2348/12000 [1:24:39<5:07:05,  1.91s/it]
 20%|█▉        | 2349/12000 [1:24:41<5:15:00,  1.96s/it]
                                                        

 20%|█▉        | 2349/12000 [1:24:41<5:15:00,  1.96s/it]
 20%|█▉        | 2350/12000 [1:24:46<7:51:50,  2.93s/it]
                                                        

 20%|█▉        | 2350/12000 [1:24:46<7:51:50,  2.93s/it]
 20%|█▉        | 2351/12000 [1:24:49<8:01:49,  3.00s/it]
                                                        

 20%|█▉        | 2351/12000 [1:24:49<8:01:49,  3.00s/it]
 20%|█▉        | 2352/12000 [1:24:51<6:56:34,  2.59s/it]
                                                        

 20%|█▉        | 2352/12000 [1:24:51<6:56:34,  2.59s/it]
 20%|█▉        | 2353/12000 [1:24:53<6:25:28,  2.40s/it]
                                                        

 20%|█▉        | 2353/12000 [1:24:53<6:25:28,  2.40s/it]
 20%|█▉        | 2354/12000 [1:24:54<5:45:38,  2.15s/it]
                                                        
{'loss': '0.799', 'grad_norm': '0.3847', 'learning_rate': '0.0001892', 'entropy': '0.7588', 'num_tokens': '1.255e+07', 'mean_token_accuracy': '0.7956', 'epoch': '0.02042'}
{'loss': '1.274', 'grad_norm': '0.4141', 'learning_rate': '0.0001891', 'entropy': '1.232', 'num_tokens': '1.256e+07', 'mean_token_accuracy': '0.7246', 'epoch': '0.02043'}
{'loss': '1.104', 'grad_norm': '0.4716', 'learning_rate': '0.0001891', 'entropy': '1.109', 'num_tokens': '1.256e+07', 'mean_token_accuracy': '0.7172', 'epoch': '0.02043'}
{'loss': '1.256', 'grad_norm': '0.4151', 'learning_rate': '0.0001891', 'entropy': '1.229', 'num_tokens': '1.257e+07', 'mean_token_accuracy': '0.7097', 'epoch': '0.02044'}
{'loss': '1.102', 'grad_norm': '0.4763', 'learning_rate': '0.0001891', 'entropy': '1.086', 'num_tokens': '1.257e+07', 'mean_token_accuracy': '0.74', 'epoch': '0.02045'}
{'loss': '0.8955', 'grad_norm': '0.3338', 'learning_rate': '0.0001891', 'entropy': '0.9909', 'num_tokens': '1.258e+07', 'mean_token_accuracy': '0.7458', 'epoch': '0.02046'}
{'loss': '0.9953', 'grad_norm': '0.4021', 'learning_rate': '0.0001891', 'entropy': '1.041', 'num_tokens': '1.258e+07', 'mean_token_accuracy': '0.756', 'epoch': '0.02047'}
{'loss': '0.8157', 'grad_norm': '0.3567', 'learning_rate': '0.0001891', 'entropy': '0.8224', 'num_tokens': '1.259e+07', 'mean_token_accuracy': '0.7987', 'epoch': '0.02048'}
{'loss': '0.7787', 'grad_norm': '0.286', 'learning_rate': '0.0001891', 'entropy': '0.779', 'num_tokens': '1.259e+07', 'mean_token_accuracy': '0.7936', 'epoch': '0.02049'}
{'loss': '0.8014', 'grad_norm': '0.3528', 'learning_rate': '0.000189', 'entropy': '0.8269', 'num_tokens': '1.26e+07', 'mean_token_accuracy': '0.7941', 'epoch': '0.0205'}
{'loss': '0.7862', 'grad_norm': '0.3063', 'learning_rate': '0.000189', 'entropy': '0.8249', 'num_tokens': '1.26e+07', 'mean_token_accuracy': '0.7965', 'epoch': '0.02051'}
{'loss': '0.8522', 'grad_norm': '0.3138', 'learning_rate': '0.000189', 'entropy': '0.8768', 'num_tokens': '1.26e+07', 'mean_token_accuracy': '0.789', 'epoch': '0.02051'}
{'loss': '0.8787', 'grad_norm': '0.4429', 'learning_rate': '0.000189', 'entropy': '0.8724', 'num_tokens': '1.261e+07', 'mean_token_accuracy': '0.7836', 'epoch': '0.02052'}
{'loss': '1.209', 'grad_norm': '0.4346', 'learning_rate': '0.000189', 'entropy': '1.269', 'num_tokens': '1.262e+07', 'mean_token_accuracy': '0.7168', 'epoch': '0.02053'}
{'loss': '0.9638', 'grad_norm': '0.3735', 'learning_rate': '0.000189', 'entropy': '0.9212', 'num_tokens': '1.262e+07', 'mean_token_accuracy': '0.7628', 'epoch': '0.02054'}
{'loss': '0.9082', 'grad_norm': '0.3532', 'learning_rate': '0.000189', 'entropy': '0.9405', 'num_tokens': '1.262e+07', 'mean_token_accuracy': '0.77', 'epoch': '0.02055'}
{'loss': '0.7021', 'grad_norm': '0.3404', 'learning_rate': '0.000189', 'entropy': '0.7385', 'num_tokens': '1.263e+07', 'mean_token_accuracy': '0.8116', 'epoch': '0.02056'}
{'loss': '0.8519', 'grad_norm': '0.397', 'learning_rate': '0.0001889', 'entropy': '0.8561', 'num_tokens': '1.264e+07', 'mean_token_accuracy': '0.7884', 'epoch': '0.02057'}
{'loss': '0.7145', 'grad_norm': '0.342', 'learning_rate': '0.0001889', 'entropy': '0.7654', 'num_tokens': '1.264e+07', 'mean_token_accuracy': '0.8095', 'epoch': '0.02058'}
{'loss': '0.9474', 'grad_norm': '0.4153', 'learning_rate': '0.0001889', 'entropy': '0.9194', 'num_tokens': '1.265e+07', 'mean_token_accuracy': '0.7708', 'epoch': '0.02058'}
{'loss': '0.7586', 'grad_norm': '0.3285', 'learning_rate': '0.0001889', 'entropy': '0.7668', 'num_tokens': '1.265e+07', 'mean_token_accuracy': '0.7924', 'epoch': '0.02059'}
{'loss': '1.164', 'grad_norm': '0.4389', 'learning_rate': '0.0001889', 'entropy': '1.165', 'num_tokens': '1.266e+07', 'mean_token_accuracy': '0.7249', 'epoch': '0.0206'}
{'loss': '0.9149', 'grad_norm': '0.4176', 'learning_rate': '0.0001889', 'entropy': '0.984', 'num_tokens': '1.266e+07', 'mean_token_accuracy': '0.7483', 'epoch': '0.02061'}
{'loss': '0.9347', 'grad_norm': '0.4092', 'learning_rate': '0.0001889', 'entropy': '0.8455', 'num_tokens': '1.267e+07', 'mean_token_accuracy': '0.7901', 'epoch': '0.02062'}
{'loss': '1.008', 'grad_norm': '0.3466', 'learning_rate': '0.0001888', 'entropy': '1.003', 'num_tokens': '1.267e+07', 'mean_token_accuracy': '0.7463', 'epoch': '0.02063'}
{'loss': '1.035', 'grad_norm': '0.3539', 'learning_rate': '0.0001888', 'entropy': '0.9412', 'num_tokens': '1.268e+07', 'mean_token_accuracy': '0.7662', 'epoch': '0.02064'}
{'loss': '1.162', 'grad_norm': '0.4579', 'learning_rate': '0.0001888', 'entropy': '1.172', 'num_tokens': '1.268e+07', 'mean_token_accuracy': '0.6991', 'epoch': '0.02065'}
{'loss': '1.106', 'grad_norm': '0.3897', 'learning_rate': '0.0001888', 'entropy': '1.135', 'num_tokens': '1.269e+07', 'mean_token_accuracy': '0.7392', 'epoch': '0.02066'}
{'loss': '0.975', 'grad_norm': '0.4493', 'learning_rate': '0.0001888', 'entropy': '0.9913', 'num_tokens': '1.269e+07', 'mean_token_accuracy': '0.7522', 'epoch': '0.02066'}
{'loss': '0.6565', 'grad_norm': '0.255', 'learning_rate': '0.0001888', 'entropy': '0.8235', 'num_tokens': '1.27e+07', 'mean_token_accuracy': '0.7903', 'epoch': '0.02067'}
{'loss': '0.5335', 'grad_norm': '0.2237', 'learning_rate': '0.0001888', 'entropy': '0.6973', 'num_tokens': '1.271e+07', 'mean_token_accuracy': '0.8335', 'epoch': '0.02068'}
{'loss': '0.757', 'grad_norm': '0.3391', 'learning_rate': '0.0001888', 'entropy': '0.7644', 'num_tokens': '1.271e+07', 'mean_token_accuracy': '0.7914', 'epoch': '0.02069'}
{'loss': '0.8249', 'grad_norm': '0.3989', 'learning_rate': '0.0001887', 'entropy': '0.8057', 'num_tokens': '1.272e+07', 'mean_token_accuracy': '0.7953', 'epoch': '0.0207'}
{'loss': '0.6429', 'grad_norm': '0.3749', 'learning_rate': '0.0001887', 'entropy': '0.6839', 'num_tokens': '1.272e+07', 'mean_token_accuracy': '0.8235', 'epoch': '0.02071'}
{'loss': '0.7419', 'grad_norm': '0.3462', 'learning_rate': '0.0001887', 'entropy': '0.7056', 'num_tokens': '1.273e+07', 'mean_token_accuracy': '0.8214', 'epoch': '0.02072'}
{'loss': '0.9268', 'grad_norm': '0.3673', 'learning_rate': '0.0001887', 'entropy': '0.9436', 'num_tokens': '1.273e+07', 'mean_token_accuracy': '0.7657', 'epoch': '0.02073'}
{'loss': '0.7572', 'grad_norm': '0.4135', 'learning_rate': '0.0001887', 'entropy': '0.8116', 'num_tokens': '1.273e+07', 'mean_token_accuracy': '0.8107', 'epoch': '0.02074'}
{'loss': '0.9543', 'grad_norm': '0.3989', 'learning_rate': '0.0001887', 'entropy': '0.9332', 'num_tokens': '1.274e+07', 'mean_token_accuracy': '0.7714', 'epoch': '0.02074'}
{'loss': '1.055', 'grad_norm': '0.3512', 'learning_rate': '0.0001887', 'entropy': '1.049', 'num_tokens': '1.274e+07', 'mean_token_accuracy': '0.7403', 'epoch': '0.02075'}
{'loss': '0.7158', 'grad_norm': '0.288', 'learning_rate': '0.0001887', 'entropy': '0.9107', 'num_tokens': '1.275e+07', 'mean_token_accuracy': '0.7774', 'epoch': '0.02076'}
{'loss': '1.17', 'grad_norm': '0.5112', 'learning_rate': '0.0001886', 'entropy': '1.009', 'num_tokens': '1.276e+07', 'mean_token_accuracy': '0.7548', 'epoch': '0.02077'}
{'loss': '0.8813', 'grad_norm': '0.3893', 'learning_rate': '0.0001886', 'entropy': '0.8703', 'num_tokens': '1.276e+07', 'mean_token_accuracy': '0.7766', 'epoch': '0.02078'}
{'loss': '1.341', 'grad_norm': '0.3764', 'learning_rate': '0.0001886', 'entropy': '1.142', 'num_tokens': '1.277e+07', 'mean_token_accuracy': '0.7183', 'epoch': '0.02079'}
{'loss': '1.039', 'grad_norm': '0.4071', 'learning_rate': '0.0001886', 'entropy': '0.9874', 'num_tokens': '1.277e+07', 'mean_token_accuracy': '0.7579', 'epoch': '0.0208'}
{'loss': '0.9278', 'grad_norm': '0.3882', 'learning_rate': '0.0001886', 'entropy': '0.9682', 'num_tokens': '1.278e+07', 'mean_token_accuracy': '0.7426', 'epoch': '0.02081'}
{'loss': '1.123', 'grad_norm': '0.3758', 'learning_rate': '0.0001886', 'entropy': '1.178', 'num_tokens': '1.278e+07', 'mean_token_accuracy': '0.7304', 'epoch': '0.02081'}
{'loss': '0.8507', 'grad_norm': '0.3857', 'learning_rate': '0.0001886', 'entropy': '0.8113', 'num_tokens': '1.279e+07', 'mean_token_accuracy': '0.7927', 'epoch': '0.02082'}

 20%|█▉        | 2354/12000 [1:24:54<5:45:38,  2.15s/it]
 20%|█▉        | 2355/12000 [1:24:56<5:15:35,  1.96s/it]
                                                        

 20%|█▉        | 2355/12000 [1:24:56<5:15:35,  1.96s/it]
 20%|█▉        | 2356/12000 [1:24:58<5:31:12,  2.06s/it]
                                                        

 20%|█▉        | 2356/12000 [1:24:58<5:31:12,  2.06s/it]
 20%|█▉        | 2357/12000 [1:25:01<6:05:56,  2.28s/it]
                                                        

 20%|█▉        | 2357/12000 [1:25:01<6:05:56,  2.28s/it]
 20%|█▉        | 2358/12000 [1:25:04<7:06:28,  2.65s/it]
                                                        

 20%|█▉        | 2358/12000 [1:25:04<7:06:28,  2.65s/it]
 20%|█▉        | 2359/12000 [1:25:06<6:17:35,  2.35s/it]
                                                        

 20%|█▉        | 2359/12000 [1:25:06<6:17:35,  2.35s/it]
 20%|█▉        | 2360/12000 [1:25:08<5:56:42,  2.22s/it]
                                                        

 20%|█▉        | 2360/12000 [1:25:08<5:56:42,  2.22s/it]
 20%|█▉        | 2361/12000 [1:25:10<5:43:44,  2.14s/it]
                                                        

 20%|█▉        | 2361/12000 [1:25:10<5:43:44,  2.14s/it]
 20%|█▉        | 2362/12000 [1:25:12<5:21:26,  2.00s/it]
                                                        

 20%|█▉        | 2362/12000 [1:25:12<5:21:26,  2.00s/it]
 20%|█▉        | 2363/12000 [1:25:15<6:11:09,  2.31s/it]
                                                        

 20%|█▉        | 2363/12000 [1:25:15<6:11:09,  2.31s/it]
 20%|█▉        | 2364/12000 [1:25:16<5:25:29,  2.03s/it]
                                                        

 20%|█▉        | 2364/12000 [1:25:16<5:25:29,  2.03s/it]
 20%|█▉        | 2365/12000 [1:25:19<5:52:58,  2.20s/it]
                                                        

 20%|█▉        | 2365/12000 [1:25:19<5:52:58,  2.20s/it]
 20%|█▉        | 2366/12000 [1:25:22<7:05:47,  2.65s/it]
                                                        

 20%|█▉        | 2366/12000 [1:25:22<7:05:47,  2.65s/it]
 20%|█▉        | 2367/12000 [1:25:28<9:10:46,  3.43s/it]
                                                        

 20%|█▉        | 2367/12000 [1:25:28<9:10:46,  3.43s/it]
 20%|█▉        | 2368/12000 [1:25:31<8:49:09,  3.30s/it]
                                                        

 20%|█▉        | 2368/12000 [1:25:31<8:49:09,  3.30s/it]
 20%|█▉        | 2369/12000 [1:25:32<7:28:14,  2.79s/it]
                                                        

 20%|█▉        | 2369/12000 [1:25:32<7:28:14,  2.79s/it]
 20%|█▉        | 2370/12000 [1:25:34<6:33:49,  2.45s/it]
                                                        

 20%|█▉        | 2370/12000 [1:25:34<6:33:49,  2.45s/it]
 20%|█▉        | 2371/12000 [1:25:38<7:37:06,  2.85s/it]
                                                        

 20%|█▉        | 2371/12000 [1:25:38<7:37:06,  2.85s/it]
 20%|█▉        | 2372/12000 [1:25:41<8:10:11,  3.05s/it]
                                                        

 20%|█▉        | 2372/12000 [1:25:41<8:10:11,  3.05s/it]
 20%|█▉        | 2373/12000 [1:25:43<6:59:58,  2.62s/it]
                                                        

 20%|█▉        | 2373/12000 [1:25:43<6:59:58,  2.62s/it]
 20%|█▉        | 2374/12000 [1:25:44<6:19:03,  2.36s/it]
                                                        

 20%|█▉        | 2374/12000 [1:25:44<6:19:03,  2.36s/it]
 20%|█▉        | 2375/12000 [1:25:46<5:44:20,  2.15s/it]
                                                        

 20%|█▉        | 2375/12000 [1:25:46<5:44:20,  2.15s/it]
 20%|█▉        | 2376/12000 [1:25:48<5:35:57,  2.09s/it]
                                                        

 20%|█▉        | 2376/12000 [1:25:48<5:35:57,  2.09s/it]
 20%|█▉        | 2377/12000 [1:25:51<6:31:42,  2.44s/it]
                                                        

 20%|█▉        | 2377/12000 [1:25:51<6:31:42,  2.44s/it]
 20%|█▉        | 2378/12000 [1:25:54<6:48:42,  2.55s/it]
                                                        

 20%|█▉        | 2378/12000 [1:25:54<6:48:42,  2.55s/it]
 20%|█▉        | 2379/12000 [1:25:56<6:08:29,  2.30s/it]
                                                        

 20%|█▉        | 2379/12000 [1:25:56<6:08:29,  2.30s/it]
 20%|█▉        | 2380/12000 [1:25:58<5:41:18,  2.13s/it]
                                                        

 20%|█▉        | 2380/12000 [1:25:58<5:41:18,  2.13s/it]
 20%|█▉        | 2381/12000 [1:25:59<5:05:11,  1.90s/it]
                                                        

 20%|█▉        | 2381/12000 [1:25:59<5:05:11,  1.90s/it]
 20%|█▉        | 2382/12000 [1:26:01<4:55:56,  1.85s/it]
                                                        

 20%|█▉        | 2382/12000 [1:26:01<4:55:56,  1.85s/it]
 20%|█▉        | 2383/12000 [1:26:02<4:41:49,  1.76s/it]
                                                        

 20%|█▉        | 2383/12000 [1:26:02<4:41:49,  1.76s/it]
 20%|█▉        | 2384/12000 [1:26:04<4:30:12,  1.69s/it]
                                                        

 20%|█▉        | 2384/12000 [1:26:04<4:30:12,  1.69s/it]
 20%|█▉        | 2385/12000 [1:26:05<4:19:51,  1.62s/it]
                                                        

 20%|█▉        | 2385/12000 [1:26:05<4:19:51,  1.62s/it]
 20%|█▉        | 2386/12000 [1:26:09<5:47:21,  2.17s/it]
                                                        

 20%|█▉        | 2386/12000 [1:26:09<5:47:21,  2.17s/it]
 20%|█▉        | 2387/12000 [1:26:12<6:23:00,  2.39s/it]
                                                        

 20%|█▉        | 2387/12000 [1:26:12<6:23:00,  2.39s/it]
 20%|█▉        | 2388/12000 [1:26:13<5:52:23,  2.20s/it]
                                                        

 20%|█▉        | 2388/12000 [1:26:13<5:52:23,  2.20s/it]
 20%|█▉        | 2389/12000 [1:26:16<6:11:37,  2.32s/it]
                                                        

 20%|█▉        | 2389/12000 [1:26:16<6:11:37,  2.32s/it]
 20%|█▉        | 2390/12000 [1:26:18<6:19:08,  2.37s/it]
                                                        

 20%|█▉        | 2390/12000 [1:26:18<6:19:08,  2.37s/it]
 20%|█▉        | 2391/12000 [1:26:21<6:33:44,  2.46s/it]
                                                        

 20%|█▉        | 2391/12000 [1:26:21<6:33:44,  2.46s/it]
 20%|█▉        | 2392/12000 [1:26:23<5:46:12,  2.16s/it]
                                                        

 20%|█▉        | 2392/12000 [1:26:23<5:46:12,  2.16s/it]
 20%|█▉        | 2393/12000 [1:26:24<5:13:45,  1.96s/it]
                                                        

 20%|█▉        | 2393/12000 [1:26:24<5:13:45,  1.96s/it]
 20%|█▉        | 2394/12000 [1:26:27<6:25:51,  2.41s/it]
                                                        

 20%|█▉        | 2394/12000 [1:26:27<6:25:51,  2.41s/it]
 20%|█▉        | 2395/12000 [1:26:29<6:01:28,  2.26s/it]
                                                        

 20%|█▉        | 2395/12000 [1:26:29<6:01:28,  2.26s/it]
 20%|█▉        | 2396/12000 [1:26:32<6:00:28,  2.25s/it]
                                                        

 20%|█▉        | 2396/12000 [1:26:32<6:00:28,  2.25s/it]
 20%|█▉        | 2397/12000 [1:26:34<5:51:48,  2.20s/it]
                                                        

 20%|█▉        | 2397/12000 [1:26:34<5:51:48,  2.20s/it]
 20%|█▉        | 2398/12000 [1:26:36<6:14:37,  2.34s/it]
                                                        

 20%|█▉        | 2398/12000 [1:26:36<6:14:37,  2.34s/it]
 20%|█▉        | 2399/12000 [1:26:39<6:30:42,  2.44s/it]
                                                        

 20%|█▉        | 2399/12000 [1:26:39<6:30:42,  2.44s/it]
 20%|██        | 2400/12000 [1:26:41<5:58:44,  2.24s/it]
                                                        

 20%|██        | 2400/12000 [1:26:41<5:58:44,  2.24s/it]
 20%|██        | 2401/12000 [1:26:46<8:25:26,  3.16s/it]
                                                        
{'loss': '1.023', 'grad_norm': '0.4376', 'learning_rate': '0.0001886', 'entropy': '1.087', 'num_tokens': '1.279e+07', 'mean_token_accuracy': '0.7521', 'epoch': '0.02083'}
{'loss': '1.14', 'grad_norm': '0.4737', 'learning_rate': '0.0001885', 'entropy': '1.167', 'num_tokens': '1.28e+07', 'mean_token_accuracy': '0.7258', 'epoch': '0.02084'}
{'loss': '1.057', 'grad_norm': '0.3313', 'learning_rate': '0.0001885', 'entropy': '1.042', 'num_tokens': '1.28e+07', 'mean_token_accuracy': '0.7498', 'epoch': '0.02085'}
{'loss': '0.842', 'grad_norm': '0.2835', 'learning_rate': '0.0001885', 'entropy': '0.7292', 'num_tokens': '1.281e+07', 'mean_token_accuracy': '0.8029', 'epoch': '0.02086'}
{'loss': '1.099', 'grad_norm': '0.3447', 'learning_rate': '0.0001885', 'entropy': '0.9806', 'num_tokens': '1.282e+07', 'mean_token_accuracy': '0.7654', 'epoch': '0.02087'}
{'loss': '0.8029', 'grad_norm': '0.3942', 'learning_rate': '0.0001885', 'entropy': '0.7797', 'num_tokens': '1.282e+07', 'mean_token_accuracy': '0.8173', 'epoch': '0.02088'}
{'loss': '1.068', 'grad_norm': '0.4223', 'learning_rate': '0.0001885', 'entropy': '1.016', 'num_tokens': '1.283e+07', 'mean_token_accuracy': '0.7427', 'epoch': '0.02089'}
{'loss': '0.9813', 'grad_norm': '0.3825', 'learning_rate': '0.0001885', 'entropy': '0.8361', 'num_tokens': '1.283e+07', 'mean_token_accuracy': '0.7896', 'epoch': '0.02089'}
{'loss': '0.9551', 'grad_norm': '0.4045', 'learning_rate': '0.0001885', 'entropy': '1.063', 'num_tokens': '1.284e+07', 'mean_token_accuracy': '0.748', 'epoch': '0.0209'}
{'loss': '0.7674', 'grad_norm': '0.2954', 'learning_rate': '0.0001884', 'entropy': '0.905', 'num_tokens': '1.284e+07', 'mean_token_accuracy': '0.7877', 'epoch': '0.02091'}
{'loss': '0.8889', 'grad_norm': '0.3316', 'learning_rate': '0.0001884', 'entropy': '0.931', 'num_tokens': '1.285e+07', 'mean_token_accuracy': '0.7659', 'epoch': '0.02092'}
{'loss': '0.949', 'grad_norm': '0.3564', 'learning_rate': '0.0001884', 'entropy': '0.9336', 'num_tokens': '1.285e+07', 'mean_token_accuracy': '0.7574', 'epoch': '0.02093'}
{'loss': '0.8503', 'grad_norm': '0.3929', 'learning_rate': '0.0001884', 'entropy': '0.7914', 'num_tokens': '1.286e+07', 'mean_token_accuracy': '0.7976', 'epoch': '0.02094'}
{'loss': '0.6568', 'grad_norm': '0.2385', 'learning_rate': '0.0001884', 'entropy': '0.6833', 'num_tokens': '1.287e+07', 'mean_token_accuracy': '0.8268', 'epoch': '0.02095'}
{'loss': '1.019', 'grad_norm': '0.4214', 'learning_rate': '0.0001884', 'entropy': '1.032', 'num_tokens': '1.288e+07', 'mean_token_accuracy': '0.7464', 'epoch': '0.02096'}
{'loss': '0.9977', 'grad_norm': '0.3789', 'learning_rate': '0.0001884', 'entropy': '0.9861', 'num_tokens': '1.288e+07', 'mean_token_accuracy': '0.7481', 'epoch': '0.02097'}
{'loss': '0.9545', 'grad_norm': '0.4163', 'learning_rate': '0.0001884', 'entropy': '0.9382', 'num_tokens': '1.289e+07', 'mean_token_accuracy': '0.7555', 'epoch': '0.02097'}
{'loss': '0.9051', 'grad_norm': '0.3212', 'learning_rate': '0.0001883', 'entropy': '0.8695', 'num_tokens': '1.289e+07', 'mean_token_accuracy': '0.7873', 'epoch': '0.02098'}
{'loss': '1.005', 'grad_norm': '0.3763', 'learning_rate': '0.0001883', 'entropy': '1.044', 'num_tokens': '1.29e+07', 'mean_token_accuracy': '0.747', 'epoch': '0.02099'}
{'loss': '0.935', 'grad_norm': '0.4408', 'learning_rate': '0.0001883', 'entropy': '0.9647', 'num_tokens': '1.29e+07', 'mean_token_accuracy': '0.769', 'epoch': '0.021'}
{'loss': '1.217', 'grad_norm': '0.4728', 'learning_rate': '0.0001883', 'entropy': '1.137', 'num_tokens': '1.291e+07', 'mean_token_accuracy': '0.7172', 'epoch': '0.02101'}
{'loss': '1.224', 'grad_norm': '0.374', 'learning_rate': '0.0001883', 'entropy': '1.272', 'num_tokens': '1.291e+07', 'mean_token_accuracy': '0.718', 'epoch': '0.02102'}
{'loss': '0.8929', 'grad_norm': '0.5007', 'learning_rate': '0.0001883', 'entropy': '0.9938', 'num_tokens': '1.292e+07', 'mean_token_accuracy': '0.7523', 'epoch': '0.02103'}
{'loss': '1.324', 'grad_norm': '0.4067', 'learning_rate': '0.0001883', 'entropy': '1.407', 'num_tokens': '1.293e+07', 'mean_token_accuracy': '0.6727', 'epoch': '0.02104'}
{'loss': '1.062', 'grad_norm': '0.3386', 'learning_rate': '0.0001882', 'entropy': '1.082', 'num_tokens': '1.293e+07', 'mean_token_accuracy': '0.7264', 'epoch': '0.02104'}
{'loss': '0.9089', 'grad_norm': '0.3694', 'learning_rate': '0.0001882', 'entropy': '0.9394', 'num_tokens': '1.294e+07', 'mean_token_accuracy': '0.7699', 'epoch': '0.02105'}
{'loss': '0.93', 'grad_norm': '0.3747', 'learning_rate': '0.0001882', 'entropy': '0.7993', 'num_tokens': '1.294e+07', 'mean_token_accuracy': '0.7957', 'epoch': '0.02106'}
{'loss': '1.104', 'grad_norm': '0.4325', 'learning_rate': '0.0001882', 'entropy': '1.122', 'num_tokens': '1.295e+07', 'mean_token_accuracy': '0.7273', 'epoch': '0.02107'}
{'loss': '1.766', 'grad_norm': '0.5314', 'learning_rate': '0.0001882', 'entropy': '1.621', 'num_tokens': '1.295e+07', 'mean_token_accuracy': '0.6277', 'epoch': '0.02108'}
{'loss': '0.7987', 'grad_norm': '0.358', 'learning_rate': '0.0001882', 'entropy': '0.8609', 'num_tokens': '1.296e+07', 'mean_token_accuracy': '0.7937', 'epoch': '0.02109'}
{'loss': '1.426', 'grad_norm': '0.4072', 'learning_rate': '0.0001882', 'entropy': '1.376', 'num_tokens': '1.296e+07', 'mean_token_accuracy': '0.673', 'epoch': '0.0211'}
{'loss': '0.9228', 'grad_norm': '0.4276', 'learning_rate': '0.0001882', 'entropy': '0.9462', 'num_tokens': '1.296e+07', 'mean_token_accuracy': '0.7594', 'epoch': '0.02111'}
{'loss': '0.8928', 'grad_norm': '0.3227', 'learning_rate': '0.0001881', 'entropy': '0.8266', 'num_tokens': '1.297e+07', 'mean_token_accuracy': '0.7967', 'epoch': '0.02112'}
{'loss': '0.9781', 'grad_norm': '0.3736', 'learning_rate': '0.0001881', 'entropy': '1.054', 'num_tokens': '1.298e+07', 'mean_token_accuracy': '0.7505', 'epoch': '0.02112'}
{'loss': '0.9682', 'grad_norm': '0.3815', 'learning_rate': '0.0001881', 'entropy': '0.9408', 'num_tokens': '1.298e+07', 'mean_token_accuracy': '0.779', 'epoch': '0.02113'}
{'loss': '0.758', 'grad_norm': '0.3052', 'learning_rate': '0.0001881', 'entropy': '0.8481', 'num_tokens': '1.299e+07', 'mean_token_accuracy': '0.8037', 'epoch': '0.02114'}
{'loss': '1.278', 'grad_norm': '0.3718', 'learning_rate': '0.0001881', 'entropy': '1.006', 'num_tokens': '1.3e+07', 'mean_token_accuracy': '0.7728', 'epoch': '0.02115'}
{'loss': '0.8021', 'grad_norm': '0.4208', 'learning_rate': '0.0001881', 'entropy': '0.7843', 'num_tokens': '1.3e+07', 'mean_token_accuracy': '0.8073', 'epoch': '0.02116'}
{'loss': '0.5948', 'grad_norm': '0.3579', 'learning_rate': '0.0001881', 'entropy': '0.6392', 'num_tokens': '1.301e+07', 'mean_token_accuracy': '0.8425', 'epoch': '0.02117'}
{'loss': '0.7961', 'grad_norm': '0.3662', 'learning_rate': '0.0001881', 'entropy': '0.8468', 'num_tokens': '1.301e+07', 'mean_token_accuracy': '0.7966', 'epoch': '0.02118'}
{'loss': '1.058', 'grad_norm': '0.5345', 'learning_rate': '0.000188', 'entropy': '1.099', 'num_tokens': '1.302e+07', 'mean_token_accuracy': '0.74', 'epoch': '0.02119'}
{'loss': '0.7733', 'grad_norm': '0.3254', 'learning_rate': '0.000188', 'entropy': '0.7244', 'num_tokens': '1.302e+07', 'mean_token_accuracy': '0.8134', 'epoch': '0.0212'}
{'loss': '0.653', 'grad_norm': '0.2772', 'learning_rate': '0.000188', 'entropy': '0.7612', 'num_tokens': '1.303e+07', 'mean_token_accuracy': '0.8209', 'epoch': '0.0212'}
{'loss': '0.9013', 'grad_norm': '0.4273', 'learning_rate': '0.000188', 'entropy': '0.8671', 'num_tokens': '1.304e+07', 'mean_token_accuracy': '0.7885', 'epoch': '0.02121'}
{'loss': '0.9157', 'grad_norm': '0.5107', 'learning_rate': '0.000188', 'entropy': '0.8733', 'num_tokens': '1.304e+07', 'mean_token_accuracy': '0.7774', 'epoch': '0.02122'}
{'loss': '0.8328', 'grad_norm': '0.3113', 'learning_rate': '0.000188', 'entropy': '0.8487', 'num_tokens': '1.305e+07', 'mean_token_accuracy': '0.8011', 'epoch': '0.02123'}
{'loss': '0.9564', 'grad_norm': '0.3574', 'learning_rate': '0.000188', 'entropy': '0.9364', 'num_tokens': '1.305e+07', 'mean_token_accuracy': '0.7743', 'epoch': '0.02124'}

 20%|██        | 2401/12000 [1:26:46<8:25:26,  3.16s/it]
 20%|██        | 2402/12000 [1:26:48<7:18:12,  2.74s/it]
                                                        

 20%|██        | 2402/12000 [1:26:48<7:18:12,  2.74s/it]
 20%|██        | 2403/12000 [1:26:50<6:35:12,  2.47s/it]
                                                        

 20%|██        | 2403/12000 [1:26:50<6:35:12,  2.47s/it]
 20%|██        | 2404/12000 [1:26:51<5:47:21,  2.17s/it]
                                                        

 20%|██        | 2404/12000 [1:26:51<5:47:21,  2.17s/it]
 20%|██        | 2405/12000 [1:26:53<5:17:43,  1.99s/it]
                                                        

 20%|██        | 2405/12000 [1:26:53<5:17:43,  1.99s/it]
 20%|██        | 2406/12000 [1:26:54<4:59:21,  1.87s/it]
                                                        

 20%|██        | 2406/12000 [1:26:54<4:59:21,  1.87s/it]
 20%|██        | 2407/12000 [1:26:56<4:48:05,  1.80s/it]
                                                        

 20%|██        | 2407/12000 [1:26:56<4:48:05,  1.80s/it]
 20%|██        | 2408/12000 [1:26:58<5:06:32,  1.92s/it]
                                                        

 20%|██        | 2408/12000 [1:26:58<5:06:32,  1.92s/it]
 20%|██        | 2409/12000 [1:27:01<5:56:37,  2.23s/it]
                                                        

 20%|██        | 2409/12000 [1:27:01<5:56:37,  2.23s/it]
 20%|██        | 2410/12000 [1:27:03<5:50:50,  2.20s/it]
                                                        

 20%|██        | 2410/12000 [1:27:03<5:50:50,  2.20s/it]
 20%|██        | 2411/12000 [1:27:05<5:38:09,  2.12s/it]
                                                        

 20%|██        | 2411/12000 [1:27:05<5:38:09,  2.12s/it]
 20%|██        | 2412/12000 [1:27:08<6:10:44,  2.32s/it]
                                                        

 20%|██        | 2412/12000 [1:27:08<6:10:44,  2.32s/it]
 20%|██        | 2413/12000 [1:27:09<5:26:45,  2.04s/it]
                                                        

 20%|██        | 2413/12000 [1:27:09<5:26:45,  2.04s/it]
 20%|██        | 2414/12000 [1:27:13<6:19:41,  2.38s/it]
                                                        

 20%|██        | 2414/12000 [1:27:13<6:19:41,  2.38s/it]
 20%|██        | 2415/12000 [1:27:15<6:14:18,  2.34s/it]
                                                        

 20%|██        | 2415/12000 [1:27:15<6:14:18,  2.34s/it]
 20%|██        | 2416/12000 [1:27:18<7:06:03,  2.67s/it]
                                                        

 20%|██        | 2416/12000 [1:27:18<7:06:03,  2.67s/it]
 20%|██        | 2417/12000 [1:27:21<6:54:29,  2.60s/it]
                                                        

 20%|██        | 2417/12000 [1:27:21<6:54:29,  2.60s/it]
 20%|██        | 2418/12000 [1:27:23<6:34:29,  2.47s/it]
                                                        

 20%|██        | 2418/12000 [1:27:23<6:34:29,  2.47s/it]
 20%|██        | 2419/12000 [1:27:28<8:39:22,  3.25s/it]
                                                        

 20%|██        | 2419/12000 [1:27:28<8:39:22,  3.25s/it]
 20%|██        | 2420/12000 [1:27:31<8:09:36,  3.07s/it]
                                                        

 20%|██        | 2420/12000 [1:27:31<8:09:36,  3.07s/it]
 20%|██        | 2421/12000 [1:27:33<7:50:36,  2.95s/it]
                                                        

 20%|██        | 2421/12000 [1:27:33<7:50:36,  2.95s/it]
 20%|██        | 2422/12000 [1:27:35<6:57:15,  2.61s/it]
                                                        

 20%|██        | 2422/12000 [1:27:35<6:57:15,  2.61s/it]
 20%|██        | 2423/12000 [1:27:37<6:10:37,  2.32s/it]
                                                        

 20%|██        | 2423/12000 [1:27:37<6:10:37,  2.32s/it]
 20%|██        | 2424/12000 [1:27:38<5:29:40,  2.07s/it]
                                                        

 20%|██        | 2424/12000 [1:27:38<5:29:40,  2.07s/it]
 20%|██        | 2425/12000 [1:27:42<7:04:14,  2.66s/it]
                                                        

 20%|██        | 2425/12000 [1:27:42<7:04:14,  2.66s/it]
 20%|██        | 2426/12000 [1:27:44<6:35:39,  2.48s/it]
                                                        

 20%|██        | 2426/12000 [1:27:44<6:35:39,  2.48s/it]
 20%|██        | 2427/12000 [1:27:46<5:51:41,  2.20s/it]
                                                        

 20%|██        | 2427/12000 [1:27:46<5:51:41,  2.20s/it]
 20%|██        | 2428/12000 [1:27:47<5:09:56,  1.94s/it]
                                                        

 20%|██        | 2428/12000 [1:27:47<5:09:56,  1.94s/it]
 20%|██        | 2429/12000 [1:27:52<7:35:38,  2.86s/it]
                                                        

 20%|██        | 2429/12000 [1:27:52<7:35:38,  2.86s/it]
 20%|██        | 2430/12000 [1:27:55<7:40:09,  2.88s/it]
                                                        

 20%|██        | 2430/12000 [1:27:55<7:40:09,  2.88s/it]
 20%|██        | 2431/12000 [1:27:57<6:35:02,  2.48s/it]
                                                        

 20%|██        | 2431/12000 [1:27:57<6:35:02,  2.48s/it]
 20%|██        | 2432/12000 [1:27:58<5:55:34,  2.23s/it]
                                                        

 20%|██        | 2432/12000 [1:27:58<5:55:34,  2.23s/it]
 20%|██        | 2433/12000 [1:28:00<5:36:34,  2.11s/it]
                                                        

 20%|██        | 2433/12000 [1:28:00<5:36:34,  2.11s/it]
 20%|██        | 2434/12000 [1:28:02<5:35:27,  2.10s/it]
                                                        

 20%|██        | 2434/12000 [1:28:02<5:35:27,  2.10s/it]
 20%|██        | 2435/12000 [1:28:04<5:15:39,  1.98s/it]
                                                        

 20%|██        | 2435/12000 [1:28:04<5:15:39,  1.98s/it]
 20%|██        | 2436/12000 [1:28:05<4:46:15,  1.80s/it]
                                                        

 20%|██        | 2436/12000 [1:28:05<4:46:15,  1.80s/it]
 20%|██        | 2437/12000 [1:28:07<4:43:13,  1.78s/it]
                                                        

 20%|██        | 2437/12000 [1:28:07<4:43:13,  1.78s/it]
 20%|██        | 2438/12000 [1:28:11<6:42:32,  2.53s/it]
                                                        

 20%|██        | 2438/12000 [1:28:11<6:42:32,  2.53s/it]
 20%|██        | 2439/12000 [1:28:14<6:50:52,  2.58s/it]
                                                        

 20%|██        | 2439/12000 [1:28:14<6:50:52,  2.58s/it]
 20%|██        | 2440/12000 [1:28:16<6:32:43,  2.46s/it]
                                                        

 20%|██        | 2440/12000 [1:28:16<6:32:43,  2.46s/it]
 20%|██        | 2441/12000 [1:28:18<5:53:34,  2.22s/it]
                                                        

 20%|██        | 2441/12000 [1:28:18<5:53:34,  2.22s/it]
 20%|██        | 2442/12000 [1:28:21<6:33:05,  2.47s/it]
                                                        

 20%|██        | 2442/12000 [1:28:21<6:33:05,  2.47s/it]
 20%|██        | 2443/12000 [1:28:23<6:01:58,  2.27s/it]
                                                        

 20%|██        | 2443/12000 [1:28:23<6:01:58,  2.27s/it]
 20%|██        | 2444/12000 [1:28:24<5:35:31,  2.11s/it]
                                                        

 20%|██        | 2444/12000 [1:28:24<5:35:31,  2.11s/it]
 20%|██        | 2445/12000 [1:28:26<5:15:17,  1.98s/it]
                                                        

 20%|██        | 2445/12000 [1:28:26<5:15:17,  1.98s/it]
 20%|██        | 2446/12000 [1:28:28<5:06:53,  1.93s/it]
                                                        

 20%|██        | 2446/12000 [1:28:28<5:06:53,  1.93s/it]
 20%|██        | 2447/12000 [1:28:30<5:39:08,  2.13s/it]
                                                        

 20%|██        | 2447/12000 [1:28:30<5:39:08,  2.13s/it]
 20%|██        | 2448/12000 [1:28:34<6:36:28,  2.49s/it]
                                                        
{'loss': '1.061', 'grad_norm': '0.3064', 'learning_rate': '0.0001879', 'entropy': '1.256', 'num_tokens': '1.306e+07', 'mean_token_accuracy': '0.7289', 'epoch': '0.02125'}
{'loss': '0.8113', 'grad_norm': '0.309', 'learning_rate': '0.0001879', 'entropy': '0.8373', 'num_tokens': '1.307e+07', 'mean_token_accuracy': '0.7927', 'epoch': '0.02126'}
{'loss': '1.438', 'grad_norm': '0.4805', 'learning_rate': '0.0001879', 'entropy': '1.29', 'num_tokens': '1.308e+07', 'mean_token_accuracy': '0.6964', 'epoch': '0.02127'}
{'loss': '0.7294', 'grad_norm': '0.3208', 'learning_rate': '0.0001879', 'entropy': '0.7676', 'num_tokens': '1.308e+07', 'mean_token_accuracy': '0.7956', 'epoch': '0.02127'}
{'loss': '1.095', 'grad_norm': '0.5393', 'learning_rate': '0.0001879', 'entropy': '1.049', 'num_tokens': '1.308e+07', 'mean_token_accuracy': '0.7659', 'epoch': '0.02128'}
{'loss': '1.276', 'grad_norm': '0.4321', 'learning_rate': '0.0001879', 'entropy': '1.237', 'num_tokens': '1.309e+07', 'mean_token_accuracy': '0.707', 'epoch': '0.02129'}
{'loss': '0.9787', 'grad_norm': '0.3779', 'learning_rate': '0.0001879', 'entropy': '1.023', 'num_tokens': '1.309e+07', 'mean_token_accuracy': '0.7563', 'epoch': '0.0213'}
{'loss': '0.776', 'grad_norm': '0.3439', 'learning_rate': '0.0001879', 'entropy': '0.8168', 'num_tokens': '1.31e+07', 'mean_token_accuracy': '0.7947', 'epoch': '0.02131'}
{'loss': '0.549', 'grad_norm': '0.2366', 'learning_rate': '0.0001878', 'entropy': '0.5939', 'num_tokens': '1.311e+07', 'mean_token_accuracy': '0.8445', 'epoch': '0.02132'}
{'loss': '1.185', 'grad_norm': '0.414', 'learning_rate': '0.0001878', 'entropy': '1.089', 'num_tokens': '1.311e+07', 'mean_token_accuracy': '0.7407', 'epoch': '0.02133'}
{'loss': '0.9232', 'grad_norm': '0.3746', 'learning_rate': '0.0001878', 'entropy': '0.9746', 'num_tokens': '1.312e+07', 'mean_token_accuracy': '0.763', 'epoch': '0.02134'}
{'loss': '0.892', 'grad_norm': '0.4066', 'learning_rate': '0.0001878', 'entropy': '0.8905', 'num_tokens': '1.312e+07', 'mean_token_accuracy': '0.7744', 'epoch': '0.02135'}
{'loss': '0.9449', 'grad_norm': '0.5118', 'learning_rate': '0.0001878', 'entropy': '0.9459', 'num_tokens': '1.313e+07', 'mean_token_accuracy': '0.7629', 'epoch': '0.02135'}
{'loss': '1.025', 'grad_norm': '0.3977', 'learning_rate': '0.0001878', 'entropy': '1.005', 'num_tokens': '1.313e+07', 'mean_token_accuracy': '0.7543', 'epoch': '0.02136'}
{'loss': '0.8789', 'grad_norm': '0.5615', 'learning_rate': '0.0001878', 'entropy': '0.779', 'num_tokens': '1.314e+07', 'mean_token_accuracy': '0.8062', 'epoch': '0.02137'}
{'loss': '0.8903', 'grad_norm': '0.3585', 'learning_rate': '0.0001877', 'entropy': '0.8709', 'num_tokens': '1.314e+07', 'mean_token_accuracy': '0.7774', 'epoch': '0.02138'}
{'loss': '0.8841', 'grad_norm': '0.3647', 'learning_rate': '0.0001877', 'entropy': '0.7903', 'num_tokens': '1.315e+07', 'mean_token_accuracy': '0.8053', 'epoch': '0.02139'}
{'loss': '0.9964', 'grad_norm': '0.3859', 'learning_rate': '0.0001877', 'entropy': '0.9913', 'num_tokens': '1.316e+07', 'mean_token_accuracy': '0.7803', 'epoch': '0.0214'}
{'loss': '1.14', 'grad_norm': '0.392', 'learning_rate': '0.0001877', 'entropy': '1.083', 'num_tokens': '1.317e+07', 'mean_token_accuracy': '0.7456', 'epoch': '0.02141'}
{'loss': '0.7276', 'grad_norm': '0.3679', 'learning_rate': '0.0001877', 'entropy': '0.651', 'num_tokens': '1.317e+07', 'mean_token_accuracy': '0.8378', 'epoch': '0.02142'}
{'loss': '0.9965', 'grad_norm': '0.3895', 'learning_rate': '0.0001877', 'entropy': '0.8945', 'num_tokens': '1.318e+07', 'mean_token_accuracy': '0.7778', 'epoch': '0.02143'}
{'loss': '1.286', 'grad_norm': '0.4893', 'learning_rate': '0.0001877', 'entropy': '1.185', 'num_tokens': '1.318e+07', 'mean_token_accuracy': '0.7252', 'epoch': '0.02143'}
{'loss': '0.7896', 'grad_norm': '0.3966', 'learning_rate': '0.0001877', 'entropy': '0.8005', 'num_tokens': '1.319e+07', 'mean_token_accuracy': '0.7874', 'epoch': '0.02144'}
{'loss': '0.6263', 'grad_norm': '0.3637', 'learning_rate': '0.0001876', 'entropy': '0.6192', 'num_tokens': '1.319e+07', 'mean_token_accuracy': '0.8396', 'epoch': '0.02145'}
{'loss': '0.8953', 'grad_norm': '0.3935', 'learning_rate': '0.0001876', 'entropy': '1.102', 'num_tokens': '1.32e+07', 'mean_token_accuracy': '0.7488', 'epoch': '0.02146'}
{'loss': '0.8828', 'grad_norm': '0.3304', 'learning_rate': '0.0001876', 'entropy': '0.8674', 'num_tokens': '1.321e+07', 'mean_token_accuracy': '0.7864', 'epoch': '0.02147'}
{'loss': '0.9381', 'grad_norm': '0.4871', 'learning_rate': '0.0001876', 'entropy': '0.9868', 'num_tokens': '1.321e+07', 'mean_token_accuracy': '0.7582', 'epoch': '0.02148'}
{'loss': '0.602', 'grad_norm': '0.304', 'learning_rate': '0.0001876', 'entropy': '0.6971', 'num_tokens': '1.322e+07', 'mean_token_accuracy': '0.8296', 'epoch': '0.02149'}
{'loss': '2.441', 'grad_norm': '0.3308', 'learning_rate': '0.0001876', 'entropy': '1.519', 'num_tokens': '1.323e+07', 'mean_token_accuracy': '0.6831', 'epoch': '0.0215'}
{'loss': '0.8711', 'grad_norm': '0.342', 'learning_rate': '0.0001876', 'entropy': '0.711', 'num_tokens': '1.323e+07', 'mean_token_accuracy': '0.8253', 'epoch': '0.02151'}
{'loss': '1.456', 'grad_norm': '0.4754', 'learning_rate': '0.0001876', 'entropy': '1.277', 'num_tokens': '1.324e+07', 'mean_token_accuracy': '0.6917', 'epoch': '0.02151'}
{'loss': '0.9633', 'grad_norm': '0.3659', 'learning_rate': '0.0001875', 'entropy': '0.9053', 'num_tokens': '1.324e+07', 'mean_token_accuracy': '0.7734', 'epoch': '0.02152'}
{'loss': '0.9713', 'grad_norm': '0.3212', 'learning_rate': '0.0001875', 'entropy': '0.9306', 'num_tokens': '1.325e+07', 'mean_token_accuracy': '0.7493', 'epoch': '0.02153'}
{'loss': '0.9649', 'grad_norm': '0.4645', 'learning_rate': '0.0001875', 'entropy': '0.9648', 'num_tokens': '1.325e+07', 'mean_token_accuracy': '0.7694', 'epoch': '0.02154'}
{'loss': '1.026', 'grad_norm': '0.3449', 'learning_rate': '0.0001875', 'entropy': '1.039', 'num_tokens': '1.326e+07', 'mean_token_accuracy': '0.7482', 'epoch': '0.02155'}
{'loss': '0.6979', 'grad_norm': '0.3242', 'learning_rate': '0.0001875', 'entropy': '0.7472', 'num_tokens': '1.326e+07', 'mean_token_accuracy': '0.8154', 'epoch': '0.02156'}
{'loss': '0.8883', 'grad_norm': '0.4977', 'learning_rate': '0.0001875', 'entropy': '0.9041', 'num_tokens': '1.327e+07', 'mean_token_accuracy': '0.7759', 'epoch': '0.02157'}
{'loss': '0.8489', 'grad_norm': '0.2974', 'learning_rate': '0.0001875', 'entropy': '0.9457', 'num_tokens': '1.327e+07', 'mean_token_accuracy': '0.7845', 'epoch': '0.02158'}
{'loss': '0.9178', 'grad_norm': '0.494', 'learning_rate': '0.0001874', 'entropy': '0.9141', 'num_tokens': '1.328e+07', 'mean_token_accuracy': '0.7828', 'epoch': '0.02158'}
{'loss': '0.9925', 'grad_norm': '0.4205', 'learning_rate': '0.0001874', 'entropy': '0.9929', 'num_tokens': '1.329e+07', 'mean_token_accuracy': '0.7442', 'epoch': '0.02159'}
{'loss': '0.813', 'grad_norm': '0.4631', 'learning_rate': '0.0001874', 'entropy': '0.7615', 'num_tokens': '1.329e+07', 'mean_token_accuracy': '0.792', 'epoch': '0.0216'}
{'loss': '1.021', 'grad_norm': '0.3323', 'learning_rate': '0.0001874', 'entropy': '1.084', 'num_tokens': '1.33e+07', 'mean_token_accuracy': '0.737', 'epoch': '0.02161'}
{'loss': '0.7078', 'grad_norm': '0.3884', 'learning_rate': '0.0001874', 'entropy': '0.7275', 'num_tokens': '1.33e+07', 'mean_token_accuracy': '0.8197', 'epoch': '0.02162'}
{'loss': '0.9604', 'grad_norm': '0.3301', 'learning_rate': '0.0001874', 'entropy': '0.9467', 'num_tokens': '1.331e+07', 'mean_token_accuracy': '0.7648', 'epoch': '0.02163'}
{'loss': '0.7957', 'grad_norm': '0.3408', 'learning_rate': '0.0001874', 'entropy': '0.8078', 'num_tokens': '1.332e+07', 'mean_token_accuracy': '0.8063', 'epoch': '0.02164'}
{'loss': '0.9484', 'grad_norm': '0.3484', 'learning_rate': '0.0001874', 'entropy': '0.9693', 'num_tokens': '1.332e+07', 'mean_token_accuracy': '0.7632', 'epoch': '0.02165'}
{'loss': '1.082', 'grad_norm': '0.3424', 'learning_rate': '0.0001873', 'entropy': '1.078', 'num_tokens': '1.333e+07', 'mean_token_accuracy': '0.7476', 'epoch': '0.02166'}

 20%|██        | 2448/12000 [1:28:34<6:36:28,  2.49s/it]
 20%|██        | 2449/12000 [1:28:36<6:18:29,  2.38s/it]
                                                        

 20%|██        | 2449/12000 [1:28:36<6:18:29,  2.38s/it]
 20%|██        | 2450/12000 [1:28:38<6:25:52,  2.42s/it]
                                                        

 20%|██        | 2450/12000 [1:28:38<6:25:52,  2.42s/it]
 20%|██        | 2451/12000 [1:28:41<6:50:59,  2.58s/it]
                                                        

 20%|██        | 2451/12000 [1:28:41<6:50:59,  2.58s/it]
 20%|██        | 2452/12000 [1:28:43<5:55:51,  2.24s/it]
                                                        

 20%|██        | 2452/12000 [1:28:43<5:55:51,  2.24s/it]
 20%|██        | 2453/12000 [1:28:44<5:19:55,  2.01s/it]
                                                        

 20%|██        | 2453/12000 [1:28:44<5:19:55,  2.01s/it]
 20%|██        | 2454/12000 [1:28:46<5:00:30,  1.89s/it]
                                                        

 20%|██        | 2454/12000 [1:28:46<5:00:30,  1.89s/it]
 20%|██        | 2455/12000 [1:28:48<5:10:41,  1.95s/it]
                                                        

 20%|██        | 2455/12000 [1:28:48<5:10:41,  1.95s/it]
 20%|██        | 2456/12000 [1:28:50<5:19:40,  2.01s/it]
                                                        

 20%|██        | 2456/12000 [1:28:50<5:19:40,  2.01s/it]
 20%|██        | 2457/12000 [1:28:52<5:09:53,  1.95s/it]
                                                        

 20%|██        | 2457/12000 [1:28:52<5:09:53,  1.95s/it]
 20%|██        | 2458/12000 [1:28:53<4:40:54,  1.77s/it]
                                                        

 20%|██        | 2458/12000 [1:28:53<4:40:54,  1.77s/it]
 20%|██        | 2459/12000 [1:28:55<4:58:35,  1.88s/it]
                                                        

 20%|██        | 2459/12000 [1:28:55<4:58:35,  1.88s/it]
 20%|██        | 2460/12000 [1:28:59<6:20:29,  2.39s/it]
                                                        

 20%|██        | 2460/12000 [1:28:59<6:20:29,  2.39s/it]
 21%|██        | 2461/12000 [1:29:03<7:16:57,  2.75s/it]
                                                        

 21%|██        | 2461/12000 [1:29:03<7:16:57,  2.75s/it]
 21%|██        | 2462/12000 [1:29:05<6:36:50,  2.50s/it]
                                                        

 21%|██        | 2462/12000 [1:29:05<6:36:50,  2.50s/it]
 21%|██        | 2463/12000 [1:29:07<6:32:36,  2.47s/it]
                                                        

 21%|██        | 2463/12000 [1:29:07<6:32:36,  2.47s/it]
 21%|██        | 2464/12000 [1:29:09<6:00:42,  2.27s/it]
                                                        

 21%|██        | 2464/12000 [1:29:09<6:00:42,  2.27s/it]
 21%|██        | 2465/12000 [1:29:10<5:22:30,  2.03s/it]
                                                        

 21%|██        | 2465/12000 [1:29:10<5:22:30,  2.03s/it]
 21%|██        | 2466/12000 [1:29:12<4:52:37,  1.84s/it]
                                                        

 21%|██        | 2466/12000 [1:29:12<4:52:37,  1.84s/it]
 21%|██        | 2467/12000 [1:29:16<6:49:37,  2.58s/it]
                                                        

 21%|██        | 2467/12000 [1:29:16<6:49:37,  2.58s/it]
 21%|██        | 2468/12000 [1:29:18<6:48:12,  2.57s/it]
                                                        

 21%|██        | 2468/12000 [1:29:18<6:48:12,  2.57s/it]
 21%|██        | 2469/12000 [1:29:20<5:56:31,  2.24s/it]
                                                        

 21%|██        | 2469/12000 [1:29:20<5:56:31,  2.24s/it]
 21%|██        | 2470/12000 [1:29:23<6:49:52,  2.58s/it]
                                                        

 21%|██        | 2470/12000 [1:29:23<6:49:52,  2.58s/it]
 21%|██        | 2471/12000 [1:29:26<6:45:19,  2.55s/it]
                                                        

 21%|██        | 2471/12000 [1:29:26<6:45:19,  2.55s/it]
 21%|██        | 2472/12000 [1:29:27<5:56:39,  2.25s/it]
                                                        

 21%|██        | 2472/12000 [1:29:27<5:56:39,  2.25s/it]
 21%|██        | 2473/12000 [1:29:29<5:24:59,  2.05s/it]
                                                        

 21%|██        | 2473/12000 [1:29:29<5:24:59,  2.05s/it]
 21%|██        | 2474/12000 [1:29:31<5:08:45,  1.94s/it]
                                                        

 21%|██        | 2474/12000 [1:29:31<5:08:45,  1.94s/it]
 21%|██        | 2475/12000 [1:29:32<4:56:41,  1.87s/it]
                                                        

 21%|██        | 2475/12000 [1:29:32<4:56:41,  1.87s/it]
 21%|██        | 2476/12000 [1:29:36<6:05:27,  2.30s/it]
                                                        

 21%|██        | 2476/12000 [1:29:36<6:05:27,  2.30s/it]
 21%|██        | 2477/12000 [1:29:37<5:23:59,  2.04s/it]
                                                        

 21%|██        | 2477/12000 [1:29:37<5:23:59,  2.04s/it]
 21%|██        | 2478/12000 [1:29:39<5:03:45,  1.91s/it]
                                                        

 21%|██        | 2478/12000 [1:29:39<5:03:45,  1.91s/it]
 21%|██        | 2479/12000 [1:29:41<5:43:28,  2.16s/it]
                                                        

 21%|██        | 2479/12000 [1:29:41<5:43:28,  2.16s/it]
 21%|██        | 2480/12000 [1:29:43<5:37:47,  2.13s/it]
                                                        

 21%|██        | 2480/12000 [1:29:43<5:37:47,  2.13s/it]
 21%|██        | 2481/12000 [1:29:46<5:37:20,  2.13s/it]
                                                        

 21%|██        | 2481/12000 [1:29:46<5:37:20,  2.13s/it]
 21%|██        | 2482/12000 [1:29:50<7:08:30,  2.70s/it]
                                                        

 21%|██        | 2482/12000 [1:29:50<7:08:30,  2.70s/it]
 21%|██        | 2483/12000 [1:29:51<6:16:04,  2.37s/it]
                                                        

 21%|██        | 2483/12000 [1:29:51<6:16:04,  2.37s/it]
 21%|██        | 2484/12000 [1:29:56<8:03:38,  3.05s/it]
                                                        

 21%|██        | 2484/12000 [1:29:56<8:03:38,  3.05s/it]
 21%|██        | 2485/12000 [1:29:58<6:55:50,  2.62s/it]
                                                        

 21%|██        | 2485/12000 [1:29:58<6:55:50,  2.62s/it]
 21%|██        | 2486/12000 [1:29:59<6:05:02,  2.30s/it]
                                                        

 21%|██        | 2486/12000 [1:29:59<6:05:02,  2.30s/it]
 21%|██        | 2487/12000 [1:30:01<5:26:19,  2.06s/it]
                                                        

 21%|██        | 2487/12000 [1:30:01<5:26:19,  2.06s/it]
 21%|██        | 2488/12000 [1:30:02<5:08:49,  1.95s/it]
                                                        

 21%|██        | 2488/12000 [1:30:02<5:08:49,  1.95s/it]
 21%|██        | 2489/12000 [1:30:05<5:41:51,  2.16s/it]
                                                        

 21%|██        | 2489/12000 [1:30:05<5:41:51,  2.16s/it]
 21%|██        | 2490/12000 [1:30:08<6:34:04,  2.49s/it]
                                                        

 21%|██        | 2490/12000 [1:30:08<6:34:04,  2.49s/it]
 21%|██        | 2491/12000 [1:30:10<6:07:18,  2.32s/it]
                                                        

 21%|██        | 2491/12000 [1:30:10<6:07:18,  2.32s/it]
 21%|██        | 2492/12000 [1:30:12<6:11:00,  2.34s/it]
                                                        

 21%|██        | 2492/12000 [1:30:12<6:11:00,  2.34s/it]
 21%|██        | 2493/12000 [1:30:14<5:46:04,  2.18s/it]
                                                        

 21%|██        | 2493/12000 [1:30:14<5:46:04,  2.18s/it]
 21%|██        | 2494/12000 [1:30:16<5:24:57,  2.05s/it]
                                                        

 21%|██        | 2494/12000 [1:30:16<5:24:57,  2.05s/it]
 21%|██        | 2495/12000 [1:30:17<4:53:12,  1.85s/it]
                                                        
{'loss': '0.6473', 'grad_norm': '0.3043', 'learning_rate': '0.0001873', 'entropy': '0.6101', 'num_tokens': '1.334e+07', 'mean_token_accuracy': '0.8511', 'epoch': '0.02166'}
{'loss': '1.095', 'grad_norm': '0.3408', 'learning_rate': '0.0001873', 'entropy': '1.042', 'num_tokens': '1.334e+07', 'mean_token_accuracy': '0.745', 'epoch': '0.02167'}
{'loss': '0.8969', 'grad_norm': '0.344', 'learning_rate': '0.0001873', 'entropy': '0.9508', 'num_tokens': '1.335e+07', 'mean_token_accuracy': '0.7724', 'epoch': '0.02168'}
{'loss': '0.6532', 'grad_norm': '0.4197', 'learning_rate': '0.0001873', 'entropy': '0.6696', 'num_tokens': '1.335e+07', 'mean_token_accuracy': '0.8279', 'epoch': '0.02169'}
{'loss': '0.9524', 'grad_norm': '0.3719', 'learning_rate': '0.0001873', 'entropy': '0.9571', 'num_tokens': '1.336e+07', 'mean_token_accuracy': '0.7674', 'epoch': '0.0217'}
{'loss': '0.8671', 'grad_norm': '0.3602', 'learning_rate': '0.0001873', 'entropy': '0.8853', 'num_tokens': '1.336e+07', 'mean_token_accuracy': '0.7737', 'epoch': '0.02171'}
{'loss': '0.8835', 'grad_norm': '0.3453', 'learning_rate': '0.0001872', 'entropy': '0.8531', 'num_tokens': '1.337e+07', 'mean_token_accuracy': '0.7869', 'epoch': '0.02172'}
{'loss': '0.8892', 'grad_norm': '0.3437', 'learning_rate': '0.0001872', 'entropy': '0.8288', 'num_tokens': '1.337e+07', 'mean_token_accuracy': '0.7943', 'epoch': '0.02173'}
{'loss': '0.9237', 'grad_norm': '0.3368', 'learning_rate': '0.0001872', 'entropy': '0.9099', 'num_tokens': '1.338e+07', 'mean_token_accuracy': '0.767', 'epoch': '0.02174'}
{'loss': '0.984', 'grad_norm': '0.3779', 'learning_rate': '0.0001872', 'entropy': '0.9207', 'num_tokens': '1.338e+07', 'mean_token_accuracy': '0.7645', 'epoch': '0.02174'}
{'loss': '1.012', 'grad_norm': '0.3469', 'learning_rate': '0.0001872', 'entropy': '1.003', 'num_tokens': '1.339e+07', 'mean_token_accuracy': '0.7498', 'epoch': '0.02175'}
{'loss': '0.7132', 'grad_norm': '0.3556', 'learning_rate': '0.0001872', 'entropy': '0.7159', 'num_tokens': '1.339e+07', 'mean_token_accuracy': '0.8159', 'epoch': '0.02176'}
{'loss': '0.7724', 'grad_norm': '0.3397', 'learning_rate': '0.0001872', 'entropy': '0.7752', 'num_tokens': '1.34e+07', 'mean_token_accuracy': '0.7933', 'epoch': '0.02177'}
{'loss': '1.02', 'grad_norm': '0.2884', 'learning_rate': '0.0001871', 'entropy': '0.9308', 'num_tokens': '1.341e+07', 'mean_token_accuracy': '0.7968', 'epoch': '0.02178'}
{'loss': '1.054', 'grad_norm': '0.3992', 'learning_rate': '0.0001871', 'entropy': '1.001', 'num_tokens': '1.341e+07', 'mean_token_accuracy': '0.7474', 'epoch': '0.02179'}
{'loss': '0.7332', 'grad_norm': '0.2605', 'learning_rate': '0.0001871', 'entropy': '0.8902', 'num_tokens': '1.342e+07', 'mean_token_accuracy': '0.7865', 'epoch': '0.0218'}
{'loss': '0.8693', 'grad_norm': '0.4209', 'learning_rate': '0.0001871', 'entropy': '0.8715', 'num_tokens': '1.342e+07', 'mean_token_accuracy': '0.7843', 'epoch': '0.02181'}
{'loss': '0.7966', 'grad_norm': '0.3058', 'learning_rate': '0.0001871', 'entropy': '0.8691', 'num_tokens': '1.343e+07', 'mean_token_accuracy': '0.7824', 'epoch': '0.02181'}
{'loss': '1.144', 'grad_norm': '0.4965', 'learning_rate': '0.0001871', 'entropy': '1.138', 'num_tokens': '1.343e+07', 'mean_token_accuracy': '0.7375', 'epoch': '0.02182'}
{'loss': '1.205', 'grad_norm': '0.3233', 'learning_rate': '0.0001871', 'entropy': '1.129', 'num_tokens': '1.344e+07', 'mean_token_accuracy': '0.7121', 'epoch': '0.02183'}
{'loss': '0.682', 'grad_norm': '0.2776', 'learning_rate': '0.0001871', 'entropy': '0.6937', 'num_tokens': '1.345e+07', 'mean_token_accuracy': '0.8269', 'epoch': '0.02184'}
{'loss': '0.4808', 'grad_norm': '0.3002', 'learning_rate': '0.000187', 'entropy': '0.4848', 'num_tokens': '1.345e+07', 'mean_token_accuracy': '0.8724', 'epoch': '0.02185'}
{'loss': '0.7599', 'grad_norm': '0.3495', 'learning_rate': '0.000187', 'entropy': '0.7512', 'num_tokens': '1.346e+07', 'mean_token_accuracy': '0.8183', 'epoch': '0.02186'}
{'loss': '0.619', 'grad_norm': '0.2698', 'learning_rate': '0.000187', 'entropy': '0.7196', 'num_tokens': '1.346e+07', 'mean_token_accuracy': '0.8293', 'epoch': '0.02187'}
{'loss': '0.8554', 'grad_norm': '0.4274', 'learning_rate': '0.000187', 'entropy': '0.9005', 'num_tokens': '1.347e+07', 'mean_token_accuracy': '0.7711', 'epoch': '0.02188'}
{'loss': '0.8925', 'grad_norm': '0.4547', 'learning_rate': '0.000187', 'entropy': '0.9632', 'num_tokens': '1.347e+07', 'mean_token_accuracy': '0.7595', 'epoch': '0.02189'}
{'loss': '0.8813', 'grad_norm': '0.4183', 'learning_rate': '0.000187', 'entropy': '0.8621', 'num_tokens': '1.348e+07', 'mean_token_accuracy': '0.779', 'epoch': '0.02189'}
{'loss': '0.9231', 'grad_norm': '0.4152', 'learning_rate': '0.000187', 'entropy': '0.8589', 'num_tokens': '1.348e+07', 'mean_token_accuracy': '0.7815', 'epoch': '0.0219'}
{'loss': '0.8587', 'grad_norm': '0.3484', 'learning_rate': '0.0001869', 'entropy': '0.8558', 'num_tokens': '1.349e+07', 'mean_token_accuracy': '0.7788', 'epoch': '0.02191'}
{'loss': '0.8662', 'grad_norm': '0.4548', 'learning_rate': '0.0001869', 'entropy': '0.864', 'num_tokens': '1.349e+07', 'mean_token_accuracy': '0.7835', 'epoch': '0.02192'}
{'loss': '1.097', 'grad_norm': '0.3951', 'learning_rate': '0.0001869', 'entropy': '1.069', 'num_tokens': '1.35e+07', 'mean_token_accuracy': '0.726', 'epoch': '0.02193'}
{'loss': '1.376', 'grad_norm': '2.063', 'learning_rate': '0.0001869', 'entropy': '1.15', 'num_tokens': '1.351e+07', 'mean_token_accuracy': '0.7431', 'epoch': '0.02194'}
{'loss': '0.9222', 'grad_norm': '0.356', 'learning_rate': '0.0001869', 'entropy': '0.8888', 'num_tokens': '1.351e+07', 'mean_token_accuracy': '0.7856', 'epoch': '0.02195'}
{'loss': '0.9677', 'grad_norm': '0.3867', 'learning_rate': '0.0001869', 'entropy': '0.8561', 'num_tokens': '1.352e+07', 'mean_token_accuracy': '0.7798', 'epoch': '0.02196'}
{'loss': '1.133', 'grad_norm': '0.3589', 'learning_rate': '0.0001869', 'entropy': '1.067', 'num_tokens': '1.352e+07', 'mean_token_accuracy': '0.7446', 'epoch': '0.02197'}
{'loss': '1.032', 'grad_norm': '0.402', 'learning_rate': '0.0001868', 'entropy': '0.9956', 'num_tokens': '1.353e+07', 'mean_token_accuracy': '0.7626', 'epoch': '0.02197'}
{'loss': '0.6948', 'grad_norm': '0.376', 'learning_rate': '0.0001868', 'entropy': '0.801', 'num_tokens': '1.354e+07', 'mean_token_accuracy': '0.8026', 'epoch': '0.02198'}
{'loss': '0.8702', 'grad_norm': '0.3412', 'learning_rate': '0.0001868', 'entropy': '0.9223', 'num_tokens': '1.354e+07', 'mean_token_accuracy': '0.758', 'epoch': '0.02199'}
{'loss': '0.8647', 'grad_norm': '0.3589', 'learning_rate': '0.0001868', 'entropy': '0.848', 'num_tokens': '1.355e+07', 'mean_token_accuracy': '0.7912', 'epoch': '0.022'}
{'loss': '0.7688', 'grad_norm': '0.3599', 'learning_rate': '0.0001868', 'entropy': '0.8331', 'num_tokens': '1.355e+07', 'mean_token_accuracy': '0.7836', 'epoch': '0.02201'}
{'loss': '1.018', 'grad_norm': '0.4602', 'learning_rate': '0.0001868', 'entropy': '1.077', 'num_tokens': '1.356e+07', 'mean_token_accuracy': '0.7442', 'epoch': '0.02202'}
{'loss': '0.8395', 'grad_norm': '0.4501', 'learning_rate': '0.0001868', 'entropy': '0.8746', 'num_tokens': '1.356e+07', 'mean_token_accuracy': '0.78', 'epoch': '0.02203'}
{'loss': '0.7762', 'grad_norm': '0.374', 'learning_rate': '0.0001868', 'entropy': '0.7732', 'num_tokens': '1.357e+07', 'mean_token_accuracy': '0.8062', 'epoch': '0.02204'}
{'loss': '1.059', 'grad_norm': '0.4338', 'learning_rate': '0.0001867', 'entropy': '1.041', 'num_tokens': '1.357e+07', 'mean_token_accuracy': '0.7298', 'epoch': '0.02204'}
{'loss': '0.9844', 'grad_norm': '0.3785', 'learning_rate': '0.0001867', 'entropy': '0.9474', 'num_tokens': '1.358e+07', 'mean_token_accuracy': '0.7663', 'epoch': '0.02205'}
{'loss': '0.8638', 'grad_norm': '0.3646', 'learning_rate': '0.0001867', 'entropy': '0.8746', 'num_tokens': '1.359e+07', 'mean_token_accuracy': '0.7741', 'epoch': '0.02206'}
{'loss': '1.003', 'grad_norm': '0.3865', 'learning_rate': '0.0001867', 'entropy': '0.908', 'num_tokens': '1.359e+07', 'mean_token_accuracy': '0.7612', 'epoch': '0.02207'}

 21%|██        | 2495/12000 [1:30:17<4:53:12,  1.85s/it]
 21%|██        | 2496/12000 [1:30:20<5:17:56,  2.01s/it]
                                                        

 21%|██        | 2496/12000 [1:30:20<5:17:56,  2.01s/it]
 21%|██        | 2497/12000 [1:30:22<5:17:01,  2.00s/it]
                                                        

 21%|██        | 2497/12000 [1:30:22<5:17:01,  2.00s/it]
 21%|██        | 2498/12000 [1:30:23<5:01:39,  1.90s/it]
                                                        

 21%|██        | 2498/12000 [1:30:23<5:01:39,  1.90s/it]
 21%|██        | 2499/12000 [1:30:25<4:43:51,  1.79s/it]
                                                        

 21%|██        | 2499/12000 [1:30:25<4:43:51,  1.79s/it]
 21%|██        | 2500/12000 [1:30:27<4:41:31,  1.78s/it]
                                                        

 21%|██        | 2500/12000 [1:30:27<4:41:31,  1.78s/it]
 21%|██        | 2501/12000 [1:30:30<5:49:18,  2.21s/it]
                                                        

 21%|██        | 2501/12000 [1:30:30<5:49:18,  2.21s/it]
 21%|██        | 2502/12000 [1:30:33<6:42:38,  2.54s/it]
                                                        

 21%|██        | 2502/12000 [1:30:33<6:42:38,  2.54s/it]
 21%|██        | 2503/12000 [1:30:35<6:05:42,  2.31s/it]
                                                        

 21%|██        | 2503/12000 [1:30:35<6:05:42,  2.31s/it]
 21%|██        | 2504/12000 [1:30:38<6:53:02,  2.61s/it]
                                                        

 21%|██        | 2504/12000 [1:30:38<6:53:02,  2.61s/it]
 21%|██        | 2505/12000 [1:30:40<6:07:33,  2.32s/it]
                                                        

 21%|██        | 2505/12000 [1:30:40<6:07:33,  2.32s/it]
 21%|██        | 2506/12000 [1:30:41<5:27:05,  2.07s/it]
                                                        

 21%|██        | 2506/12000 [1:30:41<5:27:05,  2.07s/it]
 21%|██        | 2507/12000 [1:30:43<5:12:36,  1.98s/it]
                                                        

 21%|██        | 2507/12000 [1:30:43<5:12:36,  1.98s/it]
 21%|██        | 2508/12000 [1:30:45<4:47:57,  1.82s/it]
                                                        

 21%|██        | 2508/12000 [1:30:45<4:47:57,  1.82s/it]
 21%|██        | 2509/12000 [1:30:46<4:37:30,  1.75s/it]
                                                        

 21%|██        | 2509/12000 [1:30:46<4:37:30,  1.75s/it]
 21%|██        | 2510/12000 [1:30:48<4:32:50,  1.72s/it]
                                                        

 21%|██        | 2510/12000 [1:30:48<4:32:50,  1.72s/it]
 21%|██        | 2511/12000 [1:30:50<4:27:35,  1.69s/it]
                                                        

 21%|██        | 2511/12000 [1:30:50<4:27:35,  1.69s/it]
 21%|██        | 2512/12000 [1:30:51<4:20:42,  1.65s/it]
                                                        

 21%|██        | 2512/12000 [1:30:51<4:20:42,  1.65s/it]
 21%|██        | 2513/12000 [1:30:54<5:23:57,  2.05s/it]
                                                        

 21%|██        | 2513/12000 [1:30:54<5:23:57,  2.05s/it]
 21%|██        | 2514/12000 [1:30:56<5:11:00,  1.97s/it]
                                                        

 21%|██        | 2514/12000 [1:30:56<5:11:00,  1.97s/it]
 21%|██        | 2515/12000 [1:30:57<4:54:26,  1.86s/it]
                                                        

 21%|██        | 2515/12000 [1:30:57<4:54:26,  1.86s/it]
 21%|██        | 2516/12000 [1:30:59<4:46:26,  1.81s/it]
                                                        

 21%|██        | 2516/12000 [1:30:59<4:46:26,  1.81s/it]
 21%|██        | 2517/12000 [1:31:01<4:28:52,  1.70s/it]
                                                        

 21%|██        | 2517/12000 [1:31:01<4:28:52,  1.70s/it]
 21%|██        | 2518/12000 [1:31:03<4:52:08,  1.85s/it]
                                                        

 21%|██        | 2518/12000 [1:31:03<4:52:08,  1.85s/it]
 21%|██        | 2519/12000 [1:31:05<4:45:05,  1.80s/it]
                                                        

 21%|██        | 2519/12000 [1:31:05<4:45:05,  1.80s/it]
 21%|██        | 2520/12000 [1:31:06<4:48:04,  1.82s/it]
                                                        

 21%|██        | 2520/12000 [1:31:06<4:48:04,  1.82s/it]
 21%|██        | 2521/12000 [1:31:08<4:36:37,  1.75s/it]
                                                        

 21%|██        | 2521/12000 [1:31:08<4:36:37,  1.75s/it]
 21%|██        | 2522/12000 [1:31:09<4:22:21,  1.66s/it]
                                                        

 21%|██        | 2522/12000 [1:31:09<4:22:21,  1.66s/it]
 21%|██        | 2523/12000 [1:31:11<4:19:19,  1.64s/it]
                                                        

 21%|██        | 2523/12000 [1:31:11<4:19:19,  1.64s/it]
 21%|██        | 2524/12000 [1:31:13<4:49:32,  1.83s/it]
                                                        

 21%|██        | 2524/12000 [1:31:13<4:49:32,  1.83s/it]
 21%|██        | 2525/12000 [1:31:15<4:42:58,  1.79s/it]
                                                        

 21%|██        | 2525/12000 [1:31:15<4:42:58,  1.79s/it]
 21%|██        | 2526/12000 [1:31:19<6:11:50,  2.35s/it]
                                                        

 21%|██        | 2526/12000 [1:31:19<6:11:50,  2.35s/it]
 21%|██        | 2527/12000 [1:31:20<5:37:44,  2.14s/it]
                                                        

 21%|██        | 2527/12000 [1:31:20<5:37:44,  2.14s/it]
 21%|██        | 2528/12000 [1:31:22<5:02:08,  1.91s/it]
                                                        

 21%|██        | 2528/12000 [1:31:22<5:02:08,  1.91s/it]
 21%|██        | 2529/12000 [1:31:24<5:33:00,  2.11s/it]
                                                        

 21%|██        | 2529/12000 [1:31:24<5:33:00,  2.11s/it]
 21%|██        | 2530/12000 [1:31:26<5:18:03,  2.02s/it]
                                                        

 21%|██        | 2530/12000 [1:31:26<5:18:03,  2.02s/it]
 21%|██        | 2531/12000 [1:31:29<5:49:20,  2.21s/it]
                                                        

 21%|██        | 2531/12000 [1:31:29<5:49:20,  2.21s/it]
 21%|██        | 2532/12000 [1:31:32<6:20:59,  2.41s/it]
                                                        

 21%|██        | 2532/12000 [1:31:32<6:20:59,  2.41s/it]
 21%|██        | 2533/12000 [1:31:33<5:44:16,  2.18s/it]
                                                        

 21%|██        | 2533/12000 [1:31:33<5:44:16,  2.18s/it]
 21%|██        | 2534/12000 [1:31:36<6:01:58,  2.29s/it]
                                                        

 21%|██        | 2534/12000 [1:31:36<6:01:58,  2.29s/it]
 21%|██        | 2535/12000 [1:31:37<5:32:05,  2.11s/it]
                                                        

 21%|██        | 2535/12000 [1:31:37<5:32:05,  2.11s/it]
 21%|██        | 2536/12000 [1:31:39<5:02:25,  1.92s/it]
                                                        

 21%|██        | 2536/12000 [1:31:39<5:02:25,  1.92s/it]
 21%|██        | 2537/12000 [1:31:41<4:47:59,  1.83s/it]
                                                        

 21%|██        | 2537/12000 [1:31:41<4:47:59,  1.83s/it]
 21%|██        | 2538/12000 [1:31:42<4:27:09,  1.69s/it]
                                                        

 21%|██        | 2538/12000 [1:31:42<4:27:09,  1.69s/it]
 21%|██        | 2539/12000 [1:31:43<4:16:22,  1.63s/it]
                                                        

 21%|██        | 2539/12000 [1:31:43<4:16:22,  1.63s/it]
 21%|██        | 2540/12000 [1:31:45<4:17:24,  1.63s/it]
                                                        

 21%|██        | 2540/12000 [1:31:45<4:17:24,  1.63s/it]
 21%|██        | 2541/12000 [1:31:47<4:13:49,  1.61s/it]
                                                        

 21%|██        | 2541/12000 [1:31:47<4:13:49,  1.61s/it]
 21%|██        | 2542/12000 [1:31:48<4:17:34,  1.63s/it]
                                                        
{'loss': '1.171', 'grad_norm': '0.7256', 'learning_rate': '0.0001867', 'entropy': '1.186', 'num_tokens': '1.36e+07', 'mean_token_accuracy': '0.7075', 'epoch': '0.02208'}
{'loss': '1.081', 'grad_norm': '0.3616', 'learning_rate': '0.0001867', 'entropy': '1.118', 'num_tokens': '1.36e+07', 'mean_token_accuracy': '0.7202', 'epoch': '0.02209'}
{'loss': '1.028', 'grad_norm': '0.6048', 'learning_rate': '0.0001867', 'entropy': '0.9208', 'num_tokens': '1.361e+07', 'mean_token_accuracy': '0.7728', 'epoch': '0.0221'}
{'loss': '0.7605', 'grad_norm': '0.3208', 'learning_rate': '0.0001866', 'entropy': '0.8139', 'num_tokens': '1.361e+07', 'mean_token_accuracy': '0.8014', 'epoch': '0.02211'}
{'loss': '0.9893', 'grad_norm': '0.3794', 'learning_rate': '0.0001866', 'entropy': '1.055', 'num_tokens': '1.362e+07', 'mean_token_accuracy': '0.7451', 'epoch': '0.02212'}
{'loss': '1.212', 'grad_norm': '0.407', 'learning_rate': '0.0001866', 'entropy': '1.363', 'num_tokens': '1.362e+07', 'mean_token_accuracy': '0.6993', 'epoch': '0.02212'}
{'loss': '0.827', 'grad_norm': '0.3918', 'learning_rate': '0.0001866', 'entropy': '0.8718', 'num_tokens': '1.363e+07', 'mean_token_accuracy': '0.782', 'epoch': '0.02213'}
{'loss': '0.7568', 'grad_norm': '0.4215', 'learning_rate': '0.0001866', 'entropy': '0.9023', 'num_tokens': '1.364e+07', 'mean_token_accuracy': '0.7873', 'epoch': '0.02214'}
{'loss': '1.063', 'grad_norm': '0.3344', 'learning_rate': '0.0001866', 'entropy': '1.059', 'num_tokens': '1.364e+07', 'mean_token_accuracy': '0.7513', 'epoch': '0.02215'}
{'loss': '0.6142', 'grad_norm': '0.319', 'learning_rate': '0.0001866', 'entropy': '0.6967', 'num_tokens': '1.365e+07', 'mean_token_accuracy': '0.8362', 'epoch': '0.02216'}
{'loss': '0.8152', 'grad_norm': '0.3572', 'learning_rate': '0.0001865', 'entropy': '0.8103', 'num_tokens': '1.365e+07', 'mean_token_accuracy': '0.8095', 'epoch': '0.02217'}
{'loss': '0.711', 'grad_norm': '0.3634', 'learning_rate': '0.0001865', 'entropy': '0.7967', 'num_tokens': '1.366e+07', 'mean_token_accuracy': '0.8129', 'epoch': '0.02218'}
{'loss': '1.015', 'grad_norm': '0.4251', 'learning_rate': '0.0001865', 'entropy': '1.049', 'num_tokens': '1.366e+07', 'mean_token_accuracy': '0.7504', 'epoch': '0.02219'}
{'loss': '0.8676', 'grad_norm': '0.6001', 'learning_rate': '0.0001865', 'entropy': '0.9169', 'num_tokens': '1.366e+07', 'mean_token_accuracy': '0.7805', 'epoch': '0.0222'}
{'loss': '0.9269', 'grad_norm': '0.4042', 'learning_rate': '0.0001865', 'entropy': '0.8262', 'num_tokens': '1.367e+07', 'mean_token_accuracy': '0.795', 'epoch': '0.0222'}
{'loss': '0.7338', 'grad_norm': '0.4534', 'learning_rate': '0.0001865', 'entropy': '0.7856', 'num_tokens': '1.367e+07', 'mean_token_accuracy': '0.8005', 'epoch': '0.02221'}
{'loss': '0.8745', 'grad_norm': '0.4402', 'learning_rate': '0.0001865', 'entropy': '0.8568', 'num_tokens': '1.368e+07', 'mean_token_accuracy': '0.7785', 'epoch': '0.02222'}
{'loss': '0.8521', 'grad_norm': '0.431', 'learning_rate': '0.0001865', 'entropy': '0.7828', 'num_tokens': '1.368e+07', 'mean_token_accuracy': '0.7754', 'epoch': '0.02223'}
{'loss': '0.7616', 'grad_norm': '0.4818', 'learning_rate': '0.0001864', 'entropy': '0.733', 'num_tokens': '1.369e+07', 'mean_token_accuracy': '0.8149', 'epoch': '0.02224'}
{'loss': '1.17', 'grad_norm': '0.5513', 'learning_rate': '0.0001864', 'entropy': '1.034', 'num_tokens': '1.369e+07', 'mean_token_accuracy': '0.7264', 'epoch': '0.02225'}
{'loss': '0.8677', 'grad_norm': '0.5394', 'learning_rate': '0.0001864', 'entropy': '0.7947', 'num_tokens': '1.37e+07', 'mean_token_accuracy': '0.787', 'epoch': '0.02226'}
{'loss': '0.6992', 'grad_norm': '0.3546', 'learning_rate': '0.0001864', 'entropy': '0.6931', 'num_tokens': '1.37e+07', 'mean_token_accuracy': '0.8158', 'epoch': '0.02227'}
{'loss': '0.8314', 'grad_norm': '0.3595', 'learning_rate': '0.0001864', 'entropy': '0.8086', 'num_tokens': '1.371e+07', 'mean_token_accuracy': '0.8023', 'epoch': '0.02228'}
{'loss': '0.9761', 'grad_norm': '0.3855', 'learning_rate': '0.0001864', 'entropy': '1.018', 'num_tokens': '1.371e+07', 'mean_token_accuracy': '0.756', 'epoch': '0.02228'}
{'loss': '0.9111', 'grad_norm': '0.3686', 'learning_rate': '0.0001864', 'entropy': '0.9675', 'num_tokens': '1.372e+07', 'mean_token_accuracy': '0.776', 'epoch': '0.02229'}
{'loss': '1.15', 'grad_norm': '0.3587', 'learning_rate': '0.0001863', 'entropy': '1.196', 'num_tokens': '1.372e+07', 'mean_token_accuracy': '0.7252', 'epoch': '0.0223'}
{'loss': '0.7331', 'grad_norm': '0.3693', 'learning_rate': '0.0001863', 'entropy': '0.7657', 'num_tokens': '1.373e+07', 'mean_token_accuracy': '0.8181', 'epoch': '0.02231'}
{'loss': '0.8393', 'grad_norm': '0.3268', 'learning_rate': '0.0001863', 'entropy': '0.9071', 'num_tokens': '1.373e+07', 'mean_token_accuracy': '0.7786', 'epoch': '0.02232'}
{'loss': '1.018', 'grad_norm': '0.3729', 'learning_rate': '0.0001863', 'entropy': '0.9875', 'num_tokens': '1.374e+07', 'mean_token_accuracy': '0.7542', 'epoch': '0.02233'}
{'loss': '1.051', 'grad_norm': '0.4395', 'learning_rate': '0.0001863', 'entropy': '1.131', 'num_tokens': '1.374e+07', 'mean_token_accuracy': '0.7212', 'epoch': '0.02234'}
{'loss': '1.124', 'grad_norm': '0.4508', 'learning_rate': '0.0001863', 'entropy': '1.072', 'num_tokens': '1.375e+07', 'mean_token_accuracy': '0.7413', 'epoch': '0.02235'}
{'loss': '0.9036', 'grad_norm': '0.4199', 'learning_rate': '0.0001863', 'entropy': '0.9179', 'num_tokens': '1.375e+07', 'mean_token_accuracy': '0.777', 'epoch': '0.02235'}
{'loss': '1.037', 'grad_norm': '0.4359', 'learning_rate': '0.0001862', 'entropy': '1.068', 'num_tokens': '1.376e+07', 'mean_token_accuracy': '0.7472', 'epoch': '0.02236'}
{'loss': '0.7143', 'grad_norm': '0.4787', 'learning_rate': '0.0001862', 'entropy': '0.7834', 'num_tokens': '1.376e+07', 'mean_token_accuracy': '0.8102', 'epoch': '0.02237'}
{'loss': '1.005', 'grad_norm': '0.3975', 'learning_rate': '0.0001862', 'entropy': '0.9446', 'num_tokens': '1.377e+07', 'mean_token_accuracy': '0.7753', 'epoch': '0.02238'}
{'loss': '0.9777', 'grad_norm': '0.3729', 'learning_rate': '0.0001862', 'entropy': '0.9848', 'num_tokens': '1.377e+07', 'mean_token_accuracy': '0.7585', 'epoch': '0.02239'}
{'loss': '0.9057', 'grad_norm': '0.36', 'learning_rate': '0.0001862', 'entropy': '0.8403', 'num_tokens': '1.378e+07', 'mean_token_accuracy': '0.7837', 'epoch': '0.0224'}
{'loss': '0.9628', 'grad_norm': '0.3442', 'learning_rate': '0.0001862', 'entropy': '1.093', 'num_tokens': '1.379e+07', 'mean_token_accuracy': '0.7496', 'epoch': '0.02241'}
{'loss': '0.7571', 'grad_norm': '0.3861', 'learning_rate': '0.0001862', 'entropy': '0.7781', 'num_tokens': '1.379e+07', 'mean_token_accuracy': '0.8013', 'epoch': '0.02242'}
{'loss': '0.7498', 'grad_norm': '0.3384', 'learning_rate': '0.0001861', 'entropy': '0.6911', 'num_tokens': '1.38e+07', 'mean_token_accuracy': '0.8334', 'epoch': '0.02243'}
{'loss': '0.6195', 'grad_norm': '0.3718', 'learning_rate': '0.0001861', 'entropy': '0.6584', 'num_tokens': '1.38e+07', 'mean_token_accuracy': '0.8257', 'epoch': '0.02243'}
{'loss': '0.7374', 'grad_norm': '0.3285', 'learning_rate': '0.0001861', 'entropy': '0.7497', 'num_tokens': '1.381e+07', 'mean_token_accuracy': '0.8061', 'epoch': '0.02244'}
{'loss': '1.049', 'grad_norm': '0.5277', 'learning_rate': '0.0001861', 'entropy': '1.075', 'num_tokens': '1.381e+07', 'mean_token_accuracy': '0.7364', 'epoch': '0.02245'}
{'loss': '0.8995', 'grad_norm': '0.4397', 'learning_rate': '0.0001861', 'entropy': '0.9201', 'num_tokens': '1.381e+07', 'mean_token_accuracy': '0.7663', 'epoch': '0.02246'}
{'loss': '0.831', 'grad_norm': '0.4013', 'learning_rate': '0.0001861', 'entropy': '0.8403', 'num_tokens': '1.382e+07', 'mean_token_accuracy': '0.7884', 'epoch': '0.02247'}
{'loss': '0.5953', 'grad_norm': '0.2827', 'learning_rate': '0.0001861', 'entropy': '0.6002', 'num_tokens': '1.382e+07', 'mean_token_accuracy': '0.834', 'epoch': '0.02248'}
{'loss': '0.7483', 'grad_norm': '0.4099', 'learning_rate': '0.000186', 'entropy': '0.7687', 'num_tokens': '1.383e+07', 'mean_token_accuracy': '0.7977', 'epoch': '0.02249'}

 21%|██        | 2542/12000 [1:31:48<4:17:34,  1.63s/it]
 21%|██        | 2543/12000 [1:31:50<4:06:27,  1.56s/it]
                                                        

 21%|██        | 2543/12000 [1:31:50<4:06:27,  1.56s/it]
 21%|██        | 2544/12000 [1:31:51<4:12:21,  1.60s/it]
                                                        

 21%|██        | 2544/12000 [1:31:51<4:12:21,  1.60s/it]
 21%|██        | 2545/12000 [1:31:53<4:17:41,  1.64s/it]
                                                        

 21%|██        | 2545/12000 [1:31:53<4:17:41,  1.64s/it]
 21%|██        | 2546/12000 [1:31:55<4:10:45,  1.59s/it]
                                                        

 21%|██        | 2546/12000 [1:31:55<4:10:45,  1.59s/it]
 21%|██        | 2547/12000 [1:31:57<4:39:20,  1.77s/it]
                                                        

 21%|██        | 2547/12000 [1:31:57<4:39:20,  1.77s/it]
 21%|██        | 2548/12000 [1:32:00<5:58:36,  2.28s/it]
                                                        

 21%|██        | 2548/12000 [1:32:00<5:58:36,  2.28s/it]
 21%|██        | 2549/12000 [1:32:02<5:17:04,  2.01s/it]
                                                        

 21%|██        | 2549/12000 [1:32:02<5:17:04,  2.01s/it]
 21%|██▏       | 2550/12000 [1:32:03<5:02:03,  1.92s/it]
                                                        

 21%|██▏       | 2550/12000 [1:32:03<5:02:03,  1.92s/it]
 21%|██▏       | 2551/12000 [1:32:07<6:18:18,  2.40s/it]
                                                        

 21%|██▏       | 2551/12000 [1:32:07<6:18:18,  2.40s/it]
 21%|██▏       | 2552/12000 [1:32:09<6:12:24,  2.36s/it]
                                                        

 21%|██▏       | 2552/12000 [1:32:09<6:12:24,  2.36s/it]
 21%|██▏       | 2553/12000 [1:32:11<5:34:00,  2.12s/it]
                                                        

 21%|██▏       | 2553/12000 [1:32:11<5:34:00,  2.12s/it]
 21%|██▏       | 2554/12000 [1:32:13<5:51:58,  2.24s/it]
                                                        

 21%|██▏       | 2554/12000 [1:32:13<5:51:58,  2.24s/it]
 21%|██▏       | 2555/12000 [1:32:16<6:00:49,  2.29s/it]
                                                        

 21%|██▏       | 2555/12000 [1:32:16<6:00:49,  2.29s/it]
 21%|██▏       | 2556/12000 [1:32:17<5:18:26,  2.02s/it]
                                                        

 21%|██▏       | 2556/12000 [1:32:17<5:18:26,  2.02s/it]
 21%|██▏       | 2557/12000 [1:32:19<5:28:04,  2.08s/it]
                                                        

 21%|██▏       | 2557/12000 [1:32:19<5:28:04,  2.08s/it]
 21%|██▏       | 2558/12000 [1:32:21<4:59:40,  1.90s/it]
                                                        

 21%|██▏       | 2558/12000 [1:32:21<4:59:40,  1.90s/it]
 21%|██▏       | 2559/12000 [1:32:22<4:39:21,  1.78s/it]
                                                        

 21%|██▏       | 2559/12000 [1:32:22<4:39:21,  1.78s/it]
 21%|██▏       | 2560/12000 [1:32:24<4:30:37,  1.72s/it]
                                                        

 21%|██▏       | 2560/12000 [1:32:24<4:30:37,  1.72s/it]
 21%|██▏       | 2561/12000 [1:32:25<4:28:47,  1.71s/it]
                                                        

 21%|██▏       | 2561/12000 [1:32:25<4:28:47,  1.71s/it]
 21%|██▏       | 2562/12000 [1:32:29<5:33:43,  2.12s/it]
                                                        

 21%|██▏       | 2562/12000 [1:32:29<5:33:43,  2.12s/it]
 21%|██▏       | 2563/12000 [1:32:30<5:17:41,  2.02s/it]
                                                        

 21%|██▏       | 2563/12000 [1:32:30<5:17:41,  2.02s/it]
 21%|██▏       | 2564/12000 [1:32:33<5:39:09,  2.16s/it]
                                                        

 21%|██▏       | 2564/12000 [1:32:33<5:39:09,  2.16s/it]
 21%|██▏       | 2565/12000 [1:32:35<5:53:50,  2.25s/it]
                                                        

 21%|██▏       | 2565/12000 [1:32:35<5:53:50,  2.25s/it]
 21%|██▏       | 2566/12000 [1:32:38<6:05:21,  2.32s/it]
                                                        

 21%|██▏       | 2566/12000 [1:32:38<6:05:21,  2.32s/it]
 21%|██▏       | 2567/12000 [1:32:40<6:19:46,  2.42s/it]
                                                        

 21%|██▏       | 2567/12000 [1:32:40<6:19:46,  2.42s/it]
 21%|██▏       | 2568/12000 [1:32:43<6:35:16,  2.51s/it]
                                                        

 21%|██▏       | 2568/12000 [1:32:43<6:35:16,  2.51s/it]
 21%|██▏       | 2569/12000 [1:32:45<6:08:32,  2.34s/it]
                                                        

 21%|██▏       | 2569/12000 [1:32:45<6:08:32,  2.34s/it]
 21%|██▏       | 2570/12000 [1:32:48<6:54:12,  2.64s/it]
                                                        

 21%|██▏       | 2570/12000 [1:32:48<6:54:12,  2.64s/it]
 21%|██▏       | 2571/12000 [1:32:50<6:12:25,  2.37s/it]
                                                        

 21%|██▏       | 2571/12000 [1:32:50<6:12:25,  2.37s/it]
 21%|██▏       | 2572/12000 [1:32:53<6:30:59,  2.49s/it]
                                                        

 21%|██▏       | 2572/12000 [1:32:53<6:30:59,  2.49s/it]
 21%|██▏       | 2573/12000 [1:32:55<6:24:34,  2.45s/it]
                                                        

 21%|██▏       | 2573/12000 [1:32:55<6:24:34,  2.45s/it]
 21%|██▏       | 2574/12000 [1:32:57<5:55:17,  2.26s/it]
                                                        

 21%|██▏       | 2574/12000 [1:32:57<5:55:17,  2.26s/it]
 21%|██▏       | 2575/12000 [1:32:59<5:28:20,  2.09s/it]
                                                        

 21%|██▏       | 2575/12000 [1:32:59<5:28:20,  2.09s/it]
 21%|██▏       | 2576/12000 [1:33:01<5:10:05,  1.97s/it]
                                                        

 21%|██▏       | 2576/12000 [1:33:01<5:10:05,  1.97s/it]
 21%|██▏       | 2577/12000 [1:33:03<5:26:07,  2.08s/it]
                                                        

 21%|██▏       | 2577/12000 [1:33:03<5:26:07,  2.08s/it]
 21%|██▏       | 2578/12000 [1:33:09<8:27:02,  3.23s/it]
                                                        

 21%|██▏       | 2578/12000 [1:33:09<8:27:02,  3.23s/it]
 21%|██▏       | 2579/12000 [1:33:11<7:23:37,  2.83s/it]
                                                        

 21%|██▏       | 2579/12000 [1:33:11<7:23:37,  2.83s/it]
 22%|██▏       | 2580/12000 [1:33:13<6:50:10,  2.61s/it]
                                                        

 22%|██▏       | 2580/12000 [1:33:13<6:50:10,  2.61s/it]
 22%|██▏       | 2581/12000 [1:33:14<6:04:46,  2.32s/it]
                                                        

 22%|██▏       | 2581/12000 [1:33:14<6:04:46,  2.32s/it]
 22%|██▏       | 2582/12000 [1:33:16<5:23:17,  2.06s/it]
                                                        

 22%|██▏       | 2582/12000 [1:33:16<5:23:17,  2.06s/it]
 22%|██▏       | 2583/12000 [1:33:17<4:52:02,  1.86s/it]
                                                        

 22%|██▏       | 2583/12000 [1:33:17<4:52:02,  1.86s/it]
 22%|██▏       | 2584/12000 [1:33:19<4:52:45,  1.87s/it]
                                                        

 22%|██▏       | 2584/12000 [1:33:19<4:52:45,  1.87s/it]
 22%|██▏       | 2585/12000 [1:33:20<4:29:14,  1.72s/it]
                                                        

 22%|██▏       | 2585/12000 [1:33:20<4:29:14,  1.72s/it]
 22%|██▏       | 2586/12000 [1:33:22<4:10:56,  1.60s/it]
                                                        

 22%|██▏       | 2586/12000 [1:33:22<4:10:56,  1.60s/it]
 22%|██▏       | 2587/12000 [1:33:23<4:06:20,  1.57s/it]
                                                        

 22%|██▏       | 2587/12000 [1:33:23<4:06:20,  1.57s/it]
 22%|██▏       | 2588/12000 [1:33:25<3:55:42,  1.50s/it]
                                                        

 22%|██▏       | 2588/12000 [1:33:25<3:55:42,  1.50s/it]
 22%|██▏       | 2589/12000 [1:33:27<4:23:51,  1.68s/it]
                                                        
{'loss': '1.052', 'grad_norm': '0.4808', 'learning_rate': '0.000186', 'entropy': '1.021', 'num_tokens': '1.383e+07', 'mean_token_accuracy': '0.7458', 'epoch': '0.0225'}
{'loss': '0.7265', 'grad_norm': '0.3464', 'learning_rate': '0.000186', 'entropy': '0.6819', 'num_tokens': '1.384e+07', 'mean_token_accuracy': '0.8185', 'epoch': '0.02251'}
{'loss': '0.7958', 'grad_norm': '0.3287', 'learning_rate': '0.000186', 'entropy': '0.8444', 'num_tokens': '1.384e+07', 'mean_token_accuracy': '0.7786', 'epoch': '0.02251'}
{'loss': '0.9103', 'grad_norm': '0.3788', 'learning_rate': '0.000186', 'entropy': '0.9085', 'num_tokens': '1.385e+07', 'mean_token_accuracy': '0.7628', 'epoch': '0.02252'}
{'loss': '0.9728', 'grad_norm': '0.4273', 'learning_rate': '0.000186', 'entropy': '1.052', 'num_tokens': '1.385e+07', 'mean_token_accuracy': '0.7643', 'epoch': '0.02253'}
{'loss': '0.7047', 'grad_norm': '0.365', 'learning_rate': '0.000186', 'entropy': '0.7594', 'num_tokens': '1.386e+07', 'mean_token_accuracy': '0.8093', 'epoch': '0.02254'}
{'loss': '0.7713', 'grad_norm': '0.4332', 'learning_rate': '0.0001859', 'entropy': '0.8739', 'num_tokens': '1.386e+07', 'mean_token_accuracy': '0.7848', 'epoch': '0.02255'}
{'loss': '0.9312', 'grad_norm': '0.3697', 'learning_rate': '0.0001859', 'entropy': '0.9503', 'num_tokens': '1.387e+07', 'mean_token_accuracy': '0.7615', 'epoch': '0.02256'}
{'loss': '0.707', 'grad_norm': '0.3626', 'learning_rate': '0.0001859', 'entropy': '0.7969', 'num_tokens': '1.387e+07', 'mean_token_accuracy': '0.8102', 'epoch': '0.02257'}
{'loss': '1.146', 'grad_norm': '0.4197', 'learning_rate': '0.0001859', 'entropy': '1.058', 'num_tokens': '1.388e+07', 'mean_token_accuracy': '0.7482', 'epoch': '0.02258'}
{'loss': '0.8556', 'grad_norm': '0.3468', 'learning_rate': '0.0001859', 'entropy': '0.9774', 'num_tokens': '1.388e+07', 'mean_token_accuracy': '0.7583', 'epoch': '0.02258'}
{'loss': '1.061', 'grad_norm': '0.413', 'learning_rate': '0.0001859', 'entropy': '1.054', 'num_tokens': '1.389e+07', 'mean_token_accuracy': '0.7473', 'epoch': '0.02259'}
{'loss': '0.744', 'grad_norm': '0.3643', 'learning_rate': '0.0001859', 'entropy': '0.7929', 'num_tokens': '1.389e+07', 'mean_token_accuracy': '0.8099', 'epoch': '0.0226'}
{'loss': '0.874', 'grad_norm': '0.5109', 'learning_rate': '0.0001858', 'entropy': '0.9161', 'num_tokens': '1.39e+07', 'mean_token_accuracy': '0.775', 'epoch': '0.02261'}
{'loss': '0.6506', 'grad_norm': '0.3403', 'learning_rate': '0.0001858', 'entropy': '0.6649', 'num_tokens': '1.39e+07', 'mean_token_accuracy': '0.8324', 'epoch': '0.02262'}
{'loss': '0.8823', 'grad_norm': '0.3808', 'learning_rate': '0.0001858', 'entropy': '0.8914', 'num_tokens': '1.391e+07', 'mean_token_accuracy': '0.7706', 'epoch': '0.02263'}
{'loss': '0.7768', 'grad_norm': '0.3862', 'learning_rate': '0.0001858', 'entropy': '0.7662', 'num_tokens': '1.391e+07', 'mean_token_accuracy': '0.7879', 'epoch': '0.02264'}
{'loss': '0.9985', 'grad_norm': '0.4398', 'learning_rate': '0.0001858', 'entropy': '0.9684', 'num_tokens': '1.392e+07', 'mean_token_accuracy': '0.7638', 'epoch': '0.02265'}
{'loss': '0.9357', 'grad_norm': '0.3782', 'learning_rate': '0.0001858', 'entropy': '0.944', 'num_tokens': '1.392e+07', 'mean_token_accuracy': '0.7589', 'epoch': '0.02266'}
{'loss': '1.047', 'grad_norm': '0.4526', 'learning_rate': '0.0001858', 'entropy': '1.003', 'num_tokens': '1.393e+07', 'mean_token_accuracy': '0.7488', 'epoch': '0.02266'}
{'loss': '1.443', 'grad_norm': '0.4051', 'learning_rate': '0.0001857', 'entropy': '1.305', 'num_tokens': '1.393e+07', 'mean_token_accuracy': '0.6937', 'epoch': '0.02267'}
{'loss': '1.312', 'grad_norm': '0.4257', 'learning_rate': '0.0001857', 'entropy': '1.322', 'num_tokens': '1.394e+07', 'mean_token_accuracy': '0.6901', 'epoch': '0.02268'}
{'loss': '0.5895', 'grad_norm': '0.2725', 'learning_rate': '0.0001857', 'entropy': '0.7053', 'num_tokens': '1.394e+07', 'mean_token_accuracy': '0.8237', 'epoch': '0.02269'}
{'loss': '0.9707', 'grad_norm': '0.4045', 'learning_rate': '0.0001857', 'entropy': '1.056', 'num_tokens': '1.395e+07', 'mean_token_accuracy': '0.7494', 'epoch': '0.0227'}
{'loss': '0.8323', 'grad_norm': '0.7587', 'learning_rate': '0.0001857', 'entropy': '0.9174', 'num_tokens': '1.396e+07', 'mean_token_accuracy': '0.7778', 'epoch': '0.02271'}
{'loss': '0.9179', 'grad_norm': '0.3689', 'learning_rate': '0.0001857', 'entropy': '0.9213', 'num_tokens': '1.396e+07', 'mean_token_accuracy': '0.7857', 'epoch': '0.02272'}
{'loss': '0.8595', 'grad_norm': '0.391', 'learning_rate': '0.0001857', 'entropy': '0.9154', 'num_tokens': '1.397e+07', 'mean_token_accuracy': '0.7788', 'epoch': '0.02273'}
{'loss': '0.8659', 'grad_norm': '0.3531', 'learning_rate': '0.0001857', 'entropy': '0.8551', 'num_tokens': '1.398e+07', 'mean_token_accuracy': '0.7799', 'epoch': '0.02274'}
{'loss': '0.9256', 'grad_norm': '0.367', 'learning_rate': '0.0001856', 'entropy': '0.9795', 'num_tokens': '1.398e+07', 'mean_token_accuracy': '0.7596', 'epoch': '0.02274'}
{'loss': '0.8609', 'grad_norm': '0.301', 'learning_rate': '0.0001856', 'entropy': '0.7659', 'num_tokens': '1.399e+07', 'mean_token_accuracy': '0.8079', 'epoch': '0.02275'}
{'loss': '0.9375', 'grad_norm': '0.3957', 'learning_rate': '0.0001856', 'entropy': '1.03', 'num_tokens': '1.399e+07', 'mean_token_accuracy': '0.756', 'epoch': '0.02276'}
{'loss': '0.9748', 'grad_norm': '0.4086', 'learning_rate': '0.0001856', 'entropy': '0.9641', 'num_tokens': '1.4e+07', 'mean_token_accuracy': '0.7699', 'epoch': '0.02277'}
{'loss': '0.8101', 'grad_norm': '0.2914', 'learning_rate': '0.0001856', 'entropy': '0.8', 'num_tokens': '1.4e+07', 'mean_token_accuracy': '0.8024', 'epoch': '0.02278'}
{'loss': '0.8398', 'grad_norm': '0.3748', 'learning_rate': '0.0001856', 'entropy': '0.9775', 'num_tokens': '1.401e+07', 'mean_token_accuracy': '0.7636', 'epoch': '0.02279'}
{'loss': '0.7736', 'grad_norm': '0.4574', 'learning_rate': '0.0001856', 'entropy': '0.7968', 'num_tokens': '1.401e+07', 'mean_token_accuracy': '0.7959', 'epoch': '0.0228'}
{'loss': '1.238', 'grad_norm': '0.4444', 'learning_rate': '0.0001855', 'entropy': '1.327', 'num_tokens': '1.402e+07', 'mean_token_accuracy': '0.7018', 'epoch': '0.02281'}
{'loss': '1.198', 'grad_norm': '0.4093', 'learning_rate': '0.0001855', 'entropy': '1.115', 'num_tokens': '1.403e+07', 'mean_token_accuracy': '0.7442', 'epoch': '0.02281'}
{'loss': '1.224', 'grad_norm': '0.4878', 'learning_rate': '0.0001855', 'entropy': '1.233', 'num_tokens': '1.404e+07', 'mean_token_accuracy': '0.7142', 'epoch': '0.02282'}
{'loss': '0.8532', 'grad_norm': '0.3105', 'learning_rate': '0.0001855', 'entropy': '0.8949', 'num_tokens': '1.404e+07', 'mean_token_accuracy': '0.7732', 'epoch': '0.02283'}
{'loss': '0.7722', 'grad_norm': '0.2936', 'learning_rate': '0.0001855', 'entropy': '0.7327', 'num_tokens': '1.405e+07', 'mean_token_accuracy': '0.8181', 'epoch': '0.02284'}
{'loss': '0.6362', 'grad_norm': '0.2979', 'learning_rate': '0.0001855', 'entropy': '0.648', 'num_tokens': '1.405e+07', 'mean_token_accuracy': '0.8391', 'epoch': '0.02285'}
{'loss': '0.865', 'grad_norm': '0.3839', 'learning_rate': '0.0001855', 'entropy': '0.9124', 'num_tokens': '1.406e+07', 'mean_token_accuracy': '0.7737', 'epoch': '0.02286'}
{'loss': '1.127', 'grad_norm': '0.37', 'learning_rate': '0.0001854', 'entropy': '1.103', 'num_tokens': '1.406e+07', 'mean_token_accuracy': '0.7269', 'epoch': '0.02287'}
{'loss': '0.6121', 'grad_norm': '0.2996', 'learning_rate': '0.0001854', 'entropy': '0.6823', 'num_tokens': '1.407e+07', 'mean_token_accuracy': '0.8199', 'epoch': '0.02288'}
{'loss': '0.9811', 'grad_norm': '0.416', 'learning_rate': '0.0001854', 'entropy': '1.053', 'num_tokens': '1.407e+07', 'mean_token_accuracy': '0.7476', 'epoch': '0.02289'}
{'loss': '0.9512', 'grad_norm': '0.3536', 'learning_rate': '0.0001854', 'entropy': '1.018', 'num_tokens': '1.408e+07', 'mean_token_accuracy': '0.7611', 'epoch': '0.02289'}
{'loss': '1.067', 'grad_norm': '0.4065', 'learning_rate': '0.0001854', 'entropy': '1.105', 'num_tokens': '1.408e+07', 'mean_token_accuracy': '0.7405', 'epoch': '0.0229'}

 22%|██▏       | 2589/12000 [1:33:27<4:23:51,  1.68s/it]
 22%|██▏       | 2590/12000 [1:33:28<4:21:03,  1.66s/it]
                                                        

 22%|██▏       | 2590/12000 [1:33:28<4:21:03,  1.66s/it]
 22%|██▏       | 2591/12000 [1:33:31<4:56:39,  1.89s/it]
                                                        

 22%|██▏       | 2591/12000 [1:33:31<4:56:39,  1.89s/it]
 22%|██▏       | 2592/12000 [1:33:35<7:00:16,  2.68s/it]
                                                        

 22%|██▏       | 2592/12000 [1:33:35<7:00:16,  2.68s/it]
 22%|██▏       | 2593/12000 [1:33:37<6:08:22,  2.35s/it]
                                                        

 22%|██▏       | 2593/12000 [1:33:37<6:08:22,  2.35s/it]
 22%|██▏       | 2594/12000 [1:33:39<5:47:40,  2.22s/it]
                                                        

 22%|██▏       | 2594/12000 [1:33:39<5:47:40,  2.22s/it]
 22%|██▏       | 2595/12000 [1:33:40<5:16:04,  2.02s/it]
                                                        

 22%|██▏       | 2595/12000 [1:33:40<5:16:04,  2.02s/it]
 22%|██▏       | 2596/12000 [1:33:42<4:53:18,  1.87s/it]
                                                        

 22%|██▏       | 2596/12000 [1:33:42<4:53:18,  1.87s/it]
 22%|██▏       | 2597/12000 [1:33:43<4:30:49,  1.73s/it]
                                                        

 22%|██▏       | 2597/12000 [1:33:43<4:30:49,  1.73s/it]
 22%|██▏       | 2598/12000 [1:33:45<4:14:17,  1.62s/it]
                                                        

 22%|██▏       | 2598/12000 [1:33:45<4:14:17,  1.62s/it]
 22%|██▏       | 2599/12000 [1:33:46<4:12:12,  1.61s/it]
                                                        

 22%|██▏       | 2599/12000 [1:33:46<4:12:12,  1.61s/it]
 22%|██▏       | 2600/12000 [1:33:49<5:04:38,  1.94s/it]
                                                        

 22%|██▏       | 2600/12000 [1:33:49<5:04:38,  1.94s/it]
 22%|██▏       | 2601/12000 [1:33:54<7:20:11,  2.81s/it]
                                                        

 22%|██▏       | 2601/12000 [1:33:54<7:20:11,  2.81s/it]
 22%|██▏       | 2602/12000 [1:33:58<8:44:30,  3.35s/it]
                                                        

 22%|██▏       | 2602/12000 [1:33:58<8:44:30,  3.35s/it]
 22%|██▏       | 2603/12000 [1:34:00<7:08:56,  2.74s/it]
                                                        

 22%|██▏       | 2603/12000 [1:34:00<7:08:56,  2.74s/it]
 22%|██▏       | 2604/12000 [1:34:01<6:13:11,  2.38s/it]
                                                        

 22%|██▏       | 2604/12000 [1:34:01<6:13:11,  2.38s/it]
 22%|██▏       | 2605/12000 [1:34:03<6:03:13,  2.32s/it]
                                                        

 22%|██▏       | 2605/12000 [1:34:03<6:03:13,  2.32s/it]
 22%|██▏       | 2606/12000 [1:34:05<5:19:01,  2.04s/it]
                                                        

 22%|██▏       | 2606/12000 [1:34:05<5:19:01,  2.04s/it]
 22%|██▏       | 2607/12000 [1:34:06<4:52:31,  1.87s/it]
                                                        

 22%|██▏       | 2607/12000 [1:34:06<4:52:31,  1.87s/it]
 22%|██▏       | 2608/12000 [1:34:09<5:17:37,  2.03s/it]
                                                        

 22%|██▏       | 2608/12000 [1:34:09<5:17:37,  2.03s/it]
 22%|██▏       | 2609/12000 [1:34:11<5:13:17,  2.00s/it]
                                                        

 22%|██▏       | 2609/12000 [1:34:11<5:13:17,  2.00s/it]
 22%|██▏       | 2610/12000 [1:34:12<5:03:59,  1.94s/it]
                                                        

 22%|██▏       | 2610/12000 [1:34:12<5:03:59,  1.94s/it]
 22%|██▏       | 2611/12000 [1:34:14<4:43:57,  1.81s/it]
                                                        

 22%|██▏       | 2611/12000 [1:34:14<4:43:57,  1.81s/it]
 22%|██▏       | 2612/12000 [1:34:17<6:05:10,  2.33s/it]
                                                        

 22%|██▏       | 2612/12000 [1:34:17<6:05:10,  2.33s/it]
 22%|██▏       | 2613/12000 [1:34:19<5:27:27,  2.09s/it]
                                                        

 22%|██▏       | 2613/12000 [1:34:19<5:27:27,  2.09s/it]
 22%|██▏       | 2614/12000 [1:34:24<7:44:00,  2.97s/it]
                                                        

 22%|██▏       | 2614/12000 [1:34:24<7:44:00,  2.97s/it]
 22%|██▏       | 2615/12000 [1:34:26<6:49:41,  2.62s/it]
                                                        

 22%|██▏       | 2615/12000 [1:34:26<6:49:41,  2.62s/it]
 22%|██▏       | 2616/12000 [1:34:29<6:56:36,  2.66s/it]
                                                        

 22%|██▏       | 2616/12000 [1:34:29<6:56:36,  2.66s/it]
 22%|██▏       | 2617/12000 [1:34:31<6:21:58,  2.44s/it]
                                                        

 22%|██▏       | 2617/12000 [1:34:31<6:21:58,  2.44s/it]
 22%|██▏       | 2618/12000 [1:34:33<6:12:11,  2.38s/it]
                                                        

 22%|██▏       | 2618/12000 [1:34:33<6:12:11,  2.38s/it]
 22%|██▏       | 2619/12000 [1:34:34<5:32:34,  2.13s/it]
                                                        

 22%|██▏       | 2619/12000 [1:34:34<5:32:34,  2.13s/it]
 22%|██▏       | 2620/12000 [1:34:36<4:52:41,  1.87s/it]
                                                        

 22%|██▏       | 2620/12000 [1:34:36<4:52:41,  1.87s/it]
 22%|██▏       | 2621/12000 [1:34:38<5:14:48,  2.01s/it]
                                                        

 22%|██▏       | 2621/12000 [1:34:38<5:14:48,  2.01s/it]
 22%|██▏       | 2622/12000 [1:34:39<4:40:21,  1.79s/it]
                                                        

 22%|██▏       | 2622/12000 [1:34:39<4:40:21,  1.79s/it]
 22%|██▏       | 2623/12000 [1:34:41<4:20:24,  1.67s/it]
                                                        

 22%|██▏       | 2623/12000 [1:34:41<4:20:24,  1.67s/it]
 22%|██▏       | 2624/12000 [1:34:42<4:32:17,  1.74s/it]
                                                        

 22%|██▏       | 2624/12000 [1:34:42<4:32:17,  1.74s/it]
 22%|██▏       | 2625/12000 [1:34:44<4:30:29,  1.73s/it]
                                                        

 22%|██▏       | 2625/12000 [1:34:44<4:30:29,  1.73s/it]
 22%|██▏       | 2626/12000 [1:34:46<4:34:36,  1.76s/it]
                                                        

 22%|██▏       | 2626/12000 [1:34:46<4:34:36,  1.76s/it]
 22%|██▏       | 2627/12000 [1:34:48<4:28:12,  1.72s/it]
                                                        

 22%|██▏       | 2627/12000 [1:34:48<4:28:12,  1.72s/it]
 22%|██▏       | 2628/12000 [1:34:50<4:35:26,  1.76s/it]
                                                        

 22%|██▏       | 2628/12000 [1:34:50<4:35:26,  1.76s/it]
 22%|██▏       | 2629/12000 [1:34:51<4:23:36,  1.69s/it]
                                                        

 22%|██▏       | 2629/12000 [1:34:51<4:23:36,  1.69s/it]
 22%|██▏       | 2630/12000 [1:34:53<4:36:51,  1.77s/it]
                                                        

 22%|██▏       | 2630/12000 [1:34:53<4:36:51,  1.77s/it]
 22%|██▏       | 2631/12000 [1:34:57<6:27:48,  2.48s/it]
                                                        

 22%|██▏       | 2631/12000 [1:34:57<6:27:48,  2.48s/it]
 22%|██▏       | 2632/12000 [1:34:59<6:10:41,  2.37s/it]
                                                        

 22%|██▏       | 2632/12000 [1:34:59<6:10:41,  2.37s/it]
 22%|██▏       | 2633/12000 [1:35:01<5:37:46,  2.16s/it]
                                                        

 22%|██▏       | 2633/12000 [1:35:01<5:37:46,  2.16s/it]
 22%|██▏       | 2634/12000 [1:35:02<5:07:07,  1.97s/it]
                                                        

 22%|██▏       | 2634/12000 [1:35:02<5:07:07,  1.97s/it]
 22%|██▏       | 2635/12000 [1:35:04<4:59:17,  1.92s/it]
                                                        

 22%|██▏       | 2635/12000 [1:35:04<4:59:17,  1.92s/it]
 22%|██▏       | 2636/12000 [1:35:08<6:37:25,  2.55s/it]
                                                        
{'loss': '0.8319', 'grad_norm': '0.3982', 'learning_rate': '0.0001854', 'entropy': '0.7915', 'num_tokens': '1.409e+07', 'mean_token_accuracy': '0.7923', 'epoch': '0.02291'}
{'loss': '0.642', 'grad_norm': '0.3685', 'learning_rate': '0.0001854', 'entropy': '0.6788', 'num_tokens': '1.409e+07', 'mean_token_accuracy': '0.8252', 'epoch': '0.02292'}
{'loss': '0.7375', 'grad_norm': '0.2597', 'learning_rate': '0.0001853', 'entropy': '0.7733', 'num_tokens': '1.41e+07', 'mean_token_accuracy': '0.8068', 'epoch': '0.02293'}
{'loss': '1.007', 'grad_norm': '0.3329', 'learning_rate': '0.0001853', 'entropy': '1.02', 'num_tokens': '1.411e+07', 'mean_token_accuracy': '0.7568', 'epoch': '0.02294'}
{'loss': '0.9316', 'grad_norm': '0.3677', 'learning_rate': '0.0001853', 'entropy': '0.9617', 'num_tokens': '1.411e+07', 'mean_token_accuracy': '0.7639', 'epoch': '0.02295'}
{'loss': '1.113', 'grad_norm': '0.3581', 'learning_rate': '0.0001853', 'entropy': '1.032', 'num_tokens': '1.412e+07', 'mean_token_accuracy': '0.7362', 'epoch': '0.02296'}
{'loss': '1.137', 'grad_norm': '0.4106', 'learning_rate': '0.0001853', 'entropy': '1.13', 'num_tokens': '1.412e+07', 'mean_token_accuracy': '0.7309', 'epoch': '0.02297'}
{'loss': '0.8662', 'grad_norm': '0.3582', 'learning_rate': '0.0001853', 'entropy': '0.9039', 'num_tokens': '1.413e+07', 'mean_token_accuracy': '0.7911', 'epoch': '0.02297'}
{'loss': '0.8609', 'grad_norm': '0.4058', 'learning_rate': '0.0001852', 'entropy': '0.8476', 'num_tokens': '1.413e+07', 'mean_token_accuracy': '0.7845', 'epoch': '0.02298'}
{'loss': '0.6746', 'grad_norm': '0.3358', 'learning_rate': '0.0001852', 'entropy': '0.717', 'num_tokens': '1.413e+07', 'mean_token_accuracy': '0.8266', 'epoch': '0.02299'}
{'loss': '0.8139', 'grad_norm': '0.3593', 'learning_rate': '0.0001852', 'entropy': '0.8558', 'num_tokens': '1.414e+07', 'mean_token_accuracy': '0.787', 'epoch': '0.023'}
{'loss': '1.324', 'grad_norm': '0.3268', 'learning_rate': '0.0001852', 'entropy': '1.185', 'num_tokens': '1.415e+07', 'mean_token_accuracy': '0.7241', 'epoch': '0.02301'}
{'loss': '1.187', 'grad_norm': '0.3627', 'learning_rate': '0.0001852', 'entropy': '1.186', 'num_tokens': '1.416e+07', 'mean_token_accuracy': '0.7053', 'epoch': '0.02302'}
{'loss': '1.131', 'grad_norm': '0.3519', 'learning_rate': '0.0001852', 'entropy': '0.8074', 'num_tokens': '1.416e+07', 'mean_token_accuracy': '0.8103', 'epoch': '0.02303'}
{'loss': '0.8251', 'grad_norm': '0.4493', 'learning_rate': '0.0001852', 'entropy': '0.8931', 'num_tokens': '1.417e+07', 'mean_token_accuracy': '0.7904', 'epoch': '0.02304'}
{'loss': '1.072', 'grad_norm': '0.3739', 'learning_rate': '0.0001851', 'entropy': '1.053', 'num_tokens': '1.417e+07', 'mean_token_accuracy': '0.7467', 'epoch': '0.02304'}
{'loss': '1.125', 'grad_norm': '0.3779', 'learning_rate': '0.0001851', 'entropy': '0.9523', 'num_tokens': '1.418e+07', 'mean_token_accuracy': '0.7891', 'epoch': '0.02305'}
{'loss': '0.6636', 'grad_norm': '0.3179', 'learning_rate': '0.0001851', 'entropy': '0.6968', 'num_tokens': '1.418e+07', 'mean_token_accuracy': '0.8266', 'epoch': '0.02306'}
{'loss': '1.131', 'grad_norm': '0.4039', 'learning_rate': '0.0001851', 'entropy': '1.067', 'num_tokens': '1.418e+07', 'mean_token_accuracy': '0.7523', 'epoch': '0.02307'}
{'loss': '1.142', 'grad_norm': '0.4125', 'learning_rate': '0.0001851', 'entropy': '1.115', 'num_tokens': '1.419e+07', 'mean_token_accuracy': '0.7253', 'epoch': '0.02308'}
{'loss': '0.772', 'grad_norm': '0.4103', 'learning_rate': '0.0001851', 'entropy': '0.7314', 'num_tokens': '1.42e+07', 'mean_token_accuracy': '0.8114', 'epoch': '0.02309'}
{'loss': '1.201', 'grad_norm': '0.4238', 'learning_rate': '0.0001851', 'entropy': '1.2', 'num_tokens': '1.42e+07', 'mean_token_accuracy': '0.718', 'epoch': '0.0231'}
{'loss': '0.8322', 'grad_norm': '0.3212', 'learning_rate': '0.000185', 'entropy': '0.8575', 'num_tokens': '1.421e+07', 'mean_token_accuracy': '0.7879', 'epoch': '0.02311'}
{'loss': '1.06', 'grad_norm': '0.4688', 'learning_rate': '0.000185', 'entropy': '0.9592', 'num_tokens': '1.421e+07', 'mean_token_accuracy': '0.7703', 'epoch': '0.02312'}
{'loss': '1.187', 'grad_norm': '0.5393', 'learning_rate': '0.000185', 'entropy': '1.3', 'num_tokens': '1.422e+07', 'mean_token_accuracy': '0.6964', 'epoch': '0.02312'}
{'loss': '0.9779', 'grad_norm': '0.2946', 'learning_rate': '0.000185', 'entropy': '1.037', 'num_tokens': '1.422e+07', 'mean_token_accuracy': '0.7626', 'epoch': '0.02313'}
{'loss': '0.8569', 'grad_norm': '0.3231', 'learning_rate': '0.000185', 'entropy': '0.8914', 'num_tokens': '1.423e+07', 'mean_token_accuracy': '0.7925', 'epoch': '0.02314'}
{'loss': '0.7455', 'grad_norm': '0.2873', 'learning_rate': '0.000185', 'entropy': '0.7907', 'num_tokens': '1.424e+07', 'mean_token_accuracy': '0.7962', 'epoch': '0.02315'}
{'loss': '1.278', 'grad_norm': '0.545', 'learning_rate': '0.000185', 'entropy': '1.2', 'num_tokens': '1.424e+07', 'mean_token_accuracy': '0.7007', 'epoch': '0.02316'}
{'loss': '1.128', 'grad_norm': '0.3881', 'learning_rate': '0.0001849', 'entropy': '1.06', 'num_tokens': '1.425e+07', 'mean_token_accuracy': '0.7467', 'epoch': '0.02317'}
{'loss': '0.8802', 'grad_norm': '0.3892', 'learning_rate': '0.0001849', 'entropy': '0.964', 'num_tokens': '1.425e+07', 'mean_token_accuracy': '0.7592', 'epoch': '0.02318'}
{'loss': '0.8027', 'grad_norm': '0.3506', 'learning_rate': '0.0001849', 'entropy': '0.8215', 'num_tokens': '1.426e+07', 'mean_token_accuracy': '0.7887', 'epoch': '0.02319'}
{'loss': '0.8485', 'grad_norm': '0.4206', 'learning_rate': '0.0001849', 'entropy': '0.8618', 'num_tokens': '1.426e+07', 'mean_token_accuracy': '0.7744', 'epoch': '0.0232'}
{'loss': '0.7998', 'grad_norm': '0.3731', 'learning_rate': '0.0001849', 'entropy': '0.7833', 'num_tokens': '1.427e+07', 'mean_token_accuracy': '0.7853', 'epoch': '0.0232'}
{'loss': '0.8213', 'grad_norm': '0.4324', 'learning_rate': '0.0001849', 'entropy': '0.937', 'num_tokens': '1.427e+07', 'mean_token_accuracy': '0.7658', 'epoch': '0.02321'}
{'loss': '1.329', 'grad_norm': '0.3972', 'learning_rate': '0.0001849', 'entropy': '1.194', 'num_tokens': '1.427e+07', 'mean_token_accuracy': '0.7092', 'epoch': '0.02322'}
{'loss': '0.7461', 'grad_norm': '0.3545', 'learning_rate': '0.0001848', 'entropy': '0.7575', 'num_tokens': '1.428e+07', 'mean_token_accuracy': '0.8004', 'epoch': '0.02323'}
{'loss': '0.8969', 'grad_norm': '0.3548', 'learning_rate': '0.0001848', 'entropy': '0.8879', 'num_tokens': '1.428e+07', 'mean_token_accuracy': '0.7976', 'epoch': '0.02324'}
{'loss': '0.7144', 'grad_norm': '0.4672', 'learning_rate': '0.0001848', 'entropy': '0.6875', 'num_tokens': '1.429e+07', 'mean_token_accuracy': '0.8232', 'epoch': '0.02325'}
{'loss': '0.9133', 'grad_norm': '0.4463', 'learning_rate': '0.0001848', 'entropy': '0.9127', 'num_tokens': '1.429e+07', 'mean_token_accuracy': '0.7898', 'epoch': '0.02326'}
{'loss': '0.74', 'grad_norm': '0.3123', 'learning_rate': '0.0001848', 'entropy': '0.7579', 'num_tokens': '1.43e+07', 'mean_token_accuracy': '0.7998', 'epoch': '0.02327'}
{'loss': '1.099', 'grad_norm': '0.3709', 'learning_rate': '0.0001848', 'entropy': '1.015', 'num_tokens': '1.43e+07', 'mean_token_accuracy': '0.759', 'epoch': '0.02328'}
{'loss': '0.7748', 'grad_norm': '0.3037', 'learning_rate': '0.0001848', 'entropy': '0.7741', 'num_tokens': '1.431e+07', 'mean_token_accuracy': '0.8071', 'epoch': '0.02328'}
{'loss': '0.9725', 'grad_norm': '0.4488', 'learning_rate': '0.0001847', 'entropy': '0.8697', 'num_tokens': '1.432e+07', 'mean_token_accuracy': '0.7811', 'epoch': '0.02329'}
{'loss': '0.8492', 'grad_norm': '0.343', 'learning_rate': '0.0001847', 'entropy': '0.8952', 'num_tokens': '1.432e+07', 'mean_token_accuracy': '0.7781', 'epoch': '0.0233'}
{'loss': '0.6044', 'grad_norm': '0.3058', 'learning_rate': '0.0001847', 'entropy': '0.5959', 'num_tokens': '1.433e+07', 'mean_token_accuracy': '0.845', 'epoch': '0.02331'}
{'loss': '0.6874', 'grad_norm': '0.2687', 'learning_rate': '0.0001847', 'entropy': '0.6835', 'num_tokens': '1.433e+07', 'mean_token_accuracy': '0.8262', 'epoch': '0.02332'}

 22%|██▏       | 2636/12000 [1:35:08<6:37:25,  2.55s/it]
 22%|██▏       | 2637/12000 [1:35:12<7:21:19,  2.83s/it]
                                                        

 22%|██▏       | 2637/12000 [1:35:12<7:21:19,  2.83s/it]
 22%|██▏       | 2638/12000 [1:35:17<9:08:38,  3.52s/it]
                                                        

 22%|██▏       | 2638/12000 [1:35:17<9:08:38,  3.52s/it]
 22%|██▏       | 2639/12000 [1:35:19<8:15:49,  3.18s/it]
                                                        

 22%|██▏       | 2639/12000 [1:35:19<8:15:49,  3.18s/it]
 22%|██▏       | 2640/12000 [1:35:21<6:54:25,  2.66s/it]
                                                        

 22%|██▏       | 2640/12000 [1:35:21<6:54:25,  2.66s/it]
 22%|██▏       | 2641/12000 [1:35:22<6:08:01,  2.36s/it]
                                                        

 22%|██▏       | 2641/12000 [1:35:22<6:08:01,  2.36s/it]
 22%|██▏       | 2642/12000 [1:35:24<5:31:07,  2.12s/it]
                                                        

 22%|██▏       | 2642/12000 [1:35:24<5:31:07,  2.12s/it]
 22%|██▏       | 2643/12000 [1:35:25<5:04:20,  1.95s/it]
                                                        

 22%|██▏       | 2643/12000 [1:35:25<5:04:20,  1.95s/it]
 22%|██▏       | 2644/12000 [1:35:27<5:02:37,  1.94s/it]
                                                        

 22%|██▏       | 2644/12000 [1:35:27<5:02:37,  1.94s/it]
 22%|██▏       | 2645/12000 [1:35:29<4:28:53,  1.72s/it]
                                                        

 22%|██▏       | 2645/12000 [1:35:29<4:28:53,  1.72s/it]
 22%|██▏       | 2646/12000 [1:35:30<4:13:51,  1.63s/it]
                                                        

 22%|██▏       | 2646/12000 [1:35:30<4:13:51,  1.63s/it]
 22%|██▏       | 2647/12000 [1:35:32<4:24:21,  1.70s/it]
                                                        

 22%|██▏       | 2647/12000 [1:35:32<4:24:21,  1.70s/it]
 22%|██▏       | 2648/12000 [1:35:33<4:18:16,  1.66s/it]
                                                        

 22%|██▏       | 2648/12000 [1:35:33<4:18:16,  1.66s/it]
 22%|██▏       | 2649/12000 [1:35:36<4:50:14,  1.86s/it]
                                                        

 22%|██▏       | 2649/12000 [1:35:36<4:50:14,  1.86s/it]
 22%|██▏       | 2650/12000 [1:35:38<4:45:57,  1.83s/it]
                                                        

 22%|██▏       | 2650/12000 [1:35:38<4:45:57,  1.83s/it]
 22%|██▏       | 2651/12000 [1:35:41<5:41:08,  2.19s/it]
                                                        

 22%|██▏       | 2651/12000 [1:35:41<5:41:08,  2.19s/it]
 22%|██▏       | 2652/12000 [1:35:42<5:08:00,  1.98s/it]
                                                        

 22%|██▏       | 2652/12000 [1:35:42<5:08:00,  1.98s/it]
 22%|██▏       | 2653/12000 [1:35:44<4:51:20,  1.87s/it]
                                                        

 22%|██▏       | 2653/12000 [1:35:44<4:51:20,  1.87s/it]
 22%|██▏       | 2654/12000 [1:35:45<4:34:17,  1.76s/it]
                                                        

 22%|██▏       | 2654/12000 [1:35:45<4:34:17,  1.76s/it]
 22%|██▏       | 2655/12000 [1:35:47<4:24:06,  1.70s/it]
                                                        

 22%|██▏       | 2655/12000 [1:35:47<4:24:06,  1.70s/it]
 22%|██▏       | 2656/12000 [1:35:49<4:35:28,  1.77s/it]
                                                        

 22%|██▏       | 2656/12000 [1:35:49<4:35:28,  1.77s/it]
 22%|██▏       | 2657/12000 [1:35:52<5:51:08,  2.25s/it]
                                                        

 22%|██▏       | 2657/12000 [1:35:52<5:51:08,  2.25s/it]
 22%|██▏       | 2658/12000 [1:35:56<7:00:05,  2.70s/it]
                                                        

 22%|██▏       | 2658/12000 [1:35:56<7:00:05,  2.70s/it]
 22%|██▏       | 2659/12000 [1:35:57<6:05:04,  2.34s/it]
                                                        

 22%|██▏       | 2659/12000 [1:35:57<6:05:04,  2.34s/it]
 22%|██▏       | 2660/12000 [1:35:59<5:33:11,  2.14s/it]
                                                        

 22%|██▏       | 2660/12000 [1:35:59<5:33:11,  2.14s/it]
 22%|██▏       | 2661/12000 [1:36:03<6:40:56,  2.58s/it]
                                                        

 22%|██▏       | 2661/12000 [1:36:03<6:40:56,  2.58s/it]
 22%|██▏       | 2662/12000 [1:36:04<5:45:30,  2.22s/it]
                                                        

 22%|██▏       | 2662/12000 [1:36:04<5:45:30,  2.22s/it]
 22%|██▏       | 2663/12000 [1:36:06<5:30:45,  2.13s/it]
                                                        

 22%|██▏       | 2663/12000 [1:36:06<5:30:45,  2.13s/it]
 22%|██▏       | 2664/12000 [1:36:08<5:12:24,  2.01s/it]
                                                        

 22%|██▏       | 2664/12000 [1:36:08<5:12:24,  2.01s/it]
 22%|██▏       | 2665/12000 [1:36:09<4:45:41,  1.84s/it]
                                                        

 22%|██▏       | 2665/12000 [1:36:09<4:45:41,  1.84s/it]
 22%|██▏       | 2666/12000 [1:36:10<4:26:58,  1.72s/it]
                                                        

 22%|██▏       | 2666/12000 [1:36:10<4:26:58,  1.72s/it]
 22%|██▏       | 2667/12000 [1:36:12<4:15:50,  1.64s/it]
                                                        

 22%|██▏       | 2667/12000 [1:36:12<4:15:50,  1.64s/it]
 22%|██▏       | 2668/12000 [1:36:15<4:59:12,  1.92s/it]
                                                        

 22%|██▏       | 2668/12000 [1:36:15<4:59:12,  1.92s/it]
 22%|██▏       | 2669/12000 [1:36:16<4:43:16,  1.82s/it]
                                                        

 22%|██▏       | 2669/12000 [1:36:16<4:43:16,  1.82s/it]
 22%|██▏       | 2670/12000 [1:36:18<4:32:05,  1.75s/it]
                                                        

 22%|██▏       | 2670/12000 [1:36:18<4:32:05,  1.75s/it]
 22%|██▏       | 2671/12000 [1:36:19<4:30:33,  1.74s/it]
                                                        

 22%|██▏       | 2671/12000 [1:36:19<4:30:33,  1.74s/it]
 22%|██▏       | 2672/12000 [1:36:21<4:18:10,  1.66s/it]
                                                        

 22%|██▏       | 2672/12000 [1:36:21<4:18:10,  1.66s/it]
 22%|██▏       | 2673/12000 [1:36:22<3:59:45,  1.54s/it]
                                                        

 22%|██▏       | 2673/12000 [1:36:22<3:59:45,  1.54s/it]
 22%|██▏       | 2674/12000 [1:36:24<4:00:18,  1.55s/it]
                                                        

 22%|██▏       | 2674/12000 [1:36:24<4:00:18,  1.55s/it]
 22%|██▏       | 2675/12000 [1:36:26<4:28:06,  1.73s/it]
                                                        

 22%|██▏       | 2675/12000 [1:36:26<4:28:06,  1.73s/it]
 22%|██▏       | 2676/12000 [1:36:27<4:15:54,  1.65s/it]
                                                        

 22%|██▏       | 2676/12000 [1:36:27<4:15:54,  1.65s/it]
 22%|██▏       | 2677/12000 [1:36:29<4:14:53,  1.64s/it]
                                                        

 22%|██▏       | 2677/12000 [1:36:29<4:14:53,  1.64s/it]
 22%|██▏       | 2678/12000 [1:36:32<5:24:33,  2.09s/it]
                                                        

 22%|██▏       | 2678/12000 [1:36:32<5:24:33,  2.09s/it]
 22%|██▏       | 2679/12000 [1:36:34<4:59:21,  1.93s/it]
                                                        

 22%|██▏       | 2679/12000 [1:36:34<4:59:21,  1.93s/it]
 22%|██▏       | 2680/12000 [1:36:37<6:16:48,  2.43s/it]
                                                        

 22%|██▏       | 2680/12000 [1:36:37<6:16:48,  2.43s/it]
 22%|██▏       | 2681/12000 [1:36:40<6:19:24,  2.44s/it]
                                                        

 22%|██▏       | 2681/12000 [1:36:40<6:19:24,  2.44s/it]
 22%|██▏       | 2682/12000 [1:36:42<6:10:32,  2.39s/it]
                                                        

 22%|██▏       | 2682/12000 [1:36:42<6:10:32,  2.39s/it]
 22%|██▏       | 2683/12000 [1:36:44<5:40:16,  2.19s/it]
                                                        
{'loss': '0.9631', 'grad_norm': '0.3115', 'learning_rate': '0.0001847', 'entropy': '0.9955', 'num_tokens': '1.434e+07', 'mean_token_accuracy': '0.7633', 'epoch': '0.02333'}
{'loss': '0.7877', 'grad_norm': '0.3363', 'learning_rate': '0.0001847', 'entropy': '0.8338', 'num_tokens': '1.435e+07', 'mean_token_accuracy': '0.7898', 'epoch': '0.02334'}
{'loss': '1.212', 'grad_norm': '0.2834', 'learning_rate': '0.0001847', 'entropy': '1.084', 'num_tokens': '1.436e+07', 'mean_token_accuracy': '0.7553', 'epoch': '0.02335'}
{'loss': '1.277', 'grad_norm': '0.3911', 'learning_rate': '0.0001846', 'entropy': '1.333', 'num_tokens': '1.437e+07', 'mean_token_accuracy': '0.6871', 'epoch': '0.02335'}
{'loss': '1.267', 'grad_norm': '0.4968', 'learning_rate': '0.0001846', 'entropy': '1.291', 'num_tokens': '1.437e+07', 'mean_token_accuracy': '0.6898', 'epoch': '0.02336'}
{'loss': '0.7538', 'grad_norm': '0.3567', 'learning_rate': '0.0001846', 'entropy': '0.7435', 'num_tokens': '1.437e+07', 'mean_token_accuracy': '0.8158', 'epoch': '0.02337'}
{'loss': '1.056', 'grad_norm': '0.3858', 'learning_rate': '0.0001846', 'entropy': '1.042', 'num_tokens': '1.438e+07', 'mean_token_accuracy': '0.7507', 'epoch': '0.02338'}
{'loss': '1.045', 'grad_norm': '0.4463', 'learning_rate': '0.0001846', 'entropy': '1.12', 'num_tokens': '1.438e+07', 'mean_token_accuracy': '0.7284', 'epoch': '0.02339'}
{'loss': '1.189', 'grad_norm': '0.3838', 'learning_rate': '0.0001846', 'entropy': '1.254', 'num_tokens': '1.439e+07', 'mean_token_accuracy': '0.6948', 'epoch': '0.0234'}
{'loss': '0.7348', 'grad_norm': '0.3799', 'learning_rate': '0.0001846', 'entropy': '0.7915', 'num_tokens': '1.439e+07', 'mean_token_accuracy': '0.8052', 'epoch': '0.02341'}
{'loss': '1.038', 'grad_norm': '0.3704', 'learning_rate': '0.0001845', 'entropy': '1.133', 'num_tokens': '1.44e+07', 'mean_token_accuracy': '0.7197', 'epoch': '0.02342'}
{'loss': '0.9979', 'grad_norm': '0.4178', 'learning_rate': '0.0001845', 'entropy': '1.111', 'num_tokens': '1.44e+07', 'mean_token_accuracy': '0.7453', 'epoch': '0.02343'}
{'loss': '0.8415', 'grad_norm': '0.3854', 'learning_rate': '0.0001845', 'entropy': '0.8645', 'num_tokens': '1.441e+07', 'mean_token_accuracy': '0.7711', 'epoch': '0.02343'}
{'loss': '1.062', 'grad_norm': '0.4196', 'learning_rate': '0.0001845', 'entropy': '0.9648', 'num_tokens': '1.441e+07', 'mean_token_accuracy': '0.7757', 'epoch': '0.02344'}
{'loss': '0.9401', 'grad_norm': '0.3723', 'learning_rate': '0.0001845', 'entropy': '0.9248', 'num_tokens': '1.442e+07', 'mean_token_accuracy': '0.7712', 'epoch': '0.02345'}
{'loss': '0.8412', 'grad_norm': '0.38', 'learning_rate': '0.0001845', 'entropy': '0.8256', 'num_tokens': '1.443e+07', 'mean_token_accuracy': '0.7904', 'epoch': '0.02346'}
{'loss': '0.927', 'grad_norm': '0.4017', 'learning_rate': '0.0001844', 'entropy': '0.9311', 'num_tokens': '1.443e+07', 'mean_token_accuracy': '0.7597', 'epoch': '0.02347'}
{'loss': '0.9577', 'grad_norm': '0.458', 'learning_rate': '0.0001844', 'entropy': '0.9711', 'num_tokens': '1.443e+07', 'mean_token_accuracy': '0.7671', 'epoch': '0.02348'}
{'loss': '0.7775', 'grad_norm': '0.4099', 'learning_rate': '0.0001844', 'entropy': '0.7874', 'num_tokens': '1.444e+07', 'mean_token_accuracy': '0.7902', 'epoch': '0.02349'}
{'loss': '0.6583', 'grad_norm': '0.3342', 'learning_rate': '0.0001844', 'entropy': '0.6807', 'num_tokens': '1.444e+07', 'mean_token_accuracy': '0.8202', 'epoch': '0.0235'}
{'loss': '1.098', 'grad_norm': '0.4652', 'learning_rate': '0.0001844', 'entropy': '1.051', 'num_tokens': '1.445e+07', 'mean_token_accuracy': '0.7477', 'epoch': '0.02351'}
{'loss': '1.35', 'grad_norm': '0.4475', 'learning_rate': '0.0001844', 'entropy': '1.316', 'num_tokens': '1.446e+07', 'mean_token_accuracy': '0.6689', 'epoch': '0.02351'}
{'loss': '1.079', 'grad_norm': '0.4176', 'learning_rate': '0.0001844', 'entropy': '0.9628', 'num_tokens': '1.446e+07', 'mean_token_accuracy': '0.7674', 'epoch': '0.02352'}
{'loss': '0.849', 'grad_norm': '0.5442', 'learning_rate': '0.0001843', 'entropy': '0.9385', 'num_tokens': '1.446e+07', 'mean_token_accuracy': '0.7638', 'epoch': '0.02353'}
{'loss': '1.08', 'grad_norm': '0.4551', 'learning_rate': '0.0001843', 'entropy': '1.113', 'num_tokens': '1.447e+07', 'mean_token_accuracy': '0.7255', 'epoch': '0.02354'}
{'loss': '0.6553', 'grad_norm': '0.3082', 'learning_rate': '0.0001843', 'entropy': '0.6865', 'num_tokens': '1.448e+07', 'mean_token_accuracy': '0.8292', 'epoch': '0.02355'}
{'loss': '0.8333', 'grad_norm': '0.3598', 'learning_rate': '0.0001843', 'entropy': '0.8351', 'num_tokens': '1.448e+07', 'mean_token_accuracy': '0.8038', 'epoch': '0.02356'}
{'loss': '0.7354', 'grad_norm': '0.3492', 'learning_rate': '0.0001843', 'entropy': '0.825', 'num_tokens': '1.449e+07', 'mean_token_accuracy': '0.7926', 'epoch': '0.02357'}
{'loss': '0.7782', 'grad_norm': '0.346', 'learning_rate': '0.0001843', 'entropy': '0.7569', 'num_tokens': '1.449e+07', 'mean_token_accuracy': '0.7992', 'epoch': '0.02358'}
{'loss': '0.7335', 'grad_norm': '0.3732', 'learning_rate': '0.0001843', 'entropy': '0.7761', 'num_tokens': '1.45e+07', 'mean_token_accuracy': '0.7969', 'epoch': '0.02358'}
{'loss': '0.7489', 'grad_norm': '0.3027', 'learning_rate': '0.0001842', 'entropy': '0.763', 'num_tokens': '1.45e+07', 'mean_token_accuracy': '0.8091', 'epoch': '0.02359'}
{'loss': '0.6702', 'grad_norm': '0.3171', 'learning_rate': '0.0001842', 'entropy': '0.8191', 'num_tokens': '1.45e+07', 'mean_token_accuracy': '0.7925', 'epoch': '0.0236'}
{'loss': '0.7824', 'grad_norm': '0.3493', 'learning_rate': '0.0001842', 'entropy': '0.7678', 'num_tokens': '1.451e+07', 'mean_token_accuracy': '0.8169', 'epoch': '0.02361'}
{'loss': '0.6734', 'grad_norm': '0.3877', 'learning_rate': '0.0001842', 'entropy': '0.691', 'num_tokens': '1.452e+07', 'mean_token_accuracy': '0.8301', 'epoch': '0.02362'}
{'loss': '0.8953', 'grad_norm': '0.4178', 'learning_rate': '0.0001842', 'entropy': '0.9502', 'num_tokens': '1.452e+07', 'mean_token_accuracy': '0.7697', 'epoch': '0.02363'}
{'loss': '0.743', 'grad_norm': '0.3691', 'learning_rate': '0.0001842', 'entropy': '0.7053', 'num_tokens': '1.453e+07', 'mean_token_accuracy': '0.8138', 'epoch': '0.02364'}
{'loss': '0.7746', 'grad_norm': '0.3542', 'learning_rate': '0.0001842', 'entropy': '0.7423', 'num_tokens': '1.453e+07', 'mean_token_accuracy': '0.8151', 'epoch': '0.02365'}
{'loss': '0.9229', 'grad_norm': '0.4242', 'learning_rate': '0.0001841', 'entropy': '0.9824', 'num_tokens': '1.453e+07', 'mean_token_accuracy': '0.7507', 'epoch': '0.02366'}
{'loss': '1.145', 'grad_norm': '0.4783', 'learning_rate': '0.0001841', 'entropy': '1.061', 'num_tokens': '1.454e+07', 'mean_token_accuracy': '0.7348', 'epoch': '0.02366'}
{'loss': '1.008', 'grad_norm': '0.4302', 'learning_rate': '0.0001841', 'entropy': '0.9877', 'num_tokens': '1.454e+07', 'mean_token_accuracy': '0.7609', 'epoch': '0.02367'}
{'loss': '0.782', 'grad_norm': '0.3255', 'learning_rate': '0.0001841', 'entropy': '0.7851', 'num_tokens': '1.455e+07', 'mean_token_accuracy': '0.7967', 'epoch': '0.02368'}
{'loss': '0.7252', 'grad_norm': '0.3587', 'learning_rate': '0.0001841', 'entropy': '0.7638', 'num_tokens': '1.455e+07', 'mean_token_accuracy': '0.7967', 'epoch': '0.02369'}
{'loss': '0.8452', 'grad_norm': '0.2626', 'learning_rate': '0.0001841', 'entropy': '0.9255', 'num_tokens': '1.456e+07', 'mean_token_accuracy': '0.7645', 'epoch': '0.0237'}
{'loss': '1.216', 'grad_norm': '0.4622', 'learning_rate': '0.000184', 'entropy': '1.13', 'num_tokens': '1.456e+07', 'mean_token_accuracy': '0.7419', 'epoch': '0.02371'}
{'loss': '0.5649', 'grad_norm': '0.5041', 'learning_rate': '0.000184', 'entropy': '0.562', 'num_tokens': '1.457e+07', 'mean_token_accuracy': '0.8537', 'epoch': '0.02372'}
{'loss': '0.5941', 'grad_norm': '0.2892', 'learning_rate': '0.000184', 'entropy': '0.7107', 'num_tokens': '1.457e+07', 'mean_token_accuracy': '0.8229', 'epoch': '0.02373'}
{'loss': '1.151', 'grad_norm': '0.4355', 'learning_rate': '0.000184', 'entropy': '1.073', 'num_tokens': '1.458e+07', 'mean_token_accuracy': '0.738', 'epoch': '0.02374'}

 22%|██▏       | 2683/12000 [1:36:44<5:40:16,  2.19s/it]
 22%|██▏       | 2684/12000 [1:36:45<5:05:11,  1.97s/it]
                                                        

 22%|██▏       | 2684/12000 [1:36:45<5:05:11,  1.97s/it]
 22%|██▏       | 2685/12000 [1:36:47<5:23:25,  2.08s/it]
                                                        

 22%|██▏       | 2685/12000 [1:36:47<5:23:25,  2.08s/it]
 22%|██▏       | 2686/12000 [1:36:50<5:39:21,  2.19s/it]
                                                        

 22%|██▏       | 2686/12000 [1:36:50<5:39:21,  2.19s/it]
 22%|██▏       | 2687/12000 [1:36:52<5:12:33,  2.01s/it]
                                                        

 22%|██▏       | 2687/12000 [1:36:52<5:12:33,  2.01s/it]
 22%|██▏       | 2688/12000 [1:36:53<5:07:54,  1.98s/it]
                                                        

 22%|██▏       | 2688/12000 [1:36:53<5:07:54,  1.98s/it]
 22%|██▏       | 2689/12000 [1:36:56<5:45:55,  2.23s/it]
                                                        

 22%|██▏       | 2689/12000 [1:36:56<5:45:55,  2.23s/it]
 22%|██▏       | 2690/12000 [1:36:58<5:19:37,  2.06s/it]
                                                        

 22%|██▏       | 2690/12000 [1:36:58<5:19:37,  2.06s/it]
 22%|██▏       | 2691/12000 [1:36:59<4:50:10,  1.87s/it]
                                                        

 22%|██▏       | 2691/12000 [1:36:59<4:50:10,  1.87s/it]
 22%|██▏       | 2692/12000 [1:37:01<4:31:00,  1.75s/it]
                                                        

 22%|██▏       | 2692/12000 [1:37:01<4:31:00,  1.75s/it]
 22%|██▏       | 2693/12000 [1:37:02<4:27:24,  1.72s/it]
                                                        

 22%|██▏       | 2693/12000 [1:37:02<4:27:24,  1.72s/it]
 22%|██▏       | 2694/12000 [1:37:07<6:49:18,  2.64s/it]
                                                        

 22%|██▏       | 2694/12000 [1:37:07<6:49:18,  2.64s/it]
 22%|██▏       | 2695/12000 [1:37:10<6:43:45,  2.60s/it]
                                                        

 22%|██▏       | 2695/12000 [1:37:10<6:43:45,  2.60s/it]
 22%|██▏       | 2696/12000 [1:37:11<5:53:12,  2.28s/it]
                                                        

 22%|██▏       | 2696/12000 [1:37:11<5:53:12,  2.28s/it]
 22%|██▏       | 2697/12000 [1:37:13<5:22:41,  2.08s/it]
                                                        

 22%|██▏       | 2697/12000 [1:37:13<5:22:41,  2.08s/it]
 22%|██▏       | 2698/12000 [1:37:15<5:14:20,  2.03s/it]
                                                        

 22%|██▏       | 2698/12000 [1:37:15<5:14:20,  2.03s/it]
 22%|██▏       | 2699/12000 [1:37:16<4:51:20,  1.88s/it]
                                                        

 22%|██▏       | 2699/12000 [1:37:16<4:51:20,  1.88s/it]
 22%|██▎       | 2700/12000 [1:37:18<4:59:17,  1.93s/it]
                                                        

 22%|██▎       | 2700/12000 [1:37:18<4:59:17,  1.93s/it]
 23%|██▎       | 2701/12000 [1:37:22<6:07:32,  2.37s/it]
                                                        

 23%|██▎       | 2701/12000 [1:37:22<6:07:32,  2.37s/it]
 23%|██▎       | 2702/12000 [1:37:26<7:13:22,  2.80s/it]
                                                        

 23%|██▎       | 2702/12000 [1:37:26<7:13:22,  2.80s/it]
 23%|██▎       | 2703/12000 [1:37:27<6:22:16,  2.47s/it]
                                                        

 23%|██▎       | 2703/12000 [1:37:27<6:22:16,  2.47s/it]
 23%|██▎       | 2704/12000 [1:37:29<5:57:32,  2.31s/it]
                                                        

 23%|██▎       | 2704/12000 [1:37:29<5:57:32,  2.31s/it]
 23%|██▎       | 2705/12000 [1:37:31<5:21:58,  2.08s/it]
                                                        

 23%|██▎       | 2705/12000 [1:37:31<5:21:58,  2.08s/it]
 23%|██▎       | 2706/12000 [1:37:34<6:02:42,  2.34s/it]
                                                        

 23%|██▎       | 2706/12000 [1:37:34<6:02:42,  2.34s/it]
 23%|██▎       | 2707/12000 [1:37:35<5:33:33,  2.15s/it]
                                                        

 23%|██▎       | 2707/12000 [1:37:35<5:33:33,  2.15s/it]
 23%|██▎       | 2708/12000 [1:37:37<5:08:30,  1.99s/it]
                                                        

 23%|██▎       | 2708/12000 [1:37:37<5:08:30,  1.99s/it]
 23%|██▎       | 2709/12000 [1:37:39<5:18:11,  2.05s/it]
                                                        

 23%|██▎       | 2709/12000 [1:37:39<5:18:11,  2.05s/it]
 23%|██▎       | 2710/12000 [1:37:41<4:55:11,  1.91s/it]
                                                        

 23%|██▎       | 2710/12000 [1:37:41<4:55:11,  1.91s/it]
 23%|██▎       | 2711/12000 [1:37:42<4:29:19,  1.74s/it]
                                                        

 23%|██▎       | 2711/12000 [1:37:42<4:29:19,  1.74s/it]
 23%|██▎       | 2712/12000 [1:37:45<5:18:09,  2.06s/it]
                                                        

 23%|██▎       | 2712/12000 [1:37:45<5:18:09,  2.06s/it]
 23%|██▎       | 2713/12000 [1:37:46<4:53:34,  1.90s/it]
                                                        

 23%|██▎       | 2713/12000 [1:37:46<4:53:34,  1.90s/it]
 23%|██▎       | 2714/12000 [1:37:48<4:39:16,  1.80s/it]
                                                        

 23%|██▎       | 2714/12000 [1:37:48<4:39:16,  1.80s/it]
 23%|██▎       | 2715/12000 [1:37:50<4:49:36,  1.87s/it]
                                                        

 23%|██▎       | 2715/12000 [1:37:50<4:49:36,  1.87s/it]
 23%|██▎       | 2716/12000 [1:37:52<4:49:37,  1.87s/it]
                                                        

 23%|██▎       | 2716/12000 [1:37:52<4:49:37,  1.87s/it]
 23%|██▎       | 2717/12000 [1:37:54<4:39:27,  1.81s/it]
                                                        

 23%|██▎       | 2717/12000 [1:37:54<4:39:27,  1.81s/it]
 23%|██▎       | 2718/12000 [1:37:55<4:44:07,  1.84s/it]
                                                        

 23%|██▎       | 2718/12000 [1:37:55<4:44:07,  1.84s/it]
 23%|██▎       | 2719/12000 [1:37:57<4:42:07,  1.82s/it]
                                                        

 23%|██▎       | 2719/12000 [1:37:57<4:42:07,  1.82s/it]
 23%|██▎       | 2720/12000 [1:38:00<5:23:28,  2.09s/it]
                                                        

 23%|██▎       | 2720/12000 [1:38:00<5:23:28,  2.09s/it]
 23%|██▎       | 2721/12000 [1:38:03<5:42:58,  2.22s/it]
                                                        

 23%|██▎       | 2721/12000 [1:38:03<5:42:58,  2.22s/it]
 23%|██▎       | 2722/12000 [1:38:04<5:27:29,  2.12s/it]
                                                        

 23%|██▎       | 2722/12000 [1:38:04<5:27:29,  2.12s/it]
 23%|██▎       | 2723/12000 [1:38:06<5:24:43,  2.10s/it]
                                                        

 23%|██▎       | 2723/12000 [1:38:06<5:24:43,  2.10s/it]
 23%|██▎       | 2724/12000 [1:38:10<6:31:25,  2.53s/it]
                                                        

 23%|██▎       | 2724/12000 [1:38:10<6:31:25,  2.53s/it]
 23%|██▎       | 2725/12000 [1:38:12<5:44:28,  2.23s/it]
                                                        

 23%|██▎       | 2725/12000 [1:38:12<5:44:28,  2.23s/it]
 23%|██▎       | 2726/12000 [1:38:13<5:29:51,  2.13s/it]
                                                        

 23%|██▎       | 2726/12000 [1:38:13<5:29:51,  2.13s/it]
 23%|██▎       | 2727/12000 [1:38:15<5:08:42,  2.00s/it]
                                                        

 23%|██▎       | 2727/12000 [1:38:15<5:08:42,  2.00s/it]
 23%|██▎       | 2728/12000 [1:38:17<4:50:20,  1.88s/it]
                                                        

 23%|██▎       | 2728/12000 [1:38:17<4:50:20,  1.88s/it]
 23%|██▎       | 2729/12000 [1:38:18<4:40:02,  1.81s/it]
                                                        

 23%|██▎       | 2729/12000 [1:38:18<4:40:02,  1.81s/it]
 23%|██▎       | 2730/12000 [1:38:21<5:16:19,  2.05s/it]
                                                        
{'loss': '0.727', 'grad_norm': '0.3692', 'learning_rate': '0.000184', 'entropy': '0.829', 'num_tokens': '1.459e+07', 'mean_token_accuracy': '0.7892', 'epoch': '0.02374'}
{'loss': '0.7469', 'grad_norm': '0.4159', 'learning_rate': '0.000184', 'entropy': '0.8565', 'num_tokens': '1.459e+07', 'mean_token_accuracy': '0.7931', 'epoch': '0.02375'}
{'loss': '0.8594', 'grad_norm': '0.3976', 'learning_rate': '0.000184', 'entropy': '0.8857', 'num_tokens': '1.46e+07', 'mean_token_accuracy': '0.7752', 'epoch': '0.02376'}
{'loss': '0.977', 'grad_norm': '0.4442', 'learning_rate': '0.0001839', 'entropy': '0.9638', 'num_tokens': '1.46e+07', 'mean_token_accuracy': '0.7632', 'epoch': '0.02377'}
{'loss': '0.7636', 'grad_norm': '0.3757', 'learning_rate': '0.0001839', 'entropy': '0.8036', 'num_tokens': '1.461e+07', 'mean_token_accuracy': '0.795', 'epoch': '0.02378'}
{'loss': '1.429', 'grad_norm': '0.4494', 'learning_rate': '0.0001839', 'entropy': '1.355', 'num_tokens': '1.461e+07', 'mean_token_accuracy': '0.6621', 'epoch': '0.02379'}
{'loss': '0.8935', 'grad_norm': '0.3944', 'learning_rate': '0.0001839', 'entropy': '0.8932', 'num_tokens': '1.462e+07', 'mean_token_accuracy': '0.7817', 'epoch': '0.0238'}
{'loss': '0.8813', 'grad_norm': '0.3903', 'learning_rate': '0.0001839', 'entropy': '0.9079', 'num_tokens': '1.463e+07', 'mean_token_accuracy': '0.7679', 'epoch': '0.02381'}
{'loss': '0.6865', 'grad_norm': '0.3081', 'learning_rate': '0.0001839', 'entropy': '0.7499', 'num_tokens': '1.463e+07', 'mean_token_accuracy': '0.8051', 'epoch': '0.02381'}
{'loss': '0.6828', 'grad_norm': '0.4237', 'learning_rate': '0.0001839', 'entropy': '0.6537', 'num_tokens': '1.463e+07', 'mean_token_accuracy': '0.8203', 'epoch': '0.02382'}
{'loss': '1.178', 'grad_norm': '0.4256', 'learning_rate': '0.0001838', 'entropy': '1.274', 'num_tokens': '1.464e+07', 'mean_token_accuracy': '0.6882', 'epoch': '0.02383'}
{'loss': '1.091', 'grad_norm': '0.3264', 'learning_rate': '0.0001838', 'entropy': '1.042', 'num_tokens': '1.465e+07', 'mean_token_accuracy': '0.7461', 'epoch': '0.02384'}
{'loss': '0.8304', 'grad_norm': '0.3926', 'learning_rate': '0.0001838', 'entropy': '0.7177', 'num_tokens': '1.466e+07', 'mean_token_accuracy': '0.8131', 'epoch': '0.02385'}
{'loss': '0.9908', 'grad_norm': '0.4793', 'learning_rate': '0.0001838', 'entropy': '1.015', 'num_tokens': '1.466e+07', 'mean_token_accuracy': '0.7385', 'epoch': '0.02386'}
{'loss': '0.7669', 'grad_norm': '0.4464', 'learning_rate': '0.0001838', 'entropy': '0.7835', 'num_tokens': '1.466e+07', 'mean_token_accuracy': '0.8057', 'epoch': '0.02387'}
{'loss': '0.8518', 'grad_norm': '0.5029', 'learning_rate': '0.0001838', 'entropy': '0.8474', 'num_tokens': '1.467e+07', 'mean_token_accuracy': '0.7819', 'epoch': '0.02388'}
{'loss': '1.277', 'grad_norm': '0.4519', 'learning_rate': '0.0001837', 'entropy': '1.202', 'num_tokens': '1.467e+07', 'mean_token_accuracy': '0.7159', 'epoch': '0.02389'}
{'loss': '0.8255', 'grad_norm': '0.4044', 'learning_rate': '0.0001837', 'entropy': '0.8735', 'num_tokens': '1.468e+07', 'mean_token_accuracy': '0.7789', 'epoch': '0.02389'}
{'loss': '1.061', 'grad_norm': '0.4547', 'learning_rate': '0.0001837', 'entropy': '1.073', 'num_tokens': '1.468e+07', 'mean_token_accuracy': '0.7316', 'epoch': '0.0239'}
{'loss': '0.8519', 'grad_norm': '0.3339', 'learning_rate': '0.0001837', 'entropy': '0.8219', 'num_tokens': '1.469e+07', 'mean_token_accuracy': '0.7971', 'epoch': '0.02391'}
{'loss': '0.8282', 'grad_norm': '0.3551', 'learning_rate': '0.0001837', 'entropy': '0.8291', 'num_tokens': '1.469e+07', 'mean_token_accuracy': '0.8119', 'epoch': '0.02392'}
{'loss': '0.9981', 'grad_norm': '0.3182', 'learning_rate': '0.0001837', 'entropy': '0.9058', 'num_tokens': '1.47e+07', 'mean_token_accuracy': '0.7761', 'epoch': '0.02393'}
{'loss': '0.897', 'grad_norm': '0.3652', 'learning_rate': '0.0001837', 'entropy': '0.9501', 'num_tokens': '1.47e+07', 'mean_token_accuracy': '0.7812', 'epoch': '0.02394'}
{'loss': '0.7708', 'grad_norm': '0.3961', 'learning_rate': '0.0001836', 'entropy': '0.7364', 'num_tokens': '1.471e+07', 'mean_token_accuracy': '0.8175', 'epoch': '0.02395'}
{'loss': '0.9838', 'grad_norm': '0.4348', 'learning_rate': '0.0001836', 'entropy': '0.9556', 'num_tokens': '1.472e+07', 'mean_token_accuracy': '0.7648', 'epoch': '0.02396'}
{'loss': '1.239', 'grad_norm': '0.4436', 'learning_rate': '0.0001836', 'entropy': '1.165', 'num_tokens': '1.472e+07', 'mean_token_accuracy': '0.7149', 'epoch': '0.02397'}
{'loss': '0.8039', 'grad_norm': '0.4296', 'learning_rate': '0.0001836', 'entropy': '0.8085', 'num_tokens': '1.473e+07', 'mean_token_accuracy': '0.7948', 'epoch': '0.02397'}
{'loss': '0.7091', 'grad_norm': '0.3115', 'learning_rate': '0.0001836', 'entropy': '0.7595', 'num_tokens': '1.473e+07', 'mean_token_accuracy': '0.8162', 'epoch': '0.02398'}
{'loss': '0.8442', 'grad_norm': '0.4381', 'learning_rate': '0.0001836', 'entropy': '0.9368', 'num_tokens': '1.473e+07', 'mean_token_accuracy': '0.7822', 'epoch': '0.02399'}
{'loss': '0.9223', 'grad_norm': '0.3495', 'learning_rate': '0.0001836', 'entropy': '0.9642', 'num_tokens': '1.474e+07', 'mean_token_accuracy': '0.7693', 'epoch': '0.024'}
{'loss': '0.7302', 'grad_norm': '0.3069', 'learning_rate': '0.0001835', 'entropy': '0.7459', 'num_tokens': '1.475e+07', 'mean_token_accuracy': '0.8124', 'epoch': '0.02401'}
{'loss': '0.8029', 'grad_norm': '0.3778', 'learning_rate': '0.0001835', 'entropy': '0.7661', 'num_tokens': '1.475e+07', 'mean_token_accuracy': '0.8028', 'epoch': '0.02402'}
{'loss': '0.8961', 'grad_norm': '0.3926', 'learning_rate': '0.0001835', 'entropy': '0.9497', 'num_tokens': '1.476e+07', 'mean_token_accuracy': '0.7777', 'epoch': '0.02403'}
{'loss': '0.8004', 'grad_norm': '0.3884', 'learning_rate': '0.0001835', 'entropy': '0.7487', 'num_tokens': '1.476e+07', 'mean_token_accuracy': '0.8123', 'epoch': '0.02404'}
{'loss': '0.6032', 'grad_norm': '0.3418', 'learning_rate': '0.0001835', 'entropy': '0.6696', 'num_tokens': '1.477e+07', 'mean_token_accuracy': '0.8256', 'epoch': '0.02404'}
{'loss': '1.185', 'grad_norm': '0.4377', 'learning_rate': '0.0001835', 'entropy': '1.187', 'num_tokens': '1.477e+07', 'mean_token_accuracy': '0.7093', 'epoch': '0.02405'}
{'loss': '1.267', 'grad_norm': '0.4087', 'learning_rate': '0.0001834', 'entropy': '1.159', 'num_tokens': '1.478e+07', 'mean_token_accuracy': '0.7054', 'epoch': '0.02406'}
{'loss': '0.9186', 'grad_norm': '0.3531', 'learning_rate': '0.0001834', 'entropy': '0.7841', 'num_tokens': '1.478e+07', 'mean_token_accuracy': '0.8009', 'epoch': '0.02407'}
{'loss': '0.5237', 'grad_norm': '0.3057', 'learning_rate': '0.0001834', 'entropy': '0.5538', 'num_tokens': '1.479e+07', 'mean_token_accuracy': '0.8597', 'epoch': '0.02408'}
{'loss': '1.087', 'grad_norm': '0.374', 'learning_rate': '0.0001834', 'entropy': '1.163', 'num_tokens': '1.48e+07', 'mean_token_accuracy': '0.7304', 'epoch': '0.02409'}
{'loss': '0.944', 'grad_norm': '0.4235', 'learning_rate': '0.0001834', 'entropy': '0.9476', 'num_tokens': '1.48e+07', 'mean_token_accuracy': '0.758', 'epoch': '0.0241'}
{'loss': '0.7605', 'grad_norm': '0.369', 'learning_rate': '0.0001834', 'entropy': '0.8193', 'num_tokens': '1.481e+07', 'mean_token_accuracy': '0.7949', 'epoch': '0.02411'}
{'loss': '0.7629', 'grad_norm': '0.3725', 'learning_rate': '0.0001834', 'entropy': '0.7758', 'num_tokens': '1.481e+07', 'mean_token_accuracy': '0.8023', 'epoch': '0.02412'}
{'loss': '0.8555', 'grad_norm': '0.3388', 'learning_rate': '0.0001833', 'entropy': '0.7708', 'num_tokens': '1.482e+07', 'mean_token_accuracy': '0.7958', 'epoch': '0.02412'}
{'loss': '1.111', 'grad_norm': '0.4445', 'learning_rate': '0.0001833', 'entropy': '1.201', 'num_tokens': '1.482e+07', 'mean_token_accuracy': '0.7211', 'epoch': '0.02413'}
{'loss': '0.6952', 'grad_norm': '0.4251', 'learning_rate': '0.0001833', 'entropy': '0.6753', 'num_tokens': '1.483e+07', 'mean_token_accuracy': '0.8289', 'epoch': '0.02414'}
{'loss': '0.6069', 'grad_norm': '0.3335', 'learning_rate': '0.0001833', 'entropy': '0.6776', 'num_tokens': '1.483e+07', 'mean_token_accuracy': '0.8245', 'epoch': '0.02415'}

 23%|██▎       | 2730/12000 [1:38:21<5:16:19,  2.05s/it]
 23%|██▎       | 2731/12000 [1:38:22<4:48:03,  1.86s/it]
                                                        

 23%|██▎       | 2731/12000 [1:38:22<4:48:03,  1.86s/it]
 23%|██▎       | 2732/12000 [1:38:24<4:36:56,  1.79s/it]
                                                        

 23%|██▎       | 2732/12000 [1:38:24<4:36:56,  1.79s/it]
 23%|██▎       | 2733/12000 [1:38:26<4:26:51,  1.73s/it]
                                                        

 23%|██▎       | 2733/12000 [1:38:26<4:26:51,  1.73s/it]
 23%|██▎       | 2734/12000 [1:38:27<4:18:01,  1.67s/it]
                                                        

 23%|██▎       | 2734/12000 [1:38:27<4:18:01,  1.67s/it]
 23%|██▎       | 2735/12000 [1:38:29<4:20:18,  1.69s/it]
                                                        

 23%|██▎       | 2735/12000 [1:38:29<4:20:18,  1.69s/it]
 23%|██▎       | 2736/12000 [1:38:31<4:54:10,  1.91s/it]
                                                        

 23%|██▎       | 2736/12000 [1:38:31<4:54:10,  1.91s/it]
 23%|██▎       | 2737/12000 [1:38:34<5:25:16,  2.11s/it]
                                                        

 23%|██▎       | 2737/12000 [1:38:34<5:25:16,  2.11s/it]
 23%|██▎       | 2738/12000 [1:38:37<5:58:54,  2.32s/it]
                                                        

 23%|██▎       | 2738/12000 [1:38:37<5:58:54,  2.32s/it]
 23%|██▎       | 2739/12000 [1:38:39<5:42:56,  2.22s/it]
                                                        

 23%|██▎       | 2739/12000 [1:38:39<5:42:56,  2.22s/it]
 23%|██▎       | 2740/12000 [1:38:40<5:23:43,  2.10s/it]
                                                        

 23%|██▎       | 2740/12000 [1:38:40<5:23:43,  2.10s/it]
 23%|██▎       | 2741/12000 [1:38:42<4:58:36,  1.94s/it]
                                                        

 23%|██▎       | 2741/12000 [1:38:42<4:58:36,  1.94s/it]
 23%|██▎       | 2742/12000 [1:38:44<4:41:10,  1.82s/it]
                                                        

 23%|██▎       | 2742/12000 [1:38:44<4:41:10,  1.82s/it]
 23%|██▎       | 2743/12000 [1:38:45<4:29:13,  1.75s/it]
                                                        

 23%|██▎       | 2743/12000 [1:38:45<4:29:13,  1.75s/it]
 23%|██▎       | 2744/12000 [1:38:48<5:02:55,  1.96s/it]
                                                        

 23%|██▎       | 2744/12000 [1:38:48<5:02:55,  1.96s/it]
 23%|██▎       | 2745/12000 [1:38:49<4:41:04,  1.82s/it]
                                                        

 23%|██▎       | 2745/12000 [1:38:49<4:41:04,  1.82s/it]
 23%|██▎       | 2746/12000 [1:38:53<5:57:54,  2.32s/it]
                                                        

 23%|██▎       | 2746/12000 [1:38:53<5:57:54,  2.32s/it]
 23%|██▎       | 2747/12000 [1:38:54<5:21:50,  2.09s/it]
                                                        

 23%|██▎       | 2747/12000 [1:38:54<5:21:50,  2.09s/it]
 23%|██▎       | 2748/12000 [1:38:56<5:07:01,  1.99s/it]
                                                        

 23%|██▎       | 2748/12000 [1:38:56<5:07:01,  1.99s/it]
 23%|██▎       | 2749/12000 [1:38:58<5:17:34,  2.06s/it]
                                                        

 23%|██▎       | 2749/12000 [1:38:58<5:17:34,  2.06s/it]
 23%|██▎       | 2750/12000 [1:39:00<5:16:28,  2.05s/it]
                                                        

 23%|██▎       | 2750/12000 [1:39:00<5:16:28,  2.05s/it]
 23%|██▎       | 2751/12000 [1:39:04<6:42:28,  2.61s/it]
                                                        

 23%|██▎       | 2751/12000 [1:39:04<6:42:28,  2.61s/it]
 23%|██▎       | 2752/12000 [1:39:06<6:03:18,  2.36s/it]
                                                        

 23%|██▎       | 2752/12000 [1:39:06<6:03:18,  2.36s/it]
 23%|██▎       | 2753/12000 [1:39:07<5:23:21,  2.10s/it]
                                                        

 23%|██▎       | 2753/12000 [1:39:07<5:23:21,  2.10s/it]
 23%|██▎       | 2754/12000 [1:39:10<5:29:08,  2.14s/it]
                                                        

 23%|██▎       | 2754/12000 [1:39:10<5:29:08,  2.14s/it]
 23%|██▎       | 2755/12000 [1:39:11<5:18:39,  2.07s/it]
                                                        

 23%|██▎       | 2755/12000 [1:39:11<5:18:39,  2.07s/it]
 23%|██▎       | 2756/12000 [1:39:14<5:20:26,  2.08s/it]
                                                        

 23%|██▎       | 2756/12000 [1:39:14<5:20:26,  2.08s/it]
 23%|██▎       | 2757/12000 [1:39:15<4:50:35,  1.89s/it]
                                                        

 23%|██▎       | 2757/12000 [1:39:15<4:50:35,  1.89s/it]
 23%|██▎       | 2758/12000 [1:39:17<5:00:28,  1.95s/it]
                                                        

 23%|██▎       | 2758/12000 [1:39:17<5:00:28,  1.95s/it]
 23%|██▎       | 2759/12000 [1:39:19<4:49:07,  1.88s/it]
                                                        

 23%|██▎       | 2759/12000 [1:39:19<4:49:07,  1.88s/it]
 23%|██▎       | 2760/12000 [1:39:21<5:02:32,  1.96s/it]
                                                        

 23%|██▎       | 2760/12000 [1:39:21<5:02:32,  1.96s/it]
 23%|██▎       | 2761/12000 [1:39:23<5:16:52,  2.06s/it]
                                                        

 23%|██▎       | 2761/12000 [1:39:23<5:16:52,  2.06s/it]
 23%|██▎       | 2762/12000 [1:39:25<4:56:41,  1.93s/it]
                                                        

 23%|██▎       | 2762/12000 [1:39:25<4:56:41,  1.93s/it]
 23%|██▎       | 2763/12000 [1:39:27<4:41:41,  1.83s/it]
                                                        

 23%|██▎       | 2763/12000 [1:39:27<4:41:41,  1.83s/it]
 23%|██▎       | 2764/12000 [1:39:31<6:55:49,  2.70s/it]
                                                        

 23%|██▎       | 2764/12000 [1:39:31<6:55:49,  2.70s/it]
 23%|██▎       | 2765/12000 [1:39:35<7:32:11,  2.94s/it]
                                                        

 23%|██▎       | 2765/12000 [1:39:35<7:32:11,  2.94s/it]
 23%|██▎       | 2766/12000 [1:39:36<6:26:23,  2.51s/it]
                                                        

 23%|██▎       | 2766/12000 [1:39:36<6:26:23,  2.51s/it]
 23%|██▎       | 2767/12000 [1:39:38<6:06:42,  2.38s/it]
                                                        

 23%|██▎       | 2767/12000 [1:39:38<6:06:42,  2.38s/it]
 23%|██▎       | 2768/12000 [1:39:42<7:05:21,  2.76s/it]
                                                        

 23%|██▎       | 2768/12000 [1:39:42<7:05:21,  2.76s/it]
 23%|██▎       | 2769/12000 [1:39:43<6:03:55,  2.37s/it]
                                                        

 23%|██▎       | 2769/12000 [1:39:43<6:03:55,  2.37s/it]
 23%|██▎       | 2770/12000 [1:39:45<5:23:32,  2.10s/it]
                                                        

 23%|██▎       | 2770/12000 [1:39:45<5:23:32,  2.10s/it]
 23%|██▎       | 2771/12000 [1:39:47<5:05:02,  1.98s/it]
                                                        

 23%|██▎       | 2771/12000 [1:39:47<5:05:02,  1.98s/it]
 23%|██▎       | 2772/12000 [1:39:48<4:52:29,  1.90s/it]
                                                        

 23%|██▎       | 2772/12000 [1:39:48<4:52:29,  1.90s/it]
 23%|██▎       | 2773/12000 [1:39:50<4:54:42,  1.92s/it]
                                                        

 23%|██▎       | 2773/12000 [1:39:50<4:54:42,  1.92s/it]
 23%|██▎       | 2774/12000 [1:39:53<5:45:27,  2.25s/it]
                                                        

 23%|██▎       | 2774/12000 [1:39:53<5:45:27,  2.25s/it]
 23%|██▎       | 2775/12000 [1:39:56<5:55:26,  2.31s/it]
                                                        

 23%|██▎       | 2775/12000 [1:39:56<5:55:26,  2.31s/it]
 23%|██▎       | 2776/12000 [1:39:57<5:25:18,  2.12s/it]
                                                        

 23%|██▎       | 2776/12000 [1:39:57<5:25:18,  2.12s/it]
 23%|██▎       | 2777/12000 [1:39:59<5:11:35,  2.03s/it]
                                                        
{'loss': '1.685', 'grad_norm': '0.4733', 'learning_rate': '0.0001833', 'entropy': '1.225', 'num_tokens': '1.484e+07', 'mean_token_accuracy': '0.7305', 'epoch': '0.02416'}
{'loss': '0.7457', 'grad_norm': '0.3794', 'learning_rate': '0.0001833', 'entropy': '0.8969', 'num_tokens': '1.484e+07', 'mean_token_accuracy': '0.7946', 'epoch': '0.02417'}
{'loss': '0.7423', 'grad_norm': '0.3183', 'learning_rate': '0.0001832', 'entropy': '0.7824', 'num_tokens': '1.485e+07', 'mean_token_accuracy': '0.8001', 'epoch': '0.02418'}
{'loss': '0.9445', 'grad_norm': '0.4327', 'learning_rate': '0.0001832', 'entropy': '0.9917', 'num_tokens': '1.485e+07', 'mean_token_accuracy': '0.7542', 'epoch': '0.02419'}
{'loss': '0.8014', 'grad_norm': '0.4418', 'learning_rate': '0.0001832', 'entropy': '0.8637', 'num_tokens': '1.485e+07', 'mean_token_accuracy': '0.7825', 'epoch': '0.0242'}
{'loss': '0.994', 'grad_norm': '0.4256', 'learning_rate': '0.0001832', 'entropy': '1.085', 'num_tokens': '1.486e+07', 'mean_token_accuracy': '0.749', 'epoch': '0.0242'}
{'loss': '1.012', 'grad_norm': '0.3904', 'learning_rate': '0.0001832', 'entropy': '1.034', 'num_tokens': '1.487e+07', 'mean_token_accuracy': '0.7567', 'epoch': '0.02421'}
{'loss': '0.6858', 'grad_norm': '0.2969', 'learning_rate': '0.0001832', 'entropy': '0.8098', 'num_tokens': '1.487e+07', 'mean_token_accuracy': '0.7991', 'epoch': '0.02422'}
{'loss': '0.7066', 'grad_norm': '0.3269', 'learning_rate': '0.0001832', 'entropy': '0.8356', 'num_tokens': '1.488e+07', 'mean_token_accuracy': '0.7982', 'epoch': '0.02423'}
{'loss': '1.097', 'grad_norm': '0.3982', 'learning_rate': '0.0001831', 'entropy': '1.17', 'num_tokens': '1.489e+07', 'mean_token_accuracy': '0.7318', 'epoch': '0.02424'}
{'loss': '0.9609', 'grad_norm': '0.3346', 'learning_rate': '0.0001831', 'entropy': '0.8817', 'num_tokens': '1.489e+07', 'mean_token_accuracy': '0.8009', 'epoch': '0.02425'}
{'loss': '0.804', 'grad_norm': '0.375', 'learning_rate': '0.0001831', 'entropy': '0.8337', 'num_tokens': '1.49e+07', 'mean_token_accuracy': '0.788', 'epoch': '0.02426'}
{'loss': '0.7006', 'grad_norm': '0.3654', 'learning_rate': '0.0001831', 'entropy': '0.6985', 'num_tokens': '1.49e+07', 'mean_token_accuracy': '0.8155', 'epoch': '0.02427'}
{'loss': '0.7479', 'grad_norm': '0.4536', 'learning_rate': '0.0001831', 'entropy': '0.7587', 'num_tokens': '1.49e+07', 'mean_token_accuracy': '0.7975', 'epoch': '0.02428'}
{'loss': '0.7759', 'grad_norm': '0.3567', 'learning_rate': '0.0001831', 'entropy': '0.7329', 'num_tokens': '1.491e+07', 'mean_token_accuracy': '0.7956', 'epoch': '0.02428'}
{'loss': '0.8806', 'grad_norm': '0.3947', 'learning_rate': '0.000183', 'entropy': '0.8554', 'num_tokens': '1.491e+07', 'mean_token_accuracy': '0.7824', 'epoch': '0.02429'}
{'loss': '1.084', 'grad_norm': '0.381', 'learning_rate': '0.000183', 'entropy': '1.048', 'num_tokens': '1.492e+07', 'mean_token_accuracy': '0.735', 'epoch': '0.0243'}
{'loss': '0.7565', 'grad_norm': '0.3749', 'learning_rate': '0.000183', 'entropy': '0.7878', 'num_tokens': '1.492e+07', 'mean_token_accuracy': '0.7973', 'epoch': '0.02431'}
{'loss': '0.8312', 'grad_norm': '0.4091', 'learning_rate': '0.000183', 'entropy': '0.8379', 'num_tokens': '1.493e+07', 'mean_token_accuracy': '0.7947', 'epoch': '0.02432'}
{'loss': '0.637', 'grad_norm': '0.384', 'learning_rate': '0.000183', 'entropy': '0.667', 'num_tokens': '1.493e+07', 'mean_token_accuracy': '0.8295', 'epoch': '0.02433'}
{'loss': '0.8778', 'grad_norm': '0.3966', 'learning_rate': '0.000183', 'entropy': '0.8652', 'num_tokens': '1.494e+07', 'mean_token_accuracy': '0.7986', 'epoch': '0.02434'}
{'loss': '0.8081', 'grad_norm': '0.3558', 'learning_rate': '0.000183', 'entropy': '0.8053', 'num_tokens': '1.494e+07', 'mean_token_accuracy': '0.794', 'epoch': '0.02435'}
{'loss': '0.8408', 'grad_norm': '0.3391', 'learning_rate': '0.0001829', 'entropy': '0.8367', 'num_tokens': '1.495e+07', 'mean_token_accuracy': '0.8052', 'epoch': '0.02435'}
{'loss': '0.7073', 'grad_norm': '0.3199', 'learning_rate': '0.0001829', 'entropy': '0.706', 'num_tokens': '1.495e+07', 'mean_token_accuracy': '0.8132', 'epoch': '0.02436'}
{'loss': '1.148', 'grad_norm': '0.3903', 'learning_rate': '0.0001829', 'entropy': '0.9839', 'num_tokens': '1.496e+07', 'mean_token_accuracy': '0.7402', 'epoch': '0.02437'}
{'loss': '0.7091', 'grad_norm': '0.388', 'learning_rate': '0.0001829', 'entropy': '0.7001', 'num_tokens': '1.496e+07', 'mean_token_accuracy': '0.8181', 'epoch': '0.02438'}
{'loss': '0.7857', 'grad_norm': '0.3611', 'learning_rate': '0.0001829', 'entropy': '0.802', 'num_tokens': '1.497e+07', 'mean_token_accuracy': '0.8009', 'epoch': '0.02439'}
{'loss': '1.186', 'grad_norm': '0.462', 'learning_rate': '0.0001829', 'entropy': '1.14', 'num_tokens': '1.497e+07', 'mean_token_accuracy': '0.7407', 'epoch': '0.0244'}
{'loss': '0.9882', 'grad_norm': '0.4072', 'learning_rate': '0.0001828', 'entropy': '0.9262', 'num_tokens': '1.498e+07', 'mean_token_accuracy': '0.7815', 'epoch': '0.02441'}
{'loss': '0.7328', 'grad_norm': '0.3859', 'learning_rate': '0.0001828', 'entropy': '0.7568', 'num_tokens': '1.499e+07', 'mean_token_accuracy': '0.8021', 'epoch': '0.02442'}
{'loss': '1.468', 'grad_norm': '0.5358', 'learning_rate': '0.0001828', 'entropy': '1.328', 'num_tokens': '1.499e+07', 'mean_token_accuracy': '0.6789', 'epoch': '0.02443'}
{'loss': '0.7099', 'grad_norm': '0.304', 'learning_rate': '0.0001828', 'entropy': '0.7627', 'num_tokens': '1.5e+07', 'mean_token_accuracy': '0.8052', 'epoch': '0.02443'}
{'loss': '0.8192', 'grad_norm': '0.4182', 'learning_rate': '0.0001828', 'entropy': '0.7617', 'num_tokens': '1.5e+07', 'mean_token_accuracy': '0.8042', 'epoch': '0.02444'}
{'loss': '0.8356', 'grad_norm': '0.46', 'learning_rate': '0.0001828', 'entropy': '0.8224', 'num_tokens': '1.501e+07', 'mean_token_accuracy': '0.7939', 'epoch': '0.02445'}
{'loss': '1.648', 'grad_norm': '0.7171', 'learning_rate': '0.0001828', 'entropy': '1.099', 'num_tokens': '1.501e+07', 'mean_token_accuracy': '0.7304', 'epoch': '0.02446'}
{'loss': '0.8598', 'grad_norm': '0.3751', 'learning_rate': '0.0001827', 'entropy': '0.82', 'num_tokens': '1.502e+07', 'mean_token_accuracy': '0.7932', 'epoch': '0.02447'}
{'loss': '1.075', 'grad_norm': '0.4616', 'learning_rate': '0.0001827', 'entropy': '1.149', 'num_tokens': '1.503e+07', 'mean_token_accuracy': '0.7191', 'epoch': '0.02448'}
{'loss': '1.057', 'grad_norm': '0.3721', 'learning_rate': '0.0001827', 'entropy': '1.049', 'num_tokens': '1.503e+07', 'mean_token_accuracy': '0.7484', 'epoch': '0.02449'}
{'loss': '0.735', 'grad_norm': '0.2819', 'learning_rate': '0.0001827', 'entropy': '0.717', 'num_tokens': '1.504e+07', 'mean_token_accuracy': '0.811', 'epoch': '0.0245'}
{'loss': '0.6998', 'grad_norm': '0.3942', 'learning_rate': '0.0001827', 'entropy': '0.8001', 'num_tokens': '1.504e+07', 'mean_token_accuracy': '0.7946', 'epoch': '0.02451'}
{'loss': '0.7577', 'grad_norm': '0.338', 'learning_rate': '0.0001827', 'entropy': '0.7691', 'num_tokens': '1.505e+07', 'mean_token_accuracy': '0.8122', 'epoch': '0.02451'}
{'loss': '0.7798', 'grad_norm': '0.3403', 'learning_rate': '0.0001826', 'entropy': '0.8031', 'num_tokens': '1.505e+07', 'mean_token_accuracy': '0.8021', 'epoch': '0.02452'}
{'loss': '1.072', 'grad_norm': '0.3977', 'learning_rate': '0.0001826', 'entropy': '1.066', 'num_tokens': '1.505e+07', 'mean_token_accuracy': '0.7559', 'epoch': '0.02453'}
{'loss': '0.8952', 'grad_norm': '0.3345', 'learning_rate': '0.0001826', 'entropy': '1.105', 'num_tokens': '1.506e+07', 'mean_token_accuracy': '0.7369', 'epoch': '0.02454'}
{'loss': '0.7331', 'grad_norm': '0.3379', 'learning_rate': '0.0001826', 'entropy': '0.6806', 'num_tokens': '1.507e+07', 'mean_token_accuracy': '0.8324', 'epoch': '0.02455'}
{'loss': '1.067', 'grad_norm': '0.4134', 'learning_rate': '0.0001826', 'entropy': '1.126', 'num_tokens': '1.507e+07', 'mean_token_accuracy': '0.7462', 'epoch': '0.02456'}
{'loss': '0.704', 'grad_norm': '0.3937', 'learning_rate': '0.0001826', 'entropy': '0.734', 'num_tokens': '1.508e+07', 'mean_token_accuracy': '0.8038', 'epoch': '0.02457'}

 23%|██▎       | 2777/12000 [1:39:59<5:11:35,  2.03s/it]
 23%|██▎       | 2778/12000 [1:40:01<4:50:43,  1.89s/it]
                                                        

 23%|██▎       | 2778/12000 [1:40:01<4:50:43,  1.89s/it]
 23%|██▎       | 2779/12000 [1:40:05<6:58:41,  2.72s/it]
                                                        

 23%|██▎       | 2779/12000 [1:40:05<6:58:41,  2.72s/it]
 23%|██▎       | 2780/12000 [1:40:07<6:07:39,  2.39s/it]
                                                        

 23%|██▎       | 2780/12000 [1:40:07<6:07:39,  2.39s/it]
 23%|██▎       | 2781/12000 [1:40:09<5:28:58,  2.14s/it]
                                                        

 23%|██▎       | 2781/12000 [1:40:09<5:28:58,  2.14s/it]
 23%|██▎       | 2782/12000 [1:40:10<4:52:37,  1.90s/it]
                                                        

 23%|██▎       | 2782/12000 [1:40:10<4:52:37,  1.90s/it]
 23%|██▎       | 2783/12000 [1:40:13<5:52:58,  2.30s/it]
                                                        

 23%|██▎       | 2783/12000 [1:40:13<5:52:58,  2.30s/it]
 23%|██▎       | 2784/12000 [1:40:15<5:22:27,  2.10s/it]
                                                        

 23%|██▎       | 2784/12000 [1:40:15<5:22:27,  2.10s/it]
 23%|██▎       | 2785/12000 [1:40:16<5:00:26,  1.96s/it]
                                                        

 23%|██▎       | 2785/12000 [1:40:16<5:00:26,  1.96s/it]
 23%|██▎       | 2786/12000 [1:40:18<4:31:27,  1.77s/it]
                                                        

 23%|██▎       | 2786/12000 [1:40:18<4:31:27,  1.77s/it]
 23%|██▎       | 2787/12000 [1:40:20<4:36:22,  1.80s/it]
                                                        

 23%|██▎       | 2787/12000 [1:40:20<4:36:22,  1.80s/it]
 23%|██▎       | 2788/12000 [1:40:21<4:21:32,  1.70s/it]
                                                        

 23%|██▎       | 2788/12000 [1:40:21<4:21:32,  1.70s/it]
 23%|██▎       | 2789/12000 [1:40:23<4:11:54,  1.64s/it]
                                                        

 23%|██▎       | 2789/12000 [1:40:23<4:11:54,  1.64s/it]
 23%|██▎       | 2790/12000 [1:40:28<6:47:54,  2.66s/it]
                                                        

 23%|██▎       | 2790/12000 [1:40:28<6:47:54,  2.66s/it]
 23%|██▎       | 2791/12000 [1:40:30<6:14:34,  2.44s/it]
                                                        

 23%|██▎       | 2791/12000 [1:40:30<6:14:34,  2.44s/it]
 23%|██▎       | 2792/12000 [1:40:32<5:50:49,  2.29s/it]
                                                        

 23%|██▎       | 2792/12000 [1:40:32<5:50:49,  2.29s/it]
 23%|██▎       | 2793/12000 [1:40:34<5:42:14,  2.23s/it]
                                                        

 23%|██▎       | 2793/12000 [1:40:34<5:42:14,  2.23s/it]
 23%|██▎       | 2794/12000 [1:40:35<5:11:31,  2.03s/it]
                                                        

 23%|██▎       | 2794/12000 [1:40:35<5:11:31,  2.03s/it]
 23%|██▎       | 2795/12000 [1:40:37<5:23:37,  2.11s/it]
                                                        

 23%|██▎       | 2795/12000 [1:40:37<5:23:37,  2.11s/it]
 23%|██▎       | 2796/12000 [1:40:39<5:13:39,  2.04s/it]
                                                        

 23%|██▎       | 2796/12000 [1:40:39<5:13:39,  2.04s/it]
 23%|██▎       | 2797/12000 [1:40:41<4:54:54,  1.92s/it]
                                                        

 23%|██▎       | 2797/12000 [1:40:41<4:54:54,  1.92s/it]
 23%|██▎       | 2798/12000 [1:40:43<4:56:23,  1.93s/it]
                                                        

 23%|██▎       | 2798/12000 [1:40:43<4:56:23,  1.93s/it]
 23%|██▎       | 2799/12000 [1:40:44<4:36:53,  1.81s/it]
                                                        

 23%|██▎       | 2799/12000 [1:40:44<4:36:53,  1.81s/it]
 23%|██▎       | 2800/12000 [1:40:46<4:36:36,  1.80s/it]
                                                        

 23%|██▎       | 2800/12000 [1:40:46<4:36:36,  1.80s/it]
 23%|██▎       | 2801/12000 [1:40:50<5:42:29,  2.23s/it]
                                                        

 23%|██▎       | 2801/12000 [1:40:50<5:42:29,  2.23s/it]
 23%|██▎       | 2802/12000 [1:40:51<5:04:04,  1.98s/it]
                                                        

 23%|██▎       | 2802/12000 [1:40:51<5:04:04,  1.98s/it]
 23%|██▎       | 2803/12000 [1:40:53<5:27:22,  2.14s/it]
                                                        

 23%|██▎       | 2803/12000 [1:40:53<5:27:22,  2.14s/it]
 23%|██▎       | 2804/12000 [1:40:56<5:32:12,  2.17s/it]
                                                        

 23%|██▎       | 2804/12000 [1:40:56<5:32:12,  2.17s/it]
 23%|██▎       | 2805/12000 [1:40:58<5:35:10,  2.19s/it]
                                                        

 23%|██▎       | 2805/12000 [1:40:58<5:35:10,  2.19s/it]
 23%|██▎       | 2806/12000 [1:40:59<5:00:12,  1.96s/it]
                                                        

 23%|██▎       | 2806/12000 [1:40:59<5:00:12,  1.96s/it]
 23%|██▎       | 2807/12000 [1:41:02<5:28:17,  2.14s/it]
                                                        

 23%|██▎       | 2807/12000 [1:41:02<5:28:17,  2.14s/it]
 23%|██▎       | 2808/12000 [1:41:05<5:56:41,  2.33s/it]
                                                        

 23%|██▎       | 2808/12000 [1:41:05<5:56:41,  2.33s/it]
 23%|██▎       | 2809/12000 [1:41:06<5:26:47,  2.13s/it]
                                                        

 23%|██▎       | 2809/12000 [1:41:06<5:26:47,  2.13s/it]
 23%|██▎       | 2810/12000 [1:41:08<5:03:23,  1.98s/it]
                                                        

 23%|██▎       | 2810/12000 [1:41:08<5:03:23,  1.98s/it]
 23%|██▎       | 2811/12000 [1:41:09<4:37:30,  1.81s/it]
                                                        

 23%|██▎       | 2811/12000 [1:41:09<4:37:30,  1.81s/it]
 23%|██▎       | 2812/12000 [1:41:13<6:22:40,  2.50s/it]
                                                        

 23%|██▎       | 2812/12000 [1:41:13<6:22:40,  2.50s/it]
 23%|██▎       | 2813/12000 [1:41:16<6:20:05,  2.48s/it]
                                                        

 23%|██▎       | 2813/12000 [1:41:16<6:20:05,  2.48s/it]
 23%|██▎       | 2814/12000 [1:41:17<5:31:17,  2.16s/it]
                                                        

 23%|██▎       | 2814/12000 [1:41:17<5:31:17,  2.16s/it]
 23%|██▎       | 2815/12000 [1:41:19<5:01:11,  1.97s/it]
                                                        

 23%|██▎       | 2815/12000 [1:41:19<5:01:11,  1.97s/it]
 23%|██▎       | 2816/12000 [1:41:23<6:25:45,  2.52s/it]
                                                        

 23%|██▎       | 2816/12000 [1:41:23<6:25:45,  2.52s/it]
 23%|██▎       | 2817/12000 [1:41:25<6:03:04,  2.37s/it]
                                                        

 23%|██▎       | 2817/12000 [1:41:25<6:03:04,  2.37s/it]
 23%|██▎       | 2818/12000 [1:41:27<6:17:30,  2.47s/it]
                                                        

 23%|██▎       | 2818/12000 [1:41:27<6:17:30,  2.47s/it]
 23%|██▎       | 2819/12000 [1:41:29<5:32:25,  2.17s/it]
                                                        

 23%|██▎       | 2819/12000 [1:41:29<5:32:25,  2.17s/it]
 24%|██▎       | 2820/12000 [1:41:30<5:04:08,  1.99s/it]
                                                        

 24%|██▎       | 2820/12000 [1:41:30<5:04:08,  1.99s/it]
 24%|██▎       | 2821/12000 [1:41:33<5:38:55,  2.22s/it]
                                                        

 24%|██▎       | 2821/12000 [1:41:33<5:38:55,  2.22s/it]
 24%|██▎       | 2822/12000 [1:41:36<5:59:51,  2.35s/it]
                                                        

 24%|██▎       | 2822/12000 [1:41:36<5:59:51,  2.35s/it]
 24%|██▎       | 2823/12000 [1:41:39<6:14:07,  2.45s/it]
                                                        

 24%|██▎       | 2823/12000 [1:41:39<6:14:07,  2.45s/it]
 24%|██▎       | 2824/12000 [1:41:40<5:31:48,  2.17s/it]
                                                        
{'loss': '0.9177', 'grad_norm': '0.378', 'learning_rate': '0.0001826', 'entropy': '0.9143', 'num_tokens': '1.508e+07', 'mean_token_accuracy': '0.7842', 'epoch': '0.02458'}
{'loss': '0.7453', 'grad_norm': '0.2893', 'learning_rate': '0.0001825', 'entropy': '0.7775', 'num_tokens': '1.509e+07', 'mean_token_accuracy': '0.8034', 'epoch': '0.02458'}
{'loss': '1.149', 'grad_norm': '0.3698', 'learning_rate': '0.0001825', 'entropy': '0.9516', 'num_tokens': '1.509e+07', 'mean_token_accuracy': '0.7745', 'epoch': '0.02459'}
{'loss': '0.7118', 'grad_norm': '0.4115', 'learning_rate': '0.0001825', 'entropy': '0.7438', 'num_tokens': '1.51e+07', 'mean_token_accuracy': '0.8097', 'epoch': '0.0246'}
{'loss': '1.096', 'grad_norm': '0.4382', 'learning_rate': '0.0001825', 'entropy': '1.13', 'num_tokens': '1.51e+07', 'mean_token_accuracy': '0.7208', 'epoch': '0.02461'}
{'loss': '0.9634', 'grad_norm': '0.4842', 'learning_rate': '0.0001825', 'entropy': '0.985', 'num_tokens': '1.511e+07', 'mean_token_accuracy': '0.762', 'epoch': '0.02462'}
{'loss': '0.8775', 'grad_norm': '0.3352', 'learning_rate': '0.0001825', 'entropy': '0.8772', 'num_tokens': '1.511e+07', 'mean_token_accuracy': '0.7666', 'epoch': '0.02463'}
{'loss': '0.9575', 'grad_norm': '0.3641', 'learning_rate': '0.0001824', 'entropy': '0.9627', 'num_tokens': '1.512e+07', 'mean_token_accuracy': '0.7644', 'epoch': '0.02464'}
{'loss': '0.9476', 'grad_norm': '0.3306', 'learning_rate': '0.0001824', 'entropy': '1.029', 'num_tokens': '1.512e+07', 'mean_token_accuracy': '0.7524', 'epoch': '0.02465'}
{'loss': '0.8523', 'grad_norm': '0.3281', 'learning_rate': '0.0001824', 'entropy': '0.9342', 'num_tokens': '1.513e+07', 'mean_token_accuracy': '0.7679', 'epoch': '0.02466'}
{'loss': '1.303', 'grad_norm': '0.4527', 'learning_rate': '0.0001824', 'entropy': '1.239', 'num_tokens': '1.513e+07', 'mean_token_accuracy': '0.7028', 'epoch': '0.02466'}
{'loss': '0.5883', 'grad_norm': '0.3207', 'learning_rate': '0.0001824', 'entropy': '0.5975', 'num_tokens': '1.514e+07', 'mean_token_accuracy': '0.8447', 'epoch': '0.02467'}
{'loss': '1.121', 'grad_norm': '0.4263', 'learning_rate': '0.0001824', 'entropy': '1.109', 'num_tokens': '1.514e+07', 'mean_token_accuracy': '0.7305', 'epoch': '0.02468'}
{'loss': '0.8438', 'grad_norm': '0.3312', 'learning_rate': '0.0001824', 'entropy': '0.7953', 'num_tokens': '1.515e+07', 'mean_token_accuracy': '0.7975', 'epoch': '0.02469'}
{'loss': '1.332', 'grad_norm': '0.4503', 'learning_rate': '0.0001823', 'entropy': '1.235', 'num_tokens': '1.515e+07', 'mean_token_accuracy': '0.709', 'epoch': '0.0247'}
{'loss': '1.22', 'grad_norm': '0.3966', 'learning_rate': '0.0001823', 'entropy': '1.098', 'num_tokens': '1.516e+07', 'mean_token_accuracy': '0.7211', 'epoch': '0.02471'}
{'loss': '0.6519', 'grad_norm': '0.3367', 'learning_rate': '0.0001823', 'entropy': '0.643', 'num_tokens': '1.517e+07', 'mean_token_accuracy': '0.8445', 'epoch': '0.02472'}
{'loss': '0.6612', 'grad_norm': '0.4001', 'learning_rate': '0.0001823', 'entropy': '0.6773', 'num_tokens': '1.517e+07', 'mean_token_accuracy': '0.8155', 'epoch': '0.02473'}
{'loss': '0.967', 'grad_norm': '0.4286', 'learning_rate': '0.0001823', 'entropy': '0.8999', 'num_tokens': '1.518e+07', 'mean_token_accuracy': '0.77', 'epoch': '0.02474'}
{'loss': '0.9254', 'grad_norm': '0.3646', 'learning_rate': '0.0001823', 'entropy': '0.8794', 'num_tokens': '1.518e+07', 'mean_token_accuracy': '0.7956', 'epoch': '0.02474'}
{'loss': '0.897', 'grad_norm': '0.37', 'learning_rate': '0.0001822', 'entropy': '0.911', 'num_tokens': '1.519e+07', 'mean_token_accuracy': '0.7745', 'epoch': '0.02475'}
{'loss': '1.066', 'grad_norm': '0.4759', 'learning_rate': '0.0001822', 'entropy': '1.039', 'num_tokens': '1.519e+07', 'mean_token_accuracy': '0.7447', 'epoch': '0.02476'}
{'loss': '0.6488', 'grad_norm': '0.3368', 'learning_rate': '0.0001822', 'entropy': '0.7296', 'num_tokens': '1.52e+07', 'mean_token_accuracy': '0.8101', 'epoch': '0.02477'}
{'loss': '0.9041', 'grad_norm': '0.3881', 'learning_rate': '0.0001822', 'entropy': '0.9531', 'num_tokens': '1.52e+07', 'mean_token_accuracy': '0.7714', 'epoch': '0.02478'}
{'loss': '0.8612', 'grad_norm': '0.4208', 'learning_rate': '0.0001822', 'entropy': '0.8148', 'num_tokens': '1.521e+07', 'mean_token_accuracy': '0.7833', 'epoch': '0.02479'}
{'loss': '1.044', 'grad_norm': '0.4249', 'learning_rate': '0.0001822', 'entropy': '1.11', 'num_tokens': '1.521e+07', 'mean_token_accuracy': '0.7334', 'epoch': '0.0248'}
{'loss': '0.962', 'grad_norm': '0.3586', 'learning_rate': '0.0001821', 'entropy': '1.011', 'num_tokens': '1.522e+07', 'mean_token_accuracy': '0.7629', 'epoch': '0.02481'}
{'loss': '1.274', 'grad_norm': '0.4516', 'learning_rate': '0.0001821', 'entropy': '1.189', 'num_tokens': '1.522e+07', 'mean_token_accuracy': '0.7133', 'epoch': '0.02481'}
{'loss': '0.8883', 'grad_norm': '0.3643', 'learning_rate': '0.0001821', 'entropy': '0.7581', 'num_tokens': '1.523e+07', 'mean_token_accuracy': '0.817', 'epoch': '0.02482'}
{'loss': '0.6997', 'grad_norm': '0.3366', 'learning_rate': '0.0001821', 'entropy': '0.7445', 'num_tokens': '1.523e+07', 'mean_token_accuracy': '0.8138', 'epoch': '0.02483'}
{'loss': '0.8975', 'grad_norm': '0.3603', 'learning_rate': '0.0001821', 'entropy': '0.8985', 'num_tokens': '1.524e+07', 'mean_token_accuracy': '0.7833', 'epoch': '0.02484'}
{'loss': '0.6548', 'grad_norm': '0.3706', 'learning_rate': '0.0001821', 'entropy': '0.6665', 'num_tokens': '1.525e+07', 'mean_token_accuracy': '0.8292', 'epoch': '0.02485'}
{'loss': '0.9916', 'grad_norm': '0.442', 'learning_rate': '0.0001821', 'entropy': '0.9996', 'num_tokens': '1.525e+07', 'mean_token_accuracy': '0.7629', 'epoch': '0.02486'}
{'loss': '0.7698', 'grad_norm': '0.334', 'learning_rate': '0.000182', 'entropy': '0.7824', 'num_tokens': '1.526e+07', 'mean_token_accuracy': '0.8102', 'epoch': '0.02487'}
{'loss': '0.7897', 'grad_norm': '0.3505', 'learning_rate': '0.000182', 'entropy': '0.7559', 'num_tokens': '1.527e+07', 'mean_token_accuracy': '0.8056', 'epoch': '0.02488'}
{'loss': '1.04', 'grad_norm': '0.4157', 'learning_rate': '0.000182', 'entropy': '0.954', 'num_tokens': '1.527e+07', 'mean_token_accuracy': '0.7701', 'epoch': '0.02489'}
{'loss': '0.7847', 'grad_norm': '0.2863', 'learning_rate': '0.000182', 'entropy': '0.7885', 'num_tokens': '1.528e+07', 'mean_token_accuracy': '0.812', 'epoch': '0.02489'}
{'loss': '1.014', 'grad_norm': '0.4559', 'learning_rate': '0.000182', 'entropy': '1.045', 'num_tokens': '1.528e+07', 'mean_token_accuracy': '0.7389', 'epoch': '0.0249'}
{'loss': '0.8655', 'grad_norm': '0.3294', 'learning_rate': '0.000182', 'entropy': '0.8229', 'num_tokens': '1.529e+07', 'mean_token_accuracy': '0.7936', 'epoch': '0.02491'}
{'loss': '0.8175', 'grad_norm': '0.3593', 'learning_rate': '0.0001819', 'entropy': '0.8942', 'num_tokens': '1.53e+07', 'mean_token_accuracy': '0.775', 'epoch': '0.02492'}
{'loss': '0.9486', 'grad_norm': '0.4138', 'learning_rate': '0.0001819', 'entropy': '0.8271', 'num_tokens': '1.53e+07', 'mean_token_accuracy': '0.7983', 'epoch': '0.02493'}
{'loss': '0.9634', 'grad_norm': '0.4976', 'learning_rate': '0.0001819', 'entropy': '0.9308', 'num_tokens': '1.531e+07', 'mean_token_accuracy': '0.7935', 'epoch': '0.02494'}
{'loss': '0.8908', 'grad_norm': '0.3466', 'learning_rate': '0.0001819', 'entropy': '0.8858', 'num_tokens': '1.531e+07', 'mean_token_accuracy': '0.773', 'epoch': '0.02495'}
{'loss': '1.08', 'grad_norm': '0.4906', 'learning_rate': '0.0001819', 'entropy': '1.047', 'num_tokens': '1.532e+07', 'mean_token_accuracy': '0.7218', 'epoch': '0.02496'}
{'loss': '0.9161', 'grad_norm': '0.3273', 'learning_rate': '0.0001819', 'entropy': '1.094', 'num_tokens': '1.532e+07', 'mean_token_accuracy': '0.7471', 'epoch': '0.02497'}
{'loss': '1.049', 'grad_norm': '0.344', 'learning_rate': '0.0001818', 'entropy': '1.02', 'num_tokens': '1.533e+07', 'mean_token_accuracy': '0.745', 'epoch': '0.02497'}
{'loss': '0.82', 'grad_norm': '0.3316', 'learning_rate': '0.0001818', 'entropy': '0.8124', 'num_tokens': '1.534e+07', 'mean_token_accuracy': '0.8098', 'epoch': '0.02498'}

 24%|██▎       | 2824/12000 [1:41:40<5:31:48,  2.17s/it]
 24%|██▎       | 2825/12000 [1:41:42<5:20:12,  2.09s/it]
                                                        

 24%|██▎       | 2825/12000 [1:41:42<5:20:12,  2.09s/it]
 24%|██▎       | 2826/12000 [1:41:43<4:46:18,  1.87s/it]
                                                        

 24%|██▎       | 2826/12000 [1:41:43<4:46:18,  1.87s/it]
 24%|██▎       | 2827/12000 [1:41:46<5:12:13,  2.04s/it]
                                                        

 24%|██▎       | 2827/12000 [1:41:46<5:12:13,  2.04s/it]
 24%|██▎       | 2828/12000 [1:41:47<4:56:23,  1.94s/it]
                                                        

 24%|██▎       | 2828/12000 [1:41:47<4:56:23,  1.94s/it]
 24%|██▎       | 2829/12000 [1:41:50<5:06:31,  2.01s/it]
                                                        

 24%|██▎       | 2829/12000 [1:41:50<5:06:31,  2.01s/it]
 24%|██▎       | 2830/12000 [1:41:52<5:09:31,  2.03s/it]
                                                        

 24%|██▎       | 2830/12000 [1:41:52<5:09:31,  2.03s/it]
 24%|██▎       | 2831/12000 [1:41:54<5:37:22,  2.21s/it]
                                                        

 24%|██▎       | 2831/12000 [1:41:54<5:37:22,  2.21s/it]
 24%|██▎       | 2832/12000 [1:41:57<6:00:18,  2.36s/it]
                                                        

 24%|██▎       | 2832/12000 [1:41:57<6:00:18,  2.36s/it]
 24%|██▎       | 2833/12000 [1:42:01<7:06:18,  2.79s/it]
                                                        

 24%|██▎       | 2833/12000 [1:42:01<7:06:18,  2.79s/it]
 24%|██▎       | 2834/12000 [1:42:03<6:41:51,  2.63s/it]
                                                        

 24%|██▎       | 2834/12000 [1:42:03<6:41:51,  2.63s/it]
 24%|██▎       | 2835/12000 [1:42:06<6:50:59,  2.69s/it]
                                                        

 24%|██▎       | 2835/12000 [1:42:06<6:50:59,  2.69s/it]
 24%|██▎       | 2836/12000 [1:42:08<6:03:32,  2.38s/it]
                                                        

 24%|██▎       | 2836/12000 [1:42:08<6:03:32,  2.38s/it]
 24%|██▎       | 2837/12000 [1:42:10<5:45:30,  2.26s/it]
                                                        

 24%|██▎       | 2837/12000 [1:42:10<5:45:30,  2.26s/it]
 24%|██▎       | 2838/12000 [1:42:11<5:28:38,  2.15s/it]
                                                        

 24%|██▎       | 2838/12000 [1:42:11<5:28:38,  2.15s/it]
 24%|██▎       | 2839/12000 [1:42:14<5:25:36,  2.13s/it]
                                                        

 24%|██▎       | 2839/12000 [1:42:14<5:25:36,  2.13s/it]
 24%|██▎       | 2840/12000 [1:42:15<4:48:24,  1.89s/it]
                                                        

 24%|██▎       | 2840/12000 [1:42:15<4:48:24,  1.89s/it]
 24%|██▎       | 2841/12000 [1:42:17<5:16:15,  2.07s/it]
                                                        

 24%|██▎       | 2841/12000 [1:42:17<5:16:15,  2.07s/it]
 24%|██▎       | 2842/12000 [1:42:19<4:56:08,  1.94s/it]
                                                        

 24%|██▎       | 2842/12000 [1:42:19<4:56:08,  1.94s/it]
 24%|██▎       | 2843/12000 [1:42:20<4:36:43,  1.81s/it]
                                                        

 24%|██▎       | 2843/12000 [1:42:20<4:36:43,  1.81s/it]
 24%|██▎       | 2844/12000 [1:42:22<4:18:18,  1.69s/it]
                                                        

 24%|██▎       | 2844/12000 [1:42:22<4:18:18,  1.69s/it]
 24%|██▎       | 2845/12000 [1:42:24<4:22:01,  1.72s/it]
                                                        

 24%|██▎       | 2845/12000 [1:42:24<4:22:01,  1.72s/it]
 24%|██▎       | 2846/12000 [1:42:26<4:49:46,  1.90s/it]
                                                        

 24%|██▎       | 2846/12000 [1:42:26<4:49:46,  1.90s/it]
 24%|██▎       | 2847/12000 [1:42:29<5:29:59,  2.16s/it]
                                                        

 24%|██▎       | 2847/12000 [1:42:29<5:29:59,  2.16s/it]
 24%|██▎       | 2848/12000 [1:42:30<4:59:29,  1.96s/it]
                                                        

 24%|██▎       | 2848/12000 [1:42:30<4:59:29,  1.96s/it]
 24%|██▎       | 2849/12000 [1:42:32<4:44:14,  1.86s/it]
                                                        

 24%|██▎       | 2849/12000 [1:42:32<4:44:14,  1.86s/it]
 24%|██▍       | 2850/12000 [1:42:33<4:16:45,  1.68s/it]
                                                        

 24%|██▍       | 2850/12000 [1:42:33<4:16:45,  1.68s/it]
 24%|██▍       | 2851/12000 [1:42:36<5:05:39,  2.00s/it]
                                                        

 24%|██▍       | 2851/12000 [1:42:36<5:05:39,  2.00s/it]
 24%|██▍       | 2852/12000 [1:42:38<5:02:30,  1.98s/it]
                                                        

 24%|██▍       | 2852/12000 [1:42:38<5:02:30,  1.98s/it]
 24%|██▍       | 2853/12000 [1:42:40<5:01:43,  1.98s/it]
                                                        

 24%|██▍       | 2853/12000 [1:42:40<5:01:43,  1.98s/it]
 24%|██▍       | 2854/12000 [1:42:42<5:11:49,  2.05s/it]
                                                        

 24%|██▍       | 2854/12000 [1:42:42<5:11:49,  2.05s/it]
 24%|██▍       | 2855/12000 [1:42:46<6:36:00,  2.60s/it]
                                                        

 24%|██▍       | 2855/12000 [1:42:46<6:36:00,  2.60s/it]
 24%|██▍       | 2856/12000 [1:42:47<5:39:06,  2.23s/it]
                                                        

 24%|██▍       | 2856/12000 [1:42:47<5:39:06,  2.23s/it]
 24%|██▍       | 2857/12000 [1:42:50<6:10:10,  2.43s/it]
                                                        

 24%|██▍       | 2857/12000 [1:42:50<6:10:10,  2.43s/it]
 24%|██▍       | 2858/12000 [1:42:53<6:05:52,  2.40s/it]
                                                        

 24%|██▍       | 2858/12000 [1:42:53<6:05:52,  2.40s/it]
 24%|██▍       | 2859/12000 [1:42:54<5:25:17,  2.14s/it]
                                                        

 24%|██▍       | 2859/12000 [1:42:54<5:25:17,  2.14s/it]
 24%|██▍       | 2860/12000 [1:42:56<4:55:40,  1.94s/it]
                                                        

 24%|██▍       | 2860/12000 [1:42:56<4:55:40,  1.94s/it]
 24%|██▍       | 2861/12000 [1:42:57<4:34:00,  1.80s/it]
                                                        

 24%|██▍       | 2861/12000 [1:42:57<4:34:00,  1.80s/it]
 24%|██▍       | 2862/12000 [1:42:58<4:18:55,  1.70s/it]
                                                        

 24%|██▍       | 2862/12000 [1:42:58<4:18:55,  1.70s/it]
 24%|██▍       | 2863/12000 [1:43:01<4:37:47,  1.82s/it]
                                                        

 24%|██▍       | 2863/12000 [1:43:01<4:37:47,  1.82s/it]
 24%|██▍       | 2864/12000 [1:43:02<4:33:11,  1.79s/it]
                                                        

 24%|██▍       | 2864/12000 [1:43:02<4:33:11,  1.79s/it]
 24%|██▍       | 2865/12000 [1:43:04<4:19:28,  1.70s/it]
                                                        

 24%|██▍       | 2865/12000 [1:43:04<4:19:28,  1.70s/it]
 24%|██▍       | 2866/12000 [1:43:05<4:10:49,  1.65s/it]
                                                        

 24%|██▍       | 2866/12000 [1:43:05<4:10:49,  1.65s/it]
 24%|██▍       | 2867/12000 [1:43:10<6:31:29,  2.57s/it]
                                                        

 24%|██▍       | 2867/12000 [1:43:10<6:31:29,  2.57s/it]
 24%|██▍       | 2868/12000 [1:43:11<5:35:34,  2.20s/it]
                                                        

 24%|██▍       | 2868/12000 [1:43:11<5:35:34,  2.20s/it]
 24%|██▍       | 2869/12000 [1:43:14<5:46:30,  2.28s/it]
                                                        

 24%|██▍       | 2869/12000 [1:43:14<5:46:30,  2.28s/it]
 24%|██▍       | 2870/12000 [1:43:15<5:13:36,  2.06s/it]
                                                        

 24%|██▍       | 2870/12000 [1:43:15<5:13:36,  2.06s/it]
 24%|██▍       | 2871/12000 [1:43:17<4:42:12,  1.85s/it]
                                                        
{'loss': '1.2', 'grad_norm': '0.3895', 'learning_rate': '0.0001818', 'entropy': '1.208', 'num_tokens': '1.534e+07', 'mean_token_accuracy': '0.7184', 'epoch': '0.02499'}
{'loss': '0.9551', 'grad_norm': '0.3647', 'learning_rate': '0.0001818', 'entropy': '0.9926', 'num_tokens': '1.535e+07', 'mean_token_accuracy': '0.7713', 'epoch': '0.025'}
{'loss': '0.8111', 'grad_norm': '0.3944', 'learning_rate': '0.0001818', 'entropy': '0.9382', 'num_tokens': '1.535e+07', 'mean_token_accuracy': '0.7677', 'epoch': '0.02501'}
{'loss': '1.206', 'grad_norm': '0.468', 'learning_rate': '0.0001818', 'entropy': '1.185', 'num_tokens': '1.536e+07', 'mean_token_accuracy': '0.7217', 'epoch': '0.02502'}
{'loss': '0.7599', 'grad_norm': '0.4917', 'learning_rate': '0.0001818', 'entropy': '0.771', 'num_tokens': '1.536e+07', 'mean_token_accuracy': '0.7968', 'epoch': '0.02503'}
{'loss': '0.8768', 'grad_norm': '0.4078', 'learning_rate': '0.0001817', 'entropy': '0.9282', 'num_tokens': '1.537e+07', 'mean_token_accuracy': '0.7698', 'epoch': '0.02504'}
{'loss': '1.191', 'grad_norm': '0.4033', 'learning_rate': '0.0001817', 'entropy': '1.11', 'num_tokens': '1.537e+07', 'mean_token_accuracy': '0.7231', 'epoch': '0.02505'}
{'loss': '0.667', 'grad_norm': '0.3703', 'learning_rate': '0.0001817', 'entropy': '0.7281', 'num_tokens': '1.538e+07', 'mean_token_accuracy': '0.813', 'epoch': '0.02505'}
{'loss': '0.6401', 'grad_norm': '0.2561', 'learning_rate': '0.0001817', 'entropy': '0.7273', 'num_tokens': '1.538e+07', 'mean_token_accuracy': '0.8093', 'epoch': '0.02506'}
{'loss': '0.6139', 'grad_norm': '0.2948', 'learning_rate': '0.0001817', 'entropy': '0.6044', 'num_tokens': '1.539e+07', 'mean_token_accuracy': '0.8394', 'epoch': '0.02507'}
{'loss': '1.446', 'grad_norm': '0.4471', 'learning_rate': '0.0001817', 'entropy': '1.368', 'num_tokens': '1.539e+07', 'mean_token_accuracy': '0.6711', 'epoch': '0.02508'}
{'loss': '0.9927', 'grad_norm': '0.3199', 'learning_rate': '0.0001816', 'entropy': '0.9619', 'num_tokens': '1.54e+07', 'mean_token_accuracy': '0.7589', 'epoch': '0.02509'}
{'loss': '0.7964', 'grad_norm': '0.3472', 'learning_rate': '0.0001816', 'entropy': '0.7781', 'num_tokens': '1.541e+07', 'mean_token_accuracy': '0.8013', 'epoch': '0.0251'}
{'loss': '0.8864', 'grad_norm': '0.3242', 'learning_rate': '0.0001816', 'entropy': '0.8848', 'num_tokens': '1.541e+07', 'mean_token_accuracy': '0.7688', 'epoch': '0.02511'}
{'loss': '0.5865', 'grad_norm': '0.3913', 'learning_rate': '0.0001816', 'entropy': '0.5907', 'num_tokens': '1.542e+07', 'mean_token_accuracy': '0.8457', 'epoch': '0.02512'}
{'loss': '0.8587', 'grad_norm': '0.3562', 'learning_rate': '0.0001816', 'entropy': '0.8833', 'num_tokens': '1.542e+07', 'mean_token_accuracy': '0.778', 'epoch': '0.02512'}
{'loss': '1.051', 'grad_norm': '0.4204', 'learning_rate': '0.0001816', 'entropy': '1.087', 'num_tokens': '1.543e+07', 'mean_token_accuracy': '0.7307', 'epoch': '0.02513'}
{'loss': '0.7872', 'grad_norm': '0.3251', 'learning_rate': '0.0001815', 'entropy': '0.8455', 'num_tokens': '1.543e+07', 'mean_token_accuracy': '0.8015', 'epoch': '0.02514'}
{'loss': '0.6764', 'grad_norm': '0.3858', 'learning_rate': '0.0001815', 'entropy': '0.7203', 'num_tokens': '1.544e+07', 'mean_token_accuracy': '0.8148', 'epoch': '0.02515'}
{'loss': '0.7631', 'grad_norm': '0.4287', 'learning_rate': '0.0001815', 'entropy': '0.818', 'num_tokens': '1.544e+07', 'mean_token_accuracy': '0.8009', 'epoch': '0.02516'}
{'loss': '0.8923', 'grad_norm': '0.3782', 'learning_rate': '0.0001815', 'entropy': '0.9476', 'num_tokens': '1.544e+07', 'mean_token_accuracy': '0.7709', 'epoch': '0.02517'}
{'loss': '0.6156', 'grad_norm': '0.312', 'learning_rate': '0.0001815', 'entropy': '0.6633', 'num_tokens': '1.545e+07', 'mean_token_accuracy': '0.8346', 'epoch': '0.02518'}
{'loss': '1.252', 'grad_norm': '0.4935', 'learning_rate': '0.0001815', 'entropy': '1.252', 'num_tokens': '1.546e+07', 'mean_token_accuracy': '0.6981', 'epoch': '0.02519'}
{'loss': '0.7191', 'grad_norm': '0.3293', 'learning_rate': '0.0001814', 'entropy': '0.7016', 'num_tokens': '1.546e+07', 'mean_token_accuracy': '0.822', 'epoch': '0.0252'}
{'loss': '1.176', 'grad_norm': '0.3526', 'learning_rate': '0.0001814', 'entropy': '1.105', 'num_tokens': '1.547e+07', 'mean_token_accuracy': '0.7275', 'epoch': '0.0252'}
{'loss': '0.6792', 'grad_norm': '0.3757', 'learning_rate': '0.0001814', 'entropy': '0.67', 'num_tokens': '1.547e+07', 'mean_token_accuracy': '0.818', 'epoch': '0.02521'}
{'loss': '0.9122', 'grad_norm': '0.4149', 'learning_rate': '0.0001814', 'entropy': '0.9664', 'num_tokens': '1.548e+07', 'mean_token_accuracy': '0.7642', 'epoch': '0.02522'}
{'loss': '1.039', 'grad_norm': '0.4132', 'learning_rate': '0.0001814', 'entropy': '0.9672', 'num_tokens': '1.548e+07', 'mean_token_accuracy': '0.7592', 'epoch': '0.02523'}
{'loss': '1.546', 'grad_norm': '0.4525', 'learning_rate': '0.0001814', 'entropy': '1.397', 'num_tokens': '1.549e+07', 'mean_token_accuracy': '0.6862', 'epoch': '0.02524'}
{'loss': '0.7212', 'grad_norm': '0.2928', 'learning_rate': '0.0001814', 'entropy': '0.7225', 'num_tokens': '1.549e+07', 'mean_token_accuracy': '0.804', 'epoch': '0.02525'}
{'loss': '0.971', 'grad_norm': '0.3348', 'learning_rate': '0.0001813', 'entropy': '0.9483', 'num_tokens': '1.55e+07', 'mean_token_accuracy': '0.7719', 'epoch': '0.02526'}
{'loss': '0.8565', 'grad_norm': '0.3244', 'learning_rate': '0.0001813', 'entropy': '0.8535', 'num_tokens': '1.55e+07', 'mean_token_accuracy': '0.7968', 'epoch': '0.02527'}
{'loss': '0.6265', 'grad_norm': '0.3879', 'learning_rate': '0.0001813', 'entropy': '0.6091', 'num_tokens': '1.551e+07', 'mean_token_accuracy': '0.8307', 'epoch': '0.02528'}
{'loss': '0.6554', 'grad_norm': '0.351', 'learning_rate': '0.0001813', 'entropy': '0.6272', 'num_tokens': '1.551e+07', 'mean_token_accuracy': '0.8477', 'epoch': '0.02528'}
{'loss': '1.33', 'grad_norm': '0.4321', 'learning_rate': '0.0001813', 'entropy': '1.315', 'num_tokens': '1.552e+07', 'mean_token_accuracy': '0.705', 'epoch': '0.02529'}
{'loss': '1.145', 'grad_norm': '0.4045', 'learning_rate': '0.0001813', 'entropy': '1.141', 'num_tokens': '1.553e+07', 'mean_token_accuracy': '0.7158', 'epoch': '0.0253'}
{'loss': '0.8383', 'grad_norm': '0.3481', 'learning_rate': '0.0001812', 'entropy': '0.841', 'num_tokens': '1.553e+07', 'mean_token_accuracy': '0.7915', 'epoch': '0.02531'}
{'loss': '0.9271', 'grad_norm': '0.4434', 'learning_rate': '0.0001812', 'entropy': '0.8871', 'num_tokens': '1.553e+07', 'mean_token_accuracy': '0.774', 'epoch': '0.02532'}
{'loss': '0.6915', 'grad_norm': '0.3967', 'learning_rate': '0.0001812', 'entropy': '0.7205', 'num_tokens': '1.554e+07', 'mean_token_accuracy': '0.8165', 'epoch': '0.02533'}
{'loss': '0.8937', 'grad_norm': '0.3643', 'learning_rate': '0.0001812', 'entropy': '1.064', 'num_tokens': '1.554e+07', 'mean_token_accuracy': '0.7502', 'epoch': '0.02534'}
{'loss': '0.8951', 'grad_norm': '0.4056', 'learning_rate': '0.0001812', 'entropy': '0.9015', 'num_tokens': '1.555e+07', 'mean_token_accuracy': '0.7805', 'epoch': '0.02535'}
{'loss': '0.7397', 'grad_norm': '0.379', 'learning_rate': '0.0001812', 'entropy': '0.7586', 'num_tokens': '1.555e+07', 'mean_token_accuracy': '0.8135', 'epoch': '0.02535'}
{'loss': '0.8486', 'grad_norm': '0.4074', 'learning_rate': '0.0001811', 'entropy': '0.9115', 'num_tokens': '1.556e+07', 'mean_token_accuracy': '0.7753', 'epoch': '0.02536'}
{'loss': '1.735', 'grad_norm': '0.3455', 'learning_rate': '0.0001811', 'entropy': '1.367', 'num_tokens': '1.557e+07', 'mean_token_accuracy': '0.6945', 'epoch': '0.02537'}
{'loss': '1.086', 'grad_norm': '0.5129', 'learning_rate': '0.0001811', 'entropy': '1.039', 'num_tokens': '1.557e+07', 'mean_token_accuracy': '0.7491', 'epoch': '0.02538'}
{'loss': '1.298', 'grad_norm': '0.4169', 'learning_rate': '0.0001811', 'entropy': '1.164', 'num_tokens': '1.558e+07', 'mean_token_accuracy': '0.7313', 'epoch': '0.02539'}
{'loss': '1.327', 'grad_norm': '0.5657', 'learning_rate': '0.0001811', 'entropy': '1.186', 'num_tokens': '1.558e+07', 'mean_token_accuracy': '0.6949', 'epoch': '0.0254'}

 24%|██▍       | 2871/12000 [1:43:17<4:42:12,  1.85s/it]
 24%|██▍       | 2872/12000 [1:43:19<4:48:19,  1.90s/it]
                                                        

 24%|██▍       | 2872/12000 [1:43:19<4:48:19,  1.90s/it]
 24%|██▍       | 2873/12000 [1:43:21<4:52:51,  1.93s/it]
                                                        

 24%|██▍       | 2873/12000 [1:43:21<4:52:51,  1.93s/it]
 24%|██▍       | 2874/12000 [1:43:23<5:04:38,  2.00s/it]
                                                        

 24%|██▍       | 2874/12000 [1:43:23<5:04:38,  2.00s/it]
 24%|██▍       | 2875/12000 [1:43:24<4:43:24,  1.86s/it]
                                                        

 24%|██▍       | 2875/12000 [1:43:24<4:43:24,  1.86s/it]
 24%|██▍       | 2876/12000 [1:43:27<4:53:55,  1.93s/it]
                                                        

 24%|██▍       | 2876/12000 [1:43:27<4:53:55,  1.93s/it]
 24%|██▍       | 2877/12000 [1:43:31<7:10:22,  2.83s/it]
                                                        

 24%|██▍       | 2877/12000 [1:43:31<7:10:22,  2.83s/it]
 24%|██▍       | 2878/12000 [1:43:33<6:19:51,  2.50s/it]
                                                        

 24%|██▍       | 2878/12000 [1:43:33<6:19:51,  2.50s/it]
 24%|██▍       | 2879/12000 [1:43:35<5:40:01,  2.24s/it]
                                                        

 24%|██▍       | 2879/12000 [1:43:35<5:40:01,  2.24s/it]
 24%|██▍       | 2880/12000 [1:43:38<6:06:23,  2.41s/it]
                                                        

 24%|██▍       | 2880/12000 [1:43:38<6:06:23,  2.41s/it]
 24%|██▍       | 2881/12000 [1:43:39<5:23:41,  2.13s/it]
                                                        

 24%|██▍       | 2881/12000 [1:43:39<5:23:41,  2.13s/it]
 24%|██▍       | 2882/12000 [1:43:41<5:07:07,  2.02s/it]
                                                        

 24%|██▍       | 2882/12000 [1:43:41<5:07:07,  2.02s/it]
 24%|██▍       | 2883/12000 [1:43:42<4:36:04,  1.82s/it]
                                                        

 24%|██▍       | 2883/12000 [1:43:42<4:36:04,  1.82s/it]
 24%|██▍       | 2884/12000 [1:43:45<5:32:27,  2.19s/it]
                                                        

 24%|██▍       | 2884/12000 [1:43:45<5:32:27,  2.19s/it]
 24%|██▍       | 2885/12000 [1:43:47<5:02:10,  1.99s/it]
                                                        

 24%|██▍       | 2885/12000 [1:43:47<5:02:10,  1.99s/it]
 24%|██▍       | 2886/12000 [1:43:48<4:41:19,  1.85s/it]
                                                        

 24%|██▍       | 2886/12000 [1:43:48<4:41:19,  1.85s/it]
 24%|██▍       | 2887/12000 [1:43:50<4:23:47,  1.74s/it]
                                                        

 24%|██▍       | 2887/12000 [1:43:50<4:23:47,  1.74s/it]
 24%|██▍       | 2888/12000 [1:43:52<4:36:21,  1.82s/it]
                                                        

 24%|██▍       | 2888/12000 [1:43:52<4:36:21,  1.82s/it]
 24%|██▍       | 2889/12000 [1:43:53<4:24:30,  1.74s/it]
                                                        

 24%|██▍       | 2889/12000 [1:43:53<4:24:30,  1.74s/it]
 24%|██▍       | 2890/12000 [1:43:55<4:21:17,  1.72s/it]
                                                        

 24%|██▍       | 2890/12000 [1:43:55<4:21:17,  1.72s/it]
 24%|██▍       | 2891/12000 [1:43:57<4:23:32,  1.74s/it]
                                                        

 24%|██▍       | 2891/12000 [1:43:57<4:23:32,  1.74s/it]
 24%|██▍       | 2892/12000 [1:43:59<4:28:54,  1.77s/it]
                                                        

 24%|██▍       | 2892/12000 [1:43:59<4:28:54,  1.77s/it]
 24%|██▍       | 2893/12000 [1:44:00<4:08:45,  1.64s/it]
                                                        

 24%|██▍       | 2893/12000 [1:44:00<4:08:45,  1.64s/it]
 24%|██▍       | 2894/12000 [1:44:01<4:01:23,  1.59s/it]
                                                        

 24%|██▍       | 2894/12000 [1:44:01<4:01:23,  1.59s/it]
 24%|██▍       | 2895/12000 [1:44:04<4:46:41,  1.89s/it]
                                                        

 24%|██▍       | 2895/12000 [1:44:04<4:46:41,  1.89s/it]
 24%|██▍       | 2896/12000 [1:44:05<4:23:30,  1.74s/it]
                                                        

 24%|██▍       | 2896/12000 [1:44:05<4:23:30,  1.74s/it]
 24%|██▍       | 2897/12000 [1:44:07<4:30:17,  1.78s/it]
                                                        

 24%|██▍       | 2897/12000 [1:44:07<4:30:17,  1.78s/it]
 24%|██▍       | 2898/12000 [1:44:09<4:34:51,  1.81s/it]
                                                        

 24%|██▍       | 2898/12000 [1:44:09<4:34:51,  1.81s/it]
 24%|██▍       | 2899/12000 [1:44:13<5:45:56,  2.28s/it]
                                                        

 24%|██▍       | 2899/12000 [1:44:13<5:45:56,  2.28s/it]
 24%|██▍       | 2900/12000 [1:44:16<6:19:56,  2.51s/it]
                                                        

 24%|██▍       | 2900/12000 [1:44:16<6:19:56,  2.51s/it]
 24%|██▍       | 2901/12000 [1:44:20<7:25:54,  2.94s/it]
                                                        

 24%|██▍       | 2901/12000 [1:44:20<7:25:54,  2.94s/it]
 24%|██▍       | 2902/12000 [1:44:21<6:34:12,  2.60s/it]
                                                        

 24%|██▍       | 2902/12000 [1:44:21<6:34:12,  2.60s/it]
 24%|██▍       | 2903/12000 [1:44:24<6:26:58,  2.55s/it]
                                                        

 24%|██▍       | 2903/12000 [1:44:24<6:26:58,  2.55s/it]
 24%|██▍       | 2904/12000 [1:44:26<5:51:50,  2.32s/it]
                                                        

 24%|██▍       | 2904/12000 [1:44:26<5:51:50,  2.32s/it]
 24%|██▍       | 2905/12000 [1:44:27<5:24:01,  2.14s/it]
                                                        

 24%|██▍       | 2905/12000 [1:44:27<5:24:01,  2.14s/it]
 24%|██▍       | 2906/12000 [1:44:30<5:51:28,  2.32s/it]
                                                        

 24%|██▍       | 2906/12000 [1:44:30<5:51:28,  2.32s/it]
 24%|██▍       | 2907/12000 [1:44:32<5:15:58,  2.08s/it]
                                                        

 24%|██▍       | 2907/12000 [1:44:32<5:15:58,  2.08s/it]
 24%|██▍       | 2908/12000 [1:44:33<5:01:11,  1.99s/it]
                                                        

 24%|██▍       | 2908/12000 [1:44:33<5:01:11,  1.99s/it]
 24%|██▍       | 2909/12000 [1:44:35<4:52:08,  1.93s/it]
                                                        

 24%|██▍       | 2909/12000 [1:44:35<4:52:08,  1.93s/it]
 24%|██▍       | 2910/12000 [1:44:37<4:40:43,  1.85s/it]
                                                        

 24%|██▍       | 2910/12000 [1:44:37<4:40:43,  1.85s/it]
 24%|██▍       | 2911/12000 [1:44:40<5:30:15,  2.18s/it]
                                                        

 24%|██▍       | 2911/12000 [1:44:40<5:30:15,  2.18s/it]
 24%|██▍       | 2912/12000 [1:44:41<5:02:45,  2.00s/it]
                                                        

 24%|██▍       | 2912/12000 [1:44:41<5:02:45,  2.00s/it]
 24%|██▍       | 2913/12000 [1:44:43<5:02:29,  2.00s/it]
                                                        

 24%|██▍       | 2913/12000 [1:44:43<5:02:29,  2.00s/it]
 24%|██▍       | 2914/12000 [1:44:46<5:19:54,  2.11s/it]
                                                        

 24%|██▍       | 2914/12000 [1:44:46<5:19:54,  2.11s/it]
 24%|██▍       | 2915/12000 [1:44:47<4:53:01,  1.94s/it]
                                                        

 24%|██▍       | 2915/12000 [1:44:47<4:53:01,  1.94s/it]
 24%|██▍       | 2916/12000 [1:44:49<5:05:08,  2.02s/it]
                                                        

 24%|██▍       | 2916/12000 [1:44:49<5:05:08,  2.02s/it]
 24%|██▍       | 2917/12000 [1:44:51<4:48:00,  1.90s/it]
                                                        

 24%|██▍       | 2917/12000 [1:44:51<4:48:00,  1.90s/it]
 24%|██▍       | 2918/12000 [1:44:53<4:30:16,  1.79s/it]
                                                        
{'loss': '1.139', 'grad_norm': '0.4607', 'learning_rate': '0.0001811', 'entropy': '1.211', 'num_tokens': '1.558e+07', 'mean_token_accuracy': '0.7134', 'epoch': '0.02541'}
{'loss': '0.7634', 'grad_norm': '0.3378', 'learning_rate': '0.000181', 'entropy': '0.8104', 'num_tokens': '1.559e+07', 'mean_token_accuracy': '0.7995', 'epoch': '0.02542'}
{'loss': '1.331', 'grad_norm': '0.4476', 'learning_rate': '0.000181', 'entropy': '1.221', 'num_tokens': '1.559e+07', 'mean_token_accuracy': '0.7346', 'epoch': '0.02543'}
{'loss': '0.9496', 'grad_norm': '0.4369', 'learning_rate': '0.000181', 'entropy': '0.8661', 'num_tokens': '1.56e+07', 'mean_token_accuracy': '0.7813', 'epoch': '0.02543'}
{'loss': '0.8821', 'grad_norm': '0.3931', 'learning_rate': '0.000181', 'entropy': '0.9187', 'num_tokens': '1.56e+07', 'mean_token_accuracy': '0.792', 'epoch': '0.02544'}
{'loss': '0.9912', 'grad_norm': '0.333', 'learning_rate': '0.000181', 'entropy': '0.9932', 'num_tokens': '1.561e+07', 'mean_token_accuracy': '0.7508', 'epoch': '0.02545'}
{'loss': '0.7106', 'grad_norm': '0.3417', 'learning_rate': '0.000181', 'entropy': '0.871', 'num_tokens': '1.561e+07', 'mean_token_accuracy': '0.789', 'epoch': '0.02546'}
{'loss': '1.047', 'grad_norm': '0.4371', 'learning_rate': '0.000181', 'entropy': '1.009', 'num_tokens': '1.562e+07', 'mean_token_accuracy': '0.767', 'epoch': '0.02547'}
{'loss': '0.7343', 'grad_norm': '0.3742', 'learning_rate': '0.0001809', 'entropy': '0.7487', 'num_tokens': '1.562e+07', 'mean_token_accuracy': '0.824', 'epoch': '0.02548'}
{'loss': '1.413', 'grad_norm': '0.4077', 'learning_rate': '0.0001809', 'entropy': '1.323', 'num_tokens': '1.563e+07', 'mean_token_accuracy': '0.6866', 'epoch': '0.02549'}
{'loss': '0.9094', 'grad_norm': '0.3632', 'learning_rate': '0.0001809', 'entropy': '0.9049', 'num_tokens': '1.563e+07', 'mean_token_accuracy': '0.7875', 'epoch': '0.0255'}
{'loss': '1.266', 'grad_norm': '0.4175', 'learning_rate': '0.0001809', 'entropy': '1.245', 'num_tokens': '1.564e+07', 'mean_token_accuracy': '0.7048', 'epoch': '0.02551'}
{'loss': '0.8187', 'grad_norm': '0.4478', 'learning_rate': '0.0001809', 'entropy': '0.8329', 'num_tokens': '1.564e+07', 'mean_token_accuracy': '0.7897', 'epoch': '0.02551'}
{'loss': '0.7931', 'grad_norm': '0.3486', 'learning_rate': '0.0001809', 'entropy': '0.864', 'num_tokens': '1.565e+07', 'mean_token_accuracy': '0.7722', 'epoch': '0.02552'}
{'loss': '0.6977', 'grad_norm': '0.2884', 'learning_rate': '0.0001808', 'entropy': '0.7163', 'num_tokens': '1.565e+07', 'mean_token_accuracy': '0.8184', 'epoch': '0.02553'}
{'loss': '0.7188', 'grad_norm': '0.2752', 'learning_rate': '0.0001808', 'entropy': '0.7328', 'num_tokens': '1.566e+07', 'mean_token_accuracy': '0.813', 'epoch': '0.02554'}
{'loss': '0.8864', 'grad_norm': '0.4272', 'learning_rate': '0.0001808', 'entropy': '0.9595', 'num_tokens': '1.566e+07', 'mean_token_accuracy': '0.7606', 'epoch': '0.02555'}
{'loss': '0.9958', 'grad_norm': '0.4775', 'learning_rate': '0.0001808', 'entropy': '0.978', 'num_tokens': '1.567e+07', 'mean_token_accuracy': '0.7488', 'epoch': '0.02556'}
{'loss': '0.8133', 'grad_norm': '0.4872', 'learning_rate': '0.0001808', 'entropy': '0.8434', 'num_tokens': '1.567e+07', 'mean_token_accuracy': '0.7838', 'epoch': '0.02557'}
{'loss': '0.7927', 'grad_norm': '0.3971', 'learning_rate': '0.0001808', 'entropy': '0.8469', 'num_tokens': '1.567e+07', 'mean_token_accuracy': '0.7763', 'epoch': '0.02558'}
{'loss': '0.9682', 'grad_norm': '0.3423', 'learning_rate': '0.0001807', 'entropy': '0.9737', 'num_tokens': '1.568e+07', 'mean_token_accuracy': '0.7624', 'epoch': '0.02558'}
{'loss': '0.8104', 'grad_norm': '0.3981', 'learning_rate': '0.0001807', 'entropy': '0.8377', 'num_tokens': '1.569e+07', 'mean_token_accuracy': '0.7848', 'epoch': '0.02559'}
{'loss': '0.844', 'grad_norm': '0.3303', 'learning_rate': '0.0001807', 'entropy': '0.8671', 'num_tokens': '1.569e+07', 'mean_token_accuracy': '0.7863', 'epoch': '0.0256'}
{'loss': '0.8698', 'grad_norm': '0.4272', 'learning_rate': '0.0001807', 'entropy': '0.8387', 'num_tokens': '1.569e+07', 'mean_token_accuracy': '0.778', 'epoch': '0.02561'}
{'loss': '0.8364', 'grad_norm': '0.3528', 'learning_rate': '0.0001807', 'entropy': '0.8877', 'num_tokens': '1.57e+07', 'mean_token_accuracy': '0.7801', 'epoch': '0.02562'}
{'loss': '0.9455', 'grad_norm': '0.3749', 'learning_rate': '0.0001807', 'entropy': '0.958', 'num_tokens': '1.57e+07', 'mean_token_accuracy': '0.7593', 'epoch': '0.02563'}
{'loss': '0.9209', 'grad_norm': '0.4383', 'learning_rate': '0.0001806', 'entropy': '0.8719', 'num_tokens': '1.571e+07', 'mean_token_accuracy': '0.7667', 'epoch': '0.02564'}
{'loss': '1.078', 'grad_norm': '0.4876', 'learning_rate': '0.0001806', 'entropy': '1.037', 'num_tokens': '1.571e+07', 'mean_token_accuracy': '0.745', 'epoch': '0.02565'}
{'loss': '0.695', 'grad_norm': '0.359', 'learning_rate': '0.0001806', 'entropy': '0.7473', 'num_tokens': '1.572e+07', 'mean_token_accuracy': '0.8094', 'epoch': '0.02566'}
{'loss': '1.173', 'grad_norm': '0.3869', 'learning_rate': '0.0001806', 'entropy': '1.144', 'num_tokens': '1.572e+07', 'mean_token_accuracy': '0.7202', 'epoch': '0.02566'}
{'loss': '0.9464', 'grad_norm': '0.3361', 'learning_rate': '0.0001806', 'entropy': '0.9543', 'num_tokens': '1.573e+07', 'mean_token_accuracy': '0.7627', 'epoch': '0.02567'}
{'loss': '0.7109', 'grad_norm': '0.4241', 'learning_rate': '0.0001806', 'entropy': '0.7096', 'num_tokens': '1.574e+07', 'mean_token_accuracy': '0.8102', 'epoch': '0.02568'}
{'loss': '0.6337', 'grad_norm': '0.2494', 'learning_rate': '0.0001805', 'entropy': '0.7318', 'num_tokens': '1.574e+07', 'mean_token_accuracy': '0.825', 'epoch': '0.02569'}
{'loss': '0.9128', 'grad_norm': '0.4412', 'learning_rate': '0.0001805', 'entropy': '0.8667', 'num_tokens': '1.575e+07', 'mean_token_accuracy': '0.7852', 'epoch': '0.0257'}
{'loss': '1.218', 'grad_norm': '0.4198', 'learning_rate': '0.0001805', 'entropy': '1.101', 'num_tokens': '1.575e+07', 'mean_token_accuracy': '0.7328', 'epoch': '0.02571'}
{'loss': '0.9628', 'grad_norm': '0.3824', 'learning_rate': '0.0001805', 'entropy': '0.9136', 'num_tokens': '1.576e+07', 'mean_token_accuracy': '0.7848', 'epoch': '0.02572'}
{'loss': '0.6912', 'grad_norm': '0.5479', 'learning_rate': '0.0001805', 'entropy': '0.6936', 'num_tokens': '1.576e+07', 'mean_token_accuracy': '0.8179', 'epoch': '0.02573'}
{'loss': '0.9702', 'grad_norm': '0.4281', 'learning_rate': '0.0001805', 'entropy': '0.9557', 'num_tokens': '1.577e+07', 'mean_token_accuracy': '0.7726', 'epoch': '0.02574'}
{'loss': '0.8835', 'grad_norm': '0.3546', 'learning_rate': '0.0001804', 'entropy': '0.8918', 'num_tokens': '1.577e+07', 'mean_token_accuracy': '0.7844', 'epoch': '0.02574'}
{'loss': '1.105', 'grad_norm': '0.5905', 'learning_rate': '0.0001804', 'entropy': '1.106', 'num_tokens': '1.578e+07', 'mean_token_accuracy': '0.7461', 'epoch': '0.02575'}
{'loss': '0.7129', 'grad_norm': '0.3464', 'learning_rate': '0.0001804', 'entropy': '0.872', 'num_tokens': '1.578e+07', 'mean_token_accuracy': '0.7755', 'epoch': '0.02576'}
{'loss': '0.5837', 'grad_norm': '0.3687', 'learning_rate': '0.0001804', 'entropy': '0.6425', 'num_tokens': '1.579e+07', 'mean_token_accuracy': '0.8361', 'epoch': '0.02577'}
{'loss': '1.109', 'grad_norm': '0.4014', 'learning_rate': '0.0001804', 'entropy': '1.049', 'num_tokens': '1.579e+07', 'mean_token_accuracy': '0.7579', 'epoch': '0.02578'}
{'loss': '1.041', 'grad_norm': '0.3966', 'learning_rate': '0.0001804', 'entropy': '0.9955', 'num_tokens': '1.58e+07', 'mean_token_accuracy': '0.752', 'epoch': '0.02579'}
{'loss': '0.6021', 'grad_norm': '0.3692', 'learning_rate': '0.0001803', 'entropy': '0.597', 'num_tokens': '1.58e+07', 'mean_token_accuracy': '0.8522', 'epoch': '0.0258'}
{'loss': '0.8478', 'grad_norm': '0.3052', 'learning_rate': '0.0001803', 'entropy': '0.8917', 'num_tokens': '1.581e+07', 'mean_token_accuracy': '0.7784', 'epoch': '0.02581'}
{'loss': '0.9032', 'grad_norm': '0.4361', 'learning_rate': '0.0001803', 'entropy': '0.8305', 'num_tokens': '1.581e+07', 'mean_token_accuracy': '0.7848', 'epoch': '0.02581'}

 24%|██▍       | 2918/12000 [1:44:53<4:30:16,  1.79s/it]
 24%|██▍       | 2919/12000 [1:44:54<4:34:08,  1.81s/it]
                                                        

 24%|██▍       | 2919/12000 [1:44:54<4:34:08,  1.81s/it]
 24%|██▍       | 2920/12000 [1:44:56<4:28:39,  1.78s/it]
                                                        

 24%|██▍       | 2920/12000 [1:44:56<4:28:39,  1.78s/it]
 24%|██▍       | 2921/12000 [1:45:00<6:05:32,  2.42s/it]
                                                        

 24%|██▍       | 2921/12000 [1:45:00<6:05:32,  2.42s/it]
 24%|██▍       | 2922/12000 [1:45:02<5:35:25,  2.22s/it]
                                                        

 24%|██▍       | 2922/12000 [1:45:02<5:35:25,  2.22s/it]
 24%|██▍       | 2923/12000 [1:45:05<6:30:03,  2.58s/it]
                                                        

 24%|██▍       | 2923/12000 [1:45:05<6:30:03,  2.58s/it]
 24%|██▍       | 2924/12000 [1:45:07<5:39:45,  2.25s/it]
                                                        

 24%|██▍       | 2924/12000 [1:45:07<5:39:45,  2.25s/it]
 24%|██▍       | 2925/12000 [1:45:09<5:59:09,  2.37s/it]
                                                        

 24%|██▍       | 2925/12000 [1:45:09<5:59:09,  2.37s/it]
 24%|██▍       | 2926/12000 [1:45:11<5:26:44,  2.16s/it]
                                                        

 24%|██▍       | 2926/12000 [1:45:11<5:26:44,  2.16s/it]
 24%|██▍       | 2927/12000 [1:45:13<5:10:24,  2.05s/it]
                                                        

 24%|██▍       | 2927/12000 [1:45:13<5:10:24,  2.05s/it]
 24%|██▍       | 2928/12000 [1:45:15<4:59:34,  1.98s/it]
                                                        

 24%|██▍       | 2928/12000 [1:45:15<4:59:34,  1.98s/it]
 24%|██▍       | 2929/12000 [1:45:17<5:26:20,  2.16s/it]
                                                        

 24%|██▍       | 2929/12000 [1:45:17<5:26:20,  2.16s/it]
 24%|██▍       | 2930/12000 [1:45:19<4:58:41,  1.98s/it]
                                                        

 24%|██▍       | 2930/12000 [1:45:19<4:58:41,  1.98s/it]
 24%|██▍       | 2931/12000 [1:45:21<5:15:17,  2.09s/it]
                                                        

 24%|██▍       | 2931/12000 [1:45:21<5:15:17,  2.09s/it]
 24%|██▍       | 2932/12000 [1:45:26<7:17:37,  2.90s/it]
                                                        

 24%|██▍       | 2932/12000 [1:45:26<7:17:37,  2.90s/it]
 24%|██▍       | 2933/12000 [1:45:28<6:23:17,  2.54s/it]
                                                        

 24%|██▍       | 2933/12000 [1:45:28<6:23:17,  2.54s/it]
 24%|██▍       | 2934/12000 [1:45:29<5:50:02,  2.32s/it]
                                                        

 24%|██▍       | 2934/12000 [1:45:29<5:50:02,  2.32s/it]
 24%|██▍       | 2935/12000 [1:45:31<5:26:40,  2.16s/it]
                                                        

 24%|██▍       | 2935/12000 [1:45:31<5:26:40,  2.16s/it]
 24%|██▍       | 2936/12000 [1:45:36<7:07:16,  2.83s/it]
                                                        

 24%|██▍       | 2936/12000 [1:45:36<7:07:16,  2.83s/it]
 24%|██▍       | 2937/12000 [1:45:37<6:07:47,  2.43s/it]
                                                        

 24%|██▍       | 2937/12000 [1:45:37<6:07:47,  2.43s/it]
 24%|██▍       | 2938/12000 [1:45:40<6:08:13,  2.44s/it]
                                                        

 24%|██▍       | 2938/12000 [1:45:40<6:08:13,  2.44s/it]
 24%|██▍       | 2939/12000 [1:45:41<5:36:50,  2.23s/it]
                                                        

 24%|██▍       | 2939/12000 [1:45:41<5:36:50,  2.23s/it]
 24%|██▍       | 2940/12000 [1:45:43<5:11:25,  2.06s/it]
                                                        

 24%|██▍       | 2940/12000 [1:45:43<5:11:25,  2.06s/it]
 25%|██▍       | 2941/12000 [1:45:44<4:41:40,  1.87s/it]
                                                        

 25%|██▍       | 2941/12000 [1:45:44<4:41:40,  1.87s/it]
 25%|██▍       | 2942/12000 [1:45:46<4:36:05,  1.83s/it]
                                                        

 25%|██▍       | 2942/12000 [1:45:46<4:36:05,  1.83s/it]
 25%|██▍       | 2943/12000 [1:45:51<6:39:46,  2.65s/it]
                                                        

 25%|██▍       | 2943/12000 [1:45:51<6:39:46,  2.65s/it]
 25%|██▍       | 2944/12000 [1:45:52<5:55:50,  2.36s/it]
                                                        

 25%|██▍       | 2944/12000 [1:45:52<5:55:50,  2.36s/it]
 25%|██▍       | 2945/12000 [1:45:54<5:26:08,  2.16s/it]
                                                        

 25%|██▍       | 2945/12000 [1:45:54<5:26:08,  2.16s/it]
 25%|██▍       | 2946/12000 [1:45:58<6:34:37,  2.62s/it]
                                                        

 25%|██▍       | 2946/12000 [1:45:58<6:34:37,  2.62s/it]
 25%|██▍       | 2947/12000 [1:46:00<5:56:45,  2.36s/it]
                                                        

 25%|██▍       | 2947/12000 [1:46:00<5:56:45,  2.36s/it]
 25%|██▍       | 2948/12000 [1:46:02<5:55:33,  2.36s/it]
                                                        

 25%|██▍       | 2948/12000 [1:46:02<5:55:33,  2.36s/it]
 25%|██▍       | 2949/12000 [1:46:04<5:34:56,  2.22s/it]
                                                        

 25%|██▍       | 2949/12000 [1:46:04<5:34:56,  2.22s/it]
 25%|██▍       | 2950/12000 [1:46:06<5:20:28,  2.12s/it]
                                                        

 25%|██▍       | 2950/12000 [1:46:06<5:20:28,  2.12s/it]
 25%|██▍       | 2951/12000 [1:46:08<5:41:51,  2.27s/it]
                                                        

 25%|██▍       | 2951/12000 [1:46:08<5:41:51,  2.27s/it]
 25%|██▍       | 2952/12000 [1:46:10<5:03:30,  2.01s/it]
                                                        

 25%|██▍       | 2952/12000 [1:46:10<5:03:30,  2.01s/it]
 25%|██▍       | 2953/12000 [1:46:11<4:39:01,  1.85s/it]
                                                        

 25%|██▍       | 2953/12000 [1:46:11<4:39:01,  1.85s/it]
 25%|██▍       | 2954/12000 [1:46:14<5:17:33,  2.11s/it]
                                                        

 25%|██▍       | 2954/12000 [1:46:14<5:17:33,  2.11s/it]
 25%|██▍       | 2955/12000 [1:46:15<4:55:00,  1.96s/it]
                                                        

 25%|██▍       | 2955/12000 [1:46:15<4:55:00,  1.96s/it]
 25%|██▍       | 2956/12000 [1:46:17<4:38:48,  1.85s/it]
                                                        

 25%|██▍       | 2956/12000 [1:46:17<4:38:48,  1.85s/it]
 25%|██▍       | 2957/12000 [1:46:19<4:30:10,  1.79s/it]
                                                        

 25%|██▍       | 2957/12000 [1:46:19<4:30:10,  1.79s/it]
 25%|██▍       | 2958/12000 [1:46:21<5:04:41,  2.02s/it]
                                                        

 25%|██▍       | 2958/12000 [1:46:21<5:04:41,  2.02s/it]
 25%|██▍       | 2959/12000 [1:46:23<5:08:34,  2.05s/it]
                                                        

 25%|██▍       | 2959/12000 [1:46:23<5:08:34,  2.05s/it]
 25%|██▍       | 2960/12000 [1:46:25<4:50:14,  1.93s/it]
                                                        

 25%|██▍       | 2960/12000 [1:46:25<4:50:14,  1.93s/it]
 25%|██▍       | 2961/12000 [1:46:27<4:31:12,  1.80s/it]
                                                        

 25%|██▍       | 2961/12000 [1:46:27<4:31:12,  1.80s/it]
 25%|██▍       | 2962/12000 [1:46:29<4:59:38,  1.99s/it]
                                                        

 25%|██▍       | 2962/12000 [1:46:29<4:59:38,  1.99s/it]
 25%|██▍       | 2963/12000 [1:46:30<4:38:22,  1.85s/it]
                                                        

 25%|██▍       | 2963/12000 [1:46:30<4:38:22,  1.85s/it]
 25%|██▍       | 2964/12000 [1:46:32<4:29:17,  1.79s/it]
                                                        

 25%|██▍       | 2964/12000 [1:46:32<4:29:17,  1.79s/it]
 25%|██▍       | 2965/12000 [1:46:36<6:02:18,  2.41s/it]
                                                        
{'loss': '0.8185', 'grad_norm': '0.3938', 'learning_rate': '0.0001803', 'entropy': '0.803', 'num_tokens': '1.582e+07', 'mean_token_accuracy': '0.7981', 'epoch': '0.02582'}
{'loss': '1.193', 'grad_norm': '0.3775', 'learning_rate': '0.0001803', 'entropy': '1.166', 'num_tokens': '1.582e+07', 'mean_token_accuracy': '0.7124', 'epoch': '0.02583'}
{'loss': '1.023', 'grad_norm': '0.4021', 'learning_rate': '0.0001803', 'entropy': '1.028', 'num_tokens': '1.583e+07', 'mean_token_accuracy': '0.7533', 'epoch': '0.02584'}
{'loss': '0.8029', 'grad_norm': '0.4159', 'learning_rate': '0.0001802', 'entropy': '0.8198', 'num_tokens': '1.584e+07', 'mean_token_accuracy': '0.8044', 'epoch': '0.02585'}
{'loss': '0.824', 'grad_norm': '0.3603', 'learning_rate': '0.0001802', 'entropy': '0.8481', 'num_tokens': '1.584e+07', 'mean_token_accuracy': '0.796', 'epoch': '0.02586'}
{'loss': '1.126', 'grad_norm': '0.3908', 'learning_rate': '0.0001802', 'entropy': '1.242', 'num_tokens': '1.585e+07', 'mean_token_accuracy': '0.7129', 'epoch': '0.02587'}
{'loss': '0.8781', 'grad_norm': '0.3344', 'learning_rate': '0.0001802', 'entropy': '0.7997', 'num_tokens': '1.585e+07', 'mean_token_accuracy': '0.7938', 'epoch': '0.02588'}
{'loss': '0.8283', 'grad_norm': '0.4297', 'learning_rate': '0.0001802', 'entropy': '0.7906', 'num_tokens': '1.586e+07', 'mean_token_accuracy': '0.799', 'epoch': '0.02589'}
{'loss': '1.229', 'grad_norm': '0.5083', 'learning_rate': '0.0001802', 'entropy': '1.184', 'num_tokens': '1.587e+07', 'mean_token_accuracy': '0.6879', 'epoch': '0.02589'}
{'loss': '0.9535', 'grad_norm': '0.4656', 'learning_rate': '0.0001802', 'entropy': '0.9362', 'num_tokens': '1.587e+07', 'mean_token_accuracy': '0.7637', 'epoch': '0.0259'}
{'loss': '1.059', 'grad_norm': '0.4016', 'learning_rate': '0.0001801', 'entropy': '1.061', 'num_tokens': '1.588e+07', 'mean_token_accuracy': '0.735', 'epoch': '0.02591'}
{'loss': '0.8808', 'grad_norm': '0.3206', 'learning_rate': '0.0001801', 'entropy': '0.9752', 'num_tokens': '1.588e+07', 'mean_token_accuracy': '0.765', 'epoch': '0.02592'}
{'loss': '0.6875', 'grad_norm': '0.3081', 'learning_rate': '0.0001801', 'entropy': '0.7502', 'num_tokens': '1.589e+07', 'mean_token_accuracy': '0.8133', 'epoch': '0.02593'}
{'loss': '0.7358', 'grad_norm': '0.2797', 'learning_rate': '0.0001801', 'entropy': '0.9', 'num_tokens': '1.589e+07', 'mean_token_accuracy': '0.7796', 'epoch': '0.02594'}
{'loss': '1.255', 'grad_norm': '0.3597', 'learning_rate': '0.0001801', 'entropy': '0.9987', 'num_tokens': '1.59e+07', 'mean_token_accuracy': '0.7745', 'epoch': '0.02595'}
{'loss': '0.9429', 'grad_norm': '0.5374', 'learning_rate': '0.0001801', 'entropy': '0.961', 'num_tokens': '1.591e+07', 'mean_token_accuracy': '0.7771', 'epoch': '0.02596'}
{'loss': '0.976', 'grad_norm': '0.346', 'learning_rate': '0.00018', 'entropy': '1.095', 'num_tokens': '1.591e+07', 'mean_token_accuracy': '0.7524', 'epoch': '0.02597'}
{'loss': '0.925', 'grad_norm': '0.376', 'learning_rate': '0.00018', 'entropy': '1.06', 'num_tokens': '1.591e+07', 'mean_token_accuracy': '0.7439', 'epoch': '0.02597'}
{'loss': '0.7944', 'grad_norm': '0.2799', 'learning_rate': '0.00018', 'entropy': '0.9344', 'num_tokens': '1.592e+07', 'mean_token_accuracy': '0.7711', 'epoch': '0.02598'}
{'loss': '0.6942', 'grad_norm': '0.3044', 'learning_rate': '0.00018', 'entropy': '0.6978', 'num_tokens': '1.593e+07', 'mean_token_accuracy': '0.826', 'epoch': '0.02599'}
{'loss': '0.7567', 'grad_norm': '0.356', 'learning_rate': '0.00018', 'entropy': '0.7614', 'num_tokens': '1.594e+07', 'mean_token_accuracy': '0.8021', 'epoch': '0.026'}
{'loss': '1.045', 'grad_norm': '0.4452', 'learning_rate': '0.00018', 'entropy': '1.103', 'num_tokens': '1.594e+07', 'mean_token_accuracy': '0.7401', 'epoch': '0.02601'}
{'loss': '1.022', 'grad_norm': '0.3833', 'learning_rate': '0.0001799', 'entropy': '0.9584', 'num_tokens': '1.595e+07', 'mean_token_accuracy': '0.7598', 'epoch': '0.02602'}
{'loss': '0.8009', 'grad_norm': '0.3511', 'learning_rate': '0.0001799', 'entropy': '0.8522', 'num_tokens': '1.595e+07', 'mean_token_accuracy': '0.7876', 'epoch': '0.02603'}
{'loss': '0.6948', 'grad_norm': '0.4214', 'learning_rate': '0.0001799', 'entropy': '0.6813', 'num_tokens': '1.596e+07', 'mean_token_accuracy': '0.8223', 'epoch': '0.02604'}
{'loss': '0.774', 'grad_norm': '0.4147', 'learning_rate': '0.0001799', 'entropy': '0.7592', 'num_tokens': '1.596e+07', 'mean_token_accuracy': '0.803', 'epoch': '0.02605'}
{'loss': '0.9451', 'grad_norm': '0.4425', 'learning_rate': '0.0001799', 'entropy': '0.947', 'num_tokens': '1.597e+07', 'mean_token_accuracy': '0.7603', 'epoch': '0.02605'}
{'loss': '0.827', 'grad_norm': '0.4408', 'learning_rate': '0.0001799', 'entropy': '0.816', 'num_tokens': '1.597e+07', 'mean_token_accuracy': '0.7904', 'epoch': '0.02606'}
{'loss': '0.9', 'grad_norm': '0.3613', 'learning_rate': '0.0001798', 'entropy': '0.8554', 'num_tokens': '1.598e+07', 'mean_token_accuracy': '0.7791', 'epoch': '0.02607'}
{'loss': '1.261', 'grad_norm': '0.4067', 'learning_rate': '0.0001798', 'entropy': '1.264', 'num_tokens': '1.599e+07', 'mean_token_accuracy': '0.7085', 'epoch': '0.02608'}
{'loss': '0.8475', 'grad_norm': '0.3766', 'learning_rate': '0.0001798', 'entropy': '0.8555', 'num_tokens': '1.599e+07', 'mean_token_accuracy': '0.7761', 'epoch': '0.02609'}
{'loss': '0.8755', 'grad_norm': '0.4311', 'learning_rate': '0.0001798', 'entropy': '0.7915', 'num_tokens': '1.6e+07', 'mean_token_accuracy': '0.8028', 'epoch': '0.0261'}
{'loss': '0.8474', 'grad_norm': '0.4193', 'learning_rate': '0.0001798', 'entropy': '0.8328', 'num_tokens': '1.6e+07', 'mean_token_accuracy': '0.7897', 'epoch': '0.02611'}
{'loss': '0.8475', 'grad_norm': '0.3783', 'learning_rate': '0.0001798', 'entropy': '0.8978', 'num_tokens': '1.601e+07', 'mean_token_accuracy': '0.7719', 'epoch': '0.02612'}
{'loss': '0.9603', 'grad_norm': '0.3453', 'learning_rate': '0.0001797', 'entropy': '1.078', 'num_tokens': '1.601e+07', 'mean_token_accuracy': '0.7631', 'epoch': '0.02612'}
{'loss': '0.983', 'grad_norm': '0.427', 'learning_rate': '0.0001797', 'entropy': '1.044', 'num_tokens': '1.602e+07', 'mean_token_accuracy': '0.7403', 'epoch': '0.02613'}
{'loss': '1.041', 'grad_norm': '0.4067', 'learning_rate': '0.0001797', 'entropy': '1.095', 'num_tokens': '1.602e+07', 'mean_token_accuracy': '0.7439', 'epoch': '0.02614'}
{'loss': '0.8588', 'grad_norm': '0.3893', 'learning_rate': '0.0001797', 'entropy': '0.8719', 'num_tokens': '1.603e+07', 'mean_token_accuracy': '0.7864', 'epoch': '0.02615'}
{'loss': '0.8149', 'grad_norm': '0.4581', 'learning_rate': '0.0001797', 'entropy': '0.9002', 'num_tokens': '1.603e+07', 'mean_token_accuracy': '0.7665', 'epoch': '0.02616'}
{'loss': '0.964', 'grad_norm': '0.4021', 'learning_rate': '0.0001797', 'entropy': '0.9792', 'num_tokens': '1.604e+07', 'mean_token_accuracy': '0.7508', 'epoch': '0.02617'}
{'loss': '0.941', 'grad_norm': '0.4619', 'learning_rate': '0.0001796', 'entropy': '0.9964', 'num_tokens': '1.604e+07', 'mean_token_accuracy': '0.7608', 'epoch': '0.02618'}
{'loss': '1.257', 'grad_norm': '0.4023', 'learning_rate': '0.0001796', 'entropy': '1.171', 'num_tokens': '1.605e+07', 'mean_token_accuracy': '0.7065', 'epoch': '0.02619'}
{'loss': '0.7621', 'grad_norm': '0.3558', 'learning_rate': '0.0001796', 'entropy': '0.778', 'num_tokens': '1.606e+07', 'mean_token_accuracy': '0.8067', 'epoch': '0.0262'}
{'loss': '1.09', 'grad_norm': '0.4205', 'learning_rate': '0.0001796', 'entropy': '1.036', 'num_tokens': '1.606e+07', 'mean_token_accuracy': '0.7365', 'epoch': '0.0262'}
{'loss': '0.7636', 'grad_norm': '0.38', 'learning_rate': '0.0001796', 'entropy': '0.8114', 'num_tokens': '1.607e+07', 'mean_token_accuracy': '0.7925', 'epoch': '0.02621'}
{'loss': '0.8223', 'grad_norm': '0.344', 'learning_rate': '0.0001796', 'entropy': '0.8263', 'num_tokens': '1.607e+07', 'mean_token_accuracy': '0.7793', 'epoch': '0.02622'}
{'loss': '0.8094', 'grad_norm': '0.3577', 'learning_rate': '0.0001795', 'entropy': '0.847', 'num_tokens': '1.608e+07', 'mean_token_accuracy': '0.784', 'epoch': '0.02623'}

 25%|██▍       | 2965/12000 [1:46:36<6:02:18,  2.41s/it]
 25%|██▍       | 2966/12000 [1:46:38<5:32:17,  2.21s/it]
                                                        

 25%|██▍       | 2966/12000 [1:46:38<5:32:17,  2.21s/it]
 25%|██▍       | 2967/12000 [1:46:39<4:57:44,  1.98s/it]
                                                        

 25%|██▍       | 2967/12000 [1:46:39<4:57:44,  1.98s/it]
 25%|██▍       | 2968/12000 [1:46:41<4:41:29,  1.87s/it]
                                                        

 25%|██▍       | 2968/12000 [1:46:41<4:41:29,  1.87s/it]
 25%|██▍       | 2969/12000 [1:46:45<6:43:58,  2.68s/it]
                                                        

 25%|██▍       | 2969/12000 [1:46:45<6:43:58,  2.68s/it]
 25%|██▍       | 2970/12000 [1:46:47<5:42:47,  2.28s/it]
                                                        

 25%|██▍       | 2970/12000 [1:46:47<5:42:47,  2.28s/it]
 25%|██▍       | 2971/12000 [1:46:48<4:59:32,  1.99s/it]
                                                        

 25%|██▍       | 2971/12000 [1:46:48<4:59:32,  1.99s/it]
 25%|██▍       | 2972/12000 [1:46:52<6:50:22,  2.73s/it]
                                                        

 25%|██▍       | 2972/12000 [1:46:52<6:50:22,  2.73s/it]
 25%|██▍       | 2973/12000 [1:46:55<6:46:56,  2.70s/it]
                                                        

 25%|██▍       | 2973/12000 [1:46:55<6:46:56,  2.70s/it]
 25%|██▍       | 2974/12000 [1:46:57<6:07:23,  2.44s/it]
                                                        

 25%|██▍       | 2974/12000 [1:46:57<6:07:23,  2.44s/it]
 25%|██▍       | 2975/12000 [1:46:59<5:48:58,  2.32s/it]
                                                        

 25%|██▍       | 2975/12000 [1:46:59<5:48:58,  2.32s/it]
 25%|██▍       | 2976/12000 [1:47:01<5:17:03,  2.11s/it]
                                                        

 25%|██▍       | 2976/12000 [1:47:01<5:17:03,  2.11s/it]
 25%|██▍       | 2977/12000 [1:47:02<4:48:20,  1.92s/it]
                                                        

 25%|██▍       | 2977/12000 [1:47:02<4:48:20,  1.92s/it]
 25%|██▍       | 2978/12000 [1:47:04<4:27:22,  1.78s/it]
                                                        

 25%|██▍       | 2978/12000 [1:47:04<4:27:22,  1.78s/it]
 25%|██▍       | 2979/12000 [1:47:05<4:19:40,  1.73s/it]
                                                        

 25%|██▍       | 2979/12000 [1:47:05<4:19:40,  1.73s/it]
 25%|██▍       | 2980/12000 [1:47:06<3:58:46,  1.59s/it]
                                                        

 25%|██▍       | 2980/12000 [1:47:06<3:58:46,  1.59s/it]
 25%|██▍       | 2981/12000 [1:47:08<4:01:28,  1.61s/it]
                                                        

 25%|██▍       | 2981/12000 [1:47:08<4:01:28,  1.61s/it]
 25%|██▍       | 2982/12000 [1:47:10<4:15:53,  1.70s/it]
                                                        

 25%|██▍       | 2982/12000 [1:47:10<4:15:53,  1.70s/it]
 25%|██▍       | 2983/12000 [1:47:12<4:12:05,  1.68s/it]
                                                        

 25%|██▍       | 2983/12000 [1:47:12<4:12:05,  1.68s/it]
 25%|██▍       | 2984/12000 [1:47:13<4:04:24,  1.63s/it]
                                                        

 25%|██▍       | 2984/12000 [1:47:13<4:04:24,  1.63s/it]
 25%|██▍       | 2985/12000 [1:47:15<4:17:05,  1.71s/it]
                                                        

 25%|██▍       | 2985/12000 [1:47:15<4:17:05,  1.71s/it]
 25%|██▍       | 2986/12000 [1:47:17<4:42:03,  1.88s/it]
                                                        

 25%|██▍       | 2986/12000 [1:47:17<4:42:03,  1.88s/it]
 25%|██▍       | 2987/12000 [1:47:21<6:09:13,  2.46s/it]
                                                        

 25%|██▍       | 2987/12000 [1:47:21<6:09:13,  2.46s/it]
 25%|██▍       | 2988/12000 [1:47:23<5:36:42,  2.24s/it]
                                                        

 25%|██▍       | 2988/12000 [1:47:23<5:36:42,  2.24s/it]
 25%|██▍       | 2989/12000 [1:47:25<5:26:43,  2.18s/it]
                                                        

 25%|██▍       | 2989/12000 [1:47:25<5:26:43,  2.18s/it]
 25%|██▍       | 2990/12000 [1:47:26<5:02:15,  2.01s/it]
                                                        

 25%|██▍       | 2990/12000 [1:47:26<5:02:15,  2.01s/it]
 25%|██▍       | 2991/12000 [1:47:28<4:45:56,  1.90s/it]
                                                        

 25%|██▍       | 2991/12000 [1:47:28<4:45:56,  1.90s/it]
 25%|██▍       | 2992/12000 [1:47:30<4:32:21,  1.81s/it]
                                                        

 25%|██▍       | 2992/12000 [1:47:30<4:32:21,  1.81s/it]
 25%|██▍       | 2993/12000 [1:47:34<6:39:04,  2.66s/it]
                                                        

 25%|██▍       | 2993/12000 [1:47:34<6:39:04,  2.66s/it]
 25%|██▍       | 2994/12000 [1:47:36<6:07:18,  2.45s/it]
                                                        

 25%|██▍       | 2994/12000 [1:47:36<6:07:18,  2.45s/it]
 25%|██▍       | 2995/12000 [1:47:39<6:23:11,  2.55s/it]
                                                        

 25%|██▍       | 2995/12000 [1:47:39<6:23:11,  2.55s/it]
 25%|██▍       | 2996/12000 [1:47:42<6:15:47,  2.50s/it]
                                                        

 25%|██▍       | 2996/12000 [1:47:42<6:15:47,  2.50s/it]
 25%|██▍       | 2997/12000 [1:47:44<6:15:21,  2.50s/it]
                                                        

 25%|██▍       | 2997/12000 [1:47:44<6:15:21,  2.50s/it]
 25%|██▍       | 2998/12000 [1:47:46<6:04:52,  2.43s/it]
                                                        

 25%|██▍       | 2998/12000 [1:47:46<6:04:52,  2.43s/it]
 25%|██▍       | 2999/12000 [1:47:48<5:39:45,  2.26s/it]
                                                        

 25%|██▍       | 2999/12000 [1:47:48<5:39:45,  2.26s/it]
 25%|██▌       | 3000/12000 [1:47:50<5:01:20,  2.01s/it]
                                                        

 25%|██▌       | 3000/12000 [1:47:50<5:01:20,  2.01s/it]
 25%|██▌       | 3001/12000 [1:47:53<5:52:19,  2.35s/it]
                                                        

 25%|██▌       | 3001/12000 [1:47:53<5:52:19,  2.35s/it]
 25%|██▌       | 3002/12000 [1:47:54<5:12:53,  2.09s/it]
                                                        

 25%|██▌       | 3002/12000 [1:47:54<5:12:53,  2.09s/it]
 25%|██▌       | 3003/12000 [1:47:56<4:51:09,  1.94s/it]
                                                        

 25%|██▌       | 3003/12000 [1:47:56<4:51:09,  1.94s/it]
 25%|██▌       | 3004/12000 [1:47:59<5:49:02,  2.33s/it]
                                                        

 25%|██▌       | 3004/12000 [1:47:59<5:49:02,  2.33s/it]
 25%|██▌       | 3005/12000 [1:48:01<5:24:10,  2.16s/it]
                                                        

 25%|██▌       | 3005/12000 [1:48:01<5:24:10,  2.16s/it]
 25%|██▌       | 3006/12000 [1:48:04<6:04:16,  2.43s/it]
                                                        

 25%|██▌       | 3006/12000 [1:48:04<6:04:16,  2.43s/it]
 25%|██▌       | 3007/12000 [1:48:07<6:54:22,  2.76s/it]
                                                        

 25%|██▌       | 3007/12000 [1:48:07<6:54:22,  2.76s/it]
 25%|██▌       | 3008/12000 [1:48:09<6:02:59,  2.42s/it]
                                                        

 25%|██▌       | 3008/12000 [1:48:09<6:02:59,  2.42s/it]
 25%|██▌       | 3009/12000 [1:48:13<7:13:56,  2.90s/it]
                                                        

 25%|██▌       | 3009/12000 [1:48:13<7:13:56,  2.90s/it]
 25%|██▌       | 3010/12000 [1:48:16<7:06:27,  2.85s/it]
                                                        

 25%|██▌       | 3010/12000 [1:48:16<7:06:27,  2.85s/it]
 25%|██▌       | 3011/12000 [1:48:18<7:00:04,  2.80s/it]
                                                        

 25%|██▌       | 3011/12000 [1:48:18<7:00:04,  2.80s/it]
 25%|██▌       | 3012/12000 [1:48:20<6:08:10,  2.46s/it]
                                                        
{'loss': '0.7258', 'grad_norm': '0.3854', 'learning_rate': '0.0001795', 'entropy': '0.7058', 'num_tokens': '1.608e+07', 'mean_token_accuracy': '0.8155', 'epoch': '0.02624'}
{'loss': '0.8842', 'grad_norm': '0.3499', 'learning_rate': '0.0001795', 'entropy': '1.043', 'num_tokens': '1.609e+07', 'mean_token_accuracy': '0.7577', 'epoch': '0.02625'}
{'loss': '0.9089', 'grad_norm': '0.3622', 'learning_rate': '0.0001795', 'entropy': '0.9479', 'num_tokens': '1.609e+07', 'mean_token_accuracy': '0.771', 'epoch': '0.02626'}
{'loss': '0.8243', 'grad_norm': '0.4285', 'learning_rate': '0.0001795', 'entropy': '0.8643', 'num_tokens': '1.61e+07', 'mean_token_accuracy': '0.7766', 'epoch': '0.02627'}
{'loss': '1.603', 'grad_norm': '0.4522', 'learning_rate': '0.0001795', 'entropy': '1.346', 'num_tokens': '1.61e+07', 'mean_token_accuracy': '0.6858', 'epoch': '0.02628'}
{'loss': '0.7951', 'grad_norm': '0.4684', 'learning_rate': '0.0001794', 'entropy': '0.8789', 'num_tokens': '1.611e+07', 'mean_token_accuracy': '0.7911', 'epoch': '0.02628'}
{'loss': '0.7752', 'grad_norm': '0.3921', 'learning_rate': '0.0001794', 'entropy': '0.8241', 'num_tokens': '1.611e+07', 'mean_token_accuracy': '0.7949', 'epoch': '0.02629'}
{'loss': '1.336', 'grad_norm': '0.3473', 'learning_rate': '0.0001794', 'entropy': '1.43', 'num_tokens': '1.612e+07', 'mean_token_accuracy': '0.6546', 'epoch': '0.0263'}
{'loss': '1.053', 'grad_norm': '0.4581', 'learning_rate': '0.0001794', 'entropy': '1.041', 'num_tokens': '1.613e+07', 'mean_token_accuracy': '0.7532', 'epoch': '0.02631'}
{'loss': '0.7047', 'grad_norm': '0.3517', 'learning_rate': '0.0001794', 'entropy': '0.74', 'num_tokens': '1.613e+07', 'mean_token_accuracy': '0.8183', 'epoch': '0.02632'}
{'loss': '1.094', 'grad_norm': '0.344', 'learning_rate': '0.0001794', 'entropy': '0.9329', 'num_tokens': '1.614e+07', 'mean_token_accuracy': '0.7506', 'epoch': '0.02633'}
{'loss': '0.7219', 'grad_norm': '0.3766', 'learning_rate': '0.0001793', 'entropy': '0.8369', 'num_tokens': '1.614e+07', 'mean_token_accuracy': '0.7866', 'epoch': '0.02634'}
{'loss': '1.277', 'grad_norm': '0.4351', 'learning_rate': '0.0001793', 'entropy': '1.346', 'num_tokens': '1.615e+07', 'mean_token_accuracy': '0.6971', 'epoch': '0.02635'}
{'loss': '0.9122', 'grad_norm': '0.3392', 'learning_rate': '0.0001793', 'entropy': '0.8865', 'num_tokens': '1.615e+07', 'mean_token_accuracy': '0.7853', 'epoch': '0.02635'}
{'loss': '0.7638', 'grad_norm': '0.3826', 'learning_rate': '0.0001793', 'entropy': '0.8512', 'num_tokens': '1.616e+07', 'mean_token_accuracy': '0.7864', 'epoch': '0.02636'}
{'loss': '0.9391', 'grad_norm': '0.3806', 'learning_rate': '0.0001793', 'entropy': '1.003', 'num_tokens': '1.616e+07', 'mean_token_accuracy': '0.7548', 'epoch': '0.02637'}
{'loss': '1.045', 'grad_norm': '0.4544', 'learning_rate': '0.0001793', 'entropy': '1.075', 'num_tokens': '1.617e+07', 'mean_token_accuracy': '0.7469', 'epoch': '0.02638'}
{'loss': '0.7519', 'grad_norm': '0.3137', 'learning_rate': '0.0001792', 'entropy': '0.7237', 'num_tokens': '1.617e+07', 'mean_token_accuracy': '0.8132', 'epoch': '0.02639'}
{'loss': '0.7998', 'grad_norm': '0.3463', 'learning_rate': '0.0001792', 'entropy': '0.7775', 'num_tokens': '1.618e+07', 'mean_token_accuracy': '0.8021', 'epoch': '0.0264'}
{'loss': '0.9126', 'grad_norm': '0.3106', 'learning_rate': '0.0001792', 'entropy': '0.8411', 'num_tokens': '1.618e+07', 'mean_token_accuracy': '0.7945', 'epoch': '0.02641'}
{'loss': '0.9711', 'grad_norm': '0.3861', 'learning_rate': '0.0001792', 'entropy': '0.9037', 'num_tokens': '1.619e+07', 'mean_token_accuracy': '0.7718', 'epoch': '0.02642'}
{'loss': '0.9527', 'grad_norm': '0.4365', 'learning_rate': '0.0001792', 'entropy': '0.8695', 'num_tokens': '1.619e+07', 'mean_token_accuracy': '0.7901', 'epoch': '0.02643'}
{'loss': '0.9058', 'grad_norm': '0.4064', 'learning_rate': '0.0001792', 'entropy': '0.8571', 'num_tokens': '1.62e+07', 'mean_token_accuracy': '0.7828', 'epoch': '0.02643'}
{'loss': '0.8885', 'grad_norm': '0.4501', 'learning_rate': '0.0001791', 'entropy': '0.8999', 'num_tokens': '1.621e+07', 'mean_token_accuracy': '0.7678', 'epoch': '0.02644'}
{'loss': '0.9003', 'grad_norm': '0.4144', 'learning_rate': '0.0001791', 'entropy': '0.86', 'num_tokens': '1.621e+07', 'mean_token_accuracy': '0.794', 'epoch': '0.02645'}
{'loss': '0.7293', 'grad_norm': '0.4325', 'learning_rate': '0.0001791', 'entropy': '0.7779', 'num_tokens': '1.622e+07', 'mean_token_accuracy': '0.8141', 'epoch': '0.02646'}
{'loss': '1.047', 'grad_norm': '0.4031', 'learning_rate': '0.0001791', 'entropy': '0.9167', 'num_tokens': '1.622e+07', 'mean_token_accuracy': '0.7826', 'epoch': '0.02647'}
{'loss': '0.9584', 'grad_norm': '0.4513', 'learning_rate': '0.0001791', 'entropy': '1.017', 'num_tokens': '1.622e+07', 'mean_token_accuracy': '0.7637', 'epoch': '0.02648'}
{'loss': '1.274', 'grad_norm': '0.414', 'learning_rate': '0.000179', 'entropy': '1.004', 'num_tokens': '1.623e+07', 'mean_token_accuracy': '0.7665', 'epoch': '0.02649'}
{'loss': '1.141', 'grad_norm': '0.3782', 'learning_rate': '0.000179', 'entropy': '1.076', 'num_tokens': '1.624e+07', 'mean_token_accuracy': '0.7284', 'epoch': '0.0265'}
{'loss': '0.9806', 'grad_norm': '0.3737', 'learning_rate': '0.000179', 'entropy': '0.9697', 'num_tokens': '1.624e+07', 'mean_token_accuracy': '0.7657', 'epoch': '0.02651'}
{'loss': '0.987', 'grad_norm': '0.3774', 'learning_rate': '0.000179', 'entropy': '0.9095', 'num_tokens': '1.625e+07', 'mean_token_accuracy': '0.7837', 'epoch': '0.02651'}
{'loss': '0.7584', 'grad_norm': '0.3643', 'learning_rate': '0.000179', 'entropy': '0.837', 'num_tokens': '1.626e+07', 'mean_token_accuracy': '0.7897', 'epoch': '0.02652'}
{'loss': '0.7616', 'grad_norm': '0.3041', 'learning_rate': '0.000179', 'entropy': '0.7583', 'num_tokens': '1.626e+07', 'mean_token_accuracy': '0.7985', 'epoch': '0.02653'}
{'loss': '0.7687', 'grad_norm': '0.3686', 'learning_rate': '0.0001789', 'entropy': '0.771', 'num_tokens': '1.627e+07', 'mean_token_accuracy': '0.8132', 'epoch': '0.02654'}
{'loss': '0.6843', 'grad_norm': '0.3057', 'learning_rate': '0.0001789', 'entropy': '0.6769', 'num_tokens': '1.627e+07', 'mean_token_accuracy': '0.8308', 'epoch': '0.02655'}
{'loss': '0.9812', 'grad_norm': '0.3599', 'learning_rate': '0.0001789', 'entropy': '0.9943', 'num_tokens': '1.628e+07', 'mean_token_accuracy': '0.7581', 'epoch': '0.02656'}
{'loss': '0.9624', 'grad_norm': '0.4315', 'learning_rate': '0.0001789', 'entropy': '0.9812', 'num_tokens': '1.628e+07', 'mean_token_accuracy': '0.7389', 'epoch': '0.02657'}
{'loss': '0.9112', 'grad_norm': '0.4088', 'learning_rate': '0.0001789', 'entropy': '0.9123', 'num_tokens': '1.629e+07', 'mean_token_accuracy': '0.7546', 'epoch': '0.02658'}
{'loss': '0.9376', 'grad_norm': '0.3198', 'learning_rate': '0.0001789', 'entropy': '0.9118', 'num_tokens': '1.63e+07', 'mean_token_accuracy': '0.7795', 'epoch': '0.02658'}
{'loss': '0.7678', 'grad_norm': '0.3845', 'learning_rate': '0.0001788', 'entropy': '0.707', 'num_tokens': '1.63e+07', 'mean_token_accuracy': '0.8283', 'epoch': '0.02659'}
{'loss': '1.003', 'grad_norm': '0.322', 'learning_rate': '0.0001788', 'entropy': '0.9224', 'num_tokens': '1.631e+07', 'mean_token_accuracy': '0.7548', 'epoch': '0.0266'}
{'loss': '0.8863', 'grad_norm': '0.3241', 'learning_rate': '0.0001788', 'entropy': '0.9229', 'num_tokens': '1.632e+07', 'mean_token_accuracy': '0.7739', 'epoch': '0.02661'}
{'loss': '0.748', 'grad_norm': '0.4743', 'learning_rate': '0.0001788', 'entropy': '0.8141', 'num_tokens': '1.632e+07', 'mean_token_accuracy': '0.7937', 'epoch': '0.02662'}
{'loss': '0.7237', 'grad_norm': '0.325', 'learning_rate': '0.0001788', 'entropy': '0.7259', 'num_tokens': '1.633e+07', 'mean_token_accuracy': '0.8084', 'epoch': '0.02663'}
{'loss': '0.6075', 'grad_norm': '0.334', 'learning_rate': '0.0001788', 'entropy': '0.6826', 'num_tokens': '1.633e+07', 'mean_token_accuracy': '0.8326', 'epoch': '0.02664'}
{'loss': '0.9375', 'grad_norm': '0.3645', 'learning_rate': '0.0001787', 'entropy': '0.9445', 'num_tokens': '1.634e+07', 'mean_token_accuracy': '0.7745', 'epoch': '0.02665'}

 25%|██▌       | 3012/12000 [1:48:20<6:08:10,  2.46s/it]
 25%|██▌       | 3013/12000 [1:48:21<5:18:36,  2.13s/it]
                                                        

 25%|██▌       | 3013/12000 [1:48:21<5:18:36,  2.13s/it]
 25%|██▌       | 3014/12000 [1:48:23<4:47:01,  1.92s/it]
                                                        

 25%|██▌       | 3014/12000 [1:48:23<4:47:01,  1.92s/it]
 25%|██▌       | 3015/12000 [1:48:24<4:27:32,  1.79s/it]
                                                        

 25%|██▌       | 3015/12000 [1:48:24<4:27:32,  1.79s/it]
 25%|██▌       | 3016/12000 [1:48:26<4:21:55,  1.75s/it]
                                                        

 25%|██▌       | 3016/12000 [1:48:26<4:21:55,  1.75s/it]
 25%|██▌       | 3017/12000 [1:48:27<4:08:29,  1.66s/it]
                                                        

 25%|██▌       | 3017/12000 [1:48:27<4:08:29,  1.66s/it]
 25%|██▌       | 3018/12000 [1:48:30<4:43:57,  1.90s/it]
                                                        

 25%|██▌       | 3018/12000 [1:48:30<4:43:57,  1.90s/it]
 25%|██▌       | 3019/12000 [1:48:31<4:24:51,  1.77s/it]
                                                        

 25%|██▌       | 3019/12000 [1:48:31<4:24:51,  1.77s/it]
 25%|██▌       | 3020/12000 [1:48:33<4:07:15,  1.65s/it]
                                                        

 25%|██▌       | 3020/12000 [1:48:33<4:07:15,  1.65s/it]
 25%|██▌       | 3021/12000 [1:48:36<4:55:55,  1.98s/it]
                                                        

 25%|██▌       | 3021/12000 [1:48:36<4:55:55,  1.98s/it]
 25%|██▌       | 3022/12000 [1:48:37<4:47:04,  1.92s/it]
                                                        

 25%|██▌       | 3022/12000 [1:48:37<4:47:04,  1.92s/it]
 25%|██▌       | 3023/12000 [1:48:39<4:58:39,  2.00s/it]
                                                        

 25%|██▌       | 3023/12000 [1:48:39<4:58:39,  2.00s/it]
 25%|██▌       | 3024/12000 [1:48:41<4:38:06,  1.86s/it]
                                                        

 25%|██▌       | 3024/12000 [1:48:41<4:38:06,  1.86s/it]
 25%|██▌       | 3025/12000 [1:48:42<4:21:07,  1.75s/it]
                                                        

 25%|██▌       | 3025/12000 [1:48:42<4:21:07,  1.75s/it]
 25%|██▌       | 3026/12000 [1:48:44<4:05:42,  1.64s/it]
                                                        

 25%|██▌       | 3026/12000 [1:48:44<4:05:42,  1.64s/it]
 25%|██▌       | 3027/12000 [1:48:45<3:55:34,  1.58s/it]
                                                        

 25%|██▌       | 3027/12000 [1:48:45<3:55:34,  1.58s/it]
 25%|██▌       | 3028/12000 [1:48:47<3:49:33,  1.54s/it]
                                                        

 25%|██▌       | 3028/12000 [1:48:47<3:49:33,  1.54s/it]
 25%|██▌       | 3029/12000 [1:48:52<6:16:35,  2.52s/it]
                                                        

 25%|██▌       | 3029/12000 [1:48:52<6:16:35,  2.52s/it]
 25%|██▌       | 3030/12000 [1:48:53<5:33:22,  2.23s/it]
                                                        

 25%|██▌       | 3030/12000 [1:48:53<5:33:22,  2.23s/it]
 25%|██▌       | 3031/12000 [1:48:57<6:34:10,  2.64s/it]
                                                        

 25%|██▌       | 3031/12000 [1:48:57<6:34:10,  2.64s/it]
 25%|██▌       | 3032/12000 [1:48:58<5:51:23,  2.35s/it]
                                                        

 25%|██▌       | 3032/12000 [1:48:58<5:51:23,  2.35s/it]
 25%|██▌       | 3033/12000 [1:49:01<5:51:34,  2.35s/it]
                                                        

 25%|██▌       | 3033/12000 [1:49:01<5:51:34,  2.35s/it]
 25%|██▌       | 3034/12000 [1:49:03<5:31:17,  2.22s/it]
                                                        

 25%|██▌       | 3034/12000 [1:49:03<5:31:17,  2.22s/it]
 25%|██▌       | 3035/12000 [1:49:06<6:04:12,  2.44s/it]
                                                        

 25%|██▌       | 3035/12000 [1:49:06<6:04:12,  2.44s/it]
 25%|██▌       | 3036/12000 [1:49:07<5:35:53,  2.25s/it]
                                                        

 25%|██▌       | 3036/12000 [1:49:07<5:35:53,  2.25s/it]
 25%|██▌       | 3037/12000 [1:49:09<5:18:12,  2.13s/it]
                                                        

 25%|██▌       | 3037/12000 [1:49:09<5:18:12,  2.13s/it]
 25%|██▌       | 3038/12000 [1:49:11<5:03:40,  2.03s/it]
                                                        

 25%|██▌       | 3038/12000 [1:49:11<5:03:40,  2.03s/it]
 25%|██▌       | 3039/12000 [1:49:13<5:03:59,  2.04s/it]
                                                        

 25%|██▌       | 3039/12000 [1:49:13<5:03:59,  2.04s/it]
 25%|██▌       | 3040/12000 [1:49:15<4:49:22,  1.94s/it]
                                                        

 25%|██▌       | 3040/12000 [1:49:15<4:49:22,  1.94s/it]
 25%|██▌       | 3041/12000 [1:49:17<4:44:22,  1.90s/it]
                                                        

 25%|██▌       | 3041/12000 [1:49:17<4:44:22,  1.90s/it]
 25%|██▌       | 3042/12000 [1:49:19<5:22:08,  2.16s/it]
                                                        

 25%|██▌       | 3042/12000 [1:49:19<5:22:08,  2.16s/it]
 25%|██▌       | 3043/12000 [1:49:21<5:06:50,  2.06s/it]
                                                        

 25%|██▌       | 3043/12000 [1:49:21<5:06:50,  2.06s/it]
 25%|██▌       | 3044/12000 [1:49:23<5:13:21,  2.10s/it]
                                                        

 25%|██▌       | 3044/12000 [1:49:23<5:13:21,  2.10s/it]
 25%|██▌       | 3045/12000 [1:49:26<5:18:47,  2.14s/it]
                                                        

 25%|██▌       | 3045/12000 [1:49:26<5:18:47,  2.14s/it]
 25%|██▌       | 3046/12000 [1:49:27<4:54:51,  1.98s/it]
                                                        

 25%|██▌       | 3046/12000 [1:49:27<4:54:51,  1.98s/it]
 25%|██▌       | 3047/12000 [1:49:29<4:41:33,  1.89s/it]
                                                        

 25%|██▌       | 3047/12000 [1:49:29<4:41:33,  1.89s/it]
 25%|██▌       | 3048/12000 [1:49:31<4:47:16,  1.93s/it]
                                                        

 25%|██▌       | 3048/12000 [1:49:31<4:47:16,  1.93s/it]
 25%|██▌       | 3049/12000 [1:49:33<4:59:45,  2.01s/it]
                                                        

 25%|██▌       | 3049/12000 [1:49:33<4:59:45,  2.01s/it]
 25%|██▌       | 3050/12000 [1:49:37<6:12:56,  2.50s/it]
                                                        

 25%|██▌       | 3050/12000 [1:49:37<6:12:56,  2.50s/it]
 25%|██▌       | 3051/12000 [1:49:40<6:46:00,  2.72s/it]
                                                        

 25%|██▌       | 3051/12000 [1:49:40<6:46:00,  2.72s/it]
 25%|██▌       | 3052/12000 [1:49:42<6:24:42,  2.58s/it]
                                                        

 25%|██▌       | 3052/12000 [1:49:42<6:24:42,  2.58s/it]
 25%|██▌       | 3053/12000 [1:49:47<7:48:15,  3.14s/it]
                                                        

 25%|██▌       | 3053/12000 [1:49:47<7:48:15,  3.14s/it]
 25%|██▌       | 3054/12000 [1:49:49<6:52:01,  2.76s/it]
                                                        

 25%|██▌       | 3054/12000 [1:49:49<6:52:01,  2.76s/it]
 25%|██▌       | 3055/12000 [1:49:50<6:10:57,  2.49s/it]
                                                        

 25%|██▌       | 3055/12000 [1:49:50<6:10:57,  2.49s/it]
 25%|██▌       | 3056/12000 [1:49:53<5:53:28,  2.37s/it]
                                                        

 25%|██▌       | 3056/12000 [1:49:53<5:53:28,  2.37s/it]
 25%|██▌       | 3057/12000 [1:49:54<5:30:34,  2.22s/it]
                                                        

 25%|██▌       | 3057/12000 [1:49:54<5:30:34,  2.22s/it]
 25%|██▌       | 3058/12000 [1:49:56<5:05:46,  2.05s/it]
                                                        

 25%|██▌       | 3058/12000 [1:49:56<5:05:46,  2.05s/it]
 25%|██▌       | 3059/12000 [1:49:58<5:00:22,  2.02s/it]
                                                        
{'loss': '0.633', 'grad_norm': '0.4033', 'learning_rate': '0.0001787', 'entropy': '0.6164', 'num_tokens': '1.635e+07', 'mean_token_accuracy': '0.8383', 'epoch': '0.02666'}
{'loss': '1.02', 'grad_norm': '0.3668', 'learning_rate': '0.0001787', 'entropy': '1.107', 'num_tokens': '1.635e+07', 'mean_token_accuracy': '0.7192', 'epoch': '0.02666'}
{'loss': '0.6789', 'grad_norm': '0.3593', 'learning_rate': '0.0001787', 'entropy': '0.7019', 'num_tokens': '1.635e+07', 'mean_token_accuracy': '0.8177', 'epoch': '0.02667'}
{'loss': '0.6483', 'grad_norm': '0.292', 'learning_rate': '0.0001787', 'entropy': '0.7458', 'num_tokens': '1.636e+07', 'mean_token_accuracy': '0.7958', 'epoch': '0.02668'}
{'loss': '1.028', 'grad_norm': '0.4141', 'learning_rate': '0.0001787', 'entropy': '1.112', 'num_tokens': '1.636e+07', 'mean_token_accuracy': '0.7267', 'epoch': '0.02669'}
{'loss': '1.143', 'grad_norm': '0.4821', 'learning_rate': '0.0001786', 'entropy': '1.237', 'num_tokens': '1.637e+07', 'mean_token_accuracy': '0.6994', 'epoch': '0.0267'}
{'loss': '0.8594', 'grad_norm': '0.3921', 'learning_rate': '0.0001786', 'entropy': '0.8672', 'num_tokens': '1.637e+07', 'mean_token_accuracy': '0.7903', 'epoch': '0.02671'}
{'loss': '0.6966', 'grad_norm': '0.3799', 'learning_rate': '0.0001786', 'entropy': '0.7502', 'num_tokens': '1.638e+07', 'mean_token_accuracy': '0.8055', 'epoch': '0.02672'}
{'loss': '0.754', 'grad_norm': '0.3731', 'learning_rate': '0.0001786', 'entropy': '0.8135', 'num_tokens': '1.638e+07', 'mean_token_accuracy': '0.8077', 'epoch': '0.02673'}
{'loss': '0.5701', 'grad_norm': '0.3331', 'learning_rate': '0.0001786', 'entropy': '0.6062', 'num_tokens': '1.639e+07', 'mean_token_accuracy': '0.8349', 'epoch': '0.02674'}
{'loss': '0.8465', 'grad_norm': '0.2893', 'learning_rate': '0.0001786', 'entropy': '0.8407', 'num_tokens': '1.639e+07', 'mean_token_accuracy': '0.7868', 'epoch': '0.02674'}
{'loss': '0.7501', 'grad_norm': '0.4193', 'learning_rate': '0.0001785', 'entropy': '0.7372', 'num_tokens': '1.64e+07', 'mean_token_accuracy': '0.8083', 'epoch': '0.02675'}
{'loss': '0.825', 'grad_norm': '0.4579', 'learning_rate': '0.0001785', 'entropy': '0.8495', 'num_tokens': '1.64e+07', 'mean_token_accuracy': '0.7852', 'epoch': '0.02676'}
{'loss': '0.6653', 'grad_norm': '0.3599', 'learning_rate': '0.0001785', 'entropy': '0.6706', 'num_tokens': '1.641e+07', 'mean_token_accuracy': '0.8163', 'epoch': '0.02677'}
{'loss': '1.083', 'grad_norm': '0.3628', 'learning_rate': '0.0001785', 'entropy': '1.086', 'num_tokens': '1.641e+07', 'mean_token_accuracy': '0.7472', 'epoch': '0.02678'}
{'loss': '0.8904', 'grad_norm': '0.4167', 'learning_rate': '0.0001785', 'entropy': '0.8864', 'num_tokens': '1.642e+07', 'mean_token_accuracy': '0.7811', 'epoch': '0.02679'}
{'loss': '0.8557', 'grad_norm': '0.3349', 'learning_rate': '0.0001785', 'entropy': '0.8981', 'num_tokens': '1.642e+07', 'mean_token_accuracy': '0.7843', 'epoch': '0.0268'}
{'loss': '1.249', 'grad_norm': '0.3994', 'learning_rate': '0.0001784', 'entropy': '0.9619', 'num_tokens': '1.643e+07', 'mean_token_accuracy': '0.7697', 'epoch': '0.02681'}
{'loss': '1.077', 'grad_norm': '0.4854', 'learning_rate': '0.0001784', 'entropy': '1.036', 'num_tokens': '1.643e+07', 'mean_token_accuracy': '0.7377', 'epoch': '0.02681'}
{'loss': '0.9506', 'grad_norm': '0.4722', 'learning_rate': '0.0001784', 'entropy': '0.9402', 'num_tokens': '1.644e+07', 'mean_token_accuracy': '0.7641', 'epoch': '0.02682'}
{'loss': '0.841', 'grad_norm': '0.3948', 'learning_rate': '0.0001784', 'entropy': '0.921', 'num_tokens': '1.644e+07', 'mean_token_accuracy': '0.7746', 'epoch': '0.02683'}
{'loss': '1.372', 'grad_norm': '0.4511', 'learning_rate': '0.0001784', 'entropy': '1.398', 'num_tokens': '1.645e+07', 'mean_token_accuracy': '0.675', 'epoch': '0.02684'}
{'loss': '0.7722', 'grad_norm': '0.313', 'learning_rate': '0.0001784', 'entropy': '0.9509', 'num_tokens': '1.645e+07', 'mean_token_accuracy': '0.7778', 'epoch': '0.02685'}
{'loss': '0.7982', 'grad_norm': '0.2915', 'learning_rate': '0.0001783', 'entropy': '0.908', 'num_tokens': '1.646e+07', 'mean_token_accuracy': '0.7806', 'epoch': '0.02686'}
{'loss': '0.9743', 'grad_norm': '0.4002', 'learning_rate': '0.0001783', 'entropy': '1.008', 'num_tokens': '1.647e+07', 'mean_token_accuracy': '0.7419', 'epoch': '0.02687'}
{'loss': '1.059', 'grad_norm': '0.373', 'learning_rate': '0.0001783', 'entropy': '1.053', 'num_tokens': '1.647e+07', 'mean_token_accuracy': '0.7468', 'epoch': '0.02688'}
{'loss': '1.145', 'grad_norm': '0.4199', 'learning_rate': '0.0001783', 'entropy': '1.103', 'num_tokens': '1.647e+07', 'mean_token_accuracy': '0.7281', 'epoch': '0.02689'}
{'loss': '1.266', 'grad_norm': '0.416', 'learning_rate': '0.0001783', 'entropy': '1.223', 'num_tokens': '1.648e+07', 'mean_token_accuracy': '0.6998', 'epoch': '0.02689'}
{'loss': '0.6721', 'grad_norm': '0.3517', 'learning_rate': '0.0001782', 'entropy': '0.7058', 'num_tokens': '1.648e+07', 'mean_token_accuracy': '0.8156', 'epoch': '0.0269'}
{'loss': '0.7657', 'grad_norm': '0.4144', 'learning_rate': '0.0001782', 'entropy': '0.719', 'num_tokens': '1.649e+07', 'mean_token_accuracy': '0.8161', 'epoch': '0.02691'}
{'loss': '1.241', 'grad_norm': '0.3645', 'learning_rate': '0.0001782', 'entropy': '1.324', 'num_tokens': '1.65e+07', 'mean_token_accuracy': '0.688', 'epoch': '0.02692'}
{'loss': '0.9605', 'grad_norm': '0.3373', 'learning_rate': '0.0001782', 'entropy': '0.8834', 'num_tokens': '1.65e+07', 'mean_token_accuracy': '0.781', 'epoch': '0.02693'}
{'loss': '1.117', 'grad_norm': '0.4563', 'learning_rate': '0.0001782', 'entropy': '1.136', 'num_tokens': '1.651e+07', 'mean_token_accuracy': '0.732', 'epoch': '0.02694'}
{'loss': '0.7107', 'grad_norm': '0.3526', 'learning_rate': '0.0001782', 'entropy': '0.7619', 'num_tokens': '1.651e+07', 'mean_token_accuracy': '0.8136', 'epoch': '0.02695'}
{'loss': '0.6486', 'grad_norm': '0.2995', 'learning_rate': '0.0001781', 'entropy': '0.6471', 'num_tokens': '1.652e+07', 'mean_token_accuracy': '0.8334', 'epoch': '0.02696'}
{'loss': '1.19', 'grad_norm': '0.5123', 'learning_rate': '0.0001781', 'entropy': '1.178', 'num_tokens': '1.652e+07', 'mean_token_accuracy': '0.7041', 'epoch': '0.02697'}
{'loss': '0.9561', 'grad_norm': '0.4889', 'learning_rate': '0.0001781', 'entropy': '0.9134', 'num_tokens': '1.653e+07', 'mean_token_accuracy': '0.7718', 'epoch': '0.02697'}
{'loss': '1.045', 'grad_norm': '0.4057', 'learning_rate': '0.0001781', 'entropy': '1.019', 'num_tokens': '1.653e+07', 'mean_token_accuracy': '0.7644', 'epoch': '0.02698'}
{'loss': '0.9105', 'grad_norm': '0.3193', 'learning_rate': '0.0001781', 'entropy': '0.9503', 'num_tokens': '1.654e+07', 'mean_token_accuracy': '0.7659', 'epoch': '0.02699'}
{'loss': '0.8793', 'grad_norm': '0.5288', 'learning_rate': '0.0001781', 'entropy': '0.856', 'num_tokens': '1.654e+07', 'mean_token_accuracy': '0.7723', 'epoch': '0.027'}
{'loss': '0.8813', 'grad_norm': '0.4363', 'learning_rate': '0.000178', 'entropy': '0.8171', 'num_tokens': '1.655e+07', 'mean_token_accuracy': '0.7945', 'epoch': '0.02701'}
{'loss': '0.8691', 'grad_norm': '0.3202', 'learning_rate': '0.000178', 'entropy': '0.8576', 'num_tokens': '1.656e+07', 'mean_token_accuracy': '0.7967', 'epoch': '0.02702'}
{'loss': '1.104', 'grad_norm': '0.4088', 'learning_rate': '0.000178', 'entropy': '0.9223', 'num_tokens': '1.656e+07', 'mean_token_accuracy': '0.7627', 'epoch': '0.02703'}
{'loss': '0.8728', 'grad_norm': '0.4019', 'learning_rate': '0.000178', 'entropy': '0.887', 'num_tokens': '1.657e+07', 'mean_token_accuracy': '0.7669', 'epoch': '0.02704'}
{'loss': '0.7669', 'grad_norm': '0.387', 'learning_rate': '0.000178', 'entropy': '0.7827', 'num_tokens': '1.657e+07', 'mean_token_accuracy': '0.7994', 'epoch': '0.02705'}
{'loss': '0.7897', 'grad_norm': '0.4609', 'learning_rate': '0.000178', 'entropy': '0.8101', 'num_tokens': '1.658e+07', 'mean_token_accuracy': '0.8015', 'epoch': '0.02705'}
{'loss': '0.7572', 'grad_norm': '0.4291', 'learning_rate': '0.0001779', 'entropy': '0.8001', 'num_tokens': '1.658e+07', 'mean_token_accuracy': '0.7946', 'epoch': '0.02706'}

 25%|██▌       | 3059/12000 [1:49:58<5:00:22,  2.02s/it]
 26%|██▌       | 3060/12000 [1:49:59<4:36:02,  1.85s/it]
                                                        

 26%|██▌       | 3060/12000 [1:49:59<4:36:02,  1.85s/it]
 26%|██▌       | 3061/12000 [1:50:01<4:20:19,  1.75s/it]
                                                        

 26%|██▌       | 3061/12000 [1:50:01<4:20:19,  1.75s/it]
 26%|██▌       | 3062/12000 [1:50:03<4:29:36,  1.81s/it]
                                                        

 26%|██▌       | 3062/12000 [1:50:03<4:29:36,  1.81s/it]
 26%|██▌       | 3063/12000 [1:50:05<4:23:02,  1.77s/it]
                                                        

 26%|██▌       | 3063/12000 [1:50:05<4:23:02,  1.77s/it]
 26%|██▌       | 3064/12000 [1:50:06<4:08:08,  1.67s/it]
                                                        

 26%|██▌       | 3064/12000 [1:50:06<4:08:08,  1.67s/it]
 26%|██▌       | 3065/12000 [1:50:07<3:58:28,  1.60s/it]
                                                        

 26%|██▌       | 3065/12000 [1:50:07<3:58:28,  1.60s/it]
 26%|██▌       | 3066/12000 [1:50:09<4:08:44,  1.67s/it]
                                                        

 26%|██▌       | 3066/12000 [1:50:09<4:08:44,  1.67s/it]
 26%|██▌       | 3067/12000 [1:50:11<4:17:15,  1.73s/it]
                                                        

 26%|██▌       | 3067/12000 [1:50:11<4:17:15,  1.73s/it]
 26%|██▌       | 3068/12000 [1:50:15<5:42:51,  2.30s/it]
                                                        

 26%|██▌       | 3068/12000 [1:50:15<5:42:51,  2.30s/it]
 26%|██▌       | 3069/12000 [1:50:17<5:20:55,  2.16s/it]
                                                        

 26%|██▌       | 3069/12000 [1:50:17<5:20:55,  2.16s/it]
 26%|██▌       | 3070/12000 [1:50:18<4:46:29,  1.92s/it]
                                                        

 26%|██▌       | 3070/12000 [1:50:18<4:46:29,  1.92s/it]
 26%|██▌       | 3071/12000 [1:50:20<4:30:48,  1.82s/it]
                                                        

 26%|██▌       | 3071/12000 [1:50:20<4:30:48,  1.82s/it]
 26%|██▌       | 3072/12000 [1:50:21<4:22:42,  1.77s/it]
                                                        

 26%|██▌       | 3072/12000 [1:50:21<4:22:42,  1.77s/it]
 26%|██▌       | 3073/12000 [1:50:23<4:12:44,  1.70s/it]
                                                        

 26%|██▌       | 3073/12000 [1:50:23<4:12:44,  1.70s/it]
 26%|██▌       | 3074/12000 [1:50:24<4:01:43,  1.62s/it]
                                                        

 26%|██▌       | 3074/12000 [1:50:24<4:01:43,  1.62s/it]
 26%|██▌       | 3075/12000 [1:50:29<6:02:02,  2.43s/it]
                                                        

 26%|██▌       | 3075/12000 [1:50:29<6:02:02,  2.43s/it]
 26%|██▌       | 3076/12000 [1:50:31<6:21:28,  2.56s/it]
                                                        

 26%|██▌       | 3076/12000 [1:50:31<6:21:28,  2.56s/it]
 26%|██▌       | 3077/12000 [1:50:34<6:14:34,  2.52s/it]
                                                        

 26%|██▌       | 3077/12000 [1:50:34<6:14:34,  2.52s/it]
 26%|██▌       | 3078/12000 [1:50:36<5:46:29,  2.33s/it]
                                                        

 26%|██▌       | 3078/12000 [1:50:36<5:46:29,  2.33s/it]
 26%|██▌       | 3079/12000 [1:50:38<5:35:20,  2.26s/it]
                                                        

 26%|██▌       | 3079/12000 [1:50:38<5:35:20,  2.26s/it]
 26%|██▌       | 3080/12000 [1:50:40<5:15:37,  2.12s/it]
                                                        

 26%|██▌       | 3080/12000 [1:50:40<5:15:37,  2.12s/it]
 26%|██▌       | 3081/12000 [1:50:41<4:48:31,  1.94s/it]
                                                        

 26%|██▌       | 3081/12000 [1:50:41<4:48:31,  1.94s/it]
 26%|██▌       | 3082/12000 [1:50:43<4:37:20,  1.87s/it]
                                                        

 26%|██▌       | 3082/12000 [1:50:43<4:37:20,  1.87s/it]
 26%|██▌       | 3083/12000 [1:50:45<5:06:16,  2.06s/it]
                                                        

 26%|██▌       | 3083/12000 [1:50:45<5:06:16,  2.06s/it]
 26%|██▌       | 3084/12000 [1:50:47<4:35:37,  1.85s/it]
                                                        

 26%|██▌       | 3084/12000 [1:50:47<4:35:37,  1.85s/it]
 26%|██▌       | 3085/12000 [1:50:49<5:06:15,  2.06s/it]
                                                        

 26%|██▌       | 3085/12000 [1:50:49<5:06:15,  2.06s/it]
 26%|██▌       | 3086/12000 [1:50:51<4:41:26,  1.89s/it]
                                                        

 26%|██▌       | 3086/12000 [1:50:51<4:41:26,  1.89s/it]
 26%|██▌       | 3087/12000 [1:50:52<4:28:33,  1.81s/it]
                                                        

 26%|██▌       | 3087/12000 [1:50:52<4:28:33,  1.81s/it]
 26%|██▌       | 3088/12000 [1:50:54<4:31:08,  1.83s/it]
                                                        

 26%|██▌       | 3088/12000 [1:50:54<4:31:08,  1.83s/it]
 26%|██▌       | 3089/12000 [1:50:56<4:22:06,  1.76s/it]
                                                        

 26%|██▌       | 3089/12000 [1:50:56<4:22:06,  1.76s/it]
 26%|██▌       | 3090/12000 [1:50:57<4:06:55,  1.66s/it]
                                                        

 26%|██▌       | 3090/12000 [1:50:57<4:06:55,  1.66s/it]
 26%|██▌       | 3091/12000 [1:50:59<4:05:32,  1.65s/it]
                                                        

 26%|██▌       | 3091/12000 [1:50:59<4:05:32,  1.65s/it]
 26%|██▌       | 3092/12000 [1:51:01<4:23:09,  1.77s/it]
                                                        

 26%|██▌       | 3092/12000 [1:51:01<4:23:09,  1.77s/it]
 26%|██▌       | 3093/12000 [1:51:02<4:07:11,  1.67s/it]
                                                        

 26%|██▌       | 3093/12000 [1:51:02<4:07:11,  1.67s/it]
 26%|██▌       | 3094/12000 [1:51:04<4:00:32,  1.62s/it]
                                                        

 26%|██▌       | 3094/12000 [1:51:04<4:00:32,  1.62s/it]
 26%|██▌       | 3095/12000 [1:51:06<4:12:21,  1.70s/it]
                                                        

 26%|██▌       | 3095/12000 [1:51:06<4:12:21,  1.70s/it]
 26%|██▌       | 3096/12000 [1:51:07<4:10:52,  1.69s/it]
                                                        

 26%|██▌       | 3096/12000 [1:51:07<4:10:52,  1.69s/it]
 26%|██▌       | 3097/12000 [1:51:11<5:30:42,  2.23s/it]
                                                        

 26%|██▌       | 3097/12000 [1:51:11<5:30:42,  2.23s/it]
 26%|██▌       | 3098/12000 [1:51:12<4:58:56,  2.01s/it]
                                                        

 26%|██▌       | 3098/12000 [1:51:12<4:58:56,  2.01s/it]
 26%|██▌       | 3099/12000 [1:51:14<4:30:25,  1.82s/it]
                                                        

 26%|██▌       | 3099/12000 [1:51:14<4:30:25,  1.82s/it]
 26%|██▌       | 3100/12000 [1:51:16<4:50:34,  1.96s/it]
                                                        

 26%|██▌       | 3100/12000 [1:51:16<4:50:34,  1.96s/it]
 26%|██▌       | 3101/12000 [1:51:19<5:45:22,  2.33s/it]
                                                        

 26%|██▌       | 3101/12000 [1:51:19<5:45:22,  2.33s/it]
 26%|██▌       | 3102/12000 [1:51:21<5:06:44,  2.07s/it]
                                                        

 26%|██▌       | 3102/12000 [1:51:21<5:06:44,  2.07s/it]
 26%|██▌       | 3103/12000 [1:51:23<5:05:45,  2.06s/it]
                                                        

 26%|██▌       | 3103/12000 [1:51:23<5:05:45,  2.06s/it]
 26%|██▌       | 3104/12000 [1:51:25<4:58:24,  2.01s/it]
                                                        

 26%|██▌       | 3104/12000 [1:51:25<4:58:24,  2.01s/it]
 26%|██▌       | 3105/12000 [1:51:27<5:11:45,  2.10s/it]
                                                        

 26%|██▌       | 3105/12000 [1:51:27<5:11:45,  2.10s/it]
 26%|██▌       | 3106/12000 [1:51:28<4:41:09,  1.90s/it]
                                                        
{'loss': '1.091', 'grad_norm': '0.4187', 'learning_rate': '0.0001779', 'entropy': '1.136', 'num_tokens': '1.659e+07', 'mean_token_accuracy': '0.7255', 'epoch': '0.02707'}
{'loss': '0.6766', 'grad_norm': '0.4193', 'learning_rate': '0.0001779', 'entropy': '0.7239', 'num_tokens': '1.659e+07', 'mean_token_accuracy': '0.8157', 'epoch': '0.02708'}
{'loss': '0.8277', 'grad_norm': '0.3765', 'learning_rate': '0.0001779', 'entropy': '0.8014', 'num_tokens': '1.66e+07', 'mean_token_accuracy': '0.8013', 'epoch': '0.02709'}
{'loss': '0.9862', 'grad_norm': '0.3945', 'learning_rate': '0.0001779', 'entropy': '0.9688', 'num_tokens': '1.66e+07', 'mean_token_accuracy': '0.7759', 'epoch': '0.0271'}
{'loss': '0.9717', 'grad_norm': '0.4953', 'learning_rate': '0.0001779', 'entropy': '1.043', 'num_tokens': '1.661e+07', 'mean_token_accuracy': '0.7461', 'epoch': '0.02711'}
{'loss': '1.395', 'grad_norm': '0.4698', 'learning_rate': '0.0001778', 'entropy': '1.351', 'num_tokens': '1.661e+07', 'mean_token_accuracy': '0.6914', 'epoch': '0.02712'}
{'loss': '0.7575', 'grad_norm': '0.3821', 'learning_rate': '0.0001778', 'entropy': '0.7852', 'num_tokens': '1.662e+07', 'mean_token_accuracy': '0.8107', 'epoch': '0.02712'}
{'loss': '0.8424', 'grad_norm': '0.3611', 'learning_rate': '0.0001778', 'entropy': '0.8622', 'num_tokens': '1.662e+07', 'mean_token_accuracy': '0.7909', 'epoch': '0.02713'}
{'loss': '0.6853', 'grad_norm': '0.3117', 'learning_rate': '0.0001778', 'entropy': '0.6783', 'num_tokens': '1.663e+07', 'mean_token_accuracy': '0.8224', 'epoch': '0.02714'}
{'loss': '1.144', 'grad_norm': '0.4833', 'learning_rate': '0.0001778', 'entropy': '1.063', 'num_tokens': '1.664e+07', 'mean_token_accuracy': '0.7376', 'epoch': '0.02715'}
{'loss': '0.6544', 'grad_norm': '0.97', 'learning_rate': '0.0001777', 'entropy': '0.6546', 'num_tokens': '1.664e+07', 'mean_token_accuracy': '0.8256', 'epoch': '0.02716'}
{'loss': '0.938', 'grad_norm': '0.389', 'learning_rate': '0.0001777', 'entropy': '0.8965', 'num_tokens': '1.665e+07', 'mean_token_accuracy': '0.7708', 'epoch': '0.02717'}
{'loss': '0.6011', 'grad_norm': '0.3412', 'learning_rate': '0.0001777', 'entropy': '0.5896', 'num_tokens': '1.665e+07', 'mean_token_accuracy': '0.8475', 'epoch': '0.02718'}
{'loss': '0.9443', 'grad_norm': '0.3622', 'learning_rate': '0.0001777', 'entropy': '1.046', 'num_tokens': '1.666e+07', 'mean_token_accuracy': '0.7594', 'epoch': '0.02719'}
{'loss': '0.9354', 'grad_norm': '0.4755', 'learning_rate': '0.0001777', 'entropy': '0.9386', 'num_tokens': '1.666e+07', 'mean_token_accuracy': '0.7602', 'epoch': '0.0272'}
{'loss': '1.009', 'grad_norm': '0.4171', 'learning_rate': '0.0001777', 'entropy': '1.082', 'num_tokens': '1.666e+07', 'mean_token_accuracy': '0.7244', 'epoch': '0.0272'}
{'loss': '0.746', 'grad_norm': '0.317', 'learning_rate': '0.0001776', 'entropy': '0.7668', 'num_tokens': '1.667e+07', 'mean_token_accuracy': '0.7945', 'epoch': '0.02721'}
{'loss': '1.029', 'grad_norm': '0.6161', 'learning_rate': '0.0001776', 'entropy': '0.892', 'num_tokens': '1.668e+07', 'mean_token_accuracy': '0.7659', 'epoch': '0.02722'}
{'loss': '1.125', 'grad_norm': '0.4067', 'learning_rate': '0.0001776', 'entropy': '1.064', 'num_tokens': '1.668e+07', 'mean_token_accuracy': '0.7308', 'epoch': '0.02723'}
{'loss': '0.7937', 'grad_norm': '0.3155', 'learning_rate': '0.0001776', 'entropy': '0.7506', 'num_tokens': '1.669e+07', 'mean_token_accuracy': '0.8202', 'epoch': '0.02724'}
{'loss': '0.9071', 'grad_norm': '0.3372', 'learning_rate': '0.0001776', 'entropy': '0.8637', 'num_tokens': '1.67e+07', 'mean_token_accuracy': '0.7726', 'epoch': '0.02725'}
{'loss': '0.7445', 'grad_norm': '0.4005', 'learning_rate': '0.0001776', 'entropy': '0.7275', 'num_tokens': '1.67e+07', 'mean_token_accuracy': '0.8019', 'epoch': '0.02726'}
{'loss': '0.5116', 'grad_norm': '0.2917', 'learning_rate': '0.0001775', 'entropy': '0.5322', 'num_tokens': '1.671e+07', 'mean_token_accuracy': '0.8611', 'epoch': '0.02727'}
{'loss': '1.066', 'grad_norm': '0.4637', 'learning_rate': '0.0001775', 'entropy': '1.039', 'num_tokens': '1.671e+07', 'mean_token_accuracy': '0.7488', 'epoch': '0.02728'}
{'loss': '0.7364', 'grad_norm': '0.3121', 'learning_rate': '0.0001775', 'entropy': '0.8262', 'num_tokens': '1.672e+07', 'mean_token_accuracy': '0.7935', 'epoch': '0.02728'}
{'loss': '0.8105', 'grad_norm': '0.3415', 'learning_rate': '0.0001775', 'entropy': '0.8402', 'num_tokens': '1.672e+07', 'mean_token_accuracy': '0.7925', 'epoch': '0.02729'}
{'loss': '0.8967', 'grad_norm': '0.3471', 'learning_rate': '0.0001775', 'entropy': '0.9163', 'num_tokens': '1.673e+07', 'mean_token_accuracy': '0.7696', 'epoch': '0.0273'}
{'loss': '1.232', 'grad_norm': '0.5416', 'learning_rate': '0.0001775', 'entropy': '1.325', 'num_tokens': '1.673e+07', 'mean_token_accuracy': '0.6839', 'epoch': '0.02731'}
{'loss': '0.5999', 'grad_norm': '0.3131', 'learning_rate': '0.0001774', 'entropy': '0.614', 'num_tokens': '1.674e+07', 'mean_token_accuracy': '0.8499', 'epoch': '0.02732'}
{'loss': '0.9727', 'grad_norm': '0.399', 'learning_rate': '0.0001774', 'entropy': '1.04', 'num_tokens': '1.675e+07', 'mean_token_accuracy': '0.7456', 'epoch': '0.02733'}
{'loss': '0.9042', 'grad_norm': '0.3421', 'learning_rate': '0.0001774', 'entropy': '0.9067', 'num_tokens': '1.675e+07', 'mean_token_accuracy': '0.7784', 'epoch': '0.02734'}
{'loss': '1.501', 'grad_norm': '0.4625', 'learning_rate': '0.0001774', 'entropy': '1.565', 'num_tokens': '1.675e+07', 'mean_token_accuracy': '0.6471', 'epoch': '0.02735'}
{'loss': '0.8545', 'grad_norm': '0.4145', 'learning_rate': '0.0001774', 'entropy': '0.9224', 'num_tokens': '1.676e+07', 'mean_token_accuracy': '0.7772', 'epoch': '0.02735'}
{'loss': '1.057', 'grad_norm': '0.5296', 'learning_rate': '0.0001773', 'entropy': '0.8768', 'num_tokens': '1.676e+07', 'mean_token_accuracy': '0.781', 'epoch': '0.02736'}
{'loss': '0.7714', 'grad_norm': '0.3579', 'learning_rate': '0.0001773', 'entropy': '0.7722', 'num_tokens': '1.677e+07', 'mean_token_accuracy': '0.7953', 'epoch': '0.02737'}
{'loss': '0.8562', 'grad_norm': '0.3983', 'learning_rate': '0.0001773', 'entropy': '0.7808', 'num_tokens': '1.677e+07', 'mean_token_accuracy': '0.8062', 'epoch': '0.02738'}
{'loss': '1.128', 'grad_norm': '0.342', 'learning_rate': '0.0001773', 'entropy': '1.053', 'num_tokens': '1.678e+07', 'mean_token_accuracy': '0.7502', 'epoch': '0.02739'}
{'loss': '0.6594', 'grad_norm': '0.3696', 'learning_rate': '0.0001773', 'entropy': '0.6744', 'num_tokens': '1.678e+07', 'mean_token_accuracy': '0.8156', 'epoch': '0.0274'}
{'loss': '0.8837', 'grad_norm': '0.3571', 'learning_rate': '0.0001773', 'entropy': '0.8542', 'num_tokens': '1.679e+07', 'mean_token_accuracy': '0.7855', 'epoch': '0.02741'}
{'loss': '0.8913', 'grad_norm': '0.3994', 'learning_rate': '0.0001772', 'entropy': '0.9295', 'num_tokens': '1.679e+07', 'mean_token_accuracy': '0.7713', 'epoch': '0.02742'}
{'loss': '1.117', 'grad_norm': '0.4817', 'learning_rate': '0.0001772', 'entropy': '1.102', 'num_tokens': '1.679e+07', 'mean_token_accuracy': '0.7309', 'epoch': '0.02743'}
{'loss': '0.928', 'grad_norm': '0.4432', 'learning_rate': '0.0001772', 'entropy': '0.8961', 'num_tokens': '1.68e+07', 'mean_token_accuracy': '0.7842', 'epoch': '0.02743'}
{'loss': '1.246', 'grad_norm': '0.4172', 'learning_rate': '0.0001772', 'entropy': '1.188', 'num_tokens': '1.681e+07', 'mean_token_accuracy': '0.7254', 'epoch': '0.02744'}
{'loss': '1.021', 'grad_norm': '0.5637', 'learning_rate': '0.0001772', 'entropy': '1.106', 'num_tokens': '1.681e+07', 'mean_token_accuracy': '0.7322', 'epoch': '0.02745'}
{'loss': '0.8434', 'grad_norm': '0.389', 'learning_rate': '0.0001772', 'entropy': '0.798', 'num_tokens': '1.681e+07', 'mean_token_accuracy': '0.7898', 'epoch': '0.02746'}
{'loss': '1.168', 'grad_norm': '0.4092', 'learning_rate': '0.0001771', 'entropy': '1.102', 'num_tokens': '1.682e+07', 'mean_token_accuracy': '0.7283', 'epoch': '0.02747'}
{'loss': '1.023', 'grad_norm': '0.3262', 'learning_rate': '0.0001771', 'entropy': '0.9978', 'num_tokens': '1.683e+07', 'mean_token_accuracy': '0.7621', 'epoch': '0.02748'}

 26%|██▌       | 3106/12000 [1:51:28<4:41:09,  1.90s/it]
 26%|██▌       | 3107/12000 [1:51:30<4:19:48,  1.75s/it]
                                                        

 26%|██▌       | 3107/12000 [1:51:30<4:19:48,  1.75s/it]
 26%|██▌       | 3108/12000 [1:51:32<4:52:39,  1.97s/it]
                                                        

 26%|██▌       | 3108/12000 [1:51:32<4:52:39,  1.97s/it]
 26%|██▌       | 3109/12000 [1:51:34<4:53:50,  1.98s/it]
                                                        

 26%|██▌       | 3109/12000 [1:51:34<4:53:50,  1.98s/it]
 26%|██▌       | 3110/12000 [1:51:36<4:43:49,  1.92s/it]
                                                        

 26%|██▌       | 3110/12000 [1:51:36<4:43:49,  1.92s/it]
 26%|██▌       | 3111/12000 [1:51:38<4:30:37,  1.83s/it]
                                                        

 26%|██▌       | 3111/12000 [1:51:38<4:30:37,  1.83s/it]
 26%|██▌       | 3112/12000 [1:51:39<4:18:40,  1.75s/it]
                                                        

 26%|██▌       | 3112/12000 [1:51:39<4:18:40,  1.75s/it]
 26%|██▌       | 3113/12000 [1:51:41<4:32:07,  1.84s/it]
                                                        

 26%|██▌       | 3113/12000 [1:51:41<4:32:07,  1.84s/it]
 26%|██▌       | 3114/12000 [1:51:43<4:39:16,  1.89s/it]
                                                        

 26%|██▌       | 3114/12000 [1:51:43<4:39:16,  1.89s/it]
 26%|██▌       | 3115/12000 [1:51:45<4:22:38,  1.77s/it]
                                                        

 26%|██▌       | 3115/12000 [1:51:45<4:22:38,  1.77s/it]
 26%|██▌       | 3116/12000 [1:51:48<5:03:35,  2.05s/it]
                                                        

 26%|██▌       | 3116/12000 [1:51:48<5:03:35,  2.05s/it]
 26%|██▌       | 3117/12000 [1:51:49<4:38:37,  1.88s/it]
                                                        

 26%|██▌       | 3117/12000 [1:51:49<4:38:37,  1.88s/it]
 26%|██▌       | 3118/12000 [1:51:51<4:29:31,  1.82s/it]
                                                        

 26%|██▌       | 3118/12000 [1:51:51<4:29:31,  1.82s/it]
 26%|██▌       | 3119/12000 [1:51:54<5:45:41,  2.34s/it]
                                                        

 26%|██▌       | 3119/12000 [1:51:54<5:45:41,  2.34s/it]
 26%|██▌       | 3120/12000 [1:51:57<6:00:38,  2.44s/it]
                                                        

 26%|██▌       | 3120/12000 [1:51:57<6:00:38,  2.44s/it]
 26%|██▌       | 3121/12000 [1:51:59<5:53:40,  2.39s/it]
                                                        

 26%|██▌       | 3121/12000 [1:51:59<5:53:40,  2.39s/it]
 26%|██▌       | 3122/12000 [1:52:02<5:59:22,  2.43s/it]
                                                        

 26%|██▌       | 3122/12000 [1:52:02<5:59:22,  2.43s/it]
 26%|██▌       | 3123/12000 [1:52:03<5:16:21,  2.14s/it]
                                                        

 26%|██▌       | 3123/12000 [1:52:03<5:16:21,  2.14s/it]
 26%|██▌       | 3124/12000 [1:52:05<4:59:29,  2.02s/it]
                                                        

 26%|██▌       | 3124/12000 [1:52:05<4:59:29,  2.02s/it]
 26%|██▌       | 3125/12000 [1:52:07<4:39:43,  1.89s/it]
                                                        

 26%|██▌       | 3125/12000 [1:52:07<4:39:43,  1.89s/it]
 26%|██▌       | 3126/12000 [1:52:08<4:24:56,  1.79s/it]
                                                        

 26%|██▌       | 3126/12000 [1:52:08<4:24:56,  1.79s/it]
 26%|██▌       | 3127/12000 [1:52:10<4:11:13,  1.70s/it]
                                                        

 26%|██▌       | 3127/12000 [1:52:10<4:11:13,  1.70s/it]
 26%|██▌       | 3128/12000 [1:52:11<4:02:30,  1.64s/it]
                                                        

 26%|██▌       | 3128/12000 [1:52:11<4:02:30,  1.64s/it]
 26%|██▌       | 3129/12000 [1:52:13<3:56:38,  1.60s/it]
                                                        

 26%|██▌       | 3129/12000 [1:52:13<3:56:38,  1.60s/it]
 26%|██▌       | 3130/12000 [1:52:16<5:38:12,  2.29s/it]
                                                        

 26%|██▌       | 3130/12000 [1:52:16<5:38:12,  2.29s/it]
 26%|██▌       | 3131/12000 [1:52:18<4:59:57,  2.03s/it]
                                                        

 26%|██▌       | 3131/12000 [1:52:18<4:59:57,  2.03s/it]
 26%|██▌       | 3132/12000 [1:52:20<5:19:42,  2.16s/it]
                                                        

 26%|██▌       | 3132/12000 [1:52:20<5:19:42,  2.16s/it]
 26%|██▌       | 3133/12000 [1:52:22<4:50:22,  1.96s/it]
                                                        

 26%|██▌       | 3133/12000 [1:52:22<4:50:22,  1.96s/it]
 26%|██▌       | 3134/12000 [1:52:25<5:53:44,  2.39s/it]
                                                        

 26%|██▌       | 3134/12000 [1:52:25<5:53:44,  2.39s/it]
 26%|██▌       | 3135/12000 [1:52:28<6:12:30,  2.52s/it]
                                                        

 26%|██▌       | 3135/12000 [1:52:28<6:12:30,  2.52s/it]
 26%|██▌       | 3136/12000 [1:52:31<6:31:53,  2.65s/it]
                                                        

 26%|██▌       | 3136/12000 [1:52:31<6:31:53,  2.65s/it]
 26%|██▌       | 3137/12000 [1:52:33<5:49:38,  2.37s/it]
                                                        

 26%|██▌       | 3137/12000 [1:52:33<5:49:38,  2.37s/it]
 26%|██▌       | 3138/12000 [1:52:34<5:14:41,  2.13s/it]
                                                        

 26%|██▌       | 3138/12000 [1:52:34<5:14:41,  2.13s/it]
 26%|██▌       | 3139/12000 [1:52:37<5:19:19,  2.16s/it]
                                                        

 26%|██▌       | 3139/12000 [1:52:37<5:19:19,  2.16s/it]
 26%|██▌       | 3140/12000 [1:52:39<5:42:08,  2.32s/it]
                                                        

 26%|██▌       | 3140/12000 [1:52:39<5:42:08,  2.32s/it]
 26%|██▌       | 3141/12000 [1:52:44<7:20:00,  2.98s/it]
                                                        

 26%|██▌       | 3141/12000 [1:52:44<7:20:00,  2.98s/it]
 26%|██▌       | 3142/12000 [1:52:47<7:39:22,  3.11s/it]
                                                        

 26%|██▌       | 3142/12000 [1:52:47<7:39:22,  3.11s/it]
 26%|██▌       | 3143/12000 [1:52:49<6:26:59,  2.62s/it]
                                                        

 26%|██▌       | 3143/12000 [1:52:49<6:26:59,  2.62s/it]
 26%|██▌       | 3144/12000 [1:52:51<6:28:50,  2.63s/it]
                                                        

 26%|██▌       | 3144/12000 [1:52:51<6:28:50,  2.63s/it]
 26%|██▌       | 3145/12000 [1:52:53<5:38:09,  2.29s/it]
                                                        

 26%|██▌       | 3145/12000 [1:52:53<5:38:09,  2.29s/it]
 26%|██▌       | 3146/12000 [1:52:54<5:03:18,  2.06s/it]
                                                        

 26%|██▌       | 3146/12000 [1:52:54<5:03:18,  2.06s/it]
 26%|██▌       | 3147/12000 [1:52:56<4:38:41,  1.89s/it]
                                                        

 26%|██▌       | 3147/12000 [1:52:56<4:38:41,  1.89s/it]
 26%|██▌       | 3148/12000 [1:52:58<4:54:35,  2.00s/it]
                                                        

 26%|██▌       | 3148/12000 [1:52:58<4:54:35,  2.00s/it]
 26%|██▌       | 3149/12000 [1:53:00<4:45:28,  1.94s/it]
                                                        

 26%|██▌       | 3149/12000 [1:53:00<4:45:28,  1.94s/it]
 26%|██▋       | 3150/12000 [1:53:01<4:23:20,  1.79s/it]
                                                        

 26%|██▋       | 3150/12000 [1:53:01<4:23:20,  1.79s/it]
 26%|██▋       | 3151/12000 [1:53:04<5:09:54,  2.10s/it]
                                                        

 26%|██▋       | 3151/12000 [1:53:04<5:09:54,  2.10s/it]
 26%|██▋       | 3152/12000 [1:53:07<5:32:14,  2.25s/it]
                                                        

 26%|██▋       | 3152/12000 [1:53:07<5:32:14,  2.25s/it]
 26%|██▋       | 3153/12000 [1:53:09<5:16:08,  2.14s/it]
                                                        
{'loss': '0.8829', 'grad_norm': '0.3904', 'learning_rate': '0.0001771', 'entropy': '0.9761', 'num_tokens': '1.683e+07', 'mean_token_accuracy': '0.763', 'epoch': '0.02749'}
{'loss': '1.05', 'grad_norm': '0.4478', 'learning_rate': '0.0001771', 'entropy': '1.059', 'num_tokens': '1.683e+07', 'mean_token_accuracy': '0.7407', 'epoch': '0.0275'}
{'loss': '0.9626', 'grad_norm': '0.3986', 'learning_rate': '0.0001771', 'entropy': '0.9639', 'num_tokens': '1.684e+07', 'mean_token_accuracy': '0.7678', 'epoch': '0.02751'}
{'loss': '1.075', 'grad_norm': '0.3875', 'learning_rate': '0.0001771', 'entropy': '0.9325', 'num_tokens': '1.685e+07', 'mean_token_accuracy': '0.7598', 'epoch': '0.02751'}
{'loss': '1.083', 'grad_norm': '0.4209', 'learning_rate': '0.000177', 'entropy': '1.061', 'num_tokens': '1.685e+07', 'mean_token_accuracy': '0.7367', 'epoch': '0.02752'}
{'loss': '0.8692', 'grad_norm': '0.4482', 'learning_rate': '0.000177', 'entropy': '0.9284', 'num_tokens': '1.686e+07', 'mean_token_accuracy': '0.7657', 'epoch': '0.02753'}
{'loss': '0.8826', 'grad_norm': '0.4123', 'learning_rate': '0.000177', 'entropy': '0.915', 'num_tokens': '1.686e+07', 'mean_token_accuracy': '0.7746', 'epoch': '0.02754'}
{'loss': '0.9353', 'grad_norm': '0.449', 'learning_rate': '0.000177', 'entropy': '0.9122', 'num_tokens': '1.687e+07', 'mean_token_accuracy': '0.7655', 'epoch': '0.02755'}
{'loss': '1.228', 'grad_norm': '0.4256', 'learning_rate': '0.000177', 'entropy': '1.121', 'num_tokens': '1.687e+07', 'mean_token_accuracy': '0.7146', 'epoch': '0.02756'}
{'loss': '0.7932', 'grad_norm': '0.3108', 'learning_rate': '0.0001769', 'entropy': '0.8123', 'num_tokens': '1.688e+07', 'mean_token_accuracy': '0.8023', 'epoch': '0.02757'}
{'loss': '1.03', 'grad_norm': '0.4314', 'learning_rate': '0.0001769', 'entropy': '0.9422', 'num_tokens': '1.688e+07', 'mean_token_accuracy': '0.7654', 'epoch': '0.02758'}
{'loss': '1.008', 'grad_norm': '0.4363', 'learning_rate': '0.0001769', 'entropy': '1.09', 'num_tokens': '1.689e+07', 'mean_token_accuracy': '0.7403', 'epoch': '0.02758'}
{'loss': '0.8771', 'grad_norm': '0.4232', 'learning_rate': '0.0001769', 'entropy': '0.9242', 'num_tokens': '1.689e+07', 'mean_token_accuracy': '0.7689', 'epoch': '0.02759'}
{'loss': '0.6702', 'grad_norm': '0.377', 'learning_rate': '0.0001769', 'entropy': '0.6961', 'num_tokens': '1.69e+07', 'mean_token_accuracy': '0.814', 'epoch': '0.0276'}
{'loss': '0.8248', 'grad_norm': '0.3593', 'learning_rate': '0.0001769', 'entropy': '0.8175', 'num_tokens': '1.69e+07', 'mean_token_accuracy': '0.8034', 'epoch': '0.02761'}
{'loss': '0.8385', 'grad_norm': '0.3472', 'learning_rate': '0.0001768', 'entropy': '0.8353', 'num_tokens': '1.691e+07', 'mean_token_accuracy': '0.7976', 'epoch': '0.02762'}
{'loss': '0.9076', 'grad_norm': '0.3557', 'learning_rate': '0.0001768', 'entropy': '0.9643', 'num_tokens': '1.692e+07', 'mean_token_accuracy': '0.7774', 'epoch': '0.02763'}
{'loss': '0.9451', 'grad_norm': '0.5231', 'learning_rate': '0.0001768', 'entropy': '1.015', 'num_tokens': '1.692e+07', 'mean_token_accuracy': '0.7431', 'epoch': '0.02764'}
{'loss': '1.117', 'grad_norm': '0.4408', 'learning_rate': '0.0001768', 'entropy': '1.067', 'num_tokens': '1.693e+07', 'mean_token_accuracy': '0.7629', 'epoch': '0.02765'}
{'loss': '1.167', 'grad_norm': '0.489', 'learning_rate': '0.0001768', 'entropy': '1.105', 'num_tokens': '1.693e+07', 'mean_token_accuracy': '0.7278', 'epoch': '0.02766'}
{'loss': '1.05', 'grad_norm': '0.4107', 'learning_rate': '0.0001768', 'entropy': '1.047', 'num_tokens': '1.694e+07', 'mean_token_accuracy': '0.7487', 'epoch': '0.02766'}
{'loss': '0.6634', 'grad_norm': '0.3468', 'learning_rate': '0.0001767', 'entropy': '0.6925', 'num_tokens': '1.694e+07', 'mean_token_accuracy': '0.8207', 'epoch': '0.02767'}
{'loss': '0.8387', 'grad_norm': '0.3711', 'learning_rate': '0.0001767', 'entropy': '0.8761', 'num_tokens': '1.695e+07', 'mean_token_accuracy': '0.7842', 'epoch': '0.02768'}
{'loss': '0.8125', 'grad_norm': '0.4411', 'learning_rate': '0.0001767', 'entropy': '0.8434', 'num_tokens': '1.695e+07', 'mean_token_accuracy': '0.7712', 'epoch': '0.02769'}
{'loss': '0.871', 'grad_norm': '0.345', 'learning_rate': '0.0001767', 'entropy': '0.831', 'num_tokens': '1.696e+07', 'mean_token_accuracy': '0.7937', 'epoch': '0.0277'}
{'loss': '0.766', 'grad_norm': '0.331', 'learning_rate': '0.0001767', 'entropy': '0.7634', 'num_tokens': '1.696e+07', 'mean_token_accuracy': '0.8117', 'epoch': '0.02771'}
{'loss': '0.8636', 'grad_norm': '0.3495', 'learning_rate': '0.0001766', 'entropy': '0.7461', 'num_tokens': '1.697e+07', 'mean_token_accuracy': '0.8229', 'epoch': '0.02772'}
{'loss': '0.8039', 'grad_norm': '0.3546', 'learning_rate': '0.0001766', 'entropy': '0.8018', 'num_tokens': '1.697e+07', 'mean_token_accuracy': '0.8162', 'epoch': '0.02773'}
{'loss': '0.9711', 'grad_norm': '0.4007', 'learning_rate': '0.0001766', 'entropy': '1.053', 'num_tokens': '1.698e+07', 'mean_token_accuracy': '0.756', 'epoch': '0.02774'}
{'loss': '0.9317', 'grad_norm': '0.3219', 'learning_rate': '0.0001766', 'entropy': '0.8035', 'num_tokens': '1.699e+07', 'mean_token_accuracy': '0.7991', 'epoch': '0.02774'}
{'loss': '1.267', 'grad_norm': '0.4013', 'learning_rate': '0.0001766', 'entropy': '1.28', 'num_tokens': '1.699e+07', 'mean_token_accuracy': '0.7148', 'epoch': '0.02775'}
{'loss': '0.9462', 'grad_norm': '0.465', 'learning_rate': '0.0001766', 'entropy': '0.9662', 'num_tokens': '1.7e+07', 'mean_token_accuracy': '0.7549', 'epoch': '0.02776'}
{'loss': '0.7672', 'grad_norm': '0.3665', 'learning_rate': '0.0001765', 'entropy': '0.7902', 'num_tokens': '1.7e+07', 'mean_token_accuracy': '0.8119', 'epoch': '0.02777'}
{'loss': '0.8189', 'grad_norm': '0.3422', 'learning_rate': '0.0001765', 'entropy': '0.7384', 'num_tokens': '1.701e+07', 'mean_token_accuracy': '0.8153', 'epoch': '0.02778'}
{'loss': '0.76', 'grad_norm': '0.3847', 'learning_rate': '0.0001765', 'entropy': '0.7511', 'num_tokens': '1.702e+07', 'mean_token_accuracy': '0.8069', 'epoch': '0.02779'}
{'loss': '0.7294', 'grad_norm': '0.295', 'learning_rate': '0.0001765', 'entropy': '0.8057', 'num_tokens': '1.702e+07', 'mean_token_accuracy': '0.7945', 'epoch': '0.0278'}
{'loss': '0.9515', 'grad_norm': '0.3601', 'learning_rate': '0.0001765', 'entropy': '0.8984', 'num_tokens': '1.703e+07', 'mean_token_accuracy': '0.7733', 'epoch': '0.02781'}
{'loss': '0.8424', 'grad_norm': '0.3685', 'learning_rate': '0.0001765', 'entropy': '0.88', 'num_tokens': '1.703e+07', 'mean_token_accuracy': '0.7711', 'epoch': '0.02782'}
{'loss': '0.7922', 'grad_norm': '0.3698', 'learning_rate': '0.0001764', 'entropy': '0.8908', 'num_tokens': '1.704e+07', 'mean_token_accuracy': '0.7847', 'epoch': '0.02782'}
{'loss': '1.008', 'grad_norm': '0.4344', 'learning_rate': '0.0001764', 'entropy': '0.9775', 'num_tokens': '1.704e+07', 'mean_token_accuracy': '0.7529', 'epoch': '0.02783'}
{'loss': '0.9364', 'grad_norm': '0.4249', 'learning_rate': '0.0001764', 'entropy': '0.9547', 'num_tokens': '1.705e+07', 'mean_token_accuracy': '0.7561', 'epoch': '0.02784'}
{'loss': '1.184', 'grad_norm': '0.4719', 'learning_rate': '0.0001764', 'entropy': '1.251', 'num_tokens': '1.705e+07', 'mean_token_accuracy': '0.7146', 'epoch': '0.02785'}
{'loss': '1.092', 'grad_norm': '0.3924', 'learning_rate': '0.0001764', 'entropy': '0.989', 'num_tokens': '1.706e+07', 'mean_token_accuracy': '0.7674', 'epoch': '0.02786'}
{'loss': '1.281', 'grad_norm': '0.4199', 'learning_rate': '0.0001763', 'entropy': '1.213', 'num_tokens': '1.706e+07', 'mean_token_accuracy': '0.7259', 'epoch': '0.02787'}
{'loss': '0.8328', 'grad_norm': '0.3807', 'learning_rate': '0.0001763', 'entropy': '0.9453', 'num_tokens': '1.707e+07', 'mean_token_accuracy': '0.7664', 'epoch': '0.02788'}
{'loss': '0.5828', 'grad_norm': '0.2994', 'learning_rate': '0.0001763', 'entropy': '0.6205', 'num_tokens': '1.707e+07', 'mean_token_accuracy': '0.8433', 'epoch': '0.02789'}
{'loss': '0.9923', 'grad_norm': '0.3002', 'learning_rate': '0.0001763', 'entropy': '1.084', 'num_tokens': '1.708e+07', 'mean_token_accuracy': '0.7467', 'epoch': '0.02789'}

 26%|██▋       | 3153/12000 [1:53:09<5:16:08,  2.14s/it]
 26%|██▋       | 3154/12000 [1:53:12<6:01:15,  2.45s/it]
                                                        

 26%|██▋       | 3154/12000 [1:53:12<6:01:15,  2.45s/it]
 26%|██▋       | 3155/12000 [1:53:14<5:41:03,  2.31s/it]
                                                        

 26%|██▋       | 3155/12000 [1:53:14<5:41:03,  2.31s/it]
 26%|██▋       | 3156/12000 [1:53:19<7:49:26,  3.18s/it]
                                                        

 26%|██▋       | 3156/12000 [1:53:19<7:49:26,  3.18s/it]
 26%|██▋       | 3157/12000 [1:53:22<7:27:30,  3.04s/it]
                                                        

 26%|██▋       | 3157/12000 [1:53:22<7:27:30,  3.04s/it]
 26%|██▋       | 3158/12000 [1:53:23<6:22:42,  2.60s/it]
                                                        

 26%|██▋       | 3158/12000 [1:53:23<6:22:42,  2.60s/it]
 26%|██▋       | 3159/12000 [1:53:25<5:35:26,  2.28s/it]
                                                        

 26%|██▋       | 3159/12000 [1:53:25<5:35:26,  2.28s/it]
 26%|██▋       | 3160/12000 [1:53:28<6:06:45,  2.49s/it]
                                                        

 26%|██▋       | 3160/12000 [1:53:28<6:06:45,  2.49s/it]
 26%|██▋       | 3161/12000 [1:53:30<5:36:34,  2.28s/it]
                                                        

 26%|██▋       | 3161/12000 [1:53:30<5:36:34,  2.28s/it]
 26%|██▋       | 3162/12000 [1:53:32<6:00:40,  2.45s/it]
                                                        

 26%|██▋       | 3162/12000 [1:53:32<6:00:40,  2.45s/it]
 26%|██▋       | 3163/12000 [1:53:36<6:59:20,  2.85s/it]
                                                        

 26%|██▋       | 3163/12000 [1:53:36<6:59:20,  2.85s/it]
 26%|██▋       | 3164/12000 [1:53:38<6:09:43,  2.51s/it]
                                                        

 26%|██▋       | 3164/12000 [1:53:38<6:09:43,  2.51s/it]
 26%|██▋       | 3165/12000 [1:53:40<5:36:41,  2.29s/it]
                                                        

 26%|██▋       | 3165/12000 [1:53:40<5:36:41,  2.29s/it]
 26%|██▋       | 3166/12000 [1:53:41<5:09:28,  2.10s/it]
                                                        

 26%|██▋       | 3166/12000 [1:53:41<5:09:28,  2.10s/it]
 26%|██▋       | 3167/12000 [1:53:44<5:17:02,  2.15s/it]
                                                        

 26%|██▋       | 3167/12000 [1:53:44<5:17:02,  2.15s/it]
 26%|██▋       | 3168/12000 [1:53:45<4:47:49,  1.96s/it]
                                                        

 26%|██▋       | 3168/12000 [1:53:45<4:47:49,  1.96s/it]
 26%|██▋       | 3169/12000 [1:53:48<5:07:16,  2.09s/it]
                                                        

 26%|██▋       | 3169/12000 [1:53:48<5:07:16,  2.09s/it]
 26%|██▋       | 3170/12000 [1:53:49<4:31:58,  1.85s/it]
                                                        

 26%|██▋       | 3170/12000 [1:53:49<4:31:58,  1.85s/it]
 26%|██▋       | 3171/12000 [1:53:50<4:18:12,  1.75s/it]
                                                        

 26%|██▋       | 3171/12000 [1:53:50<4:18:12,  1.75s/it]
 26%|██▋       | 3172/12000 [1:53:52<4:29:41,  1.83s/it]
                                                        

 26%|██▋       | 3172/12000 [1:53:52<4:29:41,  1.83s/it]
 26%|██▋       | 3173/12000 [1:53:54<4:24:00,  1.79s/it]
                                                        

 26%|██▋       | 3173/12000 [1:53:54<4:24:00,  1.79s/it]
 26%|██▋       | 3174/12000 [1:53:56<4:50:27,  1.97s/it]
                                                        

 26%|██▋       | 3174/12000 [1:53:56<4:50:27,  1.97s/it]
 26%|██▋       | 3175/12000 [1:53:58<4:39:43,  1.90s/it]
                                                        

 26%|██▋       | 3175/12000 [1:53:58<4:39:43,  1.90s/it]
 26%|██▋       | 3176/12000 [1:54:00<4:19:19,  1.76s/it]
                                                        

 26%|██▋       | 3176/12000 [1:54:00<4:19:19,  1.76s/it]
 26%|██▋       | 3177/12000 [1:54:03<5:13:26,  2.13s/it]
                                                        

 26%|██▋       | 3177/12000 [1:54:03<5:13:26,  2.13s/it]
 26%|██▋       | 3178/12000 [1:54:05<5:16:23,  2.15s/it]
                                                        

 26%|██▋       | 3178/12000 [1:54:05<5:16:23,  2.15s/it]
 26%|██▋       | 3179/12000 [1:54:07<4:57:40,  2.02s/it]
                                                        

 26%|██▋       | 3179/12000 [1:54:07<4:57:40,  2.02s/it]
 26%|██▋       | 3180/12000 [1:54:08<4:42:01,  1.92s/it]
                                                        

 26%|██▋       | 3180/12000 [1:54:08<4:42:01,  1.92s/it]
 27%|██▋       | 3181/12000 [1:54:10<4:30:41,  1.84s/it]
                                                        

 27%|██▋       | 3181/12000 [1:54:10<4:30:41,  1.84s/it]
 27%|██▋       | 3182/12000 [1:54:11<4:16:20,  1.74s/it]
                                                        

 27%|██▋       | 3182/12000 [1:54:11<4:16:20,  1.74s/it]
 27%|██▋       | 3183/12000 [1:54:15<5:18:35,  2.17s/it]
                                                        

 27%|██▋       | 3183/12000 [1:54:15<5:18:35,  2.17s/it]
 27%|██▋       | 3184/12000 [1:54:16<4:40:27,  1.91s/it]
                                                        

 27%|██▋       | 3184/12000 [1:54:16<4:40:27,  1.91s/it]
 27%|██▋       | 3185/12000 [1:54:20<6:00:19,  2.45s/it]
                                                        

 27%|██▋       | 3185/12000 [1:54:20<6:00:19,  2.45s/it]
 27%|██▋       | 3186/12000 [1:54:21<5:25:53,  2.22s/it]
                                                        

 27%|██▋       | 3186/12000 [1:54:21<5:25:53,  2.22s/it]
 27%|██▋       | 3187/12000 [1:54:23<5:22:47,  2.20s/it]
                                                        

 27%|██▋       | 3187/12000 [1:54:23<5:22:47,  2.20s/it]
 27%|██▋       | 3188/12000 [1:54:25<4:46:43,  1.95s/it]
                                                        

 27%|██▋       | 3188/12000 [1:54:25<4:46:43,  1.95s/it]
 27%|██▋       | 3189/12000 [1:54:27<4:46:13,  1.95s/it]
                                                        

 27%|██▋       | 3189/12000 [1:54:27<4:46:13,  1.95s/it]
 27%|██▋       | 3190/12000 [1:54:30<5:47:13,  2.36s/it]
                                                        

 27%|██▋       | 3190/12000 [1:54:30<5:47:13,  2.36s/it]
 27%|██▋       | 3191/12000 [1:54:32<5:16:29,  2.16s/it]
                                                        

 27%|██▋       | 3191/12000 [1:54:32<5:16:29,  2.16s/it]
 27%|██▋       | 3192/12000 [1:54:34<5:33:02,  2.27s/it]
                                                        

 27%|██▋       | 3192/12000 [1:54:34<5:33:02,  2.27s/it]
 27%|██▋       | 3193/12000 [1:54:36<5:13:05,  2.13s/it]
                                                        

 27%|██▋       | 3193/12000 [1:54:36<5:13:05,  2.13s/it]
 27%|██▋       | 3194/12000 [1:54:40<6:18:40,  2.58s/it]
                                                        

 27%|██▋       | 3194/12000 [1:54:40<6:18:40,  2.58s/it]
 27%|██▋       | 3195/12000 [1:54:42<6:08:56,  2.51s/it]
                                                        

 27%|██▋       | 3195/12000 [1:54:42<6:08:56,  2.51s/it]
 27%|██▋       | 3196/12000 [1:54:44<5:38:22,  2.31s/it]
                                                        

 27%|██▋       | 3196/12000 [1:54:44<5:38:22,  2.31s/it]
 27%|██▋       | 3197/12000 [1:54:45<5:01:29,  2.05s/it]
                                                        

 27%|██▋       | 3197/12000 [1:54:45<5:01:29,  2.05s/it]
 27%|██▋       | 3198/12000 [1:54:47<4:57:21,  2.03s/it]
                                                        

 27%|██▋       | 3198/12000 [1:54:47<4:57:21,  2.03s/it]
 27%|██▋       | 3199/12000 [1:54:49<5:03:57,  2.07s/it]
                                                        

 27%|██▋       | 3199/12000 [1:54:49<5:03:57,  2.07s/it]
 27%|██▋       | 3200/12000 [1:54:52<5:07:21,  2.10s/it]
                                                        
{'loss': '1.408', 'grad_norm': '0.413', 'learning_rate': '0.0001763', 'entropy': '1.147', 'num_tokens': '1.708e+07', 'mean_token_accuracy': '0.7324', 'epoch': '0.0279'}
{'loss': '1.12', 'grad_norm': '0.3783', 'learning_rate': '0.0001763', 'entropy': '1.154', 'num_tokens': '1.709e+07', 'mean_token_accuracy': '0.7287', 'epoch': '0.02791'}
{'loss': '0.9916', 'grad_norm': '0.4467', 'learning_rate': '0.0001762', 'entropy': '0.9317', 'num_tokens': '1.71e+07', 'mean_token_accuracy': '0.7739', 'epoch': '0.02792'}
{'loss': '0.621', 'grad_norm': '0.2539', 'learning_rate': '0.0001762', 'entropy': '0.8705', 'num_tokens': '1.711e+07', 'mean_token_accuracy': '0.7942', 'epoch': '0.02793'}
{'loss': '0.8348', 'grad_norm': '0.4356', 'learning_rate': '0.0001762', 'entropy': '0.8997', 'num_tokens': '1.711e+07', 'mean_token_accuracy': '0.7706', 'epoch': '0.02794'}
{'loss': '1.006', 'grad_norm': '0.3751', 'learning_rate': '0.0001762', 'entropy': '0.9765', 'num_tokens': '1.712e+07', 'mean_token_accuracy': '0.7531', 'epoch': '0.02795'}
{'loss': '0.7872', 'grad_norm': '0.3756', 'learning_rate': '0.0001762', 'entropy': '0.8319', 'num_tokens': '1.712e+07', 'mean_token_accuracy': '0.7789', 'epoch': '0.02796'}
{'loss': '0.9165', 'grad_norm': '0.4012', 'learning_rate': '0.0001761', 'entropy': '1.01', 'num_tokens': '1.713e+07', 'mean_token_accuracy': '0.7384', 'epoch': '0.02797'}
{'loss': '1.002', 'grad_norm': '0.4357', 'learning_rate': '0.0001761', 'entropy': '0.9011', 'num_tokens': '1.713e+07', 'mean_token_accuracy': '0.7725', 'epoch': '0.02797'}
{'loss': '1.32', 'grad_norm': '0.4749', 'learning_rate': '0.0001761', 'entropy': '1.205', 'num_tokens': '1.714e+07', 'mean_token_accuracy': '0.7149', 'epoch': '0.02798'}
{'loss': '1.022', 'grad_norm': '0.4141', 'learning_rate': '0.0001761', 'entropy': '0.9674', 'num_tokens': '1.715e+07', 'mean_token_accuracy': '0.7565', 'epoch': '0.02799'}
{'loss': '0.6815', 'grad_norm': '0.4156', 'learning_rate': '0.0001761', 'entropy': '0.6697', 'num_tokens': '1.715e+07', 'mean_token_accuracy': '0.8148', 'epoch': '0.028'}
{'loss': '0.7422', 'grad_norm': '0.3801', 'learning_rate': '0.0001761', 'entropy': '0.7186', 'num_tokens': '1.716e+07', 'mean_token_accuracy': '0.8169', 'epoch': '0.02801'}
{'loss': '0.8975', 'grad_norm': '0.3954', 'learning_rate': '0.000176', 'entropy': '0.8204', 'num_tokens': '1.716e+07', 'mean_token_accuracy': '0.7923', 'epoch': '0.02802'}
{'loss': '1.088', 'grad_norm': '0.5415', 'learning_rate': '0.000176', 'entropy': '1.019', 'num_tokens': '1.717e+07', 'mean_token_accuracy': '0.753', 'epoch': '0.02803'}
{'loss': '0.6476', 'grad_norm': '0.3223', 'learning_rate': '0.000176', 'entropy': '0.746', 'num_tokens': '1.717e+07', 'mean_token_accuracy': '0.8112', 'epoch': '0.02804'}
{'loss': '0.8604', 'grad_norm': '0.3492', 'learning_rate': '0.000176', 'entropy': '0.8259', 'num_tokens': '1.718e+07', 'mean_token_accuracy': '0.7978', 'epoch': '0.02805'}
{'loss': '0.8777', 'grad_norm': '0.3559', 'learning_rate': '0.000176', 'entropy': '0.9017', 'num_tokens': '1.718e+07', 'mean_token_accuracy': '0.7683', 'epoch': '0.02805'}
{'loss': '0.7398', 'grad_norm': '0.3228', 'learning_rate': '0.000176', 'entropy': '0.7644', 'num_tokens': '1.719e+07', 'mean_token_accuracy': '0.8153', 'epoch': '0.02806'}
{'loss': '0.9037', 'grad_norm': '0.3556', 'learning_rate': '0.0001759', 'entropy': '0.9446', 'num_tokens': '1.719e+07', 'mean_token_accuracy': '0.7735', 'epoch': '0.02807'}
{'loss': '1.025', 'grad_norm': '0.4655', 'learning_rate': '0.0001759', 'entropy': '1.01', 'num_tokens': '1.72e+07', 'mean_token_accuracy': '0.7659', 'epoch': '0.02808'}
{'loss': '1.014', 'grad_norm': '0.447', 'learning_rate': '0.0001759', 'entropy': '1.043', 'num_tokens': '1.721e+07', 'mean_token_accuracy': '0.7656', 'epoch': '0.02809'}
{'loss': '0.8509', 'grad_norm': '0.3553', 'learning_rate': '0.0001759', 'entropy': '0.8813', 'num_tokens': '1.721e+07', 'mean_token_accuracy': '0.7848', 'epoch': '0.0281'}
{'loss': '0.7797', 'grad_norm': '0.3208', 'learning_rate': '0.0001759', 'entropy': '0.8335', 'num_tokens': '1.721e+07', 'mean_token_accuracy': '0.8041', 'epoch': '0.02811'}
{'loss': '1.199', 'grad_norm': '0.335', 'learning_rate': '0.0001758', 'entropy': '0.9276', 'num_tokens': '1.722e+07', 'mean_token_accuracy': '0.7694', 'epoch': '0.02812'}
{'loss': '0.7109', 'grad_norm': '0.3454', 'learning_rate': '0.0001758', 'entropy': '0.7497', 'num_tokens': '1.723e+07', 'mean_token_accuracy': '0.8128', 'epoch': '0.02812'}
{'loss': '0.7452', 'grad_norm': '0.3745', 'learning_rate': '0.0001758', 'entropy': '0.78', 'num_tokens': '1.723e+07', 'mean_token_accuracy': '0.8057', 'epoch': '0.02813'}
{'loss': '1.289', 'grad_norm': '0.4589', 'learning_rate': '0.0001758', 'entropy': '1.223', 'num_tokens': '1.724e+07', 'mean_token_accuracy': '0.686', 'epoch': '0.02814'}
{'loss': '0.5401', 'grad_norm': '0.2861', 'learning_rate': '0.0001758', 'entropy': '0.5277', 'num_tokens': '1.724e+07', 'mean_token_accuracy': '0.8624', 'epoch': '0.02815'}
{'loss': '0.8261', 'grad_norm': '0.4376', 'learning_rate': '0.0001758', 'entropy': '0.7874', 'num_tokens': '1.725e+07', 'mean_token_accuracy': '0.7928', 'epoch': '0.02816'}
{'loss': '0.9663', 'grad_norm': '0.3696', 'learning_rate': '0.0001757', 'entropy': '0.8467', 'num_tokens': '1.726e+07', 'mean_token_accuracy': '0.7721', 'epoch': '0.02817'}
{'loss': '0.7965', 'grad_norm': '0.3568', 'learning_rate': '0.0001757', 'entropy': '0.7901', 'num_tokens': '1.726e+07', 'mean_token_accuracy': '0.7863', 'epoch': '0.02818'}
{'loss': '1.09', 'grad_norm': '0.427', 'learning_rate': '0.0001757', 'entropy': '1.042', 'num_tokens': '1.726e+07', 'mean_token_accuracy': '0.7387', 'epoch': '0.02819'}
{'loss': '0.9313', 'grad_norm': '0.4074', 'learning_rate': '0.0001757', 'entropy': '0.8951', 'num_tokens': '1.727e+07', 'mean_token_accuracy': '0.7735', 'epoch': '0.0282'}
{'loss': '1.159', 'grad_norm': '0.4799', 'learning_rate': '0.0001757', 'entropy': '0.9854', 'num_tokens': '1.728e+07', 'mean_token_accuracy': '0.7578', 'epoch': '0.0282'}
{'loss': '1.073', 'grad_norm': '0.514', 'learning_rate': '0.0001756', 'entropy': '1.085', 'num_tokens': '1.728e+07', 'mean_token_accuracy': '0.7309', 'epoch': '0.02821'}
{'loss': '1.092', 'grad_norm': '0.447', 'learning_rate': '0.0001756', 'entropy': '1.059', 'num_tokens': '1.728e+07', 'mean_token_accuracy': '0.739', 'epoch': '0.02822'}
{'loss': '0.995', 'grad_norm': '0.5045', 'learning_rate': '0.0001756', 'entropy': '0.8876', 'num_tokens': '1.729e+07', 'mean_token_accuracy': '0.7743', 'epoch': '0.02823'}
{'loss': '0.7975', 'grad_norm': '0.335', 'learning_rate': '0.0001756', 'entropy': '0.8106', 'num_tokens': '1.73e+07', 'mean_token_accuracy': '0.7983', 'epoch': '0.02824'}
{'loss': '0.816', 'grad_norm': '0.2975', 'learning_rate': '0.0001756', 'entropy': '0.8544', 'num_tokens': '1.73e+07', 'mean_token_accuracy': '0.7875', 'epoch': '0.02825'}
{'loss': '1.187', 'grad_norm': '0.4322', 'learning_rate': '0.0001756', 'entropy': '1.155', 'num_tokens': '1.731e+07', 'mean_token_accuracy': '0.7278', 'epoch': '0.02826'}
{'loss': '0.5983', 'grad_norm': '0.2426', 'learning_rate': '0.0001755', 'entropy': '0.7119', 'num_tokens': '1.732e+07', 'mean_token_accuracy': '0.8277', 'epoch': '0.02827'}
{'loss': '1.057', 'grad_norm': '0.3574', 'learning_rate': '0.0001755', 'entropy': '1.094', 'num_tokens': '1.732e+07', 'mean_token_accuracy': '0.7536', 'epoch': '0.02828'}
{'loss': '0.8251', 'grad_norm': '0.3691', 'learning_rate': '0.0001755', 'entropy': '0.8102', 'num_tokens': '1.733e+07', 'mean_token_accuracy': '0.8029', 'epoch': '0.02828'}
{'loss': '0.7806', 'grad_norm': '0.4527', 'learning_rate': '0.0001755', 'entropy': '0.7801', 'num_tokens': '1.733e+07', 'mean_token_accuracy': '0.8125', 'epoch': '0.02829'}
{'loss': '0.8943', 'grad_norm': '0.4675', 'learning_rate': '0.0001755', 'entropy': '0.8327', 'num_tokens': '1.734e+07', 'mean_token_accuracy': '0.8089', 'epoch': '0.0283'}
{'loss': '0.915', 'grad_norm': '0.3691', 'learning_rate': '0.0001754', 'entropy': '0.9398', 'num_tokens': '1.735e+07', 'mean_token_accuracy': '0.7766', 'epoch': '0.02831'}

 27%|██▋       | 3200/12000 [1:54:52<5:07:21,  2.10s/it]
 27%|██▋       | 3201/12000 [1:54:54<5:32:09,  2.26s/it]
                                                        

 27%|██▋       | 3201/12000 [1:54:54<5:32:09,  2.26s/it]
 27%|██▋       | 3202/12000 [1:54:56<4:59:25,  2.04s/it]
                                                        

 27%|██▋       | 3202/12000 [1:54:56<4:59:25,  2.04s/it]
 27%|██▋       | 3203/12000 [1:54:57<4:27:49,  1.83s/it]
                                                        

 27%|██▋       | 3203/12000 [1:54:57<4:27:49,  1.83s/it]
 27%|██▋       | 3204/12000 [1:54:59<4:21:37,  1.78s/it]
                                                        

 27%|██▋       | 3204/12000 [1:54:59<4:21:37,  1.78s/it]
 27%|██▋       | 3205/12000 [1:55:00<4:15:32,  1.74s/it]
                                                        

 27%|██▋       | 3205/12000 [1:55:00<4:15:32,  1.74s/it]
 27%|██▋       | 3206/12000 [1:55:04<5:17:45,  2.17s/it]
                                                        

 27%|██▋       | 3206/12000 [1:55:04<5:17:45,  2.17s/it]
 27%|██▋       | 3207/12000 [1:55:10<8:18:23,  3.40s/it]
                                                        

 27%|██▋       | 3207/12000 [1:55:10<8:18:23,  3.40s/it]
 27%|██▋       | 3208/12000 [1:55:11<6:55:05,  2.83s/it]
                                                        

 27%|██▋       | 3208/12000 [1:55:11<6:55:05,  2.83s/it]
 27%|██▋       | 3209/12000 [1:55:13<6:13:50,  2.55s/it]
                                                        

 27%|██▋       | 3209/12000 [1:55:13<6:13:50,  2.55s/it]
 27%|██▋       | 3210/12000 [1:55:15<5:30:34,  2.26s/it]
                                                        

 27%|██▋       | 3210/12000 [1:55:15<5:30:34,  2.26s/it]
 27%|██▋       | 3211/12000 [1:55:17<5:33:25,  2.28s/it]
                                                        

 27%|██▋       | 3211/12000 [1:55:17<5:33:25,  2.28s/it]
 27%|██▋       | 3212/12000 [1:55:19<4:56:33,  2.02s/it]
                                                        

 27%|██▋       | 3212/12000 [1:55:19<4:56:33,  2.02s/it]
 27%|██▋       | 3213/12000 [1:55:21<5:06:58,  2.10s/it]
                                                        

 27%|██▋       | 3213/12000 [1:55:21<5:06:58,  2.10s/it]
 27%|██▋       | 3214/12000 [1:55:22<4:44:09,  1.94s/it]
                                                        

 27%|██▋       | 3214/12000 [1:55:22<4:44:09,  1.94s/it]
 27%|██▋       | 3215/12000 [1:55:24<4:28:11,  1.83s/it]
                                                        

 27%|██▋       | 3215/12000 [1:55:24<4:28:11,  1.83s/it]
 27%|██▋       | 3216/12000 [1:55:26<4:26:34,  1.82s/it]
                                                        

 27%|██▋       | 3216/12000 [1:55:26<4:26:34,  1.82s/it]
 27%|██▋       | 3217/12000 [1:55:28<4:57:08,  2.03s/it]
                                                        

 27%|██▋       | 3217/12000 [1:55:28<4:57:08,  2.03s/it]
 27%|██▋       | 3218/12000 [1:55:30<4:39:22,  1.91s/it]
                                                        

 27%|██▋       | 3218/12000 [1:55:30<4:39:22,  1.91s/it]
 27%|██▋       | 3219/12000 [1:55:32<4:24:53,  1.81s/it]
                                                        

 27%|██▋       | 3219/12000 [1:55:32<4:24:53,  1.81s/it]
 27%|██▋       | 3220/12000 [1:55:33<4:12:23,  1.72s/it]
                                                        

 27%|██▋       | 3220/12000 [1:55:33<4:12:23,  1.72s/it]
 27%|██▋       | 3221/12000 [1:55:37<5:31:41,  2.27s/it]
                                                        

 27%|██▋       | 3221/12000 [1:55:37<5:31:41,  2.27s/it]
 27%|██▋       | 3222/12000 [1:55:38<4:55:28,  2.02s/it]
                                                        

 27%|██▋       | 3222/12000 [1:55:38<4:55:28,  2.02s/it]
 27%|██▋       | 3223/12000 [1:55:41<5:51:27,  2.40s/it]
                                                        

 27%|██▋       | 3223/12000 [1:55:41<5:51:27,  2.40s/it]
 27%|██▋       | 3224/12000 [1:55:44<5:52:53,  2.41s/it]
                                                        

 27%|██▋       | 3224/12000 [1:55:44<5:52:53,  2.41s/it]
 27%|██▋       | 3225/12000 [1:55:45<5:17:34,  2.17s/it]
                                                        

 27%|██▋       | 3225/12000 [1:55:45<5:17:34,  2.17s/it]
 27%|██▋       | 3226/12000 [1:55:48<5:17:06,  2.17s/it]
                                                        

 27%|██▋       | 3226/12000 [1:55:48<5:17:06,  2.17s/it]
 27%|██▋       | 3227/12000 [1:55:49<4:47:59,  1.97s/it]
                                                        

 27%|██▋       | 3227/12000 [1:55:49<4:47:59,  1.97s/it]
 27%|██▋       | 3228/12000 [1:55:51<4:40:08,  1.92s/it]
                                                        

 27%|██▋       | 3228/12000 [1:55:51<4:40:08,  1.92s/it]
 27%|██▋       | 3229/12000 [1:55:55<6:13:19,  2.55s/it]
                                                        

 27%|██▋       | 3229/12000 [1:55:55<6:13:19,  2.55s/it]
 27%|██▋       | 3230/12000 [1:55:56<5:23:11,  2.21s/it]
                                                        

 27%|██▋       | 3230/12000 [1:55:56<5:23:11,  2.21s/it]
 27%|██▋       | 3231/12000 [1:55:58<5:04:56,  2.09s/it]
                                                        

 27%|██▋       | 3231/12000 [1:55:58<5:04:56,  2.09s/it]
 27%|██▋       | 3232/12000 [1:56:02<6:35:06,  2.70s/it]
                                                        

 27%|██▋       | 3232/12000 [1:56:02<6:35:06,  2.70s/it]
 27%|██▋       | 3233/12000 [1:56:04<5:37:27,  2.31s/it]
                                                        

 27%|██▋       | 3233/12000 [1:56:04<5:37:27,  2.31s/it]
 27%|██▋       | 3234/12000 [1:56:08<7:27:01,  3.06s/it]
                                                        

 27%|██▋       | 3234/12000 [1:56:08<7:27:01,  3.06s/it]
 27%|██▋       | 3235/12000 [1:56:10<6:36:41,  2.72s/it]
                                                        

 27%|██▋       | 3235/12000 [1:56:10<6:36:41,  2.72s/it]
 27%|██▋       | 3236/12000 [1:56:14<7:08:30,  2.93s/it]
                                                        

 27%|██▋       | 3236/12000 [1:56:14<7:08:30,  2.93s/it]
 27%|██▋       | 3237/12000 [1:56:15<6:03:59,  2.49s/it]
                                                        

 27%|██▋       | 3237/12000 [1:56:15<6:03:59,  2.49s/it]
 27%|██▋       | 3238/12000 [1:56:18<6:24:50,  2.64s/it]
                                                        

 27%|██▋       | 3238/12000 [1:56:18<6:24:50,  2.64s/it]
 27%|██▋       | 3239/12000 [1:56:20<5:32:48,  2.28s/it]
                                                        

 27%|██▋       | 3239/12000 [1:56:20<5:32:48,  2.28s/it]
 27%|██▋       | 3240/12000 [1:56:22<5:25:34,  2.23s/it]
                                                        

 27%|██▋       | 3240/12000 [1:56:22<5:25:34,  2.23s/it]
 27%|██▋       | 3241/12000 [1:56:24<5:23:42,  2.22s/it]
                                                        

 27%|██▋       | 3241/12000 [1:56:24<5:23:42,  2.22s/it]
 27%|██▋       | 3242/12000 [1:56:25<4:46:54,  1.97s/it]
                                                        

 27%|██▋       | 3242/12000 [1:56:25<4:46:54,  1.97s/it]
 27%|██▋       | 3243/12000 [1:56:27<4:35:40,  1.89s/it]
                                                        

 27%|██▋       | 3243/12000 [1:56:27<4:35:40,  1.89s/it]
 27%|██▋       | 3244/12000 [1:56:32<6:55:59,  2.85s/it]
                                                        

 27%|██▋       | 3244/12000 [1:56:32<6:55:59,  2.85s/it]
 27%|██▋       | 3245/12000 [1:56:34<6:26:03,  2.65s/it]
                                                        

 27%|██▋       | 3245/12000 [1:56:34<6:26:03,  2.65s/it]
 27%|██▋       | 3246/12000 [1:56:36<5:46:52,  2.38s/it]
                                                        

 27%|██▋       | 3246/12000 [1:56:36<5:46:52,  2.38s/it]
 27%|██▋       | 3247/12000 [1:56:38<5:14:46,  2.16s/it]
                                                        
{'loss': '1.084', 'grad_norm': '0.4103', 'learning_rate': '0.0001754', 'entropy': '1.092', 'num_tokens': '1.735e+07', 'mean_token_accuracy': '0.7414', 'epoch': '0.02832'}
{'loss': '0.8751', 'grad_norm': '0.3877', 'learning_rate': '0.0001754', 'entropy': '0.9309', 'num_tokens': '1.736e+07', 'mean_token_accuracy': '0.7724', 'epoch': '0.02833'}
{'loss': '1', 'grad_norm': '0.4253', 'learning_rate': '0.0001754', 'entropy': '0.9429', 'num_tokens': '1.736e+07', 'mean_token_accuracy': '0.7582', 'epoch': '0.02834'}
{'loss': '1.097', 'grad_norm': '0.4399', 'learning_rate': '0.0001754', 'entropy': '1.057', 'num_tokens': '1.736e+07', 'mean_token_accuracy': '0.7459', 'epoch': '0.02835'}
{'loss': '0.9674', 'grad_norm': '0.4283', 'learning_rate': '0.0001754', 'entropy': '1.004', 'num_tokens': '1.737e+07', 'mean_token_accuracy': '0.7487', 'epoch': '0.02835'}
{'loss': '0.8067', 'grad_norm': '0.3522', 'learning_rate': '0.0001753', 'entropy': '0.7866', 'num_tokens': '1.737e+07', 'mean_token_accuracy': '0.7933', 'epoch': '0.02836'}
{'loss': '1.14', 'grad_norm': '0.4138', 'learning_rate': '0.0001753', 'entropy': '1.058', 'num_tokens': '1.738e+07', 'mean_token_accuracy': '0.748', 'epoch': '0.02837'}
{'loss': '0.8551', 'grad_norm': '0.3192', 'learning_rate': '0.0001753', 'entropy': '0.8735', 'num_tokens': '1.739e+07', 'mean_token_accuracy': '0.7838', 'epoch': '0.02838'}
{'loss': '0.8287', 'grad_norm': '0.4308', 'learning_rate': '0.0001753', 'entropy': '0.8245', 'num_tokens': '1.739e+07', 'mean_token_accuracy': '0.7835', 'epoch': '0.02839'}
{'loss': '0.8204', 'grad_norm': '0.3205', 'learning_rate': '0.0001753', 'entropy': '0.8121', 'num_tokens': '1.74e+07', 'mean_token_accuracy': '0.796', 'epoch': '0.0284'}
{'loss': '0.7836', 'grad_norm': '0.3869', 'learning_rate': '0.0001752', 'entropy': '0.8239', 'num_tokens': '1.74e+07', 'mean_token_accuracy': '0.7843', 'epoch': '0.02841'}
{'loss': '1.213', 'grad_norm': '0.5049', 'learning_rate': '0.0001752', 'entropy': '1.15', 'num_tokens': '1.741e+07', 'mean_token_accuracy': '0.7101', 'epoch': '0.02842'}
{'loss': '0.6394', 'grad_norm': '0.2748', 'learning_rate': '0.0001752', 'entropy': '0.6891', 'num_tokens': '1.741e+07', 'mean_token_accuracy': '0.8174', 'epoch': '0.02843'}
{'loss': '1.154', 'grad_norm': '0.4527', 'learning_rate': '0.0001752', 'entropy': '1.171', 'num_tokens': '1.742e+07', 'mean_token_accuracy': '0.7195', 'epoch': '0.02843'}
{'loss': '0.8647', 'grad_norm': '0.4749', 'learning_rate': '0.0001752', 'entropy': '0.8822', 'num_tokens': '1.742e+07', 'mean_token_accuracy': '0.7724', 'epoch': '0.02844'}
{'loss': '1.088', 'grad_norm': '0.4138', 'learning_rate': '0.0001752', 'entropy': '1.107', 'num_tokens': '1.743e+07', 'mean_token_accuracy': '0.7286', 'epoch': '0.02845'}
{'loss': '0.781', 'grad_norm': '0.2989', 'learning_rate': '0.0001751', 'entropy': '0.8204', 'num_tokens': '1.743e+07', 'mean_token_accuracy': '0.8007', 'epoch': '0.02846'}
{'loss': '0.8793', 'grad_norm': '0.341', 'learning_rate': '0.0001751', 'entropy': '1.002', 'num_tokens': '1.744e+07', 'mean_token_accuracy': '0.7644', 'epoch': '0.02847'}
{'loss': '0.8805', 'grad_norm': '0.3948', 'learning_rate': '0.0001751', 'entropy': '0.9402', 'num_tokens': '1.745e+07', 'mean_token_accuracy': '0.777', 'epoch': '0.02848'}
{'loss': '0.7893', 'grad_norm': '0.3874', 'learning_rate': '0.0001751', 'entropy': '0.8259', 'num_tokens': '1.745e+07', 'mean_token_accuracy': '0.7873', 'epoch': '0.02849'}
{'loss': '0.8815', 'grad_norm': '0.3853', 'learning_rate': '0.0001751', 'entropy': '0.8832', 'num_tokens': '1.745e+07', 'mean_token_accuracy': '0.7831', 'epoch': '0.0285'}
{'loss': '0.892', 'grad_norm': '0.3841', 'learning_rate': '0.000175', 'entropy': '0.8771', 'num_tokens': '1.746e+07', 'mean_token_accuracy': '0.784', 'epoch': '0.02851'}
{'loss': '0.9454', 'grad_norm': '0.4552', 'learning_rate': '0.000175', 'entropy': '0.9721', 'num_tokens': '1.747e+07', 'mean_token_accuracy': '0.7531', 'epoch': '0.02851'}
{'loss': '1.244', 'grad_norm': '0.425', 'learning_rate': '0.000175', 'entropy': '1.062', 'num_tokens': '1.747e+07', 'mean_token_accuracy': '0.7606', 'epoch': '0.02852'}
{'loss': '0.5618', 'grad_norm': '0.2625', 'learning_rate': '0.000175', 'entropy': '0.604', 'num_tokens': '1.748e+07', 'mean_token_accuracy': '0.8462', 'epoch': '0.02853'}
{'loss': '0.8325', 'grad_norm': '0.3797', 'learning_rate': '0.000175', 'entropy': '0.8484', 'num_tokens': '1.749e+07', 'mean_token_accuracy': '0.7924', 'epoch': '0.02854'}
{'loss': '1.49', 'grad_norm': '0.4369', 'learning_rate': '0.000175', 'entropy': '1.399', 'num_tokens': '1.749e+07', 'mean_token_accuracy': '0.6712', 'epoch': '0.02855'}
{'loss': '0.7575', 'grad_norm': '0.3459', 'learning_rate': '0.0001749', 'entropy': '0.8237', 'num_tokens': '1.75e+07', 'mean_token_accuracy': '0.7848', 'epoch': '0.02856'}
{'loss': '0.9755', 'grad_norm': '0.455', 'learning_rate': '0.0001749', 'entropy': '0.9028', 'num_tokens': '1.75e+07', 'mean_token_accuracy': '0.7702', 'epoch': '0.02857'}
{'loss': '0.7562', 'grad_norm': '0.3861', 'learning_rate': '0.0001749', 'entropy': '0.7846', 'num_tokens': '1.751e+07', 'mean_token_accuracy': '0.8013', 'epoch': '0.02858'}
{'loss': '1.308', 'grad_norm': '0.4086', 'learning_rate': '0.0001749', 'entropy': '1.149', 'num_tokens': '1.751e+07', 'mean_token_accuracy': '0.7235', 'epoch': '0.02858'}
{'loss': '1.066', 'grad_norm': '0.4351', 'learning_rate': '0.0001749', 'entropy': '1.003', 'num_tokens': '1.751e+07', 'mean_token_accuracy': '0.7534', 'epoch': '0.02859'}
{'loss': '1.247', 'grad_norm': '0.4544', 'learning_rate': '0.0001748', 'entropy': '1.028', 'num_tokens': '1.752e+07', 'mean_token_accuracy': '0.7573', 'epoch': '0.0286'}
{'loss': '0.7886', 'grad_norm': '0.3627', 'learning_rate': '0.0001748', 'entropy': '0.8327', 'num_tokens': '1.753e+07', 'mean_token_accuracy': '0.7998', 'epoch': '0.02861'}
{'loss': '0.9233', 'grad_norm': '0.4075', 'learning_rate': '0.0001748', 'entropy': '0.8315', 'num_tokens': '1.754e+07', 'mean_token_accuracy': '0.7922', 'epoch': '0.02862'}
{'loss': '1.053', 'grad_norm': '0.3931', 'learning_rate': '0.0001748', 'entropy': '1.066', 'num_tokens': '1.754e+07', 'mean_token_accuracy': '0.7487', 'epoch': '0.02863'}
{'loss': '1.047', 'grad_norm': '0.4106', 'learning_rate': '0.0001748', 'entropy': '0.9523', 'num_tokens': '1.755e+07', 'mean_token_accuracy': '0.7673', 'epoch': '0.02864'}
{'loss': '0.6026', 'grad_norm': '0.2929', 'learning_rate': '0.0001748', 'entropy': '0.6354', 'num_tokens': '1.755e+07', 'mean_token_accuracy': '0.8334', 'epoch': '0.02865'}
{'loss': '1.266', 'grad_norm': '0.6575', 'learning_rate': '0.0001747', 'entropy': '1.202', 'num_tokens': '1.756e+07', 'mean_token_accuracy': '0.722', 'epoch': '0.02866'}
{'loss': '0.8348', 'grad_norm': '0.401', 'learning_rate': '0.0001747', 'entropy': '0.886', 'num_tokens': '1.756e+07', 'mean_token_accuracy': '0.7782', 'epoch': '0.02866'}
{'loss': '0.7956', 'grad_norm': '0.4404', 'learning_rate': '0.0001747', 'entropy': '0.8055', 'num_tokens': '1.757e+07', 'mean_token_accuracy': '0.7818', 'epoch': '0.02867'}
{'loss': '0.887', 'grad_norm': '0.5741', 'learning_rate': '0.0001747', 'entropy': '0.9121', 'num_tokens': '1.757e+07', 'mean_token_accuracy': '0.7701', 'epoch': '0.02868'}
{'loss': '0.7134', 'grad_norm': '0.375', 'learning_rate': '0.0001747', 'entropy': '0.715', 'num_tokens': '1.758e+07', 'mean_token_accuracy': '0.8195', 'epoch': '0.02869'}
{'loss': '0.8388', 'grad_norm': '0.3742', 'learning_rate': '0.0001746', 'entropy': '0.8619', 'num_tokens': '1.758e+07', 'mean_token_accuracy': '0.7859', 'epoch': '0.0287'}
{'loss': '1.132', 'grad_norm': '0.3664', 'learning_rate': '0.0001746', 'entropy': '0.9347', 'num_tokens': '1.759e+07', 'mean_token_accuracy': '0.7699', 'epoch': '0.02871'}
{'loss': '0.927', 'grad_norm': '0.4107', 'learning_rate': '0.0001746', 'entropy': '0.9536', 'num_tokens': '1.76e+07', 'mean_token_accuracy': '0.7603', 'epoch': '0.02872'}
{'loss': '1.224', 'grad_norm': '0.4243', 'learning_rate': '0.0001746', 'entropy': '1.168', 'num_tokens': '1.76e+07', 'mean_token_accuracy': '0.7203', 'epoch': '0.02873'}

 27%|██▋       | 3247/12000 [1:56:38<5:14:46,  2.16s/it]
 27%|██▋       | 3248/12000 [1:56:40<5:06:03,  2.10s/it]
                                                        

 27%|██▋       | 3248/12000 [1:56:40<5:06:03,  2.10s/it]
 27%|██▋       | 3249/12000 [1:56:41<4:34:22,  1.88s/it]
                                                        

 27%|██▋       | 3249/12000 [1:56:41<4:34:22,  1.88s/it]
 27%|██▋       | 3250/12000 [1:56:43<4:19:32,  1.78s/it]
                                                        

 27%|██▋       | 3250/12000 [1:56:43<4:19:32,  1.78s/it]
 27%|██▋       | 3251/12000 [1:56:48<7:04:55,  2.91s/it]
                                                        

 27%|██▋       | 3251/12000 [1:56:48<7:04:55,  2.91s/it]
 27%|██▋       | 3252/12000 [1:56:50<6:24:23,  2.64s/it]
                                                        

 27%|██▋       | 3252/12000 [1:56:50<6:24:23,  2.64s/it]
 27%|██▋       | 3253/12000 [1:56:52<5:54:26,  2.43s/it]
                                                        

 27%|██▋       | 3253/12000 [1:56:52<5:54:26,  2.43s/it]
 27%|██▋       | 3254/12000 [1:56:55<5:53:01,  2.42s/it]
                                                        

 27%|██▋       | 3254/12000 [1:56:55<5:53:01,  2.42s/it]
 27%|██▋       | 3255/12000 [1:56:56<5:15:23,  2.16s/it]
                                                        

 27%|██▋       | 3255/12000 [1:56:56<5:15:23,  2.16s/it]
 27%|██▋       | 3256/12000 [1:56:58<4:53:07,  2.01s/it]
                                                        

 27%|██▋       | 3256/12000 [1:56:58<4:53:07,  2.01s/it]
 27%|██▋       | 3257/12000 [1:57:00<5:01:55,  2.07s/it]
                                                        

 27%|██▋       | 3257/12000 [1:57:00<5:01:55,  2.07s/it]
 27%|██▋       | 3258/12000 [1:57:03<5:25:39,  2.24s/it]
                                                        

 27%|██▋       | 3258/12000 [1:57:03<5:25:39,  2.24s/it]
 27%|██▋       | 3259/12000 [1:57:04<4:50:32,  1.99s/it]
                                                        

 27%|██▋       | 3259/12000 [1:57:04<4:50:32,  1.99s/it]
 27%|██▋       | 3260/12000 [1:57:06<4:35:20,  1.89s/it]
                                                        

 27%|██▋       | 3260/12000 [1:57:06<4:35:20,  1.89s/it]
 27%|██▋       | 3261/12000 [1:57:07<4:14:44,  1.75s/it]
                                                        

 27%|██▋       | 3261/12000 [1:57:07<4:14:44,  1.75s/it]
 27%|██▋       | 3262/12000 [1:57:12<6:43:25,  2.77s/it]
                                                        

 27%|██▋       | 3262/12000 [1:57:12<6:43:25,  2.77s/it]
 27%|██▋       | 3263/12000 [1:57:14<5:58:14,  2.46s/it]
                                                        

 27%|██▋       | 3263/12000 [1:57:14<5:58:14,  2.46s/it]
 27%|██▋       | 3264/12000 [1:57:16<5:48:54,  2.40s/it]
                                                        

 27%|██▋       | 3264/12000 [1:57:16<5:48:54,  2.40s/it]
 27%|██▋       | 3265/12000 [1:57:18<5:27:35,  2.25s/it]
                                                        

 27%|██▋       | 3265/12000 [1:57:18<5:27:35,  2.25s/it]
 27%|██▋       | 3266/12000 [1:57:21<6:00:35,  2.48s/it]
                                                        

 27%|██▋       | 3266/12000 [1:57:21<6:00:35,  2.48s/it]
 27%|██▋       | 3267/12000 [1:57:23<5:29:50,  2.27s/it]
                                                        

 27%|██▋       | 3267/12000 [1:57:23<5:29:50,  2.27s/it]
 27%|██▋       | 3268/12000 [1:57:25<5:31:26,  2.28s/it]
                                                        

 27%|██▋       | 3268/12000 [1:57:25<5:31:26,  2.28s/it]
 27%|██▋       | 3269/12000 [1:57:27<5:19:30,  2.20s/it]
                                                        

 27%|██▋       | 3269/12000 [1:57:27<5:19:30,  2.20s/it]
 27%|██▋       | 3270/12000 [1:57:29<4:48:37,  1.98s/it]
                                                        

 27%|██▋       | 3270/12000 [1:57:29<4:48:37,  1.98s/it]
 27%|██▋       | 3271/12000 [1:57:30<4:24:25,  1.82s/it]
                                                        

 27%|██▋       | 3271/12000 [1:57:30<4:24:25,  1.82s/it]
 27%|██▋       | 3272/12000 [1:57:32<4:36:10,  1.90s/it]
                                                        

 27%|██▋       | 3272/12000 [1:57:32<4:36:10,  1.90s/it]
 27%|██▋       | 3273/12000 [1:57:38<7:04:05,  2.92s/it]
                                                        

 27%|██▋       | 3273/12000 [1:57:38<7:04:05,  2.92s/it]
 27%|██▋       | 3274/12000 [1:57:39<6:07:05,  2.52s/it]
                                                        

 27%|██▋       | 3274/12000 [1:57:39<6:07:05,  2.52s/it]
 27%|██▋       | 3275/12000 [1:57:42<6:09:32,  2.54s/it]
                                                        

 27%|██▋       | 3275/12000 [1:57:42<6:09:32,  2.54s/it]
 27%|██▋       | 3276/12000 [1:57:43<5:31:04,  2.28s/it]
                                                        

 27%|██▋       | 3276/12000 [1:57:43<5:31:04,  2.28s/it]
 27%|██▋       | 3277/12000 [1:57:45<5:11:30,  2.14s/it]
                                                        

 27%|██▋       | 3277/12000 [1:57:45<5:11:30,  2.14s/it]
 27%|██▋       | 3278/12000 [1:57:47<4:51:19,  2.00s/it]
                                                        

 27%|██▋       | 3278/12000 [1:57:47<4:51:19,  2.00s/it]
 27%|██▋       | 3279/12000 [1:57:48<4:30:53,  1.86s/it]
                                                        

 27%|██▋       | 3279/12000 [1:57:48<4:30:53,  1.86s/it]
 27%|██▋       | 3280/12000 [1:57:50<4:11:18,  1.73s/it]
                                                        

 27%|██▋       | 3280/12000 [1:57:50<4:11:18,  1.73s/it]
 27%|██▋       | 3281/12000 [1:57:51<4:05:19,  1.69s/it]
                                                        

 27%|██▋       | 3281/12000 [1:57:51<4:05:19,  1.69s/it]
 27%|██▋       | 3282/12000 [1:57:53<4:09:00,  1.71s/it]
                                                        

 27%|██▋       | 3282/12000 [1:57:53<4:09:00,  1.71s/it]
 27%|██▋       | 3283/12000 [1:57:55<4:11:01,  1.73s/it]
                                                        

 27%|██▋       | 3283/12000 [1:57:55<4:11:01,  1.73s/it]
 27%|██▋       | 3284/12000 [1:57:57<4:05:34,  1.69s/it]
                                                        

 27%|██▋       | 3284/12000 [1:57:57<4:05:34,  1.69s/it]
 27%|██▋       | 3285/12000 [1:57:58<4:11:07,  1.73s/it]
                                                        

 27%|██▋       | 3285/12000 [1:57:58<4:11:07,  1.73s/it]
 27%|██▋       | 3286/12000 [1:58:01<4:51:20,  2.01s/it]
                                                        

 27%|██▋       | 3286/12000 [1:58:01<4:51:20,  2.01s/it]
 27%|██▋       | 3287/12000 [1:58:03<4:37:51,  1.91s/it]
                                                        

 27%|██▋       | 3287/12000 [1:58:03<4:37:51,  1.91s/it]
 27%|██▋       | 3288/12000 [1:58:05<4:36:53,  1.91s/it]
                                                        

 27%|██▋       | 3288/12000 [1:58:05<4:36:53,  1.91s/it]
 27%|██▋       | 3289/12000 [1:58:07<4:44:29,  1.96s/it]
                                                        

 27%|██▋       | 3289/12000 [1:58:07<4:44:29,  1.96s/it]
 27%|██▋       | 3290/12000 [1:58:09<4:45:23,  1.97s/it]
                                                        

 27%|██▋       | 3290/12000 [1:58:09<4:45:23,  1.97s/it]
 27%|██▋       | 3291/12000 [1:58:11<4:47:13,  1.98s/it]
                                                        

 27%|██▋       | 3291/12000 [1:58:11<4:47:13,  1.98s/it]
 27%|██▋       | 3292/12000 [1:58:12<4:28:11,  1.85s/it]
                                                        

 27%|██▋       | 3292/12000 [1:58:12<4:28:11,  1.85s/it]
 27%|██▋       | 3293/12000 [1:58:15<5:02:11,  2.08s/it]
                                                        

 27%|██▋       | 3293/12000 [1:58:15<5:02:11,  2.08s/it]
 27%|██▋       | 3294/12000 [1:58:16<4:39:33,  1.93s/it]
                                                        
{'loss': '0.8985', 'grad_norm': '0.4772', 'learning_rate': '0.0001746', 'entropy': '0.902', 'num_tokens': '1.761e+07', 'mean_token_accuracy': '0.7725', 'epoch': '0.02874'}
{'loss': '1.122', 'grad_norm': '0.3686', 'learning_rate': '0.0001746', 'entropy': '1.027', 'num_tokens': '1.761e+07', 'mean_token_accuracy': '0.7509', 'epoch': '0.02874'}
{'loss': '0.7643', 'grad_norm': '0.3674', 'learning_rate': '0.0001745', 'entropy': '0.8594', 'num_tokens': '1.762e+07', 'mean_token_accuracy': '0.7826', 'epoch': '0.02875'}
{'loss': '0.7043', 'grad_norm': '0.4097', 'learning_rate': '0.0001745', 'entropy': '0.7966', 'num_tokens': '1.762e+07', 'mean_token_accuracy': '0.8085', 'epoch': '0.02876'}
{'loss': '0.9074', 'grad_norm': '0.352', 'learning_rate': '0.0001745', 'entropy': '0.929', 'num_tokens': '1.763e+07', 'mean_token_accuracy': '0.7485', 'epoch': '0.02877'}
{'loss': '0.8145', 'grad_norm': '0.3442', 'learning_rate': '0.0001745', 'entropy': '0.7942', 'num_tokens': '1.763e+07', 'mean_token_accuracy': '0.7876', 'epoch': '0.02878'}
{'loss': '1.241', 'grad_norm': '0.3899', 'learning_rate': '0.0001745', 'entropy': '1.266', 'num_tokens': '1.764e+07', 'mean_token_accuracy': '0.698', 'epoch': '0.02879'}
{'loss': '0.8778', 'grad_norm': '0.299', 'learning_rate': '0.0001744', 'entropy': '1.022', 'num_tokens': '1.764e+07', 'mean_token_accuracy': '0.7441', 'epoch': '0.0288'}
{'loss': '0.788', 'grad_norm': '0.3736', 'learning_rate': '0.0001744', 'entropy': '0.7731', 'num_tokens': '1.765e+07', 'mean_token_accuracy': '0.8026', 'epoch': '0.02881'}
{'loss': '0.9839', 'grad_norm': '0.4523', 'learning_rate': '0.0001744', 'entropy': '1.001', 'num_tokens': '1.765e+07', 'mean_token_accuracy': '0.7423', 'epoch': '0.02882'}
{'loss': '0.7436', 'grad_norm': '0.3545', 'learning_rate': '0.0001744', 'entropy': '0.7415', 'num_tokens': '1.766e+07', 'mean_token_accuracy': '0.8126', 'epoch': '0.02882'}
{'loss': '0.7816', 'grad_norm': '0.3119', 'learning_rate': '0.0001744', 'entropy': '0.8505', 'num_tokens': '1.767e+07', 'mean_token_accuracy': '0.778', 'epoch': '0.02883'}
{'loss': '1.048', 'grad_norm': '0.4493', 'learning_rate': '0.0001744', 'entropy': '1.05', 'num_tokens': '1.767e+07', 'mean_token_accuracy': '0.7501', 'epoch': '0.02884'}
{'loss': '1.252', 'grad_norm': '0.4754', 'learning_rate': '0.0001743', 'entropy': '1.154', 'num_tokens': '1.767e+07', 'mean_token_accuracy': '0.7208', 'epoch': '0.02885'}
{'loss': '0.8119', 'grad_norm': '0.3166', 'learning_rate': '0.0001743', 'entropy': '0.8255', 'num_tokens': '1.768e+07', 'mean_token_accuracy': '0.7848', 'epoch': '0.02886'}
{'loss': '1.377', 'grad_norm': '0.4283', 'learning_rate': '0.0001743', 'entropy': '1.194', 'num_tokens': '1.769e+07', 'mean_token_accuracy': '0.7288', 'epoch': '0.02887'}
{'loss': '0.6945', 'grad_norm': '0.3227', 'learning_rate': '0.0001743', 'entropy': '0.6881', 'num_tokens': '1.769e+07', 'mean_token_accuracy': '0.8249', 'epoch': '0.02888'}
{'loss': '1.023', 'grad_norm': '0.329', 'learning_rate': '0.0001743', 'entropy': '0.9815', 'num_tokens': '1.77e+07', 'mean_token_accuracy': '0.754', 'epoch': '0.02889'}
{'loss': '1.198', 'grad_norm': '0.5037', 'learning_rate': '0.0001742', 'entropy': '1.184', 'num_tokens': '1.771e+07', 'mean_token_accuracy': '0.7166', 'epoch': '0.02889'}
{'loss': '0.684', 'grad_norm': '0.3503', 'learning_rate': '0.0001742', 'entropy': '0.6986', 'num_tokens': '1.771e+07', 'mean_token_accuracy': '0.8238', 'epoch': '0.0289'}
{'loss': '0.9758', 'grad_norm': '0.4579', 'learning_rate': '0.0001742', 'entropy': '0.9849', 'num_tokens': '1.772e+07', 'mean_token_accuracy': '0.754', 'epoch': '0.02891'}
{'loss': '0.7388', 'grad_norm': '0.3448', 'learning_rate': '0.0001742', 'entropy': '0.7727', 'num_tokens': '1.772e+07', 'mean_token_accuracy': '0.8112', 'epoch': '0.02892'}
{'loss': '1.16', 'grad_norm': '0.3994', 'learning_rate': '0.0001742', 'entropy': '1.222', 'num_tokens': '1.773e+07', 'mean_token_accuracy': '0.7091', 'epoch': '0.02893'}
{'loss': '0.7895', 'grad_norm': '0.3725', 'learning_rate': '0.0001741', 'entropy': '0.8485', 'num_tokens': '1.773e+07', 'mean_token_accuracy': '0.7925', 'epoch': '0.02894'}
{'loss': '0.6978', 'grad_norm': '0.344', 'learning_rate': '0.0001741', 'entropy': '0.7332', 'num_tokens': '1.774e+07', 'mean_token_accuracy': '0.7994', 'epoch': '0.02895'}
{'loss': '0.9008', 'grad_norm': '0.3462', 'learning_rate': '0.0001741', 'entropy': '0.854', 'num_tokens': '1.774e+07', 'mean_token_accuracy': '0.787', 'epoch': '0.02896'}
{'loss': '1.093', 'grad_norm': '0.348', 'learning_rate': '0.0001741', 'entropy': '1.04', 'num_tokens': '1.775e+07', 'mean_token_accuracy': '0.7557', 'epoch': '0.02897'}
{'loss': '0.655', 'grad_norm': '0.3429', 'learning_rate': '0.0001741', 'entropy': '0.6711', 'num_tokens': '1.775e+07', 'mean_token_accuracy': '0.8168', 'epoch': '0.02897'}
{'loss': '0.7914', 'grad_norm': '0.2852', 'learning_rate': '0.0001741', 'entropy': '0.7237', 'num_tokens': '1.776e+07', 'mean_token_accuracy': '0.8204', 'epoch': '0.02898'}
{'loss': '0.7398', 'grad_norm': '0.3303', 'learning_rate': '0.000174', 'entropy': '0.7499', 'num_tokens': '1.777e+07', 'mean_token_accuracy': '0.7998', 'epoch': '0.02899'}
{'loss': '0.9403', 'grad_norm': '0.4619', 'learning_rate': '0.000174', 'entropy': '0.87', 'num_tokens': '1.777e+07', 'mean_token_accuracy': '0.7736', 'epoch': '0.029'}
{'loss': '0.9542', 'grad_norm': '0.4113', 'learning_rate': '0.000174', 'entropy': '0.8773', 'num_tokens': '1.778e+07', 'mean_token_accuracy': '0.7809', 'epoch': '0.02901'}
{'loss': '0.6656', 'grad_norm': '0.3786', 'learning_rate': '0.000174', 'entropy': '0.6468', 'num_tokens': '1.778e+07', 'mean_token_accuracy': '0.8301', 'epoch': '0.02902'}
{'loss': '0.76', 'grad_norm': '0.4285', 'learning_rate': '0.000174', 'entropy': '0.7185', 'num_tokens': '1.779e+07', 'mean_token_accuracy': '0.8128', 'epoch': '0.02903'}
{'loss': '0.8137', 'grad_norm': '0.4399', 'learning_rate': '0.0001739', 'entropy': '0.7597', 'num_tokens': '1.779e+07', 'mean_token_accuracy': '0.802', 'epoch': '0.02904'}
{'loss': '0.9258', 'grad_norm': '0.3776', 'learning_rate': '0.0001739', 'entropy': '0.9415', 'num_tokens': '1.779e+07', 'mean_token_accuracy': '0.7649', 'epoch': '0.02905'}
{'loss': '0.9471', 'grad_norm': '0.4279', 'learning_rate': '0.0001739', 'entropy': '0.949', 'num_tokens': '1.78e+07', 'mean_token_accuracy': '0.7456', 'epoch': '0.02905'}
{'loss': '0.9193', 'grad_norm': '0.5162', 'learning_rate': '0.0001739', 'entropy': '1.001', 'num_tokens': '1.78e+07', 'mean_token_accuracy': '0.7522', 'epoch': '0.02906'}
{'loss': '0.9367', 'grad_norm': '0.4058', 'learning_rate': '0.0001739', 'entropy': '1.021', 'num_tokens': '1.781e+07', 'mean_token_accuracy': '0.7597', 'epoch': '0.02907'}
{'loss': '0.6572', 'grad_norm': '0.2908', 'learning_rate': '0.0001739', 'entropy': '0.7033', 'num_tokens': '1.782e+07', 'mean_token_accuracy': '0.8227', 'epoch': '0.02908'}
{'loss': '0.6465', 'grad_norm': '0.3516', 'learning_rate': '0.0001738', 'entropy': '0.7108', 'num_tokens': '1.782e+07', 'mean_token_accuracy': '0.8296', 'epoch': '0.02909'}
{'loss': '1.057', 'grad_norm': '0.4236', 'learning_rate': '0.0001738', 'entropy': '1.219', 'num_tokens': '1.783e+07', 'mean_token_accuracy': '0.7108', 'epoch': '0.0291'}
{'loss': '0.8237', 'grad_norm': '0.4176', 'learning_rate': '0.0001738', 'entropy': '0.8439', 'num_tokens': '1.783e+07', 'mean_token_accuracy': '0.7976', 'epoch': '0.02911'}
{'loss': '0.7033', 'grad_norm': '0.3595', 'learning_rate': '0.0001738', 'entropy': '0.6761', 'num_tokens': '1.784e+07', 'mean_token_accuracy': '0.8263', 'epoch': '0.02912'}
{'loss': '1.051', 'grad_norm': '0.387', 'learning_rate': '0.0001738', 'entropy': '1.194', 'num_tokens': '1.784e+07', 'mean_token_accuracy': '0.7104', 'epoch': '0.02912'}
{'loss': '0.6264', 'grad_norm': '0.327', 'learning_rate': '0.0001737', 'entropy': '0.6392', 'num_tokens': '1.785e+07', 'mean_token_accuracy': '0.8395', 'epoch': '0.02913'}
{'loss': '0.7808', 'grad_norm': '0.3103', 'learning_rate': '0.0001737', 'entropy': '0.7777', 'num_tokens': '1.785e+07', 'mean_token_accuracy': '0.8155', 'epoch': '0.02914'}

 27%|██▋       | 3294/12000 [1:58:16<4:39:33,  1.93s/it]
 27%|██▋       | 3295/12000 [1:58:20<6:00:55,  2.49s/it]
                                                        

 27%|██▋       | 3295/12000 [1:58:20<6:00:55,  2.49s/it]
 27%|██▋       | 3296/12000 [1:58:22<5:30:58,  2.28s/it]
                                                        

 27%|██▋       | 3296/12000 [1:58:22<5:30:58,  2.28s/it]
 27%|██▋       | 3297/12000 [1:58:24<5:01:58,  2.08s/it]
                                                        

 27%|██▋       | 3297/12000 [1:58:24<5:01:58,  2.08s/it]
 27%|██▋       | 3298/12000 [1:58:26<5:04:24,  2.10s/it]
                                                        

 27%|██▋       | 3298/12000 [1:58:26<5:04:24,  2.10s/it]
 27%|██▋       | 3299/12000 [1:58:27<4:38:58,  1.92s/it]
                                                        

 27%|██▋       | 3299/12000 [1:58:27<4:38:58,  1.92s/it]
 28%|██▊       | 3300/12000 [1:58:29<4:33:34,  1.89s/it]
                                                        

 28%|██▊       | 3300/12000 [1:58:29<4:33:34,  1.89s/it]
 28%|██▊       | 3301/12000 [1:58:32<5:33:44,  2.30s/it]
                                                        

 28%|██▊       | 3301/12000 [1:58:32<5:33:44,  2.30s/it]
 28%|██▊       | 3302/12000 [1:58:34<5:11:00,  2.15s/it]
                                                        

 28%|██▊       | 3302/12000 [1:58:34<5:11:00,  2.15s/it]
 28%|██▊       | 3303/12000 [1:58:36<4:48:45,  1.99s/it]
                                                        

 28%|██▊       | 3303/12000 [1:58:36<4:48:45,  1.99s/it]
 28%|██▊       | 3304/12000 [1:58:38<4:44:28,  1.96s/it]
                                                        

 28%|██▊       | 3304/12000 [1:58:38<4:44:28,  1.96s/it]
 28%|██▊       | 3305/12000 [1:58:39<4:34:14,  1.89s/it]
                                                        

 28%|██▊       | 3305/12000 [1:58:39<4:34:14,  1.89s/it]
 28%|██▊       | 3306/12000 [1:58:41<4:29:38,  1.86s/it]
                                                        

 28%|██▊       | 3306/12000 [1:58:41<4:29:38,  1.86s/it]
 28%|██▊       | 3307/12000 [1:58:43<4:16:19,  1.77s/it]
                                                        

 28%|██▊       | 3307/12000 [1:58:43<4:16:19,  1.77s/it]
 28%|██▊       | 3308/12000 [1:58:45<4:42:13,  1.95s/it]
                                                        

 28%|██▊       | 3308/12000 [1:58:45<4:42:13,  1.95s/it]
 28%|██▊       | 3309/12000 [1:58:47<4:41:20,  1.94s/it]
                                                        

 28%|██▊       | 3309/12000 [1:58:47<4:41:20,  1.94s/it]
 28%|██▊       | 3310/12000 [1:58:49<4:37:16,  1.91s/it]
                                                        

 28%|██▊       | 3310/12000 [1:58:49<4:37:16,  1.91s/it]
 28%|██▊       | 3311/12000 [1:58:50<4:23:59,  1.82s/it]
                                                        

 28%|██▊       | 3311/12000 [1:58:50<4:23:59,  1.82s/it]
 28%|██▊       | 3312/12000 [1:58:53<4:32:25,  1.88s/it]
                                                        

 28%|██▊       | 3312/12000 [1:58:53<4:32:25,  1.88s/it]
 28%|██▊       | 3313/12000 [1:58:55<4:53:06,  2.02s/it]
                                                        

 28%|██▊       | 3313/12000 [1:58:55<4:53:06,  2.02s/it]
 28%|██▊       | 3314/12000 [1:58:57<4:55:09,  2.04s/it]
                                                        

 28%|██▊       | 3314/12000 [1:58:57<4:55:09,  2.04s/it]
 28%|██▊       | 3315/12000 [1:58:59<4:46:21,  1.98s/it]
                                                        

 28%|██▊       | 3315/12000 [1:58:59<4:46:21,  1.98s/it]
 28%|██▊       | 3316/12000 [1:59:01<4:47:25,  1.99s/it]
                                                        

 28%|██▊       | 3316/12000 [1:59:01<4:47:25,  1.99s/it]
 28%|██▊       | 3317/12000 [1:59:05<6:07:19,  2.54s/it]
                                                        

 28%|██▊       | 3317/12000 [1:59:05<6:07:19,  2.54s/it]
 28%|██▊       | 3318/12000 [1:59:06<5:23:56,  2.24s/it]
                                                        

 28%|██▊       | 3318/12000 [1:59:06<5:23:56,  2.24s/it]
 28%|██▊       | 3319/12000 [1:59:08<5:13:14,  2.16s/it]
                                                        

 28%|██▊       | 3319/12000 [1:59:08<5:13:14,  2.16s/it]
 28%|██▊       | 3320/12000 [1:59:10<5:16:32,  2.19s/it]
                                                        

 28%|██▊       | 3320/12000 [1:59:10<5:16:32,  2.19s/it]
 28%|██▊       | 3321/12000 [1:59:12<5:00:53,  2.08s/it]
                                                        

 28%|██▊       | 3321/12000 [1:59:12<5:00:53,  2.08s/it]
 28%|██▊       | 3322/12000 [1:59:14<4:52:16,  2.02s/it]
                                                        

 28%|██▊       | 3322/12000 [1:59:14<4:52:16,  2.02s/it]
 28%|██▊       | 3323/12000 [1:59:19<6:51:35,  2.85s/it]
                                                        

 28%|██▊       | 3323/12000 [1:59:19<6:51:35,  2.85s/it]
 28%|██▊       | 3324/12000 [1:59:20<5:52:58,  2.44s/it]
                                                        

 28%|██▊       | 3324/12000 [1:59:20<5:52:58,  2.44s/it]
 28%|██▊       | 3325/12000 [1:59:24<6:36:07,  2.74s/it]
                                                        

 28%|██▊       | 3325/12000 [1:59:24<6:36:07,  2.74s/it]
 28%|██▊       | 3326/12000 [1:59:26<6:05:03,  2.53s/it]
                                                        

 28%|██▊       | 3326/12000 [1:59:26<6:05:03,  2.53s/it]
 28%|██▊       | 3327/12000 [1:59:28<5:34:29,  2.31s/it]
                                                        

 28%|██▊       | 3327/12000 [1:59:28<5:34:29,  2.31s/it]
 28%|██▊       | 3328/12000 [1:59:31<6:10:03,  2.56s/it]
                                                        

 28%|██▊       | 3328/12000 [1:59:31<6:10:03,  2.56s/it]
 28%|██▊       | 3329/12000 [1:59:33<5:37:36,  2.34s/it]
                                                        

 28%|██▊       | 3329/12000 [1:59:33<5:37:36,  2.34s/it]
 28%|██▊       | 3330/12000 [1:59:35<5:38:11,  2.34s/it]
                                                        

 28%|██▊       | 3330/12000 [1:59:35<5:38:11,  2.34s/it]
 28%|██▊       | 3331/12000 [1:59:38<6:21:28,  2.64s/it]
                                                        

 28%|██▊       | 3331/12000 [1:59:38<6:21:28,  2.64s/it]
 28%|██▊       | 3332/12000 [1:59:41<6:10:48,  2.57s/it]
                                                        

 28%|██▊       | 3332/12000 [1:59:41<6:10:48,  2.57s/it]
 28%|██▊       | 3333/12000 [1:59:43<6:17:03,  2.61s/it]
                                                        

 28%|██▊       | 3333/12000 [1:59:43<6:17:03,  2.61s/it]
 28%|██▊       | 3334/12000 [1:59:45<5:31:07,  2.29s/it]
                                                        

 28%|██▊       | 3334/12000 [1:59:45<5:31:07,  2.29s/it]
 28%|██▊       | 3335/12000 [1:59:46<4:56:23,  2.05s/it]
                                                        

 28%|██▊       | 3335/12000 [1:59:46<4:56:23,  2.05s/it]
 28%|██▊       | 3336/12000 [1:59:48<4:32:22,  1.89s/it]
                                                        

 28%|██▊       | 3336/12000 [1:59:48<4:32:22,  1.89s/it]
 28%|██▊       | 3337/12000 [1:59:50<4:32:53,  1.89s/it]
                                                        

 28%|██▊       | 3337/12000 [1:59:50<4:32:53,  1.89s/it]
 28%|██▊       | 3338/12000 [1:59:52<4:23:43,  1.83s/it]
                                                        

 28%|██▊       | 3338/12000 [1:59:52<4:23:43,  1.83s/it]
 28%|██▊       | 3339/12000 [1:59:56<6:22:55,  2.65s/it]
                                                        

 28%|██▊       | 3339/12000 [1:59:56<6:22:55,  2.65s/it]
 28%|██▊       | 3340/12000 [1:59:58<5:57:59,  2.48s/it]
                                                        

 28%|██▊       | 3340/12000 [1:59:58<5:57:59,  2.48s/it]
 28%|██▊       | 3341/12000 [2:00:01<6:25:52,  2.67s/it]
                                                        
{'loss': '0.9363', 'grad_norm': '0.395', 'learning_rate': '0.0001737', 'entropy': '1.005', 'num_tokens': '1.786e+07', 'mean_token_accuracy': '0.7667', 'epoch': '0.02915'}
{'loss': '0.8616', 'grad_norm': '0.3945', 'learning_rate': '0.0001737', 'entropy': '0.8297', 'num_tokens': '1.786e+07', 'mean_token_accuracy': '0.7849', 'epoch': '0.02916'}
{'loss': '0.8102', 'grad_norm': '0.3705', 'learning_rate': '0.0001737', 'entropy': '0.8347', 'num_tokens': '1.787e+07', 'mean_token_accuracy': '0.8005', 'epoch': '0.02917'}
{'loss': '0.7038', 'grad_norm': '0.2967', 'learning_rate': '0.0001736', 'entropy': '0.7116', 'num_tokens': '1.787e+07', 'mean_token_accuracy': '0.8141', 'epoch': '0.02918'}
{'loss': '0.8413', 'grad_norm': '0.3727', 'learning_rate': '0.0001736', 'entropy': '0.8335', 'num_tokens': '1.788e+07', 'mean_token_accuracy': '0.796', 'epoch': '0.02919'}
{'loss': '1.011', 'grad_norm': '0.4368', 'learning_rate': '0.0001736', 'entropy': '0.9156', 'num_tokens': '1.788e+07', 'mean_token_accuracy': '0.7746', 'epoch': '0.0292'}
{'loss': '1.527', 'grad_norm': '0.4566', 'learning_rate': '0.0001736', 'entropy': '1.309', 'num_tokens': '1.789e+07', 'mean_token_accuracy': '0.6976', 'epoch': '0.0292'}
{'loss': '1.062', 'grad_norm': '0.4211', 'learning_rate': '0.0001736', 'entropy': '1.139', 'num_tokens': '1.789e+07', 'mean_token_accuracy': '0.7163', 'epoch': '0.02921'}
{'loss': '0.7259', 'grad_norm': '0.4188', 'learning_rate': '0.0001736', 'entropy': '0.7533', 'num_tokens': '1.79e+07', 'mean_token_accuracy': '0.805', 'epoch': '0.02922'}
{'loss': '1.048', 'grad_norm': '0.4414', 'learning_rate': '0.0001735', 'entropy': '1.059', 'num_tokens': '1.79e+07', 'mean_token_accuracy': '0.7419', 'epoch': '0.02923'}
{'loss': '0.8886', 'grad_norm': '0.4604', 'learning_rate': '0.0001735', 'entropy': '0.8262', 'num_tokens': '1.791e+07', 'mean_token_accuracy': '0.7891', 'epoch': '0.02924'}
{'loss': '1.093', 'grad_norm': '0.4067', 'learning_rate': '0.0001735', 'entropy': '1.061', 'num_tokens': '1.791e+07', 'mean_token_accuracy': '0.7469', 'epoch': '0.02925'}
{'loss': '0.9015', 'grad_norm': '0.3717', 'learning_rate': '0.0001735', 'entropy': '0.8861', 'num_tokens': '1.792e+07', 'mean_token_accuracy': '0.7824', 'epoch': '0.02926'}
{'loss': '0.6032', 'grad_norm': '0.3049', 'learning_rate': '0.0001735', 'entropy': '0.6472', 'num_tokens': '1.792e+07', 'mean_token_accuracy': '0.8412', 'epoch': '0.02927'}
{'loss': '0.6663', 'grad_norm': '0.3481', 'learning_rate': '0.0001734', 'entropy': '0.8741', 'num_tokens': '1.793e+07', 'mean_token_accuracy': '0.789', 'epoch': '0.02928'}
{'loss': '0.9685', 'grad_norm': '0.471', 'learning_rate': '0.0001734', 'entropy': '0.9406', 'num_tokens': '1.793e+07', 'mean_token_accuracy': '0.7522', 'epoch': '0.02928'}
{'loss': '0.893', 'grad_norm': '0.3982', 'learning_rate': '0.0001734', 'entropy': '0.942', 'num_tokens': '1.794e+07', 'mean_token_accuracy': '0.7641', 'epoch': '0.02929'}
{'loss': '0.9194', 'grad_norm': '0.4177', 'learning_rate': '0.0001734', 'entropy': '0.9004', 'num_tokens': '1.794e+07', 'mean_token_accuracy': '0.7711', 'epoch': '0.0293'}
{'loss': '0.9722', 'grad_norm': '0.4115', 'learning_rate': '0.0001734', 'entropy': '0.9204', 'num_tokens': '1.795e+07', 'mean_token_accuracy': '0.7621', 'epoch': '0.02931'}
{'loss': '1.196', 'grad_norm': '0.5645', 'learning_rate': '0.0001733', 'entropy': '1.129', 'num_tokens': '1.795e+07', 'mean_token_accuracy': '0.7346', 'epoch': '0.02932'}
{'loss': '1.054', 'grad_norm': '0.3946', 'learning_rate': '0.0001733', 'entropy': '1.044', 'num_tokens': '1.796e+07', 'mean_token_accuracy': '0.7442', 'epoch': '0.02933'}
{'loss': '1.04', 'grad_norm': '0.5008', 'learning_rate': '0.0001733', 'entropy': '1.058', 'num_tokens': '1.796e+07', 'mean_token_accuracy': '0.7481', 'epoch': '0.02934'}
{'loss': '0.6582', 'grad_norm': '0.3558', 'learning_rate': '0.0001733', 'entropy': '0.716', 'num_tokens': '1.796e+07', 'mean_token_accuracy': '0.8153', 'epoch': '0.02935'}
{'loss': '0.7928', 'grad_norm': '0.4468', 'learning_rate': '0.0001733', 'entropy': '0.8314', 'num_tokens': '1.797e+07', 'mean_token_accuracy': '0.8015', 'epoch': '0.02935'}
{'loss': '0.8558', 'grad_norm': '0.3696', 'learning_rate': '0.0001733', 'entropy': '0.8948', 'num_tokens': '1.797e+07', 'mean_token_accuracy': '0.7731', 'epoch': '0.02936'}
{'loss': '1.613', 'grad_norm': '0.4428', 'learning_rate': '0.0001732', 'entropy': '1.499', 'num_tokens': '1.798e+07', 'mean_token_accuracy': '0.6483', 'epoch': '0.02937'}
{'loss': '0.7495', 'grad_norm': '0.329', 'learning_rate': '0.0001732', 'entropy': '0.8022', 'num_tokens': '1.798e+07', 'mean_token_accuracy': '0.7912', 'epoch': '0.02938'}
{'loss': '0.9065', 'grad_norm': '0.4662', 'learning_rate': '0.0001732', 'entropy': '0.9825', 'num_tokens': '1.799e+07', 'mean_token_accuracy': '0.7415', 'epoch': '0.02939'}
{'loss': '0.9319', 'grad_norm': '0.4059', 'learning_rate': '0.0001732', 'entropy': '0.9633', 'num_tokens': '1.799e+07', 'mean_token_accuracy': '0.7568', 'epoch': '0.0294'}
{'loss': '1.281', 'grad_norm': '0.4118', 'learning_rate': '0.0001732', 'entropy': '0.9417', 'num_tokens': '1.8e+07', 'mean_token_accuracy': '0.7776', 'epoch': '0.02941'}
{'loss': '0.8382', 'grad_norm': '0.3285', 'learning_rate': '0.0001731', 'entropy': '0.8514', 'num_tokens': '1.8e+07', 'mean_token_accuracy': '0.7863', 'epoch': '0.02942'}
{'loss': '0.9618', 'grad_norm': '0.2836', 'learning_rate': '0.0001731', 'entropy': '0.8985', 'num_tokens': '1.801e+07', 'mean_token_accuracy': '0.7838', 'epoch': '0.02943'}
{'loss': '1.441', 'grad_norm': '0.4742', 'learning_rate': '0.0001731', 'entropy': '1.319', 'num_tokens': '1.802e+07', 'mean_token_accuracy': '0.7042', 'epoch': '0.02943'}
{'loss': '0.9065', 'grad_norm': '0.4559', 'learning_rate': '0.0001731', 'entropy': '0.9901', 'num_tokens': '1.802e+07', 'mean_token_accuracy': '0.7573', 'epoch': '0.02944'}
{'loss': '1.214', 'grad_norm': '0.3936', 'learning_rate': '0.0001731', 'entropy': '1.061', 'num_tokens': '1.803e+07', 'mean_token_accuracy': '0.7413', 'epoch': '0.02945'}
{'loss': '0.8299', 'grad_norm': '0.4742', 'learning_rate': '0.000173', 'entropy': '0.8436', 'num_tokens': '1.803e+07', 'mean_token_accuracy': '0.7814', 'epoch': '0.02946'}
{'loss': '1.112', 'grad_norm': '0.4374', 'learning_rate': '0.000173', 'entropy': '1.021', 'num_tokens': '1.804e+07', 'mean_token_accuracy': '0.7538', 'epoch': '0.02947'}
{'loss': '1.175', 'grad_norm': '0.3628', 'learning_rate': '0.000173', 'entropy': '1.072', 'num_tokens': '1.805e+07', 'mean_token_accuracy': '0.7466', 'epoch': '0.02948'}
{'loss': '0.7311', 'grad_norm': '0.2999', 'learning_rate': '0.000173', 'entropy': '0.7966', 'num_tokens': '1.805e+07', 'mean_token_accuracy': '0.8034', 'epoch': '0.02949'}
{'loss': '1.146', 'grad_norm': '0.4414', 'learning_rate': '0.000173', 'entropy': '1.115', 'num_tokens': '1.806e+07', 'mean_token_accuracy': '0.7256', 'epoch': '0.0295'}
{'loss': '0.7299', 'grad_norm': '0.4709', 'learning_rate': '0.000173', 'entropy': '0.8001', 'num_tokens': '1.806e+07', 'mean_token_accuracy': '0.7974', 'epoch': '0.02951'}
{'loss': '0.9541', 'grad_norm': '0.3921', 'learning_rate': '0.0001729', 'entropy': '0.9635', 'num_tokens': '1.807e+07', 'mean_token_accuracy': '0.7657', 'epoch': '0.02951'}
{'loss': '0.8055', 'grad_norm': '0.3657', 'learning_rate': '0.0001729', 'entropy': '0.8381', 'num_tokens': '1.807e+07', 'mean_token_accuracy': '0.7843', 'epoch': '0.02952'}
{'loss': '0.9369', 'grad_norm': '0.3892', 'learning_rate': '0.0001729', 'entropy': '0.9005', 'num_tokens': '1.808e+07', 'mean_token_accuracy': '0.7644', 'epoch': '0.02953'}
{'loss': '0.9819', 'grad_norm': '0.3927', 'learning_rate': '0.0001729', 'entropy': '0.979', 'num_tokens': '1.808e+07', 'mean_token_accuracy': '0.741', 'epoch': '0.02954'}
{'loss': '0.8442', 'grad_norm': '0.3537', 'learning_rate': '0.0001729', 'entropy': '0.9071', 'num_tokens': '1.809e+07', 'mean_token_accuracy': '0.7778', 'epoch': '0.02955'}
{'loss': '0.9944', 'grad_norm': '0.4794', 'learning_rate': '0.0001728', 'entropy': '1.023', 'num_tokens': '1.809e+07', 'mean_token_accuracy': '0.7579', 'epoch': '0.02956'}

 28%|██▊       | 3341/12000 [2:00:01<6:25:52,  2.67s/it]
 28%|██▊       | 3342/12000 [2:00:04<6:11:12,  2.57s/it]
                                                        

 28%|██▊       | 3342/12000 [2:00:04<6:11:12,  2.57s/it]
 28%|██▊       | 3343/12000 [2:00:06<5:43:01,  2.38s/it]
                                                        

 28%|██▊       | 3343/12000 [2:00:06<5:43:01,  2.38s/it]
 28%|██▊       | 3344/12000 [2:00:07<5:12:49,  2.17s/it]
                                                        

 28%|██▊       | 3344/12000 [2:00:07<5:12:49,  2.17s/it]
 28%|██▊       | 3345/12000 [2:00:09<4:52:31,  2.03s/it]
                                                        

 28%|██▊       | 3345/12000 [2:00:09<4:52:31,  2.03s/it]
 28%|██▊       | 3346/12000 [2:00:11<4:39:11,  1.94s/it]
                                                        

 28%|██▊       | 3346/12000 [2:00:11<4:39:11,  1.94s/it]
 28%|██▊       | 3347/12000 [2:00:12<4:35:08,  1.91s/it]
                                                        

 28%|██▊       | 3347/12000 [2:00:12<4:35:08,  1.91s/it]
 28%|██▊       | 3348/12000 [2:00:15<4:54:07,  2.04s/it]
                                                        

 28%|██▊       | 3348/12000 [2:00:15<4:54:07,  2.04s/it]
 28%|██▊       | 3349/12000 [2:00:17<4:43:58,  1.97s/it]
                                                        

 28%|██▊       | 3349/12000 [2:00:17<4:43:58,  1.97s/it]
 28%|██▊       | 3350/12000 [2:00:19<4:42:36,  1.96s/it]
                                                        

 28%|██▊       | 3350/12000 [2:00:19<4:42:36,  1.96s/it]
 28%|██▊       | 3351/12000 [2:00:21<5:20:03,  2.22s/it]
                                                        

 28%|██▊       | 3351/12000 [2:00:21<5:20:03,  2.22s/it]
 28%|██▊       | 3352/12000 [2:00:23<4:57:23,  2.06s/it]
                                                        

 28%|██▊       | 3352/12000 [2:00:23<4:57:23,  2.06s/it]
 28%|██▊       | 3353/12000 [2:00:25<4:33:23,  1.90s/it]
                                                        

 28%|██▊       | 3353/12000 [2:00:25<4:33:23,  1.90s/it]
 28%|██▊       | 3354/12000 [2:00:26<4:17:03,  1.78s/it]
                                                        

 28%|██▊       | 3354/12000 [2:00:26<4:17:03,  1.78s/it]
 28%|██▊       | 3355/12000 [2:00:28<4:39:35,  1.94s/it]
                                                        

 28%|██▊       | 3355/12000 [2:00:28<4:39:35,  1.94s/it]
 28%|██▊       | 3356/12000 [2:00:30<4:35:51,  1.91s/it]
                                                        

 28%|██▊       | 3356/12000 [2:00:30<4:35:51,  1.91s/it]
 28%|██▊       | 3357/12000 [2:00:32<4:42:12,  1.96s/it]
                                                        

 28%|██▊       | 3357/12000 [2:00:32<4:42:12,  1.96s/it]
 28%|██▊       | 3358/12000 [2:00:35<5:01:42,  2.09s/it]
                                                        

 28%|██▊       | 3358/12000 [2:00:35<5:01:42,  2.09s/it]
 28%|██▊       | 3359/12000 [2:00:36<4:35:02,  1.91s/it]
                                                        

 28%|██▊       | 3359/12000 [2:00:36<4:35:02,  1.91s/it]
 28%|██▊       | 3360/12000 [2:00:38<4:29:38,  1.87s/it]
                                                        

 28%|██▊       | 3360/12000 [2:00:38<4:29:38,  1.87s/it]
 28%|██▊       | 3361/12000 [2:00:42<5:40:52,  2.37s/it]
                                                        

 28%|██▊       | 3361/12000 [2:00:42<5:40:52,  2.37s/it]
 28%|██▊       | 3362/12000 [2:00:43<4:58:35,  2.07s/it]
                                                        

 28%|██▊       | 3362/12000 [2:00:43<4:58:35,  2.07s/it]
 28%|██▊       | 3363/12000 [2:00:44<4:27:26,  1.86s/it]
                                                        

 28%|██▊       | 3363/12000 [2:00:44<4:27:26,  1.86s/it]
 28%|██▊       | 3364/12000 [2:00:46<4:07:31,  1.72s/it]
                                                        

 28%|██▊       | 3364/12000 [2:00:46<4:07:31,  1.72s/it]
 28%|██▊       | 3365/12000 [2:00:48<4:18:15,  1.79s/it]
                                                        

 28%|██▊       | 3365/12000 [2:00:48<4:18:15,  1.79s/it]
 28%|██▊       | 3366/12000 [2:00:50<4:43:01,  1.97s/it]
                                                        

 28%|██▊       | 3366/12000 [2:00:50<4:43:01,  1.97s/it]
 28%|██▊       | 3367/12000 [2:00:52<4:30:57,  1.88s/it]
                                                        

 28%|██▊       | 3367/12000 [2:00:52<4:30:57,  1.88s/it]
 28%|██▊       | 3368/12000 [2:00:53<4:13:03,  1.76s/it]
                                                        

 28%|██▊       | 3368/12000 [2:00:53<4:13:03,  1.76s/it]
 28%|██▊       | 3369/12000 [2:00:54<3:52:02,  1.61s/it]
                                                        

 28%|██▊       | 3369/12000 [2:00:54<3:52:02,  1.61s/it]
 28%|██▊       | 3370/12000 [2:00:56<3:59:18,  1.66s/it]
                                                        

 28%|██▊       | 3370/12000 [2:00:56<3:59:18,  1.66s/it]
 28%|██▊       | 3371/12000 [2:00:58<4:12:16,  1.75s/it]
                                                        

 28%|██▊       | 3371/12000 [2:00:58<4:12:16,  1.75s/it]
 28%|██▊       | 3372/12000 [2:01:00<4:06:14,  1.71s/it]
                                                        

 28%|██▊       | 3372/12000 [2:01:00<4:06:14,  1.71s/it]
 28%|██▊       | 3373/12000 [2:01:01<3:52:40,  1.62s/it]
                                                        

 28%|██▊       | 3373/12000 [2:01:01<3:52:40,  1.62s/it]
 28%|██▊       | 3374/12000 [2:01:03<4:01:34,  1.68s/it]
                                                        

 28%|██▊       | 3374/12000 [2:01:03<4:01:34,  1.68s/it]
 28%|██▊       | 3375/12000 [2:01:05<4:03:05,  1.69s/it]
                                                        

 28%|██▊       | 3375/12000 [2:01:05<4:03:05,  1.69s/it]
 28%|██▊       | 3376/12000 [2:01:07<4:10:39,  1.74s/it]
                                                        

 28%|██▊       | 3376/12000 [2:01:07<4:10:39,  1.74s/it]
 28%|██▊       | 3377/12000 [2:01:08<4:12:49,  1.76s/it]
                                                        

 28%|██▊       | 3377/12000 [2:01:08<4:12:49,  1.76s/it]
 28%|██▊       | 3378/12000 [2:01:11<4:47:40,  2.00s/it]
                                                        

 28%|██▊       | 3378/12000 [2:01:11<4:47:40,  2.00s/it]
 28%|██▊       | 3379/12000 [2:01:12<4:24:51,  1.84s/it]
                                                        

 28%|██▊       | 3379/12000 [2:01:12<4:24:51,  1.84s/it]
 28%|██▊       | 3380/12000 [2:01:14<4:16:37,  1.79s/it]
                                                        

 28%|██▊       | 3380/12000 [2:01:14<4:16:37,  1.79s/it]
 28%|██▊       | 3381/12000 [2:01:16<4:14:46,  1.77s/it]
                                                        

 28%|██▊       | 3381/12000 [2:01:16<4:14:46,  1.77s/it]
 28%|██▊       | 3382/12000 [2:01:18<4:18:53,  1.80s/it]
                                                        

 28%|██▊       | 3382/12000 [2:01:18<4:18:53,  1.80s/it]
 28%|██▊       | 3383/12000 [2:01:21<5:25:06,  2.26s/it]
                                                        

 28%|██▊       | 3383/12000 [2:01:21<5:25:06,  2.26s/it]
 28%|██▊       | 3384/12000 [2:01:23<4:56:33,  2.07s/it]
                                                        

 28%|██▊       | 3384/12000 [2:01:23<4:56:33,  2.07s/it]
 28%|██▊       | 3385/12000 [2:01:24<4:42:59,  1.97s/it]
                                                        

 28%|██▊       | 3385/12000 [2:01:24<4:42:59,  1.97s/it]
 28%|██▊       | 3386/12000 [2:01:26<4:17:45,  1.80s/it]
                                                        

 28%|██▊       | 3386/12000 [2:01:26<4:17:45,  1.80s/it]
 28%|██▊       | 3387/12000 [2:01:27<3:57:13,  1.65s/it]
                                                        

 28%|██▊       | 3387/12000 [2:01:27<3:57:13,  1.65s/it]
 28%|██▊       | 3388/12000 [2:01:29<4:09:19,  1.74s/it]
                                                        
{'loss': '1.136', 'grad_norm': '0.3939', 'learning_rate': '0.0001728', 'entropy': '1.1', 'num_tokens': '1.81e+07', 'mean_token_accuracy': '0.7293', 'epoch': '0.02957'}
{'loss': '0.9315', 'grad_norm': '0.3727', 'learning_rate': '0.0001728', 'entropy': '1.054', 'num_tokens': '1.81e+07', 'mean_token_accuracy': '0.7514', 'epoch': '0.02958'}
{'loss': '1.139', 'grad_norm': '0.428', 'learning_rate': '0.0001728', 'entropy': '1.143', 'num_tokens': '1.811e+07', 'mean_token_accuracy': '0.7263', 'epoch': '0.02958'}
{'loss': '1.08', 'grad_norm': '0.3878', 'learning_rate': '0.0001728', 'entropy': '1.073', 'num_tokens': '1.811e+07', 'mean_token_accuracy': '0.7447', 'epoch': '0.02959'}
{'loss': '0.812', 'grad_norm': '0.3261', 'learning_rate': '0.0001727', 'entropy': '0.8058', 'num_tokens': '1.812e+07', 'mean_token_accuracy': '0.7906', 'epoch': '0.0296'}
{'loss': '0.8734', 'grad_norm': '0.3756', 'learning_rate': '0.0001727', 'entropy': '0.9052', 'num_tokens': '1.812e+07', 'mean_token_accuracy': '0.7705', 'epoch': '0.02961'}
{'loss': '0.5932', 'grad_norm': '0.3176', 'learning_rate': '0.0001727', 'entropy': '0.6388', 'num_tokens': '1.813e+07', 'mean_token_accuracy': '0.8357', 'epoch': '0.02962'}
{'loss': '1.016', 'grad_norm': '0.4676', 'learning_rate': '0.0001727', 'entropy': '0.9504', 'num_tokens': '1.813e+07', 'mean_token_accuracy': '0.7724', 'epoch': '0.02963'}
{'loss': '1.145', 'grad_norm': '0.3897', 'learning_rate': '0.0001727', 'entropy': '1.199', 'num_tokens': '1.814e+07', 'mean_token_accuracy': '0.7215', 'epoch': '0.02964'}
{'loss': '0.9867', 'grad_norm': '0.4093', 'learning_rate': '0.0001727', 'entropy': '1.02', 'num_tokens': '1.815e+07', 'mean_token_accuracy': '0.7566', 'epoch': '0.02965'}
{'loss': '0.8816', 'grad_norm': '0.4272', 'learning_rate': '0.0001726', 'entropy': '0.8116', 'num_tokens': '1.815e+07', 'mean_token_accuracy': '0.7974', 'epoch': '0.02966'}
{'loss': '1.294', 'grad_norm': '0.4202', 'learning_rate': '0.0001726', 'entropy': '1.264', 'num_tokens': '1.816e+07', 'mean_token_accuracy': '0.695', 'epoch': '0.02966'}
{'loss': '0.8546', 'grad_norm': '0.4567', 'learning_rate': '0.0001726', 'entropy': '0.909', 'num_tokens': '1.816e+07', 'mean_token_accuracy': '0.7622', 'epoch': '0.02967'}
{'loss': '0.7027', 'grad_norm': '0.4387', 'learning_rate': '0.0001726', 'entropy': '0.757', 'num_tokens': '1.816e+07', 'mean_token_accuracy': '0.8035', 'epoch': '0.02968'}
{'loss': '0.6729', 'grad_norm': '0.3431', 'learning_rate': '0.0001726', 'entropy': '0.6798', 'num_tokens': '1.817e+07', 'mean_token_accuracy': '0.8285', 'epoch': '0.02969'}
{'loss': '0.7828', 'grad_norm': '0.4023', 'learning_rate': '0.0001725', 'entropy': '0.8006', 'num_tokens': '1.817e+07', 'mean_token_accuracy': '0.7967', 'epoch': '0.0297'}
{'loss': '0.9792', 'grad_norm': '0.4372', 'learning_rate': '0.0001725', 'entropy': '0.92', 'num_tokens': '1.818e+07', 'mean_token_accuracy': '0.7643', 'epoch': '0.02971'}
{'loss': '0.8209', 'grad_norm': '0.3128', 'learning_rate': '0.0001725', 'entropy': '1.011', 'num_tokens': '1.819e+07', 'mean_token_accuracy': '0.7538', 'epoch': '0.02972'}
{'loss': '0.7657', 'grad_norm': '0.3726', 'learning_rate': '0.0001725', 'entropy': '0.7714', 'num_tokens': '1.819e+07', 'mean_token_accuracy': '0.7896', 'epoch': '0.02973'}
{'loss': '0.8383', 'grad_norm': '0.4507', 'learning_rate': '0.0001725', 'entropy': '0.673', 'num_tokens': '1.82e+07', 'mean_token_accuracy': '0.8174', 'epoch': '0.02974'}
{'loss': '0.8694', 'grad_norm': '0.4041', 'learning_rate': '0.0001724', 'entropy': '0.9015', 'num_tokens': '1.82e+07', 'mean_token_accuracy': '0.777', 'epoch': '0.02974'}
{'loss': '0.9012', 'grad_norm': '0.385', 'learning_rate': '0.0001724', 'entropy': '0.9198', 'num_tokens': '1.82e+07', 'mean_token_accuracy': '0.7608', 'epoch': '0.02975'}
{'loss': '0.765', 'grad_norm': '0.4239', 'learning_rate': '0.0001724', 'entropy': '0.7951', 'num_tokens': '1.821e+07', 'mean_token_accuracy': '0.7994', 'epoch': '0.02976'}
{'loss': '0.87', 'grad_norm': '0.39', 'learning_rate': '0.0001724', 'entropy': '0.8893', 'num_tokens': '1.821e+07', 'mean_token_accuracy': '0.7767', 'epoch': '0.02977'}
{'loss': '0.7597', 'grad_norm': '0.4194', 'learning_rate': '0.0001724', 'entropy': '0.8216', 'num_tokens': '1.821e+07', 'mean_token_accuracy': '0.7909', 'epoch': '0.02978'}
{'loss': '0.8742', 'grad_norm': '0.2953', 'learning_rate': '0.0001723', 'entropy': '0.877', 'num_tokens': '1.822e+07', 'mean_token_accuracy': '0.7845', 'epoch': '0.02979'}
{'loss': '0.9659', 'grad_norm': '0.4641', 'learning_rate': '0.0001723', 'entropy': '1.097', 'num_tokens': '1.823e+07', 'mean_token_accuracy': '0.7487', 'epoch': '0.0298'}
{'loss': '0.9274', 'grad_norm': '0.3752', 'learning_rate': '0.0001723', 'entropy': '0.8899', 'num_tokens': '1.823e+07', 'mean_token_accuracy': '0.7795', 'epoch': '0.02981'}
{'loss': '1.344', 'grad_norm': '0.4872', 'learning_rate': '0.0001723', 'entropy': '1.296', 'num_tokens': '1.823e+07', 'mean_token_accuracy': '0.6867', 'epoch': '0.02982'}
{'loss': '0.9189', 'grad_norm': '0.9171', 'learning_rate': '0.0001723', 'entropy': '1.02', 'num_tokens': '1.824e+07', 'mean_token_accuracy': '0.748', 'epoch': '0.02982'}
{'loss': '0.881', 'grad_norm': '0.359', 'learning_rate': '0.0001723', 'entropy': '0.8324', 'num_tokens': '1.824e+07', 'mean_token_accuracy': '0.8046', 'epoch': '0.02983'}
{'loss': '0.8172', 'grad_norm': '0.3295', 'learning_rate': '0.0001722', 'entropy': '0.8249', 'num_tokens': '1.825e+07', 'mean_token_accuracy': '0.7841', 'epoch': '0.02984'}
{'loss': '1.243', 'grad_norm': '0.4319', 'learning_rate': '0.0001722', 'entropy': '1.279', 'num_tokens': '1.825e+07', 'mean_token_accuracy': '0.7127', 'epoch': '0.02985'}
{'loss': '0.8033', 'grad_norm': '0.4562', 'learning_rate': '0.0001722', 'entropy': '0.794', 'num_tokens': '1.826e+07', 'mean_token_accuracy': '0.7879', 'epoch': '0.02986'}
{'loss': '0.7042', 'grad_norm': '0.3081', 'learning_rate': '0.0001722', 'entropy': '0.728', 'num_tokens': '1.826e+07', 'mean_token_accuracy': '0.8148', 'epoch': '0.02987'}
{'loss': '1.14', 'grad_norm': '0.465', 'learning_rate': '0.0001722', 'entropy': '1.101', 'num_tokens': '1.827e+07', 'mean_token_accuracy': '0.7358', 'epoch': '0.02988'}
{'loss': '0.9214', 'grad_norm': '0.395', 'learning_rate': '0.0001721', 'entropy': '0.895', 'num_tokens': '1.827e+07', 'mean_token_accuracy': '0.7815', 'epoch': '0.02989'}
{'loss': '0.676', 'grad_norm': '0.3389', 'learning_rate': '0.0001721', 'entropy': '0.6884', 'num_tokens': '1.828e+07', 'mean_token_accuracy': '0.8226', 'epoch': '0.02989'}
{'loss': '0.8594', 'grad_norm': '0.4331', 'learning_rate': '0.0001721', 'entropy': '0.8067', 'num_tokens': '1.828e+07', 'mean_token_accuracy': '0.7978', 'epoch': '0.0299'}
{'loss': '0.7003', 'grad_norm': '0.4317', 'learning_rate': '0.0001721', 'entropy': '0.6992', 'num_tokens': '1.829e+07', 'mean_token_accuracy': '0.8118', 'epoch': '0.02991'}
{'loss': '0.6814', 'grad_norm': '0.3985', 'learning_rate': '0.0001721', 'entropy': '0.6913', 'num_tokens': '1.83e+07', 'mean_token_accuracy': '0.8208', 'epoch': '0.02992'}
{'loss': '0.9764', 'grad_norm': '0.4923', 'learning_rate': '0.000172', 'entropy': '0.9582', 'num_tokens': '1.83e+07', 'mean_token_accuracy': '0.7625', 'epoch': '0.02993'}
{'loss': '0.6886', 'grad_norm': '0.4439', 'learning_rate': '0.000172', 'entropy': '0.6758', 'num_tokens': '1.83e+07', 'mean_token_accuracy': '0.8248', 'epoch': '0.02994'}
{'loss': '0.7778', 'grad_norm': '0.407', 'learning_rate': '0.000172', 'entropy': '0.765', 'num_tokens': '1.831e+07', 'mean_token_accuracy': '0.8017', 'epoch': '0.02995'}
{'loss': '0.8675', 'grad_norm': '0.3914', 'learning_rate': '0.000172', 'entropy': '0.8403', 'num_tokens': '1.831e+07', 'mean_token_accuracy': '0.7992', 'epoch': '0.02996'}
{'loss': '0.7661', 'grad_norm': '0.3705', 'learning_rate': '0.000172', 'entropy': '0.7885', 'num_tokens': '1.832e+07', 'mean_token_accuracy': '0.7975', 'epoch': '0.02997'}
{'loss': '0.7964', 'grad_norm': '0.4513', 'learning_rate': '0.0001719', 'entropy': '0.8576', 'num_tokens': '1.832e+07', 'mean_token_accuracy': '0.7747', 'epoch': '0.02997'}

 28%|██▊       | 3388/12000 [2:01:29<4:09:19,  1.74s/it]
 28%|██▊       | 3389/12000 [2:01:30<3:50:48,  1.61s/it]
                                                        

 28%|██▊       | 3389/12000 [2:01:30<3:50:48,  1.61s/it]
 28%|██▊       | 3390/12000 [2:01:32<3:45:37,  1.57s/it]
                                                        

 28%|██▊       | 3390/12000 [2:01:32<3:45:37,  1.57s/it]
 28%|██▊       | 3391/12000 [2:01:34<4:01:41,  1.68s/it]
                                                        

 28%|██▊       | 3391/12000 [2:01:34<4:01:41,  1.68s/it]
 28%|██▊       | 3392/12000 [2:01:35<3:50:23,  1.61s/it]
                                                        

 28%|██▊       | 3392/12000 [2:01:35<3:50:23,  1.61s/it]
 28%|██▊       | 3393/12000 [2:01:37<3:54:16,  1.63s/it]
                                                        

 28%|██▊       | 3393/12000 [2:01:37<3:54:16,  1.63s/it]
 28%|██▊       | 3394/12000 [2:01:39<3:57:12,  1.65s/it]
                                                        

 28%|██▊       | 3394/12000 [2:01:39<3:57:12,  1.65s/it]
 28%|██▊       | 3395/12000 [2:01:41<4:37:50,  1.94s/it]
                                                        

 28%|██▊       | 3395/12000 [2:01:41<4:37:50,  1.94s/it]
 28%|██▊       | 3396/12000 [2:01:45<6:16:17,  2.62s/it]
                                                        

 28%|██▊       | 3396/12000 [2:01:45<6:16:17,  2.62s/it]
 28%|██▊       | 3397/12000 [2:01:47<5:31:05,  2.31s/it]
                                                        

 28%|██▊       | 3397/12000 [2:01:47<5:31:05,  2.31s/it]
 28%|██▊       | 3398/12000 [2:01:49<4:57:24,  2.07s/it]
                                                        

 28%|██▊       | 3398/12000 [2:01:49<4:57:24,  2.07s/it]
 28%|██▊       | 3399/12000 [2:01:50<4:46:51,  2.00s/it]
                                                        

 28%|██▊       | 3399/12000 [2:01:50<4:46:51,  2.00s/it]
 28%|██▊       | 3400/12000 [2:01:52<4:27:16,  1.86s/it]
                                                        

 28%|██▊       | 3400/12000 [2:01:52<4:27:16,  1.86s/it]
 28%|██▊       | 3401/12000 [2:01:55<5:19:46,  2.23s/it]
                                                        

 28%|██▊       | 3401/12000 [2:01:55<5:19:46,  2.23s/it]
 28%|██▊       | 3402/12000 [2:01:57<5:08:16,  2.15s/it]
                                                        

 28%|██▊       | 3402/12000 [2:01:57<5:08:16,  2.15s/it]
 28%|██▊       | 3403/12000 [2:01:58<4:38:36,  1.94s/it]
                                                        

 28%|██▊       | 3403/12000 [2:01:58<4:38:36,  1.94s/it]
 28%|██▊       | 3404/12000 [2:02:00<4:22:56,  1.84s/it]
                                                        

 28%|██▊       | 3404/12000 [2:02:00<4:22:56,  1.84s/it]
 28%|██▊       | 3405/12000 [2:02:03<5:31:13,  2.31s/it]
                                                        

 28%|██▊       | 3405/12000 [2:02:03<5:31:13,  2.31s/it]
 28%|██▊       | 3406/12000 [2:02:05<4:55:30,  2.06s/it]
                                                        

 28%|██▊       | 3406/12000 [2:02:05<4:55:30,  2.06s/it]
 28%|██▊       | 3407/12000 [2:02:06<4:27:09,  1.87s/it]
                                                        

 28%|██▊       | 3407/12000 [2:02:06<4:27:09,  1.87s/it]
 28%|██▊       | 3408/12000 [2:02:08<4:11:17,  1.75s/it]
                                                        

 28%|██▊       | 3408/12000 [2:02:08<4:11:17,  1.75s/it]
 28%|██▊       | 3409/12000 [2:02:09<3:54:52,  1.64s/it]
                                                        

 28%|██▊       | 3409/12000 [2:02:09<3:54:52,  1.64s/it]
 28%|██▊       | 3410/12000 [2:02:12<4:44:50,  1.99s/it]
                                                        

 28%|██▊       | 3410/12000 [2:02:12<4:44:50,  1.99s/it]
 28%|██▊       | 3411/12000 [2:02:14<4:30:02,  1.89s/it]
                                                        

 28%|██▊       | 3411/12000 [2:02:14<4:30:02,  1.89s/it]
 28%|██▊       | 3412/12000 [2:02:15<4:14:47,  1.78s/it]
                                                        

 28%|██▊       | 3412/12000 [2:02:15<4:14:47,  1.78s/it]
 28%|██▊       | 3413/12000 [2:02:17<4:10:28,  1.75s/it]
                                                        

 28%|██▊       | 3413/12000 [2:02:17<4:10:28,  1.75s/it]
 28%|██▊       | 3414/12000 [2:02:22<6:16:03,  2.63s/it]
                                                        

 28%|██▊       | 3414/12000 [2:02:22<6:16:03,  2.63s/it]
 28%|██▊       | 3415/12000 [2:02:24<6:01:18,  2.53s/it]
                                                        

 28%|██▊       | 3415/12000 [2:02:24<6:01:18,  2.53s/it]
 28%|██▊       | 3416/12000 [2:02:26<5:40:36,  2.38s/it]
                                                        

 28%|██▊       | 3416/12000 [2:02:26<5:40:36,  2.38s/it]
 28%|██▊       | 3417/12000 [2:02:28<5:50:47,  2.45s/it]
                                                        

 28%|██▊       | 3417/12000 [2:02:28<5:50:47,  2.45s/it]
 28%|██▊       | 3418/12000 [2:02:30<5:06:46,  2.14s/it]
                                                        

 28%|██▊       | 3418/12000 [2:02:30<5:06:46,  2.14s/it]
 28%|██▊       | 3419/12000 [2:02:31<4:35:47,  1.93s/it]
                                                        

 28%|██▊       | 3419/12000 [2:02:31<4:35:47,  1.93s/it]
 28%|██▊       | 3420/12000 [2:02:37<6:56:45,  2.91s/it]
                                                        

 28%|██▊       | 3420/12000 [2:02:37<6:56:45,  2.91s/it]
 29%|██▊       | 3421/12000 [2:02:39<6:45:04,  2.83s/it]
                                                        

 29%|██▊       | 3421/12000 [2:02:39<6:45:04,  2.83s/it]
 29%|██▊       | 3422/12000 [2:02:42<6:27:32,  2.71s/it]
                                                        

 29%|██▊       | 3422/12000 [2:02:42<6:27:32,  2.71s/it]
 29%|██▊       | 3423/12000 [2:02:45<7:14:58,  3.04s/it]
                                                        

 29%|██▊       | 3423/12000 [2:02:45<7:14:58,  3.04s/it]
 29%|██▊       | 3424/12000 [2:02:47<6:03:23,  2.54s/it]
                                                        

 29%|██▊       | 3424/12000 [2:02:47<6:03:23,  2.54s/it]
 29%|██▊       | 3425/12000 [2:02:49<5:28:50,  2.30s/it]
                                                        

 29%|██▊       | 3425/12000 [2:02:49<5:28:50,  2.30s/it]
 29%|██▊       | 3426/12000 [2:02:51<5:31:42,  2.32s/it]
                                                        

 29%|██▊       | 3426/12000 [2:02:51<5:31:42,  2.32s/it]
 29%|██▊       | 3427/12000 [2:02:55<6:31:24,  2.74s/it]
                                                        

 29%|██▊       | 3427/12000 [2:02:55<6:31:24,  2.74s/it]
 29%|██▊       | 3428/12000 [2:02:56<5:31:23,  2.32s/it]
                                                        

 29%|██▊       | 3428/12000 [2:02:56<5:31:23,  2.32s/it]
 29%|██▊       | 3429/12000 [2:03:03<9:07:45,  3.83s/it]
                                                        

 29%|██▊       | 3429/12000 [2:03:03<9:07:45,  3.83s/it]
 29%|██▊       | 3430/12000 [2:03:06<7:56:57,  3.34s/it]
                                                        

 29%|██▊       | 3430/12000 [2:03:06<7:56:57,  3.34s/it]
 29%|██▊       | 3431/12000 [2:03:07<6:38:46,  2.79s/it]
                                                        

 29%|██▊       | 3431/12000 [2:03:07<6:38:46,  2.79s/it]
 29%|██▊       | 3432/12000 [2:03:09<5:50:45,  2.46s/it]
                                                        

 29%|██▊       | 3432/12000 [2:03:09<5:50:45,  2.46s/it]
 29%|██▊       | 3433/12000 [2:03:11<5:29:22,  2.31s/it]
                                                        

 29%|██▊       | 3433/12000 [2:03:11<5:29:22,  2.31s/it]
 29%|██▊       | 3434/12000 [2:03:12<4:58:19,  2.09s/it]
                                                        

 29%|██▊       | 3434/12000 [2:03:12<4:58:19,  2.09s/it]
 29%|██▊       | 3435/12000 [2:03:14<4:32:45,  1.91s/it]
                                                        
{'loss': '0.6984', 'grad_norm': '0.3805', 'learning_rate': '0.0001719', 'entropy': '0.678', 'num_tokens': '1.833e+07', 'mean_token_accuracy': '0.8159', 'epoch': '0.02998'}
{'loss': '1.221', 'grad_norm': '0.4931', 'learning_rate': '0.0001719', 'entropy': '1.198', 'num_tokens': '1.833e+07', 'mean_token_accuracy': '0.7114', 'epoch': '0.02999'}
{'loss': '1.031', 'grad_norm': '0.4011', 'learning_rate': '0.0001719', 'entropy': '0.9951', 'num_tokens': '1.834e+07', 'mean_token_accuracy': '0.7453', 'epoch': '0.03'}
{'loss': '0.8305', 'grad_norm': '0.3504', 'learning_rate': '0.0001719', 'entropy': '0.7939', 'num_tokens': '1.834e+07', 'mean_token_accuracy': '0.7932', 'epoch': '0.03001'}
{'loss': '0.7227', 'grad_norm': '0.3382', 'learning_rate': '0.0001719', 'entropy': '0.7328', 'num_tokens': '1.835e+07', 'mean_token_accuracy': '0.8144', 'epoch': '0.03002'}
{'loss': '0.9297', 'grad_norm': '0.3521', 'learning_rate': '0.0001718', 'entropy': '1.047', 'num_tokens': '1.835e+07', 'mean_token_accuracy': '0.752', 'epoch': '0.03003'}
{'loss': '0.7722', 'grad_norm': '0.4176', 'learning_rate': '0.0001718', 'entropy': '0.7582', 'num_tokens': '1.835e+07', 'mean_token_accuracy': '0.8073', 'epoch': '0.03004'}
{'loss': '0.8805', 'grad_norm': '0.3736', 'learning_rate': '0.0001718', 'entropy': '0.8945', 'num_tokens': '1.836e+07', 'mean_token_accuracy': '0.7709', 'epoch': '0.03005'}
{'loss': '0.713', 'grad_norm': '0.33', 'learning_rate': '0.0001718', 'entropy': '0.6902', 'num_tokens': '1.837e+07', 'mean_token_accuracy': '0.8096', 'epoch': '0.03005'}
{'loss': '1.239', 'grad_norm': '0.492', 'learning_rate': '0.0001718', 'entropy': '1.306', 'num_tokens': '1.837e+07', 'mean_token_accuracy': '0.6922', 'epoch': '0.03006'}
{'loss': '1.359', 'grad_norm': '0.401', 'learning_rate': '0.0001717', 'entropy': '1.316', 'num_tokens': '1.838e+07', 'mean_token_accuracy': '0.6829', 'epoch': '0.03007'}
{'loss': '0.7041', 'grad_norm': '0.3568', 'learning_rate': '0.0001717', 'entropy': '0.762', 'num_tokens': '1.838e+07', 'mean_token_accuracy': '0.8008', 'epoch': '0.03008'}
{'loss': '0.8411', 'grad_norm': '0.3929', 'learning_rate': '0.0001717', 'entropy': '0.8379', 'num_tokens': '1.839e+07', 'mean_token_accuracy': '0.792', 'epoch': '0.03009'}
{'loss': '0.9656', 'grad_norm': '0.4731', 'learning_rate': '0.0001717', 'entropy': '0.976', 'num_tokens': '1.839e+07', 'mean_token_accuracy': '0.7558', 'epoch': '0.0301'}
{'loss': '0.647', 'grad_norm': '0.3552', 'learning_rate': '0.0001717', 'entropy': '0.6538', 'num_tokens': '1.84e+07', 'mean_token_accuracy': '0.8312', 'epoch': '0.03011'}
{'loss': '0.84', 'grad_norm': '0.3866', 'learning_rate': '0.0001716', 'entropy': '0.7979', 'num_tokens': '1.84e+07', 'mean_token_accuracy': '0.7966', 'epoch': '0.03012'}
{'loss': '1.142', 'grad_norm': '0.4421', 'learning_rate': '0.0001716', 'entropy': '1.184', 'num_tokens': '1.841e+07', 'mean_token_accuracy': '0.7162', 'epoch': '0.03012'}
{'loss': '0.7805', 'grad_norm': '0.3556', 'learning_rate': '0.0001716', 'entropy': '0.8453', 'num_tokens': '1.841e+07', 'mean_token_accuracy': '0.7932', 'epoch': '0.03013'}
{'loss': '0.6593', 'grad_norm': '0.4811', 'learning_rate': '0.0001716', 'entropy': '0.6707', 'num_tokens': '1.842e+07', 'mean_token_accuracy': '0.8313', 'epoch': '0.03014'}
{'loss': '0.8656', 'grad_norm': '0.393', 'learning_rate': '0.0001716', 'entropy': '0.839', 'num_tokens': '1.842e+07', 'mean_token_accuracy': '0.7927', 'epoch': '0.03015'}
{'loss': '1.17', 'grad_norm': '0.4154', 'learning_rate': '0.0001715', 'entropy': '1.266', 'num_tokens': '1.842e+07', 'mean_token_accuracy': '0.7168', 'epoch': '0.03016'}
{'loss': '0.6348', 'grad_norm': '0.3715', 'learning_rate': '0.0001715', 'entropy': '0.6543', 'num_tokens': '1.843e+07', 'mean_token_accuracy': '0.832', 'epoch': '0.03017'}
{'loss': '1.106', 'grad_norm': '0.3454', 'learning_rate': '0.0001715', 'entropy': '1.308', 'num_tokens': '1.844e+07', 'mean_token_accuracy': '0.6956', 'epoch': '0.03018'}
{'loss': '0.7692', 'grad_norm': '0.445', 'learning_rate': '0.0001715', 'entropy': '0.8197', 'num_tokens': '1.844e+07', 'mean_token_accuracy': '0.7894', 'epoch': '0.03019'}
{'loss': '0.682', 'grad_norm': '0.4039', 'learning_rate': '0.0001715', 'entropy': '0.7408', 'num_tokens': '1.845e+07', 'mean_token_accuracy': '0.8032', 'epoch': '0.0302'}
{'loss': '0.7813', 'grad_norm': '0.3991', 'learning_rate': '0.0001714', 'entropy': '0.7679', 'num_tokens': '1.845e+07', 'mean_token_accuracy': '0.808', 'epoch': '0.0302'}
{'loss': '1.686', 'grad_norm': '0.4058', 'learning_rate': '0.0001714', 'entropy': '1.209', 'num_tokens': '1.846e+07', 'mean_token_accuracy': '0.7331', 'epoch': '0.03021'}
{'loss': '0.6888', 'grad_norm': '0.4356', 'learning_rate': '0.0001714', 'entropy': '0.6863', 'num_tokens': '1.847e+07', 'mean_token_accuracy': '0.8274', 'epoch': '0.03022'}
{'loss': '0.9143', 'grad_norm': '0.4049', 'learning_rate': '0.0001714', 'entropy': '0.894', 'num_tokens': '1.847e+07', 'mean_token_accuracy': '0.7797', 'epoch': '0.03023'}
{'loss': '0.9577', 'grad_norm': '0.3449', 'learning_rate': '0.0001714', 'entropy': '0.8281', 'num_tokens': '1.848e+07', 'mean_token_accuracy': '0.7803', 'epoch': '0.03024'}
{'loss': '1.122', 'grad_norm': '3.444', 'learning_rate': '0.0001714', 'entropy': '1.095', 'num_tokens': '1.848e+07', 'mean_token_accuracy': '0.7249', 'epoch': '0.03025'}
{'loss': '0.6935', 'grad_norm': '0.3548', 'learning_rate': '0.0001713', 'entropy': '0.7547', 'num_tokens': '1.849e+07', 'mean_token_accuracy': '0.8166', 'epoch': '0.03026'}
{'loss': '1.122', 'grad_norm': '0.3649', 'learning_rate': '0.0001713', 'entropy': '0.9552', 'num_tokens': '1.85e+07', 'mean_token_accuracy': '0.7793', 'epoch': '0.03027'}
{'loss': '0.7567', 'grad_norm': '0.3528', 'learning_rate': '0.0001713', 'entropy': '0.867', 'num_tokens': '1.85e+07', 'mean_token_accuracy': '0.777', 'epoch': '0.03028'}
{'loss': '1.14', 'grad_norm': '0.48', 'learning_rate': '0.0001713', 'entropy': '1.128', 'num_tokens': '1.851e+07', 'mean_token_accuracy': '0.7281', 'epoch': '0.03028'}
{'loss': '1.471', 'grad_norm': '0.4119', 'learning_rate': '0.0001713', 'entropy': '1.241', 'num_tokens': '1.852e+07', 'mean_token_accuracy': '0.7238', 'epoch': '0.03029'}
{'loss': '0.8955', 'grad_norm': '0.5699', 'learning_rate': '0.0001712', 'entropy': '0.9678', 'num_tokens': '1.852e+07', 'mean_token_accuracy': '0.7586', 'epoch': '0.0303'}
{'loss': '0.6133', 'grad_norm': '0.3694', 'learning_rate': '0.0001712', 'entropy': '0.5977', 'num_tokens': '1.853e+07', 'mean_token_accuracy': '0.8546', 'epoch': '0.03031'}
{'loss': '0.8661', 'grad_norm': '0.3553', 'learning_rate': '0.0001712', 'entropy': '0.8847', 'num_tokens': '1.853e+07', 'mean_token_accuracy': '0.7755', 'epoch': '0.03032'}
{'loss': '1.104', 'grad_norm': '0.4649', 'learning_rate': '0.0001712', 'entropy': '1.079', 'num_tokens': '1.854e+07', 'mean_token_accuracy': '0.7378', 'epoch': '0.03033'}
{'loss': '0.6229', 'grad_norm': '0.3136', 'learning_rate': '0.0001712', 'entropy': '0.6572', 'num_tokens': '1.854e+07', 'mean_token_accuracy': '0.8362', 'epoch': '0.03034'}
{'loss': '0.8754', 'grad_norm': '0.3637', 'learning_rate': '0.0001711', 'entropy': '0.8055', 'num_tokens': '1.856e+07', 'mean_token_accuracy': '0.7935', 'epoch': '0.03035'}
{'loss': '0.6983', 'grad_norm': '0.4213', 'learning_rate': '0.0001711', 'entropy': '0.7615', 'num_tokens': '1.856e+07', 'mean_token_accuracy': '0.8105', 'epoch': '0.03035'}
{'loss': '0.9122', 'grad_norm': '0.5072', 'learning_rate': '0.0001711', 'entropy': '0.9556', 'num_tokens': '1.856e+07', 'mean_token_accuracy': '0.7725', 'epoch': '0.03036'}
{'loss': '0.9398', 'grad_norm': '0.4292', 'learning_rate': '0.0001711', 'entropy': '0.9576', 'num_tokens': '1.857e+07', 'mean_token_accuracy': '0.7614', 'epoch': '0.03037'}
{'loss': '0.998', 'grad_norm': '0.3819', 'learning_rate': '0.0001711', 'entropy': '0.9359', 'num_tokens': '1.858e+07', 'mean_token_accuracy': '0.7595', 'epoch': '0.03038'}
{'loss': '1.101', 'grad_norm': '0.4015', 'learning_rate': '0.000171', 'entropy': '1.072', 'num_tokens': '1.858e+07', 'mean_token_accuracy': '0.7471', 'epoch': '0.03039'}

 29%|██▊       | 3435/12000 [2:03:14<4:32:45,  1.91s/it]
 29%|██▊       | 3436/12000 [2:03:15<4:25:18,  1.86s/it]
                                                        

 29%|██▊       | 3436/12000 [2:03:15<4:25:18,  1.86s/it]
 29%|██▊       | 3437/12000 [2:03:17<4:05:53,  1.72s/it]
                                                        

 29%|██▊       | 3437/12000 [2:03:17<4:05:53,  1.72s/it]
 29%|██▊       | 3438/12000 [2:03:18<3:56:30,  1.66s/it]
                                                        

 29%|██▊       | 3438/12000 [2:03:18<3:56:30,  1.66s/it]
 29%|██▊       | 3439/12000 [2:03:20<3:47:06,  1.59s/it]
                                                        

 29%|██▊       | 3439/12000 [2:03:20<3:47:06,  1.59s/it]
 29%|██▊       | 3440/12000 [2:03:21<3:42:00,  1.56s/it]
                                                        

 29%|██▊       | 3440/12000 [2:03:21<3:42:00,  1.56s/it]
 29%|██▊       | 3441/12000 [2:03:23<3:31:59,  1.49s/it]
                                                        

 29%|██▊       | 3441/12000 [2:03:23<3:31:59,  1.49s/it]
 29%|██▊       | 3442/12000 [2:03:24<3:44:54,  1.58s/it]
                                                        

 29%|██▊       | 3442/12000 [2:03:24<3:44:54,  1.58s/it]
 29%|██▊       | 3443/12000 [2:03:27<4:24:53,  1.86s/it]
                                                        

 29%|██▊       | 3443/12000 [2:03:27<4:24:53,  1.86s/it]
 29%|██▊       | 3444/12000 [2:03:29<4:21:04,  1.83s/it]
                                                        

 29%|██▊       | 3444/12000 [2:03:29<4:21:04,  1.83s/it]
 29%|██▊       | 3445/12000 [2:03:33<6:24:31,  2.70s/it]
                                                        

 29%|██▊       | 3445/12000 [2:03:33<6:24:31,  2.70s/it]
 29%|██▊       | 3446/12000 [2:03:35<5:37:32,  2.37s/it]
                                                        

 29%|██▊       | 3446/12000 [2:03:35<5:37:32,  2.37s/it]
 29%|██▊       | 3447/12000 [2:03:37<5:04:57,  2.14s/it]
                                                        

 29%|██▊       | 3447/12000 [2:03:37<5:04:57,  2.14s/it]
 29%|██▊       | 3448/12000 [2:03:38<4:51:51,  2.05s/it]
                                                        

 29%|██▊       | 3448/12000 [2:03:38<4:51:51,  2.05s/it]
 29%|██▊       | 3449/12000 [2:03:42<6:02:18,  2.54s/it]
                                                        

 29%|██▊       | 3449/12000 [2:03:42<6:02:18,  2.54s/it]
 29%|██▉       | 3450/12000 [2:03:44<5:25:49,  2.29s/it]
                                                        

 29%|██▉       | 3450/12000 [2:03:44<5:25:49,  2.29s/it]
 29%|██▉       | 3451/12000 [2:03:47<5:55:20,  2.49s/it]
                                                        

 29%|██▉       | 3451/12000 [2:03:47<5:55:20,  2.49s/it]
 29%|██▉       | 3452/12000 [2:03:49<5:39:43,  2.38s/it]
                                                        

 29%|██▉       | 3452/12000 [2:03:49<5:39:43,  2.38s/it]
 29%|██▉       | 3453/12000 [2:03:52<6:01:55,  2.54s/it]
                                                        

 29%|██▉       | 3453/12000 [2:03:52<6:01:55,  2.54s/it]
 29%|██▉       | 3454/12000 [2:03:54<5:24:25,  2.28s/it]
                                                        

 29%|██▉       | 3454/12000 [2:03:54<5:24:25,  2.28s/it]
 29%|██▉       | 3455/12000 [2:03:55<5:01:52,  2.12s/it]
                                                        

 29%|██▉       | 3455/12000 [2:03:55<5:01:52,  2.12s/it]
 29%|██▉       | 3456/12000 [2:03:57<4:35:34,  1.94s/it]
                                                        

 29%|██▉       | 3456/12000 [2:03:57<4:35:34,  1.94s/it]
 29%|██▉       | 3457/12000 [2:03:59<4:26:40,  1.87s/it]
                                                        

 29%|██▉       | 3457/12000 [2:03:59<4:26:40,  1.87s/it]
 29%|██▉       | 3458/12000 [2:04:00<4:10:22,  1.76s/it]
                                                        

 29%|██▉       | 3458/12000 [2:04:00<4:10:22,  1.76s/it]
 29%|██▉       | 3459/12000 [2:04:01<3:54:13,  1.65s/it]
                                                        

 29%|██▉       | 3459/12000 [2:04:01<3:54:13,  1.65s/it]
 29%|██▉       | 3460/12000 [2:04:03<3:59:13,  1.68s/it]
                                                        

 29%|██▉       | 3460/12000 [2:04:03<3:59:13,  1.68s/it]
 29%|██▉       | 3461/12000 [2:04:05<4:00:47,  1.69s/it]
                                                        

 29%|██▉       | 3461/12000 [2:04:05<4:00:47,  1.69s/it]
 29%|██▉       | 3462/12000 [2:04:07<4:34:04,  1.93s/it]
                                                        

 29%|██▉       | 3462/12000 [2:04:07<4:34:04,  1.93s/it]
 29%|██▉       | 3463/12000 [2:04:09<4:24:34,  1.86s/it]
                                                        

 29%|██▉       | 3463/12000 [2:04:09<4:24:34,  1.86s/it]
 29%|██▉       | 3464/12000 [2:04:11<4:38:13,  1.96s/it]
                                                        

 29%|██▉       | 3464/12000 [2:04:11<4:38:13,  1.96s/it]
 29%|██▉       | 3465/12000 [2:04:14<5:29:26,  2.32s/it]
                                                        

 29%|██▉       | 3465/12000 [2:04:14<5:29:26,  2.32s/it]
 29%|██▉       | 3466/12000 [2:04:16<4:59:27,  2.11s/it]
                                                        

 29%|██▉       | 3466/12000 [2:04:16<4:59:27,  2.11s/it]
 29%|██▉       | 3467/12000 [2:04:18<5:10:34,  2.18s/it]
                                                        

 29%|██▉       | 3467/12000 [2:04:18<5:10:34,  2.18s/it]
 29%|██▉       | 3468/12000 [2:04:20<4:45:41,  2.01s/it]
                                                        

 29%|██▉       | 3468/12000 [2:04:20<4:45:41,  2.01s/it]
 29%|██▉       | 3469/12000 [2:04:22<5:01:39,  2.12s/it]
                                                        

 29%|██▉       | 3469/12000 [2:04:22<5:01:39,  2.12s/it]
 29%|██▉       | 3470/12000 [2:04:24<4:49:36,  2.04s/it]
                                                        

 29%|██▉       | 3470/12000 [2:04:24<4:49:36,  2.04s/it]
 29%|██▉       | 3471/12000 [2:04:29<6:42:33,  2.83s/it]
                                                        

 29%|██▉       | 3471/12000 [2:04:29<6:42:33,  2.83s/it]
 29%|██▉       | 3472/12000 [2:04:30<5:46:35,  2.44s/it]
                                                        

 29%|██▉       | 3472/12000 [2:04:30<5:46:35,  2.44s/it]
 29%|██▉       | 3473/12000 [2:04:33<5:57:30,  2.52s/it]
                                                        

 29%|██▉       | 3473/12000 [2:04:33<5:57:30,  2.52s/it]
 29%|██▉       | 3474/12000 [2:04:35<5:15:08,  2.22s/it]
                                                        

 29%|██▉       | 3474/12000 [2:04:35<5:15:08,  2.22s/it]
 29%|██▉       | 3475/12000 [2:04:36<4:43:29,  2.00s/it]
                                                        

 29%|██▉       | 3475/12000 [2:04:36<4:43:29,  2.00s/it]
 29%|██▉       | 3476/12000 [2:04:38<4:27:56,  1.89s/it]
                                                        

 29%|██▉       | 3476/12000 [2:04:38<4:27:56,  1.89s/it]
 29%|██▉       | 3477/12000 [2:04:39<4:14:44,  1.79s/it]
                                                        

 29%|██▉       | 3477/12000 [2:04:39<4:14:44,  1.79s/it]
 29%|██▉       | 3478/12000 [2:04:41<4:01:16,  1.70s/it]
                                                        

 29%|██▉       | 3478/12000 [2:04:41<4:01:16,  1.70s/it]
 29%|██▉       | 3479/12000 [2:04:42<3:56:49,  1.67s/it]
                                                        

 29%|██▉       | 3479/12000 [2:04:42<3:56:49,  1.67s/it]
 29%|██▉       | 3480/12000 [2:04:44<3:57:53,  1.68s/it]
                                                        

 29%|██▉       | 3480/12000 [2:04:44<3:57:53,  1.68s/it]
 29%|██▉       | 3481/12000 [2:04:46<3:51:56,  1.63s/it]
                                                        

 29%|██▉       | 3481/12000 [2:04:46<3:51:56,  1.63s/it]
 29%|██▉       | 3482/12000 [2:04:47<3:54:47,  1.65s/it]
                                                        
{'loss': '0.5448', 'grad_norm': '0.3325', 'learning_rate': '0.000171', 'entropy': '0.6304', 'num_tokens': '1.859e+07', 'mean_token_accuracy': '0.8416', 'epoch': '0.0304'}
{'loss': '0.8835', 'grad_norm': '0.4227', 'learning_rate': '0.000171', 'entropy': '0.8853', 'num_tokens': '1.859e+07', 'mean_token_accuracy': '0.7824', 'epoch': '0.03041'}
{'loss': '0.8637', 'grad_norm': '0.3812', 'learning_rate': '0.000171', 'entropy': '0.7968', 'num_tokens': '1.859e+07', 'mean_token_accuracy': '0.8113', 'epoch': '0.03042'}
{'loss': '0.6497', 'grad_norm': '0.3955', 'learning_rate': '0.000171', 'entropy': '0.7822', 'num_tokens': '1.86e+07', 'mean_token_accuracy': '0.8087', 'epoch': '0.03043'}
{'loss': '0.8291', 'grad_norm': '0.3076', 'learning_rate': '0.0001709', 'entropy': '0.8133', 'num_tokens': '1.86e+07', 'mean_token_accuracy': '0.7947', 'epoch': '0.03043'}
{'loss': '0.7062', 'grad_norm': '0.3291', 'learning_rate': '0.0001709', 'entropy': '0.8001', 'num_tokens': '1.861e+07', 'mean_token_accuracy': '0.8141', 'epoch': '0.03044'}
{'loss': '0.7022', 'grad_norm': '0.3432', 'learning_rate': '0.0001709', 'entropy': '0.7407', 'num_tokens': '1.861e+07', 'mean_token_accuracy': '0.8061', 'epoch': '0.03045'}
{'loss': '0.8749', 'grad_norm': '0.3788', 'learning_rate': '0.0001709', 'entropy': '0.8305', 'num_tokens': '1.862e+07', 'mean_token_accuracy': '0.795', 'epoch': '0.03046'}
{'loss': '0.6865', 'grad_norm': '0.3754', 'learning_rate': '0.0001709', 'entropy': '0.7104', 'num_tokens': '1.862e+07', 'mean_token_accuracy': '0.8218', 'epoch': '0.03047'}
{'loss': '1.238', 'grad_norm': '0.3988', 'learning_rate': '0.0001708', 'entropy': '1.187', 'num_tokens': '1.863e+07', 'mean_token_accuracy': '0.7065', 'epoch': '0.03048'}
{'loss': '1.432', 'grad_norm': '0.3327', 'learning_rate': '0.0001708', 'entropy': '1.21', 'num_tokens': '1.864e+07', 'mean_token_accuracy': '0.7135', 'epoch': '0.03049'}
{'loss': '0.675', 'grad_norm': '0.3707', 'learning_rate': '0.0001708', 'entropy': '0.6821', 'num_tokens': '1.864e+07', 'mean_token_accuracy': '0.8239', 'epoch': '0.0305'}
{'loss': '0.8571', 'grad_norm': '0.4038', 'learning_rate': '0.0001708', 'entropy': '0.8146', 'num_tokens': '1.865e+07', 'mean_token_accuracy': '0.795', 'epoch': '0.03051'}
{'loss': '1.251', 'grad_norm': '0.4164', 'learning_rate': '0.0001708', 'entropy': '1.176', 'num_tokens': '1.865e+07', 'mean_token_accuracy': '0.7053', 'epoch': '0.03051'}
{'loss': '0.7325', 'grad_norm': '0.3644', 'learning_rate': '0.0001707', 'entropy': '0.852', 'num_tokens': '1.866e+07', 'mean_token_accuracy': '0.7947', 'epoch': '0.03052'}
{'loss': '0.9055', 'grad_norm': '0.5243', 'learning_rate': '0.0001707', 'entropy': '0.8934', 'num_tokens': '1.866e+07', 'mean_token_accuracy': '0.7789', 'epoch': '0.03053'}
{'loss': '0.9857', 'grad_norm': '0.4891', 'learning_rate': '0.0001707', 'entropy': '0.9225', 'num_tokens': '1.867e+07', 'mean_token_accuracy': '0.7684', 'epoch': '0.03054'}
{'loss': '1.172', 'grad_norm': '0.382', 'learning_rate': '0.0001707', 'entropy': '1.207', 'num_tokens': '1.867e+07', 'mean_token_accuracy': '0.721', 'epoch': '0.03055'}
{'loss': '1.258', 'grad_norm': '0.4523', 'learning_rate': '0.0001707', 'entropy': '1.236', 'num_tokens': '1.868e+07', 'mean_token_accuracy': '0.7202', 'epoch': '0.03056'}
{'loss': '0.8698', 'grad_norm': '0.3755', 'learning_rate': '0.0001707', 'entropy': '0.9226', 'num_tokens': '1.869e+07', 'mean_token_accuracy': '0.7745', 'epoch': '0.03057'}
{'loss': '0.8235', 'grad_norm': '0.3923', 'learning_rate': '0.0001706', 'entropy': '0.8304', 'num_tokens': '1.869e+07', 'mean_token_accuracy': '0.7952', 'epoch': '0.03058'}
{'loss': '1.382', 'grad_norm': '0.4259', 'learning_rate': '0.0001706', 'entropy': '1.396', 'num_tokens': '1.87e+07', 'mean_token_accuracy': '0.6884', 'epoch': '0.03059'}
{'loss': '0.8451', 'grad_norm': '0.3758', 'learning_rate': '0.0001706', 'entropy': '0.8818', 'num_tokens': '1.87e+07', 'mean_token_accuracy': '0.7861', 'epoch': '0.03059'}
{'loss': '0.8252', 'grad_norm': '0.3891', 'learning_rate': '0.0001706', 'entropy': '0.8814', 'num_tokens': '1.871e+07', 'mean_token_accuracy': '0.7823', 'epoch': '0.0306'}
{'loss': '0.7714', 'grad_norm': '0.4102', 'learning_rate': '0.0001706', 'entropy': '0.8043', 'num_tokens': '1.871e+07', 'mean_token_accuracy': '0.7966', 'epoch': '0.03061'}
{'loss': '1.147', 'grad_norm': '0.3688', 'learning_rate': '0.0001705', 'entropy': '1.084', 'num_tokens': '1.871e+07', 'mean_token_accuracy': '0.7476', 'epoch': '0.03062'}
{'loss': '1.066', 'grad_norm': '0.4022', 'learning_rate': '0.0001705', 'entropy': '0.9674', 'num_tokens': '1.872e+07', 'mean_token_accuracy': '0.7665', 'epoch': '0.03063'}
{'loss': '1.077', 'grad_norm': '0.3976', 'learning_rate': '0.0001705', 'entropy': '1.031', 'num_tokens': '1.873e+07', 'mean_token_accuracy': '0.7353', 'epoch': '0.03064'}
{'loss': '0.8804', 'grad_norm': '0.4048', 'learning_rate': '0.0001705', 'entropy': '0.9649', 'num_tokens': '1.873e+07', 'mean_token_accuracy': '0.771', 'epoch': '0.03065'}
{'loss': '1.017', 'grad_norm': '0.3681', 'learning_rate': '0.0001705', 'entropy': '0.9666', 'num_tokens': '1.874e+07', 'mean_token_accuracy': '0.7766', 'epoch': '0.03066'}
{'loss': '0.7254', 'grad_norm': '0.3324', 'learning_rate': '0.0001704', 'entropy': '0.7191', 'num_tokens': '1.874e+07', 'mean_token_accuracy': '0.8269', 'epoch': '0.03066'}
{'loss': '0.6771', 'grad_norm': '0.4709', 'learning_rate': '0.0001704', 'entropy': '0.7751', 'num_tokens': '1.875e+07', 'mean_token_accuracy': '0.8202', 'epoch': '0.03067'}
{'loss': '0.7646', 'grad_norm': '0.4148', 'learning_rate': '0.0001704', 'entropy': '0.7268', 'num_tokens': '1.875e+07', 'mean_token_accuracy': '0.8108', 'epoch': '0.03068'}
{'loss': '0.6085', 'grad_norm': '0.2867', 'learning_rate': '0.0001704', 'entropy': '0.641', 'num_tokens': '1.876e+07', 'mean_token_accuracy': '0.8484', 'epoch': '0.03069'}
{'loss': '0.6254', 'grad_norm': '0.3203', 'learning_rate': '0.0001704', 'entropy': '0.696', 'num_tokens': '1.876e+07', 'mean_token_accuracy': '0.8169', 'epoch': '0.0307'}
{'loss': '1.065', 'grad_norm': '0.4985', 'learning_rate': '0.0001703', 'entropy': '1.007', 'num_tokens': '1.877e+07', 'mean_token_accuracy': '0.7478', 'epoch': '0.03071'}
{'loss': '0.6227', 'grad_norm': '0.3077', 'learning_rate': '0.0001703', 'entropy': '0.7057', 'num_tokens': '1.877e+07', 'mean_token_accuracy': '0.8239', 'epoch': '0.03072'}
{'loss': '0.694', 'grad_norm': '0.3984', 'learning_rate': '0.0001703', 'entropy': '0.6388', 'num_tokens': '1.878e+07', 'mean_token_accuracy': '0.8298', 'epoch': '0.03073'}
{'loss': '0.7924', 'grad_norm': '0.3505', 'learning_rate': '0.0001703', 'entropy': '0.8371', 'num_tokens': '1.879e+07', 'mean_token_accuracy': '0.7893', 'epoch': '0.03074'}
{'loss': '0.8735', 'grad_norm': '0.4073', 'learning_rate': '0.0001703', 'entropy': '0.8828', 'num_tokens': '1.879e+07', 'mean_token_accuracy': '0.7778', 'epoch': '0.03074'}
{'loss': '0.6754', 'grad_norm': '0.3629', 'learning_rate': '0.0001702', 'entropy': '0.7197', 'num_tokens': '1.879e+07', 'mean_token_accuracy': '0.8198', 'epoch': '0.03075'}
{'loss': '1.011', 'grad_norm': '0.42', 'learning_rate': '0.0001702', 'entropy': '1.031', 'num_tokens': '1.88e+07', 'mean_token_accuracy': '0.7533', 'epoch': '0.03076'}
{'loss': '0.9933', 'grad_norm': '0.5754', 'learning_rate': '0.0001702', 'entropy': '0.983', 'num_tokens': '1.88e+07', 'mean_token_accuracy': '0.7577', 'epoch': '0.03077'}
{'loss': '0.7492', 'grad_norm': '0.3007', 'learning_rate': '0.0001702', 'entropy': '0.768', 'num_tokens': '1.881e+07', 'mean_token_accuracy': '0.8146', 'epoch': '0.03078'}
{'loss': '0.775', 'grad_norm': '0.4166', 'learning_rate': '0.0001702', 'entropy': '0.8232', 'num_tokens': '1.881e+07', 'mean_token_accuracy': '0.792', 'epoch': '0.03079'}
{'loss': '0.9257', 'grad_norm': '0.4015', 'learning_rate': '0.0001701', 'entropy': '0.8643', 'num_tokens': '1.882e+07', 'mean_token_accuracy': '0.7842', 'epoch': '0.0308'}
{'loss': '0.7924', 'grad_norm': '0.3549', 'learning_rate': '0.0001701', 'entropy': '0.8208', 'num_tokens': '1.882e+07', 'mean_token_accuracy': '0.7883', 'epoch': '0.03081'}

 29%|██▉       | 3482/12000 [2:04:47<3:54:47,  1.65s/it]
 29%|██▉       | 3483/12000 [2:04:49<3:49:28,  1.62s/it]
                                                        

 29%|██▉       | 3483/12000 [2:04:49<3:49:28,  1.62s/it]
 29%|██▉       | 3484/12000 [2:04:51<3:53:22,  1.64s/it]
                                                        

 29%|██▉       | 3484/12000 [2:04:51<3:53:22,  1.64s/it]
 29%|██▉       | 3485/12000 [2:04:52<3:55:00,  1.66s/it]
                                                        

 29%|██▉       | 3485/12000 [2:04:52<3:55:00,  1.66s/it]
 29%|██▉       | 3486/12000 [2:04:54<4:15:57,  1.80s/it]
                                                        

 29%|██▉       | 3486/12000 [2:04:54<4:15:57,  1.80s/it]
 29%|██▉       | 3487/12000 [2:04:56<4:09:55,  1.76s/it]
                                                        

 29%|██▉       | 3487/12000 [2:04:56<4:09:55,  1.76s/it]
 29%|██▉       | 3488/12000 [2:04:58<4:30:35,  1.91s/it]
                                                        

 29%|██▉       | 3488/12000 [2:04:58<4:30:35,  1.91s/it]
 29%|██▉       | 3489/12000 [2:05:00<4:17:19,  1.81s/it]
                                                        

 29%|██▉       | 3489/12000 [2:05:00<4:17:19,  1.81s/it]
 29%|██▉       | 3490/12000 [2:05:01<3:59:05,  1.69s/it]
                                                        

 29%|██▉       | 3490/12000 [2:05:01<3:59:05,  1.69s/it]
 29%|██▉       | 3491/12000 [2:05:03<3:56:18,  1.67s/it]
                                                        

 29%|██▉       | 3491/12000 [2:05:03<3:56:18,  1.67s/it]
 29%|██▉       | 3492/12000 [2:05:05<4:03:14,  1.72s/it]
                                                        

 29%|██▉       | 3492/12000 [2:05:05<4:03:14,  1.72s/it]
 29%|██▉       | 3493/12000 [2:05:08<5:10:42,  2.19s/it]
                                                        

 29%|██▉       | 3493/12000 [2:05:08<5:10:42,  2.19s/it]
 29%|██▉       | 3494/12000 [2:05:11<5:32:33,  2.35s/it]
                                                        

 29%|██▉       | 3494/12000 [2:05:11<5:32:33,  2.35s/it]
 29%|██▉       | 3495/12000 [2:05:16<7:20:45,  3.11s/it]
                                                        

 29%|██▉       | 3495/12000 [2:05:16<7:20:45,  3.11s/it]
 29%|██▉       | 3496/12000 [2:05:17<6:10:31,  2.61s/it]
                                                        

 29%|██▉       | 3496/12000 [2:05:17<6:10:31,  2.61s/it]
 29%|██▉       | 3497/12000 [2:05:18<5:18:14,  2.25s/it]
                                                        

 29%|██▉       | 3497/12000 [2:05:18<5:18:14,  2.25s/it]
 29%|██▉       | 3498/12000 [2:05:21<5:24:11,  2.29s/it]
                                                        

 29%|██▉       | 3498/12000 [2:05:21<5:24:11,  2.29s/it]
 29%|██▉       | 3499/12000 [2:05:24<6:04:19,  2.57s/it]
                                                        

 29%|██▉       | 3499/12000 [2:05:24<6:04:19,  2.57s/it]
 29%|██▉       | 3500/12000 [2:05:26<5:17:48,  2.24s/it]
                                                        

 29%|██▉       | 3500/12000 [2:05:26<5:17:48,  2.24s/it]
 29%|██▉       | 3501/12000 [2:05:29<5:49:30,  2.47s/it]
                                                        

 29%|██▉       | 3501/12000 [2:05:29<5:49:30,  2.47s/it]
 29%|██▉       | 3502/12000 [2:05:30<5:01:57,  2.13s/it]
                                                        

 29%|██▉       | 3502/12000 [2:05:30<5:01:57,  2.13s/it]
 29%|██▉       | 3503/12000 [2:05:32<4:39:42,  1.98s/it]
                                                        

 29%|██▉       | 3503/12000 [2:05:32<4:39:42,  1.98s/it]
 29%|██▉       | 3504/12000 [2:05:34<5:03:09,  2.14s/it]
                                                        

 29%|██▉       | 3504/12000 [2:05:34<5:03:09,  2.14s/it]
 29%|██▉       | 3505/12000 [2:05:36<4:39:45,  1.98s/it]
                                                        

 29%|██▉       | 3505/12000 [2:05:36<4:39:45,  1.98s/it]
 29%|██▉       | 3506/12000 [2:05:38<4:55:33,  2.09s/it]
                                                        

 29%|██▉       | 3506/12000 [2:05:38<4:55:33,  2.09s/it]
 29%|██▉       | 3507/12000 [2:05:40<4:43:13,  2.00s/it]
                                                        

 29%|██▉       | 3507/12000 [2:05:40<4:43:13,  2.00s/it]
 29%|██▉       | 3508/12000 [2:05:43<5:21:51,  2.27s/it]
                                                        

 29%|██▉       | 3508/12000 [2:05:43<5:21:51,  2.27s/it]
 29%|██▉       | 3509/12000 [2:05:44<4:48:01,  2.04s/it]
                                                        

 29%|██▉       | 3509/12000 [2:05:44<4:48:01,  2.04s/it]
 29%|██▉       | 3510/12000 [2:05:46<4:58:01,  2.11s/it]
                                                        

 29%|██▉       | 3510/12000 [2:05:46<4:58:01,  2.11s/it]
 29%|██▉       | 3511/12000 [2:05:48<4:35:52,  1.95s/it]
                                                        

 29%|██▉       | 3511/12000 [2:05:48<4:35:52,  1.95s/it]
 29%|██▉       | 3512/12000 [2:05:53<7:00:11,  2.97s/it]
                                                        

 29%|██▉       | 3512/12000 [2:05:53<7:00:11,  2.97s/it]
 29%|██▉       | 3513/12000 [2:05:55<5:57:55,  2.53s/it]
                                                        

 29%|██▉       | 3513/12000 [2:05:55<5:57:55,  2.53s/it]
 29%|██▉       | 3514/12000 [2:05:58<6:34:35,  2.79s/it]
                                                        

 29%|██▉       | 3514/12000 [2:05:58<6:34:35,  2.79s/it]
 29%|██▉       | 3515/12000 [2:06:01<6:31:37,  2.77s/it]
                                                        

 29%|██▉       | 3515/12000 [2:06:01<6:31:37,  2.77s/it]
 29%|██▉       | 3516/12000 [2:06:03<5:57:16,  2.53s/it]
                                                        

 29%|██▉       | 3516/12000 [2:06:03<5:57:16,  2.53s/it]
 29%|██▉       | 3517/12000 [2:06:04<5:15:19,  2.23s/it]
                                                        

 29%|██▉       | 3517/12000 [2:06:04<5:15:19,  2.23s/it]
 29%|██▉       | 3518/12000 [2:06:06<4:50:15,  2.05s/it]
                                                        

 29%|██▉       | 3518/12000 [2:06:06<4:50:15,  2.05s/it]
 29%|██▉       | 3519/12000 [2:06:08<4:23:13,  1.86s/it]
                                                        

 29%|██▉       | 3519/12000 [2:06:08<4:23:13,  1.86s/it]
 29%|██▉       | 3520/12000 [2:06:10<4:40:06,  1.98s/it]
                                                        

 29%|██▉       | 3520/12000 [2:06:10<4:40:06,  1.98s/it]
 29%|██▉       | 3521/12000 [2:06:12<4:40:33,  1.99s/it]
                                                        

 29%|██▉       | 3521/12000 [2:06:12<4:40:33,  1.99s/it]
 29%|██▉       | 3522/12000 [2:06:14<4:40:19,  1.98s/it]
                                                        

 29%|██▉       | 3522/12000 [2:06:14<4:40:19,  1.98s/it]
 29%|██▉       | 3523/12000 [2:06:16<4:48:16,  2.04s/it]
                                                        

 29%|██▉       | 3523/12000 [2:06:16<4:48:16,  2.04s/it]
 29%|██▉       | 3524/12000 [2:06:18<4:45:45,  2.02s/it]
                                                        

 29%|██▉       | 3524/12000 [2:06:18<4:45:45,  2.02s/it]
 29%|██▉       | 3525/12000 [2:06:20<5:02:37,  2.14s/it]
                                                        

 29%|██▉       | 3525/12000 [2:06:20<5:02:37,  2.14s/it]
 29%|██▉       | 3526/12000 [2:06:24<5:50:10,  2.48s/it]
                                                        

 29%|██▉       | 3526/12000 [2:06:24<5:50:10,  2.48s/it]
 29%|██▉       | 3527/12000 [2:06:25<5:22:10,  2.28s/it]
                                                        

 29%|██▉       | 3527/12000 [2:06:25<5:22:10,  2.28s/it]
 29%|██▉       | 3528/12000 [2:06:28<5:23:42,  2.29s/it]
                                                        

 29%|██▉       | 3528/12000 [2:06:28<5:23:42,  2.29s/it]
 29%|██▉       | 3529/12000 [2:06:30<5:02:07,  2.14s/it]
                                                        
{'loss': '0.8861', 'grad_norm': '0.369', 'learning_rate': '0.0001701', 'entropy': '0.8493', 'num_tokens': '1.883e+07', 'mean_token_accuracy': '0.7818', 'epoch': '0.03082'}
{'loss': '0.6398', 'grad_norm': '0.3472', 'learning_rate': '0.0001701', 'entropy': '0.632', 'num_tokens': '1.883e+07', 'mean_token_accuracy': '0.8285', 'epoch': '0.03082'}
{'loss': '1.35', 'grad_norm': '0.4851', 'learning_rate': '0.0001701', 'entropy': '1.284', 'num_tokens': '1.884e+07', 'mean_token_accuracy': '0.6975', 'epoch': '0.03083'}
{'loss': '1.078', 'grad_norm': '0.3847', 'learning_rate': '0.00017', 'entropy': '1.015', 'num_tokens': '1.884e+07', 'mean_token_accuracy': '0.7509', 'epoch': '0.03084'}
{'loss': '0.9655', 'grad_norm': '0.4854', 'learning_rate': '0.00017', 'entropy': '0.9561', 'num_tokens': '1.885e+07', 'mean_token_accuracy': '0.7567', 'epoch': '0.03085'}
{'loss': '1.044', 'grad_norm': '0.3802', 'learning_rate': '0.00017', 'entropy': '0.9406', 'num_tokens': '1.885e+07', 'mean_token_accuracy': '0.7596', 'epoch': '0.03086'}
{'loss': '1.183', 'grad_norm': '0.5589', 'learning_rate': '0.00017', 'entropy': '1.125', 'num_tokens': '1.886e+07', 'mean_token_accuracy': '0.723', 'epoch': '0.03087'}
{'loss': '0.6911', 'grad_norm': '0.3488', 'learning_rate': '0.00017', 'entropy': '0.7549', 'num_tokens': '1.886e+07', 'mean_token_accuracy': '0.819', 'epoch': '0.03088'}
{'loss': '0.8834', 'grad_norm': '0.402', 'learning_rate': '0.0001699', 'entropy': '0.9326', 'num_tokens': '1.886e+07', 'mean_token_accuracy': '0.7764', 'epoch': '0.03089'}
{'loss': '1.287', 'grad_norm': '0.4261', 'learning_rate': '0.0001699', 'entropy': '1.184', 'num_tokens': '1.887e+07', 'mean_token_accuracy': '0.7082', 'epoch': '0.03089'}
{'loss': '1.047', 'grad_norm': '0.3445', 'learning_rate': '0.0001699', 'entropy': '1.086', 'num_tokens': '1.887e+07', 'mean_token_accuracy': '0.7301', 'epoch': '0.0309'}
{'loss': '1.079', 'grad_norm': '0.4142', 'learning_rate': '0.0001699', 'entropy': '1.07', 'num_tokens': '1.888e+07', 'mean_token_accuracy': '0.7329', 'epoch': '0.03091'}
{'loss': '0.6185', 'grad_norm': '0.3842', 'learning_rate': '0.0001699', 'entropy': '0.6594', 'num_tokens': '1.888e+07', 'mean_token_accuracy': '0.8299', 'epoch': '0.03092'}
{'loss': '0.8094', 'grad_norm': '0.3509', 'learning_rate': '0.0001698', 'entropy': '0.7911', 'num_tokens': '1.889e+07', 'mean_token_accuracy': '0.8041', 'epoch': '0.03093'}
{'loss': '0.8898', 'grad_norm': '0.3689', 'learning_rate': '0.0001698', 'entropy': '0.8543', 'num_tokens': '1.89e+07', 'mean_token_accuracy': '0.7899', 'epoch': '0.03094'}
{'loss': '0.6719', 'grad_norm': '0.3189', 'learning_rate': '0.0001698', 'entropy': '0.6616', 'num_tokens': '1.89e+07', 'mean_token_accuracy': '0.8285', 'epoch': '0.03095'}
{'loss': '0.8397', 'grad_norm': '0.4647', 'learning_rate': '0.0001698', 'entropy': '0.7677', 'num_tokens': '1.891e+07', 'mean_token_accuracy': '0.8032', 'epoch': '0.03096'}
{'loss': '0.9186', 'grad_norm': '0.3849', 'learning_rate': '0.0001698', 'entropy': '0.9698', 'num_tokens': '1.892e+07', 'mean_token_accuracy': '0.7672', 'epoch': '0.03097'}
{'loss': '1.26', 'grad_norm': '0.4462', 'learning_rate': '0.0001697', 'entropy': '1.222', 'num_tokens': '1.892e+07', 'mean_token_accuracy': '0.71', 'epoch': '0.03097'}
{'loss': '0.9792', 'grad_norm': '0.3576', 'learning_rate': '0.0001697', 'entropy': '0.9965', 'num_tokens': '1.893e+07', 'mean_token_accuracy': '0.7525', 'epoch': '0.03098'}
{'loss': '0.7296', 'grad_norm': '0.4078', 'learning_rate': '0.0001697', 'entropy': '0.7628', 'num_tokens': '1.893e+07', 'mean_token_accuracy': '0.8068', 'epoch': '0.03099'}
{'loss': '0.6585', 'grad_norm': '0.3742', 'learning_rate': '0.0001697', 'entropy': '0.6848', 'num_tokens': '1.893e+07', 'mean_token_accuracy': '0.825', 'epoch': '0.031'}
{'loss': '0.627', 'grad_norm': '0.3616', 'learning_rate': '0.0001697', 'entropy': '0.8523', 'num_tokens': '1.894e+07', 'mean_token_accuracy': '0.7986', 'epoch': '0.03101'}
{'loss': '1.256', 'grad_norm': '0.4754', 'learning_rate': '0.0001697', 'entropy': '1.112', 'num_tokens': '1.894e+07', 'mean_token_accuracy': '0.724', 'epoch': '0.03102'}
{'loss': '0.8653', 'grad_norm': '0.3446', 'learning_rate': '0.0001696', 'entropy': '0.9035', 'num_tokens': '1.895e+07', 'mean_token_accuracy': '0.776', 'epoch': '0.03103'}
{'loss': '1.199', 'grad_norm': '0.5856', 'learning_rate': '0.0001696', 'entropy': '1.268', 'num_tokens': '1.896e+07', 'mean_token_accuracy': '0.7222', 'epoch': '0.03104'}
{'loss': '0.9703', 'grad_norm': '0.4054', 'learning_rate': '0.0001696', 'entropy': '0.8772', 'num_tokens': '1.896e+07', 'mean_token_accuracy': '0.7809', 'epoch': '0.03105'}
{'loss': '0.8916', 'grad_norm': '0.4778', 'learning_rate': '0.0001696', 'entropy': '0.9534', 'num_tokens': '1.897e+07', 'mean_token_accuracy': '0.7695', 'epoch': '0.03105'}
{'loss': '0.894', 'grad_norm': '0.4612', 'learning_rate': '0.0001696', 'entropy': '0.79', 'num_tokens': '1.897e+07', 'mean_token_accuracy': '0.7967', 'epoch': '0.03106'}
{'loss': '0.8827', 'grad_norm': '0.4978', 'learning_rate': '0.0001695', 'entropy': '0.9882', 'num_tokens': '1.898e+07', 'mean_token_accuracy': '0.7531', 'epoch': '0.03107'}
{'loss': '1.292', 'grad_norm': '0.3465', 'learning_rate': '0.0001695', 'entropy': '1.069', 'num_tokens': '1.899e+07', 'mean_token_accuracy': '0.7537', 'epoch': '0.03108'}
{'loss': '0.7604', 'grad_norm': '0.4035', 'learning_rate': '0.0001695', 'entropy': '0.7645', 'num_tokens': '1.899e+07', 'mean_token_accuracy': '0.804', 'epoch': '0.03109'}
{'loss': '0.7754', 'grad_norm': '0.3733', 'learning_rate': '0.0001695', 'entropy': '0.7585', 'num_tokens': '1.9e+07', 'mean_token_accuracy': '0.7961', 'epoch': '0.0311'}
{'loss': '1.342', 'grad_norm': '0.4164', 'learning_rate': '0.0001695', 'entropy': '1.164', 'num_tokens': '1.9e+07', 'mean_token_accuracy': '0.714', 'epoch': '0.03111'}
{'loss': '0.8572', 'grad_norm': '0.3805', 'learning_rate': '0.0001694', 'entropy': '0.804', 'num_tokens': '1.901e+07', 'mean_token_accuracy': '0.7947', 'epoch': '0.03112'}
{'loss': '0.765', 'grad_norm': '0.4369', 'learning_rate': '0.0001694', 'entropy': '0.7875', 'num_tokens': '1.901e+07', 'mean_token_accuracy': '0.7993', 'epoch': '0.03112'}
{'loss': '0.8781', 'grad_norm': '0.3715', 'learning_rate': '0.0001694', 'entropy': '0.965', 'num_tokens': '1.902e+07', 'mean_token_accuracy': '0.7739', 'epoch': '0.03113'}
{'loss': '0.8713', 'grad_norm': '0.4295', 'learning_rate': '0.0001694', 'entropy': '0.9113', 'num_tokens': '1.902e+07', 'mean_token_accuracy': '0.771', 'epoch': '0.03114'}
{'loss': '0.9817', 'grad_norm': '0.437', 'learning_rate': '0.0001694', 'entropy': '0.922', 'num_tokens': '1.903e+07', 'mean_token_accuracy': '0.7721', 'epoch': '0.03115'}
{'loss': '0.7182', 'grad_norm': '0.3239', 'learning_rate': '0.0001693', 'entropy': '0.7542', 'num_tokens': '1.903e+07', 'mean_token_accuracy': '0.8132', 'epoch': '0.03116'}
{'loss': '0.8812', 'grad_norm': '0.3707', 'learning_rate': '0.0001693', 'entropy': '0.8678', 'num_tokens': '1.904e+07', 'mean_token_accuracy': '0.7852', 'epoch': '0.03117'}
{'loss': '0.8941', 'grad_norm': '0.3229', 'learning_rate': '0.0001693', 'entropy': '0.9409', 'num_tokens': '1.904e+07', 'mean_token_accuracy': '0.7728', 'epoch': '0.03118'}
{'loss': '0.8401', 'grad_norm': '0.3427', 'learning_rate': '0.0001693', 'entropy': '0.8533', 'num_tokens': '1.905e+07', 'mean_token_accuracy': '0.7859', 'epoch': '0.03119'}
{'loss': '0.9713', 'grad_norm': '0.3975', 'learning_rate': '0.0001693', 'entropy': '1.022', 'num_tokens': '1.906e+07', 'mean_token_accuracy': '0.7631', 'epoch': '0.0312'}
{'loss': '0.7936', 'grad_norm': '0.3572', 'learning_rate': '0.0001692', 'entropy': '0.9156', 'num_tokens': '1.906e+07', 'mean_token_accuracy': '0.7721', 'epoch': '0.0312'}
{'loss': '0.9537', 'grad_norm': '0.3554', 'learning_rate': '0.0001692', 'entropy': '0.879', 'num_tokens': '1.907e+07', 'mean_token_accuracy': '0.7716', 'epoch': '0.03121'}
{'loss': '0.5702', 'grad_norm': '0.2937', 'learning_rate': '0.0001692', 'entropy': '0.7061', 'num_tokens': '1.908e+07', 'mean_token_accuracy': '0.8296', 'epoch': '0.03122'}

 29%|██▉       | 3529/12000 [2:06:30<5:02:07,  2.14s/it]
 29%|██▉       | 3530/12000 [2:06:32<5:28:00,  2.32s/it]
                                                        

 29%|██▉       | 3530/12000 [2:06:32<5:28:00,  2.32s/it]
 29%|██▉       | 3531/12000 [2:06:36<6:25:31,  2.73s/it]
                                                        

 29%|██▉       | 3531/12000 [2:06:36<6:25:31,  2.73s/it]
 29%|██▉       | 3532/12000 [2:06:38<5:50:04,  2.48s/it]
                                                        

 29%|██▉       | 3532/12000 [2:06:38<5:50:04,  2.48s/it]
 29%|██▉       | 3533/12000 [2:06:40<5:38:38,  2.40s/it]
                                                        

 29%|██▉       | 3533/12000 [2:06:40<5:38:38,  2.40s/it]
 29%|██▉       | 3534/12000 [2:06:43<5:50:15,  2.48s/it]
                                                        

 29%|██▉       | 3534/12000 [2:06:43<5:50:15,  2.48s/it]
 29%|██▉       | 3535/12000 [2:06:44<5:09:53,  2.20s/it]
                                                        

 29%|██▉       | 3535/12000 [2:06:44<5:09:53,  2.20s/it]
 29%|██▉       | 3536/12000 [2:06:48<6:04:37,  2.58s/it]
                                                        

 29%|██▉       | 3536/12000 [2:06:48<6:04:37,  2.58s/it]
 29%|██▉       | 3537/12000 [2:06:49<5:11:47,  2.21s/it]
                                                        

 29%|██▉       | 3537/12000 [2:06:49<5:11:47,  2.21s/it]
 29%|██▉       | 3538/12000 [2:06:51<4:41:55,  2.00s/it]
                                                        

 29%|██▉       | 3538/12000 [2:06:51<4:41:55,  2.00s/it]
 29%|██▉       | 3539/12000 [2:06:53<4:50:33,  2.06s/it]
                                                        

 29%|██▉       | 3539/12000 [2:06:53<4:50:33,  2.06s/it]
 30%|██▉       | 3540/12000 [2:06:55<4:34:37,  1.95s/it]
                                                        

 30%|██▉       | 3540/12000 [2:06:55<4:34:37,  1.95s/it]
 30%|██▉       | 3541/12000 [2:06:56<4:12:16,  1.79s/it]
                                                        

 30%|██▉       | 3541/12000 [2:06:56<4:12:16,  1.79s/it]
 30%|██▉       | 3542/12000 [2:06:57<3:56:48,  1.68s/it]
                                                        

 30%|██▉       | 3542/12000 [2:06:57<3:56:48,  1.68s/it]
 30%|██▉       | 3543/12000 [2:06:59<3:55:41,  1.67s/it]
                                                        

 30%|██▉       | 3543/12000 [2:06:59<3:55:41,  1.67s/it]
 30%|██▉       | 3544/12000 [2:07:02<4:30:42,  1.92s/it]
                                                        

 30%|██▉       | 3544/12000 [2:07:02<4:30:42,  1.92s/it]
 30%|██▉       | 3545/12000 [2:07:04<5:01:45,  2.14s/it]
                                                        

 30%|██▉       | 3545/12000 [2:07:04<5:01:45,  2.14s/it]
 30%|██▉       | 3546/12000 [2:07:06<4:30:54,  1.92s/it]
                                                        

 30%|██▉       | 3546/12000 [2:07:06<4:30:54,  1.92s/it]
 30%|██▉       | 3547/12000 [2:07:07<4:21:43,  1.86s/it]
                                                        

 30%|██▉       | 3547/12000 [2:07:07<4:21:43,  1.86s/it]
 30%|██▉       | 3548/12000 [2:07:09<4:11:32,  1.79s/it]
                                                        

 30%|██▉       | 3548/12000 [2:07:09<4:11:32,  1.79s/it]
 30%|██▉       | 3549/12000 [2:07:11<4:26:26,  1.89s/it]
                                                        

 30%|██▉       | 3549/12000 [2:07:11<4:26:26,  1.89s/it]
 30%|██▉       | 3550/12000 [2:07:13<4:19:18,  1.84s/it]
                                                        

 30%|██▉       | 3550/12000 [2:07:13<4:19:18,  1.84s/it]
 30%|██▉       | 3551/12000 [2:07:16<5:03:02,  2.15s/it]
                                                        

 30%|██▉       | 3551/12000 [2:07:16<5:03:02,  2.15s/it]
 30%|██▉       | 3552/12000 [2:07:17<4:42:29,  2.01s/it]
                                                        

 30%|██▉       | 3552/12000 [2:07:17<4:42:29,  2.01s/it]
 30%|██▉       | 3553/12000 [2:07:19<4:19:52,  1.85s/it]
                                                        

 30%|██▉       | 3553/12000 [2:07:19<4:19:52,  1.85s/it]
 30%|██▉       | 3554/12000 [2:07:21<4:16:53,  1.82s/it]
                                                        

 30%|██▉       | 3554/12000 [2:07:21<4:16:53,  1.82s/it]
 30%|██▉       | 3555/12000 [2:07:23<4:23:04,  1.87s/it]
                                                        

 30%|██▉       | 3555/12000 [2:07:23<4:23:04,  1.87s/it]
 30%|██▉       | 3556/12000 [2:07:24<4:04:01,  1.73s/it]
                                                        

 30%|██▉       | 3556/12000 [2:07:24<4:04:01,  1.73s/it]
 30%|██▉       | 3557/12000 [2:07:26<3:59:26,  1.70s/it]
                                                        

 30%|██▉       | 3557/12000 [2:07:26<3:59:26,  1.70s/it]
 30%|██▉       | 3558/12000 [2:07:30<5:50:59,  2.49s/it]
                                                        

 30%|██▉       | 3558/12000 [2:07:30<5:50:59,  2.49s/it]
 30%|██▉       | 3559/12000 [2:07:31<5:10:24,  2.21s/it]
                                                        

 30%|██▉       | 3559/12000 [2:07:31<5:10:24,  2.21s/it]
 30%|██▉       | 3560/12000 [2:07:34<5:27:49,  2.33s/it]
                                                        

 30%|██▉       | 3560/12000 [2:07:34<5:27:49,  2.33s/it]
 30%|██▉       | 3561/12000 [2:07:36<5:00:24,  2.14s/it]
                                                        

 30%|██▉       | 3561/12000 [2:07:36<5:00:24,  2.14s/it]
 30%|██▉       | 3562/12000 [2:07:37<4:39:01,  1.98s/it]
                                                        

 30%|██▉       | 3562/12000 [2:07:37<4:39:01,  1.98s/it]
 30%|██▉       | 3563/12000 [2:07:39<4:18:11,  1.84s/it]
                                                        

 30%|██▉       | 3563/12000 [2:07:39<4:18:11,  1.84s/it]
 30%|██▉       | 3564/12000 [2:07:41<4:24:50,  1.88s/it]
                                                        

 30%|██▉       | 3564/12000 [2:07:41<4:24:50,  1.88s/it]
 30%|██▉       | 3565/12000 [2:07:43<4:15:52,  1.82s/it]
                                                        

 30%|██▉       | 3565/12000 [2:07:43<4:15:52,  1.82s/it]
 30%|██▉       | 3566/12000 [2:07:45<4:57:55,  2.12s/it]
                                                        

 30%|██▉       | 3566/12000 [2:07:45<4:57:55,  2.12s/it]
 30%|██▉       | 3567/12000 [2:07:47<4:36:46,  1.97s/it]
                                                        

 30%|██▉       | 3567/12000 [2:07:47<4:36:46,  1.97s/it]
 30%|██▉       | 3568/12000 [2:07:49<4:21:23,  1.86s/it]
                                                        

 30%|██▉       | 3568/12000 [2:07:49<4:21:23,  1.86s/it]
 30%|██▉       | 3569/12000 [2:07:51<5:05:05,  2.17s/it]
                                                        

 30%|██▉       | 3569/12000 [2:07:51<5:05:05,  2.17s/it]
 30%|██▉       | 3570/12000 [2:07:56<6:46:10,  2.89s/it]
                                                        

 30%|██▉       | 3570/12000 [2:07:56<6:46:10,  2.89s/it]
 30%|██▉       | 3571/12000 [2:07:57<5:42:58,  2.44s/it]
                                                        

 30%|██▉       | 3571/12000 [2:07:57<5:42:58,  2.44s/it]
 30%|██▉       | 3572/12000 [2:07:59<4:57:33,  2.12s/it]
                                                        

 30%|██▉       | 3572/12000 [2:07:59<4:57:33,  2.12s/it]
 30%|██▉       | 3573/12000 [2:08:00<4:28:20,  1.91s/it]
                                                        

 30%|██▉       | 3573/12000 [2:08:00<4:28:20,  1.91s/it]
 30%|██▉       | 3574/12000 [2:08:02<4:22:57,  1.87s/it]
                                                        

 30%|██▉       | 3574/12000 [2:08:02<4:22:57,  1.87s/it]
 30%|██▉       | 3575/12000 [2:08:03<4:03:30,  1.73s/it]
                                                        

 30%|██▉       | 3575/12000 [2:08:03<4:03:30,  1.73s/it]
 30%|██▉       | 3576/12000 [2:08:05<3:51:25,  1.65s/it]
                                                        
{'loss': '0.6618', 'grad_norm': '0.4012', 'learning_rate': '0.0001692', 'entropy': '0.6777', 'num_tokens': '1.908e+07', 'mean_token_accuracy': '0.8185', 'epoch': '0.03123'}
{'loss': '0.8976', 'grad_norm': '0.3619', 'learning_rate': '0.0001692', 'entropy': '1.013', 'num_tokens': '1.909e+07', 'mean_token_accuracy': '0.7582', 'epoch': '0.03124'}
{'loss': '1.255', 'grad_norm': '0.3884', 'learning_rate': '0.0001691', 'entropy': '0.9144', 'num_tokens': '1.91e+07', 'mean_token_accuracy': '0.7752', 'epoch': '0.03125'}
{'loss': '0.8148', 'grad_norm': '0.3964', 'learning_rate': '0.0001691', 'entropy': '0.7739', 'num_tokens': '1.91e+07', 'mean_token_accuracy': '0.8037', 'epoch': '0.03126'}
{'loss': '0.9014', 'grad_norm': '0.4014', 'learning_rate': '0.0001691', 'entropy': '0.8456', 'num_tokens': '1.911e+07', 'mean_token_accuracy': '0.7826', 'epoch': '0.03127'}
{'loss': '0.7159', 'grad_norm': '0.3044', 'learning_rate': '0.0001691', 'entropy': '0.6266', 'num_tokens': '1.912e+07', 'mean_token_accuracy': '0.8316', 'epoch': '0.03128'}
{'loss': '0.7512', 'grad_norm': '0.3212', 'learning_rate': '0.0001691', 'entropy': '0.7261', 'num_tokens': '1.912e+07', 'mean_token_accuracy': '0.8069', 'epoch': '0.03128'}
{'loss': '0.884', 'grad_norm': '0.4531', 'learning_rate': '0.000169', 'entropy': '0.8244', 'num_tokens': '1.913e+07', 'mean_token_accuracy': '0.7921', 'epoch': '0.03129'}
{'loss': '0.7321', 'grad_norm': '0.3572', 'learning_rate': '0.000169', 'entropy': '0.7366', 'num_tokens': '1.913e+07', 'mean_token_accuracy': '0.8149', 'epoch': '0.0313'}
{'loss': '0.697', 'grad_norm': '0.3505', 'learning_rate': '0.000169', 'entropy': '0.704', 'num_tokens': '1.913e+07', 'mean_token_accuracy': '0.8202', 'epoch': '0.03131'}
{'loss': '0.7093', 'grad_norm': '0.3531', 'learning_rate': '0.000169', 'entropy': '0.7963', 'num_tokens': '1.914e+07', 'mean_token_accuracy': '0.7951', 'epoch': '0.03132'}
{'loss': '1.01', 'grad_norm': '0.4136', 'learning_rate': '0.000169', 'entropy': '1.093', 'num_tokens': '1.914e+07', 'mean_token_accuracy': '0.7422', 'epoch': '0.03133'}
{'loss': '0.6077', 'grad_norm': '0.3417', 'learning_rate': '0.0001689', 'entropy': '0.6345', 'num_tokens': '1.915e+07', 'mean_token_accuracy': '0.8387', 'epoch': '0.03134'}
{'loss': '0.7633', 'grad_norm': '0.4256', 'learning_rate': '0.0001689', 'entropy': '0.8633', 'num_tokens': '1.915e+07', 'mean_token_accuracy': '0.7773', 'epoch': '0.03135'}
{'loss': '0.7565', 'grad_norm': '0.467', 'learning_rate': '0.0001689', 'entropy': '0.753', 'num_tokens': '1.916e+07', 'mean_token_accuracy': '0.805', 'epoch': '0.03135'}
{'loss': '0.944', 'grad_norm': '0.4082', 'learning_rate': '0.0001689', 'entropy': '1.027', 'num_tokens': '1.916e+07', 'mean_token_accuracy': '0.7528', 'epoch': '0.03136'}
{'loss': '1.408', 'grad_norm': '0.4606', 'learning_rate': '0.0001689', 'entropy': '1.411', 'num_tokens': '1.917e+07', 'mean_token_accuracy': '0.6714', 'epoch': '0.03137'}
{'loss': '0.879', 'grad_norm': '0.4633', 'learning_rate': '0.0001688', 'entropy': '0.9397', 'num_tokens': '1.917e+07', 'mean_token_accuracy': '0.7612', 'epoch': '0.03138'}
{'loss': '0.8575', 'grad_norm': '0.3562', 'learning_rate': '0.0001688', 'entropy': '0.8454', 'num_tokens': '1.918e+07', 'mean_token_accuracy': '0.7808', 'epoch': '0.03139'}
{'loss': '0.6231', 'grad_norm': '0.3619', 'learning_rate': '0.0001688', 'entropy': '0.618', 'num_tokens': '1.918e+07', 'mean_token_accuracy': '0.8396', 'epoch': '0.0314'}
{'loss': '0.9597', 'grad_norm': '0.3376', 'learning_rate': '0.0001688', 'entropy': '0.9078', 'num_tokens': '1.919e+07', 'mean_token_accuracy': '0.7798', 'epoch': '0.03141'}
{'loss': '0.932', 'grad_norm': '0.3852', 'learning_rate': '0.0001688', 'entropy': '0.8962', 'num_tokens': '1.92e+07', 'mean_token_accuracy': '0.7583', 'epoch': '0.03142'}
{'loss': '0.8249', 'grad_norm': '0.4106', 'learning_rate': '0.0001687', 'entropy': '0.8742', 'num_tokens': '1.92e+07', 'mean_token_accuracy': '0.7676', 'epoch': '0.03143'}
{'loss': '0.8246', 'grad_norm': '0.37', 'learning_rate': '0.0001687', 'entropy': '0.8136', 'num_tokens': '1.921e+07', 'mean_token_accuracy': '0.7968', 'epoch': '0.03143'}
{'loss': '0.6177', 'grad_norm': '0.3464', 'learning_rate': '0.0001687', 'entropy': '0.6018', 'num_tokens': '1.921e+07', 'mean_token_accuracy': '0.8421', 'epoch': '0.03144'}
{'loss': '0.8041', 'grad_norm': '0.3906', 'learning_rate': '0.0001687', 'entropy': '0.8665', 'num_tokens': '1.922e+07', 'mean_token_accuracy': '0.7973', 'epoch': '0.03145'}
{'loss': '0.8633', 'grad_norm': '0.4082', 'learning_rate': '0.0001687', 'entropy': '0.8297', 'num_tokens': '1.922e+07', 'mean_token_accuracy': '0.7823', 'epoch': '0.03146'}
{'loss': '0.9438', 'grad_norm': '0.4557', 'learning_rate': '0.0001686', 'entropy': '0.9768', 'num_tokens': '1.923e+07', 'mean_token_accuracy': '0.7525', 'epoch': '0.03147'}
{'loss': '0.8766', 'grad_norm': '0.4369', 'learning_rate': '0.0001686', 'entropy': '0.8822', 'num_tokens': '1.923e+07', 'mean_token_accuracy': '0.7735', 'epoch': '0.03148'}
{'loss': '0.7859', 'grad_norm': '0.3439', 'learning_rate': '0.0001686', 'entropy': '0.7165', 'num_tokens': '1.924e+07', 'mean_token_accuracy': '0.8194', 'epoch': '0.03149'}
{'loss': '0.9601', 'grad_norm': '0.4343', 'learning_rate': '0.0001686', 'entropy': '0.9867', 'num_tokens': '1.924e+07', 'mean_token_accuracy': '0.7609', 'epoch': '0.0315'}
{'loss': '0.5287', 'grad_norm': '0.2511', 'learning_rate': '0.0001686', 'entropy': '0.5542', 'num_tokens': '1.925e+07', 'mean_token_accuracy': '0.8757', 'epoch': '0.03151'}
{'loss': '0.8799', 'grad_norm': '0.4327', 'learning_rate': '0.0001685', 'entropy': '0.8981', 'num_tokens': '1.925e+07', 'mean_token_accuracy': '0.7828', 'epoch': '0.03151'}
{'loss': '0.8937', 'grad_norm': '0.36', 'learning_rate': '0.0001685', 'entropy': '0.9097', 'num_tokens': '1.926e+07', 'mean_token_accuracy': '0.7881', 'epoch': '0.03152'}
{'loss': '0.8678', 'grad_norm': '0.5054', 'learning_rate': '0.0001685', 'entropy': '0.9371', 'num_tokens': '1.926e+07', 'mean_token_accuracy': '0.7822', 'epoch': '0.03153'}
{'loss': '0.9115', 'grad_norm': '0.4063', 'learning_rate': '0.0001685', 'entropy': '0.8678', 'num_tokens': '1.927e+07', 'mean_token_accuracy': '0.7922', 'epoch': '0.03154'}
{'loss': '0.6636', 'grad_norm': '0.3508', 'learning_rate': '0.0001685', 'entropy': '0.6515', 'num_tokens': '1.928e+07', 'mean_token_accuracy': '0.8264', 'epoch': '0.03155'}
{'loss': '0.7893', 'grad_norm': '0.3144', 'learning_rate': '0.0001684', 'entropy': '0.8123', 'num_tokens': '1.929e+07', 'mean_token_accuracy': '0.8008', 'epoch': '0.03156'}
{'loss': '0.8976', 'grad_norm': '0.3688', 'learning_rate': '0.0001684', 'entropy': '0.8982', 'num_tokens': '1.929e+07', 'mean_token_accuracy': '0.7724', 'epoch': '0.03157'}
{'loss': '0.9301', 'grad_norm': '0.3081', 'learning_rate': '0.0001684', 'entropy': '0.8905', 'num_tokens': '1.93e+07', 'mean_token_accuracy': '0.787', 'epoch': '0.03158'}
{'loss': '0.8325', 'grad_norm': '0.4981', 'learning_rate': '0.0001684', 'entropy': '0.8145', 'num_tokens': '1.93e+07', 'mean_token_accuracy': '0.7872', 'epoch': '0.03159'}
{'loss': '1.438', 'grad_norm': '0.2845', 'learning_rate': '0.0001684', 'entropy': '1.105', 'num_tokens': '1.931e+07', 'mean_token_accuracy': '0.7168', 'epoch': '0.03159'}
{'loss': '1.267', 'grad_norm': '0.4661', 'learning_rate': '0.0001683', 'entropy': '1.346', 'num_tokens': '1.931e+07', 'mean_token_accuracy': '0.6904', 'epoch': '0.0316'}
{'loss': '0.7388', 'grad_norm': '0.3754', 'learning_rate': '0.0001683', 'entropy': '0.7634', 'num_tokens': '1.932e+07', 'mean_token_accuracy': '0.8179', 'epoch': '0.03161'}
{'loss': '0.7787', 'grad_norm': '0.372', 'learning_rate': '0.0001683', 'entropy': '0.822', 'num_tokens': '1.932e+07', 'mean_token_accuracy': '0.797', 'epoch': '0.03162'}
{'loss': '1.021', 'grad_norm': '0.2958', 'learning_rate': '0.0001683', 'entropy': '1.013', 'num_tokens': '1.933e+07', 'mean_token_accuracy': '0.7456', 'epoch': '0.03163'}
{'loss': '0.8343', 'grad_norm': '0.423', 'learning_rate': '0.0001683', 'entropy': '0.914', 'num_tokens': '1.933e+07', 'mean_token_accuracy': '0.7594', 'epoch': '0.03164'}

 30%|██▉       | 3576/12000 [2:08:05<3:51:25,  1.65s/it]
 30%|██▉       | 3577/12000 [2:08:09<5:24:51,  2.31s/it]
                                                        

 30%|██▉       | 3577/12000 [2:08:09<5:24:51,  2.31s/it]
 30%|██▉       | 3578/12000 [2:08:11<5:15:24,  2.25s/it]
                                                        

 30%|██▉       | 3578/12000 [2:08:11<5:15:24,  2.25s/it]
 30%|██▉       | 3579/12000 [2:08:12<4:38:13,  1.98s/it]
                                                        

 30%|██▉       | 3579/12000 [2:08:12<4:38:13,  1.98s/it]
 30%|██▉       | 3580/12000 [2:08:16<5:48:50,  2.49s/it]
                                                        

 30%|██▉       | 3580/12000 [2:08:16<5:48:50,  2.49s/it]
 30%|██▉       | 3581/12000 [2:08:18<5:28:28,  2.34s/it]
                                                        

 30%|██▉       | 3581/12000 [2:08:18<5:28:28,  2.34s/it]
 30%|██▉       | 3582/12000 [2:08:19<4:52:27,  2.08s/it]
                                                        

 30%|██▉       | 3582/12000 [2:08:19<4:52:27,  2.08s/it]
 30%|██▉       | 3583/12000 [2:08:21<4:27:29,  1.91s/it]
                                                        

 30%|██▉       | 3583/12000 [2:08:21<4:27:29,  1.91s/it]
 30%|██▉       | 3584/12000 [2:08:22<4:06:19,  1.76s/it]
                                                        

 30%|██▉       | 3584/12000 [2:08:22<4:06:19,  1.76s/it]
 30%|██▉       | 3585/12000 [2:08:24<4:25:24,  1.89s/it]
                                                        

 30%|██▉       | 3585/12000 [2:08:24<4:25:24,  1.89s/it]
 30%|██▉       | 3586/12000 [2:08:26<4:01:45,  1.72s/it]
                                                        

 30%|██▉       | 3586/12000 [2:08:26<4:01:45,  1.72s/it]
 30%|██▉       | 3587/12000 [2:08:27<3:50:40,  1.65s/it]
                                                        

 30%|██▉       | 3587/12000 [2:08:27<3:50:40,  1.65s/it]
 30%|██▉       | 3588/12000 [2:08:29<3:53:44,  1.67s/it]
                                                        

 30%|██▉       | 3588/12000 [2:08:29<3:53:44,  1.67s/it]
 30%|██▉       | 3589/12000 [2:08:31<4:09:03,  1.78s/it]
                                                        

 30%|██▉       | 3589/12000 [2:08:31<4:09:03,  1.78s/it]
 30%|██▉       | 3590/12000 [2:08:33<4:03:56,  1.74s/it]
                                                        

 30%|██▉       | 3590/12000 [2:08:33<4:03:56,  1.74s/it]
 30%|██▉       | 3591/12000 [2:08:35<4:14:22,  1.82s/it]
                                                        

 30%|██▉       | 3591/12000 [2:08:35<4:14:22,  1.82s/it]
 30%|██▉       | 3592/12000 [2:08:37<4:33:59,  1.96s/it]
                                                        

 30%|██▉       | 3592/12000 [2:08:37<4:33:59,  1.96s/it]
 30%|██▉       | 3593/12000 [2:08:39<4:19:07,  1.85s/it]
                                                        

 30%|██▉       | 3593/12000 [2:08:39<4:19:07,  1.85s/it]
 30%|██▉       | 3594/12000 [2:08:40<4:21:01,  1.86s/it]
                                                        

 30%|██▉       | 3594/12000 [2:08:40<4:21:01,  1.86s/it]
 30%|██▉       | 3595/12000 [2:08:43<4:51:16,  2.08s/it]
                                                        

 30%|██▉       | 3595/12000 [2:08:43<4:51:16,  2.08s/it]
 30%|██▉       | 3596/12000 [2:08:45<4:59:34,  2.14s/it]
                                                        

 30%|██▉       | 3596/12000 [2:08:45<4:59:34,  2.14s/it]
 30%|██▉       | 3597/12000 [2:08:47<4:29:07,  1.92s/it]
                                                        

 30%|██▉       | 3597/12000 [2:08:47<4:29:07,  1.92s/it]
 30%|██▉       | 3598/12000 [2:08:49<4:42:15,  2.02s/it]
                                                        

 30%|██▉       | 3598/12000 [2:08:49<4:42:15,  2.02s/it]
 30%|██▉       | 3599/12000 [2:08:50<4:11:20,  1.80s/it]
                                                        

 30%|██▉       | 3599/12000 [2:08:50<4:11:20,  1.80s/it]
 30%|███       | 3600/12000 [2:08:52<4:06:30,  1.76s/it]
                                                        

 30%|███       | 3600/12000 [2:08:52<4:06:30,  1.76s/it]
 30%|███       | 3601/12000 [2:08:55<4:44:51,  2.03s/it]
                                                        

 30%|███       | 3601/12000 [2:08:55<4:44:51,  2.03s/it]
 30%|███       | 3602/12000 [2:08:58<5:41:56,  2.44s/it]
                                                        

 30%|███       | 3602/12000 [2:08:58<5:41:56,  2.44s/it]
 30%|███       | 3603/12000 [2:09:01<6:04:02,  2.60s/it]
                                                        

 30%|███       | 3603/12000 [2:09:01<6:04:02,  2.60s/it]
 30%|███       | 3604/12000 [2:09:04<6:05:23,  2.61s/it]
                                                        

 30%|███       | 3604/12000 [2:09:04<6:05:23,  2.61s/it]
 30%|███       | 3605/12000 [2:09:05<5:33:46,  2.39s/it]
                                                        

 30%|███       | 3605/12000 [2:09:05<5:33:46,  2.39s/it]
 30%|███       | 3606/12000 [2:09:11<7:58:02,  3.42s/it]
                                                        

 30%|███       | 3606/12000 [2:09:11<7:58:02,  3.42s/it]
 30%|███       | 3607/12000 [2:09:13<6:35:22,  2.83s/it]
                                                        

 30%|███       | 3607/12000 [2:09:13<6:35:22,  2.83s/it]
 30%|███       | 3608/12000 [2:09:14<5:50:12,  2.50s/it]
                                                        

 30%|███       | 3608/12000 [2:09:14<5:50:12,  2.50s/it]
 30%|███       | 3609/12000 [2:09:16<5:16:41,  2.26s/it]
                                                        

 30%|███       | 3609/12000 [2:09:16<5:16:41,  2.26s/it]
 30%|███       | 3610/12000 [2:09:18<4:49:20,  2.07s/it]
                                                        

 30%|███       | 3610/12000 [2:09:18<4:49:20,  2.07s/it]
 30%|███       | 3611/12000 [2:09:21<5:44:18,  2.46s/it]
                                                        

 30%|███       | 3611/12000 [2:09:21<5:44:18,  2.46s/it]
 30%|███       | 3612/12000 [2:09:23<5:18:53,  2.28s/it]
                                                        

 30%|███       | 3612/12000 [2:09:23<5:18:53,  2.28s/it]
 30%|███       | 3613/12000 [2:09:24<4:41:26,  2.01s/it]
                                                        

 30%|███       | 3613/12000 [2:09:24<4:41:26,  2.01s/it]
 30%|███       | 3614/12000 [2:09:26<4:22:34,  1.88s/it]
                                                        

 30%|███       | 3614/12000 [2:09:26<4:22:34,  1.88s/it]
 30%|███       | 3615/12000 [2:09:28<4:40:50,  2.01s/it]
                                                        

 30%|███       | 3615/12000 [2:09:28<4:40:50,  2.01s/it]
 30%|███       | 3616/12000 [2:09:31<5:02:12,  2.16s/it]
                                                        

 30%|███       | 3616/12000 [2:09:31<5:02:12,  2.16s/it]
 30%|███       | 3617/12000 [2:09:34<5:44:05,  2.46s/it]
                                                        

 30%|███       | 3617/12000 [2:09:34<5:44:05,  2.46s/it]
 30%|███       | 3618/12000 [2:09:35<5:01:19,  2.16s/it]
                                                        

 30%|███       | 3618/12000 [2:09:35<5:01:19,  2.16s/it]
 30%|███       | 3619/12000 [2:09:37<4:28:36,  1.92s/it]
                                                        

 30%|███       | 3619/12000 [2:09:37<4:28:36,  1.92s/it]
 30%|███       | 3620/12000 [2:09:41<6:25:27,  2.76s/it]
                                                        

 30%|███       | 3620/12000 [2:09:41<6:25:27,  2.76s/it]
 30%|███       | 3621/12000 [2:09:43<5:35:11,  2.40s/it]
                                                        

 30%|███       | 3621/12000 [2:09:43<5:35:11,  2.40s/it]
 30%|███       | 3622/12000 [2:09:45<5:11:33,  2.23s/it]
                                                        

 30%|███       | 3622/12000 [2:09:45<5:11:33,  2.23s/it]
 30%|███       | 3623/12000 [2:09:47<4:54:58,  2.11s/it]
                                                        
{'loss': '0.804', 'grad_norm': '0.4031', 'learning_rate': '0.0001682', 'entropy': '0.7939', 'num_tokens': '1.934e+07', 'mean_token_accuracy': '0.8039', 'epoch': '0.03165'}
{'loss': '0.8942', 'grad_norm': '0.3153', 'learning_rate': '0.0001682', 'entropy': '0.8666', 'num_tokens': '1.934e+07', 'mean_token_accuracy': '0.7825', 'epoch': '0.03166'}
{'loss': '1.137', 'grad_norm': '0.4176', 'learning_rate': '0.0001682', 'entropy': '1.144', 'num_tokens': '1.935e+07', 'mean_token_accuracy': '0.7339', 'epoch': '0.03166'}
{'loss': '0.7608', 'grad_norm': '0.3666', 'learning_rate': '0.0001682', 'entropy': '0.7883', 'num_tokens': '1.935e+07', 'mean_token_accuracy': '0.82', 'epoch': '0.03167'}
{'loss': '0.5547', 'grad_norm': '0.3277', 'learning_rate': '0.0001682', 'entropy': '0.6456', 'num_tokens': '1.936e+07', 'mean_token_accuracy': '0.8267', 'epoch': '0.03168'}
{'loss': '1.138', 'grad_norm': '0.4537', 'learning_rate': '0.0001681', 'entropy': '1.093', 'num_tokens': '1.936e+07', 'mean_token_accuracy': '0.7323', 'epoch': '0.03169'}
{'loss': '1.1', 'grad_norm': '0.4213', 'learning_rate': '0.0001681', 'entropy': '1.103', 'num_tokens': '1.937e+07', 'mean_token_accuracy': '0.7141', 'epoch': '0.0317'}
{'loss': '0.8016', 'grad_norm': '0.3501', 'learning_rate': '0.0001681', 'entropy': '0.7383', 'num_tokens': '1.937e+07', 'mean_token_accuracy': '0.806', 'epoch': '0.03171'}
{'loss': '0.7797', 'grad_norm': '0.2921', 'learning_rate': '0.0001681', 'entropy': '0.778', 'num_tokens': '1.938e+07', 'mean_token_accuracy': '0.798', 'epoch': '0.03172'}
{'loss': '0.9332', 'grad_norm': '0.3578', 'learning_rate': '0.0001681', 'entropy': '0.9325', 'num_tokens': '1.938e+07', 'mean_token_accuracy': '0.7658', 'epoch': '0.03173'}
{'loss': '1.145', 'grad_norm': '0.5091', 'learning_rate': '0.000168', 'entropy': '1.098', 'num_tokens': '1.939e+07', 'mean_token_accuracy': '0.7336', 'epoch': '0.03174'}
{'loss': '0.8045', 'grad_norm': '0.3755', 'learning_rate': '0.000168', 'entropy': '0.8052', 'num_tokens': '1.939e+07', 'mean_token_accuracy': '0.789', 'epoch': '0.03174'}
{'loss': '0.8526', 'grad_norm': '0.4856', 'learning_rate': '0.000168', 'entropy': '0.8184', 'num_tokens': '1.939e+07', 'mean_token_accuracy': '0.7901', 'epoch': '0.03175'}
{'loss': '0.9944', 'grad_norm': '0.3305', 'learning_rate': '0.000168', 'entropy': '0.9434', 'num_tokens': '1.94e+07', 'mean_token_accuracy': '0.7674', 'epoch': '0.03176'}
{'loss': '0.5865', 'grad_norm': '0.3294', 'learning_rate': '0.000168', 'entropy': '0.6404', 'num_tokens': '1.941e+07', 'mean_token_accuracy': '0.8398', 'epoch': '0.03177'}
{'loss': '0.7033', 'grad_norm': '0.3848', 'learning_rate': '0.0001679', 'entropy': '0.7118', 'num_tokens': '1.941e+07', 'mean_token_accuracy': '0.8023', 'epoch': '0.03178'}
{'loss': '0.9461', 'grad_norm': '0.4045', 'learning_rate': '0.0001679', 'entropy': '0.9', 'num_tokens': '1.942e+07', 'mean_token_accuracy': '0.781', 'epoch': '0.03179'}
{'loss': '0.9157', 'grad_norm': '0.4412', 'learning_rate': '0.0001679', 'entropy': '0.9236', 'num_tokens': '1.942e+07', 'mean_token_accuracy': '0.7748', 'epoch': '0.0318'}
{'loss': '0.8012', 'grad_norm': '0.3728', 'learning_rate': '0.0001679', 'entropy': '0.868', 'num_tokens': '1.943e+07', 'mean_token_accuracy': '0.7941', 'epoch': '0.03181'}
{'loss': '0.8068', 'grad_norm': '0.3267', 'learning_rate': '0.0001678', 'entropy': '0.8834', 'num_tokens': '1.944e+07', 'mean_token_accuracy': '0.7841', 'epoch': '0.03182'}
{'loss': '0.6975', 'grad_norm': '0.3348', 'learning_rate': '0.0001678', 'entropy': '0.6639', 'num_tokens': '1.944e+07', 'mean_token_accuracy': '0.8275', 'epoch': '0.03182'}
{'loss': '0.959', 'grad_norm': '0.4504', 'learning_rate': '0.0001678', 'entropy': '1.021', 'num_tokens': '1.945e+07', 'mean_token_accuracy': '0.7533', 'epoch': '0.03183'}
{'loss': '0.9905', 'grad_norm': '0.4862', 'learning_rate': '0.0001678', 'entropy': '0.9563', 'num_tokens': '1.945e+07', 'mean_token_accuracy': '0.7714', 'epoch': '0.03184'}
{'loss': '0.8443', 'grad_norm': '0.4076', 'learning_rate': '0.0001678', 'entropy': '0.8402', 'num_tokens': '1.946e+07', 'mean_token_accuracy': '0.7814', 'epoch': '0.03185'}
{'loss': '0.859', 'grad_norm': '0.3457', 'learning_rate': '0.0001677', 'entropy': '0.8944', 'num_tokens': '1.946e+07', 'mean_token_accuracy': '0.7648', 'epoch': '0.03186'}
{'loss': '0.9083', 'grad_norm': '0.4535', 'learning_rate': '0.0001677', 'entropy': '0.8994', 'num_tokens': '1.946e+07', 'mean_token_accuracy': '0.7695', 'epoch': '0.03187'}
{'loss': '0.8704', 'grad_norm': '0.4366', 'learning_rate': '0.0001677', 'entropy': '0.8632', 'num_tokens': '1.947e+07', 'mean_token_accuracy': '0.7756', 'epoch': '0.03188'}
{'loss': '0.7585', 'grad_norm': '0.3744', 'learning_rate': '0.0001677', 'entropy': '0.829', 'num_tokens': '1.948e+07', 'mean_token_accuracy': '0.7971', 'epoch': '0.03189'}
{'loss': '0.871', 'grad_norm': '0.4127', 'learning_rate': '0.0001677', 'entropy': '0.9389', 'num_tokens': '1.948e+07', 'mean_token_accuracy': '0.7762', 'epoch': '0.03189'}
{'loss': '1.183', 'grad_norm': '0.3814', 'learning_rate': '0.0001676', 'entropy': '1.154', 'num_tokens': '1.949e+07', 'mean_token_accuracy': '0.7311', 'epoch': '0.0319'}
{'loss': '0.652', 'grad_norm': '0.2743', 'learning_rate': '0.0001676', 'entropy': '0.7031', 'num_tokens': '1.95e+07', 'mean_token_accuracy': '0.8278', 'epoch': '0.03191'}
{'loss': '0.8256', 'grad_norm': '0.3877', 'learning_rate': '0.0001676', 'entropy': '0.8916', 'num_tokens': '1.95e+07', 'mean_token_accuracy': '0.7699', 'epoch': '0.03192'}
{'loss': '0.8949', 'grad_norm': '0.4191', 'learning_rate': '0.0001676', 'entropy': '0.9247', 'num_tokens': '1.951e+07', 'mean_token_accuracy': '0.7588', 'epoch': '0.03193'}
{'loss': '1.057', 'grad_norm': '0.3895', 'learning_rate': '0.0001676', 'entropy': '0.9788', 'num_tokens': '1.951e+07', 'mean_token_accuracy': '0.7612', 'epoch': '0.03194'}
{'loss': '0.8177', 'grad_norm': '0.4551', 'learning_rate': '0.0001675', 'entropy': '0.8117', 'num_tokens': '1.952e+07', 'mean_token_accuracy': '0.7868', 'epoch': '0.03195'}
{'loss': '0.8389', 'grad_norm': '0.355', 'learning_rate': '0.0001675', 'entropy': '0.8685', 'num_tokens': '1.953e+07', 'mean_token_accuracy': '0.7916', 'epoch': '0.03196'}
{'loss': '1.142', 'grad_norm': '0.3811', 'learning_rate': '0.0001675', 'entropy': '1.054', 'num_tokens': '1.953e+07', 'mean_token_accuracy': '0.7417', 'epoch': '0.03197'}
{'loss': '0.6893', 'grad_norm': '0.2829', 'learning_rate': '0.0001675', 'entropy': '0.7292', 'num_tokens': '1.954e+07', 'mean_token_accuracy': '0.8124', 'epoch': '0.03197'}
{'loss': '0.9118', 'grad_norm': '0.3736', 'learning_rate': '0.0001675', 'entropy': '0.9025', 'num_tokens': '1.954e+07', 'mean_token_accuracy': '0.7799', 'epoch': '0.03198'}
{'loss': '0.8004', 'grad_norm': '0.3009', 'learning_rate': '0.0001674', 'entropy': '0.7278', 'num_tokens': '1.955e+07', 'mean_token_accuracy': '0.8075', 'epoch': '0.03199'}
{'loss': '0.8781', 'grad_norm': '0.4597', 'learning_rate': '0.0001674', 'entropy': '0.8296', 'num_tokens': '1.955e+07', 'mean_token_accuracy': '0.7905', 'epoch': '0.032'}
{'loss': '1.072', 'grad_norm': '0.4183', 'learning_rate': '0.0001674', 'entropy': '0.8974', 'num_tokens': '1.956e+07', 'mean_token_accuracy': '0.7747', 'epoch': '0.03201'}
{'loss': '0.7282', 'grad_norm': '0.3409', 'learning_rate': '0.0001674', 'entropy': '0.8047', 'num_tokens': '1.957e+07', 'mean_token_accuracy': '0.7991', 'epoch': '0.03202'}
{'loss': '0.8891', 'grad_norm': '0.4707', 'learning_rate': '0.0001674', 'entropy': '0.9691', 'num_tokens': '1.957e+07', 'mean_token_accuracy': '0.7566', 'epoch': '0.03203'}
{'loss': '1.138', 'grad_norm': '0.3518', 'learning_rate': '0.0001673', 'entropy': '1.028', 'num_tokens': '1.958e+07', 'mean_token_accuracy': '0.7498', 'epoch': '0.03204'}
{'loss': '1.087', 'grad_norm': '0.5165', 'learning_rate': '0.0001673', 'entropy': '1.037', 'num_tokens': '1.958e+07', 'mean_token_accuracy': '0.7467', 'epoch': '0.03205'}
{'loss': '1.135', 'grad_norm': '0.3663', 'learning_rate': '0.0001673', 'entropy': '0.9989', 'num_tokens': '1.959e+07', 'mean_token_accuracy': '0.7511', 'epoch': '0.03205'}

 30%|███       | 3623/12000 [2:09:47<4:54:58,  2.11s/it]
 30%|███       | 3624/12000 [2:09:51<6:19:01,  2.72s/it]
                                                        

 30%|███       | 3624/12000 [2:09:51<6:19:01,  2.72s/it]
 30%|███       | 3625/12000 [2:09:52<5:30:17,  2.37s/it]
                                                        

 30%|███       | 3625/12000 [2:09:52<5:30:17,  2.37s/it]
 30%|███       | 3626/12000 [2:09:54<4:55:30,  2.12s/it]
                                                        

 30%|███       | 3626/12000 [2:09:54<4:55:30,  2.12s/it]
 30%|███       | 3627/12000 [2:09:55<4:25:05,  1.90s/it]
                                                        

 30%|███       | 3627/12000 [2:09:55<4:25:05,  1.90s/it]
 30%|███       | 3628/12000 [2:09:58<4:50:52,  2.08s/it]
                                                        

 30%|███       | 3628/12000 [2:09:58<4:50:52,  2.08s/it]
 30%|███       | 3629/12000 [2:09:59<4:31:59,  1.95s/it]
                                                        

 30%|███       | 3629/12000 [2:09:59<4:31:59,  1.95s/it]
 30%|███       | 3630/12000 [2:10:02<4:45:33,  2.05s/it]
                                                        

 30%|███       | 3630/12000 [2:10:02<4:45:33,  2.05s/it]
 30%|███       | 3631/12000 [2:10:06<5:57:52,  2.57s/it]
                                                        

 30%|███       | 3631/12000 [2:10:06<5:57:52,  2.57s/it]
 30%|███       | 3632/12000 [2:10:07<5:12:40,  2.24s/it]
                                                        

 30%|███       | 3632/12000 [2:10:07<5:12:40,  2.24s/it]
 30%|███       | 3633/12000 [2:10:08<4:38:40,  2.00s/it]
                                                        

 30%|███       | 3633/12000 [2:10:08<4:38:40,  2.00s/it]
 30%|███       | 3634/12000 [2:10:11<5:06:49,  2.20s/it]
                                                        

 30%|███       | 3634/12000 [2:10:11<5:06:49,  2.20s/it]
 30%|███       | 3635/12000 [2:10:13<4:33:31,  1.96s/it]
                                                        

 30%|███       | 3635/12000 [2:10:13<4:33:31,  1.96s/it]
 30%|███       | 3636/12000 [2:10:15<4:51:15,  2.09s/it]
                                                        

 30%|███       | 3636/12000 [2:10:15<4:51:15,  2.09s/it]
 30%|███       | 3637/12000 [2:10:17<5:05:32,  2.19s/it]
                                                        

 30%|███       | 3637/12000 [2:10:17<5:05:32,  2.19s/it]
 30%|███       | 3638/12000 [2:10:19<4:53:44,  2.11s/it]
                                                        

 30%|███       | 3638/12000 [2:10:19<4:53:44,  2.11s/it]
 30%|███       | 3639/12000 [2:10:21<4:25:44,  1.91s/it]
                                                        

 30%|███       | 3639/12000 [2:10:21<4:25:44,  1.91s/it]
 30%|███       | 3640/12000 [2:10:23<4:55:09,  2.12s/it]
                                                        

 30%|███       | 3640/12000 [2:10:23<4:55:09,  2.12s/it]
 30%|███       | 3641/12000 [2:10:26<5:04:38,  2.19s/it]
                                                        

 30%|███       | 3641/12000 [2:10:26<5:04:38,  2.19s/it]
 30%|███       | 3642/12000 [2:10:27<4:33:42,  1.96s/it]
                                                        

 30%|███       | 3642/12000 [2:10:27<4:33:42,  1.96s/it]
 30%|███       | 3643/12000 [2:10:28<4:07:38,  1.78s/it]
                                                        

 30%|███       | 3643/12000 [2:10:28<4:07:38,  1.78s/it]
 30%|███       | 3644/12000 [2:10:31<4:37:44,  1.99s/it]
                                                        

 30%|███       | 3644/12000 [2:10:31<4:37:44,  1.99s/it]
 30%|███       | 3645/12000 [2:10:32<4:13:50,  1.82s/it]
                                                        

 30%|███       | 3645/12000 [2:10:32<4:13:50,  1.82s/it]
 30%|███       | 3646/12000 [2:10:36<5:21:23,  2.31s/it]
                                                        

 30%|███       | 3646/12000 [2:10:36<5:21:23,  2.31s/it]
 30%|███       | 3647/12000 [2:10:40<6:46:56,  2.92s/it]
                                                        

 30%|███       | 3647/12000 [2:10:40<6:46:56,  2.92s/it]
 30%|███       | 3648/12000 [2:10:42<5:45:06,  2.48s/it]
                                                        

 30%|███       | 3648/12000 [2:10:42<5:45:06,  2.48s/it]
 30%|███       | 3649/12000 [2:10:43<5:06:53,  2.20s/it]
                                                        

 30%|███       | 3649/12000 [2:10:43<5:06:53,  2.20s/it]
 30%|███       | 3650/12000 [2:10:46<5:27:55,  2.36s/it]
                                                        

 30%|███       | 3650/12000 [2:10:46<5:27:55,  2.36s/it]
 30%|███       | 3651/12000 [2:10:49<6:09:25,  2.65s/it]
                                                        

 30%|███       | 3651/12000 [2:10:49<6:09:25,  2.65s/it]
 30%|███       | 3652/12000 [2:10:51<5:26:05,  2.34s/it]
                                                        

 30%|███       | 3652/12000 [2:10:51<5:26:05,  2.34s/it]
 30%|███       | 3653/12000 [2:10:53<4:58:38,  2.15s/it]
                                                        

 30%|███       | 3653/12000 [2:10:53<4:58:38,  2.15s/it]
 30%|███       | 3654/12000 [2:10:54<4:36:20,  1.99s/it]
                                                        

 30%|███       | 3654/12000 [2:10:54<4:36:20,  1.99s/it]
 30%|███       | 3655/12000 [2:10:56<4:30:08,  1.94s/it]
                                                        

 30%|███       | 3655/12000 [2:10:56<4:30:08,  1.94s/it]
 30%|███       | 3656/12000 [2:10:59<5:21:00,  2.31s/it]
                                                        

 30%|███       | 3656/12000 [2:10:59<5:21:00,  2.31s/it]
 30%|███       | 3657/12000 [2:11:01<4:56:24,  2.13s/it]
                                                        

 30%|███       | 3657/12000 [2:11:01<4:56:24,  2.13s/it]
 30%|███       | 3658/12000 [2:11:03<4:50:25,  2.09s/it]
                                                        

 30%|███       | 3658/12000 [2:11:03<4:50:25,  2.09s/it]
 30%|███       | 3659/12000 [2:11:05<4:37:18,  1.99s/it]
                                                        

 30%|███       | 3659/12000 [2:11:05<4:37:18,  1.99s/it]
 30%|███       | 3660/12000 [2:11:07<4:51:58,  2.10s/it]
                                                        

 30%|███       | 3660/12000 [2:11:07<4:51:58,  2.10s/it]
 31%|███       | 3661/12000 [2:11:10<5:39:59,  2.45s/it]
                                                        

 31%|███       | 3661/12000 [2:11:10<5:39:59,  2.45s/it]
 31%|███       | 3662/12000 [2:11:12<5:09:15,  2.23s/it]
                                                        

 31%|███       | 3662/12000 [2:11:12<5:09:15,  2.23s/it]
 31%|███       | 3663/12000 [2:11:14<4:47:03,  2.07s/it]
                                                        

 31%|███       | 3663/12000 [2:11:14<4:47:03,  2.07s/it]
 31%|███       | 3664/12000 [2:11:17<5:38:02,  2.43s/it]
                                                        

 31%|███       | 3664/12000 [2:11:17<5:38:02,  2.43s/it]
 31%|███       | 3665/12000 [2:11:18<4:54:20,  2.12s/it]
                                                        

 31%|███       | 3665/12000 [2:11:18<4:54:20,  2.12s/it]
 31%|███       | 3666/12000 [2:11:21<5:04:34,  2.19s/it]
                                                        

 31%|███       | 3666/12000 [2:11:21<5:04:34,  2.19s/it]
 31%|███       | 3667/12000 [2:11:22<4:36:33,  1.99s/it]
                                                        

 31%|███       | 3667/12000 [2:11:22<4:36:33,  1.99s/it]
 31%|███       | 3668/12000 [2:11:27<6:16:27,  2.71s/it]
                                                        

 31%|███       | 3668/12000 [2:11:27<6:16:27,  2.71s/it]
 31%|███       | 3669/12000 [2:11:28<5:22:02,  2.32s/it]
                                                        

 31%|███       | 3669/12000 [2:11:28<5:22:02,  2.32s/it]
 31%|███       | 3670/12000 [2:11:29<4:47:37,  2.07s/it]
                                                        
{'loss': '1.32', 'grad_norm': '0.4938', 'learning_rate': '0.0001673', 'entropy': '1.229', 'num_tokens': '1.959e+07', 'mean_token_accuracy': '0.7066', 'epoch': '0.03206'}
{'loss': '1.159', 'grad_norm': '0.4107', 'learning_rate': '0.0001673', 'entropy': '1.119', 'num_tokens': '1.96e+07', 'mean_token_accuracy': '0.7257', 'epoch': '0.03207'}
{'loss': '0.9687', 'grad_norm': '0.3221', 'learning_rate': '0.0001672', 'entropy': '0.8962', 'num_tokens': '1.96e+07', 'mean_token_accuracy': '0.7801', 'epoch': '0.03208'}
{'loss': '0.7969', 'grad_norm': '0.4482', 'learning_rate': '0.0001672', 'entropy': '0.7798', 'num_tokens': '1.961e+07', 'mean_token_accuracy': '0.7919', 'epoch': '0.03209'}
{'loss': '0.6674', 'grad_norm': '0.3506', 'learning_rate': '0.0001672', 'entropy': '0.7189', 'num_tokens': '1.961e+07', 'mean_token_accuracy': '0.816', 'epoch': '0.0321'}
{'loss': '0.7072', 'grad_norm': '0.343', 'learning_rate': '0.0001672', 'entropy': '0.7777', 'num_tokens': '1.962e+07', 'mean_token_accuracy': '0.8075', 'epoch': '0.03211'}
{'loss': '0.923', 'grad_norm': '0.4112', 'learning_rate': '0.0001672', 'entropy': '0.9058', 'num_tokens': '1.962e+07', 'mean_token_accuracy': '0.78', 'epoch': '0.03212'}
{'loss': '0.8335', 'grad_norm': '0.3342', 'learning_rate': '0.0001671', 'entropy': '0.8107', 'num_tokens': '1.963e+07', 'mean_token_accuracy': '0.7987', 'epoch': '0.03212'}
{'loss': '0.7813', 'grad_norm': '0.345', 'learning_rate': '0.0001671', 'entropy': '0.8955', 'num_tokens': '1.964e+07', 'mean_token_accuracy': '0.7822', 'epoch': '0.03213'}
{'loss': '0.7066', 'grad_norm': '0.3468', 'learning_rate': '0.0001671', 'entropy': '0.7962', 'num_tokens': '1.964e+07', 'mean_token_accuracy': '0.7972', 'epoch': '0.03214'}
{'loss': '0.7896', 'grad_norm': '0.3739', 'learning_rate': '0.0001671', 'entropy': '0.8185', 'num_tokens': '1.965e+07', 'mean_token_accuracy': '0.7927', 'epoch': '0.03215'}
{'loss': '0.9323', 'grad_norm': '0.4039', 'learning_rate': '0.0001671', 'entropy': '0.9601', 'num_tokens': '1.965e+07', 'mean_token_accuracy': '0.7621', 'epoch': '0.03216'}
{'loss': '1.044', 'grad_norm': '0.4463', 'learning_rate': '0.000167', 'entropy': '1.054', 'num_tokens': '1.966e+07', 'mean_token_accuracy': '0.7398', 'epoch': '0.03217'}
{'loss': '1.042', 'grad_norm': '0.41', 'learning_rate': '0.000167', 'entropy': '1.138', 'num_tokens': '1.966e+07', 'mean_token_accuracy': '0.7277', 'epoch': '0.03218'}
{'loss': '0.7389', 'grad_norm': '0.3365', 'learning_rate': '0.000167', 'entropy': '0.7673', 'num_tokens': '1.967e+07', 'mean_token_accuracy': '0.8079', 'epoch': '0.03219'}
{'loss': '0.9248', 'grad_norm': '0.4509', 'learning_rate': '0.000167', 'entropy': '0.8955', 'num_tokens': '1.968e+07', 'mean_token_accuracy': '0.77', 'epoch': '0.0322'}
{'loss': '1.123', 'grad_norm': '0.4068', 'learning_rate': '0.000167', 'entropy': '1.044', 'num_tokens': '1.968e+07', 'mean_token_accuracy': '0.7448', 'epoch': '0.0322'}
{'loss': '0.7885', 'grad_norm': '0.3255', 'learning_rate': '0.0001669', 'entropy': '0.8044', 'num_tokens': '1.969e+07', 'mean_token_accuracy': '0.799', 'epoch': '0.03221'}
{'loss': '0.8766', 'grad_norm': '0.3566', 'learning_rate': '0.0001669', 'entropy': '0.9141', 'num_tokens': '1.969e+07', 'mean_token_accuracy': '0.7731', 'epoch': '0.03222'}
{'loss': '0.8064', 'grad_norm': '0.4995', 'learning_rate': '0.0001669', 'entropy': '0.8348', 'num_tokens': '1.97e+07', 'mean_token_accuracy': '0.7975', 'epoch': '0.03223'}
{'loss': '0.5347', 'grad_norm': '0.3267', 'learning_rate': '0.0001669', 'entropy': '0.6563', 'num_tokens': '1.97e+07', 'mean_token_accuracy': '0.8306', 'epoch': '0.03224'}
{'loss': '0.8667', 'grad_norm': '0.3386', 'learning_rate': '0.0001669', 'entropy': '0.9103', 'num_tokens': '1.971e+07', 'mean_token_accuracy': '0.7789', 'epoch': '0.03225'}
{'loss': '0.8814', 'grad_norm': '0.4', 'learning_rate': '0.0001668', 'entropy': '0.8688', 'num_tokens': '1.971e+07', 'mean_token_accuracy': '0.7786', 'epoch': '0.03226'}
{'loss': '0.9049', 'grad_norm': '0.4671', 'learning_rate': '0.0001668', 'entropy': '0.9549', 'num_tokens': '1.972e+07', 'mean_token_accuracy': '0.7602', 'epoch': '0.03227'}
{'loss': '1.092', 'grad_norm': '0.2569', 'learning_rate': '0.0001668', 'entropy': '0.9768', 'num_tokens': '1.973e+07', 'mean_token_accuracy': '0.7481', 'epoch': '0.03228'}
{'loss': '0.8661', 'grad_norm': '0.3823', 'learning_rate': '0.0001668', 'entropy': '0.9586', 'num_tokens': '1.973e+07', 'mean_token_accuracy': '0.7666', 'epoch': '0.03228'}
{'loss': '0.7177', 'grad_norm': '0.3514', 'learning_rate': '0.0001667', 'entropy': '0.7145', 'num_tokens': '1.974e+07', 'mean_token_accuracy': '0.8181', 'epoch': '0.03229'}
{'loss': '1.472', 'grad_norm': '0.5186', 'learning_rate': '0.0001667', 'entropy': '1.204', 'num_tokens': '1.974e+07', 'mean_token_accuracy': '0.7233', 'epoch': '0.0323'}
{'loss': '0.9692', 'grad_norm': '0.3293', 'learning_rate': '0.0001667', 'entropy': '1.058', 'num_tokens': '1.975e+07', 'mean_token_accuracy': '0.7482', 'epoch': '0.03231'}
{'loss': '0.9309', 'grad_norm': '0.3801', 'learning_rate': '0.0001667', 'entropy': '0.953', 'num_tokens': '1.975e+07', 'mean_token_accuracy': '0.75', 'epoch': '0.03232'}
{'loss': '0.6927', 'grad_norm': '0.3816', 'learning_rate': '0.0001667', 'entropy': '0.699', 'num_tokens': '1.976e+07', 'mean_token_accuracy': '0.8119', 'epoch': '0.03233'}
{'loss': '1.092', 'grad_norm': '0.4601', 'learning_rate': '0.0001666', 'entropy': '1.049', 'num_tokens': '1.976e+07', 'mean_token_accuracy': '0.7285', 'epoch': '0.03234'}
{'loss': '0.713', 'grad_norm': '0.4079', 'learning_rate': '0.0001666', 'entropy': '0.7282', 'num_tokens': '1.977e+07', 'mean_token_accuracy': '0.8124', 'epoch': '0.03235'}
{'loss': '0.755', 'grad_norm': '0.311', 'learning_rate': '0.0001666', 'entropy': '0.7565', 'num_tokens': '1.978e+07', 'mean_token_accuracy': '0.8068', 'epoch': '0.03235'}
{'loss': '0.8722', 'grad_norm': '0.3356', 'learning_rate': '0.0001666', 'entropy': '0.9232', 'num_tokens': '1.978e+07', 'mean_token_accuracy': '0.7746', 'epoch': '0.03236'}
{'loss': '0.9899', 'grad_norm': '0.3413', 'learning_rate': '0.0001666', 'entropy': '0.9673', 'num_tokens': '1.979e+07', 'mean_token_accuracy': '0.7571', 'epoch': '0.03237'}
{'loss': '0.8288', 'grad_norm': '0.3939', 'learning_rate': '0.0001665', 'entropy': '0.8208', 'num_tokens': '1.979e+07', 'mean_token_accuracy': '0.7984', 'epoch': '0.03238'}
{'loss': '0.7018', 'grad_norm': '0.2633', 'learning_rate': '0.0001665', 'entropy': '0.7224', 'num_tokens': '1.98e+07', 'mean_token_accuracy': '0.8093', 'epoch': '0.03239'}
{'loss': '0.8337', 'grad_norm': '0.3526', 'learning_rate': '0.0001665', 'entropy': '0.8555', 'num_tokens': '1.981e+07', 'mean_token_accuracy': '0.7785', 'epoch': '0.0324'}
{'loss': '0.9917', 'grad_norm': '0.5011', 'learning_rate': '0.0001665', 'entropy': '1.098', 'num_tokens': '1.981e+07', 'mean_token_accuracy': '0.7413', 'epoch': '0.03241'}
{'loss': '0.89', 'grad_norm': '0.4028', 'learning_rate': '0.0001665', 'entropy': '0.9514', 'num_tokens': '1.982e+07', 'mean_token_accuracy': '0.7532', 'epoch': '0.03242'}
{'loss': '1.213', 'grad_norm': '0.3871', 'learning_rate': '0.0001664', 'entropy': '1.017', 'num_tokens': '1.982e+07', 'mean_token_accuracy': '0.7671', 'epoch': '0.03243'}
{'loss': '0.832', 'grad_norm': '0.3475', 'learning_rate': '0.0001664', 'entropy': '0.9076', 'num_tokens': '1.983e+07', 'mean_token_accuracy': '0.7686', 'epoch': '0.03243'}
{'loss': '0.8819', 'grad_norm': '0.3405', 'learning_rate': '0.0001664', 'entropy': '0.8991', 'num_tokens': '1.983e+07', 'mean_token_accuracy': '0.781', 'epoch': '0.03244'}
{'loss': '0.9993', 'grad_norm': '0.4009', 'learning_rate': '0.0001664', 'entropy': '0.956', 'num_tokens': '1.984e+07', 'mean_token_accuracy': '0.7762', 'epoch': '0.03245'}
{'loss': '0.6866', 'grad_norm': '0.3044', 'learning_rate': '0.0001664', 'entropy': '0.7668', 'num_tokens': '1.985e+07', 'mean_token_accuracy': '0.8036', 'epoch': '0.03246'}
{'loss': '0.8205', 'grad_norm': '0.4118', 'learning_rate': '0.0001663', 'entropy': '0.7694', 'num_tokens': '1.985e+07', 'mean_token_accuracy': '0.81', 'epoch': '0.03247'}

 31%|███       | 3670/12000 [2:11:29<4:47:37,  2.07s/it]
 31%|███       | 3671/12000 [2:11:31<4:25:14,  1.91s/it]
                                                        

 31%|███       | 3671/12000 [2:11:31<4:25:14,  1.91s/it]
 31%|███       | 3672/12000 [2:11:33<4:15:51,  1.84s/it]
                                                        

 31%|███       | 3672/12000 [2:11:33<4:15:51,  1.84s/it]
 31%|███       | 3673/12000 [2:11:36<5:17:20,  2.29s/it]
                                                        

 31%|███       | 3673/12000 [2:11:36<5:17:20,  2.29s/it]
 31%|███       | 3674/12000 [2:11:38<4:53:15,  2.11s/it]
                                                        

 31%|███       | 3674/12000 [2:11:38<4:53:15,  2.11s/it]
 31%|███       | 3675/12000 [2:11:39<4:37:04,  2.00s/it]
                                                        

 31%|███       | 3675/12000 [2:11:39<4:37:04,  2.00s/it]
 31%|███       | 3676/12000 [2:11:41<4:16:44,  1.85s/it]
                                                        

 31%|███       | 3676/12000 [2:11:41<4:16:44,  1.85s/it]
 31%|███       | 3677/12000 [2:11:43<4:31:17,  1.96s/it]
                                                        

 31%|███       | 3677/12000 [2:11:43<4:31:17,  1.96s/it]
 31%|███       | 3678/12000 [2:11:45<4:14:12,  1.83s/it]
                                                        

 31%|███       | 3678/12000 [2:11:45<4:14:12,  1.83s/it]
 31%|███       | 3679/12000 [2:11:46<4:05:26,  1.77s/it]
                                                        

 31%|███       | 3679/12000 [2:11:46<4:05:26,  1.77s/it]
 31%|███       | 3680/12000 [2:11:48<3:55:53,  1.70s/it]
                                                        

 31%|███       | 3680/12000 [2:11:48<3:55:53,  1.70s/it]
 31%|███       | 3681/12000 [2:11:54<6:56:45,  3.01s/it]
                                                        

 31%|███       | 3681/12000 [2:11:54<6:56:45,  3.01s/it]
 31%|███       | 3682/12000 [2:11:55<5:53:47,  2.55s/it]
                                                        

 31%|███       | 3682/12000 [2:11:55<5:53:47,  2.55s/it]
 31%|███       | 3683/12000 [2:11:58<5:38:17,  2.44s/it]
                                                        

 31%|███       | 3683/12000 [2:11:58<5:38:17,  2.44s/it]
 31%|███       | 3684/12000 [2:11:59<5:02:29,  2.18s/it]
                                                        

 31%|███       | 3684/12000 [2:11:59<5:02:29,  2.18s/it]
 31%|███       | 3685/12000 [2:12:01<4:28:40,  1.94s/it]
                                                        

 31%|███       | 3685/12000 [2:12:01<4:28:40,  1.94s/it]
 31%|███       | 3686/12000 [2:12:03<4:39:05,  2.01s/it]
                                                        

 31%|███       | 3686/12000 [2:12:03<4:39:05,  2.01s/it]
 31%|███       | 3687/12000 [2:12:07<6:11:45,  2.68s/it]
                                                        

 31%|███       | 3687/12000 [2:12:07<6:11:45,  2.68s/it]
 31%|███       | 3688/12000 [2:12:09<5:43:05,  2.48s/it]
                                                        

 31%|███       | 3688/12000 [2:12:09<5:43:05,  2.48s/it]
 31%|███       | 3689/12000 [2:12:10<4:58:23,  2.15s/it]
                                                        

 31%|███       | 3689/12000 [2:12:10<4:58:23,  2.15s/it]
 31%|███       | 3690/12000 [2:12:14<6:15:08,  2.71s/it]
                                                        

 31%|███       | 3690/12000 [2:12:14<6:15:08,  2.71s/it]
 31%|███       | 3691/12000 [2:12:16<5:32:06,  2.40s/it]
                                                        

 31%|███       | 3691/12000 [2:12:16<5:32:06,  2.40s/it]
 31%|███       | 3692/12000 [2:12:18<4:53:36,  2.12s/it]
                                                        

 31%|███       | 3692/12000 [2:12:18<4:53:36,  2.12s/it]
 31%|███       | 3693/12000 [2:12:20<4:51:37,  2.11s/it]
                                                        

 31%|███       | 3693/12000 [2:12:20<4:51:37,  2.11s/it]
 31%|███       | 3694/12000 [2:12:21<4:36:58,  2.00s/it]
                                                        

 31%|███       | 3694/12000 [2:12:21<4:36:58,  2.00s/it]
 31%|███       | 3695/12000 [2:12:23<4:28:11,  1.94s/it]
                                                        

 31%|███       | 3695/12000 [2:12:23<4:28:11,  1.94s/it]
 31%|███       | 3696/12000 [2:12:25<4:18:30,  1.87s/it]
                                                        

 31%|███       | 3696/12000 [2:12:25<4:18:30,  1.87s/it]
 31%|███       | 3697/12000 [2:12:26<4:02:27,  1.75s/it]
                                                        

 31%|███       | 3697/12000 [2:12:26<4:02:27,  1.75s/it]
 31%|███       | 3698/12000 [2:12:28<3:58:21,  1.72s/it]
                                                        

 31%|███       | 3698/12000 [2:12:28<3:58:21,  1.72s/it]
 31%|███       | 3699/12000 [2:12:30<3:49:37,  1.66s/it]
                                                        

 31%|███       | 3699/12000 [2:12:30<3:49:37,  1.66s/it]
 31%|███       | 3700/12000 [2:12:31<3:47:17,  1.64s/it]
                                                        

 31%|███       | 3700/12000 [2:12:31<3:47:17,  1.64s/it]
 31%|███       | 3701/12000 [2:12:36<6:00:06,  2.60s/it]
                                                        

 31%|███       | 3701/12000 [2:12:36<6:00:06,  2.60s/it]
 31%|███       | 3702/12000 [2:12:39<6:17:53,  2.73s/it]
                                                        

 31%|███       | 3702/12000 [2:12:39<6:17:53,  2.73s/it]
 31%|███       | 3703/12000 [2:12:41<5:33:43,  2.41s/it]
                                                        

 31%|███       | 3703/12000 [2:12:41<5:33:43,  2.41s/it]
 31%|███       | 3704/12000 [2:12:42<4:51:44,  2.11s/it]
                                                        

 31%|███       | 3704/12000 [2:12:42<4:51:44,  2.11s/it]
 31%|███       | 3705/12000 [2:12:45<5:06:47,  2.22s/it]
                                                        

 31%|███       | 3705/12000 [2:12:45<5:06:47,  2.22s/it]
 31%|███       | 3706/12000 [2:12:46<4:53:27,  2.12s/it]
                                                        

 31%|███       | 3706/12000 [2:12:46<4:53:27,  2.12s/it]
 31%|███       | 3707/12000 [2:12:48<4:42:01,  2.04s/it]
                                                        

 31%|███       | 3707/12000 [2:12:48<4:42:01,  2.04s/it]
 31%|███       | 3708/12000 [2:12:50<4:15:44,  1.85s/it]
                                                        

 31%|███       | 3708/12000 [2:12:50<4:15:44,  1.85s/it]
 31%|███       | 3709/12000 [2:12:53<5:16:13,  2.29s/it]
                                                        

 31%|███       | 3709/12000 [2:12:53<5:16:13,  2.29s/it]
 31%|███       | 3710/12000 [2:12:54<4:43:00,  2.05s/it]
                                                        

 31%|███       | 3710/12000 [2:12:54<4:43:00,  2.05s/it]
 31%|███       | 3711/12000 [2:12:56<4:16:25,  1.86s/it]
                                                        

 31%|███       | 3711/12000 [2:12:56<4:16:25,  1.86s/it]
 31%|███       | 3712/12000 [2:13:01<6:12:11,  2.69s/it]
                                                        

 31%|███       | 3712/12000 [2:13:01<6:12:11,  2.69s/it]
 31%|███       | 3713/12000 [2:13:02<5:29:56,  2.39s/it]
                                                        

 31%|███       | 3713/12000 [2:13:02<5:29:56,  2.39s/it]
 31%|███       | 3714/12000 [2:13:04<5:15:53,  2.29s/it]
                                                        

 31%|███       | 3714/12000 [2:13:04<5:15:53,  2.29s/it]
 31%|███       | 3715/12000 [2:13:06<4:46:59,  2.08s/it]
                                                        

 31%|███       | 3715/12000 [2:13:06<4:46:59,  2.08s/it]
 31%|███       | 3716/12000 [2:13:07<4:23:39,  1.91s/it]
                                                        

 31%|███       | 3716/12000 [2:13:07<4:23:39,  1.91s/it]
 31%|███       | 3717/12000 [2:13:10<5:02:18,  2.19s/it]
                                                        
{'loss': '0.7422', 'grad_norm': '0.3608', 'learning_rate': '0.0001663', 'entropy': '0.7922', 'num_tokens': '1.985e+07', 'mean_token_accuracy': '0.7958', 'epoch': '0.03248'}
{'loss': '0.5988', 'grad_norm': '0.3573', 'learning_rate': '0.0001663', 'entropy': '0.5764', 'num_tokens': '1.986e+07', 'mean_token_accuracy': '0.8364', 'epoch': '0.03249'}
{'loss': '1.183', 'grad_norm': '0.3793', 'learning_rate': '0.0001663', 'entropy': '1.16', 'num_tokens': '1.986e+07', 'mean_token_accuracy': '0.713', 'epoch': '0.0325'}
{'loss': '0.8483', 'grad_norm': '0.3041', 'learning_rate': '0.0001663', 'entropy': '0.8758', 'num_tokens': '1.987e+07', 'mean_token_accuracy': '0.7752', 'epoch': '0.03251'}
{'loss': '1.343', 'grad_norm': '0.3955', 'learning_rate': '0.0001662', 'entropy': '1.299', 'num_tokens': '1.988e+07', 'mean_token_accuracy': '0.6976', 'epoch': '0.03251'}
{'loss': '0.7927', 'grad_norm': '0.4195', 'learning_rate': '0.0001662', 'entropy': '0.8256', 'num_tokens': '1.988e+07', 'mean_token_accuracy': '0.7903', 'epoch': '0.03252'}
{'loss': '0.9991', 'grad_norm': '0.4904', 'learning_rate': '0.0001662', 'entropy': '0.9905', 'num_tokens': '1.989e+07', 'mean_token_accuracy': '0.7518', 'epoch': '0.03253'}
{'loss': '0.6126', 'grad_norm': '0.3249', 'learning_rate': '0.0001662', 'entropy': '0.6433', 'num_tokens': '1.989e+07', 'mean_token_accuracy': '0.842', 'epoch': '0.03254'}
{'loss': '0.5974', 'grad_norm': '0.3132', 'learning_rate': '0.0001662', 'entropy': '0.6503', 'num_tokens': '1.99e+07', 'mean_token_accuracy': '0.8311', 'epoch': '0.03255'}
{'loss': '0.7045', 'grad_norm': '0.2939', 'learning_rate': '0.0001661', 'entropy': '0.7165', 'num_tokens': '1.99e+07', 'mean_token_accuracy': '0.8147', 'epoch': '0.03256'}
{'loss': '0.6648', 'grad_norm': '0.3441', 'learning_rate': '0.0001661', 'entropy': '0.7256', 'num_tokens': '1.991e+07', 'mean_token_accuracy': '0.8317', 'epoch': '0.03257'}
{'loss': '1.307', 'grad_norm': '0.3609', 'learning_rate': '0.0001661', 'entropy': '1.02', 'num_tokens': '1.992e+07', 'mean_token_accuracy': '0.7607', 'epoch': '0.03258'}
{'loss': '1.271', 'grad_norm': '0.3962', 'learning_rate': '0.0001661', 'entropy': '1.204', 'num_tokens': '1.992e+07', 'mean_token_accuracy': '0.7131', 'epoch': '0.03259'}
{'loss': '0.7046', 'grad_norm': '0.3366', 'learning_rate': '0.000166', 'entropy': '0.7484', 'num_tokens': '1.993e+07', 'mean_token_accuracy': '0.8125', 'epoch': '0.03259'}
{'loss': '0.7782', 'grad_norm': '0.4135', 'learning_rate': '0.000166', 'entropy': '0.8792', 'num_tokens': '1.993e+07', 'mean_token_accuracy': '0.7837', 'epoch': '0.0326'}
{'loss': '0.844', 'grad_norm': '0.4575', 'learning_rate': '0.000166', 'entropy': '0.8587', 'num_tokens': '1.994e+07', 'mean_token_accuracy': '0.7873', 'epoch': '0.03261'}
{'loss': '0.9526', 'grad_norm': '0.3601', 'learning_rate': '0.000166', 'entropy': '0.9382', 'num_tokens': '1.994e+07', 'mean_token_accuracy': '0.7682', 'epoch': '0.03262'}
{'loss': '0.8979', 'grad_norm': '0.3254', 'learning_rate': '0.000166', 'entropy': '0.9184', 'num_tokens': '1.996e+07', 'mean_token_accuracy': '0.7681', 'epoch': '0.03263'}
{'loss': '0.9984', 'grad_norm': '0.3563', 'learning_rate': '0.0001659', 'entropy': '0.9508', 'num_tokens': '1.996e+07', 'mean_token_accuracy': '0.7672', 'epoch': '0.03264'}
{'loss': '0.6719', 'grad_norm': '0.3339', 'learning_rate': '0.0001659', 'entropy': '0.6766', 'num_tokens': '1.997e+07', 'mean_token_accuracy': '0.8284', 'epoch': '0.03265'}
{'loss': '0.6418', 'grad_norm': '0.3235', 'learning_rate': '0.0001659', 'entropy': '0.5824', 'num_tokens': '1.997e+07', 'mean_token_accuracy': '0.8484', 'epoch': '0.03266'}
{'loss': '1.203', 'grad_norm': '0.4615', 'learning_rate': '0.0001659', 'entropy': '1.128', 'num_tokens': '1.998e+07', 'mean_token_accuracy': '0.7315', 'epoch': '0.03266'}
{'loss': '0.6674', 'grad_norm': '0.3351', 'learning_rate': '0.0001659', 'entropy': '0.649', 'num_tokens': '1.998e+07', 'mean_token_accuracy': '0.8218', 'epoch': '0.03267'}
{'loss': '1.125', 'grad_norm': '0.4495', 'learning_rate': '0.0001658', 'entropy': '1.008', 'num_tokens': '1.999e+07', 'mean_token_accuracy': '0.7425', 'epoch': '0.03268'}
{'loss': '0.908', 'grad_norm': '0.3879', 'learning_rate': '0.0001658', 'entropy': '0.911', 'num_tokens': '1.999e+07', 'mean_token_accuracy': '0.7636', 'epoch': '0.03269'}
{'loss': '0.7355', 'grad_norm': '0.3152', 'learning_rate': '0.0001658', 'entropy': '0.7533', 'num_tokens': '2e+07', 'mean_token_accuracy': '0.8081', 'epoch': '0.0327'}
{'loss': '0.9382', 'grad_norm': '0.3971', 'learning_rate': '0.0001658', 'entropy': '0.9467', 'num_tokens': '2e+07', 'mean_token_accuracy': '0.7605', 'epoch': '0.03271'}
{'loss': '1.113', 'grad_norm': '0.5066', 'learning_rate': '0.0001658', 'entropy': '1.064', 'num_tokens': '2.001e+07', 'mean_token_accuracy': '0.7371', 'epoch': '0.03272'}
{'loss': '1.206', 'grad_norm': '0.4562', 'learning_rate': '0.0001657', 'entropy': '1.229', 'num_tokens': '2.001e+07', 'mean_token_accuracy': '0.7068', 'epoch': '0.03273'}
{'loss': '0.6931', 'grad_norm': '0.3556', 'learning_rate': '0.0001657', 'entropy': '0.6683', 'num_tokens': '2.002e+07', 'mean_token_accuracy': '0.8222', 'epoch': '0.03274'}
{'loss': '0.8256', 'grad_norm': '0.4023', 'learning_rate': '0.0001657', 'entropy': '0.7976', 'num_tokens': '2.002e+07', 'mean_token_accuracy': '0.7976', 'epoch': '0.03274'}
{'loss': '0.7275', 'grad_norm': '0.3728', 'learning_rate': '0.0001657', 'entropy': '0.7582', 'num_tokens': '2.003e+07', 'mean_token_accuracy': '0.8089', 'epoch': '0.03275'}
{'loss': '1.307', 'grad_norm': '0.452', 'learning_rate': '0.0001657', 'entropy': '1.085', 'num_tokens': '2.004e+07', 'mean_token_accuracy': '0.7318', 'epoch': '0.03276'}
{'loss': '0.7624', 'grad_norm': '0.3437', 'learning_rate': '0.0001656', 'entropy': '0.7877', 'num_tokens': '2.004e+07', 'mean_token_accuracy': '0.8075', 'epoch': '0.03277'}
{'loss': '0.5307', 'grad_norm': '0.2993', 'learning_rate': '0.0001656', 'entropy': '0.6009', 'num_tokens': '2.004e+07', 'mean_token_accuracy': '0.8484', 'epoch': '0.03278'}
{'loss': '1.073', 'grad_norm': '0.3772', 'learning_rate': '0.0001656', 'entropy': '1.059', 'num_tokens': '2.005e+07', 'mean_token_accuracy': '0.7452', 'epoch': '0.03279'}
{'loss': '0.9259', 'grad_norm': '0.5344', 'learning_rate': '0.0001656', 'entropy': '0.8437', 'num_tokens': '2.006e+07', 'mean_token_accuracy': '0.7818', 'epoch': '0.0328'}
{'loss': '0.9718', 'grad_norm': '0.3826', 'learning_rate': '0.0001656', 'entropy': '0.9629', 'num_tokens': '2.006e+07', 'mean_token_accuracy': '0.7681', 'epoch': '0.03281'}
{'loss': '0.9097', 'grad_norm': '0.4199', 'learning_rate': '0.0001655', 'entropy': '0.9063', 'num_tokens': '2.007e+07', 'mean_token_accuracy': '0.7692', 'epoch': '0.03282'}
{'loss': '0.8393', 'grad_norm': '0.3682', 'learning_rate': '0.0001655', 'entropy': '0.8463', 'num_tokens': '2.008e+07', 'mean_token_accuracy': '0.7885', 'epoch': '0.03282'}
{'loss': '0.7273', 'grad_norm': '0.3596', 'learning_rate': '0.0001655', 'entropy': '0.7943', 'num_tokens': '2.008e+07', 'mean_token_accuracy': '0.7911', 'epoch': '0.03283'}
{'loss': '0.8656', 'grad_norm': '0.3784', 'learning_rate': '0.0001655', 'entropy': '0.8937', 'num_tokens': '2.008e+07', 'mean_token_accuracy': '0.7756', 'epoch': '0.03284'}
{'loss': '1.028', 'grad_norm': '0.4079', 'learning_rate': '0.0001654', 'entropy': '0.9864', 'num_tokens': '2.009e+07', 'mean_token_accuracy': '0.7635', 'epoch': '0.03285'}
{'loss': '0.8161', 'grad_norm': '0.4374', 'learning_rate': '0.0001654', 'entropy': '0.8104', 'num_tokens': '2.01e+07', 'mean_token_accuracy': '0.7791', 'epoch': '0.03286'}
{'loss': '1.118', 'grad_norm': '0.3549', 'learning_rate': '0.0001654', 'entropy': '1.02', 'num_tokens': '2.01e+07', 'mean_token_accuracy': '0.7475', 'epoch': '0.03287'}
{'loss': '0.8265', 'grad_norm': '0.4281', 'learning_rate': '0.0001654', 'entropy': '0.8931', 'num_tokens': '2.011e+07', 'mean_token_accuracy': '0.7864', 'epoch': '0.03288'}
{'loss': '0.7078', 'grad_norm': '0.3275', 'learning_rate': '0.0001654', 'entropy': '0.7338', 'num_tokens': '2.011e+07', 'mean_token_accuracy': '0.8134', 'epoch': '0.03289'}

 31%|███       | 3717/12000 [2:13:10<5:02:18,  2.19s/it]
 31%|███       | 3718/12000 [2:13:13<5:15:28,  2.29s/it]
                                                        

 31%|███       | 3718/12000 [2:13:13<5:15:28,  2.29s/it]
 31%|███       | 3719/12000 [2:13:14<4:43:31,  2.05s/it]
                                                        

 31%|███       | 3719/12000 [2:13:14<4:43:31,  2.05s/it]
 31%|███       | 3720/12000 [2:13:16<4:30:27,  1.96s/it]
                                                        

 31%|███       | 3720/12000 [2:13:16<4:30:27,  1.96s/it]
 31%|███       | 3721/12000 [2:13:17<4:10:39,  1.82s/it]
                                                        

 31%|███       | 3721/12000 [2:13:17<4:10:39,  1.82s/it]
 31%|███       | 3722/12000 [2:13:20<4:27:49,  1.94s/it]
                                                        

 31%|███       | 3722/12000 [2:13:20<4:27:49,  1.94s/it]
 31%|███       | 3723/12000 [2:13:22<4:53:21,  2.13s/it]
                                                        

 31%|███       | 3723/12000 [2:13:22<4:53:21,  2.13s/it]
 31%|███       | 3724/12000 [2:13:25<5:15:26,  2.29s/it]
                                                        

 31%|███       | 3724/12000 [2:13:25<5:15:26,  2.29s/it]
 31%|███       | 3725/12000 [2:13:26<4:36:29,  2.00s/it]
                                                        

 31%|███       | 3725/12000 [2:13:26<4:36:29,  2.00s/it]
 31%|███       | 3726/12000 [2:13:28<4:38:11,  2.02s/it]
                                                        

 31%|███       | 3726/12000 [2:13:28<4:38:11,  2.02s/it]
 31%|███       | 3727/12000 [2:13:30<4:20:10,  1.89s/it]
                                                        

 31%|███       | 3727/12000 [2:13:30<4:20:10,  1.89s/it]
 31%|███       | 3728/12000 [2:13:32<4:09:44,  1.81s/it]
                                                        

 31%|███       | 3728/12000 [2:13:32<4:09:44,  1.81s/it]
 31%|███       | 3729/12000 [2:13:33<4:12:15,  1.83s/it]
                                                        

 31%|███       | 3729/12000 [2:13:33<4:12:15,  1.83s/it]
 31%|███       | 3730/12000 [2:13:38<6:06:09,  2.66s/it]
                                                        

 31%|███       | 3730/12000 [2:13:38<6:06:09,  2.66s/it]
 31%|███       | 3731/12000 [2:13:41<6:35:32,  2.87s/it]
                                                        

 31%|███       | 3731/12000 [2:13:41<6:35:32,  2.87s/it]
 31%|███       | 3732/12000 [2:13:43<5:52:53,  2.56s/it]
                                                        

 31%|███       | 3732/12000 [2:13:43<5:52:53,  2.56s/it]
 31%|███       | 3733/12000 [2:13:45<5:37:44,  2.45s/it]
                                                        

 31%|███       | 3733/12000 [2:13:45<5:37:44,  2.45s/it]
 31%|███       | 3734/12000 [2:13:50<7:11:29,  3.13s/it]
                                                        

 31%|███       | 3734/12000 [2:13:50<7:11:29,  3.13s/it]
 31%|███       | 3735/12000 [2:13:52<6:02:24,  2.63s/it]
                                                        

 31%|███       | 3735/12000 [2:13:52<6:02:24,  2.63s/it]
 31%|███       | 3736/12000 [2:13:53<5:17:11,  2.30s/it]
                                                        

 31%|███       | 3736/12000 [2:13:53<5:17:11,  2.30s/it]
 31%|███       | 3737/12000 [2:13:55<4:39:15,  2.03s/it]
                                                        

 31%|███       | 3737/12000 [2:13:55<4:39:15,  2.03s/it]
 31%|███       | 3738/12000 [2:13:56<4:13:31,  1.84s/it]
                                                        

 31%|███       | 3738/12000 [2:13:56<4:13:31,  1.84s/it]
 31%|███       | 3739/12000 [2:13:57<3:54:22,  1.70s/it]
                                                        

 31%|███       | 3739/12000 [2:13:57<3:54:22,  1.70s/it]
 31%|███       | 3740/12000 [2:14:00<4:36:49,  2.01s/it]
                                                        

 31%|███       | 3740/12000 [2:14:00<4:36:49,  2.01s/it]
 31%|███       | 3741/12000 [2:14:01<4:07:11,  1.80s/it]
                                                        

 31%|███       | 3741/12000 [2:14:01<4:07:11,  1.80s/it]
 31%|███       | 3742/12000 [2:14:03<4:20:40,  1.89s/it]
                                                        

 31%|███       | 3742/12000 [2:14:03<4:20:40,  1.89s/it]
 31%|███       | 3743/12000 [2:14:08<6:22:45,  2.78s/it]
                                                        

 31%|███       | 3743/12000 [2:14:08<6:22:45,  2.78s/it]
 31%|███       | 3744/12000 [2:14:10<5:23:58,  2.35s/it]
                                                        

 31%|███       | 3744/12000 [2:14:10<5:23:58,  2.35s/it]
 31%|███       | 3745/12000 [2:14:12<5:13:38,  2.28s/it]
                                                        

 31%|███       | 3745/12000 [2:14:12<5:13:38,  2.28s/it]
 31%|███       | 3746/12000 [2:14:15<5:35:49,  2.44s/it]
                                                        

 31%|███       | 3746/12000 [2:14:15<5:35:49,  2.44s/it]
 31%|███       | 3747/12000 [2:14:16<5:00:58,  2.19s/it]
                                                        

 31%|███       | 3747/12000 [2:14:16<5:00:58,  2.19s/it]
 31%|███       | 3748/12000 [2:14:18<4:34:09,  1.99s/it]
                                                        

 31%|███       | 3748/12000 [2:14:18<4:34:09,  1.99s/it]
 31%|███       | 3749/12000 [2:14:20<4:33:46,  1.99s/it]
                                                        

 31%|███       | 3749/12000 [2:14:20<4:33:46,  1.99s/it]
 31%|███▏      | 3750/12000 [2:14:23<5:28:37,  2.39s/it]
                                                        

 31%|███▏      | 3750/12000 [2:14:23<5:28:37,  2.39s/it]
 31%|███▏      | 3751/12000 [2:14:26<5:35:08,  2.44s/it]
                                                        

 31%|███▏      | 3751/12000 [2:14:26<5:35:08,  2.44s/it]
 31%|███▏      | 3752/12000 [2:14:27<5:04:37,  2.22s/it]
                                                        

 31%|███▏      | 3752/12000 [2:14:27<5:04:37,  2.22s/it]
 31%|███▏      | 3753/12000 [2:14:29<4:35:20,  2.00s/it]
                                                        

 31%|███▏      | 3753/12000 [2:14:29<4:35:20,  2.00s/it]
 31%|███▏      | 3754/12000 [2:14:30<4:07:13,  1.80s/it]
                                                        

 31%|███▏      | 3754/12000 [2:14:30<4:07:13,  1.80s/it]
 31%|███▏      | 3755/12000 [2:14:33<4:58:37,  2.17s/it]
                                                        

 31%|███▏      | 3755/12000 [2:14:33<4:58:37,  2.17s/it]
 31%|███▏      | 3756/12000 [2:14:36<5:38:38,  2.46s/it]
                                                        

 31%|███▏      | 3756/12000 [2:14:36<5:38:38,  2.46s/it]
 31%|███▏      | 3757/12000 [2:14:39<6:08:39,  2.68s/it]
                                                        

 31%|███▏      | 3757/12000 [2:14:39<6:08:39,  2.68s/it]
 31%|███▏      | 3758/12000 [2:14:41<5:28:23,  2.39s/it]
                                                        

 31%|███▏      | 3758/12000 [2:14:41<5:28:23,  2.39s/it]
 31%|███▏      | 3759/12000 [2:14:44<5:50:02,  2.55s/it]
                                                        

 31%|███▏      | 3759/12000 [2:14:44<5:50:02,  2.55s/it]
 31%|███▏      | 3760/12000 [2:14:46<5:11:32,  2.27s/it]
                                                        

 31%|███▏      | 3760/12000 [2:14:46<5:11:32,  2.27s/it]
 31%|███▏      | 3761/12000 [2:14:49<5:51:38,  2.56s/it]
                                                        

 31%|███▏      | 3761/12000 [2:14:49<5:51:38,  2.56s/it]
 31%|███▏      | 3762/12000 [2:14:52<5:58:57,  2.61s/it]
                                                        

 31%|███▏      | 3762/12000 [2:14:52<5:58:57,  2.61s/it]
 31%|███▏      | 3763/12000 [2:14:54<5:27:56,  2.39s/it]
                                                        

 31%|███▏      | 3763/12000 [2:14:54<5:27:56,  2.39s/it]
 31%|███▏      | 3764/12000 [2:14:55<4:57:08,  2.16s/it]
                                                        
{'loss': '1.172', 'grad_norm': '0.4693', 'learning_rate': '0.0001653', 'entropy': '0.9951', 'num_tokens': '2.012e+07', 'mean_token_accuracy': '0.77', 'epoch': '0.03289'}
{'loss': '0.6611', 'grad_norm': '0.299', 'learning_rate': '0.0001653', 'entropy': '0.7126', 'num_tokens': '2.012e+07', 'mean_token_accuracy': '0.812', 'epoch': '0.0329'}
{'loss': '0.9097', 'grad_norm': '0.3962', 'learning_rate': '0.0001653', 'entropy': '0.9373', 'num_tokens': '2.013e+07', 'mean_token_accuracy': '0.7703', 'epoch': '0.03291'}
{'loss': '0.8529', 'grad_norm': '0.452', 'learning_rate': '0.0001653', 'entropy': '0.8712', 'num_tokens': '2.013e+07', 'mean_token_accuracy': '0.7755', 'epoch': '0.03292'}
{'loss': '0.8398', 'grad_norm': '0.4009', 'learning_rate': '0.0001653', 'entropy': '0.9341', 'num_tokens': '2.014e+07', 'mean_token_accuracy': '0.7787', 'epoch': '0.03293'}
{'loss': '1.198', 'grad_norm': '0.5127', 'learning_rate': '0.0001652', 'entropy': '1.196', 'num_tokens': '2.014e+07', 'mean_token_accuracy': '0.7214', 'epoch': '0.03294'}
{'loss': '0.7155', 'grad_norm': '0.3446', 'learning_rate': '0.0001652', 'entropy': '0.7085', 'num_tokens': '2.015e+07', 'mean_token_accuracy': '0.8075', 'epoch': '0.03295'}
{'loss': '0.959', 'grad_norm': '0.3075', 'learning_rate': '0.0001652', 'entropy': '0.9453', 'num_tokens': '2.016e+07', 'mean_token_accuracy': '0.7518', 'epoch': '0.03296'}
{'loss': '0.6709', 'grad_norm': '0.3584', 'learning_rate': '0.0001652', 'entropy': '0.6247', 'num_tokens': '2.016e+07', 'mean_token_accuracy': '0.8319', 'epoch': '0.03297'}
{'loss': '0.9651', 'grad_norm': '0.3238', 'learning_rate': '0.0001652', 'entropy': '0.9102', 'num_tokens': '2.017e+07', 'mean_token_accuracy': '0.7696', 'epoch': '0.03297'}
{'loss': '0.9533', 'grad_norm': '0.3366', 'learning_rate': '0.0001651', 'entropy': '0.9376', 'num_tokens': '2.017e+07', 'mean_token_accuracy': '0.7497', 'epoch': '0.03298'}
{'loss': '1.357', 'grad_norm': '0.442', 'learning_rate': '0.0001651', 'entropy': '1.303', 'num_tokens': '2.018e+07', 'mean_token_accuracy': '0.6892', 'epoch': '0.03299'}
{'loss': '0.9865', 'grad_norm': '0.4224', 'learning_rate': '0.0001651', 'entropy': '0.959', 'num_tokens': '2.019e+07', 'mean_token_accuracy': '0.7578', 'epoch': '0.033'}
{'loss': '1.135', 'grad_norm': '0.3717', 'learning_rate': '0.0001651', 'entropy': '0.7942', 'num_tokens': '2.019e+07', 'mean_token_accuracy': '0.8135', 'epoch': '0.03301'}
{'loss': '0.7238', 'grad_norm': '0.414', 'learning_rate': '0.000165', 'entropy': '0.8358', 'num_tokens': '2.02e+07', 'mean_token_accuracy': '0.8065', 'epoch': '0.03302'}
{'loss': '1.164', 'grad_norm': '0.4729', 'learning_rate': '0.000165', 'entropy': '1.155', 'num_tokens': '2.021e+07', 'mean_token_accuracy': '0.7232', 'epoch': '0.03303'}
{'loss': '1.329', 'grad_norm': '0.4042', 'learning_rate': '0.000165', 'entropy': '1.003', 'num_tokens': '2.021e+07', 'mean_token_accuracy': '0.759', 'epoch': '0.03304'}
{'loss': '0.8688', 'grad_norm': '0.3953', 'learning_rate': '0.000165', 'entropy': '0.8489', 'num_tokens': '2.022e+07', 'mean_token_accuracy': '0.7945', 'epoch': '0.03305'}
{'loss': '0.6525', 'grad_norm': '0.3442', 'learning_rate': '0.000165', 'entropy': '0.6834', 'num_tokens': '2.022e+07', 'mean_token_accuracy': '0.8322', 'epoch': '0.03305'}
{'loss': '1.091', 'grad_norm': '0.3714', 'learning_rate': '0.0001649', 'entropy': '1.004', 'num_tokens': '2.023e+07', 'mean_token_accuracy': '0.7564', 'epoch': '0.03306'}
{'loss': '0.6044', 'grad_norm': '0.3259', 'learning_rate': '0.0001649', 'entropy': '0.7086', 'num_tokens': '2.023e+07', 'mean_token_accuracy': '0.819', 'epoch': '0.03307'}
{'loss': '1.037', 'grad_norm': '0.3851', 'learning_rate': '0.0001649', 'entropy': '1.08', 'num_tokens': '2.023e+07', 'mean_token_accuracy': '0.7514', 'epoch': '0.03308'}
{'loss': '0.7966', 'grad_norm': '0.4495', 'learning_rate': '0.0001649', 'entropy': '0.9026', 'num_tokens': '2.024e+07', 'mean_token_accuracy': '0.7695', 'epoch': '0.03309'}
{'loss': '0.8574', 'grad_norm': '0.3851', 'learning_rate': '0.0001649', 'entropy': '0.8954', 'num_tokens': '2.025e+07', 'mean_token_accuracy': '0.7855', 'epoch': '0.0331'}
{'loss': '1.009', 'grad_norm': '0.4312', 'learning_rate': '0.0001648', 'entropy': '0.998', 'num_tokens': '2.025e+07', 'mean_token_accuracy': '0.7444', 'epoch': '0.03311'}
{'loss': '0.8811', 'grad_norm': '0.4445', 'learning_rate': '0.0001648', 'entropy': '0.8992', 'num_tokens': '2.026e+07', 'mean_token_accuracy': '0.7824', 'epoch': '0.03312'}
{'loss': '1.195', 'grad_norm': '0.3484', 'learning_rate': '0.0001648', 'entropy': '0.8937', 'num_tokens': '2.027e+07', 'mean_token_accuracy': '0.7932', 'epoch': '0.03312'}
{'loss': '0.6556', 'grad_norm': '0.3701', 'learning_rate': '0.0001648', 'entropy': '0.7257', 'num_tokens': '2.027e+07', 'mean_token_accuracy': '0.831', 'epoch': '0.03313'}
{'loss': '1.155', 'grad_norm': '0.4884', 'learning_rate': '0.0001648', 'entropy': '1.033', 'num_tokens': '2.027e+07', 'mean_token_accuracy': '0.7389', 'epoch': '0.03314'}
{'loss': '0.9165', 'grad_norm': '0.4506', 'learning_rate': '0.0001647', 'entropy': '0.9731', 'num_tokens': '2.028e+07', 'mean_token_accuracy': '0.7529', 'epoch': '0.03315'}
{'loss': '0.8976', 'grad_norm': '0.4392', 'learning_rate': '0.0001647', 'entropy': '0.8462', 'num_tokens': '2.029e+07', 'mean_token_accuracy': '0.7909', 'epoch': '0.03316'}
{'loss': '0.833', 'grad_norm': '0.3905', 'learning_rate': '0.0001647', 'entropy': '0.8428', 'num_tokens': '2.029e+07', 'mean_token_accuracy': '0.7979', 'epoch': '0.03317'}
{'loss': '0.9299', 'grad_norm': '0.3976', 'learning_rate': '0.0001647', 'entropy': '0.8681', 'num_tokens': '2.03e+07', 'mean_token_accuracy': '0.7754', 'epoch': '0.03318'}
{'loss': '0.909', 'grad_norm': '0.3699', 'learning_rate': '0.0001647', 'entropy': '1.041', 'num_tokens': '2.03e+07', 'mean_token_accuracy': '0.7508', 'epoch': '0.03319'}
{'loss': '1.21', 'grad_norm': '0.6848', 'learning_rate': '0.0001646', 'entropy': '1.217', 'num_tokens': '2.031e+07', 'mean_token_accuracy': '0.6997', 'epoch': '0.0332'}
{'loss': '0.8504', 'grad_norm': '0.4031', 'learning_rate': '0.0001646', 'entropy': '0.8417', 'num_tokens': '2.031e+07', 'mean_token_accuracy': '0.7784', 'epoch': '0.0332'}
{'loss': '0.6231', 'grad_norm': '0.3864', 'learning_rate': '0.0001646', 'entropy': '0.6144', 'num_tokens': '2.032e+07', 'mean_token_accuracy': '0.8388', 'epoch': '0.03321'}
{'loss': '0.7503', 'grad_norm': '0.3628', 'learning_rate': '0.0001646', 'entropy': '0.7498', 'num_tokens': '2.032e+07', 'mean_token_accuracy': '0.8065', 'epoch': '0.03322'}
{'loss': '1.154', 'grad_norm': '0.3758', 'learning_rate': '0.0001645', 'entropy': '1.135', 'num_tokens': '2.033e+07', 'mean_token_accuracy': '0.7356', 'epoch': '0.03323'}
{'loss': '0.73', 'grad_norm': '0.3345', 'learning_rate': '0.0001645', 'entropy': '0.7495', 'num_tokens': '2.033e+07', 'mean_token_accuracy': '0.8163', 'epoch': '0.03324'}
{'loss': '0.9319', 'grad_norm': '0.439', 'learning_rate': '0.0001645', 'entropy': '0.9982', 'num_tokens': '2.034e+07', 'mean_token_accuracy': '0.7703', 'epoch': '0.03325'}
{'loss': '0.7295', 'grad_norm': '0.3689', 'learning_rate': '0.0001645', 'entropy': '0.7694', 'num_tokens': '2.035e+07', 'mean_token_accuracy': '0.8021', 'epoch': '0.03326'}
{'loss': '0.8709', 'grad_norm': '0.3977', 'learning_rate': '0.0001645', 'entropy': '0.9057', 'num_tokens': '2.035e+07', 'mean_token_accuracy': '0.7716', 'epoch': '0.03327'}
{'loss': '0.8184', 'grad_norm': '0.3847', 'learning_rate': '0.0001644', 'entropy': '0.8642', 'num_tokens': '2.036e+07', 'mean_token_accuracy': '0.7822', 'epoch': '0.03328'}
{'loss': '0.9013', 'grad_norm': '0.3808', 'learning_rate': '0.0001644', 'entropy': '0.8888', 'num_tokens': '2.037e+07', 'mean_token_accuracy': '0.7828', 'epoch': '0.03328'}
{'loss': '1.112', 'grad_norm': '0.3087', 'learning_rate': '0.0001644', 'entropy': '1.024', 'num_tokens': '2.037e+07', 'mean_token_accuracy': '0.764', 'epoch': '0.03329'}
{'loss': '0.8414', 'grad_norm': '0.3746', 'learning_rate': '0.0001644', 'entropy': '0.8683', 'num_tokens': '2.038e+07', 'mean_token_accuracy': '0.796', 'epoch': '0.0333'}

 31%|███▏      | 3764/12000 [2:14:55<4:57:08,  2.16s/it]
 31%|███▏      | 3765/12000 [2:14:57<4:38:30,  2.03s/it]
                                                        

 31%|███▏      | 3765/12000 [2:14:57<4:38:30,  2.03s/it]
 31%|███▏      | 3766/12000 [2:14:59<4:33:13,  1.99s/it]
                                                        

 31%|███▏      | 3766/12000 [2:14:59<4:33:13,  1.99s/it]
 31%|███▏      | 3767/12000 [2:15:00<4:11:49,  1.84s/it]
                                                        

 31%|███▏      | 3767/12000 [2:15:00<4:11:49,  1.84s/it]
 31%|███▏      | 3768/12000 [2:15:02<4:04:33,  1.78s/it]
                                                        

 31%|███▏      | 3768/12000 [2:15:02<4:04:33,  1.78s/it]
 31%|███▏      | 3769/12000 [2:15:04<4:21:59,  1.91s/it]
                                                        

 31%|███▏      | 3769/12000 [2:15:04<4:21:59,  1.91s/it]
 31%|███▏      | 3770/12000 [2:15:06<4:13:29,  1.85s/it]
                                                        

 31%|███▏      | 3770/12000 [2:15:06<4:13:29,  1.85s/it]
 31%|███▏      | 3771/12000 [2:15:08<4:10:02,  1.82s/it]
                                                        

 31%|███▏      | 3771/12000 [2:15:08<4:10:02,  1.82s/it]
 31%|███▏      | 3772/12000 [2:15:09<4:06:06,  1.79s/it]
                                                        

 31%|███▏      | 3772/12000 [2:15:09<4:06:06,  1.79s/it]
 31%|███▏      | 3773/12000 [2:15:11<4:02:00,  1.76s/it]
                                                        

 31%|███▏      | 3773/12000 [2:15:11<4:02:00,  1.76s/it]
 31%|███▏      | 3774/12000 [2:15:15<5:36:30,  2.45s/it]
                                                        

 31%|███▏      | 3774/12000 [2:15:15<5:36:30,  2.45s/it]
 31%|███▏      | 3775/12000 [2:15:19<6:16:15,  2.74s/it]
                                                        

 31%|███▏      | 3775/12000 [2:15:19<6:16:15,  2.74s/it]
 31%|███▏      | 3776/12000 [2:15:20<5:24:25,  2.37s/it]
                                                        

 31%|███▏      | 3776/12000 [2:15:20<5:24:25,  2.37s/it]
 31%|███▏      | 3777/12000 [2:15:21<4:46:46,  2.09s/it]
                                                        

 31%|███▏      | 3777/12000 [2:15:21<4:46:46,  2.09s/it]
 31%|███▏      | 3778/12000 [2:15:26<6:36:14,  2.89s/it]
                                                        

 31%|███▏      | 3778/12000 [2:15:26<6:36:14,  2.89s/it]
 31%|███▏      | 3779/12000 [2:15:28<6:08:51,  2.69s/it]
                                                        

 31%|███▏      | 3779/12000 [2:15:28<6:08:51,  2.69s/it]
 32%|███▏      | 3780/12000 [2:15:30<5:16:32,  2.31s/it]
                                                        

 32%|███▏      | 3780/12000 [2:15:30<5:16:32,  2.31s/it]
 32%|███▏      | 3781/12000 [2:15:31<4:38:25,  2.03s/it]
                                                        

 32%|███▏      | 3781/12000 [2:15:31<4:38:25,  2.03s/it]
 32%|███▏      | 3782/12000 [2:15:33<4:40:09,  2.05s/it]
                                                        

 32%|███▏      | 3782/12000 [2:15:33<4:40:09,  2.05s/it]
 32%|███▏      | 3783/12000 [2:15:36<5:19:47,  2.34s/it]
                                                        

 32%|███▏      | 3783/12000 [2:15:36<5:19:47,  2.34s/it]
 32%|███▏      | 3784/12000 [2:15:38<4:46:51,  2.09s/it]
                                                        

 32%|███▏      | 3784/12000 [2:15:38<4:46:51,  2.09s/it]
 32%|███▏      | 3785/12000 [2:15:39<4:19:32,  1.90s/it]
                                                        

 32%|███▏      | 3785/12000 [2:15:39<4:19:32,  1.90s/it]
 32%|███▏      | 3786/12000 [2:15:41<4:20:40,  1.90s/it]
                                                        

 32%|███▏      | 3786/12000 [2:15:41<4:20:40,  1.90s/it]
 32%|███▏      | 3787/12000 [2:15:43<4:03:21,  1.78s/it]
                                                        

 32%|███▏      | 3787/12000 [2:15:43<4:03:21,  1.78s/it]
 32%|███▏      | 3788/12000 [2:15:44<3:49:57,  1.68s/it]
                                                        

 32%|███▏      | 3788/12000 [2:15:44<3:49:57,  1.68s/it]
 32%|███▏      | 3789/12000 [2:15:49<6:00:20,  2.63s/it]
                                                        

 32%|███▏      | 3789/12000 [2:15:49<6:00:20,  2.63s/it]
 32%|███▏      | 3790/12000 [2:15:51<5:19:30,  2.34s/it]
                                                        

 32%|███▏      | 3790/12000 [2:15:51<5:19:30,  2.34s/it]
 32%|███▏      | 3791/12000 [2:15:52<4:41:48,  2.06s/it]
                                                        

 32%|███▏      | 3791/12000 [2:15:52<4:41:48,  2.06s/it]
 32%|███▏      | 3792/12000 [2:15:54<4:26:16,  1.95s/it]
                                                        

 32%|███▏      | 3792/12000 [2:15:54<4:26:16,  1.95s/it]
 32%|███▏      | 3793/12000 [2:15:57<5:01:01,  2.20s/it]
                                                        

 32%|███▏      | 3793/12000 [2:15:57<5:01:01,  2.20s/it]
 32%|███▏      | 3794/12000 [2:15:59<4:56:30,  2.17s/it]
                                                        

 32%|███▏      | 3794/12000 [2:15:59<4:56:30,  2.17s/it]
 32%|███▏      | 3795/12000 [2:16:03<6:46:23,  2.97s/it]
                                                        

 32%|███▏      | 3795/12000 [2:16:03<6:46:23,  2.97s/it]
 32%|███▏      | 3796/12000 [2:16:05<5:42:36,  2.51s/it]
                                                        

 32%|███▏      | 3796/12000 [2:16:05<5:42:36,  2.51s/it]
 32%|███▏      | 3797/12000 [2:16:07<5:18:21,  2.33s/it]
                                                        

 32%|███▏      | 3797/12000 [2:16:07<5:18:21,  2.33s/it]
 32%|███▏      | 3798/12000 [2:16:08<4:41:18,  2.06s/it]
                                                        

 32%|███▏      | 3798/12000 [2:16:08<4:41:18,  2.06s/it]
 32%|███▏      | 3799/12000 [2:16:10<4:10:01,  1.83s/it]
                                                        

 32%|███▏      | 3799/12000 [2:16:10<4:10:01,  1.83s/it]
 32%|███▏      | 3800/12000 [2:16:13<5:23:56,  2.37s/it]
                                                        

 32%|███▏      | 3800/12000 [2:16:13<5:23:56,  2.37s/it]
 32%|███▏      | 3801/12000 [2:16:16<5:53:31,  2.59s/it]
                                                        

 32%|███▏      | 3801/12000 [2:16:16<5:53:31,  2.59s/it]
 32%|███▏      | 3802/12000 [2:16:18<5:05:37,  2.24s/it]
                                                        

 32%|███▏      | 3802/12000 [2:16:18<5:05:37,  2.24s/it]
 32%|███▏      | 3803/12000 [2:16:19<4:31:22,  1.99s/it]
                                                        

 32%|███▏      | 3803/12000 [2:16:19<4:31:22,  1.99s/it]
 32%|███▏      | 3804/12000 [2:16:21<4:21:50,  1.92s/it]
                                                        

 32%|███▏      | 3804/12000 [2:16:21<4:21:50,  1.92s/it]
 32%|███▏      | 3805/12000 [2:16:22<4:07:55,  1.82s/it]
                                                        

 32%|███▏      | 3805/12000 [2:16:22<4:07:55,  1.82s/it]
 32%|███▏      | 3806/12000 [2:16:24<4:07:17,  1.81s/it]
                                                        

 32%|███▏      | 3806/12000 [2:16:24<4:07:17,  1.81s/it]
 32%|███▏      | 3807/12000 [2:16:27<4:53:25,  2.15s/it]
                                                        

 32%|███▏      | 3807/12000 [2:16:27<4:53:25,  2.15s/it]
 32%|███▏      | 3808/12000 [2:16:29<4:20:22,  1.91s/it]
                                                        

 32%|███▏      | 3808/12000 [2:16:29<4:20:22,  1.91s/it]
 32%|███▏      | 3809/12000 [2:16:31<4:25:40,  1.95s/it]
                                                        

 32%|███▏      | 3809/12000 [2:16:31<4:25:40,  1.95s/it]
 32%|███▏      | 3810/12000 [2:16:32<4:01:17,  1.77s/it]
                                                        

 32%|███▏      | 3810/12000 [2:16:32<4:01:17,  1.77s/it]
 32%|███▏      | 3811/12000 [2:16:33<3:50:46,  1.69s/it]
                                                        
{'loss': '0.8535', 'grad_norm': '0.3584', 'learning_rate': '0.0001644', 'entropy': '0.8177', 'num_tokens': '2.038e+07', 'mean_token_accuracy': '0.7886', 'epoch': '0.03331'}
{'loss': '0.7619', 'grad_norm': '0.38', 'learning_rate': '0.0001643', 'entropy': '0.7434', 'num_tokens': '2.039e+07', 'mean_token_accuracy': '0.8064', 'epoch': '0.03332'}
{'loss': '0.7876', 'grad_norm': '0.4316', 'learning_rate': '0.0001643', 'entropy': '0.7819', 'num_tokens': '2.04e+07', 'mean_token_accuracy': '0.792', 'epoch': '0.03333'}
{'loss': '1.32', 'grad_norm': '0.4485', 'learning_rate': '0.0001643', 'entropy': '1.353', 'num_tokens': '2.04e+07', 'mean_token_accuracy': '0.6809', 'epoch': '0.03334'}
{'loss': '0.8261', 'grad_norm': '0.3677', 'learning_rate': '0.0001643', 'entropy': '0.8173', 'num_tokens': '2.041e+07', 'mean_token_accuracy': '0.7901', 'epoch': '0.03335'}
{'loss': '1.119', 'grad_norm': '0.4229', 'learning_rate': '0.0001643', 'entropy': '1.168', 'num_tokens': '2.041e+07', 'mean_token_accuracy': '0.7214', 'epoch': '0.03336'}
{'loss': '0.8209', 'grad_norm': '0.3588', 'learning_rate': '0.0001642', 'entropy': '0.8371', 'num_tokens': '2.042e+07', 'mean_token_accuracy': '0.7849', 'epoch': '0.03336'}
{'loss': '0.9912', 'grad_norm': '0.3839', 'learning_rate': '0.0001642', 'entropy': '0.9632', 'num_tokens': '2.042e+07', 'mean_token_accuracy': '0.7645', 'epoch': '0.03337'}
{'loss': '0.8123', 'grad_norm': '0.3818', 'learning_rate': '0.0001642', 'entropy': '0.8259', 'num_tokens': '2.043e+07', 'mean_token_accuracy': '0.7943', 'epoch': '0.03338'}
{'loss': '1.128', 'grad_norm': '0.4229', 'learning_rate': '0.0001642', 'entropy': '1.12', 'num_tokens': '2.043e+07', 'mean_token_accuracy': '0.7277', 'epoch': '0.03339'}
{'loss': '1.004', 'grad_norm': '0.3156', 'learning_rate': '0.0001641', 'entropy': '1.207', 'num_tokens': '2.044e+07', 'mean_token_accuracy': '0.7257', 'epoch': '0.0334'}
{'loss': '0.9317', 'grad_norm': '0.3995', 'learning_rate': '0.0001641', 'entropy': '1.03', 'num_tokens': '2.045e+07', 'mean_token_accuracy': '0.7486', 'epoch': '0.03341'}
{'loss': '0.7791', 'grad_norm': '0.3342', 'learning_rate': '0.0001641', 'entropy': '0.7817', 'num_tokens': '2.045e+07', 'mean_token_accuracy': '0.7993', 'epoch': '0.03342'}
{'loss': '0.7191', 'grad_norm': '0.3881', 'learning_rate': '0.0001641', 'entropy': '0.712', 'num_tokens': '2.046e+07', 'mean_token_accuracy': '0.8171', 'epoch': '0.03343'}
{'loss': '1.155', 'grad_norm': '0.3733', 'learning_rate': '0.0001641', 'entropy': '1.114', 'num_tokens': '2.047e+07', 'mean_token_accuracy': '0.7275', 'epoch': '0.03343'}
{'loss': '0.8897', 'grad_norm': '0.3649', 'learning_rate': '0.000164', 'entropy': '0.8044', 'num_tokens': '2.047e+07', 'mean_token_accuracy': '0.8011', 'epoch': '0.03344'}
{'loss': '0.6741', 'grad_norm': '0.322', 'learning_rate': '0.000164', 'entropy': '0.7015', 'num_tokens': '2.048e+07', 'mean_token_accuracy': '0.8265', 'epoch': '0.03345'}
{'loss': '0.9104', 'grad_norm': '0.4346', 'learning_rate': '0.000164', 'entropy': '0.9571', 'num_tokens': '2.048e+07', 'mean_token_accuracy': '0.7566', 'epoch': '0.03346'}
{'loss': '0.8375', 'grad_norm': '0.3374', 'learning_rate': '0.000164', 'entropy': '0.7983', 'num_tokens': '2.048e+07', 'mean_token_accuracy': '0.8042', 'epoch': '0.03347'}
{'loss': '1.162', 'grad_norm': '0.4513', 'learning_rate': '0.000164', 'entropy': '1.258', 'num_tokens': '2.049e+07', 'mean_token_accuracy': '0.7027', 'epoch': '0.03348'}
{'loss': '0.7891', 'grad_norm': '0.4466', 'learning_rate': '0.0001639', 'entropy': '0.796', 'num_tokens': '2.05e+07', 'mean_token_accuracy': '0.7974', 'epoch': '0.03349'}
{'loss': '0.6956', 'grad_norm': '0.3264', 'learning_rate': '0.0001639', 'entropy': '0.7946', 'num_tokens': '2.05e+07', 'mean_token_accuracy': '0.8088', 'epoch': '0.0335'}
{'loss': '0.8579', 'grad_norm': '0.3139', 'learning_rate': '0.0001639', 'entropy': '0.7978', 'num_tokens': '2.05e+07', 'mean_token_accuracy': '0.8038', 'epoch': '0.03351'}
{'loss': '0.8048', 'grad_norm': '0.3614', 'learning_rate': '0.0001639', 'entropy': '0.8146', 'num_tokens': '2.051e+07', 'mean_token_accuracy': '0.7973', 'epoch': '0.03351'}
{'loss': '0.8512', 'grad_norm': '0.3343', 'learning_rate': '0.0001638', 'entropy': '0.8359', 'num_tokens': '2.051e+07', 'mean_token_accuracy': '0.783', 'epoch': '0.03352'}
{'loss': '1.61', 'grad_norm': '0.4412', 'learning_rate': '0.0001638', 'entropy': '1.097', 'num_tokens': '2.052e+07', 'mean_token_accuracy': '0.7614', 'epoch': '0.03353'}
{'loss': '0.9616', 'grad_norm': '0.4196', 'learning_rate': '0.0001638', 'entropy': '0.9155', 'num_tokens': '2.053e+07', 'mean_token_accuracy': '0.7795', 'epoch': '0.03354'}
{'loss': '1.122', 'grad_norm': '0.4698', 'learning_rate': '0.0001638', 'entropy': '1.078', 'num_tokens': '2.053e+07', 'mean_token_accuracy': '0.7256', 'epoch': '0.03355'}
{'loss': '0.8126', 'grad_norm': '0.3917', 'learning_rate': '0.0001638', 'entropy': '0.8577', 'num_tokens': '2.054e+07', 'mean_token_accuracy': '0.7934', 'epoch': '0.03356'}
{'loss': '0.5783', 'grad_norm': '0.2971', 'learning_rate': '0.0001637', 'entropy': '0.6701', 'num_tokens': '2.054e+07', 'mean_token_accuracy': '0.836', 'epoch': '0.03357'}
{'loss': '1.426', 'grad_norm': '0.527', 'learning_rate': '0.0001637', 'entropy': '1.261', 'num_tokens': '2.055e+07', 'mean_token_accuracy': '0.7134', 'epoch': '0.03358'}
{'loss': '1.166', 'grad_norm': '0.3418', 'learning_rate': '0.0001637', 'entropy': '0.9555', 'num_tokens': '2.056e+07', 'mean_token_accuracy': '0.7754', 'epoch': '0.03359'}
{'loss': '0.9593', 'grad_norm': '0.3668', 'learning_rate': '0.0001637', 'entropy': '0.97', 'num_tokens': '2.056e+07', 'mean_token_accuracy': '0.7643', 'epoch': '0.03359'}
{'loss': '0.81', 'grad_norm': '0.3744', 'learning_rate': '0.0001637', 'entropy': '0.8041', 'num_tokens': '2.057e+07', 'mean_token_accuracy': '0.7975', 'epoch': '0.0336'}
{'loss': '0.9853', 'grad_norm': '0.4527', 'learning_rate': '0.0001636', 'entropy': '1.088', 'num_tokens': '2.057e+07', 'mean_token_accuracy': '0.7381', 'epoch': '0.03361'}
{'loss': '0.7969', 'grad_norm': '0.3903', 'learning_rate': '0.0001636', 'entropy': '0.8653', 'num_tokens': '2.057e+07', 'mean_token_accuracy': '0.7927', 'epoch': '0.03362'}
{'loss': '0.5689', 'grad_norm': '0.3658', 'learning_rate': '0.0001636', 'entropy': '0.5863', 'num_tokens': '2.058e+07', 'mean_token_accuracy': '0.8412', 'epoch': '0.03363'}
{'loss': '1.175', 'grad_norm': '0.4188', 'learning_rate': '0.0001636', 'entropy': '1.032', 'num_tokens': '2.059e+07', 'mean_token_accuracy': '0.7494', 'epoch': '0.03364'}
{'loss': '0.7968', 'grad_norm': '0.3619', 'learning_rate': '0.0001636', 'entropy': '0.8607', 'num_tokens': '2.059e+07', 'mean_token_accuracy': '0.7857', 'epoch': '0.03365'}
{'loss': '0.5723', 'grad_norm': '0.2896', 'learning_rate': '0.0001635', 'entropy': '0.6926', 'num_tokens': '2.059e+07', 'mean_token_accuracy': '0.8196', 'epoch': '0.03366'}
{'loss': '0.9567', 'grad_norm': '0.5165', 'learning_rate': '0.0001635', 'entropy': '0.8736', 'num_tokens': '2.06e+07', 'mean_token_accuracy': '0.7753', 'epoch': '0.03366'}
{'loss': '0.8925', 'grad_norm': '0.3542', 'learning_rate': '0.0001635', 'entropy': '0.972', 'num_tokens': '2.06e+07', 'mean_token_accuracy': '0.7741', 'epoch': '0.03367'}
{'loss': '0.8299', 'grad_norm': '0.3852', 'learning_rate': '0.0001635', 'entropy': '0.8513', 'num_tokens': '2.061e+07', 'mean_token_accuracy': '0.7891', 'epoch': '0.03368'}
{'loss': '0.789', 'grad_norm': '0.2783', 'learning_rate': '0.0001634', 'entropy': '0.7391', 'num_tokens': '2.062e+07', 'mean_token_accuracy': '0.8177', 'epoch': '0.03369'}
{'loss': '0.6633', 'grad_norm': '0.3551', 'learning_rate': '0.0001634', 'entropy': '0.6832', 'num_tokens': '2.062e+07', 'mean_token_accuracy': '0.8215', 'epoch': '0.0337'}
{'loss': '0.8116', 'grad_norm': '0.3541', 'learning_rate': '0.0001634', 'entropy': '0.9502', 'num_tokens': '2.063e+07', 'mean_token_accuracy': '0.7619', 'epoch': '0.03371'}
{'loss': '0.6636', 'grad_norm': '0.3482', 'learning_rate': '0.0001634', 'entropy': '0.7036', 'num_tokens': '2.063e+07', 'mean_token_accuracy': '0.8259', 'epoch': '0.03372'}

 32%|███▏      | 3811/12000 [2:16:33<3:50:46,  1.69s/it]
 32%|███▏      | 3812/12000 [2:16:36<4:38:50,  2.04s/it]
                                                        

 32%|███▏      | 3812/12000 [2:16:36<4:38:50,  2.04s/it]
 32%|███▏      | 3813/12000 [2:16:38<4:18:44,  1.90s/it]
                                                        

 32%|███▏      | 3813/12000 [2:16:38<4:18:44,  1.90s/it]
 32%|███▏      | 3814/12000 [2:16:39<4:07:56,  1.82s/it]
                                                        

 32%|███▏      | 3814/12000 [2:16:39<4:07:56,  1.82s/it]
 32%|███▏      | 3815/12000 [2:16:41<3:53:26,  1.71s/it]
                                                        

 32%|███▏      | 3815/12000 [2:16:41<3:53:26,  1.71s/it]
 32%|███▏      | 3816/12000 [2:16:42<3:39:58,  1.61s/it]
                                                        

 32%|███▏      | 3816/12000 [2:16:42<3:39:58,  1.61s/it]
 32%|███▏      | 3817/12000 [2:16:45<4:05:15,  1.80s/it]
                                                        

 32%|███▏      | 3817/12000 [2:16:45<4:05:15,  1.80s/it]
 32%|███▏      | 3818/12000 [2:16:46<4:02:19,  1.78s/it]
                                                        

 32%|███▏      | 3818/12000 [2:16:46<4:02:19,  1.78s/it]
 32%|███▏      | 3819/12000 [2:16:48<4:05:03,  1.80s/it]
                                                        

 32%|███▏      | 3819/12000 [2:16:48<4:05:03,  1.80s/it]
 32%|███▏      | 3820/12000 [2:16:51<4:32:28,  2.00s/it]
                                                        

 32%|███▏      | 3820/12000 [2:16:51<4:32:28,  2.00s/it]
 32%|███▏      | 3821/12000 [2:16:53<4:40:20,  2.06s/it]
                                                        

 32%|███▏      | 3821/12000 [2:16:53<4:40:20,  2.06s/it]
 32%|███▏      | 3822/12000 [2:16:56<5:25:09,  2.39s/it]
                                                        

 32%|███▏      | 3822/12000 [2:16:56<5:25:09,  2.39s/it]
 32%|███▏      | 3823/12000 [2:17:01<7:01:55,  3.10s/it]
                                                        

 32%|███▏      | 3823/12000 [2:17:01<7:01:55,  3.10s/it]
 32%|███▏      | 3824/12000 [2:17:02<5:51:14,  2.58s/it]
                                                        

 32%|███▏      | 3824/12000 [2:17:02<5:51:14,  2.58s/it]
 32%|███▏      | 3825/12000 [2:17:03<5:03:07,  2.22s/it]
                                                        

 32%|███▏      | 3825/12000 [2:17:03<5:03:07,  2.22s/it]
 32%|███▏      | 3826/12000 [2:17:07<5:59:43,  2.64s/it]
                                                        

 32%|███▏      | 3826/12000 [2:17:07<5:59:43,  2.64s/it]
 32%|███▏      | 3827/12000 [2:17:10<6:22:59,  2.81s/it]
                                                        

 32%|███▏      | 3827/12000 [2:17:10<6:22:59,  2.81s/it]
 32%|███▏      | 3828/12000 [2:17:12<5:27:23,  2.40s/it]
                                                        

 32%|███▏      | 3828/12000 [2:17:12<5:27:23,  2.40s/it]
 32%|███▏      | 3829/12000 [2:17:13<4:54:51,  2.17s/it]
                                                        

 32%|███▏      | 3829/12000 [2:17:13<4:54:51,  2.17s/it]
 32%|███▏      | 3830/12000 [2:17:16<5:03:15,  2.23s/it]
                                                        

 32%|███▏      | 3830/12000 [2:17:16<5:03:15,  2.23s/it]
 32%|███▏      | 3831/12000 [2:17:17<4:32:28,  2.00s/it]
                                                        

 32%|███▏      | 3831/12000 [2:17:17<4:32:28,  2.00s/it]
 32%|███▏      | 3832/12000 [2:17:19<4:06:09,  1.81s/it]
                                                        

 32%|███▏      | 3832/12000 [2:17:19<4:06:09,  1.81s/it]
 32%|███▏      | 3833/12000 [2:17:21<4:15:49,  1.88s/it]
                                                        

 32%|███▏      | 3833/12000 [2:17:21<4:15:49,  1.88s/it]
 32%|███▏      | 3834/12000 [2:17:22<4:03:30,  1.79s/it]
                                                        

 32%|███▏      | 3834/12000 [2:17:22<4:03:30,  1.79s/it]
 32%|███▏      | 3835/12000 [2:17:24<4:23:18,  1.93s/it]
                                                        

 32%|███▏      | 3835/12000 [2:17:24<4:23:18,  1.93s/it]
 32%|███▏      | 3836/12000 [2:17:26<4:23:16,  1.93s/it]
                                                        

 32%|███▏      | 3836/12000 [2:17:26<4:23:16,  1.93s/it]
 32%|███▏      | 3837/12000 [2:17:29<5:01:20,  2.21s/it]
                                                        

 32%|███▏      | 3837/12000 [2:17:29<5:01:20,  2.21s/it]
 32%|███▏      | 3838/12000 [2:17:32<5:06:26,  2.25s/it]
                                                        

 32%|███▏      | 3838/12000 [2:17:32<5:06:26,  2.25s/it]
 32%|███▏      | 3839/12000 [2:17:34<5:13:33,  2.31s/it]
                                                        

 32%|███▏      | 3839/12000 [2:17:34<5:13:33,  2.31s/it]
 32%|███▏      | 3840/12000 [2:17:35<4:40:17,  2.06s/it]
                                                        

 32%|███▏      | 3840/12000 [2:17:35<4:40:17,  2.06s/it]
 32%|███▏      | 3841/12000 [2:17:37<4:23:29,  1.94s/it]
                                                        

 32%|███▏      | 3841/12000 [2:17:37<4:23:29,  1.94s/it]
 32%|███▏      | 3842/12000 [2:17:40<4:41:10,  2.07s/it]
                                                        

 32%|███▏      | 3842/12000 [2:17:40<4:41:10,  2.07s/it]
 32%|███▏      | 3843/12000 [2:17:43<5:40:25,  2.50s/it]
                                                        

 32%|███▏      | 3843/12000 [2:17:43<5:40:25,  2.50s/it]
 32%|███▏      | 3844/12000 [2:17:46<6:10:08,  2.72s/it]
                                                        

 32%|███▏      | 3844/12000 [2:17:46<6:10:08,  2.72s/it]
 32%|███▏      | 3845/12000 [2:17:48<5:18:08,  2.34s/it]
                                                        

 32%|███▏      | 3845/12000 [2:17:48<5:18:08,  2.34s/it]
 32%|███▏      | 3846/12000 [2:17:49<4:42:51,  2.08s/it]
                                                        

 32%|███▏      | 3846/12000 [2:17:49<4:42:51,  2.08s/it]
 32%|███▏      | 3847/12000 [2:17:51<4:25:41,  1.96s/it]
                                                        

 32%|███▏      | 3847/12000 [2:17:51<4:25:41,  1.96s/it]
 32%|███▏      | 3848/12000 [2:17:52<4:05:17,  1.81s/it]
                                                        

 32%|███▏      | 3848/12000 [2:17:52<4:05:17,  1.81s/it]
 32%|███▏      | 3849/12000 [2:17:54<4:01:45,  1.78s/it]
                                                        

 32%|███▏      | 3849/12000 [2:17:54<4:01:45,  1.78s/it]
 32%|███▏      | 3850/12000 [2:17:55<3:44:03,  1.65s/it]
                                                        

 32%|███▏      | 3850/12000 [2:17:55<3:44:03,  1.65s/it]
 32%|███▏      | 3851/12000 [2:17:58<4:17:36,  1.90s/it]
                                                        

 32%|███▏      | 3851/12000 [2:17:58<4:17:36,  1.90s/it]
 32%|███▏      | 3852/12000 [2:17:59<4:04:47,  1.80s/it]
                                                        

 32%|███▏      | 3852/12000 [2:17:59<4:04:47,  1.80s/it]
 32%|███▏      | 3853/12000 [2:18:01<3:42:41,  1.64s/it]
                                                        

 32%|███▏      | 3853/12000 [2:18:01<3:42:41,  1.64s/it]
 32%|███▏      | 3854/12000 [2:18:02<3:36:44,  1.60s/it]
                                                        

 32%|███▏      | 3854/12000 [2:18:02<3:36:44,  1.60s/it]
 32%|███▏      | 3855/12000 [2:18:04<3:31:16,  1.56s/it]
                                                        

 32%|███▏      | 3855/12000 [2:18:04<3:31:16,  1.56s/it]
 32%|███▏      | 3856/12000 [2:18:05<3:25:14,  1.51s/it]
                                                        

 32%|███▏      | 3856/12000 [2:18:05<3:25:14,  1.51s/it]
 32%|███▏      | 3857/12000 [2:18:07<3:32:35,  1.57s/it]
                                                        

 32%|███▏      | 3857/12000 [2:18:07<3:32:35,  1.57s/it]
 32%|███▏      | 3858/12000 [2:18:09<3:47:41,  1.68s/it]
                                                        
{'loss': '0.7697', 'grad_norm': '0.4146', 'learning_rate': '0.0001634', 'entropy': '0.7894', 'num_tokens': '2.064e+07', 'mean_token_accuracy': '0.7946', 'epoch': '0.03373'}
{'loss': '1.194', 'grad_norm': '0.3903', 'learning_rate': '0.0001633', 'entropy': '1.392', 'num_tokens': '2.064e+07', 'mean_token_accuracy': '0.672', 'epoch': '0.03374'}
{'loss': '0.9774', 'grad_norm': '0.4306', 'learning_rate': '0.0001633', 'entropy': '0.9128', 'num_tokens': '2.065e+07', 'mean_token_accuracy': '0.7639', 'epoch': '0.03374'}
{'loss': '0.9788', 'grad_norm': '0.4145', 'learning_rate': '0.0001633', 'entropy': '0.986', 'num_tokens': '2.065e+07', 'mean_token_accuracy': '0.7645', 'epoch': '0.03375'}
{'loss': '0.5652', 'grad_norm': '0.306', 'learning_rate': '0.0001633', 'entropy': '0.575', 'num_tokens': '2.066e+07', 'mean_token_accuracy': '0.8485', 'epoch': '0.03376'}
{'loss': '0.5661', 'grad_norm': '0.3025', 'learning_rate': '0.0001633', 'entropy': '0.5745', 'num_tokens': '2.066e+07', 'mean_token_accuracy': '0.8547', 'epoch': '0.03377'}
{'loss': '0.9031', 'grad_norm': '0.3583', 'learning_rate': '0.0001632', 'entropy': '0.8717', 'num_tokens': '2.067e+07', 'mean_token_accuracy': '0.7748', 'epoch': '0.03378'}
{'loss': '0.9258', 'grad_norm': '0.4357', 'learning_rate': '0.0001632', 'entropy': '0.9377', 'num_tokens': '2.067e+07', 'mean_token_accuracy': '0.7674', 'epoch': '0.03379'}
{'loss': '0.9188', 'grad_norm': '0.4456', 'learning_rate': '0.0001632', 'entropy': '0.9077', 'num_tokens': '2.068e+07', 'mean_token_accuracy': '0.7948', 'epoch': '0.0338'}
{'loss': '0.7187', 'grad_norm': '0.3032', 'learning_rate': '0.0001632', 'entropy': '0.7786', 'num_tokens': '2.068e+07', 'mean_token_accuracy': '0.8006', 'epoch': '0.03381'}
{'loss': '0.8371', 'grad_norm': '0.3637', 'learning_rate': '0.0001631', 'entropy': '0.8339', 'num_tokens': '2.069e+07', 'mean_token_accuracy': '0.7988', 'epoch': '0.03382'}
{'loss': '1.033', 'grad_norm': '0.3851', 'learning_rate': '0.0001631', 'entropy': '1.103', 'num_tokens': '2.069e+07', 'mean_token_accuracy': '0.7281', 'epoch': '0.03382'}
{'loss': '0.5403', 'grad_norm': '0.2902', 'learning_rate': '0.0001631', 'entropy': '0.7421', 'num_tokens': '2.07e+07', 'mean_token_accuracy': '0.8116', 'epoch': '0.03383'}
{'loss': '0.7356', 'grad_norm': '0.3614', 'learning_rate': '0.0001631', 'entropy': '0.7385', 'num_tokens': '2.071e+07', 'mean_token_accuracy': '0.8123', 'epoch': '0.03384'}
{'loss': '1.071', 'grad_norm': '0.3874', 'learning_rate': '0.0001631', 'entropy': '1.04', 'num_tokens': '2.071e+07', 'mean_token_accuracy': '0.7465', 'epoch': '0.03385'}
{'loss': '0.5925', 'grad_norm': '0.2876', 'learning_rate': '0.000163', 'entropy': '0.6755', 'num_tokens': '2.072e+07', 'mean_token_accuracy': '0.8348', 'epoch': '0.03386'}
{'loss': '0.798', 'grad_norm': '0.3808', 'learning_rate': '0.000163', 'entropy': '0.8476', 'num_tokens': '2.073e+07', 'mean_token_accuracy': '0.788', 'epoch': '0.03387'}
{'loss': '0.8431', 'grad_norm': '0.3622', 'learning_rate': '0.000163', 'entropy': '0.8874', 'num_tokens': '2.073e+07', 'mean_token_accuracy': '0.777', 'epoch': '0.03388'}
{'loss': '0.6832', 'grad_norm': '0.3879', 'learning_rate': '0.000163', 'entropy': '0.6689', 'num_tokens': '2.074e+07', 'mean_token_accuracy': '0.8181', 'epoch': '0.03389'}
{'loss': '0.4163', 'grad_norm': '0.2487', 'learning_rate': '0.000163', 'entropy': '0.4476', 'num_tokens': '2.074e+07', 'mean_token_accuracy': '0.888', 'epoch': '0.03389'}
{'loss': '1.371', 'grad_norm': '0.4949', 'learning_rate': '0.0001629', 'entropy': '1.271', 'num_tokens': '2.075e+07', 'mean_token_accuracy': '0.6972', 'epoch': '0.0339'}
{'loss': '0.827', 'grad_norm': '0.4238', 'learning_rate': '0.0001629', 'entropy': '0.8142', 'num_tokens': '2.075e+07', 'mean_token_accuracy': '0.7964', 'epoch': '0.03391'}
{'loss': '0.8009', 'grad_norm': '0.5262', 'learning_rate': '0.0001629', 'entropy': '0.7861', 'num_tokens': '2.076e+07', 'mean_token_accuracy': '0.8046', 'epoch': '0.03392'}
{'loss': '0.7919', 'grad_norm': '0.3931', 'learning_rate': '0.0001629', 'entropy': '0.8661', 'num_tokens': '2.076e+07', 'mean_token_accuracy': '0.7729', 'epoch': '0.03393'}
{'loss': '0.8338', 'grad_norm': '0.3802', 'learning_rate': '0.0001628', 'entropy': '0.8782', 'num_tokens': '2.077e+07', 'mean_token_accuracy': '0.7837', 'epoch': '0.03394'}
{'loss': '0.927', 'grad_norm': '0.3852', 'learning_rate': '0.0001628', 'entropy': '0.8985', 'num_tokens': '2.077e+07', 'mean_token_accuracy': '0.7798', 'epoch': '0.03395'}
{'loss': '0.9888', 'grad_norm': '0.3153', 'learning_rate': '0.0001628', 'entropy': '0.9802', 'num_tokens': '2.078e+07', 'mean_token_accuracy': '0.7593', 'epoch': '0.03396'}
{'loss': '0.9579', 'grad_norm': '0.3642', 'learning_rate': '0.0001628', 'entropy': '0.9351', 'num_tokens': '2.079e+07', 'mean_token_accuracy': '0.7707', 'epoch': '0.03397'}
{'loss': '0.7123', 'grad_norm': '0.3461', 'learning_rate': '0.0001628', 'entropy': '0.8179', 'num_tokens': '2.079e+07', 'mean_token_accuracy': '0.7995', 'epoch': '0.03397'}
{'loss': '0.6205', 'grad_norm': '0.3379', 'learning_rate': '0.0001627', 'entropy': '0.7798', 'num_tokens': '2.08e+07', 'mean_token_accuracy': '0.8154', 'epoch': '0.03398'}
{'loss': '0.9716', 'grad_norm': '0.4256', 'learning_rate': '0.0001627', 'entropy': '1.05', 'num_tokens': '2.08e+07', 'mean_token_accuracy': '0.7484', 'epoch': '0.03399'}
{'loss': '1.243', 'grad_norm': '0.342', 'learning_rate': '0.0001627', 'entropy': '1.265', 'num_tokens': '2.081e+07', 'mean_token_accuracy': '0.6858', 'epoch': '0.034'}
{'loss': '0.7434', 'grad_norm': '0.3285', 'learning_rate': '0.0001627', 'entropy': '0.76', 'num_tokens': '2.082e+07', 'mean_token_accuracy': '0.8107', 'epoch': '0.03401'}
{'loss': '0.7294', 'grad_norm': '0.4565', 'learning_rate': '0.0001627', 'entropy': '0.7386', 'num_tokens': '2.082e+07', 'mean_token_accuracy': '0.8076', 'epoch': '0.03402'}
{'loss': '0.902', 'grad_norm': '0.4109', 'learning_rate': '0.0001626', 'entropy': '0.9724', 'num_tokens': '2.082e+07', 'mean_token_accuracy': '0.7563', 'epoch': '0.03403'}
{'loss': '1', 'grad_norm': '0.3485', 'learning_rate': '0.0001626', 'entropy': '1.078', 'num_tokens': '2.083e+07', 'mean_token_accuracy': '0.7426', 'epoch': '0.03404'}
{'loss': '1.092', 'grad_norm': '0.4119', 'learning_rate': '0.0001626', 'entropy': '1.121', 'num_tokens': '2.083e+07', 'mean_token_accuracy': '0.7217', 'epoch': '0.03405'}
{'loss': '0.9775', 'grad_norm': '0.4158', 'learning_rate': '0.0001626', 'entropy': '0.8679', 'num_tokens': '2.084e+07', 'mean_token_accuracy': '0.7856', 'epoch': '0.03405'}
{'loss': '0.7323', 'grad_norm': '0.434', 'learning_rate': '0.0001625', 'entropy': '0.7338', 'num_tokens': '2.084e+07', 'mean_token_accuracy': '0.8082', 'epoch': '0.03406'}
{'loss': '0.753', 'grad_norm': '0.341', 'learning_rate': '0.0001625', 'entropy': '0.8344', 'num_tokens': '2.085e+07', 'mean_token_accuracy': '0.7977', 'epoch': '0.03407'}
{'loss': '1.122', 'grad_norm': '0.5113', 'learning_rate': '0.0001625', 'entropy': '1.057', 'num_tokens': '2.085e+07', 'mean_token_accuracy': '0.7369', 'epoch': '0.03408'}
{'loss': '0.6665', 'grad_norm': '0.3459', 'learning_rate': '0.0001625', 'entropy': '0.6858', 'num_tokens': '2.086e+07', 'mean_token_accuracy': '0.8244', 'epoch': '0.03409'}
{'loss': '0.5116', 'grad_norm': '0.3493', 'learning_rate': '0.0001625', 'entropy': '0.5606', 'num_tokens': '2.086e+07', 'mean_token_accuracy': '0.8541', 'epoch': '0.0341'}
{'loss': '0.839', 'grad_norm': '0.6495', 'learning_rate': '0.0001624', 'entropy': '0.8949', 'num_tokens': '2.087e+07', 'mean_token_accuracy': '0.7678', 'epoch': '0.03411'}
{'loss': '0.9963', 'grad_norm': '0.4634', 'learning_rate': '0.0001624', 'entropy': '0.9869', 'num_tokens': '2.087e+07', 'mean_token_accuracy': '0.7615', 'epoch': '0.03412'}
{'loss': '0.8315', 'grad_norm': '0.3472', 'learning_rate': '0.0001624', 'entropy': '0.839', 'num_tokens': '2.087e+07', 'mean_token_accuracy': '0.7977', 'epoch': '0.03412'}
{'loss': '1.664', 'grad_norm': '0.5468', 'learning_rate': '0.0001624', 'entropy': '1.517', 'num_tokens': '2.088e+07', 'mean_token_accuracy': '0.6504', 'epoch': '0.03413'}

 32%|███▏      | 3858/12000 [2:18:09<3:47:41,  1.68s/it]
 32%|███▏      | 3859/12000 [2:18:11<4:07:18,  1.82s/it]
                                                        

 32%|███▏      | 3859/12000 [2:18:11<4:07:18,  1.82s/it]
 32%|███▏      | 3860/12000 [2:18:12<3:51:27,  1.71s/it]
                                                        

 32%|███▏      | 3860/12000 [2:18:12<3:51:27,  1.71s/it]
 32%|███▏      | 3861/12000 [2:18:14<3:41:04,  1.63s/it]
                                                        

 32%|███▏      | 3861/12000 [2:18:14<3:41:04,  1.63s/it]
 32%|███▏      | 3862/12000 [2:18:18<5:35:16,  2.47s/it]
                                                        

 32%|███▏      | 3862/12000 [2:18:18<5:35:16,  2.47s/it]
 32%|███▏      | 3863/12000 [2:18:20<5:11:13,  2.29s/it]
                                                        

 32%|███▏      | 3863/12000 [2:18:20<5:11:13,  2.29s/it]
 32%|███▏      | 3864/12000 [2:18:23<5:41:30,  2.52s/it]
                                                        

 32%|███▏      | 3864/12000 [2:18:23<5:41:30,  2.52s/it]
 32%|███▏      | 3865/12000 [2:18:25<5:01:38,  2.22s/it]
                                                        

 32%|███▏      | 3865/12000 [2:18:25<5:01:38,  2.22s/it]
 32%|███▏      | 3866/12000 [2:18:29<6:10:17,  2.73s/it]
                                                        

 32%|███▏      | 3866/12000 [2:18:29<6:10:17,  2.73s/it]
 32%|███▏      | 3867/12000 [2:18:31<6:13:02,  2.75s/it]
                                                        

 32%|███▏      | 3867/12000 [2:18:31<6:13:02,  2.75s/it]
 32%|███▏      | 3868/12000 [2:18:33<5:19:11,  2.36s/it]
                                                        

 32%|███▏      | 3868/12000 [2:18:33<5:19:11,  2.36s/it]
 32%|███▏      | 3869/12000 [2:18:34<4:43:27,  2.09s/it]
                                                        

 32%|███▏      | 3869/12000 [2:18:34<4:43:27,  2.09s/it]
 32%|███▏      | 3870/12000 [2:18:36<4:31:42,  2.01s/it]
                                                        

 32%|███▏      | 3870/12000 [2:18:36<4:31:42,  2.01s/it]
 32%|███▏      | 3871/12000 [2:18:38<4:12:19,  1.86s/it]
                                                        

 32%|███▏      | 3871/12000 [2:18:38<4:12:19,  1.86s/it]
 32%|███▏      | 3872/12000 [2:18:39<4:05:04,  1.81s/it]
                                                        

 32%|███▏      | 3872/12000 [2:18:39<4:05:04,  1.81s/it]
 32%|███▏      | 3873/12000 [2:18:41<3:42:43,  1.64s/it]
                                                        

 32%|███▏      | 3873/12000 [2:18:41<3:42:43,  1.64s/it]
 32%|███▏      | 3874/12000 [2:18:43<4:22:48,  1.94s/it]
                                                        

 32%|███▏      | 3874/12000 [2:18:43<4:22:48,  1.94s/it]
 32%|███▏      | 3875/12000 [2:18:45<4:23:16,  1.94s/it]
                                                        

 32%|███▏      | 3875/12000 [2:18:45<4:23:16,  1.94s/it]
 32%|███▏      | 3876/12000 [2:18:47<4:25:04,  1.96s/it]
                                                        

 32%|███▏      | 3876/12000 [2:18:47<4:25:04,  1.96s/it]
 32%|███▏      | 3877/12000 [2:18:49<4:14:31,  1.88s/it]
                                                        

 32%|███▏      | 3877/12000 [2:18:49<4:14:31,  1.88s/it]
 32%|███▏      | 3878/12000 [2:18:50<4:04:25,  1.81s/it]
                                                        

 32%|███▏      | 3878/12000 [2:18:50<4:04:25,  1.81s/it]
 32%|███▏      | 3879/12000 [2:18:52<3:51:26,  1.71s/it]
                                                        

 32%|███▏      | 3879/12000 [2:18:52<3:51:26,  1.71s/it]
 32%|███▏      | 3880/12000 [2:18:53<3:43:38,  1.65s/it]
                                                        

 32%|███▏      | 3880/12000 [2:18:53<3:43:38,  1.65s/it]
 32%|███▏      | 3881/12000 [2:18:55<3:42:23,  1.64s/it]
                                                        

 32%|███▏      | 3881/12000 [2:18:55<3:42:23,  1.64s/it]
 32%|███▏      | 3882/12000 [2:18:57<3:36:30,  1.60s/it]
                                                        

 32%|███▏      | 3882/12000 [2:18:57<3:36:30,  1.60s/it]
 32%|███▏      | 3883/12000 [2:18:58<3:42:03,  1.64s/it]
                                                        

 32%|███▏      | 3883/12000 [2:18:58<3:42:03,  1.64s/it]
 32%|███▏      | 3884/12000 [2:19:00<3:23:59,  1.51s/it]
                                                        

 32%|███▏      | 3884/12000 [2:19:00<3:23:59,  1.51s/it]
 32%|███▏      | 3885/12000 [2:19:01<3:25:44,  1.52s/it]
                                                        

 32%|███▏      | 3885/12000 [2:19:01<3:25:44,  1.52s/it]
 32%|███▏      | 3886/12000 [2:19:04<4:35:06,  2.03s/it]
                                                        

 32%|███▏      | 3886/12000 [2:19:04<4:35:06,  2.03s/it]
 32%|███▏      | 3887/12000 [2:19:07<4:59:04,  2.21s/it]
                                                        

 32%|███▏      | 3887/12000 [2:19:07<4:59:04,  2.21s/it]
 32%|███▏      | 3888/12000 [2:19:10<5:37:24,  2.50s/it]
                                                        

 32%|███▏      | 3888/12000 [2:19:10<5:37:24,  2.50s/it]
 32%|███▏      | 3889/12000 [2:19:11<4:54:01,  2.17s/it]
                                                        

 32%|███▏      | 3889/12000 [2:19:11<4:54:01,  2.17s/it]
 32%|███▏      | 3890/12000 [2:19:13<4:33:32,  2.02s/it]
                                                        

 32%|███▏      | 3890/12000 [2:19:13<4:33:32,  2.02s/it]
 32%|███▏      | 3891/12000 [2:19:15<4:18:08,  1.91s/it]
                                                        

 32%|███▏      | 3891/12000 [2:19:15<4:18:08,  1.91s/it]
 32%|███▏      | 3892/12000 [2:19:16<4:04:27,  1.81s/it]
                                                        

 32%|███▏      | 3892/12000 [2:19:16<4:04:27,  1.81s/it]
 32%|███▏      | 3893/12000 [2:19:18<3:43:33,  1.65s/it]
                                                        

 32%|███▏      | 3893/12000 [2:19:18<3:43:33,  1.65s/it]
 32%|███▏      | 3894/12000 [2:19:22<5:30:52,  2.45s/it]
                                                        

 32%|███▏      | 3894/12000 [2:19:22<5:30:52,  2.45s/it]
 32%|███▏      | 3895/12000 [2:19:24<5:24:53,  2.41s/it]
                                                        

 32%|███▏      | 3895/12000 [2:19:24<5:24:53,  2.41s/it]
 32%|███▏      | 3896/12000 [2:19:26<4:52:28,  2.17s/it]
                                                        

 32%|███▏      | 3896/12000 [2:19:26<4:52:28,  2.17s/it]
 32%|███▏      | 3897/12000 [2:19:27<4:15:32,  1.89s/it]
                                                        

 32%|███▏      | 3897/12000 [2:19:27<4:15:32,  1.89s/it]
 32%|███▏      | 3898/12000 [2:19:29<3:54:27,  1.74s/it]
                                                        

 32%|███▏      | 3898/12000 [2:19:29<3:54:27,  1.74s/it]
 32%|███▏      | 3899/12000 [2:19:30<3:57:10,  1.76s/it]
                                                        

 32%|███▏      | 3899/12000 [2:19:30<3:57:10,  1.76s/it]
 32%|███▎      | 3900/12000 [2:19:34<5:03:27,  2.25s/it]
                                                        

 32%|███▎      | 3900/12000 [2:19:34<5:03:27,  2.25s/it]
 33%|███▎      | 3901/12000 [2:19:37<5:33:51,  2.47s/it]
                                                        

 33%|███▎      | 3901/12000 [2:19:37<5:33:51,  2.47s/it]
 33%|███▎      | 3902/12000 [2:19:38<4:52:23,  2.17s/it]
                                                        

 33%|███▎      | 3902/12000 [2:19:38<4:52:23,  2.17s/it]
 33%|███▎      | 3903/12000 [2:19:42<5:47:06,  2.57s/it]
                                                        

 33%|███▎      | 3903/12000 [2:19:42<5:47:06,  2.57s/it]
 33%|███▎      | 3904/12000 [2:19:44<5:45:15,  2.56s/it]
                                                        

 33%|███▎      | 3904/12000 [2:19:44<5:45:15,  2.56s/it]
 33%|███▎      | 3905/12000 [2:19:46<5:19:43,  2.37s/it]
                                                        
{'loss': '1.072', 'grad_norm': '0.5016', 'learning_rate': '0.0001624', 'entropy': '1.001', 'num_tokens': '2.089e+07', 'mean_token_accuracy': '0.7587', 'epoch': '0.03414'}
{'loss': '1.27', 'grad_norm': '0.458', 'learning_rate': '0.0001623', 'entropy': '1.146', 'num_tokens': '2.089e+07', 'mean_token_accuracy': '0.7267', 'epoch': '0.03415'}
{'loss': '0.6361', 'grad_norm': '0.3096', 'learning_rate': '0.0001623', 'entropy': '0.7478', 'num_tokens': '2.09e+07', 'mean_token_accuracy': '0.8138', 'epoch': '0.03416'}
{'loss': '0.8815', 'grad_norm': '0.4239', 'learning_rate': '0.0001623', 'entropy': '0.9179', 'num_tokens': '2.09e+07', 'mean_token_accuracy': '0.776', 'epoch': '0.03417'}
{'loss': '1.35', 'grad_norm': '0.4351', 'learning_rate': '0.0001623', 'entropy': '0.9701', 'num_tokens': '2.091e+07', 'mean_token_accuracy': '0.7661', 'epoch': '0.03418'}
{'loss': '1.302', 'grad_norm': '0.3775', 'learning_rate': '0.0001622', 'entropy': '1.323', 'num_tokens': '2.091e+07', 'mean_token_accuracy': '0.6893', 'epoch': '0.03419'}
{'loss': '0.8635', 'grad_norm': '0.3689', 'learning_rate': '0.0001622', 'entropy': '0.9251', 'num_tokens': '2.092e+07', 'mean_token_accuracy': '0.7771', 'epoch': '0.0342'}
{'loss': '0.8712', 'grad_norm': '0.4135', 'learning_rate': '0.0001622', 'entropy': '0.8765', 'num_tokens': '2.093e+07', 'mean_token_accuracy': '0.7774', 'epoch': '0.0342'}
{'loss': '1.142', 'grad_norm': '0.3383', 'learning_rate': '0.0001622', 'entropy': '1.202', 'num_tokens': '2.093e+07', 'mean_token_accuracy': '0.7266', 'epoch': '0.03421'}
{'loss': '1.001', 'grad_norm': '0.3302', 'learning_rate': '0.0001622', 'entropy': '1.106', 'num_tokens': '2.094e+07', 'mean_token_accuracy': '0.7332', 'epoch': '0.03422'}
{'loss': '0.8621', 'grad_norm': '0.4843', 'learning_rate': '0.0001621', 'entropy': '0.8976', 'num_tokens': '2.094e+07', 'mean_token_accuracy': '0.7692', 'epoch': '0.03423'}
{'loss': '0.8518', 'grad_norm': '0.3197', 'learning_rate': '0.0001621', 'entropy': '0.8648', 'num_tokens': '2.095e+07', 'mean_token_accuracy': '0.7693', 'epoch': '0.03424'}
{'loss': '1.109', 'grad_norm': '0.4697', 'learning_rate': '0.0001621', 'entropy': '1.159', 'num_tokens': '2.095e+07', 'mean_token_accuracy': '0.7112', 'epoch': '0.03425'}
{'loss': '0.7462', 'grad_norm': '0.4154', 'learning_rate': '0.0001621', 'entropy': '0.737', 'num_tokens': '2.096e+07', 'mean_token_accuracy': '0.8026', 'epoch': '0.03426'}
{'loss': '1.036', 'grad_norm': '0.3681', 'learning_rate': '0.000162', 'entropy': '1.062', 'num_tokens': '2.096e+07', 'mean_token_accuracy': '0.7539', 'epoch': '0.03427'}
{'loss': '0.9335', 'grad_norm': '0.4319', 'learning_rate': '0.000162', 'entropy': '1.064', 'num_tokens': '2.097e+07', 'mean_token_accuracy': '0.7426', 'epoch': '0.03428'}
{'loss': '0.6461', 'grad_norm': '0.3753', 'learning_rate': '0.000162', 'entropy': '0.6758', 'num_tokens': '2.097e+07', 'mean_token_accuracy': '0.8265', 'epoch': '0.03428'}
{'loss': '1.274', 'grad_norm': '0.4006', 'learning_rate': '0.000162', 'entropy': '1.276', 'num_tokens': '2.098e+07', 'mean_token_accuracy': '0.6918', 'epoch': '0.03429'}
{'loss': '0.9705', 'grad_norm': '0.4136', 'learning_rate': '0.000162', 'entropy': '0.9807', 'num_tokens': '2.099e+07', 'mean_token_accuracy': '0.7726', 'epoch': '0.0343'}
{'loss': '1.133', 'grad_norm': '0.4184', 'learning_rate': '0.0001619', 'entropy': '1.04', 'num_tokens': '2.099e+07', 'mean_token_accuracy': '0.7621', 'epoch': '0.03431'}
{'loss': '0.6739', 'grad_norm': '0.4226', 'learning_rate': '0.0001619', 'entropy': '0.6793', 'num_tokens': '2.099e+07', 'mean_token_accuracy': '0.833', 'epoch': '0.03432'}
{'loss': '1.007', 'grad_norm': '0.4363', 'learning_rate': '0.0001619', 'entropy': '1.06', 'num_tokens': '2.1e+07', 'mean_token_accuracy': '0.7404', 'epoch': '0.03433'}
{'loss': '1.03', 'grad_norm': '0.4267', 'learning_rate': '0.0001619', 'entropy': '1.081', 'num_tokens': '2.1e+07', 'mean_token_accuracy': '0.7429', 'epoch': '0.03434'}
{'loss': '1.085', 'grad_norm': '0.4049', 'learning_rate': '0.0001619', 'entropy': '1.021', 'num_tokens': '2.101e+07', 'mean_token_accuracy': '0.7582', 'epoch': '0.03435'}
{'loss': '0.9227', 'grad_norm': '0.5083', 'learning_rate': '0.0001618', 'entropy': '1.023', 'num_tokens': '2.101e+07', 'mean_token_accuracy': '0.7546', 'epoch': '0.03436'}
{'loss': '0.9135', 'grad_norm': '0.4539', 'learning_rate': '0.0001618', 'entropy': '0.8865', 'num_tokens': '2.102e+07', 'mean_token_accuracy': '0.7789', 'epoch': '0.03436'}
{'loss': '0.8161', 'grad_norm': '0.4491', 'learning_rate': '0.0001618', 'entropy': '0.8409', 'num_tokens': '2.102e+07', 'mean_token_accuracy': '0.7867', 'epoch': '0.03437'}
{'loss': '0.6892', 'grad_norm': '0.3631', 'learning_rate': '0.0001618', 'entropy': '0.6564', 'num_tokens': '2.102e+07', 'mean_token_accuracy': '0.8259', 'epoch': '0.03438'}
{'loss': '0.6618', 'grad_norm': '0.2897', 'learning_rate': '0.0001617', 'entropy': '0.9757', 'num_tokens': '2.103e+07', 'mean_token_accuracy': '0.7653', 'epoch': '0.03439'}
{'loss': '0.7512', 'grad_norm': '0.3768', 'learning_rate': '0.0001617', 'entropy': '0.7526', 'num_tokens': '2.104e+07', 'mean_token_accuracy': '0.8072', 'epoch': '0.0344'}
{'loss': '0.8055', 'grad_norm': '0.3912', 'learning_rate': '0.0001617', 'entropy': '0.8556', 'num_tokens': '2.104e+07', 'mean_token_accuracy': '0.7836', 'epoch': '0.03441'}
{'loss': '0.5781', 'grad_norm': '0.3279', 'learning_rate': '0.0001617', 'entropy': '0.6171', 'num_tokens': '2.105e+07', 'mean_token_accuracy': '0.8477', 'epoch': '0.03442'}
{'loss': '0.9292', 'grad_norm': '0.4028', 'learning_rate': '0.0001617', 'entropy': '0.9593', 'num_tokens': '2.105e+07', 'mean_token_accuracy': '0.7658', 'epoch': '0.03443'}
{'loss': '0.9599', 'grad_norm': '0.4209', 'learning_rate': '0.0001616', 'entropy': '0.9835', 'num_tokens': '2.106e+07', 'mean_token_accuracy': '0.7685', 'epoch': '0.03443'}
{'loss': '0.7408', 'grad_norm': '0.3256', 'learning_rate': '0.0001616', 'entropy': '0.8674', 'num_tokens': '2.106e+07', 'mean_token_accuracy': '0.7882', 'epoch': '0.03444'}
{'loss': '0.7031', 'grad_norm': '0.3149', 'learning_rate': '0.0001616', 'entropy': '0.7725', 'num_tokens': '2.107e+07', 'mean_token_accuracy': '0.802', 'epoch': '0.03445'}
{'loss': '1.259', 'grad_norm': '0.3514', 'learning_rate': '0.0001616', 'entropy': '1.162', 'num_tokens': '2.108e+07', 'mean_token_accuracy': '0.732', 'epoch': '0.03446'}
{'loss': '1.035', 'grad_norm': '0.4046', 'learning_rate': '0.0001616', 'entropy': '0.9963', 'num_tokens': '2.108e+07', 'mean_token_accuracy': '0.7568', 'epoch': '0.03447'}
{'loss': '0.6337', 'grad_norm': '0.381', 'learning_rate': '0.0001615', 'entropy': '0.6359', 'num_tokens': '2.109e+07', 'mean_token_accuracy': '0.8337', 'epoch': '0.03448'}
{'loss': '0.9126', 'grad_norm': '0.4381', 'learning_rate': '0.0001615', 'entropy': '0.89', 'num_tokens': '2.109e+07', 'mean_token_accuracy': '0.7754', 'epoch': '0.03449'}
{'loss': '0.5408', 'grad_norm': '0.3098', 'learning_rate': '0.0001615', 'entropy': '0.5963', 'num_tokens': '2.11e+07', 'mean_token_accuracy': '0.8406', 'epoch': '0.0345'}
{'loss': '0.5413', 'grad_norm': '0.3339', 'learning_rate': '0.0001615', 'entropy': '0.5768', 'num_tokens': '2.11e+07', 'mean_token_accuracy': '0.8566', 'epoch': '0.03451'}
{'loss': '0.5623', 'grad_norm': '0.2671', 'learning_rate': '0.0001614', 'entropy': '0.6779', 'num_tokens': '2.111e+07', 'mean_token_accuracy': '0.8377', 'epoch': '0.03451'}
{'loss': '1.386', 'grad_norm': '0.4359', 'learning_rate': '0.0001614', 'entropy': '1.235', 'num_tokens': '2.112e+07', 'mean_token_accuracy': '0.7041', 'epoch': '0.03452'}
{'loss': '1.075', 'grad_norm': '0.4851', 'learning_rate': '0.0001614', 'entropy': '1.073', 'num_tokens': '2.112e+07', 'mean_token_accuracy': '0.7251', 'epoch': '0.03453'}
{'loss': '0.9577', 'grad_norm': '0.3784', 'learning_rate': '0.0001614', 'entropy': '0.9622', 'num_tokens': '2.113e+07', 'mean_token_accuracy': '0.7629', 'epoch': '0.03454'}
{'loss': '0.5962', 'grad_norm': '0.3427', 'learning_rate': '0.0001614', 'entropy': '0.5703', 'num_tokens': '2.113e+07', 'mean_token_accuracy': '0.8519', 'epoch': '0.03455'}

 33%|███▎      | 3905/12000 [2:19:46<5:19:43,  2.37s/it]
 33%|███▎      | 3906/12000 [2:19:47<4:38:02,  2.06s/it]
                                                        

 33%|███▎      | 3906/12000 [2:19:47<4:38:02,  2.06s/it]
 33%|███▎      | 3907/12000 [2:19:49<4:23:14,  1.95s/it]
                                                        

 33%|███▎      | 3907/12000 [2:19:49<4:23:14,  1.95s/it]
 33%|███▎      | 3908/12000 [2:19:52<4:41:45,  2.09s/it]
                                                        

 33%|███▎      | 3908/12000 [2:19:52<4:41:45,  2.09s/it]
 33%|███▎      | 3909/12000 [2:19:53<4:21:26,  1.94s/it]
                                                        

 33%|███▎      | 3909/12000 [2:19:53<4:21:26,  1.94s/it]
 33%|███▎      | 3910/12000 [2:19:59<7:17:55,  3.25s/it]
                                                        

 33%|███▎      | 3910/12000 [2:19:59<7:17:55,  3.25s/it]
 33%|███▎      | 3911/12000 [2:20:01<6:01:46,  2.68s/it]
                                                        

 33%|███▎      | 3911/12000 [2:20:01<6:01:46,  2.68s/it]
 33%|███▎      | 3912/12000 [2:20:03<5:23:09,  2.40s/it]
                                                        

 33%|███▎      | 3912/12000 [2:20:03<5:23:09,  2.40s/it]
 33%|███▎      | 3913/12000 [2:20:04<4:52:42,  2.17s/it]
                                                        

 33%|███▎      | 3913/12000 [2:20:04<4:52:42,  2.17s/it]
 33%|███▎      | 3914/12000 [2:20:06<4:26:38,  1.98s/it]
                                                        

 33%|███▎      | 3914/12000 [2:20:06<4:26:38,  1.98s/it]
 33%|███▎      | 3915/12000 [2:20:09<5:20:56,  2.38s/it]
                                                        

 33%|███▎      | 3915/12000 [2:20:09<5:20:56,  2.38s/it]
 33%|███▎      | 3916/12000 [2:20:11<4:58:11,  2.21s/it]
                                                        

 33%|███▎      | 3916/12000 [2:20:11<4:58:11,  2.21s/it]
 33%|███▎      | 3917/12000 [2:20:13<4:34:48,  2.04s/it]
                                                        

 33%|███▎      | 3917/12000 [2:20:13<4:34:48,  2.04s/it]
 33%|███▎      | 3918/12000 [2:20:15<4:44:40,  2.11s/it]
                                                        

 33%|███▎      | 3918/12000 [2:20:15<4:44:40,  2.11s/it]
 33%|███▎      | 3919/12000 [2:20:16<4:12:03,  1.87s/it]
                                                        

 33%|███▎      | 3919/12000 [2:20:16<4:12:03,  1.87s/it]
 33%|███▎      | 3920/12000 [2:20:17<3:49:33,  1.70s/it]
                                                        

 33%|███▎      | 3920/12000 [2:20:17<3:49:33,  1.70s/it]
 33%|███▎      | 3921/12000 [2:20:19<3:38:50,  1.63s/it]
                                                        

 33%|███▎      | 3921/12000 [2:20:19<3:38:50,  1.63s/it]
 33%|███▎      | 3922/12000 [2:20:22<4:56:14,  2.20s/it]
                                                        

 33%|███▎      | 3922/12000 [2:20:22<4:56:14,  2.20s/it]
 33%|███▎      | 3923/12000 [2:20:24<4:27:54,  1.99s/it]
                                                        

 33%|███▎      | 3923/12000 [2:20:24<4:27:54,  1.99s/it]
 33%|███▎      | 3924/12000 [2:20:25<3:57:03,  1.76s/it]
                                                        

 33%|███▎      | 3924/12000 [2:20:25<3:57:03,  1.76s/it]
 33%|███▎      | 3925/12000 [2:20:27<3:45:17,  1.67s/it]
                                                        

 33%|███▎      | 3925/12000 [2:20:27<3:45:17,  1.67s/it]
 33%|███▎      | 3926/12000 [2:20:28<3:35:43,  1.60s/it]
                                                        

 33%|███▎      | 3926/12000 [2:20:28<3:35:43,  1.60s/it]
 33%|███▎      | 3927/12000 [2:20:29<3:20:23,  1.49s/it]
                                                        

 33%|███▎      | 3927/12000 [2:20:29<3:20:23,  1.49s/it]
 33%|███▎      | 3928/12000 [2:20:31<3:12:03,  1.43s/it]
                                                        

 33%|███▎      | 3928/12000 [2:20:31<3:12:03,  1.43s/it]
 33%|███▎      | 3929/12000 [2:20:32<3:19:50,  1.49s/it]
                                                        

 33%|███▎      | 3929/12000 [2:20:32<3:19:50,  1.49s/it]
 33%|███▎      | 3930/12000 [2:20:34<3:28:12,  1.55s/it]
                                                        

 33%|███▎      | 3930/12000 [2:20:34<3:28:12,  1.55s/it]
 33%|███▎      | 3931/12000 [2:20:35<3:26:02,  1.53s/it]
                                                        

 33%|███▎      | 3931/12000 [2:20:35<3:26:02,  1.53s/it]
 33%|███▎      | 3932/12000 [2:20:39<4:33:48,  2.04s/it]
                                                        

 33%|███▎      | 3932/12000 [2:20:39<4:33:48,  2.04s/it]
 33%|███▎      | 3933/12000 [2:20:42<5:15:31,  2.35s/it]
                                                        

 33%|███▎      | 3933/12000 [2:20:42<5:15:31,  2.35s/it]
 33%|███▎      | 3934/12000 [2:20:45<5:53:09,  2.63s/it]
                                                        

 33%|███▎      | 3934/12000 [2:20:45<5:53:09,  2.63s/it]
 33%|███▎      | 3935/12000 [2:20:46<5:06:27,  2.28s/it]
                                                        

 33%|███▎      | 3935/12000 [2:20:46<5:06:27,  2.28s/it]
 33%|███▎      | 3936/12000 [2:20:48<4:49:02,  2.15s/it]
                                                        

 33%|███▎      | 3936/12000 [2:20:48<4:49:02,  2.15s/it]
 33%|███▎      | 3937/12000 [2:20:51<5:01:51,  2.25s/it]
                                                        

 33%|███▎      | 3937/12000 [2:20:51<5:01:51,  2.25s/it]
 33%|███▎      | 3938/12000 [2:20:53<4:43:23,  2.11s/it]
                                                        

 33%|███▎      | 3938/12000 [2:20:53<4:43:23,  2.11s/it]
 33%|███▎      | 3939/12000 [2:20:54<4:25:21,  1.98s/it]
                                                        

 33%|███▎      | 3939/12000 [2:20:54<4:25:21,  1.98s/it]
 33%|███▎      | 3940/12000 [2:20:57<4:51:52,  2.17s/it]
                                                        

 33%|███▎      | 3940/12000 [2:20:57<4:51:52,  2.17s/it]
 33%|███▎      | 3941/12000 [2:20:58<4:17:09,  1.91s/it]
                                                        

 33%|███▎      | 3941/12000 [2:20:58<4:17:09,  1.91s/it]
 33%|███▎      | 3942/12000 [2:21:00<4:20:33,  1.94s/it]
                                                        

 33%|███▎      | 3942/12000 [2:21:00<4:20:33,  1.94s/it]
 33%|███▎      | 3943/12000 [2:21:02<4:17:33,  1.92s/it]
                                                        

 33%|███▎      | 3943/12000 [2:21:02<4:17:33,  1.92s/it]
 33%|███▎      | 3944/12000 [2:21:04<4:28:17,  2.00s/it]
                                                        

 33%|███▎      | 3944/12000 [2:21:04<4:28:17,  2.00s/it]
 33%|███▎      | 3945/12000 [2:21:05<3:59:19,  1.78s/it]
                                                        

 33%|███▎      | 3945/12000 [2:21:05<3:59:19,  1.78s/it]
 33%|███▎      | 3946/12000 [2:21:07<3:38:01,  1.62s/it]
                                                        

 33%|███▎      | 3946/12000 [2:21:07<3:38:01,  1.62s/it]
 33%|███▎      | 3947/12000 [2:21:08<3:30:57,  1.57s/it]
                                                        

 33%|███▎      | 3947/12000 [2:21:08<3:30:57,  1.57s/it]
 33%|███▎      | 3948/12000 [2:21:09<3:21:25,  1.50s/it]
                                                        

 33%|███▎      | 3948/12000 [2:21:09<3:21:25,  1.50s/it]
 33%|███▎      | 3949/12000 [2:21:12<3:46:35,  1.69s/it]
                                                        

 33%|███▎      | 3949/12000 [2:21:12<3:46:35,  1.69s/it]
 33%|███▎      | 3950/12000 [2:21:13<3:34:41,  1.60s/it]
                                                        

 33%|███▎      | 3950/12000 [2:21:13<3:34:41,  1.60s/it]
 33%|███▎      | 3951/12000 [2:21:16<4:34:44,  2.05s/it]
                                                        

 33%|███▎      | 3951/12000 [2:21:16<4:34:44,  2.05s/it]
 33%|███▎      | 3952/12000 [2:21:18<4:38:26,  2.08s/it]
                                                        
{'loss': '0.7818', 'grad_norm': '0.4143', 'learning_rate': '0.0001613', 'entropy': '0.7424', 'num_tokens': '2.114e+07', 'mean_token_accuracy': '0.8102', 'epoch': '0.03456'}
{'loss': '0.6885', 'grad_norm': '0.449', 'learning_rate': '0.0001613', 'entropy': '0.7243', 'num_tokens': '2.114e+07', 'mean_token_accuracy': '0.8249', 'epoch': '0.03457'}
{'loss': '1.024', 'grad_norm': '0.496', 'learning_rate': '0.0001613', 'entropy': '1.011', 'num_tokens': '2.115e+07', 'mean_token_accuracy': '0.7432', 'epoch': '0.03458'}
{'loss': '1.27', 'grad_norm': '0.3914', 'learning_rate': '0.0001613', 'entropy': '1.164', 'num_tokens': '2.115e+07', 'mean_token_accuracy': '0.7163', 'epoch': '0.03459'}
{'loss': '0.8449', 'grad_norm': '0.4253', 'learning_rate': '0.0001612', 'entropy': '0.9397', 'num_tokens': '2.116e+07', 'mean_token_accuracy': '0.7649', 'epoch': '0.03459'}
{'loss': '1.231', 'grad_norm': '0.3928', 'learning_rate': '0.0001612', 'entropy': '0.8907', 'num_tokens': '2.117e+07', 'mean_token_accuracy': '0.799', 'epoch': '0.0346'}
{'loss': '0.7481', 'grad_norm': '0.3609', 'learning_rate': '0.0001612', 'entropy': '0.7818', 'num_tokens': '2.117e+07', 'mean_token_accuracy': '0.8049', 'epoch': '0.03461'}
{'loss': '1.225', 'grad_norm': '0.4869', 'learning_rate': '0.0001612', 'entropy': '1.207', 'num_tokens': '2.118e+07', 'mean_token_accuracy': '0.7203', 'epoch': '0.03462'}
{'loss': '0.5878', 'grad_norm': '0.3616', 'learning_rate': '0.0001612', 'entropy': '0.5947', 'num_tokens': '2.118e+07', 'mean_token_accuracy': '0.8393', 'epoch': '0.03463'}
{'loss': '0.7757', 'grad_norm': '0.395', 'learning_rate': '0.0001611', 'entropy': '0.7982', 'num_tokens': '2.118e+07', 'mean_token_accuracy': '0.7922', 'epoch': '0.03464'}
{'loss': '0.778', 'grad_norm': '0.3298', 'learning_rate': '0.0001611', 'entropy': '0.8321', 'num_tokens': '2.119e+07', 'mean_token_accuracy': '0.8021', 'epoch': '0.03465'}
{'loss': '1.062', 'grad_norm': '0.4114', 'learning_rate': '0.0001611', 'entropy': '1.109', 'num_tokens': '2.12e+07', 'mean_token_accuracy': '0.7224', 'epoch': '0.03466'}
{'loss': '0.5928', 'grad_norm': '0.3239', 'learning_rate': '0.0001611', 'entropy': '0.6453', 'num_tokens': '2.12e+07', 'mean_token_accuracy': '0.8405', 'epoch': '0.03466'}
{'loss': '0.7902', 'grad_norm': '0.3079', 'learning_rate': '0.0001611', 'entropy': '0.7791', 'num_tokens': '2.121e+07', 'mean_token_accuracy': '0.817', 'epoch': '0.03467'}
{'loss': '0.5153', 'grad_norm': '0.2647', 'learning_rate': '0.000161', 'entropy': '0.5777', 'num_tokens': '2.121e+07', 'mean_token_accuracy': '0.8613', 'epoch': '0.03468'}
{'loss': '0.7778', 'grad_norm': '0.3707', 'learning_rate': '0.000161', 'entropy': '0.8569', 'num_tokens': '2.122e+07', 'mean_token_accuracy': '0.802', 'epoch': '0.03469'}
{'loss': '0.6693', 'grad_norm': '0.3506', 'learning_rate': '0.000161', 'entropy': '0.7609', 'num_tokens': '2.122e+07', 'mean_token_accuracy': '0.8062', 'epoch': '0.0347'}
{'loss': '1.049', 'grad_norm': '0.4036', 'learning_rate': '0.000161', 'entropy': '0.965', 'num_tokens': '2.123e+07', 'mean_token_accuracy': '0.7642', 'epoch': '0.03471'}
{'loss': '0.6177', 'grad_norm': '0.3339', 'learning_rate': '0.0001609', 'entropy': '0.6344', 'num_tokens': '2.123e+07', 'mean_token_accuracy': '0.8491', 'epoch': '0.03472'}
{'loss': '1.055', 'grad_norm': '0.4692', 'learning_rate': '0.0001609', 'entropy': '1.018', 'num_tokens': '2.124e+07', 'mean_token_accuracy': '0.733', 'epoch': '0.03473'}
{'loss': '1.031', 'grad_norm': '0.4663', 'learning_rate': '0.0001609', 'entropy': '0.977', 'num_tokens': '2.124e+07', 'mean_token_accuracy': '0.7463', 'epoch': '0.03474'}
{'loss': '0.6953', 'grad_norm': '0.3211', 'learning_rate': '0.0001609', 'entropy': '0.7605', 'num_tokens': '2.125e+07', 'mean_token_accuracy': '0.8019', 'epoch': '0.03474'}
{'loss': '1.144', 'grad_norm': '0.5036', 'learning_rate': '0.0001609', 'entropy': '1.14', 'num_tokens': '2.125e+07', 'mean_token_accuracy': '0.7373', 'epoch': '0.03475'}
{'loss': '0.8691', 'grad_norm': '0.4159', 'learning_rate': '0.0001608', 'entropy': '0.8418', 'num_tokens': '2.125e+07', 'mean_token_accuracy': '0.7951', 'epoch': '0.03476'}
{'loss': '1.086', 'grad_norm': '0.3875', 'learning_rate': '0.0001608', 'entropy': '1.023', 'num_tokens': '2.126e+07', 'mean_token_accuracy': '0.7446', 'epoch': '0.03477'}
{'loss': '1.202', 'grad_norm': '0.4867', 'learning_rate': '0.0001608', 'entropy': '1.197', 'num_tokens': '2.126e+07', 'mean_token_accuracy': '0.7087', 'epoch': '0.03478'}
{'loss': '0.8374', 'grad_norm': '0.4073', 'learning_rate': '0.0001608', 'entropy': '0.8259', 'num_tokens': '2.127e+07', 'mean_token_accuracy': '0.7789', 'epoch': '0.03479'}
{'loss': '1.087', 'grad_norm': '0.4173', 'learning_rate': '0.0001607', 'entropy': '0.9802', 'num_tokens': '2.127e+07', 'mean_token_accuracy': '0.7545', 'epoch': '0.0348'}
{'loss': '0.9885', 'grad_norm': '0.4122', 'learning_rate': '0.0001607', 'entropy': '0.9583', 'num_tokens': '2.128e+07', 'mean_token_accuracy': '0.7705', 'epoch': '0.03481'}
{'loss': '0.5613', 'grad_norm': '0.2446', 'learning_rate': '0.0001607', 'entropy': '0.7127', 'num_tokens': '2.129e+07', 'mean_token_accuracy': '0.8245', 'epoch': '0.03482'}
{'loss': '0.8577', 'grad_norm': '0.3048', 'learning_rate': '0.0001607', 'entropy': '0.7825', 'num_tokens': '2.129e+07', 'mean_token_accuracy': '0.8168', 'epoch': '0.03482'}
{'loss': '0.7412', 'grad_norm': '0.3406', 'learning_rate': '0.0001607', 'entropy': '0.7455', 'num_tokens': '2.13e+07', 'mean_token_accuracy': '0.8067', 'epoch': '0.03483'}
{'loss': '0.7625', 'grad_norm': '0.3358', 'learning_rate': '0.0001606', 'entropy': '0.7735', 'num_tokens': '2.13e+07', 'mean_token_accuracy': '0.7951', 'epoch': '0.03484'}
{'loss': '0.9833', 'grad_norm': '0.3642', 'learning_rate': '0.0001606', 'entropy': '1.009', 'num_tokens': '2.131e+07', 'mean_token_accuracy': '0.744', 'epoch': '0.03485'}
{'loss': '0.6662', 'grad_norm': '0.304', 'learning_rate': '0.0001606', 'entropy': '0.7432', 'num_tokens': '2.131e+07', 'mean_token_accuracy': '0.8078', 'epoch': '0.03486'}
{'loss': '0.9219', 'grad_norm': '0.3435', 'learning_rate': '0.0001606', 'entropy': '0.863', 'num_tokens': '2.132e+07', 'mean_token_accuracy': '0.7972', 'epoch': '0.03487'}
{'loss': '0.6365', 'grad_norm': '0.3133', 'learning_rate': '0.0001605', 'entropy': '0.7806', 'num_tokens': '2.133e+07', 'mean_token_accuracy': '0.8014', 'epoch': '0.03488'}
{'loss': '0.7432', 'grad_norm': '0.3602', 'learning_rate': '0.0001605', 'entropy': '0.7634', 'num_tokens': '2.133e+07', 'mean_token_accuracy': '0.8024', 'epoch': '0.03489'}
{'loss': '0.6821', 'grad_norm': '0.3754', 'learning_rate': '0.0001605', 'entropy': '0.7121', 'num_tokens': '2.134e+07', 'mean_token_accuracy': '0.8159', 'epoch': '0.03489'}
{'loss': '1.023', 'grad_norm': '0.3913', 'learning_rate': '0.0001605', 'entropy': '0.9949', 'num_tokens': '2.134e+07', 'mean_token_accuracy': '0.7809', 'epoch': '0.0349'}
{'loss': '0.8116', 'grad_norm': '0.3785', 'learning_rate': '0.0001605', 'entropy': '0.836', 'num_tokens': '2.135e+07', 'mean_token_accuracy': '0.8037', 'epoch': '0.03491'}
{'loss': '1.01', 'grad_norm': '0.4475', 'learning_rate': '0.0001604', 'entropy': '1.011', 'num_tokens': '2.135e+07', 'mean_token_accuracy': '0.7417', 'epoch': '0.03492'}
{'loss': '0.8784', 'grad_norm': '0.4384', 'learning_rate': '0.0001604', 'entropy': '0.8816', 'num_tokens': '2.135e+07', 'mean_token_accuracy': '0.7727', 'epoch': '0.03493'}
{'loss': '0.8052', 'grad_norm': '0.3769', 'learning_rate': '0.0001604', 'entropy': '0.7846', 'num_tokens': '2.136e+07', 'mean_token_accuracy': '0.8005', 'epoch': '0.03494'}
{'loss': '0.8439', 'grad_norm': '0.3806', 'learning_rate': '0.0001604', 'entropy': '0.8314', 'num_tokens': '2.136e+07', 'mean_token_accuracy': '0.773', 'epoch': '0.03495'}
{'loss': '0.8581', 'grad_norm': '0.4048', 'learning_rate': '0.0001604', 'entropy': '0.8736', 'num_tokens': '2.137e+07', 'mean_token_accuracy': '0.7727', 'epoch': '0.03496'}
{'loss': '0.6484', 'grad_norm': '0.3572', 'learning_rate': '0.0001603', 'entropy': '0.6286', 'num_tokens': '2.137e+07', 'mean_token_accuracy': '0.8333', 'epoch': '0.03497'}

 33%|███▎      | 3952/12000 [2:21:18<4:38:26,  2.08s/it]
 33%|███▎      | 3953/12000 [2:21:20<4:10:03,  1.86s/it]
                                                        

 33%|███▎      | 3953/12000 [2:21:20<4:10:03,  1.86s/it]
 33%|███▎      | 3954/12000 [2:21:23<5:29:16,  2.46s/it]
                                                        

 33%|███▎      | 3954/12000 [2:21:23<5:29:16,  2.46s/it]
 33%|███▎      | 3955/12000 [2:21:25<4:47:28,  2.14s/it]
                                                        

 33%|███▎      | 3955/12000 [2:21:25<4:47:28,  2.14s/it]
 33%|███▎      | 3956/12000 [2:21:27<4:28:40,  2.00s/it]
                                                        

 33%|███▎      | 3956/12000 [2:21:27<4:28:40,  2.00s/it]
 33%|███▎      | 3957/12000 [2:21:28<4:07:40,  1.85s/it]
                                                        

 33%|███▎      | 3957/12000 [2:21:28<4:07:40,  1.85s/it]
 33%|███▎      | 3958/12000 [2:21:29<3:48:46,  1.71s/it]
                                                        

 33%|███▎      | 3958/12000 [2:21:29<3:48:46,  1.71s/it]
 33%|███▎      | 3959/12000 [2:21:31<3:42:41,  1.66s/it]
                                                        

 33%|███▎      | 3959/12000 [2:21:31<3:42:41,  1.66s/it]
 33%|███▎      | 3960/12000 [2:21:33<4:03:34,  1.82s/it]
                                                        

 33%|███▎      | 3960/12000 [2:21:33<4:03:34,  1.82s/it]
 33%|███▎      | 3961/12000 [2:21:35<3:50:43,  1.72s/it]
                                                        

 33%|███▎      | 3961/12000 [2:21:35<3:50:43,  1.72s/it]
 33%|███▎      | 3962/12000 [2:21:37<4:19:53,  1.94s/it]
                                                        

 33%|███▎      | 3962/12000 [2:21:37<4:19:53,  1.94s/it]
 33%|███▎      | 3963/12000 [2:21:39<4:05:19,  1.83s/it]
                                                        

 33%|███▎      | 3963/12000 [2:21:39<4:05:19,  1.83s/it]
 33%|███▎      | 3964/12000 [2:21:42<5:20:33,  2.39s/it]
                                                        

 33%|███▎      | 3964/12000 [2:21:42<5:20:33,  2.39s/it]
 33%|███▎      | 3965/12000 [2:21:46<5:54:53,  2.65s/it]
                                                        

 33%|███▎      | 3965/12000 [2:21:46<5:54:53,  2.65s/it]
 33%|███▎      | 3966/12000 [2:21:47<5:07:56,  2.30s/it]
                                                        

 33%|███▎      | 3966/12000 [2:21:47<5:07:56,  2.30s/it]
 33%|███▎      | 3967/12000 [2:21:49<4:43:01,  2.11s/it]
                                                        

 33%|███▎      | 3967/12000 [2:21:49<4:43:01,  2.11s/it]
 33%|███▎      | 3968/12000 [2:21:50<4:10:30,  1.87s/it]
                                                        

 33%|███▎      | 3968/12000 [2:21:50<4:10:30,  1.87s/it]
 33%|███▎      | 3969/12000 [2:21:52<4:05:43,  1.84s/it]
                                                        

 33%|███▎      | 3969/12000 [2:21:52<4:05:43,  1.84s/it]
 33%|███▎      | 3970/12000 [2:21:56<5:54:05,  2.65s/it]
                                                        

 33%|███▎      | 3970/12000 [2:21:56<5:54:05,  2.65s/it]
 33%|███▎      | 3971/12000 [2:21:58<5:05:05,  2.28s/it]
                                                        

 33%|███▎      | 3971/12000 [2:21:58<5:05:05,  2.28s/it]
 33%|███▎      | 3972/12000 [2:22:01<5:37:34,  2.52s/it]
                                                        

 33%|███▎      | 3972/12000 [2:22:01<5:37:34,  2.52s/it]
 33%|███▎      | 3973/12000 [2:22:02<4:58:32,  2.23s/it]
                                                        

 33%|███▎      | 3973/12000 [2:22:02<4:58:32,  2.23s/it]
 33%|███▎      | 3974/12000 [2:22:04<4:25:45,  1.99s/it]
                                                        

 33%|███▎      | 3974/12000 [2:22:04<4:25:45,  1.99s/it]
 33%|███▎      | 3975/12000 [2:22:08<6:03:39,  2.72s/it]
                                                        

 33%|███▎      | 3975/12000 [2:22:08<6:03:39,  2.72s/it]
 33%|███▎      | 3976/12000 [2:22:12<6:30:50,  2.92s/it]
                                                        

 33%|███▎      | 3976/12000 [2:22:12<6:30:50,  2.92s/it]
 33%|███▎      | 3977/12000 [2:22:13<5:26:38,  2.44s/it]
                                                        

 33%|███▎      | 3977/12000 [2:22:13<5:26:38,  2.44s/it]
 33%|███▎      | 3978/12000 [2:22:15<5:16:30,  2.37s/it]
                                                        

 33%|███▎      | 3978/12000 [2:22:15<5:16:30,  2.37s/it]
 33%|███▎      | 3979/12000 [2:22:17<4:45:23,  2.13s/it]
                                                        

 33%|███▎      | 3979/12000 [2:22:17<4:45:23,  2.13s/it]
 33%|███▎      | 3980/12000 [2:22:19<5:04:53,  2.28s/it]
                                                        

 33%|███▎      | 3980/12000 [2:22:19<5:04:53,  2.28s/it]
 33%|███▎      | 3981/12000 [2:22:21<4:29:25,  2.02s/it]
                                                        

 33%|███▎      | 3981/12000 [2:22:21<4:29:25,  2.02s/it]
 33%|███▎      | 3982/12000 [2:22:22<4:11:33,  1.88s/it]
                                                        

 33%|███▎      | 3982/12000 [2:22:22<4:11:33,  1.88s/it]
 33%|███▎      | 3983/12000 [2:22:25<4:22:43,  1.97s/it]
                                                        

 33%|███▎      | 3983/12000 [2:22:25<4:22:43,  1.97s/it]
 33%|███▎      | 3984/12000 [2:22:27<4:22:48,  1.97s/it]
                                                        

 33%|███▎      | 3984/12000 [2:22:27<4:22:48,  1.97s/it]
 33%|███▎      | 3985/12000 [2:22:29<4:38:55,  2.09s/it]
                                                        

 33%|███▎      | 3985/12000 [2:22:29<4:38:55,  2.09s/it]
 33%|███▎      | 3986/12000 [2:22:31<4:41:02,  2.10s/it]
                                                        

 33%|███▎      | 3986/12000 [2:22:31<4:41:02,  2.10s/it]
 33%|███▎      | 3987/12000 [2:22:32<4:15:26,  1.91s/it]
                                                        

 33%|███▎      | 3987/12000 [2:22:32<4:15:26,  1.91s/it]
 33%|███▎      | 3988/12000 [2:22:34<4:13:43,  1.90s/it]
                                                        

 33%|███▎      | 3988/12000 [2:22:34<4:13:43,  1.90s/it]
 33%|███▎      | 3989/12000 [2:22:36<4:03:50,  1.83s/it]
                                                        

 33%|███▎      | 3989/12000 [2:22:36<4:03:50,  1.83s/it]
 33%|███▎      | 3990/12000 [2:22:38<4:23:44,  1.98s/it]
                                                        

 33%|███▎      | 3990/12000 [2:22:38<4:23:44,  1.98s/it]
 33%|███▎      | 3991/12000 [2:22:40<3:56:18,  1.77s/it]
                                                        

 33%|███▎      | 3991/12000 [2:22:40<3:56:18,  1.77s/it]
 33%|███▎      | 3992/12000 [2:22:41<3:55:33,  1.76s/it]
                                                        

 33%|███▎      | 3992/12000 [2:22:41<3:55:33,  1.76s/it]
 33%|███▎      | 3993/12000 [2:22:44<4:13:16,  1.90s/it]
                                                        

 33%|███▎      | 3993/12000 [2:22:44<4:13:16,  1.90s/it]
 33%|███▎      | 3994/12000 [2:22:46<4:23:14,  1.97s/it]
                                                        

 33%|███▎      | 3994/12000 [2:22:46<4:23:14,  1.97s/it]
 33%|███▎      | 3995/12000 [2:22:48<4:15:49,  1.92s/it]
                                                        

 33%|███▎      | 3995/12000 [2:22:48<4:15:49,  1.92s/it]
 33%|███▎      | 3996/12000 [2:22:49<3:49:39,  1.72s/it]
                                                        

 33%|███▎      | 3996/12000 [2:22:49<3:49:39,  1.72s/it]
 33%|███▎      | 3997/12000 [2:22:50<3:32:34,  1.59s/it]
                                                        

 33%|███▎      | 3997/12000 [2:22:50<3:32:34,  1.59s/it]
 33%|███▎      | 3998/12000 [2:22:53<4:32:36,  2.04s/it]
                                                        

 33%|███▎      | 3998/12000 [2:22:53<4:32:36,  2.04s/it]
 33%|███▎      | 3999/12000 [2:22:56<5:05:50,  2.29s/it]
                                                        
{'loss': '0.8895', 'grad_norm': '0.3195', 'learning_rate': '0.0001603', 'entropy': '0.929', 'num_tokens': '2.138e+07', 'mean_token_accuracy': '0.7813', 'epoch': '0.03497'}
{'loss': '0.8017', 'grad_norm': '0.3741', 'learning_rate': '0.0001603', 'entropy': '0.7803', 'num_tokens': '2.138e+07', 'mean_token_accuracy': '0.7965', 'epoch': '0.03498'}
{'loss': '0.7671', 'grad_norm': '0.3843', 'learning_rate': '0.0001603', 'entropy': '0.8489', 'num_tokens': '2.139e+07', 'mean_token_accuracy': '0.7895', 'epoch': '0.03499'}
{'loss': '0.9038', 'grad_norm': '0.4654', 'learning_rate': '0.0001602', 'entropy': '0.9615', 'num_tokens': '2.139e+07', 'mean_token_accuracy': '0.7696', 'epoch': '0.035'}
{'loss': '0.8436', 'grad_norm': '0.3976', 'learning_rate': '0.0001602', 'entropy': '0.8318', 'num_tokens': '2.14e+07', 'mean_token_accuracy': '0.779', 'epoch': '0.03501'}
{'loss': '0.7579', 'grad_norm': '0.3708', 'learning_rate': '0.0001602', 'entropy': '0.7375', 'num_tokens': '2.14e+07', 'mean_token_accuracy': '0.8125', 'epoch': '0.03502'}
{'loss': '0.8517', 'grad_norm': '0.3549', 'learning_rate': '0.0001602', 'entropy': '0.8203', 'num_tokens': '2.141e+07', 'mean_token_accuracy': '0.7937', 'epoch': '0.03503'}
{'loss': '0.973', 'grad_norm': '0.4599', 'learning_rate': '0.0001602', 'entropy': '0.9687', 'num_tokens': '2.141e+07', 'mean_token_accuracy': '0.7596', 'epoch': '0.03504'}
{'loss': '1.199', 'grad_norm': '0.4446', 'learning_rate': '0.0001601', 'entropy': '1.188', 'num_tokens': '2.142e+07', 'mean_token_accuracy': '0.7149', 'epoch': '0.03505'}
{'loss': '0.8253', 'grad_norm': '0.3488', 'learning_rate': '0.0001601', 'entropy': '0.8162', 'num_tokens': '2.142e+07', 'mean_token_accuracy': '0.803', 'epoch': '0.03505'}
{'loss': '0.9104', 'grad_norm': '0.3146', 'learning_rate': '0.0001601', 'entropy': '0.9813', 'num_tokens': '2.143e+07', 'mean_token_accuracy': '0.7589', 'epoch': '0.03506'}
{'loss': '0.7936', 'grad_norm': '0.3475', 'learning_rate': '0.0001601', 'entropy': '0.8233', 'num_tokens': '2.143e+07', 'mean_token_accuracy': '0.7935', 'epoch': '0.03507'}
{'loss': '1.325', 'grad_norm': '0.3726', 'learning_rate': '0.00016', 'entropy': '1.209', 'num_tokens': '2.144e+07', 'mean_token_accuracy': '0.7096', 'epoch': '0.03508'}
{'loss': '0.7766', 'grad_norm': '0.2797', 'learning_rate': '0.00016', 'entropy': '0.7323', 'num_tokens': '2.145e+07', 'mean_token_accuracy': '0.8108', 'epoch': '0.03509'}
{'loss': '0.9605', 'grad_norm': '0.395', 'learning_rate': '0.00016', 'entropy': '0.9213', 'num_tokens': '2.146e+07', 'mean_token_accuracy': '0.7798', 'epoch': '0.0351'}
{'loss': '0.8695', 'grad_norm': '0.4129', 'learning_rate': '0.00016', 'entropy': '0.9541', 'num_tokens': '2.146e+07', 'mean_token_accuracy': '0.7594', 'epoch': '0.03511'}
{'loss': '0.8083', 'grad_norm': '0.4066', 'learning_rate': '0.00016', 'entropy': '0.7985', 'num_tokens': '2.146e+07', 'mean_token_accuracy': '0.7931', 'epoch': '0.03512'}
{'loss': '0.7418', 'grad_norm': '0.4282', 'learning_rate': '0.0001599', 'entropy': '0.7336', 'num_tokens': '2.147e+07', 'mean_token_accuracy': '0.798', 'epoch': '0.03512'}
{'loss': '1.352', 'grad_norm': '0.383', 'learning_rate': '0.0001599', 'entropy': '0.9229', 'num_tokens': '2.148e+07', 'mean_token_accuracy': '0.7763', 'epoch': '0.03513'}
{'loss': '0.8543', 'grad_norm': '0.3593', 'learning_rate': '0.0001599', 'entropy': '0.8834', 'num_tokens': '2.148e+07', 'mean_token_accuracy': '0.7835', 'epoch': '0.03514'}
{'loss': '0.86', 'grad_norm': '0.4212', 'learning_rate': '0.0001599', 'entropy': '0.8207', 'num_tokens': '2.149e+07', 'mean_token_accuracy': '0.8047', 'epoch': '0.03515'}
{'loss': '0.79', 'grad_norm': '0.4474', 'learning_rate': '0.0001598', 'entropy': '0.7783', 'num_tokens': '2.149e+07', 'mean_token_accuracy': '0.7979', 'epoch': '0.03516'}
{'loss': '0.6015', 'grad_norm': '0.2978', 'learning_rate': '0.0001598', 'entropy': '0.6787', 'num_tokens': '2.15e+07', 'mean_token_accuracy': '0.8297', 'epoch': '0.03517'}
{'loss': '0.614', 'grad_norm': '0.2355', 'learning_rate': '0.0001598', 'entropy': '0.7984', 'num_tokens': '2.15e+07', 'mean_token_accuracy': '0.7969', 'epoch': '0.03518'}
{'loss': '0.588', 'grad_norm': '0.3347', 'learning_rate': '0.0001598', 'entropy': '0.5809', 'num_tokens': '2.151e+07', 'mean_token_accuracy': '0.8471', 'epoch': '0.03519'}
{'loss': '0.9173', 'grad_norm': '0.3893', 'learning_rate': '0.0001598', 'entropy': '0.903', 'num_tokens': '2.151e+07', 'mean_token_accuracy': '0.7654', 'epoch': '0.0352'}
{'loss': '0.8469', 'grad_norm': '0.3938', 'learning_rate': '0.0001597', 'entropy': '0.8518', 'num_tokens': '2.152e+07', 'mean_token_accuracy': '0.7779', 'epoch': '0.0352'}
{'loss': '1.068', 'grad_norm': '0.4695', 'learning_rate': '0.0001597', 'entropy': '1.152', 'num_tokens': '2.152e+07', 'mean_token_accuracy': '0.7137', 'epoch': '0.03521'}
{'loss': '0.7489', 'grad_norm': '0.3874', 'learning_rate': '0.0001597', 'entropy': '0.8041', 'num_tokens': '2.153e+07', 'mean_token_accuracy': '0.8016', 'epoch': '0.03522'}
{'loss': '0.5778', 'grad_norm': '0.3182', 'learning_rate': '0.0001597', 'entropy': '0.578', 'num_tokens': '2.154e+07', 'mean_token_accuracy': '0.8483', 'epoch': '0.03523'}
{'loss': '0.5735', 'grad_norm': '0.3334', 'learning_rate': '0.0001596', 'entropy': '0.5767', 'num_tokens': '2.154e+07', 'mean_token_accuracy': '0.8488', 'epoch': '0.03524'}
{'loss': '1.432', 'grad_norm': '0.4707', 'learning_rate': '0.0001596', 'entropy': '1.395', 'num_tokens': '2.155e+07', 'mean_token_accuracy': '0.6823', 'epoch': '0.03525'}
{'loss': '0.5704', 'grad_norm': '0.366', 'learning_rate': '0.0001596', 'entropy': '0.5984', 'num_tokens': '2.155e+07', 'mean_token_accuracy': '0.8389', 'epoch': '0.03526'}
{'loss': '1.072', 'grad_norm': '0.358', 'learning_rate': '0.0001596', 'entropy': '1.027', 'num_tokens': '2.156e+07', 'mean_token_accuracy': '0.7631', 'epoch': '0.03527'}
{'loss': '0.9786', 'grad_norm': '0.4803', 'learning_rate': '0.0001596', 'entropy': '0.9496', 'num_tokens': '2.157e+07', 'mean_token_accuracy': '0.7613', 'epoch': '0.03528'}
{'loss': '0.9951', 'grad_norm': '0.3997', 'learning_rate': '0.0001595', 'entropy': '0.9894', 'num_tokens': '2.157e+07', 'mean_token_accuracy': '0.7615', 'epoch': '0.03528'}
{'loss': '1.074', 'grad_norm': '0.4211', 'learning_rate': '0.0001595', 'entropy': '1.074', 'num_tokens': '2.158e+07', 'mean_token_accuracy': '0.7472', 'epoch': '0.03529'}
{'loss': '0.9254', 'grad_norm': '0.439', 'learning_rate': '0.0001595', 'entropy': '0.9328', 'num_tokens': '2.158e+07', 'mean_token_accuracy': '0.7529', 'epoch': '0.0353'}
{'loss': '0.6891', 'grad_norm': '0.3595', 'learning_rate': '0.0001595', 'entropy': '0.716', 'num_tokens': '2.159e+07', 'mean_token_accuracy': '0.8176', 'epoch': '0.03531'}
{'loss': '0.9339', 'grad_norm': '0.4459', 'learning_rate': '0.0001594', 'entropy': '0.9111', 'num_tokens': '2.159e+07', 'mean_token_accuracy': '0.7751', 'epoch': '0.03532'}
{'loss': '0.7396', 'grad_norm': '0.3659', 'learning_rate': '0.0001594', 'entropy': '0.7119', 'num_tokens': '2.16e+07', 'mean_token_accuracy': '0.8215', 'epoch': '0.03533'}
{'loss': '0.6828', 'grad_norm': '0.4675', 'learning_rate': '0.0001594', 'entropy': '0.6819', 'num_tokens': '2.16e+07', 'mean_token_accuracy': '0.8313', 'epoch': '0.03534'}
{'loss': '0.6778', 'grad_norm': '0.3136', 'learning_rate': '0.0001594', 'entropy': '0.7233', 'num_tokens': '2.161e+07', 'mean_token_accuracy': '0.817', 'epoch': '0.03535'}
{'loss': '0.824', 'grad_norm': '0.3835', 'learning_rate': '0.0001594', 'entropy': '0.8587', 'num_tokens': '2.162e+07', 'mean_token_accuracy': '0.7893', 'epoch': '0.03536'}
{'loss': '0.733', 'grad_norm': '0.3627', 'learning_rate': '0.0001593', 'entropy': '0.7549', 'num_tokens': '2.162e+07', 'mean_token_accuracy': '0.802', 'epoch': '0.03536'}
{'loss': '0.7591', 'grad_norm': '0.3953', 'learning_rate': '0.0001593', 'entropy': '0.875', 'num_tokens': '2.162e+07', 'mean_token_accuracy': '0.7924', 'epoch': '0.03537'}
{'loss': '0.7079', 'grad_norm': '0.3919', 'learning_rate': '0.0001593', 'entropy': '0.6924', 'num_tokens': '2.163e+07', 'mean_token_accuracy': '0.8128', 'epoch': '0.03538'}

 33%|███▎      | 3999/12000 [2:22:56<5:05:50,  2.29s/it]
 33%|███▎      | 4000/12000 [2:22:58<4:44:19,  2.13s/it]
                                                        

 33%|███▎      | 4000/12000 [2:22:58<4:44:19,  2.13s/it]
 33%|███▎      | 4001/12000 [2:23:01<5:16:17,  2.37s/it]
                                                        

 33%|███▎      | 4001/12000 [2:23:01<5:16:17,  2.37s/it]
 33%|███▎      | 4002/12000 [2:23:03<4:55:34,  2.22s/it]
                                                        

 33%|███▎      | 4002/12000 [2:23:03<4:55:34,  2.22s/it]
 33%|███▎      | 4003/12000 [2:23:05<4:53:19,  2.20s/it]
                                                        

 33%|███▎      | 4003/12000 [2:23:05<4:53:19,  2.20s/it]
 33%|███▎      | 4004/12000 [2:23:06<4:22:28,  1.97s/it]
                                                        

 33%|███▎      | 4004/12000 [2:23:06<4:22:28,  1.97s/it]
 33%|███▎      | 4005/12000 [2:23:09<4:47:23,  2.16s/it]
                                                        

 33%|███▎      | 4005/12000 [2:23:09<4:47:23,  2.16s/it]
 33%|███▎      | 4006/12000 [2:23:11<4:35:19,  2.07s/it]
                                                        

 33%|███▎      | 4006/12000 [2:23:11<4:35:19,  2.07s/it]
 33%|███▎      | 4007/12000 [2:23:12<4:05:45,  1.84s/it]
                                                        

 33%|███▎      | 4007/12000 [2:23:12<4:05:45,  1.84s/it]
 33%|███▎      | 4008/12000 [2:23:13<3:45:28,  1.69s/it]
                                                        

 33%|███▎      | 4008/12000 [2:23:13<3:45:28,  1.69s/it]
 33%|███▎      | 4009/12000 [2:23:15<3:29:23,  1.57s/it]
                                                        

 33%|███▎      | 4009/12000 [2:23:15<3:29:23,  1.57s/it]
 33%|███▎      | 4010/12000 [2:23:16<3:39:38,  1.65s/it]
                                                        

 33%|███▎      | 4010/12000 [2:23:16<3:39:38,  1.65s/it]
 33%|███▎      | 4011/12000 [2:23:19<4:04:27,  1.84s/it]
                                                        

 33%|███▎      | 4011/12000 [2:23:19<4:04:27,  1.84s/it]
 33%|███▎      | 4012/12000 [2:23:20<3:45:49,  1.70s/it]
                                                        

 33%|███▎      | 4012/12000 [2:23:20<3:45:49,  1.70s/it]
 33%|███▎      | 4013/12000 [2:23:21<3:31:40,  1.59s/it]
                                                        

 33%|███▎      | 4013/12000 [2:23:21<3:31:40,  1.59s/it]
 33%|███▎      | 4014/12000 [2:23:24<4:14:19,  1.91s/it]
                                                        

 33%|███▎      | 4014/12000 [2:23:24<4:14:19,  1.91s/it]
 33%|███▎      | 4015/12000 [2:23:27<4:36:19,  2.08s/it]
                                                        

 33%|███▎      | 4015/12000 [2:23:27<4:36:19,  2.08s/it]
 33%|███▎      | 4016/12000 [2:23:30<5:20:02,  2.41s/it]
                                                        

 33%|███▎      | 4016/12000 [2:23:30<5:20:02,  2.41s/it]
 33%|███▎      | 4017/12000 [2:23:31<4:46:19,  2.15s/it]
                                                        

 33%|███▎      | 4017/12000 [2:23:31<4:46:19,  2.15s/it]
 33%|███▎      | 4018/12000 [2:23:33<4:20:59,  1.96s/it]
                                                        

 33%|███▎      | 4018/12000 [2:23:33<4:20:59,  1.96s/it]
 33%|███▎      | 4019/12000 [2:23:34<3:56:09,  1.78s/it]
                                                        

 33%|███▎      | 4019/12000 [2:23:34<3:56:09,  1.78s/it]
 34%|███▎      | 4020/12000 [2:23:37<4:53:56,  2.21s/it]
                                                        

 34%|███▎      | 4020/12000 [2:23:37<4:53:56,  2.21s/it]
 34%|███▎      | 4021/12000 [2:23:39<4:30:58,  2.04s/it]
                                                        

 34%|███▎      | 4021/12000 [2:23:39<4:30:58,  2.04s/it]
 34%|███▎      | 4022/12000 [2:23:40<4:08:52,  1.87s/it]
                                                        

 34%|███▎      | 4022/12000 [2:23:40<4:08:52,  1.87s/it]
 34%|███▎      | 4023/12000 [2:23:42<3:51:58,  1.74s/it]
                                                        

 34%|███▎      | 4023/12000 [2:23:42<3:51:58,  1.74s/it]
 34%|███▎      | 4024/12000 [2:23:44<4:08:43,  1.87s/it]
                                                        

 34%|███▎      | 4024/12000 [2:23:44<4:08:43,  1.87s/it]
 34%|███▎      | 4025/12000 [2:23:46<3:58:57,  1.80s/it]
                                                        

 34%|███▎      | 4025/12000 [2:23:46<3:58:57,  1.80s/it]
 34%|███▎      | 4026/12000 [2:23:48<4:17:24,  1.94s/it]
                                                        

 34%|███▎      | 4026/12000 [2:23:48<4:17:24,  1.94s/it]
 34%|███▎      | 4027/12000 [2:23:50<4:22:26,  1.97s/it]
                                                        

 34%|███▎      | 4027/12000 [2:23:50<4:22:26,  1.97s/it]
 34%|███▎      | 4028/12000 [2:23:52<4:02:54,  1.83s/it]
                                                        

 34%|███▎      | 4028/12000 [2:23:52<4:02:54,  1.83s/it]
 34%|███▎      | 4029/12000 [2:23:53<4:00:08,  1.81s/it]
                                                        

 34%|███▎      | 4029/12000 [2:23:53<4:00:08,  1.81s/it]
 34%|███▎      | 4030/12000 [2:23:55<3:41:35,  1.67s/it]
                                                        

 34%|███▎      | 4030/12000 [2:23:55<3:41:35,  1.67s/it]
 34%|███▎      | 4031/12000 [2:23:57<4:15:07,  1.92s/it]
                                                        

 34%|███▎      | 4031/12000 [2:23:57<4:15:07,  1.92s/it]
 34%|███▎      | 4032/12000 [2:24:02<6:09:55,  2.79s/it]
                                                        

 34%|███▎      | 4032/12000 [2:24:02<6:09:55,  2.79s/it]
 34%|███▎      | 4033/12000 [2:24:03<5:16:34,  2.38s/it]
                                                        

 34%|███▎      | 4033/12000 [2:24:03<5:16:34,  2.38s/it]
 34%|███▎      | 4034/12000 [2:24:05<4:47:39,  2.17s/it]
                                                        

 34%|███▎      | 4034/12000 [2:24:05<4:47:39,  2.17s/it]
 34%|███▎      | 4035/12000 [2:24:07<4:29:32,  2.03s/it]
                                                        

 34%|███▎      | 4035/12000 [2:24:07<4:29:32,  2.03s/it]
 34%|███▎      | 4036/12000 [2:24:08<4:11:43,  1.90s/it]
                                                        

 34%|███▎      | 4036/12000 [2:24:08<4:11:43,  1.90s/it]
 34%|███▎      | 4037/12000 [2:24:10<3:54:19,  1.77s/it]
                                                        

 34%|███▎      | 4037/12000 [2:24:10<3:54:19,  1.77s/it]
 34%|███▎      | 4038/12000 [2:24:11<3:42:47,  1.68s/it]
                                                        

 34%|███▎      | 4038/12000 [2:24:11<3:42:47,  1.68s/it]
 34%|███▎      | 4039/12000 [2:24:13<3:44:10,  1.69s/it]
                                                        

 34%|███▎      | 4039/12000 [2:24:13<3:44:10,  1.69s/it]
 34%|███▎      | 4040/12000 [2:24:15<3:38:09,  1.64s/it]
                                                        

 34%|███▎      | 4040/12000 [2:24:15<3:38:09,  1.64s/it]
 34%|███▎      | 4041/12000 [2:24:17<4:22:55,  1.98s/it]
                                                        

 34%|███▎      | 4041/12000 [2:24:17<4:22:55,  1.98s/it]
 34%|███▎      | 4042/12000 [2:24:21<5:40:14,  2.57s/it]
                                                        

 34%|███▎      | 4042/12000 [2:24:21<5:40:14,  2.57s/it]
 34%|███▎      | 4043/12000 [2:24:23<5:03:10,  2.29s/it]
                                                        

 34%|███▎      | 4043/12000 [2:24:23<5:03:10,  2.29s/it]
 34%|███▎      | 4044/12000 [2:24:25<4:47:40,  2.17s/it]
                                                        

 34%|███▎      | 4044/12000 [2:24:25<4:47:40,  2.17s/it]
 34%|███▎      | 4045/12000 [2:24:26<4:13:12,  1.91s/it]
                                                        

 34%|███▎      | 4045/12000 [2:24:26<4:13:12,  1.91s/it]
 34%|███▎      | 4046/12000 [2:24:27<3:49:02,  1.73s/it]
                                                        
{'loss': '0.7592', 'grad_norm': '0.3615', 'learning_rate': '0.0001593', 'entropy': '0.8331', 'num_tokens': '2.163e+07', 'mean_token_accuracy': '0.7857', 'epoch': '0.03539'}
{'loss': '0.8816', 'grad_norm': '0.4644', 'learning_rate': '0.0001592', 'entropy': '0.928', 'num_tokens': '2.164e+07', 'mean_token_accuracy': '0.7707', 'epoch': '0.0354'}
{'loss': '0.802', 'grad_norm': '0.4184', 'learning_rate': '0.0001592', 'entropy': '0.7775', 'num_tokens': '2.164e+07', 'mean_token_accuracy': '0.7977', 'epoch': '0.03541'}
{'loss': '0.7802', 'grad_norm': '0.3979', 'learning_rate': '0.0001592', 'entropy': '0.7516', 'num_tokens': '2.165e+07', 'mean_token_accuracy': '0.8072', 'epoch': '0.03542'}
{'loss': '0.607', 'grad_norm': '0.3728', 'learning_rate': '0.0001592', 'entropy': '0.6926', 'num_tokens': '2.166e+07', 'mean_token_accuracy': '0.8184', 'epoch': '0.03543'}
{'loss': '0.8421', 'grad_norm': '0.3533', 'learning_rate': '0.0001592', 'entropy': '0.9414', 'num_tokens': '2.166e+07', 'mean_token_accuracy': '0.7762', 'epoch': '0.03543'}
{'loss': '0.8795', 'grad_norm': '0.351', 'learning_rate': '0.0001591', 'entropy': '0.8594', 'num_tokens': '2.167e+07', 'mean_token_accuracy': '0.7763', 'epoch': '0.03544'}
{'loss': '0.9149', 'grad_norm': '0.4281', 'learning_rate': '0.0001591', 'entropy': '0.9104', 'num_tokens': '2.167e+07', 'mean_token_accuracy': '0.7772', 'epoch': '0.03545'}
{'loss': '0.6378', 'grad_norm': '0.3316', 'learning_rate': '0.0001591', 'entropy': '0.6639', 'num_tokens': '2.168e+07', 'mean_token_accuracy': '0.8336', 'epoch': '0.03546'}
{'loss': '0.6298', 'grad_norm': '0.3485', 'learning_rate': '0.0001591', 'entropy': '0.6867', 'num_tokens': '2.168e+07', 'mean_token_accuracy': '0.8287', 'epoch': '0.03547'}
{'loss': '0.8199', 'grad_norm': '0.405', 'learning_rate': '0.000159', 'entropy': '0.8656', 'num_tokens': '2.168e+07', 'mean_token_accuracy': '0.7779', 'epoch': '0.03548'}
{'loss': '1.134', 'grad_norm': '0.529', 'learning_rate': '0.000159', 'entropy': '0.9885', 'num_tokens': '2.169e+07', 'mean_token_accuracy': '0.7532', 'epoch': '0.03549'}
{'loss': '0.6531', 'grad_norm': '0.2781', 'learning_rate': '0.000159', 'entropy': '0.6316', 'num_tokens': '2.17e+07', 'mean_token_accuracy': '0.8376', 'epoch': '0.0355'}
{'loss': '0.5319', 'grad_norm': '0.309', 'learning_rate': '0.000159', 'entropy': '0.6353', 'num_tokens': '2.17e+07', 'mean_token_accuracy': '0.8255', 'epoch': '0.03551'}
{'loss': '0.764', 'grad_norm': '0.3572', 'learning_rate': '0.000159', 'entropy': '0.776', 'num_tokens': '2.17e+07', 'mean_token_accuracy': '0.8077', 'epoch': '0.03551'}
{'loss': '0.8422', 'grad_norm': '0.3342', 'learning_rate': '0.0001589', 'entropy': '1.013', 'num_tokens': '2.171e+07', 'mean_token_accuracy': '0.7596', 'epoch': '0.03552'}
{'loss': '0.6587', 'grad_norm': '0.3694', 'learning_rate': '0.0001589', 'entropy': '0.8071', 'num_tokens': '2.172e+07', 'mean_token_accuracy': '0.7928', 'epoch': '0.03553'}
{'loss': '0.6434', 'grad_norm': '0.2653', 'learning_rate': '0.0001589', 'entropy': '0.8496', 'num_tokens': '2.172e+07', 'mean_token_accuracy': '0.7892', 'epoch': '0.03554'}
{'loss': '0.8079', 'grad_norm': '0.3361', 'learning_rate': '0.0001589', 'entropy': '0.8196', 'num_tokens': '2.173e+07', 'mean_token_accuracy': '0.7906', 'epoch': '0.03555'}
{'loss': '0.9535', 'grad_norm': '0.4468', 'learning_rate': '0.0001588', 'entropy': '0.916', 'num_tokens': '2.173e+07', 'mean_token_accuracy': '0.7699', 'epoch': '0.03556'}
{'loss': '0.8344', 'grad_norm': '0.3692', 'learning_rate': '0.0001588', 'entropy': '0.831', 'num_tokens': '2.174e+07', 'mean_token_accuracy': '0.7944', 'epoch': '0.03557'}
{'loss': '0.9547', 'grad_norm': '0.3663', 'learning_rate': '0.0001588', 'entropy': '1.045', 'num_tokens': '2.174e+07', 'mean_token_accuracy': '0.7642', 'epoch': '0.03558'}
{'loss': '0.6657', 'grad_norm': '0.3236', 'learning_rate': '0.0001588', 'entropy': '0.6681', 'num_tokens': '2.175e+07', 'mean_token_accuracy': '0.8335', 'epoch': '0.03559'}
{'loss': '0.7635', 'grad_norm': '0.3096', 'learning_rate': '0.0001588', 'entropy': '0.7466', 'num_tokens': '2.175e+07', 'mean_token_accuracy': '0.805', 'epoch': '0.03559'}
{'loss': '0.6718', 'grad_norm': '0.2925', 'learning_rate': '0.0001587', 'entropy': '0.7347', 'num_tokens': '2.176e+07', 'mean_token_accuracy': '0.8194', 'epoch': '0.0356'}
{'loss': '1.073', 'grad_norm': '0.4907', 'learning_rate': '0.0001587', 'entropy': '1.029', 'num_tokens': '2.176e+07', 'mean_token_accuracy': '0.7584', 'epoch': '0.03561'}
{'loss': '0.9677', 'grad_norm': '0.3603', 'learning_rate': '0.0001587', 'entropy': '0.9669', 'num_tokens': '2.177e+07', 'mean_token_accuracy': '0.7657', 'epoch': '0.03562'}
{'loss': '1.125', 'grad_norm': '0.4192', 'learning_rate': '0.0001587', 'entropy': '1.087', 'num_tokens': '2.177e+07', 'mean_token_accuracy': '0.7197', 'epoch': '0.03563'}
{'loss': '0.7202', 'grad_norm': '0.3703', 'learning_rate': '0.0001586', 'entropy': '0.7511', 'num_tokens': '2.178e+07', 'mean_token_accuracy': '0.8121', 'epoch': '0.03564'}
{'loss': '0.6879', 'grad_norm': '0.3902', 'learning_rate': '0.0001586', 'entropy': '0.7248', 'num_tokens': '2.178e+07', 'mean_token_accuracy': '0.8138', 'epoch': '0.03565'}
{'loss': '1.179', 'grad_norm': '0.4851', 'learning_rate': '0.0001586', 'entropy': '1.194', 'num_tokens': '2.179e+07', 'mean_token_accuracy': '0.7177', 'epoch': '0.03566'}
{'loss': '1.224', 'grad_norm': '0.4575', 'learning_rate': '0.0001586', 'entropy': '1.136', 'num_tokens': '2.179e+07', 'mean_token_accuracy': '0.7326', 'epoch': '0.03566'}
{'loss': '1.161', 'grad_norm': '0.3808', 'learning_rate': '0.0001586', 'entropy': '1.113', 'num_tokens': '2.18e+07', 'mean_token_accuracy': '0.725', 'epoch': '0.03567'}
{'loss': '1.518', 'grad_norm': '0.385', 'learning_rate': '0.0001585', 'entropy': '1.247', 'num_tokens': '2.181e+07', 'mean_token_accuracy': '0.7215', 'epoch': '0.03568'}
{'loss': '0.6268', 'grad_norm': '0.3324', 'learning_rate': '0.0001585', 'entropy': '0.643', 'num_tokens': '2.181e+07', 'mean_token_accuracy': '0.8451', 'epoch': '0.03569'}
{'loss': '0.7317', 'grad_norm': '0.3595', 'learning_rate': '0.0001585', 'entropy': '0.7177', 'num_tokens': '2.182e+07', 'mean_token_accuracy': '0.8226', 'epoch': '0.0357'}
{'loss': '0.6967', 'grad_norm': '0.3001', 'learning_rate': '0.0001585', 'entropy': '0.7714', 'num_tokens': '2.182e+07', 'mean_token_accuracy': '0.8034', 'epoch': '0.03571'}
{'loss': '0.8519', 'grad_norm': '0.4256', 'learning_rate': '0.0001584', 'entropy': '0.7881', 'num_tokens': '2.183e+07', 'mean_token_accuracy': '0.8029', 'epoch': '0.03572'}
{'loss': '1.096', 'grad_norm': '0.4384', 'learning_rate': '0.0001584', 'entropy': '1.133', 'num_tokens': '2.183e+07', 'mean_token_accuracy': '0.7331', 'epoch': '0.03573'}
{'loss': '0.86', 'grad_norm': '0.4379', 'learning_rate': '0.0001584', 'entropy': '0.8987', 'num_tokens': '2.184e+07', 'mean_token_accuracy': '0.7794', 'epoch': '0.03574'}
{'loss': '0.7447', 'grad_norm': '0.4351', 'learning_rate': '0.0001584', 'entropy': '0.8152', 'num_tokens': '2.184e+07', 'mean_token_accuracy': '0.7954', 'epoch': '0.03574'}
{'loss': '0.7397', 'grad_norm': '0.4802', 'learning_rate': '0.0001584', 'entropy': '0.783', 'num_tokens': '2.185e+07', 'mean_token_accuracy': '0.7903', 'epoch': '0.03575'}
{'loss': '1.02', 'grad_norm': '0.3835', 'learning_rate': '0.0001583', 'entropy': '1.002', 'num_tokens': '2.185e+07', 'mean_token_accuracy': '0.7518', 'epoch': '0.03576'}
{'loss': '1.174', 'grad_norm': '0.3742', 'learning_rate': '0.0001583', 'entropy': '1.247', 'num_tokens': '2.186e+07', 'mean_token_accuracy': '0.7101', 'epoch': '0.03577'}
{'loss': '0.8287', 'grad_norm': '0.6847', 'learning_rate': '0.0001583', 'entropy': '0.8208', 'num_tokens': '2.186e+07', 'mean_token_accuracy': '0.7823', 'epoch': '0.03578'}
{'loss': '1.13', 'grad_norm': '0.4232', 'learning_rate': '0.0001583', 'entropy': '1.083', 'num_tokens': '2.187e+07', 'mean_token_accuracy': '0.7372', 'epoch': '0.03579'}
{'loss': '0.6887', 'grad_norm': '0.3842', 'learning_rate': '0.0001582', 'entropy': '0.7177', 'num_tokens': '2.187e+07', 'mean_token_accuracy': '0.8234', 'epoch': '0.0358'}

 34%|███▎      | 4046/12000 [2:24:27<3:49:02,  1.73s/it]
 34%|███▎      | 4047/12000 [2:24:29<3:31:00,  1.59s/it]
                                                        

 34%|███▎      | 4047/12000 [2:24:29<3:31:00,  1.59s/it]
 34%|███▎      | 4048/12000 [2:24:30<3:33:23,  1.61s/it]
                                                        

 34%|███▎      | 4048/12000 [2:24:30<3:33:23,  1.61s/it]
 34%|███▎      | 4049/12000 [2:24:32<3:37:35,  1.64s/it]
                                                        

 34%|███▎      | 4049/12000 [2:24:32<3:37:35,  1.64s/it]
 34%|███▍      | 4050/12000 [2:24:34<3:33:17,  1.61s/it]
                                                        

 34%|███▍      | 4050/12000 [2:24:34<3:33:17,  1.61s/it]
 34%|███▍      | 4051/12000 [2:24:36<4:26:17,  2.01s/it]
                                                        

 34%|███▍      | 4051/12000 [2:24:36<4:26:17,  2.01s/it]
 34%|███▍      | 4052/12000 [2:24:40<5:26:25,  2.46s/it]
                                                        

 34%|███▍      | 4052/12000 [2:24:40<5:26:25,  2.46s/it]
 34%|███▍      | 4053/12000 [2:24:42<4:52:32,  2.21s/it]
                                                        

 34%|███▍      | 4053/12000 [2:24:42<4:52:32,  2.21s/it]
 34%|███▍      | 4054/12000 [2:24:43<4:28:20,  2.03s/it]
                                                        

 34%|███▍      | 4054/12000 [2:24:43<4:28:20,  2.03s/it]
 34%|███▍      | 4055/12000 [2:24:45<4:05:57,  1.86s/it]
                                                        

 34%|███▍      | 4055/12000 [2:24:45<4:05:57,  1.86s/it]
 34%|███▍      | 4056/12000 [2:24:46<3:48:04,  1.72s/it]
                                                        

 34%|███▍      | 4056/12000 [2:24:46<3:48:04,  1.72s/it]
 34%|███▍      | 4057/12000 [2:24:48<3:39:16,  1.66s/it]
                                                        

 34%|███▍      | 4057/12000 [2:24:48<3:39:16,  1.66s/it]
 34%|███▍      | 4058/12000 [2:24:49<3:34:11,  1.62s/it]
                                                        

 34%|███▍      | 4058/12000 [2:24:49<3:34:11,  1.62s/it]
 34%|███▍      | 4059/12000 [2:24:51<3:36:09,  1.63s/it]
                                                        

 34%|███▍      | 4059/12000 [2:24:51<3:36:09,  1.63s/it]
 34%|███▍      | 4060/12000 [2:24:53<4:15:24,  1.93s/it]
                                                        

 34%|███▍      | 4060/12000 [2:24:53<4:15:24,  1.93s/it]
 34%|███▍      | 4061/12000 [2:24:56<4:29:12,  2.03s/it]
                                                        

 34%|███▍      | 4061/12000 [2:24:56<4:29:12,  2.03s/it]
 34%|███▍      | 4062/12000 [2:24:57<4:05:09,  1.85s/it]
                                                        

 34%|███▍      | 4062/12000 [2:24:57<4:05:09,  1.85s/it]
 34%|███▍      | 4063/12000 [2:24:59<3:55:33,  1.78s/it]
                                                        

 34%|███▍      | 4063/12000 [2:24:59<3:55:33,  1.78s/it]
 34%|███▍      | 4064/12000 [2:25:02<4:59:12,  2.26s/it]
                                                        

 34%|███▍      | 4064/12000 [2:25:02<4:59:12,  2.26s/it]
 34%|███▍      | 4065/12000 [2:25:04<4:38:56,  2.11s/it]
                                                        

 34%|███▍      | 4065/12000 [2:25:04<4:38:56,  2.11s/it]
 34%|███▍      | 4066/12000 [2:25:05<4:12:42,  1.91s/it]
                                                        

 34%|███▍      | 4066/12000 [2:25:05<4:12:42,  1.91s/it]
 34%|███▍      | 4067/12000 [2:25:07<3:55:07,  1.78s/it]
                                                        

 34%|███▍      | 4067/12000 [2:25:07<3:55:07,  1.78s/it]
 34%|███▍      | 4068/12000 [2:25:08<3:34:57,  1.63s/it]
                                                        

 34%|███▍      | 4068/12000 [2:25:08<3:34:57,  1.63s/it]
 34%|███▍      | 4069/12000 [2:25:09<3:22:47,  1.53s/it]
                                                        

 34%|███▍      | 4069/12000 [2:25:09<3:22:47,  1.53s/it]
 34%|███▍      | 4070/12000 [2:25:12<3:47:58,  1.72s/it]
                                                        

 34%|███▍      | 4070/12000 [2:25:12<3:47:58,  1.72s/it]
 34%|███▍      | 4071/12000 [2:25:13<3:38:31,  1.65s/it]
                                                        

 34%|███▍      | 4071/12000 [2:25:13<3:38:31,  1.65s/it]
 34%|███▍      | 4072/12000 [2:25:14<3:26:41,  1.56s/it]
                                                        

 34%|███▍      | 4072/12000 [2:25:14<3:26:41,  1.56s/it]
 34%|███▍      | 4073/12000 [2:25:19<5:35:36,  2.54s/it]
                                                        

 34%|███▍      | 4073/12000 [2:25:19<5:35:36,  2.54s/it]
 34%|███▍      | 4074/12000 [2:25:21<5:01:19,  2.28s/it]
                                                        

 34%|███▍      | 4074/12000 [2:25:21<5:01:19,  2.28s/it]
 34%|███▍      | 4075/12000 [2:25:22<4:27:52,  2.03s/it]
                                                        

 34%|███▍      | 4075/12000 [2:25:22<4:27:52,  2.03s/it]
 34%|███▍      | 4076/12000 [2:25:26<5:17:30,  2.40s/it]
                                                        

 34%|███▍      | 4076/12000 [2:25:26<5:17:30,  2.40s/it]
 34%|███▍      | 4077/12000 [2:25:27<4:44:30,  2.15s/it]
                                                        

 34%|███▍      | 4077/12000 [2:25:27<4:44:30,  2.15s/it]
 34%|███▍      | 4078/12000 [2:25:31<5:45:44,  2.62s/it]
                                                        

 34%|███▍      | 4078/12000 [2:25:31<5:45:44,  2.62s/it]
 34%|███▍      | 4079/12000 [2:25:32<5:01:58,  2.29s/it]
                                                        

 34%|███▍      | 4079/12000 [2:25:32<5:01:58,  2.29s/it]
 34%|███▍      | 4080/12000 [2:25:35<5:21:56,  2.44s/it]
                                                        

 34%|███▍      | 4080/12000 [2:25:35<5:21:56,  2.44s/it]
 34%|███▍      | 4081/12000 [2:25:40<6:44:11,  3.06s/it]
                                                        

 34%|███▍      | 4081/12000 [2:25:40<6:44:11,  3.06s/it]
 34%|███▍      | 4082/12000 [2:25:41<5:51:48,  2.67s/it]
                                                        

 34%|███▍      | 4082/12000 [2:25:41<5:51:48,  2.67s/it]
 34%|███▍      | 4083/12000 [2:25:43<5:12:20,  2.37s/it]
                                                        

 34%|███▍      | 4083/12000 [2:25:43<5:12:20,  2.37s/it]
 34%|███▍      | 4084/12000 [2:25:45<4:37:11,  2.10s/it]
                                                        

 34%|███▍      | 4084/12000 [2:25:45<4:37:11,  2.10s/it]
 34%|███▍      | 4085/12000 [2:25:49<5:48:34,  2.64s/it]
                                                        

 34%|███▍      | 4085/12000 [2:25:49<5:48:34,  2.64s/it]
 34%|███▍      | 4086/12000 [2:25:52<6:41:49,  3.05s/it]
                                                        

 34%|███▍      | 4086/12000 [2:25:52<6:41:49,  3.05s/it]
 34%|███▍      | 4087/12000 [2:25:55<6:01:03,  2.74s/it]
                                                        

 34%|███▍      | 4087/12000 [2:25:55<6:01:03,  2.74s/it]
 34%|███▍      | 4088/12000 [2:25:58<6:14:27,  2.84s/it]
                                                        

 34%|███▍      | 4088/12000 [2:25:58<6:14:27,  2.84s/it]
 34%|███▍      | 4089/12000 [2:25:59<5:16:39,  2.40s/it]
                                                        

 34%|███▍      | 4089/12000 [2:25:59<5:16:39,  2.40s/it]
 34%|███▍      | 4090/12000 [2:26:01<5:02:48,  2.30s/it]
                                                        

 34%|███▍      | 4090/12000 [2:26:01<5:02:48,  2.30s/it]
 34%|███▍      | 4091/12000 [2:26:05<6:24:48,  2.92s/it]
                                                        

 34%|███▍      | 4091/12000 [2:26:05<6:24:48,  2.92s/it]
 34%|███▍      | 4092/12000 [2:26:08<6:13:06,  2.83s/it]
                                                        

 34%|███▍      | 4092/12000 [2:26:08<6:13:06,  2.83s/it]
 34%|███▍      | 4093/12000 [2:26:09<5:15:16,  2.39s/it]
                                                        
{'loss': '0.9038', 'grad_norm': '0.4179', 'learning_rate': '0.0001582', 'entropy': '0.9192', 'num_tokens': '2.187e+07', 'mean_token_accuracy': '0.7728', 'epoch': '0.03581'}
{'loss': '0.8384', 'grad_norm': '0.4182', 'learning_rate': '0.0001582', 'entropy': '0.8953', 'num_tokens': '2.188e+07', 'mean_token_accuracy': '0.779', 'epoch': '0.03582'}
{'loss': '0.793', 'grad_norm': '0.4155', 'learning_rate': '0.0001582', 'entropy': '0.7332', 'num_tokens': '2.188e+07', 'mean_token_accuracy': '0.8055', 'epoch': '0.03582'}
{'loss': '1.002', 'grad_norm': '0.4396', 'learning_rate': '0.0001582', 'entropy': '0.9955', 'num_tokens': '2.189e+07', 'mean_token_accuracy': '0.7643', 'epoch': '0.03583'}
{'loss': '0.8047', 'grad_norm': '0.4157', 'learning_rate': '0.0001581', 'entropy': '0.7841', 'num_tokens': '2.189e+07', 'mean_token_accuracy': '0.7992', 'epoch': '0.03584'}
{'loss': '1.044', 'grad_norm': '0.4979', 'learning_rate': '0.0001581', 'entropy': '1.029', 'num_tokens': '2.19e+07', 'mean_token_accuracy': '0.7444', 'epoch': '0.03585'}
{'loss': '1.161', 'grad_norm': '0.3722', 'learning_rate': '0.0001581', 'entropy': '1.059', 'num_tokens': '2.191e+07', 'mean_token_accuracy': '0.7331', 'epoch': '0.03586'}
{'loss': '0.9961', 'grad_norm': '0.4163', 'learning_rate': '0.0001581', 'entropy': '0.9715', 'num_tokens': '2.191e+07', 'mean_token_accuracy': '0.7674', 'epoch': '0.03587'}
{'loss': '0.8702', 'grad_norm': '0.4005', 'learning_rate': '0.000158', 'entropy': '0.8367', 'num_tokens': '2.192e+07', 'mean_token_accuracy': '0.7923', 'epoch': '0.03588'}
{'loss': '0.7834', 'grad_norm': '0.407', 'learning_rate': '0.000158', 'entropy': '0.8348', 'num_tokens': '2.192e+07', 'mean_token_accuracy': '0.7905', 'epoch': '0.03589'}
{'loss': '0.6861', 'grad_norm': '0.3378', 'learning_rate': '0.000158', 'entropy': '0.7518', 'num_tokens': '2.192e+07', 'mean_token_accuracy': '0.8072', 'epoch': '0.03589'}
{'loss': '0.6145', 'grad_norm': '0.3305', 'learning_rate': '0.000158', 'entropy': '0.6131', 'num_tokens': '2.193e+07', 'mean_token_accuracy': '0.8418', 'epoch': '0.0359'}
{'loss': '0.8039', 'grad_norm': '0.3776', 'learning_rate': '0.000158', 'entropy': '0.833', 'num_tokens': '2.193e+07', 'mean_token_accuracy': '0.7841', 'epoch': '0.03591'}
{'loss': '1.404', 'grad_norm': '0.4848', 'learning_rate': '0.0001579', 'entropy': '1.347', 'num_tokens': '2.194e+07', 'mean_token_accuracy': '0.6667', 'epoch': '0.03592'}
{'loss': '0.9484', 'grad_norm': '0.4673', 'learning_rate': '0.0001579', 'entropy': '0.8771', 'num_tokens': '2.194e+07', 'mean_token_accuracy': '0.7832', 'epoch': '0.03593'}
{'loss': '1.374', 'grad_norm': '0.4112', 'learning_rate': '0.0001579', 'entropy': '1.293', 'num_tokens': '2.195e+07', 'mean_token_accuracy': '0.7027', 'epoch': '0.03594'}
{'loss': '0.7354', 'grad_norm': '0.3604', 'learning_rate': '0.0001579', 'entropy': '0.8386', 'num_tokens': '2.195e+07', 'mean_token_accuracy': '0.794', 'epoch': '0.03595'}
{'loss': '0.8186', 'grad_norm': '0.437', 'learning_rate': '0.0001578', 'entropy': '0.8415', 'num_tokens': '2.196e+07', 'mean_token_accuracy': '0.7958', 'epoch': '0.03596'}
{'loss': '0.9722', 'grad_norm': '0.4377', 'learning_rate': '0.0001578', 'entropy': '0.9278', 'num_tokens': '2.196e+07', 'mean_token_accuracy': '0.768', 'epoch': '0.03597'}
{'loss': '0.8738', 'grad_norm': '0.3739', 'learning_rate': '0.0001578', 'entropy': '0.833', 'num_tokens': '2.197e+07', 'mean_token_accuracy': '0.7854', 'epoch': '0.03597'}
{'loss': '0.7555', 'grad_norm': '0.4294', 'learning_rate': '0.0001578', 'entropy': '0.8259', 'num_tokens': '2.197e+07', 'mean_token_accuracy': '0.7851', 'epoch': '0.03598'}
{'loss': '0.7036', 'grad_norm': '0.3661', 'learning_rate': '0.0001577', 'entropy': '0.7396', 'num_tokens': '2.198e+07', 'mean_token_accuracy': '0.8013', 'epoch': '0.03599'}
{'loss': '0.6227', 'grad_norm': '0.2713', 'learning_rate': '0.0001577', 'entropy': '0.6715', 'num_tokens': '2.198e+07', 'mean_token_accuracy': '0.8235', 'epoch': '0.036'}
{'loss': '0.7534', 'grad_norm': '0.3705', 'learning_rate': '0.0001577', 'entropy': '0.8036', 'num_tokens': '2.198e+07', 'mean_token_accuracy': '0.804', 'epoch': '0.03601'}
{'loss': '0.7371', 'grad_norm': '0.3449', 'learning_rate': '0.0001577', 'entropy': '0.7177', 'num_tokens': '2.199e+07', 'mean_token_accuracy': '0.8194', 'epoch': '0.03602'}
{'loss': '0.9625', 'grad_norm': '0.4146', 'learning_rate': '0.0001577', 'entropy': '1.048', 'num_tokens': '2.199e+07', 'mean_token_accuracy': '0.7503', 'epoch': '0.03603'}
{'loss': '0.9983', 'grad_norm': '0.4379', 'learning_rate': '0.0001576', 'entropy': '0.9647', 'num_tokens': '2.2e+07', 'mean_token_accuracy': '0.7542', 'epoch': '0.03604'}
{'loss': '1.107', 'grad_norm': '0.3306', 'learning_rate': '0.0001576', 'entropy': '0.809', 'num_tokens': '2.201e+07', 'mean_token_accuracy': '0.8007', 'epoch': '0.03605'}
{'loss': '0.8272', 'grad_norm': '0.4345', 'learning_rate': '0.0001576', 'entropy': '0.8458', 'num_tokens': '2.201e+07', 'mean_token_accuracy': '0.7812', 'epoch': '0.03605'}
{'loss': '1.084', 'grad_norm': '0.3824', 'learning_rate': '0.0001576', 'entropy': '1.05', 'num_tokens': '2.202e+07', 'mean_token_accuracy': '0.7458', 'epoch': '0.03606'}
{'loss': '0.9973', 'grad_norm': '0.3924', 'learning_rate': '0.0001575', 'entropy': '1.015', 'num_tokens': '2.203e+07', 'mean_token_accuracy': '0.7527', 'epoch': '0.03607'}
{'loss': '1.049', 'grad_norm': '0.3712', 'learning_rate': '0.0001575', 'entropy': '1.006', 'num_tokens': '2.203e+07', 'mean_token_accuracy': '0.7638', 'epoch': '0.03608'}
{'loss': '1.039', 'grad_norm': '0.2897', 'learning_rate': '0.0001575', 'entropy': '1.049', 'num_tokens': '2.204e+07', 'mean_token_accuracy': '0.7421', 'epoch': '0.03609'}
{'loss': '0.688', 'grad_norm': '0.371', 'learning_rate': '0.0001575', 'entropy': '0.6947', 'num_tokens': '2.205e+07', 'mean_token_accuracy': '0.8184', 'epoch': '0.0361'}
{'loss': '1.058', 'grad_norm': '0.3637', 'learning_rate': '0.0001575', 'entropy': '1.061', 'num_tokens': '2.205e+07', 'mean_token_accuracy': '0.7374', 'epoch': '0.03611'}
{'loss': '1.515', 'grad_norm': '0.443', 'learning_rate': '0.0001574', 'entropy': '1.301', 'num_tokens': '2.206e+07', 'mean_token_accuracy': '0.705', 'epoch': '0.03612'}
{'loss': '1.098', 'grad_norm': '0.3839', 'learning_rate': '0.0001574', 'entropy': '1.14', 'num_tokens': '2.207e+07', 'mean_token_accuracy': '0.7211', 'epoch': '0.03613'}
{'loss': '0.7613', 'grad_norm': '0.3538', 'learning_rate': '0.0001574', 'entropy': '0.7566', 'num_tokens': '2.207e+07', 'mean_token_accuracy': '0.8088', 'epoch': '0.03613'}
{'loss': '0.7493', 'grad_norm': '0.4165', 'learning_rate': '0.0001574', 'entropy': '0.7606', 'num_tokens': '2.208e+07', 'mean_token_accuracy': '0.8142', 'epoch': '0.03614'}
{'loss': '0.7208', 'grad_norm': '0.3328', 'learning_rate': '0.0001573', 'entropy': '0.7735', 'num_tokens': '2.208e+07', 'mean_token_accuracy': '0.8194', 'epoch': '0.03615'}
{'loss': '1.112', 'grad_norm': '0.4241', 'learning_rate': '0.0001573', 'entropy': '0.9084', 'num_tokens': '2.209e+07', 'mean_token_accuracy': '0.7793', 'epoch': '0.03616'}
{'loss': '1.049', 'grad_norm': '0.4985', 'learning_rate': '0.0001573', 'entropy': '0.9417', 'num_tokens': '2.21e+07', 'mean_token_accuracy': '0.7618', 'epoch': '0.03617'}
{'loss': '1.001', 'grad_norm': '0.4106', 'learning_rate': '0.0001573', 'entropy': '1.034', 'num_tokens': '2.21e+07', 'mean_token_accuracy': '0.7533', 'epoch': '0.03618'}
{'loss': '0.9622', 'grad_norm': '0.3978', 'learning_rate': '0.0001573', 'entropy': '0.9562', 'num_tokens': '2.211e+07', 'mean_token_accuracy': '0.7624', 'epoch': '0.03619'}
{'loss': '1.081', 'grad_norm': '0.4083', 'learning_rate': '0.0001572', 'entropy': '1.118', 'num_tokens': '2.211e+07', 'mean_token_accuracy': '0.7433', 'epoch': '0.0362'}
{'loss': '1.054', 'grad_norm': '0.3528', 'learning_rate': '0.0001572', 'entropy': '0.9782', 'num_tokens': '2.212e+07', 'mean_token_accuracy': '0.7673', 'epoch': '0.0362'}
{'loss': '0.9183', 'grad_norm': '0.3234', 'learning_rate': '0.0001572', 'entropy': '0.8593', 'num_tokens': '2.213e+07', 'mean_token_accuracy': '0.7978', 'epoch': '0.03621'}

 34%|███▍      | 4093/12000 [2:26:09<5:15:16,  2.39s/it]
 34%|███▍      | 4094/12000 [2:26:11<4:56:52,  2.25s/it]
                                                        

 34%|███▍      | 4094/12000 [2:26:11<4:56:52,  2.25s/it]
 34%|███▍      | 4095/12000 [2:26:14<5:02:52,  2.30s/it]
                                                        

 34%|███▍      | 4095/12000 [2:26:14<5:02:52,  2.30s/it]
 34%|███▍      | 4096/12000 [2:26:16<5:07:19,  2.33s/it]
                                                        

 34%|███▍      | 4096/12000 [2:26:16<5:07:19,  2.33s/it]
 34%|███▍      | 4097/12000 [2:26:18<5:04:18,  2.31s/it]
                                                        

 34%|███▍      | 4097/12000 [2:26:18<5:04:18,  2.31s/it]
 34%|███▍      | 4098/12000 [2:26:20<4:46:05,  2.17s/it]
                                                        

 34%|███▍      | 4098/12000 [2:26:20<4:46:05,  2.17s/it]
 34%|███▍      | 4099/12000 [2:26:22<4:20:31,  1.98s/it]
                                                        

 34%|███▍      | 4099/12000 [2:26:22<4:20:31,  1.98s/it]
 34%|███▍      | 4100/12000 [2:26:23<3:56:34,  1.80s/it]
                                                        

 34%|███▍      | 4100/12000 [2:26:23<3:56:34,  1.80s/it]
 34%|███▍      | 4101/12000 [2:26:27<5:12:58,  2.38s/it]
                                                        

 34%|███▍      | 4101/12000 [2:26:27<5:12:58,  2.38s/it]
 34%|███▍      | 4102/12000 [2:26:28<4:41:09,  2.14s/it]
                                                        

 34%|███▍      | 4102/12000 [2:26:28<4:41:09,  2.14s/it]
 34%|███▍      | 4103/12000 [2:26:30<4:19:16,  1.97s/it]
                                                        

 34%|███▍      | 4103/12000 [2:26:30<4:19:16,  1.97s/it]
 34%|███▍      | 4104/12000 [2:26:33<4:49:18,  2.20s/it]
                                                        

 34%|███▍      | 4104/12000 [2:26:33<4:49:18,  2.20s/it]
 34%|███▍      | 4105/12000 [2:26:36<5:13:51,  2.39s/it]
                                                        

 34%|███▍      | 4105/12000 [2:26:36<5:13:51,  2.39s/it]
 34%|███▍      | 4106/12000 [2:26:37<4:39:22,  2.12s/it]
                                                        

 34%|███▍      | 4106/12000 [2:26:37<4:39:22,  2.12s/it]
 34%|███▍      | 4107/12000 [2:26:39<4:12:41,  1.92s/it]
                                                        

 34%|███▍      | 4107/12000 [2:26:39<4:12:41,  1.92s/it]
 34%|███▍      | 4108/12000 [2:26:42<5:14:46,  2.39s/it]
                                                        

 34%|███▍      | 4108/12000 [2:26:42<5:14:46,  2.39s/it]
 34%|███▍      | 4109/12000 [2:26:43<4:34:42,  2.09s/it]
                                                        

 34%|███▍      | 4109/12000 [2:26:43<4:34:42,  2.09s/it]
 34%|███▍      | 4110/12000 [2:26:48<6:09:39,  2.81s/it]
                                                        

 34%|███▍      | 4110/12000 [2:26:48<6:09:39,  2.81s/it]
 34%|███▍      | 4111/12000 [2:26:50<5:55:14,  2.70s/it]
                                                        

 34%|███▍      | 4111/12000 [2:26:50<5:55:14,  2.70s/it]
 34%|███▍      | 4112/12000 [2:26:53<6:04:23,  2.77s/it]
                                                        

 34%|███▍      | 4112/12000 [2:26:53<6:04:23,  2.77s/it]
 34%|███▍      | 4113/12000 [2:26:55<5:15:08,  2.40s/it]
                                                        

 34%|███▍      | 4113/12000 [2:26:55<5:15:08,  2.40s/it]
 34%|███▍      | 4114/12000 [2:26:56<4:35:06,  2.09s/it]
                                                        

 34%|███▍      | 4114/12000 [2:26:56<4:35:06,  2.09s/it]
 34%|███▍      | 4115/12000 [2:26:58<4:36:39,  2.11s/it]
                                                        

 34%|███▍      | 4115/12000 [2:26:58<4:36:39,  2.11s/it]
 34%|███▍      | 4116/12000 [2:27:00<4:09:45,  1.90s/it]
                                                        

 34%|███▍      | 4116/12000 [2:27:00<4:09:45,  1.90s/it]
 34%|███▍      | 4117/12000 [2:27:02<4:19:39,  1.98s/it]
                                                        

 34%|███▍      | 4117/12000 [2:27:02<4:19:39,  1.98s/it]
 34%|███▍      | 4118/12000 [2:27:03<3:51:52,  1.77s/it]
                                                        

 34%|███▍      | 4118/12000 [2:27:03<3:51:52,  1.77s/it]
 34%|███▍      | 4119/12000 [2:27:05<4:09:49,  1.90s/it]
                                                        

 34%|███▍      | 4119/12000 [2:27:05<4:09:49,  1.90s/it]
 34%|███▍      | 4120/12000 [2:27:07<3:56:26,  1.80s/it]
                                                        

 34%|███▍      | 4120/12000 [2:27:07<3:56:26,  1.80s/it]
 34%|███▍      | 4121/12000 [2:27:08<3:41:36,  1.69s/it]
                                                        

 34%|███▍      | 4121/12000 [2:27:08<3:41:36,  1.69s/it]
 34%|███▍      | 4122/12000 [2:27:11<4:01:37,  1.84s/it]
                                                        

 34%|███▍      | 4122/12000 [2:27:11<4:01:37,  1.84s/it]
 34%|███▍      | 4123/12000 [2:27:13<4:16:11,  1.95s/it]
                                                        

 34%|███▍      | 4123/12000 [2:27:13<4:16:11,  1.95s/it]
 34%|███▍      | 4124/12000 [2:27:14<3:59:47,  1.83s/it]
                                                        

 34%|███▍      | 4124/12000 [2:27:14<3:59:47,  1.83s/it]
 34%|███▍      | 4125/12000 [2:27:16<3:39:58,  1.68s/it]
                                                        

 34%|███▍      | 4125/12000 [2:27:16<3:39:58,  1.68s/it]
 34%|███▍      | 4126/12000 [2:27:18<4:16:17,  1.95s/it]
                                                        

 34%|███▍      | 4126/12000 [2:27:18<4:16:17,  1.95s/it]
 34%|███▍      | 4127/12000 [2:27:21<5:01:15,  2.30s/it]
                                                        

 34%|███▍      | 4127/12000 [2:27:21<5:01:15,  2.30s/it]
 34%|███▍      | 4128/12000 [2:27:24<5:01:03,  2.29s/it]
                                                        

 34%|███▍      | 4128/12000 [2:27:24<5:01:03,  2.29s/it]
 34%|███▍      | 4129/12000 [2:27:28<6:14:02,  2.85s/it]
                                                        

 34%|███▍      | 4129/12000 [2:27:28<6:14:02,  2.85s/it]
 34%|███▍      | 4130/12000 [2:27:29<5:12:07,  2.38s/it]
                                                        

 34%|███▍      | 4130/12000 [2:27:29<5:12:07,  2.38s/it]
 34%|███▍      | 4131/12000 [2:27:32<5:39:47,  2.59s/it]
                                                        

 34%|███▍      | 4131/12000 [2:27:32<5:39:47,  2.59s/it]
 34%|███▍      | 4132/12000 [2:27:34<5:18:37,  2.43s/it]
                                                        

 34%|███▍      | 4132/12000 [2:27:34<5:18:37,  2.43s/it]
 34%|███▍      | 4133/12000 [2:27:36<4:50:00,  2.21s/it]
                                                        

 34%|███▍      | 4133/12000 [2:27:36<4:50:00,  2.21s/it]
 34%|███▍      | 4134/12000 [2:27:38<4:46:39,  2.19s/it]
                                                        

 34%|███▍      | 4134/12000 [2:27:38<4:46:39,  2.19s/it]
 34%|███▍      | 4135/12000 [2:27:40<4:25:49,  2.03s/it]
                                                        

 34%|███▍      | 4135/12000 [2:27:40<4:25:49,  2.03s/it]
 34%|███▍      | 4136/12000 [2:27:41<3:52:44,  1.78s/it]
                                                        

 34%|███▍      | 4136/12000 [2:27:41<3:52:44,  1.78s/it]
 34%|███▍      | 4137/12000 [2:27:43<3:56:00,  1.80s/it]
                                                        

 34%|███▍      | 4137/12000 [2:27:43<3:56:00,  1.80s/it]
 34%|███▍      | 4138/12000 [2:27:45<4:24:46,  2.02s/it]
                                                        

 34%|███▍      | 4138/12000 [2:27:45<4:24:46,  2.02s/it]
 34%|███▍      | 4139/12000 [2:27:47<3:53:44,  1.78s/it]
                                                        

 34%|███▍      | 4139/12000 [2:27:47<3:53:44,  1.78s/it]
 34%|███▍      | 4140/12000 [2:27:48<3:46:11,  1.73s/it]
                                                        
{'loss': '1.001', 'grad_norm': '0.4088', 'learning_rate': '0.0001572', 'entropy': '0.9594', 'num_tokens': '2.213e+07', 'mean_token_accuracy': '0.758', 'epoch': '0.03622'}
{'loss': '0.5876', 'grad_norm': '0.2846', 'learning_rate': '0.0001571', 'entropy': '0.675', 'num_tokens': '2.214e+07', 'mean_token_accuracy': '0.8341', 'epoch': '0.03623'}
{'loss': '0.8664', 'grad_norm': '0.4146', 'learning_rate': '0.0001571', 'entropy': '0.908', 'num_tokens': '2.214e+07', 'mean_token_accuracy': '0.7775', 'epoch': '0.03624'}
{'loss': '0.7255', 'grad_norm': '0.3298', 'learning_rate': '0.0001571', 'entropy': '0.8118', 'num_tokens': '2.215e+07', 'mean_token_accuracy': '0.8028', 'epoch': '0.03625'}
{'loss': '0.779', 'grad_norm': '0.3427', 'learning_rate': '0.0001571', 'entropy': '0.7823', 'num_tokens': '2.216e+07', 'mean_token_accuracy': '0.814', 'epoch': '0.03626'}
{'loss': '1.022', 'grad_norm': '0.4481', 'learning_rate': '0.000157', 'entropy': '0.9178', 'num_tokens': '2.216e+07', 'mean_token_accuracy': '0.7734', 'epoch': '0.03627'}
{'loss': '0.7767', 'grad_norm': '0.3112', 'learning_rate': '0.000157', 'entropy': '0.7955', 'num_tokens': '2.217e+07', 'mean_token_accuracy': '0.8045', 'epoch': '0.03628'}
{'loss': '0.6687', 'grad_norm': '0.3741', 'learning_rate': '0.000157', 'entropy': '0.6827', 'num_tokens': '2.217e+07', 'mean_token_accuracy': '0.8199', 'epoch': '0.03628'}
{'loss': '1.084', 'grad_norm': '0.7456', 'learning_rate': '0.000157', 'entropy': '1.095', 'num_tokens': '2.218e+07', 'mean_token_accuracy': '0.7453', 'epoch': '0.03629'}
{'loss': '0.8097', 'grad_norm': '0.3694', 'learning_rate': '0.000157', 'entropy': '0.7606', 'num_tokens': '2.218e+07', 'mean_token_accuracy': '0.8043', 'epoch': '0.0363'}
{'loss': '1.066', 'grad_norm': '0.3624', 'learning_rate': '0.0001569', 'entropy': '1.035', 'num_tokens': '2.219e+07', 'mean_token_accuracy': '0.7356', 'epoch': '0.03631'}
{'loss': '1.011', 'grad_norm': '0.3307', 'learning_rate': '0.0001569', 'entropy': '1.025', 'num_tokens': '2.219e+07', 'mean_token_accuracy': '0.7564', 'epoch': '0.03632'}
{'loss': '1.012', 'grad_norm': '0.3801', 'learning_rate': '0.0001569', 'entropy': '1.025', 'num_tokens': '2.22e+07', 'mean_token_accuracy': '0.7547', 'epoch': '0.03633'}
{'loss': '0.9608', 'grad_norm': '0.3507', 'learning_rate': '0.0001569', 'entropy': '0.902', 'num_tokens': '2.221e+07', 'mean_token_accuracy': '0.7675', 'epoch': '0.03634'}
{'loss': '1.087', 'grad_norm': '0.4315', 'learning_rate': '0.0001568', 'entropy': '1.085', 'num_tokens': '2.221e+07', 'mean_token_accuracy': '0.7374', 'epoch': '0.03635'}
{'loss': '0.6622', 'grad_norm': '0.3552', 'learning_rate': '0.0001568', 'entropy': '0.8304', 'num_tokens': '2.222e+07', 'mean_token_accuracy': '0.7908', 'epoch': '0.03636'}
{'loss': '0.6292', 'grad_norm': '0.3389', 'learning_rate': '0.0001568', 'entropy': '0.6355', 'num_tokens': '2.222e+07', 'mean_token_accuracy': '0.8353', 'epoch': '0.03636'}
{'loss': '0.9415', 'grad_norm': '0.3575', 'learning_rate': '0.0001568', 'entropy': '0.9117', 'num_tokens': '2.223e+07', 'mean_token_accuracy': '0.767', 'epoch': '0.03637'}
{'loss': '0.9201', 'grad_norm': '0.3704', 'learning_rate': '0.0001568', 'entropy': '0.9574', 'num_tokens': '2.223e+07', 'mean_token_accuracy': '0.7703', 'epoch': '0.03638'}
{'loss': '0.6629', 'grad_norm': '0.2963', 'learning_rate': '0.0001567', 'entropy': '0.758', 'num_tokens': '2.224e+07', 'mean_token_accuracy': '0.8147', 'epoch': '0.03639'}
{'loss': '1.051', 'grad_norm': '0.3965', 'learning_rate': '0.0001567', 'entropy': '1.059', 'num_tokens': '2.225e+07', 'mean_token_accuracy': '0.7401', 'epoch': '0.0364'}
{'loss': '0.9019', 'grad_norm': '0.3733', 'learning_rate': '0.0001567', 'entropy': '0.9295', 'num_tokens': '2.225e+07', 'mean_token_accuracy': '0.7725', 'epoch': '0.03641'}
{'loss': '1.34', 'grad_norm': '0.5238', 'learning_rate': '0.0001567', 'entropy': '1.095', 'num_tokens': '2.226e+07', 'mean_token_accuracy': '0.7312', 'epoch': '0.03642'}
{'loss': '0.8173', 'grad_norm': '0.3457', 'learning_rate': '0.0001566', 'entropy': '0.8823', 'num_tokens': '2.226e+07', 'mean_token_accuracy': '0.7846', 'epoch': '0.03643'}
{'loss': '0.916', 'grad_norm': '0.3632', 'learning_rate': '0.0001566', 'entropy': '1.052', 'num_tokens': '2.227e+07', 'mean_token_accuracy': '0.7444', 'epoch': '0.03643'}
{'loss': '0.6624', 'grad_norm': '0.2921', 'learning_rate': '0.0001566', 'entropy': '0.6758', 'num_tokens': '2.227e+07', 'mean_token_accuracy': '0.8228', 'epoch': '0.03644'}
{'loss': '0.9098', 'grad_norm': '0.4368', 'learning_rate': '0.0001566', 'entropy': '0.8474', 'num_tokens': '2.228e+07', 'mean_token_accuracy': '0.7971', 'epoch': '0.03645'}
{'loss': '0.8737', 'grad_norm': '0.3747', 'learning_rate': '0.0001565', 'entropy': '0.9208', 'num_tokens': '2.228e+07', 'mean_token_accuracy': '0.7708', 'epoch': '0.03646'}
{'loss': '0.5731', 'grad_norm': '0.3987', 'learning_rate': '0.0001565', 'entropy': '0.5652', 'num_tokens': '2.229e+07', 'mean_token_accuracy': '0.854', 'epoch': '0.03647'}
{'loss': '0.7858', 'grad_norm': '0.3109', 'learning_rate': '0.0001565', 'entropy': '0.8107', 'num_tokens': '2.229e+07', 'mean_token_accuracy': '0.7996', 'epoch': '0.03648'}
{'loss': '0.6718', 'grad_norm': '0.3139', 'learning_rate': '0.0001565', 'entropy': '0.7067', 'num_tokens': '2.23e+07', 'mean_token_accuracy': '0.8198', 'epoch': '0.03649'}
{'loss': '0.6982', 'grad_norm': '0.2867', 'learning_rate': '0.0001565', 'entropy': '0.7084', 'num_tokens': '2.231e+07', 'mean_token_accuracy': '0.8176', 'epoch': '0.0365'}
{'loss': '0.7009', 'grad_norm': '0.3652', 'learning_rate': '0.0001564', 'entropy': '0.785', 'num_tokens': '2.231e+07', 'mean_token_accuracy': '0.8015', 'epoch': '0.03651'}
{'loss': '0.7907', 'grad_norm': '0.3199', 'learning_rate': '0.0001564', 'entropy': '0.9139', 'num_tokens': '2.232e+07', 'mean_token_accuracy': '0.782', 'epoch': '0.03651'}
{'loss': '1.228', 'grad_norm': '0.3988', 'learning_rate': '0.0001564', 'entropy': '1.078', 'num_tokens': '2.233e+07', 'mean_token_accuracy': '0.7393', 'epoch': '0.03652'}
{'loss': '0.7924', 'grad_norm': '0.3955', 'learning_rate': '0.0001564', 'entropy': '0.8513', 'num_tokens': '2.233e+07', 'mean_token_accuracy': '0.7847', 'epoch': '0.03653'}
{'loss': '1.288', 'grad_norm': '0.4748', 'learning_rate': '0.0001563', 'entropy': '1.167', 'num_tokens': '2.234e+07', 'mean_token_accuracy': '0.7221', 'epoch': '0.03654'}
{'loss': '0.839', 'grad_norm': '0.4639', 'learning_rate': '0.0001563', 'entropy': '0.8917', 'num_tokens': '2.234e+07', 'mean_token_accuracy': '0.7725', 'epoch': '0.03655'}
{'loss': '0.899', 'grad_norm': '0.3789', 'learning_rate': '0.0001563', 'entropy': '1.116', 'num_tokens': '2.235e+07', 'mean_token_accuracy': '0.7286', 'epoch': '0.03656'}
{'loss': '1.038', 'grad_norm': '0.4334', 'learning_rate': '0.0001563', 'entropy': '0.9805', 'num_tokens': '2.235e+07', 'mean_token_accuracy': '0.7618', 'epoch': '0.03657'}
{'loss': '0.5261', 'grad_norm': '0.3002', 'learning_rate': '0.0001563', 'entropy': '0.6138', 'num_tokens': '2.236e+07', 'mean_token_accuracy': '0.8467', 'epoch': '0.03658'}
{'loss': '0.7895', 'grad_norm': '0.3571', 'learning_rate': '0.0001562', 'entropy': '0.9637', 'num_tokens': '2.236e+07', 'mean_token_accuracy': '0.7776', 'epoch': '0.03659'}
{'loss': '1.434', 'grad_norm': '0.4664', 'learning_rate': '0.0001562', 'entropy': '1.344', 'num_tokens': '2.237e+07', 'mean_token_accuracy': '0.6659', 'epoch': '0.03659'}
{'loss': '0.8295', 'grad_norm': '0.4659', 'learning_rate': '0.0001562', 'entropy': '0.9026', 'num_tokens': '2.237e+07', 'mean_token_accuracy': '0.7794', 'epoch': '0.0366'}
{'loss': '0.568', 'grad_norm': '0.3745', 'learning_rate': '0.0001562', 'entropy': '0.5624', 'num_tokens': '2.238e+07', 'mean_token_accuracy': '0.8455', 'epoch': '0.03661'}
{'loss': '1.027', 'grad_norm': '0.3979', 'learning_rate': '0.0001561', 'entropy': '0.9533', 'num_tokens': '2.238e+07', 'mean_token_accuracy': '0.7736', 'epoch': '0.03662'}
{'loss': '0.7996', 'grad_norm': '0.3846', 'learning_rate': '0.0001561', 'entropy': '0.8526', 'num_tokens': '2.239e+07', 'mean_token_accuracy': '0.783', 'epoch': '0.03663'}

 34%|███▍      | 4140/12000 [2:27:48<3:46:11,  1.73s/it]
 35%|███▍      | 4141/12000 [2:27:50<3:48:07,  1.74s/it]
                                                        

 35%|███▍      | 4141/12000 [2:27:50<3:48:07,  1.74s/it]
 35%|███▍      | 4142/12000 [2:27:51<3:42:35,  1.70s/it]
                                                        

 35%|███▍      | 4142/12000 [2:27:51<3:42:35,  1.70s/it]
 35%|███▍      | 4143/12000 [2:27:53<3:42:02,  1.70s/it]
                                                        

 35%|███▍      | 4143/12000 [2:27:53<3:42:02,  1.70s/it]
 35%|███▍      | 4144/12000 [2:27:55<3:43:16,  1.71s/it]
                                                        

 35%|███▍      | 4144/12000 [2:27:55<3:43:16,  1.71s/it]
 35%|███▍      | 4145/12000 [2:27:56<3:32:42,  1.62s/it]
                                                        

 35%|███▍      | 4145/12000 [2:27:56<3:32:42,  1.62s/it]
 35%|███▍      | 4146/12000 [2:27:58<3:27:06,  1.58s/it]
                                                        

 35%|███▍      | 4146/12000 [2:27:58<3:27:06,  1.58s/it]
 35%|███▍      | 4147/12000 [2:27:59<3:24:39,  1.56s/it]
                                                        

 35%|███▍      | 4147/12000 [2:27:59<3:24:39,  1.56s/it]
 35%|███▍      | 4148/12000 [2:28:01<3:35:15,  1.64s/it]
                                                        

 35%|███▍      | 4148/12000 [2:28:01<3:35:15,  1.64s/it]
 35%|███▍      | 4149/12000 [2:28:06<5:46:32,  2.65s/it]
                                                        

 35%|███▍      | 4149/12000 [2:28:06<5:46:32,  2.65s/it]
 35%|███▍      | 4150/12000 [2:28:08<5:30:48,  2.53s/it]
                                                        

 35%|███▍      | 4150/12000 [2:28:08<5:30:48,  2.53s/it]
 35%|███▍      | 4151/12000 [2:28:14<7:45:15,  3.56s/it]
                                                        

 35%|███▍      | 4151/12000 [2:28:14<7:45:15,  3.56s/it]
 35%|███▍      | 4152/12000 [2:28:16<6:27:43,  2.96s/it]
                                                        

 35%|███▍      | 4152/12000 [2:28:16<6:27:43,  2.96s/it]
 35%|███▍      | 4153/12000 [2:28:19<6:15:14,  2.87s/it]
                                                        

 35%|███▍      | 4153/12000 [2:28:19<6:15:14,  2.87s/it]
 35%|███▍      | 4154/12000 [2:28:21<5:46:54,  2.65s/it]
                                                        

 35%|███▍      | 4154/12000 [2:28:21<5:46:54,  2.65s/it]
 35%|███▍      | 4155/12000 [2:28:22<5:01:52,  2.31s/it]
                                                        

 35%|███▍      | 4155/12000 [2:28:22<5:01:52,  2.31s/it]
 35%|███▍      | 4156/12000 [2:28:24<4:41:47,  2.16s/it]
                                                        

 35%|███▍      | 4156/12000 [2:28:24<4:41:47,  2.16s/it]
 35%|███▍      | 4157/12000 [2:28:25<4:12:49,  1.93s/it]
                                                        

 35%|███▍      | 4157/12000 [2:28:25<4:12:49,  1.93s/it]
 35%|███▍      | 4158/12000 [2:28:27<4:14:05,  1.94s/it]
                                                        

 35%|███▍      | 4158/12000 [2:28:27<4:14:05,  1.94s/it]
 35%|███▍      | 4159/12000 [2:28:29<3:58:19,  1.82s/it]
                                                        

 35%|███▍      | 4159/12000 [2:28:29<3:58:19,  1.82s/it]
 35%|███▍      | 4160/12000 [2:28:31<4:09:55,  1.91s/it]
                                                        

 35%|███▍      | 4160/12000 [2:28:31<4:09:55,  1.91s/it]
 35%|███▍      | 4161/12000 [2:28:33<3:51:06,  1.77s/it]
                                                        

 35%|███▍      | 4161/12000 [2:28:33<3:51:06,  1.77s/it]
 35%|███▍      | 4162/12000 [2:28:34<3:39:38,  1.68s/it]
                                                        

 35%|███▍      | 4162/12000 [2:28:34<3:39:38,  1.68s/it]
 35%|███▍      | 4163/12000 [2:28:36<4:06:19,  1.89s/it]
                                                        

 35%|███▍      | 4163/12000 [2:28:36<4:06:19,  1.89s/it]
 35%|███▍      | 4164/12000 [2:28:38<4:13:42,  1.94s/it]
                                                        

 35%|███▍      | 4164/12000 [2:28:38<4:13:42,  1.94s/it]
 35%|███▍      | 4165/12000 [2:28:42<4:59:51,  2.30s/it]
                                                        

 35%|███▍      | 4165/12000 [2:28:42<4:59:51,  2.30s/it]
 35%|███▍      | 4166/12000 [2:28:46<6:14:56,  2.87s/it]
                                                        

 35%|███▍      | 4166/12000 [2:28:46<6:14:56,  2.87s/it]
 35%|███▍      | 4167/12000 [2:28:47<5:12:46,  2.40s/it]
                                                        

 35%|███▍      | 4167/12000 [2:28:47<5:12:46,  2.40s/it]
 35%|███▍      | 4168/12000 [2:28:49<4:50:59,  2.23s/it]
                                                        

 35%|███▍      | 4168/12000 [2:28:49<4:50:59,  2.23s/it]
 35%|███▍      | 4169/12000 [2:28:51<4:27:01,  2.05s/it]
                                                        

 35%|███▍      | 4169/12000 [2:28:51<4:27:01,  2.05s/it]
 35%|███▍      | 4170/12000 [2:28:53<4:28:16,  2.06s/it]
                                                        

 35%|███▍      | 4170/12000 [2:28:53<4:28:16,  2.06s/it]
 35%|███▍      | 4171/12000 [2:28:54<4:11:44,  1.93s/it]
                                                        

 35%|███▍      | 4171/12000 [2:28:54<4:11:44,  1.93s/it]
 35%|███▍      | 4172/12000 [2:28:56<4:10:38,  1.92s/it]
                                                        

 35%|███▍      | 4172/12000 [2:28:56<4:10:38,  1.92s/it]
 35%|███▍      | 4173/12000 [2:28:58<4:27:35,  2.05s/it]
                                                        

 35%|███▍      | 4173/12000 [2:28:58<4:27:35,  2.05s/it]
 35%|███▍      | 4174/12000 [2:29:02<5:41:43,  2.62s/it]
                                                        

 35%|███▍      | 4174/12000 [2:29:02<5:41:43,  2.62s/it]
 35%|███▍      | 4175/12000 [2:29:04<5:17:10,  2.43s/it]
                                                        

 35%|███▍      | 4175/12000 [2:29:04<5:17:10,  2.43s/it]
 35%|███▍      | 4176/12000 [2:29:06<4:40:24,  2.15s/it]
                                                        

 35%|███▍      | 4176/12000 [2:29:06<4:40:24,  2.15s/it]
 35%|███▍      | 4177/12000 [2:29:09<5:14:16,  2.41s/it]
                                                        

 35%|███▍      | 4177/12000 [2:29:09<5:14:16,  2.41s/it]
 35%|███▍      | 4178/12000 [2:29:12<5:21:53,  2.47s/it]
                                                        

 35%|███▍      | 4178/12000 [2:29:12<5:21:53,  2.47s/it]
 35%|███▍      | 4179/12000 [2:29:13<4:50:44,  2.23s/it]
                                                        

 35%|███▍      | 4179/12000 [2:29:13<4:50:44,  2.23s/it]
 35%|███▍      | 4180/12000 [2:29:14<4:12:49,  1.94s/it]
                                                        

 35%|███▍      | 4180/12000 [2:29:14<4:12:49,  1.94s/it]
 35%|███▍      | 4181/12000 [2:29:17<4:31:11,  2.08s/it]
                                                        

 35%|███▍      | 4181/12000 [2:29:17<4:31:11,  2.08s/it]
 35%|███▍      | 4182/12000 [2:29:18<4:00:37,  1.85s/it]
                                                        

 35%|███▍      | 4182/12000 [2:29:18<4:00:37,  1.85s/it]
 35%|███▍      | 4183/12000 [2:29:20<3:44:18,  1.72s/it]
                                                        

 35%|███▍      | 4183/12000 [2:29:20<3:44:18,  1.72s/it]
 35%|███▍      | 4184/12000 [2:29:21<3:39:01,  1.68s/it]
                                                        

 35%|███▍      | 4184/12000 [2:29:21<3:39:01,  1.68s/it]
 35%|███▍      | 4185/12000 [2:29:23<3:30:52,  1.62s/it]
                                                        

 35%|███▍      | 4185/12000 [2:29:23<3:30:52,  1.62s/it]
 35%|███▍      | 4186/12000 [2:29:25<3:45:05,  1.73s/it]
                                                        

 35%|███▍      | 4186/12000 [2:29:25<3:45:05,  1.73s/it]
 35%|███▍      | 4187/12000 [2:29:26<3:44:27,  1.72s/it]
                                                        
{'loss': '1.105', 'grad_norm': '0.3952', 'learning_rate': '0.0001561', 'entropy': '1.022', 'num_tokens': '2.239e+07', 'mean_token_accuracy': '0.7505', 'epoch': '0.03664'}
{'loss': '0.7594', 'grad_norm': '0.3467', 'learning_rate': '0.0001561', 'entropy': '0.7461', 'num_tokens': '2.24e+07', 'mean_token_accuracy': '0.8088', 'epoch': '0.03665'}
{'loss': '1.211', 'grad_norm': '0.4423', 'learning_rate': '0.000156', 'entropy': '1.14', 'num_tokens': '2.24e+07', 'mean_token_accuracy': '0.725', 'epoch': '0.03666'}
{'loss': '0.9711', 'grad_norm': '0.4231', 'learning_rate': '0.000156', 'entropy': '1.004', 'num_tokens': '2.241e+07', 'mean_token_accuracy': '0.7442', 'epoch': '0.03666'}
{'loss': '0.8178', 'grad_norm': '0.3723', 'learning_rate': '0.000156', 'entropy': '0.8236', 'num_tokens': '2.241e+07', 'mean_token_accuracy': '0.7809', 'epoch': '0.03667'}
{'loss': '0.7819', 'grad_norm': '0.3628', 'learning_rate': '0.000156', 'entropy': '0.823', 'num_tokens': '2.242e+07', 'mean_token_accuracy': '0.7852', 'epoch': '0.03668'}
{'loss': '0.5882', 'grad_norm': '0.2774', 'learning_rate': '0.000156', 'entropy': '0.5892', 'num_tokens': '2.242e+07', 'mean_token_accuracy': '0.8491', 'epoch': '0.03669'}
{'loss': '0.9631', 'grad_norm': '0.3998', 'learning_rate': '0.0001559', 'entropy': '0.9396', 'num_tokens': '2.243e+07', 'mean_token_accuracy': '0.7582', 'epoch': '0.0367'}
{'loss': '1.027', 'grad_norm': '0.3997', 'learning_rate': '0.0001559', 'entropy': '0.9868', 'num_tokens': '2.243e+07', 'mean_token_accuracy': '0.7526', 'epoch': '0.03671'}
{'loss': '1.013', 'grad_norm': '0.4017', 'learning_rate': '0.0001559', 'entropy': '0.8087', 'num_tokens': '2.244e+07', 'mean_token_accuracy': '0.8018', 'epoch': '0.03672'}
{'loss': '0.9969', 'grad_norm': '0.5269', 'learning_rate': '0.0001559', 'entropy': '0.7788', 'num_tokens': '2.245e+07', 'mean_token_accuracy': '0.808', 'epoch': '0.03673'}
{'loss': '1.124', 'grad_norm': '0.3545', 'learning_rate': '0.0001558', 'entropy': '1.093', 'num_tokens': '2.245e+07', 'mean_token_accuracy': '0.7465', 'epoch': '0.03674'}
{'loss': '0.8328', 'grad_norm': '0.3879', 'learning_rate': '0.0001558', 'entropy': '0.7823', 'num_tokens': '2.246e+07', 'mean_token_accuracy': '0.8029', 'epoch': '0.03674'}
{'loss': '1.021', 'grad_norm': '0.4528', 'learning_rate': '0.0001558', 'entropy': '1.003', 'num_tokens': '2.246e+07', 'mean_token_accuracy': '0.7571', 'epoch': '0.03675'}
{'loss': '0.7925', 'grad_norm': '0.3518', 'learning_rate': '0.0001558', 'entropy': '0.8666', 'num_tokens': '2.247e+07', 'mean_token_accuracy': '0.7949', 'epoch': '0.03676'}
{'loss': '0.6908', 'grad_norm': '0.3968', 'learning_rate': '0.0001558', 'entropy': '0.8274', 'num_tokens': '2.247e+07', 'mean_token_accuracy': '0.798', 'epoch': '0.03677'}
{'loss': '0.859', 'grad_norm': '0.3587', 'learning_rate': '0.0001557', 'entropy': '0.8678', 'num_tokens': '2.248e+07', 'mean_token_accuracy': '0.7814', 'epoch': '0.03678'}
{'loss': '0.9348', 'grad_norm': '0.4427', 'learning_rate': '0.0001557', 'entropy': '1.046', 'num_tokens': '2.248e+07', 'mean_token_accuracy': '0.7368', 'epoch': '0.03679'}
{'loss': '0.7255', 'grad_norm': '0.3594', 'learning_rate': '0.0001557', 'entropy': '0.7149', 'num_tokens': '2.249e+07', 'mean_token_accuracy': '0.8119', 'epoch': '0.0368'}
{'loss': '1.018', 'grad_norm': '0.3907', 'learning_rate': '0.0001557', 'entropy': '1.073', 'num_tokens': '2.25e+07', 'mean_token_accuracy': '0.7318', 'epoch': '0.03681'}
{'loss': '1.278', 'grad_norm': '0.4333', 'learning_rate': '0.0001556', 'entropy': '1.228', 'num_tokens': '2.25e+07', 'mean_token_accuracy': '0.7088', 'epoch': '0.03682'}
{'loss': '0.8028', 'grad_norm': '0.4249', 'learning_rate': '0.0001556', 'entropy': '0.7893', 'num_tokens': '2.251e+07', 'mean_token_accuracy': '0.7924', 'epoch': '0.03682'}
{'loss': '0.5808', 'grad_norm': '0.3043', 'learning_rate': '0.0001556', 'entropy': '0.5894', 'num_tokens': '2.251e+07', 'mean_token_accuracy': '0.8477', 'epoch': '0.03683'}
{'loss': '0.8532', 'grad_norm': '0.3644', 'learning_rate': '0.0001556', 'entropy': '0.8711', 'num_tokens': '2.252e+07', 'mean_token_accuracy': '0.798', 'epoch': '0.03684'}
{'loss': '0.869', 'grad_norm': '0.3736', 'learning_rate': '0.0001555', 'entropy': '0.7845', 'num_tokens': '2.253e+07', 'mean_token_accuracy': '0.7934', 'epoch': '0.03685'}
{'loss': '0.8299', 'grad_norm': '0.4045', 'learning_rate': '0.0001555', 'entropy': '0.813', 'num_tokens': '2.253e+07', 'mean_token_accuracy': '0.7915', 'epoch': '0.03686'}
{'loss': '0.6638', 'grad_norm': '0.2898', 'learning_rate': '0.0001555', 'entropy': '0.6311', 'num_tokens': '2.254e+07', 'mean_token_accuracy': '0.843', 'epoch': '0.03687'}
{'loss': '0.6235', 'grad_norm': '0.3477', 'learning_rate': '0.0001555', 'entropy': '0.6705', 'num_tokens': '2.255e+07', 'mean_token_accuracy': '0.8211', 'epoch': '0.03688'}
{'loss': '0.9867', 'grad_norm': '0.4841', 'learning_rate': '0.0001555', 'entropy': '1.041', 'num_tokens': '2.255e+07', 'mean_token_accuracy': '0.7598', 'epoch': '0.03689'}
{'loss': '0.738', 'grad_norm': '0.437', 'learning_rate': '0.0001554', 'entropy': '0.8143', 'num_tokens': '2.256e+07', 'mean_token_accuracy': '0.7864', 'epoch': '0.03689'}
{'loss': '0.7945', 'grad_norm': '0.3619', 'learning_rate': '0.0001554', 'entropy': '0.7362', 'num_tokens': '2.256e+07', 'mean_token_accuracy': '0.807', 'epoch': '0.0369'}
{'loss': '0.8689', 'grad_norm': '0.307', 'learning_rate': '0.0001554', 'entropy': '0.8699', 'num_tokens': '2.257e+07', 'mean_token_accuracy': '0.7857', 'epoch': '0.03691'}
{'loss': '1.103', 'grad_norm': '0.4155', 'learning_rate': '0.0001554', 'entropy': '1.037', 'num_tokens': '2.258e+07', 'mean_token_accuracy': '0.7498', 'epoch': '0.03692'}
{'loss': '1.176', 'grad_norm': '0.3969', 'learning_rate': '0.0001553', 'entropy': '1.154', 'num_tokens': '2.258e+07', 'mean_token_accuracy': '0.7373', 'epoch': '0.03693'}
{'loss': '0.7621', 'grad_norm': '0.4252', 'learning_rate': '0.0001553', 'entropy': '0.8269', 'num_tokens': '2.259e+07', 'mean_token_accuracy': '0.7843', 'epoch': '0.03694'}
{'loss': '1.039', 'grad_norm': '0.392', 'learning_rate': '0.0001553', 'entropy': '1.043', 'num_tokens': '2.259e+07', 'mean_token_accuracy': '0.7395', 'epoch': '0.03695'}
{'loss': '0.6554', 'grad_norm': '0.3215', 'learning_rate': '0.0001553', 'entropy': '0.639', 'num_tokens': '2.26e+07', 'mean_token_accuracy': '0.8289', 'epoch': '0.03696'}
{'loss': '0.6996', 'grad_norm': '0.2801', 'learning_rate': '0.0001552', 'entropy': '0.7532', 'num_tokens': '2.261e+07', 'mean_token_accuracy': '0.8069', 'epoch': '0.03697'}
{'loss': '0.7208', 'grad_norm': '0.3214', 'learning_rate': '0.0001552', 'entropy': '0.681', 'num_tokens': '2.261e+07', 'mean_token_accuracy': '0.8249', 'epoch': '0.03697'}
{'loss': '0.8805', 'grad_norm': '0.4145', 'learning_rate': '0.0001552', 'entropy': '0.9102', 'num_tokens': '2.262e+07', 'mean_token_accuracy': '0.7744', 'epoch': '0.03698'}
{'loss': '0.7573', 'grad_norm': '0.3807', 'learning_rate': '0.0001552', 'entropy': '0.7512', 'num_tokens': '2.262e+07', 'mean_token_accuracy': '0.8106', 'epoch': '0.03699'}
{'loss': '0.9712', 'grad_norm': '0.3897', 'learning_rate': '0.0001552', 'entropy': '0.9302', 'num_tokens': '2.263e+07', 'mean_token_accuracy': '0.7741', 'epoch': '0.037'}
{'loss': '0.7505', 'grad_norm': '0.353', 'learning_rate': '0.0001551', 'entropy': '0.8039', 'num_tokens': '2.263e+07', 'mean_token_accuracy': '0.794', 'epoch': '0.03701'}
{'loss': '0.9803', 'grad_norm': '0.4192', 'learning_rate': '0.0001551', 'entropy': '0.9595', 'num_tokens': '2.264e+07', 'mean_token_accuracy': '0.7683', 'epoch': '0.03702'}
{'loss': '1.011', 'grad_norm': '0.4327', 'learning_rate': '0.0001551', 'entropy': '1.043', 'num_tokens': '2.264e+07', 'mean_token_accuracy': '0.7503', 'epoch': '0.03703'}
{'loss': '0.8888', 'grad_norm': '0.3756', 'learning_rate': '0.0001551', 'entropy': '0.8779', 'num_tokens': '2.265e+07', 'mean_token_accuracy': '0.7872', 'epoch': '0.03704'}
{'loss': '1.046', 'grad_norm': '0.3972', 'learning_rate': '0.000155', 'entropy': '1.085', 'num_tokens': '2.265e+07', 'mean_token_accuracy': '0.746', 'epoch': '0.03705'}

 35%|███▍      | 4187/12000 [2:29:26<3:44:27,  1.72s/it]
 35%|███▍      | 4188/12000 [2:29:29<4:28:23,  2.06s/it]
                                                        

 35%|███▍      | 4188/12000 [2:29:29<4:28:23,  2.06s/it]
 35%|███▍      | 4189/12000 [2:29:31<4:35:59,  2.12s/it]
                                                        

 35%|███▍      | 4189/12000 [2:29:31<4:35:59,  2.12s/it]
 35%|███▍      | 4190/12000 [2:29:33<4:04:24,  1.88s/it]
                                                        

 35%|███▍      | 4190/12000 [2:29:33<4:04:24,  1.88s/it]
 35%|███▍      | 4191/12000 [2:29:34<3:54:00,  1.80s/it]
                                                        

 35%|███▍      | 4191/12000 [2:29:34<3:54:00,  1.80s/it]
 35%|███▍      | 4192/12000 [2:29:37<4:08:10,  1.91s/it]
                                                        

 35%|███▍      | 4192/12000 [2:29:37<4:08:10,  1.91s/it]
 35%|███▍      | 4193/12000 [2:29:39<4:34:03,  2.11s/it]
                                                        

 35%|███▍      | 4193/12000 [2:29:39<4:34:03,  2.11s/it]
 35%|███▍      | 4194/12000 [2:29:42<5:21:29,  2.47s/it]
                                                        

 35%|███▍      | 4194/12000 [2:29:42<5:21:29,  2.47s/it]
 35%|███▍      | 4195/12000 [2:29:45<5:25:17,  2.50s/it]
                                                        

 35%|███▍      | 4195/12000 [2:29:45<5:25:17,  2.50s/it]
 35%|███▍      | 4196/12000 [2:29:48<6:02:09,  2.78s/it]
                                                        

 35%|███▍      | 4196/12000 [2:29:48<6:02:09,  2.78s/it]
 35%|███▍      | 4197/12000 [2:29:50<4:59:56,  2.31s/it]
                                                        

 35%|███▍      | 4197/12000 [2:29:50<4:59:56,  2.31s/it]
 35%|███▍      | 4198/12000 [2:29:51<4:34:37,  2.11s/it]
                                                        

 35%|███▍      | 4198/12000 [2:29:51<4:34:37,  2.11s/it]
 35%|███▍      | 4199/12000 [2:29:54<4:38:43,  2.14s/it]
                                                        

 35%|███▍      | 4199/12000 [2:29:54<4:38:43,  2.14s/it]
 35%|███▌      | 4200/12000 [2:29:55<4:08:13,  1.91s/it]
                                                        

 35%|███▌      | 4200/12000 [2:29:55<4:08:13,  1.91s/it]
 35%|███▌      | 4201/12000 [2:29:58<4:37:20,  2.13s/it]
                                                        

 35%|███▌      | 4201/12000 [2:29:58<4:37:20,  2.13s/it]
 35%|███▌      | 4202/12000 [2:29:59<4:04:30,  1.88s/it]
                                                        

 35%|███▌      | 4202/12000 [2:29:59<4:04:30,  1.88s/it]
 35%|███▌      | 4203/12000 [2:30:00<3:52:57,  1.79s/it]
                                                        

 35%|███▌      | 4203/12000 [2:30:00<3:52:57,  1.79s/it]
 35%|███▌      | 4204/12000 [2:30:02<3:36:18,  1.66s/it]
                                                        

 35%|███▌      | 4204/12000 [2:30:02<3:36:18,  1.66s/it]
 35%|███▌      | 4205/12000 [2:30:03<3:36:03,  1.66s/it]
                                                        

 35%|███▌      | 4205/12000 [2:30:03<3:36:03,  1.66s/it]
 35%|███▌      | 4206/12000 [2:30:05<3:30:41,  1.62s/it]
                                                        

 35%|███▌      | 4206/12000 [2:30:05<3:30:41,  1.62s/it]
 35%|███▌      | 4207/12000 [2:30:07<4:04:37,  1.88s/it]
                                                        

 35%|███▌      | 4207/12000 [2:30:07<4:04:37,  1.88s/it]
 35%|███▌      | 4208/12000 [2:30:09<3:49:49,  1.77s/it]
                                                        

 35%|███▌      | 4208/12000 [2:30:09<3:49:49,  1.77s/it]
 35%|███▌      | 4209/12000 [2:30:10<3:33:51,  1.65s/it]
                                                        

 35%|███▌      | 4209/12000 [2:30:10<3:33:51,  1.65s/it]
 35%|███▌      | 4210/12000 [2:30:12<3:40:14,  1.70s/it]
                                                        

 35%|███▌      | 4210/12000 [2:30:12<3:40:14,  1.70s/it]
 35%|███▌      | 4211/12000 [2:30:14<3:48:05,  1.76s/it]
                                                        

 35%|███▌      | 4211/12000 [2:30:14<3:48:05,  1.76s/it]
 35%|███▌      | 4212/12000 [2:30:16<3:58:23,  1.84s/it]
                                                        

 35%|███▌      | 4212/12000 [2:30:16<3:58:23,  1.84s/it]
 35%|███▌      | 4213/12000 [2:30:18<4:14:27,  1.96s/it]
                                                        

 35%|███▌      | 4213/12000 [2:30:18<4:14:27,  1.96s/it]
 35%|███▌      | 4214/12000 [2:30:20<4:00:05,  1.85s/it]
                                                        

 35%|███▌      | 4214/12000 [2:30:20<4:00:05,  1.85s/it]
 35%|███▌      | 4215/12000 [2:30:21<3:44:46,  1.73s/it]
                                                        

 35%|███▌      | 4215/12000 [2:30:21<3:44:46,  1.73s/it]
 35%|███▌      | 4216/12000 [2:30:23<3:43:31,  1.72s/it]
                                                        

 35%|███▌      | 4216/12000 [2:30:23<3:43:31,  1.72s/it]
 35%|███▌      | 4217/12000 [2:30:26<4:46:06,  2.21s/it]
                                                        

 35%|███▌      | 4217/12000 [2:30:26<4:46:06,  2.21s/it]
 35%|███▌      | 4218/12000 [2:30:29<4:50:34,  2.24s/it]
                                                        

 35%|███▌      | 4218/12000 [2:30:29<4:50:34,  2.24s/it]
 35%|███▌      | 4219/12000 [2:30:31<5:02:28,  2.33s/it]
                                                        

 35%|███▌      | 4219/12000 [2:30:31<5:02:28,  2.33s/it]
 35%|███▌      | 4220/12000 [2:30:33<4:18:43,  2.00s/it]
                                                        

 35%|███▌      | 4220/12000 [2:30:33<4:18:43,  2.00s/it]
 35%|███▌      | 4221/12000 [2:30:34<3:56:31,  1.82s/it]
                                                        

 35%|███▌      | 4221/12000 [2:30:34<3:56:31,  1.82s/it]
 35%|███▌      | 4222/12000 [2:30:36<3:51:15,  1.78s/it]
                                                        

 35%|███▌      | 4222/12000 [2:30:36<3:51:15,  1.78s/it]
 35%|███▌      | 4223/12000 [2:30:37<3:38:28,  1.69s/it]
                                                        

 35%|███▌      | 4223/12000 [2:30:37<3:38:28,  1.69s/it]
 35%|███▌      | 4224/12000 [2:30:39<3:36:23,  1.67s/it]
                                                        

 35%|███▌      | 4224/12000 [2:30:39<3:36:23,  1.67s/it]
 35%|███▌      | 4225/12000 [2:30:40<3:34:44,  1.66s/it]
                                                        

 35%|███▌      | 4225/12000 [2:30:40<3:34:44,  1.66s/it]
 35%|███▌      | 4226/12000 [2:30:42<3:23:07,  1.57s/it]
                                                        

 35%|███▌      | 4226/12000 [2:30:42<3:23:07,  1.57s/it]
 35%|███▌      | 4227/12000 [2:30:43<3:29:26,  1.62s/it]
                                                        

 35%|███▌      | 4227/12000 [2:30:43<3:29:26,  1.62s/it]
 35%|███▌      | 4228/12000 [2:30:49<6:00:54,  2.79s/it]
                                                        

 35%|███▌      | 4228/12000 [2:30:49<6:00:54,  2.79s/it]
 35%|███▌      | 4229/12000 [2:30:50<5:12:21,  2.41s/it]
                                                        

 35%|███▌      | 4229/12000 [2:30:50<5:12:21,  2.41s/it]
 35%|███▌      | 4230/12000 [2:30:53<4:58:20,  2.30s/it]
                                                        

 35%|███▌      | 4230/12000 [2:30:53<4:58:20,  2.30s/it]
 35%|███▌      | 4231/12000 [2:30:54<4:28:51,  2.08s/it]
                                                        

 35%|███▌      | 4231/12000 [2:30:54<4:28:51,  2.08s/it]
 35%|███▌      | 4232/12000 [2:30:56<4:14:33,  1.97s/it]
                                                        

 35%|███▌      | 4232/12000 [2:30:56<4:14:33,  1.97s/it]
 35%|███▌      | 4233/12000 [2:30:58<4:16:50,  1.98s/it]
                                                        

 35%|███▌      | 4233/12000 [2:30:58<4:16:50,  1.98s/it]
 35%|███▌      | 4234/12000 [2:30:59<3:56:03,  1.82s/it]
                                                        
{'loss': '0.729', 'grad_norm': '0.4015', 'learning_rate': '0.000155', 'entropy': '0.7401', 'num_tokens': '2.266e+07', 'mean_token_accuracy': '0.8135', 'epoch': '0.03705'}
{'loss': '0.9311', 'grad_norm': '0.3586', 'learning_rate': '0.000155', 'entropy': '1', 'num_tokens': '2.267e+07', 'mean_token_accuracy': '0.7654', 'epoch': '0.03706'}
{'loss': '1.068', 'grad_norm': '0.3796', 'learning_rate': '0.000155', 'entropy': '1.119', 'num_tokens': '2.267e+07', 'mean_token_accuracy': '0.7544', 'epoch': '0.03707'}
{'loss': '0.7033', 'grad_norm': '0.3535', 'learning_rate': '0.0001549', 'entropy': '0.7294', 'num_tokens': '2.268e+07', 'mean_token_accuracy': '0.8084', 'epoch': '0.03708'}
{'loss': '0.7607', 'grad_norm': '0.3473', 'learning_rate': '0.0001549', 'entropy': '0.8088', 'num_tokens': '2.268e+07', 'mean_token_accuracy': '0.7961', 'epoch': '0.03709'}
{'loss': '1.188', 'grad_norm': '0.431', 'learning_rate': '0.0001549', 'entropy': '1.104', 'num_tokens': '2.269e+07', 'mean_token_accuracy': '0.7426', 'epoch': '0.0371'}
{'loss': '0.696', 'grad_norm': '0.3386', 'learning_rate': '0.0001549', 'entropy': '0.8152', 'num_tokens': '2.269e+07', 'mean_token_accuracy': '0.8002', 'epoch': '0.03711'}
{'loss': '0.9633', 'grad_norm': '0.2804', 'learning_rate': '0.0001549', 'entropy': '1.007', 'num_tokens': '2.271e+07', 'mean_token_accuracy': '0.7585', 'epoch': '0.03712'}
{'loss': '0.9251', 'grad_norm': '0.3401', 'learning_rate': '0.0001548', 'entropy': '1.011', 'num_tokens': '2.271e+07', 'mean_token_accuracy': '0.7576', 'epoch': '0.03713'}
{'loss': '1.065', 'grad_norm': '0.3734', 'learning_rate': '0.0001548', 'entropy': '1.062', 'num_tokens': '2.272e+07', 'mean_token_accuracy': '0.7437', 'epoch': '0.03713'}
{'loss': '0.7631', 'grad_norm': '0.3639', 'learning_rate': '0.0001548', 'entropy': '0.7935', 'num_tokens': '2.272e+07', 'mean_token_accuracy': '0.8027', 'epoch': '0.03714'}
{'loss': '0.8268', 'grad_norm': '0.4169', 'learning_rate': '0.0001548', 'entropy': '0.8004', 'num_tokens': '2.273e+07', 'mean_token_accuracy': '0.7935', 'epoch': '0.03715'}
{'loss': '1.363', 'grad_norm': '0.4412', 'learning_rate': '0.0001547', 'entropy': '1.298', 'num_tokens': '2.273e+07', 'mean_token_accuracy': '0.696', 'epoch': '0.03716'}
{'loss': '0.9054', 'grad_norm': '0.402', 'learning_rate': '0.0001547', 'entropy': '0.8745', 'num_tokens': '2.274e+07', 'mean_token_accuracy': '0.7743', 'epoch': '0.03717'}
{'loss': '0.8444', 'grad_norm': '0.3701', 'learning_rate': '0.0001547', 'entropy': '0.8344', 'num_tokens': '2.274e+07', 'mean_token_accuracy': '0.7809', 'epoch': '0.03718'}
{'loss': '0.8593', 'grad_norm': '0.4827', 'learning_rate': '0.0001547', 'entropy': '0.8423', 'num_tokens': '2.274e+07', 'mean_token_accuracy': '0.7857', 'epoch': '0.03719'}
{'loss': '0.8395', 'grad_norm': '0.4696', 'learning_rate': '0.0001546', 'entropy': '0.8523', 'num_tokens': '2.275e+07', 'mean_token_accuracy': '0.7731', 'epoch': '0.0372'}
{'loss': '1.494', 'grad_norm': '0.5153', 'learning_rate': '0.0001546', 'entropy': '1.487', 'num_tokens': '2.275e+07', 'mean_token_accuracy': '0.6537', 'epoch': '0.0372'}
{'loss': '1.181', 'grad_norm': '0.3667', 'learning_rate': '0.0001546', 'entropy': '1.033', 'num_tokens': '2.276e+07', 'mean_token_accuracy': '0.7513', 'epoch': '0.03721'}
{'loss': '0.8109', 'grad_norm': '0.3807', 'learning_rate': '0.0001546', 'entropy': '0.8512', 'num_tokens': '2.276e+07', 'mean_token_accuracy': '0.7814', 'epoch': '0.03722'}
{'loss': '1.099', 'grad_norm': '0.3531', 'learning_rate': '0.0001546', 'entropy': '0.9959', 'num_tokens': '2.277e+07', 'mean_token_accuracy': '0.7479', 'epoch': '0.03723'}
{'loss': '1.075', 'grad_norm': '0.5071', 'learning_rate': '0.0001545', 'entropy': '1.045', 'num_tokens': '2.277e+07', 'mean_token_accuracy': '0.7499', 'epoch': '0.03724'}
{'loss': '0.866', 'grad_norm': '0.364', 'learning_rate': '0.0001545', 'entropy': '0.9053', 'num_tokens': '2.278e+07', 'mean_token_accuracy': '0.7797', 'epoch': '0.03725'}
{'loss': '0.7114', 'grad_norm': '0.4169', 'learning_rate': '0.0001545', 'entropy': '0.6946', 'num_tokens': '2.278e+07', 'mean_token_accuracy': '0.8217', 'epoch': '0.03726'}
{'loss': '1.2', 'grad_norm': '0.4159', 'learning_rate': '0.0001545', 'entropy': '1.206', 'num_tokens': '2.279e+07', 'mean_token_accuracy': '0.7137', 'epoch': '0.03727'}
{'loss': '0.7924', 'grad_norm': '0.3904', 'learning_rate': '0.0001544', 'entropy': '0.8351', 'num_tokens': '2.279e+07', 'mean_token_accuracy': '0.7921', 'epoch': '0.03728'}
{'loss': '1.153', 'grad_norm': '0.3621', 'learning_rate': '0.0001544', 'entropy': '1.085', 'num_tokens': '2.28e+07', 'mean_token_accuracy': '0.7396', 'epoch': '0.03728'}
{'loss': '0.8802', 'grad_norm': '0.4006', 'learning_rate': '0.0001544', 'entropy': '0.9111', 'num_tokens': '2.281e+07', 'mean_token_accuracy': '0.781', 'epoch': '0.03729'}
{'loss': '0.8922', 'grad_norm': '0.4072', 'learning_rate': '0.0001544', 'entropy': '0.9909', 'num_tokens': '2.281e+07', 'mean_token_accuracy': '0.761', 'epoch': '0.0373'}
{'loss': '0.7704', 'grad_norm': '0.4071', 'learning_rate': '0.0001543', 'entropy': '0.7784', 'num_tokens': '2.282e+07', 'mean_token_accuracy': '0.8054', 'epoch': '0.03731'}
{'loss': '0.6313', 'grad_norm': '0.361', 'learning_rate': '0.0001543', 'entropy': '0.6242', 'num_tokens': '2.282e+07', 'mean_token_accuracy': '0.8376', 'epoch': '0.03732'}
{'loss': '0.8192', 'grad_norm': '0.3294', 'learning_rate': '0.0001543', 'entropy': '0.8165', 'num_tokens': '2.283e+07', 'mean_token_accuracy': '0.7963', 'epoch': '0.03733'}
{'loss': '0.9472', 'grad_norm': '0.3969', 'learning_rate': '0.0001543', 'entropy': '0.9511', 'num_tokens': '2.284e+07', 'mean_token_accuracy': '0.7653', 'epoch': '0.03734'}
{'loss': '0.9989', 'grad_norm': '0.3785', 'learning_rate': '0.0001543', 'entropy': '1.009', 'num_tokens': '2.284e+07', 'mean_token_accuracy': '0.7416', 'epoch': '0.03735'}
{'loss': '1.261', 'grad_norm': '0.4723', 'learning_rate': '0.0001542', 'entropy': '1.145', 'num_tokens': '2.284e+07', 'mean_token_accuracy': '0.7241', 'epoch': '0.03736'}
{'loss': '1.153', 'grad_norm': '0.3935', 'learning_rate': '0.0001542', 'entropy': '1.092', 'num_tokens': '2.285e+07', 'mean_token_accuracy': '0.7417', 'epoch': '0.03736'}
{'loss': '0.618', 'grad_norm': '0.3177', 'learning_rate': '0.0001542', 'entropy': '0.611', 'num_tokens': '2.285e+07', 'mean_token_accuracy': '0.8408', 'epoch': '0.03737'}
{'loss': '0.9689', 'grad_norm': '0.376', 'learning_rate': '0.0001542', 'entropy': '0.9333', 'num_tokens': '2.286e+07', 'mean_token_accuracy': '0.7703', 'epoch': '0.03738'}
{'loss': '0.6258', 'grad_norm': '0.3891', 'learning_rate': '0.0001541', 'entropy': '0.6772', 'num_tokens': '2.286e+07', 'mean_token_accuracy': '0.8289', 'epoch': '0.03739'}
{'loss': '0.858', 'grad_norm': '0.4929', 'learning_rate': '0.0001541', 'entropy': '0.9197', 'num_tokens': '2.287e+07', 'mean_token_accuracy': '0.767', 'epoch': '0.0374'}
{'loss': '0.8963', 'grad_norm': '0.4084', 'learning_rate': '0.0001541', 'entropy': '0.883', 'num_tokens': '2.287e+07', 'mean_token_accuracy': '0.7715', 'epoch': '0.03741'}
{'loss': '1.155', 'grad_norm': '0.3406', 'learning_rate': '0.0001541', 'entropy': '0.9793', 'num_tokens': '2.288e+07', 'mean_token_accuracy': '0.7619', 'epoch': '0.03742'}
{'loss': '0.7589', 'grad_norm': '0.4228', 'learning_rate': '0.000154', 'entropy': '0.7113', 'num_tokens': '2.289e+07', 'mean_token_accuracy': '0.8229', 'epoch': '0.03743'}
{'loss': '1.024', 'grad_norm': '0.5337', 'learning_rate': '0.000154', 'entropy': '0.9628', 'num_tokens': '2.289e+07', 'mean_token_accuracy': '0.7594', 'epoch': '0.03743'}
{'loss': '0.8684', 'grad_norm': '0.4116', 'learning_rate': '0.000154', 'entropy': '0.8444', 'num_tokens': '2.29e+07', 'mean_token_accuracy': '0.7748', 'epoch': '0.03744'}
{'loss': '0.7381', 'grad_norm': '0.3896', 'learning_rate': '0.000154', 'entropy': '0.7392', 'num_tokens': '2.29e+07', 'mean_token_accuracy': '0.8111', 'epoch': '0.03745'}
{'loss': '0.767', 'grad_norm': '0.3461', 'learning_rate': '0.000154', 'entropy': '0.7344', 'num_tokens': '2.291e+07', 'mean_token_accuracy': '0.8116', 'epoch': '0.03746'}

 35%|███▌      | 4234/12000 [2:30:59<3:56:03,  1.82s/it]
 35%|███▌      | 4235/12000 [2:31:02<4:29:59,  2.09s/it]
                                                        

 35%|███▌      | 4235/12000 [2:31:02<4:29:59,  2.09s/it]
 35%|███▌      | 4236/12000 [2:31:04<4:18:39,  2.00s/it]
                                                        

 35%|███▌      | 4236/12000 [2:31:04<4:18:39,  2.00s/it]
 35%|███▌      | 4237/12000 [2:31:05<3:54:50,  1.82s/it]
                                                        

 35%|███▌      | 4237/12000 [2:31:05<3:54:50,  1.82s/it]
 35%|███▌      | 4238/12000 [2:31:07<3:45:51,  1.75s/it]
                                                        

 35%|███▌      | 4238/12000 [2:31:07<3:45:51,  1.75s/it]
 35%|███▌      | 4239/12000 [2:31:08<3:35:55,  1.67s/it]
                                                        

 35%|███▌      | 4239/12000 [2:31:08<3:35:55,  1.67s/it]
 35%|███▌      | 4240/12000 [2:31:12<4:48:58,  2.23s/it]
                                                        

 35%|███▌      | 4240/12000 [2:31:12<4:48:58,  2.23s/it]
 35%|███▌      | 4241/12000 [2:31:13<4:19:51,  2.01s/it]
                                                        

 35%|███▌      | 4241/12000 [2:31:13<4:19:51,  2.01s/it]
 35%|███▌      | 4242/12000 [2:31:16<4:48:06,  2.23s/it]
                                                        

 35%|███▌      | 4242/12000 [2:31:16<4:48:06,  2.23s/it]
 35%|███▌      | 4243/12000 [2:31:18<4:29:22,  2.08s/it]
                                                        

 35%|███▌      | 4243/12000 [2:31:18<4:29:22,  2.08s/it]
 35%|███▌      | 4244/12000 [2:31:19<4:11:10,  1.94s/it]
                                                        

 35%|███▌      | 4244/12000 [2:31:19<4:11:10,  1.94s/it]
 35%|███▌      | 4245/12000 [2:31:21<4:01:36,  1.87s/it]
                                                        

 35%|███▌      | 4245/12000 [2:31:21<4:01:36,  1.87s/it]
 35%|███▌      | 4246/12000 [2:31:22<3:41:07,  1.71s/it]
                                                        

 35%|███▌      | 4246/12000 [2:31:22<3:41:07,  1.71s/it]
 35%|███▌      | 4247/12000 [2:31:24<3:53:33,  1.81s/it]
                                                        

 35%|███▌      | 4247/12000 [2:31:24<3:53:33,  1.81s/it]
 35%|███▌      | 4248/12000 [2:31:29<5:36:42,  2.61s/it]
                                                        

 35%|███▌      | 4248/12000 [2:31:29<5:36:42,  2.61s/it]
 35%|███▌      | 4249/12000 [2:31:31<4:58:06,  2.31s/it]
                                                        

 35%|███▌      | 4249/12000 [2:31:31<4:58:06,  2.31s/it]
 35%|███▌      | 4250/12000 [2:31:34<5:30:25,  2.56s/it]
                                                        

 35%|███▌      | 4250/12000 [2:31:34<5:30:25,  2.56s/it]
 35%|███▌      | 4251/12000 [2:31:36<5:40:08,  2.63s/it]
                                                        

 35%|███▌      | 4251/12000 [2:31:36<5:40:08,  2.63s/it]
 35%|███▌      | 4252/12000 [2:31:39<5:36:05,  2.60s/it]
                                                        

 35%|███▌      | 4252/12000 [2:31:39<5:36:05,  2.60s/it]
 35%|███▌      | 4253/12000 [2:31:41<4:55:48,  2.29s/it]
                                                        

 35%|███▌      | 4253/12000 [2:31:41<4:55:48,  2.29s/it]
 35%|███▌      | 4254/12000 [2:31:42<4:31:46,  2.11s/it]
                                                        

 35%|███▌      | 4254/12000 [2:31:42<4:31:46,  2.11s/it]
 35%|███▌      | 4255/12000 [2:31:44<4:10:15,  1.94s/it]
                                                        

 35%|███▌      | 4255/12000 [2:31:44<4:10:15,  1.94s/it]
 35%|███▌      | 4256/12000 [2:31:45<3:52:44,  1.80s/it]
                                                        

 35%|███▌      | 4256/12000 [2:31:45<3:52:44,  1.80s/it]
 35%|███▌      | 4257/12000 [2:31:47<3:48:47,  1.77s/it]
                                                        

 35%|███▌      | 4257/12000 [2:31:47<3:48:47,  1.77s/it]
 35%|███▌      | 4258/12000 [2:31:49<3:51:32,  1.79s/it]
                                                        

 35%|███▌      | 4258/12000 [2:31:49<3:51:32,  1.79s/it]
 35%|███▌      | 4259/12000 [2:31:50<3:41:42,  1.72s/it]
                                                        

 35%|███▌      | 4259/12000 [2:31:50<3:41:42,  1.72s/it]
 36%|███▌      | 4260/12000 [2:31:54<4:52:04,  2.26s/it]
                                                        

 36%|███▌      | 4260/12000 [2:31:54<4:52:04,  2.26s/it]
 36%|███▌      | 4261/12000 [2:31:57<5:26:55,  2.53s/it]
                                                        

 36%|███▌      | 4261/12000 [2:31:57<5:26:55,  2.53s/it]
 36%|███▌      | 4262/12000 [2:32:02<7:14:06,  3.37s/it]
                                                        

 36%|███▌      | 4262/12000 [2:32:02<7:14:06,  3.37s/it]
 36%|███▌      | 4263/12000 [2:32:04<6:16:28,  2.92s/it]
                                                        

 36%|███▌      | 4263/12000 [2:32:04<6:16:28,  2.92s/it]
 36%|███▌      | 4264/12000 [2:32:06<5:35:57,  2.61s/it]
                                                        

 36%|███▌      | 4264/12000 [2:32:06<5:35:57,  2.61s/it]
 36%|███▌      | 4265/12000 [2:32:08<5:21:26,  2.49s/it]
                                                        

 36%|███▌      | 4265/12000 [2:32:08<5:21:26,  2.49s/it]
 36%|███▌      | 4266/12000 [2:32:10<4:42:51,  2.19s/it]
                                                        

 36%|███▌      | 4266/12000 [2:32:10<4:42:51,  2.19s/it]
 36%|███▌      | 4267/12000 [2:32:11<4:13:34,  1.97s/it]
                                                        

 36%|███▌      | 4267/12000 [2:32:11<4:13:34,  1.97s/it]
 36%|███▌      | 4268/12000 [2:32:14<4:48:29,  2.24s/it]
                                                        

 36%|███▌      | 4268/12000 [2:32:14<4:48:29,  2.24s/it]
 36%|███▌      | 4269/12000 [2:32:18<5:57:35,  2.78s/it]
                                                        

 36%|███▌      | 4269/12000 [2:32:18<5:57:35,  2.78s/it]
 36%|███▌      | 4270/12000 [2:32:21<5:43:52,  2.67s/it]
                                                        

 36%|███▌      | 4270/12000 [2:32:21<5:43:52,  2.67s/it]
 36%|███▌      | 4271/12000 [2:32:22<4:50:33,  2.26s/it]
                                                        

 36%|███▌      | 4271/12000 [2:32:22<4:50:33,  2.26s/it]
 36%|███▌      | 4272/12000 [2:32:26<6:20:38,  2.96s/it]
                                                        

 36%|███▌      | 4272/12000 [2:32:26<6:20:38,  2.96s/it]
 36%|███▌      | 4273/12000 [2:32:28<5:42:16,  2.66s/it]
                                                        

 36%|███▌      | 4273/12000 [2:32:28<5:42:16,  2.66s/it]
 36%|███▌      | 4274/12000 [2:32:31<5:52:57,  2.74s/it]
                                                        

 36%|███▌      | 4274/12000 [2:32:31<5:52:57,  2.74s/it]
 36%|███▌      | 4275/12000 [2:32:33<5:08:06,  2.39s/it]
                                                        

 36%|███▌      | 4275/12000 [2:32:33<5:08:06,  2.39s/it]
 36%|███▌      | 4276/12000 [2:32:34<4:34:06,  2.13s/it]
                                                        

 36%|███▌      | 4276/12000 [2:32:34<4:34:06,  2.13s/it]
 36%|███▌      | 4277/12000 [2:32:36<4:14:41,  1.98s/it]
                                                        

 36%|███▌      | 4277/12000 [2:32:36<4:14:41,  1.98s/it]
 36%|███▌      | 4278/12000 [2:32:38<4:04:20,  1.90s/it]
                                                        

 36%|███▌      | 4278/12000 [2:32:38<4:04:20,  1.90s/it]
 36%|███▌      | 4279/12000 [2:32:39<3:42:05,  1.73s/it]
                                                        

 36%|███▌      | 4279/12000 [2:32:39<3:42:05,  1.73s/it]
 36%|███▌      | 4280/12000 [2:32:41<3:52:57,  1.81s/it]
                                                        

 36%|███▌      | 4280/12000 [2:32:41<3:52:57,  1.81s/it]
 36%|███▌      | 4281/12000 [2:32:43<3:48:30,  1.78s/it]
                                                        
{'loss': '0.6793', 'grad_norm': '0.3642', 'learning_rate': '0.0001539', 'entropy': '0.6723', 'num_tokens': '2.291e+07', 'mean_token_accuracy': '0.8247', 'epoch': '0.03747'}
{'loss': '1.053', 'grad_norm': '0.4023', 'learning_rate': '0.0001539', 'entropy': '1.213', 'num_tokens': '2.292e+07', 'mean_token_accuracy': '0.7213', 'epoch': '0.03748'}
{'loss': '0.7694', 'grad_norm': '0.4227', 'learning_rate': '0.0001539', 'entropy': '0.8112', 'num_tokens': '2.292e+07', 'mean_token_accuracy': '0.7972', 'epoch': '0.03749'}
{'loss': '0.7198', 'grad_norm': '0.384', 'learning_rate': '0.0001539', 'entropy': '0.7457', 'num_tokens': '2.293e+07', 'mean_token_accuracy': '0.8125', 'epoch': '0.0375'}
{'loss': '0.9534', 'grad_norm': '0.386', 'learning_rate': '0.0001538', 'entropy': '0.973', 'num_tokens': '2.293e+07', 'mean_token_accuracy': '0.7613', 'epoch': '0.03751'}
{'loss': '1.111', 'grad_norm': '0.4401', 'learning_rate': '0.0001538', 'entropy': '1.028', 'num_tokens': '2.294e+07', 'mean_token_accuracy': '0.762', 'epoch': '0.03751'}
{'loss': '0.7838', 'grad_norm': '0.4019', 'learning_rate': '0.0001538', 'entropy': '0.8231', 'num_tokens': '2.294e+07', 'mean_token_accuracy': '0.7951', 'epoch': '0.03752'}
{'loss': '0.856', 'grad_norm': '0.4085', 'learning_rate': '0.0001538', 'entropy': '0.8769', 'num_tokens': '2.295e+07', 'mean_token_accuracy': '0.7833', 'epoch': '0.03753'}
{'loss': '0.713', 'grad_norm': '0.344', 'learning_rate': '0.0001537', 'entropy': '0.6675', 'num_tokens': '2.295e+07', 'mean_token_accuracy': '0.8317', 'epoch': '0.03754'}
{'loss': '0.9946', 'grad_norm': '0.4776', 'learning_rate': '0.0001537', 'entropy': '1.033', 'num_tokens': '2.296e+07', 'mean_token_accuracy': '0.7596', 'epoch': '0.03755'}
{'loss': '0.8773', 'grad_norm': '0.362', 'learning_rate': '0.0001537', 'entropy': '0.9116', 'num_tokens': '2.296e+07', 'mean_token_accuracy': '0.7642', 'epoch': '0.03756'}
{'loss': '0.8148', 'grad_norm': '0.349', 'learning_rate': '0.0001537', 'entropy': '0.8167', 'num_tokens': '2.297e+07', 'mean_token_accuracy': '0.7941', 'epoch': '0.03757'}
{'loss': '0.7606', 'grad_norm': '0.3407', 'learning_rate': '0.0001537', 'entropy': '0.8005', 'num_tokens': '2.297e+07', 'mean_token_accuracy': '0.7968', 'epoch': '0.03758'}
{'loss': '0.9515', 'grad_norm': '0.3783', 'learning_rate': '0.0001536', 'entropy': '0.8678', 'num_tokens': '2.298e+07', 'mean_token_accuracy': '0.7894', 'epoch': '0.03759'}
{'loss': '0.6609', 'grad_norm': '0.2519', 'learning_rate': '0.0001536', 'entropy': '0.6225', 'num_tokens': '2.299e+07', 'mean_token_accuracy': '0.8349', 'epoch': '0.03759'}
{'loss': '0.5348', 'grad_norm': '0.352', 'learning_rate': '0.0001536', 'entropy': '0.537', 'num_tokens': '2.299e+07', 'mean_token_accuracy': '0.8607', 'epoch': '0.0376'}
{'loss': '0.8943', 'grad_norm': '0.3682', 'learning_rate': '0.0001536', 'entropy': '0.9046', 'num_tokens': '2.3e+07', 'mean_token_accuracy': '0.7771', 'epoch': '0.03761'}
{'loss': '0.9991', 'grad_norm': '0.384', 'learning_rate': '0.0001535', 'entropy': '1.002', 'num_tokens': '2.301e+07', 'mean_token_accuracy': '0.7558', 'epoch': '0.03762'}
{'loss': '0.8267', 'grad_norm': '0.3604', 'learning_rate': '0.0001535', 'entropy': '0.9246', 'num_tokens': '2.301e+07', 'mean_token_accuracy': '0.7688', 'epoch': '0.03763'}
{'loss': '0.6955', 'grad_norm': '0.3528', 'learning_rate': '0.0001535', 'entropy': '0.7084', 'num_tokens': '2.302e+07', 'mean_token_accuracy': '0.8182', 'epoch': '0.03764'}
{'loss': '0.7852', 'grad_norm': '0.4617', 'learning_rate': '0.0001535', 'entropy': '0.7914', 'num_tokens': '2.302e+07', 'mean_token_accuracy': '0.7928', 'epoch': '0.03765'}
{'loss': '0.8294', 'grad_norm': '0.3777', 'learning_rate': '0.0001534', 'entropy': '0.8472', 'num_tokens': '2.303e+07', 'mean_token_accuracy': '0.7848', 'epoch': '0.03766'}
{'loss': '0.7535', 'grad_norm': '0.3294', 'learning_rate': '0.0001534', 'entropy': '0.8135', 'num_tokens': '2.303e+07', 'mean_token_accuracy': '0.7949', 'epoch': '0.03766'}
{'loss': '1.035', 'grad_norm': '0.4092', 'learning_rate': '0.0001534', 'entropy': '0.9085', 'num_tokens': '2.304e+07', 'mean_token_accuracy': '0.7834', 'epoch': '0.03767'}
{'loss': '0.904', 'grad_norm': '0.4532', 'learning_rate': '0.0001534', 'entropy': '0.9861', 'num_tokens': '2.304e+07', 'mean_token_accuracy': '0.7419', 'epoch': '0.03768'}
{'loss': '0.7794', 'grad_norm': '0.3983', 'learning_rate': '0.0001533', 'entropy': '0.7415', 'num_tokens': '2.305e+07', 'mean_token_accuracy': '0.7968', 'epoch': '0.03769'}
{'loss': '0.7119', 'grad_norm': '0.33', 'learning_rate': '0.0001533', 'entropy': '0.9952', 'num_tokens': '2.306e+07', 'mean_token_accuracy': '0.7693', 'epoch': '0.0377'}
{'loss': '0.9127', 'grad_norm': '0.3858', 'learning_rate': '0.0001533', 'entropy': '0.9854', 'num_tokens': '2.306e+07', 'mean_token_accuracy': '0.7663', 'epoch': '0.03771'}
{'loss': '1.413', 'grad_norm': '0.4073', 'learning_rate': '0.0001533', 'entropy': '1.237', 'num_tokens': '2.307e+07', 'mean_token_accuracy': '0.7413', 'epoch': '0.03772'}
{'loss': '0.7739', 'grad_norm': '0.3857', 'learning_rate': '0.0001533', 'entropy': '0.7367', 'num_tokens': '2.308e+07', 'mean_token_accuracy': '0.8154', 'epoch': '0.03773'}
{'loss': '0.9045', 'grad_norm': '0.5085', 'learning_rate': '0.0001532', 'entropy': '0.8622', 'num_tokens': '2.308e+07', 'mean_token_accuracy': '0.7876', 'epoch': '0.03774'}
{'loss': '0.6784', 'grad_norm': '0.3082', 'learning_rate': '0.0001532', 'entropy': '0.782', 'num_tokens': '2.309e+07', 'mean_token_accuracy': '0.808', 'epoch': '0.03774'}
{'loss': '0.8717', 'grad_norm': '0.4469', 'learning_rate': '0.0001532', 'entropy': '0.984', 'num_tokens': '2.309e+07', 'mean_token_accuracy': '0.7662', 'epoch': '0.03775'}
{'loss': '1.044', 'grad_norm': '0.4971', 'learning_rate': '0.0001532', 'entropy': '1.125', 'num_tokens': '2.31e+07', 'mean_token_accuracy': '0.7285', 'epoch': '0.03776'}
{'loss': '0.5771', 'grad_norm': '0.3165', 'learning_rate': '0.0001531', 'entropy': '0.8078', 'num_tokens': '2.31e+07', 'mean_token_accuracy': '0.7992', 'epoch': '0.03777'}
{'loss': '0.7339', 'grad_norm': '0.4864', 'learning_rate': '0.0001531', 'entropy': '0.8215', 'num_tokens': '2.311e+07', 'mean_token_accuracy': '0.7826', 'epoch': '0.03778'}
{'loss': '0.627', 'grad_norm': '0.3631', 'learning_rate': '0.0001531', 'entropy': '0.6219', 'num_tokens': '2.311e+07', 'mean_token_accuracy': '0.843', 'epoch': '0.03779'}
{'loss': '0.5989', 'grad_norm': '0.3658', 'learning_rate': '0.0001531', 'entropy': '0.6245', 'num_tokens': '2.312e+07', 'mean_token_accuracy': '0.8332', 'epoch': '0.0378'}
{'loss': '1.103', 'grad_norm': '0.3595', 'learning_rate': '0.000153', 'entropy': '0.9067', 'num_tokens': '2.313e+07', 'mean_token_accuracy': '0.7795', 'epoch': '0.03781'}
{'loss': '0.9727', 'grad_norm': '0.3908', 'learning_rate': '0.000153', 'entropy': '0.9053', 'num_tokens': '2.313e+07', 'mean_token_accuracy': '0.7757', 'epoch': '0.03782'}
{'loss': '1.121', 'grad_norm': '0.3034', 'learning_rate': '0.000153', 'entropy': '1.008', 'num_tokens': '2.314e+07', 'mean_token_accuracy': '0.7522', 'epoch': '0.03782'}
{'loss': '1.086', 'grad_norm': '0.4667', 'learning_rate': '0.000153', 'entropy': '0.9716', 'num_tokens': '2.314e+07', 'mean_token_accuracy': '0.7771', 'epoch': '0.03783'}
{'loss': '0.7437', 'grad_norm': '0.3926', 'learning_rate': '0.000153', 'entropy': '0.8097', 'num_tokens': '2.315e+07', 'mean_token_accuracy': '0.7887', 'epoch': '0.03784'}
{'loss': '0.7509', 'grad_norm': '0.4326', 'learning_rate': '0.0001529', 'entropy': '0.7686', 'num_tokens': '2.315e+07', 'mean_token_accuracy': '0.7989', 'epoch': '0.03785'}
{'loss': '0.9804', 'grad_norm': '0.4471', 'learning_rate': '0.0001529', 'entropy': '0.8382', 'num_tokens': '2.316e+07', 'mean_token_accuracy': '0.7979', 'epoch': '0.03786'}
{'loss': '0.9398', 'grad_norm': '0.4291', 'learning_rate': '0.0001529', 'entropy': '0.9636', 'num_tokens': '2.316e+07', 'mean_token_accuracy': '0.7693', 'epoch': '0.03787'}
{'loss': '0.7422', 'grad_norm': '0.2534', 'learning_rate': '0.0001529', 'entropy': '0.7528', 'num_tokens': '2.317e+07', 'mean_token_accuracy': '0.8063', 'epoch': '0.03788'}

 36%|███▌      | 4281/12000 [2:32:43<3:48:30,  1.78s/it]
 36%|███▌      | 4282/12000 [2:32:44<3:37:54,  1.69s/it]
                                                        

 36%|███▌      | 4282/12000 [2:32:44<3:37:54,  1.69s/it]
 36%|███▌      | 4283/12000 [2:32:49<5:33:49,  2.60s/it]
                                                        

 36%|███▌      | 4283/12000 [2:32:49<5:33:49,  2.60s/it]
 36%|███▌      | 4284/12000 [2:32:52<5:33:27,  2.59s/it]
                                                        

 36%|███▌      | 4284/12000 [2:32:52<5:33:27,  2.59s/it]
 36%|███▌      | 4285/12000 [2:32:53<4:41:48,  2.19s/it]
                                                        

 36%|███▌      | 4285/12000 [2:32:53<4:41:48,  2.19s/it]
 36%|███▌      | 4286/12000 [2:32:58<6:30:47,  3.04s/it]
                                                        

 36%|███▌      | 4286/12000 [2:32:58<6:30:47,  3.04s/it]
 36%|███▌      | 4287/12000 [2:32:59<5:22:17,  2.51s/it]
                                                        

 36%|███▌      | 4287/12000 [2:32:59<5:22:17,  2.51s/it]
 36%|███▌      | 4288/12000 [2:33:02<5:22:19,  2.51s/it]
                                                        

 36%|███▌      | 4288/12000 [2:33:02<5:22:19,  2.51s/it]
 36%|███▌      | 4289/12000 [2:33:03<4:49:00,  2.25s/it]
                                                        

 36%|███▌      | 4289/12000 [2:33:03<4:49:00,  2.25s/it]
 36%|███▌      | 4290/12000 [2:33:05<4:26:04,  2.07s/it]
                                                        

 36%|███▌      | 4290/12000 [2:33:05<4:26:04,  2.07s/it]
 36%|███▌      | 4291/12000 [2:33:08<5:07:41,  2.39s/it]
                                                        

 36%|███▌      | 4291/12000 [2:33:08<5:07:41,  2.39s/it]
 36%|███▌      | 4292/12000 [2:33:10<4:38:45,  2.17s/it]
                                                        

 36%|███▌      | 4292/12000 [2:33:10<4:38:45,  2.17s/it]
 36%|███▌      | 4293/12000 [2:33:12<4:55:24,  2.30s/it]
                                                        

 36%|███▌      | 4293/12000 [2:33:12<4:55:24,  2.30s/it]
 36%|███▌      | 4294/12000 [2:33:14<4:46:42,  2.23s/it]
                                                        

 36%|███▌      | 4294/12000 [2:33:14<4:46:42,  2.23s/it]
 36%|███▌      | 4295/12000 [2:33:18<5:39:09,  2.64s/it]
                                                        

 36%|███▌      | 4295/12000 [2:33:18<5:39:09,  2.64s/it]
 36%|███▌      | 4296/12000 [2:33:20<5:00:09,  2.34s/it]
                                                        

 36%|███▌      | 4296/12000 [2:33:20<5:00:09,  2.34s/it]
 36%|███▌      | 4297/12000 [2:33:22<4:44:47,  2.22s/it]
                                                        

 36%|███▌      | 4297/12000 [2:33:22<4:44:47,  2.22s/it]
 36%|███▌      | 4298/12000 [2:33:24<4:35:11,  2.14s/it]
                                                        

 36%|███▌      | 4298/12000 [2:33:24<4:35:11,  2.14s/it]
 36%|███▌      | 4299/12000 [2:33:25<4:18:00,  2.01s/it]
                                                        

 36%|███▌      | 4299/12000 [2:33:25<4:18:00,  2.01s/it]
 36%|███▌      | 4300/12000 [2:33:27<4:01:22,  1.88s/it]
                                                        

 36%|███▌      | 4300/12000 [2:33:27<4:01:22,  1.88s/it]
 36%|███▌      | 4301/12000 [2:33:31<5:31:39,  2.58s/it]
                                                        

 36%|███▌      | 4301/12000 [2:33:31<5:31:39,  2.58s/it]
 36%|███▌      | 4302/12000 [2:33:33<5:20:25,  2.50s/it]
                                                        

 36%|███▌      | 4302/12000 [2:33:33<5:20:25,  2.50s/it]
 36%|███▌      | 4303/12000 [2:33:35<4:48:37,  2.25s/it]
                                                        

 36%|███▌      | 4303/12000 [2:33:35<4:48:37,  2.25s/it]
 36%|███▌      | 4304/12000 [2:33:37<4:46:49,  2.24s/it]
                                                        

 36%|███▌      | 4304/12000 [2:33:37<4:46:49,  2.24s/it]
 36%|███▌      | 4305/12000 [2:33:43<6:46:16,  3.17s/it]
                                                        

 36%|███▌      | 4305/12000 [2:33:43<6:46:16,  3.17s/it]
 36%|███▌      | 4306/12000 [2:33:44<5:42:03,  2.67s/it]
                                                        

 36%|███▌      | 4306/12000 [2:33:44<5:42:03,  2.67s/it]
 36%|███▌      | 4307/12000 [2:33:47<5:35:05,  2.61s/it]
                                                        

 36%|███▌      | 4307/12000 [2:33:47<5:35:05,  2.61s/it]
 36%|███▌      | 4308/12000 [2:33:50<5:47:44,  2.71s/it]
                                                        

 36%|███▌      | 4308/12000 [2:33:50<5:47:44,  2.71s/it]
 36%|███▌      | 4309/12000 [2:33:51<5:02:13,  2.36s/it]
                                                        

 36%|███▌      | 4309/12000 [2:33:51<5:02:13,  2.36s/it]
 36%|███▌      | 4310/12000 [2:33:53<4:46:41,  2.24s/it]
                                                        

 36%|███▌      | 4310/12000 [2:33:53<4:46:41,  2.24s/it]
 36%|███▌      | 4311/12000 [2:33:56<5:32:53,  2.60s/it]
                                                        

 36%|███▌      | 4311/12000 [2:33:56<5:32:53,  2.60s/it]
 36%|███▌      | 4312/12000 [2:33:58<4:57:01,  2.32s/it]
                                                        

 36%|███▌      | 4312/12000 [2:33:58<4:57:01,  2.32s/it]
 36%|███▌      | 4313/12000 [2:34:00<4:29:54,  2.11s/it]
                                                        

 36%|███▌      | 4313/12000 [2:34:00<4:29:54,  2.11s/it]
 36%|███▌      | 4314/12000 [2:34:02<4:42:05,  2.20s/it]
                                                        

 36%|███▌      | 4314/12000 [2:34:02<4:42:05,  2.20s/it]
 36%|███▌      | 4315/12000 [2:34:04<4:26:55,  2.08s/it]
                                                        

 36%|███▌      | 4315/12000 [2:34:04<4:26:55,  2.08s/it]
 36%|███▌      | 4316/12000 [2:34:06<4:12:09,  1.97s/it]
                                                        

 36%|███▌      | 4316/12000 [2:34:06<4:12:09,  1.97s/it]
 36%|███▌      | 4317/12000 [2:34:07<3:54:40,  1.83s/it]
                                                        

 36%|███▌      | 4317/12000 [2:34:07<3:54:40,  1.83s/it]
 36%|███▌      | 4318/12000 [2:34:09<3:53:43,  1.83s/it]
                                                        

 36%|███▌      | 4318/12000 [2:34:09<3:53:43,  1.83s/it]
 36%|███▌      | 4319/12000 [2:34:11<3:54:28,  1.83s/it]
                                                        

 36%|███▌      | 4319/12000 [2:34:11<3:54:28,  1.83s/it]
 36%|███▌      | 4320/12000 [2:34:13<4:16:34,  2.00s/it]
                                                        

 36%|███▌      | 4320/12000 [2:34:13<4:16:34,  2.00s/it]
 36%|███▌      | 4321/12000 [2:34:16<4:33:37,  2.14s/it]
                                                        

 36%|███▌      | 4321/12000 [2:34:16<4:33:37,  2.14s/it]
 36%|███▌      | 4322/12000 [2:34:17<4:18:36,  2.02s/it]
                                                        

 36%|███▌      | 4322/12000 [2:34:17<4:18:36,  2.02s/it]
 36%|███▌      | 4323/12000 [2:34:20<4:43:34,  2.22s/it]
                                                        

 36%|███▌      | 4323/12000 [2:34:20<4:43:34,  2.22s/it]
 36%|███▌      | 4324/12000 [2:34:23<5:21:05,  2.51s/it]
                                                        

 36%|███▌      | 4324/12000 [2:34:23<5:21:05,  2.51s/it]
 36%|███▌      | 4325/12000 [2:34:25<4:47:01,  2.24s/it]
                                                        

 36%|███▌      | 4325/12000 [2:34:25<4:47:01,  2.24s/it]
 36%|███▌      | 4326/12000 [2:34:27<4:50:02,  2.27s/it]
                                                        

 36%|███▌      | 4326/12000 [2:34:27<4:50:02,  2.27s/it]
 36%|███▌      | 4327/12000 [2:34:31<5:37:48,  2.64s/it]
                                                        

 36%|███▌      | 4327/12000 [2:34:31<5:37:48,  2.64s/it]
 36%|███▌      | 4328/12000 [2:34:33<5:21:44,  2.52s/it]
                                                        
{'loss': '0.9638', 'grad_norm': '0.444', 'learning_rate': '0.0001528', 'entropy': '0.8696', 'num_tokens': '2.318e+07', 'mean_token_accuracy': '0.7856', 'epoch': '0.03789'}
{'loss': '1.058', 'grad_norm': '0.4587', 'learning_rate': '0.0001528', 'entropy': '1.111', 'num_tokens': '2.318e+07', 'mean_token_accuracy': '0.7341', 'epoch': '0.03789'}
{'loss': '0.9714', 'grad_norm': '0.4155', 'learning_rate': '0.0001528', 'entropy': '0.9308', 'num_tokens': '2.319e+07', 'mean_token_accuracy': '0.7722', 'epoch': '0.0379'}
{'loss': '0.8055', 'grad_norm': '0.3661', 'learning_rate': '0.0001528', 'entropy': '0.8218', 'num_tokens': '2.319e+07', 'mean_token_accuracy': '0.7794', 'epoch': '0.03791'}
{'loss': '0.6427', 'grad_norm': '0.3463', 'learning_rate': '0.0001527', 'entropy': '0.6346', 'num_tokens': '2.32e+07', 'mean_token_accuracy': '0.83', 'epoch': '0.03792'}
{'loss': '0.4151', 'grad_norm': '0.2584', 'learning_rate': '0.0001527', 'entropy': '0.5257', 'num_tokens': '2.321e+07', 'mean_token_accuracy': '0.8465', 'epoch': '0.03793'}
{'loss': '0.7752', 'grad_norm': '0.3662', 'learning_rate': '0.0001527', 'entropy': '0.7731', 'num_tokens': '2.321e+07', 'mean_token_accuracy': '0.7953', 'epoch': '0.03794'}
{'loss': '0.6926', 'grad_norm': '0.3695', 'learning_rate': '0.0001527', 'entropy': '0.7712', 'num_tokens': '2.322e+07', 'mean_token_accuracy': '0.8015', 'epoch': '0.03795'}
{'loss': '0.6432', 'grad_norm': '0.3219', 'learning_rate': '0.0001526', 'entropy': '0.6681', 'num_tokens': '2.322e+07', 'mean_token_accuracy': '0.836', 'epoch': '0.03796'}
{'loss': '1.137', 'grad_norm': '0.4446', 'learning_rate': '0.0001526', 'entropy': '1.106', 'num_tokens': '2.323e+07', 'mean_token_accuracy': '0.7247', 'epoch': '0.03797'}
{'loss': '0.9775', 'grad_norm': '0.4007', 'learning_rate': '0.0001526', 'entropy': '0.936', 'num_tokens': '2.323e+07', 'mean_token_accuracy': '0.7734', 'epoch': '0.03797'}
{'loss': '0.8222', 'grad_norm': '0.3312', 'learning_rate': '0.0001526', 'entropy': '0.6984', 'num_tokens': '2.324e+07', 'mean_token_accuracy': '0.8353', 'epoch': '0.03798'}
{'loss': '1.183', 'grad_norm': '0.4589', 'learning_rate': '0.0001526', 'entropy': '1.136', 'num_tokens': '2.324e+07', 'mean_token_accuracy': '0.7401', 'epoch': '0.03799'}
{'loss': '1.028', 'grad_norm': '0.3133', 'learning_rate': '0.0001525', 'entropy': '0.8937', 'num_tokens': '2.325e+07', 'mean_token_accuracy': '0.7853', 'epoch': '0.038'}
{'loss': '0.5972', 'grad_norm': '0.2605', 'learning_rate': '0.0001525', 'entropy': '0.6376', 'num_tokens': '2.326e+07', 'mean_token_accuracy': '0.844', 'epoch': '0.03801'}
{'loss': '0.7097', 'grad_norm': '0.4068', 'learning_rate': '0.0001525', 'entropy': '0.772', 'num_tokens': '2.326e+07', 'mean_token_accuracy': '0.8075', 'epoch': '0.03802'}
{'loss': '1.253', 'grad_norm': '0.4402', 'learning_rate': '0.0001525', 'entropy': '1.063', 'num_tokens': '2.327e+07', 'mean_token_accuracy': '0.7474', 'epoch': '0.03803'}
{'loss': '1.464', 'grad_norm': '0.4978', 'learning_rate': '0.0001524', 'entropy': '1.421', 'num_tokens': '2.327e+07', 'mean_token_accuracy': '0.6752', 'epoch': '0.03804'}
{'loss': '0.4887', 'grad_norm': '0.3387', 'learning_rate': '0.0001524', 'entropy': '0.482', 'num_tokens': '2.328e+07', 'mean_token_accuracy': '0.869', 'epoch': '0.03805'}
{'loss': '0.7462', 'grad_norm': '0.2884', 'learning_rate': '0.0001524', 'entropy': '0.7412', 'num_tokens': '2.328e+07', 'mean_token_accuracy': '0.8085', 'epoch': '0.03805'}
{'loss': '0.7936', 'grad_norm': '0.338', 'learning_rate': '0.0001524', 'entropy': '0.7962', 'num_tokens': '2.329e+07', 'mean_token_accuracy': '0.8037', 'epoch': '0.03806'}
{'loss': '0.8293', 'grad_norm': '0.2823', 'learning_rate': '0.0001523', 'entropy': '0.7998', 'num_tokens': '2.33e+07', 'mean_token_accuracy': '0.795', 'epoch': '0.03807'}
{'loss': '0.8336', 'grad_norm': '0.3946', 'learning_rate': '0.0001523', 'entropy': '0.9008', 'num_tokens': '2.33e+07', 'mean_token_accuracy': '0.7857', 'epoch': '0.03808'}
{'loss': '1.032', 'grad_norm': '0.4065', 'learning_rate': '0.0001523', 'entropy': '1.077', 'num_tokens': '2.331e+07', 'mean_token_accuracy': '0.7448', 'epoch': '0.03809'}
{'loss': '0.7322', 'grad_norm': '0.2793', 'learning_rate': '0.0001523', 'entropy': '0.859', 'num_tokens': '2.332e+07', 'mean_token_accuracy': '0.7883', 'epoch': '0.0381'}
{'loss': '0.7013', 'grad_norm': '0.3308', 'learning_rate': '0.0001522', 'entropy': '0.752', 'num_tokens': '2.332e+07', 'mean_token_accuracy': '0.8055', 'epoch': '0.03811'}
{'loss': '1.036', 'grad_norm': '0.3448', 'learning_rate': '0.0001522', 'entropy': '1.158', 'num_tokens': '2.333e+07', 'mean_token_accuracy': '0.7228', 'epoch': '0.03812'}
{'loss': '0.776', 'grad_norm': '0.3419', 'learning_rate': '0.0001522', 'entropy': '0.7453', 'num_tokens': '2.334e+07', 'mean_token_accuracy': '0.8161', 'epoch': '0.03813'}
{'loss': '0.5326', 'grad_norm': '0.3256', 'learning_rate': '0.0001522', 'entropy': '0.5755', 'num_tokens': '2.334e+07', 'mean_token_accuracy': '0.8531', 'epoch': '0.03813'}
{'loss': '1.255', 'grad_norm': '0.4282', 'learning_rate': '0.0001522', 'entropy': '1.222', 'num_tokens': '2.335e+07', 'mean_token_accuracy': '0.6974', 'epoch': '0.03814'}
{'loss': '1.063', 'grad_norm': '0.3834', 'learning_rate': '0.0001521', 'entropy': '0.9858', 'num_tokens': '2.335e+07', 'mean_token_accuracy': '0.7578', 'epoch': '0.03815'}
{'loss': '0.6114', 'grad_norm': '0.3679', 'learning_rate': '0.0001521', 'entropy': '0.6158', 'num_tokens': '2.336e+07', 'mean_token_accuracy': '0.8309', 'epoch': '0.03816'}
{'loss': '0.7989', 'grad_norm': '0.4075', 'learning_rate': '0.0001521', 'entropy': '0.9257', 'num_tokens': '2.336e+07', 'mean_token_accuracy': '0.7638', 'epoch': '0.03817'}
{'loss': '0.7494', 'grad_norm': '0.3757', 'learning_rate': '0.0001521', 'entropy': '0.763', 'num_tokens': '2.337e+07', 'mean_token_accuracy': '0.8108', 'epoch': '0.03818'}
{'loss': '1.428', 'grad_norm': '0.5318', 'learning_rate': '0.000152', 'entropy': '1.261', 'num_tokens': '2.337e+07', 'mean_token_accuracy': '0.7035', 'epoch': '0.03819'}
{'loss': '0.7776', 'grad_norm': '0.3737', 'learning_rate': '0.000152', 'entropy': '0.7715', 'num_tokens': '2.338e+07', 'mean_token_accuracy': '0.7961', 'epoch': '0.0382'}
{'loss': '0.8853', 'grad_norm': '0.3526', 'learning_rate': '0.000152', 'entropy': '0.8634', 'num_tokens': '2.338e+07', 'mean_token_accuracy': '0.78', 'epoch': '0.0382'}
{'loss': '0.9028', 'grad_norm': '0.4056', 'learning_rate': '0.000152', 'entropy': '0.9071', 'num_tokens': '2.339e+07', 'mean_token_accuracy': '0.7678', 'epoch': '0.03821'}
{'loss': '0.9322', 'grad_norm': '0.376', 'learning_rate': '0.0001519', 'entropy': '0.9042', 'num_tokens': '2.339e+07', 'mean_token_accuracy': '0.7769', 'epoch': '0.03822'}
{'loss': '0.9879', 'grad_norm': '0.371', 'learning_rate': '0.0001519', 'entropy': '0.9971', 'num_tokens': '2.34e+07', 'mean_token_accuracy': '0.7856', 'epoch': '0.03823'}
{'loss': '0.7803', 'grad_norm': '0.3635', 'learning_rate': '0.0001519', 'entropy': '0.7081', 'num_tokens': '2.341e+07', 'mean_token_accuracy': '0.8127', 'epoch': '0.03824'}
{'loss': '1.027', 'grad_norm': '0.384', 'learning_rate': '0.0001519', 'entropy': '1.11', 'num_tokens': '2.341e+07', 'mean_token_accuracy': '0.7483', 'epoch': '0.03825'}
{'loss': '0.631', 'grad_norm': '0.3551', 'learning_rate': '0.0001518', 'entropy': '0.6528', 'num_tokens': '2.342e+07', 'mean_token_accuracy': '0.8297', 'epoch': '0.03826'}
{'loss': '0.7752', 'grad_norm': '0.3633', 'learning_rate': '0.0001518', 'entropy': '0.8063', 'num_tokens': '2.343e+07', 'mean_token_accuracy': '0.8023', 'epoch': '0.03827'}
{'loss': '0.849', 'grad_norm': '0.5466', 'learning_rate': '0.0001518', 'entropy': '0.8903', 'num_tokens': '2.343e+07', 'mean_token_accuracy': '0.7814', 'epoch': '0.03828'}
{'loss': '0.9118', 'grad_norm': '0.392', 'learning_rate': '0.0001518', 'entropy': '0.9374', 'num_tokens': '2.344e+07', 'mean_token_accuracy': '0.7773', 'epoch': '0.03828'}
{'loss': '0.7971', 'grad_norm': '0.3691', 'learning_rate': '0.0001518', 'entropy': '0.8618', 'num_tokens': '2.344e+07', 'mean_token_accuracy': '0.7884', 'epoch': '0.03829'}

 36%|███▌      | 4328/12000 [2:34:33<5:21:44,  2.52s/it]
 36%|███▌      | 4329/12000 [2:34:35<4:51:21,  2.28s/it]
                                                        

 36%|███▌      | 4329/12000 [2:34:35<4:51:21,  2.28s/it]
 36%|███▌      | 4330/12000 [2:34:37<4:59:38,  2.34s/it]
                                                        

 36%|███▌      | 4330/12000 [2:34:37<4:59:38,  2.34s/it]
 36%|███▌      | 4331/12000 [2:34:39<4:30:02,  2.11s/it]
                                                        

 36%|███▌      | 4331/12000 [2:34:39<4:30:02,  2.11s/it]
 36%|███▌      | 4332/12000 [2:34:41<4:30:58,  2.12s/it]
                                                        

 36%|███▌      | 4332/12000 [2:34:41<4:30:58,  2.12s/it]
 36%|███▌      | 4333/12000 [2:34:44<4:55:08,  2.31s/it]
                                                        

 36%|███▌      | 4333/12000 [2:34:44<4:55:08,  2.31s/it]
 36%|███▌      | 4334/12000 [2:34:45<4:25:51,  2.08s/it]
                                                        

 36%|███▌      | 4334/12000 [2:34:45<4:25:51,  2.08s/it]
 36%|███▌      | 4335/12000 [2:34:48<4:51:12,  2.28s/it]
                                                        

 36%|███▌      | 4335/12000 [2:34:48<4:51:12,  2.28s/it]
 36%|███▌      | 4336/12000 [2:34:51<5:08:33,  2.42s/it]
                                                        

 36%|███▌      | 4336/12000 [2:34:51<5:08:33,  2.42s/it]
 36%|███▌      | 4337/12000 [2:34:53<5:21:26,  2.52s/it]
                                                        

 36%|███▌      | 4337/12000 [2:34:53<5:21:26,  2.52s/it]
 36%|███▌      | 4338/12000 [2:34:56<5:11:02,  2.44s/it]
                                                        

 36%|███▌      | 4338/12000 [2:34:56<5:11:02,  2.44s/it]
 36%|███▌      | 4339/12000 [2:34:58<4:49:55,  2.27s/it]
                                                        

 36%|███▌      | 4339/12000 [2:34:58<4:49:55,  2.27s/it]
 36%|███▌      | 4340/12000 [2:34:59<4:28:19,  2.10s/it]
                                                        

 36%|███▌      | 4340/12000 [2:34:59<4:28:19,  2.10s/it]
 36%|███▌      | 4341/12000 [2:35:01<4:11:24,  1.97s/it]
                                                        

 36%|███▌      | 4341/12000 [2:35:01<4:11:24,  1.97s/it]
 36%|███▌      | 4342/12000 [2:35:03<4:02:14,  1.90s/it]
                                                        

 36%|███▌      | 4342/12000 [2:35:03<4:02:14,  1.90s/it]
 36%|███▌      | 4343/12000 [2:35:04<3:55:09,  1.84s/it]
                                                        

 36%|███▌      | 4343/12000 [2:35:04<3:55:09,  1.84s/it]
 36%|███▌      | 4344/12000 [2:35:06<3:54:21,  1.84s/it]
                                                        

 36%|███▌      | 4344/12000 [2:35:06<3:54:21,  1.84s/it]
 36%|███▌      | 4345/12000 [2:35:08<3:43:58,  1.76s/it]
                                                        

 36%|███▌      | 4345/12000 [2:35:08<3:43:58,  1.76s/it]
 36%|███▌      | 4346/12000 [2:35:10<3:44:53,  1.76s/it]
                                                        

 36%|███▌      | 4346/12000 [2:35:10<3:44:53,  1.76s/it]
 36%|███▌      | 4347/12000 [2:35:12<4:18:53,  2.03s/it]
                                                        

 36%|███▌      | 4347/12000 [2:35:12<4:18:53,  2.03s/it]
 36%|███▌      | 4348/12000 [2:35:14<4:03:38,  1.91s/it]
                                                        

 36%|███▌      | 4348/12000 [2:35:14<4:03:38,  1.91s/it]
 36%|███▌      | 4349/12000 [2:35:18<5:19:26,  2.51s/it]
                                                        

 36%|███▌      | 4349/12000 [2:35:18<5:19:26,  2.51s/it]
 36%|███▋      | 4350/12000 [2:35:19<4:45:48,  2.24s/it]
                                                        

 36%|███▋      | 4350/12000 [2:35:19<4:45:48,  2.24s/it]
 36%|███▋      | 4351/12000 [2:35:22<5:14:46,  2.47s/it]
                                                        

 36%|███▋      | 4351/12000 [2:35:22<5:14:46,  2.47s/it]
 36%|███▋      | 4352/12000 [2:35:24<4:52:39,  2.30s/it]
                                                        

 36%|███▋      | 4352/12000 [2:35:24<4:52:39,  2.30s/it]
 36%|███▋      | 4353/12000 [2:35:26<4:26:46,  2.09s/it]
                                                        

 36%|███▋      | 4353/12000 [2:35:26<4:26:46,  2.09s/it]
 36%|███▋      | 4354/12000 [2:35:29<5:14:04,  2.46s/it]
                                                        

 36%|███▋      | 4354/12000 [2:35:29<5:14:04,  2.46s/it]
 36%|███▋      | 4355/12000 [2:35:31<4:42:33,  2.22s/it]
                                                        

 36%|███▋      | 4355/12000 [2:35:31<4:42:33,  2.22s/it]
 36%|███▋      | 4356/12000 [2:35:32<4:19:20,  2.04s/it]
                                                        

 36%|███▋      | 4356/12000 [2:35:32<4:19:20,  2.04s/it]
 36%|███▋      | 4357/12000 [2:35:34<4:16:48,  2.02s/it]
                                                        

 36%|███▋      | 4357/12000 [2:35:34<4:16:48,  2.02s/it]
 36%|███▋      | 4358/12000 [2:35:36<4:03:14,  1.91s/it]
                                                        

 36%|███▋      | 4358/12000 [2:35:36<4:03:14,  1.91s/it]
 36%|███▋      | 4359/12000 [2:35:39<4:33:39,  2.15s/it]
                                                        

 36%|███▋      | 4359/12000 [2:35:39<4:33:39,  2.15s/it]
 36%|███▋      | 4360/12000 [2:35:40<4:13:58,  1.99s/it]
                                                        

 36%|███▋      | 4360/12000 [2:35:40<4:13:58,  1.99s/it]
 36%|███▋      | 4361/12000 [2:35:42<4:13:28,  1.99s/it]
                                                        

 36%|███▋      | 4361/12000 [2:35:42<4:13:28,  1.99s/it]
 36%|███▋      | 4362/12000 [2:35:44<4:12:23,  1.98s/it]
                                                        

 36%|███▋      | 4362/12000 [2:35:44<4:12:23,  1.98s/it]
 36%|███▋      | 4363/12000 [2:35:46<3:55:27,  1.85s/it]
                                                        

 36%|███▋      | 4363/12000 [2:35:46<3:55:27,  1.85s/it]
 36%|███▋      | 4364/12000 [2:35:48<3:45:43,  1.77s/it]
                                                        

 36%|███▋      | 4364/12000 [2:35:48<3:45:43,  1.77s/it]
 36%|███▋      | 4365/12000 [2:35:49<3:40:19,  1.73s/it]
                                                        

 36%|███▋      | 4365/12000 [2:35:49<3:40:19,  1.73s/it]
 36%|███▋      | 4366/12000 [2:35:51<3:33:52,  1.68s/it]
                                                        

 36%|███▋      | 4366/12000 [2:35:51<3:33:52,  1.68s/it]
 36%|███▋      | 4367/12000 [2:35:53<3:55:48,  1.85s/it]
                                                        

 36%|███▋      | 4367/12000 [2:35:53<3:55:48,  1.85s/it]
 36%|███▋      | 4368/12000 [2:35:55<3:44:45,  1.77s/it]
                                                        

 36%|███▋      | 4368/12000 [2:35:55<3:44:45,  1.77s/it]
 36%|███▋      | 4369/12000 [2:35:57<4:10:06,  1.97s/it]
                                                        

 36%|███▋      | 4369/12000 [2:35:57<4:10:06,  1.97s/it]
 36%|███▋      | 4370/12000 [2:35:59<3:54:52,  1.85s/it]
                                                        

 36%|███▋      | 4370/12000 [2:35:59<3:54:52,  1.85s/it]
 36%|███▋      | 4371/12000 [2:36:02<5:06:28,  2.41s/it]
                                                        

 36%|███▋      | 4371/12000 [2:36:02<5:06:28,  2.41s/it]
 36%|███▋      | 4372/12000 [2:36:04<4:46:45,  2.26s/it]
                                                        

 36%|███▋      | 4372/12000 [2:36:04<4:46:45,  2.26s/it]
 36%|███▋      | 4373/12000 [2:36:06<4:27:18,  2.10s/it]
                                                        

 36%|███▋      | 4373/12000 [2:36:06<4:27:18,  2.10s/it]
 36%|███▋      | 4374/12000 [2:36:08<4:21:59,  2.06s/it]
                                                        

 36%|███▋      | 4374/12000 [2:36:08<4:21:59,  2.06s/it]
 36%|███▋      | 4375/12000 [2:36:10<4:06:48,  1.94s/it]
                                                        
{'loss': '0.7506', 'grad_norm': '0.3657', 'learning_rate': '0.0001517', 'entropy': '0.8255', 'num_tokens': '2.344e+07', 'mean_token_accuracy': '0.8009', 'epoch': '0.0383'}
{'loss': '0.9346', 'grad_norm': '0.4162', 'learning_rate': '0.0001517', 'entropy': '0.9473', 'num_tokens': '2.345e+07', 'mean_token_accuracy': '0.7593', 'epoch': '0.03831'}
{'loss': '1.475', 'grad_norm': '0.4659', 'learning_rate': '0.0001517', 'entropy': '1.39', 'num_tokens': '2.346e+07', 'mean_token_accuracy': '0.6652', 'epoch': '0.03832'}
{'loss': '0.7041', 'grad_norm': '0.3619', 'learning_rate': '0.0001517', 'entropy': '0.6995', 'num_tokens': '2.346e+07', 'mean_token_accuracy': '0.8118', 'epoch': '0.03833'}
{'loss': '1.096', 'grad_norm': '0.3692', 'learning_rate': '0.0001516', 'entropy': '0.952', 'num_tokens': '2.347e+07', 'mean_token_accuracy': '0.7719', 'epoch': '0.03834'}
{'loss': '0.8641', 'grad_norm': '0.3687', 'learning_rate': '0.0001516', 'entropy': '0.8457', 'num_tokens': '2.347e+07', 'mean_token_accuracy': '0.7807', 'epoch': '0.03835'}
{'loss': '0.8075', 'grad_norm': '0.4662', 'learning_rate': '0.0001516', 'entropy': '0.7344', 'num_tokens': '2.348e+07', 'mean_token_accuracy': '0.8077', 'epoch': '0.03836'}
{'loss': '0.8522', 'grad_norm': '0.3545', 'learning_rate': '0.0001516', 'entropy': '0.8257', 'num_tokens': '2.348e+07', 'mean_token_accuracy': '0.8012', 'epoch': '0.03836'}
{'loss': '0.7817', 'grad_norm': '0.3651', 'learning_rate': '0.0001515', 'entropy': '0.8685', 'num_tokens': '2.349e+07', 'mean_token_accuracy': '0.7915', 'epoch': '0.03837'}
{'loss': '0.7157', 'grad_norm': '0.3054', 'learning_rate': '0.0001515', 'entropy': '0.7144', 'num_tokens': '2.35e+07', 'mean_token_accuracy': '0.8215', 'epoch': '0.03838'}
{'loss': '1.365', 'grad_norm': '0.4614', 'learning_rate': '0.0001515', 'entropy': '1.365', 'num_tokens': '2.35e+07', 'mean_token_accuracy': '0.6774', 'epoch': '0.03839'}
{'loss': '1.017', 'grad_norm': '0.5026', 'learning_rate': '0.0001515', 'entropy': '1.085', 'num_tokens': '2.351e+07', 'mean_token_accuracy': '0.7383', 'epoch': '0.0384'}
{'loss': '0.7932', 'grad_norm': '0.4431', 'learning_rate': '0.0001514', 'entropy': '0.9156', 'num_tokens': '2.351e+07', 'mean_token_accuracy': '0.7728', 'epoch': '0.03841'}
{'loss': '0.7537', 'grad_norm': '0.3398', 'learning_rate': '0.0001514', 'entropy': '0.7585', 'num_tokens': '2.352e+07', 'mean_token_accuracy': '0.7998', 'epoch': '0.03842'}
{'loss': '0.9456', 'grad_norm': '0.5165', 'learning_rate': '0.0001514', 'entropy': '0.9376', 'num_tokens': '2.352e+07', 'mean_token_accuracy': '0.7547', 'epoch': '0.03843'}
{'loss': '0.8045', 'grad_norm': '0.4194', 'learning_rate': '0.0001514', 'entropy': '0.8655', 'num_tokens': '2.353e+07', 'mean_token_accuracy': '0.7814', 'epoch': '0.03843'}
{'loss': '0.6754', 'grad_norm': '0.412', 'learning_rate': '0.0001514', 'entropy': '0.675', 'num_tokens': '2.353e+07', 'mean_token_accuracy': '0.8122', 'epoch': '0.03844'}
{'loss': '0.7566', 'grad_norm': '0.3097', 'learning_rate': '0.0001513', 'entropy': '0.7854', 'num_tokens': '2.354e+07', 'mean_token_accuracy': '0.8035', 'epoch': '0.03845'}
{'loss': '1.255', 'grad_norm': '0.4913', 'learning_rate': '0.0001513', 'entropy': '1.121', 'num_tokens': '2.354e+07', 'mean_token_accuracy': '0.7336', 'epoch': '0.03846'}
{'loss': '0.9362', 'grad_norm': '0.3646', 'learning_rate': '0.0001513', 'entropy': '0.948', 'num_tokens': '2.355e+07', 'mean_token_accuracy': '0.7716', 'epoch': '0.03847'}
{'loss': '0.7099', 'grad_norm': '0.3472', 'learning_rate': '0.0001513', 'entropy': '0.6879', 'num_tokens': '2.355e+07', 'mean_token_accuracy': '0.8241', 'epoch': '0.03848'}
{'loss': '0.7035', 'grad_norm': '0.3751', 'learning_rate': '0.0001512', 'entropy': '0.6915', 'num_tokens': '2.356e+07', 'mean_token_accuracy': '0.8369', 'epoch': '0.03849'}
{'loss': '0.8651', 'grad_norm': '0.3332', 'learning_rate': '0.0001512', 'entropy': '0.8846', 'num_tokens': '2.356e+07', 'mean_token_accuracy': '0.787', 'epoch': '0.0385'}
{'loss': '1.017', 'grad_norm': '0.4164', 'learning_rate': '0.0001512', 'entropy': '1.003', 'num_tokens': '2.357e+07', 'mean_token_accuracy': '0.7536', 'epoch': '0.03851'}
{'loss': '1.221', 'grad_norm': '0.5548', 'learning_rate': '0.0001512', 'entropy': '1.18', 'num_tokens': '2.357e+07', 'mean_token_accuracy': '0.7361', 'epoch': '0.03851'}
{'loss': '1.176', 'grad_norm': '0.4575', 'learning_rate': '0.0001511', 'entropy': '1.339', 'num_tokens': '2.358e+07', 'mean_token_accuracy': '0.6889', 'epoch': '0.03852'}
{'loss': '0.6639', 'grad_norm': '0.3163', 'learning_rate': '0.0001511', 'entropy': '0.697', 'num_tokens': '2.358e+07', 'mean_token_accuracy': '0.8244', 'epoch': '0.03853'}
{'loss': '0.9256', 'grad_norm': '0.4063', 'learning_rate': '0.0001511', 'entropy': '0.8782', 'num_tokens': '2.359e+07', 'mean_token_accuracy': '0.7983', 'epoch': '0.03854'}
{'loss': '0.8631', 'grad_norm': '0.5145', 'learning_rate': '0.0001511', 'entropy': '0.8896', 'num_tokens': '2.359e+07', 'mean_token_accuracy': '0.7695', 'epoch': '0.03855'}
{'loss': '1.197', 'grad_norm': '0.4295', 'learning_rate': '0.000151', 'entropy': '1.142', 'num_tokens': '2.36e+07', 'mean_token_accuracy': '0.7334', 'epoch': '0.03856'}
{'loss': '0.8272', 'grad_norm': '0.4725', 'learning_rate': '0.000151', 'entropy': '0.8488', 'num_tokens': '2.36e+07', 'mean_token_accuracy': '0.7864', 'epoch': '0.03857'}
{'loss': '1.004', 'grad_norm': '0.4545', 'learning_rate': '0.000151', 'entropy': '0.994', 'num_tokens': '2.361e+07', 'mean_token_accuracy': '0.7625', 'epoch': '0.03858'}
{'loss': '0.7598', 'grad_norm': '0.3989', 'learning_rate': '0.000151', 'entropy': '0.7813', 'num_tokens': '2.361e+07', 'mean_token_accuracy': '0.8092', 'epoch': '0.03859'}
{'loss': '0.7224', 'grad_norm': '0.4011', 'learning_rate': '0.000151', 'entropy': '0.6894', 'num_tokens': '2.362e+07', 'mean_token_accuracy': '0.814', 'epoch': '0.03859'}
{'loss': '0.6592', 'grad_norm': '0.3559', 'learning_rate': '0.0001509', 'entropy': '0.6762', 'num_tokens': '2.362e+07', 'mean_token_accuracy': '0.8206', 'epoch': '0.0386'}
{'loss': '0.6049', 'grad_norm': '0.3607', 'learning_rate': '0.0001509', 'entropy': '0.6516', 'num_tokens': '2.363e+07', 'mean_token_accuracy': '0.8332', 'epoch': '0.03861'}
{'loss': '0.9479', 'grad_norm': '0.3916', 'learning_rate': '0.0001509', 'entropy': '0.9344', 'num_tokens': '2.363e+07', 'mean_token_accuracy': '0.7565', 'epoch': '0.03862'}
{'loss': '0.9697', 'grad_norm': '0.4005', 'learning_rate': '0.0001509', 'entropy': '1.007', 'num_tokens': '2.364e+07', 'mean_token_accuracy': '0.7517', 'epoch': '0.03863'}
{'loss': '0.7969', 'grad_norm': '0.3922', 'learning_rate': '0.0001508', 'entropy': '0.7724', 'num_tokens': '2.364e+07', 'mean_token_accuracy': '0.8035', 'epoch': '0.03864'}
{'loss': '0.7566', 'grad_norm': '0.3331', 'learning_rate': '0.0001508', 'entropy': '0.8414', 'num_tokens': '2.365e+07', 'mean_token_accuracy': '0.8043', 'epoch': '0.03865'}
{'loss': '0.9455', 'grad_norm': '0.399', 'learning_rate': '0.0001508', 'entropy': '0.8827', 'num_tokens': '2.365e+07', 'mean_token_accuracy': '0.767', 'epoch': '0.03866'}
{'loss': '0.5736', 'grad_norm': '0.4703', 'learning_rate': '0.0001508', 'entropy': '0.6899', 'num_tokens': '2.366e+07', 'mean_token_accuracy': '0.8241', 'epoch': '0.03866'}
{'loss': '0.6962', 'grad_norm': '0.3333', 'learning_rate': '0.0001507', 'entropy': '0.7154', 'num_tokens': '2.366e+07', 'mean_token_accuracy': '0.8111', 'epoch': '0.03867'}
{'loss': '0.9079', 'grad_norm': '0.4434', 'learning_rate': '0.0001507', 'entropy': '0.9038', 'num_tokens': '2.367e+07', 'mean_token_accuracy': '0.7722', 'epoch': '0.03868'}
{'loss': '0.8365', 'grad_norm': '0.3717', 'learning_rate': '0.0001507', 'entropy': '0.8586', 'num_tokens': '2.367e+07', 'mean_token_accuracy': '0.7879', 'epoch': '0.03869'}
{'loss': '1.013', 'grad_norm': '0.4136', 'learning_rate': '0.0001507', 'entropy': '1.027', 'num_tokens': '2.368e+07', 'mean_token_accuracy': '0.7657', 'epoch': '0.0387'}
{'loss': '1.061', 'grad_norm': '0.3246', 'learning_rate': '0.0001506', 'entropy': '1.075', 'num_tokens': '2.368e+07', 'mean_token_accuracy': '0.7511', 'epoch': '0.03871'}

 36%|███▋      | 4375/12000 [2:36:10<4:06:48,  1.94s/it]
 36%|███▋      | 4376/12000 [2:36:12<4:18:48,  2.04s/it]
                                                        

 36%|███▋      | 4376/12000 [2:36:12<4:18:48,  2.04s/it]
 36%|███▋      | 4377/12000 [2:36:13<4:04:06,  1.92s/it]
                                                        

 36%|███▋      | 4377/12000 [2:36:13<4:04:06,  1.92s/it]
 36%|███▋      | 4378/12000 [2:36:15<3:53:25,  1.84s/it]
                                                        

 36%|███▋      | 4378/12000 [2:36:15<3:53:25,  1.84s/it]
 36%|███▋      | 4379/12000 [2:36:18<4:15:35,  2.01s/it]
                                                        

 36%|███▋      | 4379/12000 [2:36:18<4:15:35,  2.01s/it]
 36%|███▋      | 4380/12000 [2:36:21<4:53:16,  2.31s/it]
                                                        

 36%|███▋      | 4380/12000 [2:36:21<4:53:16,  2.31s/it]
 37%|███▋      | 4381/12000 [2:36:23<5:16:13,  2.49s/it]
                                                        

 37%|███▋      | 4381/12000 [2:36:23<5:16:13,  2.49s/it]
 37%|███▋      | 4382/12000 [2:36:25<4:58:46,  2.35s/it]
                                                        

 37%|███▋      | 4382/12000 [2:36:25<4:58:46,  2.35s/it]
 37%|███▋      | 4383/12000 [2:36:33<8:14:50,  3.90s/it]
                                                        

 37%|███▋      | 4383/12000 [2:36:33<8:14:50,  3.90s/it]
 37%|███▋      | 4384/12000 [2:36:34<6:45:13,  3.19s/it]
                                                        

 37%|███▋      | 4384/12000 [2:36:34<6:45:13,  3.19s/it]
 37%|███▋      | 4385/12000 [2:36:36<5:47:00,  2.73s/it]
                                                        

 37%|███▋      | 4385/12000 [2:36:36<5:47:00,  2.73s/it]
 37%|███▋      | 4386/12000 [2:36:38<5:17:49,  2.50s/it]
                                                        

 37%|███▋      | 4386/12000 [2:36:38<5:17:49,  2.50s/it]
 37%|███▋      | 4387/12000 [2:36:40<4:48:30,  2.27s/it]
                                                        

 37%|███▋      | 4387/12000 [2:36:40<4:48:30,  2.27s/it]
 37%|███▋      | 4388/12000 [2:36:41<4:23:07,  2.07s/it]
                                                        

 37%|███▋      | 4388/12000 [2:36:41<4:23:07,  2.07s/it]
 37%|███▋      | 4389/12000 [2:36:45<5:05:03,  2.40s/it]
                                                        

 37%|███▋      | 4389/12000 [2:36:45<5:05:03,  2.40s/it]
 37%|███▋      | 4390/12000 [2:36:46<4:43:07,  2.23s/it]
                                                        

 37%|███▋      | 4390/12000 [2:36:46<4:43:07,  2.23s/it]
 37%|███▋      | 4391/12000 [2:36:49<4:51:36,  2.30s/it]
                                                        

 37%|███▋      | 4391/12000 [2:36:49<4:51:36,  2.30s/it]
 37%|███▋      | 4392/12000 [2:36:51<4:24:20,  2.08s/it]
                                                        

 37%|███▋      | 4392/12000 [2:36:51<4:24:20,  2.08s/it]
 37%|███▋      | 4393/12000 [2:36:55<5:45:30,  2.73s/it]
                                                        

 37%|███▋      | 4393/12000 [2:36:55<5:45:30,  2.73s/it]
 37%|███▋      | 4394/12000 [2:36:56<4:59:09,  2.36s/it]
                                                        

 37%|███▋      | 4394/12000 [2:36:56<4:59:09,  2.36s/it]
 37%|███▋      | 4395/12000 [2:36:58<4:36:20,  2.18s/it]
                                                        

 37%|███▋      | 4395/12000 [2:36:58<4:36:20,  2.18s/it]
 37%|███▋      | 4396/12000 [2:37:01<4:55:01,  2.33s/it]
                                                        

 37%|███▋      | 4396/12000 [2:37:01<4:55:01,  2.33s/it]
 37%|███▋      | 4397/12000 [2:37:03<4:39:43,  2.21s/it]
                                                        

 37%|███▋      | 4397/12000 [2:37:03<4:39:43,  2.21s/it]
 37%|███▋      | 4398/12000 [2:37:05<4:53:21,  2.32s/it]
                                                        

 37%|███▋      | 4398/12000 [2:37:05<4:53:21,  2.32s/it]
 37%|███▋      | 4399/12000 [2:37:08<5:05:47,  2.41s/it]
                                                        

 37%|███▋      | 4399/12000 [2:37:08<5:05:47,  2.41s/it]
 37%|███▋      | 4400/12000 [2:37:10<4:45:57,  2.26s/it]
                                                        

 37%|███▋      | 4400/12000 [2:37:10<4:45:57,  2.26s/it]
 37%|███▋      | 4401/12000 [2:37:13<5:23:23,  2.55s/it]
                                                        

 37%|███▋      | 4401/12000 [2:37:13<5:23:23,  2.55s/it]
 37%|███▋      | 4402/12000 [2:37:16<5:23:23,  2.55s/it]
                                                        

 37%|███▋      | 4402/12000 [2:37:16<5:23:23,  2.55s/it]
 37%|███▋      | 4403/12000 [2:37:17<4:47:32,  2.27s/it]
                                                        

 37%|███▋      | 4403/12000 [2:37:17<4:47:32,  2.27s/it]
 37%|███▋      | 4404/12000 [2:37:19<4:18:21,  2.04s/it]
                                                        

 37%|███▋      | 4404/12000 [2:37:19<4:18:21,  2.04s/it]
 37%|███▋      | 4405/12000 [2:37:21<4:43:15,  2.24s/it]
                                                        

 37%|███▋      | 4405/12000 [2:37:21<4:43:15,  2.24s/it]
 37%|███▋      | 4406/12000 [2:37:23<4:24:29,  2.09s/it]
                                                        

 37%|███▋      | 4406/12000 [2:37:23<4:24:29,  2.09s/it]
 37%|███▋      | 4407/12000 [2:37:26<5:00:17,  2.37s/it]
                                                        

 37%|███▋      | 4407/12000 [2:37:26<5:00:17,  2.37s/it]
 37%|███▋      | 4408/12000 [2:37:28<4:50:23,  2.29s/it]
                                                        

 37%|███▋      | 4408/12000 [2:37:28<4:50:23,  2.29s/it]
 37%|███▋      | 4409/12000 [2:37:30<4:24:11,  2.09s/it]
                                                        

 37%|███▋      | 4409/12000 [2:37:30<4:24:11,  2.09s/it]
 37%|███▋      | 4410/12000 [2:37:32<4:25:49,  2.10s/it]
                                                        

 37%|███▋      | 4410/12000 [2:37:32<4:25:49,  2.10s/it]
 37%|███▋      | 4411/12000 [2:37:34<4:16:06,  2.02s/it]
                                                        

 37%|███▋      | 4411/12000 [2:37:34<4:16:06,  2.02s/it]
 37%|███▋      | 4412/12000 [2:37:37<5:01:44,  2.39s/it]
                                                        

 37%|███▋      | 4412/12000 [2:37:37<5:01:44,  2.39s/it]
 37%|███▋      | 4413/12000 [2:37:40<5:26:34,  2.58s/it]
                                                        

 37%|███▋      | 4413/12000 [2:37:40<5:26:34,  2.58s/it]
 37%|███▋      | 4414/12000 [2:37:42<4:55:50,  2.34s/it]
                                                        

 37%|███▋      | 4414/12000 [2:37:42<4:55:50,  2.34s/it]
 37%|███▋      | 4415/12000 [2:37:46<5:50:14,  2.77s/it]
                                                        

 37%|███▋      | 4415/12000 [2:37:46<5:50:14,  2.77s/it]
 37%|███▋      | 4416/12000 [2:37:47<5:10:04,  2.45s/it]
                                                        

 37%|███▋      | 4416/12000 [2:37:47<5:10:04,  2.45s/it]
 37%|███▋      | 4417/12000 [2:37:50<4:59:20,  2.37s/it]
                                                        

 37%|███▋      | 4417/12000 [2:37:50<4:59:20,  2.37s/it]
 37%|███▋      | 4418/12000 [2:37:52<4:48:15,  2.28s/it]
                                                        

 37%|███▋      | 4418/12000 [2:37:52<4:48:15,  2.28s/it]
 37%|███▋      | 4419/12000 [2:37:54<4:49:22,  2.29s/it]
                                                        

 37%|███▋      | 4419/12000 [2:37:54<4:49:22,  2.29s/it]
 37%|███▋      | 4420/12000 [2:37:56<4:41:48,  2.23s/it]
                                                        

 37%|███▋      | 4420/12000 [2:37:56<4:41:48,  2.23s/it]
 37%|███▋      | 4421/12000 [2:37:58<4:37:05,  2.19s/it]
                                                        

 37%|███▋      | 4421/12000 [2:37:58<4:37:05,  2.19s/it]
 37%|███▋      | 4422/12000 [2:38:04<6:59:34,  3.32s/it]
                                                        
{'loss': '0.9391', 'grad_norm': '0.4319', 'learning_rate': '0.0001506', 'entropy': '0.9116', 'num_tokens': '2.369e+07', 'mean_token_accuracy': '0.7844', 'epoch': '0.03872'}
{'loss': '0.5881', 'grad_norm': '0.3564', 'learning_rate': '0.0001506', 'entropy': '0.643', 'num_tokens': '2.369e+07', 'mean_token_accuracy': '0.8355', 'epoch': '0.03873'}
{'loss': '0.8144', 'grad_norm': '0.3946', 'learning_rate': '0.0001506', 'entropy': '0.8157', 'num_tokens': '2.37e+07', 'mean_token_accuracy': '0.79', 'epoch': '0.03874'}
{'loss': '0.8919', 'grad_norm': '0.3738', 'learning_rate': '0.0001505', 'entropy': '0.9225', 'num_tokens': '2.37e+07', 'mean_token_accuracy': '0.7767', 'epoch': '0.03874'}
{'loss': '0.9092', 'grad_norm': '0.3791', 'learning_rate': '0.0001505', 'entropy': '0.9797', 'num_tokens': '2.371e+07', 'mean_token_accuracy': '0.7598', 'epoch': '0.03875'}
{'loss': '0.7071', 'grad_norm': '0.3396', 'learning_rate': '0.0001505', 'entropy': '0.7597', 'num_tokens': '2.371e+07', 'mean_token_accuracy': '0.8146', 'epoch': '0.03876'}
{'loss': '0.8181', 'grad_norm': '0.4445', 'learning_rate': '0.0001505', 'entropy': '0.896', 'num_tokens': '2.372e+07', 'mean_token_accuracy': '0.7852', 'epoch': '0.03877'}
{'loss': '1.027', 'grad_norm': '0.397', 'learning_rate': '0.0001505', 'entropy': '0.9684', 'num_tokens': '2.373e+07', 'mean_token_accuracy': '0.7418', 'epoch': '0.03878'}
{'loss': '0.9023', 'grad_norm': '0.3372', 'learning_rate': '0.0001504', 'entropy': '0.8523', 'num_tokens': '2.374e+07', 'mean_token_accuracy': '0.7694', 'epoch': '0.03879'}
{'loss': '0.7318', 'grad_norm': '0.3942', 'learning_rate': '0.0001504', 'entropy': '0.7406', 'num_tokens': '2.374e+07', 'mean_token_accuracy': '0.8131', 'epoch': '0.0388'}
{'loss': '0.8503', 'grad_norm': '0.3763', 'learning_rate': '0.0001504', 'entropy': '0.8357', 'num_tokens': '2.375e+07', 'mean_token_accuracy': '0.7876', 'epoch': '0.03881'}
{'loss': '0.6988', 'grad_norm': '0.4043', 'learning_rate': '0.0001504', 'entropy': '0.6438', 'num_tokens': '2.375e+07', 'mean_token_accuracy': '0.8377', 'epoch': '0.03882'}
{'loss': '0.9776', 'grad_norm': '0.4908', 'learning_rate': '0.0001503', 'entropy': '0.8971', 'num_tokens': '2.376e+07', 'mean_token_accuracy': '0.7671', 'epoch': '0.03882'}
{'loss': '0.6792', 'grad_norm': '0.4012', 'learning_rate': '0.0001503', 'entropy': '0.7474', 'num_tokens': '2.376e+07', 'mean_token_accuracy': '0.8104', 'epoch': '0.03883'}
{'loss': '0.6738', 'grad_norm': '0.2767', 'learning_rate': '0.0001503', 'entropy': '0.8008', 'num_tokens': '2.377e+07', 'mean_token_accuracy': '0.7891', 'epoch': '0.03884'}
{'loss': '0.9553', 'grad_norm': '0.4358', 'learning_rate': '0.0001503', 'entropy': '0.9261', 'num_tokens': '2.377e+07', 'mean_token_accuracy': '0.7712', 'epoch': '0.03885'}
{'loss': '1.044', 'grad_norm': '0.3432', 'learning_rate': '0.0001502', 'entropy': '1.032', 'num_tokens': '2.378e+07', 'mean_token_accuracy': '0.7458', 'epoch': '0.03886'}
{'loss': '0.7606', 'grad_norm': '0.3842', 'learning_rate': '0.0001502', 'entropy': '0.8631', 'num_tokens': '2.379e+07', 'mean_token_accuracy': '0.7839', 'epoch': '0.03887'}
{'loss': '0.9411', 'grad_norm': '0.3609', 'learning_rate': '0.0001502', 'entropy': '0.8358', 'num_tokens': '2.379e+07', 'mean_token_accuracy': '0.8', 'epoch': '0.03888'}
{'loss': '0.9113', 'grad_norm': '0.4948', 'learning_rate': '0.0001502', 'entropy': '0.9513', 'num_tokens': '2.379e+07', 'mean_token_accuracy': '0.7546', 'epoch': '0.03889'}
{'loss': '0.6939', 'grad_norm': '0.3891', 'learning_rate': '0.0001501', 'entropy': '0.7547', 'num_tokens': '2.38e+07', 'mean_token_accuracy': '0.8115', 'epoch': '0.0389'}
{'loss': '0.7298', 'grad_norm': '0.354', 'learning_rate': '0.0001501', 'entropy': '0.8642', 'num_tokens': '2.381e+07', 'mean_token_accuracy': '0.8015', 'epoch': '0.0389'}
{'loss': '0.9609', 'grad_norm': '0.3683', 'learning_rate': '0.0001501', 'entropy': '0.9642', 'num_tokens': '2.381e+07', 'mean_token_accuracy': '0.767', 'epoch': '0.03891'}
{'loss': '0.7044', 'grad_norm': '0.3372', 'learning_rate': '0.0001501', 'entropy': '0.6995', 'num_tokens': '2.382e+07', 'mean_token_accuracy': '0.8213', 'epoch': '0.03892'}
{'loss': '0.7357', 'grad_norm': '0.3502', 'learning_rate': '0.00015', 'entropy': '0.7127', 'num_tokens': '2.382e+07', 'mean_token_accuracy': '0.8204', 'epoch': '0.03893'}
{'loss': '1.135', 'grad_norm': '0.3554', 'learning_rate': '0.00015', 'entropy': '1.126', 'num_tokens': '2.383e+07', 'mean_token_accuracy': '0.7363', 'epoch': '0.03894'}
{'loss': '0.6697', 'grad_norm': '0.418', 'learning_rate': '0.00015', 'entropy': '0.697', 'num_tokens': '2.384e+07', 'mean_token_accuracy': '0.8165', 'epoch': '0.03895'}
{'loss': '0.9116', 'grad_norm': '0.3898', 'learning_rate': '0.00015', 'entropy': '0.8875', 'num_tokens': '2.384e+07', 'mean_token_accuracy': '0.777', 'epoch': '0.03896'}
{'loss': '0.7957', 'grad_norm': '0.319', 'learning_rate': '0.00015', 'entropy': '0.8222', 'num_tokens': '2.385e+07', 'mean_token_accuracy': '0.8062', 'epoch': '0.03897'}
{'loss': '0.7073', 'grad_norm': '0.3696', 'learning_rate': '0.0001499', 'entropy': '0.7125', 'num_tokens': '2.385e+07', 'mean_token_accuracy': '0.7993', 'epoch': '0.03897'}
{'loss': '0.6111', 'grad_norm': '0.369', 'learning_rate': '0.0001499', 'entropy': '0.6038', 'num_tokens': '2.386e+07', 'mean_token_accuracy': '0.8347', 'epoch': '0.03898'}
{'loss': '0.8828', 'grad_norm': '0.4401', 'learning_rate': '0.0001499', 'entropy': '0.9297', 'num_tokens': '2.386e+07', 'mean_token_accuracy': '0.7569', 'epoch': '0.03899'}
{'loss': '0.6074', 'grad_norm': '0.3312', 'learning_rate': '0.0001499', 'entropy': '0.5751', 'num_tokens': '2.387e+07', 'mean_token_accuracy': '0.8517', 'epoch': '0.039'}
{'loss': '0.96', 'grad_norm': '0.3973', 'learning_rate': '0.0001498', 'entropy': '0.9042', 'num_tokens': '2.388e+07', 'mean_token_accuracy': '0.7657', 'epoch': '0.03901'}
{'loss': '0.7941', 'grad_norm': '0.4953', 'learning_rate': '0.0001498', 'entropy': '0.7345', 'num_tokens': '2.388e+07', 'mean_token_accuracy': '0.8082', 'epoch': '0.03902'}
{'loss': '0.9548', 'grad_norm': '0.3927', 'learning_rate': '0.0001498', 'entropy': '0.9414', 'num_tokens': '2.389e+07', 'mean_token_accuracy': '0.7651', 'epoch': '0.03903'}
{'loss': '1.507', 'grad_norm': '0.5791', 'learning_rate': '0.0001498', 'entropy': '1.311', 'num_tokens': '2.389e+07', 'mean_token_accuracy': '0.7019', 'epoch': '0.03904'}
{'loss': '0.9173', 'grad_norm': '0.3604', 'learning_rate': '0.0001497', 'entropy': '0.9701', 'num_tokens': '2.39e+07', 'mean_token_accuracy': '0.7654', 'epoch': '0.03905'}
{'loss': '0.8412', 'grad_norm': '0.3878', 'learning_rate': '0.0001497', 'entropy': '0.8828', 'num_tokens': '2.391e+07', 'mean_token_accuracy': '0.7824', 'epoch': '0.03905'}
{'loss': '1.073', 'grad_norm': '0.391', 'learning_rate': '0.0001497', 'entropy': '1.07', 'num_tokens': '2.391e+07', 'mean_token_accuracy': '0.7528', 'epoch': '0.03906'}
{'loss': '0.5293', 'grad_norm': '0.3091', 'learning_rate': '0.0001497', 'entropy': '0.5847', 'num_tokens': '2.392e+07', 'mean_token_accuracy': '0.8532', 'epoch': '0.03907'}
{'loss': '0.7463', 'grad_norm': '0.3484', 'learning_rate': '0.0001496', 'entropy': '0.8151', 'num_tokens': '2.392e+07', 'mean_token_accuracy': '0.791', 'epoch': '0.03908'}
{'loss': '0.9968', 'grad_norm': '0.3293', 'learning_rate': '0.0001496', 'entropy': '0.9566', 'num_tokens': '2.393e+07', 'mean_token_accuracy': '0.7688', 'epoch': '0.03909'}
{'loss': '0.5748', 'grad_norm': '0.275', 'learning_rate': '0.0001496', 'entropy': '0.6375', 'num_tokens': '2.393e+07', 'mean_token_accuracy': '0.8423', 'epoch': '0.0391'}
{'loss': '0.736', 'grad_norm': '0.3013', 'learning_rate': '0.0001496', 'entropy': '0.7552', 'num_tokens': '2.394e+07', 'mean_token_accuracy': '0.8074', 'epoch': '0.03911'}
{'loss': '1.398', 'grad_norm': '0.4284', 'learning_rate': '0.0001495', 'entropy': '1.173', 'num_tokens': '2.394e+07', 'mean_token_accuracy': '0.7255', 'epoch': '0.03912'}
{'loss': '0.8792', 'grad_norm': '0.3546', 'learning_rate': '0.0001495', 'entropy': '0.9074', 'num_tokens': '2.395e+07', 'mean_token_accuracy': '0.7743', 'epoch': '0.03913'}

 37%|███▋      | 4422/12000 [2:38:04<6:59:34,  3.32s/it]
 37%|███▋      | 4423/12000 [2:38:06<6:02:27,  2.87s/it]
                                                        

 37%|███▋      | 4423/12000 [2:38:06<6:02:27,  2.87s/it]
 37%|███▋      | 4424/12000 [2:38:09<5:54:16,  2.81s/it]
                                                        

 37%|███▋      | 4424/12000 [2:38:09<5:54:16,  2.81s/it]
 37%|███▋      | 4425/12000 [2:38:12<6:16:43,  2.98s/it]
                                                        

 37%|███▋      | 4425/12000 [2:38:12<6:16:43,  2.98s/it]
 37%|███▋      | 4426/12000 [2:38:13<5:23:10,  2.56s/it]
                                                        

 37%|███▋      | 4426/12000 [2:38:13<5:23:10,  2.56s/it]
 37%|███▋      | 4427/12000 [2:38:15<4:58:54,  2.37s/it]
                                                        

 37%|███▋      | 4427/12000 [2:38:15<4:58:54,  2.37s/it]
 37%|███▋      | 4428/12000 [2:38:17<4:32:39,  2.16s/it]
                                                        

 37%|███▋      | 4428/12000 [2:38:17<4:32:39,  2.16s/it]
 37%|███▋      | 4429/12000 [2:38:19<4:10:05,  1.98s/it]
                                                        

 37%|███▋      | 4429/12000 [2:38:19<4:10:05,  1.98s/it]
 37%|███▋      | 4430/12000 [2:38:20<3:56:52,  1.88s/it]
                                                        

 37%|███▋      | 4430/12000 [2:38:20<3:56:52,  1.88s/it]
 37%|███▋      | 4431/12000 [2:38:22<3:46:38,  1.80s/it]
                                                        

 37%|███▋      | 4431/12000 [2:38:22<3:46:38,  1.80s/it]
 37%|███▋      | 4432/12000 [2:38:25<4:36:20,  2.19s/it]
                                                        

 37%|███▋      | 4432/12000 [2:38:25<4:36:20,  2.19s/it]
 37%|███▋      | 4433/12000 [2:38:27<4:22:23,  2.08s/it]
                                                        

 37%|███▋      | 4433/12000 [2:38:27<4:22:23,  2.08s/it]
 37%|███▋      | 4434/12000 [2:38:29<4:12:03,  2.00s/it]
                                                        

 37%|███▋      | 4434/12000 [2:38:29<4:12:03,  2.00s/it]
 37%|███▋      | 4435/12000 [2:38:30<3:56:20,  1.87s/it]
                                                        

 37%|███▋      | 4435/12000 [2:38:30<3:56:20,  1.87s/it]
 37%|███▋      | 4436/12000 [2:38:32<3:49:20,  1.82s/it]
                                                        

 37%|███▋      | 4436/12000 [2:38:32<3:49:20,  1.82s/it]
 37%|███▋      | 4437/12000 [2:38:36<5:05:31,  2.42s/it]
                                                        

 37%|███▋      | 4437/12000 [2:38:36<5:05:31,  2.42s/it]
 37%|███▋      | 4438/12000 [2:38:37<4:36:41,  2.20s/it]
                                                        

 37%|███▋      | 4438/12000 [2:38:37<4:36:41,  2.20s/it]
 37%|███▋      | 4439/12000 [2:38:39<4:12:13,  2.00s/it]
                                                        

 37%|███▋      | 4439/12000 [2:38:39<4:12:13,  2.00s/it]
 37%|███▋      | 4440/12000 [2:38:41<4:29:16,  2.14s/it]
                                                        

 37%|███▋      | 4440/12000 [2:38:41<4:29:16,  2.14s/it]
 37%|███▋      | 4441/12000 [2:38:43<4:09:38,  1.98s/it]
                                                        

 37%|███▋      | 4441/12000 [2:38:43<4:09:38,  1.98s/it]
 37%|███▋      | 4442/12000 [2:38:45<3:58:58,  1.90s/it]
                                                        

 37%|███▋      | 4442/12000 [2:38:45<3:58:58,  1.90s/it]
 37%|███▋      | 4443/12000 [2:38:46<3:50:47,  1.83s/it]
                                                        

 37%|███▋      | 4443/12000 [2:38:46<3:50:47,  1.83s/it]
 37%|███▋      | 4444/12000 [2:38:49<4:08:52,  1.98s/it]
                                                        

 37%|███▋      | 4444/12000 [2:38:49<4:08:52,  1.98s/it]
 37%|███▋      | 4445/12000 [2:38:50<3:51:02,  1.83s/it]
                                                        

 37%|███▋      | 4445/12000 [2:38:50<3:51:02,  1.83s/it]
 37%|███▋      | 4446/12000 [2:38:52<3:46:23,  1.80s/it]
                                                        

 37%|███▋      | 4446/12000 [2:38:52<3:46:23,  1.80s/it]
 37%|███▋      | 4447/12000 [2:38:54<4:09:16,  1.98s/it]
                                                        

 37%|███▋      | 4447/12000 [2:38:54<4:09:16,  1.98s/it]
 37%|███▋      | 4448/12000 [2:38:56<4:04:09,  1.94s/it]
                                                        

 37%|███▋      | 4448/12000 [2:38:56<4:04:09,  1.94s/it]
 37%|███▋      | 4449/12000 [2:38:58<4:05:16,  1.95s/it]
                                                        

 37%|███▋      | 4449/12000 [2:38:58<4:05:16,  1.95s/it]
 37%|███▋      | 4450/12000 [2:39:00<4:03:17,  1.93s/it]
                                                        

 37%|███▋      | 4450/12000 [2:39:00<4:03:17,  1.93s/it]
 37%|███▋      | 4451/12000 [2:39:03<4:42:16,  2.24s/it]
                                                        

 37%|███▋      | 4451/12000 [2:39:03<4:42:16,  2.24s/it]
 37%|███▋      | 4452/12000 [2:39:05<4:48:34,  2.29s/it]
                                                        

 37%|███▋      | 4452/12000 [2:39:05<4:48:34,  2.29s/it]
 37%|███▋      | 4453/12000 [2:39:07<4:20:03,  2.07s/it]
                                                        

 37%|███▋      | 4453/12000 [2:39:07<4:20:03,  2.07s/it]
 37%|███▋      | 4454/12000 [2:39:09<4:00:31,  1.91s/it]
                                                        

 37%|███▋      | 4454/12000 [2:39:09<4:00:31,  1.91s/it]
 37%|███▋      | 4455/12000 [2:39:11<4:28:58,  2.14s/it]
                                                        

 37%|███▋      | 4455/12000 [2:39:11<4:28:58,  2.14s/it]
 37%|███▋      | 4456/12000 [2:39:13<4:19:41,  2.07s/it]
                                                        

 37%|███▋      | 4456/12000 [2:39:13<4:19:41,  2.07s/it]
 37%|███▋      | 4457/12000 [2:39:16<4:44:20,  2.26s/it]
                                                        

 37%|███▋      | 4457/12000 [2:39:16<4:44:20,  2.26s/it]
 37%|███▋      | 4458/12000 [2:39:18<4:55:23,  2.35s/it]
                                                        

 37%|███▋      | 4458/12000 [2:39:18<4:55:23,  2.35s/it]
 37%|███▋      | 4459/12000 [2:39:22<5:39:56,  2.70s/it]
                                                        

 37%|███▋      | 4459/12000 [2:39:22<5:39:56,  2.70s/it]
 37%|███▋      | 4460/12000 [2:39:23<4:58:16,  2.37s/it]
                                                        

 37%|███▋      | 4460/12000 [2:39:23<4:58:16,  2.37s/it]
 37%|███▋      | 4461/12000 [2:39:25<4:27:15,  2.13s/it]
                                                        

 37%|███▋      | 4461/12000 [2:39:25<4:27:15,  2.13s/it]
 37%|███▋      | 4462/12000 [2:39:28<4:56:51,  2.36s/it]
                                                        

 37%|███▋      | 4462/12000 [2:39:28<4:56:51,  2.36s/it]
 37%|███▋      | 4463/12000 [2:39:33<6:21:39,  3.04s/it]
                                                        

 37%|███▋      | 4463/12000 [2:39:33<6:21:39,  3.04s/it]
 37%|███▋      | 4464/12000 [2:39:35<5:43:25,  2.73s/it]
                                                        

 37%|███▋      | 4464/12000 [2:39:35<5:43:25,  2.73s/it]
 37%|███▋      | 4465/12000 [2:39:37<5:19:27,  2.54s/it]
                                                        

 37%|███▋      | 4465/12000 [2:39:37<5:19:27,  2.54s/it]
 37%|███▋      | 4466/12000 [2:39:39<4:56:49,  2.36s/it]
                                                        

 37%|███▋      | 4466/12000 [2:39:39<4:56:49,  2.36s/it]
 37%|███▋      | 4467/12000 [2:39:41<4:50:55,  2.32s/it]
                                                        

 37%|███▋      | 4467/12000 [2:39:41<4:50:55,  2.32s/it]
 37%|███▋      | 4468/12000 [2:39:44<5:12:40,  2.49s/it]
                                                        

 37%|███▋      | 4468/12000 [2:39:44<5:12:40,  2.49s/it]
 37%|███▋      | 4469/12000 [2:39:46<4:45:30,  2.27s/it]
                                                        
{'loss': '1.032', 'grad_norm': '0.3456', 'learning_rate': '0.0001495', 'entropy': '0.8691', 'num_tokens': '2.396e+07', 'mean_token_accuracy': '0.7924', 'epoch': '0.03913'}
{'loss': '0.8381', 'grad_norm': '0.3351', 'learning_rate': '0.0001495', 'entropy': '0.8369', 'num_tokens': '2.396e+07', 'mean_token_accuracy': '0.7934', 'epoch': '0.03914'}
{'loss': '0.7882', 'grad_norm': '0.2966', 'learning_rate': '0.0001495', 'entropy': '0.8961', 'num_tokens': '2.397e+07', 'mean_token_accuracy': '0.7805', 'epoch': '0.03915'}
{'loss': '0.8539', 'grad_norm': '0.3695', 'learning_rate': '0.0001494', 'entropy': '1.009', 'num_tokens': '2.398e+07', 'mean_token_accuracy': '0.7712', 'epoch': '0.03916'}
{'loss': '0.7941', 'grad_norm': '0.317', 'learning_rate': '0.0001494', 'entropy': '0.8326', 'num_tokens': '2.398e+07', 'mean_token_accuracy': '0.7905', 'epoch': '0.03917'}
{'loss': '0.9862', 'grad_norm': '0.3864', 'learning_rate': '0.0001494', 'entropy': '1.001', 'num_tokens': '2.399e+07', 'mean_token_accuracy': '0.7563', 'epoch': '0.03918'}
{'loss': '0.875', 'grad_norm': '0.3956', 'learning_rate': '0.0001494', 'entropy': '0.8377', 'num_tokens': '2.399e+07', 'mean_token_accuracy': '0.7823', 'epoch': '0.03919'}
{'loss': '0.7768', 'grad_norm': '0.3571', 'learning_rate': '0.0001493', 'entropy': '0.8218', 'num_tokens': '2.399e+07', 'mean_token_accuracy': '0.7911', 'epoch': '0.0392'}
{'loss': '1.061', 'grad_norm': '0.4095', 'learning_rate': '0.0001493', 'entropy': '1.014', 'num_tokens': '2.4e+07', 'mean_token_accuracy': '0.7493', 'epoch': '0.0392'}
{'loss': '0.7177', 'grad_norm': '0.2935', 'learning_rate': '0.0001493', 'entropy': '0.7695', 'num_tokens': '2.4e+07', 'mean_token_accuracy': '0.7989', 'epoch': '0.03921'}
{'loss': '0.8812', 'grad_norm': '0.3162', 'learning_rate': '0.0001493', 'entropy': '0.7886', 'num_tokens': '2.401e+07', 'mean_token_accuracy': '0.7819', 'epoch': '0.03922'}
{'loss': '1.186', 'grad_norm': '0.4654', 'learning_rate': '0.0001492', 'entropy': '1.069', 'num_tokens': '2.402e+07', 'mean_token_accuracy': '0.7481', 'epoch': '0.03923'}
{'loss': '0.7205', 'grad_norm': '0.386', 'learning_rate': '0.0001492', 'entropy': '0.7046', 'num_tokens': '2.402e+07', 'mean_token_accuracy': '0.8139', 'epoch': '0.03924'}
{'loss': '0.7437', 'grad_norm': '0.4058', 'learning_rate': '0.0001492', 'entropy': '0.7212', 'num_tokens': '2.403e+07', 'mean_token_accuracy': '0.8074', 'epoch': '0.03925'}
{'loss': '0.8339', 'grad_norm': '0.3794', 'learning_rate': '0.0001492', 'entropy': '0.7872', 'num_tokens': '2.403e+07', 'mean_token_accuracy': '0.796', 'epoch': '0.03926'}
{'loss': '0.6236', 'grad_norm': '0.4371', 'learning_rate': '0.0001491', 'entropy': '0.6567', 'num_tokens': '2.404e+07', 'mean_token_accuracy': '0.8328', 'epoch': '0.03927'}
{'loss': '0.7938', 'grad_norm': '0.4292', 'learning_rate': '0.0001491', 'entropy': '0.7778', 'num_tokens': '2.404e+07', 'mean_token_accuracy': '0.8056', 'epoch': '0.03928'}
{'loss': '0.5639', 'grad_norm': '0.3257', 'learning_rate': '0.0001491', 'entropy': '0.589', 'num_tokens': '2.404e+07', 'mean_token_accuracy': '0.8398', 'epoch': '0.03928'}
{'loss': '0.5961', 'grad_norm': '0.3984', 'learning_rate': '0.0001491', 'entropy': '0.59', 'num_tokens': '2.405e+07', 'mean_token_accuracy': '0.8381', 'epoch': '0.03929'}
{'loss': '1.098', 'grad_norm': '0.3988', 'learning_rate': '0.000149', 'entropy': '1.042', 'num_tokens': '2.405e+07', 'mean_token_accuracy': '0.7389', 'epoch': '0.0393'}
{'loss': '0.8862', 'grad_norm': '0.389', 'learning_rate': '0.000149', 'entropy': '0.9439', 'num_tokens': '2.406e+07', 'mean_token_accuracy': '0.769', 'epoch': '0.03931'}
{'loss': '1.331', 'grad_norm': '0.4581', 'learning_rate': '0.000149', 'entropy': '1.326', 'num_tokens': '2.406e+07', 'mean_token_accuracy': '0.6918', 'epoch': '0.03932'}
{'loss': '1.183', 'grad_norm': '0.5292', 'learning_rate': '0.000149', 'entropy': '1.154', 'num_tokens': '2.407e+07', 'mean_token_accuracy': '0.7286', 'epoch': '0.03933'}
{'loss': '0.7437', 'grad_norm': '0.4241', 'learning_rate': '0.0001489', 'entropy': '0.756', 'num_tokens': '2.407e+07', 'mean_token_accuracy': '0.8175', 'epoch': '0.03934'}
{'loss': '0.7768', 'grad_norm': '0.3993', 'learning_rate': '0.0001489', 'entropy': '0.7958', 'num_tokens': '2.408e+07', 'mean_token_accuracy': '0.8132', 'epoch': '0.03935'}
{'loss': '0.9722', 'grad_norm': '0.4877', 'learning_rate': '0.0001489', 'entropy': '0.9721', 'num_tokens': '2.408e+07', 'mean_token_accuracy': '0.7694', 'epoch': '0.03936'}
{'loss': '1.168', 'grad_norm': '0.4775', 'learning_rate': '0.0001489', 'entropy': '1.229', 'num_tokens': '2.409e+07', 'mean_token_accuracy': '0.7034', 'epoch': '0.03936'}
{'loss': '0.7594', 'grad_norm': '0.2904', 'learning_rate': '0.0001489', 'entropy': '0.7798', 'num_tokens': '2.409e+07', 'mean_token_accuracy': '0.795', 'epoch': '0.03937'}
{'loss': '1.004', 'grad_norm': '0.3867', 'learning_rate': '0.0001488', 'entropy': '0.9695', 'num_tokens': '2.41e+07', 'mean_token_accuracy': '0.7656', 'epoch': '0.03938'}
{'loss': '0.7421', 'grad_norm': '0.3572', 'learning_rate': '0.0001488', 'entropy': '0.8102', 'num_tokens': '2.41e+07', 'mean_token_accuracy': '0.7953', 'epoch': '0.03939'}
{'loss': '0.6321', 'grad_norm': '0.2669', 'learning_rate': '0.0001488', 'entropy': '0.6705', 'num_tokens': '2.411e+07', 'mean_token_accuracy': '0.8429', 'epoch': '0.0394'}
{'loss': '0.8393', 'grad_norm': '0.49', 'learning_rate': '0.0001488', 'entropy': '0.8751', 'num_tokens': '2.411e+07', 'mean_token_accuracy': '0.7736', 'epoch': '0.03941'}
{'loss': '0.7599', 'grad_norm': '0.3559', 'learning_rate': '0.0001487', 'entropy': '0.7945', 'num_tokens': '2.412e+07', 'mean_token_accuracy': '0.8067', 'epoch': '0.03942'}
{'loss': '0.7655', 'grad_norm': '0.3676', 'learning_rate': '0.0001487', 'entropy': '0.79', 'num_tokens': '2.412e+07', 'mean_token_accuracy': '0.8059', 'epoch': '0.03943'}
{'loss': '1.177', 'grad_norm': '0.4337', 'learning_rate': '0.0001487', 'entropy': '1.024', 'num_tokens': '2.413e+07', 'mean_token_accuracy': '0.7646', 'epoch': '0.03943'}
{'loss': '0.9317', 'grad_norm': '0.3328', 'learning_rate': '0.0001487', 'entropy': '0.8492', 'num_tokens': '2.414e+07', 'mean_token_accuracy': '0.7862', 'epoch': '0.03944'}
{'loss': '1.195', 'grad_norm': '0.4873', 'learning_rate': '0.0001486', 'entropy': '1.062', 'num_tokens': '2.414e+07', 'mean_token_accuracy': '0.7305', 'epoch': '0.03945'}
{'loss': '0.6767', 'grad_norm': '0.3557', 'learning_rate': '0.0001486', 'entropy': '0.739', 'num_tokens': '2.415e+07', 'mean_token_accuracy': '0.816', 'epoch': '0.03946'}
{'loss': '1.087', 'grad_norm': '0.4466', 'learning_rate': '0.0001486', 'entropy': '1.032', 'num_tokens': '2.415e+07', 'mean_token_accuracy': '0.7499', 'epoch': '0.03947'}
{'loss': '0.7274', 'grad_norm': '0.3353', 'learning_rate': '0.0001486', 'entropy': '0.7343', 'num_tokens': '2.416e+07', 'mean_token_accuracy': '0.8029', 'epoch': '0.03948'}
{'loss': '0.926', 'grad_norm': '0.3094', 'learning_rate': '0.0001485', 'entropy': '0.8665', 'num_tokens': '2.416e+07', 'mean_token_accuracy': '0.7889', 'epoch': '0.03949'}
{'loss': '1.044', 'grad_norm': '0.386', 'learning_rate': '0.0001485', 'entropy': '0.7699', 'num_tokens': '2.417e+07', 'mean_token_accuracy': '0.8218', 'epoch': '0.0395'}
{'loss': '0.8332', 'grad_norm': '0.3819', 'learning_rate': '0.0001485', 'entropy': '0.8562', 'num_tokens': '2.418e+07', 'mean_token_accuracy': '0.7895', 'epoch': '0.03951'}
{'loss': '0.6446', 'grad_norm': '0.3384', 'learning_rate': '0.0001485', 'entropy': '0.647', 'num_tokens': '2.419e+07', 'mean_token_accuracy': '0.8325', 'epoch': '0.03951'}
{'loss': '0.784', 'grad_norm': '0.3947', 'learning_rate': '0.0001484', 'entropy': '0.7975', 'num_tokens': '2.419e+07', 'mean_token_accuracy': '0.7867', 'epoch': '0.03952'}
{'loss': '0.9439', 'grad_norm': '0.3568', 'learning_rate': '0.0001484', 'entropy': '0.9508', 'num_tokens': '2.42e+07', 'mean_token_accuracy': '0.7574', 'epoch': '0.03953'}
{'loss': '0.5384', 'grad_norm': '0.3401', 'learning_rate': '0.0001484', 'entropy': '0.562', 'num_tokens': '2.42e+07', 'mean_token_accuracy': '0.8499', 'epoch': '0.03954'}

 37%|███▋      | 4469/12000 [2:39:46<4:45:30,  2.27s/it]
 37%|███▋      | 4470/12000 [2:39:49<5:43:25,  2.74s/it]
                                                        

 37%|███▋      | 4470/12000 [2:39:49<5:43:25,  2.74s/it]
 37%|███▋      | 4471/12000 [2:39:52<5:31:33,  2.64s/it]
                                                        

 37%|███▋      | 4471/12000 [2:39:52<5:31:33,  2.64s/it]
 37%|███▋      | 4472/12000 [2:39:53<4:47:53,  2.29s/it]
                                                        

 37%|███▋      | 4472/12000 [2:39:53<4:47:53,  2.29s/it]
 37%|███▋      | 4473/12000 [2:39:55<4:21:27,  2.08s/it]
                                                        

 37%|███▋      | 4473/12000 [2:39:55<4:21:27,  2.08s/it]
 37%|███▋      | 4474/12000 [2:39:57<4:34:12,  2.19s/it]
                                                        

 37%|███▋      | 4474/12000 [2:39:57<4:34:12,  2.19s/it]
 37%|███▋      | 4475/12000 [2:39:59<4:09:58,  1.99s/it]
                                                        

 37%|███▋      | 4475/12000 [2:39:59<4:09:58,  1.99s/it]
 37%|███▋      | 4476/12000 [2:40:00<3:55:13,  1.88s/it]
                                                        

 37%|███▋      | 4476/12000 [2:40:00<3:55:13,  1.88s/it]
 37%|███▋      | 4477/12000 [2:40:05<5:20:52,  2.56s/it]
                                                        

 37%|███▋      | 4477/12000 [2:40:05<5:20:52,  2.56s/it]
 37%|███▋      | 4478/12000 [2:40:06<4:46:50,  2.29s/it]
                                                        

 37%|███▋      | 4478/12000 [2:40:06<4:46:50,  2.29s/it]
 37%|███▋      | 4479/12000 [2:40:08<4:21:08,  2.08s/it]
                                                        

 37%|███▋      | 4479/12000 [2:40:08<4:21:08,  2.08s/it]
 37%|███▋      | 4480/12000 [2:40:09<4:05:08,  1.96s/it]
                                                        

 37%|███▋      | 4480/12000 [2:40:09<4:05:08,  1.96s/it]
 37%|███▋      | 4481/12000 [2:40:14<5:50:31,  2.80s/it]
                                                        

 37%|███▋      | 4481/12000 [2:40:14<5:50:31,  2.80s/it]
 37%|███▋      | 4482/12000 [2:40:16<5:03:46,  2.42s/it]
                                                        

 37%|███▋      | 4482/12000 [2:40:16<5:03:46,  2.42s/it]
 37%|███▋      | 4483/12000 [2:40:17<4:33:05,  2.18s/it]
                                                        

 37%|███▋      | 4483/12000 [2:40:17<4:33:05,  2.18s/it]
 37%|███▋      | 4484/12000 [2:40:19<4:12:35,  2.02s/it]
                                                        

 37%|███▋      | 4484/12000 [2:40:19<4:12:35,  2.02s/it]
 37%|███▋      | 4485/12000 [2:40:22<4:46:31,  2.29s/it]
                                                        

 37%|███▋      | 4485/12000 [2:40:22<4:46:31,  2.29s/it]
 37%|███▋      | 4486/12000 [2:40:24<4:27:01,  2.13s/it]
                                                        

 37%|███▋      | 4486/12000 [2:40:24<4:27:01,  2.13s/it]
 37%|███▋      | 4487/12000 [2:40:26<4:49:03,  2.31s/it]
                                                        

 37%|███▋      | 4487/12000 [2:40:26<4:49:03,  2.31s/it]
 37%|███▋      | 4488/12000 [2:40:28<4:25:17,  2.12s/it]
                                                        

 37%|███▋      | 4488/12000 [2:40:28<4:25:17,  2.12s/it]
 37%|███▋      | 4489/12000 [2:40:30<4:08:36,  1.99s/it]
                                                        

 37%|███▋      | 4489/12000 [2:40:30<4:08:36,  1.99s/it]
 37%|███▋      | 4490/12000 [2:40:31<3:56:37,  1.89s/it]
                                                        

 37%|███▋      | 4490/12000 [2:40:31<3:56:37,  1.89s/it]
 37%|███▋      | 4491/12000 [2:40:33<3:43:45,  1.79s/it]
                                                        

 37%|███▋      | 4491/12000 [2:40:33<3:43:45,  1.79s/it]
 37%|███▋      | 4492/12000 [2:40:35<3:41:46,  1.77s/it]
                                                        

 37%|███▋      | 4492/12000 [2:40:35<3:41:46,  1.77s/it]
 37%|███▋      | 4493/12000 [2:40:37<4:16:38,  2.05s/it]
                                                        

 37%|███▋      | 4493/12000 [2:40:37<4:16:38,  2.05s/it]
 37%|███▋      | 4494/12000 [2:40:39<4:01:34,  1.93s/it]
                                                        

 37%|███▋      | 4494/12000 [2:40:39<4:01:34,  1.93s/it]
 37%|███▋      | 4495/12000 [2:40:42<4:25:43,  2.12s/it]
                                                        

 37%|███▋      | 4495/12000 [2:40:42<4:25:43,  2.12s/it]
 37%|███▋      | 4496/12000 [2:40:43<4:01:18,  1.93s/it]
                                                        

 37%|███▋      | 4496/12000 [2:40:43<4:01:18,  1.93s/it]
 37%|███▋      | 4497/12000 [2:40:45<3:50:04,  1.84s/it]
                                                        

 37%|███▋      | 4497/12000 [2:40:45<3:50:04,  1.84s/it]
 37%|███▋      | 4498/12000 [2:40:47<4:10:10,  2.00s/it]
                                                        

 37%|███▋      | 4498/12000 [2:40:47<4:10:10,  2.00s/it]
 37%|███▋      | 4499/12000 [2:40:49<4:06:19,  1.97s/it]
                                                        

 37%|███▋      | 4499/12000 [2:40:49<4:06:19,  1.97s/it]
 38%|███▊      | 4500/12000 [2:40:51<4:06:09,  1.97s/it]
                                                        

 38%|███▊      | 4500/12000 [2:40:51<4:06:09,  1.97s/it]
 38%|███▊      | 4501/12000 [2:40:54<4:54:01,  2.35s/it]
                                                        

 38%|███▊      | 4501/12000 [2:40:54<4:54:01,  2.35s/it]
 38%|███▊      | 4502/12000 [2:40:56<4:25:23,  2.12s/it]
                                                        

 38%|███▊      | 4502/12000 [2:40:56<4:25:23,  2.12s/it]
 38%|███▊      | 4503/12000 [2:41:00<5:27:00,  2.62s/it]
                                                        

 38%|███▊      | 4503/12000 [2:41:00<5:27:00,  2.62s/it]
 38%|███▊      | 4504/12000 [2:41:01<4:49:14,  2.32s/it]
                                                        

 38%|███▊      | 4504/12000 [2:41:01<4:49:14,  2.32s/it]
 38%|███▊      | 4505/12000 [2:41:03<4:24:44,  2.12s/it]
                                                        

 38%|███▊      | 4505/12000 [2:41:03<4:24:44,  2.12s/it]
 38%|███▊      | 4506/12000 [2:41:04<4:03:23,  1.95s/it]
                                                        

 38%|███▊      | 4506/12000 [2:41:04<4:03:23,  1.95s/it]
 38%|███▊      | 4507/12000 [2:41:06<3:47:37,  1.82s/it]
                                                        

 38%|███▊      | 4507/12000 [2:41:06<3:47:37,  1.82s/it]
 38%|███▊      | 4508/12000 [2:41:08<3:42:47,  1.78s/it]
                                                        

 38%|███▊      | 4508/12000 [2:41:08<3:42:47,  1.78s/it]
 38%|███▊      | 4509/12000 [2:41:10<3:59:58,  1.92s/it]
                                                        

 38%|███▊      | 4509/12000 [2:41:10<3:59:58,  1.92s/it]
 38%|███▊      | 4510/12000 [2:41:11<3:45:02,  1.80s/it]
                                                        

 38%|███▊      | 4510/12000 [2:41:11<3:45:02,  1.80s/it]
 38%|███▊      | 4511/12000 [2:41:13<3:34:29,  1.72s/it]
                                                        

 38%|███▊      | 4511/12000 [2:41:13<3:34:29,  1.72s/it]
 38%|███▊      | 4512/12000 [2:41:15<4:03:17,  1.95s/it]
                                                        

 38%|███▊      | 4512/12000 [2:41:15<4:03:17,  1.95s/it]
 38%|███▊      | 4513/12000 [2:41:18<4:19:02,  2.08s/it]
                                                        

 38%|███▊      | 4513/12000 [2:41:18<4:19:02,  2.08s/it]
 38%|███▊      | 4514/12000 [2:41:20<4:21:15,  2.09s/it]
                                                        

 38%|███▊      | 4514/12000 [2:41:20<4:21:15,  2.09s/it]
 38%|███▊      | 4515/12000 [2:41:22<4:04:58,  1.96s/it]
                                                        

 38%|███▊      | 4515/12000 [2:41:22<4:04:58,  1.96s/it]
 38%|███▊      | 4516/12000 [2:41:24<4:37:55,  2.23s/it]
                                                        
{'loss': '0.7401', 'grad_norm': '0.4583', 'learning_rate': '0.0001484', 'entropy': '0.7822', 'num_tokens': '2.421e+07', 'mean_token_accuracy': '0.7958', 'epoch': '0.03955'}
{'loss': '0.8016', 'grad_norm': '0.2894', 'learning_rate': '0.0001483', 'entropy': '0.8082', 'num_tokens': '2.421e+07', 'mean_token_accuracy': '0.7991', 'epoch': '0.03956'}
{'loss': '1.264', 'grad_norm': '0.4368', 'learning_rate': '0.0001483', 'entropy': '1.035', 'num_tokens': '2.422e+07', 'mean_token_accuracy': '0.7446', 'epoch': '0.03957'}
{'loss': '0.8951', 'grad_norm': '0.3859', 'learning_rate': '0.0001483', 'entropy': '0.9024', 'num_tokens': '2.422e+07', 'mean_token_accuracy': '0.7706', 'epoch': '0.03958'}
{'loss': '0.9759', 'grad_norm': '0.3933', 'learning_rate': '0.0001483', 'entropy': '0.9018', 'num_tokens': '2.423e+07', 'mean_token_accuracy': '0.7608', 'epoch': '0.03959'}
{'loss': '0.6808', 'grad_norm': '0.2956', 'learning_rate': '0.0001482', 'entropy': '0.6427', 'num_tokens': '2.423e+07', 'mean_token_accuracy': '0.8396', 'epoch': '0.03959'}
{'loss': '0.6237', 'grad_norm': '0.308', 'learning_rate': '0.0001482', 'entropy': '0.6195', 'num_tokens': '2.424e+07', 'mean_token_accuracy': '0.8364', 'epoch': '0.0396'}
{'loss': '0.897', 'grad_norm': '0.3914', 'learning_rate': '0.0001482', 'entropy': '0.8688', 'num_tokens': '2.424e+07', 'mean_token_accuracy': '0.7881', 'epoch': '0.03961'}
{'loss': '0.7681', 'grad_norm': '0.3306', 'learning_rate': '0.0001482', 'entropy': '0.9944', 'num_tokens': '2.425e+07', 'mean_token_accuracy': '0.7674', 'epoch': '0.03962'}
{'loss': '0.8563', 'grad_norm': '0.399', 'learning_rate': '0.0001482', 'entropy': '0.9085', 'num_tokens': '2.426e+07', 'mean_token_accuracy': '0.7756', 'epoch': '0.03963'}
{'loss': '0.6521', 'grad_norm': '0.3716', 'learning_rate': '0.0001481', 'entropy': '0.682', 'num_tokens': '2.426e+07', 'mean_token_accuracy': '0.8211', 'epoch': '0.03964'}
{'loss': '0.7115', 'grad_norm': '0.3754', 'learning_rate': '0.0001481', 'entropy': '0.6819', 'num_tokens': '2.427e+07', 'mean_token_accuracy': '0.817', 'epoch': '0.03965'}
{'loss': '0.8909', 'grad_norm': '0.3606', 'learning_rate': '0.0001481', 'entropy': '0.864', 'num_tokens': '2.427e+07', 'mean_token_accuracy': '0.7878', 'epoch': '0.03966'}
{'loss': '0.8701', 'grad_norm': '0.4482', 'learning_rate': '0.0001481', 'entropy': '0.9659', 'num_tokens': '2.428e+07', 'mean_token_accuracy': '0.7729', 'epoch': '0.03966'}
{'loss': '0.7883', 'grad_norm': '0.4091', 'learning_rate': '0.000148', 'entropy': '0.7613', 'num_tokens': '2.428e+07', 'mean_token_accuracy': '0.8061', 'epoch': '0.03967'}
{'loss': '1.028', 'grad_norm': '0.422', 'learning_rate': '0.000148', 'entropy': '1.116', 'num_tokens': '2.428e+07', 'mean_token_accuracy': '0.7253', 'epoch': '0.03968'}
{'loss': '0.7744', 'grad_norm': '0.3749', 'learning_rate': '0.000148', 'entropy': '0.8658', 'num_tokens': '2.429e+07', 'mean_token_accuracy': '0.7914', 'epoch': '0.03969'}
{'loss': '0.8386', 'grad_norm': '0.4527', 'learning_rate': '0.000148', 'entropy': '0.8225', 'num_tokens': '2.43e+07', 'mean_token_accuracy': '0.7783', 'epoch': '0.0397'}
{'loss': '0.9261', 'grad_norm': '0.3771', 'learning_rate': '0.0001479', 'entropy': '0.9254', 'num_tokens': '2.43e+07', 'mean_token_accuracy': '0.759', 'epoch': '0.03971'}
{'loss': '0.7612', 'grad_norm': '0.2771', 'learning_rate': '0.0001479', 'entropy': '0.7906', 'num_tokens': '2.431e+07', 'mean_token_accuracy': '0.7935', 'epoch': '0.03972'}
{'loss': '0.8488', 'grad_norm': '0.4831', 'learning_rate': '0.0001479', 'entropy': '0.9431', 'num_tokens': '2.431e+07', 'mean_token_accuracy': '0.7672', 'epoch': '0.03973'}
{'loss': '0.8218', 'grad_norm': '0.382', 'learning_rate': '0.0001479', 'entropy': '0.7862', 'num_tokens': '2.432e+07', 'mean_token_accuracy': '0.7918', 'epoch': '0.03974'}
{'loss': '1.07', 'grad_norm': '0.4229', 'learning_rate': '0.0001478', 'entropy': '1.091', 'num_tokens': '2.432e+07', 'mean_token_accuracy': '0.7344', 'epoch': '0.03974'}
{'loss': '0.627', 'grad_norm': '0.2896', 'learning_rate': '0.0001478', 'entropy': '0.6154', 'num_tokens': '2.433e+07', 'mean_token_accuracy': '0.8474', 'epoch': '0.03975'}
{'loss': '0.8213', 'grad_norm': '0.3291', 'learning_rate': '0.0001478', 'entropy': '0.8852', 'num_tokens': '2.433e+07', 'mean_token_accuracy': '0.7697', 'epoch': '0.03976'}
{'loss': '0.546', 'grad_norm': '0.3681', 'learning_rate': '0.0001478', 'entropy': '0.5305', 'num_tokens': '2.434e+07', 'mean_token_accuracy': '0.8525', 'epoch': '0.03977'}
{'loss': '0.6921', 'grad_norm': '0.3925', 'learning_rate': '0.0001477', 'entropy': '0.8038', 'num_tokens': '2.434e+07', 'mean_token_accuracy': '0.7972', 'epoch': '0.03978'}
{'loss': '0.8319', 'grad_norm': '0.4002', 'learning_rate': '0.0001477', 'entropy': '0.8577', 'num_tokens': '2.435e+07', 'mean_token_accuracy': '0.7751', 'epoch': '0.03979'}
{'loss': '0.5777', 'grad_norm': '0.3441', 'learning_rate': '0.0001477', 'entropy': '0.6334', 'num_tokens': '2.435e+07', 'mean_token_accuracy': '0.8228', 'epoch': '0.0398'}
{'loss': '0.8256', 'grad_norm': '0.4061', 'learning_rate': '0.0001477', 'entropy': '0.8336', 'num_tokens': '2.436e+07', 'mean_token_accuracy': '0.7859', 'epoch': '0.03981'}
{'loss': '1.145', 'grad_norm': '0.4268', 'learning_rate': '0.0001476', 'entropy': '1.053', 'num_tokens': '2.436e+07', 'mean_token_accuracy': '0.7461', 'epoch': '0.03982'}
{'loss': '0.9192', 'grad_norm': '0.5302', 'learning_rate': '0.0001476', 'entropy': '0.8972', 'num_tokens': '2.437e+07', 'mean_token_accuracy': '0.7512', 'epoch': '0.03982'}
{'loss': '0.8503', 'grad_norm': '0.3703', 'learning_rate': '0.0001476', 'entropy': '0.9149', 'num_tokens': '2.437e+07', 'mean_token_accuracy': '0.7772', 'epoch': '0.03983'}
{'loss': '0.5867', 'grad_norm': '0.3017', 'learning_rate': '0.0001476', 'entropy': '0.5931', 'num_tokens': '2.438e+07', 'mean_token_accuracy': '0.8452', 'epoch': '0.03984'}
{'loss': '0.7533', 'grad_norm': '0.3604', 'learning_rate': '0.0001475', 'entropy': '0.7831', 'num_tokens': '2.438e+07', 'mean_token_accuracy': '0.8018', 'epoch': '0.03985'}
{'loss': '0.6729', 'grad_norm': '0.4332', 'learning_rate': '0.0001475', 'entropy': '0.6534', 'num_tokens': '2.438e+07', 'mean_token_accuracy': '0.8296', 'epoch': '0.03986'}
{'loss': '0.8189', 'grad_norm': '0.3997', 'learning_rate': '0.0001475', 'entropy': '0.878', 'num_tokens': '2.439e+07', 'mean_token_accuracy': '0.7883', 'epoch': '0.03987'}
{'loss': '0.6103', 'grad_norm': '0.3247', 'learning_rate': '0.0001475', 'entropy': '0.6239', 'num_tokens': '2.439e+07', 'mean_token_accuracy': '0.8397', 'epoch': '0.03988'}
{'loss': '0.7286', 'grad_norm': '0.3474', 'learning_rate': '0.0001474', 'entropy': '0.7684', 'num_tokens': '2.44e+07', 'mean_token_accuracy': '0.8029', 'epoch': '0.03989'}
{'loss': '0.8546', 'grad_norm': '0.3866', 'learning_rate': '0.0001474', 'entropy': '0.8722', 'num_tokens': '2.44e+07', 'mean_token_accuracy': '0.7812', 'epoch': '0.0399'}
{'loss': '1.258', 'grad_norm': '0.5368', 'learning_rate': '0.0001474', 'entropy': '1.155', 'num_tokens': '2.441e+07', 'mean_token_accuracy': '0.7383', 'epoch': '0.0399'}
{'loss': '0.8427', 'grad_norm': '0.4347', 'learning_rate': '0.0001474', 'entropy': '0.8463', 'num_tokens': '2.441e+07', 'mean_token_accuracy': '0.7765', 'epoch': '0.03991'}
{'loss': '0.6526', 'grad_norm': '0.3407', 'learning_rate': '0.0001474', 'entropy': '0.6653', 'num_tokens': '2.442e+07', 'mean_token_accuracy': '0.8296', 'epoch': '0.03992'}
{'loss': '0.8323', 'grad_norm': '0.3841', 'learning_rate': '0.0001473', 'entropy': '0.9285', 'num_tokens': '2.442e+07', 'mean_token_accuracy': '0.774', 'epoch': '0.03993'}
{'loss': '0.8264', 'grad_norm': '0.336', 'learning_rate': '0.0001473', 'entropy': '0.7836', 'num_tokens': '2.443e+07', 'mean_token_accuracy': '0.8046', 'epoch': '0.03994'}
{'loss': '0.563', 'grad_norm': '0.3233', 'learning_rate': '0.0001473', 'entropy': '0.6418', 'num_tokens': '2.443e+07', 'mean_token_accuracy': '0.8401', 'epoch': '0.03995'}
{'loss': '0.9712', 'grad_norm': '0.4288', 'learning_rate': '0.0001473', 'entropy': '0.9145', 'num_tokens': '2.444e+07', 'mean_token_accuracy': '0.7614', 'epoch': '0.03996'}

 38%|███▊      | 4516/12000 [2:41:24<4:37:55,  2.23s/it]
 38%|███▊      | 4517/12000 [2:41:26<4:17:49,  2.07s/it]
                                                        

 38%|███▊      | 4517/12000 [2:41:26<4:17:49,  2.07s/it]
 38%|███▊      | 4518/12000 [2:41:28<4:05:01,  1.96s/it]
                                                        

 38%|███▊      | 4518/12000 [2:41:28<4:05:01,  1.96s/it]
 38%|███▊      | 4519/12000 [2:41:30<4:28:15,  2.15s/it]
                                                        

 38%|███▊      | 4519/12000 [2:41:30<4:28:15,  2.15s/it]
 38%|███▊      | 4520/12000 [2:41:33<4:51:18,  2.34s/it]
                                                        

 38%|███▊      | 4520/12000 [2:41:33<4:51:18,  2.34s/it]
 38%|███▊      | 4521/12000 [2:41:35<4:21:48,  2.10s/it]
                                                        

 38%|███▊      | 4521/12000 [2:41:35<4:21:48,  2.10s/it]
 38%|███▊      | 4522/12000 [2:41:37<4:23:22,  2.11s/it]
                                                        

 38%|███▊      | 4522/12000 [2:41:37<4:23:22,  2.11s/it]
 38%|███▊      | 4523/12000 [2:41:39<4:03:32,  1.95s/it]
                                                        

 38%|███▊      | 4523/12000 [2:41:39<4:03:32,  1.95s/it]
 38%|███▊      | 4524/12000 [2:41:40<3:52:32,  1.87s/it]
                                                        

 38%|███▊      | 4524/12000 [2:41:40<3:52:32,  1.87s/it]
 38%|███▊      | 4525/12000 [2:41:45<5:42:11,  2.75s/it]
                                                        

 38%|███▊      | 4525/12000 [2:41:45<5:42:11,  2.75s/it]
 38%|███▊      | 4526/12000 [2:41:47<4:58:40,  2.40s/it]
                                                        

 38%|███▊      | 4526/12000 [2:41:47<4:58:40,  2.40s/it]
 38%|███▊      | 4527/12000 [2:41:49<4:58:09,  2.39s/it]
                                                        

 38%|███▊      | 4527/12000 [2:41:49<4:58:09,  2.39s/it]
 38%|███▊      | 4528/12000 [2:41:51<4:28:16,  2.15s/it]
                                                        

 38%|███▊      | 4528/12000 [2:41:51<4:28:16,  2.15s/it]
 38%|███▊      | 4529/12000 [2:41:53<4:22:34,  2.11s/it]
                                                        

 38%|███▊      | 4529/12000 [2:41:53<4:22:34,  2.11s/it]
 38%|███▊      | 4530/12000 [2:41:54<4:09:49,  2.01s/it]
                                                        

 38%|███▊      | 4530/12000 [2:41:54<4:09:49,  2.01s/it]
 38%|███▊      | 4531/12000 [2:41:56<4:16:33,  2.06s/it]
                                                        

 38%|███▊      | 4531/12000 [2:41:56<4:16:33,  2.06s/it]
 38%|███▊      | 4532/12000 [2:41:58<4:12:43,  2.03s/it]
                                                        

 38%|███▊      | 4532/12000 [2:41:58<4:12:43,  2.03s/it]
 38%|███▊      | 4533/12000 [2:42:00<3:52:43,  1.87s/it]
                                                        

 38%|███▊      | 4533/12000 [2:42:00<3:52:43,  1.87s/it]
 38%|███▊      | 4534/12000 [2:42:02<4:02:41,  1.95s/it]
                                                        

 38%|███▊      | 4534/12000 [2:42:02<4:02:41,  1.95s/it]
 38%|███▊      | 4535/12000 [2:42:04<3:50:28,  1.85s/it]
                                                        

 38%|███▊      | 4535/12000 [2:42:04<3:50:28,  1.85s/it]
 38%|███▊      | 4536/12000 [2:42:06<4:21:56,  2.11s/it]
                                                        

 38%|███▊      | 4536/12000 [2:42:06<4:21:56,  2.11s/it]
 38%|███▊      | 4537/12000 [2:42:08<4:04:39,  1.97s/it]
                                                        

 38%|███▊      | 4537/12000 [2:42:08<4:04:39,  1.97s/it]
 38%|███▊      | 4538/12000 [2:42:10<4:17:26,  2.07s/it]
                                                        

 38%|███▊      | 4538/12000 [2:42:10<4:17:26,  2.07s/it]
 38%|███▊      | 4539/12000 [2:42:12<4:00:59,  1.94s/it]
                                                        

 38%|███▊      | 4539/12000 [2:42:12<4:00:59,  1.94s/it]
 38%|███▊      | 4540/12000 [2:42:14<3:56:07,  1.90s/it]
                                                        

 38%|███▊      | 4540/12000 [2:42:14<3:56:07,  1.90s/it]
 38%|███▊      | 4541/12000 [2:42:15<3:44:36,  1.81s/it]
                                                        

 38%|███▊      | 4541/12000 [2:42:15<3:44:36,  1.81s/it]
 38%|███▊      | 4542/12000 [2:42:18<4:09:02,  2.00s/it]
                                                        

 38%|███▊      | 4542/12000 [2:42:18<4:09:02,  2.00s/it]
 38%|███▊      | 4543/12000 [2:42:19<3:54:30,  1.89s/it]
                                                        

 38%|███▊      | 4543/12000 [2:42:19<3:54:30,  1.89s/it]
 38%|███▊      | 4544/12000 [2:42:21<3:47:09,  1.83s/it]
                                                        

 38%|███▊      | 4544/12000 [2:42:21<3:47:09,  1.83s/it]
 38%|███▊      | 4545/12000 [2:42:23<3:41:16,  1.78s/it]
                                                        

 38%|███▊      | 4545/12000 [2:42:23<3:41:16,  1.78s/it]
 38%|███▊      | 4546/12000 [2:42:28<5:41:24,  2.75s/it]
                                                        

 38%|███▊      | 4546/12000 [2:42:28<5:41:24,  2.75s/it]
 38%|███▊      | 4547/12000 [2:42:32<6:20:33,  3.06s/it]
                                                        

 38%|███▊      | 4547/12000 [2:42:32<6:20:33,  3.06s/it]
 38%|███▊      | 4548/12000 [2:42:34<5:44:17,  2.77s/it]
                                                        

 38%|███▊      | 4548/12000 [2:42:34<5:44:17,  2.77s/it]
 38%|███▊      | 4549/12000 [2:42:35<4:57:41,  2.40s/it]
                                                        

 38%|███▊      | 4549/12000 [2:42:35<4:57:41,  2.40s/it]
 38%|███▊      | 4550/12000 [2:42:37<4:25:05,  2.13s/it]
                                                        

 38%|███▊      | 4550/12000 [2:42:37<4:25:05,  2.13s/it]
 38%|███▊      | 4551/12000 [2:42:40<5:03:13,  2.44s/it]
                                                        

 38%|███▊      | 4551/12000 [2:42:40<5:03:13,  2.44s/it]
 38%|███▊      | 4552/12000 [2:42:41<4:26:14,  2.14s/it]
                                                        

 38%|███▊      | 4552/12000 [2:42:41<4:26:14,  2.14s/it]
 38%|███▊      | 4553/12000 [2:42:43<4:02:19,  1.95s/it]
                                                        

 38%|███▊      | 4553/12000 [2:42:43<4:02:19,  1.95s/it]
 38%|███▊      | 4554/12000 [2:42:45<4:10:02,  2.01s/it]
                                                        

 38%|███▊      | 4554/12000 [2:42:45<4:10:02,  2.01s/it]
 38%|███▊      | 4555/12000 [2:42:47<3:58:12,  1.92s/it]
                                                        

 38%|███▊      | 4555/12000 [2:42:47<3:58:12,  1.92s/it]
 38%|███▊      | 4556/12000 [2:42:48<3:49:34,  1.85s/it]
                                                        

 38%|███▊      | 4556/12000 [2:42:48<3:49:34,  1.85s/it]
 38%|███▊      | 4557/12000 [2:42:50<3:45:02,  1.81s/it]
                                                        

 38%|███▊      | 4557/12000 [2:42:50<3:45:02,  1.81s/it]
 38%|███▊      | 4558/12000 [2:42:53<4:21:39,  2.11s/it]
                                                        

 38%|███▊      | 4558/12000 [2:42:53<4:21:39,  2.11s/it]
 38%|███▊      | 4559/12000 [2:42:55<4:07:28,  2.00s/it]
                                                        

 38%|███▊      | 4559/12000 [2:42:55<4:07:28,  2.00s/it]
 38%|███▊      | 4560/12000 [2:42:56<3:53:20,  1.88s/it]
                                                        

 38%|███▊      | 4560/12000 [2:42:56<3:53:20,  1.88s/it]
 38%|███▊      | 4561/12000 [2:42:58<3:44:42,  1.81s/it]
                                                        

 38%|███▊      | 4561/12000 [2:42:58<3:44:42,  1.81s/it]
 38%|███▊      | 4562/12000 [2:43:00<3:45:14,  1.82s/it]
                                                        

 38%|███▊      | 4562/12000 [2:43:00<3:45:14,  1.82s/it]
 38%|███▊      | 4563/12000 [2:43:01<3:36:57,  1.75s/it]
                                                        
{'loss': '1.467', 'grad_norm': '0.3944', 'learning_rate': '0.0001472', 'entropy': '1.505', 'num_tokens': '2.444e+07', 'mean_token_accuracy': '0.6443', 'epoch': '0.03997'}
{'loss': '0.6337', 'grad_norm': '0.4358', 'learning_rate': '0.0001472', 'entropy': '0.6172', 'num_tokens': '2.445e+07', 'mean_token_accuracy': '0.8461', 'epoch': '0.03997'}
{'loss': '0.7127', 'grad_norm': '0.3369', 'learning_rate': '0.0001472', 'entropy': '0.6935', 'num_tokens': '2.445e+07', 'mean_token_accuracy': '0.8195', 'epoch': '0.03998'}
{'loss': '0.8634', 'grad_norm': '0.4359', 'learning_rate': '0.0001472', 'entropy': '0.9909', 'num_tokens': '2.446e+07', 'mean_token_accuracy': '0.7589', 'epoch': '0.03999'}
{'loss': '0.787', 'grad_norm': '0.341', 'learning_rate': '0.0001471', 'entropy': '0.872', 'num_tokens': '2.447e+07', 'mean_token_accuracy': '0.7831', 'epoch': '0.04'}
{'loss': '0.9788', 'grad_norm': '0.4452', 'learning_rate': '0.0001471', 'entropy': '0.9288', 'num_tokens': '2.447e+07', 'mean_token_accuracy': '0.7789', 'epoch': '0.04001'}
{'loss': '1', 'grad_norm': '0.3601', 'learning_rate': '0.0001471', 'entropy': '0.8881', 'num_tokens': '2.448e+07', 'mean_token_accuracy': '0.7893', 'epoch': '0.04002'}
{'loss': '0.749', 'grad_norm': '0.3377', 'learning_rate': '0.0001471', 'entropy': '0.7369', 'num_tokens': '2.448e+07', 'mean_token_accuracy': '0.812', 'epoch': '0.04003'}
{'loss': '0.7826', 'grad_norm': '0.3839', 'learning_rate': '0.000147', 'entropy': '0.8549', 'num_tokens': '2.449e+07', 'mean_token_accuracy': '0.7902', 'epoch': '0.04004'}
{'loss': '0.836', 'grad_norm': '0.3056', 'learning_rate': '0.000147', 'entropy': '0.9398', 'num_tokens': '2.449e+07', 'mean_token_accuracy': '0.7723', 'epoch': '0.04005'}
{'loss': '0.7983', 'grad_norm': '0.452', 'learning_rate': '0.000147', 'entropy': '0.8504', 'num_tokens': '2.45e+07', 'mean_token_accuracy': '0.7861', 'epoch': '0.04005'}
{'loss': '0.7827', 'grad_norm': '0.3111', 'learning_rate': '0.000147', 'entropy': '0.8192', 'num_tokens': '2.451e+07', 'mean_token_accuracy': '0.8088', 'epoch': '0.04006'}
{'loss': '0.7616', 'grad_norm': '0.464', 'learning_rate': '0.0001469', 'entropy': '0.7946', 'num_tokens': '2.451e+07', 'mean_token_accuracy': '0.7934', 'epoch': '0.04007'}
{'loss': '0.6412', 'grad_norm': '0.3234', 'learning_rate': '0.0001469', 'entropy': '0.7231', 'num_tokens': '2.451e+07', 'mean_token_accuracy': '0.8179', 'epoch': '0.04008'}
{'loss': '0.9121', 'grad_norm': '0.4501', 'learning_rate': '0.0001469', 'entropy': '0.8739', 'num_tokens': '2.452e+07', 'mean_token_accuracy': '0.7825', 'epoch': '0.04009'}
{'loss': '0.9019', 'grad_norm': '0.3802', 'learning_rate': '0.0001469', 'entropy': '0.8673', 'num_tokens': '2.453e+07', 'mean_token_accuracy': '0.7834', 'epoch': '0.0401'}
{'loss': '1.066', 'grad_norm': '0.3972', 'learning_rate': '0.0001468', 'entropy': '1.137', 'num_tokens': '2.453e+07', 'mean_token_accuracy': '0.7298', 'epoch': '0.04011'}
{'loss': '0.7969', 'grad_norm': '0.3731', 'learning_rate': '0.0001468', 'entropy': '0.8057', 'num_tokens': '2.453e+07', 'mean_token_accuracy': '0.787', 'epoch': '0.04012'}
{'loss': '0.8109', 'grad_norm': '0.3276', 'learning_rate': '0.0001468', 'entropy': '0.7852', 'num_tokens': '2.454e+07', 'mean_token_accuracy': '0.8099', 'epoch': '0.04013'}
{'loss': '1.13', 'grad_norm': '0.5364', 'learning_rate': '0.0001468', 'entropy': '1.099', 'num_tokens': '2.455e+07', 'mean_token_accuracy': '0.7284', 'epoch': '0.04013'}
{'loss': '0.8745', 'grad_norm': '0.3336', 'learning_rate': '0.0001467', 'entropy': '0.9391', 'num_tokens': '2.455e+07', 'mean_token_accuracy': '0.7634', 'epoch': '0.04014'}
{'loss': '0.688', 'grad_norm': '0.3511', 'learning_rate': '0.0001467', 'entropy': '0.7143', 'num_tokens': '2.456e+07', 'mean_token_accuracy': '0.8005', 'epoch': '0.04015'}
{'loss': '0.6306', 'grad_norm': '0.2825', 'learning_rate': '0.0001467', 'entropy': '0.658', 'num_tokens': '2.456e+07', 'mean_token_accuracy': '0.8305', 'epoch': '0.04016'}
{'loss': '0.8264', 'grad_norm': '0.3674', 'learning_rate': '0.0001467', 'entropy': '0.7929', 'num_tokens': '2.457e+07', 'mean_token_accuracy': '0.8134', 'epoch': '0.04017'}
{'loss': '0.8622', 'grad_norm': '0.3839', 'learning_rate': '0.0001466', 'entropy': '0.8219', 'num_tokens': '2.457e+07', 'mean_token_accuracy': '0.7778', 'epoch': '0.04018'}
{'loss': '0.9463', 'grad_norm': '0.3849', 'learning_rate': '0.0001466', 'entropy': '0.9871', 'num_tokens': '2.458e+07', 'mean_token_accuracy': '0.7452', 'epoch': '0.04019'}
{'loss': '0.7003', 'grad_norm': '0.326', 'learning_rate': '0.0001466', 'entropy': '0.7054', 'num_tokens': '2.458e+07', 'mean_token_accuracy': '0.8156', 'epoch': '0.0402'}
{'loss': '0.6983', 'grad_norm': '0.403', 'learning_rate': '0.0001466', 'entropy': '0.6947', 'num_tokens': '2.459e+07', 'mean_token_accuracy': '0.8248', 'epoch': '0.0402'}
{'loss': '0.8257', 'grad_norm': '0.4418', 'learning_rate': '0.0001465', 'entropy': '0.9433', 'num_tokens': '2.459e+07', 'mean_token_accuracy': '0.762', 'epoch': '0.04021'}
{'loss': '0.8463', 'grad_norm': '0.4277', 'learning_rate': '0.0001465', 'entropy': '0.8745', 'num_tokens': '2.46e+07', 'mean_token_accuracy': '0.7689', 'epoch': '0.04022'}
{'loss': '1.018', 'grad_norm': '0.3145', 'learning_rate': '0.0001465', 'entropy': '0.9788', 'num_tokens': '2.461e+07', 'mean_token_accuracy': '0.7588', 'epoch': '0.04023'}
{'loss': '1.114', 'grad_norm': '0.3808', 'learning_rate': '0.0001465', 'entropy': '1.113', 'num_tokens': '2.461e+07', 'mean_token_accuracy': '0.7277', 'epoch': '0.04024'}
{'loss': '0.7697', 'grad_norm': '0.3995', 'learning_rate': '0.0001465', 'entropy': '0.7751', 'num_tokens': '2.462e+07', 'mean_token_accuracy': '0.8023', 'epoch': '0.04025'}
{'loss': '0.785', 'grad_norm': '0.4439', 'learning_rate': '0.0001464', 'entropy': '0.8029', 'num_tokens': '2.462e+07', 'mean_token_accuracy': '0.8066', 'epoch': '0.04026'}
{'loss': '0.8132', 'grad_norm': '0.3308', 'learning_rate': '0.0001464', 'entropy': '0.8139', 'num_tokens': '2.462e+07', 'mean_token_accuracy': '0.7841', 'epoch': '0.04027'}
{'loss': '0.7842', 'grad_norm': '0.4985', 'learning_rate': '0.0001464', 'entropy': '0.7736', 'num_tokens': '2.463e+07', 'mean_token_accuracy': '0.8142', 'epoch': '0.04028'}
{'loss': '0.8689', 'grad_norm': '0.4266', 'learning_rate': '0.0001464', 'entropy': '0.9092', 'num_tokens': '2.463e+07', 'mean_token_accuracy': '0.7765', 'epoch': '0.04028'}
{'loss': '0.7293', 'grad_norm': '0.3417', 'learning_rate': '0.0001463', 'entropy': '0.78', 'num_tokens': '2.464e+07', 'mean_token_accuracy': '0.7901', 'epoch': '0.04029'}
{'loss': '0.8453', 'grad_norm': '0.397', 'learning_rate': '0.0001463', 'entropy': '0.986', 'num_tokens': '2.464e+07', 'mean_token_accuracy': '0.7695', 'epoch': '0.0403'}
{'loss': '0.6959', 'grad_norm': '0.4125', 'learning_rate': '0.0001463', 'entropy': '0.8571', 'num_tokens': '2.465e+07', 'mean_token_accuracy': '0.8028', 'epoch': '0.04031'}
{'loss': '0.7431', 'grad_norm': '0.4145', 'learning_rate': '0.0001463', 'entropy': '0.8044', 'num_tokens': '2.465e+07', 'mean_token_accuracy': '0.7843', 'epoch': '0.04032'}
{'loss': '0.7907', 'grad_norm': '0.4423', 'learning_rate': '0.0001462', 'entropy': '0.8305', 'num_tokens': '2.466e+07', 'mean_token_accuracy': '0.7777', 'epoch': '0.04033'}
{'loss': '0.9752', 'grad_norm': '0.3922', 'learning_rate': '0.0001462', 'entropy': '1.009', 'num_tokens': '2.466e+07', 'mean_token_accuracy': '0.7538', 'epoch': '0.04034'}
{'loss': '0.9448', 'grad_norm': '0.3787', 'learning_rate': '0.0001462', 'entropy': '1.029', 'num_tokens': '2.467e+07', 'mean_token_accuracy': '0.7501', 'epoch': '0.04035'}
{'loss': '0.7038', 'grad_norm': '0.4112', 'learning_rate': '0.0001462', 'entropy': '0.6881', 'num_tokens': '2.467e+07', 'mean_token_accuracy': '0.8177', 'epoch': '0.04036'}
{'loss': '0.7932', 'grad_norm': '0.4456', 'learning_rate': '0.0001461', 'entropy': '0.8254', 'num_tokens': '2.467e+07', 'mean_token_accuracy': '0.8011', 'epoch': '0.04036'}
{'loss': '1.022', 'grad_norm': '0.3966', 'learning_rate': '0.0001461', 'entropy': '0.9984', 'num_tokens': '2.468e+07', 'mean_token_accuracy': '0.7549', 'epoch': '0.04037'}

 38%|███▊      | 4563/12000 [2:43:01<3:36:57,  1.75s/it]
 38%|███▊      | 4564/12000 [2:43:04<3:51:20,  1.87s/it]
                                                        

 38%|███▊      | 4564/12000 [2:43:04<3:51:20,  1.87s/it]
 38%|███▊      | 4565/12000 [2:43:05<3:49:47,  1.85s/it]
                                                        

 38%|███▊      | 4565/12000 [2:43:05<3:49:47,  1.85s/it]
 38%|███▊      | 4566/12000 [2:43:08<4:17:16,  2.08s/it]
                                                        

 38%|███▊      | 4566/12000 [2:43:08<4:17:16,  2.08s/it]
 38%|███▊      | 4567/12000 [2:43:09<3:57:12,  1.91s/it]
                                                        

 38%|███▊      | 4567/12000 [2:43:09<3:57:12,  1.91s/it]
 38%|███▊      | 4568/12000 [2:43:12<4:10:47,  2.02s/it]
                                                        

 38%|███▊      | 4568/12000 [2:43:12<4:10:47,  2.02s/it]
 38%|███▊      | 4569/12000 [2:43:16<5:33:48,  2.70s/it]
                                                        

 38%|███▊      | 4569/12000 [2:43:16<5:33:48,  2.70s/it]
 38%|███▊      | 4570/12000 [2:43:18<5:00:08,  2.42s/it]
                                                        

 38%|███▊      | 4570/12000 [2:43:18<5:00:08,  2.42s/it]
 38%|███▊      | 4571/12000 [2:43:22<6:12:27,  3.01s/it]
                                                        

 38%|███▊      | 4571/12000 [2:43:22<6:12:27,  3.01s/it]
 38%|███▊      | 4572/12000 [2:43:24<5:24:46,  2.62s/it]
                                                        

 38%|███▊      | 4572/12000 [2:43:24<5:24:46,  2.62s/it]
 38%|███▊      | 4573/12000 [2:43:26<5:03:02,  2.45s/it]
                                                        

 38%|███▊      | 4573/12000 [2:43:26<5:03:02,  2.45s/it]
 38%|███▊      | 4574/12000 [2:43:27<4:28:04,  2.17s/it]
                                                        

 38%|███▊      | 4574/12000 [2:43:27<4:28:04,  2.17s/it]
 38%|███▊      | 4575/12000 [2:43:33<6:17:40,  3.05s/it]
                                                        

 38%|███▊      | 4575/12000 [2:43:33<6:17:40,  3.05s/it]
 38%|███▊      | 4576/12000 [2:43:36<6:16:40,  3.04s/it]
                                                        

 38%|███▊      | 4576/12000 [2:43:36<6:16:40,  3.04s/it]
 38%|███▊      | 4577/12000 [2:43:39<6:19:44,  3.07s/it]
                                                        

 38%|███▊      | 4577/12000 [2:43:39<6:19:44,  3.07s/it]
 38%|███▊      | 4578/12000 [2:43:44<7:34:00,  3.67s/it]
                                                        

 38%|███▊      | 4578/12000 [2:43:44<7:34:00,  3.67s/it]
 38%|███▊      | 4579/12000 [2:43:46<6:49:36,  3.31s/it]
                                                        

 38%|███▊      | 4579/12000 [2:43:46<6:49:36,  3.31s/it]
 38%|███▊      | 4580/12000 [2:43:48<5:43:08,  2.77s/it]
                                                        

 38%|███▊      | 4580/12000 [2:43:48<5:43:08,  2.77s/it]
 38%|███▊      | 4581/12000 [2:43:51<5:44:27,  2.79s/it]
                                                        

 38%|███▊      | 4581/12000 [2:43:51<5:44:27,  2.79s/it]
 38%|███▊      | 4582/12000 [2:43:52<5:02:14,  2.44s/it]
                                                        

 38%|███▊      | 4582/12000 [2:43:52<5:02:14,  2.44s/it]
 38%|███▊      | 4583/12000 [2:43:55<5:03:49,  2.46s/it]
                                                        

 38%|███▊      | 4583/12000 [2:43:55<5:03:49,  2.46s/it]
 38%|███▊      | 4584/12000 [2:43:57<4:40:39,  2.27s/it]
                                                        

 38%|███▊      | 4584/12000 [2:43:57<4:40:39,  2.27s/it]
 38%|███▊      | 4585/12000 [2:43:58<4:20:43,  2.11s/it]
                                                        

 38%|███▊      | 4585/12000 [2:43:58<4:20:43,  2.11s/it]
 38%|███▊      | 4586/12000 [2:44:00<4:02:35,  1.96s/it]
                                                        

 38%|███▊      | 4586/12000 [2:44:00<4:02:35,  1.96s/it]
 38%|███▊      | 4587/12000 [2:44:02<4:19:52,  2.10s/it]
                                                        

 38%|███▊      | 4587/12000 [2:44:02<4:19:52,  2.10s/it]
 38%|███▊      | 4588/12000 [2:44:04<3:58:18,  1.93s/it]
                                                        

 38%|███▊      | 4588/12000 [2:44:04<3:58:18,  1.93s/it]
 38%|███▊      | 4589/12000 [2:44:06<4:23:17,  2.13s/it]
                                                        

 38%|███▊      | 4589/12000 [2:44:06<4:23:17,  2.13s/it]
 38%|███▊      | 4590/12000 [2:44:08<4:12:17,  2.04s/it]
                                                        

 38%|███▊      | 4590/12000 [2:44:08<4:12:17,  2.04s/it]
 38%|███▊      | 4591/12000 [2:44:12<5:07:10,  2.49s/it]
                                                        

 38%|███▊      | 4591/12000 [2:44:12<5:07:10,  2.49s/it]
 38%|███▊      | 4592/12000 [2:44:14<4:41:25,  2.28s/it]
                                                        

 38%|███▊      | 4592/12000 [2:44:14<4:41:25,  2.28s/it]
 38%|███▊      | 4593/12000 [2:44:15<4:20:50,  2.11s/it]
                                                        

 38%|███▊      | 4593/12000 [2:44:15<4:20:50,  2.11s/it]
 38%|███▊      | 4594/12000 [2:44:17<4:02:00,  1.96s/it]
                                                        

 38%|███▊      | 4594/12000 [2:44:17<4:02:00,  1.96s/it]
 38%|███▊      | 4595/12000 [2:44:19<3:50:01,  1.86s/it]
                                                        

 38%|███▊      | 4595/12000 [2:44:19<3:50:01,  1.86s/it]
 38%|███▊      | 4596/12000 [2:44:21<3:51:44,  1.88s/it]
                                                        

 38%|███▊      | 4596/12000 [2:44:21<3:51:44,  1.88s/it]
 38%|███▊      | 4597/12000 [2:44:22<3:39:31,  1.78s/it]
                                                        

 38%|███▊      | 4597/12000 [2:44:22<3:39:31,  1.78s/it]
 38%|███▊      | 4598/12000 [2:44:24<3:30:23,  1.71s/it]
                                                        

 38%|███▊      | 4598/12000 [2:44:24<3:30:23,  1.71s/it]
 38%|███▊      | 4599/12000 [2:44:26<4:00:37,  1.95s/it]
                                                        

 38%|███▊      | 4599/12000 [2:44:26<4:00:37,  1.95s/it]
 38%|███▊      | 4600/12000 [2:44:28<3:47:45,  1.85s/it]
                                                        

 38%|███▊      | 4600/12000 [2:44:28<3:47:45,  1.85s/it]
 38%|███▊      | 4601/12000 [2:44:31<4:36:23,  2.24s/it]
                                                        

 38%|███▊      | 4601/12000 [2:44:31<4:36:23,  2.24s/it]
 38%|███▊      | 4602/12000 [2:44:32<4:09:02,  2.02s/it]
                                                        

 38%|███▊      | 4602/12000 [2:44:32<4:09:02,  2.02s/it]
 38%|███▊      | 4603/12000 [2:44:34<4:00:17,  1.95s/it]
                                                        

 38%|███▊      | 4603/12000 [2:44:34<4:00:17,  1.95s/it]
 38%|███▊      | 4604/12000 [2:44:36<3:48:02,  1.85s/it]
                                                        

 38%|███▊      | 4604/12000 [2:44:36<3:48:02,  1.85s/it]
 38%|███▊      | 4605/12000 [2:44:38<4:07:48,  2.01s/it]
                                                        

 38%|███▊      | 4605/12000 [2:44:38<4:07:48,  2.01s/it]
 38%|███▊      | 4606/12000 [2:44:40<3:58:54,  1.94s/it]
                                                        

 38%|███▊      | 4606/12000 [2:44:40<3:58:54,  1.94s/it]
 38%|███▊      | 4607/12000 [2:44:44<5:09:29,  2.51s/it]
                                                        

 38%|███▊      | 4607/12000 [2:44:44<5:09:29,  2.51s/it]
 38%|███▊      | 4608/12000 [2:44:46<5:06:34,  2.49s/it]
                                                        

 38%|███▊      | 4608/12000 [2:44:46<5:06:34,  2.49s/it]
 38%|███▊      | 4609/12000 [2:44:48<4:38:39,  2.26s/it]
                                                        

 38%|███▊      | 4609/12000 [2:44:48<4:38:39,  2.26s/it]
 38%|███▊      | 4610/12000 [2:44:50<4:12:38,  2.05s/it]
                                                        
{'loss': '1.017', 'grad_norm': '0.5178', 'learning_rate': '0.0001461', 'entropy': '0.9795', 'num_tokens': '2.468e+07', 'mean_token_accuracy': '0.7663', 'epoch': '0.04038'}
{'loss': '0.9274', 'grad_norm': '0.4331', 'learning_rate': '0.0001461', 'entropy': '0.8967', 'num_tokens': '2.469e+07', 'mean_token_accuracy': '0.7773', 'epoch': '0.04039'}
{'loss': '0.7107', 'grad_norm': '0.3698', 'learning_rate': '0.000146', 'entropy': '0.8552', 'num_tokens': '2.469e+07', 'mean_token_accuracy': '0.7952', 'epoch': '0.0404'}
{'loss': '0.6775', 'grad_norm': '0.3151', 'learning_rate': '0.000146', 'entropy': '0.6807', 'num_tokens': '2.47e+07', 'mean_token_accuracy': '0.827', 'epoch': '0.04041'}
{'loss': '0.6149', 'grad_norm': '0.3234', 'learning_rate': '0.000146', 'entropy': '0.6133', 'num_tokens': '2.471e+07', 'mean_token_accuracy': '0.8459', 'epoch': '0.04042'}
{'loss': '1.186', 'grad_norm': '0.4424', 'learning_rate': '0.000146', 'entropy': '1.119', 'num_tokens': '2.471e+07', 'mean_token_accuracy': '0.7364', 'epoch': '0.04043'}
{'loss': '0.9752', 'grad_norm': '0.3548', 'learning_rate': '0.0001459', 'entropy': '0.9456', 'num_tokens': '2.472e+07', 'mean_token_accuracy': '0.7637', 'epoch': '0.04043'}
{'loss': '1.127', 'grad_norm': '0.4657', 'learning_rate': '0.0001459', 'entropy': '1.112', 'num_tokens': '2.472e+07', 'mean_token_accuracy': '0.7389', 'epoch': '0.04044'}
{'loss': '0.8935', 'grad_norm': '0.2876', 'learning_rate': '0.0001459', 'entropy': '0.8949', 'num_tokens': '2.473e+07', 'mean_token_accuracy': '0.7799', 'epoch': '0.04045'}
{'loss': '1.083', 'grad_norm': '0.4267', 'learning_rate': '0.0001459', 'entropy': '1.107', 'num_tokens': '2.474e+07', 'mean_token_accuracy': '0.7359', 'epoch': '0.04046'}
{'loss': '1.118', 'grad_norm': '0.4602', 'learning_rate': '0.0001458', 'entropy': '1.089', 'num_tokens': '2.474e+07', 'mean_token_accuracy': '0.714', 'epoch': '0.04047'}
{'loss': '1.001', 'grad_norm': '0.4283', 'learning_rate': '0.0001458', 'entropy': '1.042', 'num_tokens': '2.475e+07', 'mean_token_accuracy': '0.7517', 'epoch': '0.04048'}
{'loss': '1.333', 'grad_norm': '0.4817', 'learning_rate': '0.0001458', 'entropy': '1.1', 'num_tokens': '2.476e+07', 'mean_token_accuracy': '0.7435', 'epoch': '0.04049'}
{'loss': '0.7591', 'grad_norm': '0.3569', 'learning_rate': '0.0001458', 'entropy': '0.7441', 'num_tokens': '2.476e+07', 'mean_token_accuracy': '0.8062', 'epoch': '0.0405'}
{'loss': '1.143', 'grad_norm': '0.3596', 'learning_rate': '0.0001457', 'entropy': '1.099', 'num_tokens': '2.477e+07', 'mean_token_accuracy': '0.7439', 'epoch': '0.04051'}
{'loss': '1.309', 'grad_norm': '0.3396', 'learning_rate': '0.0001457', 'entropy': '1.102', 'num_tokens': '2.478e+07', 'mean_token_accuracy': '0.7484', 'epoch': '0.04051'}
{'loss': '0.7856', 'grad_norm': '0.3708', 'learning_rate': '0.0001457', 'entropy': '0.8223', 'num_tokens': '2.479e+07', 'mean_token_accuracy': '0.7923', 'epoch': '0.04052'}
{'loss': '1.085', 'grad_norm': '0.3754', 'learning_rate': '0.0001457', 'entropy': '1.072', 'num_tokens': '2.479e+07', 'mean_token_accuracy': '0.752', 'epoch': '0.04053'}
{'loss': '0.9268', 'grad_norm': '0.4358', 'learning_rate': '0.0001456', 'entropy': '0.8666', 'num_tokens': '2.48e+07', 'mean_token_accuracy': '0.7888', 'epoch': '0.04054'}
{'loss': '0.7686', 'grad_norm': '0.3813', 'learning_rate': '0.0001456', 'entropy': '0.721', 'num_tokens': '2.481e+07', 'mean_token_accuracy': '0.8154', 'epoch': '0.04055'}
{'loss': '0.522', 'grad_norm': '0.2691', 'learning_rate': '0.0001456', 'entropy': '0.6113', 'num_tokens': '2.481e+07', 'mean_token_accuracy': '0.8429', 'epoch': '0.04056'}
{'loss': '0.9626', 'grad_norm': '0.3626', 'learning_rate': '0.0001456', 'entropy': '0.9964', 'num_tokens': '2.482e+07', 'mean_token_accuracy': '0.7558', 'epoch': '0.04057'}
{'loss': '0.8072', 'grad_norm': '0.3813', 'learning_rate': '0.0001455', 'entropy': '0.7674', 'num_tokens': '2.482e+07', 'mean_token_accuracy': '0.8078', 'epoch': '0.04058'}
{'loss': '1.099', 'grad_norm': '0.4207', 'learning_rate': '0.0001455', 'entropy': '1.075', 'num_tokens': '2.483e+07', 'mean_token_accuracy': '0.7381', 'epoch': '0.04059'}
{'loss': '1.018', 'grad_norm': '0.46', 'learning_rate': '0.0001455', 'entropy': '1.042', 'num_tokens': '2.483e+07', 'mean_token_accuracy': '0.7499', 'epoch': '0.04059'}
{'loss': '0.9552', 'grad_norm': '0.4043', 'learning_rate': '0.0001455', 'entropy': '0.9412', 'num_tokens': '2.484e+07', 'mean_token_accuracy': '0.7583', 'epoch': '0.0406'}
{'loss': '0.9455', 'grad_norm': '0.3566', 'learning_rate': '0.0001454', 'entropy': '0.7735', 'num_tokens': '2.484e+07', 'mean_token_accuracy': '0.7999', 'epoch': '0.04061'}
{'loss': '0.7462', 'grad_norm': '0.3554', 'learning_rate': '0.0001454', 'entropy': '0.7619', 'num_tokens': '2.485e+07', 'mean_token_accuracy': '0.7999', 'epoch': '0.04062'}
{'loss': '0.8154', 'grad_norm': '0.3884', 'learning_rate': '0.0001454', 'entropy': '0.8122', 'num_tokens': '2.485e+07', 'mean_token_accuracy': '0.7955', 'epoch': '0.04063'}
{'loss': '0.8571', 'grad_norm': '0.3538', 'learning_rate': '0.0001454', 'entropy': '0.804', 'num_tokens': '2.486e+07', 'mean_token_accuracy': '0.7947', 'epoch': '0.04064'}
{'loss': '0.8539', 'grad_norm': '0.3874', 'learning_rate': '0.0001453', 'entropy': '0.788', 'num_tokens': '2.486e+07', 'mean_token_accuracy': '0.7978', 'epoch': '0.04065'}
{'loss': '0.8713', 'grad_norm': '0.5193', 'learning_rate': '0.0001453', 'entropy': '0.8857', 'num_tokens': '2.487e+07', 'mean_token_accuracy': '0.7863', 'epoch': '0.04066'}
{'loss': '0.7415', 'grad_norm': '0.3887', 'learning_rate': '0.0001453', 'entropy': '0.7818', 'num_tokens': '2.487e+07', 'mean_token_accuracy': '0.8002', 'epoch': '0.04066'}
{'loss': '0.8829', 'grad_norm': '0.3164', 'learning_rate': '0.0001453', 'entropy': '0.8891', 'num_tokens': '2.488e+07', 'mean_token_accuracy': '0.7715', 'epoch': '0.04067'}
{'loss': '1.033', 'grad_norm': '0.4392', 'learning_rate': '0.0001453', 'entropy': '1.099', 'num_tokens': '2.488e+07', 'mean_token_accuracy': '0.7368', 'epoch': '0.04068'}
{'loss': '0.9964', 'grad_norm': '0.4645', 'learning_rate': '0.0001452', 'entropy': '0.957', 'num_tokens': '2.489e+07', 'mean_token_accuracy': '0.7658', 'epoch': '0.04069'}
{'loss': '1.383', 'grad_norm': '0.4171', 'learning_rate': '0.0001452', 'entropy': '1.281', 'num_tokens': '2.489e+07', 'mean_token_accuracy': '0.7145', 'epoch': '0.0407'}
{'loss': '0.6928', 'grad_norm': '0.3412', 'learning_rate': '0.0001452', 'entropy': '0.7415', 'num_tokens': '2.49e+07', 'mean_token_accuracy': '0.8273', 'epoch': '0.04071'}
{'loss': '0.7039', 'grad_norm': '0.3832', 'learning_rate': '0.0001452', 'entropy': '0.7045', 'num_tokens': '2.49e+07', 'mean_token_accuracy': '0.8094', 'epoch': '0.04072'}
{'loss': '0.8269', 'grad_norm': '0.3258', 'learning_rate': '0.0001451', 'entropy': '0.8386', 'num_tokens': '2.49e+07', 'mean_token_accuracy': '0.7999', 'epoch': '0.04073'}
{'loss': '0.8493', 'grad_norm': '0.363', 'learning_rate': '0.0001451', 'entropy': '0.9757', 'num_tokens': '2.491e+07', 'mean_token_accuracy': '0.7651', 'epoch': '0.04074'}
{'loss': '0.6148', 'grad_norm': '0.2969', 'learning_rate': '0.0001451', 'entropy': '0.6066', 'num_tokens': '2.491e+07', 'mean_token_accuracy': '0.8398', 'epoch': '0.04074'}
{'loss': '1.308', 'grad_norm': '0.4096', 'learning_rate': '0.0001451', 'entropy': '1.272', 'num_tokens': '2.492e+07', 'mean_token_accuracy': '0.7028', 'epoch': '0.04075'}
{'loss': '0.8631', 'grad_norm': '0.3973', 'learning_rate': '0.000145', 'entropy': '0.8802', 'num_tokens': '2.493e+07', 'mean_token_accuracy': '0.7766', 'epoch': '0.04076'}
{'loss': '1.186', 'grad_norm': '0.3857', 'learning_rate': '0.000145', 'entropy': '1.051', 'num_tokens': '2.493e+07', 'mean_token_accuracy': '0.7356', 'epoch': '0.04077'}
{'loss': '0.651', 'grad_norm': '0.2932', 'learning_rate': '0.000145', 'entropy': '0.5827', 'num_tokens': '2.494e+07', 'mean_token_accuracy': '0.8496', 'epoch': '0.04078'}
{'loss': '0.7619', 'grad_norm': '0.4742', 'learning_rate': '0.000145', 'entropy': '0.7579', 'num_tokens': '2.495e+07', 'mean_token_accuracy': '0.7939', 'epoch': '0.04079'}

 38%|███▊      | 4610/12000 [2:44:50<4:12:38,  2.05s/it]
 38%|███▊      | 4611/12000 [2:44:51<3:59:48,  1.95s/it]
                                                        

 38%|███▊      | 4611/12000 [2:44:51<3:59:48,  1.95s/it]
 38%|███▊      | 4612/12000 [2:44:54<4:28:12,  2.18s/it]
                                                        

 38%|███▊      | 4612/12000 [2:44:54<4:28:12,  2.18s/it]
 38%|███▊      | 4613/12000 [2:44:58<5:28:42,  2.67s/it]
                                                        

 38%|███▊      | 4613/12000 [2:44:58<5:28:42,  2.67s/it]
 38%|███▊      | 4614/12000 [2:45:00<5:11:59,  2.53s/it]
                                                        

 38%|███▊      | 4614/12000 [2:45:00<5:11:59,  2.53s/it]
 38%|███▊      | 4615/12000 [2:45:02<4:50:58,  2.36s/it]
                                                        

 38%|███▊      | 4615/12000 [2:45:02<4:50:58,  2.36s/it]
 38%|███▊      | 4616/12000 [2:45:04<4:54:08,  2.39s/it]
                                                        

 38%|███▊      | 4616/12000 [2:45:04<4:54:08,  2.39s/it]
 38%|███▊      | 4617/12000 [2:45:06<4:26:03,  2.16s/it]
                                                        

 38%|███▊      | 4617/12000 [2:45:06<4:26:03,  2.16s/it]
 38%|███▊      | 4618/12000 [2:45:10<5:22:06,  2.62s/it]
                                                        

 38%|███▊      | 4618/12000 [2:45:10<5:22:06,  2.62s/it]
 38%|███▊      | 4619/12000 [2:45:11<4:42:36,  2.30s/it]
                                                        

 38%|███▊      | 4619/12000 [2:45:11<4:42:36,  2.30s/it]
 38%|███▊      | 4620/12000 [2:45:13<4:19:49,  2.11s/it]
                                                        

 38%|███▊      | 4620/12000 [2:45:13<4:19:49,  2.11s/it]
 39%|███▊      | 4621/12000 [2:45:15<4:25:54,  2.16s/it]
                                                        

 39%|███▊      | 4621/12000 [2:45:15<4:25:54,  2.16s/it]
 39%|███▊      | 4622/12000 [2:45:17<4:09:17,  2.03s/it]
                                                        

 39%|███▊      | 4622/12000 [2:45:17<4:09:17,  2.03s/it]
 39%|███▊      | 4623/12000 [2:45:19<4:07:35,  2.01s/it]
                                                        

 39%|███▊      | 4623/12000 [2:45:19<4:07:35,  2.01s/it]
 39%|███▊      | 4624/12000 [2:45:21<3:54:32,  1.91s/it]
                                                        

 39%|███▊      | 4624/12000 [2:45:21<3:54:32,  1.91s/it]
 39%|███▊      | 4625/12000 [2:45:24<4:35:05,  2.24s/it]
                                                        

 39%|███▊      | 4625/12000 [2:45:24<4:35:05,  2.24s/it]
 39%|███▊      | 4626/12000 [2:45:25<4:07:06,  2.01s/it]
                                                        

 39%|███▊      | 4626/12000 [2:45:25<4:07:06,  2.01s/it]
 39%|███▊      | 4627/12000 [2:45:27<3:55:40,  1.92s/it]
                                                        

 39%|███▊      | 4627/12000 [2:45:27<3:55:40,  1.92s/it]
 39%|███▊      | 4628/12000 [2:45:29<3:55:08,  1.91s/it]
                                                        

 39%|███▊      | 4628/12000 [2:45:29<3:55:08,  1.91s/it]
 39%|███▊      | 4629/12000 [2:45:30<3:40:51,  1.80s/it]
                                                        

 39%|███▊      | 4629/12000 [2:45:30<3:40:51,  1.80s/it]
 39%|███▊      | 4630/12000 [2:45:32<3:33:35,  1.74s/it]
                                                        

 39%|███▊      | 4630/12000 [2:45:32<3:33:35,  1.74s/it]
 39%|███▊      | 4631/12000 [2:45:33<3:24:52,  1.67s/it]
                                                        

 39%|███▊      | 4631/12000 [2:45:33<3:24:52,  1.67s/it]
 39%|███▊      | 4632/12000 [2:45:35<3:26:37,  1.68s/it]
                                                        

 39%|███▊      | 4632/12000 [2:45:35<3:26:37,  1.68s/it]
 39%|███▊      | 4633/12000 [2:45:37<3:24:29,  1.67s/it]
                                                        

 39%|███▊      | 4633/12000 [2:45:37<3:24:29,  1.67s/it]
 39%|███▊      | 4634/12000 [2:45:38<3:27:18,  1.69s/it]
                                                        

 39%|███▊      | 4634/12000 [2:45:38<3:27:18,  1.69s/it]
 39%|███▊      | 4635/12000 [2:45:42<4:40:04,  2.28s/it]
                                                        

 39%|███▊      | 4635/12000 [2:45:42<4:40:04,  2.28s/it]
 39%|███▊      | 4636/12000 [2:45:44<4:38:52,  2.27s/it]
                                                        

 39%|███▊      | 4636/12000 [2:45:44<4:38:52,  2.27s/it]
 39%|███▊      | 4637/12000 [2:45:46<4:19:24,  2.11s/it]
                                                        

 39%|███▊      | 4637/12000 [2:45:46<4:19:24,  2.11s/it]
 39%|███▊      | 4638/12000 [2:45:48<4:07:20,  2.02s/it]
                                                        

 39%|███▊      | 4638/12000 [2:45:48<4:07:20,  2.02s/it]
 39%|███▊      | 4639/12000 [2:45:52<5:34:18,  2.73s/it]
                                                        

 39%|███▊      | 4639/12000 [2:45:52<5:34:18,  2.73s/it]
 39%|███▊      | 4640/12000 [2:45:54<5:07:54,  2.51s/it]
                                                        

 39%|███▊      | 4640/12000 [2:45:54<5:07:54,  2.51s/it]
 39%|███▊      | 4641/12000 [2:45:56<4:34:36,  2.24s/it]
                                                        

 39%|███▊      | 4641/12000 [2:45:56<4:34:36,  2.24s/it]
 39%|███▊      | 4642/12000 [2:45:58<4:22:29,  2.14s/it]
                                                        

 39%|███▊      | 4642/12000 [2:45:58<4:22:29,  2.14s/it]
 39%|███▊      | 4643/12000 [2:46:00<4:30:55,  2.21s/it]
                                                        

 39%|███▊      | 4643/12000 [2:46:00<4:30:55,  2.21s/it]
 39%|███▊      | 4644/12000 [2:46:02<4:30:38,  2.21s/it]
                                                        

 39%|███▊      | 4644/12000 [2:46:02<4:30:38,  2.21s/it]
 39%|███▊      | 4645/12000 [2:46:04<4:13:04,  2.06s/it]
                                                        

 39%|███▊      | 4645/12000 [2:46:04<4:13:04,  2.06s/it]
 39%|███▊      | 4646/12000 [2:46:06<4:00:32,  1.96s/it]
                                                        

 39%|███▊      | 4646/12000 [2:46:06<4:00:32,  1.96s/it]
 39%|███▊      | 4647/12000 [2:46:08<3:56:50,  1.93s/it]
                                                        

 39%|███▊      | 4647/12000 [2:46:08<3:56:50,  1.93s/it]
 39%|███▊      | 4648/12000 [2:46:10<3:59:39,  1.96s/it]
                                                        

 39%|███▊      | 4648/12000 [2:46:10<3:59:39,  1.96s/it]
 39%|███▊      | 4649/12000 [2:46:11<3:47:34,  1.86s/it]
                                                        

 39%|███▊      | 4649/12000 [2:46:11<3:47:34,  1.86s/it]
 39%|███▉      | 4650/12000 [2:46:13<3:35:12,  1.76s/it]
                                                        

 39%|███▉      | 4650/12000 [2:46:13<3:35:12,  1.76s/it]
 39%|███▉      | 4651/12000 [2:46:16<4:29:11,  2.20s/it]
                                                        

 39%|███▉      | 4651/12000 [2:46:16<4:29:11,  2.20s/it]
 39%|███▉      | 4652/12000 [2:46:18<4:04:35,  2.00s/it]
                                                        

 39%|███▉      | 4652/12000 [2:46:18<4:04:35,  2.00s/it]
 39%|███▉      | 4653/12000 [2:46:21<4:44:27,  2.32s/it]
                                                        

 39%|███▉      | 4653/12000 [2:46:21<4:44:27,  2.32s/it]
 39%|███▉      | 4654/12000 [2:46:23<4:58:28,  2.44s/it]
                                                        

 39%|███▉      | 4654/12000 [2:46:23<4:58:28,  2.44s/it]
 39%|███▉      | 4655/12000 [2:46:25<4:24:54,  2.16s/it]
                                                        

 39%|███▉      | 4655/12000 [2:46:25<4:24:54,  2.16s/it]
 39%|███▉      | 4656/12000 [2:46:26<3:54:50,  1.92s/it]
                                                        

 39%|███▉      | 4656/12000 [2:46:26<3:54:50,  1.92s/it]
 39%|███▉      | 4657/12000 [2:46:32<6:07:20,  3.00s/it]
                                                        
{'loss': '1.058', 'grad_norm': '0.4393', 'learning_rate': '0.0001449', 'entropy': '0.9745', 'num_tokens': '2.495e+07', 'mean_token_accuracy': '0.7701', 'epoch': '0.0408'}
{'loss': '0.8522', 'grad_norm': '0.4194', 'learning_rate': '0.0001449', 'entropy': '0.8851', 'num_tokens': '2.495e+07', 'mean_token_accuracy': '0.7769', 'epoch': '0.04081'}
{'loss': '0.8026', 'grad_norm': '0.3355', 'learning_rate': '0.0001449', 'entropy': '0.9743', 'num_tokens': '2.496e+07', 'mean_token_accuracy': '0.7585', 'epoch': '0.04082'}
{'loss': '0.7832', 'grad_norm': '0.4492', 'learning_rate': '0.0001449', 'entropy': '0.8128', 'num_tokens': '2.497e+07', 'mean_token_accuracy': '0.8004', 'epoch': '0.04082'}
{'loss': '0.828', 'grad_norm': '0.3223', 'learning_rate': '0.0001448', 'entropy': '0.7999', 'num_tokens': '2.497e+07', 'mean_token_accuracy': '0.7934', 'epoch': '0.04083'}
{'loss': '1.049', 'grad_norm': '0.4254', 'learning_rate': '0.0001448', 'entropy': '1.008', 'num_tokens': '2.498e+07', 'mean_token_accuracy': '0.7467', 'epoch': '0.04084'}
{'loss': '0.795', 'grad_norm': '0.3828', 'learning_rate': '0.0001448', 'entropy': '0.8202', 'num_tokens': '2.499e+07', 'mean_token_accuracy': '0.7878', 'epoch': '0.04085'}
{'loss': '0.5217', 'grad_norm': '0.3809', 'learning_rate': '0.0001448', 'entropy': '0.5336', 'num_tokens': '2.499e+07', 'mean_token_accuracy': '0.8499', 'epoch': '0.04086'}
{'loss': '1.257', 'grad_norm': '0.394', 'learning_rate': '0.0001447', 'entropy': '1.113', 'num_tokens': '2.5e+07', 'mean_token_accuracy': '0.7436', 'epoch': '0.04087'}
{'loss': '0.6372', 'grad_norm': '0.347', 'learning_rate': '0.0001447', 'entropy': '0.6764', 'num_tokens': '2.5e+07', 'mean_token_accuracy': '0.8291', 'epoch': '0.04088'}
{'loss': '0.813', 'grad_norm': '0.4247', 'learning_rate': '0.0001447', 'entropy': '0.8929', 'num_tokens': '2.501e+07', 'mean_token_accuracy': '0.7871', 'epoch': '0.04089'}
{'loss': '0.5928', 'grad_norm': '0.4946', 'learning_rate': '0.0001447', 'entropy': '0.807', 'num_tokens': '2.501e+07', 'mean_token_accuracy': '0.8094', 'epoch': '0.0409'}
{'loss': '0.7389', 'grad_norm': '0.3625', 'learning_rate': '0.0001446', 'entropy': '0.748', 'num_tokens': '2.502e+07', 'mean_token_accuracy': '0.81', 'epoch': '0.0409'}
{'loss': '0.8885', 'grad_norm': '0.3491', 'learning_rate': '0.0001446', 'entropy': '0.8941', 'num_tokens': '2.502e+07', 'mean_token_accuracy': '0.7832', 'epoch': '0.04091'}
{'loss': '1.009', 'grad_norm': '0.4086', 'learning_rate': '0.0001446', 'entropy': '1.125', 'num_tokens': '2.503e+07', 'mean_token_accuracy': '0.7476', 'epoch': '0.04092'}
{'loss': '0.7445', 'grad_norm': '0.4005', 'learning_rate': '0.0001446', 'entropy': '0.8846', 'num_tokens': '2.503e+07', 'mean_token_accuracy': '0.7922', 'epoch': '0.04093'}
{'loss': '0.9111', 'grad_norm': '0.3653', 'learning_rate': '0.0001445', 'entropy': '0.9162', 'num_tokens': '2.504e+07', 'mean_token_accuracy': '0.7733', 'epoch': '0.04094'}
{'loss': '1.168', 'grad_norm': '0.4836', 'learning_rate': '0.0001445', 'entropy': '1.168', 'num_tokens': '2.504e+07', 'mean_token_accuracy': '0.734', 'epoch': '0.04095'}
{'loss': '0.7827', 'grad_norm': '0.3766', 'learning_rate': '0.0001445', 'entropy': '0.7562', 'num_tokens': '2.505e+07', 'mean_token_accuracy': '0.802', 'epoch': '0.04096'}
{'loss': '0.6955', 'grad_norm': '0.3999', 'learning_rate': '0.0001445', 'entropy': '0.729', 'num_tokens': '2.505e+07', 'mean_token_accuracy': '0.8086', 'epoch': '0.04097'}
{'loss': '0.6226', 'grad_norm': '0.302', 'learning_rate': '0.0001444', 'entropy': '0.6634', 'num_tokens': '2.506e+07', 'mean_token_accuracy': '0.8231', 'epoch': '0.04097'}
{'loss': '0.7177', 'grad_norm': '0.3783', 'learning_rate': '0.0001444', 'entropy': '0.7744', 'num_tokens': '2.506e+07', 'mean_token_accuracy': '0.8158', 'epoch': '0.04098'}
{'loss': '0.646', 'grad_norm': '0.3394', 'learning_rate': '0.0001444', 'entropy': '0.6703', 'num_tokens': '2.507e+07', 'mean_token_accuracy': '0.8203', 'epoch': '0.04099'}
{'loss': '0.687', 'grad_norm': '0.3425', 'learning_rate': '0.0001444', 'entropy': '0.743', 'num_tokens': '2.507e+07', 'mean_token_accuracy': '0.8168', 'epoch': '0.041'}
{'loss': '0.8779', 'grad_norm': '0.4647', 'learning_rate': '0.0001443', 'entropy': '0.8733', 'num_tokens': '2.508e+07', 'mean_token_accuracy': '0.7686', 'epoch': '0.04101'}
{'loss': '0.6871', 'grad_norm': '0.4439', 'learning_rate': '0.0001443', 'entropy': '0.7112', 'num_tokens': '2.508e+07', 'mean_token_accuracy': '0.8015', 'epoch': '0.04102'}
{'loss': '0.624', 'grad_norm': '0.2899', 'learning_rate': '0.0001443', 'entropy': '0.6095', 'num_tokens': '2.509e+07', 'mean_token_accuracy': '0.8448', 'epoch': '0.04103'}
{'loss': '0.7399', 'grad_norm': '0.3518', 'learning_rate': '0.0001443', 'entropy': '0.7523', 'num_tokens': '2.509e+07', 'mean_token_accuracy': '0.8039', 'epoch': '0.04104'}
{'loss': '1.117', 'grad_norm': '0.4738', 'learning_rate': '0.0001442', 'entropy': '1.117', 'num_tokens': '2.51e+07', 'mean_token_accuracy': '0.7348', 'epoch': '0.04105'}
{'loss': '0.8019', 'grad_norm': '0.3971', 'learning_rate': '0.0001442', 'entropy': '0.8557', 'num_tokens': '2.511e+07', 'mean_token_accuracy': '0.7855', 'epoch': '0.04105'}
{'loss': '1.143', 'grad_norm': '0.4918', 'learning_rate': '0.0001442', 'entropy': '1.074', 'num_tokens': '2.511e+07', 'mean_token_accuracy': '0.7372', 'epoch': '0.04106'}
{'loss': '0.9489', 'grad_norm': '0.3353', 'learning_rate': '0.0001442', 'entropy': '0.9494', 'num_tokens': '2.512e+07', 'mean_token_accuracy': '0.7697', 'epoch': '0.04107'}
{'loss': '1.013', 'grad_norm': '0.4383', 'learning_rate': '0.0001441', 'entropy': '0.9893', 'num_tokens': '2.512e+07', 'mean_token_accuracy': '0.763', 'epoch': '0.04108'}
{'loss': '0.7846', 'grad_norm': '0.4', 'learning_rate': '0.0001441', 'entropy': '0.8798', 'num_tokens': '2.513e+07', 'mean_token_accuracy': '0.7905', 'epoch': '0.04109'}
{'loss': '0.8435', 'grad_norm': '0.4085', 'learning_rate': '0.0001441', 'entropy': '0.8501', 'num_tokens': '2.513e+07', 'mean_token_accuracy': '0.7933', 'epoch': '0.0411'}
{'loss': '0.8036', 'grad_norm': '0.4658', 'learning_rate': '0.0001441', 'entropy': '0.8133', 'num_tokens': '2.514e+07', 'mean_token_accuracy': '0.7881', 'epoch': '0.04111'}
{'loss': '0.7204', 'grad_norm': '0.3972', 'learning_rate': '0.000144', 'entropy': '0.7665', 'num_tokens': '2.514e+07', 'mean_token_accuracy': '0.8018', 'epoch': '0.04112'}
{'loss': '0.8228', 'grad_norm': '0.3318', 'learning_rate': '0.000144', 'entropy': '0.8381', 'num_tokens': '2.515e+07', 'mean_token_accuracy': '0.7926', 'epoch': '0.04113'}
{'loss': '1.227', 'grad_norm': '0.4293', 'learning_rate': '0.000144', 'entropy': '1.237', 'num_tokens': '2.515e+07', 'mean_token_accuracy': '0.6903', 'epoch': '0.04113'}
{'loss': '1.026', 'grad_norm': '0.3879', 'learning_rate': '0.000144', 'entropy': '1.027', 'num_tokens': '2.516e+07', 'mean_token_accuracy': '0.7634', 'epoch': '0.04114'}
{'loss': '0.7818', 'grad_norm': '0.3513', 'learning_rate': '0.0001439', 'entropy': '0.7746', 'num_tokens': '2.516e+07', 'mean_token_accuracy': '0.8075', 'epoch': '0.04115'}
{'loss': '0.8532', 'grad_norm': '0.4026', 'learning_rate': '0.0001439', 'entropy': '0.9345', 'num_tokens': '2.516e+07', 'mean_token_accuracy': '0.7788', 'epoch': '0.04116'}
{'loss': '0.8554', 'grad_norm': '0.4001', 'learning_rate': '0.0001439', 'entropy': '0.8528', 'num_tokens': '2.517e+07', 'mean_token_accuracy': '0.7813', 'epoch': '0.04117'}
{'loss': '1.12', 'grad_norm': '0.3919', 'learning_rate': '0.0001439', 'entropy': '1.24', 'num_tokens': '2.517e+07', 'mean_token_accuracy': '0.7126', 'epoch': '0.04118'}
{'loss': '0.9522', 'grad_norm': '0.3499', 'learning_rate': '0.0001438', 'entropy': '0.9423', 'num_tokens': '2.518e+07', 'mean_token_accuracy': '0.7671', 'epoch': '0.04119'}
{'loss': '0.6507', 'grad_norm': '0.3547', 'learning_rate': '0.0001438', 'entropy': '0.7469', 'num_tokens': '2.519e+07', 'mean_token_accuracy': '0.8019', 'epoch': '0.0412'}
{'loss': '1.143', 'grad_norm': '0.4457', 'learning_rate': '0.0001438', 'entropy': '1.242', 'num_tokens': '2.519e+07', 'mean_token_accuracy': '0.7287', 'epoch': '0.0412'}

 39%|███▉      | 4657/12000 [2:46:32<6:07:20,  3.00s/it]
 39%|███▉      | 4658/12000 [2:46:33<5:15:16,  2.58s/it]
                                                        

 39%|███▉      | 4658/12000 [2:46:33<5:15:16,  2.58s/it]
 39%|███▉      | 4659/12000 [2:46:35<4:33:30,  2.24s/it]
                                                        

 39%|███▉      | 4659/12000 [2:46:35<4:33:30,  2.24s/it]
 39%|███▉      | 4660/12000 [2:46:36<4:08:36,  2.03s/it]
                                                        

 39%|███▉      | 4660/12000 [2:46:36<4:08:36,  2.03s/it]
 39%|███▉      | 4661/12000 [2:46:39<4:20:45,  2.13s/it]
                                                        

 39%|███▉      | 4661/12000 [2:46:39<4:20:45,  2.13s/it]
 39%|███▉      | 4662/12000 [2:46:40<4:02:43,  1.98s/it]
                                                        

 39%|███▉      | 4662/12000 [2:46:40<4:02:43,  1.98s/it]
 39%|███▉      | 4663/12000 [2:46:43<4:12:36,  2.07s/it]
                                                        

 39%|███▉      | 4663/12000 [2:46:43<4:12:36,  2.07s/it]
 39%|███▉      | 4664/12000 [2:46:44<3:47:01,  1.86s/it]
                                                        

 39%|███▉      | 4664/12000 [2:46:44<3:47:01,  1.86s/it]
 39%|███▉      | 4665/12000 [2:46:46<4:08:57,  2.04s/it]
                                                        

 39%|███▉      | 4665/12000 [2:46:46<4:08:57,  2.04s/it]
 39%|███▉      | 4666/12000 [2:46:48<3:53:44,  1.91s/it]
                                                        

 39%|███▉      | 4666/12000 [2:46:48<3:53:44,  1.91s/it]
 39%|███▉      | 4667/12000 [2:46:51<4:37:07,  2.27s/it]
                                                        

 39%|███▉      | 4667/12000 [2:46:51<4:37:07,  2.27s/it]
 39%|███▉      | 4668/12000 [2:46:55<5:21:04,  2.63s/it]
                                                        

 39%|███▉      | 4668/12000 [2:46:55<5:21:04,  2.63s/it]
 39%|███▉      | 4669/12000 [2:46:57<5:21:19,  2.63s/it]
                                                        

 39%|███▉      | 4669/12000 [2:46:57<5:21:19,  2.63s/it]
 39%|███▉      | 4670/12000 [2:46:59<4:53:07,  2.40s/it]
                                                        

 39%|███▉      | 4670/12000 [2:46:59<4:53:07,  2.40s/it]
 39%|███▉      | 4671/12000 [2:47:02<5:10:41,  2.54s/it]
                                                        

 39%|███▉      | 4671/12000 [2:47:02<5:10:41,  2.54s/it]
 39%|███▉      | 4672/12000 [2:47:05<5:39:35,  2.78s/it]
                                                        

 39%|███▉      | 4672/12000 [2:47:05<5:39:35,  2.78s/it]
 39%|███▉      | 4673/12000 [2:47:07<4:49:01,  2.37s/it]
                                                        

 39%|███▉      | 4673/12000 [2:47:07<4:49:01,  2.37s/it]
 39%|███▉      | 4674/12000 [2:47:09<4:39:59,  2.29s/it]
                                                        

 39%|███▉      | 4674/12000 [2:47:09<4:39:59,  2.29s/it]
 39%|███▉      | 4675/12000 [2:47:11<4:25:42,  2.18s/it]
                                                        

 39%|███▉      | 4675/12000 [2:47:11<4:25:42,  2.18s/it]
 39%|███▉      | 4676/12000 [2:47:14<5:00:55,  2.47s/it]
                                                        

 39%|███▉      | 4676/12000 [2:47:14<5:00:55,  2.47s/it]
 39%|███▉      | 4677/12000 [2:47:16<4:40:51,  2.30s/it]
                                                        

 39%|███▉      | 4677/12000 [2:47:16<4:40:51,  2.30s/it]
 39%|███▉      | 4678/12000 [2:47:17<4:10:47,  2.06s/it]
                                                        

 39%|███▉      | 4678/12000 [2:47:17<4:10:47,  2.06s/it]
 39%|███▉      | 4679/12000 [2:47:21<5:01:26,  2.47s/it]
                                                        

 39%|███▉      | 4679/12000 [2:47:21<5:01:26,  2.47s/it]
 39%|███▉      | 4680/12000 [2:47:22<4:26:52,  2.19s/it]
                                                        

 39%|███▉      | 4680/12000 [2:47:22<4:26:52,  2.19s/it]
 39%|███▉      | 4681/12000 [2:47:25<4:38:32,  2.28s/it]
                                                        

 39%|███▉      | 4681/12000 [2:47:25<4:38:32,  2.28s/it]
 39%|███▉      | 4682/12000 [2:47:28<5:24:38,  2.66s/it]
                                                        

 39%|███▉      | 4682/12000 [2:47:28<5:24:38,  2.66s/it]
 39%|███▉      | 4683/12000 [2:47:30<4:48:23,  2.36s/it]
                                                        

 39%|███▉      | 4683/12000 [2:47:30<4:48:23,  2.36s/it]
 39%|███▉      | 4684/12000 [2:47:32<4:28:11,  2.20s/it]
                                                        

 39%|███▉      | 4684/12000 [2:47:32<4:28:11,  2.20s/it]
 39%|███▉      | 4685/12000 [2:47:33<4:00:44,  1.97s/it]
                                                        

 39%|███▉      | 4685/12000 [2:47:33<4:00:44,  1.97s/it]
 39%|███▉      | 4686/12000 [2:47:37<4:59:00,  2.45s/it]
                                                        

 39%|███▉      | 4686/12000 [2:47:37<4:59:00,  2.45s/it]
 39%|███▉      | 4687/12000 [2:47:40<5:08:48,  2.53s/it]
                                                        

 39%|███▉      | 4687/12000 [2:47:40<5:08:48,  2.53s/it]
 39%|███▉      | 4688/12000 [2:47:41<4:30:28,  2.22s/it]
                                                        

 39%|███▉      | 4688/12000 [2:47:41<4:30:28,  2.22s/it]
 39%|███▉      | 4689/12000 [2:47:43<4:08:37,  2.04s/it]
                                                        

 39%|███▉      | 4689/12000 [2:47:43<4:08:37,  2.04s/it]
 39%|███▉      | 4690/12000 [2:47:44<3:55:37,  1.93s/it]
                                                        

 39%|███▉      | 4690/12000 [2:47:44<3:55:37,  1.93s/it]
 39%|███▉      | 4691/12000 [2:47:46<4:04:15,  2.01s/it]
                                                        

 39%|███▉      | 4691/12000 [2:47:46<4:04:15,  2.01s/it]
 39%|███▉      | 4692/12000 [2:47:48<3:50:09,  1.89s/it]
                                                        

 39%|███▉      | 4692/12000 [2:47:48<3:50:09,  1.89s/it]
 39%|███▉      | 4693/12000 [2:47:50<3:40:56,  1.81s/it]
                                                        

 39%|███▉      | 4693/12000 [2:47:50<3:40:56,  1.81s/it]
 39%|███▉      | 4694/12000 [2:47:51<3:31:32,  1.74s/it]
                                                        

 39%|███▉      | 4694/12000 [2:47:51<3:31:32,  1.74s/it]
 39%|███▉      | 4695/12000 [2:47:54<4:14:18,  2.09s/it]
                                                        

 39%|███▉      | 4695/12000 [2:47:54<4:14:18,  2.09s/it]
 39%|███▉      | 4696/12000 [2:47:57<4:26:44,  2.19s/it]
                                                        

 39%|███▉      | 4696/12000 [2:47:57<4:26:44,  2.19s/it]
 39%|███▉      | 4697/12000 [2:47:59<4:35:44,  2.27s/it]
                                                        

 39%|███▉      | 4697/12000 [2:47:59<4:35:44,  2.27s/it]
 39%|███▉      | 4698/12000 [2:48:01<4:05:00,  2.01s/it]
                                                        

 39%|███▉      | 4698/12000 [2:48:01<4:05:00,  2.01s/it]
 39%|███▉      | 4699/12000 [2:48:02<3:50:30,  1.89s/it]
                                                        

 39%|███▉      | 4699/12000 [2:48:02<3:50:30,  1.89s/it]
 39%|███▉      | 4700/12000 [2:48:04<3:34:39,  1.76s/it]
                                                        

 39%|███▉      | 4700/12000 [2:48:04<3:34:39,  1.76s/it]
 39%|███▉      | 4701/12000 [2:48:08<5:16:00,  2.60s/it]
                                                        

 39%|███▉      | 4701/12000 [2:48:08<5:16:00,  2.60s/it]
 39%|███▉      | 4702/12000 [2:48:09<4:30:19,  2.22s/it]
                                                        

 39%|███▉      | 4702/12000 [2:48:09<4:30:19,  2.22s/it]
 39%|███▉      | 4703/12000 [2:48:11<3:57:35,  1.95s/it]
                                                        

 39%|███▉      | 4703/12000 [2:48:11<3:57:35,  1.95s/it]
 39%|███▉      | 4704/12000 [2:48:13<4:02:32,  1.99s/it]
                                                        
{'loss': '0.8046', 'grad_norm': '0.4033', 'learning_rate': '0.0001438', 'entropy': '0.8289', 'num_tokens': '2.52e+07', 'mean_token_accuracy': '0.8052', 'epoch': '0.04121'}
{'loss': '0.8306', 'grad_norm': '0.3592', 'learning_rate': '0.0001437', 'entropy': '0.749', 'num_tokens': '2.52e+07', 'mean_token_accuracy': '0.8053', 'epoch': '0.04122'}
{'loss': '1.019', 'grad_norm': '0.415', 'learning_rate': '0.0001437', 'entropy': '0.9973', 'num_tokens': '2.52e+07', 'mean_token_accuracy': '0.7546', 'epoch': '0.04123'}
{'loss': '1.049', 'grad_norm': '0.4061', 'learning_rate': '0.0001437', 'entropy': '0.9758', 'num_tokens': '2.521e+07', 'mean_token_accuracy': '0.765', 'epoch': '0.04124'}
{'loss': '0.9829', 'grad_norm': '0.498', 'learning_rate': '0.0001437', 'entropy': '0.8988', 'num_tokens': '2.522e+07', 'mean_token_accuracy': '0.7712', 'epoch': '0.04125'}
{'loss': '0.9654', 'grad_norm': '0.4263', 'learning_rate': '0.0001436', 'entropy': '1.024', 'num_tokens': '2.522e+07', 'mean_token_accuracy': '0.7455', 'epoch': '0.04126'}
{'loss': '1', 'grad_norm': '0.4412', 'learning_rate': '0.0001436', 'entropy': '0.9795', 'num_tokens': '2.523e+07', 'mean_token_accuracy': '0.7455', 'epoch': '0.04127'}
{'loss': '0.7201', 'grad_norm': '0.334', 'learning_rate': '0.0001436', 'entropy': '0.7275', 'num_tokens': '2.523e+07', 'mean_token_accuracy': '0.8151', 'epoch': '0.04128'}
{'loss': '0.8693', 'grad_norm': '0.4752', 'learning_rate': '0.0001436', 'entropy': '0.9375', 'num_tokens': '2.524e+07', 'mean_token_accuracy': '0.7648', 'epoch': '0.04128'}
{'loss': '0.5069', 'grad_norm': '0.3178', 'learning_rate': '0.0001435', 'entropy': '0.5043', 'num_tokens': '2.524e+07', 'mean_token_accuracy': '0.8709', 'epoch': '0.04129'}
{'loss': '1.015', 'grad_norm': '0.4107', 'learning_rate': '0.0001435', 'entropy': '1.041', 'num_tokens': '2.525e+07', 'mean_token_accuracy': '0.7443', 'epoch': '0.0413'}
{'loss': '0.7304', 'grad_norm': '0.2902', 'learning_rate': '0.0001435', 'entropy': '0.7692', 'num_tokens': '2.526e+07', 'mean_token_accuracy': '0.8082', 'epoch': '0.04131'}
{'loss': '0.9946', 'grad_norm': '0.4437', 'learning_rate': '0.0001435', 'entropy': '0.9964', 'num_tokens': '2.526e+07', 'mean_token_accuracy': '0.747', 'epoch': '0.04132'}
{'loss': '0.911', 'grad_norm': '0.3855', 'learning_rate': '0.0001434', 'entropy': '0.9442', 'num_tokens': '2.527e+07', 'mean_token_accuracy': '0.7628', 'epoch': '0.04133'}
{'loss': '1.035', 'grad_norm': '0.4465', 'learning_rate': '0.0001434', 'entropy': '1.04', 'num_tokens': '2.528e+07', 'mean_token_accuracy': '0.7355', 'epoch': '0.04134'}
{'loss': '0.6201', 'grad_norm': '0.268', 'learning_rate': '0.0001434', 'entropy': '0.6523', 'num_tokens': '2.528e+07', 'mean_token_accuracy': '0.8369', 'epoch': '0.04135'}
{'loss': '0.8933', 'grad_norm': '0.484', 'learning_rate': '0.0001434', 'entropy': '0.9076', 'num_tokens': '2.529e+07', 'mean_token_accuracy': '0.7902', 'epoch': '0.04136'}
{'loss': '0.7385', 'grad_norm': '0.2603', 'learning_rate': '0.0001433', 'entropy': '0.7794', 'num_tokens': '2.53e+07', 'mean_token_accuracy': '0.8003', 'epoch': '0.04136'}
{'loss': '0.8476', 'grad_norm': '0.4597', 'learning_rate': '0.0001433', 'entropy': '0.8167', 'num_tokens': '2.53e+07', 'mean_token_accuracy': '0.793', 'epoch': '0.04137'}
{'loss': '0.7845', 'grad_norm': '0.3109', 'learning_rate': '0.0001433', 'entropy': '0.8136', 'num_tokens': '2.531e+07', 'mean_token_accuracy': '0.7958', 'epoch': '0.04138'}
{'loss': '1.289', 'grad_norm': '0.5253', 'learning_rate': '0.0001433', 'entropy': '1.232', 'num_tokens': '2.531e+07', 'mean_token_accuracy': '0.7195', 'epoch': '0.04139'}
{'loss': '0.7674', 'grad_norm': '0.2512', 'learning_rate': '0.0001432', 'entropy': '0.7618', 'num_tokens': '2.532e+07', 'mean_token_accuracy': '0.8017', 'epoch': '0.0414'}
{'loss': '0.6978', 'grad_norm': '0.3325', 'learning_rate': '0.0001432', 'entropy': '0.7197', 'num_tokens': '2.532e+07', 'mean_token_accuracy': '0.8231', 'epoch': '0.04141'}
{'loss': '1.107', 'grad_norm': '0.414', 'learning_rate': '0.0001432', 'entropy': '0.9672', 'num_tokens': '2.533e+07', 'mean_token_accuracy': '0.7714', 'epoch': '0.04142'}
{'loss': '0.871', 'grad_norm': '0.364', 'learning_rate': '0.0001432', 'entropy': '0.8599', 'num_tokens': '2.533e+07', 'mean_token_accuracy': '0.7867', 'epoch': '0.04143'}
{'loss': '1.193', 'grad_norm': '0.4099', 'learning_rate': '0.0001432', 'entropy': '1.007', 'num_tokens': '2.534e+07', 'mean_token_accuracy': '0.7528', 'epoch': '0.04143'}
{'loss': '0.7565', 'grad_norm': '0.4344', 'learning_rate': '0.0001431', 'entropy': '0.7387', 'num_tokens': '2.534e+07', 'mean_token_accuracy': '0.8119', 'epoch': '0.04144'}
{'loss': '1.283', 'grad_norm': '0.4544', 'learning_rate': '0.0001431', 'entropy': '1.15', 'num_tokens': '2.535e+07', 'mean_token_accuracy': '0.7217', 'epoch': '0.04145'}
{'loss': '0.7147', 'grad_norm': '0.4342', 'learning_rate': '0.0001431', 'entropy': '0.7774', 'num_tokens': '2.535e+07', 'mean_token_accuracy': '0.8066', 'epoch': '0.04146'}
{'loss': '1.028', 'grad_norm': '0.3612', 'learning_rate': '0.0001431', 'entropy': '1.023', 'num_tokens': '2.536e+07', 'mean_token_accuracy': '0.7426', 'epoch': '0.04147'}
{'loss': '1.01', 'grad_norm': '0.5412', 'learning_rate': '0.000143', 'entropy': '1.171', 'num_tokens': '2.537e+07', 'mean_token_accuracy': '0.7292', 'epoch': '0.04148'}
{'loss': '0.6874', 'grad_norm': '0.3506', 'learning_rate': '0.000143', 'entropy': '0.804', 'num_tokens': '2.537e+07', 'mean_token_accuracy': '0.7968', 'epoch': '0.04149'}
{'loss': '0.711', 'grad_norm': '0.4279', 'learning_rate': '0.000143', 'entropy': '0.7126', 'num_tokens': '2.538e+07', 'mean_token_accuracy': '0.8058', 'epoch': '0.0415'}
{'loss': '1.09', 'grad_norm': '0.4628', 'learning_rate': '0.000143', 'entropy': '1.052', 'num_tokens': '2.538e+07', 'mean_token_accuracy': '0.7559', 'epoch': '0.04151'}
{'loss': '0.9026', 'grad_norm': '0.342', 'learning_rate': '0.0001429', 'entropy': '0.8761', 'num_tokens': '2.539e+07', 'mean_token_accuracy': '0.7831', 'epoch': '0.04151'}
{'loss': '0.9869', 'grad_norm': '0.4136', 'learning_rate': '0.0001429', 'entropy': '1.002', 'num_tokens': '2.539e+07', 'mean_token_accuracy': '0.7543', 'epoch': '0.04152'}
{'loss': '0.8312', 'grad_norm': '0.4343', 'learning_rate': '0.0001429', 'entropy': '0.8756', 'num_tokens': '2.54e+07', 'mean_token_accuracy': '0.7829', 'epoch': '0.04153'}
{'loss': '1.056', 'grad_norm': '0.428', 'learning_rate': '0.0001429', 'entropy': '1.123', 'num_tokens': '2.54e+07', 'mean_token_accuracy': '0.7298', 'epoch': '0.04154'}
{'loss': '0.7008', 'grad_norm': '0.2894', 'learning_rate': '0.0001428', 'entropy': '0.7296', 'num_tokens': '2.541e+07', 'mean_token_accuracy': '0.8156', 'epoch': '0.04155'}
{'loss': '0.8814', 'grad_norm': '0.3859', 'learning_rate': '0.0001428', 'entropy': '0.8041', 'num_tokens': '2.542e+07', 'mean_token_accuracy': '0.7994', 'epoch': '0.04156'}
{'loss': '0.7875', 'grad_norm': '0.3923', 'learning_rate': '0.0001428', 'entropy': '0.7832', 'num_tokens': '2.542e+07', 'mean_token_accuracy': '0.8069', 'epoch': '0.04157'}
{'loss': '0.6503', 'grad_norm': '0.3375', 'learning_rate': '0.0001428', 'entropy': '0.6829', 'num_tokens': '2.543e+07', 'mean_token_accuracy': '0.8283', 'epoch': '0.04158'}
{'loss': '0.8427', 'grad_norm': '0.3584', 'learning_rate': '0.0001427', 'entropy': '0.8039', 'num_tokens': '2.544e+07', 'mean_token_accuracy': '0.8062', 'epoch': '0.04159'}
{'loss': '0.9305', 'grad_norm': '0.4653', 'learning_rate': '0.0001427', 'entropy': '0.9361', 'num_tokens': '2.544e+07', 'mean_token_accuracy': '0.7603', 'epoch': '0.04159'}
{'loss': '0.7659', 'grad_norm': '0.4804', 'learning_rate': '0.0001427', 'entropy': '0.7741', 'num_tokens': '2.544e+07', 'mean_token_accuracy': '0.8137', 'epoch': '0.0416'}
{'loss': '0.7321', 'grad_norm': '0.3279', 'learning_rate': '0.0001427', 'entropy': '0.7302', 'num_tokens': '2.545e+07', 'mean_token_accuracy': '0.819', 'epoch': '0.04161'}
{'loss': '0.8827', 'grad_norm': '0.4248', 'learning_rate': '0.0001426', 'entropy': '0.8949', 'num_tokens': '2.545e+07', 'mean_token_accuracy': '0.782', 'epoch': '0.04162'}

 39%|███▉      | 4704/12000 [2:48:13<4:02:32,  1.99s/it]
 39%|███▉      | 4705/12000 [2:48:15<3:54:26,  1.93s/it]
                                                        

 39%|███▉      | 4705/12000 [2:48:15<3:54:26,  1.93s/it]
 39%|███▉      | 4706/12000 [2:48:16<3:38:52,  1.80s/it]
                                                        

 39%|███▉      | 4706/12000 [2:48:16<3:38:52,  1.80s/it]
 39%|███▉      | 4707/12000 [2:48:21<5:18:23,  2.62s/it]
                                                        

 39%|███▉      | 4707/12000 [2:48:21<5:18:23,  2.62s/it]
 39%|███▉      | 4708/12000 [2:48:23<5:02:33,  2.49s/it]
                                                        

 39%|███▉      | 4708/12000 [2:48:23<5:02:33,  2.49s/it]
 39%|███▉      | 4709/12000 [2:48:25<5:03:45,  2.50s/it]
                                                        

 39%|███▉      | 4709/12000 [2:48:25<5:03:45,  2.50s/it]
 39%|███▉      | 4710/12000 [2:48:28<4:55:59,  2.44s/it]
                                                        

 39%|███▉      | 4710/12000 [2:48:28<4:55:59,  2.44s/it]
 39%|███▉      | 4711/12000 [2:48:29<4:17:10,  2.12s/it]
                                                        

 39%|███▉      | 4711/12000 [2:48:29<4:17:10,  2.12s/it]
 39%|███▉      | 4712/12000 [2:48:31<4:03:50,  2.01s/it]
                                                        

 39%|███▉      | 4712/12000 [2:48:31<4:03:50,  2.01s/it]
 39%|███▉      | 4713/12000 [2:48:32<3:45:48,  1.86s/it]
                                                        

 39%|███▉      | 4713/12000 [2:48:32<3:45:48,  1.86s/it]
 39%|███▉      | 4714/12000 [2:48:34<3:30:29,  1.73s/it]
                                                        

 39%|███▉      | 4714/12000 [2:48:34<3:30:29,  1.73s/it]
 39%|███▉      | 4715/12000 [2:48:35<3:26:07,  1.70s/it]
                                                        

 39%|███▉      | 4715/12000 [2:48:35<3:26:07,  1.70s/it]
 39%|███▉      | 4716/12000 [2:48:37<3:19:59,  1.65s/it]
                                                        

 39%|███▉      | 4716/12000 [2:48:37<3:19:59,  1.65s/it]
 39%|███▉      | 4717/12000 [2:48:39<3:19:53,  1.65s/it]
                                                        

 39%|███▉      | 4717/12000 [2:48:39<3:19:53,  1.65s/it]
 39%|███▉      | 4718/12000 [2:48:40<3:26:05,  1.70s/it]
                                                        

 39%|███▉      | 4718/12000 [2:48:40<3:26:05,  1.70s/it]
 39%|███▉      | 4719/12000 [2:48:42<3:22:52,  1.67s/it]
                                                        

 39%|███▉      | 4719/12000 [2:48:42<3:22:52,  1.67s/it]
 39%|███▉      | 4720/12000 [2:48:44<3:30:05,  1.73s/it]
                                                        

 39%|███▉      | 4720/12000 [2:48:44<3:30:05,  1.73s/it]
 39%|███▉      | 4721/12000 [2:48:46<3:59:01,  1.97s/it]
                                                        

 39%|███▉      | 4721/12000 [2:48:46<3:59:01,  1.97s/it]
 39%|███▉      | 4722/12000 [2:48:48<3:38:36,  1.80s/it]
                                                        

 39%|███▉      | 4722/12000 [2:48:48<3:38:36,  1.80s/it]
 39%|███▉      | 4723/12000 [2:48:52<4:48:43,  2.38s/it]
                                                        

 39%|███▉      | 4723/12000 [2:48:52<4:48:43,  2.38s/it]
 39%|███▉      | 4724/12000 [2:48:53<4:09:43,  2.06s/it]
                                                        

 39%|███▉      | 4724/12000 [2:48:53<4:09:43,  2.06s/it]
 39%|███▉      | 4725/12000 [2:48:54<3:51:23,  1.91s/it]
                                                        

 39%|███▉      | 4725/12000 [2:48:54<3:51:23,  1.91s/it]
 39%|███▉      | 4726/12000 [2:48:56<3:38:45,  1.80s/it]
                                                        

 39%|███▉      | 4726/12000 [2:48:56<3:38:45,  1.80s/it]
 39%|███▉      | 4727/12000 [2:48:58<3:56:41,  1.95s/it]
                                                        

 39%|███▉      | 4727/12000 [2:48:58<3:56:41,  1.95s/it]
 39%|███▉      | 4728/12000 [2:49:00<3:59:17,  1.97s/it]
                                                        

 39%|███▉      | 4728/12000 [2:49:00<3:59:17,  1.97s/it]
 39%|███▉      | 4729/12000 [2:49:02<3:46:43,  1.87s/it]
                                                        

 39%|███▉      | 4729/12000 [2:49:02<3:46:43,  1.87s/it]
 39%|███▉      | 4730/12000 [2:49:03<3:23:31,  1.68s/it]
                                                        

 39%|███▉      | 4730/12000 [2:49:03<3:23:31,  1.68s/it]
 39%|███▉      | 4731/12000 [2:49:05<3:19:19,  1.65s/it]
                                                        

 39%|███▉      | 4731/12000 [2:49:05<3:19:19,  1.65s/it]
 39%|███▉      | 4732/12000 [2:49:07<3:45:41,  1.86s/it]
                                                        

 39%|███▉      | 4732/12000 [2:49:07<3:45:41,  1.86s/it]
 39%|███▉      | 4733/12000 [2:49:09<4:00:55,  1.99s/it]
                                                        

 39%|███▉      | 4733/12000 [2:49:09<4:00:55,  1.99s/it]
 39%|███▉      | 4734/12000 [2:49:11<3:54:32,  1.94s/it]
                                                        

 39%|███▉      | 4734/12000 [2:49:11<3:54:32,  1.94s/it]
 39%|███▉      | 4735/12000 [2:49:13<4:04:37,  2.02s/it]
                                                        

 39%|███▉      | 4735/12000 [2:49:13<4:04:37,  2.02s/it]
 39%|███▉      | 4736/12000 [2:49:16<4:20:38,  2.15s/it]
                                                        

 39%|███▉      | 4736/12000 [2:49:16<4:20:38,  2.15s/it]
 39%|███▉      | 4737/12000 [2:49:17<3:54:18,  1.94s/it]
                                                        

 39%|███▉      | 4737/12000 [2:49:17<3:54:18,  1.94s/it]
 39%|███▉      | 4738/12000 [2:49:20<4:09:03,  2.06s/it]
                                                        

 39%|███▉      | 4738/12000 [2:49:20<4:09:03,  2.06s/it]
 39%|███▉      | 4739/12000 [2:49:21<3:46:26,  1.87s/it]
                                                        

 39%|███▉      | 4739/12000 [2:49:21<3:46:26,  1.87s/it]
 40%|███▉      | 4740/12000 [2:49:23<4:05:11,  2.03s/it]
                                                        

 40%|███▉      | 4740/12000 [2:49:23<4:05:11,  2.03s/it]
 40%|███▉      | 4741/12000 [2:49:25<3:49:14,  1.89s/it]
                                                        

 40%|███▉      | 4741/12000 [2:49:25<3:49:14,  1.89s/it]
 40%|███▉      | 4742/12000 [2:49:26<3:31:07,  1.75s/it]
                                                        

 40%|███▉      | 4742/12000 [2:49:26<3:31:07,  1.75s/it]
 40%|███▉      | 4743/12000 [2:49:29<3:54:34,  1.94s/it]
                                                        

 40%|███▉      | 4743/12000 [2:49:29<3:54:34,  1.94s/it]
 40%|███▉      | 4744/12000 [2:49:31<3:48:38,  1.89s/it]
                                                        

 40%|███▉      | 4744/12000 [2:49:31<3:48:38,  1.89s/it]
 40%|███▉      | 4745/12000 [2:49:34<4:41:38,  2.33s/it]
                                                        

 40%|███▉      | 4745/12000 [2:49:34<4:41:38,  2.33s/it]
 40%|███▉      | 4746/12000 [2:49:35<4:10:42,  2.07s/it]
                                                        

 40%|███▉      | 4746/12000 [2:49:35<4:10:42,  2.07s/it]
 40%|███▉      | 4747/12000 [2:49:37<4:08:39,  2.06s/it]
                                                        

 40%|███▉      | 4747/12000 [2:49:37<4:08:39,  2.06s/it]
 40%|███▉      | 4748/12000 [2:49:39<3:48:07,  1.89s/it]
                                                        

 40%|███▉      | 4748/12000 [2:49:39<3:48:07,  1.89s/it]
 40%|███▉      | 4749/12000 [2:49:41<3:46:52,  1.88s/it]
                                                        

 40%|███▉      | 4749/12000 [2:49:41<3:46:52,  1.88s/it]
 40%|███▉      | 4750/12000 [2:49:42<3:40:16,  1.82s/it]
                                                        

 40%|███▉      | 4750/12000 [2:49:42<3:40:16,  1.82s/it]
 40%|███▉      | 4751/12000 [2:49:45<4:11:41,  2.08s/it]
                                                        
{'loss': '1.033', 'grad_norm': '0.5195', 'learning_rate': '0.0001426', 'entropy': '1.185', 'num_tokens': '2.546e+07', 'mean_token_accuracy': '0.729', 'epoch': '0.04163'}
{'loss': '0.9796', 'grad_norm': '0.5178', 'learning_rate': '0.0001426', 'entropy': '0.8903', 'num_tokens': '2.546e+07', 'mean_token_accuracy': '0.7841', 'epoch': '0.04164'}
{'loss': '0.8909', 'grad_norm': '0.3232', 'learning_rate': '0.0001426', 'entropy': '0.9345', 'num_tokens': '2.547e+07', 'mean_token_accuracy': '0.7763', 'epoch': '0.04165'}
{'loss': '1.104', 'grad_norm': '0.3863', 'learning_rate': '0.0001425', 'entropy': '1.057', 'num_tokens': '2.547e+07', 'mean_token_accuracy': '0.7456', 'epoch': '0.04166'}
{'loss': '0.6951', 'grad_norm': '0.315', 'learning_rate': '0.0001425', 'entropy': '0.7034', 'num_tokens': '2.548e+07', 'mean_token_accuracy': '0.826', 'epoch': '0.04167'}
{'loss': '1.102', 'grad_norm': '0.5949', 'learning_rate': '0.0001425', 'entropy': '0.9757', 'num_tokens': '2.548e+07', 'mean_token_accuracy': '0.7586', 'epoch': '0.04167'}
{'loss': '0.5348', 'grad_norm': '0.257', 'learning_rate': '0.0001425', 'entropy': '0.6918', 'num_tokens': '2.549e+07', 'mean_token_accuracy': '0.8252', 'epoch': '0.04168'}
{'loss': '0.9135', 'grad_norm': '0.4243', 'learning_rate': '0.0001424', 'entropy': '0.9004', 'num_tokens': '2.549e+07', 'mean_token_accuracy': '0.7697', 'epoch': '0.04169'}
{'loss': '0.6743', 'grad_norm': '0.3509', 'learning_rate': '0.0001424', 'entropy': '0.6754', 'num_tokens': '2.55e+07', 'mean_token_accuracy': '0.8228', 'epoch': '0.0417'}
{'loss': '0.5234', 'grad_norm': '0.3602', 'learning_rate': '0.0001424', 'entropy': '0.5924', 'num_tokens': '2.55e+07', 'mean_token_accuracy': '0.8469', 'epoch': '0.04171'}
{'loss': '0.7786', 'grad_norm': '0.3558', 'learning_rate': '0.0001424', 'entropy': '0.7199', 'num_tokens': '2.551e+07', 'mean_token_accuracy': '0.8194', 'epoch': '0.04172'}
{'loss': '0.8047', 'grad_norm': '0.3301', 'learning_rate': '0.0001423', 'entropy': '0.8644', 'num_tokens': '2.551e+07', 'mean_token_accuracy': '0.7772', 'epoch': '0.04173'}
{'loss': '0.7622', 'grad_norm': '0.3738', 'learning_rate': '0.0001423', 'entropy': '0.7636', 'num_tokens': '2.552e+07', 'mean_token_accuracy': '0.8098', 'epoch': '0.04174'}
{'loss': '1.11', 'grad_norm': '0.5067', 'learning_rate': '0.0001423', 'entropy': '1.092', 'num_tokens': '2.552e+07', 'mean_token_accuracy': '0.7217', 'epoch': '0.04174'}
{'loss': '0.7345', 'grad_norm': '0.4404', 'learning_rate': '0.0001423', 'entropy': '0.7668', 'num_tokens': '2.553e+07', 'mean_token_accuracy': '0.8035', 'epoch': '0.04175'}
{'loss': '1.136', 'grad_norm': '0.5172', 'learning_rate': '0.0001422', 'entropy': '1.072', 'num_tokens': '2.553e+07', 'mean_token_accuracy': '0.7265', 'epoch': '0.04176'}
{'loss': '1.034', 'grad_norm': '0.4591', 'learning_rate': '0.0001422', 'entropy': '1.1', 'num_tokens': '2.554e+07', 'mean_token_accuracy': '0.7443', 'epoch': '0.04177'}
{'loss': '1.123', 'grad_norm': '0.467', 'learning_rate': '0.0001422', 'entropy': '1.08', 'num_tokens': '2.554e+07', 'mean_token_accuracy': '0.7409', 'epoch': '0.04178'}
{'loss': '0.7178', 'grad_norm': '0.3529', 'learning_rate': '0.0001422', 'entropy': '0.7774', 'num_tokens': '2.555e+07', 'mean_token_accuracy': '0.8095', 'epoch': '0.04179'}
{'loss': '0.9382', 'grad_norm': '0.3588', 'learning_rate': '0.0001421', 'entropy': '0.9451', 'num_tokens': '2.555e+07', 'mean_token_accuracy': '0.7692', 'epoch': '0.0418'}
{'loss': '0.7923', 'grad_norm': '0.3032', 'learning_rate': '0.0001421', 'entropy': '0.7906', 'num_tokens': '2.556e+07', 'mean_token_accuracy': '0.7898', 'epoch': '0.04181'}
{'loss': '0.7527', 'grad_norm': '0.4701', 'learning_rate': '0.0001421', 'entropy': '0.8041', 'num_tokens': '2.556e+07', 'mean_token_accuracy': '0.7886', 'epoch': '0.04182'}
{'loss': '0.5442', 'grad_norm': '0.3324', 'learning_rate': '0.0001421', 'entropy': '0.5873', 'num_tokens': '2.557e+07', 'mean_token_accuracy': '0.8383', 'epoch': '0.04182'}
{'loss': '0.9357', 'grad_norm': '0.3917', 'learning_rate': '0.000142', 'entropy': '0.9273', 'num_tokens': '2.557e+07', 'mean_token_accuracy': '0.7587', 'epoch': '0.04183'}
{'loss': '0.7124', 'grad_norm': '0.3542', 'learning_rate': '0.000142', 'entropy': '0.6511', 'num_tokens': '2.558e+07', 'mean_token_accuracy': '0.8317', 'epoch': '0.04184'}
{'loss': '0.8567', 'grad_norm': '0.4686', 'learning_rate': '0.000142', 'entropy': '0.8445', 'num_tokens': '2.558e+07', 'mean_token_accuracy': '0.786', 'epoch': '0.04185'}
{'loss': '1.082', 'grad_norm': '0.5068', 'learning_rate': '0.000142', 'entropy': '1.1', 'num_tokens': '2.559e+07', 'mean_token_accuracy': '0.736', 'epoch': '0.04186'}
{'loss': '1.062', 'grad_norm': '0.4726', 'learning_rate': '0.0001419', 'entropy': '1.108', 'num_tokens': '2.559e+07', 'mean_token_accuracy': '0.7231', 'epoch': '0.04187'}
{'loss': '1.426', 'grad_norm': '0.4795', 'learning_rate': '0.0001419', 'entropy': '1.251', 'num_tokens': '2.56e+07', 'mean_token_accuracy': '0.6988', 'epoch': '0.04188'}
{'loss': '0.9197', 'grad_norm': '0.4295', 'learning_rate': '0.0001419', 'entropy': '0.8177', 'num_tokens': '2.56e+07', 'mean_token_accuracy': '0.7893', 'epoch': '0.04189'}
{'loss': '0.6213', 'grad_norm': '0.2745', 'learning_rate': '0.0001419', 'entropy': '0.6304', 'num_tokens': '2.561e+07', 'mean_token_accuracy': '0.8453', 'epoch': '0.0419'}
{'loss': '1.191', 'grad_norm': '0.3962', 'learning_rate': '0.0001418', 'entropy': '0.9596', 'num_tokens': '2.562e+07', 'mean_token_accuracy': '0.7799', 'epoch': '0.0419'}
{'loss': '0.8364', 'grad_norm': '0.472', 'learning_rate': '0.0001418', 'entropy': '0.7802', 'num_tokens': '2.562e+07', 'mean_token_accuracy': '0.8069', 'epoch': '0.04191'}
{'loss': '0.7837', 'grad_norm': '0.4058', 'learning_rate': '0.0001418', 'entropy': '0.8122', 'num_tokens': '2.563e+07', 'mean_token_accuracy': '0.802', 'epoch': '0.04192'}
{'loss': '1.397', 'grad_norm': '0.6019', 'learning_rate': '0.0001418', 'entropy': '1.286', 'num_tokens': '2.563e+07', 'mean_token_accuracy': '0.6981', 'epoch': '0.04193'}
{'loss': '0.6018', 'grad_norm': '0.3806', 'learning_rate': '0.0001417', 'entropy': '0.609', 'num_tokens': '2.564e+07', 'mean_token_accuracy': '0.8457', 'epoch': '0.04194'}
{'loss': '0.7648', 'grad_norm': '0.3157', 'learning_rate': '0.0001417', 'entropy': '0.7244', 'num_tokens': '2.564e+07', 'mean_token_accuracy': '0.8086', 'epoch': '0.04195'}
{'loss': '1.092', 'grad_norm': '0.4601', 'learning_rate': '0.0001417', 'entropy': '1.028', 'num_tokens': '2.565e+07', 'mean_token_accuracy': '0.7581', 'epoch': '0.04196'}
{'loss': '0.8667', 'grad_norm': '0.3633', 'learning_rate': '0.0001417', 'entropy': '0.8802', 'num_tokens': '2.565e+07', 'mean_token_accuracy': '0.782', 'epoch': '0.04197'}
{'loss': '0.6902', 'grad_norm': '0.3031', 'learning_rate': '0.0001416', 'entropy': '0.7713', 'num_tokens': '2.566e+07', 'mean_token_accuracy': '0.8197', 'epoch': '0.04197'}
{'loss': '1.083', 'grad_norm': '0.4549', 'learning_rate': '0.0001416', 'entropy': '1.089', 'num_tokens': '2.566e+07', 'mean_token_accuracy': '0.7336', 'epoch': '0.04198'}
{'loss': '0.6172', 'grad_norm': '0.3059', 'learning_rate': '0.0001416', 'entropy': '0.6254', 'num_tokens': '2.567e+07', 'mean_token_accuracy': '0.8353', 'epoch': '0.04199'}
{'loss': '0.7944', 'grad_norm': '0.4033', 'learning_rate': '0.0001416', 'entropy': '0.833', 'num_tokens': '2.567e+07', 'mean_token_accuracy': '0.7793', 'epoch': '0.042'}
{'loss': '0.9074', 'grad_norm': '0.4546', 'learning_rate': '0.0001415', 'entropy': '0.9468', 'num_tokens': '2.568e+07', 'mean_token_accuracy': '0.7688', 'epoch': '0.04201'}
{'loss': '0.6303', 'grad_norm': '0.3832', 'learning_rate': '0.0001415', 'entropy': '0.6967', 'num_tokens': '2.568e+07', 'mean_token_accuracy': '0.8269', 'epoch': '0.04202'}
{'loss': '0.763', 'grad_norm': '0.4146', 'learning_rate': '0.0001415', 'entropy': '0.7652', 'num_tokens': '2.569e+07', 'mean_token_accuracy': '0.8053', 'epoch': '0.04203'}
{'loss': '0.807', 'grad_norm': '0.3993', 'learning_rate': '0.0001415', 'entropy': '0.8683', 'num_tokens': '2.569e+07', 'mean_token_accuracy': '0.7903', 'epoch': '0.04204'}

 40%|███▉      | 4751/12000 [2:49:45<4:11:41,  2.08s/it]
 40%|███▉      | 4752/12000 [2:49:47<3:49:10,  1.90s/it]
                                                        

 40%|███▉      | 4752/12000 [2:49:47<3:49:10,  1.90s/it]
 40%|███▉      | 4753/12000 [2:49:48<3:36:32,  1.79s/it]
                                                        

 40%|███▉      | 4753/12000 [2:49:48<3:36:32,  1.79s/it]
 40%|███▉      | 4754/12000 [2:49:50<3:22:26,  1.68s/it]
                                                        

 40%|███▉      | 4754/12000 [2:49:50<3:22:26,  1.68s/it]
 40%|███▉      | 4755/12000 [2:49:52<3:50:53,  1.91s/it]
                                                        

 40%|███▉      | 4755/12000 [2:49:52<3:50:53,  1.91s/it]
 40%|███▉      | 4756/12000 [2:49:55<4:12:47,  2.09s/it]
                                                        

 40%|███▉      | 4756/12000 [2:49:55<4:12:47,  2.09s/it]
 40%|███▉      | 4757/12000 [2:49:56<3:58:13,  1.97s/it]
                                                        

 40%|███▉      | 4757/12000 [2:49:56<3:58:13,  1.97s/it]
 40%|███▉      | 4758/12000 [2:49:58<3:40:25,  1.83s/it]
                                                        

 40%|███▉      | 4758/12000 [2:49:58<3:40:25,  1.83s/it]
 40%|███▉      | 4759/12000 [2:49:59<3:28:35,  1.73s/it]
                                                        

 40%|███▉      | 4759/12000 [2:49:59<3:28:35,  1.73s/it]
 40%|███▉      | 4760/12000 [2:50:01<3:29:45,  1.74s/it]
                                                        

 40%|███▉      | 4760/12000 [2:50:01<3:29:45,  1.74s/it]
 40%|███▉      | 4761/12000 [2:50:03<3:49:08,  1.90s/it]
                                                        

 40%|███▉      | 4761/12000 [2:50:03<3:49:08,  1.90s/it]
 40%|███▉      | 4762/12000 [2:50:08<5:32:27,  2.76s/it]
                                                        

 40%|███▉      | 4762/12000 [2:50:08<5:32:27,  2.76s/it]
 40%|███▉      | 4763/12000 [2:50:13<6:48:43,  3.39s/it]
                                                        

 40%|███▉      | 4763/12000 [2:50:13<6:48:43,  3.39s/it]
 40%|███▉      | 4764/12000 [2:50:18<7:36:40,  3.79s/it]
                                                        

 40%|███▉      | 4764/12000 [2:50:18<7:36:40,  3.79s/it]
 40%|███▉      | 4765/12000 [2:50:19<6:17:10,  3.13s/it]
                                                        

 40%|███▉      | 4765/12000 [2:50:19<6:17:10,  3.13s/it]
 40%|███▉      | 4766/12000 [2:50:23<6:24:19,  3.19s/it]
                                                        

 40%|███▉      | 4766/12000 [2:50:23<6:24:19,  3.19s/it]
 40%|███▉      | 4767/12000 [2:50:25<5:57:18,  2.96s/it]
                                                        

 40%|███▉      | 4767/12000 [2:50:25<5:57:18,  2.96s/it]
 40%|███▉      | 4768/12000 [2:50:27<5:30:36,  2.74s/it]
                                                        

 40%|███▉      | 4768/12000 [2:50:27<5:30:36,  2.74s/it]
 40%|███▉      | 4769/12000 [2:50:29<4:59:58,  2.49s/it]
                                                        

 40%|███▉      | 4769/12000 [2:50:29<4:59:58,  2.49s/it]
 40%|███▉      | 4770/12000 [2:50:31<4:34:56,  2.28s/it]
                                                        

 40%|███▉      | 4770/12000 [2:50:31<4:34:56,  2.28s/it]
 40%|███▉      | 4771/12000 [2:50:32<4:04:17,  2.03s/it]
                                                        

 40%|███▉      | 4771/12000 [2:50:32<4:04:17,  2.03s/it]
 40%|███▉      | 4772/12000 [2:50:34<3:42:33,  1.85s/it]
                                                        

 40%|███▉      | 4772/12000 [2:50:34<3:42:33,  1.85s/it]
 40%|███▉      | 4773/12000 [2:50:36<4:08:16,  2.06s/it]
                                                        

 40%|███▉      | 4773/12000 [2:50:36<4:08:16,  2.06s/it]
 40%|███▉      | 4774/12000 [2:50:38<3:46:29,  1.88s/it]
                                                        

 40%|███▉      | 4774/12000 [2:50:38<3:46:29,  1.88s/it]
 40%|███▉      | 4775/12000 [2:50:40<4:06:11,  2.04s/it]
                                                        

 40%|███▉      | 4775/12000 [2:50:40<4:06:11,  2.04s/it]
 40%|███▉      | 4776/12000 [2:50:43<4:27:49,  2.22s/it]
                                                        

 40%|███▉      | 4776/12000 [2:50:43<4:27:49,  2.22s/it]
 40%|███▉      | 4777/12000 [2:50:47<5:54:17,  2.94s/it]
                                                        

 40%|███▉      | 4777/12000 [2:50:47<5:54:17,  2.94s/it]
 40%|███▉      | 4778/12000 [2:50:50<5:24:50,  2.70s/it]
                                                        

 40%|███▉      | 4778/12000 [2:50:50<5:24:50,  2.70s/it]
 40%|███▉      | 4779/12000 [2:50:51<4:48:35,  2.40s/it]
                                                        

 40%|███▉      | 4779/12000 [2:50:51<4:48:35,  2.40s/it]
 40%|███▉      | 4780/12000 [2:50:53<4:19:46,  2.16s/it]
                                                        

 40%|███▉      | 4780/12000 [2:50:53<4:19:46,  2.16s/it]
 40%|███▉      | 4781/12000 [2:50:55<4:01:12,  2.00s/it]
                                                        

 40%|███▉      | 4781/12000 [2:50:55<4:01:12,  2.00s/it]
 40%|███▉      | 4782/12000 [2:50:56<3:41:38,  1.84s/it]
                                                        

 40%|███▉      | 4782/12000 [2:50:56<3:41:38,  1.84s/it]
 40%|███▉      | 4783/12000 [2:50:58<3:34:17,  1.78s/it]
                                                        

 40%|███▉      | 4783/12000 [2:50:58<3:34:17,  1.78s/it]
 40%|███▉      | 4784/12000 [2:50:59<3:25:39,  1.71s/it]
                                                        

 40%|███▉      | 4784/12000 [2:50:59<3:25:39,  1.71s/it]
 40%|███▉      | 4785/12000 [2:51:01<3:32:47,  1.77s/it]
                                                        

 40%|███▉      | 4785/12000 [2:51:01<3:32:47,  1.77s/it]
 40%|███▉      | 4786/12000 [2:51:04<4:00:27,  2.00s/it]
                                                        

 40%|███▉      | 4786/12000 [2:51:04<4:00:27,  2.00s/it]
 40%|███▉      | 4787/12000 [2:51:05<3:50:57,  1.92s/it]
                                                        

 40%|███▉      | 4787/12000 [2:51:05<3:50:57,  1.92s/it]
 40%|███▉      | 4788/12000 [2:51:12<6:49:35,  3.41s/it]
                                                        

 40%|███▉      | 4788/12000 [2:51:12<6:49:35,  3.41s/it]
 40%|███▉      | 4789/12000 [2:51:15<6:31:38,  3.26s/it]
                                                        

 40%|███▉      | 4789/12000 [2:51:15<6:31:38,  3.26s/it]
 40%|███▉      | 4790/12000 [2:51:18<6:09:59,  3.08s/it]
                                                        

 40%|███▉      | 4790/12000 [2:51:18<6:09:59,  3.08s/it]
 40%|███▉      | 4791/12000 [2:51:20<5:45:11,  2.87s/it]
                                                        

 40%|███▉      | 4791/12000 [2:51:20<5:45:11,  2.87s/it]
 40%|███▉      | 4792/12000 [2:51:23<5:28:49,  2.74s/it]
                                                        

 40%|███▉      | 4792/12000 [2:51:23<5:28:49,  2.74s/it]
 40%|███▉      | 4793/12000 [2:51:27<6:38:41,  3.32s/it]
                                                        

 40%|███▉      | 4793/12000 [2:51:27<6:38:41,  3.32s/it]
 40%|███▉      | 4794/12000 [2:51:29<5:30:39,  2.75s/it]
                                                        

 40%|███▉      | 4794/12000 [2:51:29<5:30:39,  2.75s/it]
 40%|███▉      | 4795/12000 [2:51:32<5:35:43,  2.80s/it]
                                                        

 40%|███▉      | 4795/12000 [2:51:32<5:35:43,  2.80s/it]
 40%|███▉      | 4796/12000 [2:51:34<5:05:33,  2.54s/it]
                                                        

 40%|███▉      | 4796/12000 [2:51:34<5:05:33,  2.54s/it]
 40%|███▉      | 4797/12000 [2:51:35<4:28:14,  2.23s/it]
                                                        

 40%|███▉      | 4797/12000 [2:51:35<4:28:14,  2.23s/it]
 40%|███▉      | 4798/12000 [2:51:37<4:29:06,  2.24s/it]
                                                        
{'loss': '1.075', 'grad_norm': '0.5149', 'learning_rate': '0.0001414', 'entropy': '1.084', 'num_tokens': '2.57e+07', 'mean_token_accuracy': '0.7395', 'epoch': '0.04205'}
{'loss': '0.8453', 'grad_norm': '0.3647', 'learning_rate': '0.0001414', 'entropy': '0.8933', 'num_tokens': '2.57e+07', 'mean_token_accuracy': '0.7763', 'epoch': '0.04205'}
{'loss': '0.6246', 'grad_norm': '0.2969', 'learning_rate': '0.0001414', 'entropy': '0.6477', 'num_tokens': '2.571e+07', 'mean_token_accuracy': '0.8377', 'epoch': '0.04206'}
{'loss': '0.7438', 'grad_norm': '0.3784', 'learning_rate': '0.0001414', 'entropy': '0.6949', 'num_tokens': '2.571e+07', 'mean_token_accuracy': '0.8204', 'epoch': '0.04207'}
{'loss': '1.218', 'grad_norm': '0.4387', 'learning_rate': '0.0001413', 'entropy': '1.216', 'num_tokens': '2.572e+07', 'mean_token_accuracy': '0.7076', 'epoch': '0.04208'}
{'loss': '1.083', 'grad_norm': '0.3544', 'learning_rate': '0.0001413', 'entropy': '1.136', 'num_tokens': '2.572e+07', 'mean_token_accuracy': '0.7253', 'epoch': '0.04209'}
{'loss': '0.7149', 'grad_norm': '0.297', 'learning_rate': '0.0001413', 'entropy': '0.7505', 'num_tokens': '2.573e+07', 'mean_token_accuracy': '0.8049', 'epoch': '0.0421'}
{'loss': '0.6454', 'grad_norm': '0.3503', 'learning_rate': '0.0001413', 'entropy': '0.6745', 'num_tokens': '2.574e+07', 'mean_token_accuracy': '0.8193', 'epoch': '0.04211'}
{'loss': '0.8375', 'grad_norm': '0.4161', 'learning_rate': '0.0001412', 'entropy': '0.8793', 'num_tokens': '2.574e+07', 'mean_token_accuracy': '0.7911', 'epoch': '0.04212'}
{'loss': '0.5673', 'grad_norm': '0.3474', 'learning_rate': '0.0001412', 'entropy': '0.5526', 'num_tokens': '2.574e+07', 'mean_token_accuracy': '0.8545', 'epoch': '0.04213'}
{'loss': '0.9753', 'grad_norm': '0.3491', 'learning_rate': '0.0001412', 'entropy': '0.954', 'num_tokens': '2.575e+07', 'mean_token_accuracy': '0.7555', 'epoch': '0.04213'}
{'loss': '1.261', 'grad_norm': '0.3807', 'learning_rate': '0.0001412', 'entropy': '0.8881', 'num_tokens': '2.576e+07', 'mean_token_accuracy': '0.7855', 'epoch': '0.04214'}
{'loss': '1.372', 'grad_norm': '0.4061', 'learning_rate': '0.0001411', 'entropy': '1.179', 'num_tokens': '2.577e+07', 'mean_token_accuracy': '0.7235', 'epoch': '0.04215'}
{'loss': '1.475', 'grad_norm': '0.4247', 'learning_rate': '0.0001411', 'entropy': '1.327', 'num_tokens': '2.578e+07', 'mean_token_accuracy': '0.7048', 'epoch': '0.04216'}
{'loss': '0.7092', 'grad_norm': '0.3766', 'learning_rate': '0.0001411', 'entropy': '0.6926', 'num_tokens': '2.578e+07', 'mean_token_accuracy': '0.8316', 'epoch': '0.04217'}
{'loss': '0.8221', 'grad_norm': '0.4234', 'learning_rate': '0.0001411', 'entropy': '0.8773', 'num_tokens': '2.579e+07', 'mean_token_accuracy': '0.7814', 'epoch': '0.04218'}
{'loss': '1.034', 'grad_norm': '0.3731', 'learning_rate': '0.000141', 'entropy': '1.025', 'num_tokens': '2.579e+07', 'mean_token_accuracy': '0.7496', 'epoch': '0.04219'}
{'loss': '0.5655', 'grad_norm': '0.3838', 'learning_rate': '0.000141', 'entropy': '0.5856', 'num_tokens': '2.58e+07', 'mean_token_accuracy': '0.8467', 'epoch': '0.0422'}
{'loss': '0.9128', 'grad_norm': '0.4431', 'learning_rate': '0.000141', 'entropy': '0.973', 'num_tokens': '2.581e+07', 'mean_token_accuracy': '0.7589', 'epoch': '0.0422'}
{'loss': '0.7514', 'grad_norm': '0.3608', 'learning_rate': '0.000141', 'entropy': '0.7742', 'num_tokens': '2.581e+07', 'mean_token_accuracy': '0.7965', 'epoch': '0.04221'}
{'loss': '0.769', 'grad_norm': '0.3791', 'learning_rate': '0.0001409', 'entropy': '0.829', 'num_tokens': '2.582e+07', 'mean_token_accuracy': '0.7932', 'epoch': '0.04222'}
{'loss': '0.9281', 'grad_norm': '0.3717', 'learning_rate': '0.0001409', 'entropy': '0.8651', 'num_tokens': '2.582e+07', 'mean_token_accuracy': '0.7801', 'epoch': '0.04223'}
{'loss': '0.9325', 'grad_norm': '0.4163', 'learning_rate': '0.0001409', 'entropy': '0.9863', 'num_tokens': '2.583e+07', 'mean_token_accuracy': '0.7562', 'epoch': '0.04224'}
{'loss': '1.242', 'grad_norm': '0.4183', 'learning_rate': '0.0001408', 'entropy': '1.196', 'num_tokens': '2.583e+07', 'mean_token_accuracy': '0.7079', 'epoch': '0.04225'}
{'loss': '0.7711', 'grad_norm': '0.3211', 'learning_rate': '0.0001408', 'entropy': '0.7117', 'num_tokens': '2.584e+07', 'mean_token_accuracy': '0.8171', 'epoch': '0.04226'}
{'loss': '0.6987', 'grad_norm': '0.3106', 'learning_rate': '0.0001408', 'entropy': '0.6949', 'num_tokens': '2.585e+07', 'mean_token_accuracy': '0.8202', 'epoch': '0.04227'}
{'loss': '0.5172', 'grad_norm': '0.3471', 'learning_rate': '0.0001408', 'entropy': '0.56', 'num_tokens': '2.585e+07', 'mean_token_accuracy': '0.8573', 'epoch': '0.04228'}
{'loss': '1.156', 'grad_norm': '0.3897', 'learning_rate': '0.0001407', 'entropy': '1.098', 'num_tokens': '2.586e+07', 'mean_token_accuracy': '0.7335', 'epoch': '0.04228'}
{'loss': '0.5688', 'grad_norm': '0.3294', 'learning_rate': '0.0001407', 'entropy': '0.5826', 'num_tokens': '2.586e+07', 'mean_token_accuracy': '0.8426', 'epoch': '0.04229'}
{'loss': '0.7558', 'grad_norm': '0.3867', 'learning_rate': '0.0001407', 'entropy': '0.7781', 'num_tokens': '2.587e+07', 'mean_token_accuracy': '0.8023', 'epoch': '0.0423'}
{'loss': '1.43', 'grad_norm': '0.777', 'learning_rate': '0.0001407', 'entropy': '1.439', 'num_tokens': '2.587e+07', 'mean_token_accuracy': '0.6666', 'epoch': '0.04231'}
{'loss': '0.5136', 'grad_norm': '0.2945', 'learning_rate': '0.0001406', 'entropy': '0.5892', 'num_tokens': '2.588e+07', 'mean_token_accuracy': '0.849', 'epoch': '0.04232'}
{'loss': '0.8721', 'grad_norm': '0.4191', 'learning_rate': '0.0001406', 'entropy': '0.8814', 'num_tokens': '2.588e+07', 'mean_token_accuracy': '0.7854', 'epoch': '0.04233'}
{'loss': '0.92', 'grad_norm': '0.3985', 'learning_rate': '0.0001406', 'entropy': '0.9704', 'num_tokens': '2.589e+07', 'mean_token_accuracy': '0.767', 'epoch': '0.04234'}
{'loss': '0.7845', 'grad_norm': '0.344', 'learning_rate': '0.0001406', 'entropy': '0.7421', 'num_tokens': '2.589e+07', 'mean_token_accuracy': '0.8082', 'epoch': '0.04235'}
{'loss': '0.7242', 'grad_norm': '0.2953', 'learning_rate': '0.0001405', 'entropy': '0.7261', 'num_tokens': '2.59e+07', 'mean_token_accuracy': '0.8234', 'epoch': '0.04236'}
{'loss': '1.029', 'grad_norm': '0.4183', 'learning_rate': '0.0001405', 'entropy': '1.062', 'num_tokens': '2.59e+07', 'mean_token_accuracy': '0.7347', 'epoch': '0.04236'}
{'loss': '1.078', 'grad_norm': '0.3718', 'learning_rate': '0.0001405', 'entropy': '1.111', 'num_tokens': '2.591e+07', 'mean_token_accuracy': '0.7386', 'epoch': '0.04237'}
{'loss': '0.9814', 'grad_norm': '0.3772', 'learning_rate': '0.0001405', 'entropy': '0.9929', 'num_tokens': '2.592e+07', 'mean_token_accuracy': '0.7471', 'epoch': '0.04238'}
{'loss': '0.9406', 'grad_norm': '0.3767', 'learning_rate': '0.0001404', 'entropy': '1.053', 'num_tokens': '2.593e+07', 'mean_token_accuracy': '0.7416', 'epoch': '0.04239'}
{'loss': '1.156', 'grad_norm': '0.4698', 'learning_rate': '0.0001404', 'entropy': '1.144', 'num_tokens': '2.593e+07', 'mean_token_accuracy': '0.7314', 'epoch': '0.0424'}
{'loss': '0.8162', 'grad_norm': '0.3616', 'learning_rate': '0.0001404', 'entropy': '0.8012', 'num_tokens': '2.594e+07', 'mean_token_accuracy': '0.7903', 'epoch': '0.04241'}
{'loss': '1.448', 'grad_norm': '0.4215', 'learning_rate': '0.0001404', 'entropy': '1.061', 'num_tokens': '2.595e+07', 'mean_token_accuracy': '0.7573', 'epoch': '0.04242'}
{'loss': '0.7231', 'grad_norm': '0.4306', 'learning_rate': '0.0001403', 'entropy': '0.7284', 'num_tokens': '2.595e+07', 'mean_token_accuracy': '0.821', 'epoch': '0.04243'}
{'loss': '1.142', 'grad_norm': '0.4068', 'learning_rate': '0.0001403', 'entropy': '1.188', 'num_tokens': '2.596e+07', 'mean_token_accuracy': '0.7074', 'epoch': '0.04243'}
{'loss': '0.8778', 'grad_norm': '0.3991', 'learning_rate': '0.0001403', 'entropy': '0.8896', 'num_tokens': '2.596e+07', 'mean_token_accuracy': '0.7771', 'epoch': '0.04244'}
{'loss': '1.181', 'grad_norm': '0.5123', 'learning_rate': '0.0001403', 'entropy': '1.149', 'num_tokens': '2.597e+07', 'mean_token_accuracy': '0.7227', 'epoch': '0.04245'}

 40%|███▉      | 4798/12000 [2:51:37<4:29:06,  2.24s/it]
 40%|███▉      | 4799/12000 [2:51:39<4:03:24,  2.03s/it]
                                                        

 40%|███▉      | 4799/12000 [2:51:39<4:03:24,  2.03s/it]
 40%|████      | 4800/12000 [2:51:41<3:52:08,  1.93s/it]
                                                        

 40%|████      | 4800/12000 [2:51:41<3:52:08,  1.93s/it]
 40%|████      | 4801/12000 [2:51:43<4:25:00,  2.21s/it]
                                                        

 40%|████      | 4801/12000 [2:51:43<4:25:00,  2.21s/it]
 40%|████      | 4802/12000 [2:51:45<3:56:02,  1.97s/it]
                                                        

 40%|████      | 4802/12000 [2:51:45<3:56:02,  1.97s/it]
 40%|████      | 4803/12000 [2:51:47<4:01:24,  2.01s/it]
                                                        

 40%|████      | 4803/12000 [2:51:47<4:01:24,  2.01s/it]
 40%|████      | 4804/12000 [2:51:48<3:40:04,  1.83s/it]
                                                        

 40%|████      | 4804/12000 [2:51:48<3:40:04,  1.83s/it]
 40%|████      | 4805/12000 [2:51:51<4:01:18,  2.01s/it]
                                                        

 40%|████      | 4805/12000 [2:51:51<4:01:18,  2.01s/it]
 40%|████      | 4806/12000 [2:51:52<3:48:18,  1.90s/it]
                                                        

 40%|████      | 4806/12000 [2:51:52<3:48:18,  1.90s/it]
 40%|████      | 4807/12000 [2:51:55<4:16:04,  2.14s/it]
                                                        

 40%|████      | 4807/12000 [2:51:55<4:16:04,  2.14s/it]
 40%|████      | 4808/12000 [2:51:57<4:01:36,  2.02s/it]
                                                        

 40%|████      | 4808/12000 [2:51:57<4:01:36,  2.02s/it]
 40%|████      | 4809/12000 [2:51:58<3:44:13,  1.87s/it]
                                                        

 40%|████      | 4809/12000 [2:51:58<3:44:13,  1.87s/it]
 40%|████      | 4810/12000 [2:52:02<4:27:21,  2.23s/it]
                                                        

 40%|████      | 4810/12000 [2:52:02<4:27:21,  2.23s/it]
 40%|████      | 4811/12000 [2:52:03<4:05:20,  2.05s/it]
                                                        

 40%|████      | 4811/12000 [2:52:03<4:05:20,  2.05s/it]
 40%|████      | 4812/12000 [2:52:06<4:47:11,  2.40s/it]
                                                        

 40%|████      | 4812/12000 [2:52:06<4:47:11,  2.40s/it]
 40%|████      | 4813/12000 [2:52:08<4:25:59,  2.22s/it]
                                                        

 40%|████      | 4813/12000 [2:52:08<4:25:59,  2.22s/it]
 40%|████      | 4814/12000 [2:52:10<3:56:11,  1.97s/it]
                                                        

 40%|████      | 4814/12000 [2:52:10<3:56:11,  1.97s/it]
 40%|████      | 4815/12000 [2:52:11<3:46:44,  1.89s/it]
                                                        

 40%|████      | 4815/12000 [2:52:11<3:46:44,  1.89s/it]
 40%|████      | 4816/12000 [2:52:13<3:38:34,  1.83s/it]
                                                        

 40%|████      | 4816/12000 [2:52:13<3:38:34,  1.83s/it]
 40%|████      | 4817/12000 [2:52:18<5:47:28,  2.90s/it]
                                                        

 40%|████      | 4817/12000 [2:52:18<5:47:28,  2.90s/it]
 40%|████      | 4818/12000 [2:52:21<5:45:47,  2.89s/it]
                                                        

 40%|████      | 4818/12000 [2:52:21<5:45:47,  2.89s/it]
 40%|████      | 4819/12000 [2:52:23<5:02:01,  2.52s/it]
                                                        

 40%|████      | 4819/12000 [2:52:23<5:02:01,  2.52s/it]
 40%|████      | 4820/12000 [2:52:25<4:47:47,  2.40s/it]
                                                        

 40%|████      | 4820/12000 [2:52:25<4:47:47,  2.40s/it]
 40%|████      | 4821/12000 [2:52:27<4:48:59,  2.42s/it]
                                                        

 40%|████      | 4821/12000 [2:52:27<4:48:59,  2.42s/it]
 40%|████      | 4822/12000 [2:52:29<4:09:46,  2.09s/it]
                                                        

 40%|████      | 4822/12000 [2:52:29<4:09:46,  2.09s/it]
 40%|████      | 4823/12000 [2:52:31<4:26:28,  2.23s/it]
                                                        

 40%|████      | 4823/12000 [2:52:31<4:26:28,  2.23s/it]
 40%|████      | 4824/12000 [2:52:33<3:59:19,  2.00s/it]
                                                        

 40%|████      | 4824/12000 [2:52:33<3:59:19,  2.00s/it]
 40%|████      | 4825/12000 [2:52:34<3:40:44,  1.85s/it]
                                                        

 40%|████      | 4825/12000 [2:52:34<3:40:44,  1.85s/it]
 40%|████      | 4826/12000 [2:52:36<3:31:34,  1.77s/it]
                                                        

 40%|████      | 4826/12000 [2:52:36<3:31:34,  1.77s/it]
 40%|████      | 4827/12000 [2:52:38<3:34:32,  1.79s/it]
                                                        

 40%|████      | 4827/12000 [2:52:38<3:34:32,  1.79s/it]
 40%|████      | 4828/12000 [2:52:40<3:38:51,  1.83s/it]
                                                        

 40%|████      | 4828/12000 [2:52:40<3:38:51,  1.83s/it]
 40%|████      | 4829/12000 [2:52:41<3:19:02,  1.67s/it]
                                                        

 40%|████      | 4829/12000 [2:52:41<3:19:02,  1.67s/it]
 40%|████      | 4830/12000 [2:52:43<3:17:16,  1.65s/it]
                                                        

 40%|████      | 4830/12000 [2:52:43<3:17:16,  1.65s/it]
 40%|████      | 4831/12000 [2:52:45<3:50:46,  1.93s/it]
                                                        

 40%|████      | 4831/12000 [2:52:45<3:50:46,  1.93s/it]
 40%|████      | 4832/12000 [2:52:50<5:28:51,  2.75s/it]
                                                        

 40%|████      | 4832/12000 [2:52:50<5:28:51,  2.75s/it]
 40%|████      | 4833/12000 [2:52:52<5:24:14,  2.71s/it]
                                                        

 40%|████      | 4833/12000 [2:52:52<5:24:14,  2.71s/it]
 40%|████      | 4834/12000 [2:52:56<5:44:58,  2.89s/it]
                                                        

 40%|████      | 4834/12000 [2:52:56<5:44:58,  2.89s/it]
 40%|████      | 4835/12000 [2:52:58<5:20:24,  2.68s/it]
                                                        

 40%|████      | 4835/12000 [2:52:58<5:20:24,  2.68s/it]
 40%|████      | 4836/12000 [2:52:59<4:35:55,  2.31s/it]
                                                        

 40%|████      | 4836/12000 [2:52:59<4:35:55,  2.31s/it]
 40%|████      | 4837/12000 [2:53:01<4:13:45,  2.13s/it]
                                                        

 40%|████      | 4837/12000 [2:53:01<4:13:45,  2.13s/it]
 40%|████      | 4838/12000 [2:53:02<3:48:02,  1.91s/it]
                                                        

 40%|████      | 4838/12000 [2:53:02<3:48:02,  1.91s/it]
 40%|████      | 4839/12000 [2:53:04<3:45:52,  1.89s/it]
                                                        

 40%|████      | 4839/12000 [2:53:04<3:45:52,  1.89s/it]
 40%|████      | 4840/12000 [2:53:06<3:37:45,  1.82s/it]
                                                        

 40%|████      | 4840/12000 [2:53:06<3:37:45,  1.82s/it]
 40%|████      | 4841/12000 [2:53:07<3:20:01,  1.68s/it]
                                                        

 40%|████      | 4841/12000 [2:53:07<3:20:01,  1.68s/it]
 40%|████      | 4842/12000 [2:53:10<3:46:38,  1.90s/it]
                                                        

 40%|████      | 4842/12000 [2:53:10<3:46:38,  1.90s/it]
 40%|████      | 4843/12000 [2:53:11<3:35:54,  1.81s/it]
                                                        

 40%|████      | 4843/12000 [2:53:11<3:35:54,  1.81s/it]
 40%|████      | 4844/12000 [2:53:13<3:21:38,  1.69s/it]
                                                        

 40%|████      | 4844/12000 [2:53:13<3:21:38,  1.69s/it]
 40%|████      | 4845/12000 [2:53:14<3:10:00,  1.59s/it]
                                                        
{'loss': '0.758', 'grad_norm': '0.3491', 'learning_rate': '0.0001402', 'entropy': '0.7365', 'num_tokens': '2.597e+07', 'mean_token_accuracy': '0.8115', 'epoch': '0.04246'}
{'loss': '1.313', 'grad_norm': '0.4332', 'learning_rate': '0.0001402', 'entropy': '1.322', 'num_tokens': '2.598e+07', 'mean_token_accuracy': '0.6941', 'epoch': '0.04247'}
{'loss': '0.8382', 'grad_norm': '0.4065', 'learning_rate': '0.0001402', 'entropy': '0.8182', 'num_tokens': '2.598e+07', 'mean_token_accuracy': '0.7914', 'epoch': '0.04248'}
{'loss': '1.009', 'grad_norm': '0.4465', 'learning_rate': '0.0001402', 'entropy': '0.9952', 'num_tokens': '2.599e+07', 'mean_token_accuracy': '0.7524', 'epoch': '0.04249'}
{'loss': '0.6946', 'grad_norm': '0.4302', 'learning_rate': '0.0001401', 'entropy': '0.7552', 'num_tokens': '2.599e+07', 'mean_token_accuracy': '0.8204', 'epoch': '0.0425'}
{'loss': '0.5338', 'grad_norm': '0.2984', 'learning_rate': '0.0001401', 'entropy': '0.5796', 'num_tokens': '2.6e+07', 'mean_token_accuracy': '0.8562', 'epoch': '0.04251'}
{'loss': '0.8637', 'grad_norm': '0.3497', 'learning_rate': '0.0001401', 'entropy': '0.8963', 'num_tokens': '2.6e+07', 'mean_token_accuracy': '0.7903', 'epoch': '0.04251'}
{'loss': '0.6641', 'grad_norm': '0.4271', 'learning_rate': '0.0001401', 'entropy': '0.7791', 'num_tokens': '2.601e+07', 'mean_token_accuracy': '0.7998', 'epoch': '0.04252'}
{'loss': '0.6882', 'grad_norm': '0.334', 'learning_rate': '0.00014', 'entropy': '0.7031', 'num_tokens': '2.601e+07', 'mean_token_accuracy': '0.8163', 'epoch': '0.04253'}
{'loss': '0.7684', 'grad_norm': '0.3788', 'learning_rate': '0.00014', 'entropy': '0.7036', 'num_tokens': '2.602e+07', 'mean_token_accuracy': '0.828', 'epoch': '0.04254'}
{'loss': '0.5871', 'grad_norm': '0.3084', 'learning_rate': '0.00014', 'entropy': '0.5946', 'num_tokens': '2.603e+07', 'mean_token_accuracy': '0.8474', 'epoch': '0.04255'}
{'loss': '0.6359', 'grad_norm': '0.3772', 'learning_rate': '0.00014', 'entropy': '0.6445', 'num_tokens': '2.603e+07', 'mean_token_accuracy': '0.8491', 'epoch': '0.04256'}
{'loss': '1.043', 'grad_norm': '0.4734', 'learning_rate': '0.0001399', 'entropy': '1.038', 'num_tokens': '2.604e+07', 'mean_token_accuracy': '0.7621', 'epoch': '0.04257'}
{'loss': '0.7044', 'grad_norm': '0.4434', 'learning_rate': '0.0001399', 'entropy': '0.7496', 'num_tokens': '2.604e+07', 'mean_token_accuracy': '0.8048', 'epoch': '0.04258'}
{'loss': '0.4877', 'grad_norm': '0.2433', 'learning_rate': '0.0001399', 'entropy': '0.6394', 'num_tokens': '2.605e+07', 'mean_token_accuracy': '0.8352', 'epoch': '0.04259'}
{'loss': '0.9603', 'grad_norm': '0.3763', 'learning_rate': '0.0001399', 'entropy': '0.9801', 'num_tokens': '2.605e+07', 'mean_token_accuracy': '0.7499', 'epoch': '0.04259'}
{'loss': '1.044', 'grad_norm': '0.3881', 'learning_rate': '0.0001398', 'entropy': '1.063', 'num_tokens': '2.606e+07', 'mean_token_accuracy': '0.7472', 'epoch': '0.0426'}
{'loss': '1.261', 'grad_norm': '0.4256', 'learning_rate': '0.0001398', 'entropy': '1.29', 'num_tokens': '2.606e+07', 'mean_token_accuracy': '0.7198', 'epoch': '0.04261'}
{'loss': '0.9322', 'grad_norm': '0.4086', 'learning_rate': '0.0001398', 'entropy': '0.8839', 'num_tokens': '2.607e+07', 'mean_token_accuracy': '0.7774', 'epoch': '0.04262'}
{'loss': '1.186', 'grad_norm': '0.3894', 'learning_rate': '0.0001398', 'entropy': '1.061', 'num_tokens': '2.608e+07', 'mean_token_accuracy': '0.746', 'epoch': '0.04263'}
{'loss': '0.7122', 'grad_norm': '0.4214', 'learning_rate': '0.0001397', 'entropy': '0.727', 'num_tokens': '2.608e+07', 'mean_token_accuracy': '0.8047', 'epoch': '0.04264'}
{'loss': '0.9794', 'grad_norm': '0.3879', 'learning_rate': '0.0001397', 'entropy': '0.9536', 'num_tokens': '2.609e+07', 'mean_token_accuracy': '0.7831', 'epoch': '0.04265'}
{'loss': '0.7934', 'grad_norm': '0.415', 'learning_rate': '0.0001397', 'entropy': '0.7336', 'num_tokens': '2.609e+07', 'mean_token_accuracy': '0.8216', 'epoch': '0.04266'}
{'loss': '0.7547', 'grad_norm': '0.3816', 'learning_rate': '0.0001397', 'entropy': '0.6505', 'num_tokens': '2.61e+07', 'mean_token_accuracy': '0.8383', 'epoch': '0.04267'}
{'loss': '0.7897', 'grad_norm': '0.3955', 'learning_rate': '0.0001396', 'entropy': '0.8121', 'num_tokens': '2.611e+07', 'mean_token_accuracy': '0.7797', 'epoch': '0.04267'}
{'loss': '0.7697', 'grad_norm': '0.3404', 'learning_rate': '0.0001396', 'entropy': '0.7933', 'num_tokens': '2.611e+07', 'mean_token_accuracy': '0.7965', 'epoch': '0.04268'}
{'loss': '0.7799', 'grad_norm': '0.4138', 'learning_rate': '0.0001396', 'entropy': '0.8895', 'num_tokens': '2.612e+07', 'mean_token_accuracy': '0.7779', 'epoch': '0.04269'}
{'loss': '0.6597', 'grad_norm': '0.3528', 'learning_rate': '0.0001396', 'entropy': '0.6549', 'num_tokens': '2.612e+07', 'mean_token_accuracy': '0.8386', 'epoch': '0.0427'}
{'loss': '0.9511', 'grad_norm': '0.3343', 'learning_rate': '0.0001395', 'entropy': '0.9793', 'num_tokens': '2.613e+07', 'mean_token_accuracy': '0.7615', 'epoch': '0.04271'}
{'loss': '0.9528', 'grad_norm': '0.408', 'learning_rate': '0.0001395', 'entropy': '0.9438', 'num_tokens': '2.613e+07', 'mean_token_accuracy': '0.7672', 'epoch': '0.04272'}
{'loss': '0.9614', 'grad_norm': '0.4238', 'learning_rate': '0.0001395', 'entropy': '1.004', 'num_tokens': '2.614e+07', 'mean_token_accuracy': '0.7524', 'epoch': '0.04273'}
{'loss': '0.681', 'grad_norm': '0.4412', 'learning_rate': '0.0001395', 'entropy': '0.7138', 'num_tokens': '2.614e+07', 'mean_token_accuracy': '0.8304', 'epoch': '0.04274'}
{'loss': '0.6629', 'grad_norm': '0.3678', 'learning_rate': '0.0001394', 'entropy': '0.6765', 'num_tokens': '2.615e+07', 'mean_token_accuracy': '0.8265', 'epoch': '0.04274'}
{'loss': '0.65', 'grad_norm': '0.2793', 'learning_rate': '0.0001394', 'entropy': '0.752', 'num_tokens': '2.615e+07', 'mean_token_accuracy': '0.8108', 'epoch': '0.04275'}
{'loss': '0.7863', 'grad_norm': '0.2989', 'learning_rate': '0.0001394', 'entropy': '0.8129', 'num_tokens': '2.616e+07', 'mean_token_accuracy': '0.776', 'epoch': '0.04276'}
{'loss': '1.116', 'grad_norm': '0.4733', 'learning_rate': '0.0001394', 'entropy': '1.058', 'num_tokens': '2.616e+07', 'mean_token_accuracy': '0.7341', 'epoch': '0.04277'}
{'loss': '1.069', 'grad_norm': '0.3674', 'learning_rate': '0.0001393', 'entropy': '1.004', 'num_tokens': '2.617e+07', 'mean_token_accuracy': '0.7617', 'epoch': '0.04278'}
{'loss': '0.9041', 'grad_norm': '0.3603', 'learning_rate': '0.0001393', 'entropy': '0.987', 'num_tokens': '2.618e+07', 'mean_token_accuracy': '0.7677', 'epoch': '0.04279'}
{'loss': '0.6791', 'grad_norm': '0.4009', 'learning_rate': '0.0001393', 'entropy': '0.6965', 'num_tokens': '2.618e+07', 'mean_token_accuracy': '0.8172', 'epoch': '0.0428'}
{'loss': '0.8858', 'grad_norm': '0.3932', 'learning_rate': '0.0001393', 'entropy': '0.9369', 'num_tokens': '2.619e+07', 'mean_token_accuracy': '0.7728', 'epoch': '0.04281'}
{'loss': '0.6718', 'grad_norm': '0.2974', 'learning_rate': '0.0001392', 'entropy': '0.7661', 'num_tokens': '2.619e+07', 'mean_token_accuracy': '0.8081', 'epoch': '0.04282'}
{'loss': '0.992', 'grad_norm': '0.4362', 'learning_rate': '0.0001392', 'entropy': '1.031', 'num_tokens': '2.62e+07', 'mean_token_accuracy': '0.7555', 'epoch': '0.04282'}
{'loss': '0.9977', 'grad_norm': '0.4365', 'learning_rate': '0.0001392', 'entropy': '0.9642', 'num_tokens': '2.62e+07', 'mean_token_accuracy': '0.7606', 'epoch': '0.04283'}
{'loss': '0.873', 'grad_norm': '0.3357', 'learning_rate': '0.0001392', 'entropy': '0.9279', 'num_tokens': '2.621e+07', 'mean_token_accuracy': '0.7726', 'epoch': '0.04284'}
{'loss': '0.9773', 'grad_norm': '0.4983', 'learning_rate': '0.0001391', 'entropy': '0.8537', 'num_tokens': '2.621e+07', 'mean_token_accuracy': '0.7784', 'epoch': '0.04285'}
{'loss': '0.7656', 'grad_norm': '0.3763', 'learning_rate': '0.0001391', 'entropy': '0.825', 'num_tokens': '2.622e+07', 'mean_token_accuracy': '0.7913', 'epoch': '0.04286'}
{'loss': '0.8566', 'grad_norm': '0.4403', 'learning_rate': '0.0001391', 'entropy': '0.8814', 'num_tokens': '2.622e+07', 'mean_token_accuracy': '0.7797', 'epoch': '0.04287'}

 40%|████      | 4845/12000 [2:53:14<3:10:00,  1.59s/it]
 40%|████      | 4846/12000 [2:53:16<3:11:50,  1.61s/it]
                                                        

 40%|████      | 4846/12000 [2:53:16<3:11:50,  1.61s/it]
 40%|████      | 4847/12000 [2:53:18<3:34:56,  1.80s/it]
                                                        

 40%|████      | 4847/12000 [2:53:18<3:34:56,  1.80s/it]
 40%|████      | 4848/12000 [2:53:20<3:52:28,  1.95s/it]
                                                        

 40%|████      | 4848/12000 [2:53:20<3:52:28,  1.95s/it]
 40%|████      | 4849/12000 [2:53:22<3:38:34,  1.83s/it]
                                                        

 40%|████      | 4849/12000 [2:53:22<3:38:34,  1.83s/it]
 40%|████      | 4850/12000 [2:53:24<3:37:18,  1.82s/it]
                                                        

 40%|████      | 4850/12000 [2:53:24<3:37:18,  1.82s/it]
 40%|████      | 4851/12000 [2:53:28<5:01:08,  2.53s/it]
                                                        

 40%|████      | 4851/12000 [2:53:28<5:01:08,  2.53s/it]
 40%|████      | 4852/12000 [2:53:30<4:40:54,  2.36s/it]
                                                        

 40%|████      | 4852/12000 [2:53:30<4:40:54,  2.36s/it]
 40%|████      | 4853/12000 [2:53:33<5:07:08,  2.58s/it]
                                                        

 40%|████      | 4853/12000 [2:53:33<5:07:08,  2.58s/it]
 40%|████      | 4854/12000 [2:53:37<5:55:21,  2.98s/it]
                                                        

 40%|████      | 4854/12000 [2:53:37<5:55:21,  2.98s/it]
 40%|████      | 4855/12000 [2:53:39<5:25:26,  2.73s/it]
                                                        

 40%|████      | 4855/12000 [2:53:39<5:25:26,  2.73s/it]
 40%|████      | 4856/12000 [2:53:42<5:20:48,  2.69s/it]
                                                        

 40%|████      | 4856/12000 [2:53:42<5:20:48,  2.69s/it]
 40%|████      | 4857/12000 [2:53:43<4:32:12,  2.29s/it]
                                                        

 40%|████      | 4857/12000 [2:53:43<4:32:12,  2.29s/it]
 40%|████      | 4858/12000 [2:53:44<4:05:36,  2.06s/it]
                                                        

 40%|████      | 4858/12000 [2:53:44<4:05:36,  2.06s/it]
 40%|████      | 4859/12000 [2:53:46<3:42:52,  1.87s/it]
                                                        

 40%|████      | 4859/12000 [2:53:46<3:42:52,  1.87s/it]
 40%|████      | 4860/12000 [2:53:47<3:25:18,  1.73s/it]
                                                        

 40%|████      | 4860/12000 [2:53:47<3:25:18,  1.73s/it]
 41%|████      | 4861/12000 [2:53:49<3:12:41,  1.62s/it]
                                                        

 41%|████      | 4861/12000 [2:53:49<3:12:41,  1.62s/it]
 41%|████      | 4862/12000 [2:53:50<3:15:25,  1.64s/it]
                                                        

 41%|████      | 4862/12000 [2:53:50<3:15:25,  1.64s/it]
 41%|████      | 4863/12000 [2:53:52<3:19:46,  1.68s/it]
                                                        

 41%|████      | 4863/12000 [2:53:52<3:19:46,  1.68s/it]
 41%|████      | 4864/12000 [2:53:54<3:45:31,  1.90s/it]
                                                        

 41%|████      | 4864/12000 [2:53:54<3:45:31,  1.90s/it]
 41%|████      | 4865/12000 [2:53:57<3:57:58,  2.00s/it]
                                                        

 41%|████      | 4865/12000 [2:53:57<3:57:58,  2.00s/it]
 41%|████      | 4866/12000 [2:53:58<3:37:13,  1.83s/it]
                                                        

 41%|████      | 4866/12000 [2:53:58<3:37:13,  1.83s/it]
 41%|████      | 4867/12000 [2:54:00<3:55:45,  1.98s/it]
                                                        

 41%|████      | 4867/12000 [2:54:00<3:55:45,  1.98s/it]
 41%|████      | 4868/12000 [2:54:02<3:36:09,  1.82s/it]
                                                        

 41%|████      | 4868/12000 [2:54:02<3:36:09,  1.82s/it]
 41%|████      | 4869/12000 [2:54:04<3:54:49,  1.98s/it]
                                                        

 41%|████      | 4869/12000 [2:54:04<3:54:49,  1.98s/it]
 41%|████      | 4870/12000 [2:54:06<3:39:11,  1.84s/it]
                                                        

 41%|████      | 4870/12000 [2:54:06<3:39:11,  1.84s/it]
 41%|████      | 4871/12000 [2:54:07<3:26:56,  1.74s/it]
                                                        

 41%|████      | 4871/12000 [2:54:07<3:26:56,  1.74s/it]
 41%|████      | 4872/12000 [2:54:09<3:34:07,  1.80s/it]
                                                        

 41%|████      | 4872/12000 [2:54:09<3:34:07,  1.80s/it]
 41%|████      | 4873/12000 [2:54:11<3:23:00,  1.71s/it]
                                                        

 41%|████      | 4873/12000 [2:54:11<3:23:00,  1.71s/it]
 41%|████      | 4874/12000 [2:54:12<3:24:48,  1.72s/it]
                                                        

 41%|████      | 4874/12000 [2:54:12<3:24:48,  1.72s/it]
 41%|████      | 4875/12000 [2:54:14<3:14:34,  1.64s/it]
                                                        

 41%|████      | 4875/12000 [2:54:14<3:14:34,  1.64s/it]
 41%|████      | 4876/12000 [2:54:16<3:28:58,  1.76s/it]
                                                        

 41%|████      | 4876/12000 [2:54:16<3:28:58,  1.76s/it]
 41%|████      | 4877/12000 [2:54:20<4:56:01,  2.49s/it]
                                                        

 41%|████      | 4877/12000 [2:54:20<4:56:01,  2.49s/it]
 41%|████      | 4878/12000 [2:54:22<4:45:17,  2.40s/it]
                                                        

 41%|████      | 4878/12000 [2:54:22<4:45:17,  2.40s/it]
 41%|████      | 4879/12000 [2:54:24<4:10:46,  2.11s/it]
                                                        

 41%|████      | 4879/12000 [2:54:24<4:10:46,  2.11s/it]
 41%|████      | 4880/12000 [2:54:25<3:49:27,  1.93s/it]
                                                        

 41%|████      | 4880/12000 [2:54:25<3:49:27,  1.93s/it]
 41%|████      | 4881/12000 [2:54:27<3:54:44,  1.98s/it]
                                                        

 41%|████      | 4881/12000 [2:54:27<3:54:44,  1.98s/it]
 41%|████      | 4882/12000 [2:54:29<3:39:39,  1.85s/it]
                                                        

 41%|████      | 4882/12000 [2:54:29<3:39:39,  1.85s/it]
 41%|████      | 4883/12000 [2:54:31<3:55:13,  1.98s/it]
                                                        

 41%|████      | 4883/12000 [2:54:31<3:55:13,  1.98s/it]
 41%|████      | 4884/12000 [2:54:33<3:54:15,  1.98s/it]
                                                        

 41%|████      | 4884/12000 [2:54:33<3:54:15,  1.98s/it]
 41%|████      | 4885/12000 [2:54:35<4:02:50,  2.05s/it]
                                                        

 41%|████      | 4885/12000 [2:54:35<4:02:50,  2.05s/it]
 41%|████      | 4886/12000 [2:54:37<3:51:49,  1.96s/it]
                                                        

 41%|████      | 4886/12000 [2:54:37<3:51:49,  1.96s/it]
 41%|████      | 4887/12000 [2:54:39<3:38:26,  1.84s/it]
                                                        

 41%|████      | 4887/12000 [2:54:39<3:38:26,  1.84s/it]
 41%|████      | 4888/12000 [2:54:43<4:48:19,  2.43s/it]
                                                        

 41%|████      | 4888/12000 [2:54:43<4:48:19,  2.43s/it]
 41%|████      | 4889/12000 [2:54:45<4:38:15,  2.35s/it]
                                                        

 41%|████      | 4889/12000 [2:54:45<4:38:15,  2.35s/it]
 41%|████      | 4890/12000 [2:54:48<5:10:24,  2.62s/it]
                                                        

 41%|████      | 4890/12000 [2:54:48<5:10:24,  2.62s/it]
 41%|████      | 4891/12000 [2:54:50<5:01:38,  2.55s/it]
                                                        

 41%|████      | 4891/12000 [2:54:50<5:01:38,  2.55s/it]
 41%|████      | 4892/12000 [2:54:52<4:18:30,  2.18s/it]
                                                        
{'loss': '0.642', 'grad_norm': '0.4447', 'learning_rate': '0.0001391', 'entropy': '0.6978', 'num_tokens': '2.622e+07', 'mean_token_accuracy': '0.8193', 'epoch': '0.04288'}
{'loss': '1.295', 'grad_norm': '0.4673', 'learning_rate': '0.000139', 'entropy': '1.253', 'num_tokens': '2.623e+07', 'mean_token_accuracy': '0.7037', 'epoch': '0.04289'}
{'loss': '0.8656', 'grad_norm': '0.3567', 'learning_rate': '0.000139', 'entropy': '0.8842', 'num_tokens': '2.624e+07', 'mean_token_accuracy': '0.7832', 'epoch': '0.0429'}
{'loss': '0.8401', 'grad_norm': '0.3564', 'learning_rate': '0.000139', 'entropy': '0.8424', 'num_tokens': '2.624e+07', 'mean_token_accuracy': '0.7865', 'epoch': '0.0429'}
{'loss': '0.855', 'grad_norm': '0.4348', 'learning_rate': '0.000139', 'entropy': '0.9195', 'num_tokens': '2.625e+07', 'mean_token_accuracy': '0.7574', 'epoch': '0.04291'}
{'loss': '1.335', 'grad_norm': '0.5624', 'learning_rate': '0.0001389', 'entropy': '1.219', 'num_tokens': '2.625e+07', 'mean_token_accuracy': '0.7031', 'epoch': '0.04292'}
{'loss': '1.141', 'grad_norm': '0.4671', 'learning_rate': '0.0001389', 'entropy': '0.9598', 'num_tokens': '2.626e+07', 'mean_token_accuracy': '0.7623', 'epoch': '0.04293'}
{'loss': '0.6957', 'grad_norm': '0.3609', 'learning_rate': '0.0001389', 'entropy': '0.7926', 'num_tokens': '2.626e+07', 'mean_token_accuracy': '0.7946', 'epoch': '0.04294'}
{'loss': '0.7156', 'grad_norm': '0.3198', 'learning_rate': '0.0001389', 'entropy': '0.7428', 'num_tokens': '2.627e+07', 'mean_token_accuracy': '0.7896', 'epoch': '0.04295'}
{'loss': '0.7667', 'grad_norm': '0.3943', 'learning_rate': '0.0001388', 'entropy': '0.7645', 'num_tokens': '2.628e+07', 'mean_token_accuracy': '0.8104', 'epoch': '0.04296'}
{'loss': '0.6142', 'grad_norm': '0.3948', 'learning_rate': '0.0001388', 'entropy': '0.8002', 'num_tokens': '2.628e+07', 'mean_token_accuracy': '0.7933', 'epoch': '0.04297'}
{'loss': '0.7139', 'grad_norm': '0.3042', 'learning_rate': '0.0001388', 'entropy': '0.807', 'num_tokens': '2.629e+07', 'mean_token_accuracy': '0.8115', 'epoch': '0.04297'}
{'loss': '0.899', 'grad_norm': '0.4088', 'learning_rate': '0.0001388', 'entropy': '0.9226', 'num_tokens': '2.629e+07', 'mean_token_accuracy': '0.7609', 'epoch': '0.04298'}
{'loss': '0.8397', 'grad_norm': '0.3902', 'learning_rate': '0.0001387', 'entropy': '0.886', 'num_tokens': '2.63e+07', 'mean_token_accuracy': '0.78', 'epoch': '0.04299'}
{'loss': '0.6834', 'grad_norm': '0.3356', 'learning_rate': '0.0001387', 'entropy': '0.7004', 'num_tokens': '2.63e+07', 'mean_token_accuracy': '0.8133', 'epoch': '0.043'}
{'loss': '1.008', 'grad_norm': '0.4437', 'learning_rate': '0.0001387', 'entropy': '1.052', 'num_tokens': '2.63e+07', 'mean_token_accuracy': '0.7304', 'epoch': '0.04301'}
{'loss': '0.6763', 'grad_norm': '0.3103', 'learning_rate': '0.0001386', 'entropy': '0.6773', 'num_tokens': '2.631e+07', 'mean_token_accuracy': '0.8228', 'epoch': '0.04302'}
{'loss': '0.9959', 'grad_norm': '0.4074', 'learning_rate': '0.0001386', 'entropy': '0.9913', 'num_tokens': '2.631e+07', 'mean_token_accuracy': '0.7617', 'epoch': '0.04303'}
{'loss': '0.8335', 'grad_norm': '0.3892', 'learning_rate': '0.0001386', 'entropy': '0.8546', 'num_tokens': '2.632e+07', 'mean_token_accuracy': '0.7909', 'epoch': '0.04304'}
{'loss': '0.8338', 'grad_norm': '0.5041', 'learning_rate': '0.0001386', 'entropy': '0.8083', 'num_tokens': '2.632e+07', 'mean_token_accuracy': '0.791', 'epoch': '0.04305'}
{'loss': '0.9427', 'grad_norm': '0.4607', 'learning_rate': '0.0001385', 'entropy': '0.8773', 'num_tokens': '2.633e+07', 'mean_token_accuracy': '0.7827', 'epoch': '0.04305'}
{'loss': '0.841', 'grad_norm': '0.3627', 'learning_rate': '0.0001385', 'entropy': '0.8285', 'num_tokens': '2.633e+07', 'mean_token_accuracy': '0.7879', 'epoch': '0.04306'}
{'loss': '0.9264', 'grad_norm': '0.4175', 'learning_rate': '0.0001385', 'entropy': '0.8785', 'num_tokens': '2.634e+07', 'mean_token_accuracy': '0.7846', 'epoch': '0.04307'}
{'loss': '1.153', 'grad_norm': '0.5358', 'learning_rate': '0.0001385', 'entropy': '1.149', 'num_tokens': '2.634e+07', 'mean_token_accuracy': '0.7314', 'epoch': '0.04308'}
{'loss': '0.8417', 'grad_norm': '0.4041', 'learning_rate': '0.0001384', 'entropy': '0.8628', 'num_tokens': '2.635e+07', 'mean_token_accuracy': '0.7799', 'epoch': '0.04309'}
{'loss': '0.7862', 'grad_norm': '0.3377', 'learning_rate': '0.0001384', 'entropy': '0.8305', 'num_tokens': '2.636e+07', 'mean_token_accuracy': '0.8007', 'epoch': '0.0431'}
{'loss': '0.7601', 'grad_norm': '0.3865', 'learning_rate': '0.0001384', 'entropy': '0.7536', 'num_tokens': '2.636e+07', 'mean_token_accuracy': '0.809', 'epoch': '0.04311'}
{'loss': '1.377', 'grad_norm': '0.4423', 'learning_rate': '0.0001384', 'entropy': '1.375', 'num_tokens': '2.637e+07', 'mean_token_accuracy': '0.6905', 'epoch': '0.04312'}
{'loss': '0.8212', 'grad_norm': '0.3393', 'learning_rate': '0.0001383', 'entropy': '0.8436', 'num_tokens': '2.637e+07', 'mean_token_accuracy': '0.78', 'epoch': '0.04313'}
{'loss': '1.446', 'grad_norm': '0.5206', 'learning_rate': '0.0001383', 'entropy': '1.334', 'num_tokens': '2.637e+07', 'mean_token_accuracy': '0.6842', 'epoch': '0.04313'}
{'loss': '1.06', 'grad_norm': '0.4216', 'learning_rate': '0.0001383', 'entropy': '1.038', 'num_tokens': '2.638e+07', 'mean_token_accuracy': '0.7542', 'epoch': '0.04314'}
{'loss': '0.8121', 'grad_norm': '0.375', 'learning_rate': '0.0001383', 'entropy': '0.8187', 'num_tokens': '2.638e+07', 'mean_token_accuracy': '0.7992', 'epoch': '0.04315'}
{'loss': '1.271', 'grad_norm': '0.4748', 'learning_rate': '0.0001382', 'entropy': '1.239', 'num_tokens': '2.639e+07', 'mean_token_accuracy': '0.6803', 'epoch': '0.04316'}
{'loss': '1.08', 'grad_norm': '0.5394', 'learning_rate': '0.0001382', 'entropy': '1.028', 'num_tokens': '2.639e+07', 'mean_token_accuracy': '0.7507', 'epoch': '0.04317'}
{'loss': '0.7079', 'grad_norm': '0.3211', 'learning_rate': '0.0001382', 'entropy': '0.7214', 'num_tokens': '2.64e+07', 'mean_token_accuracy': '0.8048', 'epoch': '0.04318'}
{'loss': '0.8161', 'grad_norm': '0.4736', 'learning_rate': '0.0001382', 'entropy': '0.8417', 'num_tokens': '2.64e+07', 'mean_token_accuracy': '0.7828', 'epoch': '0.04319'}
{'loss': '0.8507', 'grad_norm': '0.3712', 'learning_rate': '0.0001381', 'entropy': '0.7853', 'num_tokens': '2.641e+07', 'mean_token_accuracy': '0.8063', 'epoch': '0.0432'}
{'loss': '1.087', 'grad_norm': '0.4798', 'learning_rate': '0.0001381', 'entropy': '1.052', 'num_tokens': '2.641e+07', 'mean_token_accuracy': '0.7226', 'epoch': '0.0432'}
{'loss': '0.4954', 'grad_norm': '0.2913', 'learning_rate': '0.0001381', 'entropy': '0.5863', 'num_tokens': '2.642e+07', 'mean_token_accuracy': '0.8527', 'epoch': '0.04321'}
{'loss': '1.869', 'grad_norm': '0.4428', 'learning_rate': '0.0001381', 'entropy': '1.417', 'num_tokens': '2.642e+07', 'mean_token_accuracy': '0.701', 'epoch': '0.04322'}
{'loss': '0.7783', 'grad_norm': '0.35', 'learning_rate': '0.000138', 'entropy': '0.8245', 'num_tokens': '2.643e+07', 'mean_token_accuracy': '0.7912', 'epoch': '0.04323'}
{'loss': '0.8793', 'grad_norm': '0.4113', 'learning_rate': '0.000138', 'entropy': '0.8297', 'num_tokens': '2.643e+07', 'mean_token_accuracy': '0.7959', 'epoch': '0.04324'}
{'loss': '1.019', 'grad_norm': '0.4369', 'learning_rate': '0.000138', 'entropy': '1.136', 'num_tokens': '2.644e+07', 'mean_token_accuracy': '0.7442', 'epoch': '0.04325'}
{'loss': '0.73', 'grad_norm': '0.3919', 'learning_rate': '0.000138', 'entropy': '0.8011', 'num_tokens': '2.645e+07', 'mean_token_accuracy': '0.8071', 'epoch': '0.04326'}
{'loss': '0.905', 'grad_norm': '0.5166', 'learning_rate': '0.0001379', 'entropy': '0.8871', 'num_tokens': '2.645e+07', 'mean_token_accuracy': '0.7916', 'epoch': '0.04327'}
{'loss': '1.11', 'grad_norm': '0.3097', 'learning_rate': '0.0001379', 'entropy': '1.13', 'num_tokens': '2.646e+07', 'mean_token_accuracy': '0.7529', 'epoch': '0.04328'}
{'loss': '1.143', 'grad_norm': '0.5203', 'learning_rate': '0.0001379', 'entropy': '1.097', 'num_tokens': '2.647e+07', 'mean_token_accuracy': '0.7254', 'epoch': '0.04328'}

 41%|████      | 4892/12000 [2:54:52<4:18:30,  2.18s/it]
 41%|████      | 4893/12000 [2:54:53<3:50:49,  1.95s/it]
                                                        

 41%|████      | 4893/12000 [2:54:53<3:50:49,  1.95s/it]
 41%|████      | 4894/12000 [2:54:55<3:33:01,  1.80s/it]
                                                        

 41%|████      | 4894/12000 [2:54:55<3:33:01,  1.80s/it]
 41%|████      | 4895/12000 [2:54:56<3:28:02,  1.76s/it]
                                                        

 41%|████      | 4895/12000 [2:54:56<3:28:02,  1.76s/it]
 41%|████      | 4896/12000 [2:55:00<4:46:05,  2.42s/it]
                                                        

 41%|████      | 4896/12000 [2:55:00<4:46:05,  2.42s/it]
 41%|████      | 4897/12000 [2:55:02<4:16:53,  2.17s/it]
                                                        

 41%|████      | 4897/12000 [2:55:02<4:16:53,  2.17s/it]
 41%|████      | 4898/12000 [2:55:03<3:56:10,  2.00s/it]
                                                        

 41%|████      | 4898/12000 [2:55:03<3:56:10,  2.00s/it]
 41%|████      | 4899/12000 [2:55:07<4:58:15,  2.52s/it]
                                                        

 41%|████      | 4899/12000 [2:55:07<4:58:15,  2.52s/it]
 41%|████      | 4900/12000 [2:55:09<4:24:53,  2.24s/it]
                                                        

 41%|████      | 4900/12000 [2:55:09<4:24:53,  2.24s/it]
 41%|████      | 4901/12000 [2:55:12<4:53:58,  2.48s/it]
                                                        

 41%|████      | 4901/12000 [2:55:12<4:53:58,  2.48s/it]
 41%|████      | 4902/12000 [2:55:13<4:13:05,  2.14s/it]
                                                        

 41%|████      | 4902/12000 [2:55:13<4:13:05,  2.14s/it]
 41%|████      | 4903/12000 [2:55:14<3:45:45,  1.91s/it]
                                                        

 41%|████      | 4903/12000 [2:55:14<3:45:45,  1.91s/it]
 41%|████      | 4904/12000 [2:55:17<4:15:45,  2.16s/it]
                                                        

 41%|████      | 4904/12000 [2:55:17<4:15:45,  2.16s/it]
 41%|████      | 4905/12000 [2:55:19<3:51:45,  1.96s/it]
                                                        

 41%|████      | 4905/12000 [2:55:19<3:51:45,  1.96s/it]
 41%|████      | 4906/12000 [2:55:21<4:21:12,  2.21s/it]
                                                        

 41%|████      | 4906/12000 [2:55:21<4:21:12,  2.21s/it]
 41%|████      | 4907/12000 [2:55:23<4:00:27,  2.03s/it]
                                                        

 41%|████      | 4907/12000 [2:55:23<4:00:27,  2.03s/it]
 41%|████      | 4908/12000 [2:55:25<3:48:04,  1.93s/it]
                                                        

 41%|████      | 4908/12000 [2:55:25<3:48:04,  1.93s/it]
 41%|████      | 4909/12000 [2:55:26<3:40:39,  1.87s/it]
                                                        

 41%|████      | 4909/12000 [2:55:26<3:40:39,  1.87s/it]
 41%|████      | 4910/12000 [2:55:28<3:28:11,  1.76s/it]
                                                        

 41%|████      | 4910/12000 [2:55:28<3:28:11,  1.76s/it]
 41%|████      | 4911/12000 [2:55:30<3:26:04,  1.74s/it]
                                                        

 41%|████      | 4911/12000 [2:55:30<3:26:04,  1.74s/it]
 41%|████      | 4912/12000 [2:55:31<3:20:55,  1.70s/it]
                                                        

 41%|████      | 4912/12000 [2:55:31<3:20:55,  1.70s/it]
 41%|████      | 4913/12000 [2:55:33<3:29:10,  1.77s/it]
                                                        

 41%|████      | 4913/12000 [2:55:33<3:29:10,  1.77s/it]
 41%|████      | 4914/12000 [2:55:35<3:27:31,  1.76s/it]
                                                        

 41%|████      | 4914/12000 [2:55:35<3:27:31,  1.76s/it]
 41%|████      | 4915/12000 [2:55:37<3:25:12,  1.74s/it]
                                                        

 41%|████      | 4915/12000 [2:55:37<3:25:12,  1.74s/it]
 41%|████      | 4916/12000 [2:55:38<3:17:41,  1.67s/it]
                                                        

 41%|████      | 4916/12000 [2:55:38<3:17:41,  1.67s/it]
 41%|████      | 4917/12000 [2:55:40<3:13:54,  1.64s/it]
                                                        

 41%|████      | 4917/12000 [2:55:40<3:13:54,  1.64s/it]
 41%|████      | 4918/12000 [2:55:41<3:16:42,  1.67s/it]
                                                        

 41%|████      | 4918/12000 [2:55:41<3:16:42,  1.67s/it]
 41%|████      | 4919/12000 [2:55:43<3:16:43,  1.67s/it]
                                                        

 41%|████      | 4919/12000 [2:55:43<3:16:43,  1.67s/it]
 41%|████      | 4920/12000 [2:55:47<4:39:50,  2.37s/it]
                                                        

 41%|████      | 4920/12000 [2:55:47<4:39:50,  2.37s/it]
 41%|████      | 4921/12000 [2:55:50<4:57:39,  2.52s/it]
                                                        

 41%|████      | 4921/12000 [2:55:50<4:57:39,  2.52s/it]
 41%|████      | 4922/12000 [2:55:52<4:28:21,  2.27s/it]
                                                        

 41%|████      | 4922/12000 [2:55:52<4:28:21,  2.27s/it]
 41%|████      | 4923/12000 [2:55:54<4:25:47,  2.25s/it]
                                                        

 41%|████      | 4923/12000 [2:55:54<4:25:47,  2.25s/it]
 41%|████      | 4924/12000 [2:55:55<3:59:34,  2.03s/it]
                                                        

 41%|████      | 4924/12000 [2:55:55<3:59:34,  2.03s/it]
 41%|████      | 4925/12000 [2:55:57<3:48:27,  1.94s/it]
                                                        

 41%|████      | 4925/12000 [2:55:57<3:48:27,  1.94s/it]
 41%|████      | 4926/12000 [2:55:59<3:33:00,  1.81s/it]
                                                        

 41%|████      | 4926/12000 [2:55:59<3:33:00,  1.81s/it]
 41%|████      | 4927/12000 [2:56:01<4:08:24,  2.11s/it]
                                                        

 41%|████      | 4927/12000 [2:56:01<4:08:24,  2.11s/it]
 41%|████      | 4928/12000 [2:56:03<3:53:39,  1.98s/it]
                                                        

 41%|████      | 4928/12000 [2:56:03<3:53:39,  1.98s/it]
 41%|████      | 4929/12000 [2:56:05<3:38:48,  1.86s/it]
                                                        

 41%|████      | 4929/12000 [2:56:05<3:38:48,  1.86s/it]
 41%|████      | 4930/12000 [2:56:07<3:44:31,  1.91s/it]
                                                        

 41%|████      | 4930/12000 [2:56:07<3:44:31,  1.91s/it]
 41%|████      | 4931/12000 [2:56:09<3:44:51,  1.91s/it]
                                                        

 41%|████      | 4931/12000 [2:56:09<3:44:51,  1.91s/it]
 41%|████      | 4932/12000 [2:56:11<4:03:46,  2.07s/it]
                                                        

 41%|████      | 4932/12000 [2:56:11<4:03:46,  2.07s/it]
 41%|████      | 4933/12000 [2:56:13<3:58:44,  2.03s/it]
                                                        

 41%|████      | 4933/12000 [2:56:13<3:58:44,  2.03s/it]
 41%|████      | 4934/12000 [2:56:15<3:56:22,  2.01s/it]
                                                        

 41%|████      | 4934/12000 [2:56:15<3:56:22,  2.01s/it]
 41%|████      | 4935/12000 [2:56:16<3:36:25,  1.84s/it]
                                                        

 41%|████      | 4935/12000 [2:56:16<3:36:25,  1.84s/it]
 41%|████      | 4936/12000 [2:56:18<3:42:16,  1.89s/it]
                                                        

 41%|████      | 4936/12000 [2:56:18<3:42:16,  1.89s/it]
 41%|████      | 4937/12000 [2:56:20<3:24:13,  1.73s/it]
                                                        

 41%|████      | 4937/12000 [2:56:20<3:24:13,  1.73s/it]
 41%|████      | 4938/12000 [2:56:21<3:20:07,  1.70s/it]
                                                        

 41%|████      | 4938/12000 [2:56:21<3:20:07,  1.70s/it]
 41%|████      | 4939/12000 [2:56:23<3:13:56,  1.65s/it]
                                                        
{'loss': '0.8938', 'grad_norm': '0.4272', 'learning_rate': '0.0001379', 'entropy': '0.8958', 'num_tokens': '2.647e+07', 'mean_token_accuracy': '0.7791', 'epoch': '0.04329'}
{'loss': '0.9469', 'grad_norm': '0.3968', 'learning_rate': '0.0001378', 'entropy': '0.9873', 'num_tokens': '2.647e+07', 'mean_token_accuracy': '0.7608', 'epoch': '0.0433'}
{'loss': '1.149', 'grad_norm': '0.4836', 'learning_rate': '0.0001378', 'entropy': '1.167', 'num_tokens': '2.648e+07', 'mean_token_accuracy': '0.7223', 'epoch': '0.04331'}
{'loss': '0.6786', 'grad_norm': '0.3871', 'learning_rate': '0.0001378', 'entropy': '0.6659', 'num_tokens': '2.648e+07', 'mean_token_accuracy': '0.8288', 'epoch': '0.04332'}
{'loss': '0.969', 'grad_norm': '0.3804', 'learning_rate': '0.0001378', 'entropy': '0.9696', 'num_tokens': '2.649e+07', 'mean_token_accuracy': '0.7668', 'epoch': '0.04333'}
{'loss': '0.6378', 'grad_norm': '0.3272', 'learning_rate': '0.0001377', 'entropy': '0.6959', 'num_tokens': '2.65e+07', 'mean_token_accuracy': '0.8223', 'epoch': '0.04334'}
{'loss': '0.7134', 'grad_norm': '0.3814', 'learning_rate': '0.0001377', 'entropy': '0.7365', 'num_tokens': '2.65e+07', 'mean_token_accuracy': '0.8109', 'epoch': '0.04335'}
{'loss': '0.8357', 'grad_norm': '0.3195', 'learning_rate': '0.0001377', 'entropy': '0.8235', 'num_tokens': '2.651e+07', 'mean_token_accuracy': '0.7759', 'epoch': '0.04336'}
{'loss': '0.8867', 'grad_norm': '0.3595', 'learning_rate': '0.0001377', 'entropy': '0.963', 'num_tokens': '2.651e+07', 'mean_token_accuracy': '0.7636', 'epoch': '0.04336'}
{'loss': '0.7898', 'grad_norm': '0.414', 'learning_rate': '0.0001376', 'entropy': '0.7973', 'num_tokens': '2.652e+07', 'mean_token_accuracy': '0.7975', 'epoch': '0.04337'}
{'loss': '0.8771', 'grad_norm': '0.4472', 'learning_rate': '0.0001376', 'entropy': '0.8902', 'num_tokens': '2.652e+07', 'mean_token_accuracy': '0.7819', 'epoch': '0.04338'}
{'loss': '0.8294', 'grad_norm': '0.3667', 'learning_rate': '0.0001376', 'entropy': '0.9219', 'num_tokens': '2.652e+07', 'mean_token_accuracy': '0.7728', 'epoch': '0.04339'}
{'loss': '0.9823', 'grad_norm': '0.2946', 'learning_rate': '0.0001376', 'entropy': '0.8978', 'num_tokens': '2.653e+07', 'mean_token_accuracy': '0.7714', 'epoch': '0.0434'}
{'loss': '0.8985', 'grad_norm': '0.4327', 'learning_rate': '0.0001375', 'entropy': '0.9768', 'num_tokens': '2.654e+07', 'mean_token_accuracy': '0.7501', 'epoch': '0.04341'}
{'loss': '1.22', 'grad_norm': '0.5218', 'learning_rate': '0.0001375', 'entropy': '1.177', 'num_tokens': '2.654e+07', 'mean_token_accuracy': '0.7148', 'epoch': '0.04342'}
{'loss': '1.002', 'grad_norm': '0.3832', 'learning_rate': '0.0001375', 'entropy': '0.9208', 'num_tokens': '2.655e+07', 'mean_token_accuracy': '0.7686', 'epoch': '0.04343'}
{'loss': '1.179', 'grad_norm': '0.4614', 'learning_rate': '0.0001375', 'entropy': '1.172', 'num_tokens': '2.655e+07', 'mean_token_accuracy': '0.7202', 'epoch': '0.04343'}
{'loss': '1.135', 'grad_norm': '0.4619', 'learning_rate': '0.0001374', 'entropy': '0.996', 'num_tokens': '2.656e+07', 'mean_token_accuracy': '0.7568', 'epoch': '0.04344'}
{'loss': '0.7418', 'grad_norm': '0.3739', 'learning_rate': '0.0001374', 'entropy': '0.7603', 'num_tokens': '2.656e+07', 'mean_token_accuracy': '0.7959', 'epoch': '0.04345'}
{'loss': '0.6324', 'grad_norm': '0.3005', 'learning_rate': '0.0001374', 'entropy': '0.6286', 'num_tokens': '2.657e+07', 'mean_token_accuracy': '0.8293', 'epoch': '0.04346'}
{'loss': '0.586', 'grad_norm': '0.2674', 'learning_rate': '0.0001373', 'entropy': '0.6664', 'num_tokens': '2.657e+07', 'mean_token_accuracy': '0.8192', 'epoch': '0.04347'}
{'loss': '0.8759', 'grad_norm': '0.4348', 'learning_rate': '0.0001373', 'entropy': '0.8753', 'num_tokens': '2.658e+07', 'mean_token_accuracy': '0.7828', 'epoch': '0.04348'}
{'loss': '0.8398', 'grad_norm': '0.3468', 'learning_rate': '0.0001373', 'entropy': '0.9041', 'num_tokens': '2.658e+07', 'mean_token_accuracy': '0.7755', 'epoch': '0.04349'}
{'loss': '0.8728', 'grad_norm': '0.4238', 'learning_rate': '0.0001373', 'entropy': '0.826', 'num_tokens': '2.659e+07', 'mean_token_accuracy': '0.7803', 'epoch': '0.0435'}
{'loss': '0.7852', 'grad_norm': '0.3418', 'learning_rate': '0.0001372', 'entropy': '0.7929', 'num_tokens': '2.659e+07', 'mean_token_accuracy': '0.7981', 'epoch': '0.04351'}
{'loss': '0.53', 'grad_norm': '0.3049', 'learning_rate': '0.0001372', 'entropy': '0.5562', 'num_tokens': '2.66e+07', 'mean_token_accuracy': '0.8575', 'epoch': '0.04351'}
{'loss': '0.877', 'grad_norm': '0.353', 'learning_rate': '0.0001372', 'entropy': '0.8619', 'num_tokens': '2.66e+07', 'mean_token_accuracy': '0.7865', 'epoch': '0.04352'}
{'loss': '0.7841', 'grad_norm': '0.4761', 'learning_rate': '0.0001372', 'entropy': '0.7639', 'num_tokens': '2.661e+07', 'mean_token_accuracy': '0.8004', 'epoch': '0.04353'}
{'loss': '0.6511', 'grad_norm': '0.3252', 'learning_rate': '0.0001371', 'entropy': '0.693', 'num_tokens': '2.662e+07', 'mean_token_accuracy': '0.8247', 'epoch': '0.04354'}
{'loss': '0.7807', 'grad_norm': '0.388', 'learning_rate': '0.0001371', 'entropy': '0.8014', 'num_tokens': '2.662e+07', 'mean_token_accuracy': '0.7841', 'epoch': '0.04355'}
{'loss': '0.9996', 'grad_norm': '0.4607', 'learning_rate': '0.0001371', 'entropy': '1.024', 'num_tokens': '2.663e+07', 'mean_token_accuracy': '0.7485', 'epoch': '0.04356'}
{'loss': '0.8461', 'grad_norm': '0.4155', 'learning_rate': '0.0001371', 'entropy': '0.7957', 'num_tokens': '2.663e+07', 'mean_token_accuracy': '0.7968', 'epoch': '0.04357'}
{'loss': '0.6157', 'grad_norm': '0.3708', 'learning_rate': '0.000137', 'entropy': '0.616', 'num_tokens': '2.664e+07', 'mean_token_accuracy': '0.8363', 'epoch': '0.04358'}
{'loss': '0.7503', 'grad_norm': '0.2923', 'learning_rate': '0.000137', 'entropy': '0.7251', 'num_tokens': '2.664e+07', 'mean_token_accuracy': '0.8279', 'epoch': '0.04359'}
{'loss': '0.9743', 'grad_norm': '0.4064', 'learning_rate': '0.000137', 'entropy': '0.9782', 'num_tokens': '2.665e+07', 'mean_token_accuracy': '0.7723', 'epoch': '0.04359'}
{'loss': '0.6295', 'grad_norm': '0.3375', 'learning_rate': '0.000137', 'entropy': '0.573', 'num_tokens': '2.666e+07', 'mean_token_accuracy': '0.8542', 'epoch': '0.0436'}
{'loss': '1.006', 'grad_norm': '0.4001', 'learning_rate': '0.0001369', 'entropy': '0.965', 'num_tokens': '2.666e+07', 'mean_token_accuracy': '0.7627', 'epoch': '0.04361'}
{'loss': '0.9613', 'grad_norm': '0.4694', 'learning_rate': '0.0001369', 'entropy': '0.9565', 'num_tokens': '2.666e+07', 'mean_token_accuracy': '0.7657', 'epoch': '0.04362'}
{'loss': '0.5423', 'grad_norm': '0.2695', 'learning_rate': '0.0001369', 'entropy': '0.5791', 'num_tokens': '2.667e+07', 'mean_token_accuracy': '0.8544', 'epoch': '0.04363'}
{'loss': '0.6167', 'grad_norm': '0.307', 'learning_rate': '0.0001369', 'entropy': '0.6069', 'num_tokens': '2.668e+07', 'mean_token_accuracy': '0.8467', 'epoch': '0.04364'}
{'loss': '0.975', 'grad_norm': '0.3814', 'learning_rate': '0.0001368', 'entropy': '0.9501', 'num_tokens': '2.669e+07', 'mean_token_accuracy': '0.767', 'epoch': '0.04365'}
{'loss': '0.9247', 'grad_norm': '0.4616', 'learning_rate': '0.0001368', 'entropy': '0.9096', 'num_tokens': '2.669e+07', 'mean_token_accuracy': '0.7796', 'epoch': '0.04366'}
{'loss': '0.8606', 'grad_norm': '0.347', 'learning_rate': '0.0001368', 'entropy': '0.8233', 'num_tokens': '2.67e+07', 'mean_token_accuracy': '0.7896', 'epoch': '0.04367'}
{'loss': '0.8254', 'grad_norm': '0.4448', 'learning_rate': '0.0001368', 'entropy': '0.8733', 'num_tokens': '2.67e+07', 'mean_token_accuracy': '0.7726', 'epoch': '0.04367'}
{'loss': '0.8949', 'grad_norm': '0.3762', 'learning_rate': '0.0001367', 'entropy': '0.8305', 'num_tokens': '2.671e+07', 'mean_token_accuracy': '0.7987', 'epoch': '0.04368'}
{'loss': '0.7346', 'grad_norm': '0.4163', 'learning_rate': '0.0001367', 'entropy': '0.74', 'num_tokens': '2.671e+07', 'mean_token_accuracy': '0.8192', 'epoch': '0.04369'}
{'loss': '0.8825', 'grad_norm': '0.4337', 'learning_rate': '0.0001367', 'entropy': '0.9351', 'num_tokens': '2.671e+07', 'mean_token_accuracy': '0.7768', 'epoch': '0.0437'}

 41%|████      | 4939/12000 [2:56:23<3:13:56,  1.65s/it]
 41%|████      | 4940/12000 [2:56:25<3:16:44,  1.67s/it]
                                                        

 41%|████      | 4940/12000 [2:56:25<3:16:44,  1.67s/it]
 41%|████      | 4941/12000 [2:56:27<3:29:33,  1.78s/it]
                                                        

 41%|████      | 4941/12000 [2:56:27<3:29:33,  1.78s/it]
 41%|████      | 4942/12000 [2:56:30<4:29:22,  2.29s/it]
                                                        

 41%|████      | 4942/12000 [2:56:30<4:29:22,  2.29s/it]
 41%|████      | 4943/12000 [2:56:32<4:28:53,  2.29s/it]
                                                        

 41%|████      | 4943/12000 [2:56:32<4:28:53,  2.29s/it]
 41%|████      | 4944/12000 [2:56:34<4:04:11,  2.08s/it]
                                                        

 41%|████      | 4944/12000 [2:56:34<4:04:11,  2.08s/it]
 41%|████      | 4945/12000 [2:56:37<4:36:50,  2.35s/it]
                                                        

 41%|████      | 4945/12000 [2:56:37<4:36:50,  2.35s/it]
 41%|████      | 4946/12000 [2:56:39<4:13:31,  2.16s/it]
                                                        

 41%|████      | 4946/12000 [2:56:39<4:13:31,  2.16s/it]
 41%|████      | 4947/12000 [2:56:40<3:47:05,  1.93s/it]
                                                        

 41%|████      | 4947/12000 [2:56:40<3:47:05,  1.93s/it]
 41%|████      | 4948/12000 [2:56:42<3:28:23,  1.77s/it]
                                                        

 41%|████      | 4948/12000 [2:56:42<3:28:23,  1.77s/it]
 41%|████      | 4949/12000 [2:56:43<3:19:33,  1.70s/it]
                                                        

 41%|████      | 4949/12000 [2:56:43<3:19:33,  1.70s/it]
 41%|████▏     | 4950/12000 [2:56:45<3:11:52,  1.63s/it]
                                                        

 41%|████▏     | 4950/12000 [2:56:45<3:11:52,  1.63s/it]
 41%|████▏     | 4951/12000 [2:56:48<4:29:43,  2.30s/it]
                                                        

 41%|████▏     | 4951/12000 [2:56:48<4:29:43,  2.30s/it]
 41%|████▏     | 4952/12000 [2:56:50<4:02:56,  2.07s/it]
                                                        

 41%|████▏     | 4952/12000 [2:56:50<4:02:56,  2.07s/it]
 41%|████▏     | 4953/12000 [2:56:52<3:52:54,  1.98s/it]
                                                        

 41%|████▏     | 4953/12000 [2:56:52<3:52:54,  1.98s/it]
 41%|████▏     | 4954/12000 [2:56:54<4:02:19,  2.06s/it]
                                                        

 41%|████▏     | 4954/12000 [2:56:54<4:02:19,  2.06s/it]
 41%|████▏     | 4955/12000 [2:56:56<3:52:40,  1.98s/it]
                                                        

 41%|████▏     | 4955/12000 [2:56:56<3:52:40,  1.98s/it]
 41%|████▏     | 4956/12000 [2:56:58<4:01:16,  2.06s/it]
                                                        

 41%|████▏     | 4956/12000 [2:56:58<4:01:16,  2.06s/it]
 41%|████▏     | 4957/12000 [2:57:00<3:59:03,  2.04s/it]
                                                        

 41%|████▏     | 4957/12000 [2:57:00<3:59:03,  2.04s/it]
 41%|████▏     | 4958/12000 [2:57:03<4:30:15,  2.30s/it]
                                                        

 41%|████▏     | 4958/12000 [2:57:03<4:30:15,  2.30s/it]
 41%|████▏     | 4959/12000 [2:57:05<4:13:34,  2.16s/it]
                                                        

 41%|████▏     | 4959/12000 [2:57:05<4:13:34,  2.16s/it]
 41%|████▏     | 4960/12000 [2:57:07<4:15:18,  2.18s/it]
                                                        

 41%|████▏     | 4960/12000 [2:57:07<4:15:18,  2.18s/it]
 41%|████▏     | 4961/12000 [2:57:10<4:49:15,  2.47s/it]
                                                        

 41%|████▏     | 4961/12000 [2:57:10<4:49:15,  2.47s/it]
 41%|████▏     | 4962/12000 [2:57:12<4:17:10,  2.19s/it]
                                                        

 41%|████▏     | 4962/12000 [2:57:12<4:17:10,  2.19s/it]
 41%|████▏     | 4963/12000 [2:57:13<3:53:23,  1.99s/it]
                                                        

 41%|████▏     | 4963/12000 [2:57:13<3:53:23,  1.99s/it]
 41%|████▏     | 4964/12000 [2:57:17<4:58:37,  2.55s/it]
                                                        

 41%|████▏     | 4964/12000 [2:57:17<4:58:37,  2.55s/it]
 41%|████▏     | 4965/12000 [2:57:19<4:40:33,  2.39s/it]
                                                        

 41%|████▏     | 4965/12000 [2:57:19<4:40:33,  2.39s/it]
 41%|████▏     | 4966/12000 [2:57:23<5:27:56,  2.80s/it]
                                                        

 41%|████▏     | 4966/12000 [2:57:23<5:27:56,  2.80s/it]
 41%|████▏     | 4967/12000 [2:57:24<4:38:39,  2.38s/it]
                                                        

 41%|████▏     | 4967/12000 [2:57:24<4:38:39,  2.38s/it]
 41%|████▏     | 4968/12000 [2:57:26<4:17:05,  2.19s/it]
                                                        

 41%|████▏     | 4968/12000 [2:57:26<4:17:05,  2.19s/it]
 41%|████▏     | 4969/12000 [2:57:31<6:08:48,  3.15s/it]
                                                        

 41%|████▏     | 4969/12000 [2:57:31<6:08:48,  3.15s/it]
 41%|████▏     | 4970/12000 [2:57:33<5:30:00,  2.82s/it]
                                                        

 41%|████▏     | 4970/12000 [2:57:33<5:30:00,  2.82s/it]
 41%|████▏     | 4971/12000 [2:57:35<4:41:50,  2.41s/it]
                                                        

 41%|████▏     | 4971/12000 [2:57:35<4:41:50,  2.41s/it]
 41%|████▏     | 4972/12000 [2:57:36<4:13:07,  2.16s/it]
                                                        

 41%|████▏     | 4972/12000 [2:57:36<4:13:07,  2.16s/it]
 41%|████▏     | 4973/12000 [2:57:38<3:51:37,  1.98s/it]
                                                        

 41%|████▏     | 4973/12000 [2:57:38<3:51:37,  1.98s/it]
 41%|████▏     | 4974/12000 [2:57:40<3:52:27,  1.99s/it]
                                                        

 41%|████▏     | 4974/12000 [2:57:40<3:52:27,  1.99s/it]
 41%|████▏     | 4975/12000 [2:57:42<3:41:29,  1.89s/it]
                                                        

 41%|████▏     | 4975/12000 [2:57:42<3:41:29,  1.89s/it]
 41%|████▏     | 4976/12000 [2:57:43<3:28:47,  1.78s/it]
                                                        

 41%|████▏     | 4976/12000 [2:57:43<3:28:47,  1.78s/it]
 41%|████▏     | 4977/12000 [2:57:45<3:15:39,  1.67s/it]
                                                        

 41%|████▏     | 4977/12000 [2:57:45<3:15:39,  1.67s/it]
 41%|████▏     | 4978/12000 [2:57:47<3:28:17,  1.78s/it]
                                                        

 41%|████▏     | 4978/12000 [2:57:47<3:28:17,  1.78s/it]
 41%|████▏     | 4979/12000 [2:57:48<3:26:04,  1.76s/it]
                                                        

 41%|████▏     | 4979/12000 [2:57:48<3:26:04,  1.76s/it]
 42%|████▏     | 4980/12000 [2:57:50<3:16:43,  1.68s/it]
                                                        

 42%|████▏     | 4980/12000 [2:57:50<3:16:43,  1.68s/it]
 42%|████▏     | 4981/12000 [2:57:52<3:30:33,  1.80s/it]
                                                        

 42%|████▏     | 4981/12000 [2:57:52<3:30:33,  1.80s/it]
 42%|████▏     | 4982/12000 [2:57:55<4:00:20,  2.05s/it]
                                                        

 42%|████▏     | 4982/12000 [2:57:55<4:00:20,  2.05s/it]
 42%|████▏     | 4983/12000 [2:57:57<4:15:51,  2.19s/it]
                                                        

 42%|████▏     | 4983/12000 [2:57:57<4:15:51,  2.19s/it]
 42%|████▏     | 4984/12000 [2:57:59<4:17:18,  2.20s/it]
                                                        

 42%|████▏     | 4984/12000 [2:57:59<4:17:18,  2.20s/it]
 42%|████▏     | 4985/12000 [2:58:01<4:13:10,  2.17s/it]
                                                        

 42%|████▏     | 4985/12000 [2:58:01<4:13:10,  2.17s/it]
 42%|████▏     | 4986/12000 [2:58:04<4:44:05,  2.43s/it]
                                                        
{'loss': '0.6583', 'grad_norm': '0.3975', 'learning_rate': '0.0001367', 'entropy': '0.6707', 'num_tokens': '2.672e+07', 'mean_token_accuracy': '0.8216', 'epoch': '0.04371'}
{'loss': '0.8702', 'grad_norm': '0.3965', 'learning_rate': '0.0001366', 'entropy': '0.9666', 'num_tokens': '2.672e+07', 'mean_token_accuracy': '0.7727', 'epoch': '0.04372'}
{'loss': '0.8415', 'grad_norm': '0.3365', 'learning_rate': '0.0001366', 'entropy': '0.9604', 'num_tokens': '2.673e+07', 'mean_token_accuracy': '0.7531', 'epoch': '0.04373'}
{'loss': '0.9285', 'grad_norm': '0.44', 'learning_rate': '0.0001366', 'entropy': '0.9874', 'num_tokens': '2.673e+07', 'mean_token_accuracy': '0.7527', 'epoch': '0.04374'}
{'loss': '0.821', 'grad_norm': '0.3705', 'learning_rate': '0.0001366', 'entropy': '0.8817', 'num_tokens': '2.674e+07', 'mean_token_accuracy': '0.7838', 'epoch': '0.04374'}
{'loss': '0.6811', 'grad_norm': '0.3988', 'learning_rate': '0.0001365', 'entropy': '0.7204', 'num_tokens': '2.675e+07', 'mean_token_accuracy': '0.8135', 'epoch': '0.04375'}
{'loss': '0.65', 'grad_norm': '0.3688', 'learning_rate': '0.0001365', 'entropy': '0.715', 'num_tokens': '2.675e+07', 'mean_token_accuracy': '0.8089', 'epoch': '0.04376'}
{'loss': '0.6671', 'grad_norm': '0.4277', 'learning_rate': '0.0001365', 'entropy': '0.725', 'num_tokens': '2.676e+07', 'mean_token_accuracy': '0.8195', 'epoch': '0.04377'}
{'loss': '0.8135', 'grad_norm': '0.4308', 'learning_rate': '0.0001365', 'entropy': '0.8063', 'num_tokens': '2.676e+07', 'mean_token_accuracy': '0.7994', 'epoch': '0.04378'}
{'loss': '0.7438', 'grad_norm': '0.347', 'learning_rate': '0.0001364', 'entropy': '0.7387', 'num_tokens': '2.677e+07', 'mean_token_accuracy': '0.8105', 'epoch': '0.04379'}
{'loss': '0.91', 'grad_norm': '0.492', 'learning_rate': '0.0001364', 'entropy': '0.9948', 'num_tokens': '2.677e+07', 'mean_token_accuracy': '0.7355', 'epoch': '0.0438'}
{'loss': '0.7075', 'grad_norm': '0.3926', 'learning_rate': '0.0001364', 'entropy': '0.6941', 'num_tokens': '2.678e+07', 'mean_token_accuracy': '0.8231', 'epoch': '0.04381'}
{'loss': '0.9844', 'grad_norm': '0.389', 'learning_rate': '0.0001364', 'entropy': '0.9475', 'num_tokens': '2.678e+07', 'mean_token_accuracy': '0.7464', 'epoch': '0.04382'}
{'loss': '1.018', 'grad_norm': '0.4065', 'learning_rate': '0.0001363', 'entropy': '1.054', 'num_tokens': '2.679e+07', 'mean_token_accuracy': '0.7518', 'epoch': '0.04382'}
{'loss': '0.7563', 'grad_norm': '0.441', 'learning_rate': '0.0001363', 'entropy': '0.7315', 'num_tokens': '2.679e+07', 'mean_token_accuracy': '0.7928', 'epoch': '0.04383'}
{'loss': '1.122', 'grad_norm': '0.4221', 'learning_rate': '0.0001363', 'entropy': '1.038', 'num_tokens': '2.68e+07', 'mean_token_accuracy': '0.7343', 'epoch': '0.04384'}
{'loss': '0.6703', 'grad_norm': '0.3312', 'learning_rate': '0.0001362', 'entropy': '0.6914', 'num_tokens': '2.68e+07', 'mean_token_accuracy': '0.8266', 'epoch': '0.04385'}
{'loss': '1.096', 'grad_norm': '0.5305', 'learning_rate': '0.0001362', 'entropy': '0.8945', 'num_tokens': '2.681e+07', 'mean_token_accuracy': '0.7742', 'epoch': '0.04386'}
{'loss': '0.7657', 'grad_norm': '0.445', 'learning_rate': '0.0001362', 'entropy': '0.7097', 'num_tokens': '2.681e+07', 'mean_token_accuracy': '0.8098', 'epoch': '0.04387'}
{'loss': '1.073', 'grad_norm': '0.3559', 'learning_rate': '0.0001362', 'entropy': '1.083', 'num_tokens': '2.682e+07', 'mean_token_accuracy': '0.7433', 'epoch': '0.04388'}
{'loss': '0.7684', 'grad_norm': '0.4215', 'learning_rate': '0.0001361', 'entropy': '0.8004', 'num_tokens': '2.682e+07', 'mean_token_accuracy': '0.7788', 'epoch': '0.04389'}
{'loss': '1.089', 'grad_norm': '0.4509', 'learning_rate': '0.0001361', 'entropy': '1.01', 'num_tokens': '2.683e+07', 'mean_token_accuracy': '0.7491', 'epoch': '0.0439'}
{'loss': '0.9011', 'grad_norm': '0.4042', 'learning_rate': '0.0001361', 'entropy': '1.043', 'num_tokens': '2.684e+07', 'mean_token_accuracy': '0.7425', 'epoch': '0.0439'}
{'loss': '0.9899', 'grad_norm': '0.4719', 'learning_rate': '0.0001361', 'entropy': '1.027', 'num_tokens': '2.684e+07', 'mean_token_accuracy': '0.7434', 'epoch': '0.04391'}
{'loss': '0.7782', 'grad_norm': '0.3716', 'learning_rate': '0.000136', 'entropy': '0.7458', 'num_tokens': '2.685e+07', 'mean_token_accuracy': '0.8046', 'epoch': '0.04392'}
{'loss': '0.6993', 'grad_norm': '0.3474', 'learning_rate': '0.000136', 'entropy': '0.7066', 'num_tokens': '2.685e+07', 'mean_token_accuracy': '0.8071', 'epoch': '0.04393'}
{'loss': '0.9376', 'grad_norm': '0.3038', 'learning_rate': '0.000136', 'entropy': '1.123', 'num_tokens': '2.686e+07', 'mean_token_accuracy': '0.727', 'epoch': '0.04394'}
{'loss': '1.222', 'grad_norm': '0.4268', 'learning_rate': '0.000136', 'entropy': '1.109', 'num_tokens': '2.687e+07', 'mean_token_accuracy': '0.7494', 'epoch': '0.04395'}
{'loss': '0.8758', 'grad_norm': '0.4791', 'learning_rate': '0.0001359', 'entropy': '0.9583', 'num_tokens': '2.687e+07', 'mean_token_accuracy': '0.7628', 'epoch': '0.04396'}
{'loss': '0.9511', 'grad_norm': '0.3451', 'learning_rate': '0.0001359', 'entropy': '0.9297', 'num_tokens': '2.688e+07', 'mean_token_accuracy': '0.7848', 'epoch': '0.04397'}
{'loss': '0.958', 'grad_norm': '0.3948', 'learning_rate': '0.0001359', 'entropy': '1.019', 'num_tokens': '2.689e+07', 'mean_token_accuracy': '0.7428', 'epoch': '0.04397'}
{'loss': '0.9289', 'grad_norm': '0.3986', 'learning_rate': '0.0001359', 'entropy': '0.8747', 'num_tokens': '2.689e+07', 'mean_token_accuracy': '0.78', 'epoch': '0.04398'}
{'loss': '0.7047', 'grad_norm': '0.3369', 'learning_rate': '0.0001358', 'entropy': '0.7507', 'num_tokens': '2.69e+07', 'mean_token_accuracy': '0.8172', 'epoch': '0.04399'}
{'loss': '0.8483', 'grad_norm': '0.4667', 'learning_rate': '0.0001358', 'entropy': '0.7973', 'num_tokens': '2.69e+07', 'mean_token_accuracy': '0.8038', 'epoch': '0.044'}
{'loss': '0.8939', 'grad_norm': '0.4584', 'learning_rate': '0.0001358', 'entropy': '0.9746', 'num_tokens': '2.691e+07', 'mean_token_accuracy': '0.7627', 'epoch': '0.04401'}
{'loss': '0.7096', 'grad_norm': '0.369', 'learning_rate': '0.0001358', 'entropy': '0.6832', 'num_tokens': '2.691e+07', 'mean_token_accuracy': '0.8247', 'epoch': '0.04402'}
{'loss': '0.6753', 'grad_norm': '0.3724', 'learning_rate': '0.0001357', 'entropy': '0.7235', 'num_tokens': '2.692e+07', 'mean_token_accuracy': '0.8106', 'epoch': '0.04403'}
{'loss': '0.8269', 'grad_norm': '0.3208', 'learning_rate': '0.0001357', 'entropy': '0.8497', 'num_tokens': '2.692e+07', 'mean_token_accuracy': '0.7937', 'epoch': '0.04404'}
{'loss': '0.9416', 'grad_norm': '0.4463', 'learning_rate': '0.0001357', 'entropy': '0.9688', 'num_tokens': '2.693e+07', 'mean_token_accuracy': '0.7664', 'epoch': '0.04405'}
{'loss': '0.8486', 'grad_norm': '0.4066', 'learning_rate': '0.0001357', 'entropy': '0.8505', 'num_tokens': '2.693e+07', 'mean_token_accuracy': '0.7708', 'epoch': '0.04405'}
{'loss': '1.249', 'grad_norm': '0.4616', 'learning_rate': '0.0001356', 'entropy': '0.987', 'num_tokens': '2.694e+07', 'mean_token_accuracy': '0.7593', 'epoch': '0.04406'}
{'loss': '1.401', 'grad_norm': '0.4266', 'learning_rate': '0.0001356', 'entropy': '1.275', 'num_tokens': '2.694e+07', 'mean_token_accuracy': '0.6895', 'epoch': '0.04407'}
{'loss': '0.7308', 'grad_norm': '0.3278', 'learning_rate': '0.0001356', 'entropy': '0.6843', 'num_tokens': '2.695e+07', 'mean_token_accuracy': '0.8194', 'epoch': '0.04408'}
{'loss': '0.7766', 'grad_norm': '0.3543', 'learning_rate': '0.0001356', 'entropy': '0.7616', 'num_tokens': '2.695e+07', 'mean_token_accuracy': '0.8064', 'epoch': '0.04409'}
{'loss': '0.6007', 'grad_norm': '0.3785', 'learning_rate': '0.0001355', 'entropy': '0.6029', 'num_tokens': '2.696e+07', 'mean_token_accuracy': '0.8483', 'epoch': '0.0441'}
{'loss': '0.6618', 'grad_norm': '0.3802', 'learning_rate': '0.0001355', 'entropy': '0.7269', 'num_tokens': '2.697e+07', 'mean_token_accuracy': '0.7985', 'epoch': '0.04411'}
{'loss': '0.9656', 'grad_norm': '0.3659', 'learning_rate': '0.0001355', 'entropy': '1.062', 'num_tokens': '2.697e+07', 'mean_token_accuracy': '0.7446', 'epoch': '0.04412'}

 42%|████▏     | 4986/12000 [2:58:04<4:44:05,  2.43s/it]
 42%|████▏     | 4987/12000 [2:58:06<4:08:12,  2.12s/it]
                                                        

 42%|████▏     | 4987/12000 [2:58:06<4:08:12,  2.12s/it]
 42%|████▏     | 4988/12000 [2:58:08<4:02:02,  2.07s/it]
                                                        

 42%|████▏     | 4988/12000 [2:58:08<4:02:02,  2.07s/it]
 42%|████▏     | 4989/12000 [2:58:12<5:26:34,  2.79s/it]
                                                        

 42%|████▏     | 4989/12000 [2:58:12<5:26:34,  2.79s/it]
 42%|████▏     | 4990/12000 [2:58:17<6:26:46,  3.31s/it]
                                                        

 42%|████▏     | 4990/12000 [2:58:17<6:26:46,  3.31s/it]
 42%|████▏     | 4991/12000 [2:58:19<5:31:27,  2.84s/it]
                                                        

 42%|████▏     | 4991/12000 [2:58:19<5:31:27,  2.84s/it]
 42%|████▏     | 4992/12000 [2:58:22<5:38:03,  2.89s/it]
                                                        

 42%|████▏     | 4992/12000 [2:58:22<5:38:03,  2.89s/it]
 42%|████▏     | 4993/12000 [2:58:23<4:48:34,  2.47s/it]
                                                        

 42%|████▏     | 4993/12000 [2:58:23<4:48:34,  2.47s/it]
 42%|████▏     | 4994/12000 [2:58:26<5:14:05,  2.69s/it]
                                                        

 42%|████▏     | 4994/12000 [2:58:26<5:14:05,  2.69s/it]
 42%|████▏     | 4995/12000 [2:58:29<5:07:04,  2.63s/it]
                                                        

 42%|████▏     | 4995/12000 [2:58:29<5:07:04,  2.63s/it]
 42%|████▏     | 4996/12000 [2:58:30<4:22:41,  2.25s/it]
                                                        

 42%|████▏     | 4996/12000 [2:58:30<4:22:41,  2.25s/it]
 42%|████▏     | 4997/12000 [2:58:32<3:55:47,  2.02s/it]
                                                        

 42%|████▏     | 4997/12000 [2:58:32<3:55:47,  2.02s/it]
 42%|████▏     | 4998/12000 [2:58:33<3:46:21,  1.94s/it]
                                                        

 42%|████▏     | 4998/12000 [2:58:33<3:46:21,  1.94s/it]
 42%|████▏     | 4999/12000 [2:58:36<4:05:19,  2.10s/it]
                                                        

 42%|████▏     | 4999/12000 [2:58:36<4:05:19,  2.10s/it]
 42%|████▏     | 5000/12000 [2:58:38<3:53:05,  2.00s/it]
                                                        

 42%|████▏     | 5000/12000 [2:58:38<3:53:05,  2.00s/it]
 42%|████▏     | 5001/12000 [2:58:41<4:38:13,  2.39s/it]
                                                        

 42%|████▏     | 5001/12000 [2:58:41<4:38:13,  2.39s/it]
 42%|████▏     | 5002/12000 [2:58:42<4:03:20,  2.09s/it]
                                                        

 42%|████▏     | 5002/12000 [2:58:42<4:03:20,  2.09s/it]
 42%|████▏     | 5003/12000 [2:58:44<3:47:38,  1.95s/it]
                                                        

 42%|████▏     | 5003/12000 [2:58:44<3:47:38,  1.95s/it]
 42%|████▏     | 5004/12000 [2:58:45<3:31:06,  1.81s/it]
                                                        

 42%|████▏     | 5004/12000 [2:58:45<3:31:06,  1.81s/it]
 42%|████▏     | 5005/12000 [2:58:47<3:24:18,  1.75s/it]
                                                        

 42%|████▏     | 5005/12000 [2:58:47<3:24:18,  1.75s/it]
 42%|████▏     | 5006/12000 [2:58:50<4:05:08,  2.10s/it]
                                                        

 42%|████▏     | 5006/12000 [2:58:50<4:05:08,  2.10s/it]
 42%|████▏     | 5007/12000 [2:58:52<3:48:41,  1.96s/it]
                                                        

 42%|████▏     | 5007/12000 [2:58:52<3:48:41,  1.96s/it]
 42%|████▏     | 5008/12000 [2:58:55<4:36:30,  2.37s/it]
                                                        

 42%|████▏     | 5008/12000 [2:58:55<4:36:30,  2.37s/it]
 42%|████▏     | 5009/12000 [2:58:57<4:11:40,  2.16s/it]
                                                        

 42%|████▏     | 5009/12000 [2:58:57<4:11:40,  2.16s/it]
 42%|████▏     | 5010/12000 [2:58:58<3:45:19,  1.93s/it]
                                                        

 42%|████▏     | 5010/12000 [2:58:58<3:45:19,  1.93s/it]
 42%|████▏     | 5011/12000 [2:59:00<3:34:10,  1.84s/it]
                                                        

 42%|████▏     | 5011/12000 [2:59:00<3:34:10,  1.84s/it]
 42%|████▏     | 5012/12000 [2:59:02<3:57:59,  2.04s/it]
                                                        

 42%|████▏     | 5012/12000 [2:59:02<3:57:59,  2.04s/it]
 42%|████▏     | 5013/12000 [2:59:03<3:35:14,  1.85s/it]
                                                        

 42%|████▏     | 5013/12000 [2:59:03<3:35:14,  1.85s/it]
 42%|████▏     | 5014/12000 [2:59:05<3:19:59,  1.72s/it]
                                                        

 42%|████▏     | 5014/12000 [2:59:05<3:19:59,  1.72s/it]
 42%|████▏     | 5015/12000 [2:59:07<3:29:13,  1.80s/it]
                                                        

 42%|████▏     | 5015/12000 [2:59:07<3:29:13,  1.80s/it]
 42%|████▏     | 5016/12000 [2:59:09<3:45:41,  1.94s/it]
                                                        

 42%|████▏     | 5016/12000 [2:59:09<3:45:41,  1.94s/it]
 42%|████▏     | 5017/12000 [2:59:11<3:49:58,  1.98s/it]
                                                        

 42%|████▏     | 5017/12000 [2:59:11<3:49:58,  1.98s/it]
 42%|████▏     | 5018/12000 [2:59:13<3:32:38,  1.83s/it]
                                                        

 42%|████▏     | 5018/12000 [2:59:13<3:32:38,  1.83s/it]
 42%|████▏     | 5019/12000 [2:59:14<3:16:28,  1.69s/it]
                                                        

 42%|████▏     | 5019/12000 [2:59:14<3:16:28,  1.69s/it]
 42%|████▏     | 5020/12000 [2:59:16<3:10:09,  1.63s/it]
                                                        

 42%|████▏     | 5020/12000 [2:59:16<3:10:09,  1.63s/it]
 42%|████▏     | 5021/12000 [2:59:17<3:02:54,  1.57s/it]
                                                        

 42%|████▏     | 5021/12000 [2:59:17<3:02:54,  1.57s/it]
 42%|████▏     | 5022/12000 [2:59:20<3:45:44,  1.94s/it]
                                                        

 42%|████▏     | 5022/12000 [2:59:20<3:45:44,  1.94s/it]
 42%|████▏     | 5023/12000 [2:59:22<3:53:35,  2.01s/it]
                                                        

 42%|████▏     | 5023/12000 [2:59:22<3:53:35,  2.01s/it]
 42%|████▏     | 5024/12000 [2:59:23<3:28:11,  1.79s/it]
                                                        

 42%|████▏     | 5024/12000 [2:59:23<3:28:11,  1.79s/it]
 42%|████▏     | 5025/12000 [2:59:25<3:25:01,  1.76s/it]
                                                        

 42%|████▏     | 5025/12000 [2:59:25<3:25:01,  1.76s/it]
 42%|████▏     | 5026/12000 [2:59:27<3:45:25,  1.94s/it]
                                                        

 42%|████▏     | 5026/12000 [2:59:27<3:45:25,  1.94s/it]
 42%|████▏     | 5027/12000 [2:59:29<3:25:29,  1.77s/it]
                                                        

 42%|████▏     | 5027/12000 [2:59:29<3:25:29,  1.77s/it]
 42%|████▏     | 5028/12000 [2:59:31<3:36:51,  1.87s/it]
                                                        

 42%|████▏     | 5028/12000 [2:59:31<3:36:51,  1.87s/it]
 42%|████▏     | 5029/12000 [2:59:32<3:18:38,  1.71s/it]
                                                        

 42%|████▏     | 5029/12000 [2:59:32<3:18:38,  1.71s/it]
 42%|████▏     | 5030/12000 [2:59:36<4:40:48,  2.42s/it]
                                                        

 42%|████▏     | 5030/12000 [2:59:36<4:40:48,  2.42s/it]
 42%|████▏     | 5031/12000 [2:59:39<5:11:05,  2.68s/it]
                                                        

 42%|████▏     | 5031/12000 [2:59:39<5:11:05,  2.68s/it]
 42%|████▏     | 5032/12000 [2:59:41<4:27:05,  2.30s/it]
                                                        

 42%|████▏     | 5032/12000 [2:59:41<4:27:05,  2.30s/it]
 42%|████▏     | 5033/12000 [2:59:43<4:26:39,  2.30s/it]
                                                        
{'loss': '1.016', 'grad_norm': '0.5423', 'learning_rate': '0.0001355', 'entropy': '1.068', 'num_tokens': '2.698e+07', 'mean_token_accuracy': '0.7457', 'epoch': '0.04413'}
{'loss': '0.7484', 'grad_norm': '0.3759', 'learning_rate': '0.0001354', 'entropy': '0.754', 'num_tokens': '2.698e+07', 'mean_token_accuracy': '0.8097', 'epoch': '0.04413'}
{'loss': '1.066', 'grad_norm': '0.4358', 'learning_rate': '0.0001354', 'entropy': '1.028', 'num_tokens': '2.699e+07', 'mean_token_accuracy': '0.7537', 'epoch': '0.04414'}
{'loss': '0.6075', 'grad_norm': '0.2467', 'learning_rate': '0.0001354', 'entropy': '0.7688', 'num_tokens': '2.699e+07', 'mean_token_accuracy': '0.8074', 'epoch': '0.04415'}
{'loss': '1.045', 'grad_norm': '0.3761', 'learning_rate': '0.0001353', 'entropy': '0.9258', 'num_tokens': '2.7e+07', 'mean_token_accuracy': '0.7878', 'epoch': '0.04416'}
{'loss': '0.8431', 'grad_norm': '0.33', 'learning_rate': '0.0001353', 'entropy': '0.8534', 'num_tokens': '2.701e+07', 'mean_token_accuracy': '0.7897', 'epoch': '0.04417'}
{'loss': '1.35', 'grad_norm': '0.4739', 'learning_rate': '0.0001353', 'entropy': '1.392', 'num_tokens': '2.702e+07', 'mean_token_accuracy': '0.686', 'epoch': '0.04418'}
{'loss': '1.057', 'grad_norm': '0.4389', 'learning_rate': '0.0001353', 'entropy': '1.05', 'num_tokens': '2.702e+07', 'mean_token_accuracy': '0.746', 'epoch': '0.04419'}
{'loss': '0.7635', 'grad_norm': '0.2938', 'learning_rate': '0.0001352', 'entropy': '0.7926', 'num_tokens': '2.703e+07', 'mean_token_accuracy': '0.7945', 'epoch': '0.0442'}
{'loss': '0.8449', 'grad_norm': '0.3423', 'learning_rate': '0.0001352', 'entropy': '0.9678', 'num_tokens': '2.703e+07', 'mean_token_accuracy': '0.7629', 'epoch': '0.0442'}
{'loss': '0.871', 'grad_norm': '0.4073', 'learning_rate': '0.0001352', 'entropy': '0.8785', 'num_tokens': '2.704e+07', 'mean_token_accuracy': '0.7871', 'epoch': '0.04421'}
{'loss': '0.7495', 'grad_norm': '0.3697', 'learning_rate': '0.0001352', 'entropy': '0.8248', 'num_tokens': '2.704e+07', 'mean_token_accuracy': '0.8045', 'epoch': '0.04422'}
{'loss': '0.9938', 'grad_norm': '0.4259', 'learning_rate': '0.0001351', 'entropy': '0.9934', 'num_tokens': '2.705e+07', 'mean_token_accuracy': '0.7601', 'epoch': '0.04423'}
{'loss': '0.5926', 'grad_norm': '0.354', 'learning_rate': '0.0001351', 'entropy': '0.5878', 'num_tokens': '2.705e+07', 'mean_token_accuracy': '0.8525', 'epoch': '0.04424'}
{'loss': '0.9901', 'grad_norm': '0.3914', 'learning_rate': '0.0001351', 'entropy': '1.049', 'num_tokens': '2.706e+07', 'mean_token_accuracy': '0.759', 'epoch': '0.04425'}
{'loss': '0.9351', 'grad_norm': '0.33', 'learning_rate': '0.0001351', 'entropy': '0.916', 'num_tokens': '2.707e+07', 'mean_token_accuracy': '0.7822', 'epoch': '0.04426'}
{'loss': '0.9868', 'grad_norm': '0.4063', 'learning_rate': '0.000135', 'entropy': '0.9556', 'num_tokens': '2.707e+07', 'mean_token_accuracy': '0.7514', 'epoch': '0.04427'}
{'loss': '0.8287', 'grad_norm': '0.3809', 'learning_rate': '0.000135', 'entropy': '0.8155', 'num_tokens': '2.707e+07', 'mean_token_accuracy': '0.7808', 'epoch': '0.04428'}
{'loss': '0.9037', 'grad_norm': '0.3857', 'learning_rate': '0.000135', 'entropy': '0.9323', 'num_tokens': '2.708e+07', 'mean_token_accuracy': '0.7643', 'epoch': '0.04428'}
{'loss': '0.8092', 'grad_norm': '0.343', 'learning_rate': '0.000135', 'entropy': '0.8013', 'num_tokens': '2.708e+07', 'mean_token_accuracy': '0.7861', 'epoch': '0.04429'}
{'loss': '1.008', 'grad_norm': '0.44', 'learning_rate': '0.0001349', 'entropy': '0.9101', 'num_tokens': '2.709e+07', 'mean_token_accuracy': '0.7778', 'epoch': '0.0443'}
{'loss': '0.7076', 'grad_norm': '0.3835', 'learning_rate': '0.0001349', 'entropy': '0.6914', 'num_tokens': '2.71e+07', 'mean_token_accuracy': '0.8204', 'epoch': '0.04431'}
{'loss': '0.7636', 'grad_norm': '0.361', 'learning_rate': '0.0001349', 'entropy': '0.7344', 'num_tokens': '2.71e+07', 'mean_token_accuracy': '0.7932', 'epoch': '0.04432'}
{'loss': '1.074', 'grad_norm': '0.4164', 'learning_rate': '0.0001349', 'entropy': '1.034', 'num_tokens': '2.71e+07', 'mean_token_accuracy': '0.7441', 'epoch': '0.04433'}
{'loss': '0.7142', 'grad_norm': '0.3899', 'learning_rate': '0.0001348', 'entropy': '0.7585', 'num_tokens': '2.711e+07', 'mean_token_accuracy': '0.8146', 'epoch': '0.04434'}
{'loss': '0.8847', 'grad_norm': '0.4243', 'learning_rate': '0.0001348', 'entropy': '0.9051', 'num_tokens': '2.711e+07', 'mean_token_accuracy': '0.7754', 'epoch': '0.04435'}
{'loss': '0.8266', 'grad_norm': '0.4169', 'learning_rate': '0.0001348', 'entropy': '0.8498', 'num_tokens': '2.712e+07', 'mean_token_accuracy': '0.7929', 'epoch': '0.04436'}
{'loss': '0.7709', 'grad_norm': '0.4369', 'learning_rate': '0.0001348', 'entropy': '0.728', 'num_tokens': '2.712e+07', 'mean_token_accuracy': '0.8163', 'epoch': '0.04436'}
{'loss': '0.8659', 'grad_norm': '0.4203', 'learning_rate': '0.0001347', 'entropy': '0.8311', 'num_tokens': '2.713e+07', 'mean_token_accuracy': '0.788', 'epoch': '0.04437'}
{'loss': '1.201', 'grad_norm': '0.4525', 'learning_rate': '0.0001347', 'entropy': '1.196', 'num_tokens': '2.713e+07', 'mean_token_accuracy': '0.7069', 'epoch': '0.04438'}
{'loss': '0.8666', 'grad_norm': '0.4109', 'learning_rate': '0.0001347', 'entropy': '0.7743', 'num_tokens': '2.714e+07', 'mean_token_accuracy': '0.803', 'epoch': '0.04439'}
{'loss': '1.199', 'grad_norm': '0.5096', 'learning_rate': '0.0001347', 'entropy': '1.147', 'num_tokens': '2.714e+07', 'mean_token_accuracy': '0.7281', 'epoch': '0.0444'}
{'loss': '0.7907', 'grad_norm': '0.3907', 'learning_rate': '0.0001346', 'entropy': '0.8372', 'num_tokens': '2.715e+07', 'mean_token_accuracy': '0.7927', 'epoch': '0.04441'}
{'loss': '0.8534', 'grad_norm': '0.3367', 'learning_rate': '0.0001346', 'entropy': '0.9774', 'num_tokens': '2.715e+07', 'mean_token_accuracy': '0.7611', 'epoch': '0.04442'}
{'loss': '0.6416', 'grad_norm': '0.4112', 'learning_rate': '0.0001346', 'entropy': '0.6938', 'num_tokens': '2.716e+07', 'mean_token_accuracy': '0.8216', 'epoch': '0.04443'}
{'loss': '0.7088', 'grad_norm': '0.3795', 'learning_rate': '0.0001345', 'entropy': '0.7603', 'num_tokens': '2.716e+07', 'mean_token_accuracy': '0.8046', 'epoch': '0.04444'}
{'loss': '0.6282', 'grad_norm': '0.434', 'learning_rate': '0.0001345', 'entropy': '0.6145', 'num_tokens': '2.717e+07', 'mean_token_accuracy': '0.8311', 'epoch': '0.04444'}
{'loss': '1.219', 'grad_norm': '0.5067', 'learning_rate': '0.0001345', 'entropy': '1.142', 'num_tokens': '2.717e+07', 'mean_token_accuracy': '0.722', 'epoch': '0.04445'}
{'loss': '0.9918', 'grad_norm': '0.4643', 'learning_rate': '0.0001345', 'entropy': '0.9549', 'num_tokens': '2.718e+07', 'mean_token_accuracy': '0.7724', 'epoch': '0.04446'}
{'loss': '0.8407', 'grad_norm': '0.4176', 'learning_rate': '0.0001344', 'entropy': '0.8994', 'num_tokens': '2.718e+07', 'mean_token_accuracy': '0.7743', 'epoch': '0.04447'}
{'loss': '1.226', 'grad_norm': '0.4724', 'learning_rate': '0.0001344', 'entropy': '1.074', 'num_tokens': '2.719e+07', 'mean_token_accuracy': '0.7392', 'epoch': '0.04448'}
{'loss': '0.7892', 'grad_norm': '0.3495', 'learning_rate': '0.0001344', 'entropy': '0.8216', 'num_tokens': '2.719e+07', 'mean_token_accuracy': '0.7875', 'epoch': '0.04449'}
{'loss': '1.17', 'grad_norm': '0.4289', 'learning_rate': '0.0001344', 'entropy': '1.18', 'num_tokens': '2.72e+07', 'mean_token_accuracy': '0.7241', 'epoch': '0.0445'}
{'loss': '0.8104', 'grad_norm': '0.3904', 'learning_rate': '0.0001343', 'entropy': '0.818', 'num_tokens': '2.72e+07', 'mean_token_accuracy': '0.7988', 'epoch': '0.04451'}
{'loss': '0.8696', 'grad_norm': '0.355', 'learning_rate': '0.0001343', 'entropy': '0.8459', 'num_tokens': '2.721e+07', 'mean_token_accuracy': '0.79', 'epoch': '0.04451'}
{'loss': '1.115', 'grad_norm': '0.3785', 'learning_rate': '0.0001343', 'entropy': '1.033', 'num_tokens': '2.722e+07', 'mean_token_accuracy': '0.7388', 'epoch': '0.04452'}
{'loss': '0.5552', 'grad_norm': '0.3003', 'learning_rate': '0.0001343', 'entropy': '0.6225', 'num_tokens': '2.722e+07', 'mean_token_accuracy': '0.8334', 'epoch': '0.04453'}

 42%|████▏     | 5033/12000 [2:59:43<4:26:39,  2.30s/it]
 42%|████▏     | 5034/12000 [2:59:45<4:10:00,  2.15s/it]
                                                        

 42%|████▏     | 5034/12000 [2:59:45<4:10:00,  2.15s/it]
 42%|████▏     | 5035/12000 [2:59:46<3:43:19,  1.92s/it]
                                                        

 42%|████▏     | 5035/12000 [2:59:46<3:43:19,  1.92s/it]
 42%|████▏     | 5036/12000 [2:59:48<3:41:34,  1.91s/it]
                                                        

 42%|████▏     | 5036/12000 [2:59:48<3:41:34,  1.91s/it]
 42%|████▏     | 5037/12000 [2:59:50<3:27:04,  1.78s/it]
                                                        

 42%|████▏     | 5037/12000 [2:59:50<3:27:04,  1.78s/it]
 42%|████▏     | 5038/12000 [2:59:51<3:19:56,  1.72s/it]
                                                        

 42%|████▏     | 5038/12000 [2:59:51<3:19:56,  1.72s/it]
 42%|████▏     | 5039/12000 [2:59:54<3:44:01,  1.93s/it]
                                                        

 42%|████▏     | 5039/12000 [2:59:54<3:44:01,  1.93s/it]
 42%|████▏     | 5040/12000 [2:59:56<4:00:11,  2.07s/it]
                                                        

 42%|████▏     | 5040/12000 [2:59:56<4:00:11,  2.07s/it]
 42%|████▏     | 5041/12000 [2:59:58<3:48:23,  1.97s/it]
                                                        

 42%|████▏     | 5041/12000 [2:59:58<3:48:23,  1.97s/it]
 42%|████▏     | 5042/12000 [2:59:59<3:36:57,  1.87s/it]
                                                        

 42%|████▏     | 5042/12000 [2:59:59<3:36:57,  1.87s/it]
 42%|████▏     | 5043/12000 [3:00:01<3:34:22,  1.85s/it]
                                                        

 42%|████▏     | 5043/12000 [3:00:01<3:34:22,  1.85s/it]
 42%|████▏     | 5044/12000 [3:00:03<3:21:38,  1.74s/it]
                                                        

 42%|████▏     | 5044/12000 [3:00:03<3:21:38,  1.74s/it]
 42%|████▏     | 5045/12000 [3:00:04<3:17:17,  1.70s/it]
                                                        

 42%|████▏     | 5045/12000 [3:00:04<3:17:17,  1.70s/it]
 42%|████▏     | 5046/12000 [3:00:06<3:15:39,  1.69s/it]
                                                        

 42%|████▏     | 5046/12000 [3:00:06<3:15:39,  1.69s/it]
 42%|████▏     | 5047/12000 [3:00:08<3:19:55,  1.73s/it]
                                                        

 42%|████▏     | 5047/12000 [3:00:08<3:19:55,  1.73s/it]
 42%|████▏     | 5048/12000 [3:00:10<3:42:58,  1.92s/it]
                                                        

 42%|████▏     | 5048/12000 [3:00:10<3:42:58,  1.92s/it]
 42%|████▏     | 5049/12000 [3:00:13<4:01:02,  2.08s/it]
                                                        

 42%|████▏     | 5049/12000 [3:00:13<4:01:02,  2.08s/it]
 42%|████▏     | 5050/12000 [3:00:14<3:46:12,  1.95s/it]
                                                        

 42%|████▏     | 5050/12000 [3:00:14<3:46:12,  1.95s/it]
 42%|████▏     | 5051/12000 [3:00:18<4:38:39,  2.41s/it]
                                                        

 42%|████▏     | 5051/12000 [3:00:18<4:38:39,  2.41s/it]
 42%|████▏     | 5052/12000 [3:00:24<6:50:16,  3.54s/it]
                                                        

 42%|████▏     | 5052/12000 [3:00:24<6:50:16,  3.54s/it]
 42%|████▏     | 5053/12000 [3:00:25<5:35:28,  2.90s/it]
                                                        

 42%|████▏     | 5053/12000 [3:00:25<5:35:28,  2.90s/it]
 42%|████▏     | 5054/12000 [3:00:27<4:55:31,  2.55s/it]
                                                        

 42%|████▏     | 5054/12000 [3:00:27<4:55:31,  2.55s/it]
 42%|████▏     | 5055/12000 [3:00:29<4:28:35,  2.32s/it]
                                                        

 42%|████▏     | 5055/12000 [3:00:29<4:28:35,  2.32s/it]
 42%|████▏     | 5056/12000 [3:00:31<4:21:01,  2.26s/it]
                                                        

 42%|████▏     | 5056/12000 [3:00:31<4:21:01,  2.26s/it]
 42%|████▏     | 5057/12000 [3:00:33<4:09:30,  2.16s/it]
                                                        

 42%|████▏     | 5057/12000 [3:00:33<4:09:30,  2.16s/it]
 42%|████▏     | 5058/12000 [3:00:35<4:02:31,  2.10s/it]
                                                        

 42%|████▏     | 5058/12000 [3:00:35<4:02:31,  2.10s/it]
 42%|████▏     | 5059/12000 [3:00:37<3:48:43,  1.98s/it]
                                                        

 42%|████▏     | 5059/12000 [3:00:37<3:48:43,  1.98s/it]
 42%|████▏     | 5060/12000 [3:00:39<4:06:13,  2.13s/it]
                                                        

 42%|████▏     | 5060/12000 [3:00:39<4:06:13,  2.13s/it]
 42%|████▏     | 5061/12000 [3:00:41<3:42:40,  1.93s/it]
                                                        

 42%|████▏     | 5061/12000 [3:00:41<3:42:40,  1.93s/it]
 42%|████▏     | 5062/12000 [3:00:42<3:39:11,  1.90s/it]
                                                        

 42%|████▏     | 5062/12000 [3:00:42<3:39:11,  1.90s/it]
 42%|████▏     | 5063/12000 [3:00:44<3:24:11,  1.77s/it]
                                                        

 42%|████▏     | 5063/12000 [3:00:44<3:24:11,  1.77s/it]
 42%|████▏     | 5064/12000 [3:00:45<3:07:48,  1.62s/it]
                                                        

 42%|████▏     | 5064/12000 [3:00:45<3:07:48,  1.62s/it]
 42%|████▏     | 5065/12000 [3:00:47<3:26:14,  1.78s/it]
                                                        

 42%|████▏     | 5065/12000 [3:00:47<3:26:14,  1.78s/it]
 42%|████▏     | 5066/12000 [3:00:49<3:29:19,  1.81s/it]
                                                        

 42%|████▏     | 5066/12000 [3:00:49<3:29:19,  1.81s/it]
 42%|████▏     | 5067/12000 [3:00:51<3:21:46,  1.75s/it]
                                                        

 42%|████▏     | 5067/12000 [3:00:51<3:21:46,  1.75s/it]
 42%|████▏     | 5068/12000 [3:00:52<3:22:53,  1.76s/it]
                                                        

 42%|████▏     | 5068/12000 [3:00:52<3:22:53,  1.76s/it]
 42%|████▏     | 5069/12000 [3:00:56<4:32:14,  2.36s/it]
                                                        

 42%|████▏     | 5069/12000 [3:00:56<4:32:14,  2.36s/it]
 42%|████▏     | 5070/12000 [3:00:58<3:57:29,  2.06s/it]
                                                        

 42%|████▏     | 5070/12000 [3:00:58<3:57:29,  2.06s/it]
 42%|████▏     | 5071/12000 [3:00:59<3:51:15,  2.00s/it]
                                                        

 42%|████▏     | 5071/12000 [3:00:59<3:51:15,  2.00s/it]
 42%|████▏     | 5072/12000 [3:01:01<3:30:41,  1.82s/it]
                                                        

 42%|████▏     | 5072/12000 [3:01:01<3:30:41,  1.82s/it]
 42%|████▏     | 5073/12000 [3:01:03<3:30:59,  1.83s/it]
                                                        

 42%|████▏     | 5073/12000 [3:01:03<3:30:59,  1.83s/it]
 42%|████▏     | 5074/12000 [3:01:06<4:13:57,  2.20s/it]
                                                        

 42%|████▏     | 5074/12000 [3:01:06<4:13:57,  2.20s/it]
 42%|████▏     | 5075/12000 [3:01:08<4:02:10,  2.10s/it]
                                                        

 42%|████▏     | 5075/12000 [3:01:08<4:02:10,  2.10s/it]
 42%|████▏     | 5076/12000 [3:01:10<4:23:19,  2.28s/it]
                                                        

 42%|████▏     | 5076/12000 [3:01:10<4:23:19,  2.28s/it]
 42%|████▏     | 5077/12000 [3:01:12<3:56:01,  2.05s/it]
                                                        

 42%|████▏     | 5077/12000 [3:01:12<3:56:01,  2.05s/it]
 42%|████▏     | 5078/12000 [3:01:14<3:49:08,  1.99s/it]
                                                        

 42%|████▏     | 5078/12000 [3:01:14<3:49:08,  1.99s/it]
 42%|████▏     | 5079/12000 [3:01:16<3:44:39,  1.95s/it]
                                                        

 42%|████▏     | 5079/12000 [3:01:16<3:44:39,  1.95s/it]
 42%|████▏     | 5080/12000 [3:01:17<3:36:43,  1.88s/it]
                                                        
{'loss': '0.8493', 'grad_norm': '0.3862', 'learning_rate': '0.0001342', 'entropy': '0.8133', 'num_tokens': '2.723e+07', 'mean_token_accuracy': '0.7896', 'epoch': '0.04454'}
{'loss': '0.8893', 'grad_norm': '0.4071', 'learning_rate': '0.0001342', 'entropy': '0.8475', 'num_tokens': '2.723e+07', 'mean_token_accuracy': '0.7845', 'epoch': '0.04455'}
{'loss': '0.8158', 'grad_norm': '0.4025', 'learning_rate': '0.0001342', 'entropy': '0.8305', 'num_tokens': '2.724e+07', 'mean_token_accuracy': '0.7907', 'epoch': '0.04456'}
{'loss': '1.085', 'grad_norm': '0.4299', 'learning_rate': '0.0001342', 'entropy': '1.031', 'num_tokens': '2.724e+07', 'mean_token_accuracy': '0.7575', 'epoch': '0.04457'}
{'loss': '0.7116', 'grad_norm': '0.3', 'learning_rate': '0.0001341', 'entropy': '0.7544', 'num_tokens': '2.725e+07', 'mean_token_accuracy': '0.8033', 'epoch': '0.04458'}
{'loss': '0.9212', 'grad_norm': '0.4972', 'learning_rate': '0.0001341', 'entropy': '0.9418', 'num_tokens': '2.725e+07', 'mean_token_accuracy': '0.7695', 'epoch': '0.04459'}
{'loss': '1.102', 'grad_norm': '0.3935', 'learning_rate': '0.0001341', 'entropy': '1.03', 'num_tokens': '2.726e+07', 'mean_token_accuracy': '0.748', 'epoch': '0.04459'}
{'loss': '0.5334', 'grad_norm': '0.2946', 'learning_rate': '0.0001341', 'entropy': '0.6007', 'num_tokens': '2.727e+07', 'mean_token_accuracy': '0.849', 'epoch': '0.0446'}
{'loss': '0.7276', 'grad_norm': '0.4476', 'learning_rate': '0.000134', 'entropy': '0.738', 'num_tokens': '2.727e+07', 'mean_token_accuracy': '0.8095', 'epoch': '0.04461'}
{'loss': '0.8174', 'grad_norm': '0.3612', 'learning_rate': '0.000134', 'entropy': '0.8026', 'num_tokens': '2.728e+07', 'mean_token_accuracy': '0.8021', 'epoch': '0.04462'}
{'loss': '1.056', 'grad_norm': '0.4024', 'learning_rate': '0.000134', 'entropy': '1.006', 'num_tokens': '2.728e+07', 'mean_token_accuracy': '0.7624', 'epoch': '0.04463'}
{'loss': '0.7454', 'grad_norm': '0.4199', 'learning_rate': '0.000134', 'entropy': '0.7298', 'num_tokens': '2.729e+07', 'mean_token_accuracy': '0.8264', 'epoch': '0.04464'}
{'loss': '0.9984', 'grad_norm': '0.3675', 'learning_rate': '0.0001339', 'entropy': '0.9915', 'num_tokens': '2.729e+07', 'mean_token_accuracy': '0.7544', 'epoch': '0.04465'}
{'loss': '1.098', 'grad_norm': '0.4683', 'learning_rate': '0.0001339', 'entropy': '1.054', 'num_tokens': '2.73e+07', 'mean_token_accuracy': '0.7357', 'epoch': '0.04466'}
{'loss': '0.8669', 'grad_norm': '0.3686', 'learning_rate': '0.0001339', 'entropy': '0.8727', 'num_tokens': '2.73e+07', 'mean_token_accuracy': '0.7862', 'epoch': '0.04467'}
{'loss': '0.9163', 'grad_norm': '0.3907', 'learning_rate': '0.0001338', 'entropy': '0.8672', 'num_tokens': '2.731e+07', 'mean_token_accuracy': '0.7787', 'epoch': '0.04467'}
{'loss': '1.289', 'grad_norm': '0.5238', 'learning_rate': '0.0001338', 'entropy': '1.205', 'num_tokens': '2.731e+07', 'mean_token_accuracy': '0.7071', 'epoch': '0.04468'}
{'loss': '0.6783', 'grad_norm': '0.3513', 'learning_rate': '0.0001338', 'entropy': '0.6474', 'num_tokens': '2.732e+07', 'mean_token_accuracy': '0.826', 'epoch': '0.04469'}
{'loss': '0.969', 'grad_norm': '0.3965', 'learning_rate': '0.0001338', 'entropy': '1.03', 'num_tokens': '2.732e+07', 'mean_token_accuracy': '0.7442', 'epoch': '0.0447'}
{'loss': '0.972', 'grad_norm': '0.4468', 'learning_rate': '0.0001337', 'entropy': '0.9861', 'num_tokens': '2.733e+07', 'mean_token_accuracy': '0.7652', 'epoch': '0.04471'}
{'loss': '1.165', 'grad_norm': '0.4339', 'learning_rate': '0.0001337', 'entropy': '1.134', 'num_tokens': '2.734e+07', 'mean_token_accuracy': '0.7245', 'epoch': '0.04472'}
{'loss': '0.5724', 'grad_norm': '0.3735', 'learning_rate': '0.0001337', 'entropy': '0.6158', 'num_tokens': '2.734e+07', 'mean_token_accuracy': '0.8443', 'epoch': '0.04473'}
{'loss': '1.364', 'grad_norm': '0.4534', 'learning_rate': '0.0001337', 'entropy': '1.242', 'num_tokens': '2.735e+07', 'mean_token_accuracy': '0.7044', 'epoch': '0.04474'}
{'loss': '0.9361', 'grad_norm': '0.376', 'learning_rate': '0.0001336', 'entropy': '1.029', 'num_tokens': '2.735e+07', 'mean_token_accuracy': '0.7429', 'epoch': '0.04474'}
{'loss': '1.021', 'grad_norm': '0.5229', 'learning_rate': '0.0001336', 'entropy': '1.053', 'num_tokens': '2.736e+07', 'mean_token_accuracy': '0.7375', 'epoch': '0.04475'}
{'loss': '0.8583', 'grad_norm': '0.3609', 'learning_rate': '0.0001336', 'entropy': '0.921', 'num_tokens': '2.737e+07', 'mean_token_accuracy': '0.7672', 'epoch': '0.04476'}
{'loss': '1.084', 'grad_norm': '0.4395', 'learning_rate': '0.0001336', 'entropy': '1.103', 'num_tokens': '2.737e+07', 'mean_token_accuracy': '0.7278', 'epoch': '0.04477'}
{'loss': '0.9413', 'grad_norm': '0.3937', 'learning_rate': '0.0001335', 'entropy': '0.9395', 'num_tokens': '2.738e+07', 'mean_token_accuracy': '0.7674', 'epoch': '0.04478'}
{'loss': '1.42', 'grad_norm': '0.5052', 'learning_rate': '0.0001335', 'entropy': '1.364', 'num_tokens': '2.738e+07', 'mean_token_accuracy': '0.6776', 'epoch': '0.04479'}
{'loss': '1.175', 'grad_norm': '0.4332', 'learning_rate': '0.0001335', 'entropy': '1.16', 'num_tokens': '2.739e+07', 'mean_token_accuracy': '0.7199', 'epoch': '0.0448'}
{'loss': '1.209', 'grad_norm': '0.4397', 'learning_rate': '0.0001335', 'entropy': '1.244', 'num_tokens': '2.739e+07', 'mean_token_accuracy': '0.708', 'epoch': '0.04481'}
{'loss': '0.7207', 'grad_norm': '0.3674', 'learning_rate': '0.0001334', 'entropy': '0.7545', 'num_tokens': '2.739e+07', 'mean_token_accuracy': '0.8117', 'epoch': '0.04482'}
{'loss': '1.345', 'grad_norm': '0.4116', 'learning_rate': '0.0001334', 'entropy': '1.339', 'num_tokens': '2.74e+07', 'mean_token_accuracy': '0.691', 'epoch': '0.04482'}
{'loss': '1.051', 'grad_norm': '0.4943', 'learning_rate': '0.0001334', 'entropy': '1.055', 'num_tokens': '2.74e+07', 'mean_token_accuracy': '0.7578', 'epoch': '0.04483'}
{'loss': '0.9395', 'grad_norm': '0.3856', 'learning_rate': '0.0001334', 'entropy': '0.9871', 'num_tokens': '2.741e+07', 'mean_token_accuracy': '0.7682', 'epoch': '0.04484'}
{'loss': '0.6746', 'grad_norm': '0.3841', 'learning_rate': '0.0001333', 'entropy': '0.6834', 'num_tokens': '2.741e+07', 'mean_token_accuracy': '0.8229', 'epoch': '0.04485'}
{'loss': '0.9247', 'grad_norm': '0.3664', 'learning_rate': '0.0001333', 'entropy': '0.9062', 'num_tokens': '2.742e+07', 'mean_token_accuracy': '0.7769', 'epoch': '0.04486'}
{'loss': '0.7162', 'grad_norm': '0.3216', 'learning_rate': '0.0001333', 'entropy': '0.7621', 'num_tokens': '2.743e+07', 'mean_token_accuracy': '0.8059', 'epoch': '0.04487'}
{'loss': '0.7033', 'grad_norm': '0.3369', 'learning_rate': '0.0001333', 'entropy': '0.6616', 'num_tokens': '2.743e+07', 'mean_token_accuracy': '0.8242', 'epoch': '0.04488'}
{'loss': '1.171', 'grad_norm': '0.4639', 'learning_rate': '0.0001332', 'entropy': '1.177', 'num_tokens': '2.744e+07', 'mean_token_accuracy': '0.7192', 'epoch': '0.04489'}
{'loss': '0.6089', 'grad_norm': '0.3543', 'learning_rate': '0.0001332', 'entropy': '0.5927', 'num_tokens': '2.744e+07', 'mean_token_accuracy': '0.8468', 'epoch': '0.0449'}
{'loss': '0.8911', 'grad_norm': '0.4687', 'learning_rate': '0.0001332', 'entropy': '0.8776', 'num_tokens': '2.745e+07', 'mean_token_accuracy': '0.7663', 'epoch': '0.0449'}
{'loss': '0.8613', 'grad_norm': '0.3709', 'learning_rate': '0.0001331', 'entropy': '0.8459', 'num_tokens': '2.745e+07', 'mean_token_accuracy': '0.7887', 'epoch': '0.04491'}
{'loss': '0.8474', 'grad_norm': '0.3973', 'learning_rate': '0.0001331', 'entropy': '0.7', 'num_tokens': '2.746e+07', 'mean_token_accuracy': '0.8133', 'epoch': '0.04492'}
{'loss': '0.759', 'grad_norm': '0.3989', 'learning_rate': '0.0001331', 'entropy': '0.8114', 'num_tokens': '2.746e+07', 'mean_token_accuracy': '0.7896', 'epoch': '0.04493'}
{'loss': '0.9932', 'grad_norm': '0.5029', 'learning_rate': '0.0001331', 'entropy': '0.9181', 'num_tokens': '2.747e+07', 'mean_token_accuracy': '0.7686', 'epoch': '0.04494'}
{'loss': '0.5845', 'grad_norm': '0.2779', 'learning_rate': '0.000133', 'entropy': '0.5801', 'num_tokens': '2.747e+07', 'mean_token_accuracy': '0.8572', 'epoch': '0.04495'}

 42%|████▏     | 5080/12000 [3:01:17<3:36:43,  1.88s/it]
 42%|████▏     | 5081/12000 [3:01:19<3:32:54,  1.85s/it]
                                                        

 42%|████▏     | 5081/12000 [3:01:19<3:32:54,  1.85s/it]
 42%|████▏     | 5082/12000 [3:01:21<3:23:46,  1.77s/it]
                                                        

 42%|████▏     | 5082/12000 [3:01:21<3:23:46,  1.77s/it]
 42%|████▏     | 5083/12000 [3:01:23<3:31:27,  1.83s/it]
                                                        

 42%|████▏     | 5083/12000 [3:01:23<3:31:27,  1.83s/it]
 42%|████▏     | 5084/12000 [3:01:24<3:18:08,  1.72s/it]
                                                        

 42%|████▏     | 5084/12000 [3:01:24<3:18:08,  1.72s/it]
 42%|████▏     | 5085/12000 [3:01:27<3:56:22,  2.05s/it]
                                                        

 42%|████▏     | 5085/12000 [3:01:27<3:56:22,  2.05s/it]
 42%|████▏     | 5086/12000 [3:01:28<3:31:00,  1.83s/it]
                                                        

 42%|████▏     | 5086/12000 [3:01:28<3:31:00,  1.83s/it]
 42%|████▏     | 5087/12000 [3:01:30<3:38:35,  1.90s/it]
                                                        

 42%|████▏     | 5087/12000 [3:01:30<3:38:35,  1.90s/it]
 42%|████▏     | 5088/12000 [3:01:33<4:05:34,  2.13s/it]
                                                        

 42%|████▏     | 5088/12000 [3:01:33<4:05:34,  2.13s/it]
 42%|████▏     | 5089/12000 [3:01:35<4:11:00,  2.18s/it]
                                                        

 42%|████▏     | 5089/12000 [3:01:35<4:11:00,  2.18s/it]
 42%|████▏     | 5090/12000 [3:01:37<4:00:17,  2.09s/it]
                                                        

 42%|████▏     | 5090/12000 [3:01:37<4:00:17,  2.09s/it]
 42%|████▏     | 5091/12000 [3:01:39<3:42:07,  1.93s/it]
                                                        

 42%|████▏     | 5091/12000 [3:01:39<3:42:07,  1.93s/it]
 42%|████▏     | 5092/12000 [3:01:40<3:24:10,  1.77s/it]
                                                        

 42%|████▏     | 5092/12000 [3:01:40<3:24:10,  1.77s/it]
 42%|████▏     | 5093/12000 [3:01:42<3:13:10,  1.68s/it]
                                                        

 42%|████▏     | 5093/12000 [3:01:42<3:13:10,  1.68s/it]
 42%|████▏     | 5094/12000 [3:01:43<3:10:06,  1.65s/it]
                                                        

 42%|████▏     | 5094/12000 [3:01:43<3:10:06,  1.65s/it]
 42%|████▏     | 5095/12000 [3:01:45<3:00:31,  1.57s/it]
                                                        

 42%|████▏     | 5095/12000 [3:01:45<3:00:31,  1.57s/it]
 42%|████▏     | 5096/12000 [3:01:48<4:00:16,  2.09s/it]
                                                        

 42%|████▏     | 5096/12000 [3:01:48<4:00:16,  2.09s/it]
 42%|████▏     | 5097/12000 [3:01:50<3:46:42,  1.97s/it]
                                                        

 42%|████▏     | 5097/12000 [3:01:50<3:46:42,  1.97s/it]
 42%|████▏     | 5098/12000 [3:01:51<3:32:47,  1.85s/it]
                                                        

 42%|████▏     | 5098/12000 [3:01:51<3:32:47,  1.85s/it]
 42%|████▏     | 5099/12000 [3:01:52<3:15:16,  1.70s/it]
                                                        

 42%|████▏     | 5099/12000 [3:01:52<3:15:16,  1.70s/it]
 42%|████▎     | 5100/12000 [3:01:54<3:20:12,  1.74s/it]
                                                        

 42%|████▎     | 5100/12000 [3:01:54<3:20:12,  1.74s/it]
 43%|████▎     | 5101/12000 [3:01:57<4:01:35,  2.10s/it]
                                                        

 43%|████▎     | 5101/12000 [3:01:57<4:01:35,  2.10s/it]
 43%|████▎     | 5102/12000 [3:02:04<6:34:30,  3.43s/it]
                                                        

 43%|████▎     | 5102/12000 [3:02:04<6:34:30,  3.43s/it]
 43%|████▎     | 5103/12000 [3:02:06<5:38:00,  2.94s/it]
                                                        

 43%|████▎     | 5103/12000 [3:02:06<5:38:00,  2.94s/it]
 43%|████▎     | 5104/12000 [3:02:08<5:15:31,  2.75s/it]
                                                        

 43%|████▎     | 5104/12000 [3:02:08<5:15:31,  2.75s/it]
 43%|████▎     | 5105/12000 [3:02:09<4:34:57,  2.39s/it]
                                                        

 43%|████▎     | 5105/12000 [3:02:09<4:34:57,  2.39s/it]
 43%|████▎     | 5106/12000 [3:02:11<4:01:41,  2.10s/it]
                                                        

 43%|████▎     | 5106/12000 [3:02:11<4:01:41,  2.10s/it]
 43%|████▎     | 5107/12000 [3:02:13<3:49:49,  2.00s/it]
                                                        

 43%|████▎     | 5107/12000 [3:02:13<3:49:49,  2.00s/it]
 43%|████▎     | 5108/12000 [3:02:14<3:37:08,  1.89s/it]
                                                        

 43%|████▎     | 5108/12000 [3:02:14<3:37:08,  1.89s/it]
 43%|████▎     | 5109/12000 [3:02:16<3:33:06,  1.86s/it]
                                                        

 43%|████▎     | 5109/12000 [3:02:16<3:33:06,  1.86s/it]
 43%|████▎     | 5110/12000 [3:02:18<3:42:44,  1.94s/it]
                                                        

 43%|████▎     | 5110/12000 [3:02:18<3:42:44,  1.94s/it]
 43%|████▎     | 5111/12000 [3:02:20<3:29:16,  1.82s/it]
                                                        

 43%|████▎     | 5111/12000 [3:02:20<3:29:16,  1.82s/it]
 43%|████▎     | 5112/12000 [3:02:21<3:22:56,  1.77s/it]
                                                        

 43%|████▎     | 5112/12000 [3:02:21<3:22:56,  1.77s/it]
 43%|████▎     | 5113/12000 [3:02:23<3:11:35,  1.67s/it]
                                                        

 43%|████▎     | 5113/12000 [3:02:23<3:11:35,  1.67s/it]
 43%|████▎     | 5114/12000 [3:02:24<3:01:17,  1.58s/it]
                                                        

 43%|████▎     | 5114/12000 [3:02:24<3:01:17,  1.58s/it]
 43%|████▎     | 5115/12000 [3:02:26<3:09:13,  1.65s/it]
                                                        

 43%|████▎     | 5115/12000 [3:02:26<3:09:13,  1.65s/it]
 43%|████▎     | 5116/12000 [3:02:27<3:04:28,  1.61s/it]
                                                        

 43%|████▎     | 5116/12000 [3:02:27<3:04:28,  1.61s/it]
 43%|████▎     | 5117/12000 [3:02:30<3:41:59,  1.94s/it]
                                                        

 43%|████▎     | 5117/12000 [3:02:30<3:41:59,  1.94s/it]
 43%|████▎     | 5118/12000 [3:02:33<4:22:58,  2.29s/it]
                                                        

 43%|████▎     | 5118/12000 [3:02:33<4:22:58,  2.29s/it]
 43%|████▎     | 5119/12000 [3:02:35<4:09:10,  2.17s/it]
                                                        

 43%|████▎     | 5119/12000 [3:02:35<4:09:10,  2.17s/it]
 43%|████▎     | 5120/12000 [3:02:37<3:53:56,  2.04s/it]
                                                        

 43%|████▎     | 5120/12000 [3:02:37<3:53:56,  2.04s/it]
 43%|████▎     | 5121/12000 [3:02:38<3:32:31,  1.85s/it]
                                                        

 43%|████▎     | 5121/12000 [3:02:38<3:32:31,  1.85s/it]
 43%|████▎     | 5122/12000 [3:02:41<4:03:32,  2.12s/it]
                                                        

 43%|████▎     | 5122/12000 [3:02:41<4:03:32,  2.12s/it]
 43%|████▎     | 5123/12000 [3:02:43<4:02:13,  2.11s/it]
                                                        

 43%|████▎     | 5123/12000 [3:02:43<4:02:13,  2.11s/it]
 43%|████▎     | 5124/12000 [3:02:45<3:39:39,  1.92s/it]
                                                        

 43%|████▎     | 5124/12000 [3:02:45<3:39:39,  1.92s/it]
 43%|████▎     | 5125/12000 [3:02:46<3:34:39,  1.87s/it]
                                                        

 43%|████▎     | 5125/12000 [3:02:46<3:34:39,  1.87s/it]
 43%|████▎     | 5126/12000 [3:02:48<3:31:38,  1.85s/it]
                                                        

 43%|████▎     | 5126/12000 [3:02:48<3:31:38,  1.85s/it]
 43%|████▎     | 5127/12000 [3:02:50<3:29:57,  1.83s/it]
                                                        
{'loss': '0.8977', 'grad_norm': '0.4022', 'learning_rate': '0.000133', 'entropy': '0.8965', 'num_tokens': '2.748e+07', 'mean_token_accuracy': '0.7965', 'epoch': '0.04496'}
{'loss': '0.6202', 'grad_norm': '0.3887', 'learning_rate': '0.000133', 'entropy': '0.5891', 'num_tokens': '2.749e+07', 'mean_token_accuracy': '0.8478', 'epoch': '0.04497'}
{'loss': '0.8374', 'grad_norm': '0.3985', 'learning_rate': '0.000133', 'entropy': '0.8201', 'num_tokens': '2.749e+07', 'mean_token_accuracy': '0.7932', 'epoch': '0.04497'}
{'loss': '0.9229', 'grad_norm': '0.3617', 'learning_rate': '0.0001329', 'entropy': '0.9206', 'num_tokens': '2.75e+07', 'mean_token_accuracy': '0.7756', 'epoch': '0.04498'}
{'loss': '1.025', 'grad_norm': '0.4562', 'learning_rate': '0.0001329', 'entropy': '1.034', 'num_tokens': '2.75e+07', 'mean_token_accuracy': '0.743', 'epoch': '0.04499'}
{'loss': '0.8261', 'grad_norm': '0.351', 'learning_rate': '0.0001329', 'entropy': '0.8523', 'num_tokens': '2.751e+07', 'mean_token_accuracy': '0.7907', 'epoch': '0.045'}
{'loss': '0.8971', 'grad_norm': '0.433', 'learning_rate': '0.0001329', 'entropy': '0.9448', 'num_tokens': '2.751e+07', 'mean_token_accuracy': '0.7686', 'epoch': '0.04501'}
{'loss': '0.738', 'grad_norm': '0.33', 'learning_rate': '0.0001328', 'entropy': '0.7848', 'num_tokens': '2.751e+07', 'mean_token_accuracy': '0.7926', 'epoch': '0.04502'}
{'loss': '0.8698', 'grad_norm': '0.3754', 'learning_rate': '0.0001328', 'entropy': '0.8857', 'num_tokens': '2.752e+07', 'mean_token_accuracy': '0.7988', 'epoch': '0.04503'}
{'loss': '0.915', 'grad_norm': '0.415', 'learning_rate': '0.0001328', 'entropy': '0.8354', 'num_tokens': '2.753e+07', 'mean_token_accuracy': '0.7861', 'epoch': '0.04504'}
{'loss': '0.7321', 'grad_norm': '0.3497', 'learning_rate': '0.0001328', 'entropy': '0.8628', 'num_tokens': '2.753e+07', 'mean_token_accuracy': '0.7738', 'epoch': '0.04505'}
{'loss': '0.6057', 'grad_norm': '0.3924', 'learning_rate': '0.0001327', 'entropy': '0.6883', 'num_tokens': '2.754e+07', 'mean_token_accuracy': '0.8173', 'epoch': '0.04505'}
{'loss': '0.7309', 'grad_norm': '0.3813', 'learning_rate': '0.0001327', 'entropy': '0.757', 'num_tokens': '2.754e+07', 'mean_token_accuracy': '0.8043', 'epoch': '0.04506'}
{'loss': '0.7226', 'grad_norm': '0.4278', 'learning_rate': '0.0001327', 'entropy': '0.7322', 'num_tokens': '2.754e+07', 'mean_token_accuracy': '0.8059', 'epoch': '0.04507'}
{'loss': '1.098', 'grad_norm': '0.3917', 'learning_rate': '0.0001327', 'entropy': '1.07', 'num_tokens': '2.755e+07', 'mean_token_accuracy': '0.7497', 'epoch': '0.04508'}
{'loss': '0.7406', 'grad_norm': '0.3341', 'learning_rate': '0.0001326', 'entropy': '0.769', 'num_tokens': '2.755e+07', 'mean_token_accuracy': '0.7974', 'epoch': '0.04509'}
{'loss': '0.6902', 'grad_norm': '0.3709', 'learning_rate': '0.0001326', 'entropy': '0.7138', 'num_tokens': '2.756e+07', 'mean_token_accuracy': '0.8161', 'epoch': '0.0451'}
{'loss': '1.022', 'grad_norm': '0.3823', 'learning_rate': '0.0001326', 'entropy': '1.003', 'num_tokens': '2.756e+07', 'mean_token_accuracy': '0.7404', 'epoch': '0.04511'}
{'loss': '0.6578', 'grad_norm': '0.3454', 'learning_rate': '0.0001325', 'entropy': '0.6938', 'num_tokens': '2.757e+07', 'mean_token_accuracy': '0.8185', 'epoch': '0.04512'}
{'loss': '0.8658', 'grad_norm': '0.3883', 'learning_rate': '0.0001325', 'entropy': '0.8709', 'num_tokens': '2.757e+07', 'mean_token_accuracy': '0.7813', 'epoch': '0.04513'}
{'loss': '0.7193', 'grad_norm': '0.3933', 'learning_rate': '0.0001325', 'entropy': '0.7161', 'num_tokens': '2.758e+07', 'mean_token_accuracy': '0.8144', 'epoch': '0.04513'}
{'loss': '0.7476', 'grad_norm': '0.3718', 'learning_rate': '0.0001325', 'entropy': '0.7345', 'num_tokens': '2.758e+07', 'mean_token_accuracy': '0.819', 'epoch': '0.04514'}
{'loss': '1.296', 'grad_norm': '0.3306', 'learning_rate': '0.0001324', 'entropy': '1.306', 'num_tokens': '2.76e+07', 'mean_token_accuracy': '0.6993', 'epoch': '0.04515'}
{'loss': '0.9601', 'grad_norm': '0.3579', 'learning_rate': '0.0001324', 'entropy': '0.7765', 'num_tokens': '2.76e+07', 'mean_token_accuracy': '0.8099', 'epoch': '0.04516'}
{'loss': '1.177', 'grad_norm': '0.4451', 'learning_rate': '0.0001324', 'entropy': '1.072', 'num_tokens': '2.761e+07', 'mean_token_accuracy': '0.7404', 'epoch': '0.04517'}
{'loss': '0.7235', 'grad_norm': '0.3419', 'learning_rate': '0.0001324', 'entropy': '0.6965', 'num_tokens': '2.761e+07', 'mean_token_accuracy': '0.8207', 'epoch': '0.04518'}
{'loss': '0.76', 'grad_norm': '0.4028', 'learning_rate': '0.0001323', 'entropy': '0.8023', 'num_tokens': '2.762e+07', 'mean_token_accuracy': '0.8009', 'epoch': '0.04519'}
{'loss': '1.074', 'grad_norm': '0.404', 'learning_rate': '0.0001323', 'entropy': '1.037', 'num_tokens': '2.762e+07', 'mean_token_accuracy': '0.7512', 'epoch': '0.0452'}
{'loss': '1.391', 'grad_norm': '0.4718', 'learning_rate': '0.0001323', 'entropy': '1.147', 'num_tokens': '2.763e+07', 'mean_token_accuracy': '0.7204', 'epoch': '0.0452'}
{'loss': '1.089', 'grad_norm': '0.4382', 'learning_rate': '0.0001323', 'entropy': '1.014', 'num_tokens': '2.763e+07', 'mean_token_accuracy': '0.7513', 'epoch': '0.04521'}
{'loss': '1.12', 'grad_norm': '0.3962', 'learning_rate': '0.0001322', 'entropy': '1.174', 'num_tokens': '2.764e+07', 'mean_token_accuracy': '0.7131', 'epoch': '0.04522'}
{'loss': '0.8854', 'grad_norm': '0.4156', 'learning_rate': '0.0001322', 'entropy': '0.893', 'num_tokens': '2.764e+07', 'mean_token_accuracy': '0.7736', 'epoch': '0.04523'}
{'loss': '0.7924', 'grad_norm': '0.3826', 'learning_rate': '0.0001322', 'entropy': '0.7579', 'num_tokens': '2.765e+07', 'mean_token_accuracy': '0.8067', 'epoch': '0.04524'}
{'loss': '0.9943', 'grad_norm': '0.469', 'learning_rate': '0.0001322', 'entropy': '1.056', 'num_tokens': '2.765e+07', 'mean_token_accuracy': '0.7521', 'epoch': '0.04525'}
{'loss': '0.9516', 'grad_norm': '0.402', 'learning_rate': '0.0001321', 'entropy': '0.9751', 'num_tokens': '2.766e+07', 'mean_token_accuracy': '0.7615', 'epoch': '0.04526'}
{'loss': '0.7052', 'grad_norm': '0.3585', 'learning_rate': '0.0001321', 'entropy': '0.7328', 'num_tokens': '2.766e+07', 'mean_token_accuracy': '0.8184', 'epoch': '0.04527'}
{'loss': '1.045', 'grad_norm': '0.4523', 'learning_rate': '0.0001321', 'entropy': '1.089', 'num_tokens': '2.767e+07', 'mean_token_accuracy': '0.7379', 'epoch': '0.04528'}
{'loss': '0.9056', 'grad_norm': '0.3709', 'learning_rate': '0.0001321', 'entropy': '1.037', 'num_tokens': '2.767e+07', 'mean_token_accuracy': '0.7392', 'epoch': '0.04528'}
{'loss': '0.6978', 'grad_norm': '0.3562', 'learning_rate': '0.000132', 'entropy': '0.7547', 'num_tokens': '2.768e+07', 'mean_token_accuracy': '0.8044', 'epoch': '0.04529'}
{'loss': '0.8089', 'grad_norm': '0.442', 'learning_rate': '0.000132', 'entropy': '0.8797', 'num_tokens': '2.768e+07', 'mean_token_accuracy': '0.7785', 'epoch': '0.0453'}
{'loss': '0.6885', 'grad_norm': '0.3218', 'learning_rate': '0.000132', 'entropy': '0.6872', 'num_tokens': '2.769e+07', 'mean_token_accuracy': '0.8301', 'epoch': '0.04531'}
{'loss': '0.5293', 'grad_norm': '0.3237', 'learning_rate': '0.0001319', 'entropy': '0.5251', 'num_tokens': '2.769e+07', 'mean_token_accuracy': '0.8656', 'epoch': '0.04532'}
{'loss': '0.7752', 'grad_norm': '0.3595', 'learning_rate': '0.0001319', 'entropy': '0.7801', 'num_tokens': '2.77e+07', 'mean_token_accuracy': '0.7867', 'epoch': '0.04533'}
{'loss': '0.7833', 'grad_norm': '0.3123', 'learning_rate': '0.0001319', 'entropy': '0.7934', 'num_tokens': '2.771e+07', 'mean_token_accuracy': '0.7961', 'epoch': '0.04534'}
{'loss': '0.8888', 'grad_norm': '0.3647', 'learning_rate': '0.0001319', 'entropy': '0.8533', 'num_tokens': '2.771e+07', 'mean_token_accuracy': '0.7843', 'epoch': '0.04535'}
{'loss': '0.7077', 'grad_norm': '0.3741', 'learning_rate': '0.0001318', 'entropy': '0.6649', 'num_tokens': '2.772e+07', 'mean_token_accuracy': '0.827', 'epoch': '0.04536'}
{'loss': '0.6194', 'grad_norm': '0.3593', 'learning_rate': '0.0001318', 'entropy': '0.6284', 'num_tokens': '2.772e+07', 'mean_token_accuracy': '0.8344', 'epoch': '0.04536'}

 43%|████▎     | 5127/12000 [3:02:50<3:29:57,  1.83s/it]
 43%|████▎     | 5128/12000 [3:02:51<3:18:41,  1.73s/it]
                                                        

 43%|████▎     | 5128/12000 [3:02:51<3:18:41,  1.73s/it]
 43%|████▎     | 5129/12000 [3:02:54<3:36:32,  1.89s/it]
                                                        

 43%|████▎     | 5129/12000 [3:02:54<3:36:32,  1.89s/it]
 43%|████▎     | 5130/12000 [3:02:55<3:31:50,  1.85s/it]
                                                        

 43%|████▎     | 5130/12000 [3:02:55<3:31:50,  1.85s/it]
 43%|████▎     | 5131/12000 [3:02:59<4:30:54,  2.37s/it]
                                                        

 43%|████▎     | 5131/12000 [3:02:59<4:30:54,  2.37s/it]
 43%|████▎     | 5132/12000 [3:03:00<3:57:53,  2.08s/it]
                                                        

 43%|████▎     | 5132/12000 [3:03:00<3:57:53,  2.08s/it]
 43%|████▎     | 5133/12000 [3:03:02<3:34:47,  1.88s/it]
                                                        

 43%|████▎     | 5133/12000 [3:03:02<3:34:47,  1.88s/it]
 43%|████▎     | 5134/12000 [3:03:04<3:32:41,  1.86s/it]
                                                        

 43%|████▎     | 5134/12000 [3:03:04<3:32:41,  1.86s/it]
 43%|████▎     | 5135/12000 [3:03:06<3:32:38,  1.86s/it]
                                                        

 43%|████▎     | 5135/12000 [3:03:06<3:32:38,  1.86s/it]
 43%|████▎     | 5136/12000 [3:03:07<3:23:05,  1.78s/it]
                                                        

 43%|████▎     | 5136/12000 [3:03:07<3:23:05,  1.78s/it]
 43%|████▎     | 5137/12000 [3:03:09<3:13:12,  1.69s/it]
                                                        

 43%|████▎     | 5137/12000 [3:03:09<3:13:12,  1.69s/it]
 43%|████▎     | 5138/12000 [3:03:11<3:36:32,  1.89s/it]
                                                        

 43%|████▎     | 5138/12000 [3:03:11<3:36:32,  1.89s/it]
 43%|████▎     | 5139/12000 [3:03:13<3:55:22,  2.06s/it]
                                                        

 43%|████▎     | 5139/12000 [3:03:13<3:55:22,  2.06s/it]
 43%|████▎     | 5140/12000 [3:03:18<5:21:49,  2.81s/it]
                                                        

 43%|████▎     | 5140/12000 [3:03:18<5:21:49,  2.81s/it]
 43%|████▎     | 5141/12000 [3:03:19<4:34:57,  2.41s/it]
                                                        

 43%|████▎     | 5141/12000 [3:03:19<4:34:57,  2.41s/it]
 43%|████▎     | 5142/12000 [3:03:21<4:05:39,  2.15s/it]
                                                        

 43%|████▎     | 5142/12000 [3:03:21<4:05:39,  2.15s/it]
 43%|████▎     | 5143/12000 [3:03:23<3:47:40,  1.99s/it]
                                                        

 43%|████▎     | 5143/12000 [3:03:23<3:47:40,  1.99s/it]
 43%|████▎     | 5144/12000 [3:03:24<3:31:45,  1.85s/it]
                                                        

 43%|████▎     | 5144/12000 [3:03:24<3:31:45,  1.85s/it]
 43%|████▎     | 5145/12000 [3:03:27<4:14:10,  2.22s/it]
                                                        

 43%|████▎     | 5145/12000 [3:03:27<4:14:10,  2.22s/it]
 43%|████▎     | 5146/12000 [3:03:29<3:57:27,  2.08s/it]
                                                        

 43%|████▎     | 5146/12000 [3:03:29<3:57:27,  2.08s/it]
 43%|████▎     | 5147/12000 [3:03:34<5:37:28,  2.95s/it]
                                                        

 43%|████▎     | 5147/12000 [3:03:34<5:37:28,  2.95s/it]
 43%|████▎     | 5148/12000 [3:03:36<5:13:16,  2.74s/it]
                                                        

 43%|████▎     | 5148/12000 [3:03:36<5:13:16,  2.74s/it]
 43%|████▎     | 5149/12000 [3:03:39<5:27:01,  2.86s/it]
                                                        

 43%|████▎     | 5149/12000 [3:03:39<5:27:01,  2.86s/it]
 43%|████▎     | 5150/12000 [3:03:42<5:12:32,  2.74s/it]
                                                        

 43%|████▎     | 5150/12000 [3:03:42<5:12:32,  2.74s/it]
 43%|████▎     | 5151/12000 [3:03:46<6:01:21,  3.17s/it]
                                                        

 43%|████▎     | 5151/12000 [3:03:46<6:01:21,  3.17s/it]
 43%|████▎     | 5152/12000 [3:03:48<5:07:46,  2.70s/it]
                                                        

 43%|████▎     | 5152/12000 [3:03:48<5:07:46,  2.70s/it]
 43%|████▎     | 5153/12000 [3:03:49<4:25:11,  2.32s/it]
                                                        

 43%|████▎     | 5153/12000 [3:03:49<4:25:11,  2.32s/it]
 43%|████▎     | 5154/12000 [3:03:53<5:24:26,  2.84s/it]
                                                        

 43%|████▎     | 5154/12000 [3:03:53<5:24:26,  2.84s/it]
 43%|████▎     | 5155/12000 [3:03:56<5:33:59,  2.93s/it]
                                                        

 43%|████▎     | 5155/12000 [3:03:56<5:33:59,  2.93s/it]
 43%|████▎     | 5156/12000 [3:03:58<4:48:15,  2.53s/it]
                                                        

 43%|████▎     | 5156/12000 [3:03:58<4:48:15,  2.53s/it]
 43%|████▎     | 5157/12000 [3:04:00<4:19:39,  2.28s/it]
                                                        

 43%|████▎     | 5157/12000 [3:04:00<4:19:39,  2.28s/it]
 43%|████▎     | 5158/12000 [3:04:01<4:01:09,  2.11s/it]
                                                        

 43%|████▎     | 5158/12000 [3:04:01<4:01:09,  2.11s/it]
 43%|████▎     | 5159/12000 [3:04:03<3:39:12,  1.92s/it]
                                                        

 43%|████▎     | 5159/12000 [3:04:03<3:39:12,  1.92s/it]
 43%|████▎     | 5160/12000 [3:04:04<3:24:59,  1.80s/it]
                                                        

 43%|████▎     | 5160/12000 [3:04:04<3:24:59,  1.80s/it]
 43%|████▎     | 5161/12000 [3:04:06<3:24:02,  1.79s/it]
                                                        

 43%|████▎     | 5161/12000 [3:04:06<3:24:02,  1.79s/it]
 43%|████▎     | 5162/12000 [3:04:09<4:16:55,  2.25s/it]
                                                        

 43%|████▎     | 5162/12000 [3:04:09<4:16:55,  2.25s/it]
 43%|████▎     | 5163/12000 [3:04:11<3:48:02,  2.00s/it]
                                                        

 43%|████▎     | 5163/12000 [3:04:11<3:48:02,  2.00s/it]
 43%|████▎     | 5164/12000 [3:04:13<4:05:27,  2.15s/it]
                                                        

 43%|████▎     | 5164/12000 [3:04:13<4:05:27,  2.15s/it]
 43%|████▎     | 5165/12000 [3:04:15<3:54:39,  2.06s/it]
                                                        

 43%|████▎     | 5165/12000 [3:04:15<3:54:39,  2.06s/it]
 43%|████▎     | 5166/12000 [3:04:17<3:37:14,  1.91s/it]
                                                        

 43%|████▎     | 5166/12000 [3:04:17<3:37:14,  1.91s/it]
 43%|████▎     | 5167/12000 [3:04:19<3:59:57,  2.11s/it]
                                                        

 43%|████▎     | 5167/12000 [3:04:19<3:59:57,  2.11s/it]
 43%|████▎     | 5168/12000 [3:04:21<3:52:36,  2.04s/it]
                                                        

 43%|████▎     | 5168/12000 [3:04:21<3:52:36,  2.04s/it]
 43%|████▎     | 5169/12000 [3:04:27<6:02:59,  3.19s/it]
                                                        

 43%|████▎     | 5169/12000 [3:04:27<6:02:59,  3.19s/it]
 43%|████▎     | 5170/12000 [3:04:29<5:07:25,  2.70s/it]
                                                        

 43%|████▎     | 5170/12000 [3:04:29<5:07:25,  2.70s/it]
 43%|████▎     | 5171/12000 [3:04:31<4:57:09,  2.61s/it]
                                                        

 43%|████▎     | 5171/12000 [3:04:31<4:57:09,  2.61s/it]
 43%|████▎     | 5172/12000 [3:04:33<4:21:06,  2.29s/it]
                                                        

 43%|████▎     | 5172/12000 [3:04:33<4:21:06,  2.29s/it]
 43%|████▎     | 5173/12000 [3:04:34<3:56:47,  2.08s/it]
                                                        

 43%|████▎     | 5173/12000 [3:04:34<3:56:47,  2.08s/it]
 43%|████▎     | 5174/12000 [3:04:36<3:50:23,  2.03s/it]
                                                        
{'loss': '0.7043', 'grad_norm': '0.3902', 'learning_rate': '0.0001318', 'entropy': '0.7011', 'num_tokens': '2.773e+07', 'mean_token_accuracy': '0.8084', 'epoch': '0.04537'}
{'loss': '0.8312', 'grad_norm': '0.4244', 'learning_rate': '0.0001318', 'entropy': '0.8982', 'num_tokens': '2.773e+07', 'mean_token_accuracy': '0.7721', 'epoch': '0.04538'}
{'loss': '0.9804', 'grad_norm': '0.4606', 'learning_rate': '0.0001317', 'entropy': '0.7795', 'num_tokens': '2.774e+07', 'mean_token_accuracy': '0.794', 'epoch': '0.04539'}
{'loss': '0.7651', 'grad_norm': '0.3949', 'learning_rate': '0.0001317', 'entropy': '0.7998', 'num_tokens': '2.774e+07', 'mean_token_accuracy': '0.7867', 'epoch': '0.0454'}
{'loss': '1.125', 'grad_norm': '0.3441', 'learning_rate': '0.0001317', 'entropy': '0.8865', 'num_tokens': '2.775e+07', 'mean_token_accuracy': '0.7785', 'epoch': '0.04541'}
{'loss': '0.899', 'grad_norm': '0.3901', 'learning_rate': '0.0001317', 'entropy': '0.875', 'num_tokens': '2.776e+07', 'mean_token_accuracy': '0.7738', 'epoch': '0.04542'}
{'loss': '0.7287', 'grad_norm': '0.3028', 'learning_rate': '0.0001316', 'entropy': '0.8297', 'num_tokens': '2.776e+07', 'mean_token_accuracy': '0.7886', 'epoch': '0.04543'}
{'loss': '0.9636', 'grad_norm': '0.4144', 'learning_rate': '0.0001316', 'entropy': '0.9196', 'num_tokens': '2.777e+07', 'mean_token_accuracy': '0.7812', 'epoch': '0.04544'}
{'loss': '0.8704', 'grad_norm': '0.4003', 'learning_rate': '0.0001316', 'entropy': '0.8807', 'num_tokens': '2.777e+07', 'mean_token_accuracy': '0.7953', 'epoch': '0.04544'}
{'loss': '1.254', 'grad_norm': '0.5421', 'learning_rate': '0.0001316', 'entropy': '1.199', 'num_tokens': '2.778e+07', 'mean_token_accuracy': '0.7188', 'epoch': '0.04545'}
{'loss': '0.6167', 'grad_norm': '0.3324', 'learning_rate': '0.0001315', 'entropy': '0.6739', 'num_tokens': '2.778e+07', 'mean_token_accuracy': '0.8313', 'epoch': '0.04546'}
{'loss': '0.8477', 'grad_norm': '0.466', 'learning_rate': '0.0001315', 'entropy': '0.9021', 'num_tokens': '2.779e+07', 'mean_token_accuracy': '0.7638', 'epoch': '0.04547'}
{'loss': '0.6987', 'grad_norm': '0.3983', 'learning_rate': '0.0001315', 'entropy': '0.7683', 'num_tokens': '2.779e+07', 'mean_token_accuracy': '0.8029', 'epoch': '0.04548'}
{'loss': '0.6408', 'grad_norm': '0.3139', 'learning_rate': '0.0001315', 'entropy': '0.6937', 'num_tokens': '2.78e+07', 'mean_token_accuracy': '0.8198', 'epoch': '0.04549'}
{'loss': '0.9566', 'grad_norm': '0.3737', 'learning_rate': '0.0001314', 'entropy': '1.032', 'num_tokens': '2.78e+07', 'mean_token_accuracy': '0.7425', 'epoch': '0.0455'}
{'loss': '0.6626', 'grad_norm': '0.3528', 'learning_rate': '0.0001314', 'entropy': '0.6701', 'num_tokens': '2.781e+07', 'mean_token_accuracy': '0.8256', 'epoch': '0.04551'}
{'loss': '0.6279', 'grad_norm': '0.3553', 'learning_rate': '0.0001314', 'entropy': '0.665', 'num_tokens': '2.781e+07', 'mean_token_accuracy': '0.832', 'epoch': '0.04551'}
{'loss': '0.7477', 'grad_norm': '0.3603', 'learning_rate': '0.0001313', 'entropy': '0.7228', 'num_tokens': '2.782e+07', 'mean_token_accuracy': '0.8098', 'epoch': '0.04552'}
{'loss': '1.132', 'grad_norm': '0.3719', 'learning_rate': '0.0001313', 'entropy': '1.186', 'num_tokens': '2.783e+07', 'mean_token_accuracy': '0.7215', 'epoch': '0.04553'}
{'loss': '0.8752', 'grad_norm': '0.4493', 'learning_rate': '0.0001313', 'entropy': '0.8397', 'num_tokens': '2.783e+07', 'mean_token_accuracy': '0.7775', 'epoch': '0.04554'}
{'loss': '1.152', 'grad_norm': '0.3925', 'learning_rate': '0.0001313', 'entropy': '1.009', 'num_tokens': '2.784e+07', 'mean_token_accuracy': '0.7569', 'epoch': '0.04555'}
{'loss': '0.5278', 'grad_norm': '0.3438', 'learning_rate': '0.0001312', 'entropy': '0.5804', 'num_tokens': '2.784e+07', 'mean_token_accuracy': '0.853', 'epoch': '0.04556'}
{'loss': '1.085', 'grad_norm': '0.3689', 'learning_rate': '0.0001312', 'entropy': '0.933', 'num_tokens': '2.785e+07', 'mean_token_accuracy': '0.7753', 'epoch': '0.04557'}
{'loss': '0.4985', 'grad_norm': '0.2778', 'learning_rate': '0.0001312', 'entropy': '0.5384', 'num_tokens': '2.786e+07', 'mean_token_accuracy': '0.8615', 'epoch': '0.04558'}
{'loss': '0.9139', 'grad_norm': '0.3064', 'learning_rate': '0.0001312', 'entropy': '0.9765', 'num_tokens': '2.787e+07', 'mean_token_accuracy': '0.7628', 'epoch': '0.04559'}
{'loss': '0.8544', 'grad_norm': '0.3341', 'learning_rate': '0.0001311', 'entropy': '0.873', 'num_tokens': '2.787e+07', 'mean_token_accuracy': '0.7686', 'epoch': '0.04559'}
{'loss': '0.5991', 'grad_norm': '0.3378', 'learning_rate': '0.0001311', 'entropy': '0.7241', 'num_tokens': '2.788e+07', 'mean_token_accuracy': '0.8196', 'epoch': '0.0456'}
{'loss': '0.8372', 'grad_norm': '0.3126', 'learning_rate': '0.0001311', 'entropy': '0.7153', 'num_tokens': '2.789e+07', 'mean_token_accuracy': '0.813', 'epoch': '0.04561'}
{'loss': '0.8988', 'grad_norm': '0.4', 'learning_rate': '0.0001311', 'entropy': '0.9436', 'num_tokens': '2.789e+07', 'mean_token_accuracy': '0.7593', 'epoch': '0.04562'}
{'loss': '0.7173', 'grad_norm': '0.3992', 'learning_rate': '0.000131', 'entropy': '0.7205', 'num_tokens': '2.79e+07', 'mean_token_accuracy': '0.8131', 'epoch': '0.04563'}
{'loss': '0.5835', 'grad_norm': '0.3391', 'learning_rate': '0.000131', 'entropy': '0.6623', 'num_tokens': '2.79e+07', 'mean_token_accuracy': '0.8221', 'epoch': '0.04564'}
{'loss': '1.115', 'grad_norm': '0.4357', 'learning_rate': '0.000131', 'entropy': '0.9933', 'num_tokens': '2.791e+07', 'mean_token_accuracy': '0.7671', 'epoch': '0.04565'}
{'loss': '0.7545', 'grad_norm': '0.3482', 'learning_rate': '0.000131', 'entropy': '0.741', 'num_tokens': '2.791e+07', 'mean_token_accuracy': '0.802', 'epoch': '0.04566'}
{'loss': '1.187', 'grad_norm': '0.4639', 'learning_rate': '0.0001309', 'entropy': '1.189', 'num_tokens': '2.792e+07', 'mean_token_accuracy': '0.716', 'epoch': '0.04567'}
{'loss': '0.8313', 'grad_norm': '0.514', 'learning_rate': '0.0001309', 'entropy': '0.8827', 'num_tokens': '2.792e+07', 'mean_token_accuracy': '0.7607', 'epoch': '0.04567'}
{'loss': '0.7711', 'grad_norm': '0.4255', 'learning_rate': '0.0001309', 'entropy': '0.7361', 'num_tokens': '2.793e+07', 'mean_token_accuracy': '0.8088', 'epoch': '0.04568'}
{'loss': '0.8767', 'grad_norm': '0.5148', 'learning_rate': '0.0001308', 'entropy': '0.8474', 'num_tokens': '2.793e+07', 'mean_token_accuracy': '0.781', 'epoch': '0.04569'}
{'loss': '0.8196', 'grad_norm': '0.3205', 'learning_rate': '0.0001308', 'entropy': '0.7676', 'num_tokens': '2.794e+07', 'mean_token_accuracy': '0.7954', 'epoch': '0.0457'}
{'loss': '0.8526', 'grad_norm': '0.3296', 'learning_rate': '0.0001308', 'entropy': '0.8526', 'num_tokens': '2.794e+07', 'mean_token_accuracy': '0.7761', 'epoch': '0.04571'}
{'loss': '0.5478', 'grad_norm': '0.3579', 'learning_rate': '0.0001308', 'entropy': '0.5511', 'num_tokens': '2.795e+07', 'mean_token_accuracy': '0.8549', 'epoch': '0.04572'}
{'loss': '0.7705', 'grad_norm': '0.3985', 'learning_rate': '0.0001307', 'entropy': '0.8561', 'num_tokens': '2.795e+07', 'mean_token_accuracy': '0.7724', 'epoch': '0.04573'}
{'loss': '0.9639', 'grad_norm': '0.4444', 'learning_rate': '0.0001307', 'entropy': '0.9297', 'num_tokens': '2.796e+07', 'mean_token_accuracy': '0.7682', 'epoch': '0.04574'}
{'loss': '0.776', 'grad_norm': '0.4049', 'learning_rate': '0.0001307', 'entropy': '0.7691', 'num_tokens': '2.797e+07', 'mean_token_accuracy': '0.8002', 'epoch': '0.04574'}
{'loss': '0.8451', 'grad_norm': '0.4073', 'learning_rate': '0.0001307', 'entropy': '0.8579', 'num_tokens': '2.797e+07', 'mean_token_accuracy': '0.8044', 'epoch': '0.04575'}
{'loss': '0.897', 'grad_norm': '0.4423', 'learning_rate': '0.0001306', 'entropy': '0.8947', 'num_tokens': '2.798e+07', 'mean_token_accuracy': '0.7801', 'epoch': '0.04576'}
{'loss': '0.6467', 'grad_norm': '0.4085', 'learning_rate': '0.0001306', 'entropy': '0.6789', 'num_tokens': '2.798e+07', 'mean_token_accuracy': '0.8215', 'epoch': '0.04577'}
{'loss': '0.7578', 'grad_norm': '0.4278', 'learning_rate': '0.0001306', 'entropy': '0.8088', 'num_tokens': '2.799e+07', 'mean_token_accuracy': '0.789', 'epoch': '0.04578'}

 43%|████▎     | 5174/12000 [3:04:36<3:50:23,  2.03s/it]
 43%|████▎     | 5175/12000 [3:04:38<3:40:49,  1.94s/it]
                                                        

 43%|████▎     | 5175/12000 [3:04:38<3:40:49,  1.94s/it]
 43%|████▎     | 5176/12000 [3:04:40<3:47:34,  2.00s/it]
                                                        

 43%|████▎     | 5176/12000 [3:04:40<3:47:34,  2.00s/it]
 43%|████▎     | 5177/12000 [3:04:41<3:34:18,  1.88s/it]
                                                        

 43%|████▎     | 5177/12000 [3:04:41<3:34:18,  1.88s/it]
 43%|████▎     | 5178/12000 [3:04:43<3:23:58,  1.79s/it]
                                                        

 43%|████▎     | 5178/12000 [3:04:43<3:23:58,  1.79s/it]
 43%|████▎     | 5179/12000 [3:04:45<3:29:51,  1.85s/it]
                                                        

 43%|████▎     | 5179/12000 [3:04:45<3:29:51,  1.85s/it]
 43%|████▎     | 5180/12000 [3:04:47<3:35:15,  1.89s/it]
                                                        

 43%|████▎     | 5180/12000 [3:04:47<3:35:15,  1.89s/it]
 43%|████▎     | 5181/12000 [3:04:49<3:24:14,  1.80s/it]
                                                        

 43%|████▎     | 5181/12000 [3:04:49<3:24:14,  1.80s/it]
 43%|████▎     | 5182/12000 [3:04:52<4:18:47,  2.28s/it]
                                                        

 43%|████▎     | 5182/12000 [3:04:52<4:18:47,  2.28s/it]
 43%|████▎     | 5183/12000 [3:04:54<3:57:06,  2.09s/it]
                                                        

 43%|████▎     | 5183/12000 [3:04:54<3:57:06,  2.09s/it]
 43%|████▎     | 5184/12000 [3:04:58<5:18:42,  2.81s/it]
                                                        

 43%|████▎     | 5184/12000 [3:04:58<5:18:42,  2.81s/it]
 43%|████▎     | 5185/12000 [3:05:00<4:42:16,  2.49s/it]
                                                        

 43%|████▎     | 5185/12000 [3:05:00<4:42:16,  2.49s/it]
 43%|████▎     | 5186/12000 [3:05:02<4:14:55,  2.24s/it]
                                                        

 43%|████▎     | 5186/12000 [3:05:02<4:14:55,  2.24s/it]
 43%|████▎     | 5187/12000 [3:05:03<3:51:17,  2.04s/it]
                                                        

 43%|████▎     | 5187/12000 [3:05:03<3:51:17,  2.04s/it]
 43%|████▎     | 5188/12000 [3:05:05<3:43:13,  1.97s/it]
                                                        

 43%|████▎     | 5188/12000 [3:05:05<3:43:13,  1.97s/it]
 43%|████▎     | 5189/12000 [3:05:07<3:42:41,  1.96s/it]
                                                        

 43%|████▎     | 5189/12000 [3:05:07<3:42:41,  1.96s/it]
 43%|████▎     | 5190/12000 [3:05:09<3:33:14,  1.88s/it]
                                                        

 43%|████▎     | 5190/12000 [3:05:09<3:33:14,  1.88s/it]
 43%|████▎     | 5191/12000 [3:05:11<3:49:26,  2.02s/it]
                                                        

 43%|████▎     | 5191/12000 [3:05:11<3:49:26,  2.02s/it]
 43%|████▎     | 5192/12000 [3:05:13<4:01:34,  2.13s/it]
                                                        

 43%|████▎     | 5192/12000 [3:05:13<4:01:34,  2.13s/it]
 43%|████▎     | 5193/12000 [3:05:17<4:43:35,  2.50s/it]
                                                        

 43%|████▎     | 5193/12000 [3:05:17<4:43:35,  2.50s/it]
 43%|████▎     | 5194/12000 [3:05:18<4:08:55,  2.19s/it]
                                                        

 43%|████▎     | 5194/12000 [3:05:18<4:08:55,  2.19s/it]
 43%|████▎     | 5195/12000 [3:05:19<3:40:55,  1.95s/it]
                                                        

 43%|████▎     | 5195/12000 [3:05:19<3:40:55,  1.95s/it]
 43%|████▎     | 5196/12000 [3:05:21<3:23:29,  1.79s/it]
                                                        

 43%|████▎     | 5196/12000 [3:05:21<3:23:29,  1.79s/it]
 43%|████▎     | 5197/12000 [3:05:22<3:13:05,  1.70s/it]
                                                        

 43%|████▎     | 5197/12000 [3:05:22<3:13:05,  1.70s/it]
 43%|████▎     | 5198/12000 [3:05:25<3:26:26,  1.82s/it]
                                                        

 43%|████▎     | 5198/12000 [3:05:25<3:26:26,  1.82s/it]
 43%|████▎     | 5199/12000 [3:05:26<3:30:47,  1.86s/it]
                                                        

 43%|████▎     | 5199/12000 [3:05:26<3:30:47,  1.86s/it]
 43%|████▎     | 5200/12000 [3:05:28<3:23:24,  1.79s/it]
                                                        

 43%|████▎     | 5200/12000 [3:05:28<3:23:24,  1.79s/it]
 43%|████▎     | 5201/12000 [3:05:32<4:35:23,  2.43s/it]
                                                        

 43%|████▎     | 5201/12000 [3:05:32<4:35:23,  2.43s/it]
 43%|████▎     | 5202/12000 [3:05:34<4:07:48,  2.19s/it]
                                                        

 43%|████▎     | 5202/12000 [3:05:34<4:07:48,  2.19s/it]
 43%|████▎     | 5203/12000 [3:05:35<3:52:38,  2.05s/it]
                                                        

 43%|████▎     | 5203/12000 [3:05:35<3:52:38,  2.05s/it]
 43%|████▎     | 5204/12000 [3:05:38<3:55:02,  2.08s/it]
                                                        

 43%|████▎     | 5204/12000 [3:05:38<3:55:02,  2.08s/it]
 43%|████▎     | 5205/12000 [3:05:39<3:27:11,  1.83s/it]
                                                        

 43%|████▎     | 5205/12000 [3:05:39<3:27:11,  1.83s/it]
 43%|████▎     | 5206/12000 [3:05:43<4:48:29,  2.55s/it]
                                                        

 43%|████▎     | 5206/12000 [3:05:43<4:48:29,  2.55s/it]
 43%|████▎     | 5207/12000 [3:05:45<4:22:33,  2.32s/it]
                                                        

 43%|████▎     | 5207/12000 [3:05:45<4:22:33,  2.32s/it]
 43%|████▎     | 5208/12000 [3:05:47<4:15:14,  2.25s/it]
                                                        

 43%|████▎     | 5208/12000 [3:05:47<4:15:14,  2.25s/it]
 43%|████▎     | 5209/12000 [3:05:49<3:56:18,  2.09s/it]
                                                        

 43%|████▎     | 5209/12000 [3:05:49<3:56:18,  2.09s/it]
 43%|████▎     | 5210/12000 [3:05:50<3:34:52,  1.90s/it]
                                                        

 43%|████▎     | 5210/12000 [3:05:50<3:34:52,  1.90s/it]
 43%|████▎     | 5211/12000 [3:05:52<3:44:02,  1.98s/it]
                                                        

 43%|████▎     | 5211/12000 [3:05:52<3:44:02,  1.98s/it]
 43%|████▎     | 5212/12000 [3:05:55<3:54:58,  2.08s/it]
                                                        

 43%|████▎     | 5212/12000 [3:05:55<3:54:58,  2.08s/it]
 43%|████▎     | 5213/12000 [3:05:56<3:41:22,  1.96s/it]
                                                        

 43%|████▎     | 5213/12000 [3:05:56<3:41:22,  1.96s/it]
 43%|████▎     | 5214/12000 [3:05:58<3:43:12,  1.97s/it]
                                                        

 43%|████▎     | 5214/12000 [3:05:58<3:43:12,  1.97s/it]
 43%|████▎     | 5215/12000 [3:06:00<3:31:55,  1.87s/it]
                                                        

 43%|████▎     | 5215/12000 [3:06:00<3:31:55,  1.87s/it]
 43%|████▎     | 5216/12000 [3:06:04<5:00:20,  2.66s/it]
                                                        

 43%|████▎     | 5216/12000 [3:06:04<5:00:20,  2.66s/it]
 43%|████▎     | 5217/12000 [3:06:06<4:30:35,  2.39s/it]
                                                        

 43%|████▎     | 5217/12000 [3:06:06<4:30:35,  2.39s/it]
 43%|████▎     | 5218/12000 [3:06:08<4:21:02,  2.31s/it]
                                                        

 43%|████▎     | 5218/12000 [3:06:08<4:21:02,  2.31s/it]
 43%|████▎     | 5219/12000 [3:06:11<4:31:30,  2.40s/it]
                                                        

 43%|████▎     | 5219/12000 [3:06:11<4:31:30,  2.40s/it]
 44%|████▎     | 5220/12000 [3:06:12<4:04:16,  2.16s/it]
                                                        

 44%|████▎     | 5220/12000 [3:06:12<4:04:16,  2.16s/it]
 44%|████▎     | 5221/12000 [3:06:14<3:41:14,  1.96s/it]
                                                        
{'loss': '0.8079', 'grad_norm': '0.4106', 'learning_rate': '0.0001306', 'entropy': '0.8718', 'num_tokens': '2.799e+07', 'mean_token_accuracy': '0.7721', 'epoch': '0.04579'}
{'loss': '0.8422', 'grad_norm': '0.4093', 'learning_rate': '0.0001305', 'entropy': '0.8481', 'num_tokens': '2.8e+07', 'mean_token_accuracy': '0.7966', 'epoch': '0.0458'}
{'loss': '0.8242', 'grad_norm': '0.3658', 'learning_rate': '0.0001305', 'entropy': '0.7961', 'num_tokens': '2.8e+07', 'mean_token_accuracy': '0.8023', 'epoch': '0.04581'}
{'loss': '1.059', 'grad_norm': '0.4321', 'learning_rate': '0.0001305', 'entropy': '0.9742', 'num_tokens': '2.801e+07', 'mean_token_accuracy': '0.7724', 'epoch': '0.04582'}
{'loss': '0.6665', 'grad_norm': '0.3558', 'learning_rate': '0.0001305', 'entropy': '0.7665', 'num_tokens': '2.801e+07', 'mean_token_accuracy': '0.8024', 'epoch': '0.04582'}
{'loss': '0.6321', 'grad_norm': '0.2834', 'learning_rate': '0.0001304', 'entropy': '0.6788', 'num_tokens': '2.802e+07', 'mean_token_accuracy': '0.8325', 'epoch': '0.04583'}
{'loss': '0.8435', 'grad_norm': '0.3902', 'learning_rate': '0.0001304', 'entropy': '0.7838', 'num_tokens': '2.803e+07', 'mean_token_accuracy': '0.802', 'epoch': '0.04584'}
{'loss': '0.8308', 'grad_norm': '0.4615', 'learning_rate': '0.0001304', 'entropy': '0.9304', 'num_tokens': '2.803e+07', 'mean_token_accuracy': '0.7759', 'epoch': '0.04585'}
{'loss': '0.9893', 'grad_norm': '0.5113', 'learning_rate': '0.0001304', 'entropy': '0.762', 'num_tokens': '2.804e+07', 'mean_token_accuracy': '0.8127', 'epoch': '0.04586'}
{'loss': '0.7615', 'grad_norm': '0.3494', 'learning_rate': '0.0001303', 'entropy': '0.7299', 'num_tokens': '2.805e+07', 'mean_token_accuracy': '0.8172', 'epoch': '0.04587'}
{'loss': '0.4543', 'grad_norm': '0.2992', 'learning_rate': '0.0001303', 'entropy': '0.5345', 'num_tokens': '2.805e+07', 'mean_token_accuracy': '0.8656', 'epoch': '0.04588'}
{'loss': '1.153', 'grad_norm': '0.4048', 'learning_rate': '0.0001303', 'entropy': '1.148', 'num_tokens': '2.806e+07', 'mean_token_accuracy': '0.7253', 'epoch': '0.04589'}
{'loss': '0.7645', 'grad_norm': '0.3926', 'learning_rate': '0.0001302', 'entropy': '0.763', 'num_tokens': '2.806e+07', 'mean_token_accuracy': '0.8034', 'epoch': '0.0459'}
{'loss': '0.8895', 'grad_norm': '0.4921', 'learning_rate': '0.0001302', 'entropy': '0.8698', 'num_tokens': '2.807e+07', 'mean_token_accuracy': '0.7796', 'epoch': '0.0459'}
{'loss': '0.7225', 'grad_norm': '0.3928', 'learning_rate': '0.0001302', 'entropy': '0.7492', 'num_tokens': '2.807e+07', 'mean_token_accuracy': '0.805', 'epoch': '0.04591'}
{'loss': '0.7643', 'grad_norm': '0.4157', 'learning_rate': '0.0001302', 'entropy': '0.7972', 'num_tokens': '2.808e+07', 'mean_token_accuracy': '0.7941', 'epoch': '0.04592'}
{'loss': '0.6755', 'grad_norm': '0.3022', 'learning_rate': '0.0001301', 'entropy': '0.7261', 'num_tokens': '2.808e+07', 'mean_token_accuracy': '0.8041', 'epoch': '0.04593'}
{'loss': '0.9374', 'grad_norm': '0.485', 'learning_rate': '0.0001301', 'entropy': '0.81', 'num_tokens': '2.809e+07', 'mean_token_accuracy': '0.7959', 'epoch': '0.04594'}
{'loss': '0.8913', 'grad_norm': '0.3188', 'learning_rate': '0.0001301', 'entropy': '0.8871', 'num_tokens': '2.809e+07', 'mean_token_accuracy': '0.7805', 'epoch': '0.04595'}
{'loss': '0.8483', 'grad_norm': '0.3093', 'learning_rate': '0.0001301', 'entropy': '0.8093', 'num_tokens': '2.81e+07', 'mean_token_accuracy': '0.7899', 'epoch': '0.04596'}
{'loss': '0.8211', 'grad_norm': '0.4065', 'learning_rate': '0.00013', 'entropy': '0.8517', 'num_tokens': '2.811e+07', 'mean_token_accuracy': '0.7902', 'epoch': '0.04597'}
{'loss': '0.8621', 'grad_norm': '0.5679', 'learning_rate': '0.00013', 'entropy': '0.8624', 'num_tokens': '2.811e+07', 'mean_token_accuracy': '0.7753', 'epoch': '0.04597'}
{'loss': '0.7591', 'grad_norm': '0.3609', 'learning_rate': '0.00013', 'entropy': '0.7122', 'num_tokens': '2.811e+07', 'mean_token_accuracy': '0.8134', 'epoch': '0.04598'}
{'loss': '0.6972', 'grad_norm': '0.3594', 'learning_rate': '0.00013', 'entropy': '0.7324', 'num_tokens': '2.812e+07', 'mean_token_accuracy': '0.8225', 'epoch': '0.04599'}
{'loss': '1.234', 'grad_norm': '0.4027', 'learning_rate': '0.0001299', 'entropy': '1.196', 'num_tokens': '2.812e+07', 'mean_token_accuracy': '0.7132', 'epoch': '0.046'}
{'loss': '0.8371', 'grad_norm': '0.3457', 'learning_rate': '0.0001299', 'entropy': '0.8183', 'num_tokens': '2.813e+07', 'mean_token_accuracy': '0.7905', 'epoch': '0.04601'}
{'loss': '1.174', 'grad_norm': '0.4231', 'learning_rate': '0.0001299', 'entropy': '1.191', 'num_tokens': '2.813e+07', 'mean_token_accuracy': '0.7177', 'epoch': '0.04602'}
{'loss': '0.6861', 'grad_norm': '0.3389', 'learning_rate': '0.0001299', 'entropy': '0.6498', 'num_tokens': '2.814e+07', 'mean_token_accuracy': '0.8363', 'epoch': '0.04603'}
{'loss': '0.7038', 'grad_norm': '0.4242', 'learning_rate': '0.0001298', 'entropy': '0.6965', 'num_tokens': '2.815e+07', 'mean_token_accuracy': '0.8132', 'epoch': '0.04604'}
{'loss': '0.9156', 'grad_norm': '0.4312', 'learning_rate': '0.0001298', 'entropy': '0.8693', 'num_tokens': '2.815e+07', 'mean_token_accuracy': '0.7762', 'epoch': '0.04605'}
{'loss': '0.797', 'grad_norm': '0.3313', 'learning_rate': '0.0001298', 'entropy': '0.7816', 'num_tokens': '2.816e+07', 'mean_token_accuracy': '0.7927', 'epoch': '0.04605'}
{'loss': '1.032', 'grad_norm': '0.5093', 'learning_rate': '0.0001297', 'entropy': '1.065', 'num_tokens': '2.816e+07', 'mean_token_accuracy': '0.7372', 'epoch': '0.04606'}
{'loss': '1.292', 'grad_norm': '0.4484', 'learning_rate': '0.0001297', 'entropy': '1.148', 'num_tokens': '2.817e+07', 'mean_token_accuracy': '0.7292', 'epoch': '0.04607'}
{'loss': '0.8654', 'grad_norm': '0.292', 'learning_rate': '0.0001297', 'entropy': '0.905', 'num_tokens': '2.817e+07', 'mean_token_accuracy': '0.7738', 'epoch': '0.04608'}
{'loss': '1.001', 'grad_norm': '0.3353', 'learning_rate': '0.0001297', 'entropy': '1.05', 'num_tokens': '2.818e+07', 'mean_token_accuracy': '0.7631', 'epoch': '0.04609'}
{'loss': '0.8052', 'grad_norm': '0.4452', 'learning_rate': '0.0001296', 'entropy': '0.809', 'num_tokens': '2.818e+07', 'mean_token_accuracy': '0.7984', 'epoch': '0.0461'}
{'loss': '1.187', 'grad_norm': '0.4112', 'learning_rate': '0.0001296', 'entropy': '1.109', 'num_tokens': '2.819e+07', 'mean_token_accuracy': '0.7185', 'epoch': '0.04611'}
{'loss': '0.8207', 'grad_norm': '0.4153', 'learning_rate': '0.0001296', 'entropy': '0.6932', 'num_tokens': '2.819e+07', 'mean_token_accuracy': '0.8281', 'epoch': '0.04612'}
{'loss': '0.7501', 'grad_norm': '0.4105', 'learning_rate': '0.0001296', 'entropy': '0.7184', 'num_tokens': '2.82e+07', 'mean_token_accuracy': '0.8122', 'epoch': '0.04613'}
{'loss': '0.8407', 'grad_norm': '0.4444', 'learning_rate': '0.0001295', 'entropy': '0.8345', 'num_tokens': '2.82e+07', 'mean_token_accuracy': '0.7899', 'epoch': '0.04613'}
{'loss': '0.5488', 'grad_norm': '0.3757', 'learning_rate': '0.0001295', 'entropy': '0.6174', 'num_tokens': '2.821e+07', 'mean_token_accuracy': '0.8369', 'epoch': '0.04614'}
{'loss': '0.7684', 'grad_norm': '0.411', 'learning_rate': '0.0001295', 'entropy': '0.7472', 'num_tokens': '2.822e+07', 'mean_token_accuracy': '0.8029', 'epoch': '0.04615'}
{'loss': '0.6713', 'grad_norm': '0.2493', 'learning_rate': '0.0001295', 'entropy': '0.7189', 'num_tokens': '2.822e+07', 'mean_token_accuracy': '0.8143', 'epoch': '0.04616'}
{'loss': '0.8683', 'grad_norm': '0.3845', 'learning_rate': '0.0001294', 'entropy': '0.9089', 'num_tokens': '2.823e+07', 'mean_token_accuracy': '0.7797', 'epoch': '0.04617'}
{'loss': '0.866', 'grad_norm': '0.3716', 'learning_rate': '0.0001294', 'entropy': '1.009', 'num_tokens': '2.823e+07', 'mean_token_accuracy': '0.743', 'epoch': '0.04618'}
{'loss': '0.7852', 'grad_norm': '0.4311', 'learning_rate': '0.0001294', 'entropy': '0.827', 'num_tokens': '2.824e+07', 'mean_token_accuracy': '0.7932', 'epoch': '0.04619'}
{'loss': '0.9004', 'grad_norm': '0.4916', 'learning_rate': '0.0001294', 'entropy': '0.8682', 'num_tokens': '2.824e+07', 'mean_token_accuracy': '0.7744', 'epoch': '0.0462'}

 44%|████▎     | 5221/12000 [3:06:14<3:41:14,  1.96s/it]
 44%|████▎     | 5222/12000 [3:06:16<3:28:52,  1.85s/it]
                                                        

 44%|████▎     | 5222/12000 [3:06:16<3:28:52,  1.85s/it]
 44%|████▎     | 5223/12000 [3:06:20<4:44:12,  2.52s/it]
                                                        

 44%|████▎     | 5223/12000 [3:06:20<4:44:12,  2.52s/it]
 44%|████▎     | 5224/12000 [3:06:22<4:36:08,  2.45s/it]
                                                        

 44%|████▎     | 5224/12000 [3:06:22<4:36:08,  2.45s/it]
 44%|████▎     | 5225/12000 [3:06:23<3:59:09,  2.12s/it]
                                                        

 44%|████▎     | 5225/12000 [3:06:23<3:59:09,  2.12s/it]
 44%|████▎     | 5226/12000 [3:06:25<3:46:28,  2.01s/it]
                                                        

 44%|████▎     | 5226/12000 [3:06:25<3:46:28,  2.01s/it]
 44%|████▎     | 5227/12000 [3:06:28<4:19:53,  2.30s/it]
                                                        

 44%|████▎     | 5227/12000 [3:06:28<4:19:53,  2.30s/it]
 44%|████▎     | 5228/12000 [3:06:31<4:49:31,  2.57s/it]
                                                        

 44%|████▎     | 5228/12000 [3:06:31<4:49:31,  2.57s/it]
 44%|████▎     | 5229/12000 [3:06:33<4:15:39,  2.27s/it]
                                                        

 44%|████▎     | 5229/12000 [3:06:33<4:15:39,  2.27s/it]
 44%|████▎     | 5230/12000 [3:06:34<3:47:10,  2.01s/it]
                                                        

 44%|████▎     | 5230/12000 [3:06:34<3:47:10,  2.01s/it]
 44%|████▎     | 5231/12000 [3:06:36<3:39:24,  1.94s/it]
                                                        

 44%|████▎     | 5231/12000 [3:06:36<3:39:24,  1.94s/it]
 44%|████▎     | 5232/12000 [3:06:38<3:29:04,  1.85s/it]
                                                        

 44%|████▎     | 5232/12000 [3:06:38<3:29:04,  1.85s/it]
 44%|████▎     | 5233/12000 [3:06:39<3:17:19,  1.75s/it]
                                                        

 44%|████▎     | 5233/12000 [3:06:39<3:17:19,  1.75s/it]
 44%|████▎     | 5234/12000 [3:06:41<3:07:06,  1.66s/it]
                                                        

 44%|████▎     | 5234/12000 [3:06:41<3:07:06,  1.66s/it]
 44%|████▎     | 5235/12000 [3:06:43<3:29:39,  1.86s/it]
                                                        

 44%|████▎     | 5235/12000 [3:06:43<3:29:39,  1.86s/it]
 44%|████▎     | 5236/12000 [3:06:44<3:16:55,  1.75s/it]
                                                        

 44%|████▎     | 5236/12000 [3:06:44<3:16:55,  1.75s/it]
 44%|████▎     | 5237/12000 [3:06:46<3:06:26,  1.65s/it]
                                                        

 44%|████▎     | 5237/12000 [3:06:46<3:06:26,  1.65s/it]
 44%|████▎     | 5238/12000 [3:06:47<3:09:14,  1.68s/it]
                                                        

 44%|████▎     | 5238/12000 [3:06:47<3:09:14,  1.68s/it]
 44%|████▎     | 5239/12000 [3:06:49<3:03:45,  1.63s/it]
                                                        

 44%|████▎     | 5239/12000 [3:06:49<3:03:45,  1.63s/it]
 44%|████▎     | 5240/12000 [3:06:53<4:22:36,  2.33s/it]
                                                        

 44%|████▎     | 5240/12000 [3:06:53<4:22:36,  2.33s/it]
 44%|████▎     | 5241/12000 [3:06:55<4:11:29,  2.23s/it]
                                                        

 44%|████▎     | 5241/12000 [3:06:55<4:11:29,  2.23s/it]
 44%|████▎     | 5242/12000 [3:06:56<3:43:18,  1.98s/it]
                                                        

 44%|████▎     | 5242/12000 [3:06:56<3:43:18,  1.98s/it]
 44%|████▎     | 5243/12000 [3:06:59<3:57:15,  2.11s/it]
                                                        

 44%|████▎     | 5243/12000 [3:06:59<3:57:15,  2.11s/it]
 44%|████▎     | 5244/12000 [3:07:00<3:38:40,  1.94s/it]
                                                        

 44%|████▎     | 5244/12000 [3:07:00<3:38:40,  1.94s/it]
 44%|████▎     | 5245/12000 [3:07:03<4:00:19,  2.13s/it]
                                                        

 44%|████▎     | 5245/12000 [3:07:03<4:00:19,  2.13s/it]
 44%|████▎     | 5246/12000 [3:07:06<4:39:24,  2.48s/it]
                                                        

 44%|████▎     | 5246/12000 [3:07:06<4:39:24,  2.48s/it]
 44%|████▎     | 5247/12000 [3:07:08<4:11:11,  2.23s/it]
                                                        

 44%|████▎     | 5247/12000 [3:07:08<4:11:11,  2.23s/it]
 44%|████▎     | 5248/12000 [3:07:11<4:31:44,  2.41s/it]
                                                        

 44%|████▎     | 5248/12000 [3:07:11<4:31:44,  2.41s/it]
 44%|████▎     | 5249/12000 [3:07:12<4:05:22,  2.18s/it]
                                                        

 44%|████▎     | 5249/12000 [3:07:12<4:05:22,  2.18s/it]
 44%|████▍     | 5250/12000 [3:07:16<5:01:33,  2.68s/it]
                                                        

 44%|████▍     | 5250/12000 [3:07:16<5:01:33,  2.68s/it]
 44%|████▍     | 5251/12000 [3:07:19<5:10:54,  2.76s/it]
                                                        

 44%|████▍     | 5251/12000 [3:07:19<5:10:54,  2.76s/it]
 44%|████▍     | 5252/12000 [3:07:21<4:39:13,  2.48s/it]
                                                        

 44%|████▍     | 5252/12000 [3:07:21<4:39:13,  2.48s/it]
 44%|████▍     | 5253/12000 [3:07:23<4:34:48,  2.44s/it]
                                                        

 44%|████▍     | 5253/12000 [3:07:23<4:34:48,  2.44s/it]
 44%|████▍     | 5254/12000 [3:07:25<4:03:43,  2.17s/it]
                                                        

 44%|████▍     | 5254/12000 [3:07:25<4:03:43,  2.17s/it]
 44%|████▍     | 5255/12000 [3:07:26<3:41:20,  1.97s/it]
                                                        

 44%|████▍     | 5255/12000 [3:07:26<3:41:20,  1.97s/it]
 44%|████▍     | 5256/12000 [3:07:28<3:27:13,  1.84s/it]
                                                        

 44%|████▍     | 5256/12000 [3:07:28<3:27:13,  1.84s/it]
 44%|████▍     | 5257/12000 [3:07:29<3:17:45,  1.76s/it]
                                                        

 44%|████▍     | 5257/12000 [3:07:29<3:17:45,  1.76s/it]
 44%|████▍     | 5258/12000 [3:07:32<3:28:08,  1.85s/it]
                                                        

 44%|████▍     | 5258/12000 [3:07:32<3:28:08,  1.85s/it]
 44%|████▍     | 5259/12000 [3:07:33<3:30:55,  1.88s/it]
                                                        

 44%|████▍     | 5259/12000 [3:07:33<3:30:55,  1.88s/it]
 44%|████▍     | 5260/12000 [3:07:35<3:13:31,  1.72s/it]
                                                        

 44%|████▍     | 5260/12000 [3:07:35<3:13:31,  1.72s/it]
 44%|████▍     | 5261/12000 [3:07:37<3:20:10,  1.78s/it]
                                                        

 44%|████▍     | 5261/12000 [3:07:37<3:20:10,  1.78s/it]
 44%|████▍     | 5262/12000 [3:07:39<3:28:08,  1.85s/it]
                                                        

 44%|████▍     | 5262/12000 [3:07:39<3:28:08,  1.85s/it]
 44%|████▍     | 5263/12000 [3:07:41<3:56:04,  2.10s/it]
                                                        

 44%|████▍     | 5263/12000 [3:07:41<3:56:04,  2.10s/it]
 44%|████▍     | 5264/12000 [3:07:44<3:57:05,  2.11s/it]
                                                        

 44%|████▍     | 5264/12000 [3:07:44<3:57:05,  2.11s/it]
 44%|████▍     | 5265/12000 [3:07:45<3:41:45,  1.98s/it]
                                                        

 44%|████▍     | 5265/12000 [3:07:45<3:41:45,  1.98s/it]
 44%|████▍     | 5266/12000 [3:07:47<3:44:59,  2.00s/it]
                                                        

 44%|████▍     | 5266/12000 [3:07:47<3:44:59,  2.00s/it]
 44%|████▍     | 5267/12000 [3:07:49<3:50:06,  2.05s/it]
                                                        

 44%|████▍     | 5267/12000 [3:07:49<3:50:06,  2.05s/it]
 44%|████▍     | 5268/12000 [3:07:55<5:54:16,  3.16s/it]
                                                        
{'loss': '0.6912', 'grad_norm': '0.3264', 'learning_rate': '0.0001293', 'entropy': '0.7085', 'num_tokens': '2.825e+07', 'mean_token_accuracy': '0.8101', 'epoch': '0.0462'}
{'loss': '0.8805', 'grad_norm': '0.4313', 'learning_rate': '0.0001293', 'entropy': '0.9057', 'num_tokens': '2.825e+07', 'mean_token_accuracy': '0.7701', 'epoch': '0.04621'}
{'loss': '1.115', 'grad_norm': '0.3608', 'learning_rate': '0.0001293', 'entropy': '1.192', 'num_tokens': '2.826e+07', 'mean_token_accuracy': '0.7179', 'epoch': '0.04622'}
{'loss': '1.275', 'grad_norm': '0.5226', 'learning_rate': '0.0001292', 'entropy': '1.131', 'num_tokens': '2.827e+07', 'mean_token_accuracy': '0.7261', 'epoch': '0.04623'}
{'loss': '0.8042', 'grad_norm': '0.3551', 'learning_rate': '0.0001292', 'entropy': '0.7974', 'num_tokens': '2.827e+07', 'mean_token_accuracy': '0.7946', 'epoch': '0.04624'}
{'loss': '0.6494', 'grad_norm': '0.3852', 'learning_rate': '0.0001292', 'entropy': '0.6692', 'num_tokens': '2.828e+07', 'mean_token_accuracy': '0.8311', 'epoch': '0.04625'}
{'loss': '0.8411', 'grad_norm': '0.3982', 'learning_rate': '0.0001292', 'entropy': '0.8459', 'num_tokens': '2.829e+07', 'mean_token_accuracy': '0.7818', 'epoch': '0.04626'}
{'loss': '0.8626', 'grad_norm': '0.4194', 'learning_rate': '0.0001291', 'entropy': '0.9075', 'num_tokens': '2.829e+07', 'mean_token_accuracy': '0.7788', 'epoch': '0.04627'}
{'loss': '0.862', 'grad_norm': '0.3436', 'learning_rate': '0.0001291', 'entropy': '0.855', 'num_tokens': '2.829e+07', 'mean_token_accuracy': '0.8038', 'epoch': '0.04628'}
{'loss': '0.6905', 'grad_norm': '0.3854', 'learning_rate': '0.0001291', 'entropy': '0.7074', 'num_tokens': '2.83e+07', 'mean_token_accuracy': '0.8201', 'epoch': '0.04628'}
{'loss': '0.9384', 'grad_norm': '0.4858', 'learning_rate': '0.0001291', 'entropy': '0.9457', 'num_tokens': '2.83e+07', 'mean_token_accuracy': '0.7605', 'epoch': '0.04629'}
{'loss': '1.018', 'grad_norm': '0.407', 'learning_rate': '0.000129', 'entropy': '0.9766', 'num_tokens': '2.831e+07', 'mean_token_accuracy': '0.7635', 'epoch': '0.0463'}
{'loss': '0.7403', 'grad_norm': '0.3243', 'learning_rate': '0.000129', 'entropy': '0.7594', 'num_tokens': '2.831e+07', 'mean_token_accuracy': '0.7947', 'epoch': '0.04631'}
{'loss': '1.182', 'grad_norm': '0.4688', 'learning_rate': '0.000129', 'entropy': '1.187', 'num_tokens': '2.832e+07', 'mean_token_accuracy': '0.7253', 'epoch': '0.04632'}
{'loss': '0.8709', 'grad_norm': '0.3947', 'learning_rate': '0.000129', 'entropy': '0.9193', 'num_tokens': '2.832e+07', 'mean_token_accuracy': '0.7866', 'epoch': '0.04633'}
{'loss': '0.9179', 'grad_norm': '0.4458', 'learning_rate': '0.0001289', 'entropy': '0.9938', 'num_tokens': '2.833e+07', 'mean_token_accuracy': '0.7601', 'epoch': '0.04634'}
{'loss': '0.7055', 'grad_norm': '0.393', 'learning_rate': '0.0001289', 'entropy': '0.7763', 'num_tokens': '2.833e+07', 'mean_token_accuracy': '0.8037', 'epoch': '0.04635'}
{'loss': '0.846', 'grad_norm': '0.4313', 'learning_rate': '0.0001289', 'entropy': '0.8844', 'num_tokens': '2.833e+07', 'mean_token_accuracy': '0.7703', 'epoch': '0.04636'}
{'loss': '0.7213', 'grad_norm': '0.3653', 'learning_rate': '0.0001289', 'entropy': '0.7347', 'num_tokens': '2.834e+07', 'mean_token_accuracy': '0.8081', 'epoch': '0.04636'}
{'loss': '1.088', 'grad_norm': '0.307', 'learning_rate': '0.0001288', 'entropy': '0.9915', 'num_tokens': '2.835e+07', 'mean_token_accuracy': '0.7664', 'epoch': '0.04637'}
{'loss': '0.9177', 'grad_norm': '0.3611', 'learning_rate': '0.0001288', 'entropy': '0.8448', 'num_tokens': '2.836e+07', 'mean_token_accuracy': '0.7793', 'epoch': '0.04638'}
{'loss': '0.891', 'grad_norm': '0.4353', 'learning_rate': '0.0001288', 'entropy': '0.9218', 'num_tokens': '2.836e+07', 'mean_token_accuracy': '0.7749', 'epoch': '0.04639'}
{'loss': '0.526', 'grad_norm': '0.3366', 'learning_rate': '0.0001287', 'entropy': '0.6442', 'num_tokens': '2.836e+07', 'mean_token_accuracy': '0.8444', 'epoch': '0.0464'}
{'loss': '0.8637', 'grad_norm': '0.4885', 'learning_rate': '0.0001287', 'entropy': '0.8523', 'num_tokens': '2.837e+07', 'mean_token_accuracy': '0.7795', 'epoch': '0.04641'}
{'loss': '0.6938', 'grad_norm': '0.3429', 'learning_rate': '0.0001287', 'entropy': '0.7229', 'num_tokens': '2.838e+07', 'mean_token_accuracy': '0.8195', 'epoch': '0.04642'}
{'loss': '0.7508', 'grad_norm': '0.354', 'learning_rate': '0.0001287', 'entropy': '0.7606', 'num_tokens': '2.838e+07', 'mean_token_accuracy': '0.8029', 'epoch': '0.04643'}
{'loss': '1.084', 'grad_norm': '0.5773', 'learning_rate': '0.0001286', 'entropy': '1.102', 'num_tokens': '2.839e+07', 'mean_token_accuracy': '0.731', 'epoch': '0.04644'}
{'loss': '0.9061', 'grad_norm': '0.3025', 'learning_rate': '0.0001286', 'entropy': '0.9196', 'num_tokens': '2.839e+07', 'mean_token_accuracy': '0.7452', 'epoch': '0.04644'}
{'loss': '0.8288', 'grad_norm': '0.4476', 'learning_rate': '0.0001286', 'entropy': '0.838', 'num_tokens': '2.84e+07', 'mean_token_accuracy': '0.7808', 'epoch': '0.04645'}
{'loss': '0.6453', 'grad_norm': '0.3918', 'learning_rate': '0.0001286', 'entropy': '0.628', 'num_tokens': '2.84e+07', 'mean_token_accuracy': '0.8369', 'epoch': '0.04646'}
{'loss': '0.9817', 'grad_norm': '0.3923', 'learning_rate': '0.0001285', 'entropy': '0.9335', 'num_tokens': '2.841e+07', 'mean_token_accuracy': '0.7613', 'epoch': '0.04647'}
{'loss': '1.577', 'grad_norm': '0.3878', 'learning_rate': '0.0001285', 'entropy': '1.255', 'num_tokens': '2.841e+07', 'mean_token_accuracy': '0.7172', 'epoch': '0.04648'}
{'loss': '0.6333', 'grad_norm': '0.2742', 'learning_rate': '0.0001285', 'entropy': '0.6124', 'num_tokens': '2.842e+07', 'mean_token_accuracy': '0.8342', 'epoch': '0.04649'}
{'loss': '0.8067', 'grad_norm': '0.4247', 'learning_rate': '0.0001285', 'entropy': '0.8113', 'num_tokens': '2.842e+07', 'mean_token_accuracy': '0.7952', 'epoch': '0.0465'}
{'loss': '0.7731', 'grad_norm': '0.3452', 'learning_rate': '0.0001284', 'entropy': '0.8718', 'num_tokens': '2.843e+07', 'mean_token_accuracy': '0.7791', 'epoch': '0.04651'}
{'loss': '0.7376', 'grad_norm': '0.3161', 'learning_rate': '0.0001284', 'entropy': '0.8307', 'num_tokens': '2.843e+07', 'mean_token_accuracy': '0.7845', 'epoch': '0.04651'}
{'loss': '1.05', 'grad_norm': '0.3898', 'learning_rate': '0.0001284', 'entropy': '1.079', 'num_tokens': '2.844e+07', 'mean_token_accuracy': '0.7376', 'epoch': '0.04652'}
{'loss': '1.189', 'grad_norm': '0.4131', 'learning_rate': '0.0001283', 'entropy': '1.151', 'num_tokens': '2.844e+07', 'mean_token_accuracy': '0.741', 'epoch': '0.04653'}
{'loss': '1.148', 'grad_norm': '0.3972', 'learning_rate': '0.0001283', 'entropy': '1.034', 'num_tokens': '2.845e+07', 'mean_token_accuracy': '0.7461', 'epoch': '0.04654'}
{'loss': '0.7947', 'grad_norm': '0.3505', 'learning_rate': '0.0001283', 'entropy': '0.8632', 'num_tokens': '2.845e+07', 'mean_token_accuracy': '0.7859', 'epoch': '0.04655'}
{'loss': '0.9604', 'grad_norm': '0.4463', 'learning_rate': '0.0001283', 'entropy': '1.058', 'num_tokens': '2.846e+07', 'mean_token_accuracy': '0.7399', 'epoch': '0.04656'}
{'loss': '0.6739', 'grad_norm': '0.3207', 'learning_rate': '0.0001282', 'entropy': '0.7679', 'num_tokens': '2.847e+07', 'mean_token_accuracy': '0.8056', 'epoch': '0.04657'}
{'loss': '0.9946', 'grad_norm': '0.4109', 'learning_rate': '0.0001282', 'entropy': '0.9423', 'num_tokens': '2.847e+07', 'mean_token_accuracy': '0.7728', 'epoch': '0.04658'}
{'loss': '0.7903', 'grad_norm': '0.3808', 'learning_rate': '0.0001282', 'entropy': '0.9942', 'num_tokens': '2.848e+07', 'mean_token_accuracy': '0.7656', 'epoch': '0.04659'}
{'loss': '1.028', 'grad_norm': '0.4964', 'learning_rate': '0.0001282', 'entropy': '0.982', 'num_tokens': '2.848e+07', 'mean_token_accuracy': '0.7753', 'epoch': '0.04659'}
{'loss': '1.003', 'grad_norm': '0.3879', 'learning_rate': '0.0001281', 'entropy': '0.8974', 'num_tokens': '2.849e+07', 'mean_token_accuracy': '0.7841', 'epoch': '0.0466'}
{'loss': '1.12', 'grad_norm': '0.3819', 'learning_rate': '0.0001281', 'entropy': '1.008', 'num_tokens': '2.85e+07', 'mean_token_accuracy': '0.7597', 'epoch': '0.04661'}

 44%|████▍     | 5268/12000 [3:07:55<5:54:16,  3.16s/it]
 44%|████▍     | 5269/12000 [3:08:02<8:03:12,  4.31s/it]
                                                        

 44%|████▍     | 5269/12000 [3:08:02<8:03:12,  4.31s/it]
 44%|████▍     | 5270/12000 [3:08:04<6:30:04,  3.48s/it]
                                                        

 44%|████▍     | 5270/12000 [3:08:04<6:30:04,  3.48s/it]
 44%|████▍     | 5271/12000 [3:08:06<5:32:28,  2.96s/it]
                                                        

 44%|████▍     | 5271/12000 [3:08:06<5:32:28,  2.96s/it]
 44%|████▍     | 5272/12000 [3:08:09<5:51:58,  3.14s/it]
                                                        

 44%|████▍     | 5272/12000 [3:08:09<5:51:58,  3.14s/it]
 44%|████▍     | 5273/12000 [3:08:15<7:12:51,  3.86s/it]
                                                        

 44%|████▍     | 5273/12000 [3:08:15<7:12:51,  3.86s/it]
 44%|████▍     | 5274/12000 [3:08:18<6:42:31,  3.59s/it]
                                                        

 44%|████▍     | 5274/12000 [3:08:18<6:42:31,  3.59s/it]
 44%|████▍     | 5275/12000 [3:08:20<6:19:42,  3.39s/it]
                                                        

 44%|████▍     | 5275/12000 [3:08:20<6:19:42,  3.39s/it]
 44%|████▍     | 5276/12000 [3:08:22<5:29:03,  2.94s/it]
                                                        

 44%|████▍     | 5276/12000 [3:08:22<5:29:03,  2.94s/it]
 44%|████▍     | 5277/12000 [3:08:24<4:40:00,  2.50s/it]
                                                        

 44%|████▍     | 5277/12000 [3:08:24<4:40:00,  2.50s/it]
 44%|████▍     | 5278/12000 [3:08:29<5:56:29,  3.18s/it]
                                                        

 44%|████▍     | 5278/12000 [3:08:29<5:56:29,  3.18s/it]
 44%|████▍     | 5279/12000 [3:08:30<4:54:40,  2.63s/it]
                                                        

 44%|████▍     | 5279/12000 [3:08:30<4:54:40,  2.63s/it]
 44%|████▍     | 5280/12000 [3:08:32<4:18:57,  2.31s/it]
                                                        

 44%|████▍     | 5280/12000 [3:08:32<4:18:57,  2.31s/it]
 44%|████▍     | 5281/12000 [3:08:33<3:58:26,  2.13s/it]
                                                        

 44%|████▍     | 5281/12000 [3:08:33<3:58:26,  2.13s/it]
 44%|████▍     | 5282/12000 [3:08:36<4:05:34,  2.19s/it]
                                                        

 44%|████▍     | 5282/12000 [3:08:36<4:05:34,  2.19s/it]
 44%|████▍     | 5283/12000 [3:08:37<3:50:47,  2.06s/it]
                                                        

 44%|████▍     | 5283/12000 [3:08:37<3:50:47,  2.06s/it]
 44%|████▍     | 5284/12000 [3:08:39<3:25:36,  1.84s/it]
                                                        

 44%|████▍     | 5284/12000 [3:08:39<3:25:36,  1.84s/it]
 44%|████▍     | 5285/12000 [3:08:41<3:33:00,  1.90s/it]
                                                        

 44%|████▍     | 5285/12000 [3:08:41<3:33:00,  1.90s/it]
 44%|████▍     | 5286/12000 [3:08:43<3:29:39,  1.87s/it]
                                                        

 44%|████▍     | 5286/12000 [3:08:43<3:29:39,  1.87s/it]
 44%|████▍     | 5287/12000 [3:08:44<3:28:08,  1.86s/it]
                                                        

 44%|████▍     | 5287/12000 [3:08:44<3:28:08,  1.86s/it]
 44%|████▍     | 5288/12000 [3:08:46<3:11:08,  1.71s/it]
                                                        

 44%|████▍     | 5288/12000 [3:08:46<3:11:08,  1.71s/it]
 44%|████▍     | 5289/12000 [3:08:47<3:04:55,  1.65s/it]
                                                        

 44%|████▍     | 5289/12000 [3:08:47<3:04:55,  1.65s/it]
 44%|████▍     | 5290/12000 [3:08:49<2:59:05,  1.60s/it]
                                                        

 44%|████▍     | 5290/12000 [3:08:49<2:59:05,  1.60s/it]
 44%|████▍     | 5291/12000 [3:08:50<3:00:40,  1.62s/it]
                                                        

 44%|████▍     | 5291/12000 [3:08:50<3:00:40,  1.62s/it]
 44%|████▍     | 5292/12000 [3:08:52<3:06:54,  1.67s/it]
                                                        

 44%|████▍     | 5292/12000 [3:08:52<3:06:54,  1.67s/it]
 44%|████▍     | 5293/12000 [3:08:54<3:04:22,  1.65s/it]
                                                        

 44%|████▍     | 5293/12000 [3:08:54<3:04:22,  1.65s/it]
 44%|████▍     | 5294/12000 [3:08:57<4:05:52,  2.20s/it]
                                                        

 44%|████▍     | 5294/12000 [3:08:57<4:05:52,  2.20s/it]
 44%|████▍     | 5295/12000 [3:08:59<3:45:57,  2.02s/it]
                                                        

 44%|████▍     | 5295/12000 [3:08:59<3:45:57,  2.02s/it]
 44%|████▍     | 5296/12000 [3:09:01<3:34:47,  1.92s/it]
                                                        

 44%|████▍     | 5296/12000 [3:09:01<3:34:47,  1.92s/it]
 44%|████▍     | 5297/12000 [3:09:02<3:30:06,  1.88s/it]
                                                        

 44%|████▍     | 5297/12000 [3:09:02<3:30:06,  1.88s/it]
 44%|████▍     | 5298/12000 [3:09:04<3:18:49,  1.78s/it]
                                                        

 44%|████▍     | 5298/12000 [3:09:04<3:18:49,  1.78s/it]
 44%|████▍     | 5299/12000 [3:09:06<3:23:48,  1.82s/it]
                                                        

 44%|████▍     | 5299/12000 [3:09:06<3:23:48,  1.82s/it]
 44%|████▍     | 5300/12000 [3:09:08<3:31:28,  1.89s/it]
                                                        

 44%|████▍     | 5300/12000 [3:09:08<3:31:28,  1.89s/it]
 44%|████▍     | 5301/12000 [3:09:11<4:04:14,  2.19s/it]
                                                        

 44%|████▍     | 5301/12000 [3:09:11<4:04:14,  2.19s/it]
 44%|████▍     | 5302/12000 [3:09:12<3:46:58,  2.03s/it]
                                                        

 44%|████▍     | 5302/12000 [3:09:12<3:46:58,  2.03s/it]
 44%|████▍     | 5303/12000 [3:09:14<3:30:01,  1.88s/it]
                                                        

 44%|████▍     | 5303/12000 [3:09:14<3:30:01,  1.88s/it]
 44%|████▍     | 5304/12000 [3:09:15<3:16:39,  1.76s/it]
                                                        

 44%|████▍     | 5304/12000 [3:09:15<3:16:39,  1.76s/it]
 44%|████▍     | 5305/12000 [3:09:17<3:14:16,  1.74s/it]
                                                        

 44%|████▍     | 5305/12000 [3:09:17<3:14:16,  1.74s/it]
 44%|████▍     | 5306/12000 [3:09:19<3:24:02,  1.83s/it]
                                                        

 44%|████▍     | 5306/12000 [3:09:19<3:24:02,  1.83s/it]
 44%|████▍     | 5307/12000 [3:09:21<3:14:24,  1.74s/it]
                                                        

 44%|████▍     | 5307/12000 [3:09:21<3:14:24,  1.74s/it]
 44%|████▍     | 5308/12000 [3:09:23<3:27:07,  1.86s/it]
                                                        

 44%|████▍     | 5308/12000 [3:09:23<3:27:07,  1.86s/it]
 44%|████▍     | 5309/12000 [3:09:25<3:23:56,  1.83s/it]
                                                        

 44%|████▍     | 5309/12000 [3:09:25<3:23:56,  1.83s/it]
 44%|████▍     | 5310/12000 [3:09:27<3:51:27,  2.08s/it]
                                                        

 44%|████▍     | 5310/12000 [3:09:27<3:51:27,  2.08s/it]
 44%|████▍     | 5311/12000 [3:09:29<3:55:47,  2.11s/it]
                                                        

 44%|████▍     | 5311/12000 [3:09:29<3:55:47,  2.11s/it]
 44%|████▍     | 5312/12000 [3:09:31<3:35:28,  1.93s/it]
                                                        

 44%|████▍     | 5312/12000 [3:09:31<3:35:28,  1.93s/it]
 44%|████▍     | 5313/12000 [3:09:32<3:20:21,  1.80s/it]
                                                        

 44%|████▍     | 5313/12000 [3:09:32<3:20:21,  1.80s/it]
 44%|████▍     | 5314/12000 [3:09:34<3:15:33,  1.75s/it]
                                                        

 44%|████▍     | 5314/12000 [3:09:34<3:15:33,  1.75s/it]
 44%|████▍     | 5315/12000 [3:09:38<4:41:57,  2.53s/it]
                                                        
{'loss': '0.7903', 'grad_norm': '0.3387', 'learning_rate': '0.0001281', 'entropy': '0.8209', 'num_tokens': '2.85e+07', 'mean_token_accuracy': '0.7991', 'epoch': '0.04662'}
{'loss': '0.7419', 'grad_norm': '0.3898', 'learning_rate': '0.0001281', 'entropy': '0.7677', 'num_tokens': '2.851e+07', 'mean_token_accuracy': '0.8197', 'epoch': '0.04663'}
{'loss': '1.069', 'grad_norm': '0.4406', 'learning_rate': '0.000128', 'entropy': '1.087', 'num_tokens': '2.851e+07', 'mean_token_accuracy': '0.7351', 'epoch': '0.04664'}
{'loss': '0.8177', 'grad_norm': '0.4022', 'learning_rate': '0.000128', 'entropy': '0.8639', 'num_tokens': '2.852e+07', 'mean_token_accuracy': '0.7874', 'epoch': '0.04665'}
{'loss': '1.206', 'grad_norm': '0.5695', 'learning_rate': '0.000128', 'entropy': '1.149', 'num_tokens': '2.852e+07', 'mean_token_accuracy': '0.7272', 'epoch': '0.04666'}
{'loss': '0.7613', 'grad_norm': '0.278', 'learning_rate': '0.000128', 'entropy': '0.8282', 'num_tokens': '2.853e+07', 'mean_token_accuracy': '0.79', 'epoch': '0.04667'}
{'loss': '0.8152', 'grad_norm': '0.4562', 'learning_rate': '0.0001279', 'entropy': '0.8002', 'num_tokens': '2.854e+07', 'mean_token_accuracy': '0.7975', 'epoch': '0.04667'}
{'loss': '0.7888', 'grad_norm': '0.3318', 'learning_rate': '0.0001279', 'entropy': '0.8179', 'num_tokens': '2.855e+07', 'mean_token_accuracy': '0.8016', 'epoch': '0.04668'}
{'loss': '0.9867', 'grad_norm': '0.4302', 'learning_rate': '0.0001279', 'entropy': '1.101', 'num_tokens': '2.855e+07', 'mean_token_accuracy': '0.7266', 'epoch': '0.04669'}
{'loss': '0.881', 'grad_norm': '0.4275', 'learning_rate': '0.0001278', 'entropy': '0.7523', 'num_tokens': '2.856e+07', 'mean_token_accuracy': '0.815', 'epoch': '0.0467'}
{'loss': '1.353', 'grad_norm': '0.3978', 'learning_rate': '0.0001278', 'entropy': '1.008', 'num_tokens': '2.856e+07', 'mean_token_accuracy': '0.7658', 'epoch': '0.04671'}
{'loss': '0.8637', 'grad_norm': '0.5096', 'learning_rate': '0.0001278', 'entropy': '0.8865', 'num_tokens': '2.857e+07', 'mean_token_accuracy': '0.773', 'epoch': '0.04672'}
{'loss': '0.8439', 'grad_norm': '0.4453', 'learning_rate': '0.0001278', 'entropy': '0.7879', 'num_tokens': '2.857e+07', 'mean_token_accuracy': '0.8052', 'epoch': '0.04673'}
{'loss': '0.6149', 'grad_norm': '0.3814', 'learning_rate': '0.0001277', 'entropy': '0.6337', 'num_tokens': '2.858e+07', 'mean_token_accuracy': '0.8401', 'epoch': '0.04674'}
{'loss': '0.884', 'grad_norm': '0.387', 'learning_rate': '0.0001277', 'entropy': '0.9279', 'num_tokens': '2.858e+07', 'mean_token_accuracy': '0.7767', 'epoch': '0.04674'}
{'loss': '0.9568', 'grad_norm': '0.3317', 'learning_rate': '0.0001277', 'entropy': '0.9521', 'num_tokens': '2.859e+07', 'mean_token_accuracy': '0.7685', 'epoch': '0.04675'}
{'loss': '0.9099', 'grad_norm': '0.4726', 'learning_rate': '0.0001277', 'entropy': '1.064', 'num_tokens': '2.859e+07', 'mean_token_accuracy': '0.7356', 'epoch': '0.04676'}
{'loss': '0.9061', 'grad_norm': '0.4017', 'learning_rate': '0.0001276', 'entropy': '0.8992', 'num_tokens': '2.859e+07', 'mean_token_accuracy': '0.7562', 'epoch': '0.04677'}
{'loss': '0.8147', 'grad_norm': '0.4033', 'learning_rate': '0.0001276', 'entropy': '0.7967', 'num_tokens': '2.86e+07', 'mean_token_accuracy': '0.798', 'epoch': '0.04678'}
{'loss': '0.7727', 'grad_norm': '0.3458', 'learning_rate': '0.0001276', 'entropy': '0.7531', 'num_tokens': '2.86e+07', 'mean_token_accuracy': '0.8011', 'epoch': '0.04679'}
{'loss': '0.9162', 'grad_norm': '0.3683', 'learning_rate': '0.0001276', 'entropy': '0.9769', 'num_tokens': '2.861e+07', 'mean_token_accuracy': '0.7583', 'epoch': '0.0468'}
{'loss': '0.7926', 'grad_norm': '0.3708', 'learning_rate': '0.0001275', 'entropy': '0.8492', 'num_tokens': '2.861e+07', 'mean_token_accuracy': '0.7948', 'epoch': '0.04681'}
{'loss': '0.6291', 'grad_norm': '0.3588', 'learning_rate': '0.0001275', 'entropy': '0.6278', 'num_tokens': '2.862e+07', 'mean_token_accuracy': '0.8384', 'epoch': '0.04682'}
{'loss': '0.8658', 'grad_norm': '0.3801', 'learning_rate': '0.0001275', 'entropy': '0.8345', 'num_tokens': '2.862e+07', 'mean_token_accuracy': '0.7912', 'epoch': '0.04682'}
{'loss': '0.7991', 'grad_norm': '0.3442', 'learning_rate': '0.0001274', 'entropy': '0.761', 'num_tokens': '2.863e+07', 'mean_token_accuracy': '0.8081', 'epoch': '0.04683'}
{'loss': '0.7468', 'grad_norm': '0.3726', 'learning_rate': '0.0001274', 'entropy': '0.7567', 'num_tokens': '2.863e+07', 'mean_token_accuracy': '0.7982', 'epoch': '0.04684'}
{'loss': '0.8998', 'grad_norm': '0.4604', 'learning_rate': '0.0001274', 'entropy': '0.9789', 'num_tokens': '2.864e+07', 'mean_token_accuracy': '0.7647', 'epoch': '0.04685'}
{'loss': '0.818', 'grad_norm': '0.4517', 'learning_rate': '0.0001274', 'entropy': '0.8929', 'num_tokens': '2.864e+07', 'mean_token_accuracy': '0.7782', 'epoch': '0.04686'}
{'loss': '1.037', 'grad_norm': '0.3969', 'learning_rate': '0.0001273', 'entropy': '1.002', 'num_tokens': '2.864e+07', 'mean_token_accuracy': '0.7516', 'epoch': '0.04687'}
{'loss': '1.139', 'grad_norm': '0.4248', 'learning_rate': '0.0001273', 'entropy': '1.088', 'num_tokens': '2.865e+07', 'mean_token_accuracy': '0.7331', 'epoch': '0.04688'}
{'loss': '0.7957', 'grad_norm': '0.4485', 'learning_rate': '0.0001273', 'entropy': '0.8269', 'num_tokens': '2.865e+07', 'mean_token_accuracy': '0.7817', 'epoch': '0.04689'}
{'loss': '0.684', 'grad_norm': '0.3609', 'learning_rate': '0.0001273', 'entropy': '0.6163', 'num_tokens': '2.866e+07', 'mean_token_accuracy': '0.8431', 'epoch': '0.0469'}
{'loss': '1.211', 'grad_norm': '0.5128', 'learning_rate': '0.0001272', 'entropy': '1.218', 'num_tokens': '2.866e+07', 'mean_token_accuracy': '0.7082', 'epoch': '0.0469'}
{'loss': '0.5889', 'grad_norm': '0.3734', 'learning_rate': '0.0001272', 'entropy': '0.5845', 'num_tokens': '2.867e+07', 'mean_token_accuracy': '0.8422', 'epoch': '0.04691'}
{'loss': '0.836', 'grad_norm': '0.3459', 'learning_rate': '0.0001272', 'entropy': '0.8606', 'num_tokens': '2.867e+07', 'mean_token_accuracy': '0.7869', 'epoch': '0.04692'}
{'loss': '1.205', 'grad_norm': '0.5039', 'learning_rate': '0.0001272', 'entropy': '1.223', 'num_tokens': '2.868e+07', 'mean_token_accuracy': '0.6949', 'epoch': '0.04693'}
{'loss': '0.8874', 'grad_norm': '0.3764', 'learning_rate': '0.0001271', 'entropy': '0.9144', 'num_tokens': '2.868e+07', 'mean_token_accuracy': '0.7767', 'epoch': '0.04694'}
{'loss': '0.8436', 'grad_norm': '0.4596', 'learning_rate': '0.0001271', 'entropy': '0.8701', 'num_tokens': '2.869e+07', 'mean_token_accuracy': '0.7888', 'epoch': '0.04695'}
{'loss': '0.9843', 'grad_norm': '0.412', 'learning_rate': '0.0001271', 'entropy': '1.05', 'num_tokens': '2.869e+07', 'mean_token_accuracy': '0.7584', 'epoch': '0.04696'}
{'loss': '0.7951', 'grad_norm': '0.4075', 'learning_rate': '0.0001271', 'entropy': '0.8444', 'num_tokens': '2.87e+07', 'mean_token_accuracy': '0.8077', 'epoch': '0.04697'}
{'loss': '0.8687', 'grad_norm': '0.4205', 'learning_rate': '0.000127', 'entropy': '0.8865', 'num_tokens': '2.871e+07', 'mean_token_accuracy': '0.7827', 'epoch': '0.04697'}
{'loss': '0.8498', 'grad_norm': '0.366', 'learning_rate': '0.000127', 'entropy': '0.8443', 'num_tokens': '2.871e+07', 'mean_token_accuracy': '0.7874', 'epoch': '0.04698'}
{'loss': '0.7473', 'grad_norm': '0.3274', 'learning_rate': '0.000127', 'entropy': '0.8146', 'num_tokens': '2.872e+07', 'mean_token_accuracy': '0.7944', 'epoch': '0.04699'}
{'loss': '0.8115', 'grad_norm': '0.3258', 'learning_rate': '0.0001269', 'entropy': '0.909', 'num_tokens': '2.872e+07', 'mean_token_accuracy': '0.783', 'epoch': '0.047'}
{'loss': '0.7647', 'grad_norm': '0.3855', 'learning_rate': '0.0001269', 'entropy': '0.7776', 'num_tokens': '2.873e+07', 'mean_token_accuracy': '0.807', 'epoch': '0.04701'}
{'loss': '0.8099', 'grad_norm': '0.4628', 'learning_rate': '0.0001269', 'entropy': '0.8848', 'num_tokens': '2.873e+07', 'mean_token_accuracy': '0.7838', 'epoch': '0.04702'}
{'loss': '0.7978', 'grad_norm': '0.419', 'learning_rate': '0.0001269', 'entropy': '0.8707', 'num_tokens': '2.874e+07', 'mean_token_accuracy': '0.7856', 'epoch': '0.04703'}

 44%|████▍     | 5315/12000 [3:09:38<4:41:57,  2.53s/it]
 44%|████▍     | 5316/12000 [3:09:42<5:18:31,  2.86s/it]
                                                        

 44%|████▍     | 5316/12000 [3:09:42<5:18:31,  2.86s/it]
 44%|████▍     | 5317/12000 [3:09:44<4:32:15,  2.44s/it]
                                                        

 44%|████▍     | 5317/12000 [3:09:44<4:32:15,  2.44s/it]
 44%|████▍     | 5318/12000 [3:09:46<4:34:45,  2.47s/it]
                                                        

 44%|████▍     | 5318/12000 [3:09:46<4:34:45,  2.47s/it]
 44%|████▍     | 5319/12000 [3:09:47<4:00:04,  2.16s/it]
                                                        

 44%|████▍     | 5319/12000 [3:09:47<4:00:04,  2.16s/it]
 44%|████▍     | 5320/12000 [3:09:49<3:39:47,  1.97s/it]
                                                        

 44%|████▍     | 5320/12000 [3:09:49<3:39:47,  1.97s/it]
 44%|████▍     | 5321/12000 [3:09:50<3:18:44,  1.79s/it]
                                                        

 44%|████▍     | 5321/12000 [3:09:50<3:18:44,  1.79s/it]
 44%|████▍     | 5322/12000 [3:09:52<3:13:40,  1.74s/it]
                                                        

 44%|████▍     | 5322/12000 [3:09:52<3:13:40,  1.74s/it]
 44%|████▍     | 5323/12000 [3:09:54<3:16:33,  1.77s/it]
                                                        

 44%|████▍     | 5323/12000 [3:09:54<3:16:33,  1.77s/it]
 44%|████▍     | 5324/12000 [3:09:55<3:06:14,  1.67s/it]
                                                        

 44%|████▍     | 5324/12000 [3:09:55<3:06:14,  1.67s/it]
 44%|████▍     | 5325/12000 [3:09:57<3:13:10,  1.74s/it]
                                                        

 44%|████▍     | 5325/12000 [3:09:57<3:13:10,  1.74s/it]
 44%|████▍     | 5326/12000 [3:10:00<3:45:09,  2.02s/it]
                                                        

 44%|████▍     | 5326/12000 [3:10:00<3:45:09,  2.02s/it]
 44%|████▍     | 5327/12000 [3:10:02<3:41:40,  1.99s/it]
                                                        

 44%|████▍     | 5327/12000 [3:10:02<3:41:40,  1.99s/it]
 44%|████▍     | 5328/12000 [3:10:04<3:38:04,  1.96s/it]
                                                        

 44%|████▍     | 5328/12000 [3:10:04<3:38:04,  1.96s/it]
 44%|████▍     | 5329/12000 [3:10:05<3:26:24,  1.86s/it]
                                                        

 44%|████▍     | 5329/12000 [3:10:05<3:26:24,  1.86s/it]
 44%|████▍     | 5330/12000 [3:10:08<4:01:08,  2.17s/it]
                                                        

 44%|████▍     | 5330/12000 [3:10:08<4:01:08,  2.17s/it]
 44%|████▍     | 5331/12000 [3:10:10<3:49:13,  2.06s/it]
                                                        

 44%|████▍     | 5331/12000 [3:10:10<3:49:13,  2.06s/it]
 44%|████▍     | 5332/12000 [3:10:12<3:44:03,  2.02s/it]
                                                        

 44%|████▍     | 5332/12000 [3:10:12<3:44:03,  2.02s/it]
 44%|████▍     | 5333/12000 [3:10:13<3:27:03,  1.86s/it]
                                                        

 44%|████▍     | 5333/12000 [3:10:13<3:27:03,  1.86s/it]
 44%|████▍     | 5334/12000 [3:10:16<3:57:36,  2.14s/it]
                                                        

 44%|████▍     | 5334/12000 [3:10:16<3:57:36,  2.14s/it]
 44%|████▍     | 5335/12000 [3:10:18<3:37:08,  1.95s/it]
                                                        

 44%|████▍     | 5335/12000 [3:10:18<3:37:08,  1.95s/it]
 44%|████▍     | 5336/12000 [3:10:20<3:43:58,  2.02s/it]
                                                        

 44%|████▍     | 5336/12000 [3:10:20<3:43:58,  2.02s/it]
 44%|████▍     | 5337/12000 [3:10:21<3:26:39,  1.86s/it]
                                                        

 44%|████▍     | 5337/12000 [3:10:21<3:26:39,  1.86s/it]
 44%|████▍     | 5338/12000 [3:10:25<4:17:15,  2.32s/it]
                                                        

 44%|████▍     | 5338/12000 [3:10:25<4:17:15,  2.32s/it]
 44%|████▍     | 5339/12000 [3:10:28<4:52:52,  2.64s/it]
                                                        

 44%|████▍     | 5339/12000 [3:10:28<4:52:52,  2.64s/it]
 44%|████▍     | 5340/12000 [3:10:30<4:16:01,  2.31s/it]
                                                        

 44%|████▍     | 5340/12000 [3:10:30<4:16:01,  2.31s/it]
 45%|████▍     | 5341/12000 [3:10:32<4:03:05,  2.19s/it]
                                                        

 45%|████▍     | 5341/12000 [3:10:32<4:03:05,  2.19s/it]
 45%|████▍     | 5342/12000 [3:10:33<3:53:16,  2.10s/it]
                                                        

 45%|████▍     | 5342/12000 [3:10:33<3:53:16,  2.10s/it]
 45%|████▍     | 5343/12000 [3:10:35<3:33:11,  1.92s/it]
                                                        

 45%|████▍     | 5343/12000 [3:10:35<3:33:11,  1.92s/it]
 45%|████▍     | 5344/12000 [3:10:38<4:15:21,  2.30s/it]
                                                        

 45%|████▍     | 5344/12000 [3:10:38<4:15:21,  2.30s/it]
 45%|████▍     | 5345/12000 [3:10:40<4:05:16,  2.21s/it]
                                                        

 45%|████▍     | 5345/12000 [3:10:40<4:05:16,  2.21s/it]
 45%|████▍     | 5346/12000 [3:10:43<4:15:26,  2.30s/it]
                                                        

 45%|████▍     | 5346/12000 [3:10:43<4:15:26,  2.30s/it]
 45%|████▍     | 5347/12000 [3:10:44<3:42:42,  2.01s/it]
                                                        

 45%|████▍     | 5347/12000 [3:10:44<3:42:42,  2.01s/it]
 45%|████▍     | 5348/12000 [3:10:46<3:27:02,  1.87s/it]
                                                        

 45%|████▍     | 5348/12000 [3:10:46<3:27:02,  1.87s/it]
 45%|████▍     | 5349/12000 [3:10:47<3:09:25,  1.71s/it]
                                                        

 45%|████▍     | 5349/12000 [3:10:47<3:09:25,  1.71s/it]
 45%|████▍     | 5350/12000 [3:10:50<3:49:55,  2.07s/it]
                                                        

 45%|████▍     | 5350/12000 [3:10:50<3:49:55,  2.07s/it]
 45%|████▍     | 5351/12000 [3:10:53<4:16:01,  2.31s/it]
                                                        

 45%|████▍     | 5351/12000 [3:10:53<4:16:01,  2.31s/it]
 45%|████▍     | 5352/12000 [3:10:54<3:49:22,  2.07s/it]
                                                        

 45%|████▍     | 5352/12000 [3:10:54<3:49:22,  2.07s/it]
 45%|████▍     | 5353/12000 [3:10:56<3:30:52,  1.90s/it]
                                                        

 45%|████▍     | 5353/12000 [3:10:56<3:30:52,  1.90s/it]
 45%|████▍     | 5354/12000 [3:10:57<3:20:08,  1.81s/it]
                                                        

 45%|████▍     | 5354/12000 [3:10:57<3:20:08,  1.81s/it]
 45%|████▍     | 5355/12000 [3:10:59<3:15:17,  1.76s/it]
                                                        

 45%|████▍     | 5355/12000 [3:10:59<3:15:17,  1.76s/it]
 45%|████▍     | 5356/12000 [3:11:01<3:19:10,  1.80s/it]
                                                        

 45%|████▍     | 5356/12000 [3:11:01<3:19:10,  1.80s/it]
 45%|████▍     | 5357/12000 [3:11:02<3:00:13,  1.63s/it]
                                                        

 45%|████▍     | 5357/12000 [3:11:02<3:00:13,  1.63s/it]
 45%|████▍     | 5358/12000 [3:11:04<3:01:10,  1.64s/it]
                                                        

 45%|████▍     | 5358/12000 [3:11:04<3:01:10,  1.64s/it]
 45%|████▍     | 5359/12000 [3:11:06<3:34:59,  1.94s/it]
                                                        

 45%|████▍     | 5359/12000 [3:11:06<3:34:59,  1.94s/it]
 45%|████▍     | 5360/12000 [3:11:10<4:19:30,  2.34s/it]
                                                        

 45%|████▍     | 5360/12000 [3:11:10<4:19:30,  2.34s/it]
 45%|████▍     | 5361/12000 [3:11:11<3:54:26,  2.12s/it]
                                                        

 45%|████▍     | 5361/12000 [3:11:11<3:54:26,  2.12s/it]
 45%|████▍     | 5362/12000 [3:11:13<3:44:21,  2.03s/it]
                                                        
{'loss': '1.304', 'grad_norm': '0.3767', 'learning_rate': '0.0001268', 'entropy': '1.378', 'num_tokens': '2.875e+07', 'mean_token_accuracy': '0.7016', 'epoch': '0.04704'}
{'loss': '0.8137', 'grad_norm': '0.4114', 'learning_rate': '0.0001268', 'entropy': '0.8081', 'num_tokens': '2.875e+07', 'mean_token_accuracy': '0.7902', 'epoch': '0.04705'}
{'loss': '0.7744', 'grad_norm': '0.5512', 'learning_rate': '0.0001268', 'entropy': '0.811', 'num_tokens': '2.876e+07', 'mean_token_accuracy': '0.7829', 'epoch': '0.04705'}
{'loss': '0.543', 'grad_norm': '0.4262', 'learning_rate': '0.0001268', 'entropy': '0.586', 'num_tokens': '2.876e+07', 'mean_token_accuracy': '0.8434', 'epoch': '0.04706'}
{'loss': '0.7999', 'grad_norm': '0.5167', 'learning_rate': '0.0001267', 'entropy': '0.8089', 'num_tokens': '2.877e+07', 'mean_token_accuracy': '0.7881', 'epoch': '0.04707'}
{'loss': '0.7751', 'grad_norm': '0.3825', 'learning_rate': '0.0001267', 'entropy': '0.7989', 'num_tokens': '2.877e+07', 'mean_token_accuracy': '0.7971', 'epoch': '0.04708'}
{'loss': '1.007', 'grad_norm': '0.3907', 'learning_rate': '0.0001267', 'entropy': '0.9266', 'num_tokens': '2.877e+07', 'mean_token_accuracy': '0.7735', 'epoch': '0.04709'}
{'loss': '1.034', 'grad_norm': '0.354', 'learning_rate': '0.0001267', 'entropy': '1.02', 'num_tokens': '2.878e+07', 'mean_token_accuracy': '0.7496', 'epoch': '0.0471'}
{'loss': '1.003', 'grad_norm': '0.4215', 'learning_rate': '0.0001266', 'entropy': '0.9813', 'num_tokens': '2.879e+07', 'mean_token_accuracy': '0.7561', 'epoch': '0.04711'}
{'loss': '1.029', 'grad_norm': '0.4447', 'learning_rate': '0.0001266', 'entropy': '0.9002', 'num_tokens': '2.879e+07', 'mean_token_accuracy': '0.7807', 'epoch': '0.04712'}
{'loss': '0.8657', 'grad_norm': '0.3318', 'learning_rate': '0.0001266', 'entropy': '0.8686', 'num_tokens': '2.88e+07', 'mean_token_accuracy': '0.7889', 'epoch': '0.04713'}
{'loss': '0.9499', 'grad_norm': '0.3294', 'learning_rate': '0.0001265', 'entropy': '0.94', 'num_tokens': '2.88e+07', 'mean_token_accuracy': '0.7806', 'epoch': '0.04713'}
{'loss': '0.9621', 'grad_norm': '0.3239', 'learning_rate': '0.0001265', 'entropy': '0.9706', 'num_tokens': '2.881e+07', 'mean_token_accuracy': '0.7588', 'epoch': '0.04714'}
{'loss': '1.269', 'grad_norm': '0.4558', 'learning_rate': '0.0001265', 'entropy': '1.024', 'num_tokens': '2.881e+07', 'mean_token_accuracy': '0.7542', 'epoch': '0.04715'}
{'loss': '0.9281', 'grad_norm': '0.3931', 'learning_rate': '0.0001265', 'entropy': '0.7842', 'num_tokens': '2.882e+07', 'mean_token_accuracy': '0.8189', 'epoch': '0.04716'}
{'loss': '0.8507', 'grad_norm': '0.3181', 'learning_rate': '0.0001264', 'entropy': '1.056', 'num_tokens': '2.883e+07', 'mean_token_accuracy': '0.7486', 'epoch': '0.04717'}
{'loss': '0.7439', 'grad_norm': '0.4136', 'learning_rate': '0.0001264', 'entropy': '0.7361', 'num_tokens': '2.883e+07', 'mean_token_accuracy': '0.8072', 'epoch': '0.04718'}
{'loss': '0.875', 'grad_norm': '0.3823', 'learning_rate': '0.0001264', 'entropy': '0.951', 'num_tokens': '2.884e+07', 'mean_token_accuracy': '0.764', 'epoch': '0.04719'}
{'loss': '0.8342', 'grad_norm': '0.4673', 'learning_rate': '0.0001264', 'entropy': '0.8496', 'num_tokens': '2.884e+07', 'mean_token_accuracy': '0.7772', 'epoch': '0.0472'}
{'loss': '1.033', 'grad_norm': '0.546', 'learning_rate': '0.0001263', 'entropy': '0.9662', 'num_tokens': '2.885e+07', 'mean_token_accuracy': '0.7716', 'epoch': '0.04721'}
{'loss': '0.9662', 'grad_norm': '0.3897', 'learning_rate': '0.0001263', 'entropy': '1.126', 'num_tokens': '2.885e+07', 'mean_token_accuracy': '0.7418', 'epoch': '0.04721'}
{'loss': '1.064', 'grad_norm': '0.4029', 'learning_rate': '0.0001263', 'entropy': '1.093', 'num_tokens': '2.886e+07', 'mean_token_accuracy': '0.7282', 'epoch': '0.04722'}
{'loss': '0.6909', 'grad_norm': '0.3819', 'learning_rate': '0.0001263', 'entropy': '0.7322', 'num_tokens': '2.886e+07', 'mean_token_accuracy': '0.8191', 'epoch': '0.04723'}
{'loss': '0.6744', 'grad_norm': '0.3396', 'learning_rate': '0.0001262', 'entropy': '0.6409', 'num_tokens': '2.886e+07', 'mean_token_accuracy': '0.8394', 'epoch': '0.04724'}
{'loss': '0.9195', 'grad_norm': '0.3952', 'learning_rate': '0.0001262', 'entropy': '1.096', 'num_tokens': '2.887e+07', 'mean_token_accuracy': '0.7433', 'epoch': '0.04725'}
{'loss': '0.8742', 'grad_norm': '0.4349', 'learning_rate': '0.0001262', 'entropy': '0.8767', 'num_tokens': '2.888e+07', 'mean_token_accuracy': '0.7775', 'epoch': '0.04726'}
{'loss': '1.05', 'grad_norm': '0.4316', 'learning_rate': '0.0001261', 'entropy': '0.9795', 'num_tokens': '2.888e+07', 'mean_token_accuracy': '0.7684', 'epoch': '0.04727'}
{'loss': '0.699', 'grad_norm': '0.3217', 'learning_rate': '0.0001261', 'entropy': '0.7282', 'num_tokens': '2.889e+07', 'mean_token_accuracy': '0.8135', 'epoch': '0.04728'}
{'loss': '0.9931', 'grad_norm': '0.4565', 'learning_rate': '0.0001261', 'entropy': '0.9116', 'num_tokens': '2.889e+07', 'mean_token_accuracy': '0.7501', 'epoch': '0.04728'}
{'loss': '0.8651', 'grad_norm': '0.3702', 'learning_rate': '0.0001261', 'entropy': '0.8111', 'num_tokens': '2.89e+07', 'mean_token_accuracy': '0.7927', 'epoch': '0.04729'}
{'loss': '1.021', 'grad_norm': '0.4774', 'learning_rate': '0.000126', 'entropy': '0.9828', 'num_tokens': '2.891e+07', 'mean_token_accuracy': '0.756', 'epoch': '0.0473'}
{'loss': '0.992', 'grad_norm': '0.4659', 'learning_rate': '0.000126', 'entropy': '0.9162', 'num_tokens': '2.891e+07', 'mean_token_accuracy': '0.7779', 'epoch': '0.04731'}
{'loss': '1.093', 'grad_norm': '0.3909', 'learning_rate': '0.000126', 'entropy': '1.213', 'num_tokens': '2.892e+07', 'mean_token_accuracy': '0.7072', 'epoch': '0.04732'}
{'loss': '0.7208', 'grad_norm': '0.426', 'learning_rate': '0.000126', 'entropy': '0.7604', 'num_tokens': '2.892e+07', 'mean_token_accuracy': '0.8026', 'epoch': '0.04733'}
{'loss': '1.263', 'grad_norm': '0.4913', 'learning_rate': '0.0001259', 'entropy': '1.267', 'num_tokens': '2.892e+07', 'mean_token_accuracy': '0.7144', 'epoch': '0.04734'}
{'loss': '0.7199', 'grad_norm': '0.2855', 'learning_rate': '0.0001259', 'entropy': '0.7838', 'num_tokens': '2.893e+07', 'mean_token_accuracy': '0.8107', 'epoch': '0.04735'}
{'loss': '0.7057', 'grad_norm': '0.3383', 'learning_rate': '0.0001259', 'entropy': '0.722', 'num_tokens': '2.894e+07', 'mean_token_accuracy': '0.816', 'epoch': '0.04736'}
{'loss': '1.025', 'grad_norm': '0.4238', 'learning_rate': '0.0001259', 'entropy': '0.9724', 'num_tokens': '2.894e+07', 'mean_token_accuracy': '0.7743', 'epoch': '0.04736'}
{'loss': '0.6182', 'grad_norm': '0.3677', 'learning_rate': '0.0001258', 'entropy': '0.694', 'num_tokens': '2.895e+07', 'mean_token_accuracy': '0.833', 'epoch': '0.04737'}
{'loss': '0.8299', 'grad_norm': '0.4029', 'learning_rate': '0.0001258', 'entropy': '0.8513', 'num_tokens': '2.895e+07', 'mean_token_accuracy': '0.7881', 'epoch': '0.04738'}
{'loss': '0.8192', 'grad_norm': '0.3598', 'learning_rate': '0.0001258', 'entropy': '0.8887', 'num_tokens': '2.896e+07', 'mean_token_accuracy': '0.7776', 'epoch': '0.04739'}
{'loss': '0.6743', 'grad_norm': '0.3188', 'learning_rate': '0.0001257', 'entropy': '0.6963', 'num_tokens': '2.896e+07', 'mean_token_accuracy': '0.8334', 'epoch': '0.0474'}
{'loss': '0.6859', 'grad_norm': '0.4426', 'learning_rate': '0.0001257', 'entropy': '0.6895', 'num_tokens': '2.896e+07', 'mean_token_accuracy': '0.823', 'epoch': '0.04741'}
{'loss': '0.6648', 'grad_norm': '0.3194', 'learning_rate': '0.0001257', 'entropy': '0.6902', 'num_tokens': '2.897e+07', 'mean_token_accuracy': '0.8261', 'epoch': '0.04742'}
{'loss': '0.9114', 'grad_norm': '0.3137', 'learning_rate': '0.0001257', 'entropy': '0.9481', 'num_tokens': '2.898e+07', 'mean_token_accuracy': '0.7647', 'epoch': '0.04743'}
{'loss': '0.8358', 'grad_norm': '0.3632', 'learning_rate': '0.0001256', 'entropy': '0.8257', 'num_tokens': '2.898e+07', 'mean_token_accuracy': '0.7957', 'epoch': '0.04744'}
{'loss': '0.9803', 'grad_norm': '0.3897', 'learning_rate': '0.0001256', 'entropy': '1.013', 'num_tokens': '2.899e+07', 'mean_token_accuracy': '0.7575', 'epoch': '0.04744'}

 45%|████▍     | 5362/12000 [3:11:13<3:44:21,  2.03s/it]
 45%|████▍     | 5363/12000 [3:11:15<3:28:25,  1.88s/it]
                                                        

 45%|████▍     | 5363/12000 [3:11:15<3:28:25,  1.88s/it]
 45%|████▍     | 5364/12000 [3:11:20<5:23:16,  2.92s/it]
                                                        

 45%|████▍     | 5364/12000 [3:11:20<5:23:16,  2.92s/it]
 45%|████▍     | 5365/12000 [3:11:22<4:46:13,  2.59s/it]
                                                        

 45%|████▍     | 5365/12000 [3:11:22<4:46:13,  2.59s/it]
 45%|████▍     | 5366/12000 [3:11:24<4:42:26,  2.55s/it]
                                                        

 45%|████▍     | 5366/12000 [3:11:24<4:42:26,  2.55s/it]
 45%|████▍     | 5367/12000 [3:11:26<4:11:06,  2.27s/it]
                                                        

 45%|████▍     | 5367/12000 [3:11:26<4:11:06,  2.27s/it]
 45%|████▍     | 5368/12000 [3:11:27<3:44:38,  2.03s/it]
                                                        

 45%|████▍     | 5368/12000 [3:11:27<3:44:38,  2.03s/it]
 45%|████▍     | 5369/12000 [3:11:30<4:05:40,  2.22s/it]
                                                        

 45%|████▍     | 5369/12000 [3:11:30<4:05:40,  2.22s/it]
 45%|████▍     | 5370/12000 [3:11:31<3:34:45,  1.94s/it]
                                                        

 45%|████▍     | 5370/12000 [3:11:31<3:34:45,  1.94s/it]
 45%|████▍     | 5371/12000 [3:11:33<3:18:51,  1.80s/it]
                                                        

 45%|████▍     | 5371/12000 [3:11:33<3:18:51,  1.80s/it]
 45%|████▍     | 5372/12000 [3:11:34<3:12:27,  1.74s/it]
                                                        

 45%|████▍     | 5372/12000 [3:11:34<3:12:27,  1.74s/it]
 45%|████▍     | 5373/12000 [3:11:36<2:59:18,  1.62s/it]
                                                        

 45%|████▍     | 5373/12000 [3:11:36<2:59:18,  1.62s/it]
 45%|████▍     | 5374/12000 [3:11:37<2:54:21,  1.58s/it]
                                                        

 45%|████▍     | 5374/12000 [3:11:37<2:54:21,  1.58s/it]
 45%|████▍     | 5375/12000 [3:11:39<2:53:16,  1.57s/it]
                                                        

 45%|████▍     | 5375/12000 [3:11:39<2:53:16,  1.57s/it]
 45%|████▍     | 5376/12000 [3:11:40<2:57:00,  1.60s/it]
                                                        

 45%|████▍     | 5376/12000 [3:11:40<2:57:00,  1.60s/it]
 45%|████▍     | 5377/12000 [3:11:42<3:05:09,  1.68s/it]
                                                        

 45%|████▍     | 5377/12000 [3:11:42<3:05:09,  1.68s/it]
 45%|████▍     | 5378/12000 [3:11:44<3:07:01,  1.69s/it]
                                                        

 45%|████▍     | 5378/12000 [3:11:44<3:07:01,  1.69s/it]
 45%|████▍     | 5379/12000 [3:11:45<2:59:25,  1.63s/it]
                                                        

 45%|████▍     | 5379/12000 [3:11:45<2:59:25,  1.63s/it]
 45%|████▍     | 5380/12000 [3:11:47<3:00:39,  1.64s/it]
                                                        

 45%|████▍     | 5380/12000 [3:11:47<3:00:39,  1.64s/it]
 45%|████▍     | 5381/12000 [3:11:49<3:07:06,  1.70s/it]
                                                        

 45%|████▍     | 5381/12000 [3:11:49<3:07:06,  1.70s/it]
 45%|████▍     | 5382/12000 [3:11:52<4:04:07,  2.21s/it]
                                                        

 45%|████▍     | 5382/12000 [3:11:52<4:04:07,  2.21s/it]
 45%|████▍     | 5383/12000 [3:11:54<3:41:44,  2.01s/it]
                                                        

 45%|████▍     | 5383/12000 [3:11:54<3:41:44,  2.01s/it]
 45%|████▍     | 5384/12000 [3:11:56<3:32:37,  1.93s/it]
                                                        

 45%|████▍     | 5384/12000 [3:11:56<3:32:37,  1.93s/it]
 45%|████▍     | 5385/12000 [3:11:57<3:20:42,  1.82s/it]
                                                        

 45%|████▍     | 5385/12000 [3:11:57<3:20:42,  1.82s/it]
 45%|████▍     | 5386/12000 [3:11:59<3:06:25,  1.69s/it]
                                                        

 45%|████▍     | 5386/12000 [3:11:59<3:06:25,  1.69s/it]
 45%|████▍     | 5387/12000 [3:12:00<2:58:47,  1.62s/it]
                                                        

 45%|████▍     | 5387/12000 [3:12:00<2:58:47,  1.62s/it]
 45%|████▍     | 5388/12000 [3:12:02<3:00:56,  1.64s/it]
                                                        

 45%|████▍     | 5388/12000 [3:12:02<3:00:56,  1.64s/it]
 45%|████▍     | 5389/12000 [3:12:04<3:16:06,  1.78s/it]
                                                        

 45%|████▍     | 5389/12000 [3:12:04<3:16:06,  1.78s/it]
 45%|████▍     | 5390/12000 [3:12:07<4:11:23,  2.28s/it]
                                                        

 45%|████▍     | 5390/12000 [3:12:07<4:11:23,  2.28s/it]
 45%|████▍     | 5391/12000 [3:12:09<3:54:14,  2.13s/it]
                                                        

 45%|████▍     | 5391/12000 [3:12:09<3:54:14,  2.13s/it]
 45%|████▍     | 5392/12000 [3:12:11<4:00:00,  2.18s/it]
                                                        

 45%|████▍     | 5392/12000 [3:12:11<4:00:00,  2.18s/it]
 45%|████▍     | 5393/12000 [3:12:14<4:19:27,  2.36s/it]
                                                        

 45%|████▍     | 5393/12000 [3:12:14<4:19:27,  2.36s/it]
 45%|████▍     | 5394/12000 [3:12:16<3:50:34,  2.09s/it]
                                                        

 45%|████▍     | 5394/12000 [3:12:16<3:50:34,  2.09s/it]
 45%|████▍     | 5395/12000 [3:12:18<3:57:51,  2.16s/it]
                                                        

 45%|████▍     | 5395/12000 [3:12:18<3:57:51,  2.16s/it]
 45%|████▍     | 5396/12000 [3:12:20<3:47:34,  2.07s/it]
                                                        

 45%|████▍     | 5396/12000 [3:12:20<3:47:34,  2.07s/it]
 45%|████▍     | 5397/12000 [3:12:22<3:58:07,  2.16s/it]
                                                        

 45%|████▍     | 5397/12000 [3:12:22<3:58:07,  2.16s/it]
 45%|████▍     | 5398/12000 [3:12:24<3:43:32,  2.03s/it]
                                                        

 45%|████▍     | 5398/12000 [3:12:24<3:43:32,  2.03s/it]
 45%|████▍     | 5399/12000 [3:12:25<3:29:11,  1.90s/it]
                                                        

 45%|████▍     | 5399/12000 [3:12:25<3:29:11,  1.90s/it]
 45%|████▌     | 5400/12000 [3:12:27<3:15:55,  1.78s/it]
                                                        

 45%|████▌     | 5400/12000 [3:12:27<3:15:55,  1.78s/it]
 45%|████▌     | 5401/12000 [3:12:30<4:03:32,  2.21s/it]
                                                        

 45%|████▌     | 5401/12000 [3:12:30<4:03:32,  2.21s/it]
 45%|████▌     | 5402/12000 [3:12:32<3:53:21,  2.12s/it]
                                                        

 45%|████▌     | 5402/12000 [3:12:32<3:53:21,  2.12s/it]
 45%|████▌     | 5403/12000 [3:12:35<4:06:57,  2.25s/it]
                                                        

 45%|████▌     | 5403/12000 [3:12:35<4:06:57,  2.25s/it]
 45%|████▌     | 5404/12000 [3:12:38<4:43:09,  2.58s/it]
                                                        

 45%|████▌     | 5404/12000 [3:12:38<4:43:09,  2.58s/it]
 45%|████▌     | 5405/12000 [3:12:39<4:04:24,  2.22s/it]
                                                        

 45%|████▌     | 5405/12000 [3:12:39<4:04:24,  2.22s/it]
 45%|████▌     | 5406/12000 [3:12:41<3:48:06,  2.08s/it]
                                                        

 45%|████▌     | 5406/12000 [3:12:41<3:48:06,  2.08s/it]
 45%|████▌     | 5407/12000 [3:12:44<3:57:42,  2.16s/it]
                                                        

 45%|████▌     | 5407/12000 [3:12:44<3:57:42,  2.16s/it]
 45%|████▌     | 5408/12000 [3:12:45<3:34:00,  1.95s/it]
                                                        

 45%|████▌     | 5408/12000 [3:12:45<3:34:00,  1.95s/it]
 45%|████▌     | 5409/12000 [3:12:47<3:22:21,  1.84s/it]
                                                        
{'loss': '0.8743', 'grad_norm': '0.3842', 'learning_rate': '0.0001256', 'entropy': '0.8444', 'num_tokens': '2.899e+07', 'mean_token_accuracy': '0.7995', 'epoch': '0.04745'}
{'loss': '0.8972', 'grad_norm': '0.3626', 'learning_rate': '0.0001256', 'entropy': '0.9112', 'num_tokens': '2.9e+07', 'mean_token_accuracy': '0.7678', 'epoch': '0.04746'}
{'loss': '1.109', 'grad_norm': '0.3575', 'learning_rate': '0.0001255', 'entropy': '1.002', 'num_tokens': '2.901e+07', 'mean_token_accuracy': '0.7555', 'epoch': '0.04747'}
{'loss': '0.7857', 'grad_norm': '0.4281', 'learning_rate': '0.0001255', 'entropy': '0.7769', 'num_tokens': '2.901e+07', 'mean_token_accuracy': '0.8049', 'epoch': '0.04748'}
{'loss': '1.01', 'grad_norm': '0.4036', 'learning_rate': '0.0001255', 'entropy': '0.9015', 'num_tokens': '2.902e+07', 'mean_token_accuracy': '0.779', 'epoch': '0.04749'}
{'loss': '1.095', 'grad_norm': '0.361', 'learning_rate': '0.0001255', 'entropy': '0.996', 'num_tokens': '2.902e+07', 'mean_token_accuracy': '0.7534', 'epoch': '0.0475'}
{'loss': '1.139', 'grad_norm': '0.4408', 'learning_rate': '0.0001254', 'entropy': '0.9484', 'num_tokens': '2.903e+07', 'mean_token_accuracy': '0.7573', 'epoch': '0.04751'}
{'loss': '1.367', 'grad_norm': '0.4081', 'learning_rate': '0.0001254', 'entropy': '1.157', 'num_tokens': '2.903e+07', 'mean_token_accuracy': '0.7078', 'epoch': '0.04751'}
{'loss': '0.7854', 'grad_norm': '0.4389', 'learning_rate': '0.0001254', 'entropy': '0.7559', 'num_tokens': '2.904e+07', 'mean_token_accuracy': '0.7928', 'epoch': '0.04752'}
{'loss': '0.9501', 'grad_norm': '0.3923', 'learning_rate': '0.0001253', 'entropy': '0.8904', 'num_tokens': '2.904e+07', 'mean_token_accuracy': '0.7811', 'epoch': '0.04753'}
{'loss': '0.8676', 'grad_norm': '0.4248', 'learning_rate': '0.0001253', 'entropy': '0.9065', 'num_tokens': '2.905e+07', 'mean_token_accuracy': '0.7822', 'epoch': '0.04754'}
{'loss': '0.9996', 'grad_norm': '0.5034', 'learning_rate': '0.0001253', 'entropy': '1.05', 'num_tokens': '2.905e+07', 'mean_token_accuracy': '0.7496', 'epoch': '0.04755'}
{'loss': '0.8755', 'grad_norm': '0.3219', 'learning_rate': '0.0001253', 'entropy': '0.8196', 'num_tokens': '2.905e+07', 'mean_token_accuracy': '0.7835', 'epoch': '0.04756'}
{'loss': '0.7644', 'grad_norm': '0.3561', 'learning_rate': '0.0001252', 'entropy': '0.8207', 'num_tokens': '2.906e+07', 'mean_token_accuracy': '0.7801', 'epoch': '0.04757'}
{'loss': '0.7742', 'grad_norm': '0.3745', 'learning_rate': '0.0001252', 'entropy': '0.7446', 'num_tokens': '2.907e+07', 'mean_token_accuracy': '0.8073', 'epoch': '0.04758'}
{'loss': '0.7405', 'grad_norm': '0.3296', 'learning_rate': '0.0001252', 'entropy': '0.774', 'num_tokens': '2.907e+07', 'mean_token_accuracy': '0.8108', 'epoch': '0.04759'}
{'loss': '0.6422', 'grad_norm': '0.4003', 'learning_rate': '0.0001252', 'entropy': '0.6402', 'num_tokens': '2.908e+07', 'mean_token_accuracy': '0.8418', 'epoch': '0.04759'}
{'loss': '0.9803', 'grad_norm': '0.3525', 'learning_rate': '0.0001251', 'entropy': '0.9074', 'num_tokens': '2.908e+07', 'mean_token_accuracy': '0.7656', 'epoch': '0.0476'}
{'loss': '0.7149', 'grad_norm': '0.4038', 'learning_rate': '0.0001251', 'entropy': '0.7327', 'num_tokens': '2.908e+07', 'mean_token_accuracy': '0.8159', 'epoch': '0.04761'}
{'loss': '1.134', 'grad_norm': '0.4405', 'learning_rate': '0.0001251', 'entropy': '1.162', 'num_tokens': '2.909e+07', 'mean_token_accuracy': '0.7305', 'epoch': '0.04762'}
{'loss': '0.9875', 'grad_norm': '0.463', 'learning_rate': '0.0001251', 'entropy': '1.126', 'num_tokens': '2.909e+07', 'mean_token_accuracy': '0.7275', 'epoch': '0.04763'}
{'loss': '0.6311', 'grad_norm': '0.408', 'learning_rate': '0.000125', 'entropy': '0.6681', 'num_tokens': '2.91e+07', 'mean_token_accuracy': '0.8286', 'epoch': '0.04764'}
{'loss': '0.7619', 'grad_norm': '0.3408', 'learning_rate': '0.000125', 'entropy': '0.7684', 'num_tokens': '2.91e+07', 'mean_token_accuracy': '0.8112', 'epoch': '0.04765'}
{'loss': '0.8346', 'grad_norm': '0.4531', 'learning_rate': '0.000125', 'entropy': '0.881', 'num_tokens': '2.911e+07', 'mean_token_accuracy': '0.7775', 'epoch': '0.04766'}
{'loss': '1.027', 'grad_norm': '0.3482', 'learning_rate': '0.0001249', 'entropy': '1.078', 'num_tokens': '2.911e+07', 'mean_token_accuracy': '0.7534', 'epoch': '0.04767'}
{'loss': '0.6681', 'grad_norm': '0.3475', 'learning_rate': '0.0001249', 'entropy': '0.6507', 'num_tokens': '2.912e+07', 'mean_token_accuracy': '0.8135', 'epoch': '0.04767'}
{'loss': '0.9387', 'grad_norm': '0.4598', 'learning_rate': '0.0001249', 'entropy': '0.899', 'num_tokens': '2.912e+07', 'mean_token_accuracy': '0.7785', 'epoch': '0.04768'}
{'loss': '0.966', 'grad_norm': '0.3032', 'learning_rate': '0.0001249', 'entropy': '0.9892', 'num_tokens': '2.913e+07', 'mean_token_accuracy': '0.7454', 'epoch': '0.04769'}
{'loss': '0.7936', 'grad_norm': '0.3417', 'learning_rate': '0.0001248', 'entropy': '0.7388', 'num_tokens': '2.914e+07', 'mean_token_accuracy': '0.8207', 'epoch': '0.0477'}
{'loss': '0.7623', 'grad_norm': '0.4039', 'learning_rate': '0.0001248', 'entropy': '0.722', 'num_tokens': '2.914e+07', 'mean_token_accuracy': '0.8099', 'epoch': '0.04771'}
{'loss': '0.8645', 'grad_norm': '0.4069', 'learning_rate': '0.0001248', 'entropy': '0.8613', 'num_tokens': '2.915e+07', 'mean_token_accuracy': '0.781', 'epoch': '0.04772'}
{'loss': '0.7991', 'grad_norm': '0.3211', 'learning_rate': '0.0001248', 'entropy': '0.8692', 'num_tokens': '2.916e+07', 'mean_token_accuracy': '0.7916', 'epoch': '0.04773'}
{'loss': '0.7875', 'grad_norm': '0.3919', 'learning_rate': '0.0001247', 'entropy': '0.8709', 'num_tokens': '2.916e+07', 'mean_token_accuracy': '0.7761', 'epoch': '0.04774'}
{'loss': '1.113', 'grad_norm': '0.4962', 'learning_rate': '0.0001247', 'entropy': '1.115', 'num_tokens': '2.917e+07', 'mean_token_accuracy': '0.7273', 'epoch': '0.04774'}
{'loss': '0.8524', 'grad_norm': '0.657', 'learning_rate': '0.0001247', 'entropy': '0.8672', 'num_tokens': '2.917e+07', 'mean_token_accuracy': '0.774', 'epoch': '0.04775'}
{'loss': '0.8602', 'grad_norm': '0.3692', 'learning_rate': '0.0001247', 'entropy': '0.86', 'num_tokens': '2.918e+07', 'mean_token_accuracy': '0.7869', 'epoch': '0.04776'}
{'loss': '0.7848', 'grad_norm': '0.37', 'learning_rate': '0.0001246', 'entropy': '0.7932', 'num_tokens': '2.918e+07', 'mean_token_accuracy': '0.7964', 'epoch': '0.04777'}
{'loss': '0.8357', 'grad_norm': '0.3216', 'learning_rate': '0.0001246', 'entropy': '0.8347', 'num_tokens': '2.919e+07', 'mean_token_accuracy': '0.7911', 'epoch': '0.04778'}
{'loss': '0.81', 'grad_norm': '0.3955', 'learning_rate': '0.0001246', 'entropy': '0.894', 'num_tokens': '2.919e+07', 'mean_token_accuracy': '0.7811', 'epoch': '0.04779'}
{'loss': '0.7709', 'grad_norm': '0.3996', 'learning_rate': '0.0001245', 'entropy': '0.8568', 'num_tokens': '2.92e+07', 'mean_token_accuracy': '0.7848', 'epoch': '0.0478'}
{'loss': '1.082', 'grad_norm': '0.3616', 'learning_rate': '0.0001245', 'entropy': '1.066', 'num_tokens': '2.921e+07', 'mean_token_accuracy': '0.7371', 'epoch': '0.04781'}
{'loss': '0.8342', 'grad_norm': '0.4088', 'learning_rate': '0.0001245', 'entropy': '0.8918', 'num_tokens': '2.921e+07', 'mean_token_accuracy': '0.7882', 'epoch': '0.04782'}
{'loss': '0.7852', 'grad_norm': '0.3453', 'learning_rate': '0.0001245', 'entropy': '0.7733', 'num_tokens': '2.922e+07', 'mean_token_accuracy': '0.8002', 'epoch': '0.04782'}
{'loss': '0.7102', 'grad_norm': '0.3178', 'learning_rate': '0.0001244', 'entropy': '0.6948', 'num_tokens': '2.922e+07', 'mean_token_accuracy': '0.8166', 'epoch': '0.04783'}
{'loss': '0.6611', 'grad_norm': '0.4168', 'learning_rate': '0.0001244', 'entropy': '0.6737', 'num_tokens': '2.923e+07', 'mean_token_accuracy': '0.8194', 'epoch': '0.04784'}
{'loss': '0.8437', 'grad_norm': '0.3384', 'learning_rate': '0.0001244', 'entropy': '0.9086', 'num_tokens': '2.923e+07', 'mean_token_accuracy': '0.7782', 'epoch': '0.04785'}
{'loss': '0.7334', 'grad_norm': '0.3603', 'learning_rate': '0.0001244', 'entropy': '0.751', 'num_tokens': '2.924e+07', 'mean_token_accuracy': '0.8021', 'epoch': '0.04786'}

 45%|████▌     | 5409/12000 [3:12:47<3:22:21,  1.84s/it]
 45%|████▌     | 5410/12000 [3:12:48<3:15:20,  1.78s/it]
                                                        

 45%|████▌     | 5410/12000 [3:12:48<3:15:20,  1.78s/it]
 45%|████▌     | 5411/12000 [3:12:50<3:07:43,  1.71s/it]
                                                        

 45%|████▌     | 5411/12000 [3:12:50<3:07:43,  1.71s/it]
 45%|████▌     | 5412/12000 [3:12:51<3:06:06,  1.69s/it]
                                                        

 45%|████▌     | 5412/12000 [3:12:51<3:06:06,  1.69s/it]
 45%|████▌     | 5413/12000 [3:12:53<2:59:57,  1.64s/it]
                                                        

 45%|████▌     | 5413/12000 [3:12:53<2:59:57,  1.64s/it]
 45%|████▌     | 5414/12000 [3:12:54<2:47:31,  1.53s/it]
                                                        

 45%|████▌     | 5414/12000 [3:12:54<2:47:31,  1.53s/it]
 45%|████▌     | 5415/12000 [3:12:56<2:59:21,  1.63s/it]
                                                        

 45%|████▌     | 5415/12000 [3:12:56<2:59:21,  1.63s/it]
 45%|████▌     | 5416/12000 [3:12:59<3:32:15,  1.93s/it]
                                                        

 45%|████▌     | 5416/12000 [3:12:59<3:32:15,  1.93s/it]
 45%|████▌     | 5417/12000 [3:13:00<3:22:58,  1.85s/it]
                                                        

 45%|████▌     | 5417/12000 [3:13:00<3:22:58,  1.85s/it]
 45%|████▌     | 5418/12000 [3:13:02<3:14:00,  1.77s/it]
                                                        

 45%|████▌     | 5418/12000 [3:13:02<3:14:00,  1.77s/it]
 45%|████▌     | 5419/12000 [3:13:04<3:40:10,  2.01s/it]
                                                        

 45%|████▌     | 5419/12000 [3:13:04<3:40:10,  2.01s/it]
 45%|████▌     | 5420/12000 [3:13:06<3:18:49,  1.81s/it]
                                                        

 45%|████▌     | 5420/12000 [3:13:06<3:18:49,  1.81s/it]
 45%|████▌     | 5421/12000 [3:13:07<3:07:30,  1.71s/it]
                                                        

 45%|████▌     | 5421/12000 [3:13:07<3:07:30,  1.71s/it]
 45%|████▌     | 5422/12000 [3:13:09<2:53:19,  1.58s/it]
                                                        

 45%|████▌     | 5422/12000 [3:13:09<2:53:19,  1.58s/it]
 45%|████▌     | 5423/12000 [3:13:11<3:23:30,  1.86s/it]
                                                        

 45%|████▌     | 5423/12000 [3:13:11<3:23:30,  1.86s/it]
 45%|████▌     | 5424/12000 [3:13:14<3:44:11,  2.05s/it]
                                                        

 45%|████▌     | 5424/12000 [3:13:14<3:44:11,  2.05s/it]
 45%|████▌     | 5425/12000 [3:13:15<3:21:37,  1.84s/it]
                                                        

 45%|████▌     | 5425/12000 [3:13:15<3:21:37,  1.84s/it]
 45%|████▌     | 5426/12000 [3:13:18<4:17:13,  2.35s/it]
                                                        

 45%|████▌     | 5426/12000 [3:13:18<4:17:13,  2.35s/it]
 45%|████▌     | 5427/12000 [3:13:20<4:03:46,  2.23s/it]
                                                        

 45%|████▌     | 5427/12000 [3:13:20<4:03:46,  2.23s/it]
 45%|████▌     | 5428/12000 [3:13:22<3:37:30,  1.99s/it]
                                                        

 45%|████▌     | 5428/12000 [3:13:22<3:37:30,  1.99s/it]
 45%|████▌     | 5429/12000 [3:13:24<3:44:20,  2.05s/it]
                                                        

 45%|████▌     | 5429/12000 [3:13:24<3:44:20,  2.05s/it]
 45%|████▌     | 5430/12000 [3:13:25<3:20:38,  1.83s/it]
                                                        

 45%|████▌     | 5430/12000 [3:13:25<3:20:38,  1.83s/it]
 45%|████▌     | 5431/12000 [3:13:27<3:08:04,  1.72s/it]
                                                        

 45%|████▌     | 5431/12000 [3:13:27<3:08:04,  1.72s/it]
 45%|████▌     | 5432/12000 [3:13:29<3:34:20,  1.96s/it]
                                                        

 45%|████▌     | 5432/12000 [3:13:29<3:34:20,  1.96s/it]
 45%|████▌     | 5433/12000 [3:13:33<4:18:31,  2.36s/it]
                                                        

 45%|████▌     | 5433/12000 [3:13:33<4:18:31,  2.36s/it]
 45%|████▌     | 5434/12000 [3:13:34<3:53:18,  2.13s/it]
                                                        

 45%|████▌     | 5434/12000 [3:13:34<3:53:18,  2.13s/it]
 45%|████▌     | 5435/12000 [3:13:36<3:26:37,  1.89s/it]
                                                        

 45%|████▌     | 5435/12000 [3:13:36<3:26:37,  1.89s/it]
 45%|████▌     | 5436/12000 [3:13:37<3:20:11,  1.83s/it]
                                                        

 45%|████▌     | 5436/12000 [3:13:37<3:20:11,  1.83s/it]
 45%|████▌     | 5437/12000 [3:13:39<3:04:46,  1.69s/it]
                                                        

 45%|████▌     | 5437/12000 [3:13:39<3:04:46,  1.69s/it]
 45%|████▌     | 5438/12000 [3:13:40<2:57:58,  1.63s/it]
                                                        

 45%|████▌     | 5438/12000 [3:13:40<2:57:58,  1.63s/it]
 45%|████▌     | 5439/12000 [3:13:42<3:11:54,  1.75s/it]
                                                        

 45%|████▌     | 5439/12000 [3:13:42<3:11:54,  1.75s/it]
 45%|████▌     | 5440/12000 [3:13:44<3:02:49,  1.67s/it]
                                                        

 45%|████▌     | 5440/12000 [3:13:44<3:02:49,  1.67s/it]
 45%|████▌     | 5441/12000 [3:13:47<3:46:58,  2.08s/it]
                                                        

 45%|████▌     | 5441/12000 [3:13:47<3:46:58,  2.08s/it]
 45%|████▌     | 5442/12000 [3:13:49<3:45:23,  2.06s/it]
                                                        

 45%|████▌     | 5442/12000 [3:13:49<3:45:23,  2.06s/it]
 45%|████▌     | 5443/12000 [3:13:50<3:20:12,  1.83s/it]
                                                        

 45%|████▌     | 5443/12000 [3:13:50<3:20:12,  1.83s/it]
 45%|████▌     | 5444/12000 [3:13:51<3:05:48,  1.70s/it]
                                                        

 45%|████▌     | 5444/12000 [3:13:51<3:05:48,  1.70s/it]
 45%|████▌     | 5445/12000 [3:13:53<2:58:40,  1.64s/it]
                                                        

 45%|████▌     | 5445/12000 [3:13:53<2:58:40,  1.64s/it]
 45%|████▌     | 5446/12000 [3:13:55<3:06:00,  1.70s/it]
                                                        

 45%|████▌     | 5446/12000 [3:13:55<3:06:00,  1.70s/it]
 45%|████▌     | 5447/12000 [3:13:59<4:17:31,  2.36s/it]
                                                        

 45%|████▌     | 5447/12000 [3:13:59<4:17:31,  2.36s/it]
 45%|████▌     | 5448/12000 [3:14:02<4:42:51,  2.59s/it]
                                                        

 45%|████▌     | 5448/12000 [3:14:02<4:42:51,  2.59s/it]
 45%|████▌     | 5449/12000 [3:14:03<4:00:37,  2.20s/it]
                                                        

 45%|████▌     | 5449/12000 [3:14:03<4:00:37,  2.20s/it]
 45%|████▌     | 5450/12000 [3:14:06<4:11:36,  2.30s/it]
                                                        

 45%|████▌     | 5450/12000 [3:14:06<4:11:36,  2.30s/it]
 45%|████▌     | 5451/12000 [3:14:09<4:48:12,  2.64s/it]
                                                        

 45%|████▌     | 5451/12000 [3:14:09<4:48:12,  2.64s/it]
 45%|████▌     | 5452/12000 [3:14:10<4:09:58,  2.29s/it]
                                                        

 45%|████▌     | 5452/12000 [3:14:10<4:09:58,  2.29s/it]
 45%|████▌     | 5453/12000 [3:14:12<3:56:01,  2.16s/it]
                                                        

 45%|████▌     | 5453/12000 [3:14:12<3:56:01,  2.16s/it]
 45%|████▌     | 5454/12000 [3:14:14<3:27:51,  1.91s/it]
                                                        

 45%|████▌     | 5454/12000 [3:14:14<3:27:51,  1.91s/it]
 45%|████▌     | 5455/12000 [3:14:15<3:07:42,  1.72s/it]
                                                        

 45%|████▌     | 5455/12000 [3:14:15<3:07:42,  1.72s/it]
 45%|████▌     | 5456/12000 [3:14:19<4:39:58,  2.57s/it]
                                                        
{'loss': '1.072', 'grad_norm': '0.3883', 'learning_rate': '0.0001243', 'entropy': '1.104', 'num_tokens': '2.924e+07', 'mean_token_accuracy': '0.7219', 'epoch': '0.04787'}
{'loss': '0.888', 'grad_norm': '0.4376', 'learning_rate': '0.0001243', 'entropy': '0.9502', 'num_tokens': '2.925e+07', 'mean_token_accuracy': '0.7746', 'epoch': '0.04788'}
{'loss': '0.9051', 'grad_norm': '0.4126', 'learning_rate': '0.0001243', 'entropy': '0.9172', 'num_tokens': '2.925e+07', 'mean_token_accuracy': '0.7802', 'epoch': '0.04789'}
{'loss': '0.9543', 'grad_norm': '0.4199', 'learning_rate': '0.0001243', 'entropy': '0.9543', 'num_tokens': '2.926e+07', 'mean_token_accuracy': '0.7541', 'epoch': '0.0479'}
{'loss': '0.7414', 'grad_norm': '0.3624', 'learning_rate': '0.0001242', 'entropy': '0.7463', 'num_tokens': '2.926e+07', 'mean_token_accuracy': '0.8026', 'epoch': '0.0479'}
{'loss': '0.9419', 'grad_norm': '0.4377', 'learning_rate': '0.0001242', 'entropy': '0.9496', 'num_tokens': '2.927e+07', 'mean_token_accuracy': '0.7685', 'epoch': '0.04791'}
{'loss': '0.7027', 'grad_norm': '0.3899', 'learning_rate': '0.0001242', 'entropy': '0.7132', 'num_tokens': '2.927e+07', 'mean_token_accuracy': '0.8142', 'epoch': '0.04792'}
{'loss': '0.9265', 'grad_norm': '0.443', 'learning_rate': '0.0001241', 'entropy': '0.8651', 'num_tokens': '2.928e+07', 'mean_token_accuracy': '0.7853', 'epoch': '0.04793'}
{'loss': '0.5627', 'grad_norm': '0.3308', 'learning_rate': '0.0001241', 'entropy': '0.5902', 'num_tokens': '2.928e+07', 'mean_token_accuracy': '0.853', 'epoch': '0.04794'}
{'loss': '0.6332', 'grad_norm': '0.4798', 'learning_rate': '0.0001241', 'entropy': '0.6968', 'num_tokens': '2.929e+07', 'mean_token_accuracy': '0.8147', 'epoch': '0.04795'}
{'loss': '0.6065', 'grad_norm': '0.2593', 'learning_rate': '0.0001241', 'entropy': '0.6871', 'num_tokens': '2.929e+07', 'mean_token_accuracy': '0.8086', 'epoch': '0.04796'}
{'loss': '0.8078', 'grad_norm': '0.3673', 'learning_rate': '0.000124', 'entropy': '0.86', 'num_tokens': '2.93e+07', 'mean_token_accuracy': '0.7929', 'epoch': '0.04797'}
{'loss': '1.002', 'grad_norm': '0.4766', 'learning_rate': '0.000124', 'entropy': '0.9747', 'num_tokens': '2.93e+07', 'mean_token_accuracy': '0.7571', 'epoch': '0.04797'}
{'loss': '0.9288', 'grad_norm': '0.4441', 'learning_rate': '0.000124', 'entropy': '0.9157', 'num_tokens': '2.931e+07', 'mean_token_accuracy': '0.7454', 'epoch': '0.04798'}
{'loss': '1.011', 'grad_norm': '0.4526', 'learning_rate': '0.000124', 'entropy': '0.9081', 'num_tokens': '2.931e+07', 'mean_token_accuracy': '0.7683', 'epoch': '0.04799'}
{'loss': '0.6314', 'grad_norm': '0.3587', 'learning_rate': '0.0001239', 'entropy': '0.6836', 'num_tokens': '2.932e+07', 'mean_token_accuracy': '0.8233', 'epoch': '0.048'}
{'loss': '1.033', 'grad_norm': '0.4572', 'learning_rate': '0.0001239', 'entropy': '1.111', 'num_tokens': '2.932e+07', 'mean_token_accuracy': '0.7203', 'epoch': '0.04801'}
{'loss': '0.8905', 'grad_norm': '0.4181', 'learning_rate': '0.0001239', 'entropy': '0.9124', 'num_tokens': '2.933e+07', 'mean_token_accuracy': '0.7649', 'epoch': '0.04802'}
{'loss': '0.7279', 'grad_norm': '0.4455', 'learning_rate': '0.0001239', 'entropy': '0.7322', 'num_tokens': '2.933e+07', 'mean_token_accuracy': '0.8084', 'epoch': '0.04803'}
{'loss': '0.7659', 'grad_norm': '0.314', 'learning_rate': '0.0001238', 'entropy': '0.8034', 'num_tokens': '2.934e+07', 'mean_token_accuracy': '0.8104', 'epoch': '0.04804'}
{'loss': '0.8311', 'grad_norm': '0.4374', 'learning_rate': '0.0001238', 'entropy': '0.7972', 'num_tokens': '2.935e+07', 'mean_token_accuracy': '0.7992', 'epoch': '0.04805'}
{'loss': '1.036', 'grad_norm': '0.5226', 'learning_rate': '0.0001238', 'entropy': '1.043', 'num_tokens': '2.935e+07', 'mean_token_accuracy': '0.7502', 'epoch': '0.04805'}
{'loss': '1.011', 'grad_norm': '0.5338', 'learning_rate': '0.0001237', 'entropy': '1.032', 'num_tokens': '2.935e+07', 'mean_token_accuracy': '0.739', 'epoch': '0.04806'}
{'loss': '1.028', 'grad_norm': '0.4342', 'learning_rate': '0.0001237', 'entropy': '0.952', 'num_tokens': '2.936e+07', 'mean_token_accuracy': '0.7669', 'epoch': '0.04807'}
{'loss': '0.7601', 'grad_norm': '0.337', 'learning_rate': '0.0001237', 'entropy': '0.9345', 'num_tokens': '2.937e+07', 'mean_token_accuracy': '0.7687', 'epoch': '0.04808'}
{'loss': '0.9465', 'grad_norm': '0.3841', 'learning_rate': '0.0001237', 'entropy': '1.021', 'num_tokens': '2.937e+07', 'mean_token_accuracy': '0.7578', 'epoch': '0.04809'}
{'loss': '0.8688', 'grad_norm': '0.4519', 'learning_rate': '0.0001236', 'entropy': '0.9225', 'num_tokens': '2.937e+07', 'mean_token_accuracy': '0.7584', 'epoch': '0.0481'}
{'loss': '0.9852', 'grad_norm': '0.3847', 'learning_rate': '0.0001236', 'entropy': '0.9735', 'num_tokens': '2.938e+07', 'mean_token_accuracy': '0.7667', 'epoch': '0.04811'}
{'loss': '0.7783', 'grad_norm': '0.3849', 'learning_rate': '0.0001236', 'entropy': '0.853', 'num_tokens': '2.938e+07', 'mean_token_accuracy': '0.7914', 'epoch': '0.04812'}
{'loss': '0.8526', 'grad_norm': '0.4017', 'learning_rate': '0.0001236', 'entropy': '0.966', 'num_tokens': '2.939e+07', 'mean_token_accuracy': '0.7712', 'epoch': '0.04813'}
{'loss': '0.7949', 'grad_norm': '0.4119', 'learning_rate': '0.0001235', 'entropy': '0.7354', 'num_tokens': '2.939e+07', 'mean_token_accuracy': '0.8177', 'epoch': '0.04813'}
{'loss': '0.7455', 'grad_norm': '0.3833', 'learning_rate': '0.0001235', 'entropy': '0.8163', 'num_tokens': '2.94e+07', 'mean_token_accuracy': '0.8004', 'epoch': '0.04814'}
{'loss': '0.8161', 'grad_norm': '0.3005', 'learning_rate': '0.0001235', 'entropy': '0.968', 'num_tokens': '2.94e+07', 'mean_token_accuracy': '0.7738', 'epoch': '0.04815'}
{'loss': '1.012', 'grad_norm': '0.4152', 'learning_rate': '0.0001235', 'entropy': '0.9613', 'num_tokens': '2.941e+07', 'mean_token_accuracy': '0.7542', 'epoch': '0.04816'}
{'loss': '0.797', 'grad_norm': '0.3721', 'learning_rate': '0.0001234', 'entropy': '0.7973', 'num_tokens': '2.942e+07', 'mean_token_accuracy': '0.8102', 'epoch': '0.04817'}
{'loss': '0.9731', 'grad_norm': '0.4046', 'learning_rate': '0.0001234', 'entropy': '0.9385', 'num_tokens': '2.942e+07', 'mean_token_accuracy': '0.7613', 'epoch': '0.04818'}
{'loss': '0.641', 'grad_norm': '0.3307', 'learning_rate': '0.0001234', 'entropy': '0.6505', 'num_tokens': '2.942e+07', 'mean_token_accuracy': '0.8253', 'epoch': '0.04819'}
{'loss': '1.018', 'grad_norm': '0.3709', 'learning_rate': '0.0001233', 'entropy': '0.9279', 'num_tokens': '2.943e+07', 'mean_token_accuracy': '0.7691', 'epoch': '0.0482'}
{'loss': '0.8267', 'grad_norm': '0.4308', 'learning_rate': '0.0001233', 'entropy': '0.8122', 'num_tokens': '2.944e+07', 'mean_token_accuracy': '0.7998', 'epoch': '0.04821'}
{'loss': '1.145', 'grad_norm': '0.4288', 'learning_rate': '0.0001233', 'entropy': '1.095', 'num_tokens': '2.944e+07', 'mean_token_accuracy': '0.7253', 'epoch': '0.04821'}
{'loss': '0.706', 'grad_norm': '0.357', 'learning_rate': '0.0001233', 'entropy': '0.7631', 'num_tokens': '2.944e+07', 'mean_token_accuracy': '0.7948', 'epoch': '0.04822'}
{'loss': '1.287', 'grad_norm': '0.4403', 'learning_rate': '0.0001232', 'entropy': '1.276', 'num_tokens': '2.945e+07', 'mean_token_accuracy': '0.7025', 'epoch': '0.04823'}
{'loss': '0.7537', 'grad_norm': '0.3581', 'learning_rate': '0.0001232', 'entropy': '0.7202', 'num_tokens': '2.946e+07', 'mean_token_accuracy': '0.8147', 'epoch': '0.04824'}
{'loss': '0.7637', 'grad_norm': '0.4163', 'learning_rate': '0.0001232', 'entropy': '0.7792', 'num_tokens': '2.946e+07', 'mean_token_accuracy': '0.8093', 'epoch': '0.04825'}
{'loss': '0.6866', 'grad_norm': '0.3078', 'learning_rate': '0.0001232', 'entropy': '0.7802', 'num_tokens': '2.947e+07', 'mean_token_accuracy': '0.7904', 'epoch': '0.04826'}
{'loss': '0.728', 'grad_norm': '0.3821', 'learning_rate': '0.0001231', 'entropy': '0.7528', 'num_tokens': '2.947e+07', 'mean_token_accuracy': '0.8092', 'epoch': '0.04827'}
{'loss': '0.8956', 'grad_norm': '0.4101', 'learning_rate': '0.0001231', 'entropy': '0.9456', 'num_tokens': '2.947e+07', 'mean_token_accuracy': '0.7656', 'epoch': '0.04828'}

 45%|████▌     | 5456/12000 [3:14:19<4:39:58,  2.57s/it]
 45%|████▌     | 5457/12000 [3:14:22<4:28:46,  2.46s/it]
                                                        

 45%|████▌     | 5457/12000 [3:14:22<4:28:46,  2.46s/it]
 45%|████▌     | 5458/12000 [3:14:23<4:00:13,  2.20s/it]
                                                        

 45%|████▌     | 5458/12000 [3:14:23<4:00:13,  2.20s/it]
 45%|████▌     | 5459/12000 [3:14:25<3:48:31,  2.10s/it]
                                                        

 45%|████▌     | 5459/12000 [3:14:25<3:48:31,  2.10s/it]
 46%|████▌     | 5460/12000 [3:14:27<3:30:39,  1.93s/it]
                                                        

 46%|████▌     | 5460/12000 [3:14:27<3:30:39,  1.93s/it]
 46%|████▌     | 5461/12000 [3:14:28<3:18:52,  1.82s/it]
                                                        

 46%|████▌     | 5461/12000 [3:14:28<3:18:52,  1.82s/it]
 46%|████▌     | 5462/12000 [3:14:30<3:16:03,  1.80s/it]
                                                        

 46%|████▌     | 5462/12000 [3:14:30<3:16:03,  1.80s/it]
 46%|████▌     | 5463/12000 [3:14:32<3:14:08,  1.78s/it]
                                                        

 46%|████▌     | 5463/12000 [3:14:32<3:14:08,  1.78s/it]
 46%|████▌     | 5464/12000 [3:14:33<2:57:09,  1.63s/it]
                                                        

 46%|████▌     | 5464/12000 [3:14:33<2:57:09,  1.63s/it]
 46%|████▌     | 5465/12000 [3:14:35<3:10:33,  1.75s/it]
                                                        

 46%|████▌     | 5465/12000 [3:14:35<3:10:33,  1.75s/it]
 46%|████▌     | 5466/12000 [3:14:43<6:30:43,  3.59s/it]
                                                        

 46%|████▌     | 5466/12000 [3:14:43<6:30:43,  3.59s/it]
 46%|████▌     | 5467/12000 [3:14:44<5:23:17,  2.97s/it]
                                                        

 46%|████▌     | 5467/12000 [3:14:44<5:23:17,  2.97s/it]
 46%|████▌     | 5468/12000 [3:14:46<4:33:32,  2.51s/it]
                                                        

 46%|████▌     | 5468/12000 [3:14:46<4:33:32,  2.51s/it]
 46%|████▌     | 5469/12000 [3:14:47<3:57:11,  2.18s/it]
                                                        

 46%|████▌     | 5469/12000 [3:14:47<3:57:11,  2.18s/it]
 46%|████▌     | 5470/12000 [3:14:51<4:33:33,  2.51s/it]
                                                        

 46%|████▌     | 5470/12000 [3:14:51<4:33:33,  2.51s/it]
 46%|████▌     | 5471/12000 [3:14:52<3:59:14,  2.20s/it]
                                                        

 46%|████▌     | 5471/12000 [3:14:52<3:59:14,  2.20s/it]
 46%|████▌     | 5472/12000 [3:14:53<3:32:01,  1.95s/it]
                                                        

 46%|████▌     | 5472/12000 [3:14:53<3:32:01,  1.95s/it]
 46%|████▌     | 5473/12000 [3:14:55<3:18:04,  1.82s/it]
                                                        

 46%|████▌     | 5473/12000 [3:14:55<3:18:04,  1.82s/it]
 46%|████▌     | 5474/12000 [3:15:00<4:54:29,  2.71s/it]
                                                        

 46%|████▌     | 5474/12000 [3:15:00<4:54:29,  2.71s/it]
 46%|████▌     | 5475/12000 [3:15:03<5:02:24,  2.78s/it]
                                                        

 46%|████▌     | 5475/12000 [3:15:03<5:02:24,  2.78s/it]
 46%|████▌     | 5476/12000 [3:15:04<4:14:17,  2.34s/it]
                                                        

 46%|████▌     | 5476/12000 [3:15:04<4:14:17,  2.34s/it]
 46%|████▌     | 5477/12000 [3:15:06<4:16:07,  2.36s/it]
                                                        

 46%|████▌     | 5477/12000 [3:15:06<4:16:07,  2.36s/it]
 46%|████▌     | 5478/12000 [3:15:09<4:27:07,  2.46s/it]
                                                        

 46%|████▌     | 5478/12000 [3:15:09<4:27:07,  2.46s/it]
 46%|████▌     | 5479/12000 [3:15:11<4:01:43,  2.22s/it]
                                                        

 46%|████▌     | 5479/12000 [3:15:11<4:01:43,  2.22s/it]
 46%|████▌     | 5480/12000 [3:15:14<4:22:00,  2.41s/it]
                                                        

 46%|████▌     | 5480/12000 [3:15:14<4:22:00,  2.41s/it]
 46%|████▌     | 5481/12000 [3:15:16<4:14:05,  2.34s/it]
                                                        

 46%|████▌     | 5481/12000 [3:15:16<4:14:05,  2.34s/it]
 46%|████▌     | 5482/12000 [3:15:18<4:23:03,  2.42s/it]
                                                        

 46%|████▌     | 5482/12000 [3:15:18<4:23:03,  2.42s/it]
 46%|████▌     | 5483/12000 [3:15:20<3:50:15,  2.12s/it]
                                                        

 46%|████▌     | 5483/12000 [3:15:20<3:50:15,  2.12s/it]
 46%|████▌     | 5484/12000 [3:15:21<3:28:47,  1.92s/it]
                                                        

 46%|████▌     | 5484/12000 [3:15:21<3:28:47,  1.92s/it]
 46%|████▌     | 5485/12000 [3:15:26<5:13:42,  2.89s/it]
                                                        

 46%|████▌     | 5485/12000 [3:15:26<5:13:42,  2.89s/it]
 46%|████▌     | 5486/12000 [3:15:28<4:41:42,  2.59s/it]
                                                        

 46%|████▌     | 5486/12000 [3:15:28<4:41:42,  2.59s/it]
 46%|████▌     | 5487/12000 [3:15:32<5:11:05,  2.87s/it]
                                                        

 46%|████▌     | 5487/12000 [3:15:32<5:11:05,  2.87s/it]
 46%|████▌     | 5488/12000 [3:15:33<4:29:55,  2.49s/it]
                                                        

 46%|████▌     | 5488/12000 [3:15:33<4:29:55,  2.49s/it]
 46%|████▌     | 5489/12000 [3:15:35<4:16:24,  2.36s/it]
                                                        

 46%|████▌     | 5489/12000 [3:15:35<4:16:24,  2.36s/it]
 46%|████▌     | 5490/12000 [3:15:38<4:11:30,  2.32s/it]
                                                        

 46%|████▌     | 5490/12000 [3:15:38<4:11:30,  2.32s/it]
 46%|████▌     | 5491/12000 [3:15:39<3:51:39,  2.14s/it]
                                                        

 46%|████▌     | 5491/12000 [3:15:39<3:51:39,  2.14s/it]
 46%|████▌     | 5492/12000 [3:15:43<4:42:08,  2.60s/it]
                                                        

 46%|████▌     | 5492/12000 [3:15:43<4:42:08,  2.60s/it]
 46%|████▌     | 5493/12000 [3:15:46<4:41:49,  2.60s/it]
                                                        

 46%|████▌     | 5493/12000 [3:15:46<4:41:49,  2.60s/it]
 46%|████▌     | 5494/12000 [3:15:48<4:25:40,  2.45s/it]
                                                        

 46%|████▌     | 5494/12000 [3:15:48<4:25:40,  2.45s/it]
 46%|████▌     | 5495/12000 [3:15:49<3:52:17,  2.14s/it]
                                                        

 46%|████▌     | 5495/12000 [3:15:49<3:52:17,  2.14s/it]
 46%|████▌     | 5496/12000 [3:15:51<3:36:36,  2.00s/it]
                                                        

 46%|████▌     | 5496/12000 [3:15:51<3:36:36,  2.00s/it]
 46%|████▌     | 5497/12000 [3:15:53<3:35:29,  1.99s/it]
                                                        

 46%|████▌     | 5497/12000 [3:15:53<3:35:29,  1.99s/it]
 46%|████▌     | 5498/12000 [3:15:58<5:17:33,  2.93s/it]
                                                        

 46%|████▌     | 5498/12000 [3:15:58<5:17:33,  2.93s/it]
 46%|████▌     | 5499/12000 [3:16:00<4:34:44,  2.54s/it]
                                                        

 46%|████▌     | 5499/12000 [3:16:00<4:34:44,  2.54s/it]
 46%|████▌     | 5500/12000 [3:16:01<4:10:15,  2.31s/it]
                                                        

 46%|████▌     | 5500/12000 [3:16:01<4:10:15,  2.31s/it]
 46%|████▌     | 5501/12000 [3:16:04<4:27:19,  2.47s/it]
                                                        

 46%|████▌     | 5501/12000 [3:16:04<4:27:19,  2.47s/it]
 46%|████▌     | 5502/12000 [3:16:06<3:50:44,  2.13s/it]
                                                        

 46%|████▌     | 5502/12000 [3:16:06<3:50:44,  2.13s/it]
 46%|████▌     | 5503/12000 [3:16:08<3:47:14,  2.10s/it]
                                                        
{'loss': '1.216', 'grad_norm': '0.3752', 'learning_rate': '0.0001231', 'entropy': '1.124', 'num_tokens': '2.948e+07', 'mean_token_accuracy': '0.7287', 'epoch': '0.04828'}
{'loss': '1.327', 'grad_norm': '0.4012', 'learning_rate': '0.000123', 'entropy': '1.227', 'num_tokens': '2.949e+07', 'mean_token_accuracy': '0.7127', 'epoch': '0.04829'}
{'loss': '0.9192', 'grad_norm': '0.3965', 'learning_rate': '0.000123', 'entropy': '0.8765', 'num_tokens': '2.949e+07', 'mean_token_accuracy': '0.7773', 'epoch': '0.0483'}
{'loss': '0.7445', 'grad_norm': '0.3708', 'learning_rate': '0.000123', 'entropy': '0.7266', 'num_tokens': '2.95e+07', 'mean_token_accuracy': '0.8181', 'epoch': '0.04831'}
{'loss': '0.9745', 'grad_norm': '0.4355', 'learning_rate': '0.000123', 'entropy': '0.973', 'num_tokens': '2.95e+07', 'mean_token_accuracy': '0.764', 'epoch': '0.04832'}
{'loss': '0.8639', 'grad_norm': '0.3831', 'learning_rate': '0.0001229', 'entropy': '0.935', 'num_tokens': '2.951e+07', 'mean_token_accuracy': '0.7779', 'epoch': '0.04833'}
{'loss': '1.085', 'grad_norm': '0.5589', 'learning_rate': '0.0001229', 'entropy': '1.08', 'num_tokens': '2.951e+07', 'mean_token_accuracy': '0.7487', 'epoch': '0.04834'}
{'loss': '0.8767', 'grad_norm': '0.4', 'learning_rate': '0.0001229', 'entropy': '0.9078', 'num_tokens': '2.952e+07', 'mean_token_accuracy': '0.7783', 'epoch': '0.04835'}
{'loss': '1.012', 'grad_norm': '0.4291', 'learning_rate': '0.0001229', 'entropy': '1.071', 'num_tokens': '2.952e+07', 'mean_token_accuracy': '0.7392', 'epoch': '0.04836'}
{'loss': '0.7314', 'grad_norm': '0.3942', 'learning_rate': '0.0001228', 'entropy': '0.7069', 'num_tokens': '2.953e+07', 'mean_token_accuracy': '0.8137', 'epoch': '0.04836'}
{'loss': '0.7989', 'grad_norm': '0.2968', 'learning_rate': '0.0001228', 'entropy': '0.9046', 'num_tokens': '2.954e+07', 'mean_token_accuracy': '0.7777', 'epoch': '0.04837'}
{'loss': '0.689', 'grad_norm': '0.3834', 'learning_rate': '0.0001228', 'entropy': '0.7854', 'num_tokens': '2.955e+07', 'mean_token_accuracy': '0.8017', 'epoch': '0.04838'}
{'loss': '1.013', 'grad_norm': '0.3855', 'learning_rate': '0.0001228', 'entropy': '1.028', 'num_tokens': '2.955e+07', 'mean_token_accuracy': '0.7579', 'epoch': '0.04839'}
{'loss': '0.89', 'grad_norm': '0.4054', 'learning_rate': '0.0001227', 'entropy': '0.8736', 'num_tokens': '2.956e+07', 'mean_token_accuracy': '0.7896', 'epoch': '0.0484'}
{'loss': '0.7322', 'grad_norm': '0.4429', 'learning_rate': '0.0001227', 'entropy': '0.8336', 'num_tokens': '2.956e+07', 'mean_token_accuracy': '0.7833', 'epoch': '0.04841'}
{'loss': '0.8859', 'grad_norm': '0.3567', 'learning_rate': '0.0001227', 'entropy': '0.9308', 'num_tokens': '2.956e+07', 'mean_token_accuracy': '0.7748', 'epoch': '0.04842'}
{'loss': '0.5513', 'grad_norm': '0.3486', 'learning_rate': '0.0001226', 'entropy': '0.575', 'num_tokens': '2.957e+07', 'mean_token_accuracy': '0.8445', 'epoch': '0.04843'}
{'loss': '0.8556', 'grad_norm': '0.4813', 'learning_rate': '0.0001226', 'entropy': '0.8917', 'num_tokens': '2.957e+07', 'mean_token_accuracy': '0.7752', 'epoch': '0.04844'}
{'loss': '1.043', 'grad_norm': '0.3629', 'learning_rate': '0.0001226', 'entropy': '1.037', 'num_tokens': '2.958e+07', 'mean_token_accuracy': '0.7575', 'epoch': '0.04844'}
{'loss': '0.3926', 'grad_norm': '0.2615', 'learning_rate': '0.0001226', 'entropy': '0.3925', 'num_tokens': '2.959e+07', 'mean_token_accuracy': '0.8984', 'epoch': '0.04845'}
{'loss': '0.7886', 'grad_norm': '0.4109', 'learning_rate': '0.0001225', 'entropy': '0.822', 'num_tokens': '2.959e+07', 'mean_token_accuracy': '0.7935', 'epoch': '0.04846'}
{'loss': '0.8099', 'grad_norm': '0.3447', 'learning_rate': '0.0001225', 'entropy': '0.7913', 'num_tokens': '2.96e+07', 'mean_token_accuracy': '0.7959', 'epoch': '0.04847'}
{'loss': '0.8497', 'grad_norm': '0.4621', 'learning_rate': '0.0001225', 'entropy': '0.9378', 'num_tokens': '2.961e+07', 'mean_token_accuracy': '0.7497', 'epoch': '0.04848'}
{'loss': '0.6497', 'grad_norm': '0.3546', 'learning_rate': '0.0001225', 'entropy': '0.6703', 'num_tokens': '2.961e+07', 'mean_token_accuracy': '0.8293', 'epoch': '0.04849'}
{'loss': '0.5956', 'grad_norm': '0.3727', 'learning_rate': '0.0001224', 'entropy': '0.6139', 'num_tokens': '2.962e+07', 'mean_token_accuracy': '0.8374', 'epoch': '0.0485'}
{'loss': '1.159', 'grad_norm': '0.4519', 'learning_rate': '0.0001224', 'entropy': '1.157', 'num_tokens': '2.962e+07', 'mean_token_accuracy': '0.7196', 'epoch': '0.04851'}
{'loss': '0.6327', 'grad_norm': '0.3977', 'learning_rate': '0.0001224', 'entropy': '0.6642', 'num_tokens': '2.963e+07', 'mean_token_accuracy': '0.8174', 'epoch': '0.04851'}
{'loss': '0.9016', 'grad_norm': '0.4289', 'learning_rate': '0.0001224', 'entropy': '0.836', 'num_tokens': '2.963e+07', 'mean_token_accuracy': '0.783', 'epoch': '0.04852'}
{'loss': '0.4894', 'grad_norm': '0.3036', 'learning_rate': '0.0001223', 'entropy': '0.5158', 'num_tokens': '2.964e+07', 'mean_token_accuracy': '0.8588', 'epoch': '0.04853'}
{'loss': '1.197', 'grad_norm': '0.4345', 'learning_rate': '0.0001223', 'entropy': '0.9323', 'num_tokens': '2.965e+07', 'mean_token_accuracy': '0.7747', 'epoch': '0.04854'}
{'loss': '0.9716', 'grad_norm': '0.342', 'learning_rate': '0.0001223', 'entropy': '0.9659', 'num_tokens': '2.965e+07', 'mean_token_accuracy': '0.754', 'epoch': '0.04855'}
{'loss': '0.925', 'grad_norm': '0.4385', 'learning_rate': '0.0001222', 'entropy': '0.9673', 'num_tokens': '2.966e+07', 'mean_token_accuracy': '0.7547', 'epoch': '0.04856'}
{'loss': '0.5545', 'grad_norm': '0.3205', 'learning_rate': '0.0001222', 'entropy': '0.5622', 'num_tokens': '2.967e+07', 'mean_token_accuracy': '0.8536', 'epoch': '0.04857'}
{'loss': '0.9177', 'grad_norm': '0.4063', 'learning_rate': '0.0001222', 'entropy': '0.9075', 'num_tokens': '2.967e+07', 'mean_token_accuracy': '0.772', 'epoch': '0.04858'}
{'loss': '0.826', 'grad_norm': '0.3861', 'learning_rate': '0.0001222', 'entropy': '0.8366', 'num_tokens': '2.968e+07', 'mean_token_accuracy': '0.7947', 'epoch': '0.04859'}
{'loss': '0.9238', 'grad_norm': '0.4455', 'learning_rate': '0.0001221', 'entropy': '0.9126', 'num_tokens': '2.968e+07', 'mean_token_accuracy': '0.7798', 'epoch': '0.04859'}
{'loss': '0.8846', 'grad_norm': '0.4028', 'learning_rate': '0.0001221', 'entropy': '0.8874', 'num_tokens': '2.969e+07', 'mean_token_accuracy': '0.7797', 'epoch': '0.0486'}
{'loss': '0.935', 'grad_norm': '0.3845', 'learning_rate': '0.0001221', 'entropy': '0.9744', 'num_tokens': '2.969e+07', 'mean_token_accuracy': '0.763', 'epoch': '0.04861'}
{'loss': '1.095', 'grad_norm': '0.5233', 'learning_rate': '0.0001221', 'entropy': '0.9822', 'num_tokens': '2.97e+07', 'mean_token_accuracy': '0.7539', 'epoch': '0.04862'}
{'loss': '0.7426', 'grad_norm': '0.3489', 'learning_rate': '0.000122', 'entropy': '0.7515', 'num_tokens': '2.97e+07', 'mean_token_accuracy': '0.8191', 'epoch': '0.04863'}
{'loss': '0.5867', 'grad_norm': '0.3026', 'learning_rate': '0.000122', 'entropy': '0.6012', 'num_tokens': '2.971e+07', 'mean_token_accuracy': '0.8351', 'epoch': '0.04864'}
{'loss': '0.8981', 'grad_norm': '0.4345', 'learning_rate': '0.000122', 'entropy': '0.9794', 'num_tokens': '2.972e+07', 'mean_token_accuracy': '0.7515', 'epoch': '0.04865'}
{'loss': '1.891', 'grad_norm': '0.4236', 'learning_rate': '0.0001219', 'entropy': '1.514', 'num_tokens': '2.973e+07', 'mean_token_accuracy': '0.6619', 'epoch': '0.04866'}
{'loss': '0.7431', 'grad_norm': '0.4236', 'learning_rate': '0.0001219', 'entropy': '0.7665', 'num_tokens': '2.973e+07', 'mean_token_accuracy': '0.8', 'epoch': '0.04867'}
{'loss': '0.385', 'grad_norm': '0.2953', 'learning_rate': '0.0001219', 'entropy': '0.4334', 'num_tokens': '2.974e+07', 'mean_token_accuracy': '0.8928', 'epoch': '0.04867'}
{'loss': '0.823', 'grad_norm': '0.3807', 'learning_rate': '0.0001219', 'entropy': '0.7821', 'num_tokens': '2.974e+07', 'mean_token_accuracy': '0.8057', 'epoch': '0.04868'}
{'loss': '0.8005', 'grad_norm': '0.4514', 'learning_rate': '0.0001218', 'entropy': '0.8115', 'num_tokens': '2.974e+07', 'mean_token_accuracy': '0.7937', 'epoch': '0.04869'}

 46%|████▌     | 5503/12000 [3:16:08<3:47:14,  2.10s/it]
 46%|████▌     | 5504/12000 [3:16:09<3:39:39,  2.03s/it]
                                                        

 46%|████▌     | 5504/12000 [3:16:09<3:39:39,  2.03s/it]
 46%|████▌     | 5505/12000 [3:16:12<3:48:18,  2.11s/it]
                                                        

 46%|████▌     | 5505/12000 [3:16:12<3:48:18,  2.11s/it]
 46%|████▌     | 5506/12000 [3:16:13<3:31:02,  1.95s/it]
                                                        

 46%|████▌     | 5506/12000 [3:16:13<3:31:02,  1.95s/it]
 46%|████▌     | 5507/12000 [3:16:15<3:30:55,  1.95s/it]
                                                        

 46%|████▌     | 5507/12000 [3:16:15<3:30:55,  1.95s/it]
 46%|████▌     | 5508/12000 [3:16:18<3:46:08,  2.09s/it]
                                                        

 46%|████▌     | 5508/12000 [3:16:18<3:46:08,  2.09s/it]
 46%|████▌     | 5509/12000 [3:16:19<3:31:24,  1.95s/it]
                                                        

 46%|████▌     | 5509/12000 [3:16:19<3:31:24,  1.95s/it]
 46%|████▌     | 5510/12000 [3:16:21<3:22:26,  1.87s/it]
                                                        

 46%|████▌     | 5510/12000 [3:16:21<3:22:26,  1.87s/it]
 46%|████▌     | 5511/12000 [3:16:23<3:29:12,  1.93s/it]
                                                        

 46%|████▌     | 5511/12000 [3:16:23<3:29:12,  1.93s/it]
 46%|████▌     | 5512/12000 [3:16:25<3:39:12,  2.03s/it]
                                                        

 46%|████▌     | 5512/12000 [3:16:25<3:39:12,  2.03s/it]
 46%|████▌     | 5513/12000 [3:16:27<3:18:34,  1.84s/it]
                                                        

 46%|████▌     | 5513/12000 [3:16:27<3:18:34,  1.84s/it]
 46%|████▌     | 5514/12000 [3:16:30<4:19:40,  2.40s/it]
                                                        

 46%|████▌     | 5514/12000 [3:16:30<4:19:40,  2.40s/it]
 46%|████▌     | 5515/12000 [3:16:32<3:47:03,  2.10s/it]
                                                        

 46%|████▌     | 5515/12000 [3:16:32<3:47:03,  2.10s/it]
 46%|████▌     | 5516/12000 [3:16:34<3:40:40,  2.04s/it]
                                                        

 46%|████▌     | 5516/12000 [3:16:34<3:40:40,  2.04s/it]
 46%|████▌     | 5517/12000 [3:16:35<3:16:02,  1.81s/it]
                                                        

 46%|████▌     | 5517/12000 [3:16:35<3:16:02,  1.81s/it]
 46%|████▌     | 5518/12000 [3:16:36<3:04:39,  1.71s/it]
                                                        

 46%|████▌     | 5518/12000 [3:16:36<3:04:39,  1.71s/it]
 46%|████▌     | 5519/12000 [3:16:39<3:43:35,  2.07s/it]
                                                        

 46%|████▌     | 5519/12000 [3:16:39<3:43:35,  2.07s/it]
 46%|████▌     | 5520/12000 [3:16:41<3:22:39,  1.88s/it]
                                                        

 46%|████▌     | 5520/12000 [3:16:41<3:22:39,  1.88s/it]
 46%|████▌     | 5521/12000 [3:16:43<3:30:10,  1.95s/it]
                                                        

 46%|████▌     | 5521/12000 [3:16:43<3:30:10,  1.95s/it]
 46%|████▌     | 5522/12000 [3:16:45<3:50:45,  2.14s/it]
                                                        

 46%|████▌     | 5522/12000 [3:16:45<3:50:45,  2.14s/it]
 46%|████▌     | 5523/12000 [3:16:47<3:37:23,  2.01s/it]
                                                        

 46%|████▌     | 5523/12000 [3:16:47<3:37:23,  2.01s/it]
 46%|████▌     | 5524/12000 [3:16:49<3:23:28,  1.89s/it]
                                                        

 46%|████▌     | 5524/12000 [3:16:49<3:23:28,  1.89s/it]
 46%|████▌     | 5525/12000 [3:16:50<3:15:03,  1.81s/it]
                                                        

 46%|████▌     | 5525/12000 [3:16:50<3:15:03,  1.81s/it]
 46%|████▌     | 5526/12000 [3:16:53<3:31:49,  1.96s/it]
                                                        

 46%|████▌     | 5526/12000 [3:16:53<3:31:49,  1.96s/it]
 46%|████▌     | 5527/12000 [3:16:57<4:43:01,  2.62s/it]
                                                        

 46%|████▌     | 5527/12000 [3:16:57<4:43:01,  2.62s/it]
 46%|████▌     | 5528/12000 [3:16:58<4:03:34,  2.26s/it]
                                                        

 46%|████▌     | 5528/12000 [3:16:58<4:03:34,  2.26s/it]
 46%|████▌     | 5529/12000 [3:17:00<3:49:41,  2.13s/it]
                                                        

 46%|████▌     | 5529/12000 [3:17:00<3:49:41,  2.13s/it]
 46%|████▌     | 5530/12000 [3:17:04<4:38:37,  2.58s/it]
                                                        

 46%|████▌     | 5530/12000 [3:17:04<4:38:37,  2.58s/it]
 46%|████▌     | 5531/12000 [3:17:06<4:19:54,  2.41s/it]
                                                        

 46%|████▌     | 5531/12000 [3:17:06<4:19:54,  2.41s/it]
 46%|████▌     | 5532/12000 [3:17:08<4:01:38,  2.24s/it]
                                                        

 46%|████▌     | 5532/12000 [3:17:08<4:01:38,  2.24s/it]
 46%|████▌     | 5533/12000 [3:17:10<4:05:33,  2.28s/it]
                                                        

 46%|████▌     | 5533/12000 [3:17:10<4:05:33,  2.28s/it]
 46%|████▌     | 5534/12000 [3:17:12<4:09:23,  2.31s/it]
                                                        

 46%|████▌     | 5534/12000 [3:17:12<4:09:23,  2.31s/it]
 46%|████▌     | 5535/12000 [3:17:15<4:06:48,  2.29s/it]
                                                        

 46%|████▌     | 5535/12000 [3:17:15<4:06:48,  2.29s/it]
 46%|████▌     | 5536/12000 [3:17:18<4:48:30,  2.68s/it]
                                                        

 46%|████▌     | 5536/12000 [3:17:18<4:48:30,  2.68s/it]
 46%|████▌     | 5537/12000 [3:17:21<4:37:24,  2.58s/it]
                                                        

 46%|████▌     | 5537/12000 [3:17:21<4:37:24,  2.58s/it]
 46%|████▌     | 5538/12000 [3:17:23<4:25:01,  2.46s/it]
                                                        

 46%|████▌     | 5538/12000 [3:17:23<4:25:01,  2.46s/it]
 46%|████▌     | 5539/12000 [3:17:24<3:46:33,  2.10s/it]
                                                        

 46%|████▌     | 5539/12000 [3:17:24<3:46:33,  2.10s/it]
 46%|████▌     | 5540/12000 [3:17:26<3:28:09,  1.93s/it]
                                                        

 46%|████▌     | 5540/12000 [3:17:26<3:28:09,  1.93s/it]
 46%|████▌     | 5541/12000 [3:17:28<3:40:45,  2.05s/it]
                                                        

 46%|████▌     | 5541/12000 [3:17:28<3:40:45,  2.05s/it]
 46%|████▌     | 5542/12000 [3:17:30<3:37:07,  2.02s/it]
                                                        

 46%|████▌     | 5542/12000 [3:17:30<3:37:07,  2.02s/it]
 46%|████▌     | 5543/12000 [3:17:32<3:28:34,  1.94s/it]
                                                        

 46%|████▌     | 5543/12000 [3:17:32<3:28:34,  1.94s/it]
 46%|████▌     | 5544/12000 [3:17:33<3:16:53,  1.83s/it]
                                                        

 46%|████▌     | 5544/12000 [3:17:33<3:16:53,  1.83s/it]
 46%|████▌     | 5545/12000 [3:17:35<3:10:29,  1.77s/it]
                                                        

 46%|████▌     | 5545/12000 [3:17:35<3:10:29,  1.77s/it]
 46%|████▌     | 5546/12000 [3:17:36<3:00:29,  1.68s/it]
                                                        

 46%|████▌     | 5546/12000 [3:17:36<3:00:29,  1.68s/it]
 46%|████▌     | 5547/12000 [3:17:38<2:53:09,  1.61s/it]
                                                        

 46%|████▌     | 5547/12000 [3:17:38<2:53:09,  1.61s/it]
 46%|████▌     | 5548/12000 [3:17:39<2:48:23,  1.57s/it]
                                                        

 46%|████▌     | 5548/12000 [3:17:39<2:48:23,  1.57s/it]
 46%|████▌     | 5549/12000 [3:17:44<4:31:04,  2.52s/it]
                                                        

 46%|████▌     | 5549/12000 [3:17:44<4:31:04,  2.52s/it]
 46%|████▋     | 5550/12000 [3:17:47<4:42:16,  2.63s/it]
                                                        
{'loss': '0.7863', 'grad_norm': '0.4429', 'learning_rate': '0.0001218', 'entropy': '0.8073', 'num_tokens': '2.975e+07', 'mean_token_accuracy': '0.7892', 'epoch': '0.0487'}
{'loss': '0.7497', 'grad_norm': '0.3389', 'learning_rate': '0.0001218', 'entropy': '0.8687', 'num_tokens': '2.976e+07', 'mean_token_accuracy': '0.7889', 'epoch': '0.04871'}
{'loss': '0.6834', 'grad_norm': '0.4287', 'learning_rate': '0.0001218', 'entropy': '0.7529', 'num_tokens': '2.976e+07', 'mean_token_accuracy': '0.8046', 'epoch': '0.04872'}
{'loss': '0.5973', 'grad_norm': '0.3297', 'learning_rate': '0.0001217', 'entropy': '0.6345', 'num_tokens': '2.977e+07', 'mean_token_accuracy': '0.8252', 'epoch': '0.04873'}
{'loss': '1.055', 'grad_norm': '0.4511', 'learning_rate': '0.0001217', 'entropy': '1.052', 'num_tokens': '2.977e+07', 'mean_token_accuracy': '0.7285', 'epoch': '0.04874'}
{'loss': '1.671', 'grad_norm': '0.4697', 'learning_rate': '0.0001217', 'entropy': '1.464', 'num_tokens': '2.978e+07', 'mean_token_accuracy': '0.6836', 'epoch': '0.04874'}
{'loss': '1.021', 'grad_norm': '0.4152', 'learning_rate': '0.0001217', 'entropy': '0.9811', 'num_tokens': '2.978e+07', 'mean_token_accuracy': '0.76', 'epoch': '0.04875'}
{'loss': '0.7555', 'grad_norm': '0.4178', 'learning_rate': '0.0001216', 'entropy': '0.8216', 'num_tokens': '2.979e+07', 'mean_token_accuracy': '0.8019', 'epoch': '0.04876'}
{'loss': '0.7888', 'grad_norm': '0.4759', 'learning_rate': '0.0001216', 'entropy': '0.7391', 'num_tokens': '2.979e+07', 'mean_token_accuracy': '0.7975', 'epoch': '0.04877'}
{'loss': '1.101', 'grad_norm': '0.507', 'learning_rate': '0.0001216', 'entropy': '1.05', 'num_tokens': '2.98e+07', 'mean_token_accuracy': '0.7551', 'epoch': '0.04878'}
{'loss': '0.6693', 'grad_norm': '0.406', 'learning_rate': '0.0001215', 'entropy': '0.6725', 'num_tokens': '2.98e+07', 'mean_token_accuracy': '0.8247', 'epoch': '0.04879'}
{'loss': '1.101', 'grad_norm': '0.4457', 'learning_rate': '0.0001215', 'entropy': '1.029', 'num_tokens': '2.981e+07', 'mean_token_accuracy': '0.7491', 'epoch': '0.0488'}
{'loss': '0.7492', 'grad_norm': '0.4562', 'learning_rate': '0.0001215', 'entropy': '0.8397', 'num_tokens': '2.981e+07', 'mean_token_accuracy': '0.7896', 'epoch': '0.04881'}
{'loss': '0.7048', 'grad_norm': '0.3337', 'learning_rate': '0.0001215', 'entropy': '0.6508', 'num_tokens': '2.982e+07', 'mean_token_accuracy': '0.8302', 'epoch': '0.04882'}
{'loss': '0.7002', 'grad_norm': '0.3937', 'learning_rate': '0.0001214', 'entropy': '0.7091', 'num_tokens': '2.982e+07', 'mean_token_accuracy': '0.8278', 'epoch': '0.04882'}
{'loss': '1.223', 'grad_norm': '0.4833', 'learning_rate': '0.0001214', 'entropy': '1.173', 'num_tokens': '2.982e+07', 'mean_token_accuracy': '0.7213', 'epoch': '0.04883'}
{'loss': '1.051', 'grad_norm': '0.4348', 'learning_rate': '0.0001214', 'entropy': '1.004', 'num_tokens': '2.983e+07', 'mean_token_accuracy': '0.7588', 'epoch': '0.04884'}
{'loss': '0.8235', 'grad_norm': '0.4515', 'learning_rate': '0.0001214', 'entropy': '0.8573', 'num_tokens': '2.983e+07', 'mean_token_accuracy': '0.7855', 'epoch': '0.04885'}
{'loss': '1.116', 'grad_norm': '0.4345', 'learning_rate': '0.0001213', 'entropy': '1.166', 'num_tokens': '2.984e+07', 'mean_token_accuracy': '0.7385', 'epoch': '0.04886'}
{'loss': '0.7263', 'grad_norm': '0.3705', 'learning_rate': '0.0001213', 'entropy': '0.7507', 'num_tokens': '2.985e+07', 'mean_token_accuracy': '0.8096', 'epoch': '0.04887'}
{'loss': '0.7061', 'grad_norm': '0.303', 'learning_rate': '0.0001213', 'entropy': '0.7195', 'num_tokens': '2.985e+07', 'mean_token_accuracy': '0.8199', 'epoch': '0.04888'}
{'loss': '0.8452', 'grad_norm': '0.3463', 'learning_rate': '0.0001212', 'entropy': '0.8655', 'num_tokens': '2.986e+07', 'mean_token_accuracy': '0.7923', 'epoch': '0.04889'}
{'loss': '0.685', 'grad_norm': '0.3956', 'learning_rate': '0.0001212', 'entropy': '0.7043', 'num_tokens': '2.986e+07', 'mean_token_accuracy': '0.8238', 'epoch': '0.0489'}
{'loss': '0.91', 'grad_norm': '0.392', 'learning_rate': '0.0001212', 'entropy': '0.9126', 'num_tokens': '2.987e+07', 'mean_token_accuracy': '0.7736', 'epoch': '0.0489'}
{'loss': '0.8517', 'grad_norm': '0.3713', 'learning_rate': '0.0001212', 'entropy': '0.8575', 'num_tokens': '2.988e+07', 'mean_token_accuracy': '0.7785', 'epoch': '0.04891'}
{'loss': '1.179', 'grad_norm': '0.521', 'learning_rate': '0.0001211', 'entropy': '1.169', 'num_tokens': '2.988e+07', 'mean_token_accuracy': '0.7125', 'epoch': '0.04892'}
{'loss': '0.8441', 'grad_norm': '0.4645', 'learning_rate': '0.0001211', 'entropy': '0.8563', 'num_tokens': '2.989e+07', 'mean_token_accuracy': '0.7939', 'epoch': '0.04893'}
{'loss': '1.029', 'grad_norm': '0.4194', 'learning_rate': '0.0001211', 'entropy': '1.017', 'num_tokens': '2.99e+07', 'mean_token_accuracy': '0.748', 'epoch': '0.04894'}
{'loss': '1.088', 'grad_norm': '0.401', 'learning_rate': '0.0001211', 'entropy': '1.026', 'num_tokens': '2.99e+07', 'mean_token_accuracy': '0.7367', 'epoch': '0.04895'}
{'loss': '0.8235', 'grad_norm': '0.4907', 'learning_rate': '0.000121', 'entropy': '0.8477', 'num_tokens': '2.991e+07', 'mean_token_accuracy': '0.7832', 'epoch': '0.04896'}
{'loss': '1.077', 'grad_norm': '0.445', 'learning_rate': '0.000121', 'entropy': '1.028', 'num_tokens': '2.991e+07', 'mean_token_accuracy': '0.7482', 'epoch': '0.04897'}
{'loss': '1.518', 'grad_norm': '0.5362', 'learning_rate': '0.000121', 'entropy': '1.375', 'num_tokens': '2.992e+07', 'mean_token_accuracy': '0.6818', 'epoch': '0.04897'}
{'loss': '1.08', 'grad_norm': '0.3901', 'learning_rate': '0.000121', 'entropy': '1', 'num_tokens': '2.992e+07', 'mean_token_accuracy': '0.7399', 'epoch': '0.04898'}
{'loss': '0.8149', 'grad_norm': '0.3342', 'learning_rate': '0.0001209', 'entropy': '0.7629', 'num_tokens': '2.993e+07', 'mean_token_accuracy': '0.803', 'epoch': '0.04899'}
{'loss': '0.8388', 'grad_norm': '0.3568', 'learning_rate': '0.0001209', 'entropy': '0.8536', 'num_tokens': '2.994e+07', 'mean_token_accuracy': '0.768', 'epoch': '0.049'}
{'loss': '0.5996', 'grad_norm': '0.301', 'learning_rate': '0.0001209', 'entropy': '0.6338', 'num_tokens': '2.994e+07', 'mean_token_accuracy': '0.8331', 'epoch': '0.04901'}
{'loss': '0.8355', 'grad_norm': '0.3604', 'learning_rate': '0.0001208', 'entropy': '1.011', 'num_tokens': '2.995e+07', 'mean_token_accuracy': '0.7537', 'epoch': '0.04902'}
{'loss': '1.084', 'grad_norm': '0.414', 'learning_rate': '0.0001208', 'entropy': '0.94', 'num_tokens': '2.995e+07', 'mean_token_accuracy': '0.7618', 'epoch': '0.04903'}
{'loss': '0.9398', 'grad_norm': '0.4433', 'learning_rate': '0.0001208', 'entropy': '0.8882', 'num_tokens': '2.996e+07', 'mean_token_accuracy': '0.7698', 'epoch': '0.04904'}
{'loss': '0.8072', 'grad_norm': '0.3267', 'learning_rate': '0.0001208', 'entropy': '0.8132', 'num_tokens': '2.996e+07', 'mean_token_accuracy': '0.8027', 'epoch': '0.04905'}
{'loss': '1.462', 'grad_norm': '0.5698', 'learning_rate': '0.0001207', 'entropy': '1.297', 'num_tokens': '2.997e+07', 'mean_token_accuracy': '0.6951', 'epoch': '0.04905'}
{'loss': '0.8931', 'grad_norm': '0.4525', 'learning_rate': '0.0001207', 'entropy': '0.9506', 'num_tokens': '2.997e+07', 'mean_token_accuracy': '0.7711', 'epoch': '0.04906'}
{'loss': '0.7474', 'grad_norm': '0.4005', 'learning_rate': '0.0001207', 'entropy': '0.764', 'num_tokens': '2.998e+07', 'mean_token_accuracy': '0.7977', 'epoch': '0.04907'}
{'loss': '0.8662', 'grad_norm': '0.3936', 'learning_rate': '0.0001207', 'entropy': '0.9254', 'num_tokens': '2.998e+07', 'mean_token_accuracy': '0.7732', 'epoch': '0.04908'}
{'loss': '0.8338', 'grad_norm': '0.4185', 'learning_rate': '0.0001206', 'entropy': '0.9179', 'num_tokens': '2.998e+07', 'mean_token_accuracy': '0.7663', 'epoch': '0.04909'}
{'loss': '0.7419', 'grad_norm': '0.3353', 'learning_rate': '0.0001206', 'entropy': '0.7995', 'num_tokens': '2.999e+07', 'mean_token_accuracy': '0.7977', 'epoch': '0.0491'}
{'loss': '0.8888', 'grad_norm': '0.3107', 'learning_rate': '0.0001206', 'entropy': '1.043', 'num_tokens': '3e+07', 'mean_token_accuracy': '0.7422', 'epoch': '0.04911'}

 46%|████▋     | 5550/12000 [3:17:47<4:42:16,  2.63s/it]
 46%|████▋     | 5551/12000 [3:17:50<4:47:07,  2.67s/it]
                                                        

 46%|████▋     | 5551/12000 [3:17:50<4:47:07,  2.67s/it]
 46%|████▋     | 5552/12000 [3:17:51<4:20:23,  2.42s/it]
                                                        

 46%|████▋     | 5552/12000 [3:17:51<4:20:23,  2.42s/it]
 46%|████▋     | 5553/12000 [3:17:53<3:52:19,  2.16s/it]
                                                        

 46%|████▋     | 5553/12000 [3:17:53<3:52:19,  2.16s/it]
 46%|████▋     | 5554/12000 [3:17:54<3:24:01,  1.90s/it]
                                                        

 46%|████▋     | 5554/12000 [3:17:54<3:24:01,  1.90s/it]
 46%|████▋     | 5555/12000 [3:17:56<3:16:31,  1.83s/it]
                                                        

 46%|████▋     | 5555/12000 [3:17:56<3:16:31,  1.83s/it]
 46%|████▋     | 5556/12000 [3:17:58<3:12:39,  1.79s/it]
                                                        

 46%|████▋     | 5556/12000 [3:17:58<3:12:39,  1.79s/it]
 46%|████▋     | 5557/12000 [3:17:59<3:12:14,  1.79s/it]
                                                        

 46%|████▋     | 5557/12000 [3:17:59<3:12:14,  1.79s/it]
 46%|████▋     | 5558/12000 [3:18:03<4:03:28,  2.27s/it]
                                                        

 46%|████▋     | 5558/12000 [3:18:03<4:03:28,  2.27s/it]
 46%|████▋     | 5559/12000 [3:18:05<3:46:46,  2.11s/it]
                                                        

 46%|████▋     | 5559/12000 [3:18:05<3:46:46,  2.11s/it]
 46%|████▋     | 5560/12000 [3:18:07<3:42:36,  2.07s/it]
                                                        

 46%|████▋     | 5560/12000 [3:18:07<3:42:36,  2.07s/it]
 46%|████▋     | 5561/12000 [3:18:08<3:21:52,  1.88s/it]
                                                        

 46%|████▋     | 5561/12000 [3:18:08<3:21:52,  1.88s/it]
 46%|████▋     | 5562/12000 [3:18:10<3:29:37,  1.95s/it]
                                                        

 46%|████▋     | 5562/12000 [3:18:10<3:29:37,  1.95s/it]
 46%|████▋     | 5563/12000 [3:18:12<3:13:22,  1.80s/it]
                                                        

 46%|████▋     | 5563/12000 [3:18:12<3:13:22,  1.80s/it]
 46%|████▋     | 5564/12000 [3:18:13<3:02:29,  1.70s/it]
                                                        

 46%|████▋     | 5564/12000 [3:18:13<3:02:29,  1.70s/it]
 46%|████▋     | 5565/12000 [3:18:14<2:55:55,  1.64s/it]
                                                        

 46%|████▋     | 5565/12000 [3:18:14<2:55:55,  1.64s/it]
 46%|████▋     | 5566/12000 [3:18:16<2:55:49,  1.64s/it]
                                                        

 46%|████▋     | 5566/12000 [3:18:16<2:55:49,  1.64s/it]
 46%|████▋     | 5567/12000 [3:18:18<3:10:18,  1.77s/it]
                                                        

 46%|████▋     | 5567/12000 [3:18:18<3:10:18,  1.77s/it]
 46%|████▋     | 5568/12000 [3:18:20<3:04:07,  1.72s/it]
                                                        

 46%|████▋     | 5568/12000 [3:18:20<3:04:07,  1.72s/it]
 46%|████▋     | 5569/12000 [3:18:21<2:58:12,  1.66s/it]
                                                        

 46%|████▋     | 5569/12000 [3:18:21<2:58:12,  1.66s/it]
 46%|████▋     | 5570/12000 [3:18:23<2:53:09,  1.62s/it]
                                                        

 46%|████▋     | 5570/12000 [3:18:23<2:53:09,  1.62s/it]
 46%|████▋     | 5571/12000 [3:18:25<2:57:50,  1.66s/it]
                                                        

 46%|████▋     | 5571/12000 [3:18:25<2:57:50,  1.66s/it]
 46%|████▋     | 5572/12000 [3:18:26<3:04:40,  1.72s/it]
                                                        

 46%|████▋     | 5572/12000 [3:18:26<3:04:40,  1.72s/it]
 46%|████▋     | 5573/12000 [3:18:28<3:01:31,  1.69s/it]
                                                        

 46%|████▋     | 5573/12000 [3:18:28<3:01:31,  1.69s/it]
 46%|████▋     | 5574/12000 [3:18:31<3:46:34,  2.12s/it]
                                                        

 46%|████▋     | 5574/12000 [3:18:31<3:46:34,  2.12s/it]
 46%|████▋     | 5575/12000 [3:18:33<3:25:11,  1.92s/it]
                                                        

 46%|████▋     | 5575/12000 [3:18:33<3:25:11,  1.92s/it]
 46%|████▋     | 5576/12000 [3:18:34<3:10:00,  1.77s/it]
                                                        

 46%|████▋     | 5576/12000 [3:18:34<3:10:00,  1.77s/it]
 46%|████▋     | 5577/12000 [3:18:40<5:24:57,  3.04s/it]
                                                        

 46%|████▋     | 5577/12000 [3:18:40<5:24:57,  3.04s/it]
 46%|████▋     | 5578/12000 [3:18:42<4:37:49,  2.60s/it]
                                                        

 46%|████▋     | 5578/12000 [3:18:42<4:37:49,  2.60s/it]
 46%|████▋     | 5579/12000 [3:18:43<4:04:14,  2.28s/it]
                                                        

 46%|████▋     | 5579/12000 [3:18:43<4:04:14,  2.28s/it]
 46%|████▋     | 5580/12000 [3:18:47<4:37:38,  2.59s/it]
                                                        

 46%|████▋     | 5580/12000 [3:18:47<4:37:38,  2.59s/it]
 47%|████▋     | 5581/12000 [3:18:48<4:06:57,  2.31s/it]
                                                        

 47%|████▋     | 5581/12000 [3:18:48<4:06:57,  2.31s/it]
 47%|████▋     | 5582/12000 [3:18:50<4:02:10,  2.26s/it]
                                                        

 47%|████▋     | 5582/12000 [3:18:50<4:02:10,  2.26s/it]
 47%|████▋     | 5583/12000 [3:18:52<3:32:15,  1.98s/it]
                                                        

 47%|████▋     | 5583/12000 [3:18:52<3:32:15,  1.98s/it]
 47%|████▋     | 5584/12000 [3:18:55<4:09:55,  2.34s/it]
                                                        

 47%|████▋     | 5584/12000 [3:18:55<4:09:55,  2.34s/it]
 47%|████▋     | 5585/12000 [3:18:56<3:37:58,  2.04s/it]
                                                        

 47%|████▋     | 5585/12000 [3:18:56<3:37:58,  2.04s/it]
 47%|████▋     | 5586/12000 [3:18:58<3:20:38,  1.88s/it]
                                                        

 47%|████▋     | 5586/12000 [3:18:58<3:20:38,  1.88s/it]
 47%|████▋     | 5587/12000 [3:19:00<3:40:37,  2.06s/it]
                                                        

 47%|████▋     | 5587/12000 [3:19:00<3:40:37,  2.06s/it]
 47%|████▋     | 5588/12000 [3:19:02<3:26:54,  1.94s/it]
                                                        

 47%|████▋     | 5588/12000 [3:19:02<3:26:54,  1.94s/it]
 47%|████▋     | 5589/12000 [3:19:04<3:26:39,  1.93s/it]
                                                        

 47%|████▋     | 5589/12000 [3:19:04<3:26:39,  1.93s/it]
 47%|████▋     | 5590/12000 [3:19:07<3:57:48,  2.23s/it]
                                                        

 47%|████▋     | 5590/12000 [3:19:07<3:57:48,  2.23s/it]
 47%|████▋     | 5591/12000 [3:19:09<3:58:57,  2.24s/it]
                                                        

 47%|████▋     | 5591/12000 [3:19:09<3:58:57,  2.24s/it]
 47%|████▋     | 5592/12000 [3:19:10<3:31:36,  1.98s/it]
                                                        

 47%|████▋     | 5592/12000 [3:19:10<3:31:36,  1.98s/it]
 47%|████▋     | 5593/12000 [3:19:12<3:11:19,  1.79s/it]
                                                        

 47%|████▋     | 5593/12000 [3:19:12<3:11:19,  1.79s/it]
 47%|████▋     | 5594/12000 [3:19:14<3:18:20,  1.86s/it]
                                                        

 47%|████▋     | 5594/12000 [3:19:14<3:18:20,  1.86s/it]
 47%|████▋     | 5595/12000 [3:19:15<3:17:05,  1.85s/it]
                                                        

 47%|████▋     | 5595/12000 [3:19:15<3:17:05,  1.85s/it]
 47%|████▋     | 5596/12000 [3:19:17<3:06:03,  1.74s/it]
                                                        

 47%|████▋     | 5596/12000 [3:19:17<3:06:03,  1.74s/it]
 47%|████▋     | 5597/12000 [3:19:19<3:02:24,  1.71s/it]
                                                        
{'loss': '0.8356', 'grad_norm': '0.3753', 'learning_rate': '0.0001205', 'entropy': '0.8484', 'num_tokens': '3e+07', 'mean_token_accuracy': '0.7932', 'epoch': '0.04912'}
{'loss': '0.7557', 'grad_norm': '0.4586', 'learning_rate': '0.0001205', 'entropy': '0.762', 'num_tokens': '3.001e+07', 'mean_token_accuracy': '0.8028', 'epoch': '0.04913'}
{'loss': '1.006', 'grad_norm': '0.4521', 'learning_rate': '0.0001205', 'entropy': '0.951', 'num_tokens': '3.001e+07', 'mean_token_accuracy': '0.7673', 'epoch': '0.04913'}
{'loss': '0.8663', 'grad_norm': '0.3327', 'learning_rate': '0.0001205', 'entropy': '0.9141', 'num_tokens': '3.002e+07', 'mean_token_accuracy': '0.7638', 'epoch': '0.04914'}
{'loss': '0.964', 'grad_norm': '0.4531', 'learning_rate': '0.0001204', 'entropy': '0.9837', 'num_tokens': '3.002e+07', 'mean_token_accuracy': '0.7569', 'epoch': '0.04915'}
{'loss': '0.8534', 'grad_norm': '0.4564', 'learning_rate': '0.0001204', 'entropy': '0.793', 'num_tokens': '3.002e+07', 'mean_token_accuracy': '0.7882', 'epoch': '0.04916'}
{'loss': '0.7544', 'grad_norm': '0.4376', 'learning_rate': '0.0001204', 'entropy': '0.7981', 'num_tokens': '3.003e+07', 'mean_token_accuracy': '0.7957', 'epoch': '0.04917'}
{'loss': '1.282', 'grad_norm': '0.475', 'learning_rate': '0.0001204', 'entropy': '1.201', 'num_tokens': '3.003e+07', 'mean_token_accuracy': '0.7279', 'epoch': '0.04918'}
{'loss': '1.122', 'grad_norm': '0.4244', 'learning_rate': '0.0001203', 'entropy': '0.9483', 'num_tokens': '3.004e+07', 'mean_token_accuracy': '0.7734', 'epoch': '0.04919'}
{'loss': '0.6222', 'grad_norm': '0.4106', 'learning_rate': '0.0001203', 'entropy': '0.6808', 'num_tokens': '3.004e+07', 'mean_token_accuracy': '0.8267', 'epoch': '0.0492'}
{'loss': '0.8334', 'grad_norm': '0.4672', 'learning_rate': '0.0001203', 'entropy': '0.7882', 'num_tokens': '3.005e+07', 'mean_token_accuracy': '0.7986', 'epoch': '0.04921'}
{'loss': '0.7576', 'grad_norm': '0.3352', 'learning_rate': '0.0001203', 'entropy': '0.7362', 'num_tokens': '3.005e+07', 'mean_token_accuracy': '0.8073', 'epoch': '0.04921'}
{'loss': '0.6369', 'grad_norm': '0.3643', 'learning_rate': '0.0001202', 'entropy': '0.6237', 'num_tokens': '3.006e+07', 'mean_token_accuracy': '0.8379', 'epoch': '0.04922'}
{'loss': '1.045', 'grad_norm': '0.4227', 'learning_rate': '0.0001202', 'entropy': '1.102', 'num_tokens': '3.006e+07', 'mean_token_accuracy': '0.7304', 'epoch': '0.04923'}
{'loss': '1.091', 'grad_norm': '0.4195', 'learning_rate': '0.0001202', 'entropy': '1.144', 'num_tokens': '3.007e+07', 'mean_token_accuracy': '0.7343', 'epoch': '0.04924'}
{'loss': '0.6806', 'grad_norm': '0.3734', 'learning_rate': '0.0001201', 'entropy': '0.6832', 'num_tokens': '3.007e+07', 'mean_token_accuracy': '0.8274', 'epoch': '0.04925'}
{'loss': '0.6985', 'grad_norm': '0.3775', 'learning_rate': '0.0001201', 'entropy': '0.6856', 'num_tokens': '3.008e+07', 'mean_token_accuracy': '0.8169', 'epoch': '0.04926'}
{'loss': '1.116', 'grad_norm': '0.3962', 'learning_rate': '0.0001201', 'entropy': '0.9718', 'num_tokens': '3.008e+07', 'mean_token_accuracy': '0.7565', 'epoch': '0.04927'}
{'loss': '0.7019', 'grad_norm': '0.4272', 'learning_rate': '0.0001201', 'entropy': '0.7088', 'num_tokens': '3.009e+07', 'mean_token_accuracy': '0.8168', 'epoch': '0.04928'}
{'loss': '0.6987', 'grad_norm': '0.3516', 'learning_rate': '0.00012', 'entropy': '0.7516', 'num_tokens': '3.009e+07', 'mean_token_accuracy': '0.8243', 'epoch': '0.04928'}
{'loss': '0.7127', 'grad_norm': '0.3457', 'learning_rate': '0.00012', 'entropy': '0.7239', 'num_tokens': '3.01e+07', 'mean_token_accuracy': '0.8271', 'epoch': '0.04929'}
{'loss': '0.7038', 'grad_norm': '0.3877', 'learning_rate': '0.00012', 'entropy': '0.729', 'num_tokens': '3.01e+07', 'mean_token_accuracy': '0.808', 'epoch': '0.0493'}
{'loss': '0.6538', 'grad_norm': '0.3683', 'learning_rate': '0.00012', 'entropy': '0.6361', 'num_tokens': '3.011e+07', 'mean_token_accuracy': '0.8286', 'epoch': '0.04931'}
{'loss': '0.7621', 'grad_norm': '0.442', 'learning_rate': '0.0001199', 'entropy': '0.8106', 'num_tokens': '3.012e+07', 'mean_token_accuracy': '0.7974', 'epoch': '0.04932'}
{'loss': '0.7349', 'grad_norm': '0.3646', 'learning_rate': '0.0001199', 'entropy': '0.8104', 'num_tokens': '3.012e+07', 'mean_token_accuracy': '0.7978', 'epoch': '0.04933'}
{'loss': '0.8194', 'grad_norm': '0.459', 'learning_rate': '0.0001199', 'entropy': '0.8237', 'num_tokens': '3.013e+07', 'mean_token_accuracy': '0.7894', 'epoch': '0.04934'}
{'loss': '0.6924', 'grad_norm': '0.3592', 'learning_rate': '0.0001198', 'entropy': '0.6933', 'num_tokens': '3.013e+07', 'mean_token_accuracy': '0.8225', 'epoch': '0.04935'}
{'loss': '0.5714', 'grad_norm': '0.2717', 'learning_rate': '0.0001198', 'entropy': '0.7179', 'num_tokens': '3.014e+07', 'mean_token_accuracy': '0.8186', 'epoch': '0.04936'}
{'loss': '0.873', 'grad_norm': '0.4549', 'learning_rate': '0.0001198', 'entropy': '0.8425', 'num_tokens': '3.014e+07', 'mean_token_accuracy': '0.781', 'epoch': '0.04936'}
{'loss': '0.6013', 'grad_norm': '0.3804', 'learning_rate': '0.0001198', 'entropy': '0.667', 'num_tokens': '3.015e+07', 'mean_token_accuracy': '0.8301', 'epoch': '0.04937'}
{'loss': '0.5802', 'grad_norm': '0.2984', 'learning_rate': '0.0001197', 'entropy': '0.6083', 'num_tokens': '3.015e+07', 'mean_token_accuracy': '0.8402', 'epoch': '0.04938'}
{'loss': '0.753', 'grad_norm': '0.4069', 'learning_rate': '0.0001197', 'entropy': '0.7812', 'num_tokens': '3.016e+07', 'mean_token_accuracy': '0.7916', 'epoch': '0.04939'}
{'loss': '0.8966', 'grad_norm': '0.3999', 'learning_rate': '0.0001197', 'entropy': '0.8229', 'num_tokens': '3.016e+07', 'mean_token_accuracy': '0.7931', 'epoch': '0.0494'}
{'loss': '0.8998', 'grad_norm': '0.4024', 'learning_rate': '0.0001197', 'entropy': '0.8645', 'num_tokens': '3.017e+07', 'mean_token_accuracy': '0.7671', 'epoch': '0.04941'}
{'loss': '0.5038', 'grad_norm': '0.293', 'learning_rate': '0.0001196', 'entropy': '0.6639', 'num_tokens': '3.018e+07', 'mean_token_accuracy': '0.8303', 'epoch': '0.04942'}
{'loss': '0.9712', 'grad_norm': '0.4628', 'learning_rate': '0.0001196', 'entropy': '0.9786', 'num_tokens': '3.018e+07', 'mean_token_accuracy': '0.773', 'epoch': '0.04943'}
{'loss': '0.6729', 'grad_norm': '0.3433', 'learning_rate': '0.0001196', 'entropy': '0.666', 'num_tokens': '3.018e+07', 'mean_token_accuracy': '0.8189', 'epoch': '0.04944'}
{'loss': '0.8523', 'grad_norm': '0.3603', 'learning_rate': '0.0001195', 'entropy': '0.7781', 'num_tokens': '3.019e+07', 'mean_token_accuracy': '0.8057', 'epoch': '0.04944'}
{'loss': '1.123', 'grad_norm': '0.4141', 'learning_rate': '0.0001195', 'entropy': '1.02', 'num_tokens': '3.019e+07', 'mean_token_accuracy': '0.7517', 'epoch': '0.04945'}
{'loss': '1.028', 'grad_norm': '0.4095', 'learning_rate': '0.0001195', 'entropy': '1.042', 'num_tokens': '3.02e+07', 'mean_token_accuracy': '0.74', 'epoch': '0.04946'}
{'loss': '0.7068', 'grad_norm': '0.348', 'learning_rate': '0.0001195', 'entropy': '0.842', 'num_tokens': '3.021e+07', 'mean_token_accuracy': '0.7944', 'epoch': '0.04947'}
{'loss': '0.785', 'grad_norm': '0.5764', 'learning_rate': '0.0001194', 'entropy': '0.8693', 'num_tokens': '3.021e+07', 'mean_token_accuracy': '0.7827', 'epoch': '0.04948'}
{'loss': '0.8258', 'grad_norm': '0.4428', 'learning_rate': '0.0001194', 'entropy': '0.8178', 'num_tokens': '3.022e+07', 'mean_token_accuracy': '0.7961', 'epoch': '0.04949'}
{'loss': '1.202', 'grad_norm': '0.538', 'learning_rate': '0.0001194', 'entropy': '1.206', 'num_tokens': '3.022e+07', 'mean_token_accuracy': '0.7085', 'epoch': '0.0495'}
{'loss': '0.7429', 'grad_norm': '0.3668', 'learning_rate': '0.0001194', 'entropy': '0.7819', 'num_tokens': '3.023e+07', 'mean_token_accuracy': '0.8061', 'epoch': '0.04951'}
{'loss': '0.6118', 'grad_norm': '0.3817', 'learning_rate': '0.0001193', 'entropy': '0.6239', 'num_tokens': '3.023e+07', 'mean_token_accuracy': '0.8336', 'epoch': '0.04951'}
{'loss': '0.8994', 'grad_norm': '0.4159', 'learning_rate': '0.0001193', 'entropy': '0.8288', 'num_tokens': '3.024e+07', 'mean_token_accuracy': '0.7758', 'epoch': '0.04952'}

 47%|████▋     | 5597/12000 [3:19:19<3:02:24,  1.71s/it]
 47%|████▋     | 5598/12000 [3:19:21<3:09:56,  1.78s/it]
                                                        

 47%|████▋     | 5598/12000 [3:19:21<3:09:56,  1.78s/it]
 47%|████▋     | 5599/12000 [3:19:22<2:57:45,  1.67s/it]
                                                        

 47%|████▋     | 5599/12000 [3:19:22<2:57:45,  1.67s/it]
 47%|████▋     | 5600/12000 [3:19:25<3:51:34,  2.17s/it]
                                                        

 47%|████▋     | 5600/12000 [3:19:25<3:51:34,  2.17s/it]
 47%|████▋     | 5601/12000 [3:19:29<4:34:07,  2.57s/it]
                                                        

 47%|████▋     | 5601/12000 [3:19:29<4:34:07,  2.57s/it]
 47%|████▋     | 5602/12000 [3:19:32<5:04:16,  2.85s/it]
                                                        

 47%|████▋     | 5602/12000 [3:19:32<5:04:16,  2.85s/it]
 47%|████▋     | 5603/12000 [3:19:34<4:23:54,  2.48s/it]
                                                        

 47%|████▋     | 5603/12000 [3:19:34<4:23:54,  2.48s/it]
 47%|████▋     | 5604/12000 [3:19:36<4:24:57,  2.49s/it]
                                                        

 47%|████▋     | 5604/12000 [3:19:36<4:24:57,  2.49s/it]
 47%|████▋     | 5605/12000 [3:19:38<3:58:18,  2.24s/it]
                                                        

 47%|████▋     | 5605/12000 [3:19:38<3:58:18,  2.24s/it]
 47%|████▋     | 5606/12000 [3:19:43<5:12:05,  2.93s/it]
                                                        

 47%|████▋     | 5606/12000 [3:19:43<5:12:05,  2.93s/it]
 47%|████▋     | 5607/12000 [3:19:44<4:25:27,  2.49s/it]
                                                        

 47%|████▋     | 5607/12000 [3:19:44<4:25:27,  2.49s/it]
 47%|████▋     | 5608/12000 [3:19:46<3:57:01,  2.22s/it]
                                                        

 47%|████▋     | 5608/12000 [3:19:46<3:57:01,  2.22s/it]
 47%|████▋     | 5609/12000 [3:19:47<3:39:08,  2.06s/it]
                                                        

 47%|████▋     | 5609/12000 [3:19:47<3:39:08,  2.06s/it]
 47%|████▋     | 5610/12000 [3:19:49<3:25:23,  1.93s/it]
                                                        

 47%|████▋     | 5610/12000 [3:19:49<3:25:23,  1.93s/it]
 47%|████▋     | 5611/12000 [3:19:51<3:36:52,  2.04s/it]
                                                        

 47%|████▋     | 5611/12000 [3:19:51<3:36:52,  2.04s/it]
 47%|████▋     | 5612/12000 [3:19:53<3:35:00,  2.02s/it]
                                                        

 47%|████▋     | 5612/12000 [3:19:53<3:35:00,  2.02s/it]
 47%|████▋     | 5613/12000 [3:19:55<3:42:31,  2.09s/it]
                                                        

 47%|████▋     | 5613/12000 [3:19:55<3:42:31,  2.09s/it]
 47%|████▋     | 5614/12000 [3:19:57<3:29:31,  1.97s/it]
                                                        

 47%|████▋     | 5614/12000 [3:19:57<3:29:31,  1.97s/it]
 47%|████▋     | 5615/12000 [3:19:59<3:36:09,  2.03s/it]
                                                        

 47%|████▋     | 5615/12000 [3:19:59<3:36:09,  2.03s/it]
 47%|████▋     | 5616/12000 [3:20:01<3:29:51,  1.97s/it]
                                                        

 47%|████▋     | 5616/12000 [3:20:01<3:29:51,  1.97s/it]
 47%|████▋     | 5617/12000 [3:20:03<3:16:19,  1.85s/it]
                                                        

 47%|████▋     | 5617/12000 [3:20:03<3:16:19,  1.85s/it]
 47%|████▋     | 5618/12000 [3:20:05<3:19:04,  1.87s/it]
                                                        

 47%|████▋     | 5618/12000 [3:20:05<3:19:04,  1.87s/it]
 47%|████▋     | 5619/12000 [3:20:06<3:11:29,  1.80s/it]
                                                        

 47%|████▋     | 5619/12000 [3:20:06<3:11:29,  1.80s/it]
 47%|████▋     | 5620/12000 [3:20:08<3:02:44,  1.72s/it]
                                                        

 47%|████▋     | 5620/12000 [3:20:08<3:02:44,  1.72s/it]
 47%|████▋     | 5621/12000 [3:20:11<3:39:15,  2.06s/it]
                                                        

 47%|████▋     | 5621/12000 [3:20:11<3:39:15,  2.06s/it]
 47%|████▋     | 5622/12000 [3:20:13<3:40:54,  2.08s/it]
                                                        

 47%|████▋     | 5622/12000 [3:20:13<3:40:54,  2.08s/it]
 47%|████▋     | 5623/12000 [3:20:14<3:27:47,  1.95s/it]
                                                        

 47%|████▋     | 5623/12000 [3:20:14<3:27:47,  1.95s/it]
 47%|████▋     | 5624/12000 [3:20:18<4:09:23,  2.35s/it]
                                                        

 47%|████▋     | 5624/12000 [3:20:18<4:09:23,  2.35s/it]
 47%|████▋     | 5625/12000 [3:20:21<4:36:43,  2.60s/it]
                                                        

 47%|████▋     | 5625/12000 [3:20:21<4:36:43,  2.60s/it]
 47%|████▋     | 5626/12000 [3:20:25<5:13:24,  2.95s/it]
                                                        

 47%|████▋     | 5626/12000 [3:20:25<5:13:24,  2.95s/it]
 47%|████▋     | 5627/12000 [3:20:27<5:00:07,  2.83s/it]
                                                        

 47%|████▋     | 5627/12000 [3:20:27<5:00:07,  2.83s/it]
 47%|████▋     | 5628/12000 [3:20:30<5:04:57,  2.87s/it]
                                                        

 47%|████▋     | 5628/12000 [3:20:30<5:04:57,  2.87s/it]
 47%|████▋     | 5629/12000 [3:20:33<5:15:52,  2.97s/it]
                                                        

 47%|████▋     | 5629/12000 [3:20:33<5:15:52,  2.97s/it]
 47%|████▋     | 5630/12000 [3:20:35<4:31:20,  2.56s/it]
                                                        

 47%|████▋     | 5630/12000 [3:20:35<4:31:20,  2.56s/it]
 47%|████▋     | 5631/12000 [3:20:38<4:55:36,  2.78s/it]
                                                        

 47%|████▋     | 5631/12000 [3:20:38<4:55:36,  2.78s/it]
 47%|████▋     | 5632/12000 [3:20:40<4:15:28,  2.41s/it]
                                                        

 47%|████▋     | 5632/12000 [3:20:40<4:15:28,  2.41s/it]
 47%|████▋     | 5633/12000 [3:20:41<3:44:12,  2.11s/it]
                                                        

 47%|████▋     | 5633/12000 [3:20:41<3:44:12,  2.11s/it]
 47%|████▋     | 5634/12000 [3:20:44<3:57:30,  2.24s/it]
                                                        

 47%|████▋     | 5634/12000 [3:20:44<3:57:30,  2.24s/it]
 47%|████▋     | 5635/12000 [3:20:46<3:55:02,  2.22s/it]
                                                        

 47%|████▋     | 5635/12000 [3:20:46<3:55:02,  2.22s/it]
 47%|████▋     | 5636/12000 [3:20:48<3:33:19,  2.01s/it]
                                                        

 47%|████▋     | 5636/12000 [3:20:48<3:33:19,  2.01s/it]
 47%|████▋     | 5637/12000 [3:20:49<3:28:47,  1.97s/it]
                                                        

 47%|████▋     | 5637/12000 [3:20:49<3:28:47,  1.97s/it]
 47%|████▋     | 5638/12000 [3:20:51<3:22:51,  1.91s/it]
                                                        

 47%|████▋     | 5638/12000 [3:20:51<3:22:51,  1.91s/it]
 47%|████▋     | 5639/12000 [3:20:54<3:46:09,  2.13s/it]
                                                        

 47%|████▋     | 5639/12000 [3:20:54<3:46:09,  2.13s/it]
 47%|████▋     | 5640/12000 [3:20:56<3:52:26,  2.19s/it]
                                                        

 47%|████▋     | 5640/12000 [3:20:56<3:52:26,  2.19s/it]
 47%|████▋     | 5641/12000 [3:20:58<3:26:45,  1.95s/it]
                                                        

 47%|████▋     | 5641/12000 [3:20:58<3:26:45,  1.95s/it]
 47%|████▋     | 5642/12000 [3:20:59<3:12:15,  1.81s/it]
                                                        

 47%|████▋     | 5642/12000 [3:20:59<3:12:15,  1.81s/it]
 47%|████▋     | 5643/12000 [3:21:01<3:01:38,  1.71s/it]
                                                        

 47%|████▋     | 5643/12000 [3:21:01<3:01:38,  1.71s/it]
 47%|████▋     | 5644/12000 [3:21:02<2:52:35,  1.63s/it]
                                                        
{'loss': '0.7494', 'grad_norm': '0.4374', 'learning_rate': '0.0001193', 'entropy': '0.7472', 'num_tokens': '3.024e+07', 'mean_token_accuracy': '0.8114', 'epoch': '0.04953'}
{'loss': '1.027', 'grad_norm': '0.3871', 'learning_rate': '0.0001193', 'entropy': '0.9858', 'num_tokens': '3.025e+07', 'mean_token_accuracy': '0.7531', 'epoch': '0.04954'}
{'loss': '0.9383', 'grad_norm': '0.4449', 'learning_rate': '0.0001192', 'entropy': '0.995', 'num_tokens': '3.025e+07', 'mean_token_accuracy': '0.7694', 'epoch': '0.04955'}
{'loss': '0.9415', 'grad_norm': '0.3263', 'learning_rate': '0.0001192', 'entropy': '0.9914', 'num_tokens': '3.026e+07', 'mean_token_accuracy': '0.7581', 'epoch': '0.04956'}
{'loss': '0.9664', 'grad_norm': '0.4163', 'learning_rate': '0.0001192', 'entropy': '0.9059', 'num_tokens': '3.027e+07', 'mean_token_accuracy': '0.7765', 'epoch': '0.04957'}
{'loss': '0.7186', 'grad_norm': '0.3598', 'learning_rate': '0.0001191', 'entropy': '0.7557', 'num_tokens': '3.027e+07', 'mean_token_accuracy': '0.7915', 'epoch': '0.04958'}
{'loss': '0.9189', 'grad_norm': '0.3536', 'learning_rate': '0.0001191', 'entropy': '0.826', 'num_tokens': '3.028e+07', 'mean_token_accuracy': '0.8049', 'epoch': '0.04959'}
{'loss': '1.029', 'grad_norm': '0.3707', 'learning_rate': '0.0001191', 'entropy': '1.004', 'num_tokens': '3.028e+07', 'mean_token_accuracy': '0.7632', 'epoch': '0.04959'}
{'loss': '1.087', 'grad_norm': '0.5023', 'learning_rate': '0.0001191', 'entropy': '0.9881', 'num_tokens': '3.029e+07', 'mean_token_accuracy': '0.7592', 'epoch': '0.0496'}
{'loss': '1.163', 'grad_norm': '0.4085', 'learning_rate': '0.000119', 'entropy': '1.057', 'num_tokens': '3.03e+07', 'mean_token_accuracy': '0.7494', 'epoch': '0.04961'}
{'loss': '0.7348', 'grad_norm': '0.3812', 'learning_rate': '0.000119', 'entropy': '0.77', 'num_tokens': '3.03e+07', 'mean_token_accuracy': '0.8001', 'epoch': '0.04962'}
{'loss': '0.7824', 'grad_norm': '0.536', 'learning_rate': '0.000119', 'entropy': '0.8108', 'num_tokens': '3.03e+07', 'mean_token_accuracy': '0.7767', 'epoch': '0.04963'}
{'loss': '0.659', 'grad_norm': '0.3322', 'learning_rate': '0.000119', 'entropy': '0.7264', 'num_tokens': '3.031e+07', 'mean_token_accuracy': '0.8174', 'epoch': '0.04964'}
{'loss': '0.8326', 'grad_norm': '0.4353', 'learning_rate': '0.0001189', 'entropy': '0.8369', 'num_tokens': '3.031e+07', 'mean_token_accuracy': '0.7829', 'epoch': '0.04965'}
{'loss': '0.7301', 'grad_norm': '0.3312', 'learning_rate': '0.0001189', 'entropy': '0.7589', 'num_tokens': '3.032e+07', 'mean_token_accuracy': '0.8061', 'epoch': '0.04966'}
{'loss': '0.9885', 'grad_norm': '0.4013', 'learning_rate': '0.0001189', 'entropy': '1', 'num_tokens': '3.033e+07', 'mean_token_accuracy': '0.76', 'epoch': '0.04967'}
{'loss': '0.828', 'grad_norm': '0.3974', 'learning_rate': '0.0001188', 'entropy': '0.887', 'num_tokens': '3.033e+07', 'mean_token_accuracy': '0.7809', 'epoch': '0.04967'}
{'loss': '1.021', 'grad_norm': '0.4389', 'learning_rate': '0.0001188', 'entropy': '1.015', 'num_tokens': '3.034e+07', 'mean_token_accuracy': '0.7425', 'epoch': '0.04968'}
{'loss': '0.6727', 'grad_norm': '0.3436', 'learning_rate': '0.0001188', 'entropy': '0.7097', 'num_tokens': '3.034e+07', 'mean_token_accuracy': '0.8272', 'epoch': '0.04969'}
{'loss': '0.7451', 'grad_norm': '0.3608', 'learning_rate': '0.0001188', 'entropy': '0.7553', 'num_tokens': '3.035e+07', 'mean_token_accuracy': '0.8161', 'epoch': '0.0497'}
{'loss': '0.6815', 'grad_norm': '0.3725', 'learning_rate': '0.0001187', 'entropy': '0.7139', 'num_tokens': '3.035e+07', 'mean_token_accuracy': '0.8132', 'epoch': '0.04971'}
{'loss': '0.582', 'grad_norm': '0.3759', 'learning_rate': '0.0001187', 'entropy': '0.5601', 'num_tokens': '3.036e+07', 'mean_token_accuracy': '0.8412', 'epoch': '0.04972'}
{'loss': '0.803', 'grad_norm': '0.4452', 'learning_rate': '0.0001187', 'entropy': '0.8589', 'num_tokens': '3.036e+07', 'mean_token_accuracy': '0.7797', 'epoch': '0.04973'}
{'loss': '1.029', 'grad_norm': '0.5532', 'learning_rate': '0.0001187', 'entropy': '1.042', 'num_tokens': '3.037e+07', 'mean_token_accuracy': '0.7386', 'epoch': '0.04974'}
{'loss': '0.702', 'grad_norm': '0.2901', 'learning_rate': '0.0001186', 'entropy': '0.743', 'num_tokens': '3.037e+07', 'mean_token_accuracy': '0.8227', 'epoch': '0.04974'}
{'loss': '0.9643', 'grad_norm': '0.3419', 'learning_rate': '0.0001186', 'entropy': '1.005', 'num_tokens': '3.038e+07', 'mean_token_accuracy': '0.7537', 'epoch': '0.04975'}
{'loss': '0.6896', 'grad_norm': '0.4073', 'learning_rate': '0.0001186', 'entropy': '0.6858', 'num_tokens': '3.039e+07', 'mean_token_accuracy': '0.8186', 'epoch': '0.04976'}
{'loss': '0.8265', 'grad_norm': '0.386', 'learning_rate': '0.0001185', 'entropy': '0.7706', 'num_tokens': '3.039e+07', 'mean_token_accuracy': '0.8102', 'epoch': '0.04977'}
{'loss': '0.6498', 'grad_norm': '0.3278', 'learning_rate': '0.0001185', 'entropy': '0.6677', 'num_tokens': '3.04e+07', 'mean_token_accuracy': '0.8202', 'epoch': '0.04978'}
{'loss': '1.213', 'grad_norm': '0.4775', 'learning_rate': '0.0001185', 'entropy': '1.08', 'num_tokens': '3.041e+07', 'mean_token_accuracy': '0.7388', 'epoch': '0.04979'}
{'loss': '0.9654', 'grad_norm': '0.4677', 'learning_rate': '0.0001185', 'entropy': '0.8922', 'num_tokens': '3.042e+07', 'mean_token_accuracy': '0.778', 'epoch': '0.0498'}
{'loss': '0.9164', 'grad_norm': '0.3545', 'learning_rate': '0.0001184', 'entropy': '0.7746', 'num_tokens': '3.042e+07', 'mean_token_accuracy': '0.8084', 'epoch': '0.04981'}
{'loss': '0.9432', 'grad_norm': '0.333', 'learning_rate': '0.0001184', 'entropy': '0.9199', 'num_tokens': '3.043e+07', 'mean_token_accuracy': '0.7762', 'epoch': '0.04982'}
{'loss': '0.7433', 'grad_norm': '0.377', 'learning_rate': '0.0001184', 'entropy': '0.7202', 'num_tokens': '3.044e+07', 'mean_token_accuracy': '0.8112', 'epoch': '0.04982'}
{'loss': '0.7403', 'grad_norm': '0.3407', 'learning_rate': '0.0001184', 'entropy': '0.6843', 'num_tokens': '3.044e+07', 'mean_token_accuracy': '0.825', 'epoch': '0.04983'}
{'loss': '0.7203', 'grad_norm': '0.3395', 'learning_rate': '0.0001183', 'entropy': '0.7786', 'num_tokens': '3.045e+07', 'mean_token_accuracy': '0.8053', 'epoch': '0.04984'}
{'loss': '0.595', 'grad_norm': '0.2984', 'learning_rate': '0.0001183', 'entropy': '0.5973', 'num_tokens': '3.045e+07', 'mean_token_accuracy': '0.8443', 'epoch': '0.04985'}
{'loss': '0.877', 'grad_norm': '0.5077', 'learning_rate': '0.0001183', 'entropy': '0.8105', 'num_tokens': '3.046e+07', 'mean_token_accuracy': '0.7909', 'epoch': '0.04986'}
{'loss': '0.8756', 'grad_norm': '0.4175', 'learning_rate': '0.0001183', 'entropy': '0.8203', 'num_tokens': '3.047e+07', 'mean_token_accuracy': '0.7894', 'epoch': '0.04987'}
{'loss': '0.6205', 'grad_norm': '0.3311', 'learning_rate': '0.0001182', 'entropy': '0.738', 'num_tokens': '3.047e+07', 'mean_token_accuracy': '0.8087', 'epoch': '0.04988'}
{'loss': '0.8779', 'grad_norm': '0.4353', 'learning_rate': '0.0001182', 'entropy': '0.8479', 'num_tokens': '3.048e+07', 'mean_token_accuracy': '0.79', 'epoch': '0.04989'}
{'loss': '1.3', 'grad_norm': '0.4638', 'learning_rate': '0.0001182', 'entropy': '1.163', 'num_tokens': '3.048e+07', 'mean_token_accuracy': '0.7294', 'epoch': '0.0499'}
{'loss': '1.188', 'grad_norm': '0.4833', 'learning_rate': '0.0001181', 'entropy': '1.084', 'num_tokens': '3.049e+07', 'mean_token_accuracy': '0.7403', 'epoch': '0.0499'}
{'loss': '1.147', 'grad_norm': '0.8308', 'learning_rate': '0.0001181', 'entropy': '1.162', 'num_tokens': '3.049e+07', 'mean_token_accuracy': '0.7159', 'epoch': '0.04991'}
{'loss': '0.9826', 'grad_norm': '0.457', 'learning_rate': '0.0001181', 'entropy': '0.9968', 'num_tokens': '3.05e+07', 'mean_token_accuracy': '0.752', 'epoch': '0.04992'}
{'loss': '0.8622', 'grad_norm': '0.3916', 'learning_rate': '0.0001181', 'entropy': '0.9544', 'num_tokens': '3.05e+07', 'mean_token_accuracy': '0.7711', 'epoch': '0.04993'}
{'loss': '0.714', 'grad_norm': '0.3298', 'learning_rate': '0.000118', 'entropy': '0.8047', 'num_tokens': '3.051e+07', 'mean_token_accuracy': '0.7857', 'epoch': '0.04994'}

 47%|████▋     | 5644/12000 [3:21:02<2:52:35,  1.63s/it]
 47%|████▋     | 5645/12000 [3:21:03<2:49:12,  1.60s/it]
                                                        

 47%|████▋     | 5645/12000 [3:21:03<2:49:12,  1.60s/it]
 47%|████▋     | 5646/12000 [3:21:07<3:48:12,  2.15s/it]
                                                        

 47%|████▋     | 5646/12000 [3:21:07<3:48:12,  2.15s/it]
 47%|████▋     | 5647/12000 [3:21:11<5:04:12,  2.87s/it]
                                                        

 47%|████▋     | 5647/12000 [3:21:11<5:04:12,  2.87s/it]
 47%|████▋     | 5648/12000 [3:21:15<5:22:08,  3.04s/it]
                                                        

 47%|████▋     | 5648/12000 [3:21:15<5:22:08,  3.04s/it]
 47%|████▋     | 5649/12000 [3:21:18<5:10:49,  2.94s/it]
                                                        

 47%|████▋     | 5649/12000 [3:21:18<5:10:49,  2.94s/it]
 47%|████▋     | 5650/12000 [3:21:19<4:31:52,  2.57s/it]
                                                        

 47%|████▋     | 5650/12000 [3:21:19<4:31:52,  2.57s/it]
 47%|████▋     | 5651/12000 [3:21:23<5:10:30,  2.93s/it]
                                                        

 47%|████▋     | 5651/12000 [3:21:23<5:10:30,  2.93s/it]
 47%|████▋     | 5652/12000 [3:21:26<5:13:56,  2.97s/it]
                                                        

 47%|████▋     | 5652/12000 [3:21:26<5:13:56,  2.97s/it]
 47%|████▋     | 5653/12000 [3:21:28<4:36:35,  2.61s/it]
                                                        

 47%|████▋     | 5653/12000 [3:21:28<4:36:35,  2.61s/it]
 47%|████▋     | 5654/12000 [3:21:31<4:37:13,  2.62s/it]
                                                        

 47%|████▋     | 5654/12000 [3:21:31<4:37:13,  2.62s/it]
 47%|████▋     | 5655/12000 [3:21:32<4:00:26,  2.27s/it]
                                                        

 47%|████▋     | 5655/12000 [3:21:32<4:00:26,  2.27s/it]
 47%|████▋     | 5656/12000 [3:21:34<3:35:23,  2.04s/it]
                                                        

 47%|████▋     | 5656/12000 [3:21:34<3:35:23,  2.04s/it]
 47%|████▋     | 5657/12000 [3:21:35<3:15:38,  1.85s/it]
                                                        

 47%|████▋     | 5657/12000 [3:21:35<3:15:38,  1.85s/it]
 47%|████▋     | 5658/12000 [3:21:37<3:34:08,  2.03s/it]
                                                        

 47%|████▋     | 5658/12000 [3:21:37<3:34:08,  2.03s/it]
 47%|████▋     | 5659/12000 [3:21:39<3:23:26,  1.93s/it]
                                                        

 47%|████▋     | 5659/12000 [3:21:39<3:23:26,  1.93s/it]
 47%|████▋     | 5660/12000 [3:21:42<3:48:26,  2.16s/it]
                                                        

 47%|████▋     | 5660/12000 [3:21:42<3:48:26,  2.16s/it]
 47%|████▋     | 5661/12000 [3:21:43<3:33:01,  2.02s/it]
                                                        

 47%|████▋     | 5661/12000 [3:21:43<3:33:01,  2.02s/it]
 47%|████▋     | 5662/12000 [3:21:45<3:15:43,  1.85s/it]
                                                        

 47%|████▋     | 5662/12000 [3:21:45<3:15:43,  1.85s/it]
 47%|████▋     | 5663/12000 [3:21:47<3:10:09,  1.80s/it]
                                                        

 47%|████▋     | 5663/12000 [3:21:47<3:10:09,  1.80s/it]
 47%|████▋     | 5664/12000 [3:21:48<2:57:56,  1.68s/it]
                                                        

 47%|████▋     | 5664/12000 [3:21:48<2:57:56,  1.68s/it]
 47%|████▋     | 5665/12000 [3:21:50<2:55:35,  1.66s/it]
                                                        

 47%|████▋     | 5665/12000 [3:21:50<2:55:35,  1.66s/it]
 47%|████▋     | 5666/12000 [3:21:52<3:13:37,  1.83s/it]
                                                        

 47%|████▋     | 5666/12000 [3:21:52<3:13:37,  1.83s/it]
 47%|████▋     | 5667/12000 [3:21:53<3:04:55,  1.75s/it]
                                                        

 47%|████▋     | 5667/12000 [3:21:53<3:04:55,  1.75s/it]
 47%|████▋     | 5668/12000 [3:21:57<3:59:22,  2.27s/it]
                                                        

 47%|████▋     | 5668/12000 [3:21:57<3:59:22,  2.27s/it]
 47%|████▋     | 5669/12000 [3:21:58<3:31:18,  2.00s/it]
                                                        

 47%|████▋     | 5669/12000 [3:21:58<3:31:18,  2.00s/it]
 47%|████▋     | 5670/12000 [3:22:01<3:39:52,  2.08s/it]
                                                        

 47%|████▋     | 5670/12000 [3:22:01<3:39:52,  2.08s/it]
 47%|████▋     | 5671/12000 [3:22:03<3:39:17,  2.08s/it]
                                                        

 47%|████▋     | 5671/12000 [3:22:03<3:39:17,  2.08s/it]
 47%|████▋     | 5672/12000 [3:22:04<3:25:05,  1.94s/it]
                                                        

 47%|████▋     | 5672/12000 [3:22:04<3:25:05,  1.94s/it]
 47%|████▋     | 5673/12000 [3:22:06<3:10:17,  1.80s/it]
                                                        

 47%|████▋     | 5673/12000 [3:22:06<3:10:17,  1.80s/it]
 47%|████▋     | 5674/12000 [3:22:07<3:07:24,  1.78s/it]
                                                        

 47%|████▋     | 5674/12000 [3:22:07<3:07:24,  1.78s/it]
 47%|████▋     | 5675/12000 [3:22:09<3:05:12,  1.76s/it]
                                                        

 47%|████▋     | 5675/12000 [3:22:09<3:05:12,  1.76s/it]
 47%|████▋     | 5676/12000 [3:22:11<3:02:46,  1.73s/it]
                                                        

 47%|████▋     | 5676/12000 [3:22:11<3:02:46,  1.73s/it]
 47%|████▋     | 5677/12000 [3:22:12<2:49:31,  1.61s/it]
                                                        

 47%|████▋     | 5677/12000 [3:22:12<2:49:31,  1.61s/it]
 47%|████▋     | 5678/12000 [3:22:14<2:59:46,  1.71s/it]
                                                        

 47%|████▋     | 5678/12000 [3:22:14<2:59:46,  1.71s/it]
 47%|████▋     | 5679/12000 [3:22:16<3:11:48,  1.82s/it]
                                                        

 47%|████▋     | 5679/12000 [3:22:16<3:11:48,  1.82s/it]
 47%|████▋     | 5680/12000 [3:22:18<3:19:03,  1.89s/it]
                                                        

 47%|████▋     | 5680/12000 [3:22:18<3:19:03,  1.89s/it]
 47%|████▋     | 5681/12000 [3:22:20<3:16:41,  1.87s/it]
                                                        

 47%|████▋     | 5681/12000 [3:22:20<3:16:41,  1.87s/it]
 47%|████▋     | 5682/12000 [3:22:23<3:51:18,  2.20s/it]
                                                        

 47%|████▋     | 5682/12000 [3:22:23<3:51:18,  2.20s/it]
 47%|████▋     | 5683/12000 [3:22:25<3:34:37,  2.04s/it]
                                                        

 47%|████▋     | 5683/12000 [3:22:25<3:34:37,  2.04s/it]
 47%|████▋     | 5684/12000 [3:22:27<3:49:31,  2.18s/it]
                                                        

 47%|████▋     | 5684/12000 [3:22:27<3:49:31,  2.18s/it]
 47%|████▋     | 5685/12000 [3:22:32<5:24:14,  3.08s/it]
                                                        

 47%|████▋     | 5685/12000 [3:22:32<5:24:14,  3.08s/it]
 47%|████▋     | 5686/12000 [3:22:35<5:06:34,  2.91s/it]
                                                        

 47%|████▋     | 5686/12000 [3:22:35<5:06:34,  2.91s/it]
 47%|████▋     | 5687/12000 [3:22:37<4:35:43,  2.62s/it]
                                                        

 47%|████▋     | 5687/12000 [3:22:37<4:35:43,  2.62s/it]
 47%|████▋     | 5688/12000 [3:22:41<5:35:05,  3.19s/it]
                                                        

 47%|████▋     | 5688/12000 [3:22:41<5:35:05,  3.19s/it]
 47%|████▋     | 5689/12000 [3:22:43<5:00:34,  2.86s/it]
                                                        

 47%|████▋     | 5689/12000 [3:22:43<5:00:34,  2.86s/it]
 47%|████▋     | 5690/12000 [3:22:47<5:24:05,  3.08s/it]
                                                        

 47%|████▋     | 5690/12000 [3:22:47<5:24:05,  3.08s/it]
 47%|████▋     | 5691/12000 [3:22:49<4:33:49,  2.60s/it]
                                                        
{'loss': '0.7141', 'grad_norm': '0.3999', 'learning_rate': '0.000118', 'entropy': '0.736', 'num_tokens': '3.051e+07', 'mean_token_accuracy': '0.8137', 'epoch': '0.04995'}
{'loss': '0.5628', 'grad_norm': '0.2987', 'learning_rate': '0.000118', 'entropy': '0.5981', 'num_tokens': '3.051e+07', 'mean_token_accuracy': '0.8434', 'epoch': '0.04996'}
{'loss': '0.6844', 'grad_norm': '0.3989', 'learning_rate': '0.000118', 'entropy': '0.7565', 'num_tokens': '3.052e+07', 'mean_token_accuracy': '0.8044', 'epoch': '0.04997'}
{'loss': '1.151', 'grad_norm': '0.4035', 'learning_rate': '0.0001179', 'entropy': '1.074', 'num_tokens': '3.052e+07', 'mean_token_accuracy': '0.7493', 'epoch': '0.04997'}
{'loss': '0.7353', 'grad_norm': '0.3129', 'learning_rate': '0.0001179', 'entropy': '0.7081', 'num_tokens': '3.053e+07', 'mean_token_accuracy': '0.8195', 'epoch': '0.04998'}
{'loss': '0.9616', 'grad_norm': '0.38', 'learning_rate': '0.0001179', 'entropy': '1.148', 'num_tokens': '3.054e+07', 'mean_token_accuracy': '0.7345', 'epoch': '0.04999'}
{'loss': '0.8362', 'grad_norm': '0.5234', 'learning_rate': '0.0001178', 'entropy': '0.8639', 'num_tokens': '3.055e+07', 'mean_token_accuracy': '0.7806', 'epoch': '0.05'}
{'loss': '0.5797', 'grad_norm': '0.2649', 'learning_rate': '0.0001178', 'entropy': '0.6072', 'num_tokens': '3.055e+07', 'mean_token_accuracy': '0.8416', 'epoch': '0.05001'}
{'loss': '0.7708', 'grad_norm': '0.3247', 'learning_rate': '0.0001178', 'entropy': '0.7424', 'num_tokens': '3.056e+07', 'mean_token_accuracy': '0.8081', 'epoch': '0.05002'}
{'loss': '0.6158', 'grad_norm': '0.371', 'learning_rate': '0.0001178', 'entropy': '0.6123', 'num_tokens': '3.056e+07', 'mean_token_accuracy': '0.8271', 'epoch': '0.05003'}
{'loss': '1.226', 'grad_norm': '0.4937', 'learning_rate': '0.0001177', 'entropy': '1.211', 'num_tokens': '3.057e+07', 'mean_token_accuracy': '0.713', 'epoch': '0.05004'}
{'loss': '0.66', 'grad_norm': '0.3788', 'learning_rate': '0.0001177', 'entropy': '0.7163', 'num_tokens': '3.057e+07', 'mean_token_accuracy': '0.8196', 'epoch': '0.05005'}
{'loss': '0.6828', 'grad_norm': '0.3274', 'learning_rate': '0.0001177', 'entropy': '0.6882', 'num_tokens': '3.058e+07', 'mean_token_accuracy': '0.8349', 'epoch': '0.05005'}
{'loss': '0.802', 'grad_norm': '0.4792', 'learning_rate': '0.0001177', 'entropy': '0.8504', 'num_tokens': '3.058e+07', 'mean_token_accuracy': '0.7945', 'epoch': '0.05006'}
{'loss': '0.6613', 'grad_norm': '0.3279', 'learning_rate': '0.0001176', 'entropy': '0.6502', 'num_tokens': '3.059e+07', 'mean_token_accuracy': '0.8259', 'epoch': '0.05007'}
{'loss': '0.7803', 'grad_norm': '0.3888', 'learning_rate': '0.0001176', 'entropy': '0.8378', 'num_tokens': '3.059e+07', 'mean_token_accuracy': '0.7936', 'epoch': '0.05008'}
{'loss': '0.6303', 'grad_norm': '0.3715', 'learning_rate': '0.0001176', 'entropy': '0.6018', 'num_tokens': '3.06e+07', 'mean_token_accuracy': '0.8441', 'epoch': '0.05009'}
{'loss': '0.6541', 'grad_norm': '0.406', 'learning_rate': '0.0001175', 'entropy': '0.6923', 'num_tokens': '3.061e+07', 'mean_token_accuracy': '0.8242', 'epoch': '0.0501'}
{'loss': '0.9844', 'grad_norm': '0.3885', 'learning_rate': '0.0001175', 'entropy': '0.9251', 'num_tokens': '3.061e+07', 'mean_token_accuracy': '0.759', 'epoch': '0.05011'}
{'loss': '0.7471', 'grad_norm': '0.4123', 'learning_rate': '0.0001175', 'entropy': '0.6936', 'num_tokens': '3.061e+07', 'mean_token_accuracy': '0.8049', 'epoch': '0.05012'}
{'loss': '0.8575', 'grad_norm': '0.4226', 'learning_rate': '0.0001175', 'entropy': '0.861', 'num_tokens': '3.062e+07', 'mean_token_accuracy': '0.7829', 'epoch': '0.05013'}
{'loss': '1.13', 'grad_norm': '0.5627', 'learning_rate': '0.0001174', 'entropy': '1.019', 'num_tokens': '3.062e+07', 'mean_token_accuracy': '0.7623', 'epoch': '0.05013'}
{'loss': '1.154', 'grad_norm': '0.4034', 'learning_rate': '0.0001174', 'entropy': '1.044', 'num_tokens': '3.063e+07', 'mean_token_accuracy': '0.7613', 'epoch': '0.05014'}
{'loss': '0.8517', 'grad_norm': '0.4042', 'learning_rate': '0.0001174', 'entropy': '0.8644', 'num_tokens': '3.063e+07', 'mean_token_accuracy': '0.7897', 'epoch': '0.05015'}
{'loss': '0.6533', 'grad_norm': '0.3424', 'learning_rate': '0.0001174', 'entropy': '0.6269', 'num_tokens': '3.064e+07', 'mean_token_accuracy': '0.8405', 'epoch': '0.05016'}
{'loss': '0.9378', 'grad_norm': '0.4089', 'learning_rate': '0.0001173', 'entropy': '0.9496', 'num_tokens': '3.064e+07', 'mean_token_accuracy': '0.7699', 'epoch': '0.05017'}
{'loss': '0.7449', 'grad_norm': '0.3284', 'learning_rate': '0.0001173', 'entropy': '0.7878', 'num_tokens': '3.065e+07', 'mean_token_accuracy': '0.7993', 'epoch': '0.05018'}
{'loss': '0.7431', 'grad_norm': '0.3877', 'learning_rate': '0.0001173', 'entropy': '0.7543', 'num_tokens': '3.066e+07', 'mean_token_accuracy': '0.8005', 'epoch': '0.05019'}
{'loss': '0.6665', 'grad_norm': '0.3643', 'learning_rate': '0.0001172', 'entropy': '0.664', 'num_tokens': '3.066e+07', 'mean_token_accuracy': '0.8316', 'epoch': '0.0502'}
{'loss': '1.154', 'grad_norm': '0.3912', 'learning_rate': '0.0001172', 'entropy': '1.13', 'num_tokens': '3.067e+07', 'mean_token_accuracy': '0.7258', 'epoch': '0.05021'}
{'loss': '0.6845', 'grad_norm': '0.3715', 'learning_rate': '0.0001172', 'entropy': '0.6585', 'num_tokens': '3.067e+07', 'mean_token_accuracy': '0.8306', 'epoch': '0.05021'}
{'loss': '0.5691', 'grad_norm': '0.3601', 'learning_rate': '0.0001172', 'entropy': '0.5631', 'num_tokens': '3.068e+07', 'mean_token_accuracy': '0.8544', 'epoch': '0.05022'}
{'loss': '0.7733', 'grad_norm': '0.4755', 'learning_rate': '0.0001171', 'entropy': '0.7966', 'num_tokens': '3.068e+07', 'mean_token_accuracy': '0.7852', 'epoch': '0.05023'}
{'loss': '0.9146', 'grad_norm': '0.4811', 'learning_rate': '0.0001171', 'entropy': '0.9451', 'num_tokens': '3.068e+07', 'mean_token_accuracy': '0.7682', 'epoch': '0.05024'}
{'loss': '0.9172', 'grad_norm': '0.5357', 'learning_rate': '0.0001171', 'entropy': '0.8802', 'num_tokens': '3.069e+07', 'mean_token_accuracy': '0.7825', 'epoch': '0.05025'}
{'loss': '0.7581', 'grad_norm': '0.3195', 'learning_rate': '0.0001171', 'entropy': '0.7652', 'num_tokens': '3.069e+07', 'mean_token_accuracy': '0.8021', 'epoch': '0.05026'}
{'loss': '1.259', 'grad_norm': '0.4341', 'learning_rate': '0.000117', 'entropy': '1.129', 'num_tokens': '3.07e+07', 'mean_token_accuracy': '0.7299', 'epoch': '0.05027'}
{'loss': '0.8584', 'grad_norm': '0.3738', 'learning_rate': '0.000117', 'entropy': '0.8513', 'num_tokens': '3.07e+07', 'mean_token_accuracy': '0.7921', 'epoch': '0.05028'}
{'loss': '1.108', 'grad_norm': '0.4925', 'learning_rate': '0.000117', 'entropy': '1.074', 'num_tokens': '3.071e+07', 'mean_token_accuracy': '0.7374', 'epoch': '0.05028'}
{'loss': '0.972', 'grad_norm': '0.3782', 'learning_rate': '0.0001169', 'entropy': '1.043', 'num_tokens': '3.072e+07', 'mean_token_accuracy': '0.7535', 'epoch': '0.05029'}
{'loss': '0.8832', 'grad_norm': '0.4246', 'learning_rate': '0.0001169', 'entropy': '0.7413', 'num_tokens': '3.072e+07', 'mean_token_accuracy': '0.8241', 'epoch': '0.0503'}
{'loss': '1.288', 'grad_norm': '0.3771', 'learning_rate': '0.0001169', 'entropy': '1.29', 'num_tokens': '3.073e+07', 'mean_token_accuracy': '0.6687', 'epoch': '0.05031'}
{'loss': '0.6938', 'grad_norm': '0.2946', 'learning_rate': '0.0001169', 'entropy': '0.6973', 'num_tokens': '3.074e+07', 'mean_token_accuracy': '0.8196', 'epoch': '0.05032'}
{'loss': '1.203', 'grad_norm': '0.5103', 'learning_rate': '0.0001168', 'entropy': '1.229', 'num_tokens': '3.074e+07', 'mean_token_accuracy': '0.7307', 'epoch': '0.05033'}
{'loss': '1.039', 'grad_norm': '0.3768', 'learning_rate': '0.0001168', 'entropy': '0.8606', 'num_tokens': '3.075e+07', 'mean_token_accuracy': '0.7846', 'epoch': '0.05034'}
{'loss': '1.042', 'grad_norm': '0.4763', 'learning_rate': '0.0001168', 'entropy': '0.8693', 'num_tokens': '3.076e+07', 'mean_token_accuracy': '0.7812', 'epoch': '0.05035'}
{'loss': '1.078', 'grad_norm': '0.4585', 'learning_rate': '0.0001168', 'entropy': '1.068', 'num_tokens': '3.076e+07', 'mean_token_accuracy': '0.7414', 'epoch': '0.05036'}

 47%|████▋     | 5691/12000 [3:22:49<4:33:49,  2.60s/it]
 47%|████▋     | 5692/12000 [3:22:50<4:01:55,  2.30s/it]
                                                        

 47%|████▋     | 5692/12000 [3:22:50<4:01:55,  2.30s/it]
 47%|████▋     | 5693/12000 [3:22:53<4:34:13,  2.61s/it]
                                                        

 47%|████▋     | 5693/12000 [3:22:53<4:34:13,  2.61s/it]
 47%|████▋     | 5694/12000 [3:22:55<4:01:38,  2.30s/it]
                                                        

 47%|████▋     | 5694/12000 [3:22:55<4:01:38,  2.30s/it]
 47%|████▋     | 5695/12000 [3:22:57<3:45:40,  2.15s/it]
                                                        

 47%|████▋     | 5695/12000 [3:22:57<3:45:40,  2.15s/it]
 47%|████▋     | 5696/12000 [3:22:58<3:26:24,  1.96s/it]
                                                        

 47%|████▋     | 5696/12000 [3:22:58<3:26:24,  1.96s/it]
 47%|████▋     | 5697/12000 [3:23:00<3:07:57,  1.79s/it]
                                                        

 47%|████▋     | 5697/12000 [3:23:00<3:07:57,  1.79s/it]
 47%|████▋     | 5698/12000 [3:23:01<3:04:30,  1.76s/it]
                                                        

 47%|████▋     | 5698/12000 [3:23:01<3:04:30,  1.76s/it]
 47%|████▋     | 5699/12000 [3:23:03<2:55:46,  1.67s/it]
                                                        

 47%|████▋     | 5699/12000 [3:23:03<2:55:46,  1.67s/it]
 48%|████▊     | 5700/12000 [3:23:05<2:57:28,  1.69s/it]
                                                        

 48%|████▊     | 5700/12000 [3:23:05<2:57:28,  1.69s/it]
 48%|████▊     | 5701/12000 [3:23:07<3:32:40,  2.03s/it]
                                                        

 48%|████▊     | 5701/12000 [3:23:07<3:32:40,  2.03s/it]
 48%|████▊     | 5702/12000 [3:23:09<3:20:34,  1.91s/it]
                                                        

 48%|████▊     | 5702/12000 [3:23:09<3:20:34,  1.91s/it]
 48%|████▊     | 5703/12000 [3:23:11<3:07:48,  1.79s/it]
                                                        

 48%|████▊     | 5703/12000 [3:23:11<3:07:48,  1.79s/it]
 48%|████▊     | 5704/12000 [3:23:13<3:16:25,  1.87s/it]
                                                        

 48%|████▊     | 5704/12000 [3:23:13<3:16:25,  1.87s/it]
 48%|████▊     | 5705/12000 [3:23:14<3:07:53,  1.79s/it]
                                                        

 48%|████▊     | 5705/12000 [3:23:14<3:07:53,  1.79s/it]
 48%|████▊     | 5706/12000 [3:23:16<3:18:47,  1.90s/it]
                                                        

 48%|████▊     | 5706/12000 [3:23:16<3:18:47,  1.90s/it]
 48%|████▊     | 5707/12000 [3:23:19<3:50:28,  2.20s/it]
                                                        

 48%|████▊     | 5707/12000 [3:23:19<3:50:28,  2.20s/it]
 48%|████▊     | 5708/12000 [3:23:22<3:52:47,  2.22s/it]
                                                        

 48%|████▊     | 5708/12000 [3:23:22<3:52:47,  2.22s/it]
 48%|████▊     | 5709/12000 [3:23:24<3:56:02,  2.25s/it]
                                                        

 48%|████▊     | 5709/12000 [3:23:24<3:56:02,  2.25s/it]
 48%|████▊     | 5710/12000 [3:23:26<3:36:26,  2.06s/it]
                                                        

 48%|████▊     | 5710/12000 [3:23:26<3:36:26,  2.06s/it]
 48%|████▊     | 5711/12000 [3:23:27<3:25:49,  1.96s/it]
                                                        

 48%|████▊     | 5711/12000 [3:23:27<3:25:49,  1.96s/it]
 48%|████▊     | 5712/12000 [3:23:31<4:33:13,  2.61s/it]
                                                        

 48%|████▊     | 5712/12000 [3:23:31<4:33:13,  2.61s/it]
 48%|████▊     | 5713/12000 [3:23:33<3:53:47,  2.23s/it]
                                                        

 48%|████▊     | 5713/12000 [3:23:33<3:53:47,  2.23s/it]
 48%|████▊     | 5714/12000 [3:23:35<3:44:18,  2.14s/it]
                                                        

 48%|████▊     | 5714/12000 [3:23:35<3:44:18,  2.14s/it]
 48%|████▊     | 5715/12000 [3:23:38<4:09:22,  2.38s/it]
                                                        

 48%|████▊     | 5715/12000 [3:23:38<4:09:22,  2.38s/it]
 48%|████▊     | 5716/12000 [3:23:41<4:28:38,  2.57s/it]
                                                        

 48%|████▊     | 5716/12000 [3:23:41<4:28:38,  2.57s/it]
 48%|████▊     | 5717/12000 [3:23:43<4:17:11,  2.46s/it]
                                                        

 48%|████▊     | 5717/12000 [3:23:43<4:17:11,  2.46s/it]
 48%|████▊     | 5718/12000 [3:23:45<3:58:04,  2.27s/it]
                                                        

 48%|████▊     | 5718/12000 [3:23:45<3:58:04,  2.27s/it]
 48%|████▊     | 5719/12000 [3:23:47<4:05:41,  2.35s/it]
                                                        

 48%|████▊     | 5719/12000 [3:23:47<4:05:41,  2.35s/it]
 48%|████▊     | 5720/12000 [3:23:52<5:15:08,  3.01s/it]
                                                        

 48%|████▊     | 5720/12000 [3:23:52<5:15:08,  3.01s/it]
 48%|████▊     | 5721/12000 [3:23:53<4:27:40,  2.56s/it]
                                                        

 48%|████▊     | 5721/12000 [3:23:53<4:27:40,  2.56s/it]
 48%|████▊     | 5722/12000 [3:23:56<4:35:48,  2.64s/it]
                                                        

 48%|████▊     | 5722/12000 [3:23:56<4:35:48,  2.64s/it]
 48%|████▊     | 5723/12000 [3:23:58<4:02:50,  2.32s/it]
                                                        

 48%|████▊     | 5723/12000 [3:23:58<4:02:50,  2.32s/it]
 48%|████▊     | 5724/12000 [3:23:59<3:39:25,  2.10s/it]
                                                        

 48%|████▊     | 5724/12000 [3:23:59<3:39:25,  2.10s/it]
 48%|████▊     | 5725/12000 [3:24:01<3:22:56,  1.94s/it]
                                                        

 48%|████▊     | 5725/12000 [3:24:01<3:22:56,  1.94s/it]
 48%|████▊     | 5726/12000 [3:24:02<3:16:16,  1.88s/it]
                                                        

 48%|████▊     | 5726/12000 [3:24:02<3:16:16,  1.88s/it]
 48%|████▊     | 5727/12000 [3:24:04<3:02:14,  1.74s/it]
                                                        

 48%|████▊     | 5727/12000 [3:24:04<3:02:14,  1.74s/it]
 48%|████▊     | 5728/12000 [3:24:05<2:53:51,  1.66s/it]
                                                        

 48%|████▊     | 5728/12000 [3:24:05<2:53:51,  1.66s/it]
 48%|████▊     | 5729/12000 [3:24:07<3:05:50,  1.78s/it]
                                                        

 48%|████▊     | 5729/12000 [3:24:07<3:05:50,  1.78s/it]
 48%|████▊     | 5730/12000 [3:24:09<2:59:48,  1.72s/it]
                                                        

 48%|████▊     | 5730/12000 [3:24:09<2:59:48,  1.72s/it]
 48%|████▊     | 5731/12000 [3:24:11<2:54:31,  1.67s/it]
                                                        

 48%|████▊     | 5731/12000 [3:24:11<2:54:31,  1.67s/it]
 48%|████▊     | 5732/12000 [3:24:13<3:03:47,  1.76s/it]
                                                        

 48%|████▊     | 5732/12000 [3:24:13<3:03:47,  1.76s/it]
 48%|████▊     | 5733/12000 [3:24:14<2:59:41,  1.72s/it]
                                                        

 48%|████▊     | 5733/12000 [3:24:14<2:59:41,  1.72s/it]
 48%|████▊     | 5734/12000 [3:24:18<4:01:11,  2.31s/it]
                                                        

 48%|████▊     | 5734/12000 [3:24:18<4:01:11,  2.31s/it]
 48%|████▊     | 5735/12000 [3:24:19<3:31:45,  2.03s/it]
                                                        

 48%|████▊     | 5735/12000 [3:24:19<3:31:45,  2.03s/it]
 48%|████▊     | 5736/12000 [3:24:21<3:14:20,  1.86s/it]
                                                        

 48%|████▊     | 5736/12000 [3:24:21<3:14:20,  1.86s/it]
 48%|████▊     | 5737/12000 [3:24:22<3:07:25,  1.80s/it]
                                                        

 48%|████▊     | 5737/12000 [3:24:22<3:07:25,  1.80s/it]
 48%|████▊     | 5738/12000 [3:24:25<3:35:55,  2.07s/it]
                                                        
{'loss': '1.037', 'grad_norm': '0.4722', 'learning_rate': '0.0001167', 'entropy': '1.109', 'num_tokens': '3.077e+07', 'mean_token_accuracy': '0.7224', 'epoch': '0.05036'}
{'loss': '0.7855', 'grad_norm': '0.3966', 'learning_rate': '0.0001167', 'entropy': '0.8457', 'num_tokens': '3.077e+07', 'mean_token_accuracy': '0.7858', 'epoch': '0.05037'}
{'loss': '0.9612', 'grad_norm': '0.3758', 'learning_rate': '0.0001167', 'entropy': '1.069', 'num_tokens': '3.078e+07', 'mean_token_accuracy': '0.7284', 'epoch': '0.05038'}
{'loss': '0.6941', 'grad_norm': '0.4705', 'learning_rate': '0.0001166', 'entropy': '0.7124', 'num_tokens': '3.078e+07', 'mean_token_accuracy': '0.8257', 'epoch': '0.05039'}
{'loss': '0.8812', 'grad_norm': '0.3704', 'learning_rate': '0.0001166', 'entropy': '0.8856', 'num_tokens': '3.079e+07', 'mean_token_accuracy': '0.7731', 'epoch': '0.0504'}
{'loss': '0.6401', 'grad_norm': '0.4136', 'learning_rate': '0.0001166', 'entropy': '0.6287', 'num_tokens': '3.079e+07', 'mean_token_accuracy': '0.8342', 'epoch': '0.05041'}
{'loss': '0.9193', 'grad_norm': '0.3949', 'learning_rate': '0.0001166', 'entropy': '0.9306', 'num_tokens': '3.08e+07', 'mean_token_accuracy': '0.7702', 'epoch': '0.05042'}
{'loss': '1.237', 'grad_norm': '0.4912', 'learning_rate': '0.0001165', 'entropy': '1.234', 'num_tokens': '3.08e+07', 'mean_token_accuracy': '0.7053', 'epoch': '0.05043'}
{'loss': '1.12', 'grad_norm': '0.4482', 'learning_rate': '0.0001165', 'entropy': '0.9918', 'num_tokens': '3.08e+07', 'mean_token_accuracy': '0.7719', 'epoch': '0.05044'}
{'loss': '0.8828', 'grad_norm': '0.3909', 'learning_rate': '0.0001165', 'entropy': '0.7993', 'num_tokens': '3.081e+07', 'mean_token_accuracy': '0.8066', 'epoch': '0.05044'}
{'loss': '0.8519', 'grad_norm': '0.4207', 'learning_rate': '0.0001165', 'entropy': '0.8653', 'num_tokens': '3.081e+07', 'mean_token_accuracy': '0.7856', 'epoch': '0.05045'}
{'loss': '0.5452', 'grad_norm': '0.317', 'learning_rate': '0.0001164', 'entropy': '0.5457', 'num_tokens': '3.082e+07', 'mean_token_accuracy': '0.8519', 'epoch': '0.05046'}
{'loss': '0.6885', 'grad_norm': '0.403', 'learning_rate': '0.0001164', 'entropy': '0.7058', 'num_tokens': '3.082e+07', 'mean_token_accuracy': '0.8212', 'epoch': '0.05047'}
{'loss': '1.122', 'grad_norm': '0.3501', 'learning_rate': '0.0001164', 'entropy': '0.9553', 'num_tokens': '3.083e+07', 'mean_token_accuracy': '0.7708', 'epoch': '0.05048'}
{'loss': '1.064', 'grad_norm': '0.4027', 'learning_rate': '0.0001164', 'entropy': '0.9988', 'num_tokens': '3.083e+07', 'mean_token_accuracy': '0.764', 'epoch': '0.05049'}
{'loss': '0.828', 'grad_norm': '0.3483', 'learning_rate': '0.0001163', 'entropy': '0.8295', 'num_tokens': '3.084e+07', 'mean_token_accuracy': '0.7886', 'epoch': '0.0505'}
{'loss': '1.158', 'grad_norm': '0.4883', 'learning_rate': '0.0001163', 'entropy': '1.082', 'num_tokens': '3.084e+07', 'mean_token_accuracy': '0.7326', 'epoch': '0.05051'}
{'loss': '0.8105', 'grad_norm': '0.4347', 'learning_rate': '0.0001163', 'entropy': '0.8838', 'num_tokens': '3.085e+07', 'mean_token_accuracy': '0.7853', 'epoch': '0.05051'}
{'loss': '0.9186', 'grad_norm': '0.4843', 'learning_rate': '0.0001162', 'entropy': '0.9083', 'num_tokens': '3.086e+07', 'mean_token_accuracy': '0.7757', 'epoch': '0.05052'}
{'loss': '1.016', 'grad_norm': '0.4746', 'learning_rate': '0.0001162', 'entropy': '0.9638', 'num_tokens': '3.086e+07', 'mean_token_accuracy': '0.7608', 'epoch': '0.05053'}
{'loss': '0.9619', 'grad_norm': '0.4761', 'learning_rate': '0.0001162', 'entropy': '1.009', 'num_tokens': '3.087e+07', 'mean_token_accuracy': '0.7524', 'epoch': '0.05054'}
{'loss': '0.8243', 'grad_norm': '0.3877', 'learning_rate': '0.0001162', 'entropy': '0.771', 'num_tokens': '3.087e+07', 'mean_token_accuracy': '0.7973', 'epoch': '0.05055'}
{'loss': '0.902', 'grad_norm': '0.3946', 'learning_rate': '0.0001161', 'entropy': '0.9946', 'num_tokens': '3.088e+07', 'mean_token_accuracy': '0.7654', 'epoch': '0.05056'}
{'loss': '0.9666', 'grad_norm': '0.4379', 'learning_rate': '0.0001161', 'entropy': '0.9802', 'num_tokens': '3.088e+07', 'mean_token_accuracy': '0.7556', 'epoch': '0.05057'}
{'loss': '0.764', 'grad_norm': '0.3937', 'learning_rate': '0.0001161', 'entropy': '0.8184', 'num_tokens': '3.089e+07', 'mean_token_accuracy': '0.7803', 'epoch': '0.05058'}
{'loss': '0.8497', 'grad_norm': '0.3161', 'learning_rate': '0.0001161', 'entropy': '0.8459', 'num_tokens': '3.09e+07', 'mean_token_accuracy': '0.785', 'epoch': '0.05059'}
{'loss': '1.163', 'grad_norm': '0.4256', 'learning_rate': '0.000116', 'entropy': '0.9648', 'num_tokens': '3.09e+07', 'mean_token_accuracy': '0.7566', 'epoch': '0.05059'}
{'loss': '1.073', 'grad_norm': '0.3999', 'learning_rate': '0.000116', 'entropy': '1.039', 'num_tokens': '3.091e+07', 'mean_token_accuracy': '0.7512', 'epoch': '0.0506'}
{'loss': '1.194', 'grad_norm': '0.3816', 'learning_rate': '0.000116', 'entropy': '1.161', 'num_tokens': '3.092e+07', 'mean_token_accuracy': '0.7064', 'epoch': '0.05061'}
{'loss': '0.7484', 'grad_norm': '0.4481', 'learning_rate': '0.0001159', 'entropy': '0.8446', 'num_tokens': '3.092e+07', 'mean_token_accuracy': '0.782', 'epoch': '0.05062'}
{'loss': '0.6063', 'grad_norm': '0.3397', 'learning_rate': '0.0001159', 'entropy': '0.6968', 'num_tokens': '3.093e+07', 'mean_token_accuracy': '0.8325', 'epoch': '0.05063'}
{'loss': '0.6759', 'grad_norm': '0.3782', 'learning_rate': '0.0001159', 'entropy': '0.6877', 'num_tokens': '3.093e+07', 'mean_token_accuracy': '0.8251', 'epoch': '0.05064'}
{'loss': '1.116', 'grad_norm': '0.3695', 'learning_rate': '0.0001159', 'entropy': '1.107', 'num_tokens': '3.094e+07', 'mean_token_accuracy': '0.7295', 'epoch': '0.05065'}
{'loss': '0.9951', 'grad_norm': '0.4769', 'learning_rate': '0.0001158', 'entropy': '1.01', 'num_tokens': '3.094e+07', 'mean_token_accuracy': '0.7536', 'epoch': '0.05066'}
{'loss': '1.105', 'grad_norm': '0.5456', 'learning_rate': '0.0001158', 'entropy': '1.097', 'num_tokens': '3.095e+07', 'mean_token_accuracy': '0.7292', 'epoch': '0.05067'}
{'loss': '0.8844', 'grad_norm': '0.3851', 'learning_rate': '0.0001158', 'entropy': '0.8803', 'num_tokens': '3.095e+07', 'mean_token_accuracy': '0.7769', 'epoch': '0.05067'}
{'loss': '1.449', 'grad_norm': '0.5178', 'learning_rate': '0.0001158', 'entropy': '1.442', 'num_tokens': '3.096e+07', 'mean_token_accuracy': '0.6743', 'epoch': '0.05068'}
{'loss': '0.861', 'grad_norm': '0.3727', 'learning_rate': '0.0001157', 'entropy': '0.946', 'num_tokens': '3.096e+07', 'mean_token_accuracy': '0.7627', 'epoch': '0.05069'}
{'loss': '0.8216', 'grad_norm': '0.4071', 'learning_rate': '0.0001157', 'entropy': '0.7768', 'num_tokens': '3.097e+07', 'mean_token_accuracy': '0.7943', 'epoch': '0.0507'}
{'loss': '1.089', 'grad_norm': '0.4035', 'learning_rate': '0.0001157', 'entropy': '1.057', 'num_tokens': '3.097e+07', 'mean_token_accuracy': '0.742', 'epoch': '0.05071'}
{'loss': '0.7848', 'grad_norm': '0.3499', 'learning_rate': '0.0001156', 'entropy': '0.838', 'num_tokens': '3.098e+07', 'mean_token_accuracy': '0.7899', 'epoch': '0.05072'}
{'loss': '1.343', 'grad_norm': '0.4965', 'learning_rate': '0.0001156', 'entropy': '1.202', 'num_tokens': '3.098e+07', 'mean_token_accuracy': '0.7305', 'epoch': '0.05073'}
{'loss': '0.7692', 'grad_norm': '0.3887', 'learning_rate': '0.0001156', 'entropy': '0.7216', 'num_tokens': '3.099e+07', 'mean_token_accuracy': '0.8109', 'epoch': '0.05074'}
{'loss': '0.6199', 'grad_norm': '0.3298', 'learning_rate': '0.0001156', 'entropy': '0.6426', 'num_tokens': '3.099e+07', 'mean_token_accuracy': '0.8284', 'epoch': '0.05074'}
{'loss': '0.819', 'grad_norm': '0.4282', 'learning_rate': '0.0001155', 'entropy': '0.9167', 'num_tokens': '3.099e+07', 'mean_token_accuracy': '0.775', 'epoch': '0.05075'}
{'loss': '0.8065', 'grad_norm': '0.3826', 'learning_rate': '0.0001155', 'entropy': '0.8213', 'num_tokens': '3.1e+07', 'mean_token_accuracy': '0.7915', 'epoch': '0.05076'}
{'loss': '0.7075', 'grad_norm': '0.4131', 'learning_rate': '0.0001155', 'entropy': '0.7023', 'num_tokens': '3.1e+07', 'mean_token_accuracy': '0.8114', 'epoch': '0.05077'}

 48%|████▊     | 5738/12000 [3:24:25<3:35:55,  2.07s/it]
 48%|████▊     | 5739/12000 [3:24:27<3:37:32,  2.08s/it]
                                                        

 48%|████▊     | 5739/12000 [3:24:27<3:37:32,  2.08s/it]
 48%|████▊     | 5740/12000 [3:24:29<3:22:55,  1.95s/it]
                                                        

 48%|████▊     | 5740/12000 [3:24:29<3:22:55,  1.95s/it]
 48%|████▊     | 5741/12000 [3:24:30<3:13:09,  1.85s/it]
                                                        

 48%|████▊     | 5741/12000 [3:24:30<3:13:09,  1.85s/it]
 48%|████▊     | 5742/12000 [3:24:36<5:03:44,  2.91s/it]
                                                        

 48%|████▊     | 5742/12000 [3:24:36<5:03:44,  2.91s/it]
 48%|████▊     | 5743/12000 [3:24:37<4:21:40,  2.51s/it]
                                                        

 48%|████▊     | 5743/12000 [3:24:37<4:21:40,  2.51s/it]
 48%|████▊     | 5744/12000 [3:24:39<3:54:44,  2.25s/it]
                                                        

 48%|████▊     | 5744/12000 [3:24:39<3:54:44,  2.25s/it]
 48%|████▊     | 5745/12000 [3:24:42<4:17:54,  2.47s/it]
                                                        

 48%|████▊     | 5745/12000 [3:24:42<4:17:54,  2.47s/it]
 48%|████▊     | 5746/12000 [3:24:44<3:48:59,  2.20s/it]
                                                        

 48%|████▊     | 5746/12000 [3:24:44<3:48:59,  2.20s/it]
 48%|████▊     | 5747/12000 [3:24:48<5:14:17,  3.02s/it]
                                                        

 48%|████▊     | 5747/12000 [3:24:48<5:14:17,  3.02s/it]
 48%|████▊     | 5748/12000 [3:24:50<4:42:54,  2.71s/it]
                                                        

 48%|████▊     | 5748/12000 [3:24:50<4:42:54,  2.71s/it]
 48%|████▊     | 5749/12000 [3:24:52<4:10:08,  2.40s/it]
                                                        

 48%|████▊     | 5749/12000 [3:24:52<4:10:08,  2.40s/it]
 48%|████▊     | 5750/12000 [3:24:54<3:46:00,  2.17s/it]
                                                        

 48%|████▊     | 5750/12000 [3:24:54<3:46:00,  2.17s/it]
 48%|████▊     | 5751/12000 [3:24:57<4:15:20,  2.45s/it]
                                                        

 48%|████▊     | 5751/12000 [3:24:57<4:15:20,  2.45s/it]
 48%|████▊     | 5752/12000 [3:25:00<4:28:34,  2.58s/it]
                                                        

 48%|████▊     | 5752/12000 [3:25:00<4:28:34,  2.58s/it]
 48%|████▊     | 5753/12000 [3:25:01<4:01:18,  2.32s/it]
                                                        

 48%|████▊     | 5753/12000 [3:25:01<4:01:18,  2.32s/it]
 48%|████▊     | 5754/12000 [3:25:04<4:00:13,  2.31s/it]
                                                        

 48%|████▊     | 5754/12000 [3:25:04<4:00:13,  2.31s/it]
 48%|████▊     | 5755/12000 [3:25:06<3:42:44,  2.14s/it]
                                                        

 48%|████▊     | 5755/12000 [3:25:06<3:42:44,  2.14s/it]
 48%|████▊     | 5756/12000 [3:25:09<4:19:04,  2.49s/it]
                                                        

 48%|████▊     | 5756/12000 [3:25:09<4:19:04,  2.49s/it]
 48%|████▊     | 5757/12000 [3:25:10<3:48:02,  2.19s/it]
                                                        

 48%|████▊     | 5757/12000 [3:25:10<3:48:02,  2.19s/it]
 48%|████▊     | 5758/12000 [3:25:12<3:24:50,  1.97s/it]
                                                        

 48%|████▊     | 5758/12000 [3:25:12<3:24:50,  1.97s/it]
 48%|████▊     | 5759/12000 [3:25:13<3:12:04,  1.85s/it]
                                                        

 48%|████▊     | 5759/12000 [3:25:13<3:12:04,  1.85s/it]
 48%|████▊     | 5760/12000 [3:25:15<3:08:27,  1.81s/it]
                                                        

 48%|████▊     | 5760/12000 [3:25:15<3:08:27,  1.81s/it]
 48%|████▊     | 5761/12000 [3:25:18<3:28:40,  2.01s/it]
                                                        

 48%|████▊     | 5761/12000 [3:25:18<3:28:40,  2.01s/it]
 48%|████▊     | 5762/12000 [3:25:19<3:10:46,  1.84s/it]
                                                        

 48%|████▊     | 5762/12000 [3:25:19<3:10:46,  1.84s/it]
 48%|████▊     | 5763/12000 [3:25:21<3:04:56,  1.78s/it]
                                                        

 48%|████▊     | 5763/12000 [3:25:21<3:04:56,  1.78s/it]
 48%|████▊     | 5764/12000 [3:25:22<3:02:35,  1.76s/it]
                                                        

 48%|████▊     | 5764/12000 [3:25:22<3:02:35,  1.76s/it]
 48%|████▊     | 5765/12000 [3:25:24<2:59:47,  1.73s/it]
                                                        

 48%|████▊     | 5765/12000 [3:25:24<2:59:47,  1.73s/it]
 48%|████▊     | 5766/12000 [3:25:26<3:11:47,  1.85s/it]
                                                        

 48%|████▊     | 5766/12000 [3:25:26<3:11:47,  1.85s/it]
 48%|████▊     | 5767/12000 [3:25:29<3:37:13,  2.09s/it]
                                                        

 48%|████▊     | 5767/12000 [3:25:29<3:37:13,  2.09s/it]
 48%|████▊     | 5768/12000 [3:25:30<3:25:30,  1.98s/it]
                                                        

 48%|████▊     | 5768/12000 [3:25:30<3:25:30,  1.98s/it]
 48%|████▊     | 5769/12000 [3:25:32<3:10:24,  1.83s/it]
                                                        

 48%|████▊     | 5769/12000 [3:25:32<3:10:24,  1.83s/it]
 48%|████▊     | 5770/12000 [3:25:34<3:02:45,  1.76s/it]
                                                        

 48%|████▊     | 5770/12000 [3:25:34<3:02:45,  1.76s/it]
 48%|████▊     | 5771/12000 [3:25:36<3:16:20,  1.89s/it]
                                                        

 48%|████▊     | 5771/12000 [3:25:36<3:16:20,  1.89s/it]
 48%|████▊     | 5772/12000 [3:25:37<3:04:07,  1.77s/it]
                                                        

 48%|████▊     | 5772/12000 [3:25:37<3:04:07,  1.77s/it]
 48%|████▊     | 5773/12000 [3:25:39<3:01:31,  1.75s/it]
                                                        

 48%|████▊     | 5773/12000 [3:25:39<3:01:31,  1.75s/it]
 48%|████▊     | 5774/12000 [3:25:41<3:06:18,  1.80s/it]
                                                        

 48%|████▊     | 5774/12000 [3:25:41<3:06:18,  1.80s/it]
 48%|████▊     | 5775/12000 [3:25:42<3:01:33,  1.75s/it]
                                                        

 48%|████▊     | 5775/12000 [3:25:43<3:01:33,  1.75s/it]
 48%|████▊     | 5776/12000 [3:25:44<2:53:59,  1.68s/it]
                                                        

 48%|████▊     | 5776/12000 [3:25:44<2:53:59,  1.68s/it]
 48%|████▊     | 5777/12000 [3:25:46<2:52:42,  1.67s/it]
                                                        

 48%|████▊     | 5777/12000 [3:25:46<2:52:42,  1.67s/it]
 48%|████▊     | 5778/12000 [3:25:49<3:52:57,  2.25s/it]
                                                        

 48%|████▊     | 5778/12000 [3:25:49<3:52:57,  2.25s/it]
 48%|████▊     | 5779/12000 [3:25:51<3:43:48,  2.16s/it]
                                                        

 48%|████▊     | 5779/12000 [3:25:51<3:43:48,  2.16s/it]
 48%|████▊     | 5780/12000 [3:25:53<3:29:41,  2.02s/it]
                                                        

 48%|████▊     | 5780/12000 [3:25:53<3:29:41,  2.02s/it]
 48%|████▊     | 5781/12000 [3:25:55<3:17:36,  1.91s/it]
                                                        

 48%|████▊     | 5781/12000 [3:25:55<3:17:36,  1.91s/it]
 48%|████▊     | 5782/12000 [3:25:56<3:15:53,  1.89s/it]
                                                        

 48%|████▊     | 5782/12000 [3:25:56<3:15:53,  1.89s/it]
 48%|████▊     | 5783/12000 [3:25:58<3:02:21,  1.76s/it]
                                                        

 48%|████▊     | 5783/12000 [3:25:58<3:02:21,  1.76s/it]
 48%|████▊     | 5784/12000 [3:25:59<2:51:25,  1.65s/it]
                                                        

 48%|████▊     | 5784/12000 [3:25:59<2:51:25,  1.65s/it]
 48%|████▊     | 5785/12000 [3:26:01<3:00:43,  1.74s/it]
                                                        
{'loss': '0.9229', 'grad_norm': '0.2682', 'learning_rate': '0.0001155', 'entropy': '0.7998', 'num_tokens': '3.101e+07', 'mean_token_accuracy': '0.8005', 'epoch': '0.05078'}
{'loss': '0.6893', 'grad_norm': '0.3697', 'learning_rate': '0.0001154', 'entropy': '0.6926', 'num_tokens': '3.102e+07', 'mean_token_accuracy': '0.8289', 'epoch': '0.05079'}
{'loss': '0.6911', 'grad_norm': '0.3552', 'learning_rate': '0.0001154', 'entropy': '0.6734', 'num_tokens': '3.102e+07', 'mean_token_accuracy': '0.829', 'epoch': '0.0508'}
{'loss': '1.04', 'grad_norm': '0.4156', 'learning_rate': '0.0001154', 'entropy': '1.001', 'num_tokens': '3.103e+07', 'mean_token_accuracy': '0.7478', 'epoch': '0.05081'}
{'loss': '1.55', 'grad_norm': '0.3643', 'learning_rate': '0.0001153', 'entropy': '1.414', 'num_tokens': '3.104e+07', 'mean_token_accuracy': '0.6764', 'epoch': '0.05082'}
{'loss': '0.6453', 'grad_norm': '0.3217', 'learning_rate': '0.0001153', 'entropy': '0.6689', 'num_tokens': '3.104e+07', 'mean_token_accuracy': '0.8322', 'epoch': '0.05082'}
{'loss': '0.7508', 'grad_norm': '0.3148', 'learning_rate': '0.0001153', 'entropy': '0.7505', 'num_tokens': '3.105e+07', 'mean_token_accuracy': '0.8115', 'epoch': '0.05083'}
{'loss': '0.7668', 'grad_norm': '0.3178', 'learning_rate': '0.0001153', 'entropy': '0.791', 'num_tokens': '3.106e+07', 'mean_token_accuracy': '0.7997', 'epoch': '0.05084'}
{'loss': '1.026', 'grad_norm': '0.4019', 'learning_rate': '0.0001152', 'entropy': '0.9362', 'num_tokens': '3.106e+07', 'mean_token_accuracy': '0.775', 'epoch': '0.05085'}
{'loss': '1.141', 'grad_norm': '0.3583', 'learning_rate': '0.0001152', 'entropy': '0.9659', 'num_tokens': '3.107e+07', 'mean_token_accuracy': '0.7578', 'epoch': '0.05086'}
{'loss': '0.7656', 'grad_norm': '0.3622', 'learning_rate': '0.0001152', 'entropy': '0.8464', 'num_tokens': '3.108e+07', 'mean_token_accuracy': '0.7827', 'epoch': '0.05087'}
{'loss': '1.126', 'grad_norm': '0.4737', 'learning_rate': '0.0001152', 'entropy': '1.096', 'num_tokens': '3.108e+07', 'mean_token_accuracy': '0.7419', 'epoch': '0.05088'}
{'loss': '0.6298', 'grad_norm': '0.3418', 'learning_rate': '0.0001151', 'entropy': '0.653', 'num_tokens': '3.109e+07', 'mean_token_accuracy': '0.82', 'epoch': '0.05089'}
{'loss': '0.859', 'grad_norm': '0.4201', 'learning_rate': '0.0001151', 'entropy': '0.9293', 'num_tokens': '3.109e+07', 'mean_token_accuracy': '0.7658', 'epoch': '0.0509'}
{'loss': '0.494', 'grad_norm': '0.276', 'learning_rate': '0.0001151', 'entropy': '0.5887', 'num_tokens': '3.11e+07', 'mean_token_accuracy': '0.8515', 'epoch': '0.0509'}
{'loss': '0.9216', 'grad_norm': '0.3729', 'learning_rate': '0.000115', 'entropy': '0.9507', 'num_tokens': '3.11e+07', 'mean_token_accuracy': '0.7605', 'epoch': '0.05091'}
{'loss': '0.7467', 'grad_norm': '0.3623', 'learning_rate': '0.000115', 'entropy': '0.8226', 'num_tokens': '3.111e+07', 'mean_token_accuracy': '0.8049', 'epoch': '0.05092'}
{'loss': '1.024', 'grad_norm': '0.4501', 'learning_rate': '0.000115', 'entropy': '0.9535', 'num_tokens': '3.111e+07', 'mean_token_accuracy': '0.7754', 'epoch': '0.05093'}
{'loss': '0.7966', 'grad_norm': '0.4085', 'learning_rate': '0.000115', 'entropy': '0.8545', 'num_tokens': '3.112e+07', 'mean_token_accuracy': '0.7886', 'epoch': '0.05094'}
{'loss': '0.8028', 'grad_norm': '0.4878', 'learning_rate': '0.0001149', 'entropy': '0.7474', 'num_tokens': '3.112e+07', 'mean_token_accuracy': '0.8016', 'epoch': '0.05095'}
{'loss': '0.6252', 'grad_norm': '0.3504', 'learning_rate': '0.0001149', 'entropy': '0.6434', 'num_tokens': '3.112e+07', 'mean_token_accuracy': '0.8327', 'epoch': '0.05096'}
{'loss': '0.563', 'grad_norm': '0.3158', 'learning_rate': '0.0001149', 'entropy': '0.6414', 'num_tokens': '3.113e+07', 'mean_token_accuracy': '0.8327', 'epoch': '0.05097'}
{'loss': '0.9967', 'grad_norm': '0.5991', 'learning_rate': '0.0001149', 'entropy': '0.9967', 'num_tokens': '3.113e+07', 'mean_token_accuracy': '0.7498', 'epoch': '0.05098'}
{'loss': '0.7387', 'grad_norm': '0.381', 'learning_rate': '0.0001148', 'entropy': '0.7126', 'num_tokens': '3.114e+07', 'mean_token_accuracy': '0.8123', 'epoch': '0.05098'}
{'loss': '0.8113', 'grad_norm': '0.4124', 'learning_rate': '0.0001148', 'entropy': '0.7684', 'num_tokens': '3.114e+07', 'mean_token_accuracy': '0.7998', 'epoch': '0.05099'}
{'loss': '0.7234', 'grad_norm': '0.4208', 'learning_rate': '0.0001148', 'entropy': '0.7903', 'num_tokens': '3.115e+07', 'mean_token_accuracy': '0.7993', 'epoch': '0.051'}
{'loss': '0.9384', 'grad_norm': '0.4824', 'learning_rate': '0.0001147', 'entropy': '0.9435', 'num_tokens': '3.115e+07', 'mean_token_accuracy': '0.7541', 'epoch': '0.05101'}
{'loss': '0.8961', 'grad_norm': '0.4466', 'learning_rate': '0.0001147', 'entropy': '0.9658', 'num_tokens': '3.116e+07', 'mean_token_accuracy': '0.7701', 'epoch': '0.05102'}
{'loss': '1.006', 'grad_norm': '0.4146', 'learning_rate': '0.0001147', 'entropy': '1.116', 'num_tokens': '3.117e+07', 'mean_token_accuracy': '0.7258', 'epoch': '0.05103'}
{'loss': '0.9383', 'grad_norm': '0.3182', 'learning_rate': '0.0001147', 'entropy': '0.8625', 'num_tokens': '3.117e+07', 'mean_token_accuracy': '0.7786', 'epoch': '0.05104'}
{'loss': '1.027', 'grad_norm': '0.3775', 'learning_rate': '0.0001146', 'entropy': '0.9941', 'num_tokens': '3.118e+07', 'mean_token_accuracy': '0.7592', 'epoch': '0.05105'}
{'loss': '0.7667', 'grad_norm': '0.3848', 'learning_rate': '0.0001146', 'entropy': '0.7775', 'num_tokens': '3.118e+07', 'mean_token_accuracy': '0.794', 'epoch': '0.05105'}
{'loss': '0.974', 'grad_norm': '0.3798', 'learning_rate': '0.0001146', 'entropy': '1.013', 'num_tokens': '3.119e+07', 'mean_token_accuracy': '0.7584', 'epoch': '0.05106'}
{'loss': '0.6922', 'grad_norm': '0.3775', 'learning_rate': '0.0001146', 'entropy': '0.7774', 'num_tokens': '3.119e+07', 'mean_token_accuracy': '0.7942', 'epoch': '0.05107'}
{'loss': '0.7725', 'grad_norm': '0.4775', 'learning_rate': '0.0001145', 'entropy': '0.7868', 'num_tokens': '3.12e+07', 'mean_token_accuracy': '0.8008', 'epoch': '0.05108'}
{'loss': '0.6669', 'grad_norm': '0.3642', 'learning_rate': '0.0001145', 'entropy': '0.6926', 'num_tokens': '3.12e+07', 'mean_token_accuracy': '0.8227', 'epoch': '0.05109'}
{'loss': '0.6372', 'grad_norm': '0.3833', 'learning_rate': '0.0001145', 'entropy': '0.6381', 'num_tokens': '3.121e+07', 'mean_token_accuracy': '0.8302', 'epoch': '0.0511'}
{'loss': '0.6115', 'grad_norm': '0.4073', 'learning_rate': '0.0001144', 'entropy': '0.6589', 'num_tokens': '3.121e+07', 'mean_token_accuracy': '0.8284', 'epoch': '0.05111'}
{'loss': '0.8826', 'grad_norm': '0.3887', 'learning_rate': '0.0001144', 'entropy': '0.8827', 'num_tokens': '3.122e+07', 'mean_token_accuracy': '0.7766', 'epoch': '0.05112'}
{'loss': '1.245', 'grad_norm': '0.4819', 'learning_rate': '0.0001144', 'entropy': '1.082', 'num_tokens': '3.122e+07', 'mean_token_accuracy': '0.7387', 'epoch': '0.05113'}
{'loss': '0.9717', 'grad_norm': '0.4741', 'learning_rate': '0.0001144', 'entropy': '0.9762', 'num_tokens': '3.123e+07', 'mean_token_accuracy': '0.7482', 'epoch': '0.05113'}
{'loss': '1.06', 'grad_norm': '0.3976', 'learning_rate': '0.0001143', 'entropy': '1.049', 'num_tokens': '3.123e+07', 'mean_token_accuracy': '0.7537', 'epoch': '0.05114'}
{'loss': '1.05', 'grad_norm': '0.6396', 'learning_rate': '0.0001143', 'entropy': '1.07', 'num_tokens': '3.124e+07', 'mean_token_accuracy': '0.7502', 'epoch': '0.05115'}
{'loss': '0.9739', 'grad_norm': '0.4249', 'learning_rate': '0.0001143', 'entropy': '0.9794', 'num_tokens': '3.124e+07', 'mean_token_accuracy': '0.7494', 'epoch': '0.05116'}
{'loss': '1.004', 'grad_norm': '0.3611', 'learning_rate': '0.0001143', 'entropy': '1.061', 'num_tokens': '3.125e+07', 'mean_token_accuracy': '0.7547', 'epoch': '0.05117'}
{'loss': '0.8069', 'grad_norm': '0.3561', 'learning_rate': '0.0001142', 'entropy': '0.8101', 'num_tokens': '3.125e+07', 'mean_token_accuracy': '0.8001', 'epoch': '0.05118'}
{'loss': '0.6059', 'grad_norm': '0.3429', 'learning_rate': '0.0001142', 'entropy': '0.6957', 'num_tokens': '3.126e+07', 'mean_token_accuracy': '0.8249', 'epoch': '0.05119'}

 48%|████▊     | 5785/12000 [3:26:01<3:00:43,  1.74s/it]
 48%|████▊     | 5786/12000 [3:26:03<3:13:09,  1.87s/it]
                                                        

 48%|████▊     | 5786/12000 [3:26:03<3:13:09,  1.87s/it]
 48%|████▊     | 5787/12000 [3:26:06<3:52:17,  2.24s/it]
                                                        

 48%|████▊     | 5787/12000 [3:26:06<3:52:17,  2.24s/it]
 48%|████▊     | 5788/12000 [3:26:08<3:30:52,  2.04s/it]
                                                        

 48%|████▊     | 5788/12000 [3:26:08<3:30:52,  2.04s/it]
 48%|████▊     | 5789/12000 [3:26:10<3:18:32,  1.92s/it]
                                                        

 48%|████▊     | 5789/12000 [3:26:10<3:18:32,  1.92s/it]
 48%|████▊     | 5790/12000 [3:26:12<3:23:59,  1.97s/it]
                                                        

 48%|████▊     | 5790/12000 [3:26:12<3:23:59,  1.97s/it]
 48%|████▊     | 5791/12000 [3:26:14<3:45:12,  2.18s/it]
                                                        

 48%|████▊     | 5791/12000 [3:26:14<3:45:12,  2.18s/it]
 48%|████▊     | 5792/12000 [3:26:16<3:29:54,  2.03s/it]
                                                        

 48%|████▊     | 5792/12000 [3:26:16<3:29:54,  2.03s/it]
 48%|████▊     | 5793/12000 [3:26:18<3:19:06,  1.92s/it]
                                                        

 48%|████▊     | 5793/12000 [3:26:18<3:19:06,  1.92s/it]
 48%|████▊     | 5794/12000 [3:26:19<3:09:46,  1.83s/it]
                                                        

 48%|████▊     | 5794/12000 [3:26:19<3:09:46,  1.83s/it]
 48%|████▊     | 5795/12000 [3:26:21<3:00:50,  1.75s/it]
                                                        

 48%|████▊     | 5795/12000 [3:26:21<3:00:50,  1.75s/it]
 48%|████▊     | 5796/12000 [3:26:22<2:53:07,  1.67s/it]
                                                        

 48%|████▊     | 5796/12000 [3:26:22<2:53:07,  1.67s/it]
 48%|████▊     | 5797/12000 [3:26:24<3:00:36,  1.75s/it]
                                                        

 48%|████▊     | 5797/12000 [3:26:24<3:00:36,  1.75s/it]
 48%|████▊     | 5798/12000 [3:26:27<3:34:44,  2.08s/it]
                                                        

 48%|████▊     | 5798/12000 [3:26:27<3:34:44,  2.08s/it]
 48%|████▊     | 5799/12000 [3:26:29<3:17:25,  1.91s/it]
                                                        

 48%|████▊     | 5799/12000 [3:26:29<3:17:25,  1.91s/it]
 48%|████▊     | 5800/12000 [3:26:32<4:09:18,  2.41s/it]
                                                        

 48%|████▊     | 5800/12000 [3:26:32<4:09:18,  2.41s/it]
 48%|████▊     | 5801/12000 [3:26:36<4:38:14,  2.69s/it]
                                                        

 48%|████▊     | 5801/12000 [3:26:36<4:38:14,  2.69s/it]
 48%|████▊     | 5802/12000 [3:26:37<3:59:33,  2.32s/it]
                                                        

 48%|████▊     | 5802/12000 [3:26:37<3:59:33,  2.32s/it]
 48%|████▊     | 5803/12000 [3:26:39<3:51:22,  2.24s/it]
                                                        

 48%|████▊     | 5803/12000 [3:26:39<3:51:22,  2.24s/it]
 48%|████▊     | 5804/12000 [3:26:42<4:12:32,  2.45s/it]
                                                        

 48%|████▊     | 5804/12000 [3:26:42<4:12:32,  2.45s/it]
 48%|████▊     | 5805/12000 [3:26:45<4:35:31,  2.67s/it]
                                                        

 48%|████▊     | 5805/12000 [3:26:45<4:35:31,  2.67s/it]
 48%|████▊     | 5806/12000 [3:26:48<4:23:50,  2.56s/it]
                                                        

 48%|████▊     | 5806/12000 [3:26:48<4:23:50,  2.56s/it]
 48%|████▊     | 5807/12000 [3:26:49<3:58:58,  2.32s/it]
                                                        

 48%|████▊     | 5807/12000 [3:26:49<3:58:58,  2.32s/it]
 48%|████▊     | 5808/12000 [3:26:52<4:04:29,  2.37s/it]
                                                        

 48%|████▊     | 5808/12000 [3:26:52<4:04:29,  2.37s/it]
 48%|████▊     | 5809/12000 [3:26:54<3:52:14,  2.25s/it]
                                                        

 48%|████▊     | 5809/12000 [3:26:54<3:52:14,  2.25s/it]
 48%|████▊     | 5810/12000 [3:26:55<3:25:46,  1.99s/it]
                                                        

 48%|████▊     | 5810/12000 [3:26:55<3:25:46,  1.99s/it]
 48%|████▊     | 5811/12000 [3:26:58<3:50:04,  2.23s/it]
                                                        

 48%|████▊     | 5811/12000 [3:26:58<3:50:04,  2.23s/it]
 48%|████▊     | 5812/12000 [3:27:00<3:31:52,  2.05s/it]
                                                        

 48%|████▊     | 5812/12000 [3:27:00<3:31:52,  2.05s/it]
 48%|████▊     | 5813/12000 [3:27:01<3:25:28,  1.99s/it]
                                                        

 48%|████▊     | 5813/12000 [3:27:01<3:25:28,  1.99s/it]
 48%|████▊     | 5814/12000 [3:27:03<3:13:19,  1.88s/it]
                                                        

 48%|████▊     | 5814/12000 [3:27:03<3:13:19,  1.88s/it]
 48%|████▊     | 5815/12000 [3:27:05<3:26:40,  2.00s/it]
                                                        

 48%|████▊     | 5815/12000 [3:27:05<3:26:40,  2.00s/it]
 48%|████▊     | 5816/12000 [3:27:07<3:21:23,  1.95s/it]
                                                        

 48%|████▊     | 5816/12000 [3:27:07<3:21:23,  1.95s/it]
 48%|████▊     | 5817/12000 [3:27:09<3:17:22,  1.92s/it]
                                                        

 48%|████▊     | 5817/12000 [3:27:09<3:17:22,  1.92s/it]
 48%|████▊     | 5818/12000 [3:27:11<3:22:36,  1.97s/it]
                                                        

 48%|████▊     | 5818/12000 [3:27:11<3:22:36,  1.97s/it]
 48%|████▊     | 5819/12000 [3:27:13<3:23:37,  1.98s/it]
                                                        

 48%|████▊     | 5819/12000 [3:27:13<3:23:37,  1.98s/it]
 48%|████▊     | 5820/12000 [3:27:15<3:10:50,  1.85s/it]
                                                        

 48%|████▊     | 5820/12000 [3:27:15<3:10:50,  1.85s/it]
 49%|████▊     | 5821/12000 [3:27:19<4:23:33,  2.56s/it]
                                                        

 49%|████▊     | 5821/12000 [3:27:19<4:23:33,  2.56s/it]
 49%|████▊     | 5822/12000 [3:27:22<4:53:07,  2.85s/it]
                                                        

 49%|████▊     | 5822/12000 [3:27:22<4:53:07,  2.85s/it]
 49%|████▊     | 5823/12000 [3:27:25<4:41:29,  2.73s/it]
                                                        

 49%|████▊     | 5823/12000 [3:27:25<4:41:29,  2.73s/it]
 49%|████▊     | 5824/12000 [3:27:28<4:40:26,  2.72s/it]
                                                        

 49%|████▊     | 5824/12000 [3:27:28<4:40:26,  2.72s/it]
 49%|████▊     | 5825/12000 [3:27:30<4:18:25,  2.51s/it]
                                                        

 49%|████▊     | 5825/12000 [3:27:30<4:18:25,  2.51s/it]
 49%|████▊     | 5826/12000 [3:27:31<3:51:15,  2.25s/it]
                                                        

 49%|████▊     | 5826/12000 [3:27:31<3:51:15,  2.25s/it]
 49%|████▊     | 5827/12000 [3:27:33<3:29:28,  2.04s/it]
                                                        

 49%|████▊     | 5827/12000 [3:27:33<3:29:28,  2.04s/it]
 49%|████▊     | 5828/12000 [3:27:34<3:14:38,  1.89s/it]
                                                        

 49%|████▊     | 5828/12000 [3:27:34<3:14:38,  1.89s/it]
 49%|████▊     | 5829/12000 [3:27:36<3:06:40,  1.81s/it]
                                                        

 49%|████▊     | 5829/12000 [3:27:36<3:06:40,  1.81s/it]
 49%|████▊     | 5830/12000 [3:27:38<3:13:51,  1.89s/it]
                                                        

 49%|████▊     | 5830/12000 [3:27:38<3:13:51,  1.89s/it]
 49%|████▊     | 5831/12000 [3:27:40<3:26:45,  2.01s/it]
                                                        

 49%|████▊     | 5831/12000 [3:27:40<3:26:45,  2.01s/it]
 49%|████▊     | 5832/12000 [3:27:44<4:09:15,  2.42s/it]
                                                        
{'loss': '1.034', 'grad_norm': '0.4823', 'learning_rate': '0.0001142', 'entropy': '1.029', 'num_tokens': '3.126e+07', 'mean_token_accuracy': '0.7499', 'epoch': '0.0512'}
{'loss': '0.8793', 'grad_norm': '0.3531', 'learning_rate': '0.0001141', 'entropy': '0.7623', 'num_tokens': '3.127e+07', 'mean_token_accuracy': '0.8135', 'epoch': '0.05121'}
{'loss': '0.6408', 'grad_norm': '0.3299', 'learning_rate': '0.0001141', 'entropy': '0.7799', 'num_tokens': '3.127e+07', 'mean_token_accuracy': '0.801', 'epoch': '0.05121'}
{'loss': '0.7491', 'grad_norm': '0.354', 'learning_rate': '0.0001141', 'entropy': '0.7457', 'num_tokens': '3.128e+07', 'mean_token_accuracy': '0.8106', 'epoch': '0.05122'}
{'loss': '0.8631', 'grad_norm': '0.373', 'learning_rate': '0.0001141', 'entropy': '0.8526', 'num_tokens': '3.128e+07', 'mean_token_accuracy': '0.7797', 'epoch': '0.05123'}
{'loss': '0.8001', 'grad_norm': '0.3343', 'learning_rate': '0.000114', 'entropy': '0.9151', 'num_tokens': '3.129e+07', 'mean_token_accuracy': '0.7792', 'epoch': '0.05124'}
{'loss': '0.7417', 'grad_norm': '0.3416', 'learning_rate': '0.000114', 'entropy': '1.035', 'num_tokens': '3.129e+07', 'mean_token_accuracy': '0.7498', 'epoch': '0.05125'}
{'loss': '0.8766', 'grad_norm': '0.4488', 'learning_rate': '0.000114', 'entropy': '0.854', 'num_tokens': '3.13e+07', 'mean_token_accuracy': '0.7826', 'epoch': '0.05126'}
{'loss': '0.6833', 'grad_norm': '0.4063', 'learning_rate': '0.000114', 'entropy': '0.6891', 'num_tokens': '3.13e+07', 'mean_token_accuracy': '0.8166', 'epoch': '0.05127'}
{'loss': '0.6715', 'grad_norm': '0.3482', 'learning_rate': '0.0001139', 'entropy': '0.695', 'num_tokens': '3.131e+07', 'mean_token_accuracy': '0.8301', 'epoch': '0.05128'}
{'loss': '0.9821', 'grad_norm': '0.4548', 'learning_rate': '0.0001139', 'entropy': '0.9437', 'num_tokens': '3.131e+07', 'mean_token_accuracy': '0.7704', 'epoch': '0.05128'}
{'loss': '1.007', 'grad_norm': '0.403', 'learning_rate': '0.0001139', 'entropy': '1.009', 'num_tokens': '3.132e+07', 'mean_token_accuracy': '0.7614', 'epoch': '0.05129'}
{'loss': '1.076', 'grad_norm': '0.4021', 'learning_rate': '0.0001138', 'entropy': '0.9773', 'num_tokens': '3.132e+07', 'mean_token_accuracy': '0.7579', 'epoch': '0.0513'}
{'loss': '1.308', 'grad_norm': '0.5461', 'learning_rate': '0.0001138', 'entropy': '1.127', 'num_tokens': '3.133e+07', 'mean_token_accuracy': '0.7385', 'epoch': '0.05131'}
{'loss': '0.4713', 'grad_norm': '0.2684', 'learning_rate': '0.0001138', 'entropy': '0.507', 'num_tokens': '3.133e+07', 'mean_token_accuracy': '0.8686', 'epoch': '0.05132'}
{'loss': '0.8733', 'grad_norm': '0.4461', 'learning_rate': '0.0001138', 'entropy': '0.8567', 'num_tokens': '3.134e+07', 'mean_token_accuracy': '0.7846', 'epoch': '0.05133'}
{'loss': '0.9796', 'grad_norm': '0.4697', 'learning_rate': '0.0001137', 'entropy': '0.9504', 'num_tokens': '3.134e+07', 'mean_token_accuracy': '0.7679', 'epoch': '0.05134'}
{'loss': '0.9187', 'grad_norm': '0.453', 'learning_rate': '0.0001137', 'entropy': '0.9227', 'num_tokens': '3.135e+07', 'mean_token_accuracy': '0.7745', 'epoch': '0.05135'}
{'loss': '0.9593', 'grad_norm': '0.36', 'learning_rate': '0.0001137', 'entropy': '0.9013', 'num_tokens': '3.135e+07', 'mean_token_accuracy': '0.7687', 'epoch': '0.05136'}
{'loss': '1.023', 'grad_norm': '0.4159', 'learning_rate': '0.0001137', 'entropy': '1.017', 'num_tokens': '3.136e+07', 'mean_token_accuracy': '0.7602', 'epoch': '0.05136'}
{'loss': '0.7588', 'grad_norm': '0.3163', 'learning_rate': '0.0001136', 'entropy': '0.7567', 'num_tokens': '3.137e+07', 'mean_token_accuracy': '0.8047', 'epoch': '0.05137'}
{'loss': '0.7114', 'grad_norm': '0.3924', 'learning_rate': '0.0001136', 'entropy': '0.7012', 'num_tokens': '3.137e+07', 'mean_token_accuracy': '0.8224', 'epoch': '0.05138'}
{'loss': '0.5351', 'grad_norm': '0.3354', 'learning_rate': '0.0001136', 'entropy': '0.5575', 'num_tokens': '3.138e+07', 'mean_token_accuracy': '0.8611', 'epoch': '0.05139'}
{'loss': '0.9995', 'grad_norm': '0.3566', 'learning_rate': '0.0001135', 'entropy': '0.9198', 'num_tokens': '3.139e+07', 'mean_token_accuracy': '0.7769', 'epoch': '0.0514'}
{'loss': '0.8714', 'grad_norm': '0.3961', 'learning_rate': '0.0001135', 'entropy': '0.8952', 'num_tokens': '3.139e+07', 'mean_token_accuracy': '0.7752', 'epoch': '0.05141'}
{'loss': '0.9294', 'grad_norm': '0.3941', 'learning_rate': '0.0001135', 'entropy': '0.9983', 'num_tokens': '3.139e+07', 'mean_token_accuracy': '0.7578', 'epoch': '0.05142'}
{'loss': '0.8018', 'grad_norm': '0.4052', 'learning_rate': '0.0001135', 'entropy': '0.8884', 'num_tokens': '3.14e+07', 'mean_token_accuracy': '0.774', 'epoch': '0.05143'}
{'loss': '0.6297', 'grad_norm': '0.2671', 'learning_rate': '0.0001134', 'entropy': '0.6016', 'num_tokens': '3.141e+07', 'mean_token_accuracy': '0.8428', 'epoch': '0.05144'}
{'loss': '0.8342', 'grad_norm': '0.3786', 'learning_rate': '0.0001134', 'entropy': '0.8443', 'num_tokens': '3.141e+07', 'mean_token_accuracy': '0.7968', 'epoch': '0.05144'}
{'loss': '1.066', 'grad_norm': '0.5256', 'learning_rate': '0.0001134', 'entropy': '1.177', 'num_tokens': '3.142e+07', 'mean_token_accuracy': '0.7131', 'epoch': '0.05145'}
{'loss': '0.7765', 'grad_norm': '0.3699', 'learning_rate': '0.0001134', 'entropy': '0.7159', 'num_tokens': '3.142e+07', 'mean_token_accuracy': '0.818', 'epoch': '0.05146'}
{'loss': '0.8702', 'grad_norm': '0.4884', 'learning_rate': '0.0001133', 'entropy': '0.8686', 'num_tokens': '3.143e+07', 'mean_token_accuracy': '0.771', 'epoch': '0.05147'}
{'loss': '1.326', 'grad_norm': '0.4341', 'learning_rate': '0.0001133', 'entropy': '1.112', 'num_tokens': '3.143e+07', 'mean_token_accuracy': '0.7353', 'epoch': '0.05148'}
{'loss': '0.8567', 'grad_norm': '0.4532', 'learning_rate': '0.0001133', 'entropy': '0.8852', 'num_tokens': '3.144e+07', 'mean_token_accuracy': '0.7704', 'epoch': '0.05149'}
{'loss': '1.013', 'grad_norm': '0.4158', 'learning_rate': '0.0001132', 'entropy': '0.9818', 'num_tokens': '3.144e+07', 'mean_token_accuracy': '0.7612', 'epoch': '0.0515'}
{'loss': '0.5466', 'grad_norm': '0.2695', 'learning_rate': '0.0001132', 'entropy': '0.5504', 'num_tokens': '3.145e+07', 'mean_token_accuracy': '0.8528', 'epoch': '0.05151'}
{'loss': '1.16', 'grad_norm': '0.353', 'learning_rate': '0.0001132', 'entropy': '0.9955', 'num_tokens': '3.146e+07', 'mean_token_accuracy': '0.7583', 'epoch': '0.05151'}
{'loss': '0.7426', 'grad_norm': '0.3596', 'learning_rate': '0.0001132', 'entropy': '0.7761', 'num_tokens': '3.146e+07', 'mean_token_accuracy': '0.8078', 'epoch': '0.05152'}
{'loss': '0.7276', 'grad_norm': '0.4116', 'learning_rate': '0.0001131', 'entropy': '0.8307', 'num_tokens': '3.147e+07', 'mean_token_accuracy': '0.7926', 'epoch': '0.05153'}
{'loss': '0.5448', 'grad_norm': '0.3622', 'learning_rate': '0.0001131', 'entropy': '0.5354', 'num_tokens': '3.147e+07', 'mean_token_accuracy': '0.8695', 'epoch': '0.05154'}
{'loss': '0.8273', 'grad_norm': '0.4543', 'learning_rate': '0.0001131', 'entropy': '0.7867', 'num_tokens': '3.148e+07', 'mean_token_accuracy': '0.7969', 'epoch': '0.05155'}
{'loss': '0.7716', 'grad_norm': '0.3292', 'learning_rate': '0.0001131', 'entropy': '0.8086', 'num_tokens': '3.148e+07', 'mean_token_accuracy': '0.7999', 'epoch': '0.05156'}
{'loss': '0.7452', 'grad_norm': '0.4188', 'learning_rate': '0.000113', 'entropy': '0.7786', 'num_tokens': '3.149e+07', 'mean_token_accuracy': '0.8073', 'epoch': '0.05157'}
{'loss': '0.769', 'grad_norm': '0.4811', 'learning_rate': '0.000113', 'entropy': '0.8015', 'num_tokens': '3.149e+07', 'mean_token_accuracy': '0.7943', 'epoch': '0.05158'}
{'loss': '0.7839', 'grad_norm': '0.4369', 'learning_rate': '0.000113', 'entropy': '0.7763', 'num_tokens': '3.15e+07', 'mean_token_accuracy': '0.791', 'epoch': '0.05159'}
{'loss': '1.239', 'grad_norm': '0.3726', 'learning_rate': '0.0001129', 'entropy': '0.9957', 'num_tokens': '3.15e+07', 'mean_token_accuracy': '0.7712', 'epoch': '0.05159'}
{'loss': '1.004', 'grad_norm': '0.3729', 'learning_rate': '0.0001129', 'entropy': '0.988', 'num_tokens': '3.151e+07', 'mean_token_accuracy': '0.7578', 'epoch': '0.0516'}

 49%|████▊     | 5832/12000 [3:27:44<4:09:15,  2.42s/it]
 49%|████▊     | 5833/12000 [3:27:47<4:43:06,  2.75s/it]
                                                        

 49%|████▊     | 5833/12000 [3:27:47<4:43:06,  2.75s/it]
 49%|████▊     | 5834/12000 [3:27:49<4:01:01,  2.35s/it]
                                                        

 49%|████▊     | 5834/12000 [3:27:49<4:01:01,  2.35s/it]
 49%|████▊     | 5835/12000 [3:27:51<4:04:54,  2.38s/it]
                                                        

 49%|████▊     | 5835/12000 [3:27:51<4:04:54,  2.38s/it]
 49%|████▊     | 5836/12000 [3:27:53<3:39:33,  2.14s/it]
                                                        

 49%|████▊     | 5836/12000 [3:27:53<3:39:33,  2.14s/it]
 49%|████▊     | 5837/12000 [3:27:55<3:30:40,  2.05s/it]
                                                        

 49%|████▊     | 5837/12000 [3:27:55<3:30:40,  2.05s/it]
 49%|████▊     | 5838/12000 [3:27:56<3:11:18,  1.86s/it]
                                                        

 49%|████▊     | 5838/12000 [3:27:56<3:11:18,  1.86s/it]
 49%|████▊     | 5839/12000 [3:27:58<3:02:12,  1.77s/it]
                                                        

 49%|████▊     | 5839/12000 [3:27:58<3:02:12,  1.77s/it]
 49%|████▊     | 5840/12000 [3:27:59<2:50:36,  1.66s/it]
                                                        

 49%|████▊     | 5840/12000 [3:27:59<2:50:36,  1.66s/it]
 49%|████▊     | 5841/12000 [3:28:02<3:33:07,  2.08s/it]
                                                        

 49%|████▊     | 5841/12000 [3:28:02<3:33:07,  2.08s/it]
 49%|████▊     | 5842/12000 [3:28:03<3:13:24,  1.88s/it]
                                                        

 49%|████▊     | 5842/12000 [3:28:03<3:13:24,  1.88s/it]
 49%|████▊     | 5843/12000 [3:28:05<3:03:23,  1.79s/it]
                                                        

 49%|████▊     | 5843/12000 [3:28:05<3:03:23,  1.79s/it]
 49%|████▊     | 5844/12000 [3:28:08<3:45:57,  2.20s/it]
                                                        

 49%|████▊     | 5844/12000 [3:28:08<3:45:57,  2.20s/it]
 49%|████▊     | 5845/12000 [3:28:10<3:28:26,  2.03s/it]
                                                        

 49%|████▊     | 5845/12000 [3:28:10<3:28:26,  2.03s/it]
 49%|████▊     | 5846/12000 [3:28:11<3:15:23,  1.91s/it]
                                                        

 49%|████▊     | 5846/12000 [3:28:11<3:15:23,  1.91s/it]
 49%|████▊     | 5847/12000 [3:28:13<3:03:31,  1.79s/it]
                                                        

 49%|████▊     | 5847/12000 [3:28:13<3:03:31,  1.79s/it]
 49%|████▊     | 5848/12000 [3:28:14<2:57:17,  1.73s/it]
                                                        

 49%|████▊     | 5848/12000 [3:28:14<2:57:17,  1.73s/it]
 49%|████▊     | 5849/12000 [3:28:16<3:04:22,  1.80s/it]
                                                        

 49%|████▊     | 5849/12000 [3:28:16<3:04:22,  1.80s/it]
 49%|████▉     | 5850/12000 [3:28:20<4:05:30,  2.40s/it]
                                                        

 49%|████▉     | 5850/12000 [3:28:20<4:05:30,  2.40s/it]
 49%|████▉     | 5851/12000 [3:28:23<4:15:01,  2.49s/it]
                                                        

 49%|████▉     | 5851/12000 [3:28:23<4:15:01,  2.49s/it]
 49%|████▉     | 5852/12000 [3:28:25<4:12:58,  2.47s/it]
                                                        

 49%|████▉     | 5852/12000 [3:28:25<4:12:58,  2.47s/it]
 49%|████▉     | 5853/12000 [3:28:27<3:40:06,  2.15s/it]
                                                        

 49%|████▉     | 5853/12000 [3:28:27<3:40:06,  2.15s/it]
 49%|████▉     | 5854/12000 [3:28:31<4:43:00,  2.76s/it]
                                                        

 49%|████▉     | 5854/12000 [3:28:31<4:43:00,  2.76s/it]
 49%|████▉     | 5855/12000 [3:28:33<4:19:52,  2.54s/it]
                                                        

 49%|████▉     | 5855/12000 [3:28:33<4:19:52,  2.54s/it]
 49%|████▉     | 5856/12000 [3:28:35<3:49:08,  2.24s/it]
                                                        

 49%|████▉     | 5856/12000 [3:28:35<3:49:08,  2.24s/it]
 49%|████▉     | 5857/12000 [3:28:36<3:30:14,  2.05s/it]
                                                        

 49%|████▉     | 5857/12000 [3:28:36<3:30:14,  2.05s/it]
 49%|████▉     | 5858/12000 [3:28:38<3:13:14,  1.89s/it]
                                                        

 49%|████▉     | 5858/12000 [3:28:38<3:13:14,  1.89s/it]
 49%|████▉     | 5859/12000 [3:28:40<3:26:28,  2.02s/it]
                                                        

 49%|████▉     | 5859/12000 [3:28:40<3:26:28,  2.02s/it]
 49%|████▉     | 5860/12000 [3:28:42<3:29:13,  2.04s/it]
                                                        

 49%|████▉     | 5860/12000 [3:28:42<3:29:13,  2.04s/it]
 49%|████▉     | 5861/12000 [3:28:44<3:18:39,  1.94s/it]
                                                        

 49%|████▉     | 5861/12000 [3:28:44<3:18:39,  1.94s/it]
 49%|████▉     | 5862/12000 [3:28:47<3:56:19,  2.31s/it]
                                                        

 49%|████▉     | 5862/12000 [3:28:47<3:56:19,  2.31s/it]
 49%|████▉     | 5863/12000 [3:28:49<3:45:32,  2.20s/it]
                                                        

 49%|████▉     | 5863/12000 [3:28:49<3:45:32,  2.20s/it]
 49%|████▉     | 5864/12000 [3:28:50<3:21:40,  1.97s/it]
                                                        

 49%|████▉     | 5864/12000 [3:28:50<3:21:40,  1.97s/it]
 49%|████▉     | 5865/12000 [3:28:52<3:21:54,  1.97s/it]
                                                        

 49%|████▉     | 5865/12000 [3:28:52<3:21:54,  1.97s/it]
 49%|████▉     | 5866/12000 [3:28:56<4:05:26,  2.40s/it]
                                                        

 49%|████▉     | 5866/12000 [3:28:56<4:05:26,  2.40s/it]
 49%|████▉     | 5867/12000 [3:28:57<3:35:10,  2.11s/it]
                                                        

 49%|████▉     | 5867/12000 [3:28:57<3:35:10,  2.11s/it]
 49%|████▉     | 5868/12000 [3:28:59<3:38:52,  2.14s/it]
                                                        

 49%|████▉     | 5868/12000 [3:28:59<3:38:52,  2.14s/it]
 49%|████▉     | 5869/12000 [3:29:01<3:23:35,  1.99s/it]
                                                        

 49%|████▉     | 5869/12000 [3:29:01<3:23:35,  1.99s/it]
 49%|████▉     | 5870/12000 [3:29:02<3:04:45,  1.81s/it]
                                                        

 49%|████▉     | 5870/12000 [3:29:02<3:04:45,  1.81s/it]
 49%|████▉     | 5871/12000 [3:29:04<2:54:47,  1.71s/it]
                                                        

 49%|████▉     | 5871/12000 [3:29:04<2:54:47,  1.71s/it]
 49%|████▉     | 5872/12000 [3:29:06<2:59:43,  1.76s/it]
                                                        

 49%|████▉     | 5872/12000 [3:29:06<2:59:43,  1.76s/it]
 49%|████▉     | 5873/12000 [3:29:08<3:22:24,  1.98s/it]
                                                        

 49%|████▉     | 5873/12000 [3:29:08<3:22:24,  1.98s/it]
 49%|████▉     | 5874/12000 [3:29:10<3:05:29,  1.82s/it]
                                                        

 49%|████▉     | 5874/12000 [3:29:10<3:05:29,  1.82s/it]
 49%|████▉     | 5875/12000 [3:29:13<4:02:56,  2.38s/it]
                                                        

 49%|████▉     | 5875/12000 [3:29:13<4:02:56,  2.38s/it]
 49%|████▉     | 5876/12000 [3:29:15<3:49:58,  2.25s/it]
                                                        

 49%|████▉     | 5876/12000 [3:29:15<3:49:58,  2.25s/it]
 49%|████▉     | 5877/12000 [3:29:19<4:37:53,  2.72s/it]
                                                        

 49%|████▉     | 5877/12000 [3:29:19<4:37:53,  2.72s/it]
 49%|████▉     | 5878/12000 [3:29:21<4:20:06,  2.55s/it]
                                                        

 49%|████▉     | 5878/12000 [3:29:21<4:20:06,  2.55s/it]
 49%|████▉     | 5879/12000 [3:29:24<4:35:28,  2.70s/it]
                                                        
{'loss': '0.8495', 'grad_norm': '0.3665', 'learning_rate': '0.0001129', 'entropy': '0.98', 'num_tokens': '3.152e+07', 'mean_token_accuracy': '0.7502', 'epoch': '0.05161'}
{'loss': '0.735', 'grad_norm': '0.3712', 'learning_rate': '0.0001129', 'entropy': '0.8807', 'num_tokens': '3.152e+07', 'mean_token_accuracy': '0.7852', 'epoch': '0.05162'}
{'loss': '0.6419', 'grad_norm': '0.3181', 'learning_rate': '0.0001128', 'entropy': '0.6413', 'num_tokens': '3.153e+07', 'mean_token_accuracy': '0.8354', 'epoch': '0.05163'}
{'loss': '1.044', 'grad_norm': '0.3599', 'learning_rate': '0.0001128', 'entropy': '0.9672', 'num_tokens': '3.154e+07', 'mean_token_accuracy': '0.769', 'epoch': '0.05164'}
{'loss': '0.8289', 'grad_norm': '0.4104', 'learning_rate': '0.0001128', 'entropy': '0.9417', 'num_tokens': '3.154e+07', 'mean_token_accuracy': '0.7635', 'epoch': '0.05165'}
{'loss': '0.8482', 'grad_norm': '0.3082', 'learning_rate': '0.0001128', 'entropy': '0.872', 'num_tokens': '3.155e+07', 'mean_token_accuracy': '0.7943', 'epoch': '0.05166'}
{'loss': '0.7251', 'grad_norm': '0.3632', 'learning_rate': '0.0001127', 'entropy': '0.8091', 'num_tokens': '3.155e+07', 'mean_token_accuracy': '0.8001', 'epoch': '0.05167'}
{'loss': '0.7534', 'grad_norm': '0.4031', 'learning_rate': '0.0001127', 'entropy': '0.764', 'num_tokens': '3.156e+07', 'mean_token_accuracy': '0.7972', 'epoch': '0.05167'}
{'loss': '1.104', 'grad_norm': '0.4453', 'learning_rate': '0.0001127', 'entropy': '0.8856', 'num_tokens': '3.156e+07', 'mean_token_accuracy': '0.7905', 'epoch': '0.05168'}
{'loss': '0.8308', 'grad_norm': '0.3863', 'learning_rate': '0.0001126', 'entropy': '0.8312', 'num_tokens': '3.157e+07', 'mean_token_accuracy': '0.7884', 'epoch': '0.05169'}
{'loss': '1.024', 'grad_norm': '0.5078', 'learning_rate': '0.0001126', 'entropy': '1.132', 'num_tokens': '3.157e+07', 'mean_token_accuracy': '0.7183', 'epoch': '0.0517'}
{'loss': '0.6908', 'grad_norm': '0.4895', 'learning_rate': '0.0001126', 'entropy': '0.7542', 'num_tokens': '3.158e+07', 'mean_token_accuracy': '0.8048', 'epoch': '0.05171'}
{'loss': '0.8805', 'grad_norm': '0.3776', 'learning_rate': '0.0001126', 'entropy': '0.8583', 'num_tokens': '3.158e+07', 'mean_token_accuracy': '0.7828', 'epoch': '0.05172'}
{'loss': '0.8444', 'grad_norm': '0.3534', 'learning_rate': '0.0001125', 'entropy': '0.8642', 'num_tokens': '3.158e+07', 'mean_token_accuracy': '0.7694', 'epoch': '0.05173'}
{'loss': '0.8121', 'grad_norm': '0.3552', 'learning_rate': '0.0001125', 'entropy': '0.8055', 'num_tokens': '3.159e+07', 'mean_token_accuracy': '0.7813', 'epoch': '0.05174'}
{'loss': '0.756', 'grad_norm': '0.3556', 'learning_rate': '0.0001125', 'entropy': '0.7777', 'num_tokens': '3.159e+07', 'mean_token_accuracy': '0.8008', 'epoch': '0.05174'}
{'loss': '0.7616', 'grad_norm': '0.4753', 'learning_rate': '0.0001125', 'entropy': '0.7879', 'num_tokens': '3.16e+07', 'mean_token_accuracy': '0.8049', 'epoch': '0.05175'}
{'loss': '0.9547', 'grad_norm': '0.422', 'learning_rate': '0.0001124', 'entropy': '0.8768', 'num_tokens': '3.16e+07', 'mean_token_accuracy': '0.7866', 'epoch': '0.05176'}
{'loss': '0.9683', 'grad_norm': '0.3679', 'learning_rate': '0.0001124', 'entropy': '0.9273', 'num_tokens': '3.161e+07', 'mean_token_accuracy': '0.7715', 'epoch': '0.05177'}
{'loss': '0.7217', 'grad_norm': '0.3337', 'learning_rate': '0.0001124', 'entropy': '0.7569', 'num_tokens': '3.162e+07', 'mean_token_accuracy': '0.8072', 'epoch': '0.05178'}
{'loss': '1.057', 'grad_norm': '0.4115', 'learning_rate': '0.0001123', 'entropy': '0.9129', 'num_tokens': '3.162e+07', 'mean_token_accuracy': '0.7858', 'epoch': '0.05179'}
{'loss': '0.9875', 'grad_norm': '0.4037', 'learning_rate': '0.0001123', 'entropy': '1.026', 'num_tokens': '3.163e+07', 'mean_token_accuracy': '0.7464', 'epoch': '0.0518'}
{'loss': '1.195', 'grad_norm': '0.7003', 'learning_rate': '0.0001123', 'entropy': '1.108', 'num_tokens': '3.164e+07', 'mean_token_accuracy': '0.7468', 'epoch': '0.05181'}
{'loss': '0.8172', 'grad_norm': '0.4235', 'learning_rate': '0.0001123', 'entropy': '0.7045', 'num_tokens': '3.164e+07', 'mean_token_accuracy': '0.8447', 'epoch': '0.05182'}
{'loss': '0.8512', 'grad_norm': '0.4887', 'learning_rate': '0.0001122', 'entropy': '0.8449', 'num_tokens': '3.165e+07', 'mean_token_accuracy': '0.7852', 'epoch': '0.05182'}
{'loss': '0.7693', 'grad_norm': '0.3316', 'learning_rate': '0.0001122', 'entropy': '0.7577', 'num_tokens': '3.165e+07', 'mean_token_accuracy': '0.8047', 'epoch': '0.05183'}
{'loss': '0.7906', 'grad_norm': '0.405', 'learning_rate': '0.0001122', 'entropy': '0.8308', 'num_tokens': '3.166e+07', 'mean_token_accuracy': '0.79', 'epoch': '0.05184'}
{'loss': '0.7164', 'grad_norm': '0.3682', 'learning_rate': '0.0001122', 'entropy': '0.7713', 'num_tokens': '3.166e+07', 'mean_token_accuracy': '0.8054', 'epoch': '0.05185'}
{'loss': '1.258', 'grad_norm': '0.3814', 'learning_rate': '0.0001121', 'entropy': '1.243', 'num_tokens': '3.167e+07', 'mean_token_accuracy': '0.7132', 'epoch': '0.05186'}
{'loss': '0.793', 'grad_norm': '0.4885', 'learning_rate': '0.0001121', 'entropy': '0.8429', 'num_tokens': '3.167e+07', 'mean_token_accuracy': '0.7776', 'epoch': '0.05187'}
{'loss': '0.8337', 'grad_norm': '0.3399', 'learning_rate': '0.0001121', 'entropy': '1.053', 'num_tokens': '3.168e+07', 'mean_token_accuracy': '0.751', 'epoch': '0.05188'}
{'loss': '0.8365', 'grad_norm': '0.3941', 'learning_rate': '0.000112', 'entropy': '0.8171', 'num_tokens': '3.169e+07', 'mean_token_accuracy': '0.7881', 'epoch': '0.05189'}
{'loss': '0.8994', 'grad_norm': '0.3965', 'learning_rate': '0.000112', 'entropy': '0.9865', 'num_tokens': '3.169e+07', 'mean_token_accuracy': '0.7596', 'epoch': '0.0519'}
{'loss': '0.5499', 'grad_norm': '0.3369', 'learning_rate': '0.000112', 'entropy': '0.5752', 'num_tokens': '3.17e+07', 'mean_token_accuracy': '0.8506', 'epoch': '0.0519'}
{'loss': '0.6502', 'grad_norm': '0.38', 'learning_rate': '0.000112', 'entropy': '0.639', 'num_tokens': '3.17e+07', 'mean_token_accuracy': '0.8292', 'epoch': '0.05191'}
{'loss': '0.5814', 'grad_norm': '0.3302', 'learning_rate': '0.0001119', 'entropy': '0.7026', 'num_tokens': '3.17e+07', 'mean_token_accuracy': '0.8172', 'epoch': '0.05192'}
{'loss': '0.8712', 'grad_norm': '0.476', 'learning_rate': '0.0001119', 'entropy': '0.7953', 'num_tokens': '3.171e+07', 'mean_token_accuracy': '0.7888', 'epoch': '0.05193'}
{'loss': '0.8814', 'grad_norm': '0.3984', 'learning_rate': '0.0001119', 'entropy': '0.8765', 'num_tokens': '3.171e+07', 'mean_token_accuracy': '0.7687', 'epoch': '0.05194'}
{'loss': '0.7876', 'grad_norm': '0.4069', 'learning_rate': '0.0001118', 'entropy': '0.7877', 'num_tokens': '3.172e+07', 'mean_token_accuracy': '0.7976', 'epoch': '0.05195'}
{'loss': '0.8773', 'grad_norm': '0.4438', 'learning_rate': '0.0001118', 'entropy': '0.8673', 'num_tokens': '3.172e+07', 'mean_token_accuracy': '0.7898', 'epoch': '0.05196'}
{'loss': '0.6868', 'grad_norm': '0.3868', 'learning_rate': '0.0001118', 'entropy': '0.7345', 'num_tokens': '3.173e+07', 'mean_token_accuracy': '0.8177', 'epoch': '0.05197'}
{'loss': '0.964', 'grad_norm': '0.4699', 'learning_rate': '0.0001118', 'entropy': '0.9943', 'num_tokens': '3.173e+07', 'mean_token_accuracy': '0.7558', 'epoch': '0.05198'}
{'loss': '0.8047', 'grad_norm': '0.4067', 'learning_rate': '0.0001117', 'entropy': '0.8996', 'num_tokens': '3.174e+07', 'mean_token_accuracy': '0.7701', 'epoch': '0.05198'}
{'loss': '0.6642', 'grad_norm': '0.2835', 'learning_rate': '0.0001117', 'entropy': '0.7477', 'num_tokens': '3.175e+07', 'mean_token_accuracy': '0.807', 'epoch': '0.05199'}
{'loss': '0.9972', 'grad_norm': '0.393', 'learning_rate': '0.0001117', 'entropy': '0.8992', 'num_tokens': '3.175e+07', 'mean_token_accuracy': '0.7827', 'epoch': '0.052'}
{'loss': '0.9413', 'grad_norm': '0.317', 'learning_rate': '0.0001117', 'entropy': '0.8778', 'num_tokens': '3.176e+07', 'mean_token_accuracy': '0.7759', 'epoch': '0.05201'}
{'loss': '0.8506', 'grad_norm': '0.3621', 'learning_rate': '0.0001116', 'entropy': '0.8075', 'num_tokens': '3.177e+07', 'mean_token_accuracy': '0.7932', 'epoch': '0.05202'}

 49%|████▉     | 5879/12000 [3:29:24<4:35:28,  2.70s/it]
 49%|████▉     | 5880/12000 [3:29:27<4:28:05,  2.63s/it]
                                                        

 49%|████▉     | 5880/12000 [3:29:27<4:28:05,  2.63s/it]
 49%|████▉     | 5881/12000 [3:29:28<3:51:36,  2.27s/it]
                                                        

 49%|████▉     | 5881/12000 [3:29:28<3:51:36,  2.27s/it]
 49%|████▉     | 5882/12000 [3:29:30<3:27:07,  2.03s/it]
                                                        

 49%|████▉     | 5882/12000 [3:29:30<3:27:07,  2.03s/it]
 49%|████▉     | 5883/12000 [3:29:31<3:15:10,  1.91s/it]
                                                        

 49%|████▉     | 5883/12000 [3:29:31<3:15:10,  1.91s/it]
 49%|████▉     | 5884/12000 [3:29:36<4:41:28,  2.76s/it]
                                                        

 49%|████▉     | 5884/12000 [3:29:36<4:41:28,  2.76s/it]
 49%|████▉     | 5885/12000 [3:29:38<4:09:56,  2.45s/it]
                                                        

 49%|████▉     | 5885/12000 [3:29:38<4:09:56,  2.45s/it]
 49%|████▉     | 5886/12000 [3:29:40<4:06:34,  2.42s/it]
                                                        

 49%|████▉     | 5886/12000 [3:29:40<4:06:34,  2.42s/it]
 49%|████▉     | 5887/12000 [3:29:41<3:32:13,  2.08s/it]
                                                        

 49%|████▉     | 5887/12000 [3:29:41<3:32:13,  2.08s/it]
 49%|████▉     | 5888/12000 [3:29:45<4:09:30,  2.45s/it]
                                                        

 49%|████▉     | 5888/12000 [3:29:45<4:09:30,  2.45s/it]
 49%|████▉     | 5889/12000 [3:29:46<3:43:11,  2.19s/it]
                                                        

 49%|████▉     | 5889/12000 [3:29:46<3:43:11,  2.19s/it]
 49%|████▉     | 5890/12000 [3:29:48<3:22:37,  1.99s/it]
                                                        

 49%|████▉     | 5890/12000 [3:29:48<3:22:37,  1.99s/it]
 49%|████▉     | 5891/12000 [3:29:49<3:04:30,  1.81s/it]
                                                        

 49%|████▉     | 5891/12000 [3:29:49<3:04:30,  1.81s/it]
 49%|████▉     | 5892/12000 [3:29:51<2:59:04,  1.76s/it]
                                                        

 49%|████▉     | 5892/12000 [3:29:51<2:59:04,  1.76s/it]
 49%|████▉     | 5893/12000 [3:29:54<3:34:55,  2.11s/it]
                                                        

 49%|████▉     | 5893/12000 [3:29:54<3:34:55,  2.11s/it]
 49%|████▉     | 5894/12000 [3:29:55<3:19:57,  1.96s/it]
                                                        

 49%|████▉     | 5894/12000 [3:29:55<3:19:57,  1.96s/it]
 49%|████▉     | 5895/12000 [3:29:57<3:09:05,  1.86s/it]
                                                        

 49%|████▉     | 5895/12000 [3:29:57<3:09:05,  1.86s/it]
 49%|████▉     | 5896/12000 [3:29:59<3:07:09,  1.84s/it]
                                                        

 49%|████▉     | 5896/12000 [3:29:59<3:07:09,  1.84s/it]
 49%|████▉     | 5897/12000 [3:30:01<3:08:01,  1.85s/it]
                                                        

 49%|████▉     | 5897/12000 [3:30:01<3:08:01,  1.85s/it]
 49%|████▉     | 5898/12000 [3:30:03<3:26:39,  2.03s/it]
                                                        

 49%|████▉     | 5898/12000 [3:30:03<3:26:39,  2.03s/it]
 49%|████▉     | 5899/12000 [3:30:04<3:03:23,  1.80s/it]
                                                        

 49%|████▉     | 5899/12000 [3:30:04<3:03:23,  1.80s/it]
 49%|████▉     | 5900/12000 [3:30:09<4:15:07,  2.51s/it]
                                                        

 49%|████▉     | 5900/12000 [3:30:09<4:15:07,  2.51s/it]
 49%|████▉     | 5901/12000 [3:30:12<4:31:12,  2.67s/it]
                                                        

 49%|████▉     | 5901/12000 [3:30:12<4:31:12,  2.67s/it]
 49%|████▉     | 5902/12000 [3:30:13<3:54:26,  2.31s/it]
                                                        

 49%|████▉     | 5902/12000 [3:30:13<3:54:26,  2.31s/it]
 49%|████▉     | 5903/12000 [3:30:15<3:27:41,  2.04s/it]
                                                        

 49%|████▉     | 5903/12000 [3:30:15<3:27:41,  2.04s/it]
 49%|████▉     | 5904/12000 [3:30:16<3:23:15,  2.00s/it]
                                                        

 49%|████▉     | 5904/12000 [3:30:16<3:23:15,  2.00s/it]
 49%|████▉     | 5905/12000 [3:30:19<3:46:53,  2.23s/it]
                                                        

 49%|████▉     | 5905/12000 [3:30:19<3:46:53,  2.23s/it]
 49%|████▉     | 5906/12000 [3:30:21<3:28:47,  2.06s/it]
                                                        

 49%|████▉     | 5906/12000 [3:30:21<3:28:47,  2.06s/it]
 49%|████▉     | 5907/12000 [3:30:22<3:08:33,  1.86s/it]
                                                        

 49%|████▉     | 5907/12000 [3:30:22<3:08:33,  1.86s/it]
 49%|████▉     | 5908/12000 [3:30:25<3:21:52,  1.99s/it]
                                                        

 49%|████▉     | 5908/12000 [3:30:25<3:21:52,  1.99s/it]
 49%|████▉     | 5909/12000 [3:30:28<4:00:14,  2.37s/it]
                                                        

 49%|████▉     | 5909/12000 [3:30:28<4:00:14,  2.37s/it]
 49%|████▉     | 5910/12000 [3:30:32<5:05:20,  3.01s/it]
                                                        

 49%|████▉     | 5910/12000 [3:30:32<5:05:20,  3.01s/it]
 49%|████▉     | 5911/12000 [3:30:35<5:04:49,  3.00s/it]
                                                        

 49%|████▉     | 5911/12000 [3:30:35<5:04:49,  3.00s/it]
 49%|████▉     | 5912/12000 [3:30:37<4:22:19,  2.59s/it]
                                                        

 49%|████▉     | 5912/12000 [3:30:37<4:22:19,  2.59s/it]
 49%|████▉     | 5913/12000 [3:30:39<3:52:45,  2.29s/it]
                                                        

 49%|████▉     | 5913/12000 [3:30:39<3:52:45,  2.29s/it]
 49%|████▉     | 5914/12000 [3:30:41<4:02:52,  2.39s/it]
                                                        

 49%|████▉     | 5914/12000 [3:30:41<4:02:52,  2.39s/it]
 49%|████▉     | 5915/12000 [3:30:43<3:38:42,  2.16s/it]
                                                        

 49%|████▉     | 5915/12000 [3:30:43<3:38:42,  2.16s/it]
 49%|████▉     | 5916/12000 [3:30:45<3:28:13,  2.05s/it]
                                                        

 49%|████▉     | 5916/12000 [3:30:45<3:28:13,  2.05s/it]
 49%|████▉     | 5917/12000 [3:30:46<3:23:49,  2.01s/it]
                                                        

 49%|████▉     | 5917/12000 [3:30:46<3:23:49,  2.01s/it]
 49%|████▉     | 5918/12000 [3:30:48<3:05:59,  1.83s/it]
                                                        

 49%|████▉     | 5918/12000 [3:30:48<3:05:59,  1.83s/it]
 49%|████▉     | 5919/12000 [3:30:49<2:48:46,  1.67s/it]
                                                        

 49%|████▉     | 5919/12000 [3:30:49<2:48:46,  1.67s/it]
 49%|████▉     | 5920/12000 [3:30:51<2:45:36,  1.63s/it]
                                                        

 49%|████▉     | 5920/12000 [3:30:51<2:45:36,  1.63s/it]
 49%|████▉     | 5921/12000 [3:30:52<2:39:02,  1.57s/it]
                                                        

 49%|████▉     | 5921/12000 [3:30:52<2:39:02,  1.57s/it]
 49%|████▉     | 5922/12000 [3:30:54<2:46:42,  1.65s/it]
                                                        

 49%|████▉     | 5922/12000 [3:30:54<2:46:42,  1.65s/it]
 49%|████▉     | 5923/12000 [3:30:56<3:11:21,  1.89s/it]
                                                        

 49%|████▉     | 5923/12000 [3:30:56<3:11:21,  1.89s/it]
 49%|████▉     | 5924/12000 [3:30:59<3:21:47,  1.99s/it]
                                                        

 49%|████▉     | 5924/12000 [3:30:59<3:21:47,  1.99s/it]
 49%|████▉     | 5925/12000 [3:31:01<3:34:18,  2.12s/it]
                                                        

 49%|████▉     | 5925/12000 [3:31:01<3:34:18,  2.12s/it]
 49%|████▉     | 5926/12000 [3:31:02<3:12:54,  1.91s/it]
                                                        
{'loss': '0.7395', 'grad_norm': '0.3444', 'learning_rate': '0.0001116', 'entropy': '0.7197', 'num_tokens': '3.177e+07', 'mean_token_accuracy': '0.8123', 'epoch': '0.05203'}
{'loss': '0.8095', 'grad_norm': '0.4002', 'learning_rate': '0.0001116', 'entropy': '0.7596', 'num_tokens': '3.178e+07', 'mean_token_accuracy': '0.8104', 'epoch': '0.05204'}
{'loss': '0.5585', 'grad_norm': '0.3069', 'learning_rate': '0.0001115', 'entropy': '0.6071', 'num_tokens': '3.178e+07', 'mean_token_accuracy': '0.8416', 'epoch': '0.05205'}
{'loss': '0.6819', 'grad_norm': '0.3202', 'learning_rate': '0.0001115', 'entropy': '0.6682', 'num_tokens': '3.179e+07', 'mean_token_accuracy': '0.8198', 'epoch': '0.05205'}
{'loss': '0.9259', 'grad_norm': '0.5604', 'learning_rate': '0.0001115', 'entropy': '0.8121', 'num_tokens': '3.179e+07', 'mean_token_accuracy': '0.7981', 'epoch': '0.05206'}
{'loss': '1.346', 'grad_norm': '0.3326', 'learning_rate': '0.0001115', 'entropy': '1.236', 'num_tokens': '3.18e+07', 'mean_token_accuracy': '0.6855', 'epoch': '0.05207'}
{'loss': '1.152', 'grad_norm': '0.488', 'learning_rate': '0.0001114', 'entropy': '1.084', 'num_tokens': '3.181e+07', 'mean_token_accuracy': '0.734', 'epoch': '0.05208'}
{'loss': '1.14', 'grad_norm': '0.603', 'learning_rate': '0.0001114', 'entropy': '1.159', 'num_tokens': '3.181e+07', 'mean_token_accuracy': '0.7268', 'epoch': '0.05209'}
{'loss': '0.7276', 'grad_norm': '0.3631', 'learning_rate': '0.0001114', 'entropy': '0.7516', 'num_tokens': '3.181e+07', 'mean_token_accuracy': '0.7999', 'epoch': '0.0521'}
{'loss': '1.46', 'grad_norm': '0.5174', 'learning_rate': '0.0001114', 'entropy': '1.444', 'num_tokens': '3.182e+07', 'mean_token_accuracy': '0.6762', 'epoch': '0.05211'}
{'loss': '0.9415', 'grad_norm': '0.3712', 'learning_rate': '0.0001113', 'entropy': '0.9609', 'num_tokens': '3.182e+07', 'mean_token_accuracy': '0.768', 'epoch': '0.05212'}
{'loss': '0.7439', 'grad_norm': '0.3961', 'learning_rate': '0.0001113', 'entropy': '0.8414', 'num_tokens': '3.183e+07', 'mean_token_accuracy': '0.7945', 'epoch': '0.05213'}
{'loss': '0.7945', 'grad_norm': '0.3828', 'learning_rate': '0.0001113', 'entropy': '0.8279', 'num_tokens': '3.183e+07', 'mean_token_accuracy': '0.7908', 'epoch': '0.05213'}
{'loss': '0.7783', 'grad_norm': '0.4434', 'learning_rate': '0.0001112', 'entropy': '0.8653', 'num_tokens': '3.184e+07', 'mean_token_accuracy': '0.7917', 'epoch': '0.05214'}
{'loss': '0.5155', 'grad_norm': '0.2745', 'learning_rate': '0.0001112', 'entropy': '0.5593', 'num_tokens': '3.184e+07', 'mean_token_accuracy': '0.8498', 'epoch': '0.05215'}
{'loss': '0.8697', 'grad_norm': '0.4313', 'learning_rate': '0.0001112', 'entropy': '0.8131', 'num_tokens': '3.185e+07', 'mean_token_accuracy': '0.7883', 'epoch': '0.05216'}
{'loss': '0.6052', 'grad_norm': '0.409', 'learning_rate': '0.0001112', 'entropy': '0.6118', 'num_tokens': '3.185e+07', 'mean_token_accuracy': '0.8385', 'epoch': '0.05217'}
{'loss': '0.9126', 'grad_norm': '0.3921', 'learning_rate': '0.0001111', 'entropy': '0.8767', 'num_tokens': '3.186e+07', 'mean_token_accuracy': '0.7872', 'epoch': '0.05218'}
{'loss': '0.7789', 'grad_norm': '0.3492', 'learning_rate': '0.0001111', 'entropy': '0.7233', 'num_tokens': '3.186e+07', 'mean_token_accuracy': '0.8167', 'epoch': '0.05219'}
{'loss': '0.7992', 'grad_norm': '0.359', 'learning_rate': '0.0001111', 'entropy': '0.8104', 'num_tokens': '3.187e+07', 'mean_token_accuracy': '0.788', 'epoch': '0.0522'}
{'loss': '0.7765', 'grad_norm': '0.3801', 'learning_rate': '0.0001111', 'entropy': '0.7996', 'num_tokens': '3.187e+07', 'mean_token_accuracy': '0.7945', 'epoch': '0.05221'}
{'loss': '1.152', 'grad_norm': '0.3618', 'learning_rate': '0.000111', 'entropy': '1.079', 'num_tokens': '3.188e+07', 'mean_token_accuracy': '0.7447', 'epoch': '0.05221'}
{'loss': '0.908', 'grad_norm': '0.3939', 'learning_rate': '0.000111', 'entropy': '0.9301', 'num_tokens': '3.189e+07', 'mean_token_accuracy': '0.7674', 'epoch': '0.05222'}
{'loss': '0.5616', 'grad_norm': '0.2802', 'learning_rate': '0.000111', 'entropy': '0.6561', 'num_tokens': '3.189e+07', 'mean_token_accuracy': '0.8298', 'epoch': '0.05223'}
{'loss': '0.7131', 'grad_norm': '0.4072', 'learning_rate': '0.0001109', 'entropy': '0.7631', 'num_tokens': '3.19e+07', 'mean_token_accuracy': '0.8131', 'epoch': '0.05224'}
{'loss': '0.9829', 'grad_norm': '0.3596', 'learning_rate': '0.0001109', 'entropy': '0.9224', 'num_tokens': '3.19e+07', 'mean_token_accuracy': '0.785', 'epoch': '0.05225'}
{'loss': '0.8992', 'grad_norm': '0.3856', 'learning_rate': '0.0001109', 'entropy': '0.8471', 'num_tokens': '3.191e+07', 'mean_token_accuracy': '0.7942', 'epoch': '0.05226'}
{'loss': '0.7517', 'grad_norm': '0.3908', 'learning_rate': '0.0001109', 'entropy': '0.761', 'num_tokens': '3.192e+07', 'mean_token_accuracy': '0.8021', 'epoch': '0.05227'}
{'loss': '0.6525', 'grad_norm': '0.3744', 'learning_rate': '0.0001108', 'entropy': '0.6679', 'num_tokens': '3.192e+07', 'mean_token_accuracy': '0.8232', 'epoch': '0.05228'}
{'loss': '0.9906', 'grad_norm': '0.4707', 'learning_rate': '0.0001108', 'entropy': '0.9405', 'num_tokens': '3.193e+07', 'mean_token_accuracy': '0.7664', 'epoch': '0.05228'}
{'loss': '0.7884', 'grad_norm': '0.2883', 'learning_rate': '0.0001108', 'entropy': '0.7839', 'num_tokens': '3.193e+07', 'mean_token_accuracy': '0.7964', 'epoch': '0.05229'}
{'loss': '0.5723', 'grad_norm': '0.3561', 'learning_rate': '0.0001108', 'entropy': '0.5897', 'num_tokens': '3.194e+07', 'mean_token_accuracy': '0.8568', 'epoch': '0.0523'}
{'loss': '0.8121', 'grad_norm': '0.372', 'learning_rate': '0.0001107', 'entropy': '0.87', 'num_tokens': '3.195e+07', 'mean_token_accuracy': '0.7746', 'epoch': '0.05231'}
{'loss': '0.9139', 'grad_norm': '0.4244', 'learning_rate': '0.0001107', 'entropy': '0.8902', 'num_tokens': '3.195e+07', 'mean_token_accuracy': '0.7762', 'epoch': '0.05232'}
{'loss': '0.5714', 'grad_norm': '0.3557', 'learning_rate': '0.0001107', 'entropy': '0.5759', 'num_tokens': '3.196e+07', 'mean_token_accuracy': '0.854', 'epoch': '0.05233'}
{'loss': '0.8253', 'grad_norm': '0.372', 'learning_rate': '0.0001106', 'entropy': '0.7058', 'num_tokens': '3.197e+07', 'mean_token_accuracy': '0.816', 'epoch': '0.05234'}
{'loss': '0.7017', 'grad_norm': '0.3386', 'learning_rate': '0.0001106', 'entropy': '0.6747', 'num_tokens': '3.197e+07', 'mean_token_accuracy': '0.8284', 'epoch': '0.05235'}
{'loss': '0.6525', 'grad_norm': '0.3307', 'learning_rate': '0.0001106', 'entropy': '0.6798', 'num_tokens': '3.198e+07', 'mean_token_accuracy': '0.8289', 'epoch': '0.05236'}
{'loss': '0.5371', 'grad_norm': '0.3027', 'learning_rate': '0.0001106', 'entropy': '0.5479', 'num_tokens': '3.198e+07', 'mean_token_accuracy': '0.8599', 'epoch': '0.05236'}
{'loss': '0.8951', 'grad_norm': '0.4544', 'learning_rate': '0.0001105', 'entropy': '0.9139', 'num_tokens': '3.199e+07', 'mean_token_accuracy': '0.7652', 'epoch': '0.05237'}
{'loss': '0.6788', 'grad_norm': '0.4148', 'learning_rate': '0.0001105', 'entropy': '0.7056', 'num_tokens': '3.199e+07', 'mean_token_accuracy': '0.8248', 'epoch': '0.05238'}
{'loss': '0.7716', 'grad_norm': '0.3706', 'learning_rate': '0.0001105', 'entropy': '0.7699', 'num_tokens': '3.199e+07', 'mean_token_accuracy': '0.8022', 'epoch': '0.05239'}
{'loss': '0.4289', 'grad_norm': '0.2462', 'learning_rate': '0.0001105', 'entropy': '0.483', 'num_tokens': '3.2e+07', 'mean_token_accuracy': '0.8797', 'epoch': '0.0524'}
{'loss': '0.9511', 'grad_norm': '0.4205', 'learning_rate': '0.0001104', 'entropy': '0.9411', 'num_tokens': '3.201e+07', 'mean_token_accuracy': '0.7711', 'epoch': '0.05241'}
{'loss': '0.7969', 'grad_norm': '0.3579', 'learning_rate': '0.0001104', 'entropy': '0.8686', 'num_tokens': '3.201e+07', 'mean_token_accuracy': '0.7839', 'epoch': '0.05242'}
{'loss': '0.842', 'grad_norm': '0.4291', 'learning_rate': '0.0001104', 'entropy': '0.7692', 'num_tokens': '3.202e+07', 'mean_token_accuracy': '0.8016', 'epoch': '0.05243'}
{'loss': '0.6879', 'grad_norm': '0.3805', 'learning_rate': '0.0001103', 'entropy': '0.8005', 'num_tokens': '3.202e+07', 'mean_token_accuracy': '0.7991', 'epoch': '0.05244'}

 49%|████▉     | 5926/12000 [3:31:02<3:12:54,  1.91s/it]
 49%|████▉     | 5927/12000 [3:31:04<3:06:28,  1.84s/it]
                                                        

 49%|████▉     | 5927/12000 [3:31:04<3:06:28,  1.84s/it]
 49%|████▉     | 5928/12000 [3:31:06<3:05:19,  1.83s/it]
                                                        

 49%|████▉     | 5928/12000 [3:31:06<3:05:19,  1.83s/it]
 49%|████▉     | 5929/12000 [3:31:07<2:54:39,  1.73s/it]
                                                        

 49%|████▉     | 5929/12000 [3:31:07<2:54:39,  1.73s/it]
 49%|████▉     | 5930/12000 [3:31:09<2:42:33,  1.61s/it]
                                                        

 49%|████▉     | 5930/12000 [3:31:09<2:42:33,  1.61s/it]
 49%|████▉     | 5931/12000 [3:31:10<2:35:13,  1.53s/it]
                                                        

 49%|████▉     | 5931/12000 [3:31:10<2:35:13,  1.53s/it]
 49%|████▉     | 5932/12000 [3:31:14<3:35:26,  2.13s/it]
                                                        

 49%|████▉     | 5932/12000 [3:31:14<3:35:26,  2.13s/it]
 49%|████▉     | 5933/12000 [3:31:16<3:37:13,  2.15s/it]
                                                        

 49%|████▉     | 5933/12000 [3:31:16<3:37:13,  2.15s/it]
 49%|████▉     | 5934/12000 [3:31:18<3:33:42,  2.11s/it]
                                                        

 49%|████▉     | 5934/12000 [3:31:18<3:33:42,  2.11s/it]
 49%|████▉     | 5935/12000 [3:31:20<3:40:22,  2.18s/it]
                                                        

 49%|████▉     | 5935/12000 [3:31:20<3:40:22,  2.18s/it]
 49%|████▉     | 5936/12000 [3:31:24<4:20:40,  2.58s/it]
                                                        

 49%|████▉     | 5936/12000 [3:31:24<4:20:40,  2.58s/it]
 49%|████▉     | 5937/12000 [3:31:25<3:43:04,  2.21s/it]
                                                        

 49%|████▉     | 5937/12000 [3:31:25<3:43:04,  2.21s/it]
 49%|████▉     | 5938/12000 [3:31:29<4:45:11,  2.82s/it]
                                                        

 49%|████▉     | 5938/12000 [3:31:29<4:45:11,  2.82s/it]
 49%|████▉     | 5939/12000 [3:31:32<4:48:51,  2.86s/it]
                                                        

 49%|████▉     | 5939/12000 [3:31:32<4:48:51,  2.86s/it]
 50%|████▉     | 5940/12000 [3:31:34<4:20:09,  2.58s/it]
                                                        

 50%|████▉     | 5940/12000 [3:31:34<4:20:09,  2.58s/it]
 50%|████▉     | 5941/12000 [3:31:36<3:45:53,  2.24s/it]
                                                        

 50%|████▉     | 5941/12000 [3:31:36<3:45:53,  2.24s/it]
 50%|████▉     | 5942/12000 [3:31:37<3:23:21,  2.01s/it]
                                                        

 50%|████▉     | 5942/12000 [3:31:37<3:23:21,  2.01s/it]
 50%|████▉     | 5943/12000 [3:31:41<4:25:21,  2.63s/it]
                                                        

 50%|████▉     | 5943/12000 [3:31:41<4:25:21,  2.63s/it]
 50%|████▉     | 5944/12000 [3:31:43<3:50:09,  2.28s/it]
                                                        

 50%|████▉     | 5944/12000 [3:31:43<3:50:09,  2.28s/it]
 50%|████▉     | 5945/12000 [3:31:44<3:33:25,  2.11s/it]
                                                        

 50%|████▉     | 5945/12000 [3:31:44<3:33:25,  2.11s/it]
 50%|████▉     | 5946/12000 [3:31:46<3:21:11,  1.99s/it]
                                                        

 50%|████▉     | 5946/12000 [3:31:46<3:21:11,  1.99s/it]
 50%|████▉     | 5947/12000 [3:31:48<3:03:44,  1.82s/it]
                                                        

 50%|████▉     | 5947/12000 [3:31:48<3:03:44,  1.82s/it]
 50%|████▉     | 5948/12000 [3:31:49<2:46:49,  1.65s/it]
                                                        

 50%|████▉     | 5948/12000 [3:31:49<2:46:49,  1.65s/it]
 50%|████▉     | 5949/12000 [3:31:51<2:53:53,  1.72s/it]
                                                        

 50%|████▉     | 5949/12000 [3:31:51<2:53:53,  1.72s/it]
 50%|████▉     | 5950/12000 [3:31:52<2:49:09,  1.68s/it]
                                                        

 50%|████▉     | 5950/12000 [3:31:52<2:49:09,  1.68s/it]
 50%|████▉     | 5951/12000 [3:31:56<3:38:31,  2.17s/it]
                                                        

 50%|████▉     | 5951/12000 [3:31:56<3:38:31,  2.17s/it]
 50%|████▉     | 5952/12000 [3:31:59<4:13:05,  2.51s/it]
                                                        

 50%|████▉     | 5952/12000 [3:31:59<4:13:05,  2.51s/it]
 50%|████▉     | 5953/12000 [3:32:00<3:40:34,  2.19s/it]
                                                        

 50%|████▉     | 5953/12000 [3:32:00<3:40:34,  2.19s/it]
 50%|████▉     | 5954/12000 [3:32:04<4:15:49,  2.54s/it]
                                                        

 50%|████▉     | 5954/12000 [3:32:04<4:15:49,  2.54s/it]
 50%|████▉     | 5955/12000 [3:32:05<3:50:51,  2.29s/it]
                                                        

 50%|████▉     | 5955/12000 [3:32:05<3:50:51,  2.29s/it]
 50%|████▉     | 5956/12000 [3:32:07<3:28:55,  2.07s/it]
                                                        

 50%|████▉     | 5956/12000 [3:32:07<3:28:55,  2.07s/it]
 50%|████▉     | 5957/12000 [3:32:09<3:17:00,  1.96s/it]
                                                        

 50%|████▉     | 5957/12000 [3:32:09<3:17:00,  1.96s/it]
 50%|████▉     | 5958/12000 [3:32:12<4:00:19,  2.39s/it]
                                                        

 50%|████▉     | 5958/12000 [3:32:12<4:00:19,  2.39s/it]
 50%|████▉     | 5959/12000 [3:32:14<3:41:14,  2.20s/it]
                                                        

 50%|████▉     | 5959/12000 [3:32:14<3:41:14,  2.20s/it]
 50%|████▉     | 5960/12000 [3:32:16<3:43:55,  2.22s/it]
                                                        

 50%|████▉     | 5960/12000 [3:32:16<3:43:55,  2.22s/it]
 50%|████▉     | 5961/12000 [3:32:19<3:54:00,  2.33s/it]
                                                        

 50%|████▉     | 5961/12000 [3:32:19<3:54:00,  2.33s/it]
 50%|████▉     | 5962/12000 [3:32:20<3:24:34,  2.03s/it]
                                                        

 50%|████▉     | 5962/12000 [3:32:20<3:24:34,  2.03s/it]
 50%|████▉     | 5963/12000 [3:32:22<3:39:15,  2.18s/it]
                                                        

 50%|████▉     | 5963/12000 [3:32:22<3:39:15,  2.18s/it]
 50%|████▉     | 5964/12000 [3:32:25<3:42:46,  2.21s/it]
                                                        

 50%|████▉     | 5964/12000 [3:32:25<3:42:46,  2.21s/it]
 50%|████▉     | 5965/12000 [3:32:27<3:55:49,  2.34s/it]
                                                        

 50%|████▉     | 5965/12000 [3:32:27<3:55:49,  2.34s/it]
 50%|████▉     | 5966/12000 [3:32:29<3:45:09,  2.24s/it]
                                                        

 50%|████▉     | 5966/12000 [3:32:29<3:45:09,  2.24s/it]
 50%|████▉     | 5967/12000 [3:32:31<3:21:58,  2.01s/it]
                                                        

 50%|████▉     | 5967/12000 [3:32:31<3:21:58,  2.01s/it]
 50%|████▉     | 5968/12000 [3:32:33<3:25:37,  2.05s/it]
                                                        

 50%|████▉     | 5968/12000 [3:32:33<3:25:37,  2.05s/it]
 50%|████▉     | 5969/12000 [3:32:35<3:29:40,  2.09s/it]
                                                        

 50%|████▉     | 5969/12000 [3:32:35<3:29:40,  2.09s/it]
 50%|████▉     | 5970/12000 [3:32:37<3:25:56,  2.05s/it]
                                                        

 50%|████▉     | 5970/12000 [3:32:37<3:25:56,  2.05s/it]
 50%|████▉     | 5971/12000 [3:32:39<3:12:49,  1.92s/it]
                                                        

 50%|████▉     | 5971/12000 [3:32:39<3:12:49,  1.92s/it]
 50%|████▉     | 5972/12000 [3:32:40<3:00:39,  1.80s/it]
                                                        

 50%|████▉     | 5972/12000 [3:32:40<3:00:39,  1.80s/it]
 50%|████▉     | 5973/12000 [3:32:43<3:27:25,  2.06s/it]
                                                        
{'loss': '0.723', 'grad_norm': '0.3462', 'learning_rate': '0.0001103', 'entropy': '0.7401', 'num_tokens': '3.203e+07', 'mean_token_accuracy': '0.8291', 'epoch': '0.05244'}
{'loss': '0.9274', 'grad_norm': '0.4806', 'learning_rate': '0.0001103', 'entropy': '0.8178', 'num_tokens': '3.203e+07', 'mean_token_accuracy': '0.7861', 'epoch': '0.05245'}
{'loss': '1.055', 'grad_norm': '0.4112', 'learning_rate': '0.0001103', 'entropy': '1.003', 'num_tokens': '3.204e+07', 'mean_token_accuracy': '0.7493', 'epoch': '0.05246'}
{'loss': '0.7371', 'grad_norm': '0.3863', 'learning_rate': '0.0001102', 'entropy': '0.7505', 'num_tokens': '3.204e+07', 'mean_token_accuracy': '0.7985', 'epoch': '0.05247'}
{'loss': '1.091', 'grad_norm': '0.4402', 'learning_rate': '0.0001102', 'entropy': '0.9558', 'num_tokens': '3.205e+07', 'mean_token_accuracy': '0.7602', 'epoch': '0.05248'}
{'loss': '0.4933', 'grad_norm': '0.4086', 'learning_rate': '0.0001102', 'entropy': '0.5514', 'num_tokens': '3.205e+07', 'mean_token_accuracy': '0.8524', 'epoch': '0.05249'}
{'loss': '0.7699', 'grad_norm': '0.3767', 'learning_rate': '0.0001102', 'entropy': '0.8178', 'num_tokens': '3.206e+07', 'mean_token_accuracy': '0.7848', 'epoch': '0.0525'}
{'loss': '1.099', 'grad_norm': '0.4379', 'learning_rate': '0.0001101', 'entropy': '0.9931', 'num_tokens': '3.206e+07', 'mean_token_accuracy': '0.7363', 'epoch': '0.05251'}
{'loss': '0.9899', 'grad_norm': '0.4098', 'learning_rate': '0.0001101', 'entropy': '0.9518', 'num_tokens': '3.207e+07', 'mean_token_accuracy': '0.7695', 'epoch': '0.05251'}
{'loss': '0.7224', 'grad_norm': '0.3923', 'learning_rate': '0.0001101', 'entropy': '0.8027', 'num_tokens': '3.207e+07', 'mean_token_accuracy': '0.7964', 'epoch': '0.05252'}
{'loss': '0.6945', 'grad_norm': '0.3285', 'learning_rate': '0.00011', 'entropy': '0.7345', 'num_tokens': '3.208e+07', 'mean_token_accuracy': '0.8043', 'epoch': '0.05253'}
{'loss': '0.6549', 'grad_norm': '0.3968', 'learning_rate': '0.00011', 'entropy': '0.706', 'num_tokens': '3.208e+07', 'mean_token_accuracy': '0.8228', 'epoch': '0.05254'}
{'loss': '1.253', 'grad_norm': '0.4057', 'learning_rate': '0.00011', 'entropy': '1.363', 'num_tokens': '3.209e+07', 'mean_token_accuracy': '0.6864', 'epoch': '0.05255'}
{'loss': '1.061', 'grad_norm': '0.3798', 'learning_rate': '0.00011', 'entropy': '1.064', 'num_tokens': '3.21e+07', 'mean_token_accuracy': '0.7286', 'epoch': '0.05256'}
{'loss': '0.7048', 'grad_norm': '0.3296', 'learning_rate': '0.0001099', 'entropy': '0.6851', 'num_tokens': '3.211e+07', 'mean_token_accuracy': '0.835', 'epoch': '0.05257'}
{'loss': '0.8823', 'grad_norm': '0.3572', 'learning_rate': '0.0001099', 'entropy': '0.9902', 'num_tokens': '3.211e+07', 'mean_token_accuracy': '0.7609', 'epoch': '0.05258'}
{'loss': '1.048', 'grad_norm': '0.4262', 'learning_rate': '0.0001099', 'entropy': '1.077', 'num_tokens': '3.211e+07', 'mean_token_accuracy': '0.7275', 'epoch': '0.05259'}
{'loss': '0.9406', 'grad_norm': '0.3066', 'learning_rate': '0.0001098', 'entropy': '1.042', 'num_tokens': '3.212e+07', 'mean_token_accuracy': '0.7502', 'epoch': '0.05259'}
{'loss': '0.7802', 'grad_norm': '0.43', 'learning_rate': '0.0001098', 'entropy': '0.8159', 'num_tokens': '3.213e+07', 'mean_token_accuracy': '0.79', 'epoch': '0.0526'}
{'loss': '0.9326', 'grad_norm': '0.4244', 'learning_rate': '0.0001098', 'entropy': '0.8596', 'num_tokens': '3.213e+07', 'mean_token_accuracy': '0.7821', 'epoch': '0.05261'}
{'loss': '1.02', 'grad_norm': '0.4573', 'learning_rate': '0.0001098', 'entropy': '0.9384', 'num_tokens': '3.214e+07', 'mean_token_accuracy': '0.781', 'epoch': '0.05262'}
{'loss': '0.7506', 'grad_norm': '0.3932', 'learning_rate': '0.0001097', 'entropy': '0.7361', 'num_tokens': '3.214e+07', 'mean_token_accuracy': '0.8064', 'epoch': '0.05263'}
{'loss': '0.5962', 'grad_norm': '0.3391', 'learning_rate': '0.0001097', 'entropy': '0.6428', 'num_tokens': '3.215e+07', 'mean_token_accuracy': '0.8414', 'epoch': '0.05264'}
{'loss': '0.7354', 'grad_norm': '0.3788', 'learning_rate': '0.0001097', 'entropy': '0.6865', 'num_tokens': '3.215e+07', 'mean_token_accuracy': '0.8222', 'epoch': '0.05265'}
{'loss': '0.9145', 'grad_norm': '0.4139', 'learning_rate': '0.0001097', 'entropy': '0.9494', 'num_tokens': '3.216e+07', 'mean_token_accuracy': '0.7499', 'epoch': '0.05266'}
{'loss': '0.8934', 'grad_norm': '0.3592', 'learning_rate': '0.0001096', 'entropy': '1.026', 'num_tokens': '3.216e+07', 'mean_token_accuracy': '0.7512', 'epoch': '0.05267'}
{'loss': '0.9848', 'grad_norm': '0.5271', 'learning_rate': '0.0001096', 'entropy': '0.8992', 'num_tokens': '3.217e+07', 'mean_token_accuracy': '0.7703', 'epoch': '0.05267'}
{'loss': '0.8066', 'grad_norm': '0.3505', 'learning_rate': '0.0001096', 'entropy': '0.8562', 'num_tokens': '3.217e+07', 'mean_token_accuracy': '0.7921', 'epoch': '0.05268'}
{'loss': '0.8627', 'grad_norm': '0.4012', 'learning_rate': '0.0001095', 'entropy': '0.9649', 'num_tokens': '3.218e+07', 'mean_token_accuracy': '0.7639', 'epoch': '0.05269'}
{'loss': '0.6205', 'grad_norm': '0.4133', 'learning_rate': '0.0001095', 'entropy': '0.589', 'num_tokens': '3.218e+07', 'mean_token_accuracy': '0.8449', 'epoch': '0.0527'}
{'loss': '0.9349', 'grad_norm': '0.4312', 'learning_rate': '0.0001095', 'entropy': '0.9054', 'num_tokens': '3.219e+07', 'mean_token_accuracy': '0.778', 'epoch': '0.05271'}
{'loss': '0.7374', 'grad_norm': '0.438', 'learning_rate': '0.0001095', 'entropy': '0.7981', 'num_tokens': '3.219e+07', 'mean_token_accuracy': '0.7935', 'epoch': '0.05272'}
{'loss': '0.7838', 'grad_norm': '0.3381', 'learning_rate': '0.0001094', 'entropy': '0.7885', 'num_tokens': '3.22e+07', 'mean_token_accuracy': '0.7905', 'epoch': '0.05273'}
{'loss': '0.9009', 'grad_norm': '0.4055', 'learning_rate': '0.0001094', 'entropy': '1.045', 'num_tokens': '3.22e+07', 'mean_token_accuracy': '0.7382', 'epoch': '0.05274'}
{'loss': '1.015', 'grad_norm': '0.4671', 'learning_rate': '0.0001094', 'entropy': '0.9248', 'num_tokens': '3.221e+07', 'mean_token_accuracy': '0.7585', 'epoch': '0.05274'}
{'loss': '1.071', 'grad_norm': '0.4128', 'learning_rate': '0.0001094', 'entropy': '1.032', 'num_tokens': '3.222e+07', 'mean_token_accuracy': '0.7341', 'epoch': '0.05275'}
{'loss': '0.8689', 'grad_norm': '0.3828', 'learning_rate': '0.0001093', 'entropy': '0.8728', 'num_tokens': '3.222e+07', 'mean_token_accuracy': '0.7745', 'epoch': '0.05276'}
{'loss': '0.9636', 'grad_norm': '0.4599', 'learning_rate': '0.0001093', 'entropy': '0.9159', 'num_tokens': '3.223e+07', 'mean_token_accuracy': '0.7733', 'epoch': '0.05277'}
{'loss': '0.9677', 'grad_norm': '0.3838', 'learning_rate': '0.0001093', 'entropy': '0.9422', 'num_tokens': '3.223e+07', 'mean_token_accuracy': '0.7604', 'epoch': '0.05278'}
{'loss': '1.031', 'grad_norm': '0.4483', 'learning_rate': '0.0001092', 'entropy': '1.102', 'num_tokens': '3.224e+07', 'mean_token_accuracy': '0.7343', 'epoch': '0.05279'}
{'loss': '1.353', 'grad_norm': '0.4756', 'learning_rate': '0.0001092', 'entropy': '1.235', 'num_tokens': '3.225e+07', 'mean_token_accuracy': '0.6946', 'epoch': '0.0528'}
{'loss': '0.6895', 'grad_norm': '0.3845', 'learning_rate': '0.0001092', 'entropy': '0.7049', 'num_tokens': '3.225e+07', 'mean_token_accuracy': '0.8235', 'epoch': '0.05281'}
{'loss': '0.7669', 'grad_norm': '0.3003', 'learning_rate': '0.0001092', 'entropy': '0.7009', 'num_tokens': '3.226e+07', 'mean_token_accuracy': '0.8121', 'epoch': '0.05282'}
{'loss': '0.6583', 'grad_norm': '0.329', 'learning_rate': '0.0001091', 'entropy': '0.6149', 'num_tokens': '3.226e+07', 'mean_token_accuracy': '0.8437', 'epoch': '0.05282'}
{'loss': '0.9278', 'grad_norm': '0.3972', 'learning_rate': '0.0001091', 'entropy': '0.8964', 'num_tokens': '3.227e+07', 'mean_token_accuracy': '0.786', 'epoch': '0.05283'}
{'loss': '1.117', 'grad_norm': '0.3893', 'learning_rate': '0.0001091', 'entropy': '1.033', 'num_tokens': '3.227e+07', 'mean_token_accuracy': '0.7463', 'epoch': '0.05284'}
{'loss': '0.9861', 'grad_norm': '0.4639', 'learning_rate': '0.0001091', 'entropy': '1.054', 'num_tokens': '3.228e+07', 'mean_token_accuracy': '0.7409', 'epoch': '0.05285'}

 50%|████▉     | 5973/12000 [3:32:43<3:27:25,  2.06s/it]
 50%|████▉     | 5974/12000 [3:32:44<3:09:23,  1.89s/it]
                                                        

 50%|████▉     | 5974/12000 [3:32:44<3:09:23,  1.89s/it]
 50%|████▉     | 5975/12000 [3:32:46<2:52:23,  1.72s/it]
                                                        

 50%|████▉     | 5975/12000 [3:32:46<2:52:23,  1.72s/it]
 50%|████▉     | 5976/12000 [3:32:51<4:44:43,  2.84s/it]
                                                        

 50%|████▉     | 5976/12000 [3:32:51<4:44:43,  2.84s/it]
 50%|████▉     | 5977/12000 [3:32:54<4:47:25,  2.86s/it]
                                                        

 50%|████▉     | 5977/12000 [3:32:54<4:47:25,  2.86s/it]
 50%|████▉     | 5978/12000 [3:32:56<4:01:30,  2.41s/it]
                                                        

 50%|████▉     | 5978/12000 [3:32:56<4:01:30,  2.41s/it]
 50%|████▉     | 5979/12000 [3:32:57<3:45:51,  2.25s/it]
                                                        

 50%|████▉     | 5979/12000 [3:32:57<3:45:51,  2.25s/it]
 50%|████▉     | 5980/12000 [3:32:59<3:23:46,  2.03s/it]
                                                        

 50%|████▉     | 5980/12000 [3:32:59<3:23:46,  2.03s/it]
 50%|████▉     | 5981/12000 [3:33:01<3:29:29,  2.09s/it]
                                                        

 50%|████▉     | 5981/12000 [3:33:01<3:29:29,  2.09s/it]
 50%|████▉     | 5982/12000 [3:33:03<3:31:27,  2.11s/it]
                                                        

 50%|████▉     | 5982/12000 [3:33:03<3:31:27,  2.11s/it]
 50%|████▉     | 5983/12000 [3:33:05<3:09:16,  1.89s/it]
                                                        

 50%|████▉     | 5983/12000 [3:33:05<3:09:16,  1.89s/it]
 50%|████▉     | 5984/12000 [3:33:06<2:52:15,  1.72s/it]
                                                        

 50%|████▉     | 5984/12000 [3:33:06<2:52:15,  1.72s/it]
 50%|████▉     | 5985/12000 [3:33:09<3:25:41,  2.05s/it]
                                                        

 50%|████▉     | 5985/12000 [3:33:09<3:25:41,  2.05s/it]
 50%|████▉     | 5986/12000 [3:33:11<3:37:15,  2.17s/it]
                                                        

 50%|████▉     | 5986/12000 [3:33:11<3:37:15,  2.17s/it]
 50%|████▉     | 5987/12000 [3:33:13<3:30:56,  2.10s/it]
                                                        

 50%|████▉     | 5987/12000 [3:33:13<3:30:56,  2.10s/it]
 50%|████▉     | 5988/12000 [3:33:15<3:18:33,  1.98s/it]
                                                        

 50%|████▉     | 5988/12000 [3:33:15<3:18:33,  1.98s/it]
 50%|████▉     | 5989/12000 [3:33:16<3:05:51,  1.86s/it]
                                                        

 50%|████▉     | 5989/12000 [3:33:16<3:05:51,  1.86s/it]
 50%|████▉     | 5990/12000 [3:33:18<3:00:15,  1.80s/it]
                                                        

 50%|████▉     | 5990/12000 [3:33:18<3:00:15,  1.80s/it]
 50%|████▉     | 5991/12000 [3:33:20<2:54:17,  1.74s/it]
                                                        

 50%|████▉     | 5991/12000 [3:33:20<2:54:17,  1.74s/it]
 50%|████▉     | 5992/12000 [3:33:25<4:37:22,  2.77s/it]
                                                        

 50%|████▉     | 5992/12000 [3:33:25<4:37:22,  2.77s/it]
 50%|████▉     | 5993/12000 [3:33:27<4:14:09,  2.54s/it]
                                                        

 50%|████▉     | 5993/12000 [3:33:27<4:14:09,  2.54s/it]
 50%|████▉     | 5994/12000 [3:33:29<3:52:06,  2.32s/it]
                                                        

 50%|████▉     | 5994/12000 [3:33:29<3:52:06,  2.32s/it]
 50%|████▉     | 5995/12000 [3:33:30<3:31:55,  2.12s/it]
                                                        

 50%|████▉     | 5995/12000 [3:33:30<3:31:55,  2.12s/it]
 50%|████▉     | 5996/12000 [3:33:32<3:31:17,  2.11s/it]
                                                        

 50%|████▉     | 5996/12000 [3:33:32<3:31:17,  2.11s/it]
 50%|████▉     | 5997/12000 [3:33:35<3:51:26,  2.31s/it]
                                                        

 50%|████▉     | 5997/12000 [3:33:35<3:51:26,  2.31s/it]
 50%|████▉     | 5998/12000 [3:33:40<5:11:47,  3.12s/it]
                                                        

 50%|████▉     | 5998/12000 [3:33:40<5:11:47,  3.12s/it]
 50%|████▉     | 5999/12000 [3:33:42<4:26:00,  2.66s/it]
                                                        

 50%|████▉     | 5999/12000 [3:33:42<4:26:00,  2.66s/it]
 50%|█████     | 6000/12000 [3:33:43<3:47:59,  2.28s/it]
                                                        

 50%|█████     | 6000/12000 [3:33:43<3:47:59,  2.28s/it]
 50%|█████     | 6001/12000 [3:33:46<4:01:43,  2.42s/it]
                                                        

 50%|█████     | 6001/12000 [3:33:46<4:01:43,  2.42s/it]
 50%|█████     | 6002/12000 [3:33:48<3:36:00,  2.16s/it]
                                                        

 50%|█████     | 6002/12000 [3:33:48<3:36:00,  2.16s/it]
 50%|█████     | 6003/12000 [3:33:50<3:41:12,  2.21s/it]
                                                        

 50%|█████     | 6003/12000 [3:33:50<3:41:12,  2.21s/it]
 50%|█████     | 6004/12000 [3:33:51<3:16:17,  1.96s/it]
                                                        

 50%|█████     | 6004/12000 [3:33:51<3:16:17,  1.96s/it]
 50%|█████     | 6005/12000 [3:33:53<2:57:31,  1.78s/it]
                                                        

 50%|█████     | 6005/12000 [3:33:53<2:57:31,  1.78s/it]
 50%|█████     | 6006/12000 [3:33:54<2:47:27,  1.68s/it]
                                                        

 50%|█████     | 6006/12000 [3:33:54<2:47:27,  1.68s/it]
 50%|█████     | 6007/12000 [3:33:56<2:59:38,  1.80s/it]
                                                        

 50%|█████     | 6007/12000 [3:33:56<2:59:38,  1.80s/it]
 50%|█████     | 6008/12000 [3:33:57<2:46:07,  1.66s/it]
                                                        

 50%|█████     | 6008/12000 [3:33:57<2:46:07,  1.66s/it]
 50%|█████     | 6009/12000 [3:34:00<2:59:57,  1.80s/it]
                                                        

 50%|█████     | 6009/12000 [3:34:00<2:59:57,  1.80s/it]
 50%|█████     | 6010/12000 [3:34:02<3:10:00,  1.90s/it]
                                                        

 50%|█████     | 6010/12000 [3:34:02<3:10:00,  1.90s/it]
 50%|█████     | 6011/12000 [3:34:03<2:57:37,  1.78s/it]
                                                        

 50%|█████     | 6011/12000 [3:34:03<2:57:37,  1.78s/it]
 50%|█████     | 6012/12000 [3:34:05<2:58:02,  1.78s/it]
                                                        

 50%|█████     | 6012/12000 [3:34:05<2:58:02,  1.78s/it]
 50%|█████     | 6013/12000 [3:34:07<2:52:11,  1.73s/it]
                                                        

 50%|█████     | 6013/12000 [3:34:07<2:52:11,  1.73s/it]
 50%|█████     | 6014/12000 [3:34:08<2:46:24,  1.67s/it]
                                                        

 50%|█████     | 6014/12000 [3:34:08<2:46:24,  1.67s/it]
 50%|█████     | 6015/12000 [3:34:09<2:36:31,  1.57s/it]
                                                        

 50%|█████     | 6015/12000 [3:34:09<2:36:31,  1.57s/it]
 50%|█████     | 6016/12000 [3:34:11<2:35:41,  1.56s/it]
                                                        

 50%|█████     | 6016/12000 [3:34:11<2:35:41,  1.56s/it]
 50%|█████     | 6017/12000 [3:34:13<2:43:43,  1.64s/it]
                                                        

 50%|█████     | 6017/12000 [3:34:13<2:43:43,  1.64s/it]
 50%|█████     | 6018/12000 [3:34:15<2:45:16,  1.66s/it]
                                                        

 50%|█████     | 6018/12000 [3:34:15<2:45:16,  1.66s/it]
 50%|█████     | 6019/12000 [3:34:18<3:34:33,  2.15s/it]
                                                        

 50%|█████     | 6019/12000 [3:34:18<3:34:33,  2.15s/it]
 50%|█████     | 6020/12000 [3:34:21<4:06:56,  2.48s/it]
                                                        
{'loss': '0.8665', 'grad_norm': '0.3313', 'learning_rate': '0.000109', 'entropy': '1.029', 'num_tokens': '3.229e+07', 'mean_token_accuracy': '0.7527', 'epoch': '0.05286'}
{'loss': '0.7229', 'grad_norm': '0.2828', 'learning_rate': '0.000109', 'entropy': '0.7365', 'num_tokens': '3.229e+07', 'mean_token_accuracy': '0.8027', 'epoch': '0.05287'}
{'loss': '0.7573', 'grad_norm': '0.4152', 'learning_rate': '0.000109', 'entropy': '0.7834', 'num_tokens': '3.229e+07', 'mean_token_accuracy': '0.8153', 'epoch': '0.05288'}
{'loss': '0.968', 'grad_norm': '0.3929', 'learning_rate': '0.0001089', 'entropy': '0.9791', 'num_tokens': '3.23e+07', 'mean_token_accuracy': '0.7584', 'epoch': '0.05289'}
{'loss': '0.7528', 'grad_norm': '0.2916', 'learning_rate': '0.0001089', 'entropy': '0.9027', 'num_tokens': '3.231e+07', 'mean_token_accuracy': '0.7896', 'epoch': '0.0529'}
{'loss': '0.6916', 'grad_norm': '0.401', 'learning_rate': '0.0001089', 'entropy': '0.861', 'num_tokens': '3.231e+07', 'mean_token_accuracy': '0.7836', 'epoch': '0.0529'}
{'loss': '1.244', 'grad_norm': '0.4393', 'learning_rate': '0.0001089', 'entropy': '1.117', 'num_tokens': '3.232e+07', 'mean_token_accuracy': '0.7424', 'epoch': '0.05291'}
{'loss': '1.278', 'grad_norm': '0.4005', 'learning_rate': '0.0001088', 'entropy': '1.319', 'num_tokens': '3.232e+07', 'mean_token_accuracy': '0.6947', 'epoch': '0.05292'}
{'loss': '0.9774', 'grad_norm': '0.3144', 'learning_rate': '0.0001088', 'entropy': '1.032', 'num_tokens': '3.233e+07', 'mean_token_accuracy': '0.7497', 'epoch': '0.05293'}
{'loss': '1.11', 'grad_norm': '0.4564', 'learning_rate': '0.0001088', 'entropy': '1.081', 'num_tokens': '3.233e+07', 'mean_token_accuracy': '0.737', 'epoch': '0.05294'}
{'loss': '0.8235', 'grad_norm': '0.3581', 'learning_rate': '0.0001088', 'entropy': '0.8231', 'num_tokens': '3.234e+07', 'mean_token_accuracy': '0.788', 'epoch': '0.05295'}
{'loss': '0.7548', 'grad_norm': '0.375', 'learning_rate': '0.0001087', 'entropy': '0.7746', 'num_tokens': '3.234e+07', 'mean_token_accuracy': '0.7966', 'epoch': '0.05296'}
{'loss': '0.8648', 'grad_norm': '0.4177', 'learning_rate': '0.0001087', 'entropy': '0.84', 'num_tokens': '3.235e+07', 'mean_token_accuracy': '0.7956', 'epoch': '0.05297'}
{'loss': '0.9719', 'grad_norm': '0.5038', 'learning_rate': '0.0001087', 'entropy': '0.8662', 'num_tokens': '3.236e+07', 'mean_token_accuracy': '0.7762', 'epoch': '0.05298'}
{'loss': '1.202', 'grad_norm': '0.3941', 'learning_rate': '0.0001086', 'entropy': '1.168', 'num_tokens': '3.236e+07', 'mean_token_accuracy': '0.7457', 'epoch': '0.05298'}
{'loss': '0.7576', 'grad_norm': '0.4035', 'learning_rate': '0.0001086', 'entropy': '0.7481', 'num_tokens': '3.237e+07', 'mean_token_accuracy': '0.8048', 'epoch': '0.05299'}
{'loss': '0.681', 'grad_norm': '0.3519', 'learning_rate': '0.0001086', 'entropy': '0.6846', 'num_tokens': '3.237e+07', 'mean_token_accuracy': '0.8365', 'epoch': '0.053'}
{'loss': '1.054', 'grad_norm': '0.4047', 'learning_rate': '0.0001086', 'entropy': '1.015', 'num_tokens': '3.238e+07', 'mean_token_accuracy': '0.7417', 'epoch': '0.05301'}
{'loss': '1.032', 'grad_norm': '0.3614', 'learning_rate': '0.0001085', 'entropy': '1.028', 'num_tokens': '3.238e+07', 'mean_token_accuracy': '0.7481', 'epoch': '0.05302'}
{'loss': '1.199', 'grad_norm': '0.3417', 'learning_rate': '0.0001085', 'entropy': '1.027', 'num_tokens': '3.239e+07', 'mean_token_accuracy': '0.7504', 'epoch': '0.05303'}
{'loss': '0.898', 'grad_norm': '0.42', 'learning_rate': '0.0001085', 'entropy': '0.8655', 'num_tokens': '3.24e+07', 'mean_token_accuracy': '0.7691', 'epoch': '0.05304'}
{'loss': '0.9109', 'grad_norm': '0.4174', 'learning_rate': '0.0001085', 'entropy': '0.863', 'num_tokens': '3.241e+07', 'mean_token_accuracy': '0.7848', 'epoch': '0.05305'}
{'loss': '0.7577', 'grad_norm': '0.3898', 'learning_rate': '0.0001084', 'entropy': '0.8317', 'num_tokens': '3.241e+07', 'mean_token_accuracy': '0.8045', 'epoch': '0.05305'}
{'loss': '1.062', 'grad_norm': '0.4049', 'learning_rate': '0.0001084', 'entropy': '0.9289', 'num_tokens': '3.242e+07', 'mean_token_accuracy': '0.7987', 'epoch': '0.05306'}
{'loss': '0.6374', 'grad_norm': '0.3452', 'learning_rate': '0.0001084', 'entropy': '0.6209', 'num_tokens': '3.242e+07', 'mean_token_accuracy': '0.8361', 'epoch': '0.05307'}
{'loss': '0.9395', 'grad_norm': '0.3418', 'learning_rate': '0.0001083', 'entropy': '0.9454', 'num_tokens': '3.243e+07', 'mean_token_accuracy': '0.7621', 'epoch': '0.05308'}
{'loss': '0.9061', 'grad_norm': '0.5676', 'learning_rate': '0.0001083', 'entropy': '0.9136', 'num_tokens': '3.244e+07', 'mean_token_accuracy': '0.7722', 'epoch': '0.05309'}
{'loss': '0.8768', 'grad_norm': '0.4384', 'learning_rate': '0.0001083', 'entropy': '0.8699', 'num_tokens': '3.244e+07', 'mean_token_accuracy': '0.7733', 'epoch': '0.0531'}
{'loss': '0.9889', 'grad_norm': '0.3755', 'learning_rate': '0.0001083', 'entropy': '1.005', 'num_tokens': '3.244e+07', 'mean_token_accuracy': '0.7608', 'epoch': '0.05311'}
{'loss': '1.119', 'grad_norm': '0.4145', 'learning_rate': '0.0001082', 'entropy': '1.102', 'num_tokens': '3.245e+07', 'mean_token_accuracy': '0.7432', 'epoch': '0.05312'}
{'loss': '0.9154', 'grad_norm': '0.3473', 'learning_rate': '0.0001082', 'entropy': '0.9063', 'num_tokens': '3.246e+07', 'mean_token_accuracy': '0.7815', 'epoch': '0.05313'}
{'loss': '0.8605', 'grad_norm': '0.4012', 'learning_rate': '0.0001082', 'entropy': '0.8849', 'num_tokens': '3.246e+07', 'mean_token_accuracy': '0.7781', 'epoch': '0.05313'}
{'loss': '1.017', 'grad_norm': '0.4405', 'learning_rate': '0.0001081', 'entropy': '1.056', 'num_tokens': '3.246e+07', 'mean_token_accuracy': '0.7436', 'epoch': '0.05314'}
{'loss': '0.9911', 'grad_norm': '0.4093', 'learning_rate': '0.0001081', 'entropy': '1.002', 'num_tokens': '3.247e+07', 'mean_token_accuracy': '0.7544', 'epoch': '0.05315'}
{'loss': '1.987', 'grad_norm': '0.4811', 'learning_rate': '0.0001081', 'entropy': '1.435', 'num_tokens': '3.247e+07', 'mean_token_accuracy': '0.6948', 'epoch': '0.05316'}
{'loss': '0.942', 'grad_norm': '0.3937', 'learning_rate': '0.0001081', 'entropy': '0.9996', 'num_tokens': '3.248e+07', 'mean_token_accuracy': '0.7536', 'epoch': '0.05317'}
{'loss': '0.9602', 'grad_norm': '0.4694', 'learning_rate': '0.000108', 'entropy': '0.873', 'num_tokens': '3.248e+07', 'mean_token_accuracy': '0.7812', 'epoch': '0.05318'}
{'loss': '0.5497', 'grad_norm': '0.2579', 'learning_rate': '0.000108', 'entropy': '0.5949', 'num_tokens': '3.249e+07', 'mean_token_accuracy': '0.851', 'epoch': '0.05319'}
{'loss': '0.8064', 'grad_norm': '0.407', 'learning_rate': '0.000108', 'entropy': '0.8508', 'num_tokens': '3.249e+07', 'mean_token_accuracy': '0.7866', 'epoch': '0.0532'}
{'loss': '1.163', 'grad_norm': '0.3937', 'learning_rate': '0.000108', 'entropy': '1.139', 'num_tokens': '3.25e+07', 'mean_token_accuracy': '0.7042', 'epoch': '0.05321'}
{'loss': '0.8356', 'grad_norm': '0.3747', 'learning_rate': '0.0001079', 'entropy': '0.9919', 'num_tokens': '3.25e+07', 'mean_token_accuracy': '0.7511', 'epoch': '0.05321'}
{'loss': '0.9381', 'grad_norm': '0.5237', 'learning_rate': '0.0001079', 'entropy': '0.9945', 'num_tokens': '3.251e+07', 'mean_token_accuracy': '0.7583', 'epoch': '0.05322'}
{'loss': '0.8288', 'grad_norm': '0.3472', 'learning_rate': '0.0001079', 'entropy': '0.7988', 'num_tokens': '3.251e+07', 'mean_token_accuracy': '0.7884', 'epoch': '0.05323'}
{'loss': '0.9623', 'grad_norm': '0.4901', 'learning_rate': '0.0001078', 'entropy': '0.9741', 'num_tokens': '3.251e+07', 'mean_token_accuracy': '0.748', 'epoch': '0.05324'}
{'loss': '0.7773', 'grad_norm': '0.3405', 'learning_rate': '0.0001078', 'entropy': '0.7933', 'num_tokens': '3.252e+07', 'mean_token_accuracy': '0.8013', 'epoch': '0.05325'}
{'loss': '0.7839', 'grad_norm': '0.3757', 'learning_rate': '0.0001078', 'entropy': '0.8094', 'num_tokens': '3.252e+07', 'mean_token_accuracy': '0.7985', 'epoch': '0.05326'}
{'loss': '1.017', 'grad_norm': '0.3466', 'learning_rate': '0.0001078', 'entropy': '0.9398', 'num_tokens': '3.253e+07', 'mean_token_accuracy': '0.7735', 'epoch': '0.05327'}

 50%|█████     | 6020/12000 [3:34:21<4:06:56,  2.48s/it]
 50%|█████     | 6021/12000 [3:34:23<3:56:48,  2.38s/it]
                                                        

 50%|█████     | 6021/12000 [3:34:23<3:56:48,  2.38s/it]
 50%|█████     | 6022/12000 [3:34:25<3:41:17,  2.22s/it]
                                                        

 50%|█████     | 6022/12000 [3:34:25<3:41:17,  2.22s/it]
 50%|█████     | 6023/12000 [3:34:27<3:21:45,  2.03s/it]
                                                        

 50%|█████     | 6023/12000 [3:34:27<3:21:45,  2.03s/it]
 50%|█████     | 6024/12000 [3:34:28<3:04:28,  1.85s/it]
                                                        

 50%|█████     | 6024/12000 [3:34:28<3:04:28,  1.85s/it]
 50%|█████     | 6025/12000 [3:34:29<2:49:29,  1.70s/it]
                                                        

 50%|█████     | 6025/12000 [3:34:29<2:49:29,  1.70s/it]
 50%|█████     | 6026/12000 [3:34:32<3:04:07,  1.85s/it]
                                                        

 50%|█████     | 6026/12000 [3:34:32<3:04:07,  1.85s/it]
 50%|█████     | 6027/12000 [3:34:33<2:53:40,  1.74s/it]
                                                        

 50%|█████     | 6027/12000 [3:34:33<2:53:40,  1.74s/it]
 50%|█████     | 6028/12000 [3:34:35<2:48:21,  1.69s/it]
                                                        

 50%|█████     | 6028/12000 [3:34:35<2:48:21,  1.69s/it]
 50%|█████     | 6029/12000 [3:34:36<2:41:36,  1.62s/it]
                                                        

 50%|█████     | 6029/12000 [3:34:36<2:41:36,  1.62s/it]
 50%|█████     | 6030/12000 [3:34:38<2:50:31,  1.71s/it]
                                                        

 50%|█████     | 6030/12000 [3:34:38<2:50:31,  1.71s/it]
 50%|█████     | 6031/12000 [3:34:40<2:41:44,  1.63s/it]
                                                        

 50%|█████     | 6031/12000 [3:34:40<2:41:44,  1.63s/it]
 50%|█████     | 6032/12000 [3:34:41<2:49:34,  1.70s/it]
                                                        

 50%|█████     | 6032/12000 [3:34:41<2:49:34,  1.70s/it]
 50%|█████     | 6033/12000 [3:34:43<2:42:27,  1.63s/it]
                                                        

 50%|█████     | 6033/12000 [3:34:43<2:42:27,  1.63s/it]
 50%|█████     | 6034/12000 [3:34:44<2:37:14,  1.58s/it]
                                                        

 50%|█████     | 6034/12000 [3:34:44<2:37:14,  1.58s/it]
 50%|█████     | 6035/12000 [3:34:46<2:40:33,  1.62s/it]
                                                        

 50%|█████     | 6035/12000 [3:34:46<2:40:33,  1.62s/it]
 50%|█████     | 6036/12000 [3:34:48<2:45:28,  1.66s/it]
                                                        

 50%|█████     | 6036/12000 [3:34:48<2:45:28,  1.66s/it]
 50%|█████     | 6037/12000 [3:34:50<3:06:36,  1.88s/it]
                                                        

 50%|█████     | 6037/12000 [3:34:50<3:06:36,  1.88s/it]
 50%|█████     | 6038/12000 [3:34:53<3:32:19,  2.14s/it]
                                                        

 50%|█████     | 6038/12000 [3:34:53<3:32:19,  2.14s/it]
 50%|█████     | 6039/12000 [3:34:55<3:21:48,  2.03s/it]
                                                        

 50%|█████     | 6039/12000 [3:34:55<3:21:48,  2.03s/it]
 50%|█████     | 6040/12000 [3:34:56<3:12:32,  1.94s/it]
                                                        

 50%|█████     | 6040/12000 [3:34:56<3:12:32,  1.94s/it]
 50%|█████     | 6041/12000 [3:34:58<3:04:24,  1.86s/it]
                                                        

 50%|█████     | 6041/12000 [3:34:58<3:04:24,  1.86s/it]
 50%|█████     | 6042/12000 [3:35:02<4:14:29,  2.56s/it]
                                                        

 50%|█████     | 6042/12000 [3:35:02<4:14:29,  2.56s/it]
 50%|█████     | 6043/12000 [3:35:05<4:22:13,  2.64s/it]
                                                        

 50%|█████     | 6043/12000 [3:35:05<4:22:13,  2.64s/it]
 50%|█████     | 6044/12000 [3:35:07<3:54:44,  2.36s/it]
                                                        

 50%|█████     | 6044/12000 [3:35:07<3:54:44,  2.36s/it]
 50%|█████     | 6045/12000 [3:35:10<4:18:47,  2.61s/it]
                                                        

 50%|█████     | 6045/12000 [3:35:10<4:18:47,  2.61s/it]
 50%|█████     | 6046/12000 [3:35:12<3:48:05,  2.30s/it]
                                                        

 50%|█████     | 6046/12000 [3:35:12<3:48:05,  2.30s/it]
 50%|█████     | 6047/12000 [3:35:14<3:47:20,  2.29s/it]
                                                        

 50%|█████     | 6047/12000 [3:35:14<3:47:20,  2.29s/it]
 50%|█████     | 6048/12000 [3:35:15<3:21:15,  2.03s/it]
                                                        

 50%|█████     | 6048/12000 [3:35:15<3:21:15,  2.03s/it]
 50%|█████     | 6049/12000 [3:35:17<3:07:58,  1.90s/it]
                                                        

 50%|█████     | 6049/12000 [3:35:17<3:07:58,  1.90s/it]
 50%|█████     | 6050/12000 [3:35:19<3:01:01,  1.83s/it]
                                                        

 50%|█████     | 6050/12000 [3:35:19<3:01:01,  1.83s/it]
 50%|█████     | 6051/12000 [3:35:21<3:26:19,  2.08s/it]
                                                        

 50%|█████     | 6051/12000 [3:35:21<3:26:19,  2.08s/it]
 50%|█████     | 6052/12000 [3:35:22<3:01:43,  1.83s/it]
                                                        

 50%|█████     | 6052/12000 [3:35:22<3:01:43,  1.83s/it]
 50%|█████     | 6053/12000 [3:35:24<2:49:57,  1.71s/it]
                                                        

 50%|█████     | 6053/12000 [3:35:24<2:49:57,  1.71s/it]
 50%|█████     | 6054/12000 [3:35:26<3:14:46,  1.97s/it]
                                                        

 50%|█████     | 6054/12000 [3:35:26<3:14:46,  1.97s/it]
 50%|█████     | 6055/12000 [3:35:29<3:46:34,  2.29s/it]
                                                        

 50%|█████     | 6055/12000 [3:35:29<3:46:34,  2.29s/it]
 50%|█████     | 6056/12000 [3:35:31<3:19:04,  2.01s/it]
                                                        

 50%|█████     | 6056/12000 [3:35:31<3:19:04,  2.01s/it]
 50%|█████     | 6057/12000 [3:35:33<3:25:04,  2.07s/it]
                                                        

 50%|█████     | 6057/12000 [3:35:33<3:25:04,  2.07s/it]
 50%|█████     | 6058/12000 [3:35:35<3:13:36,  1.96s/it]
                                                        

 50%|█████     | 6058/12000 [3:35:35<3:13:36,  1.96s/it]
 50%|█████     | 6059/12000 [3:35:36<3:03:41,  1.86s/it]
                                                        

 50%|█████     | 6059/12000 [3:35:36<3:03:41,  1.86s/it]
 50%|█████     | 6060/12000 [3:35:38<2:59:15,  1.81s/it]
                                                        

 50%|█████     | 6060/12000 [3:35:38<2:59:15,  1.81s/it]
 51%|█████     | 6061/12000 [3:35:43<4:22:08,  2.65s/it]
                                                        

 51%|█████     | 6061/12000 [3:35:43<4:22:08,  2.65s/it]
 51%|█████     | 6062/12000 [3:35:44<3:54:54,  2.37s/it]
                                                        

 51%|█████     | 6062/12000 [3:35:44<3:54:54,  2.37s/it]
 51%|█████     | 6063/12000 [3:35:46<3:28:15,  2.10s/it]
                                                        

 51%|█████     | 6063/12000 [3:35:46<3:28:15,  2.10s/it]
 51%|█████     | 6064/12000 [3:35:50<4:39:15,  2.82s/it]
                                                        

 51%|█████     | 6064/12000 [3:35:50<4:39:15,  2.82s/it]
 51%|█████     | 6065/12000 [3:35:52<4:04:52,  2.48s/it]
                                                        

 51%|█████     | 6065/12000 [3:35:52<4:04:52,  2.48s/it]
 51%|█████     | 6066/12000 [3:35:54<3:35:00,  2.17s/it]
                                                        

 51%|█████     | 6066/12000 [3:35:54<3:35:00,  2.17s/it]
 51%|█████     | 6067/12000 [3:35:55<3:22:31,  2.05s/it]
                                                        
{'loss': '0.8475', 'grad_norm': '0.3344', 'learning_rate': '0.0001077', 'entropy': '0.8361', 'num_tokens': '3.254e+07', 'mean_token_accuracy': '0.7886', 'epoch': '0.05328'}
{'loss': '0.9963', 'grad_norm': '0.551', 'learning_rate': '0.0001077', 'entropy': '1.021', 'num_tokens': '3.254e+07', 'mean_token_accuracy': '0.7563', 'epoch': '0.05328'}
{'loss': '0.8816', 'grad_norm': '0.4944', 'learning_rate': '0.0001077', 'entropy': '0.8228', 'num_tokens': '3.255e+07', 'mean_token_accuracy': '0.7886', 'epoch': '0.05329'}
{'loss': '0.7597', 'grad_norm': '0.3851', 'learning_rate': '0.0001077', 'entropy': '0.7451', 'num_tokens': '3.255e+07', 'mean_token_accuracy': '0.7954', 'epoch': '0.0533'}
{'loss': '0.8731', 'grad_norm': '0.4154', 'learning_rate': '0.0001076', 'entropy': '0.8764', 'num_tokens': '3.256e+07', 'mean_token_accuracy': '0.7982', 'epoch': '0.05331'}
{'loss': '0.9181', 'grad_norm': '0.5203', 'learning_rate': '0.0001076', 'entropy': '0.9489', 'num_tokens': '3.256e+07', 'mean_token_accuracy': '0.7494', 'epoch': '0.05332'}
{'loss': '1.138', 'grad_norm': '0.4313', 'learning_rate': '0.0001076', 'entropy': '1.084', 'num_tokens': '3.257e+07', 'mean_token_accuracy': '0.7379', 'epoch': '0.05333'}
{'loss': '1.04', 'grad_norm': '0.4514', 'learning_rate': '0.0001075', 'entropy': '1.018', 'num_tokens': '3.257e+07', 'mean_token_accuracy': '0.7669', 'epoch': '0.05334'}
{'loss': '0.89', 'grad_norm': '0.4826', 'learning_rate': '0.0001075', 'entropy': '0.9522', 'num_tokens': '3.258e+07', 'mean_token_accuracy': '0.7521', 'epoch': '0.05335'}
{'loss': '0.7907', 'grad_norm': '0.4676', 'learning_rate': '0.0001075', 'entropy': '0.8936', 'num_tokens': '3.258e+07', 'mean_token_accuracy': '0.796', 'epoch': '0.05336'}
{'loss': '0.9503', 'grad_norm': '0.434', 'learning_rate': '0.0001075', 'entropy': '0.825', 'num_tokens': '3.259e+07', 'mean_token_accuracy': '0.7815', 'epoch': '0.05336'}
{'loss': '1.054', 'grad_norm': '0.5235', 'learning_rate': '0.0001074', 'entropy': '1.078', 'num_tokens': '3.259e+07', 'mean_token_accuracy': '0.7427', 'epoch': '0.05337'}
{'loss': '1.057', 'grad_norm': '0.4753', 'learning_rate': '0.0001074', 'entropy': '1.037', 'num_tokens': '3.259e+07', 'mean_token_accuracy': '0.7484', 'epoch': '0.05338'}
{'loss': '0.8732', 'grad_norm': '0.4363', 'learning_rate': '0.0001074', 'entropy': '0.8596', 'num_tokens': '3.26e+07', 'mean_token_accuracy': '0.7717', 'epoch': '0.05339'}
{'loss': '0.7786', 'grad_norm': '0.3908', 'learning_rate': '0.0001074', 'entropy': '0.7548', 'num_tokens': '3.26e+07', 'mean_token_accuracy': '0.8207', 'epoch': '0.0534'}
{'loss': '1.267', 'grad_norm': '0.4301', 'learning_rate': '0.0001073', 'entropy': '1.259', 'num_tokens': '3.261e+07', 'mean_token_accuracy': '0.7069', 'epoch': '0.05341'}
{'loss': '0.7707', 'grad_norm': '0.4361', 'learning_rate': '0.0001073', 'entropy': '0.7081', 'num_tokens': '3.261e+07', 'mean_token_accuracy': '0.8223', 'epoch': '0.05342'}
{'loss': '1.203', 'grad_norm': '0.3938', 'learning_rate': '0.0001073', 'entropy': '1.272', 'num_tokens': '3.262e+07', 'mean_token_accuracy': '0.6967', 'epoch': '0.05343'}
{'loss': '0.9905', 'grad_norm': '0.4608', 'learning_rate': '0.0001072', 'entropy': '0.9952', 'num_tokens': '3.263e+07', 'mean_token_accuracy': '0.7358', 'epoch': '0.05344'}
{'loss': '0.7747', 'grad_norm': '0.3372', 'learning_rate': '0.0001072', 'entropy': '0.834', 'num_tokens': '3.263e+07', 'mean_token_accuracy': '0.7923', 'epoch': '0.05344'}
{'loss': '0.9168', 'grad_norm': '0.385', 'learning_rate': '0.0001072', 'entropy': '0.8961', 'num_tokens': '3.264e+07', 'mean_token_accuracy': '0.7844', 'epoch': '0.05345'}
{'loss': '1.014', 'grad_norm': '0.529', 'learning_rate': '0.0001072', 'entropy': '0.9453', 'num_tokens': '3.264e+07', 'mean_token_accuracy': '0.7722', 'epoch': '0.05346'}
{'loss': '0.5566', 'grad_norm': '0.3386', 'learning_rate': '0.0001071', 'entropy': '0.5922', 'num_tokens': '3.265e+07', 'mean_token_accuracy': '0.8511', 'epoch': '0.05347'}
{'loss': '0.6069', 'grad_norm': '0.4128', 'learning_rate': '0.0001071', 'entropy': '0.6328', 'num_tokens': '3.265e+07', 'mean_token_accuracy': '0.8311', 'epoch': '0.05348'}
{'loss': '0.7531', 'grad_norm': '0.4329', 'learning_rate': '0.0001071', 'entropy': '0.7565', 'num_tokens': '3.266e+07', 'mean_token_accuracy': '0.7899', 'epoch': '0.05349'}
{'loss': '0.6886', 'grad_norm': '0.2847', 'learning_rate': '0.000107', 'entropy': '0.6974', 'num_tokens': '3.267e+07', 'mean_token_accuracy': '0.8167', 'epoch': '0.0535'}
{'loss': '0.8979', 'grad_norm': '0.3585', 'learning_rate': '0.000107', 'entropy': '0.8936', 'num_tokens': '3.267e+07', 'mean_token_accuracy': '0.7861', 'epoch': '0.05351'}
{'loss': '0.6213', 'grad_norm': '0.3791', 'learning_rate': '0.000107', 'entropy': '0.5837', 'num_tokens': '3.268e+07', 'mean_token_accuracy': '0.8382', 'epoch': '0.05351'}
{'loss': '0.7585', 'grad_norm': '0.4508', 'learning_rate': '0.000107', 'entropy': '0.7917', 'num_tokens': '3.268e+07', 'mean_token_accuracy': '0.792', 'epoch': '0.05352'}
{'loss': '0.7144', 'grad_norm': '0.4184', 'learning_rate': '0.0001069', 'entropy': '0.7216', 'num_tokens': '3.269e+07', 'mean_token_accuracy': '0.815', 'epoch': '0.05353'}
{'loss': '0.8506', 'grad_norm': '0.4486', 'learning_rate': '0.0001069', 'entropy': '0.7579', 'num_tokens': '3.269e+07', 'mean_token_accuracy': '0.8054', 'epoch': '0.05354'}
{'loss': '0.7789', 'grad_norm': '0.4466', 'learning_rate': '0.0001069', 'entropy': '0.724', 'num_tokens': '3.27e+07', 'mean_token_accuracy': '0.8067', 'epoch': '0.05355'}
{'loss': '0.9047', 'grad_norm': '0.4054', 'learning_rate': '0.0001069', 'entropy': '0.8766', 'num_tokens': '3.27e+07', 'mean_token_accuracy': '0.7689', 'epoch': '0.05356'}
{'loss': '0.6461', 'grad_norm': '0.3058', 'learning_rate': '0.0001068', 'entropy': '0.6783', 'num_tokens': '3.271e+07', 'mean_token_accuracy': '0.8229', 'epoch': '0.05357'}
{'loss': '0.909', 'grad_norm': '0.4403', 'learning_rate': '0.0001068', 'entropy': '1.063', 'num_tokens': '3.271e+07', 'mean_token_accuracy': '0.7522', 'epoch': '0.05358'}
{'loss': '1.046', 'grad_norm': '0.3561', 'learning_rate': '0.0001068', 'entropy': '0.9183', 'num_tokens': '3.272e+07', 'mean_token_accuracy': '0.7726', 'epoch': '0.05359'}
{'loss': '0.7705', 'grad_norm': '0.3557', 'learning_rate': '0.0001067', 'entropy': '0.7808', 'num_tokens': '3.272e+07', 'mean_token_accuracy': '0.8027', 'epoch': '0.05359'}
{'loss': '0.9271', 'grad_norm': '0.3838', 'learning_rate': '0.0001067', 'entropy': '0.8628', 'num_tokens': '3.273e+07', 'mean_token_accuracy': '0.7832', 'epoch': '0.0536'}
{'loss': '1.126', 'grad_norm': '0.3551', 'learning_rate': '0.0001067', 'entropy': '1.105', 'num_tokens': '3.274e+07', 'mean_token_accuracy': '0.7465', 'epoch': '0.05361'}
{'loss': '0.9891', 'grad_norm': '0.3812', 'learning_rate': '0.0001067', 'entropy': '0.9164', 'num_tokens': '3.274e+07', 'mean_token_accuracy': '0.7859', 'epoch': '0.05362'}
{'loss': '0.7959', 'grad_norm': '0.4103', 'learning_rate': '0.0001066', 'entropy': '0.8371', 'num_tokens': '3.275e+07', 'mean_token_accuracy': '0.8049', 'epoch': '0.05363'}
{'loss': '1.769', 'grad_norm': '0.3813', 'learning_rate': '0.0001066', 'entropy': '1.401', 'num_tokens': '3.275e+07', 'mean_token_accuracy': '0.6904', 'epoch': '0.05364'}
{'loss': '0.9012', 'grad_norm': '0.4119', 'learning_rate': '0.0001066', 'entropy': '0.8854', 'num_tokens': '3.276e+07', 'mean_token_accuracy': '0.7801', 'epoch': '0.05365'}
{'loss': '0.8828', 'grad_norm': '0.4204', 'learning_rate': '0.0001066', 'entropy': '0.9791', 'num_tokens': '3.276e+07', 'mean_token_accuracy': '0.7623', 'epoch': '0.05366'}
{'loss': '0.9285', 'grad_norm': '0.4425', 'learning_rate': '0.0001065', 'entropy': '0.9272', 'num_tokens': '3.277e+07', 'mean_token_accuracy': '0.7706', 'epoch': '0.05367'}
{'loss': '0.7799', 'grad_norm': '0.3578', 'learning_rate': '0.0001065', 'entropy': '0.8029', 'num_tokens': '3.277e+07', 'mean_token_accuracy': '0.8042', 'epoch': '0.05367'}
{'loss': '0.6221', 'grad_norm': '0.3268', 'learning_rate': '0.0001065', 'entropy': '0.6664', 'num_tokens': '3.278e+07', 'mean_token_accuracy': '0.845', 'epoch': '0.05368'}

 51%|█████     | 6067/12000 [3:35:55<3:22:31,  2.05s/it]
 51%|█████     | 6068/12000 [3:35:57<3:17:28,  2.00s/it]
                                                        

 51%|█████     | 6068/12000 [3:35:57<3:17:28,  2.00s/it]
 51%|█████     | 6069/12000 [3:35:59<3:06:42,  1.89s/it]
                                                        

 51%|█████     | 6069/12000 [3:35:59<3:06:42,  1.89s/it]
 51%|█████     | 6070/12000 [3:36:01<3:08:27,  1.91s/it]
                                                        

 51%|█████     | 6070/12000 [3:36:01<3:08:27,  1.91s/it]
 51%|█████     | 6071/12000 [3:36:02<2:55:04,  1.77s/it]
                                                        

 51%|█████     | 6071/12000 [3:36:02<2:55:04,  1.77s/it]
 51%|█████     | 6072/12000 [3:36:07<4:18:45,  2.62s/it]
                                                        

 51%|█████     | 6072/12000 [3:36:07<4:18:45,  2.62s/it]
 51%|█████     | 6073/12000 [3:36:08<3:40:29,  2.23s/it]
                                                        

 51%|█████     | 6073/12000 [3:36:08<3:40:29,  2.23s/it]
 51%|█████     | 6074/12000 [3:36:11<3:47:31,  2.30s/it]
                                                        

 51%|█████     | 6074/12000 [3:36:11<3:47:31,  2.30s/it]
 51%|█████     | 6075/12000 [3:36:12<3:28:03,  2.11s/it]
                                                        

 51%|█████     | 6075/12000 [3:36:12<3:28:03,  2.11s/it]
 51%|█████     | 6076/12000 [3:36:14<3:03:13,  1.86s/it]
                                                        

 51%|█████     | 6076/12000 [3:36:14<3:03:13,  1.86s/it]
 51%|█████     | 6077/12000 [3:36:15<2:50:20,  1.73s/it]
                                                        

 51%|█████     | 6077/12000 [3:36:15<2:50:20,  1.73s/it]
 51%|█████     | 6078/12000 [3:36:18<3:25:44,  2.08s/it]
                                                        

 51%|█████     | 6078/12000 [3:36:18<3:25:44,  2.08s/it]
 51%|█████     | 6079/12000 [3:36:19<3:04:24,  1.87s/it]
                                                        

 51%|█████     | 6079/12000 [3:36:19<3:04:24,  1.87s/it]
 51%|█████     | 6080/12000 [3:36:22<3:16:53,  2.00s/it]
                                                        

 51%|█████     | 6080/12000 [3:36:22<3:16:53,  2.00s/it]
 51%|█████     | 6081/12000 [3:36:24<3:29:57,  2.13s/it]
                                                        

 51%|█████     | 6081/12000 [3:36:24<3:29:57,  2.13s/it]
 51%|█████     | 6082/12000 [3:36:26<3:13:20,  1.96s/it]
                                                        

 51%|█████     | 6082/12000 [3:36:26<3:13:20,  1.96s/it]
 51%|█████     | 6083/12000 [3:36:27<2:58:18,  1.81s/it]
                                                        

 51%|█████     | 6083/12000 [3:36:27<2:58:18,  1.81s/it]
 51%|█████     | 6084/12000 [3:36:29<3:04:58,  1.88s/it]
                                                        

 51%|█████     | 6084/12000 [3:36:29<3:04:58,  1.88s/it]
 51%|█████     | 6085/12000 [3:36:31<2:56:46,  1.79s/it]
                                                        

 51%|█████     | 6085/12000 [3:36:31<2:56:46,  1.79s/it]
 51%|█████     | 6086/12000 [3:36:34<3:42:09,  2.25s/it]
                                                        

 51%|█████     | 6086/12000 [3:36:34<3:42:09,  2.25s/it]
 51%|█████     | 6087/12000 [3:36:36<3:22:30,  2.05s/it]
                                                        

 51%|█████     | 6087/12000 [3:36:36<3:22:30,  2.05s/it]
 51%|█████     | 6088/12000 [3:36:38<3:28:36,  2.12s/it]
                                                        

 51%|█████     | 6088/12000 [3:36:38<3:28:36,  2.12s/it]
 51%|█████     | 6089/12000 [3:36:39<3:13:15,  1.96s/it]
                                                        

 51%|█████     | 6089/12000 [3:36:39<3:13:15,  1.96s/it]
 51%|█████     | 6090/12000 [3:36:41<3:02:33,  1.85s/it]
                                                        

 51%|█████     | 6090/12000 [3:36:41<3:02:33,  1.85s/it]
 51%|█████     | 6091/12000 [3:36:43<3:06:06,  1.89s/it]
                                                        

 51%|█████     | 6091/12000 [3:36:43<3:06:06,  1.89s/it]
 51%|█████     | 6092/12000 [3:36:44<2:50:11,  1.73s/it]
                                                        

 51%|█████     | 6092/12000 [3:36:44<2:50:11,  1.73s/it]
 51%|█████     | 6093/12000 [3:36:46<2:50:07,  1.73s/it]
                                                        

 51%|█████     | 6093/12000 [3:36:46<2:50:07,  1.73s/it]
 51%|█████     | 6094/12000 [3:36:48<2:47:10,  1.70s/it]
                                                        

 51%|█████     | 6094/12000 [3:36:48<2:47:10,  1.70s/it]
 51%|█████     | 6095/12000 [3:36:49<2:38:59,  1.62s/it]
                                                        

 51%|█████     | 6095/12000 [3:36:49<2:38:59,  1.62s/it]
 51%|█████     | 6096/12000 [3:36:51<2:33:36,  1.56s/it]
                                                        

 51%|█████     | 6096/12000 [3:36:51<2:33:36,  1.56s/it]
 51%|█████     | 6097/12000 [3:36:53<2:47:32,  1.70s/it]
                                                        

 51%|█████     | 6097/12000 [3:36:53<2:47:32,  1.70s/it]
 51%|█████     | 6098/12000 [3:36:54<2:46:22,  1.69s/it]
                                                        

 51%|█████     | 6098/12000 [3:36:54<2:46:22,  1.69s/it]
 51%|█████     | 6099/12000 [3:36:56<2:35:27,  1.58s/it]
                                                        

 51%|█████     | 6099/12000 [3:36:56<2:35:27,  1.58s/it]
 51%|█████     | 6100/12000 [3:36:57<2:36:38,  1.59s/it]
                                                        

 51%|█████     | 6100/12000 [3:36:57<2:36:38,  1.59s/it]
 51%|█████     | 6101/12000 [3:37:01<3:30:52,  2.14s/it]
                                                        

 51%|█████     | 6101/12000 [3:37:01<3:30:52,  2.14s/it]
 51%|█████     | 6102/12000 [3:37:02<3:14:12,  1.98s/it]
                                                        

 51%|█████     | 6102/12000 [3:37:02<3:14:12,  1.98s/it]
 51%|█████     | 6103/12000 [3:37:04<3:10:22,  1.94s/it]
                                                        

 51%|█████     | 6103/12000 [3:37:04<3:10:22,  1.94s/it]
 51%|█████     | 6104/12000 [3:37:06<3:11:20,  1.95s/it]
                                                        

 51%|█████     | 6104/12000 [3:37:06<3:11:20,  1.95s/it]
 51%|█████     | 6105/12000 [3:37:08<3:03:14,  1.87s/it]
                                                        

 51%|█████     | 6105/12000 [3:37:08<3:03:14,  1.87s/it]
 51%|█████     | 6106/12000 [3:37:09<2:50:38,  1.74s/it]
                                                        

 51%|█████     | 6106/12000 [3:37:09<2:50:38,  1.74s/it]
 51%|█████     | 6107/12000 [3:37:10<2:38:13,  1.61s/it]
                                                        

 51%|█████     | 6107/12000 [3:37:10<2:38:13,  1.61s/it]
 51%|█████     | 6108/12000 [3:37:14<3:29:40,  2.14s/it]
                                                        

 51%|█████     | 6108/12000 [3:37:14<3:29:40,  2.14s/it]
 51%|█████     | 6109/12000 [3:37:15<3:13:58,  1.98s/it]
                                                        

 51%|█████     | 6109/12000 [3:37:15<3:13:58,  1.98s/it]
 51%|█████     | 6110/12000 [3:37:17<3:08:32,  1.92s/it]
                                                        

 51%|█████     | 6110/12000 [3:37:17<3:08:32,  1.92s/it]
 51%|█████     | 6111/12000 [3:37:19<3:00:36,  1.84s/it]
                                                        

 51%|█████     | 6111/12000 [3:37:19<3:00:36,  1.84s/it]
 51%|█████     | 6112/12000 [3:37:21<3:03:42,  1.87s/it]
                                                        

 51%|█████     | 6112/12000 [3:37:21<3:03:42,  1.87s/it]
 51%|█████     | 6113/12000 [3:37:23<3:12:58,  1.97s/it]
                                                        

 51%|█████     | 6113/12000 [3:37:23<3:12:58,  1.97s/it]
 51%|█████     | 6114/12000 [3:37:26<3:29:35,  2.14s/it]
                                                        
{'loss': '0.7344', 'grad_norm': '0.4565', 'learning_rate': '0.0001064', 'entropy': '0.7858', 'num_tokens': '3.278e+07', 'mean_token_accuracy': '0.7963', 'epoch': '0.05369'}
{'loss': '1.149', 'grad_norm': '0.4085', 'learning_rate': '0.0001064', 'entropy': '1.054', 'num_tokens': '3.279e+07', 'mean_token_accuracy': '0.7447', 'epoch': '0.0537'}
{'loss': '1.093', 'grad_norm': '0.434', 'learning_rate': '0.0001064', 'entropy': '1.026', 'num_tokens': '3.279e+07', 'mean_token_accuracy': '0.7527', 'epoch': '0.05371'}
{'loss': '0.9098', 'grad_norm': '0.4133', 'learning_rate': '0.0001064', 'entropy': '0.8946', 'num_tokens': '3.28e+07', 'mean_token_accuracy': '0.7667', 'epoch': '0.05372'}
{'loss': '0.6871', 'grad_norm': '0.3515', 'learning_rate': '0.0001063', 'entropy': '0.7334', 'num_tokens': '3.28e+07', 'mean_token_accuracy': '0.8129', 'epoch': '0.05373'}
{'loss': '0.9561', 'grad_norm': '0.3428', 'learning_rate': '0.0001063', 'entropy': '0.7875', 'num_tokens': '3.281e+07', 'mean_token_accuracy': '0.8002', 'epoch': '0.05374'}
{'loss': '0.565', 'grad_norm': '0.3536', 'learning_rate': '0.0001063', 'entropy': '0.5747', 'num_tokens': '3.282e+07', 'mean_token_accuracy': '0.847', 'epoch': '0.05375'}
{'loss': '1.098', 'grad_norm': '0.4351', 'learning_rate': '0.0001063', 'entropy': '0.8799', 'num_tokens': '3.282e+07', 'mean_token_accuracy': '0.7812', 'epoch': '0.05375'}
{'loss': '0.8638', 'grad_norm': '0.4424', 'learning_rate': '0.0001062', 'entropy': '0.8584', 'num_tokens': '3.283e+07', 'mean_token_accuracy': '0.7796', 'epoch': '0.05376'}
{'loss': '0.9463', 'grad_norm': '0.432', 'learning_rate': '0.0001062', 'entropy': '1.077', 'num_tokens': '3.283e+07', 'mean_token_accuracy': '0.7419', 'epoch': '0.05377'}
{'loss': '0.7972', 'grad_norm': '0.3567', 'learning_rate': '0.0001062', 'entropy': '0.8632', 'num_tokens': '3.284e+07', 'mean_token_accuracy': '0.7859', 'epoch': '0.05378'}
{'loss': '0.8572', 'grad_norm': '0.348', 'learning_rate': '0.0001061', 'entropy': '0.9957', 'num_tokens': '3.284e+07', 'mean_token_accuracy': '0.7618', 'epoch': '0.05379'}
{'loss': '0.7617', 'grad_norm': '0.3683', 'learning_rate': '0.0001061', 'entropy': '0.7479', 'num_tokens': '3.285e+07', 'mean_token_accuracy': '0.814', 'epoch': '0.0538'}
{'loss': '0.6168', 'grad_norm': '0.2736', 'learning_rate': '0.0001061', 'entropy': '0.7265', 'num_tokens': '3.285e+07', 'mean_token_accuracy': '0.8104', 'epoch': '0.05381'}
{'loss': '0.6765', 'grad_norm': '0.3556', 'learning_rate': '0.0001061', 'entropy': '0.6225', 'num_tokens': '3.286e+07', 'mean_token_accuracy': '0.8379', 'epoch': '0.05382'}
{'loss': '0.8518', 'grad_norm': '0.414', 'learning_rate': '0.000106', 'entropy': '0.9179', 'num_tokens': '3.286e+07', 'mean_token_accuracy': '0.7824', 'epoch': '0.05382'}
{'loss': '0.8383', 'grad_norm': '0.3611', 'learning_rate': '0.000106', 'entropy': '0.834', 'num_tokens': '3.287e+07', 'mean_token_accuracy': '0.7909', 'epoch': '0.05383'}
{'loss': '0.6505', 'grad_norm': '0.384', 'learning_rate': '0.000106', 'entropy': '0.6351', 'num_tokens': '3.287e+07', 'mean_token_accuracy': '0.839', 'epoch': '0.05384'}
{'loss': '1.326', 'grad_norm': '0.4758', 'learning_rate': '0.0001059', 'entropy': '1.228', 'num_tokens': '3.288e+07', 'mean_token_accuracy': '0.7153', 'epoch': '0.05385'}
{'loss': '0.6495', 'grad_norm': '0.3614', 'learning_rate': '0.0001059', 'entropy': '0.6607', 'num_tokens': '3.288e+07', 'mean_token_accuracy': '0.8333', 'epoch': '0.05386'}
{'loss': '0.7513', 'grad_norm': '0.4112', 'learning_rate': '0.0001059', 'entropy': '0.751', 'num_tokens': '3.289e+07', 'mean_token_accuracy': '0.791', 'epoch': '0.05387'}
{'loss': '0.9094', 'grad_norm': '0.4608', 'learning_rate': '0.0001059', 'entropy': '0.832', 'num_tokens': '3.289e+07', 'mean_token_accuracy': '0.7842', 'epoch': '0.05388'}
{'loss': '0.9793', 'grad_norm': '0.4132', 'learning_rate': '0.0001058', 'entropy': '1.063', 'num_tokens': '3.29e+07', 'mean_token_accuracy': '0.7369', 'epoch': '0.05389'}
{'loss': '0.725', 'grad_norm': '0.4146', 'learning_rate': '0.0001058', 'entropy': '0.7468', 'num_tokens': '3.29e+07', 'mean_token_accuracy': '0.7991', 'epoch': '0.0539'}
{'loss': '1.055', 'grad_norm': '0.3612', 'learning_rate': '0.0001058', 'entropy': '0.9467', 'num_tokens': '3.291e+07', 'mean_token_accuracy': '0.7622', 'epoch': '0.0539'}
{'loss': '0.8693', 'grad_norm': '0.3866', 'learning_rate': '0.0001058', 'entropy': '0.8637', 'num_tokens': '3.291e+07', 'mean_token_accuracy': '0.7818', 'epoch': '0.05391'}
{'loss': '1.105', 'grad_norm': '0.3869', 'learning_rate': '0.0001057', 'entropy': '1.227', 'num_tokens': '3.292e+07', 'mean_token_accuracy': '0.7214', 'epoch': '0.05392'}
{'loss': '1.009', 'grad_norm': '0.4291', 'learning_rate': '0.0001057', 'entropy': '0.9585', 'num_tokens': '3.292e+07', 'mean_token_accuracy': '0.767', 'epoch': '0.05393'}
{'loss': '0.9179', 'grad_norm': '0.4152', 'learning_rate': '0.0001057', 'entropy': '0.8979', 'num_tokens': '3.293e+07', 'mean_token_accuracy': '0.7788', 'epoch': '0.05394'}
{'loss': '1.067', 'grad_norm': '0.4351', 'learning_rate': '0.0001056', 'entropy': '1.112', 'num_tokens': '3.293e+07', 'mean_token_accuracy': '0.7237', 'epoch': '0.05395'}
{'loss': '0.7112', 'grad_norm': '0.4198', 'learning_rate': '0.0001056', 'entropy': '0.7432', 'num_tokens': '3.294e+07', 'mean_token_accuracy': '0.7998', 'epoch': '0.05396'}
{'loss': '0.7833', 'grad_norm': '0.3901', 'learning_rate': '0.0001056', 'entropy': '0.7798', 'num_tokens': '3.295e+07', 'mean_token_accuracy': '0.7981', 'epoch': '0.05397'}
{'loss': '0.9408', 'grad_norm': '0.4658', 'learning_rate': '0.0001056', 'entropy': '0.9166', 'num_tokens': '3.295e+07', 'mean_token_accuracy': '0.7781', 'epoch': '0.05398'}
{'loss': '0.7771', 'grad_norm': '0.4173', 'learning_rate': '0.0001055', 'entropy': '0.8208', 'num_tokens': '3.295e+07', 'mean_token_accuracy': '0.7867', 'epoch': '0.05398'}
{'loss': '0.7289', 'grad_norm': '0.362', 'learning_rate': '0.0001055', 'entropy': '0.9428', 'num_tokens': '3.296e+07', 'mean_token_accuracy': '0.7773', 'epoch': '0.05399'}
{'loss': '1.027', 'grad_norm': '0.5129', 'learning_rate': '0.0001055', 'entropy': '1.026', 'num_tokens': '3.296e+07', 'mean_token_accuracy': '0.7516', 'epoch': '0.054'}
{'loss': '1.134', 'grad_norm': '0.4076', 'learning_rate': '0.0001055', 'entropy': '1.068', 'num_tokens': '3.297e+07', 'mean_token_accuracy': '0.7321', 'epoch': '0.05401'}
{'loss': '0.6272', 'grad_norm': '0.3336', 'learning_rate': '0.0001054', 'entropy': '0.7057', 'num_tokens': '3.297e+07', 'mean_token_accuracy': '0.8155', 'epoch': '0.05402'}
{'loss': '0.9903', 'grad_norm': '0.3641', 'learning_rate': '0.0001054', 'entropy': '0.9814', 'num_tokens': '3.298e+07', 'mean_token_accuracy': '0.7522', 'epoch': '0.05403'}
{'loss': '1.077', 'grad_norm': '0.4736', 'learning_rate': '0.0001054', 'entropy': '1.047', 'num_tokens': '3.298e+07', 'mean_token_accuracy': '0.736', 'epoch': '0.05404'}
{'loss': '0.7054', 'grad_norm': '0.338', 'learning_rate': '0.0001053', 'entropy': '0.7143', 'num_tokens': '3.299e+07', 'mean_token_accuracy': '0.8206', 'epoch': '0.05405'}
{'loss': '0.8589', 'grad_norm': '0.4387', 'learning_rate': '0.0001053', 'entropy': '0.9391', 'num_tokens': '3.299e+07', 'mean_token_accuracy': '0.7736', 'epoch': '0.05405'}
{'loss': '0.7703', 'grad_norm': '0.3981', 'learning_rate': '0.0001053', 'entropy': '0.8229', 'num_tokens': '3.3e+07', 'mean_token_accuracy': '0.7929', 'epoch': '0.05406'}
{'loss': '0.812', 'grad_norm': '0.4584', 'learning_rate': '0.0001053', 'entropy': '0.8067', 'num_tokens': '3.3e+07', 'mean_token_accuracy': '0.7908', 'epoch': '0.05407'}
{'loss': '0.6723', 'grad_norm': '0.3974', 'learning_rate': '0.0001052', 'entropy': '0.685', 'num_tokens': '3.301e+07', 'mean_token_accuracy': '0.8284', 'epoch': '0.05408'}
{'loss': '0.7756', 'grad_norm': '0.3733', 'learning_rate': '0.0001052', 'entropy': '0.783', 'num_tokens': '3.301e+07', 'mean_token_accuracy': '0.7863', 'epoch': '0.05409'}
{'loss': '0.8663', 'grad_norm': '0.4462', 'learning_rate': '0.0001052', 'entropy': '0.7604', 'num_tokens': '3.302e+07', 'mean_token_accuracy': '0.8017', 'epoch': '0.0541'}

 51%|█████     | 6114/12000 [3:37:26<3:29:35,  2.14s/it]
 51%|█████     | 6115/12000 [3:37:28<3:26:49,  2.11s/it]
                                                        

 51%|█████     | 6115/12000 [3:37:28<3:26:49,  2.11s/it]
 51%|█████     | 6116/12000 [3:37:30<3:34:47,  2.19s/it]
                                                        

 51%|█████     | 6116/12000 [3:37:30<3:34:47,  2.19s/it]
 51%|█████     | 6117/12000 [3:37:33<4:12:10,  2.57s/it]
                                                        

 51%|█████     | 6117/12000 [3:37:33<4:12:10,  2.57s/it]
 51%|█████     | 6118/12000 [3:37:36<4:07:35,  2.53s/it]
                                                        

 51%|█████     | 6118/12000 [3:37:36<4:07:35,  2.53s/it]
 51%|█████     | 6119/12000 [3:37:37<3:39:52,  2.24s/it]
                                                        

 51%|█████     | 6119/12000 [3:37:37<3:39:52,  2.24s/it]
 51%|█████     | 6120/12000 [3:37:40<4:00:29,  2.45s/it]
                                                        

 51%|█████     | 6120/12000 [3:37:40<4:00:29,  2.45s/it]
 51%|█████     | 6121/12000 [3:37:42<3:29:50,  2.14s/it]
                                                        

 51%|█████     | 6121/12000 [3:37:42<3:29:50,  2.14s/it]
 51%|█████     | 6122/12000 [3:37:43<3:08:59,  1.93s/it]
                                                        

 51%|█████     | 6122/12000 [3:37:43<3:08:59,  1.93s/it]
 51%|█████     | 6123/12000 [3:37:45<3:00:11,  1.84s/it]
                                                        

 51%|█████     | 6123/12000 [3:37:45<3:00:11,  1.84s/it]
 51%|█████     | 6124/12000 [3:37:46<2:49:36,  1.73s/it]
                                                        

 51%|█████     | 6124/12000 [3:37:46<2:49:36,  1.73s/it]
 51%|█████     | 6125/12000 [3:37:48<2:40:37,  1.64s/it]
                                                        

 51%|█████     | 6125/12000 [3:37:48<2:40:37,  1.64s/it]
 51%|█████     | 6126/12000 [3:37:51<3:16:30,  2.01s/it]
                                                        

 51%|█████     | 6126/12000 [3:37:51<3:16:30,  2.01s/it]
 51%|█████     | 6127/12000 [3:37:53<3:31:51,  2.16s/it]
                                                        

 51%|█████     | 6127/12000 [3:37:53<3:31:51,  2.16s/it]
 51%|█████     | 6128/12000 [3:37:55<3:25:35,  2.10s/it]
                                                        

 51%|█████     | 6128/12000 [3:37:55<3:25:35,  2.10s/it]
 51%|█████     | 6129/12000 [3:37:56<3:05:54,  1.90s/it]
                                                        

 51%|█████     | 6129/12000 [3:37:56<3:05:54,  1.90s/it]
 51%|█████     | 6130/12000 [3:38:01<4:35:10,  2.81s/it]
                                                        

 51%|█████     | 6130/12000 [3:38:01<4:35:10,  2.81s/it]
 51%|█████     | 6131/12000 [3:38:03<3:54:52,  2.40s/it]
                                                        

 51%|█████     | 6131/12000 [3:38:03<3:54:52,  2.40s/it]
 51%|█████     | 6132/12000 [3:38:07<4:48:34,  2.95s/it]
                                                        

 51%|█████     | 6132/12000 [3:38:07<4:48:34,  2.95s/it]
 51%|█████     | 6133/12000 [3:38:09<4:09:13,  2.55s/it]
                                                        

 51%|█████     | 6133/12000 [3:38:09<4:09:13,  2.55s/it]
 51%|█████     | 6134/12000 [3:38:11<4:08:19,  2.54s/it]
                                                        

 51%|█████     | 6134/12000 [3:38:11<4:08:19,  2.54s/it]
 51%|█████     | 6135/12000 [3:38:13<3:34:07,  2.19s/it]
                                                        

 51%|█████     | 6135/12000 [3:38:13<3:34:07,  2.19s/it]
 51%|█████     | 6136/12000 [3:38:14<3:13:03,  1.98s/it]
                                                        

 51%|█████     | 6136/12000 [3:38:14<3:13:03,  1.98s/it]
 51%|█████     | 6137/12000 [3:38:15<2:54:30,  1.79s/it]
                                                        

 51%|█████     | 6137/12000 [3:38:15<2:54:30,  1.79s/it]
 51%|█████     | 6138/12000 [3:38:18<3:15:38,  2.00s/it]
                                                        

 51%|█████     | 6138/12000 [3:38:18<3:15:38,  2.00s/it]
 51%|█████     | 6139/12000 [3:38:19<2:58:35,  1.83s/it]
                                                        

 51%|█████     | 6139/12000 [3:38:19<2:58:35,  1.83s/it]
 51%|█████     | 6140/12000 [3:38:23<3:42:05,  2.27s/it]
                                                        

 51%|█████     | 6140/12000 [3:38:23<3:42:05,  2.27s/it]
 51%|█████     | 6141/12000 [3:38:24<3:24:44,  2.10s/it]
                                                        

 51%|█████     | 6141/12000 [3:38:24<3:24:44,  2.10s/it]
 51%|█████     | 6142/12000 [3:38:26<3:12:16,  1.97s/it]
                                                        

 51%|█████     | 6142/12000 [3:38:26<3:12:16,  1.97s/it]
 51%|█████     | 6143/12000 [3:38:28<3:03:37,  1.88s/it]
                                                        

 51%|█████     | 6143/12000 [3:38:28<3:03:37,  1.88s/it]
 51%|█████     | 6144/12000 [3:38:29<2:53:34,  1.78s/it]
                                                        

 51%|█████     | 6144/12000 [3:38:29<2:53:34,  1.78s/it]
 51%|█████     | 6145/12000 [3:38:32<3:19:03,  2.04s/it]
                                                        

 51%|█████     | 6145/12000 [3:38:32<3:19:03,  2.04s/it]
 51%|█████     | 6146/12000 [3:38:33<3:05:27,  1.90s/it]
                                                        

 51%|█████     | 6146/12000 [3:38:33<3:05:27,  1.90s/it]
 51%|█████     | 6147/12000 [3:38:35<3:02:36,  1.87s/it]
                                                        

 51%|█████     | 6147/12000 [3:38:35<3:02:36,  1.87s/it]
 51%|█████     | 6148/12000 [3:38:37<2:52:09,  1.77s/it]
                                                        

 51%|█████     | 6148/12000 [3:38:37<2:52:09,  1.77s/it]
 51%|█████     | 6149/12000 [3:38:40<3:45:48,  2.32s/it]
                                                        

 51%|█████     | 6149/12000 [3:38:40<3:45:48,  2.32s/it]
 51%|█████▏    | 6150/12000 [3:38:42<3:25:55,  2.11s/it]
                                                        

 51%|█████▏    | 6150/12000 [3:38:42<3:25:55,  2.11s/it]
 51%|█████▏    | 6151/12000 [3:38:46<4:11:28,  2.58s/it]
                                                        

 51%|█████▏    | 6151/12000 [3:38:46<4:11:28,  2.58s/it]
 51%|█████▏    | 6152/12000 [3:38:50<4:51:57,  3.00s/it]
                                                        

 51%|█████▏    | 6152/12000 [3:38:50<4:51:57,  3.00s/it]
 51%|█████▏    | 6153/12000 [3:38:52<4:22:23,  2.69s/it]
                                                        

 51%|█████▏    | 6153/12000 [3:38:52<4:22:23,  2.69s/it]
 51%|█████▏    | 6154/12000 [3:38:53<3:41:44,  2.28s/it]
                                                        

 51%|█████▏    | 6154/12000 [3:38:53<3:41:44,  2.28s/it]
 51%|█████▏    | 6155/12000 [3:38:55<3:24:06,  2.10s/it]
                                                        

 51%|█████▏    | 6155/12000 [3:38:55<3:24:06,  2.10s/it]
 51%|█████▏    | 6156/12000 [3:38:56<3:02:49,  1.88s/it]
                                                        

 51%|█████▏    | 6156/12000 [3:38:56<3:02:49,  1.88s/it]
 51%|█████▏    | 6157/12000 [3:38:58<2:58:49,  1.84s/it]
                                                        

 51%|█████▏    | 6157/12000 [3:38:58<2:58:49,  1.84s/it]
 51%|█████▏    | 6158/12000 [3:38:59<2:50:14,  1.75s/it]
                                                        

 51%|█████▏    | 6158/12000 [3:38:59<2:50:14,  1.75s/it]
 51%|█████▏    | 6159/12000 [3:39:01<2:40:55,  1.65s/it]
                                                        

 51%|█████▏    | 6159/12000 [3:39:01<2:40:55,  1.65s/it]
 51%|█████▏    | 6160/12000 [3:39:03<2:48:31,  1.73s/it]
                                                        

 51%|█████▏    | 6160/12000 [3:39:03<2:48:31,  1.73s/it]
 51%|█████▏    | 6161/12000 [3:39:05<3:07:48,  1.93s/it]
                                                        
{'loss': '0.6001', 'grad_norm': '0.4156', 'learning_rate': '0.0001052', 'entropy': '0.6779', 'num_tokens': '3.303e+07', 'mean_token_accuracy': '0.8219', 'epoch': '0.05411'}
{'loss': '0.8893', 'grad_norm': '0.3821', 'learning_rate': '0.0001051', 'entropy': '0.8425', 'num_tokens': '3.303e+07', 'mean_token_accuracy': '0.7898', 'epoch': '0.05412'}
{'loss': '0.5009', 'grad_norm': '0.3298', 'learning_rate': '0.0001051', 'entropy': '0.6268', 'num_tokens': '3.304e+07', 'mean_token_accuracy': '0.8377', 'epoch': '0.05413'}
{'loss': '0.7049', 'grad_norm': '0.2928', 'learning_rate': '0.0001051', 'entropy': '0.7662', 'num_tokens': '3.305e+07', 'mean_token_accuracy': '0.8043', 'epoch': '0.05413'}
{'loss': '1.048', 'grad_norm': '0.2967', 'learning_rate': '0.000105', 'entropy': '0.9176', 'num_tokens': '3.305e+07', 'mean_token_accuracy': '0.7522', 'epoch': '0.05414'}
{'loss': '0.9854', 'grad_norm': '0.3704', 'learning_rate': '0.000105', 'entropy': '0.9865', 'num_tokens': '3.306e+07', 'mean_token_accuracy': '0.7447', 'epoch': '0.05415'}
{'loss': '0.9147', 'grad_norm': '0.3271', 'learning_rate': '0.000105', 'entropy': '0.8821', 'num_tokens': '3.307e+07', 'mean_token_accuracy': '0.7867', 'epoch': '0.05416'}
{'loss': '0.8037', 'grad_norm': '0.3436', 'learning_rate': '0.000105', 'entropy': '0.8404', 'num_tokens': '3.307e+07', 'mean_token_accuracy': '0.7758', 'epoch': '0.05417'}
{'loss': '0.7967', 'grad_norm': '0.3676', 'learning_rate': '0.0001049', 'entropy': '0.829', 'num_tokens': '3.307e+07', 'mean_token_accuracy': '0.8022', 'epoch': '0.05418'}
{'loss': '1.076', 'grad_norm': '0.3857', 'learning_rate': '0.0001049', 'entropy': '1.07', 'num_tokens': '3.308e+07', 'mean_token_accuracy': '0.7448', 'epoch': '0.05419'}
{'loss': '0.6797', 'grad_norm': '0.3449', 'learning_rate': '0.0001049', 'entropy': '0.7061', 'num_tokens': '3.308e+07', 'mean_token_accuracy': '0.8301', 'epoch': '0.0542'}
{'loss': '0.8609', 'grad_norm': '0.4148', 'learning_rate': '0.0001048', 'entropy': '0.8546', 'num_tokens': '3.309e+07', 'mean_token_accuracy': '0.7898', 'epoch': '0.05421'}
{'loss': '0.6687', 'grad_norm': '0.3295', 'learning_rate': '0.0001048', 'entropy': '0.6372', 'num_tokens': '3.31e+07', 'mean_token_accuracy': '0.8281', 'epoch': '0.05421'}
{'loss': '0.8367', 'grad_norm': '0.4053', 'learning_rate': '0.0001048', 'entropy': '0.776', 'num_tokens': '3.31e+07', 'mean_token_accuracy': '0.7969', 'epoch': '0.05422'}
{'loss': '1.137', 'grad_norm': '0.4214', 'learning_rate': '0.0001048', 'entropy': '1.156', 'num_tokens': '3.311e+07', 'mean_token_accuracy': '0.7307', 'epoch': '0.05423'}
{'loss': '0.8282', 'grad_norm': '0.4122', 'learning_rate': '0.0001047', 'entropy': '0.8488', 'num_tokens': '3.311e+07', 'mean_token_accuracy': '0.7778', 'epoch': '0.05424'}
{'loss': '0.7678', 'grad_norm': '0.3474', 'learning_rate': '0.0001047', 'entropy': '0.9778', 'num_tokens': '3.312e+07', 'mean_token_accuracy': '0.7588', 'epoch': '0.05425'}
{'loss': '0.6607', 'grad_norm': '0.3712', 'learning_rate': '0.0001047', 'entropy': '0.7022', 'num_tokens': '3.312e+07', 'mean_token_accuracy': '0.8174', 'epoch': '0.05426'}
{'loss': '0.8022', 'grad_norm': '0.3415', 'learning_rate': '0.0001047', 'entropy': '0.8254', 'num_tokens': '3.313e+07', 'mean_token_accuracy': '0.7837', 'epoch': '0.05427'}
{'loss': '0.9063', 'grad_norm': '0.4849', 'learning_rate': '0.0001046', 'entropy': '0.7752', 'num_tokens': '3.314e+07', 'mean_token_accuracy': '0.8059', 'epoch': '0.05428'}
{'loss': '0.8411', 'grad_norm': '0.4787', 'learning_rate': '0.0001046', 'entropy': '0.9233', 'num_tokens': '3.314e+07', 'mean_token_accuracy': '0.7626', 'epoch': '0.05428'}
{'loss': '0.8396', 'grad_norm': '0.3461', 'learning_rate': '0.0001046', 'entropy': '0.872', 'num_tokens': '3.315e+07', 'mean_token_accuracy': '0.7736', 'epoch': '0.05429'}
{'loss': '1.312', 'grad_norm': '0.4513', 'learning_rate': '0.0001045', 'entropy': '1.269', 'num_tokens': '3.315e+07', 'mean_token_accuracy': '0.702', 'epoch': '0.0543'}
{'loss': '0.82', 'grad_norm': '0.4209', 'learning_rate': '0.0001045', 'entropy': '0.8507', 'num_tokens': '3.316e+07', 'mean_token_accuracy': '0.7795', 'epoch': '0.05431'}
{'loss': '0.6979', 'grad_norm': '0.3947', 'learning_rate': '0.0001045', 'entropy': '0.702', 'num_tokens': '3.316e+07', 'mean_token_accuracy': '0.8179', 'epoch': '0.05432'}
{'loss': '0.7687', 'grad_norm': '0.3852', 'learning_rate': '0.0001045', 'entropy': '0.8677', 'num_tokens': '3.317e+07', 'mean_token_accuracy': '0.7983', 'epoch': '0.05433'}
{'loss': '0.7313', 'grad_norm': '0.3759', 'learning_rate': '0.0001044', 'entropy': '0.8399', 'num_tokens': '3.317e+07', 'mean_token_accuracy': '0.7875', 'epoch': '0.05434'}
{'loss': '0.7748', 'grad_norm': '0.3619', 'learning_rate': '0.0001044', 'entropy': '0.7714', 'num_tokens': '3.318e+07', 'mean_token_accuracy': '0.8063', 'epoch': '0.05435'}
{'loss': '1.015', 'grad_norm': '0.4081', 'learning_rate': '0.0001044', 'entropy': '1.076', 'num_tokens': '3.318e+07', 'mean_token_accuracy': '0.7335', 'epoch': '0.05436'}
{'loss': '0.8022', 'grad_norm': '0.3315', 'learning_rate': '0.0001044', 'entropy': '0.7963', 'num_tokens': '3.319e+07', 'mean_token_accuracy': '0.803', 'epoch': '0.05436'}
{'loss': '1.054', 'grad_norm': '0.4714', 'learning_rate': '0.0001043', 'entropy': '1.059', 'num_tokens': '3.319e+07', 'mean_token_accuracy': '0.7443', 'epoch': '0.05437'}
{'loss': '0.6977', 'grad_norm': '0.4033', 'learning_rate': '0.0001043', 'entropy': '0.8359', 'num_tokens': '3.32e+07', 'mean_token_accuracy': '0.7996', 'epoch': '0.05438'}
{'loss': '0.9608', 'grad_norm': '0.4155', 'learning_rate': '0.0001043', 'entropy': '0.9339', 'num_tokens': '3.32e+07', 'mean_token_accuracy': '0.7665', 'epoch': '0.05439'}
{'loss': '0.7949', 'grad_norm': '0.3917', 'learning_rate': '0.0001042', 'entropy': '0.8266', 'num_tokens': '3.321e+07', 'mean_token_accuracy': '0.8023', 'epoch': '0.0544'}
{'loss': '1.21', 'grad_norm': '0.4425', 'learning_rate': '0.0001042', 'entropy': '1.208', 'num_tokens': '3.322e+07', 'mean_token_accuracy': '0.689', 'epoch': '0.05441'}
{'loss': '0.8071', 'grad_norm': '0.3484', 'learning_rate': '0.0001042', 'entropy': '0.8555', 'num_tokens': '3.322e+07', 'mean_token_accuracy': '0.7881', 'epoch': '0.05442'}
{'loss': '0.6597', 'grad_norm': '0.3169', 'learning_rate': '0.0001042', 'entropy': '0.7263', 'num_tokens': '3.323e+07', 'mean_token_accuracy': '0.8173', 'epoch': '0.05443'}
{'loss': '0.6272', 'grad_norm': '0.2945', 'learning_rate': '0.0001041', 'entropy': '0.6128', 'num_tokens': '3.323e+07', 'mean_token_accuracy': '0.8473', 'epoch': '0.05444'}
{'loss': '1.343', 'grad_norm': '0.4283', 'learning_rate': '0.0001041', 'entropy': '1.356', 'num_tokens': '3.324e+07', 'mean_token_accuracy': '0.6948', 'epoch': '0.05444'}
{'loss': '1.085', 'grad_norm': '0.4189', 'learning_rate': '0.0001041', 'entropy': '0.9131', 'num_tokens': '3.325e+07', 'mean_token_accuracy': '0.7755', 'epoch': '0.05445'}
{'loss': '0.8639', 'grad_norm': '0.4142', 'learning_rate': '0.000104', 'entropy': '0.8489', 'num_tokens': '3.325e+07', 'mean_token_accuracy': '0.7851', 'epoch': '0.05446'}
{'loss': '1.038', 'grad_norm': '0.3947', 'learning_rate': '0.000104', 'entropy': '1.012', 'num_tokens': '3.326e+07', 'mean_token_accuracy': '0.7527', 'epoch': '0.05447'}
{'loss': '0.895', 'grad_norm': '0.376', 'learning_rate': '0.000104', 'entropy': '0.9532', 'num_tokens': '3.326e+07', 'mean_token_accuracy': '0.7568', 'epoch': '0.05448'}
{'loss': '0.5883', 'grad_norm': '0.3434', 'learning_rate': '0.000104', 'entropy': '0.553', 'num_tokens': '3.327e+07', 'mean_token_accuracy': '0.8488', 'epoch': '0.05449'}
{'loss': '0.8515', 'grad_norm': '0.4391', 'learning_rate': '0.0001039', 'entropy': '0.938', 'num_tokens': '3.327e+07', 'mean_token_accuracy': '0.7706', 'epoch': '0.0545'}
{'loss': '0.8527', 'grad_norm': '0.4642', 'learning_rate': '0.0001039', 'entropy': '0.8752', 'num_tokens': '3.327e+07', 'mean_token_accuracy': '0.7911', 'epoch': '0.05451'}
{'loss': '0.8289', 'grad_norm': '0.369', 'learning_rate': '0.0001039', 'entropy': '0.8676', 'num_tokens': '3.328e+07', 'mean_token_accuracy': '0.7725', 'epoch': '0.05451'}

 51%|█████▏    | 6161/12000 [3:39:05<3:07:48,  1.93s/it]
 51%|█████▏    | 6162/12000 [3:39:06<2:48:57,  1.74s/it]
                                                        

 51%|█████▏    | 6162/12000 [3:39:06<2:48:57,  1.74s/it]
 51%|█████▏    | 6163/12000 [3:39:08<2:39:29,  1.64s/it]
                                                        

 51%|█████▏    | 6163/12000 [3:39:08<2:39:29,  1.64s/it]
 51%|█████▏    | 6164/12000 [3:39:09<2:34:34,  1.59s/it]
                                                        

 51%|█████▏    | 6164/12000 [3:39:09<2:34:34,  1.59s/it]
 51%|█████▏    | 6165/12000 [3:39:11<2:50:17,  1.75s/it]
                                                        

 51%|█████▏    | 6165/12000 [3:39:11<2:50:17,  1.75s/it]
 51%|█████▏    | 6166/12000 [3:39:13<2:36:46,  1.61s/it]
                                                        

 51%|█████▏    | 6166/12000 [3:39:13<2:36:46,  1.61s/it]
 51%|█████▏    | 6167/12000 [3:39:14<2:36:50,  1.61s/it]
                                                        

 51%|█████▏    | 6167/12000 [3:39:14<2:36:50,  1.61s/it]
 51%|█████▏    | 6168/12000 [3:39:16<2:34:58,  1.59s/it]
                                                        

 51%|█████▏    | 6168/12000 [3:39:16<2:34:58,  1.59s/it]
 51%|█████▏    | 6169/12000 [3:39:18<2:41:36,  1.66s/it]
                                                        

 51%|█████▏    | 6169/12000 [3:39:18<2:41:36,  1.66s/it]
 51%|█████▏    | 6170/12000 [3:39:19<2:33:27,  1.58s/it]
                                                        

 51%|█████▏    | 6170/12000 [3:39:19<2:33:27,  1.58s/it]
 51%|█████▏    | 6171/12000 [3:39:20<2:30:49,  1.55s/it]
                                                        

 51%|█████▏    | 6171/12000 [3:39:20<2:30:49,  1.55s/it]
 51%|█████▏    | 6172/12000 [3:39:23<3:09:54,  1.96s/it]
                                                        

 51%|█████▏    | 6172/12000 [3:39:23<3:09:54,  1.96s/it]
 51%|█████▏    | 6173/12000 [3:39:26<3:32:54,  2.19s/it]
                                                        

 51%|█████▏    | 6173/12000 [3:39:26<3:32:54,  2.19s/it]
 51%|█████▏    | 6174/12000 [3:39:30<4:19:02,  2.67s/it]
                                                        

 51%|█████▏    | 6174/12000 [3:39:30<4:19:02,  2.67s/it]
 51%|█████▏    | 6175/12000 [3:39:32<4:06:36,  2.54s/it]
                                                        

 51%|█████▏    | 6175/12000 [3:39:32<4:06:36,  2.54s/it]
 51%|█████▏    | 6176/12000 [3:39:34<3:34:04,  2.21s/it]
                                                        

 51%|█████▏    | 6176/12000 [3:39:34<3:34:04,  2.21s/it]
 51%|█████▏    | 6177/12000 [3:39:35<3:10:30,  1.96s/it]
                                                        

 51%|█████▏    | 6177/12000 [3:39:35<3:10:30,  1.96s/it]
 51%|█████▏    | 6178/12000 [3:39:40<4:41:26,  2.90s/it]
                                                        

 51%|█████▏    | 6178/12000 [3:39:40<4:41:26,  2.90s/it]
 51%|█████▏    | 6179/12000 [3:39:42<4:00:36,  2.48s/it]
                                                        

 51%|█████▏    | 6179/12000 [3:39:42<4:00:36,  2.48s/it]
 52%|█████▏    | 6180/12000 [3:39:44<4:06:17,  2.54s/it]
                                                        

 52%|█████▏    | 6180/12000 [3:39:44<4:06:17,  2.54s/it]
 52%|█████▏    | 6181/12000 [3:39:47<4:09:39,  2.57s/it]
                                                        

 52%|█████▏    | 6181/12000 [3:39:47<4:09:39,  2.57s/it]
 52%|█████▏    | 6182/12000 [3:39:49<3:43:12,  2.30s/it]
                                                        

 52%|█████▏    | 6182/12000 [3:39:49<3:43:12,  2.30s/it]
 52%|█████▏    | 6183/12000 [3:39:50<3:20:44,  2.07s/it]
                                                        

 52%|█████▏    | 6183/12000 [3:39:50<3:20:44,  2.07s/it]
 52%|█████▏    | 6184/12000 [3:39:52<3:12:43,  1.99s/it]
                                                        

 52%|█████▏    | 6184/12000 [3:39:52<3:12:43,  1.99s/it]
 52%|█████▏    | 6185/12000 [3:39:53<3:01:42,  1.87s/it]
                                                        

 52%|█████▏    | 6185/12000 [3:39:53<3:01:42,  1.87s/it]
 52%|█████▏    | 6186/12000 [3:39:55<2:52:07,  1.78s/it]
                                                        

 52%|█████▏    | 6186/12000 [3:39:55<2:52:07,  1.78s/it]
 52%|█████▏    | 6187/12000 [3:39:57<2:57:59,  1.84s/it]
                                                        

 52%|█████▏    | 6187/12000 [3:39:57<2:57:59,  1.84s/it]
 52%|█████▏    | 6188/12000 [3:39:59<2:57:49,  1.84s/it]
                                                        

 52%|█████▏    | 6188/12000 [3:39:59<2:57:49,  1.84s/it]
 52%|█████▏    | 6189/12000 [3:40:00<2:48:44,  1.74s/it]
                                                        

 52%|█████▏    | 6189/12000 [3:40:00<2:48:44,  1.74s/it]
 52%|█████▏    | 6190/12000 [3:40:02<2:40:54,  1.66s/it]
                                                        

 52%|█████▏    | 6190/12000 [3:40:02<2:40:54,  1.66s/it]
 52%|█████▏    | 6191/12000 [3:40:04<2:55:35,  1.81s/it]
                                                        

 52%|█████▏    | 6191/12000 [3:40:04<2:55:35,  1.81s/it]
 52%|█████▏    | 6192/12000 [3:40:05<2:43:14,  1.69s/it]
                                                        

 52%|█████▏    | 6192/12000 [3:40:05<2:43:14,  1.69s/it]
 52%|█████▏    | 6193/12000 [3:40:07<2:41:53,  1.67s/it]
                                                        

 52%|█████▏    | 6193/12000 [3:40:07<2:41:53,  1.67s/it]
 52%|█████▏    | 6194/12000 [3:40:09<2:52:09,  1.78s/it]
                                                        

 52%|█████▏    | 6194/12000 [3:40:09<2:52:09,  1.78s/it]
 52%|█████▏    | 6195/12000 [3:40:11<2:48:28,  1.74s/it]
                                                        

 52%|█████▏    | 6195/12000 [3:40:11<2:48:28,  1.74s/it]
 52%|█████▏    | 6196/12000 [3:40:14<3:35:02,  2.22s/it]
                                                        

 52%|█████▏    | 6196/12000 [3:40:14<3:35:02,  2.22s/it]
 52%|█████▏    | 6197/12000 [3:40:16<3:15:57,  2.03s/it]
                                                        

 52%|█████▏    | 6197/12000 [3:40:16<3:15:57,  2.03s/it]
 52%|█████▏    | 6198/12000 [3:40:17<2:59:58,  1.86s/it]
                                                        

 52%|█████▏    | 6198/12000 [3:40:17<2:59:58,  1.86s/it]
 52%|█████▏    | 6199/12000 [3:40:19<3:02:32,  1.89s/it]
                                                        

 52%|█████▏    | 6199/12000 [3:40:19<3:02:32,  1.89s/it]
 52%|█████▏    | 6200/12000 [3:40:21<2:54:53,  1.81s/it]
                                                        

 52%|█████▏    | 6200/12000 [3:40:21<2:54:53,  1.81s/it]
 52%|█████▏    | 6201/12000 [3:40:26<4:35:27,  2.85s/it]
                                                        

 52%|█████▏    | 6201/12000 [3:40:26<4:35:27,  2.85s/it]
 52%|█████▏    | 6202/12000 [3:40:27<3:55:35,  2.44s/it]
                                                        

 52%|█████▏    | 6202/12000 [3:40:27<3:55:35,  2.44s/it]
 52%|█████▏    | 6203/12000 [3:40:29<3:27:07,  2.14s/it]
                                                        

 52%|█████▏    | 6203/12000 [3:40:29<3:27:07,  2.14s/it]
 52%|█████▏    | 6204/12000 [3:40:32<3:57:06,  2.45s/it]
                                                        

 52%|█████▏    | 6204/12000 [3:40:32<3:57:06,  2.45s/it]
 52%|█████▏    | 6205/12000 [3:40:34<3:38:59,  2.27s/it]
                                                        

 52%|█████▏    | 6205/12000 [3:40:34<3:38:59,  2.27s/it]
 52%|█████▏    | 6206/12000 [3:40:35<3:19:03,  2.06s/it]
                                                        

 52%|█████▏    | 6206/12000 [3:40:35<3:19:03,  2.06s/it]
 52%|█████▏    | 6207/12000 [3:40:37<3:03:30,  1.90s/it]
                                                        

 52%|█████▏    | 6207/12000 [3:40:37<3:03:30,  1.90s/it]
 52%|█████▏    | 6208/12000 [3:40:39<3:01:57,  1.88s/it]
                                                        
{'loss': '0.6389', 'grad_norm': '0.3357', 'learning_rate': '0.0001039', 'entropy': '0.7457', 'num_tokens': '3.328e+07', 'mean_token_accuracy': '0.8059', 'epoch': '0.05452'}
{'loss': '0.75', 'grad_norm': '0.3569', 'learning_rate': '0.0001038', 'entropy': '0.819', 'num_tokens': '3.329e+07', 'mean_token_accuracy': '0.7933', 'epoch': '0.05453'}
{'loss': '1.02', 'grad_norm': '0.5008', 'learning_rate': '0.0001038', 'entropy': '1.027', 'num_tokens': '3.329e+07', 'mean_token_accuracy': '0.76', 'epoch': '0.05454'}
{'loss': '0.8139', 'grad_norm': '0.4081', 'learning_rate': '0.0001038', 'entropy': '0.8338', 'num_tokens': '3.33e+07', 'mean_token_accuracy': '0.7927', 'epoch': '0.05455'}
{'loss': '0.607', 'grad_norm': '0.2737', 'learning_rate': '0.0001037', 'entropy': '0.6291', 'num_tokens': '3.33e+07', 'mean_token_accuracy': '0.8463', 'epoch': '0.05456'}
{'loss': '0.8686', 'grad_norm': '0.3817', 'learning_rate': '0.0001037', 'entropy': '0.8252', 'num_tokens': '3.331e+07', 'mean_token_accuracy': '0.7982', 'epoch': '0.05457'}
{'loss': '1.092', 'grad_norm': '0.4264', 'learning_rate': '0.0001037', 'entropy': '1.101', 'num_tokens': '3.331e+07', 'mean_token_accuracy': '0.7425', 'epoch': '0.05458'}
{'loss': '0.6064', 'grad_norm': '0.2968', 'learning_rate': '0.0001037', 'entropy': '0.65', 'num_tokens': '3.332e+07', 'mean_token_accuracy': '0.824', 'epoch': '0.05459'}
{'loss': '0.9057', 'grad_norm': '0.4291', 'learning_rate': '0.0001036', 'entropy': '0.878', 'num_tokens': '3.332e+07', 'mean_token_accuracy': '0.777', 'epoch': '0.05459'}
{'loss': '0.7107', 'grad_norm': '0.3529', 'learning_rate': '0.0001036', 'entropy': '0.7498', 'num_tokens': '3.333e+07', 'mean_token_accuracy': '0.797', 'epoch': '0.0546'}
{'loss': '0.7667', 'grad_norm': '0.3137', 'learning_rate': '0.0001036', 'entropy': '0.7898', 'num_tokens': '3.333e+07', 'mean_token_accuracy': '0.7958', 'epoch': '0.05461'}
{'loss': '0.6791', 'grad_norm': '0.3678', 'learning_rate': '0.0001036', 'entropy': '0.5999', 'num_tokens': '3.334e+07', 'mean_token_accuracy': '0.8313', 'epoch': '0.05462'}
{'loss': '0.8734', 'grad_norm': '0.4184', 'learning_rate': '0.0001035', 'entropy': '0.7971', 'num_tokens': '3.334e+07', 'mean_token_accuracy': '0.7981', 'epoch': '0.05463'}
{'loss': '0.9907', 'grad_norm': '0.4606', 'learning_rate': '0.0001035', 'entropy': '0.8775', 'num_tokens': '3.335e+07', 'mean_token_accuracy': '0.7825', 'epoch': '0.05464'}
{'loss': '0.7893', 'grad_norm': '0.4013', 'learning_rate': '0.0001035', 'entropy': '0.7588', 'num_tokens': '3.336e+07', 'mean_token_accuracy': '0.8142', 'epoch': '0.05465'}
{'loss': '0.7793', 'grad_norm': '0.4288', 'learning_rate': '0.0001034', 'entropy': '0.8289', 'num_tokens': '3.336e+07', 'mean_token_accuracy': '0.7865', 'epoch': '0.05466'}
{'loss': '0.7976', 'grad_norm': '0.4446', 'learning_rate': '0.0001034', 'entropy': '0.8091', 'num_tokens': '3.337e+07', 'mean_token_accuracy': '0.7955', 'epoch': '0.05467'}
{'loss': '1.091', 'grad_norm': '0.5748', 'learning_rate': '0.0001034', 'entropy': '0.9048', 'num_tokens': '3.337e+07', 'mean_token_accuracy': '0.7794', 'epoch': '0.05467'}
{'loss': '0.6112', 'grad_norm': '0.36', 'learning_rate': '0.0001034', 'entropy': '0.6377', 'num_tokens': '3.338e+07', 'mean_token_accuracy': '0.8217', 'epoch': '0.05468'}
{'loss': '1.116', 'grad_norm': '0.4622', 'learning_rate': '0.0001033', 'entropy': '1.121', 'num_tokens': '3.339e+07', 'mean_token_accuracy': '0.7355', 'epoch': '0.05469'}
{'loss': '0.5723', 'grad_norm': '0.2552', 'learning_rate': '0.0001033', 'entropy': '0.5763', 'num_tokens': '3.339e+07', 'mean_token_accuracy': '0.8491', 'epoch': '0.0547'}
{'loss': '0.7157', 'grad_norm': '0.458', 'learning_rate': '0.0001033', 'entropy': '0.7127', 'num_tokens': '3.34e+07', 'mean_token_accuracy': '0.8121', 'epoch': '0.05471'}
{'loss': '0.8067', 'grad_norm': '0.4551', 'learning_rate': '0.0001033', 'entropy': '0.7777', 'num_tokens': '3.34e+07', 'mean_token_accuracy': '0.8035', 'epoch': '0.05472'}
{'loss': '0.6139', 'grad_norm': '0.3782', 'learning_rate': '0.0001032', 'entropy': '0.6366', 'num_tokens': '3.341e+07', 'mean_token_accuracy': '0.8366', 'epoch': '0.05473'}
{'loss': '0.5798', 'grad_norm': '0.3222', 'learning_rate': '0.0001032', 'entropy': '0.6277', 'num_tokens': '3.341e+07', 'mean_token_accuracy': '0.8404', 'epoch': '0.05474'}
{'loss': '0.6922', 'grad_norm': '0.3727', 'learning_rate': '0.0001032', 'entropy': '0.6988', 'num_tokens': '3.342e+07', 'mean_token_accuracy': '0.818', 'epoch': '0.05475'}
{'loss': '0.7911', 'grad_norm': '0.3048', 'learning_rate': '0.0001031', 'entropy': '0.8927', 'num_tokens': '3.342e+07', 'mean_token_accuracy': '0.7807', 'epoch': '0.05475'}
{'loss': '0.9087', 'grad_norm': '0.3834', 'learning_rate': '0.0001031', 'entropy': '0.8803', 'num_tokens': '3.343e+07', 'mean_token_accuracy': '0.7712', 'epoch': '0.05476'}
{'loss': '0.8661', 'grad_norm': '0.3377', 'learning_rate': '0.0001031', 'entropy': '0.9155', 'num_tokens': '3.343e+07', 'mean_token_accuracy': '0.7721', 'epoch': '0.05477'}
{'loss': '0.6499', 'grad_norm': '0.3615', 'learning_rate': '0.0001031', 'entropy': '0.6841', 'num_tokens': '3.344e+07', 'mean_token_accuracy': '0.826', 'epoch': '0.05478'}
{'loss': '1.376', 'grad_norm': '0.4153', 'learning_rate': '0.000103', 'entropy': '1.203', 'num_tokens': '3.344e+07', 'mean_token_accuracy': '0.7197', 'epoch': '0.05479'}
{'loss': '1.132', 'grad_norm': '0.4466', 'learning_rate': '0.000103', 'entropy': '1.128', 'num_tokens': '3.345e+07', 'mean_token_accuracy': '0.7291', 'epoch': '0.0548'}
{'loss': '0.7518', 'grad_norm': '0.379', 'learning_rate': '0.000103', 'entropy': '0.7414', 'num_tokens': '3.345e+07', 'mean_token_accuracy': '0.8115', 'epoch': '0.05481'}
{'loss': '0.9997', 'grad_norm': '0.4769', 'learning_rate': '0.0001029', 'entropy': '0.9601', 'num_tokens': '3.346e+07', 'mean_token_accuracy': '0.7614', 'epoch': '0.05482'}
{'loss': '0.9349', 'grad_norm': '0.4494', 'learning_rate': '0.0001029', 'entropy': '0.977', 'num_tokens': '3.346e+07', 'mean_token_accuracy': '0.747', 'epoch': '0.05482'}
{'loss': '0.9556', 'grad_norm': '0.4389', 'learning_rate': '0.0001029', 'entropy': '0.9863', 'num_tokens': '3.347e+07', 'mean_token_accuracy': '0.7511', 'epoch': '0.05483'}
{'loss': '0.8067', 'grad_norm': '0.4461', 'learning_rate': '0.0001029', 'entropy': '0.8195', 'num_tokens': '3.347e+07', 'mean_token_accuracy': '0.7802', 'epoch': '0.05484'}
{'loss': '1.192', 'grad_norm': '0.4089', 'learning_rate': '0.0001028', 'entropy': '0.995', 'num_tokens': '3.348e+07', 'mean_token_accuracy': '0.7458', 'epoch': '0.05485'}
{'loss': '0.733', 'grad_norm': '0.3638', 'learning_rate': '0.0001028', 'entropy': '0.7523', 'num_tokens': '3.348e+07', 'mean_token_accuracy': '0.8067', 'epoch': '0.05486'}
{'loss': '1.009', 'grad_norm': '0.4514', 'learning_rate': '0.0001028', 'entropy': '0.9759', 'num_tokens': '3.349e+07', 'mean_token_accuracy': '0.7611', 'epoch': '0.05487'}
{'loss': '1.213', 'grad_norm': '0.4557', 'learning_rate': '0.0001028', 'entropy': '0.9745', 'num_tokens': '3.35e+07', 'mean_token_accuracy': '0.7788', 'epoch': '0.05488'}
{'loss': '1.04', 'grad_norm': '0.4776', 'learning_rate': '0.0001027', 'entropy': '1.05', 'num_tokens': '3.35e+07', 'mean_token_accuracy': '0.7526', 'epoch': '0.05489'}
{'loss': '0.7221', 'grad_norm': '0.3027', 'learning_rate': '0.0001027', 'entropy': '0.7774', 'num_tokens': '3.351e+07', 'mean_token_accuracy': '0.8038', 'epoch': '0.0549'}
{'loss': '0.8404', 'grad_norm': '0.3779', 'learning_rate': '0.0001027', 'entropy': '0.7626', 'num_tokens': '3.351e+07', 'mean_token_accuracy': '0.8065', 'epoch': '0.0549'}
{'loss': '0.899', 'grad_norm': '0.3951', 'learning_rate': '0.0001026', 'entropy': '0.8711', 'num_tokens': '3.352e+07', 'mean_token_accuracy': '0.7945', 'epoch': '0.05491'}
{'loss': '0.9439', 'grad_norm': '0.4283', 'learning_rate': '0.0001026', 'entropy': '0.9553', 'num_tokens': '3.352e+07', 'mean_token_accuracy': '0.7644', 'epoch': '0.05492'}
{'loss': '0.894', 'grad_norm': '0.46', 'learning_rate': '0.0001026', 'entropy': '0.9028', 'num_tokens': '3.353e+07', 'mean_token_accuracy': '0.7643', 'epoch': '0.05493'}

 52%|█████▏    | 6208/12000 [3:40:39<3:01:57,  1.88s/it]
 52%|█████▏    | 6209/12000 [3:40:41<2:56:48,  1.83s/it]
                                                        

 52%|█████▏    | 6209/12000 [3:40:41<2:56:48,  1.83s/it]
 52%|█████▏    | 6210/12000 [3:40:42<2:40:40,  1.66s/it]
                                                        

 52%|█████▏    | 6210/12000 [3:40:42<2:40:40,  1.66s/it]
 52%|█████▏    | 6211/12000 [3:40:43<2:36:35,  1.62s/it]
                                                        

 52%|█████▏    | 6211/12000 [3:40:43<2:36:35,  1.62s/it]
 52%|█████▏    | 6212/12000 [3:40:45<2:32:34,  1.58s/it]
                                                        

 52%|█████▏    | 6212/12000 [3:40:45<2:32:34,  1.58s/it]
 52%|█████▏    | 6213/12000 [3:40:46<2:32:46,  1.58s/it]
                                                        

 52%|█████▏    | 6213/12000 [3:40:46<2:32:46,  1.58s/it]
 52%|█████▏    | 6214/12000 [3:40:48<2:29:11,  1.55s/it]
                                                        

 52%|█████▏    | 6214/12000 [3:40:48<2:29:11,  1.55s/it]
 52%|█████▏    | 6215/12000 [3:40:49<2:28:42,  1.54s/it]
                                                        

 52%|█████▏    | 6215/12000 [3:40:49<2:28:42,  1.54s/it]
 52%|█████▏    | 6216/12000 [3:40:51<2:36:08,  1.62s/it]
                                                        

 52%|█████▏    | 6216/12000 [3:40:51<2:36:08,  1.62s/it]
 52%|█████▏    | 6217/12000 [3:40:53<2:37:09,  1.63s/it]
                                                        

 52%|█████▏    | 6217/12000 [3:40:53<2:37:09,  1.63s/it]
 52%|█████▏    | 6218/12000 [3:40:56<3:24:36,  2.12s/it]
                                                        

 52%|█████▏    | 6218/12000 [3:40:56<3:24:36,  2.12s/it]
 52%|█████▏    | 6219/12000 [3:40:58<3:09:07,  1.96s/it]
                                                        

 52%|█████▏    | 6219/12000 [3:40:58<3:09:07,  1.96s/it]
 52%|█████▏    | 6220/12000 [3:40:59<2:59:21,  1.86s/it]
                                                        

 52%|█████▏    | 6220/12000 [3:40:59<2:59:21,  1.86s/it]
 52%|█████▏    | 6221/12000 [3:41:02<3:21:45,  2.09s/it]
                                                        

 52%|█████▏    | 6221/12000 [3:41:02<3:21:45,  2.09s/it]
 52%|█████▏    | 6222/12000 [3:41:03<3:02:04,  1.89s/it]
                                                        

 52%|█████▏    | 6222/12000 [3:41:03<3:02:04,  1.89s/it]
 52%|█████▏    | 6223/12000 [3:41:05<2:49:46,  1.76s/it]
                                                        

 52%|█████▏    | 6223/12000 [3:41:05<2:49:46,  1.76s/it]
 52%|█████▏    | 6224/12000 [3:41:06<2:38:21,  1.65s/it]
                                                        

 52%|█████▏    | 6224/12000 [3:41:06<2:38:21,  1.65s/it]
 52%|█████▏    | 6225/12000 [3:41:09<3:17:41,  2.05s/it]
                                                        

 52%|█████▏    | 6225/12000 [3:41:09<3:17:41,  2.05s/it]
 52%|█████▏    | 6226/12000 [3:41:12<3:30:10,  2.18s/it]
                                                        

 52%|█████▏    | 6226/12000 [3:41:12<3:30:10,  2.18s/it]
 52%|█████▏    | 6227/12000 [3:41:13<3:13:51,  2.01s/it]
                                                        

 52%|█████▏    | 6227/12000 [3:41:13<3:13:51,  2.01s/it]
 52%|█████▏    | 6228/12000 [3:41:19<4:48:21,  3.00s/it]
                                                        

 52%|█████▏    | 6228/12000 [3:41:19<4:48:21,  3.00s/it]
 52%|█████▏    | 6229/12000 [3:41:21<4:35:22,  2.86s/it]
                                                        

 52%|█████▏    | 6229/12000 [3:41:21<4:35:22,  2.86s/it]
 52%|█████▏    | 6230/12000 [3:41:23<4:01:13,  2.51s/it]
                                                        

 52%|█████▏    | 6230/12000 [3:41:23<4:01:13,  2.51s/it]
 52%|█████▏    | 6231/12000 [3:41:24<3:31:33,  2.20s/it]
                                                        

 52%|█████▏    | 6231/12000 [3:41:24<3:31:33,  2.20s/it]
 52%|█████▏    | 6232/12000 [3:41:26<3:23:05,  2.11s/it]
                                                        

 52%|█████▏    | 6232/12000 [3:41:26<3:23:05,  2.11s/it]
 52%|█████▏    | 6233/12000 [3:41:28<3:17:48,  2.06s/it]
                                                        

 52%|█████▏    | 6233/12000 [3:41:28<3:17:48,  2.06s/it]
 52%|█████▏    | 6234/12000 [3:41:30<3:23:56,  2.12s/it]
                                                        

 52%|█████▏    | 6234/12000 [3:41:30<3:23:56,  2.12s/it]
 52%|█████▏    | 6235/12000 [3:41:32<3:01:53,  1.89s/it]
                                                        

 52%|█████▏    | 6235/12000 [3:41:32<3:01:53,  1.89s/it]
 52%|█████▏    | 6236/12000 [3:41:37<4:21:54,  2.73s/it]
                                                        

 52%|█████▏    | 6236/12000 [3:41:37<4:21:54,  2.73s/it]
 52%|█████▏    | 6237/12000 [3:41:39<4:06:49,  2.57s/it]
                                                        

 52%|█████▏    | 6237/12000 [3:41:39<4:06:49,  2.57s/it]
 52%|█████▏    | 6238/12000 [3:41:40<3:35:49,  2.25s/it]
                                                        

 52%|█████▏    | 6238/12000 [3:41:40<3:35:49,  2.25s/it]
 52%|█████▏    | 6239/12000 [3:41:43<3:49:22,  2.39s/it]
                                                        

 52%|█████▏    | 6239/12000 [3:41:43<3:49:22,  2.39s/it]
 52%|█████▏    | 6240/12000 [3:41:46<4:18:28,  2.69s/it]
                                                        

 52%|█████▏    | 6240/12000 [3:41:46<4:18:28,  2.69s/it]
 52%|█████▏    | 6241/12000 [3:41:48<4:00:16,  2.50s/it]
                                                        

 52%|█████▏    | 6241/12000 [3:41:48<4:00:16,  2.50s/it]
 52%|█████▏    | 6242/12000 [3:41:50<3:30:35,  2.19s/it]
                                                        

 52%|█████▏    | 6242/12000 [3:41:50<3:30:35,  2.19s/it]
 52%|█████▏    | 6243/12000 [3:41:52<3:18:39,  2.07s/it]
                                                        

 52%|█████▏    | 6243/12000 [3:41:52<3:18:39,  2.07s/it]
 52%|█████▏    | 6244/12000 [3:41:53<3:01:26,  1.89s/it]
                                                        

 52%|█████▏    | 6244/12000 [3:41:53<3:01:26,  1.89s/it]
 52%|█████▏    | 6245/12000 [3:41:55<2:53:29,  1.81s/it]
                                                        

 52%|█████▏    | 6245/12000 [3:41:55<2:53:29,  1.81s/it]
 52%|█████▏    | 6246/12000 [3:41:57<2:57:14,  1.85s/it]
                                                        

 52%|█████▏    | 6246/12000 [3:41:57<2:57:14,  1.85s/it]
 52%|█████▏    | 6247/12000 [3:41:59<3:20:35,  2.09s/it]
                                                        

 52%|█████▏    | 6247/12000 [3:41:59<3:20:35,  2.09s/it]
 52%|█████▏    | 6248/12000 [3:42:01<3:05:40,  1.94s/it]
                                                        

 52%|█████▏    | 6248/12000 [3:42:01<3:05:40,  1.94s/it]
 52%|█████▏    | 6249/12000 [3:42:03<3:02:27,  1.90s/it]
                                                        

 52%|█████▏    | 6249/12000 [3:42:03<3:02:27,  1.90s/it]
 52%|█████▏    | 6250/12000 [3:42:05<3:09:35,  1.98s/it]
                                                        

 52%|█████▏    | 6250/12000 [3:42:05<3:09:35,  1.98s/it]
 52%|█████▏    | 6251/12000 [3:42:08<3:47:16,  2.37s/it]
                                                        

 52%|█████▏    | 6251/12000 [3:42:08<3:47:16,  2.37s/it]
 52%|█████▏    | 6252/12000 [3:42:10<3:21:40,  2.11s/it]
                                                        

 52%|█████▏    | 6252/12000 [3:42:10<3:21:40,  2.11s/it]
 52%|█████▏    | 6253/12000 [3:42:12<3:27:43,  2.17s/it]
                                                        

 52%|█████▏    | 6253/12000 [3:42:12<3:27:43,  2.17s/it]
 52%|█████▏    | 6254/12000 [3:42:13<3:01:40,  1.90s/it]
                                                        

 52%|█████▏    | 6254/12000 [3:42:13<3:01:40,  1.90s/it]
 52%|█████▏    | 6255/12000 [3:42:15<2:54:33,  1.82s/it]
                                                        
{'loss': '0.9572', 'grad_norm': '0.3333', 'learning_rate': '0.0001026', 'entropy': '0.8843', 'num_tokens': '3.353e+07', 'mean_token_accuracy': '0.7805', 'epoch': '0.05494'}
{'loss': '0.7865', 'grad_norm': '0.3528', 'learning_rate': '0.0001025', 'entropy': '0.8007', 'num_tokens': '3.354e+07', 'mean_token_accuracy': '0.7997', 'epoch': '0.05495'}
{'loss': '0.9662', 'grad_norm': '0.3492', 'learning_rate': '0.0001025', 'entropy': '0.9609', 'num_tokens': '3.354e+07', 'mean_token_accuracy': '0.752', 'epoch': '0.05496'}
{'loss': '0.7847', 'grad_norm': '0.4407', 'learning_rate': '0.0001025', 'entropy': '0.8123', 'num_tokens': '3.355e+07', 'mean_token_accuracy': '0.7977', 'epoch': '0.05497'}
{'loss': '0.7355', 'grad_norm': '0.3433', 'learning_rate': '0.0001025', 'entropy': '0.7065', 'num_tokens': '3.355e+07', 'mean_token_accuracy': '0.8105', 'epoch': '0.05498'}
{'loss': '1.283', 'grad_norm': '0.4683', 'learning_rate': '0.0001024', 'entropy': '1.352', 'num_tokens': '3.356e+07', 'mean_token_accuracy': '0.6795', 'epoch': '0.05498'}
{'loss': '0.921', 'grad_norm': '0.3769', 'learning_rate': '0.0001024', 'entropy': '0.9604', 'num_tokens': '3.356e+07', 'mean_token_accuracy': '0.7601', 'epoch': '0.05499'}
{'loss': '0.6524', 'grad_norm': '0.4219', 'learning_rate': '0.0001024', 'entropy': '0.6841', 'num_tokens': '3.357e+07', 'mean_token_accuracy': '0.8292', 'epoch': '0.055'}
{'loss': '1.039', 'grad_norm': '0.4715', 'learning_rate': '0.0001023', 'entropy': '0.957', 'num_tokens': '3.357e+07', 'mean_token_accuracy': '0.7659', 'epoch': '0.05501'}
{'loss': '0.6259', 'grad_norm': '0.329', 'learning_rate': '0.0001023', 'entropy': '0.6303', 'num_tokens': '3.358e+07', 'mean_token_accuracy': '0.8335', 'epoch': '0.05502'}
{'loss': '0.8244', 'grad_norm': '0.4479', 'learning_rate': '0.0001023', 'entropy': '0.896', 'num_tokens': '3.358e+07', 'mean_token_accuracy': '0.7896', 'epoch': '0.05503'}
{'loss': '0.8118', 'grad_norm': '0.3512', 'learning_rate': '0.0001023', 'entropy': '0.8441', 'num_tokens': '3.358e+07', 'mean_token_accuracy': '0.7979', 'epoch': '0.05504'}
{'loss': '1.002', 'grad_norm': '0.3998', 'learning_rate': '0.0001022', 'entropy': '1.1', 'num_tokens': '3.359e+07', 'mean_token_accuracy': '0.7428', 'epoch': '0.05505'}
{'loss': '1.255', 'grad_norm': '0.3918', 'learning_rate': '0.0001022', 'entropy': '1.188', 'num_tokens': '3.36e+07', 'mean_token_accuracy': '0.7127', 'epoch': '0.05505'}
{'loss': '1.029', 'grad_norm': '0.4396', 'learning_rate': '0.0001022', 'entropy': '1.018', 'num_tokens': '3.36e+07', 'mean_token_accuracy': '0.7612', 'epoch': '0.05506'}
{'loss': '0.8541', 'grad_norm': '0.4312', 'learning_rate': '0.0001021', 'entropy': '0.7935', 'num_tokens': '3.36e+07', 'mean_token_accuracy': '0.7922', 'epoch': '0.05507'}
{'loss': '0.8362', 'grad_norm': '0.365', 'learning_rate': '0.0001021', 'entropy': '0.8531', 'num_tokens': '3.361e+07', 'mean_token_accuracy': '0.788', 'epoch': '0.05508'}
{'loss': '0.8', 'grad_norm': '0.3075', 'learning_rate': '0.0001021', 'entropy': '0.8022', 'num_tokens': '3.362e+07', 'mean_token_accuracy': '0.7996', 'epoch': '0.05509'}
{'loss': '1.135', 'grad_norm': '0.4752', 'learning_rate': '0.0001021', 'entropy': '1.076', 'num_tokens': '3.362e+07', 'mean_token_accuracy': '0.7576', 'epoch': '0.0551'}
{'loss': '0.8545', 'grad_norm': '0.3792', 'learning_rate': '0.000102', 'entropy': '0.895', 'num_tokens': '3.363e+07', 'mean_token_accuracy': '0.7707', 'epoch': '0.05511'}
{'loss': '1.228', 'grad_norm': '0.3952', 'learning_rate': '0.000102', 'entropy': '1.058', 'num_tokens': '3.364e+07', 'mean_token_accuracy': '0.7551', 'epoch': '0.05512'}
{'loss': '0.7371', 'grad_norm': '0.3428', 'learning_rate': '0.000102', 'entropy': '0.589', 'num_tokens': '3.364e+07', 'mean_token_accuracy': '0.8566', 'epoch': '0.05513'}
{'loss': '0.8268', 'grad_norm': '0.3972', 'learning_rate': '0.000102', 'entropy': '0.7705', 'num_tokens': '3.365e+07', 'mean_token_accuracy': '0.8101', 'epoch': '0.05513'}
{'loss': '0.6416', 'grad_norm': '0.3008', 'learning_rate': '0.0001019', 'entropy': '0.6546', 'num_tokens': '3.365e+07', 'mean_token_accuracy': '0.8224', 'epoch': '0.05514'}
{'loss': '0.79', 'grad_norm': '0.3468', 'learning_rate': '0.0001019', 'entropy': '0.8178', 'num_tokens': '3.366e+07', 'mean_token_accuracy': '0.7945', 'epoch': '0.05515'}
{'loss': '0.6241', 'grad_norm': '0.3688', 'learning_rate': '0.0001019', 'entropy': '0.6324', 'num_tokens': '3.367e+07', 'mean_token_accuracy': '0.8378', 'epoch': '0.05516'}
{'loss': '0.8759', 'grad_norm': '0.36', 'learning_rate': '0.0001018', 'entropy': '0.7569', 'num_tokens': '3.367e+07', 'mean_token_accuracy': '0.8018', 'epoch': '0.05517'}
{'loss': '0.9034', 'grad_norm': '0.3646', 'learning_rate': '0.0001018', 'entropy': '0.9598', 'num_tokens': '3.368e+07', 'mean_token_accuracy': '0.7555', 'epoch': '0.05518'}
{'loss': '1.349', 'grad_norm': '0.4531', 'learning_rate': '0.0001018', 'entropy': '1.057', 'num_tokens': '3.368e+07', 'mean_token_accuracy': '0.7681', 'epoch': '0.05519'}
{'loss': '0.9288', 'grad_norm': '0.4045', 'learning_rate': '0.0001018', 'entropy': '0.867', 'num_tokens': '3.369e+07', 'mean_token_accuracy': '0.7842', 'epoch': '0.0552'}
{'loss': '0.8248', 'grad_norm': '0.4177', 'learning_rate': '0.0001017', 'entropy': '0.8182', 'num_tokens': '3.369e+07', 'mean_token_accuracy': '0.7868', 'epoch': '0.05521'}
{'loss': '0.7438', 'grad_norm': '0.4804', 'learning_rate': '0.0001017', 'entropy': '0.672', 'num_tokens': '3.37e+07', 'mean_token_accuracy': '0.8206', 'epoch': '0.05521'}
{'loss': '0.8504', 'grad_norm': '0.4217', 'learning_rate': '0.0001017', 'entropy': '0.8633', 'num_tokens': '3.371e+07', 'mean_token_accuracy': '0.7877', 'epoch': '0.05522'}
{'loss': '0.6833', 'grad_norm': '0.292', 'learning_rate': '0.0001017', 'entropy': '0.7716', 'num_tokens': '3.371e+07', 'mean_token_accuracy': '0.8191', 'epoch': '0.05523'}
{'loss': '0.6203', 'grad_norm': '0.2797', 'learning_rate': '0.0001016', 'entropy': '0.6018', 'num_tokens': '3.372e+07', 'mean_token_accuracy': '0.8436', 'epoch': '0.05524'}
{'loss': '0.575', 'grad_norm': '0.3394', 'learning_rate': '0.0001016', 'entropy': '0.6095', 'num_tokens': '3.372e+07', 'mean_token_accuracy': '0.8378', 'epoch': '0.05525'}
{'loss': '0.6498', 'grad_norm': '0.2773', 'learning_rate': '0.0001016', 'entropy': '0.7085', 'num_tokens': '3.373e+07', 'mean_token_accuracy': '0.8224', 'epoch': '0.05526'}
{'loss': '0.8384', 'grad_norm': '0.3944', 'learning_rate': '0.0001015', 'entropy': '0.823', 'num_tokens': '3.373e+07', 'mean_token_accuracy': '0.7927', 'epoch': '0.05527'}
{'loss': '1.346', 'grad_norm': '0.4502', 'learning_rate': '0.0001015', 'entropy': '1.232', 'num_tokens': '3.374e+07', 'mean_token_accuracy': '0.7138', 'epoch': '0.05528'}
{'loss': '0.783', 'grad_norm': '0.3797', 'learning_rate': '0.0001015', 'entropy': '0.7677', 'num_tokens': '3.374e+07', 'mean_token_accuracy': '0.8138', 'epoch': '0.05528'}
{'loss': '0.686', 'grad_norm': '0.3502', 'learning_rate': '0.0001015', 'entropy': '0.7673', 'num_tokens': '3.375e+07', 'mean_token_accuracy': '0.809', 'epoch': '0.05529'}
{'loss': '0.7902', 'grad_norm': '0.3086', 'learning_rate': '0.0001014', 'entropy': '0.7178', 'num_tokens': '3.375e+07', 'mean_token_accuracy': '0.8233', 'epoch': '0.0553'}
{'loss': '0.9496', 'grad_norm': '0.4227', 'learning_rate': '0.0001014', 'entropy': '0.8893', 'num_tokens': '3.376e+07', 'mean_token_accuracy': '0.7869', 'epoch': '0.05531'}
{'loss': '0.7527', 'grad_norm': '0.3405', 'learning_rate': '0.0001014', 'entropy': '0.9618', 'num_tokens': '3.376e+07', 'mean_token_accuracy': '0.7618', 'epoch': '0.05532'}
{'loss': '1.01', 'grad_norm': '0.4684', 'learning_rate': '0.0001014', 'entropy': '1.102', 'num_tokens': '3.377e+07', 'mean_token_accuracy': '0.7349', 'epoch': '0.05533'}
{'loss': '1.092', 'grad_norm': '0.4001', 'learning_rate': '0.0001013', 'entropy': '0.9865', 'num_tokens': '3.377e+07', 'mean_token_accuracy': '0.7741', 'epoch': '0.05534'}
{'loss': '0.7675', 'grad_norm': '0.383', 'learning_rate': '0.0001013', 'entropy': '0.8651', 'num_tokens': '3.378e+07', 'mean_token_accuracy': '0.7765', 'epoch': '0.05535'}

 52%|█████▏    | 6255/12000 [3:42:15<2:54:33,  1.82s/it]
 52%|█████▏    | 6256/12000 [3:42:17<2:48:56,  1.76s/it]
                                                        

 52%|█████▏    | 6256/12000 [3:42:17<2:48:56,  1.76s/it]
 52%|█████▏    | 6257/12000 [3:42:19<2:55:45,  1.84s/it]
                                                        

 52%|█████▏    | 6257/12000 [3:42:19<2:55:45,  1.84s/it]
 52%|█████▏    | 6258/12000 [3:42:21<3:05:18,  1.94s/it]
                                                        

 52%|█████▏    | 6258/12000 [3:42:21<3:05:18,  1.94s/it]
 52%|█████▏    | 6259/12000 [3:42:22<3:00:45,  1.89s/it]
                                                        

 52%|█████▏    | 6259/12000 [3:42:22<3:00:45,  1.89s/it]
 52%|█████▏    | 6260/12000 [3:42:24<2:45:09,  1.73s/it]
                                                        

 52%|█████▏    | 6260/12000 [3:42:24<2:45:09,  1.73s/it]
 52%|█████▏    | 6261/12000 [3:42:26<2:43:24,  1.71s/it]
                                                        

 52%|█████▏    | 6261/12000 [3:42:26<2:43:24,  1.71s/it]
 52%|█████▏    | 6262/12000 [3:42:29<3:25:33,  2.15s/it]
                                                        

 52%|█████▏    | 6262/12000 [3:42:29<3:25:33,  2.15s/it]
 52%|█████▏    | 6263/12000 [3:42:31<3:22:45,  2.12s/it]
                                                        

 52%|█████▏    | 6263/12000 [3:42:31<3:22:45,  2.12s/it]
 52%|█████▏    | 6264/12000 [3:42:35<4:12:35,  2.64s/it]
                                                        

 52%|█████▏    | 6264/12000 [3:42:35<4:12:35,  2.64s/it]
 52%|█████▏    | 6265/12000 [3:42:36<3:38:35,  2.29s/it]
                                                        

 52%|█████▏    | 6265/12000 [3:42:36<3:38:35,  2.29s/it]
 52%|█████▏    | 6266/12000 [3:42:40<4:29:18,  2.82s/it]
                                                        

 52%|█████▏    | 6266/12000 [3:42:40<4:29:18,  2.82s/it]
 52%|█████▏    | 6267/12000 [3:42:42<3:48:43,  2.39s/it]
                                                        

 52%|█████▏    | 6267/12000 [3:42:42<3:48:43,  2.39s/it]
 52%|█████▏    | 6268/12000 [3:42:43<3:25:02,  2.15s/it]
                                                        

 52%|█████▏    | 6268/12000 [3:42:43<3:25:02,  2.15s/it]
 52%|█████▏    | 6269/12000 [3:42:45<3:17:27,  2.07s/it]
                                                        

 52%|█████▏    | 6269/12000 [3:42:45<3:17:27,  2.07s/it]
 52%|█████▏    | 6270/12000 [3:42:46<2:59:22,  1.88s/it]
                                                        

 52%|█████▏    | 6270/12000 [3:42:46<2:59:22,  1.88s/it]
 52%|█████▏    | 6271/12000 [3:42:49<3:15:39,  2.05s/it]
                                                        

 52%|█████▏    | 6271/12000 [3:42:49<3:15:39,  2.05s/it]
 52%|█████▏    | 6272/12000 [3:42:50<3:03:39,  1.92s/it]
                                                        

 52%|█████▏    | 6272/12000 [3:42:50<3:03:39,  1.92s/it]
 52%|█████▏    | 6273/12000 [3:42:52<3:00:53,  1.90s/it]
                                                        

 52%|█████▏    | 6273/12000 [3:42:52<3:00:53,  1.90s/it]
 52%|█████▏    | 6274/12000 [3:42:54<2:49:36,  1.78s/it]
                                                        

 52%|█████▏    | 6274/12000 [3:42:54<2:49:36,  1.78s/it]
 52%|█████▏    | 6275/12000 [3:42:55<2:45:01,  1.73s/it]
                                                        

 52%|█████▏    | 6275/12000 [3:42:55<2:45:01,  1.73s/it]
 52%|█████▏    | 6276/12000 [3:42:57<2:37:47,  1.65s/it]
                                                        

 52%|█████▏    | 6276/12000 [3:42:57<2:37:47,  1.65s/it]
 52%|█████▏    | 6277/12000 [3:43:00<3:07:41,  1.97s/it]
                                                        

 52%|█████▏    | 6277/12000 [3:43:00<3:07:41,  1.97s/it]
 52%|█████▏    | 6278/12000 [3:43:01<2:57:51,  1.87s/it]
                                                        

 52%|█████▏    | 6278/12000 [3:43:01<2:57:51,  1.87s/it]
 52%|█████▏    | 6279/12000 [3:43:03<2:46:59,  1.75s/it]
                                                        

 52%|█████▏    | 6279/12000 [3:43:03<2:46:59,  1.75s/it]
 52%|█████▏    | 6280/12000 [3:43:04<2:42:27,  1.70s/it]
                                                        

 52%|█████▏    | 6280/12000 [3:43:04<2:42:27,  1.70s/it]
 52%|█████▏    | 6281/12000 [3:43:06<2:32:37,  1.60s/it]
                                                        

 52%|█████▏    | 6281/12000 [3:43:06<2:32:37,  1.60s/it]
 52%|█████▏    | 6282/12000 [3:43:09<3:35:36,  2.26s/it]
                                                        

 52%|█████▏    | 6282/12000 [3:43:09<3:35:36,  2.26s/it]
 52%|█████▏    | 6283/12000 [3:43:11<3:18:52,  2.09s/it]
                                                        

 52%|█████▏    | 6283/12000 [3:43:11<3:18:52,  2.09s/it]
 52%|█████▏    | 6284/12000 [3:43:15<4:02:28,  2.55s/it]
                                                        

 52%|█████▏    | 6284/12000 [3:43:15<4:02:28,  2.55s/it]
 52%|█████▏    | 6285/12000 [3:43:18<4:23:38,  2.77s/it]
                                                        

 52%|█████▏    | 6285/12000 [3:43:18<4:23:38,  2.77s/it]
 52%|█████▏    | 6286/12000 [3:43:20<3:55:03,  2.47s/it]
                                                        

 52%|█████▏    | 6286/12000 [3:43:20<3:55:03,  2.47s/it]
 52%|█████▏    | 6287/12000 [3:43:21<3:24:58,  2.15s/it]
                                                        

 52%|█████▏    | 6287/12000 [3:43:21<3:24:58,  2.15s/it]
 52%|█████▏    | 6288/12000 [3:43:23<3:21:51,  2.12s/it]
                                                        

 52%|█████▏    | 6288/12000 [3:43:23<3:21:51,  2.12s/it]
 52%|█████▏    | 6289/12000 [3:43:27<4:08:11,  2.61s/it]
                                                        

 52%|█████▏    | 6289/12000 [3:43:27<4:08:11,  2.61s/it]
 52%|█████▏    | 6290/12000 [3:43:30<4:32:40,  2.87s/it]
                                                        

 52%|█████▏    | 6290/12000 [3:43:30<4:32:40,  2.87s/it]
 52%|█████▏    | 6291/12000 [3:43:32<3:58:42,  2.51s/it]
                                                        

 52%|█████▏    | 6291/12000 [3:43:32<3:58:42,  2.51s/it]
 52%|█████▏    | 6292/12000 [3:43:34<3:26:32,  2.17s/it]
                                                        

 52%|█████▏    | 6292/12000 [3:43:34<3:26:32,  2.17s/it]
 52%|█████▏    | 6293/12000 [3:43:35<3:14:30,  2.04s/it]
                                                        

 52%|█████▏    | 6293/12000 [3:43:35<3:14:30,  2.04s/it]
 52%|█████▏    | 6294/12000 [3:43:37<3:00:16,  1.90s/it]
                                                        

 52%|█████▏    | 6294/12000 [3:43:37<3:00:16,  1.90s/it]
 52%|█████▏    | 6295/12000 [3:43:40<3:31:22,  2.22s/it]
                                                        

 52%|█████▏    | 6295/12000 [3:43:40<3:31:22,  2.22s/it]
 52%|█████▏    | 6296/12000 [3:43:43<3:48:29,  2.40s/it]
                                                        

 52%|█████▏    | 6296/12000 [3:43:43<3:48:29,  2.40s/it]
 52%|█████▏    | 6297/12000 [3:43:46<4:15:45,  2.69s/it]
                                                        

 52%|█████▏    | 6297/12000 [3:43:46<4:15:45,  2.69s/it]
 52%|█████▏    | 6298/12000 [3:43:48<3:41:29,  2.33s/it]
                                                        

 52%|█████▏    | 6298/12000 [3:43:48<3:41:29,  2.33s/it]
 52%|█████▏    | 6299/12000 [3:43:52<4:32:25,  2.87s/it]
                                                        

 52%|█████▏    | 6299/12000 [3:43:52<4:32:25,  2.87s/it]
 52%|█████▎    | 6300/12000 [3:43:53<3:56:01,  2.48s/it]
                                                        

 52%|█████▎    | 6300/12000 [3:43:53<3:56:01,  2.48s/it]
 53%|█████▎    | 6301/12000 [3:43:59<5:42:27,  3.61s/it]
                                                        

 53%|█████▎    | 6301/12000 [3:43:59<5:42:27,  3.61s/it]
 53%|█████▎    | 6302/12000 [3:44:01<4:55:33,  3.11s/it]
                                                        
{'loss': '0.9152', 'grad_norm': '0.4512', 'learning_rate': '0.0001013', 'entropy': '0.9122', 'num_tokens': '3.378e+07', 'mean_token_accuracy': '0.7628', 'epoch': '0.05536'}
{'loss': '0.6732', 'grad_norm': '0.3572', 'learning_rate': '0.0001012', 'entropy': '0.8067', 'num_tokens': '3.379e+07', 'mean_token_accuracy': '0.7941', 'epoch': '0.05536'}
{'loss': '1.143', 'grad_norm': '0.3746', 'learning_rate': '0.0001012', 'entropy': '0.9614', 'num_tokens': '3.379e+07', 'mean_token_accuracy': '0.7705', 'epoch': '0.05537'}
{'loss': '0.7683', 'grad_norm': '0.3108', 'learning_rate': '0.0001012', 'entropy': '0.6942', 'num_tokens': '3.38e+07', 'mean_token_accuracy': '0.8119', 'epoch': '0.05538'}
{'loss': '0.7601', 'grad_norm': '0.4418', 'learning_rate': '0.0001012', 'entropy': '0.7944', 'num_tokens': '3.381e+07', 'mean_token_accuracy': '0.7965', 'epoch': '0.05539'}
{'loss': '0.6922', 'grad_norm': '0.405', 'learning_rate': '0.0001011', 'entropy': '0.7349', 'num_tokens': '3.381e+07', 'mean_token_accuracy': '0.8113', 'epoch': '0.0554'}
{'loss': '1.179', 'grad_norm': '0.4026', 'learning_rate': '0.0001011', 'entropy': '1.214', 'num_tokens': '3.381e+07', 'mean_token_accuracy': '0.7224', 'epoch': '0.05541'}
{'loss': '0.7014', 'grad_norm': '0.3415', 'learning_rate': '0.0001011', 'entropy': '0.7806', 'num_tokens': '3.382e+07', 'mean_token_accuracy': '0.8017', 'epoch': '0.05542'}
{'loss': '1.076', 'grad_norm': '0.5249', 'learning_rate': '0.000101', 'entropy': '1.011', 'num_tokens': '3.382e+07', 'mean_token_accuracy': '0.7428', 'epoch': '0.05543'}
{'loss': '0.8085', 'grad_norm': '0.2865', 'learning_rate': '0.000101', 'entropy': '0.8652', 'num_tokens': '3.383e+07', 'mean_token_accuracy': '0.7826', 'epoch': '0.05544'}
{'loss': '0.8594', 'grad_norm': '0.5153', 'learning_rate': '0.000101', 'entropy': '0.907', 'num_tokens': '3.384e+07', 'mean_token_accuracy': '0.7757', 'epoch': '0.05544'}
{'loss': '0.6255', 'grad_norm': '0.2558', 'learning_rate': '0.000101', 'entropy': '0.6084', 'num_tokens': '3.384e+07', 'mean_token_accuracy': '0.8475', 'epoch': '0.05545'}
{'loss': '0.6922', 'grad_norm': '0.3651', 'learning_rate': '0.0001009', 'entropy': '0.6699', 'num_tokens': '3.385e+07', 'mean_token_accuracy': '0.83', 'epoch': '0.05546'}
{'loss': '0.6555', 'grad_norm': '0.3849', 'learning_rate': '0.0001009', 'entropy': '0.6692', 'num_tokens': '3.385e+07', 'mean_token_accuracy': '0.8245', 'epoch': '0.05547'}
{'loss': '1.221', 'grad_norm': '0.3872', 'learning_rate': '0.0001009', 'entropy': '1.117', 'num_tokens': '3.386e+07', 'mean_token_accuracy': '0.7314', 'epoch': '0.05548'}
{'loss': '0.6317', 'grad_norm': '0.3979', 'learning_rate': '0.0001009', 'entropy': '0.6138', 'num_tokens': '3.386e+07', 'mean_token_accuracy': '0.8369', 'epoch': '0.05549'}
{'loss': '0.7196', 'grad_norm': '0.3966', 'learning_rate': '0.0001008', 'entropy': '0.7535', 'num_tokens': '3.387e+07', 'mean_token_accuracy': '0.8155', 'epoch': '0.0555'}
{'loss': '0.6823', 'grad_norm': '0.3774', 'learning_rate': '0.0001008', 'entropy': '0.6891', 'num_tokens': '3.388e+07', 'mean_token_accuracy': '0.8184', 'epoch': '0.05551'}
{'loss': '0.6617', 'grad_norm': '0.3664', 'learning_rate': '0.0001008', 'entropy': '0.7327', 'num_tokens': '3.388e+07', 'mean_token_accuracy': '0.8114', 'epoch': '0.05551'}
{'loss': '0.5682', 'grad_norm': '0.3279', 'learning_rate': '0.0001007', 'entropy': '0.5656', 'num_tokens': '3.389e+07', 'mean_token_accuracy': '0.8445', 'epoch': '0.05552'}
{'loss': '1.066', 'grad_norm': '0.3755', 'learning_rate': '0.0001007', 'entropy': '1.01', 'num_tokens': '3.389e+07', 'mean_token_accuracy': '0.7545', 'epoch': '0.05553'}
{'loss': '0.6922', 'grad_norm': '0.3938', 'learning_rate': '0.0001007', 'entropy': '0.7447', 'num_tokens': '3.389e+07', 'mean_token_accuracy': '0.7999', 'epoch': '0.05554'}
{'loss': '0.8362', 'grad_norm': '0.4095', 'learning_rate': '0.0001007', 'entropy': '0.8075', 'num_tokens': '3.39e+07', 'mean_token_accuracy': '0.7931', 'epoch': '0.05555'}
{'loss': '0.834', 'grad_norm': '0.5315', 'learning_rate': '0.0001006', 'entropy': '0.8362', 'num_tokens': '3.391e+07', 'mean_token_accuracy': '0.7861', 'epoch': '0.05556'}
{'loss': '0.6978', 'grad_norm': '0.3643', 'learning_rate': '0.0001006', 'entropy': '0.7171', 'num_tokens': '3.391e+07', 'mean_token_accuracy': '0.8065', 'epoch': '0.05557'}
{'loss': '0.7124', 'grad_norm': '0.3367', 'learning_rate': '0.0001006', 'entropy': '0.7988', 'num_tokens': '3.392e+07', 'mean_token_accuracy': '0.8011', 'epoch': '0.05558'}
{'loss': '0.7973', 'grad_norm': '0.3527', 'learning_rate': '0.0001006', 'entropy': '0.7969', 'num_tokens': '3.392e+07', 'mean_token_accuracy': '0.7936', 'epoch': '0.05559'}
{'loss': '0.7892', 'grad_norm': '0.3395', 'learning_rate': '0.0001005', 'entropy': '0.7567', 'num_tokens': '3.393e+07', 'mean_token_accuracy': '0.8118', 'epoch': '0.05559'}
{'loss': '0.3923', 'grad_norm': '0.2625', 'learning_rate': '0.0001005', 'entropy': '0.4585', 'num_tokens': '3.393e+07', 'mean_token_accuracy': '0.8794', 'epoch': '0.0556'}
{'loss': '0.9387', 'grad_norm': '0.3964', 'learning_rate': '0.0001005', 'entropy': '0.9605', 'num_tokens': '3.394e+07', 'mean_token_accuracy': '0.7667', 'epoch': '0.05561'}
{'loss': '0.6765', 'grad_norm': '0.3235', 'learning_rate': '0.0001004', 'entropy': '0.668', 'num_tokens': '3.395e+07', 'mean_token_accuracy': '0.824', 'epoch': '0.05562'}
{'loss': '0.7489', 'grad_norm': '0.3824', 'learning_rate': '0.0001004', 'entropy': '0.7651', 'num_tokens': '3.395e+07', 'mean_token_accuracy': '0.812', 'epoch': '0.05563'}
{'loss': '0.7369', 'grad_norm': '0.3463', 'learning_rate': '0.0001004', 'entropy': '0.7685', 'num_tokens': '3.396e+07', 'mean_token_accuracy': '0.812', 'epoch': '0.05564'}
{'loss': '0.7885', 'grad_norm': '0.4039', 'learning_rate': '0.0001004', 'entropy': '0.8344', 'num_tokens': '3.396e+07', 'mean_token_accuracy': '0.8015', 'epoch': '0.05565'}
{'loss': '1.459', 'grad_norm': '0.3946', 'learning_rate': '0.0001003', 'entropy': '1.256', 'num_tokens': '3.397e+07', 'mean_token_accuracy': '0.7164', 'epoch': '0.05566'}
{'loss': '0.9131', 'grad_norm': '0.3096', 'learning_rate': '0.0001003', 'entropy': '0.952', 'num_tokens': '3.398e+07', 'mean_token_accuracy': '0.7457', 'epoch': '0.05567'}
{'loss': '0.6038', 'grad_norm': '0.4149', 'learning_rate': '0.0001003', 'entropy': '0.6362', 'num_tokens': '3.398e+07', 'mean_token_accuracy': '0.8266', 'epoch': '0.05567'}
{'loss': '0.8956', 'grad_norm': '0.454', 'learning_rate': '0.0001002', 'entropy': '0.9006', 'num_tokens': '3.399e+07', 'mean_token_accuracy': '0.7765', 'epoch': '0.05568'}
{'loss': '0.6565', 'grad_norm': '0.3717', 'learning_rate': '0.0001002', 'entropy': '0.6746', 'num_tokens': '3.399e+07', 'mean_token_accuracy': '0.8168', 'epoch': '0.05569'}
{'loss': '0.6553', 'grad_norm': '0.3631', 'learning_rate': '0.0001002', 'entropy': '0.6464', 'num_tokens': '3.4e+07', 'mean_token_accuracy': '0.8211', 'epoch': '0.0557'}
{'loss': '0.7567', 'grad_norm': '0.374', 'learning_rate': '0.0001002', 'entropy': '0.7608', 'num_tokens': '3.4e+07', 'mean_token_accuracy': '0.8069', 'epoch': '0.05571'}
{'loss': '0.7767', 'grad_norm': '0.373', 'learning_rate': '0.0001001', 'entropy': '0.5832', 'num_tokens': '3.401e+07', 'mean_token_accuracy': '0.854', 'epoch': '0.05572'}
{'loss': '1.049', 'grad_norm': '0.4143', 'learning_rate': '0.0001001', 'entropy': '0.9815', 'num_tokens': '3.402e+07', 'mean_token_accuracy': '0.7521', 'epoch': '0.05573'}
{'loss': '0.8449', 'grad_norm': '0.4863', 'learning_rate': '0.0001001', 'entropy': '0.7954', 'num_tokens': '3.402e+07', 'mean_token_accuracy': '0.7874', 'epoch': '0.05574'}
{'loss': '0.6236', 'grad_norm': '0.2973', 'learning_rate': '0.0001001', 'entropy': '0.7245', 'num_tokens': '3.403e+07', 'mean_token_accuracy': '0.8089', 'epoch': '0.05575'}
{'loss': '0.5835', 'grad_norm': '0.3655', 'learning_rate': '0.0001', 'entropy': '0.65', 'num_tokens': '3.403e+07', 'mean_token_accuracy': '0.8351', 'epoch': '0.05575'}
{'loss': '1.007', 'grad_norm': '0.3604', 'learning_rate': '0.0001', 'entropy': '0.9553', 'num_tokens': '3.404e+07', 'mean_token_accuracy': '0.7662', 'epoch': '0.05576'}

 53%|█████▎    | 6302/12000 [3:44:01<4:55:33,  3.11s/it]
 53%|█████▎    | 6303/12000 [3:44:03<4:05:21,  2.58s/it]
                                                        

 53%|█████▎    | 6303/12000 [3:44:03<4:05:21,  2.58s/it]
 53%|█████▎    | 6304/12000 [3:44:04<3:31:14,  2.23s/it]
                                                        

 53%|█████▎    | 6304/12000 [3:44:04<3:31:14,  2.23s/it]
 53%|█████▎    | 6305/12000 [3:44:08<4:11:44,  2.65s/it]
                                                        

 53%|█████▎    | 6305/12000 [3:44:08<4:11:44,  2.65s/it]
 53%|█████▎    | 6306/12000 [3:44:09<3:41:25,  2.33s/it]
                                                        

 53%|█████▎    | 6306/12000 [3:44:09<3:41:25,  2.33s/it]
 53%|█████▎    | 6307/12000 [3:44:11<3:17:20,  2.08s/it]
                                                        

 53%|█████▎    | 6307/12000 [3:44:11<3:17:20,  2.08s/it]
 53%|█████▎    | 6308/12000 [3:44:12<3:02:49,  1.93s/it]
                                                        

 53%|█████▎    | 6308/12000 [3:44:12<3:02:49,  1.93s/it]
 53%|█████▎    | 6309/12000 [3:44:14<2:54:07,  1.84s/it]
                                                        

 53%|█████▎    | 6309/12000 [3:44:14<2:54:07,  1.84s/it]
 53%|█████▎    | 6310/12000 [3:44:16<2:56:54,  1.87s/it]
                                                        

 53%|█████▎    | 6310/12000 [3:44:16<2:56:54,  1.87s/it]
 53%|█████▎    | 6311/12000 [3:44:18<3:01:03,  1.91s/it]
                                                        

 53%|█████▎    | 6311/12000 [3:44:18<3:01:03,  1.91s/it]
 53%|█████▎    | 6312/12000 [3:44:20<2:52:48,  1.82s/it]
                                                        

 53%|█████▎    | 6312/12000 [3:44:20<2:52:48,  1.82s/it]
 53%|█████▎    | 6313/12000 [3:44:21<2:45:15,  1.74s/it]
                                                        

 53%|█████▎    | 6313/12000 [3:44:21<2:45:15,  1.74s/it]
 53%|█████▎    | 6314/12000 [3:44:23<2:35:46,  1.64s/it]
                                                        

 53%|█████▎    | 6314/12000 [3:44:23<2:35:46,  1.64s/it]
 53%|█████▎    | 6315/12000 [3:44:25<3:02:08,  1.92s/it]
                                                        

 53%|█████▎    | 6315/12000 [3:44:25<3:02:08,  1.92s/it]
 53%|█████▎    | 6316/12000 [3:44:27<2:51:06,  1.81s/it]
                                                        

 53%|█████▎    | 6316/12000 [3:44:27<2:51:06,  1.81s/it]
 53%|█████▎    | 6317/12000 [3:44:29<2:52:33,  1.82s/it]
                                                        

 53%|█████▎    | 6317/12000 [3:44:29<2:52:33,  1.82s/it]
 53%|█████▎    | 6318/12000 [3:44:31<3:18:08,  2.09s/it]
                                                        

 53%|█████▎    | 6318/12000 [3:44:31<3:18:08,  2.09s/it]
 53%|█████▎    | 6319/12000 [3:44:33<3:15:10,  2.06s/it]
                                                        

 53%|█████▎    | 6319/12000 [3:44:33<3:15:10,  2.06s/it]
 53%|█████▎    | 6320/12000 [3:44:35<3:00:02,  1.90s/it]
                                                        

 53%|█████▎    | 6320/12000 [3:44:35<3:00:02,  1.90s/it]
 53%|█████▎    | 6321/12000 [3:44:37<3:09:49,  2.01s/it]
                                                        

 53%|█████▎    | 6321/12000 [3:44:37<3:09:49,  2.01s/it]
 53%|█████▎    | 6322/12000 [3:44:38<2:51:32,  1.81s/it]
                                                        

 53%|█████▎    | 6322/12000 [3:44:38<2:51:32,  1.81s/it]
 53%|█████▎    | 6323/12000 [3:44:40<2:50:34,  1.80s/it]
                                                        

 53%|█████▎    | 6323/12000 [3:44:40<2:50:34,  1.80s/it]
 53%|█████▎    | 6324/12000 [3:44:42<2:37:18,  1.66s/it]
                                                        

 53%|█████▎    | 6324/12000 [3:44:42<2:37:18,  1.66s/it]
 53%|█████▎    | 6325/12000 [3:44:44<2:48:26,  1.78s/it]
                                                        

 53%|█████▎    | 6325/12000 [3:44:44<2:48:26,  1.78s/it]
 53%|█████▎    | 6326/12000 [3:44:45<2:42:19,  1.72s/it]
                                                        

 53%|█████▎    | 6326/12000 [3:44:45<2:42:19,  1.72s/it]
 53%|█████▎    | 6327/12000 [3:44:49<3:43:02,  2.36s/it]
                                                        

 53%|█████▎    | 6327/12000 [3:44:49<3:43:02,  2.36s/it]
 53%|█████▎    | 6328/12000 [3:44:52<3:49:21,  2.43s/it]
                                                        

 53%|█████▎    | 6328/12000 [3:44:52<3:49:21,  2.43s/it]
 53%|█████▎    | 6329/12000 [3:44:55<4:21:03,  2.76s/it]
                                                        

 53%|█████▎    | 6329/12000 [3:44:55<4:21:03,  2.76s/it]
 53%|█████▎    | 6330/12000 [3:44:57<3:48:39,  2.42s/it]
                                                        

 53%|█████▎    | 6330/12000 [3:44:57<3:48:39,  2.42s/it]
 53%|█████▎    | 6331/12000 [3:44:58<3:22:25,  2.14s/it]
                                                        

 53%|█████▎    | 6331/12000 [3:44:58<3:22:25,  2.14s/it]
 53%|█████▎    | 6332/12000 [3:45:00<3:02:19,  1.93s/it]
                                                        

 53%|█████▎    | 6332/12000 [3:45:00<3:02:19,  1.93s/it]
 53%|█████▎    | 6333/12000 [3:45:01<2:57:33,  1.88s/it]
                                                        

 53%|█████▎    | 6333/12000 [3:45:01<2:57:33,  1.88s/it]
 53%|█████▎    | 6334/12000 [3:45:03<2:38:57,  1.68s/it]
                                                        

 53%|█████▎    | 6334/12000 [3:45:03<2:38:57,  1.68s/it]
 53%|█████▎    | 6335/12000 [3:45:04<2:36:05,  1.65s/it]
                                                        

 53%|█████▎    | 6335/12000 [3:45:04<2:36:05,  1.65s/it]
 53%|█████▎    | 6336/12000 [3:45:06<2:38:55,  1.68s/it]
                                                        

 53%|█████▎    | 6336/12000 [3:45:06<2:38:55,  1.68s/it]
 53%|█████▎    | 6337/12000 [3:45:08<2:37:03,  1.66s/it]
                                                        

 53%|█████▎    | 6337/12000 [3:45:08<2:37:03,  1.66s/it]
 53%|█████▎    | 6338/12000 [3:45:09<2:29:38,  1.59s/it]
                                                        

 53%|█████▎    | 6338/12000 [3:45:09<2:29:38,  1.59s/it]
 53%|█████▎    | 6339/12000 [3:45:11<2:46:55,  1.77s/it]
                                                        

 53%|█████▎    | 6339/12000 [3:45:11<2:46:55,  1.77s/it]
 53%|█████▎    | 6340/12000 [3:45:14<3:12:27,  2.04s/it]
                                                        

 53%|█████▎    | 6340/12000 [3:45:14<3:12:27,  2.04s/it]
 53%|█████▎    | 6341/12000 [3:45:17<3:29:07,  2.22s/it]
                                                        

 53%|█████▎    | 6341/12000 [3:45:17<3:29:07,  2.22s/it]
 53%|█████▎    | 6342/12000 [3:45:18<3:12:25,  2.04s/it]
                                                        

 53%|█████▎    | 6342/12000 [3:45:18<3:12:25,  2.04s/it]
 53%|█████▎    | 6343/12000 [3:45:20<2:55:31,  1.86s/it]
                                                        

 53%|█████▎    | 6343/12000 [3:45:20<2:55:31,  1.86s/it]
 53%|█████▎    | 6344/12000 [3:45:21<2:42:38,  1.73s/it]
                                                        

 53%|█████▎    | 6344/12000 [3:45:21<2:42:38,  1.73s/it]
 53%|█████▎    | 6345/12000 [3:45:23<3:00:47,  1.92s/it]
                                                        

 53%|█████▎    | 6345/12000 [3:45:23<3:00:47,  1.92s/it]
 53%|█████▎    | 6346/12000 [3:45:27<3:50:20,  2.44s/it]
                                                        

 53%|█████▎    | 6346/12000 [3:45:27<3:50:20,  2.44s/it]
 53%|█████▎    | 6347/12000 [3:45:30<4:00:23,  2.55s/it]
                                                        

 53%|█████▎    | 6347/12000 [3:45:30<4:00:23,  2.55s/it]
 53%|█████▎    | 6348/12000 [3:45:31<3:30:10,  2.23s/it]
                                                        

 53%|█████▎    | 6348/12000 [3:45:31<3:30:10,  2.23s/it]
 53%|█████▎    | 6349/12000 [3:45:35<4:09:31,  2.65s/it]
                                                        
{'loss': '1.141', 'grad_norm': '0.4067', 'learning_rate': '9.997e-05', 'entropy': '1.074', 'num_tokens': '3.405e+07', 'mean_token_accuracy': '0.736', 'epoch': '0.05577'}
{'loss': '0.827', 'grad_norm': '0.4498', 'learning_rate': '9.994e-05', 'entropy': '0.8795', 'num_tokens': '3.405e+07', 'mean_token_accuracy': '0.7799', 'epoch': '0.05578'}
{'loss': '0.6001', 'grad_norm': '0.2988', 'learning_rate': '9.992e-05', 'entropy': '0.637', 'num_tokens': '3.406e+07', 'mean_token_accuracy': '0.8262', 'epoch': '0.05579'}
{'loss': '0.8852', 'grad_norm': '0.4175', 'learning_rate': '9.989e-05', 'entropy': '0.8757', 'num_tokens': '3.406e+07', 'mean_token_accuracy': '0.7813', 'epoch': '0.0558'}
{'loss': '0.7268', 'grad_norm': '0.5686', 'learning_rate': '9.986e-05', 'entropy': '0.7262', 'num_tokens': '3.406e+07', 'mean_token_accuracy': '0.8043', 'epoch': '0.05581'}
{'loss': '0.8331', 'grad_norm': '0.3578', 'learning_rate': '9.983e-05', 'entropy': '0.822', 'num_tokens': '3.407e+07', 'mean_token_accuracy': '0.7933', 'epoch': '0.05582'}
{'loss': '1.036', 'grad_norm': '0.4463', 'learning_rate': '9.981e-05', 'entropy': '1.081', 'num_tokens': '3.407e+07', 'mean_token_accuracy': '0.7273', 'epoch': '0.05582'}
{'loss': '0.7916', 'grad_norm': '0.444', 'learning_rate': '9.978e-05', 'entropy': '0.8153', 'num_tokens': '3.408e+07', 'mean_token_accuracy': '0.7864', 'epoch': '0.05583'}
{'loss': '0.9754', 'grad_norm': '0.4511', 'learning_rate': '9.975e-05', 'entropy': '0.9395', 'num_tokens': '3.408e+07', 'mean_token_accuracy': '0.7461', 'epoch': '0.05584'}
{'loss': '0.8751', 'grad_norm': '0.4573', 'learning_rate': '9.972e-05', 'entropy': '0.8608', 'num_tokens': '3.409e+07', 'mean_token_accuracy': '0.7971', 'epoch': '0.05585'}
{'loss': '0.6853', 'grad_norm': '0.3423', 'learning_rate': '9.97e-05', 'entropy': '0.7421', 'num_tokens': '3.409e+07', 'mean_token_accuracy': '0.8068', 'epoch': '0.05586'}
{'loss': '0.8188', 'grad_norm': '0.3724', 'learning_rate': '9.967e-05', 'entropy': '0.8454', 'num_tokens': '3.41e+07', 'mean_token_accuracy': '0.7786', 'epoch': '0.05587'}
{'loss': '0.7296', 'grad_norm': '0.3272', 'learning_rate': '9.964e-05', 'entropy': '0.813', 'num_tokens': '3.41e+07', 'mean_token_accuracy': '0.8017', 'epoch': '0.05588'}
{'loss': '1.262', 'grad_norm': '0.4589', 'learning_rate': '9.961e-05', 'entropy': '1.165', 'num_tokens': '3.411e+07', 'mean_token_accuracy': '0.7042', 'epoch': '0.05589'}
{'loss': '0.6374', 'grad_norm': '0.4235', 'learning_rate': '9.959e-05', 'entropy': '0.665', 'num_tokens': '3.411e+07', 'mean_token_accuracy': '0.8231', 'epoch': '0.0559'}
{'loss': '0.899', 'grad_norm': '0.3597', 'learning_rate': '9.956e-05', 'entropy': '0.8616', 'num_tokens': '3.412e+07', 'mean_token_accuracy': '0.7856', 'epoch': '0.0559'}
{'loss': '0.7663', 'grad_norm': '0.3491', 'learning_rate': '9.953e-05', 'entropy': '0.6929', 'num_tokens': '3.413e+07', 'mean_token_accuracy': '0.8146', 'epoch': '0.05591'}
{'loss': '1.071', 'grad_norm': '0.3856', 'learning_rate': '9.95e-05', 'entropy': '1.091', 'num_tokens': '3.413e+07', 'mean_token_accuracy': '0.7314', 'epoch': '0.05592'}
{'loss': '0.9029', 'grad_norm': '0.416', 'learning_rate': '9.948e-05', 'entropy': '0.887', 'num_tokens': '3.414e+07', 'mean_token_accuracy': '0.7905', 'epoch': '0.05593'}
{'loss': '0.81', 'grad_norm': '0.3827', 'learning_rate': '9.945e-05', 'entropy': '0.9503', 'num_tokens': '3.414e+07', 'mean_token_accuracy': '0.7566', 'epoch': '0.05594'}
{'loss': '0.778', 'grad_norm': '0.5662', 'learning_rate': '9.942e-05', 'entropy': '0.8101', 'num_tokens': '3.415e+07', 'mean_token_accuracy': '0.8015', 'epoch': '0.05595'}
{'loss': '0.8615', 'grad_norm': '0.4279', 'learning_rate': '9.939e-05', 'entropy': '0.8081', 'num_tokens': '3.415e+07', 'mean_token_accuracy': '0.7877', 'epoch': '0.05596'}
{'loss': '0.7541', 'grad_norm': '0.3916', 'learning_rate': '9.937e-05', 'entropy': '0.817', 'num_tokens': '3.415e+07', 'mean_token_accuracy': '0.7887', 'epoch': '0.05597'}
{'loss': '0.6196', 'grad_norm': '0.3495', 'learning_rate': '9.934e-05', 'entropy': '0.6229', 'num_tokens': '3.416e+07', 'mean_token_accuracy': '0.8329', 'epoch': '0.05598'}
{'loss': '0.8259', 'grad_norm': '0.3808', 'learning_rate': '9.931e-05', 'entropy': '0.8512', 'num_tokens': '3.416e+07', 'mean_token_accuracy': '0.7875', 'epoch': '0.05598'}
{'loss': '1.017', 'grad_norm': '0.4238', 'learning_rate': '9.928e-05', 'entropy': '1.034', 'num_tokens': '3.417e+07', 'mean_token_accuracy': '0.7594', 'epoch': '0.05599'}
{'loss': '1.034', 'grad_norm': '0.4868', 'learning_rate': '9.926e-05', 'entropy': '0.9208', 'num_tokens': '3.418e+07', 'mean_token_accuracy': '0.7788', 'epoch': '0.056'}
{'loss': '0.9692', 'grad_norm': '0.4099', 'learning_rate': '9.923e-05', 'entropy': '0.8862', 'num_tokens': '3.418e+07', 'mean_token_accuracy': '0.7719', 'epoch': '0.05601'}
{'loss': '0.7218', 'grad_norm': '0.4435', 'learning_rate': '9.92e-05', 'entropy': '0.7588', 'num_tokens': '3.419e+07', 'mean_token_accuracy': '0.7942', 'epoch': '0.05602'}
{'loss': '0.5087', 'grad_norm': '0.3028', 'learning_rate': '9.917e-05', 'entropy': '0.5084', 'num_tokens': '3.419e+07', 'mean_token_accuracy': '0.8708', 'epoch': '0.05603'}
{'loss': '0.715', 'grad_norm': '0.43', 'learning_rate': '9.915e-05', 'entropy': '0.804', 'num_tokens': '3.42e+07', 'mean_token_accuracy': '0.7904', 'epoch': '0.05604'}
{'loss': '1.009', 'grad_norm': '0.3819', 'learning_rate': '9.912e-05', 'entropy': '0.8209', 'num_tokens': '3.42e+07', 'mean_token_accuracy': '0.7948', 'epoch': '0.05605'}
{'loss': '0.6931', 'grad_norm': '0.5164', 'learning_rate': '9.909e-05', 'entropy': '0.7414', 'num_tokens': '3.421e+07', 'mean_token_accuracy': '0.8086', 'epoch': '0.05605'}
{'loss': '0.9805', 'grad_norm': '0.4639', 'learning_rate': '9.906e-05', 'entropy': '0.8714', 'num_tokens': '3.421e+07', 'mean_token_accuracy': '0.7834', 'epoch': '0.05606'}
{'loss': '0.9244', 'grad_norm': '0.385', 'learning_rate': '9.904e-05', 'entropy': '0.9018', 'num_tokens': '3.422e+07', 'mean_token_accuracy': '0.7643', 'epoch': '0.05607'}
{'loss': '0.8628', 'grad_norm': '0.456', 'learning_rate': '9.901e-05', 'entropy': '0.8839', 'num_tokens': '3.422e+07', 'mean_token_accuracy': '0.7875', 'epoch': '0.05608'}
{'loss': '1.041', 'grad_norm': '0.4648', 'learning_rate': '9.898e-05', 'entropy': '1.003', 'num_tokens': '3.423e+07', 'mean_token_accuracy': '0.7604', 'epoch': '0.05609'}
{'loss': '0.9762', 'grad_norm': '0.489', 'learning_rate': '9.895e-05', 'entropy': '0.879', 'num_tokens': '3.423e+07', 'mean_token_accuracy': '0.7827', 'epoch': '0.0561'}
{'loss': '0.7886', 'grad_norm': '0.4019', 'learning_rate': '9.893e-05', 'entropy': '0.8465', 'num_tokens': '3.424e+07', 'mean_token_accuracy': '0.7769', 'epoch': '0.05611'}
{'loss': '1.184', 'grad_norm': '0.4492', 'learning_rate': '9.89e-05', 'entropy': '1.194', 'num_tokens': '3.424e+07', 'mean_token_accuracy': '0.7165', 'epoch': '0.05612'}
{'loss': '0.9429', 'grad_norm': '0.3664', 'learning_rate': '9.887e-05', 'entropy': '0.9284', 'num_tokens': '3.425e+07', 'mean_token_accuracy': '0.7583', 'epoch': '0.05613'}
{'loss': '0.7913', 'grad_norm': '0.3688', 'learning_rate': '9.884e-05', 'entropy': '0.7827', 'num_tokens': '3.425e+07', 'mean_token_accuracy': '0.8121', 'epoch': '0.05613'}
{'loss': '0.8766', 'grad_norm': '0.36', 'learning_rate': '9.882e-05', 'entropy': '0.8846', 'num_tokens': '3.426e+07', 'mean_token_accuracy': '0.7986', 'epoch': '0.05614'}
{'loss': '0.9442', 'grad_norm': '0.3988', 'learning_rate': '9.879e-05', 'entropy': '0.994', 'num_tokens': '3.427e+07', 'mean_token_accuracy': '0.764', 'epoch': '0.05615'}
{'loss': '0.8951', 'grad_norm': '0.3507', 'learning_rate': '9.876e-05', 'entropy': '0.9732', 'num_tokens': '3.427e+07', 'mean_token_accuracy': '0.7563', 'epoch': '0.05616'}
{'loss': '0.9004', 'grad_norm': '0.3252', 'learning_rate': '9.873e-05', 'entropy': '0.9394', 'num_tokens': '3.428e+07', 'mean_token_accuracy': '0.7735', 'epoch': '0.05617'}
{'loss': '0.6555', 'grad_norm': '0.3213', 'learning_rate': '9.87e-05', 'entropy': '0.6611', 'num_tokens': '3.429e+07', 'mean_token_accuracy': '0.8252', 'epoch': '0.05618'}

 53%|█████▎    | 6349/12000 [3:45:35<4:09:31,  2.65s/it]
 53%|█████▎    | 6350/12000 [3:45:37<3:39:35,  2.33s/it]
                                                        

 53%|█████▎    | 6350/12000 [3:45:37<3:39:35,  2.33s/it]
 53%|█████▎    | 6351/12000 [3:45:41<4:31:20,  2.88s/it]
                                                        

 53%|█████▎    | 6351/12000 [3:45:41<4:31:20,  2.88s/it]
 53%|█████▎    | 6352/12000 [3:45:42<3:50:29,  2.45s/it]
                                                        

 53%|█████▎    | 6352/12000 [3:45:42<3:50:29,  2.45s/it]
 53%|█████▎    | 6353/12000 [3:45:44<3:39:56,  2.34s/it]
                                                        

 53%|█████▎    | 6353/12000 [3:45:44<3:39:56,  2.34s/it]
 53%|█████▎    | 6354/12000 [3:45:46<3:18:42,  2.11s/it]
                                                        

 53%|█████▎    | 6354/12000 [3:45:46<3:18:42,  2.11s/it]
 53%|█████▎    | 6355/12000 [3:45:48<3:26:23,  2.19s/it]
                                                        

 53%|█████▎    | 6355/12000 [3:45:48<3:26:23,  2.19s/it]
 53%|█████▎    | 6356/12000 [3:45:50<3:05:35,  1.97s/it]
                                                        

 53%|█████▎    | 6356/12000 [3:45:50<3:05:35,  1.97s/it]
 53%|█████▎    | 6357/12000 [3:45:52<3:24:02,  2.17s/it]
                                                        

 53%|█████▎    | 6357/12000 [3:45:52<3:24:02,  2.17s/it]
 53%|█████▎    | 6358/12000 [3:45:56<3:52:56,  2.48s/it]
                                                        

 53%|█████▎    | 6358/12000 [3:45:56<3:52:56,  2.48s/it]
 53%|█████▎    | 6359/12000 [3:45:58<4:05:54,  2.62s/it]
                                                        

 53%|█████▎    | 6359/12000 [3:45:58<4:05:54,  2.62s/it]
 53%|█████▎    | 6360/12000 [3:46:00<3:33:25,  2.27s/it]
                                                        

 53%|█████▎    | 6360/12000 [3:46:00<3:33:25,  2.27s/it]
 53%|█████▎    | 6361/12000 [3:46:02<3:23:26,  2.16s/it]
                                                        

 53%|█████▎    | 6361/12000 [3:46:02<3:23:26,  2.16s/it]
 53%|█████▎    | 6362/12000 [3:46:04<3:25:32,  2.19s/it]
                                                        

 53%|█████▎    | 6362/12000 [3:46:04<3:25:32,  2.19s/it]
 53%|█████▎    | 6363/12000 [3:46:06<3:05:57,  1.98s/it]
                                                        

 53%|█████▎    | 6363/12000 [3:46:06<3:05:57,  1.98s/it]
 53%|█████▎    | 6364/12000 [3:46:08<3:08:04,  2.00s/it]
                                                        

 53%|█████▎    | 6364/12000 [3:46:08<3:08:04,  2.00s/it]
 53%|█████▎    | 6365/12000 [3:46:09<2:52:53,  1.84s/it]
                                                        

 53%|█████▎    | 6365/12000 [3:46:09<2:52:53,  1.84s/it]
 53%|█████▎    | 6366/12000 [3:46:12<3:09:50,  2.02s/it]
                                                        

 53%|█████▎    | 6366/12000 [3:46:12<3:09:50,  2.02s/it]
 53%|█████▎    | 6367/12000 [3:46:13<3:02:43,  1.95s/it]
                                                        

 53%|█████▎    | 6367/12000 [3:46:13<3:02:43,  1.95s/it]
 53%|█████▎    | 6368/12000 [3:46:15<2:48:42,  1.80s/it]
                                                        

 53%|█████▎    | 6368/12000 [3:46:15<2:48:42,  1.80s/it]
 53%|█████▎    | 6369/12000 [3:46:16<2:38:42,  1.69s/it]
                                                        

 53%|█████▎    | 6369/12000 [3:46:16<2:38:42,  1.69s/it]
 53%|█████▎    | 6370/12000 [3:46:18<2:38:09,  1.69s/it]
                                                        

 53%|█████▎    | 6370/12000 [3:46:18<2:38:09,  1.69s/it]
 53%|█████▎    | 6371/12000 [3:46:21<3:21:26,  2.15s/it]
                                                        

 53%|█████▎    | 6371/12000 [3:46:21<3:21:26,  2.15s/it]
 53%|█████▎    | 6372/12000 [3:46:24<3:49:20,  2.45s/it]
                                                        

 53%|█████▎    | 6372/12000 [3:46:24<3:49:20,  2.45s/it]
 53%|█████▎    | 6373/12000 [3:46:26<3:44:09,  2.39s/it]
                                                        

 53%|█████▎    | 6373/12000 [3:46:26<3:44:09,  2.39s/it]
 53%|█████▎    | 6374/12000 [3:46:28<3:27:23,  2.21s/it]
                                                        

 53%|█████▎    | 6374/12000 [3:46:28<3:27:23,  2.21s/it]
 53%|█████▎    | 6375/12000 [3:46:30<3:09:26,  2.02s/it]
                                                        

 53%|█████▎    | 6375/12000 [3:46:30<3:09:26,  2.02s/it]
 53%|█████▎    | 6376/12000 [3:46:33<3:36:33,  2.31s/it]
                                                        

 53%|█████▎    | 6376/12000 [3:46:33<3:36:33,  2.31s/it]
 53%|█████▎    | 6377/12000 [3:46:34<3:17:11,  2.10s/it]
                                                        

 53%|█████▎    | 6377/12000 [3:46:34<3:17:11,  2.10s/it]
 53%|█████▎    | 6378/12000 [3:46:36<2:59:48,  1.92s/it]
                                                        

 53%|█████▎    | 6378/12000 [3:46:36<2:59:48,  1.92s/it]
 53%|█████▎    | 6379/12000 [3:46:38<2:55:12,  1.87s/it]
                                                        

 53%|█████▎    | 6379/12000 [3:46:38<2:55:12,  1.87s/it]
 53%|█████▎    | 6380/12000 [3:46:39<2:41:29,  1.72s/it]
                                                        

 53%|█████▎    | 6380/12000 [3:46:39<2:41:29,  1.72s/it]
 53%|█████▎    | 6381/12000 [3:46:41<2:41:28,  1.72s/it]
                                                        

 53%|█████▎    | 6381/12000 [3:46:41<2:41:28,  1.72s/it]
 53%|█████▎    | 6382/12000 [3:46:43<2:47:50,  1.79s/it]
                                                        

 53%|█████▎    | 6382/12000 [3:46:43<2:47:50,  1.79s/it]
 53%|█████▎    | 6383/12000 [3:46:45<3:06:52,  2.00s/it]
                                                        

 53%|█████▎    | 6383/12000 [3:46:45<3:06:52,  2.00s/it]
 53%|█████▎    | 6384/12000 [3:46:48<3:20:37,  2.14s/it]
                                                        

 53%|█████▎    | 6384/12000 [3:46:48<3:20:37,  2.14s/it]
 53%|█████▎    | 6385/12000 [3:46:50<3:26:01,  2.20s/it]
                                                        

 53%|█████▎    | 6385/12000 [3:46:50<3:26:01,  2.20s/it]
 53%|█████▎    | 6386/12000 [3:46:52<3:15:14,  2.09s/it]
                                                        

 53%|█████▎    | 6386/12000 [3:46:52<3:15:14,  2.09s/it]
 53%|█████▎    | 6387/12000 [3:46:56<4:24:22,  2.83s/it]
                                                        

 53%|█████▎    | 6387/12000 [3:46:56<4:24:22,  2.83s/it]
 53%|█████▎    | 6388/12000 [3:46:58<3:49:59,  2.46s/it]
                                                        

 53%|█████▎    | 6388/12000 [3:46:58<3:49:59,  2.46s/it]
 53%|█████▎    | 6389/12000 [3:47:00<3:26:51,  2.21s/it]
                                                        

 53%|█████▎    | 6389/12000 [3:47:00<3:26:51,  2.21s/it]
 53%|█████▎    | 6390/12000 [3:47:02<3:32:57,  2.28s/it]
                                                        

 53%|█████▎    | 6390/12000 [3:47:02<3:32:57,  2.28s/it]
 53%|█████▎    | 6391/12000 [3:47:04<3:10:24,  2.04s/it]
                                                        

 53%|█████▎    | 6391/12000 [3:47:04<3:10:24,  2.04s/it]
 53%|█████▎    | 6392/12000 [3:47:06<3:15:58,  2.10s/it]
                                                        

 53%|█████▎    | 6392/12000 [3:47:06<3:15:58,  2.10s/it]
 53%|█████▎    | 6393/12000 [3:47:09<3:50:11,  2.46s/it]
                                                        

 53%|█████▎    | 6393/12000 [3:47:09<3:50:11,  2.46s/it]
 53%|█████▎    | 6394/12000 [3:47:11<3:22:52,  2.17s/it]
                                                        

 53%|█████▎    | 6394/12000 [3:47:11<3:22:52,  2.17s/it]
 53%|█████▎    | 6395/12000 [3:47:13<3:15:23,  2.09s/it]
                                                        

 53%|█████▎    | 6395/12000 [3:47:13<3:15:23,  2.09s/it]
 53%|█████▎    | 6396/12000 [3:47:14<2:56:40,  1.89s/it]
                                                        
{'loss': '0.8803', 'grad_norm': '0.3922', 'learning_rate': '9.868e-05', 'entropy': '0.9097', 'num_tokens': '3.429e+07', 'mean_token_accuracy': '0.7789', 'epoch': '0.05619'}
{'loss': '0.7925', 'grad_norm': '0.3988', 'learning_rate': '9.865e-05', 'entropy': '0.759', 'num_tokens': '3.43e+07', 'mean_token_accuracy': '0.813', 'epoch': '0.0562'}
{'loss': '0.9092', 'grad_norm': '0.3472', 'learning_rate': '9.862e-05', 'entropy': '0.9895', 'num_tokens': '3.43e+07', 'mean_token_accuracy': '0.7777', 'epoch': '0.05621'}
{'loss': '0.8235', 'grad_norm': '0.4101', 'learning_rate': '9.859e-05', 'entropy': '0.8217', 'num_tokens': '3.431e+07', 'mean_token_accuracy': '0.7769', 'epoch': '0.05621'}
{'loss': '0.8532', 'grad_norm': '0.4134', 'learning_rate': '9.857e-05', 'entropy': '0.9575', 'num_tokens': '3.431e+07', 'mean_token_accuracy': '0.7709', 'epoch': '0.05622'}
{'loss': '1.091', 'grad_norm': '0.4149', 'learning_rate': '9.854e-05', 'entropy': '1.111', 'num_tokens': '3.432e+07', 'mean_token_accuracy': '0.7235', 'epoch': '0.05623'}
{'loss': '0.5818', 'grad_norm': '0.3623', 'learning_rate': '9.851e-05', 'entropy': '0.6369', 'num_tokens': '3.432e+07', 'mean_token_accuracy': '0.8449', 'epoch': '0.05624'}
{'loss': '1.139', 'grad_norm': '0.4833', 'learning_rate': '9.848e-05', 'entropy': '1.209', 'num_tokens': '3.433e+07', 'mean_token_accuracy': '0.7084', 'epoch': '0.05625'}
{'loss': '1.09', 'grad_norm': '0.3953', 'learning_rate': '9.846e-05', 'entropy': '1.013', 'num_tokens': '3.434e+07', 'mean_token_accuracy': '0.7572', 'epoch': '0.05626'}
{'loss': '0.7012', 'grad_norm': '0.3103', 'learning_rate': '9.843e-05', 'entropy': '0.8417', 'num_tokens': '3.434e+07', 'mean_token_accuracy': '0.7886', 'epoch': '0.05627'}
{'loss': '1.332', 'grad_norm': '0.4085', 'learning_rate': '9.84e-05', 'entropy': '1.21', 'num_tokens': '3.435e+07', 'mean_token_accuracy': '0.7134', 'epoch': '0.05628'}
{'loss': '0.6925', 'grad_norm': '0.3382', 'learning_rate': '9.837e-05', 'entropy': '0.7476', 'num_tokens': '3.435e+07', 'mean_token_accuracy': '0.8106', 'epoch': '0.05628'}
{'loss': '0.6584', 'grad_norm': '0.4533', 'learning_rate': '9.835e-05', 'entropy': '0.6755', 'num_tokens': '3.436e+07', 'mean_token_accuracy': '0.8276', 'epoch': '0.05629'}
{'loss': '0.6052', 'grad_norm': '0.3579', 'learning_rate': '9.832e-05', 'entropy': '0.7125', 'num_tokens': '3.436e+07', 'mean_token_accuracy': '0.8197', 'epoch': '0.0563'}
{'loss': '1.053', 'grad_norm': '0.4656', 'learning_rate': '9.829e-05', 'entropy': '1.113', 'num_tokens': '3.437e+07', 'mean_token_accuracy': '0.7313', 'epoch': '0.05631'}
{'loss': '0.8287', 'grad_norm': '0.357', 'learning_rate': '9.826e-05', 'entropy': '0.9147', 'num_tokens': '3.437e+07', 'mean_token_accuracy': '0.7669', 'epoch': '0.05632'}
{'loss': '0.7334', 'grad_norm': '0.3362', 'learning_rate': '9.824e-05', 'entropy': '0.6971', 'num_tokens': '3.438e+07', 'mean_token_accuracy': '0.8141', 'epoch': '0.05633'}
{'loss': '0.911', 'grad_norm': '0.4902', 'learning_rate': '9.821e-05', 'entropy': '0.8366', 'num_tokens': '3.438e+07', 'mean_token_accuracy': '0.7912', 'epoch': '0.05634'}
{'loss': '1.096', 'grad_norm': '0.3513', 'learning_rate': '9.818e-05', 'entropy': '1.055', 'num_tokens': '3.439e+07', 'mean_token_accuracy': '0.7587', 'epoch': '0.05635'}
{'loss': '0.8086', 'grad_norm': '0.4', 'learning_rate': '9.815e-05', 'entropy': '0.9064', 'num_tokens': '3.439e+07', 'mean_token_accuracy': '0.7739', 'epoch': '0.05636'}
{'loss': '0.8522', 'grad_norm': '0.5155', 'learning_rate': '9.813e-05', 'entropy': '0.9002', 'num_tokens': '3.44e+07', 'mean_token_accuracy': '0.7806', 'epoch': '0.05636'}
{'loss': '0.8737', 'grad_norm': '0.3373', 'learning_rate': '9.81e-05', 'entropy': '0.8904', 'num_tokens': '3.44e+07', 'mean_token_accuracy': '0.7896', 'epoch': '0.05637'}
{'loss': '0.8792', 'grad_norm': '0.4424', 'learning_rate': '9.807e-05', 'entropy': '0.8894', 'num_tokens': '3.441e+07', 'mean_token_accuracy': '0.7849', 'epoch': '0.05638'}
{'loss': '0.7061', 'grad_norm': '0.307', 'learning_rate': '9.804e-05', 'entropy': '0.672', 'num_tokens': '3.442e+07', 'mean_token_accuracy': '0.8394', 'epoch': '0.05639'}
{'loss': '0.7778', 'grad_norm': '0.4008', 'learning_rate': '9.802e-05', 'entropy': '0.7361', 'num_tokens': '3.442e+07', 'mean_token_accuracy': '0.7938', 'epoch': '0.0564'}
{'loss': '1.29', 'grad_norm': '0.4463', 'learning_rate': '9.799e-05', 'entropy': '1.192', 'num_tokens': '3.443e+07', 'mean_token_accuracy': '0.7145', 'epoch': '0.05641'}
{'loss': '0.8519', 'grad_norm': '0.3728', 'learning_rate': '9.796e-05', 'entropy': '0.7859', 'num_tokens': '3.443e+07', 'mean_token_accuracy': '0.8045', 'epoch': '0.05642'}
{'loss': '0.84', 'grad_norm': '0.398', 'learning_rate': '9.793e-05', 'entropy': '0.8263', 'num_tokens': '3.444e+07', 'mean_token_accuracy': '0.7836', 'epoch': '0.05643'}
{'loss': '0.7532', 'grad_norm': '0.378', 'learning_rate': '9.791e-05', 'entropy': '0.7567', 'num_tokens': '3.444e+07', 'mean_token_accuracy': '0.8048', 'epoch': '0.05644'}
{'loss': '0.7237', 'grad_norm': '0.3659', 'learning_rate': '9.788e-05', 'entropy': '0.7156', 'num_tokens': '3.445e+07', 'mean_token_accuracy': '0.8226', 'epoch': '0.05644'}
{'loss': '0.5557', 'grad_norm': '0.3691', 'learning_rate': '9.785e-05', 'entropy': '0.5758', 'num_tokens': '3.445e+07', 'mean_token_accuracy': '0.8502', 'epoch': '0.05645'}
{'loss': '0.7005', 'grad_norm': '0.36', 'learning_rate': '9.782e-05', 'entropy': '0.6826', 'num_tokens': '3.446e+07', 'mean_token_accuracy': '0.8363', 'epoch': '0.05646'}
{'loss': '0.797', 'grad_norm': '0.4729', 'learning_rate': '9.78e-05', 'entropy': '0.7956', 'num_tokens': '3.446e+07', 'mean_token_accuracy': '0.8026', 'epoch': '0.05647'}
{'loss': '0.7207', 'grad_norm': '0.4165', 'learning_rate': '9.777e-05', 'entropy': '0.706', 'num_tokens': '3.447e+07', 'mean_token_accuracy': '0.8047', 'epoch': '0.05648'}
{'loss': '0.7801', 'grad_norm': '0.2959', 'learning_rate': '9.774e-05', 'entropy': '0.8226', 'num_tokens': '3.448e+07', 'mean_token_accuracy': '0.7979', 'epoch': '0.05649'}
{'loss': '0.992', 'grad_norm': '0.3906', 'learning_rate': '9.771e-05', 'entropy': '0.9851', 'num_tokens': '3.448e+07', 'mean_token_accuracy': '0.7602', 'epoch': '0.0565'}
{'loss': '0.5794', 'grad_norm': '0.3215', 'learning_rate': '9.769e-05', 'entropy': '0.6462', 'num_tokens': '3.449e+07', 'mean_token_accuracy': '0.8332', 'epoch': '0.05651'}
{'loss': '0.8804', 'grad_norm': '0.414', 'learning_rate': '9.766e-05', 'entropy': '0.8853', 'num_tokens': '3.449e+07', 'mean_token_accuracy': '0.7866', 'epoch': '0.05652'}
{'loss': '1.136', 'grad_norm': '0.4132', 'learning_rate': '9.763e-05', 'entropy': '0.8231', 'num_tokens': '3.45e+07', 'mean_token_accuracy': '0.804', 'epoch': '0.05652'}
{'loss': '0.8789', 'grad_norm': '0.3619', 'learning_rate': '9.76e-05', 'entropy': '0.8718', 'num_tokens': '3.451e+07', 'mean_token_accuracy': '0.7783', 'epoch': '0.05653'}
{'loss': '0.6925', 'grad_norm': '0.3752', 'learning_rate': '9.758e-05', 'entropy': '0.8166', 'num_tokens': '3.451e+07', 'mean_token_accuracy': '0.7898', 'epoch': '0.05654'}
{'loss': '0.6016', 'grad_norm': '0.337', 'learning_rate': '9.755e-05', 'entropy': '0.591', 'num_tokens': '3.452e+07', 'mean_token_accuracy': '0.8375', 'epoch': '0.05655'}
{'loss': '0.9482', 'grad_norm': '0.4198', 'learning_rate': '9.752e-05', 'entropy': '0.9518', 'num_tokens': '3.452e+07', 'mean_token_accuracy': '0.7767', 'epoch': '0.05656'}
{'loss': '0.708', 'grad_norm': '0.3217', 'learning_rate': '9.749e-05', 'entropy': '0.7163', 'num_tokens': '3.453e+07', 'mean_token_accuracy': '0.8098', 'epoch': '0.05657'}
{'loss': '0.8285', 'grad_norm': '0.4066', 'learning_rate': '9.746e-05', 'entropy': '0.8003', 'num_tokens': '3.453e+07', 'mean_token_accuracy': '0.8003', 'epoch': '0.05658'}
{'loss': '0.6476', 'grad_norm': '0.3996', 'learning_rate': '9.744e-05', 'entropy': '0.6613', 'num_tokens': '3.454e+07', 'mean_token_accuracy': '0.8249', 'epoch': '0.05659'}
{'loss': '0.8798', 'grad_norm': '0.4064', 'learning_rate': '9.741e-05', 'entropy': '0.8795', 'num_tokens': '3.454e+07', 'mean_token_accuracy': '0.7783', 'epoch': '0.05659'}

 53%|█████▎    | 6396/12000 [3:47:14<2:56:40,  1.89s/it]
 53%|█████▎    | 6397/12000 [3:47:16<2:53:54,  1.86s/it]
                                                        

 53%|█████▎    | 6397/12000 [3:47:16<2:53:54,  1.86s/it]
 53%|█████▎    | 6398/12000 [3:47:19<3:40:56,  2.37s/it]
                                                        

 53%|█████▎    | 6398/12000 [3:47:19<3:40:56,  2.37s/it]
 53%|█████▎    | 6399/12000 [3:47:21<3:14:27,  2.08s/it]
                                                        

 53%|█████▎    | 6399/12000 [3:47:21<3:14:27,  2.08s/it]
 53%|█████▎    | 6400/12000 [3:47:22<3:01:13,  1.94s/it]
                                                        

 53%|█████▎    | 6400/12000 [3:47:22<3:01:13,  1.94s/it]
 53%|█████▎    | 6401/12000 [3:47:25<3:24:56,  2.20s/it]
                                                        

 53%|█████▎    | 6401/12000 [3:47:25<3:24:56,  2.20s/it]
 53%|█████▎    | 6402/12000 [3:47:29<3:59:56,  2.57s/it]
                                                        

 53%|█████▎    | 6402/12000 [3:47:29<3:59:56,  2.57s/it]
 53%|█████▎    | 6403/12000 [3:47:31<4:02:40,  2.60s/it]
                                                        

 53%|█████▎    | 6403/12000 [3:47:31<4:02:40,  2.60s/it]
 53%|█████▎    | 6404/12000 [3:47:34<3:57:57,  2.55s/it]
                                                        

 53%|█████▎    | 6404/12000 [3:47:34<3:57:57,  2.55s/it]
 53%|█████▎    | 6405/12000 [3:47:35<3:24:49,  2.20s/it]
                                                        

 53%|█████▎    | 6405/12000 [3:47:35<3:24:49,  2.20s/it]
 53%|█████▎    | 6406/12000 [3:47:36<3:00:35,  1.94s/it]
                                                        

 53%|█████▎    | 6406/12000 [3:47:36<3:00:35,  1.94s/it]
 53%|█████▎    | 6407/12000 [3:47:38<2:45:40,  1.78s/it]
                                                        

 53%|█████▎    | 6407/12000 [3:47:38<2:45:40,  1.78s/it]
 53%|█████▎    | 6408/12000 [3:47:39<2:38:09,  1.70s/it]
                                                        

 53%|█████▎    | 6408/12000 [3:47:39<2:38:09,  1.70s/it]
 53%|█████▎    | 6409/12000 [3:47:41<2:28:48,  1.60s/it]
                                                        

 53%|█████▎    | 6409/12000 [3:47:41<2:28:48,  1.60s/it]
 53%|█████▎    | 6410/12000 [3:47:42<2:22:40,  1.53s/it]
                                                        

 53%|█████▎    | 6410/12000 [3:47:42<2:22:40,  1.53s/it]
 53%|█████▎    | 6411/12000 [3:47:44<2:24:54,  1.56s/it]
                                                        

 53%|█████▎    | 6411/12000 [3:47:44<2:24:54,  1.56s/it]
 53%|█████▎    | 6412/12000 [3:47:45<2:31:49,  1.63s/it]
                                                        

 53%|█████▎    | 6412/12000 [3:47:45<2:31:49,  1.63s/it]
 53%|█████▎    | 6413/12000 [3:47:47<2:29:19,  1.60s/it]
                                                        

 53%|█████▎    | 6413/12000 [3:47:47<2:29:19,  1.60s/it]
 53%|█████▎    | 6414/12000 [3:47:50<3:14:07,  2.09s/it]
                                                        

 53%|█████▎    | 6414/12000 [3:47:50<3:14:07,  2.09s/it]
 53%|█████▎    | 6415/12000 [3:47:54<4:00:04,  2.58s/it]
                                                        

 53%|█████▎    | 6415/12000 [3:47:54<4:00:04,  2.58s/it]
 53%|█████▎    | 6416/12000 [3:47:55<3:31:27,  2.27s/it]
                                                        

 53%|█████▎    | 6416/12000 [3:47:55<3:31:27,  2.27s/it]
 53%|█████▎    | 6417/12000 [3:47:57<3:12:35,  2.07s/it]
                                                        

 53%|█████▎    | 6417/12000 [3:47:57<3:12:35,  2.07s/it]
 53%|█████▎    | 6418/12000 [3:47:59<3:07:48,  2.02s/it]
                                                        

 53%|█████▎    | 6418/12000 [3:47:59<3:07:48,  2.02s/it]
 53%|█████▎    | 6419/12000 [3:48:02<3:25:46,  2.21s/it]
                                                        

 53%|█████▎    | 6419/12000 [3:48:02<3:25:46,  2.21s/it]
 54%|█████▎    | 6420/12000 [3:48:03<3:08:32,  2.03s/it]
                                                        

 54%|█████▎    | 6420/12000 [3:48:03<3:08:32,  2.03s/it]
 54%|█████▎    | 6421/12000 [3:48:05<2:56:08,  1.89s/it]
                                                        

 54%|█████▎    | 6421/12000 [3:48:05<2:56:08,  1.89s/it]
 54%|█████▎    | 6422/12000 [3:48:08<3:21:49,  2.17s/it]
                                                        

 54%|█████▎    | 6422/12000 [3:48:08<3:21:49,  2.17s/it]
 54%|█████▎    | 6423/12000 [3:48:09<3:06:02,  2.00s/it]
                                                        

 54%|█████▎    | 6423/12000 [3:48:09<3:06:02,  2.00s/it]
 54%|█████▎    | 6424/12000 [3:48:13<3:49:49,  2.47s/it]
                                                        

 54%|█████▎    | 6424/12000 [3:48:13<3:49:49,  2.47s/it]
 54%|█████▎    | 6425/12000 [3:48:15<3:42:04,  2.39s/it]
                                                        

 54%|█████▎    | 6425/12000 [3:48:15<3:42:04,  2.39s/it]
 54%|█████▎    | 6426/12000 [3:48:17<3:24:27,  2.20s/it]
                                                        

 54%|█████▎    | 6426/12000 [3:48:17<3:24:27,  2.20s/it]
 54%|█████▎    | 6427/12000 [3:48:19<3:26:18,  2.22s/it]
                                                        

 54%|█████▎    | 6427/12000 [3:48:19<3:26:18,  2.22s/it]
 54%|█████▎    | 6428/12000 [3:48:20<3:04:11,  1.98s/it]
                                                        

 54%|█████▎    | 6428/12000 [3:48:20<3:04:11,  1.98s/it]
 54%|█████▎    | 6429/12000 [3:48:22<2:53:30,  1.87s/it]
                                                        

 54%|█████▎    | 6429/12000 [3:48:22<2:53:30,  1.87s/it]
 54%|█████▎    | 6430/12000 [3:48:24<2:52:20,  1.86s/it]
                                                        

 54%|█████▎    | 6430/12000 [3:48:24<2:52:20,  1.86s/it]
 54%|█████▎    | 6431/12000 [3:48:25<2:44:49,  1.78s/it]
                                                        

 54%|█████▎    | 6431/12000 [3:48:25<2:44:49,  1.78s/it]
 54%|█████▎    | 6432/12000 [3:48:27<2:35:30,  1.68s/it]
                                                        

 54%|█████▎    | 6432/12000 [3:48:27<2:35:30,  1.68s/it]
 54%|█████▎    | 6433/12000 [3:48:28<2:27:44,  1.59s/it]
                                                        

 54%|█████▎    | 6433/12000 [3:48:28<2:27:44,  1.59s/it]
 54%|█████▎    | 6434/12000 [3:48:32<3:19:01,  2.15s/it]
                                                        

 54%|█████▎    | 6434/12000 [3:48:32<3:19:01,  2.15s/it]
 54%|█████▎    | 6435/12000 [3:48:34<3:11:06,  2.06s/it]
                                                        

 54%|█████▎    | 6435/12000 [3:48:34<3:11:06,  2.06s/it]
 54%|█████▎    | 6436/12000 [3:48:35<2:56:42,  1.91s/it]
                                                        

 54%|█████▎    | 6436/12000 [3:48:35<2:56:42,  1.91s/it]
 54%|█████▎    | 6437/12000 [3:48:39<3:37:39,  2.35s/it]
                                                        

 54%|█████▎    | 6437/12000 [3:48:39<3:37:39,  2.35s/it]
 54%|█████▎    | 6438/12000 [3:48:40<3:19:44,  2.15s/it]
                                                        

 54%|█████▎    | 6438/12000 [3:48:40<3:19:44,  2.15s/it]
 54%|█████▎    | 6439/12000 [3:48:43<3:26:45,  2.23s/it]
                                                        

 54%|█████▎    | 6439/12000 [3:48:43<3:26:45,  2.23s/it]
 54%|█████▎    | 6440/12000 [3:48:44<3:15:47,  2.11s/it]
                                                        

 54%|█████▎    | 6440/12000 [3:48:44<3:15:47,  2.11s/it]
 54%|█████▎    | 6441/12000 [3:48:48<3:42:07,  2.40s/it]
                                                        

 54%|█████▎    | 6441/12000 [3:48:48<3:42:07,  2.40s/it]
 54%|█████▎    | 6442/12000 [3:48:49<3:15:09,  2.11s/it]
                                                        

 54%|█████▎    | 6442/12000 [3:48:49<3:15:09,  2.11s/it]
 54%|█████▎    | 6443/12000 [3:48:51<3:01:41,  1.96s/it]
                                                        
{'loss': '0.907', 'grad_norm': '0.3996', 'learning_rate': '9.738e-05', 'entropy': '0.8779', 'num_tokens': '3.455e+07', 'mean_token_accuracy': '0.7906', 'epoch': '0.0566'}
{'loss': '1.137', 'grad_norm': '0.3928', 'learning_rate': '9.735e-05', 'entropy': '1.119', 'num_tokens': '3.455e+07', 'mean_token_accuracy': '0.7282', 'epoch': '0.05661'}
{'loss': '0.9216', 'grad_norm': '0.3984', 'learning_rate': '9.733e-05', 'entropy': '0.8989', 'num_tokens': '3.456e+07', 'mean_token_accuracy': '0.7818', 'epoch': '0.05662'}
{'loss': '0.9188', 'grad_norm': '0.4213', 'learning_rate': '9.73e-05', 'entropy': '0.956', 'num_tokens': '3.456e+07', 'mean_token_accuracy': '0.7708', 'epoch': '0.05663'}
{'loss': '0.8099', 'grad_norm': '0.4047', 'learning_rate': '9.727e-05', 'entropy': '0.7974', 'num_tokens': '3.457e+07', 'mean_token_accuracy': '0.795', 'epoch': '0.05664'}
{'loss': '0.8629', 'grad_norm': '0.4289', 'learning_rate': '9.724e-05', 'entropy': '0.9747', 'num_tokens': '3.457e+07', 'mean_token_accuracy': '0.7654', 'epoch': '0.05665'}
{'loss': '0.8307', 'grad_norm': '0.331', 'learning_rate': '9.722e-05', 'entropy': '0.8144', 'num_tokens': '3.458e+07', 'mean_token_accuracy': '0.8045', 'epoch': '0.05666'}
{'loss': '0.5115', 'grad_norm': '0.2982', 'learning_rate': '9.719e-05', 'entropy': '0.6225', 'num_tokens': '3.459e+07', 'mean_token_accuracy': '0.8536', 'epoch': '0.05667'}
{'loss': '0.6687', 'grad_norm': '0.3333', 'learning_rate': '9.716e-05', 'entropy': '0.785', 'num_tokens': '3.459e+07', 'mean_token_accuracy': '0.7947', 'epoch': '0.05667'}
{'loss': '0.6944', 'grad_norm': '0.4314', 'learning_rate': '9.713e-05', 'entropy': '0.7561', 'num_tokens': '3.46e+07', 'mean_token_accuracy': '0.8118', 'epoch': '0.05668'}
{'loss': '0.7067', 'grad_norm': '0.3699', 'learning_rate': '9.711e-05', 'entropy': '0.7412', 'num_tokens': '3.46e+07', 'mean_token_accuracy': '0.8046', 'epoch': '0.05669'}
{'loss': '0.8448', 'grad_norm': '0.3729', 'learning_rate': '9.708e-05', 'entropy': '0.8321', 'num_tokens': '3.46e+07', 'mean_token_accuracy': '0.7876', 'epoch': '0.0567'}
{'loss': '0.9681', 'grad_norm': '0.3965', 'learning_rate': '9.705e-05', 'entropy': '0.9239', 'num_tokens': '3.461e+07', 'mean_token_accuracy': '0.7743', 'epoch': '0.05671'}
{'loss': '1.115', 'grad_norm': '0.4476', 'learning_rate': '9.702e-05', 'entropy': '1.066', 'num_tokens': '3.461e+07', 'mean_token_accuracy': '0.732', 'epoch': '0.05672'}
{'loss': '0.7073', 'grad_norm': '0.3712', 'learning_rate': '9.7e-05', 'entropy': '0.7227', 'num_tokens': '3.462e+07', 'mean_token_accuracy': '0.8092', 'epoch': '0.05673'}
{'loss': '0.812', 'grad_norm': '0.3645', 'learning_rate': '9.697e-05', 'entropy': '0.8007', 'num_tokens': '3.462e+07', 'mean_token_accuracy': '0.8004', 'epoch': '0.05674'}
{'loss': '0.7484', 'grad_norm': '0.497', 'learning_rate': '9.694e-05', 'entropy': '0.711', 'num_tokens': '3.462e+07', 'mean_token_accuracy': '0.8204', 'epoch': '0.05675'}
{'loss': '0.9807', 'grad_norm': '0.427', 'learning_rate': '9.691e-05', 'entropy': '0.9204', 'num_tokens': '3.463e+07', 'mean_token_accuracy': '0.7778', 'epoch': '0.05675'}
{'loss': '0.8379', 'grad_norm': '0.3844', 'learning_rate': '9.689e-05', 'entropy': '0.8844', 'num_tokens': '3.464e+07', 'mean_token_accuracy': '0.7774', 'epoch': '0.05676'}
{'loss': '1.137', 'grad_norm': '0.426', 'learning_rate': '9.686e-05', 'entropy': '1.049', 'num_tokens': '3.464e+07', 'mean_token_accuracy': '0.7461', 'epoch': '0.05677'}
{'loss': '0.8114', 'grad_norm': '0.4018', 'learning_rate': '9.683e-05', 'entropy': '0.7428', 'num_tokens': '3.465e+07', 'mean_token_accuracy': '0.812', 'epoch': '0.05678'}
{'loss': '0.7158', 'grad_norm': '0.3602', 'learning_rate': '9.68e-05', 'entropy': '0.7258', 'num_tokens': '3.465e+07', 'mean_token_accuracy': '0.8058', 'epoch': '0.05679'}
{'loss': '1.097', 'grad_norm': '0.4258', 'learning_rate': '9.678e-05', 'entropy': '1.08', 'num_tokens': '3.466e+07', 'mean_token_accuracy': '0.7474', 'epoch': '0.0568'}
{'loss': '1.16', 'grad_norm': '0.4484', 'learning_rate': '9.675e-05', 'entropy': '1.134', 'num_tokens': '3.467e+07', 'mean_token_accuracy': '0.7286', 'epoch': '0.05681'}
{'loss': '0.8111', 'grad_norm': '0.5425', 'learning_rate': '9.672e-05', 'entropy': '0.8508', 'num_tokens': '3.467e+07', 'mean_token_accuracy': '0.7792', 'epoch': '0.05682'}
{'loss': '0.907', 'grad_norm': '0.4895', 'learning_rate': '9.669e-05', 'entropy': '0.9981', 'num_tokens': '3.467e+07', 'mean_token_accuracy': '0.7448', 'epoch': '0.05682'}
{'loss': '0.8689', 'grad_norm': '0.3603', 'learning_rate': '9.667e-05', 'entropy': '0.9205', 'num_tokens': '3.468e+07', 'mean_token_accuracy': '0.7757', 'epoch': '0.05683'}
{'loss': '0.8128', 'grad_norm': '0.4448', 'learning_rate': '9.664e-05', 'entropy': '0.8332', 'num_tokens': '3.469e+07', 'mean_token_accuracy': '0.7893', 'epoch': '0.05684'}
{'loss': '0.6025', 'grad_norm': '0.256', 'learning_rate': '9.661e-05', 'entropy': '0.5965', 'num_tokens': '3.47e+07', 'mean_token_accuracy': '0.8302', 'epoch': '0.05685'}
{'loss': '0.8815', 'grad_norm': '0.3778', 'learning_rate': '9.658e-05', 'entropy': '0.8552', 'num_tokens': '3.47e+07', 'mean_token_accuracy': '0.7694', 'epoch': '0.05686'}
{'loss': '0.6104', 'grad_norm': '0.3013', 'learning_rate': '9.656e-05', 'entropy': '0.6515', 'num_tokens': '3.471e+07', 'mean_token_accuracy': '0.8305', 'epoch': '0.05687'}
{'loss': '1.026', 'grad_norm': '0.5015', 'learning_rate': '9.653e-05', 'entropy': '0.9952', 'num_tokens': '3.471e+07', 'mean_token_accuracy': '0.7648', 'epoch': '0.05688'}
{'loss': '0.849', 'grad_norm': '0.367', 'learning_rate': '9.65e-05', 'entropy': '0.8628', 'num_tokens': '3.472e+07', 'mean_token_accuracy': '0.7862', 'epoch': '0.05689'}
{'loss': '0.9836', 'grad_norm': '0.4293', 'learning_rate': '9.647e-05', 'entropy': '1.059', 'num_tokens': '3.472e+07', 'mean_token_accuracy': '0.7402', 'epoch': '0.0569'}
{'loss': '1.174', 'grad_norm': '0.4538', 'learning_rate': '9.645e-05', 'entropy': '1.092', 'num_tokens': '3.473e+07', 'mean_token_accuracy': '0.7432', 'epoch': '0.0569'}
{'loss': '0.7321', 'grad_norm': '0.4227', 'learning_rate': '9.642e-05', 'entropy': '0.7456', 'num_tokens': '3.473e+07', 'mean_token_accuracy': '0.8085', 'epoch': '0.05691'}
{'loss': '0.4767', 'grad_norm': '0.3101', 'learning_rate': '9.639e-05', 'entropy': '0.5637', 'num_tokens': '3.474e+07', 'mean_token_accuracy': '0.8562', 'epoch': '0.05692'}
{'loss': '0.6928', 'grad_norm': '0.3992', 'learning_rate': '9.636e-05', 'entropy': '0.77', 'num_tokens': '3.474e+07', 'mean_token_accuracy': '0.8039', 'epoch': '0.05693'}
{'loss': '1.298', 'grad_norm': '0.4221', 'learning_rate': '9.634e-05', 'entropy': '0.9537', 'num_tokens': '3.475e+07', 'mean_token_accuracy': '0.7703', 'epoch': '0.05694'}
{'loss': '1.325', 'grad_norm': '0.4669', 'learning_rate': '9.631e-05', 'entropy': '1.231', 'num_tokens': '3.475e+07', 'mean_token_accuracy': '0.6954', 'epoch': '0.05695'}
{'loss': '0.7151', 'grad_norm': '0.2931', 'learning_rate': '9.628e-05', 'entropy': '0.7537', 'num_tokens': '3.476e+07', 'mean_token_accuracy': '0.8053', 'epoch': '0.05696'}
{'loss': '1', 'grad_norm': '0.4569', 'learning_rate': '9.625e-05', 'entropy': '1.034', 'num_tokens': '3.476e+07', 'mean_token_accuracy': '0.7535', 'epoch': '0.05697'}
{'loss': '0.6859', 'grad_norm': '0.4311', 'learning_rate': '9.623e-05', 'entropy': '0.7127', 'num_tokens': '3.476e+07', 'mean_token_accuracy': '0.8044', 'epoch': '0.05698'}
{'loss': '0.5077', 'grad_norm': '0.3588', 'learning_rate': '9.62e-05', 'entropy': '0.6318', 'num_tokens': '3.477e+07', 'mean_token_accuracy': '0.8245', 'epoch': '0.05698'}
{'loss': '1.103', 'grad_norm': '0.4339', 'learning_rate': '9.617e-05', 'entropy': '1.03', 'num_tokens': '3.478e+07', 'mean_token_accuracy': '0.737', 'epoch': '0.05699'}
{'loss': '0.8911', 'grad_norm': '0.3321', 'learning_rate': '9.614e-05', 'entropy': '0.8721', 'num_tokens': '3.478e+07', 'mean_token_accuracy': '0.7773', 'epoch': '0.057'}
{'loss': '0.7542', 'grad_norm': '0.3883', 'learning_rate': '9.612e-05', 'entropy': '0.6984', 'num_tokens': '3.479e+07', 'mean_token_accuracy': '0.8216', 'epoch': '0.05701'}

 54%|█████▎    | 6443/12000 [3:48:51<3:01:41,  1.96s/it]
 54%|█████▎    | 6444/12000 [3:48:52<2:56:09,  1.90s/it]
                                                        

 54%|█████▎    | 6444/12000 [3:48:52<2:56:09,  1.90s/it]
 54%|█████▎    | 6445/12000 [3:48:56<3:30:31,  2.27s/it]
                                                        

 54%|█████▎    | 6445/12000 [3:48:56<3:30:31,  2.27s/it]
 54%|█████▎    | 6446/12000 [3:48:57<3:10:04,  2.05s/it]
                                                        

 54%|█████▎    | 6446/12000 [3:48:57<3:10:04,  2.05s/it]
 54%|█████▎    | 6447/12000 [3:48:59<2:53:39,  1.88s/it]
                                                        

 54%|█████▎    | 6447/12000 [3:48:59<2:53:39,  1.88s/it]
 54%|█████▎    | 6448/12000 [3:49:00<2:46:51,  1.80s/it]
                                                        

 54%|█████▎    | 6448/12000 [3:49:00<2:46:51,  1.80s/it]
 54%|█████▎    | 6449/12000 [3:49:02<2:45:50,  1.79s/it]
                                                        

 54%|█████▎    | 6449/12000 [3:49:02<2:45:50,  1.79s/it]
 54%|█████▍    | 6450/12000 [3:49:04<2:48:35,  1.82s/it]
                                                        

 54%|█████▍    | 6450/12000 [3:49:04<2:48:35,  1.82s/it]
 54%|█████▍    | 6451/12000 [3:49:08<3:46:25,  2.45s/it]
                                                        

 54%|█████▍    | 6451/12000 [3:49:08<3:46:25,  2.45s/it]
 54%|█████▍    | 6452/12000 [3:49:11<3:59:44,  2.59s/it]
                                                        

 54%|█████▍    | 6452/12000 [3:49:11<3:59:44,  2.59s/it]
 54%|█████▍    | 6453/12000 [3:49:12<3:33:41,  2.31s/it]
                                                        

 54%|█████▍    | 6453/12000 [3:49:12<3:33:41,  2.31s/it]
 54%|█████▍    | 6454/12000 [3:49:14<3:10:43,  2.06s/it]
                                                        

 54%|█████▍    | 6454/12000 [3:49:14<3:10:43,  2.06s/it]
 54%|█████▍    | 6455/12000 [3:49:17<3:28:47,  2.26s/it]
                                                        

 54%|█████▍    | 6455/12000 [3:49:17<3:28:47,  2.26s/it]
 54%|█████▍    | 6456/12000 [3:49:18<3:14:44,  2.11s/it]
                                                        

 54%|█████▍    | 6456/12000 [3:49:18<3:14:44,  2.11s/it]
 54%|█████▍    | 6457/12000 [3:49:20<3:17:39,  2.14s/it]
                                                        

 54%|█████▍    | 6457/12000 [3:49:20<3:17:39,  2.14s/it]
 54%|█████▍    | 6458/12000 [3:49:22<3:05:43,  2.01s/it]
                                                        

 54%|█████▍    | 6458/12000 [3:49:22<3:05:43,  2.01s/it]
 54%|█████▍    | 6459/12000 [3:49:26<3:45:40,  2.44s/it]
                                                        

 54%|█████▍    | 6459/12000 [3:49:26<3:45:40,  2.44s/it]
 54%|█████▍    | 6460/12000 [3:49:28<3:35:11,  2.33s/it]
                                                        

 54%|█████▍    | 6460/12000 [3:49:28<3:35:11,  2.33s/it]
 54%|█████▍    | 6461/12000 [3:49:29<3:09:32,  2.05s/it]
                                                        

 54%|█████▍    | 6461/12000 [3:49:29<3:09:32,  2.05s/it]
 54%|█████▍    | 6462/12000 [3:49:31<2:58:00,  1.93s/it]
                                                        

 54%|█████▍    | 6462/12000 [3:49:31<2:58:00,  1.93s/it]
 54%|█████▍    | 6463/12000 [3:49:32<2:49:04,  1.83s/it]
                                                        

 54%|█████▍    | 6463/12000 [3:49:32<2:49:04,  1.83s/it]
 54%|█████▍    | 6464/12000 [3:49:37<4:07:19,  2.68s/it]
                                                        

 54%|█████▍    | 6464/12000 [3:49:37<4:07:19,  2.68s/it]
 54%|█████▍    | 6465/12000 [3:49:39<3:43:14,  2.42s/it]
                                                        

 54%|█████▍    | 6465/12000 [3:49:39<3:43:14,  2.42s/it]
 54%|█████▍    | 6466/12000 [3:49:41<3:49:53,  2.49s/it]
                                                        

 54%|█████▍    | 6466/12000 [3:49:41<3:49:53,  2.49s/it]
 54%|█████▍    | 6467/12000 [3:49:44<3:56:02,  2.56s/it]
                                                        

 54%|█████▍    | 6467/12000 [3:49:44<3:56:02,  2.56s/it]
 54%|█████▍    | 6468/12000 [3:49:47<3:49:48,  2.49s/it]
                                                        

 54%|█████▍    | 6468/12000 [3:49:47<3:49:48,  2.49s/it]
 54%|█████▍    | 6469/12000 [3:49:48<3:24:33,  2.22s/it]
                                                        

 54%|█████▍    | 6469/12000 [3:49:48<3:24:33,  2.22s/it]
 54%|█████▍    | 6470/12000 [3:49:50<3:20:37,  2.18s/it]
                                                        

 54%|█████▍    | 6470/12000 [3:49:50<3:20:37,  2.18s/it]
 54%|█████▍    | 6471/12000 [3:49:52<3:12:24,  2.09s/it]
                                                        

 54%|█████▍    | 6471/12000 [3:49:52<3:12:24,  2.09s/it]
 54%|█████▍    | 6472/12000 [3:49:54<2:58:39,  1.94s/it]
                                                        

 54%|█████▍    | 6472/12000 [3:49:54<2:58:39,  1.94s/it]
 54%|█████▍    | 6473/12000 [3:49:55<2:43:30,  1.78s/it]
                                                        

 54%|█████▍    | 6473/12000 [3:49:55<2:43:30,  1.78s/it]
 54%|█████▍    | 6474/12000 [3:49:57<2:42:25,  1.76s/it]
                                                        

 54%|█████▍    | 6474/12000 [3:49:57<2:42:25,  1.76s/it]
 54%|█████▍    | 6475/12000 [3:49:58<2:35:19,  1.69s/it]
                                                        

 54%|█████▍    | 6475/12000 [3:49:58<2:35:19,  1.69s/it]
 54%|█████▍    | 6476/12000 [3:50:01<2:53:48,  1.89s/it]
                                                        

 54%|█████▍    | 6476/12000 [3:50:01<2:53:48,  1.89s/it]
 54%|█████▍    | 6477/12000 [3:50:02<2:43:45,  1.78s/it]
                                                        

 54%|█████▍    | 6477/12000 [3:50:02<2:43:45,  1.78s/it]
 54%|█████▍    | 6478/12000 [3:50:05<3:09:12,  2.06s/it]
                                                        

 54%|█████▍    | 6478/12000 [3:50:05<3:09:12,  2.06s/it]
 54%|█████▍    | 6479/12000 [3:50:06<2:55:58,  1.91s/it]
                                                        

 54%|█████▍    | 6479/12000 [3:50:06<2:55:58,  1.91s/it]
 54%|█████▍    | 6480/12000 [3:50:09<3:20:07,  2.18s/it]
                                                        

 54%|█████▍    | 6480/12000 [3:50:09<3:20:07,  2.18s/it]
 54%|█████▍    | 6481/12000 [3:50:13<3:53:33,  2.54s/it]
                                                        

 54%|█████▍    | 6481/12000 [3:50:13<3:53:33,  2.54s/it]
 54%|█████▍    | 6482/12000 [3:50:15<3:50:08,  2.50s/it]
                                                        

 54%|█████▍    | 6482/12000 [3:50:15<3:50:08,  2.50s/it]
 54%|█████▍    | 6483/12000 [3:50:17<3:21:10,  2.19s/it]
                                                        

 54%|█████▍    | 6483/12000 [3:50:17<3:21:10,  2.19s/it]
 54%|█████▍    | 6484/12000 [3:50:18<3:11:51,  2.09s/it]
                                                        

 54%|█████▍    | 6484/12000 [3:50:18<3:11:51,  2.09s/it]
 54%|█████▍    | 6485/12000 [3:50:20<2:57:39,  1.93s/it]
                                                        

 54%|█████▍    | 6485/12000 [3:50:20<2:57:39,  1.93s/it]
 54%|█████▍    | 6486/12000 [3:50:22<2:57:54,  1.94s/it]
                                                        

 54%|█████▍    | 6486/12000 [3:50:22<2:57:54,  1.94s/it]
 54%|█████▍    | 6487/12000 [3:50:24<3:06:37,  2.03s/it]
                                                        

 54%|█████▍    | 6487/12000 [3:50:24<3:06:37,  2.03s/it]
 54%|█████▍    | 6488/12000 [3:50:26<3:06:12,  2.03s/it]
                                                        

 54%|█████▍    | 6488/12000 [3:50:26<3:06:12,  2.03s/it]
 54%|█████▍    | 6489/12000 [3:50:28<2:50:51,  1.86s/it]
                                                        

 54%|█████▍    | 6489/12000 [3:50:28<2:50:51,  1.86s/it]
 54%|█████▍    | 6490/12000 [3:50:32<4:11:08,  2.73s/it]
                                                        
{'loss': '0.6954', 'grad_norm': '0.3349', 'learning_rate': '9.609e-05', 'entropy': '0.703', 'num_tokens': '3.479e+07', 'mean_token_accuracy': '0.8063', 'epoch': '0.05702'}
{'loss': '1.17', 'grad_norm': '0.4355', 'learning_rate': '9.606e-05', 'entropy': '1.097', 'num_tokens': '3.48e+07', 'mean_token_accuracy': '0.7311', 'epoch': '0.05703'}
{'loss': '0.8463', 'grad_norm': '0.3688', 'learning_rate': '9.603e-05', 'entropy': '0.9235', 'num_tokens': '3.481e+07', 'mean_token_accuracy': '0.7534', 'epoch': '0.05704'}
{'loss': '0.6432', 'grad_norm': '0.4701', 'learning_rate': '9.601e-05', 'entropy': '0.6952', 'num_tokens': '3.481e+07', 'mean_token_accuracy': '0.8116', 'epoch': '0.05705'}
{'loss': '0.6751', 'grad_norm': '0.3269', 'learning_rate': '9.598e-05', 'entropy': '0.6855', 'num_tokens': '3.482e+07', 'mean_token_accuracy': '0.8301', 'epoch': '0.05705'}
{'loss': '0.9434', 'grad_norm': '0.4272', 'learning_rate': '9.595e-05', 'entropy': '0.969', 'num_tokens': '3.482e+07', 'mean_token_accuracy': '0.763', 'epoch': '0.05706'}
{'loss': '0.6571', 'grad_norm': '0.249', 'learning_rate': '9.592e-05', 'entropy': '0.6933', 'num_tokens': '3.483e+07', 'mean_token_accuracy': '0.8231', 'epoch': '0.05707'}
{'loss': '0.8704', 'grad_norm': '0.4227', 'learning_rate': '9.59e-05', 'entropy': '0.8564', 'num_tokens': '3.483e+07', 'mean_token_accuracy': '0.7874', 'epoch': '0.05708'}
{'loss': '0.8914', 'grad_norm': '0.3702', 'learning_rate': '9.587e-05', 'entropy': '0.9637', 'num_tokens': '3.484e+07', 'mean_token_accuracy': '0.7667', 'epoch': '0.05709'}
{'loss': '1.495', 'grad_norm': '0.3859', 'learning_rate': '9.584e-05', 'entropy': '1.398', 'num_tokens': '3.484e+07', 'mean_token_accuracy': '0.6619', 'epoch': '0.0571'}
{'loss': '0.8841', 'grad_norm': '0.3671', 'learning_rate': '9.581e-05', 'entropy': '1.081', 'num_tokens': '3.485e+07', 'mean_token_accuracy': '0.7482', 'epoch': '0.05711'}
{'loss': '1.042', 'grad_norm': '0.5285', 'learning_rate': '9.578e-05', 'entropy': '1.086', 'num_tokens': '3.485e+07', 'mean_token_accuracy': '0.7559', 'epoch': '0.05712'}
{'loss': '0.821', 'grad_norm': '0.3511', 'learning_rate': '9.576e-05', 'entropy': '0.8595', 'num_tokens': '3.486e+07', 'mean_token_accuracy': '0.7753', 'epoch': '0.05713'}
{'loss': '0.5942', 'grad_norm': '0.3911', 'learning_rate': '9.573e-05', 'entropy': '0.6202', 'num_tokens': '3.486e+07', 'mean_token_accuracy': '0.8289', 'epoch': '0.05713'}
{'loss': '0.8095', 'grad_norm': '0.4027', 'learning_rate': '9.57e-05', 'entropy': '0.8222', 'num_tokens': '3.487e+07', 'mean_token_accuracy': '0.7772', 'epoch': '0.05714'}
{'loss': '1.058', 'grad_norm': '0.4809', 'learning_rate': '9.567e-05', 'entropy': '1.095', 'num_tokens': '3.487e+07', 'mean_token_accuracy': '0.739', 'epoch': '0.05715'}
{'loss': '0.6423', 'grad_norm': '0.3597', 'learning_rate': '9.565e-05', 'entropy': '0.6199', 'num_tokens': '3.488e+07', 'mean_token_accuracy': '0.84', 'epoch': '0.05716'}
{'loss': '0.5777', 'grad_norm': '0.2843', 'learning_rate': '9.562e-05', 'entropy': '0.5684', 'num_tokens': '3.488e+07', 'mean_token_accuracy': '0.8508', 'epoch': '0.05717'}
{'loss': '0.9423', 'grad_norm': '0.4617', 'learning_rate': '9.559e-05', 'entropy': '0.9547', 'num_tokens': '3.489e+07', 'mean_token_accuracy': '0.7625', 'epoch': '0.05718'}
{'loss': '0.959', 'grad_norm': '0.4043', 'learning_rate': '9.556e-05', 'entropy': '1.018', 'num_tokens': '3.489e+07', 'mean_token_accuracy': '0.7478', 'epoch': '0.05719'}
{'loss': '0.7132', 'grad_norm': '0.4576', 'learning_rate': '9.554e-05', 'entropy': '0.7359', 'num_tokens': '3.49e+07', 'mean_token_accuracy': '0.8126', 'epoch': '0.0572'}
{'loss': '1.303', 'grad_norm': '0.4663', 'learning_rate': '9.551e-05', 'entropy': '1.183', 'num_tokens': '3.491e+07', 'mean_token_accuracy': '0.717', 'epoch': '0.05721'}
{'loss': '0.737', 'grad_norm': '0.4254', 'learning_rate': '9.548e-05', 'entropy': '0.7743', 'num_tokens': '3.491e+07', 'mean_token_accuracy': '0.7918', 'epoch': '0.05721'}
{'loss': '1.113', 'grad_norm': '0.4082', 'learning_rate': '9.545e-05', 'entropy': '0.9928', 'num_tokens': '3.492e+07', 'mean_token_accuracy': '0.7569', 'epoch': '0.05722'}
{'loss': '1.131', 'grad_norm': '0.3612', 'learning_rate': '9.543e-05', 'entropy': '0.9067', 'num_tokens': '3.492e+07', 'mean_token_accuracy': '0.7751', 'epoch': '0.05723'}
{'loss': '0.9139', 'grad_norm': '0.3697', 'learning_rate': '9.54e-05', 'entropy': '0.9089', 'num_tokens': '3.493e+07', 'mean_token_accuracy': '0.7708', 'epoch': '0.05724'}
{'loss': '0.6973', 'grad_norm': '0.4104', 'learning_rate': '9.537e-05', 'entropy': '0.6546', 'num_tokens': '3.494e+07', 'mean_token_accuracy': '0.8239', 'epoch': '0.05725'}
{'loss': '1.211', 'grad_norm': '0.5702', 'learning_rate': '9.534e-05', 'entropy': '1.043', 'num_tokens': '3.494e+07', 'mean_token_accuracy': '0.7371', 'epoch': '0.05726'}
{'loss': '0.728', 'grad_norm': '0.414', 'learning_rate': '9.532e-05', 'entropy': '0.7466', 'num_tokens': '3.495e+07', 'mean_token_accuracy': '0.8096', 'epoch': '0.05727'}
{'loss': '0.87', 'grad_norm': '0.3939', 'learning_rate': '9.529e-05', 'entropy': '0.8997', 'num_tokens': '3.495e+07', 'mean_token_accuracy': '0.7747', 'epoch': '0.05728'}
{'loss': '0.8037', 'grad_norm': '0.4864', 'learning_rate': '9.526e-05', 'entropy': '0.8319', 'num_tokens': '3.495e+07', 'mean_token_accuracy': '0.7982', 'epoch': '0.05728'}
{'loss': '0.8538', 'grad_norm': '0.3826', 'learning_rate': '9.523e-05', 'entropy': '0.8176', 'num_tokens': '3.496e+07', 'mean_token_accuracy': '0.7869', 'epoch': '0.05729'}
{'loss': '0.7425', 'grad_norm': '0.418', 'learning_rate': '9.521e-05', 'entropy': '0.7452', 'num_tokens': '3.496e+07', 'mean_token_accuracy': '0.8181', 'epoch': '0.0573'}
{'loss': '1.032', 'grad_norm': '0.5022', 'learning_rate': '9.518e-05', 'entropy': '0.9797', 'num_tokens': '3.497e+07', 'mean_token_accuracy': '0.7664', 'epoch': '0.05731'}
{'loss': '0.9143', 'grad_norm': '0.472', 'learning_rate': '9.515e-05', 'entropy': '0.8623', 'num_tokens': '3.497e+07', 'mean_token_accuracy': '0.7837', 'epoch': '0.05732'}
{'loss': '0.7054', 'grad_norm': '0.3109', 'learning_rate': '9.512e-05', 'entropy': '0.7937', 'num_tokens': '3.498e+07', 'mean_token_accuracy': '0.7995', 'epoch': '0.05733'}
{'loss': '1.235', 'grad_norm': '0.5032', 'learning_rate': '9.51e-05', 'entropy': '1.125', 'num_tokens': '3.498e+07', 'mean_token_accuracy': '0.7246', 'epoch': '0.05734'}
{'loss': '0.6354', 'grad_norm': '0.3414', 'learning_rate': '9.507e-05', 'entropy': '0.6767', 'num_tokens': '3.499e+07', 'mean_token_accuracy': '0.8214', 'epoch': '0.05735'}
{'loss': '0.629', 'grad_norm': '0.3496', 'learning_rate': '9.504e-05', 'entropy': '0.6307', 'num_tokens': '3.499e+07', 'mean_token_accuracy': '0.8424', 'epoch': '0.05736'}
{'loss': '0.9715', 'grad_norm': '0.5094', 'learning_rate': '9.501e-05', 'entropy': '0.8965', 'num_tokens': '3.5e+07', 'mean_token_accuracy': '0.775', 'epoch': '0.05736'}
{'loss': '0.7575', 'grad_norm': '0.4194', 'learning_rate': '9.499e-05', 'entropy': '0.6751', 'num_tokens': '3.501e+07', 'mean_token_accuracy': '0.8222', 'epoch': '0.05737'}
{'loss': '0.8979', 'grad_norm': '0.4648', 'learning_rate': '9.496e-05', 'entropy': '0.9249', 'num_tokens': '3.501e+07', 'mean_token_accuracy': '0.761', 'epoch': '0.05738'}
{'loss': '1.048', 'grad_norm': '0.381', 'learning_rate': '9.493e-05', 'entropy': '1.041', 'num_tokens': '3.502e+07', 'mean_token_accuracy': '0.7606', 'epoch': '0.05739'}
{'loss': '1.116', 'grad_norm': '0.4518', 'learning_rate': '9.49e-05', 'entropy': '1.035', 'num_tokens': '3.502e+07', 'mean_token_accuracy': '0.7591', 'epoch': '0.0574'}
{'loss': '0.7282', 'grad_norm': '0.4095', 'learning_rate': '9.488e-05', 'entropy': '0.7211', 'num_tokens': '3.503e+07', 'mean_token_accuracy': '0.8205', 'epoch': '0.05741'}
{'loss': '0.6375', 'grad_norm': '0.3522', 'learning_rate': '9.485e-05', 'entropy': '0.6687', 'num_tokens': '3.503e+07', 'mean_token_accuracy': '0.8276', 'epoch': '0.05742'}
{'loss': '0.5811', 'grad_norm': '0.3956', 'learning_rate': '9.482e-05', 'entropy': '0.602', 'num_tokens': '3.504e+07', 'mean_token_accuracy': '0.8555', 'epoch': '0.05743'}

 54%|█████▍    | 6490/12000 [3:50:32<4:11:08,  2.73s/it]
 54%|█████▍    | 6491/12000 [3:50:36<4:28:15,  2.92s/it]
                                                        

 54%|█████▍    | 6491/12000 [3:50:36<4:28:15,  2.92s/it]
 54%|█████▍    | 6492/12000 [3:50:37<3:46:10,  2.46s/it]
                                                        

 54%|█████▍    | 6492/12000 [3:50:37<3:46:10,  2.46s/it]
 54%|█████▍    | 6493/12000 [3:50:39<3:41:48,  2.42s/it]
                                                        

 54%|█████▍    | 6493/12000 [3:50:39<3:41:48,  2.42s/it]
 54%|█████▍    | 6494/12000 [3:50:41<3:19:10,  2.17s/it]
                                                        

 54%|█████▍    | 6494/12000 [3:50:41<3:19:10,  2.17s/it]
 54%|█████▍    | 6495/12000 [3:50:43<3:18:50,  2.17s/it]
                                                        

 54%|█████▍    | 6495/12000 [3:50:43<3:18:50,  2.17s/it]
 54%|█████▍    | 6496/12000 [3:50:45<2:58:02,  1.94s/it]
                                                        

 54%|█████▍    | 6496/12000 [3:50:45<2:58:02,  1.94s/it]
 54%|█████▍    | 6497/12000 [3:50:46<2:49:07,  1.84s/it]
                                                        

 54%|█████▍    | 6497/12000 [3:50:46<2:49:07,  1.84s/it]
 54%|█████▍    | 6498/12000 [3:50:48<2:38:10,  1.72s/it]
                                                        

 54%|█████▍    | 6498/12000 [3:50:48<2:38:10,  1.72s/it]
 54%|█████▍    | 6499/12000 [3:50:50<2:48:32,  1.84s/it]
                                                        

 54%|█████▍    | 6499/12000 [3:50:50<2:48:32,  1.84s/it]
 54%|█████▍    | 6500/12000 [3:50:51<2:38:13,  1.73s/it]
                                                        

 54%|█████▍    | 6500/12000 [3:50:51<2:38:13,  1.73s/it]
 54%|█████▍    | 6501/12000 [3:50:55<3:33:11,  2.33s/it]
                                                        

 54%|█████▍    | 6501/12000 [3:50:55<3:33:11,  2.33s/it]
 54%|█████▍    | 6502/12000 [3:50:57<3:12:35,  2.10s/it]
                                                        

 54%|█████▍    | 6502/12000 [3:50:57<3:12:35,  2.10s/it]
 54%|█████▍    | 6503/12000 [3:51:01<4:20:54,  2.85s/it]
                                                        

 54%|█████▍    | 6503/12000 [3:51:01<4:20:54,  2.85s/it]
 54%|█████▍    | 6504/12000 [3:51:03<3:40:01,  2.40s/it]
                                                        

 54%|█████▍    | 6504/12000 [3:51:03<3:40:01,  2.40s/it]
 54%|█████▍    | 6505/12000 [3:51:04<3:19:49,  2.18s/it]
                                                        

 54%|█████▍    | 6505/12000 [3:51:04<3:19:49,  2.18s/it]
 54%|█████▍    | 6506/12000 [3:51:06<3:22:16,  2.21s/it]
                                                        

 54%|█████▍    | 6506/12000 [3:51:06<3:22:16,  2.21s/it]
 54%|█████▍    | 6507/12000 [3:51:08<3:14:29,  2.12s/it]
                                                        

 54%|█████▍    | 6507/12000 [3:51:08<3:14:29,  2.12s/it]
 54%|█████▍    | 6508/12000 [3:51:11<3:15:26,  2.14s/it]
                                                        

 54%|█████▍    | 6508/12000 [3:51:11<3:15:26,  2.14s/it]
 54%|█████▍    | 6509/12000 [3:51:12<3:06:59,  2.04s/it]
                                                        

 54%|█████▍    | 6509/12000 [3:51:12<3:06:59,  2.04s/it]
 54%|█████▍    | 6510/12000 [3:51:14<2:48:54,  1.85s/it]
                                                        

 54%|█████▍    | 6510/12000 [3:51:14<2:48:54,  1.85s/it]
 54%|█████▍    | 6511/12000 [3:51:16<3:07:41,  2.05s/it]
                                                        

 54%|█████▍    | 6511/12000 [3:51:16<3:07:41,  2.05s/it]
 54%|█████▍    | 6512/12000 [3:51:18<3:06:21,  2.04s/it]
                                                        

 54%|█████▍    | 6512/12000 [3:51:18<3:06:21,  2.04s/it]
 54%|█████▍    | 6513/12000 [3:51:20<3:02:36,  2.00s/it]
                                                        

 54%|█████▍    | 6513/12000 [3:51:20<3:02:36,  2.00s/it]
 54%|█████▍    | 6514/12000 [3:51:22<2:48:32,  1.84s/it]
                                                        

 54%|█████▍    | 6514/12000 [3:51:22<2:48:32,  1.84s/it]
 54%|█████▍    | 6515/12000 [3:51:23<2:37:06,  1.72s/it]
                                                        

 54%|█████▍    | 6515/12000 [3:51:23<2:37:06,  1.72s/it]
 54%|█████▍    | 6516/12000 [3:51:25<2:41:20,  1.77s/it]
                                                        

 54%|█████▍    | 6516/12000 [3:51:25<2:41:20,  1.77s/it]
 54%|█████▍    | 6517/12000 [3:51:26<2:31:15,  1.66s/it]
                                                        

 54%|█████▍    | 6517/12000 [3:51:26<2:31:15,  1.66s/it]
 54%|█████▍    | 6518/12000 [3:51:28<2:25:44,  1.60s/it]
                                                        

 54%|█████▍    | 6518/12000 [3:51:28<2:25:44,  1.60s/it]
 54%|█████▍    | 6519/12000 [3:51:29<2:20:38,  1.54s/it]
                                                        

 54%|█████▍    | 6519/12000 [3:51:29<2:20:38,  1.54s/it]
 54%|█████▍    | 6520/12000 [3:51:32<2:44:34,  1.80s/it]
                                                        

 54%|█████▍    | 6520/12000 [3:51:32<2:44:34,  1.80s/it]
 54%|█████▍    | 6521/12000 [3:51:33<2:35:28,  1.70s/it]
                                                        

 54%|█████▍    | 6521/12000 [3:51:33<2:35:28,  1.70s/it]
 54%|█████▍    | 6522/12000 [3:51:35<2:30:49,  1.65s/it]
                                                        

 54%|█████▍    | 6522/12000 [3:51:35<2:30:49,  1.65s/it]
 54%|█████▍    | 6523/12000 [3:51:36<2:25:17,  1.59s/it]
                                                        

 54%|█████▍    | 6523/12000 [3:51:36<2:25:17,  1.59s/it]
 54%|█████▍    | 6524/12000 [3:51:39<2:50:30,  1.87s/it]
                                                        

 54%|█████▍    | 6524/12000 [3:51:39<2:50:30,  1.87s/it]
 54%|█████▍    | 6525/12000 [3:51:42<3:38:52,  2.40s/it]
                                                        

 54%|█████▍    | 6525/12000 [3:51:42<3:38:52,  2.40s/it]
 54%|█████▍    | 6526/12000 [3:51:44<3:16:01,  2.15s/it]
                                                        

 54%|█████▍    | 6526/12000 [3:51:44<3:16:01,  2.15s/it]
 54%|█████▍    | 6527/12000 [3:51:45<2:54:35,  1.91s/it]
                                                        

 54%|█████▍    | 6527/12000 [3:51:45<2:54:35,  1.91s/it]
 54%|█████▍    | 6528/12000 [3:51:46<2:37:18,  1.72s/it]
                                                        

 54%|█████▍    | 6528/12000 [3:51:46<2:37:18,  1.72s/it]
 54%|█████▍    | 6529/12000 [3:51:48<2:32:54,  1.68s/it]
                                                        

 54%|█████▍    | 6529/12000 [3:51:48<2:32:54,  1.68s/it]
 54%|█████▍    | 6530/12000 [3:51:49<2:22:09,  1.56s/it]
                                                        

 54%|█████▍    | 6530/12000 [3:51:49<2:22:09,  1.56s/it]
 54%|█████▍    | 6531/12000 [3:51:51<2:19:17,  1.53s/it]
                                                        

 54%|█████▍    | 6531/12000 [3:51:51<2:19:17,  1.53s/it]
 54%|█████▍    | 6532/12000 [3:51:53<2:45:45,  1.82s/it]
                                                        

 54%|█████▍    | 6532/12000 [3:51:53<2:45:45,  1.82s/it]
 54%|█████▍    | 6533/12000 [3:51:55<2:45:24,  1.82s/it]
                                                        

 54%|█████▍    | 6533/12000 [3:51:55<2:45:24,  1.82s/it]
 54%|█████▍    | 6534/12000 [3:51:57<2:39:25,  1.75s/it]
                                                        

 54%|█████▍    | 6534/12000 [3:51:57<2:39:25,  1.75s/it]
 54%|█████▍    | 6535/12000 [3:51:58<2:32:29,  1.67s/it]
                                                        

 54%|█████▍    | 6535/12000 [3:51:58<2:32:29,  1.67s/it]
 54%|█████▍    | 6536/12000 [3:52:00<2:24:37,  1.59s/it]
                                                        

 54%|█████▍    | 6536/12000 [3:52:00<2:24:37,  1.59s/it]
 54%|█████▍    | 6537/12000 [3:52:01<2:26:29,  1.61s/it]
                                                        
{'loss': '0.6826', 'grad_norm': '0.3349', 'learning_rate': '9.479e-05', 'entropy': '0.7114', 'num_tokens': '3.505e+07', 'mean_token_accuracy': '0.8288', 'epoch': '0.05744'}
{'loss': '0.977', 'grad_norm': '0.4028', 'learning_rate': '9.477e-05', 'entropy': '0.9738', 'num_tokens': '3.505e+07', 'mean_token_accuracy': '0.7682', 'epoch': '0.05744'}
{'loss': '0.6872', 'grad_norm': '0.3397', 'learning_rate': '9.474e-05', 'entropy': '0.697', 'num_tokens': '3.506e+07', 'mean_token_accuracy': '0.8169', 'epoch': '0.05745'}
{'loss': '0.632', 'grad_norm': '0.4031', 'learning_rate': '9.471e-05', 'entropy': '0.8198', 'num_tokens': '3.506e+07', 'mean_token_accuracy': '0.7929', 'epoch': '0.05746'}
{'loss': '0.9607', 'grad_norm': '0.4177', 'learning_rate': '9.468e-05', 'entropy': '0.9667', 'num_tokens': '3.507e+07', 'mean_token_accuracy': '0.762', 'epoch': '0.05747'}
{'loss': '1.125', 'grad_norm': '0.4324', 'learning_rate': '9.466e-05', 'entropy': '1.068', 'num_tokens': '3.507e+07', 'mean_token_accuracy': '0.7487', 'epoch': '0.05748'}
{'loss': '0.5982', 'grad_norm': '0.3749', 'learning_rate': '9.463e-05', 'entropy': '0.7248', 'num_tokens': '3.508e+07', 'mean_token_accuracy': '0.8112', 'epoch': '0.05749'}
{'loss': '0.9586', 'grad_norm': '0.344', 'learning_rate': '9.46e-05', 'entropy': '0.961', 'num_tokens': '3.508e+07', 'mean_token_accuracy': '0.7606', 'epoch': '0.0575'}
{'loss': '0.7491', 'grad_norm': '0.3927', 'learning_rate': '9.457e-05', 'entropy': '0.8256', 'num_tokens': '3.509e+07', 'mean_token_accuracy': '0.7885', 'epoch': '0.05751'}
{'loss': '0.7897', 'grad_norm': '0.3517', 'learning_rate': '9.455e-05', 'entropy': '0.8578', 'num_tokens': '3.509e+07', 'mean_token_accuracy': '0.7839', 'epoch': '0.05752'}
{'loss': '0.6737', 'grad_norm': '0.3809', 'learning_rate': '9.452e-05', 'entropy': '0.6678', 'num_tokens': '3.509e+07', 'mean_token_accuracy': '0.8247', 'epoch': '0.05752'}
{'loss': '1.743', 'grad_norm': '0.4959', 'learning_rate': '9.449e-05', 'entropy': '1.627', 'num_tokens': '3.51e+07', 'mean_token_accuracy': '0.6464', 'epoch': '0.05753'}
{'loss': '0.9292', 'grad_norm': '0.419', 'learning_rate': '9.446e-05', 'entropy': '0.8644', 'num_tokens': '3.51e+07', 'mean_token_accuracy': '0.7873', 'epoch': '0.05754'}
{'loss': '0.8169', 'grad_norm': '0.4159', 'learning_rate': '9.444e-05', 'entropy': '1.003', 'num_tokens': '3.511e+07', 'mean_token_accuracy': '0.7654', 'epoch': '0.05755'}
{'loss': '0.8028', 'grad_norm': '0.4191', 'learning_rate': '9.441e-05', 'entropy': '0.7947', 'num_tokens': '3.511e+07', 'mean_token_accuracy': '0.7874', 'epoch': '0.05756'}
{'loss': '0.6837', 'grad_norm': '0.3316', 'learning_rate': '9.438e-05', 'entropy': '0.7115', 'num_tokens': '3.512e+07', 'mean_token_accuracy': '0.8199', 'epoch': '0.05757'}
{'loss': '1.156', 'grad_norm': '0.4318', 'learning_rate': '9.435e-05', 'entropy': '1.157', 'num_tokens': '3.513e+07', 'mean_token_accuracy': '0.7347', 'epoch': '0.05758'}
{'loss': '0.7452', 'grad_norm': '0.435', 'learning_rate': '9.433e-05', 'entropy': '0.692', 'num_tokens': '3.513e+07', 'mean_token_accuracy': '0.8141', 'epoch': '0.05759'}
{'loss': '0.6131', 'grad_norm': '0.4061', 'learning_rate': '9.43e-05', 'entropy': '0.6041', 'num_tokens': '3.514e+07', 'mean_token_accuracy': '0.8343', 'epoch': '0.05759'}
{'loss': '0.631', 'grad_norm': '0.351', 'learning_rate': '9.427e-05', 'entropy': '0.6637', 'num_tokens': '3.514e+07', 'mean_token_accuracy': '0.8169', 'epoch': '0.0576'}
{'loss': '0.9897', 'grad_norm': '0.4949', 'learning_rate': '9.424e-05', 'entropy': '0.9484', 'num_tokens': '3.515e+07', 'mean_token_accuracy': '0.7667', 'epoch': '0.05761'}
{'loss': '0.6968', 'grad_norm': '0.3256', 'learning_rate': '9.422e-05', 'entropy': '0.7923', 'num_tokens': '3.515e+07', 'mean_token_accuracy': '0.8101', 'epoch': '0.05762'}
{'loss': '1.123', 'grad_norm': '0.5775', 'learning_rate': '9.419e-05', 'entropy': '1.04', 'num_tokens': '3.516e+07', 'mean_token_accuracy': '0.7415', 'epoch': '0.05763'}
{'loss': '1.053', 'grad_norm': '0.3283', 'learning_rate': '9.416e-05', 'entropy': '0.9542', 'num_tokens': '3.516e+07', 'mean_token_accuracy': '0.7755', 'epoch': '0.05764'}
{'loss': '0.8898', 'grad_norm': '0.3457', 'learning_rate': '9.413e-05', 'entropy': '0.9554', 'num_tokens': '3.517e+07', 'mean_token_accuracy': '0.7581', 'epoch': '0.05765'}
{'loss': '1.075', 'grad_norm': '0.5006', 'learning_rate': '9.411e-05', 'entropy': '0.9827', 'num_tokens': '3.517e+07', 'mean_token_accuracy': '0.754', 'epoch': '0.05766'}
{'loss': '0.7594', 'grad_norm': '0.3776', 'learning_rate': '9.408e-05', 'entropy': '0.7341', 'num_tokens': '3.518e+07', 'mean_token_accuracy': '0.8044', 'epoch': '0.05767'}
{'loss': '0.9202', 'grad_norm': '0.4461', 'learning_rate': '9.405e-05', 'entropy': '0.9066', 'num_tokens': '3.518e+07', 'mean_token_accuracy': '0.772', 'epoch': '0.05767'}
{'loss': '0.6996', 'grad_norm': '0.3916', 'learning_rate': '9.402e-05', 'entropy': '0.7004', 'num_tokens': '3.519e+07', 'mean_token_accuracy': '0.8091', 'epoch': '0.05768'}
{'loss': '0.8828', 'grad_norm': '0.3911', 'learning_rate': '9.4e-05', 'entropy': '0.9701', 'num_tokens': '3.519e+07', 'mean_token_accuracy': '0.7728', 'epoch': '0.05769'}
{'loss': '1.115', 'grad_norm': '0.5177', 'learning_rate': '9.397e-05', 'entropy': '1.047', 'num_tokens': '3.52e+07', 'mean_token_accuracy': '0.7416', 'epoch': '0.0577'}
{'loss': '0.7128', 'grad_norm': '0.4815', 'learning_rate': '9.394e-05', 'entropy': '0.7409', 'num_tokens': '3.52e+07', 'mean_token_accuracy': '0.8062', 'epoch': '0.05771'}
{'loss': '0.8371', 'grad_norm': '0.4109', 'learning_rate': '9.391e-05', 'entropy': '0.8645', 'num_tokens': '3.521e+07', 'mean_token_accuracy': '0.787', 'epoch': '0.05772'}
{'loss': '1.016', 'grad_norm': '0.4544', 'learning_rate': '9.389e-05', 'entropy': '1.079', 'num_tokens': '3.521e+07', 'mean_token_accuracy': '0.7336', 'epoch': '0.05773'}
{'loss': '1.084', 'grad_norm': '0.3865', 'learning_rate': '9.386e-05', 'entropy': '1.086', 'num_tokens': '3.522e+07', 'mean_token_accuracy': '0.715', 'epoch': '0.05774'}
{'loss': '1.025', 'grad_norm': '0.4161', 'learning_rate': '9.383e-05', 'entropy': '1.022', 'num_tokens': '3.522e+07', 'mean_token_accuracy': '0.7502', 'epoch': '0.05775'}
{'loss': '0.8703', 'grad_norm': '0.4395', 'learning_rate': '9.38e-05', 'entropy': '0.8459', 'num_tokens': '3.523e+07', 'mean_token_accuracy': '0.802', 'epoch': '0.05775'}
{'loss': '0.8426', 'grad_norm': '0.4108', 'learning_rate': '9.378e-05', 'entropy': '0.8679', 'num_tokens': '3.523e+07', 'mean_token_accuracy': '0.7718', 'epoch': '0.05776'}
{'loss': '0.5642', 'grad_norm': '0.3086', 'learning_rate': '9.375e-05', 'entropy': '0.5716', 'num_tokens': '3.524e+07', 'mean_token_accuracy': '0.841', 'epoch': '0.05777'}
{'loss': '0.6372', 'grad_norm': '0.3589', 'learning_rate': '9.372e-05', 'entropy': '0.6418', 'num_tokens': '3.524e+07', 'mean_token_accuracy': '0.8334', 'epoch': '0.05778'}
{'loss': '0.7913', 'grad_norm': '0.4489', 'learning_rate': '9.369e-05', 'entropy': '0.8223', 'num_tokens': '3.524e+07', 'mean_token_accuracy': '0.8002', 'epoch': '0.05779'}
{'loss': '0.7458', 'grad_norm': '0.4339', 'learning_rate': '9.367e-05', 'entropy': '0.7331', 'num_tokens': '3.525e+07', 'mean_token_accuracy': '0.8154', 'epoch': '0.0578'}
{'loss': '0.7374', 'grad_norm': '0.3199', 'learning_rate': '9.364e-05', 'entropy': '0.7417', 'num_tokens': '3.526e+07', 'mean_token_accuracy': '0.8065', 'epoch': '0.05781'}
{'loss': '0.6244', 'grad_norm': '0.3314', 'learning_rate': '9.361e-05', 'entropy': '0.6247', 'num_tokens': '3.526e+07', 'mean_token_accuracy': '0.8322', 'epoch': '0.05782'}
{'loss': '0.5065', 'grad_norm': '0.3184', 'learning_rate': '9.358e-05', 'entropy': '0.5919', 'num_tokens': '3.527e+07', 'mean_token_accuracy': '0.8484', 'epoch': '0.05782'}
{'loss': '0.5699', 'grad_norm': '0.3725', 'learning_rate': '9.356e-05', 'entropy': '0.5968', 'num_tokens': '3.527e+07', 'mean_token_accuracy': '0.8304', 'epoch': '0.05783'}
{'loss': '0.6208', 'grad_norm': '0.3538', 'learning_rate': '9.353e-05', 'entropy': '0.6901', 'num_tokens': '3.528e+07', 'mean_token_accuracy': '0.8253', 'epoch': '0.05784'}

 54%|█████▍    | 6537/12000 [3:52:01<2:26:29,  1.61s/it]
 54%|█████▍    | 6538/12000 [3:52:03<2:17:53,  1.51s/it]
                                                        

 54%|█████▍    | 6538/12000 [3:52:03<2:17:53,  1.51s/it]
 54%|█████▍    | 6539/12000 [3:52:04<2:12:00,  1.45s/it]
                                                        

 54%|█████▍    | 6539/12000 [3:52:04<2:12:00,  1.45s/it]
 55%|█████▍    | 6540/12000 [3:52:05<2:13:45,  1.47s/it]
                                                        

 55%|█████▍    | 6540/12000 [3:52:05<2:13:45,  1.47s/it]
 55%|█████▍    | 6541/12000 [3:52:07<2:14:50,  1.48s/it]
                                                        

 55%|█████▍    | 6541/12000 [3:52:07<2:14:50,  1.48s/it]
 55%|█████▍    | 6542/12000 [3:52:08<2:12:28,  1.46s/it]
                                                        

 55%|█████▍    | 6542/12000 [3:52:08<2:12:28,  1.46s/it]
 55%|█████▍    | 6543/12000 [3:52:10<2:12:58,  1.46s/it]
                                                        

 55%|█████▍    | 6543/12000 [3:52:10<2:12:58,  1.46s/it]
 55%|█████▍    | 6544/12000 [3:52:11<2:11:47,  1.45s/it]
                                                        

 55%|█████▍    | 6544/12000 [3:52:11<2:11:47,  1.45s/it]
 55%|█████▍    | 6545/12000 [3:52:13<2:29:22,  1.64s/it]
                                                        

 55%|█████▍    | 6545/12000 [3:52:13<2:29:22,  1.64s/it]
 55%|█████▍    | 6546/12000 [3:52:16<2:55:00,  1.93s/it]
                                                        

 55%|█████▍    | 6546/12000 [3:52:16<2:55:00,  1.93s/it]
 55%|█████▍    | 6547/12000 [3:52:21<4:21:30,  2.88s/it]
                                                        

 55%|█████▍    | 6547/12000 [3:52:21<4:21:30,  2.88s/it]
 55%|█████▍    | 6548/12000 [3:52:24<4:36:03,  3.04s/it]
                                                        

 55%|█████▍    | 6548/12000 [3:52:24<4:36:03,  3.04s/it]
 55%|█████▍    | 6549/12000 [3:52:26<3:58:56,  2.63s/it]
                                                        

 55%|█████▍    | 6549/12000 [3:52:26<3:58:56,  2.63s/it]
 55%|█████▍    | 6550/12000 [3:52:28<3:42:13,  2.45s/it]
                                                        

 55%|█████▍    | 6550/12000 [3:52:28<3:42:13,  2.45s/it]
 55%|█████▍    | 6551/12000 [3:52:31<4:00:07,  2.64s/it]
                                                        

 55%|█████▍    | 6551/12000 [3:52:31<4:00:07,  2.64s/it]
 55%|█████▍    | 6552/12000 [3:52:33<3:51:20,  2.55s/it]
                                                        

 55%|█████▍    | 6552/12000 [3:52:33<3:51:20,  2.55s/it]
 55%|█████▍    | 6553/12000 [3:52:35<3:16:52,  2.17s/it]
                                                        

 55%|█████▍    | 6553/12000 [3:52:35<3:16:52,  2.17s/it]
 55%|█████▍    | 6554/12000 [3:52:36<3:03:15,  2.02s/it]
                                                        

 55%|█████▍    | 6554/12000 [3:52:36<3:03:15,  2.02s/it]
 55%|█████▍    | 6555/12000 [3:52:40<3:40:34,  2.43s/it]
                                                        

 55%|█████▍    | 6555/12000 [3:52:40<3:40:34,  2.43s/it]
 55%|█████▍    | 6556/12000 [3:52:44<4:35:37,  3.04s/it]
                                                        

 55%|█████▍    | 6556/12000 [3:52:44<4:35:37,  3.04s/it]
 55%|█████▍    | 6557/12000 [3:52:46<3:57:38,  2.62s/it]
                                                        

 55%|█████▍    | 6557/12000 [3:52:46<3:57:38,  2.62s/it]
 55%|█████▍    | 6558/12000 [3:52:48<3:30:03,  2.32s/it]
                                                        

 55%|█████▍    | 6558/12000 [3:52:48<3:30:03,  2.32s/it]
 55%|█████▍    | 6559/12000 [3:52:49<3:09:48,  2.09s/it]
                                                        

 55%|█████▍    | 6559/12000 [3:52:49<3:09:48,  2.09s/it]
 55%|█████▍    | 6560/12000 [3:52:51<2:55:36,  1.94s/it]
                                                        

 55%|█████▍    | 6560/12000 [3:52:51<2:55:36,  1.94s/it]
 55%|█████▍    | 6561/12000 [3:52:52<2:44:26,  1.81s/it]
                                                        

 55%|█████▍    | 6561/12000 [3:52:52<2:44:26,  1.81s/it]
 55%|█████▍    | 6562/12000 [3:52:54<2:36:05,  1.72s/it]
                                                        

 55%|█████▍    | 6562/12000 [3:52:54<2:36:05,  1.72s/it]
 55%|█████▍    | 6563/12000 [3:52:56<2:54:41,  1.93s/it]
                                                        

 55%|█████▍    | 6563/12000 [3:52:56<2:54:41,  1.93s/it]
 55%|█████▍    | 6564/12000 [3:52:57<2:37:54,  1.74s/it]
                                                        

 55%|█████▍    | 6564/12000 [3:52:57<2:37:54,  1.74s/it]
 55%|█████▍    | 6565/12000 [3:52:59<2:45:16,  1.82s/it]
                                                        

 55%|█████▍    | 6565/12000 [3:52:59<2:45:16,  1.82s/it]
 55%|█████▍    | 6566/12000 [3:53:01<2:37:41,  1.74s/it]
                                                        

 55%|█████▍    | 6566/12000 [3:53:01<2:37:41,  1.74s/it]
 55%|█████▍    | 6567/12000 [3:53:03<2:36:05,  1.72s/it]
                                                        

 55%|█████▍    | 6567/12000 [3:53:03<2:36:05,  1.72s/it]
 55%|█████▍    | 6568/12000 [3:53:06<3:24:01,  2.25s/it]
                                                        

 55%|█████▍    | 6568/12000 [3:53:06<3:24:01,  2.25s/it]
 55%|█████▍    | 6569/12000 [3:53:09<3:51:54,  2.56s/it]
                                                        

 55%|█████▍    | 6569/12000 [3:53:09<3:51:54,  2.56s/it]
 55%|█████▍    | 6570/12000 [3:53:11<3:19:45,  2.21s/it]
                                                        

 55%|█████▍    | 6570/12000 [3:53:11<3:19:45,  2.21s/it]
 55%|█████▍    | 6571/12000 [3:53:14<3:38:49,  2.42s/it]
                                                        

 55%|█████▍    | 6571/12000 [3:53:14<3:38:49,  2.42s/it]
 55%|█████▍    | 6572/12000 [3:53:16<3:27:22,  2.29s/it]
                                                        

 55%|█████▍    | 6572/12000 [3:53:16<3:27:22,  2.29s/it]
 55%|█████▍    | 6573/12000 [3:53:18<3:36:45,  2.40s/it]
                                                        

 55%|█████▍    | 6573/12000 [3:53:18<3:36:45,  2.40s/it]
 55%|█████▍    | 6574/12000 [3:53:20<3:09:39,  2.10s/it]
                                                        

 55%|█████▍    | 6574/12000 [3:53:20<3:09:39,  2.10s/it]
 55%|█████▍    | 6575/12000 [3:53:22<3:09:50,  2.10s/it]
                                                        

 55%|█████▍    | 6575/12000 [3:53:22<3:09:50,  2.10s/it]
 55%|█████▍    | 6576/12000 [3:53:24<3:01:34,  2.01s/it]
                                                        

 55%|█████▍    | 6576/12000 [3:53:24<3:01:34,  2.01s/it]
 55%|█████▍    | 6577/12000 [3:53:25<2:56:25,  1.95s/it]
                                                        

 55%|█████▍    | 6577/12000 [3:53:25<2:56:25,  1.95s/it]
 55%|█████▍    | 6578/12000 [3:53:28<3:05:45,  2.06s/it]
                                                        

 55%|█████▍    | 6578/12000 [3:53:28<3:05:45,  2.06s/it]
 55%|█████▍    | 6579/12000 [3:53:29<2:52:51,  1.91s/it]
                                                        

 55%|█████▍    | 6579/12000 [3:53:29<2:52:51,  1.91s/it]
 55%|█████▍    | 6580/12000 [3:53:31<2:48:15,  1.86s/it]
                                                        

 55%|█████▍    | 6580/12000 [3:53:31<2:48:15,  1.86s/it]
 55%|█████▍    | 6581/12000 [3:53:33<2:39:35,  1.77s/it]
                                                        

 55%|█████▍    | 6581/12000 [3:53:33<2:39:35,  1.77s/it]
 55%|█████▍    | 6582/12000 [3:53:35<2:42:27,  1.80s/it]
                                                        

 55%|█████▍    | 6582/12000 [3:53:35<2:42:27,  1.80s/it]
 55%|█████▍    | 6583/12000 [3:53:36<2:36:37,  1.73s/it]
                                                        

 55%|█████▍    | 6583/12000 [3:53:36<2:36:37,  1.73s/it]
 55%|█████▍    | 6584/12000 [3:53:40<3:28:10,  2.31s/it]
                                                        
{'loss': '0.9093', 'grad_norm': '0.4475', 'learning_rate': '9.35e-05', 'entropy': '0.8175', 'num_tokens': '3.528e+07', 'mean_token_accuracy': '0.7857', 'epoch': '0.05785'}
{'loss': '0.9357', 'grad_norm': '0.6042', 'learning_rate': '9.347e-05', 'entropy': '0.9228', 'num_tokens': '3.528e+07', 'mean_token_accuracy': '0.774', 'epoch': '0.05786'}
{'loss': '0.8378', 'grad_norm': '0.4352', 'learning_rate': '9.345e-05', 'entropy': '0.8616', 'num_tokens': '3.529e+07', 'mean_token_accuracy': '0.7916', 'epoch': '0.05787'}
{'loss': '1.063', 'grad_norm': '0.4716', 'learning_rate': '9.342e-05', 'entropy': '1.057', 'num_tokens': '3.529e+07', 'mean_token_accuracy': '0.755', 'epoch': '0.05788'}
{'loss': '0.6188', 'grad_norm': '0.3167', 'learning_rate': '9.339e-05', 'entropy': '0.6307', 'num_tokens': '3.53e+07', 'mean_token_accuracy': '0.8401', 'epoch': '0.05789'}
{'loss': '0.7356', 'grad_norm': '0.429', 'learning_rate': '9.336e-05', 'entropy': '0.7638', 'num_tokens': '3.53e+07', 'mean_token_accuracy': '0.8058', 'epoch': '0.0579'}
{'loss': '0.5456', 'grad_norm': '0.3271', 'learning_rate': '9.334e-05', 'entropy': '0.6149', 'num_tokens': '3.531e+07', 'mean_token_accuracy': '0.8418', 'epoch': '0.0579'}
{'loss': '0.6527', 'grad_norm': '0.4151', 'learning_rate': '9.331e-05', 'entropy': '0.6632', 'num_tokens': '3.531e+07', 'mean_token_accuracy': '0.8235', 'epoch': '0.05791'}
{'loss': '0.9573', 'grad_norm': '0.4303', 'learning_rate': '9.328e-05', 'entropy': '0.7643', 'num_tokens': '3.532e+07', 'mean_token_accuracy': '0.8028', 'epoch': '0.05792'}
{'loss': '0.6395', 'grad_norm': '0.3113', 'learning_rate': '9.325e-05', 'entropy': '0.6043', 'num_tokens': '3.532e+07', 'mean_token_accuracy': '0.8501', 'epoch': '0.05793'}
{'loss': '0.7994', 'grad_norm': '0.3521', 'learning_rate': '9.323e-05', 'entropy': '0.8075', 'num_tokens': '3.533e+07', 'mean_token_accuracy': '0.7792', 'epoch': '0.05794'}
{'loss': '1.214', 'grad_norm': '0.3979', 'learning_rate': '9.32e-05', 'entropy': '1.151', 'num_tokens': '3.534e+07', 'mean_token_accuracy': '0.7238', 'epoch': '0.05795'}
{'loss': '1.259', 'grad_norm': '0.522', 'learning_rate': '9.317e-05', 'entropy': '1.26', 'num_tokens': '3.534e+07', 'mean_token_accuracy': '0.6968', 'epoch': '0.05796'}
{'loss': '0.7887', 'grad_norm': '0.3877', 'learning_rate': '9.314e-05', 'entropy': '0.8593', 'num_tokens': '3.535e+07', 'mean_token_accuracy': '0.7954', 'epoch': '0.05797'}
{'loss': '1.043', 'grad_norm': '0.4405', 'learning_rate': '9.312e-05', 'entropy': '1.092', 'num_tokens': '3.535e+07', 'mean_token_accuracy': '0.7359', 'epoch': '0.05798'}
{'loss': '0.6068', 'grad_norm': '0.3302', 'learning_rate': '9.309e-05', 'entropy': '0.6632', 'num_tokens': '3.536e+07', 'mean_token_accuracy': '0.8342', 'epoch': '0.05798'}
{'loss': '0.7865', 'grad_norm': '0.3318', 'learning_rate': '9.306e-05', 'entropy': '0.8142', 'num_tokens': '3.536e+07', 'mean_token_accuracy': '0.7873', 'epoch': '0.05799'}
{'loss': '1.07', 'grad_norm': '0.4782', 'learning_rate': '9.303e-05', 'entropy': '0.9982', 'num_tokens': '3.537e+07', 'mean_token_accuracy': '0.7433', 'epoch': '0.058'}
{'loss': '0.8433', 'grad_norm': '0.3609', 'learning_rate': '9.301e-05', 'entropy': '0.974', 'num_tokens': '3.538e+07', 'mean_token_accuracy': '0.7682', 'epoch': '0.05801'}
{'loss': '0.6844', 'grad_norm': '0.3432', 'learning_rate': '9.298e-05', 'entropy': '0.8502', 'num_tokens': '3.538e+07', 'mean_token_accuracy': '0.7983', 'epoch': '0.05802'}
{'loss': '0.8923', 'grad_norm': '0.3808', 'learning_rate': '9.295e-05', 'entropy': '0.901', 'num_tokens': '3.539e+07', 'mean_token_accuracy': '0.778', 'epoch': '0.05803'}
{'loss': '0.9877', 'grad_norm': '0.4218', 'learning_rate': '9.292e-05', 'entropy': '1.008', 'num_tokens': '3.54e+07', 'mean_token_accuracy': '0.7529', 'epoch': '0.05804'}
{'loss': '1.247', 'grad_norm': '0.5246', 'learning_rate': '9.29e-05', 'entropy': '1.178', 'num_tokens': '3.54e+07', 'mean_token_accuracy': '0.7129', 'epoch': '0.05805'}
{'loss': '0.933', 'grad_norm': '0.4104', 'learning_rate': '9.287e-05', 'entropy': '0.9787', 'num_tokens': '3.54e+07', 'mean_token_accuracy': '0.7618', 'epoch': '0.05805'}
{'loss': '0.7244', 'grad_norm': '0.4012', 'learning_rate': '9.284e-05', 'entropy': '0.7551', 'num_tokens': '3.541e+07', 'mean_token_accuracy': '0.8035', 'epoch': '0.05806'}
{'loss': '0.802', 'grad_norm': '0.3543', 'learning_rate': '9.281e-05', 'entropy': '0.8928', 'num_tokens': '3.541e+07', 'mean_token_accuracy': '0.779', 'epoch': '0.05807'}
{'loss': '1.029', 'grad_norm': '0.4954', 'learning_rate': '9.279e-05', 'entropy': '1.007', 'num_tokens': '3.542e+07', 'mean_token_accuracy': '0.7404', 'epoch': '0.05808'}
{'loss': '0.6162', 'grad_norm': '0.3587', 'learning_rate': '9.276e-05', 'entropy': '0.7031', 'num_tokens': '3.542e+07', 'mean_token_accuracy': '0.8122', 'epoch': '0.05809'}
{'loss': '0.6225', 'grad_norm': '0.3392', 'learning_rate': '9.273e-05', 'entropy': '0.6478', 'num_tokens': '3.543e+07', 'mean_token_accuracy': '0.8204', 'epoch': '0.0581'}
{'loss': '0.898', 'grad_norm': '0.3802', 'learning_rate': '9.27e-05', 'entropy': '0.8992', 'num_tokens': '3.544e+07', 'mean_token_accuracy': '0.7768', 'epoch': '0.05811'}
{'loss': '0.8447', 'grad_norm': '0.3608', 'learning_rate': '9.268e-05', 'entropy': '0.829', 'num_tokens': '3.544e+07', 'mean_token_accuracy': '0.7792', 'epoch': '0.05812'}
{'loss': '1.127', 'grad_norm': '0.342', 'learning_rate': '9.265e-05', 'entropy': '1.125', 'num_tokens': '3.545e+07', 'mean_token_accuracy': '0.7351', 'epoch': '0.05813'}
{'loss': '0.9812', 'grad_norm': '0.4542', 'learning_rate': '9.262e-05', 'entropy': '0.9786', 'num_tokens': '3.545e+07', 'mean_token_accuracy': '0.7438', 'epoch': '0.05813'}
{'loss': '0.6388', 'grad_norm': '0.3618', 'learning_rate': '9.259e-05', 'entropy': '0.6778', 'num_tokens': '3.546e+07', 'mean_token_accuracy': '0.8265', 'epoch': '0.05814'}
{'loss': '0.7705', 'grad_norm': '0.4037', 'learning_rate': '9.257e-05', 'entropy': '0.8646', 'num_tokens': '3.546e+07', 'mean_token_accuracy': '0.7775', 'epoch': '0.05815'}
{'loss': '1.027', 'grad_norm': '0.428', 'learning_rate': '9.254e-05', 'entropy': '0.9574', 'num_tokens': '3.547e+07', 'mean_token_accuracy': '0.7649', 'epoch': '0.05816'}
{'loss': '0.7563', 'grad_norm': '0.3493', 'learning_rate': '9.251e-05', 'entropy': '0.6837', 'num_tokens': '3.548e+07', 'mean_token_accuracy': '0.8282', 'epoch': '0.05817'}
{'loss': '0.8377', 'grad_norm': '0.4588', 'learning_rate': '9.248e-05', 'entropy': '0.806', 'num_tokens': '3.548e+07', 'mean_token_accuracy': '0.7979', 'epoch': '0.05818'}
{'loss': '0.7517', 'grad_norm': '0.4212', 'learning_rate': '9.246e-05', 'entropy': '0.7163', 'num_tokens': '3.549e+07', 'mean_token_accuracy': '0.8191', 'epoch': '0.05819'}
{'loss': '0.7519', 'grad_norm': '0.3851', 'learning_rate': '9.243e-05', 'entropy': '0.7542', 'num_tokens': '3.549e+07', 'mean_token_accuracy': '0.8113', 'epoch': '0.0582'}
{'loss': '0.821', 'grad_norm': '0.4249', 'learning_rate': '9.24e-05', 'entropy': '0.808', 'num_tokens': '3.55e+07', 'mean_token_accuracy': '0.7931', 'epoch': '0.05821'}
{'loss': '0.846', 'grad_norm': '0.4292', 'learning_rate': '9.237e-05', 'entropy': '0.8714', 'num_tokens': '3.55e+07', 'mean_token_accuracy': '0.7797', 'epoch': '0.05821'}
{'loss': '0.6444', 'grad_norm': '0.3544', 'learning_rate': '9.235e-05', 'entropy': '0.6517', 'num_tokens': '3.551e+07', 'mean_token_accuracy': '0.8365', 'epoch': '0.05822'}
{'loss': '0.8174', 'grad_norm': '0.4188', 'learning_rate': '9.232e-05', 'entropy': '0.7844', 'num_tokens': '3.551e+07', 'mean_token_accuracy': '0.7928', 'epoch': '0.05823'}
{'loss': '0.8946', 'grad_norm': '0.5359', 'learning_rate': '9.229e-05', 'entropy': '0.9292', 'num_tokens': '3.551e+07', 'mean_token_accuracy': '0.7724', 'epoch': '0.05824'}
{'loss': '1.054', 'grad_norm': '0.3891', 'learning_rate': '9.226e-05', 'entropy': '0.9236', 'num_tokens': '3.552e+07', 'mean_token_accuracy': '0.7716', 'epoch': '0.05825'}
{'loss': '0.9613', 'grad_norm': '0.4622', 'learning_rate': '9.224e-05', 'entropy': '0.9524', 'num_tokens': '3.552e+07', 'mean_token_accuracy': '0.7621', 'epoch': '0.05826'}

 55%|█████▍    | 6584/12000 [3:53:40<3:28:10,  2.31s/it]
 55%|█████▍    | 6585/12000 [3:53:41<3:11:46,  2.12s/it]
                                                        

 55%|█████▍    | 6585/12000 [3:53:41<3:11:46,  2.12s/it]
 55%|█████▍    | 6586/12000 [3:53:43<2:55:41,  1.95s/it]
                                                        

 55%|█████▍    | 6586/12000 [3:53:43<2:55:41,  1.95s/it]
 55%|█████▍    | 6587/12000 [3:53:44<2:42:51,  1.81s/it]
                                                        

 55%|█████▍    | 6587/12000 [3:53:44<2:42:51,  1.81s/it]
 55%|█████▍    | 6588/12000 [3:53:46<2:36:20,  1.73s/it]
                                                        

 55%|█████▍    | 6588/12000 [3:53:46<2:36:20,  1.73s/it]
 55%|█████▍    | 6589/12000 [3:53:48<2:44:53,  1.83s/it]
                                                        

 55%|█████▍    | 6589/12000 [3:53:48<2:44:53,  1.83s/it]
 55%|█████▍    | 6590/12000 [3:53:50<2:50:29,  1.89s/it]
                                                        

 55%|█████▍    | 6590/12000 [3:53:50<2:50:29,  1.89s/it]
 55%|█████▍    | 6591/12000 [3:53:54<3:33:55,  2.37s/it]
                                                        

 55%|█████▍    | 6591/12000 [3:53:54<3:33:55,  2.37s/it]
 55%|█████▍    | 6592/12000 [3:53:56<3:26:19,  2.29s/it]
                                                        

 55%|█████▍    | 6592/12000 [3:53:56<3:26:19,  2.29s/it]
 55%|█████▍    | 6593/12000 [3:53:57<3:07:17,  2.08s/it]
                                                        

 55%|█████▍    | 6593/12000 [3:53:57<3:07:17,  2.08s/it]
 55%|█████▍    | 6594/12000 [3:53:59<2:50:20,  1.89s/it]
                                                        

 55%|█████▍    | 6594/12000 [3:53:59<2:50:20,  1.89s/it]
 55%|█████▍    | 6595/12000 [3:54:03<3:41:35,  2.46s/it]
                                                        

 55%|█████▍    | 6595/12000 [3:54:03<3:41:35,  2.46s/it]
 55%|█████▍    | 6596/12000 [3:54:04<3:25:05,  2.28s/it]
                                                        

 55%|█████▍    | 6596/12000 [3:54:04<3:25:05,  2.28s/it]
 55%|█████▍    | 6597/12000 [3:54:06<3:14:44,  2.16s/it]
                                                        

 55%|█████▍    | 6597/12000 [3:54:06<3:14:44,  2.16s/it]
 55%|█████▍    | 6598/12000 [3:54:08<2:53:37,  1.93s/it]
                                                        

 55%|█████▍    | 6598/12000 [3:54:08<2:53:37,  1.93s/it]
 55%|█████▍    | 6599/12000 [3:54:09<2:41:51,  1.80s/it]
                                                        

 55%|█████▍    | 6599/12000 [3:54:09<2:41:51,  1.80s/it]
 55%|█████▌    | 6600/12000 [3:54:11<2:41:57,  1.80s/it]
                                                        

 55%|█████▌    | 6600/12000 [3:54:11<2:41:57,  1.80s/it]
 55%|█████▌    | 6601/12000 [3:54:14<3:08:28,  2.09s/it]
                                                        

 55%|█████▌    | 6601/12000 [3:54:14<3:08:28,  2.09s/it]
 55%|█████▌    | 6602/12000 [3:54:15<2:58:43,  1.99s/it]
                                                        

 55%|█████▌    | 6602/12000 [3:54:15<2:58:43,  1.99s/it]
 55%|█████▌    | 6603/12000 [3:54:17<2:53:00,  1.92s/it]
                                                        

 55%|█████▌    | 6603/12000 [3:54:17<2:53:00,  1.92s/it]
 55%|█████▌    | 6604/12000 [3:54:19<2:41:40,  1.80s/it]
                                                        

 55%|█████▌    | 6604/12000 [3:54:19<2:41:40,  1.80s/it]
 55%|█████▌    | 6605/12000 [3:54:21<3:04:01,  2.05s/it]
                                                        

 55%|█████▌    | 6605/12000 [3:54:21<3:04:01,  2.05s/it]
 55%|█████▌    | 6606/12000 [3:54:23<2:51:57,  1.91s/it]
                                                        

 55%|█████▌    | 6606/12000 [3:54:23<2:51:57,  1.91s/it]
 55%|█████▌    | 6607/12000 [3:54:28<4:05:59,  2.74s/it]
                                                        

 55%|█████▌    | 6607/12000 [3:54:28<4:05:59,  2.74s/it]
 55%|█████▌    | 6608/12000 [3:54:32<4:41:14,  3.13s/it]
                                                        

 55%|█████▌    | 6608/12000 [3:54:32<4:41:14,  3.13s/it]
 55%|█████▌    | 6609/12000 [3:54:33<3:56:50,  2.64s/it]
                                                        

 55%|█████▌    | 6609/12000 [3:54:33<3:56:50,  2.64s/it]
 55%|█████▌    | 6610/12000 [3:54:36<3:49:31,  2.56s/it]
                                                        

 55%|█████▌    | 6610/12000 [3:54:36<3:49:31,  2.56s/it]
 55%|█████▌    | 6611/12000 [3:54:37<3:22:49,  2.26s/it]
                                                        

 55%|█████▌    | 6611/12000 [3:54:37<3:22:49,  2.26s/it]
 55%|█████▌    | 6612/12000 [3:54:39<3:02:04,  2.03s/it]
                                                        

 55%|█████▌    | 6612/12000 [3:54:39<3:02:04,  2.03s/it]
 55%|█████▌    | 6613/12000 [3:54:43<4:07:36,  2.76s/it]
                                                        

 55%|█████▌    | 6613/12000 [3:54:43<4:07:36,  2.76s/it]
 55%|█████▌    | 6614/12000 [3:54:45<3:36:53,  2.42s/it]
                                                        

 55%|█████▌    | 6614/12000 [3:54:45<3:36:53,  2.42s/it]
 55%|█████▌    | 6615/12000 [3:54:46<3:15:14,  2.18s/it]
                                                        

 55%|█████▌    | 6615/12000 [3:54:46<3:15:14,  2.18s/it]
 55%|█████▌    | 6616/12000 [3:54:49<3:35:45,  2.40s/it]
                                                        

 55%|█████▌    | 6616/12000 [3:54:49<3:35:45,  2.40s/it]
 55%|█████▌    | 6617/12000 [3:54:51<3:12:55,  2.15s/it]
                                                        

 55%|█████▌    | 6617/12000 [3:54:51<3:12:55,  2.15s/it]
 55%|█████▌    | 6618/12000 [3:54:52<2:58:08,  1.99s/it]
                                                        

 55%|█████▌    | 6618/12000 [3:54:52<2:58:08,  1.99s/it]
 55%|█████▌    | 6619/12000 [3:54:54<2:49:55,  1.89s/it]
                                                        

 55%|█████▌    | 6619/12000 [3:54:54<2:49:55,  1.89s/it]
 55%|█████▌    | 6620/12000 [3:54:56<3:00:54,  2.02s/it]
                                                        

 55%|█████▌    | 6620/12000 [3:54:56<3:00:54,  2.02s/it]
 55%|█████▌    | 6621/12000 [3:54:59<3:10:02,  2.12s/it]
                                                        

 55%|█████▌    | 6621/12000 [3:54:59<3:10:02,  2.12s/it]
 55%|█████▌    | 6622/12000 [3:55:00<2:53:20,  1.93s/it]
                                                        

 55%|█████▌    | 6622/12000 [3:55:00<2:53:20,  1.93s/it]
 55%|█████▌    | 6623/12000 [3:55:02<2:41:52,  1.81s/it]
                                                        

 55%|█████▌    | 6623/12000 [3:55:02<2:41:52,  1.81s/it]
 55%|█████▌    | 6624/12000 [3:55:05<3:22:59,  2.27s/it]
                                                        

 55%|█████▌    | 6624/12000 [3:55:05<3:22:59,  2.27s/it]
 55%|█████▌    | 6625/12000 [3:55:07<3:24:11,  2.28s/it]
                                                        

 55%|█████▌    | 6625/12000 [3:55:07<3:24:11,  2.28s/it]
 55%|█████▌    | 6626/12000 [3:55:09<3:04:51,  2.06s/it]
                                                        

 55%|█████▌    | 6626/12000 [3:55:09<3:04:51,  2.06s/it]
 55%|█████▌    | 6627/12000 [3:55:15<4:42:30,  3.15s/it]
                                                        

 55%|█████▌    | 6627/12000 [3:55:15<4:42:30,  3.15s/it]
 55%|█████▌    | 6628/12000 [3:55:17<4:09:47,  2.79s/it]
                                                        

 55%|█████▌    | 6628/12000 [3:55:17<4:09:47,  2.79s/it]
 55%|█████▌    | 6629/12000 [3:55:18<3:32:03,  2.37s/it]
                                                        

 55%|█████▌    | 6629/12000 [3:55:18<3:32:03,  2.37s/it]
 55%|█████▌    | 6630/12000 [3:55:20<3:15:31,  2.18s/it]
                                                        

 55%|█████▌    | 6630/12000 [3:55:20<3:15:31,  2.18s/it]
 55%|█████▌    | 6631/12000 [3:55:21<2:59:31,  2.01s/it]
                                                        
{'loss': '0.7518', 'grad_norm': '0.4195', 'learning_rate': '9.221e-05', 'entropy': '0.757', 'num_tokens': '3.553e+07', 'mean_token_accuracy': '0.8084', 'epoch': '0.05827'}
{'loss': '0.7666', 'grad_norm': '0.3105', 'learning_rate': '9.218e-05', 'entropy': '0.7479', 'num_tokens': '3.554e+07', 'mean_token_accuracy': '0.8099', 'epoch': '0.05828'}
{'loss': '0.9345', 'grad_norm': '0.38', 'learning_rate': '9.215e-05', 'entropy': '0.9932', 'num_tokens': '3.554e+07', 'mean_token_accuracy': '0.7617', 'epoch': '0.05828'}
{'loss': '0.6382', 'grad_norm': '0.3534', 'learning_rate': '9.213e-05', 'entropy': '0.7428', 'num_tokens': '3.555e+07', 'mean_token_accuracy': '0.8262', 'epoch': '0.05829'}
{'loss': '0.6306', 'grad_norm': '0.3511', 'learning_rate': '9.21e-05', 'entropy': '0.6676', 'num_tokens': '3.555e+07', 'mean_token_accuracy': '0.8179', 'epoch': '0.0583'}
{'loss': '0.802', 'grad_norm': '0.4353', 'learning_rate': '9.207e-05', 'entropy': '0.7649', 'num_tokens': '3.556e+07', 'mean_token_accuracy': '0.7975', 'epoch': '0.05831'}
{'loss': '1.061', 'grad_norm': '0.3851', 'learning_rate': '9.204e-05', 'entropy': '0.9447', 'num_tokens': '3.556e+07', 'mean_token_accuracy': '0.7706', 'epoch': '0.05832'}
{'loss': '0.691', 'grad_norm': '0.4246', 'learning_rate': '9.202e-05', 'entropy': '0.6856', 'num_tokens': '3.557e+07', 'mean_token_accuracy': '0.8249', 'epoch': '0.05833'}
{'loss': '0.9103', 'grad_norm': '0.378', 'learning_rate': '9.199e-05', 'entropy': '0.9964', 'num_tokens': '3.558e+07', 'mean_token_accuracy': '0.7673', 'epoch': '0.05834'}
{'loss': '0.7582', 'grad_norm': '0.4344', 'learning_rate': '9.196e-05', 'entropy': '0.8119', 'num_tokens': '3.558e+07', 'mean_token_accuracy': '0.7958', 'epoch': '0.05835'}
{'loss': '0.6174', 'grad_norm': '0.3735', 'learning_rate': '9.193e-05', 'entropy': '0.6284', 'num_tokens': '3.558e+07', 'mean_token_accuracy': '0.8364', 'epoch': '0.05836'}
{'loss': '1.225', 'grad_norm': '0.3944', 'learning_rate': '9.191e-05', 'entropy': '1.11', 'num_tokens': '3.559e+07', 'mean_token_accuracy': '0.7324', 'epoch': '0.05836'}
{'loss': '0.8891', 'grad_norm': '0.4063', 'learning_rate': '9.188e-05', 'entropy': '0.9053', 'num_tokens': '3.56e+07', 'mean_token_accuracy': '0.7708', 'epoch': '0.05837'}
{'loss': '0.821', 'grad_norm': '0.3716', 'learning_rate': '9.185e-05', 'entropy': '0.7483', 'num_tokens': '3.561e+07', 'mean_token_accuracy': '0.8029', 'epoch': '0.05838'}
{'loss': '0.6378', 'grad_norm': '0.3519', 'learning_rate': '9.182e-05', 'entropy': '0.6843', 'num_tokens': '3.561e+07', 'mean_token_accuracy': '0.822', 'epoch': '0.05839'}
{'loss': '0.638', 'grad_norm': '0.4083', 'learning_rate': '9.18e-05', 'entropy': '0.6656', 'num_tokens': '3.561e+07', 'mean_token_accuracy': '0.8234', 'epoch': '0.0584'}
{'loss': '0.8446', 'grad_norm': '0.4563', 'learning_rate': '9.177e-05', 'entropy': '0.7785', 'num_tokens': '3.562e+07', 'mean_token_accuracy': '0.8059', 'epoch': '0.05841'}
{'loss': '0.7057', 'grad_norm': '0.3194', 'learning_rate': '9.174e-05', 'entropy': '0.6757', 'num_tokens': '3.562e+07', 'mean_token_accuracy': '0.8318', 'epoch': '0.05842'}
{'loss': '0.6265', 'grad_norm': '0.359', 'learning_rate': '9.171e-05', 'entropy': '0.646', 'num_tokens': '3.563e+07', 'mean_token_accuracy': '0.8363', 'epoch': '0.05843'}
{'loss': '1.237', 'grad_norm': '0.4038', 'learning_rate': '9.169e-05', 'entropy': '1.229', 'num_tokens': '3.563e+07', 'mean_token_accuracy': '0.7104', 'epoch': '0.05844'}
{'loss': '0.7895', 'grad_norm': '0.338', 'learning_rate': '9.166e-05', 'entropy': '0.7932', 'num_tokens': '3.564e+07', 'mean_token_accuracy': '0.8043', 'epoch': '0.05844'}
{'loss': '1.092', 'grad_norm': '0.3748', 'learning_rate': '9.163e-05', 'entropy': '1.093', 'num_tokens': '3.565e+07', 'mean_token_accuracy': '0.7363', 'epoch': '0.05845'}
{'loss': '0.7791', 'grad_norm': '0.413', 'learning_rate': '9.16e-05', 'entropy': '0.7796', 'num_tokens': '3.565e+07', 'mean_token_accuracy': '0.7985', 'epoch': '0.05846'}
{'loss': '0.958', 'grad_norm': '0.3835', 'learning_rate': '9.158e-05', 'entropy': '0.6754', 'num_tokens': '3.566e+07', 'mean_token_accuracy': '0.8259', 'epoch': '0.05847'}
{'loss': '0.5396', 'grad_norm': '0.2475', 'learning_rate': '9.155e-05', 'entropy': '0.5292', 'num_tokens': '3.567e+07', 'mean_token_accuracy': '0.859', 'epoch': '0.05848'}
{'loss': '0.6976', 'grad_norm': '0.3812', 'learning_rate': '9.152e-05', 'entropy': '0.6818', 'num_tokens': '3.567e+07', 'mean_token_accuracy': '0.827', 'epoch': '0.05849'}
{'loss': '0.7503', 'grad_norm': '0.3662', 'learning_rate': '9.149e-05', 'entropy': '0.7445', 'num_tokens': '3.568e+07', 'mean_token_accuracy': '0.8168', 'epoch': '0.0585'}
{'loss': '0.6839', 'grad_norm': '0.4097', 'learning_rate': '9.147e-05', 'entropy': '0.6644', 'num_tokens': '3.568e+07', 'mean_token_accuracy': '0.8197', 'epoch': '0.05851'}
{'loss': '0.6888', 'grad_norm': '0.3735', 'learning_rate': '9.144e-05', 'entropy': '0.7515', 'num_tokens': '3.569e+07', 'mean_token_accuracy': '0.8107', 'epoch': '0.05852'}
{'loss': '0.5914', 'grad_norm': '0.3873', 'learning_rate': '9.141e-05', 'entropy': '0.6365', 'num_tokens': '3.569e+07', 'mean_token_accuracy': '0.8239', 'epoch': '0.05852'}
{'loss': '0.6855', 'grad_norm': '0.4204', 'learning_rate': '9.139e-05', 'entropy': '0.6693', 'num_tokens': '3.57e+07', 'mean_token_accuracy': '0.8224', 'epoch': '0.05853'}
{'loss': '0.6516', 'grad_norm': '0.4115', 'learning_rate': '9.136e-05', 'entropy': '0.6134', 'num_tokens': '3.57e+07', 'mean_token_accuracy': '0.8408', 'epoch': '0.05854'}
{'loss': '1.494', 'grad_norm': '0.453', 'learning_rate': '9.133e-05', 'entropy': '1.247', 'num_tokens': '3.571e+07', 'mean_token_accuracy': '0.6953', 'epoch': '0.05855'}
{'loss': '0.6269', 'grad_norm': '0.3109', 'learning_rate': '9.13e-05', 'entropy': '0.6486', 'num_tokens': '3.571e+07', 'mean_token_accuracy': '0.8282', 'epoch': '0.05856'}
{'loss': '0.6913', 'grad_norm': '0.4264', 'learning_rate': '9.128e-05', 'entropy': '0.6471', 'num_tokens': '3.572e+07', 'mean_token_accuracy': '0.8267', 'epoch': '0.05857'}
{'loss': '0.6786', 'grad_norm': '0.4091', 'learning_rate': '9.125e-05', 'entropy': '0.7084', 'num_tokens': '3.572e+07', 'mean_token_accuracy': '0.8167', 'epoch': '0.05858'}
{'loss': '0.9717', 'grad_norm': '0.3971', 'learning_rate': '9.122e-05', 'entropy': '0.7851', 'num_tokens': '3.573e+07', 'mean_token_accuracy': '0.792', 'epoch': '0.05859'}
{'loss': '0.893', 'grad_norm': '0.3601', 'learning_rate': '9.119e-05', 'entropy': '0.9552', 'num_tokens': '3.573e+07', 'mean_token_accuracy': '0.759', 'epoch': '0.05859'}
{'loss': '0.6383', 'grad_norm': '0.2903', 'learning_rate': '9.117e-05', 'entropy': '0.6634', 'num_tokens': '3.574e+07', 'mean_token_accuracy': '0.8256', 'epoch': '0.0586'}
{'loss': '0.6796', 'grad_norm': '0.3318', 'learning_rate': '9.114e-05', 'entropy': '0.6974', 'num_tokens': '3.574e+07', 'mean_token_accuracy': '0.826', 'epoch': '0.05861'}
{'loss': '0.819', 'grad_norm': '0.3933', 'learning_rate': '9.111e-05', 'entropy': '0.6757', 'num_tokens': '3.575e+07', 'mean_token_accuracy': '0.816', 'epoch': '0.05862'}
{'loss': '0.7355', 'grad_norm': '0.3274', 'learning_rate': '9.108e-05', 'entropy': '0.7181', 'num_tokens': '3.576e+07', 'mean_token_accuracy': '0.8203', 'epoch': '0.05863'}
{'loss': '0.6291', 'grad_norm': '0.4177', 'learning_rate': '9.106e-05', 'entropy': '0.647', 'num_tokens': '3.576e+07', 'mean_token_accuracy': '0.8326', 'epoch': '0.05864'}
{'loss': '0.8634', 'grad_norm': '0.3628', 'learning_rate': '9.103e-05', 'entropy': '0.8845', 'num_tokens': '3.577e+07', 'mean_token_accuracy': '0.7789', 'epoch': '0.05865'}
{'loss': '1.132', 'grad_norm': '0.3837', 'learning_rate': '9.1e-05', 'entropy': '1.137', 'num_tokens': '3.578e+07', 'mean_token_accuracy': '0.7192', 'epoch': '0.05866'}
{'loss': '0.7656', 'grad_norm': '0.478', 'learning_rate': '9.097e-05', 'entropy': '0.8288', 'num_tokens': '3.578e+07', 'mean_token_accuracy': '0.8026', 'epoch': '0.05867'}
{'loss': '0.7815', 'grad_norm': '0.4706', 'learning_rate': '9.095e-05', 'entropy': '0.7694', 'num_tokens': '3.579e+07', 'mean_token_accuracy': '0.7929', 'epoch': '0.05867'}

 55%|█████▌    | 6631/12000 [3:55:21<2:59:31,  2.01s/it]
 55%|█████▌    | 6632/12000 [3:55:23<2:46:27,  1.86s/it]
                                                        

 55%|█████▌    | 6632/12000 [3:55:23<2:46:27,  1.86s/it]
 55%|█████▌    | 6633/12000 [3:55:25<2:51:06,  1.91s/it]
                                                        

 55%|█████▌    | 6633/12000 [3:55:25<2:51:06,  1.91s/it]
 55%|█████▌    | 6634/12000 [3:55:27<2:43:54,  1.83s/it]
                                                        

 55%|█████▌    | 6634/12000 [3:55:27<2:43:54,  1.83s/it]
 55%|█████▌    | 6635/12000 [3:55:31<4:04:36,  2.74s/it]
                                                        

 55%|█████▌    | 6635/12000 [3:55:31<4:04:36,  2.74s/it]
 55%|█████▌    | 6636/12000 [3:55:33<3:30:29,  2.35s/it]
                                                        

 55%|█████▌    | 6636/12000 [3:55:33<3:30:29,  2.35s/it]
 55%|█████▌    | 6637/12000 [3:55:34<3:12:25,  2.15s/it]
                                                        

 55%|█████▌    | 6637/12000 [3:55:34<3:12:25,  2.15s/it]
 55%|█████▌    | 6638/12000 [3:55:37<3:08:59,  2.11s/it]
                                                        

 55%|█████▌    | 6638/12000 [3:55:37<3:08:59,  2.11s/it]
 55%|█████▌    | 6639/12000 [3:55:39<3:27:23,  2.32s/it]
                                                        

 55%|█████▌    | 6639/12000 [3:55:39<3:27:23,  2.32s/it]
 55%|█████▌    | 6640/12000 [3:55:41<3:04:40,  2.07s/it]
                                                        

 55%|█████▌    | 6640/12000 [3:55:41<3:04:40,  2.07s/it]
 55%|█████▌    | 6641/12000 [3:55:43<2:57:08,  1.98s/it]
                                                        

 55%|█████▌    | 6641/12000 [3:55:43<2:57:08,  1.98s/it]
 55%|█████▌    | 6642/12000 [3:55:45<2:59:40,  2.01s/it]
                                                        

 55%|█████▌    | 6642/12000 [3:55:45<2:59:40,  2.01s/it]
 55%|█████▌    | 6643/12000 [3:55:47<3:04:03,  2.06s/it]
                                                        

 55%|█████▌    | 6643/12000 [3:55:47<3:04:03,  2.06s/it]
 55%|█████▌    | 6644/12000 [3:55:48<2:48:27,  1.89s/it]
                                                        

 55%|█████▌    | 6644/12000 [3:55:48<2:48:27,  1.89s/it]
 55%|█████▌    | 6645/12000 [3:55:51<2:59:31,  2.01s/it]
                                                        

 55%|█████▌    | 6645/12000 [3:55:51<2:59:31,  2.01s/it]
 55%|█████▌    | 6646/12000 [3:55:52<2:43:32,  1.83s/it]
                                                        

 55%|█████▌    | 6646/12000 [3:55:52<2:43:32,  1.83s/it]
 55%|█████▌    | 6647/12000 [3:55:54<2:40:52,  1.80s/it]
                                                        

 55%|█████▌    | 6647/12000 [3:55:54<2:40:52,  1.80s/it]
 55%|█████▌    | 6648/12000 [3:55:55<2:38:38,  1.78s/it]
                                                        

 55%|█████▌    | 6648/12000 [3:55:55<2:38:38,  1.78s/it]
 55%|█████▌    | 6649/12000 [3:55:57<2:39:00,  1.78s/it]
                                                        

 55%|█████▌    | 6649/12000 [3:55:57<2:39:00,  1.78s/it]
 55%|█████▌    | 6650/12000 [3:55:59<2:25:48,  1.64s/it]
                                                        

 55%|█████▌    | 6650/12000 [3:55:59<2:25:48,  1.64s/it]
 55%|█████▌    | 6651/12000 [3:56:02<3:08:25,  2.11s/it]
                                                        

 55%|█████▌    | 6651/12000 [3:56:02<3:08:25,  2.11s/it]
 55%|█████▌    | 6652/12000 [3:56:03<2:53:07,  1.94s/it]
                                                        

 55%|█████▌    | 6652/12000 [3:56:03<2:53:07,  1.94s/it]
 55%|█████▌    | 6653/12000 [3:56:05<2:56:33,  1.98s/it]
                                                        

 55%|█████▌    | 6653/12000 [3:56:05<2:56:33,  1.98s/it]
 55%|█████▌    | 6654/12000 [3:56:07<2:56:38,  1.98s/it]
                                                        

 55%|█████▌    | 6654/12000 [3:56:07<2:56:38,  1.98s/it]
 55%|█████▌    | 6655/12000 [3:56:09<2:58:10,  2.00s/it]
                                                        

 55%|█████▌    | 6655/12000 [3:56:09<2:58:10,  2.00s/it]
 55%|█████▌    | 6656/12000 [3:56:12<3:09:45,  2.13s/it]
                                                        

 55%|█████▌    | 6656/12000 [3:56:12<3:09:45,  2.13s/it]
 55%|█████▌    | 6657/12000 [3:56:15<3:48:19,  2.56s/it]
                                                        

 55%|█████▌    | 6657/12000 [3:56:15<3:48:19,  2.56s/it]
 55%|█████▌    | 6658/12000 [3:56:17<3:17:49,  2.22s/it]
                                                        

 55%|█████▌    | 6658/12000 [3:56:17<3:17:49,  2.22s/it]
 55%|█████▌    | 6659/12000 [3:56:19<3:20:16,  2.25s/it]
                                                        

 55%|█████▌    | 6659/12000 [3:56:19<3:20:16,  2.25s/it]
 56%|█████▌    | 6660/12000 [3:56:21<3:02:13,  2.05s/it]
                                                        

 56%|█████▌    | 6660/12000 [3:56:21<3:02:13,  2.05s/it]
 56%|█████▌    | 6661/12000 [3:56:22<2:49:38,  1.91s/it]
                                                        

 56%|█████▌    | 6661/12000 [3:56:22<2:49:38,  1.91s/it]
 56%|█████▌    | 6662/12000 [3:56:24<2:38:24,  1.78s/it]
                                                        

 56%|█████▌    | 6662/12000 [3:56:24<2:38:24,  1.78s/it]
 56%|█████▌    | 6663/12000 [3:56:26<2:52:19,  1.94s/it]
                                                        

 56%|█████▌    | 6663/12000 [3:56:26<2:52:19,  1.94s/it]
 56%|█████▌    | 6664/12000 [3:56:28<2:48:16,  1.89s/it]
                                                        

 56%|█████▌    | 6664/12000 [3:56:28<2:48:16,  1.89s/it]
 56%|█████▌    | 6665/12000 [3:56:30<2:44:12,  1.85s/it]
                                                        

 56%|█████▌    | 6665/12000 [3:56:30<2:44:12,  1.85s/it]
 56%|█████▌    | 6666/12000 [3:56:32<3:02:32,  2.05s/it]
                                                        

 56%|█████▌    | 6666/12000 [3:56:32<3:02:32,  2.05s/it]
 56%|█████▌    | 6667/12000 [3:56:34<2:54:35,  1.96s/it]
                                                        

 56%|█████▌    | 6667/12000 [3:56:34<2:54:35,  1.96s/it]
 56%|█████▌    | 6668/12000 [3:56:35<2:39:39,  1.80s/it]
                                                        

 56%|█████▌    | 6668/12000 [3:56:35<2:39:39,  1.80s/it]
 56%|█████▌    | 6669/12000 [3:56:37<2:33:53,  1.73s/it]
                                                        

 56%|█████▌    | 6669/12000 [3:56:37<2:33:53,  1.73s/it]
 56%|█████▌    | 6670/12000 [3:56:39<2:33:47,  1.73s/it]
                                                        

 56%|█████▌    | 6670/12000 [3:56:39<2:33:47,  1.73s/it]
 56%|█████▌    | 6671/12000 [3:56:40<2:30:18,  1.69s/it]
                                                        

 56%|█████▌    | 6671/12000 [3:56:40<2:30:18,  1.69s/it]
 56%|█████▌    | 6672/12000 [3:56:42<2:24:51,  1.63s/it]
                                                        

 56%|█████▌    | 6672/12000 [3:56:42<2:24:51,  1.63s/it]
 56%|█████▌    | 6673/12000 [3:56:43<2:22:03,  1.60s/it]
                                                        

 56%|█████▌    | 6673/12000 [3:56:43<2:22:03,  1.60s/it]
 56%|█████▌    | 6674/12000 [3:56:45<2:26:51,  1.65s/it]
                                                        

 56%|█████▌    | 6674/12000 [3:56:45<2:26:51,  1.65s/it]
 56%|█████▌    | 6675/12000 [3:56:47<2:33:01,  1.72s/it]
                                                        

 56%|█████▌    | 6675/12000 [3:56:47<2:33:01,  1.72s/it]
 56%|█████▌    | 6676/12000 [3:56:49<2:31:23,  1.71s/it]
                                                        

 56%|█████▌    | 6676/12000 [3:56:49<2:31:23,  1.71s/it]
 56%|█████▌    | 6677/12000 [3:56:50<2:26:00,  1.65s/it]
                                                        

 56%|█████▌    | 6677/12000 [3:56:50<2:26:00,  1.65s/it]
 56%|█████▌    | 6678/12000 [3:56:54<3:22:30,  2.28s/it]
                                                        
{'loss': '0.6702', 'grad_norm': '0.3369', 'learning_rate': '9.092e-05', 'entropy': '0.7641', 'num_tokens': '3.579e+07', 'mean_token_accuracy': '0.8144', 'epoch': '0.05868'}
{'loss': '0.6412', 'grad_norm': '0.3896', 'learning_rate': '9.089e-05', 'entropy': '0.6335', 'num_tokens': '3.58e+07', 'mean_token_accuracy': '0.8344', 'epoch': '0.05869'}
{'loss': '0.7131', 'grad_norm': '0.4078', 'learning_rate': '9.086e-05', 'entropy': '0.6889', 'num_tokens': '3.58e+07', 'mean_token_accuracy': '0.8204', 'epoch': '0.0587'}
{'loss': '0.7771', 'grad_norm': '0.4135', 'learning_rate': '9.084e-05', 'entropy': '0.8333', 'num_tokens': '3.581e+07', 'mean_token_accuracy': '0.7926', 'epoch': '0.05871'}
{'loss': '0.8033', 'grad_norm': '0.382', 'learning_rate': '9.081e-05', 'entropy': '0.8739', 'num_tokens': '3.581e+07', 'mean_token_accuracy': '0.7727', 'epoch': '0.05872'}
{'loss': '1.012', 'grad_norm': '0.5909', 'learning_rate': '9.078e-05', 'entropy': '1.146', 'num_tokens': '3.582e+07', 'mean_token_accuracy': '0.7282', 'epoch': '0.05873'}
{'loss': '0.7137', 'grad_norm': '0.4693', 'learning_rate': '9.075e-05', 'entropy': '0.7246', 'num_tokens': '3.582e+07', 'mean_token_accuracy': '0.8017', 'epoch': '0.05874'}
{'loss': '0.8944', 'grad_norm': '0.4862', 'learning_rate': '9.073e-05', 'entropy': '0.8137', 'num_tokens': '3.583e+07', 'mean_token_accuracy': '0.7893', 'epoch': '0.05875'}
{'loss': '0.9129', 'grad_norm': '0.4109', 'learning_rate': '9.07e-05', 'entropy': '0.7968', 'num_tokens': '3.583e+07', 'mean_token_accuracy': '0.8005', 'epoch': '0.05875'}
{'loss': '0.8399', 'grad_norm': '0.3943', 'learning_rate': '9.067e-05', 'entropy': '0.8558', 'num_tokens': '3.584e+07', 'mean_token_accuracy': '0.7708', 'epoch': '0.05876'}
{'loss': '1.305', 'grad_norm': '0.4602', 'learning_rate': '9.064e-05', 'entropy': '1.239', 'num_tokens': '3.584e+07', 'mean_token_accuracy': '0.7284', 'epoch': '0.05877'}
{'loss': '0.6995', 'grad_norm': '0.3329', 'learning_rate': '9.062e-05', 'entropy': '0.7529', 'num_tokens': '3.585e+07', 'mean_token_accuracy': '0.8046', 'epoch': '0.05878'}
{'loss': '1.204', 'grad_norm': '0.5087', 'learning_rate': '9.059e-05', 'entropy': '1.01', 'num_tokens': '3.585e+07', 'mean_token_accuracy': '0.7475', 'epoch': '0.05879'}
{'loss': '0.8532', 'grad_norm': '0.391', 'learning_rate': '9.056e-05', 'entropy': '0.8675', 'num_tokens': '3.586e+07', 'mean_token_accuracy': '0.7655', 'epoch': '0.0588'}
{'loss': '0.5633', 'grad_norm': '0.2916', 'learning_rate': '9.053e-05', 'entropy': '0.6085', 'num_tokens': '3.587e+07', 'mean_token_accuracy': '0.8376', 'epoch': '0.05881'}
{'loss': '0.7207', 'grad_norm': '0.4456', 'learning_rate': '9.051e-05', 'entropy': '0.7393', 'num_tokens': '3.587e+07', 'mean_token_accuracy': '0.8135', 'epoch': '0.05882'}
{'loss': '0.6683', 'grad_norm': '0.3402', 'learning_rate': '9.048e-05', 'entropy': '0.6166', 'num_tokens': '3.587e+07', 'mean_token_accuracy': '0.8394', 'epoch': '0.05882'}
{'loss': '0.5648', 'grad_norm': '0.3018', 'learning_rate': '9.045e-05', 'entropy': '0.583', 'num_tokens': '3.588e+07', 'mean_token_accuracy': '0.8491', 'epoch': '0.05883'}
{'loss': '0.9393', 'grad_norm': '0.4309', 'learning_rate': '9.042e-05', 'entropy': '0.8671', 'num_tokens': '3.588e+07', 'mean_token_accuracy': '0.775', 'epoch': '0.05884'}
{'loss': '0.8128', 'grad_norm': '0.5071', 'learning_rate': '9.04e-05', 'entropy': '0.7681', 'num_tokens': '3.589e+07', 'mean_token_accuracy': '0.8087', 'epoch': '0.05885'}
{'loss': '0.5559', 'grad_norm': '0.2846', 'learning_rate': '9.037e-05', 'entropy': '0.6153', 'num_tokens': '3.589e+07', 'mean_token_accuracy': '0.842', 'epoch': '0.05886'}
{'loss': '0.7882', 'grad_norm': '0.3287', 'learning_rate': '9.034e-05', 'entropy': '0.7636', 'num_tokens': '3.59e+07', 'mean_token_accuracy': '0.8118', 'epoch': '0.05887'}
{'loss': '0.934', 'grad_norm': '0.3443', 'learning_rate': '9.031e-05', 'entropy': '0.9399', 'num_tokens': '3.59e+07', 'mean_token_accuracy': '0.7638', 'epoch': '0.05888'}
{'loss': '1.04', 'grad_norm': '0.4979', 'learning_rate': '9.029e-05', 'entropy': '0.9982', 'num_tokens': '3.591e+07', 'mean_token_accuracy': '0.7611', 'epoch': '0.05889'}
{'loss': '1.492', 'grad_norm': '0.4464', 'learning_rate': '9.026e-05', 'entropy': '1.131', 'num_tokens': '3.591e+07', 'mean_token_accuracy': '0.73', 'epoch': '0.0589'}
{'loss': '1.165', 'grad_norm': '0.3348', 'learning_rate': '9.023e-05', 'entropy': '1.118', 'num_tokens': '3.592e+07', 'mean_token_accuracy': '0.7193', 'epoch': '0.0589'}
{'loss': '0.8209', 'grad_norm': '0.3993', 'learning_rate': '9.021e-05', 'entropy': '0.8436', 'num_tokens': '3.592e+07', 'mean_token_accuracy': '0.7931', 'epoch': '0.05891'}
{'loss': '0.6833', 'grad_norm': '0.4394', 'learning_rate': '9.018e-05', 'entropy': '0.7543', 'num_tokens': '3.593e+07', 'mean_token_accuracy': '0.8142', 'epoch': '0.05892'}
{'loss': '0.7296', 'grad_norm': '0.4071', 'learning_rate': '9.015e-05', 'entropy': '0.9315', 'num_tokens': '3.593e+07', 'mean_token_accuracy': '0.7706', 'epoch': '0.05893'}
{'loss': '1.18', 'grad_norm': '0.4693', 'learning_rate': '9.012e-05', 'entropy': '1.183', 'num_tokens': '3.594e+07', 'mean_token_accuracy': '0.7149', 'epoch': '0.05894'}
{'loss': '0.7799', 'grad_norm': '0.3577', 'learning_rate': '9.01e-05', 'entropy': '0.8186', 'num_tokens': '3.594e+07', 'mean_token_accuracy': '0.7889', 'epoch': '0.05895'}
{'loss': '0.6256', 'grad_norm': '0.3698', 'learning_rate': '9.007e-05', 'entropy': '0.6612', 'num_tokens': '3.595e+07', 'mean_token_accuracy': '0.8252', 'epoch': '0.05896'}
{'loss': '0.6487', 'grad_norm': '0.3804', 'learning_rate': '9.004e-05', 'entropy': '0.7632', 'num_tokens': '3.595e+07', 'mean_token_accuracy': '0.8035', 'epoch': '0.05897'}
{'loss': '1.131', 'grad_norm': '0.4107', 'learning_rate': '9.001e-05', 'entropy': '1.015', 'num_tokens': '3.596e+07', 'mean_token_accuracy': '0.7462', 'epoch': '0.05898'}
{'loss': '0.9757', 'grad_norm': '0.4297', 'learning_rate': '8.999e-05', 'entropy': '0.9369', 'num_tokens': '3.596e+07', 'mean_token_accuracy': '0.7596', 'epoch': '0.05898'}
{'loss': '1.132', 'grad_norm': '0.4565', 'learning_rate': '8.996e-05', 'entropy': '1.176', 'num_tokens': '3.597e+07', 'mean_token_accuracy': '0.7156', 'epoch': '0.05899'}
{'loss': '0.6938', 'grad_norm': '0.3705', 'learning_rate': '8.993e-05', 'entropy': '0.6708', 'num_tokens': '3.597e+07', 'mean_token_accuracy': '0.8249', 'epoch': '0.059'}
{'loss': '1.097', 'grad_norm': '0.3977', 'learning_rate': '8.99e-05', 'entropy': '1.093', 'num_tokens': '3.598e+07', 'mean_token_accuracy': '0.736', 'epoch': '0.05901'}
{'loss': '0.7379', 'grad_norm': '0.3199', 'learning_rate': '8.988e-05', 'entropy': '0.7305', 'num_tokens': '3.598e+07', 'mean_token_accuracy': '0.8083', 'epoch': '0.05902'}
{'loss': '1.007', 'grad_norm': '0.383', 'learning_rate': '8.985e-05', 'entropy': '0.9225', 'num_tokens': '3.599e+07', 'mean_token_accuracy': '0.7772', 'epoch': '0.05903'}
{'loss': '0.8927', 'grad_norm': '0.465', 'learning_rate': '8.982e-05', 'entropy': '0.9776', 'num_tokens': '3.599e+07', 'mean_token_accuracy': '0.7552', 'epoch': '0.05904'}
{'loss': '0.7688', 'grad_norm': '0.4775', 'learning_rate': '8.979e-05', 'entropy': '0.8329', 'num_tokens': '3.6e+07', 'mean_token_accuracy': '0.7906', 'epoch': '0.05905'}
{'loss': '0.891', 'grad_norm': '0.4317', 'learning_rate': '8.977e-05', 'entropy': '0.916', 'num_tokens': '3.6e+07', 'mean_token_accuracy': '0.7753', 'epoch': '0.05905'}
{'loss': '0.7018', 'grad_norm': '0.4276', 'learning_rate': '8.974e-05', 'entropy': '0.646', 'num_tokens': '3.6e+07', 'mean_token_accuracy': '0.8327', 'epoch': '0.05906'}
{'loss': '0.9221', 'grad_norm': '0.4108', 'learning_rate': '8.971e-05', 'entropy': '0.8404', 'num_tokens': '3.601e+07', 'mean_token_accuracy': '0.8064', 'epoch': '0.05907'}
{'loss': '0.7401', 'grad_norm': '0.4214', 'learning_rate': '8.968e-05', 'entropy': '0.7161', 'num_tokens': '3.601e+07', 'mean_token_accuracy': '0.8073', 'epoch': '0.05908'}
{'loss': '0.886', 'grad_norm': '0.484', 'learning_rate': '8.966e-05', 'entropy': '0.9108', 'num_tokens': '3.602e+07', 'mean_token_accuracy': '0.7579', 'epoch': '0.05909'}

 56%|█████▌    | 6678/12000 [3:56:54<3:22:30,  2.28s/it]
 56%|█████▌    | 6679/12000 [3:56:57<3:33:39,  2.41s/it]
                                                        

 56%|█████▌    | 6679/12000 [3:56:57<3:33:39,  2.41s/it]
 56%|█████▌    | 6680/12000 [3:57:01<4:15:43,  2.88s/it]
                                                        

 56%|█████▌    | 6680/12000 [3:57:01<4:15:43,  2.88s/it]
 56%|█████▌    | 6681/12000 [3:57:02<3:40:36,  2.49s/it]
                                                        

 56%|█████▌    | 6681/12000 [3:57:02<3:40:36,  2.49s/it]
 56%|█████▌    | 6682/12000 [3:57:04<3:13:48,  2.19s/it]
                                                        

 56%|█████▌    | 6682/12000 [3:57:04<3:13:48,  2.19s/it]
 56%|█████▌    | 6683/12000 [3:57:06<3:14:00,  2.19s/it]
                                                        

 56%|█████▌    | 6683/12000 [3:57:06<3:14:00,  2.19s/it]
 56%|█████▌    | 6684/12000 [3:57:08<3:07:18,  2.11s/it]
                                                        

 56%|█████▌    | 6684/12000 [3:57:08<3:07:18,  2.11s/it]
 56%|█████▌    | 6685/12000 [3:57:09<2:53:50,  1.96s/it]
                                                        

 56%|█████▌    | 6685/12000 [3:57:09<2:53:50,  1.96s/it]
 56%|█████▌    | 6686/12000 [3:57:12<3:15:52,  2.21s/it]
                                                        

 56%|█████▌    | 6686/12000 [3:57:12<3:15:52,  2.21s/it]
 56%|█████▌    | 6687/12000 [3:57:14<2:59:33,  2.03s/it]
                                                        

 56%|█████▌    | 6687/12000 [3:57:14<2:59:33,  2.03s/it]
 56%|█████▌    | 6688/12000 [3:57:15<2:42:42,  1.84s/it]
                                                        

 56%|█████▌    | 6688/12000 [3:57:15<2:42:42,  1.84s/it]
 56%|█████▌    | 6689/12000 [3:57:17<2:35:27,  1.76s/it]
                                                        

 56%|█████▌    | 6689/12000 [3:57:17<2:35:27,  1.76s/it]
 56%|█████▌    | 6690/12000 [3:57:18<2:25:25,  1.64s/it]
                                                        

 56%|█████▌    | 6690/12000 [3:57:18<2:25:25,  1.64s/it]
 56%|█████▌    | 6691/12000 [3:57:20<2:25:14,  1.64s/it]
                                                        

 56%|█████▌    | 6691/12000 [3:57:20<2:25:14,  1.64s/it]
 56%|█████▌    | 6692/12000 [3:57:22<2:44:13,  1.86s/it]
                                                        

 56%|█████▌    | 6692/12000 [3:57:22<2:44:13,  1.86s/it]
 56%|█████▌    | 6693/12000 [3:57:26<3:25:14,  2.32s/it]
                                                        

 56%|█████▌    | 6693/12000 [3:57:26<3:25:14,  2.32s/it]
 56%|█████▌    | 6694/12000 [3:57:27<3:01:30,  2.05s/it]
                                                        

 56%|█████▌    | 6694/12000 [3:57:27<3:01:30,  2.05s/it]
 56%|█████▌    | 6695/12000 [3:57:29<2:50:34,  1.93s/it]
                                                        

 56%|█████▌    | 6695/12000 [3:57:29<2:50:34,  1.93s/it]
 56%|█████▌    | 6696/12000 [3:57:31<3:13:41,  2.19s/it]
                                                        

 56%|█████▌    | 6696/12000 [3:57:31<3:13:41,  2.19s/it]
 56%|█████▌    | 6697/12000 [3:57:33<3:08:39,  2.13s/it]
                                                        

 56%|█████▌    | 6697/12000 [3:57:33<3:08:39,  2.13s/it]
 56%|█████▌    | 6698/12000 [3:57:35<2:52:37,  1.95s/it]
                                                        

 56%|█████▌    | 6698/12000 [3:57:35<2:52:37,  1.95s/it]
 56%|█████▌    | 6699/12000 [3:57:37<2:50:19,  1.93s/it]
                                                        

 56%|█████▌    | 6699/12000 [3:57:37<2:50:19,  1.93s/it]
 56%|█████▌    | 6700/12000 [3:57:43<4:50:44,  3.29s/it]
                                                        

 56%|█████▌    | 6700/12000 [3:57:43<4:50:44,  3.29s/it]
 56%|█████▌    | 6701/12000 [3:57:47<4:55:50,  3.35s/it]
                                                        

 56%|█████▌    | 6701/12000 [3:57:47<4:55:50,  3.35s/it]
 56%|█████▌    | 6702/12000 [3:57:49<4:25:55,  3.01s/it]
                                                        

 56%|█████▌    | 6702/12000 [3:57:49<4:25:55,  3.01s/it]
 56%|█████▌    | 6703/12000 [3:57:51<3:58:51,  2.71s/it]
                                                        

 56%|█████▌    | 6703/12000 [3:57:51<3:58:51,  2.71s/it]
 56%|█████▌    | 6704/12000 [3:57:52<3:23:21,  2.30s/it]
                                                        

 56%|█████▌    | 6704/12000 [3:57:52<3:23:21,  2.30s/it]
 56%|█████▌    | 6705/12000 [3:57:54<3:13:03,  2.19s/it]
                                                        

 56%|█████▌    | 6705/12000 [3:57:54<3:13:03,  2.19s/it]
 56%|█████▌    | 6706/12000 [3:57:59<4:18:05,  2.93s/it]
                                                        

 56%|█████▌    | 6706/12000 [3:57:59<4:18:05,  2.93s/it]
 56%|█████▌    | 6707/12000 [3:58:03<4:36:33,  3.13s/it]
                                                        

 56%|█████▌    | 6707/12000 [3:58:03<4:36:33,  3.13s/it]
 56%|█████▌    | 6708/12000 [3:58:04<4:02:15,  2.75s/it]
                                                        

 56%|█████▌    | 6708/12000 [3:58:04<4:02:15,  2.75s/it]
 56%|█████▌    | 6709/12000 [3:58:06<3:26:56,  2.35s/it]
                                                        

 56%|█████▌    | 6709/12000 [3:58:06<3:26:56,  2.35s/it]
 56%|█████▌    | 6710/12000 [3:58:08<3:11:07,  2.17s/it]
                                                        

 56%|█████▌    | 6710/12000 [3:58:08<3:11:07,  2.17s/it]
 56%|█████▌    | 6711/12000 [3:58:12<4:13:22,  2.87s/it]
                                                        

 56%|█████▌    | 6711/12000 [3:58:12<4:13:22,  2.87s/it]
 56%|█████▌    | 6712/12000 [3:58:14<3:44:03,  2.54s/it]
                                                        

 56%|█████▌    | 6712/12000 [3:58:14<3:44:03,  2.54s/it]
 56%|█████▌    | 6713/12000 [3:58:17<3:59:47,  2.72s/it]
                                                        

 56%|█████▌    | 6713/12000 [3:58:17<3:59:47,  2.72s/it]
 56%|█████▌    | 6714/12000 [3:58:19<3:31:36,  2.40s/it]
                                                        

 56%|█████▌    | 6714/12000 [3:58:19<3:31:36,  2.40s/it]
 56%|█████▌    | 6715/12000 [3:58:20<3:06:50,  2.12s/it]
                                                        

 56%|█████▌    | 6715/12000 [3:58:20<3:06:50,  2.12s/it]
 56%|█████▌    | 6716/12000 [3:58:22<2:55:40,  1.99s/it]
                                                        

 56%|█████▌    | 6716/12000 [3:58:22<2:55:40,  1.99s/it]
 56%|█████▌    | 6717/12000 [3:58:23<2:46:01,  1.89s/it]
                                                        

 56%|█████▌    | 6717/12000 [3:58:23<2:46:01,  1.89s/it]
 56%|█████▌    | 6718/12000 [3:58:25<2:35:21,  1.76s/it]
                                                        

 56%|█████▌    | 6718/12000 [3:58:25<2:35:21,  1.76s/it]
 56%|█████▌    | 6719/12000 [3:58:26<2:28:04,  1.68s/it]
                                                        

 56%|█████▌    | 6719/12000 [3:58:26<2:28:04,  1.68s/it]
 56%|█████▌    | 6720/12000 [3:58:28<2:27:58,  1.68s/it]
                                                        

 56%|█████▌    | 6720/12000 [3:58:28<2:27:58,  1.68s/it]
 56%|█████▌    | 6721/12000 [3:58:30<2:27:19,  1.67s/it]
                                                        

 56%|█████▌    | 6721/12000 [3:58:30<2:27:19,  1.67s/it]
 56%|█████▌    | 6722/12000 [3:58:33<3:12:57,  2.19s/it]
                                                        

 56%|█████▌    | 6722/12000 [3:58:33<3:12:57,  2.19s/it]
 56%|█████▌    | 6723/12000 [3:58:34<2:49:52,  1.93s/it]
                                                        

 56%|█████▌    | 6723/12000 [3:58:34<2:49:52,  1.93s/it]
 56%|█████▌    | 6724/12000 [3:58:39<4:08:24,  2.82s/it]
                                                        

 56%|█████▌    | 6724/12000 [3:58:39<4:08:24,  2.82s/it]
 56%|█████▌    | 6725/12000 [3:58:41<3:37:42,  2.48s/it]
                                                        
{'loss': '1.018', 'grad_norm': '0.3988', 'learning_rate': '8.963e-05', 'entropy': '0.9669', 'num_tokens': '3.602e+07', 'mean_token_accuracy': '0.7531', 'epoch': '0.0591'}
{'loss': '0.8058', 'grad_norm': '0.3687', 'learning_rate': '8.96e-05', 'entropy': '0.9494', 'num_tokens': '3.603e+07', 'mean_token_accuracy': '0.7624', 'epoch': '0.05911'}
{'loss': '1.237', 'grad_norm': '0.36', 'learning_rate': '8.957e-05', 'entropy': '1.187', 'num_tokens': '3.604e+07', 'mean_token_accuracy': '0.7167', 'epoch': '0.05912'}
{'loss': '0.9231', 'grad_norm': '0.5327', 'learning_rate': '8.955e-05', 'entropy': '0.9135', 'num_tokens': '3.604e+07', 'mean_token_accuracy': '0.748', 'epoch': '0.05913'}
{'loss': '0.6592', 'grad_norm': '0.3424', 'learning_rate': '8.952e-05', 'entropy': '0.7215', 'num_tokens': '3.605e+07', 'mean_token_accuracy': '0.8159', 'epoch': '0.05913'}
{'loss': '0.7763', 'grad_norm': '0.3558', 'learning_rate': '8.949e-05', 'entropy': '0.8267', 'num_tokens': '3.605e+07', 'mean_token_accuracy': '0.7982', 'epoch': '0.05914'}
{'loss': '1.54', 'grad_norm': '0.6053', 'learning_rate': '8.946e-05', 'entropy': '1.267', 'num_tokens': '3.606e+07', 'mean_token_accuracy': '0.7024', 'epoch': '0.05915'}
{'loss': '0.7554', 'grad_norm': '0.4693', 'learning_rate': '8.944e-05', 'entropy': '0.8362', 'num_tokens': '3.606e+07', 'mean_token_accuracy': '0.7901', 'epoch': '0.05916'}
{'loss': '0.7148', 'grad_norm': '0.3544', 'learning_rate': '8.941e-05', 'entropy': '0.6876', 'num_tokens': '3.607e+07', 'mean_token_accuracy': '0.8277', 'epoch': '0.05917'}
{'loss': '0.733', 'grad_norm': '0.3858', 'learning_rate': '8.938e-05', 'entropy': '0.8227', 'num_tokens': '3.608e+07', 'mean_token_accuracy': '0.7944', 'epoch': '0.05918'}
{'loss': '0.7301', 'grad_norm': '0.3383', 'learning_rate': '8.936e-05', 'entropy': '0.7255', 'num_tokens': '3.608e+07', 'mean_token_accuracy': '0.8165', 'epoch': '0.05919'}
{'loss': '1.026', 'grad_norm': '0.5203', 'learning_rate': '8.933e-05', 'entropy': '1.165', 'num_tokens': '3.608e+07', 'mean_token_accuracy': '0.7132', 'epoch': '0.0592'}
{'loss': '1.017', 'grad_norm': '0.48', 'learning_rate': '8.93e-05', 'entropy': '1.018', 'num_tokens': '3.609e+07', 'mean_token_accuracy': '0.7478', 'epoch': '0.05921'}
{'loss': '0.6582', 'grad_norm': '0.3748', 'learning_rate': '8.927e-05', 'entropy': '0.692', 'num_tokens': '3.609e+07', 'mean_token_accuracy': '0.8329', 'epoch': '0.05921'}
{'loss': '0.8652', 'grad_norm': '0.3351', 'learning_rate': '8.925e-05', 'entropy': '0.8295', 'num_tokens': '3.61e+07', 'mean_token_accuracy': '0.7994', 'epoch': '0.05922'}
{'loss': '0.7855', 'grad_norm': '0.3284', 'learning_rate': '8.922e-05', 'entropy': '0.756', 'num_tokens': '3.611e+07', 'mean_token_accuracy': '0.8092', 'epoch': '0.05923'}
{'loss': '0.6141', 'grad_norm': '0.3247', 'learning_rate': '8.919e-05', 'entropy': '0.6161', 'num_tokens': '3.611e+07', 'mean_token_accuracy': '0.8423', 'epoch': '0.05924'}
{'loss': '0.9546', 'grad_norm': '0.347', 'learning_rate': '8.916e-05', 'entropy': '1.011', 'num_tokens': '3.612e+07', 'mean_token_accuracy': '0.7448', 'epoch': '0.05925'}
{'loss': '0.6622', 'grad_norm': '0.4116', 'learning_rate': '8.914e-05', 'entropy': '0.6962', 'num_tokens': '3.612e+07', 'mean_token_accuracy': '0.8171', 'epoch': '0.05926'}
{'loss': '0.9546', 'grad_norm': '0.455', 'learning_rate': '8.911e-05', 'entropy': '0.9522', 'num_tokens': '3.613e+07', 'mean_token_accuracy': '0.764', 'epoch': '0.05927'}
{'loss': '0.6836', 'grad_norm': '0.3029', 'learning_rate': '8.908e-05', 'entropy': '0.7503', 'num_tokens': '3.613e+07', 'mean_token_accuracy': '0.8134', 'epoch': '0.05928'}
{'loss': '0.8034', 'grad_norm': '0.3474', 'learning_rate': '8.905e-05', 'entropy': '0.8361', 'num_tokens': '3.614e+07', 'mean_token_accuracy': '0.7956', 'epoch': '0.05929'}
{'loss': '1.135', 'grad_norm': '0.3694', 'learning_rate': '8.903e-05', 'entropy': '0.8983', 'num_tokens': '3.615e+07', 'mean_token_accuracy': '0.7939', 'epoch': '0.05929'}
{'loss': '1.005', 'grad_norm': '0.3471', 'learning_rate': '8.9e-05', 'entropy': '0.9104', 'num_tokens': '3.615e+07', 'mean_token_accuracy': '0.7854', 'epoch': '0.0593'}
{'loss': '0.6113', 'grad_norm': '0.3005', 'learning_rate': '8.897e-05', 'entropy': '0.6857', 'num_tokens': '3.616e+07', 'mean_token_accuracy': '0.8334', 'epoch': '0.05931'}
{'loss': '1.05', 'grad_norm': '0.4463', 'learning_rate': '8.894e-05', 'entropy': '1.029', 'num_tokens': '3.617e+07', 'mean_token_accuracy': '0.7598', 'epoch': '0.05932'}
{'loss': '0.8974', 'grad_norm': '0.4023', 'learning_rate': '8.892e-05', 'entropy': '0.8992', 'num_tokens': '3.617e+07', 'mean_token_accuracy': '0.7819', 'epoch': '0.05933'}
{'loss': '0.77', 'grad_norm': '0.3473', 'learning_rate': '8.889e-05', 'entropy': '0.7667', 'num_tokens': '3.618e+07', 'mean_token_accuracy': '0.7982', 'epoch': '0.05934'}
{'loss': '2.927', 'grad_norm': '2.538', 'learning_rate': '8.886e-05', 'entropy': '1.857', 'num_tokens': '3.618e+07', 'mean_token_accuracy': '0.6103', 'epoch': '0.05935'}
{'loss': '0.7837', 'grad_norm': '0.3309', 'learning_rate': '8.883e-05', 'entropy': '0.8656', 'num_tokens': '3.619e+07', 'mean_token_accuracy': '0.7977', 'epoch': '0.05936'}
{'loss': '1.058', 'grad_norm': '0.4832', 'learning_rate': '8.881e-05', 'entropy': '1.078', 'num_tokens': '3.62e+07', 'mean_token_accuracy': '0.7398', 'epoch': '0.05936'}
{'loss': '0.5918', 'grad_norm': '0.3172', 'learning_rate': '8.878e-05', 'entropy': '0.6038', 'num_tokens': '3.62e+07', 'mean_token_accuracy': '0.845', 'epoch': '0.05937'}
{'loss': '0.9194', 'grad_norm': '0.4227', 'learning_rate': '8.875e-05', 'entropy': '0.9553', 'num_tokens': '3.621e+07', 'mean_token_accuracy': '0.7787', 'epoch': '0.05938'}
{'loss': '1.322', 'grad_norm': '0.3774', 'learning_rate': '8.873e-05', 'entropy': '1.288', 'num_tokens': '3.622e+07', 'mean_token_accuracy': '0.7081', 'epoch': '0.05939'}
{'loss': '0.9238', 'grad_norm': '0.47', 'learning_rate': '8.87e-05', 'entropy': '0.9139', 'num_tokens': '3.623e+07', 'mean_token_accuracy': '0.7849', 'epoch': '0.0594'}
{'loss': '0.6434', 'grad_norm': '0.3146', 'learning_rate': '8.867e-05', 'entropy': '0.7794', 'num_tokens': '3.623e+07', 'mean_token_accuracy': '0.7991', 'epoch': '0.05941'}
{'loss': '0.8474', 'grad_norm': '0.4349', 'learning_rate': '8.864e-05', 'entropy': '0.8325', 'num_tokens': '3.624e+07', 'mean_token_accuracy': '0.7861', 'epoch': '0.05942'}
{'loss': '0.9872', 'grad_norm': '0.5075', 'learning_rate': '8.862e-05', 'entropy': '1.075', 'num_tokens': '3.624e+07', 'mean_token_accuracy': '0.7492', 'epoch': '0.05943'}
{'loss': '0.826', 'grad_norm': '0.4444', 'learning_rate': '8.859e-05', 'entropy': '0.8217', 'num_tokens': '3.625e+07', 'mean_token_accuracy': '0.8041', 'epoch': '0.05944'}
{'loss': '0.7417', 'grad_norm': '0.3449', 'learning_rate': '8.856e-05', 'entropy': '0.689', 'num_tokens': '3.625e+07', 'mean_token_accuracy': '0.8265', 'epoch': '0.05944'}
{'loss': '0.7477', 'grad_norm': '0.3844', 'learning_rate': '8.853e-05', 'entropy': '0.7486', 'num_tokens': '3.626e+07', 'mean_token_accuracy': '0.8037', 'epoch': '0.05945'}
{'loss': '0.8194', 'grad_norm': '0.5181', 'learning_rate': '8.851e-05', 'entropy': '0.8138', 'num_tokens': '3.626e+07', 'mean_token_accuracy': '0.7835', 'epoch': '0.05946'}
{'loss': '0.5314', 'grad_norm': '0.3282', 'learning_rate': '8.848e-05', 'entropy': '0.5565', 'num_tokens': '3.627e+07', 'mean_token_accuracy': '0.8545', 'epoch': '0.05947'}
{'loss': '0.6474', 'grad_norm': '0.3693', 'learning_rate': '8.845e-05', 'entropy': '0.6633', 'num_tokens': '3.627e+07', 'mean_token_accuracy': '0.8259', 'epoch': '0.05948'}
{'loss': '0.6615', 'grad_norm': '0.3835', 'learning_rate': '8.842e-05', 'entropy': '0.6535', 'num_tokens': '3.628e+07', 'mean_token_accuracy': '0.8346', 'epoch': '0.05949'}
{'loss': '1.086', 'grad_norm': '0.4489', 'learning_rate': '8.84e-05', 'entropy': '1.04', 'num_tokens': '3.628e+07', 'mean_token_accuracy': '0.7389', 'epoch': '0.0595'}
{'loss': '1.091', 'grad_norm': '0.3306', 'learning_rate': '8.837e-05', 'entropy': '0.8056', 'num_tokens': '3.629e+07', 'mean_token_accuracy': '0.7994', 'epoch': '0.05951'}

 56%|█████▌    | 6725/12000 [3:58:41<3:37:42,  2.48s/it]
 56%|█████▌    | 6726/12000 [3:58:42<3:07:59,  2.14s/it]
                                                        

 56%|█████▌    | 6726/12000 [3:58:42<3:07:59,  2.14s/it]
 56%|█████▌    | 6727/12000 [3:58:45<3:23:12,  2.31s/it]
                                                        

 56%|█████▌    | 6727/12000 [3:58:45<3:23:12,  2.31s/it]
 56%|█████▌    | 6728/12000 [3:58:48<3:52:17,  2.64s/it]
                                                        

 56%|█████▌    | 6728/12000 [3:58:48<3:52:17,  2.64s/it]
 56%|█████▌    | 6729/12000 [3:58:50<3:20:36,  2.28s/it]
                                                        

 56%|█████▌    | 6729/12000 [3:58:50<3:20:36,  2.28s/it]
 56%|█████▌    | 6730/12000 [3:58:52<3:02:59,  2.08s/it]
                                                        

 56%|█████▌    | 6730/12000 [3:58:52<3:02:59,  2.08s/it]
 56%|█████▌    | 6731/12000 [3:58:53<2:46:54,  1.90s/it]
                                                        

 56%|█████▌    | 6731/12000 [3:58:53<2:46:54,  1.90s/it]
 56%|█████▌    | 6732/12000 [3:58:55<2:47:08,  1.90s/it]
                                                        

 56%|█████▌    | 6732/12000 [3:58:55<2:47:08,  1.90s/it]
 56%|█████▌    | 6733/12000 [3:58:56<2:36:38,  1.78s/it]
                                                        

 56%|█████▌    | 6733/12000 [3:58:56<2:36:38,  1.78s/it]
 56%|█████▌    | 6734/12000 [3:58:58<2:42:37,  1.85s/it]
                                                        

 56%|█████▌    | 6734/12000 [3:58:58<2:42:37,  1.85s/it]
 56%|█████▌    | 6735/12000 [3:59:00<2:31:52,  1.73s/it]
                                                        

 56%|█████▌    | 6735/12000 [3:59:00<2:31:52,  1.73s/it]
 56%|█████▌    | 6736/12000 [3:59:01<2:22:14,  1.62s/it]
                                                        

 56%|█████▌    | 6736/12000 [3:59:01<2:22:14,  1.62s/it]
 56%|█████▌    | 6737/12000 [3:59:04<2:55:09,  2.00s/it]
                                                        

 56%|█████▌    | 6737/12000 [3:59:04<2:55:09,  2.00s/it]
 56%|█████▌    | 6738/12000 [3:59:06<2:40:33,  1.83s/it]
                                                        

 56%|█████▌    | 6738/12000 [3:59:06<2:40:33,  1.83s/it]
 56%|█████▌    | 6739/12000 [3:59:08<2:48:17,  1.92s/it]
                                                        

 56%|█████▌    | 6739/12000 [3:59:08<2:48:17,  1.92s/it]
 56%|█████▌    | 6740/12000 [3:59:09<2:34:34,  1.76s/it]
                                                        

 56%|█████▌    | 6740/12000 [3:59:09<2:34:34,  1.76s/it]
 56%|█████▌    | 6741/12000 [3:59:11<2:48:09,  1.92s/it]
                                                        

 56%|█████▌    | 6741/12000 [3:59:11<2:48:09,  1.92s/it]
 56%|█████▌    | 6742/12000 [3:59:13<2:38:41,  1.81s/it]
                                                        

 56%|█████▌    | 6742/12000 [3:59:13<2:38:41,  1.81s/it]
 56%|█████▌    | 6743/12000 [3:59:15<2:50:30,  1.95s/it]
                                                        

 56%|█████▌    | 6743/12000 [3:59:15<2:50:30,  1.95s/it]
 56%|█████▌    | 6744/12000 [3:59:19<3:37:19,  2.48s/it]
                                                        

 56%|█████▌    | 6744/12000 [3:59:19<3:37:19,  2.48s/it]
 56%|█████▌    | 6745/12000 [3:59:20<3:11:26,  2.19s/it]
                                                        

 56%|█████▌    | 6745/12000 [3:59:20<3:11:26,  2.19s/it]
 56%|█████▌    | 6746/12000 [3:59:22<2:56:57,  2.02s/it]
                                                        

 56%|█████▌    | 6746/12000 [3:59:22<2:56:57,  2.02s/it]
 56%|█████▌    | 6747/12000 [3:59:24<2:47:06,  1.91s/it]
                                                        

 56%|█████▌    | 6747/12000 [3:59:24<2:47:06,  1.91s/it]
 56%|█████▌    | 6748/12000 [3:59:26<2:47:39,  1.92s/it]
                                                        

 56%|█████▌    | 6748/12000 [3:59:26<2:47:39,  1.92s/it]
 56%|█████▌    | 6749/12000 [3:59:27<2:30:16,  1.72s/it]
                                                        

 56%|█████▌    | 6749/12000 [3:59:27<2:30:16,  1.72s/it]
 56%|█████▋    | 6750/12000 [3:59:29<2:37:33,  1.80s/it]
                                                        

 56%|█████▋    | 6750/12000 [3:59:29<2:37:33,  1.80s/it]
 56%|█████▋    | 6751/12000 [3:59:32<3:05:29,  2.12s/it]
                                                        

 56%|█████▋    | 6751/12000 [3:59:32<3:05:29,  2.12s/it]
 56%|█████▋    | 6752/12000 [3:59:33<2:50:37,  1.95s/it]
                                                        

 56%|█████▋    | 6752/12000 [3:59:33<2:50:37,  1.95s/it]
 56%|█████▋    | 6753/12000 [3:59:35<2:41:01,  1.84s/it]
                                                        

 56%|█████▋    | 6753/12000 [3:59:35<2:41:01,  1.84s/it]
 56%|█████▋    | 6754/12000 [3:59:37<2:55:30,  2.01s/it]
                                                        

 56%|█████▋    | 6754/12000 [3:59:37<2:55:30,  2.01s/it]
 56%|█████▋    | 6755/12000 [3:59:39<2:56:57,  2.02s/it]
                                                        

 56%|█████▋    | 6755/12000 [3:59:39<2:56:57,  2.02s/it]
 56%|█████▋    | 6756/12000 [3:59:41<2:40:11,  1.83s/it]
                                                        

 56%|█████▋    | 6756/12000 [3:59:41<2:40:11,  1.83s/it]
 56%|█████▋    | 6757/12000 [3:59:43<2:53:09,  1.98s/it]
                                                        

 56%|█████▋    | 6757/12000 [3:59:43<2:53:09,  1.98s/it]
 56%|█████▋    | 6758/12000 [3:59:45<2:39:48,  1.83s/it]
                                                        

 56%|█████▋    | 6758/12000 [3:59:45<2:39:48,  1.83s/it]
 56%|█████▋    | 6759/12000 [3:59:46<2:40:44,  1.84s/it]
                                                        

 56%|█████▋    | 6759/12000 [3:59:46<2:40:44,  1.84s/it]
 56%|█████▋    | 6760/12000 [3:59:48<2:32:57,  1.75s/it]
                                                        

 56%|█████▋    | 6760/12000 [3:59:48<2:32:57,  1.75s/it]
 56%|█████▋    | 6761/12000 [3:59:49<2:22:00,  1.63s/it]
                                                        

 56%|█████▋    | 6761/12000 [3:59:49<2:22:00,  1.63s/it]
 56%|█████▋    | 6762/12000 [3:59:51<2:23:39,  1.65s/it]
                                                        

 56%|█████▋    | 6762/12000 [3:59:51<2:23:39,  1.65s/it]
 56%|█████▋    | 6763/12000 [3:59:52<2:15:36,  1.55s/it]
                                                        

 56%|█████▋    | 6763/12000 [3:59:52<2:15:36,  1.55s/it]
 56%|█████▋    | 6764/12000 [3:59:55<2:53:25,  1.99s/it]
                                                        

 56%|█████▋    | 6764/12000 [3:59:55<2:53:25,  1.99s/it]
 56%|█████▋    | 6765/12000 [3:59:57<2:49:44,  1.95s/it]
                                                        

 56%|█████▋    | 6765/12000 [3:59:57<2:49:44,  1.95s/it]
 56%|█████▋    | 6766/12000 [4:00:01<3:32:52,  2.44s/it]
                                                        

 56%|█████▋    | 6766/12000 [4:00:01<3:32:52,  2.44s/it]
 56%|█████▋    | 6767/12000 [4:00:02<3:05:48,  2.13s/it]
                                                        

 56%|█████▋    | 6767/12000 [4:00:02<3:05:48,  2.13s/it]
 56%|█████▋    | 6768/12000 [4:00:04<2:45:55,  1.90s/it]
                                                        

 56%|█████▋    | 6768/12000 [4:00:04<2:45:55,  1.90s/it]
 56%|█████▋    | 6769/12000 [4:00:05<2:37:16,  1.80s/it]
                                                        

 56%|█████▋    | 6769/12000 [4:00:05<2:37:16,  1.80s/it]
 56%|█████▋    | 6770/12000 [4:00:06<2:23:01,  1.64s/it]
                                                        

 56%|█████▋    | 6770/12000 [4:00:06<2:23:01,  1.64s/it]
 56%|█████▋    | 6771/12000 [4:00:09<2:35:42,  1.79s/it]
                                                        

 56%|█████▋    | 6771/12000 [4:00:09<2:35:42,  1.79s/it]
 56%|█████▋    | 6772/12000 [4:00:10<2:35:45,  1.79s/it]
                                                        
{'loss': '0.6744', 'grad_norm': '0.3785', 'learning_rate': '8.834e-05', 'entropy': '0.7698', 'num_tokens': '3.629e+07', 'mean_token_accuracy': '0.8138', 'epoch': '0.05952'}
{'loss': '0.7185', 'grad_norm': '0.4651', 'learning_rate': '8.831e-05', 'entropy': '0.7548', 'num_tokens': '3.63e+07', 'mean_token_accuracy': '0.8255', 'epoch': '0.05952'}
{'loss': '0.8606', 'grad_norm': '0.4095', 'learning_rate': '8.829e-05', 'entropy': '0.8057', 'num_tokens': '3.63e+07', 'mean_token_accuracy': '0.7987', 'epoch': '0.05953'}
{'loss': '0.8396', 'grad_norm': '0.3826', 'learning_rate': '8.826e-05', 'entropy': '0.883', 'num_tokens': '3.631e+07', 'mean_token_accuracy': '0.7797', 'epoch': '0.05954'}
{'loss': '0.9309', 'grad_norm': '0.3841', 'learning_rate': '8.823e-05', 'entropy': '0.8896', 'num_tokens': '3.632e+07', 'mean_token_accuracy': '0.7601', 'epoch': '0.05955'}
{'loss': '0.519', 'grad_norm': '0.3747', 'learning_rate': '8.821e-05', 'entropy': '0.5478', 'num_tokens': '3.632e+07', 'mean_token_accuracy': '0.8551', 'epoch': '0.05956'}
{'loss': '0.5912', 'grad_norm': '0.3829', 'learning_rate': '8.818e-05', 'entropy': '0.6099', 'num_tokens': '3.633e+07', 'mean_token_accuracy': '0.8473', 'epoch': '0.05957'}
{'loss': '0.8466', 'grad_norm': '0.4494', 'learning_rate': '8.815e-05', 'entropy': '0.8301', 'num_tokens': '3.633e+07', 'mean_token_accuracy': '0.8002', 'epoch': '0.05958'}
{'loss': '0.854', 'grad_norm': '0.373', 'learning_rate': '8.812e-05', 'entropy': '0.7689', 'num_tokens': '3.634e+07', 'mean_token_accuracy': '0.8088', 'epoch': '0.05959'}
{'loss': '0.9634', 'grad_norm': '0.3927', 'learning_rate': '8.81e-05', 'entropy': '0.9135', 'num_tokens': '3.634e+07', 'mean_token_accuracy': '0.7783', 'epoch': '0.05959'}
{'loss': '0.6285', 'grad_norm': '0.3276', 'learning_rate': '8.807e-05', 'entropy': '0.6483', 'num_tokens': '3.635e+07', 'mean_token_accuracy': '0.8269', 'epoch': '0.0596'}
{'loss': '0.9041', 'grad_norm': '0.4375', 'learning_rate': '8.804e-05', 'entropy': '0.962', 'num_tokens': '3.635e+07', 'mean_token_accuracy': '0.7695', 'epoch': '0.05961'}
{'loss': '0.6198', 'grad_norm': '0.3654', 'learning_rate': '8.801e-05', 'entropy': '0.6281', 'num_tokens': '3.636e+07', 'mean_token_accuracy': '0.8318', 'epoch': '0.05962'}
{'loss': '0.7025', 'grad_norm': '0.3769', 'learning_rate': '8.799e-05', 'entropy': '0.7037', 'num_tokens': '3.636e+07', 'mean_token_accuracy': '0.8051', 'epoch': '0.05963'}
{'loss': '0.8401', 'grad_norm': '0.4661', 'learning_rate': '8.796e-05', 'entropy': '0.7923', 'num_tokens': '3.637e+07', 'mean_token_accuracy': '0.7913', 'epoch': '0.05964'}
{'loss': '0.5297', 'grad_norm': '0.3588', 'learning_rate': '8.793e-05', 'entropy': '0.5554', 'num_tokens': '3.637e+07', 'mean_token_accuracy': '0.855', 'epoch': '0.05965'}
{'loss': '0.5818', 'grad_norm': '0.3395', 'learning_rate': '8.79e-05', 'entropy': '0.6284', 'num_tokens': '3.638e+07', 'mean_token_accuracy': '0.8329', 'epoch': '0.05966'}
{'loss': '1.061', 'grad_norm': '0.4416', 'learning_rate': '8.788e-05', 'entropy': '0.9242', 'num_tokens': '3.638e+07', 'mean_token_accuracy': '0.7762', 'epoch': '0.05967'}
{'loss': '0.6676', 'grad_norm': '0.3182', 'learning_rate': '8.785e-05', 'entropy': '0.7156', 'num_tokens': '3.639e+07', 'mean_token_accuracy': '0.8196', 'epoch': '0.05967'}
{'loss': '0.9889', 'grad_norm': '0.4237', 'learning_rate': '8.782e-05', 'entropy': '0.7901', 'num_tokens': '3.64e+07', 'mean_token_accuracy': '0.7995', 'epoch': '0.05968'}
{'loss': '0.6275', 'grad_norm': '0.2886', 'learning_rate': '8.779e-05', 'entropy': '0.6722', 'num_tokens': '3.64e+07', 'mean_token_accuracy': '0.8258', 'epoch': '0.05969'}
{'loss': '0.5882', 'grad_norm': '0.3537', 'learning_rate': '8.777e-05', 'entropy': '0.6255', 'num_tokens': '3.641e+07', 'mean_token_accuracy': '0.8397', 'epoch': '0.0597'}
{'loss': '0.9229', 'grad_norm': '0.3557', 'learning_rate': '8.774e-05', 'entropy': '0.9611', 'num_tokens': '3.641e+07', 'mean_token_accuracy': '0.755', 'epoch': '0.05971'}
{'loss': '0.7375', 'grad_norm': '0.4017', 'learning_rate': '8.771e-05', 'entropy': '0.8087', 'num_tokens': '3.642e+07', 'mean_token_accuracy': '0.7985', 'epoch': '0.05972'}
{'loss': '0.8332', 'grad_norm': '0.4216', 'learning_rate': '8.769e-05', 'entropy': '0.7981', 'num_tokens': '3.642e+07', 'mean_token_accuracy': '0.8111', 'epoch': '0.05973'}
{'loss': '1.076', 'grad_norm': '0.4187', 'learning_rate': '8.766e-05', 'entropy': '0.9607', 'num_tokens': '3.643e+07', 'mean_token_accuracy': '0.7682', 'epoch': '0.05974'}
{'loss': '0.793', 'grad_norm': '0.505', 'learning_rate': '8.763e-05', 'entropy': '0.7934', 'num_tokens': '3.643e+07', 'mean_token_accuracy': '0.7895', 'epoch': '0.05975'}
{'loss': '0.7586', 'grad_norm': '0.3922', 'learning_rate': '8.76e-05', 'entropy': '0.7559', 'num_tokens': '3.644e+07', 'mean_token_accuracy': '0.8132', 'epoch': '0.05975'}
{'loss': '0.6083', 'grad_norm': '0.2919', 'learning_rate': '8.758e-05', 'entropy': '0.6097', 'num_tokens': '3.644e+07', 'mean_token_accuracy': '0.8367', 'epoch': '0.05976'}
{'loss': '0.6727', 'grad_norm': '0.3365', 'learning_rate': '8.755e-05', 'entropy': '0.8201', 'num_tokens': '3.645e+07', 'mean_token_accuracy': '0.7935', 'epoch': '0.05977'}
{'loss': '0.7259', 'grad_norm': '0.3767', 'learning_rate': '8.752e-05', 'entropy': '0.981', 'num_tokens': '3.645e+07', 'mean_token_accuracy': '0.7637', 'epoch': '0.05978'}
{'loss': '0.8625', 'grad_norm': '0.4516', 'learning_rate': '8.749e-05', 'entropy': '0.7751', 'num_tokens': '3.646e+07', 'mean_token_accuracy': '0.812', 'epoch': '0.05979'}
{'loss': '1.038', 'grad_norm': '0.4645', 'learning_rate': '8.747e-05', 'entropy': '0.9438', 'num_tokens': '3.647e+07', 'mean_token_accuracy': '0.768', 'epoch': '0.0598'}
{'loss': '0.6612', 'grad_norm': '0.3651', 'learning_rate': '8.744e-05', 'entropy': '0.6724', 'num_tokens': '3.647e+07', 'mean_token_accuracy': '0.8216', 'epoch': '0.05981'}
{'loss': '1.067', 'grad_norm': '0.3665', 'learning_rate': '8.741e-05', 'entropy': '0.9359', 'num_tokens': '3.648e+07', 'mean_token_accuracy': '0.7659', 'epoch': '0.05982'}
{'loss': '0.6644', 'grad_norm': '0.3574', 'learning_rate': '8.738e-05', 'entropy': '0.7175', 'num_tokens': '3.648e+07', 'mean_token_accuracy': '0.8109', 'epoch': '0.05982'}
{'loss': '0.9384', 'grad_norm': '0.3655', 'learning_rate': '8.736e-05', 'entropy': '0.9763', 'num_tokens': '3.649e+07', 'mean_token_accuracy': '0.7462', 'epoch': '0.05983'}
{'loss': '0.6757', 'grad_norm': '0.2937', 'learning_rate': '8.733e-05', 'entropy': '0.7118', 'num_tokens': '3.649e+07', 'mean_token_accuracy': '0.8137', 'epoch': '0.05984'}
{'loss': '0.5149', 'grad_norm': '0.2924', 'learning_rate': '8.73e-05', 'entropy': '0.5477', 'num_tokens': '3.65e+07', 'mean_token_accuracy': '0.8513', 'epoch': '0.05985'}
{'loss': '0.6836', 'grad_norm': '0.3823', 'learning_rate': '8.728e-05', 'entropy': '0.7939', 'num_tokens': '3.65e+07', 'mean_token_accuracy': '0.8017', 'epoch': '0.05986'}
{'loss': '0.803', 'grad_norm': '0.4499', 'learning_rate': '8.725e-05', 'entropy': '0.8124', 'num_tokens': '3.651e+07', 'mean_token_accuracy': '0.7886', 'epoch': '0.05987'}
{'loss': '0.8508', 'grad_norm': '0.4005', 'learning_rate': '8.722e-05', 'entropy': '0.8913', 'num_tokens': '3.651e+07', 'mean_token_accuracy': '0.7757', 'epoch': '0.05988'}
{'loss': '0.6748', 'grad_norm': '0.2845', 'learning_rate': '8.719e-05', 'entropy': '0.7208', 'num_tokens': '3.652e+07', 'mean_token_accuracy': '0.8136', 'epoch': '0.05989'}
{'loss': '0.7307', 'grad_norm': '0.3826', 'learning_rate': '8.717e-05', 'entropy': '0.7662', 'num_tokens': '3.652e+07', 'mean_token_accuracy': '0.7994', 'epoch': '0.0599'}
{'loss': '0.7638', 'grad_norm': '0.3998', 'learning_rate': '8.714e-05', 'entropy': '0.831', 'num_tokens': '3.652e+07', 'mean_token_accuracy': '0.7954', 'epoch': '0.0599'}
{'loss': '0.8448', 'grad_norm': '0.3769', 'learning_rate': '8.711e-05', 'entropy': '0.8683', 'num_tokens': '3.653e+07', 'mean_token_accuracy': '0.7674', 'epoch': '0.05991'}
{'loss': '0.7141', 'grad_norm': '0.3841', 'learning_rate': '8.708e-05', 'entropy': '0.7627', 'num_tokens': '3.653e+07', 'mean_token_accuracy': '0.808', 'epoch': '0.05992'}

 56%|█████▋    | 6772/12000 [4:00:10<2:35:45,  1.79s/it]
 56%|█████▋    | 6773/12000 [4:00:12<2:27:08,  1.69s/it]
                                                        

 56%|█████▋    | 6773/12000 [4:00:12<2:27:08,  1.69s/it]
 56%|█████▋    | 6774/12000 [4:00:13<2:22:56,  1.64s/it]
                                                        

 56%|█████▋    | 6774/12000 [4:00:13<2:22:56,  1.64s/it]
 56%|█████▋    | 6775/12000 [4:00:15<2:19:36,  1.60s/it]
                                                        

 56%|█████▋    | 6775/12000 [4:00:15<2:19:36,  1.60s/it]
 56%|█████▋    | 6776/12000 [4:00:16<2:20:01,  1.61s/it]
                                                        

 56%|█████▋    | 6776/12000 [4:00:16<2:20:01,  1.61s/it]
 56%|█████▋    | 6777/12000 [4:00:18<2:21:49,  1.63s/it]
                                                        

 56%|█████▋    | 6777/12000 [4:00:18<2:21:49,  1.63s/it]
 56%|█████▋    | 6778/12000 [4:00:20<2:16:17,  1.57s/it]
                                                        

 56%|█████▋    | 6778/12000 [4:00:20<2:16:17,  1.57s/it]
 56%|█████▋    | 6779/12000 [4:00:21<2:26:20,  1.68s/it]
                                                        

 56%|█████▋    | 6779/12000 [4:00:21<2:26:20,  1.68s/it]
 56%|█████▋    | 6780/12000 [4:00:23<2:21:40,  1.63s/it]
                                                        

 56%|█████▋    | 6780/12000 [4:00:23<2:21:40,  1.63s/it]
 57%|█████▋    | 6781/12000 [4:00:25<2:42:34,  1.87s/it]
                                                        

 57%|█████▋    | 6781/12000 [4:00:25<2:42:34,  1.87s/it]
 57%|█████▋    | 6782/12000 [4:00:27<2:46:33,  1.92s/it]
                                                        

 57%|█████▋    | 6782/12000 [4:00:27<2:46:33,  1.92s/it]
 57%|█████▋    | 6783/12000 [4:00:30<3:14:34,  2.24s/it]
                                                        

 57%|█████▋    | 6783/12000 [4:00:30<3:14:34,  2.24s/it]
 57%|█████▋    | 6784/12000 [4:00:32<3:01:18,  2.09s/it]
                                                        

 57%|█████▋    | 6784/12000 [4:00:32<3:01:18,  2.09s/it]
 57%|█████▋    | 6785/12000 [4:00:34<2:45:05,  1.90s/it]
                                                        

 57%|█████▋    | 6785/12000 [4:00:34<2:45:05,  1.90s/it]
 57%|█████▋    | 6786/12000 [4:00:37<3:35:54,  2.48s/it]
                                                        

 57%|█████▋    | 6786/12000 [4:00:37<3:35:54,  2.48s/it]
 57%|█████▋    | 6787/12000 [4:00:39<3:15:31,  2.25s/it]
                                                        

 57%|█████▋    | 6787/12000 [4:00:39<3:15:31,  2.25s/it]
 57%|█████▋    | 6788/12000 [4:00:42<3:39:43,  2.53s/it]
                                                        

 57%|█████▋    | 6788/12000 [4:00:42<3:39:43,  2.53s/it]
 57%|█████▋    | 6789/12000 [4:00:44<3:13:26,  2.23s/it]
                                                        

 57%|█████▋    | 6789/12000 [4:00:44<3:13:26,  2.23s/it]
 57%|█████▋    | 6790/12000 [4:00:46<3:04:25,  2.12s/it]
                                                        

 57%|█████▋    | 6790/12000 [4:00:46<3:04:25,  2.12s/it]
 57%|█████▋    | 6791/12000 [4:00:50<3:54:45,  2.70s/it]
                                                        

 57%|█████▋    | 6791/12000 [4:00:50<3:54:45,  2.70s/it]
 57%|█████▋    | 6792/12000 [4:00:53<3:55:35,  2.71s/it]
                                                        

 57%|█████▋    | 6792/12000 [4:00:53<3:55:35,  2.71s/it]
 57%|█████▋    | 6793/12000 [4:00:54<3:21:30,  2.32s/it]
                                                        

 57%|█████▋    | 6793/12000 [4:00:54<3:21:30,  2.32s/it]
 57%|█████▋    | 6794/12000 [4:00:56<3:05:44,  2.14s/it]
                                                        

 57%|█████▋    | 6794/12000 [4:00:56<3:05:44,  2.14s/it]
 57%|█████▋    | 6795/12000 [4:00:57<2:43:08,  1.88s/it]
                                                        

 57%|█████▋    | 6795/12000 [4:00:57<2:43:08,  1.88s/it]
 57%|█████▋    | 6796/12000 [4:01:01<3:45:54,  2.60s/it]
                                                        

 57%|█████▋    | 6796/12000 [4:01:01<3:45:54,  2.60s/it]
 57%|█████▋    | 6797/12000 [4:01:03<3:12:35,  2.22s/it]
                                                        

 57%|█████▋    | 6797/12000 [4:01:03<3:12:35,  2.22s/it]
 57%|█████▋    | 6798/12000 [4:01:04<3:01:20,  2.09s/it]
                                                        

 57%|█████▋    | 6798/12000 [4:01:04<3:01:20,  2.09s/it]
 57%|█████▋    | 6799/12000 [4:01:09<3:54:54,  2.71s/it]
                                                        

 57%|█████▋    | 6799/12000 [4:01:09<3:54:54,  2.71s/it]
 57%|█████▋    | 6800/12000 [4:01:11<3:54:39,  2.71s/it]
                                                        

 57%|█████▋    | 6800/12000 [4:01:11<3:54:39,  2.71s/it]
 57%|█████▋    | 6801/12000 [4:01:14<3:58:26,  2.75s/it]
                                                        

 57%|█████▋    | 6801/12000 [4:01:14<3:58:26,  2.75s/it]
 57%|█████▋    | 6802/12000 [4:01:16<3:26:20,  2.38s/it]
                                                        

 57%|█████▋    | 6802/12000 [4:01:16<3:26:20,  2.38s/it]
 57%|█████▋    | 6803/12000 [4:01:17<3:08:26,  2.18s/it]
                                                        

 57%|█████▋    | 6803/12000 [4:01:17<3:08:26,  2.18s/it]
 57%|█████▋    | 6804/12000 [4:01:19<2:51:40,  1.98s/it]
                                                        

 57%|█████▋    | 6804/12000 [4:01:19<2:51:40,  1.98s/it]
 57%|█████▋    | 6805/12000 [4:01:21<2:44:27,  1.90s/it]
                                                        

 57%|█████▋    | 6805/12000 [4:01:21<2:44:27,  1.90s/it]
 57%|█████▋    | 6806/12000 [4:01:23<2:48:36,  1.95s/it]
                                                        

 57%|█████▋    | 6806/12000 [4:01:23<2:48:36,  1.95s/it]
 57%|█████▋    | 6807/12000 [4:01:24<2:44:52,  1.90s/it]
                                                        

 57%|█████▋    | 6807/12000 [4:01:24<2:44:52,  1.90s/it]
 57%|█████▋    | 6808/12000 [4:01:26<2:43:24,  1.89s/it]
                                                        

 57%|█████▋    | 6808/12000 [4:01:26<2:43:24,  1.89s/it]
 57%|█████▋    | 6809/12000 [4:01:30<3:25:39,  2.38s/it]
                                                        

 57%|█████▋    | 6809/12000 [4:01:30<3:25:39,  2.38s/it]
 57%|█████▋    | 6810/12000 [4:01:33<3:54:31,  2.71s/it]
                                                        

 57%|█████▋    | 6810/12000 [4:01:33<3:54:31,  2.71s/it]
 57%|█████▋    | 6811/12000 [4:01:35<3:25:06,  2.37s/it]
                                                        

 57%|█████▋    | 6811/12000 [4:01:35<3:25:06,  2.37s/it]
 57%|█████▋    | 6812/12000 [4:01:36<3:00:47,  2.09s/it]
                                                        

 57%|█████▋    | 6812/12000 [4:01:36<3:00:47,  2.09s/it]
 57%|█████▋    | 6813/12000 [4:01:38<2:41:27,  1.87s/it]
                                                        

 57%|█████▋    | 6813/12000 [4:01:38<2:41:27,  1.87s/it]
 57%|█████▋    | 6814/12000 [4:01:39<2:30:07,  1.74s/it]
                                                        

 57%|█████▋    | 6814/12000 [4:01:39<2:30:07,  1.74s/it]
 57%|█████▋    | 6815/12000 [4:01:40<2:20:26,  1.63s/it]
                                                        

 57%|█████▋    | 6815/12000 [4:01:40<2:20:26,  1.63s/it]
 57%|█████▋    | 6816/12000 [4:01:42<2:20:47,  1.63s/it]
                                                        

 57%|█████▋    | 6816/12000 [4:01:42<2:20:47,  1.63s/it]
 57%|█████▋    | 6817/12000 [4:01:45<2:45:34,  1.92s/it]
                                                        

 57%|█████▋    | 6817/12000 [4:01:45<2:45:34,  1.92s/it]
 57%|█████▋    | 6818/12000 [4:01:46<2:37:45,  1.83s/it]
                                                        

 57%|█████▋    | 6818/12000 [4:01:46<2:37:45,  1.83s/it]
 57%|█████▋    | 6819/12000 [4:01:48<2:28:47,  1.72s/it]
                                                        
{'loss': '0.7854', 'grad_norm': '0.4456', 'learning_rate': '8.706e-05', 'entropy': '0.8134', 'num_tokens': '3.654e+07', 'mean_token_accuracy': '0.7959', 'epoch': '0.05993'}
{'loss': '1.02', 'grad_norm': '0.3884', 'learning_rate': '8.703e-05', 'entropy': '1.031', 'num_tokens': '3.654e+07', 'mean_token_accuracy': '0.7487', 'epoch': '0.05994'}
{'loss': '0.7727', 'grad_norm': '0.4382', 'learning_rate': '8.7e-05', 'entropy': '0.7389', 'num_tokens': '3.655e+07', 'mean_token_accuracy': '0.8207', 'epoch': '0.05995'}
{'loss': '0.9798', 'grad_norm': '0.4295', 'learning_rate': '8.697e-05', 'entropy': '0.9147', 'num_tokens': '3.655e+07', 'mean_token_accuracy': '0.7803', 'epoch': '0.05996'}
{'loss': '0.9777', 'grad_norm': '0.4371', 'learning_rate': '8.695e-05', 'entropy': '1.018', 'num_tokens': '3.656e+07', 'mean_token_accuracy': '0.7473', 'epoch': '0.05997'}
{'loss': '0.8238', 'grad_norm': '0.3857', 'learning_rate': '8.692e-05', 'entropy': '0.8968', 'num_tokens': '3.656e+07', 'mean_token_accuracy': '0.7845', 'epoch': '0.05998'}
{'loss': '0.6895', 'grad_norm': '0.3551', 'learning_rate': '8.689e-05', 'entropy': '0.7293', 'num_tokens': '3.657e+07', 'mean_token_accuracy': '0.8165', 'epoch': '0.05998'}
{'loss': '0.73', 'grad_norm': '0.413', 'learning_rate': '8.687e-05', 'entropy': '0.6967', 'num_tokens': '3.657e+07', 'mean_token_accuracy': '0.8239', 'epoch': '0.05999'}
{'loss': '0.7278', 'grad_norm': '0.4327', 'learning_rate': '8.684e-05', 'entropy': '0.7964', 'num_tokens': '3.658e+07', 'mean_token_accuracy': '0.8021', 'epoch': '0.06'}
{'loss': '1.187', 'grad_norm': '0.43', 'learning_rate': '8.681e-05', 'entropy': '1.078', 'num_tokens': '3.658e+07', 'mean_token_accuracy': '0.7393', 'epoch': '0.06001'}
{'loss': '0.7158', 'grad_norm': '0.3717', 'learning_rate': '8.678e-05', 'entropy': '0.7046', 'num_tokens': '3.659e+07', 'mean_token_accuracy': '0.8169', 'epoch': '0.06002'}
{'loss': '0.8705', 'grad_norm': '0.4469', 'learning_rate': '8.676e-05', 'entropy': '0.7634', 'num_tokens': '3.66e+07', 'mean_token_accuracy': '0.8097', 'epoch': '0.06003'}
{'loss': '0.8966', 'grad_norm': '0.3778', 'learning_rate': '8.673e-05', 'entropy': '0.8866', 'num_tokens': '3.66e+07', 'mean_token_accuracy': '0.7869', 'epoch': '0.06004'}
{'loss': '0.5697', 'grad_norm': '0.321', 'learning_rate': '8.67e-05', 'entropy': '0.5818', 'num_tokens': '3.661e+07', 'mean_token_accuracy': '0.8408', 'epoch': '0.06005'}
{'loss': '0.75', 'grad_norm': '0.393', 'learning_rate': '8.667e-05', 'entropy': '0.8578', 'num_tokens': '3.662e+07', 'mean_token_accuracy': '0.7767', 'epoch': '0.06005'}
{'loss': '0.9554', 'grad_norm': '0.3953', 'learning_rate': '8.665e-05', 'entropy': '0.9495', 'num_tokens': '3.662e+07', 'mean_token_accuracy': '0.7895', 'epoch': '0.06006'}
{'loss': '0.722', 'grad_norm': '0.3592', 'learning_rate': '8.662e-05', 'entropy': '0.7809', 'num_tokens': '3.662e+07', 'mean_token_accuracy': '0.7896', 'epoch': '0.06007'}
{'loss': '0.9014', 'grad_norm': '0.4298', 'learning_rate': '8.659e-05', 'entropy': '0.8412', 'num_tokens': '3.663e+07', 'mean_token_accuracy': '0.785', 'epoch': '0.06008'}
{'loss': '0.826', 'grad_norm': '0.3864', 'learning_rate': '8.656e-05', 'entropy': '0.8307', 'num_tokens': '3.664e+07', 'mean_token_accuracy': '0.7939', 'epoch': '0.06009'}
{'loss': '0.8173', 'grad_norm': '0.3944', 'learning_rate': '8.654e-05', 'entropy': '0.7908', 'num_tokens': '3.664e+07', 'mean_token_accuracy': '0.8102', 'epoch': '0.0601'}
{'loss': '0.8688', 'grad_norm': '0.3382', 'learning_rate': '8.651e-05', 'entropy': '0.7986', 'num_tokens': '3.665e+07', 'mean_token_accuracy': '0.7932', 'epoch': '0.06011'}
{'loss': '0.8767', 'grad_norm': '0.3934', 'learning_rate': '8.648e-05', 'entropy': '0.8852', 'num_tokens': '3.666e+07', 'mean_token_accuracy': '0.7763', 'epoch': '0.06012'}
{'loss': '0.8261', 'grad_norm': '0.289', 'learning_rate': '8.646e-05', 'entropy': '0.8281', 'num_tokens': '3.666e+07', 'mean_token_accuracy': '0.7942', 'epoch': '0.06013'}
{'loss': '0.9392', 'grad_norm': '0.5089', 'learning_rate': '8.643e-05', 'entropy': '0.8804', 'num_tokens': '3.667e+07', 'mean_token_accuracy': '0.7686', 'epoch': '0.06013'}
{'loss': '0.8441', 'grad_norm': '0.3192', 'learning_rate': '8.64e-05', 'entropy': '0.8981', 'num_tokens': '3.667e+07', 'mean_token_accuracy': '0.7729', 'epoch': '0.06014'}
{'loss': '0.9668', 'grad_norm': '0.5576', 'learning_rate': '8.637e-05', 'entropy': '1.038', 'num_tokens': '3.668e+07', 'mean_token_accuracy': '0.7407', 'epoch': '0.06015'}
{'loss': '0.7563', 'grad_norm': '0.4401', 'learning_rate': '8.635e-05', 'entropy': '0.7556', 'num_tokens': '3.668e+07', 'mean_token_accuracy': '0.8044', 'epoch': '0.06016'}
{'loss': '0.8789', 'grad_norm': '0.2841', 'learning_rate': '8.632e-05', 'entropy': '0.9923', 'num_tokens': '3.669e+07', 'mean_token_accuracy': '0.7511', 'epoch': '0.06017'}
{'loss': '0.7472', 'grad_norm': '0.3948', 'learning_rate': '8.629e-05', 'entropy': '0.8265', 'num_tokens': '3.67e+07', 'mean_token_accuracy': '0.7901', 'epoch': '0.06018'}
{'loss': '0.7341', 'grad_norm': '0.3621', 'learning_rate': '8.626e-05', 'entropy': '0.738', 'num_tokens': '3.67e+07', 'mean_token_accuracy': '0.8126', 'epoch': '0.06019'}
{'loss': '0.8209', 'grad_norm': '0.3475', 'learning_rate': '8.624e-05', 'entropy': '0.8319', 'num_tokens': '3.671e+07', 'mean_token_accuracy': '0.7862', 'epoch': '0.0602'}
{'loss': '0.6762', 'grad_norm': '0.3882', 'learning_rate': '8.621e-05', 'entropy': '0.7003', 'num_tokens': '3.671e+07', 'mean_token_accuracy': '0.8275', 'epoch': '0.06021'}
{'loss': '0.8346', 'grad_norm': '0.4118', 'learning_rate': '8.618e-05', 'entropy': '0.9024', 'num_tokens': '3.672e+07', 'mean_token_accuracy': '0.7766', 'epoch': '0.06021'}
{'loss': '1.293', 'grad_norm': '0.543', 'learning_rate': '8.616e-05', 'entropy': '1.153', 'num_tokens': '3.672e+07', 'mean_token_accuracy': '0.7205', 'epoch': '0.06022'}
{'loss': '0.9549', 'grad_norm': '0.4321', 'learning_rate': '8.613e-05', 'entropy': '1.037', 'num_tokens': '3.673e+07', 'mean_token_accuracy': '0.7397', 'epoch': '0.06023'}
{'loss': '0.9259', 'grad_norm': '0.349', 'learning_rate': '8.61e-05', 'entropy': '0.8552', 'num_tokens': '3.673e+07', 'mean_token_accuracy': '0.7778', 'epoch': '0.06024'}
{'loss': '0.6656', 'grad_norm': '0.4178', 'learning_rate': '8.607e-05', 'entropy': '0.726', 'num_tokens': '3.674e+07', 'mean_token_accuracy': '0.8178', 'epoch': '0.06025'}
{'loss': '0.623', 'grad_norm': '0.2854', 'learning_rate': '8.605e-05', 'entropy': '0.6856', 'num_tokens': '3.675e+07', 'mean_token_accuracy': '0.8281', 'epoch': '0.06026'}
{'loss': '0.8998', 'grad_norm': '0.3993', 'learning_rate': '8.602e-05', 'entropy': '0.9432', 'num_tokens': '3.675e+07', 'mean_token_accuracy': '0.7757', 'epoch': '0.06027'}
{'loss': '0.6015', 'grad_norm': '0.3776', 'learning_rate': '8.599e-05', 'entropy': '0.6696', 'num_tokens': '3.675e+07', 'mean_token_accuracy': '0.8168', 'epoch': '0.06028'}
{'loss': '0.8773', 'grad_norm': '0.3872', 'learning_rate': '8.596e-05', 'entropy': '0.8747', 'num_tokens': '3.676e+07', 'mean_token_accuracy': '0.7832', 'epoch': '0.06029'}
{'loss': '1.121', 'grad_norm': '0.4387', 'learning_rate': '8.594e-05', 'entropy': '1.132', 'num_tokens': '3.676e+07', 'mean_token_accuracy': '0.73', 'epoch': '0.06029'}
{'loss': '0.7088', 'grad_norm': '0.3553', 'learning_rate': '8.591e-05', 'entropy': '0.7672', 'num_tokens': '3.677e+07', 'mean_token_accuracy': '0.8056', 'epoch': '0.0603'}
{'loss': '0.5781', 'grad_norm': '0.4059', 'learning_rate': '8.588e-05', 'entropy': '0.6859', 'num_tokens': '3.677e+07', 'mean_token_accuracy': '0.8187', 'epoch': '0.06031'}
{'loss': '0.8018', 'grad_norm': '0.4015', 'learning_rate': '8.586e-05', 'entropy': '0.7679', 'num_tokens': '3.677e+07', 'mean_token_accuracy': '0.8066', 'epoch': '0.06032'}
{'loss': '0.5494', 'grad_norm': '0.355', 'learning_rate': '8.583e-05', 'entropy': '0.5423', 'num_tokens': '3.678e+07', 'mean_token_accuracy': '0.865', 'epoch': '0.06033'}
{'loss': '0.9466', 'grad_norm': '0.4489', 'learning_rate': '8.58e-05', 'entropy': '0.9214', 'num_tokens': '3.679e+07', 'mean_token_accuracy': '0.7662', 'epoch': '0.06034'}

 57%|█████▋    | 6819/12000 [4:01:48<2:28:47,  1.72s/it]
 57%|█████▋    | 6820/12000 [4:01:49<2:22:51,  1.65s/it]
                                                        

 57%|█████▋    | 6820/12000 [4:01:49<2:22:51,  1.65s/it]
 57%|█████▋    | 6821/12000 [4:01:51<2:13:33,  1.55s/it]
                                                        

 57%|█████▋    | 6821/12000 [4:01:51<2:13:33,  1.55s/it]
 57%|█████▋    | 6822/12000 [4:01:52<2:16:44,  1.58s/it]
                                                        

 57%|█████▋    | 6822/12000 [4:01:52<2:16:44,  1.58s/it]
 57%|█████▋    | 6823/12000 [4:01:54<2:19:33,  1.62s/it]
                                                        

 57%|█████▋    | 6823/12000 [4:01:54<2:19:33,  1.62s/it]
 57%|█████▋    | 6824/12000 [4:01:56<2:31:37,  1.76s/it]
                                                        

 57%|█████▋    | 6824/12000 [4:01:56<2:31:37,  1.76s/it]
 57%|█████▋    | 6825/12000 [4:01:59<2:51:38,  1.99s/it]
                                                        

 57%|█████▋    | 6825/12000 [4:01:59<2:51:38,  1.99s/it]
 57%|█████▋    | 6826/12000 [4:02:00<2:45:12,  1.92s/it]
                                                        

 57%|█████▋    | 6826/12000 [4:02:00<2:45:12,  1.92s/it]
 57%|█████▋    | 6827/12000 [4:02:02<2:34:33,  1.79s/it]
                                                        

 57%|█████▋    | 6827/12000 [4:02:02<2:34:33,  1.79s/it]
 57%|█████▋    | 6828/12000 [4:02:03<2:30:33,  1.75s/it]
                                                        

 57%|█████▋    | 6828/12000 [4:02:03<2:30:33,  1.75s/it]
 57%|█████▋    | 6829/12000 [4:02:05<2:26:45,  1.70s/it]
                                                        

 57%|█████▋    | 6829/12000 [4:02:05<2:26:45,  1.70s/it]
 57%|█████▋    | 6830/12000 [4:02:07<2:39:07,  1.85s/it]
                                                        

 57%|█████▋    | 6830/12000 [4:02:07<2:39:07,  1.85s/it]
 57%|█████▋    | 6831/12000 [4:02:10<2:57:15,  2.06s/it]
                                                        

 57%|█████▋    | 6831/12000 [4:02:10<2:57:15,  2.06s/it]
 57%|█████▋    | 6832/12000 [4:02:13<3:36:18,  2.51s/it]
                                                        

 57%|█████▋    | 6832/12000 [4:02:13<3:36:18,  2.51s/it]
 57%|█████▋    | 6833/12000 [4:02:15<3:12:30,  2.24s/it]
                                                        

 57%|█████▋    | 6833/12000 [4:02:15<3:12:30,  2.24s/it]
 57%|█████▋    | 6834/12000 [4:02:17<2:59:08,  2.08s/it]
                                                        

 57%|█████▋    | 6834/12000 [4:02:17<2:59:08,  2.08s/it]
 57%|█████▋    | 6835/12000 [4:02:18<2:44:09,  1.91s/it]
                                                        

 57%|█████▋    | 6835/12000 [4:02:18<2:44:09,  1.91s/it]
 57%|█████▋    | 6836/12000 [4:02:23<3:56:22,  2.75s/it]
                                                        

 57%|█████▋    | 6836/12000 [4:02:23<3:56:22,  2.75s/it]
 57%|█████▋    | 6837/12000 [4:02:24<3:20:18,  2.33s/it]
                                                        

 57%|█████▋    | 6837/12000 [4:02:24<3:20:18,  2.33s/it]
 57%|█████▋    | 6838/12000 [4:02:26<2:55:35,  2.04s/it]
                                                        

 57%|█████▋    | 6838/12000 [4:02:26<2:55:35,  2.04s/it]
 57%|█████▋    | 6839/12000 [4:02:28<3:04:31,  2.15s/it]
                                                        

 57%|█████▋    | 6839/12000 [4:02:28<3:04:31,  2.15s/it]
 57%|█████▋    | 6840/12000 [4:02:30<3:07:18,  2.18s/it]
                                                        

 57%|█████▋    | 6840/12000 [4:02:30<3:07:18,  2.18s/it]
 57%|█████▋    | 6841/12000 [4:02:34<3:47:48,  2.65s/it]
                                                        

 57%|█████▋    | 6841/12000 [4:02:34<3:47:48,  2.65s/it]
 57%|█████▋    | 6842/12000 [4:02:36<3:25:19,  2.39s/it]
                                                        

 57%|█████▋    | 6842/12000 [4:02:36<3:25:19,  2.39s/it]
 57%|█████▋    | 6843/12000 [4:02:38<3:10:22,  2.22s/it]
                                                        

 57%|█████▋    | 6843/12000 [4:02:38<3:10:22,  2.22s/it]
 57%|█████▋    | 6844/12000 [4:02:39<2:54:28,  2.03s/it]
                                                        

 57%|█████▋    | 6844/12000 [4:02:39<2:54:28,  2.03s/it]
 57%|█████▋    | 6845/12000 [4:02:41<2:41:36,  1.88s/it]
                                                        

 57%|█████▋    | 6845/12000 [4:02:41<2:41:36,  1.88s/it]
 57%|█████▋    | 6846/12000 [4:02:42<2:33:41,  1.79s/it]
                                                        

 57%|█████▋    | 6846/12000 [4:02:42<2:33:41,  1.79s/it]
 57%|█████▋    | 6847/12000 [4:02:44<2:26:33,  1.71s/it]
                                                        

 57%|█████▋    | 6847/12000 [4:02:44<2:26:33,  1.71s/it]
 57%|█████▋    | 6848/12000 [4:02:45<2:16:59,  1.60s/it]
                                                        

 57%|█████▋    | 6848/12000 [4:02:45<2:16:59,  1.60s/it]
 57%|█████▋    | 6849/12000 [4:02:47<2:12:42,  1.55s/it]
                                                        

 57%|█████▋    | 6849/12000 [4:02:47<2:12:42,  1.55s/it]
 57%|█████▋    | 6850/12000 [4:02:48<2:24:07,  1.68s/it]
                                                        

 57%|█████▋    | 6850/12000 [4:02:48<2:24:07,  1.68s/it]
 57%|█████▋    | 6851/12000 [4:02:52<3:21:03,  2.34s/it]
                                                        

 57%|█████▋    | 6851/12000 [4:02:52<3:21:03,  2.34s/it]
 57%|█████▋    | 6852/12000 [4:02:56<3:51:33,  2.70s/it]
                                                        

 57%|█████▋    | 6852/12000 [4:02:56<3:51:33,  2.70s/it]
 57%|█████▋    | 6853/12000 [4:02:58<3:24:12,  2.38s/it]
                                                        

 57%|█████▋    | 6853/12000 [4:02:58<3:24:12,  2.38s/it]
 57%|█████▋    | 6854/12000 [4:03:01<3:49:32,  2.68s/it]
                                                        

 57%|█████▋    | 6854/12000 [4:03:01<3:49:32,  2.68s/it]
 57%|█████▋    | 6855/12000 [4:03:03<3:22:41,  2.36s/it]
                                                        

 57%|█████▋    | 6855/12000 [4:03:03<3:22:41,  2.36s/it]
 57%|█████▋    | 6856/12000 [4:03:05<3:17:46,  2.31s/it]
                                                        

 57%|█████▋    | 6856/12000 [4:03:05<3:17:46,  2.31s/it]
 57%|█████▋    | 6857/12000 [4:03:06<2:59:02,  2.09s/it]
                                                        

 57%|█████▋    | 6857/12000 [4:03:06<2:59:02,  2.09s/it]
 57%|█████▋    | 6858/12000 [4:03:08<2:44:47,  1.92s/it]
                                                        

 57%|█████▋    | 6858/12000 [4:03:08<2:44:47,  1.92s/it]
 57%|█████▋    | 6859/12000 [4:03:10<2:52:50,  2.02s/it]
                                                        

 57%|█████▋    | 6859/12000 [4:03:10<2:52:50,  2.02s/it]
 57%|█████▋    | 6860/12000 [4:03:14<3:40:56,  2.58s/it]
                                                        

 57%|█████▋    | 6860/12000 [4:03:14<3:40:56,  2.58s/it]
 57%|█████▋    | 6861/12000 [4:03:17<3:44:28,  2.62s/it]
                                                        

 57%|█████▋    | 6861/12000 [4:03:17<3:44:28,  2.62s/it]
 57%|█████▋    | 6862/12000 [4:03:18<3:19:14,  2.33s/it]
                                                        

 57%|█████▋    | 6862/12000 [4:03:18<3:19:14,  2.33s/it]
 57%|█████▋    | 6863/12000 [4:03:20<3:02:30,  2.13s/it]
                                                        

 57%|█████▋    | 6863/12000 [4:03:20<3:02:30,  2.13s/it]
 57%|█████▋    | 6864/12000 [4:03:23<3:19:52,  2.34s/it]
                                                        

 57%|█████▋    | 6864/12000 [4:03:23<3:19:52,  2.34s/it]
 57%|█████▋    | 6865/12000 [4:03:26<3:33:50,  2.50s/it]
                                                        

 57%|█████▋    | 6865/12000 [4:03:26<3:33:50,  2.50s/it]
 57%|█████▋    | 6866/12000 [4:03:27<3:09:51,  2.22s/it]
                                                        
{'loss': '1.024', 'grad_norm': '0.3879', 'learning_rate': '8.577e-05', 'entropy': '0.9622', 'num_tokens': '3.679e+07', 'mean_token_accuracy': '0.765', 'epoch': '0.06035'}
{'loss': '1.029', 'grad_norm': '0.4235', 'learning_rate': '8.575e-05', 'entropy': '1.064', 'num_tokens': '3.679e+07', 'mean_token_accuracy': '0.7421', 'epoch': '0.06036'}
{'loss': '0.5875', 'grad_norm': '0.3153', 'learning_rate': '8.572e-05', 'entropy': '0.575', 'num_tokens': '3.68e+07', 'mean_token_accuracy': '0.8492', 'epoch': '0.06036'}
{'loss': '0.8756', 'grad_norm': '0.4722', 'learning_rate': '8.569e-05', 'entropy': '0.9001', 'num_tokens': '3.68e+07', 'mean_token_accuracy': '0.7613', 'epoch': '0.06037'}
{'loss': '0.8244', 'grad_norm': '0.3486', 'learning_rate': '8.566e-05', 'entropy': '0.7845', 'num_tokens': '3.681e+07', 'mean_token_accuracy': '0.803', 'epoch': '0.06038'}
{'loss': '1.038', 'grad_norm': '0.5152', 'learning_rate': '8.564e-05', 'entropy': '0.9398', 'num_tokens': '3.681e+07', 'mean_token_accuracy': '0.7673', 'epoch': '0.06039'}
{'loss': '0.5252', 'grad_norm': '0.2688', 'learning_rate': '8.561e-05', 'entropy': '0.6161', 'num_tokens': '3.682e+07', 'mean_token_accuracy': '0.8506', 'epoch': '0.0604'}
{'loss': '0.9256', 'grad_norm': '0.388', 'learning_rate': '8.558e-05', 'entropy': '0.9113', 'num_tokens': '3.683e+07', 'mean_token_accuracy': '0.7774', 'epoch': '0.06041'}
{'loss': '0.7855', 'grad_norm': '0.387', 'learning_rate': '8.556e-05', 'entropy': '0.7455', 'num_tokens': '3.683e+07', 'mean_token_accuracy': '0.8164', 'epoch': '0.06042'}
{'loss': '0.9769', 'grad_norm': '0.4694', 'learning_rate': '8.553e-05', 'entropy': '0.9093', 'num_tokens': '3.684e+07', 'mean_token_accuracy': '0.7784', 'epoch': '0.06043'}
{'loss': '1.021', 'grad_norm': '0.5643', 'learning_rate': '8.55e-05', 'entropy': '1.023', 'num_tokens': '3.684e+07', 'mean_token_accuracy': '0.7582', 'epoch': '0.06044'}
{'loss': '0.954', 'grad_norm': '0.4349', 'learning_rate': '8.547e-05', 'entropy': '0.9353', 'num_tokens': '3.685e+07', 'mean_token_accuracy': '0.7626', 'epoch': '0.06044'}
{'loss': '0.7847', 'grad_norm': '0.3649', 'learning_rate': '8.545e-05', 'entropy': '0.7474', 'num_tokens': '3.685e+07', 'mean_token_accuracy': '0.805', 'epoch': '0.06045'}
{'loss': '0.7182', 'grad_norm': '0.3372', 'learning_rate': '8.542e-05', 'entropy': '0.7342', 'num_tokens': '3.686e+07', 'mean_token_accuracy': '0.8063', 'epoch': '0.06046'}
{'loss': '0.8017', 'grad_norm': '0.4197', 'learning_rate': '8.539e-05', 'entropy': '0.8072', 'num_tokens': '3.686e+07', 'mean_token_accuracy': '0.7957', 'epoch': '0.06047'}
{'loss': '0.5401', 'grad_norm': '0.3368', 'learning_rate': '8.536e-05', 'entropy': '0.609', 'num_tokens': '3.687e+07', 'mean_token_accuracy': '0.8471', 'epoch': '0.06048'}
{'loss': '0.8053', 'grad_norm': '0.4383', 'learning_rate': '8.534e-05', 'entropy': '0.8177', 'num_tokens': '3.687e+07', 'mean_token_accuracy': '0.7916', 'epoch': '0.06049'}
{'loss': '1.101', 'grad_norm': '0.3811', 'learning_rate': '8.531e-05', 'entropy': '0.8335', 'num_tokens': '3.688e+07', 'mean_token_accuracy': '0.8034', 'epoch': '0.0605'}
{'loss': '1.034', 'grad_norm': '0.4085', 'learning_rate': '8.528e-05', 'entropy': '1.09', 'num_tokens': '3.688e+07', 'mean_token_accuracy': '0.7453', 'epoch': '0.06051'}
{'loss': '0.8811', 'grad_norm': '0.4234', 'learning_rate': '8.526e-05', 'entropy': '0.9471', 'num_tokens': '3.689e+07', 'mean_token_accuracy': '0.774', 'epoch': '0.06052'}
{'loss': '0.9113', 'grad_norm': '0.3529', 'learning_rate': '8.523e-05', 'entropy': '0.8839', 'num_tokens': '3.689e+07', 'mean_token_accuracy': '0.7734', 'epoch': '0.06052'}
{'loss': '0.84', 'grad_norm': '0.3151', 'learning_rate': '8.52e-05', 'entropy': '0.8163', 'num_tokens': '3.69e+07', 'mean_token_accuracy': '0.804', 'epoch': '0.06053'}
{'loss': '0.8799', 'grad_norm': '0.3586', 'learning_rate': '8.517e-05', 'entropy': '0.8967', 'num_tokens': '3.691e+07', 'mean_token_accuracy': '0.7893', 'epoch': '0.06054'}
{'loss': '0.7425', 'grad_norm': '0.4127', 'learning_rate': '8.515e-05', 'entropy': '0.7216', 'num_tokens': '3.692e+07', 'mean_token_accuracy': '0.8138', 'epoch': '0.06055'}
{'loss': '0.6065', 'grad_norm': '0.3209', 'learning_rate': '8.512e-05', 'entropy': '0.6774', 'num_tokens': '3.692e+07', 'mean_token_accuracy': '0.8283', 'epoch': '0.06056'}
{'loss': '0.9111', 'grad_norm': '0.4072', 'learning_rate': '8.509e-05', 'entropy': '0.8675', 'num_tokens': '3.693e+07', 'mean_token_accuracy': '0.7711', 'epoch': '0.06057'}
{'loss': '0.7023', 'grad_norm': '0.4513', 'learning_rate': '8.506e-05', 'entropy': '0.7047', 'num_tokens': '3.693e+07', 'mean_token_accuracy': '0.813', 'epoch': '0.06058'}
{'loss': '0.585', 'grad_norm': '0.3358', 'learning_rate': '8.504e-05', 'entropy': '0.5911', 'num_tokens': '3.693e+07', 'mean_token_accuracy': '0.8502', 'epoch': '0.06059'}
{'loss': '0.589', 'grad_norm': '0.3643', 'learning_rate': '8.501e-05', 'entropy': '0.5547', 'num_tokens': '3.694e+07', 'mean_token_accuracy': '0.8492', 'epoch': '0.06059'}
{'loss': '0.6859', 'grad_norm': '0.3318', 'learning_rate': '8.498e-05', 'entropy': '0.6924', 'num_tokens': '3.694e+07', 'mean_token_accuracy': '0.8209', 'epoch': '0.0606'}
{'loss': '0.5504', 'grad_norm': '0.3418', 'learning_rate': '8.496e-05', 'entropy': '0.5346', 'num_tokens': '3.695e+07', 'mean_token_accuracy': '0.8573', 'epoch': '0.06061'}
{'loss': '0.9444', 'grad_norm': '0.3481', 'learning_rate': '8.493e-05', 'entropy': '0.9269', 'num_tokens': '3.695e+07', 'mean_token_accuracy': '0.7691', 'epoch': '0.06062'}
{'loss': '1.315', 'grad_norm': '0.4737', 'learning_rate': '8.49e-05', 'entropy': '1.414', 'num_tokens': '3.696e+07', 'mean_token_accuracy': '0.6631', 'epoch': '0.06063'}
{'loss': '0.8686', 'grad_norm': '0.3955', 'learning_rate': '8.487e-05', 'entropy': '0.8068', 'num_tokens': '3.697e+07', 'mean_token_accuracy': '0.8', 'epoch': '0.06064'}
{'loss': '1.051', 'grad_norm': '0.4454', 'learning_rate': '8.485e-05', 'entropy': '0.9962', 'num_tokens': '3.697e+07', 'mean_token_accuracy': '0.7496', 'epoch': '0.06065'}
{'loss': '0.7033', 'grad_norm': '0.3528', 'learning_rate': '8.482e-05', 'entropy': '0.7214', 'num_tokens': '3.698e+07', 'mean_token_accuracy': '0.8198', 'epoch': '0.06066'}
{'loss': '0.7658', 'grad_norm': '0.4373', 'learning_rate': '8.479e-05', 'entropy': '0.7847', 'num_tokens': '3.698e+07', 'mean_token_accuracy': '0.7965', 'epoch': '0.06067'}
{'loss': '0.6549', 'grad_norm': '0.2993', 'learning_rate': '8.476e-05', 'entropy': '0.6879', 'num_tokens': '3.699e+07', 'mean_token_accuracy': '0.8333', 'epoch': '0.06067'}
{'loss': '0.6934', 'grad_norm': '0.3591', 'learning_rate': '8.474e-05', 'entropy': '0.7282', 'num_tokens': '3.699e+07', 'mean_token_accuracy': '0.8072', 'epoch': '0.06068'}
{'loss': '1.185', 'grad_norm': '0.4909', 'learning_rate': '8.471e-05', 'entropy': '1.156', 'num_tokens': '3.7e+07', 'mean_token_accuracy': '0.7092', 'epoch': '0.06069'}
{'loss': '0.6947', 'grad_norm': '0.3772', 'learning_rate': '8.468e-05', 'entropy': '0.6505', 'num_tokens': '3.7e+07', 'mean_token_accuracy': '0.823', 'epoch': '0.0607'}
{'loss': '0.8101', 'grad_norm': '0.3261', 'learning_rate': '8.466e-05', 'entropy': '0.8264', 'num_tokens': '3.701e+07', 'mean_token_accuracy': '0.7957', 'epoch': '0.06071'}
{'loss': '1.078', 'grad_norm': '0.4196', 'learning_rate': '8.463e-05', 'entropy': '1.098', 'num_tokens': '3.702e+07', 'mean_token_accuracy': '0.7425', 'epoch': '0.06072'}
{'loss': '0.7284', 'grad_norm': '0.4059', 'learning_rate': '8.46e-05', 'entropy': '0.709', 'num_tokens': '3.702e+07', 'mean_token_accuracy': '0.8119', 'epoch': '0.06073'}
{'loss': '0.7004', 'grad_norm': '0.375', 'learning_rate': '8.457e-05', 'entropy': '0.7401', 'num_tokens': '3.703e+07', 'mean_token_accuracy': '0.8134', 'epoch': '0.06074'}
{'loss': '0.6833', 'grad_norm': '0.3125', 'learning_rate': '8.455e-05', 'entropy': '0.7401', 'num_tokens': '3.704e+07', 'mean_token_accuracy': '0.8033', 'epoch': '0.06075'}
{'loss': '0.5498', 'grad_norm': '0.2926', 'learning_rate': '8.452e-05', 'entropy': '0.5588', 'num_tokens': '3.704e+07', 'mean_token_accuracy': '0.8508', 'epoch': '0.06075'}

 57%|█████▋    | 6866/12000 [4:03:27<3:09:51,  2.22s/it]
 57%|█████▋    | 6867/12000 [4:03:29<3:06:04,  2.18s/it]
                                                        

 57%|█████▋    | 6867/12000 [4:03:29<3:06:04,  2.18s/it]
 57%|█████▋    | 6868/12000 [4:03:31<2:54:34,  2.04s/it]
                                                        

 57%|█████▋    | 6868/12000 [4:03:31<2:54:34,  2.04s/it]
 57%|█████▋    | 6869/12000 [4:03:33<2:45:39,  1.94s/it]
                                                        

 57%|█████▋    | 6869/12000 [4:03:33<2:45:39,  1.94s/it]
 57%|█████▋    | 6870/12000 [4:03:34<2:34:51,  1.81s/it]
                                                        

 57%|█████▋    | 6870/12000 [4:03:34<2:34:51,  1.81s/it]
 57%|█████▋    | 6871/12000 [4:03:36<2:26:29,  1.71s/it]
                                                        

 57%|█████▋    | 6871/12000 [4:03:36<2:26:29,  1.71s/it]
 57%|█████▋    | 6872/12000 [4:03:38<2:40:34,  1.88s/it]
                                                        

 57%|█████▋    | 6872/12000 [4:03:38<2:40:34,  1.88s/it]
 57%|█████▋    | 6873/12000 [4:03:40<2:33:06,  1.79s/it]
                                                        

 57%|█████▋    | 6873/12000 [4:03:40<2:33:06,  1.79s/it]
 57%|█████▋    | 6874/12000 [4:03:41<2:28:46,  1.74s/it]
                                                        

 57%|█████▋    | 6874/12000 [4:03:41<2:28:46,  1.74s/it]
 57%|█████▋    | 6875/12000 [4:03:44<2:50:24,  1.99s/it]
                                                        

 57%|█████▋    | 6875/12000 [4:03:44<2:50:24,  1.99s/it]
 57%|█████▋    | 6876/12000 [4:03:48<3:49:26,  2.69s/it]
                                                        

 57%|█████▋    | 6876/12000 [4:03:48<3:49:26,  2.69s/it]
 57%|█████▋    | 6877/12000 [4:03:50<3:27:36,  2.43s/it]
                                                        

 57%|█████▋    | 6877/12000 [4:03:50<3:27:36,  2.43s/it]
 57%|█████▋    | 6878/12000 [4:03:53<3:33:41,  2.50s/it]
                                                        

 57%|█████▋    | 6878/12000 [4:03:53<3:33:41,  2.50s/it]
 57%|█████▋    | 6879/12000 [4:03:54<3:16:17,  2.30s/it]
                                                        

 57%|█████▋    | 6879/12000 [4:03:54<3:16:17,  2.30s/it]
 57%|█████▋    | 6880/12000 [4:03:57<3:11:15,  2.24s/it]
                                                        

 57%|█████▋    | 6880/12000 [4:03:57<3:11:15,  2.24s/it]
 57%|█████▋    | 6881/12000 [4:03:59<3:16:17,  2.30s/it]
                                                        

 57%|█████▋    | 6881/12000 [4:03:59<3:16:17,  2.30s/it]
 57%|█████▋    | 6882/12000 [4:04:01<3:05:28,  2.17s/it]
                                                        

 57%|█████▋    | 6882/12000 [4:04:01<3:05:28,  2.17s/it]
 57%|█████▋    | 6883/12000 [4:04:02<2:49:14,  1.98s/it]
                                                        

 57%|█████▋    | 6883/12000 [4:04:02<2:49:14,  1.98s/it]
 57%|█████▋    | 6884/12000 [4:04:04<2:43:13,  1.91s/it]
                                                        

 57%|█████▋    | 6884/12000 [4:04:04<2:43:13,  1.91s/it]
 57%|█████▋    | 6885/12000 [4:04:08<3:28:47,  2.45s/it]
                                                        

 57%|█████▋    | 6885/12000 [4:04:08<3:28:47,  2.45s/it]
 57%|█████▋    | 6886/12000 [4:04:10<3:15:36,  2.30s/it]
                                                        

 57%|█████▋    | 6886/12000 [4:04:10<3:15:36,  2.30s/it]
 57%|█████▋    | 6887/12000 [4:04:13<3:39:42,  2.58s/it]
                                                        

 57%|█████▋    | 6887/12000 [4:04:13<3:39:42,  2.58s/it]
 57%|█████▋    | 6888/12000 [4:04:16<3:58:16,  2.80s/it]
                                                        

 57%|█████▋    | 6888/12000 [4:04:16<3:58:16,  2.80s/it]
 57%|█████▋    | 6889/12000 [4:04:18<3:30:30,  2.47s/it]
                                                        

 57%|█████▋    | 6889/12000 [4:04:18<3:30:30,  2.47s/it]
 57%|█████▋    | 6890/12000 [4:04:20<3:04:48,  2.17s/it]
                                                        

 57%|█████▋    | 6890/12000 [4:04:20<3:04:48,  2.17s/it]
 57%|█████▋    | 6891/12000 [4:04:21<2:48:22,  1.98s/it]
                                                        

 57%|█████▋    | 6891/12000 [4:04:21<2:48:22,  1.98s/it]
 57%|█████▋    | 6892/12000 [4:04:23<2:37:27,  1.85s/it]
                                                        

 57%|█████▋    | 6892/12000 [4:04:23<2:37:27,  1.85s/it]
 57%|█████▋    | 6893/12000 [4:04:24<2:29:43,  1.76s/it]
                                                        

 57%|█████▋    | 6893/12000 [4:04:24<2:29:43,  1.76s/it]
 57%|█████▋    | 6894/12000 [4:04:26<2:40:50,  1.89s/it]
                                                        

 57%|█████▋    | 6894/12000 [4:04:26<2:40:50,  1.89s/it]
 57%|█████▋    | 6895/12000 [4:04:28<2:31:43,  1.78s/it]
                                                        

 57%|█████▋    | 6895/12000 [4:04:28<2:31:43,  1.78s/it]
 57%|█████▋    | 6896/12000 [4:04:29<2:25:16,  1.71s/it]
                                                        

 57%|█████▋    | 6896/12000 [4:04:29<2:25:16,  1.71s/it]
 57%|█████▋    | 6897/12000 [4:04:31<2:24:03,  1.69s/it]
                                                        

 57%|█████▋    | 6897/12000 [4:04:31<2:24:03,  1.69s/it]
 57%|█████▋    | 6898/12000 [4:04:35<3:15:04,  2.29s/it]
                                                        

 57%|█████▋    | 6898/12000 [4:04:35<3:15:04,  2.29s/it]
 57%|█████▋    | 6899/12000 [4:04:36<2:55:41,  2.07s/it]
                                                        

 57%|█████▋    | 6899/12000 [4:04:36<2:55:41,  2.07s/it]
 57%|█████▊    | 6900/12000 [4:04:38<2:44:34,  1.94s/it]
                                                        

 57%|█████▊    | 6900/12000 [4:04:38<2:44:34,  1.94s/it]
 58%|█████▊    | 6901/12000 [4:04:41<3:09:23,  2.23s/it]
                                                        

 58%|█████▊    | 6901/12000 [4:04:41<3:09:23,  2.23s/it]
 58%|█████▊    | 6902/12000 [4:04:43<2:54:30,  2.05s/it]
                                                        

 58%|█████▊    | 6902/12000 [4:04:43<2:54:30,  2.05s/it]
 58%|█████▊    | 6903/12000 [4:04:44<2:47:37,  1.97s/it]
                                                        

 58%|█████▊    | 6903/12000 [4:04:44<2:47:37,  1.97s/it]
 58%|█████▊    | 6904/12000 [4:04:46<2:52:22,  2.03s/it]
                                                        

 58%|█████▊    | 6904/12000 [4:04:46<2:52:22,  2.03s/it]
 58%|█████▊    | 6905/12000 [4:04:48<2:44:46,  1.94s/it]
                                                        

 58%|█████▊    | 6905/12000 [4:04:48<2:44:46,  1.94s/it]
 58%|█████▊    | 6906/12000 [4:04:50<2:32:44,  1.80s/it]
                                                        

 58%|█████▊    | 6906/12000 [4:04:50<2:32:44,  1.80s/it]
 58%|█████▊    | 6907/12000 [4:04:52<2:39:54,  1.88s/it]
                                                        

 58%|█████▊    | 6907/12000 [4:04:52<2:39:54,  1.88s/it]
 58%|█████▊    | 6908/12000 [4:04:53<2:28:00,  1.74s/it]
                                                        

 58%|█████▊    | 6908/12000 [4:04:53<2:28:00,  1.74s/it]
 58%|█████▊    | 6909/12000 [4:04:55<2:27:22,  1.74s/it]
                                                        

 58%|█████▊    | 6909/12000 [4:04:55<2:27:22,  1.74s/it]
 58%|█████▊    | 6910/12000 [4:04:56<2:19:50,  1.65s/it]
                                                        

 58%|█████▊    | 6910/12000 [4:04:56<2:19:50,  1.65s/it]
 58%|█████▊    | 6911/12000 [4:05:00<3:18:08,  2.34s/it]
                                                        

 58%|█████▊    | 6911/12000 [4:05:00<3:18:08,  2.34s/it]
 58%|█████▊    | 6912/12000 [4:05:03<3:15:52,  2.31s/it]
                                                        

 58%|█████▊    | 6912/12000 [4:05:03<3:15:52,  2.31s/it]
 58%|█████▊    | 6913/12000 [4:05:04<3:02:35,  2.15s/it]
                                                        
{'loss': '0.8139', 'grad_norm': '0.4189', 'learning_rate': '8.449e-05', 'entropy': '0.8126', 'num_tokens': '3.705e+07', 'mean_token_accuracy': '0.7911', 'epoch': '0.06076'}
{'loss': '1.009', 'grad_norm': '0.512', 'learning_rate': '8.447e-05', 'entropy': '0.8964', 'num_tokens': '3.705e+07', 'mean_token_accuracy': '0.7855', 'epoch': '0.06077'}
{'loss': '0.7637', 'grad_norm': '0.3563', 'learning_rate': '8.444e-05', 'entropy': '0.8088', 'num_tokens': '3.706e+07', 'mean_token_accuracy': '0.8072', 'epoch': '0.06078'}
{'loss': '0.4982', 'grad_norm': '0.304', 'learning_rate': '8.441e-05', 'entropy': '0.5211', 'num_tokens': '3.706e+07', 'mean_token_accuracy': '0.858', 'epoch': '0.06079'}
{'loss': '0.9376', 'grad_norm': '0.4362', 'learning_rate': '8.438e-05', 'entropy': '0.9076', 'num_tokens': '3.707e+07', 'mean_token_accuracy': '0.7812', 'epoch': '0.0608'}
{'loss': '0.6712', 'grad_norm': '0.3516', 'learning_rate': '8.436e-05', 'entropy': '0.6858', 'num_tokens': '3.707e+07', 'mean_token_accuracy': '0.8323', 'epoch': '0.06081'}
{'loss': '0.7255', 'grad_norm': '0.4116', 'learning_rate': '8.433e-05', 'entropy': '0.7104', 'num_tokens': '3.708e+07', 'mean_token_accuracy': '0.8108', 'epoch': '0.06082'}
{'loss': '0.7077', 'grad_norm': '0.3446', 'learning_rate': '8.43e-05', 'entropy': '0.7067', 'num_tokens': '3.708e+07', 'mean_token_accuracy': '0.8108', 'epoch': '0.06082'}
{'loss': '0.4387', 'grad_norm': '0.3021', 'learning_rate': '8.427e-05', 'entropy': '0.4717', 'num_tokens': '3.709e+07', 'mean_token_accuracy': '0.8692', 'epoch': '0.06083'}
{'loss': '0.6349', 'grad_norm': '0.404', 'learning_rate': '8.425e-05', 'entropy': '0.7021', 'num_tokens': '3.709e+07', 'mean_token_accuracy': '0.8186', 'epoch': '0.06084'}
{'loss': '0.8176', 'grad_norm': '0.3659', 'learning_rate': '8.422e-05', 'entropy': '0.8341', 'num_tokens': '3.71e+07', 'mean_token_accuracy': '0.7834', 'epoch': '0.06085'}
{'loss': '0.8229', 'grad_norm': '0.3648', 'learning_rate': '8.419e-05', 'entropy': '0.8544', 'num_tokens': '3.711e+07', 'mean_token_accuracy': '0.787', 'epoch': '0.06086'}
{'loss': '0.7897', 'grad_norm': '0.362', 'learning_rate': '8.417e-05', 'entropy': '0.7991', 'num_tokens': '3.711e+07', 'mean_token_accuracy': '0.811', 'epoch': '0.06087'}
{'loss': '0.6312', 'grad_norm': '0.3261', 'learning_rate': '8.414e-05', 'entropy': '0.5972', 'num_tokens': '3.712e+07', 'mean_token_accuracy': '0.8444', 'epoch': '0.06088'}
{'loss': '0.8353', 'grad_norm': '0.4411', 'learning_rate': '8.411e-05', 'entropy': '0.8472', 'num_tokens': '3.713e+07', 'mean_token_accuracy': '0.7893', 'epoch': '0.06089'}
{'loss': '1.083', 'grad_norm': '0.4316', 'learning_rate': '8.408e-05', 'entropy': '1.022', 'num_tokens': '3.713e+07', 'mean_token_accuracy': '0.7547', 'epoch': '0.0609'}
{'loss': '1.077', 'grad_norm': '0.4296', 'learning_rate': '8.406e-05', 'entropy': '1.016', 'num_tokens': '3.714e+07', 'mean_token_accuracy': '0.7677', 'epoch': '0.0609'}
{'loss': '0.9387', 'grad_norm': '0.3993', 'learning_rate': '8.403e-05', 'entropy': '0.9304', 'num_tokens': '3.714e+07', 'mean_token_accuracy': '0.7848', 'epoch': '0.06091'}
{'loss': '0.9565', 'grad_norm': '0.3813', 'learning_rate': '8.4e-05', 'entropy': '0.8846', 'num_tokens': '3.715e+07', 'mean_token_accuracy': '0.7875', 'epoch': '0.06092'}
{'loss': '0.6116', 'grad_norm': '0.335', 'learning_rate': '8.398e-05', 'entropy': '0.5745', 'num_tokens': '3.715e+07', 'mean_token_accuracy': '0.848', 'epoch': '0.06093'}
{'loss': '0.6782', 'grad_norm': '0.3034', 'learning_rate': '8.395e-05', 'entropy': '0.6269', 'num_tokens': '3.716e+07', 'mean_token_accuracy': '0.8381', 'epoch': '0.06094'}
{'loss': '0.5559', 'grad_norm': '0.2732', 'learning_rate': '8.392e-05', 'entropy': '0.6312', 'num_tokens': '3.717e+07', 'mean_token_accuracy': '0.8414', 'epoch': '0.06095'}
{'loss': '0.838', 'grad_norm': '0.3568', 'learning_rate': '8.389e-05', 'entropy': '0.8751', 'num_tokens': '3.718e+07', 'mean_token_accuracy': '0.7663', 'epoch': '0.06096'}
{'loss': '0.5528', 'grad_norm': '0.306', 'learning_rate': '8.387e-05', 'entropy': '0.6126', 'num_tokens': '3.718e+07', 'mean_token_accuracy': '0.8364', 'epoch': '0.06097'}
{'loss': '0.7866', 'grad_norm': '0.4401', 'learning_rate': '8.384e-05', 'entropy': '0.7807', 'num_tokens': '3.719e+07', 'mean_token_accuracy': '0.8043', 'epoch': '0.06098'}
{'loss': '0.725', 'grad_norm': '0.4758', 'learning_rate': '8.381e-05', 'entropy': '0.7313', 'num_tokens': '3.719e+07', 'mean_token_accuracy': '0.804', 'epoch': '0.06098'}
{'loss': '0.8364', 'grad_norm': '0.3944', 'learning_rate': '8.379e-05', 'entropy': '0.826', 'num_tokens': '3.719e+07', 'mean_token_accuracy': '0.8033', 'epoch': '0.06099'}
{'loss': '1.072', 'grad_norm': '0.473', 'learning_rate': '8.376e-05', 'entropy': '1.192', 'num_tokens': '3.72e+07', 'mean_token_accuracy': '0.7203', 'epoch': '0.061'}
{'loss': '1.042', 'grad_norm': '0.3818', 'learning_rate': '8.373e-05', 'entropy': '1.081', 'num_tokens': '3.72e+07', 'mean_token_accuracy': '0.7268', 'epoch': '0.06101'}
{'loss': '0.6741', 'grad_norm': '0.4588', 'learning_rate': '8.37e-05', 'entropy': '0.7507', 'num_tokens': '3.721e+07', 'mean_token_accuracy': '0.813', 'epoch': '0.06102'}
{'loss': '0.8903', 'grad_norm': '0.3364', 'learning_rate': '8.368e-05', 'entropy': '0.9169', 'num_tokens': '3.721e+07', 'mean_token_accuracy': '0.7771', 'epoch': '0.06103'}
{'loss': '0.9663', 'grad_norm': '0.3959', 'learning_rate': '8.365e-05', 'entropy': '0.9323', 'num_tokens': '3.722e+07', 'mean_token_accuracy': '0.7642', 'epoch': '0.06104'}
{'loss': '0.7277', 'grad_norm': '0.3322', 'learning_rate': '8.362e-05', 'entropy': '0.7337', 'num_tokens': '3.722e+07', 'mean_token_accuracy': '0.8168', 'epoch': '0.06105'}
{'loss': '0.9353', 'grad_norm': '0.4283', 'learning_rate': '8.359e-05', 'entropy': '0.9328', 'num_tokens': '3.723e+07', 'mean_token_accuracy': '0.7642', 'epoch': '0.06105'}
{'loss': '0.7651', 'grad_norm': '0.4568', 'learning_rate': '8.357e-05', 'entropy': '0.756', 'num_tokens': '3.723e+07', 'mean_token_accuracy': '0.8025', 'epoch': '0.06106'}
{'loss': '0.6343', 'grad_norm': '0.2991', 'learning_rate': '8.354e-05', 'entropy': '0.6595', 'num_tokens': '3.724e+07', 'mean_token_accuracy': '0.8256', 'epoch': '0.06107'}
{'loss': '0.8066', 'grad_norm': '0.31', 'learning_rate': '8.351e-05', 'entropy': '0.772', 'num_tokens': '3.724e+07', 'mean_token_accuracy': '0.8128', 'epoch': '0.06108'}
{'loss': '0.671', 'grad_norm': '0.4168', 'learning_rate': '8.349e-05', 'entropy': '0.6518', 'num_tokens': '3.725e+07', 'mean_token_accuracy': '0.8261', 'epoch': '0.06109'}
{'loss': '0.6836', 'grad_norm': '0.3985', 'learning_rate': '8.346e-05', 'entropy': '0.644', 'num_tokens': '3.725e+07', 'mean_token_accuracy': '0.8372', 'epoch': '0.0611'}
{'loss': '0.7892', 'grad_norm': '0.4306', 'learning_rate': '8.343e-05', 'entropy': '0.7568', 'num_tokens': '3.726e+07', 'mean_token_accuracy': '0.7973', 'epoch': '0.06111'}
{'loss': '0.5944', 'grad_norm': '0.3719', 'learning_rate': '8.34e-05', 'entropy': '0.6584', 'num_tokens': '3.726e+07', 'mean_token_accuracy': '0.828', 'epoch': '0.06112'}
{'loss': '0.7857', 'grad_norm': '0.3431', 'learning_rate': '8.338e-05', 'entropy': '0.8302', 'num_tokens': '3.727e+07', 'mean_token_accuracy': '0.7895', 'epoch': '0.06113'}
{'loss': '0.9745', 'grad_norm': '0.3795', 'learning_rate': '8.335e-05', 'entropy': '0.9957', 'num_tokens': '3.727e+07', 'mean_token_accuracy': '0.7467', 'epoch': '0.06113'}
{'loss': '0.6004', 'grad_norm': '0.3512', 'learning_rate': '8.332e-05', 'entropy': '0.5468', 'num_tokens': '3.728e+07', 'mean_token_accuracy': '0.8524', 'epoch': '0.06114'}
{'loss': '0.5014', 'grad_norm': '0.2794', 'learning_rate': '8.33e-05', 'entropy': '0.5554', 'num_tokens': '3.728e+07', 'mean_token_accuracy': '0.852', 'epoch': '0.06115'}
{'loss': '0.9263', 'grad_norm': '0.3511', 'learning_rate': '8.327e-05', 'entropy': '0.9481', 'num_tokens': '3.729e+07', 'mean_token_accuracy': '0.7681', 'epoch': '0.06116'}
{'loss': '0.6835', 'grad_norm': '0.3675', 'learning_rate': '8.324e-05', 'entropy': '0.6305', 'num_tokens': '3.729e+07', 'mean_token_accuracy': '0.8379', 'epoch': '0.06117'}

 58%|█████▊    | 6913/12000 [4:05:04<3:02:35,  2.15s/it]
 58%|█████▊    | 6914/12000 [4:05:07<3:04:03,  2.17s/it]
                                                        

 58%|█████▊    | 6914/12000 [4:05:07<3:04:03,  2.17s/it]
 58%|█████▊    | 6915/12000 [4:05:08<2:50:21,  2.01s/it]
                                                        

 58%|█████▊    | 6915/12000 [4:05:08<2:50:21,  2.01s/it]
 58%|█████▊    | 6916/12000 [4:05:10<2:43:31,  1.93s/it]
                                                        

 58%|█████▊    | 6916/12000 [4:05:10<2:43:31,  1.93s/it]
 58%|█████▊    | 6917/12000 [4:05:12<2:44:49,  1.95s/it]
                                                        

 58%|█████▊    | 6917/12000 [4:05:12<2:44:49,  1.95s/it]
 58%|█████▊    | 6918/12000 [4:05:14<2:41:27,  1.91s/it]
                                                        

 58%|█████▊    | 6918/12000 [4:05:14<2:41:27,  1.91s/it]
 58%|█████▊    | 6919/12000 [4:05:15<2:30:46,  1.78s/it]
                                                        

 58%|█████▊    | 6919/12000 [4:05:15<2:30:46,  1.78s/it]
 58%|█████▊    | 6920/12000 [4:05:20<3:39:28,  2.59s/it]
                                                        

 58%|█████▊    | 6920/12000 [4:05:20<3:39:28,  2.59s/it]
 58%|█████▊    | 6921/12000 [4:05:21<3:19:42,  2.36s/it]
                                                        

 58%|█████▊    | 6921/12000 [4:05:21<3:19:42,  2.36s/it]
 58%|█████▊    | 6922/12000 [4:05:23<2:58:42,  2.11s/it]
                                                        

 58%|█████▊    | 6922/12000 [4:05:23<2:58:42,  2.11s/it]
 58%|█████▊    | 6923/12000 [4:05:26<3:14:59,  2.30s/it]
                                                        

 58%|█████▊    | 6923/12000 [4:05:26<3:14:59,  2.30s/it]
 58%|█████▊    | 6924/12000 [4:05:27<2:51:30,  2.03s/it]
                                                        

 58%|█████▊    | 6924/12000 [4:05:27<2:51:30,  2.03s/it]
 58%|█████▊    | 6925/12000 [4:05:29<2:41:41,  1.91s/it]
                                                        

 58%|█████▊    | 6925/12000 [4:05:29<2:41:41,  1.91s/it]
 58%|█████▊    | 6926/12000 [4:05:31<3:01:35,  2.15s/it]
                                                        

 58%|█████▊    | 6926/12000 [4:05:31<3:01:35,  2.15s/it]
 58%|█████▊    | 6927/12000 [4:05:33<2:51:29,  2.03s/it]
                                                        

 58%|█████▊    | 6927/12000 [4:05:33<2:51:29,  2.03s/it]
 58%|█████▊    | 6928/12000 [4:05:35<2:40:11,  1.89s/it]
                                                        

 58%|█████▊    | 6928/12000 [4:05:35<2:40:11,  1.89s/it]
 58%|█████▊    | 6929/12000 [4:05:37<2:54:28,  2.06s/it]
                                                        

 58%|█████▊    | 6929/12000 [4:05:37<2:54:28,  2.06s/it]
 58%|█████▊    | 6930/12000 [4:05:39<2:51:55,  2.03s/it]
                                                        

 58%|█████▊    | 6930/12000 [4:05:39<2:51:55,  2.03s/it]
 58%|█████▊    | 6931/12000 [4:05:41<2:57:22,  2.10s/it]
                                                        

 58%|█████▊    | 6931/12000 [4:05:41<2:57:22,  2.10s/it]
 58%|█████▊    | 6932/12000 [4:05:43<2:48:04,  1.99s/it]
                                                        

 58%|█████▊    | 6932/12000 [4:05:43<2:48:04,  1.99s/it]
 58%|█████▊    | 6933/12000 [4:05:45<2:34:12,  1.83s/it]
                                                        

 58%|█████▊    | 6933/12000 [4:05:45<2:34:12,  1.83s/it]
 58%|█████▊    | 6934/12000 [4:05:46<2:24:14,  1.71s/it]
                                                        

 58%|█████▊    | 6934/12000 [4:05:46<2:24:14,  1.71s/it]
 58%|█████▊    | 6935/12000 [4:05:49<3:00:31,  2.14s/it]
                                                        

 58%|█████▊    | 6935/12000 [4:05:49<3:00:31,  2.14s/it]
 58%|█████▊    | 6936/12000 [4:05:53<3:51:27,  2.74s/it]
                                                        

 58%|█████▊    | 6936/12000 [4:05:53<3:51:27,  2.74s/it]
 58%|█████▊    | 6937/12000 [4:05:58<4:38:08,  3.30s/it]
                                                        

 58%|█████▊    | 6937/12000 [4:05:58<4:38:08,  3.30s/it]
 58%|█████▊    | 6938/12000 [4:05:59<3:51:26,  2.74s/it]
                                                        

 58%|█████▊    | 6938/12000 [4:05:59<3:51:26,  2.74s/it]
 58%|█████▊    | 6939/12000 [4:06:01<3:20:32,  2.38s/it]
                                                        

 58%|█████▊    | 6939/12000 [4:06:01<3:20:32,  2.38s/it]
 58%|█████▊    | 6940/12000 [4:06:03<3:12:41,  2.28s/it]
                                                        

 58%|█████▊    | 6940/12000 [4:06:03<3:12:41,  2.28s/it]
 58%|█████▊    | 6941/12000 [4:06:05<2:53:44,  2.06s/it]
                                                        

 58%|█████▊    | 6941/12000 [4:06:05<2:53:44,  2.06s/it]
 58%|█████▊    | 6942/12000 [4:06:09<3:42:49,  2.64s/it]
                                                        

 58%|█████▊    | 6942/12000 [4:06:09<3:42:49,  2.64s/it]
 58%|█████▊    | 6943/12000 [4:06:11<3:27:11,  2.46s/it]
                                                        

 58%|█████▊    | 6943/12000 [4:06:11<3:27:11,  2.46s/it]
 58%|█████▊    | 6944/12000 [4:06:12<3:11:49,  2.28s/it]
                                                        

 58%|█████▊    | 6944/12000 [4:06:12<3:11:49,  2.28s/it]
 58%|█████▊    | 6945/12000 [4:06:14<2:52:58,  2.05s/it]
                                                        

 58%|█████▊    | 6945/12000 [4:06:14<2:52:58,  2.05s/it]
 58%|█████▊    | 6946/12000 [4:06:15<2:35:00,  1.84s/it]
                                                        

 58%|█████▊    | 6946/12000 [4:06:15<2:35:00,  1.84s/it]
 58%|█████▊    | 6947/12000 [4:06:18<2:47:41,  1.99s/it]
                                                        

 58%|█████▊    | 6947/12000 [4:06:18<2:47:41,  1.99s/it]
 58%|█████▊    | 6948/12000 [4:06:22<3:46:55,  2.70s/it]
                                                        

 58%|█████▊    | 6948/12000 [4:06:22<3:46:55,  2.70s/it]
 58%|█████▊    | 6949/12000 [4:06:25<3:49:36,  2.73s/it]
                                                        

 58%|█████▊    | 6949/12000 [4:06:25<3:49:36,  2.73s/it]
 58%|█████▊    | 6950/12000 [4:06:27<3:40:15,  2.62s/it]
                                                        

 58%|█████▊    | 6950/12000 [4:06:27<3:40:15,  2.62s/it]
 58%|█████▊    | 6951/12000 [4:06:30<3:57:34,  2.82s/it]
                                                        

 58%|█████▊    | 6951/12000 [4:06:30<3:57:34,  2.82s/it]
 58%|█████▊    | 6952/12000 [4:06:32<3:34:36,  2.55s/it]
                                                        

 58%|█████▊    | 6952/12000 [4:06:32<3:34:36,  2.55s/it]
 58%|█████▊    | 6953/12000 [4:06:34<3:12:00,  2.28s/it]
                                                        

 58%|█████▊    | 6953/12000 [4:06:34<3:12:00,  2.28s/it]
 58%|█████▊    | 6954/12000 [4:06:37<3:29:05,  2.49s/it]
                                                        

 58%|█████▊    | 6954/12000 [4:06:37<3:29:05,  2.49s/it]
 58%|█████▊    | 6955/12000 [4:06:41<4:03:39,  2.90s/it]
                                                        

 58%|█████▊    | 6955/12000 [4:06:41<4:03:39,  2.90s/it]
 58%|█████▊    | 6956/12000 [4:06:42<3:29:43,  2.49s/it]
                                                        

 58%|█████▊    | 6956/12000 [4:06:42<3:29:43,  2.49s/it]
 58%|█████▊    | 6957/12000 [4:06:45<3:32:15,  2.53s/it]
                                                        

 58%|█████▊    | 6957/12000 [4:06:45<3:32:15,  2.53s/it]
 58%|█████▊    | 6958/12000 [4:06:47<3:17:34,  2.35s/it]
                                                        

 58%|█████▊    | 6958/12000 [4:06:47<3:17:34,  2.35s/it]
 58%|█████▊    | 6959/12000 [4:06:49<3:08:45,  2.25s/it]
                                                        

 58%|█████▊    | 6959/12000 [4:06:49<3:08:45,  2.25s/it]
 58%|█████▊    | 6960/12000 [4:06:51<3:11:52,  2.28s/it]
                                                        
{'loss': '0.8332', 'grad_norm': '0.3609', 'learning_rate': '8.321e-05', 'entropy': '0.8555', 'num_tokens': '3.73e+07', 'mean_token_accuracy': '0.7841', 'epoch': '0.06118'}
{'loss': '1.117', 'grad_norm': '0.5518', 'learning_rate': '8.319e-05', 'entropy': '1.021', 'num_tokens': '3.73e+07', 'mean_token_accuracy': '0.7568', 'epoch': '0.06119'}
{'loss': '0.7284', 'grad_norm': '0.4972', 'learning_rate': '8.316e-05', 'entropy': '0.7753', 'num_tokens': '3.731e+07', 'mean_token_accuracy': '0.7878', 'epoch': '0.0612'}
{'loss': '0.837', 'grad_norm': '0.4723', 'learning_rate': '8.313e-05', 'entropy': '0.8312', 'num_tokens': '3.731e+07', 'mean_token_accuracy': '0.7887', 'epoch': '0.06121'}
{'loss': '0.9839', 'grad_norm': '0.4075', 'learning_rate': '8.311e-05', 'entropy': '0.9385', 'num_tokens': '3.732e+07', 'mean_token_accuracy': '0.7625', 'epoch': '0.06121'}
{'loss': '0.7968', 'grad_norm': '0.5085', 'learning_rate': '8.308e-05', 'entropy': '0.7473', 'num_tokens': '3.732e+07', 'mean_token_accuracy': '0.8134', 'epoch': '0.06122'}
{'loss': '0.7336', 'grad_norm': '0.3676', 'learning_rate': '8.305e-05', 'entropy': '0.8001', 'num_tokens': '3.733e+07', 'mean_token_accuracy': '0.803', 'epoch': '0.06123'}
{'loss': '0.856', 'grad_norm': '0.47', 'learning_rate': '8.302e-05', 'entropy': '0.8604', 'num_tokens': '3.733e+07', 'mean_token_accuracy': '0.783', 'epoch': '0.06124'}
{'loss': '0.7597', 'grad_norm': '0.514', 'learning_rate': '8.3e-05', 'entropy': '0.7483', 'num_tokens': '3.734e+07', 'mean_token_accuracy': '0.811', 'epoch': '0.06125'}
{'loss': '0.6671', 'grad_norm': '0.3302', 'learning_rate': '8.297e-05', 'entropy': '0.6921', 'num_tokens': '3.734e+07', 'mean_token_accuracy': '0.8271', 'epoch': '0.06126'}
{'loss': '0.5987', 'grad_norm': '0.3287', 'learning_rate': '8.294e-05', 'entropy': '0.6178', 'num_tokens': '3.735e+07', 'mean_token_accuracy': '0.8516', 'epoch': '0.06127'}
{'loss': '0.615', 'grad_norm': '0.3393', 'learning_rate': '8.292e-05', 'entropy': '0.6078', 'num_tokens': '3.735e+07', 'mean_token_accuracy': '0.8483', 'epoch': '0.06128'}
{'loss': '0.6925', 'grad_norm': '0.3427', 'learning_rate': '8.289e-05', 'entropy': '0.7076', 'num_tokens': '3.736e+07', 'mean_token_accuracy': '0.8129', 'epoch': '0.06129'}
{'loss': '1.141', 'grad_norm': '0.4117', 'learning_rate': '8.286e-05', 'entropy': '1.132', 'num_tokens': '3.737e+07', 'mean_token_accuracy': '0.7166', 'epoch': '0.06129'}
{'loss': '0.6738', 'grad_norm': '0.3234', 'learning_rate': '8.283e-05', 'entropy': '0.7782', 'num_tokens': '3.737e+07', 'mean_token_accuracy': '0.8111', 'epoch': '0.0613'}
{'loss': '1.069', 'grad_norm': '0.4299', 'learning_rate': '8.281e-05', 'entropy': '1.113', 'num_tokens': '3.738e+07', 'mean_token_accuracy': '0.7405', 'epoch': '0.06131'}
{'loss': '0.8644', 'grad_norm': '0.3237', 'learning_rate': '8.278e-05', 'entropy': '0.9163', 'num_tokens': '3.738e+07', 'mean_token_accuracy': '0.7804', 'epoch': '0.06132'}
{'loss': '0.9325', 'grad_norm': '0.4543', 'learning_rate': '8.275e-05', 'entropy': '0.8281', 'num_tokens': '3.739e+07', 'mean_token_accuracy': '0.7875', 'epoch': '0.06133'}
{'loss': '0.8252', 'grad_norm': '0.4205', 'learning_rate': '8.273e-05', 'entropy': '0.7855', 'num_tokens': '3.739e+07', 'mean_token_accuracy': '0.7995', 'epoch': '0.06134'}
{'loss': '0.6486', 'grad_norm': '0.4039', 'learning_rate': '8.27e-05', 'entropy': '0.7181', 'num_tokens': '3.74e+07', 'mean_token_accuracy': '0.8115', 'epoch': '0.06135'}
{'loss': '0.7178', 'grad_norm': '0.3804', 'learning_rate': '8.267e-05', 'entropy': '0.7569', 'num_tokens': '3.74e+07', 'mean_token_accuracy': '0.8076', 'epoch': '0.06136'}
{'loss': '0.6447', 'grad_norm': '0.3931', 'learning_rate': '8.264e-05', 'entropy': '0.7017', 'num_tokens': '3.74e+07', 'mean_token_accuracy': '0.8149', 'epoch': '0.06136'}
{'loss': '0.9232', 'grad_norm': '0.4464', 'learning_rate': '8.262e-05', 'entropy': '0.9722', 'num_tokens': '3.741e+07', 'mean_token_accuracy': '0.7644', 'epoch': '0.06137'}
{'loss': '1.454', 'grad_norm': '0.4323', 'learning_rate': '8.259e-05', 'entropy': '1.23', 'num_tokens': '3.742e+07', 'mean_token_accuracy': '0.717', 'epoch': '0.06138'}
{'loss': '1.655', 'grad_norm': '0.4637', 'learning_rate': '8.256e-05', 'entropy': '1.397', 'num_tokens': '3.743e+07', 'mean_token_accuracy': '0.6833', 'epoch': '0.06139'}
{'loss': '0.7359', 'grad_norm': '0.3537', 'learning_rate': '8.254e-05', 'entropy': '0.8159', 'num_tokens': '3.743e+07', 'mean_token_accuracy': '0.8122', 'epoch': '0.0614'}
{'loss': '0.8033', 'grad_norm': '0.4102', 'learning_rate': '8.251e-05', 'entropy': '0.7465', 'num_tokens': '3.744e+07', 'mean_token_accuracy': '0.8199', 'epoch': '0.06141'}
{'loss': '0.8615', 'grad_norm': '0.4253', 'learning_rate': '8.248e-05', 'entropy': '0.7625', 'num_tokens': '3.744e+07', 'mean_token_accuracy': '0.8059', 'epoch': '0.06142'}
{'loss': '0.9236', 'grad_norm': '0.4333', 'learning_rate': '8.245e-05', 'entropy': '0.8771', 'num_tokens': '3.745e+07', 'mean_token_accuracy': '0.7745', 'epoch': '0.06143'}
{'loss': '0.8475', 'grad_norm': '0.3959', 'learning_rate': '8.243e-05', 'entropy': '0.8074', 'num_tokens': '3.745e+07', 'mean_token_accuracy': '0.7883', 'epoch': '0.06144'}
{'loss': '0.8274', 'grad_norm': '0.3351', 'learning_rate': '8.24e-05', 'entropy': '0.8366', 'num_tokens': '3.746e+07', 'mean_token_accuracy': '0.7925', 'epoch': '0.06144'}
{'loss': '0.7866', 'grad_norm': '0.3779', 'learning_rate': '8.237e-05', 'entropy': '0.7572', 'num_tokens': '3.746e+07', 'mean_token_accuracy': '0.8029', 'epoch': '0.06145'}
{'loss': '1.011', 'grad_norm': '0.3834', 'learning_rate': '8.235e-05', 'entropy': '1.086', 'num_tokens': '3.747e+07', 'mean_token_accuracy': '0.7412', 'epoch': '0.06146'}
{'loss': '0.8784', 'grad_norm': '0.4461', 'learning_rate': '8.232e-05', 'entropy': '0.9028', 'num_tokens': '3.747e+07', 'mean_token_accuracy': '0.7727', 'epoch': '0.06147'}
{'loss': '1.245', 'grad_norm': '0.4226', 'learning_rate': '8.229e-05', 'entropy': '1.145', 'num_tokens': '3.748e+07', 'mean_token_accuracy': '0.7239', 'epoch': '0.06148'}
{'loss': '0.8577', 'grad_norm': '0.323', 'learning_rate': '8.226e-05', 'entropy': '0.8571', 'num_tokens': '3.749e+07', 'mean_token_accuracy': '0.7919', 'epoch': '0.06149'}
{'loss': '1.096', 'grad_norm': '0.4763', 'learning_rate': '8.224e-05', 'entropy': '0.9833', 'num_tokens': '3.75e+07', 'mean_token_accuracy': '0.7619', 'epoch': '0.0615'}
{'loss': '1.015', 'grad_norm': '0.3794', 'learning_rate': '8.221e-05', 'entropy': '1.002', 'num_tokens': '3.75e+07', 'mean_token_accuracy': '0.7472', 'epoch': '0.06151'}
{'loss': '0.6287', 'grad_norm': '0.3923', 'learning_rate': '8.218e-05', 'entropy': '0.6139', 'num_tokens': '3.751e+07', 'mean_token_accuracy': '0.8316', 'epoch': '0.06152'}
{'loss': '0.686', 'grad_norm': '0.3532', 'learning_rate': '8.216e-05', 'entropy': '0.6524', 'num_tokens': '3.752e+07', 'mean_token_accuracy': '0.8132', 'epoch': '0.06152'}
{'loss': '1.026', 'grad_norm': '0.4492', 'learning_rate': '8.213e-05', 'entropy': '1.103', 'num_tokens': '3.752e+07', 'mean_token_accuracy': '0.7473', 'epoch': '0.06153'}
{'loss': '0.8779', 'grad_norm': '0.351', 'learning_rate': '8.21e-05', 'entropy': '0.7778', 'num_tokens': '3.753e+07', 'mean_token_accuracy': '0.8183', 'epoch': '0.06154'}
{'loss': '1.263', 'grad_norm': '0.4104', 'learning_rate': '8.207e-05', 'entropy': '1.121', 'num_tokens': '3.753e+07', 'mean_token_accuracy': '0.7411', 'epoch': '0.06155'}
{'loss': '0.5233', 'grad_norm': '0.2989', 'learning_rate': '8.205e-05', 'entropy': '0.557', 'num_tokens': '3.754e+07', 'mean_token_accuracy': '0.8604', 'epoch': '0.06156'}
{'loss': '0.8193', 'grad_norm': '0.3598', 'learning_rate': '8.202e-05', 'entropy': '0.7507', 'num_tokens': '3.755e+07', 'mean_token_accuracy': '0.818', 'epoch': '0.06157'}
{'loss': '0.6893', 'grad_norm': '0.3178', 'learning_rate': '8.199e-05', 'entropy': '0.6941', 'num_tokens': '3.755e+07', 'mean_token_accuracy': '0.8126', 'epoch': '0.06158'}
{'loss': '1.006', 'grad_norm': '0.4371', 'learning_rate': '8.197e-05', 'entropy': '0.9995', 'num_tokens': '3.756e+07', 'mean_token_accuracy': '0.7472', 'epoch': '0.06159'}

 58%|█████▊    | 6960/12000 [4:06:51<3:11:52,  2.28s/it]
 58%|█████▊    | 6961/12000 [4:06:53<2:52:22,  2.05s/it]
                                                        

 58%|█████▊    | 6961/12000 [4:06:53<2:52:22,  2.05s/it]
 58%|█████▊    | 6962/12000 [4:06:55<2:42:56,  1.94s/it]
                                                        

 58%|█████▊    | 6962/12000 [4:06:55<2:42:56,  1.94s/it]
 58%|█████▊    | 6963/12000 [4:06:56<2:31:26,  1.80s/it]
                                                        

 58%|█████▊    | 6963/12000 [4:06:56<2:31:26,  1.80s/it]
 58%|█████▊    | 6964/12000 [4:06:59<3:12:46,  2.30s/it]
                                                        

 58%|█████▊    | 6964/12000 [4:06:59<3:12:46,  2.30s/it]
 58%|█████▊    | 6965/12000 [4:07:01<2:49:38,  2.02s/it]
                                                        

 58%|█████▊    | 6965/12000 [4:07:01<2:49:38,  2.02s/it]
 58%|█████▊    | 6966/12000 [4:07:02<2:40:06,  1.91s/it]
                                                        

 58%|█████▊    | 6966/12000 [4:07:02<2:40:06,  1.91s/it]
 58%|█████▊    | 6967/12000 [4:07:05<2:44:26,  1.96s/it]
                                                        

 58%|█████▊    | 6967/12000 [4:07:05<2:44:26,  1.96s/it]
 58%|█████▊    | 6968/12000 [4:07:07<2:50:56,  2.04s/it]
                                                        

 58%|█████▊    | 6968/12000 [4:07:07<2:50:56,  2.04s/it]
 58%|█████▊    | 6969/12000 [4:07:08<2:32:05,  1.81s/it]
                                                        

 58%|█████▊    | 6969/12000 [4:07:08<2:32:05,  1.81s/it]
 58%|█████▊    | 6970/12000 [4:07:10<2:26:40,  1.75s/it]
                                                        

 58%|█████▊    | 6970/12000 [4:07:10<2:26:40,  1.75s/it]
 58%|█████▊    | 6971/12000 [4:07:12<2:30:09,  1.79s/it]
                                                        

 58%|█████▊    | 6971/12000 [4:07:12<2:30:09,  1.79s/it]
 58%|█████▊    | 6972/12000 [4:07:14<2:35:01,  1.85s/it]
                                                        

 58%|█████▊    | 6972/12000 [4:07:14<2:35:01,  1.85s/it]
 58%|█████▊    | 6973/12000 [4:07:15<2:35:52,  1.86s/it]
                                                        

 58%|█████▊    | 6973/12000 [4:07:15<2:35:52,  1.86s/it]
 58%|█████▊    | 6974/12000 [4:07:17<2:28:13,  1.77s/it]
                                                        

 58%|█████▊    | 6974/12000 [4:07:17<2:28:13,  1.77s/it]
 58%|█████▊    | 6975/12000 [4:07:19<2:24:19,  1.72s/it]
                                                        

 58%|█████▊    | 6975/12000 [4:07:19<2:24:19,  1.72s/it]
 58%|█████▊    | 6976/12000 [4:07:20<2:20:50,  1.68s/it]
                                                        

 58%|█████▊    | 6976/12000 [4:07:20<2:20:50,  1.68s/it]
 58%|█████▊    | 6977/12000 [4:07:23<2:47:07,  2.00s/it]
                                                        

 58%|█████▊    | 6977/12000 [4:07:23<2:47:07,  2.00s/it]
 58%|█████▊    | 6978/12000 [4:07:25<2:37:29,  1.88s/it]
                                                        

 58%|█████▊    | 6978/12000 [4:07:25<2:37:29,  1.88s/it]
 58%|█████▊    | 6979/12000 [4:07:27<2:43:58,  1.96s/it]
                                                        

 58%|█████▊    | 6979/12000 [4:07:27<2:43:58,  1.96s/it]
 58%|█████▊    | 6980/12000 [4:07:29<2:44:52,  1.97s/it]
                                                        

 58%|█████▊    | 6980/12000 [4:07:29<2:44:52,  1.97s/it]
 58%|█████▊    | 6981/12000 [4:07:30<2:31:38,  1.81s/it]
                                                        

 58%|█████▊    | 6981/12000 [4:07:30<2:31:38,  1.81s/it]
 58%|█████▊    | 6982/12000 [4:07:32<2:41:21,  1.93s/it]
                                                        

 58%|█████▊    | 6982/12000 [4:07:32<2:41:21,  1.93s/it]
 58%|█████▊    | 6983/12000 [4:07:34<2:29:37,  1.79s/it]
                                                        

 58%|█████▊    | 6983/12000 [4:07:34<2:29:37,  1.79s/it]
 58%|█████▊    | 6984/12000 [4:07:36<2:30:06,  1.80s/it]
                                                        

 58%|█████▊    | 6984/12000 [4:07:36<2:30:06,  1.80s/it]
 58%|█████▊    | 6985/12000 [4:07:38<2:49:45,  2.03s/it]
                                                        

 58%|█████▊    | 6985/12000 [4:07:38<2:49:45,  2.03s/it]
 58%|█████▊    | 6986/12000 [4:07:42<3:28:42,  2.50s/it]
                                                        

 58%|█████▊    | 6986/12000 [4:07:42<3:28:42,  2.50s/it]
 58%|█████▊    | 6987/12000 [4:07:44<3:13:29,  2.32s/it]
                                                        

 58%|█████▊    | 6987/12000 [4:07:44<3:13:29,  2.32s/it]
 58%|█████▊    | 6988/12000 [4:07:45<2:53:25,  2.08s/it]
                                                        

 58%|█████▊    | 6988/12000 [4:07:45<2:53:25,  2.08s/it]
 58%|█████▊    | 6989/12000 [4:07:47<2:43:36,  1.96s/it]
                                                        

 58%|█████▊    | 6989/12000 [4:07:47<2:43:36,  1.96s/it]
 58%|█████▊    | 6990/12000 [4:07:49<2:55:13,  2.10s/it]
                                                        

 58%|█████▊    | 6990/12000 [4:07:49<2:55:13,  2.10s/it]
 58%|█████▊    | 6991/12000 [4:07:51<2:41:39,  1.94s/it]
                                                        

 58%|█████▊    | 6991/12000 [4:07:51<2:41:39,  1.94s/it]
 58%|█████▊    | 6992/12000 [4:07:53<2:46:44,  2.00s/it]
                                                        

 58%|█████▊    | 6992/12000 [4:07:53<2:46:44,  2.00s/it]
 58%|█████▊    | 6993/12000 [4:07:55<2:38:23,  1.90s/it]
                                                        

 58%|█████▊    | 6993/12000 [4:07:55<2:38:23,  1.90s/it]
 58%|█████▊    | 6994/12000 [4:07:57<2:51:42,  2.06s/it]
                                                        

 58%|█████▊    | 6994/12000 [4:07:57<2:51:42,  2.06s/it]
 58%|█████▊    | 6995/12000 [4:07:59<2:52:20,  2.07s/it]
                                                        

 58%|█████▊    | 6995/12000 [4:07:59<2:52:20,  2.07s/it]
 58%|█████▊    | 6996/12000 [4:08:01<2:50:35,  2.05s/it]
                                                        

 58%|█████▊    | 6996/12000 [4:08:01<2:50:35,  2.05s/it]
 58%|█████▊    | 6997/12000 [4:08:03<2:40:33,  1.93s/it]
                                                        

 58%|█████▊    | 6997/12000 [4:08:03<2:40:33,  1.93s/it]
 58%|█████▊    | 6998/12000 [4:08:04<2:34:56,  1.86s/it]
                                                        

 58%|█████▊    | 6998/12000 [4:08:04<2:34:56,  1.86s/it]
 58%|█████▊    | 6999/12000 [4:08:06<2:28:20,  1.78s/it]
                                                        

 58%|█████▊    | 6999/12000 [4:08:06<2:28:20,  1.78s/it]
 58%|█████▊    | 7000/12000 [4:08:08<2:39:51,  1.92s/it]
                                                        

 58%|█████▊    | 7000/12000 [4:08:08<2:39:51,  1.92s/it]
 58%|█████▊    | 7001/12000 [4:08:12<3:11:29,  2.30s/it]
                                                        

 58%|█████▊    | 7001/12000 [4:08:12<3:11:29,  2.30s/it]
 58%|█████▊    | 7002/12000 [4:08:13<2:55:22,  2.11s/it]
                                                        

 58%|█████▊    | 7002/12000 [4:08:13<2:55:22,  2.11s/it]
 58%|█████▊    | 7003/12000 [4:08:15<2:54:31,  2.10s/it]
                                                        

 58%|█████▊    | 7003/12000 [4:08:15<2:54:31,  2.10s/it]
 58%|█████▊    | 7004/12000 [4:08:17<2:56:00,  2.11s/it]
                                                        

 58%|█████▊    | 7004/12000 [4:08:17<2:56:00,  2.11s/it]
 58%|█████▊    | 7005/12000 [4:08:20<2:57:22,  2.13s/it]
                                                        

 58%|█████▊    | 7005/12000 [4:08:20<2:57:22,  2.13s/it]
 58%|█████▊    | 7006/12000 [4:08:21<2:42:05,  1.95s/it]
                                                        

 58%|█████▊    | 7006/12000 [4:08:21<2:42:05,  1.95s/it]
 58%|█████▊    | 7007/12000 [4:08:26<3:44:31,  2.70s/it]
                                                        
{'loss': '0.7517', 'grad_norm': '0.3917', 'learning_rate': '8.194e-05', 'entropy': '0.8534', 'num_tokens': '3.756e+07', 'mean_token_accuracy': '0.785', 'epoch': '0.06159'}
{'loss': '0.608', 'grad_norm': '0.4007', 'learning_rate': '8.191e-05', 'entropy': '0.5808', 'num_tokens': '3.757e+07', 'mean_token_accuracy': '0.8422', 'epoch': '0.0616'}
{'loss': '0.777', 'grad_norm': '0.4726', 'learning_rate': '8.188e-05', 'entropy': '0.7366', 'num_tokens': '3.757e+07', 'mean_token_accuracy': '0.8103', 'epoch': '0.06161'}
{'loss': '0.8213', 'grad_norm': '0.3966', 'learning_rate': '8.186e-05', 'entropy': '0.7639', 'num_tokens': '3.758e+07', 'mean_token_accuracy': '0.8018', 'epoch': '0.06162'}
{'loss': '0.706', 'grad_norm': '0.4612', 'learning_rate': '8.183e-05', 'entropy': '0.691', 'num_tokens': '3.758e+07', 'mean_token_accuracy': '0.8151', 'epoch': '0.06163'}
{'loss': '0.8815', 'grad_norm': '0.4234', 'learning_rate': '8.18e-05', 'entropy': '0.9309', 'num_tokens': '3.759e+07', 'mean_token_accuracy': '0.764', 'epoch': '0.06164'}
{'loss': '1.264', 'grad_norm': '0.4487', 'learning_rate': '8.178e-05', 'entropy': '1.059', 'num_tokens': '3.759e+07', 'mean_token_accuracy': '0.7481', 'epoch': '0.06165'}
{'loss': '0.9496', 'grad_norm': '0.5168', 'learning_rate': '8.175e-05', 'entropy': '0.9097', 'num_tokens': '3.759e+07', 'mean_token_accuracy': '0.7762', 'epoch': '0.06166'}
{'loss': '0.7873', 'grad_norm': '0.4114', 'learning_rate': '8.172e-05', 'entropy': '0.7624', 'num_tokens': '3.76e+07', 'mean_token_accuracy': '0.8058', 'epoch': '0.06167'}
{'loss': '0.9052', 'grad_norm': '0.5131', 'learning_rate': '8.17e-05', 'entropy': '1.111', 'num_tokens': '3.76e+07', 'mean_token_accuracy': '0.7257', 'epoch': '0.06167'}
{'loss': '0.6491', 'grad_norm': '0.3555', 'learning_rate': '8.167e-05', 'entropy': '0.6617', 'num_tokens': '3.761e+07', 'mean_token_accuracy': '0.8198', 'epoch': '0.06168'}
{'loss': '0.7808', 'grad_norm': '0.4042', 'learning_rate': '8.164e-05', 'entropy': '0.8133', 'num_tokens': '3.761e+07', 'mean_token_accuracy': '0.7753', 'epoch': '0.06169'}
{'loss': '0.7564', 'grad_norm': '0.3781', 'learning_rate': '8.161e-05', 'entropy': '0.6925', 'num_tokens': '3.762e+07', 'mean_token_accuracy': '0.8134', 'epoch': '0.0617'}
{'loss': '1.519', 'grad_norm': '0.5019', 'learning_rate': '8.159e-05', 'entropy': '1.529', 'num_tokens': '3.763e+07', 'mean_token_accuracy': '0.6457', 'epoch': '0.06171'}
{'loss': '0.7048', 'grad_norm': '0.3843', 'learning_rate': '8.156e-05', 'entropy': '0.792', 'num_tokens': '3.763e+07', 'mean_token_accuracy': '0.7942', 'epoch': '0.06172'}
{'loss': '0.84', 'grad_norm': '0.3839', 'learning_rate': '8.153e-05', 'entropy': '0.9444', 'num_tokens': '3.763e+07', 'mean_token_accuracy': '0.7732', 'epoch': '0.06173'}
{'loss': '1.241', 'grad_norm': '0.4973', 'learning_rate': '8.151e-05', 'entropy': '1.196', 'num_tokens': '3.764e+07', 'mean_token_accuracy': '0.7214', 'epoch': '0.06174'}
{'loss': '1.229', 'grad_norm': '0.4312', 'learning_rate': '8.148e-05', 'entropy': '1.144', 'num_tokens': '3.764e+07', 'mean_token_accuracy': '0.7117', 'epoch': '0.06175'}
{'loss': '0.7223', 'grad_norm': '0.3679', 'learning_rate': '8.145e-05', 'entropy': '0.8179', 'num_tokens': '3.765e+07', 'mean_token_accuracy': '0.7902', 'epoch': '0.06175'}
{'loss': '1.354', 'grad_norm': '0.4934', 'learning_rate': '8.142e-05', 'entropy': '1.279', 'num_tokens': '3.766e+07', 'mean_token_accuracy': '0.7033', 'epoch': '0.06176'}
{'loss': '1.108', 'grad_norm': '0.4532', 'learning_rate': '8.14e-05', 'entropy': '1.162', 'num_tokens': '3.766e+07', 'mean_token_accuracy': '0.7204', 'epoch': '0.06177'}
{'loss': '0.6176', 'grad_norm': '0.3575', 'learning_rate': '8.137e-05', 'entropy': '0.6324', 'num_tokens': '3.766e+07', 'mean_token_accuracy': '0.8288', 'epoch': '0.06178'}
{'loss': '0.7389', 'grad_norm': '0.3757', 'learning_rate': '8.134e-05', 'entropy': '0.6773', 'num_tokens': '3.767e+07', 'mean_token_accuracy': '0.8281', 'epoch': '0.06179'}
{'loss': '1.196', 'grad_norm': '0.437', 'learning_rate': '8.132e-05', 'entropy': '1.153', 'num_tokens': '3.767e+07', 'mean_token_accuracy': '0.724', 'epoch': '0.0618'}
{'loss': '0.9917', 'grad_norm': '0.4805', 'learning_rate': '8.129e-05', 'entropy': '1.05', 'num_tokens': '3.768e+07', 'mean_token_accuracy': '0.7512', 'epoch': '0.06181'}
{'loss': '0.8197', 'grad_norm': '0.3667', 'learning_rate': '8.126e-05', 'entropy': '0.8278', 'num_tokens': '3.768e+07', 'mean_token_accuracy': '0.7762', 'epoch': '0.06182'}
{'loss': '0.8124', 'grad_norm': '0.4259', 'learning_rate': '8.123e-05', 'entropy': '0.8892', 'num_tokens': '3.769e+07', 'mean_token_accuracy': '0.7809', 'epoch': '0.06182'}
{'loss': '0.7495', 'grad_norm': '0.4367', 'learning_rate': '8.121e-05', 'entropy': '0.8089', 'num_tokens': '3.769e+07', 'mean_token_accuracy': '0.8047', 'epoch': '0.06183'}
{'loss': '0.6357', 'grad_norm': '0.3706', 'learning_rate': '8.118e-05', 'entropy': '0.6618', 'num_tokens': '3.77e+07', 'mean_token_accuracy': '0.8286', 'epoch': '0.06184'}
{'loss': '0.6062', 'grad_norm': '0.3602', 'learning_rate': '8.115e-05', 'entropy': '0.6451', 'num_tokens': '3.77e+07', 'mean_token_accuracy': '0.8247', 'epoch': '0.06185'}
{'loss': '1.12', 'grad_norm': '0.4821', 'learning_rate': '8.113e-05', 'entropy': '0.8867', 'num_tokens': '3.771e+07', 'mean_token_accuracy': '0.785', 'epoch': '0.06186'}
{'loss': '0.8636', 'grad_norm': '0.363', 'learning_rate': '8.11e-05', 'entropy': '0.7945', 'num_tokens': '3.771e+07', 'mean_token_accuracy': '0.7949', 'epoch': '0.06187'}
{'loss': '1.071', 'grad_norm': '0.4011', 'learning_rate': '8.107e-05', 'entropy': '0.9965', 'num_tokens': '3.772e+07', 'mean_token_accuracy': '0.7603', 'epoch': '0.06188'}
{'loss': '0.7577', 'grad_norm': '0.5102', 'learning_rate': '8.105e-05', 'entropy': '0.8511', 'num_tokens': '3.772e+07', 'mean_token_accuracy': '0.7787', 'epoch': '0.06189'}
{'loss': '0.7025', 'grad_norm': '0.2949', 'learning_rate': '8.102e-05', 'entropy': '0.6497', 'num_tokens': '3.773e+07', 'mean_token_accuracy': '0.8308', 'epoch': '0.0619'}
{'loss': '1.115', 'grad_norm': '0.4233', 'learning_rate': '8.099e-05', 'entropy': '0.9845', 'num_tokens': '3.774e+07', 'mean_token_accuracy': '0.7649', 'epoch': '0.0619'}
{'loss': '0.8138', 'grad_norm': '0.3555', 'learning_rate': '8.096e-05', 'entropy': '0.8139', 'num_tokens': '3.774e+07', 'mean_token_accuracy': '0.794', 'epoch': '0.06191'}
{'loss': '0.9004', 'grad_norm': '0.3575', 'learning_rate': '8.094e-05', 'entropy': '0.8985', 'num_tokens': '3.775e+07', 'mean_token_accuracy': '0.7672', 'epoch': '0.06192'}
{'loss': '0.7387', 'grad_norm': '0.3665', 'learning_rate': '8.091e-05', 'entropy': '0.7539', 'num_tokens': '3.775e+07', 'mean_token_accuracy': '0.8016', 'epoch': '0.06193'}
{'loss': '0.7782', 'grad_norm': '0.2974', 'learning_rate': '8.088e-05', 'entropy': '0.7917', 'num_tokens': '3.776e+07', 'mean_token_accuracy': '0.7943', 'epoch': '0.06194'}
{'loss': '1.351', 'grad_norm': '0.5371', 'learning_rate': '8.086e-05', 'entropy': '1.224', 'num_tokens': '3.776e+07', 'mean_token_accuracy': '0.7069', 'epoch': '0.06195'}
{'loss': '0.8167', 'grad_norm': '0.3339', 'learning_rate': '8.083e-05', 'entropy': '0.8109', 'num_tokens': '3.777e+07', 'mean_token_accuracy': '0.7968', 'epoch': '0.06196'}
{'loss': '0.8191', 'grad_norm': '0.3677', 'learning_rate': '8.08e-05', 'entropy': '0.871', 'num_tokens': '3.777e+07', 'mean_token_accuracy': '0.7862', 'epoch': '0.06197'}
{'loss': '1.229', 'grad_norm': '0.3948', 'learning_rate': '8.077e-05', 'entropy': '1.089', 'num_tokens': '3.778e+07', 'mean_token_accuracy': '0.7486', 'epoch': '0.06198'}
{'loss': '0.8432', 'grad_norm': '0.4851', 'learning_rate': '8.075e-05', 'entropy': '0.845', 'num_tokens': '3.778e+07', 'mean_token_accuracy': '0.778', 'epoch': '0.06198'}
{'loss': '0.6215', 'grad_norm': '0.2992', 'learning_rate': '8.072e-05', 'entropy': '0.7307', 'num_tokens': '3.779e+07', 'mean_token_accuracy': '0.8059', 'epoch': '0.06199'}
{'loss': '0.6599', 'grad_norm': '0.3273', 'learning_rate': '8.069e-05', 'entropy': '0.7219', 'num_tokens': '3.779e+07', 'mean_token_accuracy': '0.8162', 'epoch': '0.062'}

 58%|█████▊    | 7007/12000 [4:08:26<3:44:31,  2.70s/it]
 58%|█████▊    | 7008/12000 [4:08:30<4:34:27,  3.30s/it]
                                                        

 58%|█████▊    | 7008/12000 [4:08:30<4:34:27,  3.30s/it]
 58%|█████▊    | 7009/12000 [4:08:34<4:42:33,  3.40s/it]
                                                        

 58%|█████▊    | 7009/12000 [4:08:34<4:42:33,  3.40s/it]
 58%|█████▊    | 7010/12000 [4:08:36<3:59:12,  2.88s/it]
                                                        

 58%|█████▊    | 7010/12000 [4:08:36<3:59:12,  2.88s/it]
 58%|█████▊    | 7011/12000 [4:08:37<3:32:27,  2.56s/it]
                                                        

 58%|█████▊    | 7011/12000 [4:08:37<3:32:27,  2.56s/it]
 58%|█████▊    | 7012/12000 [4:08:40<3:27:00,  2.49s/it]
                                                        

 58%|█████▊    | 7012/12000 [4:08:40<3:27:00,  2.49s/it]
 58%|█████▊    | 7013/12000 [4:08:41<3:04:16,  2.22s/it]
                                                        

 58%|█████▊    | 7013/12000 [4:08:41<3:04:16,  2.22s/it]
 58%|█████▊    | 7014/12000 [4:08:43<2:47:03,  2.01s/it]
                                                        

 58%|█████▊    | 7014/12000 [4:08:43<2:47:03,  2.01s/it]
 58%|█████▊    | 7015/12000 [4:08:45<2:45:56,  2.00s/it]
                                                        

 58%|█████▊    | 7015/12000 [4:08:45<2:45:56,  2.00s/it]
 58%|█████▊    | 7016/12000 [4:08:46<2:37:29,  1.90s/it]
                                                        

 58%|█████▊    | 7016/12000 [4:08:46<2:37:29,  1.90s/it]
 58%|█████▊    | 7017/12000 [4:08:48<2:36:49,  1.89s/it]
                                                        

 58%|█████▊    | 7017/12000 [4:08:48<2:36:49,  1.89s/it]
 58%|█████▊    | 7018/12000 [4:08:55<4:31:01,  3.26s/it]
                                                        

 58%|█████▊    | 7018/12000 [4:08:55<4:31:01,  3.26s/it]
 58%|█████▊    | 7019/12000 [4:08:57<4:10:57,  3.02s/it]
                                                        

 58%|█████▊    | 7019/12000 [4:08:57<4:10:57,  3.02s/it]
 58%|█████▊    | 7020/12000 [4:08:59<3:40:42,  2.66s/it]
                                                        

 58%|█████▊    | 7020/12000 [4:08:59<3:40:42,  2.66s/it]
 59%|█████▊    | 7021/12000 [4:09:01<3:21:36,  2.43s/it]
                                                        

 59%|█████▊    | 7021/12000 [4:09:01<3:21:36,  2.43s/it]
 59%|█████▊    | 7022/12000 [4:09:02<2:58:42,  2.15s/it]
                                                        

 59%|█████▊    | 7022/12000 [4:09:02<2:58:42,  2.15s/it]
 59%|█████▊    | 7023/12000 [4:09:05<3:15:00,  2.35s/it]
                                                        

 59%|█████▊    | 7023/12000 [4:09:05<3:15:00,  2.35s/it]
 59%|█████▊    | 7024/12000 [4:09:08<3:23:24,  2.45s/it]
                                                        

 59%|█████▊    | 7024/12000 [4:09:08<3:23:24,  2.45s/it]
 59%|█████▊    | 7025/12000 [4:09:14<5:05:16,  3.68s/it]
                                                        

 59%|█████▊    | 7025/12000 [4:09:14<5:05:16,  3.68s/it]
 59%|█████▊    | 7026/12000 [4:09:18<4:56:43,  3.58s/it]
                                                        

 59%|█████▊    | 7026/12000 [4:09:18<4:56:43,  3.58s/it]
 59%|█████▊    | 7027/12000 [4:09:20<4:34:17,  3.31s/it]
                                                        

 59%|█████▊    | 7027/12000 [4:09:20<4:34:17,  3.31s/it]
 59%|█████▊    | 7028/12000 [4:09:22<3:54:26,  2.83s/it]
                                                        

 59%|█████▊    | 7028/12000 [4:09:22<3:54:26,  2.83s/it]
 59%|█████▊    | 7029/12000 [4:09:25<3:49:30,  2.77s/it]
                                                        

 59%|█████▊    | 7029/12000 [4:09:25<3:49:30,  2.77s/it]
 59%|█████▊    | 7030/12000 [4:09:28<4:03:13,  2.94s/it]
                                                        

 59%|█████▊    | 7030/12000 [4:09:28<4:03:13,  2.94s/it]
 59%|█████▊    | 7031/12000 [4:09:31<4:09:48,  3.02s/it]
                                                        

 59%|█████▊    | 7031/12000 [4:09:31<4:09:48,  3.02s/it]
 59%|█████▊    | 7032/12000 [4:09:33<3:33:06,  2.57s/it]
                                                        

 59%|█████▊    | 7032/12000 [4:09:33<3:33:06,  2.57s/it]
 59%|█████▊    | 7033/12000 [4:09:35<3:20:20,  2.42s/it]
                                                        

 59%|█████▊    | 7033/12000 [4:09:35<3:20:20,  2.42s/it]
 59%|█████▊    | 7034/12000 [4:09:37<3:10:52,  2.31s/it]
                                                        

 59%|█████▊    | 7034/12000 [4:09:37<3:10:52,  2.31s/it]
 59%|█████▊    | 7035/12000 [4:09:40<3:20:42,  2.43s/it]
                                                        

 59%|█████▊    | 7035/12000 [4:09:40<3:20:42,  2.43s/it]
 59%|█████▊    | 7036/12000 [4:09:42<3:08:58,  2.28s/it]
                                                        

 59%|█████▊    | 7036/12000 [4:09:42<3:08:58,  2.28s/it]
 59%|█████▊    | 7037/12000 [4:09:44<3:21:14,  2.43s/it]
                                                        

 59%|█████▊    | 7037/12000 [4:09:44<3:21:14,  2.43s/it]
 59%|█████▊    | 7038/12000 [4:09:46<2:57:32,  2.15s/it]
                                                        

 59%|█████▊    | 7038/12000 [4:09:46<2:57:32,  2.15s/it]
 59%|█████▊    | 7039/12000 [4:09:48<2:45:57,  2.01s/it]
                                                        

 59%|█████▊    | 7039/12000 [4:09:48<2:45:57,  2.01s/it]
 59%|█████▊    | 7040/12000 [4:09:49<2:37:18,  1.90s/it]
                                                        

 59%|█████▊    | 7040/12000 [4:09:49<2:37:18,  1.90s/it]
 59%|█████▊    | 7041/12000 [4:09:51<2:38:02,  1.91s/it]
                                                        

 59%|█████▊    | 7041/12000 [4:09:51<2:38:02,  1.91s/it]
 59%|█████▊    | 7042/12000 [4:09:53<2:30:46,  1.82s/it]
                                                        

 59%|█████▊    | 7042/12000 [4:09:53<2:30:46,  1.82s/it]
 59%|█████▊    | 7043/12000 [4:09:54<2:26:56,  1.78s/it]
                                                        

 59%|█████▊    | 7043/12000 [4:09:54<2:26:56,  1.78s/it]
 59%|█████▊    | 7044/12000 [4:09:56<2:32:23,  1.84s/it]
                                                        

 59%|█████▊    | 7044/12000 [4:09:56<2:32:23,  1.84s/it]
 59%|█████▊    | 7045/12000 [4:09:58<2:30:25,  1.82s/it]
                                                        

 59%|█████▊    | 7045/12000 [4:09:58<2:30:25,  1.82s/it]
 59%|█████▊    | 7046/12000 [4:10:00<2:21:15,  1.71s/it]
                                                        

 59%|█████▊    | 7046/12000 [4:10:00<2:21:15,  1.71s/it]
 59%|█████▊    | 7047/12000 [4:10:01<2:15:17,  1.64s/it]
                                                        

 59%|█████▊    | 7047/12000 [4:10:01<2:15:17,  1.64s/it]
 59%|█████▊    | 7048/12000 [4:10:03<2:14:29,  1.63s/it]
                                                        

 59%|█████▊    | 7048/12000 [4:10:03<2:14:29,  1.63s/it]
 59%|█████▊    | 7049/12000 [4:10:04<2:15:21,  1.64s/it]
                                                        

 59%|█████▊    | 7049/12000 [4:10:04<2:15:21,  1.64s/it]
 59%|█████▉    | 7050/12000 [4:10:06<2:17:37,  1.67s/it]
                                                        

 59%|█████▉    | 7050/12000 [4:10:06<2:17:37,  1.67s/it]
 59%|█████▉    | 7051/12000 [4:10:09<2:45:58,  2.01s/it]
                                                        

 59%|█████▉    | 7051/12000 [4:10:09<2:45:58,  2.01s/it]
 59%|█████▉    | 7052/12000 [4:10:13<3:36:33,  2.63s/it]
                                                        

 59%|█████▉    | 7052/12000 [4:10:13<3:36:33,  2.63s/it]
 59%|█████▉    | 7053/12000 [4:10:15<3:09:29,  2.30s/it]
                                                        

 59%|█████▉    | 7053/12000 [4:10:15<3:09:29,  2.30s/it]
 59%|█████▉    | 7054/12000 [4:10:16<2:49:36,  2.06s/it]
                                                        
{'loss': '1.338', 'grad_norm': '0.32', 'learning_rate': '8.067e-05', 'entropy': '1.355', 'num_tokens': '3.78e+07', 'mean_token_accuracy': '0.6862', 'epoch': '0.06201'}
{'loss': '0.7688', 'grad_norm': '0.3437', 'learning_rate': '8.064e-05', 'entropy': '0.7979', 'num_tokens': '3.781e+07', 'mean_token_accuracy': '0.7944', 'epoch': '0.06202'}
{'loss': '0.668', 'grad_norm': '0.2942', 'learning_rate': '8.061e-05', 'entropy': '0.7152', 'num_tokens': '3.782e+07', 'mean_token_accuracy': '0.8292', 'epoch': '0.06203'}
{'loss': '0.9774', 'grad_norm': '0.4513', 'learning_rate': '8.059e-05', 'entropy': '0.9189', 'num_tokens': '3.782e+07', 'mean_token_accuracy': '0.7848', 'epoch': '0.06204'}
{'loss': '0.8417', 'grad_norm': '0.3942', 'learning_rate': '8.056e-05', 'entropy': '0.7341', 'num_tokens': '3.783e+07', 'mean_token_accuracy': '0.8102', 'epoch': '0.06205'}
{'loss': '0.4594', 'grad_norm': '0.2677', 'learning_rate': '8.053e-05', 'entropy': '0.4889', 'num_tokens': '3.783e+07', 'mean_token_accuracy': '0.8786', 'epoch': '0.06206'}
{'loss': '0.5916', 'grad_norm': '0.2881', 'learning_rate': '8.05e-05', 'entropy': '0.6873', 'num_tokens': '3.784e+07', 'mean_token_accuracy': '0.824', 'epoch': '0.06206'}
{'loss': '0.8535', 'grad_norm': '0.3721', 'learning_rate': '8.048e-05', 'entropy': '0.8872', 'num_tokens': '3.784e+07', 'mean_token_accuracy': '0.7745', 'epoch': '0.06207'}
{'loss': '1.137', 'grad_norm': '0.3792', 'learning_rate': '8.045e-05', 'entropy': '1.062', 'num_tokens': '3.785e+07', 'mean_token_accuracy': '0.7506', 'epoch': '0.06208'}
{'loss': '0.5405', 'grad_norm': '0.2982', 'learning_rate': '8.042e-05', 'entropy': '0.5271', 'num_tokens': '3.785e+07', 'mean_token_accuracy': '0.8616', 'epoch': '0.06209'}
{'loss': '0.8759', 'grad_norm': '0.3952', 'learning_rate': '8.04e-05', 'entropy': '0.8675', 'num_tokens': '3.786e+07', 'mean_token_accuracy': '0.7803', 'epoch': '0.0621'}
{'loss': '0.6334', 'grad_norm': '0.2574', 'learning_rate': '8.037e-05', 'entropy': '0.6179', 'num_tokens': '3.787e+07', 'mean_token_accuracy': '0.8331', 'epoch': '0.06211'}
{'loss': '0.7117', 'grad_norm': '0.3557', 'learning_rate': '8.034e-05', 'entropy': '0.763', 'num_tokens': '3.788e+07', 'mean_token_accuracy': '0.8066', 'epoch': '0.06212'}
{'loss': '0.8929', 'grad_norm': '0.3985', 'learning_rate': '8.032e-05', 'entropy': '0.8688', 'num_tokens': '3.788e+07', 'mean_token_accuracy': '0.7853', 'epoch': '0.06213'}
{'loss': '0.6792', 'grad_norm': '0.3693', 'learning_rate': '8.029e-05', 'entropy': '0.7482', 'num_tokens': '3.789e+07', 'mean_token_accuracy': '0.8047', 'epoch': '0.06213'}
{'loss': '0.5543', 'grad_norm': '0.3628', 'learning_rate': '8.026e-05', 'entropy': '0.5836', 'num_tokens': '3.789e+07', 'mean_token_accuracy': '0.8406', 'epoch': '0.06214'}
{'loss': '0.5053', 'grad_norm': '0.3262', 'learning_rate': '8.023e-05', 'entropy': '0.5785', 'num_tokens': '3.79e+07', 'mean_token_accuracy': '0.854', 'epoch': '0.06215'}
{'loss': '0.8714', 'grad_norm': '0.4444', 'learning_rate': '8.021e-05', 'entropy': '0.7881', 'num_tokens': '3.79e+07', 'mean_token_accuracy': '0.7882', 'epoch': '0.06216'}
{'loss': '0.9384', 'grad_norm': '0.2972', 'learning_rate': '8.018e-05', 'entropy': '0.9475', 'num_tokens': '3.792e+07', 'mean_token_accuracy': '0.7544', 'epoch': '0.06217'}
{'loss': '1.231', 'grad_norm': '0.4879', 'learning_rate': '8.015e-05', 'entropy': '1.109', 'num_tokens': '3.792e+07', 'mean_token_accuracy': '0.7322', 'epoch': '0.06218'}
{'loss': '0.6621', 'grad_norm': '0.3249', 'learning_rate': '8.013e-05', 'entropy': '0.6531', 'num_tokens': '3.793e+07', 'mean_token_accuracy': '0.8218', 'epoch': '0.06219'}
{'loss': '0.9161', 'grad_norm': '0.451', 'learning_rate': '8.01e-05', 'entropy': '0.8579', 'num_tokens': '3.794e+07', 'mean_token_accuracy': '0.7866', 'epoch': '0.0622'}
{'loss': '0.7895', 'grad_norm': '0.3802', 'learning_rate': '8.007e-05', 'entropy': '0.8107', 'num_tokens': '3.794e+07', 'mean_token_accuracy': '0.7894', 'epoch': '0.06221'}
{'loss': '0.4165', 'grad_norm': '0.3149', 'learning_rate': '8.005e-05', 'entropy': '0.4913', 'num_tokens': '3.795e+07', 'mean_token_accuracy': '0.8636', 'epoch': '0.06221'}
{'loss': '1.277', 'grad_norm': '0.3953', 'learning_rate': '8.002e-05', 'entropy': '1.262', 'num_tokens': '3.795e+07', 'mean_token_accuracy': '0.7049', 'epoch': '0.06222'}
{'loss': '1.005', 'grad_norm': '0.3819', 'learning_rate': '7.999e-05', 'entropy': '1.035', 'num_tokens': '3.796e+07', 'mean_token_accuracy': '0.7347', 'epoch': '0.06223'}
{'loss': '0.8431', 'grad_norm': '0.4223', 'learning_rate': '7.996e-05', 'entropy': '0.8374', 'num_tokens': '3.796e+07', 'mean_token_accuracy': '0.8022', 'epoch': '0.06224'}
{'loss': '0.9884', 'grad_norm': '0.4494', 'learning_rate': '7.994e-05', 'entropy': '1.007', 'num_tokens': '3.797e+07', 'mean_token_accuracy': '0.7554', 'epoch': '0.06225'}
{'loss': '0.7299', 'grad_norm': '0.3259', 'learning_rate': '7.991e-05', 'entropy': '0.7265', 'num_tokens': '3.798e+07', 'mean_token_accuracy': '0.8147', 'epoch': '0.06226'}
{'loss': '0.8212', 'grad_norm': '0.4124', 'learning_rate': '7.988e-05', 'entropy': '0.7718', 'num_tokens': '3.798e+07', 'mean_token_accuracy': '0.806', 'epoch': '0.06227'}
{'loss': '0.585', 'grad_norm': '0.3449', 'learning_rate': '7.986e-05', 'entropy': '0.6174', 'num_tokens': '3.799e+07', 'mean_token_accuracy': '0.8317', 'epoch': '0.06228'}
{'loss': '0.7591', 'grad_norm': '0.3808', 'learning_rate': '7.983e-05', 'entropy': '0.7157', 'num_tokens': '3.8e+07', 'mean_token_accuracy': '0.7993', 'epoch': '0.06229'}
{'loss': '0.8709', 'grad_norm': '0.4009', 'learning_rate': '7.98e-05', 'entropy': '0.8941', 'num_tokens': '3.8e+07', 'mean_token_accuracy': '0.7782', 'epoch': '0.06229'}
{'loss': '1.005', 'grad_norm': '0.4888', 'learning_rate': '7.978e-05', 'entropy': '1.003', 'num_tokens': '3.801e+07', 'mean_token_accuracy': '0.7574', 'epoch': '0.0623'}
{'loss': '0.7371', 'grad_norm': '0.3307', 'learning_rate': '7.975e-05', 'entropy': '0.7424', 'num_tokens': '3.801e+07', 'mean_token_accuracy': '0.8078', 'epoch': '0.06231'}
{'loss': '0.7591', 'grad_norm': '0.4112', 'learning_rate': '7.972e-05', 'entropy': '0.8108', 'num_tokens': '3.802e+07', 'mean_token_accuracy': '0.7852', 'epoch': '0.06232'}
{'loss': '0.4805', 'grad_norm': '0.3189', 'learning_rate': '7.969e-05', 'entropy': '0.5165', 'num_tokens': '3.802e+07', 'mean_token_accuracy': '0.8621', 'epoch': '0.06233'}
{'loss': '0.9121', 'grad_norm': '0.5274', 'learning_rate': '7.967e-05', 'entropy': '0.9061', 'num_tokens': '3.803e+07', 'mean_token_accuracy': '0.7751', 'epoch': '0.06234'}
{'loss': '0.6888', 'grad_norm': '0.4645', 'learning_rate': '7.964e-05', 'entropy': '0.6767', 'num_tokens': '3.803e+07', 'mean_token_accuracy': '0.8167', 'epoch': '0.06235'}
{'loss': '1.031', 'grad_norm': '0.3965', 'learning_rate': '7.961e-05', 'entropy': '1.079', 'num_tokens': '3.804e+07', 'mean_token_accuracy': '0.7532', 'epoch': '0.06236'}
{'loss': '0.6658', 'grad_norm': '0.363', 'learning_rate': '7.959e-05', 'entropy': '0.7192', 'num_tokens': '3.804e+07', 'mean_token_accuracy': '0.818', 'epoch': '0.06236'}
{'loss': '0.5972', 'grad_norm': '0.2949', 'learning_rate': '7.956e-05', 'entropy': '0.6472', 'num_tokens': '3.805e+07', 'mean_token_accuracy': '0.8378', 'epoch': '0.06237'}
{'loss': '0.9881', 'grad_norm': '0.3625', 'learning_rate': '7.953e-05', 'entropy': '0.8768', 'num_tokens': '3.805e+07', 'mean_token_accuracy': '0.7751', 'epoch': '0.06238'}
{'loss': '0.889', 'grad_norm': '0.4516', 'learning_rate': '7.951e-05', 'entropy': '0.83', 'num_tokens': '3.806e+07', 'mean_token_accuracy': '0.7992', 'epoch': '0.06239'}
{'loss': '0.7112', 'grad_norm': '0.4135', 'learning_rate': '7.948e-05', 'entropy': '0.7993', 'num_tokens': '3.806e+07', 'mean_token_accuracy': '0.8037', 'epoch': '0.0624'}
{'loss': '0.99', 'grad_norm': '0.4091', 'learning_rate': '7.945e-05', 'entropy': '0.9059', 'num_tokens': '3.807e+07', 'mean_token_accuracy': '0.7667', 'epoch': '0.06241'}
{'loss': '1.162', 'grad_norm': '0.4312', 'learning_rate': '7.942e-05', 'entropy': '1.21', 'num_tokens': '3.807e+07', 'mean_token_accuracy': '0.7179', 'epoch': '0.06242'}

 59%|█████▉    | 7054/12000 [4:10:16<2:49:36,  2.06s/it]
 59%|█████▉    | 7055/12000 [4:10:19<3:05:32,  2.25s/it]
                                                        

 59%|█████▉    | 7055/12000 [4:10:19<3:05:32,  2.25s/it]
 59%|█████▉    | 7056/12000 [4:10:21<3:05:41,  2.25s/it]
                                                        

 59%|█████▉    | 7056/12000 [4:10:21<3:05:41,  2.25s/it]
 59%|█████▉    | 7057/12000 [4:10:23<2:56:31,  2.14s/it]
                                                        

 59%|█████▉    | 7057/12000 [4:10:23<2:56:31,  2.14s/it]
 59%|█████▉    | 7058/12000 [4:10:25<2:43:35,  1.99s/it]
                                                        

 59%|█████▉    | 7058/12000 [4:10:25<2:43:35,  1.99s/it]
 59%|█████▉    | 7059/12000 [4:10:26<2:37:26,  1.91s/it]
                                                        

 59%|█████▉    | 7059/12000 [4:10:26<2:37:26,  1.91s/it]
 59%|█████▉    | 7060/12000 [4:10:28<2:37:03,  1.91s/it]
                                                        

 59%|█████▉    | 7060/12000 [4:10:28<2:37:03,  1.91s/it]
 59%|█████▉    | 7061/12000 [4:10:31<2:49:44,  2.06s/it]
                                                        

 59%|█████▉    | 7061/12000 [4:10:31<2:49:44,  2.06s/it]
 59%|█████▉    | 7062/12000 [4:10:32<2:39:14,  1.93s/it]
                                                        

 59%|█████▉    | 7062/12000 [4:10:32<2:39:14,  1.93s/it]
 59%|█████▉    | 7063/12000 [4:10:35<2:57:08,  2.15s/it]
                                                        

 59%|█████▉    | 7063/12000 [4:10:35<2:57:08,  2.15s/it]
 59%|█████▉    | 7064/12000 [4:10:37<2:45:16,  2.01s/it]
                                                        

 59%|█████▉    | 7064/12000 [4:10:37<2:45:16,  2.01s/it]
 59%|█████▉    | 7065/12000 [4:10:38<2:35:40,  1.89s/it]
                                                        

 59%|█████▉    | 7065/12000 [4:10:38<2:35:40,  1.89s/it]
 59%|█████▉    | 7066/12000 [4:10:40<2:23:16,  1.74s/it]
                                                        

 59%|█████▉    | 7066/12000 [4:10:40<2:23:16,  1.74s/it]
 59%|█████▉    | 7067/12000 [4:10:41<2:17:58,  1.68s/it]
                                                        

 59%|█████▉    | 7067/12000 [4:10:41<2:17:58,  1.68s/it]
 59%|█████▉    | 7068/12000 [4:10:43<2:27:27,  1.79s/it]
                                                        

 59%|█████▉    | 7068/12000 [4:10:43<2:27:27,  1.79s/it]
 59%|█████▉    | 7069/12000 [4:10:45<2:20:35,  1.71s/it]
                                                        

 59%|█████▉    | 7069/12000 [4:10:45<2:20:35,  1.71s/it]
 59%|█████▉    | 7070/12000 [4:10:47<2:35:16,  1.89s/it]
                                                        

 59%|█████▉    | 7070/12000 [4:10:47<2:35:16,  1.89s/it]
 59%|█████▉    | 7071/12000 [4:10:49<2:26:09,  1.78s/it]
                                                        

 59%|█████▉    | 7071/12000 [4:10:49<2:26:09,  1.78s/it]
 59%|█████▉    | 7072/12000 [4:10:53<3:40:33,  2.69s/it]
                                                        

 59%|█████▉    | 7072/12000 [4:10:53<3:40:33,  2.69s/it]
 59%|█████▉    | 7073/12000 [4:10:55<3:09:32,  2.31s/it]
                                                        

 59%|█████▉    | 7073/12000 [4:10:55<3:09:32,  2.31s/it]
 59%|█████▉    | 7074/12000 [4:10:59<3:54:17,  2.85s/it]
                                                        

 59%|█████▉    | 7074/12000 [4:10:59<3:54:17,  2.85s/it]
 59%|█████▉    | 7075/12000 [4:11:02<3:49:10,  2.79s/it]
                                                        

 59%|█████▉    | 7075/12000 [4:11:02<3:49:10,  2.79s/it]
 59%|█████▉    | 7076/12000 [4:11:03<3:25:52,  2.51s/it]
                                                        

 59%|█████▉    | 7076/12000 [4:11:03<3:25:52,  2.51s/it]
 59%|█████▉    | 7077/12000 [4:11:06<3:22:46,  2.47s/it]
                                                        

 59%|█████▉    | 7077/12000 [4:11:06<3:22:46,  2.47s/it]
 59%|█████▉    | 7078/12000 [4:11:08<3:28:06,  2.54s/it]
                                                        

 59%|█████▉    | 7078/12000 [4:11:08<3:28:06,  2.54s/it]
 59%|█████▉    | 7079/12000 [4:11:10<3:10:35,  2.32s/it]
                                                        

 59%|█████▉    | 7079/12000 [4:11:10<3:10:35,  2.32s/it]
 59%|█████▉    | 7080/12000 [4:11:13<3:11:15,  2.33s/it]
                                                        

 59%|█████▉    | 7080/12000 [4:11:13<3:11:15,  2.33s/it]
 59%|█████▉    | 7081/12000 [4:11:15<2:59:40,  2.19s/it]
                                                        

 59%|█████▉    | 7081/12000 [4:11:15<2:59:40,  2.19s/it]
 59%|█████▉    | 7082/12000 [4:11:19<3:59:27,  2.92s/it]
                                                        

 59%|█████▉    | 7082/12000 [4:11:19<3:59:27,  2.92s/it]
 59%|█████▉    | 7083/12000 [4:11:21<3:27:45,  2.54s/it]
                                                        

 59%|█████▉    | 7083/12000 [4:11:21<3:27:45,  2.54s/it]
 59%|█████▉    | 7084/12000 [4:11:23<3:14:42,  2.38s/it]
                                                        

 59%|█████▉    | 7084/12000 [4:11:23<3:14:42,  2.38s/it]
 59%|█████▉    | 7085/12000 [4:11:24<2:58:06,  2.17s/it]
                                                        

 59%|█████▉    | 7085/12000 [4:11:24<2:58:06,  2.17s/it]
 59%|█████▉    | 7086/12000 [4:11:27<2:59:08,  2.19s/it]
                                                        

 59%|█████▉    | 7086/12000 [4:11:27<2:59:08,  2.19s/it]
 59%|█████▉    | 7087/12000 [4:11:29<2:55:48,  2.15s/it]
                                                        

 59%|█████▉    | 7087/12000 [4:11:29<2:55:48,  2.15s/it]
 59%|█████▉    | 7088/12000 [4:11:30<2:38:09,  1.93s/it]
                                                        

 59%|█████▉    | 7088/12000 [4:11:30<2:38:09,  1.93s/it]
 59%|█████▉    | 7089/12000 [4:11:32<2:46:40,  2.04s/it]
                                                        

 59%|█████▉    | 7089/12000 [4:11:32<2:46:40,  2.04s/it]
 59%|█████▉    | 7090/12000 [4:11:34<2:35:24,  1.90s/it]
                                                        

 59%|█████▉    | 7090/12000 [4:11:34<2:35:24,  1.90s/it]
 59%|█████▉    | 7091/12000 [4:11:35<2:23:27,  1.75s/it]
                                                        

 59%|█████▉    | 7091/12000 [4:11:35<2:23:27,  1.75s/it]
 59%|█████▉    | 7092/12000 [4:11:38<2:38:51,  1.94s/it]
                                                        

 59%|█████▉    | 7092/12000 [4:11:38<2:38:51,  1.94s/it]
 59%|█████▉    | 7093/12000 [4:11:39<2:29:50,  1.83s/it]
                                                        

 59%|█████▉    | 7093/12000 [4:11:39<2:29:50,  1.83s/it]
 59%|█████▉    | 7094/12000 [4:11:41<2:23:34,  1.76s/it]
                                                        

 59%|█████▉    | 7094/12000 [4:11:41<2:23:34,  1.76s/it]
 59%|█████▉    | 7095/12000 [4:11:43<2:17:41,  1.68s/it]
                                                        

 59%|█████▉    | 7095/12000 [4:11:43<2:17:41,  1.68s/it]
 59%|█████▉    | 7096/12000 [4:11:46<3:10:17,  2.33s/it]
                                                        

 59%|█████▉    | 7096/12000 [4:11:46<3:10:17,  2.33s/it]
 59%|█████▉    | 7097/12000 [4:11:48<3:04:41,  2.26s/it]
                                                        

 59%|█████▉    | 7097/12000 [4:11:48<3:04:41,  2.26s/it]
 59%|█████▉    | 7098/12000 [4:11:51<3:09:45,  2.32s/it]
                                                        

 59%|█████▉    | 7098/12000 [4:11:51<3:09:45,  2.32s/it]
 59%|█████▉    | 7099/12000 [4:11:52<2:51:10,  2.10s/it]
                                                        

 59%|█████▉    | 7099/12000 [4:11:52<2:51:10,  2.10s/it]
 59%|█████▉    | 7100/12000 [4:11:55<2:52:44,  2.12s/it]
                                                        

 59%|█████▉    | 7100/12000 [4:11:55<2:52:44,  2.12s/it]
 59%|█████▉    | 7101/12000 [4:11:59<3:35:52,  2.64s/it]
                                                        
{'loss': '0.8646', 'grad_norm': '0.3227', 'learning_rate': '7.94e-05', 'entropy': '0.8666', 'num_tokens': '3.808e+07', 'mean_token_accuracy': '0.7844', 'epoch': '0.06243'}
{'loss': '0.6288', 'grad_norm': '0.3324', 'learning_rate': '7.937e-05', 'entropy': '0.7496', 'num_tokens': '3.808e+07', 'mean_token_accuracy': '0.8145', 'epoch': '0.06244'}
{'loss': '0.8772', 'grad_norm': '0.4066', 'learning_rate': '7.934e-05', 'entropy': '0.897', 'num_tokens': '3.809e+07', 'mean_token_accuracy': '0.7684', 'epoch': '0.06244'}
{'loss': '0.8206', 'grad_norm': '0.4125', 'learning_rate': '7.932e-05', 'entropy': '0.8362', 'num_tokens': '3.809e+07', 'mean_token_accuracy': '0.7849', 'epoch': '0.06245'}
{'loss': '0.9701', 'grad_norm': '0.391', 'learning_rate': '7.929e-05', 'entropy': '0.9587', 'num_tokens': '3.81e+07', 'mean_token_accuracy': '0.7615', 'epoch': '0.06246'}
{'loss': '0.7994', 'grad_norm': '0.413', 'learning_rate': '7.926e-05', 'entropy': '0.8158', 'num_tokens': '3.81e+07', 'mean_token_accuracy': '0.7976', 'epoch': '0.06247'}
{'loss': '0.6297', 'grad_norm': '0.3405', 'learning_rate': '7.924e-05', 'entropy': '0.653', 'num_tokens': '3.811e+07', 'mean_token_accuracy': '0.8382', 'epoch': '0.06248'}
{'loss': '0.711', 'grad_norm': '0.3025', 'learning_rate': '7.921e-05', 'entropy': '0.7776', 'num_tokens': '3.811e+07', 'mean_token_accuracy': '0.805', 'epoch': '0.06249'}
{'loss': '0.6296', 'grad_norm': '0.3785', 'learning_rate': '7.918e-05', 'entropy': '0.6156', 'num_tokens': '3.812e+07', 'mean_token_accuracy': '0.8379', 'epoch': '0.0625'}
{'loss': '1.082', 'grad_norm': '0.4287', 'learning_rate': '7.915e-05', 'entropy': '0.9647', 'num_tokens': '3.813e+07', 'mean_token_accuracy': '0.7581', 'epoch': '0.06251'}
{'loss': '0.9378', 'grad_norm': '0.4277', 'learning_rate': '7.913e-05', 'entropy': '0.8867', 'num_tokens': '3.813e+07', 'mean_token_accuracy': '0.7744', 'epoch': '0.06252'}
{'loss': '0.8425', 'grad_norm': '0.4104', 'learning_rate': '7.91e-05', 'entropy': '0.8866', 'num_tokens': '3.814e+07', 'mean_token_accuracy': '0.777', 'epoch': '0.06252'}
{'loss': '0.5843', 'grad_norm': '0.7674', 'learning_rate': '7.907e-05', 'entropy': '0.6214', 'num_tokens': '3.814e+07', 'mean_token_accuracy': '0.8361', 'epoch': '0.06253'}
{'loss': '0.5314', 'grad_norm': '0.3542', 'learning_rate': '7.905e-05', 'entropy': '0.5586', 'num_tokens': '3.814e+07', 'mean_token_accuracy': '0.8492', 'epoch': '0.06254'}
{'loss': '0.9922', 'grad_norm': '0.433', 'learning_rate': '7.902e-05', 'entropy': '0.9306', 'num_tokens': '3.815e+07', 'mean_token_accuracy': '0.7602', 'epoch': '0.06255'}
{'loss': '0.6416', 'grad_norm': '0.3525', 'learning_rate': '7.899e-05', 'entropy': '0.6849', 'num_tokens': '3.816e+07', 'mean_token_accuracy': '0.8208', 'epoch': '0.06256'}
{'loss': '0.7826', 'grad_norm': '0.4536', 'learning_rate': '7.897e-05', 'entropy': '0.6888', 'num_tokens': '3.816e+07', 'mean_token_accuracy': '0.8298', 'epoch': '0.06257'}
{'loss': '0.6473', 'grad_norm': '0.3675', 'learning_rate': '7.894e-05', 'entropy': '0.7191', 'num_tokens': '3.817e+07', 'mean_token_accuracy': '0.8105', 'epoch': '0.06258'}
{'loss': '0.7272', 'grad_norm': '0.295', 'learning_rate': '7.891e-05', 'entropy': '0.7044', 'num_tokens': '3.817e+07', 'mean_token_accuracy': '0.8044', 'epoch': '0.06259'}
{'loss': '0.6047', 'grad_norm': '0.3431', 'learning_rate': '7.889e-05', 'entropy': '0.6408', 'num_tokens': '3.818e+07', 'mean_token_accuracy': '0.8375', 'epoch': '0.06259'}
{'loss': '0.7992', 'grad_norm': '0.3707', 'learning_rate': '7.886e-05', 'entropy': '0.8784', 'num_tokens': '3.818e+07', 'mean_token_accuracy': '0.7885', 'epoch': '0.0626'}
{'loss': '0.7707', 'grad_norm': '0.3609', 'learning_rate': '7.883e-05', 'entropy': '0.7199', 'num_tokens': '3.819e+07', 'mean_token_accuracy': '0.8158', 'epoch': '0.06261'}
{'loss': '0.6973', 'grad_norm': '0.4037', 'learning_rate': '7.88e-05', 'entropy': '0.863', 'num_tokens': '3.82e+07', 'mean_token_accuracy': '0.7794', 'epoch': '0.06262'}
{'loss': '0.675', 'grad_norm': '0.2803', 'learning_rate': '7.878e-05', 'entropy': '0.6373', 'num_tokens': '3.82e+07', 'mean_token_accuracy': '0.8264', 'epoch': '0.06263'}
{'loss': '0.6762', 'grad_norm': '0.4625', 'learning_rate': '7.875e-05', 'entropy': '0.6497', 'num_tokens': '3.821e+07', 'mean_token_accuracy': '0.8321', 'epoch': '0.06264'}
{'loss': '0.8897', 'grad_norm': '0.4623', 'learning_rate': '7.872e-05', 'entropy': '0.8959', 'num_tokens': '3.821e+07', 'mean_token_accuracy': '0.7814', 'epoch': '0.06265'}
{'loss': '0.9043', 'grad_norm': '0.5761', 'learning_rate': '7.87e-05', 'entropy': '0.8617', 'num_tokens': '3.822e+07', 'mean_token_accuracy': '0.7787', 'epoch': '0.06266'}
{'loss': '0.9005', 'grad_norm': '0.3976', 'learning_rate': '7.867e-05', 'entropy': '0.9348', 'num_tokens': '3.822e+07', 'mean_token_accuracy': '0.7886', 'epoch': '0.06267'}
{'loss': '1.142', 'grad_norm': '0.3841', 'learning_rate': '7.864e-05', 'entropy': '1.006', 'num_tokens': '3.823e+07', 'mean_token_accuracy': '0.7661', 'epoch': '0.06267'}
{'loss': '0.9483', 'grad_norm': '0.4236', 'learning_rate': '7.862e-05', 'entropy': '0.9516', 'num_tokens': '3.824e+07', 'mean_token_accuracy': '0.748', 'epoch': '0.06268'}
{'loss': '0.7359', 'grad_norm': '0.4194', 'learning_rate': '7.859e-05', 'entropy': '0.8958', 'num_tokens': '3.824e+07', 'mean_token_accuracy': '0.7845', 'epoch': '0.06269'}
{'loss': '0.7167', 'grad_norm': '0.4071', 'learning_rate': '7.856e-05', 'entropy': '0.7437', 'num_tokens': '3.825e+07', 'mean_token_accuracy': '0.8008', 'epoch': '0.0627'}
{'loss': '0.6578', 'grad_norm': '0.3684', 'learning_rate': '7.854e-05', 'entropy': '0.8018', 'num_tokens': '3.825e+07', 'mean_token_accuracy': '0.7803', 'epoch': '0.06271'}
{'loss': '0.6583', 'grad_norm': '0.3046', 'learning_rate': '7.851e-05', 'entropy': '0.6828', 'num_tokens': '3.826e+07', 'mean_token_accuracy': '0.822', 'epoch': '0.06272'}
{'loss': '0.5358', 'grad_norm': '0.3763', 'learning_rate': '7.848e-05', 'entropy': '0.613', 'num_tokens': '3.826e+07', 'mean_token_accuracy': '0.8413', 'epoch': '0.06273'}
{'loss': '0.8568', 'grad_norm': '0.3488', 'learning_rate': '7.845e-05', 'entropy': '0.9936', 'num_tokens': '3.827e+07', 'mean_token_accuracy': '0.7522', 'epoch': '0.06274'}
{'loss': '0.6605', 'grad_norm': '0.3224', 'learning_rate': '7.843e-05', 'entropy': '0.6302', 'num_tokens': '3.827e+07', 'mean_token_accuracy': '0.8331', 'epoch': '0.06275'}
{'loss': '0.7394', 'grad_norm': '0.3677', 'learning_rate': '7.84e-05', 'entropy': '0.7315', 'num_tokens': '3.828e+07', 'mean_token_accuracy': '0.8126', 'epoch': '0.06275'}
{'loss': '1.144', 'grad_norm': '0.5147', 'learning_rate': '7.837e-05', 'entropy': '1.062', 'num_tokens': '3.828e+07', 'mean_token_accuracy': '0.7303', 'epoch': '0.06276'}
{'loss': '0.8182', 'grad_norm': '0.4515', 'learning_rate': '7.835e-05', 'entropy': '0.8199', 'num_tokens': '3.829e+07', 'mean_token_accuracy': '0.7789', 'epoch': '0.06277'}
{'loss': '0.7504', 'grad_norm': '0.4316', 'learning_rate': '7.832e-05', 'entropy': '0.7921', 'num_tokens': '3.829e+07', 'mean_token_accuracy': '0.8089', 'epoch': '0.06278'}
{'loss': '0.6487', 'grad_norm': '0.3659', 'learning_rate': '7.829e-05', 'entropy': '0.6607', 'num_tokens': '3.83e+07', 'mean_token_accuracy': '0.8397', 'epoch': '0.06279'}
{'loss': '0.7429', 'grad_norm': '0.4128', 'learning_rate': '7.827e-05', 'entropy': '0.7214', 'num_tokens': '3.83e+07', 'mean_token_accuracy': '0.8094', 'epoch': '0.0628'}
{'loss': '0.9206', 'grad_norm': '0.426', 'learning_rate': '7.824e-05', 'entropy': '0.9986', 'num_tokens': '3.831e+07', 'mean_token_accuracy': '0.7508', 'epoch': '0.06281'}
{'loss': '1.416', 'grad_norm': '0.4751', 'learning_rate': '7.821e-05', 'entropy': '1.218', 'num_tokens': '3.831e+07', 'mean_token_accuracy': '0.7097', 'epoch': '0.06282'}
{'loss': '0.8212', 'grad_norm': '0.4069', 'learning_rate': '7.819e-05', 'entropy': '0.8339', 'num_tokens': '3.832e+07', 'mean_token_accuracy': '0.7934', 'epoch': '0.06282'}
{'loss': '1.132', 'grad_norm': '0.386', 'learning_rate': '7.816e-05', 'entropy': '1.048', 'num_tokens': '3.832e+07', 'mean_token_accuracy': '0.7478', 'epoch': '0.06283'}

 59%|█████▉    | 7101/12000 [4:11:59<3:35:52,  2.64s/it]
 59%|█████▉    | 7102/12000 [4:12:00<3:11:10,  2.34s/it]
                                                        

 59%|█████▉    | 7102/12000 [4:12:00<3:11:10,  2.34s/it]
 59%|█████▉    | 7103/12000 [4:12:03<3:18:05,  2.43s/it]
                                                        

 59%|█████▉    | 7103/12000 [4:12:03<3:18:05,  2.43s/it]
 59%|█████▉    | 7104/12000 [4:12:05<3:13:59,  2.38s/it]
                                                        

 59%|█████▉    | 7104/12000 [4:12:05<3:13:59,  2.38s/it]
 59%|█████▉    | 7105/12000 [4:12:07<3:15:50,  2.40s/it]
                                                        

 59%|█████▉    | 7105/12000 [4:12:07<3:15:50,  2.40s/it]
 59%|█████▉    | 7106/12000 [4:12:09<2:53:45,  2.13s/it]
                                                        

 59%|█████▉    | 7106/12000 [4:12:09<2:53:45,  2.13s/it]
 59%|█████▉    | 7107/12000 [4:12:10<2:38:32,  1.94s/it]
                                                        

 59%|█████▉    | 7107/12000 [4:12:10<2:38:32,  1.94s/it]
 59%|█████▉    | 7108/12000 [4:12:13<2:42:34,  1.99s/it]
                                                        

 59%|█████▉    | 7108/12000 [4:12:13<2:42:34,  1.99s/it]
 59%|█████▉    | 7109/12000 [4:12:14<2:34:55,  1.90s/it]
                                                        

 59%|█████▉    | 7109/12000 [4:12:14<2:34:55,  1.90s/it]
 59%|█████▉    | 7110/12000 [4:12:16<2:32:14,  1.87s/it]
                                                        

 59%|█████▉    | 7110/12000 [4:12:16<2:32:14,  1.87s/it]
 59%|█████▉    | 7111/12000 [4:12:18<2:24:01,  1.77s/it]
                                                        

 59%|█████▉    | 7111/12000 [4:12:18<2:24:01,  1.77s/it]
 59%|█████▉    | 7112/12000 [4:12:19<2:20:03,  1.72s/it]
                                                        

 59%|█████▉    | 7112/12000 [4:12:19<2:20:03,  1.72s/it]
 59%|█████▉    | 7113/12000 [4:12:22<2:43:51,  2.01s/it]
                                                        

 59%|█████▉    | 7113/12000 [4:12:22<2:43:51,  2.01s/it]
 59%|█████▉    | 7114/12000 [4:12:23<2:31:29,  1.86s/it]
                                                        

 59%|█████▉    | 7114/12000 [4:12:23<2:31:29,  1.86s/it]
 59%|█████▉    | 7115/12000 [4:12:25<2:28:20,  1.82s/it]
                                                        

 59%|█████▉    | 7115/12000 [4:12:25<2:28:20,  1.82s/it]
 59%|█████▉    | 7116/12000 [4:12:27<2:21:08,  1.73s/it]
                                                        

 59%|█████▉    | 7116/12000 [4:12:27<2:21:08,  1.73s/it]
 59%|█████▉    | 7117/12000 [4:12:28<2:21:23,  1.74s/it]
                                                        

 59%|█████▉    | 7117/12000 [4:12:28<2:21:23,  1.74s/it]
 59%|█████▉    | 7118/12000 [4:12:32<3:02:20,  2.24s/it]
                                                        

 59%|█████▉    | 7118/12000 [4:12:32<3:02:20,  2.24s/it]
 59%|█████▉    | 7119/12000 [4:12:34<2:50:29,  2.10s/it]
                                                        

 59%|█████▉    | 7119/12000 [4:12:34<2:50:29,  2.10s/it]
 59%|█████▉    | 7120/12000 [4:12:35<2:44:52,  2.03s/it]
                                                        

 59%|█████▉    | 7120/12000 [4:12:35<2:44:52,  2.03s/it]
 59%|█████▉    | 7121/12000 [4:12:37<2:29:35,  1.84s/it]
                                                        

 59%|█████▉    | 7121/12000 [4:12:37<2:29:35,  1.84s/it]
 59%|█████▉    | 7122/12000 [4:12:38<2:19:23,  1.71s/it]
                                                        

 59%|█████▉    | 7122/12000 [4:12:38<2:19:23,  1.71s/it]
 59%|█████▉    | 7123/12000 [4:12:41<2:44:08,  2.02s/it]
                                                        

 59%|█████▉    | 7123/12000 [4:12:41<2:44:08,  2.02s/it]
 59%|█████▉    | 7124/12000 [4:12:43<2:35:36,  1.91s/it]
                                                        

 59%|█████▉    | 7124/12000 [4:12:43<2:35:36,  1.91s/it]
 59%|█████▉    | 7125/12000 [4:12:45<2:40:05,  1.97s/it]
                                                        

 59%|█████▉    | 7125/12000 [4:12:45<2:40:05,  1.97s/it]
 59%|█████▉    | 7126/12000 [4:12:46<2:30:02,  1.85s/it]
                                                        

 59%|█████▉    | 7126/12000 [4:12:46<2:30:02,  1.85s/it]
 59%|█████▉    | 7127/12000 [4:12:50<3:16:39,  2.42s/it]
                                                        

 59%|█████▉    | 7127/12000 [4:12:50<3:16:39,  2.42s/it]
 59%|█████▉    | 7128/12000 [4:12:52<3:03:19,  2.26s/it]
                                                        

 59%|█████▉    | 7128/12000 [4:12:52<3:03:19,  2.26s/it]
 59%|█████▉    | 7129/12000 [4:12:55<3:10:47,  2.35s/it]
                                                        

 59%|█████▉    | 7129/12000 [4:12:55<3:10:47,  2.35s/it]
 59%|█████▉    | 7130/12000 [4:12:56<2:53:03,  2.13s/it]
                                                        

 59%|█████▉    | 7130/12000 [4:12:56<2:53:03,  2.13s/it]
 59%|█████▉    | 7131/12000 [4:12:58<2:43:18,  2.01s/it]
                                                        

 59%|█████▉    | 7131/12000 [4:12:58<2:43:18,  2.01s/it]
 59%|█████▉    | 7132/12000 [4:12:59<2:28:38,  1.83s/it]
                                                        

 59%|█████▉    | 7132/12000 [4:12:59<2:28:38,  1.83s/it]
 59%|█████▉    | 7133/12000 [4:13:01<2:18:57,  1.71s/it]
                                                        

 59%|█████▉    | 7133/12000 [4:13:01<2:18:57,  1.71s/it]
 59%|█████▉    | 7134/12000 [4:13:03<2:37:32,  1.94s/it]
                                                        

 59%|█████▉    | 7134/12000 [4:13:03<2:37:32,  1.94s/it]
 59%|█████▉    | 7135/12000 [4:13:05<2:28:53,  1.84s/it]
                                                        

 59%|█████▉    | 7135/12000 [4:13:05<2:28:53,  1.84s/it]
 59%|█████▉    | 7136/12000 [4:13:07<2:37:15,  1.94s/it]
                                                        

 59%|█████▉    | 7136/12000 [4:13:07<2:37:15,  1.94s/it]
 59%|█████▉    | 7137/12000 [4:13:09<2:31:58,  1.88s/it]
                                                        

 59%|█████▉    | 7137/12000 [4:13:09<2:31:58,  1.88s/it]
 59%|█████▉    | 7138/12000 [4:13:10<2:19:37,  1.72s/it]
                                                        

 59%|█████▉    | 7138/12000 [4:13:10<2:19:37,  1.72s/it]
 59%|█████▉    | 7139/12000 [4:13:13<2:47:24,  2.07s/it]
                                                        

 59%|█████▉    | 7139/12000 [4:13:13<2:47:24,  2.07s/it]
 60%|█████▉    | 7140/12000 [4:13:17<3:35:45,  2.66s/it]
                                                        

 60%|█████▉    | 7140/12000 [4:13:17<3:35:45,  2.66s/it]
 60%|█████▉    | 7141/12000 [4:13:19<3:28:29,  2.57s/it]
                                                        

 60%|█████▉    | 7141/12000 [4:13:19<3:28:29,  2.57s/it]
 60%|█████▉    | 7142/12000 [4:13:21<3:03:28,  2.27s/it]
                                                        

 60%|█████▉    | 7142/12000 [4:13:21<3:03:28,  2.27s/it]
 60%|█████▉    | 7143/12000 [4:13:23<3:09:23,  2.34s/it]
                                                        

 60%|█████▉    | 7143/12000 [4:13:23<3:09:23,  2.34s/it]
 60%|█████▉    | 7144/12000 [4:13:25<2:51:02,  2.11s/it]
                                                        

 60%|█████▉    | 7144/12000 [4:13:25<2:51:02,  2.11s/it]
 60%|█████▉    | 7145/12000 [4:13:27<2:52:06,  2.13s/it]
                                                        

 60%|█████▉    | 7145/12000 [4:13:27<2:52:06,  2.13s/it]
 60%|█████▉    | 7146/12000 [4:13:29<2:42:39,  2.01s/it]
                                                        

 60%|█████▉    | 7146/12000 [4:13:29<2:42:39,  2.01s/it]
 60%|█████▉    | 7147/12000 [4:13:31<2:39:22,  1.97s/it]
                                                        

 60%|█████▉    | 7147/12000 [4:13:31<2:39:22,  1.97s/it]
 60%|█████▉    | 7148/12000 [4:13:33<2:33:11,  1.89s/it]
                                                        
{'loss': '0.6865', 'grad_norm': '0.3299', 'learning_rate': '7.813e-05', 'entropy': '0.7549', 'num_tokens': '3.833e+07', 'mean_token_accuracy': '0.8137', 'epoch': '0.06284'}
{'loss': '0.7878', 'grad_norm': '0.417', 'learning_rate': '7.811e-05', 'entropy': '0.7821', 'num_tokens': '3.833e+07', 'mean_token_accuracy': '0.7921', 'epoch': '0.06285'}
{'loss': '0.6731', 'grad_norm': '0.3827', 'learning_rate': '7.808e-05', 'entropy': '0.747', 'num_tokens': '3.834e+07', 'mean_token_accuracy': '0.813', 'epoch': '0.06286'}
{'loss': '0.9435', 'grad_norm': '0.4117', 'learning_rate': '7.805e-05', 'entropy': '0.8636', 'num_tokens': '3.835e+07', 'mean_token_accuracy': '0.7868', 'epoch': '0.06287'}
{'loss': '0.8476', 'grad_norm': '0.3978', 'learning_rate': '7.802e-05', 'entropy': '0.8403', 'num_tokens': '3.835e+07', 'mean_token_accuracy': '0.7856', 'epoch': '0.06288'}
{'loss': '0.559', 'grad_norm': '0.3008', 'learning_rate': '7.8e-05', 'entropy': '0.5786', 'num_tokens': '3.836e+07', 'mean_token_accuracy': '0.8366', 'epoch': '0.06289'}
{'loss': '0.7107', 'grad_norm': '0.3956', 'learning_rate': '7.797e-05', 'entropy': '0.7172', 'num_tokens': '3.836e+07', 'mean_token_accuracy': '0.8282', 'epoch': '0.0629'}
{'loss': '0.8224', 'grad_norm': '0.3819', 'learning_rate': '7.794e-05', 'entropy': '0.8158', 'num_tokens': '3.837e+07', 'mean_token_accuracy': '0.8003', 'epoch': '0.0629'}
{'loss': '1.124', 'grad_norm': '0.4029', 'learning_rate': '7.792e-05', 'entropy': '1.064', 'num_tokens': '3.837e+07', 'mean_token_accuracy': '0.7634', 'epoch': '0.06291'}
{'loss': '0.7234', 'grad_norm': '0.4337', 'learning_rate': '7.789e-05', 'entropy': '0.7422', 'num_tokens': '3.838e+07', 'mean_token_accuracy': '0.8115', 'epoch': '0.06292'}
{'loss': '0.8961', 'grad_norm': '0.4282', 'learning_rate': '7.786e-05', 'entropy': '0.931', 'num_tokens': '3.838e+07', 'mean_token_accuracy': '0.7794', 'epoch': '0.06293'}
{'loss': '0.4528', 'grad_norm': '0.3159', 'learning_rate': '7.784e-05', 'entropy': '0.527', 'num_tokens': '3.839e+07', 'mean_token_accuracy': '0.8669', 'epoch': '0.06294'}
{'loss': '0.7615', 'grad_norm': '0.4083', 'learning_rate': '7.781e-05', 'entropy': '0.7562', 'num_tokens': '3.839e+07', 'mean_token_accuracy': '0.8061', 'epoch': '0.06295'}
{'loss': '0.7309', 'grad_norm': '0.3178', 'learning_rate': '7.778e-05', 'entropy': '0.707', 'num_tokens': '3.84e+07', 'mean_token_accuracy': '0.8199', 'epoch': '0.06296'}
{'loss': '0.8884', 'grad_norm': '0.3826', 'learning_rate': '7.776e-05', 'entropy': '0.9602', 'num_tokens': '3.84e+07', 'mean_token_accuracy': '0.7687', 'epoch': '0.06297'}
{'loss': '0.7687', 'grad_norm': '0.4178', 'learning_rate': '7.773e-05', 'entropy': '0.7594', 'num_tokens': '3.84e+07', 'mean_token_accuracy': '0.8034', 'epoch': '0.06298'}
{'loss': '0.813', 'grad_norm': '0.3717', 'learning_rate': '7.77e-05', 'entropy': '0.791', 'num_tokens': '3.841e+07', 'mean_token_accuracy': '0.8124', 'epoch': '0.06298'}
{'loss': '0.9399', 'grad_norm': '0.4412', 'learning_rate': '7.767e-05', 'entropy': '0.952', 'num_tokens': '3.841e+07', 'mean_token_accuracy': '0.7682', 'epoch': '0.06299'}
{'loss': '0.7652', 'grad_norm': '0.3888', 'learning_rate': '7.765e-05', 'entropy': '0.7436', 'num_tokens': '3.842e+07', 'mean_token_accuracy': '0.7999', 'epoch': '0.063'}
{'loss': '0.7858', 'grad_norm': '0.3461', 'learning_rate': '7.762e-05', 'entropy': '0.7043', 'num_tokens': '3.842e+07', 'mean_token_accuracy': '0.8338', 'epoch': '0.06301'}
{'loss': '0.6537', 'grad_norm': '0.3572', 'learning_rate': '7.759e-05', 'entropy': '0.6114', 'num_tokens': '3.843e+07', 'mean_token_accuracy': '0.8363', 'epoch': '0.06302'}
{'loss': '1.182', 'grad_norm': '0.4779', 'learning_rate': '7.757e-05', 'entropy': '1.223', 'num_tokens': '3.843e+07', 'mean_token_accuracy': '0.7131', 'epoch': '0.06303'}
{'loss': '0.6946', 'grad_norm': '0.3662', 'learning_rate': '7.754e-05', 'entropy': '0.7318', 'num_tokens': '3.844e+07', 'mean_token_accuracy': '0.8173', 'epoch': '0.06304'}
{'loss': '1.073', 'grad_norm': '0.4542', 'learning_rate': '7.751e-05', 'entropy': '1.058', 'num_tokens': '3.844e+07', 'mean_token_accuracy': '0.7391', 'epoch': '0.06305'}
{'loss': '0.6848', 'grad_norm': '0.3365', 'learning_rate': '7.749e-05', 'entropy': '0.7093', 'num_tokens': '3.845e+07', 'mean_token_accuracy': '0.8114', 'epoch': '0.06306'}
{'loss': '0.7838', 'grad_norm': '0.3393', 'learning_rate': '7.746e-05', 'entropy': '0.8419', 'num_tokens': '3.845e+07', 'mean_token_accuracy': '0.7989', 'epoch': '0.06306'}
{'loss': '0.7106', 'grad_norm': '0.4657', 'learning_rate': '7.743e-05', 'entropy': '0.688', 'num_tokens': '3.846e+07', 'mean_token_accuracy': '0.8226', 'epoch': '0.06307'}
{'loss': '0.6032', 'grad_norm': '0.3501', 'learning_rate': '7.741e-05', 'entropy': '0.6517', 'num_tokens': '3.847e+07', 'mean_token_accuracy': '0.8375', 'epoch': '0.06308'}
{'loss': '0.8152', 'grad_norm': '0.315', 'learning_rate': '7.738e-05', 'entropy': '0.9927', 'num_tokens': '3.847e+07', 'mean_token_accuracy': '0.7426', 'epoch': '0.06309'}
{'loss': '0.6993', 'grad_norm': '0.3932', 'learning_rate': '7.735e-05', 'entropy': '0.7202', 'num_tokens': '3.848e+07', 'mean_token_accuracy': '0.8193', 'epoch': '0.0631'}
{'loss': '1.129', 'grad_norm': '0.4337', 'learning_rate': '7.733e-05', 'entropy': '1.009', 'num_tokens': '3.848e+07', 'mean_token_accuracy': '0.7619', 'epoch': '0.06311'}
{'loss': '0.8609', 'grad_norm': '0.4364', 'learning_rate': '7.73e-05', 'entropy': '0.8831', 'num_tokens': '3.849e+07', 'mean_token_accuracy': '0.7754', 'epoch': '0.06312'}
{'loss': '0.9364', 'grad_norm': '0.4162', 'learning_rate': '7.727e-05', 'entropy': '0.9214', 'num_tokens': '3.849e+07', 'mean_token_accuracy': '0.7734', 'epoch': '0.06313'}
{'loss': '0.9738', 'grad_norm': '0.4926', 'learning_rate': '7.725e-05', 'entropy': '0.9033', 'num_tokens': '3.85e+07', 'mean_token_accuracy': '0.7745', 'epoch': '0.06313'}
{'loss': '0.6113', 'grad_norm': '0.3001', 'learning_rate': '7.722e-05', 'entropy': '0.6403', 'num_tokens': '3.85e+07', 'mean_token_accuracy': '0.833', 'epoch': '0.06314'}
{'loss': '0.8054', 'grad_norm': '0.3687', 'learning_rate': '7.719e-05', 'entropy': '0.7819', 'num_tokens': '3.851e+07', 'mean_token_accuracy': '0.7922', 'epoch': '0.06315'}
{'loss': '0.5697', 'grad_norm': '0.3778', 'learning_rate': '7.716e-05', 'entropy': '0.577', 'num_tokens': '3.851e+07', 'mean_token_accuracy': '0.8514', 'epoch': '0.06316'}
{'loss': '0.6654', 'grad_norm': '0.3474', 'learning_rate': '7.714e-05', 'entropy': '0.7789', 'num_tokens': '3.852e+07', 'mean_token_accuracy': '0.8094', 'epoch': '0.06317'}
{'loss': '0.7218', 'grad_norm': '0.381', 'learning_rate': '7.711e-05', 'entropy': '0.7165', 'num_tokens': '3.852e+07', 'mean_token_accuracy': '0.8149', 'epoch': '0.06318'}
{'loss': '0.9329', 'grad_norm': '0.3929', 'learning_rate': '7.708e-05', 'entropy': '0.8388', 'num_tokens': '3.853e+07', 'mean_token_accuracy': '0.7984', 'epoch': '0.06319'}
{'loss': '0.6563', 'grad_norm': '0.3017', 'learning_rate': '7.706e-05', 'entropy': '0.6606', 'num_tokens': '3.854e+07', 'mean_token_accuracy': '0.8281', 'epoch': '0.0632'}
{'loss': '0.683', 'grad_norm': '0.3898', 'learning_rate': '7.703e-05', 'entropy': '0.7012', 'num_tokens': '3.854e+07', 'mean_token_accuracy': '0.8201', 'epoch': '0.06321'}
{'loss': '1.054', 'grad_norm': '0.4503', 'learning_rate': '7.7e-05', 'entropy': '0.9789', 'num_tokens': '3.855e+07', 'mean_token_accuracy': '0.7566', 'epoch': '0.06321'}
{'loss': '0.8415', 'grad_norm': '0.3632', 'learning_rate': '7.698e-05', 'entropy': '0.815', 'num_tokens': '3.855e+07', 'mean_token_accuracy': '0.7868', 'epoch': '0.06322'}
{'loss': '0.7545', 'grad_norm': '0.3765', 'learning_rate': '7.695e-05', 'entropy': '0.7021', 'num_tokens': '3.856e+07', 'mean_token_accuracy': '0.8198', 'epoch': '0.06323'}
{'loss': '0.6528', 'grad_norm': '0.4174', 'learning_rate': '7.692e-05', 'entropy': '0.7078', 'num_tokens': '3.856e+07', 'mean_token_accuracy': '0.8178', 'epoch': '0.06324'}
{'loss': '0.7351', 'grad_norm': '0.3768', 'learning_rate': '7.69e-05', 'entropy': '0.7466', 'num_tokens': '3.857e+07', 'mean_token_accuracy': '0.8077', 'epoch': '0.06325'}

 60%|█████▉    | 7148/12000 [4:13:33<2:33:11,  1.89s/it]
 60%|█████▉    | 7149/12000 [4:13:34<2:33:52,  1.90s/it]
                                                        

 60%|█████▉    | 7149/12000 [4:13:34<2:33:52,  1.90s/it]
 60%|█████▉    | 7150/12000 [4:13:37<2:52:09,  2.13s/it]
                                                        

 60%|█████▉    | 7150/12000 [4:13:37<2:52:09,  2.13s/it]
 60%|█████▉    | 7151/12000 [4:13:40<3:17:44,  2.45s/it]
                                                        

 60%|█████▉    | 7151/12000 [4:13:40<3:17:44,  2.45s/it]
 60%|█████▉    | 7152/12000 [4:13:42<2:55:53,  2.18s/it]
                                                        

 60%|█████▉    | 7152/12000 [4:13:42<2:55:53,  2.18s/it]
 60%|█████▉    | 7153/12000 [4:13:45<3:15:10,  2.42s/it]
                                                        

 60%|█████▉    | 7153/12000 [4:13:45<3:15:10,  2.42s/it]
 60%|█████▉    | 7154/12000 [4:13:47<3:19:21,  2.47s/it]
                                                        

 60%|█████▉    | 7154/12000 [4:13:47<3:19:21,  2.47s/it]
 60%|█████▉    | 7155/12000 [4:13:51<3:36:37,  2.68s/it]
                                                        

 60%|█████▉    | 7155/12000 [4:13:51<3:36:37,  2.68s/it]
 60%|█████▉    | 7156/12000 [4:13:53<3:18:51,  2.46s/it]
                                                        

 60%|█████▉    | 7156/12000 [4:13:53<3:18:51,  2.46s/it]
 60%|█████▉    | 7157/12000 [4:13:55<3:12:42,  2.39s/it]
                                                        

 60%|█████▉    | 7157/12000 [4:13:55<3:12:42,  2.39s/it]
 60%|█████▉    | 7158/12000 [4:13:56<2:54:05,  2.16s/it]
                                                        

 60%|█████▉    | 7158/12000 [4:13:56<2:54:05,  2.16s/it]
 60%|█████▉    | 7159/12000 [4:13:59<3:01:53,  2.25s/it]
                                                        

 60%|█████▉    | 7159/12000 [4:13:59<3:01:53,  2.25s/it]
 60%|█████▉    | 7160/12000 [4:14:03<3:59:50,  2.97s/it]
                                                        

 60%|█████▉    | 7160/12000 [4:14:04<3:59:50,  2.97s/it]
 60%|█████▉    | 7161/12000 [4:14:07<4:16:07,  3.18s/it]
                                                        

 60%|█████▉    | 7161/12000 [4:14:07<4:16:07,  3.18s/it]
 60%|█████▉    | 7162/12000 [4:14:11<4:36:34,  3.43s/it]
                                                        

 60%|█████▉    | 7162/12000 [4:14:11<4:36:34,  3.43s/it]
 60%|█████▉    | 7163/12000 [4:14:15<4:46:47,  3.56s/it]
                                                        

 60%|█████▉    | 7163/12000 [4:14:15<4:46:47,  3.56s/it]
 60%|█████▉    | 7164/12000 [4:14:17<3:59:32,  2.97s/it]
                                                        

 60%|█████▉    | 7164/12000 [4:14:17<3:59:32,  2.97s/it]
 60%|█████▉    | 7165/12000 [4:14:19<3:34:06,  2.66s/it]
                                                        

 60%|█████▉    | 7165/12000 [4:14:19<3:34:06,  2.66s/it]
 60%|█████▉    | 7166/12000 [4:14:20<3:09:42,  2.35s/it]
                                                        

 60%|█████▉    | 7166/12000 [4:14:20<3:09:42,  2.35s/it]
 60%|█████▉    | 7167/12000 [4:14:22<2:55:51,  2.18s/it]
                                                        

 60%|█████▉    | 7167/12000 [4:14:22<2:55:51,  2.18s/it]
 60%|█████▉    | 7168/12000 [4:14:24<3:00:26,  2.24s/it]
                                                        

 60%|█████▉    | 7168/12000 [4:14:24<3:00:26,  2.24s/it]
 60%|█████▉    | 7169/12000 [4:14:27<3:09:44,  2.36s/it]
                                                        

 60%|█████▉    | 7169/12000 [4:14:27<3:09:44,  2.36s/it]
 60%|█████▉    | 7170/12000 [4:14:30<3:35:35,  2.68s/it]
                                                        

 60%|█████▉    | 7170/12000 [4:14:30<3:35:35,  2.68s/it]
 60%|█████▉    | 7171/12000 [4:14:32<3:06:08,  2.31s/it]
                                                        

 60%|█████▉    | 7171/12000 [4:14:32<3:06:08,  2.31s/it]
 60%|█████▉    | 7172/12000 [4:14:34<3:04:25,  2.29s/it]
                                                        

 60%|█████▉    | 7172/12000 [4:14:34<3:04:25,  2.29s/it]
 60%|█████▉    | 7173/12000 [4:14:38<3:43:49,  2.78s/it]
                                                        

 60%|█████▉    | 7173/12000 [4:14:38<3:43:49,  2.78s/it]
 60%|█████▉    | 7174/12000 [4:14:41<3:42:16,  2.76s/it]
                                                        

 60%|█████▉    | 7174/12000 [4:14:41<3:42:16,  2.76s/it]
 60%|█████▉    | 7175/12000 [4:14:43<3:19:23,  2.48s/it]
                                                        

 60%|█████▉    | 7175/12000 [4:14:43<3:19:23,  2.48s/it]
 60%|█████▉    | 7176/12000 [4:14:44<2:53:41,  2.16s/it]
                                                        

 60%|█████▉    | 7176/12000 [4:14:44<2:53:41,  2.16s/it]
 60%|█████▉    | 7177/12000 [4:14:45<2:35:05,  1.93s/it]
                                                        

 60%|█████▉    | 7177/12000 [4:14:45<2:35:05,  1.93s/it]
 60%|█████▉    | 7178/12000 [4:14:48<2:43:58,  2.04s/it]
                                                        

 60%|█████▉    | 7178/12000 [4:14:48<2:43:58,  2.04s/it]
 60%|█████▉    | 7179/12000 [4:14:49<2:32:09,  1.89s/it]
                                                        

 60%|█████▉    | 7179/12000 [4:14:49<2:32:09,  1.89s/it]
 60%|█████▉    | 7180/12000 [4:14:51<2:22:04,  1.77s/it]
                                                        

 60%|█████▉    | 7180/12000 [4:14:51<2:22:04,  1.77s/it]
 60%|█████▉    | 7181/12000 [4:14:53<2:38:41,  1.98s/it]
                                                        

 60%|█████▉    | 7181/12000 [4:14:53<2:38:41,  1.98s/it]
 60%|█████▉    | 7182/12000 [4:14:55<2:31:24,  1.89s/it]
                                                        

 60%|█████▉    | 7182/12000 [4:14:55<2:31:24,  1.89s/it]
 60%|█████▉    | 7183/12000 [4:14:58<3:00:29,  2.25s/it]
                                                        

 60%|█████▉    | 7183/12000 [4:14:58<3:00:29,  2.25s/it]
 60%|█████▉    | 7184/12000 [4:15:02<3:41:17,  2.76s/it]
                                                        

 60%|█████▉    | 7184/12000 [4:15:02<3:41:17,  2.76s/it]
 60%|█████▉    | 7185/12000 [4:15:04<3:33:14,  2.66s/it]
                                                        

 60%|█████▉    | 7185/12000 [4:15:04<3:33:14,  2.66s/it]
 60%|█████▉    | 7186/12000 [4:15:06<3:13:04,  2.41s/it]
                                                        

 60%|█████▉    | 7186/12000 [4:15:06<3:13:04,  2.41s/it]
 60%|█████▉    | 7187/12000 [4:15:08<2:50:50,  2.13s/it]
                                                        

 60%|█████▉    | 7187/12000 [4:15:08<2:50:50,  2.13s/it]
 60%|█████▉    | 7188/12000 [4:15:10<3:08:51,  2.35s/it]
                                                        

 60%|█████▉    | 7188/12000 [4:15:10<3:08:51,  2.35s/it]
 60%|█████▉    | 7189/12000 [4:15:12<2:50:05,  2.12s/it]
                                                        

 60%|█████▉    | 7189/12000 [4:15:12<2:50:05,  2.12s/it]
 60%|█████▉    | 7190/12000 [4:15:14<2:52:31,  2.15s/it]
                                                        

 60%|█████▉    | 7190/12000 [4:15:14<2:52:31,  2.15s/it]
 60%|█████▉    | 7191/12000 [4:15:16<2:48:13,  2.10s/it]
                                                        

 60%|█████▉    | 7191/12000 [4:15:16<2:48:13,  2.10s/it]
 60%|█████▉    | 7192/12000 [4:15:18<2:39:05,  1.99s/it]
                                                        

 60%|█████▉    | 7192/12000 [4:15:18<2:39:05,  1.99s/it]
 60%|█████▉    | 7193/12000 [4:15:20<2:41:40,  2.02s/it]
                                                        

 60%|█████▉    | 7193/12000 [4:15:20<2:41:40,  2.02s/it]
 60%|█████▉    | 7194/12000 [4:15:22<2:33:49,  1.92s/it]
                                                        

 60%|█████▉    | 7194/12000 [4:15:22<2:33:49,  1.92s/it]
 60%|█████▉    | 7195/12000 [4:15:23<2:24:39,  1.81s/it]
                                                        
{'loss': '0.6672', 'grad_norm': '0.4223', 'learning_rate': '7.687e-05', 'entropy': '0.6664', 'num_tokens': '3.858e+07', 'mean_token_accuracy': '0.8249', 'epoch': '0.06326'}
{'loss': '0.5863', 'grad_norm': '0.4367', 'learning_rate': '7.684e-05', 'entropy': '0.7566', 'num_tokens': '3.858e+07', 'mean_token_accuracy': '0.8038', 'epoch': '0.06327'}
{'loss': '0.8974', 'grad_norm': '0.3702', 'learning_rate': '7.682e-05', 'entropy': '0.9604', 'num_tokens': '3.859e+07', 'mean_token_accuracy': '0.7645', 'epoch': '0.06328'}
{'loss': '0.7956', 'grad_norm': '0.5154', 'learning_rate': '7.679e-05', 'entropy': '0.8059', 'num_tokens': '3.859e+07', 'mean_token_accuracy': '0.7847', 'epoch': '0.06329'}
{'loss': '0.5709', 'grad_norm': '0.2709', 'learning_rate': '7.676e-05', 'entropy': '0.5901', 'num_tokens': '3.86e+07', 'mean_token_accuracy': '0.8375', 'epoch': '0.06329'}
{'loss': '0.8039', 'grad_norm': '0.3798', 'learning_rate': '7.674e-05', 'entropy': '0.8727', 'num_tokens': '3.86e+07', 'mean_token_accuracy': '0.7966', 'epoch': '0.0633'}
{'loss': '0.88', 'grad_norm': '0.3747', 'learning_rate': '7.671e-05', 'entropy': '0.9082', 'num_tokens': '3.861e+07', 'mean_token_accuracy': '0.7739', 'epoch': '0.06331'}
{'loss': '0.7896', 'grad_norm': '0.3845', 'learning_rate': '7.668e-05', 'entropy': '0.8359', 'num_tokens': '3.862e+07', 'mean_token_accuracy': '0.7995', 'epoch': '0.06332'}
{'loss': '0.723', 'grad_norm': '0.3399', 'learning_rate': '7.666e-05', 'entropy': '0.7106', 'num_tokens': '3.862e+07', 'mean_token_accuracy': '0.8145', 'epoch': '0.06333'}
{'loss': '0.8841', 'grad_norm': '0.4605', 'learning_rate': '7.663e-05', 'entropy': '0.881', 'num_tokens': '3.863e+07', 'mean_token_accuracy': '0.7754', 'epoch': '0.06334'}
{'loss': '0.867', 'grad_norm': '0.5269', 'learning_rate': '7.66e-05', 'entropy': '0.9369', 'num_tokens': '3.863e+07', 'mean_token_accuracy': '0.7736', 'epoch': '0.06335'}
{'loss': '0.642', 'grad_norm': '0.3398', 'learning_rate': '7.658e-05', 'entropy': '0.702', 'num_tokens': '3.864e+07', 'mean_token_accuracy': '0.8079', 'epoch': '0.06336'}
{'loss': '0.9463', 'grad_norm': '0.2771', 'learning_rate': '7.655e-05', 'entropy': '0.8636', 'num_tokens': '3.865e+07', 'mean_token_accuracy': '0.779', 'epoch': '0.06336'}
{'loss': '0.7547', 'grad_norm': '0.3144', 'learning_rate': '7.652e-05', 'entropy': '0.6368', 'num_tokens': '3.865e+07', 'mean_token_accuracy': '0.8354', 'epoch': '0.06337'}
{'loss': '0.9302', 'grad_norm': '0.4802', 'learning_rate': '7.649e-05', 'entropy': '0.9253', 'num_tokens': '3.866e+07', 'mean_token_accuracy': '0.7756', 'epoch': '0.06338'}
{'loss': '0.6498', 'grad_norm': '0.3633', 'learning_rate': '7.647e-05', 'entropy': '0.7639', 'num_tokens': '3.867e+07', 'mean_token_accuracy': '0.8163', 'epoch': '0.06339'}
{'loss': '0.6984', 'grad_norm': '0.3458', 'learning_rate': '7.644e-05', 'entropy': '0.6758', 'num_tokens': '3.867e+07', 'mean_token_accuracy': '0.825', 'epoch': '0.0634'}
{'loss': '0.6934', 'grad_norm': '0.4808', 'learning_rate': '7.641e-05', 'entropy': '0.7112', 'num_tokens': '3.868e+07', 'mean_token_accuracy': '0.8073', 'epoch': '0.06341'}
{'loss': '0.9871', 'grad_norm': '0.4316', 'learning_rate': '7.639e-05', 'entropy': '0.9419', 'num_tokens': '3.868e+07', 'mean_token_accuracy': '0.7693', 'epoch': '0.06342'}
{'loss': '0.7657', 'grad_norm': '0.4932', 'learning_rate': '7.636e-05', 'entropy': '0.7407', 'num_tokens': '3.869e+07', 'mean_token_accuracy': '0.7947', 'epoch': '0.06343'}
{'loss': '0.7349', 'grad_norm': '0.387', 'learning_rate': '7.633e-05', 'entropy': '0.722', 'num_tokens': '3.869e+07', 'mean_token_accuracy': '0.816', 'epoch': '0.06344'}
{'loss': '0.8898', 'grad_norm': '0.3618', 'learning_rate': '7.631e-05', 'entropy': '0.9722', 'num_tokens': '3.87e+07', 'mean_token_accuracy': '0.7609', 'epoch': '0.06344'}
{'loss': '0.9439', 'grad_norm': '0.3936', 'learning_rate': '7.628e-05', 'entropy': '0.8969', 'num_tokens': '3.871e+07', 'mean_token_accuracy': '0.7845', 'epoch': '0.06345'}
{'loss': '0.7859', 'grad_norm': '0.4197', 'learning_rate': '7.625e-05', 'entropy': '0.8015', 'num_tokens': '3.871e+07', 'mean_token_accuracy': '0.787', 'epoch': '0.06346'}
{'loss': '1.351', 'grad_norm': '0.6543', 'learning_rate': '7.623e-05', 'entropy': '1.119', 'num_tokens': '3.872e+07', 'mean_token_accuracy': '0.7395', 'epoch': '0.06347'}
{'loss': '1.436', 'grad_norm': '0.3499', 'learning_rate': '7.62e-05', 'entropy': '1.049', 'num_tokens': '3.873e+07', 'mean_token_accuracy': '0.7635', 'epoch': '0.06348'}
{'loss': '0.9468', 'grad_norm': '0.3819', 'learning_rate': '7.617e-05', 'entropy': '0.8679', 'num_tokens': '3.873e+07', 'mean_token_accuracy': '0.7924', 'epoch': '0.06349'}
{'loss': '1.034', 'grad_norm': '0.405', 'learning_rate': '7.615e-05', 'entropy': '0.924', 'num_tokens': '3.874e+07', 'mean_token_accuracy': '0.7676', 'epoch': '0.0635'}
{'loss': '0.6067', 'grad_norm': '0.3334', 'learning_rate': '7.612e-05', 'entropy': '0.6832', 'num_tokens': '3.874e+07', 'mean_token_accuracy': '0.8351', 'epoch': '0.06351'}
{'loss': '0.7537', 'grad_norm': '0.3629', 'learning_rate': '7.609e-05', 'entropy': '0.8273', 'num_tokens': '3.875e+07', 'mean_token_accuracy': '0.7823', 'epoch': '0.06352'}
{'loss': '0.5923', 'grad_norm': '0.3203', 'learning_rate': '7.607e-05', 'entropy': '0.6221', 'num_tokens': '3.875e+07', 'mean_token_accuracy': '0.8434', 'epoch': '0.06352'}
{'loss': '0.8793', 'grad_norm': '0.4608', 'learning_rate': '7.604e-05', 'entropy': '0.9112', 'num_tokens': '3.876e+07', 'mean_token_accuracy': '0.7795', 'epoch': '0.06353'}
{'loss': '1.052', 'grad_norm': '0.4623', 'learning_rate': '7.601e-05', 'entropy': '1.084', 'num_tokens': '3.876e+07', 'mean_token_accuracy': '0.7427', 'epoch': '0.06354'}
{'loss': '0.8624', 'grad_norm': '0.3616', 'learning_rate': '7.599e-05', 'entropy': '0.8087', 'num_tokens': '3.877e+07', 'mean_token_accuracy': '0.8045', 'epoch': '0.06355'}
{'loss': '0.655', 'grad_norm': '0.3173', 'learning_rate': '7.596e-05', 'entropy': '0.714', 'num_tokens': '3.877e+07', 'mean_token_accuracy': '0.8186', 'epoch': '0.06356'}
{'loss': '0.7238', 'grad_norm': '0.2954', 'learning_rate': '7.593e-05', 'entropy': '0.6557', 'num_tokens': '3.878e+07', 'mean_token_accuracy': '0.8345', 'epoch': '0.06357'}
{'loss': '0.8904', 'grad_norm': '0.3644', 'learning_rate': '7.591e-05', 'entropy': '0.8833', 'num_tokens': '3.879e+07', 'mean_token_accuracy': '0.7878', 'epoch': '0.06358'}
{'loss': '0.9851', 'grad_norm': '0.4113', 'learning_rate': '7.588e-05', 'entropy': '1.066', 'num_tokens': '3.879e+07', 'mean_token_accuracy': '0.7509', 'epoch': '0.06359'}
{'loss': '1.101', 'grad_norm': '0.4568', 'learning_rate': '7.585e-05', 'entropy': '1.077', 'num_tokens': '3.88e+07', 'mean_token_accuracy': '0.7349', 'epoch': '0.06359'}
{'loss': '0.6847', 'grad_norm': '0.3569', 'learning_rate': '7.583e-05', 'entropy': '0.7197', 'num_tokens': '3.88e+07', 'mean_token_accuracy': '0.814', 'epoch': '0.0636'}
{'loss': '1.057', 'grad_norm': '0.3737', 'learning_rate': '7.58e-05', 'entropy': '0.904', 'num_tokens': '3.881e+07', 'mean_token_accuracy': '0.782', 'epoch': '0.06361'}
{'loss': '0.7502', 'grad_norm': '0.5175', 'learning_rate': '7.577e-05', 'entropy': '0.7622', 'num_tokens': '3.881e+07', 'mean_token_accuracy': '0.7983', 'epoch': '0.06362'}
{'loss': '0.8688', 'grad_norm': '0.3794', 'learning_rate': '7.575e-05', 'entropy': '0.8453', 'num_tokens': '3.882e+07', 'mean_token_accuracy': '0.7891', 'epoch': '0.06363'}
{'loss': '0.9903', 'grad_norm': '0.4552', 'learning_rate': '7.572e-05', 'entropy': '0.9342', 'num_tokens': '3.882e+07', 'mean_token_accuracy': '0.7603', 'epoch': '0.06364'}
{'loss': '1.056', 'grad_norm': '0.4176', 'learning_rate': '7.569e-05', 'entropy': '0.9855', 'num_tokens': '3.883e+07', 'mean_token_accuracy': '0.7786', 'epoch': '0.06365'}
{'loss': '0.8778', 'grad_norm': '0.3533', 'learning_rate': '7.567e-05', 'entropy': '0.9666', 'num_tokens': '3.883e+07', 'mean_token_accuracy': '0.7644', 'epoch': '0.06366'}
{'loss': '1.146', 'grad_norm': '0.4692', 'learning_rate': '7.564e-05', 'entropy': '1.126', 'num_tokens': '3.884e+07', 'mean_token_accuracy': '0.7204', 'epoch': '0.06367'}

 60%|█████▉    | 7195/12000 [4:15:23<2:24:39,  1.81s/it]
 60%|█████▉    | 7196/12000 [4:15:25<2:20:12,  1.75s/it]
                                                        

 60%|█████▉    | 7196/12000 [4:15:25<2:20:12,  1.75s/it]
 60%|█████▉    | 7197/12000 [4:15:28<2:46:46,  2.08s/it]
                                                        

 60%|█████▉    | 7197/12000 [4:15:28<2:46:46,  2.08s/it]
 60%|█████▉    | 7198/12000 [4:15:33<3:54:36,  2.93s/it]
                                                        

 60%|█████▉    | 7198/12000 [4:15:33<3:54:36,  2.93s/it]
 60%|█████▉    | 7199/12000 [4:15:34<3:24:16,  2.55s/it]
                                                        

 60%|█████▉    | 7199/12000 [4:15:34<3:24:16,  2.55s/it]
 60%|██████    | 7200/12000 [4:15:39<4:18:53,  3.24s/it]
                                                        

 60%|██████    | 7200/12000 [4:15:39<4:18:53,  3.24s/it]
 60%|██████    | 7201/12000 [4:15:42<4:17:21,  3.22s/it]
                                                        

 60%|██████    | 7201/12000 [4:15:42<4:17:21,  3.22s/it]
 60%|██████    | 7202/12000 [4:15:44<3:39:02,  2.74s/it]
                                                        

 60%|██████    | 7202/12000 [4:15:44<3:39:02,  2.74s/it]
 60%|██████    | 7203/12000 [4:15:48<4:17:22,  3.22s/it]
                                                        

 60%|██████    | 7203/12000 [4:15:48<4:17:22,  3.22s/it]
 60%|██████    | 7204/12000 [4:15:50<3:41:01,  2.77s/it]
                                                        

 60%|██████    | 7204/12000 [4:15:50<3:41:01,  2.77s/it]
 60%|██████    | 7205/12000 [4:15:52<3:11:20,  2.39s/it]
                                                        

 60%|██████    | 7205/12000 [4:15:52<3:11:20,  2.39s/it]
 60%|██████    | 7206/12000 [4:15:56<3:55:35,  2.95s/it]
                                                        

 60%|██████    | 7206/12000 [4:15:56<3:55:35,  2.95s/it]
 60%|██████    | 7207/12000 [4:15:58<3:34:13,  2.68s/it]
                                                        

 60%|██████    | 7207/12000 [4:15:58<3:34:13,  2.68s/it]
 60%|██████    | 7208/12000 [4:16:01<3:46:14,  2.83s/it]
                                                        

 60%|██████    | 7208/12000 [4:16:01<3:46:14,  2.83s/it]
 60%|██████    | 7209/12000 [4:16:03<3:14:17,  2.43s/it]
                                                        

 60%|██████    | 7209/12000 [4:16:03<3:14:17,  2.43s/it]
 60%|██████    | 7210/12000 [4:16:04<2:55:56,  2.20s/it]
                                                        

 60%|██████    | 7210/12000 [4:16:04<2:55:56,  2.20s/it]
 60%|██████    | 7211/12000 [4:16:09<3:58:44,  2.99s/it]
                                                        

 60%|██████    | 7211/12000 [4:16:09<3:58:44,  2.99s/it]
 60%|██████    | 7212/12000 [4:16:11<3:24:19,  2.56s/it]
                                                        

 60%|██████    | 7212/12000 [4:16:11<3:24:19,  2.56s/it]
 60%|██████    | 7213/12000 [4:16:12<2:56:39,  2.21s/it]
                                                        

 60%|██████    | 7213/12000 [4:16:12<2:56:39,  2.21s/it]
 60%|██████    | 7214/12000 [4:16:14<2:48:22,  2.11s/it]
                                                        

 60%|██████    | 7214/12000 [4:16:14<2:48:22,  2.11s/it]
 60%|██████    | 7215/12000 [4:16:16<2:42:15,  2.03s/it]
                                                        

 60%|██████    | 7215/12000 [4:16:16<2:42:15,  2.03s/it]
 60%|██████    | 7216/12000 [4:16:19<3:16:20,  2.46s/it]
                                                        

 60%|██████    | 7216/12000 [4:16:19<3:16:20,  2.46s/it]
 60%|██████    | 7217/12000 [4:16:21<3:08:56,  2.37s/it]
                                                        

 60%|██████    | 7217/12000 [4:16:21<3:08:56,  2.37s/it]
 60%|██████    | 7218/12000 [4:16:24<3:23:19,  2.55s/it]
                                                        

 60%|██████    | 7218/12000 [4:16:24<3:23:19,  2.55s/it]
 60%|██████    | 7219/12000 [4:16:26<3:00:13,  2.26s/it]
                                                        

 60%|██████    | 7219/12000 [4:16:26<3:00:13,  2.26s/it]
 60%|██████    | 7220/12000 [4:16:27<2:39:39,  2.00s/it]
                                                        

 60%|██████    | 7220/12000 [4:16:27<2:39:39,  2.00s/it]
 60%|██████    | 7221/12000 [4:16:30<2:43:40,  2.05s/it]
                                                        

 60%|██████    | 7221/12000 [4:16:30<2:43:40,  2.05s/it]
 60%|██████    | 7222/12000 [4:16:33<3:08:09,  2.36s/it]
                                                        

 60%|██████    | 7222/12000 [4:16:33<3:08:09,  2.36s/it]
 60%|██████    | 7223/12000 [4:16:35<3:11:42,  2.41s/it]
                                                        

 60%|██████    | 7223/12000 [4:16:35<3:11:42,  2.41s/it]
 60%|██████    | 7224/12000 [4:16:37<3:00:33,  2.27s/it]
                                                        

 60%|██████    | 7224/12000 [4:16:37<3:00:33,  2.27s/it]
 60%|██████    | 7225/12000 [4:16:39<2:57:02,  2.22s/it]
                                                        

 60%|██████    | 7225/12000 [4:16:39<2:57:02,  2.22s/it]
 60%|██████    | 7226/12000 [4:16:42<3:13:09,  2.43s/it]
                                                        

 60%|██████    | 7226/12000 [4:16:42<3:13:09,  2.43s/it]
 60%|██████    | 7227/12000 [4:16:43<2:48:08,  2.11s/it]
                                                        

 60%|██████    | 7227/12000 [4:16:43<2:48:08,  2.11s/it]
 60%|██████    | 7228/12000 [4:16:47<3:28:34,  2.62s/it]
                                                        

 60%|██████    | 7228/12000 [4:16:47<3:28:34,  2.62s/it]
 60%|██████    | 7229/12000 [4:16:49<3:19:05,  2.50s/it]
                                                        

 60%|██████    | 7229/12000 [4:16:49<3:19:05,  2.50s/it]
 60%|██████    | 7230/12000 [4:16:51<2:57:22,  2.23s/it]
                                                        

 60%|██████    | 7230/12000 [4:16:51<2:57:22,  2.23s/it]
 60%|██████    | 7231/12000 [4:16:52<2:36:46,  1.97s/it]
                                                        

 60%|██████    | 7231/12000 [4:16:52<2:36:46,  1.97s/it]
 60%|██████    | 7232/12000 [4:16:54<2:23:14,  1.80s/it]
                                                        

 60%|██████    | 7232/12000 [4:16:54<2:23:14,  1.80s/it]
 60%|██████    | 7233/12000 [4:16:55<2:17:15,  1.73s/it]
                                                        

 60%|██████    | 7233/12000 [4:16:55<2:17:15,  1.73s/it]
 60%|██████    | 7234/12000 [4:16:57<2:16:56,  1.72s/it]
                                                        

 60%|██████    | 7234/12000 [4:16:57<2:16:56,  1.72s/it]
 60%|██████    | 7235/12000 [4:16:59<2:22:21,  1.79s/it]
                                                        

 60%|██████    | 7235/12000 [4:16:59<2:22:21,  1.79s/it]
 60%|██████    | 7236/12000 [4:17:01<2:36:33,  1.97s/it]
                                                        

 60%|██████    | 7236/12000 [4:17:01<2:36:33,  1.97s/it]
 60%|██████    | 7237/12000 [4:17:03<2:24:34,  1.82s/it]
                                                        

 60%|██████    | 7237/12000 [4:17:03<2:24:34,  1.82s/it]
 60%|██████    | 7238/12000 [4:17:04<2:14:58,  1.70s/it]
                                                        

 60%|██████    | 7238/12000 [4:17:04<2:14:58,  1.70s/it]
 60%|██████    | 7239/12000 [4:17:06<2:09:04,  1.63s/it]
                                                        

 60%|██████    | 7239/12000 [4:17:06<2:09:04,  1.63s/it]
 60%|██████    | 7240/12000 [4:17:08<2:16:55,  1.73s/it]
                                                        

 60%|██████    | 7240/12000 [4:17:08<2:16:55,  1.73s/it]
 60%|██████    | 7241/12000 [4:17:10<2:25:00,  1.83s/it]
                                                        

 60%|██████    | 7241/12000 [4:17:10<2:25:00,  1.83s/it]
 60%|██████    | 7242/12000 [4:17:11<2:15:02,  1.70s/it]
                                                        
{'loss': '0.8005', 'grad_norm': '0.3423', 'learning_rate': '7.561e-05', 'entropy': '0.7781', 'num_tokens': '3.884e+07', 'mean_token_accuracy': '0.7921', 'epoch': '0.06367'}
{'loss': '1.076', 'grad_norm': '0.5274', 'learning_rate': '7.559e-05', 'entropy': '1.074', 'num_tokens': '3.885e+07', 'mean_token_accuracy': '0.7341', 'epoch': '0.06368'}
{'loss': '0.6289', 'grad_norm': '0.3765', 'learning_rate': '7.556e-05', 'entropy': '0.6129', 'num_tokens': '3.886e+07', 'mean_token_accuracy': '0.8348', 'epoch': '0.06369'}
{'loss': '1.232', 'grad_norm': '0.3995', 'learning_rate': '7.553e-05', 'entropy': '0.9121', 'num_tokens': '3.886e+07', 'mean_token_accuracy': '0.7736', 'epoch': '0.0637'}
{'loss': '0.4232', 'grad_norm': '0.2353', 'learning_rate': '7.55e-05', 'entropy': '0.4271', 'num_tokens': '3.887e+07', 'mean_token_accuracy': '0.8855', 'epoch': '0.06371'}
{'loss': '0.8299', 'grad_norm': '0.2953', 'learning_rate': '7.548e-05', 'entropy': '0.752', 'num_tokens': '3.888e+07', 'mean_token_accuracy': '0.7926', 'epoch': '0.06372'}
{'loss': '0.8415', 'grad_norm': '0.4227', 'learning_rate': '7.545e-05', 'entropy': '0.8517', 'num_tokens': '3.888e+07', 'mean_token_accuracy': '0.7791', 'epoch': '0.06373'}
{'loss': '0.6745', 'grad_norm': '0.4139', 'learning_rate': '7.542e-05', 'entropy': '0.704', 'num_tokens': '3.889e+07', 'mean_token_accuracy': '0.8217', 'epoch': '0.06374'}
{'loss': '0.6666', 'grad_norm': '0.2906', 'learning_rate': '7.54e-05', 'entropy': '0.6688', 'num_tokens': '3.89e+07', 'mean_token_accuracy': '0.8332', 'epoch': '0.06375'}
{'loss': '0.6304', 'grad_norm': '0.3075', 'learning_rate': '7.537e-05', 'entropy': '0.6846', 'num_tokens': '3.89e+07', 'mean_token_accuracy': '0.8214', 'epoch': '0.06375'}
{'loss': '1.059', 'grad_norm': '0.5309', 'learning_rate': '7.534e-05', 'entropy': '1.025', 'num_tokens': '3.891e+07', 'mean_token_accuracy': '0.7381', 'epoch': '0.06376'}
{'loss': '1.032', 'grad_norm': '0.4443', 'learning_rate': '7.532e-05', 'entropy': '0.863', 'num_tokens': '3.891e+07', 'mean_token_accuracy': '0.7841', 'epoch': '0.06377'}
{'loss': '1.006', 'grad_norm': '0.4131', 'learning_rate': '7.529e-05', 'entropy': '0.8787', 'num_tokens': '3.892e+07', 'mean_token_accuracy': '0.7792', 'epoch': '0.06378'}
{'loss': '0.8828', 'grad_norm': '0.415', 'learning_rate': '7.526e-05', 'entropy': '0.9247', 'num_tokens': '3.893e+07', 'mean_token_accuracy': '0.779', 'epoch': '0.06379'}
{'loss': '0.9533', 'grad_norm': '0.4128', 'learning_rate': '7.524e-05', 'entropy': '1.063', 'num_tokens': '3.893e+07', 'mean_token_accuracy': '0.7361', 'epoch': '0.0638'}
{'loss': '0.8505', 'grad_norm': '0.4461', 'learning_rate': '7.521e-05', 'entropy': '0.9214', 'num_tokens': '3.894e+07', 'mean_token_accuracy': '0.767', 'epoch': '0.06381'}
{'loss': '0.7817', 'grad_norm': '0.3602', 'learning_rate': '7.518e-05', 'entropy': '0.7316', 'num_tokens': '3.894e+07', 'mean_token_accuracy': '0.8116', 'epoch': '0.06382'}
{'loss': '1.022', 'grad_norm': '0.4265', 'learning_rate': '7.516e-05', 'entropy': '0.8784', 'num_tokens': '3.895e+07', 'mean_token_accuracy': '0.7709', 'epoch': '0.06382'}
{'loss': '0.9314', 'grad_norm': '0.4596', 'learning_rate': '7.513e-05', 'entropy': '0.9169', 'num_tokens': '3.895e+07', 'mean_token_accuracy': '0.7566', 'epoch': '0.06383'}
{'loss': '0.8924', 'grad_norm': '0.4049', 'learning_rate': '7.51e-05', 'entropy': '0.9091', 'num_tokens': '3.896e+07', 'mean_token_accuracy': '0.7714', 'epoch': '0.06384'}
{'loss': '0.9991', 'grad_norm': '0.4374', 'learning_rate': '7.508e-05', 'entropy': '1.039', 'num_tokens': '3.896e+07', 'mean_token_accuracy': '0.7428', 'epoch': '0.06385'}
{'loss': '0.5195', 'grad_norm': '0.2481', 'learning_rate': '7.505e-05', 'entropy': '0.5523', 'num_tokens': '3.897e+07', 'mean_token_accuracy': '0.8671', 'epoch': '0.06386'}
{'loss': '0.6657', 'grad_norm': '0.2862', 'learning_rate': '7.502e-05', 'entropy': '0.6575', 'num_tokens': '3.898e+07', 'mean_token_accuracy': '0.8337', 'epoch': '0.06387'}
{'loss': '0.7071', 'grad_norm': '0.2985', 'learning_rate': '7.5e-05', 'entropy': '0.7736', 'num_tokens': '3.899e+07', 'mean_token_accuracy': '0.7972', 'epoch': '0.06388'}
{'loss': '1.064', 'grad_norm': '0.4508', 'learning_rate': '7.497e-05', 'entropy': '1.062', 'num_tokens': '3.899e+07', 'mean_token_accuracy': '0.7333', 'epoch': '0.06389'}
{'loss': '0.7746', 'grad_norm': '0.4494', 'learning_rate': '7.494e-05', 'entropy': '0.8476', 'num_tokens': '3.899e+07', 'mean_token_accuracy': '0.8052', 'epoch': '0.0639'}
{'loss': '0.7796', 'grad_norm': '0.3832', 'learning_rate': '7.492e-05', 'entropy': '0.7245', 'num_tokens': '3.9e+07', 'mean_token_accuracy': '0.8201', 'epoch': '0.0639'}
{'loss': '0.6503', 'grad_norm': '0.3531', 'learning_rate': '7.489e-05', 'entropy': '0.5915', 'num_tokens': '3.901e+07', 'mean_token_accuracy': '0.8536', 'epoch': '0.06391'}
{'loss': '1.048', 'grad_norm': '0.4154', 'learning_rate': '7.486e-05', 'entropy': '0.9989', 'num_tokens': '3.901e+07', 'mean_token_accuracy': '0.7636', 'epoch': '0.06392'}
{'loss': '0.6171', 'grad_norm': '0.3573', 'learning_rate': '7.484e-05', 'entropy': '0.6463', 'num_tokens': '3.902e+07', 'mean_token_accuracy': '0.8309', 'epoch': '0.06393'}
{'loss': '0.8654', 'grad_norm': '0.4741', 'learning_rate': '7.481e-05', 'entropy': '0.89', 'num_tokens': '3.902e+07', 'mean_token_accuracy': '0.774', 'epoch': '0.06394'}
{'loss': '0.6899', 'grad_norm': '0.385', 'learning_rate': '7.478e-05', 'entropy': '0.7719', 'num_tokens': '3.903e+07', 'mean_token_accuracy': '0.8003', 'epoch': '0.06395'}
{'loss': '0.9128', 'grad_norm': '0.42', 'learning_rate': '7.476e-05', 'entropy': '0.973', 'num_tokens': '3.904e+07', 'mean_token_accuracy': '0.7583', 'epoch': '0.06396'}
{'loss': '0.6685', 'grad_norm': '0.3313', 'learning_rate': '7.473e-05', 'entropy': '0.6533', 'num_tokens': '3.904e+07', 'mean_token_accuracy': '0.827', 'epoch': '0.06397'}
{'loss': '0.9796', 'grad_norm': '0.4818', 'learning_rate': '7.47e-05', 'entropy': '0.8803', 'num_tokens': '3.905e+07', 'mean_token_accuracy': '0.7816', 'epoch': '0.06398'}
{'loss': '0.6552', 'grad_norm': '0.4102', 'learning_rate': '7.468e-05', 'entropy': '0.6615', 'num_tokens': '3.905e+07', 'mean_token_accuracy': '0.8259', 'epoch': '0.06398'}
{'loss': '0.7913', 'grad_norm': '0.4056', 'learning_rate': '7.465e-05', 'entropy': '0.8547', 'num_tokens': '3.906e+07', 'mean_token_accuracy': '0.774', 'epoch': '0.06399'}
{'loss': '0.9209', 'grad_norm': '0.3963', 'learning_rate': '7.462e-05', 'entropy': '0.9479', 'num_tokens': '3.906e+07', 'mean_token_accuracy': '0.7622', 'epoch': '0.064'}
{'loss': '0.7113', 'grad_norm': '0.4349', 'learning_rate': '7.46e-05', 'entropy': '0.6909', 'num_tokens': '3.906e+07', 'mean_token_accuracy': '0.8224', 'epoch': '0.06401'}
{'loss': '1.148', 'grad_norm': '0.4971', 'learning_rate': '7.457e-05', 'entropy': '1.078', 'num_tokens': '3.907e+07', 'mean_token_accuracy': '0.7428', 'epoch': '0.06402'}
{'loss': '0.6433', 'grad_norm': '0.3312', 'learning_rate': '7.454e-05', 'entropy': '0.661', 'num_tokens': '3.907e+07', 'mean_token_accuracy': '0.8306', 'epoch': '0.06403'}
{'loss': '0.5755', 'grad_norm': '0.3621', 'learning_rate': '7.452e-05', 'entropy': '0.5681', 'num_tokens': '3.908e+07', 'mean_token_accuracy': '0.8545', 'epoch': '0.06404'}
{'loss': '0.7781', 'grad_norm': '0.415', 'learning_rate': '7.449e-05', 'entropy': '0.8394', 'num_tokens': '3.908e+07', 'mean_token_accuracy': '0.7819', 'epoch': '0.06405'}
{'loss': '0.9069', 'grad_norm': '0.3865', 'learning_rate': '7.446e-05', 'entropy': '0.9012', 'num_tokens': '3.909e+07', 'mean_token_accuracy': '0.7685', 'epoch': '0.06406'}
{'loss': '0.6875', 'grad_norm': '0.3471', 'learning_rate': '7.444e-05', 'entropy': '0.7325', 'num_tokens': '3.909e+07', 'mean_token_accuracy': '0.8112', 'epoch': '0.06406'}
{'loss': '0.9449', 'grad_norm': '0.4532', 'learning_rate': '7.441e-05', 'entropy': '0.9022', 'num_tokens': '3.91e+07', 'mean_token_accuracy': '0.7588', 'epoch': '0.06407'}
{'loss': '0.7579', 'grad_norm': '0.4016', 'learning_rate': '7.438e-05', 'entropy': '0.7872', 'num_tokens': '3.91e+07', 'mean_token_accuracy': '0.7956', 'epoch': '0.06408'}

 60%|██████    | 7242/12000 [4:17:11<2:15:02,  1.70s/it]
 60%|██████    | 7243/12000 [4:17:13<2:13:12,  1.68s/it]
                                                        

 60%|██████    | 7243/12000 [4:17:13<2:13:12,  1.68s/it]
 60%|██████    | 7244/12000 [4:17:14<2:09:27,  1.63s/it]
                                                        

 60%|██████    | 7244/12000 [4:17:14<2:09:27,  1.63s/it]
 60%|██████    | 7245/12000 [4:17:16<2:04:41,  1.57s/it]
                                                        

 60%|██████    | 7245/12000 [4:17:16<2:04:41,  1.57s/it]
 60%|██████    | 7246/12000 [4:17:17<2:04:32,  1.57s/it]
                                                        

 60%|██████    | 7246/12000 [4:17:17<2:04:32,  1.57s/it]
 60%|██████    | 7247/12000 [4:17:19<2:06:10,  1.59s/it]
                                                        

 60%|██████    | 7247/12000 [4:17:19<2:06:10,  1.59s/it]
 60%|██████    | 7248/12000 [4:17:21<2:25:20,  1.84s/it]
                                                        

 60%|██████    | 7248/12000 [4:17:21<2:25:20,  1.84s/it]
 60%|██████    | 7249/12000 [4:17:23<2:20:42,  1.78s/it]
                                                        

 60%|██████    | 7249/12000 [4:17:23<2:20:42,  1.78s/it]
 60%|██████    | 7250/12000 [4:17:27<3:01:30,  2.29s/it]
                                                        

 60%|██████    | 7250/12000 [4:17:27<3:01:30,  2.29s/it]
 60%|██████    | 7251/12000 [4:17:30<3:22:11,  2.55s/it]
                                                        

 60%|██████    | 7251/12000 [4:17:30<3:22:11,  2.55s/it]
 60%|██████    | 7252/12000 [4:17:32<3:08:02,  2.38s/it]
                                                        

 60%|██████    | 7252/12000 [4:17:32<3:08:02,  2.38s/it]
 60%|██████    | 7253/12000 [4:17:33<2:52:12,  2.18s/it]
                                                        

 60%|██████    | 7253/12000 [4:17:33<2:52:12,  2.18s/it]
 60%|██████    | 7254/12000 [4:17:35<2:39:10,  2.01s/it]
                                                        

 60%|██████    | 7254/12000 [4:17:35<2:39:10,  2.01s/it]
 60%|██████    | 7255/12000 [4:17:37<2:31:52,  1.92s/it]
                                                        

 60%|██████    | 7255/12000 [4:17:37<2:31:52,  1.92s/it]
 60%|██████    | 7256/12000 [4:17:38<2:19:43,  1.77s/it]
                                                        

 60%|██████    | 7256/12000 [4:17:38<2:19:43,  1.77s/it]
 60%|██████    | 7257/12000 [4:17:41<2:55:30,  2.22s/it]
                                                        

 60%|██████    | 7257/12000 [4:17:41<2:55:30,  2.22s/it]
 60%|██████    | 7258/12000 [4:17:43<2:44:49,  2.09s/it]
                                                        

 60%|██████    | 7258/12000 [4:17:43<2:44:49,  2.09s/it]
 60%|██████    | 7259/12000 [4:17:46<2:49:50,  2.15s/it]
                                                        

 60%|██████    | 7259/12000 [4:17:46<2:49:50,  2.15s/it]
 60%|██████    | 7260/12000 [4:17:48<2:52:15,  2.18s/it]
                                                        

 60%|██████    | 7260/12000 [4:17:48<2:52:15,  2.18s/it]
 61%|██████    | 7261/12000 [4:17:50<2:56:03,  2.23s/it]
                                                        

 61%|██████    | 7261/12000 [4:17:50<2:56:03,  2.23s/it]
 61%|██████    | 7262/12000 [4:17:52<2:39:56,  2.03s/it]
                                                        

 61%|██████    | 7262/12000 [4:17:52<2:39:56,  2.03s/it]
 61%|██████    | 7263/12000 [4:17:53<2:25:37,  1.84s/it]
                                                        

 61%|██████    | 7263/12000 [4:17:53<2:25:37,  1.84s/it]
 61%|██████    | 7264/12000 [4:17:55<2:20:03,  1.77s/it]
                                                        

 61%|██████    | 7264/12000 [4:17:55<2:20:03,  1.77s/it]
 61%|██████    | 7265/12000 [4:17:56<2:18:14,  1.75s/it]
                                                        

 61%|██████    | 7265/12000 [4:17:56<2:18:14,  1.75s/it]
 61%|██████    | 7266/12000 [4:17:58<2:20:00,  1.77s/it]
                                                        

 61%|██████    | 7266/12000 [4:17:58<2:20:00,  1.77s/it]
 61%|██████    | 7267/12000 [4:18:01<2:33:11,  1.94s/it]
                                                        

 61%|██████    | 7267/12000 [4:18:01<2:33:11,  1.94s/it]
 61%|██████    | 7268/12000 [4:18:03<2:48:59,  2.14s/it]
                                                        

 61%|██████    | 7268/12000 [4:18:03<2:48:59,  2.14s/it]
 61%|██████    | 7269/12000 [4:18:06<3:00:44,  2.29s/it]
                                                        

 61%|██████    | 7269/12000 [4:18:06<3:00:44,  2.29s/it]
 61%|██████    | 7270/12000 [4:18:07<2:44:05,  2.08s/it]
                                                        

 61%|██████    | 7270/12000 [4:18:07<2:44:05,  2.08s/it]
 61%|██████    | 7271/12000 [4:18:10<2:46:26,  2.11s/it]
                                                        

 61%|██████    | 7271/12000 [4:18:10<2:46:26,  2.11s/it]
 61%|██████    | 7272/12000 [4:18:15<4:01:00,  3.06s/it]
                                                        

 61%|██████    | 7272/12000 [4:18:15<4:01:00,  3.06s/it]
 61%|██████    | 7273/12000 [4:18:18<4:06:04,  3.12s/it]
                                                        

 61%|██████    | 7273/12000 [4:18:18<4:06:04,  3.12s/it]
 61%|██████    | 7274/12000 [4:18:20<3:33:44,  2.71s/it]
                                                        

 61%|██████    | 7274/12000 [4:18:20<3:33:44,  2.71s/it]
 61%|██████    | 7275/12000 [4:18:26<4:46:49,  3.64s/it]
                                                        

 61%|██████    | 7275/12000 [4:18:26<4:46:49,  3.64s/it]
 61%|██████    | 7276/12000 [4:18:28<4:06:31,  3.13s/it]
                                                        

 61%|██████    | 7276/12000 [4:18:28<4:06:31,  3.13s/it]
 61%|██████    | 7277/12000 [4:18:32<4:40:07,  3.56s/it]
                                                        

 61%|██████    | 7277/12000 [4:18:32<4:40:07,  3.56s/it]
 61%|██████    | 7278/12000 [4:18:34<3:49:55,  2.92s/it]
                                                        

 61%|██████    | 7278/12000 [4:18:34<3:49:55,  2.92s/it]
 61%|██████    | 7279/12000 [4:18:35<3:16:35,  2.50s/it]
                                                        

 61%|██████    | 7279/12000 [4:18:35<3:16:35,  2.50s/it]
 61%|██████    | 7280/12000 [4:18:37<2:56:55,  2.25s/it]
                                                        

 61%|██████    | 7280/12000 [4:18:37<2:56:55,  2.25s/it]
 61%|██████    | 7281/12000 [4:18:38<2:43:54,  2.08s/it]
                                                        

 61%|██████    | 7281/12000 [4:18:38<2:43:54,  2.08s/it]
 61%|██████    | 7282/12000 [4:18:40<2:31:58,  1.93s/it]
                                                        

 61%|██████    | 7282/12000 [4:18:40<2:31:58,  1.93s/it]
 61%|██████    | 7283/12000 [4:18:42<2:24:28,  1.84s/it]
                                                        

 61%|██████    | 7283/12000 [4:18:42<2:24:28,  1.84s/it]
 61%|██████    | 7284/12000 [4:18:43<2:12:04,  1.68s/it]
                                                        

 61%|██████    | 7284/12000 [4:18:43<2:12:04,  1.68s/it]
 61%|██████    | 7285/12000 [4:18:45<2:23:51,  1.83s/it]
                                                        

 61%|██████    | 7285/12000 [4:18:45<2:23:51,  1.83s/it]
 61%|██████    | 7286/12000 [4:18:47<2:32:18,  1.94s/it]
                                                        

 61%|██████    | 7286/12000 [4:18:47<2:32:18,  1.94s/it]
 61%|██████    | 7287/12000 [4:18:49<2:29:28,  1.90s/it]
                                                        

 61%|██████    | 7287/12000 [4:18:49<2:29:28,  1.90s/it]
 61%|██████    | 7288/12000 [4:18:51<2:19:20,  1.77s/it]
                                                        

 61%|██████    | 7288/12000 [4:18:51<2:19:20,  1.77s/it]
 61%|██████    | 7289/12000 [4:18:53<2:37:33,  2.01s/it]
                                                        
{'loss': '0.773', 'grad_norm': '0.497', 'learning_rate': '7.436e-05', 'entropy': '0.8698', 'num_tokens': '3.911e+07', 'mean_token_accuracy': '0.7816', 'epoch': '0.06409'}
{'loss': '0.5059', 'grad_norm': '0.3288', 'learning_rate': '7.433e-05', 'entropy': '0.5394', 'num_tokens': '3.911e+07', 'mean_token_accuracy': '0.8568', 'epoch': '0.0641'}
{'loss': '0.6941', 'grad_norm': '0.3631', 'learning_rate': '7.43e-05', 'entropy': '0.713', 'num_tokens': '3.912e+07', 'mean_token_accuracy': '0.8125', 'epoch': '0.06411'}
{'loss': '0.713', 'grad_norm': '0.3622', 'learning_rate': '7.428e-05', 'entropy': '0.7228', 'num_tokens': '3.912e+07', 'mean_token_accuracy': '0.8058', 'epoch': '0.06412'}
{'loss': '0.6602', 'grad_norm': '0.4239', 'learning_rate': '7.425e-05', 'entropy': '0.6489', 'num_tokens': '3.913e+07', 'mean_token_accuracy': '0.8317', 'epoch': '0.06413'}
{'loss': '0.7568', 'grad_norm': '0.4407', 'learning_rate': '7.422e-05', 'entropy': '0.7718', 'num_tokens': '3.913e+07', 'mean_token_accuracy': '0.8013', 'epoch': '0.06413'}
{'loss': '0.7315', 'grad_norm': '0.336', 'learning_rate': '7.42e-05', 'entropy': '0.8011', 'num_tokens': '3.914e+07', 'mean_token_accuracy': '0.7967', 'epoch': '0.06414'}
{'loss': '1.07', 'grad_norm': '0.3922', 'learning_rate': '7.417e-05', 'entropy': '1.137', 'num_tokens': '3.914e+07', 'mean_token_accuracy': '0.7234', 'epoch': '0.06415'}
{'loss': '1.019', 'grad_norm': '0.4907', 'learning_rate': '7.414e-05', 'entropy': '1.086', 'num_tokens': '3.915e+07', 'mean_token_accuracy': '0.7348', 'epoch': '0.06416'}
{'loss': '1.102', 'grad_norm': '0.4439', 'learning_rate': '7.412e-05', 'entropy': '1.108', 'num_tokens': '3.915e+07', 'mean_token_accuracy': '0.7267', 'epoch': '0.06417'}
{'loss': '1.226', 'grad_norm': '0.444', 'learning_rate': '7.409e-05', 'entropy': '1.149', 'num_tokens': '3.916e+07', 'mean_token_accuracy': '0.7289', 'epoch': '0.06418'}
{'loss': '0.9411', 'grad_norm': '0.3732', 'learning_rate': '7.406e-05', 'entropy': '0.9525', 'num_tokens': '3.916e+07', 'mean_token_accuracy': '0.7589', 'epoch': '0.06419'}
{'loss': '0.7007', 'grad_norm': '0.4462', 'learning_rate': '7.404e-05', 'entropy': '0.7414', 'num_tokens': '3.916e+07', 'mean_token_accuracy': '0.7968', 'epoch': '0.0642'}
{'loss': '0.7306', 'grad_norm': '0.3974', 'learning_rate': '7.401e-05', 'entropy': '0.6739', 'num_tokens': '3.917e+07', 'mean_token_accuracy': '0.8302', 'epoch': '0.06421'}
{'loss': '1.046', 'grad_norm': '0.4628', 'learning_rate': '7.399e-05', 'entropy': '0.9946', 'num_tokens': '3.917e+07', 'mean_token_accuracy': '0.754', 'epoch': '0.06421'}
{'loss': '1.39', 'grad_norm': '0.4431', 'learning_rate': '7.396e-05', 'entropy': '1.276', 'num_tokens': '3.918e+07', 'mean_token_accuracy': '0.7004', 'epoch': '0.06422'}
{'loss': '0.7757', 'grad_norm': '0.452', 'learning_rate': '7.393e-05', 'entropy': '0.7779', 'num_tokens': '3.919e+07', 'mean_token_accuracy': '0.7974', 'epoch': '0.06423'}
{'loss': '0.6487', 'grad_norm': '0.3424', 'learning_rate': '7.391e-05', 'entropy': '0.6613', 'num_tokens': '3.919e+07', 'mean_token_accuracy': '0.829', 'epoch': '0.06424'}
{'loss': '0.5615', 'grad_norm': '0.2914', 'learning_rate': '7.388e-05', 'entropy': '0.5989', 'num_tokens': '3.92e+07', 'mean_token_accuracy': '0.8463', 'epoch': '0.06425'}
{'loss': '0.8512', 'grad_norm': '0.4093', 'learning_rate': '7.385e-05', 'entropy': '0.8802', 'num_tokens': '3.92e+07', 'mean_token_accuracy': '0.7774', 'epoch': '0.06426'}
{'loss': '0.6841', 'grad_norm': '0.3894', 'learning_rate': '7.383e-05', 'entropy': '0.7692', 'num_tokens': '3.921e+07', 'mean_token_accuracy': '0.8103', 'epoch': '0.06427'}
{'loss': '0.8601', 'grad_norm': '0.4228', 'learning_rate': '7.38e-05', 'entropy': '0.8645', 'num_tokens': '3.921e+07', 'mean_token_accuracy': '0.7852', 'epoch': '0.06428'}
{'loss': '0.8862', 'grad_norm': '0.4818', 'learning_rate': '7.377e-05', 'entropy': '0.8615', 'num_tokens': '3.922e+07', 'mean_token_accuracy': '0.7876', 'epoch': '0.06429'}
{'loss': '1.075', 'grad_norm': '0.4571', 'learning_rate': '7.375e-05', 'entropy': '0.9222', 'num_tokens': '3.922e+07', 'mean_token_accuracy': '0.7846', 'epoch': '0.06429'}
{'loss': '0.7167', 'grad_norm': '0.468', 'learning_rate': '7.372e-05', 'entropy': '0.764', 'num_tokens': '3.923e+07', 'mean_token_accuracy': '0.8099', 'epoch': '0.0643'}
{'loss': '0.9866', 'grad_norm': '0.3961', 'learning_rate': '7.369e-05', 'entropy': '1.197', 'num_tokens': '3.923e+07', 'mean_token_accuracy': '0.7247', 'epoch': '0.06431'}
{'loss': '0.6792', 'grad_norm': '0.3687', 'learning_rate': '7.367e-05', 'entropy': '0.675', 'num_tokens': '3.924e+07', 'mean_token_accuracy': '0.8229', 'epoch': '0.06432'}
{'loss': '0.7171', 'grad_norm': '0.3978', 'learning_rate': '7.364e-05', 'entropy': '0.786', 'num_tokens': '3.924e+07', 'mean_token_accuracy': '0.7855', 'epoch': '0.06433'}
{'loss': '0.7409', 'grad_norm': '0.4823', 'learning_rate': '7.361e-05', 'entropy': '0.76', 'num_tokens': '3.925e+07', 'mean_token_accuracy': '0.7959', 'epoch': '0.06434'}
{'loss': '1.19', 'grad_norm': '0.4179', 'learning_rate': '7.359e-05', 'entropy': '1.14', 'num_tokens': '3.926e+07', 'mean_token_accuracy': '0.7251', 'epoch': '0.06435'}
{'loss': '0.8686', 'grad_norm': '0.4208', 'learning_rate': '7.356e-05', 'entropy': '0.9387', 'num_tokens': '3.926e+07', 'mean_token_accuracy': '0.7713', 'epoch': '0.06436'}
{'loss': '0.7497', 'grad_norm': '0.3188', 'learning_rate': '7.353e-05', 'entropy': '0.7585', 'num_tokens': '3.927e+07', 'mean_token_accuracy': '0.801', 'epoch': '0.06436'}
{'loss': '0.7375', 'grad_norm': '0.3985', 'learning_rate': '7.351e-05', 'entropy': '0.7444', 'num_tokens': '3.928e+07', 'mean_token_accuracy': '0.8066', 'epoch': '0.06437'}
{'loss': '0.5639', 'grad_norm': '0.2672', 'learning_rate': '7.348e-05', 'entropy': '0.6805', 'num_tokens': '3.929e+07', 'mean_token_accuracy': '0.8308', 'epoch': '0.06438'}
{'loss': '0.8645', 'grad_norm': '0.45', 'learning_rate': '7.345e-05', 'entropy': '0.8599', 'num_tokens': '3.929e+07', 'mean_token_accuracy': '0.7905', 'epoch': '0.06439'}
{'loss': '1.485', 'grad_norm': '0.4181', 'learning_rate': '7.343e-05', 'entropy': '1.22', 'num_tokens': '3.93e+07', 'mean_token_accuracy': '0.7217', 'epoch': '0.0644'}
{'loss': '0.7454', 'grad_norm': '0.3268', 'learning_rate': '7.34e-05', 'entropy': '0.7456', 'num_tokens': '3.931e+07', 'mean_token_accuracy': '0.797', 'epoch': '0.06441'}
{'loss': '0.6998', 'grad_norm': '0.411', 'learning_rate': '7.337e-05', 'entropy': '0.6579', 'num_tokens': '3.931e+07', 'mean_token_accuracy': '0.829', 'epoch': '0.06442'}
{'loss': '0.5892', 'grad_norm': '0.4139', 'learning_rate': '7.335e-05', 'entropy': '0.586', 'num_tokens': '3.931e+07', 'mean_token_accuracy': '0.8406', 'epoch': '0.06443'}
{'loss': '0.7676', 'grad_norm': '0.4396', 'learning_rate': '7.332e-05', 'entropy': '0.7422', 'num_tokens': '3.932e+07', 'mean_token_accuracy': '0.81', 'epoch': '0.06444'}
{'loss': '0.6024', 'grad_norm': '0.4123', 'learning_rate': '7.329e-05', 'entropy': '0.6272', 'num_tokens': '3.932e+07', 'mean_token_accuracy': '0.8353', 'epoch': '0.06444'}
{'loss': '0.6744', 'grad_norm': '0.3923', 'learning_rate': '7.327e-05', 'entropy': '0.731', 'num_tokens': '3.933e+07', 'mean_token_accuracy': '0.8041', 'epoch': '0.06445'}
{'loss': '0.6401', 'grad_norm': '0.3039', 'learning_rate': '7.324e-05', 'entropy': '0.6495', 'num_tokens': '3.933e+07', 'mean_token_accuracy': '0.8288', 'epoch': '0.06446'}
{'loss': '0.7423', 'grad_norm': '0.3372', 'learning_rate': '7.321e-05', 'entropy': '0.7464', 'num_tokens': '3.934e+07', 'mean_token_accuracy': '0.8194', 'epoch': '0.06447'}
{'loss': '0.7506', 'grad_norm': '0.3572', 'learning_rate': '7.319e-05', 'entropy': '0.737', 'num_tokens': '3.934e+07', 'mean_token_accuracy': '0.8088', 'epoch': '0.06448'}
{'loss': '0.656', 'grad_norm': '0.322', 'learning_rate': '7.316e-05', 'entropy': '0.6764', 'num_tokens': '3.935e+07', 'mean_token_accuracy': '0.8255', 'epoch': '0.06449'}
{'loss': '0.8479', 'grad_norm': '0.4877', 'learning_rate': '7.313e-05', 'entropy': '0.8551', 'num_tokens': '3.935e+07', 'mean_token_accuracy': '0.7777', 'epoch': '0.0645'}

 61%|██████    | 7289/12000 [4:18:53<2:37:33,  2.01s/it]
 61%|██████    | 7290/12000 [4:18:55<2:36:35,  1.99s/it]
                                                        

 61%|██████    | 7290/12000 [4:18:55<2:36:35,  1.99s/it]
 61%|██████    | 7291/12000 [4:18:57<2:24:28,  1.84s/it]
                                                        

 61%|██████    | 7291/12000 [4:18:57<2:24:28,  1.84s/it]
 61%|██████    | 7292/12000 [4:18:58<2:18:09,  1.76s/it]
                                                        

 61%|██████    | 7292/12000 [4:18:58<2:18:09,  1.76s/it]
 61%|██████    | 7293/12000 [4:19:00<2:14:53,  1.72s/it]
                                                        

 61%|██████    | 7293/12000 [4:19:00<2:14:53,  1.72s/it]
 61%|██████    | 7294/12000 [4:19:03<2:55:18,  2.24s/it]
                                                        

 61%|██████    | 7294/12000 [4:19:03<2:55:18,  2.24s/it]
 61%|██████    | 7295/12000 [4:19:05<2:38:12,  2.02s/it]
                                                        

 61%|██████    | 7295/12000 [4:19:05<2:38:12,  2.02s/it]
 61%|██████    | 7296/12000 [4:19:06<2:24:37,  1.84s/it]
                                                        

 61%|██████    | 7296/12000 [4:19:06<2:24:37,  1.84s/it]
 61%|██████    | 7297/12000 [4:19:08<2:19:03,  1.77s/it]
                                                        

 61%|██████    | 7297/12000 [4:19:08<2:19:03,  1.77s/it]
 61%|██████    | 7298/12000 [4:19:09<2:14:21,  1.71s/it]
                                                        

 61%|██████    | 7298/12000 [4:19:09<2:14:21,  1.71s/it]
 61%|██████    | 7299/12000 [4:19:11<2:07:02,  1.62s/it]
                                                        

 61%|██████    | 7299/12000 [4:19:11<2:07:02,  1.62s/it]
 61%|██████    | 7300/12000 [4:19:12<2:06:50,  1.62s/it]
                                                        

 61%|██████    | 7300/12000 [4:19:12<2:06:50,  1.62s/it]
 61%|██████    | 7301/12000 [4:19:16<2:43:27,  2.09s/it]
                                                        

 61%|██████    | 7301/12000 [4:19:16<2:43:27,  2.09s/it]
 61%|██████    | 7302/12000 [4:19:18<2:54:38,  2.23s/it]
                                                        

 61%|██████    | 7302/12000 [4:19:18<2:54:38,  2.23s/it]
 61%|██████    | 7303/12000 [4:19:20<2:52:45,  2.21s/it]
                                                        

 61%|██████    | 7303/12000 [4:19:20<2:52:45,  2.21s/it]
 61%|██████    | 7304/12000 [4:19:22<2:45:41,  2.12s/it]
                                                        

 61%|██████    | 7304/12000 [4:19:22<2:45:41,  2.12s/it]
 61%|██████    | 7305/12000 [4:19:26<3:21:53,  2.58s/it]
                                                        

 61%|██████    | 7305/12000 [4:19:26<3:21:53,  2.58s/it]
 61%|██████    | 7306/12000 [4:19:28<3:06:49,  2.39s/it]
                                                        

 61%|██████    | 7306/12000 [4:19:28<3:06:49,  2.39s/it]
 61%|██████    | 7307/12000 [4:19:29<2:48:11,  2.15s/it]
                                                        

 61%|██████    | 7307/12000 [4:19:29<2:48:11,  2.15s/it]
 61%|██████    | 7308/12000 [4:19:31<2:32:18,  1.95s/it]
                                                        

 61%|██████    | 7308/12000 [4:19:31<2:32:18,  1.95s/it]
 61%|██████    | 7309/12000 [4:19:33<2:25:35,  1.86s/it]
                                                        

 61%|██████    | 7309/12000 [4:19:33<2:25:35,  1.86s/it]
 61%|██████    | 7310/12000 [4:19:34<2:21:14,  1.81s/it]
                                                        

 61%|██████    | 7310/12000 [4:19:34<2:21:14,  1.81s/it]
 61%|██████    | 7311/12000 [4:19:37<2:52:08,  2.20s/it]
                                                        

 61%|██████    | 7311/12000 [4:19:37<2:52:08,  2.20s/it]
 61%|██████    | 7312/12000 [4:19:40<2:57:03,  2.27s/it]
                                                        

 61%|██████    | 7312/12000 [4:19:40<2:57:03,  2.27s/it]
 61%|██████    | 7313/12000 [4:19:41<2:40:46,  2.06s/it]
                                                        

 61%|██████    | 7313/12000 [4:19:41<2:40:46,  2.06s/it]
 61%|██████    | 7314/12000 [4:19:44<2:43:13,  2.09s/it]
                                                        

 61%|██████    | 7314/12000 [4:19:44<2:43:13,  2.09s/it]
 61%|██████    | 7315/12000 [4:19:45<2:33:07,  1.96s/it]
                                                        

 61%|██████    | 7315/12000 [4:19:45<2:33:07,  1.96s/it]
 61%|██████    | 7316/12000 [4:19:49<3:09:07,  2.42s/it]
                                                        

 61%|██████    | 7316/12000 [4:19:49<3:09:07,  2.42s/it]
 61%|██████    | 7317/12000 [4:19:50<2:46:36,  2.13s/it]
                                                        

 61%|██████    | 7317/12000 [4:19:50<2:46:36,  2.13s/it]
 61%|██████    | 7318/12000 [4:19:52<2:29:11,  1.91s/it]
                                                        

 61%|██████    | 7318/12000 [4:19:52<2:29:11,  1.91s/it]
 61%|██████    | 7319/12000 [4:19:54<2:32:42,  1.96s/it]
                                                        

 61%|██████    | 7319/12000 [4:19:54<2:32:42,  1.96s/it]
 61%|██████    | 7320/12000 [4:19:55<2:20:50,  1.81s/it]
                                                        

 61%|██████    | 7320/12000 [4:19:55<2:20:50,  1.81s/it]
 61%|██████    | 7321/12000 [4:19:57<2:28:02,  1.90s/it]
                                                        

 61%|██████    | 7321/12000 [4:19:57<2:28:02,  1.90s/it]
 61%|██████    | 7322/12000 [4:19:59<2:24:13,  1.85s/it]
                                                        

 61%|██████    | 7322/12000 [4:19:59<2:24:13,  1.85s/it]
 61%|██████    | 7323/12000 [4:20:00<2:16:08,  1.75s/it]
                                                        

 61%|██████    | 7323/12000 [4:20:00<2:16:08,  1.75s/it]
 61%|██████    | 7324/12000 [4:20:03<2:30:43,  1.93s/it]
                                                        

 61%|██████    | 7324/12000 [4:20:03<2:30:43,  1.93s/it]
 61%|██████    | 7325/12000 [4:20:04<2:21:17,  1.81s/it]
                                                        

 61%|██████    | 7325/12000 [4:20:04<2:21:17,  1.81s/it]
 61%|██████    | 7326/12000 [4:20:06<2:26:04,  1.88s/it]
                                                        

 61%|██████    | 7326/12000 [4:20:06<2:26:04,  1.88s/it]
 61%|██████    | 7327/12000 [4:20:08<2:26:56,  1.89s/it]
                                                        

 61%|██████    | 7327/12000 [4:20:08<2:26:56,  1.89s/it]
 61%|██████    | 7328/12000 [4:20:10<2:20:18,  1.80s/it]
                                                        

 61%|██████    | 7328/12000 [4:20:10<2:20:18,  1.80s/it]
 61%|██████    | 7329/12000 [4:20:12<2:26:22,  1.88s/it]
                                                        

 61%|██████    | 7329/12000 [4:20:12<2:26:22,  1.88s/it]
 61%|██████    | 7330/12000 [4:20:13<2:17:51,  1.77s/it]
                                                        

 61%|██████    | 7330/12000 [4:20:13<2:17:51,  1.77s/it]
 61%|██████    | 7331/12000 [4:20:15<2:14:31,  1.73s/it]
                                                        

 61%|██████    | 7331/12000 [4:20:15<2:14:31,  1.73s/it]
 61%|██████    | 7332/12000 [4:20:17<2:09:47,  1.67s/it]
                                                        

 61%|██████    | 7332/12000 [4:20:17<2:09:47,  1.67s/it]
 61%|██████    | 7333/12000 [4:20:19<2:20:32,  1.81s/it]
                                                        

 61%|██████    | 7333/12000 [4:20:19<2:20:32,  1.81s/it]
 61%|██████    | 7334/12000 [4:20:20<2:18:36,  1.78s/it]
                                                        

 61%|██████    | 7334/12000 [4:20:20<2:18:36,  1.78s/it]
 61%|██████    | 7335/12000 [4:20:23<2:25:35,  1.87s/it]
                                                        

 61%|██████    | 7335/12000 [4:20:23<2:25:35,  1.87s/it]
 61%|██████    | 7336/12000 [4:20:24<2:22:14,  1.83s/it]
                                                        
{'loss': '0.5399', 'grad_norm': '0.3131', 'learning_rate': '7.311e-05', 'entropy': '0.5627', 'num_tokens': '3.936e+07', 'mean_token_accuracy': '0.8456', 'epoch': '0.06451'}
{'loss': '0.4927', 'grad_norm': '0.2954', 'learning_rate': '7.308e-05', 'entropy': '0.4712', 'num_tokens': '3.936e+07', 'mean_token_accuracy': '0.8781', 'epoch': '0.06452'}
{'loss': '0.8801', 'grad_norm': '0.3576', 'learning_rate': '7.305e-05', 'entropy': '0.8123', 'num_tokens': '3.937e+07', 'mean_token_accuracy': '0.7934', 'epoch': '0.06452'}
{'loss': '1.318', 'grad_norm': '0.5267', 'learning_rate': '7.303e-05', 'entropy': '1.366', 'num_tokens': '3.937e+07', 'mean_token_accuracy': '0.6824', 'epoch': '0.06453'}
{'loss': '0.7172', 'grad_norm': '0.3965', 'learning_rate': '7.3e-05', 'entropy': '0.7861', 'num_tokens': '3.938e+07', 'mean_token_accuracy': '0.8032', 'epoch': '0.06454'}
{'loss': '0.9224', 'grad_norm': '0.4248', 'learning_rate': '7.298e-05', 'entropy': '0.8802', 'num_tokens': '3.938e+07', 'mean_token_accuracy': '0.7617', 'epoch': '0.06455'}
{'loss': '0.6157', 'grad_norm': '0.3519', 'learning_rate': '7.295e-05', 'entropy': '0.6054', 'num_tokens': '3.939e+07', 'mean_token_accuracy': '0.8295', 'epoch': '0.06456'}
{'loss': '0.6293', 'grad_norm': '0.3896', 'learning_rate': '7.292e-05', 'entropy': '0.671', 'num_tokens': '3.939e+07', 'mean_token_accuracy': '0.817', 'epoch': '0.06457'}
{'loss': '0.7583', 'grad_norm': '0.4644', 'learning_rate': '7.29e-05', 'entropy': '0.8025', 'num_tokens': '3.939e+07', 'mean_token_accuracy': '0.7954', 'epoch': '0.06458'}
{'loss': '0.9795', 'grad_norm': '0.4449', 'learning_rate': '7.287e-05', 'entropy': '1.065', 'num_tokens': '3.94e+07', 'mean_token_accuracy': '0.7355', 'epoch': '0.06459'}
{'loss': '0.9068', 'grad_norm': '0.461', 'learning_rate': '7.284e-05', 'entropy': '0.8683', 'num_tokens': '3.94e+07', 'mean_token_accuracy': '0.787', 'epoch': '0.06459'}
{'loss': '0.9325', 'grad_norm': '0.4084', 'learning_rate': '7.282e-05', 'entropy': '0.9233', 'num_tokens': '3.941e+07', 'mean_token_accuracy': '0.7704', 'epoch': '0.0646'}
{'loss': '1.191', 'grad_norm': '0.5744', 'learning_rate': '7.279e-05', 'entropy': '1.065', 'num_tokens': '3.941e+07', 'mean_token_accuracy': '0.7415', 'epoch': '0.06461'}
{'loss': '0.7706', 'grad_norm': '0.4262', 'learning_rate': '7.276e-05', 'entropy': '0.8392', 'num_tokens': '3.942e+07', 'mean_token_accuracy': '0.7813', 'epoch': '0.06462'}
{'loss': '0.729', 'grad_norm': '0.4257', 'learning_rate': '7.274e-05', 'entropy': '0.7356', 'num_tokens': '3.942e+07', 'mean_token_accuracy': '0.8106', 'epoch': '0.06463'}
{'loss': '1.039', 'grad_norm': '0.4102', 'learning_rate': '7.271e-05', 'entropy': '1.003', 'num_tokens': '3.943e+07', 'mean_token_accuracy': '0.7488', 'epoch': '0.06464'}
{'loss': '1.246', 'grad_norm': '0.3626', 'learning_rate': '7.268e-05', 'entropy': '1.238', 'num_tokens': '3.944e+07', 'mean_token_accuracy': '0.6955', 'epoch': '0.06465'}
{'loss': '1.027', 'grad_norm': '0.4274', 'learning_rate': '7.266e-05', 'entropy': '0.9808', 'num_tokens': '3.944e+07', 'mean_token_accuracy': '0.7585', 'epoch': '0.06466'}
{'loss': '0.7707', 'grad_norm': '0.3573', 'learning_rate': '7.263e-05', 'entropy': '0.7255', 'num_tokens': '3.945e+07', 'mean_token_accuracy': '0.8143', 'epoch': '0.06467'}
{'loss': '0.9206', 'grad_norm': '0.3664', 'learning_rate': '7.26e-05', 'entropy': '0.9134', 'num_tokens': '3.945e+07', 'mean_token_accuracy': '0.7687', 'epoch': '0.06467'}
{'loss': '1.022', 'grad_norm': '0.3855', 'learning_rate': '7.258e-05', 'entropy': '1.002', 'num_tokens': '3.946e+07', 'mean_token_accuracy': '0.7507', 'epoch': '0.06468'}
{'loss': '0.7412', 'grad_norm': '0.4244', 'learning_rate': '7.255e-05', 'entropy': '0.7504', 'num_tokens': '3.946e+07', 'mean_token_accuracy': '0.7961', 'epoch': '0.06469'}
{'loss': '0.8564', 'grad_norm': '0.3385', 'learning_rate': '7.252e-05', 'entropy': '1.034', 'num_tokens': '3.947e+07', 'mean_token_accuracy': '0.7449', 'epoch': '0.0647'}
{'loss': '0.8564', 'grad_norm': '0.4307', 'learning_rate': '7.25e-05', 'entropy': '0.8823', 'num_tokens': '3.947e+07', 'mean_token_accuracy': '0.777', 'epoch': '0.06471'}
{'loss': '0.649', 'grad_norm': '0.3799', 'learning_rate': '7.247e-05', 'entropy': '0.7853', 'num_tokens': '3.948e+07', 'mean_token_accuracy': '0.8176', 'epoch': '0.06472'}
{'loss': '1.086', 'grad_norm': '0.4474', 'learning_rate': '7.245e-05', 'entropy': '0.9849', 'num_tokens': '3.948e+07', 'mean_token_accuracy': '0.7729', 'epoch': '0.06473'}
{'loss': '0.7139', 'grad_norm': '0.3596', 'learning_rate': '7.242e-05', 'entropy': '0.7275', 'num_tokens': '3.948e+07', 'mean_token_accuracy': '0.821', 'epoch': '0.06474'}
{'loss': '0.7706', 'grad_norm': '0.3714', 'learning_rate': '7.239e-05', 'entropy': '0.8034', 'num_tokens': '3.949e+07', 'mean_token_accuracy': '0.8087', 'epoch': '0.06475'}
{'loss': '0.757', 'grad_norm': '0.3535', 'learning_rate': '7.237e-05', 'entropy': '0.7844', 'num_tokens': '3.949e+07', 'mean_token_accuracy': '0.8069', 'epoch': '0.06475'}
{'loss': '0.8676', 'grad_norm': '0.4349', 'learning_rate': '7.234e-05', 'entropy': '0.9174', 'num_tokens': '3.95e+07', 'mean_token_accuracy': '0.7664', 'epoch': '0.06476'}
{'loss': '0.8351', 'grad_norm': '0.4144', 'learning_rate': '7.231e-05', 'entropy': '0.7308', 'num_tokens': '3.95e+07', 'mean_token_accuracy': '0.8223', 'epoch': '0.06477'}
{'loss': '0.8015', 'grad_norm': '0.3596', 'learning_rate': '7.229e-05', 'entropy': '0.7512', 'num_tokens': '3.951e+07', 'mean_token_accuracy': '0.8224', 'epoch': '0.06478'}
{'loss': '1.113', 'grad_norm': '0.5351', 'learning_rate': '7.226e-05', 'entropy': '0.9938', 'num_tokens': '3.951e+07', 'mean_token_accuracy': '0.7581', 'epoch': '0.06479'}
{'loss': '0.9073', 'grad_norm': '0.4408', 'learning_rate': '7.223e-05', 'entropy': '0.9204', 'num_tokens': '3.952e+07', 'mean_token_accuracy': '0.7723', 'epoch': '0.0648'}
{'loss': '1.036', 'grad_norm': '0.4366', 'learning_rate': '7.221e-05', 'entropy': '1.065', 'num_tokens': '3.952e+07', 'mean_token_accuracy': '0.7461', 'epoch': '0.06481'}
{'loss': '0.8329', 'grad_norm': '0.3387', 'learning_rate': '7.218e-05', 'entropy': '0.8806', 'num_tokens': '3.952e+07', 'mean_token_accuracy': '0.7706', 'epoch': '0.06482'}
{'loss': '0.651', 'grad_norm': '0.3682', 'learning_rate': '7.215e-05', 'entropy': '0.6398', 'num_tokens': '3.953e+07', 'mean_token_accuracy': '0.8375', 'epoch': '0.06483'}
{'loss': '0.891', 'grad_norm': '0.3699', 'learning_rate': '7.213e-05', 'entropy': '0.9421', 'num_tokens': '3.953e+07', 'mean_token_accuracy': '0.7715', 'epoch': '0.06483'}
{'loss': '0.8198', 'grad_norm': '0.4466', 'learning_rate': '7.21e-05', 'entropy': '0.7984', 'num_tokens': '3.954e+07', 'mean_token_accuracy': '0.7957', 'epoch': '0.06484'}
{'loss': '0.6652', 'grad_norm': '0.351', 'learning_rate': '7.207e-05', 'entropy': '0.6607', 'num_tokens': '3.955e+07', 'mean_token_accuracy': '0.8377', 'epoch': '0.06485'}
{'loss': '1.235', 'grad_norm': '0.5182', 'learning_rate': '7.205e-05', 'entropy': '1.074', 'num_tokens': '3.955e+07', 'mean_token_accuracy': '0.7376', 'epoch': '0.06486'}
{'loss': '0.5932', 'grad_norm': '0.3874', 'learning_rate': '7.202e-05', 'entropy': '0.6142', 'num_tokens': '3.956e+07', 'mean_token_accuracy': '0.8349', 'epoch': '0.06487'}
{'loss': '1.089', 'grad_norm': '0.4428', 'learning_rate': '7.2e-05', 'entropy': '1.032', 'num_tokens': '3.956e+07', 'mean_token_accuracy': '0.7586', 'epoch': '0.06488'}
{'loss': '1.144', 'grad_norm': '0.4886', 'learning_rate': '7.197e-05', 'entropy': '1.042', 'num_tokens': '3.956e+07', 'mean_token_accuracy': '0.7395', 'epoch': '0.06489'}
{'loss': '0.6714', 'grad_norm': '0.4051', 'learning_rate': '7.194e-05', 'entropy': '0.6998', 'num_tokens': '3.957e+07', 'mean_token_accuracy': '0.8155', 'epoch': '0.0649'}
{'loss': '0.8125', 'grad_norm': '0.4287', 'learning_rate': '7.192e-05', 'entropy': '0.8819', 'num_tokens': '3.957e+07', 'mean_token_accuracy': '0.7776', 'epoch': '0.0649'}
{'loss': '0.5816', 'grad_norm': '0.334', 'learning_rate': '7.189e-05', 'entropy': '0.5777', 'num_tokens': '3.958e+07', 'mean_token_accuracy': '0.8561', 'epoch': '0.06491'}

 61%|██████    | 7336/12000 [4:20:24<2:22:14,  1.83s/it]
 61%|██████    | 7337/12000 [4:20:26<2:24:12,  1.86s/it]
                                                        

 61%|██████    | 7337/12000 [4:20:26<2:24:12,  1.86s/it]
 61%|██████    | 7338/12000 [4:20:30<3:10:09,  2.45s/it]
                                                        

 61%|██████    | 7338/12000 [4:20:30<3:10:09,  2.45s/it]
 61%|██████    | 7339/12000 [4:20:32<3:01:11,  2.33s/it]
                                                        

 61%|██████    | 7339/12000 [4:20:32<3:01:11,  2.33s/it]
 61%|██████    | 7340/12000 [4:20:35<3:09:27,  2.44s/it]
                                                        

 61%|██████    | 7340/12000 [4:20:35<3:09:27,  2.44s/it]
 61%|██████    | 7341/12000 [4:20:40<4:04:28,  3.15s/it]
                                                        

 61%|██████    | 7341/12000 [4:20:40<4:04:28,  3.15s/it]
 61%|██████    | 7342/12000 [4:20:42<3:53:48,  3.01s/it]
                                                        

 61%|██████    | 7342/12000 [4:20:42<3:53:48,  3.01s/it]
 61%|██████    | 7343/12000 [4:20:45<3:48:48,  2.95s/it]
                                                        

 61%|██████    | 7343/12000 [4:20:45<3:48:48,  2.95s/it]
 61%|██████    | 7344/12000 [4:20:47<3:16:32,  2.53s/it]
                                                        

 61%|██████    | 7344/12000 [4:20:47<3:16:32,  2.53s/it]
 61%|██████    | 7345/12000 [4:20:48<2:58:07,  2.30s/it]
                                                        

 61%|██████    | 7345/12000 [4:20:48<2:58:07,  2.30s/it]
 61%|██████    | 7346/12000 [4:20:50<2:38:15,  2.04s/it]
                                                        

 61%|██████    | 7346/12000 [4:20:50<2:38:15,  2.04s/it]
 61%|██████    | 7347/12000 [4:20:51<2:28:21,  1.91s/it]
                                                        

 61%|██████    | 7347/12000 [4:20:51<2:28:21,  1.91s/it]
 61%|██████    | 7348/12000 [4:20:53<2:15:30,  1.75s/it]
                                                        

 61%|██████    | 7348/12000 [4:20:53<2:15:30,  1.75s/it]
 61%|██████    | 7349/12000 [4:20:54<2:08:50,  1.66s/it]
                                                        

 61%|██████    | 7349/12000 [4:20:54<2:08:50,  1.66s/it]
 61%|██████▏   | 7350/12000 [4:20:56<2:02:41,  1.58s/it]
                                                        

 61%|██████▏   | 7350/12000 [4:20:56<2:02:41,  1.58s/it]
 61%|██████▏   | 7351/12000 [4:20:59<2:35:50,  2.01s/it]
                                                        

 61%|██████▏   | 7351/12000 [4:20:59<2:35:50,  2.01s/it]
 61%|██████▏   | 7352/12000 [4:21:00<2:24:32,  1.87s/it]
                                                        

 61%|██████▏   | 7352/12000 [4:21:00<2:24:32,  1.87s/it]
 61%|██████▏   | 7353/12000 [4:21:02<2:31:19,  1.95s/it]
                                                        

 61%|██████▏   | 7353/12000 [4:21:02<2:31:19,  1.95s/it]
 61%|██████▏   | 7354/12000 [4:21:04<2:24:58,  1.87s/it]
                                                        

 61%|██████▏   | 7354/12000 [4:21:04<2:24:58,  1.87s/it]
 61%|██████▏   | 7355/12000 [4:21:06<2:16:48,  1.77s/it]
                                                        

 61%|██████▏   | 7355/12000 [4:21:06<2:16:48,  1.77s/it]
 61%|██████▏   | 7356/12000 [4:21:08<2:31:25,  1.96s/it]
                                                        

 61%|██████▏   | 7356/12000 [4:21:08<2:31:25,  1.96s/it]
 61%|██████▏   | 7357/12000 [4:21:09<2:20:27,  1.82s/it]
                                                        

 61%|██████▏   | 7357/12000 [4:21:09<2:20:27,  1.82s/it]
 61%|██████▏   | 7358/12000 [4:21:11<2:09:03,  1.67s/it]
                                                        

 61%|██████▏   | 7358/12000 [4:21:11<2:09:03,  1.67s/it]
 61%|██████▏   | 7359/12000 [4:21:12<2:03:54,  1.60s/it]
                                                        

 61%|██████▏   | 7359/12000 [4:21:12<2:03:54,  1.60s/it]
 61%|██████▏   | 7360/12000 [4:21:16<2:47:57,  2.17s/it]
                                                        

 61%|██████▏   | 7360/12000 [4:21:16<2:47:57,  2.17s/it]
 61%|██████▏   | 7361/12000 [4:21:19<3:17:02,  2.55s/it]
                                                        

 61%|██████▏   | 7361/12000 [4:21:19<3:17:02,  2.55s/it]
 61%|██████▏   | 7362/12000 [4:21:21<2:54:36,  2.26s/it]
                                                        

 61%|██████▏   | 7362/12000 [4:21:21<2:54:36,  2.26s/it]
 61%|██████▏   | 7363/12000 [4:21:23<2:54:22,  2.26s/it]
                                                        

 61%|██████▏   | 7363/12000 [4:21:23<2:54:22,  2.26s/it]
 61%|██████▏   | 7364/12000 [4:21:25<2:50:40,  2.21s/it]
                                                        

 61%|██████▏   | 7364/12000 [4:21:25<2:50:40,  2.21s/it]
 61%|██████▏   | 7365/12000 [4:21:27<2:34:15,  2.00s/it]
                                                        

 61%|██████▏   | 7365/12000 [4:21:27<2:34:15,  2.00s/it]
 61%|██████▏   | 7366/12000 [4:21:28<2:19:16,  1.80s/it]
                                                        

 61%|██████▏   | 7366/12000 [4:21:28<2:19:16,  1.80s/it]
 61%|██████▏   | 7367/12000 [4:21:30<2:21:58,  1.84s/it]
                                                        

 61%|██████▏   | 7367/12000 [4:21:30<2:21:58,  1.84s/it]
 61%|██████▏   | 7368/12000 [4:21:36<3:59:44,  3.11s/it]
                                                        

 61%|██████▏   | 7368/12000 [4:21:36<3:59:44,  3.11s/it]
 61%|██████▏   | 7369/12000 [4:21:37<3:23:34,  2.64s/it]
                                                        

 61%|██████▏   | 7369/12000 [4:21:37<3:23:34,  2.64s/it]
 61%|██████▏   | 7370/12000 [4:21:40<3:11:55,  2.49s/it]
                                                        

 61%|██████▏   | 7370/12000 [4:21:40<3:11:55,  2.49s/it]
 61%|██████▏   | 7371/12000 [4:21:42<3:11:38,  2.48s/it]
                                                        

 61%|██████▏   | 7371/12000 [4:21:42<3:11:38,  2.48s/it]
 61%|██████▏   | 7372/12000 [4:21:44<2:51:18,  2.22s/it]
                                                        

 61%|██████▏   | 7372/12000 [4:21:44<2:51:18,  2.22s/it]
 61%|██████▏   | 7373/12000 [4:21:46<3:00:12,  2.34s/it]
                                                        

 61%|██████▏   | 7373/12000 [4:21:46<3:00:12,  2.34s/it]
 61%|██████▏   | 7374/12000 [4:21:48<2:43:57,  2.13s/it]
                                                        

 61%|██████▏   | 7374/12000 [4:21:48<2:43:57,  2.13s/it]
 61%|██████▏   | 7375/12000 [4:21:50<2:44:08,  2.13s/it]
                                                        

 61%|██████▏   | 7375/12000 [4:21:50<2:44:08,  2.13s/it]
 61%|██████▏   | 7376/12000 [4:21:52<2:29:35,  1.94s/it]
                                                        

 61%|██████▏   | 7376/12000 [4:21:52<2:29:35,  1.94s/it]
 61%|██████▏   | 7377/12000 [4:21:53<2:26:13,  1.90s/it]
                                                        

 61%|██████▏   | 7377/12000 [4:21:53<2:26:13,  1.90s/it]
 61%|██████▏   | 7378/12000 [4:21:55<2:13:35,  1.73s/it]
                                                        

 61%|██████▏   | 7378/12000 [4:21:55<2:13:35,  1.73s/it]
 61%|██████▏   | 7379/12000 [4:21:57<2:31:38,  1.97s/it]
                                                        

 61%|██████▏   | 7379/12000 [4:21:57<2:31:38,  1.97s/it]
 62%|██████▏   | 7380/12000 [4:22:00<2:42:57,  2.12s/it]
                                                        

 62%|██████▏   | 7380/12000 [4:22:00<2:42:57,  2.12s/it]
 62%|██████▏   | 7381/12000 [4:22:01<2:35:00,  2.01s/it]
                                                        

 62%|██████▏   | 7381/12000 [4:22:01<2:35:00,  2.01s/it]
 62%|██████▏   | 7382/12000 [4:22:05<3:15:01,  2.53s/it]
                                                        

 62%|██████▏   | 7382/12000 [4:22:05<3:15:01,  2.53s/it]
 62%|██████▏   | 7383/12000 [4:22:08<3:26:13,  2.68s/it]
                                                        
{'loss': '0.9207', 'grad_norm': '0.3465', 'learning_rate': '7.186e-05', 'entropy': '0.8756', 'num_tokens': '3.959e+07', 'mean_token_accuracy': '0.7768', 'epoch': '0.06492'}
{'loss': '0.9232', 'grad_norm': '0.5312', 'learning_rate': '7.184e-05', 'entropy': '0.7535', 'num_tokens': '3.959e+07', 'mean_token_accuracy': '0.7967', 'epoch': '0.06493'}
{'loss': '0.7352', 'grad_norm': '0.3914', 'learning_rate': '7.181e-05', 'entropy': '0.6991', 'num_tokens': '3.96e+07', 'mean_token_accuracy': '0.8232', 'epoch': '0.06494'}
{'loss': '0.7179', 'grad_norm': '0.2633', 'learning_rate': '7.178e-05', 'entropy': '0.6886', 'num_tokens': '3.96e+07', 'mean_token_accuracy': '0.8175', 'epoch': '0.06495'}
{'loss': '0.8046', 'grad_norm': '0.327', 'learning_rate': '7.176e-05', 'entropy': '0.8367', 'num_tokens': '3.961e+07', 'mean_token_accuracy': '0.7883', 'epoch': '0.06496'}
{'loss': '1.411', 'grad_norm': '0.4542', 'learning_rate': '7.173e-05', 'entropy': '1.199', 'num_tokens': '3.962e+07', 'mean_token_accuracy': '0.726', 'epoch': '0.06497'}
{'loss': '0.7496', 'grad_norm': '0.3977', 'learning_rate': '7.17e-05', 'entropy': '0.7718', 'num_tokens': '3.963e+07', 'mean_token_accuracy': '0.8035', 'epoch': '0.06498'}
{'loss': '0.9599', 'grad_norm': '0.4457', 'learning_rate': '7.168e-05', 'entropy': '0.9371', 'num_tokens': '3.963e+07', 'mean_token_accuracy': '0.7637', 'epoch': '0.06498'}
{'loss': '0.8387', 'grad_norm': '0.3939', 'learning_rate': '7.165e-05', 'entropy': '0.9286', 'num_tokens': '3.964e+07', 'mean_token_accuracy': '0.7784', 'epoch': '0.06499'}
{'loss': '1.022', 'grad_norm': '0.4691', 'learning_rate': '7.162e-05', 'entropy': '0.9804', 'num_tokens': '3.964e+07', 'mean_token_accuracy': '0.756', 'epoch': '0.065'}
{'loss': '0.7542', 'grad_norm': '0.4079', 'learning_rate': '7.16e-05', 'entropy': '0.78', 'num_tokens': '3.965e+07', 'mean_token_accuracy': '0.8013', 'epoch': '0.06501'}
{'loss': '0.6815', 'grad_norm': '0.3887', 'learning_rate': '7.157e-05', 'entropy': '0.6645', 'num_tokens': '3.965e+07', 'mean_token_accuracy': '0.8223', 'epoch': '0.06502'}
{'loss': '0.8149', 'grad_norm': '0.4083', 'learning_rate': '7.155e-05', 'entropy': '0.8242', 'num_tokens': '3.966e+07', 'mean_token_accuracy': '0.7927', 'epoch': '0.06503'}
{'loss': '0.663', 'grad_norm': '0.3298', 'learning_rate': '7.152e-05', 'entropy': '0.6871', 'num_tokens': '3.966e+07', 'mean_token_accuracy': '0.8212', 'epoch': '0.06504'}
{'loss': '0.8214', 'grad_norm': '0.3607', 'learning_rate': '7.149e-05', 'entropy': '0.8317', 'num_tokens': '3.967e+07', 'mean_token_accuracy': '0.7846', 'epoch': '0.06505'}
{'loss': '0.6707', 'grad_norm': '0.3982', 'learning_rate': '7.147e-05', 'entropy': '0.6984', 'num_tokens': '3.967e+07', 'mean_token_accuracy': '0.8225', 'epoch': '0.06506'}
{'loss': '0.5727', 'grad_norm': '0.3204', 'learning_rate': '7.144e-05', 'entropy': '0.6043', 'num_tokens': '3.967e+07', 'mean_token_accuracy': '0.8424', 'epoch': '0.06506'}
{'loss': '0.7861', 'grad_norm': '0.3841', 'learning_rate': '7.141e-05', 'entropy': '0.7296', 'num_tokens': '3.968e+07', 'mean_token_accuracy': '0.8178', 'epoch': '0.06507'}
{'loss': '0.6684', 'grad_norm': '0.3645', 'learning_rate': '7.139e-05', 'entropy': '0.6763', 'num_tokens': '3.969e+07', 'mean_token_accuracy': '0.8298', 'epoch': '0.06508'}
{'loss': '0.44', 'grad_norm': '0.3363', 'learning_rate': '7.136e-05', 'entropy': '0.5405', 'num_tokens': '3.969e+07', 'mean_token_accuracy': '0.8565', 'epoch': '0.06509'}
{'loss': '0.9468', 'grad_norm': '0.3843', 'learning_rate': '7.133e-05', 'entropy': '1.053', 'num_tokens': '3.97e+07', 'mean_token_accuracy': '0.7418', 'epoch': '0.0651'}
{'loss': '1.026', 'grad_norm': '0.3668', 'learning_rate': '7.131e-05', 'entropy': '0.9535', 'num_tokens': '3.97e+07', 'mean_token_accuracy': '0.7661', 'epoch': '0.06511'}
{'loss': '0.8198', 'grad_norm': '0.4406', 'learning_rate': '7.128e-05', 'entropy': '0.8154', 'num_tokens': '3.971e+07', 'mean_token_accuracy': '0.7906', 'epoch': '0.06512'}
{'loss': '0.7954', 'grad_norm': '0.3883', 'learning_rate': '7.126e-05', 'entropy': '0.8328', 'num_tokens': '3.971e+07', 'mean_token_accuracy': '0.7912', 'epoch': '0.06513'}
{'loss': '0.5827', 'grad_norm': '0.4059', 'learning_rate': '7.123e-05', 'entropy': '0.6073', 'num_tokens': '3.972e+07', 'mean_token_accuracy': '0.8425', 'epoch': '0.06513'}
{'loss': '0.7731', 'grad_norm': '0.3508', 'learning_rate': '7.12e-05', 'entropy': '0.7633', 'num_tokens': '3.972e+07', 'mean_token_accuracy': '0.8135', 'epoch': '0.06514'}
{'loss': '1.053', 'grad_norm': '0.3856', 'learning_rate': '7.118e-05', 'entropy': '1.07', 'num_tokens': '3.973e+07', 'mean_token_accuracy': '0.7372', 'epoch': '0.06515'}
{'loss': '0.8676', 'grad_norm': '0.4247', 'learning_rate': '7.115e-05', 'entropy': '0.829', 'num_tokens': '3.973e+07', 'mean_token_accuracy': '0.8025', 'epoch': '0.06516'}
{'loss': '0.7339', 'grad_norm': '0.3141', 'learning_rate': '7.112e-05', 'entropy': '0.7041', 'num_tokens': '3.974e+07', 'mean_token_accuracy': '0.8152', 'epoch': '0.06517'}
{'loss': '0.5937', 'grad_norm': '0.3284', 'learning_rate': '7.11e-05', 'entropy': '0.7057', 'num_tokens': '3.975e+07', 'mean_token_accuracy': '0.8334', 'epoch': '0.06518'}
{'loss': '1.039', 'grad_norm': '0.4185', 'learning_rate': '7.107e-05', 'entropy': '1.02', 'num_tokens': '3.975e+07', 'mean_token_accuracy': '0.7424', 'epoch': '0.06519'}
{'loss': '0.812', 'grad_norm': '0.3222', 'learning_rate': '7.104e-05', 'entropy': '0.7633', 'num_tokens': '3.976e+07', 'mean_token_accuracy': '0.8157', 'epoch': '0.0652'}
{'loss': '0.4934', 'grad_norm': '0.2722', 'learning_rate': '7.102e-05', 'entropy': '0.5355', 'num_tokens': '3.977e+07', 'mean_token_accuracy': '0.8603', 'epoch': '0.06521'}
{'loss': '0.9615', 'grad_norm': '0.44', 'learning_rate': '7.099e-05', 'entropy': '1.026', 'num_tokens': '3.977e+07', 'mean_token_accuracy': '0.75', 'epoch': '0.06521'}
{'loss': '0.9282', 'grad_norm': '0.3586', 'learning_rate': '7.096e-05', 'entropy': '0.9178', 'num_tokens': '3.978e+07', 'mean_token_accuracy': '0.7806', 'epoch': '0.06522'}
{'loss': '0.9137', 'grad_norm': '0.4855', 'learning_rate': '7.094e-05', 'entropy': '0.8289', 'num_tokens': '3.978e+07', 'mean_token_accuracy': '0.7952', 'epoch': '0.06523'}
{'loss': '0.8755', 'grad_norm': '0.3802', 'learning_rate': '7.091e-05', 'entropy': '0.8766', 'num_tokens': '3.979e+07', 'mean_token_accuracy': '0.7852', 'epoch': '0.06524'}
{'loss': '0.7033', 'grad_norm': '0.3809', 'learning_rate': '7.089e-05', 'entropy': '0.7251', 'num_tokens': '3.98e+07', 'mean_token_accuracy': '0.8131', 'epoch': '0.06525'}
{'loss': '0.8303', 'grad_norm': '0.4612', 'learning_rate': '7.086e-05', 'entropy': '0.8956', 'num_tokens': '3.98e+07', 'mean_token_accuracy': '0.779', 'epoch': '0.06526'}
{'loss': '1.216', 'grad_norm': '0.3994', 'learning_rate': '7.083e-05', 'entropy': '1.072', 'num_tokens': '3.981e+07', 'mean_token_accuracy': '0.7465', 'epoch': '0.06527'}
{'loss': '1.001', 'grad_norm': '0.4417', 'learning_rate': '7.081e-05', 'entropy': '0.9534', 'num_tokens': '3.981e+07', 'mean_token_accuracy': '0.7534', 'epoch': '0.06528'}
{'loss': '0.9312', 'grad_norm': '0.3873', 'learning_rate': '7.078e-05', 'entropy': '0.7822', 'num_tokens': '3.982e+07', 'mean_token_accuracy': '0.7974', 'epoch': '0.06529'}
{'loss': '0.9332', 'grad_norm': '0.5821', 'learning_rate': '7.075e-05', 'entropy': '0.8953', 'num_tokens': '3.982e+07', 'mean_token_accuracy': '0.7824', 'epoch': '0.06529'}
{'loss': '0.9056', 'grad_norm': '0.3964', 'learning_rate': '7.073e-05', 'entropy': '0.8644', 'num_tokens': '3.983e+07', 'mean_token_accuracy': '0.7754', 'epoch': '0.0653'}
{'loss': '0.664', 'grad_norm': '0.4249', 'learning_rate': '7.07e-05', 'entropy': '0.724', 'num_tokens': '3.983e+07', 'mean_token_accuracy': '0.8199', 'epoch': '0.06531'}
{'loss': '0.7322', 'grad_norm': '0.462', 'learning_rate': '7.067e-05', 'entropy': '0.7499', 'num_tokens': '3.984e+07', 'mean_token_accuracy': '0.7998', 'epoch': '0.06532'}
{'loss': '0.8993', 'grad_norm': '0.4089', 'learning_rate': '7.065e-05', 'entropy': '0.8454', 'num_tokens': '3.984e+07', 'mean_token_accuracy': '0.7863', 'epoch': '0.06533'}

 62%|██████▏   | 7383/12000 [4:22:08<3:26:13,  2.68s/it]
 62%|██████▏   | 7384/12000 [4:22:10<3:02:18,  2.37s/it]
                                                        

 62%|██████▏   | 7384/12000 [4:22:10<3:02:18,  2.37s/it]
 62%|██████▏   | 7385/12000 [4:22:13<3:18:08,  2.58s/it]
                                                        

 62%|██████▏   | 7385/12000 [4:22:13<3:18:08,  2.58s/it]
 62%|██████▏   | 7386/12000 [4:22:15<2:59:02,  2.33s/it]
                                                        

 62%|██████▏   | 7386/12000 [4:22:15<2:59:02,  2.33s/it]
 62%|██████▏   | 7387/12000 [4:22:16<2:37:16,  2.05s/it]
                                                        

 62%|██████▏   | 7387/12000 [4:22:16<2:37:16,  2.05s/it]
 62%|██████▏   | 7388/12000 [4:22:18<2:32:24,  1.98s/it]
                                                        

 62%|██████▏   | 7388/12000 [4:22:18<2:32:24,  1.98s/it]
 62%|██████▏   | 7389/12000 [4:22:20<2:42:58,  2.12s/it]
                                                        

 62%|██████▏   | 7389/12000 [4:22:20<2:42:58,  2.12s/it]
 62%|██████▏   | 7390/12000 [4:22:22<2:26:52,  1.91s/it]
                                                        

 62%|██████▏   | 7390/12000 [4:22:22<2:26:52,  1.91s/it]
 62%|██████▏   | 7391/12000 [4:22:23<2:17:33,  1.79s/it]
                                                        

 62%|██████▏   | 7391/12000 [4:22:23<2:17:33,  1.79s/it]
 62%|██████▏   | 7392/12000 [4:22:25<2:08:46,  1.68s/it]
                                                        

 62%|██████▏   | 7392/12000 [4:22:25<2:08:46,  1.68s/it]
 62%|██████▏   | 7393/12000 [4:22:26<2:03:45,  1.61s/it]
                                                        

 62%|██████▏   | 7393/12000 [4:22:26<2:03:45,  1.61s/it]
 62%|██████▏   | 7394/12000 [4:22:28<2:02:04,  1.59s/it]
                                                        

 62%|██████▏   | 7394/12000 [4:22:28<2:02:04,  1.59s/it]
 62%|██████▏   | 7395/12000 [4:22:29<2:04:09,  1.62s/it]
                                                        

 62%|██████▏   | 7395/12000 [4:22:29<2:04:09,  1.62s/it]
 62%|██████▏   | 7396/12000 [4:22:31<2:09:55,  1.69s/it]
                                                        

 62%|██████▏   | 7396/12000 [4:22:31<2:09:55,  1.69s/it]
 62%|██████▏   | 7397/12000 [4:22:33<2:10:48,  1.71s/it]
                                                        

 62%|██████▏   | 7397/12000 [4:22:33<2:10:48,  1.71s/it]
 62%|██████▏   | 7398/12000 [4:22:35<2:07:17,  1.66s/it]
                                                        

 62%|██████▏   | 7398/12000 [4:22:35<2:07:17,  1.66s/it]
 62%|██████▏   | 7399/12000 [4:22:36<2:09:34,  1.69s/it]
                                                        

 62%|██████▏   | 7399/12000 [4:22:36<2:09:34,  1.69s/it]
 62%|██████▏   | 7400/12000 [4:22:38<2:07:53,  1.67s/it]
                                                        

 62%|██████▏   | 7400/12000 [4:22:38<2:07:53,  1.67s/it]
 62%|██████▏   | 7401/12000 [4:22:41<2:39:32,  2.08s/it]
                                                        

 62%|██████▏   | 7401/12000 [4:22:41<2:39:32,  2.08s/it]
 62%|██████▏   | 7402/12000 [4:22:43<2:39:37,  2.08s/it]
                                                        

 62%|██████▏   | 7402/12000 [4:22:43<2:39:37,  2.08s/it]
 62%|██████▏   | 7403/12000 [4:22:46<2:55:25,  2.29s/it]
                                                        

 62%|██████▏   | 7403/12000 [4:22:46<2:55:25,  2.29s/it]
 62%|██████▏   | 7404/12000 [4:22:50<3:47:35,  2.97s/it]
                                                        

 62%|██████▏   | 7404/12000 [4:22:50<3:47:35,  2.97s/it]
 62%|██████▏   | 7405/12000 [4:22:52<3:14:32,  2.54s/it]
                                                        

 62%|██████▏   | 7405/12000 [4:22:52<3:14:32,  2.54s/it]
 62%|██████▏   | 7406/12000 [4:22:53<2:48:07,  2.20s/it]
                                                        

 62%|██████▏   | 7406/12000 [4:22:53<2:48:07,  2.20s/it]
 62%|██████▏   | 7407/12000 [4:22:55<2:33:34,  2.01s/it]
                                                        

 62%|██████▏   | 7407/12000 [4:22:55<2:33:34,  2.01s/it]
 62%|██████▏   | 7408/12000 [4:22:58<2:52:50,  2.26s/it]
                                                        

 62%|██████▏   | 7408/12000 [4:22:58<2:52:50,  2.26s/it]
 62%|██████▏   | 7409/12000 [4:22:59<2:34:12,  2.02s/it]
                                                        

 62%|██████▏   | 7409/12000 [4:22:59<2:34:12,  2.02s/it]
 62%|██████▏   | 7410/12000 [4:23:01<2:37:38,  2.06s/it]
                                                        

 62%|██████▏   | 7410/12000 [4:23:01<2:37:38,  2.06s/it]
 62%|██████▏   | 7411/12000 [4:23:03<2:22:10,  1.86s/it]
                                                        

 62%|██████▏   | 7411/12000 [4:23:03<2:22:10,  1.86s/it]
 62%|██████▏   | 7412/12000 [4:23:05<2:38:38,  2.07s/it]
                                                        

 62%|██████▏   | 7412/12000 [4:23:05<2:38:38,  2.07s/it]
 62%|██████▏   | 7413/12000 [4:23:07<2:33:50,  2.01s/it]
                                                        

 62%|██████▏   | 7413/12000 [4:23:07<2:33:50,  2.01s/it]
 62%|██████▏   | 7414/12000 [4:23:09<2:29:38,  1.96s/it]
                                                        

 62%|██████▏   | 7414/12000 [4:23:09<2:29:38,  1.96s/it]
 62%|██████▏   | 7415/12000 [4:23:11<2:22:34,  1.87s/it]
                                                        

 62%|██████▏   | 7415/12000 [4:23:11<2:22:34,  1.87s/it]
 62%|██████▏   | 7416/12000 [4:23:13<2:23:35,  1.88s/it]
                                                        

 62%|██████▏   | 7416/12000 [4:23:13<2:23:35,  1.88s/it]
 62%|██████▏   | 7417/12000 [4:23:15<2:43:14,  2.14s/it]
                                                        

 62%|██████▏   | 7417/12000 [4:23:15<2:43:14,  2.14s/it]
 62%|██████▏   | 7418/12000 [4:23:17<2:31:49,  1.99s/it]
                                                        

 62%|██████▏   | 7418/12000 [4:23:17<2:31:49,  1.99s/it]
 62%|██████▏   | 7419/12000 [4:23:19<2:37:09,  2.06s/it]
                                                        

 62%|██████▏   | 7419/12000 [4:23:19<2:37:09,  2.06s/it]
 62%|██████▏   | 7420/12000 [4:23:21<2:28:38,  1.95s/it]
                                                        

 62%|██████▏   | 7420/12000 [4:23:21<2:28:38,  1.95s/it]
 62%|██████▏   | 7421/12000 [4:23:22<2:19:23,  1.83s/it]
                                                        

 62%|██████▏   | 7421/12000 [4:23:22<2:19:23,  1.83s/it]
 62%|██████▏   | 7422/12000 [4:23:24<2:16:15,  1.79s/it]
                                                        

 62%|██████▏   | 7422/12000 [4:23:24<2:16:15,  1.79s/it]
 62%|██████▏   | 7423/12000 [4:23:27<2:31:36,  1.99s/it]
                                                        

 62%|██████▏   | 7423/12000 [4:23:27<2:31:36,  1.99s/it]
 62%|██████▏   | 7424/12000 [4:23:28<2:19:35,  1.83s/it]
                                                        

 62%|██████▏   | 7424/12000 [4:23:28<2:19:35,  1.83s/it]
 62%|██████▏   | 7425/12000 [4:23:32<2:59:51,  2.36s/it]
                                                        

 62%|██████▏   | 7425/12000 [4:23:32<2:59:51,  2.36s/it]
 62%|██████▏   | 7426/12000 [4:23:33<2:42:17,  2.13s/it]
                                                        

 62%|██████▏   | 7426/12000 [4:23:33<2:42:17,  2.13s/it]
 62%|██████▏   | 7427/12000 [4:23:36<2:52:13,  2.26s/it]
                                                        

 62%|██████▏   | 7427/12000 [4:23:36<2:52:13,  2.26s/it]
 62%|██████▏   | 7428/12000 [4:23:37<2:39:47,  2.10s/it]
                                                        

 62%|██████▏   | 7428/12000 [4:23:37<2:39:47,  2.10s/it]
 62%|██████▏   | 7429/12000 [4:23:40<2:39:38,  2.10s/it]
                                                        

 62%|██████▏   | 7429/12000 [4:23:40<2:39:38,  2.10s/it]
 62%|██████▏   | 7430/12000 [4:23:42<2:36:37,  2.06s/it]
                                                        
{'loss': '0.9317', 'grad_norm': '0.3306', 'learning_rate': '7.062e-05', 'entropy': '1.066', 'num_tokens': '3.985e+07', 'mean_token_accuracy': '0.7521', 'epoch': '0.06534'}
{'loss': '1.028', 'grad_norm': '0.5395', 'learning_rate': '7.06e-05', 'entropy': '0.9987', 'num_tokens': '3.985e+07', 'mean_token_accuracy': '0.7481', 'epoch': '0.06535'}
{'loss': '0.8982', 'grad_norm': '0.3708', 'learning_rate': '7.057e-05', 'entropy': '1.021', 'num_tokens': '3.986e+07', 'mean_token_accuracy': '0.7552', 'epoch': '0.06536'}
{'loss': '0.7357', 'grad_norm': '0.3937', 'learning_rate': '7.054e-05', 'entropy': '0.7384', 'num_tokens': '3.987e+07', 'mean_token_accuracy': '0.8115', 'epoch': '0.06536'}
{'loss': '0.611', 'grad_norm': '0.3646', 'learning_rate': '7.052e-05', 'entropy': '0.6448', 'num_tokens': '3.987e+07', 'mean_token_accuracy': '0.8217', 'epoch': '0.06537'}
{'loss': '1.123', 'grad_norm': '0.4558', 'learning_rate': '7.049e-05', 'entropy': '1.178', 'num_tokens': '3.988e+07', 'mean_token_accuracy': '0.7246', 'epoch': '0.06538'}
{'loss': '0.8571', 'grad_norm': '0.3106', 'learning_rate': '7.046e-05', 'entropy': '0.8204', 'num_tokens': '3.988e+07', 'mean_token_accuracy': '0.7906', 'epoch': '0.06539'}
{'loss': '0.7983', 'grad_norm': '0.4249', 'learning_rate': '7.044e-05', 'entropy': '0.8634', 'num_tokens': '3.989e+07', 'mean_token_accuracy': '0.7843', 'epoch': '0.0654'}
{'loss': '0.9806', 'grad_norm': '0.3871', 'learning_rate': '7.041e-05', 'entropy': '1.054', 'num_tokens': '3.989e+07', 'mean_token_accuracy': '0.754', 'epoch': '0.06541'}
{'loss': '0.8963', 'grad_norm': '0.4036', 'learning_rate': '7.039e-05', 'entropy': '0.9984', 'num_tokens': '3.989e+07', 'mean_token_accuracy': '0.7606', 'epoch': '0.06542'}
{'loss': '0.7255', 'grad_norm': '0.3441', 'learning_rate': '7.036e-05', 'entropy': '0.7504', 'num_tokens': '3.99e+07', 'mean_token_accuracy': '0.8058', 'epoch': '0.06543'}
{'loss': '0.8516', 'grad_norm': '0.3866', 'learning_rate': '7.033e-05', 'entropy': '0.9487', 'num_tokens': '3.99e+07', 'mean_token_accuracy': '0.7691', 'epoch': '0.06544'}
{'loss': '0.8794', 'grad_norm': '0.3573', 'learning_rate': '7.031e-05', 'entropy': '0.8579', 'num_tokens': '3.991e+07', 'mean_token_accuracy': '0.7867', 'epoch': '0.06544'}
{'loss': '0.8469', 'grad_norm': '0.4747', 'learning_rate': '7.028e-05', 'entropy': '0.7887', 'num_tokens': '3.991e+07', 'mean_token_accuracy': '0.795', 'epoch': '0.06545'}
{'loss': '0.5561', 'grad_norm': '0.3331', 'learning_rate': '7.025e-05', 'entropy': '0.649', 'num_tokens': '3.992e+07', 'mean_token_accuracy': '0.8367', 'epoch': '0.06546'}
{'loss': '0.794', 'grad_norm': '0.4086', 'learning_rate': '7.023e-05', 'entropy': '0.8292', 'num_tokens': '3.992e+07', 'mean_token_accuracy': '0.7939', 'epoch': '0.06547'}
{'loss': '0.7063', 'grad_norm': '0.3386', 'learning_rate': '7.02e-05', 'entropy': '0.7049', 'num_tokens': '3.993e+07', 'mean_token_accuracy': '0.8149', 'epoch': '0.06548'}
{'loss': '1.193', 'grad_norm': '0.4244', 'learning_rate': '7.017e-05', 'entropy': '1.194', 'num_tokens': '3.993e+07', 'mean_token_accuracy': '0.7213', 'epoch': '0.06549'}
{'loss': '0.9394', 'grad_norm': '0.3904', 'learning_rate': '7.015e-05', 'entropy': '0.9996', 'num_tokens': '3.994e+07', 'mean_token_accuracy': '0.7534', 'epoch': '0.0655'}
{'loss': '0.6011', 'grad_norm': '0.3843', 'learning_rate': '7.012e-05', 'entropy': '0.6154', 'num_tokens': '3.994e+07', 'mean_token_accuracy': '0.8429', 'epoch': '0.06551'}
{'loss': '0.9735', 'grad_norm': '0.4023', 'learning_rate': '7.01e-05', 'entropy': '0.9677', 'num_tokens': '3.995e+07', 'mean_token_accuracy': '0.7508', 'epoch': '0.06552'}
{'loss': '0.887', 'grad_norm': '0.445', 'learning_rate': '7.007e-05', 'entropy': '0.9858', 'num_tokens': '3.996e+07', 'mean_token_accuracy': '0.7614', 'epoch': '0.06552'}
{'loss': '0.8383', 'grad_norm': '0.3648', 'learning_rate': '7.004e-05', 'entropy': '0.8564', 'num_tokens': '3.996e+07', 'mean_token_accuracy': '0.7818', 'epoch': '0.06553'}
{'loss': '0.8969', 'grad_norm': '0.5449', 'learning_rate': '7.002e-05', 'entropy': '0.9044', 'num_tokens': '3.996e+07', 'mean_token_accuracy': '0.7792', 'epoch': '0.06554'}
{'loss': '0.4857', 'grad_norm': '0.3471', 'learning_rate': '6.999e-05', 'entropy': '0.4942', 'num_tokens': '3.997e+07', 'mean_token_accuracy': '0.8677', 'epoch': '0.06555'}
{'loss': '0.734', 'grad_norm': '0.4955', 'learning_rate': '6.996e-05', 'entropy': '0.7117', 'num_tokens': '3.998e+07', 'mean_token_accuracy': '0.8147', 'epoch': '0.06556'}
{'loss': '0.717', 'grad_norm': '0.3532', 'learning_rate': '6.994e-05', 'entropy': '0.7774', 'num_tokens': '3.998e+07', 'mean_token_accuracy': '0.8032', 'epoch': '0.06557'}
{'loss': '1.073', 'grad_norm': '0.4188', 'learning_rate': '6.991e-05', 'entropy': '1.101', 'num_tokens': '3.998e+07', 'mean_token_accuracy': '0.7338', 'epoch': '0.06558'}
{'loss': '1.18', 'grad_norm': '0.4145', 'learning_rate': '6.989e-05', 'entropy': '1.16', 'num_tokens': '3.999e+07', 'mean_token_accuracy': '0.7206', 'epoch': '0.06559'}
{'loss': '0.8497', 'grad_norm': '0.3495', 'learning_rate': '6.986e-05', 'entropy': '0.8131', 'num_tokens': '4e+07', 'mean_token_accuracy': '0.7965', 'epoch': '0.06559'}
{'loss': '0.8994', 'grad_norm': '0.4458', 'learning_rate': '6.983e-05', 'entropy': '0.9054', 'num_tokens': '4e+07', 'mean_token_accuracy': '0.7809', 'epoch': '0.0656'}
{'loss': '0.8416', 'grad_norm': '0.3493', 'learning_rate': '6.981e-05', 'entropy': '0.858', 'num_tokens': '4.001e+07', 'mean_token_accuracy': '0.7924', 'epoch': '0.06561'}
{'loss': '0.6774', 'grad_norm': '0.4648', 'learning_rate': '6.978e-05', 'entropy': '0.7243', 'num_tokens': '4.001e+07', 'mean_token_accuracy': '0.798', 'epoch': '0.06562'}
{'loss': '0.6512', 'grad_norm': '0.3477', 'learning_rate': '6.975e-05', 'entropy': '0.7202', 'num_tokens': '4.001e+07', 'mean_token_accuracy': '0.8336', 'epoch': '0.06563'}
{'loss': '1.056', 'grad_norm': '0.3385', 'learning_rate': '6.973e-05', 'entropy': '0.8542', 'num_tokens': '4.002e+07', 'mean_token_accuracy': '0.7736', 'epoch': '0.06564'}
{'loss': '0.9937', 'grad_norm': '0.5444', 'learning_rate': '6.97e-05', 'entropy': '1.07', 'num_tokens': '4.003e+07', 'mean_token_accuracy': '0.7389', 'epoch': '0.06565'}
{'loss': '0.7722', 'grad_norm': '0.3697', 'learning_rate': '6.968e-05', 'entropy': '0.8295', 'num_tokens': '4.003e+07', 'mean_token_accuracy': '0.7846', 'epoch': '0.06566'}
{'loss': '0.5609', 'grad_norm': '0.3193', 'learning_rate': '6.965e-05', 'entropy': '0.6265', 'num_tokens': '4.004e+07', 'mean_token_accuracy': '0.8351', 'epoch': '0.06567'}
{'loss': '0.713', 'grad_norm': '0.4462', 'learning_rate': '6.962e-05', 'entropy': '0.7366', 'num_tokens': '4.004e+07', 'mean_token_accuracy': '0.8133', 'epoch': '0.06567'}
{'loss': '1.038', 'grad_norm': '0.4892', 'learning_rate': '6.96e-05', 'entropy': '0.9791', 'num_tokens': '4.005e+07', 'mean_token_accuracy': '0.7496', 'epoch': '0.06568'}
{'loss': '1.48', 'grad_norm': '0.3938', 'learning_rate': '6.957e-05', 'entropy': '1.263', 'num_tokens': '4.006e+07', 'mean_token_accuracy': '0.6994', 'epoch': '0.06569'}
{'loss': '1.002', 'grad_norm': '0.4779', 'learning_rate': '6.954e-05', 'entropy': '0.9559', 'num_tokens': '4.006e+07', 'mean_token_accuracy': '0.7702', 'epoch': '0.0657'}
{'loss': '0.7349', 'grad_norm': '0.4377', 'learning_rate': '6.952e-05', 'entropy': '0.7774', 'num_tokens': '4.006e+07', 'mean_token_accuracy': '0.7989', 'epoch': '0.06571'}
{'loss': '0.6218', 'grad_norm': '0.3357', 'learning_rate': '6.949e-05', 'entropy': '0.6689', 'num_tokens': '4.007e+07', 'mean_token_accuracy': '0.8313', 'epoch': '0.06572'}
{'loss': '0.7831', 'grad_norm': '0.4355', 'learning_rate': '6.947e-05', 'entropy': '0.7505', 'num_tokens': '4.008e+07', 'mean_token_accuracy': '0.8102', 'epoch': '0.06573'}
{'loss': '0.8196', 'grad_norm': '0.3347', 'learning_rate': '6.944e-05', 'entropy': '0.8022', 'num_tokens': '4.008e+07', 'mean_token_accuracy': '0.7901', 'epoch': '0.06574'}
{'loss': '0.9457', 'grad_norm': '0.4408', 'learning_rate': '6.941e-05', 'entropy': '0.8733', 'num_tokens': '4.009e+07', 'mean_token_accuracy': '0.7774', 'epoch': '0.06575'}

 62%|██████▏   | 7430/12000 [4:23:42<2:36:37,  2.06s/it]
 62%|██████▏   | 7431/12000 [4:23:43<2:21:49,  1.86s/it]
                                                        

 62%|██████▏   | 7431/12000 [4:23:43<2:21:49,  1.86s/it]
 62%|██████▏   | 7432/12000 [4:23:45<2:17:26,  1.81s/it]
                                                        

 62%|██████▏   | 7432/12000 [4:23:45<2:17:26,  1.81s/it]
 62%|██████▏   | 7433/12000 [4:23:46<2:14:11,  1.76s/it]
                                                        

 62%|██████▏   | 7433/12000 [4:23:46<2:14:11,  1.76s/it]
 62%|██████▏   | 7434/12000 [4:23:48<2:07:37,  1.68s/it]
                                                        

 62%|██████▏   | 7434/12000 [4:23:48<2:07:37,  1.68s/it]
 62%|██████▏   | 7435/12000 [4:23:49<2:07:02,  1.67s/it]
                                                        

 62%|██████▏   | 7435/12000 [4:23:49<2:07:02,  1.67s/it]
 62%|██████▏   | 7436/12000 [4:23:51<2:16:12,  1.79s/it]
                                                        

 62%|██████▏   | 7436/12000 [4:23:51<2:16:12,  1.79s/it]
 62%|██████▏   | 7437/12000 [4:23:53<2:12:41,  1.74s/it]
                                                        

 62%|██████▏   | 7437/12000 [4:23:53<2:12:41,  1.74s/it]
 62%|██████▏   | 7438/12000 [4:23:55<2:08:02,  1.68s/it]
                                                        

 62%|██████▏   | 7438/12000 [4:23:55<2:08:02,  1.68s/it]
 62%|██████▏   | 7439/12000 [4:23:58<2:38:03,  2.08s/it]
                                                        

 62%|██████▏   | 7439/12000 [4:23:58<2:38:03,  2.08s/it]
 62%|██████▏   | 7440/12000 [4:23:59<2:26:10,  1.92s/it]
                                                        

 62%|██████▏   | 7440/12000 [4:23:59<2:26:10,  1.92s/it]
 62%|██████▏   | 7441/12000 [4:24:02<2:48:03,  2.21s/it]
                                                        

 62%|██████▏   | 7441/12000 [4:24:02<2:48:03,  2.21s/it]
 62%|██████▏   | 7442/12000 [4:24:04<2:44:06,  2.16s/it]
                                                        

 62%|██████▏   | 7442/12000 [4:24:04<2:44:06,  2.16s/it]
 62%|██████▏   | 7443/12000 [4:24:06<2:29:09,  1.96s/it]
                                                        

 62%|██████▏   | 7443/12000 [4:24:06<2:29:09,  1.96s/it]
 62%|██████▏   | 7444/12000 [4:24:08<2:45:56,  2.19s/it]
                                                        

 62%|██████▏   | 7444/12000 [4:24:08<2:45:56,  2.19s/it]
 62%|██████▏   | 7445/12000 [4:24:10<2:33:30,  2.02s/it]
                                                        

 62%|██████▏   | 7445/12000 [4:24:10<2:33:30,  2.02s/it]
 62%|██████▏   | 7446/12000 [4:24:12<2:24:11,  1.90s/it]
                                                        

 62%|██████▏   | 7446/12000 [4:24:12<2:24:11,  1.90s/it]
 62%|██████▏   | 7447/12000 [4:24:16<3:10:04,  2.50s/it]
                                                        

 62%|██████▏   | 7447/12000 [4:24:16<3:10:04,  2.50s/it]
 62%|██████▏   | 7448/12000 [4:24:17<2:49:47,  2.24s/it]
                                                        

 62%|██████▏   | 7448/12000 [4:24:17<2:49:47,  2.24s/it]
 62%|██████▏   | 7449/12000 [4:24:19<2:41:50,  2.13s/it]
                                                        

 62%|██████▏   | 7449/12000 [4:24:19<2:41:50,  2.13s/it]
 62%|██████▏   | 7450/12000 [4:24:22<2:52:32,  2.28s/it]
                                                        

 62%|██████▏   | 7450/12000 [4:24:22<2:52:32,  2.28s/it]
 62%|██████▏   | 7451/12000 [4:24:25<3:29:02,  2.76s/it]
                                                        

 62%|██████▏   | 7451/12000 [4:24:25<3:29:02,  2.76s/it]
 62%|██████▏   | 7452/12000 [4:24:28<3:24:54,  2.70s/it]
                                                        

 62%|██████▏   | 7452/12000 [4:24:28<3:24:54,  2.70s/it]
 62%|██████▏   | 7453/12000 [4:24:32<3:55:46,  3.11s/it]
                                                        

 62%|██████▏   | 7453/12000 [4:24:32<3:55:46,  3.11s/it]
 62%|██████▏   | 7454/12000 [4:24:34<3:27:34,  2.74s/it]
                                                        

 62%|██████▏   | 7454/12000 [4:24:34<3:27:34,  2.74s/it]
 62%|██████▏   | 7455/12000 [4:24:36<3:00:46,  2.39s/it]
                                                        

 62%|██████▏   | 7455/12000 [4:24:36<3:00:46,  2.39s/it]
 62%|██████▏   | 7456/12000 [4:24:37<2:49:32,  2.24s/it]
                                                        

 62%|██████▏   | 7456/12000 [4:24:37<2:49:32,  2.24s/it]
 62%|██████▏   | 7457/12000 [4:24:39<2:36:22,  2.07s/it]
                                                        

 62%|██████▏   | 7457/12000 [4:24:39<2:36:22,  2.07s/it]
 62%|██████▏   | 7458/12000 [4:24:42<2:57:49,  2.35s/it]
                                                        

 62%|██████▏   | 7458/12000 [4:24:42<2:57:49,  2.35s/it]
 62%|██████▏   | 7459/12000 [4:24:44<2:56:45,  2.34s/it]
                                                        

 62%|██████▏   | 7459/12000 [4:24:44<2:56:45,  2.34s/it]
 62%|██████▏   | 7460/12000 [4:24:49<3:47:17,  3.00s/it]
                                                        

 62%|██████▏   | 7460/12000 [4:24:49<3:47:17,  3.00s/it]
 62%|██████▏   | 7461/12000 [4:24:51<3:22:40,  2.68s/it]
                                                        

 62%|██████▏   | 7461/12000 [4:24:51<3:22:40,  2.68s/it]
 62%|██████▏   | 7462/12000 [4:24:54<3:21:05,  2.66s/it]
                                                        

 62%|██████▏   | 7462/12000 [4:24:54<3:21:05,  2.66s/it]
 62%|██████▏   | 7463/12000 [4:24:56<3:05:29,  2.45s/it]
                                                        

 62%|██████▏   | 7463/12000 [4:24:56<3:05:29,  2.45s/it]
 62%|██████▏   | 7464/12000 [4:24:57<2:46:32,  2.20s/it]
                                                        

 62%|██████▏   | 7464/12000 [4:24:57<2:46:32,  2.20s/it]
 62%|██████▏   | 7465/12000 [4:25:02<3:43:26,  2.96s/it]
                                                        

 62%|██████▏   | 7465/12000 [4:25:02<3:43:26,  2.96s/it]
 62%|██████▏   | 7466/12000 [4:25:03<3:13:33,  2.56s/it]
                                                        

 62%|██████▏   | 7466/12000 [4:25:03<3:13:33,  2.56s/it]
 62%|██████▏   | 7467/12000 [4:25:06<3:07:00,  2.48s/it]
                                                        

 62%|██████▏   | 7467/12000 [4:25:06<3:07:00,  2.48s/it]
 62%|██████▏   | 7468/12000 [4:25:08<2:52:01,  2.28s/it]
                                                        

 62%|██████▏   | 7468/12000 [4:25:08<2:52:01,  2.28s/it]
 62%|██████▏   | 7469/12000 [4:25:12<3:36:17,  2.86s/it]
                                                        

 62%|██████▏   | 7469/12000 [4:25:12<3:36:17,  2.86s/it]
 62%|██████▏   | 7470/12000 [4:25:14<3:14:22,  2.57s/it]
                                                        

 62%|██████▏   | 7470/12000 [4:25:14<3:14:22,  2.57s/it]
 62%|██████▏   | 7471/12000 [4:25:16<3:15:25,  2.59s/it]
                                                        

 62%|██████▏   | 7471/12000 [4:25:16<3:15:25,  2.59s/it]
 62%|██████▏   | 7472/12000 [4:25:21<4:07:58,  3.29s/it]
                                                        

 62%|██████▏   | 7472/12000 [4:25:21<4:07:58,  3.29s/it]
 62%|██████▏   | 7473/12000 [4:25:24<3:46:49,  3.01s/it]
                                                        

 62%|██████▏   | 7473/12000 [4:25:24<3:46:49,  3.01s/it]
 62%|██████▏   | 7474/12000 [4:25:27<3:47:27,  3.02s/it]
                                                        

 62%|██████▏   | 7474/12000 [4:25:27<3:47:27,  3.02s/it]
 62%|██████▏   | 7475/12000 [4:25:30<3:52:20,  3.08s/it]
                                                        

 62%|██████▏   | 7475/12000 [4:25:30<3:52:20,  3.08s/it]
 62%|██████▏   | 7476/12000 [4:25:33<3:42:28,  2.95s/it]
                                                        

 62%|██████▏   | 7476/12000 [4:25:33<3:42:28,  2.95s/it]
 62%|██████▏   | 7477/12000 [4:25:35<3:38:19,  2.90s/it]
                                                        
{'loss': '0.6753', 'grad_norm': '0.3796', 'learning_rate': '6.939e-05', 'entropy': '0.6699', 'num_tokens': '4.009e+07', 'mean_token_accuracy': '0.8227', 'epoch': '0.06575'}
{'loss': '1.079', 'grad_norm': '0.3928', 'learning_rate': '6.936e-05', 'entropy': '1.089', 'num_tokens': '4.01e+07', 'mean_token_accuracy': '0.7406', 'epoch': '0.06576'}
{'loss': '0.7838', 'grad_norm': '0.4226', 'learning_rate': '6.933e-05', 'entropy': '0.8048', 'num_tokens': '4.01e+07', 'mean_token_accuracy': '0.7923', 'epoch': '0.06577'}
{'loss': '0.8516', 'grad_norm': '0.387', 'learning_rate': '6.931e-05', 'entropy': '0.8586', 'num_tokens': '4.011e+07', 'mean_token_accuracy': '0.7789', 'epoch': '0.06578'}
{'loss': '0.7446', 'grad_norm': '0.4282', 'learning_rate': '6.928e-05', 'entropy': '0.7848', 'num_tokens': '4.011e+07', 'mean_token_accuracy': '0.8064', 'epoch': '0.06579'}
{'loss': '1.109', 'grad_norm': '0.3888', 'learning_rate': '6.926e-05', 'entropy': '1.086', 'num_tokens': '4.012e+07', 'mean_token_accuracy': '0.7289', 'epoch': '0.0658'}
{'loss': '1.207', 'grad_norm': '0.4209', 'learning_rate': '6.923e-05', 'entropy': '0.9864', 'num_tokens': '4.012e+07', 'mean_token_accuracy': '0.7568', 'epoch': '0.06581'}
{'loss': '0.8671', 'grad_norm': '0.4375', 'learning_rate': '6.92e-05', 'entropy': '0.8294', 'num_tokens': '4.013e+07', 'mean_token_accuracy': '0.8009', 'epoch': '0.06582'}
{'loss': '0.9528', 'grad_norm': '0.3732', 'learning_rate': '6.918e-05', 'entropy': '0.9771', 'num_tokens': '4.013e+07', 'mean_token_accuracy': '0.7552', 'epoch': '0.06583'}
{'loss': '0.9075', 'grad_norm': '0.3863', 'learning_rate': '6.915e-05', 'entropy': '1.044', 'num_tokens': '4.014e+07', 'mean_token_accuracy': '0.7492', 'epoch': '0.06583'}
{'loss': '0.5746', 'grad_norm': '0.2955', 'learning_rate': '6.912e-05', 'entropy': '0.5936', 'num_tokens': '4.014e+07', 'mean_token_accuracy': '0.8459', 'epoch': '0.06584'}
{'loss': '0.577', 'grad_norm': '0.2922', 'learning_rate': '6.91e-05', 'entropy': '0.5731', 'num_tokens': '4.015e+07', 'mean_token_accuracy': '0.8325', 'epoch': '0.06585'}
{'loss': '0.857', 'grad_norm': '0.3784', 'learning_rate': '6.907e-05', 'entropy': '0.822', 'num_tokens': '4.015e+07', 'mean_token_accuracy': '0.7994', 'epoch': '0.06586'}
{'loss': '0.9152', 'grad_norm': '0.42', 'learning_rate': '6.905e-05', 'entropy': '0.9667', 'num_tokens': '4.016e+07', 'mean_token_accuracy': '0.7623', 'epoch': '0.06587'}
{'loss': '0.7368', 'grad_norm': '0.3103', 'learning_rate': '6.902e-05', 'entropy': '0.8483', 'num_tokens': '4.017e+07', 'mean_token_accuracy': '0.785', 'epoch': '0.06588'}
{'loss': '0.8598', 'grad_norm': '0.4165', 'learning_rate': '6.899e-05', 'entropy': '0.8452', 'num_tokens': '4.017e+07', 'mean_token_accuracy': '0.7929', 'epoch': '0.06589'}
{'loss': '0.8066', 'grad_norm': '0.4375', 'learning_rate': '6.897e-05', 'entropy': '0.7964', 'num_tokens': '4.017e+07', 'mean_token_accuracy': '0.7891', 'epoch': '0.0659'}
{'loss': '1.007', 'grad_norm': '0.387', 'learning_rate': '6.894e-05', 'entropy': '0.9538', 'num_tokens': '4.018e+07', 'mean_token_accuracy': '0.7557', 'epoch': '0.0659'}
{'loss': '0.9217', 'grad_norm': '0.3934', 'learning_rate': '6.891e-05', 'entropy': '1.022', 'num_tokens': '4.018e+07', 'mean_token_accuracy': '0.7492', 'epoch': '0.06591'}
{'loss': '0.9447', 'grad_norm': '0.4309', 'learning_rate': '6.889e-05', 'entropy': '0.9809', 'num_tokens': '4.019e+07', 'mean_token_accuracy': '0.7559', 'epoch': '0.06592'}
{'loss': '0.7966', 'grad_norm': '0.3284', 'learning_rate': '6.886e-05', 'entropy': '0.6603', 'num_tokens': '4.02e+07', 'mean_token_accuracy': '0.8318', 'epoch': '0.06593'}
{'loss': '1.122', 'grad_norm': '0.538', 'learning_rate': '6.884e-05', 'entropy': '0.9863', 'num_tokens': '4.02e+07', 'mean_token_accuracy': '0.7493', 'epoch': '0.06594'}
{'loss': '0.9302', 'grad_norm': '0.4262', 'learning_rate': '6.881e-05', 'entropy': '0.8758', 'num_tokens': '4.021e+07', 'mean_token_accuracy': '0.7685', 'epoch': '0.06595'}
{'loss': '1.317', 'grad_norm': '0.4813', 'learning_rate': '6.878e-05', 'entropy': '1.102', 'num_tokens': '4.022e+07', 'mean_token_accuracy': '0.7201', 'epoch': '0.06596'}
{'loss': '0.516', 'grad_norm': '0.3106', 'learning_rate': '6.876e-05', 'entropy': '0.6391', 'num_tokens': '4.022e+07', 'mean_token_accuracy': '0.8246', 'epoch': '0.06597'}
{'loss': '0.6623', 'grad_norm': '0.3352', 'learning_rate': '6.873e-05', 'entropy': '0.6958', 'num_tokens': '4.023e+07', 'mean_token_accuracy': '0.8176', 'epoch': '0.06598'}
{'loss': '0.6934', 'grad_norm': '0.327', 'learning_rate': '6.871e-05', 'entropy': '0.6506', 'num_tokens': '4.023e+07', 'mean_token_accuracy': '0.8299', 'epoch': '0.06598'}
{'loss': '0.8778', 'grad_norm': '0.4026', 'learning_rate': '6.868e-05', 'entropy': '0.95', 'num_tokens': '4.024e+07', 'mean_token_accuracy': '0.7794', 'epoch': '0.06599'}
{'loss': '0.8869', 'grad_norm': '0.3642', 'learning_rate': '6.865e-05', 'entropy': '0.8948', 'num_tokens': '4.024e+07', 'mean_token_accuracy': '0.7828', 'epoch': '0.066'}
{'loss': '0.7512', 'grad_norm': '0.4116', 'learning_rate': '6.863e-05', 'entropy': '0.7508', 'num_tokens': '4.025e+07', 'mean_token_accuracy': '0.8063', 'epoch': '0.06601'}
{'loss': '0.9922', 'grad_norm': '0.3934', 'learning_rate': '6.86e-05', 'entropy': '0.836', 'num_tokens': '4.026e+07', 'mean_token_accuracy': '0.7928', 'epoch': '0.06602'}
{'loss': '1.1', 'grad_norm': '0.3497', 'learning_rate': '6.857e-05', 'entropy': '1.155', 'num_tokens': '4.026e+07', 'mean_token_accuracy': '0.7148', 'epoch': '0.06603'}
{'loss': '1.015', 'grad_norm': '0.3978', 'learning_rate': '6.855e-05', 'entropy': '1.055', 'num_tokens': '4.027e+07', 'mean_token_accuracy': '0.7613', 'epoch': '0.06604'}
{'loss': '0.9022', 'grad_norm': '0.447', 'learning_rate': '6.852e-05', 'entropy': '0.8792', 'num_tokens': '4.027e+07', 'mean_token_accuracy': '0.7815', 'epoch': '0.06605'}
{'loss': '0.8215', 'grad_norm': '0.3515', 'learning_rate': '6.85e-05', 'entropy': '0.8385', 'num_tokens': '4.028e+07', 'mean_token_accuracy': '0.7949', 'epoch': '0.06606'}
{'loss': '1.274', 'grad_norm': '0.3876', 'learning_rate': '6.847e-05', 'entropy': '1.133', 'num_tokens': '4.029e+07', 'mean_token_accuracy': '0.7189', 'epoch': '0.06606'}
{'loss': '0.7486', 'grad_norm': '0.4246', 'learning_rate': '6.844e-05', 'entropy': '0.8035', 'num_tokens': '4.029e+07', 'mean_token_accuracy': '0.7899', 'epoch': '0.06607'}
{'loss': '1.041', 'grad_norm': '0.5395', 'learning_rate': '6.842e-05', 'entropy': '0.9775', 'num_tokens': '4.03e+07', 'mean_token_accuracy': '0.7567', 'epoch': '0.06608'}
{'loss': '0.8824', 'grad_norm': '0.4057', 'learning_rate': '6.839e-05', 'entropy': '0.8914', 'num_tokens': '4.03e+07', 'mean_token_accuracy': '0.7772', 'epoch': '0.06609'}
{'loss': '0.805', 'grad_norm': '0.3652', 'learning_rate': '6.837e-05', 'entropy': '0.7206', 'num_tokens': '4.031e+07', 'mean_token_accuracy': '0.8296', 'epoch': '0.0661'}
{'loss': '0.876', 'grad_norm': '0.4294', 'learning_rate': '6.834e-05', 'entropy': '0.8743', 'num_tokens': '4.031e+07', 'mean_token_accuracy': '0.7879', 'epoch': '0.06611'}
{'loss': '1.362', 'grad_norm': '0.4623', 'learning_rate': '6.831e-05', 'entropy': '1.319', 'num_tokens': '4.032e+07', 'mean_token_accuracy': '0.6987', 'epoch': '0.06612'}
{'loss': '1.092', 'grad_norm': '0.3478', 'learning_rate': '6.829e-05', 'entropy': '0.9394', 'num_tokens': '4.033e+07', 'mean_token_accuracy': '0.7766', 'epoch': '0.06613'}
{'loss': '0.8859', 'grad_norm': '0.4059', 'learning_rate': '6.826e-05', 'entropy': '0.7986', 'num_tokens': '4.034e+07', 'mean_token_accuracy': '0.8019', 'epoch': '0.06613'}
{'loss': '0.8548', 'grad_norm': '0.3535', 'learning_rate': '6.823e-05', 'entropy': '0.7453', 'num_tokens': '4.035e+07', 'mean_token_accuracy': '0.8107', 'epoch': '0.06614'}
{'loss': '0.7919', 'grad_norm': '0.3481', 'learning_rate': '6.821e-05', 'entropy': '0.8123', 'num_tokens': '4.035e+07', 'mean_token_accuracy': '0.7949', 'epoch': '0.06615'}
{'loss': '0.7561', 'grad_norm': '0.3015', 'learning_rate': '6.818e-05', 'entropy': '0.7288', 'num_tokens': '4.036e+07', 'mean_token_accuracy': '0.8214', 'epoch': '0.06616'}

 62%|██████▏   | 7477/12000 [4:25:35<3:38:19,  2.90s/it]
 62%|██████▏   | 7478/12000 [4:25:37<3:08:20,  2.50s/it]
                                                        

 62%|██████▏   | 7478/12000 [4:25:37<3:08:20,  2.50s/it]
 62%|██████▏   | 7479/12000 [4:25:38<2:45:02,  2.19s/it]
                                                        

 62%|██████▏   | 7479/12000 [4:25:38<2:45:02,  2.19s/it]
 62%|██████▏   | 7480/12000 [4:25:40<2:43:11,  2.17s/it]
                                                        

 62%|██████▏   | 7480/12000 [4:25:40<2:43:11,  2.17s/it]
 62%|██████▏   | 7481/12000 [4:25:44<3:15:36,  2.60s/it]
                                                        

 62%|██████▏   | 7481/12000 [4:25:44<3:15:36,  2.60s/it]
 62%|██████▏   | 7482/12000 [4:25:46<3:03:11,  2.43s/it]
                                                        

 62%|██████▏   | 7482/12000 [4:25:46<3:03:11,  2.43s/it]
 62%|██████▏   | 7483/12000 [4:25:48<2:48:40,  2.24s/it]
                                                        

 62%|██████▏   | 7483/12000 [4:25:48<2:48:40,  2.24s/it]
 62%|██████▏   | 7484/12000 [4:25:50<2:37:16,  2.09s/it]
                                                        

 62%|██████▏   | 7484/12000 [4:25:50<2:37:16,  2.09s/it]
 62%|██████▏   | 7485/12000 [4:25:51<2:27:37,  1.96s/it]
                                                        

 62%|██████▏   | 7485/12000 [4:25:51<2:27:37,  1.96s/it]
 62%|██████▏   | 7486/12000 [4:25:53<2:25:23,  1.93s/it]
                                                        

 62%|██████▏   | 7486/12000 [4:25:53<2:25:23,  1.93s/it]
 62%|██████▏   | 7487/12000 [4:25:56<2:42:08,  2.16s/it]
                                                        

 62%|██████▏   | 7487/12000 [4:25:56<2:42:08,  2.16s/it]
 62%|██████▏   | 7488/12000 [4:25:58<2:42:25,  2.16s/it]
                                                        

 62%|██████▏   | 7488/12000 [4:25:58<2:42:25,  2.16s/it]
 62%|██████▏   | 7489/12000 [4:25:59<2:27:10,  1.96s/it]
                                                        

 62%|██████▏   | 7489/12000 [4:25:59<2:27:10,  1.96s/it]
 62%|██████▏   | 7490/12000 [4:26:01<2:21:39,  1.88s/it]
                                                        

 62%|██████▏   | 7490/12000 [4:26:01<2:21:39,  1.88s/it]
 62%|██████▏   | 7491/12000 [4:26:06<3:34:00,  2.85s/it]
                                                        

 62%|██████▏   | 7491/12000 [4:26:06<3:34:00,  2.85s/it]
 62%|██████▏   | 7492/12000 [4:26:08<3:01:33,  2.42s/it]
                                                        

 62%|██████▏   | 7492/12000 [4:26:08<3:01:33,  2.42s/it]
 62%|██████▏   | 7493/12000 [4:26:10<2:50:33,  2.27s/it]
                                                        

 62%|██████▏   | 7493/12000 [4:26:10<2:50:33,  2.27s/it]
 62%|██████▏   | 7494/12000 [4:26:14<3:27:50,  2.77s/it]
                                                        

 62%|██████▏   | 7494/12000 [4:26:14<3:27:50,  2.77s/it]
 62%|██████▏   | 7495/12000 [4:26:15<3:07:06,  2.49s/it]
                                                        

 62%|██████▏   | 7495/12000 [4:26:15<3:07:06,  2.49s/it]
 62%|██████▏   | 7496/12000 [4:26:17<2:46:41,  2.22s/it]
                                                        

 62%|██████▏   | 7496/12000 [4:26:17<2:46:41,  2.22s/it]
 62%|██████▏   | 7497/12000 [4:26:19<2:31:37,  2.02s/it]
                                                        

 62%|██████▏   | 7497/12000 [4:26:19<2:31:37,  2.02s/it]
 62%|██████▏   | 7498/12000 [4:26:20<2:27:31,  1.97s/it]
                                                        

 62%|██████▏   | 7498/12000 [4:26:20<2:27:31,  1.97s/it]
 62%|██████▏   | 7499/12000 [4:26:22<2:21:55,  1.89s/it]
                                                        

 62%|██████▏   | 7499/12000 [4:26:22<2:21:55,  1.89s/it]
 62%|██████▎   | 7500/12000 [4:26:25<2:39:03,  2.12s/it]
                                                        

 62%|██████▎   | 7500/12000 [4:26:25<2:39:03,  2.12s/it]
 63%|██████▎   | 7501/12000 [4:26:28<3:13:52,  2.59s/it]
                                                        

 63%|██████▎   | 7501/12000 [4:26:28<3:13:52,  2.59s/it]
 63%|██████▎   | 7502/12000 [4:26:32<3:30:32,  2.81s/it]
                                                        

 63%|██████▎   | 7502/12000 [4:26:32<3:30:32,  2.81s/it]
 63%|██████▎   | 7503/12000 [4:26:34<3:27:49,  2.77s/it]
                                                        

 63%|██████▎   | 7503/12000 [4:26:34<3:27:49,  2.77s/it]
 63%|██████▎   | 7504/12000 [4:26:36<3:03:34,  2.45s/it]
                                                        

 63%|██████▎   | 7504/12000 [4:26:36<3:03:34,  2.45s/it]
 63%|██████▎   | 7505/12000 [4:26:38<2:43:28,  2.18s/it]
                                                        

 63%|██████▎   | 7505/12000 [4:26:38<2:43:28,  2.18s/it]
 63%|██████▎   | 7506/12000 [4:26:41<3:02:16,  2.43s/it]
                                                        

 63%|██████▎   | 7506/12000 [4:26:41<3:02:16,  2.43s/it]
 63%|██████▎   | 7507/12000 [4:26:42<2:47:23,  2.24s/it]
                                                        

 63%|██████▎   | 7507/12000 [4:26:42<2:47:23,  2.24s/it]
 63%|██████▎   | 7508/12000 [4:26:46<3:23:46,  2.72s/it]
                                                        

 63%|██████▎   | 7508/12000 [4:26:46<3:23:46,  2.72s/it]
 63%|██████▎   | 7509/12000 [4:26:48<3:01:20,  2.42s/it]
                                                        

 63%|██████▎   | 7509/12000 [4:26:48<3:01:20,  2.42s/it]
 63%|██████▎   | 7510/12000 [4:26:50<2:56:00,  2.35s/it]
                                                        

 63%|██████▎   | 7510/12000 [4:26:50<2:56:00,  2.35s/it]
 63%|██████▎   | 7511/12000 [4:26:52<2:36:30,  2.09s/it]
                                                        

 63%|██████▎   | 7511/12000 [4:26:52<2:36:30,  2.09s/it]
 63%|██████▎   | 7512/12000 [4:26:53<2:23:50,  1.92s/it]
                                                        

 63%|██████▎   | 7512/12000 [4:26:53<2:23:50,  1.92s/it]
 63%|██████▎   | 7513/12000 [4:26:58<3:23:34,  2.72s/it]
                                                        

 63%|██████▎   | 7513/12000 [4:26:58<3:23:34,  2.72s/it]
 63%|██████▎   | 7514/12000 [4:27:01<3:23:03,  2.72s/it]
                                                        

 63%|██████▎   | 7514/12000 [4:27:01<3:23:03,  2.72s/it]
 63%|██████▎   | 7515/12000 [4:27:02<2:59:11,  2.40s/it]
                                                        

 63%|██████▎   | 7515/12000 [4:27:02<2:59:11,  2.40s/it]
 63%|██████▎   | 7516/12000 [4:27:05<3:07:38,  2.51s/it]
                                                        

 63%|██████▎   | 7516/12000 [4:27:05<3:07:38,  2.51s/it]
 63%|██████▎   | 7517/12000 [4:27:08<3:09:18,  2.53s/it]
                                                        

 63%|██████▎   | 7517/12000 [4:27:08<3:09:18,  2.53s/it]
 63%|██████▎   | 7518/12000 [4:27:09<2:49:57,  2.28s/it]
                                                        

 63%|██████▎   | 7518/12000 [4:27:09<2:49:57,  2.28s/it]
 63%|██████▎   | 7519/12000 [4:27:11<2:30:25,  2.01s/it]
                                                        

 63%|██████▎   | 7519/12000 [4:27:11<2:30:25,  2.01s/it]
 63%|██████▎   | 7520/12000 [4:27:12<2:21:56,  1.90s/it]
                                                        

 63%|██████▎   | 7520/12000 [4:27:12<2:21:56,  1.90s/it]
 63%|██████▎   | 7521/12000 [4:27:16<2:56:19,  2.36s/it]
                                                        

 63%|██████▎   | 7521/12000 [4:27:16<2:56:19,  2.36s/it]
 63%|██████▎   | 7522/12000 [4:27:19<3:08:04,  2.52s/it]
                                                        

 63%|██████▎   | 7522/12000 [4:27:19<3:08:04,  2.52s/it]
 63%|██████▎   | 7523/12000 [4:27:20<2:53:34,  2.33s/it]
                                                        

 63%|██████▎   | 7523/12000 [4:27:20<2:53:34,  2.33s/it]
 63%|██████▎   | 7524/12000 [4:27:22<2:37:20,  2.11s/it]
                                                        
{'loss': '1', 'grad_norm': '0.4472', 'learning_rate': '6.816e-05', 'entropy': '1.02', 'num_tokens': '4.037e+07', 'mean_token_accuracy': '0.7629', 'epoch': '0.06617'}
{'loss': '0.6101', 'grad_norm': '0.3385', 'learning_rate': '6.813e-05', 'entropy': '0.6543', 'num_tokens': '4.037e+07', 'mean_token_accuracy': '0.8261', 'epoch': '0.06618'}
{'loss': '0.9022', 'grad_norm': '0.4631', 'learning_rate': '6.81e-05', 'entropy': '0.9187', 'num_tokens': '4.038e+07', 'mean_token_accuracy': '0.7774', 'epoch': '0.06619'}
{'loss': '1.052', 'grad_norm': '0.4693', 'learning_rate': '6.808e-05', 'entropy': '0.9582', 'num_tokens': '4.038e+07', 'mean_token_accuracy': '0.7526', 'epoch': '0.0662'}
{'loss': '0.9567', 'grad_norm': '0.3707', 'learning_rate': '6.805e-05', 'entropy': '0.848', 'num_tokens': '4.039e+07', 'mean_token_accuracy': '0.797', 'epoch': '0.06621'}
{'loss': '0.7727', 'grad_norm': '0.3849', 'learning_rate': '6.803e-05', 'entropy': '0.7484', 'num_tokens': '4.04e+07', 'mean_token_accuracy': '0.7931', 'epoch': '0.06621'}
{'loss': '1.046', 'grad_norm': '0.3927', 'learning_rate': '6.8e-05', 'entropy': '1.041', 'num_tokens': '4.04e+07', 'mean_token_accuracy': '0.7425', 'epoch': '0.06622'}
{'loss': '0.677', 'grad_norm': '0.3945', 'learning_rate': '6.797e-05', 'entropy': '0.7313', 'num_tokens': '4.041e+07', 'mean_token_accuracy': '0.8232', 'epoch': '0.06623'}
{'loss': '0.6628', 'grad_norm': '0.414', 'learning_rate': '6.795e-05', 'entropy': '0.7597', 'num_tokens': '4.041e+07', 'mean_token_accuracy': '0.8068', 'epoch': '0.06624'}
{'loss': '1.027', 'grad_norm': '0.4178', 'learning_rate': '6.792e-05', 'entropy': '0.9015', 'num_tokens': '4.042e+07', 'mean_token_accuracy': '0.7824', 'epoch': '0.06625'}
{'loss': '1.307', 'grad_norm': '0.4842', 'learning_rate': '6.79e-05', 'entropy': '1.315', 'num_tokens': '4.043e+07', 'mean_token_accuracy': '0.6848', 'epoch': '0.06626'}
{'loss': '1.137', 'grad_norm': '0.4545', 'learning_rate': '6.787e-05', 'entropy': '0.9269', 'num_tokens': '4.043e+07', 'mean_token_accuracy': '0.7771', 'epoch': '0.06627'}
{'loss': '0.8242', 'grad_norm': '0.3593', 'learning_rate': '6.784e-05', 'entropy': '0.7947', 'num_tokens': '4.043e+07', 'mean_token_accuracy': '0.8109', 'epoch': '0.06628'}
{'loss': '0.7778', 'grad_norm': '0.3627', 'learning_rate': '6.782e-05', 'entropy': '0.7766', 'num_tokens': '4.044e+07', 'mean_token_accuracy': '0.8001', 'epoch': '0.06629'}
{'loss': '0.6723', 'grad_norm': '0.3606', 'learning_rate': '6.779e-05', 'entropy': '0.7514', 'num_tokens': '4.045e+07', 'mean_token_accuracy': '0.8078', 'epoch': '0.06629'}
{'loss': '0.9643', 'grad_norm': '0.3911', 'learning_rate': '6.776e-05', 'entropy': '0.9802', 'num_tokens': '4.045e+07', 'mean_token_accuracy': '0.7623', 'epoch': '0.0663'}
{'loss': '0.6412', 'grad_norm': '0.3968', 'learning_rate': '6.774e-05', 'entropy': '0.6866', 'num_tokens': '4.046e+07', 'mean_token_accuracy': '0.824', 'epoch': '0.06631'}
{'loss': '1.26', 'grad_norm': '0.3716', 'learning_rate': '6.771e-05', 'entropy': '1.176', 'num_tokens': '4.047e+07', 'mean_token_accuracy': '0.7204', 'epoch': '0.06632'}
{'loss': '0.8741', 'grad_norm': '0.465', 'learning_rate': '6.769e-05', 'entropy': '0.8416', 'num_tokens': '4.047e+07', 'mean_token_accuracy': '0.7773', 'epoch': '0.06633'}
{'loss': '0.6451', 'grad_norm': '0.3832', 'learning_rate': '6.766e-05', 'entropy': '0.6945', 'num_tokens': '4.048e+07', 'mean_token_accuracy': '0.8168', 'epoch': '0.06634'}
{'loss': '0.6572', 'grad_norm': '0.3687', 'learning_rate': '6.763e-05', 'entropy': '0.6754', 'num_tokens': '4.048e+07', 'mean_token_accuracy': '0.8211', 'epoch': '0.06635'}
{'loss': '0.7694', 'grad_norm': '0.4214', 'learning_rate': '6.761e-05', 'entropy': '0.7204', 'num_tokens': '4.049e+07', 'mean_token_accuracy': '0.8129', 'epoch': '0.06636'}
{'loss': '0.9248', 'grad_norm': '0.3506', 'learning_rate': '6.758e-05', 'entropy': '0.9673', 'num_tokens': '4.049e+07', 'mean_token_accuracy': '0.7738', 'epoch': '0.06636'}
{'loss': '1.004', 'grad_norm': '0.4486', 'learning_rate': '6.756e-05', 'entropy': '0.9802', 'num_tokens': '4.05e+07', 'mean_token_accuracy': '0.7516', 'epoch': '0.06637'}
{'loss': '0.6163', 'grad_norm': '0.3243', 'learning_rate': '6.753e-05', 'entropy': '0.6544', 'num_tokens': '4.051e+07', 'mean_token_accuracy': '0.8275', 'epoch': '0.06638'}
{'loss': '0.7628', 'grad_norm': '0.3728', 'learning_rate': '6.75e-05', 'entropy': '0.7561', 'num_tokens': '4.052e+07', 'mean_token_accuracy': '0.798', 'epoch': '0.06639'}
{'loss': '0.7184', 'grad_norm': '0.3425', 'learning_rate': '6.748e-05', 'entropy': '1.048', 'num_tokens': '4.052e+07', 'mean_token_accuracy': '0.7505', 'epoch': '0.0664'}
{'loss': '0.8857', 'grad_norm': '0.368', 'learning_rate': '6.745e-05', 'entropy': '0.8953', 'num_tokens': '4.053e+07', 'mean_token_accuracy': '0.7856', 'epoch': '0.06641'}
{'loss': '0.7851', 'grad_norm': '0.3455', 'learning_rate': '6.743e-05', 'entropy': '0.8557', 'num_tokens': '4.053e+07', 'mean_token_accuracy': '0.793', 'epoch': '0.06642'}
{'loss': '0.5405', 'grad_norm': '0.2887', 'learning_rate': '6.74e-05', 'entropy': '0.5999', 'num_tokens': '4.054e+07', 'mean_token_accuracy': '0.8417', 'epoch': '0.06643'}
{'loss': '0.8929', 'grad_norm': '0.4563', 'learning_rate': '6.737e-05', 'entropy': '0.8848', 'num_tokens': '4.055e+07', 'mean_token_accuracy': '0.7756', 'epoch': '0.06644'}
{'loss': '0.8756', 'grad_norm': '0.4152', 'learning_rate': '6.735e-05', 'entropy': '0.9771', 'num_tokens': '4.055e+07', 'mean_token_accuracy': '0.7703', 'epoch': '0.06644'}
{'loss': '1.123', 'grad_norm': '0.4375', 'learning_rate': '6.732e-05', 'entropy': '1.12', 'num_tokens': '4.056e+07', 'mean_token_accuracy': '0.7367', 'epoch': '0.06645'}
{'loss': '0.7237', 'grad_norm': '0.4176', 'learning_rate': '6.73e-05', 'entropy': '0.7929', 'num_tokens': '4.057e+07', 'mean_token_accuracy': '0.8133', 'epoch': '0.06646'}
{'loss': '0.792', 'grad_norm': '0.3754', 'learning_rate': '6.727e-05', 'entropy': '0.8216', 'num_tokens': '4.057e+07', 'mean_token_accuracy': '0.8027', 'epoch': '0.06647'}
{'loss': '0.777', 'grad_norm': '0.3984', 'learning_rate': '6.724e-05', 'entropy': '0.7456', 'num_tokens': '4.057e+07', 'mean_token_accuracy': '0.8117', 'epoch': '0.06648'}
{'loss': '0.5699', 'grad_norm': '0.339', 'learning_rate': '6.722e-05', 'entropy': '0.6329', 'num_tokens': '4.058e+07', 'mean_token_accuracy': '0.8401', 'epoch': '0.06649'}
{'loss': '0.8154', 'grad_norm': '0.3782', 'learning_rate': '6.719e-05', 'entropy': '0.7632', 'num_tokens': '4.059e+07', 'mean_token_accuracy': '0.8036', 'epoch': '0.0665'}
{'loss': '0.8186', 'grad_norm': '0.3958', 'learning_rate': '6.717e-05', 'entropy': '0.8304', 'num_tokens': '4.059e+07', 'mean_token_accuracy': '0.7849', 'epoch': '0.06651'}
{'loss': '0.7238', 'grad_norm': '0.3611', 'learning_rate': '6.714e-05', 'entropy': '0.6942', 'num_tokens': '4.06e+07', 'mean_token_accuracy': '0.8151', 'epoch': '0.06652'}
{'loss': '0.864', 'grad_norm': '0.4019', 'learning_rate': '6.711e-05', 'entropy': '0.8004', 'num_tokens': '4.061e+07', 'mean_token_accuracy': '0.7941', 'epoch': '0.06652'}
{'loss': '0.9473', 'grad_norm': '0.3906', 'learning_rate': '6.709e-05', 'entropy': '1.008', 'num_tokens': '4.061e+07', 'mean_token_accuracy': '0.7652', 'epoch': '0.06653'}
{'loss': '0.5475', 'grad_norm': '0.292', 'learning_rate': '6.706e-05', 'entropy': '0.607', 'num_tokens': '4.062e+07', 'mean_token_accuracy': '0.8324', 'epoch': '0.06654'}
{'loss': '0.8299', 'grad_norm': '0.3207', 'learning_rate': '6.704e-05', 'entropy': '0.8209', 'num_tokens': '4.062e+07', 'mean_token_accuracy': '0.7981', 'epoch': '0.06655'}
{'loss': '0.6794', 'grad_norm': '0.341', 'learning_rate': '6.701e-05', 'entropy': '0.7491', 'num_tokens': '4.063e+07', 'mean_token_accuracy': '0.8014', 'epoch': '0.06656'}
{'loss': '0.8435', 'grad_norm': '0.3377', 'learning_rate': '6.698e-05', 'entropy': '0.7746', 'num_tokens': '4.064e+07', 'mean_token_accuracy': '0.8091', 'epoch': '0.06657'}
{'loss': '0.8017', 'grad_norm': '0.461', 'learning_rate': '6.696e-05', 'entropy': '0.8051', 'num_tokens': '4.064e+07', 'mean_token_accuracy': '0.7935', 'epoch': '0.06658'}

 63%|██████▎   | 7524/12000 [4:27:22<2:37:20,  2.11s/it]
 63%|██████▎   | 7525/12000 [4:27:24<2:26:50,  1.97s/it]
                                                        

 63%|██████▎   | 7525/12000 [4:27:24<2:26:50,  1.97s/it]
 63%|██████▎   | 7526/12000 [4:27:25<2:16:50,  1.84s/it]
                                                        

 63%|██████▎   | 7526/12000 [4:27:25<2:16:50,  1.84s/it]
 63%|██████▎   | 7527/12000 [4:27:27<2:07:48,  1.71s/it]
                                                        

 63%|██████▎   | 7527/12000 [4:27:27<2:07:48,  1.71s/it]
 63%|██████▎   | 7528/12000 [4:27:28<2:06:29,  1.70s/it]
                                                        

 63%|██████▎   | 7528/12000 [4:27:28<2:06:29,  1.70s/it]
 63%|██████▎   | 7529/12000 [4:27:31<2:23:54,  1.93s/it]
                                                        

 63%|██████▎   | 7529/12000 [4:27:31<2:23:54,  1.93s/it]
 63%|██████▎   | 7530/12000 [4:27:32<2:17:55,  1.85s/it]
                                                        

 63%|██████▎   | 7530/12000 [4:27:32<2:17:55,  1.85s/it]
 63%|██████▎   | 7531/12000 [4:27:35<2:40:21,  2.15s/it]
                                                        

 63%|██████▎   | 7531/12000 [4:27:35<2:40:21,  2.15s/it]
 63%|██████▎   | 7532/12000 [4:27:37<2:27:43,  1.98s/it]
                                                        

 63%|██████▎   | 7532/12000 [4:27:37<2:27:43,  1.98s/it]
 63%|██████▎   | 7533/12000 [4:27:39<2:26:36,  1.97s/it]
                                                        

 63%|██████▎   | 7533/12000 [4:27:39<2:26:36,  1.97s/it]
 63%|██████▎   | 7534/12000 [4:27:41<2:28:12,  1.99s/it]
                                                        

 63%|██████▎   | 7534/12000 [4:27:41<2:28:12,  1.99s/it]
 63%|██████▎   | 7535/12000 [4:27:45<3:18:43,  2.67s/it]
                                                        

 63%|██████▎   | 7535/12000 [4:27:45<3:18:43,  2.67s/it]
 63%|██████▎   | 7536/12000 [4:27:47<3:01:36,  2.44s/it]
                                                        

 63%|██████▎   | 7536/12000 [4:27:47<3:01:36,  2.44s/it]
 63%|██████▎   | 7537/12000 [4:27:49<2:41:50,  2.18s/it]
                                                        

 63%|██████▎   | 7537/12000 [4:27:49<2:41:50,  2.18s/it]
 63%|██████▎   | 7538/12000 [4:27:50<2:27:04,  1.98s/it]
                                                        

 63%|██████▎   | 7538/12000 [4:27:50<2:27:04,  1.98s/it]
 63%|██████▎   | 7539/12000 [4:27:52<2:35:02,  2.09s/it]
                                                        

 63%|██████▎   | 7539/12000 [4:27:52<2:35:02,  2.09s/it]
 63%|██████▎   | 7540/12000 [4:27:57<3:32:27,  2.86s/it]
                                                        

 63%|██████▎   | 7540/12000 [4:27:57<3:32:27,  2.86s/it]
 63%|██████▎   | 7541/12000 [4:28:01<4:00:53,  3.24s/it]
                                                        

 63%|██████▎   | 7541/12000 [4:28:01<4:00:53,  3.24s/it]
 63%|██████▎   | 7542/12000 [4:28:04<3:49:55,  3.09s/it]
                                                        

 63%|██████▎   | 7542/12000 [4:28:04<3:49:55,  3.09s/it]
 63%|██████▎   | 7543/12000 [4:28:06<3:22:50,  2.73s/it]
                                                        

 63%|██████▎   | 7543/12000 [4:28:06<3:22:50,  2.73s/it]
 63%|██████▎   | 7544/12000 [4:28:08<3:02:43,  2.46s/it]
                                                        

 63%|██████▎   | 7544/12000 [4:28:08<3:02:43,  2.46s/it]
 63%|██████▎   | 7545/12000 [4:28:10<2:48:07,  2.26s/it]
                                                        

 63%|██████▎   | 7545/12000 [4:28:10<2:48:07,  2.26s/it]
 63%|██████▎   | 7546/12000 [4:28:13<3:06:13,  2.51s/it]
                                                        

 63%|██████▎   | 7546/12000 [4:28:13<3:06:13,  2.51s/it]
 63%|██████▎   | 7547/12000 [4:28:14<2:51:20,  2.31s/it]
                                                        

 63%|██████▎   | 7547/12000 [4:28:14<2:51:20,  2.31s/it]
 63%|██████▎   | 7548/12000 [4:28:16<2:32:49,  2.06s/it]
                                                        

 63%|██████▎   | 7548/12000 [4:28:16<2:32:49,  2.06s/it]
 63%|██████▎   | 7549/12000 [4:28:18<2:40:18,  2.16s/it]
                                                        

 63%|██████▎   | 7549/12000 [4:28:18<2:40:18,  2.16s/it]
 63%|██████▎   | 7550/12000 [4:28:20<2:30:08,  2.02s/it]
                                                        

 63%|██████▎   | 7550/12000 [4:28:20<2:30:08,  2.02s/it]
 63%|██████▎   | 7551/12000 [4:28:24<3:08:11,  2.54s/it]
                                                        

 63%|██████▎   | 7551/12000 [4:28:24<3:08:11,  2.54s/it]
 63%|██████▎   | 7552/12000 [4:28:25<2:46:38,  2.25s/it]
                                                        

 63%|██████▎   | 7552/12000 [4:28:25<2:46:38,  2.25s/it]
 63%|██████▎   | 7553/12000 [4:28:27<2:39:01,  2.15s/it]
                                                        

 63%|██████▎   | 7553/12000 [4:28:27<2:39:01,  2.15s/it]
 63%|██████▎   | 7554/12000 [4:28:29<2:28:04,  2.00s/it]
                                                        

 63%|██████▎   | 7554/12000 [4:28:29<2:28:04,  2.00s/it]
 63%|██████▎   | 7555/12000 [4:28:30<2:17:56,  1.86s/it]
                                                        

 63%|██████▎   | 7555/12000 [4:28:30<2:17:56,  1.86s/it]
 63%|██████▎   | 7556/12000 [4:28:32<2:15:16,  1.83s/it]
                                                        

 63%|██████▎   | 7556/12000 [4:28:32<2:15:16,  1.83s/it]
 63%|██████▎   | 7557/12000 [4:28:36<2:51:45,  2.32s/it]
                                                        

 63%|██████▎   | 7557/12000 [4:28:36<2:51:45,  2.32s/it]
 63%|██████▎   | 7558/12000 [4:28:37<2:33:51,  2.08s/it]
                                                        

 63%|██████▎   | 7558/12000 [4:28:37<2:33:51,  2.08s/it]
 63%|██████▎   | 7559/12000 [4:28:39<2:28:42,  2.01s/it]
                                                        

 63%|██████▎   | 7559/12000 [4:28:39<2:28:42,  2.01s/it]
 63%|██████▎   | 7560/12000 [4:28:41<2:28:32,  2.01s/it]
                                                        

 63%|██████▎   | 7560/12000 [4:28:41<2:28:32,  2.01s/it]
 63%|██████▎   | 7561/12000 [4:28:43<2:18:58,  1.88s/it]
                                                        

 63%|██████▎   | 7561/12000 [4:28:43<2:18:58,  1.88s/it]
 63%|██████▎   | 7562/12000 [4:28:44<2:14:59,  1.83s/it]
                                                        

 63%|██████▎   | 7562/12000 [4:28:44<2:14:59,  1.83s/it]
 63%|██████▎   | 7563/12000 [4:28:46<2:15:17,  1.83s/it]
                                                        

 63%|██████▎   | 7563/12000 [4:28:46<2:15:17,  1.83s/it]
 63%|██████▎   | 7564/12000 [4:28:48<2:10:20,  1.76s/it]
                                                        

 63%|██████▎   | 7564/12000 [4:28:48<2:10:20,  1.76s/it]
 63%|██████▎   | 7565/12000 [4:28:50<2:28:27,  2.01s/it]
                                                        

 63%|██████▎   | 7565/12000 [4:28:50<2:28:27,  2.01s/it]
 63%|██████▎   | 7566/12000 [4:28:52<2:20:34,  1.90s/it]
                                                        

 63%|██████▎   | 7566/12000 [4:28:52<2:20:34,  1.90s/it]
 63%|██████▎   | 7567/12000 [4:28:54<2:12:49,  1.80s/it]
                                                        

 63%|██████▎   | 7567/12000 [4:28:54<2:12:49,  1.80s/it]
 63%|██████▎   | 7568/12000 [4:28:55<2:04:54,  1.69s/it]
                                                        

 63%|██████▎   | 7568/12000 [4:28:55<2:04:54,  1.69s/it]
 63%|██████▎   | 7569/12000 [4:28:57<2:08:19,  1.74s/it]
                                                        

 63%|██████▎   | 7569/12000 [4:28:57<2:08:19,  1.74s/it]
 63%|██████▎   | 7570/12000 [4:28:58<2:03:59,  1.68s/it]
                                                        

 63%|██████▎   | 7570/12000 [4:28:58<2:03:59,  1.68s/it]
 63%|██████▎   | 7571/12000 [4:29:01<2:17:06,  1.86s/it]
                                                        
{'loss': '0.852', 'grad_norm': '0.4101', 'learning_rate': '6.693e-05', 'entropy': '0.8113', 'num_tokens': '4.065e+07', 'mean_token_accuracy': '0.7959', 'epoch': '0.06659'}
{'loss': '0.7221', 'grad_norm': '0.4391', 'learning_rate': '6.691e-05', 'entropy': '0.7815', 'num_tokens': '4.065e+07', 'mean_token_accuracy': '0.7923', 'epoch': '0.06659'}
{'loss': '0.6468', 'grad_norm': '0.3218', 'learning_rate': '6.688e-05', 'entropy': '0.6846', 'num_tokens': '4.066e+07', 'mean_token_accuracy': '0.8148', 'epoch': '0.0666'}
{'loss': '0.7241', 'grad_norm': '0.4559', 'learning_rate': '6.685e-05', 'entropy': '0.7476', 'num_tokens': '4.066e+07', 'mean_token_accuracy': '0.8054', 'epoch': '0.06661'}
{'loss': '0.7823', 'grad_norm': '0.4655', 'learning_rate': '6.683e-05', 'entropy': '0.7915', 'num_tokens': '4.066e+07', 'mean_token_accuracy': '0.8028', 'epoch': '0.06662'}
{'loss': '0.7724', 'grad_norm': '0.3812', 'learning_rate': '6.68e-05', 'entropy': '0.7918', 'num_tokens': '4.067e+07', 'mean_token_accuracy': '0.8008', 'epoch': '0.06663'}
{'loss': '0.5287', 'grad_norm': '0.4152', 'learning_rate': '6.678e-05', 'entropy': '0.5963', 'num_tokens': '4.067e+07', 'mean_token_accuracy': '0.8465', 'epoch': '0.06664'}
{'loss': '1.629', 'grad_norm': '0.5137', 'learning_rate': '6.675e-05', 'entropy': '1.386', 'num_tokens': '4.068e+07', 'mean_token_accuracy': '0.6619', 'epoch': '0.06665'}
{'loss': '0.9034', 'grad_norm': '0.3874', 'learning_rate': '6.672e-05', 'entropy': '0.9052', 'num_tokens': '4.069e+07', 'mean_token_accuracy': '0.7705', 'epoch': '0.06666'}
{'loss': '1.218', 'grad_norm': '0.6465', 'learning_rate': '6.67e-05', 'entropy': '1.125', 'num_tokens': '4.069e+07', 'mean_token_accuracy': '0.7344', 'epoch': '0.06667'}
{'loss': '1.042', 'grad_norm': '0.4919', 'learning_rate': '6.667e-05', 'entropy': '1.007', 'num_tokens': '4.07e+07', 'mean_token_accuracy': '0.754', 'epoch': '0.06667'}
{'loss': '0.5902', 'grad_norm': '0.3456', 'learning_rate': '6.665e-05', 'entropy': '0.5678', 'num_tokens': '4.07e+07', 'mean_token_accuracy': '0.8412', 'epoch': '0.06668'}
{'loss': '0.9986', 'grad_norm': '0.4165', 'learning_rate': '6.662e-05', 'entropy': '0.9749', 'num_tokens': '4.071e+07', 'mean_token_accuracy': '0.7596', 'epoch': '0.06669'}
{'loss': '1.064', 'grad_norm': '0.4562', 'learning_rate': '6.659e-05', 'entropy': '0.9564', 'num_tokens': '4.071e+07', 'mean_token_accuracy': '0.7631', 'epoch': '0.0667'}
{'loss': '0.593', 'grad_norm': '0.2911', 'learning_rate': '6.657e-05', 'entropy': '0.6201', 'num_tokens': '4.072e+07', 'mean_token_accuracy': '0.8382', 'epoch': '0.06671'}
{'loss': '0.7567', 'grad_norm': '0.4121', 'learning_rate': '6.654e-05', 'entropy': '0.7551', 'num_tokens': '4.072e+07', 'mean_token_accuracy': '0.8051', 'epoch': '0.06672'}
{'loss': '1.264', 'grad_norm': '0.3849', 'learning_rate': '6.652e-05', 'entropy': '0.9562', 'num_tokens': '4.073e+07', 'mean_token_accuracy': '0.7811', 'epoch': '0.06673'}
{'loss': '0.8815', 'grad_norm': '0.3582', 'learning_rate': '6.649e-05', 'entropy': '0.882', 'num_tokens': '4.074e+07', 'mean_token_accuracy': '0.7673', 'epoch': '0.06674'}
{'loss': '1.139', 'grad_norm': '0.3903', 'learning_rate': '6.646e-05', 'entropy': '1.156', 'num_tokens': '4.075e+07', 'mean_token_accuracy': '0.7323', 'epoch': '0.06675'}
{'loss': '0.8619', 'grad_norm': '0.4557', 'learning_rate': '6.644e-05', 'entropy': '0.8395', 'num_tokens': '4.075e+07', 'mean_token_accuracy': '0.7968', 'epoch': '0.06675'}
{'loss': '1.162', 'grad_norm': '0.4519', 'learning_rate': '6.641e-05', 'entropy': '1.073', 'num_tokens': '4.076e+07', 'mean_token_accuracy': '0.7442', 'epoch': '0.06676'}
{'loss': '1.342', 'grad_norm': '0.5051', 'learning_rate': '6.639e-05', 'entropy': '1.164', 'num_tokens': '4.076e+07', 'mean_token_accuracy': '0.7312', 'epoch': '0.06677'}
{'loss': '0.8608', 'grad_norm': '0.325', 'learning_rate': '6.636e-05', 'entropy': '0.8409', 'num_tokens': '4.077e+07', 'mean_token_accuracy': '0.7976', 'epoch': '0.06678'}
{'loss': '0.8895', 'grad_norm': '0.3877', 'learning_rate': '6.633e-05', 'entropy': '1.013', 'num_tokens': '4.077e+07', 'mean_token_accuracy': '0.749', 'epoch': '0.06679'}
{'loss': '0.5347', 'grad_norm': '0.344', 'learning_rate': '6.631e-05', 'entropy': '0.5385', 'num_tokens': '4.078e+07', 'mean_token_accuracy': '0.8589', 'epoch': '0.0668'}
{'loss': '1.209', 'grad_norm': '0.4143', 'learning_rate': '6.628e-05', 'entropy': '1.141', 'num_tokens': '4.078e+07', 'mean_token_accuracy': '0.7197', 'epoch': '0.06681'}
{'loss': '0.6262', 'grad_norm': '0.398', 'learning_rate': '6.626e-05', 'entropy': '0.6682', 'num_tokens': '4.079e+07', 'mean_token_accuracy': '0.8336', 'epoch': '0.06682'}
{'loss': '0.6336', 'grad_norm': '0.3052', 'learning_rate': '6.623e-05', 'entropy': '0.6608', 'num_tokens': '4.079e+07', 'mean_token_accuracy': '0.828', 'epoch': '0.06683'}
{'loss': '0.9375', 'grad_norm': '0.3758', 'learning_rate': '6.62e-05', 'entropy': '0.9462', 'num_tokens': '4.08e+07', 'mean_token_accuracy': '0.7685', 'epoch': '0.06683'}
{'loss': '1.067', 'grad_norm': '0.4227', 'learning_rate': '6.618e-05', 'entropy': '0.9507', 'num_tokens': '4.081e+07', 'mean_token_accuracy': '0.7755', 'epoch': '0.06684'}
{'loss': '0.7778', 'grad_norm': '0.3612', 'learning_rate': '6.615e-05', 'entropy': '0.7858', 'num_tokens': '4.081e+07', 'mean_token_accuracy': '0.797', 'epoch': '0.06685'}
{'loss': '0.8598', 'grad_norm': '0.3945', 'learning_rate': '6.613e-05', 'entropy': '0.9122', 'num_tokens': '4.081e+07', 'mean_token_accuracy': '0.7745', 'epoch': '0.06686'}
{'loss': '0.5159', 'grad_norm': '0.3863', 'learning_rate': '6.61e-05', 'entropy': '0.5456', 'num_tokens': '4.082e+07', 'mean_token_accuracy': '0.8677', 'epoch': '0.06687'}
{'loss': '1.023', 'grad_norm': '0.5224', 'learning_rate': '6.607e-05', 'entropy': '1.094', 'num_tokens': '4.082e+07', 'mean_token_accuracy': '0.7343', 'epoch': '0.06688'}
{'loss': '0.87', 'grad_norm': '0.4661', 'learning_rate': '6.605e-05', 'entropy': '0.9468', 'num_tokens': '4.083e+07', 'mean_token_accuracy': '0.7673', 'epoch': '0.06689'}
{'loss': '0.9583', 'grad_norm': '0.4219', 'learning_rate': '6.602e-05', 'entropy': '0.9742', 'num_tokens': '4.083e+07', 'mean_token_accuracy': '0.7663', 'epoch': '0.0669'}
{'loss': '1.163', 'grad_norm': '0.5678', 'learning_rate': '6.6e-05', 'entropy': '1.117', 'num_tokens': '4.084e+07', 'mean_token_accuracy': '0.7299', 'epoch': '0.0669'}
{'loss': '0.5906', 'grad_norm': '0.3317', 'learning_rate': '6.597e-05', 'entropy': '0.6536', 'num_tokens': '4.084e+07', 'mean_token_accuracy': '0.8324', 'epoch': '0.06691'}
{'loss': '0.6751', 'grad_norm': '0.4194', 'learning_rate': '6.594e-05', 'entropy': '0.7078', 'num_tokens': '4.085e+07', 'mean_token_accuracy': '0.8224', 'epoch': '0.06692'}
{'loss': '0.9154', 'grad_norm': '0.5458', 'learning_rate': '6.592e-05', 'entropy': '0.8341', 'num_tokens': '4.085e+07', 'mean_token_accuracy': '0.7967', 'epoch': '0.06693'}
{'loss': '0.6339', 'grad_norm': '0.3813', 'learning_rate': '6.589e-05', 'entropy': '0.6785', 'num_tokens': '4.086e+07', 'mean_token_accuracy': '0.8311', 'epoch': '0.06694'}
{'loss': '1.144', 'grad_norm': '0.3825', 'learning_rate': '6.587e-05', 'entropy': '1.052', 'num_tokens': '4.086e+07', 'mean_token_accuracy': '0.7559', 'epoch': '0.06695'}
{'loss': '0.9127', 'grad_norm': '0.4416', 'learning_rate': '6.584e-05', 'entropy': '0.9772', 'num_tokens': '4.087e+07', 'mean_token_accuracy': '0.7582', 'epoch': '0.06696'}
{'loss': '0.6948', 'grad_norm': '0.4066', 'learning_rate': '6.582e-05', 'entropy': '0.721', 'num_tokens': '4.087e+07', 'mean_token_accuracy': '0.8114', 'epoch': '0.06697'}
{'loss': '0.6268', 'grad_norm': '0.3116', 'learning_rate': '6.579e-05', 'entropy': '0.7228', 'num_tokens': '4.088e+07', 'mean_token_accuracy': '0.8153', 'epoch': '0.06698'}
{'loss': '1.136', 'grad_norm': '0.3838', 'learning_rate': '6.576e-05', 'entropy': '1.02', 'num_tokens': '4.088e+07', 'mean_token_accuracy': '0.7311', 'epoch': '0.06698'}
{'loss': '0.8015', 'grad_norm': '0.4117', 'learning_rate': '6.574e-05', 'entropy': '0.7941', 'num_tokens': '4.088e+07', 'mean_token_accuracy': '0.7957', 'epoch': '0.06699'}

 63%|██████▎   | 7571/12000 [4:29:01<2:17:06,  1.86s/it]
 63%|██████▎   | 7572/12000 [4:29:03<2:17:45,  1.87s/it]
                                                        

 63%|██████▎   | 7572/12000 [4:29:03<2:17:45,  1.87s/it]
 63%|██████▎   | 7573/12000 [4:29:06<2:44:59,  2.24s/it]
                                                        

 63%|██████▎   | 7573/12000 [4:29:06<2:44:59,  2.24s/it]
 63%|██████▎   | 7574/12000 [4:29:07<2:27:14,  2.00s/it]
                                                        

 63%|██████▎   | 7574/12000 [4:29:07<2:27:14,  2.00s/it]
 63%|██████▎   | 7575/12000 [4:29:09<2:21:52,  1.92s/it]
                                                        

 63%|██████▎   | 7575/12000 [4:29:09<2:21:52,  1.92s/it]
 63%|██████▎   | 7576/12000 [4:29:11<2:22:01,  1.93s/it]
                                                        

 63%|██████▎   | 7576/12000 [4:29:11<2:22:01,  1.93s/it]
 63%|██████▎   | 7577/12000 [4:29:14<2:41:40,  2.19s/it]
                                                        

 63%|██████▎   | 7577/12000 [4:29:14<2:41:40,  2.19s/it]
 63%|██████▎   | 7578/12000 [4:29:15<2:33:36,  2.08s/it]
                                                        

 63%|██████▎   | 7578/12000 [4:29:15<2:33:36,  2.08s/it]
 63%|██████▎   | 7579/12000 [4:29:20<3:18:00,  2.69s/it]
                                                        

 63%|██████▎   | 7579/12000 [4:29:20<3:18:00,  2.69s/it]
 63%|██████▎   | 7580/12000 [4:29:21<2:53:09,  2.35s/it]
                                                        

 63%|██████▎   | 7580/12000 [4:29:21<2:53:09,  2.35s/it]
 63%|██████▎   | 7581/12000 [4:29:23<2:38:09,  2.15s/it]
                                                        

 63%|██████▎   | 7581/12000 [4:29:23<2:38:09,  2.15s/it]
 63%|██████▎   | 7582/12000 [4:29:24<2:25:01,  1.97s/it]
                                                        

 63%|██████▎   | 7582/12000 [4:29:24<2:25:01,  1.97s/it]
 63%|██████▎   | 7583/12000 [4:29:26<2:19:45,  1.90s/it]
                                                        

 63%|██████▎   | 7583/12000 [4:29:26<2:19:45,  1.90s/it]
 63%|██████▎   | 7584/12000 [4:29:28<2:23:37,  1.95s/it]
                                                        

 63%|██████▎   | 7584/12000 [4:29:28<2:23:37,  1.95s/it]
 63%|██████▎   | 7585/12000 [4:29:30<2:16:49,  1.86s/it]
                                                        

 63%|██████▎   | 7585/12000 [4:29:30<2:16:49,  1.86s/it]
 63%|██████▎   | 7586/12000 [4:29:32<2:17:22,  1.87s/it]
                                                        

 63%|██████▎   | 7586/12000 [4:29:32<2:17:22,  1.87s/it]
 63%|██████▎   | 7587/12000 [4:29:34<2:38:24,  2.15s/it]
                                                        

 63%|██████▎   | 7587/12000 [4:29:34<2:38:24,  2.15s/it]
 63%|██████▎   | 7588/12000 [4:29:36<2:28:31,  2.02s/it]
                                                        

 63%|██████▎   | 7588/12000 [4:29:36<2:28:31,  2.02s/it]
 63%|██████▎   | 7589/12000 [4:29:38<2:34:31,  2.10s/it]
                                                        

 63%|██████▎   | 7589/12000 [4:29:38<2:34:31,  2.10s/it]
 63%|██████▎   | 7590/12000 [4:29:40<2:22:05,  1.93s/it]
                                                        

 63%|██████▎   | 7590/12000 [4:29:40<2:22:05,  1.93s/it]
 63%|██████▎   | 7591/12000 [4:29:42<2:34:05,  2.10s/it]
                                                        

 63%|██████▎   | 7591/12000 [4:29:42<2:34:05,  2.10s/it]
 63%|██████▎   | 7592/12000 [4:29:44<2:21:21,  1.92s/it]
                                                        

 63%|██████▎   | 7592/12000 [4:29:44<2:21:21,  1.92s/it]
 63%|██████▎   | 7593/12000 [4:29:46<2:26:26,  1.99s/it]
                                                        

 63%|██████▎   | 7593/12000 [4:29:46<2:26:26,  1.99s/it]
 63%|██████▎   | 7594/12000 [4:29:48<2:16:38,  1.86s/it]
                                                        

 63%|██████▎   | 7594/12000 [4:29:48<2:16:38,  1.86s/it]
 63%|██████▎   | 7595/12000 [4:29:49<2:11:27,  1.79s/it]
                                                        

 63%|██████▎   | 7595/12000 [4:29:49<2:11:27,  1.79s/it]
 63%|██████▎   | 7596/12000 [4:29:51<2:14:15,  1.83s/it]
                                                        

 63%|██████▎   | 7596/12000 [4:29:51<2:14:15,  1.83s/it]
 63%|██████▎   | 7597/12000 [4:29:53<2:09:46,  1.77s/it]
                                                        

 63%|██████▎   | 7597/12000 [4:29:53<2:09:46,  1.77s/it]
 63%|██████▎   | 7598/12000 [4:29:55<2:16:29,  1.86s/it]
                                                        

 63%|██████▎   | 7598/12000 [4:29:55<2:16:29,  1.86s/it]
 63%|██████▎   | 7599/12000 [4:29:58<2:32:34,  2.08s/it]
                                                        

 63%|██████▎   | 7599/12000 [4:29:58<2:32:34,  2.08s/it]
 63%|██████▎   | 7600/12000 [4:29:59<2:24:06,  1.97s/it]
                                                        

 63%|██████▎   | 7600/12000 [4:29:59<2:24:06,  1.97s/it]
 63%|██████▎   | 7601/12000 [4:30:04<3:33:02,  2.91s/it]
                                                        

 63%|██████▎   | 7601/12000 [4:30:04<3:33:02,  2.91s/it]
 63%|██████▎   | 7602/12000 [4:30:06<3:02:13,  2.49s/it]
                                                        

 63%|██████▎   | 7602/12000 [4:30:06<3:02:13,  2.49s/it]
 63%|██████▎   | 7603/12000 [4:30:08<2:44:05,  2.24s/it]
                                                        

 63%|██████▎   | 7603/12000 [4:30:08<2:44:05,  2.24s/it]
 63%|██████▎   | 7604/12000 [4:30:09<2:37:40,  2.15s/it]
                                                        

 63%|██████▎   | 7604/12000 [4:30:09<2:37:40,  2.15s/it]
 63%|██████▎   | 7605/12000 [4:30:12<2:39:36,  2.18s/it]
                                                        

 63%|██████▎   | 7605/12000 [4:30:12<2:39:36,  2.18s/it]
 63%|██████▎   | 7606/12000 [4:30:13<2:26:47,  2.00s/it]
                                                        

 63%|██████▎   | 7606/12000 [4:30:13<2:26:47,  2.00s/it]
 63%|██████▎   | 7607/12000 [4:30:15<2:25:18,  1.98s/it]
                                                        

 63%|██████▎   | 7607/12000 [4:30:15<2:25:18,  1.98s/it]
 63%|██████▎   | 7608/12000 [4:30:20<3:24:41,  2.80s/it]
                                                        

 63%|██████▎   | 7608/12000 [4:30:20<3:24:41,  2.80s/it]
 63%|██████▎   | 7609/12000 [4:30:22<3:01:47,  2.48s/it]
                                                        

 63%|██████▎   | 7609/12000 [4:30:22<3:01:47,  2.48s/it]
 63%|██████▎   | 7610/12000 [4:30:24<2:47:34,  2.29s/it]
                                                        

 63%|██████▎   | 7610/12000 [4:30:24<2:47:34,  2.29s/it]
 63%|██████▎   | 7611/12000 [4:30:26<2:56:32,  2.41s/it]
                                                        

 63%|██████▎   | 7611/12000 [4:30:26<2:56:32,  2.41s/it]
 63%|██████▎   | 7612/12000 [4:30:30<3:18:03,  2.71s/it]
                                                        

 63%|██████▎   | 7612/12000 [4:30:30<3:18:03,  2.71s/it]
 63%|██████▎   | 7613/12000 [4:30:32<3:16:48,  2.69s/it]
                                                        

 63%|██████▎   | 7613/12000 [4:30:32<3:16:48,  2.69s/it]
 63%|██████▎   | 7614/12000 [4:30:35<3:12:23,  2.63s/it]
                                                        

 63%|██████▎   | 7614/12000 [4:30:35<3:12:23,  2.63s/it]
 63%|██████▎   | 7615/12000 [4:30:37<3:14:15,  2.66s/it]
                                                        

 63%|██████▎   | 7615/12000 [4:30:37<3:14:15,  2.66s/it]
 63%|██████▎   | 7616/12000 [4:30:39<2:49:43,  2.32s/it]
                                                        

 63%|██████▎   | 7616/12000 [4:30:39<2:49:43,  2.32s/it]
 63%|██████▎   | 7617/12000 [4:30:40<2:30:51,  2.07s/it]
                                                        

 63%|██████▎   | 7617/12000 [4:30:40<2:30:51,  2.07s/it]
 63%|██████▎   | 7618/12000 [4:30:42<2:23:04,  1.96s/it]
                                                        
{'loss': '0.961', 'grad_norm': '0.5631', 'learning_rate': '6.571e-05', 'entropy': '0.8355', 'num_tokens': '4.089e+07', 'mean_token_accuracy': '0.7839', 'epoch': '0.067'}
{'loss': '0.8431', 'grad_norm': '0.4363', 'learning_rate': '6.569e-05', 'entropy': '0.8293', 'num_tokens': '4.089e+07', 'mean_token_accuracy': '0.7825', 'epoch': '0.06701'}
{'loss': '1.251', 'grad_norm': '0.4056', 'learning_rate': '6.566e-05', 'entropy': '1.097', 'num_tokens': '4.09e+07', 'mean_token_accuracy': '0.7275', 'epoch': '0.06702'}
{'loss': '0.894', 'grad_norm': '0.3694', 'learning_rate': '6.563e-05', 'entropy': '0.8483', 'num_tokens': '4.09e+07', 'mean_token_accuracy': '0.8002', 'epoch': '0.06703'}
{'loss': '0.5762', 'grad_norm': '0.3258', 'learning_rate': '6.561e-05', 'entropy': '0.5839', 'num_tokens': '4.091e+07', 'mean_token_accuracy': '0.8527', 'epoch': '0.06704'}
{'loss': '0.8524', 'grad_norm': '0.3947', 'learning_rate': '6.558e-05', 'entropy': '0.9308', 'num_tokens': '4.092e+07', 'mean_token_accuracy': '0.7804', 'epoch': '0.06705'}
{'loss': '0.6401', 'grad_norm': '0.2905', 'learning_rate': '6.556e-05', 'entropy': '0.755', 'num_tokens': '4.092e+07', 'mean_token_accuracy': '0.8093', 'epoch': '0.06706'}
{'loss': '1.227', 'grad_norm': '0.4275', 'learning_rate': '6.553e-05', 'entropy': '1.244', 'num_tokens': '4.093e+07', 'mean_token_accuracy': '0.699', 'epoch': '0.06706'}
{'loss': '0.8351', 'grad_norm': '0.3615', 'learning_rate': '6.55e-05', 'entropy': '0.8297', 'num_tokens': '4.094e+07', 'mean_token_accuracy': '0.7906', 'epoch': '0.06707'}
{'loss': '1.262', 'grad_norm': '0.4424', 'learning_rate': '6.548e-05', 'entropy': '1.176', 'num_tokens': '4.094e+07', 'mean_token_accuracy': '0.719', 'epoch': '0.06708'}
{'loss': '0.9201', 'grad_norm': '0.3962', 'learning_rate': '6.545e-05', 'entropy': '0.8844', 'num_tokens': '4.095e+07', 'mean_token_accuracy': '0.7746', 'epoch': '0.06709'}
{'loss': '0.8812', 'grad_norm': '0.353', 'learning_rate': '6.543e-05', 'entropy': '0.8833', 'num_tokens': '4.095e+07', 'mean_token_accuracy': '0.7748', 'epoch': '0.0671'}
{'loss': '0.8761', 'grad_norm': '0.3981', 'learning_rate': '6.54e-05', 'entropy': '0.8418', 'num_tokens': '4.096e+07', 'mean_token_accuracy': '0.795', 'epoch': '0.06711'}
{'loss': '0.8921', 'grad_norm': '0.3986', 'learning_rate': '6.538e-05', 'entropy': '0.8706', 'num_tokens': '4.096e+07', 'mean_token_accuracy': '0.7798', 'epoch': '0.06712'}
{'loss': '0.7847', 'grad_norm': '0.3287', 'learning_rate': '6.535e-05', 'entropy': '0.735', 'num_tokens': '4.097e+07', 'mean_token_accuracy': '0.815', 'epoch': '0.06713'}
{'loss': '0.6735', 'grad_norm': '0.3415', 'learning_rate': '6.532e-05', 'entropy': '0.7068', 'num_tokens': '4.097e+07', 'mean_token_accuracy': '0.8272', 'epoch': '0.06713'}
{'loss': '0.961', 'grad_norm': '0.4228', 'learning_rate': '6.53e-05', 'entropy': '0.9438', 'num_tokens': '4.098e+07', 'mean_token_accuracy': '0.7716', 'epoch': '0.06714'}
{'loss': '0.7607', 'grad_norm': '0.354', 'learning_rate': '6.527e-05', 'entropy': '0.7342', 'num_tokens': '4.099e+07', 'mean_token_accuracy': '0.8148', 'epoch': '0.06715'}
{'loss': '0.8077', 'grad_norm': '0.3308', 'learning_rate': '6.525e-05', 'entropy': '0.8173', 'num_tokens': '4.099e+07', 'mean_token_accuracy': '0.7886', 'epoch': '0.06716'}
{'loss': '0.8739', 'grad_norm': '0.3896', 'learning_rate': '6.522e-05', 'entropy': '0.8344', 'num_tokens': '4.1e+07', 'mean_token_accuracy': '0.8064', 'epoch': '0.06717'}
{'loss': '0.9984', 'grad_norm': '0.42', 'learning_rate': '6.519e-05', 'entropy': '1.041', 'num_tokens': '4.1e+07', 'mean_token_accuracy': '0.7636', 'epoch': '0.06718'}
{'loss': '0.9893', 'grad_norm': '0.5759', 'learning_rate': '6.517e-05', 'entropy': '0.9764', 'num_tokens': '4.101e+07', 'mean_token_accuracy': '0.7559', 'epoch': '0.06719'}
{'loss': '0.4452', 'grad_norm': '0.2765', 'learning_rate': '6.514e-05', 'entropy': '0.4871', 'num_tokens': '4.101e+07', 'mean_token_accuracy': '0.8722', 'epoch': '0.0672'}
{'loss': '0.6695', 'grad_norm': '0.3655', 'learning_rate': '6.512e-05', 'entropy': '0.6966', 'num_tokens': '4.102e+07', 'mean_token_accuracy': '0.8278', 'epoch': '0.06721'}
{'loss': '0.6257', 'grad_norm': '0.3994', 'learning_rate': '6.509e-05', 'entropy': '0.6747', 'num_tokens': '4.102e+07', 'mean_token_accuracy': '0.8262', 'epoch': '0.06721'}
{'loss': '1.618', 'grad_norm': '0.5207', 'learning_rate': '6.507e-05', 'entropy': '1.523', 'num_tokens': '4.103e+07', 'mean_token_accuracy': '0.6376', 'epoch': '0.06722'}
{'loss': '1.242', 'grad_norm': '0.4483', 'learning_rate': '6.504e-05', 'entropy': '1.207', 'num_tokens': '4.103e+07', 'mean_token_accuracy': '0.7023', 'epoch': '0.06723'}
{'loss': '0.8526', 'grad_norm': '0.3272', 'learning_rate': '6.501e-05', 'entropy': '0.8856', 'num_tokens': '4.104e+07', 'mean_token_accuracy': '0.7698', 'epoch': '0.06724'}
{'loss': '0.933', 'grad_norm': '0.3127', 'learning_rate': '6.499e-05', 'entropy': '0.7976', 'num_tokens': '4.105e+07', 'mean_token_accuracy': '0.8091', 'epoch': '0.06725'}
{'loss': '0.9941', 'grad_norm': '0.3843', 'learning_rate': '6.496e-05', 'entropy': '1.01', 'num_tokens': '4.105e+07', 'mean_token_accuracy': '0.7652', 'epoch': '0.06726'}
{'loss': '0.7749', 'grad_norm': '0.4086', 'learning_rate': '6.494e-05', 'entropy': '0.8109', 'num_tokens': '4.106e+07', 'mean_token_accuracy': '0.7935', 'epoch': '0.06727'}
{'loss': '0.7936', 'grad_norm': '0.42', 'learning_rate': '6.491e-05', 'entropy': '0.8629', 'num_tokens': '4.106e+07', 'mean_token_accuracy': '0.8059', 'epoch': '0.06728'}
{'loss': '1.186', 'grad_norm': '0.4989', 'learning_rate': '6.488e-05', 'entropy': '1.131', 'num_tokens': '4.107e+07', 'mean_token_accuracy': '0.7406', 'epoch': '0.06729'}
{'loss': '0.766', 'grad_norm': '0.3998', 'learning_rate': '6.486e-05', 'entropy': '0.7796', 'num_tokens': '4.107e+07', 'mean_token_accuracy': '0.8153', 'epoch': '0.06729'}
{'loss': '1.057', 'grad_norm': '0.3818', 'learning_rate': '6.483e-05', 'entropy': '0.9338', 'num_tokens': '4.108e+07', 'mean_token_accuracy': '0.7578', 'epoch': '0.0673'}
{'loss': '0.7626', 'grad_norm': '0.3685', 'learning_rate': '6.481e-05', 'entropy': '0.7727', 'num_tokens': '4.108e+07', 'mean_token_accuracy': '0.796', 'epoch': '0.06731'}
{'loss': '0.9945', 'grad_norm': '0.3796', 'learning_rate': '6.478e-05', 'entropy': '0.9952', 'num_tokens': '4.109e+07', 'mean_token_accuracy': '0.7574', 'epoch': '0.06732'}
{'loss': '0.9659', 'grad_norm': '0.3587', 'learning_rate': '6.476e-05', 'entropy': '0.6989', 'num_tokens': '4.109e+07', 'mean_token_accuracy': '0.8254', 'epoch': '0.06733'}
{'loss': '1.084', 'grad_norm': '0.4362', 'learning_rate': '6.473e-05', 'entropy': '0.9536', 'num_tokens': '4.11e+07', 'mean_token_accuracy': '0.7776', 'epoch': '0.06734'}
{'loss': '0.7373', 'grad_norm': '0.4811', 'learning_rate': '6.47e-05', 'entropy': '0.6954', 'num_tokens': '4.111e+07', 'mean_token_accuracy': '0.8051', 'epoch': '0.06735'}
{'loss': '0.8059', 'grad_norm': '0.3595', 'learning_rate': '6.468e-05', 'entropy': '0.895', 'num_tokens': '4.111e+07', 'mean_token_accuracy': '0.7807', 'epoch': '0.06736'}
{'loss': '1.053', 'grad_norm': '0.4743', 'learning_rate': '6.465e-05', 'entropy': '1.078', 'num_tokens': '4.112e+07', 'mean_token_accuracy': '0.7546', 'epoch': '0.06736'}
{'loss': '0.6826', 'grad_norm': '0.4442', 'learning_rate': '6.463e-05', 'entropy': '0.8198', 'num_tokens': '4.113e+07', 'mean_token_accuracy': '0.7797', 'epoch': '0.06737'}
{'loss': '0.7608', 'grad_norm': '0.3207', 'learning_rate': '6.46e-05', 'entropy': '0.7888', 'num_tokens': '4.113e+07', 'mean_token_accuracy': '0.8003', 'epoch': '0.06738'}
{'loss': '0.8706', 'grad_norm': '0.3644', 'learning_rate': '6.458e-05', 'entropy': '0.8717', 'num_tokens': '4.114e+07', 'mean_token_accuracy': '0.779', 'epoch': '0.06739'}
{'loss': '0.728', 'grad_norm': '0.3891', 'learning_rate': '6.455e-05', 'entropy': '0.7274', 'num_tokens': '4.114e+07', 'mean_token_accuracy': '0.8112', 'epoch': '0.0674'}
{'loss': '0.815', 'grad_norm': '0.3778', 'learning_rate': '6.452e-05', 'entropy': '0.8122', 'num_tokens': '4.115e+07', 'mean_token_accuracy': '0.8102', 'epoch': '0.06741'}

 63%|██████▎   | 7618/12000 [4:30:42<2:23:04,  1.96s/it]
 63%|██████▎   | 7619/12000 [4:30:44<2:15:23,  1.85s/it]
                                                        

 63%|██████▎   | 7619/12000 [4:30:44<2:15:23,  1.85s/it]
 64%|██████▎   | 7620/12000 [4:30:45<2:07:49,  1.75s/it]
                                                        

 64%|██████▎   | 7620/12000 [4:30:45<2:07:49,  1.75s/it]
 64%|██████▎   | 7621/12000 [4:30:47<2:02:30,  1.68s/it]
                                                        

 64%|██████▎   | 7621/12000 [4:30:47<2:02:30,  1.68s/it]
 64%|██████▎   | 7622/12000 [4:30:49<2:02:19,  1.68s/it]
                                                        

 64%|██████▎   | 7622/12000 [4:30:49<2:02:19,  1.68s/it]
 64%|██████▎   | 7623/12000 [4:30:52<2:44:55,  2.26s/it]
                                                        

 64%|██████▎   | 7623/12000 [4:30:52<2:44:55,  2.26s/it]
 64%|██████▎   | 7624/12000 [4:30:55<2:54:37,  2.39s/it]
                                                        

 64%|██████▎   | 7624/12000 [4:30:55<2:54:37,  2.39s/it]
 64%|██████▎   | 7625/12000 [4:30:56<2:33:11,  2.10s/it]
                                                        

 64%|██████▎   | 7625/12000 [4:30:56<2:33:11,  2.10s/it]
 64%|██████▎   | 7626/12000 [4:30:58<2:19:39,  1.92s/it]
                                                        

 64%|██████▎   | 7626/12000 [4:30:58<2:19:39,  1.92s/it]
 64%|██████▎   | 7627/12000 [4:30:59<2:12:06,  1.81s/it]
                                                        

 64%|██████▎   | 7627/12000 [4:30:59<2:12:06,  1.81s/it]
 64%|██████▎   | 7628/12000 [4:31:01<2:08:54,  1.77s/it]
                                                        

 64%|██████▎   | 7628/12000 [4:31:01<2:08:54,  1.77s/it]
 64%|██████▎   | 7629/12000 [4:31:03<2:03:38,  1.70s/it]
                                                        

 64%|██████▎   | 7629/12000 [4:31:03<2:03:38,  1.70s/it]
 64%|██████▎   | 7630/12000 [4:31:04<2:00:49,  1.66s/it]
                                                        

 64%|██████▎   | 7630/12000 [4:31:04<2:00:49,  1.66s/it]
 64%|██████▎   | 7631/12000 [4:31:06<1:57:00,  1.61s/it]
                                                        

 64%|██████▎   | 7631/12000 [4:31:06<1:57:00,  1.61s/it]
 64%|██████▎   | 7632/12000 [4:31:08<2:09:51,  1.78s/it]
                                                        

 64%|██████▎   | 7632/12000 [4:31:08<2:09:51,  1.78s/it]
 64%|██████▎   | 7633/12000 [4:31:10<2:22:34,  1.96s/it]
                                                        

 64%|██████▎   | 7633/12000 [4:31:10<2:22:34,  1.96s/it]
 64%|██████▎   | 7634/12000 [4:31:12<2:17:45,  1.89s/it]
                                                        

 64%|██████▎   | 7634/12000 [4:31:12<2:17:45,  1.89s/it]
 64%|██████▎   | 7635/12000 [4:31:13<2:08:36,  1.77s/it]
                                                        

 64%|██████▎   | 7635/12000 [4:31:13<2:08:36,  1.77s/it]
 64%|██████▎   | 7636/12000 [4:31:15<2:06:56,  1.75s/it]
                                                        

 64%|██████▎   | 7636/12000 [4:31:15<2:06:56,  1.75s/it]
 64%|██████▎   | 7637/12000 [4:31:17<2:09:14,  1.78s/it]
                                                        

 64%|██████▎   | 7637/12000 [4:31:17<2:09:14,  1.78s/it]
 64%|██████▎   | 7638/12000 [4:31:18<2:02:12,  1.68s/it]
                                                        

 64%|██████▎   | 7638/12000 [4:31:18<2:02:12,  1.68s/it]
 64%|██████▎   | 7639/12000 [4:31:21<2:18:21,  1.90s/it]
                                                        

 64%|██████▎   | 7639/12000 [4:31:21<2:18:21,  1.90s/it]
 64%|██████▎   | 7640/12000 [4:31:23<2:15:54,  1.87s/it]
                                                        

 64%|██████▎   | 7640/12000 [4:31:23<2:15:54,  1.87s/it]
 64%|██████▎   | 7641/12000 [4:31:24<2:11:12,  1.81s/it]
                                                        

 64%|██████▎   | 7641/12000 [4:31:24<2:11:12,  1.81s/it]
 64%|██████▎   | 7642/12000 [4:31:26<2:20:57,  1.94s/it]
                                                        

 64%|██████▎   | 7642/12000 [4:31:26<2:20:57,  1.94s/it]
 64%|██████▎   | 7643/12000 [4:31:28<2:09:45,  1.79s/it]
                                                        

 64%|██████▎   | 7643/12000 [4:31:28<2:09:45,  1.79s/it]
 64%|██████▎   | 7644/12000 [4:31:30<2:07:22,  1.75s/it]
                                                        

 64%|██████▎   | 7644/12000 [4:31:30<2:07:22,  1.75s/it]
 64%|██████▎   | 7645/12000 [4:31:34<3:10:03,  2.62s/it]
                                                        

 64%|██████▎   | 7645/12000 [4:31:34<3:10:03,  2.62s/it]
 64%|██████▎   | 7646/12000 [4:31:36<2:51:39,  2.37s/it]
                                                        

 64%|██████▎   | 7646/12000 [4:31:36<2:51:39,  2.37s/it]
 64%|██████▎   | 7647/12000 [4:31:38<2:34:43,  2.13s/it]
                                                        

 64%|██████▎   | 7647/12000 [4:31:38<2:34:43,  2.13s/it]
 64%|██████▎   | 7648/12000 [4:31:39<2:27:54,  2.04s/it]
                                                        

 64%|██████▎   | 7648/12000 [4:31:39<2:27:54,  2.04s/it]
 64%|██████▎   | 7649/12000 [4:31:41<2:25:10,  2.00s/it]
                                                        

 64%|██████▎   | 7649/12000 [4:31:41<2:25:10,  2.00s/it]
 64%|██████▍   | 7650/12000 [4:31:43<2:18:53,  1.92s/it]
                                                        

 64%|██████▍   | 7650/12000 [4:31:43<2:18:53,  1.92s/it]
 64%|██████▍   | 7651/12000 [4:31:46<2:48:52,  2.33s/it]
                                                        

 64%|██████▍   | 7651/12000 [4:31:46<2:48:52,  2.33s/it]
 64%|██████▍   | 7652/12000 [4:31:48<2:43:39,  2.26s/it]
                                                        

 64%|██████▍   | 7652/12000 [4:31:48<2:43:39,  2.26s/it]
 64%|██████▍   | 7653/12000 [4:31:50<2:29:05,  2.06s/it]
                                                        

 64%|██████▍   | 7653/12000 [4:31:50<2:29:05,  2.06s/it]
 64%|██████▍   | 7654/12000 [4:31:54<3:02:24,  2.52s/it]
                                                        

 64%|██████▍   | 7654/12000 [4:31:54<3:02:24,  2.52s/it]
 64%|██████▍   | 7655/12000 [4:31:56<3:06:06,  2.57s/it]
                                                        

 64%|██████▍   | 7655/12000 [4:31:56<3:06:06,  2.57s/it]
 64%|██████▍   | 7656/12000 [4:31:59<3:03:21,  2.53s/it]
                                                        

 64%|██████▍   | 7656/12000 [4:31:59<3:03:21,  2.53s/it]
 64%|██████▍   | 7657/12000 [4:32:01<2:49:25,  2.34s/it]
                                                        

 64%|██████▍   | 7657/12000 [4:32:01<2:49:25,  2.34s/it]
 64%|██████▍   | 7658/12000 [4:32:03<2:44:01,  2.27s/it]
                                                        

 64%|██████▍   | 7658/12000 [4:32:03<2:44:01,  2.27s/it]
 64%|██████▍   | 7659/12000 [4:32:04<2:32:24,  2.11s/it]
                                                        

 64%|██████▍   | 7659/12000 [4:32:04<2:32:24,  2.11s/it]
 64%|██████▍   | 7660/12000 [4:32:06<2:26:13,  2.02s/it]
                                                        

 64%|██████▍   | 7660/12000 [4:32:06<2:26:13,  2.02s/it]
 64%|██████▍   | 7661/12000 [4:32:08<2:26:03,  2.02s/it]
                                                        

 64%|██████▍   | 7661/12000 [4:32:08<2:26:03,  2.02s/it]
 64%|██████▍   | 7662/12000 [4:32:11<2:39:41,  2.21s/it]
                                                        

 64%|██████▍   | 7662/12000 [4:32:11<2:39:41,  2.21s/it]
 64%|██████▍   | 7663/12000 [4:32:12<2:23:30,  1.99s/it]
                                                        

 64%|██████▍   | 7663/12000 [4:32:12<2:23:30,  1.99s/it]
 64%|██████▍   | 7664/12000 [4:32:14<2:14:57,  1.87s/it]
                                                        

 64%|██████▍   | 7664/12000 [4:32:14<2:14:57,  1.87s/it]
 64%|██████▍   | 7665/12000 [4:32:16<2:21:07,  1.95s/it]
                                                        
{'loss': '1.046', 'grad_norm': '0.5192', 'learning_rate': '6.45e-05', 'entropy': '1.106', 'num_tokens': '4.115e+07', 'mean_token_accuracy': '0.7206', 'epoch': '0.06742'}
{'loss': '0.6502', 'grad_norm': '0.37', 'learning_rate': '6.447e-05', 'entropy': '0.621', 'num_tokens': '4.116e+07', 'mean_token_accuracy': '0.8298', 'epoch': '0.06743'}
{'loss': '0.8576', 'grad_norm': '0.409', 'learning_rate': '6.445e-05', 'entropy': '0.8939', 'num_tokens': '4.116e+07', 'mean_token_accuracy': '0.7872', 'epoch': '0.06744'}
{'loss': '0.7573', 'grad_norm': '0.503', 'learning_rate': '6.442e-05', 'entropy': '0.7572', 'num_tokens': '4.117e+07', 'mean_token_accuracy': '0.7954', 'epoch': '0.06744'}
{'loss': '0.5907', 'grad_norm': '0.3091', 'learning_rate': '6.439e-05', 'entropy': '0.6312', 'num_tokens': '4.117e+07', 'mean_token_accuracy': '0.834', 'epoch': '0.06745'}
{'loss': '0.6813', 'grad_norm': '0.399', 'learning_rate': '6.437e-05', 'entropy': '0.698', 'num_tokens': '4.118e+07', 'mean_token_accuracy': '0.8145', 'epoch': '0.06746'}
{'loss': '0.5928', 'grad_norm': '0.3162', 'learning_rate': '6.434e-05', 'entropy': '0.738', 'num_tokens': '4.118e+07', 'mean_token_accuracy': '0.8146', 'epoch': '0.06747'}
{'loss': '0.9018', 'grad_norm': '0.3724', 'learning_rate': '6.432e-05', 'entropy': '0.8684', 'num_tokens': '4.119e+07', 'mean_token_accuracy': '0.7863', 'epoch': '0.06748'}
{'loss': '0.9802', 'grad_norm': '0.435', 'learning_rate': '6.429e-05', 'entropy': '1.072', 'num_tokens': '4.119e+07', 'mean_token_accuracy': '0.744', 'epoch': '0.06749'}
{'loss': '0.8165', 'grad_norm': '0.4199', 'learning_rate': '6.427e-05', 'entropy': '0.7904', 'num_tokens': '4.12e+07', 'mean_token_accuracy': '0.8012', 'epoch': '0.0675'}
{'loss': '1.06', 'grad_norm': '0.4441', 'learning_rate': '6.424e-05', 'entropy': '1.067', 'num_tokens': '4.12e+07', 'mean_token_accuracy': '0.7408', 'epoch': '0.06751'}
{'loss': '0.8679', 'grad_norm': '0.3891', 'learning_rate': '6.421e-05', 'entropy': '0.9849', 'num_tokens': '4.12e+07', 'mean_token_accuracy': '0.7694', 'epoch': '0.06752'}
{'loss': '0.8439', 'grad_norm': '0.3271', 'learning_rate': '6.419e-05', 'entropy': '0.8156', 'num_tokens': '4.121e+07', 'mean_token_accuracy': '0.7958', 'epoch': '0.06752'}
{'loss': '0.7939', 'grad_norm': '0.3823', 'learning_rate': '6.416e-05', 'entropy': '0.918', 'num_tokens': '4.121e+07', 'mean_token_accuracy': '0.7693', 'epoch': '0.06753'}
{'loss': '1.062', 'grad_norm': '0.5279', 'learning_rate': '6.414e-05', 'entropy': '0.9746', 'num_tokens': '4.122e+07', 'mean_token_accuracy': '0.7701', 'epoch': '0.06754'}
{'loss': '0.4397', 'grad_norm': '0.3031', 'learning_rate': '6.411e-05', 'entropy': '0.4639', 'num_tokens': '4.123e+07', 'mean_token_accuracy': '0.877', 'epoch': '0.06755'}
{'loss': '0.5717', 'grad_norm': '0.3947', 'learning_rate': '6.409e-05', 'entropy': '0.6121', 'num_tokens': '4.123e+07', 'mean_token_accuracy': '0.8415', 'epoch': '0.06756'}
{'loss': '0.6676', 'grad_norm': '0.292', 'learning_rate': '6.406e-05', 'entropy': '0.6508', 'num_tokens': '4.124e+07', 'mean_token_accuracy': '0.8222', 'epoch': '0.06757'}
{'loss': '0.7244', 'grad_norm': '0.3658', 'learning_rate': '6.403e-05', 'entropy': '0.6956', 'num_tokens': '4.124e+07', 'mean_token_accuracy': '0.8106', 'epoch': '0.06758'}
{'loss': '0.994', 'grad_norm': '0.4302', 'learning_rate': '6.401e-05', 'entropy': '1.033', 'num_tokens': '4.125e+07', 'mean_token_accuracy': '0.7537', 'epoch': '0.06759'}
{'loss': '0.8093', 'grad_norm': '0.4157', 'learning_rate': '6.398e-05', 'entropy': '0.8261', 'num_tokens': '4.125e+07', 'mean_token_accuracy': '0.7865', 'epoch': '0.0676'}
{'loss': '0.8143', 'grad_norm': '0.384', 'learning_rate': '6.396e-05', 'entropy': '0.8699', 'num_tokens': '4.126e+07', 'mean_token_accuracy': '0.7778', 'epoch': '0.0676'}
{'loss': '0.9368', 'grad_norm': '0.4064', 'learning_rate': '6.393e-05', 'entropy': '0.908', 'num_tokens': '4.126e+07', 'mean_token_accuracy': '0.7744', 'epoch': '0.06761'}
{'loss': '0.4969', 'grad_norm': '0.2919', 'learning_rate': '6.391e-05', 'entropy': '0.5046', 'num_tokens': '4.127e+07', 'mean_token_accuracy': '0.8663', 'epoch': '0.06762'}
{'loss': '0.9135', 'grad_norm': '0.4053', 'learning_rate': '6.388e-05', 'entropy': '0.8476', 'num_tokens': '4.127e+07', 'mean_token_accuracy': '0.7807', 'epoch': '0.06763'}
{'loss': '0.7608', 'grad_norm': '0.4928', 'learning_rate': '6.385e-05', 'entropy': '0.7779', 'num_tokens': '4.127e+07', 'mean_token_accuracy': '0.8032', 'epoch': '0.06764'}
{'loss': '0.6931', 'grad_norm': '0.4031', 'learning_rate': '6.383e-05', 'entropy': '0.7065', 'num_tokens': '4.128e+07', 'mean_token_accuracy': '0.8166', 'epoch': '0.06765'}
{'loss': '0.8237', 'grad_norm': '0.3596', 'learning_rate': '6.38e-05', 'entropy': '0.8345', 'num_tokens': '4.129e+07', 'mean_token_accuracy': '0.7971', 'epoch': '0.06766'}
{'loss': '0.7276', 'grad_norm': '0.412', 'learning_rate': '6.378e-05', 'entropy': '0.8202', 'num_tokens': '4.129e+07', 'mean_token_accuracy': '0.7914', 'epoch': '0.06767'}
{'loss': '0.9891', 'grad_norm': '0.4488', 'learning_rate': '6.375e-05', 'entropy': '0.9306', 'num_tokens': '4.129e+07', 'mean_token_accuracy': '0.77', 'epoch': '0.06767'}
{'loss': '0.5989', 'grad_norm': '0.3656', 'learning_rate': '6.373e-05', 'entropy': '0.6331', 'num_tokens': '4.13e+07', 'mean_token_accuracy': '0.8284', 'epoch': '0.06768'}
{'loss': '0.591', 'grad_norm': '0.3081', 'learning_rate': '6.37e-05', 'entropy': '0.5822', 'num_tokens': '4.131e+07', 'mean_token_accuracy': '0.8442', 'epoch': '0.06769'}
{'loss': '0.7302', 'grad_norm': '0.5509', 'learning_rate': '6.367e-05', 'entropy': '0.8704', 'num_tokens': '4.131e+07', 'mean_token_accuracy': '0.7758', 'epoch': '0.0677'}
{'loss': '0.5978', 'grad_norm': '0.4042', 'learning_rate': '6.365e-05', 'entropy': '0.657', 'num_tokens': '4.132e+07', 'mean_token_accuracy': '0.819', 'epoch': '0.06771'}
{'loss': '1.065', 'grad_norm': '0.372', 'learning_rate': '6.362e-05', 'entropy': '1.062', 'num_tokens': '4.132e+07', 'mean_token_accuracy': '0.7449', 'epoch': '0.06772'}
{'loss': '0.7721', 'grad_norm': '0.4805', 'learning_rate': '6.36e-05', 'entropy': '0.8358', 'num_tokens': '4.133e+07', 'mean_token_accuracy': '0.7926', 'epoch': '0.06773'}
{'loss': '0.5824', 'grad_norm': '0.311', 'learning_rate': '6.357e-05', 'entropy': '0.6629', 'num_tokens': '4.133e+07', 'mean_token_accuracy': '0.8303', 'epoch': '0.06774'}
{'loss': '0.9039', 'grad_norm': '0.3748', 'learning_rate': '6.355e-05', 'entropy': '0.9129', 'num_tokens': '4.134e+07', 'mean_token_accuracy': '0.7832', 'epoch': '0.06775'}
{'loss': '1.265', 'grad_norm': '0.8062', 'learning_rate': '6.352e-05', 'entropy': '1.124', 'num_tokens': '4.135e+07', 'mean_token_accuracy': '0.7214', 'epoch': '0.06775'}
{'loss': '0.5817', 'grad_norm': '0.3991', 'learning_rate': '6.35e-05', 'entropy': '0.544', 'num_tokens': '4.135e+07', 'mean_token_accuracy': '0.8545', 'epoch': '0.06776'}
{'loss': '0.8646', 'grad_norm': '0.4987', 'learning_rate': '6.347e-05', 'entropy': '0.8336', 'num_tokens': '4.136e+07', 'mean_token_accuracy': '0.7878', 'epoch': '0.06777'}
{'loss': '0.6339', 'grad_norm': '0.3612', 'learning_rate': '6.344e-05', 'entropy': '0.6669', 'num_tokens': '4.136e+07', 'mean_token_accuracy': '0.8167', 'epoch': '0.06778'}
{'loss': '0.9132', 'grad_norm': '0.4377', 'learning_rate': '6.342e-05', 'entropy': '0.9124', 'num_tokens': '4.137e+07', 'mean_token_accuracy': '0.7787', 'epoch': '0.06779'}
{'loss': '0.8646', 'grad_norm': '0.4261', 'learning_rate': '6.339e-05', 'entropy': '0.8575', 'num_tokens': '4.137e+07', 'mean_token_accuracy': '0.8025', 'epoch': '0.0678'}
{'loss': '0.7542', 'grad_norm': '0.3353', 'learning_rate': '6.337e-05', 'entropy': '0.8717', 'num_tokens': '4.138e+07', 'mean_token_accuracy': '0.7904', 'epoch': '0.06781'}
{'loss': '0.7101', 'grad_norm': '0.4108', 'learning_rate': '6.334e-05', 'entropy': '0.7804', 'num_tokens': '4.138e+07', 'mean_token_accuracy': '0.8088', 'epoch': '0.06782'}
{'loss': '0.8222', 'grad_norm': '0.3911', 'learning_rate': '6.332e-05', 'entropy': '0.8694', 'num_tokens': '4.139e+07', 'mean_token_accuracy': '0.7841', 'epoch': '0.06783'}

 64%|██████▍   | 7665/12000 [4:32:16<2:21:07,  1.95s/it]
 64%|██████▍   | 7666/12000 [4:32:18<2:26:31,  2.03s/it]
                                                        

 64%|██████▍   | 7666/12000 [4:32:18<2:26:31,  2.03s/it]
 64%|██████▍   | 7667/12000 [4:32:22<3:03:11,  2.54s/it]
                                                        

 64%|██████▍   | 7667/12000 [4:32:22<3:03:11,  2.54s/it]
 64%|██████▍   | 7668/12000 [4:32:25<3:10:14,  2.63s/it]
                                                        

 64%|██████▍   | 7668/12000 [4:32:25<3:10:14,  2.63s/it]
 64%|██████▍   | 7669/12000 [4:32:28<3:14:19,  2.69s/it]
                                                        

 64%|██████▍   | 7669/12000 [4:32:28<3:14:19,  2.69s/it]
 64%|██████▍   | 7670/12000 [4:32:29<2:52:49,  2.39s/it]
                                                        

 64%|██████▍   | 7670/12000 [4:32:29<2:52:49,  2.39s/it]
 64%|██████▍   | 7671/12000 [4:32:31<2:42:48,  2.26s/it]
                                                        

 64%|██████▍   | 7671/12000 [4:32:31<2:42:48,  2.26s/it]
 64%|██████▍   | 7672/12000 [4:32:33<2:33:03,  2.12s/it]
                                                        

 64%|██████▍   | 7672/12000 [4:32:33<2:33:03,  2.12s/it]
 64%|██████▍   | 7673/12000 [4:32:35<2:18:24,  1.92s/it]
                                                        

 64%|██████▍   | 7673/12000 [4:32:35<2:18:24,  1.92s/it]
 64%|██████▍   | 7674/12000 [4:32:36<2:15:01,  1.87s/it]
                                                        

 64%|██████▍   | 7674/12000 [4:32:36<2:15:01,  1.87s/it]
 64%|██████▍   | 7675/12000 [4:32:38<2:12:42,  1.84s/it]
                                                        

 64%|██████▍   | 7675/12000 [4:32:38<2:12:42,  1.84s/it]
 64%|██████▍   | 7676/12000 [4:32:40<2:04:18,  1.72s/it]
                                                        

 64%|██████▍   | 7676/12000 [4:32:40<2:04:18,  1.72s/it]
 64%|██████▍   | 7677/12000 [4:32:41<2:00:32,  1.67s/it]
                                                        

 64%|██████▍   | 7677/12000 [4:32:41<2:00:32,  1.67s/it]
 64%|██████▍   | 7678/12000 [4:32:43<1:56:35,  1.62s/it]
                                                        

 64%|██████▍   | 7678/12000 [4:32:43<1:56:35,  1.62s/it]
 64%|██████▍   | 7679/12000 [4:32:45<2:11:16,  1.82s/it]
                                                        

 64%|██████▍   | 7679/12000 [4:32:45<2:11:16,  1.82s/it]
 64%|██████▍   | 7680/12000 [4:32:47<2:09:05,  1.79s/it]
                                                        

 64%|██████▍   | 7680/12000 [4:32:47<2:09:05,  1.79s/it]
 64%|██████▍   | 7681/12000 [4:32:48<2:05:43,  1.75s/it]
                                                        

 64%|██████▍   | 7681/12000 [4:32:48<2:05:43,  1.75s/it]
 64%|██████▍   | 7682/12000 [4:32:50<2:04:51,  1.74s/it]
                                                        

 64%|██████▍   | 7682/12000 [4:32:50<2:04:51,  1.74s/it]
 64%|██████▍   | 7683/12000 [4:32:53<2:29:18,  2.08s/it]
                                                        

 64%|██████▍   | 7683/12000 [4:32:53<2:29:18,  2.08s/it]
 64%|██████▍   | 7684/12000 [4:32:55<2:29:35,  2.08s/it]
                                                        

 64%|██████▍   | 7684/12000 [4:32:55<2:29:35,  2.08s/it]
 64%|██████▍   | 7685/12000 [4:32:57<2:25:50,  2.03s/it]
                                                        

 64%|██████▍   | 7685/12000 [4:32:57<2:25:50,  2.03s/it]
 64%|██████▍   | 7686/12000 [4:32:59<2:17:55,  1.92s/it]
                                                        

 64%|██████▍   | 7686/12000 [4:32:59<2:17:55,  1.92s/it]
 64%|██████▍   | 7687/12000 [4:33:00<2:16:38,  1.90s/it]
                                                        

 64%|██████▍   | 7687/12000 [4:33:00<2:16:38,  1.90s/it]
 64%|██████▍   | 7688/12000 [4:33:03<2:23:54,  2.00s/it]
                                                        

 64%|██████▍   | 7688/12000 [4:33:03<2:23:54,  2.00s/it]
 64%|██████▍   | 7689/12000 [4:33:07<3:17:10,  2.74s/it]
                                                        

 64%|██████▍   | 7689/12000 [4:33:07<3:17:10,  2.74s/it]
 64%|██████▍   | 7690/12000 [4:33:09<2:50:51,  2.38s/it]
                                                        

 64%|██████▍   | 7690/12000 [4:33:09<2:50:51,  2.38s/it]
 64%|██████▍   | 7691/12000 [4:33:10<2:35:17,  2.16s/it]
                                                        

 64%|██████▍   | 7691/12000 [4:33:10<2:35:17,  2.16s/it]
 64%|██████▍   | 7692/12000 [4:33:12<2:22:43,  1.99s/it]
                                                        

 64%|██████▍   | 7692/12000 [4:33:12<2:22:43,  1.99s/it]
 64%|██████▍   | 7693/12000 [4:33:14<2:25:15,  2.02s/it]
                                                        

 64%|██████▍   | 7693/12000 [4:33:14<2:25:15,  2.02s/it]
 64%|██████▍   | 7694/12000 [4:33:16<2:16:41,  1.90s/it]
                                                        

 64%|██████▍   | 7694/12000 [4:33:16<2:16:41,  1.90s/it]
 64%|██████▍   | 7695/12000 [4:33:17<2:09:45,  1.81s/it]
                                                        

 64%|██████▍   | 7695/12000 [4:33:17<2:09:45,  1.81s/it]
 64%|██████▍   | 7696/12000 [4:33:19<2:04:16,  1.73s/it]
                                                        

 64%|██████▍   | 7696/12000 [4:33:19<2:04:16,  1.73s/it]
 64%|██████▍   | 7697/12000 [4:33:21<2:07:37,  1.78s/it]
                                                        

 64%|██████▍   | 7697/12000 [4:33:21<2:07:37,  1.78s/it]
 64%|██████▍   | 7698/12000 [4:33:22<2:04:10,  1.73s/it]
                                                        

 64%|██████▍   | 7698/12000 [4:33:22<2:04:10,  1.73s/it]
 64%|██████▍   | 7699/12000 [4:33:25<2:14:27,  1.88s/it]
                                                        

 64%|██████▍   | 7699/12000 [4:33:25<2:14:27,  1.88s/it]
 64%|██████▍   | 7700/12000 [4:33:26<2:12:41,  1.85s/it]
                                                        

 64%|██████▍   | 7700/12000 [4:33:26<2:12:41,  1.85s/it]
 64%|██████▍   | 7701/12000 [4:33:31<3:13:00,  2.69s/it]
                                                        

 64%|██████▍   | 7701/12000 [4:33:31<3:13:00,  2.69s/it]
 64%|██████▍   | 7702/12000 [4:33:34<3:26:46,  2.89s/it]
                                                        

 64%|██████▍   | 7702/12000 [4:33:34<3:26:46,  2.89s/it]
 64%|██████▍   | 7703/12000 [4:33:36<3:01:18,  2.53s/it]
                                                        

 64%|██████▍   | 7703/12000 [4:33:36<3:01:18,  2.53s/it]
 64%|██████▍   | 7704/12000 [4:33:38<2:54:24,  2.44s/it]
                                                        

 64%|██████▍   | 7704/12000 [4:33:38<2:54:24,  2.44s/it]
 64%|██████▍   | 7705/12000 [4:33:40<2:36:42,  2.19s/it]
                                                        

 64%|██████▍   | 7705/12000 [4:33:40<2:36:42,  2.19s/it]
 64%|██████▍   | 7706/12000 [4:33:42<2:40:18,  2.24s/it]
                                                        

 64%|██████▍   | 7706/12000 [4:33:42<2:40:18,  2.24s/it]
 64%|██████▍   | 7707/12000 [4:33:44<2:28:20,  2.07s/it]
                                                        

 64%|██████▍   | 7707/12000 [4:33:44<2:28:20,  2.07s/it]
 64%|██████▍   | 7708/12000 [4:33:46<2:36:42,  2.19s/it]
                                                        

 64%|██████▍   | 7708/12000 [4:33:46<2:36:42,  2.19s/it]
 64%|██████▍   | 7709/12000 [4:33:48<2:27:53,  2.07s/it]
                                                        

 64%|██████▍   | 7709/12000 [4:33:48<2:27:53,  2.07s/it]
 64%|██████▍   | 7710/12000 [4:33:50<2:22:48,  2.00s/it]
                                                        

 64%|██████▍   | 7710/12000 [4:33:50<2:22:48,  2.00s/it]
 64%|██████▍   | 7711/12000 [4:33:53<2:53:14,  2.42s/it]
                                                        

 64%|██████▍   | 7711/12000 [4:33:53<2:53:14,  2.42s/it]
 64%|██████▍   | 7712/12000 [4:33:55<2:42:37,  2.28s/it]
                                                        
{'loss': '0.9272', 'grad_norm': '0.4223', 'learning_rate': '6.329e-05', 'entropy': '0.9458', 'num_tokens': '4.139e+07', 'mean_token_accuracy': '0.7649', 'epoch': '0.06783'}
{'loss': '1.031', 'grad_norm': '0.5674', 'learning_rate': '6.326e-05', 'entropy': '0.9804', 'num_tokens': '4.14e+07', 'mean_token_accuracy': '0.7548', 'epoch': '0.06784'}
{'loss': '0.8577', 'grad_norm': '0.4375', 'learning_rate': '6.324e-05', 'entropy': '0.8503', 'num_tokens': '4.141e+07', 'mean_token_accuracy': '0.7795', 'epoch': '0.06785'}
{'loss': '0.9464', 'grad_norm': '0.4081', 'learning_rate': '6.321e-05', 'entropy': '0.8765', 'num_tokens': '4.141e+07', 'mean_token_accuracy': '0.776', 'epoch': '0.06786'}
{'loss': '0.494', 'grad_norm': '0.3349', 'learning_rate': '6.319e-05', 'entropy': '0.4657', 'num_tokens': '4.142e+07', 'mean_token_accuracy': '0.8779', 'epoch': '0.06787'}
{'loss': '0.9562', 'grad_norm': '0.4643', 'learning_rate': '6.316e-05', 'entropy': '0.9795', 'num_tokens': '4.142e+07', 'mean_token_accuracy': '0.7566', 'epoch': '0.06788'}
{'loss': '1.107', 'grad_norm': '0.4466', 'learning_rate': '6.314e-05', 'entropy': '1.179', 'num_tokens': '4.143e+07', 'mean_token_accuracy': '0.7254', 'epoch': '0.06789'}
{'loss': '1.15', 'grad_norm': '0.4411', 'learning_rate': '6.311e-05', 'entropy': '1.008', 'num_tokens': '4.143e+07', 'mean_token_accuracy': '0.7583', 'epoch': '0.0679'}
{'loss': '0.7747', 'grad_norm': '0.4066', 'learning_rate': '6.309e-05', 'entropy': '0.7764', 'num_tokens': '4.144e+07', 'mean_token_accuracy': '0.8053', 'epoch': '0.0679'}
{'loss': '0.5746', 'grad_norm': '0.4235', 'learning_rate': '6.306e-05', 'entropy': '0.5666', 'num_tokens': '4.144e+07', 'mean_token_accuracy': '0.8387', 'epoch': '0.06791'}
{'loss': '1.086', 'grad_norm': '0.469', 'learning_rate': '6.303e-05', 'entropy': '1.168', 'num_tokens': '4.145e+07', 'mean_token_accuracy': '0.7267', 'epoch': '0.06792'}
{'loss': '0.6975', 'grad_norm': '0.4345', 'learning_rate': '6.301e-05', 'entropy': '0.7033', 'num_tokens': '4.145e+07', 'mean_token_accuracy': '0.8221', 'epoch': '0.06793'}
{'loss': '0.8678', 'grad_norm': '0.5642', 'learning_rate': '6.298e-05', 'entropy': '0.8039', 'num_tokens': '4.146e+07', 'mean_token_accuracy': '0.801', 'epoch': '0.06794'}
{'loss': '0.5325', 'grad_norm': '0.4278', 'learning_rate': '6.296e-05', 'entropy': '0.6029', 'num_tokens': '4.146e+07', 'mean_token_accuracy': '0.8449', 'epoch': '0.06795'}
{'loss': '0.9715', 'grad_norm': '0.3843', 'learning_rate': '6.293e-05', 'entropy': '0.9411', 'num_tokens': '4.147e+07', 'mean_token_accuracy': '0.7736', 'epoch': '0.06796'}
{'loss': '0.8742', 'grad_norm': '0.4054', 'learning_rate': '6.291e-05', 'entropy': '0.9094', 'num_tokens': '4.147e+07', 'mean_token_accuracy': '0.7708', 'epoch': '0.06797'}
{'loss': '0.6778', 'grad_norm': '0.3874', 'learning_rate': '6.288e-05', 'entropy': '0.7313', 'num_tokens': '4.148e+07', 'mean_token_accuracy': '0.8148', 'epoch': '0.06798'}
{'loss': '0.6432', 'grad_norm': '0.3465', 'learning_rate': '6.285e-05', 'entropy': '0.6685', 'num_tokens': '4.148e+07', 'mean_token_accuracy': '0.8269', 'epoch': '0.06798'}
{'loss': '1.077', 'grad_norm': '0.4253', 'learning_rate': '6.283e-05', 'entropy': '1.056', 'num_tokens': '4.149e+07', 'mean_token_accuracy': '0.7301', 'epoch': '0.06799'}
{'loss': '0.7547', 'grad_norm': '0.4367', 'learning_rate': '6.28e-05', 'entropy': '0.6763', 'num_tokens': '4.149e+07', 'mean_token_accuracy': '0.8349', 'epoch': '0.068'}
{'loss': '0.7624', 'grad_norm': '0.3504', 'learning_rate': '6.278e-05', 'entropy': '0.7587', 'num_tokens': '4.15e+07', 'mean_token_accuracy': '0.811', 'epoch': '0.06801'}
{'loss': '0.9465', 'grad_norm': '0.5187', 'learning_rate': '6.275e-05', 'entropy': '0.9762', 'num_tokens': '4.15e+07', 'mean_token_accuracy': '0.7667', 'epoch': '0.06802'}
{'loss': '0.8886', 'grad_norm': '0.3259', 'learning_rate': '6.273e-05', 'entropy': '0.8721', 'num_tokens': '4.151e+07', 'mean_token_accuracy': '0.7793', 'epoch': '0.06803'}
{'loss': '1.348', 'grad_norm': '0.4154', 'learning_rate': '6.27e-05', 'entropy': '1.189', 'num_tokens': '4.151e+07', 'mean_token_accuracy': '0.7331', 'epoch': '0.06804'}
{'loss': '0.842', 'grad_norm': '0.467', 'learning_rate': '6.268e-05', 'entropy': '0.8671', 'num_tokens': '4.152e+07', 'mean_token_accuracy': '0.7806', 'epoch': '0.06805'}
{'loss': '0.7736', 'grad_norm': '0.3484', 'learning_rate': '6.265e-05', 'entropy': '0.9197', 'num_tokens': '4.152e+07', 'mean_token_accuracy': '0.7644', 'epoch': '0.06806'}
{'loss': '0.5871', 'grad_norm': '0.4014', 'learning_rate': '6.262e-05', 'entropy': '0.5962', 'num_tokens': '4.153e+07', 'mean_token_accuracy': '0.8411', 'epoch': '0.06806'}
{'loss': '0.6343', 'grad_norm': '0.2715', 'learning_rate': '6.26e-05', 'entropy': '0.6861', 'num_tokens': '4.153e+07', 'mean_token_accuracy': '0.8221', 'epoch': '0.06807'}
{'loss': '0.937', 'grad_norm': '0.472', 'learning_rate': '6.257e-05', 'entropy': '0.9689', 'num_tokens': '4.154e+07', 'mean_token_accuracy': '0.7593', 'epoch': '0.06808'}
{'loss': '0.8592', 'grad_norm': '0.3995', 'learning_rate': '6.255e-05', 'entropy': '0.7959', 'num_tokens': '4.154e+07', 'mean_token_accuracy': '0.7827', 'epoch': '0.06809'}
{'loss': '0.8901', 'grad_norm': '0.3711', 'learning_rate': '6.252e-05', 'entropy': '0.9611', 'num_tokens': '4.155e+07', 'mean_token_accuracy': '0.7727', 'epoch': '0.0681'}
{'loss': '0.7924', 'grad_norm': '0.4374', 'learning_rate': '6.25e-05', 'entropy': '0.8151', 'num_tokens': '4.155e+07', 'mean_token_accuracy': '0.7826', 'epoch': '0.06811'}
{'loss': '0.9009', 'grad_norm': '0.3708', 'learning_rate': '6.247e-05', 'entropy': '0.8932', 'num_tokens': '4.155e+07', 'mean_token_accuracy': '0.7762', 'epoch': '0.06812'}
{'loss': '0.8273', 'grad_norm': '0.4402', 'learning_rate': '6.245e-05', 'entropy': '0.9456', 'num_tokens': '4.156e+07', 'mean_token_accuracy': '0.7592', 'epoch': '0.06813'}
{'loss': '0.8939', 'grad_norm': '0.4548', 'learning_rate': '6.242e-05', 'entropy': '0.8438', 'num_tokens': '4.156e+07', 'mean_token_accuracy': '0.7843', 'epoch': '0.06813'}
{'loss': '0.6748', 'grad_norm': '0.3988', 'learning_rate': '6.239e-05', 'entropy': '0.6492', 'num_tokens': '4.157e+07', 'mean_token_accuracy': '0.8199', 'epoch': '0.06814'}
{'loss': '0.8714', 'grad_norm': '0.3929', 'learning_rate': '6.237e-05', 'entropy': '0.952', 'num_tokens': '4.158e+07', 'mean_token_accuracy': '0.7614', 'epoch': '0.06815'}
{'loss': '0.7356', 'grad_norm': '0.3406', 'learning_rate': '6.234e-05', 'entropy': '0.7286', 'num_tokens': '4.159e+07', 'mean_token_accuracy': '0.809', 'epoch': '0.06816'}
{'loss': '0.7023', 'grad_norm': '0.3966', 'learning_rate': '6.232e-05', 'entropy': '0.7431', 'num_tokens': '4.159e+07', 'mean_token_accuracy': '0.8075', 'epoch': '0.06817'}
{'loss': '0.7352', 'grad_norm': '0.3786', 'learning_rate': '6.229e-05', 'entropy': '0.7269', 'num_tokens': '4.16e+07', 'mean_token_accuracy': '0.8162', 'epoch': '0.06818'}
{'loss': '0.828', 'grad_norm': '0.3705', 'learning_rate': '6.227e-05', 'entropy': '0.8451', 'num_tokens': '4.16e+07', 'mean_token_accuracy': '0.7877', 'epoch': '0.06819'}
{'loss': '0.673', 'grad_norm': '0.3661', 'learning_rate': '6.224e-05', 'entropy': '0.6438', 'num_tokens': '4.161e+07', 'mean_token_accuracy': '0.8343', 'epoch': '0.0682'}
{'loss': '0.7132', 'grad_norm': '0.3605', 'learning_rate': '6.222e-05', 'entropy': '0.725', 'num_tokens': '4.161e+07', 'mean_token_accuracy': '0.8224', 'epoch': '0.06821'}
{'loss': '0.8325', 'grad_norm': '0.3785', 'learning_rate': '6.219e-05', 'entropy': '0.8874', 'num_tokens': '4.162e+07', 'mean_token_accuracy': '0.7789', 'epoch': '0.06821'}
{'loss': '0.6321', 'grad_norm': '0.4037', 'learning_rate': '6.216e-05', 'entropy': '0.6371', 'num_tokens': '4.162e+07', 'mean_token_accuracy': '0.8331', 'epoch': '0.06822'}
{'loss': '0.8862', 'grad_norm': '0.4125', 'learning_rate': '6.214e-05', 'entropy': '0.8525', 'num_tokens': '4.163e+07', 'mean_token_accuracy': '0.7986', 'epoch': '0.06823'}
{'loss': '0.6539', 'grad_norm': '0.354', 'learning_rate': '6.211e-05', 'entropy': '0.6434', 'num_tokens': '4.163e+07', 'mean_token_accuracy': '0.8263', 'epoch': '0.06824'}

 64%|██████▍   | 7712/12000 [4:33:55<2:42:37,  2.28s/it]
 64%|██████▍   | 7713/12000 [4:33:57<2:25:48,  2.04s/it]
                                                        

 64%|██████▍   | 7713/12000 [4:33:57<2:25:48,  2.04s/it]
 64%|██████▍   | 7714/12000 [4:33:58<2:15:25,  1.90s/it]
                                                        

 64%|██████▍   | 7714/12000 [4:33:58<2:15:25,  1.90s/it]
 64%|██████▍   | 7715/12000 [4:34:00<2:07:36,  1.79s/it]
                                                        

 64%|██████▍   | 7715/12000 [4:34:00<2:07:36,  1.79s/it]
 64%|██████▍   | 7716/12000 [4:34:02<2:04:22,  1.74s/it]
                                                        

 64%|██████▍   | 7716/12000 [4:34:02<2:04:22,  1.74s/it]
 64%|██████▍   | 7717/12000 [4:34:04<2:14:22,  1.88s/it]
                                                        

 64%|██████▍   | 7717/12000 [4:34:04<2:14:22,  1.88s/it]
 64%|██████▍   | 7718/12000 [4:34:05<2:10:18,  1.83s/it]
                                                        

 64%|██████▍   | 7718/12000 [4:34:05<2:10:18,  1.83s/it]
 64%|██████▍   | 7719/12000 [4:34:10<3:02:08,  2.55s/it]
                                                        

 64%|██████▍   | 7719/12000 [4:34:10<3:02:08,  2.55s/it]
 64%|██████▍   | 7720/12000 [4:34:13<3:09:29,  2.66s/it]
                                                        

 64%|██████▍   | 7720/12000 [4:34:13<3:09:29,  2.66s/it]
 64%|██████▍   | 7721/12000 [4:34:15<2:58:11,  2.50s/it]
                                                        

 64%|██████▍   | 7721/12000 [4:34:15<2:58:11,  2.50s/it]
 64%|██████▍   | 7722/12000 [4:34:16<2:38:06,  2.22s/it]
                                                        

 64%|██████▍   | 7722/12000 [4:34:16<2:38:06,  2.22s/it]
 64%|██████▍   | 7723/12000 [4:34:18<2:29:44,  2.10s/it]
                                                        

 64%|██████▍   | 7723/12000 [4:34:18<2:29:44,  2.10s/it]
 64%|██████▍   | 7724/12000 [4:34:20<2:24:18,  2.02s/it]
                                                        

 64%|██████▍   | 7724/12000 [4:34:20<2:24:18,  2.02s/it]
 64%|██████▍   | 7725/12000 [4:34:22<2:14:43,  1.89s/it]
                                                        

 64%|██████▍   | 7725/12000 [4:34:22<2:14:43,  1.89s/it]
 64%|██████▍   | 7726/12000 [4:34:23<2:16:07,  1.91s/it]
                                                        

 64%|██████▍   | 7726/12000 [4:34:23<2:16:07,  1.91s/it]
 64%|██████▍   | 7727/12000 [4:34:26<2:19:51,  1.96s/it]
                                                        

 64%|██████▍   | 7727/12000 [4:34:26<2:19:51,  1.96s/it]
 64%|██████▍   | 7728/12000 [4:34:27<2:18:08,  1.94s/it]
                                                        

 64%|██████▍   | 7728/12000 [4:34:27<2:18:08,  1.94s/it]
 64%|██████▍   | 7729/12000 [4:34:30<2:32:46,  2.15s/it]
                                                        

 64%|██████▍   | 7729/12000 [4:34:30<2:32:46,  2.15s/it]
 64%|██████▍   | 7730/12000 [4:34:32<2:25:06,  2.04s/it]
                                                        

 64%|██████▍   | 7730/12000 [4:34:32<2:25:06,  2.04s/it]
 64%|██████▍   | 7731/12000 [4:34:33<2:13:23,  1.87s/it]
                                                        

 64%|██████▍   | 7731/12000 [4:34:33<2:13:23,  1.87s/it]
 64%|██████▍   | 7732/12000 [4:34:35<2:06:44,  1.78s/it]
                                                        

 64%|██████▍   | 7732/12000 [4:34:35<2:06:44,  1.78s/it]
 64%|██████▍   | 7733/12000 [4:34:38<2:43:58,  2.31s/it]
                                                        

 64%|██████▍   | 7733/12000 [4:34:38<2:43:58,  2.31s/it]
 64%|██████▍   | 7734/12000 [4:34:40<2:31:03,  2.12s/it]
                                                        

 64%|██████▍   | 7734/12000 [4:34:40<2:31:03,  2.12s/it]
 64%|██████▍   | 7735/12000 [4:34:42<2:15:48,  1.91s/it]
                                                        

 64%|██████▍   | 7735/12000 [4:34:42<2:15:48,  1.91s/it]
 64%|██████▍   | 7736/12000 [4:34:43<2:10:38,  1.84s/it]
                                                        

 64%|██████▍   | 7736/12000 [4:34:43<2:10:38,  1.84s/it]
 64%|██████▍   | 7737/12000 [4:34:46<2:34:41,  2.18s/it]
                                                        

 64%|██████▍   | 7737/12000 [4:34:46<2:34:41,  2.18s/it]
 64%|██████▍   | 7738/12000 [4:34:48<2:28:21,  2.09s/it]
                                                        

 64%|██████▍   | 7738/12000 [4:34:48<2:28:21,  2.09s/it]
 64%|██████▍   | 7739/12000 [4:34:52<3:13:50,  2.73s/it]
                                                        

 64%|██████▍   | 7739/12000 [4:34:52<3:13:50,  2.73s/it]
 64%|██████▍   | 7740/12000 [4:34:54<2:47:23,  2.36s/it]
                                                        

 64%|██████▍   | 7740/12000 [4:34:54<2:47:23,  2.36s/it]
 65%|██████▍   | 7741/12000 [4:34:56<2:47:28,  2.36s/it]
                                                        

 65%|██████▍   | 7741/12000 [4:34:56<2:47:28,  2.36s/it]
 65%|██████▍   | 7742/12000 [4:34:58<2:28:18,  2.09s/it]
                                                        

 65%|██████▍   | 7742/12000 [4:34:58<2:28:18,  2.09s/it]
 65%|██████▍   | 7743/12000 [4:35:00<2:40:34,  2.26s/it]
                                                        

 65%|██████▍   | 7743/12000 [4:35:00<2:40:34,  2.26s/it]
 65%|██████▍   | 7744/12000 [4:35:02<2:22:32,  2.01s/it]
                                                        

 65%|██████▍   | 7744/12000 [4:35:02<2:22:32,  2.01s/it]
 65%|██████▍   | 7745/12000 [4:35:04<2:30:50,  2.13s/it]
                                                        

 65%|██████▍   | 7745/12000 [4:35:04<2:30:50,  2.13s/it]
 65%|██████▍   | 7746/12000 [4:35:06<2:18:29,  1.95s/it]
                                                        

 65%|██████▍   | 7746/12000 [4:35:06<2:18:29,  1.95s/it]
 65%|██████▍   | 7747/12000 [4:35:07<2:08:33,  1.81s/it]
                                                        

 65%|██████▍   | 7747/12000 [4:35:07<2:08:33,  1.81s/it]
 65%|██████▍   | 7748/12000 [4:35:10<2:23:42,  2.03s/it]
                                                        

 65%|██████▍   | 7748/12000 [4:35:10<2:23:42,  2.03s/it]
 65%|██████▍   | 7749/12000 [4:35:11<2:14:07,  1.89s/it]
                                                        

 65%|██████▍   | 7749/12000 [4:35:11<2:14:07,  1.89s/it]
 65%|██████▍   | 7750/12000 [4:35:14<2:24:19,  2.04s/it]
                                                        

 65%|██████▍   | 7750/12000 [4:35:14<2:24:19,  2.04s/it]
 65%|██████▍   | 7751/12000 [4:35:17<2:42:36,  2.30s/it]
                                                        

 65%|██████▍   | 7751/12000 [4:35:17<2:42:36,  2.30s/it]
 65%|██████▍   | 7752/12000 [4:35:19<2:46:54,  2.36s/it]
                                                        

 65%|██████▍   | 7752/12000 [4:35:19<2:46:54,  2.36s/it]
 65%|██████▍   | 7753/12000 [4:35:21<2:29:56,  2.12s/it]
                                                        

 65%|██████▍   | 7753/12000 [4:35:21<2:29:56,  2.12s/it]
 65%|██████▍   | 7754/12000 [4:35:22<2:17:27,  1.94s/it]
                                                        

 65%|██████▍   | 7754/12000 [4:35:22<2:17:27,  1.94s/it]
 65%|██████▍   | 7755/12000 [4:35:26<2:55:02,  2.47s/it]
                                                        

 65%|██████▍   | 7755/12000 [4:35:26<2:55:02,  2.47s/it]
 65%|██████▍   | 7756/12000 [4:35:27<2:37:19,  2.22s/it]
                                                        

 65%|██████▍   | 7756/12000 [4:35:27<2:37:19,  2.22s/it]
 65%|██████▍   | 7757/12000 [4:35:30<2:35:55,  2.20s/it]
                                                        

 65%|██████▍   | 7757/12000 [4:35:30<2:35:55,  2.20s/it]
 65%|██████▍   | 7758/12000 [4:35:31<2:20:58,  1.99s/it]
                                                        

 65%|██████▍   | 7758/12000 [4:35:31<2:20:58,  1.99s/it]
 65%|██████▍   | 7759/12000 [4:35:33<2:25:31,  2.06s/it]
                                                        
{'loss': '1.414', 'grad_norm': '0.4152', 'learning_rate': '6.209e-05', 'entropy': '1.332', 'num_tokens': '4.164e+07', 'mean_token_accuracy': '0.6836', 'epoch': '0.06825'}
{'loss': '0.742', 'grad_norm': '0.3509', 'learning_rate': '6.206e-05', 'entropy': '0.7577', 'num_tokens': '4.164e+07', 'mean_token_accuracy': '0.8055', 'epoch': '0.06826'}
{'loss': '0.6283', 'grad_norm': '0.3016', 'learning_rate': '6.204e-05', 'entropy': '0.6335', 'num_tokens': '4.165e+07', 'mean_token_accuracy': '0.8283', 'epoch': '0.06827'}
{'loss': '0.6914', 'grad_norm': '0.3798', 'learning_rate': '6.201e-05', 'entropy': '0.6869', 'num_tokens': '4.165e+07', 'mean_token_accuracy': '0.8364', 'epoch': '0.06828'}
{'loss': '0.6649', 'grad_norm': '0.3808', 'learning_rate': '6.199e-05', 'entropy': '0.6832', 'num_tokens': '4.166e+07', 'mean_token_accuracy': '0.8221', 'epoch': '0.06829'}
{'loss': '0.6679', 'grad_norm': '0.3548', 'learning_rate': '6.196e-05', 'entropy': '0.7239', 'num_tokens': '4.166e+07', 'mean_token_accuracy': '0.8154', 'epoch': '0.06829'}
{'loss': '0.8965', 'grad_norm': '0.4655', 'learning_rate': '6.194e-05', 'entropy': '0.8296', 'num_tokens': '4.167e+07', 'mean_token_accuracy': '0.7792', 'epoch': '0.0683'}
{'loss': '0.7315', 'grad_norm': '0.2872', 'learning_rate': '6.191e-05', 'entropy': '0.8472', 'num_tokens': '4.168e+07', 'mean_token_accuracy': '0.7894', 'epoch': '0.06831'}
{'loss': '0.8461', 'grad_norm': '0.3905', 'learning_rate': '6.188e-05', 'entropy': '0.8419', 'num_tokens': '4.168e+07', 'mean_token_accuracy': '0.784', 'epoch': '0.06832'}
{'loss': '0.9893', 'grad_norm': '0.5587', 'learning_rate': '6.186e-05', 'entropy': '1.011', 'num_tokens': '4.169e+07', 'mean_token_accuracy': '0.7401', 'epoch': '0.06833'}
{'loss': '0.8074', 'grad_norm': '0.4465', 'learning_rate': '6.183e-05', 'entropy': '0.8329', 'num_tokens': '4.169e+07', 'mean_token_accuracy': '0.7959', 'epoch': '0.06834'}
{'loss': '0.7764', 'grad_norm': '0.3997', 'learning_rate': '6.181e-05', 'entropy': '0.9453', 'num_tokens': '4.17e+07', 'mean_token_accuracy': '0.7683', 'epoch': '0.06835'}
{'loss': '1.07', 'grad_norm': '0.4007', 'learning_rate': '6.178e-05', 'entropy': '0.9767', 'num_tokens': '4.17e+07', 'mean_token_accuracy': '0.7638', 'epoch': '0.06836'}
{'loss': '0.736', 'grad_norm': '0.3103', 'learning_rate': '6.176e-05', 'entropy': '0.7362', 'num_tokens': '4.171e+07', 'mean_token_accuracy': '0.8069', 'epoch': '0.06836'}
{'loss': '0.7342', 'grad_norm': '0.3729', 'learning_rate': '6.173e-05', 'entropy': '0.8286', 'num_tokens': '4.171e+07', 'mean_token_accuracy': '0.7901', 'epoch': '0.06837'}
{'loss': '0.916', 'grad_norm': '0.3841', 'learning_rate': '6.171e-05', 'entropy': '0.9145', 'num_tokens': '4.172e+07', 'mean_token_accuracy': '0.7651', 'epoch': '0.06838'}
{'loss': '0.9182', 'grad_norm': '0.4382', 'learning_rate': '6.168e-05', 'entropy': '0.9061', 'num_tokens': '4.172e+07', 'mean_token_accuracy': '0.7807', 'epoch': '0.06839'}
{'loss': '0.9669', 'grad_norm': '0.3705', 'learning_rate': '6.166e-05', 'entropy': '0.853', 'num_tokens': '4.173e+07', 'mean_token_accuracy': '0.7782', 'epoch': '0.0684'}
{'loss': '0.848', 'grad_norm': '0.4207', 'learning_rate': '6.163e-05', 'entropy': '0.8706', 'num_tokens': '4.173e+07', 'mean_token_accuracy': '0.7824', 'epoch': '0.06841'}
{'loss': '0.7775', 'grad_norm': '0.4862', 'learning_rate': '6.16e-05', 'entropy': '0.8109', 'num_tokens': '4.174e+07', 'mean_token_accuracy': '0.8066', 'epoch': '0.06842'}
{'loss': '0.7028', 'grad_norm': '0.3005', 'learning_rate': '6.158e-05', 'entropy': '0.788', 'num_tokens': '4.174e+07', 'mean_token_accuracy': '0.7971', 'epoch': '0.06843'}
{'loss': '0.855', 'grad_norm': '0.3986', 'learning_rate': '6.155e-05', 'entropy': '0.9393', 'num_tokens': '4.175e+07', 'mean_token_accuracy': '0.7618', 'epoch': '0.06844'}
{'loss': '0.9159', 'grad_norm': '0.3631', 'learning_rate': '6.153e-05', 'entropy': '0.8778', 'num_tokens': '4.175e+07', 'mean_token_accuracy': '0.7785', 'epoch': '0.06844'}
{'loss': '0.782', 'grad_norm': '0.3979', 'learning_rate': '6.15e-05', 'entropy': '0.8369', 'num_tokens': '4.176e+07', 'mean_token_accuracy': '0.797', 'epoch': '0.06845'}
{'loss': '0.6177', 'grad_norm': '0.3336', 'learning_rate': '6.148e-05', 'entropy': '0.6149', 'num_tokens': '4.176e+07', 'mean_token_accuracy': '0.8394', 'epoch': '0.06846'}
{'loss': '0.579', 'grad_norm': '0.3218', 'learning_rate': '6.145e-05', 'entropy': '0.5879', 'num_tokens': '4.177e+07', 'mean_token_accuracy': '0.8384', 'epoch': '0.06847'}
{'loss': '0.7547', 'grad_norm': '0.3083', 'learning_rate': '6.143e-05', 'entropy': '0.8039', 'num_tokens': '4.178e+07', 'mean_token_accuracy': '0.7896', 'epoch': '0.06848'}
{'loss': '0.9655', 'grad_norm': '0.3593', 'learning_rate': '6.14e-05', 'entropy': '1.001', 'num_tokens': '4.178e+07', 'mean_token_accuracy': '0.7482', 'epoch': '0.06849'}
{'loss': '0.5573', 'grad_norm': '0.3263', 'learning_rate': '6.138e-05', 'entropy': '0.6603', 'num_tokens': '4.179e+07', 'mean_token_accuracy': '0.8252', 'epoch': '0.0685'}
{'loss': '0.7966', 'grad_norm': '0.3943', 'learning_rate': '6.135e-05', 'entropy': '0.8868', 'num_tokens': '4.179e+07', 'mean_token_accuracy': '0.7778', 'epoch': '0.06851'}
{'loss': '0.7616', 'grad_norm': '0.387', 'learning_rate': '6.132e-05', 'entropy': '0.8689', 'num_tokens': '4.18e+07', 'mean_token_accuracy': '0.7633', 'epoch': '0.06852'}
{'loss': '0.8893', 'grad_norm': '0.4284', 'learning_rate': '6.13e-05', 'entropy': '0.9041', 'num_tokens': '4.18e+07', 'mean_token_accuracy': '0.7738', 'epoch': '0.06852'}
{'loss': '1.02', 'grad_norm': '0.4437', 'learning_rate': '6.127e-05', 'entropy': '1.008', 'num_tokens': '4.181e+07', 'mean_token_accuracy': '0.7594', 'epoch': '0.06853'}
{'loss': '0.7146', 'grad_norm': '0.3711', 'learning_rate': '6.125e-05', 'entropy': '0.7864', 'num_tokens': '4.181e+07', 'mean_token_accuracy': '0.7919', 'epoch': '0.06854'}
{'loss': '0.8298', 'grad_norm': '0.3575', 'learning_rate': '6.122e-05', 'entropy': '0.8767', 'num_tokens': '4.182e+07', 'mean_token_accuracy': '0.7755', 'epoch': '0.06855'}
{'loss': '0.9139', 'grad_norm': '0.4117', 'learning_rate': '6.12e-05', 'entropy': '0.9435', 'num_tokens': '4.182e+07', 'mean_token_accuracy': '0.7737', 'epoch': '0.06856'}
{'loss': '1.027', 'grad_norm': '0.3812', 'learning_rate': '6.117e-05', 'entropy': '1.093', 'num_tokens': '4.183e+07', 'mean_token_accuracy': '0.7349', 'epoch': '0.06857'}
{'loss': '1.006', 'grad_norm': '0.5316', 'learning_rate': '6.115e-05', 'entropy': '1.139', 'num_tokens': '4.183e+07', 'mean_token_accuracy': '0.7337', 'epoch': '0.06858'}
{'loss': '0.6551', 'grad_norm': '0.3996', 'learning_rate': '6.112e-05', 'entropy': '0.6556', 'num_tokens': '4.184e+07', 'mean_token_accuracy': '0.8248', 'epoch': '0.06859'}
{'loss': '1.045', 'grad_norm': '0.4184', 'learning_rate': '6.11e-05', 'entropy': '1.014', 'num_tokens': '4.184e+07', 'mean_token_accuracy': '0.7532', 'epoch': '0.0686'}
{'loss': '0.803', 'grad_norm': '0.421', 'learning_rate': '6.107e-05', 'entropy': '0.857', 'num_tokens': '4.185e+07', 'mean_token_accuracy': '0.7862', 'epoch': '0.0686'}
{'loss': '0.5568', 'grad_norm': '0.32', 'learning_rate': '6.105e-05', 'entropy': '0.5849', 'num_tokens': '4.185e+07', 'mean_token_accuracy': '0.8419', 'epoch': '0.06861'}
{'loss': '0.7605', 'grad_norm': '0.4036', 'learning_rate': '6.102e-05', 'entropy': '0.7717', 'num_tokens': '4.186e+07', 'mean_token_accuracy': '0.8084', 'epoch': '0.06862'}
{'loss': '0.8257', 'grad_norm': '0.3791', 'learning_rate': '6.099e-05', 'entropy': '0.7896', 'num_tokens': '4.186e+07', 'mean_token_accuracy': '0.8031', 'epoch': '0.06863'}
{'loss': '0.9225', 'grad_norm': '0.4648', 'learning_rate': '6.097e-05', 'entropy': '1.002', 'num_tokens': '4.187e+07', 'mean_token_accuracy': '0.7539', 'epoch': '0.06864'}
{'loss': '0.7143', 'grad_norm': '0.3557', 'learning_rate': '6.094e-05', 'entropy': '0.7075', 'num_tokens': '4.187e+07', 'mean_token_accuracy': '0.8189', 'epoch': '0.06865'}
{'loss': '0.6093', 'grad_norm': '0.2896', 'learning_rate': '6.092e-05', 'entropy': '0.716', 'num_tokens': '4.188e+07', 'mean_token_accuracy': '0.8231', 'epoch': '0.06866'}

 65%|██████▍   | 7759/12000 [4:35:33<2:25:31,  2.06s/it]
 65%|██████▍   | 7760/12000 [4:35:35<2:12:47,  1.88s/it]
                                                        

 65%|██████▍   | 7760/12000 [4:35:35<2:12:47,  1.88s/it]
 65%|██████▍   | 7761/12000 [4:35:36<2:04:37,  1.76s/it]
                                                        

 65%|██████▍   | 7761/12000 [4:35:36<2:04:37,  1.76s/it]
 65%|██████▍   | 7762/12000 [4:35:38<1:58:55,  1.68s/it]
                                                        

 65%|██████▍   | 7762/12000 [4:35:38<1:58:55,  1.68s/it]
 65%|██████▍   | 7763/12000 [4:35:40<2:03:22,  1.75s/it]
                                                        

 65%|██████▍   | 7763/12000 [4:35:40<2:03:22,  1.75s/it]
 65%|██████▍   | 7764/12000 [4:35:41<1:58:57,  1.68s/it]
                                                        

 65%|██████▍   | 7764/12000 [4:35:41<1:58:57,  1.68s/it]
 65%|██████▍   | 7765/12000 [4:35:43<1:53:11,  1.60s/it]
                                                        

 65%|██████▍   | 7765/12000 [4:35:43<1:53:11,  1.60s/it]
 65%|██████▍   | 7766/12000 [4:35:44<1:52:05,  1.59s/it]
                                                        

 65%|██████▍   | 7766/12000 [4:35:44<1:52:05,  1.59s/it]
 65%|██████▍   | 7767/12000 [4:35:46<1:55:55,  1.64s/it]
                                                        

 65%|██████▍   | 7767/12000 [4:35:46<1:55:55,  1.64s/it]
 65%|██████▍   | 7768/12000 [4:35:52<3:18:06,  2.81s/it]
                                                        

 65%|██████▍   | 7768/12000 [4:35:52<3:18:06,  2.81s/it]
 65%|██████▍   | 7769/12000 [4:35:53<2:51:19,  2.43s/it]
                                                        

 65%|██████▍   | 7769/12000 [4:35:53<2:51:19,  2.43s/it]
 65%|██████▍   | 7770/12000 [4:35:55<2:35:18,  2.20s/it]
                                                        

 65%|██████▍   | 7770/12000 [4:35:55<2:35:18,  2.20s/it]
 65%|██████▍   | 7771/12000 [4:35:56<2:21:48,  2.01s/it]
                                                        

 65%|██████▍   | 7771/12000 [4:35:56<2:21:48,  2.01s/it]
 65%|██████▍   | 7772/12000 [4:35:59<2:36:49,  2.23s/it]
                                                        

 65%|██████▍   | 7772/12000 [4:35:59<2:36:49,  2.23s/it]
 65%|██████▍   | 7773/12000 [4:36:01<2:27:04,  2.09s/it]
                                                        

 65%|██████▍   | 7773/12000 [4:36:01<2:27:04,  2.09s/it]
 65%|██████▍   | 7774/12000 [4:36:03<2:31:03,  2.14s/it]
                                                        

 65%|██████▍   | 7774/12000 [4:36:03<2:31:03,  2.14s/it]
 65%|██████▍   | 7775/12000 [4:36:05<2:21:26,  2.01s/it]
                                                        

 65%|██████▍   | 7775/12000 [4:36:05<2:21:26,  2.01s/it]
 65%|██████▍   | 7776/12000 [4:36:07<2:34:22,  2.19s/it]
                                                        

 65%|██████▍   | 7776/12000 [4:36:07<2:34:22,  2.19s/it]
 65%|██████▍   | 7777/12000 [4:36:12<3:19:10,  2.83s/it]
                                                        

 65%|██████▍   | 7777/12000 [4:36:12<3:19:10,  2.83s/it]
 65%|██████▍   | 7778/12000 [4:36:13<2:49:10,  2.40s/it]
                                                        

 65%|██████▍   | 7778/12000 [4:36:13<2:49:10,  2.40s/it]
 65%|██████▍   | 7779/12000 [4:36:14<2:27:49,  2.10s/it]
                                                        

 65%|██████▍   | 7779/12000 [4:36:14<2:27:49,  2.10s/it]
 65%|██████▍   | 7780/12000 [4:36:16<2:15:03,  1.92s/it]
                                                        

 65%|██████▍   | 7780/12000 [4:36:16<2:15:03,  1.92s/it]
 65%|██████▍   | 7781/12000 [4:36:18<2:11:06,  1.86s/it]
                                                        

 65%|██████▍   | 7781/12000 [4:36:18<2:11:06,  1.86s/it]
 65%|██████▍   | 7782/12000 [4:36:20<2:09:38,  1.84s/it]
                                                        

 65%|██████▍   | 7782/12000 [4:36:20<2:09:38,  1.84s/it]
 65%|██████▍   | 7783/12000 [4:36:21<2:00:04,  1.71s/it]
                                                        

 65%|██████▍   | 7783/12000 [4:36:21<2:00:04,  1.71s/it]
 65%|██████▍   | 7784/12000 [4:36:23<2:12:31,  1.89s/it]
                                                        

 65%|██████▍   | 7784/12000 [4:36:23<2:12:31,  1.89s/it]
 65%|██████▍   | 7785/12000 [4:36:25<2:04:59,  1.78s/it]
                                                        

 65%|██████▍   | 7785/12000 [4:36:25<2:04:59,  1.78s/it]
 65%|██████▍   | 7786/12000 [4:36:27<2:04:30,  1.77s/it]
                                                        

 65%|██████▍   | 7786/12000 [4:36:27<2:04:30,  1.77s/it]
 65%|██████▍   | 7787/12000 [4:36:28<2:06:22,  1.80s/it]
                                                        

 65%|██████▍   | 7787/12000 [4:36:28<2:06:22,  1.80s/it]
 65%|██████▍   | 7788/12000 [4:36:31<2:17:12,  1.95s/it]
                                                        

 65%|██████▍   | 7788/12000 [4:36:31<2:17:12,  1.95s/it]
 65%|██████▍   | 7789/12000 [4:36:32<2:10:38,  1.86s/it]
                                                        

 65%|██████▍   | 7789/12000 [4:36:32<2:10:38,  1.86s/it]
 65%|██████▍   | 7790/12000 [4:36:34<2:10:42,  1.86s/it]
                                                        

 65%|██████▍   | 7790/12000 [4:36:34<2:10:42,  1.86s/it]
 65%|██████▍   | 7791/12000 [4:36:36<2:01:58,  1.74s/it]
                                                        

 65%|██████▍   | 7791/12000 [4:36:36<2:01:58,  1.74s/it]
 65%|██████▍   | 7792/12000 [4:36:37<2:01:58,  1.74s/it]
                                                        

 65%|██████▍   | 7792/12000 [4:36:37<2:01:58,  1.74s/it]
 65%|██████▍   | 7793/12000 [4:36:40<2:20:15,  2.00s/it]
                                                        

 65%|██████▍   | 7793/12000 [4:36:40<2:20:15,  2.00s/it]
 65%|██████▍   | 7794/12000 [4:36:42<2:11:11,  1.87s/it]
                                                        

 65%|██████▍   | 7794/12000 [4:36:42<2:11:11,  1.87s/it]
 65%|██████▍   | 7795/12000 [4:36:43<2:03:37,  1.76s/it]
                                                        

 65%|██████▍   | 7795/12000 [4:36:43<2:03:37,  1.76s/it]
 65%|██████▍   | 7796/12000 [4:36:45<2:14:00,  1.91s/it]
                                                        

 65%|██████▍   | 7796/12000 [4:36:45<2:14:00,  1.91s/it]
 65%|██████▍   | 7797/12000 [4:36:48<2:31:39,  2.17s/it]
                                                        

 65%|██████▍   | 7797/12000 [4:36:48<2:31:39,  2.17s/it]
 65%|██████▍   | 7798/12000 [4:36:53<3:25:13,  2.93s/it]
                                                        

 65%|██████▍   | 7798/12000 [4:36:53<3:25:13,  2.93s/it]
 65%|██████▍   | 7799/12000 [4:36:56<3:38:53,  3.13s/it]
                                                        

 65%|██████▍   | 7799/12000 [4:36:56<3:38:53,  3.13s/it]
 65%|██████▌   | 7800/12000 [4:36:58<3:11:03,  2.73s/it]
                                                        

 65%|██████▌   | 7800/12000 [4:36:58<3:11:03,  2.73s/it]
 65%|██████▌   | 7801/12000 [4:37:01<3:18:36,  2.84s/it]
                                                        

 65%|██████▌   | 7801/12000 [4:37:01<3:18:36,  2.84s/it]
 65%|██████▌   | 7802/12000 [4:37:03<2:52:52,  2.47s/it]
                                                        

 65%|██████▌   | 7802/12000 [4:37:03<2:52:52,  2.47s/it]
 65%|██████▌   | 7803/12000 [4:37:04<2:31:43,  2.17s/it]
                                                        

 65%|██████▌   | 7803/12000 [4:37:04<2:31:43,  2.17s/it]
 65%|██████▌   | 7804/12000 [4:37:06<2:22:04,  2.03s/it]
                                                        

 65%|██████▌   | 7804/12000 [4:37:06<2:22:04,  2.03s/it]
 65%|██████▌   | 7805/12000 [4:37:08<2:15:02,  1.93s/it]
                                                        

 65%|██████▌   | 7805/12000 [4:37:08<2:15:02,  1.93s/it]
 65%|██████▌   | 7806/12000 [4:37:10<2:16:36,  1.95s/it]
                                                        
{'loss': '0.786', 'grad_norm': '0.487', 'learning_rate': '6.089e-05', 'entropy': '0.8453', 'num_tokens': '4.188e+07', 'mean_token_accuracy': '0.7879', 'epoch': '0.06867'}
{'loss': '1.093', 'grad_norm': '0.4807', 'learning_rate': '6.087e-05', 'entropy': '1.031', 'num_tokens': '4.189e+07', 'mean_token_accuracy': '0.7504', 'epoch': '0.06867'}
{'loss': '0.5731', 'grad_norm': '0.4078', 'learning_rate': '6.084e-05', 'entropy': '0.5915', 'num_tokens': '4.189e+07', 'mean_token_accuracy': '0.8491', 'epoch': '0.06868'}
{'loss': '0.7004', 'grad_norm': '0.3479', 'learning_rate': '6.082e-05', 'entropy': '0.7724', 'num_tokens': '4.19e+07', 'mean_token_accuracy': '0.8008', 'epoch': '0.06869'}
{'loss': '0.7261', 'grad_norm': '0.3432', 'learning_rate': '6.079e-05', 'entropy': '0.6771', 'num_tokens': '4.19e+07', 'mean_token_accuracy': '0.8199', 'epoch': '0.0687'}
{'loss': '0.9616', 'grad_norm': '0.4513', 'learning_rate': '6.077e-05', 'entropy': '0.9633', 'num_tokens': '4.191e+07', 'mean_token_accuracy': '0.7728', 'epoch': '0.06871'}
{'loss': '1.037', 'grad_norm': '0.5604', 'learning_rate': '6.074e-05', 'entropy': '1.114', 'num_tokens': '4.191e+07', 'mean_token_accuracy': '0.7299', 'epoch': '0.06872'}
{'loss': '0.5671', 'grad_norm': '0.322', 'learning_rate': '6.072e-05', 'entropy': '0.5517', 'num_tokens': '4.191e+07', 'mean_token_accuracy': '0.8518', 'epoch': '0.06873'}
{'loss': '0.4843', 'grad_norm': '0.3126', 'learning_rate': '6.069e-05', 'entropy': '0.5375', 'num_tokens': '4.192e+07', 'mean_token_accuracy': '0.8486', 'epoch': '0.06874'}
{'loss': '1.105', 'grad_norm': '0.4732', 'learning_rate': '6.066e-05', 'entropy': '1.259', 'num_tokens': '4.193e+07', 'mean_token_accuracy': '0.6926', 'epoch': '0.06875'}
{'loss': '0.5759', 'grad_norm': '0.3992', 'learning_rate': '6.064e-05', 'entropy': '0.66', 'num_tokens': '4.193e+07', 'mean_token_accuracy': '0.8287', 'epoch': '0.06875'}
{'loss': '0.8901', 'grad_norm': '0.4249', 'learning_rate': '6.061e-05', 'entropy': '0.8985', 'num_tokens': '4.194e+07', 'mean_token_accuracy': '0.7709', 'epoch': '0.06876'}
{'loss': '0.7343', 'grad_norm': '0.391', 'learning_rate': '6.059e-05', 'entropy': '0.7752', 'num_tokens': '4.194e+07', 'mean_token_accuracy': '0.8007', 'epoch': '0.06877'}
{'loss': '0.7781', 'grad_norm': '0.36', 'learning_rate': '6.056e-05', 'entropy': '0.8928', 'num_tokens': '4.195e+07', 'mean_token_accuracy': '0.7775', 'epoch': '0.06878'}
{'loss': '0.693', 'grad_norm': '0.3515', 'learning_rate': '6.054e-05', 'entropy': '0.6892', 'num_tokens': '4.196e+07', 'mean_token_accuracy': '0.8274', 'epoch': '0.06879'}
{'loss': '0.7099', 'grad_norm': '0.3692', 'learning_rate': '6.051e-05', 'entropy': '0.7105', 'num_tokens': '4.196e+07', 'mean_token_accuracy': '0.811', 'epoch': '0.0688'}
{'loss': '0.6283', 'grad_norm': '0.3982', 'learning_rate': '6.049e-05', 'entropy': '0.6472', 'num_tokens': '4.197e+07', 'mean_token_accuracy': '0.8242', 'epoch': '0.06881'}
{'loss': '0.8512', 'grad_norm': '0.4424', 'learning_rate': '6.046e-05', 'entropy': '0.9597', 'num_tokens': '4.197e+07', 'mean_token_accuracy': '0.7675', 'epoch': '0.06882'}
{'loss': '0.8688', 'grad_norm': '0.4233', 'learning_rate': '6.044e-05', 'entropy': '0.7659', 'num_tokens': '4.198e+07', 'mean_token_accuracy': '0.8142', 'epoch': '0.06883'}
{'loss': '0.6989', 'grad_norm': '0.3823', 'learning_rate': '6.041e-05', 'entropy': '0.7398', 'num_tokens': '4.198e+07', 'mean_token_accuracy': '0.8165', 'epoch': '0.06883'}
{'loss': '0.6999', 'grad_norm': '0.4047', 'learning_rate': '6.039e-05', 'entropy': '0.7031', 'num_tokens': '4.199e+07', 'mean_token_accuracy': '0.8178', 'epoch': '0.06884'}
{'loss': '0.8148', 'grad_norm': '0.3595', 'learning_rate': '6.036e-05', 'entropy': '0.8224', 'num_tokens': '4.199e+07', 'mean_token_accuracy': '0.8044', 'epoch': '0.06885'}
{'loss': '0.7432', 'grad_norm': '0.4246', 'learning_rate': '6.034e-05', 'entropy': '0.7939', 'num_tokens': '4.199e+07', 'mean_token_accuracy': '0.8028', 'epoch': '0.06886'}
{'loss': '0.6821', 'grad_norm': '0.3474', 'learning_rate': '6.031e-05', 'entropy': '0.6651', 'num_tokens': '4.2e+07', 'mean_token_accuracy': '0.8209', 'epoch': '0.06887'}
{'loss': '0.8873', 'grad_norm': '0.3888', 'learning_rate': '6.029e-05', 'entropy': '0.8896', 'num_tokens': '4.2e+07', 'mean_token_accuracy': '0.78', 'epoch': '0.06888'}
{'loss': '1.473', 'grad_norm': '0.4528', 'learning_rate': '6.026e-05', 'entropy': '1.321', 'num_tokens': '4.201e+07', 'mean_token_accuracy': '0.6843', 'epoch': '0.06889'}
{'loss': '0.7096', 'grad_norm': '0.3686', 'learning_rate': '6.023e-05', 'entropy': '0.7023', 'num_tokens': '4.202e+07', 'mean_token_accuracy': '0.8242', 'epoch': '0.0689'}
{'loss': '0.7339', 'grad_norm': '0.5105', 'learning_rate': '6.021e-05', 'entropy': '0.7766', 'num_tokens': '4.202e+07', 'mean_token_accuracy': '0.7956', 'epoch': '0.0689'}
{'loss': '0.5419', 'grad_norm': '0.4334', 'learning_rate': '6.018e-05', 'entropy': '0.5311', 'num_tokens': '4.202e+07', 'mean_token_accuracy': '0.8537', 'epoch': '0.06891'}
{'loss': '0.5812', 'grad_norm': '0.3199', 'learning_rate': '6.016e-05', 'entropy': '0.62', 'num_tokens': '4.203e+07', 'mean_token_accuracy': '0.8457', 'epoch': '0.06892'}
{'loss': '0.6672', 'grad_norm': '0.3343', 'learning_rate': '6.013e-05', 'entropy': '0.6788', 'num_tokens': '4.204e+07', 'mean_token_accuracy': '0.8198', 'epoch': '0.06893'}
{'loss': '1.026', 'grad_norm': '0.4458', 'learning_rate': '6.011e-05', 'entropy': '1', 'num_tokens': '4.204e+07', 'mean_token_accuracy': '0.7524', 'epoch': '0.06894'}
{'loss': '1.047', 'grad_norm': '0.4301', 'learning_rate': '6.008e-05', 'entropy': '1.081', 'num_tokens': '4.205e+07', 'mean_token_accuracy': '0.7347', 'epoch': '0.06895'}
{'loss': '0.8038', 'grad_norm': '0.4221', 'learning_rate': '6.006e-05', 'entropy': '0.8426', 'num_tokens': '4.205e+07', 'mean_token_accuracy': '0.791', 'epoch': '0.06896'}
{'loss': '0.9699', 'grad_norm': '0.4488', 'learning_rate': '6.003e-05', 'entropy': '0.876', 'num_tokens': '4.206e+07', 'mean_token_accuracy': '0.7887', 'epoch': '0.06897'}
{'loss': '0.7535', 'grad_norm': '0.4178', 'learning_rate': '6.001e-05', 'entropy': '0.7553', 'num_tokens': '4.206e+07', 'mean_token_accuracy': '0.8012', 'epoch': '0.06898'}
{'loss': '0.6', 'grad_norm': '0.3091', 'learning_rate': '5.998e-05', 'entropy': '0.6341', 'num_tokens': '4.207e+07', 'mean_token_accuracy': '0.8333', 'epoch': '0.06898'}
{'loss': '0.5536', 'grad_norm': '0.2656', 'learning_rate': '5.996e-05', 'entropy': '0.5482', 'num_tokens': '4.207e+07', 'mean_token_accuracy': '0.8528', 'epoch': '0.06899'}
{'loss': '0.8025', 'grad_norm': '0.4656', 'learning_rate': '5.993e-05', 'entropy': '0.7671', 'num_tokens': '4.208e+07', 'mean_token_accuracy': '0.8117', 'epoch': '0.069'}
{'loss': '1.782', 'grad_norm': '0.5123', 'learning_rate': '5.991e-05', 'entropy': '1.094', 'num_tokens': '4.209e+07', 'mean_token_accuracy': '0.7645', 'epoch': '0.06901'}
{'loss': '0.5454', 'grad_norm': '0.288', 'learning_rate': '5.988e-05', 'entropy': '0.5846', 'num_tokens': '4.209e+07', 'mean_token_accuracy': '0.8514', 'epoch': '0.06902'}
{'loss': '1.006', 'grad_norm': '0.4254', 'learning_rate': '5.986e-05', 'entropy': '0.8886', 'num_tokens': '4.21e+07', 'mean_token_accuracy': '0.7786', 'epoch': '0.06903'}
{'loss': '0.6867', 'grad_norm': '0.373', 'learning_rate': '5.983e-05', 'entropy': '0.7941', 'num_tokens': '4.21e+07', 'mean_token_accuracy': '0.7975', 'epoch': '0.06904'}
{'loss': '0.7783', 'grad_norm': '0.4019', 'learning_rate': '5.981e-05', 'entropy': '0.8269', 'num_tokens': '4.211e+07', 'mean_token_accuracy': '0.802', 'epoch': '0.06905'}
{'loss': '0.8499', 'grad_norm': '0.4303', 'learning_rate': '5.978e-05', 'entropy': '0.8515', 'num_tokens': '4.211e+07', 'mean_token_accuracy': '0.7931', 'epoch': '0.06906'}
{'loss': '1.401', 'grad_norm': '0.6754', 'learning_rate': '5.975e-05', 'entropy': '1.339', 'num_tokens': '4.212e+07', 'mean_token_accuracy': '0.7258', 'epoch': '0.06906'}
{'loss': '0.6921', 'grad_norm': '0.4113', 'learning_rate': '5.973e-05', 'entropy': '0.6869', 'num_tokens': '4.212e+07', 'mean_token_accuracy': '0.8107', 'epoch': '0.06907'}

 65%|██████▌   | 7806/12000 [4:37:10<2:16:36,  1.95s/it]
 65%|██████▌   | 7807/12000 [4:37:12<2:13:10,  1.91s/it]
                                                        

 65%|██████▌   | 7807/12000 [4:37:12<2:13:10,  1.91s/it]
 65%|██████▌   | 7808/12000 [4:37:13<2:13:33,  1.91s/it]
                                                        

 65%|██████▌   | 7808/12000 [4:37:13<2:13:33,  1.91s/it]
 65%|██████▌   | 7809/12000 [4:37:16<2:29:03,  2.13s/it]
                                                        

 65%|██████▌   | 7809/12000 [4:37:16<2:29:03,  2.13s/it]
 65%|██████▌   | 7810/12000 [4:37:18<2:16:25,  1.95s/it]
                                                        

 65%|██████▌   | 7810/12000 [4:37:18<2:16:25,  1.95s/it]
 65%|██████▌   | 7811/12000 [4:37:19<2:11:13,  1.88s/it]
                                                        

 65%|██████▌   | 7811/12000 [4:37:19<2:11:13,  1.88s/it]
 65%|██████▌   | 7812/12000 [4:37:21<2:13:15,  1.91s/it]
                                                        

 65%|██████▌   | 7812/12000 [4:37:21<2:13:15,  1.91s/it]
 65%|██████▌   | 7813/12000 [4:37:23<2:05:11,  1.79s/it]
                                                        

 65%|██████▌   | 7813/12000 [4:37:23<2:05:11,  1.79s/it]
 65%|██████▌   | 7814/12000 [4:37:26<2:23:36,  2.06s/it]
                                                        

 65%|██████▌   | 7814/12000 [4:37:26<2:23:36,  2.06s/it]
 65%|██████▌   | 7815/12000 [4:37:27<2:09:56,  1.86s/it]
                                                        

 65%|██████▌   | 7815/12000 [4:37:27<2:09:56,  1.86s/it]
 65%|██████▌   | 7816/12000 [4:37:28<2:02:41,  1.76s/it]
                                                        

 65%|██████▌   | 7816/12000 [4:37:28<2:02:41,  1.76s/it]
 65%|██████▌   | 7817/12000 [4:37:31<2:09:27,  1.86s/it]
                                                        

 65%|██████▌   | 7817/12000 [4:37:31<2:09:27,  1.86s/it]
 65%|██████▌   | 7818/12000 [4:37:33<2:24:34,  2.07s/it]
                                                        

 65%|██████▌   | 7818/12000 [4:37:33<2:24:34,  2.07s/it]
 65%|██████▌   | 7819/12000 [4:37:35<2:16:20,  1.96s/it]
                                                        

 65%|██████▌   | 7819/12000 [4:37:35<2:16:20,  1.96s/it]
 65%|██████▌   | 7820/12000 [4:37:37<2:10:16,  1.87s/it]
                                                        

 65%|██████▌   | 7820/12000 [4:37:37<2:10:16,  1.87s/it]
 65%|██████▌   | 7821/12000 [4:37:41<3:02:51,  2.63s/it]
                                                        

 65%|██████▌   | 7821/12000 [4:37:41<3:02:51,  2.63s/it]
 65%|██████▌   | 7822/12000 [4:37:43<2:48:36,  2.42s/it]
                                                        

 65%|██████▌   | 7822/12000 [4:37:43<2:48:36,  2.42s/it]
 65%|██████▌   | 7823/12000 [4:37:45<2:39:12,  2.29s/it]
                                                        

 65%|██████▌   | 7823/12000 [4:37:45<2:39:12,  2.29s/it]
 65%|██████▌   | 7824/12000 [4:37:46<2:26:04,  2.10s/it]
                                                        

 65%|██████▌   | 7824/12000 [4:37:46<2:26:04,  2.10s/it]
 65%|██████▌   | 7825/12000 [4:37:49<2:27:21,  2.12s/it]
                                                        

 65%|██████▌   | 7825/12000 [4:37:49<2:27:21,  2.12s/it]
 65%|██████▌   | 7826/12000 [4:37:50<2:17:05,  1.97s/it]
                                                        

 65%|██████▌   | 7826/12000 [4:37:50<2:17:05,  1.97s/it]
 65%|██████▌   | 7827/12000 [4:37:53<2:40:32,  2.31s/it]
                                                        

 65%|██████▌   | 7827/12000 [4:37:53<2:40:32,  2.31s/it]
 65%|██████▌   | 7828/12000 [4:37:55<2:27:55,  2.13s/it]
                                                        

 65%|██████▌   | 7828/12000 [4:37:55<2:27:55,  2.13s/it]
 65%|██████▌   | 7829/12000 [4:37:57<2:15:48,  1.95s/it]
                                                        

 65%|██████▌   | 7829/12000 [4:37:57<2:15:48,  1.95s/it]
 65%|██████▌   | 7830/12000 [4:37:59<2:31:16,  2.18s/it]
                                                        

 65%|██████▌   | 7830/12000 [4:37:59<2:31:16,  2.18s/it]
 65%|██████▌   | 7831/12000 [4:38:01<2:20:00,  2.02s/it]
                                                        

 65%|██████▌   | 7831/12000 [4:38:01<2:20:00,  2.02s/it]
 65%|██████▌   | 7832/12000 [4:38:04<2:37:55,  2.27s/it]
                                                        

 65%|██████▌   | 7832/12000 [4:38:04<2:37:55,  2.27s/it]
 65%|██████▌   | 7833/12000 [4:38:05<2:20:06,  2.02s/it]
                                                        

 65%|██████▌   | 7833/12000 [4:38:05<2:20:06,  2.02s/it]
 65%|██████▌   | 7834/12000 [4:38:07<2:18:25,  1.99s/it]
                                                        

 65%|██████▌   | 7834/12000 [4:38:07<2:18:25,  1.99s/it]
 65%|██████▌   | 7835/12000 [4:38:09<2:09:50,  1.87s/it]
                                                        

 65%|██████▌   | 7835/12000 [4:38:09<2:09:50,  1.87s/it]
 65%|██████▌   | 7836/12000 [4:38:11<2:08:54,  1.86s/it]
                                                        

 65%|██████▌   | 7836/12000 [4:38:11<2:08:54,  1.86s/it]
 65%|██████▌   | 7837/12000 [4:38:13<2:25:17,  2.09s/it]
                                                        

 65%|██████▌   | 7837/12000 [4:38:13<2:25:17,  2.09s/it]
 65%|██████▌   | 7838/12000 [4:38:15<2:20:37,  2.03s/it]
                                                        

 65%|██████▌   | 7838/12000 [4:38:15<2:20:37,  2.03s/it]
 65%|██████▌   | 7839/12000 [4:38:17<2:28:08,  2.14s/it]
                                                        

 65%|██████▌   | 7839/12000 [4:38:17<2:28:08,  2.14s/it]
 65%|██████▌   | 7840/12000 [4:38:19<2:16:28,  1.97s/it]
                                                        

 65%|██████▌   | 7840/12000 [4:38:19<2:16:28,  1.97s/it]
 65%|██████▌   | 7841/12000 [4:38:21<2:07:58,  1.85s/it]
                                                        

 65%|██████▌   | 7841/12000 [4:38:21<2:07:58,  1.85s/it]
 65%|██████▌   | 7842/12000 [4:38:22<2:00:07,  1.73s/it]
                                                        

 65%|██████▌   | 7842/12000 [4:38:22<2:00:07,  1.73s/it]
 65%|██████▌   | 7843/12000 [4:38:26<2:35:08,  2.24s/it]
                                                        

 65%|██████▌   | 7843/12000 [4:38:26<2:35:08,  2.24s/it]
 65%|██████▌   | 7844/12000 [4:38:28<2:46:59,  2.41s/it]
                                                        

 65%|██████▌   | 7844/12000 [4:38:28<2:46:59,  2.41s/it]
 65%|██████▌   | 7845/12000 [4:38:30<2:30:00,  2.17s/it]
                                                        

 65%|██████▌   | 7845/12000 [4:38:30<2:30:00,  2.17s/it]
 65%|██████▌   | 7846/12000 [4:38:32<2:17:47,  1.99s/it]
                                                        

 65%|██████▌   | 7846/12000 [4:38:32<2:17:47,  1.99s/it]
 65%|██████▌   | 7847/12000 [4:38:33<2:08:23,  1.85s/it]
                                                        

 65%|██████▌   | 7847/12000 [4:38:33<2:08:23,  1.85s/it]
 65%|██████▌   | 7848/12000 [4:38:35<2:02:11,  1.77s/it]
                                                        

 65%|██████▌   | 7848/12000 [4:38:35<2:02:11,  1.77s/it]
 65%|██████▌   | 7849/12000 [4:38:38<2:39:51,  2.31s/it]
                                                        

 65%|██████▌   | 7849/12000 [4:38:38<2:39:51,  2.31s/it]
 65%|██████▌   | 7850/12000 [4:38:40<2:22:07,  2.05s/it]
                                                        

 65%|██████▌   | 7850/12000 [4:38:40<2:22:07,  2.05s/it]
 65%|██████▌   | 7851/12000 [4:38:45<3:21:20,  2.91s/it]
                                                        

 65%|██████▌   | 7851/12000 [4:38:45<3:21:20,  2.91s/it]
 65%|██████▌   | 7852/12000 [4:38:46<2:55:15,  2.54s/it]
                                                        

 65%|██████▌   | 7852/12000 [4:38:46<2:55:15,  2.54s/it]
 65%|██████▌   | 7853/12000 [4:38:48<2:32:26,  2.21s/it]
                                                        
{'loss': '0.8043', 'grad_norm': '0.3889', 'learning_rate': '5.97e-05', 'entropy': '0.7639', 'num_tokens': '4.213e+07', 'mean_token_accuracy': '0.8047', 'epoch': '0.06908'}
{'loss': '0.9735', 'grad_norm': '0.3419', 'learning_rate': '5.968e-05', 'entropy': '0.8667', 'num_tokens': '4.213e+07', 'mean_token_accuracy': '0.7967', 'epoch': '0.06909'}
{'loss': '0.8393', 'grad_norm': '0.4178', 'learning_rate': '5.965e-05', 'entropy': '0.8176', 'num_tokens': '4.214e+07', 'mean_token_accuracy': '0.7834', 'epoch': '0.0691'}
{'loss': '0.5849', 'grad_norm': '0.3041', 'learning_rate': '5.963e-05', 'entropy': '0.5536', 'num_tokens': '4.214e+07', 'mean_token_accuracy': '0.8419', 'epoch': '0.06911'}
{'loss': '1.052', 'grad_norm': '0.3497', 'learning_rate': '5.96e-05', 'entropy': '1.029', 'num_tokens': '4.215e+07', 'mean_token_accuracy': '0.7468', 'epoch': '0.06912'}
{'loss': '0.6693', 'grad_norm': '0.4599', 'learning_rate': '5.958e-05', 'entropy': '0.683', 'num_tokens': '4.215e+07', 'mean_token_accuracy': '0.8147', 'epoch': '0.06913'}
{'loss': '0.8401', 'grad_norm': '0.3367', 'learning_rate': '5.955e-05', 'entropy': '0.8322', 'num_tokens': '4.216e+07', 'mean_token_accuracy': '0.785', 'epoch': '0.06913'}
{'loss': '0.6476', 'grad_norm': '0.4532', 'learning_rate': '5.953e-05', 'entropy': '0.6755', 'num_tokens': '4.216e+07', 'mean_token_accuracy': '0.8109', 'epoch': '0.06914'}
{'loss': '1.047', 'grad_norm': '0.4213', 'learning_rate': '5.95e-05', 'entropy': '1.01', 'num_tokens': '4.217e+07', 'mean_token_accuracy': '0.7523', 'epoch': '0.06915'}
{'loss': '0.7507', 'grad_norm': '0.3763', 'learning_rate': '5.948e-05', 'entropy': '0.8119', 'num_tokens': '4.218e+07', 'mean_token_accuracy': '0.8057', 'epoch': '0.06916'}
{'loss': '1.036', 'grad_norm': '0.4098', 'learning_rate': '5.945e-05', 'entropy': '0.9996', 'num_tokens': '4.218e+07', 'mean_token_accuracy': '0.746', 'epoch': '0.06917'}
{'loss': '1.027', 'grad_norm': '0.4912', 'learning_rate': '5.943e-05', 'entropy': '0.9507', 'num_tokens': '4.219e+07', 'mean_token_accuracy': '0.7666', 'epoch': '0.06918'}
{'loss': '0.7556', 'grad_norm': '0.3934', 'learning_rate': '5.94e-05', 'entropy': '0.7173', 'num_tokens': '4.219e+07', 'mean_token_accuracy': '0.8075', 'epoch': '0.06919'}
{'loss': '0.6064', 'grad_norm': '0.4298', 'learning_rate': '5.938e-05', 'entropy': '0.6554', 'num_tokens': '4.22e+07', 'mean_token_accuracy': '0.8359', 'epoch': '0.0692'}
{'loss': '1.007', 'grad_norm': '0.5233', 'learning_rate': '5.935e-05', 'entropy': '1.069', 'num_tokens': '4.22e+07', 'mean_token_accuracy': '0.753', 'epoch': '0.06921'}
{'loss': '0.7777', 'grad_norm': '0.2988', 'learning_rate': '5.933e-05', 'entropy': '0.8235', 'num_tokens': '4.221e+07', 'mean_token_accuracy': '0.8088', 'epoch': '0.06921'}
{'loss': '0.8317', 'grad_norm': '0.3956', 'learning_rate': '5.93e-05', 'entropy': '0.7823', 'num_tokens': '4.221e+07', 'mean_token_accuracy': '0.7946', 'epoch': '0.06922'}
{'loss': '0.6889', 'grad_norm': '0.3652', 'learning_rate': '5.928e-05', 'entropy': '0.6662', 'num_tokens': '4.222e+07', 'mean_token_accuracy': '0.8257', 'epoch': '0.06923'}
{'loss': '0.6144', 'grad_norm': '0.377', 'learning_rate': '5.925e-05', 'entropy': '0.6177', 'num_tokens': '4.222e+07', 'mean_token_accuracy': '0.8326', 'epoch': '0.06924'}
{'loss': '1.382', 'grad_norm': '0.4905', 'learning_rate': '5.923e-05', 'entropy': '1.248', 'num_tokens': '4.223e+07', 'mean_token_accuracy': '0.6931', 'epoch': '0.06925'}
{'loss': '0.6304', 'grad_norm': '0.3568', 'learning_rate': '5.92e-05', 'entropy': '0.6504', 'num_tokens': '4.223e+07', 'mean_token_accuracy': '0.8259', 'epoch': '0.06926'}
{'loss': '0.6103', 'grad_norm': '0.3314', 'learning_rate': '5.918e-05', 'entropy': '0.7002', 'num_tokens': '4.224e+07', 'mean_token_accuracy': '0.8323', 'epoch': '0.06927'}
{'loss': '0.8268', 'grad_norm': '0.4382', 'learning_rate': '5.915e-05', 'entropy': '0.8576', 'num_tokens': '4.225e+07', 'mean_token_accuracy': '0.7875', 'epoch': '0.06928'}
{'loss': '0.6628', 'grad_norm': '0.3643', 'learning_rate': '5.913e-05', 'entropy': '0.6876', 'num_tokens': '4.225e+07', 'mean_token_accuracy': '0.8202', 'epoch': '0.06929'}
{'loss': '0.8628', 'grad_norm': '0.3568', 'learning_rate': '5.91e-05', 'entropy': '0.9288', 'num_tokens': '4.226e+07', 'mean_token_accuracy': '0.7599', 'epoch': '0.06929'}
{'loss': '0.9364', 'grad_norm': '0.4789', 'learning_rate': '5.907e-05', 'entropy': '0.9427', 'num_tokens': '4.226e+07', 'mean_token_accuracy': '0.7896', 'epoch': '0.0693'}
{'loss': '1.079', 'grad_norm': '0.484', 'learning_rate': '5.905e-05', 'entropy': '0.9394', 'num_tokens': '4.227e+07', 'mean_token_accuracy': '0.7742', 'epoch': '0.06931'}
{'loss': '0.8411', 'grad_norm': '0.4098', 'learning_rate': '5.902e-05', 'entropy': '0.8689', 'num_tokens': '4.227e+07', 'mean_token_accuracy': '0.7855', 'epoch': '0.06932'}
{'loss': '1.146', 'grad_norm': '0.4743', 'learning_rate': '5.9e-05', 'entropy': '0.9335', 'num_tokens': '4.228e+07', 'mean_token_accuracy': '0.7894', 'epoch': '0.06933'}
{'loss': '1.053', 'grad_norm': '0.4838', 'learning_rate': '5.897e-05', 'entropy': '1.033', 'num_tokens': '4.228e+07', 'mean_token_accuracy': '0.7494', 'epoch': '0.06934'}
{'loss': '0.8257', 'grad_norm': '0.425', 'learning_rate': '5.895e-05', 'entropy': '0.8757', 'num_tokens': '4.229e+07', 'mean_token_accuracy': '0.7851', 'epoch': '0.06935'}
{'loss': '0.8231', 'grad_norm': '0.3615', 'learning_rate': '5.892e-05', 'entropy': '0.7883', 'num_tokens': '4.229e+07', 'mean_token_accuracy': '0.7925', 'epoch': '0.06936'}
{'loss': '0.5881', 'grad_norm': '0.3699', 'learning_rate': '5.89e-05', 'entropy': '0.5733', 'num_tokens': '4.23e+07', 'mean_token_accuracy': '0.8433', 'epoch': '0.06936'}
{'loss': '1.246', 'grad_norm': '0.4593', 'learning_rate': '5.887e-05', 'entropy': '1.338', 'num_tokens': '4.23e+07', 'mean_token_accuracy': '0.689', 'epoch': '0.06937'}
{'loss': '0.873', 'grad_norm': '0.4409', 'learning_rate': '5.885e-05', 'entropy': '0.9027', 'num_tokens': '4.231e+07', 'mean_token_accuracy': '0.7839', 'epoch': '0.06938'}
{'loss': '0.7391', 'grad_norm': '0.3504', 'learning_rate': '5.882e-05', 'entropy': '0.6842', 'num_tokens': '4.231e+07', 'mean_token_accuracy': '0.8256', 'epoch': '0.06939'}
{'loss': '0.9173', 'grad_norm': '0.3891', 'learning_rate': '5.88e-05', 'entropy': '0.9164', 'num_tokens': '4.232e+07', 'mean_token_accuracy': '0.7645', 'epoch': '0.0694'}
{'loss': '0.6082', 'grad_norm': '0.3723', 'learning_rate': '5.877e-05', 'entropy': '0.6637', 'num_tokens': '4.232e+07', 'mean_token_accuracy': '0.8235', 'epoch': '0.06941'}
{'loss': '0.6707', 'grad_norm': '0.381', 'learning_rate': '5.875e-05', 'entropy': '0.6864', 'num_tokens': '4.233e+07', 'mean_token_accuracy': '0.8237', 'epoch': '0.06942'}
{'loss': '0.7471', 'grad_norm': '0.4149', 'learning_rate': '5.872e-05', 'entropy': '0.7212', 'num_tokens': '4.233e+07', 'mean_token_accuracy': '0.809', 'epoch': '0.06943'}
{'loss': '0.7289', 'grad_norm': '0.3949', 'learning_rate': '5.87e-05', 'entropy': '0.7503', 'num_tokens': '4.234e+07', 'mean_token_accuracy': '0.8017', 'epoch': '0.06944'}
{'loss': '0.6929', 'grad_norm': '0.5207', 'learning_rate': '5.867e-05', 'entropy': '0.6699', 'num_tokens': '4.234e+07', 'mean_token_accuracy': '0.8191', 'epoch': '0.06944'}
{'loss': '0.9151', 'grad_norm': '0.3771', 'learning_rate': '5.865e-05', 'entropy': '0.8991', 'num_tokens': '4.235e+07', 'mean_token_accuracy': '0.7733', 'epoch': '0.06945'}
{'loss': '0.7089', 'grad_norm': '0.309', 'learning_rate': '5.862e-05', 'entropy': '0.7168', 'num_tokens': '4.235e+07', 'mean_token_accuracy': '0.8053', 'epoch': '0.06946'}
{'loss': '0.7548', 'grad_norm': '0.3615', 'learning_rate': '5.86e-05', 'entropy': '0.748', 'num_tokens': '4.236e+07', 'mean_token_accuracy': '0.8023', 'epoch': '0.06947'}
{'loss': '0.7848', 'grad_norm': '0.3346', 'learning_rate': '5.857e-05', 'entropy': '0.8475', 'num_tokens': '4.237e+07', 'mean_token_accuracy': '0.7939', 'epoch': '0.06948'}
{'loss': '1.348', 'grad_norm': '0.5214', 'learning_rate': '5.855e-05', 'entropy': '1.387', 'num_tokens': '4.237e+07', 'mean_token_accuracy': '0.6705', 'epoch': '0.06949'}

 65%|██████▌   | 7853/12000 [4:38:48<2:32:26,  2.21s/it]
 65%|██████▌   | 7854/12000 [4:38:49<2:18:16,  2.00s/it]
                                                        

 65%|██████▌   | 7854/12000 [4:38:49<2:18:16,  2.00s/it]
 65%|██████▌   | 7855/12000 [4:38:51<2:24:20,  2.09s/it]
                                                        

 65%|██████▌   | 7855/12000 [4:38:51<2:24:20,  2.09s/it]
 65%|██████▌   | 7856/12000 [4:38:53<2:22:19,  2.06s/it]
                                                        

 65%|██████▌   | 7856/12000 [4:38:53<2:22:19,  2.06s/it]
 65%|██████▌   | 7857/12000 [4:38:55<2:16:12,  1.97s/it]
                                                        

 65%|██████▌   | 7857/12000 [4:38:55<2:16:12,  1.97s/it]
 65%|██████▌   | 7858/12000 [4:39:00<3:12:39,  2.79s/it]
                                                        

 65%|██████▌   | 7858/12000 [4:39:00<3:12:39,  2.79s/it]
 65%|██████▌   | 7859/12000 [4:39:02<2:59:16,  2.60s/it]
                                                        

 65%|██████▌   | 7859/12000 [4:39:02<2:59:16,  2.60s/it]
 66%|██████▌   | 7860/12000 [4:39:04<2:36:25,  2.27s/it]
                                                        

 66%|██████▌   | 7860/12000 [4:39:04<2:36:25,  2.27s/it]
 66%|██████▌   | 7861/12000 [4:39:05<2:20:46,  2.04s/it]
                                                        

 66%|██████▌   | 7861/12000 [4:39:05<2:20:46,  2.04s/it]
 66%|██████▌   | 7862/12000 [4:39:07<2:10:01,  1.89s/it]
                                                        

 66%|██████▌   | 7862/12000 [4:39:07<2:10:01,  1.89s/it]
 66%|██████▌   | 7863/12000 [4:39:10<2:36:35,  2.27s/it]
                                                        

 66%|██████▌   | 7863/12000 [4:39:10<2:36:35,  2.27s/it]
 66%|██████▌   | 7864/12000 [4:39:11<2:17:51,  2.00s/it]
                                                        

 66%|██████▌   | 7864/12000 [4:39:11<2:17:51,  2.00s/it]
 66%|██████▌   | 7865/12000 [4:39:16<3:15:24,  2.84s/it]
                                                        

 66%|██████▌   | 7865/12000 [4:39:16<3:15:24,  2.84s/it]
 66%|██████▌   | 7866/12000 [4:39:18<2:49:26,  2.46s/it]
                                                        

 66%|██████▌   | 7866/12000 [4:39:18<2:49:26,  2.46s/it]
 66%|██████▌   | 7867/12000 [4:39:20<2:44:14,  2.38s/it]
                                                        

 66%|██████▌   | 7867/12000 [4:39:20<2:44:14,  2.38s/it]
 66%|██████▌   | 7868/12000 [4:39:21<2:30:21,  2.18s/it]
                                                        

 66%|██████▌   | 7868/12000 [4:39:21<2:30:21,  2.18s/it]
 66%|██████▌   | 7869/12000 [4:39:24<2:29:16,  2.17s/it]
                                                        

 66%|██████▌   | 7869/12000 [4:39:24<2:29:16,  2.17s/it]
 66%|██████▌   | 7870/12000 [4:39:25<2:17:07,  1.99s/it]
                                                        

 66%|██████▌   | 7870/12000 [4:39:25<2:17:07,  1.99s/it]
 66%|██████▌   | 7871/12000 [4:39:27<2:21:34,  2.06s/it]
                                                        

 66%|██████▌   | 7871/12000 [4:39:27<2:21:34,  2.06s/it]
 66%|██████▌   | 7872/12000 [4:39:29<2:09:29,  1.88s/it]
                                                        

 66%|██████▌   | 7872/12000 [4:39:29<2:09:29,  1.88s/it]
 66%|██████▌   | 7873/12000 [4:39:30<2:00:09,  1.75s/it]
                                                        

 66%|██████▌   | 7873/12000 [4:39:30<2:00:09,  1.75s/it]
 66%|██████▌   | 7874/12000 [4:39:33<2:30:25,  2.19s/it]
                                                        

 66%|██████▌   | 7874/12000 [4:39:33<2:30:25,  2.19s/it]
 66%|██████▌   | 7875/12000 [4:39:35<2:20:18,  2.04s/it]
                                                        

 66%|██████▌   | 7875/12000 [4:39:35<2:20:18,  2.04s/it]
 66%|██████▌   | 7876/12000 [4:39:37<2:12:22,  1.93s/it]
                                                        

 66%|██████▌   | 7876/12000 [4:39:37<2:12:22,  1.93s/it]
 66%|██████▌   | 7877/12000 [4:39:39<2:10:02,  1.89s/it]
                                                        

 66%|██████▌   | 7877/12000 [4:39:39<2:10:02,  1.89s/it]
 66%|██████▌   | 7878/12000 [4:39:40<2:05:42,  1.83s/it]
                                                        

 66%|██████▌   | 7878/12000 [4:39:40<2:05:42,  1.83s/it]
 66%|██████▌   | 7879/12000 [4:39:42<2:03:58,  1.81s/it]
                                                        

 66%|██████▌   | 7879/12000 [4:39:42<2:03:58,  1.81s/it]
 66%|██████▌   | 7880/12000 [4:39:45<2:23:04,  2.08s/it]
                                                        

 66%|██████▌   | 7880/12000 [4:39:45<2:23:04,  2.08s/it]
 66%|██████▌   | 7881/12000 [4:39:47<2:24:37,  2.11s/it]
                                                        

 66%|██████▌   | 7881/12000 [4:39:47<2:24:37,  2.11s/it]
 66%|██████▌   | 7882/12000 [4:39:49<2:18:54,  2.02s/it]
                                                        

 66%|██████▌   | 7882/12000 [4:39:49<2:18:54,  2.02s/it]
 66%|██████▌   | 7883/12000 [4:39:51<2:13:41,  1.95s/it]
                                                        

 66%|██████▌   | 7883/12000 [4:39:51<2:13:41,  1.95s/it]
 66%|██████▌   | 7884/12000 [4:39:53<2:31:24,  2.21s/it]
                                                        

 66%|██████▌   | 7884/12000 [4:39:53<2:31:24,  2.21s/it]
 66%|██████▌   | 7885/12000 [4:39:55<2:28:30,  2.17s/it]
                                                        

 66%|██████▌   | 7885/12000 [4:39:55<2:28:30,  2.17s/it]
 66%|██████▌   | 7886/12000 [4:39:57<2:21:32,  2.06s/it]
                                                        

 66%|██████▌   | 7886/12000 [4:39:57<2:21:32,  2.06s/it]
 66%|██████▌   | 7887/12000 [4:40:02<3:05:57,  2.71s/it]
                                                        

 66%|██████▌   | 7887/12000 [4:40:02<3:05:57,  2.71s/it]
 66%|██████▌   | 7888/12000 [4:40:03<2:49:35,  2.47s/it]
                                                        

 66%|██████▌   | 7888/12000 [4:40:03<2:49:35,  2.47s/it]
 66%|██████▌   | 7889/12000 [4:40:05<2:35:58,  2.28s/it]
                                                        

 66%|██████▌   | 7889/12000 [4:40:05<2:35:58,  2.28s/it]
 66%|██████▌   | 7890/12000 [4:40:07<2:28:48,  2.17s/it]
                                                        

 66%|██████▌   | 7890/12000 [4:40:07<2:28:48,  2.17s/it]
 66%|██████▌   | 7891/12000 [4:40:09<2:19:39,  2.04s/it]
                                                        

 66%|██████▌   | 7891/12000 [4:40:09<2:19:39,  2.04s/it]
 66%|██████▌   | 7892/12000 [4:40:12<2:43:11,  2.38s/it]
                                                        

 66%|██████▌   | 7892/12000 [4:40:12<2:43:11,  2.38s/it]
 66%|██████▌   | 7893/12000 [4:40:14<2:25:27,  2.13s/it]
                                                        

 66%|██████▌   | 7893/12000 [4:40:14<2:25:27,  2.13s/it]
 66%|██████▌   | 7894/12000 [4:40:16<2:25:57,  2.13s/it]
                                                        

 66%|██████▌   | 7894/12000 [4:40:16<2:25:57,  2.13s/it]
 66%|██████▌   | 7895/12000 [4:40:18<2:17:55,  2.02s/it]
                                                        

 66%|██████▌   | 7895/12000 [4:40:18<2:17:55,  2.02s/it]
 66%|██████▌   | 7896/12000 [4:40:19<2:15:33,  1.98s/it]
                                                        

 66%|██████▌   | 7896/12000 [4:40:19<2:15:33,  1.98s/it]
 66%|██████▌   | 7897/12000 [4:40:21<2:10:40,  1.91s/it]
                                                        

 66%|██████▌   | 7897/12000 [4:40:21<2:10:40,  1.91s/it]
 66%|██████▌   | 7898/12000 [4:40:23<2:16:18,  1.99s/it]
                                                        

 66%|██████▌   | 7898/12000 [4:40:23<2:16:18,  1.99s/it]
 66%|██████▌   | 7899/12000 [4:40:26<2:24:15,  2.11s/it]
                                                        

 66%|██████▌   | 7899/12000 [4:40:26<2:24:15,  2.11s/it]
 66%|██████▌   | 7900/12000 [4:40:27<2:10:22,  1.91s/it]
                                                        
{'loss': '0.5956', 'grad_norm': '0.3065', 'learning_rate': '5.852e-05', 'entropy': '0.604', 'num_tokens': '4.237e+07', 'mean_token_accuracy': '0.8423', 'epoch': '0.0695'}
{'loss': '0.9087', 'grad_norm': '0.4155', 'learning_rate': '5.85e-05', 'entropy': '0.9695', 'num_tokens': '4.238e+07', 'mean_token_accuracy': '0.7594', 'epoch': '0.06951'}
{'loss': '1.031', 'grad_norm': '0.4684', 'learning_rate': '5.847e-05', 'entropy': '0.8728', 'num_tokens': '4.238e+07', 'mean_token_accuracy': '0.7707', 'epoch': '0.06952'}
{'loss': '0.9209', 'grad_norm': '0.5843', 'learning_rate': '5.845e-05', 'entropy': '0.8704', 'num_tokens': '4.239e+07', 'mean_token_accuracy': '0.772', 'epoch': '0.06952'}
{'loss': '1.048', 'grad_norm': '0.4486', 'learning_rate': '5.842e-05', 'entropy': '0.9384', 'num_tokens': '4.239e+07', 'mean_token_accuracy': '0.7733', 'epoch': '0.06953'}
{'loss': '1.379', 'grad_norm': '0.5139', 'learning_rate': '5.84e-05', 'entropy': '1.062', 'num_tokens': '4.24e+07', 'mean_token_accuracy': '0.7527', 'epoch': '0.06954'}
{'loss': '0.5852', 'grad_norm': '0.3167', 'learning_rate': '5.837e-05', 'entropy': '0.6462', 'num_tokens': '4.241e+07', 'mean_token_accuracy': '0.8311', 'epoch': '0.06955'}
{'loss': '1.079', 'grad_norm': '0.546', 'learning_rate': '5.835e-05', 'entropy': '1.05', 'num_tokens': '4.241e+07', 'mean_token_accuracy': '0.7519', 'epoch': '0.06956'}
{'loss': '1.07', 'grad_norm': '0.5495', 'learning_rate': '5.832e-05', 'entropy': '1.133', 'num_tokens': '4.242e+07', 'mean_token_accuracy': '0.7229', 'epoch': '0.06957'}
{'loss': '0.8326', 'grad_norm': '0.4862', 'learning_rate': '5.83e-05', 'entropy': '0.8587', 'num_tokens': '4.242e+07', 'mean_token_accuracy': '0.7776', 'epoch': '0.06958'}
{'loss': '0.5951', 'grad_norm': '0.2759', 'learning_rate': '5.827e-05', 'entropy': '0.6785', 'num_tokens': '4.243e+07', 'mean_token_accuracy': '0.8289', 'epoch': '0.06959'}
{'loss': '0.7409', 'grad_norm': '0.4291', 'learning_rate': '5.825e-05', 'entropy': '0.8271', 'num_tokens': '4.243e+07', 'mean_token_accuracy': '0.7885', 'epoch': '0.0696'}
{'loss': '1.102', 'grad_norm': '0.4455', 'learning_rate': '5.822e-05', 'entropy': '1.133', 'num_tokens': '4.244e+07', 'mean_token_accuracy': '0.738', 'epoch': '0.0696'}
{'loss': '0.6885', 'grad_norm': '0.369', 'learning_rate': '5.82e-05', 'entropy': '0.661', 'num_tokens': '4.244e+07', 'mean_token_accuracy': '0.8272', 'epoch': '0.06961'}
{'loss': '0.7634', 'grad_norm': '0.4544', 'learning_rate': '5.817e-05', 'entropy': '0.7143', 'num_tokens': '4.245e+07', 'mean_token_accuracy': '0.8076', 'epoch': '0.06962'}
{'loss': '0.937', 'grad_norm': '0.3955', 'learning_rate': '5.815e-05', 'entropy': '0.9082', 'num_tokens': '4.245e+07', 'mean_token_accuracy': '0.7648', 'epoch': '0.06963'}
{'loss': '0.572', 'grad_norm': '0.4427', 'learning_rate': '5.812e-05', 'entropy': '0.6992', 'num_tokens': '4.246e+07', 'mean_token_accuracy': '0.8179', 'epoch': '0.06964'}
{'loss': '0.5629', 'grad_norm': '0.3911', 'learning_rate': '5.81e-05', 'entropy': '0.5852', 'num_tokens': '4.246e+07', 'mean_token_accuracy': '0.8468', 'epoch': '0.06965'}
{'loss': '1.296', 'grad_norm': '0.5468', 'learning_rate': '5.807e-05', 'entropy': '1.161', 'num_tokens': '4.247e+07', 'mean_token_accuracy': '0.7108', 'epoch': '0.06966'}
{'loss': '0.7867', 'grad_norm': '0.3634', 'learning_rate': '5.805e-05', 'entropy': '0.8386', 'num_tokens': '4.247e+07', 'mean_token_accuracy': '0.7937', 'epoch': '0.06967'}
{'loss': '0.6033', 'grad_norm': '0.335', 'learning_rate': '5.802e-05', 'entropy': '0.6279', 'num_tokens': '4.248e+07', 'mean_token_accuracy': '0.84', 'epoch': '0.06967'}
{'loss': '0.6437', 'grad_norm': '0.3748', 'learning_rate': '5.8e-05', 'entropy': '0.6942', 'num_tokens': '4.248e+07', 'mean_token_accuracy': '0.8186', 'epoch': '0.06968'}
{'loss': '0.9463', 'grad_norm': '0.484', 'learning_rate': '5.797e-05', 'entropy': '0.9119', 'num_tokens': '4.249e+07', 'mean_token_accuracy': '0.7699', 'epoch': '0.06969'}
{'loss': '0.9295', 'grad_norm': '0.4858', 'learning_rate': '5.795e-05', 'entropy': '0.9662', 'num_tokens': '4.249e+07', 'mean_token_accuracy': '0.7505', 'epoch': '0.0697'}
{'loss': '0.5942', 'grad_norm': '0.3743', 'learning_rate': '5.792e-05', 'entropy': '0.7042', 'num_tokens': '4.25e+07', 'mean_token_accuracy': '0.8275', 'epoch': '0.06971'}
{'loss': '0.895', 'grad_norm': '0.4582', 'learning_rate': '5.79e-05', 'entropy': '0.9412', 'num_tokens': '4.25e+07', 'mean_token_accuracy': '0.7655', 'epoch': '0.06972'}
{'loss': '1.041', 'grad_norm': '0.4434', 'learning_rate': '5.787e-05', 'entropy': '0.9905', 'num_tokens': '4.251e+07', 'mean_token_accuracy': '0.7664', 'epoch': '0.06973'}
{'loss': '0.6232', 'grad_norm': '0.3439', 'learning_rate': '5.785e-05', 'entropy': '0.7097', 'num_tokens': '4.251e+07', 'mean_token_accuracy': '0.8115', 'epoch': '0.06974'}
{'loss': '0.8059', 'grad_norm': '0.3838', 'learning_rate': '5.782e-05', 'entropy': '0.7564', 'num_tokens': '4.252e+07', 'mean_token_accuracy': '0.8047', 'epoch': '0.06975'}
{'loss': '0.873', 'grad_norm': '0.4443', 'learning_rate': '5.78e-05', 'entropy': '0.915', 'num_tokens': '4.252e+07', 'mean_token_accuracy': '0.7699', 'epoch': '0.06975'}
{'loss': '0.7987', 'grad_norm': '0.3678', 'learning_rate': '5.777e-05', 'entropy': '0.7607', 'num_tokens': '4.253e+07', 'mean_token_accuracy': '0.8224', 'epoch': '0.06976'}
{'loss': '0.9077', 'grad_norm': '0.4077', 'learning_rate': '5.775e-05', 'entropy': '0.8738', 'num_tokens': '4.253e+07', 'mean_token_accuracy': '0.7794', 'epoch': '0.06977'}
{'loss': '0.7775', 'grad_norm': '0.4075', 'learning_rate': '5.772e-05', 'entropy': '0.8373', 'num_tokens': '4.254e+07', 'mean_token_accuracy': '0.7938', 'epoch': '0.06978'}
{'loss': '0.728', 'grad_norm': '0.3929', 'learning_rate': '5.77e-05', 'entropy': '0.7651', 'num_tokens': '4.254e+07', 'mean_token_accuracy': '0.798', 'epoch': '0.06979'}
{'loss': '0.9178', 'grad_norm': '0.3703', 'learning_rate': '5.767e-05', 'entropy': '0.9672', 'num_tokens': '4.255e+07', 'mean_token_accuracy': '0.7661', 'epoch': '0.0698'}
{'loss': '0.8037', 'grad_norm': '0.3939', 'learning_rate': '5.765e-05', 'entropy': '0.8213', 'num_tokens': '4.255e+07', 'mean_token_accuracy': '0.797', 'epoch': '0.06981'}
{'loss': '0.6619', 'grad_norm': '0.4562', 'learning_rate': '5.762e-05', 'entropy': '0.6672', 'num_tokens': '4.256e+07', 'mean_token_accuracy': '0.8333', 'epoch': '0.06982'}
{'loss': '0.9841', 'grad_norm': '0.4226', 'learning_rate': '5.76e-05', 'entropy': '1.042', 'num_tokens': '4.256e+07', 'mean_token_accuracy': '0.7418', 'epoch': '0.06983'}
{'loss': '1.04', 'grad_norm': '0.426', 'learning_rate': '5.757e-05', 'entropy': '0.9819', 'num_tokens': '4.257e+07', 'mean_token_accuracy': '0.7573', 'epoch': '0.06983'}
{'loss': '0.9543', 'grad_norm': '0.4218', 'learning_rate': '5.755e-05', 'entropy': '0.9761', 'num_tokens': '4.258e+07', 'mean_token_accuracy': '0.7566', 'epoch': '0.06984'}
{'loss': '0.6862', 'grad_norm': '0.3878', 'learning_rate': '5.752e-05', 'entropy': '0.7559', 'num_tokens': '4.258e+07', 'mean_token_accuracy': '0.8076', 'epoch': '0.06985'}
{'loss': '0.7597', 'grad_norm': '0.3888', 'learning_rate': '5.75e-05', 'entropy': '0.7828', 'num_tokens': '4.259e+07', 'mean_token_accuracy': '0.805', 'epoch': '0.06986'}
{'loss': '1.134', 'grad_norm': '0.4371', 'learning_rate': '5.747e-05', 'entropy': '1.053', 'num_tokens': '4.259e+07', 'mean_token_accuracy': '0.736', 'epoch': '0.06987'}
{'loss': '1.466', 'grad_norm': '0.5384', 'learning_rate': '5.745e-05', 'entropy': '1.517', 'num_tokens': '4.259e+07', 'mean_token_accuracy': '0.6434', 'epoch': '0.06988'}
{'loss': '0.819', 'grad_norm': '0.4269', 'learning_rate': '5.742e-05', 'entropy': '0.856', 'num_tokens': '4.26e+07', 'mean_token_accuracy': '0.7752', 'epoch': '0.06989'}
{'loss': '0.9942', 'grad_norm': '0.3794', 'learning_rate': '5.74e-05', 'entropy': '0.9309', 'num_tokens': '4.261e+07', 'mean_token_accuracy': '0.7733', 'epoch': '0.0699'}
{'loss': '0.8722', 'grad_norm': '0.4015', 'learning_rate': '5.737e-05', 'entropy': '0.8545', 'num_tokens': '4.261e+07', 'mean_token_accuracy': '0.786', 'epoch': '0.0699'}

 66%|██████▌   | 7900/12000 [4:40:27<2:10:22,  1.91s/it]
 66%|██████▌   | 7901/12000 [4:40:30<2:34:00,  2.25s/it]
                                                        

 66%|██████▌   | 7901/12000 [4:40:30<2:34:00,  2.25s/it]
 66%|██████▌   | 7902/12000 [4:40:32<2:29:08,  2.18s/it]
                                                        

 66%|██████▌   | 7902/12000 [4:40:32<2:29:08,  2.18s/it]
 66%|██████▌   | 7903/12000 [4:40:34<2:24:12,  2.11s/it]
                                                        

 66%|██████▌   | 7903/12000 [4:40:34<2:24:12,  2.11s/it]
 66%|██████▌   | 7904/12000 [4:40:36<2:10:48,  1.92s/it]
                                                        

 66%|██████▌   | 7904/12000 [4:40:36<2:10:48,  1.92s/it]
 66%|██████▌   | 7905/12000 [4:40:38<2:28:15,  2.17s/it]
                                                        

 66%|██████▌   | 7905/12000 [4:40:38<2:28:15,  2.17s/it]
 66%|██████▌   | 7906/12000 [4:40:40<2:14:26,  1.97s/it]
                                                        

 66%|██████▌   | 7906/12000 [4:40:40<2:14:26,  1.97s/it]
 66%|██████▌   | 7907/12000 [4:40:42<2:09:40,  1.90s/it]
                                                        

 66%|██████▌   | 7907/12000 [4:40:42<2:09:40,  1.90s/it]
 66%|██████▌   | 7908/12000 [4:40:44<2:18:14,  2.03s/it]
                                                        

 66%|██████▌   | 7908/12000 [4:40:44<2:18:14,  2.03s/it]
 66%|██████▌   | 7909/12000 [4:40:48<2:58:56,  2.62s/it]
                                                        

 66%|██████▌   | 7909/12000 [4:40:48<2:58:56,  2.62s/it]
 66%|██████▌   | 7910/12000 [4:40:50<2:38:55,  2.33s/it]
                                                        

 66%|██████▌   | 7910/12000 [4:40:50<2:38:55,  2.33s/it]
 66%|██████▌   | 7911/12000 [4:40:52<2:43:18,  2.40s/it]
                                                        

 66%|██████▌   | 7911/12000 [4:40:52<2:43:18,  2.40s/it]
 66%|██████▌   | 7912/12000 [4:40:54<2:29:03,  2.19s/it]
                                                        

 66%|██████▌   | 7912/12000 [4:40:54<2:29:03,  2.19s/it]
 66%|██████▌   | 7913/12000 [4:40:56<2:23:45,  2.11s/it]
                                                        

 66%|██████▌   | 7913/12000 [4:40:56<2:23:45,  2.11s/it]
 66%|██████▌   | 7914/12000 [4:40:58<2:15:18,  1.99s/it]
                                                        

 66%|██████▌   | 7914/12000 [4:40:58<2:15:18,  1.99s/it]
 66%|██████▌   | 7915/12000 [4:40:59<2:04:47,  1.83s/it]
                                                        

 66%|██████▌   | 7915/12000 [4:40:59<2:04:47,  1.83s/it]
 66%|██████▌   | 7916/12000 [4:41:01<2:03:56,  1.82s/it]
                                                        

 66%|██████▌   | 7916/12000 [4:41:01<2:03:56,  1.82s/it]
 66%|██████▌   | 7917/12000 [4:41:03<2:09:10,  1.90s/it]
                                                        

 66%|██████▌   | 7917/12000 [4:41:03<2:09:10,  1.90s/it]
 66%|██████▌   | 7918/12000 [4:41:05<2:21:23,  2.08s/it]
                                                        

 66%|██████▌   | 7918/12000 [4:41:05<2:21:23,  2.08s/it]
 66%|██████▌   | 7919/12000 [4:41:07<2:15:20,  1.99s/it]
                                                        

 66%|██████▌   | 7919/12000 [4:41:07<2:15:20,  1.99s/it]
 66%|██████▌   | 7920/12000 [4:41:09<2:07:59,  1.88s/it]
                                                        

 66%|██████▌   | 7920/12000 [4:41:09<2:07:59,  1.88s/it]
 66%|██████▌   | 7921/12000 [4:41:11<2:17:17,  2.02s/it]
                                                        

 66%|██████▌   | 7921/12000 [4:41:11<2:17:17,  2.02s/it]
 66%|██████▌   | 7922/12000 [4:41:13<2:05:34,  1.85s/it]
                                                        

 66%|██████▌   | 7922/12000 [4:41:13<2:05:34,  1.85s/it]
 66%|██████▌   | 7923/12000 [4:41:14<2:00:20,  1.77s/it]
                                                        

 66%|██████▌   | 7923/12000 [4:41:14<2:00:20,  1.77s/it]
 66%|██████▌   | 7924/12000 [4:41:17<2:11:57,  1.94s/it]
                                                        

 66%|██████▌   | 7924/12000 [4:41:17<2:11:57,  1.94s/it]
 66%|██████▌   | 7925/12000 [4:41:18<2:02:13,  1.80s/it]
                                                        

 66%|██████▌   | 7925/12000 [4:41:18<2:02:13,  1.80s/it]
 66%|██████▌   | 7926/12000 [4:41:19<1:54:59,  1.69s/it]
                                                        

 66%|██████▌   | 7926/12000 [4:41:19<1:54:59,  1.69s/it]
 66%|██████▌   | 7927/12000 [4:41:22<2:04:31,  1.83s/it]
                                                        

 66%|██████▌   | 7927/12000 [4:41:22<2:04:31,  1.83s/it]
 66%|██████▌   | 7928/12000 [4:41:23<1:57:38,  1.73s/it]
                                                        

 66%|██████▌   | 7928/12000 [4:41:23<1:57:38,  1.73s/it]
 66%|██████▌   | 7929/12000 [4:41:25<1:52:57,  1.66s/it]
                                                        

 66%|██████▌   | 7929/12000 [4:41:25<1:52:57,  1.66s/it]
 66%|██████▌   | 7930/12000 [4:41:27<1:58:19,  1.74s/it]
                                                        

 66%|██████▌   | 7930/12000 [4:41:27<1:58:19,  1.74s/it]
 66%|██████▌   | 7931/12000 [4:41:31<2:46:50,  2.46s/it]
                                                        

 66%|██████▌   | 7931/12000 [4:41:31<2:46:50,  2.46s/it]
 66%|██████▌   | 7932/12000 [4:41:33<2:37:08,  2.32s/it]
                                                        

 66%|██████▌   | 7932/12000 [4:41:33<2:37:08,  2.32s/it]
 66%|██████▌   | 7933/12000 [4:41:34<2:27:12,  2.17s/it]
                                                        

 66%|██████▌   | 7933/12000 [4:41:34<2:27:12,  2.17s/it]
 66%|██████▌   | 7934/12000 [4:41:36<2:11:53,  1.95s/it]
                                                        

 66%|██████▌   | 7934/12000 [4:41:36<2:11:53,  1.95s/it]
 66%|██████▌   | 7935/12000 [4:41:37<2:05:11,  1.85s/it]
                                                        

 66%|██████▌   | 7935/12000 [4:41:37<2:05:11,  1.85s/it]
 66%|██████▌   | 7936/12000 [4:41:39<1:58:55,  1.76s/it]
                                                        

 66%|██████▌   | 7936/12000 [4:41:39<1:58:55,  1.76s/it]
 66%|██████▌   | 7937/12000 [4:41:41<1:55:44,  1.71s/it]
                                                        

 66%|██████▌   | 7937/12000 [4:41:41<1:55:44,  1.71s/it]
 66%|██████▌   | 7938/12000 [4:41:43<2:02:58,  1.82s/it]
                                                        

 66%|██████▌   | 7938/12000 [4:41:43<2:02:58,  1.82s/it]
 66%|██████▌   | 7939/12000 [4:41:45<2:22:50,  2.11s/it]
                                                        

 66%|██████▌   | 7939/12000 [4:41:45<2:22:50,  2.11s/it]
 66%|██████▌   | 7940/12000 [4:41:47<2:15:22,  2.00s/it]
                                                        

 66%|██████▌   | 7940/12000 [4:41:47<2:15:22,  2.00s/it]
 66%|██████▌   | 7941/12000 [4:41:49<2:08:39,  1.90s/it]
                                                        

 66%|██████▌   | 7941/12000 [4:41:49<2:08:39,  1.90s/it]
 66%|██████▌   | 7942/12000 [4:41:51<2:08:01,  1.89s/it]
                                                        

 66%|██████▌   | 7942/12000 [4:41:51<2:08:01,  1.89s/it]
 66%|██████▌   | 7943/12000 [4:41:52<1:59:49,  1.77s/it]
                                                        

 66%|██████▌   | 7943/12000 [4:41:52<1:59:49,  1.77s/it]
 66%|██████▌   | 7944/12000 [4:41:54<1:53:06,  1.67s/it]
                                                        

 66%|██████▌   | 7944/12000 [4:41:54<1:53:06,  1.67s/it]
 66%|██████▌   | 7945/12000 [4:41:57<2:19:56,  2.07s/it]
                                                        

 66%|██████▌   | 7945/12000 [4:41:57<2:19:56,  2.07s/it]
 66%|██████▌   | 7946/12000 [4:41:58<2:12:20,  1.96s/it]
                                                        

 66%|██████▌   | 7946/12000 [4:41:58<2:12:20,  1.96s/it]
 66%|██████▌   | 7947/12000 [4:42:01<2:18:51,  2.06s/it]
                                                        
{'loss': '0.708', 'grad_norm': '0.3819', 'learning_rate': '5.735e-05', 'entropy': '0.7136', 'num_tokens': '4.262e+07', 'mean_token_accuracy': '0.8072', 'epoch': '0.06991'}
{'loss': '0.9098', 'grad_norm': '0.3717', 'learning_rate': '5.732e-05', 'entropy': '0.9027', 'num_tokens': '4.262e+07', 'mean_token_accuracy': '0.7774', 'epoch': '0.06992'}
{'loss': '0.8268', 'grad_norm': '0.3511', 'learning_rate': '5.73e-05', 'entropy': '0.8263', 'num_tokens': '4.263e+07', 'mean_token_accuracy': '0.7876', 'epoch': '0.06993'}
{'loss': '0.7456', 'grad_norm': '0.4348', 'learning_rate': '5.727e-05', 'entropy': '0.734', 'num_tokens': '4.263e+07', 'mean_token_accuracy': '0.818', 'epoch': '0.06994'}
{'loss': '0.9883', 'grad_norm': '0.386', 'learning_rate': '5.725e-05', 'entropy': '0.972', 'num_tokens': '4.264e+07', 'mean_token_accuracy': '0.7448', 'epoch': '0.06995'}
{'loss': '1.075', 'grad_norm': '0.3611', 'learning_rate': '5.722e-05', 'entropy': '0.9871', 'num_tokens': '4.264e+07', 'mean_token_accuracy': '0.7633', 'epoch': '0.06996'}
{'loss': '0.7572', 'grad_norm': '0.3818', 'learning_rate': '5.72e-05', 'entropy': '0.8042', 'num_tokens': '4.265e+07', 'mean_token_accuracy': '0.8029', 'epoch': '0.06997'}
{'loss': '0.9151', 'grad_norm': '0.4529', 'learning_rate': '5.717e-05', 'entropy': '0.8604', 'num_tokens': '4.265e+07', 'mean_token_accuracy': '0.7733', 'epoch': '0.06998'}
{'loss': '0.8548', 'grad_norm': '0.4423', 'learning_rate': '5.715e-05', 'entropy': '0.8138', 'num_tokens': '4.266e+07', 'mean_token_accuracy': '0.7934', 'epoch': '0.06998'}
{'loss': '1.057', 'grad_norm': '0.5679', 'learning_rate': '5.712e-05', 'entropy': '1', 'num_tokens': '4.266e+07', 'mean_token_accuracy': '0.7491', 'epoch': '0.06999'}
{'loss': '0.7322', 'grad_norm': '0.3234', 'learning_rate': '5.71e-05', 'entropy': '0.7529', 'num_tokens': '4.267e+07', 'mean_token_accuracy': '0.804', 'epoch': '0.07'}
{'loss': '1.187', 'grad_norm': '0.4097', 'learning_rate': '5.707e-05', 'entropy': '1.071', 'num_tokens': '4.267e+07', 'mean_token_accuracy': '0.738', 'epoch': '0.07001'}
{'loss': '0.8543', 'grad_norm': '0.441', 'learning_rate': '5.705e-05', 'entropy': '0.8348', 'num_tokens': '4.268e+07', 'mean_token_accuracy': '0.7878', 'epoch': '0.07002'}
{'loss': '0.7198', 'grad_norm': '0.4193', 'learning_rate': '5.702e-05', 'entropy': '0.742', 'num_tokens': '4.268e+07', 'mean_token_accuracy': '0.8165', 'epoch': '0.07003'}
{'loss': '0.8709', 'grad_norm': '0.482', 'learning_rate': '5.7e-05', 'entropy': '0.875', 'num_tokens': '4.269e+07', 'mean_token_accuracy': '0.7552', 'epoch': '0.07004'}
{'loss': '1.163', 'grad_norm': '0.5568', 'learning_rate': '5.697e-05', 'entropy': '1.243', 'num_tokens': '4.269e+07', 'mean_token_accuracy': '0.7152', 'epoch': '0.07005'}
{'loss': '0.9093', 'grad_norm': '0.4236', 'learning_rate': '5.695e-05', 'entropy': '0.8295', 'num_tokens': '4.27e+07', 'mean_token_accuracy': '0.7909', 'epoch': '0.07006'}
{'loss': '0.8657', 'grad_norm': '0.5097', 'learning_rate': '5.692e-05', 'entropy': '0.8536', 'num_tokens': '4.27e+07', 'mean_token_accuracy': '0.7918', 'epoch': '0.07006'}
{'loss': '1.013', 'grad_norm': '0.4738', 'learning_rate': '5.69e-05', 'entropy': '0.9068', 'num_tokens': '4.271e+07', 'mean_token_accuracy': '0.7769', 'epoch': '0.07007'}
{'loss': '0.9935', 'grad_norm': '0.3796', 'learning_rate': '5.687e-05', 'entropy': '0.9085', 'num_tokens': '4.272e+07', 'mean_token_accuracy': '0.7802', 'epoch': '0.07008'}
{'loss': '0.7053', 'grad_norm': '0.547', 'learning_rate': '5.685e-05', 'entropy': '0.6625', 'num_tokens': '4.272e+07', 'mean_token_accuracy': '0.8243', 'epoch': '0.07009'}
{'loss': '1.02', 'grad_norm': '0.595', 'learning_rate': '5.682e-05', 'entropy': '0.8967', 'num_tokens': '4.272e+07', 'mean_token_accuracy': '0.7566', 'epoch': '0.0701'}
{'loss': '1.016', 'grad_norm': '0.5044', 'learning_rate': '5.68e-05', 'entropy': '1.006', 'num_tokens': '4.273e+07', 'mean_token_accuracy': '0.7452', 'epoch': '0.07011'}
{'loss': '0.6252', 'grad_norm': '0.4397', 'learning_rate': '5.677e-05', 'entropy': '0.6488', 'num_tokens': '4.273e+07', 'mean_token_accuracy': '0.827', 'epoch': '0.07012'}
{'loss': '0.9032', 'grad_norm': '0.3885', 'learning_rate': '5.675e-05', 'entropy': '0.8961', 'num_tokens': '4.274e+07', 'mean_token_accuracy': '0.7795', 'epoch': '0.07013'}
{'loss': '0.8102', 'grad_norm': '0.3968', 'learning_rate': '5.673e-05', 'entropy': '0.8462', 'num_tokens': '4.274e+07', 'mean_token_accuracy': '0.7873', 'epoch': '0.07013'}
{'loss': '0.8643', 'grad_norm': '0.4915', 'learning_rate': '5.67e-05', 'entropy': '0.9011', 'num_tokens': '4.275e+07', 'mean_token_accuracy': '0.7639', 'epoch': '0.07014'}
{'loss': '1.107', 'grad_norm': '0.4707', 'learning_rate': '5.668e-05', 'entropy': '1.174', 'num_tokens': '4.275e+07', 'mean_token_accuracy': '0.723', 'epoch': '0.07015'}
{'loss': '0.8222', 'grad_norm': '0.5182', 'learning_rate': '5.665e-05', 'entropy': '0.8941', 'num_tokens': '4.276e+07', 'mean_token_accuracy': '0.7813', 'epoch': '0.07016'}
{'loss': '0.6132', 'grad_norm': '0.3825', 'learning_rate': '5.663e-05', 'entropy': '0.6465', 'num_tokens': '4.276e+07', 'mean_token_accuracy': '0.8262', 'epoch': '0.07017'}
{'loss': '1.079', 'grad_norm': '0.3866', 'learning_rate': '5.66e-05', 'entropy': '1.098', 'num_tokens': '4.277e+07', 'mean_token_accuracy': '0.7441', 'epoch': '0.07018'}
{'loss': '0.8158', 'grad_norm': '0.379', 'learning_rate': '5.658e-05', 'entropy': '0.8195', 'num_tokens': '4.277e+07', 'mean_token_accuracy': '0.7954', 'epoch': '0.07019'}
{'loss': '0.5', 'grad_norm': '0.2728', 'learning_rate': '5.655e-05', 'entropy': '0.5067', 'num_tokens': '4.278e+07', 'mean_token_accuracy': '0.8695', 'epoch': '0.0702'}
{'loss': '0.8656', 'grad_norm': '0.4035', 'learning_rate': '5.653e-05', 'entropy': '0.8965', 'num_tokens': '4.279e+07', 'mean_token_accuracy': '0.7812', 'epoch': '0.07021'}
{'loss': '0.7924', 'grad_norm': '0.3459', 'learning_rate': '5.65e-05', 'entropy': '0.8081', 'num_tokens': '4.279e+07', 'mean_token_accuracy': '0.7813', 'epoch': '0.07021'}
{'loss': '0.6804', 'grad_norm': '0.3457', 'learning_rate': '5.648e-05', 'entropy': '0.6742', 'num_tokens': '4.28e+07', 'mean_token_accuracy': '0.8247', 'epoch': '0.07022'}
{'loss': '0.9532', 'grad_norm': '0.393', 'learning_rate': '5.645e-05', 'entropy': '0.9376', 'num_tokens': '4.28e+07', 'mean_token_accuracy': '0.7686', 'epoch': '0.07023'}
{'loss': '0.7572', 'grad_norm': '0.3673', 'learning_rate': '5.643e-05', 'entropy': '0.7584', 'num_tokens': '4.281e+07', 'mean_token_accuracy': '0.7993', 'epoch': '0.07024'}
{'loss': '0.7584', 'grad_norm': '0.39', 'learning_rate': '5.64e-05', 'entropy': '0.7406', 'num_tokens': '4.281e+07', 'mean_token_accuracy': '0.8233', 'epoch': '0.07025'}
{'loss': '1.192', 'grad_norm': '0.4482', 'learning_rate': '5.638e-05', 'entropy': '1.106', 'num_tokens': '4.282e+07', 'mean_token_accuracy': '0.728', 'epoch': '0.07026'}
{'loss': '0.811', 'grad_norm': '0.4668', 'learning_rate': '5.635e-05', 'entropy': '0.7835', 'num_tokens': '4.282e+07', 'mean_token_accuracy': '0.7896', 'epoch': '0.07027'}
{'loss': '0.8487', 'grad_norm': '0.4629', 'learning_rate': '5.633e-05', 'entropy': '0.8762', 'num_tokens': '4.283e+07', 'mean_token_accuracy': '0.7721', 'epoch': '0.07028'}
{'loss': '0.67', 'grad_norm': '0.4102', 'learning_rate': '5.63e-05', 'entropy': '0.6388', 'num_tokens': '4.283e+07', 'mean_token_accuracy': '0.8227', 'epoch': '0.07029'}
{'loss': '0.5338', 'grad_norm': '0.3442', 'learning_rate': '5.628e-05', 'entropy': '0.5878', 'num_tokens': '4.284e+07', 'mean_token_accuracy': '0.8577', 'epoch': '0.07029'}
{'loss': '0.7664', 'grad_norm': '0.391', 'learning_rate': '5.625e-05', 'entropy': '0.8092', 'num_tokens': '4.284e+07', 'mean_token_accuracy': '0.794', 'epoch': '0.0703'}
{'loss': '1.052', 'grad_norm': '0.3945', 'learning_rate': '5.623e-05', 'entropy': '1.195', 'num_tokens': '4.285e+07', 'mean_token_accuracy': '0.7163', 'epoch': '0.07031'}
{'loss': '0.8274', 'grad_norm': '0.4159', 'learning_rate': '5.62e-05', 'entropy': '0.8223', 'num_tokens': '4.285e+07', 'mean_token_accuracy': '0.7881', 'epoch': '0.07032'}

 66%|██████▌   | 7947/12000 [4:42:01<2:18:51,  2.06s/it]
 66%|██████▌   | 7948/12000 [4:42:02<2:10:50,  1.94s/it]
                                                        

 66%|██████▌   | 7948/12000 [4:42:02<2:10:50,  1.94s/it]
 66%|██████▌   | 7949/12000 [4:42:04<2:13:24,  1.98s/it]
                                                        

 66%|██████▌   | 7949/12000 [4:42:04<2:13:24,  1.98s/it]
 66%|██████▋   | 7950/12000 [4:42:06<2:12:45,  1.97s/it]
                                                        

 66%|██████▋   | 7950/12000 [4:42:06<2:12:45,  1.97s/it]
 66%|██████▋   | 7951/12000 [4:42:09<2:33:21,  2.27s/it]
                                                        

 66%|██████▋   | 7951/12000 [4:42:09<2:33:21,  2.27s/it]
 66%|██████▋   | 7952/12000 [4:42:12<2:40:36,  2.38s/it]
                                                        

 66%|██████▋   | 7952/12000 [4:42:12<2:40:36,  2.38s/it]
 66%|██████▋   | 7953/12000 [4:42:16<3:16:40,  2.92s/it]
                                                        

 66%|██████▋   | 7953/12000 [4:42:16<3:16:40,  2.92s/it]
 66%|██████▋   | 7954/12000 [4:42:18<2:52:48,  2.56s/it]
                                                        

 66%|██████▋   | 7954/12000 [4:42:18<2:52:48,  2.56s/it]
 66%|██████▋   | 7955/12000 [4:42:20<2:46:26,  2.47s/it]
                                                        

 66%|██████▋   | 7955/12000 [4:42:20<2:46:26,  2.47s/it]
 66%|██████▋   | 7956/12000 [4:42:22<2:38:30,  2.35s/it]
                                                        

 66%|██████▋   | 7956/12000 [4:42:22<2:38:30,  2.35s/it]
 66%|██████▋   | 7957/12000 [4:42:24<2:18:59,  2.06s/it]
                                                        

 66%|██████▋   | 7957/12000 [4:42:24<2:18:59,  2.06s/it]
 66%|██████▋   | 7958/12000 [4:42:25<2:08:53,  1.91s/it]
                                                        

 66%|██████▋   | 7958/12000 [4:42:25<2:08:53,  1.91s/it]
 66%|██████▋   | 7959/12000 [4:42:27<2:03:09,  1.83s/it]
                                                        

 66%|██████▋   | 7959/12000 [4:42:27<2:03:09,  1.83s/it]
 66%|██████▋   | 7960/12000 [4:42:29<2:06:23,  1.88s/it]
                                                        

 66%|██████▋   | 7960/12000 [4:42:29<2:06:23,  1.88s/it]
 66%|██████▋   | 7961/12000 [4:42:32<2:36:28,  2.32s/it]
                                                        

 66%|██████▋   | 7961/12000 [4:42:32<2:36:28,  2.32s/it]
 66%|██████▋   | 7962/12000 [4:42:34<2:17:52,  2.05s/it]
                                                        

 66%|██████▋   | 7962/12000 [4:42:34<2:17:52,  2.05s/it]
 66%|██████▋   | 7963/12000 [4:42:36<2:28:30,  2.21s/it]
                                                        

 66%|██████▋   | 7963/12000 [4:42:36<2:28:30,  2.21s/it]
 66%|██████▋   | 7964/12000 [4:42:38<2:14:16,  2.00s/it]
                                                        

 66%|██████▋   | 7964/12000 [4:42:38<2:14:16,  2.00s/it]
 66%|██████▋   | 7965/12000 [4:42:39<2:02:54,  1.83s/it]
                                                        

 66%|██████▋   | 7965/12000 [4:42:39<2:02:54,  1.83s/it]
 66%|██████▋   | 7966/12000 [4:42:42<2:29:33,  2.22s/it]
                                                        

 66%|██████▋   | 7966/12000 [4:42:42<2:29:33,  2.22s/it]
 66%|██████▋   | 7967/12000 [4:42:44<2:23:59,  2.14s/it]
                                                        

 66%|██████▋   | 7967/12000 [4:42:44<2:23:59,  2.14s/it]
 66%|██████▋   | 7968/12000 [4:42:46<2:10:49,  1.95s/it]
                                                        

 66%|██████▋   | 7968/12000 [4:42:46<2:10:49,  1.95s/it]
 66%|██████▋   | 7969/12000 [4:42:48<2:11:41,  1.96s/it]
                                                        

 66%|██████▋   | 7969/12000 [4:42:48<2:11:41,  1.96s/it]
 66%|██████▋   | 7970/12000 [4:42:49<2:02:01,  1.82s/it]
                                                        

 66%|██████▋   | 7970/12000 [4:42:49<2:02:01,  1.82s/it]
 66%|██████▋   | 7971/12000 [4:42:51<2:03:19,  1.84s/it]
                                                        

 66%|██████▋   | 7971/12000 [4:42:51<2:03:19,  1.84s/it]
 66%|██████▋   | 7972/12000 [4:42:53<1:56:04,  1.73s/it]
                                                        

 66%|██████▋   | 7972/12000 [4:42:53<1:56:04,  1.73s/it]
 66%|██████▋   | 7973/12000 [4:42:55<2:17:29,  2.05s/it]
                                                        

 66%|██████▋   | 7973/12000 [4:42:55<2:17:29,  2.05s/it]
 66%|██████▋   | 7974/12000 [4:43:00<3:09:38,  2.83s/it]
                                                        

 66%|██████▋   | 7974/12000 [4:43:00<3:09:38,  2.83s/it]
 66%|██████▋   | 7975/12000 [4:43:04<3:25:30,  3.06s/it]
                                                        

 66%|██████▋   | 7975/12000 [4:43:04<3:25:30,  3.06s/it]
 66%|██████▋   | 7976/12000 [4:43:06<3:21:43,  3.01s/it]
                                                        

 66%|██████▋   | 7976/12000 [4:43:06<3:21:43,  3.01s/it]
 66%|██████▋   | 7977/12000 [4:43:09<3:04:16,  2.75s/it]
                                                        

 66%|██████▋   | 7977/12000 [4:43:09<3:04:16,  2.75s/it]
 66%|██████▋   | 7978/12000 [4:43:10<2:39:12,  2.38s/it]
                                                        

 66%|██████▋   | 7978/12000 [4:43:10<2:39:12,  2.38s/it]
 66%|██████▋   | 7979/12000 [4:43:12<2:33:57,  2.30s/it]
                                                        

 66%|██████▋   | 7979/12000 [4:43:12<2:33:57,  2.30s/it]
 66%|██████▋   | 7980/12000 [4:43:14<2:23:00,  2.13s/it]
                                                        

 66%|██████▋   | 7980/12000 [4:43:14<2:23:00,  2.13s/it]
 67%|██████▋   | 7981/12000 [4:43:15<2:11:01,  1.96s/it]
                                                        

 67%|██████▋   | 7981/12000 [4:43:15<2:11:01,  1.96s/it]
 67%|██████▋   | 7982/12000 [4:43:17<2:11:37,  1.97s/it]
                                                        

 67%|██████▋   | 7982/12000 [4:43:17<2:11:37,  1.97s/it]
 67%|██████▋   | 7983/12000 [4:43:20<2:15:11,  2.02s/it]
                                                        

 67%|██████▋   | 7983/12000 [4:43:20<2:15:11,  2.02s/it]
 67%|██████▋   | 7984/12000 [4:43:22<2:24:26,  2.16s/it]
                                                        

 67%|██████▋   | 7984/12000 [4:43:22<2:24:26,  2.16s/it]
 67%|██████▋   | 7985/12000 [4:43:25<2:46:12,  2.48s/it]
                                                        

 67%|██████▋   | 7985/12000 [4:43:25<2:46:12,  2.48s/it]
 67%|██████▋   | 7986/12000 [4:43:27<2:37:12,  2.35s/it]
                                                        

 67%|██████▋   | 7986/12000 [4:43:27<2:37:12,  2.35s/it]
 67%|██████▋   | 7987/12000 [4:43:29<2:21:27,  2.12s/it]
                                                        

 67%|██████▋   | 7987/12000 [4:43:29<2:21:27,  2.12s/it]
 67%|██████▋   | 7988/12000 [4:43:31<2:24:42,  2.16s/it]
                                                        

 67%|██████▋   | 7988/12000 [4:43:31<2:24:42,  2.16s/it]
 67%|██████▋   | 7989/12000 [4:43:34<2:32:17,  2.28s/it]
                                                        

 67%|██████▋   | 7989/12000 [4:43:34<2:32:17,  2.28s/it]
 67%|██████▋   | 7990/12000 [4:43:35<2:16:55,  2.05s/it]
                                                        

 67%|██████▋   | 7990/12000 [4:43:35<2:16:55,  2.05s/it]
 67%|██████▋   | 7991/12000 [4:43:39<2:42:47,  2.44s/it]
                                                        

 67%|██████▋   | 7991/12000 [4:43:39<2:42:47,  2.44s/it]
 67%|██████▋   | 7992/12000 [4:43:40<2:28:05,  2.22s/it]
                                                        

 67%|██████▋   | 7992/12000 [4:43:40<2:28:05,  2.22s/it]
 67%|██████▋   | 7993/12000 [4:43:42<2:14:49,  2.02s/it]
                                                        

 67%|██████▋   | 7993/12000 [4:43:42<2:14:49,  2.02s/it]
 67%|██████▋   | 7994/12000 [4:43:44<2:06:35,  1.90s/it]
                                                        
{'loss': '0.9115', 'grad_norm': '0.3932', 'learning_rate': '5.618e-05', 'entropy': '0.8996', 'num_tokens': '4.286e+07', 'mean_token_accuracy': '0.7709', 'epoch': '0.07033'}
{'loss': '1.234', 'grad_norm': '0.6465', 'learning_rate': '5.615e-05', 'entropy': '1.262', 'num_tokens': '4.286e+07', 'mean_token_accuracy': '0.7093', 'epoch': '0.07034'}
{'loss': '0.9013', 'grad_norm': '0.4923', 'learning_rate': '5.613e-05', 'entropy': '0.8588', 'num_tokens': '4.287e+07', 'mean_token_accuracy': '0.7877', 'epoch': '0.07035'}
{'loss': '1.085', 'grad_norm': '0.4395', 'learning_rate': '5.611e-05', 'entropy': '1.068', 'num_tokens': '4.288e+07', 'mean_token_accuracy': '0.7534', 'epoch': '0.07036'}
{'loss': '0.6945', 'grad_norm': '0.3702', 'learning_rate': '5.608e-05', 'entropy': '0.7117', 'num_tokens': '4.288e+07', 'mean_token_accuracy': '0.8298', 'epoch': '0.07037'}
{'loss': '0.7769', 'grad_norm': '0.3892', 'learning_rate': '5.606e-05', 'entropy': '0.8366', 'num_tokens': '4.289e+07', 'mean_token_accuracy': '0.7927', 'epoch': '0.07037'}
{'loss': '0.9766', 'grad_norm': '0.4341', 'learning_rate': '5.603e-05', 'entropy': '0.9403', 'num_tokens': '4.289e+07', 'mean_token_accuracy': '0.7755', 'epoch': '0.07038'}
{'loss': '0.9243', 'grad_norm': '0.4084', 'learning_rate': '5.601e-05', 'entropy': '0.9008', 'num_tokens': '4.29e+07', 'mean_token_accuracy': '0.7789', 'epoch': '0.07039'}
{'loss': '0.8248', 'grad_norm': '0.3442', 'learning_rate': '5.598e-05', 'entropy': '0.8833', 'num_tokens': '4.29e+07', 'mean_token_accuracy': '0.8007', 'epoch': '0.0704'}
{'loss': '1.125', 'grad_norm': '0.3903', 'learning_rate': '5.596e-05', 'entropy': '1.036', 'num_tokens': '4.291e+07', 'mean_token_accuracy': '0.727', 'epoch': '0.07041'}
{'loss': '0.8064', 'grad_norm': '0.4752', 'learning_rate': '5.593e-05', 'entropy': '0.8115', 'num_tokens': '4.291e+07', 'mean_token_accuracy': '0.8025', 'epoch': '0.07042'}
{'loss': '0.7031', 'grad_norm': '0.412', 'learning_rate': '5.591e-05', 'entropy': '0.7255', 'num_tokens': '4.292e+07', 'mean_token_accuracy': '0.8147', 'epoch': '0.07043'}
{'loss': '0.6469', 'grad_norm': '0.4267', 'learning_rate': '5.588e-05', 'entropy': '0.7154', 'num_tokens': '4.292e+07', 'mean_token_accuracy': '0.814', 'epoch': '0.07044'}
{'loss': '0.6167', 'grad_norm': '0.4304', 'learning_rate': '5.586e-05', 'entropy': '0.7162', 'num_tokens': '4.293e+07', 'mean_token_accuracy': '0.8206', 'epoch': '0.07044'}
{'loss': '1.082', 'grad_norm': '0.3134', 'learning_rate': '5.583e-05', 'entropy': '0.967', 'num_tokens': '4.294e+07', 'mean_token_accuracy': '0.727', 'epoch': '0.07045'}
{'loss': '0.9428', 'grad_norm': '0.3995', 'learning_rate': '5.581e-05', 'entropy': '0.9992', 'num_tokens': '4.294e+07', 'mean_token_accuracy': '0.7533', 'epoch': '0.07046'}
{'loss': '1.055', 'grad_norm': '0.4632', 'learning_rate': '5.578e-05', 'entropy': '0.9772', 'num_tokens': '4.295e+07', 'mean_token_accuracy': '0.7493', 'epoch': '0.07047'}
{'loss': '0.8483', 'grad_norm': '0.4006', 'learning_rate': '5.576e-05', 'entropy': '0.8726', 'num_tokens': '4.295e+07', 'mean_token_accuracy': '0.7908', 'epoch': '0.07048'}
{'loss': '0.7948', 'grad_norm': '0.3529', 'learning_rate': '5.573e-05', 'entropy': '0.8453', 'num_tokens': '4.295e+07', 'mean_token_accuracy': '0.7837', 'epoch': '0.07049'}
{'loss': '0.7719', 'grad_norm': '0.3282', 'learning_rate': '5.571e-05', 'entropy': '0.75', 'num_tokens': '4.296e+07', 'mean_token_accuracy': '0.8031', 'epoch': '0.0705'}
{'loss': '1.049', 'grad_norm': '0.4231', 'learning_rate': '5.568e-05', 'entropy': '1.011', 'num_tokens': '4.297e+07', 'mean_token_accuracy': '0.7594', 'epoch': '0.07051'}
{'loss': '0.6634', 'grad_norm': '0.3298', 'learning_rate': '5.566e-05', 'entropy': '0.6527', 'num_tokens': '4.297e+07', 'mean_token_accuracy': '0.8357', 'epoch': '0.07052'}
{'loss': '0.7992', 'grad_norm': '0.3615', 'learning_rate': '5.564e-05', 'entropy': '0.8312', 'num_tokens': '4.298e+07', 'mean_token_accuracy': '0.7906', 'epoch': '0.07052'}
{'loss': '0.8816', 'grad_norm': '0.4718', 'learning_rate': '5.561e-05', 'entropy': '0.8972', 'num_tokens': '4.298e+07', 'mean_token_accuracy': '0.7753', 'epoch': '0.07053'}
{'loss': '0.7982', 'grad_norm': '0.3368', 'learning_rate': '5.559e-05', 'entropy': '0.822', 'num_tokens': '4.299e+07', 'mean_token_accuracy': '0.7893', 'epoch': '0.07054'}
{'loss': '0.9791', 'grad_norm': '0.3704', 'learning_rate': '5.556e-05', 'entropy': '1.012', 'num_tokens': '4.299e+07', 'mean_token_accuracy': '0.7523', 'epoch': '0.07055'}
{'loss': '0.799', 'grad_norm': '0.3856', 'learning_rate': '5.554e-05', 'entropy': '0.7467', 'num_tokens': '4.3e+07', 'mean_token_accuracy': '0.8103', 'epoch': '0.07056'}
{'loss': '1.098', 'grad_norm': '0.4068', 'learning_rate': '5.551e-05', 'entropy': '0.9358', 'num_tokens': '4.301e+07', 'mean_token_accuracy': '0.7774', 'epoch': '0.07057'}
{'loss': '0.6614', 'grad_norm': '0.4059', 'learning_rate': '5.549e-05', 'entropy': '0.6954', 'num_tokens': '4.301e+07', 'mean_token_accuracy': '0.8268', 'epoch': '0.07058'}
{'loss': '0.897', 'grad_norm': '0.4072', 'learning_rate': '5.546e-05', 'entropy': '0.8007', 'num_tokens': '4.302e+07', 'mean_token_accuracy': '0.7929', 'epoch': '0.07059'}
{'loss': '0.8618', 'grad_norm': '0.405', 'learning_rate': '5.544e-05', 'entropy': '0.8587', 'num_tokens': '4.303e+07', 'mean_token_accuracy': '0.7807', 'epoch': '0.0706'}
{'loss': '0.6474', 'grad_norm': '0.2695', 'learning_rate': '5.541e-05', 'entropy': '0.6474', 'num_tokens': '4.303e+07', 'mean_token_accuracy': '0.8225', 'epoch': '0.0706'}
{'loss': '0.9159', 'grad_norm': '0.479', 'learning_rate': '5.539e-05', 'entropy': '0.9519', 'num_tokens': '4.304e+07', 'mean_token_accuracy': '0.7626', 'epoch': '0.07061'}
{'loss': '0.6808', 'grad_norm': '0.4045', 'learning_rate': '5.536e-05', 'entropy': '0.7032', 'num_tokens': '4.304e+07', 'mean_token_accuracy': '0.8193', 'epoch': '0.07062'}
{'loss': '0.7121', 'grad_norm': '0.3285', 'learning_rate': '5.534e-05', 'entropy': '0.7233', 'num_tokens': '4.305e+07', 'mean_token_accuracy': '0.8107', 'epoch': '0.07063'}
{'loss': '0.6064', 'grad_norm': '0.4091', 'learning_rate': '5.531e-05', 'entropy': '0.5925', 'num_tokens': '4.305e+07', 'mean_token_accuracy': '0.8327', 'epoch': '0.07064'}
{'loss': '1.102', 'grad_norm': '0.5193', 'learning_rate': '5.529e-05', 'entropy': '0.9609', 'num_tokens': '4.306e+07', 'mean_token_accuracy': '0.7718', 'epoch': '0.07065'}
{'loss': '0.8404', 'grad_norm': '0.3775', 'learning_rate': '5.527e-05', 'entropy': '0.8514', 'num_tokens': '4.306e+07', 'mean_token_accuracy': '0.7772', 'epoch': '0.07066'}
{'loss': '1.297', 'grad_norm': '0.3988', 'learning_rate': '5.524e-05', 'entropy': '1.23', 'num_tokens': '4.307e+07', 'mean_token_accuracy': '0.7093', 'epoch': '0.07067'}
{'loss': '1.321', 'grad_norm': '0.4692', 'learning_rate': '5.522e-05', 'entropy': '1.258', 'num_tokens': '4.308e+07', 'mean_token_accuracy': '0.7014', 'epoch': '0.07067'}
{'loss': '0.6224', 'grad_norm': '0.4113', 'learning_rate': '5.519e-05', 'entropy': '0.6625', 'num_tokens': '4.308e+07', 'mean_token_accuracy': '0.8426', 'epoch': '0.07068'}
{'loss': '1.135', 'grad_norm': '0.3896', 'learning_rate': '5.517e-05', 'entropy': '1.004', 'num_tokens': '4.309e+07', 'mean_token_accuracy': '0.7615', 'epoch': '0.07069'}
{'loss': '0.8408', 'grad_norm': '0.4031', 'learning_rate': '5.514e-05', 'entropy': '0.8281', 'num_tokens': '4.309e+07', 'mean_token_accuracy': '0.7927', 'epoch': '0.0707'}
{'loss': '0.7968', 'grad_norm': '0.4134', 'learning_rate': '5.512e-05', 'entropy': '0.8699', 'num_tokens': '4.31e+07', 'mean_token_accuracy': '0.7844', 'epoch': '0.07071'}
{'loss': '1.129', 'grad_norm': '0.3857', 'learning_rate': '5.509e-05', 'entropy': '1.271', 'num_tokens': '4.311e+07', 'mean_token_accuracy': '0.7124', 'epoch': '0.07072'}
{'loss': '0.6215', 'grad_norm': '0.3301', 'learning_rate': '5.507e-05', 'entropy': '0.6501', 'num_tokens': '4.311e+07', 'mean_token_accuracy': '0.8319', 'epoch': '0.07073'}
{'loss': '0.9234', 'grad_norm': '0.5108', 'learning_rate': '5.504e-05', 'entropy': '0.8988', 'num_tokens': '4.311e+07', 'mean_token_accuracy': '0.7801', 'epoch': '0.07074'}

 67%|██████▋   | 7994/12000 [4:43:44<2:06:35,  1.90s/it]
 67%|██████▋   | 7995/12000 [4:43:45<2:00:07,  1.80s/it]
                                                        

 67%|██████▋   | 7995/12000 [4:43:45<2:00:07,  1.80s/it]
 67%|██████▋   | 7996/12000 [4:43:47<1:56:01,  1.74s/it]
                                                        

 67%|██████▋   | 7996/12000 [4:43:47<1:56:01,  1.74s/it]
 67%|██████▋   | 7997/12000 [4:43:50<2:29:46,  2.24s/it]
                                                        

 67%|██████▋   | 7997/12000 [4:43:50<2:29:46,  2.24s/it]
 67%|██████▋   | 7998/12000 [4:43:52<2:27:26,  2.21s/it]
                                                        

 67%|██████▋   | 7998/12000 [4:43:52<2:27:26,  2.21s/it]
 67%|██████▋   | 7999/12000 [4:43:57<3:12:35,  2.89s/it]
                                                        

 67%|██████▋   | 7999/12000 [4:43:57<3:12:35,  2.89s/it]
 67%|██████▋   | 8000/12000 [4:43:59<3:06:13,  2.79s/it]
                                                        

 67%|██████▋   | 8000/12000 [4:43:59<3:06:13,  2.79s/it]
 67%|██████▋   | 8001/12000 [4:44:02<3:10:45,  2.86s/it]
                                                        

 67%|██████▋   | 8001/12000 [4:44:02<3:10:45,  2.86s/it]
 67%|██████▋   | 8002/12000 [4:44:04<2:48:19,  2.53s/it]
                                                        

 67%|██████▋   | 8002/12000 [4:44:04<2:48:19,  2.53s/it]
 67%|██████▋   | 8003/12000 [4:44:05<2:26:15,  2.20s/it]
                                                        

 67%|██████▋   | 8003/12000 [4:44:05<2:26:15,  2.20s/it]
 67%|██████▋   | 8004/12000 [4:44:07<2:14:02,  2.01s/it]
                                                        

 67%|██████▋   | 8004/12000 [4:44:07<2:14:02,  2.01s/it]
 67%|██████▋   | 8005/12000 [4:44:09<2:03:07,  1.85s/it]
                                                        

 67%|██████▋   | 8005/12000 [4:44:09<2:03:07,  1.85s/it]
 67%|██████▋   | 8006/12000 [4:44:10<2:01:35,  1.83s/it]
                                                        

 67%|██████▋   | 8006/12000 [4:44:10<2:01:35,  1.83s/it]
 67%|██████▋   | 8007/12000 [4:44:12<1:57:39,  1.77s/it]
                                                        

 67%|██████▋   | 8007/12000 [4:44:12<1:57:39,  1.77s/it]
 67%|██████▋   | 8008/12000 [4:44:14<2:12:04,  1.99s/it]
                                                        

 67%|██████▋   | 8008/12000 [4:44:14<2:12:04,  1.99s/it]
 67%|██████▋   | 8009/12000 [4:44:16<2:08:59,  1.94s/it]
                                                        

 67%|██████▋   | 8009/12000 [4:44:16<2:08:59,  1.94s/it]
 67%|██████▋   | 8010/12000 [4:44:19<2:15:51,  2.04s/it]
                                                        

 67%|██████▋   | 8010/12000 [4:44:19<2:15:51,  2.04s/it]
 67%|██████▋   | 8011/12000 [4:44:21<2:21:35,  2.13s/it]
                                                        

 67%|██████▋   | 8011/12000 [4:44:21<2:21:35,  2.13s/it]
 67%|██████▋   | 8012/12000 [4:44:22<2:09:21,  1.95s/it]
                                                        

 67%|██████▋   | 8012/12000 [4:44:22<2:09:21,  1.95s/it]
 67%|██████▋   | 8013/12000 [4:44:26<2:35:57,  2.35s/it]
                                                        

 67%|██████▋   | 8013/12000 [4:44:26<2:35:57,  2.35s/it]
 67%|██████▋   | 8014/12000 [4:44:27<2:22:14,  2.14s/it]
                                                        

 67%|██████▋   | 8014/12000 [4:44:27<2:22:14,  2.14s/it]
 67%|██████▋   | 8015/12000 [4:44:29<2:09:59,  1.96s/it]
                                                        

 67%|██████▋   | 8015/12000 [4:44:29<2:09:59,  1.96s/it]
 67%|██████▋   | 8016/12000 [4:44:30<2:03:45,  1.86s/it]
                                                        

 67%|██████▋   | 8016/12000 [4:44:30<2:03:45,  1.86s/it]
 67%|██████▋   | 8017/12000 [4:44:32<1:55:04,  1.73s/it]
                                                        

 67%|██████▋   | 8017/12000 [4:44:32<1:55:04,  1.73s/it]
 67%|██████▋   | 8018/12000 [4:44:33<1:51:18,  1.68s/it]
                                                        

 67%|██████▋   | 8018/12000 [4:44:33<1:51:18,  1.68s/it]
 67%|██████▋   | 8019/12000 [4:44:38<2:40:07,  2.41s/it]
                                                        

 67%|██████▋   | 8019/12000 [4:44:38<2:40:07,  2.41s/it]
 67%|██████▋   | 8020/12000 [4:44:39<2:26:26,  2.21s/it]
                                                        

 67%|██████▋   | 8020/12000 [4:44:39<2:26:26,  2.21s/it]
 67%|██████▋   | 8021/12000 [4:44:41<2:16:27,  2.06s/it]
                                                        

 67%|██████▋   | 8021/12000 [4:44:41<2:16:27,  2.06s/it]
 67%|██████▋   | 8022/12000 [4:44:43<2:21:42,  2.14s/it]
                                                        

 67%|██████▋   | 8022/12000 [4:44:43<2:21:42,  2.14s/it]
 67%|██████▋   | 8023/12000 [4:44:45<2:10:37,  1.97s/it]
                                                        

 67%|██████▋   | 8023/12000 [4:44:45<2:10:37,  1.97s/it]
 67%|██████▋   | 8024/12000 [4:44:47<2:02:45,  1.85s/it]
                                                        

 67%|██████▋   | 8024/12000 [4:44:47<2:02:45,  1.85s/it]
 67%|██████▋   | 8025/12000 [4:44:48<1:56:48,  1.76s/it]
                                                        

 67%|██████▋   | 8025/12000 [4:44:48<1:56:48,  1.76s/it]
 67%|██████▋   | 8026/12000 [4:44:51<2:22:28,  2.15s/it]
                                                        

 67%|██████▋   | 8026/12000 [4:44:51<2:22:28,  2.15s/it]
 67%|██████▋   | 8027/12000 [4:44:53<2:26:44,  2.22s/it]
                                                        

 67%|██████▋   | 8027/12000 [4:44:54<2:26:44,  2.22s/it]
 67%|██████▋   | 8028/12000 [4:44:55<2:21:28,  2.14s/it]
                                                        

 67%|██████▋   | 8028/12000 [4:44:55<2:21:28,  2.14s/it]
 67%|██████▋   | 8029/12000 [4:44:57<2:13:01,  2.01s/it]
                                                        

 67%|██████▋   | 8029/12000 [4:44:57<2:13:01,  2.01s/it]
 67%|██████▋   | 8030/12000 [4:45:01<2:52:21,  2.61s/it]
                                                        

 67%|██████▋   | 8030/12000 [4:45:01<2:52:21,  2.61s/it]
 67%|██████▋   | 8031/12000 [4:45:03<2:35:59,  2.36s/it]
                                                        

 67%|██████▋   | 8031/12000 [4:45:03<2:35:59,  2.36s/it]
 67%|██████▋   | 8032/12000 [4:45:08<3:36:28,  3.27s/it]
                                                        

 67%|██████▋   | 8032/12000 [4:45:08<3:36:28,  3.27s/it]
 67%|██████▋   | 8033/12000 [4:45:11<3:29:35,  3.17s/it]
                                                        

 67%|██████▋   | 8033/12000 [4:45:11<3:29:35,  3.17s/it]
 67%|██████▋   | 8034/12000 [4:45:13<3:09:42,  2.87s/it]
                                                        

 67%|██████▋   | 8034/12000 [4:45:13<3:09:42,  2.87s/it]
 67%|██████▋   | 8035/12000 [4:45:16<2:55:28,  2.66s/it]
                                                        

 67%|██████▋   | 8035/12000 [4:45:16<2:55:28,  2.66s/it]
 67%|██████▋   | 8036/12000 [4:45:18<2:55:36,  2.66s/it]
                                                        

 67%|██████▋   | 8036/12000 [4:45:18<2:55:36,  2.66s/it]
 67%|██████▋   | 8037/12000 [4:45:20<2:35:21,  2.35s/it]
                                                        

 67%|██████▋   | 8037/12000 [4:45:20<2:35:21,  2.35s/it]
 67%|██████▋   | 8038/12000 [4:45:22<2:22:10,  2.15s/it]
                                                        

 67%|██████▋   | 8038/12000 [4:45:22<2:22:10,  2.15s/it]
 67%|██████▋   | 8039/12000 [4:45:24<2:29:51,  2.27s/it]
                                                        

 67%|██████▋   | 8039/12000 [4:45:24<2:29:51,  2.27s/it]
 67%|██████▋   | 8040/12000 [4:45:26<2:15:37,  2.05s/it]
                                                        

 67%|██████▋   | 8040/12000 [4:45:26<2:15:37,  2.05s/it]
 67%|██████▋   | 8041/12000 [4:45:29<2:43:37,  2.48s/it]
                                                        
{'loss': '0.668', 'grad_norm': '0.4304', 'learning_rate': '5.502e-05', 'entropy': '0.6853', 'num_tokens': '4.312e+07', 'mean_token_accuracy': '0.816', 'epoch': '0.07075'}
{'loss': '0.6149', 'grad_norm': '0.3578', 'learning_rate': '5.499e-05', 'entropy': '0.5971', 'num_tokens': '4.312e+07', 'mean_token_accuracy': '0.8357', 'epoch': '0.07075'}
{'loss': '0.6464', 'grad_norm': '0.3915', 'learning_rate': '5.497e-05', 'entropy': '0.6524', 'num_tokens': '4.313e+07', 'mean_token_accuracy': '0.8322', 'epoch': '0.07076'}
{'loss': '0.5368', 'grad_norm': '0.3687', 'learning_rate': '5.495e-05', 'entropy': '0.5587', 'num_tokens': '4.313e+07', 'mean_token_accuracy': '0.8507', 'epoch': '0.07077'}
{'loss': '0.9998', 'grad_norm': '0.4416', 'learning_rate': '5.492e-05', 'entropy': '0.968', 'num_tokens': '4.314e+07', 'mean_token_accuracy': '0.7666', 'epoch': '0.07078'}
{'loss': '0.4663', 'grad_norm': '0.2792', 'learning_rate': '5.49e-05', 'entropy': '0.4796', 'num_tokens': '4.315e+07', 'mean_token_accuracy': '0.8724', 'epoch': '0.07079'}
{'loss': '0.82', 'grad_norm': '0.3665', 'learning_rate': '5.487e-05', 'entropy': '0.7913', 'num_tokens': '4.316e+07', 'mean_token_accuracy': '0.7932', 'epoch': '0.0708'}
{'loss': '0.6851', 'grad_norm': '0.4573', 'learning_rate': '5.485e-05', 'entropy': '0.7758', 'num_tokens': '4.316e+07', 'mean_token_accuracy': '0.7909', 'epoch': '0.07081'}
{'loss': '0.673', 'grad_norm': '0.3901', 'learning_rate': '5.482e-05', 'entropy': '0.6561', 'num_tokens': '4.317e+07', 'mean_token_accuracy': '0.8223', 'epoch': '0.07082'}
{'loss': '0.6147', 'grad_norm': '0.3628', 'learning_rate': '5.48e-05', 'entropy': '0.6987', 'num_tokens': '4.317e+07', 'mean_token_accuracy': '0.8149', 'epoch': '0.07083'}
{'loss': '0.5933', 'grad_norm': '0.3913', 'learning_rate': '5.477e-05', 'entropy': '0.6132', 'num_tokens': '4.317e+07', 'mean_token_accuracy': '0.8389', 'epoch': '0.07083'}
{'loss': '1.019', 'grad_norm': '0.4528', 'learning_rate': '5.475e-05', 'entropy': '1.01', 'num_tokens': '4.318e+07', 'mean_token_accuracy': '0.7631', 'epoch': '0.07084'}
{'loss': '0.7113', 'grad_norm': '0.302', 'learning_rate': '5.472e-05', 'entropy': '0.7229', 'num_tokens': '4.318e+07', 'mean_token_accuracy': '0.8105', 'epoch': '0.07085'}
{'loss': '0.9815', 'grad_norm': '0.4847', 'learning_rate': '5.47e-05', 'entropy': '0.9637', 'num_tokens': '4.319e+07', 'mean_token_accuracy': '0.7555', 'epoch': '0.07086'}
{'loss': '1.118', 'grad_norm': '0.4174', 'learning_rate': '5.467e-05', 'entropy': '1.016', 'num_tokens': '4.32e+07', 'mean_token_accuracy': '0.7486', 'epoch': '0.07087'}
{'loss': '0.6922', 'grad_norm': '0.2964', 'learning_rate': '5.465e-05', 'entropy': '0.6759', 'num_tokens': '4.32e+07', 'mean_token_accuracy': '0.8212', 'epoch': '0.07088'}
{'loss': '0.8002', 'grad_norm': '0.3643', 'learning_rate': '5.463e-05', 'entropy': '0.7295', 'num_tokens': '4.321e+07', 'mean_token_accuracy': '0.8242', 'epoch': '0.07089'}
{'loss': '1.045', 'grad_norm': '0.5528', 'learning_rate': '5.46e-05', 'entropy': '1.188', 'num_tokens': '4.321e+07', 'mean_token_accuracy': '0.7247', 'epoch': '0.0709'}
{'loss': '0.6118', 'grad_norm': '0.3613', 'learning_rate': '5.458e-05', 'entropy': '0.6503', 'num_tokens': '4.322e+07', 'mean_token_accuracy': '0.8331', 'epoch': '0.0709'}
{'loss': '1.104', 'grad_norm': '0.3362', 'learning_rate': '5.455e-05', 'entropy': '0.9066', 'num_tokens': '4.322e+07', 'mean_token_accuracy': '0.7825', 'epoch': '0.07091'}
{'loss': '0.8665', 'grad_norm': '0.4894', 'learning_rate': '5.453e-05', 'entropy': '0.8508', 'num_tokens': '4.323e+07', 'mean_token_accuracy': '0.7909', 'epoch': '0.07092'}
{'loss': '0.8028', 'grad_norm': '0.3565', 'learning_rate': '5.45e-05', 'entropy': '0.8174', 'num_tokens': '4.324e+07', 'mean_token_accuracy': '0.7784', 'epoch': '0.07093'}
{'loss': '1.1', 'grad_norm': '0.3934', 'learning_rate': '5.448e-05', 'entropy': '1.014', 'num_tokens': '4.324e+07', 'mean_token_accuracy': '0.756', 'epoch': '0.07094'}
{'loss': '0.6245', 'grad_norm': '0.3863', 'learning_rate': '5.445e-05', 'entropy': '0.6609', 'num_tokens': '4.324e+07', 'mean_token_accuracy': '0.8257', 'epoch': '0.07095'}
{'loss': '0.6661', 'grad_norm': '0.4023', 'learning_rate': '5.443e-05', 'entropy': '0.6684', 'num_tokens': '4.325e+07', 'mean_token_accuracy': '0.8356', 'epoch': '0.07096'}
{'loss': '0.7869', 'grad_norm': '0.4606', 'learning_rate': '5.44e-05', 'entropy': '0.7595', 'num_tokens': '4.326e+07', 'mean_token_accuracy': '0.7983', 'epoch': '0.07097'}
{'loss': '1.086', 'grad_norm': '0.4199', 'learning_rate': '5.438e-05', 'entropy': '1.104', 'num_tokens': '4.326e+07', 'mean_token_accuracy': '0.7276', 'epoch': '0.07098'}
{'loss': '0.6086', 'grad_norm': '0.3807', 'learning_rate': '5.436e-05', 'entropy': '0.603', 'num_tokens': '4.327e+07', 'mean_token_accuracy': '0.844', 'epoch': '0.07098'}
{'loss': '0.7312', 'grad_norm': '0.3902', 'learning_rate': '5.433e-05', 'entropy': '0.6979', 'num_tokens': '4.327e+07', 'mean_token_accuracy': '0.8087', 'epoch': '0.07099'}
{'loss': '0.6777', 'grad_norm': '0.4767', 'learning_rate': '5.431e-05', 'entropy': '0.6916', 'num_tokens': '4.328e+07', 'mean_token_accuracy': '0.8176', 'epoch': '0.071'}
{'loss': '0.859', 'grad_norm': '0.3866', 'learning_rate': '5.428e-05', 'entropy': '0.8102', 'num_tokens': '4.328e+07', 'mean_token_accuracy': '0.7852', 'epoch': '0.07101'}
{'loss': '0.7016', 'grad_norm': '0.3738', 'learning_rate': '5.426e-05', 'entropy': '0.7241', 'num_tokens': '4.329e+07', 'mean_token_accuracy': '0.8139', 'epoch': '0.07102'}
{'loss': '1.276', 'grad_norm': '0.4379', 'learning_rate': '5.423e-05', 'entropy': '1.023', 'num_tokens': '4.329e+07', 'mean_token_accuracy': '0.7397', 'epoch': '0.07103'}
{'loss': '0.6013', 'grad_norm': '0.3277', 'learning_rate': '5.421e-05', 'entropy': '0.617', 'num_tokens': '4.33e+07', 'mean_token_accuracy': '0.8367', 'epoch': '0.07104'}
{'loss': '0.8546', 'grad_norm': '0.5196', 'learning_rate': '5.418e-05', 'entropy': '0.8155', 'num_tokens': '4.331e+07', 'mean_token_accuracy': '0.7869', 'epoch': '0.07105'}
{'loss': '0.8533', 'grad_norm': '0.4505', 'learning_rate': '5.416e-05', 'entropy': '0.8179', 'num_tokens': '4.331e+07', 'mean_token_accuracy': '0.7963', 'epoch': '0.07106'}
{'loss': '1.303', 'grad_norm': '0.388', 'learning_rate': '5.414e-05', 'entropy': '1.169', 'num_tokens': '4.332e+07', 'mean_token_accuracy': '0.7271', 'epoch': '0.07106'}
{'loss': '0.7535', 'grad_norm': '0.4267', 'learning_rate': '5.411e-05', 'entropy': '0.8109', 'num_tokens': '4.332e+07', 'mean_token_accuracy': '0.8017', 'epoch': '0.07107'}
{'loss': '1.063', 'grad_norm': '0.3924', 'learning_rate': '5.409e-05', 'entropy': '1.093', 'num_tokens': '4.333e+07', 'mean_token_accuracy': '0.7416', 'epoch': '0.07108'}
{'loss': '0.6853', 'grad_norm': '0.3214', 'learning_rate': '5.406e-05', 'entropy': '0.7413', 'num_tokens': '4.334e+07', 'mean_token_accuracy': '0.816', 'epoch': '0.07109'}
{'loss': '1.051', 'grad_norm': '0.4682', 'learning_rate': '5.404e-05', 'entropy': '0.9655', 'num_tokens': '4.335e+07', 'mean_token_accuracy': '0.7656', 'epoch': '0.0711'}
{'loss': '0.9312', 'grad_norm': '0.3538', 'learning_rate': '5.401e-05', 'entropy': '0.9407', 'num_tokens': '4.335e+07', 'mean_token_accuracy': '0.7663', 'epoch': '0.07111'}
{'loss': '1.17', 'grad_norm': '0.4134', 'learning_rate': '5.399e-05', 'entropy': '1.231', 'num_tokens': '4.336e+07', 'mean_token_accuracy': '0.7139', 'epoch': '0.07112'}
{'loss': '0.6861', 'grad_norm': '0.4316', 'learning_rate': '5.396e-05', 'entropy': '0.7144', 'num_tokens': '4.336e+07', 'mean_token_accuracy': '0.815', 'epoch': '0.07113'}
{'loss': '0.8665', 'grad_norm': '0.4535', 'learning_rate': '5.394e-05', 'entropy': '0.8605', 'num_tokens': '4.337e+07', 'mean_token_accuracy': '0.7895', 'epoch': '0.07113'}
{'loss': '0.7362', 'grad_norm': '0.4007', 'learning_rate': '5.391e-05', 'entropy': '0.6858', 'num_tokens': '4.337e+07', 'mean_token_accuracy': '0.8178', 'epoch': '0.07114'}
{'loss': '0.6383', 'grad_norm': '0.3141', 'learning_rate': '5.389e-05', 'entropy': '0.7068', 'num_tokens': '4.338e+07', 'mean_token_accuracy': '0.8166', 'epoch': '0.07115'}

 67%|██████▋   | 8041/12000 [4:45:29<2:43:37,  2.48s/it]
 67%|██████▋   | 8042/12000 [4:45:31<2:23:46,  2.18s/it]
                                                        

 67%|██████▋   | 8042/12000 [4:45:31<2:23:46,  2.18s/it]
 67%|██████▋   | 8043/12000 [4:45:32<2:10:29,  1.98s/it]
                                                        

 67%|██████▋   | 8043/12000 [4:45:32<2:10:29,  1.98s/it]
 67%|██████▋   | 8044/12000 [4:45:34<2:01:04,  1.84s/it]
                                                        

 67%|██████▋   | 8044/12000 [4:45:34<2:01:04,  1.84s/it]
 67%|██████▋   | 8045/12000 [4:45:35<1:57:26,  1.78s/it]
                                                        

 67%|██████▋   | 8045/12000 [4:45:35<1:57:26,  1.78s/it]
 67%|██████▋   | 8046/12000 [4:45:37<1:57:23,  1.78s/it]
                                                        

 67%|██████▋   | 8046/12000 [4:45:37<1:57:23,  1.78s/it]
 67%|██████▋   | 8047/12000 [4:45:39<1:51:52,  1.70s/it]
                                                        

 67%|██████▋   | 8047/12000 [4:45:39<1:51:52,  1.70s/it]
 67%|██████▋   | 8048/12000 [4:45:41<1:57:45,  1.79s/it]
                                                        

 67%|██████▋   | 8048/12000 [4:45:41<1:57:45,  1.79s/it]
 67%|██████▋   | 8049/12000 [4:45:43<2:02:58,  1.87s/it]
                                                        

 67%|██████▋   | 8049/12000 [4:45:43<2:02:58,  1.87s/it]
 67%|██████▋   | 8050/12000 [4:45:45<2:18:09,  2.10s/it]
                                                        

 67%|██████▋   | 8050/12000 [4:45:45<2:18:09,  2.10s/it]
 67%|██████▋   | 8051/12000 [4:45:48<2:39:55,  2.43s/it]
                                                        

 67%|██████▋   | 8051/12000 [4:45:48<2:39:55,  2.43s/it]
 67%|██████▋   | 8052/12000 [4:45:50<2:22:09,  2.16s/it]
                                                        

 67%|██████▋   | 8052/12000 [4:45:50<2:22:09,  2.16s/it]
 67%|██████▋   | 8053/12000 [4:45:52<2:15:37,  2.06s/it]
                                                        

 67%|██████▋   | 8053/12000 [4:45:52<2:15:37,  2.06s/it]
 67%|██████▋   | 8054/12000 [4:45:55<2:41:52,  2.46s/it]
                                                        

 67%|██████▋   | 8054/12000 [4:45:55<2:41:52,  2.46s/it]
 67%|██████▋   | 8055/12000 [4:45:59<3:02:35,  2.78s/it]
                                                        

 67%|██████▋   | 8055/12000 [4:45:59<3:02:35,  2.78s/it]
 67%|██████▋   | 8056/12000 [4:46:01<2:54:44,  2.66s/it]
                                                        

 67%|██████▋   | 8056/12000 [4:46:01<2:54:44,  2.66s/it]
 67%|██████▋   | 8057/12000 [4:46:03<2:32:27,  2.32s/it]
                                                        

 67%|██████▋   | 8057/12000 [4:46:03<2:32:27,  2.32s/it]
 67%|██████▋   | 8058/12000 [4:46:05<2:34:06,  2.35s/it]
                                                        

 67%|██████▋   | 8058/12000 [4:46:05<2:34:06,  2.35s/it]
 67%|██████▋   | 8059/12000 [4:46:07<2:18:43,  2.11s/it]
                                                        

 67%|██████▋   | 8059/12000 [4:46:07<2:18:43,  2.11s/it]
 67%|██████▋   | 8060/12000 [4:46:08<2:09:08,  1.97s/it]
                                                        

 67%|██████▋   | 8060/12000 [4:46:08<2:09:08,  1.97s/it]
 67%|██████▋   | 8061/12000 [4:46:10<2:09:53,  1.98s/it]
                                                        

 67%|██████▋   | 8061/12000 [4:46:10<2:09:53,  1.98s/it]
 67%|██████▋   | 8062/12000 [4:46:12<2:02:06,  1.86s/it]
                                                        

 67%|██████▋   | 8062/12000 [4:46:12<2:02:06,  1.86s/it]
 67%|██████▋   | 8063/12000 [4:46:16<2:40:28,  2.45s/it]
                                                        

 67%|██████▋   | 8063/12000 [4:46:16<2:40:28,  2.45s/it]
 67%|██████▋   | 8064/12000 [4:46:17<2:23:14,  2.18s/it]
                                                        

 67%|██████▋   | 8064/12000 [4:46:17<2:23:14,  2.18s/it]
 67%|██████▋   | 8065/12000 [4:46:20<2:33:22,  2.34s/it]
                                                        

 67%|██████▋   | 8065/12000 [4:46:20<2:33:22,  2.34s/it]
 67%|██████▋   | 8066/12000 [4:46:23<2:39:18,  2.43s/it]
                                                        

 67%|██████▋   | 8066/12000 [4:46:23<2:39:18,  2.43s/it]
 67%|██████▋   | 8067/12000 [4:46:25<2:31:34,  2.31s/it]
                                                        

 67%|██████▋   | 8067/12000 [4:46:25<2:31:34,  2.31s/it]
 67%|██████▋   | 8068/12000 [4:46:26<2:15:18,  2.06s/it]
                                                        

 67%|██████▋   | 8068/12000 [4:46:26<2:15:18,  2.06s/it]
 67%|██████▋   | 8069/12000 [4:46:31<3:16:31,  3.00s/it]
                                                        

 67%|██████▋   | 8069/12000 [4:46:31<3:16:31,  3.00s/it]
 67%|██████▋   | 8070/12000 [4:46:33<2:54:04,  2.66s/it]
                                                        

 67%|██████▋   | 8070/12000 [4:46:33<2:54:04,  2.66s/it]
 67%|██████▋   | 8071/12000 [4:46:35<2:31:37,  2.32s/it]
                                                        

 67%|██████▋   | 8071/12000 [4:46:35<2:31:37,  2.32s/it]
 67%|██████▋   | 8072/12000 [4:46:37<2:38:09,  2.42s/it]
                                                        

 67%|██████▋   | 8072/12000 [4:46:37<2:38:09,  2.42s/it]
 67%|██████▋   | 8073/12000 [4:46:39<2:18:41,  2.12s/it]
                                                        

 67%|██████▋   | 8073/12000 [4:46:39<2:18:41,  2.12s/it]
 67%|██████▋   | 8074/12000 [4:46:40<2:10:29,  1.99s/it]
                                                        

 67%|██████▋   | 8074/12000 [4:46:40<2:10:29,  1.99s/it]
 67%|██████▋   | 8075/12000 [4:46:42<1:59:47,  1.83s/it]
                                                        

 67%|██████▋   | 8075/12000 [4:46:42<1:59:47,  1.83s/it]
 67%|██████▋   | 8076/12000 [4:46:45<2:22:28,  2.18s/it]
                                                        

 67%|██████▋   | 8076/12000 [4:46:45<2:22:28,  2.18s/it]
 67%|██████▋   | 8077/12000 [4:46:47<2:12:48,  2.03s/it]
                                                        

 67%|██████▋   | 8077/12000 [4:46:47<2:12:48,  2.03s/it]
 67%|██████▋   | 8078/12000 [4:46:48<2:07:11,  1.95s/it]
                                                        

 67%|██████▋   | 8078/12000 [4:46:48<2:07:11,  1.95s/it]
 67%|██████▋   | 8079/12000 [4:46:50<1:58:57,  1.82s/it]
                                                        

 67%|██████▋   | 8079/12000 [4:46:50<1:58:57,  1.82s/it]
 67%|██████▋   | 8080/12000 [4:46:52<2:02:48,  1.88s/it]
                                                        

 67%|██████▋   | 8080/12000 [4:46:52<2:02:48,  1.88s/it]
 67%|██████▋   | 8081/12000 [4:46:54<2:02:10,  1.87s/it]
                                                        

 67%|██████▋   | 8081/12000 [4:46:54<2:02:10,  1.87s/it]
 67%|██████▋   | 8082/12000 [4:46:55<1:54:32,  1.75s/it]
                                                        

 67%|██████▋   | 8082/12000 [4:46:55<1:54:32,  1.75s/it]
 67%|██████▋   | 8083/12000 [4:46:57<1:55:13,  1.77s/it]
                                                        

 67%|██████▋   | 8083/12000 [4:46:57<1:55:13,  1.77s/it]
 67%|██████▋   | 8084/12000 [4:47:01<2:48:24,  2.58s/it]
                                                        

 67%|██████▋   | 8084/12000 [4:47:01<2:48:24,  2.58s/it]
 67%|██████▋   | 8085/12000 [4:47:05<3:06:25,  2.86s/it]
                                                        

 67%|██████▋   | 8085/12000 [4:47:05<3:06:25,  2.86s/it]
 67%|██████▋   | 8086/12000 [4:47:07<2:40:39,  2.46s/it]
                                                        

 67%|██████▋   | 8086/12000 [4:47:07<2:40:39,  2.46s/it]
 67%|██████▋   | 8087/12000 [4:47:08<2:20:19,  2.15s/it]
                                                        

 67%|██████▋   | 8087/12000 [4:47:08<2:20:19,  2.15s/it]
 67%|██████▋   | 8088/12000 [4:47:09<2:07:34,  1.96s/it]
                                                        
{'loss': '0.7977', 'grad_norm': '0.4007', 'learning_rate': '5.387e-05', 'entropy': '0.8849', 'num_tokens': '4.338e+07', 'mean_token_accuracy': '0.7696', 'epoch': '0.07116'}
{'loss': '0.9871', 'grad_norm': '0.4839', 'learning_rate': '5.384e-05', 'entropy': '1.064', 'num_tokens': '4.339e+07', 'mean_token_accuracy': '0.7511', 'epoch': '0.07117'}
{'loss': '0.7306', 'grad_norm': '0.362', 'learning_rate': '5.382e-05', 'entropy': '0.7809', 'num_tokens': '4.339e+07', 'mean_token_accuracy': '0.7928', 'epoch': '0.07118'}
{'loss': '0.6695', 'grad_norm': '0.29', 'learning_rate': '5.379e-05', 'entropy': '0.6934', 'num_tokens': '4.339e+07', 'mean_token_accuracy': '0.8295', 'epoch': '0.07119'}
{'loss': '0.8599', 'grad_norm': '0.3978', 'learning_rate': '5.377e-05', 'entropy': '0.8907', 'num_tokens': '4.34e+07', 'mean_token_accuracy': '0.7843', 'epoch': '0.0712'}
{'loss': '1.159', 'grad_norm': '0.5', 'learning_rate': '5.374e-05', 'entropy': '1.186', 'num_tokens': '4.34e+07', 'mean_token_accuracy': '0.7156', 'epoch': '0.07121'}
{'loss': '0.6131', 'grad_norm': '0.376', 'learning_rate': '5.372e-05', 'entropy': '0.6431', 'num_tokens': '4.341e+07', 'mean_token_accuracy': '0.8448', 'epoch': '0.07121'}
{'loss': '0.7645', 'grad_norm': '0.4815', 'learning_rate': '5.369e-05', 'entropy': '0.7973', 'num_tokens': '4.341e+07', 'mean_token_accuracy': '0.7952', 'epoch': '0.07122'}
{'loss': '0.5664', 'grad_norm': '0.3242', 'learning_rate': '5.367e-05', 'entropy': '0.543', 'num_tokens': '4.342e+07', 'mean_token_accuracy': '0.8584', 'epoch': '0.07123'}
{'loss': '0.8663', 'grad_norm': '0.3869', 'learning_rate': '5.365e-05', 'entropy': '0.8544', 'num_tokens': '4.342e+07', 'mean_token_accuracy': '0.7885', 'epoch': '0.07124'}
{'loss': '1.166', 'grad_norm': '0.5249', 'learning_rate': '5.362e-05', 'entropy': '1.164', 'num_tokens': '4.343e+07', 'mean_token_accuracy': '0.7234', 'epoch': '0.07125'}
{'loss': '0.6639', 'grad_norm': '0.3728', 'learning_rate': '5.36e-05', 'entropy': '0.7078', 'num_tokens': '4.343e+07', 'mean_token_accuracy': '0.8183', 'epoch': '0.07126'}
{'loss': '1.012', 'grad_norm': '0.3905', 'learning_rate': '5.357e-05', 'entropy': '0.9818', 'num_tokens': '4.344e+07', 'mean_token_accuracy': '0.7588', 'epoch': '0.07127'}
{'loss': '0.8844', 'grad_norm': '0.4164', 'learning_rate': '5.355e-05', 'entropy': '0.8411', 'num_tokens': '4.345e+07', 'mean_token_accuracy': '0.7826', 'epoch': '0.07128'}
{'loss': '1.106', 'grad_norm': '0.4706', 'learning_rate': '5.352e-05', 'entropy': '1.052', 'num_tokens': '4.346e+07', 'mean_token_accuracy': '0.7443', 'epoch': '0.07129'}
{'loss': '0.8564', 'grad_norm': '0.3479', 'learning_rate': '5.35e-05', 'entropy': '0.8206', 'num_tokens': '4.346e+07', 'mean_token_accuracy': '0.7997', 'epoch': '0.07129'}
{'loss': '0.712', 'grad_norm': '0.4265', 'learning_rate': '5.348e-05', 'entropy': '0.737', 'num_tokens': '4.347e+07', 'mean_token_accuracy': '0.8115', 'epoch': '0.0713'}
{'loss': '0.9171', 'grad_norm': '0.4572', 'learning_rate': '5.345e-05', 'entropy': '0.8371', 'num_tokens': '4.347e+07', 'mean_token_accuracy': '0.779', 'epoch': '0.07131'}
{'loss': '0.7662', 'grad_norm': '0.361', 'learning_rate': '5.343e-05', 'entropy': '0.8342', 'num_tokens': '4.348e+07', 'mean_token_accuracy': '0.7909', 'epoch': '0.07132'}
{'loss': '0.6349', 'grad_norm': '0.3375', 'learning_rate': '5.34e-05', 'entropy': '0.6285', 'num_tokens': '4.348e+07', 'mean_token_accuracy': '0.8444', 'epoch': '0.07133'}
{'loss': '1.09', 'grad_norm': '0.4241', 'learning_rate': '5.338e-05', 'entropy': '1.007', 'num_tokens': '4.349e+07', 'mean_token_accuracy': '0.7644', 'epoch': '0.07134'}
{'loss': '0.9512', 'grad_norm': '0.4671', 'learning_rate': '5.335e-05', 'entropy': '0.9249', 'num_tokens': '4.349e+07', 'mean_token_accuracy': '0.7591', 'epoch': '0.07135'}
{'loss': '0.9179', 'grad_norm': '0.4202', 'learning_rate': '5.333e-05', 'entropy': '0.9672', 'num_tokens': '4.35e+07', 'mean_token_accuracy': '0.7687', 'epoch': '0.07136'}
{'loss': '0.9972', 'grad_norm': '0.5256', 'learning_rate': '5.33e-05', 'entropy': '0.9401', 'num_tokens': '4.35e+07', 'mean_token_accuracy': '0.7818', 'epoch': '0.07137'}
{'loss': '0.7471', 'grad_norm': '0.3407', 'learning_rate': '5.328e-05', 'entropy': '0.7865', 'num_tokens': '4.351e+07', 'mean_token_accuracy': '0.7966', 'epoch': '0.07137'}
{'loss': '0.7861', 'grad_norm': '0.4314', 'learning_rate': '5.326e-05', 'entropy': '0.8108', 'num_tokens': '4.352e+07', 'mean_token_accuracy': '0.7942', 'epoch': '0.07138'}
{'loss': '0.8307', 'grad_norm': '0.4235', 'learning_rate': '5.323e-05', 'entropy': '0.7917', 'num_tokens': '4.352e+07', 'mean_token_accuracy': '0.7996', 'epoch': '0.07139'}
{'loss': '0.5864', 'grad_norm': '0.3599', 'learning_rate': '5.321e-05', 'entropy': '0.8189', 'num_tokens': '4.353e+07', 'mean_token_accuracy': '0.8204', 'epoch': '0.0714'}
{'loss': '0.8585', 'grad_norm': '0.3505', 'learning_rate': '5.318e-05', 'entropy': '0.9027', 'num_tokens': '4.354e+07', 'mean_token_accuracy': '0.7713', 'epoch': '0.07141'}
{'loss': '0.9175', 'grad_norm': '0.5142', 'learning_rate': '5.316e-05', 'entropy': '0.7783', 'num_tokens': '4.354e+07', 'mean_token_accuracy': '0.8049', 'epoch': '0.07142'}
{'loss': '0.9956', 'grad_norm': '0.4411', 'learning_rate': '5.313e-05', 'entropy': '1.006', 'num_tokens': '4.355e+07', 'mean_token_accuracy': '0.7694', 'epoch': '0.07143'}
{'loss': '0.5768', 'grad_norm': '0.3774', 'learning_rate': '5.311e-05', 'entropy': '0.6839', 'num_tokens': '4.355e+07', 'mean_token_accuracy': '0.8285', 'epoch': '0.07144'}
{'loss': '0.7318', 'grad_norm': '0.449', 'learning_rate': '5.309e-05', 'entropy': '0.7684', 'num_tokens': '4.356e+07', 'mean_token_accuracy': '0.8009', 'epoch': '0.07144'}
{'loss': '0.76', 'grad_norm': '0.4292', 'learning_rate': '5.306e-05', 'entropy': '0.7243', 'num_tokens': '4.356e+07', 'mean_token_accuracy': '0.8006', 'epoch': '0.07145'}
{'loss': '0.7599', 'grad_norm': '0.3807', 'learning_rate': '5.304e-05', 'entropy': '0.7725', 'num_tokens': '4.356e+07', 'mean_token_accuracy': '0.7892', 'epoch': '0.07146'}
{'loss': '0.666', 'grad_norm': '0.3334', 'learning_rate': '5.301e-05', 'entropy': '0.6692', 'num_tokens': '4.357e+07', 'mean_token_accuracy': '0.8338', 'epoch': '0.07147'}
{'loss': '0.8287', 'grad_norm': '0.434', 'learning_rate': '5.299e-05', 'entropy': '0.8336', 'num_tokens': '4.358e+07', 'mean_token_accuracy': '0.8066', 'epoch': '0.07148'}
{'loss': '1.257', 'grad_norm': '0.4667', 'learning_rate': '5.296e-05', 'entropy': '1.174', 'num_tokens': '4.358e+07', 'mean_token_accuracy': '0.7312', 'epoch': '0.07149'}
{'loss': '0.632', 'grad_norm': '0.3698', 'learning_rate': '5.294e-05', 'entropy': '0.6361', 'num_tokens': '4.359e+07', 'mean_token_accuracy': '0.8347', 'epoch': '0.0715'}
{'loss': '0.7893', 'grad_norm': '0.3549', 'learning_rate': '5.292e-05', 'entropy': '0.7672', 'num_tokens': '4.359e+07', 'mean_token_accuracy': '0.8002', 'epoch': '0.07151'}
{'loss': '0.8633', 'grad_norm': '0.4308', 'learning_rate': '5.289e-05', 'entropy': '0.8475', 'num_tokens': '4.36e+07', 'mean_token_accuracy': '0.7838', 'epoch': '0.07152'}
{'loss': '0.56', 'grad_norm': '0.4815', 'learning_rate': '5.287e-05', 'entropy': '0.5862', 'num_tokens': '4.36e+07', 'mean_token_accuracy': '0.8398', 'epoch': '0.07152'}
{'loss': '0.7756', 'grad_norm': '0.4144', 'learning_rate': '5.284e-05', 'entropy': '0.7503', 'num_tokens': '4.361e+07', 'mean_token_accuracy': '0.7998', 'epoch': '0.07153'}
{'loss': '0.6025', 'grad_norm': '0.2968', 'learning_rate': '5.282e-05', 'entropy': '0.6242', 'num_tokens': '4.362e+07', 'mean_token_accuracy': '0.8307', 'epoch': '0.07154'}
{'loss': '0.8589', 'grad_norm': '0.4223', 'learning_rate': '5.279e-05', 'entropy': '0.9052', 'num_tokens': '4.362e+07', 'mean_token_accuracy': '0.772', 'epoch': '0.07155'}
{'loss': '0.7791', 'grad_norm': '0.5022', 'learning_rate': '5.277e-05', 'entropy': '0.8291', 'num_tokens': '4.363e+07', 'mean_token_accuracy': '0.7971', 'epoch': '0.07156'}
{'loss': '0.5742', 'grad_norm': '0.3812', 'learning_rate': '5.274e-05', 'entropy': '0.6235', 'num_tokens': '4.363e+07', 'mean_token_accuracy': '0.8357', 'epoch': '0.07157'}

 67%|██████▋   | 8088/12000 [4:47:09<2:07:34,  1.96s/it]
 67%|██████▋   | 8089/12000 [4:47:11<1:56:42,  1.79s/it]
                                                        

 67%|██████▋   | 8089/12000 [4:47:11<1:56:42,  1.79s/it]
 67%|██████▋   | 8090/12000 [4:47:13<1:58:23,  1.82s/it]
                                                        

 67%|██████▋   | 8090/12000 [4:47:13<1:58:23,  1.82s/it]
 67%|██████▋   | 8091/12000 [4:47:14<1:57:29,  1.80s/it]
                                                        

 67%|██████▋   | 8091/12000 [4:47:14<1:57:29,  1.80s/it]
 67%|██████▋   | 8092/12000 [4:47:16<1:53:34,  1.74s/it]
                                                        

 67%|██████▋   | 8092/12000 [4:47:16<1:53:34,  1.74s/it]
 67%|██████▋   | 8093/12000 [4:47:18<1:49:07,  1.68s/it]
                                                        

 67%|██████▋   | 8093/12000 [4:47:18<1:49:07,  1.68s/it]
 67%|██████▋   | 8094/12000 [4:47:21<2:29:32,  2.30s/it]
                                                        

 67%|██████▋   | 8094/12000 [4:47:21<2:29:32,  2.30s/it]
 67%|██████▋   | 8095/12000 [4:47:24<2:28:54,  2.29s/it]
                                                        

 67%|██████▋   | 8095/12000 [4:47:24<2:28:54,  2.29s/it]
 67%|██████▋   | 8096/12000 [4:47:26<2:32:56,  2.35s/it]
                                                        

 67%|██████▋   | 8096/12000 [4:47:26<2:32:56,  2.35s/it]
 67%|██████▋   | 8097/12000 [4:47:28<2:30:29,  2.31s/it]
                                                        

 67%|██████▋   | 8097/12000 [4:47:28<2:30:29,  2.31s/it]
 67%|██████▋   | 8098/12000 [4:47:32<2:58:35,  2.75s/it]
                                                        

 67%|██████▋   | 8098/12000 [4:47:32<2:58:35,  2.75s/it]
 67%|██████▋   | 8099/12000 [4:47:34<2:32:30,  2.35s/it]
                                                        

 67%|██████▋   | 8099/12000 [4:47:34<2:32:30,  2.35s/it]
 68%|██████▊   | 8100/12000 [4:47:35<2:23:19,  2.20s/it]
                                                        

 68%|██████▊   | 8100/12000 [4:47:35<2:23:19,  2.20s/it]
 68%|██████▊   | 8101/12000 [4:47:38<2:39:33,  2.46s/it]
                                                        

 68%|██████▊   | 8101/12000 [4:47:38<2:39:33,  2.46s/it]
 68%|██████▊   | 8102/12000 [4:47:40<2:22:57,  2.20s/it]
                                                        

 68%|██████▊   | 8102/12000 [4:47:40<2:22:57,  2.20s/it]
 68%|██████▊   | 8103/12000 [4:47:42<2:11:23,  2.02s/it]
                                                        

 68%|██████▊   | 8103/12000 [4:47:42<2:11:23,  2.02s/it]
 68%|██████▊   | 8104/12000 [4:47:43<2:07:03,  1.96s/it]
                                                        

 68%|██████▊   | 8104/12000 [4:47:43<2:07:03,  1.96s/it]
 68%|██████▊   | 8105/12000 [4:47:45<1:59:49,  1.85s/it]
                                                        

 68%|██████▊   | 8105/12000 [4:47:45<1:59:49,  1.85s/it]
 68%|██████▊   | 8106/12000 [4:47:46<1:51:06,  1.71s/it]
                                                        

 68%|██████▊   | 8106/12000 [4:47:46<1:51:06,  1.71s/it]
 68%|██████▊   | 8107/12000 [4:47:50<2:26:48,  2.26s/it]
                                                        

 68%|██████▊   | 8107/12000 [4:47:50<2:26:48,  2.26s/it]
 68%|██████▊   | 8108/12000 [4:47:52<2:15:38,  2.09s/it]
                                                        

 68%|██████▊   | 8108/12000 [4:47:52<2:15:38,  2.09s/it]
 68%|██████▊   | 8109/12000 [4:47:53<2:00:46,  1.86s/it]
                                                        

 68%|██████▊   | 8109/12000 [4:47:53<2:00:46,  1.86s/it]
 68%|██████▊   | 8110/12000 [4:47:54<1:51:11,  1.72s/it]
                                                        

 68%|██████▊   | 8110/12000 [4:47:54<1:51:11,  1.72s/it]
 68%|██████▊   | 8111/12000 [4:47:56<1:47:35,  1.66s/it]
                                                        

 68%|██████▊   | 8111/12000 [4:47:56<1:47:35,  1.66s/it]
 68%|██████▊   | 8112/12000 [4:47:57<1:43:02,  1.59s/it]
                                                        

 68%|██████▊   | 8112/12000 [4:47:57<1:43:02,  1.59s/it]
 68%|██████▊   | 8113/12000 [4:47:59<1:42:30,  1.58s/it]
                                                        

 68%|██████▊   | 8113/12000 [4:47:59<1:42:30,  1.58s/it]
 68%|██████▊   | 8114/12000 [4:48:02<2:03:05,  1.90s/it]
                                                        

 68%|██████▊   | 8114/12000 [4:48:02<2:03:05,  1.90s/it]
 68%|██████▊   | 8115/12000 [4:48:03<1:53:56,  1.76s/it]
                                                        

 68%|██████▊   | 8115/12000 [4:48:03<1:53:56,  1.76s/it]
 68%|██████▊   | 8116/12000 [4:48:06<2:09:39,  2.00s/it]
                                                        

 68%|██████▊   | 8116/12000 [4:48:06<2:09:39,  2.00s/it]
 68%|██████▊   | 8117/12000 [4:48:08<2:13:20,  2.06s/it]
                                                        

 68%|██████▊   | 8117/12000 [4:48:08<2:13:20,  2.06s/it]
 68%|██████▊   | 8118/12000 [4:48:11<2:33:21,  2.37s/it]
                                                        

 68%|██████▊   | 8118/12000 [4:48:11<2:33:21,  2.37s/it]
 68%|██████▊   | 8119/12000 [4:48:13<2:20:51,  2.18s/it]
                                                        

 68%|██████▊   | 8119/12000 [4:48:13<2:20:51,  2.18s/it]
 68%|██████▊   | 8120/12000 [4:48:14<2:14:29,  2.08s/it]
                                                        

 68%|██████▊   | 8120/12000 [4:48:14<2:14:29,  2.08s/it]
 68%|██████▊   | 8121/12000 [4:48:17<2:23:21,  2.22s/it]
                                                        

 68%|██████▊   | 8121/12000 [4:48:17<2:23:21,  2.22s/it]
 68%|██████▊   | 8122/12000 [4:48:19<2:14:32,  2.08s/it]
                                                        

 68%|██████▊   | 8122/12000 [4:48:19<2:14:32,  2.08s/it]
 68%|██████▊   | 8123/12000 [4:48:20<2:02:17,  1.89s/it]
                                                        

 68%|██████▊   | 8123/12000 [4:48:20<2:02:17,  1.89s/it]
 68%|██████▊   | 8124/12000 [4:48:22<1:54:28,  1.77s/it]
                                                        

 68%|██████▊   | 8124/12000 [4:48:22<1:54:28,  1.77s/it]
 68%|██████▊   | 8125/12000 [4:48:23<1:49:07,  1.69s/it]
                                                        

 68%|██████▊   | 8125/12000 [4:48:23<1:49:07,  1.69s/it]
 68%|██████▊   | 8126/12000 [4:48:25<1:45:07,  1.63s/it]
                                                        

 68%|██████▊   | 8126/12000 [4:48:25<1:45:07,  1.63s/it]
 68%|██████▊   | 8127/12000 [4:48:26<1:42:12,  1.58s/it]
                                                        

 68%|██████▊   | 8127/12000 [4:48:26<1:42:12,  1.58s/it]
 68%|██████▊   | 8128/12000 [4:48:28<1:44:21,  1.62s/it]
                                                        

 68%|██████▊   | 8128/12000 [4:48:28<1:44:21,  1.62s/it]
 68%|██████▊   | 8129/12000 [4:48:32<2:40:52,  2.49s/it]
                                                        

 68%|██████▊   | 8129/12000 [4:48:32<2:40:52,  2.49s/it]
 68%|██████▊   | 8130/12000 [4:48:34<2:27:15,  2.28s/it]
                                                        

 68%|██████▊   | 8130/12000 [4:48:34<2:27:15,  2.28s/it]
 68%|██████▊   | 8131/12000 [4:48:37<2:33:09,  2.38s/it]
                                                        

 68%|██████▊   | 8131/12000 [4:48:37<2:33:09,  2.38s/it]
 68%|██████▊   | 8132/12000 [4:48:38<2:17:44,  2.14s/it]
                                                        

 68%|██████▊   | 8132/12000 [4:48:38<2:17:44,  2.14s/it]
 68%|██████▊   | 8133/12000 [4:48:40<2:06:45,  1.97s/it]
                                                        

 68%|██████▊   | 8133/12000 [4:48:40<2:06:45,  1.97s/it]
 68%|██████▊   | 8134/12000 [4:48:42<2:03:48,  1.92s/it]
                                                        

 68%|██████▊   | 8134/12000 [4:48:42<2:03:48,  1.92s/it]
 68%|██████▊   | 8135/12000 [4:48:45<2:26:32,  2.28s/it]
                                                        
{'loss': '0.9994', 'grad_norm': '0.4077', 'learning_rate': '5.272e-05', 'entropy': '0.9515', 'num_tokens': '4.363e+07', 'mean_token_accuracy': '0.7531', 'epoch': '0.07158'}
{'loss': '1.138', 'grad_norm': '0.4761', 'learning_rate': '5.27e-05', 'entropy': '1.138', 'num_tokens': '4.364e+07', 'mean_token_accuracy': '0.7356', 'epoch': '0.07159'}
{'loss': '0.9272', 'grad_norm': '0.4509', 'learning_rate': '5.267e-05', 'entropy': '0.9141', 'num_tokens': '4.364e+07', 'mean_token_accuracy': '0.7758', 'epoch': '0.0716'}
{'loss': '0.6932', 'grad_norm': '0.3599', 'learning_rate': '5.265e-05', 'entropy': '0.7142', 'num_tokens': '4.365e+07', 'mean_token_accuracy': '0.8228', 'epoch': '0.0716'}
{'loss': '0.9623', 'grad_norm': '0.3657', 'learning_rate': '5.262e-05', 'entropy': '0.8504', 'num_tokens': '4.365e+07', 'mean_token_accuracy': '0.7904', 'epoch': '0.07161'}
{'loss': '0.6352', 'grad_norm': '0.3772', 'learning_rate': '5.26e-05', 'entropy': '0.6579', 'num_tokens': '4.366e+07', 'mean_token_accuracy': '0.8306', 'epoch': '0.07162'}
{'loss': '0.9415', 'grad_norm': '0.3709', 'learning_rate': '5.258e-05', 'entropy': '0.9902', 'num_tokens': '4.367e+07', 'mean_token_accuracy': '0.7595', 'epoch': '0.07163'}
{'loss': '1.062', 'grad_norm': '0.4789', 'learning_rate': '5.255e-05', 'entropy': '1.021', 'num_tokens': '4.367e+07', 'mean_token_accuracy': '0.7451', 'epoch': '0.07164'}
{'loss': '0.8004', 'grad_norm': '0.4499', 'learning_rate': '5.253e-05', 'entropy': '0.6726', 'num_tokens': '4.368e+07', 'mean_token_accuracy': '0.8343', 'epoch': '0.07165'}
{'loss': '0.7051', 'grad_norm': '0.3678', 'learning_rate': '5.25e-05', 'entropy': '0.716', 'num_tokens': '4.368e+07', 'mean_token_accuracy': '0.8158', 'epoch': '0.07166'}
{'loss': '1.159', 'grad_norm': '0.3185', 'learning_rate': '5.248e-05', 'entropy': '1.357', 'num_tokens': '4.369e+07', 'mean_token_accuracy': '0.6981', 'epoch': '0.07167'}
{'loss': '0.6617', 'grad_norm': '0.3111', 'learning_rate': '5.245e-05', 'entropy': '0.6823', 'num_tokens': '4.37e+07', 'mean_token_accuracy': '0.8187', 'epoch': '0.07167'}
{'loss': '0.9998', 'grad_norm': '0.411', 'learning_rate': '5.243e-05', 'entropy': '0.9017', 'num_tokens': '4.37e+07', 'mean_token_accuracy': '0.7808', 'epoch': '0.07168'}
{'loss': '0.5988', 'grad_norm': '0.367', 'learning_rate': '5.241e-05', 'entropy': '0.613', 'num_tokens': '4.371e+07', 'mean_token_accuracy': '0.8323', 'epoch': '0.07169'}
{'loss': '0.9135', 'grad_norm': '0.3834', 'learning_rate': '5.238e-05', 'entropy': '0.8558', 'num_tokens': '4.371e+07', 'mean_token_accuracy': '0.7804', 'epoch': '0.0717'}
{'loss': '0.6749', 'grad_norm': '0.3857', 'learning_rate': '5.236e-05', 'entropy': '0.6851', 'num_tokens': '4.372e+07', 'mean_token_accuracy': '0.8329', 'epoch': '0.07171'}
{'loss': '0.708', 'grad_norm': '0.4368', 'learning_rate': '5.233e-05', 'entropy': '0.771', 'num_tokens': '4.372e+07', 'mean_token_accuracy': '0.8032', 'epoch': '0.07172'}
{'loss': '1.049', 'grad_norm': '0.4816', 'learning_rate': '5.231e-05', 'entropy': '1.156', 'num_tokens': '4.373e+07', 'mean_token_accuracy': '0.7266', 'epoch': '0.07173'}
{'loss': '0.8059', 'grad_norm': '0.4137', 'learning_rate': '5.228e-05', 'entropy': '0.8017', 'num_tokens': '4.373e+07', 'mean_token_accuracy': '0.8042', 'epoch': '0.07174'}
{'loss': '0.9895', 'grad_norm': '0.3839', 'learning_rate': '5.226e-05', 'entropy': '0.9402', 'num_tokens': '4.374e+07', 'mean_token_accuracy': '0.7768', 'epoch': '0.07175'}
{'loss': '0.8139', 'grad_norm': '0.4122', 'learning_rate': '5.224e-05', 'entropy': '0.8206', 'num_tokens': '4.374e+07', 'mean_token_accuracy': '0.7889', 'epoch': '0.07175'}
{'loss': '0.7311', 'grad_norm': '0.4532', 'learning_rate': '5.221e-05', 'entropy': '0.7579', 'num_tokens': '4.374e+07', 'mean_token_accuracy': '0.7984', 'epoch': '0.07176'}
{'loss': '0.807', 'grad_norm': '0.3325', 'learning_rate': '5.219e-05', 'entropy': '0.924', 'num_tokens': '4.375e+07', 'mean_token_accuracy': '0.7725', 'epoch': '0.07177'}
{'loss': '1.053', 'grad_norm': '0.4382', 'learning_rate': '5.216e-05', 'entropy': '1.04', 'num_tokens': '4.375e+07', 'mean_token_accuracy': '0.7474', 'epoch': '0.07178'}
{'loss': '0.9367', 'grad_norm': '0.4306', 'learning_rate': '5.214e-05', 'entropy': '0.9376', 'num_tokens': '4.376e+07', 'mean_token_accuracy': '0.7574', 'epoch': '0.07179'}
{'loss': '0.904', 'grad_norm': '0.3978', 'learning_rate': '5.211e-05', 'entropy': '0.9248', 'num_tokens': '4.376e+07', 'mean_token_accuracy': '0.7666', 'epoch': '0.0718'}
{'loss': '0.6712', 'grad_norm': '0.3414', 'learning_rate': '5.209e-05', 'entropy': '0.6942', 'num_tokens': '4.377e+07', 'mean_token_accuracy': '0.8319', 'epoch': '0.07181'}
{'loss': '0.9014', 'grad_norm': '0.4033', 'learning_rate': '5.207e-05', 'entropy': '0.8993', 'num_tokens': '4.377e+07', 'mean_token_accuracy': '0.7703', 'epoch': '0.07182'}
{'loss': '0.6159', 'grad_norm': '0.3742', 'learning_rate': '5.204e-05', 'entropy': '0.6062', 'num_tokens': '4.378e+07', 'mean_token_accuracy': '0.8405', 'epoch': '0.07183'}
{'loss': '0.9403', 'grad_norm': '0.4808', 'learning_rate': '5.202e-05', 'entropy': '0.8495', 'num_tokens': '4.378e+07', 'mean_token_accuracy': '0.7917', 'epoch': '0.07183'}
{'loss': '0.573', 'grad_norm': '0.3314', 'learning_rate': '5.199e-05', 'entropy': '0.6238', 'num_tokens': '4.379e+07', 'mean_token_accuracy': '0.8371', 'epoch': '0.07184'}
{'loss': '0.6976', 'grad_norm': '0.3353', 'learning_rate': '5.197e-05', 'entropy': '0.6977', 'num_tokens': '4.38e+07', 'mean_token_accuracy': '0.8179', 'epoch': '0.07185'}
{'loss': '0.7916', 'grad_norm': '0.396', 'learning_rate': '5.195e-05', 'entropy': '0.8155', 'num_tokens': '4.38e+07', 'mean_token_accuracy': '0.7998', 'epoch': '0.07186'}
{'loss': '0.6705', 'grad_norm': '0.4156', 'learning_rate': '5.192e-05', 'entropy': '0.6805', 'num_tokens': '4.381e+07', 'mean_token_accuracy': '0.8223', 'epoch': '0.07187'}
{'loss': '1.076', 'grad_norm': '0.7447', 'learning_rate': '5.19e-05', 'entropy': '1.073', 'num_tokens': '4.381e+07', 'mean_token_accuracy': '0.741', 'epoch': '0.07188'}
{'loss': '0.7213', 'grad_norm': '0.3027', 'learning_rate': '5.187e-05', 'entropy': '0.7736', 'num_tokens': '4.382e+07', 'mean_token_accuracy': '0.8067', 'epoch': '0.07189'}
{'loss': '0.6834', 'grad_norm': '0.3403', 'learning_rate': '5.185e-05', 'entropy': '0.6744', 'num_tokens': '4.382e+07', 'mean_token_accuracy': '0.8198', 'epoch': '0.0719'}
{'loss': '0.8227', 'grad_norm': '0.3959', 'learning_rate': '5.182e-05', 'entropy': '0.863', 'num_tokens': '4.382e+07', 'mean_token_accuracy': '0.7789', 'epoch': '0.0719'}
{'loss': '0.8512', 'grad_norm': '0.3552', 'learning_rate': '5.18e-05', 'entropy': '0.8249', 'num_tokens': '4.383e+07', 'mean_token_accuracy': '0.7853', 'epoch': '0.07191'}
{'loss': '0.5603', 'grad_norm': '0.3271', 'learning_rate': '5.178e-05', 'entropy': '0.5625', 'num_tokens': '4.383e+07', 'mean_token_accuracy': '0.8565', 'epoch': '0.07192'}
{'loss': '0.9245', 'grad_norm': '0.4179', 'learning_rate': '5.175e-05', 'entropy': '0.9275', 'num_tokens': '4.384e+07', 'mean_token_accuracy': '0.7602', 'epoch': '0.07193'}
{'loss': '0.7783', 'grad_norm': '0.3917', 'learning_rate': '5.173e-05', 'entropy': '0.772', 'num_tokens': '4.385e+07', 'mean_token_accuracy': '0.8165', 'epoch': '0.07194'}
{'loss': '0.6093', 'grad_norm': '0.3482', 'learning_rate': '5.17e-05', 'entropy': '0.5678', 'num_tokens': '4.385e+07', 'mean_token_accuracy': '0.8468', 'epoch': '0.07195'}
{'loss': '0.6528', 'grad_norm': '0.4339', 'learning_rate': '5.168e-05', 'entropy': '0.6415', 'num_tokens': '4.386e+07', 'mean_token_accuracy': '0.8396', 'epoch': '0.07196'}
{'loss': '1.231', 'grad_norm': '0.5337', 'learning_rate': '5.166e-05', 'entropy': '1.191', 'num_tokens': '4.386e+07', 'mean_token_accuracy': '0.7175', 'epoch': '0.07197'}
{'loss': '0.6261', 'grad_norm': '0.3437', 'learning_rate': '5.163e-05', 'entropy': '0.6714', 'num_tokens': '4.387e+07', 'mean_token_accuracy': '0.8236', 'epoch': '0.07198'}
{'loss': '0.7085', 'grad_norm': '0.3654', 'learning_rate': '5.161e-05', 'entropy': '0.669', 'num_tokens': '4.387e+07', 'mean_token_accuracy': '0.836', 'epoch': '0.07198'}

 68%|██████▊   | 8135/12000 [4:48:45<2:26:32,  2.28s/it]
 68%|██████▊   | 8136/12000 [4:48:46<2:10:44,  2.03s/it]
                                                        

 68%|██████▊   | 8136/12000 [4:48:46<2:10:44,  2.03s/it]
 68%|██████▊   | 8137/12000 [4:48:48<1:58:49,  1.85s/it]
                                                        

 68%|██████▊   | 8137/12000 [4:48:48<1:58:49,  1.85s/it]
 68%|██████▊   | 8138/12000 [4:48:49<1:49:42,  1.70s/it]
                                                        

 68%|██████▊   | 8138/12000 [4:48:49<1:49:42,  1.70s/it]
 68%|██████▊   | 8139/12000 [4:48:51<2:02:19,  1.90s/it]
                                                        

 68%|██████▊   | 8139/12000 [4:48:51<2:02:19,  1.90s/it]
 68%|██████▊   | 8140/12000 [4:48:53<1:53:35,  1.77s/it]
                                                        

 68%|██████▊   | 8140/12000 [4:48:53<1:53:35,  1.77s/it]
 68%|██████▊   | 8141/12000 [4:48:54<1:45:59,  1.65s/it]
                                                        

 68%|██████▊   | 8141/12000 [4:48:54<1:45:59,  1.65s/it]
 68%|██████▊   | 8142/12000 [4:48:56<1:49:05,  1.70s/it]
                                                        

 68%|██████▊   | 8142/12000 [4:48:56<1:49:05,  1.70s/it]
 68%|██████▊   | 8143/12000 [4:48:59<2:04:14,  1.93s/it]
                                                        

 68%|██████▊   | 8143/12000 [4:48:59<2:04:14,  1.93s/it]
 68%|██████▊   | 8144/12000 [4:49:00<2:00:18,  1.87s/it]
                                                        

 68%|██████▊   | 8144/12000 [4:49:00<2:00:18,  1.87s/it]
 68%|██████▊   | 8145/12000 [4:49:02<1:55:12,  1.79s/it]
                                                        

 68%|██████▊   | 8145/12000 [4:49:02<1:55:12,  1.79s/it]
 68%|██████▊   | 8146/12000 [4:49:03<1:51:00,  1.73s/it]
                                                        

 68%|██████▊   | 8146/12000 [4:49:03<1:51:00,  1.73s/it]
 68%|██████▊   | 8147/12000 [4:49:05<1:45:01,  1.64s/it]
                                                        

 68%|██████▊   | 8147/12000 [4:49:05<1:45:01,  1.64s/it]
 68%|██████▊   | 8148/12000 [4:49:07<1:45:15,  1.64s/it]
                                                        

 68%|██████▊   | 8148/12000 [4:49:07<1:45:15,  1.64s/it]
 68%|██████▊   | 8149/12000 [4:49:08<1:44:13,  1.62s/it]
                                                        

 68%|██████▊   | 8149/12000 [4:49:08<1:44:13,  1.62s/it]
 68%|██████▊   | 8150/12000 [4:49:13<2:42:08,  2.53s/it]
                                                        

 68%|██████▊   | 8150/12000 [4:49:13<2:42:08,  2.53s/it]
 68%|██████▊   | 8151/12000 [4:49:18<3:34:43,  3.35s/it]
                                                        

 68%|██████▊   | 8151/12000 [4:49:18<3:34:43,  3.35s/it]
 68%|██████▊   | 8152/12000 [4:49:20<3:02:39,  2.85s/it]
                                                        

 68%|██████▊   | 8152/12000 [4:49:20<3:02:39,  2.85s/it]
 68%|██████▊   | 8153/12000 [4:49:21<2:35:32,  2.43s/it]
                                                        

 68%|██████▊   | 8153/12000 [4:49:21<2:35:32,  2.43s/it]
 68%|██████▊   | 8154/12000 [4:49:23<2:18:34,  2.16s/it]
                                                        

 68%|██████▊   | 8154/12000 [4:49:23<2:18:34,  2.16s/it]
 68%|██████▊   | 8155/12000 [4:49:24<2:04:39,  1.95s/it]
                                                        

 68%|██████▊   | 8155/12000 [4:49:24<2:04:39,  1.95s/it]
 68%|██████▊   | 8156/12000 [4:49:26<1:54:45,  1.79s/it]
                                                        

 68%|██████▊   | 8156/12000 [4:49:26<1:54:45,  1.79s/it]
 68%|██████▊   | 8157/12000 [4:49:27<1:48:32,  1.69s/it]
                                                        

 68%|██████▊   | 8157/12000 [4:49:27<1:48:32,  1.69s/it]
 68%|██████▊   | 8158/12000 [4:49:28<1:42:45,  1.60s/it]
                                                        

 68%|██████▊   | 8158/12000 [4:49:28<1:42:45,  1.60s/it]
 68%|██████▊   | 8159/12000 [4:49:30<1:42:35,  1.60s/it]
                                                        

 68%|██████▊   | 8159/12000 [4:49:30<1:42:35,  1.60s/it]
 68%|██████▊   | 8160/12000 [4:49:32<1:48:00,  1.69s/it]
                                                        

 68%|██████▊   | 8160/12000 [4:49:32<1:48:00,  1.69s/it]
 68%|██████▊   | 8161/12000 [4:49:33<1:46:15,  1.66s/it]
                                                        

 68%|██████▊   | 8161/12000 [4:49:33<1:46:15,  1.66s/it]
 68%|██████▊   | 8162/12000 [4:49:35<1:47:44,  1.68s/it]
                                                        

 68%|██████▊   | 8162/12000 [4:49:35<1:47:44,  1.68s/it]
 68%|██████▊   | 8163/12000 [4:49:37<1:48:58,  1.70s/it]
                                                        

 68%|██████▊   | 8163/12000 [4:49:37<1:48:58,  1.70s/it]
 68%|██████▊   | 8164/12000 [4:49:38<1:45:20,  1.65s/it]
                                                        

 68%|██████▊   | 8164/12000 [4:49:38<1:45:20,  1.65s/it]
 68%|██████▊   | 8165/12000 [4:49:40<1:40:42,  1.58s/it]
                                                        

 68%|██████▊   | 8165/12000 [4:49:40<1:40:42,  1.58s/it]
 68%|██████▊   | 8166/12000 [4:49:42<1:43:19,  1.62s/it]
                                                        

 68%|██████▊   | 8166/12000 [4:49:42<1:43:19,  1.62s/it]
 68%|██████▊   | 8167/12000 [4:49:43<1:46:18,  1.66s/it]
                                                        

 68%|██████▊   | 8167/12000 [4:49:43<1:46:18,  1.66s/it]
 68%|██████▊   | 8168/12000 [4:49:46<2:04:34,  1.95s/it]
                                                        

 68%|██████▊   | 8168/12000 [4:49:46<2:04:34,  1.95s/it]
 68%|██████▊   | 8169/12000 [4:49:48<1:57:02,  1.83s/it]
                                                        

 68%|██████▊   | 8169/12000 [4:49:48<1:57:02,  1.83s/it]
 68%|██████▊   | 8170/12000 [4:49:50<2:01:56,  1.91s/it]
                                                        

 68%|██████▊   | 8170/12000 [4:49:50<2:01:56,  1.91s/it]
 68%|██████▊   | 8171/12000 [4:49:52<2:02:46,  1.92s/it]
                                                        

 68%|██████▊   | 8171/12000 [4:49:52<2:02:46,  1.92s/it]
 68%|██████▊   | 8172/12000 [4:49:53<1:55:25,  1.81s/it]
                                                        

 68%|██████▊   | 8172/12000 [4:49:53<1:55:25,  1.81s/it]
 68%|██████▊   | 8173/12000 [4:49:56<2:24:26,  2.26s/it]
                                                        

 68%|██████▊   | 8173/12000 [4:49:56<2:24:26,  2.26s/it]
 68%|██████▊   | 8174/12000 [4:49:58<2:18:49,  2.18s/it]
                                                        

 68%|██████▊   | 8174/12000 [4:49:58<2:18:49,  2.18s/it]
 68%|██████▊   | 8175/12000 [4:50:00<2:12:05,  2.07s/it]
                                                        

 68%|██████▊   | 8175/12000 [4:50:00<2:12:05,  2.07s/it]
 68%|██████▊   | 8176/12000 [4:50:02<2:10:50,  2.05s/it]
                                                        

 68%|██████▊   | 8176/12000 [4:50:02<2:10:50,  2.05s/it]
 68%|██████▊   | 8177/12000 [4:50:04<1:59:50,  1.88s/it]
                                                        

 68%|██████▊   | 8177/12000 [4:50:04<1:59:50,  1.88s/it]
 68%|██████▊   | 8178/12000 [4:50:05<1:50:41,  1.74s/it]
                                                        

 68%|██████▊   | 8178/12000 [4:50:05<1:50:41,  1.74s/it]
 68%|██████▊   | 8179/12000 [4:50:07<1:46:46,  1.68s/it]
                                                        

 68%|██████▊   | 8179/12000 [4:50:07<1:46:46,  1.68s/it]
 68%|██████▊   | 8180/12000 [4:50:10<2:14:59,  2.12s/it]
                                                        

 68%|██████▊   | 8180/12000 [4:50:10<2:14:59,  2.12s/it]
 68%|██████▊   | 8181/12000 [4:50:11<2:01:50,  1.91s/it]
                                                        

 68%|██████▊   | 8181/12000 [4:50:11<2:01:50,  1.91s/it]
 68%|██████▊   | 8182/12000 [4:50:13<1:57:42,  1.85s/it]
                                                        
{'loss': '1.077', 'grad_norm': '0.4096', 'learning_rate': '5.158e-05', 'entropy': '1.115', 'num_tokens': '4.388e+07', 'mean_token_accuracy': '0.7311', 'epoch': '0.07199'}
{'loss': '1.008', 'grad_norm': '0.3903', 'learning_rate': '5.156e-05', 'entropy': '0.9988', 'num_tokens': '4.388e+07', 'mean_token_accuracy': '0.7458', 'epoch': '0.072'}
{'loss': '0.7051', 'grad_norm': '0.3039', 'learning_rate': '5.154e-05', 'entropy': '0.6851', 'num_tokens': '4.389e+07', 'mean_token_accuracy': '0.8135', 'epoch': '0.07201'}
{'loss': '0.6185', 'grad_norm': '0.3782', 'learning_rate': '5.151e-05', 'entropy': '0.6581', 'num_tokens': '4.389e+07', 'mean_token_accuracy': '0.8198', 'epoch': '0.07202'}
{'loss': '0.8679', 'grad_norm': '0.424', 'learning_rate': '5.149e-05', 'entropy': '0.8315', 'num_tokens': '4.39e+07', 'mean_token_accuracy': '0.7881', 'epoch': '0.07203'}
{'loss': '0.6488', 'grad_norm': '0.3996', 'learning_rate': '5.146e-05', 'entropy': '0.6989', 'num_tokens': '4.39e+07', 'mean_token_accuracy': '0.8253', 'epoch': '0.07204'}
{'loss': '0.6709', 'grad_norm': '0.3201', 'learning_rate': '5.144e-05', 'entropy': '0.7292', 'num_tokens': '4.391e+07', 'mean_token_accuracy': '0.8113', 'epoch': '0.07205'}
{'loss': '0.6845', 'grad_norm': '0.3703', 'learning_rate': '5.141e-05', 'entropy': '0.6538', 'num_tokens': '4.391e+07', 'mean_token_accuracy': '0.8311', 'epoch': '0.07206'}
{'loss': '0.5001', 'grad_norm': '0.3512', 'learning_rate': '5.139e-05', 'entropy': '0.4761', 'num_tokens': '4.392e+07', 'mean_token_accuracy': '0.8761', 'epoch': '0.07206'}
{'loss': '0.7913', 'grad_norm': '0.4322', 'learning_rate': '5.137e-05', 'entropy': '0.7667', 'num_tokens': '4.392e+07', 'mean_token_accuracy': '0.8153', 'epoch': '0.07207'}
{'loss': '0.9499', 'grad_norm': '0.346', 'learning_rate': '5.134e-05', 'entropy': '0.8758', 'num_tokens': '4.393e+07', 'mean_token_accuracy': '0.7811', 'epoch': '0.07208'}
{'loss': '0.6073', 'grad_norm': '0.3313', 'learning_rate': '5.132e-05', 'entropy': '0.6476', 'num_tokens': '4.393e+07', 'mean_token_accuracy': '0.829', 'epoch': '0.07209'}
{'loss': '0.8577', 'grad_norm': '0.3605', 'learning_rate': '5.129e-05', 'entropy': '0.9178', 'num_tokens': '4.394e+07', 'mean_token_accuracy': '0.7692', 'epoch': '0.0721'}
{'loss': '0.7462', 'grad_norm': '0.3853', 'learning_rate': '5.127e-05', 'entropy': '0.7471', 'num_tokens': '4.394e+07', 'mean_token_accuracy': '0.8057', 'epoch': '0.07211'}
{'loss': '1.084', 'grad_norm': '0.4248', 'learning_rate': '5.125e-05', 'entropy': '1.091', 'num_tokens': '4.395e+07', 'mean_token_accuracy': '0.7392', 'epoch': '0.07212'}
{'loss': '1.02', 'grad_norm': '0.339', 'learning_rate': '5.122e-05', 'entropy': '0.8867', 'num_tokens': '4.396e+07', 'mean_token_accuracy': '0.7876', 'epoch': '0.07213'}
{'loss': '0.9526', 'grad_norm': '0.4297', 'learning_rate': '5.12e-05', 'entropy': '0.9045', 'num_tokens': '4.396e+07', 'mean_token_accuracy': '0.7778', 'epoch': '0.07213'}
{'loss': '0.9529', 'grad_norm': '0.3963', 'learning_rate': '5.117e-05', 'entropy': '0.8839', 'num_tokens': '4.396e+07', 'mean_token_accuracy': '0.7773', 'epoch': '0.07214'}
{'loss': '0.7194', 'grad_norm': '0.5598', 'learning_rate': '5.115e-05', 'entropy': '0.7286', 'num_tokens': '4.397e+07', 'mean_token_accuracy': '0.8104', 'epoch': '0.07215'}
{'loss': '0.7348', 'grad_norm': '0.3871', 'learning_rate': '5.113e-05', 'entropy': '0.7419', 'num_tokens': '4.397e+07', 'mean_token_accuracy': '0.8119', 'epoch': '0.07216'}
{'loss': '0.5022', 'grad_norm': '0.32', 'learning_rate': '5.11e-05', 'entropy': '0.594', 'num_tokens': '4.398e+07', 'mean_token_accuracy': '0.8553', 'epoch': '0.07217'}
{'loss': '0.7605', 'grad_norm': '0.4279', 'learning_rate': '5.108e-05', 'entropy': '0.8066', 'num_tokens': '4.398e+07', 'mean_token_accuracy': '0.8052', 'epoch': '0.07218'}
{'loss': '0.8022', 'grad_norm': '0.4373', 'learning_rate': '5.105e-05', 'entropy': '0.7953', 'num_tokens': '4.399e+07', 'mean_token_accuracy': '0.7896', 'epoch': '0.07219'}
{'loss': '1.039', 'grad_norm': '0.5463', 'learning_rate': '5.103e-05', 'entropy': '0.9543', 'num_tokens': '4.399e+07', 'mean_token_accuracy': '0.7646', 'epoch': '0.0722'}
{'loss': '0.7645', 'grad_norm': '0.4806', 'learning_rate': '5.101e-05', 'entropy': '0.8414', 'num_tokens': '4.399e+07', 'mean_token_accuracy': '0.7846', 'epoch': '0.07221'}
{'loss': '1.184', 'grad_norm': '0.3859', 'learning_rate': '5.098e-05', 'entropy': '1.119', 'num_tokens': '4.4e+07', 'mean_token_accuracy': '0.7278', 'epoch': '0.07221'}
{'loss': '0.9615', 'grad_norm': '0.3911', 'learning_rate': '5.096e-05', 'entropy': '0.9018', 'num_tokens': '4.4e+07', 'mean_token_accuracy': '0.7887', 'epoch': '0.07222'}
{'loss': '0.8303', 'grad_norm': '0.4743', 'learning_rate': '5.093e-05', 'entropy': '0.8243', 'num_tokens': '4.401e+07', 'mean_token_accuracy': '0.7963', 'epoch': '0.07223'}
{'loss': '0.6778', 'grad_norm': '0.4167', 'learning_rate': '5.091e-05', 'entropy': '0.6724', 'num_tokens': '4.401e+07', 'mean_token_accuracy': '0.8248', 'epoch': '0.07224'}
{'loss': '0.8751', 'grad_norm': '0.3412', 'learning_rate': '5.089e-05', 'entropy': '0.8906', 'num_tokens': '4.402e+07', 'mean_token_accuracy': '0.7825', 'epoch': '0.07225'}
{'loss': '0.6946', 'grad_norm': '0.3307', 'learning_rate': '5.086e-05', 'entropy': '0.8173', 'num_tokens': '4.402e+07', 'mean_token_accuracy': '0.7916', 'epoch': '0.07226'}
{'loss': '0.7653', 'grad_norm': '0.4467', 'learning_rate': '5.084e-05', 'entropy': '0.7869', 'num_tokens': '4.403e+07', 'mean_token_accuracy': '0.7956', 'epoch': '0.07227'}
{'loss': '0.6362', 'grad_norm': '0.3757', 'learning_rate': '5.081e-05', 'entropy': '0.6477', 'num_tokens': '4.403e+07', 'mean_token_accuracy': '0.835', 'epoch': '0.07228'}
{'loss': '1.174', 'grad_norm': '0.3992', 'learning_rate': '5.079e-05', 'entropy': '1.044', 'num_tokens': '4.404e+07', 'mean_token_accuracy': '0.7596', 'epoch': '0.07229'}
{'loss': '1.019', 'grad_norm': '0.6661', 'learning_rate': '5.077e-05', 'entropy': '0.9882', 'num_tokens': '4.405e+07', 'mean_token_accuracy': '0.7724', 'epoch': '0.07229'}
{'loss': '0.9787', 'grad_norm': '0.4302', 'learning_rate': '5.074e-05', 'entropy': '0.8428', 'num_tokens': '4.405e+07', 'mean_token_accuracy': '0.802', 'epoch': '0.0723'}
{'loss': '0.7371', 'grad_norm': '0.4017', 'learning_rate': '5.072e-05', 'entropy': '0.7231', 'num_tokens': '4.406e+07', 'mean_token_accuracy': '0.8094', 'epoch': '0.07231'}
{'loss': '0.6184', 'grad_norm': '0.3342', 'learning_rate': '5.069e-05', 'entropy': '0.6131', 'num_tokens': '4.406e+07', 'mean_token_accuracy': '0.8398', 'epoch': '0.07232'}
{'loss': '1.172', 'grad_norm': '0.5626', 'learning_rate': '5.067e-05', 'entropy': '1.145', 'num_tokens': '4.407e+07', 'mean_token_accuracy': '0.711', 'epoch': '0.07233'}
{'loss': '0.8195', 'grad_norm': '0.4628', 'learning_rate': '5.065e-05', 'entropy': '0.812', 'num_tokens': '4.407e+07', 'mean_token_accuracy': '0.7944', 'epoch': '0.07234'}
{'loss': '0.8386', 'grad_norm': '0.3951', 'learning_rate': '5.062e-05', 'entropy': '0.9325', 'num_tokens': '4.408e+07', 'mean_token_accuracy': '0.7679', 'epoch': '0.07235'}
{'loss': '0.838', 'grad_norm': '0.4477', 'learning_rate': '5.06e-05', 'entropy': '0.8901', 'num_tokens': '4.408e+07', 'mean_token_accuracy': '0.7801', 'epoch': '0.07236'}
{'loss': '0.6256', 'grad_norm': '0.4139', 'learning_rate': '5.057e-05', 'entropy': '0.659', 'num_tokens': '4.409e+07', 'mean_token_accuracy': '0.8217', 'epoch': '0.07237'}
{'loss': '0.6822', 'grad_norm': '0.3387', 'learning_rate': '5.055e-05', 'entropy': '0.6909', 'num_tokens': '4.409e+07', 'mean_token_accuracy': '0.8147', 'epoch': '0.07237'}
{'loss': '0.5706', 'grad_norm': '0.3226', 'learning_rate': '5.053e-05', 'entropy': '0.6429', 'num_tokens': '4.41e+07', 'mean_token_accuracy': '0.8368', 'epoch': '0.07238'}
{'loss': '0.991', 'grad_norm': '0.3745', 'learning_rate': '5.05e-05', 'entropy': '0.9517', 'num_tokens': '4.41e+07', 'mean_token_accuracy': '0.7648', 'epoch': '0.07239'}
{'loss': '0.677', 'grad_norm': '0.3179', 'learning_rate': '5.048e-05', 'entropy': '0.6994', 'num_tokens': '4.411e+07', 'mean_token_accuracy': '0.8087', 'epoch': '0.0724'}

 68%|██████▊   | 8182/12000 [4:50:13<1:57:42,  1.85s/it]
 68%|██████▊   | 8183/12000 [4:50:14<1:49:24,  1.72s/it]
                                                        

 68%|██████▊   | 8183/12000 [4:50:14<1:49:24,  1.72s/it]
 68%|██████▊   | 8184/12000 [4:50:16<1:43:49,  1.63s/it]
                                                        

 68%|██████▊   | 8184/12000 [4:50:16<1:43:49,  1.63s/it]
 68%|██████▊   | 8185/12000 [4:50:17<1:43:12,  1.62s/it]
                                                        

 68%|██████▊   | 8185/12000 [4:50:17<1:43:12,  1.62s/it]
 68%|██████▊   | 8186/12000 [4:50:20<2:00:12,  1.89s/it]
                                                        

 68%|██████▊   | 8186/12000 [4:50:20<2:00:12,  1.89s/it]
 68%|██████▊   | 8187/12000 [4:50:22<2:05:31,  1.98s/it]
                                                        

 68%|██████▊   | 8187/12000 [4:50:22<2:05:31,  1.98s/it]
 68%|██████▊   | 8188/12000 [4:50:24<1:59:34,  1.88s/it]
                                                        

 68%|██████▊   | 8188/12000 [4:50:24<1:59:34,  1.88s/it]
 68%|██████▊   | 8189/12000 [4:50:26<2:03:53,  1.95s/it]
                                                        

 68%|██████▊   | 8189/12000 [4:50:26<2:03:53,  1.95s/it]
 68%|██████▊   | 8190/12000 [4:50:28<2:08:52,  2.03s/it]
                                                        

 68%|██████▊   | 8190/12000 [4:50:28<2:08:52,  2.03s/it]
 68%|██████▊   | 8191/12000 [4:50:30<2:01:24,  1.91s/it]
                                                        

 68%|██████▊   | 8191/12000 [4:50:30<2:01:24,  1.91s/it]
 68%|██████▊   | 8192/12000 [4:50:31<1:52:25,  1.77s/it]
                                                        

 68%|██████▊   | 8192/12000 [4:50:31<1:52:25,  1.77s/it]
 68%|██████▊   | 8193/12000 [4:50:34<2:03:40,  1.95s/it]
                                                        

 68%|██████▊   | 8193/12000 [4:50:34<2:03:40,  1.95s/it]
 68%|██████▊   | 8194/12000 [4:50:35<2:01:03,  1.91s/it]
                                                        

 68%|██████▊   | 8194/12000 [4:50:35<2:01:03,  1.91s/it]
 68%|██████▊   | 8195/12000 [4:50:40<3:01:31,  2.86s/it]
                                                        

 68%|██████▊   | 8195/12000 [4:50:40<3:01:31,  2.86s/it]
 68%|██████▊   | 8196/12000 [4:50:42<2:40:37,  2.53s/it]
                                                        

 68%|██████▊   | 8196/12000 [4:50:42<2:40:37,  2.53s/it]
 68%|██████▊   | 8197/12000 [4:50:45<2:36:58,  2.48s/it]
                                                        

 68%|██████▊   | 8197/12000 [4:50:45<2:36:58,  2.48s/it]
 68%|██████▊   | 8198/12000 [4:50:46<2:19:49,  2.21s/it]
                                                        

 68%|██████▊   | 8198/12000 [4:50:46<2:19:49,  2.21s/it]
 68%|██████▊   | 8199/12000 [4:50:48<2:17:19,  2.17s/it]
                                                        

 68%|██████▊   | 8199/12000 [4:50:48<2:17:19,  2.17s/it]
 68%|██████▊   | 8200/12000 [4:50:52<2:43:18,  2.58s/it]
                                                        

 68%|██████▊   | 8200/12000 [4:50:52<2:43:18,  2.58s/it]
 68%|██████▊   | 8201/12000 [4:50:56<3:06:37,  2.95s/it]
                                                        

 68%|██████▊   | 8201/12000 [4:50:56<3:06:37,  2.95s/it]
 68%|██████▊   | 8202/12000 [4:50:57<2:40:25,  2.53s/it]
                                                        

 68%|██████▊   | 8202/12000 [4:50:57<2:40:25,  2.53s/it]
 68%|██████▊   | 8203/12000 [4:50:59<2:23:55,  2.27s/it]
                                                        

 68%|██████▊   | 8203/12000 [4:50:59<2:23:55,  2.27s/it]
 68%|██████▊   | 8204/12000 [4:51:02<2:40:14,  2.53s/it]
                                                        

 68%|██████▊   | 8204/12000 [4:51:02<2:40:14,  2.53s/it]
 68%|██████▊   | 8205/12000 [4:51:03<2:19:23,  2.20s/it]
                                                        

 68%|██████▊   | 8205/12000 [4:51:03<2:19:23,  2.20s/it]
 68%|██████▊   | 8206/12000 [4:51:05<2:07:40,  2.02s/it]
                                                        

 68%|██████▊   | 8206/12000 [4:51:05<2:07:40,  2.02s/it]
 68%|██████▊   | 8207/12000 [4:51:07<2:02:37,  1.94s/it]
                                                        

 68%|██████▊   | 8207/12000 [4:51:07<2:02:37,  1.94s/it]
 68%|██████▊   | 8208/12000 [4:51:08<1:57:51,  1.86s/it]
                                                        

 68%|██████▊   | 8208/12000 [4:51:08<1:57:51,  1.86s/it]
 68%|██████▊   | 8209/12000 [4:51:10<1:52:39,  1.78s/it]
                                                        

 68%|██████▊   | 8209/12000 [4:51:10<1:52:39,  1.78s/it]
 68%|██████▊   | 8210/12000 [4:51:11<1:46:44,  1.69s/it]
                                                        

 68%|██████▊   | 8210/12000 [4:51:11<1:46:44,  1.69s/it]
 68%|██████▊   | 8211/12000 [4:51:13<1:49:26,  1.73s/it]
                                                        

 68%|██████▊   | 8211/12000 [4:51:13<1:49:26,  1.73s/it]
 68%|██████▊   | 8212/12000 [4:51:16<2:00:08,  1.90s/it]
                                                        

 68%|██████▊   | 8212/12000 [4:51:16<2:00:08,  1.90s/it]
 68%|██████▊   | 8213/12000 [4:51:18<2:04:52,  1.98s/it]
                                                        

 68%|██████▊   | 8213/12000 [4:51:18<2:04:52,  1.98s/it]
 68%|██████▊   | 8214/12000 [4:51:19<1:54:56,  1.82s/it]
                                                        

 68%|██████▊   | 8214/12000 [4:51:19<1:54:56,  1.82s/it]
 68%|██████▊   | 8215/12000 [4:51:23<2:23:57,  2.28s/it]
                                                        

 68%|██████▊   | 8215/12000 [4:51:23<2:23:57,  2.28s/it]
 68%|██████▊   | 8216/12000 [4:51:24<2:09:50,  2.06s/it]
                                                        

 68%|██████▊   | 8216/12000 [4:51:24<2:09:50,  2.06s/it]
 68%|██████▊   | 8217/12000 [4:51:28<2:53:31,  2.75s/it]
                                                        

 68%|██████▊   | 8217/12000 [4:51:28<2:53:31,  2.75s/it]
 68%|██████▊   | 8218/12000 [4:51:30<2:29:27,  2.37s/it]
                                                        

 68%|██████▊   | 8218/12000 [4:51:30<2:29:27,  2.37s/it]
 68%|██████▊   | 8219/12000 [4:51:32<2:16:16,  2.16s/it]
                                                        

 68%|██████▊   | 8219/12000 [4:51:32<2:16:16,  2.16s/it]
 68%|██████▊   | 8220/12000 [4:51:33<2:04:53,  1.98s/it]
                                                        

 68%|██████▊   | 8220/12000 [4:51:33<2:04:53,  1.98s/it]
 69%|██████▊   | 8221/12000 [4:51:35<2:09:32,  2.06s/it]
                                                        

 69%|██████▊   | 8221/12000 [4:51:35<2:09:32,  2.06s/it]
 69%|██████▊   | 8222/12000 [4:51:38<2:12:59,  2.11s/it]
                                                        

 69%|██████▊   | 8222/12000 [4:51:38<2:12:59,  2.11s/it]
 69%|██████▊   | 8223/12000 [4:51:39<2:00:51,  1.92s/it]
                                                        

 69%|██████▊   | 8223/12000 [4:51:39<2:00:51,  1.92s/it]
 69%|██████▊   | 8224/12000 [4:51:42<2:17:47,  2.19s/it]
                                                        

 69%|██████▊   | 8224/12000 [4:51:42<2:17:47,  2.19s/it]
 69%|██████▊   | 8225/12000 [4:51:44<2:09:23,  2.06s/it]
                                                        

 69%|██████▊   | 8225/12000 [4:51:44<2:09:23,  2.06s/it]
 69%|██████▊   | 8226/12000 [4:51:45<2:03:24,  1.96s/it]
                                                        

 69%|██████▊   | 8226/12000 [4:51:45<2:03:24,  1.96s/it]
 69%|██████▊   | 8227/12000 [4:51:47<1:54:52,  1.83s/it]
                                                        

 69%|██████▊   | 8227/12000 [4:51:47<1:54:52,  1.83s/it]
 69%|██████▊   | 8228/12000 [4:51:49<1:53:57,  1.81s/it]
                                                        

 69%|██████▊   | 8228/12000 [4:51:49<1:53:57,  1.81s/it]
 69%|██████▊   | 8229/12000 [4:51:51<2:03:01,  1.96s/it]
                                                        
{'loss': '0.7563', 'grad_norm': '0.4686', 'learning_rate': '5.045e-05', 'entropy': '0.6953', 'num_tokens': '4.411e+07', 'mean_token_accuracy': '0.8114', 'epoch': '0.07241'}
{'loss': '0.5541', 'grad_norm': '0.3427', 'learning_rate': '5.043e-05', 'entropy': '0.5723', 'num_tokens': '4.412e+07', 'mean_token_accuracy': '0.8476', 'epoch': '0.07242'}
{'loss': '0.8215', 'grad_norm': '0.4086', 'learning_rate': '5.041e-05', 'entropy': '0.8281', 'num_tokens': '4.412e+07', 'mean_token_accuracy': '0.8037', 'epoch': '0.07243'}
{'loss': '0.7866', 'grad_norm': '0.3569', 'learning_rate': '5.038e-05', 'entropy': '0.807', 'num_tokens': '4.413e+07', 'mean_token_accuracy': '0.7815', 'epoch': '0.07244'}
{'loss': '0.914', 'grad_norm': '0.3929', 'learning_rate': '5.036e-05', 'entropy': '0.9478', 'num_tokens': '4.413e+07', 'mean_token_accuracy': '0.7713', 'epoch': '0.07244'}
{'loss': '0.604', 'grad_norm': '0.3797', 'learning_rate': '5.033e-05', 'entropy': '0.6245', 'num_tokens': '4.414e+07', 'mean_token_accuracy': '0.8474', 'epoch': '0.07245'}
{'loss': '0.9951', 'grad_norm': '0.4461', 'learning_rate': '5.031e-05', 'entropy': '0.9546', 'num_tokens': '4.414e+07', 'mean_token_accuracy': '0.7594', 'epoch': '0.07246'}
{'loss': '1.02', 'grad_norm': '0.4699', 'learning_rate': '5.029e-05', 'entropy': '0.8763', 'num_tokens': '4.415e+07', 'mean_token_accuracy': '0.7821', 'epoch': '0.07247'}
{'loss': '0.9024', 'grad_norm': '0.4043', 'learning_rate': '5.026e-05', 'entropy': '0.9475', 'num_tokens': '4.415e+07', 'mean_token_accuracy': '0.7665', 'epoch': '0.07248'}
{'loss': '0.7588', 'grad_norm': '0.3469', 'learning_rate': '5.024e-05', 'entropy': '0.792', 'num_tokens': '4.416e+07', 'mean_token_accuracy': '0.8025', 'epoch': '0.07249'}
{'loss': '0.6687', 'grad_norm': '0.3551', 'learning_rate': '5.021e-05', 'entropy': '0.6991', 'num_tokens': '4.416e+07', 'mean_token_accuracy': '0.8136', 'epoch': '0.0725'}
{'loss': '0.6558', 'grad_norm': '0.4081', 'learning_rate': '5.019e-05', 'entropy': '0.7092', 'num_tokens': '4.417e+07', 'mean_token_accuracy': '0.8248', 'epoch': '0.07251'}
{'loss': '0.7852', 'grad_norm': '0.4484', 'learning_rate': '5.017e-05', 'entropy': '0.7406', 'num_tokens': '4.417e+07', 'mean_token_accuracy': '0.8037', 'epoch': '0.07252'}
{'loss': '1.096', 'grad_norm': '0.586', 'learning_rate': '5.014e-05', 'entropy': '1.072', 'num_tokens': '4.418e+07', 'mean_token_accuracy': '0.7268', 'epoch': '0.07252'}
{'loss': '1.172', 'grad_norm': '0.4023', 'learning_rate': '5.012e-05', 'entropy': '1.143', 'num_tokens': '4.418e+07', 'mean_token_accuracy': '0.7185', 'epoch': '0.07253'}
{'loss': '0.7816', 'grad_norm': '0.319', 'learning_rate': '5.01e-05', 'entropy': '0.8262', 'num_tokens': '4.419e+07', 'mean_token_accuracy': '0.7887', 'epoch': '0.07254'}
{'loss': '0.7377', 'grad_norm': '0.4574', 'learning_rate': '5.007e-05', 'entropy': '0.7572', 'num_tokens': '4.419e+07', 'mean_token_accuracy': '0.8105', 'epoch': '0.07255'}
{'loss': '0.9077', 'grad_norm': '0.4269', 'learning_rate': '5.005e-05', 'entropy': '0.8559', 'num_tokens': '4.42e+07', 'mean_token_accuracy': '0.7966', 'epoch': '0.07256'}
{'loss': '1.052', 'grad_norm': '0.405', 'learning_rate': '5.002e-05', 'entropy': '1.077', 'num_tokens': '4.421e+07', 'mean_token_accuracy': '0.7472', 'epoch': '0.07257'}
{'loss': '1.344', 'grad_norm': '0.4003', 'learning_rate': '5e-05', 'entropy': '1.204', 'num_tokens': '4.421e+07', 'mean_token_accuracy': '0.7106', 'epoch': '0.07258'}
{'loss': '0.9257', 'grad_norm': '0.4011', 'learning_rate': '4.998e-05', 'entropy': '0.8653', 'num_tokens': '4.422e+07', 'mean_token_accuracy': '0.7823', 'epoch': '0.07259'}
{'loss': '0.5543', 'grad_norm': '0.3462', 'learning_rate': '4.995e-05', 'entropy': '0.5708', 'num_tokens': '4.422e+07', 'mean_token_accuracy': '0.8412', 'epoch': '0.0726'}
{'loss': '0.6489', 'grad_norm': '0.382', 'learning_rate': '4.993e-05', 'entropy': '0.5949', 'num_tokens': '4.423e+07', 'mean_token_accuracy': '0.8439', 'epoch': '0.0726'}
{'loss': '1.106', 'grad_norm': '0.5397', 'learning_rate': '4.99e-05', 'entropy': '1.135', 'num_tokens': '4.424e+07', 'mean_token_accuracy': '0.7296', 'epoch': '0.07261'}
{'loss': '0.9447', 'grad_norm': '0.52', 'learning_rate': '4.988e-05', 'entropy': '0.9179', 'num_tokens': '4.424e+07', 'mean_token_accuracy': '0.7775', 'epoch': '0.07262'}
{'loss': '0.6387', 'grad_norm': '0.3976', 'learning_rate': '4.986e-05', 'entropy': '0.6265', 'num_tokens': '4.424e+07', 'mean_token_accuracy': '0.8391', 'epoch': '0.07263'}
{'loss': '0.8538', 'grad_norm': '0.4793', 'learning_rate': '4.983e-05', 'entropy': '0.832', 'num_tokens': '4.425e+07', 'mean_token_accuracy': '0.794', 'epoch': '0.07264'}
{'loss': '0.7034', 'grad_norm': '0.3619', 'learning_rate': '4.981e-05', 'entropy': '0.7248', 'num_tokens': '4.425e+07', 'mean_token_accuracy': '0.8047', 'epoch': '0.07265'}
{'loss': '0.7106', 'grad_norm': '0.37', 'learning_rate': '4.979e-05', 'entropy': '0.8414', 'num_tokens': '4.426e+07', 'mean_token_accuracy': '0.7841', 'epoch': '0.07266'}
{'loss': '1.024', 'grad_norm': '0.3953', 'learning_rate': '4.976e-05', 'entropy': '1.077', 'num_tokens': '4.426e+07', 'mean_token_accuracy': '0.7346', 'epoch': '0.07267'}
{'loss': '0.8089', 'grad_norm': '0.4514', 'learning_rate': '4.974e-05', 'entropy': '0.8455', 'num_tokens': '4.427e+07', 'mean_token_accuracy': '0.7863', 'epoch': '0.07267'}
{'loss': '0.6605', 'grad_norm': '0.3524', 'learning_rate': '4.971e-05', 'entropy': '0.6458', 'num_tokens': '4.427e+07', 'mean_token_accuracy': '0.8251', 'epoch': '0.07268'}
{'loss': '0.5174', 'grad_norm': '0.3716', 'learning_rate': '4.969e-05', 'entropy': '0.5925', 'num_tokens': '4.428e+07', 'mean_token_accuracy': '0.8467', 'epoch': '0.07269'}
{'loss': '1.193', 'grad_norm': '0.3876', 'learning_rate': '4.967e-05', 'entropy': '0.9748', 'num_tokens': '4.429e+07', 'mean_token_accuracy': '0.7634', 'epoch': '0.0727'}
{'loss': '0.8217', 'grad_norm': '0.4933', 'learning_rate': '4.964e-05', 'entropy': '0.9355', 'num_tokens': '4.429e+07', 'mean_token_accuracy': '0.7708', 'epoch': '0.07271'}
{'loss': '0.5835', 'grad_norm': '0.4035', 'learning_rate': '4.962e-05', 'entropy': '0.5934', 'num_tokens': '4.43e+07', 'mean_token_accuracy': '0.8515', 'epoch': '0.07272'}
{'loss': '0.829', 'grad_norm': '0.4123', 'learning_rate': '4.959e-05', 'entropy': '0.8878', 'num_tokens': '4.43e+07', 'mean_token_accuracy': '0.7854', 'epoch': '0.07273'}
{'loss': '0.8213', 'grad_norm': '0.3737', 'learning_rate': '4.957e-05', 'entropy': '0.8757', 'num_tokens': '4.431e+07', 'mean_token_accuracy': '0.7881', 'epoch': '0.07274'}
{'loss': '0.7362', 'grad_norm': '0.3441', 'learning_rate': '4.955e-05', 'entropy': '0.7383', 'num_tokens': '4.431e+07', 'mean_token_accuracy': '0.8041', 'epoch': '0.07275'}
{'loss': '0.8318', 'grad_norm': '0.3355', 'learning_rate': '4.952e-05', 'entropy': '0.7372', 'num_tokens': '4.432e+07', 'mean_token_accuracy': '0.8302', 'epoch': '0.07275'}
{'loss': '0.8866', 'grad_norm': '0.3888', 'learning_rate': '4.95e-05', 'entropy': '0.7726', 'num_tokens': '4.432e+07', 'mean_token_accuracy': '0.8079', 'epoch': '0.07276'}
{'loss': '0.9367', 'grad_norm': '0.3955', 'learning_rate': '4.948e-05', 'entropy': '0.9682', 'num_tokens': '4.433e+07', 'mean_token_accuracy': '0.7584', 'epoch': '0.07277'}
{'loss': '1.031', 'grad_norm': '0.4355', 'learning_rate': '4.945e-05', 'entropy': '0.862', 'num_tokens': '4.433e+07', 'mean_token_accuracy': '0.7687', 'epoch': '0.07278'}
{'loss': '0.825', 'grad_norm': '0.4267', 'learning_rate': '4.943e-05', 'entropy': '0.815', 'num_tokens': '4.434e+07', 'mean_token_accuracy': '0.7897', 'epoch': '0.07279'}
{'loss': '0.8042', 'grad_norm': '0.4685', 'learning_rate': '4.94e-05', 'entropy': '0.7796', 'num_tokens': '4.434e+07', 'mean_token_accuracy': '0.8003', 'epoch': '0.0728'}
{'loss': '0.8731', 'grad_norm': '0.3914', 'learning_rate': '4.938e-05', 'entropy': '0.9189', 'num_tokens': '4.435e+07', 'mean_token_accuracy': '0.7765', 'epoch': '0.07281'}
{'loss': '0.8054', 'grad_norm': '0.8548', 'learning_rate': '4.936e-05', 'entropy': '0.7189', 'num_tokens': '4.435e+07', 'mean_token_accuracy': '0.8094', 'epoch': '0.07282'}

 69%|██████▊   | 8229/12000 [4:51:51<2:03:01,  1.96s/it]
 69%|██████▊   | 8230/12000 [4:51:53<1:58:11,  1.88s/it]
                                                        

 69%|██████▊   | 8230/12000 [4:51:53<1:58:11,  1.88s/it]
 69%|██████▊   | 8231/12000 [4:51:55<2:01:32,  1.93s/it]
                                                        

 69%|██████▊   | 8231/12000 [4:51:55<2:01:32,  1.93s/it]
 69%|██████▊   | 8232/12000 [4:51:57<2:01:07,  1.93s/it]
                                                        

 69%|██████▊   | 8232/12000 [4:51:57<2:01:07,  1.93s/it]
 69%|██████▊   | 8233/12000 [4:51:58<1:56:44,  1.86s/it]
                                                        

 69%|██████▊   | 8233/12000 [4:51:58<1:56:44,  1.86s/it]
 69%|██████▊   | 8234/12000 [4:52:01<2:08:32,  2.05s/it]
                                                        

 69%|██████▊   | 8234/12000 [4:52:01<2:08:32,  2.05s/it]
 69%|██████▊   | 8235/12000 [4:52:05<2:51:41,  2.74s/it]
                                                        

 69%|██████▊   | 8235/12000 [4:52:05<2:51:41,  2.74s/it]
 69%|██████▊   | 8236/12000 [4:52:07<2:27:24,  2.35s/it]
                                                        

 69%|██████▊   | 8236/12000 [4:52:07<2:27:24,  2.35s/it]
 69%|██████▊   | 8237/12000 [4:52:10<2:41:42,  2.58s/it]
                                                        

 69%|██████▊   | 8237/12000 [4:52:10<2:41:42,  2.58s/it]
 69%|██████▊   | 8238/12000 [4:52:11<2:23:44,  2.29s/it]
                                                        

 69%|██████▊   | 8238/12000 [4:52:11<2:23:44,  2.29s/it]
 69%|██████▊   | 8239/12000 [4:52:15<2:43:47,  2.61s/it]
                                                        

 69%|██████▊   | 8239/12000 [4:52:15<2:43:47,  2.61s/it]
 69%|██████▊   | 8240/12000 [4:52:16<2:24:57,  2.31s/it]
                                                        

 69%|██████▊   | 8240/12000 [4:52:16<2:24:57,  2.31s/it]
 69%|██████▊   | 8241/12000 [4:52:19<2:38:32,  2.53s/it]
                                                        

 69%|██████▊   | 8241/12000 [4:52:19<2:38:32,  2.53s/it]
 69%|██████▊   | 8242/12000 [4:52:21<2:22:07,  2.27s/it]
                                                        

 69%|██████▊   | 8242/12000 [4:52:21<2:22:07,  2.27s/it]
 69%|██████▊   | 8243/12000 [4:52:23<2:08:04,  2.05s/it]
                                                        

 69%|██████▊   | 8243/12000 [4:52:23<2:08:04,  2.05s/it]
 69%|██████▊   | 8244/12000 [4:52:24<1:56:34,  1.86s/it]
                                                        

 69%|██████▊   | 8244/12000 [4:52:24<1:56:34,  1.86s/it]
 69%|██████▊   | 8245/12000 [4:52:26<2:04:55,  2.00s/it]
                                                        

 69%|██████▊   | 8245/12000 [4:52:26<2:04:55,  2.00s/it]
 69%|██████▊   | 8246/12000 [4:52:29<2:18:10,  2.21s/it]
                                                        

 69%|██████▊   | 8246/12000 [4:52:29<2:18:10,  2.21s/it]
 69%|██████▊   | 8247/12000 [4:52:31<2:07:14,  2.03s/it]
                                                        

 69%|██████▊   | 8247/12000 [4:52:31<2:07:14,  2.03s/it]
 69%|██████▊   | 8248/12000 [4:52:33<2:03:02,  1.97s/it]
                                                        

 69%|██████▊   | 8248/12000 [4:52:33<2:03:02,  1.97s/it]
 69%|██████▊   | 8249/12000 [4:52:34<1:53:59,  1.82s/it]
                                                        

 69%|██████▊   | 8249/12000 [4:52:34<1:53:59,  1.82s/it]
 69%|██████▉   | 8250/12000 [4:52:36<1:52:52,  1.81s/it]
                                                        

 69%|██████▉   | 8250/12000 [4:52:36<1:52:52,  1.81s/it]
 69%|██████▉   | 8251/12000 [4:52:40<2:40:03,  2.56s/it]
                                                        

 69%|██████▉   | 8251/12000 [4:52:40<2:40:03,  2.56s/it]
 69%|██████▉   | 8252/12000 [4:52:42<2:24:06,  2.31s/it]
                                                        

 69%|██████▉   | 8252/12000 [4:52:42<2:24:06,  2.31s/it]
 69%|██████▉   | 8253/12000 [4:52:43<2:12:13,  2.12s/it]
                                                        

 69%|██████▉   | 8253/12000 [4:52:43<2:12:13,  2.12s/it]
 69%|██████▉   | 8254/12000 [4:52:45<2:05:14,  2.01s/it]
                                                        

 69%|██████▉   | 8254/12000 [4:52:45<2:05:14,  2.01s/it]
 69%|██████▉   | 8255/12000 [4:52:47<2:05:30,  2.01s/it]
                                                        

 69%|██████▉   | 8255/12000 [4:52:47<2:05:30,  2.01s/it]
 69%|██████▉   | 8256/12000 [4:52:50<2:26:01,  2.34s/it]
                                                        

 69%|██████▉   | 8256/12000 [4:52:50<2:26:01,  2.34s/it]
 69%|██████▉   | 8257/12000 [4:52:53<2:24:57,  2.32s/it]
                                                        

 69%|██████▉   | 8257/12000 [4:52:53<2:24:57,  2.32s/it]
 69%|██████▉   | 8258/12000 [4:52:54<2:11:29,  2.11s/it]
                                                        

 69%|██████▉   | 8258/12000 [4:52:54<2:11:29,  2.11s/it]
 69%|██████▉   | 8259/12000 [4:52:56<2:02:40,  1.97s/it]
                                                        

 69%|██████▉   | 8259/12000 [4:52:56<2:02:40,  1.97s/it]
 69%|██████▉   | 8260/12000 [4:52:58<1:57:30,  1.89s/it]
                                                        

 69%|██████▉   | 8260/12000 [4:52:58<1:57:30,  1.89s/it]
 69%|██████▉   | 8261/12000 [4:53:01<2:30:59,  2.42s/it]
                                                        

 69%|██████▉   | 8261/12000 [4:53:01<2:30:59,  2.42s/it]
 69%|██████▉   | 8262/12000 [4:53:03<2:19:03,  2.23s/it]
                                                        

 69%|██████▉   | 8262/12000 [4:53:03<2:19:03,  2.23s/it]
 69%|██████▉   | 8263/12000 [4:53:04<2:01:26,  1.95s/it]
                                                        

 69%|██████▉   | 8263/12000 [4:53:04<2:01:26,  1.95s/it]
 69%|██████▉   | 8264/12000 [4:53:07<2:11:54,  2.12s/it]
                                                        

 69%|██████▉   | 8264/12000 [4:53:07<2:11:54,  2.12s/it]
 69%|██████▉   | 8265/12000 [4:53:08<2:03:04,  1.98s/it]
                                                        

 69%|██████▉   | 8265/12000 [4:53:08<2:03:04,  1.98s/it]
 69%|██████▉   | 8266/12000 [4:53:11<2:20:55,  2.26s/it]
                                                        

 69%|██████▉   | 8266/12000 [4:53:11<2:20:55,  2.26s/it]
 69%|██████▉   | 8267/12000 [4:53:13<2:12:10,  2.12s/it]
                                                        

 69%|██████▉   | 8267/12000 [4:53:13<2:12:10,  2.12s/it]
 69%|██████▉   | 8268/12000 [4:53:15<2:04:38,  2.00s/it]
                                                        

 69%|██████▉   | 8268/12000 [4:53:15<2:04:38,  2.00s/it]
 69%|██████▉   | 8269/12000 [4:53:17<2:06:35,  2.04s/it]
                                                        

 69%|██████▉   | 8269/12000 [4:53:17<2:06:35,  2.04s/it]
 69%|██████▉   | 8270/12000 [4:53:21<2:33:23,  2.47s/it]
                                                        

 69%|██████▉   | 8270/12000 [4:53:21<2:33:23,  2.47s/it]
 69%|██████▉   | 8271/12000 [4:53:22<2:18:14,  2.22s/it]
                                                        

 69%|██████▉   | 8271/12000 [4:53:22<2:18:14,  2.22s/it]
 69%|██████▉   | 8272/12000 [4:53:24<2:08:01,  2.06s/it]
                                                        

 69%|██████▉   | 8272/12000 [4:53:24<2:08:01,  2.06s/it]
 69%|██████▉   | 8273/12000 [4:53:25<1:58:12,  1.90s/it]
                                                        

 69%|██████▉   | 8273/12000 [4:53:25<1:58:12,  1.90s/it]
 69%|██████▉   | 8274/12000 [4:53:27<1:49:30,  1.76s/it]
                                                        

 69%|██████▉   | 8274/12000 [4:53:27<1:49:30,  1.76s/it]
 69%|██████▉   | 8275/12000 [4:53:28<1:41:47,  1.64s/it]
                                                        

 69%|██████▉   | 8275/12000 [4:53:28<1:41:47,  1.64s/it]
 69%|██████▉   | 8276/12000 [4:53:30<1:40:22,  1.62s/it]
                                                        
{'loss': '0.7289', 'grad_norm': '0.4015', 'learning_rate': '4.933e-05', 'entropy': '0.76', 'num_tokens': '4.436e+07', 'mean_token_accuracy': '0.8137', 'epoch': '0.07283'}
{'loss': '0.9901', 'grad_norm': '0.3969', 'learning_rate': '4.931e-05', 'entropy': '1.039', 'num_tokens': '4.437e+07', 'mean_token_accuracy': '0.7414', 'epoch': '0.07283'}
{'loss': '0.6984', 'grad_norm': '0.2846', 'learning_rate': '4.929e-05', 'entropy': '0.6593', 'num_tokens': '4.437e+07', 'mean_token_accuracy': '0.8239', 'epoch': '0.07284'}
{'loss': '0.7557', 'grad_norm': '0.4088', 'learning_rate': '4.926e-05', 'entropy': '0.7442', 'num_tokens': '4.438e+07', 'mean_token_accuracy': '0.8232', 'epoch': '0.07285'}
{'loss': '0.6381', 'grad_norm': '0.3527', 'learning_rate': '4.924e-05', 'entropy': '0.611', 'num_tokens': '4.438e+07', 'mean_token_accuracy': '0.8309', 'epoch': '0.07286'}
{'loss': '1.443', 'grad_norm': '0.4865', 'learning_rate': '4.921e-05', 'entropy': '1.185', 'num_tokens': '4.439e+07', 'mean_token_accuracy': '0.7199', 'epoch': '0.07287'}
{'loss': '0.8143', 'grad_norm': '0.3037', 'learning_rate': '4.919e-05', 'entropy': '0.7972', 'num_tokens': '4.44e+07', 'mean_token_accuracy': '0.8041', 'epoch': '0.07288'}
{'loss': '0.7145', 'grad_norm': '0.3853', 'learning_rate': '4.917e-05', 'entropy': '0.6994', 'num_tokens': '4.44e+07', 'mean_token_accuracy': '0.8285', 'epoch': '0.07289'}
{'loss': '0.7449', 'grad_norm': '0.3498', 'learning_rate': '4.914e-05', 'entropy': '0.7474', 'num_tokens': '4.441e+07', 'mean_token_accuracy': '0.8243', 'epoch': '0.0729'}
{'loss': '0.7096', 'grad_norm': '0.3894', 'learning_rate': '4.912e-05', 'entropy': '0.7292', 'num_tokens': '4.442e+07', 'mean_token_accuracy': '0.8173', 'epoch': '0.0729'}
{'loss': '0.778', 'grad_norm': '0.3784', 'learning_rate': '4.91e-05', 'entropy': '0.7657', 'num_tokens': '4.442e+07', 'mean_token_accuracy': '0.8073', 'epoch': '0.07291'}
{'loss': '0.6119', 'grad_norm': '0.3061', 'learning_rate': '4.907e-05', 'entropy': '0.7004', 'num_tokens': '4.442e+07', 'mean_token_accuracy': '0.8212', 'epoch': '0.07292'}
{'loss': '0.6716', 'grad_norm': '0.3406', 'learning_rate': '4.905e-05', 'entropy': '0.6702', 'num_tokens': '4.443e+07', 'mean_token_accuracy': '0.8325', 'epoch': '0.07293'}
{'loss': '0.6735', 'grad_norm': '0.3044', 'learning_rate': '4.902e-05', 'entropy': '0.7164', 'num_tokens': '4.444e+07', 'mean_token_accuracy': '0.8279', 'epoch': '0.07294'}
{'loss': '0.8236', 'grad_norm': '0.3331', 'learning_rate': '4.9e-05', 'entropy': '0.9499', 'num_tokens': '4.444e+07', 'mean_token_accuracy': '0.7761', 'epoch': '0.07295'}
{'loss': '0.7959', 'grad_norm': '0.4273', 'learning_rate': '4.898e-05', 'entropy': '0.8117', 'num_tokens': '4.445e+07', 'mean_token_accuracy': '0.7968', 'epoch': '0.07296'}
{'loss': '0.5828', 'grad_norm': '0.2645', 'learning_rate': '4.895e-05', 'entropy': '0.6529', 'num_tokens': '4.445e+07', 'mean_token_accuracy': '0.8372', 'epoch': '0.07297'}
{'loss': '0.6051', 'grad_norm': '0.3455', 'learning_rate': '4.893e-05', 'entropy': '0.6315', 'num_tokens': '4.446e+07', 'mean_token_accuracy': '0.8327', 'epoch': '0.07298'}
{'loss': '1.137', 'grad_norm': '0.464', 'learning_rate': '4.891e-05', 'entropy': '1.194', 'num_tokens': '4.446e+07', 'mean_token_accuracy': '0.7086', 'epoch': '0.07298'}
{'loss': '0.7071', 'grad_norm': '0.407', 'learning_rate': '4.888e-05', 'entropy': '0.7399', 'num_tokens': '4.447e+07', 'mean_token_accuracy': '0.8082', 'epoch': '0.07299'}
{'loss': '0.8085', 'grad_norm': '0.4619', 'learning_rate': '4.886e-05', 'entropy': '0.8527', 'num_tokens': '4.447e+07', 'mean_token_accuracy': '0.7797', 'epoch': '0.073'}
{'loss': '1.13', 'grad_norm': '0.4386', 'learning_rate': '4.884e-05', 'entropy': '1.172', 'num_tokens': '4.448e+07', 'mean_token_accuracy': '0.7096', 'epoch': '0.07301'}
{'loss': '1.063', 'grad_norm': '0.4319', 'learning_rate': '4.881e-05', 'entropy': '0.8178', 'num_tokens': '4.449e+07', 'mean_token_accuracy': '0.7992', 'epoch': '0.07302'}
{'loss': '0.8395', 'grad_norm': '0.3318', 'learning_rate': '4.879e-05', 'entropy': '0.8032', 'num_tokens': '4.449e+07', 'mean_token_accuracy': '0.802', 'epoch': '0.07303'}
{'loss': '1.137', 'grad_norm': '0.4303', 'learning_rate': '4.876e-05', 'entropy': '1.156', 'num_tokens': '4.45e+07', 'mean_token_accuracy': '0.7219', 'epoch': '0.07304'}
{'loss': '0.7424', 'grad_norm': '0.4296', 'learning_rate': '4.874e-05', 'entropy': '0.7412', 'num_tokens': '4.45e+07', 'mean_token_accuracy': '0.8226', 'epoch': '0.07305'}
{'loss': '0.6271', 'grad_norm': '0.3059', 'learning_rate': '4.872e-05', 'entropy': '0.7045', 'num_tokens': '4.451e+07', 'mean_token_accuracy': '0.8306', 'epoch': '0.07306'}
{'loss': '0.9735', 'grad_norm': '0.3984', 'learning_rate': '4.869e-05', 'entropy': '1.032', 'num_tokens': '4.451e+07', 'mean_token_accuracy': '0.7539', 'epoch': '0.07306'}
{'loss': '0.8689', 'grad_norm': '0.4375', 'learning_rate': '4.867e-05', 'entropy': '0.8131', 'num_tokens': '4.452e+07', 'mean_token_accuracy': '0.7984', 'epoch': '0.07307'}
{'loss': '1.034', 'grad_norm': '0.423', 'learning_rate': '4.865e-05', 'entropy': '0.9691', 'num_tokens': '4.453e+07', 'mean_token_accuracy': '0.7564', 'epoch': '0.07308'}
{'loss': '0.7368', 'grad_norm': '0.4669', 'learning_rate': '4.862e-05', 'entropy': '0.762', 'num_tokens': '4.453e+07', 'mean_token_accuracy': '0.7937', 'epoch': '0.07309'}
{'loss': '0.7308', 'grad_norm': '0.4521', 'learning_rate': '4.86e-05', 'entropy': '0.7312', 'num_tokens': '4.454e+07', 'mean_token_accuracy': '0.811', 'epoch': '0.0731'}
{'loss': '0.8595', 'grad_norm': '0.3717', 'learning_rate': '4.857e-05', 'entropy': '0.9358', 'num_tokens': '4.454e+07', 'mean_token_accuracy': '0.7718', 'epoch': '0.07311'}
{'loss': '0.7153', 'grad_norm': '0.3849', 'learning_rate': '4.855e-05', 'entropy': '0.6249', 'num_tokens': '4.455e+07', 'mean_token_accuracy': '0.8372', 'epoch': '0.07312'}
{'loss': '0.6658', 'grad_norm': '0.3825', 'learning_rate': '4.853e-05', 'entropy': '0.6688', 'num_tokens': '4.455e+07', 'mean_token_accuracy': '0.833', 'epoch': '0.07313'}
{'loss': '0.8014', 'grad_norm': '0.3859', 'learning_rate': '4.85e-05', 'entropy': '0.9069', 'num_tokens': '4.456e+07', 'mean_token_accuracy': '0.7882', 'epoch': '0.07314'}
{'loss': '1.009', 'grad_norm': '0.4511', 'learning_rate': '4.848e-05', 'entropy': '0.9945', 'num_tokens': '4.456e+07', 'mean_token_accuracy': '0.7597', 'epoch': '0.07314'}
{'loss': '1.156', 'grad_norm': '0.4803', 'learning_rate': '4.846e-05', 'entropy': '1.111', 'num_tokens': '4.457e+07', 'mean_token_accuracy': '0.7356', 'epoch': '0.07315'}
{'loss': '0.8258', 'grad_norm': '0.3989', 'learning_rate': '4.843e-05', 'entropy': '0.8519', 'num_tokens': '4.457e+07', 'mean_token_accuracy': '0.7923', 'epoch': '0.07316'}
{'loss': '1.047', 'grad_norm': '0.374', 'learning_rate': '4.841e-05', 'entropy': '0.9297', 'num_tokens': '4.458e+07', 'mean_token_accuracy': '0.7888', 'epoch': '0.07317'}
{'loss': '0.9531', 'grad_norm': '0.3742', 'learning_rate': '4.839e-05', 'entropy': '0.8664', 'num_tokens': '4.459e+07', 'mean_token_accuracy': '0.7697', 'epoch': '0.07318'}
{'loss': '1.011', 'grad_norm': '0.3299', 'learning_rate': '4.836e-05', 'entropy': '0.9158', 'num_tokens': '4.459e+07', 'mean_token_accuracy': '0.7946', 'epoch': '0.07319'}
{'loss': '0.9011', 'grad_norm': '0.4072', 'learning_rate': '4.834e-05', 'entropy': '0.8923', 'num_tokens': '4.46e+07', 'mean_token_accuracy': '0.7582', 'epoch': '0.0732'}
{'loss': '0.762', 'grad_norm': '0.437', 'learning_rate': '4.832e-05', 'entropy': '0.8255', 'num_tokens': '4.46e+07', 'mean_token_accuracy': '0.778', 'epoch': '0.07321'}
{'loss': '0.6684', 'grad_norm': '0.4125', 'learning_rate': '4.829e-05', 'entropy': '0.7206', 'num_tokens': '4.461e+07', 'mean_token_accuracy': '0.824', 'epoch': '0.07321'}
{'loss': '0.7391', 'grad_norm': '0.452', 'learning_rate': '4.827e-05', 'entropy': '0.7497', 'num_tokens': '4.461e+07', 'mean_token_accuracy': '0.7957', 'epoch': '0.07322'}
{'loss': '0.7871', 'grad_norm': '0.4348', 'learning_rate': '4.824e-05', 'entropy': '0.8911', 'num_tokens': '4.462e+07', 'mean_token_accuracy': '0.7763', 'epoch': '0.07323'}

 69%|██████▉   | 8276/12000 [4:53:30<1:40:22,  1.62s/it]
 69%|██████▉   | 8277/12000 [4:53:31<1:37:27,  1.57s/it]
                                                        

 69%|██████▉   | 8277/12000 [4:53:31<1:37:27,  1.57s/it]
 69%|██████▉   | 8278/12000 [4:53:33<1:35:10,  1.53s/it]
                                                        

 69%|██████▉   | 8278/12000 [4:53:33<1:35:10,  1.53s/it]
 69%|██████▉   | 8279/12000 [4:53:34<1:40:28,  1.62s/it]
                                                        

 69%|██████▉   | 8279/12000 [4:53:34<1:40:28,  1.62s/it]
 69%|██████▉   | 8280/12000 [4:53:37<1:50:25,  1.78s/it]
                                                        

 69%|██████▉   | 8280/12000 [4:53:37<1:50:25,  1.78s/it]
 69%|██████▉   | 8281/12000 [4:53:38<1:45:27,  1.70s/it]
                                                        

 69%|██████▉   | 8281/12000 [4:53:38<1:45:27,  1.70s/it]
 69%|██████▉   | 8282/12000 [4:53:40<1:47:48,  1.74s/it]
                                                        

 69%|██████▉   | 8282/12000 [4:53:40<1:47:48,  1.74s/it]
 69%|██████▉   | 8283/12000 [4:53:44<2:21:18,  2.28s/it]
                                                        

 69%|██████▉   | 8283/12000 [4:53:44<2:21:18,  2.28s/it]
 69%|██████▉   | 8284/12000 [4:53:45<2:10:11,  2.10s/it]
                                                        

 69%|██████▉   | 8284/12000 [4:53:45<2:10:11,  2.10s/it]
 69%|██████▉   | 8285/12000 [4:53:47<2:02:26,  1.98s/it]
                                                        

 69%|██████▉   | 8285/12000 [4:53:47<2:02:26,  1.98s/it]
 69%|██████▉   | 8286/12000 [4:53:51<2:41:27,  2.61s/it]
                                                        

 69%|██████▉   | 8286/12000 [4:53:51<2:41:27,  2.61s/it]
 69%|██████▉   | 8287/12000 [4:53:53<2:22:22,  2.30s/it]
                                                        

 69%|██████▉   | 8287/12000 [4:53:53<2:22:22,  2.30s/it]
 69%|██████▉   | 8288/12000 [4:53:54<2:06:45,  2.05s/it]
                                                        

 69%|██████▉   | 8288/12000 [4:53:54<2:06:45,  2.05s/it]
 69%|██████▉   | 8289/12000 [4:53:57<2:32:31,  2.47s/it]
                                                        

 69%|██████▉   | 8289/12000 [4:53:57<2:32:31,  2.47s/it]
 69%|██████▉   | 8290/12000 [4:53:59<2:10:13,  2.11s/it]
                                                        

 69%|██████▉   | 8290/12000 [4:53:59<2:10:13,  2.11s/it]
 69%|██████▉   | 8291/12000 [4:54:00<2:01:21,  1.96s/it]
                                                        

 69%|██████▉   | 8291/12000 [4:54:00<2:01:21,  1.96s/it]
 69%|██████▉   | 8292/12000 [4:54:02<1:51:44,  1.81s/it]
                                                        

 69%|██████▉   | 8292/12000 [4:54:02<1:51:44,  1.81s/it]
 69%|██████▉   | 8293/12000 [4:54:03<1:47:21,  1.74s/it]
                                                        

 69%|██████▉   | 8293/12000 [4:54:03<1:47:21,  1.74s/it]
 69%|██████▉   | 8294/12000 [4:54:06<1:54:25,  1.85s/it]
                                                        

 69%|██████▉   | 8294/12000 [4:54:06<1:54:25,  1.85s/it]
 69%|██████▉   | 8295/12000 [4:54:08<2:05:26,  2.03s/it]
                                                        

 69%|██████▉   | 8295/12000 [4:54:08<2:05:26,  2.03s/it]
 69%|██████▉   | 8296/12000 [4:54:09<1:54:27,  1.85s/it]
                                                        

 69%|██████▉   | 8296/12000 [4:54:09<1:54:27,  1.85s/it]
 69%|██████▉   | 8297/12000 [4:54:11<1:45:20,  1.71s/it]
                                                        

 69%|██████▉   | 8297/12000 [4:54:11<1:45:20,  1.71s/it]
 69%|██████▉   | 8298/12000 [4:54:13<1:52:56,  1.83s/it]
                                                        

 69%|██████▉   | 8298/12000 [4:54:13<1:52:56,  1.83s/it]
 69%|██████▉   | 8299/12000 [4:54:14<1:44:59,  1.70s/it]
                                                        

 69%|██████▉   | 8299/12000 [4:54:14<1:44:59,  1.70s/it]
 69%|██████▉   | 8300/12000 [4:54:17<2:00:15,  1.95s/it]
                                                        

 69%|██████▉   | 8300/12000 [4:54:17<2:00:15,  1.95s/it]
 69%|██████▉   | 8301/12000 [4:54:20<2:23:47,  2.33s/it]
                                                        

 69%|██████▉   | 8301/12000 [4:54:20<2:23:47,  2.33s/it]
 69%|██████▉   | 8302/12000 [4:54:22<2:08:04,  2.08s/it]
                                                        

 69%|██████▉   | 8302/12000 [4:54:22<2:08:04,  2.08s/it]
 69%|██████▉   | 8303/12000 [4:54:24<2:18:37,  2.25s/it]
                                                        

 69%|██████▉   | 8303/12000 [4:54:24<2:18:37,  2.25s/it]
 69%|██████▉   | 8304/12000 [4:54:26<2:08:37,  2.09s/it]
                                                        

 69%|██████▉   | 8304/12000 [4:54:26<2:08:37,  2.09s/it]
 69%|██████▉   | 8305/12000 [4:54:29<2:36:28,  2.54s/it]
                                                        

 69%|██████▉   | 8305/12000 [4:54:29<2:36:28,  2.54s/it]
 69%|██████▉   | 8306/12000 [4:54:31<2:18:15,  2.25s/it]
                                                        

 69%|██████▉   | 8306/12000 [4:54:31<2:18:15,  2.25s/it]
 69%|██████▉   | 8307/12000 [4:54:36<3:03:13,  2.98s/it]
                                                        

 69%|██████▉   | 8307/12000 [4:54:36<3:03:13,  2.98s/it]
 69%|██████▉   | 8308/12000 [4:54:37<2:37:32,  2.56s/it]
                                                        

 69%|██████▉   | 8308/12000 [4:54:37<2:37:32,  2.56s/it]
 69%|██████▉   | 8309/12000 [4:54:39<2:17:39,  2.24s/it]
                                                        

 69%|██████▉   | 8309/12000 [4:54:39<2:17:39,  2.24s/it]
 69%|██████▉   | 8310/12000 [4:54:41<2:18:54,  2.26s/it]
                                                        

 69%|██████▉   | 8310/12000 [4:54:41<2:18:54,  2.26s/it]
 69%|██████▉   | 8311/12000 [4:54:43<2:06:18,  2.05s/it]
                                                        

 69%|██████▉   | 8311/12000 [4:54:43<2:06:18,  2.05s/it]
 69%|██████▉   | 8312/12000 [4:54:44<2:01:37,  1.98s/it]
                                                        

 69%|██████▉   | 8312/12000 [4:54:44<2:01:37,  1.98s/it]
 69%|██████▉   | 8313/12000 [4:54:46<1:58:36,  1.93s/it]
                                                        

 69%|██████▉   | 8313/12000 [4:54:46<1:58:36,  1.93s/it]
 69%|██████▉   | 8314/12000 [4:54:48<1:51:05,  1.81s/it]
                                                        

 69%|██████▉   | 8314/12000 [4:54:48<1:51:05,  1.81s/it]
 69%|██████▉   | 8315/12000 [4:54:49<1:47:35,  1.75s/it]
                                                        

 69%|██████▉   | 8315/12000 [4:54:49<1:47:35,  1.75s/it]
 69%|██████▉   | 8316/12000 [4:54:52<1:57:20,  1.91s/it]
                                                        

 69%|██████▉   | 8316/12000 [4:54:52<1:57:20,  1.91s/it]
 69%|██████▉   | 8317/12000 [4:54:56<2:44:49,  2.69s/it]
                                                        

 69%|██████▉   | 8317/12000 [4:54:56<2:44:49,  2.69s/it]
 69%|██████▉   | 8318/12000 [4:54:58<2:24:47,  2.36s/it]
                                                        

 69%|██████▉   | 8318/12000 [4:54:58<2:24:47,  2.36s/it]
 69%|██████▉   | 8319/12000 [4:54:59<2:10:27,  2.13s/it]
                                                        

 69%|██████▉   | 8319/12000 [4:54:59<2:10:27,  2.13s/it]
 69%|██████▉   | 8320/12000 [4:55:01<2:00:46,  1.97s/it]
                                                        

 69%|██████▉   | 8320/12000 [4:55:01<2:00:46,  1.97s/it]
 69%|██████▉   | 8321/12000 [4:55:02<1:51:35,  1.82s/it]
                                                        

 69%|██████▉   | 8321/12000 [4:55:02<1:51:35,  1.82s/it]
 69%|██████▉   | 8322/12000 [4:55:04<1:50:52,  1.81s/it]
                                                        

 69%|██████▉   | 8322/12000 [4:55:04<1:50:52,  1.81s/it]
 69%|██████▉   | 8323/12000 [4:55:07<2:16:20,  2.22s/it]
                                                        
{'loss': '0.7609', 'grad_norm': '0.3318', 'learning_rate': '4.822e-05', 'entropy': '0.7655', 'num_tokens': '4.462e+07', 'mean_token_accuracy': '0.8022', 'epoch': '0.07324'}
{'loss': '1.044', 'grad_norm': '0.4057', 'learning_rate': '4.82e-05', 'entropy': '1.008', 'num_tokens': '4.462e+07', 'mean_token_accuracy': '0.75', 'epoch': '0.07325'}
{'loss': '0.7462', 'grad_norm': '0.4019', 'learning_rate': '4.817e-05', 'entropy': '0.8272', 'num_tokens': '4.463e+07', 'mean_token_accuracy': '0.798', 'epoch': '0.07326'}
{'loss': '0.8556', 'grad_norm': '0.4127', 'learning_rate': '4.815e-05', 'entropy': '0.8835', 'num_tokens': '4.463e+07', 'mean_token_accuracy': '0.7806', 'epoch': '0.07327'}
{'loss': '0.7941', 'grad_norm': '0.4397', 'learning_rate': '4.813e-05', 'entropy': '0.73', 'num_tokens': '4.464e+07', 'mean_token_accuracy': '0.8178', 'epoch': '0.07328'}
{'loss': '0.9881', 'grad_norm': '0.4875', 'learning_rate': '4.81e-05', 'entropy': '0.9866', 'num_tokens': '4.464e+07', 'mean_token_accuracy': '0.749', 'epoch': '0.07329'}
{'loss': '0.6166', 'grad_norm': '0.3723', 'learning_rate': '4.808e-05', 'entropy': '0.6108', 'num_tokens': '4.465e+07', 'mean_token_accuracy': '0.8406', 'epoch': '0.07329'}
{'loss': '0.703', 'grad_norm': '0.413', 'learning_rate': '4.806e-05', 'entropy': '0.75', 'num_tokens': '4.465e+07', 'mean_token_accuracy': '0.8022', 'epoch': '0.0733'}
{'loss': '0.5792', 'grad_norm': '0.3507', 'learning_rate': '4.803e-05', 'entropy': '0.5963', 'num_tokens': '4.466e+07', 'mean_token_accuracy': '0.8371', 'epoch': '0.07331'}
{'loss': '0.7787', 'grad_norm': '0.4651', 'learning_rate': '4.801e-05', 'entropy': '0.7631', 'num_tokens': '4.466e+07', 'mean_token_accuracy': '0.8062', 'epoch': '0.07332'}
{'loss': '1.191', 'grad_norm': '0.3699', 'learning_rate': '4.799e-05', 'entropy': '1.006', 'num_tokens': '4.467e+07', 'mean_token_accuracy': '0.7411', 'epoch': '0.07333'}
{'loss': '0.7225', 'grad_norm': '0.4605', 'learning_rate': '4.796e-05', 'entropy': '0.7958', 'num_tokens': '4.468e+07', 'mean_token_accuracy': '0.7917', 'epoch': '0.07334'}
{'loss': '0.619', 'grad_norm': '0.344', 'learning_rate': '4.794e-05', 'entropy': '0.6483', 'num_tokens': '4.468e+07', 'mean_token_accuracy': '0.8163', 'epoch': '0.07335'}
{'loss': '1.055', 'grad_norm': '0.3378', 'learning_rate': '4.791e-05', 'entropy': '0.8304', 'num_tokens': '4.469e+07', 'mean_token_accuracy': '0.7994', 'epoch': '0.07336'}
{'loss': '0.4748', 'grad_norm': '0.3201', 'learning_rate': '4.789e-05', 'entropy': '0.5207', 'num_tokens': '4.469e+07', 'mean_token_accuracy': '0.8668', 'epoch': '0.07337'}
{'loss': '0.703', 'grad_norm': '0.3711', 'learning_rate': '4.787e-05', 'entropy': '0.746', 'num_tokens': '4.47e+07', 'mean_token_accuracy': '0.8002', 'epoch': '0.07337'}
{'loss': '1.258', 'grad_norm': '0.4725', 'learning_rate': '4.784e-05', 'entropy': '1.357', 'num_tokens': '4.47e+07', 'mean_token_accuracy': '0.6995', 'epoch': '0.07338'}
{'loss': '1.078', 'grad_norm': '0.407', 'learning_rate': '4.782e-05', 'entropy': '1.121', 'num_tokens': '4.471e+07', 'mean_token_accuracy': '0.7366', 'epoch': '0.07339'}
{'loss': '0.7574', 'grad_norm': '0.3908', 'learning_rate': '4.78e-05', 'entropy': '0.6282', 'num_tokens': '4.471e+07', 'mean_token_accuracy': '0.8466', 'epoch': '0.0734'}
{'loss': '1.096', 'grad_norm': '0.3942', 'learning_rate': '4.777e-05', 'entropy': '1.096', 'num_tokens': '4.472e+07', 'mean_token_accuracy': '0.7502', 'epoch': '0.07341'}
{'loss': '0.8705', 'grad_norm': '0.4874', 'learning_rate': '4.775e-05', 'entropy': '0.8259', 'num_tokens': '4.472e+07', 'mean_token_accuracy': '0.7781', 'epoch': '0.07342'}
{'loss': '0.757', 'grad_norm': '0.3174', 'learning_rate': '4.773e-05', 'entropy': '0.7507', 'num_tokens': '4.473e+07', 'mean_token_accuracy': '0.7974', 'epoch': '0.07343'}
{'loss': '0.9623', 'grad_norm': '0.4245', 'learning_rate': '4.77e-05', 'entropy': '0.8534', 'num_tokens': '4.473e+07', 'mean_token_accuracy': '0.7825', 'epoch': '0.07344'}
{'loss': '0.812', 'grad_norm': '0.363', 'learning_rate': '4.768e-05', 'entropy': '0.8477', 'num_tokens': '4.474e+07', 'mean_token_accuracy': '0.7858', 'epoch': '0.07344'}
{'loss': '0.5637', 'grad_norm': '0.3786', 'learning_rate': '4.766e-05', 'entropy': '0.6191', 'num_tokens': '4.474e+07', 'mean_token_accuracy': '0.8433', 'epoch': '0.07345'}
{'loss': '0.9051', 'grad_norm': '0.4095', 'learning_rate': '4.763e-05', 'entropy': '0.9567', 'num_tokens': '4.475e+07', 'mean_token_accuracy': '0.7636', 'epoch': '0.07346'}
{'loss': '1.019', 'grad_norm': '0.488', 'learning_rate': '4.761e-05', 'entropy': '1.003', 'num_tokens': '4.475e+07', 'mean_token_accuracy': '0.7582', 'epoch': '0.07347'}
{'loss': '0.6272', 'grad_norm': '0.3295', 'learning_rate': '4.759e-05', 'entropy': '0.6987', 'num_tokens': '4.476e+07', 'mean_token_accuracy': '0.8177', 'epoch': '0.07348'}
{'loss': '0.5833', 'grad_norm': '0.3978', 'learning_rate': '4.756e-05', 'entropy': '0.6202', 'num_tokens': '4.476e+07', 'mean_token_accuracy': '0.8416', 'epoch': '0.07349'}
{'loss': '0.5536', 'grad_norm': '0.315', 'learning_rate': '4.754e-05', 'entropy': '0.5695', 'num_tokens': '4.477e+07', 'mean_token_accuracy': '0.8561', 'epoch': '0.0735'}
{'loss': '0.8835', 'grad_norm': '0.3872', 'learning_rate': '4.752e-05', 'entropy': '0.897', 'num_tokens': '4.477e+07', 'mean_token_accuracy': '0.7836', 'epoch': '0.07351'}
{'loss': '1.122', 'grad_norm': '0.3712', 'learning_rate': '4.749e-05', 'entropy': '0.8921', 'num_tokens': '4.478e+07', 'mean_token_accuracy': '0.776', 'epoch': '0.07352'}
{'loss': '0.8652', 'grad_norm': '0.3825', 'learning_rate': '4.747e-05', 'entropy': '0.7717', 'num_tokens': '4.479e+07', 'mean_token_accuracy': '0.8158', 'epoch': '0.07352'}
{'loss': '0.5567', 'grad_norm': '0.3815', 'learning_rate': '4.745e-05', 'entropy': '0.5968', 'num_tokens': '4.479e+07', 'mean_token_accuracy': '0.8394', 'epoch': '0.07353'}
{'loss': '0.8932', 'grad_norm': '0.3888', 'learning_rate': '4.742e-05', 'entropy': '0.9888', 'num_tokens': '4.48e+07', 'mean_token_accuracy': '0.7557', 'epoch': '0.07354'}
{'loss': '0.9614', 'grad_norm': '0.4254', 'learning_rate': '4.74e-05', 'entropy': '0.9851', 'num_tokens': '4.48e+07', 'mean_token_accuracy': '0.7567', 'epoch': '0.07355'}
{'loss': '1.071', 'grad_norm': '0.496', 'learning_rate': '4.737e-05', 'entropy': '1.051', 'num_tokens': '4.481e+07', 'mean_token_accuracy': '0.741', 'epoch': '0.07356'}
{'loss': '0.8112', 'grad_norm': '0.4046', 'learning_rate': '4.735e-05', 'entropy': '0.7869', 'num_tokens': '4.481e+07', 'mean_token_accuracy': '0.7988', 'epoch': '0.07357'}
{'loss': '0.9765', 'grad_norm': '0.391', 'learning_rate': '4.733e-05', 'entropy': '0.9424', 'num_tokens': '4.482e+07', 'mean_token_accuracy': '0.7707', 'epoch': '0.07358'}
{'loss': '0.6523', 'grad_norm': '0.3989', 'learning_rate': '4.73e-05', 'entropy': '0.6837', 'num_tokens': '4.482e+07', 'mean_token_accuracy': '0.8178', 'epoch': '0.07359'}
{'loss': '0.809', 'grad_norm': '0.3704', 'learning_rate': '4.728e-05', 'entropy': '0.8812', 'num_tokens': '4.483e+07', 'mean_token_accuracy': '0.7925', 'epoch': '0.0736'}
{'loss': '1.079', 'grad_norm': '0.3295', 'learning_rate': '4.726e-05', 'entropy': '1.166', 'num_tokens': '4.484e+07', 'mean_token_accuracy': '0.7217', 'epoch': '0.0736'}
{'loss': '0.559', 'grad_norm': '0.301', 'learning_rate': '4.723e-05', 'entropy': '0.6083', 'num_tokens': '4.484e+07', 'mean_token_accuracy': '0.838', 'epoch': '0.07361'}
{'loss': '0.5849', 'grad_norm': '0.3526', 'learning_rate': '4.721e-05', 'entropy': '0.6189', 'num_tokens': '4.485e+07', 'mean_token_accuracy': '0.84', 'epoch': '0.07362'}
{'loss': '0.9009', 'grad_norm': '0.3496', 'learning_rate': '4.719e-05', 'entropy': '0.9235', 'num_tokens': '4.485e+07', 'mean_token_accuracy': '0.7574', 'epoch': '0.07363'}
{'loss': '0.7428', 'grad_norm': '0.3842', 'learning_rate': '4.716e-05', 'entropy': '0.7656', 'num_tokens': '4.486e+07', 'mean_token_accuracy': '0.8055', 'epoch': '0.07364'}
{'loss': '0.6777', 'grad_norm': '0.3562', 'learning_rate': '4.714e-05', 'entropy': '0.7108', 'num_tokens': '4.486e+07', 'mean_token_accuracy': '0.822', 'epoch': '0.07365'}

 69%|██████▉   | 8323/12000 [4:55:07<2:16:20,  2.22s/it]
 69%|██████▉   | 8324/12000 [4:55:09<2:07:19,  2.08s/it]
                                                        

 69%|██████▉   | 8324/12000 [4:55:09<2:07:19,  2.08s/it]
 69%|██████▉   | 8325/12000 [4:55:11<1:57:42,  1.92s/it]
                                                        

 69%|██████▉   | 8325/12000 [4:55:11<1:57:42,  1.92s/it]
 69%|██████▉   | 8326/12000 [4:55:12<1:51:40,  1.82s/it]
                                                        

 69%|██████▉   | 8326/12000 [4:55:12<1:51:40,  1.82s/it]
 69%|██████▉   | 8327/12000 [4:55:16<2:25:39,  2.38s/it]
                                                        

 69%|██████▉   | 8327/12000 [4:55:16<2:25:39,  2.38s/it]
 69%|██████▉   | 8328/12000 [4:55:19<2:37:58,  2.58s/it]
                                                        

 69%|██████▉   | 8328/12000 [4:55:19<2:37:58,  2.58s/it]
 69%|██████▉   | 8329/12000 [4:55:22<2:35:42,  2.54s/it]
                                                        

 69%|██████▉   | 8329/12000 [4:55:22<2:35:42,  2.54s/it]
 69%|██████▉   | 8330/12000 [4:55:25<2:49:55,  2.78s/it]
                                                        

 69%|██████▉   | 8330/12000 [4:55:25<2:49:55,  2.78s/it]
 69%|██████▉   | 8331/12000 [4:55:26<2:28:00,  2.42s/it]
                                                        

 69%|██████▉   | 8331/12000 [4:55:26<2:28:00,  2.42s/it]
 69%|██████▉   | 8332/12000 [4:55:28<2:21:25,  2.31s/it]
                                                        

 69%|██████▉   | 8332/12000 [4:55:28<2:21:25,  2.31s/it]
 69%|██████▉   | 8333/12000 [4:55:30<2:04:52,  2.04s/it]
                                                        

 69%|██████▉   | 8333/12000 [4:55:30<2:04:52,  2.04s/it]
 69%|██████▉   | 8334/12000 [4:55:32<1:58:35,  1.94s/it]
                                                        

 69%|██████▉   | 8334/12000 [4:55:32<1:58:35,  1.94s/it]
 69%|██████▉   | 8335/12000 [4:55:33<1:52:48,  1.85s/it]
                                                        

 69%|██████▉   | 8335/12000 [4:55:33<1:52:48,  1.85s/it]
 69%|██████▉   | 8336/12000 [4:55:35<1:46:23,  1.74s/it]
                                                        

 69%|██████▉   | 8336/12000 [4:55:35<1:46:23,  1.74s/it]
 69%|██████▉   | 8337/12000 [4:55:38<2:09:27,  2.12s/it]
                                                        

 69%|██████▉   | 8337/12000 [4:55:38<2:09:27,  2.12s/it]
 69%|██████▉   | 8338/12000 [4:55:40<2:09:09,  2.12s/it]
                                                        

 69%|██████▉   | 8338/12000 [4:55:40<2:09:09,  2.12s/it]
 69%|██████▉   | 8339/12000 [4:55:41<1:57:21,  1.92s/it]
                                                        

 69%|██████▉   | 8339/12000 [4:55:41<1:57:21,  1.92s/it]
 70%|██████▉   | 8340/12000 [4:55:43<1:49:03,  1.79s/it]
                                                        

 70%|██████▉   | 8340/12000 [4:55:43<1:49:03,  1.79s/it]
 70%|██████▉   | 8341/12000 [4:55:44<1:45:33,  1.73s/it]
                                                        

 70%|██████▉   | 8341/12000 [4:55:44<1:45:33,  1.73s/it]
 70%|██████▉   | 8342/12000 [4:55:46<1:44:43,  1.72s/it]
                                                        

 70%|██████▉   | 8342/12000 [4:55:46<1:44:43,  1.72s/it]
 70%|██████▉   | 8343/12000 [4:55:48<1:41:08,  1.66s/it]
                                                        

 70%|██████▉   | 8343/12000 [4:55:48<1:41:08,  1.66s/it]
 70%|██████▉   | 8344/12000 [4:55:49<1:38:08,  1.61s/it]
                                                        

 70%|██████▉   | 8344/12000 [4:55:49<1:38:08,  1.61s/it]
 70%|██████▉   | 8345/12000 [4:55:51<1:38:52,  1.62s/it]
                                                        

 70%|██████▉   | 8345/12000 [4:55:51<1:38:52,  1.62s/it]
 70%|██████▉   | 8346/12000 [4:55:53<1:57:53,  1.94s/it]
                                                        

 70%|██████▉   | 8346/12000 [4:55:53<1:57:53,  1.94s/it]
 70%|██████▉   | 8347/12000 [4:55:55<1:49:08,  1.79s/it]
                                                        

 70%|██████▉   | 8347/12000 [4:55:55<1:49:08,  1.79s/it]
 70%|██████▉   | 8348/12000 [4:55:56<1:43:48,  1.71s/it]
                                                        

 70%|██████▉   | 8348/12000 [4:55:56<1:43:48,  1.71s/it]
 70%|██████▉   | 8349/12000 [4:56:00<2:16:37,  2.25s/it]
                                                        

 70%|██████▉   | 8349/12000 [4:56:00<2:16:37,  2.25s/it]
 70%|██████▉   | 8350/12000 [4:56:02<2:17:30,  2.26s/it]
                                                        

 70%|██████▉   | 8350/12000 [4:56:02<2:17:30,  2.26s/it]
 70%|██████▉   | 8351/12000 [4:56:06<2:43:05,  2.68s/it]
                                                        

 70%|██████▉   | 8351/12000 [4:56:06<2:43:05,  2.68s/it]
 70%|██████▉   | 8352/12000 [4:56:07<2:23:00,  2.35s/it]
                                                        

 70%|██████▉   | 8352/12000 [4:56:07<2:23:00,  2.35s/it]
 70%|██████▉   | 8353/12000 [4:56:10<2:24:21,  2.37s/it]
                                                        

 70%|██████▉   | 8353/12000 [4:56:10<2:24:21,  2.37s/it]
 70%|██████▉   | 8354/12000 [4:56:12<2:12:16,  2.18s/it]
                                                        

 70%|██████▉   | 8354/12000 [4:56:12<2:12:16,  2.18s/it]
 70%|██████▉   | 8355/12000 [4:56:13<2:03:11,  2.03s/it]
                                                        

 70%|██████▉   | 8355/12000 [4:56:13<2:03:11,  2.03s/it]
 70%|██████▉   | 8356/12000 [4:56:18<2:54:31,  2.87s/it]
                                                        

 70%|██████▉   | 8356/12000 [4:56:18<2:54:31,  2.87s/it]
 70%|██████▉   | 8357/12000 [4:56:20<2:35:44,  2.57s/it]
                                                        

 70%|██████▉   | 8357/12000 [4:56:20<2:35:44,  2.57s/it]
 70%|██████▉   | 8358/12000 [4:56:22<2:19:20,  2.30s/it]
                                                        

 70%|██████▉   | 8358/12000 [4:56:22<2:19:20,  2.30s/it]
 70%|██████▉   | 8359/12000 [4:56:24<2:14:06,  2.21s/it]
                                                        

 70%|██████▉   | 8359/12000 [4:56:24<2:14:06,  2.21s/it]
 70%|██████▉   | 8360/12000 [4:56:25<2:08:05,  2.11s/it]
                                                        

 70%|██████▉   | 8360/12000 [4:56:25<2:08:05,  2.11s/it]
 70%|██████▉   | 8361/12000 [4:56:27<2:00:25,  1.99s/it]
                                                        

 70%|██████▉   | 8361/12000 [4:56:27<2:00:25,  1.99s/it]
 70%|██████▉   | 8362/12000 [4:56:30<2:08:06,  2.11s/it]
                                                        

 70%|██████▉   | 8362/12000 [4:56:30<2:08:06,  2.11s/it]
 70%|██████▉   | 8363/12000 [4:56:31<1:59:41,  1.97s/it]
                                                        

 70%|██████▉   | 8363/12000 [4:56:31<1:59:41,  1.97s/it]
 70%|██████▉   | 8364/12000 [4:56:33<1:54:51,  1.90s/it]
                                                        

 70%|██████▉   | 8364/12000 [4:56:33<1:54:51,  1.90s/it]
 70%|██████▉   | 8365/12000 [4:56:35<2:00:46,  1.99s/it]
                                                        

 70%|██████▉   | 8365/12000 [4:56:35<2:00:46,  1.99s/it]
 70%|██████▉   | 8366/12000 [4:56:38<2:08:12,  2.12s/it]
                                                        

 70%|██████▉   | 8366/12000 [4:56:38<2:08:12,  2.12s/it]
 70%|██████▉   | 8367/12000 [4:56:39<1:58:28,  1.96s/it]
                                                        

 70%|██████▉   | 8367/12000 [4:56:39<1:58:28,  1.96s/it]
 70%|██████▉   | 8368/12000 [4:56:41<1:53:25,  1.87s/it]
                                                        

 70%|██████▉   | 8368/12000 [4:56:41<1:53:25,  1.87s/it]
 70%|██████▉   | 8369/12000 [4:56:43<1:52:33,  1.86s/it]
                                                        

 70%|██████▉   | 8369/12000 [4:56:43<1:52:33,  1.86s/it]
 70%|██████▉   | 8370/12000 [4:56:44<1:46:36,  1.76s/it]
                                                        
{'loss': '0.6646', 'grad_norm': '0.2647', 'learning_rate': '4.712e-05', 'entropy': '0.6483', 'num_tokens': '4.487e+07', 'mean_token_accuracy': '0.8396', 'epoch': '0.07366'}
{'loss': '0.7182', 'grad_norm': '0.3904', 'learning_rate': '4.709e-05', 'entropy': '0.7098', 'num_tokens': '4.487e+07', 'mean_token_accuracy': '0.8094', 'epoch': '0.07367'}
{'loss': '0.7189', 'grad_norm': '0.4455', 'learning_rate': '4.707e-05', 'entropy': '0.7821', 'num_tokens': '4.488e+07', 'mean_token_accuracy': '0.8001', 'epoch': '0.07367'}
{'loss': '0.7867', 'grad_norm': '0.4167', 'learning_rate': '4.705e-05', 'entropy': '0.7752', 'num_tokens': '4.488e+07', 'mean_token_accuracy': '0.8003', 'epoch': '0.07368'}
{'loss': '0.6764', 'grad_norm': '0.3319', 'learning_rate': '4.702e-05', 'entropy': '0.7113', 'num_tokens': '4.489e+07', 'mean_token_accuracy': '0.8059', 'epoch': '0.07369'}
{'loss': '0.7471', 'grad_norm': '0.3022', 'learning_rate': '4.7e-05', 'entropy': '0.7192', 'num_tokens': '4.489e+07', 'mean_token_accuracy': '0.8114', 'epoch': '0.0737'}
{'loss': '0.8784', 'grad_norm': '0.3347', 'learning_rate': '4.698e-05', 'entropy': '0.8601', 'num_tokens': '4.49e+07', 'mean_token_accuracy': '0.7801', 'epoch': '0.07371'}
{'loss': '0.5454', 'grad_norm': '0.394', 'learning_rate': '4.695e-05', 'entropy': '0.6239', 'num_tokens': '4.491e+07', 'mean_token_accuracy': '0.828', 'epoch': '0.07372'}
{'loss': '0.7446', 'grad_norm': '0.3545', 'learning_rate': '4.693e-05', 'entropy': '0.7706', 'num_tokens': '4.491e+07', 'mean_token_accuracy': '0.8', 'epoch': '0.07373'}
{'loss': '1.152', 'grad_norm': '0.4533', 'learning_rate': '4.691e-05', 'entropy': '1.168', 'num_tokens': '4.492e+07', 'mean_token_accuracy': '0.7218', 'epoch': '0.07374'}
{'loss': '0.6082', 'grad_norm': '0.327', 'learning_rate': '4.688e-05', 'entropy': '0.7389', 'num_tokens': '4.492e+07', 'mean_token_accuracy': '0.8173', 'epoch': '0.07375'}
{'loss': '0.725', 'grad_norm': '0.4555', 'learning_rate': '4.686e-05', 'entropy': '0.746', 'num_tokens': '4.493e+07', 'mean_token_accuracy': '0.814', 'epoch': '0.07375'}
{'loss': '0.9213', 'grad_norm': '0.3524', 'learning_rate': '4.684e-05', 'entropy': '0.8676', 'num_tokens': '4.493e+07', 'mean_token_accuracy': '0.767', 'epoch': '0.07376'}
{'loss': '0.6493', 'grad_norm': '0.359', 'learning_rate': '4.681e-05', 'entropy': '0.6275', 'num_tokens': '4.494e+07', 'mean_token_accuracy': '0.8363', 'epoch': '0.07377'}
{'loss': '0.5207', 'grad_norm': '0.2947', 'learning_rate': '4.679e-05', 'entropy': '0.5102', 'num_tokens': '4.494e+07', 'mean_token_accuracy': '0.8666', 'epoch': '0.07378'}
{'loss': '0.9414', 'grad_norm': '0.3807', 'learning_rate': '4.677e-05', 'entropy': '1.017', 'num_tokens': '4.495e+07', 'mean_token_accuracy': '0.739', 'epoch': '0.07379'}
{'loss': '0.7799', 'grad_norm': '0.4001', 'learning_rate': '4.674e-05', 'entropy': '0.8404', 'num_tokens': '4.495e+07', 'mean_token_accuracy': '0.7881', 'epoch': '0.0738'}
{'loss': '0.6269', 'grad_norm': '0.3753', 'learning_rate': '4.672e-05', 'entropy': '0.6194', 'num_tokens': '4.496e+07', 'mean_token_accuracy': '0.8507', 'epoch': '0.07381'}
{'loss': '0.6103', 'grad_norm': '0.4326', 'learning_rate': '4.67e-05', 'entropy': '0.6289', 'num_tokens': '4.496e+07', 'mean_token_accuracy': '0.8391', 'epoch': '0.07382'}
{'loss': '0.9164', 'grad_norm': '0.4533', 'learning_rate': '4.667e-05', 'entropy': '0.9138', 'num_tokens': '4.497e+07', 'mean_token_accuracy': '0.7723', 'epoch': '0.07383'}
{'loss': '0.8253', 'grad_norm': '0.4223', 'learning_rate': '4.665e-05', 'entropy': '0.8461', 'num_tokens': '4.497e+07', 'mean_token_accuracy': '0.7805', 'epoch': '0.07383'}
{'loss': '0.7271', 'grad_norm': '0.4198', 'learning_rate': '4.663e-05', 'entropy': '0.7534', 'num_tokens': '4.498e+07', 'mean_token_accuracy': '0.8027', 'epoch': '0.07384'}
{'loss': '0.9491', 'grad_norm': '0.4202', 'learning_rate': '4.66e-05', 'entropy': '0.9798', 'num_tokens': '4.498e+07', 'mean_token_accuracy': '0.7657', 'epoch': '0.07385'}
{'loss': '0.9237', 'grad_norm': '0.3804', 'learning_rate': '4.658e-05', 'entropy': '0.8015', 'num_tokens': '4.499e+07', 'mean_token_accuracy': '0.795', 'epoch': '0.07386'}
{'loss': '1.134', 'grad_norm': '0.5068', 'learning_rate': '4.656e-05', 'entropy': '1.109', 'num_tokens': '4.499e+07', 'mean_token_accuracy': '0.7341', 'epoch': '0.07387'}
{'loss': '0.6841', 'grad_norm': '0.325', 'learning_rate': '4.653e-05', 'entropy': '0.7439', 'num_tokens': '4.5e+07', 'mean_token_accuracy': '0.8214', 'epoch': '0.07388'}
{'loss': '0.8561', 'grad_norm': '0.4017', 'learning_rate': '4.651e-05', 'entropy': '0.7997', 'num_tokens': '4.5e+07', 'mean_token_accuracy': '0.8165', 'epoch': '0.07389'}
{'loss': '0.5966', 'grad_norm': '0.3733', 'learning_rate': '4.649e-05', 'entropy': '0.7026', 'num_tokens': '4.501e+07', 'mean_token_accuracy': '0.8176', 'epoch': '0.0739'}
{'loss': '0.6668', 'grad_norm': '0.3791', 'learning_rate': '4.646e-05', 'entropy': '0.6837', 'num_tokens': '4.501e+07', 'mean_token_accuracy': '0.8239', 'epoch': '0.0739'}
{'loss': '0.7339', 'grad_norm': '0.3737', 'learning_rate': '4.644e-05', 'entropy': '0.7426', 'num_tokens': '4.502e+07', 'mean_token_accuracy': '0.8082', 'epoch': '0.07391'}
{'loss': '0.9505', 'grad_norm': '0.3832', 'learning_rate': '4.642e-05', 'entropy': '0.9379', 'num_tokens': '4.502e+07', 'mean_token_accuracy': '0.7664', 'epoch': '0.07392'}
{'loss': '0.537', 'grad_norm': '0.351', 'learning_rate': '4.639e-05', 'entropy': '0.5817', 'num_tokens': '4.503e+07', 'mean_token_accuracy': '0.8421', 'epoch': '0.07393'}
{'loss': '0.802', 'grad_norm': '0.3336', 'learning_rate': '4.637e-05', 'entropy': '0.7884', 'num_tokens': '4.503e+07', 'mean_token_accuracy': '0.8023', 'epoch': '0.07394'}
{'loss': '0.4619', 'grad_norm': '0.4271', 'learning_rate': '4.635e-05', 'entropy': '0.6852', 'num_tokens': '4.504e+07', 'mean_token_accuracy': '0.8279', 'epoch': '0.07395'}
{'loss': '0.799', 'grad_norm': '0.3932', 'learning_rate': '4.632e-05', 'entropy': '0.8674', 'num_tokens': '4.505e+07', 'mean_token_accuracy': '0.7866', 'epoch': '0.07396'}
{'loss': '0.5647', 'grad_norm': '0.3742', 'learning_rate': '4.63e-05', 'entropy': '0.5483', 'num_tokens': '4.505e+07', 'mean_token_accuracy': '0.8617', 'epoch': '0.07397'}
{'loss': '1.18', 'grad_norm': '0.3771', 'learning_rate': '4.628e-05', 'entropy': '1.16', 'num_tokens': '4.506e+07', 'mean_token_accuracy': '0.7271', 'epoch': '0.07398'}
{'loss': '0.5922', 'grad_norm': '0.4068', 'learning_rate': '4.625e-05', 'entropy': '0.585', 'num_tokens': '4.506e+07', 'mean_token_accuracy': '0.8361', 'epoch': '0.07398'}
{'loss': '0.7297', 'grad_norm': '0.363', 'learning_rate': '4.623e-05', 'entropy': '0.7218', 'num_tokens': '4.507e+07', 'mean_token_accuracy': '0.8165', 'epoch': '0.07399'}
{'loss': '0.7446', 'grad_norm': '0.2798', 'learning_rate': '4.621e-05', 'entropy': '0.7932', 'num_tokens': '4.508e+07', 'mean_token_accuracy': '0.8069', 'epoch': '0.074'}
{'loss': '0.4954', 'grad_norm': '0.3817', 'learning_rate': '4.618e-05', 'entropy': '0.519', 'num_tokens': '4.508e+07', 'mean_token_accuracy': '0.8603', 'epoch': '0.07401'}
{'loss': '0.6101', 'grad_norm': '0.3628', 'learning_rate': '4.616e-05', 'entropy': '0.6556', 'num_tokens': '4.509e+07', 'mean_token_accuracy': '0.8314', 'epoch': '0.07402'}
{'loss': '0.8089', 'grad_norm': '0.4664', 'learning_rate': '4.614e-05', 'entropy': '0.7482', 'num_tokens': '4.509e+07', 'mean_token_accuracy': '0.8107', 'epoch': '0.07403'}
{'loss': '0.9585', 'grad_norm': '0.5121', 'learning_rate': '4.612e-05', 'entropy': '0.9793', 'num_tokens': '4.51e+07', 'mean_token_accuracy': '0.7493', 'epoch': '0.07404'}
{'loss': '0.6822', 'grad_norm': '0.4221', 'learning_rate': '4.609e-05', 'entropy': '0.7012', 'num_tokens': '4.51e+07', 'mean_token_accuracy': '0.8166', 'epoch': '0.07405'}
{'loss': '0.7165', 'grad_norm': '0.3965', 'learning_rate': '4.607e-05', 'entropy': '0.7241', 'num_tokens': '4.511e+07', 'mean_token_accuracy': '0.8016', 'epoch': '0.07406'}
{'loss': '0.8108', 'grad_norm': '0.3608', 'learning_rate': '4.605e-05', 'entropy': '0.8673', 'num_tokens': '4.511e+07', 'mean_token_accuracy': '0.7846', 'epoch': '0.07406'}

 70%|██████▉   | 8370/12000 [4:56:44<1:46:36,  1.76s/it]
 70%|██████▉   | 8371/12000 [4:56:48<2:20:15,  2.32s/it]
                                                        

 70%|██████▉   | 8371/12000 [4:56:48<2:20:15,  2.32s/it]
 70%|██████▉   | 8372/12000 [4:56:50<2:22:47,  2.36s/it]
                                                        

 70%|██████▉   | 8372/12000 [4:56:50<2:22:47,  2.36s/it]
 70%|██████▉   | 8373/12000 [4:56:52<2:10:46,  2.16s/it]
                                                        

 70%|██████▉   | 8373/12000 [4:56:52<2:10:46,  2.16s/it]
 70%|██████▉   | 8374/12000 [4:56:54<2:03:00,  2.04s/it]
                                                        

 70%|██████▉   | 8374/12000 [4:56:54<2:03:00,  2.04s/it]
 70%|██████▉   | 8375/12000 [4:56:55<1:57:13,  1.94s/it]
                                                        

 70%|██████▉   | 8375/12000 [4:56:55<1:57:13,  1.94s/it]
 70%|██████▉   | 8376/12000 [4:56:57<1:52:16,  1.86s/it]
                                                        

 70%|██████▉   | 8376/12000 [4:56:57<1:52:16,  1.86s/it]
 70%|██████▉   | 8377/12000 [4:56:59<1:54:10,  1.89s/it]
                                                        

 70%|██████▉   | 8377/12000 [4:56:59<1:54:10,  1.89s/it]
 70%|██████▉   | 8378/12000 [4:57:01<1:46:03,  1.76s/it]
                                                        

 70%|██████▉   | 8378/12000 [4:57:01<1:46:03,  1.76s/it]
 70%|██████▉   | 8379/12000 [4:57:02<1:45:04,  1.74s/it]
                                                        

 70%|██████▉   | 8379/12000 [4:57:02<1:45:04,  1.74s/it]
 70%|██████▉   | 8380/12000 [4:57:04<1:41:05,  1.68s/it]
                                                        

 70%|██████▉   | 8380/12000 [4:57:04<1:41:05,  1.68s/it]
 70%|██████▉   | 8381/12000 [4:57:06<1:44:24,  1.73s/it]
                                                        

 70%|██████▉   | 8381/12000 [4:57:06<1:44:24,  1.73s/it]
 70%|██████▉   | 8382/12000 [4:57:07<1:46:49,  1.77s/it]
                                                        

 70%|██████▉   | 8382/12000 [4:57:07<1:46:49,  1.77s/it]
 70%|██████▉   | 8383/12000 [4:57:10<1:54:20,  1.90s/it]
                                                        

 70%|██████▉   | 8383/12000 [4:57:10<1:54:20,  1.90s/it]
 70%|██████▉   | 8384/12000 [4:57:11<1:50:46,  1.84s/it]
                                                        

 70%|██████▉   | 8384/12000 [4:57:11<1:50:46,  1.84s/it]
 70%|██████▉   | 8385/12000 [4:57:13<1:48:20,  1.80s/it]
                                                        

 70%|██████▉   | 8385/12000 [4:57:13<1:48:20,  1.80s/it]
 70%|██████▉   | 8386/12000 [4:57:15<1:45:47,  1.76s/it]
                                                        

 70%|██████▉   | 8386/12000 [4:57:15<1:45:47,  1.76s/it]
 70%|██████▉   | 8387/12000 [4:57:16<1:43:45,  1.72s/it]
                                                        

 70%|██████▉   | 8387/12000 [4:57:16<1:43:45,  1.72s/it]
 70%|██████▉   | 8388/12000 [4:57:18<1:48:39,  1.80s/it]
                                                        

 70%|██████▉   | 8388/12000 [4:57:18<1:48:39,  1.80s/it]
 70%|██████▉   | 8389/12000 [4:57:20<1:52:55,  1.88s/it]
                                                        

 70%|██████▉   | 8389/12000 [4:57:20<1:52:55,  1.88s/it]
 70%|██████▉   | 8390/12000 [4:57:23<2:02:46,  2.04s/it]
                                                        

 70%|██████▉   | 8390/12000 [4:57:23<2:02:46,  2.04s/it]
 70%|██████▉   | 8391/12000 [4:57:25<1:57:53,  1.96s/it]
                                                        

 70%|██████▉   | 8391/12000 [4:57:25<1:57:53,  1.96s/it]
 70%|██████▉   | 8392/12000 [4:57:27<1:57:21,  1.95s/it]
                                                        

 70%|██████▉   | 8392/12000 [4:57:27<1:57:21,  1.95s/it]
 70%|██████▉   | 8393/12000 [4:57:31<2:34:13,  2.57s/it]
                                                        

 70%|██████▉   | 8393/12000 [4:57:31<2:34:13,  2.57s/it]
 70%|██████▉   | 8394/12000 [4:57:33<2:32:45,  2.54s/it]
                                                        

 70%|██████▉   | 8394/12000 [4:57:33<2:32:45,  2.54s/it]
 70%|██████▉   | 8395/12000 [4:57:35<2:14:23,  2.24s/it]
                                                        

 70%|██████▉   | 8395/12000 [4:57:35<2:14:23,  2.24s/it]
 70%|██████▉   | 8396/12000 [4:57:37<2:09:45,  2.16s/it]
                                                        

 70%|██████▉   | 8396/12000 [4:57:37<2:09:45,  2.16s/it]
 70%|██████▉   | 8397/12000 [4:57:38<1:54:53,  1.91s/it]
                                                        

 70%|██████▉   | 8397/12000 [4:57:38<1:54:53,  1.91s/it]
 70%|██████▉   | 8398/12000 [4:57:40<1:58:33,  1.97s/it]
                                                        

 70%|██████▉   | 8398/12000 [4:57:40<1:58:33,  1.97s/it]
 70%|██████▉   | 8399/12000 [4:57:43<2:09:55,  2.16s/it]
                                                        

 70%|██████▉   | 8399/12000 [4:57:43<2:09:55,  2.16s/it]
 70%|███████   | 8400/12000 [4:57:44<2:03:48,  2.06s/it]
                                                        

 70%|███████   | 8400/12000 [4:57:44<2:03:48,  2.06s/it]
 70%|███████   | 8401/12000 [4:57:47<2:19:18,  2.32s/it]
                                                        

 70%|███████   | 8401/12000 [4:57:47<2:19:18,  2.32s/it]
 70%|███████   | 8402/12000 [4:57:50<2:32:28,  2.54s/it]
                                                        

 70%|███████   | 8402/12000 [4:57:50<2:32:28,  2.54s/it]
 70%|███████   | 8403/12000 [4:57:53<2:32:21,  2.54s/it]
                                                        

 70%|███████   | 8403/12000 [4:57:53<2:32:21,  2.54s/it]
 70%|███████   | 8404/12000 [4:57:56<2:37:28,  2.63s/it]
                                                        

 70%|███████   | 8404/12000 [4:57:56<2:37:28,  2.63s/it]
 70%|███████   | 8405/12000 [4:57:57<2:18:33,  2.31s/it]
                                                        

 70%|███████   | 8405/12000 [4:57:57<2:18:33,  2.31s/it]
 70%|███████   | 8406/12000 [4:58:00<2:25:10,  2.42s/it]
                                                        

 70%|███████   | 8406/12000 [4:58:00<2:25:10,  2.42s/it]
 70%|███████   | 8407/12000 [4:58:01<2:05:24,  2.09s/it]
                                                        

 70%|███████   | 8407/12000 [4:58:01<2:05:24,  2.09s/it]
 70%|███████   | 8408/12000 [4:58:03<1:56:02,  1.94s/it]
                                                        

 70%|███████   | 8408/12000 [4:58:03<1:56:02,  1.94s/it]
 70%|███████   | 8409/12000 [4:58:05<2:02:18,  2.04s/it]
                                                        

 70%|███████   | 8409/12000 [4:58:05<2:02:18,  2.04s/it]
 70%|███████   | 8410/12000 [4:58:07<2:00:59,  2.02s/it]
                                                        

 70%|███████   | 8410/12000 [4:58:07<2:00:59,  2.02s/it]
 70%|███████   | 8411/12000 [4:58:09<1:56:33,  1.95s/it]
                                                        

 70%|███████   | 8411/12000 [4:58:09<1:56:33,  1.95s/it]
 70%|███████   | 8412/12000 [4:58:11<2:04:59,  2.09s/it]
                                                        

 70%|███████   | 8412/12000 [4:58:11<2:04:59,  2.09s/it]
 70%|███████   | 8413/12000 [4:58:14<2:21:55,  2.37s/it]
                                                        

 70%|███████   | 8413/12000 [4:58:14<2:21:55,  2.37s/it]
 70%|███████   | 8414/12000 [4:58:16<2:11:30,  2.20s/it]
                                                        

 70%|███████   | 8414/12000 [4:58:16<2:11:30,  2.20s/it]
 70%|███████   | 8415/12000 [4:58:21<3:00:30,  3.02s/it]
                                                        

 70%|███████   | 8415/12000 [4:58:21<3:00:30,  3.02s/it]
 70%|███████   | 8416/12000 [4:58:23<2:32:30,  2.55s/it]
                                                        

 70%|███████   | 8416/12000 [4:58:23<2:32:30,  2.55s/it]
 70%|███████   | 8417/12000 [4:58:24<2:13:12,  2.23s/it]
                                                        
{'loss': '0.6418', 'grad_norm': '0.364', 'learning_rate': '4.602e-05', 'entropy': '0.6617', 'num_tokens': '4.512e+07', 'mean_token_accuracy': '0.8265', 'epoch': '0.07407'}
{'loss': '0.485', 'grad_norm': '0.3435', 'learning_rate': '4.6e-05', 'entropy': '0.5129', 'num_tokens': '4.512e+07', 'mean_token_accuracy': '0.8623', 'epoch': '0.07408'}
{'loss': '1.096', 'grad_norm': '0.453', 'learning_rate': '4.598e-05', 'entropy': '0.9479', 'num_tokens': '4.513e+07', 'mean_token_accuracy': '0.7626', 'epoch': '0.07409'}
{'loss': '0.8575', 'grad_norm': '0.5149', 'learning_rate': '4.595e-05', 'entropy': '0.8657', 'num_tokens': '4.513e+07', 'mean_token_accuracy': '0.781', 'epoch': '0.0741'}
{'loss': '1.055', 'grad_norm': '0.4865', 'learning_rate': '4.593e-05', 'entropy': '1.027', 'num_tokens': '4.514e+07', 'mean_token_accuracy': '0.7464', 'epoch': '0.07411'}
{'loss': '0.9265', 'grad_norm': '0.3977', 'learning_rate': '4.591e-05', 'entropy': '0.855', 'num_tokens': '4.514e+07', 'mean_token_accuracy': '0.7805', 'epoch': '0.07412'}
{'loss': '0.6831', 'grad_norm': '0.4465', 'learning_rate': '4.588e-05', 'entropy': '0.6677', 'num_tokens': '4.514e+07', 'mean_token_accuracy': '0.8292', 'epoch': '0.07413'}
{'loss': '0.7461', 'grad_norm': '0.4474', 'learning_rate': '4.586e-05', 'entropy': '0.6963', 'num_tokens': '4.515e+07', 'mean_token_accuracy': '0.8112', 'epoch': '0.07414'}
{'loss': '0.8983', 'grad_norm': '0.426', 'learning_rate': '4.584e-05', 'entropy': '0.8616', 'num_tokens': '4.515e+07', 'mean_token_accuracy': '0.7749', 'epoch': '0.07414'}
{'loss': '0.6636', 'grad_norm': '0.3858', 'learning_rate': '4.581e-05', 'entropy': '0.6793', 'num_tokens': '4.516e+07', 'mean_token_accuracy': '0.8245', 'epoch': '0.07415'}
{'loss': '0.6338', 'grad_norm': '0.3417', 'learning_rate': '4.579e-05', 'entropy': '0.619', 'num_tokens': '4.516e+07', 'mean_token_accuracy': '0.8396', 'epoch': '0.07416'}
{'loss': '1.085', 'grad_norm': '0.5303', 'learning_rate': '4.577e-05', 'entropy': '0.9204', 'num_tokens': '4.517e+07', 'mean_token_accuracy': '0.7722', 'epoch': '0.07417'}
{'loss': '1.269', 'grad_norm': '0.4813', 'learning_rate': '4.574e-05', 'entropy': '1.157', 'num_tokens': '4.517e+07', 'mean_token_accuracy': '0.7196', 'epoch': '0.07418'}
{'loss': '0.9027', 'grad_norm': '0.3485', 'learning_rate': '4.572e-05', 'entropy': '0.8899', 'num_tokens': '4.518e+07', 'mean_token_accuracy': '0.7868', 'epoch': '0.07419'}
{'loss': '0.9786', 'grad_norm': '0.4853', 'learning_rate': '4.57e-05', 'entropy': '1.072', 'num_tokens': '4.518e+07', 'mean_token_accuracy': '0.7337', 'epoch': '0.0742'}
{'loss': '0.6629', 'grad_norm': '0.3683', 'learning_rate': '4.567e-05', 'entropy': '0.7022', 'num_tokens': '4.519e+07', 'mean_token_accuracy': '0.8196', 'epoch': '0.07421'}
{'loss': '0.6243', 'grad_norm': '0.3444', 'learning_rate': '4.565e-05', 'entropy': '0.6399', 'num_tokens': '4.52e+07', 'mean_token_accuracy': '0.831', 'epoch': '0.07421'}
{'loss': '0.6794', 'grad_norm': '0.4435', 'learning_rate': '4.563e-05', 'entropy': '0.763', 'num_tokens': '4.52e+07', 'mean_token_accuracy': '0.7999', 'epoch': '0.07422'}
{'loss': '0.827', 'grad_norm': '0.3838', 'learning_rate': '4.561e-05', 'entropy': '0.8293', 'num_tokens': '4.521e+07', 'mean_token_accuracy': '0.8116', 'epoch': '0.07423'}
{'loss': '0.7043', 'grad_norm': '0.4565', 'learning_rate': '4.558e-05', 'entropy': '0.8009', 'num_tokens': '4.521e+07', 'mean_token_accuracy': '0.7955', 'epoch': '0.07424'}
{'loss': '0.8669', 'grad_norm': '0.4021', 'learning_rate': '4.556e-05', 'entropy': '0.8911', 'num_tokens': '4.522e+07', 'mean_token_accuracy': '0.7748', 'epoch': '0.07425'}
{'loss': '0.926', 'grad_norm': '0.4649', 'learning_rate': '4.554e-05', 'entropy': '0.885', 'num_tokens': '4.522e+07', 'mean_token_accuracy': '0.7705', 'epoch': '0.07426'}
{'loss': '0.6038', 'grad_norm': '0.397', 'learning_rate': '4.551e-05', 'entropy': '0.5753', 'num_tokens': '4.523e+07', 'mean_token_accuracy': '0.8374', 'epoch': '0.07427'}
{'loss': '0.867', 'grad_norm': '0.4398', 'learning_rate': '4.549e-05', 'entropy': '0.7915', 'num_tokens': '4.524e+07', 'mean_token_accuracy': '0.7875', 'epoch': '0.07428'}
{'loss': '0.6044', 'grad_norm': '0.3754', 'learning_rate': '4.547e-05', 'entropy': '0.5861', 'num_tokens': '4.524e+07', 'mean_token_accuracy': '0.8332', 'epoch': '0.07429'}
{'loss': '1.114', 'grad_norm': '0.57', 'learning_rate': '4.544e-05', 'entropy': '1.155', 'num_tokens': '4.525e+07', 'mean_token_accuracy': '0.7181', 'epoch': '0.07429'}
{'loss': '1.056', 'grad_norm': '0.3976', 'learning_rate': '4.542e-05', 'entropy': '0.9774', 'num_tokens': '4.525e+07', 'mean_token_accuracy': '0.7706', 'epoch': '0.0743'}
{'loss': '0.7415', 'grad_norm': '0.3545', 'learning_rate': '4.54e-05', 'entropy': '0.7744', 'num_tokens': '4.525e+07', 'mean_token_accuracy': '0.8016', 'epoch': '0.07431'}
{'loss': '1.04', 'grad_norm': '0.4454', 'learning_rate': '4.537e-05', 'entropy': '0.9626', 'num_tokens': '4.526e+07', 'mean_token_accuracy': '0.7546', 'epoch': '0.07432'}
{'loss': '0.7848', 'grad_norm': '0.3537', 'learning_rate': '4.535e-05', 'entropy': '0.8294', 'num_tokens': '4.527e+07', 'mean_token_accuracy': '0.7996', 'epoch': '0.07433'}
{'loss': '0.7034', 'grad_norm': '0.4566', 'learning_rate': '4.533e-05', 'entropy': '0.7355', 'num_tokens': '4.527e+07', 'mean_token_accuracy': '0.8101', 'epoch': '0.07434'}
{'loss': '0.7506', 'grad_norm': '0.353', 'learning_rate': '4.531e-05', 'entropy': '0.798', 'num_tokens': '4.528e+07', 'mean_token_accuracy': '0.8006', 'epoch': '0.07435'}
{'loss': '0.7947', 'grad_norm': '0.3361', 'learning_rate': '4.528e-05', 'entropy': '0.9125', 'num_tokens': '4.529e+07', 'mean_token_accuracy': '0.7682', 'epoch': '0.07436'}
{'loss': '0.9474', 'grad_norm': '0.417', 'learning_rate': '4.526e-05', 'entropy': '0.8513', 'num_tokens': '4.529e+07', 'mean_token_accuracy': '0.7888', 'epoch': '0.07437'}
{'loss': '0.7758', 'grad_norm': '0.3745', 'learning_rate': '4.524e-05', 'entropy': '0.7285', 'num_tokens': '4.53e+07', 'mean_token_accuracy': '0.8085', 'epoch': '0.07437'}
{'loss': '0.7653', 'grad_norm': '0.4642', 'learning_rate': '4.521e-05', 'entropy': '0.793', 'num_tokens': '4.53e+07', 'mean_token_accuracy': '0.7917', 'epoch': '0.07438'}
{'loss': '1.134', 'grad_norm': '0.4118', 'learning_rate': '4.519e-05', 'entropy': '1.04', 'num_tokens': '4.531e+07', 'mean_token_accuracy': '0.7471', 'epoch': '0.07439'}
{'loss': '0.7854', 'grad_norm': '0.467', 'learning_rate': '4.517e-05', 'entropy': '0.8164', 'num_tokens': '4.531e+07', 'mean_token_accuracy': '0.7955', 'epoch': '0.0744'}
{'loss': '1.206', 'grad_norm': '0.454', 'learning_rate': '4.514e-05', 'entropy': '1.166', 'num_tokens': '4.532e+07', 'mean_token_accuracy': '0.7252', 'epoch': '0.07441'}
{'loss': '1.184', 'grad_norm': '0.4057', 'learning_rate': '4.512e-05', 'entropy': '1.174', 'num_tokens': '4.532e+07', 'mean_token_accuracy': '0.7237', 'epoch': '0.07442'}
{'loss': '0.6735', 'grad_norm': '0.3557', 'learning_rate': '4.51e-05', 'entropy': '0.6482', 'num_tokens': '4.533e+07', 'mean_token_accuracy': '0.8347', 'epoch': '0.07443'}
{'loss': '0.6077', 'grad_norm': '0.3359', 'learning_rate': '4.507e-05', 'entropy': '0.6569', 'num_tokens': '4.533e+07', 'mean_token_accuracy': '0.8294', 'epoch': '0.07444'}
{'loss': '0.9709', 'grad_norm': '0.3478', 'learning_rate': '4.505e-05', 'entropy': '0.9952', 'num_tokens': '4.534e+07', 'mean_token_accuracy': '0.7713', 'epoch': '0.07444'}
{'loss': '0.5916', 'grad_norm': '0.3246', 'learning_rate': '4.503e-05', 'entropy': '0.7036', 'num_tokens': '4.535e+07', 'mean_token_accuracy': '0.8285', 'epoch': '0.07445'}
{'loss': '0.7998', 'grad_norm': '0.485', 'learning_rate': '4.501e-05', 'entropy': '0.8846', 'num_tokens': '4.535e+07', 'mean_token_accuracy': '0.7847', 'epoch': '0.07446'}
{'loss': '0.5507', 'grad_norm': '0.2998', 'learning_rate': '4.498e-05', 'entropy': '0.5539', 'num_tokens': '4.536e+07', 'mean_token_accuracy': '0.8588', 'epoch': '0.07447'}
{'loss': '0.647', 'grad_norm': '0.362', 'learning_rate': '4.496e-05', 'entropy': '0.7243', 'num_tokens': '4.536e+07', 'mean_token_accuracy': '0.8188', 'epoch': '0.07448'}

 70%|███████   | 8417/12000 [4:58:24<2:13:12,  2.23s/it]
 70%|███████   | 8418/12000 [4:58:26<2:11:15,  2.20s/it]
                                                        

 70%|███████   | 8418/12000 [4:58:26<2:11:15,  2.20s/it]
 70%|███████   | 8419/12000 [4:58:28<2:02:25,  2.05s/it]
                                                        

 70%|███████   | 8419/12000 [4:58:28<2:02:25,  2.05s/it]
 70%|███████   | 8420/12000 [4:58:30<1:58:20,  1.98s/it]
                                                        

 70%|███████   | 8420/12000 [4:58:30<1:58:20,  1.98s/it]
 70%|███████   | 8421/12000 [4:58:32<1:56:06,  1.95s/it]
                                                        

 70%|███████   | 8421/12000 [4:58:32<1:56:06,  1.95s/it]
 70%|███████   | 8422/12000 [4:58:34<1:56:29,  1.95s/it]
                                                        

 70%|███████   | 8422/12000 [4:58:34<1:56:29,  1.95s/it]
 70%|███████   | 8423/12000 [4:58:35<1:49:39,  1.84s/it]
                                                        

 70%|███████   | 8423/12000 [4:58:35<1:49:39,  1.84s/it]
 70%|███████   | 8424/12000 [4:58:37<1:55:45,  1.94s/it]
                                                        

 70%|███████   | 8424/12000 [4:58:37<1:55:45,  1.94s/it]
 70%|███████   | 8425/12000 [4:58:39<1:55:49,  1.94s/it]
                                                        

 70%|███████   | 8425/12000 [4:58:39<1:55:49,  1.94s/it]
 70%|███████   | 8426/12000 [4:58:42<2:07:14,  2.14s/it]
                                                        

 70%|███████   | 8426/12000 [4:58:42<2:07:14,  2.14s/it]
 70%|███████   | 8427/12000 [4:58:44<2:04:32,  2.09s/it]
                                                        

 70%|███████   | 8427/12000 [4:58:44<2:04:32,  2.09s/it]
 70%|███████   | 8428/12000 [4:58:45<1:53:02,  1.90s/it]
                                                        

 70%|███████   | 8428/12000 [4:58:45<1:53:02,  1.90s/it]
 70%|███████   | 8429/12000 [4:58:47<1:57:52,  1.98s/it]
                                                        

 70%|███████   | 8429/12000 [4:58:47<1:57:52,  1.98s/it]
 70%|███████   | 8430/12000 [4:58:50<2:05:40,  2.11s/it]
                                                        

 70%|███████   | 8430/12000 [4:58:50<2:05:40,  2.11s/it]
 70%|███████   | 8431/12000 [4:58:52<2:09:25,  2.18s/it]
                                                        

 70%|███████   | 8431/12000 [4:58:52<2:09:25,  2.18s/it]
 70%|███████   | 8432/12000 [4:58:54<1:58:05,  1.99s/it]
                                                        

 70%|███████   | 8432/12000 [4:58:54<1:58:05,  1.99s/it]
 70%|███████   | 8433/12000 [4:58:56<2:04:47,  2.10s/it]
                                                        

 70%|███████   | 8433/12000 [4:58:56<2:04:47,  2.10s/it]
 70%|███████   | 8434/12000 [4:58:58<2:00:57,  2.04s/it]
                                                        

 70%|███████   | 8434/12000 [4:58:58<2:00:57,  2.04s/it]
 70%|███████   | 8435/12000 [4:59:00<1:58:13,  1.99s/it]
                                                        

 70%|███████   | 8435/12000 [4:59:00<1:58:13,  1.99s/it]
 70%|███████   | 8436/12000 [4:59:01<1:48:59,  1.83s/it]
                                                        

 70%|███████   | 8436/12000 [4:59:01<1:48:59,  1.83s/it]
 70%|███████   | 8437/12000 [4:59:05<2:18:11,  2.33s/it]
                                                        

 70%|███████   | 8437/12000 [4:59:05<2:18:11,  2.33s/it]
 70%|███████   | 8438/12000 [4:59:07<2:18:12,  2.33s/it]
                                                        

 70%|███████   | 8438/12000 [4:59:07<2:18:12,  2.33s/it]
 70%|███████   | 8439/12000 [4:59:10<2:24:34,  2.44s/it]
                                                        

 70%|███████   | 8439/12000 [4:59:10<2:24:34,  2.44s/it]
 70%|███████   | 8440/12000 [4:59:11<2:09:48,  2.19s/it]
                                                        

 70%|███████   | 8440/12000 [4:59:11<2:09:48,  2.19s/it]
 70%|███████   | 8441/12000 [4:59:13<1:59:59,  2.02s/it]
                                                        

 70%|███████   | 8441/12000 [4:59:13<1:59:59,  2.02s/it]
 70%|███████   | 8442/12000 [4:59:15<1:55:33,  1.95s/it]
                                                        

 70%|███████   | 8442/12000 [4:59:15<1:55:33,  1.95s/it]
 70%|███████   | 8443/12000 [4:59:17<1:56:29,  1.96s/it]
                                                        

 70%|███████   | 8443/12000 [4:59:17<1:56:29,  1.96s/it]
 70%|███████   | 8444/12000 [4:59:19<1:54:00,  1.92s/it]
                                                        

 70%|███████   | 8444/12000 [4:59:19<1:54:00,  1.92s/it]
 70%|███████   | 8445/12000 [4:59:20<1:48:47,  1.84s/it]
                                                        

 70%|███████   | 8445/12000 [4:59:20<1:48:47,  1.84s/it]
 70%|███████   | 8446/12000 [4:59:22<1:41:35,  1.72s/it]
                                                        

 70%|███████   | 8446/12000 [4:59:22<1:41:35,  1.72s/it]
 70%|███████   | 8447/12000 [4:59:24<1:50:19,  1.86s/it]
                                                        

 70%|███████   | 8447/12000 [4:59:24<1:50:19,  1.86s/it]
 70%|███████   | 8448/12000 [4:59:26<1:45:36,  1.78s/it]
                                                        

 70%|███████   | 8448/12000 [4:59:26<1:45:36,  1.78s/it]
 70%|███████   | 8449/12000 [4:59:28<1:51:56,  1.89s/it]
                                                        

 70%|███████   | 8449/12000 [4:59:28<1:51:56,  1.89s/it]
 70%|███████   | 8450/12000 [4:59:29<1:47:49,  1.82s/it]
                                                        

 70%|███████   | 8450/12000 [4:59:29<1:47:49,  1.82s/it]
 70%|███████   | 8451/12000 [4:59:34<2:32:43,  2.58s/it]
                                                        

 70%|███████   | 8451/12000 [4:59:34<2:32:43,  2.58s/it]
 70%|███████   | 8452/12000 [4:59:35<2:14:56,  2.28s/it]
                                                        

 70%|███████   | 8452/12000 [4:59:35<2:14:56,  2.28s/it]
 70%|███████   | 8453/12000 [4:59:39<2:38:27,  2.68s/it]
                                                        

 70%|███████   | 8453/12000 [4:59:39<2:38:27,  2.68s/it]
 70%|███████   | 8454/12000 [4:59:41<2:34:45,  2.62s/it]
                                                        

 70%|███████   | 8454/12000 [4:59:41<2:34:45,  2.62s/it]
 70%|███████   | 8455/12000 [4:59:43<2:14:02,  2.27s/it]
                                                        

 70%|███████   | 8455/12000 [4:59:43<2:14:02,  2.27s/it]
 70%|███████   | 8456/12000 [4:59:46<2:23:33,  2.43s/it]
                                                        

 70%|███████   | 8456/12000 [4:59:46<2:23:33,  2.43s/it]
 70%|███████   | 8457/12000 [4:59:50<2:49:52,  2.88s/it]
                                                        

 70%|███████   | 8457/12000 [4:59:50<2:49:52,  2.88s/it]
 70%|███████   | 8458/12000 [4:59:53<3:03:15,  3.10s/it]
                                                        

 70%|███████   | 8458/12000 [4:59:53<3:03:15,  3.10s/it]
 70%|███████   | 8459/12000 [4:59:55<2:38:30,  2.69s/it]
                                                        

 70%|███████   | 8459/12000 [4:59:55<2:38:30,  2.69s/it]
 70%|███████   | 8460/12000 [4:59:57<2:22:59,  2.42s/it]
                                                        

 70%|███████   | 8460/12000 [4:59:57<2:22:59,  2.42s/it]
 71%|███████   | 8461/12000 [4:59:58<2:10:43,  2.22s/it]
                                                        

 71%|███████   | 8461/12000 [4:59:58<2:10:43,  2.22s/it]
 71%|███████   | 8462/12000 [5:00:00<1:59:20,  2.02s/it]
                                                        

 71%|███████   | 8462/12000 [5:00:00<1:59:20,  2.02s/it]
 71%|███████   | 8463/12000 [5:00:02<1:52:28,  1.91s/it]
                                                        

 71%|███████   | 8463/12000 [5:00:02<1:52:28,  1.91s/it]
 71%|███████   | 8464/12000 [5:00:04<2:01:32,  2.06s/it]
                                                        
{'loss': '0.6714', 'grad_norm': '0.4338', 'learning_rate': '4.494e-05', 'entropy': '0.6925', 'num_tokens': '4.537e+07', 'mean_token_accuracy': '0.81', 'epoch': '0.07449'}
{'loss': '0.926', 'grad_norm': '0.3526', 'learning_rate': '4.491e-05', 'entropy': '0.9116', 'num_tokens': '4.537e+07', 'mean_token_accuracy': '0.7749', 'epoch': '0.0745'}
{'loss': '0.9667', 'grad_norm': '0.4734', 'learning_rate': '4.489e-05', 'entropy': '1.033', 'num_tokens': '4.538e+07', 'mean_token_accuracy': '0.7469', 'epoch': '0.07451'}
{'loss': '0.8644', 'grad_norm': '0.4569', 'learning_rate': '4.487e-05', 'entropy': '0.8274', 'num_tokens': '4.538e+07', 'mean_token_accuracy': '0.7875', 'epoch': '0.07452'}
{'loss': '0.9888', 'grad_norm': '0.4192', 'learning_rate': '4.484e-05', 'entropy': '0.9154', 'num_tokens': '4.539e+07', 'mean_token_accuracy': '0.7819', 'epoch': '0.07452'}
{'loss': '0.9454', 'grad_norm': '0.5404', 'learning_rate': '4.482e-05', 'entropy': '0.9745', 'num_tokens': '4.539e+07', 'mean_token_accuracy': '0.7582', 'epoch': '0.07453'}
{'loss': '0.7621', 'grad_norm': '0.3635', 'learning_rate': '4.48e-05', 'entropy': '0.8553', 'num_tokens': '4.54e+07', 'mean_token_accuracy': '0.7908', 'epoch': '0.07454'}
{'loss': '0.7091', 'grad_norm': '0.3857', 'learning_rate': '4.478e-05', 'entropy': '0.7582', 'num_tokens': '4.54e+07', 'mean_token_accuracy': '0.8102', 'epoch': '0.07455'}
{'loss': '0.8792', 'grad_norm': '0.4314', 'learning_rate': '4.475e-05', 'entropy': '0.8432', 'num_tokens': '4.541e+07', 'mean_token_accuracy': '0.7912', 'epoch': '0.07456'}
{'loss': '0.773', 'grad_norm': '0.3813', 'learning_rate': '4.473e-05', 'entropy': '0.874', 'num_tokens': '4.541e+07', 'mean_token_accuracy': '0.7804', 'epoch': '0.07457'}
{'loss': '0.9384', 'grad_norm': '0.5405', 'learning_rate': '4.471e-05', 'entropy': '0.9209', 'num_tokens': '4.542e+07', 'mean_token_accuracy': '0.7696', 'epoch': '0.07458'}
{'loss': '0.8844', 'grad_norm': '0.4559', 'learning_rate': '4.468e-05', 'entropy': '0.926', 'num_tokens': '4.542e+07', 'mean_token_accuracy': '0.7809', 'epoch': '0.07459'}
{'loss': '0.6862', 'grad_norm': '0.3423', 'learning_rate': '4.466e-05', 'entropy': '0.7638', 'num_tokens': '4.543e+07', 'mean_token_accuracy': '0.8242', 'epoch': '0.0746'}
{'loss': '0.7812', 'grad_norm': '0.4018', 'learning_rate': '4.464e-05', 'entropy': '0.7601', 'num_tokens': '4.543e+07', 'mean_token_accuracy': '0.801', 'epoch': '0.0746'}
{'loss': '0.6598', 'grad_norm': '0.3366', 'learning_rate': '4.461e-05', 'entropy': '0.6854', 'num_tokens': '4.544e+07', 'mean_token_accuracy': '0.8176', 'epoch': '0.07461'}
{'loss': '0.7968', 'grad_norm': '0.4049', 'learning_rate': '4.459e-05', 'entropy': '0.7504', 'num_tokens': '4.544e+07', 'mean_token_accuracy': '0.8153', 'epoch': '0.07462'}
{'loss': '1.152', 'grad_norm': '0.528', 'learning_rate': '4.457e-05', 'entropy': '1.169', 'num_tokens': '4.545e+07', 'mean_token_accuracy': '0.7222', 'epoch': '0.07463'}
{'loss': '0.658', 'grad_norm': '0.4015', 'learning_rate': '4.455e-05', 'entropy': '0.6706', 'num_tokens': '4.546e+07', 'mean_token_accuracy': '0.8209', 'epoch': '0.07464'}
{'loss': '0.8563', 'grad_norm': '0.3674', 'learning_rate': '4.452e-05', 'entropy': '0.8301', 'num_tokens': '4.546e+07', 'mean_token_accuracy': '0.7839', 'epoch': '0.07465'}
{'loss': '0.5904', 'grad_norm': '0.3134', 'learning_rate': '4.45e-05', 'entropy': '0.6428', 'num_tokens': '4.547e+07', 'mean_token_accuracy': '0.8389', 'epoch': '0.07466'}
{'loss': '0.894', 'grad_norm': '0.45', 'learning_rate': '4.448e-05', 'entropy': '0.9017', 'num_tokens': '4.547e+07', 'mean_token_accuracy': '0.7752', 'epoch': '0.07467'}
{'loss': '0.819', 'grad_norm': '0.4216', 'learning_rate': '4.445e-05', 'entropy': '0.8422', 'num_tokens': '4.548e+07', 'mean_token_accuracy': '0.7906', 'epoch': '0.07467'}
{'loss': '0.9428', 'grad_norm': '0.3842', 'learning_rate': '4.443e-05', 'entropy': '0.9309', 'num_tokens': '4.549e+07', 'mean_token_accuracy': '0.7712', 'epoch': '0.07468'}
{'loss': '0.7739', 'grad_norm': '0.4221', 'learning_rate': '4.441e-05', 'entropy': '0.7762', 'num_tokens': '4.549e+07', 'mean_token_accuracy': '0.8061', 'epoch': '0.07469'}
{'loss': '0.7751', 'grad_norm': '0.4167', 'learning_rate': '4.439e-05', 'entropy': '0.8327', 'num_tokens': '4.55e+07', 'mean_token_accuracy': '0.7975', 'epoch': '0.0747'}
{'loss': '0.8093', 'grad_norm': '0.4744', 'learning_rate': '4.436e-05', 'entropy': '0.8812', 'num_tokens': '4.55e+07', 'mean_token_accuracy': '0.7778', 'epoch': '0.07471'}
{'loss': '0.7989', 'grad_norm': '0.2658', 'learning_rate': '4.434e-05', 'entropy': '0.6867', 'num_tokens': '4.551e+07', 'mean_token_accuracy': '0.8094', 'epoch': '0.07472'}
{'loss': '0.6771', 'grad_norm': '0.392', 'learning_rate': '4.432e-05', 'entropy': '0.6782', 'num_tokens': '4.551e+07', 'mean_token_accuracy': '0.8161', 'epoch': '0.07473'}
{'loss': '0.9485', 'grad_norm': '0.4524', 'learning_rate': '4.429e-05', 'entropy': '0.8688', 'num_tokens': '4.552e+07', 'mean_token_accuracy': '0.7821', 'epoch': '0.07474'}
{'loss': '0.594', 'grad_norm': '0.3157', 'learning_rate': '4.427e-05', 'entropy': '0.5973', 'num_tokens': '4.552e+07', 'mean_token_accuracy': '0.8448', 'epoch': '0.07475'}
{'loss': '0.9774', 'grad_norm': '0.5527', 'learning_rate': '4.425e-05', 'entropy': '0.8338', 'num_tokens': '4.553e+07', 'mean_token_accuracy': '0.7922', 'epoch': '0.07475'}
{'loss': '0.8766', 'grad_norm': '0.4803', 'learning_rate': '4.423e-05', 'entropy': '0.86', 'num_tokens': '4.553e+07', 'mean_token_accuracy': '0.7767', 'epoch': '0.07476'}
{'loss': '0.7389', 'grad_norm': '0.408', 'learning_rate': '4.42e-05', 'entropy': '0.887', 'num_tokens': '4.554e+07', 'mean_token_accuracy': '0.7925', 'epoch': '0.07477'}
{'loss': '0.5867', 'grad_norm': '0.4374', 'learning_rate': '4.418e-05', 'entropy': '0.6042', 'num_tokens': '4.554e+07', 'mean_token_accuracy': '0.8463', 'epoch': '0.07478'}
{'loss': '0.8559', 'grad_norm': '0.3724', 'learning_rate': '4.416e-05', 'entropy': '0.8411', 'num_tokens': '4.555e+07', 'mean_token_accuracy': '0.7899', 'epoch': '0.07479'}
{'loss': '0.715', 'grad_norm': '0.3115', 'learning_rate': '4.413e-05', 'entropy': '0.7217', 'num_tokens': '4.556e+07', 'mean_token_accuracy': '0.8053', 'epoch': '0.0748'}
{'loss': '1.039', 'grad_norm': '0.359', 'learning_rate': '4.411e-05', 'entropy': '1.076', 'num_tokens': '4.556e+07', 'mean_token_accuracy': '0.7579', 'epoch': '0.07481'}
{'loss': '0.6703', 'grad_norm': '0.4755', 'learning_rate': '4.409e-05', 'entropy': '0.7889', 'num_tokens': '4.557e+07', 'mean_token_accuracy': '0.7981', 'epoch': '0.07482'}
{'loss': '0.5676', 'grad_norm': '0.3481', 'learning_rate': '4.407e-05', 'entropy': '0.648', 'num_tokens': '4.557e+07', 'mean_token_accuracy': '0.84', 'epoch': '0.07483'}
{'loss': '0.7337', 'grad_norm': '0.4221', 'learning_rate': '4.404e-05', 'entropy': '0.7256', 'num_tokens': '4.558e+07', 'mean_token_accuracy': '0.8172', 'epoch': '0.07483'}
{'loss': '0.7455', 'grad_norm': '0.3254', 'learning_rate': '4.402e-05', 'entropy': '0.7636', 'num_tokens': '4.559e+07', 'mean_token_accuracy': '0.805', 'epoch': '0.07484'}
{'loss': '0.7006', 'grad_norm': '0.4308', 'learning_rate': '4.4e-05', 'entropy': '0.6605', 'num_tokens': '4.559e+07', 'mean_token_accuracy': '0.8333', 'epoch': '0.07485'}
{'loss': '0.9389', 'grad_norm': '0.4602', 'learning_rate': '4.397e-05', 'entropy': '0.9793', 'num_tokens': '4.56e+07', 'mean_token_accuracy': '0.7494', 'epoch': '0.07486'}
{'loss': '0.9269', 'grad_norm': '0.3504', 'learning_rate': '4.395e-05', 'entropy': '0.8973', 'num_tokens': '4.561e+07', 'mean_token_accuracy': '0.7729', 'epoch': '0.07487'}
{'loss': '0.6256', 'grad_norm': '0.4219', 'learning_rate': '4.393e-05', 'entropy': '0.6421', 'num_tokens': '4.561e+07', 'mean_token_accuracy': '0.8289', 'epoch': '0.07488'}
{'loss': '0.8746', 'grad_norm': '0.4648', 'learning_rate': '4.391e-05', 'entropy': '0.9158', 'num_tokens': '4.562e+07', 'mean_token_accuracy': '0.7522', 'epoch': '0.07489'}
{'loss': '0.6576', 'grad_norm': '0.324', 'learning_rate': '4.388e-05', 'entropy': '0.6842', 'num_tokens': '4.562e+07', 'mean_token_accuracy': '0.825', 'epoch': '0.0749'}

 71%|███████   | 8464/12000 [5:00:04<2:01:32,  2.06s/it]
 71%|███████   | 8465/12000 [5:00:06<1:56:49,  1.98s/it]
                                                        

 71%|███████   | 8465/12000 [5:00:06<1:56:49,  1.98s/it]
 71%|███████   | 8466/12000 [5:00:08<1:56:26,  1.98s/it]
                                                        

 71%|███████   | 8466/12000 [5:00:08<1:56:26,  1.98s/it]
 71%|███████   | 8467/12000 [5:00:11<2:17:26,  2.33s/it]
                                                        

 71%|███████   | 8467/12000 [5:00:11<2:17:26,  2.33s/it]
 71%|███████   | 8468/12000 [5:00:13<2:17:29,  2.34s/it]
                                                        

 71%|███████   | 8468/12000 [5:00:13<2:17:29,  2.34s/it]
 71%|███████   | 8469/12000 [5:00:16<2:28:10,  2.52s/it]
                                                        

 71%|███████   | 8469/12000 [5:00:16<2:28:10,  2.52s/it]
 71%|███████   | 8470/12000 [5:00:18<2:15:28,  2.30s/it]
                                                        

 71%|███████   | 8470/12000 [5:00:18<2:15:28,  2.30s/it]
 71%|███████   | 8471/12000 [5:00:20<1:58:55,  2.02s/it]
                                                        

 71%|███████   | 8471/12000 [5:00:20<1:58:55,  2.02s/it]
 71%|███████   | 8472/12000 [5:00:22<2:04:55,  2.12s/it]
                                                        

 71%|███████   | 8472/12000 [5:00:22<2:04:55,  2.12s/it]
 71%|███████   | 8473/12000 [5:00:24<2:11:17,  2.23s/it]
                                                        

 71%|███████   | 8473/12000 [5:00:24<2:11:17,  2.23s/it]
 71%|███████   | 8474/12000 [5:00:28<2:40:50,  2.74s/it]
                                                        

 71%|███████   | 8474/12000 [5:00:28<2:40:50,  2.74s/it]
 71%|███████   | 8475/12000 [5:00:30<2:19:51,  2.38s/it]
                                                        

 71%|███████   | 8475/12000 [5:00:30<2:19:51,  2.38s/it]
 71%|███████   | 8476/12000 [5:00:31<2:06:41,  2.16s/it]
                                                        

 71%|███████   | 8476/12000 [5:00:31<2:06:41,  2.16s/it]
 71%|███████   | 8477/12000 [5:00:33<1:56:33,  1.99s/it]
                                                        

 71%|███████   | 8477/12000 [5:00:33<1:56:33,  1.99s/it]
 71%|███████   | 8478/12000 [5:00:35<1:50:28,  1.88s/it]
                                                        

 71%|███████   | 8478/12000 [5:00:35<1:50:28,  1.88s/it]
 71%|███████   | 8479/12000 [5:00:37<1:56:18,  1.98s/it]
                                                        

 71%|███████   | 8479/12000 [5:00:37<1:56:18,  1.98s/it]
 71%|███████   | 8480/12000 [5:00:41<2:25:16,  2.48s/it]
                                                        

 71%|███████   | 8480/12000 [5:00:41<2:25:16,  2.48s/it]
 71%|███████   | 8481/12000 [5:00:42<2:08:36,  2.19s/it]
                                                        

 71%|███████   | 8481/12000 [5:00:42<2:08:36,  2.19s/it]
 71%|███████   | 8482/12000 [5:00:44<2:07:36,  2.18s/it]
                                                        

 71%|███████   | 8482/12000 [5:00:44<2:07:36,  2.18s/it]
 71%|███████   | 8483/12000 [5:00:46<2:04:14,  2.12s/it]
                                                        

 71%|███████   | 8483/12000 [5:00:46<2:04:14,  2.12s/it]
 71%|███████   | 8484/12000 [5:00:48<1:55:34,  1.97s/it]
                                                        

 71%|███████   | 8484/12000 [5:00:48<1:55:34,  1.97s/it]
 71%|███████   | 8485/12000 [5:00:49<1:47:24,  1.83s/it]
                                                        

 71%|███████   | 8485/12000 [5:00:49<1:47:24,  1.83s/it]
 71%|███████   | 8486/12000 [5:00:51<1:48:56,  1.86s/it]
                                                        

 71%|███████   | 8486/12000 [5:00:51<1:48:56,  1.86s/it]
 71%|███████   | 8487/12000 [5:00:53<1:46:33,  1.82s/it]
                                                        

 71%|███████   | 8487/12000 [5:00:53<1:46:33,  1.82s/it]
 71%|███████   | 8488/12000 [5:00:55<1:44:31,  1.79s/it]
                                                        

 71%|███████   | 8488/12000 [5:00:55<1:44:31,  1.79s/it]
 71%|███████   | 8489/12000 [5:00:56<1:40:03,  1.71s/it]
                                                        

 71%|███████   | 8489/12000 [5:00:56<1:40:03,  1.71s/it]
 71%|███████   | 8490/12000 [5:00:59<1:55:21,  1.97s/it]
                                                        

 71%|███████   | 8490/12000 [5:00:59<1:55:21,  1.97s/it]
 71%|███████   | 8491/12000 [5:01:01<1:55:12,  1.97s/it]
                                                        

 71%|███████   | 8491/12000 [5:01:01<1:55:12,  1.97s/it]
 71%|███████   | 8492/12000 [5:01:03<1:51:46,  1.91s/it]
                                                        

 71%|███████   | 8492/12000 [5:01:03<1:51:46,  1.91s/it]
 71%|███████   | 8493/12000 [5:01:05<2:02:19,  2.09s/it]
                                                        

 71%|███████   | 8493/12000 [5:01:05<2:02:19,  2.09s/it]
 71%|███████   | 8494/12000 [5:01:06<1:51:09,  1.90s/it]
                                                        

 71%|███████   | 8494/12000 [5:01:07<1:51:09,  1.90s/it]
 71%|███████   | 8495/12000 [5:01:08<1:49:40,  1.88s/it]
                                                        

 71%|███████   | 8495/12000 [5:01:08<1:49:40,  1.88s/it]
 71%|███████   | 8496/12000 [5:01:10<1:44:56,  1.80s/it]
                                                        

 71%|███████   | 8496/12000 [5:01:10<1:44:56,  1.80s/it]
 71%|███████   | 8497/12000 [5:01:12<1:51:11,  1.90s/it]
                                                        

 71%|███████   | 8497/12000 [5:01:12<1:51:11,  1.90s/it]
 71%|███████   | 8498/12000 [5:01:14<1:43:04,  1.77s/it]
                                                        

 71%|███████   | 8498/12000 [5:01:14<1:43:04,  1.77s/it]
 71%|███████   | 8499/12000 [5:01:15<1:39:01,  1.70s/it]
                                                        

 71%|███████   | 8499/12000 [5:01:15<1:39:01,  1.70s/it]
 71%|███████   | 8500/12000 [5:01:17<1:38:14,  1.68s/it]
                                                        

 71%|███████   | 8500/12000 [5:01:17<1:38:14,  1.68s/it]
 71%|███████   | 8501/12000 [5:01:20<2:09:19,  2.22s/it]
                                                        

 71%|███████   | 8501/12000 [5:01:20<2:09:19,  2.22s/it]
 71%|███████   | 8502/12000 [5:01:24<2:31:07,  2.59s/it]
                                                        

 71%|███████   | 8502/12000 [5:01:24<2:31:07,  2.59s/it]
 71%|███████   | 8503/12000 [5:01:25<2:10:10,  2.23s/it]
                                                        

 71%|███████   | 8503/12000 [5:01:25<2:10:10,  2.23s/it]
 71%|███████   | 8504/12000 [5:01:28<2:21:08,  2.42s/it]
                                                        

 71%|███████   | 8504/12000 [5:01:28<2:21:08,  2.42s/it]
 71%|███████   | 8505/12000 [5:01:30<2:21:31,  2.43s/it]
                                                        

 71%|███████   | 8505/12000 [5:01:30<2:21:31,  2.43s/it]
 71%|███████   | 8506/12000 [5:01:32<2:07:18,  2.19s/it]
                                                        

 71%|███████   | 8506/12000 [5:01:32<2:07:18,  2.19s/it]
 71%|███████   | 8507/12000 [5:01:34<1:55:59,  1.99s/it]
                                                        

 71%|███████   | 8507/12000 [5:01:34<1:55:59,  1.99s/it]
 71%|███████   | 8508/12000 [5:01:35<1:52:16,  1.93s/it]
                                                        

 71%|███████   | 8508/12000 [5:01:35<1:52:16,  1.93s/it]
 71%|███████   | 8509/12000 [5:01:37<1:47:12,  1.84s/it]
                                                        

 71%|███████   | 8509/12000 [5:01:37<1:47:12,  1.84s/it]
 71%|███████   | 8510/12000 [5:01:39<1:42:56,  1.77s/it]
                                                        

 71%|███████   | 8510/12000 [5:01:39<1:42:56,  1.77s/it]
 71%|███████   | 8511/12000 [5:01:40<1:39:36,  1.71s/it]
                                                        
{'loss': '0.9473', 'grad_norm': '0.4637', 'learning_rate': '4.386e-05', 'entropy': '0.8284', 'num_tokens': '4.563e+07', 'mean_token_accuracy': '0.7975', 'epoch': '0.0749'}
{'loss': '0.9008', 'grad_norm': '0.452', 'learning_rate': '4.384e-05', 'entropy': '0.8928', 'num_tokens': '4.563e+07', 'mean_token_accuracy': '0.7799', 'epoch': '0.07491'}
{'loss': '1.404', 'grad_norm': '0.5156', 'learning_rate': '4.381e-05', 'entropy': '1.292', 'num_tokens': '4.564e+07', 'mean_token_accuracy': '0.6913', 'epoch': '0.07492'}
{'loss': '0.896', 'grad_norm': '0.3471', 'learning_rate': '4.379e-05', 'entropy': '0.9612', 'num_tokens': '4.565e+07', 'mean_token_accuracy': '0.7612', 'epoch': '0.07493'}
{'loss': '0.6838', 'grad_norm': '0.3844', 'learning_rate': '4.377e-05', 'entropy': '0.6657', 'num_tokens': '4.565e+07', 'mean_token_accuracy': '0.8124', 'epoch': '0.07494'}
{'loss': '0.8896', 'grad_norm': '0.4126', 'learning_rate': '4.375e-05', 'entropy': '0.9679', 'num_tokens': '4.566e+07', 'mean_token_accuracy': '0.752', 'epoch': '0.07495'}
{'loss': '0.9344', 'grad_norm': '0.3983', 'learning_rate': '4.372e-05', 'entropy': '0.8811', 'num_tokens': '4.566e+07', 'mean_token_accuracy': '0.7726', 'epoch': '0.07496'}
{'loss': '0.7044', 'grad_norm': '0.4449', 'learning_rate': '4.37e-05', 'entropy': '0.7409', 'num_tokens': '4.567e+07', 'mean_token_accuracy': '0.8043', 'epoch': '0.07497'}
{'loss': '0.8025', 'grad_norm': '0.3832', 'learning_rate': '4.368e-05', 'entropy': '0.9018', 'num_tokens': '4.567e+07', 'mean_token_accuracy': '0.7802', 'epoch': '0.07498'}
{'loss': '0.6402', 'grad_norm': '0.3337', 'learning_rate': '4.365e-05', 'entropy': '0.6941', 'num_tokens': '4.568e+07', 'mean_token_accuracy': '0.8239', 'epoch': '0.07498'}
{'loss': '1.184', 'grad_norm': '0.3883', 'learning_rate': '4.363e-05', 'entropy': '1.048', 'num_tokens': '4.569e+07', 'mean_token_accuracy': '0.7541', 'epoch': '0.07499'}
{'loss': '1.074', 'grad_norm': '0.4642', 'learning_rate': '4.361e-05', 'entropy': '1.059', 'num_tokens': '4.569e+07', 'mean_token_accuracy': '0.7441', 'epoch': '0.075'}
{'loss': '0.7924', 'grad_norm': '0.4927', 'learning_rate': '4.359e-05', 'entropy': '0.8149', 'num_tokens': '4.57e+07', 'mean_token_accuracy': '0.7855', 'epoch': '0.07501'}
{'loss': '0.7339', 'grad_norm': '0.3582', 'learning_rate': '4.356e-05', 'entropy': '0.7154', 'num_tokens': '4.57e+07', 'mean_token_accuracy': '0.8223', 'epoch': '0.07502'}
{'loss': '0.708', 'grad_norm': '0.3572', 'learning_rate': '4.354e-05', 'entropy': '0.7796', 'num_tokens': '4.571e+07', 'mean_token_accuracy': '0.7939', 'epoch': '0.07503'}
{'loss': '0.9688', 'grad_norm': '0.32', 'learning_rate': '4.352e-05', 'entropy': '0.8808', 'num_tokens': '4.571e+07', 'mean_token_accuracy': '0.7875', 'epoch': '0.07504'}
{'loss': '0.8212', 'grad_norm': '0.4286', 'learning_rate': '4.35e-05', 'entropy': '0.8174', 'num_tokens': '4.572e+07', 'mean_token_accuracy': '0.7956', 'epoch': '0.07505'}
{'loss': '1.055', 'grad_norm': '0.4641', 'learning_rate': '4.347e-05', 'entropy': '1.126', 'num_tokens': '4.572e+07', 'mean_token_accuracy': '0.7232', 'epoch': '0.07506'}
{'loss': '0.6342', 'grad_norm': '0.4047', 'learning_rate': '4.345e-05', 'entropy': '0.7079', 'num_tokens': '4.573e+07', 'mean_token_accuracy': '0.8203', 'epoch': '0.07506'}
{'loss': '0.9167', 'grad_norm': '0.4418', 'learning_rate': '4.343e-05', 'entropy': '0.8678', 'num_tokens': '4.573e+07', 'mean_token_accuracy': '0.7856', 'epoch': '0.07507'}
{'loss': '0.8811', 'grad_norm': '0.3725', 'learning_rate': '4.34e-05', 'entropy': '0.8745', 'num_tokens': '4.574e+07', 'mean_token_accuracy': '0.7936', 'epoch': '0.07508'}
{'loss': '0.7194', 'grad_norm': '0.3732', 'learning_rate': '4.338e-05', 'entropy': '0.7961', 'num_tokens': '4.574e+07', 'mean_token_accuracy': '0.791', 'epoch': '0.07509'}
{'loss': '1.096', 'grad_norm': '0.46', 'learning_rate': '4.336e-05', 'entropy': '1.036', 'num_tokens': '4.575e+07', 'mean_token_accuracy': '0.745', 'epoch': '0.0751'}
{'loss': '0.8596', 'grad_norm': '0.4625', 'learning_rate': '4.334e-05', 'entropy': '0.8873', 'num_tokens': '4.575e+07', 'mean_token_accuracy': '0.7606', 'epoch': '0.07511'}
{'loss': '0.6538', 'grad_norm': '0.3784', 'learning_rate': '4.331e-05', 'entropy': '0.7365', 'num_tokens': '4.576e+07', 'mean_token_accuracy': '0.7991', 'epoch': '0.07512'}
{'loss': '0.6682', 'grad_norm': '0.2941', 'learning_rate': '4.329e-05', 'entropy': '0.71', 'num_tokens': '4.576e+07', 'mean_token_accuracy': '0.8106', 'epoch': '0.07513'}
{'loss': '0.881', 'grad_norm': '0.3827', 'learning_rate': '4.327e-05', 'entropy': '0.9339', 'num_tokens': '4.577e+07', 'mean_token_accuracy': '0.7662', 'epoch': '0.07514'}
{'loss': '1.002', 'grad_norm': '0.4436', 'learning_rate': '4.325e-05', 'entropy': '0.9662', 'num_tokens': '4.577e+07', 'mean_token_accuracy': '0.7642', 'epoch': '0.07514'}
{'loss': '1.025', 'grad_norm': '0.4548', 'learning_rate': '4.322e-05', 'entropy': '0.9422', 'num_tokens': '4.578e+07', 'mean_token_accuracy': '0.7738', 'epoch': '0.07515'}
{'loss': '0.6829', 'grad_norm': '0.3556', 'learning_rate': '4.32e-05', 'entropy': '0.6558', 'num_tokens': '4.579e+07', 'mean_token_accuracy': '0.8286', 'epoch': '0.07516'}
{'loss': '0.6557', 'grad_norm': '0.3577', 'learning_rate': '4.318e-05', 'entropy': '0.6435', 'num_tokens': '4.579e+07', 'mean_token_accuracy': '0.8254', 'epoch': '0.07517'}
{'loss': '1.113', 'grad_norm': '0.5265', 'learning_rate': '4.316e-05', 'entropy': '1.029', 'num_tokens': '4.58e+07', 'mean_token_accuracy': '0.7534', 'epoch': '0.07518'}
{'loss': '0.6723', 'grad_norm': '0.5007', 'learning_rate': '4.313e-05', 'entropy': '0.6095', 'num_tokens': '4.58e+07', 'mean_token_accuracy': '0.8348', 'epoch': '0.07519'}
{'loss': '0.989', 'grad_norm': '0.4535', 'learning_rate': '4.311e-05', 'entropy': '0.9515', 'num_tokens': '4.581e+07', 'mean_token_accuracy': '0.7683', 'epoch': '0.0752'}
{'loss': '0.6103', 'grad_norm': '0.3439', 'learning_rate': '4.309e-05', 'entropy': '0.6442', 'num_tokens': '4.581e+07', 'mean_token_accuracy': '0.8357', 'epoch': '0.07521'}
{'loss': '1.413', 'grad_norm': '0.5155', 'learning_rate': '4.306e-05', 'entropy': '1.295', 'num_tokens': '4.581e+07', 'mean_token_accuracy': '0.691', 'epoch': '0.07521'}
{'loss': '0.8259', 'grad_norm': '0.4134', 'learning_rate': '4.304e-05', 'entropy': '0.7861', 'num_tokens': '4.582e+07', 'mean_token_accuracy': '0.8089', 'epoch': '0.07522'}
{'loss': '1.635', 'grad_norm': '0.4336', 'learning_rate': '4.302e-05', 'entropy': '1.371', 'num_tokens': '4.582e+07', 'mean_token_accuracy': '0.7009', 'epoch': '0.07523'}
{'loss': '0.9882', 'grad_norm': '0.5398', 'learning_rate': '4.3e-05', 'entropy': '1.064', 'num_tokens': '4.583e+07', 'mean_token_accuracy': '0.7466', 'epoch': '0.07524'}
{'loss': '0.6853', 'grad_norm': '0.3317', 'learning_rate': '4.297e-05', 'entropy': '0.7402', 'num_tokens': '4.583e+07', 'mean_token_accuracy': '0.8104', 'epoch': '0.07525'}
{'loss': '1.012', 'grad_norm': '0.5186', 'learning_rate': '4.295e-05', 'entropy': '0.8884', 'num_tokens': '4.584e+07', 'mean_token_accuracy': '0.784', 'epoch': '0.07526'}
{'loss': '0.7704', 'grad_norm': '0.3564', 'learning_rate': '4.293e-05', 'entropy': '0.7758', 'num_tokens': '4.585e+07', 'mean_token_accuracy': '0.7918', 'epoch': '0.07527'}
{'loss': '0.6317', 'grad_norm': '0.3723', 'learning_rate': '4.291e-05', 'entropy': '0.5976', 'num_tokens': '4.585e+07', 'mean_token_accuracy': '0.8393', 'epoch': '0.07528'}
{'loss': '0.8875', 'grad_norm': '0.4127', 'learning_rate': '4.288e-05', 'entropy': '0.8812', 'num_tokens': '4.585e+07', 'mean_token_accuracy': '0.7937', 'epoch': '0.07529'}
{'loss': '0.8246', 'grad_norm': '0.464', 'learning_rate': '4.286e-05', 'entropy': '0.8264', 'num_tokens': '4.586e+07', 'mean_token_accuracy': '0.7863', 'epoch': '0.07529'}
{'loss': '1.063', 'grad_norm': '0.4465', 'learning_rate': '4.284e-05', 'entropy': '1.037', 'num_tokens': '4.586e+07', 'mean_token_accuracy': '0.7388', 'epoch': '0.0753'}
{'loss': '0.6652', 'grad_norm': '0.4224', 'learning_rate': '4.282e-05', 'entropy': '0.7086', 'num_tokens': '4.587e+07', 'mean_token_accuracy': '0.8099', 'epoch': '0.07531'}

 71%|███████   | 8511/12000 [5:01:40<1:39:36,  1.71s/it]
 71%|███████   | 8512/12000 [5:01:42<1:42:25,  1.76s/it]
                                                        

 71%|███████   | 8512/12000 [5:01:42<1:42:25,  1.76s/it]
 71%|███████   | 8513/12000 [5:01:44<1:38:27,  1.69s/it]
                                                        

 71%|███████   | 8513/12000 [5:01:44<1:38:27,  1.69s/it]
 71%|███████   | 8514/12000 [5:01:46<1:48:57,  1.88s/it]
                                                        

 71%|███████   | 8514/12000 [5:01:46<1:48:57,  1.88s/it]
 71%|███████   | 8515/12000 [5:01:47<1:43:50,  1.79s/it]
                                                        

 71%|███████   | 8515/12000 [5:01:47<1:43:50,  1.79s/it]
 71%|███████   | 8516/12000 [5:01:51<2:12:49,  2.29s/it]
                                                        

 71%|███████   | 8516/12000 [5:01:51<2:12:49,  2.29s/it]
 71%|███████   | 8517/12000 [5:01:53<2:03:48,  2.13s/it]
                                                        

 71%|███████   | 8517/12000 [5:01:53<2:03:48,  2.13s/it]
 71%|███████   | 8518/12000 [5:01:54<1:52:11,  1.93s/it]
                                                        

 71%|███████   | 8518/12000 [5:01:54<1:52:11,  1.93s/it]
 71%|███████   | 8519/12000 [5:01:56<1:49:17,  1.88s/it]
                                                        

 71%|███████   | 8519/12000 [5:01:56<1:49:17,  1.88s/it]
 71%|███████   | 8520/12000 [5:01:58<1:58:51,  2.05s/it]
                                                        

 71%|███████   | 8520/12000 [5:01:58<1:58:51,  2.05s/it]
 71%|███████   | 8521/12000 [5:02:00<1:49:52,  1.89s/it]
                                                        

 71%|███████   | 8521/12000 [5:02:00<1:49:52,  1.89s/it]
 71%|███████   | 8522/12000 [5:02:01<1:44:12,  1.80s/it]
                                                        

 71%|███████   | 8522/12000 [5:02:01<1:44:12,  1.80s/it]
 71%|███████   | 8523/12000 [5:02:03<1:38:30,  1.70s/it]
                                                        

 71%|███████   | 8523/12000 [5:02:03<1:38:30,  1.70s/it]
 71%|███████   | 8524/12000 [5:02:07<2:21:05,  2.44s/it]
                                                        

 71%|███████   | 8524/12000 [5:02:07<2:21:05,  2.44s/it]
 71%|███████   | 8525/12000 [5:02:08<2:03:23,  2.13s/it]
                                                        

 71%|███████   | 8525/12000 [5:02:08<2:03:23,  2.13s/it]
 71%|███████   | 8526/12000 [5:02:10<1:52:41,  1.95s/it]
                                                        

 71%|███████   | 8526/12000 [5:02:10<1:52:41,  1.95s/it]
 71%|███████   | 8527/12000 [5:02:12<1:51:15,  1.92s/it]
                                                        

 71%|███████   | 8527/12000 [5:02:12<1:51:15,  1.92s/it]
 71%|███████   | 8528/12000 [5:02:13<1:46:45,  1.84s/it]
                                                        

 71%|███████   | 8528/12000 [5:02:13<1:46:45,  1.84s/it]
 71%|███████   | 8529/12000 [5:02:16<1:52:59,  1.95s/it]
                                                        

 71%|███████   | 8529/12000 [5:02:16<1:52:59,  1.95s/it]
 71%|███████   | 8530/12000 [5:02:19<2:09:21,  2.24s/it]
                                                        

 71%|███████   | 8530/12000 [5:02:19<2:09:21,  2.24s/it]
 71%|███████   | 8531/12000 [5:02:25<3:13:47,  3.35s/it]
                                                        

 71%|███████   | 8531/12000 [5:02:25<3:13:47,  3.35s/it]
 71%|███████   | 8532/12000 [5:02:26<2:44:25,  2.84s/it]
                                                        

 71%|███████   | 8532/12000 [5:02:26<2:44:25,  2.84s/it]
 71%|███████   | 8533/12000 [5:02:28<2:22:10,  2.46s/it]
                                                        

 71%|███████   | 8533/12000 [5:02:28<2:22:10,  2.46s/it]
 71%|███████   | 8534/12000 [5:02:29<2:06:49,  2.20s/it]
                                                        

 71%|███████   | 8534/12000 [5:02:29<2:06:49,  2.20s/it]
 71%|███████   | 8535/12000 [5:02:31<2:04:10,  2.15s/it]
                                                        

 71%|███████   | 8535/12000 [5:02:31<2:04:10,  2.15s/it]
 71%|███████   | 8536/12000 [5:02:33<1:50:39,  1.92s/it]
                                                        

 71%|███████   | 8536/12000 [5:02:33<1:50:39,  1.92s/it]
 71%|███████   | 8537/12000 [5:02:34<1:46:06,  1.84s/it]
                                                        

 71%|███████   | 8537/12000 [5:02:34<1:46:06,  1.84s/it]
 71%|███████   | 8538/12000 [5:02:36<1:43:26,  1.79s/it]
                                                        

 71%|███████   | 8538/12000 [5:02:36<1:43:26,  1.79s/it]
 71%|███████   | 8539/12000 [5:02:38<1:37:05,  1.68s/it]
                                                        

 71%|███████   | 8539/12000 [5:02:38<1:37:05,  1.68s/it]
 71%|███████   | 8540/12000 [5:02:39<1:31:50,  1.59s/it]
                                                        

 71%|███████   | 8540/12000 [5:02:39<1:31:50,  1.59s/it]
 71%|███████   | 8541/12000 [5:02:43<2:13:47,  2.32s/it]
                                                        

 71%|███████   | 8541/12000 [5:02:43<2:13:47,  2.32s/it]
 71%|███████   | 8542/12000 [5:02:45<2:04:46,  2.16s/it]
                                                        

 71%|███████   | 8542/12000 [5:02:45<2:04:46,  2.16s/it]
 71%|███████   | 8543/12000 [5:02:47<1:57:57,  2.05s/it]
                                                        

 71%|███████   | 8543/12000 [5:02:47<1:57:57,  2.05s/it]
 71%|███████   | 8544/12000 [5:02:48<1:54:03,  1.98s/it]
                                                        

 71%|███████   | 8544/12000 [5:02:48<1:54:03,  1.98s/it]
 71%|███████   | 8545/12000 [5:02:50<1:47:34,  1.87s/it]
                                                        

 71%|███████   | 8545/12000 [5:02:50<1:47:34,  1.87s/it]
 71%|███████   | 8546/12000 [5:02:53<2:16:15,  2.37s/it]
                                                        

 71%|███████   | 8546/12000 [5:02:53<2:16:15,  2.37s/it]
 71%|███████   | 8547/12000 [5:02:55<2:00:29,  2.09s/it]
                                                        

 71%|███████   | 8547/12000 [5:02:55<2:00:29,  2.09s/it]
 71%|███████   | 8548/12000 [5:02:57<1:51:31,  1.94s/it]
                                                        

 71%|███████   | 8548/12000 [5:02:57<1:51:31,  1.94s/it]
 71%|███████   | 8549/12000 [5:02:58<1:45:42,  1.84s/it]
                                                        

 71%|███████   | 8549/12000 [5:02:58<1:45:42,  1.84s/it]
 71%|███████▏  | 8550/12000 [5:03:00<1:49:32,  1.91s/it]
                                                        

 71%|███████▏  | 8550/12000 [5:03:00<1:49:32,  1.91s/it]
 71%|███████▏  | 8551/12000 [5:03:03<2:05:28,  2.18s/it]
                                                        

 71%|███████▏  | 8551/12000 [5:03:03<2:05:28,  2.18s/it]
 71%|███████▏  | 8552/12000 [5:03:05<1:55:45,  2.01s/it]
                                                        

 71%|███████▏  | 8552/12000 [5:03:05<1:55:45,  2.01s/it]
 71%|███████▏  | 8553/12000 [5:03:07<1:56:09,  2.02s/it]
                                                        

 71%|███████▏  | 8553/12000 [5:03:07<1:56:09,  2.02s/it]
 71%|███████▏  | 8554/12000 [5:03:08<1:46:50,  1.86s/it]
                                                        

 71%|███████▏  | 8554/12000 [5:03:08<1:46:50,  1.86s/it]
 71%|███████▏  | 8555/12000 [5:03:10<1:47:53,  1.88s/it]
                                                        

 71%|███████▏  | 8555/12000 [5:03:10<1:47:53,  1.88s/it]
 71%|███████▏  | 8556/12000 [5:03:13<2:04:53,  2.18s/it]
                                                        

 71%|███████▏  | 8556/12000 [5:03:13<2:04:53,  2.18s/it]
 71%|███████▏  | 8557/12000 [5:03:15<1:54:26,  1.99s/it]
                                                        

 71%|███████▏  | 8557/12000 [5:03:15<1:54:26,  1.99s/it]
 71%|███████▏  | 8558/12000 [5:03:16<1:44:01,  1.81s/it]
                                                        
{'loss': '0.9057', 'grad_norm': '0.4034', 'learning_rate': '4.279e-05', 'entropy': '1.001', 'num_tokens': '4.587e+07', 'mean_token_accuracy': '0.7541', 'epoch': '0.07532'}
{'loss': '1.062', 'grad_norm': '0.5753', 'learning_rate': '4.277e-05', 'entropy': '1.008', 'num_tokens': '4.588e+07', 'mean_token_accuracy': '0.7591', 'epoch': '0.07533'}
{'loss': '0.7355', 'grad_norm': '0.3984', 'learning_rate': '4.275e-05', 'entropy': '0.7398', 'num_tokens': '4.588e+07', 'mean_token_accuracy': '0.8165', 'epoch': '0.07534'}
{'loss': '0.8509', 'grad_norm': '0.4399', 'learning_rate': '4.273e-05', 'entropy': '0.948', 'num_tokens': '4.589e+07', 'mean_token_accuracy': '0.771', 'epoch': '0.07535'}
{'loss': '1.161', 'grad_norm': '0.5204', 'learning_rate': '4.27e-05', 'entropy': '1.172', 'num_tokens': '4.589e+07', 'mean_token_accuracy': '0.7165', 'epoch': '0.07536'}
{'loss': '0.4496', 'grad_norm': '0.2169', 'learning_rate': '4.268e-05', 'entropy': '0.6229', 'num_tokens': '4.59e+07', 'mean_token_accuracy': '0.8439', 'epoch': '0.07537'}
{'loss': '0.7333', 'grad_norm': '0.4024', 'learning_rate': '4.266e-05', 'entropy': '0.711', 'num_tokens': '4.591e+07', 'mean_token_accuracy': '0.8232', 'epoch': '0.07537'}
{'loss': '0.6565', 'grad_norm': '0.2821', 'learning_rate': '4.263e-05', 'entropy': '0.723', 'num_tokens': '4.591e+07', 'mean_token_accuracy': '0.8181', 'epoch': '0.07538'}
{'loss': '0.5226', 'grad_norm': '0.4002', 'learning_rate': '4.261e-05', 'entropy': '0.5312', 'num_tokens': '4.592e+07', 'mean_token_accuracy': '0.8563', 'epoch': '0.07539'}
{'loss': '0.7656', 'grad_norm': '0.3889', 'learning_rate': '4.259e-05', 'entropy': '0.7242', 'num_tokens': '4.592e+07', 'mean_token_accuracy': '0.8067', 'epoch': '0.0754'}
{'loss': '0.7914', 'grad_norm': '0.4693', 'learning_rate': '4.257e-05', 'entropy': '0.8305', 'num_tokens': '4.593e+07', 'mean_token_accuracy': '0.7934', 'epoch': '0.07541'}
{'loss': '0.8758', 'grad_norm': '0.4523', 'learning_rate': '4.254e-05', 'entropy': '0.9123', 'num_tokens': '4.593e+07', 'mean_token_accuracy': '0.7738', 'epoch': '0.07542'}
{'loss': '0.7027', 'grad_norm': '0.3206', 'learning_rate': '4.252e-05', 'entropy': '0.68', 'num_tokens': '4.594e+07', 'mean_token_accuracy': '0.8196', 'epoch': '0.07543'}
{'loss': '1.069', 'grad_norm': '0.487', 'learning_rate': '4.25e-05', 'entropy': '0.9759', 'num_tokens': '4.594e+07', 'mean_token_accuracy': '0.7649', 'epoch': '0.07544'}
{'loss': '0.8765', 'grad_norm': '0.4975', 'learning_rate': '4.248e-05', 'entropy': '0.8512', 'num_tokens': '4.595e+07', 'mean_token_accuracy': '0.7912', 'epoch': '0.07544'}
{'loss': '0.8127', 'grad_norm': '0.4867', 'learning_rate': '4.245e-05', 'entropy': '0.8464', 'num_tokens': '4.595e+07', 'mean_token_accuracy': '0.7761', 'epoch': '0.07545'}
{'loss': '1.038', 'grad_norm': '0.4201', 'learning_rate': '4.243e-05', 'entropy': '0.9624', 'num_tokens': '4.595e+07', 'mean_token_accuracy': '0.7562', 'epoch': '0.07546'}
{'loss': '0.7702', 'grad_norm': '0.455', 'learning_rate': '4.241e-05', 'entropy': '0.7961', 'num_tokens': '4.596e+07', 'mean_token_accuracy': '0.7981', 'epoch': '0.07547'}
{'loss': '0.7168', 'grad_norm': '0.3667', 'learning_rate': '4.239e-05', 'entropy': '0.7574', 'num_tokens': '4.597e+07', 'mean_token_accuracy': '0.8195', 'epoch': '0.07548'}
{'loss': '0.6108', 'grad_norm': '0.354', 'learning_rate': '4.236e-05', 'entropy': '0.6987', 'num_tokens': '4.597e+07', 'mean_token_accuracy': '0.8225', 'epoch': '0.07549'}
{'loss': '1.052', 'grad_norm': '0.3636', 'learning_rate': '4.234e-05', 'entropy': '0.8013', 'num_tokens': '4.599e+07', 'mean_token_accuracy': '0.8108', 'epoch': '0.0755'}
{'loss': '0.5419', 'grad_norm': '0.3446', 'learning_rate': '4.232e-05', 'entropy': '0.6496', 'num_tokens': '4.599e+07', 'mean_token_accuracy': '0.8254', 'epoch': '0.07551'}
{'loss': '0.6512', 'grad_norm': '0.3108', 'learning_rate': '4.23e-05', 'entropy': '0.6462', 'num_tokens': '4.6e+07', 'mean_token_accuracy': '0.8365', 'epoch': '0.07552'}
{'loss': '1.094', 'grad_norm': '0.401', 'learning_rate': '4.227e-05', 'entropy': '1.048', 'num_tokens': '4.6e+07', 'mean_token_accuracy': '0.7423', 'epoch': '0.07552'}
{'loss': '1.227', 'grad_norm': '0.4628', 'learning_rate': '4.225e-05', 'entropy': '0.9708', 'num_tokens': '4.601e+07', 'mean_token_accuracy': '0.7852', 'epoch': '0.07553'}
{'loss': '0.8467', 'grad_norm': '0.4934', 'learning_rate': '4.223e-05', 'entropy': '0.8923', 'num_tokens': '4.601e+07', 'mean_token_accuracy': '0.787', 'epoch': '0.07554'}
{'loss': '0.7372', 'grad_norm': '0.4454', 'learning_rate': '4.221e-05', 'entropy': '0.7421', 'num_tokens': '4.601e+07', 'mean_token_accuracy': '0.8012', 'epoch': '0.07555'}
{'loss': '0.7386', 'grad_norm': '0.3526', 'learning_rate': '4.218e-05', 'entropy': '0.7595', 'num_tokens': '4.602e+07', 'mean_token_accuracy': '0.7926', 'epoch': '0.07556'}
{'loss': '0.8849', 'grad_norm': '0.46', 'learning_rate': '4.216e-05', 'entropy': '0.9155', 'num_tokens': '4.602e+07', 'mean_token_accuracy': '0.7825', 'epoch': '0.07557'}
{'loss': '0.8237', 'grad_norm': '0.4592', 'learning_rate': '4.214e-05', 'entropy': '0.8342', 'num_tokens': '4.603e+07', 'mean_token_accuracy': '0.7779', 'epoch': '0.07558'}
{'loss': '0.9412', 'grad_norm': '0.3145', 'learning_rate': '4.212e-05', 'entropy': '0.8898', 'num_tokens': '4.603e+07', 'mean_token_accuracy': '0.7626', 'epoch': '0.07559'}
{'loss': '0.7824', 'grad_norm': '0.3815', 'learning_rate': '4.209e-05', 'entropy': '0.7594', 'num_tokens': '4.604e+07', 'mean_token_accuracy': '0.8032', 'epoch': '0.0756'}
{'loss': '0.6094', 'grad_norm': '0.316', 'learning_rate': '4.207e-05', 'entropy': '0.5787', 'num_tokens': '4.605e+07', 'mean_token_accuracy': '0.8507', 'epoch': '0.0756'}
{'loss': '0.6468', 'grad_norm': '0.3532', 'learning_rate': '4.205e-05', 'entropy': '0.5788', 'num_tokens': '4.605e+07', 'mean_token_accuracy': '0.8485', 'epoch': '0.07561'}
{'loss': '0.7329', 'grad_norm': '0.4065', 'learning_rate': '4.203e-05', 'entropy': '0.7491', 'num_tokens': '4.606e+07', 'mean_token_accuracy': '0.7998', 'epoch': '0.07562'}
{'loss': '0.8439', 'grad_norm': '0.4003', 'learning_rate': '4.2e-05', 'entropy': '0.8545', 'num_tokens': '4.606e+07', 'mean_token_accuracy': '0.7857', 'epoch': '0.07563'}
{'loss': '0.8502', 'grad_norm': '0.4244', 'learning_rate': '4.198e-05', 'entropy': '0.8517', 'num_tokens': '4.606e+07', 'mean_token_accuracy': '0.7863', 'epoch': '0.07564'}
{'loss': '0.6896', 'grad_norm': '0.3523', 'learning_rate': '4.196e-05', 'entropy': '0.7376', 'num_tokens': '4.607e+07', 'mean_token_accuracy': '0.8077', 'epoch': '0.07565'}
{'loss': '0.5654', 'grad_norm': '0.4286', 'learning_rate': '4.194e-05', 'entropy': '0.6008', 'num_tokens': '4.607e+07', 'mean_token_accuracy': '0.8395', 'epoch': '0.07566'}
{'loss': '0.8299', 'grad_norm': '0.4661', 'learning_rate': '4.191e-05', 'entropy': '0.7597', 'num_tokens': '4.608e+07', 'mean_token_accuracy': '0.8099', 'epoch': '0.07567'}
{'loss': '1.044', 'grad_norm': '0.4649', 'learning_rate': '4.189e-05', 'entropy': '1.032', 'num_tokens': '4.608e+07', 'mean_token_accuracy': '0.7419', 'epoch': '0.07567'}
{'loss': '1.044', 'grad_norm': '0.4431', 'learning_rate': '4.187e-05', 'entropy': '1.027', 'num_tokens': '4.609e+07', 'mean_token_accuracy': '0.7583', 'epoch': '0.07568'}
{'loss': '0.5888', 'grad_norm': '0.3542', 'learning_rate': '4.185e-05', 'entropy': '0.7089', 'num_tokens': '4.609e+07', 'mean_token_accuracy': '0.8183', 'epoch': '0.07569'}
{'loss': '0.6027', 'grad_norm': '0.4214', 'learning_rate': '4.183e-05', 'entropy': '0.5938', 'num_tokens': '4.61e+07', 'mean_token_accuracy': '0.8429', 'epoch': '0.0757'}
{'loss': '0.7435', 'grad_norm': '0.5063', 'learning_rate': '4.18e-05', 'entropy': '0.6744', 'num_tokens': '4.61e+07', 'mean_token_accuracy': '0.8239', 'epoch': '0.07571'}
{'loss': '1.054', 'grad_norm': '0.4597', 'learning_rate': '4.178e-05', 'entropy': '0.9235', 'num_tokens': '4.611e+07', 'mean_token_accuracy': '0.7714', 'epoch': '0.07572'}
{'loss': '0.7296', 'grad_norm': '0.3614', 'learning_rate': '4.176e-05', 'entropy': '0.7484', 'num_tokens': '4.611e+07', 'mean_token_accuracy': '0.7981', 'epoch': '0.07573'}

 71%|███████▏  | 8558/12000 [5:03:16<1:44:01,  1.81s/it]
 71%|███████▏  | 8559/12000 [5:03:18<1:40:45,  1.76s/it]
                                                        

 71%|███████▏  | 8559/12000 [5:03:18<1:40:45,  1.76s/it]
 71%|███████▏  | 8560/12000 [5:03:21<2:06:29,  2.21s/it]
                                                        

 71%|███████▏  | 8560/12000 [5:03:21<2:06:29,  2.21s/it]
 71%|███████▏  | 8561/12000 [5:03:23<2:14:11,  2.34s/it]
                                                        

 71%|███████▏  | 8561/12000 [5:03:23<2:14:11,  2.34s/it]
 71%|███████▏  | 8562/12000 [5:03:25<1:59:41,  2.09s/it]
                                                        

 71%|███████▏  | 8562/12000 [5:03:25<1:59:41,  2.09s/it]
 71%|███████▏  | 8563/12000 [5:03:28<2:08:06,  2.24s/it]
                                                        

 71%|███████▏  | 8563/12000 [5:03:28<2:08:06,  2.24s/it]
 71%|███████▏  | 8564/12000 [5:03:29<1:55:18,  2.01s/it]
                                                        

 71%|███████▏  | 8564/12000 [5:03:29<1:55:18,  2.01s/it]
 71%|███████▏  | 8565/12000 [5:03:31<1:48:32,  1.90s/it]
                                                        

 71%|███████▏  | 8565/12000 [5:03:31<1:48:32,  1.90s/it]
 71%|███████▏  | 8566/12000 [5:03:33<1:56:55,  2.04s/it]
                                                        

 71%|███████▏  | 8566/12000 [5:03:33<1:56:55,  2.04s/it]
 71%|███████▏  | 8567/12000 [5:03:35<1:59:13,  2.08s/it]
                                                        

 71%|███████▏  | 8567/12000 [5:03:35<1:59:13,  2.08s/it]
 71%|███████▏  | 8568/12000 [5:03:39<2:27:07,  2.57s/it]
                                                        

 71%|███████▏  | 8568/12000 [5:03:39<2:27:07,  2.57s/it]
 71%|███████▏  | 8569/12000 [5:03:41<2:11:26,  2.30s/it]
                                                        

 71%|███████▏  | 8569/12000 [5:03:41<2:11:26,  2.30s/it]
 71%|███████▏  | 8570/12000 [5:03:42<2:01:22,  2.12s/it]
                                                        

 71%|███████▏  | 8570/12000 [5:03:42<2:01:22,  2.12s/it]
 71%|███████▏  | 8571/12000 [5:03:44<1:52:03,  1.96s/it]
                                                        

 71%|███████▏  | 8571/12000 [5:03:44<1:52:03,  1.96s/it]
 71%|███████▏  | 8572/12000 [5:03:46<1:49:41,  1.92s/it]
                                                        

 71%|███████▏  | 8572/12000 [5:03:46<1:49:41,  1.92s/it]
 71%|███████▏  | 8573/12000 [5:03:47<1:43:01,  1.80s/it]
                                                        

 71%|███████▏  | 8573/12000 [5:03:47<1:43:01,  1.80s/it]
 71%|███████▏  | 8574/12000 [5:03:50<1:54:29,  2.00s/it]
                                                        

 71%|███████▏  | 8574/12000 [5:03:50<1:54:29,  2.00s/it]
 71%|███████▏  | 8575/12000 [5:03:51<1:45:50,  1.85s/it]
                                                        

 71%|███████▏  | 8575/12000 [5:03:51<1:45:50,  1.85s/it]
 71%|███████▏  | 8576/12000 [5:03:53<1:40:21,  1.76s/it]
                                                        

 71%|███████▏  | 8576/12000 [5:03:53<1:40:21,  1.76s/it]
 71%|███████▏  | 8577/12000 [5:03:54<1:39:51,  1.75s/it]
                                                        

 71%|███████▏  | 8577/12000 [5:03:54<1:39:51,  1.75s/it]
 71%|███████▏  | 8578/12000 [5:03:57<2:00:45,  2.12s/it]
                                                        

 71%|███████▏  | 8578/12000 [5:03:57<2:00:45,  2.12s/it]
 71%|███████▏  | 8579/12000 [5:03:59<1:49:03,  1.91s/it]
                                                        

 71%|███████▏  | 8579/12000 [5:03:59<1:49:03,  1.91s/it]
 72%|███████▏  | 8580/12000 [5:04:01<1:59:33,  2.10s/it]
                                                        

 72%|███████▏  | 8580/12000 [5:04:01<1:59:33,  2.10s/it]
 72%|███████▏  | 8581/12000 [5:04:03<1:48:39,  1.91s/it]
                                                        

 72%|███████▏  | 8581/12000 [5:04:03<1:48:39,  1.91s/it]
 72%|███████▏  | 8582/12000 [5:04:05<1:45:16,  1.85s/it]
                                                        

 72%|███████▏  | 8582/12000 [5:04:05<1:45:16,  1.85s/it]
 72%|███████▏  | 8583/12000 [5:04:06<1:44:55,  1.84s/it]
                                                        

 72%|███████▏  | 8583/12000 [5:04:06<1:44:55,  1.84s/it]
 72%|███████▏  | 8584/12000 [5:04:09<1:57:08,  2.06s/it]
                                                        

 72%|███████▏  | 8584/12000 [5:04:09<1:57:08,  2.06s/it]
 72%|███████▏  | 8585/12000 [5:04:10<1:45:59,  1.86s/it]
                                                        

 72%|███████▏  | 8585/12000 [5:04:10<1:45:59,  1.86s/it]
 72%|███████▏  | 8586/12000 [5:04:13<1:55:53,  2.04s/it]
                                                        

 72%|███████▏  | 8586/12000 [5:04:13<1:55:53,  2.04s/it]
 72%|███████▏  | 8587/12000 [5:04:15<1:59:40,  2.10s/it]
                                                        

 72%|███████▏  | 8587/12000 [5:04:15<1:59:40,  2.10s/it]
 72%|███████▏  | 8588/12000 [5:04:17<1:53:14,  1.99s/it]
                                                        

 72%|███████▏  | 8588/12000 [5:04:17<1:53:14,  1.99s/it]
 72%|███████▏  | 8589/12000 [5:04:20<2:06:39,  2.23s/it]
                                                        

 72%|███████▏  | 8589/12000 [5:04:20<2:06:39,  2.23s/it]
 72%|███████▏  | 8590/12000 [5:04:23<2:26:39,  2.58s/it]
                                                        

 72%|███████▏  | 8590/12000 [5:04:23<2:26:39,  2.58s/it]
 72%|███████▏  | 8591/12000 [5:04:25<2:10:12,  2.29s/it]
                                                        

 72%|███████▏  | 8591/12000 [5:04:25<2:10:12,  2.29s/it]
 72%|███████▏  | 8592/12000 [5:04:29<2:39:20,  2.81s/it]
                                                        

 72%|███████▏  | 8592/12000 [5:04:29<2:39:20,  2.81s/it]
 72%|███████▏  | 8593/12000 [5:04:31<2:27:44,  2.60s/it]
                                                        

 72%|███████▏  | 8593/12000 [5:04:31<2:27:44,  2.60s/it]
 72%|███████▏  | 8594/12000 [5:04:32<2:08:24,  2.26s/it]
                                                        

 72%|███████▏  | 8594/12000 [5:04:32<2:08:24,  2.26s/it]
 72%|███████▏  | 8595/12000 [5:04:34<2:00:51,  2.13s/it]
                                                        

 72%|███████▏  | 8595/12000 [5:04:34<2:00:51,  2.13s/it]
 72%|███████▏  | 8596/12000 [5:04:36<1:50:18,  1.94s/it]
                                                        

 72%|███████▏  | 8596/12000 [5:04:36<1:50:18,  1.94s/it]
 72%|███████▏  | 8597/12000 [5:04:37<1:48:35,  1.91s/it]
                                                        

 72%|███████▏  | 8597/12000 [5:04:37<1:48:35,  1.91s/it]
 72%|███████▏  | 8598/12000 [5:04:39<1:43:31,  1.83s/it]
                                                        

 72%|███████▏  | 8598/12000 [5:04:39<1:43:31,  1.83s/it]
 72%|███████▏  | 8599/12000 [5:04:41<1:41:31,  1.79s/it]
                                                        

 72%|███████▏  | 8599/12000 [5:04:41<1:41:31,  1.79s/it]
 72%|███████▏  | 8600/12000 [5:04:43<1:51:48,  1.97s/it]
                                                        

 72%|███████▏  | 8600/12000 [5:04:43<1:51:48,  1.97s/it]
 72%|███████▏  | 8601/12000 [5:04:46<2:10:20,  2.30s/it]
                                                        

 72%|███████▏  | 8601/12000 [5:04:46<2:10:20,  2.30s/it]
 72%|███████▏  | 8602/12000 [5:04:48<1:56:02,  2.05s/it]
                                                        

 72%|███████▏  | 8602/12000 [5:04:48<1:56:02,  2.05s/it]
 72%|███████▏  | 8603/12000 [5:04:54<3:05:55,  3.28s/it]
                                                        

 72%|███████▏  | 8603/12000 [5:04:54<3:05:55,  3.28s/it]
 72%|███████▏  | 8604/12000 [5:04:56<2:46:07,  2.94s/it]
                                                        

 72%|███████▏  | 8604/12000 [5:04:56<2:46:07,  2.94s/it]
 72%|███████▏  | 8605/12000 [5:04:57<2:20:04,  2.48s/it]
                                                        
{'loss': '0.772', 'grad_norm': '0.3431', 'learning_rate': '4.174e-05', 'entropy': '0.8657', 'num_tokens': '4.612e+07', 'mean_token_accuracy': '0.7804', 'epoch': '0.07574'}
{'loss': '0.4183', 'grad_norm': '0.3309', 'learning_rate': '4.171e-05', 'entropy': '0.4329', 'num_tokens': '4.612e+07', 'mean_token_accuracy': '0.8865', 'epoch': '0.07575'}
{'loss': '1.255', 'grad_norm': '0.5406', 'learning_rate': '4.169e-05', 'entropy': '1.106', 'num_tokens': '4.613e+07', 'mean_token_accuracy': '0.7246', 'epoch': '0.07575'}
{'loss': '0.713', 'grad_norm': '0.3693', 'learning_rate': '4.167e-05', 'entropy': '0.7431', 'num_tokens': '4.614e+07', 'mean_token_accuracy': '0.8076', 'epoch': '0.07576'}
{'loss': '1.029', 'grad_norm': '0.3792', 'learning_rate': '4.165e-05', 'entropy': '1.017', 'num_tokens': '4.614e+07', 'mean_token_accuracy': '0.7598', 'epoch': '0.07577'}
{'loss': '0.9011', 'grad_norm': '0.3458', 'learning_rate': '4.162e-05', 'entropy': '0.9363', 'num_tokens': '4.615e+07', 'mean_token_accuracy': '0.7683', 'epoch': '0.07578'}
{'loss': '0.5292', 'grad_norm': '0.3334', 'learning_rate': '4.16e-05', 'entropy': '0.5738', 'num_tokens': '4.615e+07', 'mean_token_accuracy': '0.8564', 'epoch': '0.07579'}
{'loss': '0.7843', 'grad_norm': '0.4309', 'learning_rate': '4.158e-05', 'entropy': '0.8173', 'num_tokens': '4.616e+07', 'mean_token_accuracy': '0.7945', 'epoch': '0.0758'}
{'loss': '1.277', 'grad_norm': '0.5703', 'learning_rate': '4.156e-05', 'entropy': '1.209', 'num_tokens': '4.616e+07', 'mean_token_accuracy': '0.7134', 'epoch': '0.07581'}
{'loss': '0.6706', 'grad_norm': '0.3969', 'learning_rate': '4.153e-05', 'entropy': '0.7038', 'num_tokens': '4.617e+07', 'mean_token_accuracy': '0.8142', 'epoch': '0.07582'}
{'loss': '0.5906', 'grad_norm': '0.4004', 'learning_rate': '4.151e-05', 'entropy': '0.5749', 'num_tokens': '4.617e+07', 'mean_token_accuracy': '0.8551', 'epoch': '0.07583'}
{'loss': '1.241', 'grad_norm': '0.5163', 'learning_rate': '4.149e-05', 'entropy': '1.141', 'num_tokens': '4.618e+07', 'mean_token_accuracy': '0.7276', 'epoch': '0.07583'}
{'loss': '0.6121', 'grad_norm': '0.4021', 'learning_rate': '4.147e-05', 'entropy': '0.674', 'num_tokens': '4.618e+07', 'mean_token_accuracy': '0.8224', 'epoch': '0.07584'}
{'loss': '0.8622', 'grad_norm': '0.3851', 'learning_rate': '4.144e-05', 'entropy': '0.976', 'num_tokens': '4.619e+07', 'mean_token_accuracy': '0.7568', 'epoch': '0.07585'}
{'loss': '0.8929', 'grad_norm': '0.4151', 'learning_rate': '4.142e-05', 'entropy': '0.8863', 'num_tokens': '4.619e+07', 'mean_token_accuracy': '0.7796', 'epoch': '0.07586'}
{'loss': '0.703', 'grad_norm': '0.429', 'learning_rate': '4.14e-05', 'entropy': '0.8045', 'num_tokens': '4.62e+07', 'mean_token_accuracy': '0.7887', 'epoch': '0.07587'}
{'loss': '0.829', 'grad_norm': '0.4572', 'learning_rate': '4.138e-05', 'entropy': '0.9209', 'num_tokens': '4.62e+07', 'mean_token_accuracy': '0.7744', 'epoch': '0.07588'}
{'loss': '0.5557', 'grad_norm': '0.3447', 'learning_rate': '4.136e-05', 'entropy': '0.5788', 'num_tokens': '4.621e+07', 'mean_token_accuracy': '0.8519', 'epoch': '0.07589'}
{'loss': '0.5159', 'grad_norm': '0.2709', 'learning_rate': '4.133e-05', 'entropy': '0.5491', 'num_tokens': '4.621e+07', 'mean_token_accuracy': '0.848', 'epoch': '0.0759'}
{'loss': '0.742', 'grad_norm': '0.4645', 'learning_rate': '4.131e-05', 'entropy': '0.7152', 'num_tokens': '4.622e+07', 'mean_token_accuracy': '0.8064', 'epoch': '0.07591'}
{'loss': '0.6976', 'grad_norm': '0.3288', 'learning_rate': '4.129e-05', 'entropy': '0.9739', 'num_tokens': '4.622e+07', 'mean_token_accuracy': '0.7714', 'epoch': '0.07591'}
{'loss': '0.5223', 'grad_norm': '0.2717', 'learning_rate': '4.127e-05', 'entropy': '0.5769', 'num_tokens': '4.623e+07', 'mean_token_accuracy': '0.8577', 'epoch': '0.07592'}
{'loss': '0.6997', 'grad_norm': '0.3723', 'learning_rate': '4.124e-05', 'entropy': '0.691', 'num_tokens': '4.623e+07', 'mean_token_accuracy': '0.8207', 'epoch': '0.07593'}
{'loss': '0.8623', 'grad_norm': '0.3997', 'learning_rate': '4.122e-05', 'entropy': '0.9188', 'num_tokens': '4.624e+07', 'mean_token_accuracy': '0.7707', 'epoch': '0.07594'}
{'loss': '0.8995', 'grad_norm': '0.3988', 'learning_rate': '4.12e-05', 'entropy': '0.9407', 'num_tokens': '4.624e+07', 'mean_token_accuracy': '0.75', 'epoch': '0.07595'}
{'loss': '0.6972', 'grad_norm': '0.419', 'learning_rate': '4.118e-05', 'entropy': '0.6877', 'num_tokens': '4.625e+07', 'mean_token_accuracy': '0.8242', 'epoch': '0.07596'}
{'loss': '0.6588', 'grad_norm': '0.3146', 'learning_rate': '4.115e-05', 'entropy': '0.6714', 'num_tokens': '4.626e+07', 'mean_token_accuracy': '0.83', 'epoch': '0.07597'}
{'loss': '0.7673', 'grad_norm': '0.354', 'learning_rate': '4.113e-05', 'entropy': '0.7781', 'num_tokens': '4.626e+07', 'mean_token_accuracy': '0.7902', 'epoch': '0.07598'}
{'loss': '1.024', 'grad_norm': '0.4289', 'learning_rate': '4.111e-05', 'entropy': '1.049', 'num_tokens': '4.627e+07', 'mean_token_accuracy': '0.7463', 'epoch': '0.07598'}
{'loss': '0.9747', 'grad_norm': '0.5392', 'learning_rate': '4.109e-05', 'entropy': '0.8173', 'num_tokens': '4.627e+07', 'mean_token_accuracy': '0.801', 'epoch': '0.07599'}
{'loss': '0.7766', 'grad_norm': '0.4324', 'learning_rate': '4.107e-05', 'entropy': '0.7423', 'num_tokens': '4.628e+07', 'mean_token_accuracy': '0.7974', 'epoch': '0.076'}
{'loss': '0.9704', 'grad_norm': '0.4376', 'learning_rate': '4.104e-05', 'entropy': '0.8222', 'num_tokens': '4.628e+07', 'mean_token_accuracy': '0.7912', 'epoch': '0.07601'}
{'loss': '0.7338', 'grad_norm': '0.3566', 'learning_rate': '4.102e-05', 'entropy': '0.7414', 'num_tokens': '4.629e+07', 'mean_token_accuracy': '0.8092', 'epoch': '0.07602'}
{'loss': '0.8041', 'grad_norm': '0.4636', 'learning_rate': '4.1e-05', 'entropy': '0.8281', 'num_tokens': '4.629e+07', 'mean_token_accuracy': '0.8047', 'epoch': '0.07603'}
{'loss': '1.087', 'grad_norm': '0.4042', 'learning_rate': '4.098e-05', 'entropy': '0.8854', 'num_tokens': '4.63e+07', 'mean_token_accuracy': '0.7649', 'epoch': '0.07604'}
{'loss': '0.893', 'grad_norm': '0.4804', 'learning_rate': '4.095e-05', 'entropy': '0.8859', 'num_tokens': '4.631e+07', 'mean_token_accuracy': '0.7849', 'epoch': '0.07605'}
{'loss': '0.8746', 'grad_norm': '0.3926', 'learning_rate': '4.093e-05', 'entropy': '0.8488', 'num_tokens': '4.631e+07', 'mean_token_accuracy': '0.7805', 'epoch': '0.07606'}
{'loss': '0.8125', 'grad_norm': '0.4315', 'learning_rate': '4.091e-05', 'entropy': '0.8264', 'num_tokens': '4.632e+07', 'mean_token_accuracy': '0.7931', 'epoch': '0.07606'}
{'loss': '0.6009', 'grad_norm': '0.4073', 'learning_rate': '4.089e-05', 'entropy': '0.6877', 'num_tokens': '4.632e+07', 'mean_token_accuracy': '0.8258', 'epoch': '0.07607'}
{'loss': '0.6942', 'grad_norm': '0.3662', 'learning_rate': '4.087e-05', 'entropy': '0.7386', 'num_tokens': '4.633e+07', 'mean_token_accuracy': '0.8141', 'epoch': '0.07608'}
{'loss': '0.9443', 'grad_norm': '0.4275', 'learning_rate': '4.084e-05', 'entropy': '1.011', 'num_tokens': '4.633e+07', 'mean_token_accuracy': '0.748', 'epoch': '0.07609'}
{'loss': '0.7034', 'grad_norm': '0.4162', 'learning_rate': '4.082e-05', 'entropy': '0.7629', 'num_tokens': '4.634e+07', 'mean_token_accuracy': '0.7973', 'epoch': '0.0761'}
{'loss': '1.093', 'grad_norm': '0.4356', 'learning_rate': '4.08e-05', 'entropy': '0.9954', 'num_tokens': '4.635e+07', 'mean_token_accuracy': '0.7672', 'epoch': '0.07611'}
{'loss': '0.8163', 'grad_norm': '0.4272', 'learning_rate': '4.078e-05', 'entropy': '0.7714', 'num_tokens': '4.635e+07', 'mean_token_accuracy': '0.7889', 'epoch': '0.07612'}
{'loss': '0.6438', 'grad_norm': '0.4098', 'learning_rate': '4.075e-05', 'entropy': '0.6261', 'num_tokens': '4.636e+07', 'mean_token_accuracy': '0.8329', 'epoch': '0.07613'}
{'loss': '0.9114', 'grad_norm': '0.2544', 'learning_rate': '4.073e-05', 'entropy': '0.8621', 'num_tokens': '4.637e+07', 'mean_token_accuracy': '0.7934', 'epoch': '0.07614'}
{'loss': '1.123', 'grad_norm': '0.4088', 'learning_rate': '4.071e-05', 'entropy': '1', 'num_tokens': '4.637e+07', 'mean_token_accuracy': '0.7643', 'epoch': '0.07614'}

 72%|███████▏  | 8605/12000 [5:04:57<2:20:04,  2.48s/it]
 72%|███████▏  | 8606/12000 [5:05:00<2:18:29,  2.45s/it]
                                                        

 72%|███████▏  | 8606/12000 [5:05:00<2:18:29,  2.45s/it]
 72%|███████▏  | 8607/12000 [5:05:01<2:06:39,  2.24s/it]
                                                        

 72%|███████▏  | 8607/12000 [5:05:01<2:06:39,  2.24s/it]
 72%|███████▏  | 8608/12000 [5:05:04<2:15:26,  2.40s/it]
                                                        

 72%|███████▏  | 8608/12000 [5:05:04<2:15:26,  2.40s/it]
 72%|███████▏  | 8609/12000 [5:05:07<2:13:43,  2.37s/it]
                                                        

 72%|███████▏  | 8609/12000 [5:05:07<2:13:43,  2.37s/it]
 72%|███████▏  | 8610/12000 [5:05:08<1:56:57,  2.07s/it]
                                                        

 72%|███████▏  | 8610/12000 [5:05:08<1:56:57,  2.07s/it]
 72%|███████▏  | 8611/12000 [5:05:09<1:47:29,  1.90s/it]
                                                        

 72%|███████▏  | 8611/12000 [5:05:09<1:47:29,  1.90s/it]
 72%|███████▏  | 8612/12000 [5:05:13<2:18:59,  2.46s/it]
                                                        

 72%|███████▏  | 8612/12000 [5:05:13<2:18:59,  2.46s/it]
 72%|███████▏  | 8613/12000 [5:05:15<2:02:03,  2.16s/it]
                                                        

 72%|███████▏  | 8613/12000 [5:05:15<2:02:03,  2.16s/it]
 72%|███████▏  | 8614/12000 [5:05:16<1:51:16,  1.97s/it]
                                                        

 72%|███████▏  | 8614/12000 [5:05:16<1:51:16,  1.97s/it]
 72%|███████▏  | 8615/12000 [5:05:21<2:46:02,  2.94s/it]
                                                        

 72%|███████▏  | 8615/12000 [5:05:21<2:46:02,  2.94s/it]
 72%|███████▏  | 8616/12000 [5:05:24<2:34:09,  2.73s/it]
                                                        

 72%|███████▏  | 8616/12000 [5:05:24<2:34:09,  2.73s/it]
 72%|███████▏  | 8617/12000 [5:05:26<2:31:04,  2.68s/it]
                                                        

 72%|███████▏  | 8617/12000 [5:05:26<2:31:04,  2.68s/it]
 72%|███████▏  | 8618/12000 [5:05:28<2:13:29,  2.37s/it]
                                                        

 72%|███████▏  | 8618/12000 [5:05:28<2:13:29,  2.37s/it]
 72%|███████▏  | 8619/12000 [5:05:30<2:17:28,  2.44s/it]
                                                        

 72%|███████▏  | 8619/12000 [5:05:30<2:17:28,  2.44s/it]
 72%|███████▏  | 8620/12000 [5:05:32<2:08:37,  2.28s/it]
                                                        

 72%|███████▏  | 8620/12000 [5:05:32<2:08:37,  2.28s/it]
 72%|███████▏  | 8621/12000 [5:05:34<2:00:40,  2.14s/it]
                                                        

 72%|███████▏  | 8621/12000 [5:05:34<2:00:40,  2.14s/it]
 72%|███████▏  | 8622/12000 [5:05:38<2:21:22,  2.51s/it]
                                                        

 72%|███████▏  | 8622/12000 [5:05:38<2:21:22,  2.51s/it]
 72%|███████▏  | 8623/12000 [5:05:39<2:04:23,  2.21s/it]
                                                        

 72%|███████▏  | 8623/12000 [5:05:39<2:04:23,  2.21s/it]
 72%|███████▏  | 8624/12000 [5:05:41<1:54:01,  2.03s/it]
                                                        

 72%|███████▏  | 8624/12000 [5:05:41<1:54:01,  2.03s/it]
 72%|███████▏  | 8625/12000 [5:05:42<1:48:44,  1.93s/it]
                                                        

 72%|███████▏  | 8625/12000 [5:05:42<1:48:44,  1.93s/it]
 72%|███████▏  | 8626/12000 [5:05:44<1:45:07,  1.87s/it]
                                                        

 72%|███████▏  | 8626/12000 [5:05:44<1:45:07,  1.87s/it]
 72%|███████▏  | 8627/12000 [5:05:46<1:43:03,  1.83s/it]
                                                        

 72%|███████▏  | 8627/12000 [5:05:46<1:43:03,  1.83s/it]
 72%|███████▏  | 8628/12000 [5:05:47<1:37:31,  1.74s/it]
                                                        

 72%|███████▏  | 8628/12000 [5:05:47<1:37:31,  1.74s/it]
 72%|███████▏  | 8629/12000 [5:05:49<1:37:05,  1.73s/it]
                                                        

 72%|███████▏  | 8629/12000 [5:05:49<1:37:05,  1.73s/it]
 72%|███████▏  | 8630/12000 [5:05:50<1:31:35,  1.63s/it]
                                                        

 72%|███████▏  | 8630/12000 [5:05:50<1:31:35,  1.63s/it]
 72%|███████▏  | 8631/12000 [5:05:52<1:31:25,  1.63s/it]
                                                        

 72%|███████▏  | 8631/12000 [5:05:52<1:31:25,  1.63s/it]
 72%|███████▏  | 8632/12000 [5:05:54<1:32:12,  1.64s/it]
                                                        

 72%|███████▏  | 8632/12000 [5:05:54<1:32:12,  1.64s/it]
 72%|███████▏  | 8633/12000 [5:05:55<1:33:39,  1.67s/it]
                                                        

 72%|███████▏  | 8633/12000 [5:05:55<1:33:39,  1.67s/it]
 72%|███████▏  | 8634/12000 [5:05:59<2:08:05,  2.28s/it]
                                                        

 72%|███████▏  | 8634/12000 [5:05:59<2:08:05,  2.28s/it]
 72%|███████▏  | 8635/12000 [5:06:01<1:57:02,  2.09s/it]
                                                        

 72%|███████▏  | 8635/12000 [5:06:01<1:57:02,  2.09s/it]
 72%|███████▏  | 8636/12000 [5:06:03<1:50:53,  1.98s/it]
                                                        

 72%|███████▏  | 8636/12000 [5:06:03<1:50:53,  1.98s/it]
 72%|███████▏  | 8637/12000 [5:06:05<1:58:11,  2.11s/it]
                                                        

 72%|███████▏  | 8637/12000 [5:06:05<1:58:11,  2.11s/it]
 72%|███████▏  | 8638/12000 [5:06:06<1:47:25,  1.92s/it]
                                                        

 72%|███████▏  | 8638/12000 [5:06:06<1:47:25,  1.92s/it]
 72%|███████▏  | 8639/12000 [5:06:10<2:09:18,  2.31s/it]
                                                        

 72%|███████▏  | 8639/12000 [5:06:10<2:09:18,  2.31s/it]
 72%|███████▏  | 8640/12000 [5:06:11<1:59:18,  2.13s/it]
                                                        

 72%|███████▏  | 8640/12000 [5:06:11<1:59:18,  2.13s/it]
 72%|███████▏  | 8641/12000 [5:06:16<2:34:41,  2.76s/it]
                                                        

 72%|███████▏  | 8641/12000 [5:06:16<2:34:41,  2.76s/it]
 72%|███████▏  | 8642/12000 [5:06:17<2:13:59,  2.39s/it]
                                                        

 72%|███████▏  | 8642/12000 [5:06:17<2:13:59,  2.39s/it]
 72%|███████▏  | 8643/12000 [5:06:19<2:09:10,  2.31s/it]
                                                        

 72%|███████▏  | 8643/12000 [5:06:19<2:09:10,  2.31s/it]
 72%|███████▏  | 8644/12000 [5:06:21<2:06:17,  2.26s/it]
                                                        

 72%|███████▏  | 8644/12000 [5:06:21<2:06:17,  2.26s/it]
 72%|███████▏  | 8645/12000 [5:06:23<1:51:34,  2.00s/it]
                                                        

 72%|███████▏  | 8645/12000 [5:06:23<1:51:34,  2.00s/it]
 72%|███████▏  | 8646/12000 [5:06:25<1:47:58,  1.93s/it]
                                                        

 72%|███████▏  | 8646/12000 [5:06:25<1:47:58,  1.93s/it]
 72%|███████▏  | 8647/12000 [5:06:27<1:51:59,  2.00s/it]
                                                        

 72%|███████▏  | 8647/12000 [5:06:27<1:51:59,  2.00s/it]
 72%|███████▏  | 8648/12000 [5:06:28<1:46:03,  1.90s/it]
                                                        

 72%|███████▏  | 8648/12000 [5:06:28<1:46:03,  1.90s/it]
 72%|███████▏  | 8649/12000 [5:06:31<1:49:51,  1.97s/it]
                                                        

 72%|███████▏  | 8649/12000 [5:06:31<1:49:51,  1.97s/it]
 72%|███████▏  | 8650/12000 [5:06:32<1:42:17,  1.83s/it]
                                                        

 72%|███████▏  | 8650/12000 [5:06:32<1:42:17,  1.83s/it]
 72%|███████▏  | 8651/12000 [5:06:38<2:54:12,  3.12s/it]
                                                        

 72%|███████▏  | 8651/12000 [5:06:38<2:54:12,  3.12s/it]
 72%|███████▏  | 8652/12000 [5:06:40<2:31:19,  2.71s/it]
                                                        
{'loss': '0.7364', 'grad_norm': '0.4412', 'learning_rate': '4.069e-05', 'entropy': '0.7818', 'num_tokens': '4.638e+07', 'mean_token_accuracy': '0.7951', 'epoch': '0.07615'}
{'loss': '0.9533', 'grad_norm': '0.5184', 'learning_rate': '4.067e-05', 'entropy': '0.9746', 'num_tokens': '4.638e+07', 'mean_token_accuracy': '0.7621', 'epoch': '0.07616'}
{'loss': '0.7826', 'grad_norm': '0.4297', 'learning_rate': '4.064e-05', 'entropy': '0.8024', 'num_tokens': '4.639e+07', 'mean_token_accuracy': '0.7885', 'epoch': '0.07617'}
{'loss': '1.319', 'grad_norm': '0.4278', 'learning_rate': '4.062e-05', 'entropy': '1.267', 'num_tokens': '4.639e+07', 'mean_token_accuracy': '0.7054', 'epoch': '0.07618'}
{'loss': '0.8927', 'grad_norm': '0.3976', 'learning_rate': '4.06e-05', 'entropy': '1.005', 'num_tokens': '4.64e+07', 'mean_token_accuracy': '0.7598', 'epoch': '0.07619'}
{'loss': '0.7178', 'grad_norm': '0.3707', 'learning_rate': '4.058e-05', 'entropy': '0.7399', 'num_tokens': '4.64e+07', 'mean_token_accuracy': '0.8138', 'epoch': '0.0762'}
{'loss': '0.7055', 'grad_norm': '0.3536', 'learning_rate': '4.055e-05', 'entropy': '0.7445', 'num_tokens': '4.641e+07', 'mean_token_accuracy': '0.8152', 'epoch': '0.07621'}
{'loss': '0.8294', 'grad_norm': '0.335', 'learning_rate': '4.053e-05', 'entropy': '0.7876', 'num_tokens': '4.641e+07', 'mean_token_accuracy': '0.8151', 'epoch': '0.07621'}
{'loss': '0.8026', 'grad_norm': '0.3921', 'learning_rate': '4.051e-05', 'entropy': '0.8236', 'num_tokens': '4.642e+07', 'mean_token_accuracy': '0.7889', 'epoch': '0.07622'}
{'loss': '0.5491', 'grad_norm': '0.3595', 'learning_rate': '4.049e-05', 'entropy': '0.6419', 'num_tokens': '4.642e+07', 'mean_token_accuracy': '0.8392', 'epoch': '0.07623'}
{'loss': '0.8177', 'grad_norm': '0.3623', 'learning_rate': '4.047e-05', 'entropy': '0.726', 'num_tokens': '4.643e+07', 'mean_token_accuracy': '0.8195', 'epoch': '0.07624'}
{'loss': '0.4886', 'grad_norm': '0.3577', 'learning_rate': '4.044e-05', 'entropy': '0.4806', 'num_tokens': '4.644e+07', 'mean_token_accuracy': '0.871', 'epoch': '0.07625'}
{'loss': '0.5379', 'grad_norm': '0.3262', 'learning_rate': '4.042e-05', 'entropy': '0.5985', 'num_tokens': '4.645e+07', 'mean_token_accuracy': '0.843', 'epoch': '0.07626'}
{'loss': '0.7091', 'grad_norm': '0.4063', 'learning_rate': '4.04e-05', 'entropy': '0.6703', 'num_tokens': '4.645e+07', 'mean_token_accuracy': '0.8241', 'epoch': '0.07627'}
{'loss': '0.7376', 'grad_norm': '0.3622', 'learning_rate': '4.038e-05', 'entropy': '1.013', 'num_tokens': '4.646e+07', 'mean_token_accuracy': '0.7689', 'epoch': '0.07628'}
{'loss': '0.7001', 'grad_norm': '0.4673', 'learning_rate': '4.036e-05', 'entropy': '0.7447', 'num_tokens': '4.646e+07', 'mean_token_accuracy': '0.8193', 'epoch': '0.07629'}
{'loss': '0.8312', 'grad_norm': '0.4576', 'learning_rate': '4.033e-05', 'entropy': '0.8413', 'num_tokens': '4.647e+07', 'mean_token_accuracy': '0.7987', 'epoch': '0.07629'}
{'loss': '0.7665', 'grad_norm': '0.3597', 'learning_rate': '4.031e-05', 'entropy': '0.7683', 'num_tokens': '4.648e+07', 'mean_token_accuracy': '0.807', 'epoch': '0.0763'}
{'loss': '0.9225', 'grad_norm': '0.4298', 'learning_rate': '4.029e-05', 'entropy': '0.9102', 'num_tokens': '4.648e+07', 'mean_token_accuracy': '0.7862', 'epoch': '0.07631'}
{'loss': '0.6156', 'grad_norm': '0.3683', 'learning_rate': '4.027e-05', 'entropy': '0.6271', 'num_tokens': '4.648e+07', 'mean_token_accuracy': '0.8396', 'epoch': '0.07632'}
{'loss': '0.9315', 'grad_norm': '0.4599', 'learning_rate': '4.024e-05', 'entropy': '0.9319', 'num_tokens': '4.649e+07', 'mean_token_accuracy': '0.7738', 'epoch': '0.07633'}
{'loss': '0.654', 'grad_norm': '0.4332', 'learning_rate': '4.022e-05', 'entropy': '0.6467', 'num_tokens': '4.649e+07', 'mean_token_accuracy': '0.8298', 'epoch': '0.07634'}
{'loss': '1.042', 'grad_norm': '0.412', 'learning_rate': '4.02e-05', 'entropy': '1.013', 'num_tokens': '4.65e+07', 'mean_token_accuracy': '0.7642', 'epoch': '0.07635'}
{'loss': '0.6895', 'grad_norm': '0.3297', 'learning_rate': '4.018e-05', 'entropy': '0.717', 'num_tokens': '4.65e+07', 'mean_token_accuracy': '0.8263', 'epoch': '0.07636'}
{'loss': '0.6806', 'grad_norm': '0.4445', 'learning_rate': '4.016e-05', 'entropy': '0.7084', 'num_tokens': '4.651e+07', 'mean_token_accuracy': '0.8221', 'epoch': '0.07637'}
{'loss': '0.503', 'grad_norm': '0.2816', 'learning_rate': '4.013e-05', 'entropy': '0.548', 'num_tokens': '4.651e+07', 'mean_token_accuracy': '0.8436', 'epoch': '0.07637'}
{'loss': '0.9282', 'grad_norm': '0.3818', 'learning_rate': '4.011e-05', 'entropy': '0.9631', 'num_tokens': '4.652e+07', 'mean_token_accuracy': '0.7694', 'epoch': '0.07638'}
{'loss': '0.7956', 'grad_norm': '0.395', 'learning_rate': '4.009e-05', 'entropy': '0.7984', 'num_tokens': '4.652e+07', 'mean_token_accuracy': '0.7916', 'epoch': '0.07639'}
{'loss': '1.135', 'grad_norm': '0.4412', 'learning_rate': '4.007e-05', 'entropy': '1.079', 'num_tokens': '4.653e+07', 'mean_token_accuracy': '0.7418', 'epoch': '0.0764'}
{'loss': '1.008', 'grad_norm': '0.4327', 'learning_rate': '4.005e-05', 'entropy': '1.13', 'num_tokens': '4.653e+07', 'mean_token_accuracy': '0.7267', 'epoch': '0.07641'}
{'loss': '1.018', 'grad_norm': '0.4292', 'learning_rate': '4.002e-05', 'entropy': '1.001', 'num_tokens': '4.654e+07', 'mean_token_accuracy': '0.756', 'epoch': '0.07642'}
{'loss': '1.152', 'grad_norm': '0.3986', 'learning_rate': '4e-05', 'entropy': '0.9939', 'num_tokens': '4.654e+07', 'mean_token_accuracy': '0.7503', 'epoch': '0.07643'}
{'loss': '0.5362', 'grad_norm': '0.3233', 'learning_rate': '3.998e-05', 'entropy': '0.5736', 'num_tokens': '4.655e+07', 'mean_token_accuracy': '0.8531', 'epoch': '0.07644'}
{'loss': '1.051', 'grad_norm': '0.4824', 'learning_rate': '3.996e-05', 'entropy': '1.032', 'num_tokens': '4.655e+07', 'mean_token_accuracy': '0.7558', 'epoch': '0.07644'}
{'loss': '0.6605', 'grad_norm': '0.3507', 'learning_rate': '3.994e-05', 'entropy': '0.7433', 'num_tokens': '4.656e+07', 'mean_token_accuracy': '0.808', 'epoch': '0.07645'}
{'loss': '0.8832', 'grad_norm': '0.3899', 'learning_rate': '3.991e-05', 'entropy': '0.8314', 'num_tokens': '4.657e+07', 'mean_token_accuracy': '0.7783', 'epoch': '0.07646'}
{'loss': '1.389', 'grad_norm': '0.4096', 'learning_rate': '3.989e-05', 'entropy': '1.362', 'num_tokens': '4.657e+07', 'mean_token_accuracy': '0.6731', 'epoch': '0.07647'}
{'loss': '0.7487', 'grad_norm': '0.378', 'learning_rate': '3.987e-05', 'entropy': '0.7259', 'num_tokens': '4.658e+07', 'mean_token_accuracy': '0.8144', 'epoch': '0.07648'}
{'loss': '0.6697', 'grad_norm': '0.3648', 'learning_rate': '3.985e-05', 'entropy': '0.7025', 'num_tokens': '4.658e+07', 'mean_token_accuracy': '0.8161', 'epoch': '0.07649'}
{'loss': '0.9149', 'grad_norm': '0.4746', 'learning_rate': '3.983e-05', 'entropy': '0.7774', 'num_tokens': '4.659e+07', 'mean_token_accuracy': '0.8056', 'epoch': '0.0765'}
{'loss': '0.819', 'grad_norm': '0.5217', 'learning_rate': '3.98e-05', 'entropy': '0.8507', 'num_tokens': '4.659e+07', 'mean_token_accuracy': '0.7827', 'epoch': '0.07651'}
{'loss': '0.8561', 'grad_norm': '0.3626', 'learning_rate': '3.978e-05', 'entropy': '0.8155', 'num_tokens': '4.66e+07', 'mean_token_accuracy': '0.7859', 'epoch': '0.07652'}
{'loss': '0.793', 'grad_norm': '0.3855', 'learning_rate': '3.976e-05', 'entropy': '0.7743', 'num_tokens': '4.66e+07', 'mean_token_accuracy': '0.7947', 'epoch': '0.07652'}
{'loss': '0.6702', 'grad_norm': '0.3767', 'learning_rate': '3.974e-05', 'entropy': '0.6608', 'num_tokens': '4.661e+07', 'mean_token_accuracy': '0.8257', 'epoch': '0.07653'}
{'loss': '0.9172', 'grad_norm': '0.3307', 'learning_rate': '3.972e-05', 'entropy': '0.9321', 'num_tokens': '4.662e+07', 'mean_token_accuracy': '0.7733', 'epoch': '0.07654'}
{'loss': '1.179', 'grad_norm': '0.4866', 'learning_rate': '3.969e-05', 'entropy': '1.107', 'num_tokens': '4.662e+07', 'mean_token_accuracy': '0.7372', 'epoch': '0.07655'}
{'loss': '1.02', 'grad_norm': '0.3528', 'learning_rate': '3.967e-05', 'entropy': '0.7875', 'num_tokens': '4.663e+07', 'mean_token_accuracy': '0.8188', 'epoch': '0.07656'}

 72%|███████▏  | 8652/12000 [5:06:40<2:31:19,  2.71s/it]
 72%|███████▏  | 8653/12000 [5:06:41<2:12:14,  2.37s/it]
                                                        

 72%|███████▏  | 8653/12000 [5:06:41<2:12:14,  2.37s/it]
 72%|███████▏  | 8654/12000 [5:06:43<2:01:13,  2.17s/it]
                                                        

 72%|███████▏  | 8654/12000 [5:06:43<2:01:13,  2.17s/it]
 72%|███████▏  | 8655/12000 [5:06:45<1:55:06,  2.06s/it]
                                                        

 72%|███████▏  | 8655/12000 [5:06:45<1:55:06,  2.06s/it]
 72%|███████▏  | 8656/12000 [5:06:49<2:23:31,  2.58s/it]
                                                        

 72%|███████▏  | 8656/12000 [5:06:49<2:23:31,  2.58s/it]
 72%|███████▏  | 8657/12000 [5:06:50<2:05:32,  2.25s/it]
                                                        

 72%|███████▏  | 8657/12000 [5:06:50<2:05:32,  2.25s/it]
 72%|███████▏  | 8658/12000 [5:06:52<1:54:27,  2.05s/it]
                                                        

 72%|███████▏  | 8658/12000 [5:06:52<1:54:27,  2.05s/it]
 72%|███████▏  | 8659/12000 [5:06:54<1:47:27,  1.93s/it]
                                                        

 72%|███████▏  | 8659/12000 [5:06:54<1:47:27,  1.93s/it]
 72%|███████▏  | 8660/12000 [5:06:55<1:45:51,  1.90s/it]
                                                        

 72%|███████▏  | 8660/12000 [5:06:55<1:45:51,  1.90s/it]
 72%|███████▏  | 8661/12000 [5:06:57<1:41:38,  1.83s/it]
                                                        

 72%|███████▏  | 8661/12000 [5:06:57<1:41:38,  1.83s/it]
 72%|███████▏  | 8662/12000 [5:06:59<1:39:46,  1.79s/it]
                                                        

 72%|███████▏  | 8662/12000 [5:06:59<1:39:46,  1.79s/it]
 72%|███████▏  | 8663/12000 [5:07:02<2:11:54,  2.37s/it]
                                                        

 72%|███████▏  | 8663/12000 [5:07:02<2:11:54,  2.37s/it]
 72%|███████▏  | 8664/12000 [5:07:04<1:56:26,  2.09s/it]
                                                        

 72%|███████▏  | 8664/12000 [5:07:04<1:56:26,  2.09s/it]
 72%|███████▏  | 8665/12000 [5:07:05<1:43:51,  1.87s/it]
                                                        

 72%|███████▏  | 8665/12000 [5:07:05<1:43:51,  1.87s/it]
 72%|███████▏  | 8666/12000 [5:07:07<1:41:49,  1.83s/it]
                                                        

 72%|███████▏  | 8666/12000 [5:07:07<1:41:49,  1.83s/it]
 72%|███████▏  | 8667/12000 [5:07:10<1:58:49,  2.14s/it]
                                                        

 72%|███████▏  | 8667/12000 [5:07:10<1:58:49,  2.14s/it]
 72%|███████▏  | 8668/12000 [5:07:11<1:49:17,  1.97s/it]
                                                        

 72%|███████▏  | 8668/12000 [5:07:11<1:49:17,  1.97s/it]
 72%|███████▏  | 8669/12000 [5:07:13<1:42:50,  1.85s/it]
                                                        

 72%|███████▏  | 8669/12000 [5:07:13<1:42:50,  1.85s/it]
 72%|███████▏  | 8670/12000 [5:07:15<1:40:24,  1.81s/it]
                                                        

 72%|███████▏  | 8670/12000 [5:07:15<1:40:24,  1.81s/it]
 72%|███████▏  | 8671/12000 [5:07:17<1:45:01,  1.89s/it]
                                                        

 72%|███████▏  | 8671/12000 [5:07:17<1:45:01,  1.89s/it]
 72%|███████▏  | 8672/12000 [5:07:19<1:52:10,  2.02s/it]
                                                        

 72%|███████▏  | 8672/12000 [5:07:19<1:52:10,  2.02s/it]
 72%|███████▏  | 8673/12000 [5:07:20<1:40:31,  1.81s/it]
                                                        

 72%|███████▏  | 8673/12000 [5:07:20<1:40:31,  1.81s/it]
 72%|███████▏  | 8674/12000 [5:07:22<1:37:37,  1.76s/it]
                                                        

 72%|███████▏  | 8674/12000 [5:07:22<1:37:37,  1.76s/it]
 72%|███████▏  | 8675/12000 [5:07:24<1:35:53,  1.73s/it]
                                                        

 72%|███████▏  | 8675/12000 [5:07:24<1:35:53,  1.73s/it]
 72%|███████▏  | 8676/12000 [5:07:26<1:53:02,  2.04s/it]
                                                        

 72%|███████▏  | 8676/12000 [5:07:26<1:53:02,  2.04s/it]
 72%|███████▏  | 8677/12000 [5:07:29<2:02:46,  2.22s/it]
                                                        

 72%|███████▏  | 8677/12000 [5:07:29<2:02:46,  2.22s/it]
 72%|███████▏  | 8678/12000 [5:07:33<2:23:17,  2.59s/it]
                                                        

 72%|███████▏  | 8678/12000 [5:07:33<2:23:17,  2.59s/it]
 72%|███████▏  | 8679/12000 [5:07:35<2:23:40,  2.60s/it]
                                                        

 72%|███████▏  | 8679/12000 [5:07:35<2:23:40,  2.60s/it]
 72%|███████▏  | 8680/12000 [5:07:37<2:12:53,  2.40s/it]
                                                        

 72%|███████▏  | 8680/12000 [5:07:37<2:12:53,  2.40s/it]
 72%|███████▏  | 8681/12000 [5:07:38<1:55:37,  2.09s/it]
                                                        

 72%|███████▏  | 8681/12000 [5:07:38<1:55:37,  2.09s/it]
 72%|███████▏  | 8682/12000 [5:07:41<2:09:40,  2.34s/it]
                                                        

 72%|███████▏  | 8682/12000 [5:07:41<2:09:40,  2.34s/it]
 72%|███████▏  | 8683/12000 [5:07:43<1:53:14,  2.05s/it]
                                                        

 72%|███████▏  | 8683/12000 [5:07:43<1:53:14,  2.05s/it]
 72%|███████▏  | 8684/12000 [5:07:45<1:56:10,  2.10s/it]
                                                        

 72%|███████▏  | 8684/12000 [5:07:45<1:56:10,  2.10s/it]
 72%|███████▏  | 8685/12000 [5:07:47<1:47:14,  1.94s/it]
                                                        

 72%|███████▏  | 8685/12000 [5:07:47<1:47:14,  1.94s/it]
 72%|███████▏  | 8686/12000 [5:07:48<1:41:56,  1.85s/it]
                                                        

 72%|███████▏  | 8686/12000 [5:07:48<1:41:56,  1.85s/it]
 72%|███████▏  | 8687/12000 [5:07:50<1:35:42,  1.73s/it]
                                                        

 72%|███████▏  | 8687/12000 [5:07:50<1:35:42,  1.73s/it]
 72%|███████▏  | 8688/12000 [5:07:51<1:34:59,  1.72s/it]
                                                        

 72%|███████▏  | 8688/12000 [5:07:51<1:34:59,  1.72s/it]
 72%|███████▏  | 8689/12000 [5:07:53<1:39:24,  1.80s/it]
                                                        

 72%|███████▏  | 8689/12000 [5:07:53<1:39:24,  1.80s/it]
 72%|███████▏  | 8690/12000 [5:07:55<1:37:48,  1.77s/it]
                                                        

 72%|███████▏  | 8690/12000 [5:07:55<1:37:48,  1.77s/it]
 72%|███████▏  | 8691/12000 [5:07:57<1:36:11,  1.74s/it]
                                                        

 72%|███████▏  | 8691/12000 [5:07:57<1:36:11,  1.74s/it]
 72%|███████▏  | 8692/12000 [5:07:59<1:51:05,  2.02s/it]
                                                        

 72%|███████▏  | 8692/12000 [5:07:59<1:51:05,  2.02s/it]
 72%|███████▏  | 8693/12000 [5:08:01<1:43:53,  1.88s/it]
                                                        

 72%|███████▏  | 8693/12000 [5:08:01<1:43:53,  1.88s/it]
 72%|███████▏  | 8694/12000 [5:08:02<1:36:56,  1.76s/it]
                                                        

 72%|███████▏  | 8694/12000 [5:08:02<1:36:56,  1.76s/it]
 72%|███████▏  | 8695/12000 [5:08:04<1:36:51,  1.76s/it]
                                                        

 72%|███████▏  | 8695/12000 [5:08:04<1:36:51,  1.76s/it]
 72%|███████▏  | 8696/12000 [5:08:06<1:33:53,  1.70s/it]
                                                        

 72%|███████▏  | 8696/12000 [5:08:06<1:33:53,  1.70s/it]
 72%|███████▏  | 8697/12000 [5:08:08<1:41:12,  1.84s/it]
                                                        

 72%|███████▏  | 8697/12000 [5:08:08<1:41:12,  1.84s/it]
 72%|███████▏  | 8698/12000 [5:08:10<1:43:13,  1.88s/it]
                                                        

 72%|███████▏  | 8698/12000 [5:08:10<1:43:13,  1.88s/it]
 72%|███████▏  | 8699/12000 [5:08:12<1:44:16,  1.90s/it]
                                                        
{'loss': '0.8396', 'grad_norm': '0.3628', 'learning_rate': '3.965e-05', 'entropy': '0.8058', 'num_tokens': '4.663e+07', 'mean_token_accuracy': '0.8034', 'epoch': '0.07657'}
{'loss': '0.6539', 'grad_norm': '0.2934', 'learning_rate': '3.963e-05', 'entropy': '0.715', 'num_tokens': '4.664e+07', 'mean_token_accuracy': '0.8077', 'epoch': '0.07658'}
{'loss': '0.9015', 'grad_norm': '0.5242', 'learning_rate': '3.961e-05', 'entropy': '0.9232', 'num_tokens': '4.664e+07', 'mean_token_accuracy': '0.772', 'epoch': '0.07659'}
{'loss': '0.7803', 'grad_norm': '0.4634', 'learning_rate': '3.958e-05', 'entropy': '0.8523', 'num_tokens': '4.665e+07', 'mean_token_accuracy': '0.7878', 'epoch': '0.0766'}
{'loss': '1.074', 'grad_norm': '0.413', 'learning_rate': '3.956e-05', 'entropy': '1.078', 'num_tokens': '4.665e+07', 'mean_token_accuracy': '0.7421', 'epoch': '0.0766'}
{'loss': '0.5108', 'grad_norm': '0.2754', 'learning_rate': '3.954e-05', 'entropy': '0.5429', 'num_tokens': '4.666e+07', 'mean_token_accuracy': '0.863', 'epoch': '0.07661'}
{'loss': '0.6658', 'grad_norm': '0.3846', 'learning_rate': '3.952e-05', 'entropy': '0.7131', 'num_tokens': '4.666e+07', 'mean_token_accuracy': '0.8177', 'epoch': '0.07662'}
{'loss': '0.6489', 'grad_norm': '0.4139', 'learning_rate': '3.95e-05', 'entropy': '0.7059', 'num_tokens': '4.667e+07', 'mean_token_accuracy': '0.81', 'epoch': '0.07663'}
{'loss': '0.9555', 'grad_norm': '0.4676', 'learning_rate': '3.947e-05', 'entropy': '0.9833', 'num_tokens': '4.667e+07', 'mean_token_accuracy': '0.7571', 'epoch': '0.07664'}
{'loss': '0.7354', 'grad_norm': '0.4105', 'learning_rate': '3.945e-05', 'entropy': '0.7248', 'num_tokens': '4.668e+07', 'mean_token_accuracy': '0.8232', 'epoch': '0.07665'}
{'loss': '1.151', 'grad_norm': '0.5116', 'learning_rate': '3.943e-05', 'entropy': '1.054', 'num_tokens': '4.668e+07', 'mean_token_accuracy': '0.7381', 'epoch': '0.07666'}
{'loss': '0.9348', 'grad_norm': '0.4478', 'learning_rate': '3.941e-05', 'entropy': '0.8717', 'num_tokens': '4.669e+07', 'mean_token_accuracy': '0.778', 'epoch': '0.07667'}
{'loss': '0.7398', 'grad_norm': '0.4478', 'learning_rate': '3.939e-05', 'entropy': '0.7157', 'num_tokens': '4.67e+07', 'mean_token_accuracy': '0.8199', 'epoch': '0.07667'}
{'loss': '0.9798', 'grad_norm': '0.4989', 'learning_rate': '3.936e-05', 'entropy': '0.9406', 'num_tokens': '4.67e+07', 'mean_token_accuracy': '0.7518', 'epoch': '0.07668'}
{'loss': '0.8459', 'grad_norm': '0.4557', 'learning_rate': '3.934e-05', 'entropy': '0.8485', 'num_tokens': '4.67e+07', 'mean_token_accuracy': '0.7875', 'epoch': '0.07669'}
{'loss': '0.4789', 'grad_norm': '0.3265', 'learning_rate': '3.932e-05', 'entropy': '0.5144', 'num_tokens': '4.671e+07', 'mean_token_accuracy': '0.8642', 'epoch': '0.0767'}
{'loss': '0.4437', 'grad_norm': '0.3045', 'learning_rate': '3.93e-05', 'entropy': '0.4537', 'num_tokens': '4.672e+07', 'mean_token_accuracy': '0.8842', 'epoch': '0.07671'}
{'loss': '0.6526', 'grad_norm': '0.386', 'learning_rate': '3.928e-05', 'entropy': '0.7021', 'num_tokens': '4.672e+07', 'mean_token_accuracy': '0.8204', 'epoch': '0.07672'}
{'loss': '1.071', 'grad_norm': '0.4744', 'learning_rate': '3.926e-05', 'entropy': '1.077', 'num_tokens': '4.672e+07', 'mean_token_accuracy': '0.7419', 'epoch': '0.07673'}
{'loss': '0.7511', 'grad_norm': '0.4545', 'learning_rate': '3.923e-05', 'entropy': '0.7826', 'num_tokens': '4.673e+07', 'mean_token_accuracy': '0.7853', 'epoch': '0.07674'}
{'loss': '0.769', 'grad_norm': '0.3777', 'learning_rate': '3.921e-05', 'entropy': '0.7999', 'num_tokens': '4.674e+07', 'mean_token_accuracy': '0.7981', 'epoch': '0.07675'}
{'loss': '0.8181', 'grad_norm': '0.3938', 'learning_rate': '3.919e-05', 'entropy': '0.8562', 'num_tokens': '4.674e+07', 'mean_token_accuracy': '0.7931', 'epoch': '0.07675'}
{'loss': '0.5211', 'grad_norm': '0.3619', 'learning_rate': '3.917e-05', 'entropy': '0.5652', 'num_tokens': '4.675e+07', 'mean_token_accuracy': '0.855', 'epoch': '0.07676'}
{'loss': '0.666', 'grad_norm': '0.4822', 'learning_rate': '3.915e-05', 'entropy': '0.7099', 'num_tokens': '4.675e+07', 'mean_token_accuracy': '0.8244', 'epoch': '0.07677'}
{'loss': '0.8815', 'grad_norm': '0.4217', 'learning_rate': '3.912e-05', 'entropy': '1.009', 'num_tokens': '4.676e+07', 'mean_token_accuracy': '0.7631', 'epoch': '0.07678'}
{'loss': '0.8708', 'grad_norm': '0.3839', 'learning_rate': '3.91e-05', 'entropy': '0.7464', 'num_tokens': '4.676e+07', 'mean_token_accuracy': '0.8098', 'epoch': '0.07679'}
{'loss': '0.5119', 'grad_norm': '0.3689', 'learning_rate': '3.908e-05', 'entropy': '0.5481', 'num_tokens': '4.677e+07', 'mean_token_accuracy': '0.863', 'epoch': '0.0768'}
{'loss': '0.6142', 'grad_norm': '0.3513', 'learning_rate': '3.906e-05', 'entropy': '0.5598', 'num_tokens': '4.677e+07', 'mean_token_accuracy': '0.8481', 'epoch': '0.07681'}
{'loss': '1.06', 'grad_norm': '0.3381', 'learning_rate': '3.904e-05', 'entropy': '0.9802', 'num_tokens': '4.678e+07', 'mean_token_accuracy': '0.7597', 'epoch': '0.07682'}
{'loss': '0.7474', 'grad_norm': '0.4865', 'learning_rate': '3.901e-05', 'entropy': '0.7908', 'num_tokens': '4.678e+07', 'mean_token_accuracy': '0.804', 'epoch': '0.07683'}
{'loss': '1.059', 'grad_norm': '0.3803', 'learning_rate': '3.899e-05', 'entropy': '1.065', 'num_tokens': '4.679e+07', 'mean_token_accuracy': '0.7386', 'epoch': '0.07683'}
{'loss': '0.8093', 'grad_norm': '0.5482', 'learning_rate': '3.897e-05', 'entropy': '0.7967', 'num_tokens': '4.679e+07', 'mean_token_accuracy': '0.8038', 'epoch': '0.07684'}
{'loss': '1.172', 'grad_norm': '0.4762', 'learning_rate': '3.895e-05', 'entropy': '1.133', 'num_tokens': '4.68e+07', 'mean_token_accuracy': '0.7101', 'epoch': '0.07685'}
{'loss': '0.8881', 'grad_norm': '0.4637', 'learning_rate': '3.893e-05', 'entropy': '0.8516', 'num_tokens': '4.681e+07', 'mean_token_accuracy': '0.7849', 'epoch': '0.07686'}
{'loss': '0.8604', 'grad_norm': '0.4195', 'learning_rate': '3.891e-05', 'entropy': '0.8881', 'num_tokens': '4.681e+07', 'mean_token_accuracy': '0.7887', 'epoch': '0.07687'}
{'loss': '0.7755', 'grad_norm': '0.3803', 'learning_rate': '3.888e-05', 'entropy': '0.7932', 'num_tokens': '4.682e+07', 'mean_token_accuracy': '0.8026', 'epoch': '0.07688'}
{'loss': '0.7284', 'grad_norm': '0.3955', 'learning_rate': '3.886e-05', 'entropy': '0.7716', 'num_tokens': '4.682e+07', 'mean_token_accuracy': '0.7851', 'epoch': '0.07689'}
{'loss': '0.929', 'grad_norm': '0.5245', 'learning_rate': '3.884e-05', 'entropy': '0.8447', 'num_tokens': '4.682e+07', 'mean_token_accuracy': '0.7817', 'epoch': '0.0769'}
{'loss': '1.085', 'grad_norm': '0.4168', 'learning_rate': '3.882e-05', 'entropy': '1.087', 'num_tokens': '4.683e+07', 'mean_token_accuracy': '0.7504', 'epoch': '0.07691'}
{'loss': '0.6348', 'grad_norm': '0.3545', 'learning_rate': '3.88e-05', 'entropy': '0.7212', 'num_tokens': '4.683e+07', 'mean_token_accuracy': '0.8163', 'epoch': '0.07691'}
{'loss': '0.8207', 'grad_norm': '0.4112', 'learning_rate': '3.877e-05', 'entropy': '0.7267', 'num_tokens': '4.684e+07', 'mean_token_accuracy': '0.8068', 'epoch': '0.07692'}
{'loss': '0.7163', 'grad_norm': '0.3983', 'learning_rate': '3.875e-05', 'entropy': '0.733', 'num_tokens': '4.685e+07', 'mean_token_accuracy': '0.7994', 'epoch': '0.07693'}
{'loss': '0.6842', 'grad_norm': '0.4471', 'learning_rate': '3.873e-05', 'entropy': '0.6948', 'num_tokens': '4.685e+07', 'mean_token_accuracy': '0.8131', 'epoch': '0.07694'}
{'loss': '1.088', 'grad_norm': '0.4338', 'learning_rate': '3.871e-05', 'entropy': '1.006', 'num_tokens': '4.686e+07', 'mean_token_accuracy': '0.7483', 'epoch': '0.07695'}
{'loss': '0.7125', 'grad_norm': '0.3985', 'learning_rate': '3.869e-05', 'entropy': '0.6321', 'num_tokens': '4.686e+07', 'mean_token_accuracy': '0.8326', 'epoch': '0.07696'}
{'loss': '1.574', 'grad_norm': '0.5294', 'learning_rate': '3.867e-05', 'entropy': '1.565', 'num_tokens': '4.687e+07', 'mean_token_accuracy': '0.6415', 'epoch': '0.07697'}
{'loss': '0.9025', 'grad_norm': '0.489', 'learning_rate': '3.864e-05', 'entropy': '0.8639', 'num_tokens': '4.687e+07', 'mean_token_accuracy': '0.7892', 'epoch': '0.07698'}

 72%|███████▏  | 8699/12000 [5:08:12<1:44:16,  1.90s/it]
 72%|███████▎  | 8700/12000 [5:08:17<2:32:48,  2.78s/it]
                                                        

 72%|███████▎  | 8700/12000 [5:08:17<2:32:48,  2.78s/it]
 73%|███████▎  | 8701/12000 [5:08:21<2:58:13,  3.24s/it]
                                                        

 73%|███████▎  | 8701/12000 [5:08:21<2:58:13,  3.24s/it]
 73%|███████▎  | 8702/12000 [5:08:23<2:36:52,  2.85s/it]
                                                        

 73%|███████▎  | 8702/12000 [5:08:23<2:36:52,  2.85s/it]
 73%|███████▎  | 8703/12000 [5:08:25<2:21:01,  2.57s/it]
                                                        

 73%|███████▎  | 8703/12000 [5:08:25<2:21:01,  2.57s/it]
 73%|███████▎  | 8704/12000 [5:08:27<2:08:35,  2.34s/it]
                                                        

 73%|███████▎  | 8704/12000 [5:08:27<2:08:35,  2.34s/it]
 73%|███████▎  | 8705/12000 [5:08:28<1:54:13,  2.08s/it]
                                                        

 73%|███████▎  | 8705/12000 [5:08:28<1:54:13,  2.08s/it]
 73%|███████▎  | 8706/12000 [5:08:30<1:47:42,  1.96s/it]
                                                        

 73%|███████▎  | 8706/12000 [5:08:30<1:47:42,  1.96s/it]
 73%|███████▎  | 8707/12000 [5:08:33<2:14:27,  2.45s/it]
                                                        

 73%|███████▎  | 8707/12000 [5:08:33<2:14:27,  2.45s/it]
 73%|███████▎  | 8708/12000 [5:08:36<2:23:00,  2.61s/it]
                                                        

 73%|███████▎  | 8708/12000 [5:08:36<2:23:00,  2.61s/it]
 73%|███████▎  | 8709/12000 [5:08:38<2:07:19,  2.32s/it]
                                                        

 73%|███████▎  | 8709/12000 [5:08:38<2:07:19,  2.32s/it]
 73%|███████▎  | 8710/12000 [5:08:39<1:51:51,  2.04s/it]
                                                        

 73%|███████▎  | 8710/12000 [5:08:39<1:51:51,  2.04s/it]
 73%|███████▎  | 8711/12000 [5:08:41<1:45:15,  1.92s/it]
                                                        

 73%|███████▎  | 8711/12000 [5:08:41<1:45:15,  1.92s/it]
 73%|███████▎  | 8712/12000 [5:08:42<1:36:43,  1.76s/it]
                                                        

 73%|███████▎  | 8712/12000 [5:08:42<1:36:43,  1.76s/it]
 73%|███████▎  | 8713/12000 [5:08:44<1:32:39,  1.69s/it]
                                                        

 73%|███████▎  | 8713/12000 [5:08:44<1:32:39,  1.69s/it]
 73%|███████▎  | 8714/12000 [5:08:45<1:28:36,  1.62s/it]
                                                        

 73%|███████▎  | 8714/12000 [5:08:45<1:28:36,  1.62s/it]
 73%|███████▎  | 8715/12000 [5:08:47<1:32:54,  1.70s/it]
                                                        

 73%|███████▎  | 8715/12000 [5:08:47<1:32:54,  1.70s/it]
 73%|███████▎  | 8716/12000 [5:08:49<1:38:45,  1.80s/it]
                                                        

 73%|███████▎  | 8716/12000 [5:08:49<1:38:45,  1.80s/it]
 73%|███████▎  | 8717/12000 [5:08:51<1:34:05,  1.72s/it]
                                                        

 73%|███████▎  | 8717/12000 [5:08:51<1:34:05,  1.72s/it]
 73%|███████▎  | 8718/12000 [5:08:53<1:35:43,  1.75s/it]
                                                        

 73%|███████▎  | 8718/12000 [5:08:53<1:35:43,  1.75s/it]
 73%|███████▎  | 8719/12000 [5:08:55<1:39:46,  1.82s/it]
                                                        

 73%|███████▎  | 8719/12000 [5:08:55<1:39:46,  1.82s/it]
 73%|███████▎  | 8720/12000 [5:08:56<1:31:49,  1.68s/it]
                                                        

 73%|███████▎  | 8720/12000 [5:08:56<1:31:49,  1.68s/it]
 73%|███████▎  | 8721/12000 [5:08:58<1:29:03,  1.63s/it]
                                                        

 73%|███████▎  | 8721/12000 [5:08:58<1:29:03,  1.63s/it]
 73%|███████▎  | 8722/12000 [5:09:01<2:00:24,  2.20s/it]
                                                        

 73%|███████▎  | 8722/12000 [5:09:01<2:00:24,  2.20s/it]
 73%|███████▎  | 8723/12000 [5:09:03<1:53:58,  2.09s/it]
                                                        

 73%|███████▎  | 8723/12000 [5:09:03<1:53:58,  2.09s/it]
 73%|███████▎  | 8724/12000 [5:09:05<1:48:25,  1.99s/it]
                                                        

 73%|███████▎  | 8724/12000 [5:09:05<1:48:25,  1.99s/it]
 73%|███████▎  | 8725/12000 [5:09:07<1:53:02,  2.07s/it]
                                                        

 73%|███████▎  | 8725/12000 [5:09:07<1:53:02,  2.07s/it]
 73%|███████▎  | 8726/12000 [5:09:08<1:40:49,  1.85s/it]
                                                        

 73%|███████▎  | 8726/12000 [5:09:08<1:40:49,  1.85s/it]
 73%|███████▎  | 8727/12000 [5:09:10<1:34:19,  1.73s/it]
                                                        

 73%|███████▎  | 8727/12000 [5:09:10<1:34:19,  1.73s/it]
 73%|███████▎  | 8728/12000 [5:09:12<1:37:05,  1.78s/it]
                                                        

 73%|███████▎  | 8728/12000 [5:09:12<1:37:05,  1.78s/it]
 73%|███████▎  | 8729/12000 [5:09:13<1:31:44,  1.68s/it]
                                                        

 73%|███████▎  | 8729/12000 [5:09:13<1:31:44,  1.68s/it]
 73%|███████▎  | 8730/12000 [5:09:15<1:32:35,  1.70s/it]
                                                        

 73%|███████▎  | 8730/12000 [5:09:15<1:32:35,  1.70s/it]
 73%|███████▎  | 8731/12000 [5:09:16<1:29:39,  1.65s/it]
                                                        

 73%|███████▎  | 8731/12000 [5:09:16<1:29:39,  1.65s/it]
 73%|███████▎  | 8732/12000 [5:09:19<1:41:23,  1.86s/it]
                                                        

 73%|███████▎  | 8732/12000 [5:09:19<1:41:23,  1.86s/it]
 73%|███████▎  | 8733/12000 [5:09:20<1:34:00,  1.73s/it]
                                                        

 73%|███████▎  | 8733/12000 [5:09:20<1:34:00,  1.73s/it]
 73%|███████▎  | 8734/12000 [5:09:22<1:35:37,  1.76s/it]
                                                        

 73%|███████▎  | 8734/12000 [5:09:22<1:35:37,  1.76s/it]
 73%|███████▎  | 8735/12000 [5:09:24<1:47:36,  1.98s/it]
                                                        

 73%|███████▎  | 8735/12000 [5:09:24<1:47:36,  1.98s/it]
 73%|███████▎  | 8736/12000 [5:09:27<1:53:04,  2.08s/it]
                                                        

 73%|███████▎  | 8736/12000 [5:09:27<1:53:04,  2.08s/it]
 73%|███████▎  | 8737/12000 [5:09:28<1:43:14,  1.90s/it]
                                                        

 73%|███████▎  | 8737/12000 [5:09:28<1:43:14,  1.90s/it]
 73%|███████▎  | 8738/12000 [5:09:30<1:48:18,  1.99s/it]
                                                        

 73%|███████▎  | 8738/12000 [5:09:30<1:48:18,  1.99s/it]
 73%|███████▎  | 8739/12000 [5:09:32<1:44:45,  1.93s/it]
                                                        

 73%|███████▎  | 8739/12000 [5:09:32<1:44:45,  1.93s/it]
 73%|███████▎  | 8740/12000 [5:09:35<1:54:00,  2.10s/it]
                                                        

 73%|███████▎  | 8740/12000 [5:09:35<1:54:00,  2.10s/it]
 73%|███████▎  | 8741/12000 [5:09:38<2:19:54,  2.58s/it]
                                                        

 73%|███████▎  | 8741/12000 [5:09:38<2:19:54,  2.58s/it]
 73%|███████▎  | 8742/12000 [5:09:40<2:02:24,  2.25s/it]
                                                        

 73%|███████▎  | 8742/12000 [5:09:40<2:02:24,  2.25s/it]
 73%|███████▎  | 8743/12000 [5:09:41<1:47:44,  1.98s/it]
                                                        

 73%|███████▎  | 8743/12000 [5:09:41<1:47:44,  1.98s/it]
 73%|███████▎  | 8744/12000 [5:09:45<2:12:20,  2.44s/it]
                                                        

 73%|███████▎  | 8744/12000 [5:09:45<2:12:20,  2.44s/it]
 73%|███████▎  | 8745/12000 [5:09:51<3:19:17,  3.67s/it]
                                                        

 73%|███████▎  | 8745/12000 [5:09:51<3:19:17,  3.67s/it]
 73%|███████▎  | 8746/12000 [5:09:54<3:08:09,  3.47s/it]
                                                        
{'loss': '0.7387', 'grad_norm': '0.5317', 'learning_rate': '3.862e-05', 'entropy': '0.7383', 'num_tokens': '4.688e+07', 'mean_token_accuracy': '0.8071', 'epoch': '0.07698'}
{'loss': '0.6567', 'grad_norm': '0.3178', 'learning_rate': '3.86e-05', 'entropy': '0.7419', 'num_tokens': '4.689e+07', 'mean_token_accuracy': '0.8077', 'epoch': '0.07699'}
{'loss': '0.8788', 'grad_norm': '0.4139', 'learning_rate': '3.858e-05', 'entropy': '0.9169', 'num_tokens': '4.689e+07', 'mean_token_accuracy': '0.7795', 'epoch': '0.077'}
{'loss': '0.6124', 'grad_norm': '0.3217', 'learning_rate': '3.856e-05', 'entropy': '0.55', 'num_tokens': '4.69e+07', 'mean_token_accuracy': '0.8535', 'epoch': '0.07701'}
{'loss': '0.7635', 'grad_norm': '0.374', 'learning_rate': '3.854e-05', 'entropy': '0.6555', 'num_tokens': '4.69e+07', 'mean_token_accuracy': '0.8286', 'epoch': '0.07702'}
{'loss': '0.7898', 'grad_norm': '0.3898', 'learning_rate': '3.851e-05', 'entropy': '0.6743', 'num_tokens': '4.691e+07', 'mean_token_accuracy': '0.8297', 'epoch': '0.07703'}
{'loss': '1.008', 'grad_norm': '0.4158', 'learning_rate': '3.849e-05', 'entropy': '1.003', 'num_tokens': '4.691e+07', 'mean_token_accuracy': '0.7517', 'epoch': '0.07704'}
{'loss': '0.8492', 'grad_norm': '0.4735', 'learning_rate': '3.847e-05', 'entropy': '0.8342', 'num_tokens': '4.692e+07', 'mean_token_accuracy': '0.7779', 'epoch': '0.07705'}
{'loss': '1.074', 'grad_norm': '0.3821', 'learning_rate': '3.845e-05', 'entropy': '1.117', 'num_tokens': '4.693e+07', 'mean_token_accuracy': '0.74', 'epoch': '0.07706'}
{'loss': '0.8931', 'grad_norm': '0.409', 'learning_rate': '3.843e-05', 'entropy': '0.8499', 'num_tokens': '4.693e+07', 'mean_token_accuracy': '0.802', 'epoch': '0.07706'}
{'loss': '0.9093', 'grad_norm': '0.4353', 'learning_rate': '3.84e-05', 'entropy': '0.886', 'num_tokens': '4.694e+07', 'mean_token_accuracy': '0.7774', 'epoch': '0.07707'}
{'loss': '0.6337', 'grad_norm': '0.3134', 'learning_rate': '3.838e-05', 'entropy': '0.6722', 'num_tokens': '4.694e+07', 'mean_token_accuracy': '0.8271', 'epoch': '0.07708'}
{'loss': '0.6183', 'grad_norm': '0.3768', 'learning_rate': '3.836e-05', 'entropy': '0.6241', 'num_tokens': '4.695e+07', 'mean_token_accuracy': '0.8313', 'epoch': '0.07709'}
{'loss': '1.155', 'grad_norm': '0.4784', 'learning_rate': '3.834e-05', 'entropy': '1.112', 'num_tokens': '4.695e+07', 'mean_token_accuracy': '0.7283', 'epoch': '0.0771'}
{'loss': '0.8666', 'grad_norm': '0.4085', 'learning_rate': '3.832e-05', 'entropy': '0.8056', 'num_tokens': '4.696e+07', 'mean_token_accuracy': '0.7856', 'epoch': '0.07711'}
{'loss': '1.042', 'grad_norm': '0.5151', 'learning_rate': '3.83e-05', 'entropy': '1.01', 'num_tokens': '4.696e+07', 'mean_token_accuracy': '0.7498', 'epoch': '0.07712'}
{'loss': '0.9101', 'grad_norm': '0.3974', 'learning_rate': '3.827e-05', 'entropy': '0.9012', 'num_tokens': '4.697e+07', 'mean_token_accuracy': '0.7778', 'epoch': '0.07713'}
{'loss': '0.6748', 'grad_norm': '0.3303', 'learning_rate': '3.825e-05', 'entropy': '0.6449', 'num_tokens': '4.697e+07', 'mean_token_accuracy': '0.8231', 'epoch': '0.07714'}
{'loss': '0.81', 'grad_norm': '0.4419', 'learning_rate': '3.823e-05', 'entropy': '0.8486', 'num_tokens': '4.698e+07', 'mean_token_accuracy': '0.783', 'epoch': '0.07714'}
{'loss': '0.8895', 'grad_norm': '0.3845', 'learning_rate': '3.821e-05', 'entropy': '0.8632', 'num_tokens': '4.698e+07', 'mean_token_accuracy': '0.7753', 'epoch': '0.07715'}
{'loss': '1.281', 'grad_norm': '0.4216', 'learning_rate': '3.819e-05', 'entropy': '1.399', 'num_tokens': '4.699e+07', 'mean_token_accuracy': '0.6603', 'epoch': '0.07716'}
{'loss': '0.8714', 'grad_norm': '0.4278', 'learning_rate': '3.817e-05', 'entropy': '0.8912', 'num_tokens': '4.699e+07', 'mean_token_accuracy': '0.7636', 'epoch': '0.07717'}
{'loss': '0.6154', 'grad_norm': '0.3859', 'learning_rate': '3.814e-05', 'entropy': '0.64', 'num_tokens': '4.7e+07', 'mean_token_accuracy': '0.8367', 'epoch': '0.07718'}
{'loss': '0.7738', 'grad_norm': '0.4599', 'learning_rate': '3.812e-05', 'entropy': '0.8395', 'num_tokens': '4.7e+07', 'mean_token_accuracy': '0.7855', 'epoch': '0.07719'}
{'loss': '1.452', 'grad_norm': '0.4355', 'learning_rate': '3.81e-05', 'entropy': '1.21', 'num_tokens': '4.701e+07', 'mean_token_accuracy': '0.7053', 'epoch': '0.0772'}
{'loss': '0.5357', 'grad_norm': '0.3245', 'learning_rate': '3.808e-05', 'entropy': '0.5713', 'num_tokens': '4.701e+07', 'mean_token_accuracy': '0.854', 'epoch': '0.07721'}
{'loss': '0.5419', 'grad_norm': '0.2744', 'learning_rate': '3.806e-05', 'entropy': '0.5753', 'num_tokens': '4.702e+07', 'mean_token_accuracy': '0.8628', 'epoch': '0.07721'}
{'loss': '1.026', 'grad_norm': '0.4978', 'learning_rate': '3.804e-05', 'entropy': '1.069', 'num_tokens': '4.702e+07', 'mean_token_accuracy': '0.732', 'epoch': '0.07722'}
{'loss': '0.9442', 'grad_norm': '0.4318', 'learning_rate': '3.801e-05', 'entropy': '0.903', 'num_tokens': '4.702e+07', 'mean_token_accuracy': '0.7798', 'epoch': '0.07723'}
{'loss': '0.7496', 'grad_norm': '0.4221', 'learning_rate': '3.799e-05', 'entropy': '0.77', 'num_tokens': '4.703e+07', 'mean_token_accuracy': '0.7844', 'epoch': '0.07724'}
{'loss': '0.7609', 'grad_norm': '0.4642', 'learning_rate': '3.797e-05', 'entropy': '0.8308', 'num_tokens': '4.703e+07', 'mean_token_accuracy': '0.7866', 'epoch': '0.07725'}
{'loss': '0.9176', 'grad_norm': '0.4366', 'learning_rate': '3.795e-05', 'entropy': '0.8372', 'num_tokens': '4.704e+07', 'mean_token_accuracy': '0.7924', 'epoch': '0.07726'}
{'loss': '0.8328', 'grad_norm': '0.4735', 'learning_rate': '3.793e-05', 'entropy': '0.8664', 'num_tokens': '4.704e+07', 'mean_token_accuracy': '0.7798', 'epoch': '0.07727'}
{'loss': '1.087', 'grad_norm': '0.4659', 'learning_rate': '3.791e-05', 'entropy': '1.026', 'num_tokens': '4.705e+07', 'mean_token_accuracy': '0.744', 'epoch': '0.07728'}
{'loss': '0.8803', 'grad_norm': '0.3651', 'learning_rate': '3.789e-05', 'entropy': '0.9917', 'num_tokens': '4.705e+07', 'mean_token_accuracy': '0.7644', 'epoch': '0.07729'}
{'loss': '0.636', 'grad_norm': '0.3287', 'learning_rate': '3.786e-05', 'entropy': '0.6851', 'num_tokens': '4.706e+07', 'mean_token_accuracy': '0.8296', 'epoch': '0.07729'}
{'loss': '0.8533', 'grad_norm': '0.4859', 'learning_rate': '3.784e-05', 'entropy': '0.8209', 'num_tokens': '4.707e+07', 'mean_token_accuracy': '0.8095', 'epoch': '0.0773'}
{'loss': '1.069', 'grad_norm': '0.4622', 'learning_rate': '3.782e-05', 'entropy': '0.9036', 'num_tokens': '4.707e+07', 'mean_token_accuracy': '0.7891', 'epoch': '0.07731'}
{'loss': '0.6574', 'grad_norm': '0.3466', 'learning_rate': '3.78e-05', 'entropy': '0.6773', 'num_tokens': '4.708e+07', 'mean_token_accuracy': '0.8188', 'epoch': '0.07732'}
{'loss': '0.8444', 'grad_norm': '0.4682', 'learning_rate': '3.778e-05', 'entropy': '0.8368', 'num_tokens': '4.708e+07', 'mean_token_accuracy': '0.7971', 'epoch': '0.07733'}
{'loss': '0.8236', 'grad_norm': '0.3049', 'learning_rate': '3.776e-05', 'entropy': '0.7404', 'num_tokens': '4.709e+07', 'mean_token_accuracy': '0.8058', 'epoch': '0.07734'}
{'loss': '0.5169', 'grad_norm': '0.2809', 'learning_rate': '3.773e-05', 'entropy': '0.5267', 'num_tokens': '4.709e+07', 'mean_token_accuracy': '0.866', 'epoch': '0.07735'}
{'loss': '0.596', 'grad_norm': '0.3242', 'learning_rate': '3.771e-05', 'entropy': '0.6258', 'num_tokens': '4.71e+07', 'mean_token_accuracy': '0.8353', 'epoch': '0.07736'}
{'loss': '0.7693', 'grad_norm': '0.3981', 'learning_rate': '3.769e-05', 'entropy': '0.8044', 'num_tokens': '4.711e+07', 'mean_token_accuracy': '0.7814', 'epoch': '0.07737'}
{'loss': '1.108', 'grad_norm': '0.4463', 'learning_rate': '3.767e-05', 'entropy': '1.109', 'num_tokens': '4.711e+07', 'mean_token_accuracy': '0.7273', 'epoch': '0.07737'}
{'loss': '0.7998', 'grad_norm': '0.3957', 'learning_rate': '3.765e-05', 'entropy': '0.812', 'num_tokens': '4.712e+07', 'mean_token_accuracy': '0.7897', 'epoch': '0.07738'}
{'loss': '0.6987', 'grad_norm': '0.3091', 'learning_rate': '3.763e-05', 'entropy': '0.6361', 'num_tokens': '4.713e+07', 'mean_token_accuracy': '0.8368', 'epoch': '0.07739'}

 73%|███████▎  | 8746/12000 [5:09:54<3:08:09,  3.47s/it]
 73%|███████▎  | 8747/12000 [5:09:56<2:41:14,  2.97s/it]
                                                        

 73%|███████▎  | 8747/12000 [5:09:56<2:41:14,  2.97s/it]
 73%|███████▎  | 8748/12000 [5:09:59<2:43:57,  3.02s/it]
                                                        

 73%|███████▎  | 8748/12000 [5:09:59<2:43:57,  3.02s/it]
 73%|███████▎  | 8749/12000 [5:10:01<2:26:17,  2.70s/it]
                                                        

 73%|███████▎  | 8749/12000 [5:10:01<2:26:17,  2.70s/it]
 73%|███████▎  | 8750/12000 [5:10:03<2:05:15,  2.31s/it]
                                                        

 73%|███████▎  | 8750/12000 [5:10:03<2:05:15,  2.31s/it]
 73%|███████▎  | 8751/12000 [5:10:06<2:16:23,  2.52s/it]
                                                        

 73%|███████▎  | 8751/12000 [5:10:06<2:16:23,  2.52s/it]
 73%|███████▎  | 8752/12000 [5:10:07<1:59:14,  2.20s/it]
                                                        

 73%|███████▎  | 8752/12000 [5:10:07<1:59:14,  2.20s/it]
 73%|███████▎  | 8753/12000 [5:10:09<2:02:48,  2.27s/it]
                                                        

 73%|███████▎  | 8753/12000 [5:10:09<2:02:48,  2.27s/it]
 73%|███████▎  | 8754/12000 [5:10:11<1:50:15,  2.04s/it]
                                                        

 73%|███████▎  | 8754/12000 [5:10:11<1:50:15,  2.04s/it]
 73%|███████▎  | 8755/12000 [5:10:14<1:59:05,  2.20s/it]
                                                        

 73%|███████▎  | 8755/12000 [5:10:14<1:59:05,  2.20s/it]
 73%|███████▎  | 8756/12000 [5:10:16<2:02:29,  2.27s/it]
                                                        

 73%|███████▎  | 8756/12000 [5:10:16<2:02:29,  2.27s/it]
 73%|███████▎  | 8757/12000 [5:10:20<2:25:22,  2.69s/it]
                                                        

 73%|███████▎  | 8757/12000 [5:10:20<2:25:22,  2.69s/it]
 73%|███████▎  | 8758/12000 [5:10:21<2:07:09,  2.35s/it]
                                                        

 73%|███████▎  | 8758/12000 [5:10:21<2:07:09,  2.35s/it]
 73%|███████▎  | 8759/12000 [5:10:24<2:07:38,  2.36s/it]
                                                        

 73%|███████▎  | 8759/12000 [5:10:24<2:07:38,  2.36s/it]
 73%|███████▎  | 8760/12000 [5:10:26<2:12:41,  2.46s/it]
                                                        

 73%|███████▎  | 8760/12000 [5:10:26<2:12:41,  2.46s/it]
 73%|███████▎  | 8761/12000 [5:10:29<2:17:21,  2.54s/it]
                                                        

 73%|███████▎  | 8761/12000 [5:10:29<2:17:21,  2.54s/it]
 73%|███████▎  | 8762/12000 [5:10:31<2:06:56,  2.35s/it]
                                                        

 73%|███████▎  | 8762/12000 [5:10:31<2:06:56,  2.35s/it]
 73%|███████▎  | 8763/12000 [5:10:33<1:56:37,  2.16s/it]
                                                        

 73%|███████▎  | 8763/12000 [5:10:33<1:56:37,  2.16s/it]
 73%|███████▎  | 8764/12000 [5:10:34<1:48:19,  2.01s/it]
                                                        

 73%|███████▎  | 8764/12000 [5:10:34<1:48:19,  2.01s/it]
 73%|███████▎  | 8765/12000 [5:10:37<1:55:31,  2.14s/it]
                                                        

 73%|███████▎  | 8765/12000 [5:10:37<1:55:31,  2.14s/it]
 73%|███████▎  | 8766/12000 [5:10:40<2:20:15,  2.60s/it]
                                                        

 73%|███████▎  | 8766/12000 [5:10:40<2:20:15,  2.60s/it]
 73%|███████▎  | 8767/12000 [5:10:42<1:59:55,  2.23s/it]
                                                        

 73%|███████▎  | 8767/12000 [5:10:42<1:59:55,  2.23s/it]
 73%|███████▎  | 8768/12000 [5:10:43<1:49:09,  2.03s/it]
                                                        

 73%|███████▎  | 8768/12000 [5:10:43<1:49:09,  2.03s/it]
 73%|███████▎  | 8769/12000 [5:10:45<1:49:08,  2.03s/it]
                                                        

 73%|███████▎  | 8769/12000 [5:10:45<1:49:08,  2.03s/it]
 73%|███████▎  | 8770/12000 [5:10:47<1:45:22,  1.96s/it]
                                                        

 73%|███████▎  | 8770/12000 [5:10:47<1:45:22,  1.96s/it]
 73%|███████▎  | 8771/12000 [5:10:49<1:42:26,  1.90s/it]
                                                        

 73%|███████▎  | 8771/12000 [5:10:49<1:42:26,  1.90s/it]
 73%|███████▎  | 8772/12000 [5:10:51<1:43:49,  1.93s/it]
                                                        

 73%|███████▎  | 8772/12000 [5:10:51<1:43:49,  1.93s/it]
 73%|███████▎  | 8773/12000 [5:10:52<1:35:22,  1.77s/it]
                                                        

 73%|███████▎  | 8773/12000 [5:10:52<1:35:22,  1.77s/it]
 73%|███████▎  | 8774/12000 [5:10:54<1:34:13,  1.75s/it]
                                                        

 73%|███████▎  | 8774/12000 [5:10:54<1:34:13,  1.75s/it]
 73%|███████▎  | 8775/12000 [5:10:56<1:31:23,  1.70s/it]
                                                        

 73%|███████▎  | 8775/12000 [5:10:56<1:31:23,  1.70s/it]
 73%|███████▎  | 8776/12000 [5:10:57<1:30:57,  1.69s/it]
                                                        

 73%|███████▎  | 8776/12000 [5:10:57<1:30:57,  1.69s/it]
 73%|███████▎  | 8777/12000 [5:11:00<1:54:04,  2.12s/it]
                                                        

 73%|███████▎  | 8777/12000 [5:11:00<1:54:04,  2.12s/it]
 73%|███████▎  | 8778/12000 [5:11:02<1:48:58,  2.03s/it]
                                                        

 73%|███████▎  | 8778/12000 [5:11:02<1:48:58,  2.03s/it]
 73%|███████▎  | 8779/12000 [5:11:05<1:53:04,  2.11s/it]
                                                        

 73%|███████▎  | 8779/12000 [5:11:05<1:53:04,  2.11s/it]
 73%|███████▎  | 8780/12000 [5:11:06<1:47:30,  2.00s/it]
                                                        

 73%|███████▎  | 8780/12000 [5:11:06<1:47:30,  2.00s/it]
 73%|███████▎  | 8781/12000 [5:11:08<1:38:54,  1.84s/it]
                                                        

 73%|███████▎  | 8781/12000 [5:11:08<1:38:54,  1.84s/it]
 73%|███████▎  | 8782/12000 [5:11:09<1:35:01,  1.77s/it]
                                                        

 73%|███████▎  | 8782/12000 [5:11:09<1:35:01,  1.77s/it]
 73%|███████▎  | 8783/12000 [5:11:12<1:43:41,  1.93s/it]
                                                        

 73%|███████▎  | 8783/12000 [5:11:12<1:43:41,  1.93s/it]
 73%|███████▎  | 8784/12000 [5:11:15<2:07:09,  2.37s/it]
                                                        

 73%|███████▎  | 8784/12000 [5:11:15<2:07:09,  2.37s/it]
 73%|███████▎  | 8785/12000 [5:11:17<1:57:32,  2.19s/it]
                                                        

 73%|███████▎  | 8785/12000 [5:11:17<1:57:32,  2.19s/it]
 73%|███████▎  | 8786/12000 [5:11:18<1:48:38,  2.03s/it]
                                                        

 73%|███████▎  | 8786/12000 [5:11:18<1:48:38,  2.03s/it]
 73%|███████▎  | 8787/12000 [5:11:20<1:45:47,  1.98s/it]
                                                        

 73%|███████▎  | 8787/12000 [5:11:20<1:45:47,  1.98s/it]
 73%|███████▎  | 8788/12000 [5:11:24<2:09:13,  2.41s/it]
                                                        

 73%|███████▎  | 8788/12000 [5:11:24<2:09:13,  2.41s/it]
 73%|███████▎  | 8789/12000 [5:11:25<1:54:42,  2.14s/it]
                                                        

 73%|███████▎  | 8789/12000 [5:11:25<1:54:42,  2.14s/it]
 73%|███████▎  | 8790/12000 [5:11:27<1:42:29,  1.92s/it]
                                                        

 73%|███████▎  | 8790/12000 [5:11:27<1:42:29,  1.92s/it]
 73%|███████▎  | 8791/12000 [5:11:28<1:32:36,  1.73s/it]
                                                        

 73%|███████▎  | 8791/12000 [5:11:28<1:32:36,  1.73s/it]
 73%|███████▎  | 8792/12000 [5:11:30<1:30:02,  1.68s/it]
                                                        

 73%|███████▎  | 8792/12000 [5:11:30<1:30:02,  1.68s/it]
 73%|███████▎  | 8793/12000 [5:11:31<1:27:02,  1.63s/it]
                                                        
{'loss': '0.6019', 'grad_norm': '0.2503', 'learning_rate': '3.76e-05', 'entropy': '0.6393', 'num_tokens': '4.713e+07', 'mean_token_accuracy': '0.8345', 'epoch': '0.0774'}
{'loss': '1.148', 'grad_norm': '0.4779', 'learning_rate': '3.758e-05', 'entropy': '1.086', 'num_tokens': '4.714e+07', 'mean_token_accuracy': '0.7389', 'epoch': '0.07741'}
{'loss': '0.8182', 'grad_norm': '0.35', 'learning_rate': '3.756e-05', 'entropy': '0.8035', 'num_tokens': '4.715e+07', 'mean_token_accuracy': '0.8098', 'epoch': '0.07742'}
{'loss': '1.082', 'grad_norm': '0.443', 'learning_rate': '3.754e-05', 'entropy': '1.072', 'num_tokens': '4.715e+07', 'mean_token_accuracy': '0.7418', 'epoch': '0.07743'}
{'loss': '0.7874', 'grad_norm': '0.4139', 'learning_rate': '3.752e-05', 'entropy': '0.7745', 'num_tokens': '4.716e+07', 'mean_token_accuracy': '0.7944', 'epoch': '0.07744'}
{'loss': '0.8059', 'grad_norm': '0.4218', 'learning_rate': '3.75e-05', 'entropy': '0.8051', 'num_tokens': '4.716e+07', 'mean_token_accuracy': '0.7968', 'epoch': '0.07744'}
{'loss': '0.5452', 'grad_norm': '0.394', 'learning_rate': '3.748e-05', 'entropy': '0.6203', 'num_tokens': '4.716e+07', 'mean_token_accuracy': '0.8463', 'epoch': '0.07745'}
{'loss': '0.9634', 'grad_norm': '0.4755', 'learning_rate': '3.745e-05', 'entropy': '0.9337', 'num_tokens': '4.717e+07', 'mean_token_accuracy': '0.7675', 'epoch': '0.07746'}
{'loss': '0.7134', 'grad_norm': '0.4069', 'learning_rate': '3.743e-05', 'entropy': '0.7136', 'num_tokens': '4.718e+07', 'mean_token_accuracy': '0.8169', 'epoch': '0.07747'}
{'loss': '0.9837', 'grad_norm': '0.3251', 'learning_rate': '3.741e-05', 'entropy': '1.041', 'num_tokens': '4.718e+07', 'mean_token_accuracy': '0.7473', 'epoch': '0.07748'}
{'loss': '0.8439', 'grad_norm': '0.3985', 'learning_rate': '3.739e-05', 'entropy': '0.8799', 'num_tokens': '4.719e+07', 'mean_token_accuracy': '0.7716', 'epoch': '0.07749'}
{'loss': '0.9476', 'grad_norm': '0.3942', 'learning_rate': '3.737e-05', 'entropy': '0.9401', 'num_tokens': '4.72e+07', 'mean_token_accuracy': '0.7863', 'epoch': '0.0775'}
{'loss': '0.8328', 'grad_norm': '0.4736', 'learning_rate': '3.735e-05', 'entropy': '0.8991', 'num_tokens': '4.72e+07', 'mean_token_accuracy': '0.7839', 'epoch': '0.07751'}
{'loss': '1.119', 'grad_norm': '0.4251', 'learning_rate': '3.733e-05', 'entropy': '0.992', 'num_tokens': '4.721e+07', 'mean_token_accuracy': '0.7551', 'epoch': '0.07752'}
{'loss': '1.487', 'grad_norm': '0.4734', 'learning_rate': '3.73e-05', 'entropy': '1.058', 'num_tokens': '4.722e+07', 'mean_token_accuracy': '0.7756', 'epoch': '0.07752'}
{'loss': '0.9349', 'grad_norm': '0.4029', 'learning_rate': '3.728e-05', 'entropy': '0.9369', 'num_tokens': '4.722e+07', 'mean_token_accuracy': '0.7654', 'epoch': '0.07753'}
{'loss': '0.5994', 'grad_norm': '0.3936', 'learning_rate': '3.726e-05', 'entropy': '0.5721', 'num_tokens': '4.723e+07', 'mean_token_accuracy': '0.8414', 'epoch': '0.07754'}
{'loss': '0.8838', 'grad_norm': '0.4588', 'learning_rate': '3.724e-05', 'entropy': '1.051', 'num_tokens': '4.723e+07', 'mean_token_accuracy': '0.7446', 'epoch': '0.07755'}
{'loss': '0.4939', 'grad_norm': '0.3374', 'learning_rate': '3.722e-05', 'entropy': '0.5354', 'num_tokens': '4.724e+07', 'mean_token_accuracy': '0.868', 'epoch': '0.07756'}
{'loss': '0.952', 'grad_norm': '0.427', 'learning_rate': '3.72e-05', 'entropy': '0.831', 'num_tokens': '4.725e+07', 'mean_token_accuracy': '0.7934', 'epoch': '0.07757'}
{'loss': '0.9156', 'grad_norm': '0.3872', 'learning_rate': '3.718e-05', 'entropy': '0.8172', 'num_tokens': '4.725e+07', 'mean_token_accuracy': '0.8024', 'epoch': '0.07758'}
{'loss': '0.6323', 'grad_norm': '0.3786', 'learning_rate': '3.715e-05', 'entropy': '0.7331', 'num_tokens': '4.725e+07', 'mean_token_accuracy': '0.8159', 'epoch': '0.07759'}
{'loss': '0.6404', 'grad_norm': '0.369', 'learning_rate': '3.713e-05', 'entropy': '0.688', 'num_tokens': '4.726e+07', 'mean_token_accuracy': '0.825', 'epoch': '0.0776'}
{'loss': '0.8602', 'grad_norm': '0.4291', 'learning_rate': '3.711e-05', 'entropy': '0.8627', 'num_tokens': '4.727e+07', 'mean_token_accuracy': '0.7793', 'epoch': '0.0776'}
{'loss': '0.9725', 'grad_norm': '0.4456', 'learning_rate': '3.709e-05', 'entropy': '0.9702', 'num_tokens': '4.727e+07', 'mean_token_accuracy': '0.7625', 'epoch': '0.07761'}
{'loss': '0.9612', 'grad_norm': '0.4081', 'learning_rate': '3.707e-05', 'entropy': '0.9114', 'num_tokens': '4.728e+07', 'mean_token_accuracy': '0.7759', 'epoch': '0.07762'}
{'loss': '0.7226', 'grad_norm': '0.3549', 'learning_rate': '3.705e-05', 'entropy': '0.7774', 'num_tokens': '4.728e+07', 'mean_token_accuracy': '0.7942', 'epoch': '0.07763'}
{'loss': '0.9037', 'grad_norm': '0.4109', 'learning_rate': '3.703e-05', 'entropy': '0.8509', 'num_tokens': '4.729e+07', 'mean_token_accuracy': '0.7859', 'epoch': '0.07764'}
{'loss': '0.972', 'grad_norm': '0.3669', 'learning_rate': '3.7e-05', 'entropy': '0.9687', 'num_tokens': '4.729e+07', 'mean_token_accuracy': '0.7638', 'epoch': '0.07765'}
{'loss': '0.7959', 'grad_norm': '0.4311', 'learning_rate': '3.698e-05', 'entropy': '0.9312', 'num_tokens': '4.73e+07', 'mean_token_accuracy': '0.7714', 'epoch': '0.07766'}
{'loss': '1.082', 'grad_norm': '0.4504', 'learning_rate': '3.696e-05', 'entropy': '1.063', 'num_tokens': '4.73e+07', 'mean_token_accuracy': '0.7478', 'epoch': '0.07767'}
{'loss': '0.8438', 'grad_norm': '0.4254', 'learning_rate': '3.694e-05', 'entropy': '0.8815', 'num_tokens': '4.731e+07', 'mean_token_accuracy': '0.7713', 'epoch': '0.07767'}
{'loss': '0.5447', 'grad_norm': '0.3473', 'learning_rate': '3.692e-05', 'entropy': '0.5458', 'num_tokens': '4.731e+07', 'mean_token_accuracy': '0.8568', 'epoch': '0.07768'}
{'loss': '0.9574', 'grad_norm': '0.4005', 'learning_rate': '3.69e-05', 'entropy': '1.034', 'num_tokens': '4.732e+07', 'mean_token_accuracy': '0.7536', 'epoch': '0.07769'}
{'loss': '1.034', 'grad_norm': '0.4077', 'learning_rate': '3.688e-05', 'entropy': '0.9438', 'num_tokens': '4.732e+07', 'mean_token_accuracy': '0.7751', 'epoch': '0.0777'}
{'loss': '0.7491', 'grad_norm': '0.4098', 'learning_rate': '3.685e-05', 'entropy': '0.757', 'num_tokens': '4.733e+07', 'mean_token_accuracy': '0.8085', 'epoch': '0.07771'}
{'loss': '0.6361', 'grad_norm': '0.3789', 'learning_rate': '3.683e-05', 'entropy': '0.615', 'num_tokens': '4.733e+07', 'mean_token_accuracy': '0.8317', 'epoch': '0.07772'}
{'loss': '0.5981', 'grad_norm': '0.3866', 'learning_rate': '3.681e-05', 'entropy': '0.6458', 'num_tokens': '4.734e+07', 'mean_token_accuracy': '0.8419', 'epoch': '0.07773'}
{'loss': '0.6704', 'grad_norm': '0.3234', 'learning_rate': '3.679e-05', 'entropy': '0.7382', 'num_tokens': '4.735e+07', 'mean_token_accuracy': '0.821', 'epoch': '0.07774'}
{'loss': '0.6621', 'grad_norm': '0.4154', 'learning_rate': '3.677e-05', 'entropy': '0.6809', 'num_tokens': '4.735e+07', 'mean_token_accuracy': '0.8288', 'epoch': '0.07775'}
{'loss': '0.7606', 'grad_norm': '0.4099', 'learning_rate': '3.675e-05', 'entropy': '0.7762', 'num_tokens': '4.736e+07', 'mean_token_accuracy': '0.7977', 'epoch': '0.07775'}
{'loss': '0.9574', 'grad_norm': '0.3645', 'learning_rate': '3.673e-05', 'entropy': '0.9702', 'num_tokens': '4.736e+07', 'mean_token_accuracy': '0.7411', 'epoch': '0.07776'}
{'loss': '0.6682', 'grad_norm': '0.3368', 'learning_rate': '3.67e-05', 'entropy': '0.6996', 'num_tokens': '4.736e+07', 'mean_token_accuracy': '0.8176', 'epoch': '0.07777'}
{'loss': '0.6463', 'grad_norm': '0.3417', 'learning_rate': '3.668e-05', 'entropy': '0.6883', 'num_tokens': '4.737e+07', 'mean_token_accuracy': '0.817', 'epoch': '0.07778'}
{'loss': '0.7555', 'grad_norm': '0.3572', 'learning_rate': '3.666e-05', 'entropy': '0.7808', 'num_tokens': '4.737e+07', 'mean_token_accuracy': '0.8035', 'epoch': '0.07779'}
{'loss': '0.8862', 'grad_norm': '0.4111', 'learning_rate': '3.664e-05', 'entropy': '0.9278', 'num_tokens': '4.738e+07', 'mean_token_accuracy': '0.7791', 'epoch': '0.0778'}
{'loss': '0.8053', 'grad_norm': '0.4949', 'learning_rate': '3.662e-05', 'entropy': '0.8131', 'num_tokens': '4.738e+07', 'mean_token_accuracy': '0.7878', 'epoch': '0.07781'}

 73%|███████▎  | 8793/12000 [5:11:31<1:27:02,  1.63s/it]
 73%|███████▎  | 8794/12000 [5:11:33<1:36:43,  1.81s/it]
                                                        

 73%|███████▎  | 8794/12000 [5:11:33<1:36:43,  1.81s/it]
 73%|███████▎  | 8795/12000 [5:11:36<1:50:28,  2.07s/it]
                                                        

 73%|███████▎  | 8795/12000 [5:11:36<1:50:28,  2.07s/it]
 73%|███████▎  | 8796/12000 [5:11:38<1:42:07,  1.91s/it]
                                                        

 73%|███████▎  | 8796/12000 [5:11:38<1:42:07,  1.91s/it]
 73%|███████▎  | 8797/12000 [5:11:39<1:38:03,  1.84s/it]
                                                        

 73%|███████▎  | 8797/12000 [5:11:39<1:38:03,  1.84s/it]
 73%|███████▎  | 8798/12000 [5:11:41<1:32:33,  1.73s/it]
                                                        

 73%|███████▎  | 8798/12000 [5:11:41<1:32:33,  1.73s/it]
 73%|███████▎  | 8799/12000 [5:11:43<1:41:00,  1.89s/it]
                                                        

 73%|███████▎  | 8799/12000 [5:11:43<1:41:00,  1.89s/it]
 73%|███████▎  | 8800/12000 [5:11:44<1:34:40,  1.78s/it]
                                                        

 73%|███████▎  | 8800/12000 [5:11:44<1:34:40,  1.78s/it]
 73%|███████▎  | 8801/12000 [5:11:47<1:51:55,  2.10s/it]
                                                        

 73%|███████▎  | 8801/12000 [5:11:47<1:51:55,  2.10s/it]
 73%|███████▎  | 8802/12000 [5:11:49<1:44:53,  1.97s/it]
                                                        

 73%|███████▎  | 8802/12000 [5:11:49<1:44:53,  1.97s/it]
 73%|███████▎  | 8803/12000 [5:11:51<1:43:57,  1.95s/it]
                                                        

 73%|███████▎  | 8803/12000 [5:11:51<1:43:57,  1.95s/it]
 73%|███████▎  | 8804/12000 [5:11:53<1:44:38,  1.96s/it]
                                                        

 73%|███████▎  | 8804/12000 [5:11:53<1:44:38,  1.96s/it]
 73%|███████▎  | 8805/12000 [5:11:55<1:42:21,  1.92s/it]
                                                        

 73%|███████▎  | 8805/12000 [5:11:55<1:42:21,  1.92s/it]
 73%|███████▎  | 8806/12000 [5:11:56<1:37:37,  1.83s/it]
                                                        

 73%|███████▎  | 8806/12000 [5:11:56<1:37:37,  1.83s/it]
 73%|███████▎  | 8807/12000 [5:11:58<1:38:40,  1.85s/it]
                                                        

 73%|███████▎  | 8807/12000 [5:11:58<1:38:40,  1.85s/it]
 73%|███████▎  | 8808/12000 [5:12:01<1:49:22,  2.06s/it]
                                                        

 73%|███████▎  | 8808/12000 [5:12:01<1:49:22,  2.06s/it]
 73%|███████▎  | 8809/12000 [5:12:02<1:41:18,  1.91s/it]
                                                        

 73%|███████▎  | 8809/12000 [5:12:02<1:41:18,  1.91s/it]
 73%|███████▎  | 8810/12000 [5:12:07<2:21:46,  2.67s/it]
                                                        

 73%|███████▎  | 8810/12000 [5:12:07<2:21:46,  2.67s/it]
 73%|███████▎  | 8811/12000 [5:12:08<2:04:24,  2.34s/it]
                                                        

 73%|███████▎  | 8811/12000 [5:12:08<2:04:24,  2.34s/it]
 73%|███████▎  | 8812/12000 [5:12:10<1:55:52,  2.18s/it]
                                                        

 73%|███████▎  | 8812/12000 [5:12:10<1:55:52,  2.18s/it]
 73%|███████▎  | 8813/12000 [5:12:12<1:45:37,  1.99s/it]
                                                        

 73%|███████▎  | 8813/12000 [5:12:12<1:45:37,  1.99s/it]
 73%|███████▎  | 8814/12000 [5:12:13<1:41:28,  1.91s/it]
                                                        

 73%|███████▎  | 8814/12000 [5:12:13<1:41:28,  1.91s/it]
 73%|███████▎  | 8815/12000 [5:12:16<1:56:22,  2.19s/it]
                                                        

 73%|███████▎  | 8815/12000 [5:12:16<1:56:22,  2.19s/it]
 73%|███████▎  | 8816/12000 [5:12:18<1:50:07,  2.08s/it]
                                                        

 73%|███████▎  | 8816/12000 [5:12:18<1:50:07,  2.08s/it]
 73%|███████▎  | 8817/12000 [5:12:20<1:42:48,  1.94s/it]
                                                        

 73%|███████▎  | 8817/12000 [5:12:20<1:42:48,  1.94s/it]
 73%|███████▎  | 8818/12000 [5:12:22<1:54:56,  2.17s/it]
                                                        

 73%|███████▎  | 8818/12000 [5:12:22<1:54:56,  2.17s/it]
 73%|███████▎  | 8819/12000 [5:12:24<1:45:08,  1.98s/it]
                                                        

 73%|███████▎  | 8819/12000 [5:12:24<1:45:08,  1.98s/it]
 74%|███████▎  | 8820/12000 [5:12:26<1:48:15,  2.04s/it]
                                                        

 74%|███████▎  | 8820/12000 [5:12:26<1:48:15,  2.04s/it]
 74%|███████▎  | 8821/12000 [5:12:28<1:38:32,  1.86s/it]
                                                        

 74%|███████▎  | 8821/12000 [5:12:28<1:38:32,  1.86s/it]
 74%|███████▎  | 8822/12000 [5:12:29<1:32:49,  1.75s/it]
                                                        

 74%|███████▎  | 8822/12000 [5:12:29<1:32:49,  1.75s/it]
 74%|███████▎  | 8823/12000 [5:12:31<1:36:18,  1.82s/it]
                                                        

 74%|███████▎  | 8823/12000 [5:12:31<1:36:18,  1.82s/it]
 74%|███████▎  | 8824/12000 [5:12:33<1:33:50,  1.77s/it]
                                                        

 74%|███████▎  | 8824/12000 [5:12:33<1:33:50,  1.77s/it]
 74%|███████▎  | 8825/12000 [5:12:34<1:28:40,  1.68s/it]
                                                        

 74%|███████▎  | 8825/12000 [5:12:34<1:28:40,  1.68s/it]
 74%|███████▎  | 8826/12000 [5:12:36<1:31:12,  1.72s/it]
                                                        

 74%|███████▎  | 8826/12000 [5:12:36<1:31:12,  1.72s/it]
 74%|███████▎  | 8827/12000 [5:12:37<1:25:24,  1.61s/it]
                                                        

 74%|███████▎  | 8827/12000 [5:12:37<1:25:24,  1.61s/it]
 74%|███████▎  | 8828/12000 [5:12:39<1:20:08,  1.52s/it]
                                                        

 74%|███████▎  | 8828/12000 [5:12:39<1:20:08,  1.52s/it]
 74%|███████▎  | 8829/12000 [5:12:40<1:21:42,  1.55s/it]
                                                        

 74%|███████▎  | 8829/12000 [5:12:40<1:21:42,  1.55s/it]
 74%|███████▎  | 8830/12000 [5:12:42<1:29:12,  1.69s/it]
                                                        

 74%|███████▎  | 8830/12000 [5:12:42<1:29:12,  1.69s/it]
 74%|███████▎  | 8831/12000 [5:12:44<1:36:09,  1.82s/it]
                                                        

 74%|███████▎  | 8831/12000 [5:12:44<1:36:09,  1.82s/it]
 74%|███████▎  | 8832/12000 [5:12:48<2:03:19,  2.34s/it]
                                                        

 74%|███████▎  | 8832/12000 [5:12:48<2:03:19,  2.34s/it]
 74%|███████▎  | 8833/12000 [5:12:49<1:50:40,  2.10s/it]
                                                        

 74%|███████▎  | 8833/12000 [5:12:49<1:50:40,  2.10s/it]
 74%|███████▎  | 8834/12000 [5:12:51<1:41:52,  1.93s/it]
                                                        

 74%|███████▎  | 8834/12000 [5:12:51<1:41:52,  1.93s/it]
 74%|███████▎  | 8835/12000 [5:12:53<1:35:30,  1.81s/it]
                                                        

 74%|███████▎  | 8835/12000 [5:12:53<1:35:30,  1.81s/it]
 74%|███████▎  | 8836/12000 [5:12:54<1:29:18,  1.69s/it]
                                                        

 74%|███████▎  | 8836/12000 [5:12:54<1:29:18,  1.69s/it]
 74%|███████▎  | 8837/12000 [5:12:57<1:43:22,  1.96s/it]
                                                        

 74%|███████▎  | 8837/12000 [5:12:57<1:43:22,  1.96s/it]
 74%|███████▎  | 8838/12000 [5:12:59<1:45:55,  2.01s/it]
                                                        

 74%|███████▎  | 8838/12000 [5:12:59<1:45:55,  2.01s/it]
 74%|███████▎  | 8839/12000 [5:13:00<1:41:06,  1.92s/it]
                                                        

 74%|███████▎  | 8839/12000 [5:13:00<1:41:06,  1.92s/it]
 74%|███████▎  | 8840/12000 [5:13:02<1:32:40,  1.76s/it]
                                                        
{'loss': '1.021', 'grad_norm': '0.5499', 'learning_rate': '3.66e-05', 'entropy': '1.026', 'num_tokens': '4.739e+07', 'mean_token_accuracy': '0.7461', 'epoch': '0.07782'}
{'loss': '1.131', 'grad_norm': '0.4309', 'learning_rate': '3.658e-05', 'entropy': '0.9145', 'num_tokens': '4.739e+07', 'mean_token_accuracy': '0.7768', 'epoch': '0.07783'}
{'loss': '0.8452', 'grad_norm': '0.3408', 'learning_rate': '3.656e-05', 'entropy': '0.8503', 'num_tokens': '4.74e+07', 'mean_token_accuracy': '0.7796', 'epoch': '0.07783'}
{'loss': '0.8719', 'grad_norm': '0.3966', 'learning_rate': '3.653e-05', 'entropy': '0.8421', 'num_tokens': '4.74e+07', 'mean_token_accuracy': '0.7924', 'epoch': '0.07784'}
{'loss': '0.6521', 'grad_norm': '0.3857', 'learning_rate': '3.651e-05', 'entropy': '0.6532', 'num_tokens': '4.741e+07', 'mean_token_accuracy': '0.8252', 'epoch': '0.07785'}
{'loss': '0.6738', 'grad_norm': '0.4196', 'learning_rate': '3.649e-05', 'entropy': '0.6922', 'num_tokens': '4.741e+07', 'mean_token_accuracy': '0.8321', 'epoch': '0.07786'}
{'loss': '0.5889', 'grad_norm': '0.3358', 'learning_rate': '3.647e-05', 'entropy': '0.6542', 'num_tokens': '4.742e+07', 'mean_token_accuracy': '0.829', 'epoch': '0.07787'}
{'loss': '0.5093', 'grad_norm': '0.3584', 'learning_rate': '3.645e-05', 'entropy': '0.5654', 'num_tokens': '4.742e+07', 'mean_token_accuracy': '0.8591', 'epoch': '0.07788'}
{'loss': '0.8057', 'grad_norm': '0.3653', 'learning_rate': '3.643e-05', 'entropy': '0.8026', 'num_tokens': '4.743e+07', 'mean_token_accuracy': '0.7967', 'epoch': '0.07789'}
{'loss': '0.8279', 'grad_norm': '0.4183', 'learning_rate': '3.641e-05', 'entropy': '0.8754', 'num_tokens': '4.743e+07', 'mean_token_accuracy': '0.7766', 'epoch': '0.0779'}
{'loss': '0.9386', 'grad_norm': '0.4424', 'learning_rate': '3.639e-05', 'entropy': '0.8655', 'num_tokens': '4.744e+07', 'mean_token_accuracy': '0.7845', 'epoch': '0.07791'}
{'loss': '0.707', 'grad_norm': '0.3505', 'learning_rate': '3.636e-05', 'entropy': '0.7587', 'num_tokens': '4.744e+07', 'mean_token_accuracy': '0.8038', 'epoch': '0.07791'}
{'loss': '0.7058', 'grad_norm': '0.4856', 'learning_rate': '3.634e-05', 'entropy': '0.7077', 'num_tokens': '4.745e+07', 'mean_token_accuracy': '0.8077', 'epoch': '0.07792'}
{'loss': '0.7069', 'grad_norm': '0.413', 'learning_rate': '3.632e-05', 'entropy': '0.7245', 'num_tokens': '4.745e+07', 'mean_token_accuracy': '0.8148', 'epoch': '0.07793'}
{'loss': '1.031', 'grad_norm': '0.4267', 'learning_rate': '3.63e-05', 'entropy': '0.9251', 'num_tokens': '4.746e+07', 'mean_token_accuracy': '0.7784', 'epoch': '0.07794'}
{'loss': '1.014', 'grad_norm': '0.3706', 'learning_rate': '3.628e-05', 'entropy': '0.9749', 'num_tokens': '4.747e+07', 'mean_token_accuracy': '0.7683', 'epoch': '0.07795'}
{'loss': '0.7139', 'grad_norm': '0.4272', 'learning_rate': '3.626e-05', 'entropy': '0.7559', 'num_tokens': '4.747e+07', 'mean_token_accuracy': '0.7964', 'epoch': '0.07796'}
{'loss': '0.5916', 'grad_norm': '0.3346', 'learning_rate': '3.624e-05', 'entropy': '0.7178', 'num_tokens': '4.748e+07', 'mean_token_accuracy': '0.8118', 'epoch': '0.07797'}
{'loss': '0.7176', 'grad_norm': '0.4155', 'learning_rate': '3.622e-05', 'entropy': '0.7498', 'num_tokens': '4.748e+07', 'mean_token_accuracy': '0.7983', 'epoch': '0.07798'}
{'loss': '0.8558', 'grad_norm': '0.4242', 'learning_rate': '3.619e-05', 'entropy': '0.9277', 'num_tokens': '4.749e+07', 'mean_token_accuracy': '0.7756', 'epoch': '0.07798'}
{'loss': '0.6633', 'grad_norm': '0.3371', 'learning_rate': '3.617e-05', 'entropy': '0.6168', 'num_tokens': '4.749e+07', 'mean_token_accuracy': '0.8465', 'epoch': '0.07799'}
{'loss': '0.67', 'grad_norm': '0.3578', 'learning_rate': '3.615e-05', 'entropy': '0.6789', 'num_tokens': '4.75e+07', 'mean_token_accuracy': '0.8176', 'epoch': '0.078'}
{'loss': '1.164', 'grad_norm': '0.4692', 'learning_rate': '3.613e-05', 'entropy': '1.223', 'num_tokens': '4.751e+07', 'mean_token_accuracy': '0.707', 'epoch': '0.07801'}
{'loss': '0.7657', 'grad_norm': '0.4273', 'learning_rate': '3.611e-05', 'entropy': '0.7381', 'num_tokens': '4.751e+07', 'mean_token_accuracy': '0.8154', 'epoch': '0.07802'}
{'loss': '0.5429', 'grad_norm': '0.3612', 'learning_rate': '3.609e-05', 'entropy': '0.6158', 'num_tokens': '4.752e+07', 'mean_token_accuracy': '0.849', 'epoch': '0.07803'}
{'loss': '0.907', 'grad_norm': '0.4571', 'learning_rate': '3.607e-05', 'entropy': '0.7979', 'num_tokens': '4.752e+07', 'mean_token_accuracy': '0.7953', 'epoch': '0.07804'}
{'loss': '0.9396', 'grad_norm': '0.4818', 'learning_rate': '3.605e-05', 'entropy': '0.9818', 'num_tokens': '4.753e+07', 'mean_token_accuracy': '0.7612', 'epoch': '0.07805'}
{'loss': '1.206', 'grad_norm': '0.43', 'learning_rate': '3.602e-05', 'entropy': '1.139', 'num_tokens': '4.754e+07', 'mean_token_accuracy': '0.7239', 'epoch': '0.07806'}
{'loss': '0.691', 'grad_norm': '0.3419', 'learning_rate': '3.6e-05', 'entropy': '0.7503', 'num_tokens': '4.754e+07', 'mean_token_accuracy': '0.8113', 'epoch': '0.07806'}
{'loss': '0.6268', 'grad_norm': '0.3711', 'learning_rate': '3.598e-05', 'entropy': '0.6308', 'num_tokens': '4.754e+07', 'mean_token_accuracy': '0.8443', 'epoch': '0.07807'}
{'loss': '0.6909', 'grad_norm': '0.4888', 'learning_rate': '3.596e-05', 'entropy': '0.746', 'num_tokens': '4.755e+07', 'mean_token_accuracy': '0.8053', 'epoch': '0.07808'}
{'loss': '0.6718', 'grad_norm': '0.4221', 'learning_rate': '3.594e-05', 'entropy': '0.6852', 'num_tokens': '4.755e+07', 'mean_token_accuracy': '0.8169', 'epoch': '0.07809'}
{'loss': '0.9201', 'grad_norm': '0.4568', 'learning_rate': '3.592e-05', 'entropy': '0.9458', 'num_tokens': '4.756e+07', 'mean_token_accuracy': '0.7672', 'epoch': '0.0781'}
{'loss': '0.8936', 'grad_norm': '0.4518', 'learning_rate': '3.59e-05', 'entropy': '0.9031', 'num_tokens': '4.756e+07', 'mean_token_accuracy': '0.775', 'epoch': '0.07811'}
{'loss': '0.7183', 'grad_norm': '0.4483', 'learning_rate': '3.588e-05', 'entropy': '0.7795', 'num_tokens': '4.757e+07', 'mean_token_accuracy': '0.7898', 'epoch': '0.07812'}
{'loss': '0.8682', 'grad_norm': '0.3881', 'learning_rate': '3.586e-05', 'entropy': '0.9613', 'num_tokens': '4.757e+07', 'mean_token_accuracy': '0.763', 'epoch': '0.07813'}
{'loss': '1.016', 'grad_norm': '0.4521', 'learning_rate': '3.583e-05', 'entropy': '1.028', 'num_tokens': '4.757e+07', 'mean_token_accuracy': '0.743', 'epoch': '0.07814'}
{'loss': '1.002', 'grad_norm': '0.4964', 'learning_rate': '3.581e-05', 'entropy': '1.016', 'num_tokens': '4.758e+07', 'mean_token_accuracy': '0.749', 'epoch': '0.07814'}
{'loss': '1.017', 'grad_norm': '0.3947', 'learning_rate': '3.579e-05', 'entropy': '0.9365', 'num_tokens': '4.759e+07', 'mean_token_accuracy': '0.7646', 'epoch': '0.07815'}
{'loss': '0.7344', 'grad_norm': '0.4704', 'learning_rate': '3.577e-05', 'entropy': '0.7699', 'num_tokens': '4.759e+07', 'mean_token_accuracy': '0.8099', 'epoch': '0.07816'}
{'loss': '1.209', 'grad_norm': '0.498', 'learning_rate': '3.575e-05', 'entropy': '1.135', 'num_tokens': '4.759e+07', 'mean_token_accuracy': '0.7346', 'epoch': '0.07817'}
{'loss': '0.5881', 'grad_norm': '0.3241', 'learning_rate': '3.573e-05', 'entropy': '0.6583', 'num_tokens': '4.76e+07', 'mean_token_accuracy': '0.8314', 'epoch': '0.07818'}
{'loss': '1.024', 'grad_norm': '0.3904', 'learning_rate': '3.571e-05', 'entropy': '0.95', 'num_tokens': '4.76e+07', 'mean_token_accuracy': '0.7698', 'epoch': '0.07819'}
{'loss': '1.008', 'grad_norm': '0.5012', 'learning_rate': '3.569e-05', 'entropy': '0.9827', 'num_tokens': '4.761e+07', 'mean_token_accuracy': '0.771', 'epoch': '0.0782'}
{'loss': '0.5651', 'grad_norm': '0.2935', 'learning_rate': '3.566e-05', 'entropy': '0.5893', 'num_tokens': '4.761e+07', 'mean_token_accuracy': '0.8495', 'epoch': '0.07821'}
{'loss': '1.183', 'grad_norm': '0.4526', 'learning_rate': '3.564e-05', 'entropy': '1.141', 'num_tokens': '4.762e+07', 'mean_token_accuracy': '0.7295', 'epoch': '0.07821'}
{'loss': '0.6145', 'grad_norm': '0.361', 'learning_rate': '3.562e-05', 'entropy': '0.6339', 'num_tokens': '4.763e+07', 'mean_token_accuracy': '0.8288', 'epoch': '0.07822'}

 74%|███████▎  | 8840/12000 [5:13:02<1:32:40,  1.76s/it]
 74%|███████▎  | 8841/12000 [5:13:03<1:28:21,  1.68s/it]
                                                        

 74%|███████▎  | 8841/12000 [5:13:03<1:28:21,  1.68s/it]
 74%|███████▎  | 8842/12000 [5:13:06<1:43:21,  1.96s/it]
                                                        

 74%|███████▎  | 8842/12000 [5:13:06<1:43:21,  1.96s/it]
 74%|███████▎  | 8843/12000 [5:13:07<1:37:40,  1.86s/it]
                                                        

 74%|███████▎  | 8843/12000 [5:13:07<1:37:40,  1.86s/it]
 74%|███████▎  | 8844/12000 [5:13:09<1:31:10,  1.73s/it]
                                                        

 74%|███████▎  | 8844/12000 [5:13:09<1:31:10,  1.73s/it]
 74%|███████▎  | 8845/12000 [5:13:11<1:35:17,  1.81s/it]
                                                        

 74%|███████▎  | 8845/12000 [5:13:11<1:35:17,  1.81s/it]
 74%|███████▎  | 8846/12000 [5:13:12<1:30:22,  1.72s/it]
                                                        

 74%|███████▎  | 8846/12000 [5:13:12<1:30:22,  1.72s/it]
 74%|███████▎  | 8847/12000 [5:13:14<1:24:19,  1.60s/it]
                                                        

 74%|███████▎  | 8847/12000 [5:13:14<1:24:19,  1.60s/it]
 74%|███████▎  | 8848/12000 [5:13:15<1:24:59,  1.62s/it]
                                                        

 74%|███████▎  | 8848/12000 [5:13:15<1:24:59,  1.62s/it]
 74%|███████▎  | 8849/12000 [5:13:17<1:22:49,  1.58s/it]
                                                        

 74%|███████▎  | 8849/12000 [5:13:17<1:22:49,  1.58s/it]
 74%|███████▍  | 8850/12000 [5:13:18<1:21:53,  1.56s/it]
                                                        

 74%|███████▍  | 8850/12000 [5:13:18<1:21:53,  1.56s/it]
 74%|███████▍  | 8851/12000 [5:13:22<1:50:55,  2.11s/it]
                                                        

 74%|███████▍  | 8851/12000 [5:13:22<1:50:55,  2.11s/it]
 74%|███████▍  | 8852/12000 [5:13:24<1:58:59,  2.27s/it]
                                                        

 74%|███████▍  | 8852/12000 [5:13:24<1:58:59,  2.27s/it]
 74%|███████▍  | 8853/12000 [5:13:26<1:47:36,  2.05s/it]
                                                        

 74%|███████▍  | 8853/12000 [5:13:26<1:47:36,  2.05s/it]
 74%|███████▍  | 8854/12000 [5:13:30<2:22:46,  2.72s/it]
                                                        

 74%|███████▍  | 8854/12000 [5:13:30<2:22:46,  2.72s/it]
 74%|███████▍  | 8855/12000 [5:13:34<2:32:29,  2.91s/it]
                                                        

 74%|███████▍  | 8855/12000 [5:13:34<2:32:29,  2.91s/it]
 74%|███████▍  | 8856/12000 [5:13:35<2:07:36,  2.44s/it]
                                                        

 74%|███████▍  | 8856/12000 [5:13:35<2:07:36,  2.44s/it]
 74%|███████▍  | 8857/12000 [5:13:37<1:55:27,  2.20s/it]
                                                        

 74%|███████▍  | 8857/12000 [5:13:37<1:55:27,  2.20s/it]
 74%|███████▍  | 8858/12000 [5:13:39<2:05:46,  2.40s/it]
                                                        

 74%|███████▍  | 8858/12000 [5:13:39<2:05:46,  2.40s/it]
 74%|███████▍  | 8859/12000 [5:13:42<2:04:22,  2.38s/it]
                                                        

 74%|███████▍  | 8859/12000 [5:13:42<2:04:22,  2.38s/it]
 74%|███████▍  | 8860/12000 [5:13:44<2:08:08,  2.45s/it]
                                                        

 74%|███████▍  | 8860/12000 [5:13:44<2:08:08,  2.45s/it]
 74%|███████▍  | 8861/12000 [5:13:46<1:57:05,  2.24s/it]
                                                        

 74%|███████▍  | 8861/12000 [5:13:46<1:57:05,  2.24s/it]
 74%|███████▍  | 8862/12000 [5:13:47<1:42:53,  1.97s/it]
                                                        

 74%|███████▍  | 8862/12000 [5:13:47<1:42:53,  1.97s/it]
 74%|███████▍  | 8863/12000 [5:13:49<1:38:42,  1.89s/it]
                                                        

 74%|███████▍  | 8863/12000 [5:13:49<1:38:42,  1.89s/it]
 74%|███████▍  | 8864/12000 [5:13:51<1:41:34,  1.94s/it]
                                                        

 74%|███████▍  | 8864/12000 [5:13:51<1:41:34,  1.94s/it]
 74%|███████▍  | 8865/12000 [5:13:53<1:33:26,  1.79s/it]
                                                        

 74%|███████▍  | 8865/12000 [5:13:53<1:33:26,  1.79s/it]
 74%|███████▍  | 8866/12000 [5:13:54<1:33:08,  1.78s/it]
                                                        

 74%|███████▍  | 8866/12000 [5:13:54<1:33:08,  1.78s/it]
 74%|███████▍  | 8867/12000 [5:13:56<1:34:26,  1.81s/it]
                                                        

 74%|███████▍  | 8867/12000 [5:13:56<1:34:26,  1.81s/it]
 74%|███████▍  | 8868/12000 [5:13:58<1:30:44,  1.74s/it]
                                                        

 74%|███████▍  | 8868/12000 [5:13:58<1:30:44,  1.74s/it]
 74%|███████▍  | 8869/12000 [5:14:03<2:15:35,  2.60s/it]
                                                        

 74%|███████▍  | 8869/12000 [5:14:03<2:15:35,  2.60s/it]
 74%|███████▍  | 8870/12000 [5:14:06<2:33:55,  2.95s/it]
                                                        

 74%|███████▍  | 8870/12000 [5:14:06<2:33:55,  2.95s/it]
 74%|███████▍  | 8871/12000 [5:14:10<2:43:36,  3.14s/it]
                                                        

 74%|███████▍  | 8871/12000 [5:14:10<2:43:36,  3.14s/it]
 74%|███████▍  | 8872/12000 [5:14:12<2:25:31,  2.79s/it]
                                                        

 74%|███████▍  | 8872/12000 [5:14:12<2:25:31,  2.79s/it]
 74%|███████▍  | 8873/12000 [5:14:14<2:13:25,  2.56s/it]
                                                        

 74%|███████▍  | 8873/12000 [5:14:14<2:13:25,  2.56s/it]
 74%|███████▍  | 8874/12000 [5:14:19<3:00:25,  3.46s/it]
                                                        

 74%|███████▍  | 8874/12000 [5:14:19<3:00:25,  3.46s/it]
 74%|███████▍  | 8875/12000 [5:14:22<2:39:42,  3.07s/it]
                                                        

 74%|███████▍  | 8875/12000 [5:14:22<2:39:42,  3.07s/it]
 74%|███████▍  | 8876/12000 [5:14:25<2:46:45,  3.20s/it]
                                                        

 74%|███████▍  | 8876/12000 [5:14:25<2:46:45,  3.20s/it]
 74%|███████▍  | 8877/12000 [5:14:27<2:27:27,  2.83s/it]
                                                        

 74%|███████▍  | 8877/12000 [5:14:27<2:27:27,  2.83s/it]
 74%|███████▍  | 8878/12000 [5:14:29<2:13:14,  2.56s/it]
                                                        

 74%|███████▍  | 8878/12000 [5:14:29<2:13:14,  2.56s/it]
 74%|███████▍  | 8879/12000 [5:14:31<2:10:59,  2.52s/it]
                                                        

 74%|███████▍  | 8879/12000 [5:14:31<2:10:59,  2.52s/it]
 74%|███████▍  | 8880/12000 [5:14:33<1:53:17,  2.18s/it]
                                                        

 74%|███████▍  | 8880/12000 [5:14:33<1:53:17,  2.18s/it]
 74%|███████▍  | 8881/12000 [5:14:34<1:42:23,  1.97s/it]
                                                        

 74%|███████▍  | 8881/12000 [5:14:34<1:42:23,  1.97s/it]
 74%|███████▍  | 8882/12000 [5:14:38<2:03:07,  2.37s/it]
                                                        

 74%|███████▍  | 8882/12000 [5:14:38<2:03:07,  2.37s/it]
 74%|███████▍  | 8883/12000 [5:14:39<1:53:26,  2.18s/it]
                                                        

 74%|███████▍  | 8883/12000 [5:14:39<1:53:26,  2.18s/it]
 74%|███████▍  | 8884/12000 [5:14:42<2:01:13,  2.33s/it]
                                                        

 74%|███████▍  | 8884/12000 [5:14:42<2:01:13,  2.33s/it]
 74%|███████▍  | 8885/12000 [5:14:44<1:51:34,  2.15s/it]
                                                        

 74%|███████▍  | 8885/12000 [5:14:44<1:51:34,  2.15s/it]
 74%|███████▍  | 8886/12000 [5:14:45<1:41:57,  1.96s/it]
                                                        

 74%|███████▍  | 8886/12000 [5:14:45<1:41:57,  1.96s/it]
 74%|███████▍  | 8887/12000 [5:14:47<1:45:20,  2.03s/it]
                                                        
{'loss': '0.581', 'grad_norm': '0.2894', 'learning_rate': '3.56e-05', 'entropy': '0.6845', 'num_tokens': '4.763e+07', 'mean_token_accuracy': '0.8157', 'epoch': '0.07823'}
{'loss': '0.69', 'grad_norm': '0.3951', 'learning_rate': '3.558e-05', 'entropy': '0.7452', 'num_tokens': '4.763e+07', 'mean_token_accuracy': '0.8064', 'epoch': '0.07824'}
{'loss': '0.5738', 'grad_norm': '0.3076', 'learning_rate': '3.556e-05', 'entropy': '0.5952', 'num_tokens': '4.764e+07', 'mean_token_accuracy': '0.8413', 'epoch': '0.07825'}
{'loss': '0.6851', 'grad_norm': '0.4313', 'learning_rate': '3.554e-05', 'entropy': '0.73', 'num_tokens': '4.765e+07', 'mean_token_accuracy': '0.8134', 'epoch': '0.07826'}
{'loss': '0.669', 'grad_norm': '0.3355', 'learning_rate': '3.552e-05', 'entropy': '0.7034', 'num_tokens': '4.765e+07', 'mean_token_accuracy': '0.8129', 'epoch': '0.07827'}
{'loss': '0.639', 'grad_norm': '0.344', 'learning_rate': '3.55e-05', 'entropy': '0.6705', 'num_tokens': '4.766e+07', 'mean_token_accuracy': '0.8357', 'epoch': '0.07828'}
{'loss': '0.7296', 'grad_norm': '0.3502', 'learning_rate': '3.548e-05', 'entropy': '0.7544', 'num_tokens': '4.766e+07', 'mean_token_accuracy': '0.8168', 'epoch': '0.07829'}
{'loss': '0.8401', 'grad_norm': '0.4029', 'learning_rate': '3.545e-05', 'entropy': '0.8571', 'num_tokens': '4.766e+07', 'mean_token_accuracy': '0.776', 'epoch': '0.07829'}
{'loss': '0.7681', 'grad_norm': '0.3949', 'learning_rate': '3.543e-05', 'entropy': '0.772', 'num_tokens': '4.767e+07', 'mean_token_accuracy': '0.8041', 'epoch': '0.0783'}
{'loss': '0.7948', 'grad_norm': '0.3762', 'learning_rate': '3.541e-05', 'entropy': '0.804', 'num_tokens': '4.768e+07', 'mean_token_accuracy': '0.8073', 'epoch': '0.07831'}
{'loss': '0.7455', 'grad_norm': '0.4174', 'learning_rate': '3.539e-05', 'entropy': '0.7867', 'num_tokens': '4.768e+07', 'mean_token_accuracy': '0.8', 'epoch': '0.07832'}
{'loss': '0.9261', 'grad_norm': '0.3973', 'learning_rate': '3.537e-05', 'entropy': '0.9593', 'num_tokens': '4.769e+07', 'mean_token_accuracy': '0.7582', 'epoch': '0.07833'}
{'loss': '0.7268', 'grad_norm': '0.3482', 'learning_rate': '3.535e-05', 'entropy': '0.7609', 'num_tokens': '4.769e+07', 'mean_token_accuracy': '0.8013', 'epoch': '0.07834'}
{'loss': '0.7462', 'grad_norm': '0.3577', 'learning_rate': '3.533e-05', 'entropy': '0.7725', 'num_tokens': '4.77e+07', 'mean_token_accuracy': '0.7996', 'epoch': '0.07835'}
{'loss': '0.6908', 'grad_norm': '0.276', 'learning_rate': '3.531e-05', 'entropy': '0.7495', 'num_tokens': '4.77e+07', 'mean_token_accuracy': '0.8045', 'epoch': '0.07836'}
{'loss': '0.8346', 'grad_norm': '0.3539', 'learning_rate': '3.529e-05', 'entropy': '0.8532', 'num_tokens': '4.771e+07', 'mean_token_accuracy': '0.7796', 'epoch': '0.07837'}
{'loss': '0.6889', 'grad_norm': '0.3623', 'learning_rate': '3.526e-05', 'entropy': '0.7478', 'num_tokens': '4.771e+07', 'mean_token_accuracy': '0.8073', 'epoch': '0.07837'}
{'loss': '0.5763', 'grad_norm': '0.3551', 'learning_rate': '3.524e-05', 'entropy': '0.5831', 'num_tokens': '4.772e+07', 'mean_token_accuracy': '0.837', 'epoch': '0.07838'}
{'loss': '0.6563', 'grad_norm': '0.3296', 'learning_rate': '3.522e-05', 'entropy': '0.6862', 'num_tokens': '4.772e+07', 'mean_token_accuracy': '0.8183', 'epoch': '0.07839'}
{'loss': '0.9807', 'grad_norm': '0.3945', 'learning_rate': '3.52e-05', 'entropy': '0.9904', 'num_tokens': '4.773e+07', 'mean_token_accuracy': '0.7576', 'epoch': '0.0784'}
{'loss': '0.8482', 'grad_norm': '0.4083', 'learning_rate': '3.518e-05', 'entropy': '0.8425', 'num_tokens': '4.774e+07', 'mean_token_accuracy': '0.7902', 'epoch': '0.07841'}
{'loss': '0.8493', 'grad_norm': '0.3964', 'learning_rate': '3.516e-05', 'entropy': '0.8094', 'num_tokens': '4.774e+07', 'mean_token_accuracy': '0.8097', 'epoch': '0.07842'}
{'loss': '0.6978', 'grad_norm': '0.2992', 'learning_rate': '3.514e-05', 'entropy': '0.7075', 'num_tokens': '4.775e+07', 'mean_token_accuracy': '0.8128', 'epoch': '0.07843'}
{'loss': '0.7768', 'grad_norm': '0.5204', 'learning_rate': '3.512e-05', 'entropy': '0.7788', 'num_tokens': '4.775e+07', 'mean_token_accuracy': '0.7978', 'epoch': '0.07844'}
{'loss': '0.672', 'grad_norm': '0.4332', 'learning_rate': '3.51e-05', 'entropy': '0.6724', 'num_tokens': '4.776e+07', 'mean_token_accuracy': '0.8256', 'epoch': '0.07844'}
{'loss': '0.6989', 'grad_norm': '0.3739', 'learning_rate': '3.508e-05', 'entropy': '0.7553', 'num_tokens': '4.776e+07', 'mean_token_accuracy': '0.8141', 'epoch': '0.07845'}
{'loss': '0.7112', 'grad_norm': '0.4095', 'learning_rate': '3.506e-05', 'entropy': '0.6762', 'num_tokens': '4.777e+07', 'mean_token_accuracy': '0.8297', 'epoch': '0.07846'}
{'loss': '1.112', 'grad_norm': '0.6099', 'learning_rate': '3.503e-05', 'entropy': '1.018', 'num_tokens': '4.777e+07', 'mean_token_accuracy': '0.7614', 'epoch': '0.07847'}
{'loss': '0.5893', 'grad_norm': '0.2951', 'learning_rate': '3.501e-05', 'entropy': '0.6681', 'num_tokens': '4.778e+07', 'mean_token_accuracy': '0.8292', 'epoch': '0.07848'}
{'loss': '1', 'grad_norm': '0.4349', 'learning_rate': '3.499e-05', 'entropy': '0.91', 'num_tokens': '4.779e+07', 'mean_token_accuracy': '0.7691', 'epoch': '0.07849'}
{'loss': '1.165', 'grad_norm': '0.4285', 'learning_rate': '3.497e-05', 'entropy': '1.07', 'num_tokens': '4.779e+07', 'mean_token_accuracy': '0.736', 'epoch': '0.0785'}
{'loss': '1.086', 'grad_norm': '0.4018', 'learning_rate': '3.495e-05', 'entropy': '1.143', 'num_tokens': '4.78e+07', 'mean_token_accuracy': '0.694', 'epoch': '0.07851'}
{'loss': '0.7129', 'grad_norm': '0.3644', 'learning_rate': '3.493e-05', 'entropy': '0.7489', 'num_tokens': '4.781e+07', 'mean_token_accuracy': '0.8084', 'epoch': '0.07852'}
{'loss': '0.6888', 'grad_norm': '0.4059', 'learning_rate': '3.491e-05', 'entropy': '0.7488', 'num_tokens': '4.781e+07', 'mean_token_accuracy': '0.8094', 'epoch': '0.07852'}
{'loss': '1.031', 'grad_norm': '0.3744', 'learning_rate': '3.489e-05', 'entropy': '0.873', 'num_tokens': '4.782e+07', 'mean_token_accuracy': '0.7965', 'epoch': '0.07853'}
{'loss': '0.8307', 'grad_norm': '0.4', 'learning_rate': '3.487e-05', 'entropy': '0.7427', 'num_tokens': '4.783e+07', 'mean_token_accuracy': '0.8129', 'epoch': '0.07854'}
{'loss': '0.8719', 'grad_norm': '0.4595', 'learning_rate': '3.485e-05', 'entropy': '0.8482', 'num_tokens': '4.783e+07', 'mean_token_accuracy': '0.7909', 'epoch': '0.07855'}
{'loss': '0.8265', 'grad_norm': '0.3802', 'learning_rate': '3.482e-05', 'entropy': '0.7605', 'num_tokens': '4.784e+07', 'mean_token_accuracy': '0.8085', 'epoch': '0.07856'}
{'loss': '0.7717', 'grad_norm': '0.5131', 'learning_rate': '3.48e-05', 'entropy': '0.7235', 'num_tokens': '4.784e+07', 'mean_token_accuracy': '0.8095', 'epoch': '0.07857'}
{'loss': '0.8666', 'grad_norm': '0.412', 'learning_rate': '3.478e-05', 'entropy': '0.8011', 'num_tokens': '4.785e+07', 'mean_token_accuracy': '0.8018', 'epoch': '0.07858'}
{'loss': '0.8577', 'grad_norm': '0.402', 'learning_rate': '3.476e-05', 'entropy': '0.8683', 'num_tokens': '4.785e+07', 'mean_token_accuracy': '0.7971', 'epoch': '0.07859'}
{'loss': '0.6764', 'grad_norm': '0.3465', 'learning_rate': '3.474e-05', 'entropy': '0.7036', 'num_tokens': '4.786e+07', 'mean_token_accuracy': '0.8335', 'epoch': '0.0786'}
{'loss': '0.9435', 'grad_norm': '0.3289', 'learning_rate': '3.472e-05', 'entropy': '1.019', 'num_tokens': '4.787e+07', 'mean_token_accuracy': '0.7491', 'epoch': '0.0786'}
{'loss': '0.6069', 'grad_norm': '0.3254', 'learning_rate': '3.47e-05', 'entropy': '0.6062', 'num_tokens': '4.787e+07', 'mean_token_accuracy': '0.8487', 'epoch': '0.07861'}
{'loss': '1.1', 'grad_norm': '0.4454', 'learning_rate': '3.468e-05', 'entropy': '1.095', 'num_tokens': '4.788e+07', 'mean_token_accuracy': '0.7257', 'epoch': '0.07862'}
{'loss': '1.169', 'grad_norm': '0.5681', 'learning_rate': '3.466e-05', 'entropy': '1.117', 'num_tokens': '4.788e+07', 'mean_token_accuracy': '0.7254', 'epoch': '0.07863'}
{'loss': '1.473', 'grad_norm': '0.5118', 'learning_rate': '3.464e-05', 'entropy': '1.459', 'num_tokens': '4.789e+07', 'mean_token_accuracy': '0.6584', 'epoch': '0.07864'}

 74%|███████▍  | 8887/12000 [5:14:47<1:45:20,  2.03s/it]
 74%|███████▍  | 8888/12000 [5:14:49<1:38:56,  1.91s/it]
                                                        

 74%|███████▍  | 8888/12000 [5:14:49<1:38:56,  1.91s/it]
 74%|███████▍  | 8889/12000 [5:14:51<1:36:41,  1.86s/it]
                                                        

 74%|███████▍  | 8889/12000 [5:14:51<1:36:41,  1.86s/it]
 74%|███████▍  | 8890/12000 [5:14:52<1:31:45,  1.77s/it]
                                                        

 74%|███████▍  | 8890/12000 [5:14:52<1:31:45,  1.77s/it]
 74%|███████▍  | 8891/12000 [5:14:54<1:31:59,  1.78s/it]
                                                        

 74%|███████▍  | 8891/12000 [5:14:54<1:31:59,  1.78s/it]
 74%|███████▍  | 8892/12000 [5:14:56<1:28:22,  1.71s/it]
                                                        

 74%|███████▍  | 8892/12000 [5:14:56<1:28:22,  1.71s/it]
 74%|███████▍  | 8893/12000 [5:14:57<1:28:46,  1.71s/it]
                                                        

 74%|███████▍  | 8893/12000 [5:14:57<1:28:46,  1.71s/it]
 74%|███████▍  | 8894/12000 [5:15:00<1:34:11,  1.82s/it]
                                                        

 74%|███████▍  | 8894/12000 [5:15:00<1:34:11,  1.82s/it]
 74%|███████▍  | 8895/12000 [5:15:01<1:34:05,  1.82s/it]
                                                        

 74%|███████▍  | 8895/12000 [5:15:01<1:34:05,  1.82s/it]
 74%|███████▍  | 8896/12000 [5:15:03<1:31:47,  1.77s/it]
                                                        

 74%|███████▍  | 8896/12000 [5:15:03<1:31:47,  1.77s/it]
 74%|███████▍  | 8897/12000 [5:15:05<1:27:59,  1.70s/it]
                                                        

 74%|███████▍  | 8897/12000 [5:15:05<1:27:59,  1.70s/it]
 74%|███████▍  | 8898/12000 [5:15:08<1:55:52,  2.24s/it]
                                                        

 74%|███████▍  | 8898/12000 [5:15:08<1:55:52,  2.24s/it]
 74%|███████▍  | 8899/12000 [5:15:10<1:47:05,  2.07s/it]
                                                        

 74%|███████▍  | 8899/12000 [5:15:10<1:47:05,  2.07s/it]
 74%|███████▍  | 8900/12000 [5:15:11<1:40:46,  1.95s/it]
                                                        

 74%|███████▍  | 8900/12000 [5:15:11<1:40:46,  1.95s/it]
 74%|███████▍  | 8901/12000 [5:15:15<2:07:23,  2.47s/it]
                                                        

 74%|███████▍  | 8901/12000 [5:15:15<2:07:23,  2.47s/it]
 74%|███████▍  | 8902/12000 [5:15:17<1:56:24,  2.25s/it]
                                                        

 74%|███████▍  | 8902/12000 [5:15:17<1:56:24,  2.25s/it]
 74%|███████▍  | 8903/12000 [5:15:19<1:59:53,  2.32s/it]
                                                        

 74%|███████▍  | 8903/12000 [5:15:19<1:59:53,  2.32s/it]
 74%|███████▍  | 8904/12000 [5:15:21<1:47:22,  2.08s/it]
                                                        

 74%|███████▍  | 8904/12000 [5:15:21<1:47:22,  2.08s/it]
 74%|███████▍  | 8905/12000 [5:15:22<1:39:26,  1.93s/it]
                                                        

 74%|███████▍  | 8905/12000 [5:15:22<1:39:26,  1.93s/it]
 74%|███████▍  | 8906/12000 [5:15:24<1:32:58,  1.80s/it]
                                                        

 74%|███████▍  | 8906/12000 [5:15:24<1:32:58,  1.80s/it]
 74%|███████▍  | 8907/12000 [5:15:25<1:27:43,  1.70s/it]
                                                        

 74%|███████▍  | 8907/12000 [5:15:25<1:27:43,  1.70s/it]
 74%|███████▍  | 8908/12000 [5:15:27<1:25:36,  1.66s/it]
                                                        

 74%|███████▍  | 8908/12000 [5:15:27<1:25:36,  1.66s/it]
 74%|███████▍  | 8909/12000 [5:15:29<1:30:15,  1.75s/it]
                                                        

 74%|███████▍  | 8909/12000 [5:15:29<1:30:15,  1.75s/it]
 74%|███████▍  | 8910/12000 [5:15:30<1:26:53,  1.69s/it]
                                                        

 74%|███████▍  | 8910/12000 [5:15:30<1:26:53,  1.69s/it]
 74%|███████▍  | 8911/12000 [5:15:32<1:28:53,  1.73s/it]
                                                        

 74%|███████▍  | 8911/12000 [5:15:32<1:28:53,  1.73s/it]
 74%|███████▍  | 8912/12000 [5:15:34<1:29:59,  1.75s/it]
                                                        

 74%|███████▍  | 8912/12000 [5:15:34<1:29:59,  1.75s/it]
 74%|███████▍  | 8913/12000 [5:15:36<1:37:57,  1.90s/it]
                                                        

 74%|███████▍  | 8913/12000 [5:15:36<1:37:57,  1.90s/it]
 74%|███████▍  | 8914/12000 [5:15:39<1:51:42,  2.17s/it]
                                                        

 74%|███████▍  | 8914/12000 [5:15:39<1:51:42,  2.17s/it]
 74%|███████▍  | 8915/12000 [5:15:42<1:59:18,  2.32s/it]
                                                        

 74%|███████▍  | 8915/12000 [5:15:42<1:59:18,  2.32s/it]
 74%|███████▍  | 8916/12000 [5:15:44<2:01:24,  2.36s/it]
                                                        

 74%|███████▍  | 8916/12000 [5:15:44<2:01:24,  2.36s/it]
 74%|███████▍  | 8917/12000 [5:15:46<1:47:54,  2.10s/it]
                                                        

 74%|███████▍  | 8917/12000 [5:15:46<1:47:54,  2.10s/it]
 74%|███████▍  | 8918/12000 [5:15:48<1:46:44,  2.08s/it]
                                                        

 74%|███████▍  | 8918/12000 [5:15:48<1:46:44,  2.08s/it]
 74%|███████▍  | 8919/12000 [5:15:50<1:42:57,  2.01s/it]
                                                        

 74%|███████▍  | 8919/12000 [5:15:50<1:42:57,  2.01s/it]
 74%|███████▍  | 8920/12000 [5:15:54<2:21:08,  2.75s/it]
                                                        

 74%|███████▍  | 8920/12000 [5:15:54<2:21:08,  2.75s/it]
 74%|███████▍  | 8921/12000 [5:15:59<2:51:43,  3.35s/it]
                                                        

 74%|███████▍  | 8921/12000 [5:15:59<2:51:43,  3.35s/it]
 74%|███████▍  | 8922/12000 [5:16:00<2:26:19,  2.85s/it]
                                                        

 74%|███████▍  | 8922/12000 [5:16:01<2:26:19,  2.85s/it]
 74%|███████▍  | 8923/12000 [5:16:04<2:29:57,  2.92s/it]
                                                        

 74%|███████▍  | 8923/12000 [5:16:04<2:29:57,  2.92s/it]
 74%|███████▍  | 8924/12000 [5:16:05<2:10:14,  2.54s/it]
                                                        

 74%|███████▍  | 8924/12000 [5:16:05<2:10:14,  2.54s/it]
 74%|███████▍  | 8925/12000 [5:16:07<1:56:30,  2.27s/it]
                                                        

 74%|███████▍  | 8925/12000 [5:16:07<1:56:30,  2.27s/it]
 74%|███████▍  | 8926/12000 [5:16:08<1:45:40,  2.06s/it]
                                                        

 74%|███████▍  | 8926/12000 [5:16:08<1:45:40,  2.06s/it]
 74%|███████▍  | 8927/12000 [5:16:10<1:39:26,  1.94s/it]
                                                        

 74%|███████▍  | 8927/12000 [5:16:10<1:39:26,  1.94s/it]
 74%|███████▍  | 8928/12000 [5:16:12<1:32:40,  1.81s/it]
                                                        

 74%|███████▍  | 8928/12000 [5:16:12<1:32:40,  1.81s/it]
 74%|███████▍  | 8929/12000 [5:16:16<2:19:38,  2.73s/it]
                                                        

 74%|███████▍  | 8929/12000 [5:16:16<2:19:38,  2.73s/it]
 74%|███████▍  | 8930/12000 [5:16:19<2:22:15,  2.78s/it]
                                                        

 74%|███████▍  | 8930/12000 [5:16:19<2:22:15,  2.78s/it]
 74%|███████▍  | 8931/12000 [5:16:21<2:07:15,  2.49s/it]
                                                        

 74%|███████▍  | 8931/12000 [5:16:21<2:07:15,  2.49s/it]
 74%|███████▍  | 8932/12000 [5:16:23<1:56:15,  2.27s/it]
                                                        

 74%|███████▍  | 8932/12000 [5:16:23<1:56:15,  2.27s/it]
 74%|███████▍  | 8933/12000 [5:16:25<1:50:23,  2.16s/it]
                                                        

 74%|███████▍  | 8933/12000 [5:16:25<1:50:23,  2.16s/it]
 74%|███████▍  | 8934/12000 [5:16:27<1:46:25,  2.08s/it]
                                                        
{'loss': '1.09', 'grad_norm': '0.4027', 'learning_rate': '3.462e-05', 'entropy': '1.019', 'num_tokens': '4.789e+07', 'mean_token_accuracy': '0.7608', 'epoch': '0.07865'}
{'loss': '0.6351', 'grad_norm': '0.451', 'learning_rate': '3.46e-05', 'entropy': '0.6915', 'num_tokens': '4.79e+07', 'mean_token_accuracy': '0.8212', 'epoch': '0.07866'}
{'loss': '0.7779', 'grad_norm': '0.3308', 'learning_rate': '3.457e-05', 'entropy': '0.8395', 'num_tokens': '4.79e+07', 'mean_token_accuracy': '0.8007', 'epoch': '0.07867'}
{'loss': '0.8027', 'grad_norm': '0.4024', 'learning_rate': '3.455e-05', 'entropy': '0.8294', 'num_tokens': '4.791e+07', 'mean_token_accuracy': '0.7723', 'epoch': '0.07868'}
{'loss': '0.8533', 'grad_norm': '0.4033', 'learning_rate': '3.453e-05', 'entropy': '0.8816', 'num_tokens': '4.791e+07', 'mean_token_accuracy': '0.7819', 'epoch': '0.07868'}
{'loss': '0.9654', 'grad_norm': '0.4729', 'learning_rate': '3.451e-05', 'entropy': '0.9306', 'num_tokens': '4.792e+07', 'mean_token_accuracy': '0.7768', 'epoch': '0.07869'}
{'loss': '0.8742', 'grad_norm': '0.4068', 'learning_rate': '3.449e-05', 'entropy': '0.9044', 'num_tokens': '4.792e+07', 'mean_token_accuracy': '0.7589', 'epoch': '0.0787'}
{'loss': '0.7871', 'grad_norm': '0.3571', 'learning_rate': '3.447e-05', 'entropy': '0.8454', 'num_tokens': '4.793e+07', 'mean_token_accuracy': '0.7867', 'epoch': '0.07871'}
{'loss': '0.9013', 'grad_norm': '0.4138', 'learning_rate': '3.445e-05', 'entropy': '0.8806', 'num_tokens': '4.793e+07', 'mean_token_accuracy': '0.7695', 'epoch': '0.07872'}
{'loss': '0.8188', 'grad_norm': '0.4369', 'learning_rate': '3.443e-05', 'entropy': '0.8283', 'num_tokens': '4.794e+07', 'mean_token_accuracy': '0.7819', 'epoch': '0.07873'}
{'loss': '0.7346', 'grad_norm': '0.3757', 'learning_rate': '3.441e-05', 'entropy': '0.8068', 'num_tokens': '4.794e+07', 'mean_token_accuracy': '0.8125', 'epoch': '0.07874'}
{'loss': '0.758', 'grad_norm': '0.4797', 'learning_rate': '3.439e-05', 'entropy': '0.7807', 'num_tokens': '4.795e+07', 'mean_token_accuracy': '0.7966', 'epoch': '0.07875'}
{'loss': '0.6269', 'grad_norm': '0.3996', 'learning_rate': '3.437e-05', 'entropy': '0.6033', 'num_tokens': '4.795e+07', 'mean_token_accuracy': '0.8406', 'epoch': '0.07875'}
{'loss': '0.816', 'grad_norm': '0.3763', 'learning_rate': '3.435e-05', 'entropy': '0.7548', 'num_tokens': '4.796e+07', 'mean_token_accuracy': '0.8021', 'epoch': '0.07876'}
{'loss': '0.7685', 'grad_norm': '0.3469', 'learning_rate': '3.432e-05', 'entropy': '0.7616', 'num_tokens': '4.796e+07', 'mean_token_accuracy': '0.8134', 'epoch': '0.07877'}
{'loss': '1.128', 'grad_norm': '0.4281', 'learning_rate': '3.43e-05', 'entropy': '1.067', 'num_tokens': '4.797e+07', 'mean_token_accuracy': '0.7499', 'epoch': '0.07878'}
{'loss': '0.443', 'grad_norm': '0.3611', 'learning_rate': '3.428e-05', 'entropy': '0.5542', 'num_tokens': '4.797e+07', 'mean_token_accuracy': '0.8577', 'epoch': '0.07879'}
{'loss': '1.002', 'grad_norm': '0.5125', 'learning_rate': '3.426e-05', 'entropy': '0.9615', 'num_tokens': '4.798e+07', 'mean_token_accuracy': '0.7586', 'epoch': '0.0788'}
{'loss': '0.9929', 'grad_norm': '0.4059', 'learning_rate': '3.424e-05', 'entropy': '0.9318', 'num_tokens': '4.798e+07', 'mean_token_accuracy': '0.7569', 'epoch': '0.07881'}
{'loss': '0.9157', 'grad_norm': '0.4889', 'learning_rate': '3.422e-05', 'entropy': '1.007', 'num_tokens': '4.799e+07', 'mean_token_accuracy': '0.7565', 'epoch': '0.07882'}
{'loss': '0.9393', 'grad_norm': '0.3969', 'learning_rate': '3.42e-05', 'entropy': '0.922', 'num_tokens': '4.799e+07', 'mean_token_accuracy': '0.7798', 'epoch': '0.07883'}
{'loss': '0.7807', 'grad_norm': '0.5017', 'learning_rate': '3.418e-05', 'entropy': '0.8205', 'num_tokens': '4.8e+07', 'mean_token_accuracy': '0.7988', 'epoch': '0.07883'}
{'loss': '1.047', 'grad_norm': '0.4233', 'learning_rate': '3.416e-05', 'entropy': '1.132', 'num_tokens': '4.8e+07', 'mean_token_accuracy': '0.7284', 'epoch': '0.07884'}
{'loss': '0.8867', 'grad_norm': '0.428', 'learning_rate': '3.414e-05', 'entropy': '0.8462', 'num_tokens': '4.801e+07', 'mean_token_accuracy': '0.7986', 'epoch': '0.07885'}
{'loss': '0.7265', 'grad_norm': '0.4615', 'learning_rate': '3.412e-05', 'entropy': '0.7679', 'num_tokens': '4.801e+07', 'mean_token_accuracy': '0.7966', 'epoch': '0.07886'}
{'loss': '1.013', 'grad_norm': '0.4605', 'learning_rate': '3.41e-05', 'entropy': '0.9781', 'num_tokens': '4.802e+07', 'mean_token_accuracy': '0.7741', 'epoch': '0.07887'}
{'loss': '0.864', 'grad_norm': '0.3992', 'learning_rate': '3.408e-05', 'entropy': '0.8789', 'num_tokens': '4.802e+07', 'mean_token_accuracy': '0.7883', 'epoch': '0.07888'}
{'loss': '0.7392', 'grad_norm': '0.3498', 'learning_rate': '3.406e-05', 'entropy': '0.7598', 'num_tokens': '4.803e+07', 'mean_token_accuracy': '0.8062', 'epoch': '0.07889'}
{'loss': '1.105', 'grad_norm': '0.35', 'learning_rate': '3.403e-05', 'entropy': '1.07', 'num_tokens': '4.804e+07', 'mean_token_accuracy': '0.7282', 'epoch': '0.0789'}
{'loss': '0.9108', 'grad_norm': '0.5049', 'learning_rate': '3.401e-05', 'entropy': '0.8091', 'num_tokens': '4.804e+07', 'mean_token_accuracy': '0.7896', 'epoch': '0.07891'}
{'loss': '0.7265', 'grad_norm': '0.4083', 'learning_rate': '3.399e-05', 'entropy': '0.7029', 'num_tokens': '4.805e+07', 'mean_token_accuracy': '0.8177', 'epoch': '0.07891'}
{'loss': '1.023', 'grad_norm': '0.3876', 'learning_rate': '3.397e-05', 'entropy': '0.9463', 'num_tokens': '4.805e+07', 'mean_token_accuracy': '0.7622', 'epoch': '0.07892'}
{'loss': '0.8972', 'grad_norm': '0.3934', 'learning_rate': '3.395e-05', 'entropy': '0.8632', 'num_tokens': '4.806e+07', 'mean_token_accuracy': '0.7778', 'epoch': '0.07893'}
{'loss': '0.5095', 'grad_norm': '0.321', 'learning_rate': '3.393e-05', 'entropy': '0.5274', 'num_tokens': '4.807e+07', 'mean_token_accuracy': '0.859', 'epoch': '0.07894'}
{'loss': '0.9454', 'grad_norm': '0.3882', 'learning_rate': '3.391e-05', 'entropy': '0.7076', 'num_tokens': '4.807e+07', 'mean_token_accuracy': '0.8197', 'epoch': '0.07895'}
{'loss': '0.7892', 'grad_norm': '0.3865', 'learning_rate': '3.389e-05', 'entropy': '0.8866', 'num_tokens': '4.808e+07', 'mean_token_accuracy': '0.7835', 'epoch': '0.07896'}
{'loss': '0.9932', 'grad_norm': '0.4411', 'learning_rate': '3.387e-05', 'entropy': '0.9617', 'num_tokens': '4.809e+07', 'mean_token_accuracy': '0.7582', 'epoch': '0.07897'}
{'loss': '0.7156', 'grad_norm': '0.36', 'learning_rate': '3.385e-05', 'entropy': '0.6953', 'num_tokens': '4.809e+07', 'mean_token_accuracy': '0.817', 'epoch': '0.07898'}
{'loss': '0.7101', 'grad_norm': '0.3498', 'learning_rate': '3.383e-05', 'entropy': '0.7297', 'num_tokens': '4.81e+07', 'mean_token_accuracy': '0.8139', 'epoch': '0.07898'}
{'loss': '0.9847', 'grad_norm': '0.5173', 'learning_rate': '3.381e-05', 'entropy': '0.8948', 'num_tokens': '4.81e+07', 'mean_token_accuracy': '0.7848', 'epoch': '0.07899'}
{'loss': '0.6068', 'grad_norm': '0.39', 'learning_rate': '3.379e-05', 'entropy': '0.5836', 'num_tokens': '4.811e+07', 'mean_token_accuracy': '0.8458', 'epoch': '0.079'}
{'loss': '0.585', 'grad_norm': '0.4676', 'learning_rate': '3.377e-05', 'entropy': '0.6561', 'num_tokens': '4.811e+07', 'mean_token_accuracy': '0.8264', 'epoch': '0.07901'}
{'loss': '1.267', 'grad_norm': '0.2158', 'learning_rate': '3.374e-05', 'entropy': '0.882', 'num_tokens': '4.812e+07', 'mean_token_accuracy': '0.7616', 'epoch': '0.07902'}
{'loss': '1.191', 'grad_norm': '0.4703', 'learning_rate': '3.372e-05', 'entropy': '1.139', 'num_tokens': '4.813e+07', 'mean_token_accuracy': '0.7286', 'epoch': '0.07903'}
{'loss': '0.9061', 'grad_norm': '0.5112', 'learning_rate': '3.37e-05', 'entropy': '0.9162', 'num_tokens': '4.813e+07', 'mean_token_accuracy': '0.7833', 'epoch': '0.07904'}
{'loss': '0.8885', 'grad_norm': '0.4178', 'learning_rate': '3.368e-05', 'entropy': '0.8381', 'num_tokens': '4.814e+07', 'mean_token_accuracy': '0.7837', 'epoch': '0.07905'}
{'loss': '0.8218', 'grad_norm': '0.395', 'learning_rate': '3.366e-05', 'entropy': '0.773', 'num_tokens': '4.814e+07', 'mean_token_accuracy': '0.7982', 'epoch': '0.07906'}

 74%|███████▍  | 8934/12000 [5:16:27<1:46:25,  2.08s/it]
 74%|███████▍  | 8935/12000 [5:16:29<1:47:27,  2.10s/it]
                                                        

 74%|███████▍  | 8935/12000 [5:16:29<1:47:27,  2.10s/it]
 74%|███████▍  | 8936/12000 [5:16:31<1:43:42,  2.03s/it]
                                                        

 74%|███████▍  | 8936/12000 [5:16:31<1:43:42,  2.03s/it]
 74%|███████▍  | 8937/12000 [5:16:33<1:41:26,  1.99s/it]
                                                        

 74%|███████▍  | 8937/12000 [5:16:33<1:41:26,  1.99s/it]
 74%|███████▍  | 8938/12000 [5:16:36<1:58:38,  2.32s/it]
                                                        

 74%|███████▍  | 8938/12000 [5:16:36<1:58:38,  2.32s/it]
 74%|███████▍  | 8939/12000 [5:16:37<1:47:10,  2.10s/it]
                                                        

 74%|███████▍  | 8939/12000 [5:16:37<1:47:10,  2.10s/it]
 74%|███████▍  | 8940/12000 [5:16:39<1:41:29,  1.99s/it]
                                                        

 74%|███████▍  | 8940/12000 [5:16:39<1:41:29,  1.99s/it]
 75%|███████▍  | 8941/12000 [5:16:42<1:55:45,  2.27s/it]
                                                        

 75%|███████▍  | 8941/12000 [5:16:42<1:55:45,  2.27s/it]
 75%|███████▍  | 8942/12000 [5:16:46<2:25:29,  2.85s/it]
                                                        

 75%|███████▍  | 8942/12000 [5:16:46<2:25:29,  2.85s/it]
 75%|███████▍  | 8943/12000 [5:16:48<2:05:52,  2.47s/it]
                                                        

 75%|███████▍  | 8943/12000 [5:16:48<2:05:52,  2.47s/it]
 75%|███████▍  | 8944/12000 [5:16:49<1:52:00,  2.20s/it]
                                                        

 75%|███████▍  | 8944/12000 [5:16:49<1:52:00,  2.20s/it]
 75%|███████▍  | 8945/12000 [5:16:52<1:57:57,  2.32s/it]
                                                        

 75%|███████▍  | 8945/12000 [5:16:52<1:57:57,  2.32s/it]
 75%|███████▍  | 8946/12000 [5:16:54<1:52:13,  2.20s/it]
                                                        

 75%|███████▍  | 8946/12000 [5:16:54<1:52:13,  2.20s/it]
 75%|███████▍  | 8947/12000 [5:16:56<1:48:31,  2.13s/it]
                                                        

 75%|███████▍  | 8947/12000 [5:16:56<1:48:31,  2.13s/it]
 75%|███████▍  | 8948/12000 [5:16:58<1:42:25,  2.01s/it]
                                                        

 75%|███████▍  | 8948/12000 [5:16:58<1:42:25,  2.01s/it]
 75%|███████▍  | 8949/12000 [5:17:04<2:41:58,  3.19s/it]
                                                        

 75%|███████▍  | 8949/12000 [5:17:04<2:41:58,  3.19s/it]
 75%|███████▍  | 8950/12000 [5:17:06<2:26:31,  2.88s/it]
                                                        

 75%|███████▍  | 8950/12000 [5:17:06<2:26:31,  2.88s/it]
 75%|███████▍  | 8951/12000 [5:17:09<2:34:25,  3.04s/it]
                                                        

 75%|███████▍  | 8951/12000 [5:17:09<2:34:25,  3.04s/it]
 75%|███████▍  | 8952/12000 [5:17:12<2:26:42,  2.89s/it]
                                                        

 75%|███████▍  | 8952/12000 [5:17:12<2:26:42,  2.89s/it]
 75%|███████▍  | 8953/12000 [5:17:13<2:09:31,  2.55s/it]
                                                        

 75%|███████▍  | 8953/12000 [5:17:13<2:09:31,  2.55s/it]
 75%|███████▍  | 8954/12000 [5:17:16<2:03:27,  2.43s/it]
                                                        

 75%|███████▍  | 8954/12000 [5:17:16<2:03:27,  2.43s/it]
 75%|███████▍  | 8955/12000 [5:17:17<1:46:59,  2.11s/it]
                                                        

 75%|███████▍  | 8955/12000 [5:17:17<1:46:59,  2.11s/it]
 75%|███████▍  | 8956/12000 [5:17:21<2:13:57,  2.64s/it]
                                                        

 75%|███████▍  | 8956/12000 [5:17:21<2:13:57,  2.64s/it]
 75%|███████▍  | 8957/12000 [5:17:22<1:57:51,  2.32s/it]
                                                        

 75%|███████▍  | 8957/12000 [5:17:22<1:57:51,  2.32s/it]
 75%|███████▍  | 8958/12000 [5:17:24<1:45:16,  2.08s/it]
                                                        

 75%|███████▍  | 8958/12000 [5:17:24<1:45:16,  2.08s/it]
 75%|███████▍  | 8959/12000 [5:17:26<1:41:12,  2.00s/it]
                                                        

 75%|███████▍  | 8959/12000 [5:17:26<1:41:12,  2.00s/it]
 75%|███████▍  | 8960/12000 [5:17:27<1:32:55,  1.83s/it]
                                                        

 75%|███████▍  | 8960/12000 [5:17:27<1:32:55,  1.83s/it]
 75%|███████▍  | 8961/12000 [5:17:30<1:53:57,  2.25s/it]
                                                        

 75%|███████▍  | 8961/12000 [5:17:30<1:53:57,  2.25s/it]
 75%|███████▍  | 8962/12000 [5:17:32<1:47:31,  2.12s/it]
                                                        

 75%|███████▍  | 8962/12000 [5:17:32<1:47:31,  2.12s/it]
 75%|███████▍  | 8963/12000 [5:17:34<1:39:30,  1.97s/it]
                                                        

 75%|███████▍  | 8963/12000 [5:17:34<1:39:30,  1.97s/it]
 75%|███████▍  | 8964/12000 [5:17:38<2:08:02,  2.53s/it]
                                                        

 75%|███████▍  | 8964/12000 [5:17:38<2:08:02,  2.53s/it]
 75%|███████▍  | 8965/12000 [5:17:39<1:54:44,  2.27s/it]
                                                        

 75%|███████▍  | 8965/12000 [5:17:39<1:54:44,  2.27s/it]
 75%|███████▍  | 8966/12000 [5:17:41<1:46:00,  2.10s/it]
                                                        

 75%|███████▍  | 8966/12000 [5:17:41<1:46:00,  2.10s/it]
 75%|███████▍  | 8967/12000 [5:17:43<1:48:13,  2.14s/it]
                                                        

 75%|███████▍  | 8967/12000 [5:17:43<1:48:13,  2.14s/it]
 75%|███████▍  | 8968/12000 [5:17:46<1:56:07,  2.30s/it]
                                                        

 75%|███████▍  | 8968/12000 [5:17:46<1:56:07,  2.30s/it]
 75%|███████▍  | 8969/12000 [5:17:47<1:43:36,  2.05s/it]
                                                        

 75%|███████▍  | 8969/12000 [5:17:47<1:43:36,  2.05s/it]
 75%|███████▍  | 8970/12000 [5:17:49<1:37:52,  1.94s/it]
                                                        

 75%|███████▍  | 8970/12000 [5:17:49<1:37:52,  1.94s/it]
 75%|███████▍  | 8971/12000 [5:17:51<1:36:53,  1.92s/it]
                                                        

 75%|███████▍  | 8971/12000 [5:17:51<1:36:53,  1.92s/it]
 75%|███████▍  | 8972/12000 [5:17:52<1:29:22,  1.77s/it]
                                                        

 75%|███████▍  | 8972/12000 [5:17:52<1:29:22,  1.77s/it]
 75%|███████▍  | 8973/12000 [5:17:54<1:25:34,  1.70s/it]
                                                        

 75%|███████▍  | 8973/12000 [5:17:54<1:25:34,  1.70s/it]
 75%|███████▍  | 8974/12000 [5:17:56<1:27:37,  1.74s/it]
                                                        

 75%|███████▍  | 8974/12000 [5:17:56<1:27:37,  1.74s/it]
 75%|███████▍  | 8975/12000 [5:17:57<1:22:29,  1.64s/it]
                                                        

 75%|███████▍  | 8975/12000 [5:17:57<1:22:29,  1.64s/it]
 75%|███████▍  | 8976/12000 [5:18:00<1:37:14,  1.93s/it]
                                                        

 75%|███████▍  | 8976/12000 [5:18:00<1:37:14,  1.93s/it]
 75%|███████▍  | 8977/12000 [5:18:02<1:35:23,  1.89s/it]
                                                        

 75%|███████▍  | 8977/12000 [5:18:02<1:35:23,  1.89s/it]
 75%|███████▍  | 8978/12000 [5:18:03<1:29:25,  1.78s/it]
                                                        

 75%|███████▍  | 8978/12000 [5:18:03<1:29:25,  1.78s/it]
 75%|███████▍  | 8979/12000 [5:18:06<1:41:00,  2.01s/it]
                                                        

 75%|███████▍  | 8979/12000 [5:18:06<1:41:00,  2.01s/it]
 75%|███████▍  | 8980/12000 [5:18:07<1:31:41,  1.82s/it]
                                                        

 75%|███████▍  | 8980/12000 [5:18:07<1:31:41,  1.82s/it]
 75%|███████▍  | 8981/12000 [5:18:09<1:30:19,  1.80s/it]
                                                        
{'loss': '0.7413', 'grad_norm': '0.3747', 'learning_rate': '3.364e-05', 'entropy': '0.7966', 'num_tokens': '4.814e+07', 'mean_token_accuracy': '0.7768', 'epoch': '0.07906'}
{'loss': '0.8335', 'grad_norm': '0.4016', 'learning_rate': '3.362e-05', 'entropy': '0.8075', 'num_tokens': '4.815e+07', 'mean_token_accuracy': '0.7983', 'epoch': '0.07907'}
{'loss': '1.085', 'grad_norm': '0.501', 'learning_rate': '3.36e-05', 'entropy': '1.02', 'num_tokens': '4.815e+07', 'mean_token_accuracy': '0.7468', 'epoch': '0.07908'}
{'loss': '0.7166', 'grad_norm': '0.4031', 'learning_rate': '3.358e-05', 'entropy': '0.6967', 'num_tokens': '4.816e+07', 'mean_token_accuracy': '0.8196', 'epoch': '0.07909'}
{'loss': '0.9675', 'grad_norm': '0.4592', 'learning_rate': '3.356e-05', 'entropy': '0.9301', 'num_tokens': '4.817e+07', 'mean_token_accuracy': '0.7746', 'epoch': '0.0791'}
{'loss': '0.6959', 'grad_norm': '0.4178', 'learning_rate': '3.354e-05', 'entropy': '0.6755', 'num_tokens': '4.817e+07', 'mean_token_accuracy': '0.8297', 'epoch': '0.07911'}
{'loss': '0.8738', 'grad_norm': '0.399', 'learning_rate': '3.352e-05', 'entropy': '0.8026', 'num_tokens': '4.818e+07', 'mean_token_accuracy': '0.8003', 'epoch': '0.07912'}
{'loss': '0.741', 'grad_norm': '0.3414', 'learning_rate': '3.35e-05', 'entropy': '0.7429', 'num_tokens': '4.818e+07', 'mean_token_accuracy': '0.8072', 'epoch': '0.07913'}
{'loss': '0.7906', 'grad_norm': '0.4188', 'learning_rate': '3.348e-05', 'entropy': '0.7159', 'num_tokens': '4.819e+07', 'mean_token_accuracy': '0.8095', 'epoch': '0.07914'}
{'loss': '0.899', 'grad_norm': '0.4768', 'learning_rate': '3.346e-05', 'entropy': '0.8913', 'num_tokens': '4.819e+07', 'mean_token_accuracy': '0.7684', 'epoch': '0.07914'}
{'loss': '0.6621', 'grad_norm': '0.3425', 'learning_rate': '3.344e-05', 'entropy': '0.6465', 'num_tokens': '4.82e+07', 'mean_token_accuracy': '0.8307', 'epoch': '0.07915'}
{'loss': '0.8395', 'grad_norm': '0.3661', 'learning_rate': '3.342e-05', 'entropy': '0.8926', 'num_tokens': '4.821e+07', 'mean_token_accuracy': '0.7834', 'epoch': '0.07916'}
{'loss': '0.5597', 'grad_norm': '0.3285', 'learning_rate': '3.339e-05', 'entropy': '0.5471', 'num_tokens': '4.821e+07', 'mean_token_accuracy': '0.8522', 'epoch': '0.07917'}
{'loss': '0.7473', 'grad_norm': '0.4491', 'learning_rate': '3.337e-05', 'entropy': '0.6973', 'num_tokens': '4.822e+07', 'mean_token_accuracy': '0.8134', 'epoch': '0.07918'}
{'loss': '0.9099', 'grad_norm': '0.5526', 'learning_rate': '3.335e-05', 'entropy': '0.9533', 'num_tokens': '4.822e+07', 'mean_token_accuracy': '0.7507', 'epoch': '0.07919'}
{'loss': '0.9998', 'grad_norm': '0.3256', 'learning_rate': '3.333e-05', 'entropy': '0.8244', 'num_tokens': '4.823e+07', 'mean_token_accuracy': '0.7966', 'epoch': '0.0792'}
{'loss': '1.213', 'grad_norm': '0.4794', 'learning_rate': '3.331e-05', 'entropy': '1.131', 'num_tokens': '4.824e+07', 'mean_token_accuracy': '0.7294', 'epoch': '0.07921'}
{'loss': '0.8296', 'grad_norm': '0.4001', 'learning_rate': '3.329e-05', 'entropy': '0.8342', 'num_tokens': '4.824e+07', 'mean_token_accuracy': '0.7908', 'epoch': '0.07921'}
{'loss': '0.5585', 'grad_norm': '0.309', 'learning_rate': '3.327e-05', 'entropy': '0.6176', 'num_tokens': '4.825e+07', 'mean_token_accuracy': '0.8447', 'epoch': '0.07922'}
{'loss': '0.8272', 'grad_norm': '0.4187', 'learning_rate': '3.325e-05', 'entropy': '0.7602', 'num_tokens': '4.825e+07', 'mean_token_accuracy': '0.8106', 'epoch': '0.07923'}
{'loss': '0.8225', 'grad_norm': '0.39', 'learning_rate': '3.323e-05', 'entropy': '0.7832', 'num_tokens': '4.826e+07', 'mean_token_accuracy': '0.8106', 'epoch': '0.07924'}
{'loss': '0.8488', 'grad_norm': '0.4665', 'learning_rate': '3.321e-05', 'entropy': '0.9302', 'num_tokens': '4.826e+07', 'mean_token_accuracy': '0.7694', 'epoch': '0.07925'}
{'loss': '0.7463', 'grad_norm': '0.2761', 'learning_rate': '3.319e-05', 'entropy': '0.7424', 'num_tokens': '4.827e+07', 'mean_token_accuracy': '0.8155', 'epoch': '0.07926'}
{'loss': '0.5101', 'grad_norm': '0.3306', 'learning_rate': '3.317e-05', 'entropy': '0.6237', 'num_tokens': '4.828e+07', 'mean_token_accuracy': '0.8385', 'epoch': '0.07927'}
{'loss': '0.7069', 'grad_norm': '0.4842', 'learning_rate': '3.315e-05', 'entropy': '0.7319', 'num_tokens': '4.828e+07', 'mean_token_accuracy': '0.8166', 'epoch': '0.07928'}
{'loss': '0.9325', 'grad_norm': '0.4153', 'learning_rate': '3.313e-05', 'entropy': '0.919', 'num_tokens': '4.828e+07', 'mean_token_accuracy': '0.779', 'epoch': '0.07929'}
{'loss': '0.9318', 'grad_norm': '0.5138', 'learning_rate': '3.311e-05', 'entropy': '0.977', 'num_tokens': '4.829e+07', 'mean_token_accuracy': '0.7575', 'epoch': '0.07929'}
{'loss': '0.7738', 'grad_norm': '0.3779', 'learning_rate': '3.309e-05', 'entropy': '0.7546', 'num_tokens': '4.83e+07', 'mean_token_accuracy': '0.8149', 'epoch': '0.0793'}
{'loss': '0.8484', 'grad_norm': '0.4284', 'learning_rate': '3.307e-05', 'entropy': '0.8114', 'num_tokens': '4.83e+07', 'mean_token_accuracy': '0.7951', 'epoch': '0.07931'}
{'loss': '0.8953', 'grad_norm': '0.3851', 'learning_rate': '3.305e-05', 'entropy': '0.8459', 'num_tokens': '4.831e+07', 'mean_token_accuracy': '0.7843', 'epoch': '0.07932'}
{'loss': '0.9524', 'grad_norm': '0.4179', 'learning_rate': '3.303e-05', 'entropy': '0.9174', 'num_tokens': '4.831e+07', 'mean_token_accuracy': '0.7662', 'epoch': '0.07933'}
{'loss': '0.7821', 'grad_norm': '0.377', 'learning_rate': '3.301e-05', 'entropy': '0.768', 'num_tokens': '4.832e+07', 'mean_token_accuracy': '0.8142', 'epoch': '0.07934'}
{'loss': '0.9196', 'grad_norm': '0.4536', 'learning_rate': '3.298e-05', 'entropy': '0.9163', 'num_tokens': '4.832e+07', 'mean_token_accuracy': '0.7734', 'epoch': '0.07935'}
{'loss': '0.7741', 'grad_norm': '0.3864', 'learning_rate': '3.296e-05', 'entropy': '0.8178', 'num_tokens': '4.833e+07', 'mean_token_accuracy': '0.7838', 'epoch': '0.07936'}
{'loss': '0.746', 'grad_norm': '0.3418', 'learning_rate': '3.294e-05', 'entropy': '0.7384', 'num_tokens': '4.833e+07', 'mean_token_accuracy': '0.8034', 'epoch': '0.07937'}
{'loss': '0.5148', 'grad_norm': '0.3006', 'learning_rate': '3.292e-05', 'entropy': '0.4928', 'num_tokens': '4.834e+07', 'mean_token_accuracy': '0.8613', 'epoch': '0.07937'}
{'loss': '0.8184', 'grad_norm': '0.4332', 'learning_rate': '3.29e-05', 'entropy': '0.8017', 'num_tokens': '4.834e+07', 'mean_token_accuracy': '0.81', 'epoch': '0.07938'}
{'loss': '0.7081', 'grad_norm': '0.384', 'learning_rate': '3.288e-05', 'entropy': '0.6856', 'num_tokens': '4.835e+07', 'mean_token_accuracy': '0.8158', 'epoch': '0.07939'}
{'loss': '0.746', 'grad_norm': '0.4184', 'learning_rate': '3.286e-05', 'entropy': '0.7965', 'num_tokens': '4.836e+07', 'mean_token_accuracy': '0.8051', 'epoch': '0.0794'}
{'loss': '0.6279', 'grad_norm': '0.3941', 'learning_rate': '3.284e-05', 'entropy': '0.631', 'num_tokens': '4.836e+07', 'mean_token_accuracy': '0.8269', 'epoch': '0.07941'}
{'loss': '0.5267', 'grad_norm': '0.3279', 'learning_rate': '3.282e-05', 'entropy': '0.5539', 'num_tokens': '4.837e+07', 'mean_token_accuracy': '0.8492', 'epoch': '0.07942'}
{'loss': '1.041', 'grad_norm': '0.4498', 'learning_rate': '3.28e-05', 'entropy': '1.013', 'num_tokens': '4.837e+07', 'mean_token_accuracy': '0.7563', 'epoch': '0.07943'}
{'loss': '1.001', 'grad_norm': '0.3948', 'learning_rate': '3.278e-05', 'entropy': '0.9938', 'num_tokens': '4.837e+07', 'mean_token_accuracy': '0.7549', 'epoch': '0.07944'}
{'loss': '1.232', 'grad_norm': '0.4541', 'learning_rate': '3.276e-05', 'entropy': '1.183', 'num_tokens': '4.838e+07', 'mean_token_accuracy': '0.7207', 'epoch': '0.07944'}
{'loss': '0.8587', 'grad_norm': '0.4433', 'learning_rate': '3.274e-05', 'entropy': '0.8974', 'num_tokens': '4.838e+07', 'mean_token_accuracy': '0.762', 'epoch': '0.07945'}
{'loss': '0.4943', 'grad_norm': '0.3428', 'learning_rate': '3.272e-05', 'entropy': '0.4894', 'num_tokens': '4.839e+07', 'mean_token_accuracy': '0.8637', 'epoch': '0.07946'}
{'loss': '1.064', 'grad_norm': '0.3988', 'learning_rate': '3.27e-05', 'entropy': '1.101', 'num_tokens': '4.839e+07', 'mean_token_accuracy': '0.7224', 'epoch': '0.07947'}

 75%|███████▍  | 8981/12000 [5:18:09<1:30:19,  1.80s/it]
 75%|███████▍  | 8982/12000 [5:18:11<1:35:48,  1.90s/it]
                                                        

 75%|███████▍  | 8982/12000 [5:18:11<1:35:48,  1.90s/it]
 75%|███████▍  | 8983/12000 [5:18:14<1:47:44,  2.14s/it]
                                                        

 75%|███████▍  | 8983/12000 [5:18:14<1:47:44,  2.14s/it]
 75%|███████▍  | 8984/12000 [5:18:15<1:37:40,  1.94s/it]
                                                        

 75%|███████▍  | 8984/12000 [5:18:15<1:37:40,  1.94s/it]
 75%|███████▍  | 8985/12000 [5:18:16<1:29:22,  1.78s/it]
                                                        

 75%|███████▍  | 8985/12000 [5:18:16<1:29:22,  1.78s/it]
 75%|███████▍  | 8986/12000 [5:18:20<2:01:21,  2.42s/it]
                                                        

 75%|███████▍  | 8986/12000 [5:18:20<2:01:21,  2.42s/it]
 75%|███████▍  | 8987/12000 [5:18:22<1:50:05,  2.19s/it]
                                                        

 75%|███████▍  | 8987/12000 [5:18:22<1:50:05,  2.19s/it]
 75%|███████▍  | 8988/12000 [5:18:24<1:46:13,  2.12s/it]
                                                        

 75%|███████▍  | 8988/12000 [5:18:24<1:46:13,  2.12s/it]
 75%|███████▍  | 8989/12000 [5:18:26<1:43:49,  2.07s/it]
                                                        

 75%|███████▍  | 8989/12000 [5:18:26<1:43:49,  2.07s/it]
 75%|███████▍  | 8990/12000 [5:18:27<1:33:04,  1.86s/it]
                                                        

 75%|███████▍  | 8990/12000 [5:18:27<1:33:04,  1.86s/it]
 75%|███████▍  | 8991/12000 [5:18:30<1:41:07,  2.02s/it]
                                                        

 75%|███████▍  | 8991/12000 [5:18:30<1:41:07,  2.02s/it]
 75%|███████▍  | 8992/12000 [5:18:31<1:36:29,  1.92s/it]
                                                        

 75%|███████▍  | 8992/12000 [5:18:31<1:36:29,  1.92s/it]
 75%|███████▍  | 8993/12000 [5:18:33<1:37:39,  1.95s/it]
                                                        

 75%|███████▍  | 8993/12000 [5:18:33<1:37:39,  1.95s/it]
 75%|███████▍  | 8994/12000 [5:18:35<1:37:59,  1.96s/it]
                                                        

 75%|███████▍  | 8994/12000 [5:18:35<1:37:59,  1.96s/it]
 75%|███████▍  | 8995/12000 [5:18:38<1:42:29,  2.05s/it]
                                                        

 75%|███████▍  | 8995/12000 [5:18:38<1:42:29,  2.05s/it]
 75%|███████▍  | 8996/12000 [5:18:39<1:39:17,  1.98s/it]
                                                        

 75%|███████▍  | 8996/12000 [5:18:39<1:39:17,  1.98s/it]
 75%|███████▍  | 8997/12000 [5:18:41<1:31:13,  1.82s/it]
                                                        

 75%|███████▍  | 8997/12000 [5:18:41<1:31:13,  1.82s/it]
 75%|███████▍  | 8998/12000 [5:18:43<1:28:19,  1.77s/it]
                                                        

 75%|███████▍  | 8998/12000 [5:18:43<1:28:19,  1.77s/it]
 75%|███████▍  | 8999/12000 [5:18:45<1:33:48,  1.88s/it]
                                                        

 75%|███████▍  | 8999/12000 [5:18:45<1:33:48,  1.88s/it]
 75%|███████▌  | 9000/12000 [5:18:46<1:28:53,  1.78s/it]
                                                        

 75%|███████▌  | 9000/12000 [5:18:46<1:28:53,  1.78s/it]
 75%|███████▌  | 9001/12000 [5:18:49<1:44:58,  2.10s/it]
                                                        

 75%|███████▌  | 9001/12000 [5:18:49<1:44:58,  2.10s/it]
 75%|███████▌  | 9002/12000 [5:18:51<1:46:52,  2.14s/it]
                                                        

 75%|███████▌  | 9002/12000 [5:18:51<1:46:52,  2.14s/it]
 75%|███████▌  | 9003/12000 [5:18:53<1:38:06,  1.96s/it]
                                                        

 75%|███████▌  | 9003/12000 [5:18:53<1:38:06,  1.96s/it]
 75%|███████▌  | 9004/12000 [5:18:54<1:30:43,  1.82s/it]
                                                        

 75%|███████▌  | 9004/12000 [5:18:54<1:30:43,  1.82s/it]
 75%|███████▌  | 9005/12000 [5:18:56<1:28:00,  1.76s/it]
                                                        

 75%|███████▌  | 9005/12000 [5:18:56<1:28:00,  1.76s/it]
 75%|███████▌  | 9006/12000 [5:18:58<1:28:59,  1.78s/it]
                                                        

 75%|███████▌  | 9006/12000 [5:18:58<1:28:59,  1.78s/it]
 75%|███████▌  | 9007/12000 [5:18:59<1:23:36,  1.68s/it]
                                                        

 75%|███████▌  | 9007/12000 [5:18:59<1:23:36,  1.68s/it]
 75%|███████▌  | 9008/12000 [5:19:03<2:00:49,  2.42s/it]
                                                        

 75%|███████▌  | 9008/12000 [5:19:03<2:00:49,  2.42s/it]
 75%|███████▌  | 9009/12000 [5:19:05<1:54:57,  2.31s/it]
                                                        

 75%|███████▌  | 9009/12000 [5:19:05<1:54:57,  2.31s/it]
 75%|███████▌  | 9010/12000 [5:19:07<1:45:13,  2.11s/it]
                                                        

 75%|███████▌  | 9010/12000 [5:19:07<1:45:13,  2.11s/it]
 75%|███████▌  | 9011/12000 [5:19:10<1:56:43,  2.34s/it]
                                                        

 75%|███████▌  | 9011/12000 [5:19:10<1:56:43,  2.34s/it]
 75%|███████▌  | 9012/12000 [5:19:12<1:46:20,  2.14s/it]
                                                        

 75%|███████▌  | 9012/12000 [5:19:12<1:46:20,  2.14s/it]
 75%|███████▌  | 9013/12000 [5:19:13<1:38:22,  1.98s/it]
                                                        

 75%|███████▌  | 9013/12000 [5:19:13<1:38:22,  1.98s/it]
 75%|███████▌  | 9014/12000 [5:19:16<1:50:53,  2.23s/it]
                                                        

 75%|███████▌  | 9014/12000 [5:19:16<1:50:53,  2.23s/it]
 75%|███████▌  | 9015/12000 [5:19:17<1:38:14,  1.97s/it]
                                                        

 75%|███████▌  | 9015/12000 [5:19:17<1:38:14,  1.97s/it]
 75%|███████▌  | 9016/12000 [5:19:20<1:49:42,  2.21s/it]
                                                        

 75%|███████▌  | 9016/12000 [5:19:20<1:49:42,  2.21s/it]
 75%|███████▌  | 9017/12000 [5:19:22<1:45:59,  2.13s/it]
                                                        

 75%|███████▌  | 9017/12000 [5:19:22<1:45:59,  2.13s/it]
 75%|███████▌  | 9018/12000 [5:19:24<1:41:35,  2.04s/it]
                                                        

 75%|███████▌  | 9018/12000 [5:19:24<1:41:35,  2.04s/it]
 75%|███████▌  | 9019/12000 [5:19:26<1:36:20,  1.94s/it]
                                                        

 75%|███████▌  | 9019/12000 [5:19:26<1:36:20,  1.94s/it]
 75%|███████▌  | 9020/12000 [5:19:27<1:31:54,  1.85s/it]
                                                        

 75%|███████▌  | 9020/12000 [5:19:27<1:31:54,  1.85s/it]
 75%|███████▌  | 9021/12000 [5:19:30<1:46:39,  2.15s/it]
                                                        

 75%|███████▌  | 9021/12000 [5:19:30<1:46:39,  2.15s/it]
 75%|███████▌  | 9022/12000 [5:19:32<1:38:07,  1.98s/it]
                                                        

 75%|███████▌  | 9022/12000 [5:19:32<1:38:07,  1.98s/it]
 75%|███████▌  | 9023/12000 [5:19:33<1:30:36,  1.83s/it]
                                                        

 75%|███████▌  | 9023/12000 [5:19:33<1:30:36,  1.83s/it]
 75%|███████▌  | 9024/12000 [5:19:36<1:43:34,  2.09s/it]
                                                        

 75%|███████▌  | 9024/12000 [5:19:36<1:43:34,  2.09s/it]
 75%|███████▌  | 9025/12000 [5:19:38<1:36:52,  1.95s/it]
                                                        

 75%|███████▌  | 9025/12000 [5:19:38<1:36:52,  1.95s/it]
 75%|███████▌  | 9026/12000 [5:19:39<1:30:04,  1.82s/it]
                                                        

 75%|███████▌  | 9026/12000 [5:19:39<1:30:04,  1.82s/it]
 75%|███████▌  | 9027/12000 [5:19:41<1:28:43,  1.79s/it]
                                                        

 75%|███████▌  | 9027/12000 [5:19:41<1:28:43,  1.79s/it]
 75%|███████▌  | 9028/12000 [5:19:43<1:37:57,  1.98s/it]
                                                        
{'loss': '0.7149', 'grad_norm': '0.4373', 'learning_rate': '3.268e-05', 'entropy': '0.6838', 'num_tokens': '4.84e+07', 'mean_token_accuracy': '0.8292', 'epoch': '0.07948'}
{'loss': '0.7588', 'grad_norm': '0.3787', 'learning_rate': '3.266e-05', 'entropy': '0.6933', 'num_tokens': '4.841e+07', 'mean_token_accuracy': '0.8195', 'epoch': '0.07949'}
{'loss': '1.169', 'grad_norm': '0.4285', 'learning_rate': '3.264e-05', 'entropy': '1.062', 'num_tokens': '4.841e+07', 'mean_token_accuracy': '0.7315', 'epoch': '0.0795'}
{'loss': '0.9908', 'grad_norm': '0.4498', 'learning_rate': '3.262e-05', 'entropy': '0.9753', 'num_tokens': '4.842e+07', 'mean_token_accuracy': '0.7537', 'epoch': '0.07951'}
{'loss': '0.9612', 'grad_norm': '0.5167', 'learning_rate': '3.26e-05', 'entropy': '0.9172', 'num_tokens': '4.842e+07', 'mean_token_accuracy': '0.7805', 'epoch': '0.07952'}
{'loss': '0.5986', 'grad_norm': '0.2891', 'learning_rate': '3.258e-05', 'entropy': '0.5556', 'num_tokens': '4.843e+07', 'mean_token_accuracy': '0.8569', 'epoch': '0.07952'}
{'loss': '0.7956', 'grad_norm': '0.4779', 'learning_rate': '3.256e-05', 'entropy': '0.7576', 'num_tokens': '4.843e+07', 'mean_token_accuracy': '0.7974', 'epoch': '0.07953'}
{'loss': '0.8458', 'grad_norm': '0.55', 'learning_rate': '3.254e-05', 'entropy': '0.8244', 'num_tokens': '4.844e+07', 'mean_token_accuracy': '0.7882', 'epoch': '0.07954'}
{'loss': '0.6108', 'grad_norm': '0.3449', 'learning_rate': '3.252e-05', 'entropy': '0.6119', 'num_tokens': '4.844e+07', 'mean_token_accuracy': '0.8394', 'epoch': '0.07955'}
{'loss': '0.9705', 'grad_norm': '0.411', 'learning_rate': '3.25e-05', 'entropy': '0.9609', 'num_tokens': '4.845e+07', 'mean_token_accuracy': '0.7731', 'epoch': '0.07956'}
{'loss': '0.9334', 'grad_norm': '0.4506', 'learning_rate': '3.247e-05', 'entropy': '0.9009', 'num_tokens': '4.845e+07', 'mean_token_accuracy': '0.7719', 'epoch': '0.07957'}
{'loss': '0.6865', 'grad_norm': '0.4458', 'learning_rate': '3.245e-05', 'entropy': '0.683', 'num_tokens': '4.846e+07', 'mean_token_accuracy': '0.8134', 'epoch': '0.07958'}
{'loss': '0.7102', 'grad_norm': '0.3601', 'learning_rate': '3.243e-05', 'entropy': '0.7276', 'num_tokens': '4.846e+07', 'mean_token_accuracy': '0.8194', 'epoch': '0.07959'}
{'loss': '0.8102', 'grad_norm': '0.4266', 'learning_rate': '3.241e-05', 'entropy': '0.7942', 'num_tokens': '4.847e+07', 'mean_token_accuracy': '0.7892', 'epoch': '0.0796'}
{'loss': '0.7647', 'grad_norm': '0.3028', 'learning_rate': '3.239e-05', 'entropy': '0.7296', 'num_tokens': '4.847e+07', 'mean_token_accuracy': '0.8171', 'epoch': '0.0796'}
{'loss': '0.6245', 'grad_norm': '0.4343', 'learning_rate': '3.237e-05', 'entropy': '0.6095', 'num_tokens': '4.848e+07', 'mean_token_accuracy': '0.8463', 'epoch': '0.07961'}
{'loss': '0.8462', 'grad_norm': '0.3894', 'learning_rate': '3.235e-05', 'entropy': '0.8617', 'num_tokens': '4.848e+07', 'mean_token_accuracy': '0.79', 'epoch': '0.07962'}
{'loss': '0.71', 'grad_norm': '0.4267', 'learning_rate': '3.233e-05', 'entropy': '0.7567', 'num_tokens': '4.849e+07', 'mean_token_accuracy': '0.7937', 'epoch': '0.07963'}
{'loss': '0.8804', 'grad_norm': '0.5094', 'learning_rate': '3.231e-05', 'entropy': '0.8937', 'num_tokens': '4.849e+07', 'mean_token_accuracy': '0.7748', 'epoch': '0.07964'}
{'loss': '1.102', 'grad_norm': '0.5513', 'learning_rate': '3.229e-05', 'entropy': '1.164', 'num_tokens': '4.85e+07', 'mean_token_accuracy': '0.7138', 'epoch': '0.07965'}
{'loss': '0.7906', 'grad_norm': '0.3942', 'learning_rate': '3.227e-05', 'entropy': '0.8062', 'num_tokens': '4.85e+07', 'mean_token_accuracy': '0.8017', 'epoch': '0.07966'}
{'loss': '0.4589', 'grad_norm': '0.2674', 'learning_rate': '3.225e-05', 'entropy': '0.4719', 'num_tokens': '4.851e+07', 'mean_token_accuracy': '0.8786', 'epoch': '0.07967'}
{'loss': '0.7116', 'grad_norm': '0.4019', 'learning_rate': '3.223e-05', 'entropy': '0.7061', 'num_tokens': '4.851e+07', 'mean_token_accuracy': '0.8175', 'epoch': '0.07968'}
{'loss': '0.99', 'grad_norm': '0.3883', 'learning_rate': '3.221e-05', 'entropy': '1.005', 'num_tokens': '4.852e+07', 'mean_token_accuracy': '0.756', 'epoch': '0.07968'}
{'loss': '0.6782', 'grad_norm': '0.5382', 'learning_rate': '3.219e-05', 'entropy': '0.7374', 'num_tokens': '4.852e+07', 'mean_token_accuracy': '0.8087', 'epoch': '0.07969'}
{'loss': '0.59', 'grad_norm': '0.5554', 'learning_rate': '3.217e-05', 'entropy': '0.5753', 'num_tokens': '4.853e+07', 'mean_token_accuracy': '0.8445', 'epoch': '0.0797'}
{'loss': '0.7399', 'grad_norm': '0.3843', 'learning_rate': '3.215e-05', 'entropy': '0.746', 'num_tokens': '4.853e+07', 'mean_token_accuracy': '0.8098', 'epoch': '0.07971'}
{'loss': '1.226', 'grad_norm': '0.5395', 'learning_rate': '3.213e-05', 'entropy': '1.168', 'num_tokens': '4.854e+07', 'mean_token_accuracy': '0.7159', 'epoch': '0.07972'}
{'loss': '0.7564', 'grad_norm': '0.3549', 'learning_rate': '3.211e-05', 'entropy': '0.8907', 'num_tokens': '4.854e+07', 'mean_token_accuracy': '0.7759', 'epoch': '0.07973'}
{'loss': '1.1', 'grad_norm': '0.5718', 'learning_rate': '3.209e-05', 'entropy': '1.109', 'num_tokens': '4.854e+07', 'mean_token_accuracy': '0.7259', 'epoch': '0.07974'}
{'loss': '0.7902', 'grad_norm': '0.3797', 'learning_rate': '3.207e-05', 'entropy': '0.8171', 'num_tokens': '4.855e+07', 'mean_token_accuracy': '0.8', 'epoch': '0.07975'}
{'loss': '0.58', 'grad_norm': '0.3487', 'learning_rate': '3.205e-05', 'entropy': '0.5855', 'num_tokens': '4.856e+07', 'mean_token_accuracy': '0.8482', 'epoch': '0.07975'}
{'loss': '0.7065', 'grad_norm': '0.3974', 'learning_rate': '3.203e-05', 'entropy': '0.7606', 'num_tokens': '4.856e+07', 'mean_token_accuracy': '0.8001', 'epoch': '0.07976'}
{'loss': '0.8933', 'grad_norm': '0.3975', 'learning_rate': '3.201e-05', 'entropy': '0.7837', 'num_tokens': '4.857e+07', 'mean_token_accuracy': '0.8004', 'epoch': '0.07977'}
{'loss': '0.5124', 'grad_norm': '0.3824', 'learning_rate': '3.199e-05', 'entropy': '0.5461', 'num_tokens': '4.857e+07', 'mean_token_accuracy': '0.8597', 'epoch': '0.07978'}
{'loss': '0.6333', 'grad_norm': '0.3358', 'learning_rate': '3.197e-05', 'entropy': '0.6826', 'num_tokens': '4.858e+07', 'mean_token_accuracy': '0.8334', 'epoch': '0.07979'}
{'loss': '0.8678', 'grad_norm': '0.3477', 'learning_rate': '3.195e-05', 'entropy': '0.8532', 'num_tokens': '4.859e+07', 'mean_token_accuracy': '0.7951', 'epoch': '0.0798'}
{'loss': '0.9586', 'grad_norm': '0.355', 'learning_rate': '3.193e-05', 'entropy': '0.9006', 'num_tokens': '4.859e+07', 'mean_token_accuracy': '0.7825', 'epoch': '0.07981'}
{'loss': '0.6137', 'grad_norm': '0.2991', 'learning_rate': '3.191e-05', 'entropy': '0.6599', 'num_tokens': '4.86e+07', 'mean_token_accuracy': '0.8391', 'epoch': '0.07982'}
{'loss': '0.6543', 'grad_norm': '0.3313', 'learning_rate': '3.189e-05', 'entropy': '0.6773', 'num_tokens': '4.86e+07', 'mean_token_accuracy': '0.8136', 'epoch': '0.07983'}
{'loss': '0.793', 'grad_norm': '0.4899', 'learning_rate': '3.187e-05', 'entropy': '0.7416', 'num_tokens': '4.861e+07', 'mean_token_accuracy': '0.8088', 'epoch': '0.07983'}
{'loss': '0.7399', 'grad_norm': '0.3889', 'learning_rate': '3.185e-05', 'entropy': '0.7359', 'num_tokens': '4.861e+07', 'mean_token_accuracy': '0.804', 'epoch': '0.07984'}
{'loss': '0.6781', 'grad_norm': '0.3982', 'learning_rate': '3.183e-05', 'entropy': '0.723', 'num_tokens': '4.862e+07', 'mean_token_accuracy': '0.8098', 'epoch': '0.07985'}
{'loss': '0.5908', 'grad_norm': '0.3488', 'learning_rate': '3.181e-05', 'entropy': '0.722', 'num_tokens': '4.863e+07', 'mean_token_accuracy': '0.8192', 'epoch': '0.07986'}
{'loss': '0.6963', 'grad_norm': '0.3917', 'learning_rate': '3.179e-05', 'entropy': '0.6801', 'num_tokens': '4.863e+07', 'mean_token_accuracy': '0.8231', 'epoch': '0.07987'}
{'loss': '0.7949', 'grad_norm': '0.4013', 'learning_rate': '3.177e-05', 'entropy': '0.8061', 'num_tokens': '4.863e+07', 'mean_token_accuracy': '0.7898', 'epoch': '0.07988'}
{'loss': '0.6526', 'grad_norm': '0.431', 'learning_rate': '3.175e-05', 'entropy': '0.7642', 'num_tokens': '4.864e+07', 'mean_token_accuracy': '0.8105', 'epoch': '0.07989'}

 75%|███████▌  | 9028/12000 [5:19:43<1:37:57,  1.98s/it]
 75%|███████▌  | 9029/12000 [5:19:45<1:28:53,  1.80s/it]
                                                        

 75%|███████▌  | 9029/12000 [5:19:45<1:28:53,  1.80s/it]
 75%|███████▌  | 9030/12000 [5:19:48<1:54:08,  2.31s/it]
                                                        

 75%|███████▌  | 9030/12000 [5:19:48<1:54:08,  2.31s/it]
 75%|███████▌  | 9031/12000 [5:19:50<1:44:40,  2.12s/it]
                                                        

 75%|███████▌  | 9031/12000 [5:19:50<1:44:40,  2.12s/it]
 75%|███████▌  | 9032/12000 [5:19:51<1:34:43,  1.91s/it]
                                                        

 75%|███████▌  | 9032/12000 [5:19:51<1:34:43,  1.91s/it]
 75%|███████▌  | 9033/12000 [5:19:54<1:45:28,  2.13s/it]
                                                        

 75%|███████▌  | 9033/12000 [5:19:54<1:45:28,  2.13s/it]
 75%|███████▌  | 9034/12000 [5:19:55<1:35:59,  1.94s/it]
                                                        

 75%|███████▌  | 9034/12000 [5:19:55<1:35:59,  1.94s/it]
 75%|███████▌  | 9035/12000 [5:19:57<1:33:04,  1.88s/it]
                                                        

 75%|███████▌  | 9035/12000 [5:19:57<1:33:04,  1.88s/it]
 75%|███████▌  | 9036/12000 [5:20:01<2:00:03,  2.43s/it]
                                                        

 75%|███████▌  | 9036/12000 [5:20:01<2:00:03,  2.43s/it]
 75%|███████▌  | 9037/12000 [5:20:02<1:46:11,  2.15s/it]
                                                        

 75%|███████▌  | 9037/12000 [5:20:02<1:46:11,  2.15s/it]
 75%|███████▌  | 9038/12000 [5:20:04<1:37:42,  1.98s/it]
                                                        

 75%|███████▌  | 9038/12000 [5:20:04<1:37:42,  1.98s/it]
 75%|███████▌  | 9039/12000 [5:20:05<1:30:17,  1.83s/it]
                                                        

 75%|███████▌  | 9039/12000 [5:20:05<1:30:17,  1.83s/it]
 75%|███████▌  | 9040/12000 [5:20:07<1:27:25,  1.77s/it]
                                                        

 75%|███████▌  | 9040/12000 [5:20:07<1:27:25,  1.77s/it]
 75%|███████▌  | 9041/12000 [5:20:09<1:26:45,  1.76s/it]
                                                        

 75%|███████▌  | 9041/12000 [5:20:09<1:26:45,  1.76s/it]
 75%|███████▌  | 9042/12000 [5:20:10<1:24:32,  1.71s/it]
                                                        

 75%|███████▌  | 9042/12000 [5:20:10<1:24:32,  1.71s/it]
 75%|███████▌  | 9043/12000 [5:20:12<1:24:27,  1.71s/it]
                                                        

 75%|███████▌  | 9043/12000 [5:20:12<1:24:27,  1.71s/it]
 75%|███████▌  | 9044/12000 [5:20:14<1:29:12,  1.81s/it]
                                                        

 75%|███████▌  | 9044/12000 [5:20:14<1:29:12,  1.81s/it]
 75%|███████▌  | 9045/12000 [5:20:16<1:36:45,  1.96s/it]
                                                        

 75%|███████▌  | 9045/12000 [5:20:16<1:36:45,  1.96s/it]
 75%|███████▌  | 9046/12000 [5:20:18<1:31:01,  1.85s/it]
                                                        

 75%|███████▌  | 9046/12000 [5:20:18<1:31:01,  1.85s/it]
 75%|███████▌  | 9047/12000 [5:20:19<1:24:21,  1.71s/it]
                                                        

 75%|███████▌  | 9047/12000 [5:20:19<1:24:21,  1.71s/it]
 75%|███████▌  | 9048/12000 [5:20:21<1:30:44,  1.84s/it]
                                                        

 75%|███████▌  | 9048/12000 [5:20:21<1:30:44,  1.84s/it]
 75%|███████▌  | 9049/12000 [5:20:23<1:26:52,  1.77s/it]
                                                        

 75%|███████▌  | 9049/12000 [5:20:23<1:26:52,  1.77s/it]
 75%|███████▌  | 9050/12000 [5:20:24<1:21:30,  1.66s/it]
                                                        

 75%|███████▌  | 9050/12000 [5:20:24<1:21:30,  1.66s/it]
 75%|███████▌  | 9051/12000 [5:20:27<1:40:28,  2.04s/it]
                                                        

 75%|███████▌  | 9051/12000 [5:20:27<1:40:28,  2.04s/it]
 75%|███████▌  | 9052/12000 [5:20:31<2:05:00,  2.54s/it]
                                                        

 75%|███████▌  | 9052/12000 [5:20:31<2:05:00,  2.54s/it]
 75%|███████▌  | 9053/12000 [5:20:33<2:00:30,  2.45s/it]
                                                        

 75%|███████▌  | 9053/12000 [5:20:33<2:00:30,  2.45s/it]
 75%|███████▌  | 9054/12000 [5:20:37<2:18:08,  2.81s/it]
                                                        

 75%|███████▌  | 9054/12000 [5:20:37<2:18:08,  2.81s/it]
 75%|███████▌  | 9055/12000 [5:20:39<2:00:21,  2.45s/it]
                                                        

 75%|███████▌  | 9055/12000 [5:20:39<2:00:21,  2.45s/it]
 75%|███████▌  | 9056/12000 [5:20:40<1:45:06,  2.14s/it]
                                                        

 75%|███████▌  | 9056/12000 [5:20:40<1:45:06,  2.14s/it]
 75%|███████▌  | 9057/12000 [5:20:42<1:36:44,  1.97s/it]
                                                        

 75%|███████▌  | 9057/12000 [5:20:42<1:36:44,  1.97s/it]
 75%|███████▌  | 9058/12000 [5:20:44<1:42:28,  2.09s/it]
                                                        

 75%|███████▌  | 9058/12000 [5:20:44<1:42:28,  2.09s/it]
 75%|███████▌  | 9059/12000 [5:20:45<1:34:05,  1.92s/it]
                                                        

 75%|███████▌  | 9059/12000 [5:20:45<1:34:05,  1.92s/it]
 76%|███████▌  | 9060/12000 [5:20:48<1:47:11,  2.19s/it]
                                                        

 76%|███████▌  | 9060/12000 [5:20:48<1:47:11,  2.19s/it]
 76%|███████▌  | 9061/12000 [5:20:50<1:43:32,  2.11s/it]
                                                        

 76%|███████▌  | 9061/12000 [5:20:50<1:43:32,  2.11s/it]
 76%|███████▌  | 9062/12000 [5:20:53<1:46:48,  2.18s/it]
                                                        

 76%|███████▌  | 9062/12000 [5:20:53<1:46:48,  2.18s/it]
 76%|███████▌  | 9063/12000 [5:20:54<1:37:13,  1.99s/it]
                                                        

 76%|███████▌  | 9063/12000 [5:20:54<1:37:13,  1.99s/it]
 76%|███████▌  | 9064/12000 [5:20:56<1:35:32,  1.95s/it]
                                                        

 76%|███████▌  | 9064/12000 [5:20:56<1:35:32,  1.95s/it]
 76%|███████▌  | 9065/12000 [5:20:58<1:35:48,  1.96s/it]
                                                        

 76%|███████▌  | 9065/12000 [5:20:58<1:35:48,  1.96s/it]
 76%|███████▌  | 9066/12000 [5:21:00<1:35:55,  1.96s/it]
                                                        

 76%|███████▌  | 9066/12000 [5:21:00<1:35:55,  1.96s/it]
 76%|███████▌  | 9067/12000 [5:21:02<1:33:21,  1.91s/it]
                                                        

 76%|███████▌  | 9067/12000 [5:21:02<1:33:21,  1.91s/it]
 76%|███████▌  | 9068/12000 [5:21:04<1:34:52,  1.94s/it]
                                                        

 76%|███████▌  | 9068/12000 [5:21:04<1:34:52,  1.94s/it]
 76%|███████▌  | 9069/12000 [5:21:07<1:50:27,  2.26s/it]
                                                        

 76%|███████▌  | 9069/12000 [5:21:07<1:50:27,  2.26s/it]
 76%|███████▌  | 9070/12000 [5:21:08<1:39:02,  2.03s/it]
                                                        

 76%|███████▌  | 9070/12000 [5:21:08<1:39:02,  2.03s/it]
 76%|███████▌  | 9071/12000 [5:21:10<1:36:05,  1.97s/it]
                                                        

 76%|███████▌  | 9071/12000 [5:21:10<1:36:05,  1.97s/it]
 76%|███████▌  | 9072/12000 [5:21:12<1:41:22,  2.08s/it]
                                                        

 76%|███████▌  | 9072/12000 [5:21:12<1:41:22,  2.08s/it]
 76%|███████▌  | 9073/12000 [5:21:14<1:40:55,  2.07s/it]
                                                        

 76%|███████▌  | 9073/12000 [5:21:14<1:40:55,  2.07s/it]
 76%|███████▌  | 9074/12000 [5:21:19<2:22:37,  2.92s/it]
                                                        

 76%|███████▌  | 9074/12000 [5:21:19<2:22:37,  2.92s/it]
 76%|███████▌  | 9075/12000 [5:21:21<2:07:01,  2.61s/it]
                                                        
{'loss': '0.6542', 'grad_norm': '0.361', 'learning_rate': '3.173e-05', 'entropy': '0.6361', 'num_tokens': '4.865e+07', 'mean_token_accuracy': '0.838', 'epoch': '0.0799'}
{'loss': '0.7136', 'grad_norm': '0.3585', 'learning_rate': '3.171e-05', 'entropy': '0.7678', 'num_tokens': '4.865e+07', 'mean_token_accuracy': '0.7988', 'epoch': '0.07991'}
{'loss': '0.7539', 'grad_norm': '0.4025', 'learning_rate': '3.169e-05', 'entropy': '0.7137', 'num_tokens': '4.866e+07', 'mean_token_accuracy': '0.8181', 'epoch': '0.07991'}
{'loss': '0.9117', 'grad_norm': '0.4614', 'learning_rate': '3.167e-05', 'entropy': '0.8611', 'num_tokens': '4.866e+07', 'mean_token_accuracy': '0.7814', 'epoch': '0.07992'}
{'loss': '0.8579', 'grad_norm': '0.43', 'learning_rate': '3.165e-05', 'entropy': '0.8611', 'num_tokens': '4.866e+07', 'mean_token_accuracy': '0.787', 'epoch': '0.07993'}
{'loss': '1.021', 'grad_norm': '0.5487', 'learning_rate': '3.163e-05', 'entropy': '0.9718', 'num_tokens': '4.867e+07', 'mean_token_accuracy': '0.7585', 'epoch': '0.07994'}
{'loss': '1.288', 'grad_norm': '0.5248', 'learning_rate': '3.161e-05', 'entropy': '1.171', 'num_tokens': '4.867e+07', 'mean_token_accuracy': '0.7129', 'epoch': '0.07995'}
{'loss': '1.195', 'grad_norm': '0.3978', 'learning_rate': '3.159e-05', 'entropy': '1.192', 'num_tokens': '4.868e+07', 'mean_token_accuracy': '0.7246', 'epoch': '0.07996'}
{'loss': '0.9129', 'grad_norm': '0.4696', 'learning_rate': '3.157e-05', 'entropy': '0.9148', 'num_tokens': '4.869e+07', 'mean_token_accuracy': '0.7749', 'epoch': '0.07997'}
{'loss': '0.6341', 'grad_norm': '0.292', 'learning_rate': '3.155e-05', 'entropy': '0.6304', 'num_tokens': '4.869e+07', 'mean_token_accuracy': '0.8241', 'epoch': '0.07998'}
{'loss': '0.5405', 'grad_norm': '0.3091', 'learning_rate': '3.153e-05', 'entropy': '0.5951', 'num_tokens': '4.87e+07', 'mean_token_accuracy': '0.8457', 'epoch': '0.07998'}
{'loss': '0.8165', 'grad_norm': '0.3449', 'learning_rate': '3.151e-05', 'entropy': '0.8481', 'num_tokens': '4.87e+07', 'mean_token_accuracy': '0.7987', 'epoch': '0.07999'}
{'loss': '0.8543', 'grad_norm': '0.4193', 'learning_rate': '3.149e-05', 'entropy': '0.8976', 'num_tokens': '4.871e+07', 'mean_token_accuracy': '0.767', 'epoch': '0.08'}
{'loss': '1.288', 'grad_norm': '0.4574', 'learning_rate': '3.146e-05', 'entropy': '1.259', 'num_tokens': '4.871e+07', 'mean_token_accuracy': '0.7035', 'epoch': '0.08001'}
{'loss': '1.01', 'grad_norm': '0.5156', 'learning_rate': '3.144e-05', 'entropy': '0.9303', 'num_tokens': '4.871e+07', 'mean_token_accuracy': '0.7776', 'epoch': '0.08002'}
{'loss': '0.7612', 'grad_norm': '0.3899', 'learning_rate': '3.142e-05', 'entropy': '0.8303', 'num_tokens': '4.872e+07', 'mean_token_accuracy': '0.7988', 'epoch': '0.08003'}
{'loss': '0.7387', 'grad_norm': '0.3652', 'learning_rate': '3.14e-05', 'entropy': '0.7283', 'num_tokens': '4.873e+07', 'mean_token_accuracy': '0.8263', 'epoch': '0.08004'}
{'loss': '0.9204', 'grad_norm': '0.3852', 'learning_rate': '3.138e-05', 'entropy': '0.9632', 'num_tokens': '4.873e+07', 'mean_token_accuracy': '0.7609', 'epoch': '0.08005'}
{'loss': '0.6616', 'grad_norm': '0.4574', 'learning_rate': '3.136e-05', 'entropy': '0.7265', 'num_tokens': '4.874e+07', 'mean_token_accuracy': '0.8076', 'epoch': '0.08006'}
{'loss': '0.998', 'grad_norm': '0.4591', 'learning_rate': '3.134e-05', 'entropy': '0.9481', 'num_tokens': '4.874e+07', 'mean_token_accuracy': '0.7728', 'epoch': '0.08006'}
{'loss': '1.144', 'grad_norm': '0.4176', 'learning_rate': '3.132e-05', 'entropy': '0.9849', 'num_tokens': '4.875e+07', 'mean_token_accuracy': '0.7647', 'epoch': '0.08007'}
{'loss': '0.6784', 'grad_norm': '0.3892', 'learning_rate': '3.13e-05', 'entropy': '0.6728', 'num_tokens': '4.875e+07', 'mean_token_accuracy': '0.8252', 'epoch': '0.08008'}
{'loss': '0.7328', 'grad_norm': '0.4073', 'learning_rate': '3.128e-05', 'entropy': '0.735', 'num_tokens': '4.876e+07', 'mean_token_accuracy': '0.813', 'epoch': '0.08009'}
{'loss': '0.6729', 'grad_norm': '0.3819', 'learning_rate': '3.126e-05', 'entropy': '0.8082', 'num_tokens': '4.876e+07', 'mean_token_accuracy': '0.799', 'epoch': '0.0801'}
{'loss': '0.7537', 'grad_norm': '0.4826', 'learning_rate': '3.124e-05', 'entropy': '0.763', 'num_tokens': '4.877e+07', 'mean_token_accuracy': '0.8028', 'epoch': '0.08011'}
{'loss': '0.5902', 'grad_norm': '0.3547', 'learning_rate': '3.122e-05', 'entropy': '0.6149', 'num_tokens': '4.877e+07', 'mean_token_accuracy': '0.8517', 'epoch': '0.08012'}
{'loss': '0.9438', 'grad_norm': '0.3709', 'learning_rate': '3.12e-05', 'entropy': '0.9868', 'num_tokens': '4.878e+07', 'mean_token_accuracy': '0.7419', 'epoch': '0.08013'}
{'loss': '0.6094', 'grad_norm': '0.3557', 'learning_rate': '3.118e-05', 'entropy': '0.6918', 'num_tokens': '4.878e+07', 'mean_token_accuracy': '0.8272', 'epoch': '0.08014'}
{'loss': '0.7062', 'grad_norm': '0.3495', 'learning_rate': '3.116e-05', 'entropy': '0.7675', 'num_tokens': '4.879e+07', 'mean_token_accuracy': '0.8004', 'epoch': '0.08014'}
{'loss': '0.614', 'grad_norm': '0.43', 'learning_rate': '3.114e-05', 'entropy': '0.6434', 'num_tokens': '4.879e+07', 'mean_token_accuracy': '0.8388', 'epoch': '0.08015'}
{'loss': '1.076', 'grad_norm': '0.4365', 'learning_rate': '3.112e-05', 'entropy': '1.041', 'num_tokens': '4.88e+07', 'mean_token_accuracy': '0.7493', 'epoch': '0.08016'}
{'loss': '0.718', 'grad_norm': '0.3624', 'learning_rate': '3.11e-05', 'entropy': '0.7395', 'num_tokens': '4.88e+07', 'mean_token_accuracy': '0.816', 'epoch': '0.08017'}
{'loss': '0.702', 'grad_norm': '0.4164', 'learning_rate': '3.108e-05', 'entropy': '0.661', 'num_tokens': '4.881e+07', 'mean_token_accuracy': '0.8301', 'epoch': '0.08018'}
{'loss': '0.6939', 'grad_norm': '0.4041', 'learning_rate': '3.106e-05', 'entropy': '0.7947', 'num_tokens': '4.881e+07', 'mean_token_accuracy': '0.8037', 'epoch': '0.08019'}
{'loss': '1.276', 'grad_norm': '0.429', 'learning_rate': '3.104e-05', 'entropy': '1.115', 'num_tokens': '4.882e+07', 'mean_token_accuracy': '0.7238', 'epoch': '0.0802'}
{'loss': '0.6141', 'grad_norm': '0.4165', 'learning_rate': '3.102e-05', 'entropy': '0.6323', 'num_tokens': '4.882e+07', 'mean_token_accuracy': '0.8313', 'epoch': '0.08021'}
{'loss': '0.7714', 'grad_norm': '0.4189', 'learning_rate': '3.1e-05', 'entropy': '0.773', 'num_tokens': '4.883e+07', 'mean_token_accuracy': '0.8046', 'epoch': '0.08021'}
{'loss': '0.8292', 'grad_norm': '0.4568', 'learning_rate': '3.098e-05', 'entropy': '0.8567', 'num_tokens': '4.883e+07', 'mean_token_accuracy': '0.7853', 'epoch': '0.08022'}
{'loss': '0.5943', 'grad_norm': '0.4661', 'learning_rate': '3.096e-05', 'entropy': '0.6404', 'num_tokens': '4.884e+07', 'mean_token_accuracy': '0.8317', 'epoch': '0.08023'}
{'loss': '0.6901', 'grad_norm': '0.3181', 'learning_rate': '3.094e-05', 'entropy': '0.7676', 'num_tokens': '4.884e+07', 'mean_token_accuracy': '0.7941', 'epoch': '0.08024'}
{'loss': '1.125', 'grad_norm': '0.568', 'learning_rate': '3.093e-05', 'entropy': '1.065', 'num_tokens': '4.885e+07', 'mean_token_accuracy': '0.731', 'epoch': '0.08025'}
{'loss': '1.143', 'grad_norm': '0.4479', 'learning_rate': '3.091e-05', 'entropy': '0.9473', 'num_tokens': '4.886e+07', 'mean_token_accuracy': '0.7619', 'epoch': '0.08026'}
{'loss': '1.052', 'grad_norm': '0.4982', 'learning_rate': '3.089e-05', 'entropy': '0.9807', 'num_tokens': '4.886e+07', 'mean_token_accuracy': '0.7637', 'epoch': '0.08027'}
{'loss': '1.261', 'grad_norm': '0.4337', 'learning_rate': '3.087e-05', 'entropy': '1.222', 'num_tokens': '4.887e+07', 'mean_token_accuracy': '0.7178', 'epoch': '0.08028'}
{'loss': '0.8196', 'grad_norm': '0.3946', 'learning_rate': '3.085e-05', 'entropy': '0.8616', 'num_tokens': '4.887e+07', 'mean_token_accuracy': '0.7856', 'epoch': '0.08029'}
{'loss': '0.7802', 'grad_norm': '0.4062', 'learning_rate': '3.083e-05', 'entropy': '0.7407', 'num_tokens': '4.888e+07', 'mean_token_accuracy': '0.8001', 'epoch': '0.08029'}
{'loss': '0.7566', 'grad_norm': '0.4553', 'learning_rate': '3.081e-05', 'entropy': '0.7723', 'num_tokens': '4.888e+07', 'mean_token_accuracy': '0.7978', 'epoch': '0.0803'}

 76%|███████▌  | 9075/12000 [5:21:21<2:07:01,  2.61s/it]
 76%|███████▌  | 9076/12000 [5:21:25<2:23:38,  2.95s/it]
                                                        

 76%|███████▌  | 9076/12000 [5:21:25<2:23:38,  2.95s/it]
 76%|███████▌  | 9077/12000 [5:21:27<2:06:05,  2.59s/it]
                                                        

 76%|███████▌  | 9077/12000 [5:21:27<2:06:05,  2.59s/it]
 76%|███████▌  | 9078/12000 [5:21:28<1:52:53,  2.32s/it]
                                                        

 76%|███████▌  | 9078/12000 [5:21:28<1:52:53,  2.32s/it]
 76%|███████▌  | 9079/12000 [5:21:31<1:49:49,  2.26s/it]
                                                        

 76%|███████▌  | 9079/12000 [5:21:31<1:49:49,  2.26s/it]
 76%|███████▌  | 9080/12000 [5:21:32<1:38:54,  2.03s/it]
                                                        

 76%|███████▌  | 9080/12000 [5:21:32<1:38:54,  2.03s/it]
 76%|███████▌  | 9081/12000 [5:21:34<1:32:23,  1.90s/it]
                                                        

 76%|███████▌  | 9081/12000 [5:21:34<1:32:23,  1.90s/it]
 76%|███████▌  | 9082/12000 [5:21:36<1:36:24,  1.98s/it]
                                                        

 76%|███████▌  | 9082/12000 [5:21:36<1:36:24,  1.98s/it]
 76%|███████▌  | 9083/12000 [5:21:38<1:36:35,  1.99s/it]
                                                        

 76%|███████▌  | 9083/12000 [5:21:38<1:36:35,  1.99s/it]
 76%|███████▌  | 9084/12000 [5:21:39<1:30:07,  1.85s/it]
                                                        

 76%|███████▌  | 9084/12000 [5:21:39<1:30:07,  1.85s/it]
 76%|███████▌  | 9085/12000 [5:21:41<1:23:39,  1.72s/it]
                                                        

 76%|███████▌  | 9085/12000 [5:21:41<1:23:39,  1.72s/it]
 76%|███████▌  | 9086/12000 [5:21:42<1:20:07,  1.65s/it]
                                                        

 76%|███████▌  | 9086/12000 [5:21:42<1:20:07,  1.65s/it]
 76%|███████▌  | 9087/12000 [5:21:44<1:19:07,  1.63s/it]
                                                        

 76%|███████▌  | 9087/12000 [5:21:44<1:19:07,  1.63s/it]
 76%|███████▌  | 9088/12000 [5:21:46<1:33:41,  1.93s/it]
                                                        

 76%|███████▌  | 9088/12000 [5:21:46<1:33:41,  1.93s/it]
 76%|███████▌  | 9089/12000 [5:21:48<1:27:42,  1.81s/it]
                                                        

 76%|███████▌  | 9089/12000 [5:21:48<1:27:42,  1.81s/it]
 76%|███████▌  | 9090/12000 [5:21:50<1:26:20,  1.78s/it]
                                                        

 76%|███████▌  | 9090/12000 [5:21:50<1:26:20,  1.78s/it]
 76%|███████▌  | 9091/12000 [5:21:51<1:24:02,  1.73s/it]
                                                        

 76%|███████▌  | 9091/12000 [5:21:51<1:24:02,  1.73s/it]
 76%|███████▌  | 9092/12000 [5:21:53<1:20:04,  1.65s/it]
                                                        

 76%|███████▌  | 9092/12000 [5:21:53<1:20:04,  1.65s/it]
 76%|███████▌  | 9093/12000 [5:21:54<1:18:06,  1.61s/it]
                                                        

 76%|███████▌  | 9093/12000 [5:21:54<1:18:06,  1.61s/it]
 76%|███████▌  | 9094/12000 [5:21:56<1:21:20,  1.68s/it]
                                                        

 76%|███████▌  | 9094/12000 [5:21:56<1:21:20,  1.68s/it]
 76%|███████▌  | 9095/12000 [5:21:58<1:29:20,  1.85s/it]
                                                        

 76%|███████▌  | 9095/12000 [5:21:58<1:29:20,  1.85s/it]
 76%|███████▌  | 9096/12000 [5:22:03<2:11:01,  2.71s/it]
                                                        

 76%|███████▌  | 9096/12000 [5:22:03<2:11:01,  2.71s/it]
 76%|███████▌  | 9097/12000 [5:22:06<2:12:12,  2.73s/it]
                                                        

 76%|███████▌  | 9097/12000 [5:22:06<2:12:12,  2.73s/it]
 76%|███████▌  | 9098/12000 [5:22:08<2:00:34,  2.49s/it]
                                                        

 76%|███████▌  | 9098/12000 [5:22:08<2:00:34,  2.49s/it]
 76%|███████▌  | 9099/12000 [5:22:09<1:47:47,  2.23s/it]
                                                        

 76%|███████▌  | 9099/12000 [5:22:09<1:47:47,  2.23s/it]
 76%|███████▌  | 9100/12000 [5:22:11<1:36:19,  1.99s/it]
                                                        

 76%|███████▌  | 9100/12000 [5:22:11<1:36:19,  1.99s/it]
 76%|███████▌  | 9101/12000 [5:22:15<2:07:50,  2.65s/it]
                                                        

 76%|███████▌  | 9101/12000 [5:22:15<2:07:50,  2.65s/it]
 76%|███████▌  | 9102/12000 [5:22:17<1:53:28,  2.35s/it]
                                                        

 76%|███████▌  | 9102/12000 [5:22:17<1:53:28,  2.35s/it]
 76%|███████▌  | 9103/12000 [5:22:18<1:42:28,  2.12s/it]
                                                        

 76%|███████▌  | 9103/12000 [5:22:18<1:42:28,  2.12s/it]
 76%|███████▌  | 9104/12000 [5:22:21<1:44:11,  2.16s/it]
                                                        

 76%|███████▌  | 9104/12000 [5:22:21<1:44:11,  2.16s/it]
 76%|███████▌  | 9105/12000 [5:22:22<1:33:37,  1.94s/it]
                                                        

 76%|███████▌  | 9105/12000 [5:22:22<1:33:37,  1.94s/it]
 76%|███████▌  | 9106/12000 [5:22:24<1:31:48,  1.90s/it]
                                                        

 76%|███████▌  | 9106/12000 [5:22:24<1:31:48,  1.90s/it]
 76%|███████▌  | 9107/12000 [5:22:26<1:34:25,  1.96s/it]
                                                        

 76%|███████▌  | 9107/12000 [5:22:26<1:34:25,  1.96s/it]
 76%|███████▌  | 9108/12000 [5:22:28<1:34:57,  1.97s/it]
                                                        

 76%|███████▌  | 9108/12000 [5:22:28<1:34:57,  1.97s/it]
 76%|███████▌  | 9109/12000 [5:22:31<1:58:46,  2.47s/it]
                                                        

 76%|███████▌  | 9109/12000 [5:22:31<1:58:46,  2.47s/it]
 76%|███████▌  | 9110/12000 [5:22:33<1:42:44,  2.13s/it]
                                                        

 76%|███████▌  | 9110/12000 [5:22:33<1:42:44,  2.13s/it]
 76%|███████▌  | 9111/12000 [5:22:35<1:37:16,  2.02s/it]
                                                        

 76%|███████▌  | 9111/12000 [5:22:35<1:37:16,  2.02s/it]
 76%|███████▌  | 9112/12000 [5:22:36<1:33:16,  1.94s/it]
                                                        

 76%|███████▌  | 9112/12000 [5:22:36<1:33:16,  1.94s/it]
 76%|███████▌  | 9113/12000 [5:22:38<1:28:14,  1.83s/it]
                                                        

 76%|███████▌  | 9113/12000 [5:22:38<1:28:14,  1.83s/it]
 76%|███████▌  | 9114/12000 [5:22:39<1:23:20,  1.73s/it]
                                                        

 76%|███████▌  | 9114/12000 [5:22:39<1:23:20,  1.73s/it]
 76%|███████▌  | 9115/12000 [5:22:41<1:21:04,  1.69s/it]
                                                        

 76%|███████▌  | 9115/12000 [5:22:41<1:21:04,  1.69s/it]
 76%|███████▌  | 9116/12000 [5:22:42<1:18:26,  1.63s/it]
                                                        

 76%|███████▌  | 9116/12000 [5:22:43<1:18:26,  1.63s/it]
 76%|███████▌  | 9117/12000 [5:22:47<2:02:41,  2.55s/it]
                                                        

 76%|███████▌  | 9117/12000 [5:22:47<2:02:41,  2.55s/it]
 76%|███████▌  | 9118/12000 [5:22:49<1:55:08,  2.40s/it]
                                                        

 76%|███████▌  | 9118/12000 [5:22:49<1:55:08,  2.40s/it]
 76%|███████▌  | 9119/12000 [5:22:51<1:50:23,  2.30s/it]
                                                        

 76%|███████▌  | 9119/12000 [5:22:51<1:50:23,  2.30s/it]
 76%|███████▌  | 9120/12000 [5:22:53<1:41:57,  2.12s/it]
                                                        

 76%|███████▌  | 9120/12000 [5:22:53<1:41:57,  2.12s/it]
 76%|███████▌  | 9121/12000 [5:22:55<1:42:29,  2.14s/it]
                                                        

 76%|███████▌  | 9121/12000 [5:22:55<1:42:29,  2.14s/it]
 76%|███████▌  | 9122/12000 [5:22:57<1:32:14,  1.92s/it]
                                                        
{'loss': '0.8243', 'grad_norm': '0.4857', 'learning_rate': '3.079e-05', 'entropy': '0.8243', 'num_tokens': '4.889e+07', 'mean_token_accuracy': '0.7759', 'epoch': '0.08031'}
{'loss': '0.5767', 'grad_norm': '0.4077', 'learning_rate': '3.077e-05', 'entropy': '0.6652', 'num_tokens': '4.89e+07', 'mean_token_accuracy': '0.8325', 'epoch': '0.08032'}
{'loss': '0.6782', 'grad_norm': '0.4436', 'learning_rate': '3.075e-05', 'entropy': '0.6954', 'num_tokens': '4.89e+07', 'mean_token_accuracy': '0.8215', 'epoch': '0.08033'}
{'loss': '0.7239', 'grad_norm': '0.3814', 'learning_rate': '3.073e-05', 'entropy': '0.7165', 'num_tokens': '4.891e+07', 'mean_token_accuracy': '0.8268', 'epoch': '0.08034'}
{'loss': '0.6215', 'grad_norm': '0.3292', 'learning_rate': '3.071e-05', 'entropy': '0.6232', 'num_tokens': '4.892e+07', 'mean_token_accuracy': '0.8376', 'epoch': '0.08035'}
{'loss': '0.8116', 'grad_norm': '0.4754', 'learning_rate': '3.069e-05', 'entropy': '0.8335', 'num_tokens': '4.892e+07', 'mean_token_accuracy': '0.7949', 'epoch': '0.08036'}
{'loss': '0.7028', 'grad_norm': '0.4587', 'learning_rate': '3.067e-05', 'entropy': '0.7117', 'num_tokens': '4.893e+07', 'mean_token_accuracy': '0.81', 'epoch': '0.08037'}
{'loss': '0.895', 'grad_norm': '0.4598', 'learning_rate': '3.065e-05', 'entropy': '0.8127', 'num_tokens': '4.893e+07', 'mean_token_accuracy': '0.79', 'epoch': '0.08037'}
{'loss': '0.8819', 'grad_norm': '0.459', 'learning_rate': '3.063e-05', 'entropy': '0.838', 'num_tokens': '4.894e+07', 'mean_token_accuracy': '0.783', 'epoch': '0.08038'}
{'loss': '0.6317', 'grad_norm': '0.4684', 'learning_rate': '3.061e-05', 'entropy': '0.629', 'num_tokens': '4.894e+07', 'mean_token_accuracy': '0.8271', 'epoch': '0.08039'}
{'loss': '0.8527', 'grad_norm': '0.4028', 'learning_rate': '3.059e-05', 'entropy': '0.8935', 'num_tokens': '4.895e+07', 'mean_token_accuracy': '0.7741', 'epoch': '0.0804'}
{'loss': '0.7033', 'grad_norm': '0.3665', 'learning_rate': '3.057e-05', 'entropy': '0.7651', 'num_tokens': '4.895e+07', 'mean_token_accuracy': '0.8022', 'epoch': '0.08041'}
{'loss': '0.5921', 'grad_norm': '0.3526', 'learning_rate': '3.055e-05', 'entropy': '0.6265', 'num_tokens': '4.896e+07', 'mean_token_accuracy': '0.832', 'epoch': '0.08042'}
{'loss': '1.017', 'grad_norm': '0.3554', 'learning_rate': '3.053e-05', 'entropy': '1.064', 'num_tokens': '4.896e+07', 'mean_token_accuracy': '0.7393', 'epoch': '0.08043'}
{'loss': '0.7964', 'grad_norm': '0.3982', 'learning_rate': '3.051e-05', 'entropy': '0.8029', 'num_tokens': '4.897e+07', 'mean_token_accuracy': '0.7963', 'epoch': '0.08044'}
{'loss': '0.7241', 'grad_norm': '0.3955', 'learning_rate': '3.049e-05', 'entropy': '0.8007', 'num_tokens': '4.897e+07', 'mean_token_accuracy': '0.8044', 'epoch': '0.08044'}
{'loss': '1.041', 'grad_norm': '0.3966', 'learning_rate': '3.047e-05', 'entropy': '0.9583', 'num_tokens': '4.898e+07', 'mean_token_accuracy': '0.7623', 'epoch': '0.08045'}
{'loss': '0.9159', 'grad_norm': '0.5608', 'learning_rate': '3.045e-05', 'entropy': '0.9056', 'num_tokens': '4.898e+07', 'mean_token_accuracy': '0.7781', 'epoch': '0.08046'}
{'loss': '0.7603', 'grad_norm': '0.47', 'learning_rate': '3.043e-05', 'entropy': '0.7362', 'num_tokens': '4.898e+07', 'mean_token_accuracy': '0.8047', 'epoch': '0.08047'}
{'loss': '1.159', 'grad_norm': '0.4562', 'learning_rate': '3.041e-05', 'entropy': '1.118', 'num_tokens': '4.899e+07', 'mean_token_accuracy': '0.7275', 'epoch': '0.08048'}
{'loss': '0.9571', 'grad_norm': '0.431', 'learning_rate': '3.039e-05', 'entropy': '0.8371', 'num_tokens': '4.899e+07', 'mean_token_accuracy': '0.7927', 'epoch': '0.08049'}
{'loss': '0.6394', 'grad_norm': '0.3389', 'learning_rate': '3.037e-05', 'entropy': '0.6106', 'num_tokens': '4.9e+07', 'mean_token_accuracy': '0.8435', 'epoch': '0.0805'}
{'loss': '0.6452', 'grad_norm': '0.3795', 'learning_rate': '3.035e-05', 'entropy': '0.6553', 'num_tokens': '4.901e+07', 'mean_token_accuracy': '0.8246', 'epoch': '0.08051'}
{'loss': '0.9241', 'grad_norm': '0.4127', 'learning_rate': '3.033e-05', 'entropy': '0.9062', 'num_tokens': '4.901e+07', 'mean_token_accuracy': '0.7777', 'epoch': '0.08052'}
{'loss': '1.208', 'grad_norm': '0.44', 'learning_rate': '3.031e-05', 'entropy': '1.152', 'num_tokens': '4.902e+07', 'mean_token_accuracy': '0.7227', 'epoch': '0.08052'}
{'loss': '0.6183', 'grad_norm': '0.3988', 'learning_rate': '3.029e-05', 'entropy': '0.6713', 'num_tokens': '4.902e+07', 'mean_token_accuracy': '0.8339', 'epoch': '0.08053'}
{'loss': '1.15', 'grad_norm': '0.394', 'learning_rate': '3.027e-05', 'entropy': '1.082', 'num_tokens': '4.903e+07', 'mean_token_accuracy': '0.7426', 'epoch': '0.08054'}
{'loss': '0.8824', 'grad_norm': '0.4457', 'learning_rate': '3.025e-05', 'entropy': '0.9073', 'num_tokens': '4.904e+07', 'mean_token_accuracy': '0.7889', 'epoch': '0.08055'}
{'loss': '0.738', 'grad_norm': '0.4097', 'learning_rate': '3.023e-05', 'entropy': '0.7848', 'num_tokens': '4.904e+07', 'mean_token_accuracy': '0.8022', 'epoch': '0.08056'}
{'loss': '1.112', 'grad_norm': '0.4411', 'learning_rate': '3.021e-05', 'entropy': '1.107', 'num_tokens': '4.905e+07', 'mean_token_accuracy': '0.7364', 'epoch': '0.08057'}
{'loss': '0.8586', 'grad_norm': '0.4897', 'learning_rate': '3.019e-05', 'entropy': '0.8935', 'num_tokens': '4.905e+07', 'mean_token_accuracy': '0.7613', 'epoch': '0.08058'}
{'loss': '0.9185', 'grad_norm': '0.3413', 'learning_rate': '3.017e-05', 'entropy': '0.85', 'num_tokens': '4.906e+07', 'mean_token_accuracy': '0.8003', 'epoch': '0.08059'}
{'loss': '0.7212', 'grad_norm': '0.4086', 'learning_rate': '3.015e-05', 'entropy': '0.763', 'num_tokens': '4.906e+07', 'mean_token_accuracy': '0.805', 'epoch': '0.0806'}
{'loss': '1.057', 'grad_norm': '0.4407', 'learning_rate': '3.013e-05', 'entropy': '1.062', 'num_tokens': '4.907e+07', 'mean_token_accuracy': '0.7516', 'epoch': '0.0806'}
{'loss': '1.089', 'grad_norm': '0.4728', 'learning_rate': '3.011e-05', 'entropy': '1.041', 'num_tokens': '4.907e+07', 'mean_token_accuracy': '0.7546', 'epoch': '0.08061'}
{'loss': '0.6889', 'grad_norm': '0.3118', 'learning_rate': '3.009e-05', 'entropy': '0.7241', 'num_tokens': '4.908e+07', 'mean_token_accuracy': '0.8106', 'epoch': '0.08062'}
{'loss': '0.7571', 'grad_norm': '0.3831', 'learning_rate': '3.007e-05', 'entropy': '0.8034', 'num_tokens': '4.908e+07', 'mean_token_accuracy': '0.7891', 'epoch': '0.08063'}
{'loss': '0.7546', 'grad_norm': '0.3748', 'learning_rate': '3.005e-05', 'entropy': '0.764', 'num_tokens': '4.909e+07', 'mean_token_accuracy': '0.8034', 'epoch': '0.08064'}
{'loss': '0.6167', 'grad_norm': '0.3295', 'learning_rate': '3.003e-05', 'entropy': '0.6403', 'num_tokens': '4.909e+07', 'mean_token_accuracy': '0.8408', 'epoch': '0.08065'}
{'loss': '0.6683', 'grad_norm': '0.396', 'learning_rate': '3.001e-05', 'entropy': '0.6995', 'num_tokens': '4.91e+07', 'mean_token_accuracy': '0.8037', 'epoch': '0.08066'}
{'loss': '0.6878', 'grad_norm': '0.4388', 'learning_rate': '2.999e-05', 'entropy': '0.6835', 'num_tokens': '4.91e+07', 'mean_token_accuracy': '0.8212', 'epoch': '0.08067'}
{'loss': '0.9087', 'grad_norm': '0.3604', 'learning_rate': '2.997e-05', 'entropy': '0.9297', 'num_tokens': '4.911e+07', 'mean_token_accuracy': '0.7645', 'epoch': '0.08068'}
{'loss': '0.6459', 'grad_norm': '0.3512', 'learning_rate': '2.995e-05', 'entropy': '0.6446', 'num_tokens': '4.911e+07', 'mean_token_accuracy': '0.8317', 'epoch': '0.08068'}
{'loss': '1.069', 'grad_norm': '0.5388', 'learning_rate': '2.994e-05', 'entropy': '1.026', 'num_tokens': '4.912e+07', 'mean_token_accuracy': '0.7463', 'epoch': '0.08069'}
{'loss': '1.093', 'grad_norm': '0.4882', 'learning_rate': '2.992e-05', 'entropy': '1.103', 'num_tokens': '4.912e+07', 'mean_token_accuracy': '0.7247', 'epoch': '0.0807'}
{'loss': '0.7037', 'grad_norm': '0.3853', 'learning_rate': '2.99e-05', 'entropy': '0.714', 'num_tokens': '4.913e+07', 'mean_token_accuracy': '0.8165', 'epoch': '0.08071'}
{'loss': '1.398', 'grad_norm': '0.5185', 'learning_rate': '2.988e-05', 'entropy': '1.074', 'num_tokens': '4.913e+07', 'mean_token_accuracy': '0.7309', 'epoch': '0.08072'}

 76%|███████▌  | 9122/12000 [5:22:57<1:32:14,  1.92s/it]
 76%|███████▌  | 9123/12000 [5:22:58<1:27:16,  1.82s/it]
                                                        

 76%|███████▌  | 9123/12000 [5:22:58<1:27:16,  1.82s/it]
 76%|███████▌  | 9124/12000 [5:23:00<1:29:01,  1.86s/it]
                                                        

 76%|███████▌  | 9124/12000 [5:23:00<1:29:01,  1.86s/it]
 76%|███████▌  | 9125/12000 [5:23:02<1:24:57,  1.77s/it]
                                                        

 76%|███████▌  | 9125/12000 [5:23:02<1:24:57,  1.77s/it]
 76%|███████▌  | 9126/12000 [5:23:05<1:47:46,  2.25s/it]
                                                        

 76%|███████▌  | 9126/12000 [5:23:05<1:47:46,  2.25s/it]
 76%|███████▌  | 9127/12000 [5:23:07<1:40:04,  2.09s/it]
                                                        

 76%|███████▌  | 9127/12000 [5:23:07<1:40:04,  2.09s/it]
 76%|███████▌  | 9128/12000 [5:23:09<1:40:35,  2.10s/it]
                                                        

 76%|███████▌  | 9128/12000 [5:23:09<1:40:35,  2.10s/it]
 76%|███████▌  | 9129/12000 [5:23:10<1:30:00,  1.88s/it]
                                                        

 76%|███████▌  | 9129/12000 [5:23:10<1:30:00,  1.88s/it]
 76%|███████▌  | 9130/12000 [5:23:13<1:48:10,  2.26s/it]
                                                        

 76%|███████▌  | 9130/12000 [5:23:13<1:48:10,  2.26s/it]
 76%|███████▌  | 9131/12000 [5:23:16<1:54:23,  2.39s/it]
                                                        

 76%|███████▌  | 9131/12000 [5:23:16<1:54:23,  2.39s/it]
 76%|███████▌  | 9132/12000 [5:23:19<2:02:52,  2.57s/it]
                                                        

 76%|███████▌  | 9132/12000 [5:23:19<2:02:52,  2.57s/it]
 76%|███████▌  | 9133/12000 [5:23:21<1:53:05,  2.37s/it]
                                                        

 76%|███████▌  | 9133/12000 [5:23:21<1:53:05,  2.37s/it]
 76%|███████▌  | 9134/12000 [5:23:27<2:49:03,  3.54s/it]
                                                        

 76%|███████▌  | 9134/12000 [5:23:27<2:49:03,  3.54s/it]
 76%|███████▌  | 9135/12000 [5:23:29<2:19:40,  2.93s/it]
                                                        

 76%|███████▌  | 9135/12000 [5:23:29<2:19:40,  2.93s/it]
 76%|███████▌  | 9136/12000 [5:23:34<2:46:59,  3.50s/it]
                                                        

 76%|███████▌  | 9136/12000 [5:23:34<2:46:59,  3.50s/it]
 76%|███████▌  | 9137/12000 [5:23:36<2:27:07,  3.08s/it]
                                                        

 76%|███████▌  | 9137/12000 [5:23:36<2:27:07,  3.08s/it]
 76%|███████▌  | 9138/12000 [5:23:37<2:05:50,  2.64s/it]
                                                        

 76%|███████▌  | 9138/12000 [5:23:37<2:05:50,  2.64s/it]
 76%|███████▌  | 9139/12000 [5:23:41<2:17:25,  2.88s/it]
                                                        

 76%|███████▌  | 9139/12000 [5:23:41<2:17:25,  2.88s/it]
 76%|███████▌  | 9140/12000 [5:23:43<2:01:35,  2.55s/it]
                                                        

 76%|███████▌  | 9140/12000 [5:23:43<2:01:35,  2.55s/it]
 76%|███████▌  | 9141/12000 [5:23:44<1:49:59,  2.31s/it]
                                                        

 76%|███████▌  | 9141/12000 [5:23:44<1:49:59,  2.31s/it]
 76%|███████▌  | 9142/12000 [5:23:47<1:49:26,  2.30s/it]
                                                        

 76%|███████▌  | 9142/12000 [5:23:47<1:49:26,  2.30s/it]
 76%|███████▌  | 9143/12000 [5:23:50<1:59:31,  2.51s/it]
                                                        

 76%|███████▌  | 9143/12000 [5:23:50<1:59:31,  2.51s/it]
 76%|███████▌  | 9144/12000 [5:23:51<1:44:00,  2.19s/it]
                                                        

 76%|███████▌  | 9144/12000 [5:23:51<1:44:00,  2.19s/it]
 76%|███████▌  | 9145/12000 [5:23:53<1:46:07,  2.23s/it]
                                                        

 76%|███████▌  | 9145/12000 [5:23:53<1:46:07,  2.23s/it]
 76%|███████▌  | 9146/12000 [5:23:56<1:59:07,  2.50s/it]
                                                        

 76%|███████▌  | 9146/12000 [5:23:56<1:59:07,  2.50s/it]
 76%|███████▌  | 9147/12000 [5:23:58<1:49:58,  2.31s/it]
                                                        

 76%|███████▌  | 9147/12000 [5:23:58<1:49:58,  2.31s/it]
 76%|███████▌  | 9148/12000 [5:24:00<1:41:36,  2.14s/it]
                                                        

 76%|███████▌  | 9148/12000 [5:24:00<1:41:36,  2.14s/it]
 76%|███████▌  | 9149/12000 [5:24:02<1:38:05,  2.06s/it]
                                                        

 76%|███████▌  | 9149/12000 [5:24:02<1:38:05,  2.06s/it]
 76%|███████▋  | 9150/12000 [5:24:04<1:30:39,  1.91s/it]
                                                        

 76%|███████▋  | 9150/12000 [5:24:04<1:30:39,  1.91s/it]
 76%|███████▋  | 9151/12000 [5:24:08<2:02:06,  2.57s/it]
                                                        

 76%|███████▋  | 9151/12000 [5:24:08<2:02:06,  2.57s/it]
 76%|███████▋  | 9152/12000 [5:24:09<1:48:17,  2.28s/it]
                                                        

 76%|███████▋  | 9152/12000 [5:24:09<1:48:17,  2.28s/it]
 76%|███████▋  | 9153/12000 [5:24:12<1:53:45,  2.40s/it]
                                                        

 76%|███████▋  | 9153/12000 [5:24:12<1:53:45,  2.40s/it]
 76%|███████▋  | 9154/12000 [5:24:13<1:41:30,  2.14s/it]
                                                        

 76%|███████▋  | 9154/12000 [5:24:13<1:41:30,  2.14s/it]
 76%|███████▋  | 9155/12000 [5:24:15<1:34:25,  1.99s/it]
                                                        

 76%|███████▋  | 9155/12000 [5:24:15<1:34:25,  1.99s/it]
 76%|███████▋  | 9156/12000 [5:24:17<1:32:34,  1.95s/it]
                                                        

 76%|███████▋  | 9156/12000 [5:24:17<1:32:34,  1.95s/it]
 76%|███████▋  | 9157/12000 [5:24:18<1:25:20,  1.80s/it]
                                                        

 76%|███████▋  | 9157/12000 [5:24:18<1:25:20,  1.80s/it]
 76%|███████▋  | 9158/12000 [5:24:20<1:21:14,  1.72s/it]
                                                        

 76%|███████▋  | 9158/12000 [5:24:20<1:21:14,  1.72s/it]
 76%|███████▋  | 9159/12000 [5:24:22<1:20:58,  1.71s/it]
                                                        

 76%|███████▋  | 9159/12000 [5:24:22<1:20:58,  1.71s/it]
 76%|███████▋  | 9160/12000 [5:24:23<1:17:00,  1.63s/it]
                                                        

 76%|███████▋  | 9160/12000 [5:24:23<1:17:00,  1.63s/it]
 76%|███████▋  | 9161/12000 [5:24:26<1:42:50,  2.17s/it]
                                                        

 76%|███████▋  | 9161/12000 [5:24:26<1:42:50,  2.17s/it]
 76%|███████▋  | 9162/12000 [5:24:28<1:34:37,  2.00s/it]
                                                        

 76%|███████▋  | 9162/12000 [5:24:28<1:34:37,  2.00s/it]
 76%|███████▋  | 9163/12000 [5:24:30<1:30:35,  1.92s/it]
                                                        

 76%|███████▋  | 9163/12000 [5:24:30<1:30:35,  1.92s/it]
 76%|███████▋  | 9164/12000 [5:24:31<1:26:59,  1.84s/it]
                                                        

 76%|███████▋  | 9164/12000 [5:24:31<1:26:59,  1.84s/it]
 76%|███████▋  | 9165/12000 [5:24:33<1:23:04,  1.76s/it]
                                                        

 76%|███████▋  | 9165/12000 [5:24:33<1:23:04,  1.76s/it]
 76%|███████▋  | 9166/12000 [5:24:35<1:20:51,  1.71s/it]
                                                        

 76%|███████▋  | 9166/12000 [5:24:35<1:20:51,  1.71s/it]
 76%|███████▋  | 9167/12000 [5:24:36<1:18:16,  1.66s/it]
                                                        

 76%|███████▋  | 9167/12000 [5:24:36<1:18:16,  1.66s/it]
 76%|███████▋  | 9168/12000 [5:24:38<1:17:12,  1.64s/it]
                                                        

 76%|███████▋  | 9168/12000 [5:24:38<1:17:12,  1.64s/it]
 76%|███████▋  | 9169/12000 [5:24:40<1:29:07,  1.89s/it]
                                                        
{'loss': '0.818', 'grad_norm': '0.3348', 'learning_rate': '2.986e-05', 'entropy': '0.8174', 'num_tokens': '4.914e+07', 'mean_token_accuracy': '0.789', 'epoch': '0.08073'}
{'loss': '1.029', 'grad_norm': '0.4173', 'learning_rate': '2.984e-05', 'entropy': '0.9942', 'num_tokens': '4.914e+07', 'mean_token_accuracy': '0.7629', 'epoch': '0.08074'}
{'loss': '0.7186', 'grad_norm': '0.4644', 'learning_rate': '2.982e-05', 'entropy': '0.7659', 'num_tokens': '4.915e+07', 'mean_token_accuracy': '0.8031', 'epoch': '0.08075'}
{'loss': '0.8318', 'grad_norm': '0.3953', 'learning_rate': '2.98e-05', 'entropy': '0.848', 'num_tokens': '4.915e+07', 'mean_token_accuracy': '0.7896', 'epoch': '0.08075'}
{'loss': '0.4627', 'grad_norm': '0.2503', 'learning_rate': '2.978e-05', 'entropy': '0.4663', 'num_tokens': '4.916e+07', 'mean_token_accuracy': '0.872', 'epoch': '0.08076'}
{'loss': '1.041', 'grad_norm': '0.4639', 'learning_rate': '2.976e-05', 'entropy': '1.007', 'num_tokens': '4.916e+07', 'mean_token_accuracy': '0.7529', 'epoch': '0.08077'}
{'loss': '0.9392', 'grad_norm': '0.4278', 'learning_rate': '2.974e-05', 'entropy': '0.8726', 'num_tokens': '4.917e+07', 'mean_token_accuracy': '0.7965', 'epoch': '0.08078'}
{'loss': '0.7245', 'grad_norm': '0.3836', 'learning_rate': '2.972e-05', 'entropy': '0.8008', 'num_tokens': '4.917e+07', 'mean_token_accuracy': '0.7925', 'epoch': '0.08079'}
{'loss': '0.912', 'grad_norm': '0.3336', 'learning_rate': '2.97e-05', 'entropy': '1.062', 'num_tokens': '4.918e+07', 'mean_token_accuracy': '0.7388', 'epoch': '0.0808'}
{'loss': '1.094', 'grad_norm': '0.437', 'learning_rate': '2.968e-05', 'entropy': '0.9985', 'num_tokens': '4.919e+07', 'mean_token_accuracy': '0.7553', 'epoch': '0.08081'}
{'loss': '0.7288', 'grad_norm': '0.3603', 'learning_rate': '2.966e-05', 'entropy': '0.9295', 'num_tokens': '4.92e+07', 'mean_token_accuracy': '0.7781', 'epoch': '0.08082'}
{'loss': '1.063', 'grad_norm': '0.5044', 'learning_rate': '2.964e-05', 'entropy': '0.9879', 'num_tokens': '4.92e+07', 'mean_token_accuracy': '0.7571', 'epoch': '0.08083'}
{'loss': '0.6057', 'grad_norm': '0.2193', 'learning_rate': '2.962e-05', 'entropy': '0.6001', 'num_tokens': '4.921e+07', 'mean_token_accuracy': '0.8487', 'epoch': '0.08083'}
{'loss': '0.8762', 'grad_norm': '0.4113', 'learning_rate': '2.96e-05', 'entropy': '0.9109', 'num_tokens': '4.922e+07', 'mean_token_accuracy': '0.7605', 'epoch': '0.08084'}
{'loss': '0.8832', 'grad_norm': '0.3871', 'learning_rate': '2.958e-05', 'entropy': '0.8958', 'num_tokens': '4.923e+07', 'mean_token_accuracy': '0.7814', 'epoch': '0.08085'}
{'loss': '0.9655', 'grad_norm': '0.4157', 'learning_rate': '2.956e-05', 'entropy': '0.8476', 'num_tokens': '4.923e+07', 'mean_token_accuracy': '0.7995', 'epoch': '0.08086'}
{'loss': '0.5512', 'grad_norm': '0.4267', 'learning_rate': '2.954e-05', 'entropy': '0.5548', 'num_tokens': '4.923e+07', 'mean_token_accuracy': '0.8479', 'epoch': '0.08087'}
{'loss': '0.7237', 'grad_norm': '0.4345', 'learning_rate': '2.952e-05', 'entropy': '0.7435', 'num_tokens': '4.924e+07', 'mean_token_accuracy': '0.8034', 'epoch': '0.08088'}
{'loss': '0.7516', 'grad_norm': '0.4608', 'learning_rate': '2.95e-05', 'entropy': '0.7327', 'num_tokens': '4.924e+07', 'mean_token_accuracy': '0.8161', 'epoch': '0.08089'}
{'loss': '0.6443', 'grad_norm': '0.3502', 'learning_rate': '2.948e-05', 'entropy': '0.637', 'num_tokens': '4.925e+07', 'mean_token_accuracy': '0.8347', 'epoch': '0.0809'}
{'loss': '0.9825', 'grad_norm': '0.3951', 'learning_rate': '2.946e-05', 'entropy': '1.032', 'num_tokens': '4.925e+07', 'mean_token_accuracy': '0.7514', 'epoch': '0.08091'}
{'loss': '0.7249', 'grad_norm': '0.3116', 'learning_rate': '2.945e-05', 'entropy': '0.8228', 'num_tokens': '4.926e+07', 'mean_token_accuracy': '0.7938', 'epoch': '0.08091'}
{'loss': '0.6847', 'grad_norm': '0.3477', 'learning_rate': '2.943e-05', 'entropy': '0.6672', 'num_tokens': '4.927e+07', 'mean_token_accuracy': '0.8338', 'epoch': '0.08092'}
{'loss': '0.7279', 'grad_norm': '0.3507', 'learning_rate': '2.941e-05', 'entropy': '0.7968', 'num_tokens': '4.927e+07', 'mean_token_accuracy': '0.7865', 'epoch': '0.08093'}
{'loss': '0.7994', 'grad_norm': '0.3403', 'learning_rate': '2.939e-05', 'entropy': '0.7953', 'num_tokens': '4.928e+07', 'mean_token_accuracy': '0.8054', 'epoch': '0.08094'}
{'loss': '0.7517', 'grad_norm': '0.3139', 'learning_rate': '2.937e-05', 'entropy': '0.7599', 'num_tokens': '4.929e+07', 'mean_token_accuracy': '0.8038', 'epoch': '0.08095'}
{'loss': '0.603', 'grad_norm': '0.3799', 'learning_rate': '2.935e-05', 'entropy': '0.6548', 'num_tokens': '4.929e+07', 'mean_token_accuracy': '0.838', 'epoch': '0.08096'}
{'loss': '0.9462', 'grad_norm': '0.5052', 'learning_rate': '2.933e-05', 'entropy': '0.9053', 'num_tokens': '4.929e+07', 'mean_token_accuracy': '0.7765', 'epoch': '0.08097'}
{'loss': '0.781', 'grad_norm': '0.5387', 'learning_rate': '2.931e-05', 'entropy': '0.8056', 'num_tokens': '4.93e+07', 'mean_token_accuracy': '0.8033', 'epoch': '0.08098'}
{'loss': '0.7284', 'grad_norm': '0.4533', 'learning_rate': '2.929e-05', 'entropy': '0.7863', 'num_tokens': '4.93e+07', 'mean_token_accuracy': '0.808', 'epoch': '0.08098'}
{'loss': '0.7285', 'grad_norm': '0.3885', 'learning_rate': '2.927e-05', 'entropy': '0.8098', 'num_tokens': '4.931e+07', 'mean_token_accuracy': '0.7956', 'epoch': '0.08099'}
{'loss': '1.208', 'grad_norm': '0.3659', 'learning_rate': '2.925e-05', 'entropy': '0.9728', 'num_tokens': '4.932e+07', 'mean_token_accuracy': '0.7745', 'epoch': '0.081'}
{'loss': '0.5639', 'grad_norm': '0.2803', 'learning_rate': '2.923e-05', 'entropy': '0.5766', 'num_tokens': '4.932e+07', 'mean_token_accuracy': '0.8381', 'epoch': '0.08101'}
{'loss': '0.779', 'grad_norm': '0.4702', 'learning_rate': '2.921e-05', 'entropy': '0.7753', 'num_tokens': '4.932e+07', 'mean_token_accuracy': '0.803', 'epoch': '0.08102'}
{'loss': '0.8787', 'grad_norm': '0.3896', 'learning_rate': '2.919e-05', 'entropy': '0.8351', 'num_tokens': '4.933e+07', 'mean_token_accuracy': '0.7982', 'epoch': '0.08103'}
{'loss': '0.5935', 'grad_norm': '0.4074', 'learning_rate': '2.917e-05', 'entropy': '0.6648', 'num_tokens': '4.934e+07', 'mean_token_accuracy': '0.8188', 'epoch': '0.08104'}
{'loss': '0.8096', 'grad_norm': '0.4236', 'learning_rate': '2.915e-05', 'entropy': '0.8392', 'num_tokens': '4.934e+07', 'mean_token_accuracy': '0.7877', 'epoch': '0.08105'}
{'loss': '0.9271', 'grad_norm': '0.384', 'learning_rate': '2.913e-05', 'entropy': '0.8986', 'num_tokens': '4.934e+07', 'mean_token_accuracy': '0.7814', 'epoch': '0.08106'}
{'loss': '0.893', 'grad_norm': '0.4002', 'learning_rate': '2.911e-05', 'entropy': '0.9049', 'num_tokens': '4.935e+07', 'mean_token_accuracy': '0.7669', 'epoch': '0.08106'}
{'loss': '0.6995', 'grad_norm': '0.355', 'learning_rate': '2.909e-05', 'entropy': '0.7519', 'num_tokens': '4.935e+07', 'mean_token_accuracy': '0.8002', 'epoch': '0.08107'}
{'loss': '0.6233', 'grad_norm': '0.3852', 'learning_rate': '2.908e-05', 'entropy': '0.6229', 'num_tokens': '4.936e+07', 'mean_token_accuracy': '0.823', 'epoch': '0.08108'}
{'loss': '0.8083', 'grad_norm': '0.3908', 'learning_rate': '2.906e-05', 'entropy': '0.7777', 'num_tokens': '4.936e+07', 'mean_token_accuracy': '0.803', 'epoch': '0.08109'}
{'loss': '0.5761', 'grad_norm': '0.3794', 'learning_rate': '2.904e-05', 'entropy': '0.7013', 'num_tokens': '4.937e+07', 'mean_token_accuracy': '0.8219', 'epoch': '0.0811'}
{'loss': '0.7208', 'grad_norm': '0.4143', 'learning_rate': '2.902e-05', 'entropy': '0.747', 'num_tokens': '4.937e+07', 'mean_token_accuracy': '0.8048', 'epoch': '0.08111'}
{'loss': '0.7001', 'grad_norm': '0.355', 'learning_rate': '2.9e-05', 'entropy': '0.7088', 'num_tokens': '4.938e+07', 'mean_token_accuracy': '0.8147', 'epoch': '0.08112'}
{'loss': '0.6598', 'grad_norm': '0.3403', 'learning_rate': '2.898e-05', 'entropy': '0.6882', 'num_tokens': '4.938e+07', 'mean_token_accuracy': '0.8098', 'epoch': '0.08113'}
{'loss': '0.7828', 'grad_norm': '0.3973', 'learning_rate': '2.896e-05', 'entropy': '0.773', 'num_tokens': '4.939e+07', 'mean_token_accuracy': '0.8064', 'epoch': '0.08114'}

 76%|███████▋  | 9169/12000 [5:24:40<1:29:07,  1.89s/it]
 76%|███████▋  | 9170/12000 [5:24:42<1:24:12,  1.79s/it]
                                                        

 76%|███████▋  | 9170/12000 [5:24:42<1:24:12,  1.79s/it]
 76%|███████▋  | 9171/12000 [5:24:44<1:26:44,  1.84s/it]
                                                        

 76%|███████▋  | 9171/12000 [5:24:44<1:26:44,  1.84s/it]
 76%|███████▋  | 9172/12000 [5:24:45<1:23:00,  1.76s/it]
                                                        

 76%|███████▋  | 9172/12000 [5:24:45<1:23:00,  1.76s/it]
 76%|███████▋  | 9173/12000 [5:24:47<1:20:29,  1.71s/it]
                                                        

 76%|███████▋  | 9173/12000 [5:24:47<1:20:29,  1.71s/it]
 76%|███████▋  | 9174/12000 [5:24:48<1:18:02,  1.66s/it]
                                                        

 76%|███████▋  | 9174/12000 [5:24:48<1:18:02,  1.66s/it]
 76%|███████▋  | 9175/12000 [5:24:50<1:14:59,  1.59s/it]
                                                        

 76%|███████▋  | 9175/12000 [5:24:50<1:14:59,  1.59s/it]
 76%|███████▋  | 9176/12000 [5:24:51<1:14:52,  1.59s/it]
                                                        

 76%|███████▋  | 9176/12000 [5:24:51<1:14:52,  1.59s/it]
 76%|███████▋  | 9177/12000 [5:24:53<1:13:14,  1.56s/it]
                                                        

 76%|███████▋  | 9177/12000 [5:24:53<1:13:14,  1.56s/it]
 76%|███████▋  | 9178/12000 [5:24:55<1:15:54,  1.61s/it]
                                                        

 76%|███████▋  | 9178/12000 [5:24:55<1:15:54,  1.61s/it]
 76%|███████▋  | 9179/12000 [5:24:57<1:21:03,  1.72s/it]
                                                        

 76%|███████▋  | 9179/12000 [5:24:57<1:21:03,  1.72s/it]
 76%|███████▋  | 9180/12000 [5:24:59<1:23:57,  1.79s/it]
                                                        

 76%|███████▋  | 9180/12000 [5:24:59<1:23:57,  1.79s/it]
 77%|███████▋  | 9181/12000 [5:25:00<1:21:46,  1.74s/it]
                                                        

 77%|███████▋  | 9181/12000 [5:25:00<1:21:46,  1.74s/it]
 77%|███████▋  | 9182/12000 [5:25:02<1:17:38,  1.65s/it]
                                                        

 77%|███████▋  | 9182/12000 [5:25:02<1:17:38,  1.65s/it]
 77%|███████▋  | 9183/12000 [5:25:07<2:08:31,  2.74s/it]
                                                        

 77%|███████▋  | 9183/12000 [5:25:07<2:08:31,  2.74s/it]
 77%|███████▋  | 9184/12000 [5:25:09<1:53:46,  2.42s/it]
                                                        

 77%|███████▋  | 9184/12000 [5:25:09<1:53:46,  2.42s/it]
 77%|███████▋  | 9185/12000 [5:25:11<1:45:48,  2.26s/it]
                                                        

 77%|███████▋  | 9185/12000 [5:25:11<1:45:48,  2.26s/it]
 77%|███████▋  | 9186/12000 [5:25:12<1:40:19,  2.14s/it]
                                                        

 77%|███████▋  | 9186/12000 [5:25:12<1:40:19,  2.14s/it]
 77%|███████▋  | 9187/12000 [5:25:16<2:06:06,  2.69s/it]
                                                        

 77%|███████▋  | 9187/12000 [5:25:16<2:06:06,  2.69s/it]
 77%|███████▋  | 9188/12000 [5:25:18<1:51:51,  2.39s/it]
                                                        

 77%|███████▋  | 9188/12000 [5:25:18<1:51:51,  2.39s/it]
 77%|███████▋  | 9189/12000 [5:25:21<1:55:24,  2.46s/it]
                                                        

 77%|███████▋  | 9189/12000 [5:25:21<1:55:24,  2.46s/it]
 77%|███████▋  | 9190/12000 [5:25:22<1:44:40,  2.24s/it]
                                                        

 77%|███████▋  | 9190/12000 [5:25:22<1:44:40,  2.24s/it]
 77%|███████▋  | 9191/12000 [5:25:25<1:47:32,  2.30s/it]
                                                        

 77%|███████▋  | 9191/12000 [5:25:25<1:47:32,  2.30s/it]
 77%|███████▋  | 9192/12000 [5:25:26<1:37:16,  2.08s/it]
                                                        

 77%|███████▋  | 9192/12000 [5:25:26<1:37:16,  2.08s/it]
 77%|███████▋  | 9193/12000 [5:25:30<1:54:38,  2.45s/it]
                                                        

 77%|███████▋  | 9193/12000 [5:25:30<1:54:38,  2.45s/it]
 77%|███████▋  | 9194/12000 [5:25:32<1:47:57,  2.31s/it]
                                                        

 77%|███████▋  | 9194/12000 [5:25:32<1:47:57,  2.31s/it]
 77%|███████▋  | 9195/12000 [5:25:33<1:38:29,  2.11s/it]
                                                        

 77%|███████▋  | 9195/12000 [5:25:33<1:38:29,  2.11s/it]
 77%|███████▋  | 9196/12000 [5:25:35<1:38:31,  2.11s/it]
                                                        

 77%|███████▋  | 9196/12000 [5:25:35<1:38:31,  2.11s/it]
 77%|███████▋  | 9197/12000 [5:25:38<1:47:11,  2.29s/it]
                                                        

 77%|███████▋  | 9197/12000 [5:25:38<1:47:11,  2.29s/it]
 77%|███████▋  | 9198/12000 [5:25:40<1:38:08,  2.10s/it]
                                                        

 77%|███████▋  | 9198/12000 [5:25:40<1:38:08,  2.10s/it]
 77%|███████▋  | 9199/12000 [5:25:41<1:30:39,  1.94s/it]
                                                        

 77%|███████▋  | 9199/12000 [5:25:41<1:30:39,  1.94s/it]
 77%|███████▋  | 9200/12000 [5:25:43<1:29:20,  1.91s/it]
                                                        

 77%|███████▋  | 9200/12000 [5:25:43<1:29:20,  1.91s/it]
 77%|███████▋  | 9201/12000 [5:25:46<1:46:39,  2.29s/it]
                                                        

 77%|███████▋  | 9201/12000 [5:25:46<1:46:39,  2.29s/it]
 77%|███████▋  | 9202/12000 [5:25:48<1:34:40,  2.03s/it]
                                                        

 77%|███████▋  | 9202/12000 [5:25:48<1:34:40,  2.03s/it]
 77%|███████▋  | 9203/12000 [5:25:50<1:42:37,  2.20s/it]
                                                        

 77%|███████▋  | 9203/12000 [5:25:50<1:42:37,  2.20s/it]
 77%|███████▋  | 9204/12000 [5:25:53<1:42:38,  2.20s/it]
                                                        

 77%|███████▋  | 9204/12000 [5:25:53<1:42:38,  2.20s/it]
 77%|███████▋  | 9205/12000 [5:25:56<2:01:50,  2.62s/it]
                                                        

 77%|███████▋  | 9205/12000 [5:25:56<2:01:50,  2.62s/it]
 77%|███████▋  | 9206/12000 [5:25:58<1:47:27,  2.31s/it]
                                                        

 77%|███████▋  | 9206/12000 [5:25:58<1:47:27,  2.31s/it]
 77%|███████▋  | 9207/12000 [5:26:01<1:56:13,  2.50s/it]
                                                        

 77%|███████▋  | 9207/12000 [5:26:01<1:56:13,  2.50s/it]
 77%|███████▋  | 9208/12000 [5:26:02<1:40:17,  2.16s/it]
                                                        

 77%|███████▋  | 9208/12000 [5:26:02<1:40:17,  2.16s/it]
 77%|███████▋  | 9209/12000 [5:26:06<2:00:28,  2.59s/it]
                                                        

 77%|███████▋  | 9209/12000 [5:26:06<2:00:28,  2.59s/it]
 77%|███████▋  | 9210/12000 [5:26:08<1:52:41,  2.42s/it]
                                                        

 77%|███████▋  | 9210/12000 [5:26:08<1:52:41,  2.42s/it]
 77%|███████▋  | 9211/12000 [5:26:09<1:40:19,  2.16s/it]
                                                        

 77%|███████▋  | 9211/12000 [5:26:09<1:40:19,  2.16s/it]
 77%|███████▋  | 9212/12000 [5:26:11<1:29:48,  1.93s/it]
                                                        

 77%|███████▋  | 9212/12000 [5:26:11<1:29:48,  1.93s/it]
 77%|███████▋  | 9213/12000 [5:26:14<1:48:40,  2.34s/it]
                                                        

 77%|███████▋  | 9213/12000 [5:26:14<1:48:40,  2.34s/it]
 77%|███████▋  | 9214/12000 [5:26:15<1:35:49,  2.06s/it]
                                                        

 77%|███████▋  | 9214/12000 [5:26:15<1:35:49,  2.06s/it]
 77%|███████▋  | 9215/12000 [5:26:17<1:31:01,  1.96s/it]
                                                        

 77%|███████▋  | 9215/12000 [5:26:17<1:31:01,  1.96s/it]
 77%|███████▋  | 9216/12000 [5:26:19<1:24:27,  1.82s/it]
                                                        
{'loss': '0.7751', 'grad_norm': '0.4444', 'learning_rate': '2.894e-05', 'entropy': '0.9139', 'num_tokens': '4.939e+07', 'mean_token_accuracy': '0.7723', 'epoch': '0.08114'}
{'loss': '0.6754', 'grad_norm': '0.4145', 'learning_rate': '2.892e-05', 'entropy': '0.7543', 'num_tokens': '4.94e+07', 'mean_token_accuracy': '0.8027', 'epoch': '0.08115'}
{'loss': '1.158', 'grad_norm': '0.5178', 'learning_rate': '2.89e-05', 'entropy': '1.259', 'num_tokens': '4.94e+07', 'mean_token_accuracy': '0.7085', 'epoch': '0.08116'}
{'loss': '0.615', 'grad_norm': '0.3612', 'learning_rate': '2.888e-05', 'entropy': '0.6854', 'num_tokens': '4.941e+07', 'mean_token_accuracy': '0.8207', 'epoch': '0.08117'}
{'loss': '1.092', 'grad_norm': '0.4113', 'learning_rate': '2.886e-05', 'entropy': '0.9978', 'num_tokens': '4.941e+07', 'mean_token_accuracy': '0.7577', 'epoch': '0.08118'}
{'loss': '0.7625', 'grad_norm': '0.4925', 'learning_rate': '2.884e-05', 'entropy': '0.7676', 'num_tokens': '4.942e+07', 'mean_token_accuracy': '0.7918', 'epoch': '0.08119'}
{'loss': '0.831', 'grad_norm': '0.3958', 'learning_rate': '2.882e-05', 'entropy': '0.9507', 'num_tokens': '4.942e+07', 'mean_token_accuracy': '0.7593', 'epoch': '0.0812'}
{'loss': '0.813', 'grad_norm': '0.425', 'learning_rate': '2.88e-05', 'entropy': '0.8439', 'num_tokens': '4.942e+07', 'mean_token_accuracy': '0.7815', 'epoch': '0.08121'}
{'loss': '0.6078', 'grad_norm': '0.3182', 'learning_rate': '2.878e-05', 'entropy': '0.6543', 'num_tokens': '4.943e+07', 'mean_token_accuracy': '0.8153', 'epoch': '0.08121'}
{'loss': '1.013', 'grad_norm': '0.4193', 'learning_rate': '2.877e-05', 'entropy': '1.009', 'num_tokens': '4.943e+07', 'mean_token_accuracy': '0.7448', 'epoch': '0.08122'}
{'loss': '0.7064', 'grad_norm': '0.3725', 'learning_rate': '2.875e-05', 'entropy': '0.7054', 'num_tokens': '4.944e+07', 'mean_token_accuracy': '0.8337', 'epoch': '0.08123'}
{'loss': '1.301', 'grad_norm': '0.5134', 'learning_rate': '2.873e-05', 'entropy': '1.203', 'num_tokens': '4.945e+07', 'mean_token_accuracy': '0.707', 'epoch': '0.08124'}
{'loss': '0.9253', 'grad_norm': '0.3762', 'learning_rate': '2.871e-05', 'entropy': '0.9086', 'num_tokens': '4.945e+07', 'mean_token_accuracy': '0.7727', 'epoch': '0.08125'}
{'loss': '0.8861', 'grad_norm': '0.4009', 'learning_rate': '2.869e-05', 'entropy': '0.9818', 'num_tokens': '4.946e+07', 'mean_token_accuracy': '0.7446', 'epoch': '0.08126'}
{'loss': '0.8083', 'grad_norm': '0.3977', 'learning_rate': '2.867e-05', 'entropy': '0.7873', 'num_tokens': '4.946e+07', 'mean_token_accuracy': '0.7944', 'epoch': '0.08127'}
{'loss': '0.6935', 'grad_norm': '0.3882', 'learning_rate': '2.865e-05', 'entropy': '0.6673', 'num_tokens': '4.947e+07', 'mean_token_accuracy': '0.8272', 'epoch': '0.08128'}
{'loss': '0.813', 'grad_norm': '0.3244', 'learning_rate': '2.863e-05', 'entropy': '0.844', 'num_tokens': '4.947e+07', 'mean_token_accuracy': '0.796', 'epoch': '0.08129'}
{'loss': '0.5647', 'grad_norm': '0.3078', 'learning_rate': '2.861e-05', 'entropy': '0.5747', 'num_tokens': '4.948e+07', 'mean_token_accuracy': '0.8561', 'epoch': '0.08129'}
{'loss': '0.6971', 'grad_norm': '0.3176', 'learning_rate': '2.859e-05', 'entropy': '0.6815', 'num_tokens': '4.949e+07', 'mean_token_accuracy': '0.8388', 'epoch': '0.0813'}
{'loss': '1.012', 'grad_norm': '0.476', 'learning_rate': '2.857e-05', 'entropy': '0.9863', 'num_tokens': '4.949e+07', 'mean_token_accuracy': '0.7463', 'epoch': '0.08131'}
{'loss': '0.7879', 'grad_norm': '0.3758', 'learning_rate': '2.855e-05', 'entropy': '0.7402', 'num_tokens': '4.95e+07', 'mean_token_accuracy': '0.8147', 'epoch': '0.08132'}
{'loss': '0.8831', 'grad_norm': '0.5006', 'learning_rate': '2.853e-05', 'entropy': '0.8908', 'num_tokens': '4.95e+07', 'mean_token_accuracy': '0.7749', 'epoch': '0.08133'}
{'loss': '0.672', 'grad_norm': '0.3015', 'learning_rate': '2.851e-05', 'entropy': '0.7213', 'num_tokens': '4.951e+07', 'mean_token_accuracy': '0.8111', 'epoch': '0.08134'}
{'loss': '0.7163', 'grad_norm': '0.3907', 'learning_rate': '2.849e-05', 'entropy': '0.7278', 'num_tokens': '4.952e+07', 'mean_token_accuracy': '0.8142', 'epoch': '0.08135'}
{'loss': '0.9337', 'grad_norm': '0.4097', 'learning_rate': '2.848e-05', 'entropy': '0.8776', 'num_tokens': '4.952e+07', 'mean_token_accuracy': '0.7801', 'epoch': '0.08136'}
{'loss': '0.6777', 'grad_norm': '0.4006', 'learning_rate': '2.846e-05', 'entropy': '0.6726', 'num_tokens': '4.953e+07', 'mean_token_accuracy': '0.8278', 'epoch': '0.08137'}
{'loss': '0.8028', 'grad_norm': '0.3785', 'learning_rate': '2.844e-05', 'entropy': '0.8385', 'num_tokens': '4.953e+07', 'mean_token_accuracy': '0.7985', 'epoch': '0.08137'}
{'loss': '1.021', 'grad_norm': '0.3732', 'learning_rate': '2.842e-05', 'entropy': '1.07', 'num_tokens': '4.954e+07', 'mean_token_accuracy': '0.7436', 'epoch': '0.08138'}
{'loss': '0.5928', 'grad_norm': '0.3344', 'learning_rate': '2.84e-05', 'entropy': '0.6519', 'num_tokens': '4.955e+07', 'mean_token_accuracy': '0.8416', 'epoch': '0.08139'}
{'loss': '0.5969', 'grad_norm': '0.3752', 'learning_rate': '2.838e-05', 'entropy': '0.6403', 'num_tokens': '4.955e+07', 'mean_token_accuracy': '0.8299', 'epoch': '0.0814'}
{'loss': '0.5082', 'grad_norm': '0.4276', 'learning_rate': '2.836e-05', 'entropy': '0.5242', 'num_tokens': '4.956e+07', 'mean_token_accuracy': '0.8661', 'epoch': '0.08141'}
{'loss': '0.9429', 'grad_norm': '0.4327', 'learning_rate': '2.834e-05', 'entropy': '1.03', 'num_tokens': '4.956e+07', 'mean_token_accuracy': '0.7609', 'epoch': '0.08142'}
{'loss': '1.091', 'grad_norm': '0.3395', 'learning_rate': '2.832e-05', 'entropy': '1.049', 'num_tokens': '4.957e+07', 'mean_token_accuracy': '0.7412', 'epoch': '0.08143'}
{'loss': '0.9278', 'grad_norm': '0.4623', 'learning_rate': '2.83e-05', 'entropy': '0.9148', 'num_tokens': '4.957e+07', 'mean_token_accuracy': '0.7653', 'epoch': '0.08144'}
{'loss': '1.058', 'grad_norm': '0.4313', 'learning_rate': '2.828e-05', 'entropy': '1.054', 'num_tokens': '4.958e+07', 'mean_token_accuracy': '0.7482', 'epoch': '0.08145'}
{'loss': '0.8584', 'grad_norm': '0.4582', 'learning_rate': '2.826e-05', 'entropy': '0.7969', 'num_tokens': '4.959e+07', 'mean_token_accuracy': '0.7922', 'epoch': '0.08145'}
{'loss': '0.8675', 'grad_norm': '0.427', 'learning_rate': '2.824e-05', 'entropy': '0.9202', 'num_tokens': '4.959e+07', 'mean_token_accuracy': '0.7708', 'epoch': '0.08146'}
{'loss': '0.8228', 'grad_norm': '0.45', 'learning_rate': '2.823e-05', 'entropy': '0.7945', 'num_tokens': '4.96e+07', 'mean_token_accuracy': '0.8026', 'epoch': '0.08147'}
{'loss': '0.9254', 'grad_norm': '0.4682', 'learning_rate': '2.821e-05', 'entropy': '0.9959', 'num_tokens': '4.96e+07', 'mean_token_accuracy': '0.7519', 'epoch': '0.08148'}
{'loss': '0.6413', 'grad_norm': '0.4562', 'learning_rate': '2.819e-05', 'entropy': '0.6531', 'num_tokens': '4.961e+07', 'mean_token_accuracy': '0.824', 'epoch': '0.08149'}
{'loss': '1.524', 'grad_norm': '0.3929', 'learning_rate': '2.817e-05', 'entropy': '1.012', 'num_tokens': '4.961e+07', 'mean_token_accuracy': '0.7756', 'epoch': '0.0815'}
{'loss': '0.9096', 'grad_norm': '0.3795', 'learning_rate': '2.815e-05', 'entropy': '0.9193', 'num_tokens': '4.962e+07', 'mean_token_accuracy': '0.7771', 'epoch': '0.08151'}
{'loss': '0.6231', 'grad_norm': '0.294', 'learning_rate': '2.813e-05', 'entropy': '0.6142', 'num_tokens': '4.962e+07', 'mean_token_accuracy': '0.8419', 'epoch': '0.08152'}
{'loss': '0.6905', 'grad_norm': '0.3277', 'learning_rate': '2.811e-05', 'entropy': '0.7318', 'num_tokens': '4.963e+07', 'mean_token_accuracy': '0.8145', 'epoch': '0.08152'}
{'loss': '0.4413', 'grad_norm': '0.21', 'learning_rate': '2.809e-05', 'entropy': '0.514', 'num_tokens': '4.964e+07', 'mean_token_accuracy': '0.8605', 'epoch': '0.08153'}
{'loss': '0.5402', 'grad_norm': '0.3437', 'learning_rate': '2.807e-05', 'entropy': '0.6152', 'num_tokens': '4.964e+07', 'mean_token_accuracy': '0.8414', 'epoch': '0.08154'}
{'loss': '0.979', 'grad_norm': '0.3897', 'learning_rate': '2.805e-05', 'entropy': '0.9792', 'num_tokens': '4.964e+07', 'mean_token_accuracy': '0.7674', 'epoch': '0.08155'}

 77%|███████▋  | 9216/12000 [5:26:19<1:24:27,  1.82s/it]
 77%|███████▋  | 9217/12000 [5:26:20<1:21:52,  1.77s/it]
                                                        

 77%|███████▋  | 9217/12000 [5:26:20<1:21:52,  1.77s/it]
 77%|███████▋  | 9218/12000 [5:26:22<1:19:34,  1.72s/it]
                                                        

 77%|███████▋  | 9218/12000 [5:26:22<1:19:34,  1.72s/it]
 77%|███████▋  | 9219/12000 [5:26:24<1:31:04,  1.96s/it]
                                                        

 77%|███████▋  | 9219/12000 [5:26:24<1:31:04,  1.96s/it]
 77%|███████▋  | 9220/12000 [5:26:28<1:56:48,  2.52s/it]
                                                        

 77%|███████▋  | 9220/12000 [5:26:28<1:56:48,  2.52s/it]
 77%|███████▋  | 9221/12000 [5:26:30<1:43:27,  2.23s/it]
                                                        

 77%|███████▋  | 9221/12000 [5:26:30<1:43:27,  2.23s/it]
 77%|███████▋  | 9222/12000 [5:26:31<1:36:01,  2.07s/it]
                                                        

 77%|███████▋  | 9222/12000 [5:26:31<1:36:01,  2.07s/it]
 77%|███████▋  | 9223/12000 [5:26:33<1:28:21,  1.91s/it]
                                                        

 77%|███████▋  | 9223/12000 [5:26:33<1:28:21,  1.91s/it]
 77%|███████▋  | 9224/12000 [5:26:35<1:23:26,  1.80s/it]
                                                        

 77%|███████▋  | 9224/12000 [5:26:35<1:23:26,  1.80s/it]
 77%|███████▋  | 9225/12000 [5:26:36<1:19:36,  1.72s/it]
                                                        

 77%|███████▋  | 9225/12000 [5:26:36<1:19:36,  1.72s/it]
 77%|███████▋  | 9226/12000 [5:26:38<1:15:50,  1.64s/it]
                                                        

 77%|███████▋  | 9226/12000 [5:26:38<1:15:50,  1.64s/it]
 77%|███████▋  | 9227/12000 [5:26:41<1:41:23,  2.19s/it]
                                                        

 77%|███████▋  | 9227/12000 [5:26:41<1:41:23,  2.19s/it]
 77%|███████▋  | 9228/12000 [5:26:42<1:29:50,  1.94s/it]
                                                        

 77%|███████▋  | 9228/12000 [5:26:42<1:29:50,  1.94s/it]
 77%|███████▋  | 9229/12000 [5:26:44<1:24:44,  1.83s/it]
                                                        

 77%|███████▋  | 9229/12000 [5:26:44<1:24:44,  1.83s/it]
 77%|███████▋  | 9230/12000 [5:26:45<1:20:04,  1.73s/it]
                                                        

 77%|███████▋  | 9230/12000 [5:26:45<1:20:04,  1.73s/it]
 77%|███████▋  | 9231/12000 [5:26:48<1:24:11,  1.82s/it]
                                                        

 77%|███████▋  | 9231/12000 [5:26:48<1:24:11,  1.82s/it]
 77%|███████▋  | 9232/12000 [5:26:49<1:23:48,  1.82s/it]
                                                        

 77%|███████▋  | 9232/12000 [5:26:49<1:23:48,  1.82s/it]
 77%|███████▋  | 9233/12000 [5:26:52<1:31:40,  1.99s/it]
                                                        

 77%|███████▋  | 9233/12000 [5:26:52<1:31:40,  1.99s/it]
 77%|███████▋  | 9234/12000 [5:26:54<1:36:55,  2.10s/it]
                                                        

 77%|███████▋  | 9234/12000 [5:26:54<1:36:55,  2.10s/it]
 77%|███████▋  | 9235/12000 [5:26:56<1:30:45,  1.97s/it]
                                                        

 77%|███████▋  | 9235/12000 [5:26:56<1:30:45,  1.97s/it]
 77%|███████▋  | 9236/12000 [5:26:57<1:27:59,  1.91s/it]
                                                        

 77%|███████▋  | 9236/12000 [5:26:57<1:27:59,  1.91s/it]
 77%|███████▋  | 9237/12000 [5:26:59<1:21:13,  1.76s/it]
                                                        

 77%|███████▋  | 9237/12000 [5:26:59<1:21:13,  1.76s/it]
 77%|███████▋  | 9238/12000 [5:27:02<1:33:30,  2.03s/it]
                                                        

 77%|███████▋  | 9238/12000 [5:27:02<1:33:30,  2.03s/it]
 77%|███████▋  | 9239/12000 [5:27:03<1:27:41,  1.91s/it]
                                                        

 77%|███████▋  | 9239/12000 [5:27:03<1:27:41,  1.91s/it]
 77%|███████▋  | 9240/12000 [5:27:06<1:36:14,  2.09s/it]
                                                        

 77%|███████▋  | 9240/12000 [5:27:06<1:36:14,  2.09s/it]
 77%|███████▋  | 9241/12000 [5:27:08<1:39:00,  2.15s/it]
                                                        

 77%|███████▋  | 9241/12000 [5:27:08<1:39:00,  2.15s/it]
 77%|███████▋  | 9242/12000 [5:27:10<1:31:09,  1.98s/it]
                                                        

 77%|███████▋  | 9242/12000 [5:27:10<1:31:09,  1.98s/it]
 77%|███████▋  | 9243/12000 [5:27:11<1:23:02,  1.81s/it]
                                                        

 77%|███████▋  | 9243/12000 [5:27:11<1:23:02,  1.81s/it]
 77%|███████▋  | 9244/12000 [5:27:13<1:19:03,  1.72s/it]
                                                        

 77%|███████▋  | 9244/12000 [5:27:13<1:19:03,  1.72s/it]
 77%|███████▋  | 9245/12000 [5:27:15<1:33:04,  2.03s/it]
                                                        

 77%|███████▋  | 9245/12000 [5:27:15<1:33:04,  2.03s/it]
 77%|███████▋  | 9246/12000 [5:27:17<1:26:35,  1.89s/it]
                                                        

 77%|███████▋  | 9246/12000 [5:27:17<1:26:35,  1.89s/it]
 77%|███████▋  | 9247/12000 [5:27:19<1:27:55,  1.92s/it]
                                                        

 77%|███████▋  | 9247/12000 [5:27:19<1:27:55,  1.92s/it]
 77%|███████▋  | 9248/12000 [5:27:21<1:33:31,  2.04s/it]
                                                        

 77%|███████▋  | 9248/12000 [5:27:21<1:33:31,  2.04s/it]
 77%|███████▋  | 9249/12000 [5:27:25<1:56:24,  2.54s/it]
                                                        

 77%|███████▋  | 9249/12000 [5:27:25<1:56:24,  2.54s/it]
 77%|███████▋  | 9250/12000 [5:27:26<1:43:41,  2.26s/it]
                                                        

 77%|███████▋  | 9250/12000 [5:27:26<1:43:41,  2.26s/it]
 77%|███████▋  | 9251/12000 [5:27:30<1:58:58,  2.60s/it]
                                                        

 77%|███████▋  | 9251/12000 [5:27:30<1:58:58,  2.60s/it]
 77%|███████▋  | 9252/12000 [5:27:32<1:46:53,  2.33s/it]
                                                        

 77%|███████▋  | 9252/12000 [5:27:32<1:46:53,  2.33s/it]
 77%|███████▋  | 9253/12000 [5:27:33<1:35:28,  2.09s/it]
                                                        

 77%|███████▋  | 9253/12000 [5:27:33<1:35:28,  2.09s/it]
 77%|███████▋  | 9254/12000 [5:27:35<1:29:01,  1.95s/it]
                                                        

 77%|███████▋  | 9254/12000 [5:27:35<1:29:01,  1.95s/it]
 77%|███████▋  | 9255/12000 [5:27:37<1:28:49,  1.94s/it]
                                                        

 77%|███████▋  | 9255/12000 [5:27:37<1:28:49,  1.94s/it]
 77%|███████▋  | 9256/12000 [5:27:38<1:24:06,  1.84s/it]
                                                        

 77%|███████▋  | 9256/12000 [5:27:38<1:24:06,  1.84s/it]
 77%|███████▋  | 9257/12000 [5:27:40<1:18:33,  1.72s/it]
                                                        

 77%|███████▋  | 9257/12000 [5:27:40<1:18:33,  1.72s/it]
 77%|███████▋  | 9258/12000 [5:27:42<1:22:40,  1.81s/it]
                                                        

 77%|███████▋  | 9258/12000 [5:27:42<1:22:40,  1.81s/it]
 77%|███████▋  | 9259/12000 [5:27:43<1:18:48,  1.73s/it]
                                                        

 77%|███████▋  | 9259/12000 [5:27:43<1:18:48,  1.73s/it]
 77%|███████▋  | 9260/12000 [5:27:45<1:19:02,  1.73s/it]
                                                        

 77%|███████▋  | 9260/12000 [5:27:45<1:19:02,  1.73s/it]
 77%|███████▋  | 9261/12000 [5:27:47<1:20:16,  1.76s/it]
                                                        

 77%|███████▋  | 9261/12000 [5:27:47<1:20:16,  1.76s/it]
 77%|███████▋  | 9262/12000 [5:27:50<1:35:12,  2.09s/it]
                                                        

 77%|███████▋  | 9262/12000 [5:27:50<1:35:12,  2.09s/it]
 77%|███████▋  | 9263/12000 [5:27:52<1:37:49,  2.14s/it]
                                                        
{'loss': '0.8875', 'grad_norm': '0.4546', 'learning_rate': '2.803e-05', 'entropy': '0.9166', 'num_tokens': '4.965e+07', 'mean_token_accuracy': '0.7633', 'epoch': '0.08156'}
{'loss': '0.6101', 'grad_norm': '0.3944', 'learning_rate': '2.801e-05', 'entropy': '0.639', 'num_tokens': '4.965e+07', 'mean_token_accuracy': '0.8279', 'epoch': '0.08157'}
{'loss': '0.8557', 'grad_norm': '0.4134', 'learning_rate': '2.8e-05', 'entropy': '0.8512', 'num_tokens': '4.966e+07', 'mean_token_accuracy': '0.7902', 'epoch': '0.08158'}
{'loss': '0.7371', 'grad_norm': '0.3355', 'learning_rate': '2.798e-05', 'entropy': '0.7527', 'num_tokens': '4.967e+07', 'mean_token_accuracy': '0.8099', 'epoch': '0.08159'}
{'loss': '0.953', 'grad_norm': '0.4762', 'learning_rate': '2.796e-05', 'entropy': '0.8742', 'num_tokens': '4.967e+07', 'mean_token_accuracy': '0.7877', 'epoch': '0.0816'}
{'loss': '0.6605', 'grad_norm': '0.4192', 'learning_rate': '2.794e-05', 'entropy': '0.6786', 'num_tokens': '4.968e+07', 'mean_token_accuracy': '0.8301', 'epoch': '0.0816'}
{'loss': '0.6705', 'grad_norm': '0.3954', 'learning_rate': '2.792e-05', 'entropy': '0.713', 'num_tokens': '4.968e+07', 'mean_token_accuracy': '0.8243', 'epoch': '0.08161'}
{'loss': '0.6805', 'grad_norm': '0.3392', 'learning_rate': '2.79e-05', 'entropy': '0.7597', 'num_tokens': '4.969e+07', 'mean_token_accuracy': '0.8051', 'epoch': '0.08162'}
{'loss': '0.7035', 'grad_norm': '0.3557', 'learning_rate': '2.788e-05', 'entropy': '0.7676', 'num_tokens': '4.969e+07', 'mean_token_accuracy': '0.808', 'epoch': '0.08163'}
{'loss': '0.7367', 'grad_norm': '0.4676', 'learning_rate': '2.786e-05', 'entropy': '0.7809', 'num_tokens': '4.97e+07', 'mean_token_accuracy': '0.7915', 'epoch': '0.08164'}
{'loss': '0.7933', 'grad_norm': '0.4383', 'learning_rate': '2.784e-05', 'entropy': '0.8209', 'num_tokens': '4.97e+07', 'mean_token_accuracy': '0.7913', 'epoch': '0.08165'}
{'loss': '1.001', 'grad_norm': '0.4676', 'learning_rate': '2.782e-05', 'entropy': '1.044', 'num_tokens': '4.971e+07', 'mean_token_accuracy': '0.742', 'epoch': '0.08166'}
{'loss': '0.8052', 'grad_norm': '0.3233', 'learning_rate': '2.78e-05', 'entropy': '0.7987', 'num_tokens': '4.971e+07', 'mean_token_accuracy': '0.7945', 'epoch': '0.08167'}
{'loss': '0.8447', 'grad_norm': '0.5136', 'learning_rate': '2.779e-05', 'entropy': '0.9788', 'num_tokens': '4.971e+07', 'mean_token_accuracy': '0.7632', 'epoch': '0.08168'}
{'loss': '0.844', 'grad_norm': '0.4196', 'learning_rate': '2.777e-05', 'entropy': '0.8538', 'num_tokens': '4.972e+07', 'mean_token_accuracy': '0.7906', 'epoch': '0.08168'}
{'loss': '0.8365', 'grad_norm': '0.3899', 'learning_rate': '2.775e-05', 'entropy': '0.831', 'num_tokens': '4.972e+07', 'mean_token_accuracy': '0.7865', 'epoch': '0.08169'}
{'loss': '0.8489', 'grad_norm': '0.4295', 'learning_rate': '2.773e-05', 'entropy': '0.8039', 'num_tokens': '4.973e+07', 'mean_token_accuracy': '0.8064', 'epoch': '0.0817'}
{'loss': '0.7172', 'grad_norm': '0.4015', 'learning_rate': '2.771e-05', 'entropy': '0.6614', 'num_tokens': '4.974e+07', 'mean_token_accuracy': '0.8299', 'epoch': '0.08171'}
{'loss': '1.236', 'grad_norm': '0.4525', 'learning_rate': '2.769e-05', 'entropy': '1.167', 'num_tokens': '4.974e+07', 'mean_token_accuracy': '0.6946', 'epoch': '0.08172'}
{'loss': '1.138', 'grad_norm': '0.4577', 'learning_rate': '2.767e-05', 'entropy': '1.145', 'num_tokens': '4.975e+07', 'mean_token_accuracy': '0.7263', 'epoch': '0.08173'}
{'loss': '0.8601', 'grad_norm': '0.4644', 'learning_rate': '2.765e-05', 'entropy': '0.8653', 'num_tokens': '4.975e+07', 'mean_token_accuracy': '0.7873', 'epoch': '0.08174'}
{'loss': '0.5283', 'grad_norm': '0.3279', 'learning_rate': '2.763e-05', 'entropy': '0.5639', 'num_tokens': '4.976e+07', 'mean_token_accuracy': '0.8485', 'epoch': '0.08175'}
{'loss': '0.6929', 'grad_norm': '0.4331', 'learning_rate': '2.761e-05', 'entropy': '0.7443', 'num_tokens': '4.976e+07', 'mean_token_accuracy': '0.811', 'epoch': '0.08175'}
{'loss': '0.5658', 'grad_norm': '0.437', 'learning_rate': '2.76e-05', 'entropy': '0.5889', 'num_tokens': '4.977e+07', 'mean_token_accuracy': '0.8334', 'epoch': '0.08176'}
{'loss': '0.8176', 'grad_norm': '0.402', 'learning_rate': '2.758e-05', 'entropy': '0.9027', 'num_tokens': '4.977e+07', 'mean_token_accuracy': '0.7675', 'epoch': '0.08177'}
{'loss': '1.474', 'grad_norm': '0.3479', 'learning_rate': '2.756e-05', 'entropy': '1.267', 'num_tokens': '4.978e+07', 'mean_token_accuracy': '0.7007', 'epoch': '0.08178'}
{'loss': '0.5348', 'grad_norm': '0.3853', 'learning_rate': '2.754e-05', 'entropy': '0.5764', 'num_tokens': '4.979e+07', 'mean_token_accuracy': '0.8463', 'epoch': '0.08179'}
{'loss': '0.688', 'grad_norm': '0.3575', 'learning_rate': '2.752e-05', 'entropy': '0.6791', 'num_tokens': '4.979e+07', 'mean_token_accuracy': '0.8247', 'epoch': '0.0818'}
{'loss': '0.624', 'grad_norm': '0.4057', 'learning_rate': '2.75e-05', 'entropy': '0.6108', 'num_tokens': '4.979e+07', 'mean_token_accuracy': '0.8427', 'epoch': '0.08181'}
{'loss': '0.9607', 'grad_norm': '0.3245', 'learning_rate': '2.748e-05', 'entropy': '0.8122', 'num_tokens': '4.98e+07', 'mean_token_accuracy': '0.7869', 'epoch': '0.08182'}
{'loss': '0.9033', 'grad_norm': '0.4748', 'learning_rate': '2.746e-05', 'entropy': '0.9243', 'num_tokens': '4.981e+07', 'mean_token_accuracy': '0.7714', 'epoch': '0.08183'}
{'loss': '1.114', 'grad_norm': '0.4592', 'learning_rate': '2.744e-05', 'entropy': '1.02', 'num_tokens': '4.981e+07', 'mean_token_accuracy': '0.7451', 'epoch': '0.08183'}
{'loss': '0.9033', 'grad_norm': '0.3161', 'learning_rate': '2.742e-05', 'entropy': '0.9565', 'num_tokens': '4.982e+07', 'mean_token_accuracy': '0.7662', 'epoch': '0.08184'}
{'loss': '0.7658', 'grad_norm': '0.4134', 'learning_rate': '2.741e-05', 'entropy': '0.7687', 'num_tokens': '4.982e+07', 'mean_token_accuracy': '0.8014', 'epoch': '0.08185'}
{'loss': '0.5128', 'grad_norm': '0.3482', 'learning_rate': '2.739e-05', 'entropy': '0.5207', 'num_tokens': '4.983e+07', 'mean_token_accuracy': '0.8643', 'epoch': '0.08186'}
{'loss': '0.9016', 'grad_norm': '0.4172', 'learning_rate': '2.737e-05', 'entropy': '0.9869', 'num_tokens': '4.983e+07', 'mean_token_accuracy': '0.7512', 'epoch': '0.08187'}
{'loss': '1.066', 'grad_norm': '0.3893', 'learning_rate': '2.735e-05', 'entropy': '1.041', 'num_tokens': '4.984e+07', 'mean_token_accuracy': '0.7446', 'epoch': '0.08188'}
{'loss': '0.6061', 'grad_norm': '0.3964', 'learning_rate': '2.733e-05', 'entropy': '0.6083', 'num_tokens': '4.984e+07', 'mean_token_accuracy': '0.8317', 'epoch': '0.08189'}
{'loss': '1.121', 'grad_norm': '0.4212', 'learning_rate': '2.731e-05', 'entropy': '1.101', 'num_tokens': '4.985e+07', 'mean_token_accuracy': '0.7473', 'epoch': '0.0819'}
{'loss': '0.7834', 'grad_norm': '0.4717', 'learning_rate': '2.729e-05', 'entropy': '0.7225', 'num_tokens': '4.985e+07', 'mean_token_accuracy': '0.8213', 'epoch': '0.08191'}
{'loss': '0.6146', 'grad_norm': '0.3389', 'learning_rate': '2.727e-05', 'entropy': '0.5985', 'num_tokens': '4.986e+07', 'mean_token_accuracy': '0.8414', 'epoch': '0.08191'}
{'loss': '0.7523', 'grad_norm': '0.4996', 'learning_rate': '2.725e-05', 'entropy': '0.7773', 'num_tokens': '4.986e+07', 'mean_token_accuracy': '0.8016', 'epoch': '0.08192'}
{'loss': '0.5658', 'grad_norm': '0.3277', 'learning_rate': '2.724e-05', 'entropy': '0.5415', 'num_tokens': '4.987e+07', 'mean_token_accuracy': '0.8577', 'epoch': '0.08193'}
{'loss': '0.7649', 'grad_norm': '0.425', 'learning_rate': '2.722e-05', 'entropy': '0.7638', 'num_tokens': '4.987e+07', 'mean_token_accuracy': '0.8106', 'epoch': '0.08194'}
{'loss': '0.832', 'grad_norm': '0.4425', 'learning_rate': '2.72e-05', 'entropy': '0.864', 'num_tokens': '4.988e+07', 'mean_token_accuracy': '0.7823', 'epoch': '0.08195'}
{'loss': '0.7725', 'grad_norm': '0.444', 'learning_rate': '2.718e-05', 'entropy': '0.7507', 'num_tokens': '4.988e+07', 'mean_token_accuracy': '0.8078', 'epoch': '0.08196'}
{'loss': '0.7255', 'grad_norm': '0.3294', 'learning_rate': '2.716e-05', 'entropy': '0.695', 'num_tokens': '4.989e+07', 'mean_token_accuracy': '0.8296', 'epoch': '0.08197'}

 77%|███████▋  | 9263/12000 [5:27:52<1:37:49,  2.14s/it]
 77%|███████▋  | 9264/12000 [5:27:54<1:32:29,  2.03s/it]
                                                        

 77%|███████▋  | 9264/12000 [5:27:54<1:32:29,  2.03s/it]
 77%|███████▋  | 9265/12000 [5:27:55<1:27:04,  1.91s/it]
                                                        

 77%|███████▋  | 9265/12000 [5:27:55<1:27:04,  1.91s/it]
 77%|███████▋  | 9266/12000 [5:27:58<1:35:21,  2.09s/it]
                                                        

 77%|███████▋  | 9266/12000 [5:27:58<1:35:21,  2.09s/it]
 77%|███████▋  | 9267/12000 [5:28:00<1:37:26,  2.14s/it]
                                                        

 77%|███████▋  | 9267/12000 [5:28:00<1:37:26,  2.14s/it]
 77%|███████▋  | 9268/12000 [5:28:02<1:33:57,  2.06s/it]
                                                        

 77%|███████▋  | 9268/12000 [5:28:02<1:33:57,  2.06s/it]
 77%|███████▋  | 9269/12000 [5:28:03<1:26:52,  1.91s/it]
                                                        

 77%|███████▋  | 9269/12000 [5:28:03<1:26:52,  1.91s/it]
 77%|███████▋  | 9270/12000 [5:28:05<1:22:16,  1.81s/it]
                                                        

 77%|███████▋  | 9270/12000 [5:28:05<1:22:16,  1.81s/it]
 77%|███████▋  | 9271/12000 [5:28:09<1:46:09,  2.33s/it]
                                                        

 77%|███████▋  | 9271/12000 [5:28:09<1:46:09,  2.33s/it]
 77%|███████▋  | 9272/12000 [5:28:10<1:34:51,  2.09s/it]
                                                        

 77%|███████▋  | 9272/12000 [5:28:10<1:34:51,  2.09s/it]
 77%|███████▋  | 9273/12000 [5:28:12<1:36:24,  2.12s/it]
                                                        

 77%|███████▋  | 9273/12000 [5:28:12<1:36:24,  2.12s/it]
 77%|███████▋  | 9274/12000 [5:28:14<1:31:16,  2.01s/it]
                                                        

 77%|███████▋  | 9274/12000 [5:28:14<1:31:16,  2.01s/it]
 77%|███████▋  | 9275/12000 [5:28:16<1:30:54,  2.00s/it]
                                                        

 77%|███████▋  | 9275/12000 [5:28:16<1:30:54,  2.00s/it]
 77%|███████▋  | 9276/12000 [5:28:18<1:34:24,  2.08s/it]
                                                        

 77%|███████▋  | 9276/12000 [5:28:18<1:34:24,  2.08s/it]
 77%|███████▋  | 9277/12000 [5:28:20<1:26:48,  1.91s/it]
                                                        

 77%|███████▋  | 9277/12000 [5:28:20<1:26:48,  1.91s/it]
 77%|███████▋  | 9278/12000 [5:28:22<1:24:07,  1.85s/it]
                                                        

 77%|███████▋  | 9278/12000 [5:28:22<1:24:07,  1.85s/it]
 77%|███████▋  | 9279/12000 [5:28:23<1:17:35,  1.71s/it]
                                                        

 77%|███████▋  | 9279/12000 [5:28:23<1:17:35,  1.71s/it]
 77%|███████▋  | 9280/12000 [5:28:24<1:15:40,  1.67s/it]
                                                        

 77%|███████▋  | 9280/12000 [5:28:24<1:15:40,  1.67s/it]
 77%|███████▋  | 9281/12000 [5:28:26<1:15:10,  1.66s/it]
                                                        

 77%|███████▋  | 9281/12000 [5:28:26<1:15:10,  1.66s/it]
 77%|███████▋  | 9282/12000 [5:28:28<1:15:25,  1.66s/it]
                                                        

 77%|███████▋  | 9282/12000 [5:28:28<1:15:25,  1.66s/it]
 77%|███████▋  | 9283/12000 [5:28:30<1:18:21,  1.73s/it]
                                                        

 77%|███████▋  | 9283/12000 [5:28:30<1:18:21,  1.73s/it]
 77%|███████▋  | 9284/12000 [5:28:31<1:14:03,  1.64s/it]
                                                        

 77%|███████▋  | 9284/12000 [5:28:31<1:14:03,  1.64s/it]
 77%|███████▋  | 9285/12000 [5:28:33<1:13:28,  1.62s/it]
                                                        

 77%|███████▋  | 9285/12000 [5:28:33<1:13:28,  1.62s/it]
 77%|███████▋  | 9286/12000 [5:28:35<1:17:08,  1.71s/it]
                                                        

 77%|███████▋  | 9286/12000 [5:28:35<1:17:08,  1.71s/it]
 77%|███████▋  | 9287/12000 [5:28:36<1:17:52,  1.72s/it]
                                                        

 77%|███████▋  | 9287/12000 [5:28:36<1:17:52,  1.72s/it]
 77%|███████▋  | 9288/12000 [5:28:38<1:22:57,  1.84s/it]
                                                        

 77%|███████▋  | 9288/12000 [5:28:38<1:22:57,  1.84s/it]
 77%|███████▋  | 9289/12000 [5:28:40<1:24:04,  1.86s/it]
                                                        

 77%|███████▋  | 9289/12000 [5:28:40<1:24:04,  1.86s/it]
 77%|███████▋  | 9290/12000 [5:28:42<1:23:01,  1.84s/it]
                                                        

 77%|███████▋  | 9290/12000 [5:28:42<1:23:01,  1.84s/it]
 77%|███████▋  | 9291/12000 [5:28:44<1:20:05,  1.77s/it]
                                                        

 77%|███████▋  | 9291/12000 [5:28:44<1:20:05,  1.77s/it]
 77%|███████▋  | 9292/12000 [5:28:46<1:29:24,  1.98s/it]
                                                        

 77%|███████▋  | 9292/12000 [5:28:46<1:29:24,  1.98s/it]
 77%|███████▋  | 9293/12000 [5:28:50<1:52:44,  2.50s/it]
                                                        

 77%|███████▋  | 9293/12000 [5:28:50<1:52:44,  2.50s/it]
 77%|███████▋  | 9294/12000 [5:28:52<1:44:25,  2.32s/it]
                                                        

 77%|███████▋  | 9294/12000 [5:28:52<1:44:25,  2.32s/it]
 77%|███████▋  | 9295/12000 [5:28:53<1:34:55,  2.11s/it]
                                                        

 77%|███████▋  | 9295/12000 [5:28:53<1:34:55,  2.11s/it]
 77%|███████▋  | 9296/12000 [5:28:55<1:32:29,  2.05s/it]
                                                        

 77%|███████▋  | 9296/12000 [5:28:55<1:32:29,  2.05s/it]
 77%|███████▋  | 9297/12000 [5:28:57<1:28:02,  1.95s/it]
                                                        

 77%|███████▋  | 9297/12000 [5:28:57<1:28:02,  1.95s/it]
 77%|███████▋  | 9298/12000 [5:28:58<1:20:12,  1.78s/it]
                                                        

 77%|███████▋  | 9298/12000 [5:28:58<1:20:12,  1.78s/it]
 77%|███████▋  | 9299/12000 [5:29:02<1:44:57,  2.33s/it]
                                                        

 77%|███████▋  | 9299/12000 [5:29:02<1:44:57,  2.33s/it]
 78%|███████▊  | 9300/12000 [5:29:04<1:41:38,  2.26s/it]
                                                        

 78%|███████▊  | 9300/12000 [5:29:04<1:41:38,  2.26s/it]
 78%|███████▊  | 9301/12000 [5:29:07<1:50:14,  2.45s/it]
                                                        

 78%|███████▊  | 9301/12000 [5:29:07<1:50:14,  2.45s/it]
 78%|███████▊  | 9302/12000 [5:29:09<1:41:19,  2.25s/it]
                                                        

 78%|███████▊  | 9302/12000 [5:29:09<1:41:19,  2.25s/it]
 78%|███████▊  | 9303/12000 [5:29:11<1:33:58,  2.09s/it]
                                                        

 78%|███████▊  | 9303/12000 [5:29:11<1:33:58,  2.09s/it]
 78%|███████▊  | 9304/12000 [5:29:12<1:25:20,  1.90s/it]
                                                        

 78%|███████▊  | 9304/12000 [5:29:12<1:25:20,  1.90s/it]
 78%|███████▊  | 9305/12000 [5:29:14<1:23:15,  1.85s/it]
                                                        

 78%|███████▊  | 9305/12000 [5:29:14<1:23:15,  1.85s/it]
 78%|███████▊  | 9306/12000 [5:29:16<1:25:11,  1.90s/it]
                                                        

 78%|███████▊  | 9306/12000 [5:29:16<1:25:11,  1.90s/it]
 78%|███████▊  | 9307/12000 [5:29:18<1:23:06,  1.85s/it]
                                                        

 78%|███████▊  | 9307/12000 [5:29:18<1:23:06,  1.85s/it]
 78%|███████▊  | 9308/12000 [5:29:20<1:35:54,  2.14s/it]
                                                        

 78%|███████▊  | 9308/12000 [5:29:20<1:35:54,  2.14s/it]
 78%|███████▊  | 9309/12000 [5:29:22<1:30:07,  2.01s/it]
                                                        

 78%|███████▊  | 9309/12000 [5:29:22<1:30:07,  2.01s/it]
 78%|███████▊  | 9310/12000 [5:29:23<1:21:22,  1.82s/it]
                                                        
{'loss': '1.054', 'grad_norm': '0.4099', 'learning_rate': '2.714e-05', 'entropy': '1.046', 'num_tokens': '4.99e+07', 'mean_token_accuracy': '0.754', 'epoch': '0.08198'}
{'loss': '0.7968', 'grad_norm': '0.4227', 'learning_rate': '2.712e-05', 'entropy': '0.8506', 'num_tokens': '4.99e+07', 'mean_token_accuracy': '0.766', 'epoch': '0.08198'}
{'loss': '0.7594', 'grad_norm': '0.3792', 'learning_rate': '2.71e-05', 'entropy': '0.784', 'num_tokens': '4.991e+07', 'mean_token_accuracy': '0.801', 'epoch': '0.08199'}
{'loss': '1.366', 'grad_norm': '0.4473', 'learning_rate': '2.708e-05', 'entropy': '1.244', 'num_tokens': '4.991e+07', 'mean_token_accuracy': '0.6829', 'epoch': '0.082'}
{'loss': '0.7327', 'grad_norm': '0.3504', 'learning_rate': '2.707e-05', 'entropy': '0.6751', 'num_tokens': '4.992e+07', 'mean_token_accuracy': '0.8281', 'epoch': '0.08201'}
{'loss': '1.145', 'grad_norm': '0.4075', 'learning_rate': '2.705e-05', 'entropy': '1.064', 'num_tokens': '4.993e+07', 'mean_token_accuracy': '0.7406', 'epoch': '0.08202'}
{'loss': '0.8388', 'grad_norm': '0.4615', 'learning_rate': '2.703e-05', 'entropy': '0.8582', 'num_tokens': '4.993e+07', 'mean_token_accuracy': '0.777', 'epoch': '0.08203'}
{'loss': '0.6789', 'grad_norm': '0.3808', 'learning_rate': '2.701e-05', 'entropy': '0.7477', 'num_tokens': '4.993e+07', 'mean_token_accuracy': '0.8213', 'epoch': '0.08204'}
{'loss': '1.069', 'grad_norm': '0.5008', 'learning_rate': '2.699e-05', 'entropy': '1.081', 'num_tokens': '4.994e+07', 'mean_token_accuracy': '0.7425', 'epoch': '0.08205'}
{'loss': '0.6837', 'grad_norm': '0.4103', 'learning_rate': '2.697e-05', 'entropy': '0.7049', 'num_tokens': '4.994e+07', 'mean_token_accuracy': '0.8172', 'epoch': '0.08206'}
{'loss': '0.5574', 'grad_norm': '0.2925', 'learning_rate': '2.695e-05', 'entropy': '0.6841', 'num_tokens': '4.995e+07', 'mean_token_accuracy': '0.8132', 'epoch': '0.08206'}
{'loss': '0.7749', 'grad_norm': '0.3477', 'learning_rate': '2.693e-05', 'entropy': '0.7574', 'num_tokens': '4.996e+07', 'mean_token_accuracy': '0.7956', 'epoch': '0.08207'}
{'loss': '1.08', 'grad_norm': '0.5585', 'learning_rate': '2.691e-05', 'entropy': '0.9949', 'num_tokens': '4.996e+07', 'mean_token_accuracy': '0.7415', 'epoch': '0.08208'}
{'loss': '0.8933', 'grad_norm': '0.4369', 'learning_rate': '2.69e-05', 'entropy': '0.8257', 'num_tokens': '4.997e+07', 'mean_token_accuracy': '0.7909', 'epoch': '0.08209'}
{'loss': '0.9368', 'grad_norm': '0.3657', 'learning_rate': '2.688e-05', 'entropy': '0.9524', 'num_tokens': '4.997e+07', 'mean_token_accuracy': '0.7556', 'epoch': '0.0821'}
{'loss': '0.6178', 'grad_norm': '0.3909', 'learning_rate': '2.686e-05', 'entropy': '0.6302', 'num_tokens': '4.998e+07', 'mean_token_accuracy': '0.8302', 'epoch': '0.08211'}
{'loss': '0.6678', 'grad_norm': '0.3794', 'learning_rate': '2.684e-05', 'entropy': '0.7259', 'num_tokens': '4.998e+07', 'mean_token_accuracy': '0.8122', 'epoch': '0.08212'}
{'loss': '0.6917', 'grad_norm': '0.3353', 'learning_rate': '2.682e-05', 'entropy': '0.6787', 'num_tokens': '4.998e+07', 'mean_token_accuracy': '0.8232', 'epoch': '0.08213'}
{'loss': '0.9092', 'grad_norm': '0.4071', 'learning_rate': '2.68e-05', 'entropy': '0.8514', 'num_tokens': '4.999e+07', 'mean_token_accuracy': '0.7985', 'epoch': '0.08214'}
{'loss': '0.7791', 'grad_norm': '0.4613', 'learning_rate': '2.678e-05', 'entropy': '0.7189', 'num_tokens': '4.999e+07', 'mean_token_accuracy': '0.8192', 'epoch': '0.08214'}
{'loss': '1.396', 'grad_norm': '0.4726', 'learning_rate': '2.676e-05', 'entropy': '1.337', 'num_tokens': '5e+07', 'mean_token_accuracy': '0.6799', 'epoch': '0.08215'}
{'loss': '0.9994', 'grad_norm': '0.4218', 'learning_rate': '2.675e-05', 'entropy': '0.9725', 'num_tokens': '5e+07', 'mean_token_accuracy': '0.7682', 'epoch': '0.08216'}
{'loss': '1.042', 'grad_norm': '0.4623', 'learning_rate': '2.673e-05', 'entropy': '1.08', 'num_tokens': '5.001e+07', 'mean_token_accuracy': '0.7429', 'epoch': '0.08217'}
{'loss': '0.5458', 'grad_norm': '0.3493', 'learning_rate': '2.671e-05', 'entropy': '0.5682', 'num_tokens': '5.001e+07', 'mean_token_accuracy': '0.8446', 'epoch': '0.08218'}
{'loss': '0.9173', 'grad_norm': '0.4913', 'learning_rate': '2.669e-05', 'entropy': '0.8756', 'num_tokens': '5.002e+07', 'mean_token_accuracy': '0.7775', 'epoch': '0.08219'}
{'loss': '0.8636', 'grad_norm': '0.5061', 'learning_rate': '2.667e-05', 'entropy': '0.8666', 'num_tokens': '5.002e+07', 'mean_token_accuracy': '0.7946', 'epoch': '0.0822'}
{'loss': '1.066', 'grad_norm': '0.4366', 'learning_rate': '2.665e-05', 'entropy': '1.051', 'num_tokens': '5.003e+07', 'mean_token_accuracy': '0.7416', 'epoch': '0.08221'}
{'loss': '0.6547', 'grad_norm': '0.5643', 'learning_rate': '2.663e-05', 'entropy': '0.6813', 'num_tokens': '5.003e+07', 'mean_token_accuracy': '0.821', 'epoch': '0.08221'}
{'loss': '0.8183', 'grad_norm': '0.3595', 'learning_rate': '2.661e-05', 'entropy': '0.8159', 'num_tokens': '5.004e+07', 'mean_token_accuracy': '0.7994', 'epoch': '0.08222'}
{'loss': '0.6997', 'grad_norm': '0.3927', 'learning_rate': '2.66e-05', 'entropy': '0.7212', 'num_tokens': '5.004e+07', 'mean_token_accuracy': '0.8141', 'epoch': '0.08223'}
{'loss': '0.6929', 'grad_norm': '0.4884', 'learning_rate': '2.658e-05', 'entropy': '0.7351', 'num_tokens': '5.005e+07', 'mean_token_accuracy': '0.7981', 'epoch': '0.08224'}
{'loss': '0.9699', 'grad_norm': '0.5196', 'learning_rate': '2.656e-05', 'entropy': '0.9468', 'num_tokens': '5.005e+07', 'mean_token_accuracy': '0.761', 'epoch': '0.08225'}
{'loss': '0.9148', 'grad_norm': '0.3953', 'learning_rate': '2.654e-05', 'entropy': '0.846', 'num_tokens': '5.006e+07', 'mean_token_accuracy': '0.8015', 'epoch': '0.08226'}
{'loss': '0.7864', 'grad_norm': '0.3878', 'learning_rate': '2.652e-05', 'entropy': '0.8157', 'num_tokens': '5.006e+07', 'mean_token_accuracy': '0.8004', 'epoch': '0.08227'}
{'loss': '0.6998', 'grad_norm': '0.3911', 'learning_rate': '2.65e-05', 'entropy': '0.6939', 'num_tokens': '5.007e+07', 'mean_token_accuracy': '0.8167', 'epoch': '0.08228'}
{'loss': '1.081', 'grad_norm': '0.4822', 'learning_rate': '2.648e-05', 'entropy': '1.177', 'num_tokens': '5.007e+07', 'mean_token_accuracy': '0.7199', 'epoch': '0.08229'}
{'loss': '0.5983', 'grad_norm': '0.3027', 'learning_rate': '2.646e-05', 'entropy': '0.7253', 'num_tokens': '5.008e+07', 'mean_token_accuracy': '0.8106', 'epoch': '0.08229'}
{'loss': '0.8538', 'grad_norm': '0.4119', 'learning_rate': '2.645e-05', 'entropy': '0.831', 'num_tokens': '5.009e+07', 'mean_token_accuracy': '0.7962', 'epoch': '0.0823'}
{'loss': '0.6693', 'grad_norm': '0.3663', 'learning_rate': '2.643e-05', 'entropy': '0.7659', 'num_tokens': '5.009e+07', 'mean_token_accuracy': '0.8102', 'epoch': '0.08231'}
{'loss': '0.6826', 'grad_norm': '0.397', 'learning_rate': '2.641e-05', 'entropy': '0.668', 'num_tokens': '5.01e+07', 'mean_token_accuracy': '0.8144', 'epoch': '0.08232'}
{'loss': '0.7912', 'grad_norm': '0.3748', 'learning_rate': '2.639e-05', 'entropy': '0.8383', 'num_tokens': '5.01e+07', 'mean_token_accuracy': '0.7823', 'epoch': '0.08233'}
{'loss': '0.7831', 'grad_norm': '0.4296', 'learning_rate': '2.637e-05', 'entropy': '0.7688', 'num_tokens': '5.011e+07', 'mean_token_accuracy': '0.7965', 'epoch': '0.08234'}
{'loss': '0.9225', 'grad_norm': '0.4196', 'learning_rate': '2.635e-05', 'entropy': '0.9501', 'num_tokens': '5.011e+07', 'mean_token_accuracy': '0.7649', 'epoch': '0.08235'}
{'loss': '1.122', 'grad_norm': '0.3722', 'learning_rate': '2.633e-05', 'entropy': '1.024', 'num_tokens': '5.012e+07', 'mean_token_accuracy': '0.765', 'epoch': '0.08236'}
{'loss': '0.8006', 'grad_norm': '0.4796', 'learning_rate': '2.632e-05', 'entropy': '0.7317', 'num_tokens': '5.012e+07', 'mean_token_accuracy': '0.8162', 'epoch': '0.08237'}
{'loss': '0.9585', 'grad_norm': '0.3722', 'learning_rate': '2.63e-05', 'entropy': '0.9922', 'num_tokens': '5.013e+07', 'mean_token_accuracy': '0.7671', 'epoch': '0.08237'}
{'loss': '1.191', 'grad_norm': '0.6666', 'learning_rate': '2.628e-05', 'entropy': '1.2', 'num_tokens': '5.013e+07', 'mean_token_accuracy': '0.7227', 'epoch': '0.08238'}

 78%|███████▊  | 9310/12000 [5:29:23<1:21:22,  1.82s/it]
 78%|███████▊  | 9311/12000 [5:29:25<1:18:51,  1.76s/it]
                                                        

 78%|███████▊  | 9311/12000 [5:29:25<1:18:51,  1.76s/it]
 78%|███████▊  | 9312/12000 [5:29:27<1:14:40,  1.67s/it]
                                                        

 78%|███████▊  | 9312/12000 [5:29:27<1:14:40,  1.67s/it]
 78%|███████▊  | 9313/12000 [5:29:28<1:16:32,  1.71s/it]
                                                        

 78%|███████▊  | 9313/12000 [5:29:28<1:16:32,  1.71s/it]
 78%|███████▊  | 9314/12000 [5:29:32<1:39:31,  2.22s/it]
                                                        

 78%|███████▊  | 9314/12000 [5:29:32<1:39:31,  2.22s/it]
 78%|███████▊  | 9315/12000 [5:29:35<1:56:26,  2.60s/it]
                                                        

 78%|███████▊  | 9315/12000 [5:29:35<1:56:26,  2.60s/it]
 78%|███████▊  | 9316/12000 [5:29:38<2:02:40,  2.74s/it]
                                                        

 78%|███████▊  | 9316/12000 [5:29:38<2:02:40,  2.74s/it]
 78%|███████▊  | 9317/12000 [5:29:40<1:48:29,  2.43s/it]
                                                        

 78%|███████▊  | 9317/12000 [5:29:40<1:48:29,  2.43s/it]
 78%|███████▊  | 9318/12000 [5:29:41<1:35:09,  2.13s/it]
                                                        

 78%|███████▊  | 9318/12000 [5:29:41<1:35:09,  2.13s/it]
 78%|███████▊  | 9319/12000 [5:29:43<1:33:24,  2.09s/it]
                                                        

 78%|███████▊  | 9319/12000 [5:29:43<1:33:24,  2.09s/it]
 78%|███████▊  | 9320/12000 [5:29:45<1:28:58,  1.99s/it]
                                                        

 78%|███████▊  | 9320/12000 [5:29:45<1:28:58,  1.99s/it]
 78%|███████▊  | 9321/12000 [5:29:47<1:31:23,  2.05s/it]
                                                        

 78%|███████▊  | 9321/12000 [5:29:47<1:31:23,  2.05s/it]
 78%|███████▊  | 9322/12000 [5:29:49<1:23:59,  1.88s/it]
                                                        

 78%|███████▊  | 9322/12000 [5:29:49<1:23:59,  1.88s/it]
 78%|███████▊  | 9323/12000 [5:29:52<1:44:16,  2.34s/it]
                                                        

 78%|███████▊  | 9323/12000 [5:29:52<1:44:16,  2.34s/it]
 78%|███████▊  | 9324/12000 [5:29:57<2:18:42,  3.11s/it]
                                                        

 78%|███████▊  | 9324/12000 [5:29:57<2:18:42,  3.11s/it]
 78%|███████▊  | 9325/12000 [5:30:00<2:09:10,  2.90s/it]
                                                        

 78%|███████▊  | 9325/12000 [5:30:00<2:09:10,  2.90s/it]
 78%|███████▊  | 9326/12000 [5:30:01<1:49:18,  2.45s/it]
                                                        

 78%|███████▊  | 9326/12000 [5:30:01<1:49:18,  2.45s/it]
 78%|███████▊  | 9327/12000 [5:30:02<1:35:54,  2.15s/it]
                                                        

 78%|███████▊  | 9327/12000 [5:30:02<1:35:54,  2.15s/it]
 78%|███████▊  | 9328/12000 [5:30:05<1:42:02,  2.29s/it]
                                                        

 78%|███████▊  | 9328/12000 [5:30:05<1:42:02,  2.29s/it]
 78%|███████▊  | 9329/12000 [5:30:07<1:32:24,  2.08s/it]
                                                        

 78%|███████▊  | 9329/12000 [5:30:07<1:32:24,  2.08s/it]
 78%|███████▊  | 9330/12000 [5:30:09<1:40:55,  2.27s/it]
                                                        

 78%|███████▊  | 9330/12000 [5:30:09<1:40:55,  2.27s/it]
 78%|███████▊  | 9331/12000 [5:30:12<1:45:05,  2.36s/it]
                                                        

 78%|███████▊  | 9331/12000 [5:30:12<1:45:05,  2.36s/it]
 78%|███████▊  | 9332/12000 [5:30:14<1:39:54,  2.25s/it]
                                                        

 78%|███████▊  | 9332/12000 [5:30:14<1:39:54,  2.25s/it]
 78%|███████▊  | 9333/12000 [5:30:16<1:42:30,  2.31s/it]
                                                        

 78%|███████▊  | 9333/12000 [5:30:16<1:42:30,  2.31s/it]
 78%|███████▊  | 9334/12000 [5:30:18<1:36:46,  2.18s/it]
                                                        

 78%|███████▊  | 9334/12000 [5:30:18<1:36:46,  2.18s/it]
 78%|███████▊  | 9335/12000 [5:30:20<1:34:18,  2.12s/it]
                                                        

 78%|███████▊  | 9335/12000 [5:30:20<1:34:18,  2.12s/it]
 78%|███████▊  | 9336/12000 [5:30:22<1:26:41,  1.95s/it]
                                                        

 78%|███████▊  | 9336/12000 [5:30:22<1:26:41,  1.95s/it]
 78%|███████▊  | 9337/12000 [5:30:26<1:57:27,  2.65s/it]
                                                        

 78%|███████▊  | 9337/12000 [5:30:26<1:57:27,  2.65s/it]
 78%|███████▊  | 9338/12000 [5:30:28<1:49:57,  2.48s/it]
                                                        

 78%|███████▊  | 9338/12000 [5:30:28<1:49:57,  2.48s/it]
 78%|███████▊  | 9339/12000 [5:30:31<1:55:00,  2.59s/it]
                                                        

 78%|███████▊  | 9339/12000 [5:30:31<1:55:00,  2.59s/it]
 78%|███████▊  | 9340/12000 [5:30:33<1:41:28,  2.29s/it]
                                                        

 78%|███████▊  | 9340/12000 [5:30:33<1:41:28,  2.29s/it]
 78%|███████▊  | 9341/12000 [5:30:34<1:31:43,  2.07s/it]
                                                        

 78%|███████▊  | 9341/12000 [5:30:34<1:31:43,  2.07s/it]
 78%|███████▊  | 9342/12000 [5:30:36<1:31:22,  2.06s/it]
                                                        

 78%|███████▊  | 9342/12000 [5:30:36<1:31:22,  2.06s/it]
 78%|███████▊  | 9343/12000 [5:30:38<1:23:25,  1.88s/it]
                                                        

 78%|███████▊  | 9343/12000 [5:30:38<1:23:25,  1.88s/it]
 78%|███████▊  | 9344/12000 [5:30:39<1:19:15,  1.79s/it]
                                                        

 78%|███████▊  | 9344/12000 [5:30:39<1:19:15,  1.79s/it]
 78%|███████▊  | 9345/12000 [5:30:41<1:15:43,  1.71s/it]
                                                        

 78%|███████▊  | 9345/12000 [5:30:41<1:15:43,  1.71s/it]
 78%|███████▊  | 9346/12000 [5:30:44<1:30:16,  2.04s/it]
                                                        

 78%|███████▊  | 9346/12000 [5:30:44<1:30:16,  2.04s/it]
 78%|███████▊  | 9347/12000 [5:30:46<1:32:52,  2.10s/it]
                                                        

 78%|███████▊  | 9347/12000 [5:30:46<1:32:52,  2.10s/it]
 78%|███████▊  | 9348/12000 [5:30:47<1:23:09,  1.88s/it]
                                                        

 78%|███████▊  | 9348/12000 [5:30:47<1:23:09,  1.88s/it]
 78%|███████▊  | 9349/12000 [5:30:48<1:15:53,  1.72s/it]
                                                        

 78%|███████▊  | 9349/12000 [5:30:48<1:15:53,  1.72s/it]
 78%|███████▊  | 9350/12000 [5:30:50<1:14:36,  1.69s/it]
                                                        

 78%|███████▊  | 9350/12000 [5:30:50<1:14:36,  1.69s/it]
 78%|███████▊  | 9351/12000 [5:30:54<1:46:54,  2.42s/it]
                                                        

 78%|███████▊  | 9351/12000 [5:30:54<1:46:54,  2.42s/it]
 78%|███████▊  | 9352/12000 [5:30:56<1:34:35,  2.14s/it]
                                                        

 78%|███████▊  | 9352/12000 [5:30:56<1:34:35,  2.14s/it]
 78%|███████▊  | 9353/12000 [5:30:57<1:26:38,  1.96s/it]
                                                        

 78%|███████▊  | 9353/12000 [5:30:57<1:26:38,  1.96s/it]
 78%|███████▊  | 9354/12000 [5:31:00<1:35:50,  2.17s/it]
                                                        

 78%|███████▊  | 9354/12000 [5:31:00<1:35:50,  2.17s/it]
 78%|███████▊  | 9355/12000 [5:31:02<1:31:28,  2.08s/it]
                                                        

 78%|███████▊  | 9355/12000 [5:31:02<1:31:28,  2.08s/it]
 78%|███████▊  | 9356/12000 [5:31:04<1:34:34,  2.15s/it]
                                                        

 78%|███████▊  | 9356/12000 [5:31:04<1:34:34,  2.15s/it]
 78%|███████▊  | 9357/12000 [5:31:06<1:29:22,  2.03s/it]
                                                        
{'loss': '0.6884', 'grad_norm': '0.3688', 'learning_rate': '2.626e-05', 'entropy': '0.7354', 'num_tokens': '5.013e+07', 'mean_token_accuracy': '0.803', 'epoch': '0.08239'}
{'loss': '0.8897', 'grad_norm': '0.4161', 'learning_rate': '2.624e-05', 'entropy': '0.8984', 'num_tokens': '5.014e+07', 'mean_token_accuracy': '0.7793', 'epoch': '0.0824'}
{'loss': '0.7609', 'grad_norm': '0.4975', 'learning_rate': '2.622e-05', 'entropy': '0.7654', 'num_tokens': '5.014e+07', 'mean_token_accuracy': '0.8041', 'epoch': '0.08241'}
{'loss': '1.114', 'grad_norm': '0.4177', 'learning_rate': '2.62e-05', 'entropy': '0.9823', 'num_tokens': '5.015e+07', 'mean_token_accuracy': '0.7612', 'epoch': '0.08242'}
{'loss': '0.8626', 'grad_norm': '0.4582', 'learning_rate': '2.619e-05', 'entropy': '0.855', 'num_tokens': '5.015e+07', 'mean_token_accuracy': '0.8009', 'epoch': '0.08243'}
{'loss': '0.7759', 'grad_norm': '0.4614', 'learning_rate': '2.617e-05', 'entropy': '0.8629', 'num_tokens': '5.016e+07', 'mean_token_accuracy': '0.781', 'epoch': '0.08244'}
{'loss': '1.025', 'grad_norm': '0.3898', 'learning_rate': '2.615e-05', 'entropy': '1.034', 'num_tokens': '5.017e+07', 'mean_token_accuracy': '0.7377', 'epoch': '0.08245'}
{'loss': '0.9432', 'grad_norm': '0.4115', 'learning_rate': '2.613e-05', 'entropy': '0.8802', 'num_tokens': '5.017e+07', 'mean_token_accuracy': '0.7823', 'epoch': '0.08245'}
{'loss': '0.9307', 'grad_norm': '0.3229', 'learning_rate': '2.611e-05', 'entropy': '0.9882', 'num_tokens': '5.018e+07', 'mean_token_accuracy': '0.74', 'epoch': '0.08246'}
{'loss': '0.6907', 'grad_norm': '0.4084', 'learning_rate': '2.609e-05', 'entropy': '0.7205', 'num_tokens': '5.018e+07', 'mean_token_accuracy': '0.819', 'epoch': '0.08247'}
{'loss': '0.686', 'grad_norm': '0.4471', 'learning_rate': '2.607e-05', 'entropy': '0.6953', 'num_tokens': '5.019e+07', 'mean_token_accuracy': '0.8083', 'epoch': '0.08248'}
{'loss': '0.8619', 'grad_norm': '0.3489', 'learning_rate': '2.606e-05', 'entropy': '0.9178', 'num_tokens': '5.019e+07', 'mean_token_accuracy': '0.7785', 'epoch': '0.08249'}
{'loss': '1.187', 'grad_norm': '0.4279', 'learning_rate': '2.604e-05', 'entropy': '1.216', 'num_tokens': '5.02e+07', 'mean_token_accuracy': '0.7123', 'epoch': '0.0825'}
{'loss': '0.5593', 'grad_norm': '0.2806', 'learning_rate': '2.602e-05', 'entropy': '0.5329', 'num_tokens': '5.021e+07', 'mean_token_accuracy': '0.87', 'epoch': '0.08251'}
{'loss': '0.8637', 'grad_norm': '0.2991', 'learning_rate': '2.6e-05', 'entropy': '0.715', 'num_tokens': '5.022e+07', 'mean_token_accuracy': '0.8252', 'epoch': '0.08252'}
{'loss': '0.7857', 'grad_norm': '0.3666', 'learning_rate': '2.598e-05', 'entropy': '0.7198', 'num_tokens': '5.022e+07', 'mean_token_accuracy': '0.8131', 'epoch': '0.08252'}
{'loss': '0.6634', 'grad_norm': '0.3553', 'learning_rate': '2.596e-05', 'entropy': '0.7498', 'num_tokens': '5.023e+07', 'mean_token_accuracy': '0.8077', 'epoch': '0.08253'}
{'loss': '0.8415', 'grad_norm': '0.505', 'learning_rate': '2.594e-05', 'entropy': '0.979', 'num_tokens': '5.023e+07', 'mean_token_accuracy': '0.751', 'epoch': '0.08254'}
{'loss': '0.7046', 'grad_norm': '0.3404', 'learning_rate': '2.593e-05', 'entropy': '0.879', 'num_tokens': '5.024e+07', 'mean_token_accuracy': '0.7943', 'epoch': '0.08255'}
{'loss': '0.9062', 'grad_norm': '0.44', 'learning_rate': '2.591e-05', 'entropy': '0.8756', 'num_tokens': '5.024e+07', 'mean_token_accuracy': '0.7773', 'epoch': '0.08256'}
{'loss': '0.9372', 'grad_norm': '0.4446', 'learning_rate': '2.589e-05', 'entropy': '0.9257', 'num_tokens': '5.025e+07', 'mean_token_accuracy': '0.7727', 'epoch': '0.08257'}
{'loss': '1.104', 'grad_norm': '0.5222', 'learning_rate': '2.587e-05', 'entropy': '0.964', 'num_tokens': '5.026e+07', 'mean_token_accuracy': '0.7625', 'epoch': '0.08258'}
{'loss': '0.8248', 'grad_norm': '0.3938', 'learning_rate': '2.585e-05', 'entropy': '0.8732', 'num_tokens': '5.026e+07', 'mean_token_accuracy': '0.7815', 'epoch': '0.08259'}
{'loss': '0.6754', 'grad_norm': '0.4491', 'learning_rate': '2.583e-05', 'entropy': '0.6661', 'num_tokens': '5.027e+07', 'mean_token_accuracy': '0.8156', 'epoch': '0.0826'}
{'loss': '1.061', 'grad_norm': '0.5458', 'learning_rate': '2.581e-05', 'entropy': '0.9915', 'num_tokens': '5.027e+07', 'mean_token_accuracy': '0.7471', 'epoch': '0.0826'}
{'loss': '0.8571', 'grad_norm': '0.4263', 'learning_rate': '2.58e-05', 'entropy': '0.8242', 'num_tokens': '5.028e+07', 'mean_token_accuracy': '0.791', 'epoch': '0.08261'}
{'loss': '0.6563', 'grad_norm': '0.3116', 'learning_rate': '2.578e-05', 'entropy': '0.6843', 'num_tokens': '5.028e+07', 'mean_token_accuracy': '0.8162', 'epoch': '0.08262'}
{'loss': '0.8696', 'grad_norm': '0.3948', 'learning_rate': '2.576e-05', 'entropy': '0.8464', 'num_tokens': '5.029e+07', 'mean_token_accuracy': '0.7797', 'epoch': '0.08263'}
{'loss': '0.6633', 'grad_norm': '0.3542', 'learning_rate': '2.574e-05', 'entropy': '0.6598', 'num_tokens': '5.029e+07', 'mean_token_accuracy': '0.829', 'epoch': '0.08264'}
{'loss': '1.029', 'grad_norm': '0.3345', 'learning_rate': '2.572e-05', 'entropy': '1.089', 'num_tokens': '5.03e+07', 'mean_token_accuracy': '0.7477', 'epoch': '0.08265'}
{'loss': '0.7093', 'grad_norm': '0.4494', 'learning_rate': '2.57e-05', 'entropy': '0.682', 'num_tokens': '5.031e+07', 'mean_token_accuracy': '0.8326', 'epoch': '0.08266'}
{'loss': '0.8575', 'grad_norm': '0.3827', 'learning_rate': '2.569e-05', 'entropy': '0.8207', 'num_tokens': '5.031e+07', 'mean_token_accuracy': '0.7906', 'epoch': '0.08267'}
{'loss': '0.9575', 'grad_norm': '0.3736', 'learning_rate': '2.567e-05', 'entropy': '1.14', 'num_tokens': '5.032e+07', 'mean_token_accuracy': '0.7119', 'epoch': '0.08268'}
{'loss': '0.6017', 'grad_norm': '0.4278', 'learning_rate': '2.565e-05', 'entropy': '0.6434', 'num_tokens': '5.032e+07', 'mean_token_accuracy': '0.8328', 'epoch': '0.08268'}
{'loss': '0.6289', 'grad_norm': '0.3321', 'learning_rate': '2.563e-05', 'entropy': '0.7004', 'num_tokens': '5.032e+07', 'mean_token_accuracy': '0.8266', 'epoch': '0.08269'}
{'loss': '0.7759', 'grad_norm': '0.4007', 'learning_rate': '2.561e-05', 'entropy': '0.7796', 'num_tokens': '5.033e+07', 'mean_token_accuracy': '0.8047', 'epoch': '0.0827'}
{'loss': '0.7192', 'grad_norm': '0.3425', 'learning_rate': '2.559e-05', 'entropy': '0.7402', 'num_tokens': '5.034e+07', 'mean_token_accuracy': '0.8083', 'epoch': '0.08271'}
{'loss': '0.9331', 'grad_norm': '0.4488', 'learning_rate': '2.557e-05', 'entropy': '0.9585', 'num_tokens': '5.034e+07', 'mean_token_accuracy': '0.7599', 'epoch': '0.08272'}
{'loss': '0.8825', 'grad_norm': '0.4323', 'learning_rate': '2.556e-05', 'entropy': '0.8977', 'num_tokens': '5.035e+07', 'mean_token_accuracy': '0.7748', 'epoch': '0.08273'}
{'loss': '0.7268', 'grad_norm': '0.3978', 'learning_rate': '2.554e-05', 'entropy': '0.7709', 'num_tokens': '5.035e+07', 'mean_token_accuracy': '0.807', 'epoch': '0.08274'}
{'loss': '0.7111', 'grad_norm': '0.3308', 'learning_rate': '2.552e-05', 'entropy': '0.7746', 'num_tokens': '5.035e+07', 'mean_token_accuracy': '0.8087', 'epoch': '0.08275'}
{'loss': '0.6278', 'grad_norm': '0.3722', 'learning_rate': '2.55e-05', 'entropy': '0.7202', 'num_tokens': '5.036e+07', 'mean_token_accuracy': '0.8117', 'epoch': '0.08275'}
{'loss': '0.9828', 'grad_norm': '0.374', 'learning_rate': '2.548e-05', 'entropy': '0.964', 'num_tokens': '5.036e+07', 'mean_token_accuracy': '0.7579', 'epoch': '0.08276'}
{'loss': '0.7239', 'grad_norm': '0.4449', 'learning_rate': '2.546e-05', 'entropy': '0.7535', 'num_tokens': '5.037e+07', 'mean_token_accuracy': '0.7942', 'epoch': '0.08277'}
{'loss': '0.7914', 'grad_norm': '0.3888', 'learning_rate': '2.545e-05', 'entropy': '0.8438', 'num_tokens': '5.037e+07', 'mean_token_accuracy': '0.7837', 'epoch': '0.08278'}
{'loss': '0.602', 'grad_norm': '0.3704', 'learning_rate': '2.543e-05', 'entropy': '0.6163', 'num_tokens': '5.038e+07', 'mean_token_accuracy': '0.8447', 'epoch': '0.08279'}
{'loss': '0.7567', 'grad_norm': '0.3335', 'learning_rate': '2.541e-05', 'entropy': '0.8548', 'num_tokens': '5.039e+07', 'mean_token_accuracy': '0.7956', 'epoch': '0.0828'}

 78%|███████▊  | 9357/12000 [5:31:06<1:29:22,  2.03s/it]
 78%|███████▊  | 9358/12000 [5:31:08<1:27:23,  1.98s/it]
                                                        

 78%|███████▊  | 9358/12000 [5:31:08<1:27:23,  1.98s/it]
 78%|███████▊  | 9359/12000 [5:31:11<1:46:00,  2.41s/it]
                                                        

 78%|███████▊  | 9359/12000 [5:31:11<1:46:00,  2.41s/it]
 78%|███████▊  | 9360/12000 [5:31:13<1:34:16,  2.14s/it]
                                                        

 78%|███████▊  | 9360/12000 [5:31:13<1:34:16,  2.14s/it]
 78%|███████▊  | 9361/12000 [5:31:15<1:32:13,  2.10s/it]
                                                        

 78%|███████▊  | 9361/12000 [5:31:15<1:32:13,  2.10s/it]
 78%|███████▊  | 9362/12000 [5:31:18<1:43:06,  2.35s/it]
                                                        

 78%|███████▊  | 9362/12000 [5:31:18<1:43:06,  2.35s/it]
 78%|███████▊  | 9363/12000 [5:31:23<2:28:53,  3.39s/it]
                                                        

 78%|███████▊  | 9363/12000 [5:31:23<2:28:53,  3.39s/it]
 78%|███████▊  | 9364/12000 [5:31:25<2:06:49,  2.89s/it]
                                                        

 78%|███████▊  | 9364/12000 [5:31:25<2:06:49,  2.89s/it]
 78%|███████▊  | 9365/12000 [5:31:27<1:48:57,  2.48s/it]
                                                        

 78%|███████▊  | 9365/12000 [5:31:27<1:48:57,  2.48s/it]
 78%|███████▊  | 9366/12000 [5:31:28<1:40:34,  2.29s/it]
                                                        

 78%|███████▊  | 9366/12000 [5:31:28<1:40:34,  2.29s/it]
 78%|███████▊  | 9367/12000 [5:31:31<1:37:27,  2.22s/it]
                                                        

 78%|███████▊  | 9367/12000 [5:31:31<1:37:27,  2.22s/it]
 78%|███████▊  | 9368/12000 [5:31:34<1:47:33,  2.45s/it]
                                                        

 78%|███████▊  | 9368/12000 [5:31:34<1:47:33,  2.45s/it]
 78%|███████▊  | 9369/12000 [5:31:36<1:52:21,  2.56s/it]
                                                        

 78%|███████▊  | 9369/12000 [5:31:36<1:52:21,  2.56s/it]
 78%|███████▊  | 9370/12000 [5:31:38<1:45:28,  2.41s/it]
                                                        

 78%|███████▊  | 9370/12000 [5:31:38<1:45:28,  2.41s/it]
 78%|███████▊  | 9371/12000 [5:31:40<1:39:05,  2.26s/it]
                                                        

 78%|███████▊  | 9371/12000 [5:31:40<1:39:05,  2.26s/it]
 78%|███████▊  | 9372/12000 [5:31:42<1:27:25,  2.00s/it]
                                                        

 78%|███████▊  | 9372/12000 [5:31:42<1:27:25,  2.00s/it]
 78%|███████▊  | 9373/12000 [5:31:43<1:20:06,  1.83s/it]
                                                        

 78%|███████▊  | 9373/12000 [5:31:43<1:20:06,  1.83s/it]
 78%|███████▊  | 9374/12000 [5:31:46<1:31:03,  2.08s/it]
                                                        

 78%|███████▊  | 9374/12000 [5:31:46<1:31:03,  2.08s/it]
 78%|███████▊  | 9375/12000 [5:31:48<1:27:53,  2.01s/it]
                                                        

 78%|███████▊  | 9375/12000 [5:31:48<1:27:53,  2.01s/it]
 78%|███████▊  | 9376/12000 [5:31:49<1:24:55,  1.94s/it]
                                                        

 78%|███████▊  | 9376/12000 [5:31:49<1:24:55,  1.94s/it]
 78%|███████▊  | 9377/12000 [5:31:51<1:19:38,  1.82s/it]
                                                        

 78%|███████▊  | 9377/12000 [5:31:51<1:19:38,  1.82s/it]
 78%|███████▊  | 9378/12000 [5:31:53<1:23:54,  1.92s/it]
                                                        

 78%|███████▊  | 9378/12000 [5:31:53<1:23:54,  1.92s/it]
 78%|███████▊  | 9379/12000 [5:31:55<1:21:46,  1.87s/it]
                                                        

 78%|███████▊  | 9379/12000 [5:31:55<1:21:46,  1.87s/it]
 78%|███████▊  | 9380/12000 [5:31:57<1:19:39,  1.82s/it]
                                                        

 78%|███████▊  | 9380/12000 [5:31:57<1:19:39,  1.82s/it]
 78%|███████▊  | 9381/12000 [5:32:00<1:43:51,  2.38s/it]
                                                        

 78%|███████▊  | 9381/12000 [5:32:00<1:43:51,  2.38s/it]
 78%|███████▊  | 9382/12000 [5:32:02<1:38:35,  2.26s/it]
                                                        

 78%|███████▊  | 9382/12000 [5:32:02<1:38:35,  2.26s/it]
 78%|███████▊  | 9383/12000 [5:32:04<1:29:41,  2.06s/it]
                                                        

 78%|███████▊  | 9383/12000 [5:32:04<1:29:41,  2.06s/it]
 78%|███████▊  | 9384/12000 [5:32:06<1:25:52,  1.97s/it]
                                                        

 78%|███████▊  | 9384/12000 [5:32:06<1:25:52,  1.97s/it]
 78%|███████▊  | 9385/12000 [5:32:08<1:25:33,  1.96s/it]
                                                        

 78%|███████▊  | 9385/12000 [5:32:08<1:25:33,  1.96s/it]
 78%|███████▊  | 9386/12000 [5:32:09<1:18:57,  1.81s/it]
                                                        

 78%|███████▊  | 9386/12000 [5:32:09<1:18:57,  1.81s/it]
 78%|███████▊  | 9387/12000 [5:32:11<1:19:38,  1.83s/it]
                                                        

 78%|███████▊  | 9387/12000 [5:32:11<1:19:38,  1.83s/it]
 78%|███████▊  | 9388/12000 [5:32:13<1:23:45,  1.92s/it]
                                                        

 78%|███████▊  | 9388/12000 [5:32:13<1:23:45,  1.92s/it]
 78%|███████▊  | 9389/12000 [5:32:15<1:20:24,  1.85s/it]
                                                        

 78%|███████▊  | 9389/12000 [5:32:15<1:20:24,  1.85s/it]
 78%|███████▊  | 9390/12000 [5:32:16<1:14:14,  1.71s/it]
                                                        

 78%|███████▊  | 9390/12000 [5:32:16<1:14:14,  1.71s/it]
 78%|███████▊  | 9391/12000 [5:32:18<1:11:03,  1.63s/it]
                                                        

 78%|███████▊  | 9391/12000 [5:32:18<1:11:03,  1.63s/it]
 78%|███████▊  | 9392/12000 [5:32:20<1:21:02,  1.86s/it]
                                                        

 78%|███████▊  | 9392/12000 [5:32:20<1:21:02,  1.86s/it]
 78%|███████▊  | 9393/12000 [5:32:21<1:16:27,  1.76s/it]
                                                        

 78%|███████▊  | 9393/12000 [5:32:21<1:16:27,  1.76s/it]
 78%|███████▊  | 9394/12000 [5:32:23<1:14:44,  1.72s/it]
                                                        

 78%|███████▊  | 9394/12000 [5:32:23<1:14:44,  1.72s/it]
 78%|███████▊  | 9395/12000 [5:32:25<1:13:39,  1.70s/it]
                                                        

 78%|███████▊  | 9395/12000 [5:32:25<1:13:39,  1.70s/it]
 78%|███████▊  | 9396/12000 [5:32:26<1:11:00,  1.64s/it]
                                                        

 78%|███████▊  | 9396/12000 [5:32:26<1:11:00,  1.64s/it]
 78%|███████▊  | 9397/12000 [5:32:28<1:10:00,  1.61s/it]
                                                        

 78%|███████▊  | 9397/12000 [5:32:28<1:10:00,  1.61s/it]
 78%|███████▊  | 9398/12000 [5:32:29<1:11:05,  1.64s/it]
                                                        

 78%|███████▊  | 9398/12000 [5:32:29<1:11:05,  1.64s/it]
 78%|███████▊  | 9399/12000 [5:32:31<1:07:46,  1.56s/it]
                                                        

 78%|███████▊  | 9399/12000 [5:32:31<1:07:46,  1.56s/it]
 78%|███████▊  | 9400/12000 [5:32:33<1:13:45,  1.70s/it]
                                                        

 78%|███████▊  | 9400/12000 [5:32:33<1:13:45,  1.70s/it]
 78%|███████▊  | 9401/12000 [5:32:36<1:31:13,  2.11s/it]
                                                        

 78%|███████▊  | 9401/12000 [5:32:36<1:31:13,  2.11s/it]
 78%|███████▊  | 9402/12000 [5:32:38<1:34:58,  2.19s/it]
                                                        

 78%|███████▊  | 9402/12000 [5:32:38<1:34:58,  2.19s/it]
 78%|███████▊  | 9403/12000 [5:32:42<1:50:13,  2.55s/it]
                                                        

 78%|███████▊  | 9403/12000 [5:32:42<1:50:13,  2.55s/it]
 78%|███████▊  | 9404/12000 [5:32:46<2:10:31,  3.02s/it]
                                                        
{'loss': '0.5806', 'grad_norm': '0.4158', 'learning_rate': '2.539e-05', 'entropy': '0.6005', 'num_tokens': '5.039e+07', 'mean_token_accuracy': '0.8405', 'epoch': '0.08281'}
{'loss': '0.7581', 'grad_norm': '0.421', 'learning_rate': '2.537e-05', 'entropy': '0.7495', 'num_tokens': '5.04e+07', 'mean_token_accuracy': '0.8016', 'epoch': '0.08282'}
{'loss': '0.6048', 'grad_norm': '0.3479', 'learning_rate': '2.535e-05', 'entropy': '0.6352', 'num_tokens': '5.04e+07', 'mean_token_accuracy': '0.8335', 'epoch': '0.08283'}
{'loss': '0.7215', 'grad_norm': '0.4173', 'learning_rate': '2.534e-05', 'entropy': '0.7527', 'num_tokens': '5.041e+07', 'mean_token_accuracy': '0.8094', 'epoch': '0.08283'}
{'loss': '0.7674', 'grad_norm': '0.384', 'learning_rate': '2.532e-05', 'entropy': '0.7734', 'num_tokens': '5.041e+07', 'mean_token_accuracy': '0.8032', 'epoch': '0.08284'}
{'loss': '0.8808', 'grad_norm': '0.4698', 'learning_rate': '2.53e-05', 'entropy': '0.7958', 'num_tokens': '5.042e+07', 'mean_token_accuracy': '0.8018', 'epoch': '0.08285'}
{'loss': '1.149', 'grad_norm': '0.3245', 'learning_rate': '2.528e-05', 'entropy': '1.08', 'num_tokens': '5.043e+07', 'mean_token_accuracy': '0.7465', 'epoch': '0.08286'}
{'loss': '1.11', 'grad_norm': '0.4772', 'learning_rate': '2.526e-05', 'entropy': '1.095', 'num_tokens': '5.043e+07', 'mean_token_accuracy': '0.7405', 'epoch': '0.08287'}
{'loss': '0.6845', 'grad_norm': '0.4476', 'learning_rate': '2.524e-05', 'entropy': '0.7897', 'num_tokens': '5.044e+07', 'mean_token_accuracy': '0.8049', 'epoch': '0.08288'}
{'loss': '0.6759', 'grad_norm': '0.358', 'learning_rate': '2.523e-05', 'entropy': '0.7087', 'num_tokens': '5.044e+07', 'mean_token_accuracy': '0.8254', 'epoch': '0.08289'}
{'loss': '0.6808', 'grad_norm': '0.3404', 'learning_rate': '2.521e-05', 'entropy': '0.75', 'num_tokens': '5.045e+07', 'mean_token_accuracy': '0.8156', 'epoch': '0.0829'}
{'loss': '0.5738', 'grad_norm': '0.364', 'learning_rate': '2.519e-05', 'entropy': '0.617', 'num_tokens': '5.046e+07', 'mean_token_accuracy': '0.8248', 'epoch': '0.08291'}
{'loss': '0.8797', 'grad_norm': '0.3753', 'learning_rate': '2.517e-05', 'entropy': '0.8031', 'num_tokens': '5.047e+07', 'mean_token_accuracy': '0.7963', 'epoch': '0.08291'}
{'loss': '0.7448', 'grad_norm': '0.4638', 'learning_rate': '2.515e-05', 'entropy': '0.6925', 'num_tokens': '5.047e+07', 'mean_token_accuracy': '0.8248', 'epoch': '0.08292'}
{'loss': '0.627', 'grad_norm': '0.3034', 'learning_rate': '2.513e-05', 'entropy': '0.6246', 'num_tokens': '5.048e+07', 'mean_token_accuracy': '0.8405', 'epoch': '0.08293'}
{'loss': '0.7384', 'grad_norm': '0.4672', 'learning_rate': '2.512e-05', 'entropy': '0.7484', 'num_tokens': '5.048e+07', 'mean_token_accuracy': '0.8099', 'epoch': '0.08294'}
{'loss': '0.6847', 'grad_norm': '0.3277', 'learning_rate': '2.51e-05', 'entropy': '0.79', 'num_tokens': '5.049e+07', 'mean_token_accuracy': '0.8045', 'epoch': '0.08295'}
{'loss': '0.6937', 'grad_norm': '0.3526', 'learning_rate': '2.508e-05', 'entropy': '0.6806', 'num_tokens': '5.049e+07', 'mean_token_accuracy': '0.8195', 'epoch': '0.08296'}
{'loss': '0.6561', 'grad_norm': '0.3858', 'learning_rate': '2.506e-05', 'entropy': '0.647', 'num_tokens': '5.05e+07', 'mean_token_accuracy': '0.8227', 'epoch': '0.08297'}
{'loss': '0.7386', 'grad_norm': '0.4634', 'learning_rate': '2.504e-05', 'entropy': '0.7832', 'num_tokens': '5.051e+07', 'mean_token_accuracy': '0.7816', 'epoch': '0.08298'}
{'loss': '0.5434', 'grad_norm': '0.3709', 'learning_rate': '2.503e-05', 'entropy': '0.6007', 'num_tokens': '5.051e+07', 'mean_token_accuracy': '0.8354', 'epoch': '0.08298'}
{'loss': '1.176', 'grad_norm': '0.3949', 'learning_rate': '2.501e-05', 'entropy': '1.079', 'num_tokens': '5.052e+07', 'mean_token_accuracy': '0.7372', 'epoch': '0.08299'}
{'loss': '0.6203', 'grad_norm': '0.363', 'learning_rate': '2.499e-05', 'entropy': '0.6573', 'num_tokens': '5.052e+07', 'mean_token_accuracy': '0.8251', 'epoch': '0.083'}
{'loss': '0.7491', 'grad_norm': '0.4181', 'learning_rate': '2.497e-05', 'entropy': '0.7186', 'num_tokens': '5.053e+07', 'mean_token_accuracy': '0.8112', 'epoch': '0.08301'}
{'loss': '0.6132', 'grad_norm': '0.411', 'learning_rate': '2.495e-05', 'entropy': '0.6404', 'num_tokens': '5.053e+07', 'mean_token_accuracy': '0.8296', 'epoch': '0.08302'}
{'loss': '0.7992', 'grad_norm': '0.3483', 'learning_rate': '2.493e-05', 'entropy': '0.7894', 'num_tokens': '5.054e+07', 'mean_token_accuracy': '0.7921', 'epoch': '0.08303'}
{'loss': '0.829', 'grad_norm': '0.3811', 'learning_rate': '2.492e-05', 'entropy': '0.8126', 'num_tokens': '5.054e+07', 'mean_token_accuracy': '0.7906', 'epoch': '0.08304'}
{'loss': '0.7104', 'grad_norm': '0.383', 'learning_rate': '2.49e-05', 'entropy': '0.7031', 'num_tokens': '5.055e+07', 'mean_token_accuracy': '0.8235', 'epoch': '0.08305'}
{'loss': '0.836', 'grad_norm': '0.4821', 'learning_rate': '2.488e-05', 'entropy': '0.8589', 'num_tokens': '5.055e+07', 'mean_token_accuracy': '0.7987', 'epoch': '0.08306'}
{'loss': '1.067', 'grad_norm': '0.501', 'learning_rate': '2.486e-05', 'entropy': '1.143', 'num_tokens': '5.056e+07', 'mean_token_accuracy': '0.7385', 'epoch': '0.08306'}
{'loss': '0.7047', 'grad_norm': '0.3577', 'learning_rate': '2.484e-05', 'entropy': '0.7371', 'num_tokens': '5.056e+07', 'mean_token_accuracy': '0.8036', 'epoch': '0.08307'}
{'loss': '0.8808', 'grad_norm': '0.4427', 'learning_rate': '2.483e-05', 'entropy': '0.8768', 'num_tokens': '5.057e+07', 'mean_token_accuracy': '0.7694', 'epoch': '0.08308'}
{'loss': '0.6224', 'grad_norm': '0.4363', 'learning_rate': '2.481e-05', 'entropy': '0.6556', 'num_tokens': '5.057e+07', 'mean_token_accuracy': '0.8103', 'epoch': '0.08309'}
{'loss': '0.8491', 'grad_norm': '0.4622', 'learning_rate': '2.479e-05', 'entropy': '0.8512', 'num_tokens': '5.057e+07', 'mean_token_accuracy': '0.7931', 'epoch': '0.0831'}
{'loss': '0.7229', 'grad_norm': '0.4484', 'learning_rate': '2.477e-05', 'entropy': '0.7999', 'num_tokens': '5.058e+07', 'mean_token_accuracy': '0.8073', 'epoch': '0.08311'}
{'loss': '0.9648', 'grad_norm': '0.5351', 'learning_rate': '2.475e-05', 'entropy': '0.8359', 'num_tokens': '5.058e+07', 'mean_token_accuracy': '0.7819', 'epoch': '0.08312'}
{'loss': '0.5695', 'grad_norm': '0.3151', 'learning_rate': '2.473e-05', 'entropy': '0.6015', 'num_tokens': '5.059e+07', 'mean_token_accuracy': '0.8466', 'epoch': '0.08313'}
{'loss': '0.6757', 'grad_norm': '0.4326', 'learning_rate': '2.472e-05', 'entropy': '0.7515', 'num_tokens': '5.059e+07', 'mean_token_accuracy': '0.8098', 'epoch': '0.08314'}
{'loss': '0.7362', 'grad_norm': '0.5111', 'learning_rate': '2.47e-05', 'entropy': '0.7709', 'num_tokens': '5.06e+07', 'mean_token_accuracy': '0.8071', 'epoch': '0.08314'}
{'loss': '0.874', 'grad_norm': '0.3854', 'learning_rate': '2.468e-05', 'entropy': '0.8731', 'num_tokens': '5.06e+07', 'mean_token_accuracy': '0.7754', 'epoch': '0.08315'}
{'loss': '0.6352', 'grad_norm': '0.391', 'learning_rate': '2.466e-05', 'entropy': '0.6095', 'num_tokens': '5.061e+07', 'mean_token_accuracy': '0.8422', 'epoch': '0.08316'}
{'loss': '0.8222', 'grad_norm': '0.4184', 'learning_rate': '2.464e-05', 'entropy': '0.8093', 'num_tokens': '5.061e+07', 'mean_token_accuracy': '0.8087', 'epoch': '0.08317'}
{'loss': '0.9599', 'grad_norm': '0.4321', 'learning_rate': '2.463e-05', 'entropy': '0.9619', 'num_tokens': '5.062e+07', 'mean_token_accuracy': '0.7584', 'epoch': '0.08318'}
{'loss': '1.011', 'grad_norm': '0.4996', 'learning_rate': '2.461e-05', 'entropy': '0.9652', 'num_tokens': '5.062e+07', 'mean_token_accuracy': '0.7507', 'epoch': '0.08319'}
{'loss': '0.7624', 'grad_norm': '0.4801', 'learning_rate': '2.459e-05', 'entropy': '0.729', 'num_tokens': '5.063e+07', 'mean_token_accuracy': '0.8017', 'epoch': '0.0832'}
{'loss': '0.6955', 'grad_norm': '0.3431', 'learning_rate': '2.457e-05', 'entropy': '0.6375', 'num_tokens': '5.063e+07', 'mean_token_accuracy': '0.8479', 'epoch': '0.08321'}
{'loss': '0.8213', 'grad_norm': '0.3693', 'learning_rate': '2.455e-05', 'entropy': '0.9255', 'num_tokens': '5.064e+07', 'mean_token_accuracy': '0.7713', 'epoch': '0.08321'}

 78%|███████▊  | 9404/12000 [5:32:46<2:10:31,  3.02s/it]
 78%|███████▊  | 9405/12000 [5:32:47<1:52:35,  2.60s/it]
                                                        

 78%|███████▊  | 9405/12000 [5:32:47<1:52:35,  2.60s/it]
 78%|███████▊  | 9406/12000 [5:32:49<1:37:21,  2.25s/it]
                                                        

 78%|███████▊  | 9406/12000 [5:32:49<1:37:21,  2.25s/it]
 78%|███████▊  | 9407/12000 [5:32:51<1:37:13,  2.25s/it]
                                                        

 78%|███████▊  | 9407/12000 [5:32:51<1:37:13,  2.25s/it]
 78%|███████▊  | 9408/12000 [5:32:53<1:28:16,  2.04s/it]
                                                        

 78%|███████▊  | 9408/12000 [5:32:53<1:28:16,  2.04s/it]
 78%|███████▊  | 9409/12000 [5:32:55<1:29:54,  2.08s/it]
                                                        

 78%|███████▊  | 9409/12000 [5:32:55<1:29:54,  2.08s/it]
 78%|███████▊  | 9410/12000 [5:32:59<1:58:54,  2.75s/it]
                                                        

 78%|███████▊  | 9410/12000 [5:32:59<1:58:54,  2.75s/it]
 78%|███████▊  | 9411/12000 [5:33:01<1:41:36,  2.35s/it]
                                                        

 78%|███████▊  | 9411/12000 [5:33:01<1:41:36,  2.35s/it]
 78%|███████▊  | 9412/12000 [5:33:02<1:28:33,  2.05s/it]
                                                        

 78%|███████▊  | 9412/12000 [5:33:02<1:28:33,  2.05s/it]
 78%|███████▊  | 9413/12000 [5:33:04<1:26:11,  2.00s/it]
                                                        

 78%|███████▊  | 9413/12000 [5:33:04<1:26:11,  2.00s/it]
 78%|███████▊  | 9414/12000 [5:33:06<1:25:54,  1.99s/it]
                                                        

 78%|███████▊  | 9414/12000 [5:33:06<1:25:54,  1.99s/it]
 78%|███████▊  | 9415/12000 [5:33:07<1:21:07,  1.88s/it]
                                                        

 78%|███████▊  | 9415/12000 [5:33:07<1:21:07,  1.88s/it]
 78%|███████▊  | 9416/12000 [5:33:09<1:17:56,  1.81s/it]
                                                        

 78%|███████▊  | 9416/12000 [5:33:09<1:17:56,  1.81s/it]
 78%|███████▊  | 9417/12000 [5:33:11<1:25:11,  1.98s/it]
                                                        

 78%|███████▊  | 9417/12000 [5:33:11<1:25:11,  1.98s/it]
 78%|███████▊  | 9418/12000 [5:33:13<1:22:15,  1.91s/it]
                                                        

 78%|███████▊  | 9418/12000 [5:33:13<1:22:15,  1.91s/it]
 78%|███████▊  | 9419/12000 [5:33:15<1:19:09,  1.84s/it]
                                                        

 78%|███████▊  | 9419/12000 [5:33:15<1:19:09,  1.84s/it]
 78%|███████▊  | 9420/12000 [5:33:17<1:18:35,  1.83s/it]
                                                        

 78%|███████▊  | 9420/12000 [5:33:17<1:18:35,  1.83s/it]
 79%|███████▊  | 9421/12000 [5:33:18<1:12:41,  1.69s/it]
                                                        

 79%|███████▊  | 9421/12000 [5:33:18<1:12:41,  1.69s/it]
 79%|███████▊  | 9422/12000 [5:33:20<1:12:18,  1.68s/it]
                                                        

 79%|███████▊  | 9422/12000 [5:33:20<1:12:18,  1.68s/it]
 79%|███████▊  | 9423/12000 [5:33:21<1:10:22,  1.64s/it]
                                                        

 79%|███████▊  | 9423/12000 [5:33:21<1:10:22,  1.64s/it]
 79%|███████▊  | 9424/12000 [5:33:23<1:08:25,  1.59s/it]
                                                        

 79%|███████▊  | 9424/12000 [5:33:23<1:08:25,  1.59s/it]
 79%|███████▊  | 9425/12000 [5:33:26<1:36:02,  2.24s/it]
                                                        

 79%|███████▊  | 9425/12000 [5:33:26<1:36:02,  2.24s/it]
 79%|███████▊  | 9426/12000 [5:33:31<1:59:28,  2.78s/it]
                                                        

 79%|███████▊  | 9426/12000 [5:33:31<1:59:28,  2.78s/it]
 79%|███████▊  | 9427/12000 [5:33:33<1:51:14,  2.59s/it]
                                                        

 79%|███████▊  | 9427/12000 [5:33:33<1:51:14,  2.59s/it]
 79%|███████▊  | 9428/12000 [5:33:35<1:44:15,  2.43s/it]
                                                        

 79%|███████▊  | 9428/12000 [5:33:35<1:44:15,  2.43s/it]
 79%|███████▊  | 9429/12000 [5:33:36<1:31:40,  2.14s/it]
                                                        

 79%|███████▊  | 9429/12000 [5:33:36<1:31:40,  2.14s/it]
 79%|███████▊  | 9430/12000 [5:33:38<1:29:11,  2.08s/it]
                                                        

 79%|███████▊  | 9430/12000 [5:33:38<1:29:11,  2.08s/it]
 79%|███████▊  | 9431/12000 [5:33:40<1:23:13,  1.94s/it]
                                                        

 79%|███████▊  | 9431/12000 [5:33:40<1:23:13,  1.94s/it]
 79%|███████▊  | 9432/12000 [5:33:42<1:30:48,  2.12s/it]
                                                        

 79%|███████▊  | 9432/12000 [5:33:42<1:30:48,  2.12s/it]
 79%|███████▊  | 9433/12000 [5:33:46<1:48:59,  2.55s/it]
                                                        

 79%|███████▊  | 9433/12000 [5:33:46<1:48:59,  2.55s/it]
 79%|███████▊  | 9434/12000 [5:33:47<1:36:31,  2.26s/it]
                                                        

 79%|███████▊  | 9434/12000 [5:33:47<1:36:31,  2.26s/it]
 79%|███████▊  | 9435/12000 [5:33:49<1:27:31,  2.05s/it]
                                                        

 79%|███████▊  | 9435/12000 [5:33:49<1:27:31,  2.05s/it]
 79%|███████▊  | 9436/12000 [5:33:51<1:28:48,  2.08s/it]
                                                        

 79%|███████▊  | 9436/12000 [5:33:51<1:28:48,  2.08s/it]
 79%|███████▊  | 9437/12000 [5:33:53<1:23:33,  1.96s/it]
                                                        

 79%|███████▊  | 9437/12000 [5:33:53<1:23:33,  1.96s/it]
 79%|███████▊  | 9438/12000 [5:33:55<1:24:04,  1.97s/it]
                                                        

 79%|███████▊  | 9438/12000 [5:33:55<1:24:04,  1.97s/it]
 79%|███████▊  | 9439/12000 [5:33:57<1:31:04,  2.13s/it]
                                                        

 79%|███████▊  | 9439/12000 [5:33:57<1:31:04,  2.13s/it]
 79%|███████▊  | 9440/12000 [5:33:59<1:23:25,  1.96s/it]
                                                        

 79%|███████▊  | 9440/12000 [5:33:59<1:23:25,  1.96s/it]
 79%|███████▊  | 9441/12000 [5:34:02<1:38:14,  2.30s/it]
                                                        

 79%|███████▊  | 9441/12000 [5:34:02<1:38:14,  2.30s/it]
 79%|███████▊  | 9442/12000 [5:34:03<1:27:51,  2.06s/it]
                                                        

 79%|███████▊  | 9442/12000 [5:34:03<1:27:51,  2.06s/it]
 79%|███████▊  | 9443/12000 [5:34:07<1:43:25,  2.43s/it]
                                                        

 79%|███████▊  | 9443/12000 [5:34:07<1:43:25,  2.43s/it]
 79%|███████▊  | 9444/12000 [5:34:08<1:34:25,  2.22s/it]
                                                        

 79%|███████▊  | 9444/12000 [5:34:08<1:34:25,  2.22s/it]
 79%|███████▊  | 9445/12000 [5:34:10<1:26:44,  2.04s/it]
                                                        

 79%|███████▊  | 9445/12000 [5:34:10<1:26:44,  2.04s/it]
 79%|███████▊  | 9446/12000 [5:34:13<1:36:34,  2.27s/it]
                                                        

 79%|███████▊  | 9446/12000 [5:34:13<1:36:34,  2.27s/it]
 79%|███████▊  | 9447/12000 [5:34:16<1:50:11,  2.59s/it]
                                                        

 79%|███████▊  | 9447/12000 [5:34:16<1:50:11,  2.59s/it]
 79%|███████▊  | 9448/12000 [5:34:20<2:03:45,  2.91s/it]
                                                        

 79%|███████▊  | 9448/12000 [5:34:20<2:03:45,  2.91s/it]
 79%|███████▊  | 9449/12000 [5:34:21<1:46:46,  2.51s/it]
                                                        

 79%|███████▊  | 9449/12000 [5:34:21<1:46:46,  2.51s/it]
 79%|███████▉  | 9450/12000 [5:34:24<1:44:13,  2.45s/it]
                                                        

 79%|███████▉  | 9450/12000 [5:34:24<1:44:13,  2.45s/it]
 79%|███████▉  | 9451/12000 [5:34:27<1:53:43,  2.68s/it]
                                                        
{'loss': '0.8527', 'grad_norm': '0.296', 'learning_rate': '2.454e-05', 'entropy': '0.8794', 'num_tokens': '5.065e+07', 'mean_token_accuracy': '0.7727', 'epoch': '0.08322'}
{'loss': '0.6835', 'grad_norm': '0.3192', 'learning_rate': '2.452e-05', 'entropy': '0.6873', 'num_tokens': '5.065e+07', 'mean_token_accuracy': '0.8328', 'epoch': '0.08323'}
{'loss': '0.9008', 'grad_norm': '0.4407', 'learning_rate': '2.45e-05', 'entropy': '0.902', 'num_tokens': '5.066e+07', 'mean_token_accuracy': '0.7739', 'epoch': '0.08324'}
{'loss': '0.7313', 'grad_norm': '0.3554', 'learning_rate': '2.448e-05', 'entropy': '0.7201', 'num_tokens': '5.066e+07', 'mean_token_accuracy': '0.8027', 'epoch': '0.08325'}
{'loss': '0.826', 'grad_norm': '0.4615', 'learning_rate': '2.446e-05', 'entropy': '0.8239', 'num_tokens': '5.067e+07', 'mean_token_accuracy': '0.7903', 'epoch': '0.08326'}
{'loss': '0.9064', 'grad_norm': '0.3625', 'learning_rate': '2.444e-05', 'entropy': '0.9506', 'num_tokens': '5.067e+07', 'mean_token_accuracy': '0.7633', 'epoch': '0.08327'}
{'loss': '0.9461', 'grad_norm': '0.3336', 'learning_rate': '2.443e-05', 'entropy': '0.831', 'num_tokens': '5.068e+07', 'mean_token_accuracy': '0.8014', 'epoch': '0.08328'}
{'loss': '1.078', 'grad_norm': '0.4784', 'learning_rate': '2.441e-05', 'entropy': '1.062', 'num_tokens': '5.069e+07', 'mean_token_accuracy': '0.7419', 'epoch': '0.08329'}
{'loss': '0.916', 'grad_norm': '0.4565', 'learning_rate': '2.439e-05', 'entropy': '0.9471', 'num_tokens': '5.069e+07', 'mean_token_accuracy': '0.7583', 'epoch': '0.08329'}
{'loss': '0.7224', 'grad_norm': '0.3413', 'learning_rate': '2.437e-05', 'entropy': '0.8002', 'num_tokens': '5.069e+07', 'mean_token_accuracy': '0.8051', 'epoch': '0.0833'}
{'loss': '0.7745', 'grad_norm': '0.4707', 'learning_rate': '2.435e-05', 'entropy': '0.697', 'num_tokens': '5.07e+07', 'mean_token_accuracy': '0.8271', 'epoch': '0.08331'}
{'loss': '0.7657', 'grad_norm': '0.3929', 'learning_rate': '2.434e-05', 'entropy': '0.7297', 'num_tokens': '5.07e+07', 'mean_token_accuracy': '0.8131', 'epoch': '0.08332'}
{'loss': '0.7819', 'grad_norm': '0.3648', 'learning_rate': '2.432e-05', 'entropy': '0.7275', 'num_tokens': '5.071e+07', 'mean_token_accuracy': '0.8129', 'epoch': '0.08333'}
{'loss': '0.7269', 'grad_norm': '0.373', 'learning_rate': '2.43e-05', 'entropy': '0.7123', 'num_tokens': '5.072e+07', 'mean_token_accuracy': '0.8184', 'epoch': '0.08334'}
{'loss': '0.8752', 'grad_norm': '0.3955', 'learning_rate': '2.428e-05', 'entropy': '0.9397', 'num_tokens': '5.072e+07', 'mean_token_accuracy': '0.7593', 'epoch': '0.08335'}
{'loss': '0.8497', 'grad_norm': '0.4593', 'learning_rate': '2.426e-05', 'entropy': '0.8675', 'num_tokens': '5.073e+07', 'mean_token_accuracy': '0.7861', 'epoch': '0.08336'}
{'loss': '0.7073', 'grad_norm': '0.4631', 'learning_rate': '2.425e-05', 'entropy': '0.6725', 'num_tokens': '5.073e+07', 'mean_token_accuracy': '0.8261', 'epoch': '0.08337'}
{'loss': '0.7852', 'grad_norm': '0.3679', 'learning_rate': '2.423e-05', 'entropy': '0.9244', 'num_tokens': '5.073e+07', 'mean_token_accuracy': '0.7692', 'epoch': '0.08337'}
{'loss': '1.169', 'grad_norm': '0.5219', 'learning_rate': '2.421e-05', 'entropy': '1.15', 'num_tokens': '5.074e+07', 'mean_token_accuracy': '0.7267', 'epoch': '0.08338'}
{'loss': '0.6742', 'grad_norm': '0.3619', 'learning_rate': '2.419e-05', 'entropy': '0.6644', 'num_tokens': '5.074e+07', 'mean_token_accuracy': '0.826', 'epoch': '0.08339'}
{'loss': '0.6754', 'grad_norm': '0.4499', 'learning_rate': '2.417e-05', 'entropy': '0.7229', 'num_tokens': '5.075e+07', 'mean_token_accuracy': '0.81', 'epoch': '0.0834'}
{'loss': '0.8711', 'grad_norm': '0.4243', 'learning_rate': '2.416e-05', 'entropy': '0.9279', 'num_tokens': '5.075e+07', 'mean_token_accuracy': '0.7688', 'epoch': '0.08341'}
{'loss': '0.5368', 'grad_norm': '0.3327', 'learning_rate': '2.414e-05', 'entropy': '0.6641', 'num_tokens': '5.076e+07', 'mean_token_accuracy': '0.8301', 'epoch': '0.08342'}
{'loss': '0.8271', 'grad_norm': '0.4422', 'learning_rate': '2.412e-05', 'entropy': '0.8658', 'num_tokens': '5.077e+07', 'mean_token_accuracy': '0.7856', 'epoch': '0.08343'}
{'loss': '0.858', 'grad_norm': '0.4227', 'learning_rate': '2.41e-05', 'entropy': '0.9506', 'num_tokens': '5.077e+07', 'mean_token_accuracy': '0.7766', 'epoch': '0.08344'}
{'loss': '0.7639', 'grad_norm': '0.4661', 'learning_rate': '2.408e-05', 'entropy': '0.8659', 'num_tokens': '5.078e+07', 'mean_token_accuracy': '0.7847', 'epoch': '0.08345'}
{'loss': '0.8443', 'grad_norm': '0.3787', 'learning_rate': '2.407e-05', 'entropy': '0.8016', 'num_tokens': '5.078e+07', 'mean_token_accuracy': '0.7987', 'epoch': '0.08345'}
{'loss': '0.7324', 'grad_norm': '0.4858', 'learning_rate': '2.405e-05', 'entropy': '0.7733', 'num_tokens': '5.079e+07', 'mean_token_accuracy': '0.7963', 'epoch': '0.08346'}
{'loss': '0.5141', 'grad_norm': '0.3271', 'learning_rate': '2.403e-05', 'entropy': '0.6707', 'num_tokens': '5.079e+07', 'mean_token_accuracy': '0.8279', 'epoch': '0.08347'}
{'loss': '0.9276', 'grad_norm': '0.4329', 'learning_rate': '2.401e-05', 'entropy': '0.9002', 'num_tokens': '5.08e+07', 'mean_token_accuracy': '0.7659', 'epoch': '0.08348'}
{'loss': '0.8719', 'grad_norm': '0.3497', 'learning_rate': '2.4e-05', 'entropy': '0.9454', 'num_tokens': '5.081e+07', 'mean_token_accuracy': '0.7697', 'epoch': '0.08349'}
{'loss': '0.5793', 'grad_norm': '0.3462', 'learning_rate': '2.398e-05', 'entropy': '0.6091', 'num_tokens': '5.081e+07', 'mean_token_accuracy': '0.8486', 'epoch': '0.0835'}
{'loss': '0.6898', 'grad_norm': '0.3167', 'learning_rate': '2.396e-05', 'entropy': '0.727', 'num_tokens': '5.082e+07', 'mean_token_accuracy': '0.8235', 'epoch': '0.08351'}
{'loss': '0.5233', 'grad_norm': '0.4163', 'learning_rate': '2.394e-05', 'entropy': '0.5239', 'num_tokens': '5.082e+07', 'mean_token_accuracy': '0.8576', 'epoch': '0.08352'}
{'loss': '0.8509', 'grad_norm': '0.4279', 'learning_rate': '2.392e-05', 'entropy': '0.8506', 'num_tokens': '5.083e+07', 'mean_token_accuracy': '0.7973', 'epoch': '0.08352'}
{'loss': '0.8856', 'grad_norm': '0.4024', 'learning_rate': '2.391e-05', 'entropy': '0.8146', 'num_tokens': '5.084e+07', 'mean_token_accuracy': '0.7989', 'epoch': '0.08353'}
{'loss': '0.6388', 'grad_norm': '0.3992', 'learning_rate': '2.389e-05', 'entropy': '0.6589', 'num_tokens': '5.084e+07', 'mean_token_accuracy': '0.837', 'epoch': '0.08354'}
{'loss': '0.7345', 'grad_norm': '0.2799', 'learning_rate': '2.387e-05', 'entropy': '0.8527', 'num_tokens': '5.085e+07', 'mean_token_accuracy': '0.7801', 'epoch': '0.08355'}
{'loss': '0.5583', 'grad_norm': '0.4095', 'learning_rate': '2.385e-05', 'entropy': '0.5681', 'num_tokens': '5.085e+07', 'mean_token_accuracy': '0.8518', 'epoch': '0.08356'}
{'loss': '0.4798', 'grad_norm': '0.2795', 'learning_rate': '2.383e-05', 'entropy': '0.5157', 'num_tokens': '5.086e+07', 'mean_token_accuracy': '0.8592', 'epoch': '0.08357'}
{'loss': '0.716', 'grad_norm': '0.4628', 'learning_rate': '2.382e-05', 'entropy': '0.8223', 'num_tokens': '5.086e+07', 'mean_token_accuracy': '0.7923', 'epoch': '0.08358'}
{'loss': '0.8789', 'grad_norm': '0.4062', 'learning_rate': '2.38e-05', 'entropy': '0.8654', 'num_tokens': '5.087e+07', 'mean_token_accuracy': '0.7846', 'epoch': '0.08359'}
{'loss': '0.6529', 'grad_norm': '0.3755', 'learning_rate': '2.378e-05', 'entropy': '0.6742', 'num_tokens': '5.087e+07', 'mean_token_accuracy': '0.8288', 'epoch': '0.0836'}
{'loss': '0.8712', 'grad_norm': '0.4465', 'learning_rate': '2.376e-05', 'entropy': '0.9114', 'num_tokens': '5.088e+07', 'mean_token_accuracy': '0.7766', 'epoch': '0.0836'}
{'loss': '0.9408', 'grad_norm': '0.3372', 'learning_rate': '2.375e-05', 'entropy': '0.8346', 'num_tokens': '5.089e+07', 'mean_token_accuracy': '0.7847', 'epoch': '0.08361'}
{'loss': '0.6109', 'grad_norm': '0.3041', 'learning_rate': '2.373e-05', 'entropy': '0.5997', 'num_tokens': '5.089e+07', 'mean_token_accuracy': '0.8367', 'epoch': '0.08362'}
{'loss': '0.7312', 'grad_norm': '0.4215', 'learning_rate': '2.371e-05', 'entropy': '0.7955', 'num_tokens': '5.09e+07', 'mean_token_accuracy': '0.7965', 'epoch': '0.08363'}

 79%|███████▉  | 9451/12000 [5:34:27<1:53:43,  2.68s/it]
 79%|███████▉  | 9452/12000 [5:34:29<1:39:59,  2.35s/it]
                                                        

 79%|███████▉  | 9452/12000 [5:34:29<1:39:59,  2.35s/it]
 79%|███████▉  | 9453/12000 [5:34:30<1:31:34,  2.16s/it]
                                                        

 79%|███████▉  | 9453/12000 [5:34:30<1:31:34,  2.16s/it]
 79%|███████▉  | 9454/12000 [5:34:32<1:29:57,  2.12s/it]
                                                        

 79%|███████▉  | 9454/12000 [5:34:32<1:29:57,  2.12s/it]
 79%|███████▉  | 9455/12000 [5:34:34<1:30:10,  2.13s/it]
                                                        

 79%|███████▉  | 9455/12000 [5:34:34<1:30:10,  2.13s/it]
 79%|███████▉  | 9456/12000 [5:34:36<1:27:47,  2.07s/it]
                                                        

 79%|███████▉  | 9456/12000 [5:34:36<1:27:47,  2.07s/it]
 79%|███████▉  | 9457/12000 [5:34:39<1:28:51,  2.10s/it]
                                                        

 79%|███████▉  | 9457/12000 [5:34:39<1:28:51,  2.10s/it]
 79%|███████▉  | 9458/12000 [5:34:42<1:42:39,  2.42s/it]
                                                        

 79%|███████▉  | 9458/12000 [5:34:42<1:42:39,  2.42s/it]
 79%|███████▉  | 9459/12000 [5:34:44<1:34:22,  2.23s/it]
                                                        

 79%|███████▉  | 9459/12000 [5:34:44<1:34:22,  2.23s/it]
 79%|███████▉  | 9460/12000 [5:34:45<1:27:43,  2.07s/it]
                                                        

 79%|███████▉  | 9460/12000 [5:34:45<1:27:43,  2.07s/it]
 79%|███████▉  | 9461/12000 [5:34:48<1:39:28,  2.35s/it]
                                                        

 79%|███████▉  | 9461/12000 [5:34:48<1:39:28,  2.35s/it]
 79%|███████▉  | 9462/12000 [5:34:50<1:31:33,  2.16s/it]
                                                        

 79%|███████▉  | 9462/12000 [5:34:50<1:31:33,  2.16s/it]
 79%|███████▉  | 9463/12000 [5:34:52<1:27:14,  2.06s/it]
                                                        

 79%|███████▉  | 9463/12000 [5:34:52<1:27:14,  2.06s/it]
 79%|███████▉  | 9464/12000 [5:34:55<1:35:34,  2.26s/it]
                                                        

 79%|███████▉  | 9464/12000 [5:34:55<1:35:34,  2.26s/it]
 79%|███████▉  | 9465/12000 [5:34:57<1:37:04,  2.30s/it]
                                                        

 79%|███████▉  | 9465/12000 [5:34:57<1:37:04,  2.30s/it]
 79%|███████▉  | 9466/12000 [5:34:59<1:31:39,  2.17s/it]
                                                        

 79%|███████▉  | 9466/12000 [5:34:59<1:31:39,  2.17s/it]
 79%|███████▉  | 9467/12000 [5:35:01<1:29:33,  2.12s/it]
                                                        

 79%|███████▉  | 9467/12000 [5:35:01<1:29:33,  2.12s/it]
 79%|███████▉  | 9468/12000 [5:35:02<1:22:51,  1.96s/it]
                                                        

 79%|███████▉  | 9468/12000 [5:35:02<1:22:51,  1.96s/it]
 79%|███████▉  | 9469/12000 [5:35:06<1:41:12,  2.40s/it]
                                                        

 79%|███████▉  | 9469/12000 [5:35:06<1:41:12,  2.40s/it]
 79%|███████▉  | 9470/12000 [5:35:07<1:28:39,  2.10s/it]
                                                        

 79%|███████▉  | 9470/12000 [5:35:07<1:28:39,  2.10s/it]
 79%|███████▉  | 9471/12000 [5:35:09<1:25:20,  2.02s/it]
                                                        

 79%|███████▉  | 9471/12000 [5:35:09<1:25:20,  2.02s/it]
 79%|███████▉  | 9472/12000 [5:35:11<1:21:08,  1.93s/it]
                                                        

 79%|███████▉  | 9472/12000 [5:35:11<1:21:08,  1.93s/it]
 79%|███████▉  | 9473/12000 [5:35:13<1:23:55,  1.99s/it]
                                                        

 79%|███████▉  | 9473/12000 [5:35:13<1:23:55,  1.99s/it]
 79%|███████▉  | 9474/12000 [5:35:14<1:16:47,  1.82s/it]
                                                        

 79%|███████▉  | 9474/12000 [5:35:14<1:16:47,  1.82s/it]
 79%|███████▉  | 9475/12000 [5:35:16<1:13:03,  1.74s/it]
                                                        

 79%|███████▉  | 9475/12000 [5:35:16<1:13:03,  1.74s/it]
 79%|███████▉  | 9476/12000 [5:35:17<1:10:57,  1.69s/it]
                                                        

 79%|███████▉  | 9476/12000 [5:35:17<1:10:57,  1.69s/it]
 79%|███████▉  | 9477/12000 [5:35:20<1:26:34,  2.06s/it]
                                                        

 79%|███████▉  | 9477/12000 [5:35:20<1:26:34,  2.06s/it]
 79%|███████▉  | 9478/12000 [5:35:22<1:18:30,  1.87s/it]
                                                        

 79%|███████▉  | 9478/12000 [5:35:22<1:18:30,  1.87s/it]
 79%|███████▉  | 9479/12000 [5:35:23<1:15:16,  1.79s/it]
                                                        

 79%|███████▉  | 9479/12000 [5:35:23<1:15:16,  1.79s/it]
 79%|███████▉  | 9480/12000 [5:35:25<1:15:28,  1.80s/it]
                                                        

 79%|███████▉  | 9480/12000 [5:35:25<1:15:28,  1.80s/it]
 79%|███████▉  | 9481/12000 [5:35:27<1:09:14,  1.65s/it]
                                                        

 79%|███████▉  | 9481/12000 [5:35:27<1:09:14,  1.65s/it]
 79%|███████▉  | 9482/12000 [5:35:28<1:08:01,  1.62s/it]
                                                        

 79%|███████▉  | 9482/12000 [5:35:28<1:08:01,  1.62s/it]
 79%|███████▉  | 9483/12000 [5:35:31<1:18:40,  1.88s/it]
                                                        

 79%|███████▉  | 9483/12000 [5:35:31<1:18:40,  1.88s/it]
 79%|███████▉  | 9484/12000 [5:35:33<1:26:25,  2.06s/it]
                                                        

 79%|███████▉  | 9484/12000 [5:35:33<1:26:25,  2.06s/it]
 79%|███████▉  | 9485/12000 [5:35:35<1:19:30,  1.90s/it]
                                                        

 79%|███████▉  | 9485/12000 [5:35:35<1:19:30,  1.90s/it]
 79%|███████▉  | 9486/12000 [5:35:36<1:15:17,  1.80s/it]
                                                        

 79%|███████▉  | 9486/12000 [5:35:36<1:15:17,  1.80s/it]
 79%|███████▉  | 9487/12000 [5:35:40<1:40:38,  2.40s/it]
                                                        

 79%|███████▉  | 9487/12000 [5:35:40<1:40:38,  2.40s/it]
 79%|███████▉  | 9488/12000 [5:35:42<1:40:48,  2.41s/it]
                                                        

 79%|███████▉  | 9488/12000 [5:35:42<1:40:48,  2.41s/it]
 79%|███████▉  | 9489/12000 [5:35:44<1:35:26,  2.28s/it]
                                                        

 79%|███████▉  | 9489/12000 [5:35:44<1:35:26,  2.28s/it]
 79%|███████▉  | 9490/12000 [5:35:46<1:27:02,  2.08s/it]
                                                        

 79%|███████▉  | 9490/12000 [5:35:46<1:27:02,  2.08s/it]
 79%|███████▉  | 9491/12000 [5:35:53<2:23:23,  3.43s/it]
                                                        

 79%|███████▉  | 9491/12000 [5:35:53<2:23:23,  3.43s/it]
 79%|███████▉  | 9492/12000 [5:35:54<2:01:44,  2.91s/it]
                                                        

 79%|███████▉  | 9492/12000 [5:35:54<2:01:44,  2.91s/it]
 79%|███████▉  | 9493/12000 [5:35:57<2:04:11,  2.97s/it]
                                                        

 79%|███████▉  | 9493/12000 [5:35:57<2:04:11,  2.97s/it]
 79%|███████▉  | 9494/12000 [5:36:01<2:14:45,  3.23s/it]
                                                        

 79%|███████▉  | 9494/12000 [5:36:01<2:14:45,  3.23s/it]
 79%|███████▉  | 9495/12000 [5:36:03<1:55:13,  2.76s/it]
                                                        

 79%|███████▉  | 9495/12000 [5:36:03<1:55:13,  2.76s/it]
 79%|███████▉  | 9496/12000 [5:36:04<1:40:14,  2.40s/it]
                                                        

 79%|███████▉  | 9496/12000 [5:36:04<1:40:14,  2.40s/it]
 79%|███████▉  | 9497/12000 [5:36:06<1:29:51,  2.15s/it]
                                                        

 79%|███████▉  | 9497/12000 [5:36:06<1:29:51,  2.15s/it]
 79%|███████▉  | 9498/12000 [5:36:08<1:29:54,  2.16s/it]
                                                        
{'loss': '0.8242', 'grad_norm': '0.4735', 'learning_rate': '2.369e-05', 'entropy': '0.8176', 'num_tokens': '5.09e+07', 'mean_token_accuracy': '0.8033', 'epoch': '0.08364'}
{'loss': '0.9908', 'grad_norm': '0.3878', 'learning_rate': '2.367e-05', 'entropy': '1.005', 'num_tokens': '5.091e+07', 'mean_token_accuracy': '0.7452', 'epoch': '0.08365'}
{'loss': '1.01', 'grad_norm': '0.449', 'learning_rate': '2.366e-05', 'entropy': '0.9697', 'num_tokens': '5.091e+07', 'mean_token_accuracy': '0.7549', 'epoch': '0.08366'}
{'loss': '0.6566', 'grad_norm': '0.4468', 'learning_rate': '2.364e-05', 'entropy': '0.7054', 'num_tokens': '5.092e+07', 'mean_token_accuracy': '0.8145', 'epoch': '0.08367'}
{'loss': '0.7912', 'grad_norm': '0.3677', 'learning_rate': '2.362e-05', 'entropy': '0.8719', 'num_tokens': '5.092e+07', 'mean_token_accuracy': '0.7857', 'epoch': '0.08368'}
{'loss': '0.9201', 'grad_norm': '0.4031', 'learning_rate': '2.36e-05', 'entropy': '0.8589', 'num_tokens': '5.093e+07', 'mean_token_accuracy': '0.7695', 'epoch': '0.08368'}
{'loss': '0.9566', 'grad_norm': '0.5061', 'learning_rate': '2.358e-05', 'entropy': '1.008', 'num_tokens': '5.094e+07', 'mean_token_accuracy': '0.7533', 'epoch': '0.08369'}
{'loss': '0.8664', 'grad_norm': '0.3948', 'learning_rate': '2.357e-05', 'entropy': '0.995', 'num_tokens': '5.094e+07', 'mean_token_accuracy': '0.7623', 'epoch': '0.0837'}
{'loss': '0.8137', 'grad_norm': '0.5111', 'learning_rate': '2.355e-05', 'entropy': '0.852', 'num_tokens': '5.095e+07', 'mean_token_accuracy': '0.7874', 'epoch': '0.08371'}
{'loss': '0.7106', 'grad_norm': '0.4657', 'learning_rate': '2.353e-05', 'entropy': '0.6667', 'num_tokens': '5.095e+07', 'mean_token_accuracy': '0.8342', 'epoch': '0.08372'}
{'loss': '1.354', 'grad_norm': '0.4728', 'learning_rate': '2.351e-05', 'entropy': '1.397', 'num_tokens': '5.096e+07', 'mean_token_accuracy': '0.6948', 'epoch': '0.08373'}
{'loss': '0.9249', 'grad_norm': '0.3703', 'learning_rate': '2.35e-05', 'entropy': '0.892', 'num_tokens': '5.097e+07', 'mean_token_accuracy': '0.7832', 'epoch': '0.08374'}
{'loss': '0.8312', 'grad_norm': '0.4636', 'learning_rate': '2.348e-05', 'entropy': '0.8648', 'num_tokens': '5.097e+07', 'mean_token_accuracy': '0.7863', 'epoch': '0.08375'}
{'loss': '0.79', 'grad_norm': '0.3103', 'learning_rate': '2.346e-05', 'entropy': '0.7189', 'num_tokens': '5.098e+07', 'mean_token_accuracy': '0.8207', 'epoch': '0.08375'}
{'loss': '1.217', 'grad_norm': '0.4029', 'learning_rate': '2.344e-05', 'entropy': '1.145', 'num_tokens': '5.098e+07', 'mean_token_accuracy': '0.7394', 'epoch': '0.08376'}
{'loss': '0.771', 'grad_norm': '0.4213', 'learning_rate': '2.343e-05', 'entropy': '0.8228', 'num_tokens': '5.099e+07', 'mean_token_accuracy': '0.7854', 'epoch': '0.08377'}
{'loss': '0.7685', 'grad_norm': '0.3795', 'learning_rate': '2.341e-05', 'entropy': '0.6738', 'num_tokens': '5.1e+07', 'mean_token_accuracy': '0.8153', 'epoch': '0.08378'}
{'loss': '0.8303', 'grad_norm': '0.3868', 'learning_rate': '2.339e-05', 'entropy': '0.8054', 'num_tokens': '5.1e+07', 'mean_token_accuracy': '0.8003', 'epoch': '0.08379'}
{'loss': '0.769', 'grad_norm': '0.4198', 'learning_rate': '2.337e-05', 'entropy': '0.8153', 'num_tokens': '5.101e+07', 'mean_token_accuracy': '0.7935', 'epoch': '0.0838'}
{'loss': '0.4853', 'grad_norm': '0.368', 'learning_rate': '2.335e-05', 'entropy': '0.487', 'num_tokens': '5.101e+07', 'mean_token_accuracy': '0.8746', 'epoch': '0.08381'}
{'loss': '0.9574', 'grad_norm': '0.4168', 'learning_rate': '2.334e-05', 'entropy': '0.9149', 'num_tokens': '5.102e+07', 'mean_token_accuracy': '0.7741', 'epoch': '0.08382'}
{'loss': '0.7454', 'grad_norm': '0.4724', 'learning_rate': '2.332e-05', 'entropy': '0.7763', 'num_tokens': '5.102e+07', 'mean_token_accuracy': '0.8023', 'epoch': '0.08383'}
{'loss': '0.7596', 'grad_norm': '0.4043', 'learning_rate': '2.33e-05', 'entropy': '0.7206', 'num_tokens': '5.103e+07', 'mean_token_accuracy': '0.8017', 'epoch': '0.08383'}
{'loss': '0.8832', 'grad_norm': '0.4374', 'learning_rate': '2.328e-05', 'entropy': '0.9132', 'num_tokens': '5.103e+07', 'mean_token_accuracy': '0.7739', 'epoch': '0.08384'}
{'loss': '0.6959', 'grad_norm': '0.4077', 'learning_rate': '2.327e-05', 'entropy': '0.7278', 'num_tokens': '5.103e+07', 'mean_token_accuracy': '0.8064', 'epoch': '0.08385'}
{'loss': '0.9222', 'grad_norm': '0.3889', 'learning_rate': '2.325e-05', 'entropy': '0.956', 'num_tokens': '5.104e+07', 'mean_token_accuracy': '0.7738', 'epoch': '0.08386'}
{'loss': '0.6165', 'grad_norm': '0.3774', 'learning_rate': '2.323e-05', 'entropy': '0.6187', 'num_tokens': '5.105e+07', 'mean_token_accuracy': '0.84', 'epoch': '0.08387'}
{'loss': '0.7486', 'grad_norm': '0.3929', 'learning_rate': '2.321e-05', 'entropy': '0.8138', 'num_tokens': '5.105e+07', 'mean_token_accuracy': '0.7806', 'epoch': '0.08388'}
{'loss': '0.9517', 'grad_norm': '0.4302', 'learning_rate': '2.32e-05', 'entropy': '0.9024', 'num_tokens': '5.106e+07', 'mean_token_accuracy': '0.7591', 'epoch': '0.08389'}
{'loss': '0.9585', 'grad_norm': '0.5119', 'learning_rate': '2.318e-05', 'entropy': '0.9034', 'num_tokens': '5.106e+07', 'mean_token_accuracy': '0.7675', 'epoch': '0.0839'}
{'loss': '0.9008', 'grad_norm': '0.438', 'learning_rate': '2.316e-05', 'entropy': '0.9437', 'num_tokens': '5.107e+07', 'mean_token_accuracy': '0.7613', 'epoch': '0.08391'}
{'loss': '0.9665', 'grad_norm': '0.4015', 'learning_rate': '2.314e-05', 'entropy': '0.9348', 'num_tokens': '5.107e+07', 'mean_token_accuracy': '0.7834', 'epoch': '0.08391'}
{'loss': '0.8087', 'grad_norm': '0.3879', 'learning_rate': '2.312e-05', 'entropy': '0.7871', 'num_tokens': '5.108e+07', 'mean_token_accuracy': '0.7894', 'epoch': '0.08392'}
{'loss': '1.048', 'grad_norm': '0.4209', 'learning_rate': '2.311e-05', 'entropy': '0.9973', 'num_tokens': '5.108e+07', 'mean_token_accuracy': '0.7562', 'epoch': '0.08393'}
{'loss': '0.5465', 'grad_norm': '0.3127', 'learning_rate': '2.309e-05', 'entropy': '0.5589', 'num_tokens': '5.109e+07', 'mean_token_accuracy': '0.8514', 'epoch': '0.08394'}
{'loss': '0.8311', 'grad_norm': '0.432', 'learning_rate': '2.307e-05', 'entropy': '0.7908', 'num_tokens': '5.109e+07', 'mean_token_accuracy': '0.7957', 'epoch': '0.08395'}
{'loss': '1.029', 'grad_norm': '0.3361', 'learning_rate': '2.305e-05', 'entropy': '0.9136', 'num_tokens': '5.11e+07', 'mean_token_accuracy': '0.7871', 'epoch': '0.08396'}
{'loss': '0.802', 'grad_norm': '0.3942', 'learning_rate': '2.304e-05', 'entropy': '0.7553', 'num_tokens': '5.111e+07', 'mean_token_accuracy': '0.8061', 'epoch': '0.08397'}
{'loss': '0.9642', 'grad_norm': '0.3838', 'learning_rate': '2.302e-05', 'entropy': '0.9519', 'num_tokens': '5.111e+07', 'mean_token_accuracy': '0.7624', 'epoch': '0.08398'}
{'loss': '0.8163', 'grad_norm': '0.4488', 'learning_rate': '2.3e-05', 'entropy': '0.8805', 'num_tokens': '5.112e+07', 'mean_token_accuracy': '0.7736', 'epoch': '0.08398'}
{'loss': '0.566', 'grad_norm': '0.2478', 'learning_rate': '2.298e-05', 'entropy': '0.9613', 'num_tokens': '5.113e+07', 'mean_token_accuracy': '0.7694', 'epoch': '0.08399'}
{'loss': '1.177', 'grad_norm': '0.4472', 'learning_rate': '2.297e-05', 'entropy': '1.205', 'num_tokens': '5.113e+07', 'mean_token_accuracy': '0.706', 'epoch': '0.084'}
{'loss': '1.16', 'grad_norm': '0.3574', 'learning_rate': '2.295e-05', 'entropy': '0.97', 'num_tokens': '5.114e+07', 'mean_token_accuracy': '0.7736', 'epoch': '0.08401'}
{'loss': '0.9191', 'grad_norm': '0.3217', 'learning_rate': '2.293e-05', 'entropy': '0.7526', 'num_tokens': '5.115e+07', 'mean_token_accuracy': '0.7857', 'epoch': '0.08402'}
{'loss': '0.8589', 'grad_norm': '0.4898', 'learning_rate': '2.291e-05', 'entropy': '0.9387', 'num_tokens': '5.115e+07', 'mean_token_accuracy': '0.7878', 'epoch': '0.08403'}
{'loss': '0.7341', 'grad_norm': '0.4598', 'learning_rate': '2.29e-05', 'entropy': '0.7879', 'num_tokens': '5.116e+07', 'mean_token_accuracy': '0.7986', 'epoch': '0.08404'}
{'loss': '0.5318', 'grad_norm': '0.2876', 'learning_rate': '2.288e-05', 'entropy': '0.5307', 'num_tokens': '5.116e+07', 'mean_token_accuracy': '0.8638', 'epoch': '0.08405'}

 79%|███████▉  | 9498/12000 [5:36:08<1:29:54,  2.16s/it]
 79%|███████▉  | 9499/12000 [5:36:11<1:38:15,  2.36s/it]
                                                        

 79%|███████▉  | 9499/12000 [5:36:11<1:38:15,  2.36s/it]
 79%|███████▉  | 9500/12000 [5:36:12<1:27:16,  2.09s/it]
                                                        

 79%|███████▉  | 9500/12000 [5:36:12<1:27:16,  2.09s/it]
 79%|███████▉  | 9501/12000 [5:36:15<1:39:16,  2.38s/it]
                                                        

 79%|███████▉  | 9501/12000 [5:36:15<1:39:16,  2.38s/it]
 79%|███████▉  | 9502/12000 [5:36:17<1:28:10,  2.12s/it]
                                                        

 79%|███████▉  | 9502/12000 [5:36:17<1:28:10,  2.12s/it]
 79%|███████▉  | 9503/12000 [5:36:19<1:23:04,  2.00s/it]
                                                        

 79%|███████▉  | 9503/12000 [5:36:19<1:23:04,  2.00s/it]
 79%|███████▉  | 9504/12000 [5:36:21<1:21:11,  1.95s/it]
                                                        

 79%|███████▉  | 9504/12000 [5:36:21<1:21:11,  1.95s/it]
 79%|███████▉  | 9505/12000 [5:36:22<1:15:26,  1.81s/it]
                                                        

 79%|███████▉  | 9505/12000 [5:36:22<1:15:26,  1.81s/it]
 79%|███████▉  | 9506/12000 [5:36:24<1:12:52,  1.75s/it]
                                                        

 79%|███████▉  | 9506/12000 [5:36:24<1:12:52,  1.75s/it]
 79%|███████▉  | 9507/12000 [5:36:26<1:23:24,  2.01s/it]
                                                        

 79%|███████▉  | 9507/12000 [5:36:26<1:23:24,  2.01s/it]
 79%|███████▉  | 9508/12000 [5:36:28<1:18:08,  1.88s/it]
                                                        

 79%|███████▉  | 9508/12000 [5:36:28<1:18:08,  1.88s/it]
 79%|███████▉  | 9509/12000 [5:36:32<1:43:07,  2.48s/it]
                                                        

 79%|███████▉  | 9509/12000 [5:36:32<1:43:07,  2.48s/it]
 79%|███████▉  | 9510/12000 [5:36:33<1:31:43,  2.21s/it]
                                                        

 79%|███████▉  | 9510/12000 [5:36:33<1:31:43,  2.21s/it]
 79%|███████▉  | 9511/12000 [5:36:35<1:23:53,  2.02s/it]
                                                        

 79%|███████▉  | 9511/12000 [5:36:35<1:23:53,  2.02s/it]
 79%|███████▉  | 9512/12000 [5:36:36<1:16:57,  1.86s/it]
                                                        

 79%|███████▉  | 9512/12000 [5:36:36<1:16:57,  1.86s/it]
 79%|███████▉  | 9513/12000 [5:36:40<1:35:49,  2.31s/it]
                                                        

 79%|███████▉  | 9513/12000 [5:36:40<1:35:49,  2.31s/it]
 79%|███████▉  | 9514/12000 [5:36:41<1:28:41,  2.14s/it]
                                                        

 79%|███████▉  | 9514/12000 [5:36:41<1:28:41,  2.14s/it]
 79%|███████▉  | 9515/12000 [5:36:44<1:32:02,  2.22s/it]
                                                        

 79%|███████▉  | 9515/12000 [5:36:44<1:32:02,  2.22s/it]
 79%|███████▉  | 9516/12000 [5:36:45<1:22:03,  1.98s/it]
                                                        

 79%|███████▉  | 9516/12000 [5:36:45<1:22:03,  1.98s/it]
 79%|███████▉  | 9517/12000 [5:36:49<1:38:46,  2.39s/it]
                                                        

 79%|███████▉  | 9517/12000 [5:36:49<1:38:46,  2.39s/it]
 79%|███████▉  | 9518/12000 [5:36:53<2:00:48,  2.92s/it]
                                                        

 79%|███████▉  | 9518/12000 [5:36:53<2:00:48,  2.92s/it]
 79%|███████▉  | 9519/12000 [5:36:54<1:42:20,  2.48s/it]
                                                        

 79%|███████▉  | 9519/12000 [5:36:54<1:42:20,  2.48s/it]
 79%|███████▉  | 9520/12000 [5:36:57<1:44:48,  2.54s/it]
                                                        

 79%|███████▉  | 9520/12000 [5:36:57<1:44:48,  2.54s/it]
 79%|███████▉  | 9521/12000 [5:36:59<1:40:57,  2.44s/it]
                                                        

 79%|███████▉  | 9521/12000 [5:36:59<1:40:57,  2.44s/it]
 79%|███████▉  | 9522/12000 [5:37:01<1:29:21,  2.16s/it]
                                                        

 79%|███████▉  | 9522/12000 [5:37:01<1:29:21,  2.16s/it]
 79%|███████▉  | 9523/12000 [5:37:02<1:20:09,  1.94s/it]
                                                        

 79%|███████▉  | 9523/12000 [5:37:02<1:20:09,  1.94s/it]
 79%|███████▉  | 9524/12000 [5:37:04<1:24:06,  2.04s/it]
                                                        

 79%|███████▉  | 9524/12000 [5:37:04<1:24:06,  2.04s/it]
 79%|███████▉  | 9525/12000 [5:37:07<1:35:30,  2.32s/it]
                                                        

 79%|███████▉  | 9525/12000 [5:37:07<1:35:30,  2.32s/it]
 79%|███████▉  | 9526/12000 [5:37:09<1:32:11,  2.24s/it]
                                                        

 79%|███████▉  | 9526/12000 [5:37:09<1:32:11,  2.24s/it]
 79%|███████▉  | 9527/12000 [5:37:11<1:24:19,  2.05s/it]
                                                        

 79%|███████▉  | 9527/12000 [5:37:11<1:24:19,  2.05s/it]
 79%|███████▉  | 9528/12000 [5:37:13<1:22:03,  1.99s/it]
                                                        

 79%|███████▉  | 9528/12000 [5:37:13<1:22:03,  1.99s/it]
 79%|███████▉  | 9529/12000 [5:37:14<1:16:34,  1.86s/it]
                                                        

 79%|███████▉  | 9529/12000 [5:37:14<1:16:34,  1.86s/it]
 79%|███████▉  | 9530/12000 [5:37:16<1:15:41,  1.84s/it]
                                                        

 79%|███████▉  | 9530/12000 [5:37:16<1:15:41,  1.84s/it]
 79%|███████▉  | 9531/12000 [5:37:18<1:14:14,  1.80s/it]
                                                        

 79%|███████▉  | 9531/12000 [5:37:18<1:14:14,  1.80s/it]
 79%|███████▉  | 9532/12000 [5:37:21<1:25:51,  2.09s/it]
                                                        

 79%|███████▉  | 9532/12000 [5:37:21<1:25:51,  2.09s/it]
 79%|███████▉  | 9533/12000 [5:37:22<1:19:55,  1.94s/it]
                                                        

 79%|███████▉  | 9533/12000 [5:37:22<1:19:55,  1.94s/it]
 79%|███████▉  | 9534/12000 [5:37:24<1:13:24,  1.79s/it]
                                                        

 79%|███████▉  | 9534/12000 [5:37:24<1:13:24,  1.79s/it]
 79%|███████▉  | 9535/12000 [5:37:27<1:33:13,  2.27s/it]
                                                        

 79%|███████▉  | 9535/12000 [5:37:27<1:33:13,  2.27s/it]
 79%|███████▉  | 9536/12000 [5:37:29<1:26:09,  2.10s/it]
                                                        

 79%|███████▉  | 9536/12000 [5:37:29<1:26:09,  2.10s/it]
 79%|███████▉  | 9537/12000 [5:37:30<1:20:32,  1.96s/it]
                                                        

 79%|███████▉  | 9537/12000 [5:37:30<1:20:32,  1.96s/it]
 79%|███████▉  | 9538/12000 [5:37:32<1:17:03,  1.88s/it]
                                                        

 79%|███████▉  | 9538/12000 [5:37:32<1:17:03,  1.88s/it]
 79%|███████▉  | 9539/12000 [5:37:35<1:26:50,  2.12s/it]
                                                        

 79%|███████▉  | 9539/12000 [5:37:35<1:26:50,  2.12s/it]
 80%|███████▉  | 9540/12000 [5:37:36<1:21:43,  1.99s/it]
                                                        

 80%|███████▉  | 9540/12000 [5:37:36<1:21:43,  1.99s/it]
 80%|███████▉  | 9541/12000 [5:37:39<1:22:49,  2.02s/it]
                                                        

 80%|███████▉  | 9541/12000 [5:37:39<1:22:49,  2.02s/it]
 80%|███████▉  | 9542/12000 [5:37:41<1:32:41,  2.26s/it]
                                                        

 80%|███████▉  | 9542/12000 [5:37:41<1:32:41,  2.26s/it]
 80%|███████▉  | 9543/12000 [5:37:46<2:00:39,  2.95s/it]
                                                        

 80%|███████▉  | 9543/12000 [5:37:46<2:00:39,  2.95s/it]
 80%|███████▉  | 9544/12000 [5:37:48<1:47:18,  2.62s/it]
                                                        

 80%|███████▉  | 9544/12000 [5:37:48<1:47:18,  2.62s/it]
 80%|███████▉  | 9545/12000 [5:37:49<1:33:18,  2.28s/it]
                                                        
{'loss': '0.5939', 'grad_norm': '0.375', 'learning_rate': '2.286e-05', 'entropy': '0.68', 'num_tokens': '5.117e+07', 'mean_token_accuracy': '0.8325', 'epoch': '0.08406'}
{'loss': '0.7546', 'grad_norm': '0.2892', 'learning_rate': '2.284e-05', 'entropy': '0.8033', 'num_tokens': '5.117e+07', 'mean_token_accuracy': '0.8075', 'epoch': '0.08406'}
{'loss': '1.039', 'grad_norm': '0.562', 'learning_rate': '2.283e-05', 'entropy': '1.011', 'num_tokens': '5.118e+07', 'mean_token_accuracy': '0.7398', 'epoch': '0.08407'}
{'loss': '1.237', 'grad_norm': '0.532', 'learning_rate': '2.281e-05', 'entropy': '1.283', 'num_tokens': '5.118e+07', 'mean_token_accuracy': '0.6974', 'epoch': '0.08408'}
{'loss': '0.6048', 'grad_norm': '0.3571', 'learning_rate': '2.279e-05', 'entropy': '0.7244', 'num_tokens': '5.119e+07', 'mean_token_accuracy': '0.8137', 'epoch': '0.08409'}
{'loss': '0.7095', 'grad_norm': '0.3407', 'learning_rate': '2.277e-05', 'entropy': '0.7639', 'num_tokens': '5.119e+07', 'mean_token_accuracy': '0.81', 'epoch': '0.0841'}
{'loss': '1.314', 'grad_norm': '0.4268', 'learning_rate': '2.276e-05', 'entropy': '1.114', 'num_tokens': '5.12e+07', 'mean_token_accuracy': '0.7521', 'epoch': '0.08411'}
{'loss': '0.8611', 'grad_norm': '0.4182', 'learning_rate': '2.274e-05', 'entropy': '0.8755', 'num_tokens': '5.12e+07', 'mean_token_accuracy': '0.7799', 'epoch': '0.08412'}
{'loss': '0.6221', 'grad_norm': '0.5043', 'learning_rate': '2.272e-05', 'entropy': '0.7341', 'num_tokens': '5.121e+07', 'mean_token_accuracy': '0.8016', 'epoch': '0.08413'}
{'loss': '0.5147', 'grad_norm': '0.3135', 'learning_rate': '2.27e-05', 'entropy': '0.5339', 'num_tokens': '5.121e+07', 'mean_token_accuracy': '0.8583', 'epoch': '0.08414'}
{'loss': '1.001', 'grad_norm': '0.5494', 'learning_rate': '2.269e-05', 'entropy': '1.013', 'num_tokens': '5.122e+07', 'mean_token_accuracy': '0.7413', 'epoch': '0.08414'}
{'loss': '0.6153', 'grad_norm': '0.3033', 'learning_rate': '2.267e-05', 'entropy': '0.7104', 'num_tokens': '5.122e+07', 'mean_token_accuracy': '0.8195', 'epoch': '0.08415'}
{'loss': '0.8096', 'grad_norm': '0.4025', 'learning_rate': '2.265e-05', 'entropy': '0.781', 'num_tokens': '5.123e+07', 'mean_token_accuracy': '0.8061', 'epoch': '0.08416'}
{'loss': '1.026', 'grad_norm': '0.4069', 'learning_rate': '2.263e-05', 'entropy': '0.9509', 'num_tokens': '5.124e+07', 'mean_token_accuracy': '0.7631', 'epoch': '0.08417'}
{'loss': '0.703', 'grad_norm': '0.4301', 'learning_rate': '2.262e-05', 'entropy': '0.7156', 'num_tokens': '5.124e+07', 'mean_token_accuracy': '0.8257', 'epoch': '0.08418'}
{'loss': '0.8913', 'grad_norm': '0.4099', 'learning_rate': '2.26e-05', 'entropy': '0.8271', 'num_tokens': '5.125e+07', 'mean_token_accuracy': '0.7974', 'epoch': '0.08419'}
{'loss': '0.6252', 'grad_norm': '0.3975', 'learning_rate': '2.258e-05', 'entropy': '0.6635', 'num_tokens': '5.125e+07', 'mean_token_accuracy': '0.82', 'epoch': '0.0842'}
{'loss': '1.135', 'grad_norm': '0.4191', 'learning_rate': '2.256e-05', 'entropy': '1.066', 'num_tokens': '5.126e+07', 'mean_token_accuracy': '0.7399', 'epoch': '0.08421'}
{'loss': '0.6778', 'grad_norm': '0.3748', 'learning_rate': '2.255e-05', 'entropy': '0.7406', 'num_tokens': '5.126e+07', 'mean_token_accuracy': '0.8055', 'epoch': '0.08421'}
{'loss': '1.013', 'grad_norm': '0.4438', 'learning_rate': '2.253e-05', 'entropy': '1.026', 'num_tokens': '5.127e+07', 'mean_token_accuracy': '0.7403', 'epoch': '0.08422'}
{'loss': '0.8307', 'grad_norm': '0.3405', 'learning_rate': '2.251e-05', 'entropy': '0.8666', 'num_tokens': '5.128e+07', 'mean_token_accuracy': '0.7853', 'epoch': '0.08423'}
{'loss': '0.7745', 'grad_norm': '0.3957', 'learning_rate': '2.249e-05', 'entropy': '0.799', 'num_tokens': '5.128e+07', 'mean_token_accuracy': '0.8084', 'epoch': '0.08424'}
{'loss': '1.176', 'grad_norm': '0.4351', 'learning_rate': '2.248e-05', 'entropy': '0.9787', 'num_tokens': '5.129e+07', 'mean_token_accuracy': '0.7712', 'epoch': '0.08425'}
{'loss': '0.7683', 'grad_norm': '0.3797', 'learning_rate': '2.246e-05', 'entropy': '0.7733', 'num_tokens': '5.129e+07', 'mean_token_accuracy': '0.8116', 'epoch': '0.08426'}
{'loss': '0.8303', 'grad_norm': '0.4705', 'learning_rate': '2.244e-05', 'entropy': '0.9133', 'num_tokens': '5.13e+07', 'mean_token_accuracy': '0.7586', 'epoch': '0.08427'}
{'loss': '0.8631', 'grad_norm': '0.4829', 'learning_rate': '2.242e-05', 'entropy': '0.9072', 'num_tokens': '5.13e+07', 'mean_token_accuracy': '0.7716', 'epoch': '0.08428'}
{'loss': '0.6194', 'grad_norm': '0.3151', 'learning_rate': '2.241e-05', 'entropy': '0.6191', 'num_tokens': '5.131e+07', 'mean_token_accuracy': '0.8493', 'epoch': '0.08429'}
{'loss': '0.712', 'grad_norm': '0.3702', 'learning_rate': '2.239e-05', 'entropy': '0.7786', 'num_tokens': '5.131e+07', 'mean_token_accuracy': '0.8028', 'epoch': '0.08429'}
{'loss': '0.905', 'grad_norm': '0.569', 'learning_rate': '2.237e-05', 'entropy': '0.7888', 'num_tokens': '5.132e+07', 'mean_token_accuracy': '0.7855', 'epoch': '0.0843'}
{'loss': '0.8803', 'grad_norm': '0.4315', 'learning_rate': '2.235e-05', 'entropy': '0.7786', 'num_tokens': '5.132e+07', 'mean_token_accuracy': '0.7967', 'epoch': '0.08431'}
{'loss': '0.8587', 'grad_norm': '0.42', 'learning_rate': '2.234e-05', 'entropy': '0.9362', 'num_tokens': '5.133e+07', 'mean_token_accuracy': '0.7751', 'epoch': '0.08432'}
{'loss': '0.891', 'grad_norm': '0.4083', 'learning_rate': '2.232e-05', 'entropy': '0.8654', 'num_tokens': '5.133e+07', 'mean_token_accuracy': '0.7756', 'epoch': '0.08433'}
{'loss': '0.949', 'grad_norm': '0.4397', 'learning_rate': '2.23e-05', 'entropy': '0.9144', 'num_tokens': '5.134e+07', 'mean_token_accuracy': '0.7772', 'epoch': '0.08434'}
{'loss': '0.8349', 'grad_norm': '0.4462', 'learning_rate': '2.229e-05', 'entropy': '0.7804', 'num_tokens': '5.134e+07', 'mean_token_accuracy': '0.8023', 'epoch': '0.08435'}
{'loss': '0.4673', 'grad_norm': '0.3461', 'learning_rate': '2.227e-05', 'entropy': '0.4669', 'num_tokens': '5.135e+07', 'mean_token_accuracy': '0.872', 'epoch': '0.08436'}
{'loss': '0.9744', 'grad_norm': '0.4745', 'learning_rate': '2.225e-05', 'entropy': '1.003', 'num_tokens': '5.135e+07', 'mean_token_accuracy': '0.7538', 'epoch': '0.08437'}
{'loss': '0.6377', 'grad_norm': '0.3567', 'learning_rate': '2.223e-05', 'entropy': '0.7035', 'num_tokens': '5.136e+07', 'mean_token_accuracy': '0.8174', 'epoch': '0.08437'}
{'loss': '0.9432', 'grad_norm': '0.4243', 'learning_rate': '2.222e-05', 'entropy': '0.9899', 'num_tokens': '5.136e+07', 'mean_token_accuracy': '0.7683', 'epoch': '0.08438'}
{'loss': '0.4999', 'grad_norm': '0.3184', 'learning_rate': '2.22e-05', 'entropy': '0.5431', 'num_tokens': '5.137e+07', 'mean_token_accuracy': '0.8659', 'epoch': '0.08439'}
{'loss': '0.7194', 'grad_norm': '0.3772', 'learning_rate': '2.218e-05', 'entropy': '0.7877', 'num_tokens': '5.137e+07', 'mean_token_accuracy': '0.7871', 'epoch': '0.0844'}
{'loss': '0.7815', 'grad_norm': '0.2872', 'learning_rate': '2.216e-05', 'entropy': '0.7954', 'num_tokens': '5.138e+07', 'mean_token_accuracy': '0.7936', 'epoch': '0.08441'}
{'loss': '0.8831', 'grad_norm': '0.3238', 'learning_rate': '2.215e-05', 'entropy': '0.8793', 'num_tokens': '5.138e+07', 'mean_token_accuracy': '0.7828', 'epoch': '0.08442'}
{'loss': '0.715', 'grad_norm': '0.3926', 'learning_rate': '2.213e-05', 'entropy': '0.7424', 'num_tokens': '5.139e+07', 'mean_token_accuracy': '0.8106', 'epoch': '0.08443'}
{'loss': '0.7462', 'grad_norm': '0.4069', 'learning_rate': '2.211e-05', 'entropy': '0.7432', 'num_tokens': '5.14e+07', 'mean_token_accuracy': '0.8151', 'epoch': '0.08444'}
{'loss': '1.199', 'grad_norm': '0.3812', 'learning_rate': '2.209e-05', 'entropy': '1.158', 'num_tokens': '5.14e+07', 'mean_token_accuracy': '0.7105', 'epoch': '0.08445'}
{'loss': '1.291', 'grad_norm': '0.4514', 'learning_rate': '2.208e-05', 'entropy': '0.9364', 'num_tokens': '5.141e+07', 'mean_token_accuracy': '0.7567', 'epoch': '0.08445'}
{'loss': '0.5882', 'grad_norm': '0.3535', 'learning_rate': '2.206e-05', 'entropy': '0.6041', 'num_tokens': '5.142e+07', 'mean_token_accuracy': '0.8452', 'epoch': '0.08446'}

 80%|███████▉  | 9545/12000 [5:37:49<1:33:18,  2.28s/it]
 80%|███████▉  | 9546/12000 [5:37:51<1:21:55,  2.00s/it]
                                                        

 80%|███████▉  | 9546/12000 [5:37:51<1:21:55,  2.00s/it]
 80%|███████▉  | 9547/12000 [5:37:52<1:15:11,  1.84s/it]
                                                        

 80%|███████▉  | 9547/12000 [5:37:52<1:15:11,  1.84s/it]
 80%|███████▉  | 9548/12000 [5:37:55<1:24:45,  2.07s/it]
                                                        

 80%|███████▉  | 9548/12000 [5:37:55<1:24:45,  2.07s/it]
 80%|███████▉  | 9549/12000 [5:37:56<1:17:20,  1.89s/it]
                                                        

 80%|███████▉  | 9549/12000 [5:37:56<1:17:20,  1.89s/it]
 80%|███████▉  | 9550/12000 [5:37:59<1:24:44,  2.08s/it]
                                                        

 80%|███████▉  | 9550/12000 [5:37:59<1:24:44,  2.08s/it]
 80%|███████▉  | 9551/12000 [5:38:02<1:35:10,  2.33s/it]
                                                        

 80%|███████▉  | 9551/12000 [5:38:02<1:35:10,  2.33s/it]
 80%|███████▉  | 9552/12000 [5:38:03<1:25:36,  2.10s/it]
                                                        

 80%|███████▉  | 9552/12000 [5:38:03<1:25:36,  2.10s/it]
 80%|███████▉  | 9553/12000 [5:38:05<1:17:11,  1.89s/it]
                                                        

 80%|███████▉  | 9553/12000 [5:38:05<1:17:11,  1.89s/it]
 80%|███████▉  | 9554/12000 [5:38:06<1:14:52,  1.84s/it]
                                                        

 80%|███████▉  | 9554/12000 [5:38:06<1:14:52,  1.84s/it]
 80%|███████▉  | 9555/12000 [5:38:08<1:18:01,  1.91s/it]
                                                        

 80%|███████▉  | 9555/12000 [5:38:08<1:18:01,  1.91s/it]
 80%|███████▉  | 9556/12000 [5:38:10<1:10:39,  1.73s/it]
                                                        

 80%|███████▉  | 9556/12000 [5:38:10<1:10:39,  1.73s/it]
 80%|███████▉  | 9557/12000 [5:38:13<1:29:53,  2.21s/it]
                                                        

 80%|███████▉  | 9557/12000 [5:38:13<1:29:53,  2.21s/it]
 80%|███████▉  | 9558/12000 [5:38:15<1:30:17,  2.22s/it]
                                                        

 80%|███████▉  | 9558/12000 [5:38:15<1:30:17,  2.22s/it]
 80%|███████▉  | 9559/12000 [5:38:17<1:20:58,  1.99s/it]
                                                        

 80%|███████▉  | 9559/12000 [5:38:17<1:20:58,  1.99s/it]
 80%|███████▉  | 9560/12000 [5:38:18<1:14:51,  1.84s/it]
                                                        

 80%|███████▉  | 9560/12000 [5:38:18<1:14:51,  1.84s/it]
 80%|███████▉  | 9561/12000 [5:38:20<1:09:28,  1.71s/it]
                                                        

 80%|███████▉  | 9561/12000 [5:38:20<1:09:28,  1.71s/it]
 80%|███████▉  | 9562/12000 [5:38:21<1:08:09,  1.68s/it]
                                                        

 80%|███████▉  | 9562/12000 [5:38:21<1:08:09,  1.68s/it]
 80%|███████▉  | 9563/12000 [5:38:23<1:09:23,  1.71s/it]
                                                        

 80%|███████▉  | 9563/12000 [5:38:23<1:09:23,  1.71s/it]
 80%|███████▉  | 9564/12000 [5:38:25<1:19:09,  1.95s/it]
                                                        

 80%|███████▉  | 9564/12000 [5:38:25<1:19:09,  1.95s/it]
 80%|███████▉  | 9565/12000 [5:38:28<1:21:22,  2.01s/it]
                                                        

 80%|███████▉  | 9565/12000 [5:38:28<1:21:22,  2.01s/it]
 80%|███████▉  | 9566/12000 [5:38:29<1:18:12,  1.93s/it]
                                                        

 80%|███████▉  | 9566/12000 [5:38:29<1:18:12,  1.93s/it]
 80%|███████▉  | 9567/12000 [5:38:33<1:33:33,  2.31s/it]
                                                        

 80%|███████▉  | 9567/12000 [5:38:33<1:33:33,  2.31s/it]
 80%|███████▉  | 9568/12000 [5:38:34<1:23:38,  2.06s/it]
                                                        

 80%|███████▉  | 9568/12000 [5:38:34<1:23:38,  2.06s/it]
 80%|███████▉  | 9569/12000 [5:38:37<1:33:46,  2.31s/it]
                                                        

 80%|███████▉  | 9569/12000 [5:38:37<1:33:46,  2.31s/it]
 80%|███████▉  | 9570/12000 [5:38:38<1:22:39,  2.04s/it]
                                                        

 80%|███████▉  | 9570/12000 [5:38:38<1:22:39,  2.04s/it]
 80%|███████▉  | 9571/12000 [5:38:40<1:15:04,  1.85s/it]
                                                        

 80%|███████▉  | 9571/12000 [5:38:40<1:15:04,  1.85s/it]
 80%|███████▉  | 9572/12000 [5:38:42<1:23:53,  2.07s/it]
                                                        

 80%|███████▉  | 9572/12000 [5:38:42<1:23:53,  2.07s/it]
 80%|███████▉  | 9573/12000 [5:38:45<1:32:26,  2.29s/it]
                                                        

 80%|███████▉  | 9573/12000 [5:38:45<1:32:26,  2.29s/it]
 80%|███████▉  | 9574/12000 [5:38:48<1:36:34,  2.39s/it]
                                                        

 80%|███████▉  | 9574/12000 [5:38:48<1:36:34,  2.39s/it]
 80%|███████▉  | 9575/12000 [5:38:49<1:25:16,  2.11s/it]
                                                        

 80%|███████▉  | 9575/12000 [5:38:49<1:25:16,  2.11s/it]
 80%|███████▉  | 9576/12000 [5:38:52<1:27:20,  2.16s/it]
                                                        

 80%|███████▉  | 9576/12000 [5:38:52<1:27:20,  2.16s/it]
 80%|███████▉  | 9577/12000 [5:38:54<1:34:12,  2.33s/it]
                                                        

 80%|███████▉  | 9577/12000 [5:38:54<1:34:12,  2.33s/it]
 80%|███████▉  | 9578/12000 [5:38:56<1:27:06,  2.16s/it]
                                                        

 80%|███████▉  | 9578/12000 [5:38:56<1:27:06,  2.16s/it]
 80%|███████▉  | 9579/12000 [5:38:59<1:42:36,  2.54s/it]
                                                        

 80%|███████▉  | 9579/12000 [5:38:59<1:42:36,  2.54s/it]
 80%|███████▉  | 9580/12000 [5:39:01<1:35:29,  2.37s/it]
                                                        

 80%|███████▉  | 9580/12000 [5:39:01<1:35:29,  2.37s/it]
 80%|███████▉  | 9581/12000 [5:39:03<1:28:58,  2.21s/it]
                                                        

 80%|███████▉  | 9581/12000 [5:39:03<1:28:58,  2.21s/it]
 80%|███████▉  | 9582/12000 [5:39:05<1:29:21,  2.22s/it]
                                                        

 80%|███████▉  | 9582/12000 [5:39:05<1:29:21,  2.22s/it]
 80%|███████▉  | 9583/12000 [5:39:07<1:26:39,  2.15s/it]
                                                        

 80%|███████▉  | 9583/12000 [5:39:07<1:26:39,  2.15s/it]
 80%|███████▉  | 9584/12000 [5:39:09<1:21:49,  2.03s/it]
                                                        

 80%|███████▉  | 9584/12000 [5:39:09<1:21:49,  2.03s/it]
 80%|███████▉  | 9585/12000 [5:39:12<1:26:19,  2.14s/it]
                                                        

 80%|███████▉  | 9585/12000 [5:39:12<1:26:19,  2.14s/it]
 80%|███████▉  | 9586/12000 [5:39:15<1:35:14,  2.37s/it]
                                                        

 80%|███████▉  | 9586/12000 [5:39:15<1:35:14,  2.37s/it]
 80%|███████▉  | 9587/12000 [5:39:16<1:24:41,  2.11s/it]
                                                        

 80%|███████▉  | 9587/12000 [5:39:16<1:24:41,  2.11s/it]
 80%|███████▉  | 9588/12000 [5:39:18<1:17:44,  1.93s/it]
                                                        

 80%|███████▉  | 9588/12000 [5:39:18<1:17:44,  1.93s/it]
 80%|███████▉  | 9589/12000 [5:39:20<1:25:34,  2.13s/it]
                                                        

 80%|███████▉  | 9589/12000 [5:39:20<1:25:34,  2.13s/it]
 80%|███████▉  | 9590/12000 [5:39:22<1:23:25,  2.08s/it]
                                                        

 80%|███████▉  | 9590/12000 [5:39:22<1:23:25,  2.08s/it]
 80%|███████▉  | 9591/12000 [5:39:24<1:16:50,  1.91s/it]
                                                        

 80%|███████▉  | 9591/12000 [5:39:24<1:16:50,  1.91s/it]
 80%|███████▉  | 9592/12000 [5:39:26<1:22:26,  2.05s/it]
                                                        
{'loss': '0.8147', 'grad_norm': '0.4984', 'learning_rate': '2.204e-05', 'entropy': '0.9701', 'num_tokens': '5.142e+07', 'mean_token_accuracy': '0.7586', 'epoch': '0.08447'}
{'loss': '0.9739', 'grad_norm': '0.3965', 'learning_rate': '2.203e-05', 'entropy': '0.959', 'num_tokens': '5.142e+07', 'mean_token_accuracy': '0.7611', 'epoch': '0.08448'}
{'loss': '0.9092', 'grad_norm': '0.3786', 'learning_rate': '2.201e-05', 'entropy': '1.039', 'num_tokens': '5.143e+07', 'mean_token_accuracy': '0.7461', 'epoch': '0.08449'}
{'loss': '0.5541', 'grad_norm': '0.347', 'learning_rate': '2.199e-05', 'entropy': '0.5536', 'num_tokens': '5.144e+07', 'mean_token_accuracy': '0.8587', 'epoch': '0.0845'}
{'loss': '0.8315', 'grad_norm': '0.457', 'learning_rate': '2.197e-05', 'entropy': '0.8037', 'num_tokens': '5.144e+07', 'mean_token_accuracy': '0.8082', 'epoch': '0.08451'}
{'loss': '0.6923', 'grad_norm': '0.3546', 'learning_rate': '2.196e-05', 'entropy': '0.7646', 'num_tokens': '5.145e+07', 'mean_token_accuracy': '0.8138', 'epoch': '0.08452'}
{'loss': '0.7068', 'grad_norm': '0.346', 'learning_rate': '2.194e-05', 'entropy': '0.7512', 'num_tokens': '5.145e+07', 'mean_token_accuracy': '0.8224', 'epoch': '0.08452'}
{'loss': '0.9607', 'grad_norm': '0.3989', 'learning_rate': '2.192e-05', 'entropy': '0.9699', 'num_tokens': '5.145e+07', 'mean_token_accuracy': '0.7685', 'epoch': '0.08453'}
{'loss': '0.9685', 'grad_norm': '0.4651', 'learning_rate': '2.191e-05', 'entropy': '1.041', 'num_tokens': '5.146e+07', 'mean_token_accuracy': '0.7423', 'epoch': '0.08454'}
{'loss': '1.009', 'grad_norm': '0.367', 'learning_rate': '2.189e-05', 'entropy': '0.9848', 'num_tokens': '5.146e+07', 'mean_token_accuracy': '0.7619', 'epoch': '0.08455'}
{'loss': '0.8482', 'grad_norm': '0.4981', 'learning_rate': '2.187e-05', 'entropy': '0.7592', 'num_tokens': '5.147e+07', 'mean_token_accuracy': '0.8092', 'epoch': '0.08456'}
{'loss': '0.7266', 'grad_norm': '0.3827', 'learning_rate': '2.185e-05', 'entropy': '0.783', 'num_tokens': '5.147e+07', 'mean_token_accuracy': '0.8015', 'epoch': '0.08457'}
{'loss': '0.8447', 'grad_norm': '0.3994', 'learning_rate': '2.184e-05', 'entropy': '0.8999', 'num_tokens': '5.148e+07', 'mean_token_accuracy': '0.7772', 'epoch': '0.08458'}
{'loss': '1.195', 'grad_norm': '0.4297', 'learning_rate': '2.182e-05', 'entropy': '1.151', 'num_tokens': '5.148e+07', 'mean_token_accuracy': '0.722', 'epoch': '0.08459'}
{'loss': '1.078', 'grad_norm': '0.4306', 'learning_rate': '2.18e-05', 'entropy': '1.098', 'num_tokens': '5.149e+07', 'mean_token_accuracy': '0.7367', 'epoch': '0.0846'}
{'loss': '0.9138', 'grad_norm': '0.4287', 'learning_rate': '2.178e-05', 'entropy': '0.9026', 'num_tokens': '5.149e+07', 'mean_token_accuracy': '0.7835', 'epoch': '0.0846'}
{'loss': '0.9373', 'grad_norm': '0.5094', 'learning_rate': '2.177e-05', 'entropy': '0.9488', 'num_tokens': '5.15e+07', 'mean_token_accuracy': '0.759', 'epoch': '0.08461'}
{'loss': '0.5728', 'grad_norm': '0.3839', 'learning_rate': '2.175e-05', 'entropy': '0.5727', 'num_tokens': '5.15e+07', 'mean_token_accuracy': '0.8558', 'epoch': '0.08462'}
{'loss': '0.7555', 'grad_norm': '0.4693', 'learning_rate': '2.173e-05', 'entropy': '0.7718', 'num_tokens': '5.151e+07', 'mean_token_accuracy': '0.81', 'epoch': '0.08463'}
{'loss': '0.8903', 'grad_norm': '0.4175', 'learning_rate': '2.172e-05', 'entropy': '0.9622', 'num_tokens': '5.151e+07', 'mean_token_accuracy': '0.7736', 'epoch': '0.08464'}
{'loss': '1.039', 'grad_norm': '0.4623', 'learning_rate': '2.17e-05', 'entropy': '0.985', 'num_tokens': '5.152e+07', 'mean_token_accuracy': '0.7468', 'epoch': '0.08465'}
{'loss': '0.7685', 'grad_norm': '0.3791', 'learning_rate': '2.168e-05', 'entropy': '0.7796', 'num_tokens': '5.153e+07', 'mean_token_accuracy': '0.8163', 'epoch': '0.08466'}
{'loss': '0.6975', 'grad_norm': '0.3769', 'learning_rate': '2.166e-05', 'entropy': '0.6528', 'num_tokens': '5.153e+07', 'mean_token_accuracy': '0.8332', 'epoch': '0.08467'}
{'loss': '0.7828', 'grad_norm': '0.3956', 'learning_rate': '2.165e-05', 'entropy': '0.7663', 'num_tokens': '5.154e+07', 'mean_token_accuracy': '0.8099', 'epoch': '0.08468'}
{'loss': '0.5555', 'grad_norm': '0.3392', 'learning_rate': '2.163e-05', 'entropy': '0.5534', 'num_tokens': '5.154e+07', 'mean_token_accuracy': '0.8598', 'epoch': '0.08468'}
{'loss': '0.7351', 'grad_norm': '0.4835', 'learning_rate': '2.161e-05', 'entropy': '0.7438', 'num_tokens': '5.155e+07', 'mean_token_accuracy': '0.8099', 'epoch': '0.08469'}
{'loss': '0.739', 'grad_norm': '0.4755', 'learning_rate': '2.16e-05', 'entropy': '0.7834', 'num_tokens': '5.155e+07', 'mean_token_accuracy': '0.7944', 'epoch': '0.0847'}
{'loss': '0.6185', 'grad_norm': '0.3292', 'learning_rate': '2.158e-05', 'entropy': '0.713', 'num_tokens': '5.156e+07', 'mean_token_accuracy': '0.8184', 'epoch': '0.08471'}
{'loss': '0.7511', 'grad_norm': '0.393', 'learning_rate': '2.156e-05', 'entropy': '0.7136', 'num_tokens': '5.157e+07', 'mean_token_accuracy': '0.8179', 'epoch': '0.08472'}
{'loss': '1.059', 'grad_norm': '0.3644', 'learning_rate': '2.155e-05', 'entropy': '0.8733', 'num_tokens': '5.157e+07', 'mean_token_accuracy': '0.7882', 'epoch': '0.08473'}
{'loss': '0.6565', 'grad_norm': '0.3776', 'learning_rate': '2.153e-05', 'entropy': '0.6463', 'num_tokens': '5.158e+07', 'mean_token_accuracy': '0.8257', 'epoch': '0.08474'}
{'loss': '0.9072', 'grad_norm': '0.5177', 'learning_rate': '2.151e-05', 'entropy': '0.874', 'num_tokens': '5.158e+07', 'mean_token_accuracy': '0.7791', 'epoch': '0.08475'}
{'loss': '0.7358', 'grad_norm': '0.3523', 'learning_rate': '2.149e-05', 'entropy': '0.8393', 'num_tokens': '5.159e+07', 'mean_token_accuracy': '0.7863', 'epoch': '0.08475'}
{'loss': '0.9253', 'grad_norm': '0.439', 'learning_rate': '2.148e-05', 'entropy': '0.9155', 'num_tokens': '5.16e+07', 'mean_token_accuracy': '0.7682', 'epoch': '0.08476'}
{'loss': '0.6887', 'grad_norm': '0.3791', 'learning_rate': '2.146e-05', 'entropy': '0.756', 'num_tokens': '5.16e+07', 'mean_token_accuracy': '0.8176', 'epoch': '0.08477'}
{'loss': '0.9584', 'grad_norm': '0.3643', 'learning_rate': '2.144e-05', 'entropy': '1.019', 'num_tokens': '5.161e+07', 'mean_token_accuracy': '0.7635', 'epoch': '0.08478'}
{'loss': '0.6341', 'grad_norm': '0.3151', 'learning_rate': '2.143e-05', 'entropy': '0.5972', 'num_tokens': '5.161e+07', 'mean_token_accuracy': '0.8367', 'epoch': '0.08479'}
{'loss': '0.834', 'grad_norm': '0.4223', 'learning_rate': '2.141e-05', 'entropy': '0.8147', 'num_tokens': '5.162e+07', 'mean_token_accuracy': '0.801', 'epoch': '0.0848'}
{'loss': '0.7394', 'grad_norm': '0.4739', 'learning_rate': '2.139e-05', 'entropy': '0.7664', 'num_tokens': '5.162e+07', 'mean_token_accuracy': '0.8042', 'epoch': '0.08481'}
{'loss': '0.9385', 'grad_norm': '0.3689', 'learning_rate': '2.137e-05', 'entropy': '1.005', 'num_tokens': '5.163e+07', 'mean_token_accuracy': '0.7597', 'epoch': '0.08482'}
{'loss': '0.968', 'grad_norm': '0.4518', 'learning_rate': '2.136e-05', 'entropy': '1.059', 'num_tokens': '5.163e+07', 'mean_token_accuracy': '0.7383', 'epoch': '0.08483'}
{'loss': '0.6986', 'grad_norm': '0.3015', 'learning_rate': '2.134e-05', 'entropy': '0.7982', 'num_tokens': '5.164e+07', 'mean_token_accuracy': '0.7922', 'epoch': '0.08483'}
{'loss': '1.035', 'grad_norm': '0.3637', 'learning_rate': '2.132e-05', 'entropy': '1.064', 'num_tokens': '5.165e+07', 'mean_token_accuracy': '0.7482', 'epoch': '0.08484'}
{'loss': '0.4658', 'grad_norm': '0.3201', 'learning_rate': '2.131e-05', 'entropy': '0.4711', 'num_tokens': '5.165e+07', 'mean_token_accuracy': '0.8703', 'epoch': '0.08485'}
{'loss': '0.7712', 'grad_norm': '0.3492', 'learning_rate': '2.129e-05', 'entropy': '0.7613', 'num_tokens': '5.166e+07', 'mean_token_accuracy': '0.7993', 'epoch': '0.08486'}
{'loss': '0.7811', 'grad_norm': '0.3614', 'learning_rate': '2.127e-05', 'entropy': '0.8321', 'num_tokens': '5.166e+07', 'mean_token_accuracy': '0.7922', 'epoch': '0.08487'}
{'loss': '0.4582', 'grad_norm': '0.3391', 'learning_rate': '2.126e-05', 'entropy': '0.5802', 'num_tokens': '5.167e+07', 'mean_token_accuracy': '0.8445', 'epoch': '0.08488'}

 80%|███████▉  | 9592/12000 [5:39:26<1:22:26,  2.05s/it]
 80%|███████▉  | 9593/12000 [5:39:28<1:16:04,  1.90s/it]
                                                        

 80%|███████▉  | 9593/12000 [5:39:28<1:16:04,  1.90s/it]
 80%|███████▉  | 9594/12000 [5:39:29<1:12:45,  1.81s/it]
                                                        

 80%|███████▉  | 9594/12000 [5:39:29<1:12:45,  1.81s/it]
 80%|███████▉  | 9595/12000 [5:39:32<1:19:33,  1.98s/it]
                                                        

 80%|███████▉  | 9595/12000 [5:39:32<1:19:33,  1.98s/it]
 80%|███████▉  | 9596/12000 [5:39:33<1:15:14,  1.88s/it]
                                                        

 80%|███████▉  | 9596/12000 [5:39:33<1:15:14,  1.88s/it]
 80%|███████▉  | 9597/12000 [5:39:35<1:19:40,  1.99s/it]
                                                        

 80%|███████▉  | 9597/12000 [5:39:35<1:19:40,  1.99s/it]
 80%|███████▉  | 9598/12000 [5:39:37<1:12:40,  1.82s/it]
                                                        

 80%|███████▉  | 9598/12000 [5:39:37<1:12:40,  1.82s/it]
 80%|███████▉  | 9599/12000 [5:39:39<1:11:09,  1.78s/it]
                                                        

 80%|███████▉  | 9599/12000 [5:39:39<1:11:09,  1.78s/it]
 80%|████████  | 9600/12000 [5:39:41<1:23:01,  2.08s/it]
                                                        

 80%|████████  | 9600/12000 [5:39:41<1:23:01,  2.08s/it]
 80%|████████  | 9601/12000 [5:39:47<2:12:40,  3.32s/it]
                                                        

 80%|████████  | 9601/12000 [5:39:47<2:12:40,  3.32s/it]
 80%|████████  | 9602/12000 [5:39:50<2:08:37,  3.22s/it]
                                                        

 80%|████████  | 9602/12000 [5:39:50<2:08:37,  3.22s/it]
 80%|████████  | 9603/12000 [5:39:54<2:13:31,  3.34s/it]
                                                        

 80%|████████  | 9603/12000 [5:39:54<2:13:31,  3.34s/it]
 80%|████████  | 9604/12000 [5:39:56<1:52:53,  2.83s/it]
                                                        

 80%|████████  | 9604/12000 [5:39:56<1:52:53,  2.83s/it]
 80%|████████  | 9605/12000 [5:39:57<1:38:36,  2.47s/it]
                                                        

 80%|████████  | 9605/12000 [5:39:57<1:38:36,  2.47s/it]
 80%|████████  | 9606/12000 [5:39:59<1:24:51,  2.13s/it]
                                                        

 80%|████████  | 9606/12000 [5:39:59<1:24:51,  2.13s/it]
 80%|████████  | 9607/12000 [5:40:00<1:17:54,  1.95s/it]
                                                        

 80%|████████  | 9607/12000 [5:40:00<1:17:54,  1.95s/it]
 80%|████████  | 9608/12000 [5:40:02<1:13:47,  1.85s/it]
                                                        

 80%|████████  | 9608/12000 [5:40:02<1:13:47,  1.85s/it]
 80%|████████  | 9609/12000 [5:40:04<1:13:09,  1.84s/it]
                                                        

 80%|████████  | 9609/12000 [5:40:04<1:13:09,  1.84s/it]
 80%|████████  | 9610/12000 [5:40:05<1:11:34,  1.80s/it]
                                                        

 80%|████████  | 9610/12000 [5:40:05<1:11:34,  1.80s/it]
 80%|████████  | 9611/12000 [5:40:07<1:09:23,  1.74s/it]
                                                        

 80%|████████  | 9611/12000 [5:40:07<1:09:23,  1.74s/it]
 80%|████████  | 9612/12000 [5:40:09<1:08:13,  1.71s/it]
                                                        

 80%|████████  | 9612/12000 [5:40:09<1:08:13,  1.71s/it]
 80%|████████  | 9613/12000 [5:40:10<1:08:29,  1.72s/it]
                                                        

 80%|████████  | 9613/12000 [5:40:10<1:08:29,  1.72s/it]
 80%|████████  | 9614/12000 [5:40:14<1:29:09,  2.24s/it]
                                                        

 80%|████████  | 9614/12000 [5:40:14<1:29:09,  2.24s/it]
 80%|████████  | 9615/12000 [5:40:16<1:23:44,  2.11s/it]
                                                        

 80%|████████  | 9615/12000 [5:40:16<1:23:44,  2.11s/it]
 80%|████████  | 9616/12000 [5:40:17<1:18:36,  1.98s/it]
                                                        

 80%|████████  | 9616/12000 [5:40:17<1:18:36,  1.98s/it]
 80%|████████  | 9617/12000 [5:40:20<1:25:09,  2.14s/it]
                                                        

 80%|████████  | 9617/12000 [5:40:20<1:25:09,  2.14s/it]
 80%|████████  | 9618/12000 [5:40:21<1:19:07,  1.99s/it]
                                                        

 80%|████████  | 9618/12000 [5:40:21<1:19:07,  1.99s/it]
 80%|████████  | 9619/12000 [5:40:24<1:28:20,  2.23s/it]
                                                        

 80%|████████  | 9619/12000 [5:40:24<1:28:20,  2.23s/it]
 80%|████████  | 9620/12000 [5:40:27<1:35:02,  2.40s/it]
                                                        

 80%|████████  | 9620/12000 [5:40:27<1:35:02,  2.40s/it]
 80%|████████  | 9621/12000 [5:40:29<1:32:24,  2.33s/it]
                                                        

 80%|████████  | 9621/12000 [5:40:29<1:32:24,  2.33s/it]
 80%|████████  | 9622/12000 [5:40:34<1:56:24,  2.94s/it]
                                                        

 80%|████████  | 9622/12000 [5:40:34<1:56:24,  2.94s/it]
 80%|████████  | 9623/12000 [5:40:35<1:40:36,  2.54s/it]
                                                        

 80%|████████  | 9623/12000 [5:40:35<1:40:36,  2.54s/it]
 80%|████████  | 9624/12000 [5:40:37<1:28:13,  2.23s/it]
                                                        

 80%|████████  | 9624/12000 [5:40:37<1:28:13,  2.23s/it]
 80%|████████  | 9625/12000 [5:40:39<1:24:21,  2.13s/it]
                                                        

 80%|████████  | 9625/12000 [5:40:39<1:24:21,  2.13s/it]
 80%|████████  | 9626/12000 [5:40:40<1:17:43,  1.96s/it]
                                                        

 80%|████████  | 9626/12000 [5:40:40<1:17:43,  1.96s/it]
 80%|████████  | 9627/12000 [5:40:42<1:21:43,  2.07s/it]
                                                        

 80%|████████  | 9627/12000 [5:40:42<1:21:43,  2.07s/it]
 80%|████████  | 9628/12000 [5:40:44<1:16:54,  1.95s/it]
                                                        

 80%|████████  | 9628/12000 [5:40:44<1:16:54,  1.95s/it]
 80%|████████  | 9629/12000 [5:40:46<1:17:46,  1.97s/it]
                                                        

 80%|████████  | 9629/12000 [5:40:46<1:17:46,  1.97s/it]
 80%|████████  | 9630/12000 [5:40:48<1:14:57,  1.90s/it]
                                                        

 80%|████████  | 9630/12000 [5:40:48<1:14:57,  1.90s/it]
 80%|████████  | 9631/12000 [5:40:50<1:13:05,  1.85s/it]
                                                        

 80%|████████  | 9631/12000 [5:40:50<1:13:05,  1.85s/it]
 80%|████████  | 9632/12000 [5:40:51<1:08:32,  1.74s/it]
                                                        

 80%|████████  | 9632/12000 [5:40:51<1:08:32,  1.74s/it]
 80%|████████  | 9633/12000 [5:40:53<1:06:58,  1.70s/it]
                                                        

 80%|████████  | 9633/12000 [5:40:53<1:06:58,  1.70s/it]
 80%|████████  | 9634/12000 [5:40:54<1:06:09,  1.68s/it]
                                                        

 80%|████████  | 9634/12000 [5:40:54<1:06:09,  1.68s/it]
 80%|████████  | 9635/12000 [5:40:56<1:05:08,  1.65s/it]
                                                        

 80%|████████  | 9635/12000 [5:40:56<1:05:08,  1.65s/it]
 80%|████████  | 9636/12000 [5:40:59<1:16:09,  1.93s/it]
                                                        

 80%|████████  | 9636/12000 [5:40:59<1:16:09,  1.93s/it]
 80%|████████  | 9637/12000 [5:41:00<1:12:19,  1.84s/it]
                                                        

 80%|████████  | 9637/12000 [5:41:00<1:12:19,  1.84s/it]
 80%|████████  | 9638/12000 [5:41:02<1:08:56,  1.75s/it]
                                                        

 80%|████████  | 9638/12000 [5:41:02<1:08:56,  1.75s/it]
 80%|████████  | 9639/12000 [5:41:04<1:12:15,  1.84s/it]
                                                        
{'loss': '0.4392', 'grad_norm': '0.3172', 'learning_rate': '2.124e-05', 'entropy': '0.5199', 'num_tokens': '5.168e+07', 'mean_token_accuracy': '0.8584', 'epoch': '0.08489'}
{'loss': '0.5136', 'grad_norm': '0.2954', 'learning_rate': '2.122e-05', 'entropy': '0.5274', 'num_tokens': '5.168e+07', 'mean_token_accuracy': '0.8621', 'epoch': '0.0849'}
{'loss': '0.6377', 'grad_norm': '0.4337', 'learning_rate': '2.12e-05', 'entropy': '0.7359', 'num_tokens': '5.169e+07', 'mean_token_accuracy': '0.8107', 'epoch': '0.08491'}
{'loss': '1.089', 'grad_norm': '0.3956', 'learning_rate': '2.119e-05', 'entropy': '1.178', 'num_tokens': '5.169e+07', 'mean_token_accuracy': '0.7225', 'epoch': '0.08491'}
{'loss': '0.6594', 'grad_norm': '0.4188', 'learning_rate': '2.117e-05', 'entropy': '0.6982', 'num_tokens': '5.17e+07', 'mean_token_accuracy': '0.8103', 'epoch': '0.08492'}
{'loss': '0.9083', 'grad_norm': '0.3921', 'learning_rate': '2.115e-05', 'entropy': '0.859', 'num_tokens': '5.17e+07', 'mean_token_accuracy': '0.784', 'epoch': '0.08493'}
{'loss': '0.9337', 'grad_norm': '0.3955', 'learning_rate': '2.114e-05', 'entropy': '0.9169', 'num_tokens': '5.171e+07', 'mean_token_accuracy': '0.7791', 'epoch': '0.08494'}
{'loss': '0.4809', 'grad_norm': '0.3366', 'learning_rate': '2.112e-05', 'entropy': '0.4535', 'num_tokens': '5.171e+07', 'mean_token_accuracy': '0.8788', 'epoch': '0.08495'}
{'loss': '0.567', 'grad_norm': '0.3827', 'learning_rate': '2.11e-05', 'entropy': '0.6219', 'num_tokens': '5.172e+07', 'mean_token_accuracy': '0.8327', 'epoch': '0.08496'}
{'loss': '0.8877', 'grad_norm': '0.3577', 'learning_rate': '2.109e-05', 'entropy': '0.9642', 'num_tokens': '5.173e+07', 'mean_token_accuracy': '0.7727', 'epoch': '0.08497'}
{'loss': '1.065', 'grad_norm': '0.4663', 'learning_rate': '2.107e-05', 'entropy': '0.9988', 'num_tokens': '5.173e+07', 'mean_token_accuracy': '0.7595', 'epoch': '0.08498'}
{'loss': '0.8994', 'grad_norm': '0.3275', 'learning_rate': '2.105e-05', 'entropy': '0.9117', 'num_tokens': '5.174e+07', 'mean_token_accuracy': '0.7822', 'epoch': '0.08498'}
{'loss': '0.7765', 'grad_norm': '0.4', 'learning_rate': '2.104e-05', 'entropy': '0.7772', 'num_tokens': '5.175e+07', 'mean_token_accuracy': '0.8072', 'epoch': '0.08499'}
{'loss': '0.6668', 'grad_norm': '0.4035', 'learning_rate': '2.102e-05', 'entropy': '0.6903', 'num_tokens': '5.175e+07', 'mean_token_accuracy': '0.8238', 'epoch': '0.085'}
{'loss': '0.5958', 'grad_norm': '0.2968', 'learning_rate': '2.1e-05', 'entropy': '0.7025', 'num_tokens': '5.176e+07', 'mean_token_accuracy': '0.8113', 'epoch': '0.08501'}
{'loss': '1.29', 'grad_norm': '0.5146', 'learning_rate': '2.098e-05', 'entropy': '1.222', 'num_tokens': '5.176e+07', 'mean_token_accuracy': '0.7137', 'epoch': '0.08502'}
{'loss': '0.7312', 'grad_norm': '0.3959', 'learning_rate': '2.097e-05', 'entropy': '0.8568', 'num_tokens': '5.176e+07', 'mean_token_accuracy': '0.7683', 'epoch': '0.08503'}
{'loss': '0.856', 'grad_norm': '0.5023', 'learning_rate': '2.095e-05', 'entropy': '0.9373', 'num_tokens': '5.177e+07', 'mean_token_accuracy': '0.7631', 'epoch': '0.08504'}
{'loss': '0.8222', 'grad_norm': '0.417', 'learning_rate': '2.093e-05', 'entropy': '0.8392', 'num_tokens': '5.177e+07', 'mean_token_accuracy': '0.7898', 'epoch': '0.08505'}
{'loss': '0.8167', 'grad_norm': '0.496', 'learning_rate': '2.092e-05', 'entropy': '0.8076', 'num_tokens': '5.178e+07', 'mean_token_accuracy': '0.7829', 'epoch': '0.08506'}
{'loss': '0.9783', 'grad_norm': '0.4581', 'learning_rate': '2.09e-05', 'entropy': '0.9938', 'num_tokens': '5.178e+07', 'mean_token_accuracy': '0.7605', 'epoch': '0.08506'}
{'loss': '0.7214', 'grad_norm': '0.3802', 'learning_rate': '2.088e-05', 'entropy': '0.7223', 'num_tokens': '5.179e+07', 'mean_token_accuracy': '0.8133', 'epoch': '0.08507'}
{'loss': '0.8642', 'grad_norm': '0.3961', 'learning_rate': '2.087e-05', 'entropy': '0.8121', 'num_tokens': '5.18e+07', 'mean_token_accuracy': '0.7942', 'epoch': '0.08508'}
{'loss': '0.8578', 'grad_norm': '0.4148', 'learning_rate': '2.085e-05', 'entropy': '0.8651', 'num_tokens': '5.18e+07', 'mean_token_accuracy': '0.7699', 'epoch': '0.08509'}
{'loss': '0.6773', 'grad_norm': '0.385', 'learning_rate': '2.083e-05', 'entropy': '0.6751', 'num_tokens': '5.181e+07', 'mean_token_accuracy': '0.8246', 'epoch': '0.0851'}
{'loss': '0.8122', 'grad_norm': '0.3957', 'learning_rate': '2.082e-05', 'entropy': '0.8334', 'num_tokens': '5.181e+07', 'mean_token_accuracy': '0.7828', 'epoch': '0.08511'}
{'loss': '0.7823', 'grad_norm': '0.3618', 'learning_rate': '2.08e-05', 'entropy': '0.7576', 'num_tokens': '5.182e+07', 'mean_token_accuracy': '0.8082', 'epoch': '0.08512'}
{'loss': '0.9012', 'grad_norm': '0.3536', 'learning_rate': '2.078e-05', 'entropy': '0.8946', 'num_tokens': '5.183e+07', 'mean_token_accuracy': '0.7819', 'epoch': '0.08513'}
{'loss': '0.3725', 'grad_norm': '0.2422', 'learning_rate': '2.077e-05', 'entropy': '0.4754', 'num_tokens': '5.183e+07', 'mean_token_accuracy': '0.8799', 'epoch': '0.08514'}
{'loss': '0.7195', 'grad_norm': '0.3297', 'learning_rate': '2.075e-05', 'entropy': '0.7634', 'num_tokens': '5.184e+07', 'mean_token_accuracy': '0.8149', 'epoch': '0.08514'}
{'loss': '0.8206', 'grad_norm': '0.3762', 'learning_rate': '2.073e-05', 'entropy': '0.7886', 'num_tokens': '5.185e+07', 'mean_token_accuracy': '0.802', 'epoch': '0.08515'}
{'loss': '0.8016', 'grad_norm': '0.46', 'learning_rate': '2.072e-05', 'entropy': '0.8643', 'num_tokens': '5.185e+07', 'mean_token_accuracy': '0.7846', 'epoch': '0.08516'}
{'loss': '0.5278', 'grad_norm': '0.3594', 'learning_rate': '2.07e-05', 'entropy': '0.5046', 'num_tokens': '5.185e+07', 'mean_token_accuracy': '0.8694', 'epoch': '0.08517'}
{'loss': '1.095', 'grad_norm': '0.381', 'learning_rate': '2.068e-05', 'entropy': '1.081', 'num_tokens': '5.186e+07', 'mean_token_accuracy': '0.7429', 'epoch': '0.08518'}
{'loss': '0.6113', 'grad_norm': '0.3106', 'learning_rate': '2.066e-05', 'entropy': '0.614', 'num_tokens': '5.186e+07', 'mean_token_accuracy': '0.8412', 'epoch': '0.08519'}
{'loss': '0.7692', 'grad_norm': '0.3803', 'learning_rate': '2.065e-05', 'entropy': '0.6681', 'num_tokens': '5.187e+07', 'mean_token_accuracy': '0.8207', 'epoch': '0.0852'}
{'loss': '1.032', 'grad_norm': '0.4527', 'learning_rate': '2.063e-05', 'entropy': '1.001', 'num_tokens': '5.187e+07', 'mean_token_accuracy': '0.7605', 'epoch': '0.08521'}
{'loss': '1.028', 'grad_norm': '0.4673', 'learning_rate': '2.061e-05', 'entropy': '1.021', 'num_tokens': '5.188e+07', 'mean_token_accuracy': '0.7603', 'epoch': '0.08522'}
{'loss': '0.9832', 'grad_norm': '0.4446', 'learning_rate': '2.06e-05', 'entropy': '0.9909', 'num_tokens': '5.188e+07', 'mean_token_accuracy': '0.7648', 'epoch': '0.08522'}
{'loss': '0.8674', 'grad_norm': '0.4491', 'learning_rate': '2.058e-05', 'entropy': '0.9308', 'num_tokens': '5.189e+07', 'mean_token_accuracy': '0.7659', 'epoch': '0.08523'}
{'loss': '0.8011', 'grad_norm': '0.4001', 'learning_rate': '2.056e-05', 'entropy': '0.8183', 'num_tokens': '5.189e+07', 'mean_token_accuracy': '0.7926', 'epoch': '0.08524'}
{'loss': '0.5619', 'grad_norm': '0.2952', 'learning_rate': '2.055e-05', 'entropy': '0.5736', 'num_tokens': '5.19e+07', 'mean_token_accuracy': '0.8552', 'epoch': '0.08525'}
{'loss': '0.8801', 'grad_norm': '0.4489', 'learning_rate': '2.053e-05', 'entropy': '0.8805', 'num_tokens': '5.19e+07', 'mean_token_accuracy': '0.7833', 'epoch': '0.08526'}
{'loss': '0.7657', 'grad_norm': '0.3769', 'learning_rate': '2.051e-05', 'entropy': '0.8122', 'num_tokens': '5.191e+07', 'mean_token_accuracy': '0.8005', 'epoch': '0.08527'}
{'loss': '0.9396', 'grad_norm': '0.5343', 'learning_rate': '2.05e-05', 'entropy': '0.8733', 'num_tokens': '5.191e+07', 'mean_token_accuracy': '0.7765', 'epoch': '0.08528'}
{'loss': '1.259', 'grad_norm': '0.583', 'learning_rate': '2.048e-05', 'entropy': '1.161', 'num_tokens': '5.192e+07', 'mean_token_accuracy': '0.713', 'epoch': '0.08529'}
{'loss': '0.7173', 'grad_norm': '0.3381', 'learning_rate': '2.046e-05', 'entropy': '0.7288', 'num_tokens': '5.192e+07', 'mean_token_accuracy': '0.8117', 'epoch': '0.08529'}

 80%|████████  | 9639/12000 [5:41:04<1:12:15,  1.84s/it]
 80%|████████  | 9640/12000 [5:41:05<1:10:57,  1.80s/it]
                                                        

 80%|████████  | 9640/12000 [5:41:05<1:10:57,  1.80s/it]
 80%|████████  | 9641/12000 [5:41:10<1:47:58,  2.75s/it]
                                                        

 80%|████████  | 9641/12000 [5:41:10<1:47:58,  2.75s/it]
 80%|████████  | 9642/12000 [5:41:13<1:42:05,  2.60s/it]
                                                        

 80%|████████  | 9642/12000 [5:41:13<1:42:05,  2.60s/it]
 80%|████████  | 9643/12000 [5:41:14<1:29:17,  2.27s/it]
                                                        

 80%|████████  | 9643/12000 [5:41:14<1:29:17,  2.27s/it]
 80%|████████  | 9644/12000 [5:41:18<1:49:57,  2.80s/it]
                                                        

 80%|████████  | 9644/12000 [5:41:18<1:49:57,  2.80s/it]
 80%|████████  | 9645/12000 [5:41:21<1:45:14,  2.68s/it]
                                                        

 80%|████████  | 9645/12000 [5:41:21<1:45:14,  2.68s/it]
 80%|████████  | 9646/12000 [5:41:23<1:43:13,  2.63s/it]
                                                        

 80%|████████  | 9646/12000 [5:41:23<1:43:13,  2.63s/it]
 80%|████████  | 9647/12000 [5:41:25<1:30:44,  2.31s/it]
                                                        

 80%|████████  | 9647/12000 [5:41:25<1:30:44,  2.31s/it]
 80%|████████  | 9648/12000 [5:41:26<1:24:50,  2.16s/it]
                                                        

 80%|████████  | 9648/12000 [5:41:26<1:24:50,  2.16s/it]
 80%|████████  | 9649/12000 [5:41:29<1:27:05,  2.22s/it]
                                                        

 80%|████████  | 9649/12000 [5:41:29<1:27:05,  2.22s/it]
 80%|████████  | 9650/12000 [5:41:32<1:38:32,  2.52s/it]
                                                        

 80%|████████  | 9650/12000 [5:41:32<1:38:32,  2.52s/it]
 80%|████████  | 9651/12000 [5:41:35<1:46:52,  2.73s/it]
                                                        

 80%|████████  | 9651/12000 [5:41:35<1:46:52,  2.73s/it]
 80%|████████  | 9652/12000 [5:41:37<1:36:19,  2.46s/it]
                                                        

 80%|████████  | 9652/12000 [5:41:37<1:36:19,  2.46s/it]
 80%|████████  | 9653/12000 [5:41:39<1:25:11,  2.18s/it]
                                                        

 80%|████████  | 9653/12000 [5:41:39<1:25:11,  2.18s/it]
 80%|████████  | 9654/12000 [5:41:40<1:18:26,  2.01s/it]
                                                        

 80%|████████  | 9654/12000 [5:41:40<1:18:26,  2.01s/it]
 80%|████████  | 9655/12000 [5:41:42<1:14:20,  1.90s/it]
                                                        

 80%|████████  | 9655/12000 [5:41:42<1:14:20,  1.90s/it]
 80%|████████  | 9656/12000 [5:41:43<1:10:22,  1.80s/it]
                                                        

 80%|████████  | 9656/12000 [5:41:43<1:10:22,  1.80s/it]
 80%|████████  | 9657/12000 [5:41:45<1:08:48,  1.76s/it]
                                                        

 80%|████████  | 9657/12000 [5:41:45<1:08:48,  1.76s/it]
 80%|████████  | 9658/12000 [5:41:47<1:09:00,  1.77s/it]
                                                        

 80%|████████  | 9658/12000 [5:41:47<1:09:00,  1.77s/it]
 80%|████████  | 9659/12000 [5:41:50<1:20:24,  2.06s/it]
                                                        

 80%|████████  | 9659/12000 [5:41:50<1:20:24,  2.06s/it]
 80%|████████  | 9660/12000 [5:41:51<1:17:20,  1.98s/it]
                                                        

 80%|████████  | 9660/12000 [5:41:51<1:17:20,  1.98s/it]
 81%|████████  | 9661/12000 [5:41:53<1:14:16,  1.91s/it]
                                                        

 81%|████████  | 9661/12000 [5:41:53<1:14:16,  1.91s/it]
 81%|████████  | 9662/12000 [5:41:55<1:11:08,  1.83s/it]
                                                        

 81%|████████  | 9662/12000 [5:41:55<1:11:08,  1.83s/it]
 81%|████████  | 9663/12000 [5:41:57<1:09:43,  1.79s/it]
                                                        

 81%|████████  | 9663/12000 [5:41:57<1:09:43,  1.79s/it]
 81%|████████  | 9664/12000 [5:41:59<1:12:18,  1.86s/it]
                                                        

 81%|████████  | 9664/12000 [5:41:59<1:12:18,  1.86s/it]
 81%|████████  | 9665/12000 [5:42:01<1:18:43,  2.02s/it]
                                                        

 81%|████████  | 9665/12000 [5:42:01<1:18:43,  2.02s/it]
 81%|████████  | 9666/12000 [5:42:05<1:37:10,  2.50s/it]
                                                        

 81%|████████  | 9666/12000 [5:42:05<1:37:10,  2.50s/it]
 81%|████████  | 9667/12000 [5:42:06<1:27:26,  2.25s/it]
                                                        

 81%|████████  | 9667/12000 [5:42:06<1:27:26,  2.25s/it]
 81%|████████  | 9668/12000 [5:42:08<1:19:10,  2.04s/it]
                                                        

 81%|████████  | 9668/12000 [5:42:08<1:19:10,  2.04s/it]
 81%|████████  | 9669/12000 [5:42:09<1:13:54,  1.90s/it]
                                                        

 81%|████████  | 9669/12000 [5:42:09<1:13:54,  1.90s/it]
 81%|████████  | 9670/12000 [5:42:12<1:20:45,  2.08s/it]
                                                        

 81%|████████  | 9670/12000 [5:42:12<1:20:45,  2.08s/it]
 81%|████████  | 9671/12000 [5:42:15<1:34:33,  2.44s/it]
                                                        

 81%|████████  | 9671/12000 [5:42:15<1:34:33,  2.44s/it]
 81%|████████  | 9672/12000 [5:42:17<1:26:29,  2.23s/it]
                                                        

 81%|████████  | 9672/12000 [5:42:17<1:26:29,  2.23s/it]
 81%|████████  | 9673/12000 [5:42:22<1:59:06,  3.07s/it]
                                                        

 81%|████████  | 9673/12000 [5:42:22<1:59:06,  3.07s/it]
 81%|████████  | 9674/12000 [5:42:23<1:41:43,  2.62s/it]
                                                        

 81%|████████  | 9674/12000 [5:42:23<1:41:43,  2.62s/it]
 81%|████████  | 9675/12000 [5:42:25<1:30:20,  2.33s/it]
                                                        

 81%|████████  | 9675/12000 [5:42:25<1:30:20,  2.33s/it]
 81%|████████  | 9676/12000 [5:42:27<1:23:12,  2.15s/it]
                                                        

 81%|████████  | 9676/12000 [5:42:27<1:23:12,  2.15s/it]
 81%|████████  | 9677/12000 [5:42:29<1:20:54,  2.09s/it]
                                                        

 81%|████████  | 9677/12000 [5:42:29<1:20:54,  2.09s/it]
 81%|████████  | 9678/12000 [5:42:31<1:16:48,  1.98s/it]
                                                        

 81%|████████  | 9678/12000 [5:42:31<1:16:48,  1.98s/it]
 81%|████████  | 9679/12000 [5:42:32<1:11:01,  1.84s/it]
                                                        

 81%|████████  | 9679/12000 [5:42:32<1:11:01,  1.84s/it]
 81%|████████  | 9680/12000 [5:42:34<1:08:52,  1.78s/it]
                                                        

 81%|████████  | 9680/12000 [5:42:34<1:08:52,  1.78s/it]
 81%|████████  | 9681/12000 [5:42:36<1:19:06,  2.05s/it]
                                                        

 81%|████████  | 9681/12000 [5:42:36<1:19:06,  2.05s/it]
 81%|████████  | 9682/12000 [5:42:38<1:14:54,  1.94s/it]
                                                        

 81%|████████  | 9682/12000 [5:42:38<1:14:54,  1.94s/it]
 81%|████████  | 9683/12000 [5:42:40<1:11:01,  1.84s/it]
                                                        

 81%|████████  | 9683/12000 [5:42:40<1:11:01,  1.84s/it]
 81%|████████  | 9684/12000 [5:42:42<1:12:54,  1.89s/it]
                                                        

 81%|████████  | 9684/12000 [5:42:42<1:12:54,  1.89s/it]
 81%|████████  | 9685/12000 [5:42:44<1:14:32,  1.93s/it]
                                                        

 81%|████████  | 9685/12000 [5:42:44<1:14:32,  1.93s/it]
 81%|████████  | 9686/12000 [5:42:46<1:15:56,  1.97s/it]
                                                        
{'loss': '0.9689', 'grad_norm': '0.5827', 'learning_rate': '2.045e-05', 'entropy': '0.9197', 'num_tokens': '5.193e+07', 'mean_token_accuracy': '0.7691', 'epoch': '0.0853'}
{'loss': '0.6414', 'grad_norm': '0.4459', 'learning_rate': '2.043e-05', 'entropy': '0.7154', 'num_tokens': '5.193e+07', 'mean_token_accuracy': '0.8085', 'epoch': '0.08531'}
{'loss': '1.16', 'grad_norm': '0.3806', 'learning_rate': '2.041e-05', 'entropy': '1.094', 'num_tokens': '5.194e+07', 'mean_token_accuracy': '0.7624', 'epoch': '0.08532'}
{'loss': '0.8383', 'grad_norm': '0.3934', 'learning_rate': '2.04e-05', 'entropy': '0.9143', 'num_tokens': '5.195e+07', 'mean_token_accuracy': '0.7801', 'epoch': '0.08533'}
{'loss': '0.8515', 'grad_norm': '0.3624', 'learning_rate': '2.038e-05', 'entropy': '0.8843', 'num_tokens': '5.195e+07', 'mean_token_accuracy': '0.7709', 'epoch': '0.08534'}
{'loss': '0.63', 'grad_norm': '0.3682', 'learning_rate': '2.036e-05', 'entropy': '0.5736', 'num_tokens': '5.196e+07', 'mean_token_accuracy': '0.8589', 'epoch': '0.08535'}
{'loss': '0.5379', 'grad_norm': '0.2418', 'learning_rate': '2.035e-05', 'entropy': '0.5544', 'num_tokens': '5.196e+07', 'mean_token_accuracy': '0.8623', 'epoch': '0.08536'}
{'loss': '0.9831', 'grad_norm': '0.4108', 'learning_rate': '2.033e-05', 'entropy': '0.9759', 'num_tokens': '5.197e+07', 'mean_token_accuracy': '0.7668', 'epoch': '0.08537'}
{'loss': '0.6036', 'grad_norm': '0.3537', 'learning_rate': '2.031e-05', 'entropy': '0.6565', 'num_tokens': '5.198e+07', 'mean_token_accuracy': '0.833', 'epoch': '0.08537'}
{'loss': '0.9698', 'grad_norm': '0.433', 'learning_rate': '2.03e-05', 'entropy': '0.9899', 'num_tokens': '5.198e+07', 'mean_token_accuracy': '0.7661', 'epoch': '0.08538'}
{'loss': '1.368', 'grad_norm': '0.4682', 'learning_rate': '2.028e-05', 'entropy': '1.18', 'num_tokens': '5.199e+07', 'mean_token_accuracy': '0.7322', 'epoch': '0.08539'}
{'loss': '0.9832', 'grad_norm': '0.4168', 'learning_rate': '2.026e-05', 'entropy': '1.027', 'num_tokens': '5.199e+07', 'mean_token_accuracy': '0.736', 'epoch': '0.0854'}
{'loss': '0.6717', 'grad_norm': '0.3782', 'learning_rate': '2.025e-05', 'entropy': '0.741', 'num_tokens': '5.2e+07', 'mean_token_accuracy': '0.8055', 'epoch': '0.08541'}
{'loss': '0.8055', 'grad_norm': '0.4054', 'learning_rate': '2.023e-05', 'entropy': '0.8601', 'num_tokens': '5.2e+07', 'mean_token_accuracy': '0.7828', 'epoch': '0.08542'}
{'loss': '0.8359', 'grad_norm': '0.4712', 'learning_rate': '2.021e-05', 'entropy': '0.8738', 'num_tokens': '5.201e+07', 'mean_token_accuracy': '0.7832', 'epoch': '0.08543'}
{'loss': '0.8304', 'grad_norm': '0.4319', 'learning_rate': '2.02e-05', 'entropy': '0.8251', 'num_tokens': '5.201e+07', 'mean_token_accuracy': '0.7811', 'epoch': '0.08544'}
{'loss': '0.7399', 'grad_norm': '0.3741', 'learning_rate': '2.018e-05', 'entropy': '0.7438', 'num_tokens': '5.202e+07', 'mean_token_accuracy': '0.8146', 'epoch': '0.08545'}
{'loss': '0.642', 'grad_norm': '0.3638', 'learning_rate': '2.016e-05', 'entropy': '0.7713', 'num_tokens': '5.202e+07', 'mean_token_accuracy': '0.8062', 'epoch': '0.08545'}
{'loss': '0.772', 'grad_norm': '0.3559', 'learning_rate': '2.015e-05', 'entropy': '0.7797', 'num_tokens': '5.203e+07', 'mean_token_accuracy': '0.8106', 'epoch': '0.08546'}
{'loss': '0.7799', 'grad_norm': '0.4892', 'learning_rate': '2.013e-05', 'entropy': '0.8546', 'num_tokens': '5.203e+07', 'mean_token_accuracy': '0.7833', 'epoch': '0.08547'}
{'loss': '0.68', 'grad_norm': '0.4348', 'learning_rate': '2.011e-05', 'entropy': '0.828', 'num_tokens': '5.204e+07', 'mean_token_accuracy': '0.8032', 'epoch': '0.08548'}
{'loss': '0.7631', 'grad_norm': '0.4746', 'learning_rate': '2.01e-05', 'entropy': '0.765', 'num_tokens': '5.204e+07', 'mean_token_accuracy': '0.7986', 'epoch': '0.08549'}
{'loss': '0.5626', 'grad_norm': '0.3675', 'learning_rate': '2.008e-05', 'entropy': '0.6332', 'num_tokens': '5.205e+07', 'mean_token_accuracy': '0.84', 'epoch': '0.0855'}
{'loss': '0.9899', 'grad_norm': '0.4894', 'learning_rate': '2.006e-05', 'entropy': '1.001', 'num_tokens': '5.205e+07', 'mean_token_accuracy': '0.7526', 'epoch': '0.08551'}
{'loss': '0.6344', 'grad_norm': '0.5021', 'learning_rate': '2.005e-05', 'entropy': '0.7214', 'num_tokens': '5.205e+07', 'mean_token_accuracy': '0.8125', 'epoch': '0.08552'}
{'loss': '0.9395', 'grad_norm': '0.3553', 'learning_rate': '2.003e-05', 'entropy': '0.9862', 'num_tokens': '5.206e+07', 'mean_token_accuracy': '0.7564', 'epoch': '0.08552'}
{'loss': '0.6656', 'grad_norm': '0.3863', 'learning_rate': '2.001e-05', 'entropy': '0.6931', 'num_tokens': '5.206e+07', 'mean_token_accuracy': '0.8321', 'epoch': '0.08553'}
{'loss': '0.6715', 'grad_norm': '0.3283', 'learning_rate': '2e-05', 'entropy': '0.671', 'num_tokens': '5.207e+07', 'mean_token_accuracy': '0.8229', 'epoch': '0.08554'}
{'loss': '0.9288', 'grad_norm': '0.5181', 'learning_rate': '1.998e-05', 'entropy': '0.9571', 'num_tokens': '5.207e+07', 'mean_token_accuracy': '0.7489', 'epoch': '0.08555'}
{'loss': '0.7908', 'grad_norm': '0.4033', 'learning_rate': '1.997e-05', 'entropy': '0.8299', 'num_tokens': '5.208e+07', 'mean_token_accuracy': '0.786', 'epoch': '0.08556'}
{'loss': '0.7548', 'grad_norm': '0.4111', 'learning_rate': '1.995e-05', 'entropy': '0.7753', 'num_tokens': '5.208e+07', 'mean_token_accuracy': '0.8095', 'epoch': '0.08557'}
{'loss': '0.5119', 'grad_norm': '0.3981', 'learning_rate': '1.993e-05', 'entropy': '0.5012', 'num_tokens': '5.208e+07', 'mean_token_accuracy': '0.8668', 'epoch': '0.08558'}
{'loss': '0.9225', 'grad_norm': '0.4099', 'learning_rate': '1.992e-05', 'entropy': '0.9771', 'num_tokens': '5.209e+07', 'mean_token_accuracy': '0.7469', 'epoch': '0.08559'}
{'loss': '0.9944', 'grad_norm': '0.5032', 'learning_rate': '1.99e-05', 'entropy': '0.9637', 'num_tokens': '5.21e+07', 'mean_token_accuracy': '0.7479', 'epoch': '0.0856'}
{'loss': '0.7506', 'grad_norm': '0.2961', 'learning_rate': '1.988e-05', 'entropy': '0.7789', 'num_tokens': '5.21e+07', 'mean_token_accuracy': '0.8013', 'epoch': '0.0856'}
{'loss': '0.6607', 'grad_norm': '0.3757', 'learning_rate': '1.987e-05', 'entropy': '0.6763', 'num_tokens': '5.211e+07', 'mean_token_accuracy': '0.8224', 'epoch': '0.08561'}
{'loss': '0.5484', 'grad_norm': '0.3745', 'learning_rate': '1.985e-05', 'entropy': '0.5649', 'num_tokens': '5.211e+07', 'mean_token_accuracy': '0.848', 'epoch': '0.08562'}
{'loss': '0.6661', 'grad_norm': '0.3635', 'learning_rate': '1.983e-05', 'entropy': '0.7698', 'num_tokens': '5.212e+07', 'mean_token_accuracy': '0.804', 'epoch': '0.08563'}
{'loss': '0.7898', 'grad_norm': '0.3536', 'learning_rate': '1.982e-05', 'entropy': '0.7749', 'num_tokens': '5.213e+07', 'mean_token_accuracy': '0.8135', 'epoch': '0.08564'}
{'loss': '0.6189', 'grad_norm': '0.3852', 'learning_rate': '1.98e-05', 'entropy': '0.6527', 'num_tokens': '5.213e+07', 'mean_token_accuracy': '0.8321', 'epoch': '0.08565'}
{'loss': '1.06', 'grad_norm': '0.5071', 'learning_rate': '1.978e-05', 'entropy': '1.028', 'num_tokens': '5.213e+07', 'mean_token_accuracy': '0.7551', 'epoch': '0.08566'}
{'loss': '0.668', 'grad_norm': '0.4177', 'learning_rate': '1.977e-05', 'entropy': '0.6931', 'num_tokens': '5.214e+07', 'mean_token_accuracy': '0.8118', 'epoch': '0.08567'}
{'loss': '0.9316', 'grad_norm': '0.4279', 'learning_rate': '1.975e-05', 'entropy': '0.9861', 'num_tokens': '5.215e+07', 'mean_token_accuracy': '0.7622', 'epoch': '0.08568'}
{'loss': '0.6891', 'grad_norm': '0.429', 'learning_rate': '1.973e-05', 'entropy': '0.7252', 'num_tokens': '5.215e+07', 'mean_token_accuracy': '0.801', 'epoch': '0.08568'}
{'loss': '0.5452', 'grad_norm': '0.3471', 'learning_rate': '1.972e-05', 'entropy': '0.5574', 'num_tokens': '5.215e+07', 'mean_token_accuracy': '0.8455', 'epoch': '0.08569'}
{'loss': '0.7742', 'grad_norm': '0.3514', 'learning_rate': '1.97e-05', 'entropy': '0.7346', 'num_tokens': '5.216e+07', 'mean_token_accuracy': '0.8077', 'epoch': '0.0857'}
{'loss': '0.6109', 'grad_norm': '0.3618', 'learning_rate': '1.969e-05', 'entropy': '0.7078', 'num_tokens': '5.217e+07', 'mean_token_accuracy': '0.8196', 'epoch': '0.08571'}

 81%|████████  | 9686/12000 [5:42:46<1:15:56,  1.97s/it]
 81%|████████  | 9687/12000 [5:42:47<1:09:58,  1.82s/it]
                                                        

 81%|████████  | 9687/12000 [5:42:47<1:09:58,  1.82s/it]
 81%|████████  | 9688/12000 [5:42:52<1:38:46,  2.56s/it]
                                                        

 81%|████████  | 9688/12000 [5:42:52<1:38:46,  2.56s/it]
 81%|████████  | 9689/12000 [5:42:53<1:27:23,  2.27s/it]
                                                        

 81%|████████  | 9689/12000 [5:42:53<1:27:23,  2.27s/it]
 81%|████████  | 9690/12000 [5:42:55<1:21:19,  2.11s/it]
                                                        

 81%|████████  | 9690/12000 [5:42:55<1:21:19,  2.11s/it]
 81%|████████  | 9691/12000 [5:42:57<1:17:32,  2.01s/it]
                                                        

 81%|████████  | 9691/12000 [5:42:57<1:17:32,  2.01s/it]
 81%|████████  | 9692/12000 [5:42:58<1:13:58,  1.92s/it]
                                                        

 81%|████████  | 9692/12000 [5:42:58<1:13:58,  1.92s/it]
 81%|████████  | 9693/12000 [5:43:00<1:12:32,  1.89s/it]
                                                        

 81%|████████  | 9693/12000 [5:43:00<1:12:32,  1.89s/it]
 81%|████████  | 9694/12000 [5:43:03<1:18:25,  2.04s/it]
                                                        

 81%|████████  | 9694/12000 [5:43:03<1:18:25,  2.04s/it]
 81%|████████  | 9695/12000 [5:43:05<1:25:12,  2.22s/it]
                                                        

 81%|████████  | 9695/12000 [5:43:05<1:25:12,  2.22s/it]
 81%|████████  | 9696/12000 [5:43:07<1:16:44,  2.00s/it]
                                                        

 81%|████████  | 9696/12000 [5:43:07<1:16:44,  2.00s/it]
 81%|████████  | 9697/12000 [5:43:11<1:40:38,  2.62s/it]
                                                        

 81%|████████  | 9697/12000 [5:43:11<1:40:38,  2.62s/it]
 81%|████████  | 9698/12000 [5:43:12<1:27:32,  2.28s/it]
                                                        

 81%|████████  | 9698/12000 [5:43:12<1:27:32,  2.28s/it]
 81%|████████  | 9699/12000 [5:43:14<1:18:23,  2.04s/it]
                                                        

 81%|████████  | 9699/12000 [5:43:14<1:18:23,  2.04s/it]
 81%|████████  | 9700/12000 [5:43:15<1:14:58,  1.96s/it]
                                                        

 81%|████████  | 9700/12000 [5:43:15<1:14:58,  1.96s/it]
 81%|████████  | 9701/12000 [5:43:18<1:26:54,  2.27s/it]
                                                        

 81%|████████  | 9701/12000 [5:43:18<1:26:54,  2.27s/it]
 81%|████████  | 9702/12000 [5:43:21<1:27:49,  2.29s/it]
                                                        

 81%|████████  | 9702/12000 [5:43:21<1:27:49,  2.29s/it]
 81%|████████  | 9703/12000 [5:43:23<1:22:24,  2.15s/it]
                                                        

 81%|████████  | 9703/12000 [5:43:23<1:22:24,  2.15s/it]
 81%|████████  | 9704/12000 [5:43:25<1:19:51,  2.09s/it]
                                                        

 81%|████████  | 9704/12000 [5:43:25<1:19:51,  2.09s/it]
 81%|████████  | 9705/12000 [5:43:27<1:23:28,  2.18s/it]
                                                        

 81%|████████  | 9705/12000 [5:43:27<1:23:28,  2.18s/it]
 81%|████████  | 9706/12000 [5:43:29<1:21:18,  2.13s/it]
                                                        

 81%|████████  | 9706/12000 [5:43:29<1:21:18,  2.13s/it]
 81%|████████  | 9707/12000 [5:43:31<1:14:40,  1.95s/it]
                                                        

 81%|████████  | 9707/12000 [5:43:31<1:14:40,  1.95s/it]
 81%|████████  | 9708/12000 [5:43:32<1:14:37,  1.95s/it]
                                                        

 81%|████████  | 9708/12000 [5:43:32<1:14:37,  1.95s/it]
 81%|████████  | 9709/12000 [5:43:34<1:11:15,  1.87s/it]
                                                        

 81%|████████  | 9709/12000 [5:43:34<1:11:15,  1.87s/it]
 81%|████████  | 9710/12000 [5:43:38<1:29:58,  2.36s/it]
                                                        

 81%|████████  | 9710/12000 [5:43:38<1:29:58,  2.36s/it]
 81%|████████  | 9711/12000 [5:43:40<1:25:05,  2.23s/it]
                                                        

 81%|████████  | 9711/12000 [5:43:40<1:25:05,  2.23s/it]
 81%|████████  | 9712/12000 [5:43:44<1:51:56,  2.94s/it]
                                                        

 81%|████████  | 9712/12000 [5:43:44<1:51:56,  2.94s/it]
 81%|████████  | 9713/12000 [5:43:47<1:49:29,  2.87s/it]
                                                        

 81%|████████  | 9713/12000 [5:43:47<1:49:29,  2.87s/it]
 81%|████████  | 9714/12000 [5:43:48<1:32:56,  2.44s/it]
                                                        

 81%|████████  | 9714/12000 [5:43:48<1:32:56,  2.44s/it]
 81%|████████  | 9715/12000 [5:43:50<1:21:10,  2.13s/it]
                                                        

 81%|████████  | 9715/12000 [5:43:50<1:21:10,  2.13s/it]
 81%|████████  | 9716/12000 [5:43:51<1:16:30,  2.01s/it]
                                                        

 81%|████████  | 9716/12000 [5:43:51<1:16:30,  2.01s/it]
 81%|████████  | 9717/12000 [5:43:53<1:15:15,  1.98s/it]
                                                        

 81%|████████  | 9717/12000 [5:43:53<1:15:15,  1.98s/it]
 81%|████████  | 9718/12000 [5:43:55<1:10:23,  1.85s/it]
                                                        

 81%|████████  | 9718/12000 [5:43:55<1:10:23,  1.85s/it]
 81%|████████  | 9719/12000 [5:43:56<1:06:52,  1.76s/it]
                                                        

 81%|████████  | 9719/12000 [5:43:56<1:06:52,  1.76s/it]
 81%|████████  | 9720/12000 [5:43:59<1:13:38,  1.94s/it]
                                                        

 81%|████████  | 9720/12000 [5:43:59<1:13:38,  1.94s/it]
 81%|████████  | 9721/12000 [5:44:00<1:07:32,  1.78s/it]
                                                        

 81%|████████  | 9721/12000 [5:44:00<1:07:32,  1.78s/it]
 81%|████████  | 9722/12000 [5:44:02<1:07:42,  1.78s/it]
                                                        

 81%|████████  | 9722/12000 [5:44:02<1:07:42,  1.78s/it]
 81%|████████  | 9723/12000 [5:44:03<1:03:36,  1.68s/it]
                                                        

 81%|████████  | 9723/12000 [5:44:03<1:03:36,  1.68s/it]
 81%|████████  | 9724/12000 [5:44:05<1:03:11,  1.67s/it]
                                                        

 81%|████████  | 9724/12000 [5:44:05<1:03:11,  1.67s/it]
 81%|████████  | 9725/12000 [5:44:07<1:03:13,  1.67s/it]
                                                        

 81%|████████  | 9725/12000 [5:44:07<1:03:13,  1.67s/it]
 81%|████████  | 9726/12000 [5:44:09<1:12:16,  1.91s/it]
                                                        

 81%|████████  | 9726/12000 [5:44:09<1:12:16,  1.91s/it]
 81%|████████  | 9727/12000 [5:44:13<1:33:46,  2.48s/it]
                                                        

 81%|████████  | 9727/12000 [5:44:13<1:33:46,  2.48s/it]
 81%|████████  | 9728/12000 [5:44:15<1:22:54,  2.19s/it]
                                                        

 81%|████████  | 9728/12000 [5:44:15<1:22:54,  2.19s/it]
 81%|████████  | 9729/12000 [5:44:17<1:24:27,  2.23s/it]
                                                        

 81%|████████  | 9729/12000 [5:44:17<1:24:27,  2.23s/it]
 81%|████████  | 9730/12000 [5:44:18<1:16:12,  2.01s/it]
                                                        

 81%|████████  | 9730/12000 [5:44:18<1:16:12,  2.01s/it]
 81%|████████  | 9731/12000 [5:44:20<1:09:28,  1.84s/it]
                                                        

 81%|████████  | 9731/12000 [5:44:20<1:09:28,  1.84s/it]
 81%|████████  | 9732/12000 [5:44:24<1:31:53,  2.43s/it]
                                                        

 81%|████████  | 9732/12000 [5:44:24<1:31:53,  2.43s/it]
 81%|████████  | 9733/12000 [5:44:25<1:22:35,  2.19s/it]
                                                        
{'loss': '0.838', 'grad_norm': '0.3859', 'learning_rate': '1.967e-05', 'entropy': '0.8011', 'num_tokens': '5.217e+07', 'mean_token_accuracy': '0.7922', 'epoch': '0.08572'}
{'loss': '0.7976', 'grad_norm': '0.3533', 'learning_rate': '1.965e-05', 'entropy': '0.8171', 'num_tokens': '5.218e+07', 'mean_token_accuracy': '0.7825', 'epoch': '0.08573'}
{'loss': '0.5226', 'grad_norm': '0.3035', 'learning_rate': '1.964e-05', 'entropy': '0.5733', 'num_tokens': '5.218e+07', 'mean_token_accuracy': '0.8493', 'epoch': '0.08574'}
{'loss': '0.914', 'grad_norm': '0.4574', 'learning_rate': '1.962e-05', 'entropy': '1.026', 'num_tokens': '5.219e+07', 'mean_token_accuracy': '0.762', 'epoch': '0.08575'}
{'loss': '0.8223', 'grad_norm': '0.4278', 'learning_rate': '1.96e-05', 'entropy': '0.8253', 'num_tokens': '5.219e+07', 'mean_token_accuracy': '0.7891', 'epoch': '0.08575'}
{'loss': '0.9933', 'grad_norm': '0.4827', 'learning_rate': '1.959e-05', 'entropy': '0.9865', 'num_tokens': '5.22e+07', 'mean_token_accuracy': '0.7525', 'epoch': '0.08576'}
{'loss': '0.5357', 'grad_norm': '0.3594', 'learning_rate': '1.957e-05', 'entropy': '0.5688', 'num_tokens': '5.22e+07', 'mean_token_accuracy': '0.8528', 'epoch': '0.08577'}
{'loss': '0.7442', 'grad_norm': '0.3607', 'learning_rate': '1.955e-05', 'entropy': '0.728', 'num_tokens': '5.221e+07', 'mean_token_accuracy': '0.8047', 'epoch': '0.08578'}
{'loss': '0.9813', 'grad_norm': '0.401', 'learning_rate': '1.954e-05', 'entropy': '1.094', 'num_tokens': '5.221e+07', 'mean_token_accuracy': '0.7351', 'epoch': '0.08579'}
{'loss': '0.6541', 'grad_norm': '0.4001', 'learning_rate': '1.952e-05', 'entropy': '0.6203', 'num_tokens': '5.222e+07', 'mean_token_accuracy': '0.8379', 'epoch': '0.0858'}
{'loss': '0.7672', 'grad_norm': '0.3606', 'learning_rate': '1.951e-05', 'entropy': '0.8033', 'num_tokens': '5.222e+07', 'mean_token_accuracy': '0.7918', 'epoch': '0.08581'}
{'loss': '0.8865', 'grad_norm': '0.3519', 'learning_rate': '1.949e-05', 'entropy': '0.8956', 'num_tokens': '5.223e+07', 'mean_token_accuracy': '0.7809', 'epoch': '0.08582'}
{'loss': '1.008', 'grad_norm': '0.5176', 'learning_rate': '1.947e-05', 'entropy': '0.9682', 'num_tokens': '5.224e+07', 'mean_token_accuracy': '0.7534', 'epoch': '0.08583'}
{'loss': '0.5697', 'grad_norm': '0.3222', 'learning_rate': '1.946e-05', 'entropy': '0.6061', 'num_tokens': '5.224e+07', 'mean_token_accuracy': '0.8419', 'epoch': '0.08583'}
{'loss': '1.139', 'grad_norm': '0.4483', 'learning_rate': '1.944e-05', 'entropy': '1.126', 'num_tokens': '5.225e+07', 'mean_token_accuracy': '0.7187', 'epoch': '0.08584'}
{'loss': '0.5957', 'grad_norm': '0.358', 'learning_rate': '1.942e-05', 'entropy': '0.6418', 'num_tokens': '5.225e+07', 'mean_token_accuracy': '0.8358', 'epoch': '0.08585'}
{'loss': '0.8448', 'grad_norm': '0.3993', 'learning_rate': '1.941e-05', 'entropy': '0.8466', 'num_tokens': '5.226e+07', 'mean_token_accuracy': '0.7896', 'epoch': '0.08586'}
{'loss': '0.7358', 'grad_norm': '0.4695', 'learning_rate': '1.939e-05', 'entropy': '0.7385', 'num_tokens': '5.226e+07', 'mean_token_accuracy': '0.8001', 'epoch': '0.08587'}
{'loss': '0.6547', 'grad_norm': '0.3724', 'learning_rate': '1.937e-05', 'entropy': '0.6867', 'num_tokens': '5.227e+07', 'mean_token_accuracy': '0.8163', 'epoch': '0.08588'}
{'loss': '0.6858', 'grad_norm': '0.42', 'learning_rate': '1.936e-05', 'entropy': '0.8396', 'num_tokens': '5.227e+07', 'mean_token_accuracy': '0.793', 'epoch': '0.08589'}
{'loss': '0.8732', 'grad_norm': '0.475', 'learning_rate': '1.934e-05', 'entropy': '0.8626', 'num_tokens': '5.228e+07', 'mean_token_accuracy': '0.7918', 'epoch': '0.0859'}
{'loss': '0.8426', 'grad_norm': '0.3883', 'learning_rate': '1.933e-05', 'entropy': '0.8629', 'num_tokens': '5.228e+07', 'mean_token_accuracy': '0.775', 'epoch': '0.08591'}
{'loss': '0.7978', 'grad_norm': '0.3884', 'learning_rate': '1.931e-05', 'entropy': '0.8409', 'num_tokens': '5.229e+07', 'mean_token_accuracy': '0.7769', 'epoch': '0.08591'}
{'loss': '0.8126', 'grad_norm': '0.4268', 'learning_rate': '1.929e-05', 'entropy': '0.8996', 'num_tokens': '5.229e+07', 'mean_token_accuracy': '0.7735', 'epoch': '0.08592'}
{'loss': '0.7099', 'grad_norm': '0.4325', 'learning_rate': '1.928e-05', 'entropy': '0.7338', 'num_tokens': '5.23e+07', 'mean_token_accuracy': '0.811', 'epoch': '0.08593'}
{'loss': '1.046', 'grad_norm': '0.5435', 'learning_rate': '1.926e-05', 'entropy': '1.051', 'num_tokens': '5.23e+07', 'mean_token_accuracy': '0.7397', 'epoch': '0.08594'}
{'loss': '1.216', 'grad_norm': '0.3717', 'learning_rate': '1.924e-05', 'entropy': '1.139', 'num_tokens': '5.231e+07', 'mean_token_accuracy': '0.7205', 'epoch': '0.08595'}
{'loss': '0.5242', 'grad_norm': '0.4015', 'learning_rate': '1.923e-05', 'entropy': '0.5471', 'num_tokens': '5.232e+07', 'mean_token_accuracy': '0.8598', 'epoch': '0.08596'}
{'loss': '0.7994', 'grad_norm': '0.3565', 'learning_rate': '1.921e-05', 'entropy': '0.8125', 'num_tokens': '5.232e+07', 'mean_token_accuracy': '0.7922', 'epoch': '0.08597'}
{'loss': '0.817', 'grad_norm': '0.4844', 'learning_rate': '1.92e-05', 'entropy': '0.8102', 'num_tokens': '5.232e+07', 'mean_token_accuracy': '0.7881', 'epoch': '0.08598'}
{'loss': '0.9478', 'grad_norm': '0.4085', 'learning_rate': '1.918e-05', 'entropy': '0.948', 'num_tokens': '5.233e+07', 'mean_token_accuracy': '0.7572', 'epoch': '0.08598'}
{'loss': '0.8731', 'grad_norm': '0.4665', 'learning_rate': '1.916e-05', 'entropy': '0.8937', 'num_tokens': '5.233e+07', 'mean_token_accuracy': '0.7775', 'epoch': '0.08599'}
{'loss': '0.8447', 'grad_norm': '0.3739', 'learning_rate': '1.915e-05', 'entropy': '0.9244', 'num_tokens': '5.234e+07', 'mean_token_accuracy': '0.7707', 'epoch': '0.086'}
{'loss': '0.7769', 'grad_norm': '0.378', 'learning_rate': '1.913e-05', 'entropy': '0.8277', 'num_tokens': '5.234e+07', 'mean_token_accuracy': '0.7907', 'epoch': '0.08601'}
{'loss': '0.7509', 'grad_norm': '0.3313', 'learning_rate': '1.911e-05', 'entropy': '0.7608', 'num_tokens': '5.235e+07', 'mean_token_accuracy': '0.8027', 'epoch': '0.08602'}
{'loss': '0.4291', 'grad_norm': '0.3504', 'learning_rate': '1.91e-05', 'entropy': '0.4626', 'num_tokens': '5.235e+07', 'mean_token_accuracy': '0.8803', 'epoch': '0.08603'}
{'loss': '1.054', 'grad_norm': '0.4096', 'learning_rate': '1.908e-05', 'entropy': '1.186', 'num_tokens': '5.236e+07', 'mean_token_accuracy': '0.7306', 'epoch': '0.08604'}
{'loss': '0.7087', 'grad_norm': '0.45', 'learning_rate': '1.907e-05', 'entropy': '0.7802', 'num_tokens': '5.236e+07', 'mean_token_accuracy': '0.8039', 'epoch': '0.08605'}
{'loss': '0.7314', 'grad_norm': '0.4257', 'learning_rate': '1.905e-05', 'entropy': '0.7716', 'num_tokens': '5.237e+07', 'mean_token_accuracy': '0.7943', 'epoch': '0.08606'}
{'loss': '0.9096', 'grad_norm': '0.4847', 'learning_rate': '1.903e-05', 'entropy': '0.9057', 'num_tokens': '5.237e+07', 'mean_token_accuracy': '0.7627', 'epoch': '0.08606'}
{'loss': '0.8771', 'grad_norm': '0.4064', 'learning_rate': '1.902e-05', 'entropy': '0.8616', 'num_tokens': '5.238e+07', 'mean_token_accuracy': '0.7939', 'epoch': '0.08607'}
{'loss': '0.8294', 'grad_norm': '0.3135', 'learning_rate': '1.9e-05', 'entropy': '1.008', 'num_tokens': '5.239e+07', 'mean_token_accuracy': '0.7658', 'epoch': '0.08608'}
{'loss': '0.8545', 'grad_norm': '0.3985', 'learning_rate': '1.899e-05', 'entropy': '0.9112', 'num_tokens': '5.239e+07', 'mean_token_accuracy': '0.7733', 'epoch': '0.08609'}
{'loss': '1.033', 'grad_norm': '0.3965', 'learning_rate': '1.897e-05', 'entropy': '0.995', 'num_tokens': '5.24e+07', 'mean_token_accuracy': '0.7485', 'epoch': '0.0861'}
{'loss': '0.7904', 'grad_norm': '0.3536', 'learning_rate': '1.895e-05', 'entropy': '0.7581', 'num_tokens': '5.24e+07', 'mean_token_accuracy': '0.8088', 'epoch': '0.08611'}
{'loss': '0.8274', 'grad_norm': '0.5056', 'learning_rate': '1.894e-05', 'entropy': '0.7971', 'num_tokens': '5.24e+07', 'mean_token_accuracy': '0.7963', 'epoch': '0.08612'}
{'loss': '1.143', 'grad_norm': '0.4582', 'learning_rate': '1.892e-05', 'entropy': '1.111', 'num_tokens': '5.241e+07', 'mean_token_accuracy': '0.7297', 'epoch': '0.08613'}

 81%|████████  | 9733/12000 [5:44:25<1:22:35,  2.19s/it]
 81%|████████  | 9734/12000 [5:44:28<1:29:06,  2.36s/it]
                                                        

 81%|████████  | 9734/12000 [5:44:28<1:29:06,  2.36s/it]
 81%|████████  | 9735/12000 [5:44:30<1:20:22,  2.13s/it]
                                                        

 81%|████████  | 9735/12000 [5:44:30<1:20:22,  2.13s/it]
 81%|████████  | 9736/12000 [5:44:31<1:16:11,  2.02s/it]
                                                        

 81%|████████  | 9736/12000 [5:44:31<1:16:11,  2.02s/it]
 81%|████████  | 9737/12000 [5:44:34<1:22:07,  2.18s/it]
                                                        

 81%|████████  | 9737/12000 [5:44:34<1:22:07,  2.18s/it]
 81%|████████  | 9738/12000 [5:44:36<1:21:17,  2.16s/it]
                                                        

 81%|████████  | 9738/12000 [5:44:36<1:21:17,  2.16s/it]
 81%|████████  | 9739/12000 [5:44:38<1:14:23,  1.97s/it]
                                                        

 81%|████████  | 9739/12000 [5:44:38<1:14:23,  1.97s/it]
 81%|████████  | 9740/12000 [5:44:44<2:09:30,  3.44s/it]
                                                        

 81%|████████  | 9740/12000 [5:44:44<2:09:30,  3.44s/it]
 81%|████████  | 9741/12000 [5:44:46<1:53:40,  3.02s/it]
                                                        

 81%|████████  | 9741/12000 [5:44:46<1:53:40,  3.02s/it]
 81%|████████  | 9742/12000 [5:44:48<1:41:04,  2.69s/it]
                                                        

 81%|████████  | 9742/12000 [5:44:48<1:41:04,  2.69s/it]
 81%|████████  | 9743/12000 [5:44:51<1:36:45,  2.57s/it]
                                                        

 81%|████████  | 9743/12000 [5:44:51<1:36:45,  2.57s/it]
 81%|████████  | 9744/12000 [5:44:52<1:24:47,  2.26s/it]
                                                        

 81%|████████  | 9744/12000 [5:44:52<1:24:47,  2.26s/it]
 81%|████████  | 9745/12000 [5:44:54<1:16:02,  2.02s/it]
                                                        

 81%|████████  | 9745/12000 [5:44:54<1:16:02,  2.02s/it]
 81%|████████  | 9746/12000 [5:44:57<1:34:24,  2.51s/it]
                                                        

 81%|████████  | 9746/12000 [5:44:57<1:34:24,  2.51s/it]
 81%|████████  | 9747/12000 [5:44:59<1:23:40,  2.23s/it]
                                                        

 81%|████████  | 9747/12000 [5:44:59<1:23:40,  2.23s/it]
 81%|████████  | 9748/12000 [5:45:01<1:22:08,  2.19s/it]
                                                        

 81%|████████  | 9748/12000 [5:45:01<1:22:08,  2.19s/it]
 81%|████████  | 9749/12000 [5:45:02<1:14:11,  1.98s/it]
                                                        

 81%|████████  | 9749/12000 [5:45:02<1:14:11,  1.98s/it]
 81%|████████▏ | 9750/12000 [5:45:04<1:09:18,  1.85s/it]
                                                        

 81%|████████▏ | 9750/12000 [5:45:04<1:09:18,  1.85s/it]
 81%|████████▏ | 9751/12000 [5:45:08<1:33:19,  2.49s/it]
                                                        

 81%|████████▏ | 9751/12000 [5:45:08<1:33:19,  2.49s/it]
 81%|████████▏ | 9752/12000 [5:45:10<1:22:27,  2.20s/it]
                                                        

 81%|████████▏ | 9752/12000 [5:45:10<1:22:27,  2.20s/it]
 81%|████████▏ | 9753/12000 [5:45:11<1:15:41,  2.02s/it]
                                                        

 81%|████████▏ | 9753/12000 [5:45:11<1:15:41,  2.02s/it]
 81%|████████▏ | 9754/12000 [5:45:15<1:33:38,  2.50s/it]
                                                        

 81%|████████▏ | 9754/12000 [5:45:15<1:33:38,  2.50s/it]
 81%|████████▏ | 9755/12000 [5:45:16<1:23:50,  2.24s/it]
                                                        

 81%|████████▏ | 9755/12000 [5:45:16<1:23:50,  2.24s/it]
 81%|████████▏ | 9756/12000 [5:45:18<1:13:54,  1.98s/it]
                                                        

 81%|████████▏ | 9756/12000 [5:45:18<1:13:54,  1.98s/it]
 81%|████████▏ | 9757/12000 [5:45:19<1:08:54,  1.84s/it]
                                                        

 81%|████████▏ | 9757/12000 [5:45:19<1:08:54,  1.84s/it]
 81%|████████▏ | 9758/12000 [5:45:21<1:04:59,  1.74s/it]
                                                        

 81%|████████▏ | 9758/12000 [5:45:21<1:04:59,  1.74s/it]
 81%|████████▏ | 9759/12000 [5:45:23<1:05:05,  1.74s/it]
                                                        

 81%|████████▏ | 9759/12000 [5:45:23<1:05:05,  1.74s/it]
 81%|████████▏ | 9760/12000 [5:45:24<1:05:08,  1.74s/it]
                                                        

 81%|████████▏ | 9760/12000 [5:45:24<1:05:08,  1.74s/it]
 81%|████████▏ | 9761/12000 [5:45:27<1:13:02,  1.96s/it]
                                                        

 81%|████████▏ | 9761/12000 [5:45:27<1:13:02,  1.96s/it]
 81%|████████▏ | 9762/12000 [5:45:28<1:08:40,  1.84s/it]
                                                        

 81%|████████▏ | 9762/12000 [5:45:28<1:08:40,  1.84s/it]
 81%|████████▏ | 9763/12000 [5:45:30<1:05:23,  1.75s/it]
                                                        

 81%|████████▏ | 9763/12000 [5:45:30<1:05:23,  1.75s/it]
 81%|████████▏ | 9764/12000 [5:45:33<1:21:40,  2.19s/it]
                                                        

 81%|████████▏ | 9764/12000 [5:45:33<1:21:40,  2.19s/it]
 81%|████████▏ | 9765/12000 [5:45:35<1:16:40,  2.06s/it]
                                                        

 81%|████████▏ | 9765/12000 [5:45:35<1:16:40,  2.06s/it]
 81%|████████▏ | 9766/12000 [5:45:37<1:14:40,  2.01s/it]
                                                        

 81%|████████▏ | 9766/12000 [5:45:37<1:14:40,  2.01s/it]
 81%|████████▏ | 9767/12000 [5:45:40<1:26:37,  2.33s/it]
                                                        

 81%|████████▏ | 9767/12000 [5:45:40<1:26:37,  2.33s/it]
 81%|████████▏ | 9768/12000 [5:45:41<1:18:34,  2.11s/it]
                                                        

 81%|████████▏ | 9768/12000 [5:45:41<1:18:34,  2.11s/it]
 81%|████████▏ | 9769/12000 [5:45:43<1:18:11,  2.10s/it]
                                                        

 81%|████████▏ | 9769/12000 [5:45:43<1:18:11,  2.10s/it]
 81%|████████▏ | 9770/12000 [5:45:45<1:16:55,  2.07s/it]
                                                        

 81%|████████▏ | 9770/12000 [5:45:45<1:16:55,  2.07s/it]
 81%|████████▏ | 9771/12000 [5:45:47<1:11:12,  1.92s/it]
                                                        

 81%|████████▏ | 9771/12000 [5:45:47<1:11:12,  1.92s/it]
 81%|████████▏ | 9772/12000 [5:45:50<1:24:15,  2.27s/it]
                                                        

 81%|████████▏ | 9772/12000 [5:45:50<1:24:15,  2.27s/it]
 81%|████████▏ | 9773/12000 [5:45:51<1:14:36,  2.01s/it]
                                                        

 81%|████████▏ | 9773/12000 [5:45:51<1:14:36,  2.01s/it]
 81%|████████▏ | 9774/12000 [5:45:53<1:10:47,  1.91s/it]
                                                        

 81%|████████▏ | 9774/12000 [5:45:53<1:10:47,  1.91s/it]
 81%|████████▏ | 9775/12000 [5:45:55<1:09:56,  1.89s/it]
                                                        

 81%|████████▏ | 9775/12000 [5:45:55<1:09:56,  1.89s/it]
 81%|████████▏ | 9776/12000 [5:45:58<1:27:03,  2.35s/it]
                                                        

 81%|████████▏ | 9776/12000 [5:45:58<1:27:03,  2.35s/it]
 81%|████████▏ | 9777/12000 [5:46:00<1:16:44,  2.07s/it]
                                                        

 81%|████████▏ | 9777/12000 [5:46:00<1:16:44,  2.07s/it]
 81%|████████▏ | 9778/12000 [5:46:02<1:20:26,  2.17s/it]
                                                        

 81%|████████▏ | 9778/12000 [5:46:02<1:20:26,  2.17s/it]
 81%|████████▏ | 9779/12000 [5:46:04<1:14:19,  2.01s/it]
                                                        

 81%|████████▏ | 9779/12000 [5:46:04<1:14:19,  2.01s/it]
 82%|████████▏ | 9780/12000 [5:46:08<1:35:32,  2.58s/it]
                                                        
{'loss': '0.7346', 'grad_norm': '0.3992', 'learning_rate': '1.89e-05', 'entropy': '0.6893', 'num_tokens': '5.242e+07', 'mean_token_accuracy': '0.8218', 'epoch': '0.08614'}
{'loss': '0.5484', 'grad_norm': '0.2676', 'learning_rate': '1.889e-05', 'entropy': '0.5309', 'num_tokens': '5.242e+07', 'mean_token_accuracy': '0.8539', 'epoch': '0.08614'}
{'loss': '0.7662', 'grad_norm': '0.4043', 'learning_rate': '1.887e-05', 'entropy': '0.7615', 'num_tokens': '5.243e+07', 'mean_token_accuracy': '0.809', 'epoch': '0.08615'}
{'loss': '0.8214', 'grad_norm': '0.4936', 'learning_rate': '1.886e-05', 'entropy': '0.8549', 'num_tokens': '5.243e+07', 'mean_token_accuracy': '0.7901', 'epoch': '0.08616'}
{'loss': '0.6996', 'grad_norm': '0.3514', 'learning_rate': '1.884e-05', 'entropy': '0.7144', 'num_tokens': '5.244e+07', 'mean_token_accuracy': '0.8142', 'epoch': '0.08617'}
{'loss': '0.7384', 'grad_norm': '0.3371', 'learning_rate': '1.882e-05', 'entropy': '0.7946', 'num_tokens': '5.244e+07', 'mean_token_accuracy': '0.8035', 'epoch': '0.08618'}
{'loss': '0.5503', 'grad_norm': '0.3638', 'learning_rate': '1.881e-05', 'entropy': '0.5603', 'num_tokens': '5.245e+07', 'mean_token_accuracy': '0.8585', 'epoch': '0.08619'}
{'loss': '0.9046', 'grad_norm': '0.3215', 'learning_rate': '1.879e-05', 'entropy': '0.7802', 'num_tokens': '5.246e+07', 'mean_token_accuracy': '0.8236', 'epoch': '0.0862'}
{'loss': '1.221', 'grad_norm': '0.5282', 'learning_rate': '1.878e-05', 'entropy': '1.199', 'num_tokens': '5.246e+07', 'mean_token_accuracy': '0.7097', 'epoch': '0.08621'}
{'loss': '0.6912', 'grad_norm': '0.3845', 'learning_rate': '1.876e-05', 'entropy': '0.6826', 'num_tokens': '5.247e+07', 'mean_token_accuracy': '0.8208', 'epoch': '0.08622'}
{'loss': '0.7162', 'grad_norm': '0.3948', 'learning_rate': '1.874e-05', 'entropy': '0.709', 'num_tokens': '5.248e+07', 'mean_token_accuracy': '0.8204', 'epoch': '0.08622'}
{'loss': '1.16', 'grad_norm': '0.4505', 'learning_rate': '1.873e-05', 'entropy': '1.176', 'num_tokens': '5.248e+07', 'mean_token_accuracy': '0.7141', 'epoch': '0.08623'}
{'loss': '0.8355', 'grad_norm': '0.3836', 'learning_rate': '1.871e-05', 'entropy': '0.8479', 'num_tokens': '5.248e+07', 'mean_token_accuracy': '0.8056', 'epoch': '0.08624'}
{'loss': '0.6247', 'grad_norm': '0.2989', 'learning_rate': '1.87e-05', 'entropy': '0.7111', 'num_tokens': '5.249e+07', 'mean_token_accuracy': '0.8069', 'epoch': '0.08625'}
{'loss': '0.9487', 'grad_norm': '0.4517', 'learning_rate': '1.868e-05', 'entropy': '1.034', 'num_tokens': '5.25e+07', 'mean_token_accuracy': '0.7458', 'epoch': '0.08626'}
{'loss': '0.748', 'grad_norm': '0.3633', 'learning_rate': '1.866e-05', 'entropy': '0.7307', 'num_tokens': '5.25e+07', 'mean_token_accuracy': '0.7955', 'epoch': '0.08627'}
{'loss': '0.9689', 'grad_norm': '0.4115', 'learning_rate': '1.865e-05', 'entropy': '0.9521', 'num_tokens': '5.251e+07', 'mean_token_accuracy': '0.7558', 'epoch': '0.08628'}
{'loss': '0.7691', 'grad_norm': '0.4769', 'learning_rate': '1.863e-05', 'entropy': '0.7829', 'num_tokens': '5.251e+07', 'mean_token_accuracy': '0.7926', 'epoch': '0.08629'}
{'loss': '0.9823', 'grad_norm': '0.4434', 'learning_rate': '1.862e-05', 'entropy': '0.9381', 'num_tokens': '5.252e+07', 'mean_token_accuracy': '0.7681', 'epoch': '0.08629'}
{'loss': '0.9502', 'grad_norm': '0.5105', 'learning_rate': '1.86e-05', 'entropy': '1.008', 'num_tokens': '5.252e+07', 'mean_token_accuracy': '0.755', 'epoch': '0.0863'}
{'loss': '0.9298', 'grad_norm': '0.3844', 'learning_rate': '1.858e-05', 'entropy': '0.9544', 'num_tokens': '5.253e+07', 'mean_token_accuracy': '0.7711', 'epoch': '0.08631'}
{'loss': '0.6407', 'grad_norm': '0.3444', 'learning_rate': '1.857e-05', 'entropy': '0.6798', 'num_tokens': '5.253e+07', 'mean_token_accuracy': '0.8211', 'epoch': '0.08632'}
{'loss': '1.024', 'grad_norm': '0.5529', 'learning_rate': '1.855e-05', 'entropy': '1.01', 'num_tokens': '5.253e+07', 'mean_token_accuracy': '0.7482', 'epoch': '0.08633'}
{'loss': '0.5429', 'grad_norm': '0.385', 'learning_rate': '1.854e-05', 'entropy': '0.5796', 'num_tokens': '5.254e+07', 'mean_token_accuracy': '0.8481', 'epoch': '0.08634'}
{'loss': '0.8751', 'grad_norm': '0.4006', 'learning_rate': '1.852e-05', 'entropy': '0.8963', 'num_tokens': '5.254e+07', 'mean_token_accuracy': '0.7814', 'epoch': '0.08635'}
{'loss': '0.84', 'grad_norm': '0.4419', 'learning_rate': '1.85e-05', 'entropy': '0.8259', 'num_tokens': '5.255e+07', 'mean_token_accuracy': '0.7986', 'epoch': '0.08636'}
{'loss': '1.468', 'grad_norm': '0.532', 'learning_rate': '1.849e-05', 'entropy': '1.325', 'num_tokens': '5.255e+07', 'mean_token_accuracy': '0.7006', 'epoch': '0.08637'}
{'loss': '0.9879', 'grad_norm': '0.375', 'learning_rate': '1.847e-05', 'entropy': '0.9012', 'num_tokens': '5.256e+07', 'mean_token_accuracy': '0.7853', 'epoch': '0.08637'}
{'loss': '0.9527', 'grad_norm': '0.4039', 'learning_rate': '1.846e-05', 'entropy': '0.9434', 'num_tokens': '5.256e+07', 'mean_token_accuracy': '0.7612', 'epoch': '0.08638'}
{'loss': '0.7818', 'grad_norm': '0.3702', 'learning_rate': '1.844e-05', 'entropy': '0.7716', 'num_tokens': '5.257e+07', 'mean_token_accuracy': '0.8171', 'epoch': '0.08639'}
{'loss': '0.65', 'grad_norm': '0.3771', 'learning_rate': '1.842e-05', 'entropy': '0.7001', 'num_tokens': '5.257e+07', 'mean_token_accuracy': '0.8232', 'epoch': '0.0864'}
{'loss': '1.163', 'grad_norm': '0.4155', 'learning_rate': '1.841e-05', 'entropy': '0.9147', 'num_tokens': '5.258e+07', 'mean_token_accuracy': '0.7938', 'epoch': '0.08641'}
{'loss': '0.6189', 'grad_norm': '0.4557', 'learning_rate': '1.839e-05', 'entropy': '0.6492', 'num_tokens': '5.259e+07', 'mean_token_accuracy': '0.8239', 'epoch': '0.08642'}
{'loss': '0.833', 'grad_norm': '0.4225', 'learning_rate': '1.838e-05', 'entropy': '0.8858', 'num_tokens': '5.259e+07', 'mean_token_accuracy': '0.7646', 'epoch': '0.08643'}
{'loss': '0.8233', 'grad_norm': '0.3696', 'learning_rate': '1.836e-05', 'entropy': '0.7519', 'num_tokens': '5.26e+07', 'mean_token_accuracy': '0.8147', 'epoch': '0.08644'}
{'loss': '0.7395', 'grad_norm': '0.3603', 'learning_rate': '1.834e-05', 'entropy': '0.6977', 'num_tokens': '5.26e+07', 'mean_token_accuracy': '0.8183', 'epoch': '0.08645'}
{'loss': '0.8656', 'grad_norm': '0.4242', 'learning_rate': '1.833e-05', 'entropy': '0.7433', 'num_tokens': '5.261e+07', 'mean_token_accuracy': '0.8066', 'epoch': '0.08645'}
{'loss': '0.6172', 'grad_norm': '0.3558', 'learning_rate': '1.831e-05', 'entropy': '0.6277', 'num_tokens': '5.262e+07', 'mean_token_accuracy': '0.8378', 'epoch': '0.08646'}
{'loss': '0.9534', 'grad_norm': '0.4546', 'learning_rate': '1.83e-05', 'entropy': '0.8787', 'num_tokens': '5.262e+07', 'mean_token_accuracy': '0.7706', 'epoch': '0.08647'}
{'loss': '0.9509', 'grad_norm': '0.4412', 'learning_rate': '1.828e-05', 'entropy': '0.9105', 'num_tokens': '5.263e+07', 'mean_token_accuracy': '0.7702', 'epoch': '0.08648'}
{'loss': '0.6905', 'grad_norm': '0.3391', 'learning_rate': '1.826e-05', 'entropy': '0.8096', 'num_tokens': '5.263e+07', 'mean_token_accuracy': '0.7921', 'epoch': '0.08649'}
{'loss': '0.5697', 'grad_norm': '0.343', 'learning_rate': '1.825e-05', 'entropy': '0.6339', 'num_tokens': '5.264e+07', 'mean_token_accuracy': '0.8316', 'epoch': '0.0865'}
{'loss': '0.7248', 'grad_norm': '0.5297', 'learning_rate': '1.823e-05', 'entropy': '0.7639', 'num_tokens': '5.264e+07', 'mean_token_accuracy': '0.7987', 'epoch': '0.08651'}
{'loss': '0.7653', 'grad_norm': '0.3845', 'learning_rate': '1.822e-05', 'entropy': '0.7376', 'num_tokens': '5.265e+07', 'mean_token_accuracy': '0.8107', 'epoch': '0.08652'}
{'loss': '0.7896', 'grad_norm': '0.498', 'learning_rate': '1.82e-05', 'entropy': '0.7582', 'num_tokens': '5.265e+07', 'mean_token_accuracy': '0.8159', 'epoch': '0.08652'}
{'loss': '0.6821', 'grad_norm': '0.3019', 'learning_rate': '1.819e-05', 'entropy': '0.7213', 'num_tokens': '5.266e+07', 'mean_token_accuracy': '0.8228', 'epoch': '0.08653'}
{'loss': '0.7277', 'grad_norm': '0.4204', 'learning_rate': '1.817e-05', 'entropy': '0.7504', 'num_tokens': '5.266e+07', 'mean_token_accuracy': '0.8017', 'epoch': '0.08654'}

 82%|████████▏ | 9780/12000 [5:46:08<1:35:32,  2.58s/it]
 82%|████████▏ | 9781/12000 [5:46:10<1:26:48,  2.35s/it]
                                                        

 82%|████████▏ | 9781/12000 [5:46:10<1:26:48,  2.35s/it]
 82%|████████▏ | 9782/12000 [5:46:11<1:17:20,  2.09s/it]
                                                        

 82%|████████▏ | 9782/12000 [5:46:11<1:17:20,  2.09s/it]
 82%|████████▏ | 9783/12000 [5:46:13<1:18:32,  2.13s/it]
                                                        

 82%|████████▏ | 9783/12000 [5:46:13<1:18:32,  2.13s/it]
 82%|████████▏ | 9784/12000 [5:46:15<1:11:21,  1.93s/it]
                                                        

 82%|████████▏ | 9784/12000 [5:46:15<1:11:21,  1.93s/it]
 82%|████████▏ | 9785/12000 [5:46:16<1:07:26,  1.83s/it]
                                                        

 82%|████████▏ | 9785/12000 [5:46:16<1:07:26,  1.83s/it]
 82%|████████▏ | 9786/12000 [5:46:18<1:07:01,  1.82s/it]
                                                        

 82%|████████▏ | 9786/12000 [5:46:18<1:07:01,  1.82s/it]
 82%|████████▏ | 9787/12000 [5:46:20<1:03:05,  1.71s/it]
                                                        

 82%|████████▏ | 9787/12000 [5:46:20<1:03:05,  1.71s/it]
 82%|████████▏ | 9788/12000 [5:46:22<1:06:45,  1.81s/it]
                                                        

 82%|████████▏ | 9788/12000 [5:46:22<1:06:45,  1.81s/it]
 82%|████████▏ | 9789/12000 [5:46:23<1:05:57,  1.79s/it]
                                                        

 82%|████████▏ | 9789/12000 [5:46:23<1:05:57,  1.79s/it]
 82%|████████▏ | 9790/12000 [5:46:26<1:13:52,  2.01s/it]
                                                        

 82%|████████▏ | 9790/12000 [5:46:26<1:13:52,  2.01s/it]
 82%|████████▏ | 9791/12000 [5:46:27<1:07:43,  1.84s/it]
                                                        

 82%|████████▏ | 9791/12000 [5:46:27<1:07:43,  1.84s/it]
 82%|████████▏ | 9792/12000 [5:46:29<1:09:53,  1.90s/it]
                                                        

 82%|████████▏ | 9792/12000 [5:46:29<1:09:53,  1.90s/it]
 82%|████████▏ | 9793/12000 [5:46:31<1:07:57,  1.85s/it]
                                                        

 82%|████████▏ | 9793/12000 [5:46:31<1:07:57,  1.85s/it]
 82%|████████▏ | 9794/12000 [5:46:33<1:03:35,  1.73s/it]
                                                        

 82%|████████▏ | 9794/12000 [5:46:33<1:03:35,  1.73s/it]
 82%|████████▏ | 9795/12000 [5:46:34<1:04:18,  1.75s/it]
                                                        

 82%|████████▏ | 9795/12000 [5:46:34<1:04:18,  1.75s/it]
 82%|████████▏ | 9796/12000 [5:46:37<1:08:57,  1.88s/it]
                                                        

 82%|████████▏ | 9796/12000 [5:46:37<1:08:57,  1.88s/it]
 82%|████████▏ | 9797/12000 [5:46:38<1:08:43,  1.87s/it]
                                                        

 82%|████████▏ | 9797/12000 [5:46:38<1:08:43,  1.87s/it]
 82%|████████▏ | 9798/12000 [5:46:42<1:29:31,  2.44s/it]
                                                        

 82%|████████▏ | 9798/12000 [5:46:42<1:29:31,  2.44s/it]
 82%|████████▏ | 9799/12000 [5:46:44<1:17:41,  2.12s/it]
                                                        

 82%|████████▏ | 9799/12000 [5:46:44<1:17:41,  2.12s/it]
 82%|████████▏ | 9800/12000 [5:46:45<1:13:26,  2.00s/it]
                                                        

 82%|████████▏ | 9800/12000 [5:46:45<1:13:26,  2.00s/it]
 82%|████████▏ | 9801/12000 [5:46:48<1:24:48,  2.31s/it]
                                                        

 82%|████████▏ | 9801/12000 [5:46:48<1:24:48,  2.31s/it]
 82%|████████▏ | 9802/12000 [5:46:51<1:25:28,  2.33s/it]
                                                        

 82%|████████▏ | 9802/12000 [5:46:51<1:25:28,  2.33s/it]
 82%|████████▏ | 9803/12000 [5:46:53<1:23:29,  2.28s/it]
                                                        

 82%|████████▏ | 9803/12000 [5:46:53<1:23:29,  2.28s/it]
 82%|████████▏ | 9804/12000 [5:46:56<1:34:30,  2.58s/it]
                                                        

 82%|████████▏ | 9804/12000 [5:46:56<1:34:30,  2.58s/it]
 82%|████████▏ | 9805/12000 [5:46:58<1:22:03,  2.24s/it]
                                                        

 82%|████████▏ | 9805/12000 [5:46:58<1:22:03,  2.24s/it]
 82%|████████▏ | 9806/12000 [5:46:59<1:14:03,  2.03s/it]
                                                        

 82%|████████▏ | 9806/12000 [5:46:59<1:14:03,  2.03s/it]
 82%|████████▏ | 9807/12000 [5:47:01<1:09:34,  1.90s/it]
                                                        

 82%|████████▏ | 9807/12000 [5:47:01<1:09:34,  1.90s/it]
 82%|████████▏ | 9808/12000 [5:47:03<1:18:00,  2.14s/it]
                                                        

 82%|████████▏ | 9808/12000 [5:47:03<1:18:00,  2.14s/it]
 82%|████████▏ | 9809/12000 [5:47:05<1:12:39,  1.99s/it]
                                                        

 82%|████████▏ | 9809/12000 [5:47:05<1:12:39,  1.99s/it]
 82%|████████▏ | 9810/12000 [5:47:06<1:06:05,  1.81s/it]
                                                        

 82%|████████▏ | 9810/12000 [5:47:06<1:06:05,  1.81s/it]
 82%|████████▏ | 9811/12000 [5:47:08<1:03:38,  1.74s/it]
                                                        

 82%|████████▏ | 9811/12000 [5:47:08<1:03:38,  1.74s/it]
 82%|████████▏ | 9812/12000 [5:47:10<1:02:01,  1.70s/it]
                                                        

 82%|████████▏ | 9812/12000 [5:47:10<1:02:01,  1.70s/it]
 82%|████████▏ | 9813/12000 [5:47:11<59:56,  1.64s/it]  
                                                      

 82%|████████▏ | 9813/12000 [5:47:11<59:56,  1.64s/it]
 82%|████████▏ | 9814/12000 [5:47:13<1:00:11,  1.65s/it]
                                                        

 82%|████████▏ | 9814/12000 [5:47:13<1:00:11,  1.65s/it]
 82%|████████▏ | 9815/12000 [5:47:15<1:02:07,  1.71s/it]
                                                        

 82%|████████▏ | 9815/12000 [5:47:15<1:02:07,  1.71s/it]
 82%|████████▏ | 9816/12000 [5:47:17<1:05:18,  1.79s/it]
                                                        

 82%|████████▏ | 9816/12000 [5:47:17<1:05:18,  1.79s/it]
 82%|████████▏ | 9817/12000 [5:47:19<1:14:28,  2.05s/it]
                                                        

 82%|████████▏ | 9817/12000 [5:47:19<1:14:28,  2.05s/it]
 82%|████████▏ | 9818/12000 [5:47:21<1:15:38,  2.08s/it]
                                                        

 82%|████████▏ | 9818/12000 [5:47:21<1:15:38,  2.08s/it]
 82%|████████▏ | 9819/12000 [5:47:23<1:13:12,  2.01s/it]
                                                        

 82%|████████▏ | 9819/12000 [5:47:23<1:13:12,  2.01s/it]
 82%|████████▏ | 9820/12000 [5:47:27<1:31:47,  2.53s/it]
                                                        

 82%|████████▏ | 9820/12000 [5:47:27<1:31:47,  2.53s/it]
 82%|████████▏ | 9821/12000 [5:47:29<1:20:24,  2.21s/it]
                                                        

 82%|████████▏ | 9821/12000 [5:47:29<1:20:24,  2.21s/it]
 82%|████████▏ | 9822/12000 [5:47:30<1:16:54,  2.12s/it]
                                                        

 82%|████████▏ | 9822/12000 [5:47:30<1:16:54,  2.12s/it]
 82%|████████▏ | 9823/12000 [5:47:32<1:13:55,  2.04s/it]
                                                        

 82%|████████▏ | 9823/12000 [5:47:32<1:13:55,  2.04s/it]
 82%|████████▏ | 9824/12000 [5:47:34<1:07:47,  1.87s/it]
                                                        

 82%|████████▏ | 9824/12000 [5:47:34<1:07:47,  1.87s/it]
 82%|████████▏ | 9825/12000 [5:47:36<1:12:06,  1.99s/it]
                                                        

 82%|████████▏ | 9825/12000 [5:47:36<1:12:06,  1.99s/it]
 82%|████████▏ | 9826/12000 [5:47:38<1:08:53,  1.90s/it]
                                                        

 82%|████████▏ | 9826/12000 [5:47:38<1:08:53,  1.90s/it]
 82%|████████▏ | 9827/12000 [5:47:41<1:25:33,  2.36s/it]
                                                        
{'loss': '0.6813', 'grad_norm': '0.3122', 'learning_rate': '1.815e-05', 'entropy': '0.6926', 'num_tokens': '5.267e+07', 'mean_token_accuracy': '0.8204', 'epoch': '0.08655'}
{'loss': '0.7376', 'grad_norm': '0.3652', 'learning_rate': '1.814e-05', 'entropy': '0.6563', 'num_tokens': '5.267e+07', 'mean_token_accuracy': '0.828', 'epoch': '0.08656'}
{'loss': '0.8575', 'grad_norm': '0.3327', 'learning_rate': '1.812e-05', 'entropy': '0.9096', 'num_tokens': '5.268e+07', 'mean_token_accuracy': '0.7672', 'epoch': '0.08657'}
{'loss': '0.8214', 'grad_norm': '0.4508', 'learning_rate': '1.811e-05', 'entropy': '0.781', 'num_tokens': '5.269e+07', 'mean_token_accuracy': '0.8023', 'epoch': '0.08658'}
{'loss': '0.6427', 'grad_norm': '0.4197', 'learning_rate': '1.809e-05', 'entropy': '0.7379', 'num_tokens': '5.269e+07', 'mean_token_accuracy': '0.8075', 'epoch': '0.08659'}
{'loss': '0.7674', 'grad_norm': '0.4305', 'learning_rate': '1.807e-05', 'entropy': '0.7403', 'num_tokens': '5.269e+07', 'mean_token_accuracy': '0.8101', 'epoch': '0.0866'}
{'loss': '0.7626', 'grad_norm': '0.3662', 'learning_rate': '1.806e-05', 'entropy': '0.7622', 'num_tokens': '5.27e+07', 'mean_token_accuracy': '0.8032', 'epoch': '0.0866'}
{'loss': '0.8412', 'grad_norm': '0.3644', 'learning_rate': '1.804e-05', 'entropy': '0.8645', 'num_tokens': '5.27e+07', 'mean_token_accuracy': '0.7794', 'epoch': '0.08661'}
{'loss': '0.6318', 'grad_norm': '0.3512', 'learning_rate': '1.803e-05', 'entropy': '0.603', 'num_tokens': '5.271e+07', 'mean_token_accuracy': '0.8361', 'epoch': '0.08662'}
{'loss': '1.044', 'grad_norm': '0.4545', 'learning_rate': '1.801e-05', 'entropy': '1.19', 'num_tokens': '5.271e+07', 'mean_token_accuracy': '0.7199', 'epoch': '0.08663'}
{'loss': '1.209', 'grad_norm': '0.3868', 'learning_rate': '1.8e-05', 'entropy': '1.149', 'num_tokens': '5.272e+07', 'mean_token_accuracy': '0.7456', 'epoch': '0.08664'}
{'loss': '0.5914', 'grad_norm': '0.3933', 'learning_rate': '1.798e-05', 'entropy': '0.6511', 'num_tokens': '5.272e+07', 'mean_token_accuracy': '0.8267', 'epoch': '0.08665'}
{'loss': '0.9041', 'grad_norm': '0.4096', 'learning_rate': '1.796e-05', 'entropy': '0.8154', 'num_tokens': '5.273e+07', 'mean_token_accuracy': '0.7956', 'epoch': '0.08666'}
{'loss': '0.7561', 'grad_norm': '0.4375', 'learning_rate': '1.795e-05', 'entropy': '0.6566', 'num_tokens': '5.274e+07', 'mean_token_accuracy': '0.8314', 'epoch': '0.08667'}
{'loss': '0.8157', 'grad_norm': '0.3345', 'learning_rate': '1.793e-05', 'entropy': '0.8092', 'num_tokens': '5.274e+07', 'mean_token_accuracy': '0.7895', 'epoch': '0.08668'}
{'loss': '0.6368', 'grad_norm': '0.4365', 'learning_rate': '1.792e-05', 'entropy': '0.6019', 'num_tokens': '5.275e+07', 'mean_token_accuracy': '0.8255', 'epoch': '0.08668'}
{'loss': '0.7913', 'grad_norm': '0.4836', 'learning_rate': '1.79e-05', 'entropy': '0.7218', 'num_tokens': '5.275e+07', 'mean_token_accuracy': '0.8157', 'epoch': '0.08669'}
{'loss': '0.6119', 'grad_norm': '0.3573', 'learning_rate': '1.789e-05', 'entropy': '0.606', 'num_tokens': '5.276e+07', 'mean_token_accuracy': '0.8388', 'epoch': '0.0867'}
{'loss': '0.9501', 'grad_norm': '0.4021', 'learning_rate': '1.787e-05', 'entropy': '0.8674', 'num_tokens': '5.276e+07', 'mean_token_accuracy': '0.7871', 'epoch': '0.08671'}
{'loss': '0.7116', 'grad_norm': '0.4114', 'learning_rate': '1.785e-05', 'entropy': '0.8021', 'num_tokens': '5.277e+07', 'mean_token_accuracy': '0.7975', 'epoch': '0.08672'}
{'loss': '1.131', 'grad_norm': '0.4443', 'learning_rate': '1.784e-05', 'entropy': '1.194', 'num_tokens': '5.277e+07', 'mean_token_accuracy': '0.7225', 'epoch': '0.08673'}
{'loss': '0.9774', 'grad_norm': '0.4164', 'learning_rate': '1.782e-05', 'entropy': '0.9087', 'num_tokens': '5.278e+07', 'mean_token_accuracy': '0.7698', 'epoch': '0.08674'}
{'loss': '0.5828', 'grad_norm': '0.3544', 'learning_rate': '1.781e-05', 'entropy': '0.7005', 'num_tokens': '5.278e+07', 'mean_token_accuracy': '0.8158', 'epoch': '0.08675'}
{'loss': '0.7054', 'grad_norm': '0.3622', 'learning_rate': '1.779e-05', 'entropy': '0.7274', 'num_tokens': '5.279e+07', 'mean_token_accuracy': '0.823', 'epoch': '0.08675'}
{'loss': '1.062', 'grad_norm': '0.4376', 'learning_rate': '1.778e-05', 'entropy': '1.042', 'num_tokens': '5.279e+07', 'mean_token_accuracy': '0.7448', 'epoch': '0.08676'}
{'loss': '0.6428', 'grad_norm': '0.3615', 'learning_rate': '1.776e-05', 'entropy': '0.6252', 'num_tokens': '5.28e+07', 'mean_token_accuracy': '0.837', 'epoch': '0.08677'}
{'loss': '0.8908', 'grad_norm': '0.3416', 'learning_rate': '1.774e-05', 'entropy': '0.8515', 'num_tokens': '5.28e+07', 'mean_token_accuracy': '0.7812', 'epoch': '0.08678'}
{'loss': '0.764', 'grad_norm': '0.3253', 'learning_rate': '1.773e-05', 'entropy': '0.7702', 'num_tokens': '5.281e+07', 'mean_token_accuracy': '0.8077', 'epoch': '0.08679'}
{'loss': '0.8951', 'grad_norm': '0.3483', 'learning_rate': '1.771e-05', 'entropy': '0.8364', 'num_tokens': '5.281e+07', 'mean_token_accuracy': '0.7863', 'epoch': '0.0868'}
{'loss': '0.7887', 'grad_norm': '0.4686', 'learning_rate': '1.77e-05', 'entropy': '0.8791', 'num_tokens': '5.282e+07', 'mean_token_accuracy': '0.7765', 'epoch': '0.08681'}
{'loss': '0.7588', 'grad_norm': '0.4371', 'learning_rate': '1.768e-05', 'entropy': '0.7659', 'num_tokens': '5.282e+07', 'mean_token_accuracy': '0.8108', 'epoch': '0.08682'}
{'loss': '0.8487', 'grad_norm': '0.4966', 'learning_rate': '1.767e-05', 'entropy': '0.9715', 'num_tokens': '5.283e+07', 'mean_token_accuracy': '0.77', 'epoch': '0.08683'}
{'loss': '0.5523', 'grad_norm': '0.3737', 'learning_rate': '1.765e-05', 'entropy': '0.5552', 'num_tokens': '5.283e+07', 'mean_token_accuracy': '0.8492', 'epoch': '0.08683'}
{'loss': '0.9882', 'grad_norm': '0.4412', 'learning_rate': '1.763e-05', 'entropy': '1.151', 'num_tokens': '5.284e+07', 'mean_token_accuracy': '0.7329', 'epoch': '0.08684'}
{'loss': '0.9452', 'grad_norm': '0.471', 'learning_rate': '1.762e-05', 'entropy': '0.9812', 'num_tokens': '5.284e+07', 'mean_token_accuracy': '0.7522', 'epoch': '0.08685'}
{'loss': '0.8878', 'grad_norm': '0.4124', 'learning_rate': '1.76e-05', 'entropy': '0.935', 'num_tokens': '5.285e+07', 'mean_token_accuracy': '0.7667', 'epoch': '0.08686'}
{'loss': '0.7722', 'grad_norm': '0.4586', 'learning_rate': '1.759e-05', 'entropy': '0.7241', 'num_tokens': '5.285e+07', 'mean_token_accuracy': '0.804', 'epoch': '0.08687'}
{'loss': '0.8709', 'grad_norm': '0.4401', 'learning_rate': '1.757e-05', 'entropy': '0.852', 'num_tokens': '5.286e+07', 'mean_token_accuracy': '0.7713', 'epoch': '0.08688'}
{'loss': '0.6507', 'grad_norm': '0.401', 'learning_rate': '1.756e-05', 'entropy': '0.6342', 'num_tokens': '5.286e+07', 'mean_token_accuracy': '0.8383', 'epoch': '0.08689'}
{'loss': '0.9301', 'grad_norm': '0.4372', 'learning_rate': '1.754e-05', 'entropy': '0.7925', 'num_tokens': '5.287e+07', 'mean_token_accuracy': '0.8086', 'epoch': '0.0869'}
{'loss': '0.7697', 'grad_norm': '0.4001', 'learning_rate': '1.752e-05', 'entropy': '0.785', 'num_tokens': '5.288e+07', 'mean_token_accuracy': '0.7906', 'epoch': '0.08691'}
{'loss': '0.5553', 'grad_norm': '0.293', 'learning_rate': '1.751e-05', 'entropy': '0.6312', 'num_tokens': '5.288e+07', 'mean_token_accuracy': '0.8439', 'epoch': '0.08691'}
{'loss': '0.8875', 'grad_norm': '0.4542', 'learning_rate': '1.749e-05', 'entropy': '0.8228', 'num_tokens': '5.289e+07', 'mean_token_accuracy': '0.7783', 'epoch': '0.08692'}
{'loss': '0.8527', 'grad_norm': '0.5708', 'learning_rate': '1.748e-05', 'entropy': '0.8307', 'num_tokens': '5.289e+07', 'mean_token_accuracy': '0.7907', 'epoch': '0.08693'}
{'loss': '0.75', 'grad_norm': '0.4031', 'learning_rate': '1.746e-05', 'entropy': '0.8404', 'num_tokens': '5.289e+07', 'mean_token_accuracy': '0.7905', 'epoch': '0.08694'}
{'loss': '0.7886', 'grad_norm': '0.3502', 'learning_rate': '1.745e-05', 'entropy': '0.7211', 'num_tokens': '5.29e+07', 'mean_token_accuracy': '0.8085', 'epoch': '0.08695'}
{'loss': '0.6897', 'grad_norm': '0.4163', 'learning_rate': '1.743e-05', 'entropy': '0.7476', 'num_tokens': '5.29e+07', 'mean_token_accuracy': '0.8011', 'epoch': '0.08696'}

 82%|████████▏ | 9827/12000 [5:47:41<1:25:33,  2.36s/it]
 82%|████████▏ | 9828/12000 [5:47:43<1:18:20,  2.16s/it]
                                                        

 82%|████████▏ | 9828/12000 [5:47:43<1:18:20,  2.16s/it]
 82%|████████▏ | 9829/12000 [5:47:45<1:14:12,  2.05s/it]
                                                        

 82%|████████▏ | 9829/12000 [5:47:45<1:14:12,  2.05s/it]
 82%|████████▏ | 9830/12000 [5:47:46<1:07:14,  1.86s/it]
                                                        

 82%|████████▏ | 9830/12000 [5:47:46<1:07:14,  1.86s/it]
 82%|████████▏ | 9831/12000 [5:47:48<1:04:37,  1.79s/it]
                                                        

 82%|████████▏ | 9831/12000 [5:47:48<1:04:37,  1.79s/it]
 82%|████████▏ | 9832/12000 [5:47:50<1:06:42,  1.85s/it]
                                                        

 82%|████████▏ | 9832/12000 [5:47:50<1:06:42,  1.85s/it]
 82%|████████▏ | 9833/12000 [5:47:51<1:03:21,  1.75s/it]
                                                        

 82%|████████▏ | 9833/12000 [5:47:51<1:03:21,  1.75s/it]
 82%|████████▏ | 9834/12000 [5:47:53<1:02:20,  1.73s/it]
                                                        

 82%|████████▏ | 9834/12000 [5:47:53<1:02:20,  1.73s/it]
 82%|████████▏ | 9835/12000 [5:47:55<1:11:23,  1.98s/it]
                                                        

 82%|████████▏ | 9835/12000 [5:47:55<1:11:23,  1.98s/it]
 82%|████████▏ | 9836/12000 [5:47:57<1:09:09,  1.92s/it]
                                                        

 82%|████████▏ | 9836/12000 [5:47:57<1:09:09,  1.92s/it]
 82%|████████▏ | 9837/12000 [5:47:59<1:05:56,  1.83s/it]
                                                        

 82%|████████▏ | 9837/12000 [5:47:59<1:05:56,  1.83s/it]
 82%|████████▏ | 9838/12000 [5:48:00<1:02:29,  1.73s/it]
                                                        

 82%|████████▏ | 9838/12000 [5:48:00<1:02:29,  1.73s/it]
 82%|████████▏ | 9839/12000 [5:48:03<1:13:17,  2.03s/it]
                                                        

 82%|████████▏ | 9839/12000 [5:48:03<1:13:17,  2.03s/it]
 82%|████████▏ | 9840/12000 [5:48:05<1:17:38,  2.16s/it]
                                                        

 82%|████████▏ | 9840/12000 [5:48:05<1:17:38,  2.16s/it]
 82%|████████▏ | 9841/12000 [5:48:07<1:12:33,  2.02s/it]
                                                        

 82%|████████▏ | 9841/12000 [5:48:07<1:12:33,  2.02s/it]
 82%|████████▏ | 9842/12000 [5:48:11<1:35:27,  2.65s/it]
                                                        

 82%|████████▏ | 9842/12000 [5:48:11<1:35:27,  2.65s/it]
 82%|████████▏ | 9843/12000 [5:48:13<1:27:05,  2.42s/it]
                                                        

 82%|████████▏ | 9843/12000 [5:48:13<1:27:05,  2.42s/it]
 82%|████████▏ | 9844/12000 [5:48:15<1:20:47,  2.25s/it]
                                                        

 82%|████████▏ | 9844/12000 [5:48:15<1:20:47,  2.25s/it]
 82%|████████▏ | 9845/12000 [5:48:17<1:20:08,  2.23s/it]
                                                        

 82%|████████▏ | 9845/12000 [5:48:17<1:20:08,  2.23s/it]
 82%|████████▏ | 9846/12000 [5:48:19<1:13:09,  2.04s/it]
                                                        

 82%|████████▏ | 9846/12000 [5:48:19<1:13:09,  2.04s/it]
 82%|████████▏ | 9847/12000 [5:48:20<1:07:22,  1.88s/it]
                                                        

 82%|████████▏ | 9847/12000 [5:48:20<1:07:22,  1.88s/it]
 82%|████████▏ | 9848/12000 [5:48:22<1:02:57,  1.76s/it]
                                                        

 82%|████████▏ | 9848/12000 [5:48:22<1:02:57,  1.76s/it]
 82%|████████▏ | 9849/12000 [5:48:25<1:19:32,  2.22s/it]
                                                        

 82%|████████▏ | 9849/12000 [5:48:25<1:19:32,  2.22s/it]
 82%|████████▏ | 9850/12000 [5:48:27<1:15:54,  2.12s/it]
                                                        

 82%|████████▏ | 9850/12000 [5:48:27<1:15:54,  2.12s/it]
 82%|████████▏ | 9851/12000 [5:48:31<1:34:08,  2.63s/it]
                                                        

 82%|████████▏ | 9851/12000 [5:48:31<1:34:08,  2.63s/it]
 82%|████████▏ | 9852/12000 [5:48:34<1:42:29,  2.86s/it]
                                                        

 82%|████████▏ | 9852/12000 [5:48:34<1:42:29,  2.86s/it]
 82%|████████▏ | 9853/12000 [5:48:36<1:30:02,  2.52s/it]
                                                        

 82%|████████▏ | 9853/12000 [5:48:36<1:30:02,  2.52s/it]
 82%|████████▏ | 9854/12000 [5:48:39<1:37:53,  2.74s/it]
                                                        

 82%|████████▏ | 9854/12000 [5:48:39<1:37:53,  2.74s/it]
 82%|████████▏ | 9855/12000 [5:48:41<1:30:26,  2.53s/it]
                                                        

 82%|████████▏ | 9855/12000 [5:48:41<1:30:26,  2.53s/it]
 82%|████████▏ | 9856/12000 [5:48:43<1:19:28,  2.22s/it]
                                                        

 82%|████████▏ | 9856/12000 [5:48:43<1:19:28,  2.22s/it]
 82%|████████▏ | 9857/12000 [5:48:45<1:15:03,  2.10s/it]
                                                        

 82%|████████▏ | 9857/12000 [5:48:45<1:15:03,  2.10s/it]
 82%|████████▏ | 9858/12000 [5:48:47<1:14:38,  2.09s/it]
                                                        

 82%|████████▏ | 9858/12000 [5:48:47<1:14:38,  2.09s/it]
 82%|████████▏ | 9859/12000 [5:48:48<1:09:34,  1.95s/it]
                                                        

 82%|████████▏ | 9859/12000 [5:48:48<1:09:34,  1.95s/it]
 82%|████████▏ | 9860/12000 [5:48:51<1:14:05,  2.08s/it]
                                                        

 82%|████████▏ | 9860/12000 [5:48:51<1:14:05,  2.08s/it]
 82%|████████▏ | 9861/12000 [5:48:53<1:12:01,  2.02s/it]
                                                        

 82%|████████▏ | 9861/12000 [5:48:53<1:12:01,  2.02s/it]
 82%|████████▏ | 9862/12000 [5:48:54<1:06:21,  1.86s/it]
                                                        

 82%|████████▏ | 9862/12000 [5:48:54<1:06:21,  1.86s/it]
 82%|████████▏ | 9863/12000 [5:48:55<1:00:56,  1.71s/it]
                                                        

 82%|████████▏ | 9863/12000 [5:48:55<1:00:56,  1.71s/it]
 82%|████████▏ | 9864/12000 [5:49:02<1:51:33,  3.13s/it]
                                                        

 82%|████████▏ | 9864/12000 [5:49:02<1:51:33,  3.13s/it]
 82%|████████▏ | 9865/12000 [5:49:03<1:34:14,  2.65s/it]
                                                        

 82%|████████▏ | 9865/12000 [5:49:03<1:34:14,  2.65s/it]
 82%|████████▏ | 9866/12000 [5:49:05<1:23:57,  2.36s/it]
                                                        

 82%|████████▏ | 9866/12000 [5:49:05<1:23:57,  2.36s/it]
 82%|████████▏ | 9867/12000 [5:49:07<1:15:16,  2.12s/it]
                                                        

 82%|████████▏ | 9867/12000 [5:49:07<1:15:16,  2.12s/it]
 82%|████████▏ | 9868/12000 [5:49:08<1:10:38,  1.99s/it]
                                                        

 82%|████████▏ | 9868/12000 [5:49:08<1:10:38,  1.99s/it]
 82%|████████▏ | 9869/12000 [5:49:10<1:05:41,  1.85s/it]
                                                        

 82%|████████▏ | 9869/12000 [5:49:10<1:05:41,  1.85s/it]
 82%|████████▏ | 9870/12000 [5:49:13<1:17:07,  2.17s/it]
                                                        

 82%|████████▏ | 9870/12000 [5:49:13<1:17:07,  2.17s/it]
 82%|████████▏ | 9871/12000 [5:49:14<1:10:34,  1.99s/it]
                                                        

 82%|████████▏ | 9871/12000 [5:49:14<1:10:34,  1.99s/it]
 82%|████████▏ | 9872/12000 [5:49:16<1:05:10,  1.84s/it]
                                                        

 82%|████████▏ | 9872/12000 [5:49:16<1:05:10,  1.84s/it]
 82%|████████▏ | 9873/12000 [5:49:18<1:06:18,  1.87s/it]
                                                        

 82%|████████▏ | 9873/12000 [5:49:18<1:06:18,  1.87s/it]
 82%|████████▏ | 9874/12000 [5:49:21<1:19:37,  2.25s/it]
                                                        
{'loss': '0.8538', 'grad_norm': '0.4175', 'learning_rate': '1.742e-05', 'entropy': '0.9445', 'num_tokens': '5.291e+07', 'mean_token_accuracy': '0.7542', 'epoch': '0.08697'}
{'loss': '0.7824', 'grad_norm': '0.4484', 'learning_rate': '1.74e-05', 'entropy': '0.7939', 'num_tokens': '5.292e+07', 'mean_token_accuracy': '0.7986', 'epoch': '0.08698'}
{'loss': '0.6847', 'grad_norm': '0.4033', 'learning_rate': '1.738e-05', 'entropy': '0.6324', 'num_tokens': '5.292e+07', 'mean_token_accuracy': '0.8294', 'epoch': '0.08698'}
{'loss': '0.79', 'grad_norm': '0.3855', 'learning_rate': '1.737e-05', 'entropy': '0.8242', 'num_tokens': '5.293e+07', 'mean_token_accuracy': '0.7942', 'epoch': '0.08699'}
{'loss': '0.9994', 'grad_norm': '0.5012', 'learning_rate': '1.735e-05', 'entropy': '0.9787', 'num_tokens': '5.293e+07', 'mean_token_accuracy': '0.7699', 'epoch': '0.087'}
{'loss': '1.193', 'grad_norm': '0.542', 'learning_rate': '1.734e-05', 'entropy': '1.103', 'num_tokens': '5.294e+07', 'mean_token_accuracy': '0.7403', 'epoch': '0.08701'}
{'loss': '1.072', 'grad_norm': '0.476', 'learning_rate': '1.732e-05', 'entropy': '1.021', 'num_tokens': '5.294e+07', 'mean_token_accuracy': '0.7419', 'epoch': '0.08702'}
{'loss': '1.168', 'grad_norm': '0.4542', 'learning_rate': '1.731e-05', 'entropy': '1.156', 'num_tokens': '5.294e+07', 'mean_token_accuracy': '0.7097', 'epoch': '0.08703'}
{'loss': '0.7269', 'grad_norm': '0.3752', 'learning_rate': '1.729e-05', 'entropy': '0.754', 'num_tokens': '5.295e+07', 'mean_token_accuracy': '0.8123', 'epoch': '0.08704'}
{'loss': '0.7054', 'grad_norm': '0.3239', 'learning_rate': '1.728e-05', 'entropy': '0.6854', 'num_tokens': '5.296e+07', 'mean_token_accuracy': '0.8275', 'epoch': '0.08705'}
{'loss': '0.7967', 'grad_norm': '0.47', 'learning_rate': '1.726e-05', 'entropy': '0.7295', 'num_tokens': '5.296e+07', 'mean_token_accuracy': '0.8033', 'epoch': '0.08706'}
{'loss': '0.7138', 'grad_norm': '0.3793', 'learning_rate': '1.725e-05', 'entropy': '0.7885', 'num_tokens': '5.297e+07', 'mean_token_accuracy': '0.7828', 'epoch': '0.08706'}
{'loss': '0.8327', 'grad_norm': '0.3679', 'learning_rate': '1.723e-05', 'entropy': '0.8003', 'num_tokens': '5.297e+07', 'mean_token_accuracy': '0.7901', 'epoch': '0.08707'}
{'loss': '0.5244', 'grad_norm': '0.3403', 'learning_rate': '1.721e-05', 'entropy': '0.4983', 'num_tokens': '5.298e+07', 'mean_token_accuracy': '0.8692', 'epoch': '0.08708'}
{'loss': '0.8977', 'grad_norm': '0.3987', 'learning_rate': '1.72e-05', 'entropy': '0.8537', 'num_tokens': '5.298e+07', 'mean_token_accuracy': '0.7807', 'epoch': '0.08709'}
{'loss': '0.8642', 'grad_norm': '0.4353', 'learning_rate': '1.718e-05', 'entropy': '0.8402', 'num_tokens': '5.299e+07', 'mean_token_accuracy': '0.7886', 'epoch': '0.0871'}
{'loss': '0.5072', 'grad_norm': '0.3179', 'learning_rate': '1.717e-05', 'entropy': '0.5904', 'num_tokens': '5.299e+07', 'mean_token_accuracy': '0.8486', 'epoch': '0.08711'}
{'loss': '0.9146', 'grad_norm': '0.429', 'learning_rate': '1.715e-05', 'entropy': '0.9645', 'num_tokens': '5.3e+07', 'mean_token_accuracy': '0.7439', 'epoch': '0.08712'}
{'loss': '1.047', 'grad_norm': '0.3668', 'learning_rate': '1.714e-05', 'entropy': '0.9071', 'num_tokens': '5.301e+07', 'mean_token_accuracy': '0.7747', 'epoch': '0.08713'}
{'loss': '0.79', 'grad_norm': '0.4484', 'learning_rate': '1.712e-05', 'entropy': '0.8406', 'num_tokens': '5.301e+07', 'mean_token_accuracy': '0.7819', 'epoch': '0.08714'}
{'loss': '0.9876', 'grad_norm': '0.4243', 'learning_rate': '1.711e-05', 'entropy': '0.9525', 'num_tokens': '5.302e+07', 'mean_token_accuracy': '0.7601', 'epoch': '0.08714'}
{'loss': '0.6727', 'grad_norm': '0.3749', 'learning_rate': '1.709e-05', 'entropy': '0.6872', 'num_tokens': '5.302e+07', 'mean_token_accuracy': '0.8168', 'epoch': '0.08715'}
{'loss': '0.4549', 'grad_norm': '0.3675', 'learning_rate': '1.708e-05', 'entropy': '0.4218', 'num_tokens': '5.303e+07', 'mean_token_accuracy': '0.8851', 'epoch': '0.08716'}
{'loss': '0.9245', 'grad_norm': '0.3895', 'learning_rate': '1.706e-05', 'entropy': '0.954', 'num_tokens': '5.303e+07', 'mean_token_accuracy': '0.7741', 'epoch': '0.08717'}
{'loss': '0.9749', 'grad_norm': '0.4157', 'learning_rate': '1.704e-05', 'entropy': '0.9353', 'num_tokens': '5.304e+07', 'mean_token_accuracy': '0.7697', 'epoch': '0.08718'}
{'loss': '1.253', 'grad_norm': '0.4406', 'learning_rate': '1.703e-05', 'entropy': '1.021', 'num_tokens': '5.305e+07', 'mean_token_accuracy': '0.7431', 'epoch': '0.08719'}
{'loss': '0.7584', 'grad_norm': '0.4633', 'learning_rate': '1.701e-05', 'entropy': '0.8114', 'num_tokens': '5.305e+07', 'mean_token_accuracy': '0.7986', 'epoch': '0.0872'}
{'loss': '0.9886', 'grad_norm': '0.358', 'learning_rate': '1.7e-05', 'entropy': '0.8965', 'num_tokens': '5.306e+07', 'mean_token_accuracy': '0.7729', 'epoch': '0.08721'}
{'loss': '0.7748', 'grad_norm': '0.3809', 'learning_rate': '1.698e-05', 'entropy': '0.736', 'num_tokens': '5.306e+07', 'mean_token_accuracy': '0.7968', 'epoch': '0.08722'}
{'loss': '0.7638', 'grad_norm': '0.4796', 'learning_rate': '1.697e-05', 'entropy': '0.7758', 'num_tokens': '5.307e+07', 'mean_token_accuracy': '0.8042', 'epoch': '0.08722'}
{'loss': '0.7472', 'grad_norm': '0.4599', 'learning_rate': '1.695e-05', 'entropy': '0.763', 'num_tokens': '5.307e+07', 'mean_token_accuracy': '0.7925', 'epoch': '0.08723'}
{'loss': '1.359', 'grad_norm': '0.4841', 'learning_rate': '1.694e-05', 'entropy': '1.286', 'num_tokens': '5.308e+07', 'mean_token_accuracy': '0.7181', 'epoch': '0.08724'}
{'loss': '0.8186', 'grad_norm': '0.3788', 'learning_rate': '1.692e-05', 'entropy': '0.8073', 'num_tokens': '5.308e+07', 'mean_token_accuracy': '0.8049', 'epoch': '0.08725'}
{'loss': '0.9056', 'grad_norm': '0.5028', 'learning_rate': '1.691e-05', 'entropy': '0.9091', 'num_tokens': '5.309e+07', 'mean_token_accuracy': '0.7805', 'epoch': '0.08726'}
{'loss': '0.7609', 'grad_norm': '0.3972', 'learning_rate': '1.689e-05', 'entropy': '0.838', 'num_tokens': '5.31e+07', 'mean_token_accuracy': '0.7922', 'epoch': '0.08727'}
{'loss': '0.8056', 'grad_norm': '0.3867', 'learning_rate': '1.688e-05', 'entropy': '0.8143', 'num_tokens': '5.31e+07', 'mean_token_accuracy': '0.7927', 'epoch': '0.08728'}
{'loss': '0.6699', 'grad_norm': '0.3111', 'learning_rate': '1.686e-05', 'entropy': '0.7181', 'num_tokens': '5.31e+07', 'mean_token_accuracy': '0.8074', 'epoch': '0.08729'}
{'loss': '1.135', 'grad_norm': '0.3686', 'learning_rate': '1.685e-05', 'entropy': '0.9726', 'num_tokens': '5.311e+07', 'mean_token_accuracy': '0.772', 'epoch': '0.08729'}
{'loss': '0.7103', 'grad_norm': '0.4588', 'learning_rate': '1.683e-05', 'entropy': '0.7488', 'num_tokens': '5.312e+07', 'mean_token_accuracy': '0.8064', 'epoch': '0.0873'}
{'loss': '0.7596', 'grad_norm': '0.4412', 'learning_rate': '1.681e-05', 'entropy': '0.8094', 'num_tokens': '5.312e+07', 'mean_token_accuracy': '0.8006', 'epoch': '0.08731'}
{'loss': '0.7112', 'grad_norm': '0.364', 'learning_rate': '1.68e-05', 'entropy': '0.7837', 'num_tokens': '5.313e+07', 'mean_token_accuracy': '0.7908', 'epoch': '0.08732'}
{'loss': '0.6554', 'grad_norm': '0.4115', 'learning_rate': '1.678e-05', 'entropy': '0.685', 'num_tokens': '5.313e+07', 'mean_token_accuracy': '0.8248', 'epoch': '0.08733'}
{'loss': '0.9119', 'grad_norm': '0.4868', 'learning_rate': '1.677e-05', 'entropy': '0.9406', 'num_tokens': '5.313e+07', 'mean_token_accuracy': '0.7711', 'epoch': '0.08734'}
{'loss': '0.7187', 'grad_norm': '0.3855', 'learning_rate': '1.675e-05', 'entropy': '0.722', 'num_tokens': '5.314e+07', 'mean_token_accuracy': '0.8124', 'epoch': '0.08735'}
{'loss': '0.6912', 'grad_norm': '0.335', 'learning_rate': '1.674e-05', 'entropy': '0.7125', 'num_tokens': '5.315e+07', 'mean_token_accuracy': '0.8195', 'epoch': '0.08736'}
{'loss': '0.5698', 'grad_norm': '0.3452', 'learning_rate': '1.672e-05', 'entropy': '0.5524', 'num_tokens': '5.315e+07', 'mean_token_accuracy': '0.859', 'epoch': '0.08737'}
{'loss': '0.7956', 'grad_norm': '0.4035', 'learning_rate': '1.671e-05', 'entropy': '0.7627', 'num_tokens': '5.316e+07', 'mean_token_accuracy': '0.8113', 'epoch': '0.08737'}

 82%|████████▏ | 9874/12000 [5:49:21<1:19:37,  2.25s/it]
 82%|████████▏ | 9875/12000 [5:49:24<1:31:49,  2.59s/it]
                                                        

 82%|████████▏ | 9875/12000 [5:49:24<1:31:49,  2.59s/it]
 82%|████████▏ | 9876/12000 [5:49:26<1:23:39,  2.36s/it]
                                                        

 82%|████████▏ | 9876/12000 [5:49:26<1:23:39,  2.36s/it]
 82%|████████▏ | 9877/12000 [5:49:28<1:17:39,  2.19s/it]
                                                        

 82%|████████▏ | 9877/12000 [5:49:28<1:17:39,  2.19s/it]
 82%|████████▏ | 9878/12000 [5:49:30<1:19:53,  2.26s/it]
                                                        

 82%|████████▏ | 9878/12000 [5:49:30<1:19:53,  2.26s/it]
 82%|████████▏ | 9879/12000 [5:49:32<1:17:46,  2.20s/it]
                                                        

 82%|████████▏ | 9879/12000 [5:49:32<1:17:46,  2.20s/it]
 82%|████████▏ | 9880/12000 [5:49:34<1:14:13,  2.10s/it]
                                                        

 82%|████████▏ | 9880/12000 [5:49:34<1:14:13,  2.10s/it]
 82%|████████▏ | 9881/12000 [5:49:36<1:06:48,  1.89s/it]
                                                        

 82%|████████▏ | 9881/12000 [5:49:36<1:06:48,  1.89s/it]
 82%|████████▏ | 9882/12000 [5:49:38<1:08:22,  1.94s/it]
                                                        

 82%|████████▏ | 9882/12000 [5:49:38<1:08:22,  1.94s/it]
 82%|████████▏ | 9883/12000 [5:49:39<1:05:58,  1.87s/it]
                                                        

 82%|████████▏ | 9883/12000 [5:49:39<1:05:58,  1.87s/it]
 82%|████████▏ | 9884/12000 [5:49:41<1:03:05,  1.79s/it]
                                                        

 82%|████████▏ | 9884/12000 [5:49:41<1:03:05,  1.79s/it]
 82%|████████▏ | 9885/12000 [5:49:42<1:00:21,  1.71s/it]
                                                        

 82%|████████▏ | 9885/12000 [5:49:42<1:00:21,  1.71s/it]
 82%|████████▏ | 9886/12000 [5:49:46<1:20:49,  2.29s/it]
                                                        

 82%|████████▏ | 9886/12000 [5:49:46<1:20:49,  2.29s/it]
 82%|████████▏ | 9887/12000 [5:49:48<1:19:16,  2.25s/it]
                                                        

 82%|████████▏ | 9887/12000 [5:49:48<1:19:16,  2.25s/it]
 82%|████████▏ | 9888/12000 [5:49:52<1:32:01,  2.61s/it]
                                                        

 82%|████████▏ | 9888/12000 [5:49:52<1:32:01,  2.61s/it]
 82%|████████▏ | 9889/12000 [5:49:53<1:19:42,  2.27s/it]
                                                        

 82%|████████▏ | 9889/12000 [5:49:53<1:19:42,  2.27s/it]
 82%|████████▏ | 9890/12000 [5:49:55<1:17:27,  2.20s/it]
                                                        

 82%|████████▏ | 9890/12000 [5:49:55<1:17:27,  2.20s/it]
 82%|████████▏ | 9891/12000 [5:49:57<1:11:52,  2.04s/it]
                                                        

 82%|████████▏ | 9891/12000 [5:49:57<1:11:52,  2.04s/it]
 82%|████████▏ | 9892/12000 [5:49:59<1:07:39,  1.93s/it]
                                                        

 82%|████████▏ | 9892/12000 [5:49:59<1:07:39,  1.93s/it]
 82%|████████▏ | 9893/12000 [5:50:00<1:02:59,  1.79s/it]
                                                        

 82%|████████▏ | 9893/12000 [5:50:00<1:02:59,  1.79s/it]
 82%|████████▏ | 9894/12000 [5:50:02<1:01:27,  1.75s/it]
                                                        

 82%|████████▏ | 9894/12000 [5:50:02<1:01:27,  1.75s/it]
 82%|████████▏ | 9895/12000 [5:50:03<1:00:40,  1.73s/it]
                                                        

 82%|████████▏ | 9895/12000 [5:50:03<1:00:40,  1.73s/it]
 82%|████████▏ | 9896/12000 [5:50:07<1:19:54,  2.28s/it]
                                                        

 82%|████████▏ | 9896/12000 [5:50:07<1:19:54,  2.28s/it]
 82%|████████▏ | 9897/12000 [5:50:08<1:11:36,  2.04s/it]
                                                        

 82%|████████▏ | 9897/12000 [5:50:08<1:11:36,  2.04s/it]
 82%|████████▏ | 9898/12000 [5:50:10<1:10:54,  2.02s/it]
                                                        

 82%|████████▏ | 9898/12000 [5:50:10<1:10:54,  2.02s/it]
 82%|████████▏ | 9899/12000 [5:50:12<1:05:37,  1.87s/it]
                                                        

 82%|████████▏ | 9899/12000 [5:50:12<1:05:37,  1.87s/it]
 82%|████████▎ | 9900/12000 [5:50:16<1:23:17,  2.38s/it]
                                                        

 82%|████████▎ | 9900/12000 [5:50:16<1:23:17,  2.38s/it]
 83%|████████▎ | 9901/12000 [5:50:19<1:33:58,  2.69s/it]
                                                        

 83%|████████▎ | 9901/12000 [5:50:19<1:33:58,  2.69s/it]
 83%|████████▎ | 9902/12000 [5:50:20<1:21:56,  2.34s/it]
                                                        

 83%|████████▎ | 9902/12000 [5:50:20<1:21:56,  2.34s/it]
 83%|████████▎ | 9903/12000 [5:50:22<1:15:45,  2.17s/it]
                                                        

 83%|████████▎ | 9903/12000 [5:50:22<1:15:45,  2.17s/it]
 83%|████████▎ | 9904/12000 [5:50:24<1:12:12,  2.07s/it]
                                                        

 83%|████████▎ | 9904/12000 [5:50:24<1:12:12,  2.07s/it]
 83%|████████▎ | 9905/12000 [5:50:26<1:07:46,  1.94s/it]
                                                        

 83%|████████▎ | 9905/12000 [5:50:26<1:07:46,  1.94s/it]
 83%|████████▎ | 9906/12000 [5:50:27<1:04:26,  1.85s/it]
                                                        

 83%|████████▎ | 9906/12000 [5:50:27<1:04:26,  1.85s/it]
 83%|████████▎ | 9907/12000 [5:50:29<1:05:46,  1.89s/it]
                                                        

 83%|████████▎ | 9907/12000 [5:50:29<1:05:46,  1.89s/it]
 83%|████████▎ | 9908/12000 [5:50:33<1:24:03,  2.41s/it]
                                                        

 83%|████████▎ | 9908/12000 [5:50:33<1:24:03,  2.41s/it]
 83%|████████▎ | 9909/12000 [5:50:35<1:19:50,  2.29s/it]
                                                        

 83%|████████▎ | 9909/12000 [5:50:35<1:19:50,  2.29s/it]
 83%|████████▎ | 9910/12000 [5:50:37<1:22:32,  2.37s/it]
                                                        

 83%|████████▎ | 9910/12000 [5:50:37<1:22:32,  2.37s/it]
 83%|████████▎ | 9911/12000 [5:50:40<1:21:25,  2.34s/it]
                                                        

 83%|████████▎ | 9911/12000 [5:50:40<1:21:25,  2.34s/it]
 83%|████████▎ | 9912/12000 [5:50:41<1:11:54,  2.07s/it]
                                                        

 83%|████████▎ | 9912/12000 [5:50:41<1:11:54,  2.07s/it]
 83%|████████▎ | 9913/12000 [5:50:46<1:41:21,  2.91s/it]
                                                        

 83%|████████▎ | 9913/12000 [5:50:46<1:41:21,  2.91s/it]
 83%|████████▎ | 9914/12000 [5:50:49<1:38:59,  2.85s/it]
                                                        

 83%|████████▎ | 9914/12000 [5:50:49<1:38:59,  2.85s/it]
 83%|████████▎ | 9915/12000 [5:50:52<1:43:21,  2.97s/it]
                                                        

 83%|████████▎ | 9915/12000 [5:50:52<1:43:21,  2.97s/it]
 83%|████████▎ | 9916/12000 [5:50:54<1:28:18,  2.54s/it]
                                                        

 83%|████████▎ | 9916/12000 [5:50:54<1:28:18,  2.54s/it]
 83%|████████▎ | 9917/12000 [5:50:55<1:18:09,  2.25s/it]
                                                        

 83%|████████▎ | 9917/12000 [5:50:55<1:18:09,  2.25s/it]
 83%|████████▎ | 9918/12000 [5:50:57<1:13:41,  2.12s/it]
                                                        

 83%|████████▎ | 9918/12000 [5:50:57<1:13:41,  2.12s/it]
 83%|████████▎ | 9919/12000 [5:50:59<1:10:53,  2.04s/it]
                                                        

 83%|████████▎ | 9919/12000 [5:50:59<1:10:53,  2.04s/it]
 83%|████████▎ | 9920/12000 [5:51:00<1:05:59,  1.90s/it]
                                                        

 83%|████████▎ | 9920/12000 [5:51:00<1:05:59,  1.90s/it]
 83%|████████▎ | 9921/12000 [5:51:02<1:00:55,  1.76s/it]
                                                        
{'loss': '0.6624', 'grad_norm': '0.3135', 'learning_rate': '1.669e-05', 'entropy': '0.7183', 'num_tokens': '5.316e+07', 'mean_token_accuracy': '0.8136', 'epoch': '0.08738'}
{'loss': '1.098', 'grad_norm': '0.4922', 'learning_rate': '1.668e-05', 'entropy': '1.014', 'num_tokens': '5.317e+07', 'mean_token_accuracy': '0.7398', 'epoch': '0.08739'}
{'loss': '0.6171', 'grad_norm': '0.4414', 'learning_rate': '1.666e-05', 'entropy': '0.5933', 'num_tokens': '5.318e+07', 'mean_token_accuracy': '0.8297', 'epoch': '0.0874'}
{'loss': '0.9065', 'grad_norm': '0.5143', 'learning_rate': '1.665e-05', 'entropy': '0.9468', 'num_tokens': '5.318e+07', 'mean_token_accuracy': '0.7736', 'epoch': '0.08741'}
{'loss': '0.5547', 'grad_norm': '0.3356', 'learning_rate': '1.663e-05', 'entropy': '0.5729', 'num_tokens': '5.319e+07', 'mean_token_accuracy': '0.8488', 'epoch': '0.08742'}
{'loss': '0.8574', 'grad_norm': '0.461', 'learning_rate': '1.662e-05', 'entropy': '0.7884', 'num_tokens': '5.319e+07', 'mean_token_accuracy': '0.806', 'epoch': '0.08743'}
{'loss': '0.5954', 'grad_norm': '0.3542', 'learning_rate': '1.66e-05', 'entropy': '0.5818', 'num_tokens': '5.32e+07', 'mean_token_accuracy': '0.8427', 'epoch': '0.08744'}
{'loss': '0.7128', 'grad_norm': '0.3835', 'learning_rate': '1.659e-05', 'entropy': '0.8572', 'num_tokens': '5.32e+07', 'mean_token_accuracy': '0.7824', 'epoch': '0.08745'}
{'loss': '1.518', 'grad_norm': '0.5371', 'learning_rate': '1.657e-05', 'entropy': '1.146', 'num_tokens': '5.321e+07', 'mean_token_accuracy': '0.7427', 'epoch': '0.08745'}
{'loss': '0.6369', 'grad_norm': '0.4567', 'learning_rate': '1.656e-05', 'entropy': '0.6894', 'num_tokens': '5.321e+07', 'mean_token_accuracy': '0.8159', 'epoch': '0.08746'}
{'loss': '0.8945', 'grad_norm': '0.4693', 'learning_rate': '1.654e-05', 'entropy': '0.8485', 'num_tokens': '5.322e+07', 'mean_token_accuracy': '0.7805', 'epoch': '0.08747'}
{'loss': '0.7185', 'grad_norm': '0.5091', 'learning_rate': '1.653e-05', 'entropy': '0.7337', 'num_tokens': '5.322e+07', 'mean_token_accuracy': '0.8076', 'epoch': '0.08748'}
{'loss': '0.9005', 'grad_norm': '0.4334', 'learning_rate': '1.651e-05', 'entropy': '0.838', 'num_tokens': '5.323e+07', 'mean_token_accuracy': '0.7894', 'epoch': '0.08749'}
{'loss': '0.8257', 'grad_norm': '0.3648', 'learning_rate': '1.649e-05', 'entropy': '0.7832', 'num_tokens': '5.323e+07', 'mean_token_accuracy': '0.8047', 'epoch': '0.0875'}
{'loss': '0.8848', 'grad_norm': '0.3748', 'learning_rate': '1.648e-05', 'entropy': '0.8123', 'num_tokens': '5.324e+07', 'mean_token_accuracy': '0.7962', 'epoch': '0.08751'}
{'loss': '0.8426', 'grad_norm': '0.4763', 'learning_rate': '1.646e-05', 'entropy': '0.9329', 'num_tokens': '5.325e+07', 'mean_token_accuracy': '0.7592', 'epoch': '0.08752'}
{'loss': '0.6476', 'grad_norm': '0.3345', 'learning_rate': '1.645e-05', 'entropy': '0.6478', 'num_tokens': '5.325e+07', 'mean_token_accuracy': '0.8434', 'epoch': '0.08752'}
{'loss': '0.8003', 'grad_norm': '0.425', 'learning_rate': '1.643e-05', 'entropy': '0.7836', 'num_tokens': '5.326e+07', 'mean_token_accuracy': '0.8122', 'epoch': '0.08753'}
{'loss': '0.7455', 'grad_norm': '0.371', 'learning_rate': '1.642e-05', 'entropy': '0.7553', 'num_tokens': '5.326e+07', 'mean_token_accuracy': '0.8128', 'epoch': '0.08754'}
{'loss': '0.828', 'grad_norm': '0.351', 'learning_rate': '1.64e-05', 'entropy': '0.8475', 'num_tokens': '5.327e+07', 'mean_token_accuracy': '0.7825', 'epoch': '0.08755'}
{'loss': '0.7826', 'grad_norm': '0.355', 'learning_rate': '1.639e-05', 'entropy': '0.7423', 'num_tokens': '5.327e+07', 'mean_token_accuracy': '0.8024', 'epoch': '0.08756'}
{'loss': '0.7561', 'grad_norm': '0.4921', 'learning_rate': '1.637e-05', 'entropy': '0.7643', 'num_tokens': '5.328e+07', 'mean_token_accuracy': '0.8065', 'epoch': '0.08757'}
{'loss': '0.8083', 'grad_norm': '0.4119', 'learning_rate': '1.636e-05', 'entropy': '0.9452', 'num_tokens': '5.328e+07', 'mean_token_accuracy': '0.761', 'epoch': '0.08758'}
{'loss': '0.694', 'grad_norm': '0.4202', 'learning_rate': '1.634e-05', 'entropy': '0.7106', 'num_tokens': '5.329e+07', 'mean_token_accuracy': '0.8181', 'epoch': '0.08759'}
{'loss': '0.7063', 'grad_norm': '0.3391', 'learning_rate': '1.633e-05', 'entropy': '0.6921', 'num_tokens': '5.329e+07', 'mean_token_accuracy': '0.827', 'epoch': '0.0876'}
{'loss': '0.6952', 'grad_norm': '0.3778', 'learning_rate': '1.631e-05', 'entropy': '0.7092', 'num_tokens': '5.33e+07', 'mean_token_accuracy': '0.8224', 'epoch': '0.0876'}
{'loss': '0.873', 'grad_norm': '0.3261', 'learning_rate': '1.63e-05', 'entropy': '0.8615', 'num_tokens': '5.331e+07', 'mean_token_accuracy': '0.7827', 'epoch': '0.08761'}
{'loss': '0.4857', 'grad_norm': '0.3552', 'learning_rate': '1.628e-05', 'entropy': '0.4902', 'num_tokens': '5.331e+07', 'mean_token_accuracy': '0.8769', 'epoch': '0.08762'}
{'loss': '0.6887', 'grad_norm': '0.4424', 'learning_rate': '1.627e-05', 'entropy': '0.7277', 'num_tokens': '5.332e+07', 'mean_token_accuracy': '0.8146', 'epoch': '0.08763'}
{'loss': '0.6861', 'grad_norm': '0.4086', 'learning_rate': '1.625e-05', 'entropy': '0.7006', 'num_tokens': '5.332e+07', 'mean_token_accuracy': '0.8152', 'epoch': '0.08764'}
{'loss': '0.5862', 'grad_norm': '0.3561', 'learning_rate': '1.624e-05', 'entropy': '0.5794', 'num_tokens': '5.333e+07', 'mean_token_accuracy': '0.8421', 'epoch': '0.08765'}
{'loss': '0.8366', 'grad_norm': '0.4524', 'learning_rate': '1.622e-05', 'entropy': '0.7744', 'num_tokens': '5.333e+07', 'mean_token_accuracy': '0.7963', 'epoch': '0.08766'}
{'loss': '0.7422', 'grad_norm': '0.4075', 'learning_rate': '1.621e-05', 'entropy': '0.7518', 'num_tokens': '5.334e+07', 'mean_token_accuracy': '0.8007', 'epoch': '0.08767'}
{'loss': '0.8633', 'grad_norm': '0.4317', 'learning_rate': '1.619e-05', 'entropy': '0.7466', 'num_tokens': '5.335e+07', 'mean_token_accuracy': '0.8153', 'epoch': '0.08768'}
{'loss': '0.5916', 'grad_norm': '0.3501', 'learning_rate': '1.618e-05', 'entropy': '0.6529', 'num_tokens': '5.335e+07', 'mean_token_accuracy': '0.8323', 'epoch': '0.08768'}
{'loss': '0.6238', 'grad_norm': '0.2756', 'learning_rate': '1.616e-05', 'entropy': '0.6044', 'num_tokens': '5.336e+07', 'mean_token_accuracy': '0.8408', 'epoch': '0.08769'}
{'loss': '0.7504', 'grad_norm': '0.4123', 'learning_rate': '1.615e-05', 'entropy': '0.7233', 'num_tokens': '5.337e+07', 'mean_token_accuracy': '0.811', 'epoch': '0.0877'}
{'loss': '0.9639', 'grad_norm': '0.4657', 'learning_rate': '1.613e-05', 'entropy': '0.8743', 'num_tokens': '5.337e+07', 'mean_token_accuracy': '0.7772', 'epoch': '0.08771'}
{'loss': '0.9464', 'grad_norm': '0.4332', 'learning_rate': '1.612e-05', 'entropy': '1.014', 'num_tokens': '5.338e+07', 'mean_token_accuracy': '0.7625', 'epoch': '0.08772'}
{'loss': '0.8531', 'grad_norm': '0.3628', 'learning_rate': '1.61e-05', 'entropy': '0.8026', 'num_tokens': '5.339e+07', 'mean_token_accuracy': '0.7918', 'epoch': '0.08773'}
{'loss': '0.6796', 'grad_norm': '0.3726', 'learning_rate': '1.609e-05', 'entropy': '0.7168', 'num_tokens': '5.339e+07', 'mean_token_accuracy': '0.8267', 'epoch': '0.08774'}
{'loss': '0.8299', 'grad_norm': '0.3423', 'learning_rate': '1.607e-05', 'entropy': '0.7844', 'num_tokens': '5.34e+07', 'mean_token_accuracy': '0.7942', 'epoch': '0.08775'}
{'loss': '0.7193', 'grad_norm': '0.3965', 'learning_rate': '1.606e-05', 'entropy': '0.7049', 'num_tokens': '5.34e+07', 'mean_token_accuracy': '0.8176', 'epoch': '0.08775'}
{'loss': '0.6186', 'grad_norm': '0.3605', 'learning_rate': '1.604e-05', 'entropy': '0.6493', 'num_tokens': '5.341e+07', 'mean_token_accuracy': '0.8419', 'epoch': '0.08776'}
{'loss': '1.039', 'grad_norm': '0.3939', 'learning_rate': '1.603e-05', 'entropy': '0.9958', 'num_tokens': '5.341e+07', 'mean_token_accuracy': '0.746', 'epoch': '0.08777'}
{'loss': '0.8352', 'grad_norm': '0.4281', 'learning_rate': '1.601e-05', 'entropy': '0.8299', 'num_tokens': '5.342e+07', 'mean_token_accuracy': '0.7806', 'epoch': '0.08778'}
{'loss': '0.8453', 'grad_norm': '0.4528', 'learning_rate': '1.6e-05', 'entropy': '0.8116', 'num_tokens': '5.343e+07', 'mean_token_accuracy': '0.7933', 'epoch': '0.08779'}

 83%|████████▎ | 9921/12000 [5:51:02<1:00:55,  1.76s/it]
 83%|████████▎ | 9922/12000 [5:51:04<1:04:28,  1.86s/it]
                                                        

 83%|████████▎ | 9922/12000 [5:51:04<1:04:28,  1.86s/it]
 83%|████████▎ | 9923/12000 [5:51:07<1:16:30,  2.21s/it]
                                                        

 83%|████████▎ | 9923/12000 [5:51:07<1:16:30,  2.21s/it]
 83%|████████▎ | 9924/12000 [5:51:09<1:16:40,  2.22s/it]
                                                        

 83%|████████▎ | 9924/12000 [5:51:09<1:16:40,  2.22s/it]
 83%|████████▎ | 9925/12000 [5:51:11<1:13:15,  2.12s/it]
                                                        

 83%|████████▎ | 9925/12000 [5:51:11<1:13:15,  2.12s/it]
 83%|████████▎ | 9926/12000 [5:51:14<1:17:33,  2.24s/it]
                                                        

 83%|████████▎ | 9926/12000 [5:51:14<1:17:33,  2.24s/it]
 83%|████████▎ | 9927/12000 [5:51:15<1:11:27,  2.07s/it]
                                                        

 83%|████████▎ | 9927/12000 [5:51:15<1:11:27,  2.07s/it]
 83%|████████▎ | 9928/12000 [5:51:17<1:04:33,  1.87s/it]
                                                        

 83%|████████▎ | 9928/12000 [5:51:17<1:04:33,  1.87s/it]
 83%|████████▎ | 9929/12000 [5:51:19<1:13:33,  2.13s/it]
                                                        

 83%|████████▎ | 9929/12000 [5:51:19<1:13:33,  2.13s/it]
 83%|████████▎ | 9930/12000 [5:51:23<1:29:39,  2.60s/it]
                                                        

 83%|████████▎ | 9930/12000 [5:51:23<1:29:39,  2.60s/it]
 83%|████████▎ | 9931/12000 [5:51:25<1:17:51,  2.26s/it]
                                                        

 83%|████████▎ | 9931/12000 [5:51:25<1:17:51,  2.26s/it]
 83%|████████▎ | 9932/12000 [5:51:26<1:10:54,  2.06s/it]
                                                        

 83%|████████▎ | 9932/12000 [5:51:26<1:10:54,  2.06s/it]
 83%|████████▎ | 9933/12000 [5:51:27<1:03:23,  1.84s/it]
                                                        

 83%|████████▎ | 9933/12000 [5:51:27<1:03:23,  1.84s/it]
 83%|████████▎ | 9934/12000 [5:51:29<57:17,  1.66s/it]  
                                                      

 83%|████████▎ | 9934/12000 [5:51:29<57:17,  1.66s/it]
 83%|████████▎ | 9935/12000 [5:51:31<1:02:36,  1.82s/it]
                                                        

 83%|████████▎ | 9935/12000 [5:51:31<1:02:36,  1.82s/it]
 83%|████████▎ | 9936/12000 [5:51:33<1:06:23,  1.93s/it]
                                                        

 83%|████████▎ | 9936/12000 [5:51:33<1:06:23,  1.93s/it]
 83%|████████▎ | 9937/12000 [5:51:35<1:07:10,  1.95s/it]
                                                        

 83%|████████▎ | 9937/12000 [5:51:35<1:07:10,  1.95s/it]
 83%|████████▎ | 9938/12000 [5:51:37<1:02:37,  1.82s/it]
                                                        

 83%|████████▎ | 9938/12000 [5:51:37<1:02:37,  1.82s/it]
 83%|████████▎ | 9939/12000 [5:51:38<1:00:42,  1.77s/it]
                                                        

 83%|████████▎ | 9939/12000 [5:51:38<1:00:42,  1.77s/it]
 83%|████████▎ | 9940/12000 [5:51:40<1:01:50,  1.80s/it]
                                                        

 83%|████████▎ | 9940/12000 [5:51:40<1:01:50,  1.80s/it]
 83%|████████▎ | 9941/12000 [5:51:43<1:10:24,  2.05s/it]
                                                        

 83%|████████▎ | 9941/12000 [5:51:43<1:10:24,  2.05s/it]
 83%|████████▎ | 9942/12000 [5:51:44<1:04:31,  1.88s/it]
                                                        

 83%|████████▎ | 9942/12000 [5:51:44<1:04:31,  1.88s/it]
 83%|████████▎ | 9943/12000 [5:51:46<59:57,  1.75s/it]  
                                                      

 83%|████████▎ | 9943/12000 [5:51:46<59:57,  1.75s/it]
 83%|████████▎ | 9944/12000 [5:51:47<59:12,  1.73s/it]
                                                      

 83%|████████▎ | 9944/12000 [5:51:47<59:12,  1.73s/it]
 83%|████████▎ | 9945/12000 [5:51:52<1:29:11,  2.60s/it]
                                                        

 83%|████████▎ | 9945/12000 [5:51:52<1:29:11,  2.60s/it]
 83%|████████▎ | 9946/12000 [5:51:54<1:21:32,  2.38s/it]
                                                        

 83%|████████▎ | 9946/12000 [5:51:54<1:21:32,  2.38s/it]
 83%|████████▎ | 9947/12000 [5:51:56<1:14:20,  2.17s/it]
                                                        

 83%|████████▎ | 9947/12000 [5:51:56<1:14:20,  2.17s/it]
 83%|████████▎ | 9948/12000 [5:51:59<1:25:36,  2.50s/it]
                                                        

 83%|████████▎ | 9948/12000 [5:51:59<1:25:36,  2.50s/it]
 83%|████████▎ | 9949/12000 [5:52:01<1:16:42,  2.24s/it]
                                                        

 83%|████████▎ | 9949/12000 [5:52:01<1:16:42,  2.24s/it]
 83%|████████▎ | 9950/12000 [5:52:03<1:18:26,  2.30s/it]
                                                        

 83%|████████▎ | 9950/12000 [5:52:03<1:18:26,  2.30s/it]
 83%|████████▎ | 9951/12000 [5:52:06<1:30:15,  2.64s/it]
                                                        

 83%|████████▎ | 9951/12000 [5:52:06<1:30:15,  2.64s/it]
 83%|████████▎ | 9952/12000 [5:52:11<1:48:32,  3.18s/it]
                                                        

 83%|████████▎ | 9952/12000 [5:52:11<1:48:32,  3.18s/it]
 83%|████████▎ | 9953/12000 [5:52:13<1:42:27,  3.00s/it]
                                                        

 83%|████████▎ | 9953/12000 [5:52:13<1:42:27,  3.00s/it]
 83%|████████▎ | 9954/12000 [5:52:15<1:29:28,  2.62s/it]
                                                        

 83%|████████▎ | 9954/12000 [5:52:15<1:29:28,  2.62s/it]
 83%|████████▎ | 9955/12000 [5:52:17<1:21:25,  2.39s/it]
                                                        

 83%|████████▎ | 9955/12000 [5:52:17<1:21:25,  2.39s/it]
 83%|████████▎ | 9956/12000 [5:52:19<1:18:24,  2.30s/it]
                                                        

 83%|████████▎ | 9956/12000 [5:52:19<1:18:24,  2.30s/it]
 83%|████████▎ | 9957/12000 [5:52:21<1:16:34,  2.25s/it]
                                                        

 83%|████████▎ | 9957/12000 [5:52:21<1:16:34,  2.25s/it]
 83%|████████▎ | 9958/12000 [5:52:23<1:11:41,  2.11s/it]
                                                        

 83%|████████▎ | 9958/12000 [5:52:23<1:11:41,  2.11s/it]
 83%|████████▎ | 9959/12000 [5:52:25<1:07:41,  1.99s/it]
                                                        

 83%|████████▎ | 9959/12000 [5:52:25<1:07:41,  1.99s/it]
 83%|████████▎ | 9960/12000 [5:52:26<1:04:55,  1.91s/it]
                                                        

 83%|████████▎ | 9960/12000 [5:52:26<1:04:55,  1.91s/it]
 83%|████████▎ | 9961/12000 [5:52:28<1:01:49,  1.82s/it]
                                                        

 83%|████████▎ | 9961/12000 [5:52:28<1:01:49,  1.82s/it]
 83%|████████▎ | 9962/12000 [5:52:30<1:01:30,  1.81s/it]
                                                        

 83%|████████▎ | 9962/12000 [5:52:30<1:01:30,  1.81s/it]
 83%|████████▎ | 9963/12000 [5:52:32<1:00:28,  1.78s/it]
                                                        

 83%|████████▎ | 9963/12000 [5:52:32<1:00:28,  1.78s/it]
 83%|████████▎ | 9964/12000 [5:52:33<58:07,  1.71s/it]  
                                                      

 83%|████████▎ | 9964/12000 [5:52:33<58:07,  1.71s/it]
 83%|████████▎ | 9965/12000 [5:52:35<56:24,  1.66s/it]
                                                      

 83%|████████▎ | 9965/12000 [5:52:35<56:24,  1.66s/it]
 83%|████████▎ | 9966/12000 [5:52:36<56:43,  1.67s/it]
                                                      

 83%|████████▎ | 9966/12000 [5:52:36<56:43,  1.67s/it]
 83%|████████▎ | 9967/12000 [5:52:41<1:28:59,  2.63s/it]
                                                        

 83%|████████▎ | 9967/12000 [5:52:41<1:28:59,  2.63s/it]
 83%|████████▎ | 9968/12000 [5:52:43<1:24:19,  2.49s/it]
                                                        
{'loss': '1.178', 'grad_norm': '0.5129', 'learning_rate': '1.598e-05', 'entropy': '1.169', 'num_tokens': '5.343e+07', 'mean_token_accuracy': '0.7085', 'epoch': '0.0878'}
{'loss': '0.8973', 'grad_norm': '0.4343', 'learning_rate': '1.597e-05', 'entropy': '0.8718', 'num_tokens': '5.344e+07', 'mean_token_accuracy': '0.7701', 'epoch': '0.08781'}
{'loss': '0.7756', 'grad_norm': '0.3088', 'learning_rate': '1.595e-05', 'entropy': '0.8638', 'num_tokens': '5.344e+07', 'mean_token_accuracy': '0.7921', 'epoch': '0.08782'}
{'loss': '0.7086', 'grad_norm': '0.4381', 'learning_rate': '1.594e-05', 'entropy': '0.6351', 'num_tokens': '5.345e+07', 'mean_token_accuracy': '0.8402', 'epoch': '0.08783'}
{'loss': '0.9653', 'grad_norm': '0.4945', 'learning_rate': '1.592e-05', 'entropy': '0.879', 'num_tokens': '5.345e+07', 'mean_token_accuracy': '0.7835', 'epoch': '0.08783'}
{'loss': '0.6311', 'grad_norm': '0.2856', 'learning_rate': '1.591e-05', 'entropy': '0.6151', 'num_tokens': '5.346e+07', 'mean_token_accuracy': '0.836', 'epoch': '0.08784'}
{'loss': '0.9538', 'grad_norm': '0.4897', 'learning_rate': '1.589e-05', 'entropy': '0.9606', 'num_tokens': '5.347e+07', 'mean_token_accuracy': '0.7506', 'epoch': '0.08785'}
{'loss': '0.7687', 'grad_norm': '0.3802', 'learning_rate': '1.588e-05', 'entropy': '0.7792', 'num_tokens': '5.347e+07', 'mean_token_accuracy': '0.8044', 'epoch': '0.08786'}
{'loss': '1.081', 'grad_norm': '0.5422', 'learning_rate': '1.586e-05', 'entropy': '0.9566', 'num_tokens': '5.348e+07', 'mean_token_accuracy': '0.7613', 'epoch': '0.08787'}
{'loss': '0.9921', 'grad_norm': '0.4883', 'learning_rate': '1.585e-05', 'entropy': '0.9728', 'num_tokens': '5.348e+07', 'mean_token_accuracy': '0.7528', 'epoch': '0.08788'}
{'loss': '0.6182', 'grad_norm': '0.4402', 'learning_rate': '1.583e-05', 'entropy': '0.6895', 'num_tokens': '5.349e+07', 'mean_token_accuracy': '0.8166', 'epoch': '0.08789'}
{'loss': '0.5015', 'grad_norm': '0.3774', 'learning_rate': '1.582e-05', 'entropy': '0.5124', 'num_tokens': '5.349e+07', 'mean_token_accuracy': '0.8707', 'epoch': '0.0879'}
{'loss': '0.9728', 'grad_norm': '0.4783', 'learning_rate': '1.58e-05', 'entropy': '1.013', 'num_tokens': '5.349e+07', 'mean_token_accuracy': '0.744', 'epoch': '0.08791'}
{'loss': '0.793', 'grad_norm': '0.4381', 'learning_rate': '1.579e-05', 'entropy': '0.8565', 'num_tokens': '5.35e+07', 'mean_token_accuracy': '0.7857', 'epoch': '0.08791'}
{'loss': '1.044', 'grad_norm': '0.4943', 'learning_rate': '1.577e-05', 'entropy': '1.084', 'num_tokens': '5.35e+07', 'mean_token_accuracy': '0.7384', 'epoch': '0.08792'}
{'loss': '1.078', 'grad_norm': '0.4564', 'learning_rate': '1.576e-05', 'entropy': '1.088', 'num_tokens': '5.351e+07', 'mean_token_accuracy': '0.733', 'epoch': '0.08793'}
{'loss': '1.05', 'grad_norm': '0.5205', 'learning_rate': '1.574e-05', 'entropy': '1.042', 'num_tokens': '5.351e+07', 'mean_token_accuracy': '0.761', 'epoch': '0.08794'}
{'loss': '0.8823', 'grad_norm': '0.5101', 'learning_rate': '1.573e-05', 'entropy': '0.9277', 'num_tokens': '5.352e+07', 'mean_token_accuracy': '0.7726', 'epoch': '0.08795'}
{'loss': '1.103', 'grad_norm': '0.4811', 'learning_rate': '1.572e-05', 'entropy': '1.063', 'num_tokens': '5.352e+07', 'mean_token_accuracy': '0.7264', 'epoch': '0.08796'}
{'loss': '0.7245', 'grad_norm': '0.3779', 'learning_rate': '1.57e-05', 'entropy': '0.7127', 'num_tokens': '5.353e+07', 'mean_token_accuracy': '0.8115', 'epoch': '0.08797'}
{'loss': '0.7932', 'grad_norm': '0.3189', 'learning_rate': '1.569e-05', 'entropy': '0.7157', 'num_tokens': '5.353e+07', 'mean_token_accuracy': '0.8425', 'epoch': '0.08798'}
{'loss': '0.6099', 'grad_norm': '0.4182', 'learning_rate': '1.567e-05', 'entropy': '0.6113', 'num_tokens': '5.354e+07', 'mean_token_accuracy': '0.8373', 'epoch': '0.08799'}
{'loss': '0.5763', 'grad_norm': '0.3829', 'learning_rate': '1.566e-05', 'entropy': '0.6393', 'num_tokens': '5.354e+07', 'mean_token_accuracy': '0.8389', 'epoch': '0.08799'}
{'loss': '0.9106', 'grad_norm': '0.4537', 'learning_rate': '1.564e-05', 'entropy': '0.9388', 'num_tokens': '5.354e+07', 'mean_token_accuracy': '0.7769', 'epoch': '0.088'}
{'loss': '0.5482', 'grad_norm': '0.2706', 'learning_rate': '1.563e-05', 'entropy': '0.6479', 'num_tokens': '5.355e+07', 'mean_token_accuracy': '0.8539', 'epoch': '0.08801'}
{'loss': '0.9712', 'grad_norm': '0.3919', 'learning_rate': '1.561e-05', 'entropy': '1.026', 'num_tokens': '5.356e+07', 'mean_token_accuracy': '0.7529', 'epoch': '0.08802'}
{'loss': '0.5935', 'grad_norm': '0.329', 'learning_rate': '1.56e-05', 'entropy': '0.5694', 'num_tokens': '5.356e+07', 'mean_token_accuracy': '0.8439', 'epoch': '0.08803'}
{'loss': '0.9695', 'grad_norm': '0.4076', 'learning_rate': '1.558e-05', 'entropy': '0.8834', 'num_tokens': '5.357e+07', 'mean_token_accuracy': '0.7608', 'epoch': '0.08804'}
{'loss': '0.5669', 'grad_norm': '0.3703', 'learning_rate': '1.557e-05', 'entropy': '0.654', 'num_tokens': '5.358e+07', 'mean_token_accuracy': '0.8295', 'epoch': '0.08805'}
{'loss': '0.7395', 'grad_norm': '0.3833', 'learning_rate': '1.555e-05', 'entropy': '0.7874', 'num_tokens': '5.358e+07', 'mean_token_accuracy': '0.7994', 'epoch': '0.08806'}
{'loss': '0.8187', 'grad_norm': '0.3946', 'learning_rate': '1.554e-05', 'entropy': '0.8558', 'num_tokens': '5.359e+07', 'mean_token_accuracy': '0.7958', 'epoch': '0.08806'}
{'loss': '0.8083', 'grad_norm': '0.3854', 'learning_rate': '1.552e-05', 'entropy': '0.7734', 'num_tokens': '5.36e+07', 'mean_token_accuracy': '0.8036', 'epoch': '0.08807'}
{'loss': '0.7075', 'grad_norm': '0.4476', 'learning_rate': '1.551e-05', 'entropy': '0.7262', 'num_tokens': '5.36e+07', 'mean_token_accuracy': '0.8053', 'epoch': '0.08808'}
{'loss': '1.246', 'grad_norm': '0.4098', 'learning_rate': '1.549e-05', 'entropy': '1.172', 'num_tokens': '5.361e+07', 'mean_token_accuracy': '0.7168', 'epoch': '0.08809'}
{'loss': '0.9051', 'grad_norm': '0.4429', 'learning_rate': '1.548e-05', 'entropy': '0.8698', 'num_tokens': '5.361e+07', 'mean_token_accuracy': '0.7797', 'epoch': '0.0881'}
{'loss': '0.7859', 'grad_norm': '0.5221', 'learning_rate': '1.546e-05', 'entropy': '0.6696', 'num_tokens': '5.362e+07', 'mean_token_accuracy': '0.8272', 'epoch': '0.08811'}
{'loss': '1.197', 'grad_norm': '0.4015', 'learning_rate': '1.545e-05', 'entropy': '1.131', 'num_tokens': '5.362e+07', 'mean_token_accuracy': '0.7342', 'epoch': '0.08812'}
{'loss': '0.9456', 'grad_norm': '0.4189', 'learning_rate': '1.543e-05', 'entropy': '0.9779', 'num_tokens': '5.363e+07', 'mean_token_accuracy': '0.7616', 'epoch': '0.08813'}
{'loss': '0.7998', 'grad_norm': '0.41', 'learning_rate': '1.542e-05', 'entropy': '0.7759', 'num_tokens': '5.363e+07', 'mean_token_accuracy': '0.7946', 'epoch': '0.08814'}
{'loss': '0.5293', 'grad_norm': '0.3453', 'learning_rate': '1.541e-05', 'entropy': '0.522', 'num_tokens': '5.364e+07', 'mean_token_accuracy': '0.8612', 'epoch': '0.08814'}
{'loss': '0.8185', 'grad_norm': '0.4122', 'learning_rate': '1.539e-05', 'entropy': '0.8015', 'num_tokens': '5.364e+07', 'mean_token_accuracy': '0.7921', 'epoch': '0.08815'}
{'loss': '0.5462', 'grad_norm': '0.3472', 'learning_rate': '1.538e-05', 'entropy': '0.5514', 'num_tokens': '5.365e+07', 'mean_token_accuracy': '0.8536', 'epoch': '0.08816'}
{'loss': '0.5597', 'grad_norm': '0.4221', 'learning_rate': '1.536e-05', 'entropy': '0.5457', 'num_tokens': '5.365e+07', 'mean_token_accuracy': '0.8557', 'epoch': '0.08817'}
{'loss': '0.8027', 'grad_norm': '0.3918', 'learning_rate': '1.535e-05', 'entropy': '0.8657', 'num_tokens': '5.366e+07', 'mean_token_accuracy': '0.7846', 'epoch': '0.08818'}
{'loss': '1.153', 'grad_norm': '0.4903', 'learning_rate': '1.533e-05', 'entropy': '1.045', 'num_tokens': '5.366e+07', 'mean_token_accuracy': '0.7441', 'epoch': '0.08819'}
{'loss': '0.6487', 'grad_norm': '0.4184', 'learning_rate': '1.532e-05', 'entropy': '0.7672', 'num_tokens': '5.367e+07', 'mean_token_accuracy': '0.8043', 'epoch': '0.0882'}
{'loss': '1.426', 'grad_norm': '0.3885', 'learning_rate': '1.53e-05', 'entropy': '1.098', 'num_tokens': '5.368e+07', 'mean_token_accuracy': '0.7502', 'epoch': '0.08821'}

 83%|████████▎ | 9968/12000 [5:52:43<1:24:19,  2.49s/it]
 83%|████████▎ | 9969/12000 [5:52:45<1:15:24,  2.23s/it]
                                                        

 83%|████████▎ | 9969/12000 [5:52:45<1:15:24,  2.23s/it]
 83%|████████▎ | 9970/12000 [5:52:47<1:09:06,  2.04s/it]
                                                        

 83%|████████▎ | 9970/12000 [5:52:47<1:09:06,  2.04s/it]
 83%|████████▎ | 9971/12000 [5:52:48<1:03:04,  1.87s/it]
                                                        

 83%|████████▎ | 9971/12000 [5:52:48<1:03:04,  1.87s/it]
 83%|████████▎ | 9972/12000 [5:52:49<58:58,  1.74s/it]  
                                                      

 83%|████████▎ | 9972/12000 [5:52:49<58:58,  1.74s/it]
 83%|████████▎ | 9973/12000 [5:52:51<58:04,  1.72s/it]
                                                      

 83%|████████▎ | 9973/12000 [5:52:51<58:04,  1.72s/it]
 83%|████████▎ | 9974/12000 [5:52:55<1:14:46,  2.21s/it]
                                                        

 83%|████████▎ | 9974/12000 [5:52:55<1:14:46,  2.21s/it]
 83%|████████▎ | 9975/12000 [5:52:57<1:13:06,  2.17s/it]
                                                        

 83%|████████▎ | 9975/12000 [5:52:57<1:13:06,  2.17s/it]
 83%|████████▎ | 9976/12000 [5:52:58<1:05:56,  1.95s/it]
                                                        

 83%|████████▎ | 9976/12000 [5:52:58<1:05:56,  1.95s/it]
 83%|████████▎ | 9977/12000 [5:53:00<1:01:24,  1.82s/it]
                                                        

 83%|████████▎ | 9977/12000 [5:53:00<1:01:24,  1.82s/it]
 83%|████████▎ | 9978/12000 [5:53:02<1:08:22,  2.03s/it]
                                                        

 83%|████████▎ | 9978/12000 [5:53:02<1:08:22,  2.03s/it]
 83%|████████▎ | 9979/12000 [5:53:05<1:21:24,  2.42s/it]
                                                        

 83%|████████▎ | 9979/12000 [5:53:05<1:21:24,  2.42s/it]
 83%|████████▎ | 9980/12000 [5:53:07<1:10:45,  2.10s/it]
                                                        

 83%|████████▎ | 9980/12000 [5:53:07<1:10:45,  2.10s/it]
 83%|████████▎ | 9981/12000 [5:53:08<1:06:54,  1.99s/it]
                                                        

 83%|████████▎ | 9981/12000 [5:53:08<1:06:54,  1.99s/it]
 83%|████████▎ | 9982/12000 [5:53:10<1:07:07,  2.00s/it]
                                                        

 83%|████████▎ | 9982/12000 [5:53:10<1:07:07,  2.00s/it]
 83%|████████▎ | 9983/12000 [5:53:12<1:01:41,  1.83s/it]
                                                        

 83%|████████▎ | 9983/12000 [5:53:12<1:01:41,  1.83s/it]
 83%|████████▎ | 9984/12000 [5:53:15<1:17:15,  2.30s/it]
                                                        

 83%|████████▎ | 9984/12000 [5:53:15<1:17:15,  2.30s/it]
 83%|████████▎ | 9985/12000 [5:53:17<1:09:32,  2.07s/it]
                                                        

 83%|████████▎ | 9985/12000 [5:53:17<1:09:32,  2.07s/it]
 83%|████████▎ | 9986/12000 [5:53:18<1:04:13,  1.91s/it]
                                                        

 83%|████████▎ | 9986/12000 [5:53:18<1:04:13,  1.91s/it]
 83%|████████▎ | 9987/12000 [5:53:20<1:02:21,  1.86s/it]
                                                        

 83%|████████▎ | 9987/12000 [5:53:20<1:02:21,  1.86s/it]
 83%|████████▎ | 9988/12000 [5:53:22<1:00:54,  1.82s/it]
                                                        

 83%|████████▎ | 9988/12000 [5:53:22<1:00:54,  1.82s/it]
 83%|████████▎ | 9989/12000 [5:53:24<1:01:36,  1.84s/it]
                                                        

 83%|████████▎ | 9989/12000 [5:53:24<1:01:36,  1.84s/it]
 83%|████████▎ | 9990/12000 [5:53:27<1:16:57,  2.30s/it]
                                                        

 83%|████████▎ | 9990/12000 [5:53:27<1:16:57,  2.30s/it]
 83%|████████▎ | 9991/12000 [5:53:30<1:19:46,  2.38s/it]
                                                        

 83%|████████▎ | 9991/12000 [5:53:30<1:19:46,  2.38s/it]
 83%|████████▎ | 9992/12000 [5:53:31<1:11:15,  2.13s/it]
                                                        

 83%|████████▎ | 9992/12000 [5:53:31<1:11:15,  2.13s/it]
 83%|████████▎ | 9993/12000 [5:53:33<1:05:12,  1.95s/it]
                                                        

 83%|████████▎ | 9993/12000 [5:53:33<1:05:12,  1.95s/it]
 83%|████████▎ | 9994/12000 [5:53:34<1:01:02,  1.83s/it]
                                                        

 83%|████████▎ | 9994/12000 [5:53:34<1:01:02,  1.83s/it]
 83%|████████▎ | 9995/12000 [5:53:37<1:05:31,  1.96s/it]
                                                        

 83%|████████▎ | 9995/12000 [5:53:37<1:05:31,  1.96s/it]
 83%|████████▎ | 9996/12000 [5:53:40<1:20:48,  2.42s/it]
                                                        

 83%|████████▎ | 9996/12000 [5:53:40<1:20:48,  2.42s/it]
 83%|████████▎ | 9997/12000 [5:53:42<1:11:27,  2.14s/it]
                                                        

 83%|████████▎ | 9997/12000 [5:53:42<1:11:27,  2.14s/it]
 83%|████████▎ | 9998/12000 [5:53:45<1:21:37,  2.45s/it]
                                                        

 83%|████████▎ | 9998/12000 [5:53:45<1:21:37,  2.45s/it]
 83%|████████▎ | 9999/12000 [5:53:47<1:15:57,  2.28s/it]
                                                        

 83%|████████▎ | 9999/12000 [5:53:47<1:15:57,  2.28s/it]
 83%|████████▎ | 10000/12000 [5:53:48<1:09:09,  2.07s/it]
                                                         

 83%|████████▎ | 10000/12000 [5:53:48<1:09:09,  2.07s/it]
 83%|████████▎ | 10001/12000 [5:53:52<1:25:38,  2.57s/it]
                                                         

 83%|████████▎ | 10001/12000 [5:53:52<1:25:38,  2.57s/it]
 83%|████████▎ | 10002/12000 [5:53:54<1:22:49,  2.49s/it]
                                                         

 83%|████████▎ | 10002/12000 [5:53:54<1:22:49,  2.49s/it]
 83%|████████▎ | 10003/12000 [5:53:56<1:13:50,  2.22s/it]
                                                         

 83%|████████▎ | 10003/12000 [5:53:56<1:13:50,  2.22s/it]
 83%|████████▎ | 10004/12000 [5:53:57<1:06:58,  2.01s/it]
                                                         

 83%|████████▎ | 10004/12000 [5:53:57<1:06:58,  2.01s/it]
 83%|████████▎ | 10005/12000 [5:54:02<1:34:15,  2.83s/it]
                                                         

 83%|████████▎ | 10005/12000 [5:54:02<1:34:15,  2.83s/it]
 83%|████████▎ | 10006/12000 [5:54:06<1:42:37,  3.09s/it]
                                                         

 83%|████████▎ | 10006/12000 [5:54:06<1:42:37,  3.09s/it]
 83%|████████▎ | 10007/12000 [5:54:07<1:27:58,  2.65s/it]
                                                         

 83%|████████▎ | 10007/12000 [5:54:07<1:27:58,  2.65s/it]
 83%|████████▎ | 10008/12000 [5:54:13<1:56:03,  3.50s/it]
                                                         

 83%|████████▎ | 10008/12000 [5:54:13<1:56:03,  3.50s/it]
 83%|████████▎ | 10009/12000 [5:54:16<1:49:33,  3.30s/it]
                                                         

 83%|████████▎ | 10009/12000 [5:54:16<1:49:33,  3.30s/it]
 83%|████████▎ | 10010/12000 [5:54:17<1:29:38,  2.70s/it]
                                                         

 83%|████████▎ | 10010/12000 [5:54:17<1:29:38,  2.70s/it]
 83%|████████▎ | 10011/12000 [5:54:19<1:18:39,  2.37s/it]
                                                         

 83%|████████▎ | 10011/12000 [5:54:19<1:18:39,  2.37s/it]
 83%|████████▎ | 10012/12000 [5:54:21<1:19:55,  2.41s/it]
                                                         

 83%|████████▎ | 10012/12000 [5:54:21<1:19:55,  2.41s/it]
 83%|████████▎ | 10013/12000 [5:54:23<1:16:27,  2.31s/it]
                                                         

 83%|████████▎ | 10013/12000 [5:54:23<1:16:27,  2.31s/it]
 83%|████████▎ | 10014/12000 [5:54:27<1:30:26,  2.73s/it]
                                                         

 83%|████████▎ | 10014/12000 [5:54:27<1:30:26,  2.73s/it]
 83%|████████▎ | 10015/12000 [5:54:30<1:32:04,  2.78s/it]
                                                         
{'loss': '0.7194', 'grad_norm': '0.4127', 'learning_rate': '1.529e-05', 'entropy': '0.9386', 'num_tokens': '5.368e+07', 'mean_token_accuracy': '0.7722', 'epoch': '0.08822'}
{'loss': '1.255', 'grad_norm': '0.5705', 'learning_rate': '1.527e-05', 'entropy': '1.297', 'num_tokens': '5.369e+07', 'mean_token_accuracy': '0.6967', 'epoch': '0.08822'}
{'loss': '0.6386', 'grad_norm': '0.3699', 'learning_rate': '1.526e-05', 'entropy': '0.7562', 'num_tokens': '5.369e+07', 'mean_token_accuracy': '0.8105', 'epoch': '0.08823'}
{'loss': '0.659', 'grad_norm': '0.4438', 'learning_rate': '1.524e-05', 'entropy': '0.7124', 'num_tokens': '5.37e+07', 'mean_token_accuracy': '0.818', 'epoch': '0.08824'}
{'loss': '0.7659', 'grad_norm': '0.4611', 'learning_rate': '1.523e-05', 'entropy': '0.7621', 'num_tokens': '5.37e+07', 'mean_token_accuracy': '0.8004', 'epoch': '0.08825'}
{'loss': '0.7049', 'grad_norm': '0.4567', 'learning_rate': '1.521e-05', 'entropy': '0.7044', 'num_tokens': '5.371e+07', 'mean_token_accuracy': '0.826', 'epoch': '0.08826'}
{'loss': '0.6682', 'grad_norm': '0.4119', 'learning_rate': '1.52e-05', 'entropy': '0.6575', 'num_tokens': '5.371e+07', 'mean_token_accuracy': '0.8219', 'epoch': '0.08827'}
{'loss': '0.7598', 'grad_norm': '0.3886', 'learning_rate': '1.519e-05', 'entropy': '0.8215', 'num_tokens': '5.372e+07', 'mean_token_accuracy': '0.8018', 'epoch': '0.08828'}
{'loss': '0.9309', 'grad_norm': '0.4404', 'learning_rate': '1.517e-05', 'entropy': '0.8919', 'num_tokens': '5.372e+07', 'mean_token_accuracy': '0.7745', 'epoch': '0.08829'}
{'loss': '1.089', 'grad_norm': '0.4491', 'learning_rate': '1.516e-05', 'entropy': '1.042', 'num_tokens': '5.372e+07', 'mean_token_accuracy': '0.7378', 'epoch': '0.08829'}
{'loss': '0.5331', 'grad_norm': '0.3366', 'learning_rate': '1.514e-05', 'entropy': '0.5015', 'num_tokens': '5.373e+07', 'mean_token_accuracy': '0.8652', 'epoch': '0.0883'}
{'loss': '0.5389', 'grad_norm': '0.2786', 'learning_rate': '1.513e-05', 'entropy': '0.6329', 'num_tokens': '5.374e+07', 'mean_token_accuracy': '0.8379', 'epoch': '0.08831'}
{'loss': '0.7402', 'grad_norm': '0.3552', 'learning_rate': '1.511e-05', 'entropy': '0.7136', 'num_tokens': '5.374e+07', 'mean_token_accuracy': '0.8127', 'epoch': '0.08832'}
{'loss': '0.7425', 'grad_norm': '0.3945', 'learning_rate': '1.51e-05', 'entropy': '0.7321', 'num_tokens': '5.375e+07', 'mean_token_accuracy': '0.808', 'epoch': '0.08833'}
{'loss': '0.6984', 'grad_norm': '0.3428', 'learning_rate': '1.508e-05', 'entropy': '0.7191', 'num_tokens': '5.375e+07', 'mean_token_accuracy': '0.8148', 'epoch': '0.08834'}
{'loss': '0.6376', 'grad_norm': '0.3398', 'learning_rate': '1.507e-05', 'entropy': '0.6903', 'num_tokens': '5.376e+07', 'mean_token_accuracy': '0.8212', 'epoch': '0.08835'}
{'loss': '0.8691', 'grad_norm': '0.389', 'learning_rate': '1.505e-05', 'entropy': '0.9126', 'num_tokens': '5.377e+07', 'mean_token_accuracy': '0.7791', 'epoch': '0.08836'}
{'loss': '0.942', 'grad_norm': '0.457', 'learning_rate': '1.504e-05', 'entropy': '0.9102', 'num_tokens': '5.377e+07', 'mean_token_accuracy': '0.7699', 'epoch': '0.08837'}
{'loss': '0.5175', 'grad_norm': '0.3272', 'learning_rate': '1.503e-05', 'entropy': '0.5909', 'num_tokens': '5.378e+07', 'mean_token_accuracy': '0.8522', 'epoch': '0.08837'}
{'loss': '0.849', 'grad_norm': '0.4006', 'learning_rate': '1.501e-05', 'entropy': '0.8762', 'num_tokens': '5.378e+07', 'mean_token_accuracy': '0.7664', 'epoch': '0.08838'}
{'loss': '0.6367', 'grad_norm': '0.4005', 'learning_rate': '1.5e-05', 'entropy': '0.6803', 'num_tokens': '5.379e+07', 'mean_token_accuracy': '0.8324', 'epoch': '0.08839'}
{'loss': '0.8998', 'grad_norm': '0.4809', 'learning_rate': '1.498e-05', 'entropy': '0.9085', 'num_tokens': '5.379e+07', 'mean_token_accuracy': '0.771', 'epoch': '0.0884'}
{'loss': '0.8473', 'grad_norm': '0.3308', 'learning_rate': '1.497e-05', 'entropy': '0.8369', 'num_tokens': '5.38e+07', 'mean_token_accuracy': '0.7802', 'epoch': '0.08841'}
{'loss': '0.8599', 'grad_norm': '0.4095', 'learning_rate': '1.495e-05', 'entropy': '0.9132', 'num_tokens': '5.38e+07', 'mean_token_accuracy': '0.7647', 'epoch': '0.08842'}
{'loss': '0.932', 'grad_norm': '0.452', 'learning_rate': '1.494e-05', 'entropy': '0.9365', 'num_tokens': '5.381e+07', 'mean_token_accuracy': '0.7679', 'epoch': '0.08843'}
{'loss': '0.9722', 'grad_norm': '0.4803', 'learning_rate': '1.492e-05', 'entropy': '0.9157', 'num_tokens': '5.381e+07', 'mean_token_accuracy': '0.7681', 'epoch': '0.08844'}
{'loss': '0.9669', 'grad_norm': '0.421', 'learning_rate': '1.491e-05', 'entropy': '0.9643', 'num_tokens': '5.382e+07', 'mean_token_accuracy': '0.7641', 'epoch': '0.08845'}
{'loss': '0.6063', 'grad_norm': '0.4144', 'learning_rate': '1.489e-05', 'entropy': '0.6359', 'num_tokens': '5.382e+07', 'mean_token_accuracy': '0.8309', 'epoch': '0.08845'}
{'loss': '1.001', 'grad_norm': '0.4175', 'learning_rate': '1.488e-05', 'entropy': '1.055', 'num_tokens': '5.383e+07', 'mean_token_accuracy': '0.7468', 'epoch': '0.08846'}
{'loss': '0.7239', 'grad_norm': '0.4142', 'learning_rate': '1.487e-05', 'entropy': '0.7687', 'num_tokens': '5.383e+07', 'mean_token_accuracy': '0.8177', 'epoch': '0.08847'}
{'loss': '0.8237', 'grad_norm': '0.3324', 'learning_rate': '1.485e-05', 'entropy': '0.8944', 'num_tokens': '5.384e+07', 'mean_token_accuracy': '0.7746', 'epoch': '0.08848'}
{'loss': '0.932', 'grad_norm': '0.5062', 'learning_rate': '1.484e-05', 'entropy': '0.9457', 'num_tokens': '5.385e+07', 'mean_token_accuracy': '0.7462', 'epoch': '0.08849'}
{'loss': '0.981', 'grad_norm': '0.5532', 'learning_rate': '1.482e-05', 'entropy': '1.023', 'num_tokens': '5.385e+07', 'mean_token_accuracy': '0.7538', 'epoch': '0.0885'}
{'loss': '1.153', 'grad_norm': '0.5617', 'learning_rate': '1.481e-05', 'entropy': '1.112', 'num_tokens': '5.386e+07', 'mean_token_accuracy': '0.7256', 'epoch': '0.08851'}
{'loss': '0.681', 'grad_norm': '0.3555', 'learning_rate': '1.479e-05', 'entropy': '0.7069', 'num_tokens': '5.386e+07', 'mean_token_accuracy': '0.8269', 'epoch': '0.08852'}
{'loss': '0.8932', 'grad_norm': '0.4317', 'learning_rate': '1.478e-05', 'entropy': '0.8578', 'num_tokens': '5.387e+07', 'mean_token_accuracy': '0.776', 'epoch': '0.08852'}
{'loss': '0.4715', 'grad_norm': '0.2644', 'learning_rate': '1.476e-05', 'entropy': '0.5175', 'num_tokens': '5.387e+07', 'mean_token_accuracy': '0.8663', 'epoch': '0.08853'}
{'loss': '0.7678', 'grad_norm': '0.3073', 'learning_rate': '1.475e-05', 'entropy': '0.7277', 'num_tokens': '5.388e+07', 'mean_token_accuracy': '0.8171', 'epoch': '0.08854'}
{'loss': '0.7468', 'grad_norm': '0.3204', 'learning_rate': '1.474e-05', 'entropy': '0.8131', 'num_tokens': '5.389e+07', 'mean_token_accuracy': '0.7825', 'epoch': '0.08855'}
{'loss': '0.8194', 'grad_norm': '0.5207', 'learning_rate': '1.472e-05', 'entropy': '0.8319', 'num_tokens': '5.389e+07', 'mean_token_accuracy': '0.7993', 'epoch': '0.08856'}
{'loss': '0.9908', 'grad_norm': '0.3378', 'learning_rate': '1.471e-05', 'entropy': '0.8358', 'num_tokens': '5.39e+07', 'mean_token_accuracy': '0.8042', 'epoch': '0.08857'}
{'loss': '0.6924', 'grad_norm': '0.3606', 'learning_rate': '1.469e-05', 'entropy': '0.6827', 'num_tokens': '5.391e+07', 'mean_token_accuracy': '0.8165', 'epoch': '0.08858'}
{'loss': '0.684', 'grad_norm': '0.3992', 'learning_rate': '1.468e-05', 'entropy': '0.7462', 'num_tokens': '5.391e+07', 'mean_token_accuracy': '0.8021', 'epoch': '0.08859'}
{'loss': '0.5937', 'grad_norm': '0.3632', 'learning_rate': '1.466e-05', 'entropy': '0.5981', 'num_tokens': '5.392e+07', 'mean_token_accuracy': '0.834', 'epoch': '0.0886'}
{'loss': '0.6372', 'grad_norm': '0.3511', 'learning_rate': '1.465e-05', 'entropy': '0.6848', 'num_tokens': '5.393e+07', 'mean_token_accuracy': '0.826', 'epoch': '0.0886'}
{'loss': '1.165', 'grad_norm': '0.7404', 'learning_rate': '1.464e-05', 'entropy': '1.107', 'num_tokens': '5.393e+07', 'mean_token_accuracy': '0.7308', 'epoch': '0.08861'}
{'loss': '0.6937', 'grad_norm': '0.3098', 'learning_rate': '1.462e-05', 'entropy': '0.7629', 'num_tokens': '5.394e+07', 'mean_token_accuracy': '0.8008', 'epoch': '0.08862'}

 83%|████████▎ | 10015/12000 [5:54:30<1:32:04,  2.78s/it]
 83%|████████▎ | 10016/12000 [5:54:32<1:25:24,  2.58s/it]
                                                         

 83%|████████▎ | 10016/12000 [5:54:32<1:25:24,  2.58s/it]
 83%|████████▎ | 10017/12000 [5:54:34<1:15:53,  2.30s/it]
                                                         

 83%|████████▎ | 10017/12000 [5:54:34<1:15:53,  2.30s/it]
 83%|████████▎ | 10018/12000 [5:54:38<1:39:27,  3.01s/it]
                                                         

 83%|████████▎ | 10018/12000 [5:54:38<1:39:27,  3.01s/it]
 83%|████████▎ | 10019/12000 [5:54:40<1:24:12,  2.55s/it]
                                                         

 83%|████████▎ | 10019/12000 [5:54:40<1:24:12,  2.55s/it]
 84%|████████▎ | 10020/12000 [5:54:41<1:13:23,  2.22s/it]
                                                         

 84%|████████▎ | 10020/12000 [5:54:41<1:13:23,  2.22s/it]
 84%|████████▎ | 10021/12000 [5:54:43<1:06:36,  2.02s/it]
                                                         

 84%|████████▎ | 10021/12000 [5:54:43<1:06:36,  2.02s/it]
 84%|████████▎ | 10022/12000 [5:54:44<1:00:30,  1.84s/it]
                                                         

 84%|████████▎ | 10022/12000 [5:54:44<1:00:30,  1.84s/it]
 84%|████████▎ | 10023/12000 [5:54:46<56:36,  1.72s/it]  
                                                       

 84%|████████▎ | 10023/12000 [5:54:46<56:36,  1.72s/it]
 84%|████████▎ | 10024/12000 [5:54:47<54:39,  1.66s/it]
                                                       

 84%|████████▎ | 10024/12000 [5:54:47<54:39,  1.66s/it]
 84%|████████▎ | 10025/12000 [5:54:49<53:55,  1.64s/it]
                                                       

 84%|████████▎ | 10025/12000 [5:54:49<53:55,  1.64s/it]
 84%|████████▎ | 10026/12000 [5:54:51<1:03:50,  1.94s/it]
                                                         

 84%|████████▎ | 10026/12000 [5:54:51<1:03:50,  1.94s/it]
 84%|████████▎ | 10027/12000 [5:54:53<59:48,  1.82s/it]  
                                                       

 84%|████████▎ | 10027/12000 [5:54:53<59:48,  1.82s/it]
 84%|████████▎ | 10028/12000 [5:54:54<56:23,  1.72s/it]
                                                       

 84%|████████▎ | 10028/12000 [5:54:54<56:23,  1.72s/it]
 84%|████████▎ | 10029/12000 [5:54:56<58:35,  1.78s/it]
                                                       

 84%|████████▎ | 10029/12000 [5:54:56<58:35,  1.78s/it]
 84%|████████▎ | 10030/12000 [5:54:58<56:00,  1.71s/it]
                                                       

 84%|████████▎ | 10030/12000 [5:54:58<56:00,  1.71s/it]
 84%|████████▎ | 10031/12000 [5:54:59<52:43,  1.61s/it]
                                                       

 84%|████████▎ | 10031/12000 [5:54:59<52:43,  1.61s/it]
 84%|████████▎ | 10032/12000 [5:55:01<52:29,  1.60s/it]
                                                       

 84%|████████▎ | 10032/12000 [5:55:01<52:29,  1.60s/it]
 84%|████████▎ | 10033/12000 [5:55:02<50:36,  1.54s/it]
                                                       

 84%|████████▎ | 10033/12000 [5:55:02<50:36,  1.54s/it]
 84%|████████▎ | 10034/12000 [5:55:04<49:45,  1.52s/it]
                                                       

 84%|████████▎ | 10034/12000 [5:55:04<49:45,  1.52s/it]
 84%|████████▎ | 10035/12000 [5:55:05<49:35,  1.51s/it]
                                                       

 84%|████████▎ | 10035/12000 [5:55:05<49:35,  1.51s/it]
 84%|████████▎ | 10036/12000 [5:55:07<51:35,  1.58s/it]
                                                       

 84%|████████▎ | 10036/12000 [5:55:07<51:35,  1.58s/it]
 84%|████████▎ | 10037/12000 [5:55:08<50:54,  1.56s/it]
                                                       

 84%|████████▎ | 10037/12000 [5:55:08<50:54,  1.56s/it]
 84%|████████▎ | 10038/12000 [5:55:10<48:39,  1.49s/it]
                                                       

 84%|████████▎ | 10038/12000 [5:55:10<48:39,  1.49s/it]
 84%|████████▎ | 10039/12000 [5:55:12<52:39,  1.61s/it]
                                                       

 84%|████████▎ | 10039/12000 [5:55:12<52:39,  1.61s/it]
 84%|████████▎ | 10040/12000 [5:55:16<1:19:18,  2.43s/it]
                                                         

 84%|████████▎ | 10040/12000 [5:55:16<1:19:18,  2.43s/it]
 84%|████████▎ | 10041/12000 [5:55:18<1:11:12,  2.18s/it]
                                                         

 84%|████████▎ | 10041/12000 [5:55:18<1:11:12,  2.18s/it]
 84%|████████▎ | 10042/12000 [5:55:20<1:15:24,  2.31s/it]
                                                         

 84%|████████▎ | 10042/12000 [5:55:20<1:15:24,  2.31s/it]
 84%|████████▎ | 10043/12000 [5:55:22<1:11:26,  2.19s/it]
                                                         

 84%|████████▎ | 10043/12000 [5:55:22<1:11:26,  2.19s/it]
 84%|████████▎ | 10044/12000 [5:55:24<1:08:54,  2.11s/it]
                                                         

 84%|████████▎ | 10044/12000 [5:55:24<1:08:54,  2.11s/it]
 84%|████████▎ | 10045/12000 [5:55:26<1:05:43,  2.02s/it]
                                                         

 84%|████████▎ | 10045/12000 [5:55:26<1:05:43,  2.02s/it]
 84%|████████▎ | 10046/12000 [5:55:29<1:17:18,  2.37s/it]
                                                         

 84%|████████▎ | 10046/12000 [5:55:29<1:17:18,  2.37s/it]
 84%|████████▎ | 10047/12000 [5:55:31<1:09:44,  2.14s/it]
                                                         

 84%|████████▎ | 10047/12000 [5:55:31<1:09:44,  2.14s/it]
 84%|████████▎ | 10048/12000 [5:55:32<1:04:25,  1.98s/it]
                                                         

 84%|████████▎ | 10048/12000 [5:55:32<1:04:25,  1.98s/it]
 84%|████████▎ | 10049/12000 [5:55:34<1:03:41,  1.96s/it]
                                                         

 84%|████████▎ | 10049/12000 [5:55:34<1:03:41,  1.96s/it]
 84%|████████▍ | 10050/12000 [5:55:36<1:00:43,  1.87s/it]
                                                         

 84%|████████▍ | 10050/12000 [5:55:36<1:00:43,  1.87s/it]
 84%|████████▍ | 10051/12000 [5:55:41<1:28:39,  2.73s/it]
                                                         

 84%|████████▍ | 10051/12000 [5:55:41<1:28:39,  2.73s/it]
 84%|████████▍ | 10052/12000 [5:55:42<1:16:19,  2.35s/it]
                                                         

 84%|████████▍ | 10052/12000 [5:55:42<1:16:19,  2.35s/it]
 84%|████████▍ | 10053/12000 [5:55:47<1:43:14,  3.18s/it]
                                                         

 84%|████████▍ | 10053/12000 [5:55:47<1:43:14,  3.18s/it]
 84%|████████▍ | 10054/12000 [5:55:49<1:25:51,  2.65s/it]
                                                         

 84%|████████▍ | 10054/12000 [5:55:49<1:25:51,  2.65s/it]
 84%|████████▍ | 10055/12000 [5:55:51<1:21:16,  2.51s/it]
                                                         

 84%|████████▍ | 10055/12000 [5:55:51<1:21:16,  2.51s/it]
 84%|████████▍ | 10056/12000 [5:55:52<1:13:53,  2.28s/it]
                                                         

 84%|████████▍ | 10056/12000 [5:55:52<1:13:53,  2.28s/it]
 84%|████████▍ | 10057/12000 [5:55:55<1:13:17,  2.26s/it]
                                                         

 84%|████████▍ | 10057/12000 [5:55:55<1:13:17,  2.26s/it]
 84%|████████▍ | 10058/12000 [5:55:56<1:05:40,  2.03s/it]
                                                         

 84%|████████▍ | 10058/12000 [5:55:56<1:05:40,  2.03s/it]
 84%|████████▍ | 10059/12000 [5:55:59<1:11:17,  2.20s/it]
                                                         

 84%|████████▍ | 10059/12000 [5:55:59<1:11:17,  2.20s/it]
 84%|████████▍ | 10060/12000 [5:56:00<1:04:42,  2.00s/it]
                                                         

 84%|████████▍ | 10060/12000 [5:56:00<1:04:42,  2.00s/it]
 84%|████████▍ | 10061/12000 [5:56:02<1:00:39,  1.88s/it]
                                                         

 84%|████████▍ | 10061/12000 [5:56:02<1:00:39,  1.88s/it]
 84%|████████▍ | 10062/12000 [5:56:05<1:16:52,  2.38s/it]
                                                         
{'loss': '0.9129', 'grad_norm': '0.4036', 'learning_rate': '1.461e-05', 'entropy': '0.9691', 'num_tokens': '5.395e+07', 'mean_token_accuracy': '0.7606', 'epoch': '0.08863'}
{'loss': '0.8773', 'grad_norm': '0.3481', 'learning_rate': '1.459e-05', 'entropy': '0.9289', 'num_tokens': '5.395e+07', 'mean_token_accuracy': '0.78', 'epoch': '0.08864'}
{'loss': '0.8137', 'grad_norm': '0.4438', 'learning_rate': '1.458e-05', 'entropy': '0.7693', 'num_tokens': '5.395e+07', 'mean_token_accuracy': '0.8077', 'epoch': '0.08865'}
{'loss': '0.6853', 'grad_norm': '0.336', 'learning_rate': '1.456e-05', 'entropy': '0.7058', 'num_tokens': '5.396e+07', 'mean_token_accuracy': '0.8165', 'epoch': '0.08866'}
{'loss': '0.7023', 'grad_norm': '0.3898', 'learning_rate': '1.455e-05', 'entropy': '0.6939', 'num_tokens': '5.397e+07', 'mean_token_accuracy': '0.8131', 'epoch': '0.08867'}
{'loss': '0.6498', 'grad_norm': '0.3453', 'learning_rate': '1.454e-05', 'entropy': '0.6759', 'num_tokens': '5.397e+07', 'mean_token_accuracy': '0.8123', 'epoch': '0.08868'}
{'loss': '0.7028', 'grad_norm': '0.426', 'learning_rate': '1.452e-05', 'entropy': '0.7453', 'num_tokens': '5.398e+07', 'mean_token_accuracy': '0.82', 'epoch': '0.08868'}
{'loss': '0.7995', 'grad_norm': '0.524', 'learning_rate': '1.451e-05', 'entropy': '0.8545', 'num_tokens': '5.398e+07', 'mean_token_accuracy': '0.7816', 'epoch': '0.08869'}
{'loss': '0.6626', 'grad_norm': '0.3894', 'learning_rate': '1.449e-05', 'entropy': '0.7304', 'num_tokens': '5.398e+07', 'mean_token_accuracy': '0.8155', 'epoch': '0.0887'}
{'loss': '0.7141', 'grad_norm': '0.4652', 'learning_rate': '1.448e-05', 'entropy': '0.776', 'num_tokens': '5.399e+07', 'mean_token_accuracy': '0.8046', 'epoch': '0.08871'}
{'loss': '0.7135', 'grad_norm': '0.409', 'learning_rate': '1.446e-05', 'entropy': '0.732', 'num_tokens': '5.399e+07', 'mean_token_accuracy': '0.816', 'epoch': '0.08872'}
{'loss': '0.9278', 'grad_norm': '0.4464', 'learning_rate': '1.445e-05', 'entropy': '0.8994', 'num_tokens': '5.4e+07', 'mean_token_accuracy': '0.7787', 'epoch': '0.08873'}
{'loss': '0.6007', 'grad_norm': '0.4348', 'learning_rate': '1.444e-05', 'entropy': '0.6348', 'num_tokens': '5.4e+07', 'mean_token_accuracy': '0.8319', 'epoch': '0.08874'}
{'loss': '0.5707', 'grad_norm': '0.435', 'learning_rate': '1.442e-05', 'entropy': '0.598', 'num_tokens': '5.401e+07', 'mean_token_accuracy': '0.8398', 'epoch': '0.08875'}
{'loss': '1.002', 'grad_norm': '0.4468', 'learning_rate': '1.441e-05', 'entropy': '1.021', 'num_tokens': '5.401e+07', 'mean_token_accuracy': '0.7465', 'epoch': '0.08875'}
{'loss': '0.9459', 'grad_norm': '0.4489', 'learning_rate': '1.439e-05', 'entropy': '1.022', 'num_tokens': '5.402e+07', 'mean_token_accuracy': '0.75', 'epoch': '0.08876'}
{'loss': '0.6127', 'grad_norm': '0.3778', 'learning_rate': '1.438e-05', 'entropy': '0.6298', 'num_tokens': '5.402e+07', 'mean_token_accuracy': '0.8371', 'epoch': '0.08877'}
{'loss': '0.5563', 'grad_norm': '0.3964', 'learning_rate': '1.436e-05', 'entropy': '0.5676', 'num_tokens': '5.403e+07', 'mean_token_accuracy': '0.8473', 'epoch': '0.08878'}
{'loss': '1.016', 'grad_norm': '0.5268', 'learning_rate': '1.435e-05', 'entropy': '1.049', 'num_tokens': '5.403e+07', 'mean_token_accuracy': '0.7411', 'epoch': '0.08879'}
{'loss': '0.8073', 'grad_norm': '0.3817', 'learning_rate': '1.434e-05', 'entropy': '0.851', 'num_tokens': '5.403e+07', 'mean_token_accuracy': '0.7844', 'epoch': '0.0888'}
{'loss': '0.8653', 'grad_norm': '0.4051', 'learning_rate': '1.432e-05', 'entropy': '0.7788', 'num_tokens': '5.404e+07', 'mean_token_accuracy': '0.7985', 'epoch': '0.08881'}
{'loss': '0.9729', 'grad_norm': '0.4248', 'learning_rate': '1.431e-05', 'entropy': '0.8988', 'num_tokens': '5.404e+07', 'mean_token_accuracy': '0.7861', 'epoch': '0.08882'}
{'loss': '1.244', 'grad_norm': '0.4455', 'learning_rate': '1.429e-05', 'entropy': '1.105', 'num_tokens': '5.405e+07', 'mean_token_accuracy': '0.7188', 'epoch': '0.08883'}
{'loss': '0.7641', 'grad_norm': '0.3726', 'learning_rate': '1.428e-05', 'entropy': '0.787', 'num_tokens': '5.405e+07', 'mean_token_accuracy': '0.7921', 'epoch': '0.08883'}
{'loss': '0.9144', 'grad_norm': '0.3471', 'learning_rate': '1.426e-05', 'entropy': '0.8611', 'num_tokens': '5.406e+07', 'mean_token_accuracy': '0.7893', 'epoch': '0.08884'}
{'loss': '0.988', 'grad_norm': '0.4238', 'learning_rate': '1.425e-05', 'entropy': '0.798', 'num_tokens': '5.407e+07', 'mean_token_accuracy': '0.799', 'epoch': '0.08885'}
{'loss': '0.7036', 'grad_norm': '0.3849', 'learning_rate': '1.424e-05', 'entropy': '0.7067', 'num_tokens': '5.407e+07', 'mean_token_accuracy': '0.8186', 'epoch': '0.08886'}
{'loss': '0.8664', 'grad_norm': '0.4508', 'learning_rate': '1.422e-05', 'entropy': '0.8326', 'num_tokens': '5.408e+07', 'mean_token_accuracy': '0.7839', 'epoch': '0.08887'}
{'loss': '0.7762', 'grad_norm': '0.4036', 'learning_rate': '1.421e-05', 'entropy': '0.7626', 'num_tokens': '5.408e+07', 'mean_token_accuracy': '0.8014', 'epoch': '0.08888'}
{'loss': '1.153', 'grad_norm': '0.4565', 'learning_rate': '1.419e-05', 'entropy': '0.976', 'num_tokens': '5.409e+07', 'mean_token_accuracy': '0.7727', 'epoch': '0.08889'}
{'loss': '1.416', 'grad_norm': '0.5057', 'learning_rate': '1.418e-05', 'entropy': '1.152', 'num_tokens': '5.409e+07', 'mean_token_accuracy': '0.7226', 'epoch': '0.0889'}
{'loss': '0.4992', 'grad_norm': '0.2156', 'learning_rate': '1.417e-05', 'entropy': '0.663', 'num_tokens': '5.41e+07', 'mean_token_accuracy': '0.825', 'epoch': '0.08891'}
{'loss': '0.9171', 'grad_norm': '0.4752', 'learning_rate': '1.415e-05', 'entropy': '0.9754', 'num_tokens': '5.411e+07', 'mean_token_accuracy': '0.7548', 'epoch': '0.08891'}
{'loss': '0.9801', 'grad_norm': '0.4095', 'learning_rate': '1.414e-05', 'entropy': '1.048', 'num_tokens': '5.411e+07', 'mean_token_accuracy': '0.7504', 'epoch': '0.08892'}
{'loss': '0.8775', 'grad_norm': '0.434', 'learning_rate': '1.412e-05', 'entropy': '0.7681', 'num_tokens': '5.412e+07', 'mean_token_accuracy': '0.8132', 'epoch': '0.08893'}
{'loss': '0.439', 'grad_norm': '0.2918', 'learning_rate': '1.411e-05', 'entropy': '0.4554', 'num_tokens': '5.412e+07', 'mean_token_accuracy': '0.8719', 'epoch': '0.08894'}
{'loss': '0.8018', 'grad_norm': '0.3192', 'learning_rate': '1.409e-05', 'entropy': '0.8684', 'num_tokens': '5.413e+07', 'mean_token_accuracy': '0.798', 'epoch': '0.08895'}
{'loss': '0.6403', 'grad_norm': '0.3712', 'learning_rate': '1.408e-05', 'entropy': '0.6601', 'num_tokens': '5.413e+07', 'mean_token_accuracy': '0.8353', 'epoch': '0.08896'}
{'loss': '0.5451', 'grad_norm': '0.3124', 'learning_rate': '1.407e-05', 'entropy': '0.5764', 'num_tokens': '5.414e+07', 'mean_token_accuracy': '0.856', 'epoch': '0.08897'}
{'loss': '0.887', 'grad_norm': '0.5587', 'learning_rate': '1.405e-05', 'entropy': '0.8924', 'num_tokens': '5.415e+07', 'mean_token_accuracy': '0.7829', 'epoch': '0.08898'}
{'loss': '0.6253', 'grad_norm': '0.3315', 'learning_rate': '1.404e-05', 'entropy': '0.5697', 'num_tokens': '5.415e+07', 'mean_token_accuracy': '0.8542', 'epoch': '0.08899'}
{'loss': '0.8257', 'grad_norm': '0.392', 'learning_rate': '1.402e-05', 'entropy': '0.8445', 'num_tokens': '5.416e+07', 'mean_token_accuracy': '0.778', 'epoch': '0.08899'}
{'loss': '0.8063', 'grad_norm': '0.4403', 'learning_rate': '1.401e-05', 'entropy': '0.765', 'num_tokens': '5.416e+07', 'mean_token_accuracy': '0.7994', 'epoch': '0.089'}
{'loss': '0.9788', 'grad_norm': '0.4341', 'learning_rate': '1.4e-05', 'entropy': '1.01', 'num_tokens': '5.417e+07', 'mean_token_accuracy': '0.7498', 'epoch': '0.08901'}
{'loss': '0.6277', 'grad_norm': '0.3389', 'learning_rate': '1.398e-05', 'entropy': '0.6144', 'num_tokens': '5.417e+07', 'mean_token_accuracy': '0.844', 'epoch': '0.08902'}
{'loss': '0.911', 'grad_norm': '0.4143', 'learning_rate': '1.397e-05', 'entropy': '0.864', 'num_tokens': '5.418e+07', 'mean_token_accuracy': '0.7783', 'epoch': '0.08903'}
{'loss': '0.7184', 'grad_norm': '0.3214', 'learning_rate': '1.395e-05', 'entropy': '0.7764', 'num_tokens': '5.418e+07', 'mean_token_accuracy': '0.7969', 'epoch': '0.08904'}

 84%|████████▍ | 10062/12000 [5:56:05<1:16:52,  2.38s/it]
 84%|████████▍ | 10063/12000 [5:56:08<1:20:14,  2.49s/it]
                                                         

 84%|████████▍ | 10063/12000 [5:56:08<1:20:14,  2.49s/it]
 84%|████████▍ | 10064/12000 [5:56:10<1:12:28,  2.25s/it]
                                                         

 84%|████████▍ | 10064/12000 [5:56:10<1:12:28,  2.25s/it]
 84%|████████▍ | 10065/12000 [5:56:11<1:06:00,  2.05s/it]
                                                         

 84%|████████▍ | 10065/12000 [5:56:11<1:06:00,  2.05s/it]
 84%|████████▍ | 10066/12000 [5:56:13<1:02:34,  1.94s/it]
                                                         

 84%|████████▍ | 10066/12000 [5:56:13<1:02:34,  1.94s/it]
 84%|████████▍ | 10067/12000 [5:56:15<1:02:23,  1.94s/it]
                                                         

 84%|████████▍ | 10067/12000 [5:56:15<1:02:23,  1.94s/it]
 84%|████████▍ | 10068/12000 [5:56:17<59:49,  1.86s/it]  
                                                       

 84%|████████▍ | 10068/12000 [5:56:17<59:49,  1.86s/it]
 84%|████████▍ | 10069/12000 [5:56:19<1:03:53,  1.99s/it]
                                                         

 84%|████████▍ | 10069/12000 [5:56:19<1:03:53,  1.99s/it]
 84%|████████▍ | 10070/12000 [5:56:21<1:02:18,  1.94s/it]
                                                         

 84%|████████▍ | 10070/12000 [5:56:21<1:02:18,  1.94s/it]
 84%|████████▍ | 10071/12000 [5:56:23<1:07:37,  2.10s/it]
                                                         

 84%|████████▍ | 10071/12000 [5:56:23<1:07:37,  2.10s/it]
 84%|████████▍ | 10072/12000 [5:56:26<1:08:41,  2.14s/it]
                                                         

 84%|████████▍ | 10072/12000 [5:56:26<1:08:41,  2.14s/it]
 84%|████████▍ | 10073/12000 [5:56:27<1:04:14,  2.00s/it]
                                                         

 84%|████████▍ | 10073/12000 [5:56:27<1:04:14,  2.00s/it]
 84%|████████▍ | 10074/12000 [5:56:29<1:02:02,  1.93s/it]
                                                         

 84%|████████▍ | 10074/12000 [5:56:29<1:02:02,  1.93s/it]
 84%|████████▍ | 10075/12000 [5:56:31<59:15,  1.85s/it]  
                                                       

 84%|████████▍ | 10075/12000 [5:56:31<59:15,  1.85s/it]
 84%|████████▍ | 10076/12000 [5:56:33<1:06:53,  2.09s/it]
                                                         

 84%|████████▍ | 10076/12000 [5:56:33<1:06:53,  2.09s/it]
 84%|████████▍ | 10077/12000 [5:56:35<1:03:45,  1.99s/it]
                                                         

 84%|████████▍ | 10077/12000 [5:56:35<1:03:45,  1.99s/it]
 84%|████████▍ | 10078/12000 [5:56:40<1:31:39,  2.86s/it]
                                                         

 84%|████████▍ | 10078/12000 [5:56:40<1:31:39,  2.86s/it]
 84%|████████▍ | 10079/12000 [5:56:42<1:19:28,  2.48s/it]
                                                         

 84%|████████▍ | 10079/12000 [5:56:42<1:19:28,  2.48s/it]
 84%|████████▍ | 10080/12000 [5:56:43<1:08:54,  2.15s/it]
                                                         

 84%|████████▍ | 10080/12000 [5:56:43<1:08:54,  2.15s/it]
 84%|████████▍ | 10081/12000 [5:56:44<1:02:03,  1.94s/it]
                                                         

 84%|████████▍ | 10081/12000 [5:56:44<1:02:03,  1.94s/it]
 84%|████████▍ | 10082/12000 [5:56:46<57:46,  1.81s/it]  
                                                       

 84%|████████▍ | 10082/12000 [5:56:46<57:46,  1.81s/it]
 84%|████████▍ | 10083/12000 [5:56:48<1:02:56,  1.97s/it]
                                                         

 84%|████████▍ | 10083/12000 [5:56:48<1:02:56,  1.97s/it]
 84%|████████▍ | 10084/12000 [5:56:52<1:16:43,  2.40s/it]
                                                         

 84%|████████▍ | 10084/12000 [5:56:52<1:16:43,  2.40s/it]
 84%|████████▍ | 10085/12000 [5:56:53<1:10:19,  2.20s/it]
                                                         

 84%|████████▍ | 10085/12000 [5:56:53<1:10:19,  2.20s/it]
 84%|████████▍ | 10086/12000 [5:56:56<1:14:01,  2.32s/it]
                                                         

 84%|████████▍ | 10086/12000 [5:56:56<1:14:01,  2.32s/it]
 84%|████████▍ | 10087/12000 [5:56:57<1:06:07,  2.07s/it]
                                                         

 84%|████████▍ | 10087/12000 [5:56:57<1:06:07,  2.07s/it]
 84%|████████▍ | 10088/12000 [5:57:00<1:10:39,  2.22s/it]
                                                         

 84%|████████▍ | 10088/12000 [5:57:00<1:10:39,  2.22s/it]
 84%|████████▍ | 10089/12000 [5:57:02<1:11:27,  2.24s/it]
                                                         

 84%|████████▍ | 10089/12000 [5:57:02<1:11:27,  2.24s/it]
 84%|████████▍ | 10090/12000 [5:57:05<1:14:21,  2.34s/it]
                                                         

 84%|████████▍ | 10090/12000 [5:57:05<1:14:21,  2.34s/it]
 84%|████████▍ | 10091/12000 [5:57:07<1:15:54,  2.39s/it]
                                                         

 84%|████████▍ | 10091/12000 [5:57:07<1:15:54,  2.39s/it]
 84%|████████▍ | 10092/12000 [5:57:09<1:08:06,  2.14s/it]
                                                         

 84%|████████▍ | 10092/12000 [5:57:09<1:08:06,  2.14s/it]
 84%|████████▍ | 10093/12000 [5:57:12<1:14:38,  2.35s/it]
                                                         

 84%|████████▍ | 10093/12000 [5:57:12<1:14:38,  2.35s/it]
 84%|████████▍ | 10094/12000 [5:57:14<1:15:22,  2.37s/it]
                                                         

 84%|████████▍ | 10094/12000 [5:57:14<1:15:22,  2.37s/it]
 84%|████████▍ | 10095/12000 [5:57:16<1:06:10,  2.08s/it]
                                                         

 84%|████████▍ | 10095/12000 [5:57:16<1:06:10,  2.08s/it]
 84%|████████▍ | 10096/12000 [5:57:17<1:02:19,  1.96s/it]
                                                         

 84%|████████▍ | 10096/12000 [5:57:17<1:02:19,  1.96s/it]
 84%|████████▍ | 10097/12000 [5:57:19<59:43,  1.88s/it]  
                                                       

 84%|████████▍ | 10097/12000 [5:57:19<59:43,  1.88s/it]
 84%|████████▍ | 10098/12000 [5:57:21<59:14,  1.87s/it]
                                                       

 84%|████████▍ | 10098/12000 [5:57:21<59:14,  1.87s/it]
 84%|████████▍ | 10099/12000 [5:57:23<58:10,  1.84s/it]
                                                       

 84%|████████▍ | 10099/12000 [5:57:23<58:10,  1.84s/it]
 84%|████████▍ | 10100/12000 [5:57:26<1:09:58,  2.21s/it]
                                                         

 84%|████████▍ | 10100/12000 [5:57:26<1:09:58,  2.21s/it]
 84%|████████▍ | 10101/12000 [5:57:29<1:19:20,  2.51s/it]
                                                         

 84%|████████▍ | 10101/12000 [5:57:29<1:19:20,  2.51s/it]
 84%|████████▍ | 10102/12000 [5:57:31<1:12:52,  2.30s/it]
                                                         

 84%|████████▍ | 10102/12000 [5:57:31<1:12:52,  2.30s/it]
 84%|████████▍ | 10103/12000 [5:57:32<1:05:21,  2.07s/it]
                                                         

 84%|████████▍ | 10103/12000 [5:57:32<1:05:21,  2.07s/it]
 84%|████████▍ | 10104/12000 [5:57:36<1:17:30,  2.45s/it]
                                                         

 84%|████████▍ | 10104/12000 [5:57:36<1:17:30,  2.45s/it]
 84%|████████▍ | 10105/12000 [5:57:37<1:09:33,  2.20s/it]
                                                         

 84%|████████▍ | 10105/12000 [5:57:37<1:09:33,  2.20s/it]
 84%|████████▍ | 10106/12000 [5:57:41<1:22:01,  2.60s/it]
                                                         

 84%|████████▍ | 10106/12000 [5:57:41<1:22:01,  2.60s/it]
 84%|████████▍ | 10107/12000 [5:57:43<1:14:46,  2.37s/it]
                                                         

 84%|████████▍ | 10107/12000 [5:57:43<1:14:46,  2.37s/it]
 84%|████████▍ | 10108/12000 [5:57:44<1:05:10,  2.07s/it]
                                                         

 84%|████████▍ | 10108/12000 [5:57:44<1:05:10,  2.07s/it]
 84%|████████▍ | 10109/12000 [5:57:46<1:06:01,  2.09s/it]
                                                         
{'loss': '0.5178', 'grad_norm': '0.401', 'learning_rate': '1.394e-05', 'entropy': '0.5675', 'num_tokens': '5.419e+07', 'mean_token_accuracy': '0.8452', 'epoch': '0.08905'}
{'loss': '0.9457', 'grad_norm': '0.3623', 'learning_rate': '1.393e-05', 'entropy': '0.7644', 'num_tokens': '5.42e+07', 'mean_token_accuracy': '0.8008', 'epoch': '0.08906'}
{'loss': '1.165', 'grad_norm': '0.469', 'learning_rate': '1.391e-05', 'entropy': '1.158', 'num_tokens': '5.42e+07', 'mean_token_accuracy': '0.72', 'epoch': '0.08906'}
{'loss': '0.4905', 'grad_norm': '0.3184', 'learning_rate': '1.39e-05', 'entropy': '0.5079', 'num_tokens': '5.421e+07', 'mean_token_accuracy': '0.859', 'epoch': '0.08907'}
{'loss': '0.8981', 'grad_norm': '0.3976', 'learning_rate': '1.388e-05', 'entropy': '0.9056', 'num_tokens': '5.421e+07', 'mean_token_accuracy': '0.7632', 'epoch': '0.08908'}
{'loss': '1.09', 'grad_norm': '0.4431', 'learning_rate': '1.387e-05', 'entropy': '1.077', 'num_tokens': '5.422e+07', 'mean_token_accuracy': '0.7426', 'epoch': '0.08909'}
{'loss': '0.859', 'grad_norm': '0.4571', 'learning_rate': '1.386e-05', 'entropy': '0.8164', 'num_tokens': '5.422e+07', 'mean_token_accuracy': '0.7913', 'epoch': '0.0891'}
{'loss': '0.6549', 'grad_norm': '0.2806', 'learning_rate': '1.384e-05', 'entropy': '0.6584', 'num_tokens': '5.423e+07', 'mean_token_accuracy': '0.8133', 'epoch': '0.08911'}
{'loss': '0.668', 'grad_norm': '0.351', 'learning_rate': '1.383e-05', 'entropy': '0.6051', 'num_tokens': '5.423e+07', 'mean_token_accuracy': '0.837', 'epoch': '0.08912'}
{'loss': '1.138', 'grad_norm': '0.4247', 'learning_rate': '1.381e-05', 'entropy': '0.9798', 'num_tokens': '5.424e+07', 'mean_token_accuracy': '0.7594', 'epoch': '0.08913'}
{'loss': '0.9295', 'grad_norm': '0.5445', 'learning_rate': '1.38e-05', 'entropy': '0.8576', 'num_tokens': '5.425e+07', 'mean_token_accuracy': '0.7806', 'epoch': '0.08914'}
{'loss': '0.9248', 'grad_norm': '0.5031', 'learning_rate': '1.379e-05', 'entropy': '0.8803', 'num_tokens': '5.425e+07', 'mean_token_accuracy': '0.7776', 'epoch': '0.08914'}
{'loss': '0.6043', 'grad_norm': '0.4557', 'learning_rate': '1.377e-05', 'entropy': '0.6826', 'num_tokens': '5.426e+07', 'mean_token_accuracy': '0.8226', 'epoch': '0.08915'}
{'loss': '0.7985', 'grad_norm': '0.4023', 'learning_rate': '1.376e-05', 'entropy': '0.8929', 'num_tokens': '5.426e+07', 'mean_token_accuracy': '0.772', 'epoch': '0.08916'}
{'loss': '1.032', 'grad_norm': '0.38', 'learning_rate': '1.374e-05', 'entropy': '1.073', 'num_tokens': '5.427e+07', 'mean_token_accuracy': '0.7405', 'epoch': '0.08917'}
{'loss': '0.631', 'grad_norm': '0.3374', 'learning_rate': '1.373e-05', 'entropy': '0.6949', 'num_tokens': '5.427e+07', 'mean_token_accuracy': '0.8108', 'epoch': '0.08918'}
{'loss': '1.534', 'grad_norm': '0.3673', 'learning_rate': '1.372e-05', 'entropy': '1.357', 'num_tokens': '5.428e+07', 'mean_token_accuracy': '0.6571', 'epoch': '0.08919'}
{'loss': '0.8515', 'grad_norm': '0.436', 'learning_rate': '1.37e-05', 'entropy': '0.8195', 'num_tokens': '5.429e+07', 'mean_token_accuracy': '0.8029', 'epoch': '0.0892'}
{'loss': '0.7889', 'grad_norm': '0.4143', 'learning_rate': '1.369e-05', 'entropy': '0.827', 'num_tokens': '5.429e+07', 'mean_token_accuracy': '0.7895', 'epoch': '0.08921'}
{'loss': '0.9665', 'grad_norm': '0.4019', 'learning_rate': '1.367e-05', 'entropy': '1.052', 'num_tokens': '5.43e+07', 'mean_token_accuracy': '0.7472', 'epoch': '0.08922'}
{'loss': '0.4855', 'grad_norm': '0.2272', 'learning_rate': '1.366e-05', 'entropy': '0.5212', 'num_tokens': '5.43e+07', 'mean_token_accuracy': '0.845', 'epoch': '0.08922'}
{'loss': '0.9793', 'grad_norm': '0.3436', 'learning_rate': '1.365e-05', 'entropy': '0.9438', 'num_tokens': '5.431e+07', 'mean_token_accuracy': '0.7823', 'epoch': '0.08923'}
{'loss': '0.5624', 'grad_norm': '0.3414', 'learning_rate': '1.363e-05', 'entropy': '0.6036', 'num_tokens': '5.431e+07', 'mean_token_accuracy': '0.8425', 'epoch': '0.08924'}
{'loss': '0.7911', 'grad_norm': '0.4627', 'learning_rate': '1.362e-05', 'entropy': '0.8541', 'num_tokens': '5.432e+07', 'mean_token_accuracy': '0.7747', 'epoch': '0.08925'}
{'loss': '0.7554', 'grad_norm': '0.3762', 'learning_rate': '1.36e-05', 'entropy': '0.9425', 'num_tokens': '5.432e+07', 'mean_token_accuracy': '0.7621', 'epoch': '0.08926'}
{'loss': '0.7576', 'grad_norm': '0.3279', 'learning_rate': '1.359e-05', 'entropy': '0.7392', 'num_tokens': '5.433e+07', 'mean_token_accuracy': '0.8112', 'epoch': '0.08927'}
{'loss': '0.8682', 'grad_norm': '0.4483', 'learning_rate': '1.358e-05', 'entropy': '0.8329', 'num_tokens': '5.433e+07', 'mean_token_accuracy': '0.789', 'epoch': '0.08928'}
{'loss': '0.798', 'grad_norm': '0.4622', 'learning_rate': '1.356e-05', 'entropy': '0.7436', 'num_tokens': '5.434e+07', 'mean_token_accuracy': '0.8161', 'epoch': '0.08929'}
{'loss': '1.039', 'grad_norm': '0.4693', 'learning_rate': '1.355e-05', 'entropy': '1.014', 'num_tokens': '5.434e+07', 'mean_token_accuracy': '0.7447', 'epoch': '0.08929'}
{'loss': '0.3983', 'grad_norm': '0.2934', 'learning_rate': '1.354e-05', 'entropy': '0.5261', 'num_tokens': '5.435e+07', 'mean_token_accuracy': '0.8539', 'epoch': '0.0893'}
{'loss': '0.6539', 'grad_norm': '0.4248', 'learning_rate': '1.352e-05', 'entropy': '0.644', 'num_tokens': '5.436e+07', 'mean_token_accuracy': '0.8249', 'epoch': '0.08931'}
{'loss': '0.8097', 'grad_norm': '0.4034', 'learning_rate': '1.351e-05', 'entropy': '0.8653', 'num_tokens': '5.436e+07', 'mean_token_accuracy': '0.7703', 'epoch': '0.08932'}
{'loss': '0.516', 'grad_norm': '0.3576', 'learning_rate': '1.349e-05', 'entropy': '0.5948', 'num_tokens': '5.437e+07', 'mean_token_accuracy': '0.8495', 'epoch': '0.08933'}
{'loss': '0.8322', 'grad_norm': '0.5129', 'learning_rate': '1.348e-05', 'entropy': '0.6971', 'num_tokens': '5.437e+07', 'mean_token_accuracy': '0.8117', 'epoch': '0.08934'}
{'loss': '0.5343', 'grad_norm': '0.3332', 'learning_rate': '1.347e-05', 'entropy': '0.6696', 'num_tokens': '5.438e+07', 'mean_token_accuracy': '0.8315', 'epoch': '0.08935'}
{'loss': '0.7671', 'grad_norm': '0.4786', 'learning_rate': '1.345e-05', 'entropy': '0.724', 'num_tokens': '5.438e+07', 'mean_token_accuracy': '0.8182', 'epoch': '0.08936'}
{'loss': '1.021', 'grad_norm': '0.4021', 'learning_rate': '1.344e-05', 'entropy': '0.8616', 'num_tokens': '5.439e+07', 'mean_token_accuracy': '0.7863', 'epoch': '0.08937'}
{'loss': '0.7881', 'grad_norm': '0.4233', 'learning_rate': '1.343e-05', 'entropy': '0.775', 'num_tokens': '5.439e+07', 'mean_token_accuracy': '0.804', 'epoch': '0.08937'}
{'loss': '0.6129', 'grad_norm': '0.3034', 'learning_rate': '1.341e-05', 'entropy': '0.7812', 'num_tokens': '5.44e+07', 'mean_token_accuracy': '0.8058', 'epoch': '0.08938'}
{'loss': '0.6417', 'grad_norm': '0.389', 'learning_rate': '1.34e-05', 'entropy': '0.6695', 'num_tokens': '5.44e+07', 'mean_token_accuracy': '0.8303', 'epoch': '0.08939'}
{'loss': '0.8281', 'grad_norm': '0.4237', 'learning_rate': '1.338e-05', 'entropy': '0.8796', 'num_tokens': '5.441e+07', 'mean_token_accuracy': '0.7817', 'epoch': '0.0894'}
{'loss': '0.8062', 'grad_norm': '0.4233', 'learning_rate': '1.337e-05', 'entropy': '0.8358', 'num_tokens': '5.441e+07', 'mean_token_accuracy': '0.7815', 'epoch': '0.08941'}
{'loss': '0.6081', 'grad_norm': '0.3254', 'learning_rate': '1.336e-05', 'entropy': '0.7182', 'num_tokens': '5.442e+07', 'mean_token_accuracy': '0.8018', 'epoch': '0.08942'}
{'loss': '0.7503', 'grad_norm': '0.4707', 'learning_rate': '1.334e-05', 'entropy': '0.7281', 'num_tokens': '5.442e+07', 'mean_token_accuracy': '0.8121', 'epoch': '0.08943'}
{'loss': '0.6224', 'grad_norm': '0.3703', 'learning_rate': '1.333e-05', 'entropy': '0.6821', 'num_tokens': '5.443e+07', 'mean_token_accuracy': '0.8292', 'epoch': '0.08944'}
{'loss': '0.8182', 'grad_norm': '0.4549', 'learning_rate': '1.331e-05', 'entropy': '0.8287', 'num_tokens': '5.443e+07', 'mean_token_accuracy': '0.7916', 'epoch': '0.08945'}
{'loss': '1.041', 'grad_norm': '0.434', 'learning_rate': '1.33e-05', 'entropy': '1.033', 'num_tokens': '5.444e+07', 'mean_token_accuracy': '0.7399', 'epoch': '0.08945'}

 84%|████████▍ | 10109/12000 [5:57:46<1:06:01,  2.09s/it]
 84%|████████▍ | 10110/12000 [5:57:48<1:01:32,  1.95s/it]
                                                         

 84%|████████▍ | 10110/12000 [5:57:48<1:01:32,  1.95s/it]
 84%|████████▍ | 10111/12000 [5:57:49<57:18,  1.82s/it]  
                                                       

 84%|████████▍ | 10111/12000 [5:57:49<57:18,  1.82s/it]
 84%|████████▍ | 10112/12000 [5:57:51<53:34,  1.70s/it]
                                                       

 84%|████████▍ | 10112/12000 [5:57:51<53:34,  1.70s/it]
 84%|████████▍ | 10113/12000 [5:57:53<55:44,  1.77s/it]
                                                       

 84%|████████▍ | 10113/12000 [5:57:53<55:44,  1.77s/it]
 84%|████████▍ | 10114/12000 [5:57:54<53:13,  1.69s/it]
                                                       

 84%|████████▍ | 10114/12000 [5:57:54<53:13,  1.69s/it]
 84%|████████▍ | 10115/12000 [5:57:56<50:50,  1.62s/it]
                                                       

 84%|████████▍ | 10115/12000 [5:57:56<50:50,  1.62s/it]
 84%|████████▍ | 10116/12000 [5:57:58<56:01,  1.78s/it]
                                                       

 84%|████████▍ | 10116/12000 [5:57:58<56:01,  1.78s/it]
 84%|████████▍ | 10117/12000 [5:58:00<57:43,  1.84s/it]
                                                       

 84%|████████▍ | 10117/12000 [5:58:00<57:43,  1.84s/it]
 84%|████████▍ | 10118/12000 [5:58:02<1:00:28,  1.93s/it]
                                                         

 84%|████████▍ | 10118/12000 [5:58:02<1:00:28,  1.93s/it]
 84%|████████▍ | 10119/12000 [5:58:04<58:32,  1.87s/it]  
                                                       

 84%|████████▍ | 10119/12000 [5:58:04<58:32,  1.87s/it]
 84%|████████▍ | 10120/12000 [5:58:06<1:06:26,  2.12s/it]
                                                         

 84%|████████▍ | 10120/12000 [5:58:06<1:06:26,  2.12s/it]
 84%|████████▍ | 10121/12000 [5:58:08<1:02:33,  2.00s/it]
                                                         

 84%|████████▍ | 10121/12000 [5:58:08<1:02:33,  2.00s/it]
 84%|████████▍ | 10122/12000 [5:58:09<57:13,  1.83s/it]  
                                                       

 84%|████████▍ | 10122/12000 [5:58:09<57:13,  1.83s/it]
 84%|████████▍ | 10123/12000 [5:58:11<54:02,  1.73s/it]
                                                       

 84%|████████▍ | 10123/12000 [5:58:11<54:02,  1.73s/it]
 84%|████████▍ | 10124/12000 [5:58:12<52:59,  1.69s/it]
                                                       

 84%|████████▍ | 10124/12000 [5:58:12<52:59,  1.69s/it]
 84%|████████▍ | 10125/12000 [5:58:14<50:54,  1.63s/it]
                                                       

 84%|████████▍ | 10125/12000 [5:58:14<50:54,  1.63s/it]
 84%|████████▍ | 10126/12000 [5:58:15<49:01,  1.57s/it]
                                                       

 84%|████████▍ | 10126/12000 [5:58:15<49:01,  1.57s/it]
 84%|████████▍ | 10127/12000 [5:58:17<53:28,  1.71s/it]
                                                       

 84%|████████▍ | 10127/12000 [5:58:17<53:28,  1.71s/it]
 84%|████████▍ | 10128/12000 [5:58:21<1:12:53,  2.34s/it]
                                                         

 84%|████████▍ | 10128/12000 [5:58:21<1:12:53,  2.34s/it]
 84%|████████▍ | 10129/12000 [5:58:23<1:06:42,  2.14s/it]
                                                         

 84%|████████▍ | 10129/12000 [5:58:23<1:06:42,  2.14s/it]
 84%|████████▍ | 10130/12000 [5:58:24<1:00:05,  1.93s/it]
                                                         

 84%|████████▍ | 10130/12000 [5:58:24<1:00:05,  1.93s/it]
 84%|████████▍ | 10131/12000 [5:58:26<56:59,  1.83s/it]  
                                                       

 84%|████████▍ | 10131/12000 [5:58:26<56:59,  1.83s/it]
 84%|████████▍ | 10132/12000 [5:58:28<1:00:44,  1.95s/it]
                                                         

 84%|████████▍ | 10132/12000 [5:58:28<1:00:44,  1.95s/it]
 84%|████████▍ | 10133/12000 [5:58:30<57:03,  1.83s/it]  
                                                       

 84%|████████▍ | 10133/12000 [5:58:30<57:03,  1.83s/it]
 84%|████████▍ | 10134/12000 [5:58:32<59:41,  1.92s/it]
                                                       

 84%|████████▍ | 10134/12000 [5:58:32<59:41,  1.92s/it]
 84%|████████▍ | 10135/12000 [5:58:33<57:04,  1.84s/it]
                                                       

 84%|████████▍ | 10135/12000 [5:58:33<57:04,  1.84s/it]
 84%|████████▍ | 10136/12000 [5:58:36<58:58,  1.90s/it]
                                                       

 84%|████████▍ | 10136/12000 [5:58:36<58:58,  1.90s/it]
 84%|████████▍ | 10137/12000 [5:58:38<1:00:09,  1.94s/it]
                                                         

 84%|████████▍ | 10137/12000 [5:58:38<1:00:09,  1.94s/it]
 84%|████████▍ | 10138/12000 [5:58:39<58:57,  1.90s/it]  
                                                       

 84%|████████▍ | 10138/12000 [5:58:39<58:57,  1.90s/it]
 84%|████████▍ | 10139/12000 [5:58:41<56:15,  1.81s/it]
                                                       

 84%|████████▍ | 10139/12000 [5:58:41<56:15,  1.81s/it]
 84%|████████▍ | 10140/12000 [5:58:43<53:53,  1.74s/it]
                                                       

 84%|████████▍ | 10140/12000 [5:58:43<53:53,  1.74s/it]
 85%|████████▍ | 10141/12000 [5:58:44<55:20,  1.79s/it]
                                                       

 85%|████████▍ | 10141/12000 [5:58:44<55:20,  1.79s/it]
 85%|████████▍ | 10142/12000 [5:58:46<54:52,  1.77s/it]
                                                       

 85%|████████▍ | 10142/12000 [5:58:46<54:52,  1.77s/it]
 85%|████████▍ | 10143/12000 [5:58:48<54:05,  1.75s/it]
                                                       

 85%|████████▍ | 10143/12000 [5:58:48<54:05,  1.75s/it]
 85%|████████▍ | 10144/12000 [5:58:51<1:03:09,  2.04s/it]
                                                         

 85%|████████▍ | 10144/12000 [5:58:51<1:03:09,  2.04s/it]
 85%|████████▍ | 10145/12000 [5:58:55<1:26:25,  2.80s/it]
                                                         

 85%|████████▍ | 10145/12000 [5:58:55<1:26:25,  2.80s/it]
 85%|████████▍ | 10146/12000 [5:58:57<1:18:20,  2.54s/it]
                                                         

 85%|████████▍ | 10146/12000 [5:58:57<1:18:20,  2.54s/it]
 85%|████████▍ | 10147/12000 [5:59:00<1:23:18,  2.70s/it]
                                                         

 85%|████████▍ | 10147/12000 [5:59:00<1:23:18,  2.70s/it]
 85%|████████▍ | 10148/12000 [5:59:02<1:11:35,  2.32s/it]
                                                         

 85%|████████▍ | 10148/12000 [5:59:02<1:11:35,  2.32s/it]
 85%|████████▍ | 10149/12000 [5:59:03<1:03:38,  2.06s/it]
                                                         

 85%|████████▍ | 10149/12000 [5:59:03<1:03:38,  2.06s/it]
 85%|████████▍ | 10150/12000 [5:59:07<1:18:24,  2.54s/it]
                                                         

 85%|████████▍ | 10150/12000 [5:59:07<1:18:24,  2.54s/it]
 85%|████████▍ | 10151/12000 [5:59:10<1:24:02,  2.73s/it]
                                                         

 85%|████████▍ | 10151/12000 [5:59:10<1:24:02,  2.73s/it]
 85%|████████▍ | 10152/12000 [5:59:12<1:16:28,  2.48s/it]
                                                         

 85%|████████▍ | 10152/12000 [5:59:12<1:16:28,  2.48s/it]
 85%|████████▍ | 10153/12000 [5:59:14<1:14:37,  2.42s/it]
                                                         

 85%|████████▍ | 10153/12000 [5:59:14<1:14:37,  2.42s/it]
 85%|████████▍ | 10154/12000 [5:59:17<1:15:55,  2.47s/it]
                                                         

 85%|████████▍ | 10154/12000 [5:59:17<1:15:55,  2.47s/it]
 85%|████████▍ | 10155/12000 [5:59:18<1:09:25,  2.26s/it]
                                                         

 85%|████████▍ | 10155/12000 [5:59:18<1:09:25,  2.26s/it]
 85%|████████▍ | 10156/12000 [5:59:20<1:04:50,  2.11s/it]
                                                         
{'loss': '1.12', 'grad_norm': '0.4251', 'learning_rate': '1.329e-05', 'entropy': '1.11', 'num_tokens': '5.444e+07', 'mean_token_accuracy': '0.7564', 'epoch': '0.08946'}
{'loss': '0.8631', 'grad_norm': '0.5231', 'learning_rate': '1.327e-05', 'entropy': '0.8753', 'num_tokens': '5.445e+07', 'mean_token_accuracy': '0.7769', 'epoch': '0.08947'}
{'loss': '0.5682', 'grad_norm': '0.3522', 'learning_rate': '1.326e-05', 'entropy': '0.6419', 'num_tokens': '5.445e+07', 'mean_token_accuracy': '0.8284', 'epoch': '0.08948'}
{'loss': '0.9938', 'grad_norm': '0.3676', 'learning_rate': '1.325e-05', 'entropy': '0.9792', 'num_tokens': '5.446e+07', 'mean_token_accuracy': '0.7582', 'epoch': '0.08949'}
{'loss': '0.751', 'grad_norm': '0.3727', 'learning_rate': '1.323e-05', 'entropy': '0.7896', 'num_tokens': '5.446e+07', 'mean_token_accuracy': '0.8011', 'epoch': '0.0895'}
{'loss': '0.7168', 'grad_norm': '0.3787', 'learning_rate': '1.322e-05', 'entropy': '0.7179', 'num_tokens': '5.446e+07', 'mean_token_accuracy': '0.8142', 'epoch': '0.08951'}
{'loss': '0.7332', 'grad_norm': '0.3375', 'learning_rate': '1.321e-05', 'entropy': '0.7712', 'num_tokens': '5.447e+07', 'mean_token_accuracy': '0.7983', 'epoch': '0.08952'}
{'loss': '1.147', 'grad_norm': '0.489', 'learning_rate': '1.319e-05', 'entropy': '1.117', 'num_tokens': '5.447e+07', 'mean_token_accuracy': '0.7256', 'epoch': '0.08952'}
{'loss': '0.5287', 'grad_norm': '0.4192', 'learning_rate': '1.318e-05', 'entropy': '0.5306', 'num_tokens': '5.448e+07', 'mean_token_accuracy': '0.8532', 'epoch': '0.08953'}
{'loss': '0.9772', 'grad_norm': '0.429', 'learning_rate': '1.316e-05', 'entropy': '1.002', 'num_tokens': '5.449e+07', 'mean_token_accuracy': '0.7405', 'epoch': '0.08954'}
{'loss': '0.5212', 'grad_norm': '0.3743', 'learning_rate': '1.315e-05', 'entropy': '0.54', 'num_tokens': '5.449e+07', 'mean_token_accuracy': '0.8462', 'epoch': '0.08955'}
{'loss': '0.6661', 'grad_norm': '0.3227', 'learning_rate': '1.314e-05', 'entropy': '0.7842', 'num_tokens': '5.45e+07', 'mean_token_accuracy': '0.802', 'epoch': '0.08956'}
{'loss': '0.8503', 'grad_norm': '0.4871', 'learning_rate': '1.312e-05', 'entropy': '0.9004', 'num_tokens': '5.451e+07', 'mean_token_accuracy': '0.7851', 'epoch': '0.08957'}
{'loss': '0.6812', 'grad_norm': '0.3378', 'learning_rate': '1.311e-05', 'entropy': '0.7048', 'num_tokens': '5.451e+07', 'mean_token_accuracy': '0.8049', 'epoch': '0.08958'}
{'loss': '0.9221', 'grad_norm': '0.4065', 'learning_rate': '1.31e-05', 'entropy': '0.8942', 'num_tokens': '5.452e+07', 'mean_token_accuracy': '0.7889', 'epoch': '0.08959'}
{'loss': '0.7851', 'grad_norm': '0.4162', 'learning_rate': '1.308e-05', 'entropy': '0.8462', 'num_tokens': '5.452e+07', 'mean_token_accuracy': '0.794', 'epoch': '0.0896'}
{'loss': '0.6928', 'grad_norm': '0.3373', 'learning_rate': '1.307e-05', 'entropy': '0.717', 'num_tokens': '5.452e+07', 'mean_token_accuracy': '0.8093', 'epoch': '0.0896'}
{'loss': '0.7515', 'grad_norm': '0.4573', 'learning_rate': '1.306e-05', 'entropy': '0.7297', 'num_tokens': '5.453e+07', 'mean_token_accuracy': '0.8166', 'epoch': '0.08961'}
{'loss': '0.8661', 'grad_norm': '0.3498', 'learning_rate': '1.304e-05', 'entropy': '0.8164', 'num_tokens': '5.453e+07', 'mean_token_accuracy': '0.7889', 'epoch': '0.08962'}
{'loss': '0.954', 'grad_norm': '0.4223', 'learning_rate': '1.303e-05', 'entropy': '0.9152', 'num_tokens': '5.454e+07', 'mean_token_accuracy': '0.7722', 'epoch': '0.08963'}
{'loss': '0.6066', 'grad_norm': '0.4189', 'learning_rate': '1.301e-05', 'entropy': '0.6458', 'num_tokens': '5.454e+07', 'mean_token_accuracy': '0.8317', 'epoch': '0.08964'}
{'loss': '0.7435', 'grad_norm': '0.364', 'learning_rate': '1.3e-05', 'entropy': '0.7774', 'num_tokens': '5.455e+07', 'mean_token_accuracy': '0.8051', 'epoch': '0.08965'}
{'loss': '0.6014', 'grad_norm': '0.3642', 'learning_rate': '1.299e-05', 'entropy': '0.5826', 'num_tokens': '5.455e+07', 'mean_token_accuracy': '0.8418', 'epoch': '0.08966'}
{'loss': '0.8867', 'grad_norm': '0.4705', 'learning_rate': '1.297e-05', 'entropy': '0.9285', 'num_tokens': '5.456e+07', 'mean_token_accuracy': '0.7926', 'epoch': '0.08967'}
{'loss': '0.7097', 'grad_norm': '0.3398', 'learning_rate': '1.296e-05', 'entropy': '0.6786', 'num_tokens': '5.457e+07', 'mean_token_accuracy': '0.8237', 'epoch': '0.08968'}
{'loss': '0.9537', 'grad_norm': '0.4215', 'learning_rate': '1.295e-05', 'entropy': '0.8749', 'num_tokens': '5.457e+07', 'mean_token_accuracy': '0.7659', 'epoch': '0.08968'}
{'loss': '0.9222', 'grad_norm': '0.4796', 'learning_rate': '1.293e-05', 'entropy': '0.8696', 'num_tokens': '5.458e+07', 'mean_token_accuracy': '0.7734', 'epoch': '0.08969'}
{'loss': '1.017', 'grad_norm': '0.451', 'learning_rate': '1.292e-05', 'entropy': '0.9846', 'num_tokens': '5.458e+07', 'mean_token_accuracy': '0.7494', 'epoch': '0.0897'}
{'loss': '1.087', 'grad_norm': '0.4537', 'learning_rate': '1.291e-05', 'entropy': '0.9604', 'num_tokens': '5.459e+07', 'mean_token_accuracy': '0.7644', 'epoch': '0.08971'}
{'loss': '0.8858', 'grad_norm': '0.4451', 'learning_rate': '1.289e-05', 'entropy': '0.8457', 'num_tokens': '5.459e+07', 'mean_token_accuracy': '0.7869', 'epoch': '0.08972'}
{'loss': '0.8838', 'grad_norm': '0.4064', 'learning_rate': '1.288e-05', 'entropy': '0.841', 'num_tokens': '5.46e+07', 'mean_token_accuracy': '0.7848', 'epoch': '0.08973'}
{'loss': '0.7625', 'grad_norm': '0.3711', 'learning_rate': '1.287e-05', 'entropy': '0.7683', 'num_tokens': '5.46e+07', 'mean_token_accuracy': '0.8048', 'epoch': '0.08974'}
{'loss': '1.287', 'grad_norm': '0.4347', 'learning_rate': '1.285e-05', 'entropy': '1.005', 'num_tokens': '5.461e+07', 'mean_token_accuracy': '0.7577', 'epoch': '0.08975'}
{'loss': '0.5156', 'grad_norm': '0.3387', 'learning_rate': '1.284e-05', 'entropy': '0.5519', 'num_tokens': '5.461e+07', 'mean_token_accuracy': '0.8463', 'epoch': '0.08975'}
{'loss': '0.6989', 'grad_norm': '0.3834', 'learning_rate': '1.282e-05', 'entropy': '0.6913', 'num_tokens': '5.462e+07', 'mean_token_accuracy': '0.8204', 'epoch': '0.08976'}
{'loss': '0.8971', 'grad_norm': '0.3958', 'learning_rate': '1.281e-05', 'entropy': '0.9281', 'num_tokens': '5.462e+07', 'mean_token_accuracy': '0.7662', 'epoch': '0.08977'}
{'loss': '0.6815', 'grad_norm': '0.3119', 'learning_rate': '1.28e-05', 'entropy': '0.7887', 'num_tokens': '5.463e+07', 'mean_token_accuracy': '0.7952', 'epoch': '0.08978'}
{'loss': '0.6996', 'grad_norm': '0.3412', 'learning_rate': '1.278e-05', 'entropy': '0.6975', 'num_tokens': '5.464e+07', 'mean_token_accuracy': '0.8021', 'epoch': '0.08979'}
{'loss': '0.7053', 'grad_norm': '0.3198', 'learning_rate': '1.277e-05', 'entropy': '0.6666', 'num_tokens': '5.464e+07', 'mean_token_accuracy': '0.8274', 'epoch': '0.0898'}
{'loss': '0.6833', 'grad_norm': '0.4289', 'learning_rate': '1.276e-05', 'entropy': '0.7348', 'num_tokens': '5.465e+07', 'mean_token_accuracy': '0.8105', 'epoch': '0.08981'}
{'loss': '0.7598', 'grad_norm': '0.4178', 'learning_rate': '1.274e-05', 'entropy': '0.7373', 'num_tokens': '5.465e+07', 'mean_token_accuracy': '0.8011', 'epoch': '0.08982'}
{'loss': '0.6355', 'grad_norm': '0.367', 'learning_rate': '1.273e-05', 'entropy': '0.6126', 'num_tokens': '5.466e+07', 'mean_token_accuracy': '0.8355', 'epoch': '0.08983'}
{'loss': '1.145', 'grad_norm': '0.4158', 'learning_rate': '1.272e-05', 'entropy': '1.088', 'num_tokens': '5.466e+07', 'mean_token_accuracy': '0.7437', 'epoch': '0.08983'}
{'loss': '0.8406', 'grad_norm': '0.3337', 'learning_rate': '1.27e-05', 'entropy': '0.8896', 'num_tokens': '5.467e+07', 'mean_token_accuracy': '0.7873', 'epoch': '0.08984'}
{'loss': '0.563', 'grad_norm': '0.3148', 'learning_rate': '1.269e-05', 'entropy': '0.7535', 'num_tokens': '5.467e+07', 'mean_token_accuracy': '0.8011', 'epoch': '0.08985'}
{'loss': '0.6142', 'grad_norm': '0.3653', 'learning_rate': '1.268e-05', 'entropy': '0.6852', 'num_tokens': '5.468e+07', 'mean_token_accuracy': '0.8333', 'epoch': '0.08986'}
{'loss': '0.7614', 'grad_norm': '0.387', 'learning_rate': '1.266e-05', 'entropy': '0.7525', 'num_tokens': '5.469e+07', 'mean_token_accuracy': '0.8103', 'epoch': '0.08987'}

 85%|████████▍ | 10156/12000 [5:59:20<1:04:50,  2.11s/it]
 85%|████████▍ | 10157/12000 [5:59:22<58:11,  1.89s/it]  
                                                       

 85%|████████▍ | 10157/12000 [5:59:22<58:11,  1.89s/it]
 85%|████████▍ | 10158/12000 [5:59:24<1:04:44,  2.11s/it]
                                                         

 85%|████████▍ | 10158/12000 [5:59:24<1:04:44,  2.11s/it]
 85%|████████▍ | 10159/12000 [5:59:26<1:00:55,  1.99s/it]
                                                         

 85%|████████▍ | 10159/12000 [5:59:26<1:00:55,  1.99s/it]
 85%|████████▍ | 10160/12000 [5:59:29<1:11:10,  2.32s/it]
                                                         

 85%|████████▍ | 10160/12000 [5:59:29<1:11:10,  2.32s/it]
 85%|████████▍ | 10161/12000 [5:59:31<1:11:54,  2.35s/it]
                                                         

 85%|████████▍ | 10161/12000 [5:59:31<1:11:54,  2.35s/it]
 85%|████████▍ | 10162/12000 [5:59:33<1:09:25,  2.27s/it]
                                                         

 85%|████████▍ | 10162/12000 [5:59:33<1:09:25,  2.27s/it]
 85%|████████▍ | 10163/12000 [5:59:36<1:10:58,  2.32s/it]
                                                         

 85%|████████▍ | 10163/12000 [5:59:36<1:10:58,  2.32s/it]
 85%|████████▍ | 10164/12000 [5:59:38<1:04:30,  2.11s/it]
                                                         

 85%|████████▍ | 10164/12000 [5:59:38<1:04:30,  2.11s/it]
 85%|████████▍ | 10165/12000 [5:59:40<1:05:34,  2.14s/it]
                                                         

 85%|████████▍ | 10165/12000 [5:59:40<1:05:34,  2.14s/it]
 85%|████████▍ | 10166/12000 [5:59:43<1:11:56,  2.35s/it]
                                                         

 85%|████████▍ | 10166/12000 [5:59:43<1:11:56,  2.35s/it]
 85%|████████▍ | 10167/12000 [5:59:44<1:04:46,  2.12s/it]
                                                         

 85%|████████▍ | 10167/12000 [5:59:44<1:04:46,  2.12s/it]
 85%|████████▍ | 10168/12000 [5:59:46<1:00:19,  1.98s/it]
                                                         

 85%|████████▍ | 10168/12000 [5:59:46<1:00:19,  1.98s/it]
 85%|████████▍ | 10169/12000 [5:59:49<1:08:06,  2.23s/it]
                                                         

 85%|████████▍ | 10169/12000 [5:59:49<1:08:06,  2.23s/it]
 85%|████████▍ | 10170/12000 [5:59:52<1:15:03,  2.46s/it]
                                                         

 85%|████████▍ | 10170/12000 [5:59:52<1:15:03,  2.46s/it]
 85%|████████▍ | 10171/12000 [5:59:53<1:08:22,  2.24s/it]
                                                         

 85%|████████▍ | 10171/12000 [5:59:53<1:08:22,  2.24s/it]
 85%|████████▍ | 10172/12000 [5:59:57<1:25:08,  2.79s/it]
                                                         

 85%|████████▍ | 10172/12000 [5:59:57<1:25:08,  2.79s/it]
 85%|████████▍ | 10173/12000 [5:59:59<1:16:07,  2.50s/it]
                                                         

 85%|████████▍ | 10173/12000 [5:59:59<1:16:07,  2.50s/it]
 85%|████████▍ | 10174/12000 [6:00:03<1:26:31,  2.84s/it]
                                                         

 85%|████████▍ | 10174/12000 [6:00:03<1:26:31,  2.84s/it]
 85%|████████▍ | 10175/12000 [6:00:04<1:13:58,  2.43s/it]
                                                         

 85%|████████▍ | 10175/12000 [6:00:04<1:13:58,  2.43s/it]
 85%|████████▍ | 10176/12000 [6:00:06<1:08:18,  2.25s/it]
                                                         

 85%|████████▍ | 10176/12000 [6:00:06<1:08:18,  2.25s/it]
 85%|████████▍ | 10177/12000 [6:00:08<1:01:01,  2.01s/it]
                                                         

 85%|████████▍ | 10177/12000 [6:00:08<1:01:01,  2.01s/it]
 85%|████████▍ | 10178/12000 [6:00:09<56:14,  1.85s/it]  
                                                       

 85%|████████▍ | 10178/12000 [6:00:09<56:14,  1.85s/it]
 85%|████████▍ | 10179/12000 [6:00:11<53:22,  1.76s/it]
                                                       

 85%|████████▍ | 10179/12000 [6:00:11<53:22,  1.76s/it]
 85%|████████▍ | 10180/12000 [6:00:12<51:03,  1.68s/it]
                                                       

 85%|████████▍ | 10180/12000 [6:00:12<51:03,  1.68s/it]
 85%|████████▍ | 10181/12000 [6:00:14<56:35,  1.87s/it]
                                                       

 85%|████████▍ | 10181/12000 [6:00:14<56:35,  1.87s/it]
 85%|████████▍ | 10182/12000 [6:00:16<54:16,  1.79s/it]
                                                       

 85%|████████▍ | 10182/12000 [6:00:16<54:16,  1.79s/it]
 85%|████████▍ | 10183/12000 [6:00:18<51:24,  1.70s/it]
                                                       

 85%|████████▍ | 10183/12000 [6:00:18<51:24,  1.70s/it]
 85%|████████▍ | 10184/12000 [6:00:19<52:00,  1.72s/it]
                                                       

 85%|████████▍ | 10184/12000 [6:00:19<52:00,  1.72s/it]
 85%|████████▍ | 10185/12000 [6:00:21<50:15,  1.66s/it]
                                                       

 85%|████████▍ | 10185/12000 [6:00:21<50:15,  1.66s/it]
 85%|████████▍ | 10186/12000 [6:00:23<50:47,  1.68s/it]
                                                       

 85%|████████▍ | 10186/12000 [6:00:23<50:47,  1.68s/it]
 85%|████████▍ | 10187/12000 [6:00:24<50:06,  1.66s/it]
                                                       

 85%|████████▍ | 10187/12000 [6:00:24<50:06,  1.66s/it]
 85%|████████▍ | 10188/12000 [6:00:26<47:48,  1.58s/it]
                                                       

 85%|████████▍ | 10188/12000 [6:00:26<47:48,  1.58s/it]
 85%|████████▍ | 10189/12000 [6:00:28<50:31,  1.67s/it]
                                                       

 85%|████████▍ | 10189/12000 [6:00:28<50:31,  1.67s/it]
 85%|████████▍ | 10190/12000 [6:00:29<49:23,  1.64s/it]
                                                       

 85%|████████▍ | 10190/12000 [6:00:29<49:23,  1.64s/it]
 85%|████████▍ | 10191/12000 [6:00:32<56:51,  1.89s/it]
                                                       

 85%|████████▍ | 10191/12000 [6:00:32<56:51,  1.89s/it]
 85%|████████▍ | 10192/12000 [6:00:33<53:59,  1.79s/it]
                                                       

 85%|████████▍ | 10192/12000 [6:00:33<53:59,  1.79s/it]
 85%|████████▍ | 10193/12000 [6:00:37<1:10:46,  2.35s/it]
                                                         

 85%|████████▍ | 10193/12000 [6:00:37<1:10:46,  2.35s/it]
 85%|████████▍ | 10194/12000 [6:00:39<1:05:23,  2.17s/it]
                                                         

 85%|████████▍ | 10194/12000 [6:00:39<1:05:23,  2.17s/it]
 85%|████████▍ | 10195/12000 [6:00:41<1:05:40,  2.18s/it]
                                                         

 85%|████████▍ | 10195/12000 [6:00:41<1:05:40,  2.18s/it]
 85%|████████▍ | 10196/12000 [6:00:43<1:05:43,  2.19s/it]
                                                         

 85%|████████▍ | 10196/12000 [6:00:43<1:05:43,  2.19s/it]
 85%|████████▍ | 10197/12000 [6:00:45<1:03:01,  2.10s/it]
                                                         

 85%|████████▍ | 10197/12000 [6:00:45<1:03:01,  2.10s/it]
 85%|████████▍ | 10198/12000 [6:00:46<58:50,  1.96s/it]  
                                                       

 85%|████████▍ | 10198/12000 [6:00:46<58:50,  1.96s/it]
 85%|████████▍ | 10199/12000 [6:00:48<56:58,  1.90s/it]
                                                       

 85%|████████▍ | 10199/12000 [6:00:48<56:58,  1.90s/it]
 85%|████████▌ | 10200/12000 [6:00:50<55:47,  1.86s/it]
                                                       

 85%|████████▌ | 10200/12000 [6:00:50<55:47,  1.86s/it]
 85%|████████▌ | 10201/12000 [6:00:56<1:33:41,  3.12s/it]
                                                         

 85%|████████▌ | 10201/12000 [6:00:56<1:33:41,  3.12s/it]
 85%|████████▌ | 10202/12000 [6:00:58<1:20:09,  2.67s/it]
                                                         

 85%|████████▌ | 10202/12000 [6:00:58<1:20:09,  2.67s/it]
 85%|████████▌ | 10203/12000 [6:00:59<1:12:19,  2.42s/it]
                                                         
{'loss': '0.5691', 'grad_norm': '0.4217', 'learning_rate': '1.265e-05', 'entropy': '0.5886', 'num_tokens': '5.469e+07', 'mean_token_accuracy': '0.8465', 'epoch': '0.08988'}
{'loss': '0.7173', 'grad_norm': '0.394', 'learning_rate': '1.264e-05', 'entropy': '0.8217', 'num_tokens': '5.47e+07', 'mean_token_accuracy': '0.8075', 'epoch': '0.08989'}
{'loss': '0.7501', 'grad_norm': '0.3795', 'learning_rate': '1.262e-05', 'entropy': '0.676', 'num_tokens': '5.47e+07', 'mean_token_accuracy': '0.8231', 'epoch': '0.0899'}
{'loss': '0.8216', 'grad_norm': '0.415', 'learning_rate': '1.261e-05', 'entropy': '0.762', 'num_tokens': '5.471e+07', 'mean_token_accuracy': '0.8066', 'epoch': '0.08991'}
{'loss': '0.9444', 'grad_norm': '0.4094', 'learning_rate': '1.26e-05', 'entropy': '0.9565', 'num_tokens': '5.471e+07', 'mean_token_accuracy': '0.7689', 'epoch': '0.08991'}
{'loss': '0.9997', 'grad_norm': '0.3406', 'learning_rate': '1.258e-05', 'entropy': '0.953', 'num_tokens': '5.472e+07', 'mean_token_accuracy': '0.7848', 'epoch': '0.08992'}
{'loss': '0.4635', 'grad_norm': '0.3024', 'learning_rate': '1.257e-05', 'entropy': '0.449', 'num_tokens': '5.473e+07', 'mean_token_accuracy': '0.8858', 'epoch': '0.08993'}
{'loss': '0.823', 'grad_norm': '0.394', 'learning_rate': '1.256e-05', 'entropy': '0.7765', 'num_tokens': '5.473e+07', 'mean_token_accuracy': '0.7936', 'epoch': '0.08994'}
{'loss': '0.904', 'grad_norm': '0.425', 'learning_rate': '1.254e-05', 'entropy': '0.894', 'num_tokens': '5.474e+07', 'mean_token_accuracy': '0.7751', 'epoch': '0.08995'}
{'loss': '0.6926', 'grad_norm': '0.3946', 'learning_rate': '1.253e-05', 'entropy': '0.7875', 'num_tokens': '5.474e+07', 'mean_token_accuracy': '0.8028', 'epoch': '0.08996'}
{'loss': '0.5605', 'grad_norm': '0.2924', 'learning_rate': '1.252e-05', 'entropy': '0.63', 'num_tokens': '5.475e+07', 'mean_token_accuracy': '0.845', 'epoch': '0.08997'}
{'loss': '0.5894', 'grad_norm': '0.4007', 'learning_rate': '1.25e-05', 'entropy': '0.6262', 'num_tokens': '5.475e+07', 'mean_token_accuracy': '0.8421', 'epoch': '0.08998'}
{'loss': '0.9026', 'grad_norm': '0.3998', 'learning_rate': '1.249e-05', 'entropy': '0.9413', 'num_tokens': '5.476e+07', 'mean_token_accuracy': '0.7628', 'epoch': '0.08999'}
{'loss': '0.6045', 'grad_norm': '0.283', 'learning_rate': '1.248e-05', 'entropy': '0.7142', 'num_tokens': '5.477e+07', 'mean_token_accuracy': '0.8147', 'epoch': '0.08999'}
{'loss': '0.7965', 'grad_norm': '0.3757', 'learning_rate': '1.246e-05', 'entropy': '0.9289', 'num_tokens': '5.477e+07', 'mean_token_accuracy': '0.7734', 'epoch': '0.09'}
{'loss': '0.7221', 'grad_norm': '0.4229', 'learning_rate': '1.245e-05', 'entropy': '0.6892', 'num_tokens': '5.478e+07', 'mean_token_accuracy': '0.8207', 'epoch': '0.09001'}
{'loss': '1.029', 'grad_norm': '0.3937', 'learning_rate': '1.244e-05', 'entropy': '0.956', 'num_tokens': '5.478e+07', 'mean_token_accuracy': '0.7616', 'epoch': '0.09002'}
{'loss': '0.8667', 'grad_norm': '0.4196', 'learning_rate': '1.242e-05', 'entropy': '0.8818', 'num_tokens': '5.479e+07', 'mean_token_accuracy': '0.7811', 'epoch': '0.09003'}
{'loss': '0.7032', 'grad_norm': '0.3399', 'learning_rate': '1.241e-05', 'entropy': '0.6913', 'num_tokens': '5.48e+07', 'mean_token_accuracy': '0.832', 'epoch': '0.09004'}
{'loss': '0.8075', 'grad_norm': '0.4165', 'learning_rate': '1.24e-05', 'entropy': '0.8231', 'num_tokens': '5.48e+07', 'mean_token_accuracy': '0.8116', 'epoch': '0.09005'}
{'loss': '0.5462', 'grad_norm': '0.3665', 'learning_rate': '1.238e-05', 'entropy': '0.5387', 'num_tokens': '5.481e+07', 'mean_token_accuracy': '0.8568', 'epoch': '0.09006'}
{'loss': '0.8856', 'grad_norm': '0.428', 'learning_rate': '1.237e-05', 'entropy': '0.9129', 'num_tokens': '5.481e+07', 'mean_token_accuracy': '0.7842', 'epoch': '0.09006'}
{'loss': '0.8557', 'grad_norm': '0.3966', 'learning_rate': '1.236e-05', 'entropy': '0.9023', 'num_tokens': '5.482e+07', 'mean_token_accuracy': '0.7686', 'epoch': '0.09007'}
{'loss': '0.8062', 'grad_norm': '0.4274', 'learning_rate': '1.234e-05', 'entropy': '0.8346', 'num_tokens': '5.482e+07', 'mean_token_accuracy': '0.7868', 'epoch': '0.09008'}
{'loss': '0.8823', 'grad_norm': '0.3991', 'learning_rate': '1.233e-05', 'entropy': '0.8547', 'num_tokens': '5.483e+07', 'mean_token_accuracy': '0.7838', 'epoch': '0.09009'}
{'loss': '0.9498', 'grad_norm': '0.4911', 'learning_rate': '1.232e-05', 'entropy': '0.9298', 'num_tokens': '5.483e+07', 'mean_token_accuracy': '0.7759', 'epoch': '0.0901'}
{'loss': '0.9441', 'grad_norm': '0.5598', 'learning_rate': '1.23e-05', 'entropy': '0.9494', 'num_tokens': '5.484e+07', 'mean_token_accuracy': '0.7673', 'epoch': '0.09011'}
{'loss': '0.9441', 'grad_norm': '0.3934', 'learning_rate': '1.229e-05', 'entropy': '0.909', 'num_tokens': '5.484e+07', 'mean_token_accuracy': '0.7862', 'epoch': '0.09012'}
{'loss': '0.8421', 'grad_norm': '0.4019', 'learning_rate': '1.228e-05', 'entropy': '0.8214', 'num_tokens': '5.485e+07', 'mean_token_accuracy': '0.7967', 'epoch': '0.09013'}
{'loss': '0.6752', 'grad_norm': '0.3859', 'learning_rate': '1.226e-05', 'entropy': '0.6456', 'num_tokens': '5.485e+07', 'mean_token_accuracy': '0.8293', 'epoch': '0.09014'}
{'loss': '0.6396', 'grad_norm': '0.3317', 'learning_rate': '1.225e-05', 'entropy': '0.846', 'num_tokens': '5.486e+07', 'mean_token_accuracy': '0.7957', 'epoch': '0.09014'}
{'loss': '0.4924', 'grad_norm': '0.3728', 'learning_rate': '1.224e-05', 'entropy': '0.5039', 'num_tokens': '5.486e+07', 'mean_token_accuracy': '0.8705', 'epoch': '0.09015'}
{'loss': '0.8065', 'grad_norm': '0.3626', 'learning_rate': '1.222e-05', 'entropy': '0.8124', 'num_tokens': '5.486e+07', 'mean_token_accuracy': '0.7852', 'epoch': '0.09016'}
{'loss': '0.5869', 'grad_norm': '0.3535', 'learning_rate': '1.221e-05', 'entropy': '0.6405', 'num_tokens': '5.487e+07', 'mean_token_accuracy': '0.8342', 'epoch': '0.09017'}
{'loss': '0.8216', 'grad_norm': '0.4834', 'learning_rate': '1.22e-05', 'entropy': '0.8155', 'num_tokens': '5.487e+07', 'mean_token_accuracy': '0.7895', 'epoch': '0.09018'}
{'loss': '0.7076', 'grad_norm': '0.3758', 'learning_rate': '1.218e-05', 'entropy': '0.685', 'num_tokens': '5.488e+07', 'mean_token_accuracy': '0.8187', 'epoch': '0.09019'}
{'loss': '1.15', 'grad_norm': '0.4682', 'learning_rate': '1.217e-05', 'entropy': '1.112', 'num_tokens': '5.489e+07', 'mean_token_accuracy': '0.7435', 'epoch': '0.0902'}
{'loss': '0.9181', 'grad_norm': '0.4447', 'learning_rate': '1.216e-05', 'entropy': '0.8787', 'num_tokens': '5.489e+07', 'mean_token_accuracy': '0.7743', 'epoch': '0.09021'}
{'loss': '0.6559', 'grad_norm': '0.3945', 'learning_rate': '1.214e-05', 'entropy': '0.7161', 'num_tokens': '5.49e+07', 'mean_token_accuracy': '0.8255', 'epoch': '0.09022'}
{'loss': '0.8834', 'grad_norm': '0.3985', 'learning_rate': '1.213e-05', 'entropy': '0.8577', 'num_tokens': '5.49e+07', 'mean_token_accuracy': '0.7898', 'epoch': '0.09022'}
{'loss': '0.6977', 'grad_norm': '0.3395', 'learning_rate': '1.212e-05', 'entropy': '0.6448', 'num_tokens': '5.491e+07', 'mean_token_accuracy': '0.827', 'epoch': '0.09023'}
{'loss': '0.7784', 'grad_norm': '0.4327', 'learning_rate': '1.211e-05', 'entropy': '0.7139', 'num_tokens': '5.491e+07', 'mean_token_accuracy': '0.8116', 'epoch': '0.09024'}
{'loss': '0.4528', 'grad_norm': '0.3047', 'learning_rate': '1.209e-05', 'entropy': '0.461', 'num_tokens': '5.492e+07', 'mean_token_accuracy': '0.8725', 'epoch': '0.09025'}
{'loss': '0.7042', 'grad_norm': '0.4052', 'learning_rate': '1.208e-05', 'entropy': '0.6976', 'num_tokens': '5.492e+07', 'mean_token_accuracy': '0.821', 'epoch': '0.09026'}
{'loss': '0.7212', 'grad_norm': '0.4401', 'learning_rate': '1.207e-05', 'entropy': '0.7522', 'num_tokens': '5.493e+07', 'mean_token_accuracy': '0.8082', 'epoch': '0.09027'}
{'loss': '1.811', 'grad_norm': '0.4488', 'learning_rate': '1.205e-05', 'entropy': '1.332', 'num_tokens': '5.494e+07', 'mean_token_accuracy': '0.701', 'epoch': '0.09028'}
{'loss': '0.5724', 'grad_norm': '0.4047', 'learning_rate': '1.204e-05', 'entropy': '0.5735', 'num_tokens': '5.494e+07', 'mean_token_accuracy': '0.847', 'epoch': '0.09029'}

 85%|████████▌ | 10203/12000 [6:00:59<1:12:19,  2.42s/it]
 85%|████████▌ | 10204/12000 [6:01:01<1:06:13,  2.21s/it]
                                                         

 85%|████████▌ | 10204/12000 [6:01:01<1:06:13,  2.21s/it]
 85%|████████▌ | 10205/12000 [6:01:03<59:17,  1.98s/it]  
                                                       

 85%|████████▌ | 10205/12000 [6:01:03<59:17,  1.98s/it]
 85%|████████▌ | 10206/12000 [6:01:05<1:00:25,  2.02s/it]
                                                         

 85%|████████▌ | 10206/12000 [6:01:05<1:00:25,  2.02s/it]
 85%|████████▌ | 10207/12000 [6:01:07<1:01:39,  2.06s/it]
                                                         

 85%|████████▌ | 10207/12000 [6:01:07<1:01:39,  2.06s/it]
 85%|████████▌ | 10208/12000 [6:01:08<56:10,  1.88s/it]  
                                                       

 85%|████████▌ | 10208/12000 [6:01:08<56:10,  1.88s/it]
 85%|████████▌ | 10209/12000 [6:01:10<53:12,  1.78s/it]
                                                       

 85%|████████▌ | 10209/12000 [6:01:10<53:12,  1.78s/it]
 85%|████████▌ | 10210/12000 [6:01:12<52:58,  1.78s/it]
                                                       

 85%|████████▌ | 10210/12000 [6:01:12<52:58,  1.78s/it]
 85%|████████▌ | 10211/12000 [6:01:13<49:57,  1.68s/it]
                                                       

 85%|████████▌ | 10211/12000 [6:01:13<49:57,  1.68s/it]
 85%|████████▌ | 10212/12000 [6:01:15<51:53,  1.74s/it]
                                                       

 85%|████████▌ | 10212/12000 [6:01:15<51:53,  1.74s/it]
 85%|████████▌ | 10213/12000 [6:01:20<1:22:19,  2.76s/it]
                                                         

 85%|████████▌ | 10213/12000 [6:01:20<1:22:19,  2.76s/it]
 85%|████████▌ | 10214/12000 [6:01:22<1:11:28,  2.40s/it]
                                                         

 85%|████████▌ | 10214/12000 [6:01:22<1:11:28,  2.40s/it]
 85%|████████▌ | 10215/12000 [6:01:26<1:24:21,  2.84s/it]
                                                         

 85%|████████▌ | 10215/12000 [6:01:26<1:24:21,  2.84s/it]
 85%|████████▌ | 10216/12000 [6:01:27<1:13:13,  2.46s/it]
                                                         

 85%|████████▌ | 10216/12000 [6:01:27<1:13:13,  2.46s/it]
 85%|████████▌ | 10217/12000 [6:01:29<1:04:34,  2.17s/it]
                                                         

 85%|████████▌ | 10217/12000 [6:01:29<1:04:34,  2.17s/it]
 85%|████████▌ | 10218/12000 [6:01:30<57:53,  1.95s/it]  
                                                       

 85%|████████▌ | 10218/12000 [6:01:30<57:53,  1.95s/it]
 85%|████████▌ | 10219/12000 [6:01:32<58:27,  1.97s/it]
                                                       

 85%|████████▌ | 10219/12000 [6:01:32<58:27,  1.97s/it]
 85%|████████▌ | 10220/12000 [6:01:34<1:00:54,  2.05s/it]
                                                         

 85%|████████▌ | 10220/12000 [6:01:34<1:00:54,  2.05s/it]
 85%|████████▌ | 10221/12000 [6:01:36<58:17,  1.97s/it]  
                                                       

 85%|████████▌ | 10221/12000 [6:01:36<58:17,  1.97s/it]
 85%|████████▌ | 10222/12000 [6:01:39<1:02:12,  2.10s/it]
                                                         

 85%|████████▌ | 10222/12000 [6:01:39<1:02:12,  2.10s/it]
 85%|████████▌ | 10223/12000 [6:01:41<1:09:24,  2.34s/it]
                                                         

 85%|████████▌ | 10223/12000 [6:01:41<1:09:24,  2.34s/it]
 85%|████████▌ | 10224/12000 [6:01:43<1:03:28,  2.14s/it]
                                                         

 85%|████████▌ | 10224/12000 [6:01:43<1:03:28,  2.14s/it]
 85%|████████▌ | 10225/12000 [6:01:45<59:03,  2.00s/it]  
                                                       

 85%|████████▌ | 10225/12000 [6:01:45<59:03,  2.00s/it]
 85%|████████▌ | 10226/12000 [6:01:47<56:45,  1.92s/it]
                                                       

 85%|████████▌ | 10226/12000 [6:01:47<56:45,  1.92s/it]
 85%|████████▌ | 10227/12000 [6:01:48<55:51,  1.89s/it]
                                                       

 85%|████████▌ | 10227/12000 [6:01:48<55:51,  1.89s/it]
 85%|████████▌ | 10228/12000 [6:01:52<1:10:09,  2.38s/it]
                                                         

 85%|████████▌ | 10228/12000 [6:01:52<1:10:09,  2.38s/it]
 85%|████████▌ | 10229/12000 [6:01:54<1:03:42,  2.16s/it]
                                                         

 85%|████████▌ | 10229/12000 [6:01:54<1:03:42,  2.16s/it]
 85%|████████▌ | 10230/12000 [6:01:56<1:02:53,  2.13s/it]
                                                         

 85%|████████▌ | 10230/12000 [6:01:56<1:02:53,  2.13s/it]
 85%|████████▌ | 10231/12000 [6:01:57<58:43,  1.99s/it]  
                                                       

 85%|████████▌ | 10231/12000 [6:01:57<58:43,  1.99s/it]
 85%|████████▌ | 10232/12000 [6:01:59<55:30,  1.88s/it]
                                                       

 85%|████████▌ | 10232/12000 [6:01:59<55:30,  1.88s/it]
 85%|████████▌ | 10233/12000 [6:02:01<53:44,  1.82s/it]
                                                       

 85%|████████▌ | 10233/12000 [6:02:01<53:44,  1.82s/it]
 85%|████████▌ | 10234/12000 [6:02:02<54:29,  1.85s/it]
                                                       

 85%|████████▌ | 10234/12000 [6:02:02<54:29,  1.85s/it]
 85%|████████▌ | 10235/12000 [6:02:04<51:56,  1.77s/it]
                                                       

 85%|████████▌ | 10235/12000 [6:02:04<51:56,  1.77s/it]
 85%|████████▌ | 10236/12000 [6:02:06<50:26,  1.72s/it]
                                                       

 85%|████████▌ | 10236/12000 [6:02:06<50:26,  1.72s/it]
 85%|████████▌ | 10237/12000 [6:02:09<1:07:51,  2.31s/it]
                                                         

 85%|████████▌ | 10237/12000 [6:02:09<1:07:51,  2.31s/it]
 85%|████████▌ | 10238/12000 [6:02:11<1:04:38,  2.20s/it]
                                                         

 85%|████████▌ | 10238/12000 [6:02:11<1:04:38,  2.20s/it]
 85%|████████▌ | 10239/12000 [6:02:13<1:01:05,  2.08s/it]
                                                         

 85%|████████▌ | 10239/12000 [6:02:13<1:01:05,  2.08s/it]
 85%|████████▌ | 10240/12000 [6:02:15<58:16,  1.99s/it]  
                                                       

 85%|████████▌ | 10240/12000 [6:02:15<58:16,  1.99s/it]
 85%|████████▌ | 10241/12000 [6:02:17<57:02,  1.95s/it]
                                                       

 85%|████████▌ | 10241/12000 [6:02:17<57:02,  1.95s/it]
 85%|████████▌ | 10242/12000 [6:02:18<53:09,  1.81s/it]
                                                       

 85%|████████▌ | 10242/12000 [6:02:18<53:09,  1.81s/it]
 85%|████████▌ | 10243/12000 [6:02:20<50:08,  1.71s/it]
                                                       

 85%|████████▌ | 10243/12000 [6:02:20<50:08,  1.71s/it]
 85%|████████▌ | 10244/12000 [6:02:21<46:56,  1.60s/it]
                                                       

 85%|████████▌ | 10244/12000 [6:02:21<46:56,  1.60s/it]
 85%|████████▌ | 10245/12000 [6:02:24<1:00:13,  2.06s/it]
                                                         

 85%|████████▌ | 10245/12000 [6:02:24<1:00:13,  2.06s/it]
 85%|████████▌ | 10246/12000 [6:02:26<56:27,  1.93s/it]  
                                                       

 85%|████████▌ | 10246/12000 [6:02:26<56:27,  1.93s/it]
 85%|████████▌ | 10247/12000 [6:02:27<51:19,  1.76s/it]
                                                       

 85%|████████▌ | 10247/12000 [6:02:27<51:19,  1.76s/it]
 85%|████████▌ | 10248/12000 [6:02:29<49:39,  1.70s/it]
                                                       

 85%|████████▌ | 10248/12000 [6:02:29<49:39,  1.70s/it]
 85%|████████▌ | 10249/12000 [6:02:31<51:56,  1.78s/it]
                                                       

 85%|████████▌ | 10249/12000 [6:02:31<51:56,  1.78s/it]
 85%|████████▌ | 10250/12000 [6:02:34<1:01:54,  2.12s/it]
                                                         
{'loss': '0.5909', 'grad_norm': '0.3386', 'learning_rate': '1.203e-05', 'entropy': '0.5799', 'num_tokens': '5.495e+07', 'mean_token_accuracy': '0.8537', 'epoch': '0.09029'}
{'loss': '0.7189', 'grad_norm': '0.4379', 'learning_rate': '1.201e-05', 'entropy': '0.6892', 'num_tokens': '5.495e+07', 'mean_token_accuracy': '0.8154', 'epoch': '0.0903'}
{'loss': '0.625', 'grad_norm': '0.3578', 'learning_rate': '1.2e-05', 'entropy': '0.6556', 'num_tokens': '5.496e+07', 'mean_token_accuracy': '0.8303', 'epoch': '0.09031'}
{'loss': '0.5939', 'grad_norm': '0.3089', 'learning_rate': '1.199e-05', 'entropy': '0.5559', 'num_tokens': '5.496e+07', 'mean_token_accuracy': '0.8508', 'epoch': '0.09032'}
{'loss': '0.8853', 'grad_norm': '0.5252', 'learning_rate': '1.197e-05', 'entropy': '0.8013', 'num_tokens': '5.497e+07', 'mean_token_accuracy': '0.7792', 'epoch': '0.09033'}
{'loss': '0.6284', 'grad_norm': '0.3671', 'learning_rate': '1.196e-05', 'entropy': '0.643', 'num_tokens': '5.497e+07', 'mean_token_accuracy': '0.8379', 'epoch': '0.09034'}
{'loss': '0.6901', 'grad_norm': '0.523', 'learning_rate': '1.195e-05', 'entropy': '0.7592', 'num_tokens': '5.498e+07', 'mean_token_accuracy': '0.8054', 'epoch': '0.09035'}
{'loss': '1.212', 'grad_norm': '0.5491', 'learning_rate': '1.193e-05', 'entropy': '1.16', 'num_tokens': '5.498e+07', 'mean_token_accuracy': '0.7231', 'epoch': '0.09036'}
{'loss': '0.5596', 'grad_norm': '0.3922', 'learning_rate': '1.192e-05', 'entropy': '0.5996', 'num_tokens': '5.499e+07', 'mean_token_accuracy': '0.8508', 'epoch': '0.09037'}
{'loss': '0.9791', 'grad_norm': '0.3876', 'learning_rate': '1.191e-05', 'entropy': '0.9439', 'num_tokens': '5.499e+07', 'mean_token_accuracy': '0.7724', 'epoch': '0.09037'}
{'loss': '0.6768', 'grad_norm': '0.2561', 'learning_rate': '1.19e-05', 'entropy': '0.8002', 'num_tokens': '5.5e+07', 'mean_token_accuracy': '0.7918', 'epoch': '0.09038'}
{'loss': '0.8412', 'grad_norm': '0.4338', 'learning_rate': '1.188e-05', 'entropy': '0.9315', 'num_tokens': '5.501e+07', 'mean_token_accuracy': '0.7741', 'epoch': '0.09039'}
{'loss': '0.6337', 'grad_norm': '0.3643', 'learning_rate': '1.187e-05', 'entropy': '0.601', 'num_tokens': '5.501e+07', 'mean_token_accuracy': '0.8443', 'epoch': '0.0904'}
{'loss': '1.058', 'grad_norm': '0.4519', 'learning_rate': '1.186e-05', 'entropy': '1.016', 'num_tokens': '5.502e+07', 'mean_token_accuracy': '0.7573', 'epoch': '0.09041'}
{'loss': '0.8109', 'grad_norm': '0.3497', 'learning_rate': '1.184e-05', 'entropy': '0.8417', 'num_tokens': '5.502e+07', 'mean_token_accuracy': '0.7817', 'epoch': '0.09042'}
{'loss': '0.63', 'grad_norm': '0.3372', 'learning_rate': '1.183e-05', 'entropy': '0.6606', 'num_tokens': '5.502e+07', 'mean_token_accuracy': '0.8106', 'epoch': '0.09043'}
{'loss': '0.7892', 'grad_norm': '0.3818', 'learning_rate': '1.182e-05', 'entropy': '0.7807', 'num_tokens': '5.503e+07', 'mean_token_accuracy': '0.8008', 'epoch': '0.09044'}
{'loss': '0.7994', 'grad_norm': '0.5061', 'learning_rate': '1.18e-05', 'entropy': '0.8658', 'num_tokens': '5.503e+07', 'mean_token_accuracy': '0.796', 'epoch': '0.09045'}
{'loss': '0.7058', 'grad_norm': '0.4319', 'learning_rate': '1.179e-05', 'entropy': '0.6431', 'num_tokens': '5.504e+07', 'mean_token_accuracy': '0.8371', 'epoch': '0.09045'}
{'loss': '0.5015', 'grad_norm': '0.286', 'learning_rate': '1.178e-05', 'entropy': '0.6063', 'num_tokens': '5.504e+07', 'mean_token_accuracy': '0.8321', 'epoch': '0.09046'}
{'loss': '0.926', 'grad_norm': '0.4238', 'learning_rate': '1.177e-05', 'entropy': '1.059', 'num_tokens': '5.505e+07', 'mean_token_accuracy': '0.7498', 'epoch': '0.09047'}
{'loss': '0.8089', 'grad_norm': '0.4322', 'learning_rate': '1.175e-05', 'entropy': '0.8789', 'num_tokens': '5.506e+07', 'mean_token_accuracy': '0.7768', 'epoch': '0.09048'}
{'loss': '0.9505', 'grad_norm': '0.5114', 'learning_rate': '1.174e-05', 'entropy': '0.9618', 'num_tokens': '5.506e+07', 'mean_token_accuracy': '0.7606', 'epoch': '0.09049'}
{'loss': '0.7281', 'grad_norm': '0.5076', 'learning_rate': '1.173e-05', 'entropy': '0.8429', 'num_tokens': '5.507e+07', 'mean_token_accuracy': '0.7876', 'epoch': '0.0905'}
{'loss': '0.724', 'grad_norm': '0.4033', 'learning_rate': '1.171e-05', 'entropy': '0.6837', 'num_tokens': '5.507e+07', 'mean_token_accuracy': '0.8178', 'epoch': '0.09051'}
{'loss': '1.093', 'grad_norm': '0.3438', 'learning_rate': '1.17e-05', 'entropy': '0.921', 'num_tokens': '5.508e+07', 'mean_token_accuracy': '0.7776', 'epoch': '0.09052'}
{'loss': '0.6844', 'grad_norm': '0.3551', 'learning_rate': '1.169e-05', 'entropy': '0.7025', 'num_tokens': '5.508e+07', 'mean_token_accuracy': '0.8297', 'epoch': '0.09052'}
{'loss': '0.8184', 'grad_norm': '0.3393', 'learning_rate': '1.168e-05', 'entropy': '0.8226', 'num_tokens': '5.509e+07', 'mean_token_accuracy': '0.8003', 'epoch': '0.09053'}
{'loss': '0.878', 'grad_norm': '0.3804', 'learning_rate': '1.166e-05', 'entropy': '0.7804', 'num_tokens': '5.51e+07', 'mean_token_accuracy': '0.8037', 'epoch': '0.09054'}
{'loss': '0.665', 'grad_norm': '0.3521', 'learning_rate': '1.165e-05', 'entropy': '0.6437', 'num_tokens': '5.51e+07', 'mean_token_accuracy': '0.8287', 'epoch': '0.09055'}
{'loss': '1.015', 'grad_norm': '0.4373', 'learning_rate': '1.164e-05', 'entropy': '0.9686', 'num_tokens': '5.511e+07', 'mean_token_accuracy': '0.7642', 'epoch': '0.09056'}
{'loss': '0.8894', 'grad_norm': '0.4347', 'learning_rate': '1.162e-05', 'entropy': '0.9232', 'num_tokens': '5.511e+07', 'mean_token_accuracy': '0.7806', 'epoch': '0.09057'}
{'loss': '0.8572', 'grad_norm': '0.3849', 'learning_rate': '1.161e-05', 'entropy': '0.9014', 'num_tokens': '5.512e+07', 'mean_token_accuracy': '0.7695', 'epoch': '0.09058'}
{'loss': '0.6844', 'grad_norm': '0.377', 'learning_rate': '1.16e-05', 'entropy': '0.6502', 'num_tokens': '5.512e+07', 'mean_token_accuracy': '0.8305', 'epoch': '0.09059'}
{'loss': '0.8298', 'grad_norm': '0.3686', 'learning_rate': '1.158e-05', 'entropy': '0.7781', 'num_tokens': '5.513e+07', 'mean_token_accuracy': '0.8038', 'epoch': '0.0906'}
{'loss': '0.7662', 'grad_norm': '0.3972', 'learning_rate': '1.157e-05', 'entropy': '0.7467', 'num_tokens': '5.513e+07', 'mean_token_accuracy': '0.8062', 'epoch': '0.0906'}
{'loss': '0.7427', 'grad_norm': '0.4185', 'learning_rate': '1.156e-05', 'entropy': '0.7401', 'num_tokens': '5.514e+07', 'mean_token_accuracy': '0.8163', 'epoch': '0.09061'}
{'loss': '0.8672', 'grad_norm': '0.5657', 'learning_rate': '1.155e-05', 'entropy': '0.8501', 'num_tokens': '5.514e+07', 'mean_token_accuracy': '0.78', 'epoch': '0.09062'}
{'loss': '0.817', 'grad_norm': '0.4313', 'learning_rate': '1.153e-05', 'entropy': '0.7415', 'num_tokens': '5.515e+07', 'mean_token_accuracy': '0.816', 'epoch': '0.09063'}
{'loss': '0.983', 'grad_norm': '0.5225', 'learning_rate': '1.152e-05', 'entropy': '0.9911', 'num_tokens': '5.515e+07', 'mean_token_accuracy': '0.7587', 'epoch': '0.09064'}
{'loss': '0.8044', 'grad_norm': '0.3611', 'learning_rate': '1.151e-05', 'entropy': '0.8112', 'num_tokens': '5.516e+07', 'mean_token_accuracy': '0.7901', 'epoch': '0.09065'}
{'loss': '0.6546', 'grad_norm': '0.3441', 'learning_rate': '1.149e-05', 'entropy': '0.6239', 'num_tokens': '5.516e+07', 'mean_token_accuracy': '0.835', 'epoch': '0.09066'}
{'loss': '0.4918', 'grad_norm': '0.3382', 'learning_rate': '1.148e-05', 'entropy': '0.5307', 'num_tokens': '5.517e+07', 'mean_token_accuracy': '0.8601', 'epoch': '0.09067'}
{'loss': '0.6291', 'grad_norm': '0.378', 'learning_rate': '1.147e-05', 'entropy': '0.6757', 'num_tokens': '5.517e+07', 'mean_token_accuracy': '0.8247', 'epoch': '0.09068'}
{'loss': '0.7337', 'grad_norm': '0.3708', 'learning_rate': '1.146e-05', 'entropy': '0.7537', 'num_tokens': '5.518e+07', 'mean_token_accuracy': '0.8028', 'epoch': '0.09068'}
{'loss': '0.8725', 'grad_norm': '0.5283', 'learning_rate': '1.144e-05', 'entropy': '0.9006', 'num_tokens': '5.518e+07', 'mean_token_accuracy': '0.7922', 'epoch': '0.09069'}
{'loss': '1.053', 'grad_norm': '0.4587', 'learning_rate': '1.143e-05', 'entropy': '1.08', 'num_tokens': '5.519e+07', 'mean_token_accuracy': '0.7167', 'epoch': '0.0907'}

 85%|████████▌ | 10250/12000 [6:02:34<1:01:54,  2.12s/it]
 85%|████████▌ | 10251/12000 [6:02:37<1:10:48,  2.43s/it]
                                                         

 85%|████████▌ | 10251/12000 [6:02:37<1:10:48,  2.43s/it]
 85%|████████▌ | 10252/12000 [6:02:39<1:06:55,  2.30s/it]
                                                         

 85%|████████▌ | 10252/12000 [6:02:39<1:06:55,  2.30s/it]
 85%|████████▌ | 10253/12000 [6:02:40<59:52,  2.06s/it]  
                                                       

 85%|████████▌ | 10253/12000 [6:02:40<59:52,  2.06s/it]
 85%|████████▌ | 10254/12000 [6:02:44<1:13:49,  2.54s/it]
                                                         

 85%|████████▌ | 10254/12000 [6:02:44<1:13:49,  2.54s/it]
 85%|████████▌ | 10255/12000 [6:02:45<1:03:38,  2.19s/it]
                                                         

 85%|████████▌ | 10255/12000 [6:02:45<1:03:38,  2.19s/it]
 85%|████████▌ | 10256/12000 [6:02:47<59:12,  2.04s/it]  
                                                       

 85%|████████▌ | 10256/12000 [6:02:47<59:12,  2.04s/it]
 85%|████████▌ | 10257/12000 [6:02:49<55:33,  1.91s/it]
                                                       

 85%|████████▌ | 10257/12000 [6:02:49<55:33,  1.91s/it]
 85%|████████▌ | 10258/12000 [6:02:50<51:57,  1.79s/it]
                                                       

 85%|████████▌ | 10258/12000 [6:02:50<51:57,  1.79s/it]
 85%|████████▌ | 10259/12000 [6:02:53<1:04:56,  2.24s/it]
                                                         

 85%|████████▌ | 10259/12000 [6:02:53<1:04:56,  2.24s/it]
 86%|████████▌ | 10260/12000 [6:02:55<1:01:51,  2.13s/it]
                                                         

 86%|████████▌ | 10260/12000 [6:02:55<1:01:51,  2.13s/it]
 86%|████████▌ | 10261/12000 [6:02:57<1:00:29,  2.09s/it]
                                                         

 86%|████████▌ | 10261/12000 [6:02:57<1:00:29,  2.09s/it]
 86%|████████▌ | 10262/12000 [6:02:59<58:17,  2.01s/it]  
                                                       

 86%|████████▌ | 10262/12000 [6:02:59<58:17,  2.01s/it]
 86%|████████▌ | 10263/12000 [6:03:01<54:42,  1.89s/it]
                                                       

 86%|████████▌ | 10263/12000 [6:03:01<54:42,  1.89s/it]
 86%|████████▌ | 10264/12000 [6:03:02<51:57,  1.80s/it]
                                                       

 86%|████████▌ | 10264/12000 [6:03:02<51:57,  1.80s/it]
 86%|████████▌ | 10265/12000 [6:03:04<53:41,  1.86s/it]
                                                       

 86%|████████▌ | 10265/12000 [6:03:04<53:41,  1.86s/it]
 86%|████████▌ | 10266/12000 [6:03:07<58:38,  2.03s/it]
                                                       

 86%|████████▌ | 10266/12000 [6:03:07<58:38,  2.03s/it]
 86%|████████▌ | 10267/12000 [6:03:13<1:34:19,  3.27s/it]
                                                         

 86%|████████▌ | 10267/12000 [6:03:13<1:34:19,  3.27s/it]
 86%|████████▌ | 10268/12000 [6:03:15<1:25:36,  2.97s/it]
                                                         

 86%|████████▌ | 10268/12000 [6:03:15<1:25:36,  2.97s/it]
 86%|████████▌ | 10269/12000 [6:03:17<1:13:08,  2.54s/it]
                                                         

 86%|████████▌ | 10269/12000 [6:03:17<1:13:08,  2.54s/it]
 86%|████████▌ | 10270/12000 [6:03:18<1:04:48,  2.25s/it]
                                                         

 86%|████████▌ | 10270/12000 [6:03:18<1:04:48,  2.25s/it]
 86%|████████▌ | 10271/12000 [6:03:20<58:49,  2.04s/it]  
                                                       

 86%|████████▌ | 10271/12000 [6:03:20<58:49,  2.04s/it]
 86%|████████▌ | 10272/12000 [6:03:22<59:24,  2.06s/it]
                                                       

 86%|████████▌ | 10272/12000 [6:03:22<59:24,  2.06s/it]
 86%|████████▌ | 10273/12000 [6:03:24<59:09,  2.06s/it]
                                                       

 86%|████████▌ | 10273/12000 [6:03:24<59:09,  2.06s/it]
 86%|████████▌ | 10274/12000 [6:03:25<54:57,  1.91s/it]
                                                       

 86%|████████▌ | 10274/12000 [6:03:25<54:57,  1.91s/it]
 86%|████████▌ | 10275/12000 [6:03:27<53:23,  1.86s/it]
                                                       

 86%|████████▌ | 10275/12000 [6:03:27<53:23,  1.86s/it]
 86%|████████▌ | 10276/12000 [6:03:29<50:29,  1.76s/it]
                                                       

 86%|████████▌ | 10276/12000 [6:03:29<50:29,  1.76s/it]
 86%|████████▌ | 10277/12000 [6:03:30<49:33,  1.73s/it]
                                                       

 86%|████████▌ | 10277/12000 [6:03:30<49:33,  1.73s/it]
 86%|████████▌ | 10278/12000 [6:03:32<48:19,  1.68s/it]
                                                       

 86%|████████▌ | 10278/12000 [6:03:32<48:19,  1.68s/it]
 86%|████████▌ | 10279/12000 [6:03:33<45:56,  1.60s/it]
                                                       

 86%|████████▌ | 10279/12000 [6:03:33<45:56,  1.60s/it]
 86%|████████▌ | 10280/12000 [6:03:35<45:17,  1.58s/it]
                                                       

 86%|████████▌ | 10280/12000 [6:03:35<45:17,  1.58s/it]
 86%|████████▌ | 10281/12000 [6:03:39<1:02:45,  2.19s/it]
                                                         

 86%|████████▌ | 10281/12000 [6:03:39<1:02:45,  2.19s/it]
 86%|████████▌ | 10282/12000 [6:03:44<1:32:42,  3.24s/it]
                                                         

 86%|████████▌ | 10282/12000 [6:03:44<1:32:42,  3.24s/it]
 86%|████████▌ | 10283/12000 [6:03:47<1:27:56,  3.07s/it]
                                                         

 86%|████████▌ | 10283/12000 [6:03:47<1:27:56,  3.07s/it]
 86%|████████▌ | 10284/12000 [6:03:48<1:13:56,  2.59s/it]
                                                         

 86%|████████▌ | 10284/12000 [6:03:48<1:13:56,  2.59s/it]
 86%|████████▌ | 10285/12000 [6:03:50<1:08:00,  2.38s/it]
                                                         

 86%|████████▌ | 10285/12000 [6:03:50<1:08:00,  2.38s/it]
 86%|████████▌ | 10286/12000 [6:03:54<1:18:24,  2.75s/it]
                                                         

 86%|████████▌ | 10286/12000 [6:03:54<1:18:24,  2.75s/it]
 86%|████████▌ | 10287/12000 [6:03:55<1:07:09,  2.35s/it]
                                                         

 86%|████████▌ | 10287/12000 [6:03:55<1:07:09,  2.35s/it]
 86%|████████▌ | 10288/12000 [6:03:57<59:51,  2.10s/it]  
                                                       

 86%|████████▌ | 10288/12000 [6:03:57<59:51,  2.10s/it]
 86%|████████▌ | 10289/12000 [6:04:00<1:07:08,  2.35s/it]
                                                         

 86%|████████▌ | 10289/12000 [6:04:00<1:07:08,  2.35s/it]
 86%|████████▌ | 10290/12000 [6:04:01<1:00:59,  2.14s/it]
                                                         

 86%|████████▌ | 10290/12000 [6:04:01<1:00:59,  2.14s/it]
 86%|████████▌ | 10291/12000 [6:04:03<58:54,  2.07s/it]  
                                                       

 86%|████████▌ | 10291/12000 [6:04:03<58:54,  2.07s/it]
 86%|████████▌ | 10292/12000 [6:04:05<55:25,  1.95s/it]
                                                       

 86%|████████▌ | 10292/12000 [6:04:05<55:25,  1.95s/it]
 86%|████████▌ | 10293/12000 [6:04:07<53:57,  1.90s/it]
                                                       

 86%|████████▌ | 10293/12000 [6:04:07<53:57,  1.90s/it]
 86%|████████▌ | 10294/12000 [6:04:08<52:23,  1.84s/it]
                                                       

 86%|████████▌ | 10294/12000 [6:04:08<52:23,  1.84s/it]
 86%|████████▌ | 10295/12000 [6:04:11<57:22,  2.02s/it]
                                                       

 86%|████████▌ | 10295/12000 [6:04:11<57:22,  2.02s/it]
 86%|████████▌ | 10296/12000 [6:04:12<52:34,  1.85s/it]
                                                       

 86%|████████▌ | 10296/12000 [6:04:12<52:34,  1.85s/it]
 86%|████████▌ | 10297/12000 [6:04:14<49:09,  1.73s/it]
                                                       
{'loss': '0.6691', 'grad_norm': '0.4056', 'learning_rate': '1.142e-05', 'entropy': '0.6658', 'num_tokens': '5.519e+07', 'mean_token_accuracy': '0.8301', 'epoch': '0.09071'}
{'loss': '0.6772', 'grad_norm': '0.3524', 'learning_rate': '1.141e-05', 'entropy': '0.6944', 'num_tokens': '5.52e+07', 'mean_token_accuracy': '0.8184', 'epoch': '0.09072'}
{'loss': '0.7541', 'grad_norm': '0.4412', 'learning_rate': '1.139e-05', 'entropy': '0.735', 'num_tokens': '5.52e+07', 'mean_token_accuracy': '0.8308', 'epoch': '0.09073'}
{'loss': '0.6405', 'grad_norm': '0.3823', 'learning_rate': '1.138e-05', 'entropy': '0.7119', 'num_tokens': '5.521e+07', 'mean_token_accuracy': '0.8247', 'epoch': '0.09074'}
{'loss': '0.948', 'grad_norm': '0.3093', 'learning_rate': '1.137e-05', 'entropy': '0.8494', 'num_tokens': '5.522e+07', 'mean_token_accuracy': '0.7756', 'epoch': '0.09075'}
{'loss': '0.6117', 'grad_norm': '0.3885', 'learning_rate': '1.135e-05', 'entropy': '0.6296', 'num_tokens': '5.522e+07', 'mean_token_accuracy': '0.8266', 'epoch': '0.09076'}
{'loss': '0.6084', 'grad_norm': '0.438', 'learning_rate': '1.134e-05', 'entropy': '0.6253', 'num_tokens': '5.523e+07', 'mean_token_accuracy': '0.8279', 'epoch': '0.09076'}
{'loss': '1.017', 'grad_norm': '0.4482', 'learning_rate': '1.133e-05', 'entropy': '0.9152', 'num_tokens': '5.523e+07', 'mean_token_accuracy': '0.7704', 'epoch': '0.09077'}
{'loss': '0.6979', 'grad_norm': '0.3518', 'learning_rate': '1.132e-05', 'entropy': '0.7529', 'num_tokens': '5.523e+07', 'mean_token_accuracy': '0.8179', 'epoch': '0.09078'}
{'loss': '0.5633', 'grad_norm': '0.4307', 'learning_rate': '1.13e-05', 'entropy': '0.5711', 'num_tokens': '5.524e+07', 'mean_token_accuracy': '0.8544', 'epoch': '0.09079'}
{'loss': '0.7153', 'grad_norm': '0.3409', 'learning_rate': '1.129e-05', 'entropy': '0.7734', 'num_tokens': '5.524e+07', 'mean_token_accuracy': '0.8045', 'epoch': '0.0908'}
{'loss': '0.9571', 'grad_norm': '0.5121', 'learning_rate': '1.128e-05', 'entropy': '0.9013', 'num_tokens': '5.525e+07', 'mean_token_accuracy': '0.777', 'epoch': '0.09081'}
{'loss': '0.6619', 'grad_norm': '0.3408', 'learning_rate': '1.127e-05', 'entropy': '0.746', 'num_tokens': '5.526e+07', 'mean_token_accuracy': '0.8249', 'epoch': '0.09082'}
{'loss': '0.851', 'grad_norm': '0.4213', 'learning_rate': '1.125e-05', 'entropy': '0.8826', 'num_tokens': '5.526e+07', 'mean_token_accuracy': '0.7715', 'epoch': '0.09083'}
{'loss': '0.9744', 'grad_norm': '0.4262', 'learning_rate': '1.124e-05', 'entropy': '1.08', 'num_tokens': '5.526e+07', 'mean_token_accuracy': '0.7331', 'epoch': '0.09083'}
{'loss': '0.9495', 'grad_norm': '0.4306', 'learning_rate': '1.123e-05', 'entropy': '0.9559', 'num_tokens': '5.527e+07', 'mean_token_accuracy': '0.764', 'epoch': '0.09084'}
{'loss': '0.8819', 'grad_norm': '0.3886', 'learning_rate': '1.121e-05', 'entropy': '0.7889', 'num_tokens': '5.528e+07', 'mean_token_accuracy': '0.8094', 'epoch': '0.09085'}
{'loss': '0.9388', 'grad_norm': '0.2989', 'learning_rate': '1.12e-05', 'entropy': '0.9172', 'num_tokens': '5.529e+07', 'mean_token_accuracy': '0.7791', 'epoch': '0.09086'}
{'loss': '0.7871', 'grad_norm': '0.3716', 'learning_rate': '1.119e-05', 'entropy': '0.8234', 'num_tokens': '5.529e+07', 'mean_token_accuracy': '0.7979', 'epoch': '0.09087'}
{'loss': '1.11', 'grad_norm': '0.4543', 'learning_rate': '1.118e-05', 'entropy': '1.089', 'num_tokens': '5.53e+07', 'mean_token_accuracy': '0.7383', 'epoch': '0.09088'}
{'loss': '0.8084', 'grad_norm': '0.3913', 'learning_rate': '1.116e-05', 'entropy': '0.9177', 'num_tokens': '5.53e+07', 'mean_token_accuracy': '0.7705', 'epoch': '0.09089'}
{'loss': '0.8158', 'grad_norm': '0.449', 'learning_rate': '1.115e-05', 'entropy': '0.8093', 'num_tokens': '5.531e+07', 'mean_token_accuracy': '0.7943', 'epoch': '0.0909'}
{'loss': '0.6941', 'grad_norm': '0.3679', 'learning_rate': '1.114e-05', 'entropy': '0.7244', 'num_tokens': '5.531e+07', 'mean_token_accuracy': '0.8067', 'epoch': '0.09091'}
{'loss': '0.805', 'grad_norm': '0.4043', 'learning_rate': '1.113e-05', 'entropy': '0.8041', 'num_tokens': '5.532e+07', 'mean_token_accuracy': '0.7957', 'epoch': '0.09091'}
{'loss': '0.7642', 'grad_norm': '0.4493', 'learning_rate': '1.111e-05', 'entropy': '0.8628', 'num_tokens': '5.532e+07', 'mean_token_accuracy': '0.7894', 'epoch': '0.09092'}
{'loss': '0.6119', 'grad_norm': '0.2947', 'learning_rate': '1.11e-05', 'entropy': '0.6519', 'num_tokens': '5.533e+07', 'mean_token_accuracy': '0.8311', 'epoch': '0.09093'}
{'loss': '0.5811', 'grad_norm': '0.3368', 'learning_rate': '1.109e-05', 'entropy': '0.5958', 'num_tokens': '5.533e+07', 'mean_token_accuracy': '0.8444', 'epoch': '0.09094'}
{'loss': '0.7552', 'grad_norm': '0.3976', 'learning_rate': '1.108e-05', 'entropy': '0.7308', 'num_tokens': '5.534e+07', 'mean_token_accuracy': '0.8106', 'epoch': '0.09095'}
{'loss': '1.135', 'grad_norm': '0.4225', 'learning_rate': '1.106e-05', 'entropy': '1.124', 'num_tokens': '5.534e+07', 'mean_token_accuracy': '0.7233', 'epoch': '0.09096'}
{'loss': '0.7596', 'grad_norm': '0.4535', 'learning_rate': '1.105e-05', 'entropy': '0.8134', 'num_tokens': '5.534e+07', 'mean_token_accuracy': '0.7956', 'epoch': '0.09097'}
{'loss': '0.7333', 'grad_norm': '0.4349', 'learning_rate': '1.104e-05', 'entropy': '0.7292', 'num_tokens': '5.535e+07', 'mean_token_accuracy': '0.8179', 'epoch': '0.09098'}
{'loss': '0.7692', 'grad_norm': '0.4693', 'learning_rate': '1.102e-05', 'entropy': '0.7926', 'num_tokens': '5.535e+07', 'mean_token_accuracy': '0.7962', 'epoch': '0.09099'}
{'loss': '0.8472', 'grad_norm': '0.3285', 'learning_rate': '1.101e-05', 'entropy': '0.8794', 'num_tokens': '5.536e+07', 'mean_token_accuracy': '0.7822', 'epoch': '0.09099'}
{'loss': '0.8336', 'grad_norm': '0.5709', 'learning_rate': '1.1e-05', 'entropy': '0.7927', 'num_tokens': '5.537e+07', 'mean_token_accuracy': '0.7921', 'epoch': '0.091'}
{'loss': '0.6561', 'grad_norm': '0.3845', 'learning_rate': '1.099e-05', 'entropy': '0.6707', 'num_tokens': '5.537e+07', 'mean_token_accuracy': '0.8274', 'epoch': '0.09101'}
{'loss': '0.808', 'grad_norm': '0.4226', 'learning_rate': '1.097e-05', 'entropy': '0.7773', 'num_tokens': '5.538e+07', 'mean_token_accuracy': '0.8005', 'epoch': '0.09102'}
{'loss': '0.873', 'grad_norm': '0.4285', 'learning_rate': '1.096e-05', 'entropy': '0.8854', 'num_tokens': '5.539e+07', 'mean_token_accuracy': '0.7746', 'epoch': '0.09103'}
{'loss': '0.6755', 'grad_norm': '0.3583', 'learning_rate': '1.095e-05', 'entropy': '0.7353', 'num_tokens': '5.539e+07', 'mean_token_accuracy': '0.8192', 'epoch': '0.09104'}
{'loss': '0.7189', 'grad_norm': '0.3305', 'learning_rate': '1.094e-05', 'entropy': '0.713', 'num_tokens': '5.54e+07', 'mean_token_accuracy': '0.8246', 'epoch': '0.09105'}
{'loss': '0.8555', 'grad_norm': '0.3256', 'learning_rate': '1.092e-05', 'entropy': '0.9769', 'num_tokens': '5.541e+07', 'mean_token_accuracy': '0.7612', 'epoch': '0.09106'}
{'loss': '0.6549', 'grad_norm': '0.371', 'learning_rate': '1.091e-05', 'entropy': '0.6796', 'num_tokens': '5.541e+07', 'mean_token_accuracy': '0.8234', 'epoch': '0.09106'}
{'loss': '0.9377', 'grad_norm': '0.3441', 'learning_rate': '1.09e-05', 'entropy': '0.7664', 'num_tokens': '5.542e+07', 'mean_token_accuracy': '0.8232', 'epoch': '0.09107'}
{'loss': '0.8376', 'grad_norm': '0.4295', 'learning_rate': '1.089e-05', 'entropy': '0.8147', 'num_tokens': '5.542e+07', 'mean_token_accuracy': '0.7825', 'epoch': '0.09108'}
{'loss': '1.028', 'grad_norm': '0.4461', 'learning_rate': '1.087e-05', 'entropy': '1.02', 'num_tokens': '5.543e+07', 'mean_token_accuracy': '0.7489', 'epoch': '0.09109'}
{'loss': '0.6609', 'grad_norm': '0.4448', 'learning_rate': '1.086e-05', 'entropy': '0.6411', 'num_tokens': '5.543e+07', 'mean_token_accuracy': '0.8413', 'epoch': '0.0911'}
{'loss': '0.9497', 'grad_norm': '0.4101', 'learning_rate': '1.085e-05', 'entropy': '0.8996', 'num_tokens': '5.544e+07', 'mean_token_accuracy': '0.7767', 'epoch': '0.09111'}
{'loss': '0.8807', 'grad_norm': '0.4223', 'learning_rate': '1.084e-05', 'entropy': '0.8483', 'num_tokens': '5.544e+07', 'mean_token_accuracy': '0.7935', 'epoch': '0.09112'}

 86%|████████▌ | 10297/12000 [6:04:14<49:09,  1.73s/it]
 86%|████████▌ | 10298/12000 [6:04:15<47:14,  1.67s/it]
                                                       

 86%|████████▌ | 10298/12000 [6:04:15<47:14,  1.67s/it]
 86%|████████▌ | 10299/12000 [6:04:20<1:10:43,  2.49s/it]
                                                         

 86%|████████▌ | 10299/12000 [6:04:20<1:10:43,  2.49s/it]
 86%|████████▌ | 10300/12000 [6:04:21<1:01:33,  2.17s/it]
                                                         

 86%|████████▌ | 10300/12000 [6:04:21<1:01:33,  2.17s/it]
 86%|████████▌ | 10301/12000 [6:04:24<1:07:30,  2.38s/it]
                                                         

 86%|████████▌ | 10301/12000 [6:04:24<1:07:30,  2.38s/it]
 86%|████████▌ | 10302/12000 [6:04:26<1:03:28,  2.24s/it]
                                                         

 86%|████████▌ | 10302/12000 [6:04:26<1:03:28,  2.24s/it]
 86%|████████▌ | 10303/12000 [6:04:30<1:15:19,  2.66s/it]
                                                         

 86%|████████▌ | 10303/12000 [6:04:30<1:15:19,  2.66s/it]
 86%|████████▌ | 10304/12000 [6:04:32<1:09:42,  2.47s/it]
                                                         

 86%|████████▌ | 10304/12000 [6:04:32<1:09:42,  2.47s/it]
 86%|████████▌ | 10305/12000 [6:04:33<1:02:27,  2.21s/it]
                                                         

 86%|████████▌ | 10305/12000 [6:04:33<1:02:27,  2.21s/it]
 86%|████████▌ | 10306/12000 [6:04:36<1:09:04,  2.45s/it]
                                                         

 86%|████████▌ | 10306/12000 [6:04:36<1:09:04,  2.45s/it]
 86%|████████▌ | 10307/12000 [6:04:38<1:03:52,  2.26s/it]
                                                         

 86%|████████▌ | 10307/12000 [6:04:38<1:03:52,  2.26s/it]
 86%|████████▌ | 10308/12000 [6:04:40<58:17,  2.07s/it]  
                                                       

 86%|████████▌ | 10308/12000 [6:04:40<58:17,  2.07s/it]
 86%|████████▌ | 10309/12000 [6:04:42<1:00:00,  2.13s/it]
                                                         

 86%|████████▌ | 10309/12000 [6:04:42<1:00:00,  2.13s/it]
 86%|████████▌ | 10310/12000 [6:04:44<56:14,  2.00s/it]  
                                                       

 86%|████████▌ | 10310/12000 [6:04:44<56:14,  2.00s/it]
 86%|████████▌ | 10311/12000 [6:04:47<1:05:26,  2.32s/it]
                                                         

 86%|████████▌ | 10311/12000 [6:04:47<1:05:26,  2.32s/it]
 86%|████████▌ | 10312/12000 [6:04:48<57:43,  2.05s/it]  
                                                       

 86%|████████▌ | 10312/12000 [6:04:48<57:43,  2.05s/it]
 86%|████████▌ | 10313/12000 [6:04:50<55:15,  1.97s/it]
                                                       

 86%|████████▌ | 10313/12000 [6:04:50<55:15,  1.97s/it]
 86%|████████▌ | 10314/12000 [6:04:51<51:24,  1.83s/it]
                                                       

 86%|████████▌ | 10314/12000 [6:04:51<51:24,  1.83s/it]
 86%|████████▌ | 10315/12000 [6:04:53<50:26,  1.80s/it]
                                                       

 86%|████████▌ | 10315/12000 [6:04:53<50:26,  1.80s/it]
 86%|████████▌ | 10316/12000 [6:04:55<48:57,  1.74s/it]
                                                       

 86%|████████▌ | 10316/12000 [6:04:55<48:57,  1.74s/it]
 86%|████████▌ | 10317/12000 [6:04:57<50:30,  1.80s/it]
                                                       

 86%|████████▌ | 10317/12000 [6:04:57<50:30,  1.80s/it]
 86%|████████▌ | 10318/12000 [6:04:58<50:46,  1.81s/it]
                                                       

 86%|████████▌ | 10318/12000 [6:04:58<50:46,  1.81s/it]
 86%|████████▌ | 10319/12000 [6:05:02<1:05:35,  2.34s/it]
                                                         

 86%|████████▌ | 10319/12000 [6:05:02<1:05:35,  2.34s/it]
 86%|████████▌ | 10320/12000 [6:05:04<1:03:25,  2.27s/it]
                                                         

 86%|████████▌ | 10320/12000 [6:05:04<1:03:25,  2.27s/it]
 86%|████████▌ | 10321/12000 [6:05:06<57:03,  2.04s/it]  
                                                       

 86%|████████▌ | 10321/12000 [6:05:06<57:03,  2.04s/it]
 86%|████████▌ | 10322/12000 [6:05:07<52:58,  1.89s/it]
                                                       

 86%|████████▌ | 10322/12000 [6:05:07<52:58,  1.89s/it]
 86%|████████▌ | 10323/12000 [6:05:09<52:32,  1.88s/it]
                                                       

 86%|████████▌ | 10323/12000 [6:05:09<52:32,  1.88s/it]
 86%|████████▌ | 10324/12000 [6:05:11<52:24,  1.88s/it]
                                                       

 86%|████████▌ | 10324/12000 [6:05:11<52:24,  1.88s/it]
 86%|████████▌ | 10325/12000 [6:05:15<1:12:20,  2.59s/it]
                                                         

 86%|████████▌ | 10325/12000 [6:05:15<1:12:20,  2.59s/it]
 86%|████████▌ | 10326/12000 [6:05:17<1:03:54,  2.29s/it]
                                                         

 86%|████████▌ | 10326/12000 [6:05:17<1:03:54,  2.29s/it]
 86%|████████▌ | 10327/12000 [6:05:19<1:00:09,  2.16s/it]
                                                         

 86%|████████▌ | 10327/12000 [6:05:19<1:00:09,  2.16s/it]
 86%|████████▌ | 10328/12000 [6:05:21<58:56,  2.12s/it]  
                                                       

 86%|████████▌ | 10328/12000 [6:05:21<58:56,  2.12s/it]
 86%|████████▌ | 10329/12000 [6:05:24<1:08:45,  2.47s/it]
                                                         

 86%|████████▌ | 10329/12000 [6:05:24<1:08:45,  2.47s/it]
 86%|████████▌ | 10330/12000 [6:05:26<1:03:15,  2.27s/it]
                                                         

 86%|████████▌ | 10330/12000 [6:05:26<1:03:15,  2.27s/it]
 86%|████████▌ | 10331/12000 [6:05:27<56:42,  2.04s/it]  
                                                       

 86%|████████▌ | 10331/12000 [6:05:27<56:42,  2.04s/it]
 86%|████████▌ | 10332/12000 [6:05:29<53:47,  1.94s/it]
                                                       

 86%|████████▌ | 10332/12000 [6:05:29<53:47,  1.94s/it]
 86%|████████▌ | 10333/12000 [6:05:31<57:14,  2.06s/it]
                                                       

 86%|████████▌ | 10333/12000 [6:05:31<57:14,  2.06s/it]
 86%|████████▌ | 10334/12000 [6:05:34<1:05:25,  2.36s/it]
                                                         

 86%|████████▌ | 10334/12000 [6:05:34<1:05:25,  2.36s/it]
 86%|████████▌ | 10335/12000 [6:05:36<1:01:32,  2.22s/it]
                                                         

 86%|████████▌ | 10335/12000 [6:05:36<1:01:32,  2.22s/it]
 86%|████████▌ | 10336/12000 [6:05:42<1:28:25,  3.19s/it]
                                                         

 86%|████████▌ | 10336/12000 [6:05:42<1:28:25,  3.19s/it]
 86%|████████▌ | 10337/12000 [6:05:44<1:21:50,  2.95s/it]
                                                         

 86%|████████▌ | 10337/12000 [6:05:44<1:21:50,  2.95s/it]
 86%|████████▌ | 10338/12000 [6:05:46<1:13:08,  2.64s/it]
                                                         

 86%|████████▌ | 10338/12000 [6:05:46<1:13:08,  2.64s/it]
 86%|████████▌ | 10339/12000 [6:05:48<1:04:10,  2.32s/it]
                                                         

 86%|████████▌ | 10339/12000 [6:05:48<1:04:10,  2.32s/it]
 86%|████████▌ | 10340/12000 [6:05:49<59:15,  2.14s/it]  
                                                       

 86%|████████▌ | 10340/12000 [6:05:49<59:15,  2.14s/it]
 86%|████████▌ | 10341/12000 [6:05:51<54:43,  1.98s/it]
                                                       

 86%|████████▌ | 10341/12000 [6:05:51<54:43,  1.98s/it]
 86%|████████▌ | 10342/12000 [6:05:53<55:22,  2.00s/it]
                                                       

 86%|████████▌ | 10342/12000 [6:05:53<55:22,  2.00s/it]
 86%|████████▌ | 10343/12000 [6:05:54<51:30,  1.87s/it]
                                                       

 86%|████████▌ | 10343/12000 [6:05:54<51:30,  1.87s/it]
 86%|████████▌ | 10344/12000 [6:05:56<49:37,  1.80s/it]
                                                       
{'loss': '0.7962', 'grad_norm': '0.405', 'learning_rate': '1.082e-05', 'entropy': '0.7832', 'num_tokens': '5.545e+07', 'mean_token_accuracy': '0.7946', 'epoch': '0.09113'}
{'loss': '0.6565', 'grad_norm': '0.3966', 'learning_rate': '1.081e-05', 'entropy': '0.7339', 'num_tokens': '5.545e+07', 'mean_token_accuracy': '0.8018', 'epoch': '0.09114'}
{'loss': '0.7699', 'grad_norm': '0.3253', 'learning_rate': '1.08e-05', 'entropy': '0.7934', 'num_tokens': '5.546e+07', 'mean_token_accuracy': '0.8002', 'epoch': '0.09114'}
{'loss': '0.7482', 'grad_norm': '0.3822', 'learning_rate': '1.079e-05', 'entropy': '0.7802', 'num_tokens': '5.546e+07', 'mean_token_accuracy': '0.8013', 'epoch': '0.09115'}
{'loss': '0.7962', 'grad_norm': '0.403', 'learning_rate': '1.077e-05', 'entropy': '0.8383', 'num_tokens': '5.547e+07', 'mean_token_accuracy': '0.7796', 'epoch': '0.09116'}
{'loss': '0.9153', 'grad_norm': '0.451', 'learning_rate': '1.076e-05', 'entropy': '0.8609', 'num_tokens': '5.547e+07', 'mean_token_accuracy': '0.7872', 'epoch': '0.09117'}
{'loss': '0.79', 'grad_norm': '0.526', 'learning_rate': '1.075e-05', 'entropy': '0.8954', 'num_tokens': '5.548e+07', 'mean_token_accuracy': '0.7763', 'epoch': '0.09118'}
{'loss': '0.7742', 'grad_norm': '0.3751', 'learning_rate': '1.074e-05', 'entropy': '0.8158', 'num_tokens': '5.548e+07', 'mean_token_accuracy': '0.7874', 'epoch': '0.09119'}
{'loss': '0.5331', 'grad_norm': '0.407', 'learning_rate': '1.072e-05', 'entropy': '0.5391', 'num_tokens': '5.549e+07', 'mean_token_accuracy': '0.8557', 'epoch': '0.0912'}
{'loss': '1.098', 'grad_norm': '0.4777', 'learning_rate': '1.071e-05', 'entropy': '1.06', 'num_tokens': '5.549e+07', 'mean_token_accuracy': '0.7326', 'epoch': '0.09121'}
{'loss': '1.133', 'grad_norm': '0.5981', 'learning_rate': '1.07e-05', 'entropy': '1.052', 'num_tokens': '5.55e+07', 'mean_token_accuracy': '0.7411', 'epoch': '0.09122'}
{'loss': '0.7737', 'grad_norm': '0.3733', 'learning_rate': '1.069e-05', 'entropy': '0.8512', 'num_tokens': '5.55e+07', 'mean_token_accuracy': '0.7894', 'epoch': '0.09122'}
{'loss': '0.8926', 'grad_norm': '0.4027', 'learning_rate': '1.068e-05', 'entropy': '0.9952', 'num_tokens': '5.551e+07', 'mean_token_accuracy': '0.7617', 'epoch': '0.09123'}
{'loss': '0.6712', 'grad_norm': '0.5036', 'learning_rate': '1.066e-05', 'entropy': '0.6806', 'num_tokens': '5.551e+07', 'mean_token_accuracy': '0.8184', 'epoch': '0.09124'}
{'loss': '0.95', 'grad_norm': '0.4512', 'learning_rate': '1.065e-05', 'entropy': '0.8766', 'num_tokens': '5.552e+07', 'mean_token_accuracy': '0.7836', 'epoch': '0.09125'}
{'loss': '0.9734', 'grad_norm': '0.4028', 'learning_rate': '1.064e-05', 'entropy': '0.9367', 'num_tokens': '5.552e+07', 'mean_token_accuracy': '0.75', 'epoch': '0.09126'}
{'loss': '0.707', 'grad_norm': '0.411', 'learning_rate': '1.063e-05', 'entropy': '0.7483', 'num_tokens': '5.553e+07', 'mean_token_accuracy': '0.8156', 'epoch': '0.09127'}
{'loss': '0.687', 'grad_norm': '0.4002', 'learning_rate': '1.061e-05', 'entropy': '0.6746', 'num_tokens': '5.553e+07', 'mean_token_accuracy': '0.8263', 'epoch': '0.09128'}
{'loss': '0.8834', 'grad_norm': '0.4851', 'learning_rate': '1.06e-05', 'entropy': '0.8468', 'num_tokens': '5.554e+07', 'mean_token_accuracy': '0.7751', 'epoch': '0.09129'}
{'loss': '0.9814', 'grad_norm': '0.4188', 'learning_rate': '1.059e-05', 'entropy': '0.9899', 'num_tokens': '5.554e+07', 'mean_token_accuracy': '0.7544', 'epoch': '0.09129'}
{'loss': '0.6719', 'grad_norm': '0.4176', 'learning_rate': '1.058e-05', 'entropy': '0.7031', 'num_tokens': '5.555e+07', 'mean_token_accuracy': '0.8075', 'epoch': '0.0913'}
{'loss': '0.6508', 'grad_norm': '0.3398', 'learning_rate': '1.056e-05', 'entropy': '0.626', 'num_tokens': '5.555e+07', 'mean_token_accuracy': '0.8349', 'epoch': '0.09131'}
{'loss': '0.4075', 'grad_norm': '0.3878', 'learning_rate': '1.055e-05', 'entropy': '0.4342', 'num_tokens': '5.556e+07', 'mean_token_accuracy': '0.8835', 'epoch': '0.09132'}
{'loss': '0.7063', 'grad_norm': '0.3563', 'learning_rate': '1.054e-05', 'entropy': '0.6953', 'num_tokens': '5.557e+07', 'mean_token_accuracy': '0.8164', 'epoch': '0.09133'}
{'loss': '0.7628', 'grad_norm': '0.4118', 'learning_rate': '1.053e-05', 'entropy': '0.7542', 'num_tokens': '5.557e+07', 'mean_token_accuracy': '0.8188', 'epoch': '0.09134'}
{'loss': '0.9651', 'grad_norm': '0.4469', 'learning_rate': '1.051e-05', 'entropy': '0.8912', 'num_tokens': '5.558e+07', 'mean_token_accuracy': '0.7688', 'epoch': '0.09135'}
{'loss': '1.041', 'grad_norm': '0.3595', 'learning_rate': '1.05e-05', 'entropy': '1.042', 'num_tokens': '5.558e+07', 'mean_token_accuracy': '0.7442', 'epoch': '0.09136'}
{'loss': '0.965', 'grad_norm': '0.4341', 'learning_rate': '1.049e-05', 'entropy': '0.9717', 'num_tokens': '5.559e+07', 'mean_token_accuracy': '0.7761', 'epoch': '0.09137'}
{'loss': '1.125', 'grad_norm': '0.489', 'learning_rate': '1.048e-05', 'entropy': '1.123', 'num_tokens': '5.559e+07', 'mean_token_accuracy': '0.7315', 'epoch': '0.09137'}
{'loss': '0.668', 'grad_norm': '0.3823', 'learning_rate': '1.047e-05', 'entropy': '0.7181', 'num_tokens': '5.56e+07', 'mean_token_accuracy': '0.8155', 'epoch': '0.09138'}
{'loss': '0.6662', 'grad_norm': '0.45', 'learning_rate': '1.045e-05', 'entropy': '0.6441', 'num_tokens': '5.56e+07', 'mean_token_accuracy': '0.8141', 'epoch': '0.09139'}
{'loss': '0.9195', 'grad_norm': '0.4653', 'learning_rate': '1.044e-05', 'entropy': '0.8582', 'num_tokens': '5.561e+07', 'mean_token_accuracy': '0.773', 'epoch': '0.0914'}
{'loss': '0.7857', 'grad_norm': '0.3619', 'learning_rate': '1.043e-05', 'entropy': '0.7424', 'num_tokens': '5.561e+07', 'mean_token_accuracy': '0.8096', 'epoch': '0.09141'}
{'loss': '0.8646', 'grad_norm': '0.5529', 'learning_rate': '1.042e-05', 'entropy': '0.8486', 'num_tokens': '5.562e+07', 'mean_token_accuracy': '0.7797', 'epoch': '0.09142'}
{'loss': '0.7343', 'grad_norm': '0.3641', 'learning_rate': '1.04e-05', 'entropy': '0.8054', 'num_tokens': '5.562e+07', 'mean_token_accuracy': '0.7961', 'epoch': '0.09143'}
{'loss': '1.258', 'grad_norm': '0.4954', 'learning_rate': '1.039e-05', 'entropy': '1.124', 'num_tokens': '5.563e+07', 'mean_token_accuracy': '0.724', 'epoch': '0.09144'}
{'loss': '0.731', 'grad_norm': '0.2836', 'learning_rate': '1.038e-05', 'entropy': '0.7053', 'num_tokens': '5.563e+07', 'mean_token_accuracy': '0.8176', 'epoch': '0.09145'}
{'loss': '0.6629', 'grad_norm': '0.3483', 'learning_rate': '1.037e-05', 'entropy': '0.7162', 'num_tokens': '5.564e+07', 'mean_token_accuracy': '0.8167', 'epoch': '0.09145'}
{'loss': '0.7078', 'grad_norm': '0.3544', 'learning_rate': '1.036e-05', 'entropy': '0.704', 'num_tokens': '5.565e+07', 'mean_token_accuracy': '0.825', 'epoch': '0.09146'}
{'loss': '0.9344', 'grad_norm': '0.3131', 'learning_rate': '1.034e-05', 'entropy': '0.7775', 'num_tokens': '5.566e+07', 'mean_token_accuracy': '0.8018', 'epoch': '0.09147'}
{'loss': '0.8898', 'grad_norm': '0.5136', 'learning_rate': '1.033e-05', 'entropy': '0.8324', 'num_tokens': '5.566e+07', 'mean_token_accuracy': '0.7954', 'epoch': '0.09148'}
{'loss': '0.7039', 'grad_norm': '0.3601', 'learning_rate': '1.032e-05', 'entropy': '0.7556', 'num_tokens': '5.567e+07', 'mean_token_accuracy': '0.805', 'epoch': '0.09149'}
{'loss': '0.9539', 'grad_norm': '0.4339', 'learning_rate': '1.031e-05', 'entropy': '1.004', 'num_tokens': '5.567e+07', 'mean_token_accuracy': '0.767', 'epoch': '0.0915'}
{'loss': '0.735', 'grad_norm': '0.3936', 'learning_rate': '1.029e-05', 'entropy': '0.8357', 'num_tokens': '5.568e+07', 'mean_token_accuracy': '0.802', 'epoch': '0.09151'}
{'loss': '0.5275', 'grad_norm': '0.2944', 'learning_rate': '1.028e-05', 'entropy': '0.5586', 'num_tokens': '5.568e+07', 'mean_token_accuracy': '0.859', 'epoch': '0.09152'}
{'loss': '1.027', 'grad_norm': '0.3826', 'learning_rate': '1.027e-05', 'entropy': '0.9365', 'num_tokens': '5.569e+07', 'mean_token_accuracy': '0.764', 'epoch': '0.09152'}
{'loss': '0.6026', 'grad_norm': '0.332', 'learning_rate': '1.026e-05', 'entropy': '0.7382', 'num_tokens': '5.569e+07', 'mean_token_accuracy': '0.8279', 'epoch': '0.09153'}

 86%|████████▌ | 10344/12000 [6:05:56<49:37,  1.80s/it]
 86%|████████▌ | 10345/12000 [6:05:58<46:54,  1.70s/it]
                                                       

 86%|████████▌ | 10345/12000 [6:05:58<46:54,  1.70s/it]
 86%|████████▌ | 10346/12000 [6:05:59<46:06,  1.67s/it]
                                                       

 86%|████████▌ | 10346/12000 [6:05:59<46:06,  1.67s/it]
 86%|████████▌ | 10347/12000 [6:06:03<1:05:29,  2.38s/it]
                                                         

 86%|████████▌ | 10347/12000 [6:06:03<1:05:29,  2.38s/it]
 86%|████████▌ | 10348/12000 [6:06:06<1:06:06,  2.40s/it]
                                                         

 86%|████████▌ | 10348/12000 [6:06:06<1:06:06,  2.40s/it]
 86%|████████▌ | 10349/12000 [6:06:08<1:06:47,  2.43s/it]
                                                         

 86%|████████▌ | 10349/12000 [6:06:08<1:06:47,  2.43s/it]
 86%|████████▋ | 10350/12000 [6:06:10<1:01:20,  2.23s/it]
                                                         

 86%|████████▋ | 10350/12000 [6:06:10<1:01:20,  2.23s/it]
 86%|████████▋ | 10351/12000 [6:06:13<1:09:33,  2.53s/it]
                                                         

 86%|████████▋ | 10351/12000 [6:06:13<1:09:33,  2.53s/it]
 86%|████████▋ | 10352/12000 [6:06:15<1:03:41,  2.32s/it]
                                                         

 86%|████████▋ | 10352/12000 [6:06:15<1:03:41,  2.32s/it]
 86%|████████▋ | 10353/12000 [6:06:18<1:05:50,  2.40s/it]
                                                         

 86%|████████▋ | 10353/12000 [6:06:18<1:05:50,  2.40s/it]
 86%|████████▋ | 10354/12000 [6:06:19<58:20,  2.13s/it]  
                                                       

 86%|████████▋ | 10354/12000 [6:06:19<58:20,  2.13s/it]
 86%|████████▋ | 10355/12000 [6:06:21<57:02,  2.08s/it]
                                                       

 86%|████████▋ | 10355/12000 [6:06:21<57:02,  2.08s/it]
 86%|████████▋ | 10356/12000 [6:06:23<54:25,  1.99s/it]
                                                       

 86%|████████▋ | 10356/12000 [6:06:23<54:25,  1.99s/it]
 86%|████████▋ | 10357/12000 [6:06:25<52:26,  1.92s/it]
                                                       

 86%|████████▋ | 10357/12000 [6:06:25<52:26,  1.92s/it]
 86%|████████▋ | 10358/12000 [6:06:27<53:40,  1.96s/it]
                                                       

 86%|████████▋ | 10358/12000 [6:06:27<53:40,  1.96s/it]
 86%|████████▋ | 10359/12000 [6:06:29<53:00,  1.94s/it]
                                                       

 86%|████████▋ | 10359/12000 [6:06:29<53:00,  1.94s/it]
 86%|████████▋ | 10360/12000 [6:06:31<55:10,  2.02s/it]
                                                       

 86%|████████▋ | 10360/12000 [6:06:31<55:10,  2.02s/it]
 86%|████████▋ | 10361/12000 [6:06:33<53:40,  1.96s/it]
                                                       

 86%|████████▋ | 10361/12000 [6:06:33<53:40,  1.96s/it]
 86%|████████▋ | 10362/12000 [6:06:34<49:51,  1.83s/it]
                                                       

 86%|████████▋ | 10362/12000 [6:06:34<49:51,  1.83s/it]
 86%|████████▋ | 10363/12000 [6:06:38<1:10:57,  2.60s/it]
                                                         

 86%|████████▋ | 10363/12000 [6:06:38<1:10:57,  2.60s/it]
 86%|████████▋ | 10364/12000 [6:06:40<1:03:15,  2.32s/it]
                                                         

 86%|████████▋ | 10364/12000 [6:06:40<1:03:15,  2.32s/it]
 86%|████████▋ | 10365/12000 [6:06:42<1:02:34,  2.30s/it]
                                                         

 86%|████████▋ | 10365/12000 [6:06:42<1:02:34,  2.30s/it]
 86%|████████▋ | 10366/12000 [6:06:44<59:11,  2.17s/it]  
                                                       

 86%|████████▋ | 10366/12000 [6:06:44<59:11,  2.17s/it]
 86%|████████▋ | 10367/12000 [6:06:46<55:31,  2.04s/it]
                                                       

 86%|████████▋ | 10367/12000 [6:06:46<55:31,  2.04s/it]
 86%|████████▋ | 10368/12000 [6:06:48<58:49,  2.16s/it]
                                                       

 86%|████████▋ | 10368/12000 [6:06:48<58:49,  2.16s/it]
 86%|████████▋ | 10369/12000 [6:06:53<1:14:54,  2.76s/it]
                                                         

 86%|████████▋ | 10369/12000 [6:06:53<1:14:54,  2.76s/it]
 86%|████████▋ | 10370/12000 [6:06:54<1:07:17,  2.48s/it]
                                                         

 86%|████████▋ | 10370/12000 [6:06:54<1:07:17,  2.48s/it]
 86%|████████▋ | 10371/12000 [6:06:56<1:01:38,  2.27s/it]
                                                         

 86%|████████▋ | 10371/12000 [6:06:56<1:01:38,  2.27s/it]
 86%|████████▋ | 10372/12000 [6:06:58<56:14,  2.07s/it]  
                                                       

 86%|████████▋ | 10372/12000 [6:06:58<56:14,  2.07s/it]
 86%|████████▋ | 10373/12000 [6:06:59<52:41,  1.94s/it]
                                                       

 86%|████████▋ | 10373/12000 [6:06:59<52:41,  1.94s/it]
 86%|████████▋ | 10374/12000 [6:07:01<49:41,  1.83s/it]
                                                       

 86%|████████▋ | 10374/12000 [6:07:01<49:41,  1.83s/it]
 86%|████████▋ | 10375/12000 [6:07:03<51:57,  1.92s/it]
                                                       

 86%|████████▋ | 10375/12000 [6:07:03<51:57,  1.92s/it]
 86%|████████▋ | 10376/12000 [6:07:05<49:30,  1.83s/it]
                                                       

 86%|████████▋ | 10376/12000 [6:07:05<49:30,  1.83s/it]
 86%|████████▋ | 10377/12000 [6:07:07<50:48,  1.88s/it]
                                                       

 86%|████████▋ | 10377/12000 [6:07:07<50:48,  1.88s/it]
 86%|████████▋ | 10378/12000 [6:07:08<48:26,  1.79s/it]
                                                       

 86%|████████▋ | 10378/12000 [6:07:08<48:26,  1.79s/it]
 86%|████████▋ | 10379/12000 [6:07:11<57:21,  2.12s/it]
                                                       

 86%|████████▋ | 10379/12000 [6:07:11<57:21,  2.12s/it]
 86%|████████▋ | 10380/12000 [6:07:13<53:56,  2.00s/it]
                                                       

 86%|████████▋ | 10380/12000 [6:07:13<53:56,  2.00s/it]
 87%|████████▋ | 10381/12000 [6:07:17<1:08:26,  2.54s/it]
                                                         

 87%|████████▋ | 10381/12000 [6:07:17<1:08:26,  2.54s/it]
 87%|████████▋ | 10382/12000 [6:07:19<1:04:00,  2.37s/it]
                                                         

 87%|████████▋ | 10382/12000 [6:07:19<1:04:00,  2.37s/it]
 87%|████████▋ | 10383/12000 [6:07:21<1:06:39,  2.47s/it]
                                                         

 87%|████████▋ | 10383/12000 [6:07:21<1:06:39,  2.47s/it]
 87%|████████▋ | 10384/12000 [6:07:23<1:01:42,  2.29s/it]
                                                         

 87%|████████▋ | 10384/12000 [6:07:23<1:01:42,  2.29s/it]
 87%|████████▋ | 10385/12000 [6:07:26<1:01:46,  2.29s/it]
                                                         

 87%|████████▋ | 10385/12000 [6:07:26<1:01:46,  2.29s/it]
 87%|████████▋ | 10386/12000 [6:07:27<55:30,  2.06s/it]  
                                                       

 87%|████████▋ | 10386/12000 [6:07:27<55:30,  2.06s/it]
 87%|████████▋ | 10387/12000 [6:07:29<55:27,  2.06s/it]
                                                       

 87%|████████▋ | 10387/12000 [6:07:29<55:27,  2.06s/it]
 87%|████████▋ | 10388/12000 [6:07:31<52:07,  1.94s/it]
                                                       

 87%|████████▋ | 10388/12000 [6:07:31<52:07,  1.94s/it]
 87%|████████▋ | 10389/12000 [6:07:33<50:17,  1.87s/it]
                                                       

 87%|████████▋ | 10389/12000 [6:07:33<50:17,  1.87s/it]
 87%|████████▋ | 10390/12000 [6:07:35<51:12,  1.91s/it]
                                                       

 87%|████████▋ | 10390/12000 [6:07:35<51:12,  1.91s/it]
 87%|████████▋ | 10391/12000 [6:07:38<1:03:58,  2.39s/it]
                                                         
{'loss': '0.9512', 'grad_norm': '0.44', 'learning_rate': '1.025e-05', 'entropy': '0.9515', 'num_tokens': '5.57e+07', 'mean_token_accuracy': '0.7694', 'epoch': '0.09154'}
{'loss': '0.8538', 'grad_norm': '0.4092', 'learning_rate': '1.023e-05', 'entropy': '0.9486', 'num_tokens': '5.57e+07', 'mean_token_accuracy': '0.7791', 'epoch': '0.09155'}
{'loss': '0.952', 'grad_norm': '0.3903', 'learning_rate': '1.022e-05', 'entropy': '0.9083', 'num_tokens': '5.571e+07', 'mean_token_accuracy': '0.7614', 'epoch': '0.09156'}
{'loss': '1.019', 'grad_norm': '0.4632', 'learning_rate': '1.021e-05', 'entropy': '1.111', 'num_tokens': '5.571e+07', 'mean_token_accuracy': '0.7354', 'epoch': '0.09157'}
{'loss': '1.017', 'grad_norm': '0.4738', 'learning_rate': '1.02e-05', 'entropy': '0.9212', 'num_tokens': '5.572e+07', 'mean_token_accuracy': '0.7721', 'epoch': '0.09158'}
{'loss': '1.307', 'grad_norm': '0.4444', 'learning_rate': '1.019e-05', 'entropy': '1.296', 'num_tokens': '5.572e+07', 'mean_token_accuracy': '0.6977', 'epoch': '0.09159'}
{'loss': '1.232', 'grad_norm': '0.419', 'learning_rate': '1.017e-05', 'entropy': '1.245', 'num_tokens': '5.573e+07', 'mean_token_accuracy': '0.7038', 'epoch': '0.0916'}
{'loss': '0.9134', 'grad_norm': '0.45', 'learning_rate': '1.016e-05', 'entropy': '0.9149', 'num_tokens': '5.573e+07', 'mean_token_accuracy': '0.7724', 'epoch': '0.0916'}
{'loss': '1.009', 'grad_norm': '0.4906', 'learning_rate': '1.015e-05', 'entropy': '1.093', 'num_tokens': '5.574e+07', 'mean_token_accuracy': '0.7234', 'epoch': '0.09161'}
{'loss': '0.7236', 'grad_norm': '0.3529', 'learning_rate': '1.014e-05', 'entropy': '0.7127', 'num_tokens': '5.574e+07', 'mean_token_accuracy': '0.8166', 'epoch': '0.09162'}
{'loss': '0.7553', 'grad_norm': '0.3725', 'learning_rate': '1.012e-05', 'entropy': '0.7847', 'num_tokens': '5.575e+07', 'mean_token_accuracy': '0.8085', 'epoch': '0.09163'}
{'loss': '0.6076', 'grad_norm': '0.3604', 'learning_rate': '1.011e-05', 'entropy': '0.5914', 'num_tokens': '5.575e+07', 'mean_token_accuracy': '0.8408', 'epoch': '0.09164'}
{'loss': '0.8849', 'grad_norm': '0.4171', 'learning_rate': '1.01e-05', 'entropy': '0.9213', 'num_tokens': '5.576e+07', 'mean_token_accuracy': '0.7645', 'epoch': '0.09165'}
{'loss': '0.507', 'grad_norm': '0.3773', 'learning_rate': '1.009e-05', 'entropy': '0.5058', 'num_tokens': '5.576e+07', 'mean_token_accuracy': '0.8606', 'epoch': '0.09166'}
{'loss': '0.9453', 'grad_norm': '0.3715', 'learning_rate': '1.008e-05', 'entropy': '0.8962', 'num_tokens': '5.577e+07', 'mean_token_accuracy': '0.7746', 'epoch': '0.09167'}
{'loss': '0.8051', 'grad_norm': '0.3933', 'learning_rate': '1.006e-05', 'entropy': '0.7755', 'num_tokens': '5.577e+07', 'mean_token_accuracy': '0.8126', 'epoch': '0.09168'}
{'loss': '1.13', 'grad_norm': '0.4263', 'learning_rate': '1.005e-05', 'entropy': '0.8877', 'num_tokens': '5.578e+07', 'mean_token_accuracy': '0.7856', 'epoch': '0.09168'}
{'loss': '1.014', 'grad_norm': '0.4446', 'learning_rate': '1.004e-05', 'entropy': '0.9384', 'num_tokens': '5.579e+07', 'mean_token_accuracy': '0.7743', 'epoch': '0.09169'}
{'loss': '0.6447', 'grad_norm': '0.3931', 'learning_rate': '1.003e-05', 'entropy': '0.665', 'num_tokens': '5.579e+07', 'mean_token_accuracy': '0.8218', 'epoch': '0.0917'}
{'loss': '0.7841', 'grad_norm': '0.3981', 'learning_rate': '1.002e-05', 'entropy': '0.7728', 'num_tokens': '5.58e+07', 'mean_token_accuracy': '0.8172', 'epoch': '0.09171'}
{'loss': '0.5541', 'grad_norm': '0.3719', 'learning_rate': '1e-05', 'entropy': '0.5477', 'num_tokens': '5.58e+07', 'mean_token_accuracy': '0.8492', 'epoch': '0.09172'}
{'loss': '0.785', 'grad_norm': '0.4198', 'learning_rate': '9.992e-06', 'entropy': '0.6812', 'num_tokens': '5.581e+07', 'mean_token_accuracy': '0.8328', 'epoch': '0.09173'}
{'loss': '1.008', 'grad_norm': '0.4313', 'learning_rate': '9.98e-06', 'entropy': '0.9666', 'num_tokens': '5.581e+07', 'mean_token_accuracy': '0.7615', 'epoch': '0.09174'}
{'loss': '0.839', 'grad_norm': '0.3852', 'learning_rate': '9.968e-06', 'entropy': '0.8739', 'num_tokens': '5.582e+07', 'mean_token_accuracy': '0.7825', 'epoch': '0.09175'}
{'loss': '0.6321', 'grad_norm': '0.3355', 'learning_rate': '9.956e-06', 'entropy': '0.7634', 'num_tokens': '5.582e+07', 'mean_token_accuracy': '0.8055', 'epoch': '0.09176'}
{'loss': '0.7056', 'grad_norm': '0.3631', 'learning_rate': '9.944e-06', 'entropy': '0.674', 'num_tokens': '5.583e+07', 'mean_token_accuracy': '0.8246', 'epoch': '0.09176'}
{'loss': '0.8209', 'grad_norm': '0.5144', 'learning_rate': '9.932e-06', 'entropy': '0.8883', 'num_tokens': '5.583e+07', 'mean_token_accuracy': '0.7663', 'epoch': '0.09177'}
{'loss': '0.7534', 'grad_norm': '0.3602', 'learning_rate': '9.92e-06', 'entropy': '0.7688', 'num_tokens': '5.584e+07', 'mean_token_accuracy': '0.8004', 'epoch': '0.09178'}
{'loss': '0.8358', 'grad_norm': '0.4047', 'learning_rate': '9.908e-06', 'entropy': '0.9568', 'num_tokens': '5.584e+07', 'mean_token_accuracy': '0.7688', 'epoch': '0.09179'}
{'loss': '0.6452', 'grad_norm': '0.4172', 'learning_rate': '9.896e-06', 'entropy': '0.7157', 'num_tokens': '5.585e+07', 'mean_token_accuracy': '0.8189', 'epoch': '0.0918'}
{'loss': '0.9058', 'grad_norm': '0.4426', 'learning_rate': '9.884e-06', 'entropy': '0.8896', 'num_tokens': '5.585e+07', 'mean_token_accuracy': '0.773', 'epoch': '0.09181'}
{'loss': '0.6972', 'grad_norm': '0.3661', 'learning_rate': '9.872e-06', 'entropy': '0.6965', 'num_tokens': '5.586e+07', 'mean_token_accuracy': '0.823', 'epoch': '0.09182'}
{'loss': '0.8981', 'grad_norm': '0.4347', 'learning_rate': '9.86e-06', 'entropy': '0.8947', 'num_tokens': '5.586e+07', 'mean_token_accuracy': '0.7716', 'epoch': '0.09183'}
{'loss': '0.7445', 'grad_norm': '0.3841', 'learning_rate': '9.849e-06', 'entropy': '0.7843', 'num_tokens': '5.587e+07', 'mean_token_accuracy': '0.8016', 'epoch': '0.09183'}
{'loss': '0.7166', 'grad_norm': '0.3527', 'learning_rate': '9.837e-06', 'entropy': '0.7301', 'num_tokens': '5.587e+07', 'mean_token_accuracy': '0.8222', 'epoch': '0.09184'}
{'loss': '0.6723', 'grad_norm': '0.3613', 'learning_rate': '9.825e-06', 'entropy': '0.7148', 'num_tokens': '5.588e+07', 'mean_token_accuracy': '0.8128', 'epoch': '0.09185'}
{'loss': '0.548', 'grad_norm': '0.3255', 'learning_rate': '9.813e-06', 'entropy': '0.6165', 'num_tokens': '5.589e+07', 'mean_token_accuracy': '0.8415', 'epoch': '0.09186'}
{'loss': '1.686', 'grad_norm': '0.319', 'learning_rate': '9.801e-06', 'entropy': '1.221', 'num_tokens': '5.589e+07', 'mean_token_accuracy': '0.685', 'epoch': '0.09187'}
{'loss': '1.134', 'grad_norm': '0.5179', 'learning_rate': '9.789e-06', 'entropy': '0.9568', 'num_tokens': '5.59e+07', 'mean_token_accuracy': '0.7577', 'epoch': '0.09188'}
{'loss': '0.7491', 'grad_norm': '0.3486', 'learning_rate': '9.777e-06', 'entropy': '0.7399', 'num_tokens': '5.59e+07', 'mean_token_accuracy': '0.8079', 'epoch': '0.09189'}
{'loss': '0.606', 'grad_norm': '0.3752', 'learning_rate': '9.765e-06', 'entropy': '0.5912', 'num_tokens': '5.591e+07', 'mean_token_accuracy': '0.8283', 'epoch': '0.0919'}
{'loss': '1.386', 'grad_norm': '0.4308', 'learning_rate': '9.753e-06', 'entropy': '1.059', 'num_tokens': '5.592e+07', 'mean_token_accuracy': '0.7347', 'epoch': '0.09191'}
{'loss': '0.641', 'grad_norm': '0.3901', 'learning_rate': '9.741e-06', 'entropy': '0.6735', 'num_tokens': '5.592e+07', 'mean_token_accuracy': '0.8246', 'epoch': '0.09191'}
{'loss': '0.8552', 'grad_norm': '0.4597', 'learning_rate': '9.73e-06', 'entropy': '0.8914', 'num_tokens': '5.593e+07', 'mean_token_accuracy': '0.7701', 'epoch': '0.09192'}
{'loss': '0.5435', 'grad_norm': '0.3673', 'learning_rate': '9.718e-06', 'entropy': '0.5533', 'num_tokens': '5.593e+07', 'mean_token_accuracy': '0.8511', 'epoch': '0.09193'}
{'loss': '0.8103', 'grad_norm': '0.4436', 'learning_rate': '9.706e-06', 'entropy': '0.7543', 'num_tokens': '5.594e+07', 'mean_token_accuracy': '0.8214', 'epoch': '0.09194'}
{'loss': '0.7142', 'grad_norm': '0.3474', 'learning_rate': '9.694e-06', 'entropy': '0.6988', 'num_tokens': '5.594e+07', 'mean_token_accuracy': '0.8132', 'epoch': '0.09195'}

 87%|████████▋ | 10391/12000 [6:07:38<1:03:58,  2.39s/it]
 87%|████████▋ | 10392/12000 [6:07:41<1:04:26,  2.40s/it]
                                                         

 87%|████████▋ | 10392/12000 [6:07:41<1:04:26,  2.40s/it]
 87%|████████▋ | 10393/12000 [6:07:42<59:17,  2.21s/it]  
                                                       

 87%|████████▋ | 10393/12000 [6:07:42<59:17,  2.21s/it]
 87%|████████▋ | 10394/12000 [6:07:45<1:01:55,  2.31s/it]
                                                         

 87%|████████▋ | 10394/12000 [6:07:45<1:01:55,  2.31s/it]
 87%|████████▋ | 10395/12000 [6:07:46<55:51,  2.09s/it]  
                                                       

 87%|████████▋ | 10395/12000 [6:07:46<55:51,  2.09s/it]
 87%|████████▋ | 10396/12000 [6:07:48<52:11,  1.95s/it]
                                                       

 87%|████████▋ | 10396/12000 [6:07:48<52:11,  1.95s/it]
 87%|████████▋ | 10397/12000 [6:07:50<48:51,  1.83s/it]
                                                       

 87%|████████▋ | 10397/12000 [6:07:50<48:51,  1.83s/it]
 87%|████████▋ | 10398/12000 [6:07:53<57:40,  2.16s/it]
                                                       

 87%|████████▋ | 10398/12000 [6:07:53<57:40,  2.16s/it]
 87%|████████▋ | 10399/12000 [6:07:54<52:02,  1.95s/it]
                                                       

 87%|████████▋ | 10399/12000 [6:07:54<52:02,  1.95s/it]
 87%|████████▋ | 10400/12000 [6:07:56<52:01,  1.95s/it]
                                                       

 87%|████████▋ | 10400/12000 [6:07:56<52:01,  1.95s/it]
 87%|████████▋ | 10401/12000 [6:07:59<59:37,  2.24s/it]
                                                       

 87%|████████▋ | 10401/12000 [6:07:59<59:37,  2.24s/it]
 87%|████████▋ | 10402/12000 [6:08:00<54:18,  2.04s/it]
                                                       

 87%|████████▋ | 10402/12000 [6:08:00<54:18,  2.04s/it]
 87%|████████▋ | 10403/12000 [6:08:03<56:56,  2.14s/it]
                                                       

 87%|████████▋ | 10403/12000 [6:08:03<56:56,  2.14s/it]
 87%|████████▋ | 10404/12000 [6:08:04<52:58,  1.99s/it]
                                                       

 87%|████████▋ | 10404/12000 [6:08:04<52:58,  1.99s/it]
 87%|████████▋ | 10405/12000 [6:08:06<49:21,  1.86s/it]
                                                       

 87%|████████▋ | 10405/12000 [6:08:06<49:21,  1.86s/it]
 87%|████████▋ | 10406/12000 [6:08:08<47:47,  1.80s/it]
                                                       

 87%|████████▋ | 10406/12000 [6:08:08<47:47,  1.80s/it]
 87%|████████▋ | 10407/12000 [6:08:10<52:03,  1.96s/it]
                                                       

 87%|████████▋ | 10407/12000 [6:08:10<52:03,  1.96s/it]
 87%|████████▋ | 10408/12000 [6:08:12<49:28,  1.86s/it]
                                                       

 87%|████████▋ | 10408/12000 [6:08:12<49:28,  1.86s/it]
 87%|████████▋ | 10409/12000 [6:08:13<48:54,  1.84s/it]
                                                       

 87%|████████▋ | 10409/12000 [6:08:13<48:54,  1.84s/it]
 87%|████████▋ | 10410/12000 [6:08:15<47:47,  1.80s/it]
                                                       

 87%|████████▋ | 10410/12000 [6:08:15<47:47,  1.80s/it]
 87%|████████▋ | 10411/12000 [6:08:17<48:17,  1.82s/it]
                                                       

 87%|████████▋ | 10411/12000 [6:08:17<48:17,  1.82s/it]
 87%|████████▋ | 10412/12000 [6:08:19<53:13,  2.01s/it]
                                                       

 87%|████████▋ | 10412/12000 [6:08:19<53:13,  2.01s/it]
 87%|████████▋ | 10413/12000 [6:08:23<1:05:21,  2.47s/it]
                                                         

 87%|████████▋ | 10413/12000 [6:08:23<1:05:21,  2.47s/it]
 87%|████████▋ | 10414/12000 [6:08:25<59:02,  2.23s/it]  
                                                       

 87%|████████▋ | 10414/12000 [6:08:25<59:02,  2.23s/it]
 87%|████████▋ | 10415/12000 [6:08:26<55:41,  2.11s/it]
                                                       

 87%|████████▋ | 10415/12000 [6:08:26<55:41,  2.11s/it]
 87%|████████▋ | 10416/12000 [6:08:28<50:13,  1.90s/it]
                                                       

 87%|████████▋ | 10416/12000 [6:08:28<50:13,  1.90s/it]
 87%|████████▋ | 10417/12000 [6:08:29<47:01,  1.78s/it]
                                                       

 87%|████████▋ | 10417/12000 [6:08:29<47:01,  1.78s/it]
 87%|████████▋ | 10418/12000 [6:08:31<46:21,  1.76s/it]
                                                       

 87%|████████▋ | 10418/12000 [6:08:31<46:21,  1.76s/it]
 87%|████████▋ | 10419/12000 [6:08:33<44:18,  1.68s/it]
                                                       

 87%|████████▋ | 10419/12000 [6:08:33<44:18,  1.68s/it]
 87%|████████▋ | 10420/12000 [6:08:34<42:24,  1.61s/it]
                                                       

 87%|████████▋ | 10420/12000 [6:08:34<42:24,  1.61s/it]
 87%|████████▋ | 10421/12000 [6:08:36<41:25,  1.57s/it]
                                                       

 87%|████████▋ | 10421/12000 [6:08:36<41:25,  1.57s/it]
 87%|████████▋ | 10422/12000 [6:08:38<47:18,  1.80s/it]
                                                       

 87%|████████▋ | 10422/12000 [6:08:38<47:18,  1.80s/it]
 87%|████████▋ | 10423/12000 [6:08:40<53:45,  2.05s/it]
                                                       

 87%|████████▋ | 10423/12000 [6:08:40<53:45,  2.05s/it]
 87%|████████▋ | 10424/12000 [6:08:42<49:14,  1.87s/it]
                                                       

 87%|████████▋ | 10424/12000 [6:08:42<49:14,  1.87s/it]
 87%|████████▋ | 10425/12000 [6:08:46<1:06:44,  2.54s/it]
                                                         

 87%|████████▋ | 10425/12000 [6:08:46<1:06:44,  2.54s/it]
 87%|████████▋ | 10426/12000 [6:08:48<58:56,  2.25s/it]  
                                                       

 87%|████████▋ | 10426/12000 [6:08:48<58:56,  2.25s/it]
 87%|████████▋ | 10427/12000 [6:08:49<53:32,  2.04s/it]
                                                       

 87%|████████▋ | 10427/12000 [6:08:49<53:32,  2.04s/it]
 87%|████████▋ | 10428/12000 [6:08:51<48:41,  1.86s/it]
                                                       

 87%|████████▋ | 10428/12000 [6:08:51<48:41,  1.86s/it]
 87%|████████▋ | 10429/12000 [6:08:52<47:59,  1.83s/it]
                                                       

 87%|████████▋ | 10429/12000 [6:08:52<47:59,  1.83s/it]
 87%|████████▋ | 10430/12000 [6:08:54<44:57,  1.72s/it]
                                                       

 87%|████████▋ | 10430/12000 [6:08:54<44:57,  1.72s/it]
 87%|████████▋ | 10431/12000 [6:08:57<52:58,  2.03s/it]
                                                       

 87%|████████▋ | 10431/12000 [6:08:57<52:58,  2.03s/it]
 87%|████████▋ | 10432/12000 [6:08:58<49:20,  1.89s/it]
                                                       

 87%|████████▋ | 10432/12000 [6:08:58<49:20,  1.89s/it]
 87%|████████▋ | 10433/12000 [6:09:00<46:01,  1.76s/it]
                                                       

 87%|████████▋ | 10433/12000 [6:09:00<46:01,  1.76s/it]
 87%|████████▋ | 10434/12000 [6:09:02<54:28,  2.09s/it]
                                                       

 87%|████████▋ | 10434/12000 [6:09:02<54:28,  2.09s/it]
 87%|████████▋ | 10435/12000 [6:09:07<1:11:29,  2.74s/it]
                                                         

 87%|████████▋ | 10435/12000 [6:09:07<1:11:29,  2.74s/it]
 87%|████████▋ | 10436/12000 [6:09:09<1:05:48,  2.52s/it]
                                                         

 87%|████████▋ | 10436/12000 [6:09:09<1:05:48,  2.52s/it]
 87%|████████▋ | 10437/12000 [6:09:12<1:12:32,  2.79s/it]
                                                         

 87%|████████▋ | 10437/12000 [6:09:12<1:12:32,  2.79s/it]
 87%|████████▋ | 10438/12000 [6:09:14<1:04:34,  2.48s/it]
                                                         
{'loss': '0.7716', 'grad_norm': '0.4198', 'learning_rate': '9.682e-06', 'entropy': '0.7663', 'num_tokens': '5.595e+07', 'mean_token_accuracy': '0.7975', 'epoch': '0.09196'}
{'loss': '0.7401', 'grad_norm': '0.4257', 'learning_rate': '9.67e-06', 'entropy': '0.7541', 'num_tokens': '5.595e+07', 'mean_token_accuracy': '0.7945', 'epoch': '0.09197'}
{'loss': '0.9837', 'grad_norm': '0.4834', 'learning_rate': '9.659e-06', 'entropy': '0.9361', 'num_tokens': '5.596e+07', 'mean_token_accuracy': '0.7635', 'epoch': '0.09198'}
{'loss': '0.915', 'grad_norm': '0.382', 'learning_rate': '9.647e-06', 'entropy': '0.8468', 'num_tokens': '5.596e+07', 'mean_token_accuracy': '0.8005', 'epoch': '0.09199'}
{'loss': '0.6332', 'grad_norm': '0.2911', 'learning_rate': '9.635e-06', 'entropy': '0.6604', 'num_tokens': '5.597e+07', 'mean_token_accuracy': '0.831', 'epoch': '0.09199'}
{'loss': '0.7282', 'grad_norm': '0.4301', 'learning_rate': '9.623e-06', 'entropy': '0.7563', 'num_tokens': '5.598e+07', 'mean_token_accuracy': '0.8028', 'epoch': '0.092'}
{'loss': '0.6952', 'grad_norm': '0.3831', 'learning_rate': '9.611e-06', 'entropy': '0.7246', 'num_tokens': '5.598e+07', 'mean_token_accuracy': '0.8155', 'epoch': '0.09201'}
{'loss': '0.7344', 'grad_norm': '0.3565', 'learning_rate': '9.6e-06', 'entropy': '0.7621', 'num_tokens': '5.599e+07', 'mean_token_accuracy': '0.8149', 'epoch': '0.09202'}
{'loss': '0.7297', 'grad_norm': '0.4456', 'learning_rate': '9.588e-06', 'entropy': '0.7694', 'num_tokens': '5.599e+07', 'mean_token_accuracy': '0.8093', 'epoch': '0.09203'}
{'loss': '0.8696', 'grad_norm': '0.4003', 'learning_rate': '9.576e-06', 'entropy': '0.9117', 'num_tokens': '5.6e+07', 'mean_token_accuracy': '0.7792', 'epoch': '0.09204'}
{'loss': '0.5787', 'grad_norm': '0.2946', 'learning_rate': '9.564e-06', 'entropy': '0.6575', 'num_tokens': '5.6e+07', 'mean_token_accuracy': '0.8195', 'epoch': '0.09205'}
{'loss': '0.7573', 'grad_norm': '0.4469', 'learning_rate': '9.553e-06', 'entropy': '0.7623', 'num_tokens': '5.6e+07', 'mean_token_accuracy': '0.8074', 'epoch': '0.09206'}
{'loss': '0.9453', 'grad_norm': '0.4289', 'learning_rate': '9.541e-06', 'entropy': '0.9452', 'num_tokens': '5.601e+07', 'mean_token_accuracy': '0.7653', 'epoch': '0.09206'}
{'loss': '1.006', 'grad_norm': '0.4181', 'learning_rate': '9.529e-06', 'entropy': '0.9188', 'num_tokens': '5.602e+07', 'mean_token_accuracy': '0.7702', 'epoch': '0.09207'}
{'loss': '0.6705', 'grad_norm': '0.4749', 'learning_rate': '9.517e-06', 'entropy': '0.7445', 'num_tokens': '5.602e+07', 'mean_token_accuracy': '0.8152', 'epoch': '0.09208'}
{'loss': '1.18', 'grad_norm': '0.4519', 'learning_rate': '9.506e-06', 'entropy': '1.083', 'num_tokens': '5.603e+07', 'mean_token_accuracy': '0.7422', 'epoch': '0.09209'}
{'loss': '1.014', 'grad_norm': '0.4113', 'learning_rate': '9.494e-06', 'entropy': '0.9368', 'num_tokens': '5.603e+07', 'mean_token_accuracy': '0.7657', 'epoch': '0.0921'}
{'loss': '0.7043', 'grad_norm': '0.4191', 'learning_rate': '9.482e-06', 'entropy': '0.7515', 'num_tokens': '5.604e+07', 'mean_token_accuracy': '0.8165', 'epoch': '0.09211'}
{'loss': '0.9821', 'grad_norm': '0.382', 'learning_rate': '9.47e-06', 'entropy': '1.005', 'num_tokens': '5.604e+07', 'mean_token_accuracy': '0.7461', 'epoch': '0.09212'}
{'loss': '1.03', 'grad_norm': '0.3838', 'learning_rate': '9.459e-06', 'entropy': '0.9269', 'num_tokens': '5.605e+07', 'mean_token_accuracy': '0.7675', 'epoch': '0.09213'}
{'loss': '0.6068', 'grad_norm': '0.4293', 'learning_rate': '9.447e-06', 'entropy': '0.6675', 'num_tokens': '5.605e+07', 'mean_token_accuracy': '0.8263', 'epoch': '0.09214'}
{'loss': '0.532', 'grad_norm': '0.3104', 'learning_rate': '9.435e-06', 'entropy': '0.5589', 'num_tokens': '5.606e+07', 'mean_token_accuracy': '0.8534', 'epoch': '0.09214'}
{'loss': '0.6131', 'grad_norm': '0.3999', 'learning_rate': '9.424e-06', 'entropy': '0.6451', 'num_tokens': '5.606e+07', 'mean_token_accuracy': '0.8284', 'epoch': '0.09215'}
{'loss': '0.8633', 'grad_norm': '0.3964', 'learning_rate': '9.412e-06', 'entropy': '0.9063', 'num_tokens': '5.607e+07', 'mean_token_accuracy': '0.7674', 'epoch': '0.09216'}
{'loss': '0.8501', 'grad_norm': '0.4459', 'learning_rate': '9.4e-06', 'entropy': '0.8513', 'num_tokens': '5.608e+07', 'mean_token_accuracy': '0.791', 'epoch': '0.09217'}
{'loss': '1.042', 'grad_norm': '0.518', 'learning_rate': '9.389e-06', 'entropy': '1.002', 'num_tokens': '5.608e+07', 'mean_token_accuracy': '0.7594', 'epoch': '0.09218'}
{'loss': '0.9462', 'grad_norm': '0.4904', 'learning_rate': '9.377e-06', 'entropy': '0.9063', 'num_tokens': '5.608e+07', 'mean_token_accuracy': '0.7707', 'epoch': '0.09219'}
{'loss': '0.5408', 'grad_norm': '0.3826', 'learning_rate': '9.365e-06', 'entropy': '0.5429', 'num_tokens': '5.609e+07', 'mean_token_accuracy': '0.8489', 'epoch': '0.0922'}
{'loss': '1.206', 'grad_norm': '0.4996', 'learning_rate': '9.354e-06', 'entropy': '1.2', 'num_tokens': '5.609e+07', 'mean_token_accuracy': '0.7256', 'epoch': '0.09221'}
{'loss': '0.5565', 'grad_norm': '0.3332', 'learning_rate': '9.342e-06', 'entropy': '0.5977', 'num_tokens': '5.61e+07', 'mean_token_accuracy': '0.8433', 'epoch': '0.09222'}
{'loss': '0.6903', 'grad_norm': '0.4159', 'learning_rate': '9.33e-06', 'entropy': '0.7359', 'num_tokens': '5.61e+07', 'mean_token_accuracy': '0.8073', 'epoch': '0.09222'}
{'loss': '0.8266', 'grad_norm': '0.3491', 'learning_rate': '9.319e-06', 'entropy': '0.8265', 'num_tokens': '5.611e+07', 'mean_token_accuracy': '0.805', 'epoch': '0.09223'}
{'loss': '0.6309', 'grad_norm': '0.3329', 'learning_rate': '9.307e-06', 'entropy': '0.8513', 'num_tokens': '5.611e+07', 'mean_token_accuracy': '0.7868', 'epoch': '0.09224'}
{'loss': '0.5173', 'grad_norm': '0.324', 'learning_rate': '9.296e-06', 'entropy': '0.5442', 'num_tokens': '5.612e+07', 'mean_token_accuracy': '0.8458', 'epoch': '0.09225'}
{'loss': '0.4616', 'grad_norm': '0.2646', 'learning_rate': '9.284e-06', 'entropy': '0.633', 'num_tokens': '5.612e+07', 'mean_token_accuracy': '0.8433', 'epoch': '0.09226'}
{'loss': '0.6297', 'grad_norm': '0.5069', 'learning_rate': '9.272e-06', 'entropy': '0.6951', 'num_tokens': '5.613e+07', 'mean_token_accuracy': '0.8224', 'epoch': '0.09227'}
{'loss': '0.6849', 'grad_norm': '0.3898', 'learning_rate': '9.261e-06', 'entropy': '0.7436', 'num_tokens': '5.613e+07', 'mean_token_accuracy': '0.8155', 'epoch': '0.09228'}
{'loss': '1.065', 'grad_norm': '0.5421', 'learning_rate': '9.249e-06', 'entropy': '0.9961', 'num_tokens': '5.614e+07', 'mean_token_accuracy': '0.7565', 'epoch': '0.09229'}
{'loss': '0.8682', 'grad_norm': '0.4298', 'learning_rate': '9.238e-06', 'entropy': '0.8949', 'num_tokens': '5.614e+07', 'mean_token_accuracy': '0.7653', 'epoch': '0.09229'}
{'loss': '0.6106', 'grad_norm': '0.4143', 'learning_rate': '9.226e-06', 'entropy': '0.6343', 'num_tokens': '5.615e+07', 'mean_token_accuracy': '0.8387', 'epoch': '0.0923'}
{'loss': '0.7959', 'grad_norm': '0.3346', 'learning_rate': '9.215e-06', 'entropy': '0.8662', 'num_tokens': '5.615e+07', 'mean_token_accuracy': '0.7838', 'epoch': '0.09231'}
{'loss': '0.5374', 'grad_norm': '0.3717', 'learning_rate': '9.203e-06', 'entropy': '0.5228', 'num_tokens': '5.616e+07', 'mean_token_accuracy': '0.8668', 'epoch': '0.09232'}
{'loss': '0.94', 'grad_norm': '0.4121', 'learning_rate': '9.191e-06', 'entropy': '0.9339', 'num_tokens': '5.616e+07', 'mean_token_accuracy': '0.7693', 'epoch': '0.09233'}
{'loss': '1.345', 'grad_norm': '0.4167', 'learning_rate': '9.18e-06', 'entropy': '1.198', 'num_tokens': '5.617e+07', 'mean_token_accuracy': '0.7151', 'epoch': '0.09234'}
{'loss': '1.353', 'grad_norm': '0.4233', 'learning_rate': '9.168e-06', 'entropy': '1.349', 'num_tokens': '5.617e+07', 'mean_token_accuracy': '0.6829', 'epoch': '0.09235'}
{'loss': '1.331', 'grad_norm': '0.5702', 'learning_rate': '9.157e-06', 'entropy': '1.211', 'num_tokens': '5.618e+07', 'mean_token_accuracy': '0.7041', 'epoch': '0.09236'}
{'loss': '0.7608', 'grad_norm': '0.3629', 'learning_rate': '9.145e-06', 'entropy': '0.7336', 'num_tokens': '5.619e+07', 'mean_token_accuracy': '0.8064', 'epoch': '0.09237'}

 87%|████████▋ | 10438/12000 [6:09:14<1:04:34,  2.48s/it]
 87%|████████▋ | 10439/12000 [6:09:16<1:03:24,  2.44s/it]
                                                         

 87%|████████▋ | 10439/12000 [6:09:16<1:03:24,  2.44s/it]
 87%|████████▋ | 10440/12000 [6:09:18<56:28,  2.17s/it]  
                                                       

 87%|████████▋ | 10440/12000 [6:09:18<56:28,  2.17s/it]
 87%|████████▋ | 10441/12000 [6:09:20<56:25,  2.17s/it]
                                                       

 87%|████████▋ | 10441/12000 [6:09:20<56:25,  2.17s/it]
 87%|████████▋ | 10442/12000 [6:09:22<52:36,  2.03s/it]
                                                       

 87%|████████▋ | 10442/12000 [6:09:22<52:36,  2.03s/it]
 87%|████████▋ | 10443/12000 [6:09:26<1:13:25,  2.83s/it]
                                                         

 87%|████████▋ | 10443/12000 [6:09:26<1:13:25,  2.83s/it]
 87%|████████▋ | 10444/12000 [6:09:28<1:06:32,  2.57s/it]
                                                         

 87%|████████▋ | 10444/12000 [6:09:28<1:06:32,  2.57s/it]
 87%|████████▋ | 10445/12000 [6:09:30<57:52,  2.23s/it]  
                                                       

 87%|████████▋ | 10445/12000 [6:09:30<57:52,  2.23s/it]
 87%|████████▋ | 10446/12000 [6:09:32<56:39,  2.19s/it]
                                                       

 87%|████████▋ | 10446/12000 [6:09:32<56:39,  2.19s/it]
 87%|████████▋ | 10447/12000 [6:09:34<1:00:00,  2.32s/it]
                                                         

 87%|████████▋ | 10447/12000 [6:09:34<1:00:00,  2.32s/it]
 87%|████████▋ | 10448/12000 [6:09:36<53:19,  2.06s/it]  
                                                       

 87%|████████▋ | 10448/12000 [6:09:36<53:19,  2.06s/it]
 87%|████████▋ | 10449/12000 [6:09:37<48:48,  1.89s/it]
                                                       

 87%|████████▋ | 10449/12000 [6:09:37<48:48,  1.89s/it]
 87%|████████▋ | 10450/12000 [6:09:39<44:47,  1.73s/it]
                                                       

 87%|████████▋ | 10450/12000 [6:09:39<44:47,  1.73s/it]
 87%|████████▋ | 10451/12000 [6:09:42<54:15,  2.10s/it]
                                                       

 87%|████████▋ | 10451/12000 [6:09:42<54:15,  2.10s/it]
 87%|████████▋ | 10452/12000 [6:09:44<52:28,  2.03s/it]
                                                       

 87%|████████▋ | 10452/12000 [6:09:44<52:28,  2.03s/it]
 87%|████████▋ | 10453/12000 [6:09:45<47:16,  1.83s/it]
                                                       

 87%|████████▋ | 10453/12000 [6:09:45<47:16,  1.83s/it]
 87%|████████▋ | 10454/12000 [6:09:47<51:13,  1.99s/it]
                                                       

 87%|████████▋ | 10454/12000 [6:09:47<51:13,  1.99s/it]
 87%|████████▋ | 10455/12000 [6:09:49<50:49,  1.97s/it]
                                                       

 87%|████████▋ | 10455/12000 [6:09:49<50:49,  1.97s/it]
 87%|████████▋ | 10456/12000 [6:09:51<47:04,  1.83s/it]
                                                       

 87%|████████▋ | 10456/12000 [6:09:51<47:04,  1.83s/it]
 87%|████████▋ | 10457/12000 [6:09:55<1:02:18,  2.42s/it]
                                                         

 87%|████████▋ | 10457/12000 [6:09:55<1:02:18,  2.42s/it]
 87%|████████▋ | 10458/12000 [6:09:58<1:08:05,  2.65s/it]
                                                         

 87%|████████▋ | 10458/12000 [6:09:58<1:08:05,  2.65s/it]
 87%|████████▋ | 10459/12000 [6:09:59<59:35,  2.32s/it]  
                                                       

 87%|████████▋ | 10459/12000 [6:09:59<59:35,  2.32s/it]
 87%|████████▋ | 10460/12000 [6:10:01<54:15,  2.11s/it]
                                                       

 87%|████████▋ | 10460/12000 [6:10:01<54:15,  2.11s/it]
 87%|████████▋ | 10461/12000 [6:10:02<47:48,  1.86s/it]
                                                       

 87%|████████▋ | 10461/12000 [6:10:02<47:48,  1.86s/it]
 87%|████████▋ | 10462/12000 [6:10:04<50:19,  1.96s/it]
                                                       

 87%|████████▋ | 10462/12000 [6:10:04<50:19,  1.96s/it]
 87%|████████▋ | 10463/12000 [6:10:06<46:25,  1.81s/it]
                                                       

 87%|████████▋ | 10463/12000 [6:10:06<46:25,  1.81s/it]
 87%|████████▋ | 10464/12000 [6:10:08<44:59,  1.76s/it]
                                                       

 87%|████████▋ | 10464/12000 [6:10:08<44:59,  1.76s/it]
 87%|████████▋ | 10465/12000 [6:10:09<44:09,  1.73s/it]
                                                       

 87%|████████▋ | 10465/12000 [6:10:09<44:09,  1.73s/it]
 87%|████████▋ | 10466/12000 [6:10:11<44:16,  1.73s/it]
                                                       

 87%|████████▋ | 10466/12000 [6:10:11<44:16,  1.73s/it]
 87%|████████▋ | 10467/12000 [6:10:12<42:21,  1.66s/it]
                                                       

 87%|████████▋ | 10467/12000 [6:10:12<42:21,  1.66s/it]
 87%|████████▋ | 10468/12000 [6:10:14<42:26,  1.66s/it]
                                                       

 87%|████████▋ | 10468/12000 [6:10:14<42:26,  1.66s/it]
 87%|████████▋ | 10469/12000 [6:10:16<41:59,  1.65s/it]
                                                       

 87%|████████▋ | 10469/12000 [6:10:16<41:59,  1.65s/it]
 87%|████████▋ | 10470/12000 [6:10:18<44:40,  1.75s/it]
                                                       

 87%|████████▋ | 10470/12000 [6:10:18<44:40,  1.75s/it]
 87%|████████▋ | 10471/12000 [6:10:19<43:38,  1.71s/it]
                                                       

 87%|████████▋ | 10471/12000 [6:10:19<43:38,  1.71s/it]
 87%|████████▋ | 10472/12000 [6:10:21<42:41,  1.68s/it]
                                                       

 87%|████████▋ | 10472/12000 [6:10:21<42:41,  1.68s/it]
 87%|████████▋ | 10473/12000 [6:10:23<44:10,  1.74s/it]
                                                       

 87%|████████▋ | 10473/12000 [6:10:23<44:10,  1.74s/it]
 87%|████████▋ | 10474/12000 [6:10:24<42:41,  1.68s/it]
                                                       

 87%|████████▋ | 10474/12000 [6:10:24<42:41,  1.68s/it]
 87%|████████▋ | 10475/12000 [6:10:26<43:05,  1.70s/it]
                                                       

 87%|████████▋ | 10475/12000 [6:10:26<43:05,  1.70s/it]
 87%|████████▋ | 10476/12000 [6:10:28<42:56,  1.69s/it]
                                                       

 87%|████████▋ | 10476/12000 [6:10:28<42:56,  1.69s/it]
 87%|████████▋ | 10477/12000 [6:10:29<41:51,  1.65s/it]
                                                       

 87%|████████▋ | 10477/12000 [6:10:29<41:51,  1.65s/it]
 87%|████████▋ | 10478/12000 [6:10:31<43:09,  1.70s/it]
                                                       

 87%|████████▋ | 10478/12000 [6:10:31<43:09,  1.70s/it]
 87%|████████▋ | 10479/12000 [6:10:35<1:00:53,  2.40s/it]
                                                         

 87%|████████▋ | 10479/12000 [6:10:35<1:00:53,  2.40s/it]
 87%|████████▋ | 10480/12000 [6:10:37<55:03,  2.17s/it]  
                                                       

 87%|████████▋ | 10480/12000 [6:10:37<55:03,  2.17s/it]
 87%|████████▋ | 10481/12000 [6:10:39<54:09,  2.14s/it]
                                                       

 87%|████████▋ | 10481/12000 [6:10:39<54:09,  2.14s/it]
 87%|████████▋ | 10482/12000 [6:10:40<49:20,  1.95s/it]
                                                       

 87%|████████▋ | 10482/12000 [6:10:40<49:20,  1.95s/it]
 87%|████████▋ | 10483/12000 [6:10:42<50:12,  1.99s/it]
                                                       

 87%|████████▋ | 10483/12000 [6:10:42<50:12,  1.99s/it]
 87%|████████▋ | 10484/12000 [6:10:44<46:15,  1.83s/it]
                                                       

 87%|████████▋ | 10484/12000 [6:10:44<46:15,  1.83s/it]
 87%|████████▋ | 10485/12000 [6:10:46<47:34,  1.88s/it]
                                                       
{'loss': '0.9751', 'grad_norm': '0.4306', 'learning_rate': '9.134e-06', 'entropy': '0.8784', 'num_tokens': '5.619e+07', 'mean_token_accuracy': '0.7946', 'epoch': '0.09237'}
{'loss': '0.6213', 'grad_norm': '0.3909', 'learning_rate': '9.122e-06', 'entropy': '0.6536', 'num_tokens': '5.62e+07', 'mean_token_accuracy': '0.8258', 'epoch': '0.09238'}
{'loss': '0.941', 'grad_norm': '0.4319', 'learning_rate': '9.111e-06', 'entropy': '0.8866', 'num_tokens': '5.62e+07', 'mean_token_accuracy': '0.788', 'epoch': '0.09239'}
{'loss': '1.049', 'grad_norm': '0.4558', 'learning_rate': '9.099e-06', 'entropy': '0.9414', 'num_tokens': '5.621e+07', 'mean_token_accuracy': '0.7728', 'epoch': '0.0924'}
{'loss': '0.6513', 'grad_norm': '0.347', 'learning_rate': '9.088e-06', 'entropy': '0.7119', 'num_tokens': '5.621e+07', 'mean_token_accuracy': '0.8052', 'epoch': '0.09241'}
{'loss': '1.102', 'grad_norm': '0.3988', 'learning_rate': '9.076e-06', 'entropy': '0.9479', 'num_tokens': '5.622e+07', 'mean_token_accuracy': '0.7276', 'epoch': '0.09242'}
{'loss': '0.7093', 'grad_norm': '0.4014', 'learning_rate': '9.065e-06', 'entropy': '0.7149', 'num_tokens': '5.622e+07', 'mean_token_accuracy': '0.827', 'epoch': '0.09243'}
{'loss': '0.6805', 'grad_norm': '0.3903', 'learning_rate': '9.053e-06', 'entropy': '0.6995', 'num_tokens': '5.623e+07', 'mean_token_accuracy': '0.829', 'epoch': '0.09244'}
{'loss': '0.8641', 'grad_norm': '0.4687', 'learning_rate': '9.042e-06', 'entropy': '0.8632', 'num_tokens': '5.623e+07', 'mean_token_accuracy': '0.8009', 'epoch': '0.09245'}
{'loss': '0.7126', 'grad_norm': '0.4422', 'learning_rate': '9.031e-06', 'entropy': '0.7568', 'num_tokens': '5.624e+07', 'mean_token_accuracy': '0.8059', 'epoch': '0.09245'}
{'loss': '0.8073', 'grad_norm': '0.4496', 'learning_rate': '9.019e-06', 'entropy': '0.8095', 'num_tokens': '5.624e+07', 'mean_token_accuracy': '0.7907', 'epoch': '0.09246'}
{'loss': '0.9333', 'grad_norm': '0.467', 'learning_rate': '9.008e-06', 'entropy': '0.9724', 'num_tokens': '5.625e+07', 'mean_token_accuracy': '0.741', 'epoch': '0.09247'}
{'loss': '0.5104', 'grad_norm': '0.4343', 'learning_rate': '8.996e-06', 'entropy': '0.5447', 'num_tokens': '5.625e+07', 'mean_token_accuracy': '0.849', 'epoch': '0.09248'}
{'loss': '0.7961', 'grad_norm': '0.4037', 'learning_rate': '8.985e-06', 'entropy': '0.8278', 'num_tokens': '5.625e+07', 'mean_token_accuracy': '0.788', 'epoch': '0.09249'}
{'loss': '1.077', 'grad_norm': '0.4532', 'learning_rate': '8.973e-06', 'entropy': '0.9153', 'num_tokens': '5.626e+07', 'mean_token_accuracy': '0.7674', 'epoch': '0.0925'}
{'loss': '0.6573', 'grad_norm': '0.393', 'learning_rate': '8.962e-06', 'entropy': '0.6426', 'num_tokens': '5.626e+07', 'mean_token_accuracy': '0.841', 'epoch': '0.09251'}
{'loss': '0.8492', 'grad_norm': '0.3328', 'learning_rate': '8.951e-06', 'entropy': '0.8877', 'num_tokens': '5.627e+07', 'mean_token_accuracy': '0.7761', 'epoch': '0.09252'}
{'loss': '0.8762', 'grad_norm': '0.3831', 'learning_rate': '8.939e-06', 'entropy': '0.8589', 'num_tokens': '5.627e+07', 'mean_token_accuracy': '0.7839', 'epoch': '0.09252'}
{'loss': '0.6881', 'grad_norm': '0.3554', 'learning_rate': '8.928e-06', 'entropy': '0.756', 'num_tokens': '5.628e+07', 'mean_token_accuracy': '0.7951', 'epoch': '0.09253'}
{'loss': '0.7289', 'grad_norm': '0.4078', 'learning_rate': '8.916e-06', 'entropy': '0.7118', 'num_tokens': '5.628e+07', 'mean_token_accuracy': '0.8174', 'epoch': '0.09254'}
{'loss': '1.041', 'grad_norm': '0.4029', 'learning_rate': '8.905e-06', 'entropy': '1.022', 'num_tokens': '5.629e+07', 'mean_token_accuracy': '0.7675', 'epoch': '0.09255'}
{'loss': '0.9306', 'grad_norm': '0.4355', 'learning_rate': '8.894e-06', 'entropy': '0.9457', 'num_tokens': '5.63e+07', 'mean_token_accuracy': '0.7559', 'epoch': '0.09256'}
{'loss': '0.6584', 'grad_norm': '0.3846', 'learning_rate': '8.882e-06', 'entropy': '0.6755', 'num_tokens': '5.63e+07', 'mean_token_accuracy': '0.8211', 'epoch': '0.09257'}
{'loss': '0.8214', 'grad_norm': '0.3829', 'learning_rate': '8.871e-06', 'entropy': '0.8818', 'num_tokens': '5.63e+07', 'mean_token_accuracy': '0.7732', 'epoch': '0.09258'}
{'loss': '0.9595', 'grad_norm': '0.391', 'learning_rate': '8.86e-06', 'entropy': '0.9039', 'num_tokens': '5.631e+07', 'mean_token_accuracy': '0.7707', 'epoch': '0.09259'}
{'loss': '0.587', 'grad_norm': '0.37', 'learning_rate': '8.848e-06', 'entropy': '0.6073', 'num_tokens': '5.631e+07', 'mean_token_accuracy': '0.8332', 'epoch': '0.0926'}
{'loss': '0.6552', 'grad_norm': '0.3531', 'learning_rate': '8.837e-06', 'entropy': '0.6571', 'num_tokens': '5.632e+07', 'mean_token_accuracy': '0.8255', 'epoch': '0.0926'}
{'loss': '0.6458', 'grad_norm': '0.3508', 'learning_rate': '8.826e-06', 'entropy': '0.668', 'num_tokens': '5.633e+07', 'mean_token_accuracy': '0.8356', 'epoch': '0.09261'}
{'loss': '0.6138', 'grad_norm': '0.4193', 'learning_rate': '8.814e-06', 'entropy': '0.5999', 'num_tokens': '5.633e+07', 'mean_token_accuracy': '0.8429', 'epoch': '0.09262'}
{'loss': '0.7448', 'grad_norm': '0.3803', 'learning_rate': '8.803e-06', 'entropy': '0.7518', 'num_tokens': '5.633e+07', 'mean_token_accuracy': '0.8094', 'epoch': '0.09263'}
{'loss': '0.7946', 'grad_norm': '0.4331', 'learning_rate': '8.792e-06', 'entropy': '0.8578', 'num_tokens': '5.634e+07', 'mean_token_accuracy': '0.7804', 'epoch': '0.09264'}
{'loss': '0.7701', 'grad_norm': '0.4138', 'learning_rate': '8.78e-06', 'entropy': '0.7462', 'num_tokens': '5.634e+07', 'mean_token_accuracy': '0.799', 'epoch': '0.09265'}
{'loss': '0.8865', 'grad_norm': '0.4274', 'learning_rate': '8.769e-06', 'entropy': '0.8309', 'num_tokens': '5.635e+07', 'mean_token_accuracy': '0.7999', 'epoch': '0.09266'}
{'loss': '0.4657', 'grad_norm': '0.3275', 'learning_rate': '8.758e-06', 'entropy': '0.4983', 'num_tokens': '5.636e+07', 'mean_token_accuracy': '0.8662', 'epoch': '0.09267'}
{'loss': '0.7036', 'grad_norm': '0.3687', 'learning_rate': '8.747e-06', 'entropy': '0.7179', 'num_tokens': '5.636e+07', 'mean_token_accuracy': '0.8085', 'epoch': '0.09268'}
{'loss': '0.8572', 'grad_norm': '0.3922', 'learning_rate': '8.735e-06', 'entropy': '0.8364', 'num_tokens': '5.637e+07', 'mean_token_accuracy': '0.7937', 'epoch': '0.09268'}
{'loss': '0.5577', 'grad_norm': '0.3654', 'learning_rate': '8.724e-06', 'entropy': '0.5827', 'num_tokens': '5.637e+07', 'mean_token_accuracy': '0.8538', 'epoch': '0.09269'}
{'loss': '0.8374', 'grad_norm': '0.5748', 'learning_rate': '8.713e-06', 'entropy': '0.8847', 'num_tokens': '5.638e+07', 'mean_token_accuracy': '0.78', 'epoch': '0.0927'}
{'loss': '1.283', 'grad_norm': '0.437', 'learning_rate': '8.702e-06', 'entropy': '1.319', 'num_tokens': '5.638e+07', 'mean_token_accuracy': '0.6827', 'epoch': '0.09271'}
{'loss': '0.7905', 'grad_norm': '0.4215', 'learning_rate': '8.69e-06', 'entropy': '0.8436', 'num_tokens': '5.639e+07', 'mean_token_accuracy': '0.7916', 'epoch': '0.09272'}
{'loss': '1.086', 'grad_norm': '0.4546', 'learning_rate': '8.679e-06', 'entropy': '0.9655', 'num_tokens': '5.639e+07', 'mean_token_accuracy': '0.7585', 'epoch': '0.09273'}
{'loss': '0.9952', 'grad_norm': '0.4405', 'learning_rate': '8.668e-06', 'entropy': '1.005', 'num_tokens': '5.64e+07', 'mean_token_accuracy': '0.748', 'epoch': '0.09274'}
{'loss': '0.7051', 'grad_norm': '0.3724', 'learning_rate': '8.657e-06', 'entropy': '0.6969', 'num_tokens': '5.64e+07', 'mean_token_accuracy': '0.8294', 'epoch': '0.09275'}
{'loss': '0.7811', 'grad_norm': '0.3207', 'learning_rate': '8.645e-06', 'entropy': '0.7288', 'num_tokens': '5.641e+07', 'mean_token_accuracy': '0.8146', 'epoch': '0.09276'}
{'loss': '0.7294', 'grad_norm': '0.3973', 'learning_rate': '8.634e-06', 'entropy': '0.7822', 'num_tokens': '5.641e+07', 'mean_token_accuracy': '0.7951', 'epoch': '0.09276'}
{'loss': '0.8082', 'grad_norm': '0.5017', 'learning_rate': '8.623e-06', 'entropy': '0.7111', 'num_tokens': '5.642e+07', 'mean_token_accuracy': '0.8164', 'epoch': '0.09277'}
{'loss': '0.8307', 'grad_norm': '0.3946', 'learning_rate': '8.612e-06', 'entropy': '0.8305', 'num_tokens': '5.642e+07', 'mean_token_accuracy': '0.7859', 'epoch': '0.09278'}

 87%|████████▋ | 10485/12000 [6:10:46<47:34,  1.88s/it]
 87%|████████▋ | 10486/12000 [6:10:47<45:08,  1.79s/it]
                                                       

 87%|████████▋ | 10486/12000 [6:10:47<45:08,  1.79s/it]
 87%|████████▋ | 10487/12000 [6:10:49<45:24,  1.80s/it]
                                                       

 87%|████████▋ | 10487/12000 [6:10:49<45:24,  1.80s/it]
 87%|████████▋ | 10488/12000 [6:10:51<43:09,  1.71s/it]
                                                       

 87%|████████▋ | 10488/12000 [6:10:51<43:09,  1.71s/it]
 87%|████████▋ | 10489/12000 [6:10:52<41:09,  1.63s/it]
                                                       

 87%|████████▋ | 10489/12000 [6:10:52<41:09,  1.63s/it]
 87%|████████▋ | 10490/12000 [6:10:54<41:13,  1.64s/it]
                                                       

 87%|████████▋ | 10490/12000 [6:10:54<41:13,  1.64s/it]
 87%|████████▋ | 10491/12000 [6:10:56<41:21,  1.64s/it]
                                                       

 87%|████████▋ | 10491/12000 [6:10:56<41:21,  1.64s/it]
 87%|████████▋ | 10492/12000 [6:10:57<41:00,  1.63s/it]
                                                       

 87%|████████▋ | 10492/12000 [6:10:57<41:00,  1.63s/it]
 87%|████████▋ | 10493/12000 [6:10:59<40:26,  1.61s/it]
                                                       

 87%|████████▋ | 10493/12000 [6:10:59<40:26,  1.61s/it]
 87%|████████▋ | 10494/12000 [6:11:00<39:09,  1.56s/it]
                                                       

 87%|████████▋ | 10494/12000 [6:11:00<39:09,  1.56s/it]
 87%|████████▋ | 10495/12000 [6:11:02<38:14,  1.52s/it]
                                                       

 87%|████████▋ | 10495/12000 [6:11:02<38:14,  1.52s/it]
 87%|████████▋ | 10496/12000 [6:11:04<44:36,  1.78s/it]
                                                       

 87%|████████▋ | 10496/12000 [6:11:04<44:36,  1.78s/it]
 87%|████████▋ | 10497/12000 [6:11:05<41:45,  1.67s/it]
                                                       

 87%|████████▋ | 10497/12000 [6:11:05<41:45,  1.67s/it]
 87%|████████▋ | 10498/12000 [6:11:07<41:44,  1.67s/it]
                                                       

 87%|████████▋ | 10498/12000 [6:11:07<41:44,  1.67s/it]
 87%|████████▋ | 10499/12000 [6:11:09<44:55,  1.80s/it]
                                                       

 87%|████████▋ | 10499/12000 [6:11:09<44:55,  1.80s/it]
 88%|████████▊ | 10500/12000 [6:11:11<46:46,  1.87s/it]
                                                       

 88%|████████▊ | 10500/12000 [6:11:11<46:46,  1.87s/it]
 88%|████████▊ | 10501/12000 [6:11:16<1:11:21,  2.86s/it]
                                                         

 88%|████████▊ | 10501/12000 [6:11:16<1:11:21,  2.86s/it]
 88%|████████▊ | 10502/12000 [6:11:18<1:03:05,  2.53s/it]
                                                         

 88%|████████▊ | 10502/12000 [6:11:18<1:03:05,  2.53s/it]
 88%|████████▊ | 10503/12000 [6:11:20<56:55,  2.28s/it]  
                                                       

 88%|████████▊ | 10503/12000 [6:11:20<56:55,  2.28s/it]
 88%|████████▊ | 10504/12000 [6:11:21<51:57,  2.08s/it]
                                                       

 88%|████████▊ | 10504/12000 [6:11:21<51:57,  2.08s/it]
 88%|████████▊ | 10505/12000 [6:11:23<49:08,  1.97s/it]
                                                       

 88%|████████▊ | 10505/12000 [6:11:23<49:08,  1.97s/it]
 88%|████████▊ | 10506/12000 [6:11:25<46:47,  1.88s/it]
                                                       

 88%|████████▊ | 10506/12000 [6:11:25<46:47,  1.88s/it]
 88%|████████▊ | 10507/12000 [6:11:28<55:27,  2.23s/it]
                                                       

 88%|████████▊ | 10507/12000 [6:11:28<55:27,  2.23s/it]
 88%|████████▊ | 10508/12000 [6:11:30<52:59,  2.13s/it]
                                                       

 88%|████████▊ | 10508/12000 [6:11:30<52:59,  2.13s/it]
 88%|████████▊ | 10509/12000 [6:11:31<49:45,  2.00s/it]
                                                       

 88%|████████▊ | 10509/12000 [6:11:31<49:45,  2.00s/it]
 88%|████████▊ | 10510/12000 [6:11:33<47:08,  1.90s/it]
                                                       

 88%|████████▊ | 10510/12000 [6:11:33<47:08,  1.90s/it]
 88%|████████▊ | 10511/12000 [6:11:35<46:10,  1.86s/it]
                                                       

 88%|████████▊ | 10511/12000 [6:11:35<46:10,  1.86s/it]
 88%|████████▊ | 10512/12000 [6:11:37<45:46,  1.85s/it]
                                                       

 88%|████████▊ | 10512/12000 [6:11:37<45:46,  1.85s/it]
 88%|████████▊ | 10513/12000 [6:11:39<47:56,  1.93s/it]
                                                       

 88%|████████▊ | 10513/12000 [6:11:39<47:56,  1.93s/it]
 88%|████████▊ | 10514/12000 [6:11:40<45:05,  1.82s/it]
                                                       

 88%|████████▊ | 10514/12000 [6:11:40<45:05,  1.82s/it]
 88%|████████▊ | 10515/12000 [6:11:42<46:26,  1.88s/it]
                                                       

 88%|████████▊ | 10515/12000 [6:11:42<46:26,  1.88s/it]
 88%|████████▊ | 10516/12000 [6:11:44<44:47,  1.81s/it]
                                                       

 88%|████████▊ | 10516/12000 [6:11:44<44:47,  1.81s/it]
 88%|████████▊ | 10517/12000 [6:11:46<43:04,  1.74s/it]
                                                       

 88%|████████▊ | 10517/12000 [6:11:46<43:04,  1.74s/it]
 88%|████████▊ | 10518/12000 [6:11:47<42:49,  1.73s/it]
                                                       

 88%|████████▊ | 10518/12000 [6:11:47<42:49,  1.73s/it]
 88%|████████▊ | 10519/12000 [6:11:50<48:38,  1.97s/it]
                                                       

 88%|████████▊ | 10519/12000 [6:11:50<48:38,  1.97s/it]
 88%|████████▊ | 10520/12000 [6:11:52<52:32,  2.13s/it]
                                                       

 88%|████████▊ | 10520/12000 [6:11:52<52:32,  2.13s/it]
 88%|████████▊ | 10521/12000 [6:11:54<47:31,  1.93s/it]
                                                       

 88%|████████▊ | 10521/12000 [6:11:54<47:31,  1.93s/it]
 88%|████████▊ | 10522/12000 [6:11:56<47:10,  1.91s/it]
                                                       

 88%|████████▊ | 10522/12000 [6:11:56<47:10,  1.91s/it]
 88%|████████▊ | 10523/12000 [6:11:59<1:00:04,  2.44s/it]
                                                         

 88%|████████▊ | 10523/12000 [6:11:59<1:00:04,  2.44s/it]
 88%|████████▊ | 10524/12000 [6:12:02<59:54,  2.44s/it]  
                                                       

 88%|████████▊ | 10524/12000 [6:12:02<59:54,  2.44s/it]
 88%|████████▊ | 10525/12000 [6:12:03<52:07,  2.12s/it]
                                                       

 88%|████████▊ | 10525/12000 [6:12:03<52:07,  2.12s/it]
 88%|████████▊ | 10526/12000 [6:12:07<1:02:27,  2.54s/it]
                                                         

 88%|████████▊ | 10526/12000 [6:12:07<1:02:27,  2.54s/it]
 88%|████████▊ | 10527/12000 [6:12:08<54:31,  2.22s/it]  
                                                       

 88%|████████▊ | 10527/12000 [6:12:08<54:31,  2.22s/it]
 88%|████████▊ | 10528/12000 [6:12:10<49:18,  2.01s/it]
                                                       

 88%|████████▊ | 10528/12000 [6:12:10<49:18,  2.01s/it]
 88%|████████▊ | 10529/12000 [6:12:12<49:01,  2.00s/it]
                                                       

 88%|████████▊ | 10529/12000 [6:12:12<49:01,  2.00s/it]
 88%|████████▊ | 10530/12000 [6:12:13<45:30,  1.86s/it]
                                                       

 88%|████████▊ | 10530/12000 [6:12:13<45:30,  1.86s/it]
 88%|████████▊ | 10531/12000 [6:12:16<50:36,  2.07s/it]
                                                       

 88%|████████▊ | 10531/12000 [6:12:16<50:36,  2.07s/it]
 88%|████████▊ | 10532/12000 [6:12:18<53:07,  2.17s/it]
                                                       
{'loss': '1.04', 'grad_norm': '0.531', 'learning_rate': '8.601e-06', 'entropy': '0.9777', 'num_tokens': '5.643e+07', 'mean_token_accuracy': '0.7643', 'epoch': '0.09279'}
{'loss': '0.9475', 'grad_norm': '0.4306', 'learning_rate': '8.589e-06', 'entropy': '0.8228', 'num_tokens': '5.643e+07', 'mean_token_accuracy': '0.7868', 'epoch': '0.0928'}
{'loss': '1.055', 'grad_norm': '0.3949', 'learning_rate': '8.578e-06', 'entropy': '0.957', 'num_tokens': '5.644e+07', 'mean_token_accuracy': '0.7653', 'epoch': '0.09281'}
{'loss': '0.7206', 'grad_norm': '0.4137', 'learning_rate': '8.567e-06', 'entropy': '0.8086', 'num_tokens': '5.644e+07', 'mean_token_accuracy': '0.8034', 'epoch': '0.09282'}
{'loss': '0.9656', 'grad_norm': '0.4204', 'learning_rate': '8.556e-06', 'entropy': '0.9463', 'num_tokens': '5.644e+07', 'mean_token_accuracy': '0.7548', 'epoch': '0.09283'}
{'loss': '0.7422', 'grad_norm': '0.4306', 'learning_rate': '8.545e-06', 'entropy': '0.8495', 'num_tokens': '5.645e+07', 'mean_token_accuracy': '0.7945', 'epoch': '0.09283'}
{'loss': '0.596', 'grad_norm': '0.3546', 'learning_rate': '8.534e-06', 'entropy': '0.6591', 'num_tokens': '5.645e+07', 'mean_token_accuracy': '0.8351', 'epoch': '0.09284'}
{'loss': '0.7175', 'grad_norm': '0.361', 'learning_rate': '8.523e-06', 'entropy': '0.6657', 'num_tokens': '5.646e+07', 'mean_token_accuracy': '0.8289', 'epoch': '0.09285'}
{'loss': '0.7379', 'grad_norm': '0.3988', 'learning_rate': '8.511e-06', 'entropy': '0.7223', 'num_tokens': '5.646e+07', 'mean_token_accuracy': '0.8193', 'epoch': '0.09286'}
{'loss': '0.6754', 'grad_norm': '0.5071', 'learning_rate': '8.5e-06', 'entropy': '0.6542', 'num_tokens': '5.647e+07', 'mean_token_accuracy': '0.842', 'epoch': '0.09287'}
{'loss': '0.877', 'grad_norm': '0.4506', 'learning_rate': '8.489e-06', 'entropy': '0.8799', 'num_tokens': '5.647e+07', 'mean_token_accuracy': '0.7892', 'epoch': '0.09288'}
{'loss': '0.822', 'grad_norm': '0.4426', 'learning_rate': '8.478e-06', 'entropy': '0.863', 'num_tokens': '5.648e+07', 'mean_token_accuracy': '0.7884', 'epoch': '0.09289'}
{'loss': '0.5928', 'grad_norm': '0.4083', 'learning_rate': '8.467e-06', 'entropy': '0.7289', 'num_tokens': '5.648e+07', 'mean_token_accuracy': '0.821', 'epoch': '0.0929'}
{'loss': '0.8789', 'grad_norm': '0.476', 'learning_rate': '8.456e-06', 'entropy': '0.8991', 'num_tokens': '5.648e+07', 'mean_token_accuracy': '0.7756', 'epoch': '0.09291'}
{'loss': '0.6598', 'grad_norm': '0.3314', 'learning_rate': '8.445e-06', 'entropy': '0.7064', 'num_tokens': '5.649e+07', 'mean_token_accuracy': '0.8265', 'epoch': '0.09291'}
{'loss': '1.268', 'grad_norm': '0.4473', 'learning_rate': '8.434e-06', 'entropy': '1.123', 'num_tokens': '5.649e+07', 'mean_token_accuracy': '0.7429', 'epoch': '0.09292'}
{'loss': '1.028', 'grad_norm': '0.4086', 'learning_rate': '8.423e-06', 'entropy': '1.007', 'num_tokens': '5.65e+07', 'mean_token_accuracy': '0.7633', 'epoch': '0.09293'}
{'loss': '0.7411', 'grad_norm': '0.4677', 'learning_rate': '8.412e-06', 'entropy': '0.7373', 'num_tokens': '5.65e+07', 'mean_token_accuracy': '0.8021', 'epoch': '0.09294'}
{'loss': '0.8228', 'grad_norm': '0.4657', 'learning_rate': '8.401e-06', 'entropy': '0.751', 'num_tokens': '5.651e+07', 'mean_token_accuracy': '0.8108', 'epoch': '0.09295'}
{'loss': '0.8436', 'grad_norm': '0.3819', 'learning_rate': '8.389e-06', 'entropy': '0.8417', 'num_tokens': '5.651e+07', 'mean_token_accuracy': '0.7879', 'epoch': '0.09296'}
{'loss': '1.192', 'grad_norm': '0.43', 'learning_rate': '8.378e-06', 'entropy': '0.9846', 'num_tokens': '5.652e+07', 'mean_token_accuracy': '0.7625', 'epoch': '0.09297'}
{'loss': '0.8816', 'grad_norm': '0.4428', 'learning_rate': '8.367e-06', 'entropy': '0.8151', 'num_tokens': '5.652e+07', 'mean_token_accuracy': '0.7869', 'epoch': '0.09298'}
{'loss': '0.6372', 'grad_norm': '0.3821', 'learning_rate': '8.356e-06', 'entropy': '0.7795', 'num_tokens': '5.653e+07', 'mean_token_accuracy': '0.7862', 'epoch': '0.09299'}
{'loss': '0.8378', 'grad_norm': '0.4344', 'learning_rate': '8.345e-06', 'entropy': '0.7702', 'num_tokens': '5.654e+07', 'mean_token_accuracy': '0.805', 'epoch': '0.09299'}
{'loss': '0.7641', 'grad_norm': '0.3514', 'learning_rate': '8.334e-06', 'entropy': '0.7918', 'num_tokens': '5.654e+07', 'mean_token_accuracy': '0.7837', 'epoch': '0.093'}
{'loss': '0.6589', 'grad_norm': '0.5107', 'learning_rate': '8.323e-06', 'entropy': '0.7486', 'num_tokens': '5.654e+07', 'mean_token_accuracy': '0.8022', 'epoch': '0.09301'}
{'loss': '0.974', 'grad_norm': '0.4681', 'learning_rate': '8.312e-06', 'entropy': '0.8593', 'num_tokens': '5.655e+07', 'mean_token_accuracy': '0.7747', 'epoch': '0.09302'}
{'loss': '0.7033', 'grad_norm': '0.3942', 'learning_rate': '8.301e-06', 'entropy': '0.6612', 'num_tokens': '5.655e+07', 'mean_token_accuracy': '0.8444', 'epoch': '0.09303'}
{'loss': '0.6303', 'grad_norm': '0.3549', 'learning_rate': '8.29e-06', 'entropy': '0.752', 'num_tokens': '5.656e+07', 'mean_token_accuracy': '0.8078', 'epoch': '0.09304'}
{'loss': '0.6719', 'grad_norm': '0.4318', 'learning_rate': '8.279e-06', 'entropy': '0.6437', 'num_tokens': '5.656e+07', 'mean_token_accuracy': '0.8333', 'epoch': '0.09305'}
{'loss': '0.6271', 'grad_norm': '0.4063', 'learning_rate': '8.268e-06', 'entropy': '0.6344', 'num_tokens': '5.657e+07', 'mean_token_accuracy': '0.8266', 'epoch': '0.09306'}
{'loss': '0.7466', 'grad_norm': '0.4831', 'learning_rate': '8.257e-06', 'entropy': '0.6901', 'num_tokens': '5.657e+07', 'mean_token_accuracy': '0.821', 'epoch': '0.09306'}
{'loss': '0.6358', 'grad_norm': '0.4327', 'learning_rate': '8.246e-06', 'entropy': '0.6482', 'num_tokens': '5.658e+07', 'mean_token_accuracy': '0.8283', 'epoch': '0.09307'}
{'loss': '0.6523', 'grad_norm': '0.679', 'learning_rate': '8.235e-06', 'entropy': '0.6417', 'num_tokens': '5.658e+07', 'mean_token_accuracy': '0.8202', 'epoch': '0.09308'}
{'loss': '0.8857', 'grad_norm': '0.4651', 'learning_rate': '8.225e-06', 'entropy': '0.8243', 'num_tokens': '5.659e+07', 'mean_token_accuracy': '0.798', 'epoch': '0.09309'}
{'loss': '0.8803', 'grad_norm': '0.4168', 'learning_rate': '8.214e-06', 'entropy': '0.9067', 'num_tokens': '5.66e+07', 'mean_token_accuracy': '0.7717', 'epoch': '0.0931'}
{'loss': '0.925', 'grad_norm': '0.4463', 'learning_rate': '8.203e-06', 'entropy': '0.9119', 'num_tokens': '5.66e+07', 'mean_token_accuracy': '0.7577', 'epoch': '0.09311'}
{'loss': '0.8655', 'grad_norm': '0.4299', 'learning_rate': '8.192e-06', 'entropy': '0.8196', 'num_tokens': '5.661e+07', 'mean_token_accuracy': '0.7884', 'epoch': '0.09312'}
{'loss': '0.971', 'grad_norm': '0.4783', 'learning_rate': '8.181e-06', 'entropy': '0.9774', 'num_tokens': '5.661e+07', 'mean_token_accuracy': '0.7585', 'epoch': '0.09313'}
{'loss': '0.8122', 'grad_norm': '0.3478', 'learning_rate': '8.17e-06', 'entropy': '0.9174', 'num_tokens': '5.662e+07', 'mean_token_accuracy': '0.7688', 'epoch': '0.09314'}
{'loss': '0.6681', 'grad_norm': '0.3361', 'learning_rate': '8.159e-06', 'entropy': '0.6838', 'num_tokens': '5.662e+07', 'mean_token_accuracy': '0.8273', 'epoch': '0.09314'}
{'loss': '0.9383', 'grad_norm': '0.4268', 'learning_rate': '8.148e-06', 'entropy': '0.8479', 'num_tokens': '5.663e+07', 'mean_token_accuracy': '0.7995', 'epoch': '0.09315'}
{'loss': '0.8006', 'grad_norm': '0.395', 'learning_rate': '8.137e-06', 'entropy': '0.905', 'num_tokens': '5.663e+07', 'mean_token_accuracy': '0.7688', 'epoch': '0.09316'}
{'loss': '0.7381', 'grad_norm': '0.4157', 'learning_rate': '8.126e-06', 'entropy': '0.809', 'num_tokens': '5.664e+07', 'mean_token_accuracy': '0.7862', 'epoch': '0.09317'}
{'loss': '0.7273', 'grad_norm': '0.4011', 'learning_rate': '8.115e-06', 'entropy': '0.7348', 'num_tokens': '5.664e+07', 'mean_token_accuracy': '0.8094', 'epoch': '0.09318'}
{'loss': '0.85', 'grad_norm': '0.4918', 'learning_rate': '8.105e-06', 'entropy': '0.8769', 'num_tokens': '5.665e+07', 'mean_token_accuracy': '0.7592', 'epoch': '0.09319'}
{'loss': '0.5588', 'grad_norm': '0.3577', 'learning_rate': '8.094e-06', 'entropy': '0.5938', 'num_tokens': '5.665e+07', 'mean_token_accuracy': '0.85', 'epoch': '0.0932'}

 88%|████████▊ | 10532/12000 [6:12:18<53:07,  2.17s/it]
 88%|████████▊ | 10533/12000 [6:12:20<51:30,  2.11s/it]
                                                       

 88%|████████▊ | 10533/12000 [6:12:20<51:30,  2.11s/it]
 88%|████████▊ | 10534/12000 [6:12:22<47:45,  1.95s/it]
                                                       

 88%|████████▊ | 10534/12000 [6:12:22<47:45,  1.95s/it]
 88%|████████▊ | 10535/12000 [6:12:23<45:12,  1.85s/it]
                                                       

 88%|████████▊ | 10535/12000 [6:12:23<45:12,  1.85s/it]
 88%|████████▊ | 10536/12000 [6:12:25<42:46,  1.75s/it]
                                                       

 88%|████████▊ | 10536/12000 [6:12:25<42:46,  1.75s/it]
 88%|████████▊ | 10537/12000 [6:12:27<42:41,  1.75s/it]
                                                       

 88%|████████▊ | 10537/12000 [6:12:27<42:41,  1.75s/it]
 88%|████████▊ | 10538/12000 [6:12:29<44:54,  1.84s/it]
                                                       

 88%|████████▊ | 10538/12000 [6:12:29<44:54,  1.84s/it]
 88%|████████▊ | 10539/12000 [6:12:31<49:09,  2.02s/it]
                                                       

 88%|████████▊ | 10539/12000 [6:12:31<49:09,  2.02s/it]
 88%|████████▊ | 10540/12000 [6:12:33<47:35,  1.96s/it]
                                                       

 88%|████████▊ | 10540/12000 [6:12:33<47:35,  1.96s/it]
 88%|████████▊ | 10541/12000 [6:12:34<44:39,  1.84s/it]
                                                       

 88%|████████▊ | 10541/12000 [6:12:34<44:39,  1.84s/it]
 88%|████████▊ | 10542/12000 [6:12:37<53:05,  2.18s/it]
                                                       

 88%|████████▊ | 10542/12000 [6:12:37<53:05,  2.18s/it]
 88%|████████▊ | 10543/12000 [6:12:39<51:39,  2.13s/it]
                                                       

 88%|████████▊ | 10543/12000 [6:12:39<51:39,  2.13s/it]
 88%|████████▊ | 10544/12000 [6:12:41<47:11,  1.94s/it]
                                                       

 88%|████████▊ | 10544/12000 [6:12:41<47:11,  1.94s/it]
 88%|████████▊ | 10545/12000 [6:12:45<1:00:35,  2.50s/it]
                                                         

 88%|████████▊ | 10545/12000 [6:12:45<1:00:35,  2.50s/it]
 88%|████████▊ | 10546/12000 [6:12:49<1:16:06,  3.14s/it]
                                                         

 88%|████████▊ | 10546/12000 [6:12:49<1:16:06,  3.14s/it]
 88%|████████▊ | 10547/12000 [6:12:52<1:11:35,  2.96s/it]
                                                         

 88%|████████▊ | 10547/12000 [6:12:52<1:11:35,  2.96s/it]
 88%|████████▊ | 10548/12000 [6:12:55<1:11:47,  2.97s/it]
                                                         

 88%|████████▊ | 10548/12000 [6:12:55<1:11:47,  2.97s/it]
 88%|████████▊ | 10549/12000 [6:12:57<1:06:02,  2.73s/it]
                                                         

 88%|████████▊ | 10549/12000 [6:12:57<1:06:02,  2.73s/it]
 88%|████████▊ | 10550/12000 [6:13:00<1:05:06,  2.69s/it]
                                                         

 88%|████████▊ | 10550/12000 [6:13:00<1:05:06,  2.69s/it]
 88%|████████▊ | 10551/12000 [6:13:03<1:07:23,  2.79s/it]
                                                         

 88%|████████▊ | 10551/12000 [6:13:03<1:07:23,  2.79s/it]
 88%|████████▊ | 10552/12000 [6:13:05<1:06:20,  2.75s/it]
                                                         

 88%|████████▊ | 10552/12000 [6:13:05<1:06:20,  2.75s/it]
 88%|████████▊ | 10553/12000 [6:13:07<58:22,  2.42s/it]  
                                                       

 88%|████████▊ | 10553/12000 [6:13:07<58:22,  2.42s/it]
 88%|████████▊ | 10554/12000 [6:13:08<51:18,  2.13s/it]
                                                       

 88%|████████▊ | 10554/12000 [6:13:08<51:18,  2.13s/it]
 88%|████████▊ | 10555/12000 [6:13:10<48:57,  2.03s/it]
                                                       

 88%|████████▊ | 10555/12000 [6:13:10<48:57,  2.03s/it]
 88%|████████▊ | 10556/12000 [6:13:12<45:10,  1.88s/it]
                                                       

 88%|████████▊ | 10556/12000 [6:13:12<45:10,  1.88s/it]
 88%|████████▊ | 10557/12000 [6:13:14<44:36,  1.85s/it]
                                                       

 88%|████████▊ | 10557/12000 [6:13:14<44:36,  1.85s/it]
 88%|████████▊ | 10558/12000 [6:13:16<51:19,  2.14s/it]
                                                       

 88%|████████▊ | 10558/12000 [6:13:16<51:19,  2.14s/it]
 88%|████████▊ | 10559/12000 [6:13:18<48:29,  2.02s/it]
                                                       

 88%|████████▊ | 10559/12000 [6:13:18<48:29,  2.02s/it]
 88%|████████▊ | 10560/12000 [6:13:21<51:37,  2.15s/it]
                                                       

 88%|████████▊ | 10560/12000 [6:13:21<51:37,  2.15s/it]
 88%|████████▊ | 10561/12000 [6:13:22<47:00,  1.96s/it]
                                                       

 88%|████████▊ | 10561/12000 [6:13:22<47:00,  1.96s/it]
 88%|████████▊ | 10562/12000 [6:13:24<43:23,  1.81s/it]
                                                       

 88%|████████▊ | 10562/12000 [6:13:24<43:23,  1.81s/it]
 88%|████████▊ | 10563/12000 [6:13:26<46:41,  1.95s/it]
                                                       

 88%|████████▊ | 10563/12000 [6:13:26<46:41,  1.95s/it]
 88%|████████▊ | 10564/12000 [6:13:28<47:59,  2.01s/it]
                                                       

 88%|████████▊ | 10564/12000 [6:13:28<47:59,  2.01s/it]
 88%|████████▊ | 10565/12000 [6:13:31<53:30,  2.24s/it]
                                                       

 88%|████████▊ | 10565/12000 [6:13:31<53:30,  2.24s/it]
 88%|████████▊ | 10566/12000 [6:13:33<53:18,  2.23s/it]
                                                       

 88%|████████▊ | 10566/12000 [6:13:33<53:18,  2.23s/it]
 88%|████████▊ | 10567/12000 [6:13:36<1:01:39,  2.58s/it]
                                                         

 88%|████████▊ | 10567/12000 [6:13:36<1:01:39,  2.58s/it]
 88%|████████▊ | 10568/12000 [6:13:38<55:01,  2.31s/it]  
                                                       

 88%|████████▊ | 10568/12000 [6:13:38<55:01,  2.31s/it]
 88%|████████▊ | 10569/12000 [6:13:39<48:09,  2.02s/it]
                                                       

 88%|████████▊ | 10569/12000 [6:13:39<48:09,  2.02s/it]
 88%|████████▊ | 10570/12000 [6:13:41<45:48,  1.92s/it]
                                                       

 88%|████████▊ | 10570/12000 [6:13:41<45:48,  1.92s/it]
 88%|████████▊ | 10571/12000 [6:13:43<45:51,  1.93s/it]
                                                       

 88%|████████▊ | 10571/12000 [6:13:43<45:51,  1.93s/it]
 88%|████████▊ | 10572/12000 [6:13:45<43:56,  1.85s/it]
                                                       

 88%|████████▊ | 10572/12000 [6:13:45<43:56,  1.85s/it]
 88%|████████▊ | 10573/12000 [6:13:46<42:04,  1.77s/it]
                                                       

 88%|████████▊ | 10573/12000 [6:13:46<42:04,  1.77s/it]
 88%|████████▊ | 10574/12000 [6:13:48<45:16,  1.90s/it]
                                                       

 88%|████████▊ | 10574/12000 [6:13:48<45:16,  1.90s/it]
 88%|████████▊ | 10575/12000 [6:13:50<43:51,  1.85s/it]
                                                       

 88%|████████▊ | 10575/12000 [6:13:50<43:51,  1.85s/it]
 88%|████████▊ | 10576/12000 [6:13:52<41:26,  1.75s/it]
                                                       

 88%|████████▊ | 10576/12000 [6:13:52<41:26,  1.75s/it]
 88%|████████▊ | 10577/12000 [6:13:54<44:22,  1.87s/it]
                                                       

 88%|████████▊ | 10577/12000 [6:13:54<44:22,  1.87s/it]
 88%|████████▊ | 10578/12000 [6:13:55<41:23,  1.75s/it]
                                                       

 88%|████████▊ | 10578/12000 [6:13:55<41:23,  1.75s/it]
 88%|████████▊ | 10579/12000 [6:13:57<41:49,  1.77s/it]
                                                       
{'loss': '0.868', 'grad_norm': '0.3851', 'learning_rate': '8.083e-06', 'entropy': '0.7911', 'num_tokens': '5.666e+07', 'mean_token_accuracy': '0.7976', 'epoch': '0.09321'}
{'loss': '0.6545', 'grad_norm': '0.4352', 'learning_rate': '8.072e-06', 'entropy': '0.6815', 'num_tokens': '5.667e+07', 'mean_token_accuracy': '0.8229', 'epoch': '0.09322'}
{'loss': '0.929', 'grad_norm': '0.4497', 'learning_rate': '8.061e-06', 'entropy': '0.947', 'num_tokens': '5.667e+07', 'mean_token_accuracy': '0.7622', 'epoch': '0.09322'}
{'loss': '0.7295', 'grad_norm': '0.3647', 'learning_rate': '8.05e-06', 'entropy': '0.8004', 'num_tokens': '5.668e+07', 'mean_token_accuracy': '0.8108', 'epoch': '0.09323'}
{'loss': '0.5612', 'grad_norm': '0.4599', 'learning_rate': '8.039e-06', 'entropy': '0.5842', 'num_tokens': '5.668e+07', 'mean_token_accuracy': '0.8496', 'epoch': '0.09324'}
{'loss': '0.7861', 'grad_norm': '0.3984', 'learning_rate': '8.029e-06', 'entropy': '0.7805', 'num_tokens': '5.669e+07', 'mean_token_accuracy': '0.7938', 'epoch': '0.09325'}
{'loss': '1.091', 'grad_norm': '0.4343', 'learning_rate': '8.018e-06', 'entropy': '1.049', 'num_tokens': '5.669e+07', 'mean_token_accuracy': '0.7419', 'epoch': '0.09326'}
{'loss': '0.8011', 'grad_norm': '0.3496', 'learning_rate': '8.007e-06', 'entropy': '0.8014', 'num_tokens': '5.67e+07', 'mean_token_accuracy': '0.7961', 'epoch': '0.09327'}
{'loss': '0.5603', 'grad_norm': '0.4532', 'learning_rate': '7.996e-06', 'entropy': '0.5451', 'num_tokens': '5.67e+07', 'mean_token_accuracy': '0.8519', 'epoch': '0.09328'}
{'loss': '1.045', 'grad_norm': '0.4732', 'learning_rate': '7.985e-06', 'entropy': '0.9523', 'num_tokens': '5.671e+07', 'mean_token_accuracy': '0.7645', 'epoch': '0.09329'}
{'loss': '0.8124', 'grad_norm': '0.4406', 'learning_rate': '7.975e-06', 'entropy': '0.8252', 'num_tokens': '5.671e+07', 'mean_token_accuracy': '0.7874', 'epoch': '0.09329'}
{'loss': '0.8267', 'grad_norm': '0.3631', 'learning_rate': '7.964e-06', 'entropy': '0.8554', 'num_tokens': '5.672e+07', 'mean_token_accuracy': '0.7857', 'epoch': '0.0933'}
{'loss': '1.036', 'grad_norm': '0.526', 'learning_rate': '7.953e-06', 'entropy': '1.035', 'num_tokens': '5.672e+07', 'mean_token_accuracy': '0.7473', 'epoch': '0.09331'}
{'loss': '1.302', 'grad_norm': '0.5428', 'learning_rate': '7.942e-06', 'entropy': '1.195', 'num_tokens': '5.673e+07', 'mean_token_accuracy': '0.7078', 'epoch': '0.09332'}
{'loss': '1.091', 'grad_norm': '0.2838', 'learning_rate': '7.932e-06', 'entropy': '0.8615', 'num_tokens': '5.674e+07', 'mean_token_accuracy': '0.784', 'epoch': '0.09333'}
{'loss': '0.9655', 'grad_norm': '0.3891', 'learning_rate': '7.921e-06', 'entropy': '0.8292', 'num_tokens': '5.675e+07', 'mean_token_accuracy': '0.7942', 'epoch': '0.09334'}
{'loss': '1.085', 'grad_norm': '0.4561', 'learning_rate': '7.91e-06', 'entropy': '0.9911', 'num_tokens': '5.675e+07', 'mean_token_accuracy': '0.7599', 'epoch': '0.09335'}
{'loss': '0.9815', 'grad_norm': '0.4083', 'learning_rate': '7.899e-06', 'entropy': '0.9295', 'num_tokens': '5.676e+07', 'mean_token_accuracy': '0.773', 'epoch': '0.09336'}
{'loss': '0.8611', 'grad_norm': '0.3271', 'learning_rate': '7.889e-06', 'entropy': '1.023', 'num_tokens': '5.677e+07', 'mean_token_accuracy': '0.754', 'epoch': '0.09337'}
{'loss': '0.6104', 'grad_norm': '0.342', 'learning_rate': '7.878e-06', 'entropy': '0.6016', 'num_tokens': '5.677e+07', 'mean_token_accuracy': '0.8472', 'epoch': '0.09337'}
{'loss': '0.6495', 'grad_norm': '0.3653', 'learning_rate': '7.867e-06', 'entropy': '0.6158', 'num_tokens': '5.678e+07', 'mean_token_accuracy': '0.8367', 'epoch': '0.09338'}
{'loss': '0.6042', 'grad_norm': '0.4255', 'learning_rate': '7.856e-06', 'entropy': '0.6413', 'num_tokens': '5.678e+07', 'mean_token_accuracy': '0.8305', 'epoch': '0.09339'}
{'loss': '1.049', 'grad_norm': '0.4531', 'learning_rate': '7.846e-06', 'entropy': '1.119', 'num_tokens': '5.679e+07', 'mean_token_accuracy': '0.7416', 'epoch': '0.0934'}
{'loss': '0.5603', 'grad_norm': '0.3485', 'learning_rate': '7.835e-06', 'entropy': '0.5896', 'num_tokens': '5.679e+07', 'mean_token_accuracy': '0.8513', 'epoch': '0.09341'}
{'loss': '0.5885', 'grad_norm': '0.3869', 'learning_rate': '7.824e-06', 'entropy': '0.5967', 'num_tokens': '5.68e+07', 'mean_token_accuracy': '0.8399', 'epoch': '0.09342'}
{'loss': '0.7632', 'grad_norm': '0.3027', 'learning_rate': '7.814e-06', 'entropy': '0.8058', 'num_tokens': '5.68e+07', 'mean_token_accuracy': '0.797', 'epoch': '0.09343'}
{'loss': '0.6644', 'grad_norm': '0.3253', 'learning_rate': '7.803e-06', 'entropy': '0.816', 'num_tokens': '5.681e+07', 'mean_token_accuracy': '0.8016', 'epoch': '0.09344'}
{'loss': '1.002', 'grad_norm': '0.3686', 'learning_rate': '7.792e-06', 'entropy': '0.9256', 'num_tokens': '5.681e+07', 'mean_token_accuracy': '0.7682', 'epoch': '0.09345'}
{'loss': '1.078', 'grad_norm': '0.4278', 'learning_rate': '7.782e-06', 'entropy': '1.077', 'num_tokens': '5.682e+07', 'mean_token_accuracy': '0.7575', 'epoch': '0.09345'}
{'loss': '0.6743', 'grad_norm': '0.3724', 'learning_rate': '7.771e-06', 'entropy': '0.687', 'num_tokens': '5.683e+07', 'mean_token_accuracy': '0.823', 'epoch': '0.09346'}
{'loss': '0.5079', 'grad_norm': '0.3901', 'learning_rate': '7.76e-06', 'entropy': '0.5239', 'num_tokens': '5.683e+07', 'mean_token_accuracy': '0.8749', 'epoch': '0.09347'}
{'loss': '0.9739', 'grad_norm': '0.3944', 'learning_rate': '7.75e-06', 'entropy': '0.9277', 'num_tokens': '5.684e+07', 'mean_token_accuracy': '0.775', 'epoch': '0.09348'}
{'loss': '0.6565', 'grad_norm': '0.3978', 'learning_rate': '7.739e-06', 'entropy': '0.7826', 'num_tokens': '5.684e+07', 'mean_token_accuracy': '0.7978', 'epoch': '0.09349'}
{'loss': '0.831', 'grad_norm': '0.3369', 'learning_rate': '7.728e-06', 'entropy': '0.7815', 'num_tokens': '5.685e+07', 'mean_token_accuracy': '0.8091', 'epoch': '0.0935'}
{'loss': '1.026', 'grad_norm': '0.3981', 'learning_rate': '7.718e-06', 'entropy': '0.9758', 'num_tokens': '5.685e+07', 'mean_token_accuracy': '0.7553', 'epoch': '0.09351'}
{'loss': '0.693', 'grad_norm': '0.3801', 'learning_rate': '7.707e-06', 'entropy': '0.6876', 'num_tokens': '5.686e+07', 'mean_token_accuracy': '0.8235', 'epoch': '0.09352'}
{'loss': '1.007', 'grad_norm': '0.453', 'learning_rate': '7.697e-06', 'entropy': '1.029', 'num_tokens': '5.686e+07', 'mean_token_accuracy': '0.7494', 'epoch': '0.09353'}
{'loss': '0.6516', 'grad_norm': '0.3738', 'learning_rate': '7.686e-06', 'entropy': '0.6593', 'num_tokens': '5.687e+07', 'mean_token_accuracy': '0.8378', 'epoch': '0.09353'}
{'loss': '0.5578', 'grad_norm': '0.3407', 'learning_rate': '7.675e-06', 'entropy': '0.5668', 'num_tokens': '5.687e+07', 'mean_token_accuracy': '0.8473', 'epoch': '0.09354'}
{'loss': '1.269', 'grad_norm': '0.4257', 'learning_rate': '7.665e-06', 'entropy': '1.177', 'num_tokens': '5.688e+07', 'mean_token_accuracy': '0.7216', 'epoch': '0.09355'}
{'loss': '0.6369', 'grad_norm': '0.4757', 'learning_rate': '7.654e-06', 'entropy': '0.6509', 'num_tokens': '5.688e+07', 'mean_token_accuracy': '0.8241', 'epoch': '0.09356'}
{'loss': '0.985', 'grad_norm': '0.4843', 'learning_rate': '7.644e-06', 'entropy': '0.9801', 'num_tokens': '5.689e+07', 'mean_token_accuracy': '0.7346', 'epoch': '0.09357'}
{'loss': '0.8703', 'grad_norm': '0.4253', 'learning_rate': '7.633e-06', 'entropy': '0.8396', 'num_tokens': '5.689e+07', 'mean_token_accuracy': '0.7922', 'epoch': '0.09358'}
{'loss': '0.5849', 'grad_norm': '0.3157', 'learning_rate': '7.623e-06', 'entropy': '0.5938', 'num_tokens': '5.69e+07', 'mean_token_accuracy': '0.8381', 'epoch': '0.09359'}
{'loss': '0.9736', 'grad_norm': '0.4305', 'learning_rate': '7.612e-06', 'entropy': '1.008', 'num_tokens': '5.69e+07', 'mean_token_accuracy': '0.7567', 'epoch': '0.0936'}
{'loss': '0.7242', 'grad_norm': '0.3695', 'learning_rate': '7.602e-06', 'entropy': '0.867', 'num_tokens': '5.691e+07', 'mean_token_accuracy': '0.7981', 'epoch': '0.0936'}
{'loss': '0.5979', 'grad_norm': '0.3726', 'learning_rate': '7.591e-06', 'entropy': '0.6923', 'num_tokens': '5.691e+07', 'mean_token_accuracy': '0.8313', 'epoch': '0.09361'}

 88%|████████▊ | 10579/12000 [6:13:57<41:49,  1.77s/it]
 88%|████████▊ | 10580/12000 [6:13:59<41:10,  1.74s/it]
                                                       

 88%|████████▊ | 10580/12000 [6:13:59<41:10,  1.74s/it]
 88%|████████▊ | 10581/12000 [6:14:01<41:31,  1.76s/it]
                                                       

 88%|████████▊ | 10581/12000 [6:14:01<41:31,  1.76s/it]
 88%|████████▊ | 10582/12000 [6:14:02<39:55,  1.69s/it]
                                                       

 88%|████████▊ | 10582/12000 [6:14:02<39:55,  1.69s/it]
 88%|████████▊ | 10583/12000 [6:14:04<39:20,  1.67s/it]
                                                       

 88%|████████▊ | 10583/12000 [6:14:04<39:20,  1.67s/it]
 88%|████████▊ | 10584/12000 [6:14:06<39:54,  1.69s/it]
                                                       

 88%|████████▊ | 10584/12000 [6:14:06<39:54,  1.69s/it]
 88%|████████▊ | 10585/12000 [6:14:07<37:55,  1.61s/it]
                                                       

 88%|████████▊ | 10585/12000 [6:14:07<37:55,  1.61s/it]
 88%|████████▊ | 10586/12000 [6:14:10<50:14,  2.13s/it]
                                                       

 88%|████████▊ | 10586/12000 [6:14:10<50:14,  2.13s/it]
 88%|████████▊ | 10587/12000 [6:14:12<45:06,  1.92s/it]
                                                       

 88%|████████▊ | 10587/12000 [6:14:12<45:06,  1.92s/it]
 88%|████████▊ | 10588/12000 [6:14:13<43:14,  1.84s/it]
                                                       

 88%|████████▊ | 10588/12000 [6:14:13<43:14,  1.84s/it]
 88%|████████▊ | 10589/12000 [6:14:18<1:03:42,  2.71s/it]
                                                         

 88%|████████▊ | 10589/12000 [6:14:18<1:03:42,  2.71s/it]
 88%|████████▊ | 10590/12000 [6:14:20<1:00:00,  2.55s/it]
                                                         

 88%|████████▊ | 10590/12000 [6:14:20<1:00:00,  2.55s/it]
 88%|████████▊ | 10591/12000 [6:14:22<56:36,  2.41s/it]  
                                                       

 88%|████████▊ | 10591/12000 [6:14:22<56:36,  2.41s/it]
 88%|████████▊ | 10592/12000 [6:14:25<56:52,  2.42s/it]
                                                       

 88%|████████▊ | 10592/12000 [6:14:25<56:52,  2.42s/it]
 88%|████████▊ | 10593/12000 [6:14:28<1:02:04,  2.65s/it]
                                                         

 88%|████████▊ | 10593/12000 [6:14:28<1:02:04,  2.65s/it]
 88%|████████▊ | 10594/12000 [6:14:29<53:34,  2.29s/it]  
                                                       

 88%|████████▊ | 10594/12000 [6:14:29<53:34,  2.29s/it]
 88%|████████▊ | 10595/12000 [6:14:31<48:23,  2.07s/it]
                                                       

 88%|████████▊ | 10595/12000 [6:14:31<48:23,  2.07s/it]
 88%|████████▊ | 10596/12000 [6:14:34<52:01,  2.22s/it]
                                                       

 88%|████████▊ | 10596/12000 [6:14:34<52:01,  2.22s/it]
 88%|████████▊ | 10597/12000 [6:14:36<54:32,  2.33s/it]
                                                       

 88%|████████▊ | 10597/12000 [6:14:36<54:32,  2.33s/it]
 88%|████████▊ | 10598/12000 [6:14:38<48:39,  2.08s/it]
                                                       

 88%|████████▊ | 10598/12000 [6:14:38<48:39,  2.08s/it]
 88%|████████▊ | 10599/12000 [6:14:39<45:57,  1.97s/it]
                                                       

 88%|████████▊ | 10599/12000 [6:14:39<45:57,  1.97s/it]
 88%|████████▊ | 10600/12000 [6:14:42<48:10,  2.06s/it]
                                                       

 88%|████████▊ | 10600/12000 [6:14:42<48:10,  2.06s/it]
 88%|████████▊ | 10601/12000 [6:14:45<55:21,  2.37s/it]
                                                       

 88%|████████▊ | 10601/12000 [6:14:45<55:21,  2.37s/it]
 88%|████████▊ | 10602/12000 [6:14:46<49:21,  2.12s/it]
                                                       

 88%|████████▊ | 10602/12000 [6:14:46<49:21,  2.12s/it]
 88%|████████▊ | 10603/12000 [6:14:48<44:39,  1.92s/it]
                                                       

 88%|████████▊ | 10603/12000 [6:14:48<44:39,  1.92s/it]
 88%|████████▊ | 10604/12000 [6:14:52<58:53,  2.53s/it]
                                                       

 88%|████████▊ | 10604/12000 [6:14:52<58:53,  2.53s/it]
 88%|████████▊ | 10605/12000 [6:14:55<1:02:01,  2.67s/it]
                                                         

 88%|████████▊ | 10605/12000 [6:14:55<1:02:01,  2.67s/it]
 88%|████████▊ | 10606/12000 [6:14:56<53:55,  2.32s/it]  
                                                       

 88%|████████▊ | 10606/12000 [6:14:56<53:55,  2.32s/it]
 88%|████████▊ | 10607/12000 [6:14:58<50:24,  2.17s/it]
                                                       

 88%|████████▊ | 10607/12000 [6:14:58<50:24,  2.17s/it]
 88%|████████▊ | 10608/12000 [6:14:59<45:20,  1.95s/it]
                                                       

 88%|████████▊ | 10608/12000 [6:14:59<45:20,  1.95s/it]
 88%|████████▊ | 10609/12000 [6:15:01<45:10,  1.95s/it]
                                                       

 88%|████████▊ | 10609/12000 [6:15:01<45:10,  1.95s/it]
 88%|████████▊ | 10610/12000 [6:15:03<43:27,  1.88s/it]
                                                       

 88%|████████▊ | 10610/12000 [6:15:03<43:27,  1.88s/it]
 88%|████████▊ | 10611/12000 [6:15:08<1:02:19,  2.69s/it]
                                                         

 88%|████████▊ | 10611/12000 [6:15:08<1:02:19,  2.69s/it]
 88%|████████▊ | 10612/12000 [6:15:10<58:28,  2.53s/it]  
                                                       

 88%|████████▊ | 10612/12000 [6:15:10<58:28,  2.53s/it]
 88%|████████▊ | 10613/12000 [6:15:11<52:23,  2.27s/it]
                                                       

 88%|████████▊ | 10613/12000 [6:15:11<52:23,  2.27s/it]
 88%|████████▊ | 10614/12000 [6:15:13<47:00,  2.03s/it]
                                                       

 88%|████████▊ | 10614/12000 [6:15:13<47:00,  2.03s/it]
 88%|████████▊ | 10615/12000 [6:15:15<44:56,  1.95s/it]
                                                       

 88%|████████▊ | 10615/12000 [6:15:15<44:56,  1.95s/it]
 88%|████████▊ | 10616/12000 [6:15:16<41:51,  1.81s/it]
                                                       

 88%|████████▊ | 10616/12000 [6:15:16<41:51,  1.81s/it]
 88%|████████▊ | 10617/12000 [6:15:19<47:39,  2.07s/it]
                                                       

 88%|████████▊ | 10617/12000 [6:15:19<47:39,  2.07s/it]
 88%|████████▊ | 10618/12000 [6:15:21<48:29,  2.11s/it]
                                                       

 88%|████████▊ | 10618/12000 [6:15:21<48:29,  2.11s/it]
 88%|████████▊ | 10619/12000 [6:15:23<46:04,  2.00s/it]
                                                       

 88%|████████▊ | 10619/12000 [6:15:23<46:04,  2.00s/it]
 88%|████████▊ | 10620/12000 [6:15:26<50:36,  2.20s/it]
                                                       

 88%|████████▊ | 10620/12000 [6:15:26<50:36,  2.20s/it]
 89%|████████▊ | 10621/12000 [6:15:27<45:32,  1.98s/it]
                                                       

 89%|████████▊ | 10621/12000 [6:15:27<45:32,  1.98s/it]
 89%|████████▊ | 10622/12000 [6:15:28<42:12,  1.84s/it]
                                                       

 89%|████████▊ | 10622/12000 [6:15:28<42:12,  1.84s/it]
 89%|████████▊ | 10623/12000 [6:15:30<40:41,  1.77s/it]
                                                       

 89%|████████▊ | 10623/12000 [6:15:30<40:41,  1.77s/it]
 89%|████████▊ | 10624/12000 [6:15:32<38:47,  1.69s/it]
                                                       

 89%|████████▊ | 10624/12000 [6:15:32<38:47,  1.69s/it]
 89%|████████▊ | 10625/12000 [6:15:33<37:43,  1.65s/it]
                                                       

 89%|████████▊ | 10625/12000 [6:15:33<37:43,  1.65s/it]
 89%|████████▊ | 10626/12000 [6:15:35<36:12,  1.58s/it]
                                                       
{'loss': '0.5194', 'grad_norm': '0.3846', 'learning_rate': '7.58e-06', 'entropy': '0.5435', 'num_tokens': '5.692e+07', 'mean_token_accuracy': '0.8578', 'epoch': '0.09362'}
{'loss': '0.8519', 'grad_norm': '0.4646', 'learning_rate': '7.57e-06', 'entropy': '0.8176', 'num_tokens': '5.692e+07', 'mean_token_accuracy': '0.7869', 'epoch': '0.09363'}
{'loss': '0.5474', 'grad_norm': '0.3972', 'learning_rate': '7.559e-06', 'entropy': '0.5325', 'num_tokens': '5.693e+07', 'mean_token_accuracy': '0.8587', 'epoch': '0.09364'}
{'loss': '0.6001', 'grad_norm': '0.3779', 'learning_rate': '7.549e-06', 'entropy': '0.6751', 'num_tokens': '5.693e+07', 'mean_token_accuracy': '0.8185', 'epoch': '0.09365'}
{'loss': '0.8555', 'grad_norm': '0.442', 'learning_rate': '7.538e-06', 'entropy': '0.8867', 'num_tokens': '5.694e+07', 'mean_token_accuracy': '0.7785', 'epoch': '0.09366'}
{'loss': '0.8101', 'grad_norm': '0.4071', 'learning_rate': '7.528e-06', 'entropy': '0.8084', 'num_tokens': '5.694e+07', 'mean_token_accuracy': '0.796', 'epoch': '0.09367'}
{'loss': '0.6006', 'grad_norm': '0.3545', 'learning_rate': '7.517e-06', 'entropy': '0.6212', 'num_tokens': '5.695e+07', 'mean_token_accuracy': '0.8338', 'epoch': '0.09368'}
{'loss': '0.7068', 'grad_norm': '0.3282', 'learning_rate': '7.507e-06', 'entropy': '0.7335', 'num_tokens': '5.696e+07', 'mean_token_accuracy': '0.8011', 'epoch': '0.09368'}
{'loss': '0.5292', 'grad_norm': '0.3961', 'learning_rate': '7.496e-06', 'entropy': '0.5286', 'num_tokens': '5.696e+07', 'mean_token_accuracy': '0.8608', 'epoch': '0.09369'}
{'loss': '0.7103', 'grad_norm': '0.3526', 'learning_rate': '7.486e-06', 'entropy': '0.7678', 'num_tokens': '5.696e+07', 'mean_token_accuracy': '0.8008', 'epoch': '0.0937'}
{'loss': '0.8172', 'grad_norm': '0.3425', 'learning_rate': '7.476e-06', 'entropy': '0.7517', 'num_tokens': '5.697e+07', 'mean_token_accuracy': '0.7938', 'epoch': '0.09371'}
{'loss': '1.047', 'grad_norm': '0.353', 'learning_rate': '7.465e-06', 'entropy': '0.9576', 'num_tokens': '5.698e+07', 'mean_token_accuracy': '0.7754', 'epoch': '0.09372'}
{'loss': '0.7757', 'grad_norm': '0.3971', 'learning_rate': '7.455e-06', 'entropy': '0.7753', 'num_tokens': '5.698e+07', 'mean_token_accuracy': '0.806', 'epoch': '0.09373'}
{'loss': '0.573', 'grad_norm': '0.3701', 'learning_rate': '7.444e-06', 'entropy': '0.5903', 'num_tokens': '5.699e+07', 'mean_token_accuracy': '0.85', 'epoch': '0.09374'}
{'loss': '1.098', 'grad_norm': '0.5483', 'learning_rate': '7.434e-06', 'entropy': '0.9366', 'num_tokens': '5.7e+07', 'mean_token_accuracy': '0.7632', 'epoch': '0.09375'}
{'loss': '0.8326', 'grad_norm': '0.5218', 'learning_rate': '7.423e-06', 'entropy': '0.8526', 'num_tokens': '5.7e+07', 'mean_token_accuracy': '0.7977', 'epoch': '0.09376'}
{'loss': '0.8229', 'grad_norm': '0.4162', 'learning_rate': '7.413e-06', 'entropy': '0.8207', 'num_tokens': '5.701e+07', 'mean_token_accuracy': '0.7879', 'epoch': '0.09376'}
{'loss': '1.015', 'grad_norm': '0.4531', 'learning_rate': '7.403e-06', 'entropy': '1.055', 'num_tokens': '5.701e+07', 'mean_token_accuracy': '0.7528', 'epoch': '0.09377'}
{'loss': '0.6059', 'grad_norm': '0.3088', 'learning_rate': '7.392e-06', 'entropy': '0.5872', 'num_tokens': '5.702e+07', 'mean_token_accuracy': '0.8444', 'epoch': '0.09378'}
{'loss': '0.9508', 'grad_norm': '0.4227', 'learning_rate': '7.382e-06', 'entropy': '1.007', 'num_tokens': '5.702e+07', 'mean_token_accuracy': '0.7553', 'epoch': '0.09379'}
{'loss': '0.6587', 'grad_norm': '0.3795', 'learning_rate': '7.371e-06', 'entropy': '0.6683', 'num_tokens': '5.703e+07', 'mean_token_accuracy': '0.8227', 'epoch': '0.0938'}
{'loss': '0.6996', 'grad_norm': '0.3725', 'learning_rate': '7.361e-06', 'entropy': '0.7143', 'num_tokens': '5.703e+07', 'mean_token_accuracy': '0.8123', 'epoch': '0.09381'}
{'loss': '0.6155', 'grad_norm': '0.3862', 'learning_rate': '7.351e-06', 'entropy': '0.6297', 'num_tokens': '5.704e+07', 'mean_token_accuracy': '0.8408', 'epoch': '0.09382'}
{'loss': '0.6507', 'grad_norm': '0.4258', 'learning_rate': '7.34e-06', 'entropy': '0.7826', 'num_tokens': '5.704e+07', 'mean_token_accuracy': '0.7969', 'epoch': '0.09383'}
{'loss': '0.8846', 'grad_norm': '0.502', 'learning_rate': '7.33e-06', 'entropy': '0.9689', 'num_tokens': '5.705e+07', 'mean_token_accuracy': '0.7653', 'epoch': '0.09383'}
{'loss': '0.8007', 'grad_norm': '0.3127', 'learning_rate': '7.32e-06', 'entropy': '0.9609', 'num_tokens': '5.705e+07', 'mean_token_accuracy': '0.7686', 'epoch': '0.09384'}
{'loss': '0.6584', 'grad_norm': '0.3336', 'learning_rate': '7.309e-06', 'entropy': '0.6109', 'num_tokens': '5.706e+07', 'mean_token_accuracy': '0.8436', 'epoch': '0.09385'}
{'loss': '0.5787', 'grad_norm': '0.3736', 'learning_rate': '7.299e-06', 'entropy': '0.5687', 'num_tokens': '5.707e+07', 'mean_token_accuracy': '0.8374', 'epoch': '0.09386'}
{'loss': '0.8035', 'grad_norm': '0.4269', 'learning_rate': '7.288e-06', 'entropy': '0.761', 'num_tokens': '5.707e+07', 'mean_token_accuracy': '0.8048', 'epoch': '0.09387'}
{'loss': '0.8215', 'grad_norm': '0.4756', 'learning_rate': '7.278e-06', 'entropy': '0.8265', 'num_tokens': '5.708e+07', 'mean_token_accuracy': '0.7873', 'epoch': '0.09388'}
{'loss': '0.9519', 'grad_norm': '0.4593', 'learning_rate': '7.268e-06', 'entropy': '0.9142', 'num_tokens': '5.708e+07', 'mean_token_accuracy': '0.7792', 'epoch': '0.09389'}
{'loss': '0.668', 'grad_norm': '0.4272', 'learning_rate': '7.258e-06', 'entropy': '0.7538', 'num_tokens': '5.709e+07', 'mean_token_accuracy': '0.8105', 'epoch': '0.0939'}
{'loss': '0.9248', 'grad_norm': '0.4155', 'learning_rate': '7.247e-06', 'entropy': '0.8642', 'num_tokens': '5.709e+07', 'mean_token_accuracy': '0.7994', 'epoch': '0.09391'}
{'loss': '1.005', 'grad_norm': '0.4493', 'learning_rate': '7.237e-06', 'entropy': '1.013', 'num_tokens': '5.71e+07', 'mean_token_accuracy': '0.7508', 'epoch': '0.09391'}
{'loss': '0.7823', 'grad_norm': '0.3901', 'learning_rate': '7.227e-06', 'entropy': '0.8351', 'num_tokens': '5.711e+07', 'mean_token_accuracy': '0.7974', 'epoch': '0.09392'}
{'loss': '0.6698', 'grad_norm': '0.4322', 'learning_rate': '7.216e-06', 'entropy': '0.6641', 'num_tokens': '5.711e+07', 'mean_token_accuracy': '0.8225', 'epoch': '0.09393'}
{'loss': '0.8287', 'grad_norm': '0.331', 'learning_rate': '7.206e-06', 'entropy': '0.8845', 'num_tokens': '5.711e+07', 'mean_token_accuracy': '0.7808', 'epoch': '0.09394'}
{'loss': '0.8024', 'grad_norm': '0.3765', 'learning_rate': '7.196e-06', 'entropy': '0.7638', 'num_tokens': '5.712e+07', 'mean_token_accuracy': '0.8031', 'epoch': '0.09395'}
{'loss': '0.8781', 'grad_norm': '0.3499', 'learning_rate': '7.186e-06', 'entropy': '0.8007', 'num_tokens': '5.713e+07', 'mean_token_accuracy': '0.7995', 'epoch': '0.09396'}
{'loss': '0.908', 'grad_norm': '0.4346', 'learning_rate': '7.175e-06', 'entropy': '0.929', 'num_tokens': '5.713e+07', 'mean_token_accuracy': '0.7569', 'epoch': '0.09397'}
{'loss': '0.9196', 'grad_norm': '0.4801', 'learning_rate': '7.165e-06', 'entropy': '0.9985', 'num_tokens': '5.714e+07', 'mean_token_accuracy': '0.7642', 'epoch': '0.09398'}
{'loss': '0.8193', 'grad_norm': '0.5014', 'learning_rate': '7.155e-06', 'entropy': '0.7897', 'num_tokens': '5.714e+07', 'mean_token_accuracy': '0.7963', 'epoch': '0.09399'}
{'loss': '0.944', 'grad_norm': '0.4934', 'learning_rate': '7.145e-06', 'entropy': '1.017', 'num_tokens': '5.715e+07', 'mean_token_accuracy': '0.756', 'epoch': '0.09399'}
{'loss': '0.644', 'grad_norm': '0.2772', 'learning_rate': '7.134e-06', 'entropy': '0.6526', 'num_tokens': '5.715e+07', 'mean_token_accuracy': '0.8251', 'epoch': '0.094'}
{'loss': '0.6339', 'grad_norm': '0.4245', 'learning_rate': '7.124e-06', 'entropy': '0.6323', 'num_tokens': '5.716e+07', 'mean_token_accuracy': '0.8285', 'epoch': '0.09401'}
{'loss': '0.7909', 'grad_norm': '0.4995', 'learning_rate': '7.114e-06', 'entropy': '0.8334', 'num_tokens': '5.716e+07', 'mean_token_accuracy': '0.7919', 'epoch': '0.09402'}
{'loss': '0.6234', 'grad_norm': '0.3839', 'learning_rate': '7.104e-06', 'entropy': '0.6209', 'num_tokens': '5.717e+07', 'mean_token_accuracy': '0.8276', 'epoch': '0.09403'}

 89%|████████▊ | 10626/12000 [6:15:35<36:12,  1.58s/it]
 89%|████████▊ | 10627/12000 [6:15:36<35:58,  1.57s/it]
                                                       

 89%|████████▊ | 10627/12000 [6:15:36<35:58,  1.57s/it]
 89%|████████▊ | 10628/12000 [6:15:38<35:41,  1.56s/it]
                                                       

 89%|████████▊ | 10628/12000 [6:15:38<35:41,  1.56s/it]
 89%|████████▊ | 10629/12000 [6:15:40<38:46,  1.70s/it]
                                                       

 89%|████████▊ | 10629/12000 [6:15:40<38:46,  1.70s/it]
 89%|████████▊ | 10630/12000 [6:15:42<40:54,  1.79s/it]
                                                       

 89%|████████▊ | 10630/12000 [6:15:42<40:54,  1.79s/it]
 89%|████████▊ | 10631/12000 [6:15:43<40:24,  1.77s/it]
                                                       

 89%|████████▊ | 10631/12000 [6:15:43<40:24,  1.77s/it]
 89%|████████▊ | 10632/12000 [6:15:47<56:03,  2.46s/it]
                                                       

 89%|████████▊ | 10632/12000 [6:15:47<56:03,  2.46s/it]
 89%|████████▊ | 10633/12000 [6:15:49<49:22,  2.17s/it]
                                                       

 89%|████████▊ | 10633/12000 [6:15:49<49:22,  2.17s/it]
 89%|████████▊ | 10634/12000 [6:15:51<50:49,  2.23s/it]
                                                       

 89%|████████▊ | 10634/12000 [6:15:51<50:49,  2.23s/it]
 89%|████████▊ | 10635/12000 [6:15:53<45:43,  2.01s/it]
                                                       

 89%|████████▊ | 10635/12000 [6:15:53<45:43,  2.01s/it]
 89%|████████▊ | 10636/12000 [6:15:56<54:07,  2.38s/it]
                                                       

 89%|████████▊ | 10636/12000 [6:15:56<54:07,  2.38s/it]
 89%|████████▊ | 10637/12000 [6:15:58<50:01,  2.20s/it]
                                                       

 89%|████████▊ | 10637/12000 [6:15:58<50:01,  2.20s/it]
 89%|████████▊ | 10638/12000 [6:15:59<44:58,  1.98s/it]
                                                       

 89%|████████▊ | 10638/12000 [6:15:59<44:58,  1.98s/it]
 89%|████████▊ | 10639/12000 [6:16:01<42:25,  1.87s/it]
                                                       

 89%|████████▊ | 10639/12000 [6:16:01<42:25,  1.87s/it]
 89%|████████▊ | 10640/12000 [6:16:03<43:15,  1.91s/it]
                                                       

 89%|████████▊ | 10640/12000 [6:16:03<43:15,  1.91s/it]
 89%|████████▊ | 10641/12000 [6:16:05<42:22,  1.87s/it]
                                                       

 89%|████████▊ | 10641/12000 [6:16:05<42:22,  1.87s/it]
 89%|████████▊ | 10642/12000 [6:16:06<40:02,  1.77s/it]
                                                       

 89%|████████▊ | 10642/12000 [6:16:06<40:02,  1.77s/it]
 89%|████████▊ | 10643/12000 [6:16:08<37:36,  1.66s/it]
                                                       

 89%|████████▊ | 10643/12000 [6:16:08<37:36,  1.66s/it][WARNING  | transformer_engine.pytorch.dot_product_attention.utils]: Supported flash-attn versions are >= 2.1.1, <= 2.8.0.post2. Found flash-attn 2.8.3.
[transformers] `torch_dtype` is deprecated! Use `dtype` instead!
Loading tokenizer...
Loading model BF16 on ROCm...

Loading weights:   0%|          | 0/398 [00:00<?, ?it/s]
Loading weights:   0%|          | 1/398 [00:00<04:19,  1.53it/s]
Loading weights:   2%|▏         | 9/398 [00:00<00:25, 15.37it/s]
Loading weights:   4%|▍         | 16/398 [00:00<00:14, 25.83it/s]
Loading weights:   7%|▋         | 27/398 [00:00<00:08, 41.99it/s]
Loading weights:  10%|▉         | 38/398 [00:01<00:06, 54.16it/s]
Loading weights:  12%|█▏        | 49/398 [00:01<00:05, 63.23it/s]
Loading weights:  15%|█▌        | 60/398 [00:01<00:04, 69.66it/s]
Loading weights:  18%|█▊        | 71/398 [00:01<00:04, 74.44it/s]
Loading weights:  21%|██        | 82/398 [00:01<00:04, 78.11it/s]
Loading weights:  23%|██▎       | 93/398 [00:01<00:03, 80.89it/s]
Loading weights:  26%|██▌       | 104/398 [00:01<00:03, 82.86it/s]
Loading weights:  29%|██▉       | 115/398 [00:02<00:03, 84.34it/s]
Loading weights:  32%|███▏      | 126/398 [00:02<00:03, 85.54it/s]
Loading weights:  34%|███▍      | 137/398 [00:02<00:03, 86.43it/s]
Loading weights:  37%|███▋      | 148/398 [00:02<00:02, 86.61it/s]
Loading weights:  40%|███▉      | 159/398 [00:02<00:02, 87.55it/s]
Loading weights:  43%|████▎     | 170/398 [00:02<00:02, 87.66it/s]
Loading weights:  45%|████▌     | 181/398 [00:02<00:02, 87.01it/s]
Loading weights:  48%|████▊     | 191/398 [00:02<00:02, 89.94it/s]
Loading weights:  51%|█████     | 202/398 [00:02<00:02, 89.54it/s]
Loading weights:  53%|█████▎    | 212/398 [00:03<00:02, 91.48it/s]
Loading weights:  56%|█████▌    | 223/398 [00:03<00:01, 89.04it/s]
Loading weights:  59%|█████▊    | 233/398 [00:03<00:01, 85.48it/s]
Loading weights:  61%|██████    | 243/398 [00:03<00:01, 89.10it/s]
Loading weights:  64%|██████▎   | 253/398 [00:03<00:01, 85.31it/s]
Loading weights:  66%|██████▌   | 262/398 [00:03<00:01, 83.28it/s]
Loading weights:  68%|██████▊   | 271/398 [00:03<00:01, 83.09it/s]
Loading weights:  70%|███████   | 280/398 [00:03<00:01, 79.45it/s]
Loading weights:  73%|███████▎  | 291/398 [00:04<00:01, 82.09it/s]
Loading weights:  76%|███████▌  | 302/398 [00:04<00:01, 83.75it/s]
Loading weights:  79%|███████▊  | 313/398 [00:04<00:01, 83.58it/s]
Loading weights:  81%|████████▏ | 324/398 [00:04<00:00, 84.55it/s]
Loading weights:  84%|████████▍ | 335/398 [00:04<00:00, 84.82it/s]
Loading weights:  87%|████████▋ | 346/398 [00:04<00:00, 85.66it/s]
Loading weights:  90%|████████▉ | 357/398 [00:04<00:00, 85.80it/s]
Loading weights:  92%|█████████▏| 368/398 [00:04<00:00, 86.18it/s]
Loading weights:  95%|█████████▌| 379/398 [00:05<00:00, 86.01it/s]
Loading weights:  98%|█████████▊| 390/398 [00:05<00:00, 87.03it/s]
Loading weights: 100%|██████████| 398/398 [00:05<00:00, 76.05it/s]
[transformers] warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
Attaching LoRA...
trainable params: 132,120,576 || all params: 4,154,588,672 || trainable%: 3.1801
Loaded 1,807,941 conversations from /workspace/datasets/cybersec/cybersec_master.jsonl
Prepared 1,807,941 training texts

Adding EOS to train dataset:   0%|          | 0/1807941 [00:00<?, ? examples/s]
Adding EOS to train dataset:   0%|          | 5288/1807941 [00:00<00:34, 52671.59 examples/s]
Adding EOS to train dataset:   1%|          | 10573/1807941 [00:00<00:34, 52772.77 examples/s]
Adding EOS to train dataset:   1%|          | 16000/1807941 [00:00<00:33, 53237.30 examples/s]
Adding EOS to train dataset:   1%|          | 21506/1807941 [00:00<00:33, 53950.01 examples/s]
Adding EOS to train dataset:   2%|▏         | 29172/1807941 [00:00<00:33, 52647.58 examples/s]
Adding EOS to train dataset:   2%|▏         | 37000/1807941 [00:00<00:33, 52316.71 examples/s]
Adding EOS to train dataset:   2%|▏         | 44915/1807941 [00:00<00:33, 52474.78 examples/s]
Adding EOS to train dataset:   3%|▎         | 51807/1807941 [00:01<00:34, 50253.29 examples/s]
Adding EOS to train dataset:   3%|▎         | 57044/1807941 [00:01<00:34, 50766.18 examples/s]
Adding EOS to train dataset:   3%|▎         | 62454/1807941 [00:01<00:33, 51628.80 examples/s]
Adding EOS to train dataset:   4%|▍         | 67871/1807941 [00:01<00:33, 52312.83 examples/s]
Adding EOS to train dataset:   4%|▍         | 73149/1807941 [00:01<00:33, 52438.72 examples/s]
Adding EOS to train dataset:   4%|▍         | 78626/1807941 [00:01<00:32, 53096.82 examples/s]
Adding EOS to train dataset:   5%|▍         | 84000/1807941 [00:01<00:32, 53007.64 examples/s]
Adding EOS to train dataset:   5%|▍         | 90129/1807941 [00:01<00:35, 48375.36 examples/s]
Adding EOS to train dataset:   5%|▌         | 95405/1807941 [00:01<00:34, 49542.08 examples/s]
Adding EOS to train dataset:   6%|▌         | 100783/1807941 [00:01<00:33, 50608.37 examples/s]
Adding EOS to train dataset:   6%|▌         | 106000/1807941 [00:02<00:33, 50870.62 examples/s]
Adding EOS to train dataset:   6%|▌         | 111212/1807941 [00:02<00:33, 51227.16 examples/s]
Adding EOS to train dataset:   7%|▋         | 118898/1807941 [00:02<00:32, 51228.47 examples/s]
Adding EOS to train dataset:   7%|▋         | 126431/1807941 [00:02<00:33, 50869.07 examples/s]
Adding EOS to train dataset:   7%|▋         | 131585/1807941 [00:02<00:33, 50731.17 examples/s]
Adding EOS to train dataset:   8%|▊         | 136875/1807941 [00:02<00:32, 51298.66 examples/s]
Adding EOS to train dataset:   8%|▊         | 144478/1807941 [00:02<00:32, 51074.80 examples/s]
Adding EOS to train dataset:   8%|▊         | 149669/1807941 [00:02<00:32, 51239.70 examples/s]
Adding EOS to train dataset:   9%|▊         | 154984/1807941 [00:03<00:31, 51747.64 examples/s]
Adding EOS to train dataset:   9%|▉         | 162734/1807941 [00:03<00:31, 51648.81 examples/s]
Adding EOS to train dataset:   9%|▉         | 170299/1807941 [00:03<00:31, 51230.95 examples/s]
Adding EOS to train dataset:  10%|▉         | 177442/1807941 [00:03<00:37, 43841.47 examples/s]
Adding EOS to train dataset:  10%|█         | 182280/1807941 [00:03<00:36, 44815.46 examples/s]
Adding EOS to train dataset:  10%|█         | 187196/1807941 [00:03<00:35, 45834.53 examples/s]
Adding EOS to train dataset:  11%|█         | 192152/1807941 [00:03<00:34, 46769.18 examples/s]
Adding EOS to train dataset:  11%|█         | 197340/1807941 [00:03<00:33, 48117.92 examples/s]
Adding EOS to train dataset:  11%|█         | 202442/1807941 [00:04<00:32, 48909.45 examples/s]
Adding EOS to train dataset:  12%|█▏        | 209944/1807941 [00:04<00:32, 49314.55 examples/s]
Adding EOS to train dataset:  12%|█▏        | 214970/1807941 [00:04<00:32, 49557.85 examples/s]
Adding EOS to train dataset:  12%|█▏        | 220000/1807941 [00:04<00:32, 49472.58 examples/s]
Adding EOS to train dataset:  12%|█▏        | 225000/1807941 [00:04<00:31, 49609.00 examples/s]
Adding EOS to train dataset:  13%|█▎        | 230072/1807941 [00:04<00:31, 49923.78 examples/s]
Adding EOS to train dataset:  13%|█▎        | 235453/1807941 [00:04<00:30, 51043.45 examples/s]
Adding EOS to train dataset:  13%|█▎        | 240746/1807941 [00:04<00:30, 51478.10 examples/s]
Adding EOS to train dataset:  14%|█▎        | 248451/1807941 [00:04<00:30, 51430.63 examples/s]
Adding EOS to train dataset:  14%|█▍        | 253621/1807941 [00:05<00:30, 51224.13 examples/s]
Adding EOS to train dataset:  14%|█▍        | 261245/1807941 [00:05<00:30, 51076.68 examples/s]
Adding EOS to train dataset:  15%|█▍        | 266401/1807941 [00:05<00:30, 51198.72 examples/s]
Adding EOS to train dataset:  15%|█▌        | 274000/1807941 [00:05<00:30, 50947.86 examples/s]
Adding EOS to train dataset:  15%|█▌        | 279166/1807941 [00:05<00:29, 51125.38 examples/s]
Adding EOS to train dataset:  16%|█▌        | 286996/1807941 [00:05<00:29, 51500.04 examples/s]
Adding EOS to train dataset:  16%|█▋        | 294851/1807941 [00:05<00:29, 51785.72 examples/s]
Adding EOS to train dataset:  17%|█▋        | 302652/1807941 [00:05<00:29, 51803.11 examples/s]
Adding EOS to train dataset:  17%|█▋        | 307906/1807941 [00:06<00:28, 51974.84 examples/s]
Adding EOS to train dataset:  17%|█▋        | 315633/1807941 [00:06<00:28, 51818.79 examples/s]
Adding EOS to train dataset:  18%|█▊        | 323198/1807941 [00:06<00:28, 51367.64 examples/s]
Adding EOS to train dataset:  18%|█▊        | 328566/1807941 [00:06<00:28, 51910.37 examples/s]
Adding EOS to train dataset:  19%|█▊        | 336473/1807941 [00:06<00:28, 52178.02 examples/s]
Adding EOS to train dataset:  19%|█▉        | 342034/1807941 [00:06<00:27, 53007.26 examples/s]
Adding EOS to train dataset:  19%|█▉        | 347831/1807941 [00:06<00:27, 53887.86 examples/s]
Adding EOS to train dataset:  20%|█▉        | 354000/1807941 [00:07<00:36, 40196.65 examples/s]
Adding EOS to train dataset:  20%|█▉        | 359522/1807941 [00:07<00:33, 43426.10 examples/s]
Adding EOS to train dataset:  20%|██        | 365000/1807941 [00:07<00:31, 46007.87 examples/s]
Adding EOS to train dataset:  20%|██        | 370491/1807941 [00:07<00:29, 48241.31 examples/s]
Adding EOS to train dataset:  21%|██        | 376000/1807941 [00:07<00:28, 49850.85 examples/s]
Adding EOS to train dataset:  21%|██        | 381596/1807941 [00:07<00:27, 51517.66 examples/s]
Adding EOS to train dataset:  21%|██▏       | 387192/1807941 [00:07<00:26, 52759.99 examples/s]
Adding EOS to train dataset:  22%|██▏       | 392993/1807941 [00:07<00:26, 54199.37 examples/s]
Adding EOS to train dataset:  22%|██▏       | 401397/1807941 [00:07<00:25, 54889.32 examples/s]
Adding EOS to train dataset:  23%|██▎       | 406985/1807941 [00:08<00:25, 55148.88 examples/s]
Adding EOS to train dataset:  23%|██▎       | 414616/1807941 [00:08<00:26, 53303.23 examples/s]
Adding EOS to train dataset:  23%|██▎       | 422394/1807941 [00:08<00:26, 52807.27 examples/s]
Adding EOS to train dataset:  24%|██▍       | 430055/1807941 [00:08<00:26, 52237.51 examples/s]
Adding EOS to train dataset:  24%|██▍       | 438000/1807941 [00:08<00:26, 52219.45 examples/s]
Adding EOS to train dataset:  25%|██▍       | 443309/1807941 [00:08<00:26, 52418.68 examples/s]
Adding EOS to train dataset:  25%|██▍       | 451189/1807941 [00:08<00:25, 52454.78 examples/s]
Adding EOS to train dataset:  25%|██▌       | 456489/1807941 [00:08<00:25, 52585.16 examples/s]
Adding EOS to train dataset:  26%|██▌       | 464175/1807941 [00:09<00:25, 51914.40 examples/s]
Adding EOS to train dataset:  26%|██▌       | 469398/1807941 [00:09<00:25, 51989.82 examples/s]
Adding EOS to train dataset:  26%|██▋       | 476701/1807941 [00:09<00:26, 50855.09 examples/s]
Adding EOS to train dataset:  27%|██▋       | 484000/1807941 [00:09<00:26, 50036.00 examples/s]
Adding EOS to train dataset:  27%|██▋       | 491648/1807941 [00:09<00:26, 50337.31 examples/s]
Adding EOS to train dataset:  28%|██▊       | 499000/1807941 [00:09<00:26, 49714.14 examples/s]
Adding EOS to train dataset:  28%|██▊       | 504000/1807941 [00:09<00:26, 49617.88 examples/s]
Adding EOS to train dataset:  28%|██▊       | 509000/1807941 [00:10<00:26, 49481.51 examples/s]
Adding EOS to train dataset:  29%|██▊       | 516536/1807941 [00:10<00:25, 49739.30 examples/s]
Adding EOS to train dataset:  29%|██▉       | 522378/1807941 [00:10<00:24, 51877.85 examples/s]
Adding EOS to train dataset:  29%|██▉       | 528120/1807941 [00:10<00:24, 53318.88 examples/s]
Adding EOS to train dataset:  30%|██▉       | 533828/1807941 [00:10<00:23, 54293.14 examples/s]
Adding EOS to train dataset:  30%|██▉       | 539458/1807941 [00:10<00:23, 54848.99 examples/s]
Adding EOS to train dataset:  30%|███       | 545292/1807941 [00:10<00:22, 55842.10 examples/s]
Adding EOS to train dataset:  30%|███       | 551221/1807941 [00:10<00:22, 56836.88 examples/s]
Adding EOS to train dataset:  31%|███       | 557396/1807941 [00:10<00:21, 58272.11 examples/s]
Adding EOS to train dataset:  31%|███       | 563784/1807941 [00:10<00:20, 59922.15 examples/s]
Adding EOS to train dataset:  32%|███▏      | 569973/1807941 [00:11<00:20, 60502.08 examples/s]
Adding EOS to train dataset:  32%|███▏      | 576268/1807941 [00:11<00:20, 60996.23 examples/s]
Adding EOS to train dataset:  32%|███▏      | 582411/1807941 [00:11<00:20, 61123.67 examples/s]
Adding EOS to train dataset:  33%|███▎      | 588612/1807941 [00:11<00:19, 61385.27 examples/s]
Adding EOS to train dataset:  33%|███▎      | 595620/1807941 [00:11<00:18, 63979.05 examples/s]
Adding EOS to train dataset:  33%|███▎      | 602460/1807941 [00:11<00:18, 65297.87 examples/s]
Adding EOS to train dataset:  34%|███▎      | 609150/1807941 [00:11<00:18, 65776.94 examples/s]
Adding EOS to train dataset:  34%|███▍      | 616068/1807941 [00:11<00:17, 66791.52 examples/s]
Adding EOS to train dataset:  34%|███▍      | 622790/1807941 [00:11<00:17, 66917.78 examples/s]
Adding EOS to train dataset:  35%|███▍      | 632632/1807941 [00:12<00:17, 66403.11 examples/s]
Adding EOS to train dataset:  36%|███▌      | 642248/1807941 [00:12<00:17, 65576.50 examples/s]
Adding EOS to train dataset:  36%|███▌      | 651833/1807941 [00:12<00:17, 65005.56 examples/s]
Adding EOS to train dataset:  37%|███▋      | 660318/1807941 [00:12<00:18, 62251.41 examples/s]
Adding EOS to train dataset:  37%|███▋      | 669061/1807941 [00:12<00:18, 60849.02 examples/s]
Adding EOS to train dataset:  38%|███▊      | 678234/1807941 [00:12<00:18, 60941.18 examples/s]
Adding EOS to train dataset:  38%|███▊      | 687032/1807941 [00:12<00:18, 60233.70 examples/s]
Adding EOS to train dataset:  38%|███▊      | 695813/1807941 [00:13<00:18, 59710.23 examples/s]
Adding EOS to train dataset:  39%|███▉      | 704643/1807941 [00:13<00:18, 59451.81 examples/s]
Adding EOS to train dataset:  39%|███▉      | 713128/1807941 [00:13<00:18, 58576.47 examples/s]
Adding EOS to train dataset:  40%|███▉      | 721909/1807941 [00:13<00:18, 58562.39 examples/s]
Adding EOS to train dataset:  40%|████      | 728171/1807941 [00:13<00:18, 59357.93 examples/s]
Adding EOS to train dataset:  41%|████      | 734384/1807941 [00:13<00:17, 60026.37 examples/s]
Adding EOS to train dataset:  41%|████      | 740865/1807941 [00:13<00:17, 61252.43 examples/s]
Adding EOS to train dataset:  41%|████▏     | 747413/1807941 [00:13<00:16, 62384.22 examples/s]
Adding EOS to train dataset:  42%|████▏     | 753891/1807941 [00:14<00:16, 63042.28 examples/s]
Adding EOS to train dataset:  42%|████▏     | 760792/1807941 [00:14<00:16, 64733.67 examples/s]
Adding EOS to train dataset:  42%|████▏     | 767695/1807941 [00:14<00:15, 65970.50 examples/s]
Adding EOS to train dataset:  43%|████▎     | 775410/1807941 [00:14<00:17, 60415.27 examples/s]
Adding EOS to train dataset:  43%|████▎     | 783751/1807941 [00:14<00:17, 58643.37 examples/s]
Adding EOS to train dataset:  44%|████▎     | 790430/1807941 [00:15<00:40, 25062.28 examples/s]
Adding EOS to train dataset:  44%|████▍     | 796502/1807941 [00:15<00:34, 29665.67 examples/s]
Adding EOS to train dataset:  44%|████▍     | 802457/1807941 [00:15<00:29, 34319.09 examples/s]
Adding EOS to train dataset:  45%|████▍     | 808487/1807941 [00:15<00:25, 39046.21 examples/s]
Adding EOS to train dataset:  45%|████▌     | 814335/1807941 [00:15<00:23, 43055.71 examples/s]
Adding EOS to train dataset:  45%|████▌     | 820064/1807941 [00:15<00:21, 46294.27 examples/s]
Adding EOS to train dataset:  46%|████▌     | 828279/1807941 [00:15<00:19, 49070.49 examples/s]
Adding EOS to train dataset:  46%|████▌     | 833876/1807941 [00:15<00:19, 50710.01 examples/s]
Adding EOS to train dataset:  47%|████▋     | 842070/1807941 [00:16<00:18, 52031.66 examples/s]
Adding EOS to train dataset:  47%|████▋     | 847656/1807941 [00:16<00:18, 52960.39 examples/s]
Adding EOS to train dataset:  47%|████▋     | 853854/1807941 [00:16<00:17, 55285.32 examples/s]
Adding EOS to train dataset:  48%|████▊     | 859879/1807941 [00:16<00:16, 56619.14 examples/s]
Adding EOS to train dataset:  48%|████▊     | 866000/1807941 [00:16<00:16, 57863.88 examples/s]
Adding EOS to train dataset:  48%|████▊     | 872150/1807941 [00:16<00:15, 58873.29 examples/s]
Adding EOS to train dataset:  49%|████▊     | 878392/1807941 [00:16<00:15, 59893.03 examples/s]
Adding EOS to train dataset:  49%|████▉     | 884657/1807941 [00:16<00:15, 60695.60 examples/s]
Adding EOS to train dataset:  49%|████▉     | 890886/1807941 [00:16<00:14, 61162.03 examples/s]
Adding EOS to train dataset:  50%|████▉     | 900006/1807941 [00:17<00:14, 60843.15 examples/s]
Adding EOS to train dataset:  50%|█████     | 906269/1807941 [00:17<00:14, 61319.78 examples/s]
Adding EOS to train dataset:  50%|█████     | 912563/1807941 [00:17<00:14, 61767.91 examples/s]
Adding EOS to train dataset:  51%|█████     | 918812/1807941 [00:17<00:14, 61969.57 examples/s]
Adding EOS to train dataset:  51%|█████     | 925272/1807941 [00:17<00:14, 62681.46 examples/s]
Adding EOS to train dataset:  52%|█████▏    | 931711/1807941 [00:17<00:13, 63176.41 examples/s]
Adding EOS to train dataset:  52%|█████▏    | 938402/1807941 [00:17<00:13, 64238.16 examples/s]
Adding EOS to train dataset:  52%|█████▏    | 945000/1807941 [00:17<00:13, 64523.09 examples/s]
Adding EOS to train dataset:  53%|█████▎    | 954245/1807941 [00:17<00:13, 63385.44 examples/s]
Adding EOS to train dataset:  53%|█████▎    | 961000/1807941 [00:18<00:13, 64426.45 examples/s]
Adding EOS to train dataset:  54%|█████▎    | 967745/1807941 [00:18<00:12, 65263.77 examples/s]
Adding EOS to train dataset:  54%|█████▍    | 974411/1807941 [00:18<00:12, 65659.64 examples/s]
Adding EOS to train dataset:  54%|█████▍    | 981024/1807941 [00:18<00:12, 65791.99 examples/s]
Adding EOS to train dataset:  55%|█████▍    | 991026/1807941 [00:18<00:12, 66127.48 examples/s]
Adding EOS to train dataset:  55%|█████▌    | 997740/1807941 [00:18<00:12, 66395.68 examples/s]
Adding EOS to train dataset:  56%|█████▌    | 1004388/1807941 [00:18<00:12, 66417.35 examples/s]
Adding EOS to train dataset:  56%|█████▌    | 1013643/1807941 [00:18<00:12, 64659.79 examples/s]
Adding EOS to train dataset:  57%|█████▋    | 1023056/1807941 [00:18<00:12, 63996.57 examples/s]
Adding EOS to train dataset:  57%|█████▋    | 1032588/1807941 [00:19<00:12, 63843.12 examples/s]
Adding EOS to train dataset:  58%|█████▊    | 1042065/1807941 [00:19<00:12, 63629.21 examples/s]
Adding EOS to train dataset:  58%|█████▊    | 1051559/1807941 [00:19<00:11, 63519.62 examples/s]
Adding EOS to train dataset:  59%|█████▊    | 1061064/1807941 [00:19<00:11, 63468.22 examples/s]
Adding EOS to train dataset:  59%|█████▉    | 1067445/1807941 [00:19<00:11, 63544.17 examples/s]
Adding EOS to train dataset:  59%|█████▉    | 1073912/1807941 [00:19<00:11, 63817.17 examples/s]
Adding EOS to train dataset:  60%|█████▉    | 1083474/1807941 [00:19<00:11, 63789.87 examples/s]
Adding EOS to train dataset:  60%|██████    | 1089899/1807941 [00:20<00:11, 63902.13 examples/s]
Adding EOS to train dataset:  61%|██████    | 1099550/1807941 [00:20<00:11, 64051.21 examples/s]
Adding EOS to train dataset:  61%|██████    | 1106001/1807941 [00:20<00:10, 64163.11 examples/s]
Adding EOS to train dataset:  62%|██████▏   | 1112443/1807941 [00:20<00:10, 64228.00 examples/s]
Adding EOS to train dataset:  62%|██████▏   | 1122007/1807941 [00:20<00:10, 64055.93 examples/s]
Adding EOS to train dataset:  62%|██████▏   | 1128521/1807941 [00:20<00:10, 64329.09 examples/s]
Adding EOS to train dataset:  63%|██████▎   | 1138006/1807941 [00:20<00:10, 63938.86 examples/s]
Adding EOS to train dataset:  63%|██████▎   | 1147598/1807941 [00:20<00:10, 63940.00 examples/s]
Adding EOS to train dataset:  64%|██████▍   | 1157242/1807941 [00:21<00:10, 64041.15 examples/s]
Adding EOS to train dataset:  65%|██████▍   | 1166744/1807941 [00:21<00:10, 63816.88 examples/s]
Adding EOS to train dataset:  65%|██████▌   | 1175891/1807941 [00:21<00:10, 62931.79 examples/s]
Adding EOS to train dataset:  66%|██████▌   | 1185270/1807941 [00:21<00:09, 62695.90 examples/s]
Adding EOS to train dataset:  66%|██████▌   | 1191794/1807941 [00:21<00:09, 63270.82 examples/s]
Adding EOS to train dataset:  66%|██████▋   | 1198162/1807941 [00:21<00:09, 63369.19 examples/s]
Adding EOS to train dataset:  67%|██████▋   | 1205000/1807941 [00:21<00:09, 64555.00 examples/s]
Adding EOS to train dataset:  67%|██████▋   | 1211687/1807941 [00:21<00:09, 65177.28 examples/s]
Adding EOS to train dataset:  67%|██████▋   | 1218401/1807941 [00:22<00:08, 65719.76 examples/s]
Adding EOS to train dataset:  68%|██████▊   | 1227475/1807941 [00:22<00:09, 63762.60 examples/s]
Adding EOS to train dataset:  68%|██████▊   | 1235589/1807941 [00:22<00:10, 53599.41 examples/s]
Adding EOS to train dataset:  69%|██████▊   | 1241458/1807941 [00:22<00:10, 54752.86 examples/s]
Adding EOS to train dataset:  69%|██████▉   | 1250000/1807941 [00:22<00:10, 55419.47 examples/s]
Adding EOS to train dataset:  69%|██████▉   | 1255950/1807941 [00:22<00:09, 56255.57 examples/s]
Adding EOS to train dataset:  70%|██████▉   | 1264458/1807941 [00:22<00:09, 56409.64 examples/s]
Adding EOS to train dataset:  70%|███████   | 1272000/1807941 [00:23<00:09, 54165.25 examples/s]
Adding EOS to train dataset:  71%|███████   | 1278075/1807941 [00:23<00:09, 55713.14 examples/s]
Adding EOS to train dataset:  71%|███████   | 1284319/1807941 [00:23<00:09, 57403.91 examples/s]
Adding EOS to train dataset:  71%|███████▏  | 1290615/1807941 [00:23<00:08, 58870.95 examples/s]
Adding EOS to train dataset:  72%|███████▏  | 1296809/1807941 [00:23<00:08, 59710.23 examples/s]
Adding EOS to train dataset:  72%|███████▏  | 1302950/1807941 [00:23<00:08, 60187.78 examples/s]
Adding EOS to train dataset:  72%|███████▏  | 1309191/1807941 [00:23<00:08, 60821.92 examples/s]
Adding EOS to train dataset:  73%|███████▎  | 1315414/1807941 [00:23<00:08, 61228.54 examples/s]
Adding EOS to train dataset:  73%|███████▎  | 1321758/1807941 [00:23<00:07, 61872.69 examples/s]
Adding EOS to train dataset:  73%|███████▎  | 1328438/1807941 [00:23<00:07, 63326.03 examples/s]
Adding EOS to train dataset:  74%|███████▍  | 1334912/1807941 [00:24<00:07, 63743.12 examples/s]
Adding EOS to train dataset:  74%|███████▍  | 1341457/1807941 [00:24<00:07, 64247.02 examples/s]
Adding EOS to train dataset:  75%|███████▍  | 1351154/1807941 [00:24<00:07, 64399.85 examples/s]
Adding EOS to train dataset:  75%|███████▌  | 1359213/1807941 [00:24<00:07, 60581.49 examples/s]
Adding EOS to train dataset:  76%|███████▌  | 1365582/1807941 [00:24<00:07, 61364.51 examples/s]
Adding EOS to train dataset:  76%|███████▌  | 1372286/1807941 [00:24<00:06, 62874.69 examples/s]
Adding EOS to train dataset:  76%|███████▋  | 1378943/1807941 [00:24<00:06, 63888.85 examples/s]
Adding EOS to train dataset:  77%|███████▋  | 1385683/1807941 [00:24<00:06, 64879.41 examples/s]
Adding EOS to train dataset:  77%|███████▋  | 1392433/1807941 [00:24<00:06, 65631.54 examples/s]
Adding EOS to train dataset:  77%|███████▋  | 1399201/1807941 [00:25<00:06, 66223.94 examples/s]
Adding EOS to train dataset:  78%|███████▊  | 1405989/1807941 [00:25<00:06, 66708.58 examples/s]
Adding EOS to train dataset:  78%|███████▊  | 1412851/1807941 [00:25<00:05, 67270.61 examples/s]
Adding EOS to train dataset:  79%|███████▊  | 1419631/1807941 [00:25<00:05, 67425.07 examples/s]
Adding EOS to train dataset:  79%|███████▉  | 1428787/1807941 [00:25<00:05, 64938.52 examples/s]
Adding EOS to train dataset:  80%|███████▉  | 1438157/1807941 [00:25<00:05, 64055.32 examples/s]
Adding EOS to train dataset:  80%|████████  | 1447657/1807941 [00:25<00:05, 63807.93 examples/s]
Adding EOS to train dataset:  81%|████████  | 1456944/1807941 [00:25<00:05, 63186.46 examples/s]
Adding EOS to train dataset:  81%|████████  | 1466168/1807941 [00:26<00:05, 62639.64 examples/s]
Adding EOS to train dataset:  82%|████████▏ | 1475356/1807941 [00:26<00:05, 62205.60 examples/s]
Adding EOS to train dataset:  82%|████████▏ | 1484251/1807941 [00:26<00:05, 61306.69 examples/s]
Adding EOS to train dataset:  83%|████████▎ | 1493000/1807941 [00:26<00:05, 60228.45 examples/s]
Adding EOS to train dataset:  83%|████████▎ | 1499119/1807941 [00:26<00:05, 60356.72 examples/s]
Adding EOS to train dataset:  83%|████████▎ | 1505309/1807941 [00:26<00:04, 60729.02 examples/s]
Adding EOS to train dataset:  84%|████████▎ | 1511679/1807941 [00:26<00:04, 61492.34 examples/s]
Adding EOS to train dataset:  84%|████████▍ | 1520902/1807941 [00:26<00:04, 61488.26 examples/s]
Adding EOS to train dataset:  85%|████████▍ | 1529871/1807941 [00:27<00:04, 60913.14 examples/s]
Adding EOS to train dataset:  85%|████████▌ | 1539000/1807941 [00:27<00:04, 60785.55 examples/s]
Adding EOS to train dataset:  86%|████████▌ | 1548066/1807941 [00:27<00:04, 60522.43 examples/s]
Adding EOS to train dataset:  86%|████████▌ | 1556779/1807941 [00:27<00:04, 59760.84 examples/s]
Adding EOS to train dataset:  87%|████████▋ | 1565554/1807941 [00:27<00:04, 59368.69 examples/s]
Adding EOS to train dataset:  87%|████████▋ | 1572000/1807941 [00:27<00:04, 52982.77 examples/s]
Adding EOS to train dataset:  87%|████████▋ | 1578298/1807941 [00:28<00:05, 45057.76 examples/s]
Adding EOS to train dataset:  88%|████████▊ | 1584592/1807941 [00:28<00:05, 43787.97 examples/s]
Adding EOS to train dataset:  88%|████████▊ | 1591000/1807941 [00:28<00:05, 43105.32 examples/s]
Adding EOS to train dataset:  88%|████████▊ | 1595657/1807941 [00:28<00:04, 43834.89 examples/s]
Adding EOS to train dataset:  89%|████████▊ | 1601843/1807941 [00:28<00:04, 43024.82 examples/s]
Adding EOS to train dataset:  89%|████████▉ | 1608199/1807941 [00:28<00:04, 42156.00 examples/s]
Adding EOS to train dataset:  89%|████████▉ | 1612580/1807941 [00:28<00:04, 42523.76 examples/s]
Adding EOS to train dataset:  90%|████████▉ | 1619000/1807941 [00:29<00:04, 42092.49 examples/s]
Adding EOS to train dataset:  90%|████████▉ | 1625158/1807941 [00:29<00:04, 41765.11 examples/s]
Adding EOS to train dataset:  90%|█████████ | 1629507/1807941 [00:29<00:04, 42160.44 examples/s]
Adding EOS to train dataset:  90%|█████████ | 1635856/1807941 [00:29<00:04, 42210.10 examples/s]
Adding EOS to train dataset:  91%|█████████ | 1641922/1807941 [00:29<00:03, 41640.11 examples/s]
Adding EOS to train dataset:  91%|█████████ | 1648102/1807941 [00:29<00:03, 40830.34 examples/s]
Adding EOS to train dataset:  91%|█████████▏| 1652259/1807941 [00:29<00:03, 40997.14 examples/s]
Adding EOS to train dataset:  92%|█████████▏| 1657000/1807941 [00:29<00:03, 42328.32 examples/s]
Adding EOS to train dataset:  92%|█████████▏| 1661539/1807941 [00:30<00:03, 43116.19 examples/s]
Adding EOS to train dataset:  92%|█████████▏| 1666931/1807941 [00:30<00:03, 46025.08 examples/s]
Adding EOS to train dataset:  92%|█████████▏| 1672125/1807941 [00:30<00:02, 47667.17 examples/s]
Adding EOS to train dataset:  93%|█████████▎| 1677640/1807941 [00:30<00:02, 49791.32 examples/s]
Adding EOS to train dataset:  93%|█████████▎| 1683000/1807941 [00:30<00:02, 50442.87 examples/s]
Adding EOS to train dataset:  93%|█████████▎| 1688576/1807941 [00:30<00:02, 51985.05 examples/s]
Adding EOS to train dataset:  94%|█████████▎| 1694000/1807941 [00:30<00:02, 52459.71 examples/s]
Adding EOS to train dataset:  94%|█████████▍| 1699502/1807941 [00:30<00:02, 53211.47 examples/s]
Adding EOS to train dataset:  94%|█████████▍| 1704916/1807941 [00:30<00:01, 53484.18 examples/s]
Adding EOS to train dataset:  95%|█████████▍| 1712952/1807941 [00:31<00:01, 53514.28 examples/s]
Adding EOS to train dataset:  95%|█████████▌| 1719384/1807941 [00:32<00:06, 13811.84 examples/s]
Adding EOS to train dataset:  95%|█████████▌| 1725004/1807941 [00:32<00:04, 17466.53 examples/s]
Adding EOS to train dataset:  96%|█████████▌| 1731000/1807941 [00:32<00:03, 22071.65 examples/s]
Adding EOS to train dataset:  96%|█████████▌| 1736835/1807941 [00:32<00:02, 26974.85 examples/s]
Adding EOS to train dataset:  96%|█████████▋| 1742615/1807941 [00:32<00:02, 31963.00 examples/s]
Adding EOS to train dataset:  97%|█████████▋| 1748276/1807941 [00:32<00:01, 36590.23 examples/s]
Adding EOS to train dataset:  97%|█████████▋| 1754029/1807941 [00:32<00:01, 41017.21 examples/s]
Adding EOS to train dataset:  97%|█████████▋| 1759994/1807941 [00:32<00:01, 45242.66 examples/s]
Adding EOS to train dataset:  98%|█████████▊| 1768089/1807941 [00:33<00:00, 48134.13 examples/s]
Adding EOS to train dataset:  98%|█████████▊| 1775836/1807941 [00:33<00:00, 49260.64 examples/s]
Adding EOS to train dataset:  99%|█████████▊| 1783000/1807941 [00:33<00:00, 48473.89 examples/s]
Adding EOS to train dataset:  99%|█████████▉| 1790003/1807941 [00:33<00:00, 47919.63 examples/s]
Adding EOS to train dataset:  99%|█████████▉| 1795273/1807941 [00:33<00:00, 49004.30 examples/s]
Adding EOS to train dataset: 100%|█████████▉| 1800695/1807941 [00:33<00:00, 50279.90 examples/s]
Adding EOS to train dataset: 100%|█████████▉| 1806187/1807941 [00:33<00:00, 51477.25 examples/s]
Adding EOS to train dataset: 100%|██████████| 1807941/1807941 [00:33<00:00, 53279.71 examples/s]

Tokenizing train dataset:   0%|          | 0/1807941 [00:00<?, ? examples/s]
Tokenizing train dataset:   0%|          | 111/1807941 [00:00<27:22, 1100.86 examples/s]
Tokenizing train dataset:   0%|          | 229/1807941 [00:00<26:27, 1139.07 examples/s]
Tokenizing train dataset:   0%|          | 349/1807941 [00:00<25:52, 1164.28 examples/s]
Tokenizing train dataset:   0%|          | 469/1807941 [00:00<25:38, 1174.88 examples/s]
Tokenizing train dataset:   0%|          | 648/1807941 [00:00<25:33, 1178.83 examples/s]
Tokenizing train dataset:   0%|          | 826/1807941 [00:00<25:34, 1177.69 examples/s]
Tokenizing train dataset:   0%|          | 951/1807941 [00:00<25:14, 1193.31 examples/s]
Tokenizing train dataset:   0%|          | 1112/1807941 [00:01<29:59, 1004.16 examples/s]
Tokenizing train dataset:   0%|          | 1229/1807941 [00:01<28:53, 1042.23 examples/s]
Tokenizing train dataset:   0%|          | 1355/1807941 [00:01<27:29, 1095.32 examples/s]
Tokenizing train dataset:   0%|          | 1478/1807941 [00:01<26:39, 1129.07 examples/s]
Tokenizing train dataset:   0%|          | 1604/1807941 [00:01<25:53, 1162.70 examples/s]
Tokenizing train dataset:   0%|          | 1731/1807941 [00:01<25:16, 1190.66 examples/s]
Tokenizing train dataset:   0%|          | 1860/1807941 [00:01<24:43, 1217.21 examples/s]
Tokenizing train dataset:   0%|          | 1992/1807941 [00:01<24:10, 1244.94 examples/s]
Tokenizing train dataset:   0%|          | 2157/1807941 [00:01<25:23, 1185.29 examples/s]
Tokenizing train dataset:   0%|          | 2290/1807941 [00:01<24:39, 1220.53 examples/s]
Tokenizing train dataset:   0%|          | 2427/1807941 [00:02<23:52, 1260.41 examples/s]
Tokenizing train dataset:   0%|          | 2570/1807941 [00:02<23:01, 1306.85 examples/s]
Tokenizing train dataset:   0%|          | 2706/1807941 [00:02<22:48, 1319.52 examples/s]
Tokenizing train dataset:   0%|          | 2851/1807941 [00:02<22:11, 1355.76 examples/s]
Tokenizing train dataset:   0%|          | 2992/1807941 [00:02<21:58, 1369.09 examples/s]
Tokenizing train dataset:   0%|          | 3170/1807941 [00:02<23:14, 1294.15 examples/s]
Tokenizing train dataset:   0%|          | 3311/1807941 [00:02<22:44, 1322.90 examples/s]
Tokenizing train dataset:   0%|          | 3449/1807941 [00:02<22:29, 1337.41 examples/s]
Tokenizing train dataset:   0%|          | 3591/1807941 [00:02<22:09, 1356.86 examples/s]
Tokenizing train dataset:   0%|          | 3790/1807941 [00:03<22:26, 1340.16 examples/s]
Tokenizing train dataset:   0%|          | 3928/1807941 [00:03<22:18, 1348.10 examples/s]
Tokenizing train dataset:   0%|          | 4095/1807941 [00:03<23:54, 1257.52 examples/s]
Tokenizing train dataset:   0%|          | 4225/1807941 [00:03<23:46, 1264.45 examples/s]
Tokenizing train dataset:   0%|          | 4361/1807941 [00:03<23:19, 1288.48 examples/s]
Tokenizing train dataset:   0%|          | 4547/1807941 [00:03<23:43, 1267.08 examples/s]
Tokenizing train dataset:   0%|          | 4681/1807941 [00:03<23:28, 1280.45 examples/s]
Tokenizing train dataset:   0%|          | 4811/1807941 [00:03<23:27, 1280.90 examples/s]
Tokenizing train dataset:   0%|          | 4999/1807941 [00:04<23:43, 1266.26 examples/s]
Tokenizing train dataset:   0%|          | 5157/1807941 [00:04<25:14, 1190.07 examples/s]
Tokenizing train dataset:   0%|          | 5292/1807941 [00:04<24:30, 1226.20 examples/s]
Tokenizing train dataset:   0%|          | 5427/1807941 [00:04<23:55, 1255.37 examples/s]
Tokenizing train dataset:   0%|          | 5557/1807941 [00:04<23:45, 1264.35 examples/s]
Tokenizing train dataset:   0%|          | 5693/1807941 [00:04<23:17, 1289.23 examples/s]
Tokenizing train dataset:   0%|          | 5826/1807941 [00:04<23:09, 1296.70 examples/s]
Tokenizing train dataset:   0%|          | 5961/1807941 [00:04<22:57, 1308.43 examples/s]
Tokenizing train dataset:   0%|          | 6144/1807941 [00:04<23:54, 1256.30 examples/s]
Tokenizing train dataset:   0%|          | 6284/1807941 [00:05<23:13, 1292.81 examples/s]
Tokenizing train dataset:   0%|          | 6418/1807941 [00:05<23:03, 1302.29 examples/s]
Tokenizing train dataset:   0%|          | 6620/1807941 [00:05<22:48, 1315.92 examples/s]
Tokenizing train dataset:   0%|          | 6767/1807941 [00:05<22:10, 1353.37 examples/s]
Tokenizing train dataset:   0%|          | 6908/1807941 [00:05<21:59, 1365.37 examples/s]
Tokenizing train dataset:   0%|          | 7078/1807941 [00:05<23:31, 1275.75 examples/s]
Tokenizing train dataset:   0%|          | 7262/1807941 [00:05<23:52, 1257.43 examples/s]
Tokenizing train dataset:   0%|          | 7392/1807941 [00:05<23:42, 1265.98 examples/s]
Tokenizing train dataset:   0%|          | 7571/1807941 [00:06<24:13, 1238.24 examples/s]
Tokenizing train dataset:   0%|          | 7759/1807941 [00:06<24:09, 1241.83 examples/s]
Tokenizing train dataset:   0%|          | 7946/1807941 [00:06<24:11, 1240.19 examples/s]
Tokenizing train dataset:   0%|          | 8122/1807941 [00:06<25:34, 1172.90 examples/s]
Tokenizing train dataset:   0%|          | 8249/1807941 [00:06<25:08, 1193.17 examples/s]
Tokenizing train dataset:   0%|          | 8425/1807941 [00:06<25:23, 1181.30 examples/s]
Tokenizing train dataset:   0%|          | 8552/1807941 [00:06<24:58, 1200.74 examples/s]
Tokenizing train dataset:   0%|          | 8724/1807941 [00:07<25:26, 1178.66 examples/s]
Tokenizing train dataset:   0%|          | 8910/1807941 [00:07<25:04, 1195.97 examples/s]
Tokenizing train dataset:   1%|          | 9065/1807941 [00:07<26:15, 1141.71 examples/s]
Tokenizing train dataset:   1%|          | 9230/1807941 [00:07<26:36, 1126.37 examples/s]
Tokenizing train dataset:   1%|          | 9351/1807941 [00:07<26:14, 1142.19 examples/s]
Tokenizing train dataset:   1%|          | 9519/1807941 [00:07<26:30, 1130.86 examples/s]
Tokenizing train dataset:   1%|          | 9685/1807941 [00:07<26:46, 1119.44 examples/s]
Tokenizing train dataset:   1%|          | 9816/1807941 [00:07<25:49, 1160.48 examples/s]
Tokenizing train dataset:   1%|          | 9939/1807941 [00:08<25:28, 1176.00 examples/s]
Tokenizing train dataset:   1%|          | 10064/1807941 [00:08<26:44, 1120.20 examples/s]
Tokenizing train dataset:   1%|          | 10192/1807941 [00:08<25:52, 1158.30 examples/s]
Tokenizing train dataset:   1%|          | 10313/1807941 [00:08<25:37, 1168.98 examples/s]
Tokenizing train dataset:   1%|          | 10451/1807941 [00:08<27:51, 1075.20 examples/s]
Tokenizing train dataset:   1%|          | 10604/1807941 [00:08<28:23, 1055.01 examples/s]
Tokenizing train dataset:   1%|          | 10724/1807941 [00:08<27:32, 1087.82 examples/s]
Tokenizing train dataset:   1%|          | 10851/1807941 [00:08<26:26, 1133.00 examples/s]
Tokenizing train dataset:   1%|          | 11000/1807941 [00:09<26:28, 1131.19 examples/s]
Tokenizing train dataset:   1%|          | 11139/1807941 [00:09<25:01, 1196.97 examples/s]
Tokenizing train dataset:   1%|          | 11275/1807941 [00:09<24:11, 1238.05 examples/s]
Tokenizing train dataset:   1%|          | 11417/1807941 [00:09<23:19, 1283.61 examples/s]
Tokenizing train dataset:   1%|          | 11562/1807941 [00:09<22:34, 1326.47 examples/s]
Tokenizing train dataset:   1%|          | 11755/1807941 [00:09<22:56, 1305.16 examples/s]
Tokenizing train dataset:   1%|          | 11948/1807941 [00:09<23:05, 1296.66 examples/s]
Tokenizing train dataset:   1%|          | 12124/1807941 [00:09<24:53, 1202.76 examples/s]
Tokenizing train dataset:   1%|          | 12247/1807941 [00:10<24:47, 1207.23 examples/s]
Tokenizing train dataset:   1%|          | 12378/1807941 [00:10<24:19, 1230.67 examples/s]
Tokenizing train dataset:   1%|          | 12567/1807941 [00:10<24:10, 1237.71 examples/s]
Tokenizing train dataset:   1%|          | 12751/1807941 [00:10<24:17, 1231.92 examples/s]
Tokenizing train dataset:   1%|          | 12884/1807941 [00:10<23:51, 1253.76 examples/s]
Tokenizing train dataset:   1%|          | 13056/1807941 [00:10<25:47, 1159.71 examples/s]
Tokenizing train dataset:   1%|          | 13235/1807941 [00:10<25:37, 1167.32 examples/s]
Tokenizing train dataset:   1%|          | 13359/1807941 [00:10<25:20, 1180.51 examples/s]
Tokenizing train dataset:   1%|          | 13541/1807941 [00:11<25:12, 1186.11 examples/s]
Tokenizing train dataset:   1%|          | 13664/1807941 [00:11<25:03, 1193.48 examples/s]
Tokenizing train dataset:   1%|          | 13788/1807941 [00:11<24:51, 1202.85 examples/s]
Tokenizing train dataset:   1%|          | 13911/1807941 [00:11<24:45, 1207.80 examples/s]
Tokenizing train dataset:   1%|          | 14074/1807941 [00:11<25:45, 1160.90 examples/s]
Tokenizing train dataset:   1%|          | 14197/1807941 [00:11<25:26, 1175.02 examples/s]
Tokenizing train dataset:   1%|          | 14375/1807941 [00:11<25:24, 1176.55 examples/s]
Tokenizing train dataset:   1%|          | 14502/1807941 [00:11<24:58, 1197.19 examples/s]
Tokenizing train dataset:   1%|          | 14623/1807941 [00:12<24:56, 1198.15 examples/s]
Tokenizing train dataset:   1%|          | 14807/1807941 [00:12<24:51, 1201.91 examples/s]
Tokenizing train dataset:   1%|          | 14928/1807941 [00:12<24:50, 1203.11 examples/s]
Tokenizing train dataset:   1%|          | 15087/1807941 [00:12<26:01, 1148.45 examples/s]
Tokenizing train dataset:   1%|          | 15204/1807941 [00:12<26:02, 1147.21 examples/s]
Tokenizing train dataset:   1%|          | 15374/1807941 [00:12<26:24, 1131.16 examples/s]
Tokenizing train dataset:   1%|          | 15516/1807941 [00:12<28:01, 1065.98 examples/s]
Tokenizing train dataset:   1%|          | 15656/1807941 [00:12<29:15, 1021.14 examples/s]
Tokenizing train dataset:   1%|          | 15771/1807941 [00:13<28:29, 1048.62 examples/s]
Tokenizing train dataset:   1%|          | 15881/1807941 [00:13<28:09, 1060.51 examples/s]
Tokenizing train dataset:   1%|          | 16004/1807941 [00:13<30:44, 971.61 examples/s] 
Tokenizing train dataset:   1%|          | 16104/1807941 [00:13<30:33, 977.06 examples/s]
Tokenizing train dataset:   1%|          | 16219/1807941 [00:13<29:17, 1019.61 examples/s]
Tokenizing train dataset:   1%|          | 16327/1807941 [00:13<28:52, 1033.83 examples/s]
Tokenizing train dataset:   1%|          | 16437/1807941 [00:13<28:23, 1051.46 examples/s]
Tokenizing train dataset:   1%|          | 16548/1807941 [00:13<28:05, 1062.87 examples/s]
Tokenizing train dataset:   1%|          | 16699/1807941 [00:13<28:47, 1036.80 examples/s]
Tokenizing train dataset:   1%|          | 16868/1807941 [00:14<28:00, 1065.64 examples/s]
Tokenizing train dataset:   1%|          | 16983/1807941 [00:14<27:32, 1083.85 examples/s]
Tokenizing train dataset:   1%|          | 17120/1807941 [00:14<29:11, 1022.17 examples/s]
Tokenizing train dataset:   1%|          | 17252/1807941 [00:14<27:14, 1095.49 examples/s]
Tokenizing train dataset:   1%|          | 17377/1807941 [00:14<26:23, 1131.04 examples/s]
Tokenizing train dataset:   1%|          | 17499/1807941 [00:14<25:53, 1152.87 examples/s]
Tokenizing train dataset:   1%|          | 17621/1807941 [00:14<25:36, 1165.47 examples/s]
Tokenizing train dataset:   1%|          | 17799/1807941 [00:14<25:31, 1168.64 examples/s]
Tokenizing train dataset:   1%|          | 17919/1807941 [00:15<25:23, 1175.27 examples/s]
Tokenizing train dataset:   1%|          | 18084/1807941 [00:15<26:01, 1146.10 examples/s]
Tokenizing train dataset:   1%|          | 18253/1807941 [00:15<26:13, 1137.56 examples/s]
Tokenizing train dataset:   1%|          | 18384/1807941 [00:15<25:21, 1176.50 examples/s]
Tokenizing train dataset:   1%|          | 18511/1807941 [00:15<24:55, 1196.76 examples/s]
Tokenizing train dataset:   1%|          | 18687/1807941 [00:15<25:08, 1186.07 examples/s]
Tokenizing train dataset:   1%|          | 18818/1807941 [00:15<24:33, 1214.34 examples/s]
Tokenizing train dataset:   1%|          | 18998/1807941 [00:15<24:45, 1204.25 examples/s]
Tokenizing train dataset:   1%|          | 19133/1807941 [00:16<27:10, 1097.14 examples/s]
Tokenizing train dataset:   1%|          | 19269/1807941 [00:16<25:43, 1158.58 examples/s]
Tokenizing train dataset:   1%|          | 19395/1807941 [00:16<25:14, 1180.94 examples/s]
Tokenizing train dataset:   1%|          | 19518/1807941 [00:16<25:01, 1191.09 examples/s]
Tokenizing train dataset:   1%|          | 19646/1807941 [00:16<24:33, 1213.30 examples/s]
Tokenizing train dataset:   1%|          | 19824/1807941 [00:16<24:51, 1198.90 examples/s]
Tokenizing train dataset:   1%|          | 19949/1807941 [00:16<24:39, 1208.86 examples/s]
Tokenizing train dataset:   1%|          | 20119/1807941 [00:16<26:15, 1134.82 examples/s]
Tokenizing train dataset:   1%|          | 20241/1807941 [00:17<25:52, 1151.15 examples/s]
Tokenizing train dataset:   1%|          | 20418/1807941 [00:17<25:42, 1158.64 examples/s]
Tokenizing train dataset:   1%|          | 20559/1807941 [00:17<27:29, 1083.38 examples/s]
Tokenizing train dataset:   1%|          | 20719/1807941 [00:17<27:40, 1076.05 examples/s]
Tokenizing train dataset:   1%|          | 20887/1807941 [00:17<27:24, 1086.94 examples/s]
Tokenizing train dataset:   1%|          | 21000/1807941 [00:17<29:30, 1009.41 examples/s]
Tokenizing train dataset:   1%|          | 21109/1807941 [00:17<29:03, 1024.86 examples/s]
Tokenizing train dataset:   1%|          | 21221/1807941 [00:17<28:24, 1048.20 examples/s]
Tokenizing train dataset:   1%|          | 21338/1807941 [00:18<27:39, 1076.81 examples/s]
Tokenizing train dataset:   1%|          | 21455/1807941 [00:18<27:03, 1100.49 examples/s]
Tokenizing train dataset:   1%|          | 21581/1807941 [00:18<26:05, 1141.40 examples/s]
Tokenizing train dataset:   1%|          | 21702/1807941 [00:18<25:41, 1158.94 examples/s]
Tokenizing train dataset:   1%|          | 21844/1807941 [00:18<24:09, 1231.82 examples/s]
Tokenizing train dataset:   1%|          | 22000/1807941 [00:18<27:02, 1100.84 examples/s]
Tokenizing train dataset:   1%|          | 22124/1807941 [00:18<26:16, 1132.74 examples/s]
Tokenizing train dataset:   1%|          | 22245/1807941 [00:18<25:50, 1151.40 examples/s]
Tokenizing train dataset:   1%|          | 22363/1807941 [00:18<25:46, 1154.74 examples/s]
Tokenizing train dataset:   1%|          | 22484/1807941 [00:19<25:28, 1168.47 examples/s]
Tokenizing train dataset:   1%|▏         | 22608/1807941 [00:19<25:03, 1187.06 examples/s]
Tokenizing train dataset:   1%|▏         | 22778/1807941 [00:19<25:34, 1163.15 examples/s]
Tokenizing train dataset:   1%|▏         | 22954/1807941 [00:19<25:36, 1161.71 examples/s]
Tokenizing train dataset:   1%|▏         | 23099/1807941 [00:19<27:58, 1063.46 examples/s]
Tokenizing train dataset:   1%|▏         | 23264/1807941 [00:19<27:44, 1072.40 examples/s]
Tokenizing train dataset:   1%|▏         | 23387/1807941 [00:19<26:51, 1107.28 examples/s]
Tokenizing train dataset:   1%|▏         | 23514/1807941 [00:19<25:58, 1144.89 examples/s]
Tokenizing train dataset:   1%|▏         | 23643/1807941 [00:20<25:11, 1180.72 examples/s]
Tokenizing train dataset:   1%|▏         | 23817/1807941 [00:20<25:26, 1168.79 examples/s]
Tokenizing train dataset:   1%|▏         | 23936/1807941 [00:20<25:23, 1171.25 examples/s]
Tokenizing train dataset:   1%|▏         | 24077/1807941 [00:20<27:25, 1084.12 examples/s]
Tokenizing train dataset:   1%|▏         | 24205/1807941 [00:20<26:17, 1131.09 examples/s]
Tokenizing train dataset:   1%|▏         | 24329/1807941 [00:20<25:43, 1155.56 examples/s]
Tokenizing train dataset:   1%|▏         | 24448/1807941 [00:20<25:33, 1162.77 examples/s]
Tokenizing train dataset:   1%|▏         | 24567/1807941 [00:20<25:28, 1166.87 examples/s]
Tokenizing train dataset:   1%|▏         | 24688/1807941 [00:20<25:14, 1177.43 examples/s]
Tokenizing train dataset:   1%|▏         | 24839/1807941 [00:21<26:46, 1110.25 examples/s]
Tokenizing train dataset:   1%|▏         | 24961/1807941 [00:21<26:09, 1136.20 examples/s]
Tokenizing train dataset:   1%|▏         | 25100/1807941 [00:21<28:25, 1045.52 examples/s]
Tokenizing train dataset:   1%|▏         | 25265/1807941 [00:21<28:01, 1060.24 examples/s]
Tokenizing train dataset:   1%|▏         | 25398/1807941 [00:21<26:24, 1125.06 examples/s]
Tokenizing train dataset:   1%|▏         | 25536/1807941 [00:21<25:00, 1187.54 examples/s]
Tokenizing train dataset:   1%|▏         | 25671/1807941 [00:21<24:11, 1227.94 examples/s]
Tokenizing train dataset:   1%|▏         | 25861/1807941 [00:22<23:57, 1239.57 examples/s]
Tokenizing train dataset:   1%|▏         | 26000/1807941 [00:22<24:21, 1218.87 examples/s]
Tokenizing train dataset:   1%|▏         | 26176/1807941 [00:22<21:52, 1357.33 examples/s]
Tokenizing train dataset:   1%|▏         | 26360/1807941 [00:22<22:43, 1306.61 examples/s]
Tokenizing train dataset:   1%|▏         | 26563/1807941 [00:22<22:31, 1318.23 examples/s]
Tokenizing train dataset:   1%|▏         | 26707/1807941 [00:22<22:04, 1344.91 examples/s]
Tokenizing train dataset:   1%|▏         | 26846/1807941 [00:22<21:55, 1353.66 examples/s]
Tokenizing train dataset:   1%|▏         | 27023/1807941 [00:22<22:59, 1290.74 examples/s]
Tokenizing train dataset:   2%|▏         | 27219/1807941 [00:23<22:57, 1293.04 examples/s]
Tokenizing train dataset:   2%|▏         | 27352/1807941 [00:23<22:47, 1301.69 examples/s]
Tokenizing train dataset:   2%|▏         | 27548/1807941 [00:23<22:49, 1299.61 examples/s]
Tokenizing train dataset:   2%|▏         | 27687/1807941 [00:23<22:27, 1320.92 examples/s]
Tokenizing train dataset:   2%|▏         | 27830/1807941 [00:23<22:00, 1348.00 examples/s]
Tokenizing train dataset:   2%|▏         | 27969/1807941 [00:23<21:50, 1358.14 examples/s]
Tokenizing train dataset:   2%|▏         | 28141/1807941 [00:23<23:12, 1278.12 examples/s]
Tokenizing train dataset:   2%|▏         | 28283/1807941 [00:23<22:38, 1310.28 examples/s]
Tokenizing train dataset:   2%|▏         | 28417/1807941 [00:23<22:31, 1316.73 examples/s]
Tokenizing train dataset:   2%|▏         | 28603/1807941 [00:24<23:04, 1284.87 examples/s]
Tokenizing train dataset:   2%|▏         | 28754/1807941 [00:24<22:07, 1340.75 examples/s]
Tokenizing train dataset:   2%|▏         | 28905/1807941 [00:24<21:26, 1383.37 examples/s]
Tokenizing train dataset:   2%|▏         | 29091/1807941 [00:24<22:22, 1325.00 examples/s]
Tokenizing train dataset:   2%|▏         | 29239/1807941 [00:24<21:45, 1362.32 examples/s]
Tokenizing train dataset:   2%|▏         | 29390/1807941 [00:24<21:12, 1397.21 examples/s]
Tokenizing train dataset:   2%|▏         | 29548/1807941 [00:24<20:31, 1443.85 examples/s]
Tokenizing train dataset:   2%|▏         | 29729/1807941 [00:24<21:57, 1350.06 examples/s]
Tokenizing train dataset:   2%|▏         | 29906/1807941 [00:25<23:04, 1284.33 examples/s]
Tokenizing train dataset:   2%|▏         | 30057/1807941 [00:25<25:45, 1150.11 examples/s]
Tokenizing train dataset:   2%|▏         | 30181/1807941 [00:25<25:22, 1167.46 examples/s]
Tokenizing train dataset:   2%|▏         | 30332/1807941 [00:25<23:40, 1251.16 examples/s]
Tokenizing train dataset:   2%|▏         | 30479/1807941 [00:25<22:40, 1306.62 examples/s]
Tokenizing train dataset:   2%|▏         | 30668/1807941 [00:25<23:01, 1286.23 examples/s]
Tokenizing train dataset:   2%|▏         | 30852/1807941 [00:25<23:25, 1264.51 examples/s]
Tokenizing train dataset:   2%|▏         | 31000/1807941 [00:25<25:50, 1146.05 examples/s]
Tokenizing train dataset:   2%|▏         | 31118/1807941 [00:26<25:43, 1151.52 examples/s]
Tokenizing train dataset:   2%|▏         | 31293/1807941 [00:26<25:39, 1153.73 examples/s]
Tokenizing train dataset:   2%|▏         | 31413/1807941 [00:26<25:29, 1161.64 examples/s]
Tokenizing train dataset:   2%|▏         | 31537/1807941 [00:26<25:05, 1180.26 examples/s]
Tokenizing train dataset:   2%|▏         | 31674/1807941 [00:26<24:04, 1230.01 examples/s]
Tokenizing train dataset:   2%|▏         | 31849/1807941 [00:26<24:33, 1205.06 examples/s]
Tokenizing train dataset:   2%|▏         | 31978/1807941 [00:26<24:12, 1222.75 examples/s]
Tokenizing train dataset:   2%|▏         | 32141/1807941 [00:26<25:18, 1169.77 examples/s]
Tokenizing train dataset:   2%|▏         | 32289/1807941 [00:27<23:44, 1246.29 examples/s]
Tokenizing train dataset:   2%|▏         | 32473/1807941 [00:27<23:57, 1234.98 examples/s]
Tokenizing train dataset:   2%|▏         | 32656/1807941 [00:27<24:05, 1227.73 examples/s]
Tokenizing train dataset:   2%|▏         | 32784/1807941 [00:27<23:54, 1237.09 examples/s]
Tokenizing train dataset:   2%|▏         | 32974/1807941 [00:27<23:47, 1243.42 examples/s]
Tokenizing train dataset:   2%|▏         | 33141/1807941 [00:27<24:44, 1195.60 examples/s]
Tokenizing train dataset:   2%|▏         | 33309/1807941 [00:27<25:17, 1169.06 examples/s]
Tokenizing train dataset:   2%|▏         | 33488/1807941 [00:28<25:14, 1171.75 examples/s]
Tokenizing train dataset:   2%|▏         | 33663/1807941 [00:28<25:21, 1166.30 examples/s]
Tokenizing train dataset:   2%|▏         | 33790/1807941 [00:28<24:53, 1187.69 examples/s]
Tokenizing train dataset:   2%|▏         | 33915/1807941 [00:28<24:36, 1201.76 examples/s]
Tokenizing train dataset:   2%|▏         | 34071/1807941 [00:28<25:50, 1143.98 examples/s]
Tokenizing train dataset:   2%|▏         | 34203/1807941 [00:28<24:58, 1183.67 examples/s]
Tokenizing train dataset:   2%|▏         | 34348/1807941 [00:28<23:39, 1249.68 examples/s]
Tokenizing train dataset:   2%|▏         | 34485/1807941 [00:28<23:04, 1280.71 examples/s]
Tokenizing train dataset:   2%|▏         | 34644/1807941 [00:28<21:39, 1364.62 examples/s]
Tokenizing train dataset:   2%|▏         | 34840/1807941 [00:29<22:02, 1340.65 examples/s]
Tokenizing train dataset:   2%|▏         | 35000/1807941 [00:29<25:05, 1177.41 examples/s]
Tokenizing train dataset:   2%|▏         | 35148/1807941 [00:29<26:32, 1112.87 examples/s]
Tokenizing train dataset:   2%|▏         | 35265/1807941 [00:29<26:18, 1122.83 examples/s]
Tokenizing train dataset:   2%|▏         | 35386/1807941 [00:29<25:51, 1142.46 examples/s]
Tokenizing train dataset:   2%|▏         | 35506/1807941 [00:29<25:34, 1155.37 examples/s]
Tokenizing train dataset:   2%|▏         | 35625/1807941 [00:29<25:26, 1160.79 examples/s]
Tokenizing train dataset:   2%|▏         | 35752/1807941 [00:29<24:52, 1187.72 examples/s]
Tokenizing train dataset:   2%|▏         | 35874/1807941 [00:30<24:44, 1193.54 examples/s]
Tokenizing train dataset:   2%|▏         | 35999/1807941 [00:30<24:30, 1204.98 examples/s]
Tokenizing train dataset:   2%|▏         | 36155/1807941 [00:30<25:54, 1139.88 examples/s]
Tokenizing train dataset:   2%|▏         | 36317/1807941 [00:30<26:28, 1115.04 examples/s]
Tokenizing train dataset:   2%|▏         | 36469/1807941 [00:30<27:25, 1076.44 examples/s]
Tokenizing train dataset:   2%|▏         | 36600/1807941 [00:30<26:06, 1130.96 examples/s]
Tokenizing train dataset:   2%|▏         | 36718/1807941 [00:30<25:50, 1142.11 examples/s]
Tokenizing train dataset:   2%|▏         | 36839/1807941 [00:30<25:32, 1155.46 examples/s]
Tokenizing train dataset:   2%|▏         | 36962/1807941 [00:30<25:10, 1172.65 examples/s]
Tokenizing train dataset:   2%|▏         | 37130/1807941 [00:31<26:13, 1125.59 examples/s]
Tokenizing train dataset:   2%|▏         | 37256/1807941 [00:31<25:30, 1156.64 examples/s]
Tokenizing train dataset:   2%|▏         | 37385/1807941 [00:31<24:50, 1188.17 examples/s]
Tokenizing train dataset:   2%|▏         | 37506/1807941 [00:31<24:44, 1192.26 examples/s]
Tokenizing train dataset:   2%|▏         | 37631/1807941 [00:31<24:26, 1207.53 examples/s]
Tokenizing train dataset:   2%|▏         | 37814/1807941 [00:31<24:22, 1210.64 examples/s]
Tokenizing train dataset:   2%|▏         | 37949/1807941 [00:31<23:42, 1243.98 examples/s]
Tokenizing train dataset:   2%|▏         | 38079/1807941 [00:31<23:58, 1230.43 examples/s]
Tokenizing train dataset:   2%|▏         | 38223/1807941 [00:32<22:55, 1286.38 examples/s]
Tokenizing train dataset:   2%|▏         | 38409/1807941 [00:32<23:19, 1264.04 examples/s]
Tokenizing train dataset:   2%|▏         | 38545/1807941 [00:32<22:55, 1286.74 examples/s]
Tokenizing train dataset:   2%|▏         | 38681/1807941 [00:32<22:38, 1302.30 examples/s]
Tokenizing train dataset:   2%|▏         | 38866/1807941 [00:32<23:10, 1272.33 examples/s]
Tokenizing train dataset:   2%|▏         | 39019/1807941 [00:32<24:56, 1182.33 examples/s]
Tokenizing train dataset:   2%|▏         | 39148/1807941 [00:32<24:27, 1205.10 examples/s]
Tokenizing train dataset:   2%|▏         | 39280/1807941 [00:32<23:55, 1232.08 examples/s]
Tokenizing train dataset:   2%|▏         | 39466/1807941 [00:33<23:55, 1231.85 examples/s]
Tokenizing train dataset:   2%|▏         | 39642/1807941 [00:33<24:20, 1210.36 examples/s]
Tokenizing train dataset:   2%|▏         | 39766/1807941 [00:33<24:15, 1214.92 examples/s]
Tokenizing train dataset:   2%|▏         | 39892/1807941 [00:33<24:09, 1220.16 examples/s]
Tokenizing train dataset:   2%|▏         | 40060/1807941 [00:33<25:33, 1153.09 examples/s]
Tokenizing train dataset:   2%|▏         | 40218/1807941 [00:33<26:23, 1116.03 examples/s]
Tokenizing train dataset:   2%|▏         | 40392/1807941 [00:33<26:07, 1127.29 examples/s]
Tokenizing train dataset:   2%|▏         | 40521/1807941 [00:33<25:20, 1162.36 examples/s]
Tokenizing train dataset:   2%|▏         | 40645/1807941 [00:34<24:58, 1179.64 examples/s]
Tokenizing train dataset:   2%|▏         | 40771/1807941 [00:34<24:32, 1199.96 examples/s]
Tokenizing train dataset:   2%|▏         | 40896/1807941 [00:34<24:18, 1211.21 examples/s]
Tokenizing train dataset:   2%|▏         | 41061/1807941 [00:34<25:46, 1142.45 examples/s]
Tokenizing train dataset:   2%|▏         | 41191/1807941 [00:34<24:58, 1178.95 examples/s]
Tokenizing train dataset:   2%|▏         | 41364/1807941 [00:34<25:16, 1164.87 examples/s]
Tokenizing train dataset:   2%|▏         | 41487/1807941 [00:34<24:57, 1179.28 examples/s]
Tokenizing train dataset:   2%|▏         | 41650/1807941 [00:34<25:44, 1143.83 examples/s]
Tokenizing train dataset:   2%|▏         | 41772/1807941 [00:35<25:21, 1160.55 examples/s]
Tokenizing train dataset:   2%|▏         | 41900/1807941 [00:35<24:43, 1190.37 examples/s]
Tokenizing train dataset:   2%|▏         | 42061/1807941 [00:35<28:06, 1046.91 examples/s]
Tokenizing train dataset:   2%|▏         | 42177/1807941 [00:35<27:27, 1071.47 examples/s]
Tokenizing train dataset:   2%|▏         | 42304/1807941 [00:35<26:18, 1118.53 examples/s]
Tokenizing train dataset:   2%|▏         | 42452/1807941 [00:35<24:15, 1212.87 examples/s]
Tokenizing train dataset:   2%|▏         | 42593/1807941 [00:35<23:14, 1266.25 examples/s]
Tokenizing train dataset:   2%|▏         | 42779/1807941 [00:35<23:29, 1252.39 examples/s]
Tokenizing train dataset:   2%|▏         | 42954/1807941 [00:36<24:06, 1220.44 examples/s]
Tokenizing train dataset:   2%|▏         | 43114/1807941 [00:36<26:09, 1124.18 examples/s]
Tokenizing train dataset:   2%|▏         | 43291/1807941 [00:36<25:50, 1138.36 examples/s]
Tokenizing train dataset:   2%|▏         | 43425/1807941 [00:36<24:52, 1182.16 examples/s]
Tokenizing train dataset:   2%|▏         | 43548/1807941 [00:36<24:41, 1191.19 examples/s]
Tokenizing train dataset:   2%|▏         | 43679/1807941 [00:36<24:11, 1215.26 examples/s]
Tokenizing train dataset:   2%|▏         | 43862/1807941 [00:36<24:14, 1212.46 examples/s]
Tokenizing train dataset:   2%|▏         | 44000/1807941 [00:36<24:44, 1187.85 examples/s]
Tokenizing train dataset:   2%|▏         | 44122/1807941 [00:37<24:38, 1192.98 examples/s]
Tokenizing train dataset:   2%|▏         | 44268/1807941 [00:37<23:19, 1260.23 examples/s]
Tokenizing train dataset:   2%|▏         | 44454/1807941 [00:37<23:32, 1248.50 examples/s]
Tokenizing train dataset:   2%|▏         | 44580/1807941 [00:37<23:31, 1248.94 examples/s]
Tokenizing train dataset:   2%|▏         | 44768/1807941 [00:37<23:32, 1247.92 examples/s]
Tokenizing train dataset:   2%|▏         | 44896/1807941 [00:37<23:25, 1254.60 examples/s]
Tokenizing train dataset:   2%|▏         | 45063/1807941 [00:37<25:07, 1169.52 examples/s]
Tokenizing train dataset:   2%|▏         | 45184/1807941 [00:37<24:58, 1176.20 examples/s]
Tokenizing train dataset:   3%|▎         | 45309/1807941 [00:37<24:36, 1193.62 examples/s]
Tokenizing train dataset:   3%|▎         | 45437/1807941 [00:38<24:10, 1215.29 examples/s]
Tokenizing train dataset:   3%|▎         | 45561/1807941 [00:38<24:05, 1219.45 examples/s]
Tokenizing train dataset:   3%|▎         | 45688/1807941 [00:38<23:52, 1230.48 examples/s]
Tokenizing train dataset:   3%|▎         | 45837/1807941 [00:38<25:49, 1136.96 examples/s]
Tokenizing train dataset:   3%|▎         | 46000/1807941 [00:38<27:59, 1049.37 examples/s]
Tokenizing train dataset:   3%|▎         | 46135/1807941 [00:38<26:14, 1118.63 examples/s]
Tokenizing train dataset:   3%|▎         | 46261/1807941 [00:38<25:30, 1150.80 examples/s]
Tokenizing train dataset:   3%|▎         | 46397/1807941 [00:38<24:21, 1205.42 examples/s]
Tokenizing train dataset:   3%|▎         | 46541/1807941 [00:39<23:08, 1268.54 examples/s]
Tokenizing train dataset:   3%|▎         | 46687/1807941 [00:39<22:14, 1319.63 examples/s]
Tokenizing train dataset:   3%|▎         | 46882/1807941 [00:39<22:23, 1310.58 examples/s]
Tokenizing train dataset:   3%|▎         | 47075/1807941 [00:39<23:21, 1256.83 examples/s]
Tokenizing train dataset:   3%|▎         | 47211/1807941 [00:39<22:55, 1280.06 examples/s]
Tokenizing train dataset:   3%|▎         | 47347/1807941 [00:39<22:38, 1295.77 examples/s]
Tokenizing train dataset:   3%|▎         | 47485/1807941 [00:39<22:16, 1316.85 examples/s]
Tokenizing train dataset:   3%|▎         | 47689/1807941 [00:39<22:04, 1329.42 examples/s]
Tokenizing train dataset:   3%|▎         | 47853/1807941 [00:39<20:51, 1405.91 examples/s]
Tokenizing train dataset:   3%|▎         | 47998/1807941 [00:40<20:44, 1414.57 examples/s]
Tokenizing train dataset:   3%|▎         | 48178/1807941 [00:40<22:01, 1332.12 examples/s]
Tokenizing train dataset:   3%|▎         | 48327/1807941 [00:40<21:22, 1371.90 examples/s]
Tokenizing train dataset:   3%|▎         | 48476/1807941 [00:40<20:56, 1399.84 examples/s]
Tokenizing train dataset:   3%|▎         | 48619/1807941 [00:40<20:52, 1404.69 examples/s]
Tokenizing train dataset:   3%|▎         | 48826/1807941 [00:40<21:01, 1394.47 examples/s]
Tokenizing train dataset:   3%|▎         | 48969/1807941 [00:40<20:56, 1400.38 examples/s]
Tokenizing train dataset:   3%|▎         | 49147/1807941 [00:40<22:13, 1318.76 examples/s]
Tokenizing train dataset:   3%|▎         | 49288/1807941 [00:41<21:56, 1336.10 examples/s]
Tokenizing train dataset:   3%|▎         | 49469/1807941 [00:41<20:04, 1459.87 examples/s]
Tokenizing train dataset:   3%|▎         | 49662/1807941 [00:41<18:27, 1587.03 examples/s]
Tokenizing train dataset:   3%|▎         | 49888/1807941 [00:41<18:52, 1552.42 examples/s]
Tokenizing train dataset:   3%|▎         | 50078/1807941 [00:41<20:13, 1449.18 examples/s]
Tokenizing train dataset:   3%|▎         | 50245/1807941 [00:41<21:58, 1333.22 examples/s]
Tokenizing train dataset:   3%|▎         | 50393/1807941 [00:41<21:27, 1364.86 examples/s]
Tokenizing train dataset:   3%|▎         | 50534/1807941 [00:41<21:21, 1371.40 examples/s]
Tokenizing train dataset:   3%|▎         | 50739/1807941 [00:42<21:23, 1368.76 examples/s]
Tokenizing train dataset:   3%|▎         | 50888/1807941 [00:42<20:58, 1396.61 examples/s]
Tokenizing train dataset:   3%|▎         | 51070/1807941 [00:42<22:18, 1312.78 examples/s]
Tokenizing train dataset:   3%|▎         | 51256/1807941 [00:42<22:48, 1283.26 examples/s]
Tokenizing train dataset:   3%|▎         | 51393/1807941 [00:42<22:28, 1302.84 examples/s]
Tokenizing train dataset:   3%|▎         | 51532/1807941 [00:42<22:09, 1321.48 examples/s]
Tokenizing train dataset:   3%|▎         | 51711/1807941 [00:42<23:00, 1272.44 examples/s]
Tokenizing train dataset:   3%|▎         | 51859/1807941 [00:42<24:57, 1172.86 examples/s]
Tokenizing train dataset:   3%|▎         | 52000/1807941 [00:43<26:55, 1086.97 examples/s]
Tokenizing train dataset:   3%|▎         | 52130/1807941 [00:43<25:48, 1133.55 examples/s]
Tokenizing train dataset:   3%|▎         | 52258/1807941 [00:43<25:00, 1169.77 examples/s]
Tokenizing train dataset:   3%|▎         | 52386/1807941 [00:43<24:26, 1197.41 examples/s]
Tokenizing train dataset:   3%|▎         | 52565/1807941 [00:43<24:31, 1192.77 examples/s]
Tokenizing train dataset:   3%|▎         | 52693/1807941 [00:43<24:07, 1212.82 examples/s]
Tokenizing train dataset:   3%|▎         | 52877/1807941 [00:43<24:02, 1216.36 examples/s]
Tokenizing train dataset:   3%|▎         | 53018/1807941 [00:43<26:06, 1120.49 examples/s]
Tokenizing train dataset:   3%|▎         | 53147/1807941 [00:44<25:16, 1157.15 examples/s]
Tokenizing train dataset:   3%|▎         | 53280/1807941 [00:44<24:20, 1201.14 examples/s]
Tokenizing train dataset:   3%|▎         | 53410/1807941 [00:44<23:52, 1224.72 examples/s]
Tokenizing train dataset:   3%|▎         | 53537/1807941 [00:44<23:41, 1234.12 examples/s]
Tokenizing train dataset:   3%|▎         | 53678/1807941 [00:44<22:50, 1280.31 examples/s]
Tokenizing train dataset:   3%|▎         | 53813/1807941 [00:44<22:30, 1298.56 examples/s]
Tokenizing train dataset:   3%|▎         | 54000/1807941 [00:44<23:47, 1228.54 examples/s]
Tokenizing train dataset:   3%|▎         | 54170/1807941 [00:44<24:28, 1194.47 examples/s]
Tokenizing train dataset:   3%|▎         | 54356/1807941 [00:45<24:12, 1207.59 examples/s]
Tokenizing train dataset:   3%|▎         | 54490/1807941 [00:45<23:35, 1238.61 examples/s]
Tokenizing train dataset:   3%|▎         | 54630/1807941 [00:45<22:54, 1275.97 examples/s]
Tokenizing train dataset:   3%|▎         | 54760/1807941 [00:45<22:50, 1279.50 examples/s]
Tokenizing train dataset:   3%|▎         | 54952/1807941 [00:45<22:52, 1277.58 examples/s]
Tokenizing train dataset:   3%|▎         | 55087/1807941 [00:45<25:28, 1146.80 examples/s]
Tokenizing train dataset:   3%|▎         | 55260/1807941 [00:45<25:28, 1146.55 examples/s]
Tokenizing train dataset:   3%|▎         | 55437/1807941 [00:45<25:19, 1153.59 examples/s]
Tokenizing train dataset:   3%|▎         | 55563/1807941 [00:46<24:50, 1175.98 examples/s]
Tokenizing train dataset:   3%|▎         | 55692/1807941 [00:46<24:18, 1201.27 examples/s]
Tokenizing train dataset:   3%|▎         | 55817/1807941 [00:46<24:06, 1211.43 examples/s]
Tokenizing train dataset:   3%|▎         | 55947/1807941 [00:46<23:41, 1232.21 examples/s]
Tokenizing train dataset:   3%|▎         | 56072/1807941 [00:46<24:12, 1206.47 examples/s]
Tokenizing train dataset:   3%|▎         | 56202/1807941 [00:46<23:45, 1228.56 examples/s]
Tokenizing train dataset:   3%|▎         | 56336/1807941 [00:46<23:14, 1256.13 examples/s]
Tokenizing train dataset:   3%|▎         | 56467/1807941 [00:46<23:00, 1269.09 examples/s]
Tokenizing train dataset:   3%|▎         | 56607/1807941 [00:46<22:22, 1304.32 examples/s]
Tokenizing train dataset:   3%|▎         | 56796/1807941 [00:47<22:46, 1281.64 examples/s]
Tokenizing train dataset:   3%|▎         | 56980/1807941 [00:47<23:12, 1257.59 examples/s]
Tokenizing train dataset:   3%|▎         | 57146/1807941 [00:47<24:14, 1203.95 examples/s]
Tokenizing train dataset:   3%|▎         | 57281/1807941 [00:47<23:36, 1235.64 examples/s]
Tokenizing train dataset:   3%|▎         | 57408/1807941 [00:47<23:28, 1243.20 examples/s]
Tokenizing train dataset:   3%|▎         | 57574/1807941 [00:47<24:34, 1187.49 examples/s]
Tokenizing train dataset:   3%|▎         | 57751/1807941 [00:47<24:40, 1182.09 examples/s]
Tokenizing train dataset:   3%|▎         | 57918/1807941 [00:47<25:10, 1158.44 examples/s]
Tokenizing train dataset:   3%|▎         | 58085/1807941 [00:48<25:30, 1143.14 examples/s]
Tokenizing train dataset:   3%|▎         | 58245/1807941 [00:48<26:08, 1115.20 examples/s]
Tokenizing train dataset:   3%|▎         | 58418/1807941 [00:48<25:55, 1124.53 examples/s]
Tokenizing train dataset:   3%|▎         | 58547/1807941 [00:48<25:10, 1158.50 examples/s]
Tokenizing train dataset:   3%|▎         | 58674/1807941 [00:48<24:37, 1184.08 examples/s]
Tokenizing train dataset:   3%|▎         | 58798/1807941 [00:48<24:20, 1197.45 examples/s]
Tokenizing train dataset:   3%|▎         | 58926/1807941 [00:48<23:59, 1215.00 examples/s]
Tokenizing train dataset:   3%|▎         | 59088/1807941 [00:48<25:03, 1163.17 examples/s]
Tokenizing train dataset:   3%|▎         | 59268/1807941 [00:49<24:53, 1171.10 examples/s]
Tokenizing train dataset:   3%|▎         | 59391/1807941 [00:49<24:37, 1183.54 examples/s]
Tokenizing train dataset:   3%|▎         | 59525/1807941 [00:49<23:53, 1219.47 examples/s]
Tokenizing train dataset:   3%|▎         | 59715/1807941 [00:49<23:37, 1233.45 examples/s]
Tokenizing train dataset:   3%|▎         | 59845/1807941 [00:49<23:20, 1248.42 examples/s]
Tokenizing train dataset:   3%|▎         | 59977/1807941 [00:49<23:02, 1264.12 examples/s]
Tokenizing train dataset:   3%|▎         | 60129/1807941 [00:49<24:53, 1170.23 examples/s]
Tokenizing train dataset:   3%|▎         | 60307/1807941 [00:49<24:51, 1171.46 examples/s]
Tokenizing train dataset:   3%|▎         | 60484/1807941 [00:50<24:54, 1169.41 examples/s]
Tokenizing train dataset:   3%|▎         | 60615/1807941 [00:50<24:17, 1199.03 examples/s]
Tokenizing train dataset:   3%|▎         | 60746/1807941 [00:50<23:50, 1221.56 examples/s]
Tokenizing train dataset:   3%|▎         | 60872/1807941 [00:50<23:41, 1228.93 examples/s]
Tokenizing train dataset:   3%|▎         | 61000/1807941 [00:50<24:51, 1170.89 examples/s]
Tokenizing train dataset:   3%|▎         | 61132/1807941 [00:50<24:05, 1208.20 examples/s]
Tokenizing train dataset:   3%|▎         | 61256/1807941 [00:50<23:56, 1215.93 examples/s]
Tokenizing train dataset:   3%|▎         | 61390/1807941 [00:50<23:21, 1245.77 examples/s]
Tokenizing train dataset:   3%|▎         | 61567/1807941 [00:51<23:54, 1217.41 examples/s]
Tokenizing train dataset:   3%|▎         | 61701/1807941 [00:51<23:21, 1246.10 examples/s]
Tokenizing train dataset:   3%|▎         | 61883/1807941 [00:51<23:39, 1229.64 examples/s]
Tokenizing train dataset:   3%|▎         | 62058/1807941 [00:51<25:01, 1162.42 examples/s]
Tokenizing train dataset:   3%|▎         | 62187/1807941 [00:51<24:24, 1191.93 examples/s]
Tokenizing train dataset:   3%|▎         | 62313/1807941 [00:51<24:04, 1208.22 examples/s]
Tokenizing train dataset:   3%|▎         | 62499/1807941 [00:51<23:55, 1215.56 examples/s]
Tokenizing train dataset:   3%|▎         | 62632/1807941 [00:51<23:26, 1240.67 examples/s]
Tokenizing train dataset:   3%|▎         | 62816/1807941 [00:52<23:36, 1232.17 examples/s]
Tokenizing train dataset:   3%|▎         | 62998/1807941 [00:52<23:51, 1218.68 examples/s]
Tokenizing train dataset:   3%|▎         | 63163/1807941 [00:52<24:43, 1176.47 examples/s]
Tokenizing train dataset:   4%|▎         | 63292/1807941 [00:52<24:12, 1201.36 examples/s]
Tokenizing train dataset:   4%|▎         | 63480/1807941 [00:52<23:57, 1213.36 examples/s]
Tokenizing train dataset:   4%|▎         | 63623/1807941 [00:52<23:00, 1263.43 examples/s]
Tokenizing train dataset:   4%|▎         | 63800/1807941 [00:52<23:36, 1231.03 examples/s]
Tokenizing train dataset:   4%|▎         | 63938/1807941 [00:52<22:58, 1265.30 examples/s]
Tokenizing train dataset:   4%|▎         | 64120/1807941 [00:53<24:18, 1196.02 examples/s]
Tokenizing train dataset:   4%|▎         | 64305/1807941 [00:53<24:10, 1202.02 examples/s]
Tokenizing train dataset:   4%|▎         | 64485/1807941 [00:53<24:16, 1196.63 examples/s]
Tokenizing train dataset:   4%|▎         | 64639/1807941 [00:53<25:26, 1142.19 examples/s]
Tokenizing train dataset:   4%|▎         | 64813/1807941 [00:53<25:23, 1144.34 examples/s]
Tokenizing train dataset:   4%|▎         | 64945/1807941 [00:53<24:34, 1181.70 examples/s]
Tokenizing train dataset:   4%|▎         | 65123/1807941 [00:54<25:15, 1149.78 examples/s]
Tokenizing train dataset:   4%|▎         | 65255/1807941 [00:54<24:28, 1186.66 examples/s]
Tokenizing train dataset:   4%|▎         | 65378/1807941 [00:54<24:18, 1194.78 examples/s]
Tokenizing train dataset:   4%|▎         | 65505/1807941 [00:54<23:59, 1210.84 examples/s]
Tokenizing train dataset:   4%|▎         | 65631/1807941 [00:54<23:49, 1219.04 examples/s]
Tokenizing train dataset:   4%|▎         | 65758/1807941 [00:54<23:35, 1230.46 examples/s]
Tokenizing train dataset:   4%|▎         | 65893/1807941 [00:54<22:59, 1263.15 examples/s]
Tokenizing train dataset:   4%|▎         | 66058/1807941 [00:54<24:28, 1185.87 examples/s]
Tokenizing train dataset:   4%|▎         | 66191/1807941 [00:54<23:43, 1223.17 examples/s]
Tokenizing train dataset:   4%|▎         | 66325/1807941 [00:54<23:10, 1252.87 examples/s]
Tokenizing train dataset:   4%|▎         | 66521/1807941 [00:55<22:52, 1269.05 examples/s]
Tokenizing train dataset:   4%|▎         | 66655/1807941 [00:55<22:37, 1282.42 examples/s]
Tokenizing train dataset:   4%|▎         | 66848/1807941 [00:55<22:39, 1280.54 examples/s]
Tokenizing train dataset:   4%|▎         | 66977/1807941 [00:55<22:40, 1279.77 examples/s]
Tokenizing train dataset:   4%|▎         | 67147/1807941 [00:55<23:38, 1227.24 examples/s]
Tokenizing train dataset:   4%|▎         | 67286/1807941 [00:55<22:55, 1265.06 examples/s]
Tokenizing train dataset:   4%|▎         | 67416/1807941 [00:55<22:48, 1272.30 examples/s]
Tokenizing train dataset:   4%|▎         | 67551/1807941 [00:55<22:30, 1288.95 examples/s]
Tokenizing train dataset:   4%|▎         | 67746/1807941 [00:56<22:32, 1286.88 examples/s]
Tokenizing train dataset:   4%|▍         | 67880/1807941 [00:56<22:21, 1296.72 examples/s]
Tokenizing train dataset:   4%|▍         | 68060/1807941 [00:56<24:11, 1198.72 examples/s]
Tokenizing train dataset:   4%|▍         | 68184/1807941 [00:56<24:02, 1206.38 examples/s]
Tokenizing train dataset:   4%|▍         | 68321/1807941 [00:56<23:17, 1244.56 examples/s]
Tokenizing train dataset:   4%|▍         | 68514/1807941 [00:56<23:04, 1256.24 examples/s]
Tokenizing train dataset:   4%|▍         | 68646/1807941 [00:56<22:51, 1268.46 examples/s]
Tokenizing train dataset:   4%|▍         | 68782/1807941 [00:56<22:28, 1289.68 examples/s]
Tokenizing train dataset:   4%|▍         | 68964/1807941 [00:57<23:05, 1255.09 examples/s]
Tokenizing train dataset:   4%|▍         | 69128/1807941 [00:57<24:12, 1197.40 examples/s]
Tokenizing train dataset:   4%|▍         | 69258/1807941 [00:57<23:45, 1219.55 examples/s]
Tokenizing train dataset:   4%|▍         | 69386/1807941 [00:57<23:32, 1231.04 examples/s]
Tokenizing train dataset:   4%|▍         | 69522/1807941 [00:57<22:57, 1261.78 examples/s]
Tokenizing train dataset:   4%|▍         | 69655/1807941 [00:57<22:40, 1277.37 examples/s]
Tokenizing train dataset:   4%|▍         | 69854/1807941 [00:57<22:24, 1292.76 examples/s]
Tokenizing train dataset:   4%|▍         | 69996/1807941 [00:57<21:54, 1321.87 examples/s]
Tokenizing train dataset:   4%|▍         | 70173/1807941 [00:58<22:52, 1266.46 examples/s]
Tokenizing train dataset:   4%|▍         | 70340/1807941 [00:58<23:54, 1210.90 examples/s]
Tokenizing train dataset:   4%|▍         | 70477/1807941 [00:58<23:13, 1247.01 examples/s]
Tokenizing train dataset:   4%|▍         | 70611/1807941 [00:58<22:49, 1268.84 examples/s]
Tokenizing train dataset:   4%|▍         | 70747/1807941 [00:58<22:26, 1290.36 examples/s]
Tokenizing train dataset:   4%|▍         | 70881/1807941 [00:58<22:14, 1302.08 examples/s]
Tokenizing train dataset:   4%|▍         | 71051/1807941 [00:58<23:33, 1228.80 examples/s]
Tokenizing train dataset:   4%|▍         | 71213/1807941 [00:58<24:41, 1172.65 examples/s]
Tokenizing train dataset:   4%|▍         | 71342/1807941 [00:58<24:06, 1200.92 examples/s]
Tokenizing train dataset:   4%|▍         | 71470/1807941 [00:59<23:42, 1220.75 examples/s]
Tokenizing train dataset:   4%|▍         | 71599/1807941 [00:59<23:22, 1237.79 examples/s]
Tokenizing train dataset:   4%|▍         | 71727/1807941 [00:59<23:12, 1246.63 examples/s]
Tokenizing train dataset:   4%|▍         | 71865/1807941 [00:59<22:34, 1281.72 examples/s]
Tokenizing train dataset:   4%|▍         | 71997/1807941 [00:59<22:23, 1292.16 examples/s]
Tokenizing train dataset:   4%|▍         | 72164/1807941 [00:59<23:41, 1221.45 examples/s]
Tokenizing train dataset:   4%|▍         | 72291/1807941 [00:59<23:28, 1232.58 examples/s]
Tokenizing train dataset:   4%|▍         | 72478/1807941 [00:59<23:33, 1227.83 examples/s]
Tokenizing train dataset:   4%|▍         | 72613/1807941 [00:59<23:02, 1255.11 examples/s]
Tokenizing train dataset:   4%|▍         | 72741/1807941 [01:00<23:00, 1256.85 examples/s]
Tokenizing train dataset:   4%|▍         | 72926/1807941 [01:00<23:15, 1243.42 examples/s]
Tokenizing train dataset:   4%|▍         | 73070/1807941 [01:00<25:16, 1144.29 examples/s]
Tokenizing train dataset:   4%|▍         | 73195/1807941 [01:00<24:44, 1168.73 examples/s]
Tokenizing train dataset:   4%|▍         | 73315/1807941 [01:00<24:35, 1175.70 examples/s]
Tokenizing train dataset:   4%|▍         | 73448/1807941 [01:00<23:48, 1214.11 examples/s]
Tokenizing train dataset:   4%|▍         | 73576/1807941 [01:00<23:30, 1229.67 examples/s]
Tokenizing train dataset:   4%|▍         | 73760/1807941 [01:00<23:36, 1223.90 examples/s]
Tokenizing train dataset:   4%|▍         | 73949/1807941 [01:01<23:25, 1234.06 examples/s]
Tokenizing train dataset:   4%|▍         | 74074/1807941 [01:01<24:13, 1193.23 examples/s]
Tokenizing train dataset:   4%|▍         | 74249/1807941 [01:01<21:41, 1331.97 examples/s]
Tokenizing train dataset:   4%|▍         | 74439/1807941 [01:01<22:09, 1304.26 examples/s]
Tokenizing train dataset:   4%|▍         | 74638/1807941 [01:01<22:04, 1308.80 examples/s]
Tokenizing train dataset:   4%|▍         | 74826/1807941 [01:01<22:23, 1289.69 examples/s]
Tokenizing train dataset:   4%|▍         | 75000/1807941 [01:01<24:00, 1203.40 examples/s]
Tokenizing train dataset:   4%|▍         | 75126/1807941 [01:02<23:49, 1212.00 examples/s]
Tokenizing train dataset:   4%|▍         | 75257/1807941 [01:02<23:26, 1232.33 examples/s]
Tokenizing train dataset:   4%|▍         | 75447/1807941 [01:02<23:16, 1240.15 examples/s]
Tokenizing train dataset:   4%|▍         | 75621/1807941 [01:02<23:51, 1209.93 examples/s]
Tokenizing train dataset:   4%|▍         | 75750/1807941 [01:02<23:32, 1226.70 examples/s]
Tokenizing train dataset:   4%|▍         | 75929/1807941 [01:02<23:47, 1213.68 examples/s]
Tokenizing train dataset:   4%|▍         | 76081/1807941 [01:02<25:13, 1144.37 examples/s]
Tokenizing train dataset:   4%|▍         | 76205/1807941 [01:02<24:46, 1164.95 examples/s]
Tokenizing train dataset:   4%|▍         | 76327/1807941 [01:03<24:32, 1176.16 examples/s]
Tokenizing train dataset:   4%|▍         | 76450/1807941 [01:03<24:15, 1189.25 examples/s]
Tokenizing train dataset:   4%|▍         | 76601/1807941 [01:03<22:37, 1275.42 examples/s]
Tokenizing train dataset:   4%|▍         | 76762/1807941 [01:03<21:07, 1366.19 examples/s]
Tokenizing train dataset:   4%|▍         | 76960/1807941 [01:03<21:26, 1345.47 examples/s]
Tokenizing train dataset:   4%|▍         | 77122/1807941 [01:03<23:12, 1243.25 examples/s]
Tokenizing train dataset:   4%|▍         | 77301/1807941 [01:03<23:38, 1220.25 examples/s]
Tokenizing train dataset:   4%|▍         | 77430/1807941 [01:03<23:22, 1234.06 examples/s]
Tokenizing train dataset:   4%|▍         | 77565/1807941 [01:04<22:50, 1262.86 examples/s]
Tokenizing train dataset:   4%|▍         | 77755/1807941 [01:04<22:51, 1261.11 examples/s]
Tokenizing train dataset:   4%|▍         | 77918/1807941 [01:04<24:01, 1200.17 examples/s]
Tokenizing train dataset:   4%|▍         | 78078/1807941 [01:04<24:56, 1155.71 examples/s]
Tokenizing train dataset:   4%|▍         | 78249/1807941 [01:04<25:07, 1147.49 examples/s]
Tokenizing train dataset:   4%|▍         | 78368/1807941 [01:04<24:57, 1155.13 examples/s]
Tokenizing train dataset:   4%|▍         | 78487/1807941 [01:04<24:47, 1162.71 examples/s]
Tokenizing train dataset:   4%|▍         | 78615/1807941 [01:04<24:12, 1190.53 examples/s]
Tokenizing train dataset:   4%|▍         | 78799/1807941 [01:05<24:03, 1197.54 examples/s]
Tokenizing train dataset:   4%|▍         | 78922/1807941 [01:05<23:56, 1203.45 examples/s]
Tokenizing train dataset:   4%|▍         | 79087/1807941 [01:05<24:46, 1162.99 examples/s]
Tokenizing train dataset:   4%|▍         | 79205/1807941 [01:05<24:41, 1166.86 examples/s]
Tokenizing train dataset:   4%|▍         | 79327/1807941 [01:05<24:25, 1179.41 examples/s]
Tokenizing train dataset:   4%|▍         | 79446/1807941 [01:05<24:26, 1178.39 examples/s]
Tokenizing train dataset:   4%|▍         | 79575/1807941 [01:05<23:53, 1205.46 examples/s]
Tokenizing train dataset:   4%|▍         | 79701/1807941 [01:05<23:40, 1216.77 examples/s]
Tokenizing train dataset:   4%|▍         | 79874/1807941 [01:05<24:13, 1188.63 examples/s]
Tokenizing train dataset:   4%|▍         | 80000/1807941 [01:06<25:25, 1132.94 examples/s]
Tokenizing train dataset:   4%|▍         | 80121/1807941 [01:06<25:00, 1151.60 examples/s]
Tokenizing train dataset:   4%|▍         | 80239/1807941 [01:06<24:51, 1158.75 examples/s]
Tokenizing train dataset:   4%|▍         | 80359/1807941 [01:06<24:43, 1164.67 examples/s]
Tokenizing train dataset:   4%|▍         | 80489/1807941 [01:06<23:58, 1200.90 examples/s]
Tokenizing train dataset:   4%|▍         | 80620/1807941 [01:06<23:23, 1230.57 examples/s]
Tokenizing train dataset:   4%|▍         | 80802/1807941 [01:06<23:37, 1218.24 examples/s]
Tokenizing train dataset:   4%|▍         | 80983/1807941 [01:06<23:44, 1212.16 examples/s]
Tokenizing train dataset:   4%|▍         | 81166/1807941 [01:07<23:44, 1212.30 examples/s]
Tokenizing train dataset:   4%|▍         | 81343/1807941 [01:07<24:00, 1198.60 examples/s]
Tokenizing train dataset:   5%|▍         | 81500/1807941 [01:07<22:24, 1284.54 examples/s]
Tokenizing train dataset:   5%|▍         | 81695/1807941 [01:07<22:20, 1287.48 examples/s]
Tokenizing train dataset:   5%|▍         | 81832/1807941 [01:07<22:02, 1305.31 examples/s]
Tokenizing train dataset:   5%|▍         | 82000/1807941 [01:07<23:33, 1220.73 examples/s]
Tokenizing train dataset:   5%|▍         | 82174/1807941 [01:07<24:00, 1198.31 examples/s]
Tokenizing train dataset:   5%|▍         | 82297/1807941 [01:07<23:55, 1201.84 examples/s]
Tokenizing train dataset:   5%|▍         | 82421/1807941 [01:08<23:46, 1210.03 examples/s]
Tokenizing train dataset:   5%|▍         | 82546/1807941 [01:08<23:34, 1220.05 examples/s]
Tokenizing train dataset:   5%|▍         | 82721/1807941 [01:08<24:02, 1196.06 examples/s]
Tokenizing train dataset:   5%|▍         | 82855/1807941 [01:08<23:23, 1228.73 examples/s]
Tokenizing train dataset:   5%|▍         | 83009/1807941 [01:08<24:56, 1152.67 examples/s]
Tokenizing train dataset:   5%|▍         | 83181/1807941 [01:08<25:04, 1146.43 examples/s]
Tokenizing train dataset:   5%|▍         | 83314/1807941 [01:08<24:11, 1188.50 examples/s]
Tokenizing train dataset:   5%|▍         | 83445/1807941 [01:08<23:36, 1217.52 examples/s]
Tokenizing train dataset:   5%|▍         | 83631/1807941 [01:09<23:27, 1225.01 examples/s]
Tokenizing train dataset:   5%|▍         | 83817/1807941 [01:09<23:28, 1224.10 examples/s]
Tokenizing train dataset:   5%|▍         | 83953/1807941 [01:09<22:54, 1254.29 examples/s]
Tokenizing train dataset:   5%|▍         | 84114/1807941 [01:09<27:59, 1026.69 examples/s]
Tokenizing train dataset:   5%|▍         | 84231/1807941 [01:09<27:11, 1056.68 examples/s]
Tokenizing train dataset:   5%|▍         | 84398/1807941 [01:09<26:49, 1070.85 examples/s]
Tokenizing train dataset:   5%|▍         | 84519/1807941 [01:09<26:06, 1100.23 examples/s]
Tokenizing train dataset:   5%|▍         | 84649/1807941 [01:09<25:01, 1147.98 examples/s]
Tokenizing train dataset:   5%|▍         | 84770/1807941 [01:10<24:43, 1161.24 examples/s]
Tokenizing train dataset:   5%|▍         | 84946/1807941 [01:10<24:40, 1163.78 examples/s]
Tokenizing train dataset:   5%|▍         | 85067/1807941 [01:10<25:50, 1111.24 examples/s]
Tokenizing train dataset:   5%|▍         | 85190/1807941 [01:10<25:10, 1140.24 examples/s]
Tokenizing train dataset:   5%|▍         | 85359/1807941 [01:10<25:20, 1133.06 examples/s]
Tokenizing train dataset:   5%|▍         | 85486/1807941 [01:10<24:39, 1164.30 examples/s]
Tokenizing train dataset:   5%|▍         | 85615/1807941 [01:10<24:02, 1194.12 examples/s]
Tokenizing train dataset:   5%|▍         | 85793/1807941 [01:10<24:09, 1188.08 examples/s]
Tokenizing train dataset:   5%|▍         | 85966/1807941 [01:11<24:28, 1172.49 examples/s]
Tokenizing train dataset:   5%|▍         | 86095/1807941 [01:11<27:28, 1044.42 examples/s]
Tokenizing train dataset:   5%|▍         | 86211/1807941 [01:11<26:50, 1069.10 examples/s]
Tokenizing train dataset:   5%|▍         | 86334/1807941 [01:11<25:53, 1107.88 examples/s]
Tokenizing train dataset:   5%|▍         | 86456/1807941 [01:11<25:15, 1136.04 examples/s]
Tokenizing train dataset:   5%|▍         | 86581/1807941 [01:11<24:37, 1165.44 examples/s]
Tokenizing train dataset:   5%|▍         | 86754/1807941 [01:11<24:45, 1158.78 examples/s]
Tokenizing train dataset:   5%|▍         | 86881/1807941 [01:11<24:12, 1184.86 examples/s]
Tokenizing train dataset:   5%|▍         | 87061/1807941 [01:12<25:16, 1134.59 examples/s]
Tokenizing train dataset:   5%|▍         | 87194/1807941 [01:12<24:17, 1180.50 examples/s]
Tokenizing train dataset:   5%|▍         | 87322/1807941 [01:12<23:50, 1203.22 examples/s]
Tokenizing train dataset:   5%|▍         | 87450/1807941 [01:12<23:29, 1220.72 examples/s]
Tokenizing train dataset:   5%|▍         | 87576/1807941 [01:12<23:21, 1227.59 examples/s]
Tokenizing train dataset:   5%|▍         | 87701/1807941 [01:12<23:18, 1230.08 examples/s]
Tokenizing train dataset:   5%|▍         | 87831/1807941 [01:12<23:00, 1245.88 examples/s]
Tokenizing train dataset:   5%|▍         | 88000/1807941 [01:12<24:35, 1165.88 examples/s]
Tokenizing train dataset:   5%|▍         | 88161/1807941 [01:12<22:24, 1278.95 examples/s]
Tokenizing train dataset:   5%|▍         | 88345/1807941 [01:13<22:46, 1258.00 examples/s]
Tokenizing train dataset:   5%|▍         | 88490/1807941 [01:13<21:57, 1304.72 examples/s]
Tokenizing train dataset:   5%|▍         | 88670/1807941 [01:13<22:39, 1264.95 examples/s]
Tokenizing train dataset:   5%|▍         | 88861/1807941 [01:13<22:40, 1263.80 examples/s]
Tokenizing train dataset:   5%|▍         | 88990/1807941 [01:13<22:38, 1265.73 examples/s]
Tokenizing train dataset:   5%|▍         | 89153/1807941 [01:13<23:51, 1200.80 examples/s]
Tokenizing train dataset:   5%|▍         | 89306/1807941 [01:13<22:26, 1276.08 examples/s]
Tokenizing train dataset:   5%|▍         | 89493/1807941 [01:14<22:42, 1261.49 examples/s]
Tokenizing train dataset:   5%|▍         | 89626/1807941 [01:14<22:26, 1276.18 examples/s]
Tokenizing train dataset:   5%|▍         | 89817/1807941 [01:14<22:33, 1269.72 examples/s]
Tokenizing train dataset:   5%|▍         | 89949/1807941 [01:14<22:20, 1281.37 examples/s]
Tokenizing train dataset:   5%|▍         | 90126/1807941 [01:14<23:29, 1219.06 examples/s]
Tokenizing train dataset:   5%|▍         | 90257/1807941 [01:14<23:07, 1237.67 examples/s]
Tokenizing train dataset:   5%|▌         | 90441/1807941 [01:14<23:17, 1229.30 examples/s]
Tokenizing train dataset:   5%|▌         | 90633/1807941 [01:14<23:02, 1242.15 examples/s]
Tokenizing train dataset:   5%|▌         | 90775/1807941 [01:15<22:19, 1281.54 examples/s]
Tokenizing train dataset:   5%|▌         | 90917/1807941 [01:15<21:49, 1311.67 examples/s]
Tokenizing train dataset:   5%|▌         | 91094/1807941 [01:15<22:38, 1263.81 examples/s]
Tokenizing train dataset:   5%|▌         | 91232/1807941 [01:15<22:10, 1290.47 examples/s]
Tokenizing train dataset:   5%|▌         | 91366/1807941 [01:15<22:00, 1300.04 examples/s]
Tokenizing train dataset:   5%|▌         | 91502/1807941 [01:15<21:46, 1314.13 examples/s]
Tokenizing train dataset:   5%|▌         | 91641/1807941 [01:15<21:29, 1331.40 examples/s]
Tokenizing train dataset:   5%|▌         | 91850/1807941 [01:15<21:11, 1350.16 examples/s]
Tokenizing train dataset:   5%|▌         | 92012/1807941 [01:16<22:49, 1253.10 examples/s]
Tokenizing train dataset:   5%|▌         | 92149/1807941 [01:16<22:20, 1279.70 examples/s]
Tokenizing train dataset:   5%|▌         | 92283/1807941 [01:16<22:08, 1291.16 examples/s]
Tokenizing train dataset:   5%|▌         | 92423/1807941 [01:16<21:41, 1318.40 examples/s]
Tokenizing train dataset:   5%|▌         | 92567/1807941 [01:16<21:11, 1349.27 examples/s]
Tokenizing train dataset:   5%|▌         | 92775/1807941 [01:16<20:59, 1361.95 examples/s]
Tokenizing train dataset:   5%|▌         | 92923/1807941 [01:16<20:33, 1390.35 examples/s]
Tokenizing train dataset:   5%|▌         | 93102/1807941 [01:16<21:43, 1315.90 examples/s]
Tokenizing train dataset:   5%|▌         | 93250/1807941 [01:16<21:05, 1354.93 examples/s]
Tokenizing train dataset:   5%|▌         | 93450/1807941 [01:17<21:16, 1343.39 examples/s]
Tokenizing train dataset:   5%|▌         | 93586/1807941 [01:17<21:16, 1342.85 examples/s]
Tokenizing train dataset:   5%|▌         | 93773/1807941 [01:17<21:53, 1305.40 examples/s]
Tokenizing train dataset:   5%|▌         | 93962/1807941 [01:17<22:10, 1288.27 examples/s]
Tokenizing train dataset:   5%|▌         | 94127/1807941 [01:17<23:38, 1207.94 examples/s]
Tokenizing train dataset:   5%|▌         | 94311/1807941 [01:17<23:36, 1209.90 examples/s]
Tokenizing train dataset:   5%|▌         | 94496/1807941 [01:17<23:31, 1214.00 examples/s]
Tokenizing train dataset:   5%|▌         | 94623/1807941 [01:18<23:20, 1223.12 examples/s]
Tokenizing train dataset:   5%|▌         | 94801/1807941 [01:18<23:38, 1208.06 examples/s]
Tokenizing train dataset:   5%|▌         | 94987/1807941 [01:18<23:30, 1214.24 examples/s]
Tokenizing train dataset:   5%|▌         | 95151/1807941 [01:18<24:20, 1172.89 examples/s]
Tokenizing train dataset:   5%|▌         | 95277/1807941 [01:18<23:58, 1190.87 examples/s]
Tokenizing train dataset:   5%|▌         | 95403/1807941 [01:18<23:41, 1204.58 examples/s]
Tokenizing train dataset:   5%|▌         | 95536/1807941 [01:18<23:06, 1234.94 examples/s]
Tokenizing train dataset:   5%|▌         | 95667/1807941 [01:18<22:45, 1254.07 examples/s]
Tokenizing train dataset:   5%|▌         | 95807/1807941 [01:19<22:06, 1290.93 examples/s]
Tokenizing train dataset:   5%|▌         | 95941/1807941 [01:19<21:56, 1300.32 examples/s]
Tokenizing train dataset:   5%|▌         | 96136/1807941 [01:19<22:44, 1254.12 examples/s]
Tokenizing train dataset:   5%|▌         | 96272/1807941 [01:19<22:16, 1281.00 examples/s]
Tokenizing train dataset:   5%|▌         | 96458/1807941 [01:19<22:34, 1263.49 examples/s]
Tokenizing train dataset:   5%|▌         | 96650/1807941 [01:19<22:30, 1267.62 examples/s]
Tokenizing train dataset:   5%|▌         | 96827/1807941 [01:19<23:03, 1237.03 examples/s]
Tokenizing train dataset:   5%|▌         | 96957/1807941 [01:19<22:47, 1250.75 examples/s]
Tokenizing train dataset:   5%|▌         | 97127/1807941 [01:20<23:54, 1192.91 examples/s]
Tokenizing train dataset:   5%|▌         | 97252/1807941 [01:20<23:41, 1203.79 examples/s]
Tokenizing train dataset:   5%|▌         | 97435/1807941 [01:20<23:37, 1206.29 examples/s]
Tokenizing train dataset:   5%|▌         | 97557/1807941 [01:20<23:36, 1207.12 examples/s]
Tokenizing train dataset:   5%|▌         | 97679/1807941 [01:20<23:36, 1207.52 examples/s]
Tokenizing train dataset:   5%|▌         | 97808/1807941 [01:20<23:12, 1228.45 examples/s]
Tokenizing train dataset:   5%|▌         | 97967/1807941 [01:20<21:29, 1325.91 examples/s]
Tokenizing train dataset:   5%|▌         | 98106/1807941 [01:20<21:14, 1341.54 examples/s]
Tokenizing train dataset:   5%|▌         | 98302/1807941 [01:20<18:47, 1516.35 examples/s]
Tokenizing train dataset:   5%|▌         | 98497/1807941 [01:21<17:22, 1639.58 examples/s]
Tokenizing train dataset:   5%|▌         | 98701/1807941 [01:21<18:39, 1527.15 examples/s]
Tokenizing train dataset:   5%|▌         | 98894/1807941 [01:21<19:48, 1437.64 examples/s]
Tokenizing train dataset:   5%|▌         | 99067/1807941 [01:21<21:16, 1338.44 examples/s]
Tokenizing train dataset:   5%|▌         | 99211/1807941 [01:21<20:57, 1359.13 examples/s]
Tokenizing train dataset:   5%|▌         | 99396/1807941 [01:21<21:43, 1310.89 examples/s]
Tokenizing train dataset:   6%|▌         | 99590/1807941 [01:21<21:55, 1298.98 examples/s]
Tokenizing train dataset:   6%|▌         | 99724/1807941 [01:21<21:49, 1304.32 examples/s]
Tokenizing train dataset:   6%|▌         | 99913/1807941 [01:22<22:08, 1285.51 examples/s]
Tokenizing train dataset:   6%|▌         | 100064/1807941 [01:22<23:53, 1191.48 examples/s]
Tokenizing train dataset:   6%|▌         | 100243/1807941 [01:22<23:54, 1190.31 examples/s]
Tokenizing train dataset:   6%|▌         | 100368/1807941 [01:22<23:41, 1201.54 examples/s]
Tokenizing train dataset:   6%|▌         | 100493/1807941 [01:22<23:30, 1210.62 examples/s]
Tokenizing train dataset:   6%|▌         | 100657/1807941 [01:22<24:23, 1166.38 examples/s]
Tokenizing train dataset:   6%|▌         | 100781/1807941 [01:22<24:06, 1180.14 examples/s]
Tokenizing train dataset:   6%|▌         | 100905/1807941 [01:23<23:49, 1194.03 examples/s]
Tokenizing train dataset:   6%|▌         | 101048/1807941 [01:23<26:07, 1088.87 examples/s]
Tokenizing train dataset:   6%|▌         | 101160/1807941 [01:23<25:59, 1094.27 examples/s]
Tokenizing train dataset:   6%|▌         | 101277/1807941 [01:23<25:34, 1112.41 examples/s]
Tokenizing train dataset:   6%|▌         | 101446/1807941 [01:23<25:28, 1116.37 examples/s]
Tokenizing train dataset:   6%|▌         | 101572/1807941 [01:23<24:41, 1151.93 examples/s]
Tokenizing train dataset:   6%|▌         | 101696/1807941 [01:23<24:13, 1173.62 examples/s]
Tokenizing train dataset:   6%|▌         | 101874/1807941 [01:23<24:11, 1175.49 examples/s]
Tokenizing train dataset:   6%|▌         | 101999/1807941 [01:23<23:53, 1190.30 examples/s]
Tokenizing train dataset:   6%|▌         | 102168/1807941 [01:24<24:25, 1164.07 examples/s]
Tokenizing train dataset:   6%|▌         | 102294/1807941 [01:24<23:59, 1185.03 examples/s]
Tokenizing train dataset:   6%|▌         | 102466/1807941 [01:24<24:19, 1168.92 examples/s]
Tokenizing train dataset:   6%|▌         | 102602/1807941 [01:24<23:25, 1213.55 examples/s]
Tokenizing train dataset:   6%|▌         | 102729/1807941 [01:24<23:11, 1225.46 examples/s]
Tokenizing train dataset:   6%|▌         | 102918/1807941 [01:24<23:04, 1231.29 examples/s]
Tokenizing train dataset:   6%|▌         | 103075/1807941 [01:24<24:24, 1164.04 examples/s]
Tokenizing train dataset:   6%|▌         | 103200/1807941 [01:24<24:03, 1180.68 examples/s]
Tokenizing train dataset:   6%|▌         | 103327/1807941 [01:25<23:37, 1202.34 examples/s]
Tokenizing train dataset:   6%|▌         | 103503/1807941 [01:25<23:52, 1189.66 examples/s]
Tokenizing train dataset:   6%|▌         | 103631/1807941 [01:25<23:29, 1209.24 examples/s]
Tokenizing train dataset:   6%|▌         | 103754/1807941 [01:25<23:27, 1211.07 examples/s]
Tokenizing train dataset:   6%|▌         | 103878/1807941 [01:25<23:21, 1215.99 examples/s]
Tokenizing train dataset:   6%|▌         | 104058/1807941 [01:25<24:42, 1149.06 examples/s]
Tokenizing train dataset:   6%|▌         | 104234/1807941 [01:25<24:35, 1154.58 examples/s]
Tokenizing train dataset:   6%|▌         | 104357/1807941 [01:25<24:15, 1170.83 examples/s]
Tokenizing train dataset:   6%|▌         | 104481/1807941 [01:26<23:57, 1185.32 examples/s]
Tokenizing train dataset:   6%|▌         | 104602/1807941 [01:26<23:50, 1190.64 examples/s]
Tokenizing train dataset:   6%|▌         | 104783/1807941 [01:26<23:49, 1191.40 examples/s]
Tokenizing train dataset:   6%|▌         | 104908/1807941 [01:26<23:36, 1201.95 examples/s]
Tokenizing train dataset:   6%|▌         | 105060/1807941 [01:26<25:05, 1130.76 examples/s]
Tokenizing train dataset:   6%|▌         | 105241/1807941 [01:26<24:34, 1154.48 examples/s]
Tokenizing train dataset:   6%|▌         | 105422/1807941 [01:26<24:16, 1168.59 examples/s]
Tokenizing train dataset:   6%|▌         | 105541/1807941 [01:26<24:13, 1170.85 examples/s]
Tokenizing train dataset:   6%|▌         | 105666/1807941 [01:27<23:53, 1187.24 examples/s]
Tokenizing train dataset:   6%|▌         | 105786/1807941 [01:27<23:52, 1188.14 examples/s]
Tokenizing train dataset:   6%|▌         | 105906/1807941 [01:27<23:54, 1186.53 examples/s]
Tokenizing train dataset:   6%|▌         | 106067/1807941 [01:27<24:52, 1140.60 examples/s]
Tokenizing train dataset:   6%|▌         | 106184/1807941 [01:27<24:44, 1146.09 examples/s]
Tokenizing train dataset:   6%|▌         | 106314/1807941 [01:27<23:56, 1184.78 examples/s]
Tokenizing train dataset:   6%|▌         | 106435/1807941 [01:27<23:50, 1189.09 examples/s]
Tokenizing train dataset:   6%|▌         | 106610/1807941 [01:27<24:07, 1175.46 examples/s]
Tokenizing train dataset:   6%|▌         | 106733/1807941 [01:27<23:53, 1187.07 examples/s]
Tokenizing train dataset:   6%|▌         | 106858/1807941 [01:28<23:35, 1201.46 examples/s]
Tokenizing train dataset:   6%|▌         | 106987/1807941 [01:28<23:11, 1222.36 examples/s]
Tokenizing train dataset:   6%|▌         | 107135/1807941 [01:28<25:00, 1133.39 examples/s]
Tokenizing train dataset:   6%|▌         | 107258/1807941 [01:28<24:29, 1157.28 examples/s]
Tokenizing train dataset:   6%|▌         | 107379/1807941 [01:28<24:13, 1169.66 examples/s]
Tokenizing train dataset:   6%|▌         | 107501/1807941 [01:28<24:01, 1179.27 examples/s]
Tokenizing train dataset:   6%|▌         | 107623/1807941 [01:28<23:49, 1189.06 examples/s]
Tokenizing train dataset:   6%|▌         | 107798/1807941 [01:28<24:03, 1177.77 examples/s]
Tokenizing train dataset:   6%|▌         | 107922/1807941 [01:28<23:45, 1192.97 examples/s]
Tokenizing train dataset:   6%|▌         | 108083/1807941 [01:29<24:45, 1144.65 examples/s]
Tokenizing train dataset:   6%|▌         | 108200/1807941 [01:29<24:37, 1150.22 examples/s]
Tokenizing train dataset:   6%|▌         | 108329/1807941 [01:29<23:54, 1184.73 examples/s]
Tokenizing train dataset:   6%|▌         | 108488/1807941 [01:29<25:00, 1132.88 examples/s]
Tokenizing train dataset:   6%|▌         | 108606/1807941 [01:29<24:47, 1142.65 examples/s]
Tokenizing train dataset:   6%|▌         | 108726/1807941 [01:29<24:31, 1155.02 examples/s]
Tokenizing train dataset:   6%|▌         | 108895/1807941 [01:29<24:49, 1140.51 examples/s]
Tokenizing train dataset:   6%|▌         | 109053/1807941 [01:30<26:15, 1078.55 examples/s]
Tokenizing train dataset:   6%|▌         | 109182/1807941 [01:30<25:11, 1123.71 examples/s]
Tokenizing train dataset:   6%|▌         | 109303/1807941 [01:30<24:45, 1143.74 examples/s]
Tokenizing train dataset:   6%|▌         | 109423/1807941 [01:30<24:28, 1156.96 examples/s]
Tokenizing train dataset:   6%|▌         | 109547/1807941 [01:30<24:01, 1178.08 examples/s]
Tokenizing train dataset:   6%|▌         | 109676/1807941 [01:30<23:27, 1206.52 examples/s]
Tokenizing train dataset:   6%|▌         | 109802/1807941 [01:30<23:15, 1216.73 examples/s]
Tokenizing train dataset:   6%|▌         | 109936/1807941 [01:30<22:40, 1248.49 examples/s]
Tokenizing train dataset:   6%|▌         | 110086/1807941 [01:30<24:34, 1151.76 examples/s]
Tokenizing train dataset:   6%|▌         | 110262/1807941 [01:31<24:28, 1156.20 examples/s]
Tokenizing train dataset:   6%|▌         | 110389/1807941 [01:31<23:56, 1181.60 examples/s]
Tokenizing train dataset:   6%|▌         | 110517/1807941 [01:31<23:28, 1204.85 examples/s]
Tokenizing train dataset:   6%|▌         | 110696/1807941 [01:31<23:36, 1197.82 examples/s]
Tokenizing train dataset:   6%|▌         | 110833/1807941 [01:31<22:52, 1236.20 examples/s]
Tokenizing train dataset:   6%|▌         | 110964/1807941 [01:31<22:35, 1252.35 examples/s]
Tokenizing train dataset:   6%|▌         | 111121/1807941 [01:31<24:07, 1172.47 examples/s]
Tokenizing train dataset:   6%|▌         | 111242/1807941 [01:31<23:56, 1180.97 examples/s]
Tokenizing train dataset:   6%|▌         | 111362/1807941 [01:31<23:55, 1181.78 examples/s]
Tokenizing train dataset:   6%|▌         | 111491/1807941 [01:32<23:31, 1202.17 examples/s]
Tokenizing train dataset:   6%|▌         | 111652/1807941 [01:32<24:33, 1150.82 examples/s]
Tokenizing train dataset:   6%|▌         | 111818/1807941 [01:32<25:12, 1121.37 examples/s]
Tokenizing train dataset:   6%|▌         | 111979/1807941 [01:32<28:29, 991.87 examples/s] 
Tokenizing train dataset:   6%|▌         | 112135/1807941 [01:32<28:07, 1005.07 examples/s]
Tokenizing train dataset:   6%|▌         | 112248/1807941 [01:32<27:24, 1031.16 examples/s]
Tokenizing train dataset:   6%|▌         | 112373/1807941 [01:32<26:07, 1081.79 examples/s]
Tokenizing train dataset:   6%|▌         | 112499/1807941 [01:33<28:26, 993.79 examples/s] 
Tokenizing train dataset:   6%|▌         | 112603/1807941 [01:33<28:19, 997.79 examples/s]
Tokenizing train dataset:   6%|▌         | 112731/1807941 [01:33<26:27, 1067.80 examples/s]
Tokenizing train dataset:   6%|▌         | 112897/1807941 [01:33<26:09, 1079.85 examples/s]
Tokenizing train dataset:   6%|▋         | 113057/1807941 [01:33<26:28, 1066.90 examples/s]
Tokenizing train dataset:   6%|▋         | 113167/1807941 [01:33<26:19, 1073.25 examples/s]
Tokenizing train dataset:   6%|▋         | 113288/1807941 [01:33<25:30, 1107.23 examples/s]
Tokenizing train dataset:   6%|▋         | 113454/1807941 [01:33<25:33, 1105.10 examples/s]
Tokenizing train dataset:   6%|▋         | 113582/1807941 [01:33<24:38, 1145.73 examples/s]
Tokenizing train dataset:   6%|▋         | 113746/1807941 [01:34<25:06, 1124.69 examples/s]
Tokenizing train dataset:   6%|▋         | 113919/1807941 [01:34<25:07, 1124.06 examples/s]
Tokenizing train dataset:   6%|▋         | 114081/1807941 [01:34<25:28, 1108.16 examples/s]
Tokenizing train dataset:   6%|▋         | 114210/1807941 [01:34<24:35, 1147.85 examples/s]
Tokenizing train dataset:   6%|▋         | 114339/1807941 [01:34<23:55, 1179.41 examples/s]
Tokenizing train dataset:   6%|▋         | 114469/1807941 [01:34<23:19, 1210.03 examples/s]
Tokenizing train dataset:   6%|▋         | 114617/1807941 [01:34<25:02, 1126.95 examples/s]
Tokenizing train dataset:   6%|▋         | 114789/1807941 [01:35<24:56, 1131.26 examples/s]
Tokenizing train dataset:   6%|▋         | 114917/1807941 [01:35<24:15, 1163.40 examples/s]
Tokenizing train dataset:   6%|▋         | 115065/1807941 [01:35<25:41, 1098.18 examples/s]
Tokenizing train dataset:   6%|▋         | 115189/1807941 [01:35<24:55, 1132.17 examples/s]
Tokenizing train dataset:   6%|▋         | 115313/1807941 [01:35<24:22, 1157.22 examples/s]
Tokenizing train dataset:   6%|▋         | 115443/1807941 [01:35<23:40, 1191.78 examples/s]
Tokenizing train dataset:   6%|▋         | 115587/1807941 [01:35<25:50, 1091.48 examples/s]
Tokenizing train dataset:   6%|▋         | 115721/1807941 [01:35<27:46, 1015.45 examples/s]
Tokenizing train dataset:   6%|▋         | 115832/1807941 [01:36<30:34, 922.55 examples/s] 
Tokenizing train dataset:   6%|▋         | 115961/1807941 [01:36<31:26, 896.92 examples/s]
Tokenizing train dataset:   6%|▋         | 116066/1807941 [01:36<31:33, 893.65 examples/s]
Tokenizing train dataset:   6%|▋         | 116200/1807941 [01:36<28:11, 999.90 examples/s]
Tokenizing train dataset:   6%|▋         | 116332/1807941 [01:36<26:05, 1080.47 examples/s]
Tokenizing train dataset:   6%|▋         | 116471/1807941 [01:36<24:16, 1161.45 examples/s]
Tokenizing train dataset:   6%|▋         | 116647/1807941 [01:36<24:18, 1159.94 examples/s]
Tokenizing train dataset:   6%|▋         | 116779/1807941 [01:36<23:32, 1197.09 examples/s]
Tokenizing train dataset:   6%|▋         | 116941/1807941 [01:37<24:27, 1152.39 examples/s]
Tokenizing train dataset:   6%|▋         | 117104/1807941 [01:37<26:05, 1080.30 examples/s]
Tokenizing train dataset:   6%|▋         | 117232/1807941 [01:37<25:01, 1125.90 examples/s]
Tokenizing train dataset:   6%|▋         | 117355/1807941 [01:37<24:32, 1148.31 examples/s]
Tokenizing train dataset:   6%|▋         | 117511/1807941 [01:37<25:28, 1105.91 examples/s]
Tokenizing train dataset:   7%|▋         | 117626/1807941 [01:37<25:16, 1114.53 examples/s]
Tokenizing train dataset:   7%|▋         | 117757/1807941 [01:37<30:57, 910.10 examples/s] 
Tokenizing train dataset:   7%|▋         | 117881/1807941 [01:37<28:39, 983.01 examples/s]
Tokenizing train dataset:   7%|▋         | 118000/1807941 [01:38<28:39, 982.87 examples/s]
Tokenizing train dataset:   7%|▋         | 118107/1807941 [01:38<28:07, 1001.40 examples/s]
Tokenizing train dataset:   7%|▋         | 118215/1807941 [01:38<27:37, 1019.36 examples/s]
Tokenizing train dataset:   7%|▋         | 118336/1807941 [01:38<26:19, 1069.55 examples/s]
Tokenizing train dataset:   7%|▋         | 118468/1807941 [01:38<24:46, 1136.86 examples/s]
Tokenizing train dataset:   7%|▋         | 118621/1807941 [01:38<25:58, 1083.82 examples/s]
Tokenizing train dataset:   7%|▋         | 118753/1807941 [01:38<24:38, 1142.76 examples/s]
Tokenizing train dataset:   7%|▋         | 118912/1807941 [01:38<28:36, 984.28 examples/s] 
Tokenizing train dataset:   7%|▋         | 119064/1807941 [01:39<29:30, 953.67 examples/s]
Tokenizing train dataset:   7%|▋         | 119176/1807941 [01:39<28:27, 988.87 examples/s]
Tokenizing train dataset:   7%|▋         | 119283/1807941 [01:39<28:00, 1004.87 examples/s]
Tokenizing train dataset:   7%|▋         | 119390/1807941 [01:39<27:36, 1019.09 examples/s]
Tokenizing train dataset:   7%|▋         | 119503/1807941 [01:39<26:51, 1048.02 examples/s]
Tokenizing train dataset:   7%|▋         | 119662/1807941 [01:39<26:49, 1048.71 examples/s]
Tokenizing train dataset:   7%|▋         | 119817/1807941 [01:39<27:00, 1041.65 examples/s]
Tokenizing train dataset:   7%|▋         | 119927/1807941 [01:39<26:42, 1053.09 examples/s]
Tokenizing train dataset:   7%|▋         | 120053/1807941 [01:40<28:57, 971.54 examples/s] 
Tokenizing train dataset:   7%|▋         | 120173/1807941 [01:40<27:24, 1026.32 examples/s]
Tokenizing train dataset:   7%|▋         | 120304/1807941 [01:40<25:40, 1095.29 examples/s]
Tokenizing train dataset:   7%|▋         | 120470/1807941 [01:40<25:37, 1097.68 examples/s]
Tokenizing train dataset:   7%|▋         | 120642/1807941 [01:40<25:19, 1110.70 examples/s]
Tokenizing train dataset:   7%|▋         | 120766/1807941 [01:40<24:39, 1140.55 examples/s]
Tokenizing train dataset:   7%|▋         | 120883/1807941 [01:40<24:33, 1144.83 examples/s]
Tokenizing train dataset:   7%|▋         | 121027/1807941 [01:40<26:11, 1073.29 examples/s]
Tokenizing train dataset:   7%|▋         | 121146/1807941 [01:41<25:35, 1098.30 examples/s]
Tokenizing train dataset:   7%|▋         | 121274/1807941 [01:41<24:35, 1142.91 examples/s]
Tokenizing train dataset:   7%|▋         | 121398/1807941 [01:41<24:04, 1167.38 examples/s]
Tokenizing train dataset:   7%|▋         | 121584/1807941 [01:41<23:37, 1189.48 examples/s]
Tokenizing train dataset:   7%|▋         | 121711/1807941 [01:41<23:17, 1206.68 examples/s]
Tokenizing train dataset:   7%|▋         | 121852/1807941 [01:41<25:25, 1105.27 examples/s]
Tokenizing train dataset:   7%|▋         | 121973/1807941 [01:41<28:13, 995.46 examples/s] 
Tokenizing train dataset:   7%|▋         | 122078/1807941 [01:42<33:08, 847.82 examples/s]
Tokenizing train dataset:   7%|▋         | 122201/1807941 [01:42<33:51, 829.99 examples/s]
Tokenizing train dataset:   7%|▋         | 122297/1807941 [01:42<36:23, 771.96 examples/s]
Tokenizing train dataset:   7%|▋         | 122393/1807941 [01:42<38:41, 726.05 examples/s]
Tokenizing train dataset:   7%|▋         | 122487/1807941 [01:42<40:47, 688.51 examples/s]
Tokenizing train dataset:   7%|▋         | 122569/1807941 [01:42<39:17, 714.89 examples/s]
Tokenizing train dataset:   7%|▋         | 122686/1807941 [01:42<38:17, 733.58 examples/s]
Tokenizing train dataset:   7%|▋         | 122803/1807941 [01:42<33:39, 834.41 examples/s]
Tokenizing train dataset:   7%|▋         | 122906/1807941 [01:43<31:58, 878.09 examples/s]
Tokenizing train dataset:   7%|▋         | 123000/1807941 [01:43<36:09, 776.75 examples/s]
Tokenizing train dataset:   7%|▋         | 123117/1807941 [01:43<32:13, 871.24 examples/s]
Tokenizing train dataset:   7%|▋         | 123243/1807941 [01:43<28:55, 970.63 examples/s]
Tokenizing train dataset:   7%|▋         | 123364/1807941 [01:43<30:59, 906.08 examples/s]
Tokenizing train dataset:   7%|▋         | 123517/1807941 [01:43<29:48, 941.90 examples/s]
Tokenizing train dataset:   7%|▋         | 123633/1807941 [01:43<28:14, 993.98 examples/s]
Tokenizing train dataset:   7%|▋         | 123762/1807941 [01:43<26:15, 1069.04 examples/s]
Tokenizing train dataset:   7%|▋         | 123881/1807941 [01:44<25:35, 1096.66 examples/s]
Tokenizing train dataset:   7%|▋         | 124010/1807941 [01:44<27:50, 1008.26 examples/s]
Tokenizing train dataset:   7%|▋         | 124127/1807941 [01:44<26:48, 1046.63 examples/s]
Tokenizing train dataset:   7%|▋         | 124248/1807941 [01:44<25:45, 1089.18 examples/s]
Tokenizing train dataset:   7%|▋         | 124372/1807941 [01:44<24:49, 1130.01 examples/s]
Tokenizing train dataset:   7%|▋         | 124498/1807941 [01:44<24:08, 1162.57 examples/s]
Tokenizing train dataset:   7%|▋         | 124625/1807941 [01:44<23:35, 1188.92 examples/s]
Tokenizing train dataset:   7%|▋         | 124748/1807941 [01:44<23:25, 1197.39 examples/s]
Tokenizing train dataset:   7%|▋         | 124881/1807941 [01:44<22:44, 1233.89 examples/s]
Tokenizing train dataset:   7%|▋         | 125064/1807941 [01:45<23:42, 1182.91 examples/s]
Tokenizing train dataset:   7%|▋         | 125190/1807941 [01:45<23:21, 1200.58 examples/s]
Tokenizing train dataset:   7%|▋         | 125315/1807941 [01:45<23:08, 1211.94 examples/s]
Tokenizing train dataset:   7%|▋         | 125455/1807941 [01:45<25:23, 1104.61 examples/s]
Tokenizing train dataset:   7%|▋         | 125606/1807941 [01:45<29:15, 958.13 examples/s] 
Tokenizing train dataset:   7%|▋         | 125740/1807941 [01:45<30:01, 933.64 examples/s]
Tokenizing train dataset:   7%|▋         | 125891/1807941 [01:45<29:25, 952.74 examples/s]
Tokenizing train dataset:   7%|▋         | 126000/1807941 [01:46<30:25, 921.41 examples/s]
Tokenizing train dataset:   7%|▋         | 126155/1807941 [01:46<29:23, 953.47 examples/s]
Tokenizing train dataset:   7%|▋         | 126280/1807941 [01:46<27:33, 1017.17 examples/s]
Tokenizing train dataset:   7%|▋         | 126405/1807941 [01:46<26:09, 1071.27 examples/s]
Tokenizing train dataset:   7%|▋         | 126531/1807941 [01:46<25:06, 1116.20 examples/s]
Tokenizing train dataset:   7%|▋         | 126653/1807941 [01:46<24:33, 1140.74 examples/s]
Tokenizing train dataset:   7%|▋         | 126771/1807941 [01:46<24:23, 1148.59 examples/s]
Tokenizing train dataset:   7%|▋         | 126892/1807941 [01:46<24:02, 1165.05 examples/s]
Tokenizing train dataset:   7%|▋         | 127068/1807941 [01:46<24:55, 1123.80 examples/s]
Tokenizing train dataset:   7%|▋         | 127190/1807941 [01:47<24:24, 1147.96 examples/s]
Tokenizing train dataset:   7%|▋         | 127313/1807941 [01:47<24:03, 1164.23 examples/s]
Tokenizing train dataset:   7%|▋         | 127447/1807941 [01:47<26:24, 1060.39 examples/s]
Tokenizing train dataset:   7%|▋         | 127569/1807941 [01:47<32:14, 868.73 examples/s] 
Tokenizing train dataset:   7%|▋         | 127695/1807941 [01:47<29:20, 954.40 examples/s]
Tokenizing train dataset:   7%|▋         | 127813/1807941 [01:47<31:13, 896.90 examples/s]
Tokenizing train dataset:   7%|▋         | 127934/1807941 [01:47<32:17, 866.97 examples/s]
Tokenizing train dataset:   7%|▋         | 128061/1807941 [01:48<32:35, 859.13 examples/s]
Tokenizing train dataset:   7%|▋         | 128174/1807941 [01:48<30:27, 919.37 examples/s]
Tokenizing train dataset:   7%|▋         | 128291/1807941 [01:48<28:35, 979.04 examples/s]
Tokenizing train dataset:   7%|▋         | 128419/1807941 [01:48<26:34, 1053.53 examples/s]
Tokenizing train dataset:   7%|▋         | 128539/1807941 [01:48<25:38, 1091.62 examples/s]
Tokenizing train dataset:   7%|▋         | 128664/1807941 [01:48<24:42, 1132.79 examples/s]
Tokenizing train dataset:   7%|▋         | 128796/1807941 [01:48<27:06, 1032.58 examples/s]
Tokenizing train dataset:   7%|▋         | 128971/1807941 [01:48<25:57, 1077.97 examples/s]
Tokenizing train dataset:   7%|▋         | 129108/1807941 [01:49<28:00, 999.25 examples/s] 
Tokenizing train dataset:   7%|▋         | 129238/1807941 [01:49<26:11, 1067.89 examples/s]
Tokenizing train dataset:   7%|▋         | 129374/1807941 [01:49<24:36, 1136.90 examples/s]
Tokenizing train dataset:   7%|▋         | 129506/1807941 [01:49<23:39, 1182.70 examples/s]
Tokenizing train dataset:   7%|▋         | 129645/1807941 [01:49<22:37, 1236.68 examples/s]
Tokenizing train dataset:   7%|▋         | 129825/1807941 [01:49<22:56, 1218.87 examples/s]
Tokenizing train dataset:   7%|▋         | 129958/1807941 [01:49<22:26, 1245.93 examples/s]
Tokenizing train dataset:   7%|▋         | 130133/1807941 [01:49<23:18, 1199.84 examples/s]
Tokenizing train dataset:   7%|▋         | 130279/1807941 [01:49<22:10, 1261.37 examples/s]
Tokenizing train dataset:   7%|▋         | 130420/1807941 [01:50<21:33, 1296.59 examples/s]
Tokenizing train dataset:   7%|▋         | 130563/1807941 [01:50<21:01, 1329.96 examples/s]
Tokenizing train dataset:   7%|▋         | 130703/1807941 [01:50<23:45, 1177.01 examples/s]
Tokenizing train dataset:   7%|▋         | 130888/1807941 [01:50<23:26, 1192.17 examples/s]
Tokenizing train dataset:   7%|▋         | 131039/1807941 [01:50<24:46, 1127.93 examples/s]
Tokenizing train dataset:   7%|▋         | 131173/1807941 [01:50<26:48, 1042.22 examples/s]
Tokenizing train dataset:   7%|▋         | 131331/1807941 [01:50<26:49, 1041.76 examples/s]
Tokenizing train dataset:   7%|▋         | 131437/1807941 [01:51<26:48, 1042.52 examples/s]
Tokenizing train dataset:   7%|▋         | 131566/1807941 [01:51<25:23, 1100.62 examples/s]
Tokenizing train dataset:   7%|▋         | 131692/1807941 [01:51<24:29, 1140.44 examples/s]
Tokenizing train dataset:   7%|▋         | 131829/1807941 [01:51<23:16, 1200.26 examples/s]
Tokenizing train dataset:   7%|▋         | 131962/1807941 [01:51<22:40, 1232.23 examples/s]
Tokenizing train dataset:   7%|▋         | 132107/1807941 [01:51<24:45, 1128.03 examples/s]
Tokenizing train dataset:   7%|▋         | 132271/1807941 [01:51<25:05, 1112.96 examples/s]
Tokenizing train dataset:   7%|▋         | 132408/1807941 [01:51<23:44, 1175.86 examples/s]
Tokenizing train dataset:   7%|▋         | 132540/1807941 [01:51<23:02, 1212.12 examples/s]
Tokenizing train dataset:   7%|▋         | 132664/1807941 [01:52<22:55, 1217.53 examples/s]
Tokenizing train dataset:   7%|▋         | 132807/1807941 [01:52<21:57, 1271.49 examples/s]
Tokenizing train dataset:   7%|▋         | 132986/1807941 [01:52<22:32, 1238.50 examples/s]
Tokenizing train dataset:   7%|▋         | 133164/1807941 [01:52<22:53, 1218.93 examples/s]
Tokenizing train dataset:   7%|▋         | 133298/1807941 [01:52<25:14, 1105.83 examples/s]
Tokenizing train dataset:   7%|▋         | 133454/1807941 [01:52<25:48, 1081.36 examples/s]
Tokenizing train dataset:   7%|▋         | 133574/1807941 [01:52<25:10, 1108.34 examples/s]
Tokenizing train dataset:   7%|▋         | 133692/1807941 [01:52<24:47, 1125.31 examples/s]
Tokenizing train dataset:   7%|▋         | 133817/1807941 [01:53<24:09, 1154.58 examples/s]
Tokenizing train dataset:   7%|▋         | 133938/1807941 [01:53<23:58, 1163.59 examples/s]
Tokenizing train dataset:   7%|▋         | 134075/1807941 [01:53<26:10, 1065.54 examples/s]
Tokenizing train dataset:   7%|▋         | 134199/1807941 [01:53<25:11, 1107.70 examples/s]
Tokenizing train dataset:   7%|▋         | 134313/1807941 [01:53<25:00, 1115.01 examples/s]
Tokenizing train dataset:   7%|▋         | 134440/1807941 [01:53<24:06, 1156.80 examples/s]
Tokenizing train dataset:   7%|▋         | 134575/1807941 [01:53<23:04, 1208.86 examples/s]
Tokenizing train dataset:   7%|▋         | 134760/1807941 [01:53<23:00, 1211.98 examples/s]
Tokenizing train dataset:   7%|▋         | 134900/1807941 [01:53<22:09, 1258.45 examples/s]
Tokenizing train dataset:   7%|▋         | 135050/1807941 [01:54<24:08, 1155.28 examples/s]
Tokenizing train dataset:   7%|▋         | 135216/1807941 [01:54<24:33, 1134.98 examples/s]
Tokenizing train dataset:   7%|▋         | 135338/1807941 [01:54<24:11, 1152.64 examples/s]
Tokenizing train dataset:   7%|▋         | 135503/1807941 [01:54<24:39, 1130.71 examples/s]
Tokenizing train dataset:   8%|▊         | 135625/1807941 [01:54<24:14, 1150.08 examples/s]
Tokenizing train dataset:   8%|▊         | 135749/1807941 [01:54<23:49, 1169.47 examples/s]
Tokenizing train dataset:   8%|▊         | 135878/1807941 [01:54<23:15, 1198.17 examples/s]
Tokenizing train dataset:   8%|▊         | 136006/1807941 [01:54<26:12, 1063.35 examples/s]
Tokenizing train dataset:   8%|▊         | 136142/1807941 [01:55<30:52, 902.29 examples/s] 
Tokenizing train dataset:   8%|▊         | 136243/1807941 [01:55<33:34, 829.83 examples/s]
Tokenizing train dataset:   8%|▊         | 136348/1807941 [01:55<35:27, 785.82 examples/s]
Tokenizing train dataset:   8%|▊         | 136487/1807941 [01:55<33:53, 821.79 examples/s]
Tokenizing train dataset:   8%|▊         | 136590/1807941 [01:55<32:08, 866.75 examples/s]
Tokenizing train dataset:   8%|▊         | 136707/1807941 [01:55<29:47, 934.72 examples/s]
Tokenizing train dataset:   8%|▊         | 136836/1807941 [01:55<27:12, 1023.52 examples/s]
Tokenizing train dataset:   8%|▊         | 136962/1807941 [01:56<25:38, 1086.02 examples/s]
Tokenizing train dataset:   8%|▊         | 137097/1807941 [01:56<28:10, 988.43 examples/s] 
Tokenizing train dataset:   8%|▊         | 137215/1807941 [01:56<26:56, 1033.50 examples/s]
Tokenizing train dataset:   8%|▊         | 137337/1807941 [01:56<25:47, 1079.86 examples/s]
Tokenizing train dataset:   8%|▊         | 137468/1807941 [01:56<24:25, 1139.74 examples/s]
Tokenizing train dataset:   8%|▊         | 137598/1807941 [01:56<23:33, 1181.86 examples/s]
Tokenizing train dataset:   8%|▊         | 137728/1807941 [01:56<22:56, 1213.37 examples/s]
Tokenizing train dataset:   8%|▊         | 137884/1807941 [01:56<24:20, 1143.85 examples/s]
Tokenizing train dataset:   8%|▊         | 138005/1807941 [01:57<27:11, 1023.67 examples/s]
Tokenizing train dataset:   8%|▊         | 138151/1807941 [01:57<27:52, 998.10 examples/s] 
Tokenizing train dataset:   8%|▊         | 138256/1807941 [01:57<27:35, 1008.47 examples/s]
Tokenizing train dataset:   8%|▊         | 138393/1807941 [01:57<28:43, 968.42 examples/s] 
Tokenizing train dataset:   8%|▊         | 138524/1807941 [01:57<26:30, 1049.49 examples/s]
Tokenizing train dataset:   8%|▊         | 138637/1807941 [01:57<26:00, 1069.85 examples/s]
Tokenizing train dataset:   8%|▊         | 138773/1807941 [01:57<24:16, 1146.34 examples/s]
Tokenizing train dataset:   8%|▊         | 138903/1807941 [01:57<23:28, 1184.60 examples/s]
Tokenizing train dataset:   8%|▊         | 139072/1807941 [01:57<24:05, 1154.50 examples/s]
Tokenizing train dataset:   8%|▊         | 139199/1807941 [01:58<23:30, 1183.11 examples/s]
Tokenizing train dataset:   8%|▊         | 139370/1807941 [01:58<23:54, 1163.05 examples/s]
Tokenizing train dataset:   8%|▊         | 139503/1807941 [01:58<23:07, 1202.68 examples/s]
Tokenizing train dataset:   8%|▊         | 139657/1807941 [01:58<24:27, 1136.78 examples/s]
Tokenizing train dataset:   8%|▊         | 139787/1807941 [01:58<23:40, 1174.56 examples/s]
Tokenizing train dataset:   8%|▊         | 139939/1807941 [01:58<24:54, 1116.09 examples/s]
Tokenizing train dataset:   8%|▊         | 140059/1807941 [01:58<25:56, 1071.43 examples/s]
Tokenizing train dataset:   8%|▊         | 140179/1807941 [01:58<25:13, 1101.74 examples/s]
Tokenizing train dataset:   8%|▊         | 140345/1807941 [01:59<25:14, 1101.12 examples/s]
Tokenizing train dataset:   8%|▊         | 140462/1807941 [01:59<24:53, 1116.28 examples/s]
Tokenizing train dataset:   8%|▊         | 140575/1807941 [01:59<24:51, 1117.54 examples/s]
Tokenizing train dataset:   8%|▊         | 140690/1807941 [01:59<24:43, 1123.90 examples/s]
Tokenizing train dataset:   8%|▊         | 140861/1807941 [01:59<24:41, 1125.22 examples/s]
Tokenizing train dataset:   8%|▊         | 140985/1807941 [01:59<24:07, 1151.43 examples/s]
Tokenizing train dataset:   8%|▊         | 141143/1807941 [01:59<24:57, 1112.97 examples/s]
Tokenizing train dataset:   8%|▊         | 141309/1807941 [01:59<25:11, 1102.75 examples/s]
Tokenizing train dataset:   8%|▊         | 141430/1807941 [02:00<24:40, 1125.45 examples/s]
Tokenizing train dataset:   8%|▊         | 141552/1807941 [02:00<24:10, 1148.92 examples/s]
Tokenizing train dataset:   8%|▊         | 141673/1807941 [02:00<23:52, 1163.37 examples/s]
Tokenizing train dataset:   8%|▊         | 141797/1807941 [02:00<23:27, 1184.18 examples/s]
Tokenizing train dataset:   8%|▊         | 141938/1807941 [02:00<22:16, 1246.58 examples/s]
Tokenizing train dataset:   8%|▊         | 142123/1807941 [02:00<23:40, 1172.35 examples/s]
Tokenizing train dataset:   8%|▊         | 142257/1807941 [02:00<22:52, 1213.21 examples/s]
Tokenizing train dataset:   8%|▊         | 142382/1807941 [02:00<22:43, 1221.57 examples/s]
Tokenizing train dataset:   8%|▊         | 142567/1807941 [02:00<22:43, 1220.97 examples/s]
Tokenizing train dataset:   8%|▊         | 142693/1807941 [02:01<22:36, 1227.76 examples/s]
Tokenizing train dataset:   8%|▊         | 142830/1807941 [02:01<21:59, 1262.30 examples/s]
Tokenizing train dataset:   8%|▊         | 143000/1807941 [02:01<23:23, 1186.18 examples/s]
Tokenizing train dataset:   8%|▊         | 143181/1807941 [02:01<23:18, 1190.07 examples/s]
Tokenizing train dataset:   8%|▊         | 143305/1807941 [02:01<23:09, 1197.85 examples/s]
Tokenizing train dataset:   8%|▊         | 143428/1807941 [02:01<23:08, 1199.04 examples/s]
Tokenizing train dataset:   8%|▊         | 143591/1807941 [02:01<23:58, 1156.88 examples/s]
Tokenizing train dataset:   8%|▊         | 143711/1807941 [02:01<23:49, 1164.34 examples/s]
Tokenizing train dataset:   8%|▊         | 143848/1807941 [02:02<32:11, 861.57 examples/s] 
Tokenizing train dataset:   8%|▊         | 143949/1807941 [02:02<38:02, 729.03 examples/s]
Tokenizing train dataset:   8%|▊         | 144060/1807941 [02:02<38:01, 729.41 examples/s]
Tokenizing train dataset:   8%|▊         | 144178/1807941 [02:02<33:48, 820.17 examples/s]
Tokenizing train dataset:   8%|▊         | 144331/1807941 [02:02<28:21, 977.58 examples/s]
Tokenizing train dataset:   8%|▊         | 144470/1807941 [02:02<25:48, 1074.52 examples/s]
Tokenizing train dataset:   8%|▊         | 144598/1807941 [02:02<24:37, 1125.85 examples/s]
Tokenizing train dataset:   8%|▊         | 144771/1807941 [02:03<24:28, 1132.67 examples/s]
Tokenizing train dataset:   8%|▊         | 144895/1807941 [02:03<23:57, 1156.56 examples/s]
Tokenizing train dataset:   8%|▊         | 145069/1807941 [02:03<24:09, 1146.82 examples/s]
Tokenizing train dataset:   8%|▊         | 145245/1807941 [02:03<24:06, 1149.08 examples/s]
Tokenizing train dataset:   8%|▊         | 145423/1807941 [02:03<23:56, 1157.22 examples/s]
Tokenizing train dataset:   8%|▊         | 145547/1807941 [02:03<23:35, 1174.52 examples/s]
Tokenizing train dataset:   8%|▊         | 145681/1807941 [02:03<25:47, 1074.06 examples/s]
Tokenizing train dataset:   8%|▊         | 145812/1807941 [02:04<24:31, 1129.18 examples/s]
Tokenizing train dataset:   8%|▊         | 145936/1807941 [02:04<23:57, 1155.81 examples/s]
Tokenizing train dataset:   8%|▊         | 146061/1807941 [02:04<24:45, 1118.85 examples/s]
Tokenizing train dataset:   8%|▊         | 146192/1807941 [02:04<27:02, 1023.92 examples/s]
Tokenizing train dataset:   8%|▊         | 146330/1807941 [02:04<24:56, 1110.09 examples/s]
Tokenizing train dataset:   8%|▊         | 146469/1807941 [02:04<23:25, 1182.40 examples/s]
Tokenizing train dataset:   8%|▊         | 146608/1807941 [02:04<22:22, 1237.32 examples/s]
Tokenizing train dataset:   8%|▊         | 146762/1807941 [02:04<23:57, 1155.84 examples/s]
Tokenizing train dataset:   8%|▊         | 146915/1807941 [02:05<25:02, 1105.84 examples/s]
Tokenizing train dataset:   8%|▊         | 147078/1807941 [02:05<25:15, 1095.97 examples/s]
Tokenizing train dataset:   8%|▊         | 147223/1807941 [02:05<23:29, 1177.94 examples/s]
Tokenizing train dataset:   8%|▊         | 147351/1807941 [02:05<23:01, 1202.32 examples/s]
Tokenizing train dataset:   8%|▊         | 147492/1807941 [02:05<22:08, 1250.12 examples/s]
Tokenizing train dataset:   8%|▊         | 147625/1807941 [02:05<21:47, 1269.37 examples/s]
Tokenizing train dataset:   8%|▊         | 147769/1807941 [02:05<21:03, 1314.37 examples/s]
Tokenizing train dataset:   8%|▊         | 147913/1807941 [02:05<20:32, 1346.92 examples/s]
Tokenizing train dataset:   8%|▊         | 148080/1807941 [02:05<22:03, 1254.33 examples/s]
Tokenizing train dataset:   8%|▊         | 148216/1807941 [02:06<21:37, 1279.22 examples/s]
Tokenizing train dataset:   8%|▊         | 148382/1807941 [02:06<22:49, 1211.70 examples/s]
Tokenizing train dataset:   8%|▊         | 148526/1807941 [02:06<21:47, 1268.81 examples/s]
Tokenizing train dataset:   8%|▊         | 148660/1807941 [02:06<21:32, 1283.43 examples/s]
Tokenizing train dataset:   8%|▊         | 148806/1807941 [02:06<20:47, 1329.78 examples/s]
Tokenizing train dataset:   8%|▊         | 148950/1807941 [02:06<20:21, 1358.13 examples/s]
Tokenizing train dataset:   8%|▊         | 149172/1807941 [02:06<20:53, 1323.82 examples/s]
Tokenizing train dataset:   8%|▊         | 149359/1807941 [02:06<21:23, 1291.78 examples/s]
Tokenizing train dataset:   8%|▊         | 149495/1807941 [02:07<21:07, 1307.94 examples/s]
Tokenizing train dataset:   8%|▊         | 149644/1807941 [02:07<20:27, 1350.79 examples/s]
Tokenizing train dataset:   8%|▊         | 149785/1807941 [02:07<20:18, 1360.69 examples/s]
Tokenizing train dataset:   8%|▊         | 149923/1807941 [02:07<20:16, 1362.43 examples/s]
Tokenizing train dataset:   8%|▊         | 150097/1807941 [02:07<21:30, 1284.42 examples/s]
Tokenizing train dataset:   8%|▊         | 150234/1807941 [02:07<21:09, 1306.17 examples/s]
Tokenizing train dataset:   8%|▊         | 150376/1807941 [02:07<20:41, 1334.59 examples/s]
Tokenizing train dataset:   8%|▊         | 150521/1807941 [02:07<20:16, 1362.96 examples/s]
Tokenizing train dataset:   8%|▊         | 150674/1807941 [02:07<19:39, 1405.53 examples/s]
Tokenizing train dataset:   8%|▊         | 150842/1807941 [02:08<24:20, 1134.95 examples/s]
Tokenizing train dataset:   8%|▊         | 150985/1807941 [02:08<28:39, 963.70 examples/s] 
Tokenizing train dataset:   8%|▊         | 151114/1807941 [02:08<32:41, 844.82 examples/s]
Tokenizing train dataset:   8%|▊         | 151232/1807941 [02:08<33:36, 821.53 examples/s]
Tokenizing train dataset:   8%|▊         | 151359/1807941 [02:08<33:30, 823.86 examples/s]
Tokenizing train dataset:   8%|▊         | 151507/1807941 [02:08<31:55, 864.59 examples/s]
Tokenizing train dataset:   8%|▊         | 151607/1807941 [02:09<34:21, 803.30 examples/s]
Tokenizing train dataset:   8%|▊         | 151699/1807941 [02:09<33:23, 826.80 examples/s]
Tokenizing train dataset:   8%|▊         | 151805/1807941 [02:09<31:22, 879.70 examples/s]
Tokenizing train dataset:   8%|▊         | 151942/1807941 [02:09<27:35, 1000.26 examples/s]
Tokenizing train dataset:   8%|▊         | 152075/1807941 [02:09<26:49, 1028.98 examples/s]
Tokenizing train dataset:   8%|▊         | 152225/1807941 [02:09<23:58, 1150.88 examples/s]
Tokenizing train dataset:   8%|▊         | 152376/1807941 [02:09<22:08, 1246.37 examples/s]
Tokenizing train dataset:   8%|▊         | 152522/1807941 [02:09<21:11, 1301.61 examples/s]
Tokenizing train dataset:   8%|▊         | 152695/1807941 [02:09<22:12, 1242.13 examples/s]
Tokenizing train dataset:   8%|▊         | 152835/1807941 [02:10<21:32, 1280.93 examples/s]
Tokenizing train dataset:   8%|▊         | 152976/1807941 [02:10<21:00, 1313.07 examples/s]
Tokenizing train dataset:   8%|▊         | 153160/1807941 [02:10<21:37, 1274.88 examples/s]
Tokenizing train dataset:   8%|▊         | 153333/1807941 [02:10<22:28, 1227.22 examples/s]
Tokenizing train dataset:   8%|▊         | 153482/1807941 [02:10<21:24, 1288.43 examples/s]
Tokenizing train dataset:   8%|▊         | 153625/1807941 [02:10<20:51, 1321.93 examples/s]
Tokenizing train dataset:   9%|▊         | 153817/1807941 [02:10<21:11, 1301.37 examples/s]
Tokenizing train dataset:   9%|▊         | 154000/1807941 [02:11<22:40, 1216.05 examples/s]
Tokenizing train dataset:   9%|▊         | 154125/1807941 [02:11<22:33, 1221.76 examples/s]
Tokenizing train dataset:   9%|▊         | 154252/1807941 [02:11<22:25, 1229.33 examples/s]
Tokenizing train dataset:   9%|▊         | 154383/1807941 [02:11<22:04, 1248.88 examples/s]
Tokenizing train dataset:   9%|▊         | 154512/1807941 [02:11<21:55, 1256.66 examples/s]
Tokenizing train dataset:   9%|▊         | 154644/1807941 [02:11<21:39, 1272.22 examples/s]
Tokenizing train dataset:   9%|▊         | 154778/1807941 [02:11<21:22, 1289.10 examples/s]
Tokenizing train dataset:   9%|▊         | 154917/1807941 [02:11<24:01, 1146.87 examples/s]
Tokenizing train dataset:   9%|▊         | 155055/1807941 [02:11<29:03, 947.90 examples/s] 
Tokenizing train dataset:   9%|▊         | 155186/1807941 [02:12<26:46, 1028.79 examples/s]
Tokenizing train dataset:   9%|▊         | 155321/1807941 [02:12<24:55, 1105.31 examples/s]
Tokenizing train dataset:   9%|▊         | 155451/1807941 [02:12<23:54, 1152.14 examples/s]
Tokenizing train dataset:   9%|▊         | 155589/1807941 [02:12<22:44, 1211.22 examples/s]
Tokenizing train dataset:   9%|▊         | 155724/1807941 [02:12<22:05, 1246.14 examples/s]
Tokenizing train dataset:   9%|▊         | 155881/1807941 [02:12<26:33, 1036.74 examples/s]
Tokenizing train dataset:   9%|▊         | 156000/1807941 [02:12<26:32, 1037.20 examples/s]
Tokenizing train dataset:   9%|▊         | 156118/1807941 [02:12<25:41, 1071.49 examples/s]
Tokenizing train dataset:   9%|▊         | 156249/1807941 [02:12<24:21, 1130.04 examples/s]
Tokenizing train dataset:   9%|▊         | 156384/1807941 [02:13<23:12, 1186.13 examples/s]
Tokenizing train dataset:   9%|▊         | 156513/1807941 [02:13<22:43, 1211.20 examples/s]
Tokenizing train dataset:   9%|▊         | 156644/1807941 [02:13<22:14, 1237.29 examples/s]
Tokenizing train dataset:   9%|▊         | 156827/1807941 [02:13<22:24, 1228.16 examples/s]
Tokenizing train dataset:   9%|▊         | 156965/1807941 [02:13<21:43, 1266.32 examples/s]
Tokenizing train dataset:   9%|▊         | 157130/1807941 [02:13<22:55, 1200.48 examples/s]
Tokenizing train dataset:   9%|▊         | 157317/1807941 [02:13<22:42, 1211.21 examples/s]
Tokenizing train dataset:   9%|▊         | 157502/1807941 [02:14<22:37, 1215.69 examples/s]
Tokenizing train dataset:   9%|▊         | 157673/1807941 [02:14<23:07, 1189.44 examples/s]
Tokenizing train dataset:   9%|▊         | 157848/1807941 [02:14<23:29, 1170.82 examples/s]
Tokenizing train dataset:   9%|▊         | 157977/1807941 [02:14<28:24, 967.92 examples/s] 
Tokenizing train dataset:   9%|▊         | 158122/1807941 [02:14<28:29, 964.96 examples/s]
Tokenizing train dataset:   9%|▉         | 158262/1807941 [02:14<26:02, 1055.63 examples/s]
Tokenizing train dataset:   9%|▉         | 158386/1807941 [02:14<25:11, 1091.02 examples/s]
Tokenizing train dataset:   9%|▉         | 158565/1807941 [02:15<24:30, 1121.95 examples/s]
Tokenizing train dataset:   9%|▉         | 158687/1807941 [02:15<24:02, 1143.02 examples/s]
Tokenizing train dataset:   9%|▉         | 158818/1807941 [02:15<23:14, 1182.90 examples/s]
Tokenizing train dataset:   9%|▉         | 158950/1807941 [02:15<22:35, 1216.14 examples/s]
Tokenizing train dataset:   9%|▉         | 159116/1807941 [02:15<24:16, 1132.39 examples/s]
Tokenizing train dataset:   9%|▉         | 159243/1807941 [02:15<23:38, 1162.32 examples/s]
Tokenizing train dataset:   9%|▉         | 159379/1807941 [02:15<25:41, 1069.76 examples/s]
Tokenizing train dataset:   9%|▉         | 159517/1807941 [02:15<30:08, 911.54 examples/s] 
Tokenizing train dataset:   9%|▉         | 159635/1807941 [02:16<31:33, 870.43 examples/s]
Tokenizing train dataset:   9%|▉         | 159737/1807941 [02:16<30:27, 901.74 examples/s]
Tokenizing train dataset:   9%|▉         | 159884/1807941 [02:16<33:19, 824.12 examples/s]
Tokenizing train dataset:   9%|▉         | 160000/1807941 [02:16<32:33, 843.44 examples/s]
Tokenizing train dataset:   9%|▉         | 160129/1807941 [02:16<29:11, 941.05 examples/s]
Tokenizing train dataset:   9%|▉         | 160265/1807941 [02:16<26:21, 1041.83 examples/s]
Tokenizing train dataset:   9%|▉         | 160385/1807941 [02:16<25:24, 1080.62 examples/s]
Tokenizing train dataset:   9%|▉         | 160537/1807941 [02:16<26:03, 1053.93 examples/s]
Tokenizing train dataset:   9%|▉         | 160688/1807941 [02:17<26:30, 1035.64 examples/s]
Tokenizing train dataset:   9%|▉         | 160817/1807941 [02:17<25:02, 1096.07 examples/s]
Tokenizing train dataset:   9%|▉         | 160944/1807941 [02:17<24:07, 1137.62 examples/s]
Tokenizing train dataset:   9%|▉         | 161070/1807941 [02:17<24:23, 1125.29 examples/s]
Tokenizing train dataset:   9%|▉         | 161201/1807941 [02:17<23:23, 1173.53 examples/s]
Tokenizing train dataset:   9%|▉         | 161332/1807941 [02:17<22:41, 1209.09 examples/s]
Tokenizing train dataset:   9%|▉         | 161465/1807941 [02:17<22:04, 1243.11 examples/s]
Tokenizing train dataset:   9%|▉         | 161653/1807941 [02:17<22:03, 1243.77 examples/s]
Tokenizing train dataset:   9%|▉         | 161833/1807941 [02:18<22:22, 1225.93 examples/s]
Tokenizing train dataset:   9%|▉         | 161968/1807941 [02:18<21:52, 1253.93 examples/s]
Tokenizing train dataset:   9%|▉         | 162115/1807941 [02:18<26:40, 1028.37 examples/s]
Tokenizing train dataset:   9%|▉         | 162238/1807941 [02:18<25:35, 1072.03 examples/s]
Tokenizing train dataset:   9%|▉         | 162364/1807941 [02:18<24:32, 1117.45 examples/s]
Tokenizing train dataset:   9%|▉         | 162496/1807941 [02:18<23:27, 1169.30 examples/s]
Tokenizing train dataset:   9%|▉         | 162683/1807941 [02:18<23:01, 1191.01 examples/s]
Tokenizing train dataset:   9%|▉         | 162843/1807941 [02:18<21:13, 1291.70 examples/s]
Tokenizing train dataset:   9%|▉         | 162990/1807941 [02:19<20:30, 1336.60 examples/s]
Tokenizing train dataset:   9%|▉         | 163178/1807941 [02:19<29:19, 934.93 examples/s] 
Tokenizing train dataset:   9%|▉         | 163295/1807941 [02:19<27:58, 980.00 examples/s]
Tokenizing train dataset:   9%|▉         | 163413/1807941 [02:19<26:48, 1022.46 examples/s]
Tokenizing train dataset:   9%|▉         | 163532/1807941 [02:19<25:51, 1060.19 examples/s]
Tokenizing train dataset:   9%|▉         | 163664/1807941 [02:19<24:22, 1124.37 examples/s]
Tokenizing train dataset:   9%|▉         | 163802/1807941 [02:19<23:02, 1189.48 examples/s]
Tokenizing train dataset:   9%|▉         | 163986/1807941 [02:19<22:51, 1198.43 examples/s]
Tokenizing train dataset:   9%|▉         | 164135/1807941 [02:20<24:22, 1124.09 examples/s]
Tokenizing train dataset:   9%|▉         | 164318/1807941 [02:20<23:46, 1152.15 examples/s]
Tokenizing train dataset:   9%|▉         | 164444/1807941 [02:20<23:17, 1175.76 examples/s]
Tokenizing train dataset:   9%|▉         | 164577/1807941 [02:20<22:35, 1212.21 examples/s]
Tokenizing train dataset:   9%|▉         | 164754/1807941 [02:20<22:51, 1198.01 examples/s]
Tokenizing train dataset:   9%|▉         | 164876/1807941 [02:20<22:51, 1198.38 examples/s]
Tokenizing train dataset:   9%|▉         | 165000/1807941 [02:20<24:11, 1131.87 examples/s]
Tokenizing train dataset:   9%|▉         | 165116/1807941 [02:21<27:25, 998.10 examples/s] 
Tokenizing train dataset:   9%|▉         | 165245/1807941 [02:21<32:09, 851.15 examples/s]
Tokenizing train dataset:   9%|▉         | 165340/1807941 [02:21<35:01, 781.74 examples/s]
Tokenizing train dataset:   9%|▉         | 165445/1807941 [02:21<36:12, 756.08 examples/s]
Tokenizing train dataset:   9%|▉         | 165569/1807941 [02:21<31:49, 860.28 examples/s]
Tokenizing train dataset:   9%|▉         | 165703/1807941 [02:21<28:09, 972.31 examples/s]
Tokenizing train dataset:   9%|▉         | 165829/1807941 [02:21<26:14, 1042.79 examples/s]
Tokenizing train dataset:   9%|▉         | 165954/1807941 [02:21<24:59, 1095.30 examples/s]
Tokenizing train dataset:   9%|▉         | 166110/1807941 [02:22<26:49, 1019.96 examples/s]
Tokenizing train dataset:   9%|▉         | 166227/1807941 [02:22<25:56, 1054.99 examples/s]
Tokenizing train dataset:   9%|▉         | 166351/1807941 [02:22<24:52, 1100.05 examples/s]
Tokenizing train dataset:   9%|▉         | 166469/1807941 [02:22<24:26, 1119.42 examples/s]
Tokenizing train dataset:   9%|▉         | 166591/1807941 [02:22<23:53, 1144.81 examples/s]
Tokenizing train dataset:   9%|▉         | 166757/1807941 [02:22<24:20, 1123.65 examples/s]
Tokenizing train dataset:   9%|▉         | 166884/1807941 [02:22<26:41, 1024.65 examples/s]
Tokenizing train dataset:   9%|▉         | 166994/1807941 [02:22<26:17, 1039.91 examples/s]
Tokenizing train dataset:   9%|▉         | 167162/1807941 [02:23<28:53, 946.60 examples/s] 
Tokenizing train dataset:   9%|▉         | 167281/1807941 [02:23<27:26, 996.69 examples/s]
Tokenizing train dataset:   9%|▉         | 167401/1807941 [02:23<26:09, 1045.58 examples/s]
Tokenizing train dataset:   9%|▉         | 167525/1807941 [02:23<25:01, 1092.34 examples/s]
Tokenizing train dataset:   9%|▉         | 167656/1807941 [02:23<23:46, 1149.48 examples/s]
Tokenizing train dataset:   9%|▉         | 167804/1807941 [02:23<25:10, 1085.86 examples/s]
Tokenizing train dataset:   9%|▉         | 167975/1807941 [02:23<27:50, 981.74 examples/s] 
Tokenizing train dataset:   9%|▉         | 168115/1807941 [02:24<28:19, 964.91 examples/s]
Tokenizing train dataset:   9%|▉         | 168243/1807941 [02:24<26:24, 1034.66 examples/s]
Tokenizing train dataset:   9%|▉         | 168377/1807941 [02:24<24:40, 1107.12 examples/s]
Tokenizing train dataset:   9%|▉         | 168494/1807941 [02:24<24:24, 1119.81 examples/s]
Tokenizing train dataset:   9%|▉         | 168636/1807941 [02:24<22:48, 1198.32 examples/s]
Tokenizing train dataset:   9%|▉         | 168769/1807941 [02:24<22:10, 1231.54 examples/s]
Tokenizing train dataset:   9%|▉         | 168901/1807941 [02:24<21:47, 1253.60 examples/s]
Tokenizing train dataset:   9%|▉         | 169072/1807941 [02:24<22:40, 1204.77 examples/s]
Tokenizing train dataset:   9%|▉         | 169203/1807941 [02:24<22:15, 1227.16 examples/s]
Tokenizing train dataset:   9%|▉         | 169339/1807941 [02:24<21:39, 1260.75 examples/s]
Tokenizing train dataset:   9%|▉         | 169519/1807941 [02:25<24:54, 1096.29 examples/s]
Tokenizing train dataset:   9%|▉         | 169684/1807941 [02:25<24:57, 1094.25 examples/s]
Tokenizing train dataset:   9%|▉         | 169857/1807941 [02:25<24:36, 1109.66 examples/s]
Tokenizing train dataset:   9%|▉         | 169994/1807941 [02:25<23:22, 1167.80 examples/s]
Tokenizing train dataset:   9%|▉         | 170155/1807941 [02:25<24:03, 1134.84 examples/s]
Tokenizing train dataset:   9%|▉         | 170281/1807941 [02:25<23:27, 1163.49 examples/s]
Tokenizing train dataset:   9%|▉         | 170410/1807941 [02:25<22:50, 1194.74 examples/s]
Tokenizing train dataset:   9%|▉         | 170547/1807941 [02:26<22:02, 1237.81 examples/s]
Tokenizing train dataset:   9%|▉         | 170683/1807941 [02:26<21:29, 1269.38 examples/s]
Tokenizing train dataset:   9%|▉         | 170816/1807941 [02:26<21:15, 1283.26 examples/s]
Tokenizing train dataset:   9%|▉         | 171000/1807941 [02:26<22:03, 1236.97 examples/s]
Tokenizing train dataset:   9%|▉         | 171133/1807941 [02:26<21:40, 1258.93 examples/s]
Tokenizing train dataset:   9%|▉         | 171275/1807941 [02:26<20:57, 1301.77 examples/s]
Tokenizing train dataset:   9%|▉         | 171425/1807941 [02:26<20:06, 1355.94 examples/s]
Tokenizing train dataset:   9%|▉         | 171576/1807941 [02:26<19:31, 1396.81 examples/s]
Tokenizing train dataset:  10%|▉         | 171784/1807941 [02:26<19:35, 1391.65 examples/s]
Tokenizing train dataset:  10%|▉         | 171945/1807941 [02:27<21:28, 1269.99 examples/s]
Tokenizing train dataset:  10%|▉         | 172075/1807941 [02:27<24:52, 1096.19 examples/s]
Tokenizing train dataset:  10%|▉         | 172212/1807941 [02:27<23:31, 1158.51 examples/s]
Tokenizing train dataset:  10%|▉         | 172359/1807941 [02:27<22:06, 1232.94 examples/s]
Tokenizing train dataset:  10%|▉         | 172496/1807941 [02:27<21:29, 1267.88 examples/s]
Tokenizing train dataset:  10%|▉         | 172660/1807941 [02:27<22:44, 1198.26 examples/s]
Tokenizing train dataset:  10%|▉         | 172796/1807941 [02:27<22:01, 1237.23 examples/s]
Tokenizing train dataset:  10%|▉         | 172936/1807941 [02:27<21:18, 1278.90 examples/s]
Tokenizing train dataset:  10%|▉         | 173067/1807941 [02:28<22:18, 1221.64 examples/s]
Tokenizing train dataset:  10%|▉         | 173212/1807941 [02:28<21:17, 1279.47 examples/s]
Tokenizing train dataset:  10%|▉         | 173350/1807941 [02:28<20:53, 1304.21 examples/s]
Tokenizing train dataset:  10%|▉         | 173489/1807941 [02:28<20:35, 1322.38 examples/s]
Tokenizing train dataset:  10%|▉         | 173625/1807941 [02:28<20:27, 1331.67 examples/s]
Tokenizing train dataset:  10%|▉         | 173768/1807941 [02:28<20:05, 1355.50 examples/s]
Tokenizing train dataset:  10%|▉         | 173942/1807941 [02:28<21:19, 1277.41 examples/s]
Tokenizing train dataset:  10%|▉         | 174076/1807941 [02:28<23:54, 1139.29 examples/s]
Tokenizing train dataset:  10%|▉         | 174216/1807941 [02:28<22:38, 1202.18 examples/s]
Tokenizing train dataset:  10%|▉         | 174352/1807941 [02:29<24:51, 1095.56 examples/s]
Tokenizing train dataset:  10%|▉         | 174473/1807941 [02:29<24:20, 1118.32 examples/s]
Tokenizing train dataset:  10%|▉         | 174606/1807941 [02:29<23:14, 1171.53 examples/s]
Tokenizing train dataset:  10%|▉         | 174735/1807941 [02:29<22:39, 1201.01 examples/s]
Tokenizing train dataset:  10%|▉         | 174860/1807941 [02:29<22:29, 1209.90 examples/s]
Tokenizing train dataset:  10%|▉         | 174994/1807941 [02:29<25:00, 1088.41 examples/s]
Tokenizing train dataset:  10%|▉         | 175139/1807941 [02:29<26:06, 1042.25 examples/s]
Tokenizing train dataset:  10%|▉         | 175256/1807941 [02:29<28:33, 952.94 examples/s] 
Tokenizing train dataset:  10%|▉         | 175382/1807941 [02:30<33:07, 821.21 examples/s]
Tokenizing train dataset:  10%|▉         | 175534/1807941 [02:30<31:04, 875.40 examples/s]
Tokenizing train dataset:  10%|▉         | 175628/1807941 [02:30<30:37, 888.18 examples/s]
Tokenizing train dataset:  10%|▉         | 175743/1807941 [02:30<28:40, 948.50 examples/s]
Tokenizing train dataset:  10%|▉         | 175895/1807941 [02:30<28:05, 968.38 examples/s]
Tokenizing train dataset:  10%|▉         | 176000/1807941 [02:30<31:58, 850.46 examples/s]
Tokenizing train dataset:  10%|▉         | 176103/1807941 [02:30<30:34, 889.38 examples/s]
Tokenizing train dataset:  10%|▉         | 176214/1807941 [02:31<32:37, 833.55 examples/s]
Tokenizing train dataset:  10%|▉         | 176316/1807941 [02:31<31:10, 872.16 examples/s]
Tokenizing train dataset:  10%|▉         | 176416/1807941 [02:31<30:04, 904.12 examples/s]
Tokenizing train dataset:  10%|▉         | 176544/1807941 [02:31<27:07, 1002.31 examples/s]
Tokenizing train dataset:  10%|▉         | 176680/1807941 [02:31<24:43, 1099.52 examples/s]
Tokenizing train dataset:  10%|▉         | 176815/1807941 [02:31<23:17, 1167.35 examples/s]
Tokenizing train dataset:  10%|▉         | 176980/1807941 [02:31<23:55, 1135.94 examples/s]
Tokenizing train dataset:  10%|▉         | 177125/1807941 [02:31<25:15, 1075.89 examples/s]
Tokenizing train dataset:  10%|▉         | 177255/1807941 [02:32<24:04, 1128.73 examples/s]
Tokenizing train dataset:  10%|▉         | 177378/1807941 [02:32<23:36, 1151.23 examples/s]
Tokenizing train dataset:  10%|▉         | 177500/1807941 [02:32<23:17, 1166.65 examples/s]
Tokenizing train dataset:  10%|▉         | 177646/1807941 [02:32<24:52, 1092.42 examples/s]
Tokenizing train dataset:  10%|▉         | 177782/1807941 [02:32<29:58, 906.30 examples/s] 
Tokenizing train dataset:  10%|▉         | 177901/1807941 [02:32<28:03, 968.00 examples/s]
Tokenizing train dataset:  10%|▉         | 178059/1807941 [02:32<27:41, 980.91 examples/s]
Tokenizing train dataset:  10%|▉         | 178182/1807941 [02:32<29:19, 926.27 examples/s]
Tokenizing train dataset:  10%|▉         | 178280/1807941 [02:33<32:32, 834.68 examples/s]
Tokenizing train dataset:  10%|▉         | 178392/1807941 [02:33<30:18, 896.09 examples/s]
Tokenizing train dataset:  10%|▉         | 178495/1807941 [02:33<29:15, 928.29 examples/s]
Tokenizing train dataset:  10%|▉         | 178609/1807941 [02:33<35:02, 774.77 examples/s]
Tokenizing train dataset:  10%|▉         | 178723/1807941 [02:33<31:43, 856.08 examples/s]
Tokenizing train dataset:  10%|▉         | 178832/1807941 [02:33<29:46, 911.73 examples/s]
Tokenizing train dataset:  10%|▉         | 178959/1807941 [02:33<27:09, 999.82 examples/s]
Tokenizing train dataset:  10%|▉         | 179123/1807941 [02:34<27:41, 980.25 examples/s]
Tokenizing train dataset:  10%|▉         | 179246/1807941 [02:34<26:08, 1038.32 examples/s]
Tokenizing train dataset:  10%|▉         | 179376/1807941 [02:34<24:39, 1100.44 examples/s]
Tokenizing train dataset:  10%|▉         | 179511/1807941 [02:34<23:17, 1164.83 examples/s]
Tokenizing train dataset:  10%|▉         | 179656/1807941 [02:34<24:53, 1090.16 examples/s]
Tokenizing train dataset:  10%|▉         | 179779/1807941 [02:34<24:07, 1124.96 examples/s]
Tokenizing train dataset:  10%|▉         | 179950/1807941 [02:34<24:05, 1126.37 examples/s]
Tokenizing train dataset:  10%|▉         | 180109/1807941 [02:34<25:49, 1050.77 examples/s]
Tokenizing train dataset:  10%|▉         | 180221/1807941 [02:35<25:26, 1066.27 examples/s]
Tokenizing train dataset:  10%|▉         | 180341/1807941 [02:35<27:49, 974.99 examples/s] 
Tokenizing train dataset:  10%|▉         | 180482/1807941 [02:35<25:11, 1076.72 examples/s]
Tokenizing train dataset:  10%|▉         | 180603/1807941 [02:35<24:30, 1106.53 examples/s]
Tokenizing train dataset:  10%|▉         | 180722/1807941 [02:35<24:02, 1127.91 examples/s]
Tokenizing train dataset:  10%|█         | 180848/1807941 [02:35<23:21, 1160.94 examples/s]
Tokenizing train dataset:  10%|█         | 180976/1807941 [02:35<22:49, 1188.39 examples/s]
Tokenizing train dataset:  10%|█         | 181111/1807941 [02:35<25:16, 1072.72 examples/s]
Tokenizing train dataset:  10%|█         | 181247/1807941 [02:35<26:52, 1008.56 examples/s]
Tokenizing train dataset:  10%|█         | 181360/1807941 [02:36<26:08, 1037.20 examples/s]
Tokenizing train dataset:  10%|█         | 181520/1807941 [02:36<25:55, 1045.65 examples/s]
Tokenizing train dataset:  10%|█         | 181641/1807941 [02:36<24:58, 1085.15 examples/s]
Tokenizing train dataset:  10%|█         | 181756/1807941 [02:36<24:37, 1100.40 examples/s]
Tokenizing train dataset:  10%|█         | 181879/1807941 [02:36<23:55, 1132.58 examples/s]
Tokenizing train dataset:  10%|█         | 182000/1807941 [02:36<25:11, 1075.58 examples/s]
Tokenizing train dataset:  10%|█         | 182117/1807941 [02:36<24:40, 1098.49 examples/s]
Tokenizing train dataset:  10%|█         | 182242/1807941 [02:36<23:48, 1138.21 examples/s]
Tokenizing train dataset:  10%|█         | 182359/1807941 [02:36<23:38, 1146.26 examples/s]
Tokenizing train dataset:  10%|█         | 182476/1807941 [02:37<23:31, 1151.45 examples/s]
Tokenizing train dataset:  10%|█         | 182593/1807941 [02:37<23:25, 1156.13 examples/s]
Tokenizing train dataset:  10%|█         | 182712/1807941 [02:37<23:14, 1165.55 examples/s]
Tokenizing train dataset:  10%|█         | 182834/1807941 [02:37<23:02, 1175.78 examples/s]
Tokenizing train dataset:  10%|█         | 182977/1807941 [02:37<25:00, 1082.78 examples/s]
Tokenizing train dataset:  10%|█         | 183091/1807941 [02:37<28:02, 965.60 examples/s] 
Tokenizing train dataset:  10%|█         | 183210/1807941 [02:37<26:34, 1018.93 examples/s]
Tokenizing train dataset:  10%|█         | 183337/1807941 [02:37<25:02, 1081.17 examples/s]
Tokenizing train dataset:  10%|█         | 183456/1807941 [02:37<24:23, 1109.72 examples/s]
Tokenizing train dataset:  10%|█         | 183573/1807941 [02:38<24:06, 1122.75 examples/s]
Tokenizing train dataset:  10%|█         | 183693/1807941 [02:38<23:49, 1136.20 examples/s]
Tokenizing train dataset:  10%|█         | 183811/1807941 [02:38<23:36, 1146.50 examples/s]
Tokenizing train dataset:  10%|█         | 183927/1807941 [02:38<23:34, 1147.96 examples/s]
Tokenizing train dataset:  10%|█         | 184083/1807941 [02:38<24:32, 1102.74 examples/s]
Tokenizing train dataset:  10%|█         | 184205/1807941 [02:38<23:56, 1130.08 examples/s]
Tokenizing train dataset:  10%|█         | 184321/1807941 [02:38<23:51, 1134.48 examples/s]
Tokenizing train dataset:  10%|█         | 184446/1807941 [02:38<23:13, 1164.87 examples/s]
Tokenizing train dataset:  10%|█         | 184568/1807941 [02:38<22:56, 1179.46 examples/s]
Tokenizing train dataset:  10%|█         | 184743/1807941 [02:39<23:05, 1171.72 examples/s]
Tokenizing train dataset:  10%|█         | 184888/1807941 [02:39<24:42, 1094.48 examples/s]
Tokenizing train dataset:  10%|█         | 185013/1807941 [02:39<26:58, 1002.57 examples/s]
Tokenizing train dataset:  10%|█         | 185130/1807941 [02:39<26:01, 1039.56 examples/s]
Tokenizing train dataset:  10%|█         | 185262/1807941 [02:39<27:29, 983.83 examples/s] 
Tokenizing train dataset:  10%|█         | 185414/1807941 [02:39<27:21, 988.36 examples/s]
Tokenizing train dataset:  10%|█         | 185538/1807941 [02:39<28:58, 933.39 examples/s]
Tokenizing train dataset:  10%|█         | 185648/1807941 [02:40<31:10, 867.25 examples/s]
Tokenizing train dataset:  10%|█         | 185772/1807941 [02:40<28:25, 950.91 examples/s]
Tokenizing train dataset:  10%|█         | 185911/1807941 [02:40<25:33, 1057.45 examples/s]
Tokenizing train dataset:  10%|█         | 186061/1807941 [02:40<26:36, 1016.14 examples/s]
Tokenizing train dataset:  10%|█         | 186186/1807941 [02:40<25:13, 1071.50 examples/s]
Tokenizing train dataset:  10%|█         | 186317/1807941 [02:40<23:51, 1132.45 examples/s]
Tokenizing train dataset:  10%|█         | 186488/1807941 [02:40<23:50, 1133.11 examples/s]
Tokenizing train dataset:  10%|█         | 186607/1807941 [02:40<23:35, 1145.72 examples/s]
Tokenizing train dataset:  10%|█         | 186727/1807941 [02:40<23:19, 1158.33 examples/s]
Tokenizing train dataset:  10%|█         | 186902/1807941 [02:41<23:19, 1158.71 examples/s]
Tokenizing train dataset:  10%|█         | 187052/1807941 [02:41<24:40, 1094.67 examples/s]
Tokenizing train dataset:  10%|█         | 187188/1807941 [02:41<26:22, 1024.31 examples/s]
Tokenizing train dataset:  10%|█         | 187301/1807941 [02:41<28:53, 934.80 examples/s] 
Tokenizing train dataset:  10%|█         | 187426/1807941 [02:41<26:48, 1007.17 examples/s]
Tokenizing train dataset:  10%|█         | 187555/1807941 [02:41<25:06, 1075.45 examples/s]
Tokenizing train dataset:  10%|█         | 187699/1807941 [02:41<23:07, 1167.44 examples/s]
Tokenizing train dataset:  10%|█         | 187830/1807941 [02:42<22:28, 1201.69 examples/s]
Tokenizing train dataset:  10%|█         | 187959/1807941 [02:42<22:04, 1223.18 examples/s]
Tokenizing train dataset:  10%|█         | 188115/1807941 [02:42<23:55, 1128.02 examples/s]
Tokenizing train dataset:  10%|█         | 188244/1807941 [02:42<23:08, 1166.77 examples/s]
Tokenizing train dataset:  10%|█         | 188404/1807941 [02:42<26:47, 1007.40 examples/s]
Tokenizing train dataset:  10%|█         | 188517/1807941 [02:42<29:12, 924.26 examples/s] 
Tokenizing train dataset:  10%|█         | 188654/1807941 [02:42<29:32, 913.59 examples/s]
Tokenizing train dataset:  10%|█         | 188779/1807941 [02:42<27:19, 987.84 examples/s]
Tokenizing train dataset:  10%|█         | 188899/1807941 [02:43<26:01, 1036.98 examples/s]
Tokenizing train dataset:  10%|█         | 189064/1807941 [02:43<25:58, 1038.92 examples/s]
Tokenizing train dataset:  10%|█         | 189199/1807941 [02:43<27:28, 982.24 examples/s] 
Tokenizing train dataset:  10%|█         | 189316/1807941 [02:43<32:35, 827.62 examples/s]
Tokenizing train dataset:  10%|█         | 189456/1807941 [02:43<31:29, 856.52 examples/s]
Tokenizing train dataset:  10%|█         | 189589/1807941 [02:43<28:11, 956.61 examples/s]
Tokenizing train dataset:  10%|█         | 189718/1807941 [02:43<26:09, 1031.36 examples/s]
Tokenizing train dataset:  11%|█         | 189871/1807941 [02:44<23:22, 1153.83 examples/s]
Tokenizing train dataset:  11%|█         | 190011/1807941 [02:44<22:10, 1216.42 examples/s]
Tokenizing train dataset:  11%|█         | 190198/1807941 [02:44<19:20, 1394.45 examples/s]
Tokenizing train dataset:  11%|█         | 190387/1807941 [02:44<17:36, 1530.61 examples/s]
Tokenizing train dataset:  11%|█         | 190560/1807941 [02:44<19:28, 1384.40 examples/s]
Tokenizing train dataset:  11%|█         | 190751/1807941 [02:44<20:04, 1342.17 examples/s]
Tokenizing train dataset:  11%|█         | 190946/1807941 [02:44<20:21, 1323.52 examples/s]
Tokenizing train dataset:  11%|█         | 191129/1807941 [02:44<21:15, 1267.17 examples/s]
Tokenizing train dataset:  11%|█         | 191311/1807941 [02:45<21:36, 1247.09 examples/s]
Tokenizing train dataset:  11%|█         | 191471/1807941 [02:45<22:40, 1187.95 examples/s]
Tokenizing train dataset:  11%|█         | 191619/1807941 [02:45<24:00, 1122.08 examples/s]
Tokenizing train dataset:  11%|█         | 191761/1807941 [02:45<22:39, 1188.38 examples/s]
Tokenizing train dataset:  11%|█         | 191922/1807941 [02:45<23:27, 1148.20 examples/s]
Tokenizing train dataset:  11%|█         | 192078/1807941 [02:45<24:14, 1110.97 examples/s]
Tokenizing train dataset:  11%|█         | 192215/1807941 [02:45<23:02, 1168.67 examples/s]
Tokenizing train dataset:  11%|█         | 192341/1807941 [02:46<22:38, 1189.68 examples/s]
Tokenizing train dataset:  11%|█         | 192528/1807941 [02:46<22:21, 1204.02 examples/s]
Tokenizing train dataset:  11%|█         | 192694/1807941 [02:46<22:59, 1170.60 examples/s]
Tokenizing train dataset:  11%|█         | 192886/1807941 [02:46<22:21, 1204.36 examples/s]
Tokenizing train dataset:  11%|█         | 193069/1807941 [02:46<22:27, 1198.26 examples/s]
Tokenizing train dataset:  11%|█         | 193202/1807941 [02:46<21:58, 1224.42 examples/s]
Tokenizing train dataset:  11%|█         | 193383/1807941 [02:46<22:06, 1217.51 examples/s]
Tokenizing train dataset:  11%|█         | 193520/1807941 [02:47<24:07, 1115.45 examples/s]
Tokenizing train dataset:  11%|█         | 193639/1807941 [02:47<23:47, 1130.54 examples/s]
Tokenizing train dataset:  11%|█         | 193765/1807941 [02:47<23:11, 1160.33 examples/s]
Tokenizing train dataset:  11%|█         | 193942/1807941 [02:47<23:09, 1161.82 examples/s]
Tokenizing train dataset:  11%|█         | 194063/1807941 [02:47<24:07, 1115.06 examples/s]
Tokenizing train dataset:  11%|█         | 194189/1807941 [02:47<23:25, 1148.39 examples/s]
Tokenizing train dataset:  11%|█         | 194343/1807941 [02:47<24:23, 1102.91 examples/s]
Tokenizing train dataset:  11%|█         | 194508/1807941 [02:47<24:29, 1098.05 examples/s]
Tokenizing train dataset:  11%|█         | 194640/1807941 [02:48<23:23, 1149.35 examples/s]
Tokenizing train dataset:  11%|█         | 194760/1807941 [02:48<23:08, 1161.72 examples/s]
Tokenizing train dataset:  11%|█         | 194882/1807941 [02:48<26:01, 1032.95 examples/s]
Tokenizing train dataset:  11%|█         | 195000/1807941 [02:48<26:40, 1008.02 examples/s]
Tokenizing train dataset:  11%|█         | 195127/1807941 [02:48<25:04, 1071.70 examples/s]
Tokenizing train dataset:  11%|█         | 195248/1807941 [02:48<24:19, 1105.23 examples/s]
Tokenizing train dataset:  11%|█         | 195372/1807941 [02:48<23:33, 1141.05 examples/s]
Tokenizing train dataset:  11%|█         | 195497/1807941 [02:48<22:59, 1168.87 examples/s]
Tokenizing train dataset:  11%|█         | 195621/1807941 [02:48<22:40, 1185.34 examples/s]
Tokenizing train dataset:  11%|█         | 195753/1807941 [02:48<21:59, 1222.15 examples/s]
Tokenizing train dataset:  11%|█         | 195885/1807941 [02:49<21:33, 1246.75 examples/s]
Tokenizing train dataset:  11%|█         | 196065/1807941 [02:49<22:12, 1209.88 examples/s]
Tokenizing train dataset:  11%|█         | 196195/1807941 [02:49<21:48, 1231.94 examples/s]
Tokenizing train dataset:  11%|█         | 196327/1807941 [02:49<21:26, 1252.75 examples/s]
Tokenizing train dataset:  11%|█         | 196469/1807941 [02:49<26:36, 1009.35 examples/s]
Tokenizing train dataset:  11%|█         | 196595/1807941 [02:49<25:13, 1064.32 examples/s]
Tokenizing train dataset:  11%|█         | 196715/1807941 [02:49<24:31, 1095.19 examples/s]
Tokenizing train dataset:  11%|█         | 196866/1807941 [02:50<25:19, 1060.42 examples/s]
Tokenizing train dataset:  11%|█         | 196983/1807941 [02:50<24:44, 1085.34 examples/s]
Tokenizing train dataset:  11%|█         | 197155/1807941 [02:50<24:19, 1103.72 examples/s]
Tokenizing train dataset:  11%|█         | 197331/1807941 [02:50<23:53, 1123.81 examples/s]
Tokenizing train dataset:  11%|█         | 197465/1807941 [02:50<22:52, 1173.35 examples/s]
Tokenizing train dataset:  11%|█         | 197586/1807941 [02:50<22:44, 1179.81 examples/s]
Tokenizing train dataset:  11%|█         | 197766/1807941 [02:50<22:38, 1185.66 examples/s]
Tokenizing train dataset:  11%|█         | 197934/1807941 [02:50<23:08, 1159.76 examples/s]
Tokenizing train dataset:  11%|█         | 198067/1807941 [02:51<25:04, 1070.16 examples/s]
Tokenizing train dataset:  11%|█         | 198224/1807941 [02:51<25:19, 1059.04 examples/s]
Tokenizing train dataset:  11%|█         | 198335/1807941 [02:51<25:07, 1068.01 examples/s]
Tokenizing train dataset:  11%|█         | 198449/1807941 [02:51<24:44, 1084.21 examples/s]
Tokenizing train dataset:  11%|█         | 198573/1807941 [02:51<23:51, 1124.26 examples/s]
Tokenizing train dataset:  11%|█         | 198700/1807941 [02:51<23:04, 1161.93 examples/s]
Tokenizing train dataset:  11%|█         | 198824/1807941 [02:51<22:43, 1179.92 examples/s]
Tokenizing train dataset:  11%|█         | 198991/1807941 [02:51<23:14, 1153.46 examples/s]
Tokenizing train dataset:  11%|█         | 199169/1807941 [02:52<23:03, 1162.51 examples/s]
Tokenizing train dataset:  11%|█         | 199290/1807941 [02:52<22:52, 1171.76 examples/s]
Tokenizing train dataset:  11%|█         | 199466/1807941 [02:52<22:55, 1169.78 examples/s]
Tokenizing train dataset:  11%|█         | 199638/1807941 [02:52<23:07, 1158.92 examples/s]
Tokenizing train dataset:  11%|█         | 199766/1807941 [02:52<22:36, 1185.72 examples/s]
Tokenizing train dataset:  11%|█         | 199894/1807941 [02:52<22:10, 1209.01 examples/s]
Tokenizing train dataset:  11%|█         | 200056/1807941 [02:52<23:16, 1151.16 examples/s]
Tokenizing train dataset:  11%|█         | 200234/1807941 [02:52<23:05, 1160.23 examples/s]
Tokenizing train dataset:  11%|█         | 200404/1807941 [02:53<23:19, 1148.42 examples/s]
Tokenizing train dataset:  11%|█         | 200524/1807941 [02:53<23:09, 1157.21 examples/s]
Tokenizing train dataset:  11%|█         | 200644/1807941 [02:53<22:58, 1165.57 examples/s]
Tokenizing train dataset:  11%|█         | 200762/1807941 [02:53<22:56, 1167.52 examples/s]
Tokenizing train dataset:  11%|█         | 200895/1807941 [02:53<22:11, 1207.38 examples/s]
Tokenizing train dataset:  11%|█         | 201058/1807941 [02:53<23:13, 1153.15 examples/s]
Tokenizing train dataset:  11%|█         | 201237/1807941 [02:53<23:04, 1160.42 examples/s]
Tokenizing train dataset:  11%|█         | 201398/1807941 [02:53<23:49, 1123.48 examples/s]
Tokenizing train dataset:  11%|█         | 201567/1807941 [02:54<23:54, 1119.92 examples/s]
Tokenizing train dataset:  11%|█         | 201730/1807941 [02:54<24:13, 1105.05 examples/s]
Tokenizing train dataset:  11%|█         | 201893/1807941 [02:54<24:24, 1096.38 examples/s]
Tokenizing train dataset:  11%|█         | 202049/1807941 [02:54<25:53, 1033.61 examples/s]
Tokenizing train dataset:  11%|█         | 202171/1807941 [02:54<24:58, 1071.75 examples/s]
Tokenizing train dataset:  11%|█         | 202285/1807941 [02:54<24:40, 1084.84 examples/s]
Tokenizing train dataset:  11%|█         | 202402/1807941 [02:54<24:12, 1105.25 examples/s]
Tokenizing train dataset:  11%|█         | 202518/1807941 [02:54<23:58, 1116.42 examples/s]
Tokenizing train dataset:  11%|█         | 202642/1807941 [02:55<23:17, 1148.57 examples/s]
Tokenizing train dataset:  11%|█         | 202793/1807941 [02:55<27:54, 958.54 examples/s] 
Tokenizing train dataset:  11%|█         | 202931/1807941 [02:55<25:17, 1057.51 examples/s]
Tokenizing train dataset:  11%|█         | 203089/1807941 [02:55<25:22, 1053.79 examples/s]
Tokenizing train dataset:  11%|█         | 203263/1807941 [02:55<24:38, 1085.17 examples/s]
Tokenizing train dataset:  11%|█▏        | 203425/1807941 [02:55<24:42, 1082.36 examples/s]
Tokenizing train dataset:  11%|█▏        | 203551/1807941 [02:55<23:49, 1122.19 examples/s]
Tokenizing train dataset:  11%|█▏        | 203724/1807941 [02:56<23:42, 1128.11 examples/s]
Tokenizing train dataset:  11%|█▏        | 203842/1807941 [02:56<23:29, 1138.26 examples/s]
Tokenizing train dataset:  11%|█▏        | 203965/1807941 [02:56<23:03, 1159.61 examples/s]
Tokenizing train dataset:  11%|█▏        | 204118/1807941 [02:56<24:31, 1089.99 examples/s]
Tokenizing train dataset:  11%|█▏        | 204241/1807941 [02:56<23:50, 1120.72 examples/s]
Tokenizing train dataset:  11%|█▏        | 204366/1807941 [02:56<23:13, 1151.08 examples/s]
Tokenizing train dataset:  11%|█▏        | 204494/1807941 [02:56<22:33, 1184.44 examples/s]
Tokenizing train dataset:  11%|█▏        | 204642/1807941 [02:56<24:12, 1104.08 examples/s]
Tokenizing train dataset:  11%|█▏        | 204758/1807941 [02:57<23:54, 1117.24 examples/s]
Tokenizing train dataset:  11%|█▏        | 204885/1807941 [02:57<23:07, 1155.30 examples/s]
Tokenizing train dataset:  11%|█▏        | 205056/1807941 [02:57<24:15, 1101.30 examples/s]
Tokenizing train dataset:  11%|█▏        | 205169/1807941 [02:57<24:10, 1104.71 examples/s]
Tokenizing train dataset:  11%|█▏        | 205288/1807941 [02:57<23:44, 1124.78 examples/s]
Tokenizing train dataset:  11%|█▏        | 205472/1807941 [02:57<23:03, 1158.52 examples/s]
Tokenizing train dataset:  11%|█▏        | 205598/1807941 [02:57<22:34, 1182.91 examples/s]
Tokenizing train dataset:  11%|█▏        | 205779/1807941 [02:57<22:27, 1188.56 examples/s]
Tokenizing train dataset:  11%|█▏        | 205947/1807941 [02:58<22:56, 1164.07 examples/s]
Tokenizing train dataset:  11%|█▏        | 206083/1807941 [02:58<24:46, 1077.63 examples/s]
Tokenizing train dataset:  11%|█▏        | 206200/1807941 [02:58<24:20, 1096.34 examples/s]
Tokenizing train dataset:  11%|█▏        | 206313/1807941 [02:58<24:13, 1101.86 examples/s]
Tokenizing train dataset:  11%|█▏        | 206440/1807941 [02:58<23:18, 1144.90 examples/s]
Tokenizing train dataset:  11%|█▏        | 206561/1807941 [02:58<23:00, 1159.91 examples/s]
Tokenizing train dataset:  11%|█▏        | 206697/1807941 [02:58<21:59, 1213.72 examples/s]
Tokenizing train dataset:  11%|█▏        | 206841/1807941 [02:58<20:53, 1276.99 examples/s]
Tokenizing train dataset:  11%|█▏        | 207004/1807941 [02:58<22:14, 1199.36 examples/s]
Tokenizing train dataset:  11%|█▏        | 207129/1807941 [02:59<22:01, 1210.91 examples/s]
Tokenizing train dataset:  11%|█▏        | 207255/1807941 [02:59<21:52, 1219.72 examples/s]
Tokenizing train dataset:  11%|█▏        | 207380/1807941 [02:59<21:46, 1225.29 examples/s]
Tokenizing train dataset:  11%|█▏        | 207511/1807941 [02:59<21:25, 1245.18 examples/s]
Tokenizing train dataset:  11%|█▏        | 207638/1807941 [02:59<21:18, 1251.91 examples/s]
Tokenizing train dataset:  11%|█▏        | 207809/1807941 [02:59<22:07, 1204.98 examples/s]
Tokenizing train dataset:  12%|█▏        | 207990/1807941 [02:59<22:16, 1197.43 examples/s]
Tokenizing train dataset:  12%|█▏        | 208159/1807941 [02:59<25:18, 1053.25 examples/s]
Tokenizing train dataset:  12%|█▏        | 208287/1807941 [03:00<24:13, 1100.85 examples/s]
Tokenizing train dataset:  12%|█▏        | 208402/1807941 [03:00<24:06, 1105.86 examples/s]
Tokenizing train dataset:  12%|█▏        | 208555/1807941 [03:00<27:36, 965.48 examples/s] 
Tokenizing train dataset:  12%|█▏        | 208670/1807941 [03:00<26:30, 1005.47 examples/s]
Tokenizing train dataset:  12%|█▏        | 208792/1807941 [03:00<25:12, 1057.18 examples/s]
Tokenizing train dataset:  12%|█▏        | 208909/1807941 [03:00<24:36, 1083.03 examples/s]
Tokenizing train dataset:  12%|█▏        | 209065/1807941 [03:00<25:00, 1065.58 examples/s]
Tokenizing train dataset:  12%|█▏        | 209186/1807941 [03:00<24:11, 1101.38 examples/s]
Tokenizing train dataset:  12%|█▏        | 209321/1807941 [03:01<22:53, 1164.32 examples/s]
Tokenizing train dataset:  12%|█▏        | 209453/1807941 [03:01<25:12, 1057.16 examples/s]
Tokenizing train dataset:  12%|█▏        | 209594/1807941 [03:01<26:24, 1008.69 examples/s]
Tokenizing train dataset:  12%|█▏        | 209708/1807941 [03:01<25:37, 1039.51 examples/s]
Tokenizing train dataset:  12%|█▏        | 209842/1807941 [03:01<23:55, 1113.48 examples/s]
Tokenizing train dataset:  12%|█▏        | 209976/1807941 [03:01<22:46, 1169.58 examples/s]
Tokenizing train dataset:  12%|█▏        | 210140/1807941 [03:01<23:21, 1139.88 examples/s]
Tokenizing train dataset:  12%|█▏        | 210271/1807941 [03:01<22:31, 1182.37 examples/s]
Tokenizing train dataset:  12%|█▏        | 210406/1807941 [03:01<21:44, 1224.31 examples/s]
Tokenizing train dataset:  12%|█▏        | 210536/1807941 [03:02<21:23, 1244.40 examples/s]
Tokenizing train dataset:  12%|█▏        | 210671/1807941 [03:02<20:55, 1272.49 examples/s]
Tokenizing train dataset:  12%|█▏        | 210805/1807941 [03:02<20:37, 1290.86 examples/s]
Tokenizing train dataset:  12%|█▏        | 210937/1807941 [03:02<20:34, 1293.90 examples/s]
Tokenizing train dataset:  12%|█▏        | 211133/1807941 [03:02<21:42, 1226.30 examples/s]
Tokenizing train dataset:  12%|█▏        | 211276/1807941 [03:02<20:51, 1275.72 examples/s]
Tokenizing train dataset:  12%|█▏        | 211414/1807941 [03:02<20:29, 1298.65 examples/s]
Tokenizing train dataset:  12%|█▏        | 211562/1807941 [03:02<19:46, 1345.60 examples/s]
Tokenizing train dataset:  12%|█▏        | 211702/1807941 [03:02<19:34, 1358.55 examples/s]
Tokenizing train dataset:  12%|█▏        | 211843/1807941 [03:03<19:24, 1370.39 examples/s]
Tokenizing train dataset:  12%|█▏        | 212020/1807941 [03:03<20:32, 1294.58 examples/s]
Tokenizing train dataset:  12%|█▏        | 212154/1807941 [03:03<20:21, 1306.32 examples/s]
Tokenizing train dataset:  12%|█▏        | 212295/1807941 [03:03<19:59, 1330.61 examples/s]
Tokenizing train dataset:  12%|█▏        | 212500/1807941 [03:03<19:50, 1339.85 examples/s]
Tokenizing train dataset:  12%|█▏        | 212693/1807941 [03:03<20:10, 1317.95 examples/s]
Tokenizing train dataset:  12%|█▏        | 212886/1807941 [03:03<20:22, 1304.76 examples/s]
Tokenizing train dataset:  12%|█▏        | 213065/1807941 [03:04<21:20, 1245.45 examples/s]
Tokenizing train dataset:  12%|█▏        | 213196/1807941 [03:04<21:08, 1256.94 examples/s]
Tokenizing train dataset:  12%|█▏        | 213328/1807941 [03:04<20:57, 1268.22 examples/s]
Tokenizing train dataset:  12%|█▏        | 213464/1807941 [03:04<20:37, 1288.94 examples/s]
Tokenizing train dataset:  12%|█▏        | 213600/1807941 [03:04<20:21, 1304.85 examples/s]
Tokenizing train dataset:  12%|█▏        | 213791/1807941 [03:04<20:37, 1288.46 examples/s]
Tokenizing train dataset:  12%|█▏        | 213980/1807941 [03:04<20:50, 1274.88 examples/s]
Tokenizing train dataset:  12%|█▏        | 214129/1807941 [03:04<22:30, 1180.15 examples/s]
Tokenizing train dataset:  12%|█▏        | 214308/1807941 [03:05<22:28, 1182.10 examples/s]
Tokenizing train dataset:  12%|█▏        | 214431/1807941 [03:05<22:20, 1188.77 examples/s]
Tokenizing train dataset:  12%|█▏        | 214554/1807941 [03:05<22:10, 1197.29 examples/s]
Tokenizing train dataset:  12%|█▏        | 214680/1807941 [03:05<21:53, 1212.64 examples/s]
Tokenizing train dataset:  12%|█▏        | 214865/1807941 [03:05<21:48, 1217.28 examples/s]
Tokenizing train dataset:  12%|█▏        | 214988/1807941 [03:05<21:50, 1215.89 examples/s]
Tokenizing train dataset:  12%|█▏        | 215154/1807941 [03:05<22:37, 1173.56 examples/s]
Tokenizing train dataset:  12%|█▏        | 215280/1807941 [03:05<22:16, 1191.99 examples/s]
Tokenizing train dataset:  12%|█▏        | 215406/1807941 [03:05<21:57, 1208.52 examples/s]
Tokenizing train dataset:  12%|█▏        | 215529/1807941 [03:06<21:52, 1212.88 examples/s]
Tokenizing train dataset:  12%|█▏        | 215652/1807941 [03:06<21:53, 1212.47 examples/s]
Tokenizing train dataset:  12%|█▏        | 215780/1807941 [03:06<21:35, 1228.62 examples/s]
Tokenizing train dataset:  12%|█▏        | 215910/1807941 [03:06<21:15, 1247.84 examples/s]
Tokenizing train dataset:  12%|█▏        | 216074/1807941 [03:06<22:20, 1187.33 examples/s]
Tokenizing train dataset:  12%|█▏        | 216212/1807941 [03:06<21:28, 1235.14 examples/s]
Tokenizing train dataset:  12%|█▏        | 216338/1807941 [03:06<21:22, 1241.12 examples/s]
Tokenizing train dataset:  12%|█▏        | 216468/1807941 [03:06<21:07, 1255.38 examples/s]
Tokenizing train dataset:  12%|█▏        | 216659/1807941 [03:06<21:04, 1258.46 examples/s]
Tokenizing train dataset:  12%|█▏        | 216791/1807941 [03:07<20:51, 1271.47 examples/s]
Tokenizing train dataset:  12%|█▏        | 216921/1807941 [03:07<20:46, 1276.42 examples/s]
Tokenizing train dataset:  12%|█▏        | 217092/1807941 [03:07<21:42, 1221.42 examples/s]
Tokenizing train dataset:  12%|█▏        | 217221/1807941 [03:07<21:25, 1237.05 examples/s]
Tokenizing train dataset:  12%|█▏        | 217352/1807941 [03:07<21:09, 1253.20 examples/s]
Tokenizing train dataset:  12%|█▏        | 217483/1807941 [03:07<20:57, 1265.02 examples/s]
Tokenizing train dataset:  12%|█▏        | 217612/1807941 [03:07<20:53, 1268.86 examples/s]
Tokenizing train dataset:  12%|█▏        | 217800/1807941 [03:07<21:00, 1261.33 examples/s]
Tokenizing train dataset:  12%|█▏        | 217927/1807941 [03:07<21:03, 1258.88 examples/s]
Tokenizing train dataset:  12%|█▏        | 218092/1807941 [03:08<22:06, 1198.68 examples/s]
Tokenizing train dataset:  12%|█▏        | 218277/1807941 [03:08<21:57, 1206.57 examples/s]
Tokenizing train dataset:  12%|█▏        | 218404/1807941 [03:08<21:43, 1219.72 examples/s]
Tokenizing train dataset:  12%|█▏        | 218535/1807941 [03:08<21:19, 1242.35 examples/s]
Tokenizing train dataset:  12%|█▏        | 218664/1807941 [03:08<21:09, 1251.92 examples/s]
Tokenizing train dataset:  12%|█▏        | 218848/1807941 [03:08<21:26, 1235.38 examples/s]
Tokenizing train dataset:  12%|█▏        | 218981/1807941 [03:08<21:06, 1254.91 examples/s]
Tokenizing train dataset:  12%|█▏        | 219150/1807941 [03:08<21:56, 1206.49 examples/s]
Tokenizing train dataset:  12%|█▏        | 219278/1807941 [03:09<21:38, 1223.44 examples/s]
Tokenizing train dataset:  12%|█▏        | 219463/1807941 [03:09<21:38, 1223.53 examples/s]
Tokenizing train dataset:  12%|█▏        | 219593/1807941 [03:09<21:21, 1239.79 examples/s]
Tokenizing train dataset:  12%|█▏        | 219723/1807941 [03:09<21:05, 1255.20 examples/s]
Tokenizing train dataset:  12%|█▏        | 219852/1807941 [03:09<20:58, 1262.06 examples/s]
Tokenizing train dataset:  12%|█▏        | 219987/1807941 [03:09<20:38, 1282.17 examples/s]
Tokenizing train dataset:  12%|█▏        | 220157/1807941 [03:09<21:37, 1223.64 examples/s]
Tokenizing train dataset:  12%|█▏        | 220292/1807941 [03:09<21:07, 1252.56 examples/s]
Tokenizing train dataset:  12%|█▏        | 220427/1807941 [03:09<20:42, 1277.42 examples/s]
Tokenizing train dataset:  12%|█▏        | 220613/1807941 [03:10<21:01, 1258.74 examples/s]
Tokenizing train dataset:  12%|█▏        | 220803/1807941 [03:10<21:01, 1258.22 examples/s]
Tokenizing train dataset:  12%|█▏        | 220993/1807941 [03:10<21:01, 1258.25 examples/s]
Tokenizing train dataset:  12%|█▏        | 221150/1807941 [03:10<22:14, 1189.22 examples/s]
Tokenizing train dataset:  12%|█▏        | 221278/1807941 [03:10<21:53, 1208.13 examples/s]
Tokenizing train dataset:  12%|█▏        | 221408/1807941 [03:10<21:30, 1229.02 examples/s]
Tokenizing train dataset:  12%|█▏        | 221535/1807941 [03:10<21:21, 1237.92 examples/s]
Tokenizing train dataset:  12%|█▏        | 221717/1807941 [03:11<21:37, 1222.95 examples/s]
Tokenizing train dataset:  12%|█▏        | 221847/1807941 [03:11<21:19, 1239.80 examples/s]
Tokenizing train dataset:  12%|█▏        | 221976/1807941 [03:11<21:08, 1250.65 examples/s]
Tokenizing train dataset:  12%|█▏        | 222137/1807941 [03:11<22:21, 1182.28 examples/s]
Tokenizing train dataset:  12%|█▏        | 222268/1807941 [03:11<21:48, 1212.18 examples/s]
Tokenizing train dataset:  12%|█▏        | 222393/1807941 [03:11<21:39, 1219.78 examples/s]
Tokenizing train dataset:  12%|█▏        | 222524/1807941 [03:11<21:19, 1238.61 examples/s]
Tokenizing train dataset:  12%|█▏        | 222651/1807941 [03:11<21:14, 1243.60 examples/s]
Tokenizing train dataset:  12%|█▏        | 222782/1807941 [03:11<20:58, 1259.64 examples/s]
Tokenizing train dataset:  12%|█▏        | 222910/1807941 [03:11<20:54, 1263.26 examples/s]
Tokenizing train dataset:  12%|█▏        | 223075/1807941 [03:12<22:01, 1199.46 examples/s]
Tokenizing train dataset:  12%|█▏        | 223200/1807941 [03:12<21:49, 1210.18 examples/s]
Tokenizing train dataset:  12%|█▏        | 223329/1807941 [03:12<21:29, 1229.29 examples/s]
Tokenizing train dataset:  12%|█▏        | 223457/1807941 [03:12<21:16, 1240.90 examples/s]
Tokenizing train dataset:  12%|█▏        | 223585/1807941 [03:12<21:10, 1246.57 examples/s]
Tokenizing train dataset:  12%|█▏        | 223712/1807941 [03:12<21:08, 1249.09 examples/s]
Tokenizing train dataset:  12%|█▏        | 223899/1807941 [03:12<21:13, 1243.93 examples/s]
Tokenizing train dataset:  12%|█▏        | 224063/1807941 [03:12<22:18, 1183.67 examples/s]
Tokenizing train dataset:  12%|█▏        | 224187/1807941 [03:13<22:07, 1193.11 examples/s]
Tokenizing train dataset:  12%|█▏        | 224316/1807941 [03:13<21:45, 1213.36 examples/s]
Tokenizing train dataset:  12%|█▏        | 224443/1807941 [03:13<21:31, 1226.41 examples/s]
Tokenizing train dataset:  12%|█▏        | 224577/1807941 [03:13<21:02, 1253.79 examples/s]
Tokenizing train dataset:  12%|█▏        | 224711/1807941 [03:13<20:41, 1275.65 examples/s]
Tokenizing train dataset:  12%|█▏        | 224845/1807941 [03:13<20:23, 1293.68 examples/s]
Tokenizing train dataset:  12%|█▏        | 225008/1807941 [03:13<21:48, 1209.38 examples/s]
Tokenizing train dataset:  12%|█▏        | 225141/1807941 [03:13<21:18, 1238.40 examples/s]
Tokenizing train dataset:  12%|█▏        | 225280/1807941 [03:13<20:37, 1278.41 examples/s]
Tokenizing train dataset:  12%|█▏        | 225412/1807941 [03:14<20:28, 1287.69 examples/s]
Tokenizing train dataset:  12%|█▏        | 225549/1807941 [03:14<20:08, 1309.10 examples/s]
Tokenizing train dataset:  12%|█▏        | 225681/1807941 [03:14<20:07, 1310.27 examples/s]
Tokenizing train dataset:  12%|█▏        | 225813/1807941 [03:14<20:06, 1311.35 examples/s]
Tokenizing train dataset:  12%|█▏        | 225946/1807941 [03:14<20:01, 1316.45 examples/s]
Tokenizing train dataset:  13%|█▎        | 226124/1807941 [03:14<21:42, 1214.44 examples/s]
Tokenizing train dataset:  13%|█▎        | 226254/1807941 [03:14<21:19, 1236.14 examples/s]
Tokenizing train dataset:  13%|█▎        | 226385/1807941 [03:14<21:00, 1254.87 examples/s]
Tokenizing train dataset:  13%|█▎        | 226514/1807941 [03:14<20:55, 1259.74 examples/s]
Tokenizing train dataset:  13%|█▎        | 226645/1807941 [03:14<20:44, 1270.30 examples/s]
Tokenizing train dataset:  13%|█▎        | 226776/1807941 [03:15<20:37, 1277.53 examples/s]
Tokenizing train dataset:  13%|█▎        | 226908/1807941 [03:15<20:30, 1284.86 examples/s]
Tokenizing train dataset:  13%|█▎        | 227076/1807941 [03:15<21:40, 1215.84 examples/s]
Tokenizing train dataset:  13%|█▎        | 227204/1807941 [03:15<21:25, 1230.08 examples/s]
Tokenizing train dataset:  13%|█▎        | 227331/1807941 [03:15<21:14, 1240.36 examples/s]
Tokenizing train dataset:  13%|█▎        | 227461/1807941 [03:15<21:00, 1254.26 examples/s]
Tokenizing train dataset:  13%|█▎        | 227594/1807941 [03:15<20:42, 1271.87 examples/s]
Tokenizing train dataset:  13%|█▎        | 227728/1807941 [03:15<20:24, 1290.14 examples/s]
Tokenizing train dataset:  13%|█▎        | 227873/1807941 [03:15<19:46, 1332.01 examples/s]
Tokenizing train dataset:  13%|█▎        | 228062/1807941 [03:16<21:01, 1252.50 examples/s]
Tokenizing train dataset:  13%|█▎        | 228191/1807941 [03:16<20:52, 1261.53 examples/s]
Tokenizing train dataset:  13%|█▎        | 228323/1807941 [03:16<20:38, 1275.00 examples/s]
Tokenizing train dataset:  13%|█▎        | 228458/1807941 [03:16<20:22, 1292.50 examples/s]
Tokenizing train dataset:  13%|█▎        | 228598/1807941 [03:16<19:55, 1321.11 examples/s]
Tokenizing train dataset:  13%|█▎        | 228734/1807941 [03:16<19:47, 1330.34 examples/s]
Tokenizing train dataset:  13%|█▎        | 228876/1807941 [03:16<19:26, 1354.04 examples/s]
Tokenizing train dataset:  13%|█▎        | 229070/1807941 [03:16<20:03, 1311.77 examples/s]
Tokenizing train dataset:  13%|█▎        | 229209/1807941 [03:16<19:48, 1328.39 examples/s]
Tokenizing train dataset:  13%|█▎        | 229350/1807941 [03:17<19:32, 1346.22 examples/s]
Tokenizing train dataset:  13%|█▎        | 229497/1807941 [03:17<19:06, 1377.12 examples/s]
Tokenizing train dataset:  13%|█▎        | 229646/1807941 [03:17<18:42, 1406.08 examples/s]
Tokenizing train dataset:  13%|█▎        | 229792/1807941 [03:17<18:33, 1417.66 examples/s]
Tokenizing train dataset:  13%|█▎        | 229940/1807941 [03:17<18:21, 1432.47 examples/s]
Tokenizing train dataset:  13%|█▎        | 230145/1807941 [03:17<19:14, 1366.70 examples/s]
Tokenizing train dataset:  13%|█▎        | 230348/1807941 [03:17<19:19, 1360.41 examples/s]
Tokenizing train dataset:  13%|█▎        | 230489/1807941 [03:17<19:12, 1368.17 examples/s]
Tokenizing train dataset:  13%|█▎        | 230628/1807941 [03:17<19:10, 1370.49 examples/s]
Tokenizing train dataset:  13%|█▎        | 230834/1807941 [03:18<19:12, 1368.36 examples/s]
Tokenizing train dataset:  13%|█▎        | 231011/1807941 [03:18<20:11, 1301.73 examples/s]
Tokenizing train dataset:  13%|█▎        | 231145/1807941 [03:18<20:05, 1308.27 examples/s]
Tokenizing train dataset:  13%|█▎        | 231281/1807941 [03:18<19:54, 1319.41 examples/s]
Tokenizing train dataset:  13%|█▎        | 231482/1807941 [03:18<19:51, 1323.48 examples/s]
Tokenizing train dataset:  13%|█▎        | 231677/1807941 [03:18<20:01, 1311.74 examples/s]
Tokenizing train dataset:  13%|█▎        | 231872/1807941 [03:18<20:06, 1306.13 examples/s]
Tokenizing train dataset:  13%|█▎        | 232061/1807941 [03:19<21:18, 1232.77 examples/s]
Tokenizing train dataset:  13%|█▎        | 232190/1807941 [03:19<21:07, 1243.33 examples/s]
Tokenizing train dataset:  13%|█▎        | 232319/1807941 [03:19<20:57, 1253.02 examples/s]
Tokenizing train dataset:  13%|█▎        | 232449/1807941 [03:19<20:48, 1261.53 examples/s]
Tokenizing train dataset:  13%|█▎        | 232643/1807941 [03:19<20:39, 1271.13 examples/s]
Tokenizing train dataset:  13%|█▎        | 232830/1807941 [03:19<20:51, 1259.01 examples/s]
Tokenizing train dataset:  13%|█▎        | 232962/1807941 [03:19<20:39, 1270.71 examples/s]
Tokenizing train dataset:  13%|█▎        | 233127/1807941 [03:19<21:56, 1196.09 examples/s]
Tokenizing train dataset:  13%|█▎        | 233252/1807941 [03:20<21:44, 1207.49 examples/s]
Tokenizing train dataset:  13%|█▎        | 233383/1807941 [03:20<21:16, 1233.38 examples/s]
Tokenizing train dataset:  13%|█▎        | 233510/1807941 [03:20<21:08, 1241.62 examples/s]
Tokenizing train dataset:  13%|█▎        | 233649/1807941 [03:20<20:28, 1281.24 examples/s]
Tokenizing train dataset:  13%|█▎        | 233781/1807941 [03:20<20:23, 1287.10 examples/s]
Tokenizing train dataset:  13%|█▎        | 233913/1807941 [03:20<20:16, 1294.28 examples/s]
Tokenizing train dataset:  13%|█▎        | 234083/1807941 [03:20<21:19, 1230.44 examples/s]
Tokenizing train dataset:  13%|█▎        | 234224/1807941 [03:20<20:33, 1275.69 examples/s]
Tokenizing train dataset:  13%|█▎        | 234355/1807941 [03:20<20:27, 1281.85 examples/s]
Tokenizing train dataset:  13%|█▎        | 234486/1807941 [03:21<20:21, 1288.33 examples/s]
Tokenizing train dataset:  13%|█▎        | 234625/1807941 [03:21<19:57, 1314.08 examples/s]
Tokenizing train dataset:  13%|█▎        | 234821/1807941 [03:21<20:02, 1308.43 examples/s]
Tokenizing train dataset:  13%|█▎        | 235000/1807941 [03:21<21:14, 1234.51 examples/s]
Tokenizing train dataset:  13%|█▎        | 235133/1807941 [03:21<20:53, 1255.21 examples/s]
Tokenizing train dataset:  13%|█▎        | 235324/1807941 [03:21<20:49, 1258.27 examples/s]
Tokenizing train dataset:  13%|█▎        | 235515/1807941 [03:21<20:47, 1260.33 examples/s]
Tokenizing train dataset:  13%|█▎        | 235699/1807941 [03:21<21:01, 1246.18 examples/s]
Tokenizing train dataset:  13%|█▎        | 235888/1807941 [03:22<21:00, 1247.60 examples/s]
Tokenizing train dataset:  13%|█▎        | 236065/1807941 [03:22<21:48, 1201.40 examples/s]
Tokenizing train dataset:  13%|█▎        | 236191/1807941 [03:22<21:38, 1210.36 examples/s]
Tokenizing train dataset:  13%|█▎        | 236325/1807941 [03:22<21:07, 1239.53 examples/s]
Tokenizing train dataset:  13%|█▎        | 236452/1807941 [03:22<21:03, 1243.77 examples/s]
Tokenizing train dataset:  13%|█▎        | 236580/1807941 [03:22<20:56, 1250.84 examples/s]
Tokenizing train dataset:  13%|█▎        | 236706/1807941 [03:22<20:57, 1249.73 examples/s]
Tokenizing train dataset:  13%|█▎        | 236889/1807941 [03:22<21:10, 1236.38 examples/s]
Tokenizing train dataset:  13%|█▎        | 237062/1807941 [03:23<21:59, 1190.34 examples/s]
Tokenizing train dataset:  13%|█▎        | 237243/1807941 [03:23<21:56, 1193.47 examples/s]
Tokenizing train dataset:  13%|█▎        | 237369/1807941 [03:23<21:41, 1207.10 examples/s]
Tokenizing train dataset:  13%|█▎        | 237500/1807941 [03:23<21:16, 1229.82 examples/s]
Tokenizing train dataset:  13%|█▎        | 237685/1807941 [03:23<21:18, 1228.17 examples/s]
Tokenizing train dataset:  13%|█▎        | 237811/1807941 [03:23<21:14, 1232.17 examples/s]
Tokenizing train dataset:  13%|█▎        | 237998/1807941 [03:23<21:14, 1231.62 examples/s]
Tokenizing train dataset:  13%|█▎        | 238166/1807941 [03:24<21:58, 1190.69 examples/s]
Tokenizing train dataset:  13%|█▎        | 238300/1807941 [03:24<21:21, 1224.54 examples/s]
Tokenizing train dataset:  13%|█▎        | 238435/1807941 [03:24<20:51, 1254.14 examples/s]
Tokenizing train dataset:  13%|█▎        | 238622/1807941 [03:24<20:57, 1248.39 examples/s]
Tokenizing train dataset:  13%|█▎        | 238751/1807941 [03:24<20:51, 1254.19 examples/s]
Tokenizing train dataset:  13%|█▎        | 238881/1807941 [03:24<20:44, 1261.14 examples/s]
Tokenizing train dataset:  13%|█▎        | 239055/1807941 [03:24<22:31, 1160.51 examples/s]
Tokenizing train dataset:  13%|█▎        | 239178/1807941 [03:24<22:15, 1174.72 examples/s]
Tokenizing train dataset:  13%|█▎        | 239309/1807941 [03:24<21:38, 1207.88 examples/s]
Tokenizing train dataset:  13%|█▎        | 239482/1807941 [03:25<22:03, 1184.85 examples/s]
Tokenizing train dataset:  13%|█▎        | 239607/1807941 [03:25<21:47, 1199.77 examples/s]
Tokenizing train dataset:  13%|█▎        | 239796/1807941 [03:25<21:29, 1215.98 examples/s]
Tokenizing train dataset:  13%|█▎        | 239923/1807941 [03:25<21:16, 1228.45 examples/s]
Tokenizing train dataset:  13%|█▎        | 240071/1807941 [03:25<22:53, 1141.86 examples/s]
Tokenizing train dataset:  13%|█▎        | 240205/1807941 [03:25<21:57, 1190.38 examples/s]
Tokenizing train dataset:  13%|█▎        | 240339/1807941 [03:25<21:18, 1225.78 examples/s]
Tokenizing train dataset:  13%|█▎        | 240522/1807941 [03:25<21:24, 1220.35 examples/s]
Tokenizing train dataset:  13%|█▎        | 240650/1807941 [03:26<21:12, 1231.59 examples/s]
Tokenizing train dataset:  13%|█▎        | 240776/1807941 [03:26<21:07, 1235.97 examples/s]
Tokenizing train dataset:  13%|█▎        | 240912/1807941 [03:26<20:36, 1267.81 examples/s]
Tokenizing train dataset:  13%|█▎        | 241077/1807941 [03:26<21:42, 1203.17 examples/s]
Tokenizing train dataset:  13%|█▎        | 241201/1807941 [03:26<21:34, 1210.20 examples/s]
Tokenizing train dataset:  13%|█▎        | 241331/1807941 [03:26<21:09, 1233.71 examples/s]
Tokenizing train dataset:  13%|█▎        | 241513/1807941 [03:26<21:18, 1225.28 examples/s]
Tokenizing train dataset:  13%|█▎        | 241638/1807941 [03:26<21:13, 1229.96 examples/s]
Tokenizing train dataset:  13%|█▎        | 241763/1807941 [03:26<21:09, 1234.08 examples/s]
Tokenizing train dataset:  13%|█▎        | 241940/1807941 [03:27<21:35, 1208.46 examples/s]
Tokenizing train dataset:  13%|█▎        | 242066/1807941 [03:27<22:28, 1161.45 examples/s]
Tokenizing train dataset:  13%|█▎        | 242193/1807941 [03:27<21:57, 1188.16 examples/s]
Tokenizing train dataset:  13%|█▎        | 242324/1807941 [03:27<21:24, 1218.67 examples/s]
Tokenizing train dataset:  13%|█▎        | 242461/1807941 [03:27<20:44, 1257.61 examples/s]
Tokenizing train dataset:  13%|█▎        | 242591/1807941 [03:27<20:35, 1266.87 examples/s]
Tokenizing train dataset:  13%|█▎        | 242772/1807941 [03:27<21:02, 1240.08 examples/s]
Tokenizing train dataset:  13%|█▎        | 242906/1807941 [03:27<20:37, 1264.72 examples/s]
Tokenizing train dataset:  13%|█▎        | 243080/1807941 [03:28<21:18, 1223.99 examples/s]
Tokenizing train dataset:  13%|█▎        | 243211/1807941 [03:28<20:59, 1242.47 examples/s]
Tokenizing train dataset:  13%|█▎        | 243341/1807941 [03:28<20:44, 1257.24 examples/s]
Tokenizing train dataset:  13%|█▎        | 243534/1807941 [03:28<20:36, 1265.66 examples/s]
Tokenizing train dataset:  13%|█▎        | 243671/1807941 [03:28<20:12, 1289.70 examples/s]
Tokenizing train dataset:  13%|█▎        | 243803/1807941 [03:28<20:10, 1292.20 examples/s]
Tokenizing train dataset:  13%|█▎        | 243988/1807941 [03:28<20:33, 1267.44 examples/s]
Tokenizing train dataset:  14%|█▎        | 244148/1807941 [03:28<21:45, 1197.71 examples/s]
Tokenizing train dataset:  14%|█▎        | 244280/1807941 [03:29<21:16, 1224.73 examples/s]
Tokenizing train dataset:  14%|█▎        | 244412/1807941 [03:29<20:53, 1246.90 examples/s]
Tokenizing train dataset:  14%|█▎        | 244540/1807941 [03:29<20:47, 1253.52 examples/s]
Tokenizing train dataset:  14%|█▎        | 244723/1807941 [03:29<21:04, 1236.09 examples/s]
Tokenizing train dataset:  14%|█▎        | 244905/1807941 [03:29<21:19, 1221.37 examples/s]
Tokenizing train dataset:  14%|█▎        | 245059/1807941 [03:29<22:47, 1143.27 examples/s]
Tokenizing train dataset:  14%|█▎        | 245177/1807941 [03:29<22:40, 1148.63 examples/s]
Tokenizing train dataset:  14%|█▎        | 245299/1807941 [03:29<22:22, 1163.84 examples/s]
Tokenizing train dataset:  14%|█▎        | 245420/1807941 [03:29<22:09, 1175.02 examples/s]
Tokenizing train dataset:  14%|█▎        | 245549/1807941 [03:30<21:38, 1202.78 examples/s]
Tokenizing train dataset:  14%|█▎        | 245679/1807941 [03:30<21:16, 1223.72 examples/s]
Tokenizing train dataset:  14%|█▎        | 245811/1807941 [03:30<20:50, 1248.87 examples/s]
Tokenizing train dataset:  14%|█▎        | 245937/1807941 [03:30<20:48, 1251.32 examples/s]
Tokenizing train dataset:  14%|█▎        | 246063/1807941 [03:30<22:05, 1178.27 examples/s]
Tokenizing train dataset:  14%|█▎        | 246190/1807941 [03:30<21:39, 1201.70 examples/s]
Tokenizing train dataset:  14%|█▎        | 246320/1807941 [03:30<21:11, 1227.78 examples/s]
Tokenizing train dataset:  14%|█▎        | 246458/1807941 [03:30<20:29, 1270.17 examples/s]
Tokenizing train dataset:  14%|█▎        | 246598/1807941 [03:30<19:57, 1303.32 examples/s]
Tokenizing train dataset:  14%|█▎        | 246786/1807941 [03:31<20:17, 1281.86 examples/s]
Tokenizing train dataset:  14%|█▎        | 246917/1807941 [03:31<20:13, 1286.68 examples/s]
Tokenizing train dataset:  14%|█▎        | 247080/1807941 [03:31<21:30, 1209.39 examples/s]
Tokenizing train dataset:  14%|█▎        | 247212/1807941 [03:31<21:03, 1235.10 examples/s]
Tokenizing train dataset:  14%|█▎        | 247350/1807941 [03:31<20:27, 1271.55 examples/s]
Tokenizing train dataset:  14%|█▎        | 247484/1807941 [03:31<20:11, 1287.86 examples/s]
Tokenizing train dataset:  14%|█▎        | 247616/1807941 [03:31<20:06, 1292.98 examples/s]
Tokenizing train dataset:  14%|█▎        | 247808/1807941 [03:31<20:16, 1282.68 examples/s]
Tokenizing train dataset:  14%|█▎        | 247982/1807941 [03:32<21:00, 1237.10 examples/s]
Tokenizing train dataset:  14%|█▎        | 248138/1807941 [03:32<22:13, 1169.70 examples/s]
Tokenizing train dataset:  14%|█▎        | 248259/1807941 [03:32<22:04, 1177.84 examples/s]
Tokenizing train dataset:  14%|█▎        | 248382/1807941 [03:32<21:52, 1188.66 examples/s]
Tokenizing train dataset:  14%|█▎        | 248507/1807941 [03:32<21:36, 1202.65 examples/s]
Tokenizing train dataset:  14%|█▍        | 248674/1807941 [03:32<22:17, 1166.13 examples/s]
Tokenizing train dataset:  14%|█▍        | 248792/1807941 [03:32<22:17, 1165.61 examples/s]
Tokenizing train dataset:  14%|█▍        | 248910/1807941 [03:32<22:15, 1166.97 examples/s]
Tokenizing train dataset:  14%|█▍        | 249063/1807941 [03:32<23:22, 1111.65 examples/s]
Tokenizing train dataset:  14%|█▍        | 249180/1807941 [03:33<23:05, 1125.20 examples/s]
Tokenizing train dataset:  14%|█▍        | 249298/1807941 [03:33<22:49, 1137.98 examples/s]
Tokenizing train dataset:  14%|█▍        | 249422/1807941 [03:33<22:19, 1163.93 examples/s]
Tokenizing train dataset:  14%|█▍        | 249542/1807941 [03:33<22:11, 1170.54 examples/s]
Tokenizing train dataset:  14%|█▍        | 249667/1807941 [03:33<21:47, 1192.20 examples/s]
Tokenizing train dataset:  14%|█▍        | 249846/1807941 [03:33<21:50, 1188.48 examples/s]
Tokenizing train dataset:  14%|█▍        | 249968/1807941 [03:33<21:45, 1193.17 examples/s]
Tokenizing train dataset:  14%|█▍        | 250124/1807941 [03:33<22:55, 1132.18 examples/s]
Tokenizing train dataset:  14%|█▍        | 250248/1807941 [03:33<22:25, 1157.92 examples/s]
Tokenizing train dataset:  14%|█▍        | 250369/1807941 [03:34<22:12, 1168.99 examples/s]
Tokenizing train dataset:  14%|█▍        | 250498/1807941 [03:34<21:38, 1199.86 examples/s]
Tokenizing train dataset:  14%|█▍        | 250628/1807941 [03:34<21:13, 1223.03 examples/s]
Tokenizing train dataset:  14%|█▍        | 250811/1807941 [03:34<21:17, 1218.43 examples/s]
Tokenizing train dataset:  14%|█▍        | 250937/1807941 [03:34<21:11, 1224.36 examples/s]
Tokenizing train dataset:  14%|█▍        | 251061/1807941 [03:34<22:23, 1159.16 examples/s]
Tokenizing train dataset:  14%|█▍        | 251185/1807941 [03:34<22:00, 1178.56 examples/s]
Tokenizing train dataset:  14%|█▍        | 251317/1807941 [03:34<21:21, 1214.91 examples/s]
Tokenizing train dataset:  14%|█▍        | 251443/1807941 [03:34<21:10, 1225.39 examples/s]
Tokenizing train dataset:  14%|█▍        | 251627/1807941 [03:35<21:14, 1221.14 examples/s]
Tokenizing train dataset:  14%|█▍        | 251750/1807941 [03:35<21:14, 1220.87 examples/s]
Tokenizing train dataset:  14%|█▍        | 251932/1807941 [03:35<21:20, 1215.62 examples/s]
Tokenizing train dataset:  14%|█▍        | 252081/1807941 [03:35<22:47, 1138.08 examples/s]
Tokenizing train dataset:  14%|█▍        | 252251/1807941 [03:35<22:52, 1133.57 examples/s]
Tokenizing train dataset:  14%|█▍        | 252368/1807941 [03:35<22:46, 1138.05 examples/s]
Tokenizing train dataset:  14%|█▍        | 252544/1807941 [03:35<22:36, 1146.92 examples/s]
Tokenizing train dataset:  14%|█▍        | 252714/1807941 [03:36<22:44, 1139.52 examples/s]
Tokenizing train dataset:  14%|█▍        | 252829/1807941 [03:36<22:44, 1139.68 examples/s]
Tokenizing train dataset:  14%|█▍        | 253000/1807941 [03:36<23:37, 1096.78 examples/s]
Tokenizing train dataset:  14%|█▍        | 253120/1807941 [03:36<23:07, 1120.50 examples/s]
Tokenizing train dataset:  14%|█▍        | 253246/1807941 [03:36<22:27, 1153.41 examples/s]
Tokenizing train dataset:  14%|█▍        | 253366/1807941 [03:36<22:15, 1163.88 examples/s]
Tokenizing train dataset:  14%|█▍        | 253493/1807941 [03:36<21:46, 1189.91 examples/s]
Tokenizing train dataset:  14%|█▍        | 253678/1807941 [03:36<21:33, 1201.82 examples/s]
Tokenizing train dataset:  14%|█▍        | 253804/1807941 [03:36<21:19, 1214.63 examples/s]
Tokenizing train dataset:  14%|█▍        | 253928/1807941 [03:37<21:14, 1219.21 examples/s]
Tokenizing train dataset:  14%|█▍        | 254086/1807941 [03:37<22:27, 1153.53 examples/s]
Tokenizing train dataset:  14%|█▍        | 254261/1807941 [03:37<22:23, 1156.14 examples/s]
Tokenizing train dataset:  14%|█▍        | 254436/1807941 [03:37<22:25, 1154.48 examples/s]
Tokenizing train dataset:  14%|█▍        | 254558/1807941 [03:37<22:11, 1167.07 examples/s]
Tokenizing train dataset:  14%|█▍        | 254679/1807941 [03:37<22:05, 1172.04 examples/s]
Tokenizing train dataset:  14%|█▍        | 254799/1807941 [03:37<21:59, 1177.39 examples/s]
Tokenizing train dataset:  14%|█▍        | 254925/1807941 [03:37<21:36, 1198.07 examples/s]
Tokenizing train dataset:  14%|█▍        | 255081/1807941 [03:38<22:47, 1135.91 examples/s]
Tokenizing train dataset:  14%|█▍        | 255205/1807941 [03:38<22:17, 1160.72 examples/s]
Tokenizing train dataset:  14%|█▍        | 255325/1807941 [03:38<22:09, 1167.90 examples/s]
Tokenizing train dataset:  14%|█▍        | 255500/1807941 [03:38<22:15, 1162.57 examples/s]
Tokenizing train dataset:  14%|█▍        | 255628/1807941 [03:38<21:47, 1187.40 examples/s]
Tokenizing train dataset:  14%|█▍        | 255810/1807941 [03:38<21:40, 1193.47 examples/s]
Tokenizing train dataset:  14%|█▍        | 255933/1807941 [03:38<21:32, 1200.53 examples/s]
Tokenizing train dataset:  14%|█▍        | 256089/1807941 [03:38<22:39, 1141.77 examples/s]
Tokenizing train dataset:  14%|█▍        | 256208/1807941 [03:39<22:27, 1151.66 examples/s]
Tokenizing train dataset:  14%|█▍        | 256327/1807941 [03:39<22:18, 1159.02 examples/s]
Tokenizing train dataset:  14%|█▍        | 256501/1807941 [03:39<22:23, 1155.09 examples/s]
Tokenizing train dataset:  14%|█▍        | 256672/1807941 [03:39<22:30, 1149.04 examples/s]
Tokenizing train dataset:  14%|█▍        | 256846/1807941 [03:39<22:28, 1149.93 examples/s]
Tokenizing train dataset:  14%|█▍        | 257000/1807941 [03:39<23:45, 1088.16 examples/s]
Tokenizing train dataset:  14%|█▍        | 257114/1807941 [03:39<23:32, 1097.80 examples/s]
Tokenizing train dataset:  14%|█▍        | 257226/1807941 [03:39<23:28, 1100.91 examples/s]
Tokenizing train dataset:  14%|█▍        | 257345/1807941 [03:40<23:03, 1120.44 examples/s]
Tokenizing train dataset:  14%|█▍        | 257510/1807941 [03:40<23:18, 1108.37 examples/s]
Tokenizing train dataset:  14%|█▍        | 257627/1807941 [03:40<23:02, 1121.76 examples/s]
Tokenizing train dataset:  14%|█▍        | 257744/1807941 [03:40<22:50, 1130.78 examples/s]
Tokenizing train dataset:  14%|█▍        | 257860/1807941 [03:40<22:44, 1136.05 examples/s]
Tokenizing train dataset:  14%|█▍        | 257980/1807941 [03:40<22:28, 1149.76 examples/s]
Tokenizing train dataset:  14%|█▍        | 258132/1807941 [03:40<23:34, 1095.40 examples/s]
Tokenizing train dataset:  14%|█▍        | 258251/1807941 [03:40<23:04, 1119.44 examples/s]
Tokenizing train dataset:  14%|█▍        | 258377/1807941 [03:40<22:20, 1155.77 examples/s]
Tokenizing train dataset:  14%|█▍        | 258550/1807941 [03:41<22:25, 1151.33 examples/s]
Tokenizing train dataset:  14%|█▍        | 258677/1807941 [03:41<21:54, 1178.85 examples/s]
Tokenizing train dataset:  14%|█▍        | 258843/1807941 [03:41<22:26, 1150.67 examples/s]
Tokenizing train dataset:  14%|█▍        | 258973/1807941 [03:41<21:44, 1187.27 examples/s]
Tokenizing train dataset:  14%|█▍        | 259119/1807941 [03:41<23:17, 1108.53 examples/s]
Tokenizing train dataset:  14%|█▍        | 259236/1807941 [03:41<23:00, 1121.52 examples/s]
Tokenizing train dataset:  14%|█▍        | 259357/1807941 [03:41<22:33, 1143.75 examples/s]
Tokenizing train dataset:  14%|█▍        | 259532/1807941 [03:41<22:27, 1148.86 examples/s]
Tokenizing train dataset:  14%|█▍        | 259705/1807941 [03:42<22:29, 1146.96 examples/s]
Tokenizing train dataset:  14%|█▍        | 259878/1807941 [03:42<22:30, 1146.41 examples/s]
Tokenizing train dataset:  14%|█▍        | 260022/1807941 [03:42<23:47, 1084.38 examples/s]
Tokenizing train dataset:  14%|█▍        | 260133/1807941 [03:42<23:42, 1087.99 examples/s]
Tokenizing train dataset:  14%|█▍        | 260250/1807941 [03:42<23:22, 1103.35 examples/s]
Tokenizing train dataset:  14%|█▍        | 260414/1807941 [03:42<23:34, 1094.17 examples/s]
Tokenizing train dataset:  14%|█▍        | 260584/1807941 [03:42<23:21, 1103.95 examples/s]
Tokenizing train dataset:  14%|█▍        | 260695/1807941 [03:43<23:22, 1103.32 examples/s]
Tokenizing train dataset:  14%|█▍        | 260810/1807941 [03:43<23:12, 1111.41 examples/s]
Tokenizing train dataset:  14%|█▍        | 260925/1807941 [03:43<23:03, 1118.37 examples/s]
Tokenizing train dataset:  14%|█▍        | 261066/1807941 [03:43<24:31, 1051.52 examples/s]
Tokenizing train dataset:  14%|█▍        | 261177/1807941 [03:43<24:15, 1062.78 examples/s]
Tokenizing train dataset:  14%|█▍        | 261293/1807941 [03:43<23:44, 1085.48 examples/s]
Tokenizing train dataset:  14%|█▍        | 261409/1807941 [03:43<23:22, 1103.06 examples/s]
Tokenizing train dataset:  14%|█▍        | 261526/1807941 [03:43<23:02, 1118.28 examples/s]
Tokenizing train dataset:  14%|█▍        | 261646/1807941 [03:43<22:38, 1138.20 examples/s]
Tokenizing train dataset:  14%|█▍        | 261821/1807941 [03:44<22:30, 1144.56 examples/s]
Tokenizing train dataset:  14%|█▍        | 261939/1807941 [03:44<22:26, 1147.75 examples/s]
Tokenizing train dataset:  14%|█▍        | 262087/1807941 [03:44<23:46, 1083.50 examples/s]
Tokenizing train dataset:  15%|█▍        | 262259/1807941 [03:44<23:24, 1100.19 examples/s]
Tokenizing train dataset:  15%|█▍        | 262377/1807941 [03:44<23:01, 1118.59 examples/s]
Tokenizing train dataset:  15%|█▍        | 262507/1807941 [03:44<22:08, 1163.55 examples/s]
Tokenizing train dataset:  15%|█▍        | 262630/1807941 [03:44<21:51, 1178.09 examples/s]
Tokenizing train dataset:  15%|█▍        | 262754/1807941 [03:44<21:37, 1191.32 examples/s]
Tokenizing train dataset:  15%|█▍        | 262879/1807941 [03:44<21:22, 1204.39 examples/s]
Tokenizing train dataset:  15%|█▍        | 263058/1807941 [03:45<22:54, 1124.27 examples/s]
Tokenizing train dataset:  15%|█▍        | 263181/1807941 [03:45<22:25, 1147.77 examples/s]
Tokenizing train dataset:  15%|█▍        | 263303/1807941 [03:45<22:07, 1163.39 examples/s]
Tokenizing train dataset:  15%|█▍        | 263428/1807941 [03:45<21:43, 1185.06 examples/s]
Tokenizing train dataset:  15%|█▍        | 263555/1807941 [03:45<21:22, 1204.46 examples/s]
Tokenizing train dataset:  15%|█▍        | 263686/1807941 [03:45<20:53, 1232.28 examples/s]
Tokenizing train dataset:  15%|█▍        | 263867/1807941 [03:45<21:06, 1218.78 examples/s]
Tokenizing train dataset:  15%|█▍        | 263991/1807941 [03:45<21:05, 1219.84 examples/s]
Tokenizing train dataset:  15%|█▍        | 264143/1807941 [03:46<22:31, 1141.95 examples/s]
Tokenizing train dataset:  15%|█▍        | 264263/1807941 [03:46<22:18, 1152.99 examples/s]
Tokenizing train dataset:  15%|█▍        | 264436/1807941 [03:46<22:20, 1151.52 examples/s]
Tokenizing train dataset:  15%|█▍        | 264555/1807941 [03:46<22:10, 1160.15 examples/s]
Tokenizing train dataset:  15%|█▍        | 264679/1807941 [03:46<21:51, 1176.79 examples/s]
Tokenizing train dataset:  15%|█▍        | 264856/1807941 [03:46<21:55, 1172.57 examples/s]
Tokenizing train dataset:  15%|█▍        | 264980/1807941 [03:46<21:37, 1189.35 examples/s]
Tokenizing train dataset:  15%|█▍        | 265139/1807941 [03:46<22:33, 1139.54 examples/s]
Tokenizing train dataset:  15%|█▍        | 265257/1807941 [03:47<22:24, 1147.02 examples/s]
Tokenizing train dataset:  15%|█▍        | 265380/1807941 [03:47<22:02, 1166.14 examples/s]
Tokenizing train dataset:  15%|█▍        | 265501/1807941 [03:47<21:50, 1176.79 examples/s]
Tokenizing train dataset:  15%|█▍        | 265625/1807941 [03:47<21:31, 1193.80 examples/s]
Tokenizing train dataset:  15%|█▍        | 265748/1807941 [03:47<21:25, 1199.74 examples/s]
Tokenizing train dataset:  15%|█▍        | 265869/1807941 [03:47<21:24, 1200.32 examples/s]
Tokenizing train dataset:  15%|█▍        | 265995/1807941 [03:47<21:09, 1214.79 examples/s]
Tokenizing train dataset:  15%|█▍        | 266156/1807941 [03:47<22:13, 1155.87 examples/s]
Tokenizing train dataset:  15%|█▍        | 266329/1807941 [03:47<22:21, 1148.86 examples/s]
Tokenizing train dataset:  15%|█▍        | 266450/1807941 [03:48<22:08, 1160.37 examples/s]
Tokenizing train dataset:  15%|█▍        | 266631/1807941 [03:48<21:53, 1173.57 examples/s]
Tokenizing train dataset:  15%|█▍        | 266808/1807941 [03:48<21:55, 1171.84 examples/s]
Tokenizing train dataset:  15%|█▍        | 266932/1807941 [03:48<21:38, 1186.51 examples/s]
Tokenizing train dataset:  15%|█▍        | 267086/1807941 [03:48<22:45, 1128.74 examples/s]
Tokenizing train dataset:  15%|█▍        | 267209/1807941 [03:48<22:17, 1151.56 examples/s]
Tokenizing train dataset:  15%|█▍        | 267328/1807941 [03:48<22:10, 1157.72 examples/s]
Tokenizing train dataset:  15%|█▍        | 267454/1807941 [03:48<21:41, 1183.54 examples/s]
Tokenizing train dataset:  15%|█▍        | 267579/1807941 [03:48<21:23, 1199.66 examples/s]
Tokenizing train dataset:  15%|█▍        | 267707/1807941 [03:49<21:06, 1216.19 examples/s]
Tokenizing train dataset:  15%|█▍        | 267888/1807941 [03:49<21:13, 1209.39 examples/s]
Tokenizing train dataset:  15%|█▍        | 268059/1807941 [03:49<22:36, 1135.11 examples/s]
Tokenizing train dataset:  15%|█▍        | 268180/1807941 [03:49<22:17, 1151.62 examples/s]
Tokenizing train dataset:  15%|█▍        | 268306/1807941 [03:49<21:47, 1177.24 examples/s]
Tokenizing train dataset:  15%|█▍        | 268489/1807941 [03:49<21:34, 1189.68 examples/s]
Tokenizing train dataset:  15%|█▍        | 268671/1807941 [03:49<21:30, 1192.40 examples/s]
Tokenizing train dataset:  15%|█▍        | 268795/1807941 [03:50<21:20, 1202.09 examples/s]
Tokenizing train dataset:  15%|█▍        | 268918/1807941 [03:50<21:15, 1206.78 examples/s]
Tokenizing train dataset:  15%|█▍        | 269070/1807941 [03:50<22:34, 1136.15 examples/s]
Tokenizing train dataset:  15%|█▍        | 269191/1807941 [03:50<22:14, 1153.20 examples/s]
Tokenizing train dataset:  15%|█▍        | 269310/1807941 [03:50<22:08, 1158.35 examples/s]
Tokenizing train dataset:  15%|█▍        | 269487/1807941 [03:50<22:01, 1164.05 examples/s]
Tokenizing train dataset:  15%|█▍        | 269618/1807941 [03:50<21:21, 1200.07 examples/s]
Tokenizing train dataset:  15%|█▍        | 269753/1807941 [03:50<20:42, 1238.28 examples/s]
Tokenizing train dataset:  15%|█▍        | 269942/1807941 [03:50<20:39, 1240.83 examples/s]
Tokenizing train dataset:  15%|█▍        | 270124/1807941 [03:51<21:38, 1184.41 examples/s]
Tokenizing train dataset:  15%|█▍        | 270262/1807941 [03:51<20:50, 1229.53 examples/s]
Tokenizing train dataset:  15%|█▍        | 270397/1807941 [03:51<20:23, 1256.82 examples/s]
Tokenizing train dataset:  15%|█▍        | 270531/1807941 [03:51<20:04, 1276.21 examples/s]
Tokenizing train dataset:  15%|█▍        | 270666/1807941 [03:51<19:47, 1294.01 examples/s]
Tokenizing train dataset:  15%|█▍        | 270861/1807941 [03:51<19:50, 1291.60 examples/s]
Tokenizing train dataset:  15%|█▍        | 271014/1807941 [03:51<21:25, 1195.85 examples/s]
Tokenizing train dataset:  15%|█▍        | 271144/1807941 [03:51<21:00, 1219.30 examples/s]
Tokenizing train dataset:  15%|█▌        | 271285/1807941 [03:52<20:12, 1267.48 examples/s]
Tokenizing train dataset:  15%|█▌        | 271419/1807941 [03:52<19:55, 1285.69 examples/s]
Tokenizing train dataset:  15%|█▌        | 271562/1807941 [03:52<19:20, 1323.51 examples/s]
Tokenizing train dataset:  15%|█▌        | 271696/1807941 [03:52<19:16, 1327.92 examples/s]
Tokenizing train dataset:  15%|█▌        | 271890/1807941 [03:52<19:31, 1311.09 examples/s]
Tokenizing train dataset:  15%|█▌        | 272058/1807941 [03:52<21:13, 1205.65 examples/s]
Tokenizing train dataset:  15%|█▌        | 272232/1807941 [03:52<21:31, 1188.86 examples/s]
Tokenizing train dataset:  15%|█▌        | 272412/1807941 [03:52<21:31, 1188.50 examples/s]
Tokenizing train dataset:  15%|█▌        | 272535/1807941 [03:53<21:24, 1195.42 examples/s]
Tokenizing train dataset:  15%|█▌        | 272658/1807941 [03:53<21:17, 1201.59 examples/s]
Tokenizing train dataset:  15%|█▌        | 272834/1807941 [03:53<21:30, 1189.91 examples/s]
Tokenizing train dataset:  15%|█▌        | 272957/1807941 [03:53<21:21, 1198.06 examples/s]
Tokenizing train dataset:  15%|█▌        | 273120/1807941 [03:53<22:41, 1127.31 examples/s]
Tokenizing train dataset:  15%|█▌        | 273245/1807941 [03:53<22:08, 1155.23 examples/s]
Tokenizing train dataset:  15%|█▌        | 273373/1807941 [03:53<21:36, 1183.71 examples/s]
Tokenizing train dataset:  15%|█▌        | 273500/1807941 [03:53<21:15, 1202.89 examples/s]
Tokenizing train dataset:  15%|█▌        | 273622/1807941 [03:53<21:13, 1204.43 examples/s]
Tokenizing train dataset:  15%|█▌        | 273800/1807941 [03:54<21:23, 1195.09 examples/s]
Tokenizing train dataset:  15%|█▌        | 273923/1807941 [03:54<21:18, 1200.10 examples/s]
Tokenizing train dataset:  15%|█▌        | 274063/1807941 [03:54<23:11, 1101.98 examples/s]
Tokenizing train dataset:  15%|█▌        | 274182/1807941 [03:54<22:46, 1122.46 examples/s]
Tokenizing train dataset:  15%|█▌        | 274302/1807941 [03:54<22:25, 1139.88 examples/s]
Tokenizing train dataset:  15%|█▌        | 274420/1807941 [03:54<22:14, 1148.97 examples/s]
Tokenizing train dataset:  15%|█▌        | 274593/1807941 [03:54<22:19, 1144.87 examples/s]
Tokenizing train dataset:  15%|█▌        | 274768/1807941 [03:54<22:14, 1149.14 examples/s]
Tokenizing train dataset:  15%|█▌        | 274885/1807941 [03:55<22:10, 1151.88 examples/s]
Tokenizing train dataset:  15%|█▌        | 275063/1807941 [03:55<23:11, 1101.90 examples/s]
Tokenizing train dataset:  15%|█▌        | 275177/1807941 [03:55<23:02, 1108.48 examples/s]
Tokenizing train dataset:  15%|█▌        | 275302/1807941 [03:55<22:22, 1141.51 examples/s]
Tokenizing train dataset:  15%|█▌        | 275418/1807941 [03:55<22:19, 1144.29 examples/s]
Tokenizing train dataset:  15%|█▌        | 275537/1807941 [03:55<22:09, 1152.42 examples/s]
Tokenizing train dataset:  15%|█▌        | 275660/1807941 [03:55<21:47, 1172.20 examples/s]
Tokenizing train dataset:  15%|█▌        | 275782/1807941 [03:55<21:39, 1179.40 examples/s]
Tokenizing train dataset:  15%|█▌        | 275957/1807941 [03:56<21:48, 1171.05 examples/s]
Tokenizing train dataset:  15%|█▌        | 276120/1807941 [03:56<22:56, 1112.74 examples/s]
Tokenizing train dataset:  15%|█▌        | 276243/1807941 [03:56<22:23, 1140.23 examples/s]
Tokenizing train dataset:  15%|█▌        | 276370/1807941 [03:56<21:46, 1171.96 examples/s]
Tokenizing train dataset:  15%|█▌        | 276540/1807941 [03:56<22:05, 1155.35 examples/s]
Tokenizing train dataset:  15%|█▌        | 276716/1807941 [03:56<22:02, 1157.94 examples/s]
Tokenizing train dataset:  15%|█▌        | 276882/1807941 [03:56<22:22, 1140.62 examples/s]
Tokenizing train dataset:  15%|█▌        | 277000/1807941 [03:56<23:22, 1091.86 examples/s]
Tokenizing train dataset:  15%|█▌        | 277115/1807941 [03:57<23:09, 1101.82 examples/s]
Tokenizing train dataset:  15%|█▌        | 277233/1807941 [03:57<22:46, 1120.06 examples/s]
Tokenizing train dataset:  15%|█▌        | 277356/1807941 [03:57<22:15, 1145.74 examples/s]
Tokenizing train dataset:  15%|█▌        | 277475/1807941 [03:57<22:04, 1155.82 examples/s]
Tokenizing train dataset:  15%|█▌        | 277599/1807941 [03:57<21:40, 1177.05 examples/s]
Tokenizing train dataset:  15%|█▌        | 277727/1807941 [03:57<21:11, 1203.50 examples/s]
Tokenizing train dataset:  15%|█▌        | 277858/1807941 [03:57<20:42, 1231.41 examples/s]
Tokenizing train dataset:  15%|█▌        | 278007/1807941 [03:57<22:25, 1136.66 examples/s]
Tokenizing train dataset:  15%|█▌        | 278125/1807941 [03:57<22:18, 1143.06 examples/s]
Tokenizing train dataset:  15%|█▌        | 278259/1807941 [03:58<21:20, 1194.22 examples/s]
Tokenizing train dataset:  15%|█▌        | 278429/1807941 [03:58<21:48, 1169.14 examples/s]
Tokenizing train dataset:  15%|█▌        | 278554/1807941 [03:58<21:26, 1188.39 examples/s]
Tokenizing train dataset:  15%|█▌        | 278681/1807941 [03:58<21:03, 1210.09 examples/s]
Tokenizing train dataset:  15%|█▌        | 278812/1807941 [03:58<20:37, 1235.89 examples/s]
Tokenizing train dataset:  15%|█▌        | 278942/1807941 [03:58<20:20, 1252.29 examples/s]
Tokenizing train dataset:  15%|█▌        | 279090/1807941 [03:58<22:13, 1146.11 examples/s]
Tokenizing train dataset:  15%|█▌        | 279221/1807941 [03:58<21:25, 1189.19 examples/s]
Tokenizing train dataset:  15%|█▌        | 279361/1807941 [03:58<23:17, 1094.01 examples/s]
Tokenizing train dataset:  15%|█▌        | 279534/1807941 [03:59<22:58, 1108.90 examples/s]
Tokenizing train dataset:  15%|█▌        | 279650/1807941 [03:59<22:45, 1118.85 examples/s]
Tokenizing train dataset:  15%|█▌        | 279768/1807941 [03:59<22:29, 1132.14 examples/s]
Tokenizing train dataset:  15%|█▌        | 279938/1807941 [03:59<22:32, 1130.06 examples/s]
Tokenizing train dataset:  15%|█▌        | 280079/1807941 [03:59<23:56, 1063.35 examples/s]
Tokenizing train dataset:  15%|█▌        | 280196/1807941 [03:59<23:27, 1085.74 examples/s]
Tokenizing train dataset:  16%|█▌        | 280358/1807941 [03:59<23:31, 1082.22 examples/s]
Tokenizing train dataset:  16%|█▌        | 280474/1807941 [03:59<23:11, 1097.77 examples/s]
Tokenizing train dataset:  16%|█▌        | 280585/1807941 [04:00<23:09, 1098.97 examples/s]
Tokenizing train dataset:  16%|█▌        | 280702/1807941 [04:00<22:50, 1114.16 examples/s]
Tokenizing train dataset:  16%|█▌        | 280864/1807941 [04:00<23:10, 1098.36 examples/s]
Tokenizing train dataset:  16%|█▌        | 280977/1807941 [04:00<23:04, 1103.05 examples/s]
Tokenizing train dataset:  16%|█▌        | 281118/1807941 [04:00<24:29, 1038.99 examples/s]
Tokenizing train dataset:  16%|█▌        | 281226/1807941 [04:00<24:18, 1046.59 examples/s]
Tokenizing train dataset:  16%|█▌        | 281341/1807941 [04:00<23:45, 1071.00 examples/s]
Tokenizing train dataset:  16%|█▌        | 281453/1807941 [04:00<23:32, 1080.91 examples/s]
Tokenizing train dataset:  16%|█▌        | 281571/1807941 [04:00<23:00, 1105.99 examples/s]
Tokenizing train dataset:  16%|█▌        | 281692/1807941 [04:01<22:26, 1133.28 examples/s]
Tokenizing train dataset:  16%|█▌        | 281808/1807941 [04:01<22:22, 1137.05 examples/s]
Tokenizing train dataset:  16%|█▌        | 281926/1807941 [04:01<22:10, 1146.67 examples/s]
Tokenizing train dataset:  16%|█▌        | 282088/1807941 [04:01<26:13, 969.83 examples/s] 
Tokenizing train dataset:  16%|█▌        | 282206/1807941 [04:01<24:59, 1017.22 examples/s]
Tokenizing train dataset:  16%|█▌        | 282323/1807941 [04:01<24:06, 1054.52 examples/s]
Tokenizing train dataset:  16%|█▌        | 282485/1807941 [04:01<23:57, 1061.51 examples/s]
Tokenizing train dataset:  16%|█▌        | 282602/1807941 [04:01<23:21, 1088.27 examples/s]
Tokenizing train dataset:  16%|█▌        | 282716/1807941 [04:02<23:11, 1096.15 examples/s]
Tokenizing train dataset:  16%|█▌        | 282840/1807941 [04:02<22:25, 1133.16 examples/s]
Tokenizing train dataset:  16%|█▌        | 282962/1807941 [04:02<21:59, 1155.90 examples/s]
Tokenizing train dataset:  16%|█▌        | 283123/1807941 [04:02<22:57, 1106.74 examples/s]
Tokenizing train dataset:  16%|█▌        | 283280/1807941 [04:02<23:27, 1082.97 examples/s]
Tokenizing train dataset:  16%|█▌        | 283403/1807941 [04:02<22:45, 1116.35 examples/s]
Tokenizing train dataset:  16%|█▌        | 283528/1807941 [04:02<22:06, 1148.99 examples/s]
Tokenizing train dataset:  16%|█▌        | 283655/1807941 [04:02<21:33, 1178.83 examples/s]
Tokenizing train dataset:  16%|█▌        | 283781/1807941 [04:02<21:11, 1198.80 examples/s]
Tokenizing train dataset:  16%|█▌        | 283905/1807941 [04:03<21:00, 1208.90 examples/s]
Tokenizing train dataset:  16%|█▌        | 284062/1807941 [04:03<22:14, 1142.30 examples/s]
Tokenizing train dataset:  16%|█▌        | 284234/1807941 [04:03<22:13, 1142.73 examples/s]
Tokenizing train dataset:  16%|█▌        | 284351/1807941 [04:03<22:08, 1146.82 examples/s]
Tokenizing train dataset:  16%|█▌        | 284470/1807941 [04:03<21:59, 1154.54 examples/s]
Tokenizing train dataset:  16%|█▌        | 284593/1807941 [04:03<21:41, 1170.72 examples/s]
Tokenizing train dataset:  16%|█▌        | 284712/1807941 [04:03<21:38, 1173.06 examples/s]
Tokenizing train dataset:  16%|█▌        | 284834/1807941 [04:03<21:24, 1185.76 examples/s]
Tokenizing train dataset:  16%|█▌        | 284976/1807941 [04:04<23:15, 1091.62 examples/s]
Tokenizing train dataset:  16%|█▌        | 285116/1807941 [04:04<24:40, 1028.86 examples/s]
Tokenizing train dataset:  16%|█▌        | 285235/1807941 [04:04<23:48, 1066.07 examples/s]
Tokenizing train dataset:  16%|█▌        | 285346/1807941 [04:04<23:36, 1074.92 examples/s]
Tokenizing train dataset:  16%|█▌        | 285461/1807941 [04:04<23:15, 1091.14 examples/s]
Tokenizing train dataset:  16%|█▌        | 285630/1807941 [04:04<23:06, 1098.29 examples/s]
Tokenizing train dataset:  16%|█▌        | 285745/1807941 [04:04<22:51, 1110.02 examples/s]
Tokenizing train dataset:  16%|█▌        | 285863/1807941 [04:04<22:29, 1128.27 examples/s]
Tokenizing train dataset:  16%|█▌        | 285982/1807941 [04:04<22:11, 1143.20 examples/s]
Tokenizing train dataset:  16%|█▌        | 286141/1807941 [04:05<22:52, 1108.44 examples/s]
Tokenizing train dataset:  16%|█▌        | 286278/1807941 [04:05<21:36, 1173.22 examples/s]
Tokenizing train dataset:  16%|█▌        | 286407/1807941 [04:05<21:06, 1201.69 examples/s]
Tokenizing train dataset:  16%|█▌        | 286535/1807941 [04:05<20:44, 1222.89 examples/s]
Tokenizing train dataset:  16%|█▌        | 286715/1807941 [04:05<20:57, 1209.84 examples/s]
Tokenizing train dataset:  16%|█▌        | 286844/1807941 [04:05<20:37, 1229.13 examples/s]
Tokenizing train dataset:  16%|█▌        | 287000/1807941 [04:05<22:26, 1129.92 examples/s]
Tokenizing train dataset:  16%|█▌        | 287137/1807941 [04:05<24:06, 1051.38 examples/s]
Tokenizing train dataset:  16%|█▌        | 287295/1807941 [04:06<24:10, 1048.53 examples/s]
Tokenizing train dataset:  16%|█▌        | 287412/1807941 [04:06<23:37, 1072.41 examples/s]
Tokenizing train dataset:  16%|█▌        | 287527/1807941 [04:06<23:14, 1090.67 examples/s]
Tokenizing train dataset:  16%|█▌        | 287645/1807941 [04:06<22:47, 1111.96 examples/s]
Tokenizing train dataset:  16%|█▌        | 287760/1807941 [04:06<22:36, 1120.35 examples/s]
Tokenizing train dataset:  16%|█▌        | 287874/1807941 [04:06<22:36, 1120.70 examples/s]
Tokenizing train dataset:  16%|█▌        | 287998/1807941 [04:06<21:59, 1151.50 examples/s]
Tokenizing train dataset:  16%|█▌        | 288154/1807941 [04:06<22:54, 1105.66 examples/s]
Tokenizing train dataset:  16%|█▌        | 288274/1807941 [04:06<22:27, 1128.00 examples/s]
Tokenizing train dataset:  16%|█▌        | 288394/1807941 [04:07<22:05, 1146.24 examples/s]
Tokenizing train dataset:  16%|█▌        | 288518/1807941 [04:07<21:38, 1170.49 examples/s]
Tokenizing train dataset:  16%|█▌        | 288644/1807941 [04:07<21:12, 1193.98 examples/s]
Tokenizing train dataset:  16%|█▌        | 288822/1807941 [04:07<21:17, 1189.15 examples/s]
Tokenizing train dataset:  16%|█▌        | 288944/1807941 [04:07<21:12, 1193.69 examples/s]
Tokenizing train dataset:  16%|█▌        | 289065/1807941 [04:07<22:06, 1145.28 examples/s]
Tokenizing train dataset:  16%|█▌        | 289226/1807941 [04:07<22:43, 1113.93 examples/s]
Tokenizing train dataset:  16%|█▌        | 289391/1807941 [04:07<22:56, 1103.50 examples/s]
Tokenizing train dataset:  16%|█▌        | 289510/1807941 [04:08<22:36, 1119.58 examples/s]
Tokenizing train dataset:  16%|█▌        | 289631/1807941 [04:08<22:11, 1140.28 examples/s]
Tokenizing train dataset:  16%|█▌        | 289803/1807941 [04:08<22:11, 1139.85 examples/s]
Tokenizing train dataset:  16%|█▌        | 289928/1807941 [04:08<21:42, 1165.84 examples/s]
Tokenizing train dataset:  16%|█▌        | 290062/1807941 [04:08<23:43, 1066.55 examples/s]
Tokenizing train dataset:  16%|█▌        | 290179/1807941 [04:08<23:12, 1089.86 examples/s]
Tokenizing train dataset:  16%|█▌        | 290308/1807941 [04:08<22:11, 1139.86 examples/s]
Tokenizing train dataset:  16%|█▌        | 290436/1807941 [04:08<21:33, 1173.24 examples/s]
Tokenizing train dataset:  16%|█▌        | 290567/1807941 [04:08<20:55, 1209.06 examples/s]
Tokenizing train dataset:  16%|█▌        | 290695/1807941 [04:09<20:35, 1227.70 examples/s]
Tokenizing train dataset:  16%|█▌        | 290822/1807941 [04:09<20:25, 1237.65 examples/s]
Tokenizing train dataset:  16%|█▌        | 291000/1807941 [04:09<21:34, 1172.21 examples/s]
Tokenizing train dataset:  16%|█▌        | 291142/1807941 [04:09<23:09, 1091.25 examples/s]
Tokenizing train dataset:  16%|█▌        | 291260/1807941 [04:09<22:44, 1111.75 examples/s]
Tokenizing train dataset:  16%|█▌        | 291400/1807941 [04:09<21:19, 1185.08 examples/s]
Tokenizing train dataset:  16%|█▌        | 291531/1807941 [04:09<20:48, 1214.89 examples/s]
Tokenizing train dataset:  16%|█▌        | 291662/1807941 [04:09<20:26, 1235.99 examples/s]
Tokenizing train dataset:  16%|█▌        | 291834/1807941 [04:09<18:26, 1370.13 examples/s]
Tokenizing train dataset:  16%|█▌        | 291999/1807941 [04:10<17:27, 1447.86 examples/s]
Tokenizing train dataset:  16%|█▌        | 292206/1807941 [04:10<17:47, 1419.76 examples/s]
Tokenizing train dataset:  16%|█▌        | 292375/1807941 [04:10<16:56, 1490.30 examples/s]
Tokenizing train dataset:  16%|█▌        | 292543/1807941 [04:10<16:23, 1541.36 examples/s]
Tokenizing train dataset:  16%|█▌        | 292705/1807941 [04:10<16:10, 1560.85 examples/s]
Tokenizing train dataset:  16%|█▌        | 292874/1807941 [04:10<15:48, 1597.17 examples/s]
Tokenizing train dataset:  16%|█▌        | 293097/1807941 [04:10<16:16, 1551.61 examples/s]
Tokenizing train dataset:  16%|█▌        | 293264/1807941 [04:10<15:57, 1581.39 examples/s]
Tokenizing train dataset:  16%|█▌        | 293431/1807941 [04:10<15:44, 1603.08 examples/s]
Tokenizing train dataset:  16%|█▌        | 293600/1807941 [04:11<15:32, 1623.74 examples/s]
Tokenizing train dataset:  16%|█▌        | 293765/1807941 [04:11<15:31, 1625.39 examples/s]
Tokenizing train dataset:  16%|█▋        | 293929/1807941 [04:11<15:30, 1626.79 examples/s]
Tokenizing train dataset:  16%|█▋        | 294163/1807941 [04:11<16:05, 1568.46 examples/s]
Tokenizing train dataset:  16%|█▋        | 294330/1807941 [04:11<15:49, 1593.82 examples/s]
Tokenizing train dataset:  16%|█▋        | 294499/1807941 [04:11<15:36, 1615.70 examples/s]
Tokenizing train dataset:  16%|█▋        | 294663/1807941 [04:11<15:34, 1620.05 examples/s]
Tokenizing train dataset:  16%|█▋        | 294870/1807941 [04:11<16:32, 1525.13 examples/s]
Tokenizing train dataset:  16%|█▋        | 295032/1807941 [04:12<18:26, 1367.01 examples/s]
Tokenizing train dataset:  16%|█▋        | 295214/1807941 [04:12<19:13, 1311.12 examples/s]
Tokenizing train dataset:  16%|█▋        | 295409/1807941 [04:12<19:21, 1302.73 examples/s]
Tokenizing train dataset:  16%|█▋        | 295601/1807941 [04:12<19:30, 1292.30 examples/s]
Tokenizing train dataset:  16%|█▋        | 295791/1807941 [04:12<19:39, 1281.61 examples/s]
Tokenizing train dataset:  16%|█▋        | 295983/1807941 [04:12<19:43, 1277.09 examples/s]
Tokenizing train dataset:  16%|█▋        | 296135/1807941 [04:12<21:07, 1192.90 examples/s]
Tokenizing train dataset:  16%|█▋        | 296257/1807941 [04:13<21:01, 1197.89 examples/s]
Tokenizing train dataset:  16%|█▋        | 296380/1807941 [04:13<20:54, 1204.71 examples/s]
Tokenizing train dataset:  16%|█▋        | 296507/1807941 [04:13<20:39, 1218.92 examples/s]
Tokenizing train dataset:  16%|█▋        | 296679/1807941 [04:13<21:13, 1186.87 examples/s]
Tokenizing train dataset:  16%|█▋        | 296813/1807941 [04:13<20:36, 1222.04 examples/s]
Tokenizing train dataset:  16%|█▋        | 296940/1807941 [04:13<20:27, 1231.36 examples/s]
Tokenizing train dataset:  16%|█▋        | 297110/1807941 [04:13<22:10, 1135.67 examples/s]
Tokenizing train dataset:  16%|█▋        | 297242/1807941 [04:13<21:21, 1178.77 examples/s]
Tokenizing train dataset:  16%|█▋        | 297421/1807941 [04:14<21:19, 1180.93 examples/s]
Tokenizing train dataset:  16%|█▋        | 297547/1807941 [04:14<20:59, 1199.01 examples/s]
Tokenizing train dataset:  16%|█▋        | 297677/1807941 [04:14<20:34, 1223.67 examples/s]
Tokenizing train dataset:  16%|█▋        | 297807/1807941 [04:14<20:16, 1241.80 examples/s]
Tokenizing train dataset:  16%|█▋        | 297992/1807941 [04:14<20:22, 1234.78 examples/s]
Tokenizing train dataset:  16%|█▋        | 298144/1807941 [04:14<21:45, 1156.47 examples/s]
Tokenizing train dataset:  16%|█▋        | 298304/1807941 [04:14<22:24, 1122.56 examples/s]
Tokenizing train dataset:  17%|█▋        | 298451/1807941 [04:14<23:23, 1075.63 examples/s]
Tokenizing train dataset:  17%|█▋        | 298577/1807941 [04:15<22:31, 1116.88 examples/s]
Tokenizing train dataset:  17%|█▋        | 298706/1807941 [04:15<21:43, 1157.95 examples/s]
Tokenizing train dataset:  17%|█▋        | 298833/1807941 [04:15<21:14, 1184.44 examples/s]
Tokenizing train dataset:  17%|█▋        | 298980/1807941 [04:15<19:56, 1261.49 examples/s]
Tokenizing train dataset:  17%|█▋        | 299132/1807941 [04:15<21:32, 1167.79 examples/s]
Tokenizing train dataset:  17%|█▋        | 299269/1807941 [04:15<20:40, 1215.78 examples/s]
Tokenizing train dataset:  17%|█▋        | 299397/1807941 [04:15<20:26, 1229.79 examples/s]
Tokenizing train dataset:  17%|█▋        | 299581/1807941 [04:15<20:34, 1222.32 examples/s]
Tokenizing train dataset:  17%|█▋        | 299706/1807941 [04:15<20:27, 1228.42 examples/s]
Tokenizing train dataset:  17%|█▋        | 299831/1807941 [04:16<20:24, 1231.70 examples/s]
Tokenizing train dataset:  17%|█▋        | 300000/1807941 [04:16<21:59, 1142.57 examples/s]
Tokenizing train dataset:  17%|█▋        | 300138/1807941 [04:16<20:57, 1199.42 examples/s]
Tokenizing train dataset:  17%|█▋        | 300317/1807941 [04:16<21:03, 1193.25 examples/s]
Tokenizing train dataset:  17%|█▋        | 300439/1807941 [04:16<20:57, 1198.96 examples/s]
Tokenizing train dataset:  17%|█▋        | 300565/1807941 [04:16<20:44, 1210.79 examples/s]
Tokenizing train dataset:  17%|█▋        | 300738/1807941 [04:16<21:11, 1185.20 examples/s]
Tokenizing train dataset:  17%|█▋        | 300858/1807941 [04:16<21:11, 1184.86 examples/s]
Tokenizing train dataset:  17%|█▋        | 300982/1807941 [04:17<20:58, 1197.72 examples/s]
Tokenizing train dataset:  17%|█▋        | 301136/1807941 [04:17<22:11, 1131.96 examples/s]
Tokenizing train dataset:  17%|█▋        | 301302/1807941 [04:17<22:24, 1120.52 examples/s]
Tokenizing train dataset:  17%|█▋        | 301461/1807941 [04:17<22:51, 1098.57 examples/s]
Tokenizing train dataset:  17%|█▋        | 301602/1807941 [04:17<24:04, 1043.11 examples/s]
Tokenizing train dataset:  17%|█▋        | 301731/1807941 [04:17<22:51, 1098.34 examples/s]
Tokenizing train dataset:  17%|█▋        | 301854/1807941 [04:17<22:15, 1127.62 examples/s]
Tokenizing train dataset:  17%|█▋        | 301973/1807941 [04:17<21:58, 1142.36 examples/s]
Tokenizing train dataset:  17%|█▋        | 302126/1807941 [04:18<22:55, 1095.05 examples/s]
Tokenizing train dataset:  17%|█▋        | 302300/1807941 [04:18<22:30, 1114.50 examples/s]
Tokenizing train dataset:  17%|█▋        | 302425/1807941 [04:18<21:53, 1145.88 examples/s]
Tokenizing train dataset:  17%|█▋        | 302593/1807941 [04:18<22:08, 1132.97 examples/s]
Tokenizing train dataset:  17%|█▋        | 302713/1807941 [04:18<21:51, 1147.39 examples/s]
Tokenizing train dataset:  17%|█▋        | 302841/1807941 [04:18<21:15, 1179.83 examples/s]
Tokenizing train dataset:  17%|█▋        | 303000/1807941 [04:18<22:49, 1098.72 examples/s]
Tokenizing train dataset:  17%|█▋        | 303160/1807941 [04:19<23:05, 1086.18 examples/s]
Tokenizing train dataset:  17%|█▋        | 303286/1807941 [04:19<22:17, 1124.72 examples/s]
Tokenizing train dataset:  17%|█▋        | 303414/1807941 [04:19<21:37, 1159.80 examples/s]
Tokenizing train dataset:  17%|█▋        | 303589/1807941 [04:19<21:37, 1159.60 examples/s]
Tokenizing train dataset:  17%|█▋        | 303711/1807941 [04:19<21:25, 1170.37 examples/s]
Tokenizing train dataset:  17%|█▋        | 303843/1807941 [04:19<20:47, 1205.54 examples/s]
Tokenizing train dataset:  17%|█▋        | 303968/1807941 [04:19<20:38, 1214.50 examples/s]
Tokenizing train dataset:  17%|█▋        | 304124/1807941 [04:19<21:51, 1146.65 examples/s]
Tokenizing train dataset:  17%|█▋        | 304266/1807941 [04:19<20:37, 1215.17 examples/s]
Tokenizing train dataset:  17%|█▋        | 304415/1807941 [04:20<19:27, 1287.65 examples/s]
Tokenizing train dataset:  17%|█▋        | 304637/1807941 [04:20<18:28, 1356.21 examples/s]
Tokenizing train dataset:  17%|█▋        | 304814/1807941 [04:20<17:09, 1460.51 examples/s]
Tokenizing train dataset:  17%|█▋        | 305000/1807941 [04:20<18:21, 1364.29 examples/s]
Tokenizing train dataset:  17%|█▋        | 305192/1807941 [04:20<18:47, 1333.20 examples/s]
Tokenizing train dataset:  17%|█▋        | 305371/1807941 [04:20<19:31, 1282.45 examples/s]
Tokenizing train dataset:  17%|█▋        | 305514/1807941 [04:20<21:19, 1174.69 examples/s]
Tokenizing train dataset:  17%|█▋        | 305642/1807941 [04:20<20:55, 1196.52 examples/s]
Tokenizing train dataset:  17%|█▋        | 305786/1807941 [04:21<19:59, 1251.84 examples/s]
Tokenizing train dataset:  17%|█▋        | 305914/1807941 [04:21<19:54, 1257.73 examples/s]
Tokenizing train dataset:  17%|█▋        | 306079/1807941 [04:21<20:54, 1196.72 examples/s]
Tokenizing train dataset:  17%|█▋        | 306206/1807941 [04:21<20:37, 1213.51 examples/s]
Tokenizing train dataset:  17%|█▋        | 306332/1807941 [04:21<20:27, 1223.29 examples/s]
Tokenizing train dataset:  17%|█▋        | 306462/1807941 [04:21<20:08, 1242.15 examples/s]
Tokenizing train dataset:  17%|█▋        | 306639/1807941 [04:21<20:34, 1215.96 examples/s]
Tokenizing train dataset:  17%|█▋        | 306815/1807941 [04:21<23:26, 1067.62 examples/s]
Tokenizing train dataset:  17%|█▋        | 306958/1807941 [04:22<21:48, 1147.37 examples/s]
Tokenizing train dataset:  17%|█▋        | 307124/1807941 [04:22<22:35, 1107.55 examples/s]
Tokenizing train dataset:  17%|█▋        | 307258/1807941 [04:22<21:34, 1159.13 examples/s]
Tokenizing train dataset:  17%|█▋        | 307391/1807941 [04:22<20:50, 1199.94 examples/s]
Tokenizing train dataset:  17%|█▋        | 307575/1807941 [04:22<20:44, 1205.45 examples/s]
Tokenizing train dataset:  17%|█▋        | 307699/1807941 [04:22<20:38, 1211.57 examples/s]
Tokenizing train dataset:  17%|█▋        | 307878/1807941 [04:22<20:48, 1201.55 examples/s]
Tokenizing train dataset:  17%|█▋        | 308018/1807941 [04:23<22:30, 1110.36 examples/s]
Tokenizing train dataset:  17%|█▋        | 308152/1807941 [04:23<21:30, 1162.14 examples/s]
Tokenizing train dataset:  17%|█▋        | 308321/1807941 [04:23<19:19, 1293.56 examples/s]
Tokenizing train dataset:  17%|█▋        | 308493/1807941 [04:23<17:47, 1404.93 examples/s]
Tokenizing train dataset:  17%|█▋        | 308664/1807941 [04:23<16:49, 1485.45 examples/s]
Tokenizing train dataset:  17%|█▋        | 308832/1807941 [04:23<16:15, 1537.50 examples/s]
Tokenizing train dataset:  17%|█▋        | 309000/1807941 [04:23<16:32, 1510.41 examples/s]
Tokenizing train dataset:  17%|█▋        | 309192/1807941 [04:23<17:36, 1418.17 examples/s]
Tokenizing train dataset:  17%|█▋        | 309382/1807941 [04:23<18:19, 1363.30 examples/s]
Tokenizing train dataset:  17%|█▋        | 309561/1807941 [04:24<19:10, 1302.71 examples/s]
Tokenizing train dataset:  17%|█▋        | 309760/1807941 [04:24<19:05, 1308.35 examples/s]
Tokenizing train dataset:  17%|█▋        | 309893/1807941 [04:24<19:01, 1312.23 examples/s]
Tokenizing train dataset:  17%|█▋        | 310076/1807941 [04:24<19:33, 1276.77 examples/s]
Tokenizing train dataset:  17%|█▋        | 310214/1807941 [04:24<19:12, 1299.60 examples/s]
Tokenizing train dataset:  17%|█▋        | 310404/1807941 [04:24<19:26, 1284.21 examples/s]
Tokenizing train dataset:  17%|█▋        | 310545/1807941 [04:24<19:00, 1312.36 examples/s]
Tokenizing train dataset:  17%|█▋        | 310716/1807941 [04:24<20:01, 1246.32 examples/s]
Tokenizing train dataset:  17%|█▋        | 310878/1807941 [04:25<20:57, 1190.52 examples/s]
Tokenizing train dataset:  17%|█▋        | 311017/1807941 [04:25<22:36, 1103.53 examples/s]
Tokenizing train dataset:  17%|█▋        | 311185/1807941 [04:25<22:33, 1105.53 examples/s]
Tokenizing train dataset:  17%|█▋        | 311321/1807941 [04:25<21:27, 1162.72 examples/s]
Tokenizing train dataset:  17%|█▋        | 311442/1807941 [04:25<21:16, 1172.78 examples/s]
Tokenizing train dataset:  17%|█▋        | 311575/1807941 [04:25<20:33, 1213.18 examples/s]
Tokenizing train dataset:  17%|█▋        | 311709/1807941 [04:25<20:01, 1245.21 examples/s]
Tokenizing train dataset:  17%|█▋        | 311846/1807941 [04:25<19:32, 1275.84 examples/s]
Tokenizing train dataset:  17%|█▋        | 312000/1807941 [04:26<21:21, 1167.51 examples/s]
Tokenizing train dataset:  17%|█▋        | 312132/1807941 [04:26<20:41, 1204.79 examples/s]
Tokenizing train dataset:  17%|█▋        | 312267/1807941 [04:26<20:03, 1242.74 examples/s]
Tokenizing train dataset:  17%|█▋        | 312406/1807941 [04:26<19:25, 1282.65 examples/s]
Tokenizing train dataset:  17%|█▋        | 312539/1807941 [04:26<19:17, 1292.35 examples/s]
Tokenizing train dataset:  17%|█▋        | 312680/1807941 [04:26<18:48, 1324.47 examples/s]
Tokenizing train dataset:  17%|█▋        | 312818/1807941 [04:26<18:38, 1336.14 examples/s]
Tokenizing train dataset:  17%|█▋        | 312954/1807941 [04:26<18:35, 1340.27 examples/s]
Tokenizing train dataset:  17%|█▋        | 313120/1807941 [04:26<20:33, 1212.33 examples/s]
Tokenizing train dataset:  17%|█▋        | 313299/1807941 [04:27<20:44, 1201.07 examples/s]
Tokenizing train dataset:  17%|█▋        | 313481/1807941 [04:27<20:43, 1201.62 examples/s]
Tokenizing train dataset:  17%|█▋        | 313665/1807941 [04:27<20:38, 1206.37 examples/s]
Tokenizing train dataset:  17%|█▋        | 313844/1807941 [04:27<20:46, 1198.34 examples/s]
Tokenizing train dataset:  17%|█▋        | 313967/1807941 [04:27<20:43, 1201.56 examples/s]
Tokenizing train dataset:  17%|█▋        | 314129/1807941 [04:27<21:56, 1134.65 examples/s]
Tokenizing train dataset:  17%|█▋        | 314308/1807941 [04:27<21:41, 1147.99 examples/s]
Tokenizing train dataset:  17%|█▋        | 314426/1807941 [04:28<21:35, 1152.60 examples/s]
Tokenizing train dataset:  17%|█▋        | 314552/1807941 [04:28<21:09, 1176.17 examples/s]
Tokenizing train dataset:  17%|█▋        | 314679/1807941 [04:28<20:51, 1193.46 examples/s]
Tokenizing train dataset:  17%|█▋        | 314800/1807941 [04:28<20:48, 1196.28 examples/s]
Tokenizing train dataset:  17%|█▋        | 314922/1807941 [04:28<20:44, 1200.03 examples/s]
Tokenizing train dataset:  17%|█▋        | 315072/1807941 [04:28<22:08, 1123.61 examples/s]
Tokenizing train dataset:  17%|█▋        | 315242/1807941 [04:28<22:09, 1122.58 examples/s]
Tokenizing train dataset:  17%|█▋        | 315367/1807941 [04:28<21:35, 1152.26 examples/s]
Tokenizing train dataset:  17%|█▋        | 315489/1807941 [04:28<21:20, 1165.93 examples/s]
Tokenizing train dataset:  17%|█▋        | 315614/1807941 [04:29<20:58, 1185.84 examples/s]
Tokenizing train dataset:  17%|█▋        | 315756/1807941 [04:29<22:45, 1093.08 examples/s]
Tokenizing train dataset:  17%|█▋        | 315908/1807941 [04:29<23:25, 1061.60 examples/s]
Tokenizing train dataset:  17%|█▋        | 316068/1807941 [04:29<23:26, 1060.91 examples/s]
Tokenizing train dataset:  17%|█▋        | 316182/1807941 [04:29<23:06, 1076.10 examples/s]
Tokenizing train dataset:  17%|█▋        | 316305/1807941 [04:29<22:19, 1113.58 examples/s]
Tokenizing train dataset:  18%|█▊        | 316425/1807941 [04:29<21:56, 1132.54 examples/s]
Tokenizing train dataset:  18%|█▊        | 316604/1807941 [04:30<21:36, 1150.43 examples/s]
Tokenizing train dataset:  18%|█▊        | 316730/1807941 [04:30<21:08, 1175.87 examples/s]
Tokenizing train dataset:  18%|█▊        | 316860/1807941 [04:30<20:34, 1207.94 examples/s]
Tokenizing train dataset:  18%|█▊        | 316985/1807941 [04:30<20:27, 1214.35 examples/s]
Tokenizing train dataset:  18%|█▊        | 317123/1807941 [04:30<22:33, 1101.42 examples/s]
Tokenizing train dataset:  18%|█▊        | 317242/1807941 [04:30<22:10, 1120.67 examples/s]
Tokenizing train dataset:  18%|█▊        | 317365/1807941 [04:30<21:39, 1147.13 examples/s]
Tokenizing train dataset:  18%|█▊        | 317491/1807941 [04:30<21:07, 1175.85 examples/s]
Tokenizing train dataset:  18%|█▊        | 317614/1807941 [04:30<20:56, 1185.67 examples/s]
Tokenizing train dataset:  18%|█▊        | 317739/1807941 [04:30<20:40, 1201.76 examples/s]
Tokenizing train dataset:  18%|█▊        | 317866/1807941 [04:31<20:24, 1217.07 examples/s]
Tokenizing train dataset:  18%|█▊        | 317995/1807941 [04:31<20:04, 1236.71 examples/s]
Tokenizing train dataset:  18%|█▊        | 318150/1807941 [04:31<21:30, 1154.03 examples/s]
Tokenizing train dataset:  18%|█▊        | 318285/1807941 [04:31<20:40, 1200.71 examples/s]
Tokenizing train dataset:  18%|█▊        | 318463/1807941 [04:31<20:50, 1191.53 examples/s]
Tokenizing train dataset:  18%|█▊        | 318592/1807941 [04:31<20:28, 1212.65 examples/s]
Tokenizing train dataset:  18%|█▊        | 318717/1807941 [04:31<20:21, 1219.36 examples/s]
Tokenizing train dataset:  18%|█▊        | 318841/1807941 [04:31<20:17, 1222.90 examples/s]
Tokenizing train dataset:  18%|█▊        | 318965/1807941 [04:31<20:14, 1226.38 examples/s]
Tokenizing train dataset:  18%|█▊        | 319098/1807941 [04:32<23:04, 1075.09 examples/s]
Tokenizing train dataset:  18%|█▊        | 319223/1807941 [04:32<22:10, 1118.83 examples/s]
Tokenizing train dataset:  18%|█▊        | 319394/1807941 [04:32<22:05, 1123.40 examples/s]
Tokenizing train dataset:  18%|█▊        | 319523/1807941 [04:32<21:20, 1162.52 examples/s]
Tokenizing train dataset:  18%|█▊        | 319664/1807941 [04:32<22:59, 1079.00 examples/s]
Tokenizing train dataset:  18%|█▊        | 319775/1807941 [04:32<22:51, 1085.13 examples/s]
Tokenizing train dataset:  18%|█▊        | 319897/1807941 [04:32<22:12, 1116.92 examples/s]
Tokenizing train dataset:  18%|█▊        | 320057/1807941 [04:33<23:15, 1065.97 examples/s]
Tokenizing train dataset:  18%|█▊        | 320184/1807941 [04:33<22:15, 1113.61 examples/s]
Tokenizing train dataset:  18%|█▊        | 320310/1807941 [04:33<21:35, 1148.40 examples/s]
Tokenizing train dataset:  18%|█▊        | 320433/1807941 [04:33<21:12, 1168.72 examples/s]
Tokenizing train dataset:  18%|█▊        | 320559/1807941 [04:33<20:47, 1191.84 examples/s]
Tokenizing train dataset:  18%|█▊        | 320742/1807941 [04:33<20:42, 1197.01 examples/s]
Tokenizing train dataset:  18%|█▊        | 320878/1807941 [04:33<20:02, 1236.47 examples/s]
Tokenizing train dataset:  18%|█▊        | 321062/1807941 [04:33<20:50, 1188.86 examples/s]
Tokenizing train dataset:  18%|█▊        | 321247/1807941 [04:33<20:39, 1199.35 examples/s]
Tokenizing train dataset:  18%|█▊        | 321370/1807941 [04:34<20:32, 1205.94 examples/s]
Tokenizing train dataset:  18%|█▊        | 321503/1807941 [04:34<20:04, 1233.91 examples/s]
Tokenizing train dataset:  18%|█▊        | 321683/1807941 [04:34<20:20, 1218.10 examples/s]
Tokenizing train dataset:  18%|█▊        | 321861/1807941 [04:34<20:34, 1203.94 examples/s]
Tokenizing train dataset:  18%|█▊        | 321989/1807941 [04:34<20:20, 1217.42 examples/s]
Tokenizing train dataset:  18%|█▊        | 322144/1807941 [04:34<21:29, 1152.40 examples/s]
Tokenizing train dataset:  18%|█▊        | 322271/1807941 [04:34<21:00, 1179.06 examples/s]
Tokenizing train dataset:  18%|█▊        | 322396/1807941 [04:34<20:44, 1193.93 examples/s]
Tokenizing train dataset:  18%|█▊        | 322518/1807941 [04:35<20:37, 1200.21 examples/s]
Tokenizing train dataset:  18%|█▊        | 322643/1807941 [04:35<20:26, 1210.84 examples/s]
Tokenizing train dataset:  18%|█▊        | 322770/1807941 [04:35<20:13, 1223.74 examples/s]
Tokenizing train dataset:  18%|█▊        | 322958/1807941 [04:35<20:06, 1230.57 examples/s]
Tokenizing train dataset:  18%|█▊        | 323109/1807941 [04:35<21:46, 1136.33 examples/s]
Tokenizing train dataset:  18%|█▊        | 323245/1807941 [04:35<23:22, 1058.30 examples/s]
Tokenizing train dataset:  18%|█▊        | 323371/1807941 [04:35<22:22, 1105.69 examples/s]
Tokenizing train dataset:  18%|█▊        | 323492/1807941 [04:35<21:51, 1131.71 examples/s]
Tokenizing train dataset:  18%|█▊        | 323653/1807941 [04:36<19:41, 1256.10 examples/s]
Tokenizing train dataset:  18%|█▊        | 323815/1807941 [04:36<18:17, 1352.68 examples/s]
Tokenizing train dataset:  18%|█▊        | 323986/1807941 [04:36<19:28, 1269.70 examples/s]
Tokenizing train dataset:  18%|█▊        | 324151/1807941 [04:36<20:27, 1208.81 examples/s]
Tokenizing train dataset:  18%|█▊        | 324337/1807941 [04:36<20:20, 1215.98 examples/s]
Tokenizing train dataset:  18%|█▊        | 324465/1807941 [04:36<20:08, 1227.03 examples/s]
Tokenizing train dataset:  18%|█▊        | 324599/1807941 [04:36<19:43, 1253.36 examples/s]
Tokenizing train dataset:  18%|█▊        | 324782/1807941 [04:36<19:57, 1238.26 examples/s]
Tokenizing train dataset:  18%|█▊        | 324964/1807941 [04:37<20:10, 1225.33 examples/s]
Tokenizing train dataset:  18%|█▊        | 325150/1807941 [04:37<20:08, 1227.29 examples/s]
Tokenizing train dataset:  18%|█▊        | 325321/1807941 [04:37<18:27, 1339.20 examples/s]
Tokenizing train dataset:  18%|█▊        | 325478/1807941 [04:37<17:43, 1394.45 examples/s]
Tokenizing train dataset:  18%|█▊        | 325664/1807941 [04:37<18:27, 1338.29 examples/s]
Tokenizing train dataset:  18%|█▊        | 325847/1807941 [04:37<19:02, 1296.81 examples/s]
Tokenizing train dataset:  18%|█▊        | 326019/1807941 [04:37<19:50, 1244.41 examples/s]
Tokenizing train dataset:  18%|█▊        | 326151/1807941 [04:37<19:36, 1259.00 examples/s]
Tokenizing train dataset:  18%|█▊        | 326281/1807941 [04:38<19:30, 1265.68 examples/s]
Tokenizing train dataset:  18%|█▊        | 326473/1807941 [04:38<19:30, 1266.07 examples/s]
Tokenizing train dataset:  18%|█▊        | 326608/1807941 [04:38<19:12, 1284.90 examples/s]
Tokenizing train dataset:  18%|█▊        | 326749/1807941 [04:38<18:48, 1312.18 examples/s]
Tokenizing train dataset:  18%|█▊        | 326896/1807941 [04:38<18:15, 1351.79 examples/s]
Tokenizing train dataset:  18%|█▊        | 327074/1807941 [04:38<19:09, 1288.73 examples/s]
Tokenizing train dataset:  18%|█▊        | 327210/1807941 [04:38<18:54, 1304.63 examples/s]
Tokenizing train dataset:  18%|█▊        | 327413/1807941 [04:38<18:41, 1319.67 examples/s]
Tokenizing train dataset:  18%|█▊        | 327548/1807941 [04:39<18:36, 1325.68 examples/s]
Tokenizing train dataset:  18%|█▊        | 327692/1807941 [04:39<18:13, 1353.99 examples/s]
Tokenizing train dataset:  18%|█▊        | 327840/1807941 [04:39<20:17, 1216.11 examples/s]
Tokenizing train dataset:  18%|█▊        | 328000/1807941 [04:39<21:47, 1132.24 examples/s]
Tokenizing train dataset:  18%|█▊        | 328127/1807941 [04:39<21:15, 1160.50 examples/s]
Tokenizing train dataset:  18%|█▊        | 328293/1807941 [04:39<21:41, 1136.91 examples/s]
Tokenizing train dataset:  18%|█▊        | 328422/1807941 [04:39<21:03, 1171.30 examples/s]
Tokenizing train dataset:  18%|█▊        | 328546/1807941 [04:39<20:47, 1185.53 examples/s]
Tokenizing train dataset:  18%|█▊        | 328706/1807941 [04:40<21:42, 1135.28 examples/s]
Tokenizing train dataset:  18%|█▊        | 328826/1807941 [04:40<21:26, 1149.61 examples/s]
Tokenizing train dataset:  18%|█▊        | 328949/1807941 [04:40<21:07, 1166.61 examples/s]
Tokenizing train dataset:  18%|█▊        | 329123/1807941 [04:40<22:14, 1108.11 examples/s]
Tokenizing train dataset:  18%|█▊        | 329244/1807941 [04:40<21:49, 1129.48 examples/s]
Tokenizing train dataset:  18%|█▊        | 329369/1807941 [04:40<21:15, 1159.43 examples/s]
Tokenizing train dataset:  18%|█▊        | 329493/1807941 [04:40<20:54, 1178.69 examples/s]
Tokenizing train dataset:  18%|█▊        | 329679/1807941 [04:40<20:36, 1195.90 examples/s]
Tokenizing train dataset:  18%|█▊        | 329808/1807941 [04:40<20:13, 1218.26 examples/s]
Tokenizing train dataset:  18%|█▊        | 329934/1807941 [04:41<20:02, 1229.05 examples/s]
Tokenizing train dataset:  18%|█▊        | 330098/1807941 [04:41<50:49, 484.68 examples/s] 
Tokenizing train dataset:  18%|█▊        | 330202/1807941 [04:41<44:29, 553.57 examples/s]
Tokenizing train dataset:  18%|█▊        | 330312/1807941 [04:42<38:43, 636.05 examples/s]
Tokenizing train dataset:  18%|█▊        | 330435/1807941 [04:42<33:16, 740.20 examples/s]
Tokenizing train dataset:  18%|█▊        | 330561/1807941 [04:42<29:08, 844.85 examples/s]
Tokenizing train dataset:  18%|█▊        | 330693/1807941 [04:42<25:54, 950.33 examples/s]
Tokenizing train dataset:  18%|█▊        | 330827/1807941 [04:42<23:36, 1042.82 examples/s]
Tokenizing train dataset:  18%|█▊        | 331000/1807941 [04:42<23:47, 1034.48 examples/s]
Tokenizing train dataset:  18%|█▊        | 331127/1807941 [04:42<22:37, 1088.14 examples/s]
Tokenizing train dataset:  18%|█▊        | 331261/1807941 [04:42<21:24, 1149.20 examples/s]
Tokenizing train dataset:  18%|█▊        | 331450/1807941 [04:42<20:46, 1184.33 examples/s]
Tokenizing train dataset:  18%|█▊        | 331585/1807941 [04:43<20:07, 1222.45 examples/s]
Tokenizing train dataset:  18%|█▊        | 331716/1807941 [04:43<19:48, 1241.63 examples/s]
Tokenizing train dataset:  18%|█▊        | 331894/1807941 [04:43<20:13, 1216.49 examples/s]
Tokenizing train dataset:  18%|█▊        | 332031/1807941 [04:43<22:08, 1110.83 examples/s]
Tokenizing train dataset:  18%|█▊        | 332156/1807941 [04:43<21:32, 1141.77 examples/s]
Tokenizing train dataset:  18%|█▊        | 332290/1807941 [04:43<20:37, 1192.18 examples/s]
Tokenizing train dataset:  18%|█▊        | 332431/1807941 [04:43<19:41, 1248.52 examples/s]
Tokenizing train dataset:  18%|█▊        | 332581/1807941 [04:43<18:41, 1315.06 examples/s]
Tokenizing train dataset:  18%|█▊        | 332715/1807941 [04:43<18:38, 1318.89 examples/s]
Tokenizing train dataset:  18%|█▊        | 332900/1807941 [04:44<19:09, 1283.17 examples/s]
Tokenizing train dataset:  18%|█▊        | 333065/1807941 [04:44<20:14, 1214.37 examples/s]
Tokenizing train dataset:  18%|█▊        | 333201/1807941 [04:44<19:41, 1248.33 examples/s]
Tokenizing train dataset:  18%|█▊        | 333366/1807941 [04:44<20:37, 1191.70 examples/s]
Tokenizing train dataset:  18%|█▊        | 333500/1807941 [04:44<20:02, 1226.65 examples/s]
Tokenizing train dataset:  18%|█▊        | 333630/1807941 [04:44<19:44, 1244.50 examples/s]
Tokenizing train dataset:  18%|█▊        | 333762/1807941 [04:44<19:26, 1264.18 examples/s]
Tokenizing train dataset:  18%|█▊        | 333908/1807941 [04:44<18:39, 1317.15 examples/s]
Tokenizing train dataset:  18%|█▊        | 334085/1807941 [04:45<19:28, 1260.89 examples/s]
Tokenizing train dataset:  18%|█▊        | 334221/1807941 [04:45<19:06, 1285.68 examples/s]
Tokenizing train dataset:  18%|█▊        | 334416/1807941 [04:45<19:04, 1287.28 examples/s]
Tokenizing train dataset:  19%|█▊        | 334603/1807941 [04:45<19:19, 1270.37 examples/s]
Tokenizing train dataset:  19%|█▊        | 334734/1807941 [04:45<19:15, 1275.22 examples/s]
Tokenizing train dataset:  19%|█▊        | 334913/1807941 [04:45<19:47, 1240.89 examples/s]
Tokenizing train dataset:  19%|█▊        | 335065/1807941 [04:45<21:09, 1160.39 examples/s]
Tokenizing train dataset:  19%|█▊        | 335249/1807941 [04:46<20:50, 1177.48 examples/s]
Tokenizing train dataset:  19%|█▊        | 335374/1807941 [04:46<20:34, 1192.75 examples/s]
Tokenizing train dataset:  19%|█▊        | 335556/1807941 [04:46<20:30, 1197.01 examples/s]
Tokenizing train dataset:  19%|█▊        | 335696/1807941 [04:46<19:44, 1242.78 examples/s]
Tokenizing train dataset:  19%|█▊        | 335838/1807941 [04:46<19:06, 1284.44 examples/s]
Tokenizing train dataset:  19%|█▊        | 336000/1807941 [04:46<20:37, 1189.39 examples/s]
Tokenizing train dataset:  19%|█▊        | 336173/1807941 [04:46<20:53, 1174.10 examples/s]
Tokenizing train dataset:  19%|█▊        | 336305/1807941 [04:46<20:18, 1207.26 examples/s]
Tokenizing train dataset:  19%|█▊        | 336479/1807941 [04:47<20:38, 1187.71 examples/s]
Tokenizing train dataset:  19%|█▊        | 336606/1807941 [04:47<20:21, 1204.24 examples/s]
Tokenizing train dataset:  19%|█▊        | 336771/1807941 [04:47<21:04, 1163.79 examples/s]
Tokenizing train dataset:  19%|█▊        | 336923/1807941 [04:47<22:05, 1109.46 examples/s]
Tokenizing train dataset:  19%|█▊        | 337071/1807941 [04:47<22:57, 1067.74 examples/s]
Tokenizing train dataset:  19%|█▊        | 337191/1807941 [04:47<22:22, 1095.31 examples/s]
Tokenizing train dataset:  19%|█▊        | 337324/1807941 [04:47<21:16, 1152.21 examples/s]
Tokenizing train dataset:  19%|█▊        | 337450/1807941 [04:47<20:48, 1177.94 examples/s]
Tokenizing train dataset:  19%|█▊        | 337576/1807941 [04:47<20:25, 1199.40 examples/s]
Tokenizing train dataset:  19%|█▊        | 337781/1807941 [04:48<19:30, 1255.89 examples/s]
Tokenizing train dataset:  19%|█▊        | 337910/1807941 [04:48<19:22, 1264.30 examples/s]
Tokenizing train dataset:  19%|█▊        | 338063/1807941 [04:48<20:55, 1170.74 examples/s]
Tokenizing train dataset:  19%|█▊        | 338188/1807941 [04:48<20:37, 1188.14 examples/s]
Tokenizing train dataset:  19%|█▊        | 338310/1807941 [04:48<20:31, 1193.35 examples/s]
Tokenizing train dataset:  19%|█▊        | 338435/1807941 [04:48<20:19, 1205.44 examples/s]
Tokenizing train dataset:  19%|█▊        | 338601/1807941 [04:48<21:05, 1161.22 examples/s]
Tokenizing train dataset:  19%|█▊        | 338762/1807941 [04:49<21:41, 1129.19 examples/s]
Tokenizing train dataset:  19%|█▊        | 338884/1807941 [04:49<21:16, 1150.96 examples/s]
Tokenizing train dataset:  19%|█▉        | 339030/1807941 [04:49<22:32, 1085.90 examples/s]
Tokenizing train dataset:  19%|█▉        | 339164/1807941 [04:49<21:22, 1145.04 examples/s]
Tokenizing train dataset:  19%|█▉        | 339288/1807941 [04:49<20:59, 1166.43 examples/s]
Tokenizing train dataset:  19%|█▉        | 339411/1807941 [04:49<20:43, 1181.03 examples/s]
Tokenizing train dataset:  19%|█▉        | 339541/1807941 [04:49<20:12, 1210.83 examples/s]
Tokenizing train dataset:  19%|█▉        | 339673/1807941 [04:49<19:46, 1237.61 examples/s]
Tokenizing train dataset:  19%|█▉        | 339807/1807941 [04:49<19:21, 1264.02 examples/s]
Tokenizing train dataset:  19%|█▉        | 339992/1807941 [04:50<19:37, 1246.21 examples/s]
Tokenizing train dataset:  19%|█▉        | 340156/1807941 [04:50<20:33, 1189.51 examples/s]
Tokenizing train dataset:  19%|█▉        | 340340/1807941 [04:50<20:24, 1198.65 examples/s]
Tokenizing train dataset:  19%|█▉        | 340478/1807941 [04:50<19:44, 1238.59 examples/s]
Tokenizing train dataset:  19%|█▉        | 340659/1807941 [04:50<19:58, 1224.72 examples/s]
Tokenizing train dataset:  19%|█▉        | 340797/1807941 [04:50<19:24, 1259.43 examples/s]
Tokenizing train dataset:  19%|█▉        | 340929/1807941 [04:50<19:11, 1273.66 examples/s]
Tokenizing train dataset:  19%|█▉        | 341083/1807941 [04:50<18:52, 1295.06 examples/s]
Tokenizing train dataset:  19%|█▉        | 341229/1807941 [04:50<18:18, 1335.47 examples/s]
Tokenizing train dataset:  19%|█▉        | 341421/1807941 [04:51<18:37, 1312.57 examples/s]
Tokenizing train dataset:  19%|█▉        | 341558/1807941 [04:51<18:27, 1323.95 examples/s]
Tokenizing train dataset:  19%|█▉        | 341732/1807941 [04:51<19:19, 1264.39 examples/s]
Tokenizing train dataset:  19%|█▉        | 341891/1807941 [04:51<20:29, 1192.28 examples/s]
Tokenizing train dataset:  19%|█▉        | 342063/1807941 [04:51<21:34, 1132.07 examples/s]
Tokenizing train dataset:  19%|█▉        | 342207/1807941 [04:51<20:21, 1199.78 examples/s]
Tokenizing train dataset:  19%|█▉        | 342339/1807941 [04:51<19:55, 1225.63 examples/s]
Tokenizing train dataset:  19%|█▉        | 342467/1807941 [04:52<19:44, 1237.59 examples/s]
Tokenizing train dataset:  19%|█▉        | 342595/1807941 [04:52<19:35, 1246.95 examples/s]
Tokenizing train dataset:  19%|█▉        | 342724/1807941 [04:52<19:26, 1255.74 examples/s]
Tokenizing train dataset:  19%|█▉        | 342914/1807941 [04:52<19:28, 1254.16 examples/s]
Tokenizing train dataset:  19%|█▉        | 343072/1807941 [04:52<20:42, 1179.04 examples/s]
Tokenizing train dataset:  19%|█▉        | 343194/1807941 [04:52<20:32, 1188.20 examples/s]
Tokenizing train dataset:  19%|█▉        | 343316/1807941 [04:52<20:26, 1194.22 examples/s]
Tokenizing train dataset:  19%|█▉        | 343494/1807941 [04:52<20:36, 1184.22 examples/s]
Tokenizing train dataset:  19%|█▉        | 343620/1807941 [04:52<20:20, 1199.40 examples/s]
Tokenizing train dataset:  19%|█▉        | 343747/1807941 [04:53<20:05, 1214.44 examples/s]
Tokenizing train dataset:  19%|█▉        | 343878/1807941 [04:53<19:41, 1238.69 examples/s]
Tokenizing train dataset:  19%|█▉        | 344068/1807941 [04:53<20:23, 1196.14 examples/s]
Tokenizing train dataset:  19%|█▉        | 344200/1807941 [04:53<19:56, 1223.38 examples/s]
Tokenizing train dataset:  19%|█▉        | 344330/1807941 [04:53<19:39, 1241.07 examples/s]
Tokenizing train dataset:  19%|█▉        | 344461/1807941 [04:53<19:24, 1256.68 examples/s]
Tokenizing train dataset:  19%|█▉        | 344597/1807941 [04:53<19:00, 1282.71 examples/s]
Tokenizing train dataset:  19%|█▉        | 344776/1807941 [04:53<19:34, 1245.72 examples/s]
Tokenizing train dataset:  19%|█▉        | 344932/1807941 [04:54<20:48, 1171.39 examples/s]
Tokenizing train dataset:  19%|█▉        | 345076/1807941 [04:54<22:11, 1098.79 examples/s]
Tokenizing train dataset:  19%|█▉        | 345238/1807941 [04:54<22:20, 1091.12 examples/s]
Tokenizing train dataset:  19%|█▉        | 345361/1807941 [04:54<21:44, 1121.39 examples/s]
Tokenizing train dataset:  19%|█▉        | 345536/1807941 [04:54<21:34, 1129.63 examples/s]
Tokenizing train dataset:  19%|█▉        | 345653/1807941 [04:54<21:25, 1137.65 examples/s]
Tokenizing train dataset:  19%|█▉        | 345772/1807941 [04:54<21:13, 1148.53 examples/s]
Tokenizing train dataset:  19%|█▉        | 345907/1807941 [04:54<20:17, 1200.80 examples/s]
Tokenizing train dataset:  19%|█▉        | 346070/1807941 [04:55<21:04, 1156.20 examples/s]
Tokenizing train dataset:  19%|█▉        | 346203/1807941 [04:55<20:19, 1198.93 examples/s]
Tokenizing train dataset:  19%|█▉        | 346338/1807941 [04:55<19:40, 1237.82 examples/s]
Tokenizing train dataset:  19%|█▉        | 346479/1807941 [04:55<19:00, 1280.95 examples/s]
Tokenizing train dataset:  19%|█▉        | 346615/1807941 [04:55<18:45, 1298.09 examples/s]
Tokenizing train dataset:  19%|█▉        | 346752/1807941 [04:55<18:30, 1315.68 examples/s]
Tokenizing train dataset:  19%|█▉        | 346893/1807941 [04:55<18:09, 1340.67 examples/s]
Tokenizing train dataset:  19%|█▉        | 347067/1807941 [04:55<19:11, 1268.38 examples/s]
Tokenizing train dataset:  19%|█▉        | 347228/1807941 [04:55<17:57, 1355.26 examples/s]
Tokenizing train dataset:  19%|█▉        | 347430/1807941 [04:56<18:02, 1349.36 examples/s]
Tokenizing train dataset:  19%|█▉        | 347631/1807941 [04:56<18:08, 1341.40 examples/s]
Tokenizing train dataset:  19%|█▉        | 347769/1807941 [04:56<18:03, 1348.23 examples/s]
Tokenizing train dataset:  19%|█▉        | 347955/1807941 [04:56<18:37, 1306.60 examples/s]
Tokenizing train dataset:  19%|█▉        | 348121/1807941 [04:56<20:14, 1201.78 examples/s]
Tokenizing train dataset:  19%|█▉        | 348249/1807941 [04:56<19:58, 1217.88 examples/s]
Tokenizing train dataset:  19%|█▉        | 348376/1807941 [04:56<19:47, 1229.34 examples/s]
Tokenizing train dataset:  19%|█▉        | 348501/1807941 [04:56<19:45, 1231.08 examples/s]
Tokenizing train dataset:  19%|█▉        | 348628/1807941 [04:57<19:38, 1238.10 examples/s]
Tokenizing train dataset:  19%|█▉        | 348762/1807941 [04:57<19:14, 1263.55 examples/s]
Tokenizing train dataset:  19%|█▉        | 348939/1807941 [04:57<19:47, 1228.36 examples/s]
Tokenizing train dataset:  19%|█▉        | 349115/1807941 [04:57<21:24, 1135.31 examples/s]
Tokenizing train dataset:  19%|█▉        | 349247/1807941 [04:57<20:39, 1176.81 examples/s]
Tokenizing train dataset:  19%|█▉        | 349424/1807941 [04:57<20:41, 1175.24 examples/s]
Tokenizing train dataset:  19%|█▉        | 349587/1807941 [04:57<21:12, 1145.90 examples/s]
Tokenizing train dataset:  19%|█▉        | 349704/1807941 [04:57<21:07, 1150.38 examples/s]
Tokenizing train dataset:  19%|█▉        | 349832/1807941 [04:58<20:34, 1181.08 examples/s]
Tokenizing train dataset:  19%|█▉        | 350000/1807941 [04:58<22:25, 1083.61 examples/s]
Tokenizing train dataset:  19%|█▉        | 350118/1807941 [04:58<21:58, 1105.67 examples/s]
Tokenizing train dataset:  19%|█▉        | 350249/1807941 [04:58<21:03, 1153.84 examples/s]
Tokenizing train dataset:  19%|█▉        | 350370/1807941 [04:58<20:50, 1165.14 examples/s]
Tokenizing train dataset:  19%|█▉        | 350531/1807941 [04:58<21:33, 1126.65 examples/s]
Tokenizing train dataset:  19%|█▉        | 350648/1807941 [04:58<21:23, 1135.26 examples/s]
Tokenizing train dataset:  19%|█▉        | 350812/1807941 [04:58<21:46, 1115.21 examples/s]
Tokenizing train dataset:  19%|█▉        | 350944/1807941 [04:59<20:52, 1163.33 examples/s]
Tokenizing train dataset:  19%|█▉        | 351064/1807941 [04:59<21:42, 1118.87 examples/s]
Tokenizing train dataset:  19%|█▉        | 351180/1807941 [04:59<21:32, 1127.43 examples/s]
Tokenizing train dataset:  19%|█▉        | 351335/1807941 [04:59<19:33, 1241.72 examples/s]
Tokenizing train dataset:  19%|█▉        | 351531/1807941 [04:59<19:15, 1260.18 examples/s]
Tokenizing train dataset:  19%|█▉        | 351722/1807941 [04:59<19:13, 1262.16 examples/s]
Tokenizing train dataset:  19%|█▉        | 351852/1807941 [04:59<19:08, 1267.39 examples/s]
Tokenizing train dataset:  19%|█▉        | 352007/1807941 [04:59<20:29, 1184.14 examples/s]
Tokenizing train dataset:  19%|█▉        | 352183/1807941 [05:00<20:36, 1177.48 examples/s]
Tokenizing train dataset:  19%|█▉        | 352313/1807941 [05:00<20:07, 1205.33 examples/s]
Tokenizing train dataset:  19%|█▉        | 352442/1807941 [05:00<19:48, 1224.91 examples/s]
Tokenizing train dataset:  20%|█▉        | 352569/1807941 [05:00<19:37, 1235.46 examples/s]
Tokenizing train dataset:  20%|█▉        | 352697/1807941 [05:00<19:28, 1245.58 examples/s]
Tokenizing train dataset:  20%|█▉        | 352893/1807941 [05:00<19:08, 1266.78 examples/s]
Tokenizing train dataset:  20%|█▉        | 353070/1807941 [05:00<19:38, 1234.56 examples/s]
Tokenizing train dataset:  20%|█▉        | 353260/1807941 [05:00<19:31, 1241.87 examples/s]
Tokenizing train dataset:  20%|█▉        | 353398/1807941 [05:01<19:03, 1272.18 examples/s]
Tokenizing train dataset:  20%|█▉        | 353560/1807941 [05:01<20:10, 1201.76 examples/s]
Tokenizing train dataset:  20%|█▉        | 353721/1807941 [05:01<20:58, 1155.81 examples/s]
Tokenizing train dataset:  20%|█▉        | 353856/1807941 [05:01<20:11, 1200.47 examples/s]
Tokenizing train dataset:  20%|█▉        | 354000/1807941 [05:01<20:23, 1188.74 examples/s]
Tokenizing train dataset:  20%|█▉        | 354126/1807941 [05:01<20:07, 1203.55 examples/s]
Tokenizing train dataset:  20%|█▉        | 354311/1807941 [05:01<19:58, 1212.81 examples/s]
Tokenizing train dataset:  20%|█▉        | 354448/1807941 [05:01<19:23, 1249.71 examples/s]
Tokenizing train dataset:  20%|█▉        | 354581/1807941 [05:02<19:09, 1264.63 examples/s]
Tokenizing train dataset:  20%|█▉        | 354770/1807941 [05:02<19:14, 1258.36 examples/s]
Tokenizing train dataset:  20%|█▉        | 354959/1807941 [05:02<19:18, 1254.13 examples/s]
Tokenizing train dataset:  20%|█▉        | 355136/1807941 [05:02<20:06, 1203.71 examples/s]
Tokenizing train dataset:  20%|█▉        | 355260/1807941 [05:02<20:01, 1208.91 examples/s]
Tokenizing train dataset:  20%|█▉        | 355391/1807941 [05:02<19:39, 1231.94 examples/s]
Tokenizing train dataset:  20%|█▉        | 355521/1807941 [05:02<19:25, 1245.86 examples/s]
Tokenizing train dataset:  20%|█▉        | 355700/1807941 [05:02<19:48, 1222.09 examples/s]
Tokenizing train dataset:  20%|█▉        | 355879/1807941 [05:03<20:02, 1207.85 examples/s]
Tokenizing train dataset:  20%|█▉        | 356059/1807941 [05:03<21:07, 1145.60 examples/s]
Tokenizing train dataset:  20%|█▉        | 356180/1807941 [05:03<20:53, 1157.80 examples/s]
Tokenizing train dataset:  20%|█▉        | 356312/1807941 [05:03<20:15, 1193.96 examples/s]
Tokenizing train dataset:  20%|█▉        | 356435/1807941 [05:03<20:07, 1202.54 examples/s]
Tokenizing train dataset:  20%|█▉        | 356570/1807941 [05:03<19:30, 1239.94 examples/s]
Tokenizing train dataset:  20%|█▉        | 356727/1807941 [05:03<20:45, 1164.74 examples/s]
Tokenizing train dataset:  20%|█▉        | 356888/1807941 [05:03<21:23, 1130.32 examples/s]
Tokenizing train dataset:  20%|█▉        | 357058/1807941 [05:04<22:35, 1070.29 examples/s]
Tokenizing train dataset:  20%|█▉        | 357182/1807941 [05:04<21:51, 1105.97 examples/s]
Tokenizing train dataset:  20%|█▉        | 357301/1807941 [05:04<21:28, 1125.71 examples/s]
Tokenizing train dataset:  20%|█▉        | 357425/1807941 [05:04<20:57, 1153.17 examples/s]
Tokenizing train dataset:  20%|█▉        | 357576/1807941 [05:04<21:58, 1099.66 examples/s]
Tokenizing train dataset:  20%|█▉        | 357695/1807941 [05:04<21:33, 1121.60 examples/s]
Tokenizing train dataset:  20%|█▉        | 357819/1807941 [05:04<21:01, 1149.33 examples/s]
Tokenizing train dataset:  20%|█▉        | 357942/1807941 [05:04<20:38, 1170.51 examples/s]
Tokenizing train dataset:  20%|█▉        | 358119/1807941 [05:05<21:41, 1113.75 examples/s]
Tokenizing train dataset:  20%|█▉        | 358240/1807941 [05:05<21:15, 1136.14 examples/s]
Tokenizing train dataset:  20%|█▉        | 358359/1807941 [05:05<21:05, 1145.16 examples/s]
Tokenizing train dataset:  20%|█▉        | 358488/1807941 [05:05<20:26, 1181.92 examples/s]
Tokenizing train dataset:  20%|█▉        | 358637/1807941 [05:05<19:05, 1265.40 examples/s]
Tokenizing train dataset:  20%|█▉        | 358823/1807941 [05:05<19:17, 1251.80 examples/s]
Tokenizing train dataset:  20%|█▉        | 358951/1807941 [05:05<19:13, 1255.68 examples/s]
Tokenizing train dataset:  20%|█▉        | 359116/1807941 [05:05<20:58, 1151.60 examples/s]
Tokenizing train dataset:  20%|█▉        | 359245/1807941 [05:05<20:24, 1182.80 examples/s]
Tokenizing train dataset:  20%|█▉        | 359368/1807941 [05:06<20:14, 1192.39 examples/s]
Tokenizing train dataset:  20%|█▉        | 359544/1807941 [05:06<20:24, 1182.87 examples/s]
Tokenizing train dataset:  20%|█▉        | 359670/1807941 [05:06<20:08, 1198.01 examples/s]
Tokenizing train dataset:  20%|█▉        | 359838/1807941 [05:06<20:40, 1167.27 examples/s]
Tokenizing train dataset:  20%|█▉        | 360000/1807941 [05:06<22:06, 1091.14 examples/s]
Tokenizing train dataset:  20%|█▉        | 360128/1807941 [05:06<21:18, 1132.26 examples/s]
Tokenizing train dataset:  20%|█▉        | 360249/1807941 [05:06<21:00, 1148.59 examples/s]
Tokenizing train dataset:  20%|█▉        | 360368/1807941 [05:06<20:55, 1153.33 examples/s]
Tokenizing train dataset:  20%|█▉        | 360495/1807941 [05:07<20:24, 1182.37 examples/s]
Tokenizing train dataset:  20%|█▉        | 360678/1807941 [05:07<20:12, 1193.46 examples/s]
Tokenizing train dataset:  20%|█▉        | 360859/1807941 [05:07<20:10, 1195.30 examples/s]
Tokenizing train dataset:  20%|█▉        | 360995/1807941 [05:07<19:32, 1234.30 examples/s]
Tokenizing train dataset:  20%|█▉        | 361141/1807941 [05:07<21:06, 1142.50 examples/s]
Tokenizing train dataset:  20%|█▉        | 361287/1807941 [05:07<19:48, 1217.28 examples/s]
Tokenizing train dataset:  20%|█▉        | 361412/1807941 [05:07<19:43, 1222.30 examples/s]
Tokenizing train dataset:  20%|██        | 361595/1807941 [05:07<19:48, 1216.82 examples/s]
Tokenizing train dataset:  20%|██        | 361770/1807941 [05:08<20:07, 1198.12 examples/s]
Tokenizing train dataset:  20%|██        | 361898/1807941 [05:08<19:50, 1214.94 examples/s]
Tokenizing train dataset:  20%|██        | 362059/1807941 [05:08<20:54, 1152.99 examples/s]
Tokenizing train dataset:  20%|██        | 362178/1807941 [05:08<20:46, 1160.12 examples/s]
Tokenizing train dataset:  20%|██        | 362307/1807941 [05:08<20:13, 1191.70 examples/s]
Tokenizing train dataset:  20%|██        | 362437/1807941 [05:08<19:48, 1216.64 examples/s]
Tokenizing train dataset:  20%|██        | 362563/1807941 [05:08<19:36, 1228.15 examples/s]
Tokenizing train dataset:  20%|██        | 362692/1807941 [05:08<19:22, 1242.71 examples/s]
Tokenizing train dataset:  20%|██        | 362822/1807941 [05:08<19:11, 1254.71 examples/s]
Tokenizing train dataset:  20%|██        | 362954/1807941 [05:09<18:57, 1270.06 examples/s]
Tokenizing train dataset:  20%|██        | 363134/1807941 [05:09<19:36, 1227.83 examples/s]
Tokenizing train dataset:  20%|██        | 363272/1807941 [05:09<19:02, 1264.82 examples/s]
Tokenizing train dataset:  20%|██        | 363439/1807941 [05:09<19:55, 1207.85 examples/s]
Tokenizing train dataset:  20%|██        | 363573/1807941 [05:09<19:25, 1239.19 examples/s]
Tokenizing train dataset:  20%|██        | 363699/1807941 [05:09<19:22, 1241.88 examples/s]
Tokenizing train dataset:  20%|██        | 363833/1807941 [05:09<19:00, 1266.46 examples/s]
Tokenizing train dataset:  20%|██        | 363961/1807941 [05:09<18:59, 1267.00 examples/s]
Tokenizing train dataset:  20%|██        | 364119/1807941 [05:10<20:52, 1153.14 examples/s]
Tokenizing train dataset:  20%|██        | 364245/1807941 [05:10<20:24, 1179.26 examples/s]
Tokenizing train dataset:  20%|██        | 364427/1807941 [05:10<20:14, 1188.95 examples/s]
Tokenizing train dataset:  20%|██        | 364565/1807941 [05:10<19:28, 1235.49 examples/s]
Tokenizing train dataset:  20%|██        | 364701/1807941 [05:10<19:01, 1264.03 examples/s]
Tokenizing train dataset:  20%|██        | 364898/1807941 [05:10<18:48, 1278.89 examples/s]
Tokenizing train dataset:  20%|██        | 365058/1807941 [05:10<20:13, 1188.79 examples/s]
Tokenizing train dataset:  20%|██        | 365179/1807941 [05:10<20:12, 1190.10 examples/s]
Tokenizing train dataset:  20%|██        | 365310/1807941 [05:11<19:45, 1217.03 examples/s]
Tokenizing train dataset:  20%|██        | 365440/1807941 [05:11<19:28, 1234.79 examples/s]
Tokenizing train dataset:  20%|██        | 365620/1807941 [05:11<17:17, 1389.60 examples/s]
Tokenizing train dataset:  20%|██        | 365782/1807941 [05:11<16:32, 1452.76 examples/s]
Tokenizing train dataset:  20%|██        | 365979/1807941 [05:11<17:13, 1395.19 examples/s]
Tokenizing train dataset:  20%|██        | 366129/1807941 [05:11<19:10, 1253.11 examples/s]
Tokenizing train dataset:  20%|██        | 366267/1807941 [05:11<18:45, 1281.23 examples/s]
Tokenizing train dataset:  20%|██        | 366463/1807941 [05:11<18:39, 1288.15 examples/s]
Tokenizing train dataset:  20%|██        | 366598/1807941 [05:11<18:28, 1300.30 examples/s]
Tokenizing train dataset:  20%|██        | 366809/1807941 [05:12<18:00, 1333.17 examples/s]
Tokenizing train dataset:  20%|██        | 366946/1807941 [05:12<17:55, 1339.46 examples/s]
Tokenizing train dataset:  20%|██        | 367099/1807941 [05:12<20:34, 1167.50 examples/s]
Tokenizing train dataset:  20%|██        | 367227/1807941 [05:12<20:08, 1192.42 examples/s]
Tokenizing train dataset:  20%|██        | 367361/1807941 [05:12<19:32, 1228.34 examples/s]
Tokenizing train dataset:  20%|██        | 367501/1807941 [05:12<18:58, 1265.72 examples/s]
Tokenizing train dataset:  20%|██        | 367636/1807941 [05:12<18:39, 1286.32 examples/s]
Tokenizing train dataset:  20%|██        | 367769/1807941 [05:12<18:31, 1295.12 examples/s]
Tokenizing train dataset:  20%|██        | 367909/1807941 [05:13<18:07, 1324.42 examples/s]
Tokenizing train dataset:  20%|██        | 368076/1807941 [05:13<19:19, 1241.77 examples/s]
Tokenizing train dataset:  20%|██        | 368223/1807941 [05:13<18:28, 1298.46 examples/s]
Tokenizing train dataset:  20%|██        | 368363/1807941 [05:13<18:06, 1325.26 examples/s]
Tokenizing train dataset:  20%|██        | 368564/1807941 [05:13<18:06, 1324.61 examples/s]
Tokenizing train dataset:  20%|██        | 368706/1807941 [05:13<17:48, 1347.11 examples/s]
Tokenizing train dataset:  20%|██        | 368860/1807941 [05:13<17:10, 1396.66 examples/s]
Tokenizing train dataset:  20%|██        | 369034/1807941 [05:13<18:25, 1301.61 examples/s]
Tokenizing train dataset:  20%|██        | 369228/1807941 [05:14<18:31, 1294.75 examples/s]
Tokenizing train dataset:  20%|██        | 369425/1807941 [05:14<18:32, 1293.02 examples/s]
Tokenizing train dataset:  20%|██        | 369559/1807941 [05:14<18:25, 1301.14 examples/s]
Tokenizing train dataset:  20%|██        | 369752/1807941 [05:14<18:33, 1291.65 examples/s]
Tokenizing train dataset:  20%|██        | 369887/1807941 [05:14<18:23, 1303.04 examples/s]
Tokenizing train dataset:  20%|██        | 370063/1807941 [05:14<19:18, 1241.04 examples/s]
Tokenizing train dataset:  20%|██        | 370255/1807941 [05:14<19:09, 1251.12 examples/s]
Tokenizing train dataset:  20%|██        | 370382/1807941 [05:14<19:07, 1252.82 examples/s]
Tokenizing train dataset:  20%|██        | 370563/1807941 [05:15<19:29, 1229.18 examples/s]
Tokenizing train dataset:  21%|██        | 370690/1807941 [05:15<19:23, 1235.72 examples/s]
Tokenizing train dataset:  21%|██        | 370875/1807941 [05:15<19:25, 1233.24 examples/s]
Tokenizing train dataset:  21%|██        | 371010/1807941 [05:15<21:21, 1121.63 examples/s]
Tokenizing train dataset:  21%|██        | 371137/1807941 [05:15<20:47, 1152.08 examples/s]
Tokenizing train dataset:  21%|██        | 371266/1807941 [05:15<20:12, 1184.70 examples/s]
Tokenizing train dataset:  21%|██        | 371390/1807941 [05:15<19:59, 1197.43 examples/s]
Tokenizing train dataset:  21%|██        | 371515/1807941 [05:15<19:48, 1208.83 examples/s]
Tokenizing train dataset:  21%|██        | 371642/1807941 [05:16<19:34, 1222.98 examples/s]
Tokenizing train dataset:  21%|██        | 371768/1807941 [05:16<19:26, 1231.50 examples/s]
Tokenizing train dataset:  21%|██        | 371955/1807941 [05:16<19:24, 1233.46 examples/s]
Tokenizing train dataset:  21%|██        | 372124/1807941 [05:16<20:25, 1171.68 examples/s]
Tokenizing train dataset:  21%|██        | 372304/1807941 [05:16<20:19, 1177.70 examples/s]
Tokenizing train dataset:  21%|██        | 372436/1807941 [05:16<19:45, 1210.53 examples/s]
Tokenizing train dataset:  21%|██        | 372624/1807941 [05:16<19:35, 1221.40 examples/s]
Tokenizing train dataset:  21%|██        | 372751/1807941 [05:16<19:25, 1231.18 examples/s]
Tokenizing train dataset:  21%|██        | 372882/1807941 [05:17<19:07, 1251.00 examples/s]
Tokenizing train dataset:  21%|██        | 373065/1807941 [05:17<20:11, 1184.79 examples/s]
Tokenizing train dataset:  21%|██        | 373253/1807941 [05:17<19:52, 1203.36 examples/s]
Tokenizing train dataset:  21%|██        | 373382/1807941 [05:17<19:33, 1222.81 examples/s]
Tokenizing train dataset:  21%|██        | 373514/1807941 [05:17<19:12, 1244.42 examples/s]
Tokenizing train dataset:  21%|██        | 373650/1807941 [05:17<18:50, 1268.96 examples/s]
Tokenizing train dataset:  21%|██        | 373838/1807941 [05:17<18:57, 1260.38 examples/s]
Tokenizing train dataset:  21%|██        | 374000/1807941 [05:17<20:04, 1190.66 examples/s]
Tokenizing train dataset:  21%|██        | 374124/1807941 [05:18<19:55, 1199.75 examples/s]
Tokenizing train dataset:  21%|██        | 374247/1807941 [05:18<19:53, 1200.95 examples/s]
Tokenizing train dataset:  21%|██        | 374387/1807941 [05:18<19:03, 1253.60 examples/s]
Tokenizing train dataset:  21%|██        | 374517/1807941 [05:18<18:53, 1264.84 examples/s]
Tokenizing train dataset:  21%|██        | 374703/1807941 [05:18<19:04, 1252.66 examples/s]
Tokenizing train dataset:  21%|██        | 374856/1807941 [05:18<20:25, 1169.63 examples/s]
Tokenizing train dataset:  21%|██        | 374985/1807941 [05:18<19:56, 1197.33 examples/s]
Tokenizing train dataset:  21%|██        | 375134/1807941 [05:18<21:15, 1123.69 examples/s]
Tokenizing train dataset:  21%|██        | 375257/1807941 [05:19<20:49, 1146.18 examples/s]
Tokenizing train dataset:  21%|██        | 375430/1807941 [05:19<20:49, 1146.54 examples/s]
Tokenizing train dataset:  21%|██        | 375554/1807941 [05:19<20:27, 1167.28 examples/s]
Tokenizing train dataset:  21%|██        | 375679/1807941 [05:19<20:08, 1184.87 examples/s]
Tokenizing train dataset:  21%|██        | 375810/1807941 [05:19<19:39, 1214.48 examples/s]
Tokenizing train dataset:  21%|██        | 375935/1807941 [05:19<19:35, 1218.36 examples/s]
Tokenizing train dataset:  21%|██        | 376060/1807941 [05:19<20:42, 1151.99 examples/s]
Tokenizing train dataset:  21%|██        | 376184/1807941 [05:19<20:17, 1175.77 examples/s]
Tokenizing train dataset:  21%|██        | 376307/1807941 [05:19<20:06, 1186.72 examples/s]
Tokenizing train dataset:  21%|██        | 376441/1807941 [05:19<19:25, 1228.27 examples/s]
Tokenizing train dataset:  21%|██        | 376569/1807941 [05:20<19:14, 1239.57 examples/s]
Tokenizing train dataset:  21%|██        | 376750/1807941 [05:20<19:29, 1223.60 examples/s]
Tokenizing train dataset:  21%|██        | 376929/1807941 [05:20<19:44, 1207.98 examples/s]
Tokenizing train dataset:  21%|██        | 377076/1807941 [05:20<21:06, 1129.77 examples/s]
Tokenizing train dataset:  21%|██        | 377198/1807941 [05:20<20:44, 1149.99 examples/s]
Tokenizing train dataset:  21%|██        | 377323/1807941 [05:20<20:20, 1171.82 examples/s]
Tokenizing train dataset:  21%|██        | 377495/1807941 [05:20<20:33, 1159.67 examples/s]
Tokenizing train dataset:  21%|██        | 377615/1807941 [05:20<20:22, 1169.55 examples/s]
Tokenizing train dataset:  21%|██        | 377735/1807941 [05:21<20:19, 1173.05 examples/s]
Tokenizing train dataset:  21%|██        | 377866/1807941 [05:21<19:46, 1205.21 examples/s]
Tokenizing train dataset:  21%|██        | 377999/1807941 [05:21<19:17, 1235.78 examples/s]
Tokenizing train dataset:  21%|██        | 378153/1807941 [05:21<20:36, 1156.07 examples/s]
Tokenizing train dataset:  21%|██        | 378336/1807941 [05:21<20:17, 1173.81 examples/s]
Tokenizing train dataset:  21%|██        | 378462/1807941 [05:21<19:59, 1191.41 examples/s]
Tokenizing train dataset:  21%|██        | 378585/1807941 [05:21<19:52, 1198.58 examples/s]
Tokenizing train dataset:  21%|██        | 378748/1807941 [05:21<20:38, 1154.09 examples/s]
Tokenizing train dataset:  21%|██        | 378866/1807941 [05:22<20:33, 1158.50 examples/s]
Tokenizing train dataset:  21%|██        | 378985/1807941 [05:22<20:27, 1164.17 examples/s]
Tokenizing train dataset:  21%|██        | 379131/1807941 [05:22<21:49, 1090.80 examples/s]
Tokenizing train dataset:  21%|██        | 379256/1807941 [05:22<21:06, 1127.97 examples/s]
Tokenizing train dataset:  21%|██        | 379382/1807941 [05:22<20:31, 1160.44 examples/s]
Tokenizing train dataset:  21%|██        | 379501/1807941 [05:22<20:25, 1165.67 examples/s]
Tokenizing train dataset:  21%|██        | 379624/1807941 [05:22<20:09, 1181.10 examples/s]
Tokenizing train dataset:  21%|██        | 379748/1807941 [05:22<19:54, 1195.54 examples/s]
Tokenizing train dataset:  21%|██        | 379870/1807941 [05:22<19:49, 1200.96 examples/s]
Tokenizing train dataset:  21%|██        | 380012/1807941 [05:23<21:38, 1099.31 examples/s]
Tokenizing train dataset:  21%|██        | 380130/1807941 [05:23<21:18, 1116.84 examples/s]
Tokenizing train dataset:  21%|██        | 380255/1807941 [05:23<20:39, 1152.08 examples/s]
Tokenizing train dataset:  21%|██        | 380375/1807941 [05:23<20:29, 1161.48 examples/s]
Tokenizing train dataset:  21%|██        | 380557/1807941 [05:23<20:10, 1179.08 examples/s]
Tokenizing train dataset:  21%|██        | 380712/1807941 [05:23<18:39, 1275.18 examples/s]
Tokenizing train dataset:  21%|██        | 380893/1807941 [05:23<19:04, 1246.45 examples/s]
Tokenizing train dataset:  21%|██        | 381060/1807941 [05:23<20:26, 1163.53 examples/s]
Tokenizing train dataset:  21%|██        | 381241/1807941 [05:24<20:18, 1170.61 examples/s]
Tokenizing train dataset:  21%|██        | 381372/1807941 [05:24<19:49, 1199.38 examples/s]
Tokenizing train dataset:  21%|██        | 381496/1807941 [05:24<19:41, 1207.75 examples/s]
Tokenizing train dataset:  21%|██        | 381628/1807941 [05:24<19:16, 1233.46 examples/s]
Tokenizing train dataset:  21%|██        | 381810/1807941 [05:24<19:26, 1222.19 examples/s]
Tokenizing train dataset:  21%|██        | 381994/1807941 [05:24<19:30, 1218.73 examples/s]
Tokenizing train dataset:  21%|██        | 382168/1807941 [05:24<19:52, 1195.23 examples/s]
Tokenizing train dataset:  21%|██        | 382325/1807941 [05:24<18:34, 1279.47 examples/s]
Tokenizing train dataset:  21%|██        | 382456/1807941 [05:25<18:27, 1286.99 examples/s]
Tokenizing train dataset:  21%|██        | 382639/1807941 [05:25<18:50, 1261.17 examples/s]
Tokenizing train dataset:  21%|██        | 382768/1807941 [05:25<18:46, 1265.54 examples/s]
Tokenizing train dataset:  21%|██        | 382938/1807941 [05:25<19:31, 1216.49 examples/s]
Tokenizing train dataset:  21%|██        | 383098/1807941 [05:25<20:23, 1164.34 examples/s]
Tokenizing train dataset:  21%|██        | 383264/1807941 [05:25<20:47, 1142.24 examples/s]
Tokenizing train dataset:  21%|██        | 383382/1807941 [05:25<20:39, 1149.05 examples/s]
Tokenizing train dataset:  21%|██        | 383507/1807941 [05:25<20:14, 1172.39 examples/s]
Tokenizing train dataset:  21%|██        | 383636/1807941 [05:26<19:44, 1201.95 examples/s]
Tokenizing train dataset:  21%|██        | 383817/1807941 [05:26<19:47, 1199.35 examples/s]
Tokenizing train dataset:  21%|██        | 383943/1807941 [05:26<19:33, 1213.15 examples/s]
Tokenizing train dataset:  21%|██        | 384088/1807941 [05:26<21:07, 1123.21 examples/s]
Tokenizing train dataset:  21%|██▏       | 384206/1807941 [05:26<20:52, 1136.83 examples/s]
Tokenizing train dataset:  21%|██▏       | 384325/1807941 [05:26<20:39, 1148.89 examples/s]
Tokenizing train dataset:  21%|██▏       | 384451/1807941 [05:26<20:09, 1176.88 examples/s]
Tokenizing train dataset:  21%|██▏       | 384577/1807941 [05:26<19:49, 1197.09 examples/s]
Tokenizing train dataset:  21%|██▏       | 384706/1807941 [05:26<19:25, 1221.26 examples/s]
Tokenizing train dataset:  21%|██▏       | 384835/1807941 [05:27<19:09, 1237.64 examples/s]
Tokenizing train dataset:  21%|██▏       | 384979/1807941 [05:27<18:20, 1293.20 examples/s]
Tokenizing train dataset:  21%|██▏       | 385163/1807941 [05:27<18:47, 1262.43 examples/s]
Tokenizing train dataset:  21%|██▏       | 385308/1807941 [05:27<18:07, 1307.92 examples/s]
Tokenizing train dataset:  21%|██▏       | 385444/1807941 [05:27<18:00, 1316.82 examples/s]
Tokenizing train dataset:  21%|██▏       | 385580/1807941 [05:27<17:52, 1325.83 examples/s]
Tokenizing train dataset:  21%|██▏       | 385720/1807941 [05:27<17:38, 1344.05 examples/s]
Tokenizing train dataset:  21%|██▏       | 385922/1807941 [05:27<17:41, 1339.72 examples/s]
Tokenizing train dataset:  21%|██▏       | 386084/1807941 [05:28<19:02, 1244.23 examples/s]
Tokenizing train dataset:  21%|██▏       | 386225/1807941 [05:28<18:27, 1284.22 examples/s]
Tokenizing train dataset:  21%|██▏       | 386425/1807941 [05:28<18:13, 1299.55 examples/s]
Tokenizing train dataset:  21%|██▏       | 386561/1807941 [05:28<18:03, 1312.40 examples/s]
Tokenizing train dataset:  21%|██▏       | 386698/1807941 [05:28<17:55, 1321.25 examples/s]
Tokenizing train dataset:  21%|██▏       | 386896/1807941 [05:28<17:58, 1317.18 examples/s]
Tokenizing train dataset:  21%|██▏       | 387069/1807941 [05:28<18:52, 1254.95 examples/s]
Tokenizing train dataset:  21%|██▏       | 387254/1807941 [05:28<19:01, 1245.04 examples/s]
Tokenizing train dataset:  21%|██▏       | 387380/1807941 [05:29<19:02, 1243.85 examples/s]
Tokenizing train dataset:  21%|██▏       | 387525/1807941 [05:29<18:18, 1293.27 examples/s]
Tokenizing train dataset:  21%|██▏       | 387658/1807941 [05:29<18:11, 1300.75 examples/s]
Tokenizing train dataset:  21%|██▏       | 387846/1807941 [05:29<18:31, 1277.62 examples/s]
Tokenizing train dataset:  21%|██▏       | 388011/1807941 [05:29<19:27, 1216.26 examples/s]
Tokenizing train dataset:  21%|██▏       | 388142/1807941 [05:29<19:07, 1237.27 examples/s]
Tokenizing train dataset:  21%|██▏       | 388269/1807941 [05:29<19:01, 1243.45 examples/s]
Tokenizing train dataset:  21%|██▏       | 388399/1807941 [05:29<18:51, 1254.92 examples/s]
Tokenizing train dataset:  21%|██▏       | 388535/1807941 [05:29<18:29, 1279.10 examples/s]
Tokenizing train dataset:  21%|██▏       | 388685/1807941 [05:30<17:38, 1341.19 examples/s]
Tokenizing train dataset:  22%|██▏       | 388826/1807941 [05:30<17:26, 1356.64 examples/s]
Tokenizing train dataset:  22%|██▏       | 388963/1807941 [05:30<17:25, 1357.40 examples/s]
Tokenizing train dataset:  22%|██▏       | 389164/1807941 [05:30<17:33, 1346.24 examples/s]
Tokenizing train dataset:  22%|██▏       | 389380/1807941 [05:30<17:11, 1375.70 examples/s]
Tokenizing train dataset:  22%|██▏       | 389525/1807941 [05:30<16:59, 1391.14 examples/s]
Tokenizing train dataset:  22%|██▏       | 389674/1807941 [05:30<16:43, 1412.98 examples/s]
Tokenizing train dataset:  22%|██▏       | 389889/1807941 [05:30<16:43, 1413.23 examples/s]
Tokenizing train dataset:  22%|██▏       | 390073/1807941 [05:31<17:33, 1346.47 examples/s]
Tokenizing train dataset:  22%|██▏       | 390212/1807941 [05:31<17:26, 1354.30 examples/s]
Tokenizing train dataset:  22%|██▏       | 390419/1807941 [05:31<17:24, 1357.60 examples/s]
Tokenizing train dataset:  22%|██▏       | 390616/1807941 [05:31<17:38, 1339.28 examples/s]
Tokenizing train dataset:  22%|██▏       | 390758/1807941 [05:31<17:25, 1355.47 examples/s]
Tokenizing train dataset:  22%|██▏       | 390960/1807941 [05:31<17:30, 1349.09 examples/s]
Tokenizing train dataset:  22%|██▏       | 391127/1807941 [05:31<18:42, 1262.04 examples/s]
Tokenizing train dataset:  22%|██▏       | 391312/1807941 [05:32<18:52, 1250.47 examples/s]
Tokenizing train dataset:  22%|██▏       | 391443/1807941 [05:32<18:42, 1261.78 examples/s]
Tokenizing train dataset:  22%|██▏       | 391633/1807941 [05:32<18:42, 1262.28 examples/s]
Tokenizing train dataset:  22%|██▏       | 391762/1807941 [05:32<18:38, 1266.28 examples/s]
Tokenizing train dataset:  22%|██▏       | 391946/1807941 [05:32<18:53, 1249.47 examples/s]
Tokenizing train dataset:  22%|██▏       | 392129/1807941 [05:32<19:50, 1189.33 examples/s]
Tokenizing train dataset:  22%|██▏       | 392258/1807941 [05:32<19:30, 1209.95 examples/s]
Tokenizing train dataset:  22%|██▏       | 392382/1807941 [05:32<19:26, 1213.33 examples/s]
Tokenizing train dataset:  22%|██▏       | 392564/1807941 [05:33<19:28, 1210.95 examples/s]
Tokenizing train dataset:  22%|██▏       | 392740/1807941 [05:33<19:45, 1193.62 examples/s]
Tokenizing train dataset:  22%|██▏       | 392866/1807941 [05:33<19:32, 1207.34 examples/s]
Tokenizing train dataset:  22%|██▏       | 392996/1807941 [05:33<19:11, 1228.31 examples/s]
Tokenizing train dataset:  22%|██▏       | 393151/1807941 [05:33<20:25, 1154.67 examples/s]
Tokenizing train dataset:  22%|██▏       | 393271/1807941 [05:33<20:16, 1162.81 examples/s]
Tokenizing train dataset:  22%|██▏       | 393400/1807941 [05:33<19:45, 1192.92 examples/s]
Tokenizing train dataset:  22%|██▏       | 393521/1807941 [05:33<19:42, 1195.76 examples/s]
Tokenizing train dataset:  22%|██▏       | 393701/1807941 [05:33<19:42, 1195.75 examples/s]
Tokenizing train dataset:  22%|██▏       | 393825/1807941 [05:34<19:35, 1203.26 examples/s]
Tokenizing train dataset:  22%|██▏       | 394000/1807941 [05:34<20:42, 1137.54 examples/s]
Tokenizing train dataset:  22%|██▏       | 394136/1807941 [05:34<19:48, 1189.75 examples/s]
Tokenizing train dataset:  22%|██▏       | 394264/1807941 [05:34<19:27, 1210.46 examples/s]
Tokenizing train dataset:  22%|██▏       | 394398/1807941 [05:34<18:59, 1240.94 examples/s]
Tokenizing train dataset:  22%|██▏       | 394589/1807941 [05:34<18:49, 1250.94 examples/s]
Tokenizing train dataset:  22%|██▏       | 394720/1807941 [05:34<18:39, 1262.28 examples/s]
Tokenizing train dataset:  22%|██▏       | 394902/1807941 [05:34<18:58, 1241.19 examples/s]
Tokenizing train dataset:  22%|██▏       | 395060/1807941 [05:35<20:10, 1167.13 examples/s]
Tokenizing train dataset:  22%|██▏       | 395233/1807941 [05:35<18:06, 1299.76 examples/s]
Tokenizing train dataset:  22%|██▏       | 395428/1807941 [05:35<16:04, 1464.15 examples/s]
Tokenizing train dataset:  22%|██▏       | 395586/1807941 [05:35<15:47, 1490.32 examples/s]
Tokenizing train dataset:  22%|██▏       | 395774/1807941 [05:35<16:48, 1400.49 examples/s]
Tokenizing train dataset:  22%|██▏       | 395961/1807941 [05:35<17:29, 1345.08 examples/s]
Tokenizing train dataset:  22%|██▏       | 396127/1807941 [05:35<18:37, 1263.48 examples/s]
Tokenizing train dataset:  22%|██▏       | 396310/1807941 [05:36<18:52, 1246.29 examples/s]
Tokenizing train dataset:  22%|██▏       | 396494/1807941 [05:36<19:01, 1236.43 examples/s]
Tokenizing train dataset:  22%|██▏       | 396683/1807941 [05:36<18:59, 1238.49 examples/s]
Tokenizing train dataset:  22%|██▏       | 396857/1807941 [05:36<19:25, 1210.53 examples/s]
Tokenizing train dataset:  22%|██▏       | 396983/1807941 [05:36<19:16, 1219.62 examples/s]
Tokenizing train dataset:  22%|██▏       | 397126/1807941 [05:36<20:48, 1129.94 examples/s]
Tokenizing train dataset:  22%|██▏       | 397300/1807941 [05:36<20:41, 1136.03 examples/s]
Tokenizing train dataset:  22%|██▏       | 397427/1807941 [05:36<20:11, 1164.64 examples/s]
Tokenizing train dataset:  22%|██▏       | 397553/1807941 [05:37<19:49, 1185.33 examples/s]
Tokenizing train dataset:  22%|██▏       | 397683/1807941 [05:37<19:23, 1212.45 examples/s]
Tokenizing train dataset:  22%|██▏       | 397811/1807941 [05:37<19:06, 1229.44 examples/s]
Tokenizing train dataset:  22%|██▏       | 397996/1807941 [05:37<19:08, 1227.35 examples/s]
Tokenizing train dataset:  22%|██▏       | 398148/1807941 [05:37<20:24, 1150.87 examples/s]
Tokenizing train dataset:  22%|██▏       | 398275/1807941 [05:37<19:55, 1178.82 examples/s]
Tokenizing train dataset:  22%|██▏       | 398398/1807941 [05:37<19:45, 1188.84 examples/s]
Tokenizing train dataset:  22%|██▏       | 398572/1807941 [05:37<19:58, 1176.24 examples/s]
Tokenizing train dataset:  22%|██▏       | 398691/1807941 [05:38<19:55, 1178.69 examples/s]
Tokenizing train dataset:  22%|██▏       | 398817/1807941 [05:38<19:37, 1196.60 examples/s]
Tokenizing train dataset:  22%|██▏       | 398940/1807941 [05:38<19:34, 1200.14 examples/s]
Tokenizing train dataset:  22%|██▏       | 399121/1807941 [05:38<20:40, 1135.64 examples/s]
Tokenizing train dataset:  22%|██▏       | 399240/1807941 [05:38<20:27, 1147.28 examples/s]
Tokenizing train dataset:  22%|██▏       | 399365/1807941 [05:38<20:03, 1170.14 examples/s]
Tokenizing train dataset:  22%|██▏       | 399489/1807941 [05:38<19:47, 1185.82 examples/s]
Tokenizing train dataset:  22%|██▏       | 399614/1807941 [05:38<19:31, 1201.81 examples/s]
Tokenizing train dataset:  22%|██▏       | 399745/1807941 [05:38<19:02, 1232.06 examples/s]
Tokenizing train dataset:  22%|██▏       | 399926/1807941 [05:39<19:15, 1218.51 examples/s]
Tokenizing train dataset:  22%|██▏       | 400082/1807941 [05:39<20:20, 1153.14 examples/s]
Tokenizing train dataset:  22%|██▏       | 400208/1807941 [05:39<19:56, 1176.44 examples/s]
Tokenizing train dataset:  22%|██▏       | 400333/1807941 [05:39<19:38, 1194.72 examples/s]
Tokenizing train dataset:  22%|██▏       | 400457/1807941 [05:39<19:28, 1204.27 examples/s]
Tokenizing train dataset:  22%|██▏       | 400590/1807941 [05:39<19:00, 1234.38 examples/s]
Tokenizing train dataset:  22%|██▏       | 400773/1807941 [05:39<19:06, 1227.21 examples/s]
Tokenizing train dataset:  22%|██▏       | 400958/1807941 [05:39<19:07, 1225.62 examples/s]
Tokenizing train dataset:  22%|██▏       | 401117/1807941 [05:40<20:23, 1149.78 examples/s]
Tokenizing train dataset:  22%|██▏       | 401237/1807941 [05:40<20:11, 1161.02 examples/s]
Tokenizing train dataset:  22%|██▏       | 401413/1807941 [05:40<20:12, 1159.97 examples/s]
Tokenizing train dataset:  22%|██▏       | 401542/1807941 [05:40<19:42, 1189.21 examples/s]
Tokenizing train dataset:  22%|██▏       | 401709/1807941 [05:40<20:16, 1156.30 examples/s]
Tokenizing train dataset:  22%|██▏       | 401831/1807941 [05:40<20:03, 1168.37 examples/s]
Tokenizing train dataset:  22%|██▏       | 401950/1807941 [05:40<19:59, 1172.25 examples/s]
Tokenizing train dataset:  22%|██▏       | 402121/1807941 [05:40<21:11, 1105.92 examples/s]
Tokenizing train dataset:  22%|██▏       | 402248/1807941 [05:41<20:28, 1144.30 examples/s]
Tokenizing train dataset:  22%|██▏       | 402367/1807941 [05:41<20:20, 1151.91 examples/s]
Tokenizing train dataset:  22%|██▏       | 402485/1807941 [05:41<20:17, 1154.69 examples/s]
Tokenizing train dataset:  22%|██▏       | 402661/1807941 [05:41<20:12, 1159.19 examples/s]
Tokenizing train dataset:  22%|██▏       | 402787/1807941 [05:41<19:49, 1181.27 examples/s]
Tokenizing train dataset:  22%|██▏       | 402969/1807941 [05:41<19:42, 1188.49 examples/s]
Tokenizing train dataset:  22%|██▏       | 403120/1807941 [05:41<20:49, 1123.92 examples/s]
Tokenizing train dataset:  22%|██▏       | 403238/1807941 [05:41<20:39, 1133.09 examples/s]
Tokenizing train dataset:  22%|██▏       | 403358/1807941 [05:42<20:24, 1146.71 examples/s]
Tokenizing train dataset:  22%|██▏       | 403481/1807941 [05:42<20:04, 1165.72 examples/s]
Tokenizing train dataset:  22%|██▏       | 403605/1807941 [05:42<19:46, 1183.86 examples/s]
Tokenizing train dataset:  22%|██▏       | 403728/1807941 [05:42<19:38, 1191.87 examples/s]
Tokenizing train dataset:  22%|██▏       | 403898/1807941 [05:42<20:03, 1167.05 examples/s]
Tokenizing train dataset:  22%|██▏       | 404037/1807941 [05:42<22:36, 1034.99 examples/s]
Tokenizing train dataset:  22%|██▏       | 404172/1807941 [05:42<23:43, 985.99 examples/s] 
Tokenizing train dataset:  22%|██▏       | 404290/1807941 [05:42<22:46, 1027.16 examples/s]
Tokenizing train dataset:  22%|██▏       | 404403/1807941 [05:43<22:17, 1049.62 examples/s]
Tokenizing train dataset:  22%|██▏       | 404551/1807941 [05:43<22:51, 1023.41 examples/s]
Tokenizing train dataset:  22%|██▏       | 404656/1807941 [05:43<22:50, 1024.12 examples/s]
Tokenizing train dataset:  22%|██▏       | 404770/1807941 [05:43<22:12, 1052.73 examples/s]
Tokenizing train dataset:  22%|██▏       | 404892/1807941 [05:43<21:21, 1094.52 examples/s]
Tokenizing train dataset:  22%|██▏       | 405057/1807941 [05:43<21:52, 1068.55 examples/s]
Tokenizing train dataset:  22%|██▏       | 405174/1807941 [05:43<21:26, 1090.07 examples/s]
Tokenizing train dataset:  22%|██▏       | 405337/1807941 [05:43<21:32, 1085.11 examples/s]
Tokenizing train dataset:  22%|██▏       | 405460/1807941 [05:43<20:51, 1120.62 examples/s]
Tokenizing train dataset:  22%|██▏       | 405582/1807941 [05:44<20:28, 1141.30 examples/s]
Tokenizing train dataset:  22%|██▏       | 405707/1807941 [05:44<20:00, 1167.80 examples/s]
Tokenizing train dataset:  22%|██▏       | 405880/1807941 [05:44<20:08, 1160.24 examples/s]
Tokenizing train dataset:  22%|██▏       | 406000/1807941 [05:44<21:06, 1106.56 examples/s]
Tokenizing train dataset:  22%|██▏       | 406121/1807941 [05:44<20:39, 1131.07 examples/s]
Tokenizing train dataset:  22%|██▏       | 406265/1807941 [05:44<19:15, 1212.57 examples/s]
Tokenizing train dataset:  22%|██▏       | 406398/1807941 [05:44<18:47, 1242.57 examples/s]
Tokenizing train dataset:  22%|██▏       | 406565/1807941 [05:44<19:38, 1189.03 examples/s]
Tokenizing train dataset:  22%|██▏       | 406691/1807941 [05:45<22:01, 1060.40 examples/s]
Tokenizing train dataset:  23%|██▎       | 406850/1807941 [05:45<22:07, 1055.19 examples/s]
Tokenizing train dataset:  23%|██▎       | 406985/1807941 [05:45<20:45, 1124.62 examples/s]
Tokenizing train dataset:  23%|██▎       | 407151/1807941 [05:45<20:57, 1113.83 examples/s]
Tokenizing train dataset:  23%|██▎       | 407278/1807941 [05:45<20:16, 1151.03 examples/s]
Tokenizing train dataset:  23%|██▎       | 407443/1807941 [05:45<20:40, 1129.34 examples/s]
Tokenizing train dataset:  23%|██▎       | 407629/1807941 [05:45<20:03, 1163.25 examples/s]
Tokenizing train dataset:  23%|██▎       | 407760/1807941 [05:45<19:32, 1194.38 examples/s]
Tokenizing train dataset:  23%|██▎       | 407903/1807941 [05:46<21:04, 1107.28 examples/s]
Tokenizing train dataset:  23%|██▎       | 408067/1807941 [05:46<22:03, 1057.37 examples/s]
Tokenizing train dataset:  23%|██▎       | 408185/1807941 [05:46<21:33, 1082.29 examples/s]
Tokenizing train dataset:  23%|██▎       | 408315/1807941 [05:46<20:32, 1135.50 examples/s]
Tokenizing train dataset:  23%|██▎       | 408460/1807941 [05:46<19:10, 1216.78 examples/s]
Tokenizing train dataset:  23%|██▎       | 408644/1807941 [05:46<19:16, 1209.79 examples/s]
Tokenizing train dataset:  23%|██▎       | 408774/1807941 [05:46<21:21, 1091.90 examples/s]
Tokenizing train dataset:  23%|██▎       | 408903/1807941 [05:47<20:29, 1137.70 examples/s]
Tokenizing train dataset:  23%|██▎       | 409061/1807941 [05:47<21:21, 1091.33 examples/s]
Tokenizing train dataset:  23%|██▎       | 409184/1807941 [05:47<20:45, 1123.35 examples/s]
Tokenizing train dataset:  23%|██▎       | 409302/1807941 [05:47<20:33, 1134.20 examples/s]
Tokenizing train dataset:  23%|██▎       | 409421/1807941 [05:47<20:21, 1144.89 examples/s]
Tokenizing train dataset:  23%|██▎       | 409568/1807941 [05:47<18:55, 1231.35 examples/s]
Tokenizing train dataset:  23%|██▎       | 409697/1807941 [05:47<18:41, 1246.84 examples/s]
Tokenizing train dataset:  23%|██▎       | 409829/1807941 [05:47<18:24, 1266.01 examples/s]
Tokenizing train dataset:  23%|██▎       | 410000/1807941 [05:47<20:06, 1158.81 examples/s]
Tokenizing train dataset:  23%|██▎       | 410122/1807941 [05:48<19:52, 1172.49 examples/s]
Tokenizing train dataset:  23%|██▎       | 410248/1807941 [05:48<19:32, 1192.29 examples/s]
Tokenizing train dataset:  23%|██▎       | 410378/1807941 [05:48<19:05, 1220.47 examples/s]
Tokenizing train dataset:  23%|██▎       | 410524/1807941 [05:48<18:05, 1287.05 examples/s]
Tokenizing train dataset:  23%|██▎       | 410677/1807941 [05:48<19:48, 1176.12 examples/s]
Tokenizing train dataset:  23%|██▎       | 410829/1807941 [05:48<23:26, 993.33 examples/s] 
Tokenizing train dataset:  23%|██▎       | 410971/1807941 [05:48<26:27, 880.06 examples/s]
Tokenizing train dataset:  23%|██▎       | 411072/1807941 [05:49<29:48, 780.89 examples/s]
Tokenizing train dataset:  23%|██▎       | 411163/1807941 [05:49<28:51, 806.76 examples/s]
Tokenizing train dataset:  23%|██▎       | 411285/1807941 [05:49<25:54, 898.20 examples/s]
Tokenizing train dataset:  23%|██▎       | 411429/1807941 [05:49<25:31, 911.56 examples/s]
Tokenizing train dataset:  23%|██▎       | 411548/1807941 [05:49<23:53, 973.80 examples/s]
Tokenizing train dataset:  23%|██▎       | 411692/1807941 [05:49<24:09, 963.19 examples/s]
Tokenizing train dataset:  23%|██▎       | 411816/1807941 [05:49<22:36, 1028.88 examples/s]
Tokenizing train dataset:  23%|██▎       | 411945/1807941 [05:49<21:16, 1093.36 examples/s]
Tokenizing train dataset:  23%|██▎       | 412111/1807941 [05:50<22:46, 1021.10 examples/s]
Tokenizing train dataset:  23%|██▎       | 412232/1807941 [05:50<21:52, 1063.57 examples/s]
Tokenizing train dataset:  23%|██▎       | 412356/1807941 [05:50<21:02, 1105.80 examples/s]
Tokenizing train dataset:  23%|██▎       | 412480/1807941 [05:50<20:23, 1140.51 examples/s]
Tokenizing train dataset:  23%|██▎       | 412603/1807941 [05:50<20:00, 1162.37 examples/s]
Tokenizing train dataset:  23%|██▎       | 412724/1807941 [05:50<19:47, 1174.63 examples/s]
Tokenizing train dataset:  23%|██▎       | 412845/1807941 [05:50<19:39, 1182.65 examples/s]
Tokenizing train dataset:  23%|██▎       | 412999/1807941 [05:50<20:49, 1116.38 examples/s]
Tokenizing train dataset:  23%|██▎       | 413118/1807941 [05:51<26:03, 892.09 examples/s] 
Tokenizing train dataset:  23%|██▎       | 413264/1807941 [05:51<25:26, 913.72 examples/s]
Tokenizing train dataset:  23%|██▎       | 413371/1807941 [05:51<24:32, 946.92 examples/s]
Tokenizing train dataset:  23%|██▎       | 413485/1807941 [05:51<23:26, 991.36 examples/s]
Tokenizing train dataset:  23%|██▎       | 413603/1807941 [05:51<22:24, 1036.70 examples/s]
Tokenizing train dataset:  23%|██▎       | 413723/1807941 [05:51<21:33, 1077.84 examples/s]
Tokenizing train dataset:  23%|██▎       | 413894/1807941 [05:51<21:10, 1096.96 examples/s]
Tokenizing train dataset:  23%|██▎       | 414065/1807941 [05:51<21:51, 1062.77 examples/s]
Tokenizing train dataset:  23%|██▎       | 414187/1807941 [05:52<21:08, 1098.69 examples/s]
Tokenizing train dataset:  23%|██▎       | 414333/1807941 [05:52<22:14, 1044.42 examples/s]
Tokenizing train dataset:  23%|██▎       | 414475/1807941 [05:52<22:59, 1010.09 examples/s]
Tokenizing train dataset:  23%|██▎       | 414598/1807941 [05:52<24:38, 942.26 examples/s] 
Tokenizing train dataset:  23%|██▎       | 414708/1807941 [05:52<23:47, 976.13 examples/s]
Tokenizing train dataset:  23%|██▎       | 414832/1807941 [05:52<22:21, 1038.65 examples/s]
Tokenizing train dataset:  23%|██▎       | 414958/1807941 [05:52<21:14, 1093.37 examples/s]
Tokenizing train dataset:  23%|██▎       | 415118/1807941 [05:52<22:22, 1037.61 examples/s]
Tokenizing train dataset:  23%|██▎       | 415252/1807941 [05:53<23:33, 985.26 examples/s] 
Tokenizing train dataset:  23%|██▎       | 415362/1807941 [05:53<22:58, 1010.04 examples/s]
Tokenizing train dataset:  23%|██▎       | 415515/1807941 [05:53<23:00, 1008.68 examples/s]
Tokenizing train dataset:  23%|██▎       | 415640/1807941 [05:53<21:45, 1066.46 examples/s]
Tokenizing train dataset:  23%|██▎       | 415766/1807941 [05:53<20:50, 1113.20 examples/s]
Tokenizing train dataset:  23%|██▎       | 415883/1807941 [05:53<20:37, 1124.91 examples/s]
Tokenizing train dataset:  23%|██▎       | 416000/1807941 [05:53<21:52, 1060.79 examples/s]
Tokenizing train dataset:  23%|██▎       | 416121/1807941 [05:53<21:06, 1098.64 examples/s]
Tokenizing train dataset:  23%|██▎       | 416253/1807941 [05:53<20:03, 1156.35 examples/s]
Tokenizing train dataset:  23%|██▎       | 416381/1807941 [05:54<19:30, 1189.09 examples/s]
Tokenizing train dataset:  23%|██▎       | 416515/1807941 [05:54<21:44, 1066.73 examples/s]
Tokenizing train dataset:  23%|██▎       | 416635/1807941 [05:54<21:03, 1100.86 examples/s]
Tokenizing train dataset:  23%|██▎       | 416751/1807941 [05:54<20:46, 1116.52 examples/s]
Tokenizing train dataset:  23%|██▎       | 416877/1807941 [05:54<23:01, 1006.74 examples/s]
Tokenizing train dataset:  23%|██▎       | 417000/1807941 [05:54<23:15, 996.43 examples/s] 
Tokenizing train dataset:  23%|██▎       | 417154/1807941 [05:54<23:07, 1002.67 examples/s]
Tokenizing train dataset:  23%|██▎       | 417278/1807941 [05:54<21:52, 1059.56 examples/s]
Tokenizing train dataset:  23%|██▎       | 417401/1807941 [05:55<21:04, 1099.70 examples/s]
Tokenizing train dataset:  23%|██▎       | 417549/1807941 [05:55<22:04, 1050.04 examples/s]
Tokenizing train dataset:  23%|██▎       | 417681/1807941 [05:55<20:45, 1115.82 examples/s]
Tokenizing train dataset:  23%|██▎       | 417811/1807941 [05:55<19:56, 1161.60 examples/s]
Tokenizing train dataset:  23%|██▎       | 417932/1807941 [05:55<19:44, 1173.67 examples/s]
Tokenizing train dataset:  23%|██▎       | 418061/1807941 [05:55<21:57, 1054.62 examples/s]
Tokenizing train dataset:  23%|██▎       | 418193/1807941 [05:55<20:39, 1121.31 examples/s]
Tokenizing train dataset:  23%|██▎       | 418321/1807941 [05:55<22:50, 1014.00 examples/s]
Tokenizing train dataset:  23%|██▎       | 418448/1807941 [05:56<21:34, 1073.27 examples/s]
Tokenizing train dataset:  23%|██▎       | 418568/1807941 [05:56<21:00, 1102.37 examples/s]
Tokenizing train dataset:  23%|██▎       | 418685/1807941 [05:56<20:43, 1116.97 examples/s]
Tokenizing train dataset:  23%|██▎       | 418800/1807941 [05:56<20:35, 1123.93 examples/s]
Tokenizing train dataset:  23%|██▎       | 418925/1807941 [05:56<20:01, 1156.17 examples/s]
Tokenizing train dataset:  23%|██▎       | 419082/1807941 [05:56<20:48, 1112.61 examples/s]
Tokenizing train dataset:  23%|██▎       | 419215/1807941 [05:56<19:48, 1168.67 examples/s]
Tokenizing train dataset:  23%|██▎       | 419352/1807941 [05:56<18:58, 1219.26 examples/s]
Tokenizing train dataset:  23%|██▎       | 419529/1807941 [05:56<19:16, 1200.85 examples/s]
Tokenizing train dataset:  23%|██▎       | 419702/1807941 [05:57<19:35, 1181.31 examples/s]
Tokenizing train dataset:  23%|██▎       | 419822/1807941 [05:57<19:33, 1182.60 examples/s]
Tokenizing train dataset:  23%|██▎       | 419951/1807941 [05:57<19:08, 1208.14 examples/s]
Tokenizing train dataset:  23%|██▎       | 420086/1807941 [05:57<21:57, 1053.00 examples/s]
Tokenizing train dataset:  23%|██▎       | 420225/1807941 [05:57<25:29, 907.60 examples/s] 
Tokenizing train dataset:  23%|██▎       | 420343/1807941 [05:57<26:38, 868.25 examples/s]
Tokenizing train dataset:  23%|██▎       | 420448/1807941 [05:57<25:32, 905.45 examples/s]
Tokenizing train dataset:  23%|██▎       | 420567/1807941 [05:58<23:47, 971.62 examples/s]
Tokenizing train dataset:  23%|██▎       | 420689/1807941 [05:58<22:23, 1032.64 examples/s]
Tokenizing train dataset:  23%|██▎       | 420805/1807941 [05:58<21:51, 1057.85 examples/s]
Tokenizing train dataset:  23%|██▎       | 420931/1807941 [05:58<20:49, 1109.68 examples/s]
Tokenizing train dataset:  23%|██▎       | 421082/1807941 [05:58<21:38, 1068.17 examples/s]
Tokenizing train dataset:  23%|██▎       | 421211/1807941 [05:58<20:34, 1123.32 examples/s]
Tokenizing train dataset:  23%|██▎       | 421340/1807941 [05:58<19:50, 1165.16 examples/s]
Tokenizing train dataset:  23%|██▎       | 421483/1807941 [05:58<21:18, 1084.21 examples/s]
Tokenizing train dataset:  23%|██▎       | 421634/1807941 [05:58<21:54, 1054.50 examples/s]
Tokenizing train dataset:  23%|██▎       | 421796/1807941 [05:59<21:50, 1057.59 examples/s]
Tokenizing train dataset:  23%|██▎       | 421916/1807941 [05:59<21:11, 1089.71 examples/s]
Tokenizing train dataset:  23%|██▎       | 422074/1807941 [05:59<21:28, 1075.35 examples/s]
Tokenizing train dataset:  23%|██▎       | 422211/1807941 [05:59<20:09, 1145.68 examples/s]
Tokenizing train dataset:  23%|██▎       | 422331/1807941 [05:59<19:57, 1157.07 examples/s]
Tokenizing train dataset:  23%|██▎       | 422462/1807941 [05:59<19:19, 1195.01 examples/s]
Tokenizing train dataset:  23%|██▎       | 422626/1807941 [05:59<20:00, 1153.87 examples/s]
Tokenizing train dataset:  23%|██▎       | 422790/1807941 [05:59<20:26, 1129.53 examples/s]
Tokenizing train dataset:  23%|██▎       | 422969/1807941 [06:00<20:13, 1141.22 examples/s]
Tokenizing train dataset:  23%|██▎       | 423110/1807941 [06:00<21:38, 1066.15 examples/s]
Tokenizing train dataset:  23%|██▎       | 423229/1807941 [06:00<21:07, 1092.88 examples/s]
Tokenizing train dataset:  23%|██▎       | 423346/1807941 [06:00<20:48, 1108.84 examples/s]
Tokenizing train dataset:  23%|██▎       | 423473/1807941 [06:00<20:09, 1144.89 examples/s]
Tokenizing train dataset:  23%|██▎       | 423642/1807941 [06:00<20:20, 1134.33 examples/s]
Tokenizing train dataset:  23%|██▎       | 423759/1807941 [06:00<20:13, 1140.47 examples/s]
Tokenizing train dataset:  23%|██▎       | 423881/1807941 [06:00<19:54, 1158.98 examples/s]
Tokenizing train dataset:  23%|██▎       | 424000/1807941 [06:01<20:44, 1112.11 examples/s]
Tokenizing train dataset:  23%|██▎       | 424133/1807941 [06:01<19:44, 1168.36 examples/s]
Tokenizing train dataset:  23%|██▎       | 424262/1807941 [06:01<19:14, 1198.57 examples/s]
Tokenizing train dataset:  23%|██▎       | 424388/1807941 [06:01<18:59, 1214.08 examples/s]
Tokenizing train dataset:  23%|██▎       | 424515/1807941 [06:01<18:46, 1227.83 examples/s]
Tokenizing train dataset:  23%|██▎       | 424652/1807941 [06:01<18:10, 1268.37 examples/s]
Tokenizing train dataset:  23%|██▎       | 424782/1807941 [06:01<18:04, 1275.05 examples/s]
Tokenizing train dataset:  24%|██▎       | 424912/1807941 [06:01<18:00, 1279.92 examples/s]
Tokenizing train dataset:  24%|██▎       | 425067/1807941 [06:01<19:30, 1181.29 examples/s]
Tokenizing train dataset:  24%|██▎       | 425242/1807941 [06:02<19:39, 1171.79 examples/s]
Tokenizing train dataset:  24%|██▎       | 425364/1807941 [06:02<24:39, 934.42 examples/s] 
Tokenizing train dataset:  24%|██▎       | 425521/1807941 [06:02<26:20, 874.44 examples/s]
Tokenizing train dataset:  24%|██▎       | 425662/1807941 [06:02<23:26, 982.49 examples/s]
Tokenizing train dataset:  24%|██▎       | 425800/1807941 [06:02<21:31, 1070.40 examples/s]
Tokenizing train dataset:  24%|██▎       | 425917/1807941 [06:02<21:06, 1091.21 examples/s]
Tokenizing train dataset:  24%|██▎       | 426072/1807941 [06:02<21:32, 1069.46 examples/s]
Tokenizing train dataset:  24%|██▎       | 426197/1807941 [06:03<20:41, 1112.72 examples/s]
Tokenizing train dataset:  24%|██▎       | 426318/1807941 [06:03<20:15, 1136.98 examples/s]
Tokenizing train dataset:  24%|██▎       | 426451/1807941 [06:03<19:33, 1177.09 examples/s]
Tokenizing train dataset:  24%|██▎       | 426624/1807941 [06:03<19:44, 1166.01 examples/s]
Tokenizing train dataset:  24%|██▎       | 426754/1807941 [06:03<19:13, 1197.08 examples/s]
Tokenizing train dataset:  24%|██▎       | 426890/1807941 [06:03<18:34, 1238.76 examples/s]
Tokenizing train dataset:  24%|██▎       | 427018/1807941 [06:03<21:04, 1092.48 examples/s]
Tokenizing train dataset:  24%|██▎       | 427153/1807941 [06:03<19:54, 1156.08 examples/s]
Tokenizing train dataset:  24%|██▎       | 427284/1807941 [06:03<19:17, 1193.18 examples/s]
Tokenizing train dataset:  24%|██▎       | 427429/1807941 [06:04<20:48, 1105.43 examples/s]
Tokenizing train dataset:  24%|██▎       | 427556/1807941 [06:04<20:04, 1145.65 examples/s]
Tokenizing train dataset:  24%|██▎       | 427690/1807941 [06:04<19:15, 1194.19 examples/s]
Tokenizing train dataset:  24%|██▎       | 427820/1807941 [06:04<18:50, 1220.61 examples/s]
Tokenizing train dataset:  24%|██▎       | 427963/1807941 [06:04<18:01, 1276.28 examples/s]
Tokenizing train dataset:  24%|██▎       | 428125/1807941 [06:04<19:26, 1182.50 examples/s]
Tokenizing train dataset:  24%|██▎       | 428289/1807941 [06:04<20:06, 1143.68 examples/s]
Tokenizing train dataset:  24%|██▎       | 428424/1807941 [06:04<19:18, 1191.21 examples/s]
Tokenizing train dataset:  24%|██▎       | 428550/1807941 [06:05<19:01, 1208.28 examples/s]
Tokenizing train dataset:  24%|██▎       | 428691/1807941 [06:05<18:15, 1259.10 examples/s]
Tokenizing train dataset:  24%|██▎       | 428834/1807941 [06:05<17:36, 1305.72 examples/s]
Tokenizing train dataset:  24%|██▎       | 428972/1807941 [06:05<17:22, 1322.31 examples/s]
Tokenizing train dataset:  24%|██▎       | 429152/1807941 [06:05<18:02, 1273.68 examples/s]
Tokenizing train dataset:  24%|██▎       | 429293/1807941 [06:05<17:34, 1306.84 examples/s]
Tokenizing train dataset:  24%|██▍       | 429490/1807941 [06:05<17:35, 1305.63 examples/s]
Tokenizing train dataset:  24%|██▍       | 429623/1807941 [06:05<17:31, 1310.22 examples/s]
Tokenizing train dataset:  24%|██▍       | 429760/1807941 [06:05<17:21, 1323.03 examples/s]
Tokenizing train dataset:  24%|██▍       | 429900/1807941 [06:06<19:32, 1175.55 examples/s]
Tokenizing train dataset:  24%|██▍       | 430034/1807941 [06:06<24:09, 950.40 examples/s] 
Tokenizing train dataset:  24%|██▍       | 430149/1807941 [06:06<25:43, 892.63 examples/s]
Tokenizing train dataset:  24%|██▍       | 430272/1807941 [06:06<26:26, 868.52 examples/s]
Tokenizing train dataset:  24%|██▍       | 430393/1807941 [06:06<27:06, 847.14 examples/s]
Tokenizing train dataset:  24%|██▍       | 430482/1807941 [06:06<26:55, 852.67 examples/s]
Tokenizing train dataset:  24%|██▍       | 430598/1807941 [06:06<24:51, 923.56 examples/s]
Tokenizing train dataset:  24%|██▍       | 430733/1807941 [06:07<22:15, 1030.91 examples/s]
Tokenizing train dataset:  24%|██▍       | 430866/1807941 [06:07<20:45, 1105.80 examples/s]
Tokenizing train dataset:  24%|██▍       | 431000/1807941 [06:07<21:05, 1087.83 examples/s]
Tokenizing train dataset:  24%|██▍       | 431167/1807941 [06:07<20:58, 1093.87 examples/s]
Tokenizing train dataset:  24%|██▍       | 431292/1807941 [06:07<20:17, 1130.79 examples/s]
Tokenizing train dataset:  24%|██▍       | 431421/1807941 [06:07<19:35, 1170.58 examples/s]
Tokenizing train dataset:  24%|██▍       | 431589/1807941 [06:07<20:00, 1146.95 examples/s]
Tokenizing train dataset:  24%|██▍       | 431716/1807941 [06:07<19:28, 1177.66 examples/s]
Tokenizing train dataset:  24%|██▍       | 431898/1807941 [06:08<19:21, 1184.81 examples/s]
Tokenizing train dataset:  24%|██▍       | 432063/1807941 [06:08<20:21, 1126.58 examples/s]
Tokenizing train dataset:  24%|██▍       | 432191/1807941 [06:08<19:45, 1160.77 examples/s]
Tokenizing train dataset:  24%|██▍       | 432330/1807941 [06:08<18:51, 1216.17 examples/s]
Tokenizing train dataset:  24%|██▍       | 432471/1807941 [06:08<18:08, 1263.87 examples/s]
Tokenizing train dataset:  24%|██▍       | 432640/1807941 [06:08<18:56, 1209.85 examples/s]
Tokenizing train dataset:  24%|██▍       | 432802/1807941 [06:08<19:44, 1160.87 examples/s]
Tokenizing train dataset:  24%|██▍       | 432934/1807941 [06:08<19:10, 1194.98 examples/s]
Tokenizing train dataset:  24%|██▍       | 433064/1807941 [06:09<19:56, 1149.52 examples/s]
Tokenizing train dataset:  24%|██▍       | 433199/1807941 [06:09<19:07, 1198.41 examples/s]
Tokenizing train dataset:  24%|██▍       | 433343/1807941 [06:09<20:38, 1110.01 examples/s]
Tokenizing train dataset:  24%|██▍       | 433467/1807941 [06:09<20:06, 1139.40 examples/s]
Tokenizing train dataset:  24%|██▍       | 433596/1807941 [06:09<19:27, 1177.64 examples/s]
Tokenizing train dataset:  24%|██▍       | 433729/1807941 [06:09<18:50, 1215.41 examples/s]
Tokenizing train dataset:  24%|██▍       | 433919/1807941 [06:09<18:35, 1231.32 examples/s]
Tokenizing train dataset:  24%|██▍       | 434090/1807941 [06:09<19:08, 1195.75 examples/s]
Tokenizing train dataset:  24%|██▍       | 434216/1807941 [06:09<18:57, 1208.09 examples/s]
Tokenizing train dataset:  24%|██▍       | 434340/1807941 [06:10<18:49, 1215.98 examples/s]
Tokenizing train dataset:  24%|██▍       | 434516/1807941 [06:10<19:07, 1196.98 examples/s]
Tokenizing train dataset:  24%|██▍       | 434683/1807941 [06:10<21:58, 1041.45 examples/s]
Tokenizing train dataset:  24%|██▍       | 434816/1807941 [06:10<20:45, 1102.83 examples/s]
Tokenizing train dataset:  24%|██▍       | 434938/1807941 [06:10<20:15, 1129.38 examples/s]
Tokenizing train dataset:  24%|██▍       | 435062/1807941 [06:10<20:57, 1091.32 examples/s]
Tokenizing train dataset:  24%|██▍       | 435191/1807941 [06:10<20:03, 1140.23 examples/s]
Tokenizing train dataset:  24%|██▍       | 435328/1807941 [06:10<19:04, 1199.44 examples/s]
Tokenizing train dataset:  24%|██▍       | 435452/1807941 [06:11<18:54, 1209.95 examples/s]
Tokenizing train dataset:  24%|██▍       | 435607/1807941 [06:11<20:01, 1142.35 examples/s]
Tokenizing train dataset:  24%|██▍       | 435748/1807941 [06:11<23:53, 956.97 examples/s] 
Tokenizing train dataset:  24%|██▍       | 435869/1807941 [06:11<25:09, 908.91 examples/s]
Tokenizing train dataset:  24%|██▍       | 435972/1807941 [06:11<24:28, 933.99 examples/s]
Tokenizing train dataset:  24%|██▍       | 436071/1807941 [06:11<24:09, 946.39 examples/s]
Tokenizing train dataset:  24%|██▍       | 436203/1807941 [06:11<21:58, 1040.09 examples/s]
Tokenizing train dataset:  24%|██▍       | 436315/1807941 [06:11<21:33, 1060.11 examples/s]
Tokenizing train dataset:  24%|██▍       | 436486/1807941 [06:12<21:03, 1085.79 examples/s]
Tokenizing train dataset:  24%|██▍       | 436613/1807941 [06:12<20:11, 1132.17 examples/s]
Tokenizing train dataset:  24%|██▍       | 436749/1807941 [06:12<19:14, 1188.09 examples/s]
Tokenizing train dataset:  24%|██▍       | 436881/1807941 [06:12<18:41, 1222.38 examples/s]
Tokenizing train dataset:  24%|██▍       | 437059/1807941 [06:12<19:36, 1165.03 examples/s]
Tokenizing train dataset:  24%|██▍       | 437183/1807941 [06:12<19:20, 1180.82 examples/s]
Tokenizing train dataset:  24%|██▍       | 437324/1807941 [06:12<18:25, 1239.27 examples/s]
Tokenizing train dataset:  24%|██▍       | 437484/1807941 [06:12<19:30, 1170.70 examples/s]
Tokenizing train dataset:  24%|██▍       | 437620/1807941 [06:13<18:45, 1217.56 examples/s]
Tokenizing train dataset:  24%|██▍       | 437757/1807941 [06:13<18:09, 1257.53 examples/s]
Tokenizing train dataset:  24%|██▍       | 437904/1807941 [06:13<17:21, 1315.43 examples/s]
Tokenizing train dataset:  24%|██▍       | 438066/1807941 [06:13<18:38, 1224.78 examples/s]
Tokenizing train dataset:  24%|██▍       | 438250/1807941 [06:13<18:42, 1220.17 examples/s]
Tokenizing train dataset:  24%|██▍       | 438376/1807941 [06:13<18:37, 1225.85 examples/s]
Tokenizing train dataset:  24%|██▍       | 438519/1807941 [06:13<17:52, 1276.69 examples/s]
Tokenizing train dataset:  24%|██▍       | 438711/1807941 [06:13<17:55, 1272.70 examples/s]
Tokenizing train dataset:  24%|██▍       | 438840/1807941 [06:14<17:53, 1275.79 examples/s]
Tokenizing train dataset:  24%|██▍       | 438978/1807941 [06:14<17:31, 1301.66 examples/s]
Tokenizing train dataset:  24%|██▍       | 439136/1807941 [06:14<18:56, 1204.26 examples/s]
Tokenizing train dataset:  24%|██▍       | 439261/1807941 [06:14<18:49, 1211.43 examples/s]
Tokenizing train dataset:  24%|██▍       | 439411/1807941 [06:14<22:34, 1010.51 examples/s]
Tokenizing train dataset:  24%|██▍       | 439559/1807941 [06:14<25:16, 902.38 examples/s] 
Tokenizing train dataset:  24%|██▍       | 439690/1807941 [06:14<23:09, 984.41 examples/s]
Tokenizing train dataset:  24%|██▍       | 439820/1807941 [06:14<21:37, 1054.60 examples/s]
Tokenizing train dataset:  24%|██▍       | 439942/1807941 [06:15<20:53, 1091.67 examples/s]
Tokenizing train dataset:  24%|██▍       | 440058/1807941 [06:15<22:09, 1029.09 examples/s]
Tokenizing train dataset:  24%|██▍       | 440188/1807941 [06:15<20:47, 1096.49 examples/s]
Tokenizing train dataset:  24%|██▍       | 440324/1807941 [06:15<19:36, 1162.20 examples/s]
Tokenizing train dataset:  24%|██▍       | 440465/1807941 [06:15<21:11, 1075.23 examples/s]
Tokenizing train dataset:  24%|██▍       | 440599/1807941 [06:15<22:35, 1008.71 examples/s]
Tokenizing train dataset:  24%|██▍       | 440727/1807941 [06:15<21:13, 1073.40 examples/s]
Tokenizing train dataset:  24%|██▍       | 440855/1807941 [06:15<20:14, 1125.82 examples/s]
Tokenizing train dataset:  24%|██▍       | 440992/1807941 [06:16<19:08, 1190.27 examples/s]
Tokenizing train dataset:  24%|██▍       | 441138/1807941 [06:16<23:13, 981.15 examples/s] 
Tokenizing train dataset:  24%|██▍       | 441246/1807941 [06:16<22:45, 1000.96 examples/s]
Tokenizing train dataset:  24%|██▍       | 441368/1807941 [06:16<21:38, 1052.23 examples/s]
Tokenizing train dataset:  24%|██▍       | 441479/1807941 [06:16<21:24, 1063.60 examples/s]
Tokenizing train dataset:  24%|██▍       | 441610/1807941 [06:16<20:11, 1127.82 examples/s]
Tokenizing train dataset:  24%|██▍       | 441735/1807941 [06:16<19:37, 1160.66 examples/s]
Tokenizing train dataset:  24%|██▍       | 441860/1807941 [06:16<19:13, 1184.13 examples/s]
Tokenizing train dataset:  24%|██▍       | 441983/1807941 [06:16<19:04, 1193.62 examples/s]
Tokenizing train dataset:  24%|██▍       | 442130/1807941 [06:17<20:33, 1107.25 examples/s]
Tokenizing train dataset:  24%|██▍       | 442259/1807941 [06:17<22:33, 1009.09 examples/s]
Tokenizing train dataset:  24%|██▍       | 442370/1807941 [06:17<22:03, 1031.61 examples/s]
Tokenizing train dataset:  24%|██▍       | 442499/1807941 [06:17<20:43, 1097.70 examples/s]
Tokenizing train dataset:  24%|██▍       | 442629/1807941 [06:17<19:47, 1149.92 examples/s]
Tokenizing train dataset:  24%|██▍       | 442752/1807941 [06:17<19:26, 1170.21 examples/s]
Tokenizing train dataset:  24%|██▍       | 442876/1807941 [06:17<19:12, 1184.85 examples/s]
Tokenizing train dataset:  25%|██▍       | 443000/1807941 [06:17<20:06, 1131.68 examples/s]
Tokenizing train dataset:  25%|██▍       | 443116/1807941 [06:17<20:00, 1137.13 examples/s]
Tokenizing train dataset:  25%|██▍       | 443250/1807941 [06:18<19:04, 1192.81 examples/s]
Tokenizing train dataset:  25%|██▍       | 443389/1807941 [06:18<18:13, 1247.85 examples/s]
Tokenizing train dataset:  25%|██▍       | 443522/1807941 [06:18<17:56, 1266.98 examples/s]
Tokenizing train dataset:  25%|██▍       | 443652/1807941 [06:18<17:52, 1271.87 examples/s]
Tokenizing train dataset:  25%|██▍       | 443828/1807941 [06:18<18:29, 1229.71 examples/s]
Tokenizing train dataset:  25%|██▍       | 444000/1807941 [06:18<19:48, 1147.53 examples/s]
Tokenizing train dataset:  25%|██▍       | 444134/1807941 [06:18<19:04, 1191.49 examples/s]
Tokenizing train dataset:  25%|██▍       | 444304/1807941 [06:18<19:27, 1168.40 examples/s]
Tokenizing train dataset:  25%|██▍       | 444436/1807941 [06:19<18:52, 1204.08 examples/s]
Tokenizing train dataset:  25%|██▍       | 444576/1807941 [06:19<18:07, 1253.42 examples/s]
Tokenizing train dataset:  25%|██▍       | 444712/1807941 [06:19<17:46, 1277.70 examples/s]
Tokenizing train dataset:  25%|██▍       | 444845/1807941 [06:19<17:37, 1288.44 examples/s]
Tokenizing train dataset:  25%|██▍       | 444990/1807941 [06:19<17:04, 1330.88 examples/s]
Tokenizing train dataset:  25%|██▍       | 445155/1807941 [06:19<18:26, 1232.03 examples/s]
Tokenizing train dataset:  25%|██▍       | 445298/1807941 [06:19<17:43, 1280.98 examples/s]
Tokenizing train dataset:  25%|██▍       | 445492/1807941 [06:19<17:44, 1280.13 examples/s]
Tokenizing train dataset:  25%|██▍       | 445634/1807941 [06:19<17:17, 1313.29 examples/s]
Tokenizing train dataset:  25%|██▍       | 445813/1807941 [06:20<17:54, 1267.23 examples/s]
Tokenizing train dataset:  25%|██▍       | 445969/1807941 [06:20<19:21, 1172.97 examples/s]
Tokenizing train dataset:  25%|██▍       | 446138/1807941 [06:20<22:37, 1003.00 examples/s]
Tokenizing train dataset:  25%|██▍       | 446310/1807941 [06:20<21:51, 1038.41 examples/s]
Tokenizing train dataset:  25%|██▍       | 446462/1807941 [06:20<22:05, 1027.29 examples/s]
Tokenizing train dataset:  25%|██▍       | 446604/1807941 [06:20<22:38, 1002.39 examples/s]
Tokenizing train dataset:  25%|██▍       | 446772/1807941 [06:21<21:56, 1034.19 examples/s]
Tokenizing train dataset:  25%|██▍       | 446918/1807941 [06:21<20:11, 1123.15 examples/s]
Tokenizing train dataset:  25%|██▍       | 447063/1807941 [06:21<21:27, 1057.27 examples/s]
Tokenizing train dataset:  25%|██▍       | 447188/1807941 [06:21<20:38, 1098.48 examples/s]
Tokenizing train dataset:  25%|██▍       | 447318/1807941 [06:21<19:45, 1147.68 examples/s]
Tokenizing train dataset:  25%|██▍       | 447441/1807941 [06:21<19:26, 1166.15 examples/s]
Tokenizing train dataset:  25%|██▍       | 447573/1807941 [06:21<21:23, 1059.94 examples/s]
Tokenizing train dataset:  25%|██▍       | 447728/1807941 [06:21<21:37, 1048.30 examples/s]
Tokenizing train dataset:  25%|██▍       | 447865/1807941 [06:22<22:37, 1001.64 examples/s]
Tokenizing train dataset:  25%|██▍       | 448000/1807941 [06:22<24:44, 915.93 examples/s] 
Tokenizing train dataset:  25%|██▍       | 448107/1807941 [06:22<26:41, 849.30 examples/s]
Tokenizing train dataset:  25%|██▍       | 448231/1807941 [06:22<24:14, 935.14 examples/s]
Tokenizing train dataset:  25%|██▍       | 448357/1807941 [06:22<22:24, 1010.94 examples/s]
Tokenizing train dataset:  25%|██▍       | 448486/1807941 [06:22<20:59, 1079.31 examples/s]
Tokenizing train dataset:  25%|██▍       | 448614/1807941 [06:22<20:01, 1131.71 examples/s]
Tokenizing train dataset:  25%|██▍       | 448759/1807941 [06:22<18:37, 1215.95 examples/s]
Tokenizing train dataset:  25%|██▍       | 448926/1807941 [06:23<19:17, 1173.73 examples/s]
Tokenizing train dataset:  25%|██▍       | 449068/1807941 [06:23<20:44, 1091.94 examples/s]
Tokenizing train dataset:  25%|██▍       | 449190/1807941 [06:23<20:16, 1116.69 examples/s]
Tokenizing train dataset:  25%|██▍       | 449371/1807941 [06:23<19:45, 1145.83 examples/s]
Tokenizing train dataset:  25%|██▍       | 449504/1807941 [06:23<19:02, 1188.98 examples/s]
Tokenizing train dataset:  25%|██▍       | 449642/1807941 [06:23<18:17, 1237.31 examples/s]
Tokenizing train dataset:  25%|██▍       | 449772/1807941 [06:23<18:04, 1252.06 examples/s]
Tokenizing train dataset:  25%|██▍       | 449939/1807941 [06:23<18:54, 1196.73 examples/s]
Tokenizing train dataset:  25%|██▍       | 450085/1807941 [06:24<20:17, 1115.73 examples/s]
Tokenizing train dataset:  25%|██▍       | 450212/1807941 [06:24<19:41, 1149.43 examples/s]
Tokenizing train dataset:  25%|██▍       | 450338/1807941 [06:24<19:13, 1176.59 examples/s]
Tokenizing train dataset:  25%|██▍       | 450459/1807941 [06:24<19:05, 1185.10 examples/s]
Tokenizing train dataset:  25%|██▍       | 450596/1807941 [06:24<18:22, 1230.65 examples/s]
Tokenizing train dataset:  25%|██▍       | 450724/1807941 [06:24<18:13, 1241.05 examples/s]
Tokenizing train dataset:  25%|██▍       | 450851/1807941 [06:24<18:08, 1246.30 examples/s]
Tokenizing train dataset:  25%|██▍       | 451001/1807941 [06:24<19:42, 1147.28 examples/s]
Tokenizing train dataset:  25%|██▍       | 451147/1807941 [06:25<20:52, 1083.44 examples/s]
Tokenizing train dataset:  25%|██▍       | 451266/1807941 [06:25<20:24, 1107.98 examples/s]
Tokenizing train dataset:  25%|██▍       | 451387/1807941 [06:25<19:56, 1133.36 examples/s]
Tokenizing train dataset:  25%|██▍       | 451506/1807941 [06:25<19:47, 1142.62 examples/s]
Tokenizing train dataset:  25%|██▍       | 451628/1807941 [06:25<19:29, 1159.46 examples/s]
Tokenizing train dataset:  25%|██▍       | 451758/1807941 [06:25<18:54, 1195.78 examples/s]
Tokenizing train dataset:  25%|██▍       | 451896/1807941 [06:25<18:07, 1247.16 examples/s]
Tokenizing train dataset:  25%|██▌       | 452066/1807941 [06:25<18:53, 1195.78 examples/s]
Tokenizing train dataset:  25%|██▌       | 452194/1807941 [06:25<18:35, 1215.52 examples/s]
Tokenizing train dataset:  25%|██▌       | 452359/1807941 [06:26<19:19, 1169.00 examples/s]
Tokenizing train dataset:  25%|██▌       | 452482/1807941 [06:26<19:15, 1172.56 examples/s]
Tokenizing train dataset:  25%|██▌       | 452629/1807941 [06:26<20:31, 1100.27 examples/s]
Tokenizing train dataset:  25%|██▌       | 452744/1807941 [06:26<23:07, 977.00 examples/s] 
Tokenizing train dataset:  25%|██▌       | 452869/1807941 [06:26<21:41, 1040.93 examples/s]
Tokenizing train dataset:  25%|██▌       | 453000/1807941 [06:26<21:39, 1042.40 examples/s]
Tokenizing train dataset:  25%|██▌       | 453124/1807941 [06:26<20:42, 1090.33 examples/s]
Tokenizing train dataset:  25%|██▌       | 453252/1807941 [06:26<19:50, 1137.83 examples/s]
Tokenizing train dataset:  25%|██▌       | 453376/1807941 [06:26<19:22, 1164.88 examples/s]
Tokenizing train dataset:  25%|██▌       | 453556/1807941 [06:27<19:14, 1173.63 examples/s]
Tokenizing train dataset:  25%|██▌       | 453683/1807941 [06:27<18:53, 1194.95 examples/s]
Tokenizing train dataset:  25%|██▌       | 453810/1807941 [06:27<21:12, 1064.23 examples/s]
Tokenizing train dataset:  25%|██▌       | 453982/1807941 [06:27<20:42, 1089.88 examples/s]
Tokenizing train dataset:  25%|██▌       | 454149/1807941 [06:27<20:36, 1095.08 examples/s]
Tokenizing train dataset:  25%|██▌       | 454270/1807941 [06:27<20:09, 1119.11 examples/s]
Tokenizing train dataset:  25%|██▌       | 454448/1807941 [06:27<19:47, 1139.80 examples/s]
Tokenizing train dataset:  25%|██▌       | 454578/1807941 [06:28<21:35, 1044.94 examples/s]
Tokenizing train dataset:  25%|██▌       | 454718/1807941 [06:28<24:55, 905.15 examples/s] 
Tokenizing train dataset:  25%|██▌       | 454845/1807941 [06:28<22:59, 981.12 examples/s]
Tokenizing train dataset:  25%|██▌       | 454971/1807941 [06:28<21:35, 1044.66 examples/s]
Tokenizing train dataset:  25%|██▌       | 455101/1807941 [06:28<22:58, 981.08 examples/s] 
Tokenizing train dataset:  25%|██▌       | 455237/1807941 [06:28<26:16, 857.89 examples/s]
Tokenizing train dataset:  25%|██▌       | 455364/1807941 [06:28<23:51, 944.91 examples/s]
Tokenizing train dataset:  25%|██▌       | 455495/1807941 [06:29<21:54, 1029.14 examples/s]
Tokenizing train dataset:  25%|██▌       | 455675/1807941 [06:29<18:28, 1220.44 examples/s]
Tokenizing train dataset:  25%|██▌       | 455861/1807941 [06:29<16:14, 1387.36 examples/s]
Tokenizing train dataset:  25%|██▌       | 456017/1807941 [06:29<17:56, 1255.90 examples/s]
Tokenizing train dataset:  25%|██▌       | 456204/1807941 [06:29<18:03, 1247.09 examples/s]
Tokenizing train dataset:  25%|██▌       | 456378/1807941 [06:29<18:31, 1216.18 examples/s]
Tokenizing train dataset:  25%|██▌       | 456559/1807941 [06:29<18:38, 1208.65 examples/s]
Tokenizing train dataset:  25%|██▌       | 456693/1807941 [06:29<20:18, 1108.70 examples/s]
Tokenizing train dataset:  25%|██▌       | 456822/1807941 [06:30<19:37, 1147.03 examples/s]
Tokenizing train dataset:  25%|██▌       | 456986/1807941 [06:30<19:59, 1126.38 examples/s]
Tokenizing train dataset:  25%|██▌       | 457134/1807941 [06:30<20:54, 1076.94 examples/s]
Tokenizing train dataset:  25%|██▌       | 457244/1807941 [06:30<20:49, 1080.59 examples/s]
Tokenizing train dataset:  25%|██▌       | 457369/1807941 [06:30<20:06, 1119.65 examples/s]
Tokenizing train dataset:  25%|██▌       | 457538/1807941 [06:30<20:05, 1119.93 examples/s]
Tokenizing train dataset:  25%|██▌       | 457677/1807941 [06:30<19:00, 1184.37 examples/s]
Tokenizing train dataset:  25%|██▌       | 457808/1807941 [06:30<18:32, 1213.88 examples/s]
Tokenizing train dataset:  25%|██▌       | 457980/1807941 [06:31<18:59, 1185.18 examples/s]
Tokenizing train dataset:  25%|██▌       | 458124/1807941 [06:31<21:09, 1063.34 examples/s]
Tokenizing train dataset:  25%|██▌       | 458295/1807941 [06:31<20:44, 1084.65 examples/s]
Tokenizing train dataset:  25%|██▌       | 458417/1807941 [06:31<20:09, 1115.34 examples/s]
Tokenizing train dataset:  25%|██▌       | 458579/1807941 [06:31<20:25, 1101.23 examples/s]
Tokenizing train dataset:  25%|██▌       | 458703/1807941 [06:31<19:50, 1133.32 examples/s]
Tokenizing train dataset:  25%|██▌       | 458825/1807941 [06:31<19:30, 1152.39 examples/s]
Tokenizing train dataset:  25%|██▌       | 458969/1807941 [06:32<20:47, 1081.44 examples/s]
Tokenizing train dataset:  25%|██▌       | 459117/1807941 [06:32<21:39, 1038.19 examples/s]
Tokenizing train dataset:  25%|██▌       | 459233/1807941 [06:32<21:07, 1063.74 examples/s]
Tokenizing train dataset:  25%|██▌       | 459359/1807941 [06:32<20:11, 1112.94 examples/s]
Tokenizing train dataset:  25%|██▌       | 459487/1807941 [06:32<19:28, 1153.89 examples/s]
Tokenizing train dataset:  25%|██▌       | 459619/1807941 [06:32<18:46, 1197.31 examples/s]
Tokenizing train dataset:  25%|██▌       | 459744/1807941 [06:32<18:34, 1209.21 examples/s]
Tokenizing train dataset:  25%|██▌       | 459873/1807941 [06:32<18:14, 1231.13 examples/s]
Tokenizing train dataset:  25%|██▌       | 460013/1807941 [06:32<22:45, 987.18 examples/s] 
Tokenizing train dataset:  25%|██▌       | 460122/1807941 [06:33<22:15, 1009.17 examples/s]
Tokenizing train dataset:  25%|██▌       | 460291/1807941 [06:33<21:27, 1046.75 examples/s]
Tokenizing train dataset:  25%|██▌       | 460414/1807941 [06:33<20:37, 1088.88 examples/s]
Tokenizing train dataset:  25%|██▌       | 460532/1807941 [06:33<20:11, 1111.84 examples/s]
Tokenizing train dataset:  25%|██▌       | 460665/1807941 [06:33<19:11, 1169.98 examples/s]
Tokenizing train dataset:  25%|██▌       | 460785/1807941 [06:33<19:05, 1176.10 examples/s]
Tokenizing train dataset:  25%|██▌       | 460908/1807941 [06:33<18:51, 1190.56 examples/s]
Tokenizing train dataset:  26%|██▌       | 461049/1807941 [06:33<20:59, 1069.36 examples/s]
Tokenizing train dataset:  26%|██▌       | 461213/1807941 [06:34<20:53, 1074.17 examples/s]
Tokenizing train dataset:  26%|██▌       | 461334/1807941 [06:34<20:17, 1106.15 examples/s]
Tokenizing train dataset:  26%|██▌       | 461464/1807941 [06:34<19:26, 1154.65 examples/s]
Tokenizing train dataset:  26%|██▌       | 461584/1807941 [06:34<19:17, 1163.59 examples/s]
Tokenizing train dataset:  26%|██▌       | 461769/1807941 [06:34<18:56, 1184.12 examples/s]
Tokenizing train dataset:  26%|██▌       | 461899/1807941 [06:34<18:30, 1211.95 examples/s]
Tokenizing train dataset:  26%|██▌       | 462058/1807941 [06:34<19:49, 1131.40 examples/s]
Tokenizing train dataset:  26%|██▌       | 462229/1807941 [06:34<19:49, 1131.37 examples/s]
Tokenizing train dataset:  26%|██▌       | 462355/1807941 [06:35<19:19, 1160.70 examples/s]
Tokenizing train dataset:  26%|██▌       | 462484/1807941 [06:35<18:49, 1191.16 examples/s]
Tokenizing train dataset:  26%|██▌       | 462607/1807941 [06:35<18:43, 1197.41 examples/s]
Tokenizing train dataset:  26%|██▌       | 462792/1807941 [06:35<18:35, 1205.81 examples/s]
Tokenizing train dataset:  26%|██▌       | 462914/1807941 [06:35<18:33, 1207.59 examples/s]
Tokenizing train dataset:  26%|██▌       | 463087/1807941 [06:35<18:53, 1186.97 examples/s]
Tokenizing train dataset:  26%|██▌       | 463217/1807941 [06:35<18:27, 1214.03 examples/s]
Tokenizing train dataset:  26%|██▌       | 463390/1807941 [06:35<18:49, 1190.84 examples/s]
Tokenizing train dataset:  26%|██▌       | 463562/1807941 [06:36<19:05, 1173.61 examples/s]
Tokenizing train dataset:  26%|██▌       | 463682/1807941 [06:36<19:03, 1175.57 examples/s]
Tokenizing train dataset:  26%|██▌       | 463803/1807941 [06:36<18:55, 1183.55 examples/s]
Tokenizing train dataset:  26%|██▌       | 463929/1807941 [06:36<18:37, 1202.68 examples/s]
Tokenizing train dataset:  26%|██▌       | 464084/1807941 [06:36<19:41, 1137.19 examples/s]
Tokenizing train dataset:  26%|██▌       | 464201/1807941 [06:36<19:36, 1142.17 examples/s]
Tokenizing train dataset:  26%|██▌       | 464374/1807941 [06:36<19:36, 1141.92 examples/s]
Tokenizing train dataset:  26%|██▌       | 464514/1807941 [06:36<18:36, 1203.28 examples/s]
Tokenizing train dataset:  26%|██▌       | 464644/1807941 [06:36<18:17, 1223.62 examples/s]
Tokenizing train dataset:  26%|██▌       | 464769/1807941 [06:37<18:12, 1228.99 examples/s]
Tokenizing train dataset:  26%|██▌       | 464899/1807941 [06:37<17:55, 1248.29 examples/s]
Tokenizing train dataset:  26%|██▌       | 465065/1807941 [06:37<18:46, 1192.22 examples/s]
Tokenizing train dataset:  26%|██▌       | 465191/1807941 [06:37<18:31, 1207.85 examples/s]
Tokenizing train dataset:  26%|██▌       | 465322/1807941 [06:37<18:07, 1234.05 examples/s]
Tokenizing train dataset:  26%|██▌       | 465454/1807941 [06:37<17:50, 1253.53 examples/s]
Tokenizing train dataset:  26%|██▌       | 465624/1807941 [06:37<18:33, 1205.51 examples/s]
Tokenizing train dataset:  26%|██▌       | 465759/1807941 [06:37<18:01, 1241.00 examples/s]
Tokenizing train dataset:  26%|██▌       | 465890/1807941 [06:37<17:49, 1255.17 examples/s]
Tokenizing train dataset:  26%|██▌       | 466071/1807941 [06:38<18:48, 1188.77 examples/s]
Tokenizing train dataset:  26%|██▌       | 466202/1807941 [06:38<18:24, 1215.24 examples/s]
Tokenizing train dataset:  26%|██▌       | 466332/1807941 [06:38<18:05, 1236.02 examples/s]
Tokenizing train dataset:  26%|██▌       | 466506/1807941 [06:38<18:33, 1204.54 examples/s]
Tokenizing train dataset:  26%|██▌       | 466688/1807941 [06:38<18:38, 1199.36 examples/s]
Tokenizing train dataset:  26%|██▌       | 466871/1807941 [06:38<18:34, 1203.45 examples/s]
Tokenizing train dataset:  26%|██▌       | 467000/1807941 [06:38<19:17, 1158.38 examples/s]
Tokenizing train dataset:  26%|██▌       | 467177/1807941 [06:39<19:12, 1163.33 examples/s]
Tokenizing train dataset:  26%|██▌       | 467348/1807941 [06:39<19:24, 1151.34 examples/s]
Tokenizing train dataset:  26%|██▌       | 467469/1807941 [06:39<19:17, 1158.53 examples/s]
Tokenizing train dataset:  26%|██▌       | 467638/1807941 [06:39<19:29, 1145.71 examples/s]
Tokenizing train dataset:  26%|██▌       | 467777/1807941 [06:39<20:48, 1073.03 examples/s]
Tokenizing train dataset:  26%|██▌       | 467893/1807941 [06:39<20:26, 1092.43 examples/s]
Tokenizing train dataset:  26%|██▌       | 468049/1807941 [06:39<21:41, 1029.15 examples/s]
Tokenizing train dataset:  26%|██▌       | 468171/1807941 [06:40<23:16, 959.68 examples/s] 
Tokenizing train dataset:  26%|██▌       | 468282/1807941 [06:40<22:28, 993.40 examples/s]
Tokenizing train dataset:  26%|██▌       | 468402/1807941 [06:40<21:24, 1042.71 examples/s]
Tokenizing train dataset:  26%|██▌       | 468527/1807941 [06:40<20:25, 1093.12 examples/s]
Tokenizing train dataset:  26%|██▌       | 468654/1807941 [06:40<19:35, 1139.31 examples/s]
Tokenizing train dataset:  26%|██▌       | 468786/1807941 [06:40<21:30, 1037.73 examples/s]
Tokenizing train dataset:  26%|██▌       | 468921/1807941 [06:40<22:37, 986.49 examples/s] 
Tokenizing train dataset:  26%|██▌       | 469026/1807941 [06:40<31:05, 717.57 examples/s]
Tokenizing train dataset:  26%|██▌       | 469126/1807941 [06:41<34:49, 640.60 examples/s]
Tokenizing train dataset:  26%|██▌       | 469219/1807941 [06:41<32:06, 694.98 examples/s]
Tokenizing train dataset:  26%|██▌       | 469320/1807941 [06:41<29:17, 761.68 examples/s]
Tokenizing train dataset:  26%|██▌       | 469416/1807941 [06:41<27:42, 804.93 examples/s]
Tokenizing train dataset:  26%|██▌       | 469521/1807941 [06:41<25:46, 865.69 examples/s]
Tokenizing train dataset:  26%|██▌       | 469633/1807941 [06:41<23:57, 930.77 examples/s]
Tokenizing train dataset:  26%|██▌       | 469748/1807941 [06:41<22:34, 988.10 examples/s]
Tokenizing train dataset:  26%|██▌       | 469853/1807941 [06:41<22:16, 1001.49 examples/s]
Tokenizing train dataset:  26%|██▌       | 469963/1807941 [06:41<21:43, 1026.42 examples/s]
Tokenizing train dataset:  26%|██▌       | 470106/1807941 [06:42<23:24, 952.47 examples/s] 
Tokenizing train dataset:  26%|██▌       | 470212/1807941 [06:42<22:47, 978.36 examples/s]
Tokenizing train dataset:  26%|██▌       | 470328/1807941 [06:42<21:50, 1020.77 examples/s]
Tokenizing train dataset:  26%|██▌       | 470436/1807941 [06:42<21:47, 1023.31 examples/s]
Tokenizing train dataset:  26%|██▌       | 470584/1807941 [06:42<22:08, 1006.43 examples/s]
Tokenizing train dataset:  26%|██▌       | 470706/1807941 [06:42<23:48, 936.02 examples/s] 
Tokenizing train dataset:  26%|██▌       | 470824/1807941 [06:42<28:03, 794.07 examples/s]
Tokenizing train dataset:  26%|██▌       | 470923/1807941 [06:43<26:48, 831.40 examples/s]
Tokenizing train dataset:  26%|██▌       | 471047/1807941 [06:43<27:35, 807.48 examples/s]
Tokenizing train dataset:  26%|██▌       | 471139/1807941 [06:43<26:47, 831.84 examples/s]
Tokenizing train dataset:  26%|██▌       | 471281/1807941 [06:43<25:49, 862.48 examples/s]
Tokenizing train dataset:  26%|██▌       | 471380/1807941 [06:43<25:08, 885.74 examples/s]
Tokenizing train dataset:  26%|██▌       | 471472/1807941 [06:43<25:05, 887.99 examples/s]
Tokenizing train dataset:  26%|██▌       | 471609/1807941 [06:43<25:05, 887.80 examples/s]
Tokenizing train dataset:  26%|██▌       | 471719/1807941 [06:44<27:00, 824.67 examples/s]
Tokenizing train dataset:  26%|██▌       | 471810/1807941 [06:44<29:37, 751.52 examples/s]
Tokenizing train dataset:  26%|██▌       | 471900/1807941 [06:44<31:55, 697.65 examples/s]
Tokenizing train dataset:  26%|██▌       | 472000/1807941 [06:44<34:35, 643.73 examples/s]
Tokenizing train dataset:  26%|██▌       | 472074/1807941 [06:44<33:34, 663.24 examples/s]
Tokenizing train dataset:  26%|██▌       | 472158/1807941 [06:44<35:37, 625.05 examples/s]
Tokenizing train dataset:  26%|██▌       | 472255/1807941 [06:44<35:30, 627.05 examples/s]
Tokenizing train dataset:  26%|██▌       | 472321/1807941 [06:45<35:24, 628.54 examples/s]
Tokenizing train dataset:  26%|██▌       | 472413/1807941 [06:45<36:05, 616.81 examples/s]
Tokenizing train dataset:  26%|██▌       | 472502/1807941 [06:45<37:02, 600.79 examples/s]
Tokenizing train dataset:  26%|██▌       | 472586/1807941 [06:45<38:09, 583.31 examples/s]
Tokenizing train dataset:  26%|██▌       | 472677/1807941 [06:45<37:55, 586.82 examples/s]
Tokenizing train dataset:  26%|██▌       | 472752/1807941 [06:45<39:47, 559.13 examples/s]
Tokenizing train dataset:  26%|██▌       | 472814/1807941 [06:45<38:56, 571.51 examples/s]
Tokenizing train dataset:  26%|██▌       | 472876/1807941 [06:45<38:28, 578.26 examples/s]
Tokenizing train dataset:  26%|██▌       | 472956/1807941 [06:46<39:44, 559.84 examples/s]
Tokenizing train dataset:  26%|██▌       | 473022/1807941 [06:46<45:29, 489.07 examples/s]
Tokenizing train dataset:  26%|██▌       | 473080/1807941 [06:46<43:47, 508.11 examples/s]
Tokenizing train dataset:  26%|██▌       | 473138/1807941 [06:46<42:42, 520.91 examples/s]
Tokenizing train dataset:  26%|██▌       | 473215/1807941 [06:46<43:10, 515.22 examples/s]
Tokenizing train dataset:  26%|██▌       | 473292/1807941 [06:46<43:20, 513.23 examples/s]
Tokenizing train dataset:  26%|██▌       | 473350/1807941 [06:46<42:11, 527.25 examples/s]
Tokenizing train dataset:  26%|██▌       | 473405/1807941 [06:47<41:54, 530.69 examples/s]
Tokenizing train dataset:  26%|██▌       | 473483/1807941 [06:47<42:25, 524.16 examples/s]
Tokenizing train dataset:  26%|██▌       | 473544/1807941 [06:47<40:56, 543.23 examples/s]
Tokenizing train dataset:  26%|██▌       | 473607/1807941 [06:47<39:22, 564.86 examples/s]
Tokenizing train dataset:  26%|██▌       | 473684/1807941 [06:47<41:03, 541.63 examples/s]
Tokenizing train dataset:  26%|██▌       | 473752/1807941 [06:47<38:37, 575.78 examples/s]
Tokenizing train dataset:  26%|██▌       | 473833/1807941 [06:47<39:56, 556.78 examples/s]
Tokenizing train dataset:  26%|██▌       | 473920/1807941 [06:47<39:31, 562.49 examples/s]
Tokenizing train dataset:  26%|██▌       | 473981/1807941 [06:48<39:10, 567.56 examples/s]
Tokenizing train dataset:  26%|██▌       | 474057/1807941 [06:48<42:22, 524.61 examples/s]
Tokenizing train dataset:  26%|██▌       | 474123/1807941 [06:48<44:59, 494.12 examples/s]
Tokenizing train dataset:  26%|██▌       | 474205/1807941 [06:48<43:43, 508.38 examples/s]
Tokenizing train dataset:  26%|██▌       | 474280/1807941 [06:48<44:09, 503.34 examples/s]
Tokenizing train dataset:  26%|██▌       | 474341/1807941 [06:48<42:24, 524.15 examples/s]
Tokenizing train dataset:  26%|██▌       | 474411/1807941 [06:48<39:20, 564.88 examples/s]
Tokenizing train dataset:  26%|██▌       | 474484/1807941 [06:48<36:47, 604.02 examples/s]
Tokenizing train dataset:  26%|██▌       | 474553/1807941 [06:49<35:41, 622.58 examples/s]
Tokenizing train dataset:  26%|██▋       | 474622/1807941 [06:49<34:41, 640.67 examples/s]
Tokenizing train dataset:  26%|██▋       | 474695/1807941 [06:49<33:38, 660.37 examples/s]
Tokenizing train dataset:  26%|██▋       | 474795/1807941 [06:49<33:52, 655.81 examples/s]
Tokenizing train dataset:  26%|██▋       | 474882/1807941 [06:49<35:29, 625.92 examples/s]
Tokenizing train dataset:  26%|██▋       | 474947/1807941 [06:49<35:12, 631.10 examples/s]
Tokenizing train dataset:  26%|██▋       | 475031/1807941 [06:49<39:02, 568.95 examples/s]
Tokenizing train dataset:  26%|██▋       | 475103/1807941 [06:49<36:48, 603.45 examples/s]
Tokenizing train dataset:  26%|██▋       | 475177/1807941 [06:50<40:11, 552.56 examples/s]
Tokenizing train dataset:  26%|██▋       | 475257/1807941 [06:50<36:22, 610.55 examples/s]
Tokenizing train dataset:  26%|██▋       | 475337/1807941 [06:50<38:07, 582.63 examples/s]
Tokenizing train dataset:  26%|██▋       | 475418/1807941 [06:50<35:05, 632.85 examples/s]
Tokenizing train dataset:  26%|██▋       | 475488/1807941 [06:50<34:25, 645.17 examples/s]
Tokenizing train dataset:  26%|██▋       | 475555/1807941 [06:50<34:13, 648.88 examples/s]
Tokenizing train dataset:  26%|██▋       | 475622/1807941 [06:50<34:11, 649.45 examples/s]
Tokenizing train dataset:  26%|██▋       | 475712/1807941 [06:50<35:36, 623.45 examples/s]
Tokenizing train dataset:  26%|██▋       | 475776/1807941 [06:51<35:29, 625.71 examples/s]
Tokenizing train dataset:  26%|██▋       | 475872/1807941 [06:51<35:38, 622.77 examples/s]
Tokenizing train dataset:  26%|██▋       | 475950/1807941 [06:51<37:57, 584.86 examples/s]
Tokenizing train dataset:  26%|██▋       | 476024/1807941 [06:51<41:46, 531.47 examples/s]
Tokenizing train dataset:  26%|██▋       | 476085/1807941 [06:51<45:02, 492.87 examples/s]
Tokenizing train dataset:  26%|██▋       | 476156/1807941 [06:51<45:46, 484.93 examples/s]
Tokenizing train dataset:  26%|██▋       | 476216/1807941 [06:52<48:45, 455.27 examples/s]
Tokenizing train dataset:  26%|██▋       | 476270/1807941 [06:52<46:56, 472.83 examples/s]
Tokenizing train dataset:  26%|██▋       | 476349/1807941 [06:52<45:27, 488.26 examples/s]
Tokenizing train dataset:  26%|██▋       | 476408/1807941 [06:52<43:52, 505.73 examples/s]
Tokenizing train dataset:  26%|██▋       | 476476/1807941 [06:52<40:32, 547.29 examples/s]
Tokenizing train dataset:  26%|██▋       | 476557/1807941 [06:52<41:00, 541.04 examples/s]
Tokenizing train dataset:  26%|██▋       | 476617/1807941 [06:52<40:03, 553.90 examples/s]
Tokenizing train dataset:  26%|██▋       | 476681/1807941 [06:52<38:47, 572.00 examples/s]
Tokenizing train dataset:  26%|██▋       | 476748/1807941 [06:52<37:29, 591.75 examples/s]
Tokenizing train dataset:  26%|██▋       | 476820/1807941 [06:53<40:28, 548.16 examples/s]
Tokenizing train dataset:  26%|██▋       | 476897/1807941 [06:53<41:43, 531.70 examples/s]
Tokenizing train dataset:  26%|██▋       | 476968/1807941 [06:53<43:46, 506.71 examples/s]
Tokenizing train dataset:  26%|██▋       | 477030/1807941 [06:53<48:19, 459.07 examples/s]
Tokenizing train dataset:  26%|██▋       | 477084/1807941 [06:53<46:45, 474.38 examples/s]
Tokenizing train dataset:  26%|██▋       | 477152/1807941 [06:53<47:51, 463.40 examples/s]
Tokenizing train dataset:  26%|██▋       | 477207/1807941 [06:53<46:16, 479.24 examples/s]
Tokenizing train dataset:  26%|██▋       | 477271/1807941 [06:54<48:38, 455.95 examples/s]
Tokenizing train dataset:  26%|██▋       | 477329/1807941 [06:54<45:54, 483.13 examples/s]
Tokenizing train dataset:  26%|██▋       | 477407/1807941 [06:54<45:16, 489.72 examples/s]
Tokenizing train dataset:  26%|██▋       | 477480/1807941 [06:54<45:31, 487.16 examples/s]
Tokenizing train dataset:  26%|██▋       | 477538/1807941 [06:54<44:07, 502.50 examples/s]
Tokenizing train dataset:  26%|██▋       | 477625/1807941 [06:54<42:17, 524.19 examples/s]
Tokenizing train dataset:  26%|██▋       | 477688/1807941 [06:54<40:26, 548.24 examples/s]
Tokenizing train dataset:  26%|██▋       | 477759/1807941 [06:54<37:44, 587.42 examples/s]
Tokenizing train dataset:  26%|██▋       | 477839/1807941 [06:55<39:28, 561.63 examples/s]
Tokenizing train dataset:  26%|██▋       | 477909/1807941 [06:55<42:25, 522.47 examples/s]
Tokenizing train dataset:  26%|██▋       | 477964/1807941 [06:55<42:14, 524.67 examples/s]
Tokenizing train dataset:  26%|██▋       | 478067/1807941 [06:55<39:17, 564.13 examples/s]
Tokenizing train dataset:  26%|██▋       | 478138/1807941 [06:55<37:08, 596.69 examples/s]
Tokenizing train dataset:  26%|██▋       | 478219/1807941 [06:55<38:39, 573.30 examples/s]
Tokenizing train dataset:  26%|██▋       | 478293/1807941 [06:55<36:08, 613.06 examples/s]
Tokenizing train dataset:  26%|██▋       | 478362/1807941 [06:55<35:30, 624.20 examples/s]
Tokenizing train dataset:  26%|██▋       | 478457/1807941 [06:56<35:40, 621.21 examples/s]
Tokenizing train dataset:  26%|██▋       | 478552/1807941 [06:56<35:46, 619.25 examples/s]
Tokenizing train dataset:  26%|██▋       | 478623/1807941 [06:56<34:42, 638.32 examples/s]
Tokenizing train dataset:  26%|██▋       | 478707/1807941 [06:56<36:29, 606.99 examples/s]
Tokenizing train dataset:  26%|██▋       | 478771/1807941 [06:56<36:06, 613.45 examples/s]
Tokenizing train dataset:  26%|██▋       | 478845/1807941 [06:56<34:28, 642.47 examples/s]
Tokenizing train dataset:  26%|██▋       | 478915/1807941 [06:56<33:46, 655.82 examples/s]
Tokenizing train dataset:  26%|██▋       | 478986/1807941 [06:56<37:42, 587.51 examples/s]
Tokenizing train dataset:  26%|██▋       | 479067/1807941 [06:57<45:07, 490.86 examples/s]
Tokenizing train dataset:  27%|██▋       | 479146/1807941 [06:57<44:25, 498.53 examples/s]
Tokenizing train dataset:  27%|██▋       | 479216/1807941 [06:57<40:58, 540.45 examples/s]
Tokenizing train dataset:  27%|██▋       | 479294/1807941 [06:57<41:39, 531.51 examples/s]
Tokenizing train dataset:  27%|██▋       | 479366/1807941 [06:57<38:35, 573.83 examples/s]
Tokenizing train dataset:  27%|██▋       | 479429/1807941 [06:57<37:43, 586.84 examples/s]
Tokenizing train dataset:  27%|██▋       | 479497/1807941 [06:57<36:19, 609.50 examples/s]
Tokenizing train dataset:  27%|██▋       | 479598/1807941 [06:58<35:15, 628.01 examples/s]
Tokenizing train dataset:  27%|██▋       | 479663/1807941 [06:58<35:09, 629.62 examples/s]
Tokenizing train dataset:  27%|██▋       | 479735/1807941 [06:58<33:53, 653.08 examples/s]
Tokenizing train dataset:  27%|██▋       | 479813/1807941 [06:58<37:00, 598.01 examples/s]
Tokenizing train dataset:  27%|██▋       | 479875/1807941 [06:58<36:49, 600.99 examples/s]
Tokenizing train dataset:  27%|██▋       | 479941/1807941 [06:58<35:59, 614.83 examples/s]
Tokenizing train dataset:  27%|██▋       | 480025/1807941 [06:58<39:41, 557.50 examples/s]
Tokenizing train dataset:  27%|██▋       | 480101/1807941 [06:58<36:41, 603.21 examples/s]
Tokenizing train dataset:  27%|██▋       | 480195/1807941 [06:59<36:23, 608.10 examples/s]
Tokenizing train dataset:  27%|██▋       | 480282/1807941 [06:59<37:16, 593.67 examples/s]
Tokenizing train dataset:  27%|██▋       | 480378/1807941 [06:59<36:44, 602.18 examples/s]
Tokenizing train dataset:  27%|██▋       | 480468/1807941 [06:59<37:03, 597.07 examples/s]
Tokenizing train dataset:  27%|██▋       | 480553/1807941 [06:59<37:46, 585.54 examples/s]
Tokenizing train dataset:  27%|██▋       | 480638/1807941 [06:59<38:46, 570.60 examples/s]
Tokenizing train dataset:  27%|██▋       | 480721/1807941 [07:00<39:25, 561.14 examples/s]
Tokenizing train dataset:  27%|██▋       | 480794/1807941 [07:00<37:06, 596.11 examples/s]
Tokenizing train dataset:  27%|██▋       | 480879/1807941 [07:00<37:50, 584.47 examples/s]
Tokenizing train dataset:  27%|██▋       | 480946/1807941 [07:00<36:44, 601.91 examples/s]
Tokenizing train dataset:  27%|██▋       | 481036/1807941 [07:00<41:31, 532.48 examples/s]
Tokenizing train dataset:  27%|██▋       | 481097/1807941 [07:00<40:44, 542.84 examples/s]
Tokenizing train dataset:  27%|██▋       | 481190/1807941 [07:00<39:21, 561.79 examples/s]
Tokenizing train dataset:  27%|██▋       | 481254/1807941 [07:00<38:14, 578.27 examples/s]
Tokenizing train dataset:  27%|██▋       | 481316/1807941 [07:01<37:46, 585.32 examples/s]
Tokenizing train dataset:  27%|██▋       | 481385/1807941 [07:01<36:14, 610.16 examples/s]
Tokenizing train dataset:  27%|██▋       | 481479/1807941 [07:01<36:03, 613.19 examples/s]
Tokenizing train dataset:  27%|██▋       | 481551/1807941 [07:01<34:42, 636.92 examples/s]
Tokenizing train dataset:  27%|██▋       | 481646/1807941 [07:01<35:25, 624.01 examples/s]
Tokenizing train dataset:  27%|██▋       | 481729/1807941 [07:01<36:57, 598.08 examples/s]
Tokenizing train dataset:  27%|██▋       | 481806/1807941 [07:01<34:42, 636.72 examples/s]
Tokenizing train dataset:  27%|██▋       | 481873/1807941 [07:01<34:36, 638.71 examples/s]
Tokenizing train dataset:  27%|██▋       | 481941/1807941 [07:02<38:38, 571.96 examples/s]
Tokenizing train dataset:  27%|██▋       | 482000/1807941 [07:02<40:43, 542.54 examples/s]
Tokenizing train dataset:  27%|██▋       | 482067/1807941 [07:02<44:02, 501.67 examples/s]
Tokenizing train dataset:  27%|██▋       | 482131/1807941 [07:02<41:28, 532.79 examples/s]
Tokenizing train dataset:  27%|██▋       | 482221/1807941 [07:02<35:20, 625.07 examples/s]
Tokenizing train dataset:  27%|██▋       | 482315/1807941 [07:02<31:10, 708.54 examples/s]
Tokenizing train dataset:  27%|██▋       | 482401/1807941 [07:02<29:30, 748.68 examples/s]
Tokenizing train dataset:  27%|██▋       | 482492/1807941 [07:02<27:55, 791.03 examples/s]
Tokenizing train dataset:  27%|██▋       | 482583/1807941 [07:02<26:51, 822.54 examples/s]
Tokenizing train dataset:  27%|██▋       | 482676/1807941 [07:03<26:00, 849.33 examples/s]
Tokenizing train dataset:  27%|██▋       | 482788/1807941 [07:03<27:25, 805.41 examples/s]
Tokenizing train dataset:  27%|██▋       | 482879/1807941 [07:03<26:37, 829.41 examples/s]
Tokenizing train dataset:  27%|██▋       | 483000/1807941 [07:03<28:51, 765.41 examples/s]
Tokenizing train dataset:  27%|██▋       | 483114/1807941 [07:03<29:06, 758.75 examples/s]
Tokenizing train dataset:  27%|██▋       | 483199/1807941 [07:03<28:25, 776.58 examples/s]
Tokenizing train dataset:  27%|██▋       | 483287/1807941 [07:03<27:34, 800.71 examples/s]
Tokenizing train dataset:  27%|██▋       | 483402/1807941 [07:03<28:09, 783.76 examples/s]
Tokenizing train dataset:  27%|██▋       | 483529/1807941 [07:04<27:37, 799.16 examples/s]
Tokenizing train dataset:  27%|██▋       | 483635/1807941 [07:04<25:41, 859.38 examples/s]
Tokenizing train dataset:  27%|██▋       | 483730/1807941 [07:04<25:18, 872.32 examples/s]
Tokenizing train dataset:  27%|██▋       | 483857/1807941 [07:04<25:48, 855.02 examples/s]
Tokenizing train dataset:  27%|██▋       | 483986/1807941 [07:04<25:49, 854.70 examples/s]
Tokenizing train dataset:  27%|██▋       | 484086/1807941 [07:04<28:30, 773.81 examples/s]
Tokenizing train dataset:  27%|██▋       | 484173/1807941 [07:04<27:47, 794.02 examples/s]
Tokenizing train dataset:  27%|██▋       | 484255/1807941 [07:05<27:39, 797.57 examples/s]
Tokenizing train dataset:  27%|██▋       | 484352/1807941 [07:05<26:11, 842.29 examples/s]
Tokenizing train dataset:  27%|██▋       | 484473/1807941 [07:05<26:43, 825.53 examples/s]
Tokenizing train dataset:  27%|██▋       | 484602/1807941 [07:05<26:24, 835.14 examples/s]
Tokenizing train dataset:  27%|██▋       | 484701/1807941 [07:05<25:19, 870.74 examples/s]
Tokenizing train dataset:  27%|██▋       | 484828/1807941 [07:05<25:44, 856.84 examples/s]
Tokenizing train dataset:  27%|██▋       | 484945/1807941 [07:05<26:43, 825.18 examples/s]
Tokenizing train dataset:  27%|██▋       | 485052/1807941 [07:05<28:07, 784.12 examples/s]
Tokenizing train dataset:  27%|██▋       | 485137/1807941 [07:06<27:36, 798.41 examples/s]
Tokenizing train dataset:  27%|██▋       | 485221/1807941 [07:06<27:31, 801.08 examples/s]
Tokenizing train dataset:  27%|██▋       | 485324/1807941 [07:06<25:50, 852.92 examples/s]
Tokenizing train dataset:  27%|██▋       | 485439/1807941 [07:06<26:53, 819.80 examples/s]
Tokenizing train dataset:  27%|██▋       | 485570/1807941 [07:06<26:38, 827.25 examples/s]
Tokenizing train dataset:  27%|██▋       | 485701/1807941 [07:06<26:29, 831.82 examples/s]
Tokenizing train dataset:  27%|██▋       | 485813/1807941 [07:06<27:28, 802.21 examples/s]
Tokenizing train dataset:  27%|██▋       | 485903/1807941 [07:07<26:55, 818.32 examples/s]
Tokenizing train dataset:  27%|██▋       | 486002/1807941 [07:07<28:51, 763.40 examples/s]
Tokenizing train dataset:  27%|██▋       | 486123/1807941 [07:07<28:40, 768.33 examples/s]
Tokenizing train dataset:  27%|██▋       | 486224/1807941 [07:07<26:53, 819.31 examples/s]
Tokenizing train dataset:  27%|██▋       | 486343/1807941 [07:07<27:17, 807.30 examples/s]
Tokenizing train dataset:  27%|██▋       | 486426/1807941 [07:07<27:12, 809.41 examples/s]
Tokenizing train dataset:  27%|██▋       | 486518/1807941 [07:07<26:18, 837.12 examples/s]
Tokenizing train dataset:  27%|██▋       | 486606/1807941 [07:07<25:59, 847.27 examples/s]
Tokenizing train dataset:  27%|██▋       | 486692/1807941 [07:07<25:55, 849.51 examples/s]
Tokenizing train dataset:  27%|██▋       | 486808/1807941 [07:08<26:59, 816.01 examples/s]
Tokenizing train dataset:  27%|██▋       | 486931/1807941 [07:08<27:17, 806.89 examples/s]
Tokenizing train dataset:  27%|██▋       | 487037/1807941 [07:08<29:07, 756.08 examples/s]
Tokenizing train dataset:  27%|██▋       | 487150/1807941 [07:08<29:20, 750.13 examples/s]
Tokenizing train dataset:  27%|██▋       | 487235/1807941 [07:08<28:40, 767.72 examples/s]
Tokenizing train dataset:  27%|██▋       | 487321/1807941 [07:08<27:56, 787.64 examples/s]
Tokenizing train dataset:  27%|██▋       | 487434/1807941 [07:08<28:35, 769.80 examples/s]
Tokenizing train dataset:  27%|██▋       | 487549/1807941 [07:09<28:46, 764.82 examples/s]
Tokenizing train dataset:  27%|██▋       | 487633/1807941 [07:09<28:16, 778.35 examples/s]
Tokenizing train dataset:  27%|██▋       | 487722/1807941 [07:09<27:36, 797.13 examples/s]
Tokenizing train dataset:  27%|██▋       | 487835/1807941 [07:09<28:22, 775.18 examples/s]
Tokenizing train dataset:  27%|██▋       | 487915/1807941 [07:09<28:10, 780.80 examples/s]
Tokenizing train dataset:  27%|██▋       | 488000/1807941 [07:09<30:04, 731.41 examples/s]
Tokenizing train dataset:  27%|██▋       | 488080/1807941 [07:09<29:31, 745.19 examples/s]
Tokenizing train dataset:  27%|██▋       | 488187/1807941 [07:09<30:17, 725.99 examples/s]
Tokenizing train dataset:  27%|██▋       | 488271/1807941 [07:10<29:17, 750.92 examples/s]
Tokenizing train dataset:  27%|██▋       | 488373/1807941 [07:10<30:28, 721.56 examples/s]
Tokenizing train dataset:  27%|██▋       | 488483/1807941 [07:10<30:31, 720.30 examples/s]
Tokenizing train dataset:  27%|██▋       | 488561/1807941 [07:10<30:01, 732.49 examples/s]
Tokenizing train dataset:  27%|██▋       | 488642/1807941 [07:10<29:17, 750.52 examples/s]
Tokenizing train dataset:  27%|██▋       | 488719/1807941 [07:10<29:09, 754.17 examples/s]
Tokenizing train dataset:  27%|██▋       | 488817/1807941 [07:10<26:57, 815.69 examples/s]
Tokenizing train dataset:  27%|██▋       | 488901/1807941 [07:10<26:58, 815.09 examples/s]
Tokenizing train dataset:  27%|██▋       | 488996/1807941 [07:10<25:52, 849.40 examples/s]
Tokenizing train dataset:  27%|██▋       | 489092/1807941 [07:11<28:37, 767.74 examples/s]
Tokenizing train dataset:  27%|██▋       | 489204/1807941 [07:11<29:05, 755.70 examples/s]
Tokenizing train dataset:  27%|██▋       | 489305/1807941 [07:11<27:04, 811.93 examples/s]
Tokenizing train dataset:  27%|██▋       | 489399/1807941 [07:11<26:01, 844.36 examples/s]
Tokenizing train dataset:  27%|██▋       | 489505/1807941 [07:11<24:22, 901.76 examples/s]
Tokenizing train dataset:  27%|██▋       | 489634/1807941 [07:11<24:52, 883.11 examples/s]
Tokenizing train dataset:  27%|██▋       | 489774/1807941 [07:11<24:31, 895.85 examples/s]
Tokenizing train dataset:  27%|██▋       | 489874/1807941 [07:11<23:52, 920.41 examples/s]
Tokenizing train dataset:  27%|██▋       | 490000/1807941 [07:12<25:34, 859.03 examples/s]
Tokenizing train dataset:  27%|██▋       | 490097/1807941 [07:12<24:53, 882.60 examples/s]
Tokenizing train dataset:  27%|██▋       | 490198/1807941 [07:12<24:02, 913.76 examples/s]
Tokenizing train dataset:  27%|██▋       | 490294/1807941 [07:12<23:49, 921.87 examples/s]
Tokenizing train dataset:  27%|██▋       | 490430/1807941 [07:12<24:00, 914.83 examples/s]
Tokenizing train dataset:  27%|██▋       | 490524/1807941 [07:12<23:50, 921.14 examples/s]
Tokenizing train dataset:  27%|██▋       | 490623/1807941 [07:12<23:30, 934.00 examples/s]
Tokenizing train dataset:  27%|██▋       | 490723/1807941 [07:12<23:06, 950.01 examples/s]
Tokenizing train dataset:  27%|██▋       | 490854/1807941 [07:13<24:00, 914.41 examples/s]
Tokenizing train dataset:  27%|██▋       | 490969/1807941 [07:13<25:27, 862.13 examples/s]
Tokenizing train dataset:  27%|██▋       | 491076/1807941 [07:13<28:19, 774.81 examples/s]
Tokenizing train dataset:  27%|██▋       | 491157/1807941 [07:13<28:04, 781.62 examples/s]
Tokenizing train dataset:  27%|██▋       | 491267/1807941 [07:13<28:43, 763.85 examples/s]
Tokenizing train dataset:  27%|██▋       | 491349/1807941 [07:13<28:23, 772.69 examples/s]
Tokenizing train dataset:  27%|██▋       | 491459/1807941 [07:13<29:04, 754.71 examples/s]
Tokenizing train dataset:  27%|██▋       | 491543/1807941 [07:13<28:19, 774.71 examples/s]
Tokenizing train dataset:  27%|██▋       | 491629/1807941 [07:14<27:35, 795.14 examples/s]
Tokenizing train dataset:  27%|██▋       | 491742/1807941 [07:14<28:21, 773.60 examples/s]
Tokenizing train dataset:  27%|██▋       | 491835/1807941 [07:14<27:06, 809.30 examples/s]
Tokenizing train dataset:  27%|██▋       | 491930/1807941 [07:14<25:57, 844.98 examples/s]
Tokenizing train dataset:  27%|██▋       | 492047/1807941 [07:14<27:17, 803.36 examples/s]
Tokenizing train dataset:  27%|██▋       | 492134/1807941 [07:14<26:47, 818.79 examples/s]
Tokenizing train dataset:  27%|██▋       | 492227/1807941 [07:14<25:55, 846.02 examples/s]
Tokenizing train dataset:  27%|██▋       | 492346/1807941 [07:14<26:38, 822.80 examples/s]
Tokenizing train dataset:  27%|██▋       | 492471/1807941 [07:15<26:41, 821.27 examples/s]
Tokenizing train dataset:  27%|██▋       | 492583/1807941 [07:15<27:43, 790.86 examples/s]
Tokenizing train dataset:  27%|██▋       | 492699/1807941 [07:15<28:06, 780.00 examples/s]
Tokenizing train dataset:  27%|██▋       | 492793/1807941 [07:15<26:52, 815.44 examples/s]
Tokenizing train dataset:  27%|██▋       | 492922/1807941 [07:15<26:27, 828.16 examples/s]
Tokenizing train dataset:  27%|██▋       | 493013/1807941 [07:15<29:01, 755.04 examples/s]
Tokenizing train dataset:  27%|██▋       | 493115/1807941 [07:15<26:56, 813.50 examples/s]
Tokenizing train dataset:  27%|██▋       | 493231/1807941 [07:16<27:30, 796.44 examples/s]
Tokenizing train dataset:  27%|██▋       | 493332/1807941 [07:16<25:53, 846.19 examples/s]
Tokenizing train dataset:  27%|██▋       | 493422/1807941 [07:16<25:34, 856.83 examples/s]
Tokenizing train dataset:  27%|██▋       | 493521/1807941 [07:16<24:38, 889.09 examples/s]
Tokenizing train dataset:  27%|██▋       | 493639/1807941 [07:16<25:48, 848.76 examples/s]
Tokenizing train dataset:  27%|██▋       | 493759/1807941 [07:16<26:30, 826.24 examples/s]
Tokenizing train dataset:  27%|██▋       | 493878/1807941 [07:16<26:56, 812.78 examples/s]
Tokenizing train dataset:  27%|██▋       | 493962/1807941 [07:16<27:00, 810.82 examples/s]
Tokenizing train dataset:  27%|██▋       | 494074/1807941 [07:17<29:38, 738.63 examples/s]
Tokenizing train dataset:  27%|██▋       | 494164/1807941 [07:17<28:17, 773.85 examples/s]
Tokenizing train dataset:  27%|██▋       | 494271/1807941 [07:17<26:00, 842.06 examples/s]
Tokenizing train dataset:  27%|██▋       | 494400/1807941 [07:17<25:59, 842.04 examples/s]
Tokenizing train dataset:  27%|██▋       | 494514/1807941 [07:17<26:53, 813.88 examples/s]
Tokenizing train dataset:  27%|██▋       | 494622/1807941 [07:17<28:04, 779.75 examples/s]
Tokenizing train dataset:  27%|██▋       | 494729/1807941 [07:17<28:54, 757.13 examples/s]
Tokenizing train dataset:  27%|██▋       | 494822/1807941 [07:18<27:32, 794.70 examples/s]
Tokenizing train dataset:  27%|██▋       | 494944/1807941 [07:18<27:22, 799.52 examples/s]
Tokenizing train dataset:  27%|██▋       | 495051/1807941 [07:18<28:27, 768.78 examples/s]
Tokenizing train dataset:  27%|██▋       | 495137/1807941 [07:18<27:47, 787.12 examples/s]
Tokenizing train dataset:  27%|██▋       | 495219/1807941 [07:18<27:37, 792.07 examples/s]
Tokenizing train dataset:  27%|██▋       | 495303/1807941 [07:18<27:22, 799.02 examples/s]
Tokenizing train dataset:  27%|██▋       | 495392/1807941 [07:18<26:38, 820.87 examples/s]
Tokenizing train dataset:  27%|██▋       | 495481/1807941 [07:18<26:04, 838.84 examples/s]
Tokenizing train dataset:  27%|██▋       | 495598/1807941 [07:18<26:53, 813.23 examples/s]
Tokenizing train dataset:  27%|██▋       | 495724/1807941 [07:19<26:39, 820.49 examples/s]
Tokenizing train dataset:  27%|██▋       | 495813/1807941 [07:19<26:10, 835.37 examples/s]
Tokenizing train dataset:  27%|██▋       | 495898/1807941 [07:19<26:18, 831.39 examples/s]
Tokenizing train dataset:  27%|██▋       | 495990/1807941 [07:19<25:38, 852.52 examples/s]
Tokenizing train dataset:  27%|██▋       | 496118/1807941 [07:19<28:57, 754.89 examples/s]
Tokenizing train dataset:  27%|██▋       | 496238/1807941 [07:19<28:30, 766.98 examples/s]
Tokenizing train dataset:  27%|██▋       | 496329/1807941 [07:19<27:22, 798.32 examples/s]
Tokenizing train dataset:  27%|██▋       | 496414/1807941 [07:20<26:59, 809.94 examples/s]
Tokenizing train dataset:  27%|██▋       | 496504/1807941 [07:20<26:17, 831.23 examples/s]
Tokenizing train dataset:  27%|██▋       | 496589/1807941 [07:20<26:19, 830.17 examples/s]
Tokenizing train dataset:  27%|██▋       | 496704/1807941 [07:20<27:12, 803.34 examples/s]
Tokenizing train dataset:  27%|██▋       | 496789/1807941 [07:20<26:52, 813.04 examples/s]
Tokenizing train dataset:  27%|██▋       | 496908/1807941 [07:20<27:13, 802.39 examples/s]
Tokenizing train dataset:  27%|██▋       | 497000/1807941 [07:20<27:31, 793.80 examples/s]
Tokenizing train dataset:  27%|██▋       | 497090/1807941 [07:20<26:37, 820.38 examples/s]
Tokenizing train dataset:  28%|██▊       | 497213/1807941 [07:20<26:48, 814.78 examples/s]
Tokenizing train dataset:  28%|██▊       | 497306/1807941 [07:21<25:59, 840.60 examples/s]
Tokenizing train dataset:  28%|██▊       | 497409/1807941 [07:21<27:58, 780.61 examples/s]
Tokenizing train dataset:  28%|██▊       | 497498/1807941 [07:21<27:13, 802.13 examples/s]
Tokenizing train dataset:  28%|██▊       | 497588/1807941 [07:21<26:41, 818.27 examples/s]
Tokenizing train dataset:  28%|██▊       | 497709/1807941 [07:21<26:56, 810.56 examples/s]
Tokenizing train dataset:  28%|██▊       | 497825/1807941 [07:21<27:29, 794.16 examples/s]
Tokenizing train dataset:  28%|██▊       | 497940/1807941 [07:21<28:08, 775.85 examples/s]
Tokenizing train dataset:  28%|██▊       | 498047/1807941 [07:22<29:04, 751.04 examples/s]
Tokenizing train dataset:  28%|██▊       | 498125/1807941 [07:22<28:50, 756.79 examples/s]
Tokenizing train dataset:  28%|██▊       | 498206/1807941 [07:22<28:24, 768.48 examples/s]
Tokenizing train dataset:  28%|██▊       | 498321/1807941 [07:22<28:46, 758.40 examples/s]
Tokenizing train dataset:  28%|██▊       | 498403/1807941 [07:22<28:16, 771.82 examples/s]
Tokenizing train dataset:  28%|██▊       | 498506/1807941 [07:22<29:30, 739.39 examples/s]
Tokenizing train dataset:  28%|██▊       | 498581/1807941 [07:22<29:27, 740.71 examples/s]
Tokenizing train dataset:  28%|██▊       | 498666/1807941 [07:22<28:35, 763.18 examples/s]
Tokenizing train dataset:  28%|██▊       | 498756/1807941 [07:22<27:23, 796.67 examples/s]
Tokenizing train dataset:  28%|██▊       | 498843/1807941 [07:23<26:47, 814.30 examples/s]
Tokenizing train dataset:  28%|██▊       | 498952/1807941 [07:23<24:29, 890.47 examples/s]
Tokenizing train dataset:  28%|██▊       | 499049/1807941 [07:23<27:25, 795.66 examples/s]
Tokenizing train dataset:  28%|██▊       | 499148/1807941 [07:23<25:53, 842.66 examples/s]
Tokenizing train dataset:  28%|██▊       | 499258/1807941 [07:23<27:22, 796.55 examples/s]
Tokenizing train dataset:  28%|██▊       | 499352/1807941 [07:23<26:19, 828.55 examples/s]
Tokenizing train dataset:  28%|██▊       | 499444/1807941 [07:23<25:50, 844.00 examples/s]
Tokenizing train dataset:  28%|██▊       | 499537/1807941 [07:23<25:15, 863.09 examples/s]
Tokenizing train dataset:  28%|██▊       | 499655/1807941 [07:24<26:28, 823.45 examples/s]
Tokenizing train dataset:  28%|██▊       | 499786/1807941 [07:24<26:07, 834.58 examples/s]
Tokenizing train dataset:  28%|██▊       | 499880/1807941 [07:24<25:22, 859.32 examples/s]
Tokenizing train dataset:  28%|██▊       | 499997/1807941 [07:24<26:16, 829.67 examples/s]
Tokenizing train dataset:  28%|██▊       | 500104/1807941 [07:24<27:40, 787.61 examples/s]
Tokenizing train dataset:  28%|██▊       | 500225/1807941 [07:24<27:31, 791.99 examples/s]
Tokenizing train dataset:  28%|██▊       | 500339/1807941 [07:24<28:00, 778.11 examples/s]
Tokenizing train dataset:  28%|██▊       | 500434/1807941 [07:25<26:52, 810.73 examples/s]
Tokenizing train dataset:  28%|██▊       | 500519/1807941 [07:25<26:38, 817.68 examples/s]
Tokenizing train dataset:  28%|██▊       | 500646/1807941 [07:25<26:26, 824.23 examples/s]
Tokenizing train dataset:  28%|██▊       | 500731/1807941 [07:25<26:33, 820.38 examples/s]
Tokenizing train dataset:  28%|██▊       | 500851/1807941 [07:25<26:54, 809.42 examples/s]
Tokenizing train dataset:  28%|██▊       | 500941/1807941 [07:25<26:14, 830.36 examples/s]
Tokenizing train dataset:  28%|██▊       | 501049/1807941 [07:25<27:50, 782.27 examples/s]
Tokenizing train dataset:  28%|██▊       | 501132/1807941 [07:25<27:32, 790.98 examples/s]
Tokenizing train dataset:  28%|██▊       | 501243/1807941 [07:26<28:12, 771.98 examples/s]
Tokenizing train dataset:  28%|██▊       | 501349/1807941 [07:26<29:07, 747.79 examples/s]
Tokenizing train dataset:  28%|██▊       | 501432/1807941 [07:26<28:25, 765.84 examples/s]
Tokenizing train dataset:  28%|██▊       | 501546/1807941 [07:26<28:47, 756.12 examples/s]
Tokenizing train dataset:  28%|██▊       | 501654/1807941 [07:26<29:32, 736.99 examples/s]
Tokenizing train dataset:  28%|██▊       | 501732/1807941 [07:26<29:20, 741.82 examples/s]
Tokenizing train dataset:  28%|██▊       | 501845/1807941 [07:26<29:20, 741.74 examples/s]
Tokenizing train dataset:  28%|██▊       | 501929/1807941 [07:26<28:30, 763.43 examples/s]
Tokenizing train dataset:  28%|██▊       | 502042/1807941 [07:27<29:32, 736.90 examples/s]
Tokenizing train dataset:  28%|██▊       | 502121/1807941 [07:27<29:11, 745.57 examples/s]
Tokenizing train dataset:  28%|██▊       | 502199/1807941 [07:27<28:59, 750.53 examples/s]
Tokenizing train dataset:  28%|██▊       | 502289/1807941 [07:27<27:40, 786.46 examples/s]
Tokenizing train dataset:  28%|██▊       | 502380/1807941 [07:27<26:35, 818.19 examples/s]
Tokenizing train dataset:  28%|██▊       | 502499/1807941 [07:27<27:03, 804.17 examples/s]
Tokenizing train dataset:  28%|██▊       | 502619/1807941 [07:27<27:11, 800.15 examples/s]
Tokenizing train dataset:  28%|██▊       | 502702/1807941 [07:27<26:57, 806.73 examples/s]
Tokenizing train dataset:  28%|██▊       | 502834/1807941 [07:28<26:16, 827.74 examples/s]
Tokenizing train dataset:  28%|██▊       | 502927/1807941 [07:28<25:34, 850.30 examples/s]
Tokenizing train dataset:  28%|██▊       | 503041/1807941 [07:28<27:43, 784.59 examples/s]
Tokenizing train dataset:  28%|██▊       | 503131/1807941 [07:28<26:48, 811.08 examples/s]
Tokenizing train dataset:  28%|██▊       | 503254/1807941 [07:28<26:47, 811.42 examples/s]
Tokenizing train dataset:  28%|██▊       | 503345/1807941 [07:28<26:06, 832.96 examples/s]
Tokenizing train dataset:  28%|██▊       | 503468/1807941 [07:28<26:18, 826.61 examples/s]
Tokenizing train dataset:  28%|██▊       | 503575/1807941 [07:29<27:38, 786.32 examples/s]
Tokenizing train dataset:  28%|██▊       | 503676/1807941 [07:29<29:09, 745.30 examples/s]
Tokenizing train dataset:  28%|██▊       | 503775/1807941 [07:29<30:33, 711.40 examples/s]
Tokenizing train dataset:  28%|██▊       | 503857/1807941 [07:29<32:59, 658.76 examples/s]
Tokenizing train dataset:  28%|██▊       | 503940/1807941 [07:29<34:50, 623.79 examples/s]
Tokenizing train dataset:  28%|██▊       | 504009/1807941 [07:29<37:56, 572.87 examples/s]
Tokenizing train dataset:  28%|██▊       | 504077/1807941 [07:29<36:30, 595.36 examples/s]
Tokenizing train dataset:  28%|██▊       | 504140/1807941 [07:29<36:14, 599.67 examples/s]
Tokenizing train dataset:  28%|██▊       | 504229/1807941 [07:30<36:41, 592.07 examples/s]
Tokenizing train dataset:  28%|██▊       | 504307/1807941 [07:30<38:54, 558.47 examples/s]
Tokenizing train dataset:  28%|██▊       | 504387/1807941 [07:30<35:28, 612.56 examples/s]
Tokenizing train dataset:  28%|██▊       | 504478/1807941 [07:30<36:00, 603.20 examples/s]
Tokenizing train dataset:  28%|██▊       | 504549/1807941 [07:30<34:37, 627.26 examples/s]
Tokenizing train dataset:  28%|██▊       | 504619/1807941 [07:30<34:17, 633.57 examples/s]
Tokenizing train dataset:  28%|██▊       | 504687/1807941 [07:30<33:42, 644.23 examples/s]
Tokenizing train dataset:  28%|██▊       | 504762/1807941 [07:30<32:21, 671.35 examples/s]
Tokenizing train dataset:  28%|██▊       | 504858/1807941 [07:31<33:16, 652.76 examples/s]
Tokenizing train dataset:  28%|██▊       | 504957/1807941 [07:31<33:27, 649.10 examples/s]
Tokenizing train dataset:  28%|██▊       | 505042/1807941 [07:31<35:36, 609.83 examples/s]
Tokenizing train dataset:  28%|██▊       | 505111/1807941 [07:31<34:44, 624.91 examples/s]
Tokenizing train dataset:  28%|██▊       | 505200/1807941 [07:31<35:53, 604.92 examples/s]
Tokenizing train dataset:  28%|██▊       | 505265/1807941 [07:31<35:27, 612.29 examples/s]
Tokenizing train dataset:  28%|██▊       | 505331/1807941 [07:31<34:56, 621.37 examples/s]
Tokenizing train dataset:  28%|██▊       | 505400/1807941 [07:31<34:01, 638.02 examples/s]
Tokenizing train dataset:  28%|██▊       | 505470/1807941 [07:32<33:21, 650.65 examples/s]
Tokenizing train dataset:  28%|██▊       | 505547/1807941 [07:32<31:55, 679.96 examples/s]
Tokenizing train dataset:  28%|██▊       | 505642/1807941 [07:32<33:09, 654.67 examples/s]
Tokenizing train dataset:  28%|██▊       | 505714/1807941 [07:32<32:31, 667.25 examples/s]
Tokenizing train dataset:  28%|██▊       | 505795/1807941 [07:32<30:55, 701.75 examples/s]
Tokenizing train dataset:  28%|██▊       | 505882/1807941 [07:32<33:12, 653.57 examples/s]
Tokenizing train dataset:  28%|██▊       | 505954/1807941 [07:32<32:29, 667.90 examples/s]
Tokenizing train dataset:  28%|██▊       | 506027/1807941 [07:32<37:21, 580.86 examples/s]
Tokenizing train dataset:  28%|██▊       | 506109/1807941 [07:33<33:57, 638.79 examples/s]
Tokenizing train dataset:  28%|██▊       | 506187/1807941 [07:33<32:15, 672.44 examples/s]
Tokenizing train dataset:  28%|██▊       | 506263/1807941 [07:33<35:25, 612.43 examples/s]
Tokenizing train dataset:  28%|██▊       | 506328/1807941 [07:33<39:43, 546.12 examples/s]
Tokenizing train dataset:  28%|██▊       | 506386/1807941 [07:33<44:10, 491.04 examples/s]
Tokenizing train dataset:  28%|██▊       | 506461/1807941 [07:33<44:05, 492.00 examples/s]
Tokenizing train dataset:  28%|██▊       | 506535/1807941 [07:33<44:30, 487.31 examples/s]
Tokenizing train dataset:  28%|██▊       | 506589/1807941 [07:34<44:02, 492.46 examples/s]
Tokenizing train dataset:  28%|██▊       | 506650/1807941 [07:34<41:58, 516.70 examples/s]
Tokenizing train dataset:  28%|██▊       | 506737/1807941 [07:34<35:51, 604.73 examples/s]
Tokenizing train dataset:  28%|██▊       | 506818/1807941 [07:34<33:02, 656.34 examples/s]
Tokenizing train dataset:  28%|██▊       | 506896/1807941 [07:34<31:31, 687.75 examples/s]
Tokenizing train dataset:  28%|██▊       | 506967/1807941 [07:34<31:22, 691.15 examples/s]
Tokenizing train dataset:  28%|██▊       | 507046/1807941 [07:34<34:11, 634.22 examples/s]
Tokenizing train dataset:  28%|██▊       | 507126/1807941 [07:34<32:26, 668.38 examples/s]
Tokenizing train dataset:  28%|██▊       | 507201/1807941 [07:34<31:29, 688.55 examples/s]
Tokenizing train dataset:  28%|██▊       | 507301/1807941 [07:35<32:11, 673.54 examples/s]
Tokenizing train dataset:  28%|██▊       | 507378/1807941 [07:35<31:11, 694.93 examples/s]
Tokenizing train dataset:  28%|██▊       | 507459/1807941 [07:35<30:02, 721.63 examples/s]
Tokenizing train dataset:  28%|██▊       | 507564/1807941 [07:35<30:36, 708.24 examples/s]
Tokenizing train dataset:  28%|██▊       | 507637/1807941 [07:35<30:28, 711.29 examples/s]
Tokenizing train dataset:  28%|██▊       | 507727/1807941 [07:35<32:28, 667.20 examples/s]
Tokenizing train dataset:  28%|██▊       | 507824/1807941 [07:35<33:01, 656.10 examples/s]
Tokenizing train dataset:  28%|██▊       | 507911/1807941 [07:35<34:42, 624.19 examples/s]
Tokenizing train dataset:  28%|██▊       | 507980/1807941 [07:36<34:05, 635.56 examples/s]
Tokenizing train dataset:  28%|██▊       | 508058/1807941 [07:36<37:02, 584.81 examples/s]
Tokenizing train dataset:  28%|██▊       | 508124/1807941 [07:36<40:33, 534.11 examples/s]
Tokenizing train dataset:  28%|██▊       | 508198/1807941 [07:36<41:41, 519.69 examples/s]
Tokenizing train dataset:  28%|██▊       | 508256/1807941 [07:36<40:48, 530.90 examples/s]
Tokenizing train dataset:  28%|██▊       | 508328/1807941 [07:36<42:20, 511.52 examples/s]
Tokenizing train dataset:  28%|██▊       | 508382/1807941 [07:36<41:55, 516.69 examples/s]
Tokenizing train dataset:  28%|██▊       | 508459/1807941 [07:37<42:15, 512.47 examples/s]
Tokenizing train dataset:  28%|██▊       | 508512/1807941 [07:37<42:04, 514.64 examples/s]
Tokenizing train dataset:  28%|██▊       | 508566/1807941 [07:37<42:01, 515.28 examples/s]
Tokenizing train dataset:  28%|██▊       | 508626/1807941 [07:37<40:24, 535.92 examples/s]
Tokenizing train dataset:  28%|██▊       | 508703/1807941 [07:37<41:29, 521.92 examples/s]
Tokenizing train dataset:  28%|██▊       | 508785/1807941 [07:37<41:12, 525.43 examples/s]
Tokenizing train dataset:  28%|██▊       | 508865/1807941 [07:37<41:17, 524.30 examples/s]
Tokenizing train dataset:  28%|██▊       | 508945/1807941 [07:37<41:27, 522.13 examples/s]
Tokenizing train dataset:  28%|██▊       | 509000/1807941 [07:38<46:04, 469.87 examples/s]
Tokenizing train dataset:  28%|██▊       | 509049/1807941 [07:38<45:50, 472.23 examples/s]
Tokenizing train dataset:  28%|██▊       | 509097/1807941 [07:38<45:49, 472.39 examples/s]
Tokenizing train dataset:  28%|██▊       | 509164/1807941 [07:38<46:59, 460.58 examples/s]
Tokenizing train dataset:  28%|██▊       | 509215/1807941 [07:38<46:00, 470.40 examples/s]
Tokenizing train dataset:  28%|██▊       | 509266/1807941 [07:38<45:14, 478.35 examples/s]
Tokenizing train dataset:  28%|██▊       | 509329/1807941 [07:38<47:40, 453.98 examples/s]
Tokenizing train dataset:  28%|██▊       | 509402/1807941 [07:39<46:55, 461.27 examples/s]
Tokenizing train dataset:  28%|██▊       | 509463/1807941 [07:39<49:38, 435.99 examples/s]
Tokenizing train dataset:  28%|██▊       | 509510/1807941 [07:39<49:13, 439.70 examples/s]
Tokenizing train dataset:  28%|██▊       | 509558/1807941 [07:39<48:13, 448.69 examples/s]
Tokenizing train dataset:  28%|██▊       | 509611/1807941 [07:39<46:10, 468.67 examples/s]
Tokenizing train dataset:  28%|██▊       | 509661/1807941 [07:39<45:57, 470.81 examples/s]
Tokenizing train dataset:  28%|██▊       | 509713/1807941 [07:39<44:48, 482.82 examples/s]
Tokenizing train dataset:  28%|██▊       | 509786/1807941 [07:39<44:56, 481.50 examples/s]
Tokenizing train dataset:  28%|██▊       | 509839/1807941 [07:39<44:15, 488.75 examples/s]
Tokenizing train dataset:  28%|██▊       | 509917/1807941 [07:40<43:34, 496.41 examples/s]
Tokenizing train dataset:  28%|██▊       | 509996/1807941 [07:40<43:19, 499.30 examples/s]
Tokenizing train dataset:  28%|██▊       | 510051/1807941 [07:40<49:24, 437.78 examples/s]
Tokenizing train dataset:  28%|██▊       | 510113/1807941 [07:40<45:25, 476.11 examples/s]
Tokenizing train dataset:  28%|██▊       | 510169/1807941 [07:40<43:47, 493.83 examples/s]
Tokenizing train dataset:  28%|██▊       | 510225/1807941 [07:40<42:28, 509.16 examples/s]
Tokenizing train dataset:  28%|██▊       | 510278/1807941 [07:40<42:06, 513.68 examples/s]
Tokenizing train dataset:  28%|██▊       | 510334/1807941 [07:40<41:25, 522.11 examples/s]
Tokenizing train dataset:  28%|██▊       | 510412/1807941 [07:41<41:37, 519.46 examples/s]
Tokenizing train dataset:  28%|██▊       | 510466/1807941 [07:41<41:49, 516.94 examples/s]
Tokenizing train dataset:  28%|██▊       | 510529/1807941 [07:41<45:08, 478.93 examples/s]
Tokenizing train dataset:  28%|██▊       | 510586/1807941 [07:41<43:25, 497.91 examples/s]
Tokenizing train dataset:  28%|██▊       | 510650/1807941 [07:41<40:50, 529.44 examples/s]
Tokenizing train dataset:  28%|██▊       | 510704/1807941 [07:41<41:06, 526.00 examples/s]
Tokenizing train dataset:  28%|██▊       | 510758/1807941 [07:41<41:22, 522.54 examples/s]
Tokenizing train dataset:  28%|██▊       | 510831/1807941 [07:41<42:53, 503.98 examples/s]
Tokenizing train dataset:  28%|██▊       | 510883/1807941 [07:42<42:54, 503.87 examples/s]
Tokenizing train dataset:  28%|██▊       | 510942/1807941 [07:42<41:31, 520.57 examples/s]
Tokenizing train dataset:  28%|██▊       | 510997/1807941 [07:42<41:13, 524.24 examples/s]
Tokenizing train dataset:  28%|██▊       | 511056/1807941 [07:42<47:22, 456.21 examples/s]
Tokenizing train dataset:  28%|██▊       | 511107/1807941 [07:42<46:31, 464.62 examples/s]
Tokenizing train dataset:  28%|██▊       | 511160/1807941 [07:42<45:06, 479.17 examples/s]
Tokenizing train dataset:  28%|██▊       | 511213/1807941 [07:42<44:08, 489.58 examples/s]
Tokenizing train dataset:  28%|██▊       | 511279/1807941 [07:42<46:06, 468.70 examples/s]
Tokenizing train dataset:  28%|██▊       | 511329/1807941 [07:42<45:32, 474.56 examples/s]
Tokenizing train dataset:  28%|██▊       | 511407/1807941 [07:43<44:41, 483.60 examples/s]
Tokenizing train dataset:  28%|██▊       | 511482/1807941 [07:43<44:34, 484.82 examples/s]
Tokenizing train dataset:  28%|██▊       | 511534/1807941 [07:43<44:16, 488.03 examples/s]
Tokenizing train dataset:  28%|██▊       | 511604/1807941 [07:43<45:05, 479.09 examples/s]
Tokenizing train dataset:  28%|██▊       | 511663/1807941 [07:43<42:45, 505.32 examples/s]
Tokenizing train dataset:  28%|██▊       | 511721/1807941 [07:43<41:16, 523.41 examples/s]
Tokenizing train dataset:  28%|██▊       | 511792/1807941 [07:43<43:03, 501.64 examples/s]
Tokenizing train dataset:  28%|██▊       | 511846/1807941 [07:43<42:42, 505.72 examples/s]
Tokenizing train dataset:  28%|██▊       | 511908/1807941 [07:44<45:47, 471.78 examples/s]
Tokenizing train dataset:  28%|██▊       | 511959/1807941 [07:44<45:36, 473.56 examples/s]
Tokenizing train dataset:  28%|██▊       | 512027/1807941 [07:44<51:06, 422.64 examples/s]
Tokenizing train dataset:  28%|██▊       | 512078/1807941 [07:44<49:18, 438.01 examples/s]
Tokenizing train dataset:  28%|██▊       | 512131/1807941 [07:44<46:57, 459.90 examples/s]
Tokenizing train dataset:  28%|██▊       | 512179/1807941 [07:44<46:44, 462.05 examples/s]
Tokenizing train dataset:  28%|██▊       | 512243/1807941 [07:44<42:40, 506.02 examples/s]
Tokenizing train dataset:  28%|██▊       | 512300/1807941 [07:44<41:28, 520.72 examples/s]
Tokenizing train dataset:  28%|██▊       | 512376/1807941 [07:45<42:04, 513.11 examples/s]
Tokenizing train dataset:  28%|██▊       | 512443/1807941 [07:45<44:14, 488.10 examples/s]
Tokenizing train dataset:  28%|██▊       | 512530/1807941 [07:45<37:07, 581.62 examples/s]
Tokenizing train dataset:  28%|██▊       | 512601/1807941 [07:45<35:09, 614.19 examples/s]
Tokenizing train dataset:  28%|██▊       | 512681/1807941 [07:45<37:05, 582.01 examples/s]
Tokenizing train dataset:  28%|██▊       | 512745/1807941 [07:45<41:14, 523.38 examples/s]
Tokenizing train dataset:  28%|██▊       | 512800/1807941 [07:45<46:08, 467.78 examples/s]
Tokenizing train dataset:  28%|██▊       | 512862/1807941 [07:46<48:43, 442.96 examples/s]
Tokenizing train dataset:  28%|██▊       | 512931/1807941 [07:46<48:31, 444.84 examples/s]
Tokenizing train dataset:  28%|██▊       | 512984/1807941 [07:46<46:39, 462.56 examples/s]
Tokenizing train dataset:  28%|██▊       | 513045/1807941 [07:46<53:45, 401.44 examples/s]
Tokenizing train dataset:  28%|██▊       | 513099/1807941 [07:46<50:08, 430.46 examples/s]
Tokenizing train dataset:  28%|██▊       | 513148/1807941 [07:46<48:56, 440.88 examples/s]
Tokenizing train dataset:  28%|██▊       | 513227/1807941 [07:46<46:23, 465.22 examples/s]
Tokenizing train dataset:  28%|██▊       | 513297/1807941 [07:47<46:39, 462.47 examples/s]
Tokenizing train dataset:  28%|██▊       | 513348/1807941 [07:47<45:45, 471.53 examples/s]
Tokenizing train dataset:  28%|██▊       | 513417/1807941 [07:47<46:23, 465.10 examples/s]
Tokenizing train dataset:  28%|██▊       | 513470/1807941 [07:47<45:13, 477.00 examples/s]
Tokenizing train dataset:  28%|██▊       | 513548/1807941 [07:47<44:00, 490.25 examples/s]
Tokenizing train dataset:  28%|██▊       | 513621/1807941 [07:47<44:27, 485.21 examples/s]
Tokenizing train dataset:  28%|██▊       | 513700/1807941 [07:47<43:34, 495.08 examples/s]
Tokenizing train dataset:  28%|██▊       | 513753/1807941 [07:47<43:09, 499.78 examples/s]
Tokenizing train dataset:  28%|██▊       | 513820/1807941 [07:48<44:59, 479.30 examples/s]
Tokenizing train dataset:  28%|██▊       | 513871/1807941 [07:48<44:27, 485.06 examples/s]
Tokenizing train dataset:  28%|██▊       | 513921/1807941 [07:48<44:24, 485.72 examples/s]
Tokenizing train dataset:  28%|██▊       | 513976/1807941 [07:48<43:25, 496.62 examples/s]
Tokenizing train dataset:  28%|██▊       | 514048/1807941 [07:48<51:07, 421.75 examples/s]
Tokenizing train dataset:  28%|██▊       | 514110/1807941 [07:48<46:28, 463.96 examples/s]
Tokenizing train dataset:  28%|██▊       | 514184/1807941 [07:48<45:43, 471.50 examples/s]
Tokenizing train dataset:  28%|██▊       | 514244/1807941 [07:48<43:06, 500.13 examples/s]
Tokenizing train dataset:  28%|██▊       | 514300/1807941 [07:49<41:55, 514.17 examples/s]
Tokenizing train dataset:  28%|██▊       | 514383/1807941 [07:49<41:20, 521.58 examples/s]
Tokenizing train dataset:  28%|██▊       | 514442/1807941 [07:49<40:26, 533.09 examples/s]
Tokenizing train dataset:  28%|██▊       | 514501/1807941 [07:49<39:41, 543.03 examples/s]
Tokenizing train dataset:  28%|██▊       | 514582/1807941 [07:49<35:15, 611.25 examples/s]
Tokenizing train dataset:  28%|██▊       | 514663/1807941 [07:49<42:01, 512.96 examples/s]
Tokenizing train dataset:  28%|██▊       | 514729/1807941 [07:49<39:30, 545.53 examples/s]
Tokenizing train dataset:  28%|██▊       | 514802/1807941 [07:50<41:22, 520.82 examples/s]
Tokenizing train dataset:  28%|██▊       | 514939/1807941 [07:50<33:26, 644.34 examples/s]
Tokenizing train dataset:  28%|██▊       | 515162/1807941 [07:50<22:02, 977.83 examples/s]
Tokenizing train dataset:  29%|██▊       | 515485/1807941 [07:50<14:07, 1524.73 examples/s]
Tokenizing train dataset:  29%|██▊       | 515813/1807941 [07:50<10:54, 1975.25 examples/s]
Tokenizing train dataset:  29%|██▊       | 516088/1807941 [07:50<09:52, 2181.32 examples/s]
Tokenizing train dataset:  29%|██▊       | 516343/1807941 [07:50<12:04, 1782.13 examples/s]
Tokenizing train dataset:  29%|██▊       | 516561/1807941 [07:50<12:55, 1665.51 examples/s]
Tokenizing train dataset:  29%|██▊       | 516773/1807941 [07:51<13:34, 1585.51 examples/s]
Tokenizing train dataset:  29%|██▊       | 516991/1807941 [07:51<14:00, 1535.81 examples/s]
Tokenizing train dataset:  29%|██▊       | 517190/1807941 [07:51<14:37, 1470.16 examples/s]
Tokenizing train dataset:  29%|██▊       | 517353/1807941 [07:51<14:17, 1504.18 examples/s]
Tokenizing train dataset:  29%|██▊       | 517520/1807941 [07:51<13:57, 1540.57 examples/s]
Tokenizing train dataset:  29%|██▊       | 517692/1807941 [07:51<13:34, 1584.87 examples/s]
Tokenizing train dataset:  29%|██▊       | 517855/1807941 [07:51<13:28, 1595.26 examples/s]
Tokenizing train dataset:  29%|██▊       | 518081/1807941 [07:51<13:48, 1556.34 examples/s]
Tokenizing train dataset:  29%|██▊       | 518241/1807941 [07:52<13:43, 1566.53 examples/s]
Tokenizing train dataset:  29%|██▊       | 518423/1807941 [07:52<13:10, 1631.86 examples/s]
Tokenizing train dataset:  29%|██▊       | 518648/1807941 [07:52<13:36, 1579.93 examples/s]
Tokenizing train dataset:  29%|██▊       | 518822/1807941 [07:52<13:15, 1619.68 examples/s]
Tokenizing train dataset:  29%|██▊       | 519000/1807941 [07:52<13:28, 1594.54 examples/s]
Tokenizing train dataset:  29%|██▊       | 519185/1807941 [07:52<12:56, 1659.46 examples/s]
Tokenizing train dataset:  29%|██▊       | 519408/1807941 [07:52<13:28, 1592.83 examples/s]
Tokenizing train dataset:  29%|██▊       | 519655/1807941 [07:52<13:22, 1605.36 examples/s]
Tokenizing train dataset:  29%|██▉       | 519965/1807941 [07:53<10:53, 1969.98 examples/s]
Tokenizing train dataset:  29%|██▉       | 520208/1807941 [07:53<10:16, 2088.42 examples/s]
Tokenizing train dataset:  29%|██▉       | 520489/1807941 [07:53<09:24, 2280.43 examples/s]
Tokenizing train dataset:  29%|██▉       | 520781/1807941 [07:53<08:44, 2453.81 examples/s]
Tokenizing train dataset:  29%|██▉       | 521164/1807941 [07:53<08:37, 2488.17 examples/s]
Tokenizing train dataset:  29%|██▉       | 521421/1807941 [07:53<08:33, 2507.38 examples/s]
Tokenizing train dataset:  29%|██▉       | 521691/1807941 [07:53<08:22, 2558.39 examples/s]
Tokenizing train dataset:  29%|██▉       | 522039/1807941 [07:53<08:41, 2465.99 examples/s]
Tokenizing train dataset:  29%|██▉       | 522411/1807941 [07:53<08:42, 2461.63 examples/s]
Tokenizing train dataset:  29%|██▉       | 522682/1807941 [07:54<08:29, 2520.37 examples/s]
Tokenizing train dataset:  29%|██▉       | 523009/1807941 [07:54<08:55, 2399.75 examples/s]
Tokenizing train dataset:  29%|██▉       | 523331/1807941 [07:54<09:15, 2314.54 examples/s]
Tokenizing train dataset:  29%|██▉       | 523664/1807941 [07:54<09:23, 2279.60 examples/s]
Tokenizing train dataset:  29%|██▉       | 523950/1807941 [07:54<08:52, 2412.77 examples/s]
Tokenizing train dataset:  29%|██▉       | 524202/1807941 [07:54<08:46, 2438.12 examples/s]
Tokenizing train dataset:  29%|██▉       | 524491/1807941 [07:54<08:22, 2553.64 examples/s]
Tokenizing train dataset:  29%|██▉       | 524785/1807941 [07:54<08:03, 2656.10 examples/s]
Tokenizing train dataset:  29%|██▉       | 525175/1807941 [07:55<08:07, 2632.22 examples/s]
Tokenizing train dataset:  29%|██▉       | 525471/1807941 [07:55<07:52, 2715.13 examples/s]
Tokenizing train dataset:  29%|██▉       | 525778/1807941 [07:55<07:36, 2809.79 examples/s]
Tokenizing train dataset:  29%|██▉       | 526208/1807941 [07:55<07:33, 2828.77 examples/s]
Tokenizing train dataset:  29%|██▉       | 526592/1807941 [07:55<07:48, 2733.75 examples/s]
Tokenizing train dataset:  29%|██▉       | 526947/1807941 [07:55<08:10, 2610.92 examples/s]
Tokenizing train dataset:  29%|██▉       | 527301/1807941 [07:55<08:27, 2525.63 examples/s]
Tokenizing train dataset:  29%|██▉       | 527634/1807941 [07:56<08:48, 2423.11 examples/s]
Tokenizing train dataset:  29%|██▉       | 527972/1807941 [07:56<09:00, 2366.54 examples/s]
Tokenizing train dataset:  29%|██▉       | 528281/1807941 [07:56<09:23, 2271.43 examples/s]
Tokenizing train dataset:  29%|██▉       | 528541/1807941 [07:56<09:05, 2343.25 examples/s]
Tokenizing train dataset:  29%|██▉       | 528861/1807941 [07:56<09:23, 2271.77 examples/s]
Tokenizing train dataset:  29%|██▉       | 529146/1807941 [07:56<09:53, 2152.88 examples/s]
Tokenizing train dataset:  29%|██▉       | 529452/1807941 [07:56<10:04, 2116.30 examples/s]
Tokenizing train dataset:  29%|██▉       | 529750/1807941 [07:57<11:22, 1871.74 examples/s]
Tokenizing train dataset:  29%|██▉       | 530000/1807941 [07:57<10:52, 1958.84 examples/s]
Tokenizing train dataset:  29%|██▉       | 530287/1807941 [07:57<09:50, 2162.94 examples/s]
Tokenizing train dataset:  29%|██▉       | 530578/1807941 [07:57<10:12, 2086.34 examples/s]
Tokenizing train dataset:  29%|██▉       | 530812/1807941 [07:57<12:23, 1718.75 examples/s]
Tokenizing train dataset:  29%|██▉       | 531010/1807941 [07:57<13:17, 1600.90 examples/s]
Tokenizing train dataset:  29%|██▉       | 531253/1807941 [07:57<13:16, 1603.23 examples/s]
Tokenizing train dataset:  29%|██▉       | 531464/1807941 [07:58<15:14, 1395.19 examples/s]
Tokenizing train dataset:  29%|██▉       | 531648/1807941 [07:58<15:48, 1345.78 examples/s]
Tokenizing train dataset:  29%|██▉       | 531806/1807941 [07:58<16:51, 1261.19 examples/s]
Tokenizing train dataset:  29%|██▉       | 531940/1807941 [07:58<16:42, 1272.43 examples/s]
Tokenizing train dataset:  29%|██▉       | 532119/1807941 [07:58<17:54, 1187.42 examples/s]
Tokenizing train dataset:  29%|██▉       | 532289/1807941 [07:58<18:18, 1161.04 examples/s]
Tokenizing train dataset:  29%|██▉       | 532442/1807941 [07:59<19:06, 1112.56 examples/s]
Tokenizing train dataset:  29%|██▉       | 532603/1807941 [07:59<19:26, 1093.52 examples/s]
Tokenizing train dataset:  29%|██▉       | 532745/1807941 [07:59<20:25, 1040.24 examples/s]
Tokenizing train dataset:  29%|██▉       | 532882/1807941 [07:59<19:08, 1110.54 examples/s]
Tokenizing train dataset:  29%|██▉       | 533030/1807941 [07:59<17:45, 1196.84 examples/s]
Tokenizing train dataset:  29%|██▉       | 533177/1807941 [07:59<16:49, 1263.10 examples/s]
Tokenizing train dataset:  29%|██▉       | 533340/1807941 [07:59<17:45, 1196.67 examples/s]
Tokenizing train dataset:  30%|██▉       | 533497/1807941 [07:59<16:31, 1285.05 examples/s]
Tokenizing train dataset:  30%|██▉       | 533638/1807941 [08:00<16:07, 1317.24 examples/s]
Tokenizing train dataset:  30%|██▉       | 533823/1807941 [08:00<14:32, 1460.51 examples/s]
Tokenizing train dataset:  30%|██▉       | 534000/1807941 [08:00<16:14, 1306.72 examples/s]
Tokenizing train dataset:  30%|██▉       | 534163/1807941 [08:00<15:19, 1385.74 examples/s]
Tokenizing train dataset:  30%|██▉       | 534353/1807941 [08:00<15:49, 1341.23 examples/s]
Tokenizing train dataset:  30%|██▉       | 534591/1807941 [08:00<14:58, 1417.08 examples/s]
Tokenizing train dataset:  30%|██▉       | 534741/1807941 [08:00<14:50, 1429.71 examples/s]
Tokenizing train dataset:  30%|██▉       | 534900/1807941 [08:00<14:26, 1468.94 examples/s]
Tokenizing train dataset:  30%|██▉       | 535073/1807941 [08:01<15:41, 1352.14 examples/s]
Tokenizing train dataset:  30%|██▉       | 535276/1807941 [08:01<15:42, 1349.76 examples/s]
Tokenizing train dataset:  30%|██▉       | 535480/1807941 [08:01<15:49, 1340.16 examples/s]
Tokenizing train dataset:  30%|██▉       | 535654/1807941 [08:01<16:32, 1281.91 examples/s]
Tokenizing train dataset:  30%|██▉       | 535871/1807941 [08:01<16:00, 1324.67 examples/s]
Tokenizing train dataset:  30%|██▉       | 536035/1807941 [08:01<16:57, 1249.61 examples/s]
Tokenizing train dataset:  30%|██▉       | 536168/1807941 [08:01<16:44, 1265.55 examples/s]
Tokenizing train dataset:  30%|██▉       | 536340/1807941 [08:01<15:26, 1372.82 examples/s]
Tokenizing train dataset:  30%|██▉       | 536489/1807941 [08:02<17:09, 1234.53 examples/s]
Tokenizing train dataset:  30%|██▉       | 536643/1807941 [08:02<16:12, 1307.21 examples/s]
Tokenizing train dataset:  30%|██▉       | 536787/1807941 [08:02<15:48, 1339.52 examples/s]
Tokenizing train dataset:  30%|██▉       | 536990/1807941 [08:02<15:45, 1343.53 examples/s]
Tokenizing train dataset:  30%|██▉       | 537138/1807941 [08:02<15:24, 1374.47 examples/s]
Tokenizing train dataset:  30%|██▉       | 537343/1807941 [08:02<13:39, 1551.32 examples/s]
Tokenizing train dataset:  30%|██▉       | 537502/1807941 [08:02<13:34, 1559.31 examples/s]
Tokenizing train dataset:  30%|██▉       | 537768/1807941 [08:02<12:56, 1635.00 examples/s]
Tokenizing train dataset:  30%|██▉       | 538000/1807941 [08:03<11:48, 1793.36 examples/s]
Tokenizing train dataset:  30%|██▉       | 538248/1807941 [08:03<10:42, 1974.65 examples/s]
Tokenizing train dataset:  30%|██▉       | 538470/1807941 [08:03<10:22, 2037.82 examples/s]
Tokenizing train dataset:  30%|██▉       | 538761/1807941 [08:03<10:35, 1996.76 examples/s]
Tokenizing train dataset:  30%|██▉       | 539024/1807941 [08:03<11:05, 1908.11 examples/s]
Tokenizing train dataset:  30%|██▉       | 539317/1807941 [08:03<09:47, 2159.92 examples/s]
Tokenizing train dataset:  30%|██▉       | 539624/1807941 [08:03<09:59, 2116.65 examples/s]
Tokenizing train dataset:  30%|██▉       | 539929/1807941 [08:03<09:00, 2345.71 examples/s]
Tokenizing train dataset:  30%|██▉       | 540173/1807941 [08:04<08:55, 2368.43 examples/s]
Tokenizing train dataset:  30%|██▉       | 540427/1807941 [08:04<08:45, 2411.96 examples/s]
Tokenizing train dataset:  30%|██▉       | 540679/1807941 [08:04<08:39, 2440.43 examples/s]
Tokenizing train dataset:  30%|██▉       | 541000/1807941 [08:04<09:16, 2276.59 examples/s]
Tokenizing train dataset:  30%|██▉       | 541311/1807941 [08:04<09:34, 2205.08 examples/s]
Tokenizing train dataset:  30%|██▉       | 541630/1807941 [08:04<09:42, 2174.92 examples/s]
Tokenizing train dataset:  30%|██▉       | 541891/1807941 [08:04<09:16, 2275.35 examples/s]
Tokenizing train dataset:  30%|██▉       | 542250/1807941 [08:04<09:18, 2266.11 examples/s]
Tokenizing train dataset:  30%|███       | 542492/1807941 [08:05<09:10, 2300.79 examples/s]
Tokenizing train dataset:  30%|███       | 542747/1807941 [08:05<08:55, 2362.74 examples/s]
Tokenizing train dataset:  30%|███       | 542991/1807941 [08:05<08:51, 2380.88 examples/s]
Tokenizing train dataset:  30%|███       | 543285/1807941 [08:05<09:28, 2223.80 examples/s]
Tokenizing train dataset:  30%|███       | 543593/1807941 [08:05<09:45, 2160.70 examples/s]
Tokenizing train dataset:  30%|███       | 543841/1807941 [08:05<09:25, 2237.26 examples/s]
Tokenizing train dataset:  30%|███       | 544136/1807941 [08:05<09:50, 2140.62 examples/s]
Tokenizing train dataset:  30%|███       | 544372/1807941 [08:05<09:36, 2190.98 examples/s]
Tokenizing train dataset:  30%|███       | 544608/1807941 [08:06<10:41, 1969.07 examples/s]
Tokenizing train dataset:  30%|███       | 544852/1807941 [08:06<11:22, 1851.65 examples/s]
Tokenizing train dataset:  30%|███       | 545116/1807941 [08:06<11:35, 1815.43 examples/s]
Tokenizing train dataset:  30%|███       | 545321/1807941 [08:06<11:16, 1866.28 examples/s]
Tokenizing train dataset:  30%|███       | 545545/1807941 [08:06<10:45, 1954.44 examples/s]
Tokenizing train dataset:  30%|███       | 545772/1807941 [08:06<10:20, 2035.21 examples/s]
Tokenizing train dataset:  30%|███       | 545999/1807941 [08:06<10:01, 2098.18 examples/s]
Tokenizing train dataset:  30%|███       | 546245/1807941 [08:06<09:34, 2197.98 examples/s]
Tokenizing train dataset:  30%|███       | 546481/1807941 [08:06<09:24, 2234.47 examples/s]
Tokenizing train dataset:  30%|███       | 546794/1807941 [08:07<08:26, 2491.29 examples/s]
Tokenizing train dataset:  30%|███       | 547085/1807941 [08:07<08:03, 2610.43 examples/s]
Tokenizing train dataset:  30%|███       | 547404/1807941 [08:07<07:33, 2778.54 examples/s]
Tokenizing train dataset:  30%|███       | 547698/1807941 [08:07<07:26, 2822.88 examples/s]
Tokenizing train dataset:  30%|███       | 547988/1807941 [08:07<07:23, 2840.74 examples/s]
Tokenizing train dataset:  30%|███       | 548393/1807941 [08:07<07:32, 2781.27 examples/s]
Tokenizing train dataset:  30%|███       | 548687/1807941 [08:07<07:26, 2821.56 examples/s]
Tokenizing train dataset:  30%|███       | 548989/1807941 [08:07<07:18, 2873.66 examples/s]
Tokenizing train dataset:  30%|███       | 549382/1807941 [08:07<07:33, 2775.00 examples/s]
Tokenizing train dataset:  30%|███       | 549666/1807941 [08:08<07:31, 2789.54 examples/s]
Tokenizing train dataset:  30%|███       | 549953/1807941 [08:08<07:28, 2807.46 examples/s]
Tokenizing train dataset:  30%|███       | 550351/1807941 [08:08<07:37, 2748.90 examples/s]
Tokenizing train dataset:  30%|███       | 550651/1807941 [08:08<07:27, 2810.31 examples/s]
Tokenizing train dataset:  30%|███       | 550938/1807941 [08:08<07:26, 2817.49 examples/s]
Tokenizing train dataset:  30%|███       | 551348/1807941 [08:08<07:31, 2783.13 examples/s]
Tokenizing train dataset:  31%|███       | 551638/1807941 [08:08<07:26, 2812.17 examples/s]
Tokenizing train dataset:  31%|███       | 551929/1807941 [08:08<07:22, 2836.01 examples/s]
Tokenizing train dataset:  31%|███       | 552346/1807941 [08:09<07:26, 2811.97 examples/s]
Tokenizing train dataset:  31%|███       | 552647/1807941 [08:09<07:18, 2861.13 examples/s]
Tokenizing train dataset:  31%|███       | 552951/1807941 [08:09<07:11, 2906.75 examples/s]
Tokenizing train dataset:  31%|███       | 553373/1807941 [08:09<07:17, 2869.80 examples/s]
Tokenizing train dataset:  31%|███       | 553695/1807941 [08:09<07:03, 2958.76 examples/s]
Tokenizing train dataset:  31%|███       | 554000/1807941 [08:09<07:08, 2928.19 examples/s]
Tokenizing train dataset:  31%|███       | 554301/1807941 [08:09<07:05, 2949.07 examples/s]
Tokenizing train dataset:  31%|███       | 554606/1807941 [08:09<07:01, 2975.61 examples/s]
Tokenizing train dataset:  31%|███       | 555020/1807941 [08:09<07:13, 2890.98 examples/s]
Tokenizing train dataset:  31%|███       | 555319/1807941 [08:10<07:09, 2915.39 examples/s]
Tokenizing train dataset:  31%|███       | 555619/1807941 [08:10<07:06, 2936.41 examples/s]
Tokenizing train dataset:  31%|███       | 556047/1807941 [08:10<07:11, 2900.97 examples/s]
Tokenizing train dataset:  31%|███       | 556349/1807941 [08:10<07:07, 2929.10 examples/s]
Tokenizing train dataset:  31%|███       | 556795/1807941 [08:10<07:05, 2939.14 examples/s]
Tokenizing train dataset:  31%|███       | 557099/1807941 [08:10<07:02, 2962.72 examples/s]
Tokenizing train dataset:  31%|███       | 557543/1807941 [08:10<07:02, 2956.81 examples/s]
Tokenizing train dataset:  31%|███       | 557982/1807941 [08:10<07:04, 2943.48 examples/s]
Tokenizing train dataset:  31%|███       | 558401/1807941 [08:11<07:12, 2891.64 examples/s]
Tokenizing train dataset:  31%|███       | 558692/1807941 [08:11<07:11, 2894.54 examples/s]
Tokenizing train dataset:  31%|███       | 559000/1807941 [08:11<07:13, 2879.28 examples/s]
Tokenizing train dataset:  31%|███       | 559426/1807941 [08:11<07:16, 2860.03 examples/s]
Tokenizing train dataset:  31%|███       | 559714/1807941 [08:11<07:16, 2862.24 examples/s]
Tokenizing train dataset:  31%|███       | 560144/1807941 [08:11<07:20, 2835.35 examples/s]
Tokenizing train dataset:  31%|███       | 560454/1807941 [08:11<07:10, 2898.40 examples/s]
Tokenizing train dataset:  31%|███       | 560891/1807941 [08:11<07:10, 2896.81 examples/s]
Tokenizing train dataset:  31%|███       | 561309/1807941 [08:12<07:16, 2856.63 examples/s]
Tokenizing train dataset:  31%|███       | 561730/1807941 [08:12<07:19, 2836.50 examples/s]
Tokenizing train dataset:  31%|███       | 562033/1807941 [08:12<07:12, 2880.66 examples/s]
Tokenizing train dataset:  31%|███       | 562453/1807941 [08:12<07:16, 2850.74 examples/s]
Tokenizing train dataset:  31%|███       | 562743/1807941 [08:12<07:15, 2859.52 examples/s]
Tokenizing train dataset:  31%|███       | 563163/1807941 [08:12<07:24, 2800.18 examples/s]
Tokenizing train dataset:  31%|███       | 563468/1807941 [08:12<07:15, 2856.22 examples/s]
Tokenizing train dataset:  31%|███       | 563765/1807941 [08:12<07:11, 2882.55 examples/s]
Tokenizing train dataset:  31%|███       | 564059/1807941 [08:13<07:09, 2895.58 examples/s]
Tokenizing train dataset:  31%|███       | 564504/1807941 [08:13<07:05, 2919.98 examples/s]
Tokenizing train dataset:  31%|███       | 564831/1807941 [08:13<06:53, 3006.28 examples/s]
Tokenizing train dataset:  31%|███▏      | 565220/1807941 [08:13<07:14, 2857.11 examples/s]
Tokenizing train dataset:  31%|███▏      | 565530/1807941 [08:13<07:06, 2916.18 examples/s]
Tokenizing train dataset:  31%|███▏      | 565883/1807941 [08:13<06:43, 3075.80 examples/s]
Tokenizing train dataset:  31%|███▏      | 566325/1807941 [08:13<06:50, 3025.98 examples/s]
Tokenizing train dataset:  31%|███▏      | 566738/1807941 [08:13<07:03, 2932.17 examples/s]
Tokenizing train dataset:  31%|███▏      | 567071/1807941 [08:14<06:50, 3025.46 examples/s]
Tokenizing train dataset:  31%|███▏      | 567499/1807941 [08:14<06:58, 2963.14 examples/s]
Tokenizing train dataset:  31%|███▏      | 567825/1807941 [08:14<06:48, 3034.33 examples/s]
Tokenizing train dataset:  31%|███▏      | 568257/1807941 [08:14<06:56, 2976.19 examples/s]
Tokenizing train dataset:  31%|███▏      | 568577/1807941 [08:14<07:41, 2687.59 examples/s]
Tokenizing train dataset:  31%|███▏      | 568914/1807941 [08:14<10:47, 1913.65 examples/s]
Tokenizing train dataset:  31%|███▏      | 569148/1807941 [08:15<13:25, 1538.28 examples/s]
Tokenizing train dataset:  31%|███▏      | 569497/1807941 [08:15<11:02, 1870.00 examples/s]
Tokenizing train dataset:  32%|███▏      | 569805/1807941 [08:15<09:47, 2105.72 examples/s]
Tokenizing train dataset:  32%|███▏      | 570208/1807941 [08:15<09:04, 2274.03 examples/s]
Tokenizing train dataset:  32%|███▏      | 570559/1807941 [08:15<08:07, 2539.38 examples/s]
Tokenizing train dataset:  32%|███▏      | 570892/1807941 [08:15<07:33, 2726.42 examples/s]
Tokenizing train dataset:  32%|███▏      | 571317/1807941 [08:15<07:28, 2757.36 examples/s]
Tokenizing train dataset:  32%|███▏      | 571757/1807941 [08:16<07:19, 2811.60 examples/s]
Tokenizing train dataset:  32%|███▏      | 572080/1807941 [08:16<11:30, 1788.80 examples/s]
Tokenizing train dataset:  32%|███▏      | 572342/1807941 [08:16<13:27, 1530.97 examples/s]
Tokenizing train dataset:  32%|███▏      | 572552/1807941 [08:16<14:47, 1392.31 examples/s]
Tokenizing train dataset:  32%|███▏      | 572796/1807941 [08:17<15:23, 1337.80 examples/s]
Tokenizing train dataset:  32%|███▏      | 572951/1807941 [08:17<16:18, 1261.52 examples/s]
Tokenizing train dataset:  32%|███▏      | 573097/1807941 [08:17<17:58, 1144.49 examples/s]
Tokenizing train dataset:  32%|███▏      | 573227/1807941 [08:17<19:15, 1068.54 examples/s]
Tokenizing train dataset:  32%|███▏      | 573348/1807941 [08:17<18:47, 1094.51 examples/s]
Tokenizing train dataset:  32%|███▏      | 573492/1807941 [08:17<19:39, 1046.40 examples/s]
Tokenizing train dataset:  32%|███▏      | 573633/1807941 [08:17<20:22, 1009.89 examples/s]
Tokenizing train dataset:  32%|███▏      | 573775/1807941 [08:18<20:52, 985.26 examples/s] 
Tokenizing train dataset:  32%|███▏      | 573882/1807941 [08:18<20:33, 1000.49 examples/s]
Tokenizing train dataset:  32%|███▏      | 574002/1807941 [08:18<19:39, 1046.32 examples/s]
Tokenizing train dataset:  32%|███▏      | 574268/1807941 [08:18<14:09, 1452.71 examples/s]
Tokenizing train dataset:  32%|███▏      | 574520/1807941 [08:18<11:50, 1735.25 examples/s]
Tokenizing train dataset:  32%|███▏      | 574785/1807941 [08:18<10:22, 1980.58 examples/s]
Tokenizing train dataset:  32%|███▏      | 575039/1807941 [08:18<09:37, 2135.13 examples/s]
Tokenizing train dataset:  32%|███▏      | 575303/1807941 [08:18<09:01, 2278.18 examples/s]
Tokenizing train dataset:  32%|███▏      | 575556/1807941 [08:18<08:44, 2347.64 examples/s]
Tokenizing train dataset:  32%|███▏      | 575836/1807941 [08:18<08:17, 2476.74 examples/s]
Tokenizing train dataset:  32%|███▏      | 576097/1807941 [08:19<08:09, 2514.01 examples/s]
Tokenizing train dataset:  32%|███▏      | 576395/1807941 [08:19<11:13, 1829.56 examples/s]
Tokenizing train dataset:  32%|███▏      | 576623/1807941 [08:19<13:08, 1562.31 examples/s]
Tokenizing train dataset:  32%|███▏      | 576853/1807941 [08:19<15:47, 1299.52 examples/s]
Tokenizing train dataset:  32%|███▏      | 577061/1807941 [08:19<16:52, 1215.35 examples/s]
Tokenizing train dataset:  32%|███▏      | 577210/1807941 [08:20<17:43, 1157.04 examples/s]
Tokenizing train dataset:  32%|███▏      | 577340/1807941 [08:20<19:05, 1073.97 examples/s]
Tokenizing train dataset:  32%|███▏      | 577483/1807941 [08:20<19:49, 1034.17 examples/s]
Tokenizing train dataset:  32%|███▏      | 577618/1807941 [08:20<20:44, 988.53 examples/s] 
Tokenizing train dataset:  32%|███▏      | 577752/1807941 [08:20<21:21, 959.68 examples/s]
Tokenizing train dataset:  32%|███▏      | 578000/1807941 [08:20<16:37, 1233.37 examples/s]
Tokenizing train dataset:  32%|███▏      | 578228/1807941 [08:20<13:57, 1468.04 examples/s]
Tokenizing train dataset:  32%|███▏      | 578493/1807941 [08:21<11:46, 1739.64 examples/s]
Tokenizing train dataset:  32%|███▏      | 578686/1807941 [08:21<16:01, 1278.72 examples/s]
Tokenizing train dataset:  32%|███▏      | 578846/1807941 [08:21<18:30, 1107.13 examples/s]
Tokenizing train dataset:  32%|███▏      | 579000/1807941 [08:21<19:01, 1076.83 examples/s]
Tokenizing train dataset:  32%|███▏      | 579245/1807941 [08:21<15:07, 1353.82 examples/s]
Tokenizing train dataset:  32%|███▏      | 579426/1807941 [08:21<14:06, 1450.95 examples/s]
Tokenizing train dataset:  32%|███▏      | 579615/1807941 [08:22<16:25, 1246.63 examples/s]
Tokenizing train dataset:  32%|███▏      | 579805/1807941 [08:22<16:24, 1247.94 examples/s]
Tokenizing train dataset:  32%|███▏      | 579944/1807941 [08:22<16:02, 1275.35 examples/s]
Tokenizing train dataset:  32%|███▏      | 580104/1807941 [08:22<17:37, 1161.08 examples/s]
Tokenizing train dataset:  32%|███▏      | 580254/1807941 [08:22<18:26, 1109.40 examples/s]
Tokenizing train dataset:  32%|███▏      | 580391/1807941 [08:22<17:44, 1153.59 examples/s]
Tokenizing train dataset:  32%|███▏      | 580522/1807941 [08:22<17:11, 1190.14 examples/s]
Tokenizing train dataset:  32%|███▏      | 580687/1807941 [08:23<17:52, 1144.53 examples/s]
Tokenizing train dataset:  32%|███▏      | 580834/1807941 [08:23<16:44, 1221.30 examples/s]
Tokenizing train dataset:  32%|███▏      | 581000/1807941 [08:23<18:03, 1132.58 examples/s]
Tokenizing train dataset:  32%|███▏      | 581123/1807941 [08:23<17:42, 1155.07 examples/s]
Tokenizing train dataset:  32%|███▏      | 581265/1807941 [08:23<16:45, 1220.38 examples/s]
Tokenizing train dataset:  32%|███▏      | 581422/1807941 [08:23<17:47, 1149.25 examples/s]
Tokenizing train dataset:  32%|███▏      | 581604/1807941 [08:23<19:33, 1044.63 examples/s]
Tokenizing train dataset:  32%|███▏      | 581728/1807941 [08:23<18:48, 1086.44 examples/s]
Tokenizing train dataset:  32%|███▏      | 581852/1807941 [08:24<18:23, 1110.90 examples/s]
Tokenizing train dataset:  32%|███▏      | 581988/1807941 [08:24<19:45, 1034.37 examples/s]
Tokenizing train dataset:  32%|███▏      | 582119/1807941 [08:24<18:41, 1093.32 examples/s]
Tokenizing train dataset:  32%|███▏      | 582255/1807941 [08:24<22:15, 917.87 examples/s] 
Tokenizing train dataset:  32%|███▏      | 582411/1807941 [08:24<19:19, 1056.93 examples/s]
Tokenizing train dataset:  32%|███▏      | 582548/1807941 [08:24<20:20, 1003.88 examples/s]
Tokenizing train dataset:  32%|███▏      | 582736/1807941 [08:24<16:55, 1206.63 examples/s]
Tokenizing train dataset:  32%|███▏      | 583000/1807941 [08:25<13:28, 1515.98 examples/s]
Tokenizing train dataset:  32%|███▏      | 583253/1807941 [08:25<11:31, 1770.63 examples/s]
Tokenizing train dataset:  32%|███▏      | 583468/1807941 [08:25<12:30, 1630.63 examples/s]
Tokenizing train dataset:  32%|███▏      | 583680/1807941 [08:25<16:09, 1263.34 examples/s]
Tokenizing train dataset:  32%|███▏      | 583859/1807941 [08:25<17:59, 1134.30 examples/s]
Tokenizing train dataset:  32%|███▏      | 584000/1807941 [08:25<17:20, 1175.96 examples/s]
Tokenizing train dataset:  32%|███▏      | 584239/1807941 [08:25<14:11, 1436.69 examples/s]
Tokenizing train dataset:  32%|███▏      | 584447/1807941 [08:26<14:27, 1410.54 examples/s]
Tokenizing train dataset:  32%|███▏      | 584619/1807941 [08:26<15:23, 1324.48 examples/s]
Tokenizing train dataset:  32%|███▏      | 584763/1807941 [08:26<16:53, 1206.79 examples/s]
Tokenizing train dataset:  32%|███▏      | 584922/1807941 [08:26<19:25, 1049.25 examples/s]
Tokenizing train dataset:  32%|███▏      | 585047/1807941 [08:26<20:55, 973.66 examples/s] 
Tokenizing train dataset:  32%|███▏      | 585156/1807941 [08:26<20:25, 997.43 examples/s]
Tokenizing train dataset:  32%|███▏      | 585276/1807941 [08:26<19:38, 1037.07 examples/s]
Tokenizing train dataset:  32%|███▏      | 585420/1807941 [08:27<20:15, 1005.42 examples/s]
Tokenizing train dataset:  32%|███▏      | 585561/1807941 [08:27<20:51, 977.01 examples/s] 
Tokenizing train dataset:  32%|███▏      | 585662/1807941 [08:27<20:46, 980.33 examples/s]
Tokenizing train dataset:  32%|███▏      | 585771/1807941 [08:27<20:20, 1001.48 examples/s]
Tokenizing train dataset:  32%|███▏      | 585877/1807941 [08:27<20:08, 1010.86 examples/s]
Tokenizing train dataset:  32%|███▏      | 586065/1807941 [08:27<16:22, 1243.72 examples/s]
Tokenizing train dataset:  32%|███▏      | 586307/1807941 [08:27<12:58, 1568.80 examples/s]
Tokenizing train dataset:  32%|███▏      | 586529/1807941 [08:27<11:37, 1751.39 examples/s]
Tokenizing train dataset:  32%|███▏      | 586760/1807941 [08:27<10:39, 1908.22 examples/s]
Tokenizing train dataset:  32%|███▏      | 586968/1807941 [08:28<11:56, 1703.13 examples/s]
Tokenizing train dataset:  32%|███▏      | 587176/1807941 [08:28<14:24, 1412.15 examples/s]
Tokenizing train dataset:  32%|███▏      | 587369/1807941 [08:28<14:57, 1359.50 examples/s]
Tokenizing train dataset:  33%|███▎      | 587589/1807941 [08:28<13:08, 1548.35 examples/s]
Tokenizing train dataset:  33%|███▎      | 587768/1807941 [08:28<12:40, 1604.06 examples/s]
Tokenizing train dataset:  33%|███▎      | 587973/1807941 [08:28<14:55, 1363.06 examples/s]
Tokenizing train dataset:  33%|███▎      | 588165/1807941 [08:29<16:48, 1209.50 examples/s]
Tokenizing train dataset:  33%|███▎      | 588373/1807941 [08:29<16:12, 1254.47 examples/s]
Tokenizing train dataset:  33%|███▎      | 588594/1807941 [08:29<13:58, 1454.67 examples/s]
Tokenizing train dataset:  33%|███▎      | 588762/1807941 [08:29<13:29, 1506.24 examples/s]
Tokenizing train dataset:  33%|███▎      | 588927/1807941 [08:29<14:54, 1362.47 examples/s]
Tokenizing train dataset:  33%|███▎      | 589099/1807941 [08:29<15:47, 1286.08 examples/s]
Tokenizing train dataset:  33%|███▎      | 589299/1807941 [08:29<14:00, 1449.95 examples/s]
Tokenizing train dataset:  33%|███▎      | 589508/1807941 [08:29<12:37, 1608.11 examples/s]
Tokenizing train dataset:  33%|███▎      | 589749/1807941 [08:30<11:10, 1815.58 examples/s]
Tokenizing train dataset:  33%|███▎      | 589951/1807941 [08:30<10:52, 1865.60 examples/s]
Tokenizing train dataset:  33%|███▎      | 590216/1807941 [08:30<09:45, 2081.16 examples/s]
Tokenizing train dataset:  33%|███▎      | 590785/1807941 [08:30<06:31, 3108.54 examples/s]
Tokenizing train dataset:  33%|███▎      | 591435/1807941 [08:30<04:57, 4087.11 examples/s]
Tokenizing train dataset:  33%|███▎      | 592110/1807941 [08:30<04:09, 4863.89 examples/s]
Tokenizing train dataset:  33%|███▎      | 592813/1807941 [08:30<03:40, 5499.08 examples/s]
Tokenizing train dataset:  33%|███▎      | 593492/1807941 [08:30<03:26, 5877.79 examples/s]
Tokenizing train dataset:  33%|███▎      | 594172/1807941 [08:30<03:17, 6150.92 examples/s]
Tokenizing train dataset:  33%|███▎      | 594906/1807941 [08:30<03:06, 6502.06 examples/s]
Tokenizing train dataset:  33%|███▎      | 595591/1807941 [08:31<03:03, 6604.49 examples/s]
Tokenizing train dataset:  33%|███▎      | 596261/1807941 [08:31<03:02, 6630.14 examples/s]
Tokenizing train dataset:  33%|███▎      | 596929/1807941 [08:31<03:02, 6641.57 examples/s]
Tokenizing train dataset:  33%|███▎      | 597600/1807941 [08:31<03:01, 6657.90 examples/s]
Tokenizing train dataset:  33%|███▎      | 598269/1807941 [08:31<03:01, 6666.42 examples/s]
Tokenizing train dataset:  33%|███▎      | 598966/1807941 [08:31<02:58, 6756.64 examples/s]
Tokenizing train dataset:  33%|███▎      | 599646/1807941 [08:31<02:58, 6767.42 examples/s]
Tokenizing train dataset:  33%|███▎      | 600359/1807941 [08:31<02:57, 6814.35 examples/s]
Tokenizing train dataset:  33%|███▎      | 601062/1807941 [08:31<02:55, 6875.78 examples/s]
Tokenizing train dataset:  33%|███▎      | 601862/1807941 [08:32<03:37, 5540.71 examples/s]
Tokenizing train dataset:  33%|███▎      | 602584/1807941 [08:32<04:12, 4769.85 examples/s]
Tokenizing train dataset:  33%|███▎      | 603138/1807941 [08:32<04:30, 4446.28 examples/s]
Tokenizing train dataset:  33%|███▎      | 603741/1807941 [08:32<04:38, 4318.17 examples/s]
Tokenizing train dataset:  33%|███▎      | 604294/1807941 [08:32<04:51, 4124.02 examples/s]
Tokenizing train dataset:  33%|███▎      | 604905/1807941 [08:32<04:53, 4104.85 examples/s]
Tokenizing train dataset:  33%|███▎      | 605441/1807941 [08:32<05:05, 3941.93 examples/s]
Tokenizing train dataset:  34%|███▎      | 605990/1807941 [08:33<05:11, 3853.92 examples/s]
Tokenizing train dataset:  34%|███▎      | 606543/1807941 [08:33<05:16, 3799.74 examples/s]
Tokenizing train dataset:  34%|███▎      | 607076/1807941 [08:33<05:22, 3722.26 examples/s]
Tokenizing train dataset:  34%|███▎      | 607543/1807941 [08:33<05:39, 3533.92 examples/s]
Tokenizing train dataset:  34%|███▎      | 608030/1807941 [08:33<05:48, 3445.63 examples/s]
Tokenizing train dataset:  34%|███▎      | 608528/1807941 [08:33<05:52, 3404.84 examples/s]
Tokenizing train dataset:  34%|███▎      | 608886/1807941 [08:34<05:48, 3443.30 examples/s]
Tokenizing train dataset:  34%|███▎      | 609391/1807941 [08:34<05:51, 3414.25 examples/s]
Tokenizing train dataset:  34%|███▎      | 609744/1807941 [08:34<05:48, 3436.60 examples/s]
Tokenizing train dataset:  34%|███▍      | 610189/1807941 [08:34<06:05, 3275.02 examples/s]
Tokenizing train dataset:  34%|███▍      | 610680/1807941 [08:34<06:06, 3270.51 examples/s]
Tokenizing train dataset:  34%|███▍      | 611147/1807941 [08:34<06:17, 3168.65 examples/s]
Tokenizing train dataset:  34%|███▍      | 611480/1807941 [08:34<06:13, 3202.04 examples/s]
Tokenizing train dataset:  34%|███▍      | 611805/1807941 [08:34<06:12, 3211.17 examples/s]
Tokenizing train dataset:  34%|███▍      | 612284/1807941 [08:35<06:13, 3201.53 examples/s]
Tokenizing train dataset:  34%|███▍      | 612630/1807941 [08:35<06:06, 3263.37 examples/s]
Tokenizing train dataset:  34%|███▍      | 612971/1807941 [08:35<06:02, 3298.35 examples/s]
Tokenizing train dataset:  34%|███▍      | 613441/1807941 [08:35<06:09, 3236.42 examples/s]
Tokenizing train dataset:  34%|███▍      | 613768/1807941 [08:35<06:08, 3242.04 examples/s]
Tokenizing train dataset:  34%|███▍      | 614177/1807941 [08:35<06:30, 3056.46 examples/s]
Tokenizing train dataset:  34%|███▍      | 614509/1807941 [08:35<06:22, 3118.56 examples/s]
Tokenizing train dataset:  34%|███▍      | 614841/1807941 [08:35<06:16, 3169.18 examples/s]
Tokenizing train dataset:  34%|███▍      | 615263/1807941 [08:36<06:39, 2985.32 examples/s]
Tokenizing train dataset:  34%|███▍      | 615620/1807941 [08:36<06:20, 3130.42 examples/s]
Tokenizing train dataset:  34%|███▍      | 616006/1807941 [08:36<06:46, 2930.31 examples/s]
Tokenizing train dataset:  34%|███▍      | 616437/1807941 [08:36<06:49, 2908.28 examples/s]
Tokenizing train dataset:  34%|███▍      | 616792/1807941 [08:36<06:29, 3060.98 examples/s]
Tokenizing train dataset:  34%|███▍      | 617165/1807941 [08:36<06:08, 3230.32 examples/s]
Tokenizing train dataset:  34%|███▍      | 617549/1807941 [08:36<05:51, 3390.41 examples/s]
Tokenizing train dataset:  34%|███▍      | 617971/1807941 [08:36<06:14, 3175.57 examples/s]
Tokenizing train dataset:  34%|███▍      | 618458/1807941 [08:37<06:12, 3196.46 examples/s]
Tokenizing train dataset:  34%|███▍      | 618812/1807941 [08:37<06:02, 3278.33 examples/s]
Tokenizing train dataset:  34%|███▍      | 619302/1807941 [08:37<06:03, 3271.97 examples/s]
Tokenizing train dataset:  34%|███▍      | 619659/1807941 [08:37<05:55, 3342.05 examples/s]
Tokenizing train dataset:  34%|███▍      | 620028/1807941 [08:37<06:31, 3033.53 examples/s]
Tokenizing train dataset:  34%|███▍      | 620372/1807941 [08:37<06:19, 3130.34 examples/s]
Tokenizing train dataset:  34%|███▍      | 620821/1807941 [08:37<06:25, 3079.41 examples/s]
Tokenizing train dataset:  34%|███▍      | 621158/1807941 [08:37<06:25, 3076.71 examples/s]
Tokenizing train dataset:  34%|███▍      | 621501/1807941 [08:38<07:04, 2796.67 examples/s]
Tokenizing train dataset:  34%|███▍      | 621837/1807941 [08:38<06:45, 2926.28 examples/s]
Tokenizing train dataset:  34%|███▍      | 622187/1807941 [08:38<07:15, 2720.85 examples/s]
Tokenizing train dataset:  34%|███▍      | 622498/1807941 [08:38<07:01, 2814.26 examples/s]
Tokenizing train dataset:  34%|███▍      | 622825/1807941 [08:38<06:44, 2929.93 examples/s]
Tokenizing train dataset:  34%|███▍      | 623216/1807941 [08:38<07:01, 2811.46 examples/s]
Tokenizing train dataset:  34%|███▍      | 623519/1807941 [08:38<06:53, 2865.72 examples/s]
Tokenizing train dataset:  35%|███▍      | 623906/1807941 [08:38<07:08, 2762.33 examples/s]
Tokenizing train dataset:  35%|███▍      | 624207/1807941 [08:38<06:59, 2821.28 examples/s]
Tokenizing train dataset:  35%|███▍      | 624640/1807941 [08:39<06:56, 2840.91 examples/s]
Tokenizing train dataset:  35%|███▍      | 625000/1807941 [08:39<06:33, 3005.80 examples/s]
Tokenizing train dataset:  35%|███▍      | 625366/1807941 [08:39<06:12, 3171.00 examples/s]
Tokenizing train dataset:  35%|███▍      | 625855/1807941 [08:39<06:09, 3199.40 examples/s]
Tokenizing train dataset:  35%|███▍      | 626306/1807941 [08:39<07:00, 2810.33 examples/s]
Tokenizing train dataset:  35%|███▍      | 626667/1807941 [08:39<07:20, 2684.63 examples/s]
Tokenizing train dataset:  35%|███▍      | 627000/1807941 [08:40<07:47, 2527.05 examples/s]
Tokenizing train dataset:  35%|███▍      | 627351/1807941 [08:40<07:10, 2743.85 examples/s]
Tokenizing train dataset:  35%|███▍      | 627685/1807941 [08:40<06:49, 2885.70 examples/s]
Tokenizing train dataset:  35%|███▍      | 628080/1807941 [08:40<07:01, 2797.93 examples/s]
Tokenizing train dataset:  35%|███▍      | 628463/1807941 [08:40<07:14, 2716.18 examples/s]
Tokenizing train dataset:  35%|███▍      | 628819/1807941 [08:40<06:44, 2913.42 examples/s]
Tokenizing train dataset:  35%|███▍      | 629126/1807941 [08:40<07:30, 2615.36 examples/s]
Tokenizing train dataset:  35%|███▍      | 629475/1807941 [08:40<06:57, 2825.00 examples/s]
Tokenizing train dataset:  35%|███▍      | 629807/1807941 [08:41<07:30, 2612.63 examples/s]
Tokenizing train dataset:  35%|███▍      | 630194/1807941 [08:41<07:33, 2597.30 examples/s]
Tokenizing train dataset:  35%|███▍      | 630535/1807941 [08:41<07:02, 2788.93 examples/s]
Tokenizing train dataset:  35%|███▍      | 630943/1807941 [08:41<07:06, 2762.86 examples/s]
Tokenizing train dataset:  35%|███▍      | 631245/1807941 [08:41<10:27, 1874.16 examples/s]
Tokenizing train dataset:  35%|███▍      | 631520/1807941 [08:42<13:17, 1475.54 examples/s]
Tokenizing train dataset:  35%|███▍      | 631761/1807941 [08:42<15:04, 1300.95 examples/s]
Tokenizing train dataset:  35%|███▍      | 631988/1807941 [08:42<15:41, 1249.52 examples/s]
Tokenizing train dataset:  35%|███▍      | 632161/1807941 [08:42<17:12, 1139.19 examples/s]
Tokenizing train dataset:  35%|███▍      | 632306/1807941 [08:42<17:52, 1096.60 examples/s]
Tokenizing train dataset:  35%|███▍      | 632433/1807941 [08:42<19:00, 1030.67 examples/s]
Tokenizing train dataset:  35%|███▍      | 632590/1807941 [08:43<19:02, 1028.33 examples/s]
Tokenizing train dataset:  35%|███▍      | 632727/1807941 [08:43<19:47, 990.04 examples/s] 
Tokenizing train dataset:  35%|███▌      | 632865/1807941 [08:43<20:14, 967.77 examples/s]
Tokenizing train dataset:  35%|███▌      | 632965/1807941 [08:43<20:11, 970.19 examples/s]
Tokenizing train dataset:  35%|███▌      | 633097/1807941 [08:43<22:02, 888.07 examples/s]
Tokenizing train dataset:  35%|███▌      | 633291/1807941 [08:43<17:41, 1106.71 examples/s]
Tokenizing train dataset:  35%|███▌      | 633622/1807941 [08:43<12:04, 1621.87 examples/s]
Tokenizing train dataset:  35%|███▌      | 633864/1807941 [08:44<10:45, 1819.94 examples/s]
Tokenizing train dataset:  35%|███▌      | 634079/1807941 [08:44<10:16, 1905.02 examples/s]
Tokenizing train dataset:  35%|███▌      | 634355/1807941 [08:44<09:09, 2136.73 examples/s]
Tokenizing train dataset:  35%|███▌      | 634583/1807941 [08:44<08:59, 2174.78 examples/s]
Tokenizing train dataset:  35%|███▌      | 634865/1807941 [08:44<08:18, 2355.43 examples/s]
Tokenizing train dataset:  35%|███▌      | 635240/1807941 [08:44<08:18, 2353.62 examples/s]
Tokenizing train dataset:  35%|███▌      | 635644/1807941 [08:44<07:55, 2466.48 examples/s]
Tokenizing train dataset:  35%|███▌      | 635922/1807941 [08:44<07:41, 2541.91 examples/s]
Tokenizing train dataset:  35%|███▌      | 636272/1807941 [08:44<07:54, 2466.87 examples/s]
Tokenizing train dataset:  35%|███▌      | 636527/1807941 [08:45<07:51, 2485.25 examples/s]
Tokenizing train dataset:  35%|███▌      | 636782/1807941 [08:45<07:48, 2500.39 examples/s]
Tokenizing train dataset:  35%|███▌      | 637124/1807941 [08:45<08:04, 2416.25 examples/s]
Tokenizing train dataset:  35%|███▌      | 637484/1807941 [08:45<08:05, 2408.75 examples/s]
Tokenizing train dataset:  35%|███▌      | 637752/1807941 [08:45<07:53, 2473.16 examples/s]
Tokenizing train dataset:  35%|███▌      | 638059/1807941 [08:45<08:23, 2324.79 examples/s]
Tokenizing train dataset:  35%|███▌      | 638308/1807941 [08:45<08:14, 2363.67 examples/s]
Tokenizing train dataset:  35%|███▌      | 638575/1807941 [08:45<07:59, 2439.55 examples/s]
Tokenizing train dataset:  35%|███▌      | 638832/1807941 [08:46<07:53, 2470.73 examples/s]
Tokenizing train dataset:  35%|███▌      | 639082/1807941 [08:46<10:08, 1920.28 examples/s]
Tokenizing train dataset:  35%|███▌      | 639336/1807941 [08:46<12:55, 1507.81 examples/s]
Tokenizing train dataset:  35%|███▌      | 639544/1807941 [08:46<15:42, 1239.57 examples/s]
Tokenizing train dataset:  35%|███▌      | 639734/1807941 [08:46<17:00, 1144.47 examples/s]
Tokenizing train dataset:  35%|███▌      | 639891/1807941 [08:47<17:27, 1115.52 examples/s]
Tokenizing train dataset:  35%|███▌      | 640038/1807941 [08:47<19:46, 984.60 examples/s] 
Tokenizing train dataset:  35%|███▌      | 640153/1807941 [08:47<21:02, 924.90 examples/s]
Tokenizing train dataset:  35%|███▌      | 640306/1807941 [08:47<20:34, 945.64 examples/s]
Tokenizing train dataset:  35%|███▌      | 640436/1807941 [08:47<21:06, 921.58 examples/s]
Tokenizing train dataset:  35%|███▌      | 640539/1807941 [08:47<22:51, 851.16 examples/s]
Tokenizing train dataset:  35%|███▌      | 640737/1807941 [08:48<19:55, 976.20 examples/s]
Tokenizing train dataset:  35%|███▌      | 641000/1807941 [08:48<15:09, 1282.79 examples/s]
Tokenizing train dataset:  35%|███▌      | 641273/1807941 [08:48<12:05, 1608.68 examples/s]
Tokenizing train dataset:  35%|███▌      | 641539/1807941 [08:48<10:26, 1862.13 examples/s]
Tokenizing train dataset:  35%|███▌      | 641761/1807941 [08:48<09:57, 1952.20 examples/s]
Tokenizing train dataset:  36%|███▌      | 642000/1807941 [08:48<09:26, 2056.66 examples/s]
Tokenizing train dataset:  36%|███▌      | 642276/1807941 [08:48<08:38, 2249.42 examples/s]
Tokenizing train dataset:  36%|███▌      | 642549/1807941 [08:48<08:08, 2384.47 examples/s]
Tokenizing train dataset:  36%|███▌      | 642814/1807941 [08:49<12:38, 1535.14 examples/s]
Tokenizing train dataset:  36%|███▌      | 643019/1807941 [08:49<15:30, 1251.83 examples/s]
Tokenizing train dataset:  36%|███▌      | 643222/1807941 [08:49<13:56, 1392.14 examples/s]
Tokenizing train dataset:  36%|███▌      | 643464/1807941 [08:49<12:06, 1603.38 examples/s]
Tokenizing train dataset:  36%|███▌      | 643745/1807941 [08:49<10:20, 1876.77 examples/s]
Tokenizing train dataset:  36%|███▌      | 643999/1807941 [08:49<09:31, 2038.31 examples/s]
Tokenizing train dataset:  36%|███▌      | 644253/1807941 [08:49<08:57, 2165.00 examples/s]
Tokenizing train dataset:  36%|███▌      | 644521/1807941 [08:50<10:47, 1797.89 examples/s]
Tokenizing train dataset:  36%|███▌      | 644750/1807941 [08:50<11:22, 1704.49 examples/s]
Tokenizing train dataset:  36%|███▌      | 645000/1807941 [08:50<11:36, 1668.87 examples/s]
Tokenizing train dataset:  36%|███▌      | 645270/1807941 [08:50<11:21, 1707.15 examples/s]
Tokenizing train dataset:  36%|███▌      | 645499/1807941 [08:50<10:33, 1834.62 examples/s]
Tokenizing train dataset:  36%|███▌      | 645723/1807941 [08:50<12:28, 1551.92 examples/s]
Tokenizing train dataset:  36%|███▌      | 645961/1807941 [08:50<11:11, 1730.29 examples/s]
Tokenizing train dataset:  36%|███▌      | 646291/1807941 [08:51<11:25, 1693.91 examples/s]
Tokenizing train dataset:  36%|███▌      | 646547/1807941 [08:51<11:27, 1689.47 examples/s]
Tokenizing train dataset:  36%|███▌      | 646789/1807941 [08:51<11:38, 1662.39 examples/s]
Tokenizing train dataset:  36%|███▌      | 647000/1807941 [08:51<11:21, 1702.33 examples/s]
Tokenizing train dataset:  36%|███▌      | 647208/1807941 [08:51<12:04, 1601.05 examples/s]
Tokenizing train dataset:  36%|███▌      | 647414/1807941 [08:51<12:40, 1526.49 examples/s]
Tokenizing train dataset:  36%|███▌      | 647666/1807941 [08:51<11:03, 1747.81 examples/s]
Tokenizing train dataset:  36%|███▌      | 647857/1807941 [08:52<13:30, 1430.53 examples/s]
Tokenizing train dataset:  36%|███▌      | 648023/1807941 [08:52<15:57, 1211.23 examples/s]
Tokenizing train dataset:  36%|███▌      | 648163/1807941 [08:52<15:30, 1246.53 examples/s]
Tokenizing train dataset:  36%|███▌      | 648388/1807941 [08:52<13:08, 1471.26 examples/s]
Tokenizing train dataset:  36%|███▌      | 648588/1807941 [08:52<13:36, 1420.59 examples/s]
Tokenizing train dataset:  36%|███▌      | 648752/1807941 [08:52<14:41, 1314.48 examples/s]
Tokenizing train dataset:  36%|███▌      | 648962/1807941 [08:52<12:54, 1495.52 examples/s]
Tokenizing train dataset:  36%|███▌      | 649137/1807941 [08:53<14:02, 1375.89 examples/s]
Tokenizing train dataset:  36%|███▌      | 649310/1807941 [08:53<16:26, 1173.95 examples/s]
Tokenizing train dataset:  36%|███▌      | 649492/1807941 [08:53<14:43, 1311.33 examples/s]
Tokenizing train dataset:  36%|███▌      | 649641/1807941 [08:53<16:02, 1203.09 examples/s]
Tokenizing train dataset:  36%|███▌      | 649885/1807941 [08:53<12:59, 1485.11 examples/s]
Tokenizing train dataset:  36%|███▌      | 650130/1807941 [08:53<11:28, 1681.91 examples/s]
Tokenizing train dataset:  36%|███▌      | 650418/1807941 [08:53<09:42, 1987.20 examples/s]
Tokenizing train dataset:  36%|███▌      | 650697/1807941 [08:54<08:46, 2199.30 examples/s]
Tokenizing train dataset:  36%|███▌      | 650972/1807941 [08:54<08:12, 2349.27 examples/s]
Tokenizing train dataset:  36%|███▌      | 651235/1807941 [08:54<10:13, 1885.37 examples/s]
Tokenizing train dataset:  36%|███▌      | 651466/1807941 [08:54<13:16, 1452.83 examples/s]
Tokenizing train dataset:  36%|███▌      | 651664/1807941 [08:54<14:55, 1291.13 examples/s]
Tokenizing train dataset:  36%|███▌      | 651873/1807941 [08:54<13:21, 1443.21 examples/s]
Tokenizing train dataset:  36%|███▌      | 652059/1807941 [08:54<12:35, 1530.15 examples/s]
Tokenizing train dataset:  36%|███▌      | 652325/1807941 [08:55<10:44, 1793.64 examples/s]
Tokenizing train dataset:  36%|███▌      | 652612/1807941 [08:55<09:20, 2061.43 examples/s]
Tokenizing train dataset:  36%|███▌      | 652840/1807941 [08:55<09:06, 2113.33 examples/s]
Tokenizing train dataset:  36%|███▌      | 653210/1807941 [08:55<08:36, 2237.55 examples/s]
Tokenizing train dataset:  36%|███▌      | 653476/1807941 [08:55<11:24, 1687.05 examples/s]
Tokenizing train dataset:  36%|███▌      | 653690/1807941 [08:55<14:15, 1349.82 examples/s]
Tokenizing train dataset:  36%|███▌      | 653883/1807941 [08:56<16:57, 1133.87 examples/s]
Tokenizing train dataset:  36%|███▌      | 654058/1807941 [08:56<18:27, 1041.52 examples/s]
Tokenizing train dataset:  36%|███▌      | 654204/1807941 [08:56<18:46, 1024.23 examples/s]
Tokenizing train dataset:  36%|███▌      | 654325/1807941 [08:56<18:15, 1053.51 examples/s]
Tokenizing train dataset:  36%|███▌      | 654464/1807941 [08:56<19:00, 1010.94 examples/s]
Tokenizing train dataset:  36%|███▌      | 654603/1807941 [08:56<19:32, 983.65 examples/s] 
Tokenizing train dataset:  36%|███▌      | 654757/1807941 [08:57<19:21, 992.89 examples/s]
Tokenizing train dataset:  36%|███▌      | 654900/1807941 [08:57<19:37, 979.22 examples/s]
Tokenizing train dataset:  36%|███▌      | 655052/1807941 [08:57<17:32, 1095.70 examples/s]
Tokenizing train dataset:  36%|███▌      | 655316/1807941 [08:57<13:10, 1458.43 examples/s]
Tokenizing train dataset:  36%|███▋      | 655514/1807941 [08:57<12:05, 1588.81 examples/s]
Tokenizing train dataset:  36%|███▋      | 655737/1807941 [08:57<10:56, 1755.90 examples/s]
Tokenizing train dataset:  36%|███▋      | 656000/1807941 [08:57<09:54, 1936.81 examples/s]
Tokenizing train dataset:  36%|███▋      | 656215/1807941 [08:57<09:37, 1993.09 examples/s]
Tokenizing train dataset:  36%|███▋      | 656525/1807941 [08:58<09:30, 2018.90 examples/s]
Tokenizing train dataset:  36%|███▋      | 656767/1807941 [08:58<09:02, 2120.07 examples/s]
Tokenizing train dataset:  36%|███▋      | 656999/1807941 [08:58<08:50, 2170.91 examples/s]
Tokenizing train dataset:  36%|███▋      | 657285/1807941 [08:58<09:16, 2069.44 examples/s]
Tokenizing train dataset:  36%|███▋      | 657520/1807941 [08:58<08:58, 2137.92 examples/s]
Tokenizing train dataset:  36%|███▋      | 657873/1807941 [08:58<08:40, 2208.85 examples/s]
Tokenizing train dataset:  36%|███▋      | 658191/1807941 [08:58<08:49, 2173.27 examples/s]
Tokenizing train dataset:  36%|███▋      | 658533/1807941 [08:58<08:40, 2206.20 examples/s]
Tokenizing train dataset:  36%|███▋      | 658789/1807941 [08:59<10:24, 1840.50 examples/s]
Tokenizing train dataset:  36%|███▋      | 658992/1807941 [08:59<11:17, 1695.55 examples/s]
Tokenizing train dataset:  36%|███▋      | 659190/1807941 [08:59<12:05, 1583.30 examples/s]
Tokenizing train dataset:  36%|███▋      | 659397/1807941 [08:59<12:35, 1519.28 examples/s]
Tokenizing train dataset:  36%|███▋      | 659596/1807941 [08:59<11:49, 1619.15 examples/s]
Tokenizing train dataset:  36%|███▋      | 659855/1807941 [08:59<10:21, 1847.90 examples/s]
Tokenizing train dataset:  37%|███▋      | 660068/1807941 [08:59<09:58, 1916.94 examples/s]
Tokenizing train dataset:  37%|███▋      | 660278/1807941 [08:59<09:44, 1963.93 examples/s]
Tokenizing train dataset:  37%|███▋      | 660511/1807941 [09:00<11:53, 1607.05 examples/s]
Tokenizing train dataset:  37%|███▋      | 660719/1807941 [09:00<13:50, 1381.72 examples/s]
Tokenizing train dataset:  37%|███▋      | 660932/1807941 [09:00<13:45, 1390.05 examples/s]
Tokenizing train dataset:  37%|███▋      | 661092/1807941 [09:00<13:32, 1410.74 examples/s]
Tokenizing train dataset:  37%|███▋      | 661306/1807941 [09:00<12:07, 1576.45 examples/s]
Tokenizing train dataset:  37%|███▋      | 661521/1807941 [09:00<11:07, 1716.82 examples/s]
Tokenizing train dataset:  37%|███▋      | 661734/1807941 [09:00<10:28, 1824.08 examples/s]
Tokenizing train dataset:  37%|███▋      | 661927/1807941 [09:01<10:20, 1847.69 examples/s]
Tokenizing train dataset:  37%|███▋      | 662174/1807941 [09:01<10:51, 1758.93 examples/s]
Tokenizing train dataset:  37%|███▋      | 662395/1807941 [09:01<10:11, 1871.91 examples/s]
Tokenizing train dataset:  37%|███▋      | 662612/1807941 [09:01<09:46, 1951.63 examples/s]
Tokenizing train dataset:  37%|███▋      | 662922/1807941 [09:01<09:35, 1989.66 examples/s]
Tokenizing train dataset:  37%|███▋      | 663187/1807941 [09:01<09:59, 1909.35 examples/s]
Tokenizing train dataset:  37%|███▋      | 663461/1807941 [09:01<10:12, 1869.92 examples/s]
Tokenizing train dataset:  37%|███▋      | 663724/1807941 [09:02<10:25, 1830.33 examples/s]
Tokenizing train dataset:  37%|███▋      | 663915/1807941 [09:02<10:19, 1846.03 examples/s]
Tokenizing train dataset:  37%|███▋      | 664111/1807941 [09:02<10:15, 1859.73 examples/s]
Tokenizing train dataset:  37%|███▋      | 664338/1807941 [09:02<09:42, 1962.33 examples/s]
Tokenizing train dataset:  37%|███▋      | 664568/1807941 [09:02<09:16, 2053.48 examples/s]
Tokenizing train dataset:  37%|███▋      | 664810/1807941 [09:02<08:51, 2152.47 examples/s]
Tokenizing train dataset:  37%|███▋      | 665140/1807941 [09:02<08:47, 2167.89 examples/s]
Tokenizing train dataset:  37%|███▋      | 665384/1807941 [09:02<08:30, 2237.23 examples/s]
Tokenizing train dataset:  37%|███▋      | 665627/1807941 [09:02<08:19, 2286.31 examples/s]
Tokenizing train dataset:  37%|███▋      | 665865/1807941 [09:02<08:14, 2310.21 examples/s]
Tokenizing train dataset:  37%|███▋      | 666198/1807941 [09:03<08:22, 2272.95 examples/s]
Tokenizing train dataset:  37%|███▋      | 666435/1807941 [09:03<08:17, 2296.66 examples/s]
Tokenizing train dataset:  37%|███▋      | 666679/1807941 [09:03<08:10, 2328.52 examples/s]
Tokenizing train dataset:  37%|███▋      | 666917/1807941 [09:03<08:07, 2341.00 examples/s]
Tokenizing train dataset:  37%|███▋      | 667243/1807941 [09:03<08:21, 2274.84 examples/s]
Tokenizing train dataset:  37%|███▋      | 667480/1807941 [09:03<08:16, 2299.08 examples/s]
Tokenizing train dataset:  37%|███▋      | 667721/1807941 [09:03<08:10, 2326.19 examples/s]
Tokenizing train dataset:  37%|███▋      | 667967/1807941 [09:03<08:03, 2359.74 examples/s]
Tokenizing train dataset:  37%|███▋      | 668295/1807941 [09:04<08:17, 2291.65 examples/s]
Tokenizing train dataset:  37%|███▋      | 668540/1807941 [09:04<08:08, 2330.79 examples/s]
Tokenizing train dataset:  37%|███▋      | 668779/1807941 [09:04<08:06, 2343.43 examples/s]
Tokenizing train dataset:  37%|███▋      | 669119/1807941 [09:04<08:17, 2289.30 examples/s]
Tokenizing train dataset:  37%|███▋      | 669361/1807941 [09:04<08:10, 2320.86 examples/s]
Tokenizing train dataset:  37%|███▋      | 669601/1807941 [09:04<08:06, 2339.14 examples/s]
Tokenizing train dataset:  37%|███▋      | 669843/1807941 [09:04<08:02, 2357.81 examples/s]
Tokenizing train dataset:  37%|███▋      | 670175/1807941 [09:04<08:14, 2299.21 examples/s]
Tokenizing train dataset:  37%|███▋      | 670419/1807941 [09:04<08:07, 2332.10 examples/s]
Tokenizing train dataset:  37%|███▋      | 670662/1807941 [09:05<08:02, 2356.94 examples/s]
Tokenizing train dataset:  37%|███▋      | 670905/1807941 [09:05<07:59, 2373.69 examples/s]
Tokenizing train dataset:  37%|███▋      | 671236/1807941 [09:05<08:14, 2298.84 examples/s]
Tokenizing train dataset:  37%|███▋      | 671471/1807941 [09:05<08:12, 2307.81 examples/s]
Tokenizing train dataset:  37%|███▋      | 671712/1807941 [09:05<08:07, 2332.02 examples/s]
Tokenizing train dataset:  37%|███▋      | 671952/1807941 [09:05<08:04, 2346.54 examples/s]
Tokenizing train dataset:  37%|███▋      | 672273/1807941 [09:05<08:21, 2265.94 examples/s]
Tokenizing train dataset:  37%|███▋      | 672510/1807941 [09:05<08:15, 2289.83 examples/s]
Tokenizing train dataset:  37%|███▋      | 672747/1807941 [09:05<08:11, 2307.33 examples/s]
Tokenizing train dataset:  37%|███▋      | 672990/1807941 [09:06<08:05, 2336.63 examples/s]
Tokenizing train dataset:  37%|███▋      | 673320/1807941 [09:06<08:16, 2283.46 examples/s]
Tokenizing train dataset:  37%|███▋      | 673562/1807941 [09:06<08:09, 2316.25 examples/s]
Tokenizing train dataset:  37%|███▋      | 673798/1807941 [09:06<08:07, 2326.38 examples/s]
Tokenizing train dataset:  37%|███▋      | 674134/1807941 [09:06<08:14, 2291.09 examples/s]
Tokenizing train dataset:  37%|███▋      | 674379/1807941 [09:06<08:06, 2329.10 examples/s]
Tokenizing train dataset:  37%|███▋      | 674619/1807941 [09:06<08:03, 2345.33 examples/s]
Tokenizing train dataset:  37%|███▋      | 674973/1807941 [09:06<08:02, 2347.50 examples/s]
Tokenizing train dataset:  37%|███▋      | 675305/1807941 [09:07<08:12, 2298.59 examples/s]
Tokenizing train dataset:  37%|███▋      | 675556/1807941 [09:07<08:02, 2348.50 examples/s]
Tokenizing train dataset:  37%|███▋      | 675809/1807941 [09:07<07:52, 2394.42 examples/s]
Tokenizing train dataset:  37%|███▋      | 676169/1807941 [09:07<07:52, 2393.94 examples/s]
Tokenizing train dataset:  37%|███▋      | 676430/1807941 [09:07<07:42, 2444.86 examples/s]
Tokenizing train dataset:  37%|███▋      | 676687/1807941 [09:07<07:37, 2475.15 examples/s]
Tokenizing train dataset:  37%|███▋      | 676946/1807941 [09:07<07:31, 2506.31 examples/s]
Tokenizing train dataset:  37%|███▋      | 677306/1807941 [09:07<07:38, 2465.23 examples/s]
Tokenizing train dataset:  37%|███▋      | 677691/1807941 [09:07<07:32, 2496.66 examples/s]
Tokenizing train dataset:  38%|███▊      | 678000/1807941 [09:08<07:15, 2591.61 examples/s]
Tokenizing train dataset:  38%|███▊      | 678314/1807941 [09:08<06:53, 2729.09 examples/s]
Tokenizing train dataset:  38%|███▊      | 678656/1807941 [09:08<06:27, 2913.97 examples/s]
Tokenizing train dataset:  38%|███▊      | 678997/1807941 [09:08<06:10, 3047.80 examples/s]
Tokenizing train dataset:  38%|███▊      | 679468/1807941 [09:08<06:06, 3080.23 examples/s]
Tokenizing train dataset:  38%|███▊      | 679808/1807941 [09:08<05:56, 3161.17 examples/s]
Tokenizing train dataset:  38%|███▊      | 680297/1807941 [09:08<05:53, 3193.19 examples/s]
Tokenizing train dataset:  38%|███▊      | 680637/1807941 [09:08<05:47, 3242.44 examples/s]
Tokenizing train dataset:  38%|███▊      | 680985/1807941 [09:09<05:41, 3302.33 examples/s]
Tokenizing train dataset:  38%|███▊      | 681465/1807941 [09:09<05:45, 3262.52 examples/s]
Tokenizing train dataset:  38%|███▊      | 681871/1807941 [09:09<06:07, 3067.66 examples/s]
Tokenizing train dataset:  38%|███▊      | 682212/1807941 [09:09<07:26, 2523.93 examples/s]
Tokenizing train dataset:  38%|███▊      | 682551/1807941 [09:09<08:26, 2223.44 examples/s]
Tokenizing train dataset:  38%|███▊      | 682857/1807941 [09:09<08:38, 2170.76 examples/s]
Tokenizing train dataset:  38%|███▊      | 683128/1807941 [09:10<09:04, 2066.20 examples/s]
Tokenizing train dataset:  38%|███▊      | 683351/1807941 [09:10<08:55, 2098.76 examples/s]
Tokenizing train dataset:  38%|███▊      | 683585/1807941 [09:10<08:42, 2153.93 examples/s]
Tokenizing train dataset:  38%|███▊      | 683822/1807941 [09:10<08:29, 2206.04 examples/s]
Tokenizing train dataset:  38%|███▊      | 684159/1807941 [09:10<08:26, 2216.87 examples/s]
Tokenizing train dataset:  38%|███▊      | 684405/1807941 [09:10<08:13, 2276.31 examples/s]
Tokenizing train dataset:  38%|███▊      | 684659/1807941 [09:10<07:59, 2343.79 examples/s]
Tokenizing train dataset:  38%|███▊      | 684905/1807941 [09:10<07:52, 2374.45 examples/s]
Tokenizing train dataset:  38%|███▊      | 685244/1807941 [09:10<08:02, 2327.81 examples/s]
Tokenizing train dataset:  38%|███▊      | 685494/1807941 [09:11<07:53, 2370.91 examples/s]
Tokenizing train dataset:  38%|███▊      | 685746/1807941 [09:11<10:58, 1703.57 examples/s]
Tokenizing train dataset:  38%|███▊      | 685955/1807941 [09:11<12:45, 1465.73 examples/s]
Tokenizing train dataset:  38%|███▊      | 686130/1807941 [09:11<14:49, 1260.85 examples/s]
Tokenizing train dataset:  38%|███▊      | 686324/1807941 [09:11<16:03, 1164.50 examples/s]
Tokenizing train dataset:  38%|███▊      | 686466/1807941 [09:12<16:57, 1102.40 examples/s]
Tokenizing train dataset:  38%|███▊      | 686609/1807941 [09:12<17:39, 1058.74 examples/s]
Tokenizing train dataset:  38%|███▊      | 686771/1807941 [09:12<17:36, 1060.96 examples/s]
Tokenizing train dataset:  38%|███▊      | 686891/1807941 [09:12<18:58, 984.88 examples/s] 
Tokenizing train dataset:  38%|███▊      | 687000/1807941 [09:12<19:23, 963.36 examples/s]
Tokenizing train dataset:  38%|███▊      | 687151/1807941 [09:12<19:14, 971.02 examples/s]
Tokenizing train dataset:  38%|███▊      | 687292/1807941 [09:12<19:34, 954.39 examples/s]
Tokenizing train dataset:  38%|███▊      | 687440/1807941 [09:13<19:31, 956.57 examples/s]
Tokenizing train dataset:  38%|███▊      | 687580/1807941 [09:13<19:49, 941.79 examples/s]
Tokenizing train dataset:  38%|███▊      | 687686/1807941 [09:13<19:20, 965.37 examples/s]
Tokenizing train dataset:  38%|███▊      | 687792/1807941 [09:13<19:02, 980.85 examples/s]
Tokenizing train dataset:  38%|███▊      | 687942/1807941 [09:13<19:09, 974.49 examples/s]
Tokenizing train dataset:  38%|███▊      | 688082/1807941 [09:13<19:50, 941.02 examples/s]
Tokenizing train dataset:  38%|███▊      | 688217/1807941 [09:13<20:27, 911.98 examples/s]
Tokenizing train dataset:  38%|███▊      | 688313/1807941 [09:13<20:22, 915.50 examples/s]
Tokenizing train dataset:  38%|███▊      | 688406/1807941 [09:14<20:21, 916.49 examples/s]
Tokenizing train dataset:  38%|███▊      | 688510/1807941 [09:14<19:46, 943.35 examples/s]
Tokenizing train dataset:  38%|███▊      | 688666/1807941 [09:14<19:07, 975.04 examples/s]
Tokenizing train dataset:  38%|███▊      | 688815/1807941 [09:14<19:03, 978.26 examples/s]
Tokenizing train dataset:  38%|███▊      | 688921/1807941 [09:14<18:42, 996.75 examples/s]
Tokenizing train dataset:  38%|███▊      | 689057/1807941 [09:14<21:48, 855.35 examples/s]
Tokenizing train dataset:  38%|███▊      | 689158/1807941 [09:14<23:19, 799.44 examples/s]
Tokenizing train dataset:  38%|███▊      | 689386/1807941 [09:15<16:31, 1127.61 examples/s]
Tokenizing train dataset:  38%|███▊      | 689694/1807941 [09:15<11:40, 1596.65 examples/s]
Tokenizing train dataset:  38%|███▊      | 689985/1807941 [09:15<09:39, 1930.69 examples/s]
Tokenizing train dataset:  38%|███▊      | 690230/1807941 [09:15<09:00, 2067.10 examples/s]
Tokenizing train dataset:  38%|███▊      | 690482/1807941 [09:15<08:29, 2191.12 examples/s]
Tokenizing train dataset:  38%|███▊      | 690758/1807941 [09:15<07:55, 2349.45 examples/s]
Tokenizing train dataset:  38%|███▊      | 691003/1807941 [09:15<07:49, 2377.15 examples/s]
Tokenizing train dataset:  38%|███▊      | 691277/1807941 [09:15<07:30, 2481.30 examples/s]
Tokenizing train dataset:  38%|███▊      | 691548/1807941 [09:15<07:18, 2544.86 examples/s]
Tokenizing train dataset:  38%|███▊      | 691821/1807941 [09:15<07:10, 2593.57 examples/s]
Tokenizing train dataset:  38%|███▊      | 692186/1807941 [09:16<07:21, 2525.82 examples/s]
Tokenizing train dataset:  38%|███▊      | 692453/1807941 [09:16<08:15, 2250.20 examples/s]
Tokenizing train dataset:  38%|███▊      | 692691/1807941 [09:16<08:09, 2279.76 examples/s]
Tokenizing train dataset:  38%|███▊      | 692939/1807941 [09:16<09:01, 2057.83 examples/s]
Tokenizing train dataset:  38%|███▊      | 693224/1807941 [09:16<09:18, 1997.54 examples/s]
Tokenizing train dataset:  38%|███▊      | 693493/1807941 [09:16<10:42, 1735.06 examples/s]
Tokenizing train dataset:  38%|███▊      | 693729/1807941 [09:17<11:00, 1685.95 examples/s]
Tokenizing train dataset:  38%|███▊      | 693952/1807941 [09:17<11:30, 1614.15 examples/s]
Tokenizing train dataset:  38%|███▊      | 694168/1807941 [09:17<11:54, 1559.11 examples/s]
Tokenizing train dataset:  38%|███▊      | 694340/1807941 [09:17<11:39, 1591.87 examples/s]
Tokenizing train dataset:  38%|███▊      | 694582/1807941 [09:17<11:39, 1590.78 examples/s]
Tokenizing train dataset:  38%|███▊      | 694811/1807941 [09:17<11:51, 1563.64 examples/s]
Tokenizing train dataset:  38%|███▊      | 694973/1807941 [09:17<11:47, 1573.28 examples/s]
Tokenizing train dataset:  38%|███▊      | 695188/1807941 [09:17<12:10, 1523.32 examples/s]
Tokenizing train dataset:  38%|███▊      | 695348/1807941 [09:18<12:10, 1522.68 examples/s]
Tokenizing train dataset:  38%|███▊      | 695521/1807941 [09:18<11:47, 1572.60 examples/s]
Tokenizing train dataset:  38%|███▊      | 695750/1807941 [09:18<11:59, 1546.10 examples/s]
Tokenizing train dataset:  38%|███▊      | 695940/1807941 [09:18<11:21, 1632.11 examples/s]
Tokenizing train dataset:  39%|███▊      | 696112/1807941 [09:18<12:41, 1460.26 examples/s]
Tokenizing train dataset:  39%|███▊      | 696295/1807941 [09:18<11:59, 1545.98 examples/s]
Tokenizing train dataset:  39%|███▊      | 696476/1807941 [09:18<11:30, 1609.26 examples/s]
Tokenizing train dataset:  39%|███▊      | 696663/1807941 [09:18<11:02, 1676.41 examples/s]
Tokenizing train dataset:  39%|███▊      | 696903/1807941 [09:19<11:15, 1644.84 examples/s]
Tokenizing train dataset:  39%|███▊      | 697150/1807941 [09:19<11:16, 1643.03 examples/s]
Tokenizing train dataset:  39%|███▊      | 697331/1807941 [09:19<11:00, 1680.73 examples/s]
Tokenizing train dataset:  39%|███▊      | 697503/1807941 [09:19<10:57, 1689.91 examples/s]
Tokenizing train dataset:  39%|███▊      | 697676/1807941 [09:19<10:54, 1696.51 examples/s]
Tokenizing train dataset:  39%|███▊      | 697859/1807941 [09:19<10:43, 1725.03 examples/s]
Tokenizing train dataset:  39%|███▊      | 698092/1807941 [09:19<11:12, 1650.04 examples/s]
Tokenizing train dataset:  39%|███▊      | 698278/1807941 [09:19<10:51, 1702.55 examples/s]
Tokenizing train dataset:  39%|███▊      | 698491/1807941 [09:19<10:10, 1817.13 examples/s]
Tokenizing train dataset:  39%|███▊      | 698723/1807941 [09:20<09:28, 1952.75 examples/s]
Tokenizing train dataset:  39%|███▊      | 698942/1807941 [09:20<09:11, 2011.50 examples/s]
Tokenizing train dataset:  39%|███▊      | 699185/1807941 [09:20<09:55, 1860.92 examples/s]
Tokenizing train dataset:  39%|███▊      | 699456/1807941 [09:20<10:02, 1839.64 examples/s]
Tokenizing train dataset:  39%|███▊      | 699687/1807941 [09:20<10:38, 1735.70 examples/s]
Tokenizing train dataset:  39%|███▊      | 699871/1807941 [09:20<10:29, 1759.64 examples/s]
Tokenizing train dataset:  39%|███▊      | 700094/1807941 [09:20<11:11, 1648.92 examples/s]
Tokenizing train dataset:  39%|███▊      | 700272/1807941 [09:20<11:00, 1675.76 examples/s]
Tokenizing train dataset:  39%|███▊      | 700451/1807941 [09:21<10:50, 1702.85 examples/s]
Tokenizing train dataset:  39%|███▉      | 700701/1807941 [09:21<10:56, 1687.38 examples/s]
Tokenizing train dataset:  39%|███▉      | 700921/1807941 [09:21<11:27, 1610.45 examples/s]
Tokenizing train dataset:  39%|███▉      | 701087/1807941 [09:21<11:41, 1578.92 examples/s]
Tokenizing train dataset:  39%|███▉      | 701249/1807941 [09:21<11:38, 1584.75 examples/s]
Tokenizing train dataset:  39%|███▉      | 701424/1807941 [09:21<11:20, 1626.56 examples/s]
Tokenizing train dataset:  39%|███▉      | 701593/1807941 [09:21<11:13, 1642.75 examples/s]
Tokenizing train dataset:  39%|███▉      | 701766/1807941 [09:21<11:04, 1664.32 examples/s]
Tokenizing train dataset:  39%|███▉      | 701941/1807941 [09:21<10:56, 1685.85 examples/s]
Tokenizing train dataset:  39%|███▉      | 702166/1807941 [09:22<11:26, 1611.09 examples/s]
Tokenizing train dataset:  39%|███▉      | 702330/1807941 [09:22<11:24, 1615.81 examples/s]
Tokenizing train dataset:  39%|███▉      | 702568/1807941 [09:22<11:30, 1601.15 examples/s]
Tokenizing train dataset:  39%|███▉      | 702745/1807941 [09:22<11:13, 1640.65 examples/s]
Tokenizing train dataset:  39%|███▉      | 702980/1807941 [09:22<11:26, 1610.56 examples/s]
Tokenizing train dataset:  39%|███▉      | 703182/1807941 [09:22<12:12, 1507.93 examples/s]
Tokenizing train dataset:  39%|███▉      | 703361/1807941 [09:22<11:42, 1572.57 examples/s]
Tokenizing train dataset:  39%|███▉      | 703539/1807941 [09:22<11:22, 1619.08 examples/s]
Tokenizing train dataset:  39%|███▉      | 703714/1807941 [09:23<11:08, 1651.87 examples/s]
Tokenizing train dataset:  39%|███▉      | 703966/1807941 [09:23<11:05, 1658.72 examples/s]
Tokenizing train dataset:  39%|███▉      | 704205/1807941 [09:23<11:14, 1635.96 examples/s]
Tokenizing train dataset:  39%|███▉      | 704387/1807941 [09:23<10:56, 1680.71 examples/s]
Tokenizing train dataset:  39%|███▉      | 704575/1807941 [09:23<10:38, 1728.30 examples/s]
Tokenizing train dataset:  39%|███▉      | 704826/1807941 [09:23<10:48, 1701.88 examples/s]
Tokenizing train dataset:  39%|███▉      | 705052/1807941 [09:23<11:15, 1633.91 examples/s]
Tokenizing train dataset:  39%|███▉      | 705249/1807941 [09:24<10:43, 1713.45 examples/s]
Tokenizing train dataset:  39%|███▉      | 705433/1807941 [09:24<11:54, 1542.09 examples/s]
Tokenizing train dataset:  39%|███▉      | 705626/1807941 [09:24<14:03, 1307.45 examples/s]
Tokenizing train dataset:  39%|███▉      | 705790/1807941 [09:24<16:14, 1130.73 examples/s]
Tokenizing train dataset:  39%|███▉      | 705936/1807941 [09:24<16:58, 1081.64 examples/s]
Tokenizing train dataset:  39%|███▉      | 706066/1807941 [09:24<18:18, 1003.19 examples/s]
Tokenizing train dataset:  39%|███▉      | 706215/1807941 [09:25<18:32, 990.65 examples/s] 
Tokenizing train dataset:  39%|███▉      | 706494/1807941 [09:25<13:23, 1370.17 examples/s]
Tokenizing train dataset:  39%|███▉      | 706913/1807941 [09:25<09:04, 2022.85 examples/s]
Tokenizing train dataset:  39%|███▉      | 707202/1807941 [09:25<08:12, 2237.09 examples/s]
Tokenizing train dataset:  39%|███▉      | 707560/1807941 [09:25<08:01, 2284.41 examples/s]
Tokenizing train dataset:  39%|███▉      | 707815/1807941 [09:25<08:49, 2079.02 examples/s]
Tokenizing train dataset:  39%|███▉      | 708072/1807941 [09:25<10:23, 1763.28 examples/s]
Tokenizing train dataset:  39%|███▉      | 708305/1807941 [09:25<10:50, 1689.95 examples/s]
Tokenizing train dataset:  39%|███▉      | 708526/1807941 [09:26<11:16, 1624.24 examples/s]
Tokenizing train dataset:  39%|███▉      | 708739/1807941 [09:26<11:44, 1561.24 examples/s]
Tokenizing train dataset:  39%|███▉      | 708915/1807941 [09:26<11:25, 1602.25 examples/s]
Tokenizing train dataset:  39%|███▉      | 709108/1807941 [09:26<12:13, 1498.85 examples/s]
Tokenizing train dataset:  39%|███▉      | 709324/1807941 [09:26<12:26, 1472.01 examples/s]
Tokenizing train dataset:  39%|███▉      | 709537/1807941 [09:26<12:35, 1454.03 examples/s]
Tokenizing train dataset:  39%|███▉      | 709728/1807941 [09:26<13:05, 1397.41 examples/s]
Tokenizing train dataset:  39%|███▉      | 709915/1807941 [09:27<13:39, 1339.77 examples/s]
Tokenizing train dataset:  39%|███▉      | 710068/1807941 [09:27<13:46, 1328.20 examples/s]
Tokenizing train dataset:  39%|███▉      | 710208/1807941 [09:27<13:37, 1343.33 examples/s]
Tokenizing train dataset:  39%|███▉      | 710403/1807941 [09:27<13:52, 1318.37 examples/s]
Tokenizing train dataset:  39%|███▉      | 710569/1807941 [09:27<14:40, 1246.31 examples/s]
Tokenizing train dataset:  39%|███▉      | 710702/1807941 [09:27<14:32, 1258.24 examples/s]
Tokenizing train dataset:  39%|███▉      | 710914/1807941 [09:27<14:00, 1304.69 examples/s]
Tokenizing train dataset:  39%|███▉      | 711105/1807941 [09:28<14:08, 1292.63 examples/s]
Tokenizing train dataset:  39%|███▉      | 711243/1807941 [09:28<13:55, 1312.47 examples/s]
Tokenizing train dataset:  39%|███▉      | 711452/1807941 [09:28<13:40, 1336.70 examples/s]
Tokenizing train dataset:  39%|███▉      | 711608/1807941 [09:28<13:14, 1380.03 examples/s]
Tokenizing train dataset:  39%|███▉      | 711761/1807941 [09:28<12:56, 1411.51 examples/s]
Tokenizing train dataset:  39%|███▉      | 711969/1807941 [09:28<13:03, 1398.44 examples/s]
Tokenizing train dataset:  39%|███▉      | 712113/1807941 [09:28<13:00, 1403.79 examples/s]
Tokenizing train dataset:  39%|███▉      | 712267/1807941 [09:28<12:42, 1436.74 examples/s]
Tokenizing train dataset:  39%|███▉      | 712426/1807941 [09:28<12:22, 1474.87 examples/s]
Tokenizing train dataset:  39%|███▉      | 712646/1807941 [09:29<12:27, 1464.62 examples/s]
Tokenizing train dataset:  39%|███▉      | 712814/1807941 [09:29<12:13, 1493.61 examples/s]
Tokenizing train dataset:  39%|███▉      | 713008/1807941 [09:29<12:59, 1404.61 examples/s]
Tokenizing train dataset:  39%|███▉      | 713219/1807941 [09:29<12:59, 1403.95 examples/s]
Tokenizing train dataset:  39%|███▉      | 713361/1807941 [09:29<13:02, 1398.40 examples/s]
Tokenizing train dataset:  39%|███▉      | 713541/1807941 [09:29<13:43, 1328.77 examples/s]
Tokenizing train dataset:  39%|███▉      | 713735/1807941 [09:29<13:53, 1312.88 examples/s]
Tokenizing train dataset:  39%|███▉      | 713901/1807941 [09:30<14:40, 1243.02 examples/s]
Tokenizing train dataset:  39%|███▉      | 714064/1807941 [09:30<15:27, 1179.52 examples/s]
Tokenizing train dataset:  40%|███▉      | 714220/1807941 [09:30<17:23, 1048.16 examples/s]
Tokenizing train dataset:  40%|███▉      | 714374/1807941 [09:30<15:57, 1142.01 examples/s]
Tokenizing train dataset:  40%|███▉      | 714539/1807941 [09:30<16:11, 1125.39 examples/s]
Tokenizing train dataset:  40%|███▉      | 714676/1807941 [09:30<15:27, 1178.38 examples/s]
Tokenizing train dataset:  40%|███▉      | 714800/1807941 [09:30<15:17, 1191.83 examples/s]
Tokenizing train dataset:  40%|███▉      | 714952/1807941 [09:31<14:19, 1271.50 examples/s]
Tokenizing train dataset:  40%|███▉      | 715125/1807941 [09:31<15:15, 1193.28 examples/s]
Tokenizing train dataset:  40%|███▉      | 715283/1807941 [09:31<15:59, 1139.09 examples/s]
Tokenizing train dataset:  40%|███▉      | 715457/1807941 [09:31<16:00, 1137.99 examples/s]
Tokenizing train dataset:  40%|███▉      | 715625/1807941 [09:31<16:06, 1130.04 examples/s]
Tokenizing train dataset:  40%|███▉      | 715755/1807941 [09:31<15:35, 1167.11 examples/s]
Tokenizing train dataset:  40%|███▉      | 715906/1807941 [09:31<16:22, 1111.54 examples/s]
Tokenizing train dataset:  40%|███▉      | 716041/1807941 [09:32<17:29, 1040.89 examples/s]
Tokenizing train dataset:  40%|███▉      | 716157/1807941 [09:32<17:03, 1066.73 examples/s]
Tokenizing train dataset:  40%|███▉      | 716298/1807941 [09:32<15:50, 1148.83 examples/s]
Tokenizing train dataset:  40%|███▉      | 716472/1807941 [09:32<16:08, 1127.51 examples/s]
Tokenizing train dataset:  40%|███▉      | 716596/1807941 [09:32<15:59, 1137.51 examples/s]
Tokenizing train dataset:  40%|███▉      | 716742/1807941 [09:32<16:51, 1078.99 examples/s]
Tokenizing train dataset:  40%|███▉      | 716858/1807941 [09:32<16:39, 1092.17 examples/s]
Tokenizing train dataset:  40%|███▉      | 716980/1807941 [09:32<16:35, 1095.43 examples/s]
Tokenizing train dataset:  40%|███▉      | 717145/1807941 [09:33<16:47, 1082.34 examples/s]
Tokenizing train dataset:  40%|███▉      | 717312/1807941 [09:33<16:42, 1087.57 examples/s]
Tokenizing train dataset:  40%|███▉      | 717442/1807941 [09:33<16:02, 1132.53 examples/s]
Tokenizing train dataset:  40%|███▉      | 717587/1807941 [09:33<17:32, 1036.29 examples/s]
Tokenizing train dataset:  40%|███▉      | 717729/1807941 [09:33<18:05, 1004.52 examples/s]
Tokenizing train dataset:  40%|███▉      | 717867/1807941 [09:33<16:41, 1088.88 examples/s]
Tokenizing train dataset:  40%|███▉      | 717989/1807941 [09:33<16:19, 1113.04 examples/s]
Tokenizing train dataset:  40%|███▉      | 718194/1807941 [09:33<15:32, 1168.95 examples/s]
Tokenizing train dataset:  40%|███▉      | 718437/1807941 [09:34<12:19, 1473.74 examples/s]
Tokenizing train dataset:  40%|███▉      | 718684/1807941 [09:34<10:30, 1728.49 examples/s]
Tokenizing train dataset:  40%|███▉      | 718930/1807941 [09:34<09:26, 1922.84 examples/s]
Tokenizing train dataset:  40%|███▉      | 719144/1807941 [09:34<09:09, 1981.87 examples/s]
Tokenizing train dataset:  40%|███▉      | 719393/1807941 [09:34<08:32, 2123.08 examples/s]
Tokenizing train dataset:  40%|███▉      | 719641/1807941 [09:34<08:09, 2221.49 examples/s]
Tokenizing train dataset:  40%|███▉      | 719883/1807941 [09:34<07:58, 2275.29 examples/s]
Tokenizing train dataset:  40%|███▉      | 720122/1807941 [09:34<08:06, 2235.90 examples/s]
Tokenizing train dataset:  40%|███▉      | 720366/1807941 [09:34<07:54, 2291.56 examples/s]
Tokenizing train dataset:  40%|███▉      | 720613/1807941 [09:34<07:44, 2341.90 examples/s]
Tokenizing train dataset:  40%|███▉      | 720857/1807941 [09:35<07:38, 2369.54 examples/s]
Tokenizing train dataset:  40%|███▉      | 721196/1807941 [09:35<07:47, 2323.19 examples/s]
Tokenizing train dataset:  40%|███▉      | 721444/1807941 [09:35<07:40, 2360.65 examples/s]
Tokenizing train dataset:  40%|███▉      | 721692/1807941 [09:35<07:34, 2389.92 examples/s]
Tokenizing train dataset:  40%|███▉      | 721936/1807941 [09:35<07:31, 2403.81 examples/s]
Tokenizing train dataset:  40%|███▉      | 722274/1807941 [09:35<07:43, 2341.25 examples/s]
Tokenizing train dataset:  40%|███▉      | 722523/1807941 [09:35<07:36, 2377.89 examples/s]
Tokenizing train dataset:  40%|███▉      | 722765/1807941 [09:35<07:34, 2387.88 examples/s]
Tokenizing train dataset:  40%|███▉      | 723116/1807941 [09:36<07:53, 2289.61 examples/s]
Tokenizing train dataset:  40%|████      | 723359/1807941 [09:36<07:47, 2322.01 examples/s]
Tokenizing train dataset:  40%|████      | 723603/1807941 [09:36<07:41, 2350.49 examples/s]
Tokenizing train dataset:  40%|████      | 723849/1807941 [09:36<07:35, 2377.57 examples/s]
Tokenizing train dataset:  40%|████      | 724180/1807941 [09:36<07:49, 2309.90 examples/s]
Tokenizing train dataset:  40%|████      | 724422/1807941 [09:36<07:43, 2336.14 examples/s]
Tokenizing train dataset:  40%|████      | 724781/1807941 [09:36<07:40, 2350.27 examples/s]
Tokenizing train dataset:  40%|████      | 725047/1807941 [09:36<09:20, 1931.12 examples/s]
Tokenizing train dataset:  40%|████      | 725307/1807941 [09:37<09:40, 1865.73 examples/s]
Tokenizing train dataset:  40%|████      | 725630/1807941 [09:37<08:19, 2168.26 examples/s]
Tokenizing train dataset:  40%|████      | 726019/1807941 [09:37<06:58, 2583.33 examples/s]
Tokenizing train dataset:  40%|████      | 726441/1807941 [09:37<06:00, 3000.44 examples/s]
Tokenizing train dataset:  40%|████      | 726876/1807941 [09:37<05:21, 3359.85 examples/s]
Tokenizing train dataset:  40%|████      | 727239/1807941 [09:37<06:42, 2681.66 examples/s]
Tokenizing train dataset:  40%|████      | 727608/1807941 [09:37<07:38, 2357.06 examples/s]
Tokenizing train dataset:  40%|████      | 727973/1807941 [09:38<07:34, 2374.61 examples/s]
Tokenizing train dataset:  40%|████      | 728340/1807941 [09:38<08:15, 2178.36 examples/s]
Tokenizing train dataset:  40%|████      | 728613/1807941 [09:38<08:39, 2076.20 examples/s]
Tokenizing train dataset:  40%|████      | 728837/1807941 [09:38<09:25, 1908.03 examples/s]
Tokenizing train dataset:  40%|████      | 729067/1807941 [09:39<31:53, 563.94 examples/s] 
Tokenizing train dataset:  40%|████      | 729273/1807941 [09:39<26:22, 681.50 examples/s]
Tokenizing train dataset:  40%|████      | 729567/1807941 [09:40<19:45, 909.31 examples/s]
Tokenizing train dataset:  40%|████      | 729858/1807941 [09:40<15:28, 1161.23 examples/s]
Tokenizing train dataset:  40%|████      | 730087/1807941 [09:40<13:28, 1332.57 examples/s]
Tokenizing train dataset:  40%|████      | 730345/1807941 [09:40<11:33, 1554.31 examples/s]
Tokenizing train dataset:  40%|████      | 730621/1807941 [09:40<09:59, 1798.22 examples/s]
Tokenizing train dataset:  40%|████      | 731004/1807941 [09:40<08:51, 2025.85 examples/s]
Tokenizing train dataset:  40%|████      | 731411/1807941 [09:40<07:13, 2482.56 examples/s]
Tokenizing train dataset:  40%|████      | 731842/1807941 [09:40<06:08, 2923.46 examples/s]
Tokenizing train dataset:  41%|████      | 732235/1807941 [09:40<05:38, 3181.28 examples/s]
Tokenizing train dataset:  41%|████      | 732654/1807941 [09:41<05:11, 3449.80 examples/s]
Tokenizing train dataset:  41%|████      | 733041/1807941 [09:41<05:01, 3562.78 examples/s]
Tokenizing train dataset:  41%|████      | 733462/1807941 [09:41<04:47, 3743.18 examples/s]
Tokenizing train dataset:  41%|████      | 733855/1807941 [09:41<04:43, 3792.35 examples/s]
Tokenizing train dataset:  41%|████      | 734431/1807941 [09:41<04:41, 3807.90 examples/s]
Tokenizing train dataset:  41%|████      | 734835/1807941 [09:41<04:37, 3866.98 examples/s]
Tokenizing train dataset:  41%|████      | 735425/1807941 [09:41<04:36, 3885.49 examples/s]
Tokenizing train dataset:  41%|████      | 736000/1807941 [09:41<04:40, 3823.98 examples/s]
Tokenizing train dataset:  41%|████      | 736396/1807941 [09:41<04:38, 3854.41 examples/s]
Tokenizing train dataset:  41%|████      | 736803/1807941 [09:42<04:34, 3906.87 examples/s]
Tokenizing train dataset:  41%|████      | 737202/1807941 [09:42<04:37, 3862.50 examples/s]
Tokenizing train dataset:  41%|████      | 737606/1807941 [09:42<04:33, 3908.77 examples/s]
Tokenizing train dataset:  41%|████      | 738000/1807941 [09:42<04:36, 3864.28 examples/s]
Tokenizing train dataset:  41%|████      | 738410/1807941 [09:42<04:32, 3927.51 examples/s]
Tokenizing train dataset:  41%|████      | 738816/1807941 [09:42<04:29, 3960.35 examples/s]
Tokenizing train dataset:  41%|████      | 739409/1807941 [09:42<04:31, 3931.76 examples/s]
Tokenizing train dataset:  41%|████      | 739809/1807941 [09:42<04:30, 3948.21 examples/s]
Tokenizing train dataset:  41%|████      | 740401/1807941 [09:42<04:33, 3903.73 examples/s]
Tokenizing train dataset:  41%|████      | 740805/1807941 [09:43<04:31, 3935.47 examples/s]
Tokenizing train dataset:  41%|████      | 741403/1807941 [09:43<04:33, 3900.83 examples/s]
Tokenizing train dataset:  41%|████      | 741809/1807941 [09:43<04:30, 3939.25 examples/s]
Tokenizing train dataset:  41%|████      | 742403/1807941 [09:43<04:32, 3905.02 examples/s]
Tokenizing train dataset:  41%|████      | 742807/1807941 [09:43<04:30, 3934.09 examples/s]
Tokenizing train dataset:  41%|████      | 743205/1807941 [09:43<04:32, 3904.90 examples/s]
Tokenizing train dataset:  41%|████      | 743608/1807941 [09:43<04:30, 3937.02 examples/s]
Tokenizing train dataset:  41%|████      | 744198/1807941 [09:43<04:32, 3907.39 examples/s]
Tokenizing train dataset:  41%|████      | 744609/1807941 [09:44<04:28, 3957.37 examples/s]
Tokenizing train dataset:  41%|████      | 745203/1807941 [09:44<04:29, 3949.40 examples/s]
Tokenizing train dataset:  41%|████      | 745615/1807941 [09:44<04:26, 3989.63 examples/s]
Tokenizing train dataset:  41%|████▏     | 746017/1807941 [09:44<04:25, 3995.56 examples/s]
Tokenizing train dataset:  41%|████▏     | 746442/1807941 [09:44<04:21, 4061.81 examples/s]
Tokenizing train dataset:  41%|████▏     | 746875/1807941 [09:44<04:16, 4133.75 examples/s]
Tokenizing train dataset:  41%|████▏     | 747515/1807941 [09:44<04:13, 4179.15 examples/s]
Tokenizing train dataset:  41%|████▏     | 747962/1807941 [09:44<04:09, 4251.76 examples/s]
Tokenizing train dataset:  41%|████▏     | 748604/1807941 [09:45<04:08, 4259.16 examples/s]
Tokenizing train dataset:  41%|████▏     | 749250/1807941 [09:45<04:07, 4270.76 examples/s]
Tokenizing train dataset:  41%|████▏     | 749683/1807941 [09:45<04:07, 4283.72 examples/s]
Tokenizing train dataset:  41%|████▏     | 750117/1807941 [09:45<04:06, 4295.72 examples/s]
Tokenizing train dataset:  42%|████▏     | 750568/1807941 [09:45<04:36, 3823.33 examples/s]
Tokenizing train dataset:  42%|████▏     | 751013/1807941 [09:45<06:41, 2633.85 examples/s]
Tokenizing train dataset:  42%|████▏     | 751336/1807941 [09:46<07:39, 2299.68 examples/s]
Tokenizing train dataset:  42%|████▏     | 751617/1807941 [09:46<08:03, 2186.67 examples/s]
Tokenizing train dataset:  42%|████▏     | 751905/1807941 [09:46<08:20, 2110.19 examples/s]
Tokenizing train dataset:  42%|████▏     | 752220/1807941 [09:46<09:08, 1926.29 examples/s]
Tokenizing train dataset:  42%|████▏     | 752437/1807941 [09:46<09:47, 1795.98 examples/s]
Tokenizing train dataset:  42%|████▏     | 752731/1807941 [09:46<09:34, 1837.26 examples/s]
Tokenizing train dataset:  42%|████▏     | 752949/1807941 [09:46<09:13, 1906.59 examples/s]
Tokenizing train dataset:  42%|████▏     | 753215/1807941 [09:47<09:25, 1863.69 examples/s]
Tokenizing train dataset:  42%|████▏     | 753450/1807941 [09:47<09:56, 1768.77 examples/s]
Tokenizing train dataset:  42%|████▏     | 753885/1807941 [09:47<07:29, 2347.44 examples/s]
Tokenizing train dataset:  42%|████▏     | 754294/1807941 [09:47<06:20, 2768.77 examples/s]
Tokenizing train dataset:  42%|████▏     | 754758/1807941 [09:47<05:24, 3248.29 examples/s]
Tokenizing train dataset:  42%|████▏     | 755187/1807941 [09:47<04:58, 3525.84 examples/s]
Tokenizing train dataset:  42%|████▏     | 755645/1807941 [09:47<04:35, 3816.64 examples/s]
Tokenizing train dataset:  42%|████▏     | 756192/1807941 [09:47<04:40, 3751.13 examples/s]
Tokenizing train dataset:  42%|████▏     | 756586/1807941 [09:47<04:36, 3799.17 examples/s]
Tokenizing train dataset:  42%|████▏     | 757000/1807941 [09:48<04:32, 3852.23 examples/s]
Tokenizing train dataset:  42%|████▏     | 757433/1807941 [09:48<04:23, 3982.07 examples/s]
Tokenizing train dataset:  42%|████▏     | 757872/1807941 [09:48<04:16, 4096.55 examples/s]
Tokenizing train dataset:  42%|████▏     | 758502/1807941 [09:48<04:13, 4132.15 examples/s]
Tokenizing train dataset:  42%|████▏     | 758947/1807941 [09:48<04:08, 4214.15 examples/s]
Tokenizing train dataset:  42%|████▏     | 759595/1807941 [09:48<04:06, 4248.11 examples/s]
Tokenizing train dataset:  42%|████▏     | 760237/1807941 [09:48<04:06, 4257.00 examples/s]
Tokenizing train dataset:  42%|████▏     | 760691/1807941 [09:48<04:02, 4323.08 examples/s]
Tokenizing train dataset:  42%|████▏     | 761317/1807941 [09:49<04:05, 4267.21 examples/s]
Tokenizing train dataset:  42%|████▏     | 761946/1807941 [09:49<04:06, 4240.29 examples/s]
Tokenizing train dataset:  42%|████▏     | 762512/1807941 [09:49<04:15, 4088.84 examples/s]
Tokenizing train dataset:  42%|████▏     | 763079/1807941 [09:49<04:21, 3989.96 examples/s]
Tokenizing train dataset:  42%|████▏     | 763678/1807941 [09:49<04:21, 3988.94 examples/s]
Tokenizing train dataset:  42%|████▏     | 764242/1807941 [09:49<04:26, 3916.19 examples/s]
Tokenizing train dataset:  42%|████▏     | 764638/1807941 [09:49<04:25, 3924.63 examples/s]
Tokenizing train dataset:  42%|████▏     | 765197/1807941 [09:50<04:30, 3858.87 examples/s]
Tokenizing train dataset:  42%|████▏     | 765768/1807941 [09:50<04:31, 3839.74 examples/s]
Tokenizing train dataset:  42%|████▏     | 766204/1807941 [09:50<05:28, 3172.23 examples/s]
Tokenizing train dataset:  42%|████▏     | 766592/1807941 [09:50<06:55, 2504.92 examples/s]
Tokenizing train dataset:  42%|████▏     | 766919/1807941 [09:50<07:46, 2230.42 examples/s]
Tokenizing train dataset:  42%|████▏     | 767188/1807941 [09:51<07:43, 2245.69 examples/s]
Tokenizing train dataset:  42%|████▏     | 767627/1807941 [09:51<06:28, 2675.95 examples/s]
Tokenizing train dataset:  42%|████▏     | 767977/1807941 [09:51<06:05, 2847.44 examples/s]
Tokenizing train dataset:  42%|████▏     | 768318/1807941 [09:52<19:14, 900.15 examples/s] 
Tokenizing train dataset:  43%|████▎     | 768547/1807941 [09:52<19:12, 901.67 examples/s]
Tokenizing train dataset:  43%|████▎     | 768750/1807941 [09:52<18:49, 919.83 examples/s]
Tokenizing train dataset:  43%|████▎     | 768923/1807941 [09:53<20:38, 839.13 examples/s]
Tokenizing train dataset:  43%|████▎     | 769070/1807941 [09:53<22:26, 771.33 examples/s]
Tokenizing train dataset:  43%|████▎     | 769184/1807941 [09:53<21:25, 807.79 examples/s]
Tokenizing train dataset:  43%|████▎     | 769314/1807941 [09:53<21:17, 813.25 examples/s]
Tokenizing train dataset:  43%|████▎     | 769419/1807941 [09:53<22:13, 778.52 examples/s]
Tokenizing train dataset:  43%|████▎     | 769516/1807941 [09:53<21:21, 810.25 examples/s]
Tokenizing train dataset:  43%|████▎     | 769628/1807941 [09:53<20:36, 839.87 examples/s]
Tokenizing train dataset:  43%|████▎     | 769767/1807941 [09:54<20:25, 847.02 examples/s]
Tokenizing train dataset:  43%|████▎     | 769910/1807941 [09:54<24:38, 702.05 examples/s]
Tokenizing train dataset:  43%|████▎     | 770000/1807941 [09:54<27:45, 623.25 examples/s]
Tokenizing train dataset:  43%|████▎     | 770125/1807941 [09:54<25:55, 667.39 examples/s]
Tokenizing train dataset:  43%|████▎     | 770218/1807941 [09:54<27:10, 636.45 examples/s]
Tokenizing train dataset:  43%|████▎     | 770289/1807941 [09:54<26:46, 646.11 examples/s]
Tokenizing train dataset:  43%|████▎     | 770375/1807941 [09:55<32:58, 524.51 examples/s]
Tokenizing train dataset:  43%|████▎     | 770471/1807941 [09:55<30:33, 565.71 examples/s]
Tokenizing train dataset:  43%|████▎     | 770539/1807941 [09:55<38:18, 451.38 examples/s]
Tokenizing train dataset:  43%|████▎     | 770609/1807941 [09:55<39:38, 436.08 examples/s]
Tokenizing train dataset:  43%|████▎     | 770676/1807941 [09:55<42:40, 405.05 examples/s]
Tokenizing train dataset:  43%|████▎     | 770762/1807941 [09:56<36:45, 470.33 examples/s]
Tokenizing train dataset:  43%|████▎     | 770841/1807941 [09:56<33:06, 521.96 examples/s]
Tokenizing train dataset:  43%|████▎     | 770904/1807941 [09:56<31:52, 542.31 examples/s]
Tokenizing train dataset:  43%|████▎     | 771000/1807941 [09:56<29:38, 583.12 examples/s]
Tokenizing train dataset:  43%|████▎     | 771106/1807941 [09:56<31:13, 553.37 examples/s]
Tokenizing train dataset:  43%|████▎     | 771183/1807941 [09:57<47:06, 366.77 examples/s]
Tokenizing train dataset:  43%|████▎     | 771234/1807941 [09:57<49:47, 347.03 examples/s]
Tokenizing train dataset:  43%|████▎     | 771287/1807941 [09:57<54:25, 317.41 examples/s]
Tokenizing train dataset:  43%|████▎     | 771340/1807941 [09:57<57:22, 301.12 examples/s]
Tokenizing train dataset:  43%|████▎     | 771391/1807941 [09:57<55:16, 312.54 examples/s]
Tokenizing train dataset:  43%|████▎     | 771430/1807941 [09:57<54:09, 318.94 examples/s]
Tokenizing train dataset:  43%|████▎     | 771465/1807941 [09:58<53:07, 325.15 examples/s]
Tokenizing train dataset:  43%|████▎     | 771511/1807941 [09:58<55:55, 308.86 examples/s]
Tokenizing train dataset:  43%|████▎     | 771577/1807941 [09:58<45:17, 381.39 examples/s]
Tokenizing train dataset:  43%|████▎     | 771633/1807941 [09:58<48:49, 353.77 examples/s]
Tokenizing train dataset:  43%|████▎     | 771705/1807941 [09:58<47:07, 366.54 examples/s]
Tokenizing train dataset:  43%|████▎     | 771744/1807941 [09:58<1:08:01, 253.90 examples/s]
Tokenizing train dataset:  43%|████▎     | 771815/1807941 [09:59<53:58, 319.97 examples/s]  
Tokenizing train dataset:  43%|████▎     | 771857/1807941 [09:59<56:58, 303.06 examples/s]
Tokenizing train dataset:  43%|████▎     | 771895/1807941 [09:59<1:26:40, 199.23 examples/s]
Tokenizing train dataset:  43%|████▎     | 771959/1807941 [09:59<1:12:08, 239.33 examples/s]
Tokenizing train dataset:  43%|████▎     | 772000/1807941 [09:59<1:05:50, 262.22 examples/s]
Tokenizing train dataset:  43%|████▎     | 772071/1807941 [10:00<1:26:26, 199.73 examples/s]
Tokenizing train dataset:  43%|████▎     | 772171/1807941 [10:00<57:51, 298.34 examples/s]  
Tokenizing train dataset:  43%|████▎     | 772245/1807941 [10:00<55:09, 312.97 examples/s]
Tokenizing train dataset:  43%|████▎     | 772356/1807941 [10:00<39:32, 436.47 examples/s]
Tokenizing train dataset:  43%|████▎     | 772465/1807941 [10:01<56:13, 306.91 examples/s]
Tokenizing train dataset:  43%|████▎     | 772538/1807941 [10:01<51:44, 333.47 examples/s]
Tokenizing train dataset:  43%|████▎     | 772587/1807941 [10:01<54:59, 313.78 examples/s]
Tokenizing train dataset:  43%|████▎     | 772644/1807941 [10:01<54:08, 318.72 examples/s]
Tokenizing train dataset:  43%|████▎     | 772708/1807941 [10:02<52:35, 328.10 examples/s]
Tokenizing train dataset:  43%|████▎     | 772756/1807941 [10:02<1:30:48, 190.01 examples/s]
Tokenizing train dataset:  43%|████▎     | 772788/1807941 [10:03<2:18:25, 124.64 examples/s]
Tokenizing train dataset:  43%|████▎     | 772834/1807941 [10:03<2:08:51, 133.88 examples/s]
Tokenizing train dataset:  43%|████▎     | 772861/1807941 [10:03<1:57:46, 146.48 examples/s]
Tokenizing train dataset:  43%|████▎     | 772887/1807941 [10:03<1:49:02, 158.21 examples/s]
Tokenizing train dataset:  43%|████▎     | 772949/1807941 [10:04<1:21:42, 211.12 examples/s]
Tokenizing train dataset:  43%|████▎     | 772980/1807941 [10:04<1:24:06, 205.07 examples/s]
Tokenizing train dataset:  43%|████▎     | 773006/1807941 [10:04<1:54:37, 150.47 examples/s]
Tokenizing train dataset:  43%|████▎     | 773045/1807941 [10:04<1:49:02, 158.19 examples/s]
Tokenizing train dataset:  43%|████▎     | 773167/1807941 [10:04<55:32, 310.47 examples/s]  
Tokenizing train dataset:  43%|████▎     | 773216/1807941 [10:05<1:29:05, 193.57 examples/s]
Tokenizing train dataset:  43%|████▎     | 773253/1807941 [10:05<1:22:10, 209.86 examples/s]
Tokenizing train dataset:  43%|████▎     | 773306/1807941 [10:05<1:07:54, 253.92 examples/s]
Tokenizing train dataset:  43%|████▎     | 773348/1807941 [10:06<1:40:20, 171.84 examples/s]
Tokenizing train dataset:  43%|████▎     | 773501/1807941 [10:06<49:29, 348.34 examples/s]  
Tokenizing train dataset:  43%|████▎     | 773570/1807941 [10:06<44:42, 385.63 examples/s]
Tokenizing train dataset:  43%|████▎     | 773669/1807941 [10:06<43:47, 393.57 examples/s]
Tokenizing train dataset:  43%|████▎     | 773753/1807941 [10:06<48:09, 357.91 examples/s]
Tokenizing train dataset:  43%|████▎     | 773865/1807941 [10:07<50:41, 339.95 examples/s]
Tokenizing train dataset:  43%|████▎     | 773916/1807941 [10:07<1:09:01, 249.65 examples/s]
Tokenizing train dataset:  43%|████▎     | 773974/1807941 [10:07<1:00:32, 284.63 examples/s][transformers] Token indices sequence length is longer than the specified maximum sequence length for this model (153397 > 131072). Running this sequence through the model will result in indexing errors

Tokenizing train dataset:  43%|████▎     | 774023/1807941 [10:08<1:36:42, 178.18 examples/s]
Tokenizing train dataset:  43%|████▎     | 774057/1807941 [10:08<2:10:13, 132.31 examples/s]
Tokenizing train dataset:  43%|████▎     | 774131/1807941 [10:09<1:31:50, 187.59 examples/s]
Tokenizing train dataset:  43%|████▎     | 774193/1807941 [10:09<1:31:33, 188.18 examples/s]
Tokenizing train dataset:  43%|████▎     | 774226/1807941 [10:09<1:24:48, 203.16 examples/s]
Tokenizing train dataset:  43%|████▎     | 774298/1807941 [10:09<1:15:26, 228.33 examples/s]
Tokenizing train dataset:  43%|████▎     | 774348/1807941 [10:09<1:16:05, 226.37 examples/s]
Tokenizing train dataset:  43%|████▎     | 774410/1807941 [10:10<1:04:27, 267.22 examples/s]
Tokenizing train dataset:  43%|████▎     | 774488/1807941 [10:10<1:07:19, 255.85 examples/s]
Tokenizing train dataset:  43%|████▎     | 774569/1807941 [10:10<59:29, 289.47 examples/s]  
Tokenizing train dataset:  43%|████▎     | 774620/1807941 [10:10<55:04, 312.66 examples/s]
Tokenizing train dataset:  43%|████▎     | 774720/1807941 [10:10<43:48, 393.09 examples/s]
Tokenizing train dataset:  43%|████▎     | 774821/1807941 [10:11<42:07, 408.77 examples/s]
Tokenizing train dataset:  43%|████▎     | 774941/1807941 [10:11<31:47, 541.61 examples/s]
Tokenizing train dataset:  43%|████▎     | 775050/1807941 [10:11<39:44, 433.14 examples/s]
Tokenizing train dataset:  43%|████▎     | 775172/1807941 [10:12<54:24, 316.40 examples/s]
Tokenizing train dataset:  43%|████▎     | 775232/1807941 [10:12<56:04, 306.95 examples/s]
Tokenizing train dataset:  43%|████▎     | 775276/1807941 [10:12<54:14, 317.29 examples/s]
Tokenizing train dataset:  43%|████▎     | 775318/1807941 [10:12<1:05:02, 264.61 examples/s]
Tokenizing train dataset:  43%|████▎     | 775381/1807941 [10:12<55:01, 312.80 examples/s]  
Tokenizing train dataset:  43%|████▎     | 775460/1807941 [10:13<45:43, 376.38 examples/s]
Tokenizing train dataset:  43%|████▎     | 775525/1807941 [10:13<54:21, 316.55 examples/s]
Tokenizing train dataset:  43%|████▎     | 775605/1807941 [10:13<55:05, 312.32 examples/s]
Tokenizing train dataset:  43%|████▎     | 775660/1807941 [10:13<55:46, 308.50 examples/s]
Tokenizing train dataset:  43%|████▎     | 775720/1807941 [10:13<48:58, 351.22 examples/s]
Tokenizing train dataset:  43%|████▎     | 775818/1807941 [10:14<37:00, 464.83 examples/s]
Tokenizing train dataset:  43%|████▎     | 775894/1807941 [10:14<49:31, 347.34 examples/s]
Tokenizing train dataset:  43%|████▎     | 775953/1807941 [10:14<55:19, 310.87 examples/s]
Tokenizing train dataset:  43%|████▎     | 776000/1807941 [10:14<55:59, 307.17 examples/s]
Tokenizing train dataset:  43%|████▎     | 776090/1807941 [10:14<42:09, 407.89 examples/s]
Tokenizing train dataset:  43%|████▎     | 776148/1807941 [10:14<38:59, 440.96 examples/s]
Tokenizing train dataset:  43%|████▎     | 776252/1807941 [10:15<31:03, 553.65 examples/s]
Tokenizing train dataset:  43%|████▎     | 776320/1807941 [10:15<29:30, 582.57 examples/s]
Tokenizing train dataset:  43%|████▎     | 776446/1807941 [10:15<26:36, 646.11 examples/s]
Tokenizing train dataset:  43%|████▎     | 776574/1807941 [10:15<24:37, 698.05 examples/s]
Tokenizing train dataset:  43%|████▎     | 776647/1807941 [10:15<30:17, 567.48 examples/s]
Tokenizing train dataset:  43%|████▎     | 776739/1807941 [10:15<33:50, 507.96 examples/s]
Tokenizing train dataset:  43%|████▎     | 776834/1807941 [10:16<29:34, 581.01 examples/s]
Tokenizing train dataset:  43%|████▎     | 776903/1807941 [10:16<57:06, 300.94 examples/s]
Tokenizing train dataset:  43%|████▎     | 776968/1807941 [10:16<49:54, 344.27 examples/s]
Tokenizing train dataset:  43%|████▎     | 777045/1807941 [10:16<45:17, 379.39 examples/s]
Tokenizing train dataset:  43%|████▎     | 777156/1807941 [10:17<34:05, 504.04 examples/s]
Tokenizing train dataset:  43%|████▎     | 777260/1807941 [10:17<45:32, 377.20 examples/s]
Tokenizing train dataset:  43%|████▎     | 777359/1807941 [10:17<40:06, 428.19 examples/s]
Tokenizing train dataset:  43%|████▎     | 777458/1807941 [10:17<34:52, 492.51 examples/s]
Tokenizing train dataset:  43%|████▎     | 777561/1807941 [10:17<35:18, 486.28 examples/s]
Tokenizing train dataset:  43%|████▎     | 777678/1807941 [10:18<30:54, 555.69 examples/s]
Tokenizing train dataset:  43%|████▎     | 777757/1807941 [10:18<29:22, 584.39 examples/s]
Tokenizing train dataset:  43%|████▎     | 777877/1807941 [10:18<48:24, 354.67 examples/s]
Tokenizing train dataset:  43%|████▎     | 778000/1807941 [10:19<46:03, 372.70 examples/s]
Tokenizing train dataset:  43%|████▎     | 778073/1807941 [10:19<43:02, 398.72 examples/s]
Tokenizing train dataset:  43%|████▎     | 778166/1807941 [10:19<39:03, 439.45 examples/s]
Tokenizing train dataset:  43%|████▎     | 778224/1807941 [10:19<39:32, 433.95 examples/s]
Tokenizing train dataset:  43%|████▎     | 778309/1807941 [10:19<34:08, 502.73 examples/s]
Tokenizing train dataset:  43%|████▎     | 778373/1807941 [10:19<40:25, 424.47 examples/s]
Tokenizing train dataset:  43%|████▎     | 778440/1807941 [10:19<37:03, 463.09 examples/s]
Tokenizing train dataset:  43%|████▎     | 778542/1807941 [10:20<30:04, 570.47 examples/s]
Tokenizing train dataset:  43%|████▎     | 778629/1807941 [10:20<32:45, 523.71 examples/s]
Tokenizing train dataset:  43%|████▎     | 778741/1807941 [10:20<26:29, 647.53 examples/s]
Tokenizing train dataset:  43%|████▎     | 778857/1807941 [10:20<27:01, 634.47 examples/s]
Tokenizing train dataset:  43%|████▎     | 778944/1807941 [10:21<44:51, 382.31 examples/s]
Tokenizing train dataset:  43%|████▎     | 779000/1807941 [10:21<1:24:39, 202.56 examples/s]
Tokenizing train dataset:  43%|████▎     | 779074/1807941 [10:21<1:08:43, 249.49 examples/s]
Tokenizing train dataset:  43%|████▎     | 779137/1807941 [10:22<1:01:39, 278.12 examples/s]
Tokenizing train dataset:  43%|████▎     | 779205/1807941 [10:22<56:08, 305.42 examples/s]  
Tokenizing train dataset:  43%|████▎     | 779321/1807941 [10:22<40:05, 427.60 examples/s]
Tokenizing train dataset:  43%|████▎     | 779389/1807941 [10:22<44:43, 383.36 examples/s]
Tokenizing train dataset:  43%|████▎     | 779474/1807941 [10:22<37:07, 461.69 examples/s]
Tokenizing train dataset:  43%|████▎     | 779559/1807941 [10:22<32:18, 530.63 examples/s]
Tokenizing train dataset:  43%|████▎     | 779655/1807941 [10:23<38:46, 442.04 examples/s]
Tokenizing train dataset:  43%|████▎     | 779764/1807941 [10:23<31:08, 550.37 examples/s]
Tokenizing train dataset:  43%|████▎     | 779916/1807941 [10:23<25:19, 676.45 examples/s]
Tokenizing train dataset:  43%|████▎     | 780000/1807941 [10:23<32:57, 519.89 examples/s]
Tokenizing train dataset:  43%|████▎     | 780128/1807941 [10:23<26:55, 636.32 examples/s]
Tokenizing train dataset:  43%|████▎     | 780236/1807941 [10:24<32:07, 533.29 examples/s]
Tokenizing train dataset:  43%|████▎     | 780306/1807941 [10:24<36:07, 474.08 examples/s]
Tokenizing train dataset:  43%|████▎     | 780402/1807941 [10:24<32:02, 534.36 examples/s]
Tokenizing train dataset:  43%|████▎     | 780481/1807941 [10:24<30:26, 562.52 examples/s]
Tokenizing train dataset:  43%|████▎     | 780632/1807941 [10:24<22:33, 759.24 examples/s]
Tokenizing train dataset:  43%|████▎     | 780736/1807941 [10:24<20:49, 822.28 examples/s]
Tokenizing train dataset:  43%|████▎     | 780830/1807941 [10:25<31:39, 540.82 examples/s]
Tokenizing train dataset:  43%|████▎     | 780941/1807941 [10:25<34:02, 502.87 examples/s]
Tokenizing train dataset:  43%|████▎     | 781032/1807941 [10:25<32:17, 530.00 examples/s]
Tokenizing train dataset:  43%|████▎     | 781103/1807941 [10:25<40:28, 422.75 examples/s]
Tokenizing train dataset:  43%|████▎     | 781227/1807941 [10:25<36:24, 469.98 examples/s]
Tokenizing train dataset:  43%|████▎     | 781311/1807941 [10:26<32:57, 519.18 examples/s]
Tokenizing train dataset:  43%|████▎     | 781407/1807941 [10:26<34:42, 492.84 examples/s]
Tokenizing train dataset:  43%|████▎     | 781483/1807941 [10:26<45:31, 375.78 examples/s]
Tokenizing train dataset:  43%|████▎     | 781579/1807941 [10:26<54:46, 312.30 examples/s]
Tokenizing train dataset:  43%|████▎     | 781621/1807941 [10:27<53:18, 320.85 examples/s]
Tokenizing train dataset:  43%|████▎     | 781707/1807941 [10:27<49:28, 345.74 examples/s]
Tokenizing train dataset:  43%|████▎     | 781778/1807941 [10:27<42:32, 402.01 examples/s]
Tokenizing train dataset:  43%|████▎     | 781892/1807941 [10:27<44:02, 388.32 examples/s]
Tokenizing train dataset:  43%|████▎     | 781949/1807941 [10:27<41:04, 416.37 examples/s]
Tokenizing train dataset:  43%|████▎     | 782000/1807941 [10:27<42:05, 406.17 examples/s]
Tokenizing train dataset:  43%|████▎     | 782052/1807941 [10:28<54:36, 313.08 examples/s]
Tokenizing train dataset:  43%|████▎     | 782134/1807941 [10:28<43:04, 396.97 examples/s]
Tokenizing train dataset:  43%|████▎     | 782194/1807941 [10:28<40:08, 425.88 examples/s]
Tokenizing train dataset:  43%|████▎     | 782261/1807941 [10:28<1:03:15, 270.21 examples/s]
Tokenizing train dataset:  43%|████▎     | 782318/1807941 [10:29<54:24, 314.16 examples/s]  
Tokenizing train dataset:  43%|████▎     | 782371/1807941 [10:29<51:22, 332.74 examples/s]
Tokenizing train dataset:  43%|████▎     | 782437/1807941 [10:29<45:45, 373.46 examples/s]
Tokenizing train dataset:  43%|████▎     | 782573/1807941 [10:29<29:44, 574.71 examples/s]
Tokenizing train dataset:  43%|████▎     | 782683/1807941 [10:30<56:22, 303.06 examples/s]
Tokenizing train dataset:  43%|████▎     | 782766/1807941 [10:30<47:36, 358.86 examples/s]
Tokenizing train dataset:  43%|████▎     | 782844/1807941 [10:30<49:58, 341.88 examples/s]
Tokenizing train dataset:  43%|████▎     | 782928/1807941 [10:30<48:15, 354.04 examples/s]
Tokenizing train dataset:  43%|████▎     | 783000/1807941 [10:30<43:04, 396.61 examples/s]
Tokenizing train dataset:  43%|████▎     | 783066/1807941 [10:30<41:42, 409.51 examples/s]
Tokenizing train dataset:  43%|████▎     | 783178/1807941 [10:31<31:33, 541.20 examples/s]
Tokenizing train dataset:  43%|████▎     | 783253/1807941 [10:31<45:09, 378.12 examples/s]
Tokenizing train dataset:  43%|████▎     | 783310/1807941 [10:31<50:34, 337.61 examples/s]
Tokenizing train dataset:  43%|████▎     | 783400/1807941 [10:31<40:00, 426.87 examples/s]
Tokenizing train dataset:  43%|████▎     | 783470/1807941 [10:31<36:56, 462.30 examples/s]
Tokenizing train dataset:  43%|████▎     | 783530/1807941 [10:31<35:13, 484.73 examples/s]
Tokenizing train dataset:  43%|████▎     | 783680/1807941 [10:32<25:02, 681.84 examples/s]
Tokenizing train dataset:  43%|████▎     | 783769/1807941 [10:32<37:53, 450.48 examples/s]
Tokenizing train dataset:  43%|████▎     | 783895/1807941 [10:32<32:13, 529.53 examples/s]
Tokenizing train dataset:  43%|████▎     | 783990/1807941 [10:32<33:41, 506.60 examples/s]
Tokenizing train dataset:  43%|████▎     | 784059/1807941 [10:32<31:58, 533.67 examples/s]
Tokenizing train dataset:  43%|████▎     | 784150/1807941 [10:33<36:18, 469.96 examples/s]
Tokenizing train dataset:  43%|████▎     | 784241/1807941 [10:33<31:12, 546.83 examples/s]
Tokenizing train dataset:  43%|████▎     | 784319/1807941 [10:34<1:16:56, 221.73 examples/s]
Tokenizing train dataset:  43%|████▎     | 784377/1807941 [10:34<1:21:35, 209.06 examples/s]
Tokenizing train dataset:  43%|████▎     | 784418/1807941 [10:35<1:47:04, 159.32 examples/s]
Tokenizing train dataset:  43%|████▎     | 784456/1807941 [10:35<2:21:26, 120.61 examples/s]
Tokenizing train dataset:  43%|████▎     | 784486/1807941 [10:35<2:09:10, 132.05 examples/s]
Tokenizing train dataset:  43%|████▎     | 784526/1807941 [10:35<1:47:57, 157.98 examples/s]
Tokenizing train dataset:  43%|████▎     | 784572/1807941 [10:36<1:39:08, 172.04 examples/s]
Tokenizing train dataset:  43%|████▎     | 784647/1807941 [10:36<1:12:35, 234.93 examples/s]
Tokenizing train dataset:  43%|████▎     | 784711/1807941 [10:36<1:06:16, 257.34 examples/s]
Tokenizing train dataset:  43%|████▎     | 784777/1807941 [10:36<53:46, 317.14 examples/s]  
Tokenizing train dataset:  43%|████▎     | 784826/1807941 [10:36<51:58, 328.04 examples/s]
Tokenizing train dataset:  43%|████▎     | 784878/1807941 [10:36<48:11, 353.82 examples/s]
Tokenizing train dataset:  43%|████▎     | 784980/1807941 [10:36<38:04, 447.82 examples/s]
Tokenizing train dataset:  43%|████▎     | 785053/1807941 [10:37<1:05:43, 259.39 examples/s]
Tokenizing train dataset:  43%|████▎     | 785104/1807941 [10:37<1:06:05, 257.96 examples/s]
Tokenizing train dataset:  43%|████▎     | 785148/1807941 [10:38<1:19:32, 214.31 examples/s]
Tokenizing train dataset:  43%|████▎     | 785286/1807941 [10:38<46:22, 367.52 examples/s]  
Tokenizing train dataset:  43%|████▎     | 785398/1807941 [10:38<35:03, 486.11 examples/s]
Tokenizing train dataset:  43%|████▎     | 785511/1807941 [10:38<28:43, 593.08 examples/s]
Tokenizing train dataset:  43%|████▎     | 785625/1807941 [10:38<26:56, 632.38 examples/s]
Tokenizing train dataset:  43%|████▎     | 785713/1807941 [10:38<26:01, 654.68 examples/s]
Tokenizing train dataset:  43%|████▎     | 785827/1807941 [10:38<23:40, 719.38 examples/s]
Tokenizing train dataset:  43%|████▎     | 785952/1807941 [10:38<22:58, 741.28 examples/s]
Tokenizing train dataset:  43%|████▎     | 786077/1807941 [10:39<20:59, 811.45 examples/s]
Tokenizing train dataset:  43%|████▎     | 786170/1807941 [10:39<22:14, 765.78 examples/s]
Tokenizing train dataset:  43%|████▎     | 786280/1807941 [10:39<21:29, 792.03 examples/s]
Tokenizing train dataset:  44%|████▎     | 786486/1807941 [10:39<15:34, 1093.53 examples/s]
Tokenizing train dataset:  44%|████▎     | 786617/1807941 [10:39<17:50, 953.64 examples/s] 
Tokenizing train dataset:  44%|████▎     | 786738/1807941 [10:39<17:00, 1000.27 examples/s]
Tokenizing train dataset:  44%|████▎     | 786883/1807941 [10:39<17:47, 956.92 examples/s] 
Tokenizing train dataset:  44%|████▎     | 787012/1807941 [10:40<18:22, 925.60 examples/s]
Tokenizing train dataset:  44%|████▎     | 787176/1807941 [10:40<22:06, 769.44 examples/s]
Tokenizing train dataset:  44%|████▎     | 787275/1807941 [10:40<21:34, 788.67 examples/s]
Tokenizing train dataset:  44%|████▎     | 787418/1807941 [10:40<18:28, 920.97 examples/s]
Tokenizing train dataset:  44%|████▎     | 787565/1807941 [10:40<16:16, 1044.47 examples/s]
Tokenizing train dataset:  44%|████▎     | 787683/1807941 [10:40<15:52, 1071.41 examples/s]
Tokenizing train dataset:  44%|████▎     | 787823/1807941 [10:40<14:42, 1155.74 examples/s]
Tokenizing train dataset:  44%|████▎     | 787949/1807941 [10:41<20:01, 849.19 examples/s] 
Tokenizing train dataset:  44%|████▎     | 788056/1807941 [10:41<26:18, 646.10 examples/s]
Tokenizing train dataset:  44%|████▎     | 788172/1807941 [10:41<25:12, 674.10 examples/s]
Tokenizing train dataset:  44%|████▎     | 788303/1807941 [10:41<23:30, 723.02 examples/s]
Tokenizing train dataset:  44%|████▎     | 788395/1807941 [10:41<22:19, 760.99 examples/s]
Tokenizing train dataset:  44%|████▎     | 788487/1807941 [10:41<24:33, 692.09 examples/s]
Tokenizing train dataset:  44%|████▎     | 788573/1807941 [10:42<23:29, 723.33 examples/s]
Tokenizing train dataset:  44%|████▎     | 788732/1807941 [10:42<18:24, 922.40 examples/s]
Tokenizing train dataset:  44%|████▎     | 789000/1807941 [10:42<12:40, 1339.06 examples/s]
Tokenizing train dataset:  44%|████▎     | 789342/1807941 [10:42<09:02, 1877.53 examples/s]
Tokenizing train dataset:  44%|████▎     | 789691/1807941 [10:42<07:20, 2312.13 examples/s]
Tokenizing train dataset:  44%|████▎     | 790016/1807941 [10:42<06:35, 2571.73 examples/s]
Tokenizing train dataset:  44%|████▎     | 790364/1807941 [10:42<05:59, 2827.23 examples/s]
Tokenizing train dataset:  44%|████▎     | 790702/1807941 [10:42<05:40, 2984.44 examples/s]
Tokenizing train dataset:  44%|████▍     | 791036/1807941 [10:42<05:29, 3085.49 examples/s]
Tokenizing train dataset:  44%|████▍     | 791362/1807941 [10:42<05:24, 3134.26 examples/s]
Tokenizing train dataset:  44%|████▍     | 791685/1807941 [10:43<05:21, 3160.69 examples/s]
Tokenizing train dataset:  44%|████▍     | 792019/1807941 [10:43<05:16, 3212.31 examples/s]
Tokenizing train dataset:  44%|████▍     | 792363/1807941 [10:43<05:09, 3279.55 examples/s]
Tokenizing train dataset:  44%|████▍     | 792875/1807941 [10:43<05:05, 3326.22 examples/s]
Tokenizing train dataset:  44%|████▍     | 793344/1807941 [10:43<05:12, 3251.41 examples/s]
Tokenizing train dataset:  44%|████▍     | 793712/1807941 [10:43<05:40, 2979.59 examples/s]
Tokenizing train dataset:  44%|████▍     | 794020/1807941 [10:43<05:37, 3002.32 examples/s]
Tokenizing train dataset:  44%|████▍     | 794347/1807941 [10:43<05:30, 3067.17 examples/s]
Tokenizing train dataset:  44%|████▍     | 794704/1807941 [10:44<05:16, 3199.07 examples/s]
Tokenizing train dataset:  44%|████▍     | 795034/1807941 [10:44<05:13, 3226.52 examples/s]
Tokenizing train dataset:  44%|████▍     | 795382/1807941 [10:44<05:07, 3290.70 examples/s]
Tokenizing train dataset:  44%|████▍     | 795881/1807941 [10:44<05:06, 3301.55 examples/s]
Tokenizing train dataset:  44%|████▍     | 796365/1807941 [10:44<05:09, 3271.12 examples/s]
Tokenizing train dataset:  44%|████▍     | 796698/1807941 [10:44<05:08, 3282.66 examples/s]
Tokenizing train dataset:  44%|████▍     | 797139/1807941 [10:44<05:21, 3144.45 examples/s]
Tokenizing train dataset:  44%|████▍     | 797539/1807941 [10:44<05:39, 2977.06 examples/s]
Tokenizing train dataset:  44%|████▍     | 797857/1807941 [10:45<05:33, 3024.85 examples/s]
Tokenizing train dataset:  44%|████▍     | 798313/1807941 [10:45<05:33, 3025.52 examples/s]
Tokenizing train dataset:  44%|████▍     | 798772/1807941 [10:45<05:32, 3031.62 examples/s]
Tokenizing train dataset:  44%|████▍     | 799189/1807941 [10:45<05:42, 2949.18 examples/s]
Tokenizing train dataset:  44%|████▍     | 799597/1807941 [10:45<05:50, 2875.44 examples/s]
Tokenizing train dataset:  44%|████▍     | 799887/1807941 [10:45<05:50, 2878.65 examples/s]
Tokenizing train dataset:  44%|████▍     | 800290/1807941 [10:45<05:58, 2811.51 examples/s]
Tokenizing train dataset:  44%|████▍     | 800605/1807941 [10:45<05:48, 2889.98 examples/s]
Tokenizing train dataset:  44%|████▍     | 801003/1807941 [10:46<05:58, 2807.44 examples/s]
Tokenizing train dataset:  44%|████▍     | 801299/1807941 [10:46<05:54, 2842.18 examples/s]
Tokenizing train dataset:  44%|████▍     | 801601/1807941 [10:46<05:48, 2883.72 examples/s]
Tokenizing train dataset:  44%|████▍     | 801902/1807941 [10:46<05:45, 2915.43 examples/s]
Tokenizing train dataset:  44%|████▍     | 802340/1807941 [10:46<05:45, 2913.95 examples/s]
Tokenizing train dataset:  44%|████▍     | 802715/1807941 [10:46<06:03, 2767.86 examples/s]
Tokenizing train dataset:  44%|████▍     | 803106/1807941 [10:46<06:10, 2712.65 examples/s]
Tokenizing train dataset:  44%|████▍     | 803413/1807941 [10:46<05:59, 2795.44 examples/s]
Tokenizing train dataset:  44%|████▍     | 803701/1807941 [10:47<05:56, 2813.58 examples/s]
Tokenizing train dataset:  44%|████▍     | 804097/1807941 [10:47<06:05, 2748.95 examples/s]
Tokenizing train dataset:  44%|████▍     | 804412/1807941 [10:47<05:52, 2847.44 examples/s]
Tokenizing train dataset:  45%|████▍     | 804718/1807941 [10:47<05:45, 2901.13 examples/s]
Tokenizing train dataset:  45%|████▍     | 805131/1807941 [10:47<05:54, 2825.76 examples/s]
Tokenizing train dataset:  45%|████▍     | 805429/1807941 [10:47<05:49, 2864.48 examples/s]
Tokenizing train dataset:  45%|████▍     | 805764/1807941 [10:47<06:20, 2637.07 examples/s]
Tokenizing train dataset:  45%|████▍     | 806164/1807941 [10:47<06:18, 2645.41 examples/s]
Tokenizing train dataset:  45%|████▍     | 806561/1807941 [10:48<06:19, 2641.91 examples/s]
Tokenizing train dataset:  45%|████▍     | 806835/1807941 [10:48<06:15, 2663.03 examples/s]
Tokenizing train dataset:  45%|████▍     | 807117/1807941 [10:48<06:10, 2698.73 examples/s]
Tokenizing train dataset:  45%|████▍     | 807464/1807941 [10:48<06:30, 2559.66 examples/s]
Tokenizing train dataset:  45%|████▍     | 807772/1807941 [10:48<06:12, 2686.58 examples/s]
Tokenizing train dataset:  45%|████▍     | 808064/1807941 [10:48<06:04, 2744.79 examples/s]
Tokenizing train dataset:  45%|████▍     | 808439/1807941 [10:48<06:16, 2653.71 examples/s]
Tokenizing train dataset:  45%|████▍     | 808851/1807941 [10:48<06:12, 2682.30 examples/s]
Tokenizing train dataset:  45%|████▍     | 809282/1807941 [10:49<06:04, 2740.35 examples/s]
Tokenizing train dataset:  45%|████▍     | 809569/1807941 [10:49<06:00, 2768.08 examples/s]
Tokenizing train dataset:  45%|████▍     | 809851/1807941 [10:49<05:59, 2779.11 examples/s]
Tokenizing train dataset:  45%|████▍     | 810206/1807941 [10:49<06:19, 2629.68 examples/s]
Tokenizing train dataset:  45%|████▍     | 810602/1807941 [10:49<06:19, 2628.97 examples/s]
Tokenizing train dataset:  45%|████▍     | 810894/1807941 [10:49<06:09, 2697.64 examples/s]
Tokenizing train dataset:  45%|████▍     | 811286/1807941 [10:49<06:14, 2664.85 examples/s]
Tokenizing train dataset:  45%|████▍     | 811684/1807941 [10:50<06:14, 2658.69 examples/s]
Tokenizing train dataset:  45%|████▍     | 811975/1807941 [10:50<06:06, 2715.50 examples/s]
Tokenizing train dataset:  45%|████▍     | 812384/1807941 [10:50<06:06, 2715.75 examples/s]
Tokenizing train dataset:  45%|████▍     | 812660/1807941 [10:50<06:05, 2722.71 examples/s]
Tokenizing train dataset:  45%|████▍     | 812975/1807941 [10:50<05:51, 2830.70 examples/s]
Tokenizing train dataset:  45%|████▍     | 813303/1807941 [10:50<09:31, 1741.91 examples/s]
Tokenizing train dataset:  45%|████▍     | 813552/1807941 [10:51<11:18, 1465.13 examples/s]
Tokenizing train dataset:  45%|████▌     | 813812/1807941 [10:51<11:44, 1410.98 examples/s]
Tokenizing train dataset:  45%|████▌     | 814070/1807941 [10:51<10:16, 1610.87 examples/s]
Tokenizing train dataset:  45%|████▌     | 814370/1807941 [10:51<08:47, 1883.01 examples/s]
Tokenizing train dataset:  45%|████▌     | 814716/1807941 [10:51<08:15, 2002.86 examples/s]
Tokenizing train dataset:  45%|████▌     | 814948/1807941 [10:51<07:59, 2069.86 examples/s]
Tokenizing train dataset:  45%|████▌     | 815199/1807941 [10:51<07:36, 2173.00 examples/s]
Tokenizing train dataset:  45%|████▌     | 815505/1807941 [10:51<06:54, 2394.74 examples/s]
Tokenizing train dataset:  45%|████▌     | 815769/1807941 [10:52<06:43, 2459.12 examples/s]
Tokenizing train dataset:  45%|████▌     | 816047/1807941 [10:52<06:29, 2545.06 examples/s]
Tokenizing train dataset:  45%|████▌     | 816329/1807941 [10:52<06:18, 2620.79 examples/s]
Tokenizing train dataset:  45%|████▌     | 816703/1807941 [10:52<06:25, 2568.91 examples/s]
Tokenizing train dataset:  45%|████▌     | 816985/1807941 [10:52<06:16, 2632.30 examples/s]
Tokenizing train dataset:  45%|████▌     | 817300/1807941 [10:52<08:28, 1949.53 examples/s]
Tokenizing train dataset:  45%|████▌     | 817538/1807941 [10:53<10:44, 1536.79 examples/s]
Tokenizing train dataset:  45%|████▌     | 817733/1807941 [10:53<12:17, 1343.43 examples/s]
Tokenizing train dataset:  45%|████▌     | 817936/1807941 [10:53<13:21, 1235.12 examples/s]
Tokenizing train dataset:  45%|████▌     | 818082/1807941 [10:53<14:08, 1166.03 examples/s]
Tokenizing train dataset:  45%|████▌     | 818250/1807941 [10:53<14:25, 1143.61 examples/s]
Tokenizing train dataset:  45%|████▌     | 818404/1807941 [10:53<14:50, 1111.13 examples/s]
Tokenizing train dataset:  45%|████▌     | 818544/1807941 [10:54<15:40, 1052.21 examples/s]
Tokenizing train dataset:  45%|████▌     | 818653/1807941 [10:54<15:34, 1058.27 examples/s]
Tokenizing train dataset:  45%|████▌     | 818806/1807941 [10:54<15:53, 1037.79 examples/s]
Tokenizing train dataset:  45%|████▌     | 818949/1807941 [10:54<16:24, 1004.74 examples/s]
Tokenizing train dataset:  45%|████▌     | 819094/1807941 [10:54<17:25, 946.02 examples/s] 
Tokenizing train dataset:  45%|████▌     | 819243/1807941 [10:54<17:11, 958.39 examples/s]
Tokenizing train dataset:  45%|████▌     | 819446/1807941 [10:54<13:50, 1189.84 examples/s]
Tokenizing train dataset:  45%|████▌     | 819714/1807941 [10:54<10:41, 1539.61 examples/s]
Tokenizing train dataset:  45%|████▌     | 819927/1807941 [10:55<09:45, 1686.19 examples/s]
Tokenizing train dataset:  45%|████▌     | 820137/1807941 [10:55<09:10, 1794.61 examples/s]
Tokenizing train dataset:  45%|████▌     | 820383/1807941 [10:55<08:19, 1976.10 examples/s]
Tokenizing train dataset:  45%|████▌     | 820674/1807941 [10:55<07:21, 2237.29 examples/s]
Tokenizing train dataset:  45%|████▌     | 820931/1807941 [10:55<07:03, 2330.33 examples/s]
Tokenizing train dataset:  45%|████▌     | 821171/1807941 [10:55<07:00, 2347.60 examples/s]
Tokenizing train dataset:  45%|████▌     | 821412/1807941 [10:55<06:57, 2360.26 examples/s]
Tokenizing train dataset:  45%|████▌     | 821656/1807941 [10:55<06:54, 2380.44 examples/s]
Tokenizing train dataset:  45%|████▌     | 821932/1807941 [10:55<06:35, 2490.24 examples/s]
Tokenizing train dataset:  45%|████▌     | 822196/1807941 [10:56<08:27, 1940.47 examples/s]
Tokenizing train dataset:  45%|████▌     | 822416/1807941 [10:56<10:17, 1595.80 examples/s]
Tokenizing train dataset:  46%|████▌     | 822624/1807941 [10:56<11:50, 1387.45 examples/s]
Tokenizing train dataset:  46%|████▌     | 822789/1807941 [10:56<13:45, 1193.75 examples/s]
Tokenizing train dataset:  46%|████▌     | 822943/1807941 [10:56<14:22, 1141.75 examples/s]
Tokenizing train dataset:  46%|████▌     | 823101/1807941 [10:57<15:18, 1071.90 examples/s]
Tokenizing train dataset:  46%|████▌     | 823218/1807941 [10:57<16:34, 990.20 examples/s] 
Tokenizing train dataset:  46%|████▌     | 823357/1807941 [10:57<16:56, 968.39 examples/s]
Tokenizing train dataset:  46%|████▌     | 823504/1807941 [10:57<17:02, 963.16 examples/s]
Tokenizing train dataset:  46%|████▌     | 823635/1807941 [10:57<17:33, 934.54 examples/s]
Tokenizing train dataset:  46%|████▌     | 823836/1807941 [10:57<15:37, 1050.15 examples/s]
Tokenizing train dataset:  46%|████▌     | 824029/1807941 [10:57<13:13, 1239.76 examples/s]
Tokenizing train dataset:  46%|████▌     | 824272/1807941 [10:57<10:47, 1518.57 examples/s]
Tokenizing train dataset:  46%|████▌     | 824482/1807941 [10:58<09:51, 1663.17 examples/s]
Tokenizing train dataset:  46%|████▌     | 824760/1807941 [10:58<08:22, 1957.22 examples/s]
Tokenizing train dataset:  46%|████▌     | 825000/1807941 [10:58<12:35, 1301.10 examples/s]
Tokenizing train dataset:  46%|████▌     | 825216/1807941 [10:58<14:35, 1121.95 examples/s]
Tokenizing train dataset:  46%|████▌     | 825417/1807941 [10:58<12:51, 1273.12 examples/s]
Tokenizing train dataset:  46%|████▌     | 825674/1807941 [10:58<10:41, 1532.05 examples/s]
Tokenizing train dataset:  46%|████▌     | 825895/1807941 [10:59<10:53, 1503.87 examples/s]
Tokenizing train dataset:  46%|████▌     | 826109/1807941 [10:59<12:16, 1333.45 examples/s]
Tokenizing train dataset:  46%|████▌     | 826313/1807941 [10:59<12:16, 1333.00 examples/s]
Tokenizing train dataset:  46%|████▌     | 826541/1807941 [10:59<11:48, 1384.62 examples/s]
Tokenizing train dataset:  46%|████▌     | 826742/1807941 [10:59<13:12, 1238.71 examples/s]
Tokenizing train dataset:  46%|████▌     | 826902/1807941 [10:59<12:31, 1305.61 examples/s]
Tokenizing train dataset:  46%|████▌     | 827105/1807941 [11:00<12:24, 1317.04 examples/s]
Tokenizing train dataset:  46%|████▌     | 827270/1807941 [11:00<13:04, 1249.48 examples/s]
Tokenizing train dataset:  46%|████▌     | 827452/1807941 [11:00<11:53, 1373.93 examples/s]
Tokenizing train dataset:  46%|████▌     | 827667/1807941 [11:00<11:44, 1390.93 examples/s]
Tokenizing train dataset:  46%|████▌     | 827891/1807941 [11:00<11:32, 1415.85 examples/s]
Tokenizing train dataset:  46%|████▌     | 828077/1807941 [11:00<13:16, 1230.60 examples/s]
Tokenizing train dataset:  46%|████▌     | 828233/1807941 [11:00<13:55, 1173.05 examples/s]
Tokenizing train dataset:  46%|████▌     | 828366/1807941 [11:01<13:33, 1203.49 examples/s]
Tokenizing train dataset:  46%|████▌     | 828514/1807941 [11:01<12:58, 1258.04 examples/s]
Tokenizing train dataset:  46%|████▌     | 828679/1807941 [11:01<13:35, 1200.50 examples/s]
Tokenizing train dataset:  46%|████▌     | 828835/1807941 [11:01<12:45, 1279.29 examples/s]
Tokenizing train dataset:  46%|████▌     | 829000/1807941 [11:01<15:08, 1077.71 examples/s]
Tokenizing train dataset:  46%|████▌     | 829154/1807941 [11:01<13:50, 1179.03 examples/s]
Tokenizing train dataset:  46%|████▌     | 829313/1807941 [11:01<14:19, 1139.07 examples/s]
Tokenizing train dataset:  46%|████▌     | 829574/1807941 [11:02<10:59, 1482.42 examples/s]
Tokenizing train dataset:  46%|████▌     | 829849/1807941 [11:02<09:03, 1799.58 examples/s]
Tokenizing train dataset:  46%|████▌     | 830072/1807941 [11:02<08:31, 1911.26 examples/s]
Tokenizing train dataset:  46%|████▌     | 830312/1807941 [11:02<10:12, 1597.34 examples/s]
Tokenizing train dataset:  46%|████▌     | 830535/1807941 [11:02<12:39, 1286.49 examples/s]
Tokenizing train dataset:  46%|████▌     | 830744/1807941 [11:02<13:33, 1201.17 examples/s]
Tokenizing train dataset:  46%|████▌     | 830946/1807941 [11:02<12:00, 1356.55 examples/s]
Tokenizing train dataset:  46%|████▌     | 831133/1807941 [11:03<11:06, 1464.73 examples/s]
Tokenizing train dataset:  46%|████▌     | 831362/1807941 [11:03<09:50, 1654.65 examples/s]
Tokenizing train dataset:  46%|████▌     | 831577/1807941 [11:03<11:29, 1416.20 examples/s]
Tokenizing train dataset:  46%|████▌     | 831785/1807941 [11:03<12:48, 1270.47 examples/s]
Tokenizing train dataset:  46%|████▌     | 831946/1807941 [11:03<14:49, 1096.75 examples/s]
Tokenizing train dataset:  46%|████▌     | 832094/1807941 [11:03<16:43, 972.27 examples/s] 
Tokenizing train dataset:  46%|████▌     | 832246/1807941 [11:04<16:38, 977.46 examples/s]
Tokenizing train dataset:  46%|████▌     | 832353/1807941 [11:04<16:21, 993.87 examples/s]
Tokenizing train dataset:  46%|████▌     | 832477/1807941 [11:04<15:32, 1046.03 examples/s]
Tokenizing train dataset:  46%|████▌     | 832641/1807941 [11:04<15:21, 1058.30 examples/s]
Tokenizing train dataset:  46%|████▌     | 832783/1807941 [11:04<15:56, 1019.10 examples/s]
Tokenizing train dataset:  46%|████▌     | 832894/1807941 [11:04<15:40, 1037.16 examples/s]
Tokenizing train dataset:  46%|████▌     | 833012/1807941 [11:04<17:06, 949.57 examples/s] 
Tokenizing train dataset:  46%|████▌     | 833203/1807941 [11:04<13:48, 1176.14 examples/s]
Tokenizing train dataset:  46%|████▌     | 833443/1807941 [11:05<10:57, 1483.11 examples/s]
Tokenizing train dataset:  46%|████▌     | 833684/1807941 [11:05<09:24, 1726.95 examples/s]
Tokenizing train dataset:  46%|████▌     | 833929/1807941 [11:05<08:26, 1921.47 examples/s]
Tokenizing train dataset:  46%|████▌     | 834238/1807941 [11:05<08:14, 1970.51 examples/s]
Tokenizing train dataset:  46%|████▌     | 834472/1807941 [11:05<09:55, 1633.47 examples/s]
Tokenizing train dataset:  46%|████▌     | 834657/1807941 [11:05<10:45, 1507.64 examples/s]
Tokenizing train dataset:  46%|████▌     | 834838/1807941 [11:05<11:30, 1408.53 examples/s]
Tokenizing train dataset:  46%|████▌     | 835015/1807941 [11:06<10:53, 1488.20 examples/s]
Tokenizing train dataset:  46%|████▌     | 835258/1807941 [11:06<09:27, 1712.65 examples/s]
Tokenizing train dataset:  46%|████▌     | 835479/1807941 [11:06<11:05, 1461.91 examples/s]
Tokenizing train dataset:  46%|████▌     | 835641/1807941 [11:06<12:04, 1341.79 examples/s]
Tokenizing train dataset:  46%|████▌     | 835795/1807941 [11:06<13:03, 1240.69 examples/s]
Tokenizing train dataset:  46%|████▌     | 835979/1807941 [11:06<11:47, 1374.00 examples/s]
Tokenizing train dataset:  46%|████▌     | 836152/1807941 [11:06<11:06, 1458.93 examples/s]
Tokenizing train dataset:  46%|████▋     | 836373/1807941 [11:06<09:53, 1636.77 examples/s]
Tokenizing train dataset:  46%|████▋     | 836600/1807941 [11:07<08:58, 1805.27 examples/s]
Tokenizing train dataset:  46%|████▋     | 836842/1807941 [11:07<08:11, 1973.92 examples/s]
Tokenizing train dataset:  46%|████▋     | 837050/1807941 [11:07<08:04, 2001.84 examples/s]
Tokenizing train dataset:  46%|████▋     | 837313/1807941 [11:07<07:24, 2181.30 examples/s]
Tokenizing train dataset:  46%|████▋     | 837637/1807941 [11:07<06:30, 2484.47 examples/s]
Tokenizing train dataset:  46%|████▋     | 837939/1807941 [11:07<07:04, 2283.95 examples/s]
Tokenizing train dataset:  46%|████▋     | 838233/1807941 [11:07<08:21, 1935.33 examples/s]
Tokenizing train dataset:  46%|████▋     | 838517/1807941 [11:08<10:10, 1587.37 examples/s]
Tokenizing train dataset:  46%|████▋     | 838733/1807941 [11:08<10:27, 1545.61 examples/s]
Tokenizing train dataset:  46%|████▋     | 838967/1807941 [11:08<10:28, 1542.51 examples/s]
Tokenizing train dataset:  46%|████▋     | 839159/1807941 [11:08<11:01, 1465.60 examples/s]
Tokenizing train dataset:  46%|████▋     | 839387/1807941 [11:08<10:58, 1471.05 examples/s]
Tokenizing train dataset:  46%|████▋     | 839578/1807941 [11:08<10:20, 1560.98 examples/s]
Tokenizing train dataset:  46%|████▋     | 839792/1807941 [11:08<10:40, 1511.74 examples/s]
Tokenizing train dataset:  46%|████▋     | 839949/1807941 [11:09<10:35, 1524.33 examples/s]
Tokenizing train dataset:  46%|████▋     | 840197/1807941 [11:09<10:20, 1559.68 examples/s]
Tokenizing train dataset:  46%|████▋     | 840390/1807941 [11:09<09:51, 1635.43 examples/s]
Tokenizing train dataset:  46%|████▋     | 840589/1807941 [11:09<09:21, 1721.43 examples/s]
Tokenizing train dataset:  47%|████▋     | 840833/1807941 [11:09<09:33, 1686.07 examples/s]
Tokenizing train dataset:  47%|████▋     | 841076/1807941 [11:09<09:53, 1629.02 examples/s]
Tokenizing train dataset:  47%|████▋     | 841299/1807941 [11:09<10:11, 1581.70 examples/s]
Tokenizing train dataset:  47%|████▋     | 841517/1807941 [11:10<10:29, 1534.27 examples/s]
Tokenizing train dataset:  47%|████▋     | 841733/1807941 [11:10<10:44, 1499.86 examples/s]
Tokenizing train dataset:  47%|████▋     | 841915/1807941 [11:10<12:36, 1276.53 examples/s]
Tokenizing train dataset:  47%|████▋     | 842105/1807941 [11:10<12:45, 1261.83 examples/s]
Tokenizing train dataset:  47%|████▋     | 842236/1807941 [11:10<12:42, 1266.44 examples/s]
Tokenizing train dataset:  47%|████▋     | 842401/1807941 [11:10<13:19, 1208.29 examples/s]
Tokenizing train dataset:  47%|████▋     | 842529/1807941 [11:10<13:20, 1206.58 examples/s]
Tokenizing train dataset:  47%|████▋     | 842658/1807941 [11:10<13:08, 1224.03 examples/s]
Tokenizing train dataset:  47%|████▋     | 842846/1807941 [11:11<13:03, 1231.49 examples/s]
Tokenizing train dataset:  47%|████▋     | 843003/1807941 [11:11<15:25, 1042.85 examples/s]
Tokenizing train dataset:  47%|████▋     | 843127/1807941 [11:11<18:19, 877.80 examples/s] 
Tokenizing train dataset:  47%|████▋     | 843243/1807941 [11:11<18:57, 848.35 examples/s]
Tokenizing train dataset:  47%|████▋     | 843340/1807941 [11:11<18:34, 865.50 examples/s]
Tokenizing train dataset:  47%|████▋     | 843440/1807941 [11:11<20:20, 790.15 examples/s]
Tokenizing train dataset:  47%|████▋     | 843557/1807941 [11:12<18:26, 871.64 examples/s]
Tokenizing train dataset:  47%|████▋     | 843698/1807941 [11:12<20:16, 792.69 examples/s]
Tokenizing train dataset:  47%|████▋     | 843792/1807941 [11:12<24:16, 661.99 examples/s]
Tokenizing train dataset:  47%|████▋     | 843870/1807941 [11:12<26:03, 616.71 examples/s]
Tokenizing train dataset:  47%|████▋     | 843941/1807941 [11:12<25:19, 634.50 examples/s]
Tokenizing train dataset:  47%|████▋     | 844063/1807941 [11:12<26:32, 605.42 examples/s]
Tokenizing train dataset:  47%|████▋     | 844140/1807941 [11:13<25:16, 635.37 examples/s]
Tokenizing train dataset:  47%|████▋     | 844225/1807941 [11:13<23:37, 679.77 examples/s]
Tokenizing train dataset:  47%|████▋     | 844331/1807941 [11:13<21:00, 764.26 examples/s]
Tokenizing train dataset:  47%|████▋     | 844442/1807941 [11:13<21:28, 747.95 examples/s]
Tokenizing train dataset:  47%|████▋     | 844552/1807941 [11:13<19:17, 832.35 examples/s]
Tokenizing train dataset:  47%|████▋     | 844648/1807941 [11:13<18:35, 863.25 examples/s]
Tokenizing train dataset:  47%|████▋     | 844739/1807941 [11:13<22:16, 720.51 examples/s]
Tokenizing train dataset:  47%|████▋     | 844818/1807941 [11:14<27:54, 575.01 examples/s]
Tokenizing train dataset:  47%|████▋     | 844906/1807941 [11:14<30:57, 518.34 examples/s]
Tokenizing train dataset:  47%|████▋     | 844986/1807941 [11:14<28:10, 569.53 examples/s]
Tokenizing train dataset:  47%|████▋     | 845122/1807941 [11:14<22:57, 698.87 examples/s]
Tokenizing train dataset:  47%|████▋     | 845717/1807941 [11:14<08:29, 1888.23 examples/s]
Tokenizing train dataset:  47%|████▋     | 846309/1807941 [11:14<05:38, 2843.46 examples/s]
Tokenizing train dataset:  47%|████▋     | 846905/1807941 [11:14<04:23, 3642.00 examples/s]
Tokenizing train dataset:  47%|████▋     | 847479/1807941 [11:14<03:48, 4204.30 examples/s]
Tokenizing train dataset:  47%|████▋     | 848024/1807941 [11:14<03:31, 4548.24 examples/s]
Tokenizing train dataset:  47%|████▋     | 848603/1807941 [11:15<03:15, 4895.50 examples/s]
Tokenizing train dataset:  47%|████▋     | 849180/1807941 [11:15<03:06, 5146.21 examples/s]
Tokenizing train dataset:  47%|████▋     | 849787/1807941 [11:15<02:57, 5412.52 examples/s]
Tokenizing train dataset:  47%|████▋     | 850428/1807941 [11:15<03:12, 4970.83 examples/s]
Tokenizing train dataset:  47%|████▋     | 851010/1807941 [11:15<03:28, 4579.61 examples/s]
Tokenizing train dataset:  47%|████▋     | 851533/1807941 [11:15<03:47, 4207.34 examples/s]
Tokenizing train dataset:  47%|████▋     | 852032/1807941 [11:15<04:03, 3919.09 examples/s]
Tokenizing train dataset:  47%|████▋     | 852561/1807941 [11:16<04:12, 3789.51 examples/s]
Tokenizing train dataset:  47%|████▋     | 853049/1807941 [11:16<04:23, 3619.76 examples/s]
Tokenizing train dataset:  47%|████▋     | 853564/1807941 [11:16<04:28, 3557.44 examples/s]
Tokenizing train dataset:  47%|████▋     | 854054/1807941 [11:16<04:35, 3466.20 examples/s]
Tokenizing train dataset:  47%|████▋     | 854568/1807941 [11:16<04:36, 3451.39 examples/s]
Tokenizing train dataset:  47%|████▋     | 855054/1807941 [11:16<04:41, 3383.27 examples/s]
Tokenizing train dataset:  47%|████▋     | 855398/1807941 [11:16<04:40, 3393.53 examples/s]
Tokenizing train dataset:  47%|████▋     | 855747/1807941 [11:16<04:38, 3413.74 examples/s]
Tokenizing train dataset:  47%|████▋     | 856256/1807941 [11:17<04:39, 3404.55 examples/s]
Tokenizing train dataset:  47%|████▋     | 856613/1807941 [11:17<04:36, 3442.47 examples/s]
Tokenizing train dataset:  47%|████▋     | 856975/1807941 [11:17<04:32, 3484.31 examples/s]
Tokenizing train dataset:  47%|████▋     | 857332/1807941 [11:17<04:31, 3506.72 examples/s]
Tokenizing train dataset:  47%|████▋     | 857698/1807941 [11:17<04:27, 3546.64 examples/s]
Tokenizing train dataset:  47%|████▋     | 858231/1807941 [11:17<04:27, 3547.07 examples/s]
Tokenizing train dataset:  48%|████▊     | 858778/1807941 [11:17<04:25, 3577.78 examples/s]
Tokenizing train dataset:  48%|████▊     | 859246/1807941 [11:17<04:37, 3422.32 examples/s]
Tokenizing train dataset:  48%|████▊     | 859754/1807941 [11:18<04:38, 3403.14 examples/s]
Tokenizing train dataset:  48%|████▊     | 860241/1807941 [11:18<04:42, 3351.47 examples/s]
Tokenizing train dataset:  48%|████▊     | 860619/1807941 [11:18<04:34, 3447.64 examples/s]
Tokenizing train dataset:  48%|████▊     | 861000/1807941 [11:18<04:30, 3494.36 examples/s]
Tokenizing train dataset:  48%|████▊     | 861374/1807941 [11:18<04:26, 3555.22 examples/s]
Tokenizing train dataset:  48%|████▊     | 861851/1807941 [11:18<04:37, 3412.52 examples/s]
Tokenizing train dataset:  48%|████▊     | 862203/1807941 [11:18<04:41, 3361.45 examples/s]
Tokenizing train dataset:  48%|████▊     | 862591/1807941 [11:18<04:30, 3494.32 examples/s]
Tokenizing train dataset:  48%|████▊     | 862985/1807941 [11:19<04:21, 3611.43 examples/s]
Tokenizing train dataset:  48%|████▊     | 863497/1807941 [11:19<04:27, 3535.21 examples/s]
Tokenizing train dataset:  48%|████▊     | 863961/1807941 [11:19<04:40, 3370.87 examples/s]
Tokenizing train dataset:  48%|████▊     | 864449/1807941 [11:19<04:43, 3331.29 examples/s]
Tokenizing train dataset:  48%|████▊     | 864816/1807941 [11:19<04:36, 3408.93 examples/s]
Tokenizing train dataset:  48%|████▊     | 865303/1807941 [11:19<04:41, 3350.19 examples/s]
Tokenizing train dataset:  48%|████▊     | 865684/1807941 [11:19<04:32, 3459.91 examples/s]
Tokenizing train dataset:  48%|████▊     | 866096/1807941 [11:20<05:29, 2854.97 examples/s]
Tokenizing train dataset:  48%|████▊     | 866565/1807941 [11:20<05:20, 2934.11 examples/s]
Tokenizing train dataset:  48%|████▊     | 866903/1807941 [11:20<05:10, 3031.20 examples/s]
Tokenizing train dataset:  48%|████▊     | 867300/1807941 [11:20<05:23, 2904.44 examples/s]
Tokenizing train dataset:  48%|████▊     | 867648/1807941 [11:20<05:09, 3037.87 examples/s]
Tokenizing train dataset:  48%|████▊     | 867997/1807941 [11:20<04:58, 3149.71 examples/s]
Tokenizing train dataset:  48%|████▊     | 868463/1807941 [11:20<04:59, 3131.81 examples/s]
Tokenizing train dataset:  48%|████▊     | 868910/1807941 [11:20<05:05, 3078.31 examples/s]
Tokenizing train dataset:  48%|████▊     | 869358/1807941 [11:21<05:08, 3047.33 examples/s]
Tokenizing train dataset:  48%|████▊     | 869673/1807941 [11:21<05:06, 3063.09 examples/s]
Tokenizing train dataset:  48%|████▊     | 870078/1807941 [11:21<05:54, 2644.53 examples/s]
Tokenizing train dataset:  48%|████▊     | 870493/1807941 [11:21<05:14, 2977.85 examples/s]
Tokenizing train dataset:  48%|████▊     | 870828/1807941 [11:21<05:05, 3065.50 examples/s]
Tokenizing train dataset:  48%|████▊     | 871176/1807941 [11:21<04:58, 3139.23 examples/s]
Tokenizing train dataset:  48%|████▊     | 871525/1807941 [11:21<04:49, 3230.59 examples/s]
Tokenizing train dataset:  48%|████▊     | 871864/1807941 [11:21<04:46, 3272.54 examples/s]
Tokenizing train dataset:  48%|████▊     | 872246/1807941 [11:22<05:50, 2669.77 examples/s]
Tokenizing train dataset:  48%|████▊     | 872562/1807941 [11:22<05:35, 2786.02 examples/s]
Tokenizing train dataset:  48%|████▊     | 872893/1807941 [11:22<05:20, 2915.08 examples/s]
Tokenizing train dataset:  48%|████▊     | 873305/1807941 [11:22<05:27, 2852.50 examples/s]
Tokenizing train dataset:  48%|████▊     | 873750/1807941 [11:22<05:23, 2889.43 examples/s]
Tokenizing train dataset:  48%|████▊     | 874048/1807941 [11:22<05:21, 2909.19 examples/s]
Tokenizing train dataset:  48%|████▊     | 874453/1807941 [11:22<05:29, 2831.80 examples/s]
Tokenizing train dataset:  48%|████▊     | 874803/1807941 [11:22<05:11, 2992.97 examples/s]
Tokenizing train dataset:  48%|████▊     | 875189/1807941 [11:23<05:27, 2844.31 examples/s]
Tokenizing train dataset:  48%|████▊     | 875485/1807941 [11:23<06:05, 2553.27 examples/s]
Tokenizing train dataset:  48%|████▊     | 875809/1807941 [11:23<05:43, 2714.83 examples/s]
Tokenizing train dataset:  48%|████▊     | 876240/1807941 [11:23<05:36, 2764.99 examples/s]
Tokenizing train dataset:  48%|████▊     | 876564/1807941 [11:23<05:23, 2878.61 examples/s]
Tokenizing train dataset:  49%|████▊     | 876899/1807941 [11:23<05:10, 2996.29 examples/s]
Tokenizing train dataset:  49%|████▊     | 877353/1807941 [11:23<05:09, 3003.06 examples/s]
Tokenizing train dataset:  49%|████▊     | 877704/1807941 [11:23<04:57, 3130.10 examples/s]
Tokenizing train dataset:  49%|████▊     | 878181/1807941 [11:24<04:55, 3142.95 examples/s]
Tokenizing train dataset:  49%|████▊     | 878650/1807941 [11:24<04:56, 3133.67 examples/s]
Tokenizing train dataset:  49%|████▊     | 878992/1807941 [11:24<04:50, 3198.56 examples/s]
Tokenizing train dataset:  49%|████▊     | 879487/1807941 [11:24<04:47, 3225.86 examples/s]
Tokenizing train dataset:  49%|████▊     | 879821/1807941 [11:24<04:45, 3251.26 examples/s]
Tokenizing train dataset:  49%|████▊     | 880169/1807941 [11:24<04:46, 3234.59 examples/s]
Tokenizing train dataset:  49%|████▊     | 880522/1807941 [11:24<04:40, 3310.77 examples/s]
Tokenizing train dataset:  49%|████▊     | 880873/1807941 [11:24<04:35, 3362.36 examples/s]
Tokenizing train dataset:  49%|████▉     | 881414/1807941 [11:25<04:28, 3448.64 examples/s]
Tokenizing train dataset:  49%|████▉     | 881845/1807941 [11:25<04:45, 3243.91 examples/s]
Tokenizing train dataset:  49%|████▉     | 882208/1807941 [11:25<05:46, 2672.06 examples/s]
Tokenizing train dataset:  49%|████▉     | 882569/1807941 [11:25<05:57, 2589.81 examples/s]
Tokenizing train dataset:  49%|████▉     | 882901/1807941 [11:25<05:36, 2747.81 examples/s]
Tokenizing train dataset:  49%|████▉     | 883212/1807941 [11:25<05:26, 2833.50 examples/s]
Tokenizing train dataset:  49%|████▉     | 883554/1807941 [11:25<05:10, 2980.10 examples/s]
Tokenizing train dataset:  49%|████▉     | 883902/1807941 [11:25<04:56, 3112.10 examples/s]
Tokenizing train dataset:  49%|████▉     | 884228/1807941 [11:26<04:53, 3151.79 examples/s]
Tokenizing train dataset:  49%|████▉     | 884559/1807941 [11:26<04:49, 3194.40 examples/s]
Tokenizing train dataset:  49%|████▉     | 884889/1807941 [11:26<04:46, 3221.84 examples/s]
Tokenizing train dataset:  49%|████▉     | 885347/1807941 [11:26<04:52, 3154.95 examples/s]
Tokenizing train dataset:  49%|████▉     | 885682/1807941 [11:26<04:48, 3202.26 examples/s]
Tokenizing train dataset:  49%|████▉     | 886066/1807941 [11:26<05:10, 2967.05 examples/s]
Tokenizing train dataset:  49%|████▉     | 886395/1807941 [11:26<05:02, 3048.45 examples/s]
Tokenizing train dataset:  49%|████▉     | 886841/1807941 [11:26<05:05, 3016.82 examples/s]
Tokenizing train dataset:  49%|████▉     | 887258/1807941 [11:27<05:13, 2934.64 examples/s]
Tokenizing train dataset:  49%|████▉     | 887719/1807941 [11:27<05:09, 2975.75 examples/s]
Tokenizing train dataset:  49%|████▉     | 888174/1807941 [11:27<05:11, 2952.50 examples/s]
Tokenizing train dataset:  49%|████▉     | 888529/1807941 [11:27<04:57, 3088.09 examples/s]
Tokenizing train dataset:  49%|████▉     | 888850/1807941 [11:27<04:54, 3117.12 examples/s]
Tokenizing train dataset:  49%|████▉     | 889173/1807941 [11:27<04:52, 3143.57 examples/s]
Tokenizing train dataset:  49%|████▉     | 889513/1807941 [11:27<04:46, 3210.34 examples/s]
Tokenizing train dataset:  49%|████▉     | 889937/1807941 [11:27<05:00, 3059.82 examples/s]
Tokenizing train dataset:  49%|████▉     | 890379/1807941 [11:28<05:04, 3016.28 examples/s]
Tokenizing train dataset:  49%|████▉     | 890813/1807941 [11:28<05:08, 2972.74 examples/s]
Tokenizing train dataset:  49%|████▉     | 891216/1807941 [11:28<05:18, 2879.55 examples/s]
Tokenizing train dataset:  49%|████▉     | 891519/1807941 [11:28<05:14, 2913.45 examples/s]
Tokenizing train dataset:  49%|████▉     | 891867/1807941 [11:28<05:00, 3052.45 examples/s]
Tokenizing train dataset:  49%|████▉     | 892263/1807941 [11:28<05:15, 2904.13 examples/s]
Tokenizing train dataset:  49%|████▉     | 892612/1807941 [11:28<05:37, 2708.24 examples/s]
Tokenizing train dataset:  49%|████▉     | 892909/1807941 [11:29<06:10, 2470.33 examples/s]
Tokenizing train dataset:  49%|████▉     | 893193/1807941 [11:29<05:57, 2556.38 examples/s]
Tokenizing train dataset:  49%|████▉     | 893552/1807941 [11:29<05:24, 2814.76 examples/s]
Tokenizing train dataset:  49%|████▉     | 893858/1807941 [11:29<05:17, 2876.39 examples/s]
Tokenizing train dataset:  49%|████▉     | 894159/1807941 [11:29<05:16, 2883.03 examples/s]
Tokenizing train dataset:  49%|████▉     | 894514/1807941 [11:29<04:59, 3054.77 examples/s]
Tokenizing train dataset:  50%|████▉     | 894991/1807941 [11:29<04:54, 3097.25 examples/s]
Tokenizing train dataset:  50%|████▉     | 895305/1807941 [11:29<04:53, 3106.05 examples/s]
Tokenizing train dataset:  50%|████▉     | 895652/1807941 [11:29<04:44, 3202.26 examples/s]
Tokenizing train dataset:  50%|████▉     | 896000/1807941 [11:30<04:39, 3260.05 examples/s]
Tokenizing train dataset:  50%|████▉     | 896334/1807941 [11:30<04:38, 3278.95 examples/s]
Tokenizing train dataset:  50%|████▉     | 896667/1807941 [11:30<04:36, 3291.93 examples/s]
Tokenizing train dataset:  50%|████▉     | 897132/1807941 [11:30<04:43, 3216.69 examples/s]
Tokenizing train dataset:  50%|████▉     | 897458/1807941 [11:30<04:42, 3225.03 examples/s]
Tokenizing train dataset:  50%|████▉     | 897954/1807941 [11:30<04:39, 3252.89 examples/s]
Tokenizing train dataset:  50%|████▉     | 898410/1807941 [11:30<04:46, 3175.65 examples/s]
Tokenizing train dataset:  50%|████▉     | 898901/1807941 [11:30<04:43, 3203.98 examples/s]
Tokenizing train dataset:  50%|████▉     | 899371/1807941 [11:31<04:46, 3176.68 examples/s]
Tokenizing train dataset:  50%|████▉     | 899714/1807941 [11:31<04:40, 3235.43 examples/s]
Tokenizing train dataset:  50%|████▉     | 900104/1807941 [11:31<05:03, 2994.53 examples/s]
Tokenizing train dataset:  50%|████▉     | 900544/1807941 [11:31<05:05, 2970.59 examples/s]
Tokenizing train dataset:  50%|████▉     | 900875/1807941 [11:31<04:58, 3043.63 examples/s]
Tokenizing train dataset:  50%|████▉     | 901208/1807941 [11:31<05:29, 2752.05 examples/s]
Tokenizing train dataset:  50%|████▉     | 901603/1807941 [11:31<04:58, 3037.20 examples/s]
Tokenizing train dataset:  50%|████▉     | 901980/1807941 [11:31<04:41, 3222.03 examples/s]
Tokenizing train dataset:  50%|████▉     | 902365/1807941 [11:32<04:27, 3389.49 examples/s]
Tokenizing train dataset:  50%|████▉     | 902773/1807941 [11:32<04:12, 3577.84 examples/s]
Tokenizing train dataset:  50%|████▉     | 903258/1807941 [11:32<04:22, 3448.25 examples/s]
Tokenizing train dataset:  50%|████▉     | 903732/1807941 [11:32<04:30, 3342.99 examples/s]
Tokenizing train dataset:  50%|█████     | 904202/1807941 [11:32<04:36, 3270.07 examples/s]
Tokenizing train dataset:  50%|█████     | 904686/1807941 [11:32<04:37, 3253.00 examples/s]
Tokenizing train dataset:  50%|█████     | 905083/1807941 [11:32<04:54, 3060.86 examples/s]
Tokenizing train dataset:  50%|█████     | 905471/1807941 [11:33<05:10, 2910.33 examples/s]
Tokenizing train dataset:  50%|█████     | 905916/1807941 [11:33<05:08, 2924.40 examples/s]
Tokenizing train dataset:  50%|█████     | 906400/1807941 [11:33<04:59, 3013.85 examples/s]
Tokenizing train dataset:  50%|█████     | 906751/1807941 [11:33<04:48, 3122.39 examples/s]
Tokenizing train dataset:  50%|█████     | 907081/1807941 [11:33<04:44, 3161.35 examples/s]
Tokenizing train dataset:  50%|█████     | 907407/1807941 [11:33<04:42, 3184.22 examples/s]
Tokenizing train dataset:  50%|█████     | 907731/1807941 [11:33<04:41, 3197.17 examples/s]
Tokenizing train dataset:  50%|█████     | 908214/1807941 [11:33<04:41, 3200.53 examples/s]
Tokenizing train dataset:  50%|█████     | 908556/1807941 [11:33<04:36, 3250.91 examples/s]
Tokenizing train dataset:  50%|█████     | 909000/1807941 [11:34<04:46, 3132.38 examples/s]
Tokenizing train dataset:  50%|█████     | 909326/1807941 [11:34<04:44, 3162.83 examples/s]
Tokenizing train dataset:  50%|█████     | 909832/1807941 [11:34<04:37, 3234.90 examples/s]
Tokenizing train dataset:  50%|█████     | 910187/1807941 [11:34<04:33, 3286.80 examples/s]
Tokenizing train dataset:  50%|█████     | 910542/1807941 [11:34<05:02, 2964.37 examples/s]
Tokenizing train dataset:  50%|█████     | 910862/1807941 [11:34<04:57, 3019.44 examples/s]
Tokenizing train dataset:  50%|█████     | 911214/1807941 [11:34<05:22, 2783.13 examples/s]
Tokenizing train dataset:  50%|█████     | 911575/1807941 [11:34<05:00, 2987.38 examples/s]
Tokenizing train dataset:  50%|█████     | 911912/1807941 [11:35<04:50, 3084.87 examples/s]
Tokenizing train dataset:  50%|█████     | 912377/1807941 [11:35<04:49, 3088.29 examples/s]
Tokenizing train dataset:  50%|█████     | 912711/1807941 [11:35<04:44, 3149.20 examples/s]
Tokenizing train dataset:  51%|█████     | 913206/1807941 [11:35<04:39, 3199.64 examples/s]
Tokenizing train dataset:  51%|█████     | 913636/1807941 [11:35<04:50, 3083.34 examples/s]
Tokenizing train dataset:  51%|█████     | 913985/1807941 [11:35<04:41, 3176.83 examples/s]
Tokenizing train dataset:  51%|█████     | 914439/1807941 [11:35<04:46, 3122.50 examples/s]
Tokenizing train dataset:  51%|█████     | 914910/1807941 [11:36<04:45, 3126.86 examples/s]
Tokenizing train dataset:  51%|█████     | 915272/1807941 [11:36<04:35, 3237.95 examples/s]
Tokenizing train dataset:  51%|█████     | 915652/1807941 [11:36<04:24, 3379.22 examples/s]
Tokenizing train dataset:  51%|█████     | 916008/1807941 [11:36<04:20, 3426.68 examples/s]
Tokenizing train dataset:  51%|█████     | 916375/1807941 [11:36<04:15, 3490.55 examples/s]
Tokenizing train dataset:  51%|█████     | 916857/1807941 [11:36<04:23, 3381.07 examples/s]
Tokenizing train dataset:  51%|█████     | 917361/1807941 [11:36<04:24, 3363.38 examples/s]
Tokenizing train dataset:  51%|█████     | 917762/1807941 [11:36<04:12, 3520.98 examples/s]
Tokenizing train dataset:  51%|█████     | 918237/1807941 [11:36<04:21, 3396.38 examples/s]
Tokenizing train dataset:  51%|█████     | 918598/1807941 [11:37<04:18, 3446.86 examples/s]
Tokenizing train dataset:  51%|█████     | 918976/1807941 [11:37<04:11, 3529.66 examples/s]
Tokenizing train dataset:  51%|█████     | 919483/1807941 [11:37<04:15, 3473.48 examples/s]
Tokenizing train dataset:  51%|█████     | 919903/1807941 [11:37<04:33, 3243.71 examples/s]
Tokenizing train dataset:  51%|█████     | 920332/1807941 [11:37<04:44, 3116.38 examples/s]
Tokenizing train dataset:  51%|█████     | 920732/1807941 [11:37<04:26, 3323.96 examples/s]
Tokenizing train dataset:  51%|█████     | 921203/1807941 [11:37<04:32, 3258.93 examples/s]
Tokenizing train dataset:  51%|█████     | 921705/1807941 [11:38<04:29, 3285.81 examples/s]
Tokenizing train dataset:  51%|█████     | 922193/1807941 [11:38<04:30, 3272.54 examples/s]
Tokenizing train dataset:  51%|█████     | 922541/1807941 [11:38<04:26, 3316.55 examples/s]
Tokenizing train dataset:  51%|█████     | 922884/1807941 [11:38<04:24, 3341.93 examples/s]
Tokenizing train dataset:  51%|█████     | 923341/1807941 [11:38<04:33, 3237.73 examples/s]
Tokenizing train dataset:  51%|█████     | 923700/1807941 [11:38<04:26, 3320.43 examples/s]
Tokenizing train dataset:  51%|█████     | 924206/1807941 [11:38<04:24, 3335.86 examples/s]
Tokenizing train dataset:  51%|█████     | 924558/1807941 [11:38<04:21, 3379.05 examples/s]
Tokenizing train dataset:  51%|█████     | 925014/1807941 [11:39<04:31, 3257.40 examples/s]
Tokenizing train dataset:  51%|█████     | 925356/1807941 [11:39<04:28, 3291.56 examples/s]
Tokenizing train dataset:  51%|█████     | 925694/1807941 [11:39<07:23, 1988.14 examples/s]
Tokenizing train dataset:  51%|█████     | 925995/1807941 [11:39<09:16, 1585.01 examples/s]
Tokenizing train dataset:  51%|█████     | 926364/1807941 [11:40<08:52, 1655.02 examples/s]
Tokenizing train dataset:  51%|█████▏    | 926698/1807941 [11:40<07:36, 1930.37 examples/s]
Tokenizing train dataset:  51%|█████▏    | 927012/1807941 [11:40<06:47, 2159.60 examples/s]
Tokenizing train dataset:  51%|█████▏    | 927341/1807941 [11:40<06:07, 2398.34 examples/s]
Tokenizing train dataset:  51%|█████▏    | 927667/1807941 [11:40<05:38, 2597.32 examples/s]
Tokenizing train dataset:  51%|█████▏    | 928010/1807941 [11:40<05:53, 2492.50 examples/s]
Tokenizing train dataset:  51%|█████▏    | 928360/1807941 [11:40<06:01, 2436.35 examples/s]
Tokenizing train dataset:  51%|█████▏    | 928709/1807941 [11:40<05:28, 2679.68 examples/s]
Tokenizing train dataset:  51%|█████▏    | 929106/1807941 [11:40<05:29, 2665.54 examples/s]
Tokenizing train dataset:  51%|█████▏    | 929409/1807941 [11:41<05:19, 2751.18 examples/s]
Tokenizing train dataset:  51%|█████▏    | 929745/1807941 [11:41<05:02, 2904.78 examples/s]
Tokenizing train dataset:  51%|█████▏    | 930066/1807941 [11:41<04:54, 2985.29 examples/s]
Tokenizing train dataset:  51%|█████▏    | 930531/1807941 [11:41<04:49, 3025.91 examples/s]
Tokenizing train dataset:  51%|█████▏    | 930994/1807941 [11:41<04:48, 3042.13 examples/s]
Tokenizing train dataset:  52%|█████▏    | 931442/1807941 [11:41<04:50, 3016.65 examples/s]
Tokenizing train dataset:  52%|█████▏    | 931811/1807941 [11:41<04:36, 3172.62 examples/s]
Tokenizing train dataset:  52%|█████▏    | 932149/1807941 [11:41<04:31, 3223.27 examples/s]
Tokenizing train dataset:  52%|█████▏    | 932632/1807941 [11:42<04:31, 3219.12 examples/s]
Tokenizing train dataset:  52%|█████▏    | 932997/1807941 [11:42<04:23, 3324.32 examples/s]
Tokenizing train dataset:  52%|█████▏    | 933347/1807941 [11:42<04:20, 3363.00 examples/s]
Tokenizing train dataset:  52%|█████▏    | 933709/1807941 [11:42<04:14, 3429.63 examples/s]
Tokenizing train dataset:  52%|█████▏    | 934077/1807941 [11:42<04:09, 3497.32 examples/s]
Tokenizing train dataset:  52%|█████▏    | 934448/1807941 [11:42<04:05, 3556.24 examples/s]
Tokenizing train dataset:  52%|█████▏    | 934814/1807941 [11:42<04:03, 3582.65 examples/s]
Tokenizing train dataset:  52%|█████▏    | 935350/1807941 [11:42<04:07, 3521.20 examples/s]
Tokenizing train dataset:  52%|█████▏    | 935884/1807941 [11:42<04:06, 3533.29 examples/s]
Tokenizing train dataset:  52%|█████▏    | 936387/1807941 [11:43<04:11, 3469.31 examples/s]
Tokenizing train dataset:  52%|█████▏    | 936746/1807941 [11:43<04:09, 3496.60 examples/s]
Tokenizing train dataset:  52%|█████▏    | 937273/1807941 [11:43<04:08, 3497.73 examples/s]
Tokenizing train dataset:  52%|█████▏    | 937629/1807941 [11:43<04:07, 3512.89 examples/s]
Tokenizing train dataset:  52%|█████▏    | 937987/1807941 [11:43<04:06, 3528.91 examples/s]
Tokenizing train dataset:  52%|█████▏    | 938454/1807941 [11:43<04:17, 3377.37 examples/s]
Tokenizing train dataset:  52%|█████▏    | 938953/1807941 [11:43<04:18, 3355.75 examples/s]
Tokenizing train dataset:  52%|█████▏    | 939427/1807941 [11:44<04:23, 3290.20 examples/s]
Tokenizing train dataset:  52%|█████▏    | 939793/1807941 [11:44<04:17, 3374.78 examples/s]
Tokenizing train dataset:  52%|█████▏    | 940268/1807941 [11:44<04:22, 3299.79 examples/s]
Tokenizing train dataset:  52%|█████▏    | 940734/1807941 [11:44<04:28, 3234.55 examples/s]
Tokenizing train dataset:  52%|█████▏    | 941091/1807941 [11:44<04:21, 3313.11 examples/s]
Tokenizing train dataset:  52%|█████▏    | 941439/1807941 [11:44<04:18, 3351.43 examples/s]
Tokenizing train dataset:  52%|█████▏    | 941805/1807941 [11:44<04:12, 3425.95 examples/s]
Tokenizing train dataset:  52%|█████▏    | 942322/1807941 [11:44<04:12, 3431.18 examples/s]
Tokenizing train dataset:  52%|█████▏    | 942687/1807941 [11:44<04:08, 3483.00 examples/s]
Tokenizing train dataset:  52%|█████▏    | 943047/1807941 [11:45<04:06, 3510.50 examples/s]
Tokenizing train dataset:  52%|█████▏    | 943535/1807941 [11:45<04:13, 3411.79 examples/s]
Tokenizing train dataset:  52%|█████▏    | 943998/1807941 [11:45<04:22, 3295.78 examples/s]
Tokenizing train dataset:  52%|█████▏    | 944429/1807941 [11:45<04:33, 3152.79 examples/s]
Tokenizing train dataset:  52%|█████▏    | 944774/1807941 [11:45<04:59, 2878.30 examples/s]
Tokenizing train dataset:  52%|█████▏    | 945214/1807941 [11:45<04:58, 2893.38 examples/s]
Tokenizing train dataset:  52%|█████▏    | 945542/1807941 [11:45<04:49, 2979.21 examples/s]
Tokenizing train dataset:  52%|█████▏    | 945929/1807941 [11:46<04:29, 3195.99 examples/s]
Tokenizing train dataset:  52%|█████▏    | 946258/1807941 [11:46<05:02, 2844.86 examples/s]
Tokenizing train dataset:  52%|█████▏    | 946717/1807941 [11:46<04:55, 2911.65 examples/s]
Tokenizing train dataset:  52%|█████▏    | 947048/1807941 [11:46<04:46, 3000.65 examples/s]
Tokenizing train dataset:  52%|█████▏    | 947389/1807941 [11:46<04:37, 3101.90 examples/s]
Tokenizing train dataset:  52%|█████▏    | 947752/1807941 [11:46<04:25, 3239.79 examples/s]
Tokenizing train dataset:  52%|█████▏    | 948096/1807941 [11:46<04:21, 3293.11 examples/s]
Tokenizing train dataset:  52%|█████▏    | 948444/1807941 [11:46<04:17, 3343.30 examples/s]
Tokenizing train dataset:  52%|█████▏    | 948947/1807941 [11:46<04:17, 3342.36 examples/s]
Tokenizing train dataset:  53%|█████▎    | 949414/1807941 [11:47<04:23, 3258.95 examples/s]
Tokenizing train dataset:  53%|█████▎    | 949784/1807941 [11:47<04:14, 3366.42 examples/s]
Tokenizing train dataset:  53%|█████▎    | 950225/1807941 [11:47<04:26, 3216.24 examples/s]
Tokenizing train dataset:  53%|█████▎    | 950578/1807941 [11:47<04:20, 3292.02 examples/s]
Tokenizing train dataset:  53%|█████▎    | 950948/1807941 [11:47<04:12, 3398.44 examples/s]
Tokenizing train dataset:  53%|█████▎    | 951438/1807941 [11:47<04:16, 3345.59 examples/s]
Tokenizing train dataset:  53%|█████▎    | 951914/1807941 [11:47<04:20, 3285.37 examples/s]
Tokenizing train dataset:  53%|█████▎    | 952374/1807941 [11:48<04:26, 3211.22 examples/s]
Tokenizing train dataset:  53%|█████▎    | 952734/1807941 [11:48<04:19, 3299.48 examples/s]
Tokenizing train dataset:  53%|█████▎    | 953195/1807941 [11:48<04:25, 3218.09 examples/s]
Tokenizing train dataset:  53%|█████▎    | 953605/1807941 [11:48<04:09, 3429.85 examples/s]
Tokenizing train dataset:  53%|█████▎    | 953968/1807941 [11:48<04:05, 3479.98 examples/s]
Tokenizing train dataset:  53%|█████▎    | 954526/1807941 [11:48<03:59, 3561.24 examples/s]
Tokenizing train dataset:  53%|█████▎    | 955036/1807941 [11:48<04:03, 3505.02 examples/s]
Tokenizing train dataset:  53%|█████▎    | 955393/1807941 [11:48<04:02, 3517.13 examples/s]
Tokenizing train dataset:  53%|█████▎    | 955771/1807941 [11:48<03:57, 3580.94 examples/s]
Tokenizing train dataset:  53%|█████▎    | 956301/1807941 [11:49<03:59, 3561.69 examples/s]
Tokenizing train dataset:  53%|█████▎    | 956683/1807941 [11:49<03:54, 3626.44 examples/s]
Tokenizing train dataset:  53%|█████▎    | 957213/1807941 [11:49<03:57, 3589.56 examples/s]
Tokenizing train dataset:  53%|█████▎    | 957762/1807941 [11:49<03:55, 3611.11 examples/s]
Tokenizing train dataset:  53%|█████▎    | 958159/1807941 [11:49<03:49, 3695.89 examples/s]
Tokenizing train dataset:  53%|█████▎    | 958664/1807941 [11:49<03:57, 3580.04 examples/s]
Tokenizing train dataset:  53%|█████▎    | 959179/1807941 [11:49<04:01, 3510.86 examples/s]
Tokenizing train dataset:  53%|█████▎    | 959571/1807941 [11:50<03:55, 3606.38 examples/s]
Tokenizing train dataset:  53%|█████▎    | 959971/1807941 [11:50<03:49, 3700.91 examples/s]
Tokenizing train dataset:  53%|█████▎    | 960495/1807941 [11:50<03:53, 3624.42 examples/s]
Tokenizing train dataset:  53%|█████▎    | 961000/1807941 [11:50<03:59, 3532.46 examples/s]
Tokenizing train dataset:  53%|█████▎    | 961508/1807941 [11:50<04:03, 3481.86 examples/s]
Tokenizing train dataset:  53%|█████▎    | 961864/1807941 [11:50<04:01, 3497.75 examples/s]
Tokenizing train dataset:  53%|█████▎    | 962307/1807941 [11:50<04:15, 3312.63 examples/s]
Tokenizing train dataset:  53%|█████▎    | 962664/1807941 [11:50<04:10, 3374.11 examples/s]
Tokenizing train dataset:  53%|█████▎    | 963103/1807941 [11:51<04:22, 3217.91 examples/s]
Tokenizing train dataset:  53%|█████▎    | 963577/1807941 [11:51<04:24, 3194.45 examples/s]
Tokenizing train dataset:  53%|█████▎    | 964000/1807941 [11:51<04:34, 3072.94 examples/s]
Tokenizing train dataset:  53%|█████▎    | 964321/1807941 [11:51<04:31, 3102.05 examples/s]
Tokenizing train dataset:  53%|█████▎    | 964750/1807941 [11:51<04:39, 3018.17 examples/s]
Tokenizing train dataset:  53%|█████▎    | 965173/1807941 [11:51<04:45, 2952.10 examples/s]
Tokenizing train dataset:  53%|█████▎    | 965615/1807941 [11:51<04:45, 2947.70 examples/s]
Tokenizing train dataset:  53%|█████▎    | 966012/1807941 [11:52<04:55, 2850.83 examples/s]
Tokenizing train dataset:  53%|█████▎    | 966306/1807941 [11:52<04:53, 2867.62 examples/s]
Tokenizing train dataset:  53%|█████▎    | 966613/1807941 [11:52<04:48, 2914.18 examples/s]
Tokenizing train dataset:  53%|█████▎    | 966926/1807941 [11:52<04:43, 2968.44 examples/s]
Tokenizing train dataset:  54%|█████▎    | 967316/1807941 [11:52<04:56, 2831.67 examples/s]
Tokenizing train dataset:  54%|█████▎    | 967751/1807941 [11:52<04:54, 2852.07 examples/s]
Tokenizing train dataset:  54%|█████▎    | 968112/1807941 [11:52<05:15, 2660.18 examples/s]
Tokenizing train dataset:  54%|█████▎    | 968420/1807941 [11:52<05:04, 2756.58 examples/s]
Tokenizing train dataset:  54%|█████▎    | 968814/1807941 [11:53<05:09, 2706.90 examples/s]
Tokenizing train dataset:  54%|█████▎    | 969090/1807941 [11:53<05:08, 2716.48 examples/s]
Tokenizing train dataset:  54%|█████▎    | 969467/1807941 [11:53<04:40, 2984.03 examples/s]
Tokenizing train dataset:  54%|█████▎    | 969845/1807941 [11:53<04:22, 3193.82 examples/s]
Tokenizing train dataset:  54%|█████▎    | 970182/1807941 [11:53<04:19, 3224.72 examples/s]
Tokenizing train dataset:  54%|█████▎    | 970560/1807941 [11:53<04:07, 3378.07 examples/s]
Tokenizing train dataset:  54%|█████▎    | 970959/1807941 [11:53<03:55, 3551.35 examples/s]
Tokenizing train dataset:  54%|█████▎    | 971479/1807941 [11:53<03:57, 3516.21 examples/s]
Tokenizing train dataset:  54%|█████▍    | 971891/1807941 [11:53<03:47, 3673.51 examples/s]
Tokenizing train dataset:  54%|█████▍    | 972470/1807941 [11:54<03:43, 3737.96 examples/s]
Tokenizing train dataset:  54%|█████▍    | 972979/1807941 [11:54<03:50, 3615.66 examples/s]
Tokenizing train dataset:  54%|█████▍    | 973518/1807941 [11:54<03:51, 3605.73 examples/s]
Tokenizing train dataset:  54%|█████▍    | 973899/1807941 [11:54<03:48, 3650.76 examples/s]
Tokenizing train dataset:  54%|█████▍    | 974430/1807941 [11:54<03:50, 3610.29 examples/s]
Tokenizing train dataset:  54%|█████▍    | 974958/1807941 [11:54<03:52, 3577.92 examples/s]
Tokenizing train dataset:  54%|█████▍    | 975448/1807941 [11:54<03:59, 3476.44 examples/s]
Tokenizing train dataset:  54%|█████▍    | 975873/1807941 [11:55<03:47, 3650.73 examples/s]
Tokenizing train dataset:  54%|█████▍    | 976290/1807941 [11:55<04:07, 3356.49 examples/s]
Tokenizing train dataset:  54%|█████▍    | 976647/1807941 [11:55<04:04, 3405.40 examples/s]
Tokenizing train dataset:  54%|█████▍    | 976996/1807941 [11:55<04:02, 3424.07 examples/s]
Tokenizing train dataset:  54%|█████▍    | 977489/1807941 [11:55<04:06, 3372.19 examples/s]
Tokenizing train dataset:  54%|█████▍    | 977887/1807941 [11:55<03:55, 3523.82 examples/s]
Tokenizing train dataset:  54%|█████▍    | 978406/1807941 [11:55<03:57, 3496.96 examples/s]
Tokenizing train dataset:  54%|█████▍    | 978916/1807941 [11:55<03:59, 3462.51 examples/s]
Tokenizing train dataset:  54%|█████▍    | 979389/1807941 [11:56<04:06, 3360.45 examples/s]
Tokenizing train dataset:  54%|█████▍    | 979781/1807941 [11:56<03:57, 3487.66 examples/s]
Tokenizing train dataset:  54%|█████▍    | 980271/1807941 [11:56<04:02, 3409.51 examples/s]
Tokenizing train dataset:  54%|█████▍    | 980758/1807941 [11:56<04:06, 3354.76 examples/s]
Tokenizing train dataset:  54%|█████▍    | 981205/1807941 [11:56<04:15, 3233.44 examples/s]
Tokenizing train dataset:  54%|█████▍    | 981558/1807941 [11:56<04:10, 3300.60 examples/s]
Tokenizing train dataset:  54%|█████▍    | 981896/1807941 [11:56<04:09, 3314.51 examples/s]
Tokenizing train dataset:  54%|█████▍    | 982411/1807941 [11:57<04:06, 3353.69 examples/s]
Tokenizing train dataset:  54%|█████▍    | 982873/1807941 [11:57<04:13, 3259.75 examples/s]
Tokenizing train dataset:  54%|█████▍    | 983315/1807941 [11:57<04:21, 3155.43 examples/s]
Tokenizing train dataset:  54%|█████▍    | 983770/1807941 [11:57<04:24, 3114.05 examples/s]
Tokenizing train dataset:  54%|█████▍    | 984211/1807941 [11:57<04:29, 3057.07 examples/s]
Tokenizing train dataset:  54%|█████▍    | 984654/1807941 [11:57<04:32, 3021.45 examples/s]
Tokenizing train dataset:  54%|█████▍    | 985036/1807941 [11:57<04:46, 2872.00 examples/s]
Tokenizing train dataset:  55%|█████▍    | 985347/1807941 [11:58<04:41, 2923.92 examples/s]
Tokenizing train dataset:  55%|█████▍    | 985794/1807941 [11:58<04:39, 2940.10 examples/s]
Tokenizing train dataset:  55%|█████▍    | 986101/1807941 [11:58<04:36, 2969.22 examples/s]
Tokenizing train dataset:  55%|█████▍    | 986416/1807941 [11:58<04:32, 3009.29 examples/s]
Tokenizing train dataset:  55%|█████▍    | 986734/1807941 [11:58<04:29, 3052.55 examples/s]
Tokenizing train dataset:  55%|█████▍    | 987198/1807941 [11:58<04:27, 3064.11 examples/s]
Tokenizing train dataset:  55%|█████▍    | 987651/1807941 [11:58<04:29, 3046.03 examples/s]
Tokenizing train dataset:  55%|█████▍    | 987980/1807941 [11:58<04:24, 3103.68 examples/s]
Tokenizing train dataset:  55%|█████▍    | 988343/1807941 [11:59<04:46, 2865.38 examples/s]
Tokenizing train dataset:  55%|█████▍    | 988701/1807941 [11:59<04:29, 3039.16 examples/s]
Tokenizing train dataset:  55%|█████▍    | 989082/1807941 [11:59<04:13, 3235.07 examples/s]
Tokenizing train dataset:  55%|█████▍    | 989533/1807941 [11:59<04:19, 3150.18 examples/s]
Tokenizing train dataset:  55%|█████▍    | 989905/1807941 [11:59<04:08, 3294.36 examples/s]
Tokenizing train dataset:  55%|█████▍    | 990257/1807941 [11:59<04:03, 3352.17 examples/s]
Tokenizing train dataset:  55%|█████▍    | 990627/1807941 [11:59<03:57, 3445.90 examples/s]
Tokenizing train dataset:  55%|█████▍    | 991000/1807941 [11:59<03:51, 3524.20 examples/s]
Tokenizing train dataset:  55%|█████▍    | 991380/1807941 [11:59<03:46, 3601.13 examples/s]
Tokenizing train dataset:  55%|█████▍    | 991870/1807941 [12:00<03:55, 3471.73 examples/s]
Tokenizing train dataset:  55%|█████▍    | 992391/1807941 [12:00<03:55, 3469.65 examples/s]
Tokenizing train dataset:  55%|█████▍    | 992838/1807941 [12:00<04:06, 3303.45 examples/s]
Tokenizing train dataset:  55%|█████▍    | 993175/1807941 [12:00<04:05, 3316.47 examples/s]
Tokenizing train dataset:  55%|█████▍    | 993529/1807941 [12:00<04:01, 3370.21 examples/s]
Tokenizing train dataset:  55%|█████▍    | 993921/1807941 [12:00<03:51, 3515.65 examples/s]
Tokenizing train dataset:  55%|█████▍    | 994287/1807941 [12:00<03:48, 3553.18 examples/s]
Tokenizing train dataset:  55%|█████▌    | 994683/1807941 [12:00<03:41, 3667.07 examples/s]
Tokenizing train dataset:  55%|█████▌    | 995054/1807941 [12:00<03:41, 3674.88 examples/s]
Tokenizing train dataset:  55%|█████▌    | 995600/1807941 [12:01<03:42, 3655.76 examples/s]
Tokenizing train dataset:  55%|█████▌    | 996116/1807941 [12:01<03:47, 3576.00 examples/s]
Tokenizing train dataset:  55%|█████▌    | 996611/1807941 [12:01<03:53, 3479.85 examples/s]
Tokenizing train dataset:  55%|█████▌    | 997061/1807941 [12:01<04:04, 3321.63 examples/s]
Tokenizing train dataset:  55%|█████▌    | 997570/1807941 [12:01<04:02, 3336.51 examples/s]
Tokenizing train dataset:  55%|█████▌    | 997917/1807941 [12:01<04:00, 3364.87 examples/s]
Tokenizing train dataset:  55%|█████▌    | 998403/1807941 [12:01<04:03, 3321.66 examples/s]
Tokenizing train dataset:  55%|█████▌    | 998871/1807941 [12:02<04:08, 3252.69 examples/s]
Tokenizing train dataset:  55%|█████▌    | 999338/1807941 [12:02<04:12, 3203.13 examples/s]
Tokenizing train dataset:  55%|█████▌    | 999664/1807941 [12:02<04:11, 3215.52 examples/s]
Tokenizing train dataset:  55%|█████▌    | 999998/1807941 [12:02<04:08, 3244.93 examples/s]
Tokenizing train dataset:  55%|█████▌    | 1000451/1807941 [12:02<04:15, 3164.78 examples/s]
Tokenizing train dataset:  55%|█████▌    | 1000781/1807941 [12:02<04:12, 3194.69 examples/s]
Tokenizing train dataset:  55%|█████▌    | 1001216/1807941 [12:02<04:21, 3089.18 examples/s]
Tokenizing train dataset:  55%|█████▌    | 1001539/1807941 [12:02<04:18, 3121.51 examples/s]
Tokenizing train dataset:  55%|█████▌    | 1002000/1807941 [12:03<04:23, 3053.73 examples/s]
Tokenizing train dataset:  55%|█████▌    | 1002309/1807941 [12:03<04:23, 3059.96 examples/s]
Tokenizing train dataset:  55%|█████▌    | 1002634/1807941 [12:03<04:19, 3107.42 examples/s]
Tokenizing train dataset:  55%|█████▌    | 1003111/1807941 [12:03<04:17, 3126.91 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1003551/1807941 [12:03<04:23, 3056.16 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1003873/1807941 [12:03<04:19, 3092.76 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1004347/1807941 [12:03<04:18, 3111.70 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1004793/1807941 [12:04<04:22, 3063.81 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1005102/1807941 [12:04<09:06, 1468.90 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1005359/1807941 [12:04<10:36, 1261.15 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1005760/1807941 [12:04<08:12, 1629.91 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1006083/1807941 [12:05<07:04, 1887.47 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1006382/1807941 [12:05<07:00, 1906.78 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1006654/1807941 [12:05<08:24, 1589.03 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1006913/1807941 [12:05<08:55, 1496.29 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1007113/1807941 [12:05<09:56, 1342.28 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1007306/1807941 [12:06<10:08, 1316.28 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1007454/1807941 [12:06<10:51, 1228.55 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1007613/1807941 [12:06<10:16, 1297.97 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1007830/1807941 [12:06<08:57, 1487.51 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1008000/1807941 [12:06<08:51, 1505.74 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1008172/1807941 [12:06<08:33, 1558.26 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1008401/1807941 [12:06<08:40, 1537.05 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1008584/1807941 [12:06<08:18, 1603.20 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1008757/1807941 [12:07<09:13, 1442.76 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1008921/1807941 [12:07<10:11, 1306.36 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1009065/1807941 [12:07<11:09, 1193.70 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1009197/1807941 [12:07<10:54, 1220.60 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1009397/1807941 [12:07<10:44, 1238.17 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1009525/1807941 [12:07<10:41, 1245.11 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1009695/1807941 [12:07<11:06, 1198.35 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1009861/1807941 [12:07<11:23, 1167.98 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1009982/1807941 [12:08<11:20, 1172.50 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1010159/1807941 [12:08<11:21, 1170.06 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1010331/1807941 [12:08<11:28, 1158.03 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1010455/1807941 [12:08<11:19, 1174.15 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1010591/1807941 [12:08<10:55, 1216.30 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1010759/1807941 [12:08<11:15, 1180.75 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1010890/1807941 [12:08<11:00, 1207.03 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1011053/1807941 [12:09<11:41, 1135.45 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1011174/1807941 [12:09<11:32, 1150.92 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1011340/1807941 [12:09<11:59, 1106.63 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1011464/1807941 [12:09<11:47, 1125.17 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1011594/1807941 [12:09<11:25, 1162.19 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1011718/1807941 [12:09<11:15, 1178.38 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1011891/1807941 [12:09<11:32, 1149.76 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1012056/1807941 [12:09<12:12, 1086.97 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1012193/1807941 [12:10<11:38, 1139.32 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1012335/1807941 [12:10<12:21, 1073.34 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1012466/1807941 [12:10<11:48, 1122.85 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1012625/1807941 [12:10<12:02, 1100.40 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1012759/1807941 [12:10<11:27, 1156.37 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1012898/1807941 [12:10<10:58, 1207.08 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1013068/1807941 [12:10<11:16, 1174.11 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1013195/1807941 [12:10<11:07, 1190.94 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1013320/1807941 [12:10<10:59, 1204.39 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1013448/1807941 [12:11<10:51, 1219.46 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1013644/1807941 [12:11<10:41, 1239.13 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1013821/1807941 [12:11<10:56, 1209.29 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1013957/1807941 [12:11<10:44, 1232.16 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1014138/1807941 [12:11<11:04, 1194.72 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1014325/1807941 [12:11<10:56, 1209.41 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1014560/1807941 [12:11<08:57, 1475.82 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1014790/1807941 [12:11<07:51, 1682.21 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1015000/1807941 [12:12<07:25, 1778.40 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1015226/1807941 [12:12<06:55, 1907.78 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1015470/1807941 [12:12<06:25, 2053.73 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1015708/1807941 [12:12<06:09, 2145.18 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1015952/1807941 [12:12<05:55, 2226.56 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1016289/1807941 [12:12<05:54, 2231.24 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1016534/1807941 [12:12<05:45, 2288.94 examples/s]
Tokenizing train dataset:  56%|█████▌    | 1016775/1807941 [12:12<05:41, 2317.92 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1017121/1807941 [12:13<05:43, 2304.16 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1017367/1807941 [12:13<05:37, 2342.59 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1017619/1807941 [12:13<05:30, 2387.96 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1017871/1807941 [12:13<05:26, 2422.09 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1018210/1807941 [12:13<05:34, 2358.12 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1018457/1807941 [12:13<05:31, 2384.35 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1018707/1807941 [12:13<05:26, 2413.57 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1018958/1807941 [12:13<05:23, 2438.52 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1019299/1807941 [12:13<05:32, 2372.08 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1019541/1807941 [12:14<05:31, 2381.75 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1019874/1807941 [12:14<05:39, 2321.80 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1020169/1807941 [12:14<05:58, 2198.96 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1020492/1807941 [12:14<06:00, 2181.55 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1020818/1807941 [12:14<06:01, 2175.06 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1021113/1807941 [12:14<06:13, 2105.87 examples/s]
Tokenizing train dataset:  56%|█████▋    | 1021328/1807941 [12:14<06:12, 2114.20 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1021545/1807941 [12:14<06:10, 2123.65 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1021864/1807941 [12:15<06:11, 2118.26 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1022183/1807941 [12:15<06:10, 2118.14 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1022486/1807941 [12:15<06:16, 2084.51 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1022776/1807941 [12:15<06:26, 2032.96 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1023043/1807941 [12:15<06:42, 1951.97 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1023241/1807941 [12:15<06:41, 1954.06 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1023446/1807941 [12:15<06:37, 1975.95 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1023730/1807941 [12:16<06:43, 1945.40 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1023932/1807941 [12:16<06:39, 1961.52 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1024204/1807941 [12:16<06:50, 1908.91 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1024491/1807941 [12:16<06:50, 1906.22 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1024745/1807941 [12:16<07:06, 1835.28 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1024955/1807941 [12:16<06:52, 1895.92 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1025195/1807941 [12:16<06:27, 2019.60 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1025463/1807941 [12:16<05:57, 2187.46 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1025771/1807941 [12:17<06:06, 2132.77 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1026074/1807941 [12:17<06:14, 2090.40 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1026293/1807941 [12:17<06:09, 2113.95 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1026673/1807941 [12:17<05:06, 2548.05 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1027041/1807941 [12:17<04:33, 2850.75 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1027444/1807941 [12:17<04:05, 3175.94 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1027855/1807941 [12:17<03:47, 3432.71 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1028227/1807941 [12:17<03:42, 3511.48 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1028608/1807941 [12:17<03:36, 3596.84 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1029048/1807941 [12:18<03:53, 3335.45 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1029501/1807941 [12:18<04:30, 2879.98 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1029845/1807941 [12:18<04:48, 2693.55 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1030149/1807941 [12:18<05:13, 2483.41 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1030465/1807941 [12:18<05:28, 2363.97 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1030776/1807941 [12:18<05:41, 2272.44 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1031056/1807941 [12:19<06:01, 2147.28 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1031366/1807941 [12:19<06:06, 2119.99 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1031674/1807941 [12:19<06:10, 2096.81 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1032014/1807941 [12:19<06:01, 2146.88 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1032405/1807941 [12:19<05:05, 2535.54 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1032684/1807941 [12:19<05:34, 2314.28 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1032963/1807941 [12:19<05:57, 2169.04 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1033220/1807941 [12:20<06:22, 2025.13 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1033496/1807941 [12:20<06:33, 1966.34 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1033776/1807941 [12:20<06:40, 1932.83 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1034023/1807941 [12:20<07:00, 1842.49 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1034293/1807941 [12:20<07:03, 1827.16 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1034479/1807941 [12:20<07:02, 1831.82 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1034753/1807941 [12:20<07:03, 1825.81 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1034937/1807941 [12:20<07:03, 1826.10 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1035188/1807941 [12:21<07:16, 1770.80 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1035371/1807941 [12:21<07:13, 1781.85 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1035555/1807941 [12:21<07:10, 1794.18 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1035739/1807941 [12:21<07:08, 1801.93 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1035925/1807941 [12:21<07:05, 1815.73 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1036181/1807941 [12:21<07:19, 1757.08 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1036364/1807941 [12:21<07:14, 1773.82 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1036548/1807941 [12:21<07:11, 1788.28 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1036731/1807941 [12:22<07:09, 1796.79 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1036997/1807941 [12:22<07:12, 1783.87 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1037235/1807941 [12:22<07:30, 1710.01 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1037412/1807941 [12:22<07:27, 1723.20 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1037590/1807941 [12:22<07:24, 1733.45 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1037768/1807941 [12:22<07:23, 1738.39 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1037944/1807941 [12:22<07:22, 1740.53 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1038187/1807941 [12:22<07:34, 1693.71 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1038368/1807941 [12:22<07:27, 1721.60 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1038553/1807941 [12:23<07:19, 1752.24 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1038736/1807941 [12:23<07:14, 1769.49 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1038922/1807941 [12:23<07:09, 1790.93 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1039182/1807941 [12:23<07:20, 1743.81 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1039364/1807941 [12:23<07:16, 1761.70 examples/s]
Tokenizing train dataset:  57%|█████▋    | 1039549/1807941 [12:23<07:10, 1783.09 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1039735/1807941 [12:23<07:06, 1800.09 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1039920/1807941 [12:23<07:04, 1809.78 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1040183/1807941 [12:23<07:17, 1753.39 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1040367/1807941 [12:24<07:13, 1772.52 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1040554/1807941 [12:24<07:06, 1797.60 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1040736/1807941 [12:24<07:06, 1800.83 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1040924/1807941 [12:24<07:01, 1821.04 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1041191/1807941 [12:24<07:07, 1793.51 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1041380/1807941 [12:24<07:02, 1816.23 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1041579/1807941 [12:24<06:51, 1861.14 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1041779/1807941 [12:24<06:44, 1895.46 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1041979/1807941 [12:24<06:38, 1923.00 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1042236/1807941 [12:25<06:55, 1840.93 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1042424/1807941 [12:25<06:53, 1849.64 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1042613/1807941 [12:25<06:51, 1858.17 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1042801/1807941 [12:25<06:50, 1861.97 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1043056/1807941 [12:25<07:05, 1798.20 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1043241/1807941 [12:25<07:03, 1807.52 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1043427/1807941 [12:25<06:59, 1820.83 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1043611/1807941 [12:25<06:59, 1821.62 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1043796/1807941 [12:25<06:58, 1827.55 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1043985/1807941 [12:26<06:54, 1843.80 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1044237/1807941 [12:26<07:09, 1779.03 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1044424/1807941 [12:26<07:04, 1800.27 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1044609/1807941 [12:26<07:01, 1812.51 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1044794/1807941 [12:26<06:59, 1817.93 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1044979/1807941 [12:26<06:58, 1824.13 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1045224/1807941 [12:26<07:16, 1747.24 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1045482/1807941 [12:26<07:19, 1733.77 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1045738/1807941 [12:27<07:22, 1721.03 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1045914/1807941 [12:27<07:21, 1726.88 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1046089/1807941 [12:27<07:31, 1686.74 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1046270/1807941 [12:27<07:23, 1716.70 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1046455/1807941 [12:27<07:15, 1750.12 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1046642/1807941 [12:27<07:07, 1780.35 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1046825/1807941 [12:27<07:04, 1792.20 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1047089/1807941 [12:27<07:14, 1749.75 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1047353/1807941 [12:27<07:14, 1749.77 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1047540/1807941 [12:28<07:07, 1776.96 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1047726/1807941 [12:28<07:03, 1797.19 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1047917/1807941 [12:28<06:56, 1826.68 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1048196/1807941 [12:28<06:58, 1814.61 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1048390/1807941 [12:28<06:52, 1843.18 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1048590/1807941 [12:28<06:43, 1882.59 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1048791/1807941 [12:28<06:36, 1915.33 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1048996/1807941 [12:28<06:29, 1950.24 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1049276/1807941 [12:28<06:36, 1915.35 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1049480/1807941 [12:29<06:29, 1947.17 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1049683/1807941 [12:29<06:25, 1965.89 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1049887/1807941 [12:29<06:22, 1983.79 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1050165/1807941 [12:29<06:32, 1932.16 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1050369/1807941 [12:29<06:26, 1960.32 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1050577/1807941 [12:29<06:20, 1988.80 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1050789/1807941 [12:29<06:14, 2020.72 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1050998/1807941 [12:29<06:11, 2037.49 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1051273/1807941 [12:29<06:26, 1957.76 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1051477/1807941 [12:30<06:22, 1975.63 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1051687/1807941 [12:30<06:16, 2007.48 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1051892/1807941 [12:30<06:14, 2016.79 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1052183/1807941 [12:30<06:21, 1983.39 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1052398/1807941 [12:30<06:13, 2023.39 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1052611/1807941 [12:30<06:08, 2047.55 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1052827/1807941 [12:30<06:03, 2075.27 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1053118/1807941 [12:30<06:13, 2021.66 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1053335/1807941 [12:30<06:06, 2057.88 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1053548/1807941 [12:31<06:03, 2075.22 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1053762/1807941 [12:31<06:00, 2092.72 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1053981/1807941 [12:31<05:56, 2116.63 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1054277/1807941 [12:31<06:06, 2057.47 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1054575/1807941 [12:31<06:11, 2029.28 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1054864/1807941 [12:31<06:18, 1990.69 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1055128/1807941 [12:31<06:33, 1914.64 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1055414/1807941 [12:32<06:34, 1908.33 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1055607/1807941 [12:32<06:34, 1908.73 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1055895/1807941 [12:32<06:33, 1909.71 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1056157/1807941 [12:32<06:45, 1852.89 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1056346/1807941 [12:32<06:44, 1859.02 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1056535/1807941 [12:32<06:43, 1864.22 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1056725/1807941 [12:32<06:41, 1868.98 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1057000/1807941 [12:32<06:54, 1811.63 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1057275/1807941 [12:33<06:53, 1815.86 examples/s]
Tokenizing train dataset:  58%|█████▊    | 1057459/1807941 [12:33<06:52, 1820.11 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1057648/1807941 [12:33<06:48, 1836.35 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1057836/1807941 [12:33<06:46, 1845.67 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1058094/1807941 [12:33<06:57, 1797.25 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1058278/1807941 [12:33<06:54, 1806.73 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1058466/1807941 [12:33<06:51, 1821.56 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1058653/1807941 [12:33<06:48, 1833.72 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1058932/1807941 [12:33<06:47, 1837.50 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1059187/1807941 [12:34<06:59, 1785.69 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1059372/1807941 [12:34<06:56, 1798.73 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1059561/1807941 [12:34<06:50, 1821.88 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1059751/1807941 [12:34<06:46, 1841.66 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1059952/1807941 [12:34<06:36, 1885.02 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1060222/1807941 [12:34<06:43, 1850.96 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1060424/1807941 [12:34<06:35, 1891.36 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1060624/1807941 [12:34<06:29, 1919.52 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1060823/1807941 [12:34<06:25, 1937.74 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1061099/1807941 [12:35<06:36, 1885.75 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1061299/1807941 [12:35<06:30, 1912.47 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1061503/1807941 [12:35<06:23, 1945.39 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1061705/1807941 [12:35<06:19, 1964.17 examples/s]
Tokenizing train dataset:  59%|█████▊    | 1061908/1807941 [12:35<06:16, 1982.16 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1062201/1807941 [12:35<06:24, 1938.97 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1062487/1807941 [12:35<06:27, 1922.97 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1062765/1807941 [12:35<06:33, 1895.96 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1063016/1807941 [12:36<06:48, 1821.46 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1063201/1807941 [12:36<06:47, 1825.51 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1063390/1807941 [12:36<06:45, 1838.32 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1063579/1807941 [12:36<06:42, 1850.40 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1063766/1807941 [12:36<06:41, 1852.95 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1063955/1807941 [12:36<06:40, 1859.23 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1064206/1807941 [12:36<06:56, 1785.99 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1064394/1807941 [12:36<06:51, 1807.68 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1064585/1807941 [12:36<06:45, 1831.87 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1064770/1807941 [12:37<06:44, 1835.40 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1064959/1807941 [12:37<06:42, 1847.59 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1065215/1807941 [12:37<06:54, 1791.49 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1065400/1807941 [12:37<06:51, 1803.73 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1065589/1807941 [12:37<06:47, 1823.49 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1065777/1807941 [12:37<06:44, 1836.44 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1065966/1807941 [12:37<06:41, 1850.04 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1066211/1807941 [12:37<07:01, 1761.18 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1066474/1807941 [12:38<07:02, 1754.00 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1066654/1807941 [12:38<07:00, 1761.09 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1066832/1807941 [12:38<07:00, 1764.15 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1067088/1807941 [12:38<07:08, 1728.33 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1067269/1807941 [12:38<07:03, 1748.18 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1067455/1807941 [12:38<06:57, 1774.95 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1067641/1807941 [12:38<06:51, 1797.49 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1067826/1807941 [12:38<06:49, 1809.47 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1068090/1807941 [12:38<06:59, 1763.48 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1068272/1807941 [12:39<06:56, 1775.84 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1068455/1807941 [12:39<06:53, 1787.97 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1068640/1807941 [12:39<06:50, 1801.72 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1068826/1807941 [12:39<06:47, 1814.33 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1069087/1807941 [12:39<07:01, 1753.88 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1069345/1807941 [12:39<07:04, 1739.59 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1069521/1807941 [12:39<07:04, 1739.97 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1069783/1807941 [12:39<07:04, 1739.51 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1069958/1807941 [12:40<07:04, 1739.80 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1070197/1807941 [12:40<07:17, 1686.64 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1070375/1807941 [12:40<07:12, 1706.74 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1070557/1807941 [12:40<07:05, 1733.88 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1070736/1807941 [12:40<07:01, 1747.09 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1070923/1807941 [12:40<06:54, 1777.44 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1071184/1807941 [12:40<07:03, 1739.88 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1071366/1807941 [12:40<06:58, 1759.20 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1071551/1807941 [12:40<06:53, 1781.45 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1071736/1807941 [12:41<06:49, 1797.40 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1071925/1807941 [12:41<06:43, 1822.76 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1072183/1807941 [12:41<06:56, 1765.95 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1072453/1807941 [12:41<06:54, 1773.70 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1072640/1807941 [12:41<06:49, 1794.73 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1072828/1807941 [12:41<06:45, 1813.10 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1073092/1807941 [12:41<06:53, 1777.02 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1073276/1807941 [12:41<06:50, 1788.87 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1073461/1807941 [12:41<06:47, 1801.96 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1073646/1807941 [12:42<06:45, 1811.38 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1073916/1807941 [12:42<06:47, 1801.56 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1074184/1807941 [12:42<06:57, 1759.40 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1074367/1807941 [12:42<06:53, 1776.07 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1074555/1807941 [12:42<06:47, 1801.07 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1074740/1807941 [12:42<06:44, 1811.38 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1074927/1807941 [12:42<06:41, 1826.18 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1075184/1807941 [12:42<06:53, 1770.88 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1075365/1807941 [12:43<06:51, 1778.86 examples/s]
Tokenizing train dataset:  59%|█████▉    | 1075631/1807941 [12:43<06:53, 1771.57 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1075812/1807941 [12:43<06:51, 1779.95 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1075996/1807941 [12:43<06:48, 1792.66 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1076241/1807941 [12:43<07:02, 1731.96 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1076420/1807941 [12:43<06:59, 1743.24 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1076602/1807941 [12:43<06:54, 1763.01 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1076780/1807941 [12:43<06:53, 1766.16 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1076961/1807941 [12:43<06:51, 1774.97 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1077206/1807941 [12:44<07:04, 1719.91 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1077386/1807941 [12:44<07:00, 1737.02 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1077564/1807941 [12:44<06:57, 1748.10 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1077746/1807941 [12:44<06:53, 1765.54 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1077931/1807941 [12:44<06:49, 1784.52 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1078179/1807941 [12:44<07:01, 1729.70 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1078360/1807941 [12:44<06:57, 1748.77 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1078539/1807941 [12:44<06:54, 1758.80 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1078720/1807941 [12:44<06:51, 1770.04 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1078900/1807941 [12:45<06:50, 1775.70 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1079143/1807941 [12:45<07:05, 1711.02 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1079323/1807941 [12:45<07:00, 1730.73 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1079503/1807941 [12:45<06:56, 1748.16 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1079683/1807941 [12:45<06:53, 1760.45 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1079866/1807941 [12:45<06:49, 1778.05 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1080092/1807941 [12:45<07:15, 1672.06 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1080268/1807941 [12:45<07:09, 1694.38 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1080450/1807941 [12:45<07:01, 1726.27 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1080630/1807941 [12:46<06:56, 1744.44 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1080810/1807941 [12:46<06:53, 1757.67 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1080993/1807941 [12:46<06:49, 1776.32 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1081238/1807941 [12:46<07:03, 1715.87 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1081415/1807941 [12:46<07:00, 1727.62 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1081596/1807941 [12:46<06:55, 1746.98 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1081776/1807941 [12:46<06:52, 1760.52 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1081960/1807941 [12:46<06:48, 1778.94 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1082196/1807941 [12:46<07:07, 1697.54 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1082456/1807941 [12:47<07:04, 1707.35 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1082639/1807941 [12:47<06:58, 1733.67 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1082823/1807941 [12:47<06:52, 1759.67 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1083092/1807941 [12:47<06:53, 1752.51 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1083278/1807941 [12:47<06:47, 1777.24 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1083466/1807941 [12:47<06:41, 1803.26 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1083656/1807941 [12:47<06:36, 1827.34 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1083911/1807941 [12:47<06:47, 1777.10 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1084175/1807941 [12:48<07:01, 1718.46 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1084357/1807941 [12:48<06:55, 1739.63 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1084543/1807941 [12:48<06:48, 1770.38 examples/s]
Tokenizing train dataset:  60%|█████▉    | 1084725/1807941 [12:48<06:45, 1782.99 examples/s]
Tokenizing train dataset:  60%|██████    | 1084911/1807941 [12:48<06:41, 1801.97 examples/s]
Tokenizing train dataset:  60%|██████    | 1085182/1807941 [12:48<06:52, 1753.92 examples/s]
Tokenizing train dataset:  60%|██████    | 1085366/1807941 [12:48<06:47, 1773.06 examples/s]
Tokenizing train dataset:  60%|██████    | 1085551/1807941 [12:48<06:43, 1790.10 examples/s]
Tokenizing train dataset:  60%|██████    | 1085736/1807941 [12:48<06:40, 1804.59 examples/s]
Tokenizing train dataset:  60%|██████    | 1085922/1807941 [12:49<06:37, 1818.56 examples/s]
Tokenizing train dataset:  60%|██████    | 1086180/1807941 [12:49<06:50, 1758.30 examples/s]
Tokenizing train dataset:  60%|██████    | 1086363/1807941 [12:49<06:46, 1774.24 examples/s]
Tokenizing train dataset:  60%|██████    | 1086549/1807941 [12:49<06:42, 1794.36 examples/s]
Tokenizing train dataset:  60%|██████    | 1086734/1807941 [12:49<06:39, 1806.89 examples/s]
Tokenizing train dataset:  60%|██████    | 1086920/1807941 [12:49<06:36, 1819.12 examples/s]
Tokenizing train dataset:  60%|██████    | 1087181/1807941 [12:49<06:49, 1758.10 examples/s]
Tokenizing train dataset:  60%|██████    | 1087362/1807941 [12:49<06:47, 1767.54 examples/s]
Tokenizing train dataset:  60%|██████    | 1087546/1807941 [12:49<06:43, 1786.84 examples/s]
Tokenizing train dataset:  60%|██████    | 1087732/1807941 [12:50<06:39, 1802.71 examples/s]
Tokenizing train dataset:  60%|██████    | 1087920/1807941 [12:50<06:34, 1823.32 examples/s]
Tokenizing train dataset:  60%|██████    | 1088188/1807941 [12:50<06:43, 1784.98 examples/s]
Tokenizing train dataset:  60%|██████    | 1088378/1807941 [12:50<06:37, 1812.05 examples/s]
Tokenizing train dataset:  60%|██████    | 1088569/1807941 [12:50<06:31, 1838.00 examples/s]
Tokenizing train dataset:  60%|██████    | 1088764/1807941 [12:50<06:25, 1867.34 examples/s]
Tokenizing train dataset:  60%|██████    | 1088964/1807941 [12:50<06:17, 1903.28 examples/s]
Tokenizing train dataset:  60%|██████    | 1089242/1807941 [12:50<06:22, 1879.80 examples/s]
Tokenizing train dataset:  60%|██████    | 1089444/1807941 [12:50<06:15, 1914.97 examples/s]
Tokenizing train dataset:  60%|██████    | 1089647/1807941 [12:51<06:09, 1946.04 examples/s]
Tokenizing train dataset:  60%|██████    | 1089854/1807941 [12:51<06:03, 1977.20 examples/s]
Tokenizing train dataset:  60%|██████    | 1090138/1807941 [12:51<06:10, 1939.87 examples/s]
Tokenizing train dataset:  60%|██████    | 1090337/1807941 [12:51<06:08, 1949.91 examples/s]
Tokenizing train dataset:  60%|██████    | 1090539/1807941 [12:51<06:05, 1965.34 examples/s]
Tokenizing train dataset:  60%|██████    | 1090751/1807941 [12:51<05:57, 2004.72 examples/s]
Tokenizing train dataset:  60%|██████    | 1090965/1807941 [12:51<05:50, 2042.74 examples/s]
Tokenizing train dataset:  60%|██████    | 1091262/1807941 [12:51<05:55, 2014.77 examples/s]
Tokenizing train dataset:  60%|██████    | 1091472/1807941 [12:51<05:51, 2035.48 examples/s]
Tokenizing train dataset:  60%|██████    | 1091684/1807941 [12:52<05:48, 2056.00 examples/s]
Tokenizing train dataset:  60%|██████    | 1091898/1807941 [12:52<05:44, 2077.84 examples/s]
Tokenizing train dataset:  60%|██████    | 1092214/1807941 [12:52<05:50, 2042.81 examples/s]
Tokenizing train dataset:  60%|██████    | 1092430/1807941 [12:52<05:45, 2072.05 examples/s]
Tokenizing train dataset:  60%|██████    | 1092649/1807941 [12:52<05:40, 2098.96 examples/s]
Tokenizing train dataset:  60%|██████    | 1092867/1807941 [12:52<05:37, 2119.37 examples/s]
Tokenizing train dataset:  60%|██████    | 1093172/1807941 [12:52<05:42, 2084.40 examples/s]
Tokenizing train dataset:  60%|██████    | 1093387/1807941 [12:52<05:40, 2100.24 examples/s]
Tokenizing train dataset:  60%|██████    | 1093604/1807941 [12:53<05:37, 2117.48 examples/s]
Tokenizing train dataset:  61%|██████    | 1093823/1807941 [12:53<05:34, 2136.76 examples/s]
Tokenizing train dataset:  61%|██████    | 1094131/1807941 [12:53<05:39, 2101.74 examples/s]
Tokenizing train dataset:  61%|██████    | 1094347/1807941 [12:53<05:37, 2113.15 examples/s]
Tokenizing train dataset:  61%|██████    | 1094564/1807941 [12:53<05:35, 2126.60 examples/s]
Tokenizing train dataset:  61%|██████    | 1094788/1807941 [12:53<05:30, 2155.53 examples/s]
Tokenizing train dataset:  61%|██████    | 1095109/1807941 [12:53<05:33, 2135.63 examples/s]
Tokenizing train dataset:  61%|██████    | 1095331/1807941 [12:53<05:30, 2155.76 examples/s]
Tokenizing train dataset:  61%|██████    | 1095550/1807941 [12:53<05:29, 2161.09 examples/s]
Tokenizing train dataset:  61%|██████    | 1095773/1807941 [12:54<05:27, 2177.58 examples/s]
Tokenizing train dataset:  61%|██████    | 1095996/1807941 [12:54<05:25, 2189.94 examples/s]
Tokenizing train dataset:  61%|██████    | 1096291/1807941 [12:54<05:38, 2104.32 examples/s]
Tokenizing train dataset:  61%|██████    | 1096512/1807941 [12:54<05:34, 2129.01 examples/s]
Tokenizing train dataset:  61%|██████    | 1096824/1807941 [12:54<05:37, 2105.42 examples/s]
Tokenizing train dataset:  61%|██████    | 1097099/1807941 [12:54<05:53, 2009.19 examples/s]
Tokenizing train dataset:  61%|██████    | 1097391/1807941 [12:54<05:58, 1983.57 examples/s]
Tokenizing train dataset:  61%|██████    | 1097683/1807941 [12:54<06:00, 1967.85 examples/s]
Tokenizing train dataset:  61%|██████    | 1097978/1807941 [12:55<06:01, 1963.40 examples/s]
Tokenizing train dataset:  61%|██████    | 1098241/1807941 [12:55<06:14, 1894.64 examples/s]
Tokenizing train dataset:  61%|██████    | 1098434/1807941 [12:55<06:13, 1899.04 examples/s]
Tokenizing train dataset:  61%|██████    | 1098628/1807941 [12:55<06:12, 1905.68 examples/s]
Tokenizing train dataset:  61%|██████    | 1098821/1807941 [12:55<06:11, 1908.58 examples/s]
Tokenizing train dataset:  61%|██████    | 1099092/1807941 [12:55<06:25, 1839.55 examples/s]
Tokenizing train dataset:  61%|██████    | 1099364/1807941 [12:55<06:27, 1827.05 examples/s]
Tokenizing train dataset:  61%|██████    | 1099554/1807941 [12:55<06:24, 1843.63 examples/s]
Tokenizing train dataset:  61%|██████    | 1099742/1807941 [12:56<06:22, 1851.25 examples/s]
Tokenizing train dataset:  61%|██████    | 1099934/1807941 [12:56<06:19, 1866.95 examples/s]
Tokenizing train dataset:  61%|██████    | 1100195/1807941 [12:56<06:29, 1815.08 examples/s]
Tokenizing train dataset:  61%|██████    | 1100474/1807941 [12:56<06:27, 1827.69 examples/s]
Tokenizing train dataset:  61%|██████    | 1100664/1807941 [12:56<06:23, 1842.31 examples/s]
Tokenizing train dataset:  61%|██████    | 1100944/1807941 [12:56<06:22, 1849.10 examples/s]
Tokenizing train dataset:  61%|██████    | 1101199/1807941 [12:56<06:33, 1798.10 examples/s]
Tokenizing train dataset:  61%|██████    | 1101388/1807941 [12:56<06:28, 1818.03 examples/s]
Tokenizing train dataset:  61%|██████    | 1101580/1807941 [12:57<06:23, 1842.68 examples/s]
Tokenizing train dataset:  61%|██████    | 1101773/1807941 [12:57<06:18, 1864.14 examples/s]
Tokenizing train dataset:  61%|██████    | 1101969/1807941 [12:57<06:14, 1887.22 examples/s]
Tokenizing train dataset:  61%|██████    | 1102236/1807941 [12:57<06:22, 1844.29 examples/s]
Tokenizing train dataset:  61%|██████    | 1102432/1807941 [12:57<06:17, 1869.91 examples/s]
Tokenizing train dataset:  61%|██████    | 1102639/1807941 [12:57<06:07, 1921.66 examples/s]
Tokenizing train dataset:  61%|██████    | 1102839/1807941 [12:57<06:03, 1938.40 examples/s]
Tokenizing train dataset:  61%|██████    | 1103100/1807941 [12:57<06:19, 1859.69 examples/s]
Tokenizing train dataset:  61%|██████    | 1103382/1807941 [12:58<06:18, 1860.28 examples/s]
Tokenizing train dataset:  61%|██████    | 1103575/1807941 [12:58<06:15, 1874.04 examples/s]
Tokenizing train dataset:  61%|██████    | 1103765/1807941 [12:58<06:15, 1877.73 examples/s]
Tokenizing train dataset:  61%|██████    | 1103957/1807941 [12:58<06:13, 1886.15 examples/s]
Tokenizing train dataset:  61%|██████    | 1104216/1807941 [12:58<06:26, 1822.49 examples/s]
Tokenizing train dataset:  61%|██████    | 1104408/1807941 [12:58<06:21, 1846.16 examples/s]
Tokenizing train dataset:  61%|██████    | 1104604/1807941 [12:58<06:15, 1873.93 examples/s]
Tokenizing train dataset:  61%|██████    | 1104798/1807941 [12:58<06:12, 1889.64 examples/s]
Tokenizing train dataset:  61%|██████    | 1104991/1807941 [12:58<06:10, 1899.72 examples/s]
Tokenizing train dataset:  61%|██████    | 1105249/1807941 [12:59<06:24, 1825.95 examples/s]
Tokenizing train dataset:  61%|██████    | 1105445/1807941 [12:59<06:17, 1858.50 examples/s]
Tokenizing train dataset:  61%|██████    | 1105653/1807941 [12:59<06:06, 1915.54 examples/s]
Tokenizing train dataset:  61%|██████    | 1105864/1807941 [12:59<05:56, 1966.98 examples/s]
Tokenizing train dataset:  61%|██████    | 1106141/1807941 [12:59<06:06, 1916.46 examples/s]
Tokenizing train dataset:  61%|██████    | 1106338/1807941 [12:59<06:04, 1927.12 examples/s]
Tokenizing train dataset:  61%|██████    | 1106546/1807941 [12:59<05:56, 1965.29 examples/s]
Tokenizing train dataset:  61%|██████    | 1106757/1807941 [12:59<05:49, 2003.70 examples/s]
Tokenizing train dataset:  61%|██████    | 1107029/1807941 [12:59<06:03, 1927.24 examples/s]
Tokenizing train dataset:  61%|██████    | 1107316/1807941 [13:00<06:04, 1920.69 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1107603/1807941 [13:00<06:05, 1916.45 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1107891/1807941 [13:00<06:05, 1913.98 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1108153/1807941 [13:00<06:16, 1858.71 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1108346/1807941 [13:00<06:13, 1874.06 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1108542/1807941 [13:00<06:09, 1892.96 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1108736/1807941 [13:00<06:07, 1903.69 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1108929/1807941 [13:00<06:06, 1908.39 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1109187/1807941 [13:01<06:20, 1834.67 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1109460/1807941 [13:01<06:22, 1826.17 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1109651/1807941 [13:01<06:18, 1844.26 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1109842/1807941 [13:01<06:15, 1856.65 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1110101/1807941 [13:01<06:26, 1805.76 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1110288/1807941 [13:01<06:23, 1819.66 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1110476/1807941 [13:01<06:20, 1831.74 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1110667/1807941 [13:01<06:17, 1848.57 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1110859/1807941 [13:02<06:13, 1866.68 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1111121/1807941 [13:02<06:23, 1818.35 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1111311/1807941 [13:02<06:19, 1837.36 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1111502/1807941 [13:02<06:15, 1853.93 examples/s]
Tokenizing train dataset:  61%|██████▏   | 1111693/1807941 [13:02<06:13, 1866.57 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1111887/1807941 [13:02<06:09, 1882.10 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1112149/1807941 [13:02<06:21, 1825.54 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1112339/1807941 [13:02<06:17, 1842.26 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1112527/1807941 [13:02<06:15, 1851.40 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1112808/1807941 [13:03<06:14, 1855.37 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1112997/1807941 [13:03<06:13, 1861.96 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1113255/1807941 [13:03<06:23, 1809.52 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1113445/1807941 [13:03<06:19, 1827.96 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1113634/1807941 [13:03<06:17, 1840.80 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1113822/1807941 [13:03<06:15, 1847.84 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1114091/1807941 [13:03<06:26, 1793.61 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1114275/1807941 [13:03<06:24, 1801.83 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1114462/1807941 [13:03<06:21, 1815.50 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1114650/1807941 [13:04<06:18, 1831.03 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1114836/1807941 [13:04<06:17, 1835.49 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1115092/1807941 [13:04<06:28, 1781.54 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1115278/1807941 [13:04<06:24, 1799.99 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1115467/1807941 [13:04<06:19, 1823.48 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1115657/1807941 [13:04<06:15, 1844.17 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1115848/1807941 [13:04<06:12, 1858.40 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1116104/1807941 [13:04<06:25, 1794.47 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1116289/1807941 [13:04<06:22, 1808.54 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1116478/1807941 [13:05<06:18, 1825.40 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1116668/1807941 [13:05<06:14, 1844.58 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1116857/1807941 [13:05<06:12, 1855.63 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1117112/1807941 [13:05<06:25, 1792.69 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1117301/1807941 [13:05<06:20, 1816.24 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1117490/1807941 [13:05<06:16, 1833.49 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1117763/1807941 [13:05<06:18, 1825.16 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1117952/1807941 [13:05<06:15, 1837.20 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1118207/1807941 [13:06<06:26, 1786.44 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1118393/1807941 [13:06<06:22, 1802.47 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1118578/1807941 [13:06<06:20, 1813.18 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1118762/1807941 [13:06<06:18, 1818.86 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1118952/1807941 [13:06<06:14, 1838.31 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1119209/1807941 [13:06<06:25, 1788.27 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1119400/1807941 [13:06<06:18, 1817.94 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1119590/1807941 [13:06<06:14, 1837.38 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1119778/1807941 [13:06<06:12, 1845.20 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1120029/1807941 [13:07<06:26, 1778.88 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1120217/1807941 [13:07<06:21, 1804.16 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1120407/1807941 [13:07<06:16, 1825.99 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1120599/1807941 [13:07<06:11, 1849.31 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1120790/1807941 [13:07<06:08, 1863.95 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1120991/1807941 [13:07<06:00, 1903.35 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1121254/1807941 [13:07<06:12, 1843.04 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1121443/1807941 [13:07<06:10, 1852.72 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1121705/1807941 [13:07<06:18, 1811.30 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1121891/1807941 [13:08<06:16, 1820.84 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1122145/1807941 [13:08<06:27, 1771.50 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1122335/1807941 [13:08<06:20, 1801.77 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1122527/1807941 [13:08<06:14, 1828.11 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1122719/1807941 [13:08<06:10, 1849.57 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1122915/1807941 [13:08<06:04, 1879.18 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1123186/1807941 [13:08<06:16, 1816.69 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1123376/1807941 [13:08<06:12, 1836.89 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1123566/1807941 [13:08<06:09, 1851.46 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1123756/1807941 [13:09<06:07, 1864.10 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1123947/1807941 [13:09<06:04, 1875.14 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1124208/1807941 [13:09<06:15, 1820.68 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1124404/1807941 [13:09<06:08, 1855.49 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1124606/1807941 [13:09<06:00, 1896.99 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1124809/1807941 [13:09<05:53, 1931.25 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1125102/1807941 [13:09<05:56, 1913.97 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1125305/1807941 [13:09<05:51, 1942.40 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1125511/1807941 [13:09<05:45, 1972.61 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1125810/1807941 [13:10<05:44, 1978.72 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1126098/1807941 [13:10<05:52, 1935.87 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1126298/1807941 [13:10<05:49, 1948.73 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1126504/1807941 [13:10<05:44, 1976.45 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1126705/1807941 [13:10<05:43, 1981.88 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1126908/1807941 [13:10<05:42, 1991.07 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1127211/1807941 [13:10<05:45, 1970.38 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1127417/1807941 [13:10<05:42, 1988.69 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1127620/1807941 [13:11<05:40, 1996.44 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1127835/1807941 [13:11<05:34, 2035.13 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1128125/1807941 [13:11<05:40, 1996.09 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1128336/1807941 [13:11<05:35, 2023.17 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1128643/1807941 [13:11<05:34, 2028.91 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1128854/1807941 [13:11<05:31, 2048.11 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1129141/1807941 [13:11<05:39, 1998.04 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1129347/1807941 [13:11<05:37, 2012.20 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1129557/1807941 [13:12<05:33, 2032.36 examples/s]
Tokenizing train dataset:  62%|██████▏   | 1129769/1807941 [13:12<05:30, 2053.55 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1129976/1807941 [13:12<05:30, 2054.39 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1130257/1807941 [13:12<05:41, 1982.48 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1130463/1807941 [13:12<05:38, 1998.76 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1130679/1807941 [13:12<05:32, 2036.16 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1130896/1807941 [13:12<05:27, 2069.94 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1131208/1807941 [13:12<05:34, 2024.40 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1131419/1807941 [13:12<05:30, 2045.26 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1131717/1807941 [13:13<05:34, 2021.56 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1132000/1807941 [13:13<05:49, 1933.71 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1132279/1807941 [13:13<05:54, 1906.88 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1132558/1807941 [13:13<05:57, 1888.57 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1132838/1807941 [13:13<05:59, 1877.99 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1133104/1807941 [13:13<06:06, 1841.38 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1133294/1807941 [13:13<06:45, 1664.03 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1133508/1807941 [13:14<07:04, 1588.66 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1133706/1807941 [13:14<07:28, 1504.92 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1133927/1807941 [13:14<07:31, 1492.73 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1134108/1807941 [13:14<07:28, 1500.88 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1134421/1807941 [13:14<05:58, 1880.22 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1134680/1807941 [13:14<05:27, 2053.85 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1134913/1807941 [13:14<05:17, 2123.09 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1135161/1807941 [13:14<05:03, 2218.52 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1135551/1807941 [13:15<04:10, 2687.38 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1136000/1807941 [13:15<03:31, 3183.18 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1136430/1807941 [13:15<03:11, 3499.31 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1136836/1807941 [13:15<03:03, 3660.76 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1137222/1807941 [13:15<03:00, 3715.64 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1137639/1807941 [13:15<02:54, 3844.24 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1138199/1807941 [13:15<02:56, 3797.93 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1138784/1807941 [13:15<02:54, 3833.35 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1139367/1807941 [13:16<02:53, 3849.56 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1139765/1807941 [13:16<02:52, 3877.76 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1140349/1807941 [13:16<02:52, 3880.91 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1140760/1807941 [13:16<02:49, 3935.94 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1141348/1807941 [13:16<02:49, 3927.46 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1141757/1807941 [13:16<02:47, 3966.36 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1142342/1807941 [13:16<02:48, 3940.12 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1142793/1807941 [13:16<02:43, 4079.57 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1143228/1807941 [13:16<02:40, 4134.95 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1143657/1807941 [13:17<02:39, 4174.22 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1144290/1807941 [13:17<02:38, 4184.21 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1144727/1807941 [13:17<02:36, 4226.57 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1145375/1807941 [13:17<02:35, 4257.21 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1145848/1807941 [13:17<02:31, 4376.11 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1146292/1807941 [13:17<02:30, 4389.79 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1146774/1807941 [13:17<02:26, 4504.36 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1147410/1807941 [13:17<02:31, 4352.68 examples/s]
Tokenizing train dataset:  63%|██████▎   | 1148000/1807941 [13:18<02:37, 4186.40 examples/s]
Tokenizing train dataset:  64%|██████▎   | 1148597/1807941 [13:18<02:40, 4115.78 examples/s]
Tokenizing train dataset:  64%|██████▎   | 1149206/1807941 [13:18<02:42, 4043.81 examples/s]
Tokenizing train dataset:  64%|██████▎   | 1149619/1807941 [13:18<02:42, 4062.26 examples/s]
Tokenizing train dataset:  64%|██████▎   | 1150222/1807941 [13:18<02:42, 4045.19 examples/s]
Tokenizing train dataset:  64%|██████▎   | 1150683/1807941 [13:18<02:37, 4178.21 examples/s]
Tokenizing train dataset:  64%|██████▎   | 1151121/1807941 [13:18<02:35, 4227.81 examples/s]
Tokenizing train dataset:  64%|██████▎   | 1151566/1807941 [13:18<02:33, 4283.58 examples/s]
Tokenizing train dataset:  64%|██████▎   | 1152007/1807941 [13:19<02:31, 4315.71 examples/s]
Tokenizing train dataset:  64%|██████▎   | 1152446/1807941 [13:19<02:31, 4331.69 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1153022/1807941 [13:19<02:38, 4143.87 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1153626/1807941 [13:19<02:39, 4097.32 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1154206/1807941 [13:19<02:42, 4016.51 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1154615/1807941 [13:19<02:42, 4032.59 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1155200/1807941 [13:19<02:43, 3982.47 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1155603/1807941 [13:19<02:43, 3990.37 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1156206/1807941 [13:20<02:44, 3958.73 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1156605/1807941 [13:20<02:44, 3964.61 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1157202/1807941 [13:20<02:44, 3948.74 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1157611/1807941 [13:20<02:43, 3980.13 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1158199/1807941 [13:20<02:45, 3919.96 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1158602/1807941 [13:20<02:44, 3943.49 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1159000/1807941 [13:20<02:45, 3912.20 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1159395/1807941 [13:20<02:45, 3916.98 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1159863/1807941 [13:21<02:58, 3622.16 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1160430/1807941 [13:21<02:56, 3673.03 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1160845/1807941 [13:21<02:50, 3787.34 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1161407/1807941 [13:21<02:51, 3769.75 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1161798/1807941 [13:21<02:49, 3802.60 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1162355/1807941 [13:21<02:51, 3769.12 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1162746/1807941 [13:21<02:49, 3801.64 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1163306/1807941 [13:21<02:50, 3775.24 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1163697/1807941 [13:22<02:49, 3805.74 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1164243/1807941 [13:22<02:51, 3746.08 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1164806/1807941 [13:22<02:51, 3744.22 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1165189/1807941 [13:22<02:53, 3704.48 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1165561/1807941 [13:22<03:15, 3287.47 examples/s]
Tokenizing train dataset:  64%|██████▍   | 1165990/1807941 [13:22<04:08, 2584.60 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1166334/1807941 [13:23<04:16, 2501.41 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1166633/1807941 [13:23<04:32, 2354.91 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1166946/1807941 [13:23<04:41, 2275.06 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1167282/1807941 [13:23<04:43, 2262.67 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1167524/1807941 [13:23<04:39, 2293.78 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1167834/1807941 [13:23<04:48, 2215.29 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1168133/1807941 [13:23<04:58, 2143.51 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1168372/1807941 [13:23<04:51, 2197.67 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1168610/1807941 [13:24<05:20, 1993.04 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1168856/1807941 [13:24<05:40, 1877.29 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1169084/1807941 [13:24<06:02, 1762.89 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1169333/1807941 [13:24<06:09, 1727.27 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1169592/1807941 [13:24<06:10, 1723.82 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1169767/1807941 [13:24<06:09, 1728.29 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1170000/1807941 [13:24<06:24, 1660.78 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1170240/1807941 [13:25<06:29, 1637.87 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1170486/1807941 [13:25<06:30, 1634.30 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1170652/1807941 [13:25<06:29, 1637.00 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1170819/1807941 [13:25<06:27, 1642.70 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1171000/1807941 [13:25<06:32, 1623.07 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1171177/1807941 [13:25<06:23, 1660.24 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1171346/1807941 [13:25<06:22, 1665.28 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1171597/1807941 [13:25<06:22, 1663.51 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1171771/1807941 [13:26<06:18, 1680.07 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1171953/1807941 [13:26<06:10, 1715.21 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1172185/1807941 [13:26<06:24, 1652.08 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1172359/1807941 [13:26<06:19, 1673.67 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1172542/1807941 [13:26<06:10, 1713.97 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1172722/1807941 [13:26<06:05, 1736.21 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1172904/1807941 [13:26<06:01, 1756.48 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1173095/1807941 [13:26<06:05, 1735.83 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1173282/1807941 [13:26<05:57, 1773.61 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1173470/1807941 [13:27<05:51, 1802.76 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1173660/1807941 [13:27<05:46, 1830.37 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1173860/1807941 [13:27<05:37, 1879.31 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1174143/1807941 [13:27<05:37, 1877.39 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1174345/1807941 [13:27<05:30, 1914.78 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1174540/1807941 [13:27<05:29, 1922.89 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1174736/1807941 [13:27<05:28, 1929.73 examples/s]
Tokenizing train dataset:  65%|██████▍   | 1174939/1807941 [13:27<05:23, 1955.37 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1175202/1807941 [13:27<05:37, 1873.57 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1175399/1807941 [13:28<05:33, 1894.67 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1175591/1807941 [13:28<05:33, 1897.40 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1175787/1807941 [13:28<05:30, 1911.99 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1175992/1807941 [13:28<05:24, 1948.30 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1176281/1807941 [13:28<05:26, 1933.31 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1176492/1807941 [13:28<05:19, 1977.13 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1176698/1807941 [13:28<05:16, 1995.38 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1176913/1807941 [13:28<05:10, 2035.16 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1177211/1807941 [13:28<05:15, 1997.72 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1177420/1807941 [13:29<05:12, 2019.63 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1177633/1807941 [13:29<05:07, 2046.92 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1177844/1807941 [13:29<05:05, 2062.34 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1178128/1807941 [13:29<05:15, 1995.63 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1178341/1807941 [13:29<05:10, 2028.19 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1178557/1807941 [13:29<05:05, 2061.29 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1178771/1807941 [13:29<05:02, 2082.23 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1178985/1807941 [13:29<05:00, 2095.23 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1179278/1807941 [13:29<05:08, 2037.74 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1179494/1807941 [13:30<05:03, 2067.60 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1179704/1807941 [13:30<05:03, 2071.88 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1179917/1807941 [13:30<05:01, 2084.16 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1180212/1807941 [13:30<05:08, 2034.84 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1180512/1807941 [13:30<05:10, 2019.48 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1180801/1807941 [13:30<05:15, 1985.95 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1181068/1807941 [13:30<05:26, 1917.07 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1181349/1807941 [13:30<05:29, 1899.96 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1181629/1807941 [13:31<05:31, 1887.99 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1181909/1807941 [13:31<05:33, 1879.91 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1182182/1807941 [13:31<05:44, 1814.28 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1182456/1807941 [13:31<05:44, 1814.25 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1182643/1807941 [13:31<05:42, 1824.41 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1182837/1807941 [13:31<05:37, 1849.61 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1183114/1807941 [13:31<05:38, 1845.48 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1183318/1807941 [13:32<05:30, 1890.07 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1183602/1807941 [13:32<05:30, 1888.35 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1183797/1807941 [13:32<05:28, 1901.11 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1184000/1807941 [13:32<05:31, 1881.37 examples/s]
Tokenizing train dataset:  65%|██████▌   | 1184200/1807941 [13:32<05:26, 1909.24 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1184395/1807941 [13:32<05:24, 1919.19 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1184679/1807941 [13:32<05:27, 1904.57 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1184959/1807941 [13:32<05:29, 1888.87 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1185214/1807941 [13:33<05:41, 1821.13 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1185398/1807941 [13:33<05:41, 1823.70 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1185587/1807941 [13:33<05:38, 1837.22 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1185774/1807941 [13:33<05:37, 1843.27 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1185963/1807941 [13:33<05:35, 1852.85 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1186216/1807941 [13:33<05:48, 1786.26 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1186401/1807941 [13:33<05:45, 1801.12 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1186593/1807941 [13:33<05:39, 1830.27 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1186787/1807941 [13:33<05:34, 1858.76 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1186982/1807941 [13:34<05:30, 1881.57 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1187243/1807941 [13:34<05:40, 1824.54 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1187435/1807941 [13:34<05:36, 1845.76 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1187628/1807941 [13:34<05:32, 1866.28 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1187832/1807941 [13:34<05:24, 1911.57 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1188109/1807941 [13:34<05:28, 1884.61 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1188390/1807941 [13:34<05:30, 1875.71 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1188668/1807941 [13:34<05:32, 1863.44 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1188948/1807941 [13:35<05:32, 1862.84 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1189198/1807941 [13:35<05:43, 1799.61 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1189380/1807941 [13:35<05:43, 1801.24 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1189568/1807941 [13:35<05:40, 1816.68 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1189757/1807941 [13:35<05:37, 1834.36 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1189947/1807941 [13:35<05:34, 1850.02 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1190204/1807941 [13:35<05:43, 1796.83 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1190390/1807941 [13:35<05:41, 1809.95 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1190661/1807941 [13:36<05:41, 1805.51 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1190847/1807941 [13:36<05:39, 1817.46 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1191103/1807941 [13:36<05:47, 1774.77 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1191285/1807941 [13:36<05:45, 1784.62 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1191472/1807941 [13:36<05:41, 1803.73 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1191657/1807941 [13:36<05:39, 1813.56 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1191844/1807941 [13:36<05:37, 1825.86 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1192101/1807941 [13:36<05:45, 1781.05 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1192287/1807941 [13:36<05:42, 1799.36 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1192471/1807941 [13:37<05:40, 1806.13 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1192658/1807941 [13:37<05:37, 1823.45 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1192844/1807941 [13:37<05:36, 1830.60 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1193098/1807941 [13:37<05:46, 1774.28 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1193281/1807941 [13:37<05:44, 1785.46 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1193466/1807941 [13:37<05:41, 1799.54 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1193654/1807941 [13:37<05:37, 1820.77 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1193841/1807941 [13:37<05:35, 1831.99 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1194087/1807941 [13:37<05:53, 1736.16 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1194265/1807941 [13:38<05:51, 1747.49 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1194449/1807941 [13:38<05:46, 1768.31 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1194632/1807941 [13:38<05:43, 1784.16 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1194816/1807941 [13:38<05:41, 1797.81 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1195000/1807941 [13:38<05:51, 1745.18 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1195180/1807941 [13:38<05:48, 1757.50 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1195364/1807941 [13:38<05:44, 1776.54 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1195548/1807941 [13:38<05:42, 1790.58 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1195732/1807941 [13:38<05:39, 1802.34 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1195918/1807941 [13:38<05:36, 1817.21 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1196182/1807941 [13:39<05:48, 1754.61 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1196365/1807941 [13:39<05:45, 1772.01 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1196551/1807941 [13:39<05:41, 1792.68 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1196737/1807941 [13:39<05:38, 1807.71 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1196921/1807941 [13:39<05:36, 1813.90 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1197183/1807941 [13:39<05:48, 1753.27 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1197368/1807941 [13:39<05:43, 1776.09 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1197556/1807941 [13:39<05:39, 1799.83 examples/s]
Tokenizing train dataset:  66%|██████▌   | 1197742/1807941 [13:39<05:36, 1812.76 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1197927/1807941 [13:40<05:35, 1819.97 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1198182/1807941 [13:40<05:46, 1762.19 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1198361/1807941 [13:40<05:44, 1767.95 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1198546/1807941 [13:40<05:41, 1785.87 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1198728/1807941 [13:40<05:40, 1791.37 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1198918/1807941 [13:40<05:34, 1818.13 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1199182/1807941 [13:40<05:45, 1762.50 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1199366/1807941 [13:40<05:41, 1782.15 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1199550/1807941 [13:41<05:38, 1795.59 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1199736/1807941 [13:41<05:35, 1810.48 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1199922/1807941 [13:41<05:34, 1820.26 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1200170/1807941 [13:41<05:50, 1735.12 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1200352/1807941 [13:41<05:45, 1756.54 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1200537/1807941 [13:41<05:41, 1779.15 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1200724/1807941 [13:41<05:36, 1802.05 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1200914/1807941 [13:41<05:32, 1827.57 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1201190/1807941 [13:41<05:38, 1793.36 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1201382/1807941 [13:42<05:32, 1822.75 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1201573/1807941 [13:42<05:28, 1843.72 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1201769/1807941 [13:42<05:23, 1874.31 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1202027/1807941 [13:42<05:34, 1810.96 examples/s]
Tokenizing train dataset:  66%|██████▋   | 1202214/1807941 [13:42<05:32, 1822.63 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1202401/1807941 [13:42<05:30, 1832.93 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1202590/1807941 [13:42<05:27, 1848.02 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1202778/1807941 [13:42<05:26, 1853.83 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1202968/1807941 [13:42<05:24, 1864.35 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1203223/1807941 [13:43<05:36, 1798.61 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1203410/1807941 [13:43<05:33, 1814.91 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1203600/1807941 [13:43<05:29, 1834.90 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1203787/1807941 [13:43<05:27, 1842.71 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1203973/1807941 [13:43<05:27, 1844.98 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1204226/1807941 [13:43<05:39, 1779.01 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1204411/1807941 [13:43<05:36, 1792.50 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1204598/1807941 [13:43<05:33, 1810.25 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1204784/1807941 [13:43<05:31, 1821.89 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1204971/1807941 [13:43<05:29, 1830.65 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1205223/1807941 [13:44<05:40, 1768.20 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1205409/1807941 [13:44<05:36, 1791.78 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1205597/1807941 [13:44<05:32, 1810.68 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1205784/1807941 [13:44<05:29, 1824.72 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1205970/1807941 [13:44<05:28, 1833.31 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1206223/1807941 [13:44<05:39, 1773.48 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1206409/1807941 [13:44<05:35, 1794.67 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1206597/1807941 [13:44<05:30, 1816.98 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1206788/1807941 [13:44<05:26, 1841.26 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1206985/1807941 [13:45<05:20, 1874.09 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1207258/1807941 [13:45<05:25, 1847.57 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1207451/1807941 [13:45<05:21, 1868.11 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1207644/1807941 [13:45<05:18, 1882.16 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1207844/1807941 [13:45<05:13, 1912.49 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1208098/1807941 [13:45<05:28, 1826.79 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1208371/1807941 [13:45<05:29, 1821.64 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1208564/1807941 [13:45<05:24, 1846.73 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1208760/1807941 [13:46<05:20, 1871.71 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1208965/1807941 [13:46<05:12, 1916.13 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1209246/1807941 [13:46<05:15, 1897.82 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1209452/1807941 [13:46<05:09, 1936.70 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1209664/1807941 [13:46<05:01, 1981.97 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1209952/1807941 [13:46<05:05, 1957.56 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1210218/1807941 [13:46<05:16, 1891.39 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1210424/1807941 [13:46<05:09, 1932.68 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1210624/1807941 [13:47<05:06, 1947.23 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1210923/1807941 [13:47<05:04, 1961.09 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1211201/1807941 [13:47<05:11, 1917.04 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1211404/1807941 [13:47<05:07, 1942.15 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1211612/1807941 [13:47<05:01, 1977.38 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1211816/1807941 [13:47<04:59, 1990.33 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1212094/1807941 [13:47<05:08, 1934.40 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1212349/1807941 [13:47<05:21, 1851.12 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1212624/1807941 [13:48<05:23, 1839.28 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1212899/1807941 [13:48<05:24, 1832.11 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1213149/1807941 [13:48<05:34, 1777.60 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1213333/1807941 [13:48<05:32, 1789.99 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1213517/1807941 [13:48<05:29, 1801.36 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1213701/1807941 [13:48<05:28, 1811.04 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1213891/1807941 [13:48<05:24, 1832.17 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1214145/1807941 [13:48<05:34, 1777.31 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1214332/1807941 [13:49<05:29, 1800.30 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1214524/1807941 [13:49<05:24, 1831.08 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1214717/1807941 [13:49<05:19, 1855.87 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1214910/1807941 [13:49<05:16, 1875.07 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1215201/1807941 [13:49<05:17, 1865.98 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1215400/1807941 [13:49<05:12, 1896.32 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1215600/1807941 [13:49<05:08, 1921.76 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1215809/1807941 [13:49<05:01, 1965.82 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1216094/1807941 [13:49<05:11, 1900.04 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1216379/1807941 [13:50<05:12, 1895.25 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1216662/1807941 [13:50<05:13, 1888.73 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1216942/1807941 [13:50<05:14, 1879.61 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1217196/1807941 [13:50<05:25, 1816.67 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1217382/1807941 [13:50<05:23, 1824.74 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1217571/1807941 [13:50<05:21, 1836.76 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1217758/1807941 [13:50<05:20, 1842.21 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1217947/1807941 [13:50<05:18, 1853.11 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1218199/1807941 [13:51<05:30, 1785.76 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1218384/1807941 [13:51<05:27, 1799.66 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1218572/1807941 [13:51<05:24, 1818.69 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1218759/1807941 [13:51<05:22, 1829.66 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1218947/1807941 [13:51<05:19, 1840.93 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1219201/1807941 [13:51<05:30, 1781.54 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1219387/1807941 [13:51<05:26, 1801.17 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1219574/1807941 [13:51<05:23, 1817.74 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1219760/1807941 [13:51<05:21, 1828.88 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1219947/1807941 [13:52<05:19, 1839.19 examples/s]
Tokenizing train dataset:  67%|██████▋   | 1220201/1807941 [13:52<05:30, 1779.00 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1220386/1807941 [13:52<05:27, 1795.44 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1220574/1807941 [13:52<05:23, 1815.26 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1220759/1807941 [13:52<05:22, 1821.74 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1220949/1807941 [13:52<05:18, 1840.46 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1221203/1807941 [13:52<05:29, 1781.73 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1221393/1807941 [13:52<05:23, 1810.90 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1221589/1807941 [13:52<05:16, 1850.41 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1221781/1807941 [13:53<05:13, 1869.34 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1221980/1807941 [13:53<05:08, 1901.23 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1222236/1807941 [13:53<05:21, 1821.75 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1222423/1807941 [13:53<05:19, 1832.74 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1222611/1807941 [13:53<05:17, 1842.71 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1222798/1807941 [13:53<05:17, 1845.16 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1223052/1807941 [13:53<05:27, 1785.94 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1223237/1807941 [13:53<05:24, 1802.38 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1223422/1807941 [13:53<05:22, 1813.71 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1223610/1807941 [13:54<05:19, 1830.17 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1223796/1807941 [13:54<05:18, 1836.24 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1224027/1807941 [13:54<05:39, 1721.08 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1224270/1807941 [13:54<05:47, 1680.55 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1224494/1807941 [13:54<06:01, 1613.21 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1224745/1807941 [13:54<05:19, 1825.80 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1224976/1807941 [13:54<05:38, 1723.90 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1225216/1807941 [13:55<05:46, 1680.83 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1225410/1807941 [13:55<05:35, 1737.65 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1225608/1807941 [13:55<05:23, 1797.49 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1225868/1807941 [13:55<05:28, 1772.10 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1226100/1807941 [13:55<05:43, 1692.83 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1226350/1807941 [13:55<05:48, 1670.90 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1226567/1807941 [13:55<06:48, 1424.32 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1226770/1807941 [13:56<06:55, 1400.37 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1226938/1807941 [13:56<07:34, 1278.14 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1227120/1807941 [13:56<08:28, 1141.39 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1227290/1807941 [13:56<08:31, 1136.18 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1227449/1807941 [13:56<08:41, 1112.24 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1227572/1807941 [13:56<08:30, 1136.06 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1227746/1807941 [13:56<08:33, 1129.92 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1227887/1807941 [13:57<08:06, 1191.95 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1228074/1807941 [13:57<08:13, 1176.17 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1228246/1807941 [13:57<12:24, 778.58 examples/s] 
Tokenizing train dataset:  68%|██████▊   | 1228344/1807941 [13:57<14:13, 678.92 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1228479/1807941 [13:57<12:15, 788.21 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1228597/1807941 [13:58<11:13, 859.89 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1228748/1807941 [13:58<10:45, 896.67 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1228855/1807941 [13:58<11:42, 824.24 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1228974/1807941 [13:58<14:43, 654.98 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229051/1807941 [13:59<24:01, 401.49 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229122/1807941 [13:59<23:44, 406.21 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229187/1807941 [14:00<47:04, 204.93 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229246/1807941 [14:00<1:03:30, 151.86 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229278/1807941 [14:01<1:04:11, 150.24 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229333/1807941 [14:01<52:05, 185.10 examples/s]  
Tokenizing train dataset:  68%|██████▊   | 1229371/1807941 [14:01<54:04, 178.33 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229404/1807941 [14:01<50:19, 191.63 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229436/1807941 [14:01<50:39, 190.32 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229467/1807941 [14:01<51:05, 188.70 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229502/1807941 [14:02<1:03:08, 152.69 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229538/1807941 [14:02<54:14, 177.73 examples/s]  
Tokenizing train dataset:  68%|██████▊   | 1229581/1807941 [14:02<49:59, 192.83 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229616/1807941 [14:02<59:24, 162.24 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229646/1807941 [14:03<1:06:20, 145.28 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229692/1807941 [14:03<52:07, 184.87 examples/s]  
Tokenizing train dataset:  68%|██████▊   | 1229726/1807941 [14:03<47:24, 203.25 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229766/1807941 [14:03<50:23, 191.25 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229802/1807941 [14:03<47:52, 201.30 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229831/1807941 [14:04<54:39, 176.27 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229877/1807941 [14:04<1:04:16, 149.91 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229925/1807941 [14:04<1:05:24, 147.30 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229955/1807941 [14:04<1:00:05, 160.30 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1229995/1807941 [14:05<50:52, 189.32 examples/s]  
Tokenizing train dataset:  68%|██████▊   | 1230028/1807941 [14:05<1:21:24, 118.32 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230047/1807941 [14:05<1:27:11, 110.47 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230118/1807941 [14:05<51:07, 188.39 examples/s]  
Tokenizing train dataset:  68%|██████▊   | 1230177/1807941 [14:06<1:00:16, 159.78 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230203/1807941 [14:06<1:16:41, 125.55 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230229/1807941 [14:07<1:36:34, 99.70 examples/s] 
Tokenizing train dataset:  68%|██████▊   | 1230276/1807941 [14:07<1:11:08, 135.34 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230314/1807941 [14:07<58:46, 163.78 examples/s]  
Tokenizing train dataset:  68%|██████▊   | 1230358/1807941 [14:07<48:33, 198.27 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230392/1807941 [14:07<46:15, 208.09 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230427/1807941 [14:07<49:41, 193.69 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230481/1807941 [14:08<56:19, 170.86 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230519/1807941 [14:08<49:00, 196.35 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230556/1807941 [14:08<53:12, 180.87 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230597/1807941 [14:09<1:01:46, 155.78 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230661/1807941 [14:09<43:07, 223.07 examples/s]  
Tokenizing train dataset:  68%|██████▊   | 1230704/1807941 [14:09<39:05, 246.11 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230759/1807941 [14:09<32:21, 297.27 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230839/1807941 [14:09<24:51, 386.84 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230930/1807941 [14:09<20:33, 467.78 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1230987/1807941 [14:09<26:00, 369.65 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231036/1807941 [14:10<34:29, 278.74 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231074/1807941 [14:10<43:49, 219.39 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231166/1807941 [14:10<29:55, 321.24 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231228/1807941 [14:10<36:11, 265.59 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231280/1807941 [14:11<43:26, 221.23 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231346/1807941 [14:11<34:33, 278.04 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231426/1807941 [14:11<27:02, 355.35 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231483/1807941 [14:11<24:42, 388.84 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231542/1807941 [14:11<23:04, 416.47 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231608/1807941 [14:11<21:01, 457.00 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231703/1807941 [14:12<21:30, 446.35 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231799/1807941 [14:12<21:40, 442.89 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231869/1807941 [14:12<37:05, 258.88 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1231921/1807941 [14:12<33:52, 283.35 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232000/1807941 [14:13<28:51, 332.58 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232072/1807941 [14:13<24:17, 395.17 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232164/1807941 [14:13<23:08, 414.67 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232226/1807941 [14:13<35:31, 270.13 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232273/1807941 [14:14<34:43, 276.34 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232314/1807941 [14:14<32:22, 296.34 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232375/1807941 [14:14<31:10, 307.67 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232425/1807941 [14:14<34:51, 275.18 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232479/1807941 [14:14<30:00, 319.60 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232542/1807941 [14:14<30:27, 314.85 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232598/1807941 [14:15<38:52, 246.71 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232658/1807941 [14:15<33:59, 282.04 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232705/1807941 [14:15<30:44, 311.88 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232782/1807941 [14:15<23:53, 401.30 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232854/1807941 [14:15<20:52, 459.28 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1232958/1807941 [14:15<19:24, 493.57 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233050/1807941 [14:15<17:56, 533.80 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233137/1807941 [14:16<20:02, 478.01 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233211/1807941 [14:16<21:09, 452.73 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233275/1807941 [14:16<24:02, 398.42 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233326/1807941 [14:16<22:57, 417.26 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233395/1807941 [14:16<23:43, 403.66 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233454/1807941 [14:17<26:36, 359.79 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233496/1807941 [14:17<26:45, 357.79 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233580/1807941 [14:17<22:41, 421.89 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233670/1807941 [14:17<20:05, 476.21 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233720/1807941 [14:17<21:28, 445.81 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233799/1807941 [14:17<18:38, 513.15 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233893/1807941 [14:17<17:29, 546.94 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1233963/1807941 [14:18<20:40, 462.64 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234050/1807941 [14:18<20:53, 457.83 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234144/1807941 [14:18<20:19, 470.66 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234198/1807941 [14:18<22:08, 431.86 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234250/1807941 [14:18<26:49, 356.49 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234315/1807941 [14:19<23:24, 408.30 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234382/1807941 [14:19<23:48, 401.47 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234432/1807941 [14:19<22:44, 420.35 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234524/1807941 [14:19<19:53, 480.44 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234591/1807941 [14:19<22:14, 429.64 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234664/1807941 [14:19<19:28, 490.54 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234748/1807941 [14:19<20:32, 465.01 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234861/1807941 [14:20<16:05, 593.55 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1234946/1807941 [14:20<24:48, 384.94 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1235000/1807941 [14:20<27:10, 351.40 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1235066/1807941 [14:20<27:26, 347.97 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1235140/1807941 [14:20<23:20, 408.99 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1235210/1807941 [14:21<26:33, 359.38 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1235285/1807941 [14:21<22:20, 427.19 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1235355/1807941 [14:21<26:07, 365.40 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1235717/1807941 [14:21<10:05, 944.87 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1236000/1807941 [14:21<07:13, 1318.20 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1236378/1807941 [14:21<05:06, 1867.44 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1236750/1807941 [14:21<04:06, 2313.89 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1237102/1807941 [14:22<03:37, 2621.23 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1237478/1807941 [14:22<03:15, 2923.44 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1237845/1807941 [14:22<03:02, 3127.52 examples/s]
Tokenizing train dataset:  68%|██████▊   | 1238184/1807941 [14:22<02:58, 3196.90 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1238554/1807941 [14:22<02:50, 3339.81 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1239021/1807941 [14:22<02:55, 3248.90 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1239479/1807941 [14:22<02:59, 3175.64 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1239817/1807941 [14:22<02:56, 3225.20 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1240319/1807941 [14:23<02:53, 3266.30 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1240789/1807941 [14:23<04:54, 1923.05 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1241066/1807941 [14:23<04:35, 2056.01 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1241339/1807941 [14:23<06:35, 1433.06 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1241589/1807941 [14:24<06:27, 1462.02 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1241832/1807941 [14:24<05:50, 1617.38 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1242125/1807941 [14:24<05:32, 1701.86 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1242358/1807941 [14:24<05:09, 1824.97 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1242571/1807941 [14:24<04:59, 1890.22 examples/s]
Tokenizing train dataset:  69%|██████▊   | 1242804/1807941 [14:24<04:43, 1994.68 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1243111/1807941 [14:24<04:41, 2009.16 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1243405/1807941 [14:25<05:14, 1792.95 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1243655/1807941 [14:25<05:22, 1751.60 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1243898/1807941 [14:25<04:57, 1898.42 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1244115/1807941 [14:25<04:53, 1923.43 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1244333/1807941 [14:25<04:44, 1981.59 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1244572/1807941 [14:25<05:05, 1843.82 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1244847/1807941 [14:25<05:06, 1839.02 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1245104/1807941 [14:25<05:13, 1793.56 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1245300/1807941 [14:26<05:45, 1629.17 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1245521/1807941 [14:26<05:57, 1574.04 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1245709/1807941 [14:26<05:42, 1640.87 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1245883/1807941 [14:26<05:38, 1661.06 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1246103/1807941 [14:26<05:52, 1592.96 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1246350/1807941 [14:26<05:50, 1604.07 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1246536/1807941 [14:26<05:38, 1660.46 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1246717/1807941 [14:26<05:33, 1680.87 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1246914/1807941 [14:27<06:02, 1547.26 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1247138/1807941 [14:27<06:08, 1521.45 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1247343/1807941 [14:27<05:40, 1644.64 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1247607/1807941 [14:27<05:33, 1679.82 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1247868/1807941 [14:27<05:31, 1687.64 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1248059/1807941 [14:27<05:22, 1737.46 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1248253/1807941 [14:27<05:13, 1785.61 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1248472/1807941 [14:27<04:56, 1889.17 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1248665/1807941 [14:28<04:54, 1898.26 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1248890/1807941 [14:28<04:40, 1993.75 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1249121/1807941 [14:28<04:34, 2037.27 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1249378/1807941 [14:28<04:15, 2185.02 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1249628/1807941 [14:28<04:05, 2271.24 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1249879/1807941 [14:28<03:58, 2339.17 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1250119/1807941 [14:28<04:05, 2276.48 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1250361/1807941 [14:28<04:00, 2317.09 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1250611/1807941 [14:28<03:55, 2367.92 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1250867/1807941 [14:28<03:50, 2418.90 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1251193/1807941 [14:29<04:00, 2319.52 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1251449/1807941 [14:29<03:53, 2381.89 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1251693/1807941 [14:29<03:52, 2395.51 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1251943/1807941 [14:29<03:49, 2422.18 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1252289/1807941 [14:29<03:54, 2373.88 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1252532/1807941 [14:29<03:52, 2387.41 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1252787/1807941 [14:29<03:48, 2429.09 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1253145/1807941 [14:29<03:50, 2409.58 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1253401/1807941 [14:30<03:46, 2447.17 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1253668/1807941 [14:30<03:41, 2505.48 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1253972/1807941 [14:30<03:58, 2326.42 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1254281/1807941 [14:30<04:37, 1992.62 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1254524/1807941 [14:30<04:55, 1873.09 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1254765/1807941 [14:30<05:09, 1787.95 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1254983/1807941 [14:30<05:29, 1676.33 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1255182/1807941 [14:31<05:53, 1563.50 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1255417/1807941 [14:31<05:53, 1563.75 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1255580/1807941 [14:31<05:50, 1574.94 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1255776/1807941 [14:31<05:31, 1666.18 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1255976/1807941 [14:31<05:16, 1745.97 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1256217/1807941 [14:31<05:25, 1694.15 examples/s]
Tokenizing train dataset:  69%|██████▉   | 1256399/1807941 [14:31<05:21, 1716.99 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1256589/1807941 [14:31<05:12, 1762.07 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1256831/1807941 [14:32<05:23, 1704.69 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1257073/1807941 [14:32<05:37, 1633.41 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1257312/1807941 [14:32<05:41, 1613.64 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1257505/1807941 [14:32<06:06, 1502.58 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1257709/1807941 [14:32<06:19, 1450.54 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1257868/1807941 [14:32<06:12, 1477.26 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1258094/1807941 [14:32<06:10, 1484.74 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1258312/1807941 [14:33<06:13, 1471.13 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1258469/1807941 [14:33<06:08, 1491.47 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1258625/1807941 [14:33<06:04, 1507.66 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1258780/1807941 [14:33<06:02, 1516.13 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1258984/1807941 [14:33<06:17, 1452.79 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1259211/1807941 [14:33<06:12, 1471.62 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1259361/1807941 [14:33<06:12, 1474.45 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1259552/1807941 [14:33<06:30, 1403.63 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1259739/1807941 [14:34<06:47, 1345.55 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1259931/1807941 [14:34<06:56, 1315.79 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1260072/1807941 [14:34<07:42, 1185.58 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1260222/1807941 [14:34<08:05, 1127.37 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1260402/1807941 [14:34<07:56, 1148.05 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1260560/1807941 [14:34<07:20, 1243.73 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1260753/1807941 [14:34<07:16, 1253.54 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1260912/1807941 [14:35<06:52, 1326.38 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1261096/1807941 [14:35<07:04, 1288.63 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1261247/1807941 [14:35<06:48, 1338.06 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1261413/1807941 [14:35<06:25, 1418.69 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1261578/1807941 [14:35<07:00, 1299.48 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1261788/1807941 [14:35<06:52, 1322.49 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1261926/1807941 [14:35<06:49, 1334.55 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1262110/1807941 [14:36<07:49, 1162.27 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1262276/1807941 [14:36<07:57, 1142.55 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1262399/1807941 [14:36<07:50, 1160.42 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1262525/1807941 [14:36<07:42, 1179.32 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1262704/1807941 [14:36<07:41, 1182.19 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1262853/1807941 [14:36<08:08, 1116.17 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1262973/1807941 [14:36<08:00, 1133.15 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1263096/1807941 [14:36<08:53, 1021.66 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1263251/1807941 [14:37<09:46, 928.81 examples/s] 
Tokenizing train dataset:  70%|██████▉   | 1263396/1807941 [14:37<09:49, 923.04 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1263551/1807941 [14:37<08:36, 1053.18 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1263685/1807941 [14:37<08:06, 1118.08 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1263810/1807941 [14:37<07:53, 1149.37 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1263935/1807941 [14:37<07:42, 1175.66 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1264086/1807941 [14:37<08:09, 1111.69 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1264207/1807941 [14:37<07:58, 1136.01 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1264368/1807941 [14:38<08:09, 1110.57 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1264530/1807941 [14:38<08:15, 1095.62 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1264659/1807941 [14:38<07:56, 1139.29 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1264815/1807941 [14:38<08:11, 1104.45 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1264976/1807941 [14:38<08:17, 1092.12 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1265089/1807941 [14:38<09:23, 963.95 examples/s] 
Tokenizing train dataset:  70%|██████▉   | 1265222/1807941 [14:38<09:39, 935.91 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1265366/1807941 [14:39<10:20, 874.34 examples/s]
Tokenizing train dataset:  70%|██████▉   | 1265471/1807941 [14:39<09:56, 908.92 examples/s]
Tokenizing train dataset:  70%|███████   | 1265620/1807941 [14:39<09:41, 932.84 examples/s]
Tokenizing train dataset:  70%|███████   | 1265769/1807941 [14:39<09:30, 949.64 examples/s]
Tokenizing train dataset:  70%|███████   | 1265891/1807941 [14:39<10:03, 897.73 examples/s]
Tokenizing train dataset:  70%|███████   | 1265991/1807941 [14:39<09:58, 906.17 examples/s]
Tokenizing train dataset:  70%|███████   | 1266085/1807941 [14:39<11:31, 783.50 examples/s]
Tokenizing train dataset:  70%|███████   | 1266190/1807941 [14:40<11:57, 755.58 examples/s]
Tokenizing train dataset:  70%|███████   | 1266290/1807941 [14:40<11:11, 806.20 examples/s]
Tokenizing train dataset:  70%|███████   | 1266421/1807941 [14:40<10:58, 821.98 examples/s]
Tokenizing train dataset:  70%|███████   | 1266514/1807941 [14:40<10:41, 844.49 examples/s]
Tokenizing train dataset:  70%|███████   | 1266637/1807941 [14:40<10:50, 832.52 examples/s]
Tokenizing train dataset:  70%|███████   | 1266746/1807941 [14:40<11:23, 791.44 examples/s]
Tokenizing train dataset:  70%|███████   | 1266848/1807941 [14:40<11:57, 754.41 examples/s]
Tokenizing train dataset:  70%|███████   | 1266954/1807941 [14:41<12:14, 736.65 examples/s]
Tokenizing train dataset:  70%|███████   | 1267049/1807941 [14:41<12:54, 698.33 examples/s]
Tokenizing train dataset:  70%|███████   | 1267154/1807941 [14:41<13:02, 690.81 examples/s]
Tokenizing train dataset:  70%|███████   | 1267263/1807941 [14:41<12:56, 696.50 examples/s]
Tokenizing train dataset:  70%|███████   | 1267337/1807941 [14:41<12:53, 698.63 examples/s]
Tokenizing train dataset:  70%|███████   | 1267412/1807941 [14:41<12:41, 709.75 examples/s]
Tokenizing train dataset:  70%|███████   | 1267484/1807941 [14:41<12:47, 704.00 examples/s]
Tokenizing train dataset:  70%|███████   | 1267567/1807941 [14:41<12:17, 732.33 examples/s]
Tokenizing train dataset:  70%|███████   | 1267678/1807941 [14:42<12:25, 725.10 examples/s]
Tokenizing train dataset:  70%|███████   | 1267755/1807941 [14:42<12:13, 736.03 examples/s]
Tokenizing train dataset:  70%|███████   | 1267852/1807941 [14:42<13:04, 688.45 examples/s]
Tokenizing train dataset:  70%|███████   | 1267941/1807941 [14:42<13:58, 643.96 examples/s]
Tokenizing train dataset:  70%|███████   | 1268040/1807941 [14:42<14:28, 621.88 examples/s]
Tokenizing train dataset:  70%|███████   | 1268105/1807941 [14:42<14:21, 626.36 examples/s]
Tokenizing train dataset:  70%|███████   | 1268176/1807941 [14:42<14:01, 641.38 examples/s]
Tokenizing train dataset:  70%|███████   | 1268269/1807941 [14:43<14:16, 630.13 examples/s]
Tokenizing train dataset:  70%|███████   | 1268353/1807941 [14:43<14:57, 601.49 examples/s]
Tokenizing train dataset:  70%|███████   | 1268423/1807941 [14:43<14:27, 621.78 examples/s]
Tokenizing train dataset:  70%|███████   | 1268517/1807941 [14:43<14:33, 617.60 examples/s]
Tokenizing train dataset:  70%|███████   | 1268594/1807941 [14:43<13:47, 651.88 examples/s]
Tokenizing train dataset:  70%|███████   | 1268678/1807941 [14:43<14:38, 614.07 examples/s]
Tokenizing train dataset:  70%|███████   | 1268772/1807941 [14:43<14:39, 613.35 examples/s]
Tokenizing train dataset:  70%|███████   | 1268851/1807941 [14:44<15:21, 585.01 examples/s]
Tokenizing train dataset:  70%|███████   | 1268947/1807941 [14:44<15:05, 595.14 examples/s]
Tokenizing train dataset:  70%|███████   | 1269022/1807941 [14:44<16:39, 539.30 examples/s]
Tokenizing train dataset:  70%|███████   | 1269098/1807941 [14:44<15:21, 584.59 examples/s]
Tokenizing train dataset:  70%|███████   | 1269204/1807941 [14:44<14:31, 618.17 examples/s]
Tokenizing train dataset:  70%|███████   | 1269280/1807941 [14:44<13:58, 642.77 examples/s]
Tokenizing train dataset:  70%|███████   | 1269375/1807941 [14:44<14:16, 628.47 examples/s]
Tokenizing train dataset:  70%|███████   | 1269440/1807941 [14:45<14:16, 628.79 examples/s]
Tokenizing train dataset:  70%|███████   | 1269521/1807941 [14:45<15:00, 597.73 examples/s]
Tokenizing train dataset:  70%|███████   | 1269584/1807941 [14:45<14:54, 602.11 examples/s]
Tokenizing train dataset:  70%|███████   | 1269660/1807941 [14:45<15:49, 567.21 examples/s]
Tokenizing train dataset:  70%|███████   | 1269725/1807941 [14:45<15:18, 585.92 examples/s]
Tokenizing train dataset:  70%|███████   | 1269790/1807941 [14:45<14:54, 601.30 examples/s]
Tokenizing train dataset:  70%|███████   | 1269860/1807941 [14:45<16:26, 545.64 examples/s]
Tokenizing train dataset:  70%|███████   | 1269921/1807941 [14:45<16:05, 557.29 examples/s]
Tokenizing train dataset:  70%|███████   | 1269979/1807941 [14:45<16:04, 557.61 examples/s]
Tokenizing train dataset:  70%|███████   | 1270056/1807941 [14:46<17:45, 504.67 examples/s]
Tokenizing train dataset:  70%|███████   | 1270135/1807941 [14:46<17:46, 504.26 examples/s]
Tokenizing train dataset:  70%|███████   | 1270188/1807941 [14:46<17:39, 507.34 examples/s]
Tokenizing train dataset:  70%|███████   | 1270250/1807941 [14:46<16:47, 533.95 examples/s]
Tokenizing train dataset:  70%|███████   | 1270310/1807941 [14:46<16:18, 549.35 examples/s]
Tokenizing train dataset:  70%|███████   | 1270399/1807941 [14:46<16:01, 558.83 examples/s]
Tokenizing train dataset:  70%|███████   | 1270459/1807941 [14:46<15:54, 563.36 examples/s]
Tokenizing train dataset:  70%|███████   | 1270524/1807941 [14:46<15:43, 569.83 examples/s]
Tokenizing train dataset:  70%|███████   | 1270583/1807941 [14:47<15:47, 567.12 examples/s]
Tokenizing train dataset:  70%|███████   | 1270643/1807941 [14:47<15:37, 572.84 examples/s]
Tokenizing train dataset:  70%|███████   | 1270732/1807941 [14:47<15:29, 577.83 examples/s]
Tokenizing train dataset:  70%|███████   | 1271000/1807941 [14:47<08:27, 1059.02 examples/s]
Tokenizing train dataset:  70%|███████   | 1271320/1807941 [14:47<05:32, 1611.53 examples/s]
Tokenizing train dataset:  70%|███████   | 1271638/1807941 [14:47<04:24, 2030.47 examples/s]
Tokenizing train dataset:  70%|███████   | 1271947/1807941 [14:47<03:50, 2324.54 examples/s]
Tokenizing train dataset:  70%|███████   | 1272334/1807941 [14:47<03:41, 2416.68 examples/s]
Tokenizing train dataset:  70%|███████   | 1272655/1807941 [14:48<03:24, 2620.84 examples/s]
Tokenizing train dataset:  70%|███████   | 1272952/1807941 [14:48<03:17, 2711.34 examples/s]
Tokenizing train dataset:  70%|███████   | 1273345/1807941 [14:48<03:20, 2672.44 examples/s]
Tokenizing train dataset:  70%|███████   | 1273643/1807941 [14:48<03:14, 2750.20 examples/s]
Tokenizing train dataset:  70%|███████   | 1273963/1807941 [14:48<03:06, 2866.99 examples/s]
Tokenizing train dataset:  70%|███████   | 1274372/1807941 [14:48<03:09, 2814.01 examples/s]
Tokenizing train dataset:  71%|███████   | 1274710/1807941 [14:48<03:00, 2959.22 examples/s]
Tokenizing train dataset:  71%|███████   | 1275175/1807941 [14:48<02:57, 3005.50 examples/s]
Tokenizing train dataset:  71%|███████   | 1275512/1807941 [14:48<02:52, 3092.92 examples/s]
Tokenizing train dataset:  71%|███████   | 1275842/1807941 [14:49<02:49, 3145.18 examples/s]
Tokenizing train dataset:  71%|███████   | 1276171/1807941 [14:49<02:48, 3160.17 examples/s]
Tokenizing train dataset:  71%|███████   | 1276523/1807941 [14:49<02:43, 3260.23 examples/s]
Tokenizing train dataset:  71%|███████   | 1276885/1807941 [14:49<02:38, 3360.39 examples/s]
Tokenizing train dataset:  71%|███████   | 1277408/1807941 [14:49<02:35, 3403.63 examples/s]
Tokenizing train dataset:  71%|███████   | 1277774/1807941 [14:49<02:32, 3469.50 examples/s]
Tokenizing train dataset:  71%|███████   | 1278151/1807941 [14:49<02:29, 3549.45 examples/s]
Tokenizing train dataset:  71%|███████   | 1278547/1807941 [14:49<02:24, 3663.36 examples/s]
Tokenizing train dataset:  71%|███████   | 1278950/1807941 [14:49<02:20, 3766.68 examples/s]
Tokenizing train dataset:  71%|███████   | 1279329/1807941 [14:50<02:20, 3769.77 examples/s]
Tokenizing train dataset:  71%|███████   | 1279712/1807941 [14:50<02:19, 3786.55 examples/s]
Tokenizing train dataset:  71%|███████   | 1280276/1807941 [14:50<02:19, 3772.92 examples/s]
Tokenizing train dataset:  71%|███████   | 1280820/1807941 [14:50<02:21, 3717.50 examples/s]
Tokenizing train dataset:  71%|███████   | 1281204/1807941 [14:50<02:20, 3739.75 examples/s]
Tokenizing train dataset:  71%|███████   | 1281616/1807941 [14:50<02:17, 3838.31 examples/s]
Tokenizing train dataset:  71%|███████   | 1282186/1807941 [14:50<02:18, 3801.92 examples/s]
Tokenizing train dataset:  71%|███████   | 1282749/1807941 [14:50<02:18, 3782.16 examples/s]
Tokenizing train dataset:  71%|███████   | 1283273/1807941 [14:51<02:22, 3684.26 examples/s]
Tokenizing train dataset:  71%|███████   | 1283647/1807941 [14:51<02:21, 3695.34 examples/s]
Tokenizing train dataset:  71%|███████   | 1284191/1807941 [14:51<02:22, 3662.87 examples/s]
Tokenizing train dataset:  71%|███████   | 1284569/1807941 [14:51<02:21, 3689.01 examples/s]
Tokenizing train dataset:  71%|███████   | 1284941/1807941 [14:51<02:21, 3693.43 examples/s]
Tokenizing train dataset:  71%|███████   | 1285487/1807941 [14:51<02:22, 3671.91 examples/s]
Tokenizing train dataset:  71%|███████   | 1285874/1807941 [14:51<02:20, 3717.94 examples/s]
Tokenizing train dataset:  71%|███████   | 1286408/1807941 [14:51<02:22, 3659.48 examples/s]
Tokenizing train dataset:  71%|███████   | 1286788/1807941 [14:52<02:21, 3692.20 examples/s]
Tokenizing train dataset:  71%|███████   | 1287320/1807941 [14:52<02:23, 3636.63 examples/s]
Tokenizing train dataset:  71%|███████   | 1287693/1807941 [14:52<02:22, 3658.82 examples/s]
Tokenizing train dataset:  71%|███████▏  | 1288259/1807941 [14:52<02:20, 3693.90 examples/s]
Tokenizing train dataset:  71%|███████▏  | 1288668/1807941 [14:52<02:17, 3789.89 examples/s]
Tokenizing train dataset:  71%|███████▏  | 1289227/1807941 [14:52<02:17, 3765.01 examples/s]
Tokenizing train dataset:  71%|███████▏  | 1289630/1807941 [14:52<02:15, 3828.77 examples/s]
Tokenizing train dataset:  71%|███████▏  | 1290209/1807941 [14:52<02:15, 3834.22 examples/s]
Tokenizing train dataset:  71%|███████▏  | 1290719/1807941 [14:53<02:20, 3686.08 examples/s]
Tokenizing train dataset:  71%|███████▏  | 1291191/1807941 [14:53<02:27, 3510.26 examples/s]
Tokenizing train dataset:  71%|███████▏  | 1291678/1807941 [14:53<02:30, 3422.71 examples/s]
Tokenizing train dataset:  71%|███████▏  | 1292129/1807941 [14:53<02:36, 3292.13 examples/s]
Tokenizing train dataset:  71%|███████▏  | 1292615/1807941 [14:53<02:37, 3273.26 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1293049/1807941 [14:53<02:43, 3153.42 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1293374/1807941 [14:53<02:42, 3173.04 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1293705/1807941 [14:54<02:40, 3202.94 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1294159/1807941 [14:54<02:43, 3134.72 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1294479/1807941 [14:54<02:43, 3144.52 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1294811/1807941 [14:54<02:40, 3189.14 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1295133/1807941 [14:54<02:40, 3194.99 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1295479/1807941 [14:54<02:36, 3266.17 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1295830/1807941 [14:54<02:33, 3333.77 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1296342/1807941 [14:54<02:33, 3325.44 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1296701/1807941 [14:54<02:30, 3390.80 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1297180/1807941 [14:55<02:34, 3313.17 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1297662/1807941 [14:55<02:35, 3277.17 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1298119/1807941 [14:55<02:39, 3200.28 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1298597/1807941 [14:55<02:39, 3191.70 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1298922/1807941 [14:55<02:38, 3203.97 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1299378/1807941 [14:55<02:41, 3145.65 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1299702/1807941 [14:55<02:40, 3166.47 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1300165/1807941 [14:56<02:43, 3103.15 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1300484/1807941 [14:56<02:42, 3122.25 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1300960/1807941 [14:56<02:41, 3135.20 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1301397/1807941 [14:56<02:45, 3061.18 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1301728/1807941 [14:56<02:42, 3120.12 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1302165/1807941 [14:56<02:46, 3045.81 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1302480/1807941 [14:56<02:44, 3067.90 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1302829/1807941 [14:56<02:39, 3174.50 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1303165/1807941 [14:57<02:40, 3148.58 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1303496/1807941 [14:57<02:38, 3190.23 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1303844/1807941 [14:57<02:34, 3268.50 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1304321/1807941 [14:57<02:38, 3179.97 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1304660/1807941 [14:57<02:35, 3233.37 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1305119/1807941 [14:57<02:38, 3168.83 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1305440/1807941 [14:57<02:38, 3177.50 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1305911/1807941 [14:57<02:38, 3158.82 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1306333/1807941 [14:58<02:45, 3037.55 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1306640/1807941 [14:58<02:44, 3040.57 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1306951/1807941 [14:58<02:43, 3055.14 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1307389/1807941 [14:58<02:46, 3004.10 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1307705/1807941 [14:58<02:44, 3032.70 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1308153/1807941 [14:58<02:45, 3013.97 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1308461/1807941 [14:58<02:44, 3029.32 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1308782/1807941 [14:58<02:42, 3076.61 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1309245/1807941 [14:58<02:42, 3077.10 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1309562/1807941 [14:59<02:40, 3099.26 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1310000/1807941 [14:59<02:44, 3030.52 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1310322/1807941 [14:59<02:41, 3076.95 examples/s]
Tokenizing train dataset:  72%|███████▏  | 1310641/1807941 [14:59<02:40, 3104.96 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1310965/1807941 [14:59<02:38, 3138.96 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1311392/1807941 [14:59<02:43, 3028.77 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1311715/1807941 [14:59<02:41, 3078.66 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1312171/1807941 [14:59<02:41, 3062.52 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1312501/1807941 [15:00<02:38, 3121.42 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1312839/1807941 [15:00<02:35, 3189.01 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1313325/1807941 [15:00<02:37, 3149.20 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1313651/1807941 [15:00<02:35, 3174.12 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1313992/1807941 [15:00<02:32, 3233.60 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1314443/1807941 [15:00<02:36, 3149.16 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1314773/1807941 [15:00<02:34, 3185.30 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1315222/1807941 [15:00<02:38, 3115.30 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1315694/1807941 [15:01<02:37, 3121.12 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1316158/1807941 [15:01<02:39, 3076.12 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1316489/1807941 [15:01<02:37, 3128.28 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1316814/1807941 [15:01<02:35, 3156.07 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1317272/1807941 [15:01<02:37, 3115.16 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1317596/1807941 [15:01<02:36, 3141.88 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1317927/1807941 [15:01<02:33, 3183.35 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1318381/1807941 [15:01<02:36, 3123.72 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1318710/1807941 [15:02<02:34, 3164.12 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1319185/1807941 [15:02<02:34, 3159.21 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1319514/1807941 [15:02<02:33, 3188.68 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1319856/1807941 [15:02<02:30, 3243.57 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1320351/1807941 [15:02<02:29, 3262.06 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1320702/1807941 [15:02<02:26, 3323.14 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1321160/1807941 [15:02<02:31, 3221.07 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1321646/1807941 [15:02<02:30, 3223.39 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1321977/1807941 [15:03<02:29, 3242.18 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1322440/1807941 [15:03<02:32, 3184.29 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1322917/1807941 [15:03<02:32, 3180.96 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1323385/1807941 [15:03<02:33, 3157.99 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1323702/1807941 [15:03<02:33, 3157.70 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1324156/1807941 [15:03<02:35, 3106.07 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1324480/1807941 [15:03<02:34, 3135.52 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1324803/1807941 [15:03<02:32, 3157.82 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1325285/1807941 [15:04<02:32, 3174.93 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1325611/1807941 [15:04<02:30, 3194.84 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1325937/1807941 [15:04<02:30, 3209.12 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1326397/1807941 [15:04<02:32, 3152.39 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1326733/1807941 [15:04<02:30, 3203.28 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1327220/1807941 [15:04<02:29, 3216.01 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1327550/1807941 [15:04<02:28, 3235.43 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1327878/1807941 [15:04<02:27, 3244.72 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1328326/1807941 [15:05<02:32, 3148.36 examples/s]
Tokenizing train dataset:  73%|███████▎  | 1328649/1807941 [15:05<02:31, 3165.43 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1329090/1807941 [15:05<02:35, 3083.68 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1329413/1807941 [15:05<02:33, 3115.89 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1329741/1807941 [15:05<02:31, 3156.60 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1330209/1807941 [15:05<02:32, 3132.31 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1330527/1807941 [15:05<02:31, 3142.13 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1331000/1807941 [15:05<02:34, 3079.87 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1331321/1807941 [15:05<02:33, 3109.94 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1331644/1807941 [15:06<02:31, 3137.99 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1331961/1807941 [15:06<02:31, 3144.34 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1332411/1807941 [15:06<02:34, 3086.87 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1332726/1807941 [15:06<02:33, 3101.32 examples/s]
Tokenizing train dataset:  74%|███████▎  | 1333172/1807941 [15:06<02:35, 3054.74 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1333483/1807941 [15:06<02:34, 3066.24 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1333946/1807941 [15:06<02:34, 3069.88 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1334373/1807941 [15:06<02:38, 2992.65 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1334681/1807941 [15:07<02:37, 3011.93 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1334991/1807941 [15:07<02:36, 3029.75 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1335434/1807941 [15:07<02:37, 2995.05 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1335755/1807941 [15:07<02:35, 3045.51 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1336200/1807941 [15:07<02:36, 3015.09 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1336518/1807941 [15:07<02:34, 3049.93 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1336826/1807941 [15:07<02:34, 3056.07 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1337277/1807941 [15:07<02:35, 3034.31 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1337602/1807941 [15:08<02:32, 3086.03 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1338035/1807941 [15:08<02:36, 3011.52 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1338348/1807941 [15:08<02:34, 3038.99 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1338665/1807941 [15:08<02:32, 3070.28 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1338978/1807941 [15:08<02:32, 3083.95 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1339410/1807941 [15:08<02:36, 3003.18 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1339728/1807941 [15:08<02:33, 3048.30 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1340179/1807941 [15:08<02:34, 3028.45 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1340505/1807941 [15:08<02:31, 3085.60 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1340824/1807941 [15:09<02:30, 3109.39 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1341282/1807941 [15:09<02:31, 3085.81 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1341593/1807941 [15:09<02:30, 3089.83 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1341928/1807941 [15:09<02:27, 3158.04 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1342365/1807941 [15:09<02:31, 3065.63 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1342693/1807941 [15:09<02:29, 3118.68 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1343168/1807941 [15:09<02:28, 3124.28 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1343499/1807941 [15:09<02:26, 3168.22 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1343834/1807941 [15:10<02:24, 3213.13 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1344174/1807941 [15:10<02:23, 3221.15 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1344509/1807941 [15:10<02:22, 3253.74 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1344985/1807941 [15:10<02:23, 3219.73 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1345429/1807941 [15:10<02:27, 3127.03 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1345907/1807941 [15:10<02:26, 3145.01 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1346348/1807941 [15:10<02:30, 3076.72 examples/s]
Tokenizing train dataset:  74%|███████▍  | 1346675/1807941 [15:10<02:27, 3117.72 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1346989/1807941 [15:11<02:27, 3122.40 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1347429/1807941 [15:11<02:30, 3052.84 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1347754/1807941 [15:11<02:28, 3100.25 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1348199/1807941 [15:11<02:30, 3050.89 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1348519/1807941 [15:11<02:28, 3084.54 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1348845/1807941 [15:11<02:26, 3127.77 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1349161/1807941 [15:11<02:28, 3082.72 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1349479/1807941 [15:11<02:27, 3107.83 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1349794/1807941 [15:11<02:26, 3119.22 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1350250/1807941 [15:12<02:28, 3085.16 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1350566/1807941 [15:12<02:27, 3102.16 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1350887/1807941 [15:12<02:26, 3129.16 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1351320/1807941 [15:12<02:30, 3037.13 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1351629/1807941 [15:12<02:29, 3047.89 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1351948/1807941 [15:12<02:27, 3084.90 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1352388/1807941 [15:12<02:30, 3026.39 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1352709/1807941 [15:12<02:28, 3072.41 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1353162/1807941 [15:13<02:29, 3047.96 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1353625/1807941 [15:13<02:28, 3058.19 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1353947/1807941 [15:13<02:26, 3094.92 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1354402/1807941 [15:13<02:27, 3073.26 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1354711/1807941 [15:13<02:27, 3076.35 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1355155/1807941 [15:13<02:30, 3009.30 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1355477/1807941 [15:13<02:27, 3058.97 examples/s]
Tokenizing train dataset:  75%|███████▍  | 1355790/1807941 [15:13<02:26, 3076.11 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1356250/1807941 [15:14<02:27, 3069.47 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1356562/1807941 [15:14<02:26, 3079.65 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1356883/1807941 [15:14<02:24, 3110.97 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1357337/1807941 [15:14<02:26, 3077.44 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1357665/1807941 [15:14<02:24, 3124.79 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1358000/1807941 [15:14<02:23, 3128.41 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1358329/1807941 [15:14<02:21, 3171.80 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1358671/1807941 [15:14<02:18, 3239.58 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1359000/1807941 [15:14<02:19, 3218.24 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1359330/1807941 [15:15<02:18, 3239.67 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1359658/1807941 [15:15<02:17, 3248.91 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1359999/1807941 [15:15<02:16, 3292.54 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1360488/1807941 [15:15<02:16, 3277.64 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1360824/1807941 [15:15<02:15, 3296.85 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1361169/1807941 [15:15<02:16, 3271.85 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1361525/1807941 [15:15<02:13, 3347.84 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1362000/1807941 [15:15<02:16, 3265.08 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1362490/1807941 [15:16<02:16, 3261.75 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1362977/1807941 [15:16<02:16, 3252.71 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1363424/1807941 [15:16<02:20, 3162.20 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1363747/1807941 [15:16<02:19, 3176.93 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1364191/1807941 [15:16<02:23, 3102.41 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1364514/1807941 [15:16<02:21, 3131.31 examples/s]
Tokenizing train dataset:  75%|███████▌  | 1364839/1807941 [15:16<02:20, 3159.66 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1365281/1807941 [15:16<02:23, 3077.84 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1365599/1807941 [15:17<02:22, 3101.82 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1365921/1807941 [15:17<02:21, 3131.42 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1366395/1807941 [15:17<02:20, 3135.18 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1366722/1807941 [15:17<02:19, 3166.66 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1367041/1807941 [15:17<02:19, 3169.29 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1367368/1807941 [15:17<02:17, 3194.97 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1367696/1807941 [15:17<02:16, 3216.38 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1368169/1807941 [15:17<02:17, 3188.62 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1368635/1807941 [15:17<02:19, 3155.37 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1369083/1807941 [15:18<02:21, 3095.82 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1369559/1807941 [15:18<02:20, 3116.00 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1370000/1807941 [15:18<02:23, 3049.80 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1370460/1807941 [15:18<02:23, 3051.42 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1370770/1807941 [15:18<02:22, 3058.89 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1371210/1807941 [15:18<02:24, 3014.02 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1371514/1807941 [15:18<02:24, 3017.15 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1371835/1807941 [15:19<02:22, 3061.32 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1372273/1807941 [15:19<02:24, 3007.63 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1372584/1807941 [15:19<02:23, 3031.35 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1372900/1807941 [15:19<02:22, 3063.19 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1373342/1807941 [15:19<02:24, 3016.30 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1373663/1807941 [15:19<02:21, 3063.08 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1373992/1807941 [15:19<02:19, 3120.58 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1374460/1807941 [15:19<02:19, 3116.06 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1374806/1807941 [15:19<02:15, 3201.79 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1375285/1807941 [15:20<02:15, 3194.93 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1375637/1807941 [15:20<02:12, 3274.29 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1375988/1807941 [15:20<02:09, 3329.86 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1376479/1807941 [15:20<02:10, 3306.82 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1376824/1807941 [15:20<02:09, 3341.75 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1377169/1807941 [15:20<02:10, 3303.05 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1377519/1807941 [15:20<02:08, 3354.54 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1377870/1807941 [15:20<02:06, 3397.51 examples/s]
Tokenizing train dataset:  76%|███████▌  | 1378334/1807941 [15:21<02:11, 3275.69 examples/s]
Tokenizing train dataset:  76%|███████▋  | 1378692/1807941 [15:21<02:08, 3351.55 examples/s]
Tokenizing train dataset:  76%|███████▋  | 1379183/1807941 [15:21<02:09, 3316.70 examples/s]
Tokenizing train dataset:  76%|███████▋  | 1379524/1807941 [15:21<02:08, 3338.15 examples/s]
Tokenizing train dataset:  76%|███████▋  | 1379880/1807941 [15:21<02:06, 3395.93 examples/s]
Tokenizing train dataset:  76%|███████▋  | 1380387/1807941 [15:21<02:06, 3388.87 examples/s]
Tokenizing train dataset:  76%|███████▋  | 1380737/1807941 [15:21<02:05, 3415.73 examples/s]
Tokenizing train dataset:  76%|███████▋  | 1381243/1807941 [15:21<02:05, 3396.66 examples/s]
Tokenizing train dataset:  76%|███████▋  | 1381611/1807941 [15:21<02:03, 3464.30 examples/s]
Tokenizing train dataset:  76%|███████▋  | 1382092/1807941 [15:22<02:06, 3371.94 examples/s]
Tokenizing train dataset:  76%|███████▋  | 1382436/1807941 [15:22<02:05, 3387.68 examples/s]
Tokenizing train dataset:  76%|███████▋  | 1382904/1807941 [15:22<02:09, 3292.84 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1383353/1807941 [15:22<02:13, 3190.21 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1383815/1807941 [15:22<02:14, 3151.49 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1384263/1807941 [15:22<02:16, 3097.04 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1384583/1807941 [15:22<02:15, 3119.27 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1384915/1807941 [15:23<02:13, 3167.39 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1385348/1807941 [15:23<02:17, 3066.96 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1385671/1807941 [15:23<02:16, 3104.52 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1386000/1807941 [15:23<02:16, 3093.66 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1386320/1807941 [15:23<02:15, 3118.54 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1386640/1807941 [15:23<02:14, 3137.96 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1386957/1807941 [15:23<02:13, 3145.01 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1387406/1807941 [15:23<02:16, 3083.11 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1387717/1807941 [15:23<02:16, 3087.57 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1388172/1807941 [15:24<02:16, 3065.21 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1388491/1807941 [15:24<02:15, 3095.28 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1388953/1807941 [15:24<02:15, 3083.50 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1389395/1807941 [15:24<02:17, 3034.40 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1389719/1807941 [15:24<02:15, 3081.00 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1390166/1807941 [15:24<02:17, 3042.69 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1390494/1807941 [15:24<02:14, 3097.16 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1390817/1807941 [15:24<02:13, 3128.75 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1391267/1807941 [15:25<02:15, 3074.60 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1391596/1807941 [15:25<02:13, 3125.46 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1391917/1807941 [15:25<02:12, 3144.37 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1392391/1807941 [15:25<02:12, 3144.47 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1392714/1807941 [15:25<02:11, 3164.95 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1393168/1807941 [15:25<02:13, 3114.55 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1393483/1807941 [15:25<02:12, 3122.76 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1393801/1807941 [15:25<02:12, 3134.89 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1394258/1807941 [15:26<02:13, 3100.62 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1394581/1807941 [15:26<02:12, 3131.28 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1394899/1807941 [15:26<02:11, 3142.19 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1395348/1807941 [15:26<02:13, 3086.09 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1395676/1807941 [15:26<02:11, 3132.61 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1395994/1807941 [15:26<02:11, 3142.33 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1396415/1807941 [15:26<02:16, 3014.38 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1396878/1807941 [15:26<02:15, 3036.36 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1397299/1807941 [15:27<02:18, 2955.90 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1397622/1807941 [15:27<02:16, 3016.87 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1397966/1807941 [15:27<02:11, 3120.48 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1398447/1807941 [15:27<02:10, 3145.03 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1398783/1807941 [15:27<02:08, 3195.90 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1399252/1807941 [15:27<02:09, 3167.48 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1399729/1807941 [15:27<02:08, 3167.41 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1400156/1807941 [15:27<02:13, 3061.91 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1400620/1807941 [15:28<02:12, 3066.39 examples/s]
Tokenizing train dataset:  77%|███████▋  | 1401046/1807941 [15:28<02:15, 2994.66 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1401352/1807941 [15:28<02:15, 3008.26 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1401665/1807941 [15:28<02:13, 3034.23 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1401993/1807941 [15:28<02:11, 3095.18 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1402428/1807941 [15:28<02:14, 3021.74 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1402758/1807941 [15:28<02:11, 3089.62 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1403208/1807941 [15:28<02:12, 3057.71 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1403524/1807941 [15:29<02:11, 3081.49 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1403845/1807941 [15:29<02:09, 3112.15 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1404159/1807941 [15:29<02:12, 3047.74 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1404465/1807941 [15:29<02:12, 3047.58 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1404775/1807941 [15:29<02:11, 3061.71 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1405217/1807941 [15:29<02:13, 3011.63 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1405544/1807941 [15:29<02:10, 3076.21 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1405877/1807941 [15:29<02:07, 3143.15 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1406303/1807941 [15:29<02:12, 3024.91 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1406621/1807941 [15:30<02:11, 3062.26 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1406939/1807941 [15:30<02:09, 3089.55 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1407398/1807941 [15:30<02:10, 3073.19 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1407708/1807941 [15:30<02:10, 3075.54 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1408124/1807941 [15:30<02:14, 2965.71 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1408444/1807941 [15:30<02:12, 3023.33 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1408757/1807941 [15:30<02:10, 3049.75 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1409189/1807941 [15:30<02:13, 2985.67 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1409504/1807941 [15:31<02:11, 3025.21 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1409819/1807941 [15:31<02:10, 3056.93 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1410264/1807941 [15:31<02:11, 3018.32 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1410593/1807941 [15:31<02:08, 3086.96 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1410908/1807941 [15:31<02:08, 3099.13 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1411359/1807941 [15:31<02:09, 3061.96 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1411684/1807941 [15:31<02:07, 3108.61 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1412000/1807941 [15:31<02:08, 3071.63 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1412324/1807941 [15:31<02:06, 3115.70 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1412651/1807941 [15:32<02:05, 3156.88 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1413093/1807941 [15:32<02:08, 3075.76 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1413417/1807941 [15:32<02:06, 3116.61 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1413744/1807941 [15:32<02:04, 3155.52 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1414219/1807941 [15:32<02:04, 3157.11 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1414561/1807941 [15:32<02:02, 3223.39 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1414911/1807941 [15:32<01:59, 3295.71 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1415398/1807941 [15:32<01:59, 3276.02 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1415760/1807941 [15:32<01:56, 3363.42 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1416231/1807941 [15:33<01:59, 3280.60 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1416720/1807941 [15:33<01:59, 3268.09 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1417174/1807941 [15:33<02:02, 3186.45 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1417504/1807941 [15:33<02:01, 3211.22 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1417829/1807941 [15:33<02:01, 3217.03 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1418284/1807941 [15:33<02:03, 3148.10 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1418740/1807941 [15:33<02:05, 3109.85 examples/s]
Tokenizing train dataset:  78%|███████▊  | 1419193/1807941 [15:34<02:06, 3079.29 examples/s]
Tokenizing train dataset:  79%|███████▊  | 1419516/1807941 [15:34<02:04, 3111.99 examples/s]
Tokenizing train dataset:  79%|███████▊  | 1419992/1807941 [15:34<02:03, 3129.37 examples/s]
Tokenizing train dataset:  79%|███████▊  | 1420455/1807941 [15:34<02:04, 3112.80 examples/s]
Tokenizing train dataset:  79%|███████▊  | 1420802/1807941 [15:34<02:01, 3193.42 examples/s]
Tokenizing train dataset:  79%|███████▊  | 1421293/1807941 [15:34<02:00, 3218.96 examples/s]
Tokenizing train dataset:  79%|███████▊  | 1421648/1807941 [15:34<01:57, 3297.27 examples/s]
Tokenizing train dataset:  79%|███████▊  | 1422000/1807941 [15:34<01:56, 3299.62 examples/s]
Tokenizing train dataset:  79%|███████▊  | 1422339/1807941 [15:35<01:56, 3319.75 examples/s]
Tokenizing train dataset:  79%|███████▊  | 1422836/1807941 [15:35<01:56, 3315.91 examples/s]
Tokenizing train dataset:  79%|███████▊  | 1423321/1807941 [15:35<01:58, 3239.58 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1423812/1807941 [15:35<01:58, 3246.92 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1424270/1807941 [15:35<02:00, 3183.21 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1424593/1807941 [15:35<02:00, 3192.15 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1425035/1807941 [15:35<02:03, 3108.22 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1425482/1807941 [15:36<02:04, 3063.44 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1425806/1807941 [15:36<02:03, 3102.82 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1426271/1807941 [15:36<02:03, 3098.43 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1426747/1807941 [15:36<02:02, 3119.50 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1427185/1807941 [15:36<02:04, 3052.29 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1427506/1807941 [15:36<02:03, 3086.56 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1427823/1807941 [15:36<02:02, 3105.49 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1428278/1807941 [15:36<02:03, 3076.09 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1428597/1807941 [15:37<02:02, 3103.00 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1428921/1807941 [15:37<02:00, 3136.35 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1429388/1807941 [15:37<02:01, 3122.15 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1429705/1807941 [15:37<02:00, 3129.93 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1430157/1807941 [15:37<02:03, 3062.03 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1430479/1807941 [15:37<02:01, 3098.81 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1430807/1807941 [15:37<01:59, 3144.00 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1431238/1807941 [15:37<02:03, 3043.34 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1431546/1807941 [15:38<02:03, 3050.06 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1432000/1807941 [15:38<02:05, 2989.27 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1432326/1807941 [15:38<02:02, 3054.23 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1432669/1807941 [15:38<01:59, 3150.45 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1433000/1807941 [15:38<01:58, 3162.23 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1433339/1807941 [15:38<01:56, 3222.57 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1433680/1807941 [15:38<01:54, 3273.08 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1434168/1807941 [15:38<01:55, 3247.99 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1434504/1807941 [15:38<01:54, 3273.74 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1434841/1807941 [15:39<01:53, 3297.11 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1435177/1807941 [15:39<01:54, 3260.52 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1435549/1807941 [15:39<01:49, 3388.89 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1435937/1807941 [15:39<01:45, 3526.51 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1436310/1807941 [15:39<01:43, 3583.43 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1436690/1807941 [15:39<01:41, 3645.61 examples/s]
Tokenizing train dataset:  79%|███████▉  | 1437207/1807941 [15:39<01:44, 3564.60 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1437680/1807941 [15:39<01:48, 3414.34 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1438110/1807941 [15:39<01:54, 3223.94 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1438570/1807941 [15:40<01:56, 3170.99 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1439000/1807941 [15:40<02:00, 3049.81 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1439343/1807941 [15:40<01:57, 3133.25 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1439694/1807941 [15:40<01:54, 3224.38 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1440189/1807941 [15:40<01:53, 3246.25 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1440531/1807941 [15:40<01:51, 3286.25 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1440881/1807941 [15:40<01:49, 3339.36 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1441372/1807941 [15:41<01:50, 3310.06 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1441718/1807941 [15:41<01:49, 3346.66 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1442221/1807941 [15:41<01:49, 3344.76 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1442716/1807941 [15:41<01:49, 3325.44 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1443175/1807941 [15:41<01:52, 3228.19 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1443653/1807941 [15:41<01:53, 3209.88 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1443978/1807941 [15:41<01:53, 3215.82 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1444449/1807941 [15:41<01:54, 3182.52 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1444788/1807941 [15:42<01:52, 3231.08 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1445259/1807941 [15:42<01:53, 3197.37 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1445601/1807941 [15:42<01:51, 3250.50 examples/s]
Tokenizing train dataset:  80%|███████▉  | 1445944/1807941 [15:42<01:49, 3293.15 examples/s]
Tokenizing train dataset:  80%|████████  | 1446424/1807941 [15:42<01:51, 3255.88 examples/s]
Tokenizing train dataset:  80%|████████  | 1446776/1807941 [15:42<01:48, 3317.37 examples/s]
Tokenizing train dataset:  80%|████████  | 1447269/1807941 [15:42<01:49, 3302.68 examples/s]
Tokenizing train dataset:  80%|████████  | 1447751/1807941 [15:42<01:50, 3269.80 examples/s]
Tokenizing train dataset:  80%|████████  | 1448201/1807941 [15:43<01:53, 3180.35 examples/s]
Tokenizing train dataset:  80%|████████  | 1448655/1807941 [15:43<01:54, 3127.10 examples/s]
Tokenizing train dataset:  80%|████████  | 1449090/1807941 [15:43<01:57, 3053.82 examples/s]
Tokenizing train dataset:  80%|████████  | 1449398/1807941 [15:43<01:57, 3059.69 examples/s]
Tokenizing train dataset:  80%|████████  | 1449858/1807941 [15:43<01:56, 3060.59 examples/s]
Tokenizing train dataset:  80%|████████  | 1450325/1807941 [15:43<01:57, 3033.77 examples/s]
Tokenizing train dataset:  80%|████████  | 1450673/1807941 [15:43<01:53, 3134.70 examples/s]
Tokenizing train dataset:  80%|████████  | 1451000/1807941 [15:44<01:53, 3143.79 examples/s]
Tokenizing train dataset:  80%|████████  | 1451346/1807941 [15:44<01:50, 3224.79 examples/s]
Tokenizing train dataset:  80%|████████  | 1451679/1807941 [15:44<01:49, 3246.36 examples/s]
Tokenizing train dataset:  80%|████████  | 1452158/1807941 [15:44<01:51, 3204.15 examples/s]
Tokenizing train dataset:  80%|████████  | 1452496/1807941 [15:44<01:49, 3245.44 examples/s]
Tokenizing train dataset:  80%|████████  | 1452833/1807941 [15:44<01:48, 3276.40 examples/s]
Tokenizing train dataset:  80%|████████  | 1453166/1807941 [15:44<01:49, 3236.94 examples/s]
Tokenizing train dataset:  80%|████████  | 1453494/1807941 [15:44<01:49, 3246.88 examples/s]
Tokenizing train dataset:  80%|████████  | 1453839/1807941 [15:44<01:47, 3302.94 examples/s]
Tokenizing train dataset:  80%|████████  | 1454174/1807941 [15:44<01:48, 3272.64 examples/s]
Tokenizing train dataset:  80%|████████  | 1454507/1807941 [15:45<01:47, 3282.62 examples/s]
Tokenizing train dataset:  80%|████████  | 1454839/1807941 [15:45<01:47, 3288.82 examples/s]
Tokenizing train dataset:  80%|████████  | 1455324/1807941 [15:45<01:48, 3262.09 examples/s]
Tokenizing train dataset:  81%|████████  | 1455665/1807941 [15:45<01:46, 3296.67 examples/s]
Tokenizing train dataset:  81%|████████  | 1456010/1807941 [15:45<01:45, 3336.50 examples/s]
Tokenizing train dataset:  81%|████████  | 1456400/1807941 [15:45<01:40, 3493.39 examples/s]
Tokenizing train dataset:  81%|████████  | 1456903/1807941 [15:45<01:42, 3434.73 examples/s]
Tokenizing train dataset:  81%|████████  | 1457428/1807941 [15:45<01:41, 3455.95 examples/s]
Tokenizing train dataset:  81%|████████  | 1457796/1807941 [15:46<01:39, 3510.37 examples/s]
Tokenizing train dataset:  81%|████████  | 1458305/1807941 [15:46<01:40, 3466.42 examples/s]
Tokenizing train dataset:  81%|████████  | 1458823/1807941 [15:46<01:40, 3460.30 examples/s]
Tokenizing train dataset:  81%|████████  | 1459312/1807941 [15:46<01:42, 3391.37 examples/s]
Tokenizing train dataset:  81%|████████  | 1459659/1807941 [15:46<01:42, 3408.39 examples/s]
Tokenizing train dataset:  81%|████████  | 1460128/1807941 [15:46<01:45, 3310.04 examples/s]
Tokenizing train dataset:  81%|████████  | 1460632/1807941 [15:46<01:44, 3324.19 examples/s]
Tokenizing train dataset:  81%|████████  | 1461100/1807941 [15:47<01:46, 3256.64 examples/s]
Tokenizing train dataset:  81%|████████  | 1461592/1807941 [15:47<01:46, 3261.49 examples/s]
Tokenizing train dataset:  81%|████████  | 1461926/1807941 [15:47<01:45, 3277.41 examples/s]
Tokenizing train dataset:  81%|████████  | 1462401/1807941 [15:47<01:46, 3238.03 examples/s]
Tokenizing train dataset:  81%|████████  | 1462749/1807941 [15:47<01:44, 3293.60 examples/s]
Tokenizing train dataset:  81%|████████  | 1463243/1807941 [15:47<01:44, 3288.32 examples/s]
Tokenizing train dataset:  81%|████████  | 1463597/1807941 [15:47<01:42, 3348.42 examples/s]
Tokenizing train dataset:  81%|████████  | 1464034/1807941 [15:47<01:47, 3193.97 examples/s]
Tokenizing train dataset:  81%|████████  | 1464524/1807941 [15:48<01:46, 3214.89 examples/s]
Tokenizing train dataset:  81%|████████  | 1464864/1807941 [15:48<01:45, 3252.55 examples/s]
Tokenizing train dataset:  81%|████████  | 1465337/1807941 [15:48<01:46, 3202.94 examples/s]
Tokenizing train dataset:  81%|████████  | 1465688/1807941 [15:48<01:44, 3273.90 examples/s]
Tokenizing train dataset:  81%|████████  | 1466175/1807941 [15:48<01:44, 3259.03 examples/s]
Tokenizing train dataset:  81%|████████  | 1466515/1807941 [15:48<01:43, 3290.66 examples/s]
Tokenizing train dataset:  81%|████████  | 1466853/1807941 [15:48<01:43, 3310.82 examples/s]
Tokenizing train dataset:  81%|████████  | 1467343/1807941 [15:48<01:44, 3266.52 examples/s]
Tokenizing train dataset:  81%|████████  | 1467848/1807941 [15:49<01:43, 3296.50 examples/s]
Tokenizing train dataset:  81%|████████  | 1468348/1807941 [15:49<01:43, 3283.27 examples/s]
Tokenizing train dataset:  81%|████████  | 1468688/1807941 [15:49<01:42, 3302.61 examples/s]
Tokenizing train dataset:  81%|████████▏ | 1469179/1807941 [15:49<01:42, 3290.61 examples/s]
Tokenizing train dataset:  81%|████████▏ | 1469519/1807941 [15:49<01:42, 3315.50 examples/s]
Tokenizing train dataset:  81%|████████▏ | 1469889/1807941 [15:49<01:39, 3413.06 examples/s]
Tokenizing train dataset:  81%|████████▏ | 1470375/1807941 [15:49<01:40, 3346.26 examples/s]
Tokenizing train dataset:  81%|████████▏ | 1470730/1807941 [15:49<01:39, 3388.45 examples/s]
Tokenizing train dataset:  81%|████████▏ | 1471228/1807941 [15:50<01:40, 3362.35 examples/s]
Tokenizing train dataset:  81%|████████▏ | 1471580/1807941 [15:50<01:38, 3398.29 examples/s]
Tokenizing train dataset:  81%|████████▏ | 1471959/1807941 [15:50<01:35, 3500.16 examples/s]
Tokenizing train dataset:  81%|████████▏ | 1472447/1807941 [15:50<01:38, 3409.13 examples/s]
Tokenizing train dataset:  81%|████████▏ | 1472796/1807941 [15:50<01:37, 3428.61 examples/s]
Tokenizing train dataset:  81%|████████▏ | 1473269/1807941 [15:50<01:40, 3328.47 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1473743/1807941 [15:50<01:42, 3267.47 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1474208/1807941 [15:51<01:43, 3211.80 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1474532/1807941 [15:51<01:43, 3215.55 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1475000/1807941 [15:51<01:46, 3132.36 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1475468/1807941 [15:51<01:46, 3126.14 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1475891/1807941 [15:51<01:50, 3016.47 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1476264/1807941 [15:51<01:56, 2842.23 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1476656/1807941 [15:51<02:00, 2754.60 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1476944/1807941 [15:51<01:59, 2777.77 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1477342/1807941 [15:52<02:01, 2717.99 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1477646/1807941 [15:52<01:58, 2791.21 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1478027/1807941 [15:52<02:02, 2704.20 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1478425/1807941 [15:52<02:02, 2684.25 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1478822/1807941 [15:52<02:03, 2670.28 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1479189/1807941 [15:52<02:07, 2583.12 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1479602/1807941 [15:52<02:05, 2611.87 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1480000/1807941 [15:53<02:06, 2587.54 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1480373/1807941 [15:53<02:08, 2549.54 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1480752/1807941 [15:53<02:08, 2539.04 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1481129/1807941 [15:53<02:10, 2509.29 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1481483/1807941 [15:53<02:12, 2456.18 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1481737/1807941 [15:53<02:12, 2470.09 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1482053/1807941 [15:54<02:18, 2348.63 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1482393/1807941 [15:54<02:20, 2321.19 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1482737/1807941 [15:54<02:21, 2295.19 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1483000/1807941 [15:54<02:21, 2303.25 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1483338/1807941 [15:54<02:22, 2281.58 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1483579/1807941 [15:54<02:20, 2304.01 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1483842/1807941 [15:54<02:15, 2383.09 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1484144/1807941 [15:54<02:24, 2241.12 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1484375/1807941 [15:55<02:23, 2250.45 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1484705/1807941 [15:55<02:26, 2209.77 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1485000/1807941 [15:55<02:34, 2090.12 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1485322/1807941 [15:55<02:33, 2106.06 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1485635/1807941 [15:55<02:33, 2097.95 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1485958/1807941 [15:55<02:32, 2113.31 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1486248/1807941 [15:55<02:37, 2047.68 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1486485/1807941 [15:56<02:31, 2116.92 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1486709/1807941 [15:56<02:29, 2145.60 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1487001/1807941 [15:56<02:34, 2072.60 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1487237/1807941 [15:56<02:30, 2133.68 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1487562/1807941 [15:56<02:29, 2144.55 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1487896/1807941 [15:56<02:27, 2165.92 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1488205/1807941 [15:56<02:30, 2125.81 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1488529/1807941 [15:57<02:30, 2126.71 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1488746/1807941 [15:57<02:29, 2133.82 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1489006/1807941 [15:57<02:21, 2246.63 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1489286/1807941 [15:57<02:13, 2388.72 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1489548/1807941 [15:57<02:09, 2449.60 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1489802/1807941 [15:57<02:08, 2470.79 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1490064/1807941 [15:57<02:06, 2508.26 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1490464/1807941 [15:57<02:03, 2563.53 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1490727/1807941 [15:57<02:03, 2566.98 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1491058/1807941 [15:58<02:10, 2433.02 examples/s]
Tokenizing train dataset:  82%|████████▏ | 1491424/1807941 [15:58<02:10, 2431.59 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1491784/1807941 [15:58<02:10, 2419.72 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1492106/1807941 [15:58<02:16, 2319.92 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1492341/1807941 [15:58<02:15, 2323.75 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1492579/1807941 [15:58<02:15, 2323.75 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1492818/1807941 [15:58<02:15, 2334.10 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1493115/1807941 [15:58<02:23, 2198.34 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1493357/1807941 [15:59<02:19, 2252.63 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1493589/1807941 [15:59<02:18, 2268.64 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1493930/1807941 [15:59<02:18, 2266.53 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1494177/1807941 [15:59<02:15, 2315.41 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1494446/1807941 [15:59<02:09, 2412.12 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1494690/1807941 [15:59<02:09, 2416.96 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1495015/1807941 [15:59<02:14, 2322.66 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1495279/1807941 [15:59<02:10, 2402.01 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1495529/1807941 [15:59<02:08, 2425.43 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1495775/1807941 [16:00<02:08, 2432.75 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1496145/1807941 [16:00<02:08, 2432.41 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1496416/1807941 [16:00<02:04, 2501.93 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1496801/1807941 [16:00<02:03, 2517.21 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1497165/1807941 [16:00<02:05, 2484.67 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1497434/1807941 [16:00<02:02, 2533.12 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1497699/1807941 [16:00<02:01, 2557.62 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1497962/1807941 [16:00<02:00, 2575.13 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1498316/1807941 [16:01<02:04, 2491.93 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1498577/1807941 [16:01<02:02, 2520.88 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1498971/1807941 [16:01<02:01, 2542.56 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1499327/1807941 [16:01<02:04, 2473.32 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1499693/1807941 [16:01<02:05, 2460.24 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1499974/1807941 [16:01<02:01, 2537.60 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1500342/1807941 [16:01<02:03, 2499.46 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1500609/1807941 [16:01<02:01, 2539.09 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1500866/1807941 [16:02<02:01, 2529.00 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1501208/1807941 [16:02<02:05, 2437.91 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1501464/1807941 [16:02<02:05, 2447.55 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1501718/1807941 [16:02<02:04, 2462.18 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1501977/1807941 [16:02<02:02, 2494.53 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1502348/1807941 [16:02<02:03, 2482.96 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1502655/1807941 [16:02<01:55, 2633.63 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1502951/1807941 [16:02<01:52, 2719.04 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1503339/1807941 [16:03<01:54, 2667.68 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1503630/1807941 [16:03<01:51, 2728.06 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1503921/1807941 [16:03<01:49, 2773.76 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1504327/1807941 [16:03<01:50, 2746.82 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1504630/1807941 [16:03<01:47, 2816.21 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1505011/1807941 [16:03<01:51, 2707.11 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1505410/1807941 [16:03<01:52, 2689.53 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1505689/1807941 [16:03<01:51, 2711.05 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1506046/1807941 [16:04<01:56, 2590.44 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1506435/1807941 [16:04<01:56, 2587.81 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1506786/1807941 [16:04<02:00, 2504.82 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1507138/1807941 [16:04<02:02, 2453.44 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1507406/1807941 [16:04<02:00, 2501.68 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1507664/1807941 [16:04<01:59, 2518.92 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1508000/1807941 [16:04<02:04, 2412.08 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1508254/1807941 [16:04<02:02, 2441.91 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1508506/1807941 [16:05<02:01, 2459.47 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1508872/1807941 [16:05<02:02, 2442.14 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1509192/1807941 [16:05<02:08, 2331.17 examples/s]
Tokenizing train dataset:  83%|████████▎ | 1509447/1807941 [16:05<02:05, 2381.82 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1509711/1807941 [16:05<02:01, 2445.13 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1509988/1807941 [16:05<01:58, 2523.96 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1510383/1807941 [16:05<01:56, 2560.87 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1510694/1807941 [16:05<01:50, 2699.86 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1510993/1807941 [16:05<01:46, 2777.41 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1511390/1807941 [16:06<01:48, 2727.14 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1511684/1807941 [16:06<01:46, 2780.53 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1512000/1807941 [16:06<01:45, 2812.78 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1512301/1807941 [16:06<01:43, 2865.26 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1512726/1807941 [16:06<01:43, 2851.01 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1513150/1807941 [16:06<01:47, 2751.70 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1513534/1807941 [16:06<01:49, 2687.24 examples/s]
Tokenizing train dataset:  84%|████████▎ | 1513807/1807941 [16:07<01:49, 2695.05 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1514180/1807941 [16:07<01:52, 2621.94 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1514452/1807941 [16:07<01:51, 2639.75 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1514745/1807941 [16:07<01:48, 2712.23 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1515141/1807941 [16:07<01:51, 2635.57 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1515518/1807941 [16:07<01:52, 2592.86 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1515804/1807941 [16:07<01:50, 2655.49 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1516128/1807941 [16:07<01:57, 2476.49 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1516505/1807941 [16:08<01:57, 2485.61 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1516879/1807941 [16:08<01:57, 2483.88 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1517130/1807941 [16:08<01:58, 2460.96 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1517406/1807941 [16:08<01:54, 2531.75 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1517677/1807941 [16:08<01:52, 2572.98 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1518033/1807941 [16:08<01:56, 2494.15 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1518313/1807941 [16:08<01:52, 2568.68 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1518603/1807941 [16:08<01:48, 2655.53 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1518982/1807941 [16:09<01:50, 2604.28 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1519363/1807941 [16:09<01:51, 2579.25 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1519636/1807941 [16:09<01:50, 2608.79 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1520000/1807941 [16:09<01:53, 2542.80 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1520289/1807941 [16:09<01:49, 2627.03 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1520589/1807941 [16:09<01:45, 2723.56 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1520874/1807941 [16:09<01:44, 2755.05 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1521158/1807941 [16:09<01:43, 2777.72 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1521469/1807941 [16:09<01:40, 2863.44 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1521893/1807941 [16:10<01:40, 2839.16 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1522288/1807941 [16:10<01:43, 2750.45 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1522665/1807941 [16:10<01:46, 2668.36 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1522941/1807941 [16:10<01:45, 2688.77 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1523328/1807941 [16:10<01:47, 2648.71 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1523600/1807941 [16:10<01:46, 2665.89 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1523901/1807941 [16:10<01:43, 2752.83 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1524273/1807941 [16:10<01:46, 2653.95 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1524662/1807941 [16:11<01:47, 2625.38 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1525000/1807941 [16:11<01:55, 2456.29 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1525255/1807941 [16:11<01:54, 2475.09 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1525516/1807941 [16:11<01:52, 2505.09 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1525775/1807941 [16:11<01:51, 2525.70 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1526136/1807941 [16:11<01:53, 2473.09 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1526393/1807941 [16:11<01:52, 2493.45 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1526721/1807941 [16:11<01:58, 2382.99 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1527042/1807941 [16:12<02:02, 2298.47 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1527305/1807941 [16:12<01:58, 2365.34 examples/s]
Tokenizing train dataset:  84%|████████▍ | 1527551/1807941 [16:12<01:57, 2377.56 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1527926/1807941 [16:12<01:55, 2414.74 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1528280/1807941 [16:12<01:56, 2392.11 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1528527/1807941 [16:12<01:56, 2407.34 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1528788/1807941 [16:12<01:53, 2457.97 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1529064/1807941 [16:12<01:49, 2537.86 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1529325/1807941 [16:13<01:48, 2556.52 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1529703/1807941 [16:13<01:49, 2539.65 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1529999/1807941 [16:13<01:45, 2638.94 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1530360/1807941 [16:13<01:48, 2552.26 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1530722/1807941 [16:13<01:51, 2483.59 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1531000/1807941 [16:13<01:49, 2527.43 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1531273/1807941 [16:13<01:47, 2577.22 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1531656/1807941 [16:13<01:48, 2544.99 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1532000/1807941 [16:14<01:53, 2426.09 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1532246/1807941 [16:14<01:53, 2431.00 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1532588/1807941 [16:14<01:55, 2378.29 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1532938/1807941 [16:14<01:56, 2360.95 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1533299/1807941 [16:14<01:55, 2373.78 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1533652/1807941 [16:14<01:56, 2359.64 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1533906/1807941 [16:14<01:54, 2396.56 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1534254/1807941 [16:15<01:55, 2362.43 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1534493/1807941 [16:15<01:55, 2366.75 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1534822/1807941 [16:15<01:59, 2289.24 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1535067/1807941 [16:15<01:57, 2317.68 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1535311/1807941 [16:15<01:56, 2345.98 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1535577/1807941 [16:15<01:52, 2429.59 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1535851/1807941 [16:15<01:48, 2513.16 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1536243/1807941 [16:15<01:46, 2547.62 examples/s]
Tokenizing train dataset:  85%|████████▍ | 1536533/1807941 [16:15<01:42, 2636.55 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1536824/1807941 [16:16<01:40, 2709.63 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1537214/1807941 [16:16<01:41, 2667.59 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1537497/1807941 [16:16<01:39, 2706.47 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1537775/1807941 [16:16<01:39, 2721.13 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1538139/1807941 [16:16<01:43, 2600.22 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1538440/1807941 [16:16<01:52, 2395.09 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1538740/1807941 [16:16<01:59, 2260.19 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1539000/1807941 [16:17<02:08, 2086.23 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1539239/1807941 [16:17<02:04, 2153.79 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1539487/1807941 [16:17<02:00, 2232.41 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1539735/1807941 [16:17<01:56, 2293.26 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1540032/1807941 [16:17<02:03, 2175.37 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1540380/1807941 [16:17<02:00, 2218.13 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1540613/1807941 [16:17<01:59, 2243.40 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1540925/1807941 [16:17<02:02, 2176.63 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1541204/1807941 [16:18<02:08, 2070.82 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1541517/1807941 [16:18<02:09, 2059.12 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1541749/1807941 [16:18<02:06, 2103.05 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1541968/1807941 [16:18<02:05, 2122.55 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1542279/1807941 [16:18<02:07, 2089.94 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1542550/1807941 [16:18<02:13, 1994.97 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1542788/1807941 [16:18<02:07, 2079.72 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1543066/1807941 [16:18<02:12, 2001.52 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1543273/1807941 [16:19<02:11, 2014.29 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1543487/1807941 [16:19<02:09, 2042.43 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1543778/1807941 [16:19<02:12, 1999.25 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1543999/1807941 [16:19<02:08, 2049.13 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1544283/1807941 [16:19<02:12, 1993.07 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1544542/1807941 [16:19<02:18, 1895.60 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1544741/1807941 [16:19<02:17, 1915.34 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1544959/1807941 [16:19<02:12, 1981.23 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1545257/1807941 [16:20<02:12, 1978.27 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1545467/1807941 [16:20<02:11, 2001.57 examples/s]
Tokenizing train dataset:  85%|████████▌ | 1545686/1807941 [16:20<02:08, 2048.55 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1545895/1807941 [16:20<02:07, 2057.35 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1546106/1807941 [16:20<02:07, 2051.32 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1546392/1807941 [16:20<01:54, 2277.84 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1546680/1807941 [16:20<01:46, 2446.84 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1546972/1807941 [16:20<01:40, 2584.21 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1547238/1807941 [16:20<01:40, 2605.41 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1547543/1807941 [16:20<01:35, 2734.46 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1547844/1807941 [16:21<01:32, 2812.48 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1548261/1807941 [16:21<01:32, 2795.00 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1548545/1807941 [16:21<01:32, 2804.02 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1548951/1807941 [16:21<01:33, 2755.82 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1549261/1807941 [16:21<01:43, 2509.37 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1549571/1807941 [16:21<01:49, 2360.85 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1549812/1807941 [16:21<01:48, 2370.69 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1550111/1807941 [16:22<01:55, 2230.23 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1550407/1807941 [16:22<02:00, 2132.84 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1550734/1807941 [16:22<01:59, 2146.81 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1551013/1807941 [16:22<02:05, 2043.47 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1551248/1807941 [16:22<02:01, 2104.81 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1551545/1807941 [16:22<02:04, 2051.93 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1551834/1807941 [16:22<02:07, 2006.66 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1552131/1807941 [16:23<02:08, 1990.28 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1552434/1807941 [16:23<02:08, 1994.59 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1552652/1807941 [16:23<02:05, 2032.16 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1552936/1807941 [16:23<02:08, 1984.69 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1553138/1807941 [16:23<02:07, 1991.01 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1553353/1807941 [16:23<02:05, 2029.72 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1553579/1807941 [16:23<02:01, 2088.64 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1553800/1807941 [16:23<02:00, 2117.49 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1554027/1807941 [16:23<01:57, 2154.33 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1554321/1807941 [16:24<01:46, 2376.67 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1554636/1807941 [16:24<01:37, 2598.02 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1554928/1807941 [16:24<01:34, 2689.46 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1555222/1807941 [16:24<01:31, 2760.32 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1555530/1807941 [16:24<01:28, 2851.57 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1555847/1807941 [16:24<01:25, 2935.54 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1556270/1807941 [16:24<01:27, 2888.65 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1556730/1807941 [16:24<01:25, 2945.71 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1557172/1807941 [16:25<01:25, 2944.66 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1557479/1807941 [16:25<01:24, 2971.49 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1557904/1807941 [16:25<01:25, 2922.52 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1558297/1807941 [16:25<01:28, 2820.44 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1558709/1807941 [16:25<01:29, 2787.33 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1558993/1807941 [16:25<01:28, 2798.70 examples/s]
Tokenizing train dataset:  86%|████████▌ | 1559343/1807941 [16:25<01:34, 2639.94 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1559745/1807941 [16:25<01:33, 2650.62 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1560132/1807941 [16:26<01:35, 2599.51 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1560405/1807941 [16:26<01:34, 2625.76 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1560681/1807941 [16:26<01:33, 2657.88 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1560965/1807941 [16:26<01:31, 2704.36 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1561367/1807941 [16:26<01:31, 2681.00 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1561665/1807941 [16:26<01:29, 2753.20 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1561961/1807941 [16:26<01:27, 2805.60 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1562379/1807941 [16:26<01:27, 2791.46 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1562807/1807941 [16:27<01:27, 2808.45 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1563224/1807941 [16:27<01:27, 2796.03 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1563521/1807941 [16:27<01:26, 2836.45 examples/s]
Tokenizing train dataset:  86%|████████▋ | 1563836/1807941 [16:27<01:23, 2914.41 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1564273/1807941 [16:27<01:23, 2910.30 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1564574/1807941 [16:27<01:22, 2933.47 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1564887/1807941 [16:27<01:21, 2981.54 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1565312/1807941 [16:27<01:22, 2924.54 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1565608/1807941 [16:28<01:22, 2932.05 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1566014/1807941 [16:28<01:24, 2848.22 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1566311/1807941 [16:28<01:23, 2876.63 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1566724/1807941 [16:28<01:25, 2830.56 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1567083/1807941 [16:28<01:29, 2683.21 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1567467/1807941 [16:28<01:31, 2641.04 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1567850/1807941 [16:28<01:31, 2610.76 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1568187/1807941 [16:29<01:36, 2481.69 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1568454/1807941 [16:29<01:34, 2521.95 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1568719/1807941 [16:29<01:33, 2550.24 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1569041/1807941 [16:29<01:39, 2410.66 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1569313/1807941 [16:29<01:36, 2484.39 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1569680/1807941 [16:29<01:36, 2460.10 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1569937/1807941 [16:29<02:01, 1965.86 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570157/1807941 [16:31<07:21, 538.09 examples/s] 
Tokenizing train dataset:  87%|████████▋ | 1570312/1807941 [16:32<10:43, 369.25 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570430/1807941 [16:32<13:11, 300.00 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570522/1807941 [16:33<15:06, 261.86 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570586/1807941 [16:33<16:35, 238.31 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570642/1807941 [16:34<17:54, 220.82 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570683/1807941 [16:34<18:55, 208.92 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570718/1807941 [16:34<19:42, 200.58 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570751/1807941 [16:34<20:34, 192.16 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570775/1807941 [16:35<21:17, 185.60 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570800/1807941 [16:35<21:57, 179.97 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570824/1807941 [16:35<22:33, 175.18 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570848/1807941 [16:35<23:09, 170.66 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570872/1807941 [16:35<23:36, 167.33 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570896/1807941 [16:35<24:07, 163.73 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570920/1807941 [16:36<24:46, 159.43 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570945/1807941 [16:36<24:54, 158.63 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570962/1807941 [16:36<25:01, 157.83 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570978/1807941 [16:36<24:59, 158.07 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1570995/1807941 [16:36<25:03, 157.55 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571016/1807941 [16:36<31:45, 124.34 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571032/1807941 [16:36<30:04, 131.32 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571050/1807941 [16:37<28:13, 139.86 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571068/1807941 [16:37<26:48, 147.24 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571086/1807941 [16:37<25:51, 152.63 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571104/1807941 [16:37<25:13, 156.49 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571123/1807941 [16:37<24:12, 163.09 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571141/1807941 [16:37<23:45, 166.14 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571159/1807941 [16:37<23:24, 168.64 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571179/1807941 [16:37<22:49, 172.94 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571197/1807941 [16:37<22:55, 172.12 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571215/1807941 [16:37<23:01, 171.41 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571233/1807941 [16:38<23:03, 171.07 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571251/1807941 [16:38<23:09, 170.35 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571269/1807941 [16:38<23:21, 168.88 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571286/1807941 [16:38<23:33, 167.42 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571304/1807941 [16:38<23:44, 166.06 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571322/1807941 [16:38<23:46, 165.90 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571339/1807941 [16:38<23:52, 165.14 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571364/1807941 [16:38<24:20, 161.95 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571382/1807941 [16:38<24:11, 162.93 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571399/1807941 [16:39<24:16, 162.43 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571416/1807941 [16:39<24:20, 161.95 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571441/1807941 [16:39<24:29, 160.94 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571459/1807941 [16:39<24:22, 161.71 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571477/1807941 [16:39<24:07, 163.37 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571495/1807941 [16:39<24:02, 163.87 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571513/1807941 [16:39<23:53, 164.98 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571530/1807941 [16:39<23:52, 165.02 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571548/1807941 [16:39<23:45, 165.84 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571566/1807941 [16:40<23:30, 167.60 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571585/1807941 [16:40<23:02, 170.97 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571603/1807941 [16:40<23:00, 171.18 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571622/1807941 [16:40<22:38, 173.92 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571641/1807941 [16:40<22:21, 176.21 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571660/1807941 [16:40<22:05, 178.28 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571679/1807941 [16:40<22:08, 177.80 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571698/1807941 [16:40<22:04, 178.37 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571718/1807941 [16:40<21:40, 181.58 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571738/1807941 [16:41<21:24, 183.94 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571758/1807941 [16:41<21:16, 184.96 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571777/1807941 [16:41<21:30, 183.06 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571804/1807941 [16:41<22:07, 177.88 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571824/1807941 [16:41<21:54, 179.65 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571844/1807941 [16:41<21:44, 181.00 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571864/1807941 [16:41<21:33, 182.53 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571883/1807941 [16:41<21:38, 181.74 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571903/1807941 [16:41<21:25, 183.63 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571931/1807941 [16:42<21:26, 183.51 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571951/1807941 [16:42<21:14, 185.23 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571970/1807941 [16:42<21:19, 184.37 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1571990/1807941 [16:42<21:13, 185.31 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572016/1807941 [16:42<28:54, 136.02 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572032/1807941 [16:42<28:01, 140.29 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572048/1807941 [16:42<27:14, 144.28 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572066/1807941 [16:43<26:21, 149.19 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572084/1807941 [16:43<25:43, 152.83 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572102/1807941 [16:43<25:06, 156.58 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572127/1807941 [16:43<24:56, 157.55 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572145/1807941 [16:43<24:45, 158.69 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572162/1807941 [16:43<24:35, 159.75 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572180/1807941 [16:43<24:20, 161.45 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572197/1807941 [16:43<24:25, 160.91 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572215/1807941 [16:43<23:53, 164.45 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572234/1807941 [16:44<23:24, 167.84 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572253/1807941 [16:44<22:58, 170.97 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572271/1807941 [16:44<22:51, 171.80 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572290/1807941 [16:44<22:39, 173.33 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572310/1807941 [16:44<22:12, 176.87 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572330/1807941 [16:44<21:53, 179.40 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572356/1807941 [16:44<22:28, 174.72 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572374/1807941 [16:44<22:51, 171.76 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572392/1807941 [16:44<23:07, 169.81 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572410/1807941 [16:45<23:22, 167.88 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572428/1807941 [16:45<23:30, 166.96 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572446/1807941 [16:45<23:36, 166.21 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572463/1807941 [16:45<23:44, 165.31 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572481/1807941 [16:45<23:46, 165.05 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572499/1807941 [16:45<23:41, 165.57 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572522/1807941 [16:45<24:48, 158.15 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572538/1807941 [16:45<24:58, 157.13 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572556/1807941 [16:45<24:45, 158.45 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572573/1807941 [16:46<24:36, 159.40 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572591/1807941 [16:46<24:23, 160.77 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572608/1807941 [16:46<24:28, 160.23 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572625/1807941 [16:46<24:27, 160.32 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572643/1807941 [16:46<24:20, 161.15 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572663/1807941 [16:46<23:18, 168.27 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572686/1807941 [16:46<21:17, 184.21 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572710/1807941 [16:46<19:53, 197.07 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572732/1807941 [16:46<19:30, 200.95 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572755/1807941 [16:47<19:00, 206.25 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572780/1807941 [16:47<18:11, 215.49 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572812/1807941 [16:47<18:34, 210.89 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572835/1807941 [16:47<18:21, 213.53 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572859/1807941 [16:47<18:05, 216.49 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572883/1807941 [16:47<17:50, 219.59 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572906/1807941 [16:47<17:46, 220.29 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572930/1807941 [16:47<17:46, 220.40 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572964/1807941 [16:47<17:51, 219.23 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1572988/1807941 [16:48<17:32, 223.31 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573012/1807941 [16:48<22:56, 170.71 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573034/1807941 [16:48<21:35, 181.30 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573059/1807941 [16:48<20:04, 195.04 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573088/1807941 [16:48<18:02, 216.91 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573112/1807941 [16:48<17:52, 219.05 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573135/1807941 [16:48<17:39, 221.57 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573161/1807941 [16:48<16:55, 231.15 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573192/1807941 [16:49<15:36, 250.80 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573223/1807941 [16:49<14:44, 265.34 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573251/1807941 [16:49<14:35, 268.09 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573279/1807941 [16:49<14:46, 264.57 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573316/1807941 [16:49<15:19, 255.09 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573345/1807941 [16:49<14:50, 263.51 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573376/1807941 [16:49<14:13, 274.98 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573408/1807941 [16:49<13:42, 285.01 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573438/1807941 [16:49<13:31, 288.97 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573474/1807941 [16:50<14:39, 266.63 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573506/1807941 [16:50<16:05, 242.93 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573539/1807941 [16:50<17:08, 227.96 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573574/1807941 [16:50<17:08, 227.92 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573608/1807941 [16:50<17:20, 225.22 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573631/1807941 [16:50<17:24, 224.42 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573654/1807941 [16:50<17:32, 222.51 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573678/1807941 [16:51<17:24, 224.29 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573701/1807941 [16:51<17:35, 221.89 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573725/1807941 [16:51<17:22, 224.73 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573751/1807941 [16:51<16:56, 230.35 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573775/1807941 [16:51<16:52, 231.18 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573801/1807941 [16:51<16:24, 237.87 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573827/1807941 [16:51<16:20, 238.75 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573852/1807941 [16:51<16:17, 239.59 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573877/1807941 [16:51<16:25, 237.50 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573902/1807941 [16:51<16:28, 236.68 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573927/1807941 [16:52<16:22, 238.19 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573951/1807941 [16:52<16:28, 236.82 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1573982/1807941 [16:52<17:36, 221.38 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574011/1807941 [16:52<22:04, 176.56 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574032/1807941 [16:52<21:19, 182.85 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574053/1807941 [16:52<20:56, 186.20 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574083/1807941 [16:52<18:28, 211.01 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574115/1807941 [16:52<16:29, 236.20 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574144/1807941 [16:53<15:51, 245.79 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574182/1807941 [16:53<15:52, 245.38 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574218/1807941 [16:53<16:09, 241.17 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574243/1807941 [16:53<16:08, 241.22 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574277/1807941 [16:53<16:38, 233.99 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574306/1807941 [16:53<15:53, 245.04 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574335/1807941 [16:53<15:22, 253.19 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574366/1807941 [16:53<14:41, 264.95 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574394/1807941 [16:54<14:38, 265.81 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574423/1807941 [16:54<14:22, 270.65 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574451/1807941 [16:54<14:25, 269.88 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574482/1807941 [16:54<13:56, 279.24 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574512/1807941 [16:54<13:47, 282.21 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574546/1807941 [16:54<15:13, 255.36 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574579/1807941 [16:54<16:09, 240.82 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574604/1807941 [16:54<16:15, 239.15 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574639/1807941 [16:55<16:36, 234.22 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574677/1807941 [16:55<16:18, 238.32 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574702/1807941 [16:55<16:11, 240.04 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574731/1807941 [16:55<15:36, 249.10 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574760/1807941 [16:55<15:05, 257.48 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574789/1807941 [16:55<14:47, 262.75 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574828/1807941 [16:55<15:10, 256.03 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574854/1807941 [16:55<15:10, 256.03 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574883/1807941 [16:55<14:47, 262.69 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574922/1807941 [16:56<14:58, 259.26 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574950/1807941 [16:56<14:50, 261.67 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1574989/1807941 [16:56<15:02, 258.22 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575025/1807941 [16:56<18:09, 213.79 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575049/1807941 [16:56<17:53, 216.86 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575073/1807941 [16:56<17:45, 218.63 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575100/1807941 [16:56<16:52, 229.93 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575130/1807941 [16:57<15:48, 245.49 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575157/1807941 [16:57<15:32, 249.67 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575186/1807941 [16:57<15:05, 256.94 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575214/1807941 [16:57<14:54, 260.08 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575244/1807941 [16:57<14:30, 267.22 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575272/1807941 [16:57<14:35, 265.68 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575299/1807941 [16:57<14:39, 264.55 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575329/1807941 [16:57<14:14, 272.06 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575369/1807941 [16:57<14:25, 268.72 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575399/1807941 [16:58<14:09, 273.61 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575428/1807941 [16:58<14:12, 272.84 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575456/1807941 [16:58<14:14, 272.22 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575496/1807941 [16:58<14:33, 266.09 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575525/1807941 [16:58<14:24, 268.91 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575556/1807941 [16:58<13:58, 277.15 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575584/1807941 [16:58<13:57, 277.49 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575616/1807941 [16:58<13:29, 287.16 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575651/1807941 [16:58<12:47, 302.70 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575682/1807941 [16:59<12:55, 299.41 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575713/1807941 [16:59<13:02, 296.96 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575756/1807941 [16:59<13:18, 290.92 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575794/1807941 [16:59<13:58, 276.72 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575832/1807941 [16:59<14:34, 265.57 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575868/1807941 [16:59<15:16, 253.14 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575896/1807941 [16:59<14:56, 258.97 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575925/1807941 [16:59<14:38, 264.18 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575953/1807941 [17:00<14:35, 265.05 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1575983/1807941 [17:00<14:09, 273.16 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576013/1807941 [17:00<17:39, 219.00 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576041/1807941 [17:00<16:40, 231.75 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576066/1807941 [17:00<16:36, 232.72 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576100/1807941 [17:00<16:53, 228.69 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576124/1807941 [17:00<16:58, 227.63 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576148/1807941 [17:00<16:49, 229.51 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576180/1807941 [17:01<17:37, 219.08 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576203/1807941 [17:01<17:41, 218.21 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576236/1807941 [17:01<18:02, 214.05 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576269/1807941 [17:01<18:11, 212.25 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576291/1807941 [17:01<18:10, 212.47 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576315/1807941 [17:01<17:53, 215.70 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576338/1807941 [17:01<17:52, 216.00 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576360/1807941 [17:01<17:58, 214.78 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576384/1807941 [17:02<17:46, 217.10 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576410/1807941 [17:02<17:09, 224.86 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576437/1807941 [17:02<16:23, 235.34 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576462/1807941 [17:02<16:21, 235.79 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576486/1807941 [17:02<16:21, 235.88 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576513/1807941 [17:02<15:51, 243.32 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576539/1807941 [17:02<15:47, 244.19 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576565/1807941 [17:02<15:37, 246.76 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576592/1807941 [17:02<15:32, 248.18 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576620/1807941 [17:02<15:07, 254.93 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576647/1807941 [17:03<15:02, 256.37 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576687/1807941 [17:03<14:55, 258.13 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576720/1807941 [17:03<14:02, 274.56 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576751/1807941 [17:03<13:39, 282.16 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576792/1807941 [17:03<13:58, 275.58 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576821/1807941 [17:03<13:53, 277.14 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576849/1807941 [17:03<13:56, 276.11 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576889/1807941 [17:03<14:15, 270.19 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576929/1807941 [17:04<14:33, 264.57 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576960/1807941 [17:04<14:05, 273.04 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1576989/1807941 [17:04<13:59, 275.06 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577024/1807941 [17:04<17:39, 217.86 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577053/1807941 [17:04<16:43, 230.06 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577080/1807941 [17:04<16:16, 236.48 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577106/1807941 [17:04<15:55, 241.58 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577135/1807941 [17:04<15:17, 251.49 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577174/1807941 [17:05<15:18, 251.13 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577200/1807941 [17:05<15:16, 251.81 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577227/1807941 [17:05<15:11, 253.18 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577255/1807941 [17:05<14:58, 256.86 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577282/1807941 [17:05<14:56, 257.21 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577319/1807941 [17:05<15:24, 249.49 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577345/1807941 [17:05<15:21, 250.26 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577371/1807941 [17:05<15:18, 250.98 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577398/1807941 [17:06<15:12, 252.61 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577425/1807941 [17:06<15:15, 251.77 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577451/1807941 [17:06<15:23, 249.60 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577488/1807941 [17:06<15:35, 246.31 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577515/1807941 [17:06<15:24, 249.16 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577541/1807941 [17:06<15:24, 249.10 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577566/1807941 [17:06<15:32, 247.00 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577592/1807941 [17:06<15:28, 248.22 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577618/1807941 [17:06<15:24, 249.07 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577645/1807941 [17:07<15:06, 254.09 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577671/1807941 [17:07<15:02, 255.25 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577699/1807941 [17:07<14:48, 259.23 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577726/1807941 [17:07<14:44, 260.38 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577753/1807941 [17:07<14:41, 261.23 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577780/1807941 [17:07<14:44, 260.32 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577820/1807941 [17:07<14:50, 258.32 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577848/1807941 [17:07<14:44, 260.14 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577885/1807941 [17:07<15:06, 253.71 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577920/1807941 [17:08<15:44, 243.54 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577954/1807941 [17:08<16:20, 234.68 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1577978/1807941 [17:08<16:24, 233.56 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578012/1807941 [17:08<19:14, 199.24 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578037/1807941 [17:08<18:27, 207.53 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578063/1807941 [17:08<17:37, 217.41 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578088/1807941 [17:08<17:08, 223.55 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578118/1807941 [17:09<15:57, 240.12 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578147/1807941 [17:09<15:12, 251.83 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578174/1807941 [17:09<15:10, 252.48 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578202/1807941 [17:09<14:55, 256.64 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578229/1807941 [17:09<14:50, 257.85 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578256/1807941 [17:09<14:48, 258.51 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578296/1807941 [17:09<14:48, 258.38 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578325/1807941 [17:09<14:31, 263.36 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578365/1807941 [17:09<14:34, 262.39 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578394/1807941 [17:10<14:26, 264.84 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578431/1807941 [17:10<15:01, 254.62 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578462/1807941 [17:10<16:22, 233.49 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578496/1807941 [17:10<15:03, 253.84 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578523/1807941 [17:10<15:04, 253.73 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578549/1807941 [17:10<15:04, 253.64 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578575/1807941 [17:10<15:05, 253.30 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578614/1807941 [17:10<15:07, 252.74 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578653/1807941 [17:11<15:09, 252.09 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578679/1807941 [17:11<15:12, 251.30 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578705/1807941 [17:11<15:19, 249.44 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578740/1807941 [17:11<15:56, 239.71 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578777/1807941 [17:11<16:01, 238.42 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578802/1807941 [17:11<16:01, 238.26 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578829/1807941 [17:11<15:38, 244.18 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578857/1807941 [17:11<15:16, 250.05 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578894/1807941 [17:12<15:33, 245.46 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578920/1807941 [17:12<15:20, 248.71 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578947/1807941 [17:12<15:11, 251.14 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1578976/1807941 [17:12<14:43, 259.29 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579013/1807941 [17:12<17:39, 216.02 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579039/1807941 [17:12<17:11, 221.83 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579067/1807941 [17:12<16:14, 234.85 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579095/1807941 [17:12<15:53, 240.10 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579124/1807941 [17:13<15:18, 249.13 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579164/1807941 [17:13<13:15, 287.71 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579205/1807941 [17:13<13:38, 279.40 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579246/1807941 [17:13<13:53, 274.44 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579287/1807941 [17:13<14:07, 269.86 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579326/1807941 [17:13<14:21, 265.49 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579364/1807941 [17:13<14:47, 257.50 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579403/1807941 [17:14<14:52, 255.98 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579429/1807941 [17:14<14:56, 255.00 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579457/1807941 [17:14<14:46, 257.69 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579483/1807941 [17:14<14:52, 255.84 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579512/1807941 [17:14<14:34, 261.28 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579544/1807941 [17:14<13:48, 275.64 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579577/1807941 [17:14<13:13, 287.82 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579607/1807941 [17:14<13:06, 290.24 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579647/1807941 [17:14<13:34, 280.14 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579686/1807941 [17:15<14:07, 269.30 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579714/1807941 [17:15<14:14, 267.18 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579742/1807941 [17:15<14:16, 266.43 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579779/1807941 [17:15<14:48, 256.72 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579816/1807941 [17:15<15:10, 250.67 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579853/1807941 [17:15<15:28, 245.62 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579888/1807941 [17:15<15:58, 237.84 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579923/1807941 [17:16<16:22, 232.17 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579949/1807941 [17:16<16:09, 235.23 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1579973/1807941 [17:16<16:14, 233.96 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580000/1807941 [17:16<18:23, 206.58 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580052/1807941 [17:16<13:39, 278.22 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580105/1807941 [17:16<11:09, 340.35 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580157/1807941 [17:16<09:50, 385.59 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580208/1807941 [17:16<09:06, 416.91 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580261/1807941 [17:17<08:31, 444.74 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580314/1807941 [17:17<08:09, 464.67 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580367/1807941 [17:17<07:51, 482.41 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580423/1807941 [17:17<07:33, 501.90 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580476/1807941 [17:17<07:28, 507.66 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580534/1807941 [17:17<07:12, 526.24 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580590/1807941 [17:17<07:06, 532.91 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580645/1807941 [17:17<07:03, 536.65 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580722/1807941 [17:17<07:14, 522.79 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580775/1807941 [17:17<07:14, 522.56 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580828/1807941 [17:18<07:16, 520.51 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580905/1807941 [17:18<07:21, 513.86 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1580958/1807941 [17:18<07:20, 515.28 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581024/1807941 [17:18<08:07, 465.66 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581073/1807941 [17:18<08:03, 468.77 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581127/1807941 [17:18<07:48, 483.92 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581182/1807941 [17:18<07:34, 499.11 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581234/1807941 [17:18<07:30, 503.32 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581286/1807941 [17:19<07:28, 504.97 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581340/1807941 [17:19<07:22, 512.34 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581395/1807941 [17:19<07:17, 518.28 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581448/1807941 [17:19<07:17, 517.50 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581503/1807941 [17:19<07:13, 522.45 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581560/1807941 [17:19<07:04, 533.03 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581614/1807941 [17:19<07:04, 532.86 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581669/1807941 [17:19<07:03, 533.95 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581723/1807941 [17:19<07:03, 534.50 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581777/1807941 [17:19<07:04, 532.83 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581831/1807941 [17:20<07:03, 533.89 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581887/1807941 [17:20<06:59, 539.06 examples/s]
Tokenizing train dataset:  87%|████████▋ | 1581945/1807941 [17:20<06:53, 547.09 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582000/1807941 [17:20<07:26, 505.91 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582060/1807941 [17:20<07:06, 529.10 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582117/1807941 [17:20<06:58, 539.95 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582175/1807941 [17:20<06:51, 548.93 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582233/1807941 [17:20<06:47, 553.72 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582290/1807941 [17:20<06:46, 555.32 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582372/1807941 [17:21<06:53, 545.51 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582428/1807941 [17:21<06:52, 546.11 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582483/1807941 [17:21<06:54, 543.90 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582539/1807941 [17:21<06:53, 545.27 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582594/1807941 [17:21<06:54, 544.19 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582650/1807941 [17:21<06:51, 547.70 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582707/1807941 [17:21<06:50, 548.87 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582763/1807941 [17:21<06:50, 548.21 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582819/1807941 [17:21<06:51, 547.32 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582877/1807941 [17:21<06:46, 553.99 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1582933/1807941 [17:22<06:46, 553.01 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583000/1807941 [17:22<07:38, 491.03 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583053/1807941 [17:22<07:30, 499.25 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583108/1807941 [17:22<07:21, 509.54 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583161/1807941 [17:22<07:19, 511.36 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583214/1807941 [17:22<07:17, 513.20 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583266/1807941 [17:22<07:17, 513.51 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583319/1807941 [17:22<07:15, 515.78 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583372/1807941 [17:22<07:13, 518.55 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583427/1807941 [17:23<07:07, 525.06 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583483/1807941 [17:23<07:02, 531.62 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583540/1807941 [17:23<06:55, 540.49 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583598/1807941 [17:23<06:50, 546.43 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583653/1807941 [17:23<06:50, 546.43 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583708/1807941 [17:23<06:52, 543.82 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583789/1807941 [17:23<06:54, 540.59 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583870/1807941 [17:23<06:58, 535.44 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583925/1807941 [17:23<06:58, 534.83 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1583979/1807941 [17:24<06:58, 534.80 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584054/1807941 [17:24<07:38, 488.71 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584108/1807941 [17:24<07:29, 497.54 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584162/1807941 [17:24<07:22, 506.05 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584218/1807941 [17:24<07:11, 518.80 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584276/1807941 [17:24<06:59, 533.41 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584333/1807941 [17:24<06:52, 541.52 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584392/1807941 [17:24<06:44, 553.30 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584453/1807941 [17:24<06:34, 566.68 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584537/1807941 [17:25<06:38, 560.53 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584619/1807941 [17:25<06:43, 553.13 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584675/1807941 [17:25<06:44, 552.54 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584733/1807941 [17:25<06:40, 557.63 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584791/1807941 [17:25<06:37, 561.26 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584849/1807941 [17:25<06:36, 563.08 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584909/1807941 [17:25<06:31, 570.00 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1584994/1807941 [17:25<06:33, 566.59 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585056/1807941 [17:26<07:16, 510.12 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585110/1807941 [17:26<07:11, 515.98 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585165/1807941 [17:26<07:06, 522.33 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585219/1807941 [17:26<07:04, 524.95 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585295/1807941 [17:26<07:11, 516.42 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585349/1807941 [17:26<07:07, 520.31 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585402/1807941 [17:26<07:06, 521.77 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585480/1807941 [17:26<07:10, 516.66 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585556/1807941 [17:27<07:15, 510.41 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585609/1807941 [17:27<07:13, 513.34 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585662/1807941 [17:27<07:11, 514.70 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585716/1807941 [17:27<07:09, 517.48 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585771/1807941 [17:27<07:04, 523.72 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585826/1807941 [17:27<06:59, 529.08 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585884/1807941 [17:27<06:51, 540.27 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1585941/1807941 [17:27<06:47, 545.29 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586000/1807941 [17:27<07:45, 476.41 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586053/1807941 [17:28<07:36, 486.26 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586106/1807941 [17:28<07:26, 497.12 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586160/1807941 [17:28<07:17, 506.50 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586215/1807941 [17:28<07:10, 515.27 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586268/1807941 [17:28<07:09, 516.38 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586323/1807941 [17:28<07:03, 523.40 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586378/1807941 [17:28<06:58, 528.87 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586450/1807941 [17:28<07:18, 504.55 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586523/1807941 [17:28<07:27, 495.00 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586579/1807941 [17:29<07:15, 508.00 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586638/1807941 [17:29<06:59, 527.19 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586711/1807941 [17:29<07:14, 509.64 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586779/1807941 [17:29<07:31, 489.48 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586850/1807941 [17:29<07:39, 481.07 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586920/1807941 [17:29<07:46, 474.08 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1586968/1807941 [17:29<07:47, 472.77 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587025/1807941 [17:29<08:25, 437.27 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587074/1807941 [17:30<08:14, 446.36 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587126/1807941 [17:30<07:57, 462.89 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587195/1807941 [17:30<08:00, 459.07 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587264/1807941 [17:30<08:01, 458.11 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587311/1807941 [17:30<07:59, 459.98 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587361/1807941 [17:30<07:52, 466.45 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587411/1807941 [17:30<07:46, 472.30 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587460/1807941 [17:30<07:44, 474.29 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587520/1807941 [17:31<07:14, 507.35 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587574/1807941 [17:31<07:06, 516.46 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587626/1807941 [17:31<07:09, 512.94 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587678/1807941 [17:31<07:08, 513.89 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587756/1807941 [17:31<07:09, 513.11 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587833/1807941 [17:31<07:12, 509.47 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587889/1807941 [17:31<07:03, 520.09 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1587946/1807941 [17:31<06:55, 529.86 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588000/1807941 [17:31<07:34, 484.26 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588060/1807941 [17:32<07:09, 511.73 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588123/1807941 [17:32<06:45, 541.79 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588205/1807941 [17:32<06:47, 538.74 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588286/1807941 [17:32<06:49, 536.51 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588342/1807941 [17:32<06:46, 540.17 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588399/1807941 [17:32<06:42, 546.00 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588456/1807941 [17:32<06:39, 549.05 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588535/1807941 [17:32<06:46, 539.77 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588613/1807941 [17:33<06:53, 530.47 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588690/1807941 [17:33<07:01, 520.62 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588765/1807941 [17:33<07:08, 511.67 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588841/1807941 [17:33<07:12, 506.07 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588913/1807941 [17:33<07:21, 495.74 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1588986/1807941 [17:33<07:28, 488.59 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589047/1807941 [17:34<08:12, 444.07 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589096/1807941 [17:34<08:03, 452.27 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589144/1807941 [17:34<08:00, 455.74 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589193/1807941 [17:34<07:52, 462.54 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589241/1807941 [17:34<07:50, 464.50 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589290/1807941 [17:34<07:48, 467.19 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589342/1807941 [17:34<07:36, 479.20 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589393/1807941 [17:34<07:30, 485.49 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589442/1807941 [17:34<07:32, 482.94 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589513/1807941 [17:34<07:40, 474.59 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589563/1807941 [17:35<07:36, 478.51 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589612/1807941 [17:35<07:36, 478.49 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589664/1807941 [17:35<07:28, 486.51 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589714/1807941 [17:35<07:27, 487.58 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589786/1807941 [17:35<07:32, 482.59 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589835/1807941 [17:35<07:30, 483.76 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589886/1807941 [17:35<07:24, 490.25 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1589952/1807941 [17:35<07:45, 468.30 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590000/1807941 [17:36<08:31, 426.05 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590051/1807941 [17:36<08:10, 444.46 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590103/1807941 [17:36<07:52, 460.63 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590156/1807941 [17:36<07:36, 477.00 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590207/1807941 [17:36<07:31, 482.70 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590263/1807941 [17:36<07:13, 501.58 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590316/1807941 [17:36<07:07, 509.43 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590369/1807941 [17:36<07:03, 513.54 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590421/1807941 [17:36<07:04, 512.07 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590475/1807941 [17:36<06:59, 518.38 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590531/1807941 [17:37<06:52, 526.67 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590599/1807941 [17:37<07:19, 494.26 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590665/1807941 [17:37<07:41, 470.56 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590728/1807941 [17:37<08:03, 449.56 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590779/1807941 [17:37<07:51, 460.83 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590840/1807941 [17:37<07:17, 495.67 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590897/1807941 [17:37<07:02, 513.37 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1590955/1807941 [17:37<06:50, 528.24 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591028/1807941 [17:38<07:26, 486.34 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591089/1807941 [17:38<07:01, 514.15 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591164/1807941 [17:38<07:10, 502.98 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591242/1807941 [17:38<07:08, 505.53 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591298/1807941 [17:38<06:59, 516.68 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591373/1807941 [17:38<07:06, 508.16 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591449/1807941 [17:38<07:10, 502.72 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591500/1807941 [17:39<07:12, 500.48 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591552/1807941 [17:39<07:12, 500.81 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591627/1807941 [17:39<07:14, 497.47 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591677/1807941 [17:39<07:16, 495.05 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591751/1807941 [17:39<07:19, 492.01 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591801/1807941 [17:39<07:20, 490.40 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591873/1807941 [17:39<07:29, 480.21 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1591942/1807941 [17:39<07:38, 471.08 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592000/1807941 [17:40<08:21, 430.93 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592046/1807941 [17:40<08:15, 435.90 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592093/1807941 [17:40<08:09, 440.90 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592141/1807941 [17:40<08:00, 449.31 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592191/1807941 [17:40<07:51, 457.91 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592239/1807941 [17:40<07:46, 462.29 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592293/1807941 [17:40<07:27, 482.42 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592347/1807941 [17:40<07:12, 498.17 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592420/1807941 [17:40<07:18, 491.05 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592494/1807941 [17:41<07:20, 489.50 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592564/1807941 [17:41<07:28, 480.70 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592631/1807941 [17:41<07:40, 467.08 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592679/1807941 [17:41<07:39, 468.38 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592728/1807941 [17:41<07:35, 472.03 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592777/1807941 [17:41<07:34, 473.52 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592825/1807941 [17:41<07:32, 475.00 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592874/1807941 [17:41<07:32, 474.88 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592924/1807941 [17:42<07:28, 479.37 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1592975/1807941 [17:42<07:23, 485.02 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593029/1807941 [17:42<08:14, 434.49 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593076/1807941 [17:42<08:07, 441.20 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593124/1807941 [17:42<07:57, 450.22 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593174/1807941 [17:42<07:46, 460.81 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593224/1807941 [17:42<07:39, 467.42 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593274/1807941 [17:42<07:32, 473.97 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593325/1807941 [17:42<07:24, 482.41 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593376/1807941 [17:43<07:18, 488.82 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593430/1807941 [17:43<07:07, 501.44 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593484/1807941 [17:43<07:00, 510.22 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593539/1807941 [17:43<06:53, 517.89 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593592/1807941 [17:43<06:55, 516.31 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593648/1807941 [17:43<06:46, 527.14 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593703/1807941 [17:43<06:42, 531.92 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593758/1807941 [17:43<06:40, 534.16 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593812/1807941 [17:43<06:43, 531.29 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593887/1807941 [17:43<06:55, 515.79 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1593964/1807941 [17:44<06:59, 510.68 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594025/1807941 [17:44<07:35, 470.14 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594097/1807941 [17:44<07:35, 469.56 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594148/1807941 [17:44<07:27, 477.83 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594199/1807941 [17:44<07:22, 483.17 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594250/1807941 [17:44<07:19, 486.59 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594320/1807941 [17:44<07:28, 476.31 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594369/1807941 [17:45<07:25, 479.25 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594424/1807941 [17:45<07:10, 495.93 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594494/1807941 [17:45<07:25, 479.59 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594563/1807941 [17:45<07:35, 468.63 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594618/1807941 [17:45<07:18, 486.76 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594691/1807941 [17:45<07:23, 481.35 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594752/1807941 [17:45<07:50, 453.18 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594801/1807941 [17:45<07:43, 459.42 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594854/1807941 [17:46<07:28, 475.52 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594904/1807941 [17:46<07:24, 478.84 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1594954/1807941 [17:46<07:20, 483.68 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595024/1807941 [17:46<08:05, 438.26 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595072/1807941 [17:46<07:55, 447.89 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595142/1807941 [17:46<07:51, 451.03 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595189/1807941 [17:46<07:50, 451.88 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595240/1807941 [17:46<07:38, 463.45 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595309/1807941 [17:47<07:42, 459.67 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595359/1807941 [17:47<07:34, 467.43 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595409/1807941 [17:47<07:27, 474.52 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595459/1807941 [17:47<07:24, 478.02 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595509/1807941 [17:47<07:22, 480.40 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595564/1807941 [17:47<07:07, 496.51 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595616/1807941 [17:47<07:02, 502.15 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595667/1807941 [17:47<07:05, 499.29 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595731/1807941 [17:47<07:33, 467.97 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595800/1807941 [17:48<07:40, 460.21 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595860/1807941 [17:48<08:05, 436.97 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595924/1807941 [17:48<08:11, 431.74 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1595968/1807941 [17:48<08:13, 429.82 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596024/1807941 [17:48<09:01, 391.06 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596071/1807941 [17:48<08:39, 407.93 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596120/1807941 [17:48<08:17, 426.03 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596166/1807941 [17:48<08:09, 432.36 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596213/1807941 [17:49<08:00, 441.08 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596261/1807941 [17:49<07:50, 450.21 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596308/1807941 [17:49<07:44, 455.14 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596357/1807941 [17:49<07:38, 461.11 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596405/1807941 [17:49<07:36, 463.66 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596454/1807941 [17:49<07:31, 468.39 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596503/1807941 [17:49<07:27, 472.35 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596554/1807941 [17:49<07:21, 478.61 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596604/1807941 [17:49<07:19, 481.12 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596653/1807941 [17:49<07:20, 479.66 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596702/1807941 [17:50<07:21, 478.13 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596775/1807941 [17:50<07:21, 478.78 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596825/1807941 [17:50<07:18, 481.57 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596896/1807941 [17:50<07:24, 474.66 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1596965/1807941 [17:50<07:32, 466.36 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597024/1807941 [17:50<08:10, 430.17 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597070/1807941 [17:50<08:06, 433.39 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597134/1807941 [17:51<08:11, 429.11 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597199/1807941 [17:51<08:12, 428.13 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597247/1807941 [17:51<07:59, 439.23 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597295/1807941 [17:51<07:51, 446.49 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597341/1807941 [17:51<07:50, 447.60 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597389/1807941 [17:51<07:42, 455.00 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597437/1807941 [17:51<07:37, 460.26 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597487/1807941 [17:51<07:29, 467.74 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597534/1807941 [17:51<07:30, 466.77 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597581/1807941 [17:52<07:31, 466.33 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597628/1807941 [17:52<07:32, 464.28 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597677/1807941 [17:52<07:29, 468.27 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597725/1807941 [17:52<07:27, 469.41 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597774/1807941 [17:52<07:25, 471.61 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597839/1807941 [17:52<07:44, 452.17 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597906/1807941 [17:52<07:51, 445.44 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1597954/1807941 [17:52<07:45, 450.81 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598001/1807941 [17:52<08:45, 399.54 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598059/1807941 [17:53<08:55, 391.95 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598102/1807941 [17:53<08:46, 398.58 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598148/1807941 [17:53<08:30, 411.18 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598195/1807941 [17:53<08:13, 424.88 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598240/1807941 [17:53<08:07, 430.07 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598290/1807941 [17:53<07:49, 447.01 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598340/1807941 [17:53<07:36, 458.78 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598390/1807941 [17:53<07:28, 467.69 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598443/1807941 [17:53<07:14, 482.11 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598497/1807941 [17:54<07:02, 495.56 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598553/1807941 [17:54<06:48, 513.07 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598607/1807941 [17:54<06:43, 519.37 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598660/1807941 [17:54<06:41, 520.90 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598736/1807941 [17:54<06:47, 513.47 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598788/1807941 [17:54<06:47, 513.51 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598860/1807941 [17:54<06:59, 498.64 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598913/1807941 [17:54<06:53, 505.35 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1598964/1807941 [17:54<06:54, 504.48 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599030/1807941 [17:55<07:16, 478.14 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599084/1807941 [17:55<07:03, 492.87 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599140/1807941 [17:55<06:51, 507.64 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599199/1807941 [17:55<06:35, 527.84 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599253/1807941 [17:55<06:33, 530.77 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599309/1807941 [17:55<06:30, 534.68 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599365/1807941 [17:55<06:28, 536.63 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599421/1807941 [17:55<06:26, 538.90 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599476/1807941 [17:55<06:26, 538.88 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599533/1807941 [17:56<06:23, 544.05 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599616/1807941 [17:56<06:22, 544.65 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599671/1807941 [17:56<06:24, 542.18 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599726/1807941 [17:56<06:24, 541.44 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599784/1807941 [17:56<06:18, 549.32 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599841/1807941 [17:56<06:16, 553.25 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599897/1807941 [17:56<06:15, 553.71 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1599953/1807941 [17:56<06:16, 552.11 examples/s]
Tokenizing train dataset:  88%|████████▊ | 1600025/1807941 [17:56<07:10, 483.07 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600099/1807941 [17:57<07:10, 482.89 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600149/1807941 [17:57<07:08, 484.86 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600199/1807941 [17:57<07:07, 486.00 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600273/1807941 [17:57<07:07, 485.28 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600345/1807941 [17:57<07:12, 479.73 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600396/1807941 [17:57<07:08, 483.99 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600466/1807941 [17:57<07:18, 473.60 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600514/1807941 [17:58<07:19, 471.96 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600583/1807941 [17:58<07:26, 463.98 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600631/1807941 [17:58<07:26, 464.31 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600679/1807941 [17:58<07:23, 466.86 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600726/1807941 [17:58<07:26, 464.43 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600774/1807941 [17:58<07:25, 465.49 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600821/1807941 [17:58<07:25, 465.18 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600869/1807941 [17:58<07:23, 466.75 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600917/1807941 [17:58<07:23, 466.54 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1600967/1807941 [17:58<07:18, 471.75 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601024/1807941 [17:59<08:04, 427.15 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601073/1807941 [17:59<07:48, 441.49 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601122/1807941 [17:59<07:38, 451.51 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601168/1807941 [17:59<07:37, 452.02 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601216/1807941 [17:59<07:30, 458.65 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601265/1807941 [17:59<07:24, 465.34 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601315/1807941 [17:59<07:15, 473.99 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601366/1807941 [17:59<07:09, 481.41 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601417/1807941 [17:59<07:04, 486.36 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601470/1807941 [18:00<06:55, 496.68 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601523/1807941 [18:00<06:49, 504.52 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601577/1807941 [18:00<06:42, 512.40 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601633/1807941 [18:00<06:33, 524.48 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601704/1807941 [18:00<06:52, 500.45 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601757/1807941 [18:00<06:48, 504.63 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601829/1807941 [18:00<07:00, 489.60 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601886/1807941 [18:00<06:47, 506.04 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1601961/1807941 [18:01<06:52, 499.74 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602027/1807941 [18:01<07:27, 460.51 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602078/1807941 [18:01<07:17, 471.00 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602134/1807941 [18:01<06:58, 491.33 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602191/1807941 [18:01<06:43, 510.22 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602246/1807941 [18:01<06:35, 520.04 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602301/1807941 [18:01<06:31, 525.17 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602380/1807941 [18:01<06:32, 523.59 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602454/1807941 [18:02<06:46, 505.76 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602529/1807941 [18:02<06:49, 501.46 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602580/1807941 [18:02<06:51, 499.56 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602656/1807941 [18:02<06:51, 498.91 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602711/1807941 [18:02<06:42, 509.67 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602763/1807941 [18:02<06:41, 511.10 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602816/1807941 [18:02<06:37, 515.74 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602893/1807941 [18:02<06:41, 511.00 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602946/1807941 [18:02<06:38, 514.48 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1602999/1807941 [18:03<06:39, 513.39 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603055/1807941 [18:03<07:23, 461.70 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603107/1807941 [18:03<07:11, 474.91 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603159/1807941 [18:03<07:03, 483.74 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603217/1807941 [18:03<06:45, 504.87 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603292/1807941 [18:03<06:51, 497.52 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603344/1807941 [18:03<06:49, 499.50 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603416/1807941 [18:03<06:55, 491.65 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603467/1807941 [18:04<06:55, 492.26 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603518/1807941 [18:04<06:54, 492.70 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603572/1807941 [18:04<06:47, 501.88 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603644/1807941 [18:04<06:55, 491.21 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603695/1807941 [18:04<06:55, 491.58 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603767/1807941 [18:04<07:01, 484.69 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603841/1807941 [18:04<07:02, 483.52 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603890/1807941 [18:04<07:02, 482.59 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603943/1807941 [18:05<06:55, 490.87 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1603993/1807941 [18:05<06:56, 489.14 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1604051/1807941 [18:05<07:38, 444.71 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1604099/1807941 [18:05<07:31, 451.12 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1604148/1807941 [18:05<07:25, 457.66 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1604199/1807941 [18:05<07:14, 468.53 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1604252/1807941 [18:05<07:03, 481.42 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1604305/1807941 [18:05<06:53, 492.25 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1604356/1807941 [18:05<06:51, 494.68 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1604430/1807941 [18:06<06:56, 488.85 examples/s]
Tokenizing train dataset:  89%|████████▊ | 1604504/1807941 [18:06<06:56, 488.73 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1604557/1807941 [18:06<06:49, 496.53 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1604629/1807941 [18:06<06:55, 488.84 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1604679/1807941 [18:06<06:58, 486.17 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1604729/1807941 [18:06<06:56, 488.32 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1604779/1807941 [18:06<06:54, 490.10 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1604829/1807941 [18:06<06:53, 490.97 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1604902/1807941 [18:07<06:57, 486.11 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1604976/1807941 [18:07<06:58, 485.34 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605027/1807941 [18:07<07:29, 451.73 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605078/1807941 [18:07<07:16, 464.84 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605131/1807941 [18:07<07:03, 478.52 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605185/1807941 [18:07<06:50, 493.55 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605240/1807941 [18:07<06:41, 505.26 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605292/1807941 [18:07<06:40, 506.44 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605365/1807941 [18:07<06:48, 496.18 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605419/1807941 [18:08<06:39, 506.72 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605496/1807941 [18:08<06:40, 505.74 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605547/1807941 [18:08<06:42, 503.14 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605601/1807941 [18:08<06:36, 510.59 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605655/1807941 [18:08<06:31, 516.39 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605707/1807941 [18:08<06:33, 514.53 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605783/1807941 [18:08<06:38, 507.22 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605855/1807941 [18:08<06:47, 495.43 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605906/1807941 [18:09<06:47, 495.19 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1605960/1807941 [18:09<06:40, 504.35 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606027/1807941 [18:09<07:09, 470.02 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606080/1807941 [18:09<06:57, 483.34 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606132/1807941 [18:09<06:51, 490.10 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606187/1807941 [18:09<06:40, 503.64 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606243/1807941 [18:09<06:30, 515.88 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606297/1807941 [18:09<06:27, 520.38 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606375/1807941 [18:09<06:30, 516.83 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606428/1807941 [18:10<06:29, 517.54 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606482/1807941 [18:10<06:27, 520.14 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606536/1807941 [18:10<06:27, 520.40 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606591/1807941 [18:10<06:23, 525.05 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606645/1807941 [18:10<06:22, 526.16 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606700/1807941 [18:10<06:19, 530.72 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606754/1807941 [18:10<06:20, 528.70 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606808/1807941 [18:10<06:19, 530.05 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606862/1807941 [18:10<06:20, 528.78 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606918/1807941 [18:10<06:16, 534.34 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1606976/1807941 [18:11<06:09, 543.64 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607056/1807941 [18:11<06:44, 496.18 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607113/1807941 [18:11<06:32, 511.95 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607171/1807941 [18:11<06:20, 527.77 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607231/1807941 [18:11<06:08, 545.01 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607291/1807941 [18:11<06:00, 557.25 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607350/1807941 [18:11<05:55, 564.35 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607411/1807941 [18:11<05:49, 573.03 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607472/1807941 [18:11<05:45, 579.63 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607558/1807941 [18:12<05:50, 572.13 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607618/1807941 [18:12<05:47, 576.66 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607679/1807941 [18:12<05:44, 582.15 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607765/1807941 [18:12<05:49, 572.62 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607823/1807941 [18:12<05:51, 569.71 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607887/1807941 [18:12<05:41, 585.42 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1607951/1807941 [18:12<05:35, 596.11 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608030/1807941 [18:12<06:11, 538.10 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608090/1807941 [18:13<06:02, 551.08 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608148/1807941 [18:13<05:58, 557.03 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608211/1807941 [18:13<05:48, 572.97 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608272/1807941 [18:13<05:43, 581.43 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608331/1807941 [18:13<05:44, 579.76 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608394/1807941 [18:13<05:37, 591.06 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608484/1807941 [18:13<05:36, 592.39 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608572/1807941 [18:13<05:40, 586.20 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608634/1807941 [18:14<05:36, 593.11 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608695/1807941 [18:14<05:34, 594.95 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608755/1807941 [18:14<05:35, 594.29 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608837/1807941 [18:14<05:46, 574.09 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608915/1807941 [18:14<05:59, 553.96 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1608996/1807941 [18:14<06:04, 545.35 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609058/1807941 [18:14<06:38, 499.61 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609111/1807941 [18:14<06:32, 506.01 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609165/1807941 [18:15<06:27, 512.58 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609217/1807941 [18:15<06:28, 512.17 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609273/1807941 [18:15<06:20, 521.74 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609328/1807941 [18:15<06:18, 525.09 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609381/1807941 [18:15<06:19, 522.95 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609456/1807941 [18:15<06:29, 509.87 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609510/1807941 [18:15<06:24, 515.69 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609585/1807941 [18:15<06:32, 504.81 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609658/1807941 [18:15<06:39, 495.78 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609732/1807941 [18:16<06:42, 492.12 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609783/1807941 [18:16<06:40, 494.42 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609834/1807941 [18:16<06:39, 495.61 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609885/1807941 [18:16<06:39, 495.89 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1609938/1807941 [18:16<06:34, 501.48 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610000/1807941 [18:16<07:15, 454.12 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610050/1807941 [18:16<07:06, 463.76 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610098/1807941 [18:16<07:04, 466.20 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610148/1807941 [18:17<06:58, 472.53 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610201/1807941 [18:17<06:47, 485.29 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610254/1807941 [18:17<06:38, 495.86 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610307/1807941 [18:17<06:32, 503.56 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610360/1807941 [18:17<06:29, 507.02 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610413/1807941 [18:17<06:27, 510.38 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610468/1807941 [18:17<06:20, 518.65 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610523/1807941 [18:17<06:16, 524.78 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610577/1807941 [18:17<06:13, 528.02 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610655/1807941 [18:17<06:17, 523.18 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610732/1807941 [18:18<06:20, 517.82 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610788/1807941 [18:18<06:14, 525.93 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610842/1807941 [18:18<06:12, 528.80 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610899/1807941 [18:18<06:07, 535.84 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1610957/1807941 [18:18<06:00, 545.83 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611027/1807941 [18:18<06:44, 486.77 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611078/1807941 [18:18<06:41, 490.01 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611153/1807941 [18:18<06:41, 490.00 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611204/1807941 [18:19<06:40, 491.52 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611256/1807941 [18:19<06:34, 498.65 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611310/1807941 [18:19<06:29, 505.41 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611385/1807941 [18:19<06:32, 500.48 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611436/1807941 [18:19<06:32, 500.28 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611490/1807941 [18:19<06:24, 510.64 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611544/1807941 [18:19<06:21, 514.41 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611599/1807941 [18:19<06:17, 520.46 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611653/1807941 [18:19<06:14, 523.85 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611729/1807941 [18:20<06:23, 511.26 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611783/1807941 [18:20<06:19, 517.19 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611839/1807941 [18:20<06:13, 525.58 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611894/1807941 [18:20<06:11, 527.66 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1611947/1807941 [18:20<06:12, 526.47 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612000/1807941 [18:20<06:56, 470.33 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612050/1807941 [18:20<06:51, 476.41 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612121/1807941 [18:20<06:54, 472.08 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612171/1807941 [18:21<06:50, 477.30 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612222/1807941 [18:21<06:44, 483.56 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612273/1807941 [18:21<06:40, 488.24 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612325/1807941 [18:21<06:35, 495.22 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612376/1807941 [18:21<06:33, 497.22 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612452/1807941 [18:21<06:33, 496.84 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612502/1807941 [18:21<06:35, 493.91 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612578/1807941 [18:21<06:35, 494.33 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612634/1807941 [18:21<06:23, 509.30 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612687/1807941 [18:22<06:22, 510.48 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612756/1807941 [18:22<06:37, 490.66 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612831/1807941 [18:22<06:37, 490.56 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612901/1807941 [18:22<06:46, 479.39 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1612953/1807941 [18:22<06:41, 485.71 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613024/1807941 [18:22<07:19, 443.53 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613073/1807941 [18:22<07:12, 450.90 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613123/1807941 [18:22<07:03, 459.85 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613176/1807941 [18:23<06:49, 475.90 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613229/1807941 [18:23<06:39, 487.97 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613280/1807941 [18:23<06:36, 490.79 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613334/1807941 [18:23<06:26, 503.10 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613403/1807941 [18:23<06:43, 482.64 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613471/1807941 [18:23<06:55, 468.03 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613539/1807941 [18:23<07:04, 457.87 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613586/1807941 [18:23<07:05, 457.05 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613655/1807941 [18:24<07:07, 454.65 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613702/1807941 [18:24<07:07, 454.87 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613769/1807941 [18:24<07:11, 449.65 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613815/1807941 [18:24<07:11, 449.71 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613861/1807941 [18:24<07:10, 451.12 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613908/1807941 [18:24<07:05, 455.66 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1613957/1807941 [18:24<07:01, 460.70 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614022/1807941 [18:24<07:49, 412.69 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614066/1807941 [18:25<07:42, 419.09 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614112/1807941 [18:25<07:33, 427.51 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614161/1807941 [18:25<07:20, 439.85 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614209/1807941 [18:25<07:12, 447.74 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614256/1807941 [18:25<07:08, 452.21 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614304/1807941 [18:25<07:04, 456.23 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614351/1807941 [18:25<07:04, 456.09 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614418/1807941 [18:25<07:11, 448.89 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614487/1807941 [18:25<07:10, 449.09 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614554/1807941 [18:26<07:14, 444.84 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614600/1807941 [18:26<07:14, 444.85 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614646/1807941 [18:26<07:14, 445.09 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614712/1807941 [18:26<07:18, 440.22 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614757/1807941 [18:26<07:17, 441.38 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614822/1807941 [18:26<07:23, 435.67 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614868/1807941 [18:26<07:18, 439.93 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614913/1807941 [18:26<07:17, 440.75 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1614981/1807941 [18:27<07:18, 440.28 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615047/1807941 [18:27<08:04, 398.00 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615094/1807941 [18:27<07:46, 413.17 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615139/1807941 [18:27<07:38, 420.14 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615187/1807941 [18:27<07:25, 432.84 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615234/1807941 [18:27<07:17, 440.22 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615282/1807941 [18:27<07:07, 450.21 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615348/1807941 [18:27<07:14, 442.81 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615394/1807941 [18:28<07:13, 444.68 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615441/1807941 [18:28<07:09, 447.98 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615493/1807941 [18:28<06:52, 466.19 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615543/1807941 [18:28<06:45, 474.13 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615592/1807941 [18:28<06:45, 474.54 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615642/1807941 [18:28<06:42, 477.79 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615691/1807941 [18:28<06:41, 478.83 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615742/1807941 [18:28<06:36, 484.96 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615792/1807941 [18:28<06:33, 488.26 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615842/1807941 [18:28<06:34, 487.37 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615891/1807941 [18:29<06:34, 486.54 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615944/1807941 [18:29<06:27, 495.57 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1615997/1807941 [18:29<06:21, 503.55 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616050/1807941 [18:29<07:18, 438.09 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616101/1807941 [18:29<06:59, 456.94 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616153/1807941 [18:29<06:47, 470.67 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616204/1807941 [18:29<06:39, 480.09 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616257/1807941 [18:29<06:30, 490.56 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616307/1807941 [18:29<06:29, 492.29 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616360/1807941 [18:30<06:23, 499.97 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616415/1807941 [18:30<06:14, 511.33 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616469/1807941 [18:30<06:11, 515.63 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616525/1807941 [18:30<06:05, 524.41 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616603/1807941 [18:30<06:08, 519.31 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616659/1807941 [18:30<06:02, 527.75 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616715/1807941 [18:30<05:58, 533.13 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616770/1807941 [18:30<05:56, 535.88 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616828/1807941 [18:30<05:49, 546.87 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616888/1807941 [18:31<05:41, 559.01 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1616948/1807941 [18:31<05:36, 568.26 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617030/1807941 [18:31<06:11, 513.81 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617086/1807941 [18:31<06:04, 524.28 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617146/1807941 [18:31<05:51, 542.47 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617207/1807941 [18:31<05:41, 558.29 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617266/1807941 [18:31<05:39, 562.41 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617347/1807941 [18:31<05:45, 551.54 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617426/1807941 [18:32<05:52, 540.01 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617481/1807941 [18:32<05:53, 539.36 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617539/1807941 [18:32<05:46, 548.86 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617618/1807941 [18:32<05:54, 536.35 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617694/1807941 [18:32<06:03, 523.01 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617772/1807941 [18:32<06:08, 516.13 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617847/1807941 [18:32<06:13, 509.42 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617921/1807941 [18:32<06:19, 501.03 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1617973/1807941 [18:33<06:16, 504.46 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1618032/1807941 [18:33<06:49, 463.92 examples/s]
Tokenizing train dataset:  89%|████████▉ | 1618083/1807941 [18:33<06:41, 473.29 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618134/1807941 [18:33<06:34, 481.18 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618205/1807941 [18:33<06:38, 476.34 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618278/1807941 [18:33<06:38, 475.71 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618330/1807941 [18:33<06:31, 484.06 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618382/1807941 [18:33<06:26, 489.88 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618432/1807941 [18:34<06:27, 488.92 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618503/1807941 [18:34<06:35, 478.65 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618575/1807941 [18:34<06:36, 477.41 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618646/1807941 [18:34<06:40, 473.13 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618697/1807941 [18:34<06:35, 478.46 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618747/1807941 [18:34<06:32, 481.43 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618797/1807941 [18:34<06:31, 483.60 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618847/1807941 [18:34<06:27, 487.49 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618898/1807941 [18:35<06:24, 491.98 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1618949/1807941 [18:35<06:24, 491.59 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619001/1807941 [18:35<07:17, 431.97 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619050/1807941 [18:35<07:03, 446.46 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619100/1807941 [18:35<06:51, 459.34 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619153/1807941 [18:35<06:36, 476.55 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619205/1807941 [18:35<06:28, 486.29 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619259/1807941 [18:35<06:17, 499.95 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619312/1807941 [18:35<06:12, 505.80 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619379/1807941 [18:36<06:35, 476.75 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619433/1807941 [18:36<06:25, 489.05 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619507/1807941 [18:36<06:25, 488.72 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619578/1807941 [18:36<06:30, 482.00 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619630/1807941 [18:36<06:26, 487.09 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619705/1807941 [18:36<06:27, 486.20 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619777/1807941 [18:36<06:31, 481.07 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619847/1807941 [18:37<06:36, 474.12 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619918/1807941 [18:37<06:39, 470.92 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1619967/1807941 [18:37<06:37, 473.09 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620023/1807941 [18:37<07:18, 428.18 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620070/1807941 [18:37<07:12, 434.11 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620117/1807941 [18:37<07:06, 440.86 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620163/1807941 [18:37<07:02, 444.53 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620212/1807941 [18:37<06:54, 452.86 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620267/1807941 [18:37<06:32, 478.70 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620321/1807941 [18:38<06:19, 493.77 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620375/1807941 [18:38<06:12, 503.29 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620427/1807941 [18:38<06:11, 505.33 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620482/1807941 [18:38<06:04, 514.46 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620534/1807941 [18:38<06:05, 512.77 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620610/1807941 [18:38<06:10, 505.79 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620661/1807941 [18:38<06:12, 503.29 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620735/1807941 [18:38<06:19, 493.77 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620788/1807941 [18:38<06:13, 501.22 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620862/1807941 [18:39<06:16, 497.08 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620915/1807941 [18:39<06:10, 504.71 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1620966/1807941 [18:39<06:11, 503.44 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621024/1807941 [18:39<07:04, 440.62 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621072/1807941 [18:39<06:58, 446.69 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621119/1807941 [18:39<06:53, 451.55 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621168/1807941 [18:39<06:48, 457.55 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621217/1807941 [18:39<06:42, 464.03 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621266/1807941 [18:40<06:38, 468.45 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621314/1807941 [18:40<06:37, 469.25 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621363/1807941 [18:40<06:35, 471.86 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621415/1807941 [18:40<06:25, 484.10 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621469/1807941 [18:40<06:14, 497.99 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621543/1807941 [18:40<06:19, 491.17 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621616/1807941 [18:40<06:24, 484.51 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621688/1807941 [18:40<06:27, 480.32 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621738/1807941 [18:40<06:27, 480.70 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621787/1807941 [18:41<06:28, 479.66 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621860/1807941 [18:41<06:28, 479.46 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1621931/1807941 [18:41<06:33, 473.04 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622000/1807941 [18:41<07:12, 430.33 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622048/1807941 [18:41<07:01, 440.71 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622102/1807941 [18:41<06:41, 462.70 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622152/1807941 [18:41<06:33, 471.70 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622206/1807941 [18:41<06:20, 488.39 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622262/1807941 [18:42<06:08, 504.15 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622321/1807941 [18:42<05:52, 526.27 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622378/1807941 [18:42<05:47, 534.60 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622459/1807941 [18:42<05:47, 533.81 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622520/1807941 [18:42<05:37, 550.14 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622576/1807941 [18:42<05:37, 549.78 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622654/1807941 [18:42<05:46, 535.36 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622728/1807941 [18:42<05:56, 519.85 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622801/1807941 [18:43<06:07, 504.24 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622876/1807941 [18:43<06:09, 500.93 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1622950/1807941 [18:43<06:13, 495.38 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623000/1807941 [18:43<06:44, 457.47 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623049/1807941 [18:43<06:40, 461.61 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623120/1807941 [18:43<06:39, 463.17 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623171/1807941 [18:43<06:31, 472.47 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623219/1807941 [18:44<06:32, 471.12 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623267/1807941 [18:44<06:31, 471.57 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623317/1807941 [18:44<06:27, 476.38 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623368/1807941 [18:44<06:23, 481.19 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623423/1807941 [18:44<06:10, 498.54 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623478/1807941 [18:44<06:00, 511.24 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623533/1807941 [18:44<05:54, 519.85 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623588/1807941 [18:44<05:50, 525.79 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623663/1807941 [18:44<06:00, 511.65 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623737/1807941 [18:45<06:06, 502.24 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623796/1807941 [18:45<05:52, 521.69 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623875/1807941 [18:45<05:53, 520.16 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1623930/1807941 [18:45<05:51, 523.92 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624000/1807941 [18:45<06:31, 470.14 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624071/1807941 [18:45<06:32, 468.02 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624119/1807941 [18:45<06:31, 469.20 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624168/1807941 [18:45<06:28, 472.48 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624217/1807941 [18:46<06:26, 475.51 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624266/1807941 [18:46<06:23, 478.49 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624315/1807941 [18:46<06:21, 480.74 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624365/1807941 [18:46<06:21, 481.16 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624415/1807941 [18:46<06:20, 481.80 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624465/1807941 [18:46<06:19, 483.91 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624514/1807941 [18:46<06:20, 482.51 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624564/1807941 [18:46<06:19, 483.25 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624615/1807941 [18:46<06:15, 487.66 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624685/1807941 [18:47<06:26, 474.35 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624758/1807941 [18:47<06:26, 473.90 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624809/1807941 [18:47<06:19, 481.95 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624863/1807941 [18:47<06:09, 495.49 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624922/1807941 [18:47<05:53, 517.79 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1624978/1807941 [18:47<05:47, 526.45 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625052/1807941 [18:47<06:22, 477.69 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625104/1807941 [18:47<06:15, 486.56 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625154/1807941 [18:47<06:15, 487.30 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625212/1807941 [18:48<05:57, 510.72 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625271/1807941 [18:48<05:45, 529.25 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625330/1807941 [18:48<05:35, 544.05 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625388/1807941 [18:48<05:30, 552.84 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625448/1807941 [18:48<05:23, 564.26 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625506/1807941 [18:48<05:21, 566.88 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625567/1807941 [18:48<05:15, 577.97 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625632/1807941 [18:48<05:05, 596.31 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625693/1807941 [18:48<05:05, 596.28 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625784/1807941 [18:49<05:05, 597.01 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625849/1807941 [18:49<04:59, 608.39 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625911/1807941 [18:49<04:57, 610.99 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1625999/1807941 [18:49<05:04, 598.01 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626086/1807941 [18:49<05:45, 527.10 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626168/1807941 [18:49<05:42, 531.01 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626223/1807941 [18:49<05:40, 533.63 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626303/1807941 [18:49<05:42, 530.73 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626357/1807941 [18:50<05:41, 531.15 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626434/1807941 [18:50<05:49, 520.07 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626487/1807941 [18:50<05:49, 518.56 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626562/1807941 [18:50<05:55, 510.32 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626639/1807941 [18:50<05:56, 508.78 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626691/1807941 [18:50<05:57, 507.18 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626744/1807941 [18:50<05:55, 509.86 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626802/1807941 [18:50<05:44, 526.46 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626860/1807941 [18:51<05:36, 538.76 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626919/1807941 [18:51<05:30, 548.43 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1626976/1807941 [18:51<05:28, 551.43 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1627051/1807941 [18:51<06:13, 484.52 examples/s]
Tokenizing train dataset:  90%|████████▉ | 1627107/1807941 [18:51<06:02, 498.76 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627161/1807941 [18:51<05:55, 507.95 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627233/1807941 [18:51<06:05, 494.62 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627289/1807941 [18:51<05:56, 506.96 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627369/1807941 [18:52<05:51, 513.60 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627423/1807941 [18:52<05:48, 518.25 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627476/1807941 [18:52<05:48, 518.38 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627529/1807941 [18:52<05:48, 518.40 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627584/1807941 [18:52<05:45, 522.43 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627638/1807941 [18:52<05:44, 523.92 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627693/1807941 [18:52<05:41, 527.39 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627748/1807941 [18:52<05:39, 531.09 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627805/1807941 [18:52<05:32, 541.24 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627860/1807941 [18:52<05:33, 540.60 examples/s]
Tokenizing train dataset:  90%|█████████ | 1627918/1807941 [18:53<05:27, 549.03 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628000/1807941 [18:53<06:04, 493.53 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628054/1807941 [18:53<05:57, 503.82 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628130/1807941 [18:53<05:58, 501.25 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628205/1807941 [18:53<06:00, 497.94 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628256/1807941 [18:53<06:01, 497.24 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628307/1807941 [18:53<06:01, 497.07 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628383/1807941 [18:54<06:02, 495.13 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628435/1807941 [18:54<05:58, 500.26 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628487/1807941 [18:54<05:57, 502.32 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628538/1807941 [18:54<05:58, 500.81 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628612/1807941 [18:54<06:02, 494.44 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628662/1807941 [18:54<06:01, 495.27 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628712/1807941 [18:54<06:01, 495.80 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628763/1807941 [18:54<06:01, 495.59 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628814/1807941 [18:54<06:01, 495.29 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628864/1807941 [18:55<06:01, 495.03 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628917/1807941 [18:55<05:55, 504.17 examples/s]
Tokenizing train dataset:  90%|█████████ | 1628969/1807941 [18:55<05:52, 507.45 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629027/1807941 [18:55<06:33, 455.03 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629076/1807941 [18:55<06:26, 462.42 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629132/1807941 [18:55<06:07, 486.35 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629183/1807941 [18:55<06:05, 489.22 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629237/1807941 [18:55<05:56, 501.62 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629292/1807941 [18:55<05:49, 510.54 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629365/1807941 [18:56<05:59, 496.56 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629416/1807941 [18:56<05:58, 498.07 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629493/1807941 [18:56<05:56, 500.95 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629549/1807941 [18:56<05:47, 513.40 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629603/1807941 [18:56<05:43, 519.42 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629656/1807941 [18:56<05:42, 519.93 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629710/1807941 [18:56<05:42, 520.96 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629764/1807941 [18:56<05:40, 523.43 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629839/1807941 [18:56<05:48, 511.42 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629913/1807941 [18:57<05:56, 499.06 examples/s]
Tokenizing train dataset:  90%|█████████ | 1629988/1807941 [18:57<05:59, 495.43 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630049/1807941 [18:57<06:34, 450.72 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630096/1807941 [18:57<06:32, 453.03 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630145/1807941 [18:57<06:27, 459.07 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630197/1807941 [18:57<06:14, 474.14 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630246/1807941 [18:57<06:12, 477.03 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630320/1807941 [18:57<06:10, 479.17 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630389/1807941 [18:58<06:16, 471.10 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630439/1807941 [18:58<06:14, 473.87 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630487/1807941 [18:58<06:17, 470.41 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630538/1807941 [18:58<06:11, 477.58 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630586/1807941 [18:58<06:13, 474.85 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630636/1807941 [18:58<06:09, 479.39 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630685/1807941 [18:58<06:08, 481.00 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630735/1807941 [18:58<06:06, 483.15 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630784/1807941 [18:58<06:08, 481.29 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630857/1807941 [18:59<06:09, 479.10 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630906/1807941 [18:59<06:10, 478.23 examples/s]
Tokenizing train dataset:  90%|█████████ | 1630956/1807941 [18:59<06:09, 479.40 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631025/1807941 [18:59<06:45, 436.01 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631074/1807941 [18:59<06:35, 447.59 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631123/1807941 [18:59<06:27, 456.41 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631175/1807941 [18:59<06:16, 469.99 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631227/1807941 [18:59<06:08, 480.08 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631279/1807941 [19:00<06:02, 487.02 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631332/1807941 [19:00<05:56, 495.11 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631382/1807941 [19:00<05:55, 495.99 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631458/1807941 [19:00<05:54, 498.43 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631533/1807941 [19:00<05:56, 494.28 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631584/1807941 [19:00<05:56, 494.05 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631634/1807941 [19:00<05:56, 494.52 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631684/1807941 [19:00<05:56, 493.99 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631739/1807941 [19:00<05:47, 506.72 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631815/1807941 [19:01<05:49, 503.72 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631888/1807941 [19:01<05:57, 492.54 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631938/1807941 [19:01<05:56, 493.16 examples/s]
Tokenizing train dataset:  90%|█████████ | 1631988/1807941 [19:01<05:57, 492.20 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632048/1807941 [19:01<06:34, 445.42 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632099/1807941 [19:01<06:23, 458.77 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632148/1807941 [19:01<06:18, 464.51 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632199/1807941 [19:01<06:11, 473.42 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632248/1807941 [19:02<06:11, 473.39 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632298/1807941 [19:02<06:06, 479.32 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632372/1807941 [19:02<06:06, 479.43 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632427/1807941 [19:02<05:55, 494.17 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632480/1807941 [19:02<05:51, 498.88 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632555/1807941 [19:02<05:54, 494.35 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632607/1807941 [19:02<05:50, 500.20 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632661/1807941 [19:02<05:45, 507.60 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632714/1807941 [19:02<05:44, 508.90 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632792/1807941 [19:03<05:44, 508.94 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632845/1807941 [19:03<05:40, 513.83 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632898/1807941 [19:03<05:38, 517.71 examples/s]
Tokenizing train dataset:  90%|█████████ | 1632951/1807941 [19:03<05:37, 519.13 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633026/1807941 [19:03<06:23, 456.36 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633077/1807941 [19:03<06:13, 467.60 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633129/1807941 [19:03<06:04, 479.29 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633182/1807941 [19:03<05:57, 489.00 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633234/1807941 [19:03<05:53, 494.49 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633287/1807941 [19:04<05:48, 501.67 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633362/1807941 [19:04<05:49, 499.31 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633414/1807941 [19:04<05:46, 503.21 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633467/1807941 [19:04<05:43, 507.25 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633519/1807941 [19:04<05:43, 507.84 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633574/1807941 [19:04<05:36, 518.66 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633628/1807941 [19:04<05:34, 521.79 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633681/1807941 [19:04<05:32, 523.51 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633736/1807941 [19:04<05:30, 526.67 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633792/1807941 [19:05<05:27, 532.51 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633849/1807941 [19:05<05:23, 538.74 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633907/1807941 [19:05<05:16, 549.81 examples/s]
Tokenizing train dataset:  90%|█████████ | 1633965/1807941 [19:05<05:11, 557.75 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634027/1807941 [19:05<05:51, 494.71 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634083/1807941 [19:05<05:41, 509.58 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634139/1807941 [19:05<05:33, 520.60 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634196/1807941 [19:05<05:25, 532.99 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634255/1807941 [19:05<05:18, 546.18 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634312/1807941 [19:06<05:15, 549.71 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634370/1807941 [19:06<05:13, 553.43 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634430/1807941 [19:06<05:08, 562.59 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634487/1807941 [19:06<05:07, 563.88 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634546/1807941 [19:06<05:04, 569.62 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634630/1807941 [19:06<05:07, 563.79 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634711/1807941 [19:06<05:14, 550.65 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634792/1807941 [19:06<05:18, 542.96 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634869/1807941 [19:07<05:26, 529.85 examples/s]
Tokenizing train dataset:  90%|█████████ | 1634949/1807941 [19:07<05:28, 526.86 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635026/1807941 [19:07<05:56, 484.97 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635079/1807941 [19:07<05:51, 492.02 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635130/1807941 [19:07<05:48, 495.31 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635183/1807941 [19:07<05:43, 503.18 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635238/1807941 [19:07<05:36, 513.57 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635290/1807941 [19:07<05:36, 512.83 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635342/1807941 [19:08<05:35, 513.85 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635395/1807941 [19:08<05:33, 517.43 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635450/1807941 [19:08<05:28, 525.10 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635508/1807941 [19:08<05:20, 538.83 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635566/1807941 [19:08<05:15, 546.90 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635621/1807941 [19:08<05:15, 545.85 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635680/1807941 [19:08<05:09, 556.30 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635742/1807941 [19:08<05:01, 571.37 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635802/1807941 [19:08<04:58, 576.99 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635863/1807941 [19:08<04:53, 585.37 examples/s]
Tokenizing train dataset:  90%|█████████ | 1635952/1807941 [19:09<04:54, 584.52 examples/s]
Tokenizing train dataset:  90%|█████████ | 1636031/1807941 [19:09<05:13, 548.79 examples/s]
Tokenizing train dataset:  90%|█████████ | 1636093/1807941 [19:09<05:04, 563.93 examples/s]
Tokenizing train dataset:  90%|█████████ | 1636155/1807941 [19:09<04:58, 575.27 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636215/1807941 [19:09<04:56, 579.42 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636277/1807941 [19:09<04:52, 587.48 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636341/1807941 [19:09<04:46, 599.34 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636403/1807941 [19:09<04:45, 600.15 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636489/1807941 [19:09<04:50, 589.22 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636575/1807941 [19:10<04:54, 581.63 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636660/1807941 [19:10<04:58, 573.74 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636745/1807941 [19:10<05:02, 566.72 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636829/1807941 [19:10<05:05, 559.91 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636886/1807941 [19:10<05:06, 558.29 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636942/1807941 [19:10<05:07, 555.73 examples/s]
Tokenizing train dataset:  91%|█████████ | 1636998/1807941 [19:10<05:07, 555.35 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637059/1807941 [19:11<05:42, 499.32 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637112/1807941 [19:11<05:38, 504.35 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637166/1807941 [19:11<05:34, 510.81 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637220/1807941 [19:11<05:30, 516.14 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637274/1807941 [19:11<05:28, 519.25 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637327/1807941 [19:11<05:27, 520.21 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637406/1807941 [19:11<05:29, 517.89 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637460/1807941 [19:11<05:27, 521.13 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637515/1807941 [19:11<05:23, 526.11 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637569/1807941 [19:12<05:23, 527.03 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637623/1807941 [19:12<05:24, 525.57 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637676/1807941 [19:12<05:24, 524.84 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637741/1807941 [19:12<05:50, 485.06 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637806/1807941 [19:12<06:06, 463.71 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637875/1807941 [19:12<06:09, 459.91 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637922/1807941 [19:12<06:11, 457.92 examples/s]
Tokenizing train dataset:  91%|█████████ | 1637969/1807941 [19:12<06:09, 460.30 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638022/1807941 [19:13<07:03, 401.20 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638066/1807941 [19:13<06:56, 407.40 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638109/1807941 [19:13<06:53, 410.85 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638152/1807941 [19:13<06:51, 412.89 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638195/1807941 [19:13<06:49, 414.69 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638239/1807941 [19:13<06:45, 418.62 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638282/1807941 [19:13<06:45, 418.69 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638325/1807941 [19:13<06:43, 420.79 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638389/1807941 [19:13<06:44, 419.11 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638432/1807941 [19:14<06:43, 419.65 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638476/1807941 [19:14<06:42, 421.48 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638519/1807941 [19:14<06:43, 419.93 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638562/1807941 [19:14<06:43, 419.77 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638605/1807941 [19:14<06:42, 421.22 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638648/1807941 [19:14<06:42, 420.68 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638691/1807941 [19:14<06:42, 420.40 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638735/1807941 [19:14<06:40, 422.50 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638778/1807941 [19:14<06:40, 422.79 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638822/1807941 [19:14<06:36, 426.57 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638865/1807941 [19:15<06:38, 423.80 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638909/1807941 [19:15<06:38, 423.75 examples/s]
Tokenizing train dataset:  91%|█████████ | 1638972/1807941 [19:15<06:41, 421.12 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639019/1807941 [19:15<07:40, 366.99 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639061/1807941 [19:15<07:27, 377.59 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639103/1807941 [19:15<07:15, 387.38 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639145/1807941 [19:15<07:06, 395.43 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639188/1807941 [19:15<06:58, 403.61 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639230/1807941 [19:16<06:54, 407.16 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639274/1807941 [19:16<06:48, 413.35 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639320/1807941 [19:16<06:39, 422.50 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639363/1807941 [19:16<06:39, 421.80 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639407/1807941 [19:16<06:36, 425.07 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639469/1807941 [19:16<06:43, 417.87 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639512/1807941 [19:16<06:42, 418.30 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639556/1807941 [19:16<06:40, 420.65 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639599/1807941 [19:16<06:39, 421.20 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639642/1807941 [19:16<06:41, 419.26 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639684/1807941 [19:17<06:45, 415.27 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639727/1807941 [19:17<06:44, 416.16 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639771/1807941 [19:17<06:39, 420.91 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639815/1807941 [19:17<06:36, 423.80 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639878/1807941 [19:17<06:40, 419.28 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639923/1807941 [19:17<06:36, 423.40 examples/s]
Tokenizing train dataset:  91%|█████████ | 1639967/1807941 [19:17<06:35, 424.41 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640020/1807941 [19:17<07:34, 369.82 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640061/1807941 [19:18<07:24, 377.28 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640103/1807941 [19:18<07:15, 385.24 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640145/1807941 [19:18<07:08, 391.74 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640187/1807941 [19:18<07:00, 399.17 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640230/1807941 [19:18<06:53, 405.30 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640273/1807941 [19:18<06:50, 408.26 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640317/1807941 [19:18<06:45, 413.22 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640360/1807941 [19:18<06:43, 415.16 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640403/1807941 [19:18<06:42, 416.54 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640447/1807941 [19:18<06:38, 419.96 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640490/1807941 [19:19<06:39, 419.14 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640534/1807941 [19:19<06:38, 420.15 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640579/1807941 [19:19<06:33, 425.09 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640623/1807941 [19:19<06:31, 427.21 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640668/1807941 [19:19<06:28, 430.96 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640733/1807941 [19:19<06:31, 427.00 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640779/1807941 [19:19<06:24, 435.30 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640826/1807941 [19:19<06:17, 442.32 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640872/1807941 [19:19<06:14, 446.18 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640917/1807941 [19:20<06:16, 444.07 examples/s]
Tokenizing train dataset:  91%|█████████ | 1640963/1807941 [19:20<06:12, 448.08 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641024/1807941 [19:20<06:56, 400.37 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641072/1807941 [19:20<06:39, 417.29 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641117/1807941 [19:20<06:33, 423.52 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641166/1807941 [19:20<06:19, 439.35 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641212/1807941 [19:20<06:16, 442.27 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641257/1807941 [19:20<06:16, 442.16 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641302/1807941 [19:20<06:17, 441.69 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641369/1807941 [19:21<06:19, 439.33 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641414/1807941 [19:21<06:18, 439.79 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641459/1807941 [19:21<06:18, 439.41 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641505/1807941 [19:21<06:16, 441.74 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641571/1807941 [19:21<06:20, 437.18 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641615/1807941 [19:21<06:21, 436.10 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641661/1807941 [19:21<06:19, 437.66 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641726/1807941 [19:21<06:22, 434.31 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641790/1807941 [19:22<06:26, 429.78 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641834/1807941 [19:22<06:26, 429.48 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641879/1807941 [19:22<06:24, 432.09 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641925/1807941 [19:22<06:20, 435.81 examples/s]
Tokenizing train dataset:  91%|█████████ | 1641970/1807941 [19:22<06:20, 436.45 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642022/1807941 [19:22<07:16, 380.22 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642067/1807941 [19:22<07:00, 394.18 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642111/1807941 [19:22<06:49, 405.10 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642157/1807941 [19:22<06:37, 416.79 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642201/1807941 [19:23<06:33, 421.60 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642249/1807941 [19:23<06:20, 435.27 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642296/1807941 [19:23<06:12, 444.35 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642342/1807941 [19:23<06:10, 447.19 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642391/1807941 [19:23<06:01, 458.48 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642438/1807941 [19:23<06:00, 458.92 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642487/1807941 [19:23<05:55, 465.53 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642537/1807941 [19:23<05:50, 472.56 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642586/1807941 [19:23<05:48, 474.48 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642654/1807941 [19:24<05:57, 462.46 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642721/1807941 [19:24<06:04, 452.71 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642788/1807941 [19:24<06:09, 446.83 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642855/1807941 [19:24<06:13, 441.67 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642920/1807941 [19:24<06:17, 436.72 examples/s]
Tokenizing train dataset:  91%|█████████ | 1642964/1807941 [19:24<06:18, 435.54 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643022/1807941 [19:24<07:02, 390.69 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643064/1807941 [19:25<06:56, 396.24 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643106/1807941 [19:25<06:51, 400.68 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643150/1807941 [19:25<06:44, 407.61 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643195/1807941 [19:25<06:36, 415.94 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643240/1807941 [19:25<06:31, 420.75 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643285/1807941 [19:25<06:27, 425.28 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643331/1807941 [19:25<06:22, 430.36 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643375/1807941 [19:25<06:21, 431.28 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643421/1807941 [19:25<06:16, 437.52 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643468/1807941 [19:25<06:10, 444.38 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643514/1807941 [19:26<06:07, 447.57 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643562/1807941 [19:26<06:01, 454.22 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643610/1807941 [19:26<05:58, 459.02 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643660/1807941 [19:26<05:51, 466.86 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643708/1807941 [19:26<05:50, 468.26 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643755/1807941 [19:26<05:52, 466.40 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643822/1807941 [19:26<05:58, 457.36 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643868/1807941 [19:26<06:00, 455.29 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643932/1807941 [19:26<06:11, 442.03 examples/s]
Tokenizing train dataset:  91%|█████████ | 1643996/1807941 [19:27<06:18, 433.12 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644040/1807941 [19:27<07:14, 377.11 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644082/1807941 [19:27<07:04, 386.36 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644125/1807941 [19:27<06:53, 396.60 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644167/1807941 [19:27<06:48, 400.98 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644208/1807941 [19:27<06:47, 402.13 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644252/1807941 [19:27<06:40, 409.21 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644295/1807941 [19:27<06:35, 413.35 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644339/1807941 [19:27<06:32, 416.30 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644382/1807941 [19:28<06:33, 415.79 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644425/1807941 [19:28<06:31, 417.66 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644469/1807941 [19:28<06:28, 420.34 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644514/1807941 [19:28<06:24, 424.74 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644557/1807941 [19:28<06:25, 423.77 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644601/1807941 [19:28<06:25, 424.02 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644644/1807941 [19:28<06:26, 422.89 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644687/1807941 [19:28<06:24, 424.61 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644730/1807941 [19:28<06:25, 423.88 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644774/1807941 [19:29<06:23, 425.20 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644818/1807941 [19:29<06:22, 426.48 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644862/1807941 [19:29<06:20, 428.21 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644908/1807941 [19:29<06:17, 432.22 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644952/1807941 [19:29<06:16, 432.48 examples/s]
Tokenizing train dataset:  91%|█████████ | 1644996/1807941 [19:29<06:19, 429.50 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645042/1807941 [19:29<07:21, 368.77 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645085/1807941 [19:29<07:07, 381.39 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645128/1807941 [19:29<06:54, 392.35 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645172/1807941 [19:29<06:44, 402.35 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645215/1807941 [19:30<06:38, 408.11 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645261/1807941 [19:30<06:27, 419.44 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645306/1807941 [19:30<06:22, 425.43 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645349/1807941 [19:30<06:23, 423.60 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645412/1807941 [19:30<06:25, 421.15 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645474/1807941 [19:30<06:31, 414.53 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645516/1807941 [19:30<06:32, 413.98 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645558/1807941 [19:30<06:31, 414.43 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645601/1807941 [19:31<06:31, 414.85 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645645/1807941 [19:31<06:28, 417.77 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645688/1807941 [19:31<06:28, 417.88 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645732/1807941 [19:31<06:26, 420.02 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645775/1807941 [19:31<06:27, 418.16 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645818/1807941 [19:31<06:27, 418.78 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645861/1807941 [19:31<06:28, 417.24 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645903/1807941 [19:31<06:31, 414.10 examples/s]
Tokenizing train dataset:  91%|█████████ | 1645965/1807941 [19:31<06:34, 410.32 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646020/1807941 [19:32<07:32, 358.13 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646061/1807941 [19:32<07:19, 368.47 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646101/1807941 [19:32<07:11, 375.33 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646143/1807941 [19:32<07:02, 382.93 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646184/1807941 [19:32<06:56, 388.53 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646227/1807941 [19:32<06:49, 395.30 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646269/1807941 [19:32<06:43, 400.32 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646311/1807941 [19:32<06:39, 404.36 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646353/1807941 [19:32<06:38, 405.83 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646397/1807941 [19:33<06:32, 411.82 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646441/1807941 [19:33<06:27, 417.26 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646484/1807941 [19:33<06:26, 418.18 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646527/1807941 [19:33<06:26, 417.74 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646569/1807941 [19:33<06:27, 416.33 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646611/1807941 [19:33<06:29, 414.52 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646654/1807941 [19:33<06:27, 415.89 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646696/1807941 [19:33<06:29, 414.23 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646740/1807941 [19:33<06:26, 416.72 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646782/1807941 [19:33<06:30, 412.75 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646826/1807941 [19:34<06:26, 416.75 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646869/1807941 [19:34<06:24, 418.63 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646914/1807941 [19:34<06:20, 423.22 examples/s]
Tokenizing train dataset:  91%|█████████ | 1646978/1807941 [19:34<06:20, 422.53 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647021/1807941 [19:34<07:14, 370.55 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647064/1807941 [19:34<07:00, 382.91 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647106/1807941 [19:34<06:50, 391.83 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647149/1807941 [19:34<06:41, 399.99 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647191/1807941 [19:34<06:37, 404.34 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647233/1807941 [19:35<06:36, 405.60 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647276/1807941 [19:35<06:32, 409.03 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647318/1807941 [19:35<06:31, 410.47 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647360/1807941 [19:35<06:30, 410.72 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647402/1807941 [19:35<06:29, 411.75 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647444/1807941 [19:35<06:29, 411.69 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647488/1807941 [19:35<06:26, 415.01 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647530/1807941 [19:35<06:25, 415.73 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647573/1807941 [19:35<06:24, 417.43 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647616/1807941 [19:35<06:23, 417.75 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647658/1807941 [19:36<06:25, 415.65 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647700/1807941 [19:36<06:25, 415.79 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647743/1807941 [19:36<06:23, 417.61 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647787/1807941 [19:36<06:21, 419.64 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647831/1807941 [19:36<06:19, 422.09 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647874/1807941 [19:36<06:18, 422.67 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647920/1807941 [19:36<06:13, 428.49 examples/s]
Tokenizing train dataset:  91%|█████████ | 1647965/1807941 [19:36<06:09, 432.41 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648022/1807941 [19:36<06:56, 384.41 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648066/1807941 [19:37<06:42, 396.89 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648113/1807941 [19:37<06:26, 413.36 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648160/1807941 [19:37<06:16, 424.83 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648208/1807941 [19:37<06:06, 436.01 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648256/1807941 [19:37<05:58, 444.94 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648305/1807941 [19:37<05:51, 454.10 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648353/1807941 [19:37<05:47, 458.62 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648404/1807941 [19:37<05:40, 468.38 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648452/1807941 [19:37<05:40, 467.98 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648499/1807941 [19:37<05:41, 466.97 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648567/1807941 [19:38<05:48, 456.83 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648634/1807941 [19:38<05:53, 450.53 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648702/1807941 [19:38<05:57, 445.89 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648747/1807941 [19:38<05:58, 444.46 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648793/1807941 [19:38<05:58, 443.82 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648838/1807941 [19:38<05:59, 442.14 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648884/1807941 [19:38<05:59, 442.09 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648929/1807941 [19:38<06:00, 440.88 examples/s]
Tokenizing train dataset:  91%|█████████ | 1648992/1807941 [19:39<06:08, 431.52 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649042/1807941 [19:39<07:03, 374.77 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649085/1807941 [19:39<06:52, 384.88 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649128/1807941 [19:39<06:42, 394.39 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649172/1807941 [19:39<06:34, 402.65 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649216/1807941 [19:39<06:27, 409.19 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649261/1807941 [19:39<06:21, 415.74 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649306/1807941 [19:39<06:16, 421.59 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649350/1807941 [19:40<06:13, 424.07 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649395/1807941 [19:40<06:09, 429.46 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649441/1807941 [19:40<06:05, 433.62 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649488/1807941 [19:40<05:57, 443.62 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649535/1807941 [19:40<05:52, 449.19 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649582/1807941 [19:40<05:49, 452.86 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649631/1807941 [19:40<05:43, 460.74 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649679/1807941 [19:40<05:41, 463.13 examples/s]
Tokenizing train dataset:  91%|█████████ | 1649729/1807941 [19:40<05:36, 470.03 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1649779/1807941 [19:40<05:31, 477.51 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1649829/1807941 [19:41<05:29, 479.31 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1649879/1807941 [19:41<05:28, 481.36 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1649948/1807941 [19:41<05:36, 469.40 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650000/1807941 [19:41<06:34, 400.08 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650062/1807941 [19:41<06:34, 400.16 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650104/1807941 [19:41<06:32, 402.54 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650147/1807941 [19:41<06:27, 407.32 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650192/1807941 [19:41<06:19, 416.21 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650238/1807941 [19:42<06:11, 424.06 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650282/1807941 [19:42<06:10, 425.02 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650325/1807941 [19:42<06:12, 423.48 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650368/1807941 [19:42<06:13, 421.42 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650414/1807941 [19:42<06:08, 427.81 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650459/1807941 [19:42<06:05, 431.05 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650505/1807941 [19:42<06:01, 435.55 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650552/1807941 [19:42<05:55, 442.24 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650599/1807941 [19:42<05:51, 448.25 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650646/1807941 [19:42<05:47, 453.01 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650692/1807941 [19:43<05:47, 452.37 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650741/1807941 [19:43<05:40, 461.63 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650789/1807941 [19:43<05:37, 466.32 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650838/1807941 [19:43<05:34, 469.00 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650888/1807941 [19:43<05:31, 474.33 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650937/1807941 [19:43<05:29, 475.90 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1650988/1807941 [19:43<05:25, 482.63 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651049/1807941 [19:43<06:06, 427.97 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651097/1807941 [19:43<05:56, 440.24 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651148/1807941 [19:44<05:44, 455.31 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651198/1807941 [19:44<05:36, 465.42 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651249/1807941 [19:44<05:29, 476.02 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651300/1807941 [19:44<05:24, 482.64 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651371/1807941 [19:44<05:28, 476.30 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651439/1807941 [19:44<05:35, 467.08 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651506/1807941 [19:44<05:41, 457.99 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651574/1807941 [19:44<05:45, 452.96 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651620/1807941 [19:45<05:47, 450.45 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651685/1807941 [19:45<05:53, 441.83 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651730/1807941 [19:45<05:54, 440.72 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651796/1807941 [19:45<05:56, 438.12 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651840/1807941 [19:45<05:57, 437.00 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651900/1807941 [19:45<06:12, 418.54 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1651962/1807941 [19:45<06:16, 414.79 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652021/1807941 [19:46<06:51, 378.47 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652063/1807941 [19:46<06:45, 384.86 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652105/1807941 [19:46<06:38, 391.28 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652149/1807941 [19:46<06:29, 399.82 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652192/1807941 [19:46<06:24, 405.09 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652236/1807941 [19:46<06:19, 410.62 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652280/1807941 [19:46<06:15, 414.35 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652324/1807941 [19:46<06:11, 418.73 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652386/1807941 [19:46<06:16, 413.67 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652428/1807941 [19:47<06:16, 413.00 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652470/1807941 [19:47<06:15, 413.97 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652512/1807941 [19:47<06:17, 411.89 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652557/1807941 [19:47<06:12, 417.54 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652602/1807941 [19:47<06:06, 423.39 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652646/1807941 [19:47<06:03, 426.96 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652691/1807941 [19:47<05:59, 432.45 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652738/1807941 [19:47<05:51, 441.42 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652803/1807941 [19:47<05:57, 433.90 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652848/1807941 [19:48<05:57, 433.74 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652893/1807941 [19:48<05:56, 434.86 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652956/1807941 [19:48<06:05, 423.94 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1652999/1807941 [19:48<06:06, 423.21 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653061/1807941 [19:48<06:57, 371.40 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653103/1807941 [19:48<06:49, 378.52 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653146/1807941 [19:48<06:38, 388.49 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653214/1807941 [19:48<05:35, 460.75 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653285/1807941 [19:49<04:55, 523.80 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653358/1807941 [19:49<04:27, 576.94 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653425/1807941 [19:49<04:17, 599.28 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653497/1807941 [19:49<04:04, 631.00 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653570/1807941 [19:49<03:55, 656.54 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653674/1807941 [19:49<03:51, 665.95 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653747/1807941 [19:49<03:46, 680.09 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653826/1807941 [19:49<03:38, 705.58 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653905/1807941 [19:49<03:32, 726.19 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1653981/1807941 [19:49<03:29, 733.67 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1654069/1807941 [19:50<03:57, 647.71 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1654142/1807941 [19:50<03:50, 666.66 examples/s]
Tokenizing train dataset:  91%|█████████▏| 1654246/1807941 [19:50<03:48, 673.64 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1654318/1807941 [19:50<03:44, 683.57 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1654414/1807941 [19:50<03:50, 664.93 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1654521/1807941 [19:50<03:46, 677.21 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1654628/1807941 [19:50<03:43, 685.08 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1654705/1807941 [19:51<03:38, 702.01 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1654808/1807941 [19:51<03:40, 693.37 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1654886/1807941 [19:51<03:35, 711.31 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1654967/1807941 [19:51<03:28, 734.95 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1655076/1807941 [19:51<03:39, 696.98 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1655154/1807941 [19:51<03:33, 714.91 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1655235/1807941 [19:51<03:27, 737.65 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1655316/1807941 [19:51<03:22, 754.25 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1655396/1807941 [19:51<03:20, 761.71 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1655476/1807941 [19:52<03:18, 769.75 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1655554/1807941 [19:52<03:17, 770.99 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1655670/1807941 [19:52<03:18, 768.26 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1655749/1807941 [19:52<03:17, 769.20 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1655827/1807941 [19:52<03:17, 770.27 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1655943/1807941 [19:52<03:17, 769.16 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1656041/1807941 [19:52<03:29, 725.43 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1656120/1807941 [19:52<03:25, 740.58 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1656203/1807941 [19:53<03:19, 762.12 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1656312/1807941 [19:53<03:22, 748.51 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1656391/1807941 [19:53<03:19, 758.44 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1656502/1807941 [19:53<03:22, 748.47 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1656619/1807941 [19:53<03:20, 754.05 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1656697/1807941 [19:53<03:19, 756.55 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1656804/1807941 [19:53<03:25, 736.81 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1656910/1807941 [19:54<03:28, 725.67 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1656986/1807941 [19:54<03:26, 732.45 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657069/1807941 [19:54<03:45, 669.64 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657142/1807941 [19:54<03:41, 680.51 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657214/1807941 [19:54<03:38, 688.67 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657286/1807941 [19:54<03:36, 696.62 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657357/1807941 [19:54<03:35, 697.19 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657428/1807941 [19:54<03:35, 697.48 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657515/1807941 [19:54<03:51, 649.56 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657599/1807941 [19:55<04:33, 550.18 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657662/1807941 [19:55<04:54, 509.88 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657724/1807941 [19:55<05:13, 479.10 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657788/1807941 [19:55<05:26, 460.25 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657851/1807941 [19:55<05:36, 446.50 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657914/1807941 [19:55<05:43, 436.63 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1657977/1807941 [19:56<05:48, 430.13 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658040/1807941 [19:56<05:52, 425.85 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658109/1807941 [19:56<05:09, 483.63 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658184/1807941 [19:56<04:34, 545.63 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658249/1807941 [19:56<04:21, 571.43 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658344/1807941 [19:56<04:14, 588.58 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658433/1807941 [19:56<04:14, 586.75 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658504/1807941 [19:56<04:04, 611.58 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658571/1807941 [19:57<03:58, 625.01 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658661/1807941 [19:57<04:03, 613.12 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658752/1807941 [19:57<04:05, 606.96 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658822/1807941 [19:57<03:57, 627.43 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658891/1807941 [19:57<03:52, 640.65 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1658988/1807941 [19:57<03:53, 638.21 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659061/1807941 [19:57<04:15, 582.98 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659127/1807941 [19:57<04:07, 600.17 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659192/1807941 [19:58<04:03, 609.87 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659270/1807941 [19:58<03:48, 651.64 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659340/1807941 [19:58<03:44, 661.53 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659449/1807941 [19:58<03:36, 684.40 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659520/1807941 [19:58<03:35, 689.24 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659618/1807941 [19:58<03:40, 673.61 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659712/1807941 [19:58<03:47, 652.13 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659781/1807941 [19:58<04:13, 585.11 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659843/1807941 [19:59<04:43, 523.11 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659914/1807941 [19:59<05:23, 457.02 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1659972/1807941 [19:59<05:39, 435.26 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660019/1807941 [19:59<06:22, 386.77 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660078/1807941 [19:59<06:22, 386.97 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660119/1807941 [19:59<06:19, 389.60 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660160/1807941 [19:59<06:17, 391.41 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660201/1807941 [20:00<06:15, 393.83 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660244/1807941 [20:00<06:07, 401.54 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660287/1807941 [20:00<06:03, 406.50 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660331/1807941 [20:00<05:55, 415.01 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660375/1807941 [20:00<05:52, 418.15 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660419/1807941 [20:00<05:49, 421.58 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660464/1807941 [20:00<05:44, 427.89 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660508/1807941 [20:00<05:43, 429.41 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660553/1807941 [20:00<05:40, 432.28 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660631/1807941 [20:00<04:37, 531.54 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660734/1807941 [20:01<03:41, 663.60 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660850/1807941 [20:01<03:02, 805.60 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1660951/1807941 [20:01<02:50, 860.60 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1661050/1807941 [20:01<02:58, 822.72 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1661152/1807941 [20:01<02:48, 870.46 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1661243/1807941 [20:01<02:46, 879.45 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1661373/1807941 [20:01<02:48, 870.49 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1661492/1807941 [20:01<02:54, 838.27 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1661615/1807941 [20:02<03:00, 810.49 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1661724/1807941 [20:02<03:07, 778.52 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1661841/1807941 [20:02<03:09, 769.57 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1661951/1807941 [20:02<03:13, 755.00 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1662043/1807941 [20:02<03:26, 706.39 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1662128/1807941 [20:02<03:18, 736.02 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1662203/1807941 [20:02<03:18, 735.13 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1662284/1807941 [20:03<03:13, 750.95 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1662372/1807941 [20:03<03:06, 781.35 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1662467/1807941 [20:03<02:55, 826.80 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1662559/1807941 [20:03<02:51, 845.97 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1662674/1807941 [20:03<03:00, 804.68 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1662761/1807941 [20:03<02:57, 817.97 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1662861/1807941 [20:03<02:47, 863.58 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1662955/1807941 [20:03<02:44, 882.08 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1663080/1807941 [20:03<02:59, 806.68 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1663197/1807941 [20:04<03:04, 782.94 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1663317/1807941 [20:04<03:03, 786.75 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1663440/1807941 [20:04<03:02, 790.26 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1663534/1807941 [20:04<02:55, 821.75 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1663665/1807941 [20:04<02:52, 836.48 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1663757/1807941 [20:04<02:48, 855.08 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1663846/1807941 [20:04<02:47, 859.26 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1663933/1807941 [20:04<02:47, 859.73 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1664044/1807941 [20:05<02:59, 803.78 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1664129/1807941 [20:05<02:56, 813.68 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1664251/1807941 [20:05<02:59, 802.34 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1664335/1807941 [20:05<02:58, 806.29 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1664419/1807941 [20:05<02:57, 807.25 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1664506/1807941 [20:05<02:55, 818.46 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1664597/1807941 [20:05<02:51, 836.99 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1664713/1807941 [20:05<02:56, 809.73 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1664798/1807941 [20:06<02:55, 816.27 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1664891/1807941 [20:06<02:53, 825.75 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1664975/1807941 [20:06<02:53, 825.05 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1665077/1807941 [20:06<03:13, 736.94 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1665186/1807941 [20:06<03:15, 731.22 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1665266/1807941 [20:06<03:11, 744.69 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1665350/1807941 [20:06<03:07, 762.10 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1665461/1807941 [20:06<03:09, 751.87 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1665539/1807941 [20:07<03:08, 756.23 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1665624/1807941 [20:07<03:06, 763.40 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1665710/1807941 [20:07<03:01, 784.10 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1665833/1807941 [20:07<02:59, 790.33 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1665919/1807941 [20:07<02:56, 803.11 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1666000/1807941 [20:07<03:08, 753.37 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1666079/1807941 [20:07<03:06, 762.36 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1666161/1807941 [20:07<03:02, 776.06 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1666251/1807941 [20:07<02:55, 807.44 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1666354/1807941 [20:08<02:43, 867.46 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1666449/1807941 [20:08<02:39, 886.77 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1666580/1807941 [20:08<02:41, 873.87 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1666695/1807941 [20:08<02:49, 833.26 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1666806/1807941 [20:08<02:56, 798.32 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1666909/1807941 [20:08<03:06, 758.04 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1667000/1807941 [20:08<03:19, 705.18 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1667084/1807941 [20:09<03:13, 727.26 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1667169/1807941 [20:09<03:06, 755.20 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1667279/1807941 [20:09<03:09, 743.92 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1667358/1807941 [20:09<03:06, 753.02 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1667477/1807941 [20:09<03:04, 762.62 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1667557/1807941 [20:09<03:03, 766.86 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1667638/1807941 [20:09<03:01, 773.55 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1667753/1807941 [20:09<03:07, 746.98 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1667836/1807941 [20:10<03:02, 766.08 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1667949/1807941 [20:10<03:04, 757.62 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1668045/1807941 [20:10<03:15, 715.13 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1668119/1807941 [20:10<03:14, 718.89 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1668214/1807941 [20:10<03:23, 686.48 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1668296/1807941 [20:10<03:14, 716.78 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1668384/1807941 [20:10<03:05, 750.88 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1668471/1807941 [20:10<02:58, 780.68 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1668552/1807941 [20:10<02:58, 781.73 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1668671/1807941 [20:11<02:57, 782.98 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1668755/1807941 [20:11<02:54, 795.79 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1668840/1807941 [20:11<02:53, 799.48 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1668936/1807941 [20:11<02:45, 839.13 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1669048/1807941 [20:11<02:57, 782.56 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1669163/1807941 [20:11<03:00, 770.38 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1669276/1807941 [20:11<03:05, 748.74 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1669379/1807941 [20:12<03:10, 725.49 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1669458/1807941 [20:12<03:07, 738.29 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1669565/1807941 [20:12<03:10, 727.58 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1669644/1807941 [20:12<03:07, 739.11 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1669720/1807941 [20:12<03:06, 739.45 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1669829/1807941 [20:12<03:08, 731.11 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1669922/1807941 [20:12<03:20, 689.85 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1670000/1807941 [20:12<03:29, 659.65 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1670071/1807941 [20:13<03:26, 668.59 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1670141/1807941 [20:13<03:25, 669.78 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1670214/1807941 [20:13<03:22, 681.64 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1670287/1807941 [20:13<03:18, 694.01 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1670369/1807941 [20:13<03:08, 727.92 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1670478/1807941 [20:13<03:10, 722.74 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1670555/1807941 [20:13<03:07, 733.04 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1670673/1807941 [20:13<03:03, 746.84 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1670759/1807941 [20:13<02:57, 773.44 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1670842/1807941 [20:14<02:55, 782.77 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1670928/1807941 [20:14<02:50, 802.88 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1671043/1807941 [20:14<03:00, 760.22 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1671156/1807941 [20:14<03:01, 753.19 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1671243/1807941 [20:14<02:55, 779.35 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1671351/1807941 [20:14<03:01, 752.77 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1671430/1807941 [20:14<02:59, 760.45 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1671513/1807941 [20:14<02:55, 775.17 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1671593/1807941 [20:15<02:55, 778.81 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1671676/1807941 [20:15<02:53, 787.29 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1671785/1807941 [20:15<02:58, 760.70 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1671866/1807941 [20:15<02:56, 772.88 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1671952/1807941 [20:15<02:51, 793.55 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1672049/1807941 [20:15<02:52, 785.58 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1672151/1807941 [20:15<02:40, 847.72 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1672244/1807941 [20:15<02:36, 868.53 examples/s]
Tokenizing train dataset:  92%|█████████▏| 1672332/1807941 [20:15<02:36, 864.37 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1672455/1807941 [20:16<02:40, 842.73 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1672542/1807941 [20:16<02:40, 844.55 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1672630/1807941 [20:16<02:39, 846.43 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1672720/1807941 [20:16<02:37, 858.76 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1672843/1807941 [20:16<02:42, 833.83 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1672969/1807941 [20:16<02:42, 828.17 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1673084/1807941 [20:16<02:53, 777.28 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1673198/1807941 [20:17<02:55, 768.63 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1673281/1807941 [20:17<02:52, 779.70 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1673415/1807941 [20:17<02:44, 815.41 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1673503/1807941 [20:17<02:42, 826.39 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1673602/1807941 [20:17<02:35, 866.50 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1673691/1807941 [20:17<02:34, 871.30 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1673823/1807941 [20:17<02:34, 868.64 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1673961/1807941 [20:17<02:32, 877.75 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1674086/1807941 [20:18<02:44, 815.61 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1674171/1807941 [20:18<02:43, 818.74 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1674256/1807941 [20:18<02:42, 822.51 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1674374/1807941 [20:18<02:45, 807.04 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1674463/1807941 [20:18<02:42, 823.24 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1674561/1807941 [20:18<02:35, 857.32 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1674680/1807941 [20:18<02:40, 829.21 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1674801/1807941 [20:18<02:42, 817.21 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1674918/1807941 [20:19<02:46, 800.13 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1675000/1807941 [20:19<02:53, 764.15 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1675096/1807941 [20:19<02:44, 806.10 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1675183/1807941 [20:19<02:42, 817.61 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1675300/1807941 [20:19<02:45, 801.10 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1675384/1807941 [20:19<02:43, 809.33 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1675476/1807941 [20:19<02:39, 831.90 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1675609/1807941 [20:19<02:37, 842.08 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1675731/1807941 [20:20<02:41, 818.47 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1675858/1807941 [20:20<02:40, 820.59 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1675947/1807941 [20:20<02:39, 829.09 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1676039/1807941 [20:20<02:56, 747.09 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1676125/1807941 [20:20<02:50, 771.09 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1676214/1807941 [20:20<02:44, 799.78 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1676296/1807941 [20:20<02:43, 803.06 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1676381/1807941 [20:20<02:41, 813.74 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1676468/1807941 [20:21<02:40, 819.59 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1676554/1807941 [20:21<02:38, 828.12 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1676674/1807941 [20:21<02:41, 812.68 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1676764/1807941 [20:21<02:37, 833.07 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1676850/1807941 [20:21<02:36, 838.47 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1676935/1807941 [20:21<02:35, 840.05 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1677060/1807941 [20:21<02:36, 835.43 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1677183/1807941 [20:21<02:38, 825.81 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1677268/1807941 [20:21<02:37, 827.81 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1677359/1807941 [20:22<02:34, 844.74 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1677484/1807941 [20:22<02:35, 837.64 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1677609/1807941 [20:22<02:37, 829.80 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1677724/1807941 [20:22<02:41, 806.83 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1677811/1807941 [20:22<02:40, 812.71 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1677900/1807941 [20:22<02:36, 830.79 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1677992/1807941 [20:22<02:32, 849.94 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1678081/1807941 [20:22<02:52, 753.37 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1678194/1807941 [20:23<02:53, 749.49 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1678277/1807941 [20:23<02:48, 768.77 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1678364/1807941 [20:23<02:43, 793.51 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1678481/1807941 [20:23<02:45, 781.17 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1678581/1807941 [20:23<02:56, 733.13 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1678657/1807941 [20:23<02:54, 739.16 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1678742/1807941 [20:23<02:48, 764.93 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1678828/1807941 [20:23<02:43, 788.85 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1678947/1807941 [20:24<02:43, 788.23 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1679034/1807941 [20:24<03:01, 710.76 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1679117/1807941 [20:24<02:54, 738.75 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1679226/1807941 [20:24<02:56, 731.21 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1679311/1807941 [20:24<02:49, 758.44 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1679433/1807941 [20:24<02:45, 774.48 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1679547/1807941 [20:24<02:48, 763.97 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1679633/1807941 [20:25<02:43, 784.21 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1679719/1807941 [20:25<02:40, 801.36 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1679832/1807941 [20:25<02:43, 782.60 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1679912/1807941 [20:25<02:43, 783.54 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1680000/1807941 [20:25<03:02, 700.52 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1680108/1807941 [20:25<03:02, 699.85 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1680181/1807941 [20:25<03:01, 704.74 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1680270/1807941 [20:25<02:50, 749.70 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1680352/1807941 [20:25<02:46, 765.82 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1680446/1807941 [20:26<02:37, 810.21 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1680554/1807941 [20:26<02:47, 762.76 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1680642/1807941 [20:26<02:40, 790.97 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1680724/1807941 [20:26<02:39, 795.86 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1680809/1807941 [20:26<02:37, 808.78 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1680923/1807941 [20:26<02:43, 774.77 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1681014/1807941 [20:26<02:58, 710.01 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1681089/1807941 [20:26<02:56, 718.83 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1681191/1807941 [20:27<03:00, 702.86 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1681263/1807941 [20:27<02:59, 705.60 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1681336/1807941 [20:27<02:58, 710.83 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1681412/1807941 [20:27<02:55, 721.40 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1681491/1807941 [20:27<02:52, 733.77 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1681604/1807941 [20:27<02:53, 728.92 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1681682/1807941 [20:27<02:50, 739.31 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1681761/1807941 [20:27<02:48, 748.35 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1681841/1807941 [20:27<02:45, 761.19 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1681925/1807941 [20:28<02:42, 777.56 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1682010/1807941 [20:28<03:01, 692.08 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1682095/1807941 [20:28<02:52, 728.39 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1682170/1807941 [20:28<02:51, 731.88 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1682260/1807941 [20:28<02:42, 774.21 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1682375/1807941 [20:28<02:49, 739.23 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1682458/1807941 [20:28<02:45, 759.86 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1682579/1807941 [20:28<02:43, 768.59 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1682665/1807941 [20:29<02:39, 786.00 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1682768/1807941 [20:29<02:47, 748.97 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1682868/1807941 [20:29<02:54, 718.04 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1682948/1807941 [20:29<02:49, 735.51 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1683035/1807941 [20:29<03:04, 675.74 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1683116/1807941 [20:29<02:57, 704.53 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1683199/1807941 [20:29<02:50, 732.55 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1683290/1807941 [20:29<02:40, 776.96 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1683372/1807941 [20:30<02:38, 785.60 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1683486/1807941 [20:30<02:42, 767.53 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1683600/1807941 [20:30<02:43, 758.98 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1683683/1807941 [20:30<02:40, 774.57 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1683768/1807941 [20:30<02:36, 793.50 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1683854/1807941 [20:30<02:34, 805.20 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1683971/1807941 [20:30<02:36, 793.08 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1684074/1807941 [20:33<16:30, 125.11 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1684153/1807941 [20:33<12:59, 158.81 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1684236/1807941 [20:33<10:05, 204.17 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1684311/1807941 [20:33<08:10, 251.95 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1684405/1807941 [20:33<06:17, 327.67 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1684493/1807941 [20:33<05:06, 403.14 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1684589/1807941 [20:33<04:09, 493.85 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1684676/1807941 [20:33<03:38, 564.68 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1684767/1807941 [20:34<03:37, 565.93 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1684859/1807941 [20:34<03:12, 640.22 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1684958/1807941 [20:34<02:51, 718.99 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1685080/1807941 [20:34<02:52, 710.44 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1685169/1807941 [20:34<02:43, 751.58 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1685270/1807941 [20:34<02:30, 814.39 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1685368/1807941 [20:34<02:24, 849.11 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1685459/1807941 [20:34<02:21, 863.57 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1685576/1807941 [20:34<02:27, 829.94 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1685665/1807941 [20:35<02:25, 841.82 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1685761/1807941 [20:35<02:20, 870.69 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1685903/1807941 [20:35<02:16, 895.81 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1685995/1807941 [20:35<02:15, 898.27 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1686121/1807941 [20:35<02:21, 860.40 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1686215/1807941 [20:35<02:19, 874.91 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1686316/1807941 [20:35<02:14, 907.08 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1686447/1807941 [20:35<02:18, 878.20 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1686573/1807941 [20:36<02:20, 864.04 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1686674/1807941 [20:36<02:15, 893.40 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1686787/1807941 [20:36<02:23, 841.77 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1686913/1807941 [20:36<02:24, 839.73 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1687000/1807941 [20:36<02:32, 792.52 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1687118/1807941 [20:36<02:34, 780.87 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1687200/1807941 [20:36<02:33, 788.89 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1687294/1807941 [20:36<02:26, 824.15 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1687380/1807941 [20:37<02:25, 831.30 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1687500/1807941 [20:37<02:27, 817.98 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1687593/1807941 [20:37<02:27, 814.59 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1687718/1807941 [20:37<02:27, 817.65 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1687802/1807941 [20:37<02:26, 819.28 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1687885/1807941 [20:37<02:26, 818.12 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1687986/1807941 [20:37<02:36, 764.73 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1688082/1807941 [20:38<02:46, 720.32 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1688164/1807941 [20:38<02:41, 743.24 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1688244/1807941 [20:38<02:38, 757.16 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1688346/1807941 [20:38<02:24, 825.65 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1688433/1807941 [20:38<02:23, 835.62 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1688522/1807941 [20:38<02:21, 846.19 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1688658/1807941 [20:38<02:17, 866.79 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1688746/1807941 [20:38<02:17, 865.45 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1688849/1807941 [20:38<02:11, 904.77 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1688977/1807941 [20:39<02:15, 876.97 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1689090/1807941 [20:39<02:24, 824.62 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1689190/1807941 [20:39<02:17, 863.50 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1689308/1807941 [20:39<02:22, 834.36 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1689437/1807941 [20:39<02:21, 840.26 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1689536/1807941 [20:39<02:16, 868.32 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1689631/1807941 [20:39<02:13, 886.62 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1689761/1807941 [20:39<02:14, 877.99 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1689896/1807941 [20:40<02:13, 881.09 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1690000/1807941 [20:40<02:31, 779.11 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1690117/1807941 [20:40<02:31, 777.88 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1690201/1807941 [20:40<02:29, 785.68 examples/s]
Tokenizing train dataset:  93%|█████████▎| 1690309/1807941 [20:40<02:34, 761.35 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1690435/1807941 [20:40<02:30, 783.03 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1690527/1807941 [20:40<02:25, 805.51 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1690637/1807941 [20:41<02:31, 772.69 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1690737/1807941 [20:41<02:40, 729.81 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1690822/1807941 [20:41<02:35, 754.75 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1690903/1807941 [20:41<02:32, 765.19 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1690996/1807941 [20:41<02:24, 807.14 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1691098/1807941 [20:41<02:35, 750.09 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1691187/1807941 [20:41<02:29, 781.13 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1691273/1807941 [20:41<02:26, 797.09 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1691358/1807941 [20:42<02:23, 810.00 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1691467/1807941 [20:42<02:30, 771.55 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1691556/1807941 [20:42<02:25, 800.44 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1691653/1807941 [20:42<02:17, 843.41 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1691771/1807941 [20:42<02:22, 813.58 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1691860/1807941 [20:42<02:19, 832.06 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1691980/1807941 [20:42<02:22, 814.93 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1692094/1807941 [20:42<02:26, 793.47 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1692188/1807941 [20:43<02:21, 819.31 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1692283/1807941 [20:43<02:16, 844.94 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1692396/1807941 [20:43<02:22, 808.88 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1692479/1807941 [20:43<02:26, 790.46 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1692566/1807941 [20:43<02:22, 808.51 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1692687/1807941 [20:43<02:24, 796.32 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1692809/1807941 [20:43<02:27, 782.95 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1692897/1807941 [20:43<02:23, 802.00 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1692982/1807941 [20:44<02:21, 810.91 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1693081/1807941 [20:44<02:35, 738.22 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1693163/1807941 [20:44<02:32, 753.06 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1693256/1807941 [20:44<02:46, 689.90 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1693331/1807941 [20:44<02:44, 698.25 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1693413/1807941 [20:44<02:37, 728.21 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1693489/1807941 [20:44<02:35, 735.76 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1693576/1807941 [20:44<02:28, 769.38 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1693687/1807941 [20:45<02:31, 755.76 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1693777/1807941 [20:45<02:24, 790.55 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1693866/1807941 [20:45<02:20, 811.90 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1693983/1807941 [20:45<02:23, 796.53 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1694086/1807941 [20:45<02:30, 756.51 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1694196/1807941 [20:45<02:34, 738.08 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1694277/1807941 [20:45<02:31, 748.99 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1694395/1807941 [20:45<02:29, 759.79 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1694509/1807941 [20:46<02:29, 757.66 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1694596/1807941 [20:46<02:25, 779.85 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1694716/1807941 [20:46<02:24, 782.92 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1694808/1807941 [20:46<02:19, 811.60 examples/s]
Tokenizing train dataset:  94%|█████████▎| 1694891/1807941 [20:46<02:18, 813.47 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1694975/1807941 [20:46<02:18, 818.09 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1695088/1807941 [20:46<02:24, 780.73 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1695180/1807941 [20:46<02:18, 815.53 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1695296/1807941 [20:47<02:21, 796.90 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1695416/1807941 [20:47<02:21, 795.01 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1695536/1807941 [20:47<02:21, 791.97 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1695643/1807941 [20:47<02:26, 764.59 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1695759/1807941 [20:47<02:28, 754.52 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1695848/1807941 [20:47<02:23, 779.05 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1695955/1807941 [20:47<02:29, 750.56 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1696049/1807941 [20:48<02:38, 704.52 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1696138/1807941 [20:48<02:30, 744.28 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1696232/1807941 [20:48<02:21, 789.37 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1696350/1807941 [20:48<02:21, 786.08 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1696458/1807941 [20:48<02:26, 760.96 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1696576/1807941 [20:48<02:25, 763.39 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1696659/1807941 [20:48<02:23, 777.44 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1696753/1807941 [20:48<02:33, 725.69 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1696834/1807941 [20:49<02:29, 742.63 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1696939/1807941 [20:49<02:34, 720.50 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1697053/1807941 [20:49<02:49, 654.47 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1697144/1807941 [20:49<02:36, 707.58 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1697252/1807941 [20:49<02:36, 708.58 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1697329/1807941 [20:49<02:33, 721.30 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1697445/1807941 [20:49<02:30, 733.02 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1697533/1807941 [20:50<02:23, 766.94 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1697640/1807941 [20:50<02:28, 742.81 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1697725/1807941 [20:50<02:24, 762.28 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1697808/1807941 [20:50<02:22, 774.10 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1697890/1807941 [20:50<02:20, 783.20 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1697972/1807941 [20:50<02:19, 790.15 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1698068/1807941 [20:50<02:32, 718.25 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1698171/1807941 [20:50<02:35, 705.38 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1698249/1807941 [20:51<02:32, 720.86 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1698337/1807941 [20:51<02:24, 758.60 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1698435/1807941 [20:51<02:32, 720.10 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1698513/1807941 [20:51<02:29, 733.77 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1698605/1807941 [20:51<02:20, 780.69 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1698709/1807941 [20:51<02:28, 735.52 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1698818/1807941 [20:51<02:30, 723.78 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1698932/1807941 [20:51<02:29, 730.79 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1699044/1807941 [20:52<02:29, 730.42 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1699136/1807941 [20:52<02:21, 769.87 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1699245/1807941 [20:52<02:24, 751.96 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1699336/1807941 [20:52<02:17, 788.34 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1699418/1807941 [20:52<02:16, 793.55 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1699542/1807941 [20:52<02:15, 798.26 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1699662/1807941 [20:52<02:15, 797.21 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1699747/1807941 [20:52<02:13, 808.52 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1699840/1807941 [20:53<02:09, 836.78 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1699926/1807941 [20:53<02:08, 840.60 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1700033/1807941 [20:53<02:25, 742.07 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1700115/1807941 [20:53<02:22, 754.25 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1700227/1807941 [20:53<02:24, 747.11 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1700315/1807941 [20:53<02:18, 777.46 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1700405/1807941 [20:53<02:13, 808.14 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1700526/1807941 [20:53<02:13, 802.85 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1700629/1807941 [20:54<02:21, 756.85 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1700726/1807941 [20:54<02:29, 718.48 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1700808/1807941 [20:54<02:24, 740.56 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1700895/1807941 [20:54<02:19, 768.97 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1700976/1807941 [20:54<02:21, 753.73 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1701086/1807941 [20:54<02:26, 728.19 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1701173/1807941 [20:54<02:20, 762.19 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1701252/1807941 [20:54<02:18, 767.78 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1701359/1807941 [20:55<02:23, 744.42 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1701440/1807941 [20:55<02:20, 757.65 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1701521/1807941 [20:55<02:18, 765.95 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1701604/1807941 [20:55<02:16, 780.65 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1701683/1807941 [20:55<02:16, 778.06 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1701774/1807941 [20:55<02:12, 801.87 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1701856/1807941 [20:55<02:14, 789.61 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1701973/1807941 [20:55<02:18, 764.09 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1702083/1807941 [20:56<02:23, 739.54 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1702176/1807941 [20:56<02:16, 774.86 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1702269/1807941 [20:56<02:10, 809.74 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1702352/1807941 [20:56<02:09, 812.57 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1702448/1807941 [20:56<02:03, 851.40 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1702537/1807941 [20:56<02:03, 854.84 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1702678/1807941 [20:56<01:59, 883.30 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1702769/1807941 [20:56<01:58, 884.12 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1702858/1807941 [20:56<01:59, 882.37 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1702948/1807941 [20:57<01:58, 884.04 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1703070/1807941 [20:57<02:02, 855.21 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1703176/1807941 [20:57<01:55, 903.39 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1703272/1807941 [20:57<01:54, 915.28 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1703401/1807941 [20:57<01:57, 890.91 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1703505/1807941 [20:57<01:52, 924.89 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1703606/1807941 [20:57<01:50, 942.03 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1703746/1807941 [20:57<01:52, 928.70 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1703879/1807941 [20:58<01:54, 906.13 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1704000/1807941 [20:58<02:07, 812.04 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1704102/1807941 [20:58<02:01, 856.58 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1704191/1807941 [20:58<02:00, 864.19 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1704314/1807941 [20:58<02:02, 847.11 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1704408/1807941 [20:58<01:59, 867.91 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1704527/1807941 [20:58<02:03, 839.27 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1704651/1807941 [20:58<02:04, 830.80 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1704769/1807941 [20:59<02:06, 814.87 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1704893/1807941 [20:59<02:06, 814.71 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1704986/1807941 [20:59<02:02, 840.14 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1705078/1807941 [20:59<02:15, 761.07 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1705176/1807941 [20:59<02:06, 811.00 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1705260/1807941 [20:59<02:05, 816.14 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1705356/1807941 [20:59<02:00, 851.58 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1705443/1807941 [20:59<02:00, 853.38 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1705533/1807941 [21:00<01:58, 865.07 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1705621/1807941 [21:00<01:58, 863.93 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1705731/1807941 [21:00<02:06, 807.35 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1705816/1807941 [21:00<02:05, 815.27 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1705939/1807941 [21:00<02:05, 814.96 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1706034/1807941 [21:00<02:17, 743.11 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1706117/1807941 [21:00<02:14, 759.25 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1706208/1807941 [21:00<02:08, 792.69 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1706315/1807941 [21:01<02:13, 758.53 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1706421/1807941 [21:01<02:17, 736.95 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1706513/1807941 [21:01<02:10, 776.43 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1706595/1807941 [21:01<02:09, 785.48 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1706677/1807941 [21:01<02:07, 792.69 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1706799/1807941 [21:01<02:06, 796.99 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1706922/1807941 [21:01<02:06, 797.83 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1707038/1807941 [21:01<02:14, 752.02 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1707152/1807941 [21:02<02:14, 748.26 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1707242/1807941 [21:02<02:09, 779.56 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1707361/1807941 [21:02<02:08, 782.15 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1707452/1807941 [21:02<02:04, 808.98 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1707571/1807941 [21:02<02:05, 800.57 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1707691/1807941 [21:02<02:06, 794.13 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1707784/1807941 [21:02<02:01, 823.23 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1707880/1807941 [21:03<01:57, 854.22 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1707990/1807941 [21:03<02:03, 808.25 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1708098/1807941 [21:03<02:08, 776.91 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1708183/1807941 [21:03<02:05, 793.00 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1708270/1807941 [21:03<02:03, 810.04 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1708392/1807941 [21:03<02:04, 801.75 examples/s]
Tokenizing train dataset:  94%|█████████▍| 1708477/1807941 [21:03<02:02, 811.63 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1708572/1807941 [21:03<01:57, 845.44 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1708665/1807941 [21:03<01:54, 864.66 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1708791/1807941 [21:04<01:56, 852.18 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1708916/1807941 [21:04<01:59, 825.80 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1709000/1807941 [21:04<02:07, 775.18 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1709081/1807941 [21:04<02:06, 779.90 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1709186/1807941 [21:04<02:12, 746.52 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1709270/1807941 [21:04<02:08, 765.45 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1709381/1807941 [21:04<02:10, 754.63 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1709501/1807941 [21:05<02:08, 764.87 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1709586/1807941 [21:05<02:06, 779.81 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1709698/1807941 [21:05<02:08, 764.04 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1709786/1807941 [21:05<02:05, 784.97 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1709868/1807941 [21:05<02:03, 793.67 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1709952/1807941 [21:05<02:01, 803.37 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1710047/1807941 [21:05<02:12, 738.16 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1710159/1807941 [21:05<02:12, 738.26 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1710242/1807941 [21:06<02:08, 758.72 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1710353/1807941 [21:06<02:10, 747.53 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1710450/1807941 [21:06<02:17, 710.21 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1710527/1807941 [21:06<02:15, 721.52 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1710615/1807941 [21:06<02:08, 758.38 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1710725/1807941 [21:06<02:10, 746.67 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1710835/1807941 [21:06<02:11, 736.30 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1710912/1807941 [21:06<02:10, 741.86 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1710989/1807941 [21:07<02:09, 746.06 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1711084/1807941 [21:07<02:17, 703.11 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1711157/1807941 [21:07<02:17, 706.35 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1711229/1807941 [21:07<02:16, 705.94 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1711309/1807941 [21:07<02:12, 728.29 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1711386/1807941 [21:07<02:11, 736.19 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1711497/1807941 [21:07<02:11, 732.82 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1711586/1807941 [21:07<02:04, 772.98 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1711695/1807941 [21:08<02:07, 753.33 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1711807/1807941 [21:08<02:08, 748.40 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1711921/1807941 [21:08<02:08, 747.26 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1712000/1807941 [21:08<02:13, 717.63 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1712081/1807941 [21:08<02:10, 737.34 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1712171/1807941 [21:08<02:03, 773.96 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1712262/1807941 [21:08<01:59, 803.81 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1712384/1807941 [21:08<02:05, 763.05 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1712477/1807941 [21:09<01:59, 800.41 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1712566/1807941 [21:09<01:56, 818.79 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1712685/1807941 [21:09<01:58, 804.44 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1712786/1807941 [21:09<01:51, 854.92 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1712874/1807941 [21:09<01:51, 856.31 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1712968/1807941 [21:09<01:48, 876.66 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1713087/1807941 [21:09<01:58, 802.16 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1713206/1807941 [21:09<01:59, 790.69 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1713315/1807941 [21:10<02:03, 766.47 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1713394/1807941 [21:10<02:02, 770.61 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1713503/1807941 [21:10<02:05, 754.16 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1713611/1807941 [21:10<02:07, 739.86 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1713726/1807941 [21:10<02:06, 743.13 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1713802/1807941 [21:10<02:07, 740.57 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1713878/1807941 [21:10<02:06, 742.70 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1713957/1807941 [21:10<02:05, 751.34 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1714046/1807941 [21:11<02:15, 692.73 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1714123/1807941 [21:11<02:12, 707.98 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1714200/1807941 [21:11<02:09, 722.24 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1714316/1807941 [21:11<02:06, 737.59 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1714393/1807941 [21:11<02:05, 742.82 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1714484/1807941 [21:11<01:59, 784.26 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1714603/1807941 [21:11<02:00, 775.61 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1714686/1807941 [21:11<01:58, 787.99 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1714780/1807941 [21:12<01:52, 825.51 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1714890/1807941 [21:12<01:57, 789.38 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1714981/1807941 [21:12<02:08, 723.81 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715077/1807941 [21:12<02:29, 620.46 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715152/1807941 [21:12<02:25, 638.03 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715237/1807941 [21:12<02:17, 672.47 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715317/1807941 [21:12<02:12, 701.41 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715413/1807941 [21:12<02:16, 678.69 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715491/1807941 [21:13<02:12, 698.82 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715566/1807941 [21:13<02:10, 708.94 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715643/1807941 [21:13<02:08, 717.02 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715753/1807941 [21:13<02:07, 721.87 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715862/1807941 [21:13<02:07, 721.60 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1715947/1807941 [21:13<02:02, 748.11 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1716034/1807941 [21:13<02:13, 686.32 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1716137/1807941 [21:14<02:15, 675.55 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1716210/1807941 [21:14<02:14, 684.24 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1716281/1807941 [21:14<02:13, 688.61 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1716365/1807941 [21:14<02:05, 727.31 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1716450/1807941 [21:14<02:18, 662.89 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1716527/1807941 [21:14<02:12, 687.76 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1716679/1807941 [21:14<01:41, 903.26 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1716830/1807941 [21:14<01:25, 1067.39 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1716991/1807941 [21:14<01:14, 1219.03 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1717198/1807941 [21:15<01:11, 1277.21 examples/s]
Tokenizing train dataset:  95%|█████████▍| 1717389/1807941 [21:15<01:02, 1441.01 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1717591/1807941 [21:15<00:56, 1595.87 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1717789/1807941 [21:15<00:53, 1700.23 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1717985/1807941 [21:15<00:50, 1770.39 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1718215/1807941 [21:15<00:53, 1678.97 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1718521/1807941 [21:15<00:43, 2042.53 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1718818/1807941 [21:15<00:38, 2294.38 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1719063/1807941 [21:15<00:43, 2046.39 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1719280/1807941 [21:16<00:53, 1642.59 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1719513/1807941 [21:16<01:00, 1460.53 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1719683/1807941 [21:16<01:04, 1364.05 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1719910/1807941 [21:16<00:56, 1554.59 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1720189/1807941 [21:16<00:54, 1624.37 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1720480/1807941 [21:16<00:45, 1914.33 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1720861/1807941 [21:16<00:36, 2376.01 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1721168/1807941 [21:17<00:38, 2260.33 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1721498/1807941 [21:17<00:38, 2236.78 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1721827/1807941 [21:17<00:38, 2218.29 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1722118/1807941 [21:17<00:40, 2128.97 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1722447/1807941 [21:17<00:39, 2145.20 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1722766/1807941 [21:17<00:39, 2137.10 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1723019/1807941 [21:17<00:42, 1997.16 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1723273/1807941 [21:18<00:49, 1726.42 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1723499/1807941 [21:18<00:55, 1510.99 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1723689/1807941 [21:18<01:04, 1314.80 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1723839/1807941 [21:19<01:45, 796.87 examples/s] 
Tokenizing train dataset:  95%|█████████▌| 1723982/1807941 [21:19<01:35, 883.22 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1724144/1807941 [21:19<01:30, 926.01 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1724301/1807941 [21:19<01:20, 1038.96 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1724436/1807941 [21:19<01:15, 1100.47 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1724574/1807941 [21:19<01:11, 1160.27 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1724722/1807941 [21:19<01:07, 1235.52 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1724877/1807941 [21:19<01:03, 1313.76 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1725067/1807941 [21:19<01:04, 1283.45 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1725213/1807941 [21:20<01:02, 1325.38 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1725363/1807941 [21:20<01:00, 1369.23 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1725571/1807941 [21:20<00:59, 1374.33 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1725715/1807941 [21:20<00:59, 1387.30 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1725869/1807941 [21:20<00:57, 1424.56 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1726071/1807941 [21:20<00:59, 1371.18 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1726211/1807941 [21:20<00:59, 1376.55 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1726359/1807941 [21:20<00:58, 1400.61 examples/s]
Tokenizing train dataset:  95%|█████████▌| 1726511/1807941 [21:20<00:56, 1431.91 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1726668/1807941 [21:21<01:28, 913.45 examples/s] 
Tokenizing train dataset:  96%|█████████▌| 1726848/1807941 [21:21<01:29, 904.34 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1726993/1807941 [21:21<01:20, 1007.36 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1727167/1807941 [21:21<01:16, 1049.08 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1727309/1807941 [21:21<01:11, 1128.57 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1727453/1807941 [21:21<01:07, 1199.96 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1727599/1807941 [21:22<01:36, 828.31 examples/s] 
Tokenizing train dataset:  96%|█████████▌| 1727735/1807941 [21:22<01:26, 928.67 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1727870/1807941 [21:22<01:18, 1017.44 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1728000/1807941 [21:22<01:17, 1037.98 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1728134/1807941 [21:22<01:11, 1109.52 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1728262/1807941 [21:22<01:09, 1151.61 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1728396/1807941 [21:22<01:06, 1199.09 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1728538/1807941 [21:23<01:03, 1259.36 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1728687/1807941 [21:23<00:59, 1324.03 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1728832/1807941 [21:23<00:58, 1357.16 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1728978/1807941 [21:23<00:57, 1385.22 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1729154/1807941 [21:23<01:00, 1301.05 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1729288/1807941 [21:23<01:00, 1307.97 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1729482/1807941 [21:23<01:00, 1297.95 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1729683/1807941 [21:23<00:59, 1306.83 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1729825/1807941 [21:23<00:58, 1332.77 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1730000/1807941 [21:24<01:03, 1227.72 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1730135/1807941 [21:24<01:01, 1255.29 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1730272/1807941 [21:24<01:00, 1282.10 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1730414/1807941 [21:24<00:58, 1317.38 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1730603/1807941 [21:24<01:29, 865.59 examples/s] 
Tokenizing train dataset:  96%|█████████▌| 1730730/1807941 [21:25<01:53, 677.78 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1730837/1807941 [21:25<01:52, 683.31 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1730978/1807941 [21:25<01:35, 810.01 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1731135/1807941 [21:25<01:27, 873.20 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1731274/1807941 [21:25<01:18, 979.05 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1731407/1807941 [21:25<01:12, 1055.62 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1731545/1807941 [21:25<01:07, 1133.22 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1731676/1807941 [21:25<01:04, 1174.46 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1731812/1807941 [21:26<01:02, 1222.09 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1731943/1807941 [21:26<01:01, 1244.72 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1732132/1807941 [21:26<01:02, 1212.69 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1732309/1807941 [21:26<01:03, 1199.10 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1732485/1807941 [21:26<01:03, 1189.37 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1732616/1807941 [21:26<01:02, 1213.85 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1732741/1807941 [21:26<01:01, 1219.77 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1732870/1807941 [21:26<01:00, 1234.85 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1733000/1807941 [21:26<01:03, 1185.33 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1733120/1807941 [21:27<01:03, 1186.31 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1733243/1807941 [21:27<01:02, 1195.44 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1733370/1807941 [21:27<01:01, 1211.31 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1733509/1807941 [21:27<00:58, 1262.37 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1733704/1807941 [21:27<00:58, 1272.32 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1733840/1807941 [21:27<00:57, 1292.57 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1733986/1807941 [21:27<00:55, 1336.71 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1734126/1807941 [21:27<01:02, 1177.77 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1734250/1807941 [21:28<01:01, 1191.60 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1734425/1807941 [21:28<01:02, 1177.28 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1734554/1807941 [21:28<01:00, 1203.84 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1734744/1807941 [21:28<00:59, 1224.42 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1734877/1807941 [21:28<00:58, 1247.56 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1735023/1807941 [21:28<01:03, 1151.62 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1735151/1807941 [21:28<01:01, 1177.02 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1735274/1807941 [21:28<01:01, 1186.22 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1735406/1807941 [21:28<00:59, 1218.26 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1735588/1807941 [21:29<00:59, 1212.36 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1735720/1807941 [21:29<00:58, 1237.79 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1735909/1807941 [21:29<00:57, 1242.04 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1736074/1807941 [21:29<01:00, 1192.25 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1736199/1807941 [21:29<00:59, 1203.89 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1736343/1807941 [21:29<00:56, 1257.82 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1736478/1807941 [21:30<01:26, 824.15 examples/s] 
Tokenizing train dataset:  96%|█████████▌| 1736640/1807941 [21:30<01:27, 813.67 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1736778/1807941 [21:30<01:17, 919.91 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1736912/1807941 [21:30<01:10, 1008.43 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1737078/1807941 [21:30<01:08, 1036.26 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1737219/1807941 [21:30<01:03, 1120.67 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1737359/1807941 [21:30<00:59, 1188.35 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1737499/1807941 [21:30<00:56, 1239.96 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1737648/1807941 [21:30<00:53, 1304.94 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1737822/1807941 [21:31<01:03, 1106.02 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1737953/1807941 [21:31<01:00, 1153.07 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1738124/1807941 [21:31<01:02, 1122.48 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1738296/1807941 [21:31<01:01, 1127.63 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1738466/1807941 [21:31<01:01, 1126.53 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1738594/1807941 [21:31<00:59, 1156.44 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1738717/1807941 [21:31<00:59, 1172.11 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1738852/1807941 [21:32<00:56, 1216.34 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1739000/1807941 [21:32<01:01, 1117.48 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1739159/1807941 [21:32<01:02, 1094.69 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1739287/1807941 [21:32<01:00, 1136.53 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1739412/1807941 [21:32<00:58, 1162.58 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1739538/1807941 [21:32<00:57, 1185.97 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1739670/1807941 [21:32<00:56, 1213.82 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1739796/1807941 [21:32<00:55, 1223.15 examples/s]
Tokenizing train dataset:  96%|█████████▌| 1739986/1807941 [21:33<00:55, 1234.66 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1740150/1807941 [21:33<00:57, 1182.38 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1740284/1807941 [21:33<00:55, 1217.62 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1740409/1807941 [21:33<00:55, 1221.27 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1740546/1807941 [21:33<00:53, 1258.47 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1740715/1807941 [21:33<00:55, 1204.74 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1740843/1807941 [21:33<00:54, 1222.30 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1740975/1807941 [21:33<00:53, 1246.46 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1741134/1807941 [21:33<00:57, 1169.57 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1741268/1807941 [21:34<00:55, 1206.21 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1741404/1807941 [21:34<01:00, 1097.54 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1741556/1807941 [21:34<00:55, 1200.98 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1741702/1807941 [21:34<01:06, 993.66 examples/s] 
Tokenizing train dataset:  96%|█████████▋| 1741868/1807941 [21:34<00:57, 1140.32 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1742004/1807941 [21:34<01:02, 1062.20 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1742132/1807941 [21:34<01:06, 992.16 examples/s] 
Tokenizing train dataset:  96%|█████████▋| 1742259/1807941 [21:35<01:09, 944.81 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1742389/1807941 [21:35<01:11, 912.34 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1742484/1807941 [21:35<01:11, 915.20 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1742641/1807941 [21:35<01:08, 951.52 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1742747/1807941 [21:35<01:06, 975.77 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1742873/1807941 [21:35<01:02, 1044.81 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1743000/1807941 [21:35<01:02, 1034.75 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1743124/1807941 [21:35<00:59, 1085.65 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1743241/1807941 [21:36<00:58, 1104.56 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1743365/1807941 [21:36<00:56, 1139.87 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1743489/1807941 [21:36<00:55, 1167.38 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1743613/1807941 [21:36<00:54, 1181.59 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1743736/1807941 [21:36<00:53, 1192.26 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1743860/1807941 [21:36<00:53, 1205.08 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1744000/1807941 [21:36<00:59, 1076.49 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1744135/1807941 [21:36<01:18, 810.07 examples/s] 
Tokenizing train dataset:  96%|█████████▋| 1744259/1807941 [21:37<01:10, 899.48 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1744381/1807941 [21:37<01:05, 970.98 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1744511/1807941 [21:37<01:00, 1050.14 examples/s]
Tokenizing train dataset:  96%|█████████▋| 1744636/1807941 [21:37<00:57, 1100.14 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1744763/1807941 [21:37<00:55, 1143.73 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1744930/1807941 [21:37<00:55, 1130.39 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1745068/1807941 [21:37<01:00, 1045.49 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1745182/1807941 [21:37<00:58, 1064.86 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1745292/1807941 [21:37<00:58, 1069.29 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1745456/1807941 [21:38<00:58, 1074.93 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1745585/1807941 [21:38<00:55, 1126.85 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1745703/1807941 [21:38<00:54, 1140.28 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1745829/1807941 [21:38<00:53, 1169.67 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1745999/1807941 [21:38<00:53, 1152.71 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1746146/1807941 [21:38<00:56, 1086.58 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1746258/1807941 [21:38<00:56, 1093.48 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1746372/1807941 [21:38<00:55, 1100.89 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1746486/1807941 [21:39<00:55, 1109.21 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1746598/1807941 [21:39<00:55, 1106.57 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1746721/1807941 [21:39<00:53, 1138.57 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1746852/1807941 [21:39<00:51, 1185.96 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1747001/1807941 [21:39<00:54, 1108.23 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1747164/1807941 [21:39<00:55, 1096.95 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1747277/1807941 [21:39<00:54, 1104.29 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1747438/1807941 [21:39<00:55, 1090.37 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1747609/1807941 [21:40<00:54, 1105.40 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1747737/1807941 [21:40<00:52, 1145.20 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1747854/1807941 [21:40<00:52, 1149.20 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1747978/1807941 [21:40<00:51, 1170.21 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1748112/1807941 [21:40<00:56, 1066.33 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1748231/1807941 [21:40<00:54, 1096.17 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1748353/1807941 [21:40<00:52, 1125.25 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1748472/1807941 [21:40<00:52, 1139.16 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1748597/1807941 [21:40<00:50, 1168.31 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1748771/1807941 [21:41<00:50, 1160.77 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1748904/1807941 [21:41<00:49, 1201.69 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1749062/1807941 [21:41<00:51, 1142.79 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1749236/1807941 [21:41<00:51, 1147.28 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1749419/1807941 [21:41<00:50, 1167.45 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1749548/1807941 [21:41<00:48, 1194.99 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1749677/1807941 [21:41<00:47, 1214.51 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1749856/1807941 [21:41<00:48, 1201.91 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1750000/1807941 [21:42<00:54, 1072.64 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1750118/1807941 [21:42<00:59, 977.80 examples/s] 
Tokenizing train dataset:  97%|█████████▋| 1750219/1807941 [21:42<01:05, 874.61 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1750334/1807941 [21:42<01:23, 693.33 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1750441/1807941 [21:42<01:22, 693.77 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1750585/1807941 [21:42<01:15, 763.43 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1750732/1807941 [21:43<01:03, 906.00 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1750878/1807941 [21:43<00:55, 1028.54 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1751000/1807941 [21:43<00:55, 1024.55 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1751148/1807941 [21:43<00:49, 1136.01 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1751297/1807941 [21:43<00:46, 1225.66 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1751451/1807941 [21:43<00:43, 1307.42 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1751604/1807941 [21:43<00:41, 1366.99 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1751750/1807941 [21:43<00:40, 1390.67 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1751902/1807941 [21:43<00:39, 1423.65 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1752096/1807941 [21:44<00:40, 1371.10 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1752240/1807941 [21:44<00:40, 1386.33 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1752388/1807941 [21:44<00:39, 1409.81 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1752537/1807941 [21:44<00:38, 1428.74 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1752687/1807941 [21:44<00:38, 1445.33 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1752904/1807941 [21:44<00:38, 1442.37 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1753091/1807941 [21:44<00:40, 1371.16 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1753235/1807941 [21:44<00:39, 1386.03 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1753380/1807941 [21:44<00:38, 1400.63 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1753523/1807941 [21:45<00:38, 1405.63 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1753669/1807941 [21:45<00:38, 1420.06 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1753875/1807941 [21:45<00:38, 1399.17 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1754065/1807941 [21:45<00:41, 1298.24 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1754248/1807941 [21:45<00:42, 1269.76 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1754433/1807941 [21:45<00:42, 1254.25 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1754567/1807941 [21:45<00:41, 1271.87 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1754699/1807941 [21:46<00:41, 1281.19 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1754834/1807941 [21:46<00:40, 1295.59 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1754965/1807941 [21:46<00:40, 1297.83 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1755127/1807941 [21:46<00:43, 1205.85 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1755256/1807941 [21:46<00:43, 1224.62 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1755398/1807941 [21:46<00:41, 1274.70 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1755534/1807941 [21:46<00:40, 1295.48 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1755679/1807941 [21:46<00:39, 1334.55 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1755877/1807941 [21:46<00:39, 1323.02 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1756031/1807941 [21:47<00:42, 1216.16 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1756164/1807941 [21:47<00:41, 1242.61 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1756307/1807941 [21:47<00:40, 1287.56 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1756456/1807941 [21:47<00:49, 1050.36 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1756585/1807941 [21:47<00:46, 1098.78 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1756725/1807941 [21:47<00:49, 1042.19 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1756863/1807941 [21:47<00:51, 1001.46 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1756978/1807941 [21:48<01:07, 754.16 examples/s] 
Tokenizing train dataset:  97%|█████████▋| 1757094/1807941 [21:48<01:09, 733.11 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1757203/1807941 [21:48<01:09, 729.80 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1757291/1807941 [21:48<01:06, 758.98 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1757415/1807941 [21:48<00:58, 864.93 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1757532/1807941 [21:48<01:00, 831.46 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1757653/1807941 [21:48<01:01, 821.32 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1757780/1807941 [21:49<01:00, 825.55 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1757871/1807941 [21:49<00:59, 840.47 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1757968/1807941 [21:49<01:04, 772.94 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1758066/1807941 [21:49<01:11, 700.91 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1758150/1807941 [21:49<01:15, 655.70 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1758225/1807941 [21:49<01:13, 675.66 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1758318/1807941 [21:49<01:07, 734.90 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1758455/1807941 [21:50<00:55, 894.40 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1758592/1807941 [21:50<00:48, 1019.45 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1758728/1807941 [21:50<00:44, 1110.94 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1758871/1807941 [21:50<00:40, 1197.56 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1759000/1807941 [21:50<00:42, 1158.34 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1759137/1807941 [21:50<00:40, 1215.85 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1759261/1807941 [21:50<00:39, 1221.66 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1759400/1807941 [21:50<00:44, 1101.96 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1759537/1807941 [21:50<00:52, 914.62 examples/s] 
Tokenizing train dataset:  97%|█████████▋| 1759683/1807941 [21:51<00:46, 1036.83 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1759873/1807941 [21:51<00:43, 1108.08 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1760033/1807941 [21:51<00:43, 1092.27 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1760175/1807941 [21:51<00:41, 1163.09 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1760308/1807941 [21:51<00:39, 1202.60 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1760472/1807941 [21:51<00:40, 1160.06 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1760597/1807941 [21:51<00:40, 1179.41 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1760745/1807941 [21:52<00:52, 896.66 examples/s] 
Tokenizing train dataset:  97%|█████████▋| 1760865/1807941 [21:52<00:49, 953.19 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1761000/1807941 [21:52<01:09, 679.99 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1761131/1807941 [21:52<00:59, 789.57 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1761246/1807941 [21:52<00:59, 781.36 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1761354/1807941 [21:52<00:55, 840.91 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1761469/1807941 [21:53<01:02, 739.74 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1761634/1807941 [21:53<00:49, 927.89 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1761801/1807941 [21:53<00:46, 981.72 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1761916/1807941 [21:53<00:45, 1018.06 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1762075/1807941 [21:53<00:42, 1079.07 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1762254/1807941 [21:53<00:41, 1113.98 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1762421/1807941 [21:53<00:41, 1107.64 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1762579/1807941 [21:54<00:41, 1086.00 examples/s]
Tokenizing train dataset:  97%|█████████▋| 1762705/1807941 [21:54<00:45, 1000.03 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1762826/1807941 [21:54<00:48, 930.42 examples/s] 
Tokenizing train dataset:  98%|█████████▊| 1762934/1807941 [21:54<00:57, 784.03 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1763048/1807941 [21:54<01:03, 707.67 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1763168/1807941 [21:54<00:55, 801.63 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1763285/1807941 [21:55<00:56, 784.60 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1763395/1807941 [21:55<01:03, 699.71 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1763487/1807941 [21:55<00:59, 743.35 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1763578/1807941 [21:55<00:57, 773.47 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1763676/1807941 [21:55<00:54, 818.89 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1763795/1807941 [21:55<00:54, 808.94 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1763886/1807941 [21:55<00:52, 832.54 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1763976/1807941 [21:55<00:51, 846.76 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1764100/1807941 [21:56<01:00, 723.50 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1764202/1807941 [21:56<00:55, 788.17 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1764309/1807941 [21:56<00:50, 856.55 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1764412/1807941 [21:56<00:55, 787.16 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1764501/1807941 [21:56<01:00, 715.92 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1764579/1807941 [21:56<00:59, 727.02 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1764687/1807941 [21:56<01:00, 718.25 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1764802/1807941 [21:57<00:59, 730.73 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1764892/1807941 [21:57<00:56, 768.26 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1764984/1807941 [21:57<00:53, 804.86 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1765086/1807941 [21:57<00:56, 755.44 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1765179/1807941 [21:57<01:07, 632.91 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1765261/1807941 [21:57<01:03, 672.47 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1765361/1807941 [21:57<00:57, 746.99 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1765463/1807941 [21:57<01:06, 642.69 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1765604/1807941 [21:58<00:58, 727.29 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1765714/1807941 [21:58<00:52, 804.10 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1765840/1807941 [21:58<00:46, 906.88 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1765953/1807941 [21:58<00:43, 954.80 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1766062/1807941 [21:58<00:48, 856.38 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1766183/1807941 [21:58<00:49, 837.06 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1766296/1807941 [21:58<00:46, 904.50 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1766429/1807941 [21:58<00:46, 896.01 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1766526/1807941 [21:59<01:02, 667.44 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1766616/1807941 [21:59<00:57, 712.93 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1766762/1807941 [21:59<00:46, 878.49 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1766870/1807941 [21:59<01:00, 676.75 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1766966/1807941 [21:59<01:01, 665.49 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1767080/1807941 [22:00<01:04, 629.74 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1767155/1807941 [22:00<01:02, 651.85 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1767255/1807941 [22:00<01:02, 650.33 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1767354/1807941 [22:00<01:02, 652.14 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1767488/1807941 [22:00<00:50, 802.24 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1767582/1807941 [22:00<00:48, 830.32 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1767675/1807941 [22:00<00:59, 678.72 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1767792/1807941 [22:01<01:03, 635.10 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1767887/1807941 [22:01<01:03, 633.05 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1767969/1807941 [22:01<00:59, 670.56 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1768062/1807941 [22:01<01:05, 610.21 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1768135/1807941 [22:01<01:02, 634.57 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1768203/1807941 [22:01<01:01, 644.47 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1768275/1807941 [22:01<01:00, 656.52 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1768378/1807941 [22:01<00:52, 749.31 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1768474/1807941 [22:02<01:09, 565.41 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1768554/1807941 [22:02<01:04, 609.38 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1768657/1807941 [22:02<01:02, 630.19 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1768743/1807941 [22:02<00:57, 676.16 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1768829/1807941 [22:02<00:54, 719.26 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1768923/1807941 [22:02<00:50, 775.14 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1769025/1807941 [22:02<00:53, 724.18 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1769127/1807941 [22:03<00:55, 702.98 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1769218/1807941 [22:03<00:57, 669.60 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1769329/1807941 [22:03<00:50, 767.95 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1769429/1807941 [22:03<00:59, 652.10 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1769501/1807941 [22:03<01:10, 542.89 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1769594/1807941 [22:03<01:08, 563.06 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1769692/1807941 [22:04<01:11, 532.39 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1769768/1807941 [22:04<01:06, 576.69 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1769878/1807941 [22:04<00:55, 688.52 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1769979/1807941 [22:04<00:49, 764.40 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1770084/1807941 [22:04<00:52, 720.70 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1770214/1807941 [22:04<00:49, 762.66 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1770295/1807941 [22:04<00:54, 686.07 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1770385/1807941 [22:05<01:03, 593.86 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1770463/1807941 [22:05<01:12, 514.95 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1770546/1807941 [22:05<01:05, 572.77 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1770648/1807941 [22:05<01:02, 600.07 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1770756/1807941 [22:05<00:58, 633.52 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1770832/1807941 [22:05<00:56, 659.90 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1770920/1807941 [22:05<00:52, 706.55 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1771000/1807941 [22:06<00:57, 640.47 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1771087/1807941 [22:06<00:59, 616.27 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1771169/1807941 [22:06<01:02, 590.43 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1771264/1807941 [22:06<00:54, 672.07 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1771408/1807941 [22:06<00:42, 859.55 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1771540/1807941 [22:06<00:37, 977.67 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1771681/1807941 [22:06<00:33, 1092.83 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1771799/1807941 [22:06<00:37, 972.04 examples/s] 
Tokenizing train dataset:  98%|█████████▊| 1771939/1807941 [22:07<00:42, 850.96 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1772034/1807941 [22:07<01:00, 591.95 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1772141/1807941 [22:07<00:58, 613.71 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1772219/1807941 [22:07<01:00, 586.34 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1772297/1807941 [22:07<00:57, 620.67 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1772375/1807941 [22:07<00:54, 651.19 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1772479/1807941 [22:08<00:47, 741.41 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1772601/1807941 [22:08<00:41, 860.85 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1772696/1807941 [22:08<00:40, 880.69 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1772811/1807941 [22:08<00:36, 951.55 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1772924/1807941 [22:08<00:40, 873.24 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773039/1807941 [22:08<00:47, 732.18 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773170/1807941 [22:08<00:45, 771.08 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773254/1807941 [22:09<00:49, 695.72 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773332/1807941 [22:09<01:00, 576.40 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773402/1807941 [22:09<01:09, 495.19 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773474/1807941 [22:09<01:10, 487.35 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773527/1807941 [22:09<01:09, 494.62 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773582/1807941 [22:09<01:08, 500.68 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773646/1807941 [22:09<01:04, 530.72 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773731/1807941 [22:09<00:56, 606.39 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773860/1807941 [22:10<00:43, 783.31 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1773956/1807941 [22:10<00:46, 728.25 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774054/1807941 [22:10<00:57, 588.95 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774136/1807941 [22:10<00:58, 575.22 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774207/1807941 [22:10<01:02, 541.38 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774291/1807941 [22:10<00:55, 604.30 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774370/1807941 [22:10<00:52, 641.58 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774449/1807941 [22:11<01:02, 539.30 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774511/1807941 [22:11<01:14, 449.48 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774580/1807941 [22:11<01:07, 493.76 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774652/1807941 [22:11<01:08, 484.98 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774731/1807941 [22:11<01:07, 490.51 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774804/1807941 [22:11<01:07, 487.43 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774873/1807941 [22:12<01:02, 528.26 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1774944/1807941 [22:12<00:57, 569.90 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775029/1807941 [22:12<01:03, 522.24 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775088/1807941 [22:12<01:01, 536.46 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775155/1807941 [22:12<01:05, 501.59 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775222/1807941 [22:12<01:00, 540.26 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775281/1807941 [22:12<00:59, 549.74 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775363/1807941 [22:12<00:59, 546.70 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775439/1807941 [22:13<01:01, 529.86 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775495/1807941 [22:13<01:00, 532.54 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775584/1807941 [22:13<00:52, 621.78 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775667/1807941 [22:13<00:47, 674.04 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775747/1807941 [22:13<00:59, 543.93 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775808/1807941 [22:13<01:12, 445.41 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775866/1807941 [22:14<01:15, 424.36 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775932/1807941 [22:14<01:22, 386.32 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1775983/1807941 [22:14<01:18, 407.70 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776053/1807941 [22:14<01:25, 373.37 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776111/1807941 [22:14<01:17, 411.68 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776167/1807941 [22:14<01:11, 443.70 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776230/1807941 [22:14<01:13, 432.66 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776279/1807941 [22:15<01:11, 444.57 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776328/1807941 [22:15<01:09, 454.81 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776398/1807941 [22:15<01:09, 455.68 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776451/1807941 [22:15<01:06, 473.13 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776503/1807941 [22:15<01:05, 483.31 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776557/1807941 [22:15<01:03, 495.58 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776627/1807941 [22:15<01:05, 481.06 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776681/1807941 [22:15<01:03, 495.32 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776752/1807941 [22:15<00:57, 544.04 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776825/1807941 [22:16<01:00, 514.65 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776903/1807941 [22:16<01:00, 511.54 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1776961/1807941 [22:16<00:59, 522.85 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777019/1807941 [22:16<01:08, 453.31 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777068/1807941 [22:16<01:07, 459.57 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777118/1807941 [22:16<01:05, 468.87 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777171/1807941 [22:16<01:03, 481.51 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777229/1807941 [22:16<01:01, 499.77 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777316/1807941 [22:17<00:50, 601.41 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777386/1807941 [22:17<00:56, 541.01 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777443/1807941 [22:17<00:56, 543.75 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777507/1807941 [22:17<01:01, 497.41 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777587/1807941 [22:17<01:00, 505.71 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777646/1807941 [22:17<00:57, 524.41 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777700/1807941 [22:17<00:57, 523.91 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777762/1807941 [22:17<00:55, 546.65 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777839/1807941 [22:18<00:57, 527.40 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1777915/1807941 [22:18<00:58, 513.54 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778000/1807941 [22:18<01:02, 482.70 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778098/1807941 [22:18<00:50, 589.46 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778190/1807941 [22:18<00:49, 595.59 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778268/1807941 [22:18<00:52, 568.54 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778355/1807941 [22:19<00:57, 514.32 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778415/1807941 [22:19<01:02, 475.22 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778480/1807941 [22:19<01:10, 418.25 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778531/1807941 [22:19<01:07, 434.89 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778591/1807941 [22:19<01:09, 421.71 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778680/1807941 [22:19<01:02, 467.22 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778739/1807941 [22:19<00:59, 491.11 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778803/1807941 [22:20<01:02, 467.73 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778869/1807941 [22:20<01:04, 451.74 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778935/1807941 [22:20<01:05, 446.20 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1778985/1807941 [22:20<01:03, 455.67 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779059/1807941 [22:20<01:17, 373.57 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779120/1807941 [22:20<01:16, 379.08 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779182/1807941 [22:21<01:14, 385.81 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779225/1807941 [22:21<01:13, 390.99 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779282/1807941 [22:21<01:14, 386.52 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779337/1807941 [22:21<01:15, 377.77 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779388/1807941 [22:21<01:10, 405.64 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779443/1807941 [22:21<01:13, 387.89 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779497/1807941 [22:21<01:07, 421.04 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779569/1807941 [22:21<01:04, 436.55 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779627/1807941 [22:22<01:07, 416.90 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779677/1807941 [22:22<01:04, 434.86 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779735/1807941 [22:22<01:07, 415.04 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779790/1807941 [22:22<01:03, 445.32 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779861/1807941 [22:22<01:02, 450.78 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779917/1807941 [22:22<01:06, 418.36 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1779967/1807941 [22:22<01:04, 431.96 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780029/1807941 [22:23<01:11, 390.65 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780084/1807941 [22:23<01:13, 377.74 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780125/1807941 [22:23<01:13, 379.81 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780172/1807941 [22:23<01:09, 398.82 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780233/1807941 [22:23<01:09, 396.52 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780286/1807941 [22:23<01:13, 377.84 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780325/1807941 [22:23<01:13, 377.03 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780364/1807941 [22:23<01:12, 379.67 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780404/1807941 [22:24<01:11, 383.32 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780463/1807941 [22:24<01:11, 383.62 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780502/1807941 [22:24<01:11, 383.80 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780544/1807941 [22:24<01:09, 392.60 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780598/1807941 [22:24<01:03, 430.55 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780656/1807941 [22:24<01:06, 412.37 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780717/1807941 [22:24<01:07, 406.24 examples/s]
Tokenizing train dataset:  98%|█████████▊| 1780764/1807941 [22:24<01:04, 419.70 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1780826/1807941 [22:25<01:05, 412.13 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1780887/1807941 [22:25<01:06, 405.59 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1780947/1807941 [22:25<01:00, 448.58 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1780999/1807941 [22:25<00:58, 462.53 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781059/1807941 [22:25<01:10, 382.69 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781118/1807941 [22:25<01:10, 380.77 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781176/1807941 [22:25<01:10, 380.59 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781216/1807941 [22:26<01:09, 382.30 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781259/1807941 [22:26<01:08, 391.96 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781305/1807941 [22:26<01:05, 407.21 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781366/1807941 [22:26<00:58, 457.40 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781423/1807941 [22:26<01:01, 428.69 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781485/1807941 [22:26<01:02, 421.37 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781541/1807941 [22:26<01:05, 402.98 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781597/1807941 [22:27<01:07, 388.70 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781651/1807941 [22:27<01:09, 378.38 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781706/1807941 [22:27<01:10, 371.66 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781768/1807941 [22:27<01:08, 381.67 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781821/1807941 [22:27<01:03, 411.67 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1781924/1807941 [22:27<00:46, 559.36 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782000/1807941 [22:27<00:45, 572.80 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782066/1807941 [22:27<00:43, 592.90 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782190/1807941 [22:27<00:33, 760.72 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782272/1807941 [22:28<00:42, 597.95 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782345/1807941 [22:28<00:50, 507.78 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782419/1807941 [22:28<00:56, 455.05 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782473/1807941 [22:28<01:00, 423.42 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782534/1807941 [22:28<01:01, 415.32 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782619/1807941 [22:29<00:55, 453.05 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782679/1807941 [22:29<00:52, 478.54 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782737/1807941 [22:29<00:50, 499.69 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782795/1807941 [22:29<00:48, 518.15 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782863/1807941 [22:29<00:44, 557.69 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1782950/1807941 [22:29<00:44, 561.39 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783024/1807941 [22:29<00:52, 470.23 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783077/1807941 [22:30<00:57, 430.76 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783157/1807941 [22:30<00:53, 459.01 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783255/1807941 [22:30<00:43, 570.08 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783330/1807941 [22:30<00:40, 609.77 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783408/1807941 [22:30<00:42, 572.41 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783489/1807941 [22:30<00:48, 503.25 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783548/1807941 [22:30<00:52, 466.84 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783608/1807941 [22:31<01:00, 402.79 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783655/1807941 [22:31<01:04, 376.33 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783739/1807941 [22:31<00:56, 424.93 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783794/1807941 [22:31<01:05, 366.28 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783849/1807941 [22:31<01:06, 362.88 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783887/1807941 [22:31<01:05, 364.88 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783942/1807941 [22:32<01:06, 361.09 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1783991/1807941 [22:32<01:08, 347.97 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784034/1807941 [22:32<01:20, 297.79 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784068/1807941 [22:32<01:18, 304.08 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784104/1807941 [22:32<01:15, 314.56 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784138/1807941 [22:32<01:14, 319.32 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784192/1807941 [22:32<01:03, 374.09 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784232/1807941 [22:32<01:02, 377.29 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784277/1807941 [22:33<01:08, 345.58 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784314/1807941 [22:33<01:08, 346.53 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784365/1807941 [22:33<01:09, 340.74 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784406/1807941 [22:33<01:06, 352.79 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784444/1807941 [22:33<01:05, 356.24 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784481/1807941 [22:33<01:05, 357.40 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784559/1807941 [22:33<00:56, 411.87 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784604/1807941 [22:33<01:03, 369.70 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784656/1807941 [22:34<01:05, 356.34 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784701/1807941 [22:34<01:09, 334.85 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784738/1807941 [22:34<01:08, 340.08 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784786/1807941 [22:34<01:10, 329.37 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784835/1807941 [22:34<01:11, 324.57 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784871/1807941 [22:34<01:10, 328.74 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784905/1807941 [22:34<01:10, 329.05 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1784954/1807941 [22:35<01:10, 325.06 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1785000/1807941 [22:35<01:15, 305.29 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1785052/1807941 [22:35<01:05, 351.06 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1785099/1807941 [22:35<01:08, 334.98 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1785146/1807941 [22:35<01:09, 325.74 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1785182/1807941 [22:35<01:08, 330.82 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1785216/1807941 [22:35<01:08, 330.44 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1785250/1807941 [22:35<01:08, 330.79 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1785284/1807941 [22:36<01:08, 329.30 examples/s]
Tokenizing train dataset:  99%|█████████▊| 1785319/1807941 [22:36<01:08, 330.55 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785367/1807941 [22:36<01:09, 323.52 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785415/1807941 [22:36<01:02, 362.44 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785455/1807941 [22:36<01:01, 367.57 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785511/1807941 [22:36<01:01, 366.98 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785563/1807941 [22:36<01:03, 354.86 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785612/1807941 [22:36<01:04, 343.77 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785647/1807941 [22:37<01:05, 339.83 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785697/1807941 [22:37<01:06, 335.30 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785733/1807941 [22:37<01:05, 337.91 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785785/1807941 [22:37<01:05, 336.60 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785825/1807941 [22:37<01:03, 346.55 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785875/1807941 [22:37<01:04, 340.60 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785916/1807941 [22:37<01:02, 354.28 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1785995/1807941 [22:37<00:47, 465.12 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786045/1807941 [22:38<01:04, 339.01 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786096/1807941 [22:38<01:05, 334.48 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786147/1807941 [22:38<01:05, 334.67 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786195/1807941 [22:38<01:06, 326.47 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786243/1807941 [22:38<01:08, 319.05 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786278/1807941 [22:38<01:07, 322.89 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786328/1807941 [22:39<01:06, 323.53 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786379/1807941 [22:39<01:06, 326.41 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786455/1807941 [22:39<00:51, 420.98 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786507/1807941 [22:39<00:54, 390.49 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786551/1807941 [22:39<00:59, 357.40 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786600/1807941 [22:39<01:01, 344.29 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786649/1807941 [22:39<01:03, 335.23 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786698/1807941 [22:40<01:04, 330.42 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786732/1807941 [22:40<01:04, 328.73 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786767/1807941 [22:40<01:03, 332.54 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786844/1807941 [22:40<00:47, 442.51 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1786962/1807941 [22:40<00:32, 637.03 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787059/1807941 [22:40<00:36, 568.11 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787202/1807941 [22:40<00:26, 768.19 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787302/1807941 [22:40<00:28, 730.40 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787389/1807941 [22:41<00:37, 548.94 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787465/1807941 [22:41<00:45, 450.96 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787533/1807941 [22:41<00:49, 415.78 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787596/1807941 [22:41<00:52, 383.87 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787645/1807941 [22:42<00:55, 365.46 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787685/1807941 [22:42<00:54, 368.61 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787736/1807941 [22:42<00:51, 394.39 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787798/1807941 [22:42<00:45, 444.38 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787854/1807941 [22:42<00:42, 470.50 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787917/1807941 [22:42<00:39, 510.13 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1787978/1807941 [22:42<00:37, 534.03 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1788035/1807941 [22:42<00:40, 491.49 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1788098/1807941 [22:42<00:37, 526.39 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1788155/1807941 [22:42<00:36, 536.49 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1788238/1807941 [22:43<00:32, 609.13 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1788302/1807941 [22:43<00:36, 533.18 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1788361/1807941 [22:43<00:36, 542.62 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1788447/1807941 [22:43<00:31, 627.00 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1788589/1807941 [22:43<00:22, 843.87 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1788724/1807941 [22:43<00:19, 984.50 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1788867/1807941 [22:43<00:17, 1109.51 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1789000/1807941 [22:43<00:17, 1055.33 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1789122/1807941 [22:44<00:19, 963.94 examples/s] 
Tokenizing train dataset:  99%|█████████▉| 1789231/1807941 [22:44<00:21, 882.98 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1789327/1807941 [22:44<00:26, 715.86 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1789438/1807941 [22:44<00:25, 718.66 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1789529/1807941 [22:44<00:29, 618.20 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1789612/1807941 [22:44<00:33, 544.34 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1789714/1807941 [22:45<00:28, 634.94 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1789787/1807941 [22:45<00:34, 533.22 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1789892/1807941 [22:45<00:31, 575.34 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1789969/1807941 [22:45<00:32, 553.30 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1790034/1807941 [22:45<00:37, 475.49 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1790117/1807941 [22:45<00:33, 539.82 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1790196/1807941 [22:46<00:36, 484.10 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1790284/1807941 [22:46<00:31, 563.69 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1790374/1807941 [22:46<00:33, 517.20 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1790462/1807941 [22:46<00:29, 589.30 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1790529/1807941 [22:46<00:32, 538.41 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1790624/1807941 [22:46<00:27, 629.58 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1790723/1807941 [22:46<00:24, 713.66 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1790829/1807941 [22:47<00:27, 631.63 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1790935/1807941 [22:47<00:23, 725.79 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1791051/1807941 [22:47<00:27, 603.77 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1791165/1807941 [22:47<00:23, 709.83 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1791255/1807941 [22:47<00:24, 668.11 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1791335/1807941 [22:47<00:26, 625.22 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1791411/1807941 [22:47<00:28, 586.44 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1791481/1807941 [22:48<00:27, 608.41 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1791560/1807941 [22:48<00:28, 578.99 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1791634/1807941 [22:48<00:26, 610.95 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1791713/1807941 [22:48<00:31, 519.62 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1791809/1807941 [22:48<00:26, 611.55 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1791961/1807941 [22:48<00:19, 825.95 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1792065/1807941 [22:48<00:18, 854.33 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1792211/1807941 [22:48<00:15, 1010.41 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1792358/1807941 [22:49<00:13, 1133.09 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1792495/1807941 [22:49<00:12, 1190.47 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1792683/1807941 [22:49<00:12, 1208.94 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1792859/1807941 [22:49<00:11, 1351.30 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1793000/1807941 [22:49<00:11, 1337.48 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1793177/1807941 [22:49<00:10, 1453.43 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1793350/1807941 [22:49<00:09, 1527.52 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1793522/1807941 [22:49<00:09, 1579.43 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1793685/1807941 [22:49<00:08, 1588.78 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1793847/1807941 [22:50<00:12, 1098.35 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1793979/1807941 [22:50<00:17, 781.26 examples/s] 
Tokenizing train dataset:  99%|█████████▉| 1794088/1807941 [22:50<00:18, 755.40 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1794196/1807941 [22:51<00:24, 549.88 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1794290/1807941 [22:51<00:24, 565.36 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1794376/1807941 [22:51<00:22, 609.39 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1794451/1807941 [22:51<00:27, 487.10 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1794544/1807941 [22:51<00:23, 562.96 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1794632/1807941 [22:51<00:21, 624.72 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1794719/1807941 [22:52<00:26, 504.55 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1794801/1807941 [22:52<00:23, 563.07 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1794891/1807941 [22:52<00:20, 632.77 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1794975/1807941 [22:52<00:19, 679.87 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1795058/1807941 [22:52<00:23, 544.36 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1795149/1807941 [22:52<00:22, 560.20 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1795248/1807941 [22:52<00:19, 650.33 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1795322/1807941 [22:52<00:18, 665.79 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1795428/1807941 [22:53<00:20, 605.90 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1795524/1807941 [22:53<00:18, 679.66 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1795609/1807941 [22:53<00:17, 711.62 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1795695/1807941 [22:53<00:20, 593.06 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1795769/1807941 [22:53<00:19, 621.13 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1795860/1807941 [22:53<00:17, 689.17 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1795940/1807941 [22:53<00:19, 629.33 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1796045/1807941 [22:54<00:19, 621.01 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1796147/1807941 [22:54<00:16, 710.64 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1796269/1807941 [22:54<00:17, 666.84 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1796377/1807941 [22:54<00:15, 755.50 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1796482/1807941 [22:54<00:15, 733.44 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1796578/1807941 [22:54<00:16, 700.62 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1796658/1807941 [22:54<00:17, 645.31 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1796750/1807941 [22:55<00:17, 631.44 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1796929/1807941 [22:55<00:12, 887.76 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1797082/1807941 [22:55<00:11, 913.73 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1797256/1807941 [22:55<00:09, 1104.46 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1797406/1807941 [22:55<00:08, 1202.25 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1797572/1807941 [22:55<00:07, 1316.47 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1797724/1807941 [22:55<00:10, 969.71 examples/s] 
Tokenizing train dataset:  99%|█████████▉| 1797850/1807941 [22:56<00:11, 842.38 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1797986/1807941 [22:56<00:10, 944.49 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1798123/1807941 [22:56<00:09, 1038.23 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1798302/1807941 [22:56<00:07, 1217.38 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1798486/1807941 [22:56<00:06, 1373.00 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1798677/1807941 [22:56<00:06, 1513.89 examples/s]
Tokenizing train dataset:  99%|█████████▉| 1798866/1807941 [22:56<00:05, 1613.07 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1799108/1807941 [22:56<00:05, 1609.31 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1799283/1807941 [22:56<00:05, 1644.01 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1799454/1807941 [22:57<00:05, 1660.40 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1799634/1807941 [22:57<00:04, 1694.65 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1799902/1807941 [22:57<00:04, 1725.62 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1800143/1807941 [22:57<00:04, 1681.54 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1800403/1807941 [22:57<00:04, 1694.51 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1800590/1807941 [22:57<00:04, 1734.45 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1800854/1807941 [22:57<00:04, 1737.50 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1801085/1807941 [22:57<00:04, 1658.42 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1801322/1807941 [22:58<00:04, 1631.77 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1801490/1807941 [22:58<00:03, 1640.67 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1801656/1807941 [22:58<00:03, 1642.62 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1801902/1807941 [22:58<00:03, 1637.33 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1802117/1807941 [22:58<00:03, 1568.20 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1802289/1807941 [22:58<00:03, 1602.33 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1802454/1807941 [22:58<00:03, 1614.11 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1802624/1807941 [22:58<00:03, 1635.67 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1802794/1807941 [22:59<00:03, 1651.97 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1803008/1807941 [22:59<00:03, 1563.33 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1803249/1807941 [22:59<00:02, 1566.05 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1803417/1807941 [22:59<00:02, 1592.85 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1803590/1807941 [22:59<00:02, 1626.01 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1803762/1807941 [22:59<00:02, 1648.76 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1803938/1807941 [22:59<00:02, 1678.26 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1804174/1807941 [22:59<00:02, 1596.20 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1804349/1807941 [23:00<00:02, 1634.94 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1804524/1807941 [23:00<00:02, 1644.14 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1804701/1807941 [23:00<00:01, 1676.25 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1804880/1807941 [23:00<00:01, 1706.12 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1805106/1807941 [23:00<00:01, 1628.25 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1805279/1807941 [23:00<00:01, 1652.65 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1805456/1807941 [23:00<00:01, 1681.73 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1805637/1807941 [23:01<00:02, 998.75 examples/s] 
Tokenizing train dataset: 100%|█████████▉| 1805781/1807941 [23:01<00:02, 772.41 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1805922/1807941 [23:01<00:02, 749.38 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1806045/1807941 [23:01<00:02, 660.25 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1806142/1807941 [23:01<00:02, 656.41 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1806246/1807941 [23:02<00:02, 662.78 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1806351/1807941 [23:02<00:02, 667.97 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1806433/1807941 [23:02<00:02, 696.50 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1806599/1807941 [23:02<00:01, 904.05 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1806735/1807941 [23:02<00:01, 896.27 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1806907/1807941 [23:02<00:00, 1085.04 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1807111/1807941 [23:02<00:00, 1173.21 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1807291/1807941 [23:02<00:00, 1321.47 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1807466/1807941 [23:03<00:00, 1429.20 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1807634/1807941 [23:03<00:00, 1493.52 examples/s]
Tokenizing train dataset: 100%|█████████▉| 1807813/1807941 [23:03<00:00, 1572.88 examples/s]
Tokenizing train dataset: 100%|██████████| 1807941/1807941 [23:03<00:00, 1306.89 examples/s]
[transformers] The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.
Starting training...

  0%|          | 0/12000 [00:00<?, ?it/s]
 88%|████████▊ | 10601/12000 [00:07<00:00, 1422.48it/s]
                                                       

 88%|████████▊ | 10601/12000 [00:07<00:00, 1422.48it/s]
                                                       

 88%|████████▊ | 10602/12000 [00:09<00:00, 1422.48it/s]
                                                       

 88%|████████▊ | 10603/12000 [00:10<00:00, 1422.48it/s]
                                                       

 88%|████████▊ | 10604/12000 [00:14<00:00, 1422.48it/s]
                                                       

 88%|████████▊ | 10605/12000 [00:17<00:00, 1422.48it/s]
                                                       

 88%|████████▊ | 10606/12000 [00:19<00:00, 1422.48it/s]
                                                       

 88%|████████▊ | 10607/12000 [00:20<00:00, 1422.48it/s]
                                                       

 88%|████████▊ | 10608/12000 [00:22<00:00, 1422.48it/s]
 88%|████████▊ | 10608/12000 [00:23<00:00, 1422.48it/s]
 88%|████████▊ | 10609/12000 [00:24<00:04, 336.28it/s] 
                                                      

 88%|████████▊ | 10609/12000 [00:24<00:04, 336.28it/s]
 88%|████████▊ | 10610/12000 [00:26<00:04, 302.08it/s]
                                                      

 88%|████████▊ | 10610/12000 [00:26<00:04, 302.08it/s]
                                                      

 88%|████████▊ | 10611/12000 [00:28<00:04, 302.08it/s]
                                                      

 88%|████████▊ | 10612/12000 [00:30<00:04, 302.08it/s]
                                                      

 88%|████████▊ | 10613/12000 [00:32<00:04, 302.08it/s]
                                                      

 88%|████████▊ | 10614/12000 [00:34<00:04, 302.08it/s]
                                                      

 88%|████████▊ | 10615/12000 [00:35<00:04, 302.08it/s]
                                                      

 88%|████████▊ | 10616/12000 [00:37<00:04, 302.08it/s]
                                                      

 88%|████████▊ | 10617/12000 [00:39<00:04, 302.08it/s]
                                                      

 88%|████████▊ | 10618/12000 [00:42<00:04, 302.08it/s]
 88%|████████▊ | 10618/12000 [00:43<00:04, 302.08it/s]
 88%|████████▊ | 10619/12000 [00:43<00:11, 122.42it/s]
                                                      

 88%|████████▊ | 10619/12000 [00:43<00:11, 122.42it/s]
 88%|████████▊ | 10620/12000 [00:46<00:12, 108.61it/s]
                                                      

 88%|████████▊ | 10620/12000 [00:46<00:12, 108.61it/s]
                                                      

 89%|████████▊ | 10621/12000 [00:47<00:12, 108.61it/s]
                                                      

 89%|████████▊ | 10622/12000 [00:51<00:12, 108.61it/s]
                                                      

 89%|████████▊ | 10623/12000 [00:52<00:12, 108.61it/s]
                                                      

 89%|████████▊ | 10624/12000 [00:54<00:12, 108.61it/s]
                                                      

 89%|████████▊ | 10625/12000 [00:55<00:12, 108.61it/s]
                                                      

 89%|████████▊ | 10626/12000 [00:57<00:12, 108.61it/s]
                                                      

 89%|████████▊ | 10627/12000 [00:58<00:12, 108.61it/s]
                                                      

 89%|████████▊ | 10628/12000 [01:00<00:12, 108.61it/s]
                                                      

 89%|████████▊ | 10629/12000 [01:02<00:12, 108.61it/s]
 89%|████████▊ | 10629/12000 [01:03<00:12, 108.61it/s]
 89%|████████▊ | 10630/12000 [01:04<00:26, 52.06it/s] 
                                                     

 89%|████████▊ | 10630/12000 [01:04<00:26, 52.06it/s]
 89%|████████▊ | 10631/12000 [01:06<00:28, 48.73it/s]
                                                     

 89%|████████▊ | 10631/12000 [01:06<00:28, 48.73it/s]
                                                     

 89%|████████▊ | 10632/12000 [01:08<00:28, 48.73it/s]
                                                     

 89%|████████▊ | 10633/12000 [01:09<00:28, 48.73it/s]
                                                     

 89%|████████▊ | 10634/12000 [01:12<00:28, 48.73it/s]
                                                     

 89%|████████▊ | 10635/12000 [01:13<00:28, 48.73it/s]
                                                     

 89%|████████▊ | 10636/12000 [01:16<00:27, 48.73it/s]
                                                     

 89%|████████▊ | 10637/12000 [01:18<00:27, 48.73it/s]
                                                     

 89%|████████▊ | 10638/12000 [01:20<00:27, 48.73it/s]
                                                     

 89%|████████▊ | 10639/12000 [01:21<00:27, 48.73it/s]
 89%|████████▊ | 10639/12000 [01:23<00:27, 48.73it/s]
 89%|████████▊ | 10640/12000 [01:23<00:54, 25.09it/s]
                                                     

 89%|████████▊ | 10640/12000 [01:23<00:54, 25.09it/s]
 89%|████████▊ | 10641/12000 [01:25<00:57, 23.46it/s]
                                                     

 89%|████████▊ | 10641/12000 [01:25<00:57, 23.46it/s]
                                                     

 89%|████████▊ | 10642/12000 [01:26<00:57, 23.46it/s]
                                                     

 89%|████████▊ | 10643/12000 [01:28<00:57, 23.46it/s]
                                                     

 89%|████████▊ | 10644/12000 [01:32<00:57, 23.46it/s]
                                                     

 89%|████████▊ | 10645/12000 [01:34<00:57, 23.46it/s]
                                                     

 89%|████████▊ | 10646/12000 [01:36<00:57, 23.46it/s]
                                                     

 89%|████████▊ | 10647/12000 [01:38<00:57, 23.46it/s]
                                                     
{'loss': '0.6155', 'grad_norm': '0.3862', 'learning_rate': '7.351e-06', 'entropy': '0.6297', 'num_tokens': '4973', 'mean_token_accuracy': '0.8408', 'epoch': '0.09382'}
{'loss': '0.6507', 'grad_norm': '0.4258', 'learning_rate': '7.34e-06', 'entropy': '0.7826', 'num_tokens': '9188', 'mean_token_accuracy': '0.7969', 'epoch': '0.09383'}
{'loss': '0.8846', 'grad_norm': '0.502', 'learning_rate': '7.33e-06', 'entropy': '0.9689', 'num_tokens': '1.26e+04', 'mean_token_accuracy': '0.7653', 'epoch': '0.09383'}
{'loss': '0.8007', 'grad_norm': '0.3127', 'learning_rate': '7.32e-06', 'entropy': '0.9609', 'num_tokens': '2.041e+04', 'mean_token_accuracy': '0.7686', 'epoch': '0.09384'}
{'loss': '0.6584', 'grad_norm': '0.3336', 'learning_rate': '7.309e-06', 'entropy': '0.6109', 'num_tokens': '2.883e+04', 'mean_token_accuracy': '0.8436', 'epoch': '0.09385'}
{'loss': '0.5787', 'grad_norm': '0.3736', 'learning_rate': '7.299e-06', 'entropy': '0.5687', 'num_tokens': '3.277e+04', 'mean_token_accuracy': '0.8374', 'epoch': '0.09386'}
{'loss': '0.8035', 'grad_norm': '0.4269', 'learning_rate': '7.288e-06', 'entropy': '0.761', 'num_tokens': '3.801e+04', 'mean_token_accuracy': '0.8048', 'epoch': '0.09387'}
{'loss': '0.8215', 'grad_norm': '0.4756', 'learning_rate': '7.278e-06', 'entropy': '0.8265', 'num_tokens': '4.264e+04', 'mean_token_accuracy': '0.7873', 'epoch': '0.09388'}
{'loss': '0.9519', 'grad_norm': '0.4593', 'learning_rate': '7.268e-06', 'entropy': '0.9142', 'num_tokens': '4.748e+04', 'mean_token_accuracy': '0.7792', 'epoch': '0.09389'}
{'loss': '0.668', 'grad_norm': '0.4272', 'learning_rate': '7.258e-06', 'entropy': '0.7538', 'num_tokens': '5.253e+04', 'mean_token_accuracy': '0.8105', 'epoch': '0.0939'}
{'loss': '0.9248', 'grad_norm': '0.4155', 'learning_rate': '7.247e-06', 'entropy': '0.8642', 'num_tokens': '6.078e+04', 'mean_token_accuracy': '0.7994', 'epoch': '0.09391'}
{'loss': '1.005', 'grad_norm': '0.4493', 'learning_rate': '7.237e-06', 'entropy': '1.013', 'num_tokens': '6.665e+04', 'mean_token_accuracy': '0.7508', 'epoch': '0.09391'}
{'loss': '0.7823', 'grad_norm': '0.3901', 'learning_rate': '7.227e-06', 'entropy': '0.8351', 'num_tokens': '7.241e+04', 'mean_token_accuracy': '0.7974', 'epoch': '0.09392'}
{'loss': '0.6698', 'grad_norm': '0.4322', 'learning_rate': '7.216e-06', 'entropy': '0.6641', 'num_tokens': '7.572e+04', 'mean_token_accuracy': '0.8225', 'epoch': '0.09393'}
{'loss': '0.8287', 'grad_norm': '0.331', 'learning_rate': '7.206e-06', 'entropy': '0.8845', 'num_tokens': '8.034e+04', 'mean_token_accuracy': '0.7808', 'epoch': '0.09394'}
{'loss': '0.8024', 'grad_norm': '0.3765', 'learning_rate': '7.196e-06', 'entropy': '0.7638', 'num_tokens': '8.558e+04', 'mean_token_accuracy': '0.8031', 'epoch': '0.09395'}
{'loss': '0.8781', 'grad_norm': '0.3499', 'learning_rate': '7.186e-06', 'entropy': '0.8007', 'num_tokens': '9.326e+04', 'mean_token_accuracy': '0.7995', 'epoch': '0.09396'}
{'loss': '0.908', 'grad_norm': '0.4346', 'learning_rate': '7.175e-06', 'entropy': '0.929', 'num_tokens': '9.948e+04', 'mean_token_accuracy': '0.7569', 'epoch': '0.09397'}
{'loss': '0.9196', 'grad_norm': '0.4801', 'learning_rate': '7.165e-06', 'entropy': '0.9985', 'num_tokens': '1.043e+05', 'mean_token_accuracy': '0.7642', 'epoch': '0.09398'}
{'loss': '0.8193', 'grad_norm': '0.5014', 'learning_rate': '7.155e-06', 'entropy': '0.7897', 'num_tokens': '1.106e+05', 'mean_token_accuracy': '0.7963', 'epoch': '0.09399'}
{'loss': '0.944', 'grad_norm': '0.4934', 'learning_rate': '7.145e-06', 'entropy': '1.017', 'num_tokens': '1.141e+05', 'mean_token_accuracy': '0.756', 'epoch': '0.09399'}
{'loss': '0.644', 'grad_norm': '0.2772', 'learning_rate': '7.134e-06', 'entropy': '0.6526', 'num_tokens': '1.195e+05', 'mean_token_accuracy': '0.8251', 'epoch': '0.094'}
{'loss': '0.6339', 'grad_norm': '0.4245', 'learning_rate': '7.124e-06', 'entropy': '0.6323', 'num_tokens': '1.239e+05', 'mean_token_accuracy': '0.8285', 'epoch': '0.09401'}
{'loss': '0.7909', 'grad_norm': '0.4995', 'learning_rate': '7.114e-06', 'entropy': '0.8334', 'num_tokens': '1.274e+05', 'mean_token_accuracy': '0.7919', 'epoch': '0.09402'}
{'loss': '0.6234', 'grad_norm': '0.3839', 'learning_rate': '7.104e-06', 'entropy': '0.6209', 'num_tokens': '1.319e+05', 'mean_token_accuracy': '0.8276', 'epoch': '0.09403'}
{'loss': '0.9321', 'grad_norm': '0.4419', 'learning_rate': '7.094e-06', 'entropy': '0.945', 'num_tokens': '1.364e+05', 'mean_token_accuracy': '0.7697', 'epoch': '0.09404'}
{'loss': '0.5348', 'grad_norm': '0.3441', 'learning_rate': '7.083e-06', 'entropy': '0.5524', 'num_tokens': '1.417e+05', 'mean_token_accuracy': '0.8624', 'epoch': '0.09405'}
{'loss': '0.5737', 'grad_norm': '0.3745', 'learning_rate': '7.073e-06', 'entropy': '0.6428', 'num_tokens': '1.462e+05', 'mean_token_accuracy': '0.8405', 'epoch': '0.09406'}
{'loss': '1.214', 'grad_norm': '0.4693', 'learning_rate': '7.063e-06', 'entropy': '0.8941', 'num_tokens': '1.507e+05', 'mean_token_accuracy': '0.7849', 'epoch': '0.09406'}
{'loss': '1.095', 'grad_norm': '0.5206', 'learning_rate': '7.053e-06', 'entropy': '1.08', 'num_tokens': '1.558e+05', 'mean_token_accuracy': '0.7315', 'epoch': '0.09407'}
{'loss': '0.865', 'grad_norm': '0.4193', 'learning_rate': '7.043e-06', 'entropy': '0.8297', 'num_tokens': '1.608e+05', 'mean_token_accuracy': '0.7803', 'epoch': '0.09408'}
{'loss': '0.9315', 'grad_norm': '0.4599', 'learning_rate': '7.032e-06', 'entropy': '0.9283', 'num_tokens': '1.676e+05', 'mean_token_accuracy': '0.7719', 'epoch': '0.09409'}
{'loss': '0.6748', 'grad_norm': '0.3745', 'learning_rate': '7.022e-06', 'entropy': '0.6953', 'num_tokens': '1.711e+05', 'mean_token_accuracy': '0.8101', 'epoch': '0.0941'}
{'loss': '1.059', 'grad_norm': '0.3686', 'learning_rate': '7.012e-06', 'entropy': '0.9389', 'num_tokens': '1.775e+05', 'mean_token_accuracy': '0.7714', 'epoch': '0.09411'}
{'loss': '0.7333', 'grad_norm': '0.3983', 'learning_rate': '7.002e-06', 'entropy': '0.7294', 'num_tokens': '1.823e+05', 'mean_token_accuracy': '0.8129', 'epoch': '0.09412'}
{'loss': '0.5273', 'grad_norm': '0.3076', 'learning_rate': '6.992e-06', 'entropy': '0.5713', 'num_tokens': '1.891e+05', 'mean_token_accuracy': '0.8527', 'epoch': '0.09413'}
{'loss': '0.9224', 'grad_norm': '0.5537', 'learning_rate': '6.982e-06', 'entropy': '1.011', 'num_tokens': '1.933e+05', 'mean_token_accuracy': '0.7501', 'epoch': '0.09414'}
{'loss': '0.6757', 'grad_norm': '0.4469', 'learning_rate': '6.972e-06', 'entropy': '0.6996', 'num_tokens': '1.965e+05', 'mean_token_accuracy': '0.8275', 'epoch': '0.09414'}
{'loss': '0.6611', 'grad_norm': '0.4764', 'learning_rate': '6.962e-06', 'entropy': '0.6831', 'num_tokens': '2.01e+05', 'mean_token_accuracy': '0.8194', 'epoch': '0.09415'}
{'loss': '0.855', 'grad_norm': '0.5603', 'learning_rate': '6.952e-06', 'entropy': '0.8177', 'num_tokens': '2.061e+05', 'mean_token_accuracy': '0.7979', 'epoch': '0.09416'}
{'loss': '1.406', 'grad_norm': '0.4989', 'learning_rate': '6.941e-06', 'entropy': '1.244', 'num_tokens': '2.109e+05', 'mean_token_accuracy': '0.7039', 'epoch': '0.09417'}
{'loss': '0.5916', 'grad_norm': '0.2887', 'learning_rate': '6.931e-06', 'entropy': '0.5934', 'num_tokens': '2.159e+05', 'mean_token_accuracy': '0.8383', 'epoch': '0.09418'}
{'loss': '0.8365', 'grad_norm': '0.3961', 'learning_rate': '6.921e-06', 'entropy': '0.8201', 'num_tokens': '2.205e+05', 'mean_token_accuracy': '0.7951', 'epoch': '0.09419'}
{'loss': '0.5249', 'grad_norm': '0.3614', 'learning_rate': '6.911e-06', 'entropy': '0.6115', 'num_tokens': '2.268e+05', 'mean_token_accuracy': '0.8409', 'epoch': '0.0942'}
{'loss': '1.073', 'grad_norm': '0.4514', 'learning_rate': '6.901e-06', 'entropy': '1.051', 'num_tokens': '2.319e+05', 'mean_token_accuracy': '0.74', 'epoch': '0.09421'}
{'loss': '1.149', 'grad_norm': '0.5139', 'learning_rate': '6.891e-06', 'entropy': '1.127', 'num_tokens': '2.359e+05', 'mean_token_accuracy': '0.7266', 'epoch': '0.09422'}
{'loss': '0.7904', 'grad_norm': '0.3498', 'learning_rate': '6.881e-06', 'entropy': '0.8225', 'num_tokens': '2.419e+05', 'mean_token_accuracy': '0.802', 'epoch': '0.09422'}

 89%|████████▊ | 10648/12000 [01:41<00:57, 23.46it/s]
 89%|████████▊ | 10648/12000 [01:43<00:57, 23.46it/s]
 89%|████████▊ | 10649/12000 [01:43<01:51, 12.16it/s]
                                                     

 89%|████████▊ | 10649/12000 [01:43<01:51, 12.16it/s]
 89%|████████▉ | 10650/12000 [01:46<02:03, 10.91it/s]
                                                     

 89%|████████▉ | 10650/12000 [01:46<02:03, 10.91it/s]
                                                     

 89%|████████▉ | 10651/12000 [01:49<02:03, 10.91it/s]
                                                     

 89%|████████▉ | 10652/12000 [01:51<02:03, 10.91it/s]
                                                     

 89%|████████▉ | 10653/12000 [01:52<02:03, 10.91it/s]
                                                     

 89%|████████▉ | 10654/12000 [01:56<02:03, 10.91it/s]
                                                     

 89%|████████▉ | 10655/12000 [01:57<02:03, 10.91it/s]
                                                     

 89%|████████▉ | 10656/12000 [01:59<02:03, 10.91it/s]
                                                     

 89%|████████▉ | 10657/12000 [02:01<02:03, 10.91it/s]
                                                     

 89%|████████▉ | 10658/12000 [02:02<02:03, 10.91it/s]
 89%|████████▉ | 10658/12000 [02:03<02:03, 10.91it/s]
 89%|████████▉ | 10659/12000 [02:04<03:43,  5.99it/s]
                                                     

 89%|████████▉ | 10659/12000 [02:04<03:43,  5.99it/s]
 89%|████████▉ | 10660/12000 [02:06<04:00,  5.58it/s]
                                                     

 89%|████████▉ | 10660/12000 [02:06<04:00,  5.58it/s]
                                                     

 89%|████████▉ | 10661/12000 [02:08<04:00,  5.58it/s]
                                                     

 89%|████████▉ | 10662/12000 [02:09<03:59,  5.58it/s]
                                                     

 89%|████████▉ | 10663/12000 [02:11<03:59,  5.58it/s]
                                                     

 89%|████████▉ | 10664/12000 [02:13<03:59,  5.58it/s]
                                                     

 89%|████████▉ | 10665/12000 [02:15<03:59,  5.58it/s]
 89%|████████▉ | 10666/12000 [02:18<06:06,  3.64it/s]
                                                     

 89%|████████▉ | 10666/12000 [02:18<06:06,  3.64it/s]
 89%|████████▉ | 10667/12000 [02:20<06:26,  3.45it/s]
                                                     

 89%|████████▉ | 10667/12000 [02:20<06:26,  3.45it/s]
                                                     

 89%|████████▉ | 10668/12000 [02:22<06:25,  3.45it/s]
                                                     

 89%|████████▉ | 10669/12000 [02:25<06:25,  3.45it/s]
                                                     

 89%|████████▉ | 10670/12000 [02:27<06:25,  3.45it/s]
 89%|████████▉ | 10671/12000 [02:30<09:22,  2.36it/s]
                                                     

 89%|████████▉ | 10671/12000 [02:30<09:22,  2.36it/s]
                                                     

 89%|████████▉ | 10672/12000 [02:31<09:21,  2.36it/s]
                                                     

 89%|████████▉ | 10673/12000 [02:33<09:21,  2.36it/s]
 89%|████████▉ | 10674/12000 [02:35<11:05,  1.99it/s]
                                                     

 89%|████████▉ | 10674/12000 [02:35<11:05,  1.99it/s]
                                                     

 89%|████████▉ | 10675/12000 [02:37<11:05,  1.99it/s]
 89%|████████▉ | 10676/12000 [02:38<12:43,  1.73it/s]
                                                     

 89%|████████▉ | 10676/12000 [02:38<12:43,  1.73it/s]
                                                     

 89%|████████▉ | 10677/12000 [02:39<12:42,  1.73it/s]
 89%|████████▉ | 10678/12000 [02:42<14:50,  1.49it/s]
                                                     

 89%|████████▉ | 10678/12000 [02:42<14:50,  1.49it/s]
 89%|████████▉ | 10679/12000 [02:44<16:01,  1.37it/s]
                                                     

 89%|████████▉ | 10679/12000 [02:44<16:01,  1.37it/s]
 89%|████████▉ | 10680/12000 [02:45<17:05,  1.29it/s]
                                                     

 89%|████████▉ | 10680/12000 [02:45<17:05,  1.29it/s]
 89%|████████▉ | 10681/12000 [02:46<18:15,  1.20it/s]
                                                     

 89%|████████▉ | 10681/12000 [02:46<18:15,  1.20it/s]
 89%|████████▉ | 10682/12000 [02:48<19:58,  1.10it/s]
                                                     

 89%|████████▉ | 10682/12000 [02:48<19:58,  1.10it/s]
 89%|████████▉ | 10683/12000 [02:56<40:54,  1.86s/it]
                                                     

 89%|████████▉ | 10683/12000 [02:56<40:54,  1.86s/it]
 89%|████████▉ | 10684/12000 [02:57<39:05,  1.78s/it]
                                                     

 89%|████████▉ | 10684/12000 [02:57<39:05,  1.78s/it]
 89%|████████▉ | 10685/12000 [02:59<38:01,  1.74s/it]
                                                     

 89%|████████▉ | 10685/12000 [02:59<38:01,  1.74s/it]
 89%|████████▉ | 10686/12000 [03:00<36:05,  1.65s/it]
                                                     

 89%|████████▉ | 10686/12000 [03:00<36:05,  1.65s/it]
 89%|████████▉ | 10687/12000 [03:02<35:46,  1.64s/it]
                                                     

 89%|████████▉ | 10687/12000 [03:02<35:46,  1.64s/it]
 89%|████████▉ | 10688/12000 [03:05<47:44,  2.18s/it]
                                                     

 89%|████████▉ | 10688/12000 [03:05<47:44,  2.18s/it]
 89%|████████▉ | 10689/12000 [03:07<43:12,  1.98s/it]
                                                     

 89%|████████▉ | 10689/12000 [03:07<43:12,  1.98s/it]
 89%|████████▉ | 10690/12000 [03:08<41:39,  1.91s/it]
                                                     

 89%|████████▉ | 10690/12000 [03:08<41:39,  1.91s/it]
 89%|████████▉ | 10691/12000 [03:10<40:46,  1.87s/it]
                                                     

 89%|████████▉ | 10691/12000 [03:10<40:46,  1.87s/it]
 89%|████████▉ | 10692/12000 [03:12<43:12,  1.98s/it]
                                                     

 89%|████████▉ | 10692/12000 [03:12<43:12,  1.98s/it]
 89%|████████▉ | 10693/12000 [03:14<40:09,  1.84s/it]
                                                     

 89%|████████▉ | 10693/12000 [03:14<40:09,  1.84s/it]
 89%|████████▉ | 10694/12000 [03:16<38:12,  1.76s/it]
                                                     

 89%|████████▉ | 10694/12000 [03:16<38:12,  1.76s/it]
 89%|████████▉ | 10695/12000 [03:17<36:56,  1.70s/it]
                                                     
{'loss': '1.151', 'grad_norm': '0.4152', 'learning_rate': '6.871e-06', 'entropy': '1.2', 'num_tokens': '2.487e+05', 'mean_token_accuracy': '0.7225', 'epoch': '0.09423'}
{'loss': '0.5768', 'grad_norm': '0.4173', 'learning_rate': '6.861e-06', 'entropy': '0.5858', 'num_tokens': '2.543e+05', 'mean_token_accuracy': '0.8477', 'epoch': '0.09424'}
{'loss': '1.197', 'grad_norm': '0.3649', 'learning_rate': '6.851e-06', 'entropy': '1.1', 'num_tokens': '2.631e+05', 'mean_token_accuracy': '0.7376', 'epoch': '0.09425'}
{'loss': '1.085', 'grad_norm': '0.4914', 'learning_rate': '6.841e-06', 'entropy': '1.066', 'num_tokens': '2.67e+05', 'mean_token_accuracy': '0.7125', 'epoch': '0.09426'}
{'loss': '1.162', 'grad_norm': '0.5338', 'learning_rate': '6.831e-06', 'entropy': '1.206', 'num_tokens': '2.72e+05', 'mean_token_accuracy': '0.7062', 'epoch': '0.09427'}
{'loss': '0.7909', 'grad_norm': '0.3817', 'learning_rate': '6.821e-06', 'entropy': '0.8389', 'num_tokens': '2.765e+05', 'mean_token_accuracy': '0.787', 'epoch': '0.09428'}
{'loss': '1.186', 'grad_norm': '0.3801', 'learning_rate': '6.811e-06', 'entropy': '1.211', 'num_tokens': '2.84e+05', 'mean_token_accuracy': '0.7132', 'epoch': '0.09429'}
{'loss': '0.5798', 'grad_norm': '0.3243', 'learning_rate': '6.801e-06', 'entropy': '0.611', 'num_tokens': '2.895e+05', 'mean_token_accuracy': '0.8389', 'epoch': '0.09429'}
{'loss': '0.6896', 'grad_norm': '0.3796', 'learning_rate': '6.791e-06', 'entropy': '0.6936', 'num_tokens': '2.957e+05', 'mean_token_accuracy': '0.8215', 'epoch': '0.0943'}
{'loss': '0.7373', 'grad_norm': '0.563', 'learning_rate': '6.781e-06', 'entropy': '0.7886', 'num_tokens': '2.999e+05', 'mean_token_accuracy': '0.8009', 'epoch': '0.09431'}
{'loss': '0.904', 'grad_norm': '0.5141', 'learning_rate': '6.771e-06', 'entropy': '0.9546', 'num_tokens': '3.047e+05', 'mean_token_accuracy': '0.7625', 'epoch': '0.09432'}
{'loss': '0.9673', 'grad_norm': '0.4877', 'learning_rate': '6.761e-06', 'entropy': '0.914', 'num_tokens': '3.086e+05', 'mean_token_accuracy': '0.7853', 'epoch': '0.09433'}
{'loss': '0.862', 'grad_norm': '0.4329', 'learning_rate': '6.751e-06', 'entropy': '0.7506', 'num_tokens': '3.141e+05', 'mean_token_accuracy': '0.811', 'epoch': '0.09434'}
{'loss': '0.5463', 'grad_norm': '0.4153', 'learning_rate': '6.741e-06', 'entropy': '0.5565', 'num_tokens': '3.191e+05', 'mean_token_accuracy': '0.8451', 'epoch': '0.09435'}
{'loss': '0.7599', 'grad_norm': '0.3989', 'learning_rate': '6.731e-06', 'entropy': '0.7724', 'num_tokens': '3.235e+05', 'mean_token_accuracy': '0.8034', 'epoch': '0.09436'}
{'loss': '0.9014', 'grad_norm': '0.4439', 'learning_rate': '6.721e-06', 'entropy': '0.7958', 'num_tokens': '3.293e+05', 'mean_token_accuracy': '0.811', 'epoch': '0.09437'}
{'loss': '0.8566', 'grad_norm': '0.4881', 'learning_rate': '6.711e-06', 'entropy': '0.8454', 'num_tokens': '3.347e+05', 'mean_token_accuracy': '0.7844', 'epoch': '0.09437'}
{'loss': '0.6734', 'grad_norm': '0.3884', 'learning_rate': '6.701e-06', 'entropy': '0.6658', 'num_tokens': '3.397e+05', 'mean_token_accuracy': '0.8349', 'epoch': '0.09438'}
{'loss': '1.038', 'grad_norm': '0.5162', 'learning_rate': '6.691e-06', 'entropy': '1.168', 'num_tokens': '3.431e+05', 'mean_token_accuracy': '0.7218', 'epoch': '0.09439'}
{'loss': '0.6525', 'grad_norm': '0.3248', 'learning_rate': '6.682e-06', 'entropy': '0.6809', 'num_tokens': '3.474e+05', 'mean_token_accuracy': '0.8162', 'epoch': '0.0944'}
{'loss': '0.7547', 'grad_norm': '0.3508', 'learning_rate': '6.672e-06', 'entropy': '0.7786', 'num_tokens': '3.539e+05', 'mean_token_accuracy': '0.8045', 'epoch': '0.09441'}
{'loss': '0.7249', 'grad_norm': '0.3558', 'learning_rate': '6.662e-06', 'entropy': '0.8268', 'num_tokens': '3.619e+05', 'mean_token_accuracy': '0.7876', 'epoch': '0.09442'}
{'loss': '0.8476', 'grad_norm': '0.4625', 'learning_rate': '6.652e-06', 'entropy': '0.82', 'num_tokens': '3.674e+05', 'mean_token_accuracy': '0.7908', 'epoch': '0.09443'}
{'loss': '0.7739', 'grad_norm': '0.3463', 'learning_rate': '6.642e-06', 'entropy': '0.7226', 'num_tokens': '3.732e+05', 'mean_token_accuracy': '0.8023', 'epoch': '0.09444'}
{'loss': '1.042', 'grad_norm': '0.4161', 'learning_rate': '6.632e-06', 'entropy': '0.9638', 'num_tokens': '3.791e+05', 'mean_token_accuracy': '0.7673', 'epoch': '0.09445'}
{'loss': '0.7421', 'grad_norm': '0.4597', 'learning_rate': '6.622e-06', 'entropy': '0.7505', 'num_tokens': '3.843e+05', 'mean_token_accuracy': '0.8', 'epoch': '0.09445'}
{'loss': '0.7379', 'grad_norm': '0.39', 'learning_rate': '6.612e-06', 'entropy': '0.7523', 'num_tokens': '3.894e+05', 'mean_token_accuracy': '0.801', 'epoch': '0.09446'}
{'loss': '0.7094', 'grad_norm': '0.3904', 'learning_rate': '6.603e-06', 'entropy': '0.7151', 'num_tokens': '3.952e+05', 'mean_token_accuracy': '0.8197', 'epoch': '0.09447'}
{'loss': '0.5799', 'grad_norm': '0.343', 'learning_rate': '6.593e-06', 'entropy': '0.6379', 'num_tokens': '3.996e+05', 'mean_token_accuracy': '0.8354', 'epoch': '0.09448'}
{'loss': '0.9307', 'grad_norm': '0.474', 'learning_rate': '6.583e-06', 'entropy': '1.009', 'num_tokens': '4.026e+05', 'mean_token_accuracy': '0.7547', 'epoch': '0.09449'}
{'loss': '0.7615', 'grad_norm': '0.3788', 'learning_rate': '6.573e-06', 'entropy': '0.8486', 'num_tokens': '4.087e+05', 'mean_token_accuracy': '0.794', 'epoch': '0.0945'}
{'loss': '0.732', 'grad_norm': '0.4095', 'learning_rate': '6.563e-06', 'entropy': '0.7595', 'num_tokens': '4.146e+05', 'mean_token_accuracy': '0.8151', 'epoch': '0.09451'}
{'loss': '0.7709', 'grad_norm': '0.3746', 'learning_rate': '6.553e-06', 'entropy': '0.8373', 'num_tokens': '4.187e+05', 'mean_token_accuracy': '0.7842', 'epoch': '0.09452'}
{'loss': '0.5888', 'grad_norm': '0.3905', 'learning_rate': '6.544e-06', 'entropy': '0.6492', 'num_tokens': '4.22e+05', 'mean_token_accuracy': '0.8292', 'epoch': '0.09453'}
{'loss': '0.6929', 'grad_norm': '0.4593', 'learning_rate': '6.534e-06', 'entropy': '0.736', 'num_tokens': '4.251e+05', 'mean_token_accuracy': '0.8216', 'epoch': '0.09453'}
{'loss': '0.8592', 'grad_norm': '0.3058', 'learning_rate': '6.524e-06', 'entropy': '0.9341', 'num_tokens': '4.365e+05', 'mean_token_accuracy': '0.7811', 'epoch': '0.09454'}
{'loss': '0.7129', 'grad_norm': '0.4113', 'learning_rate': '6.514e-06', 'entropy': '0.7214', 'num_tokens': '4.402e+05', 'mean_token_accuracy': '0.8229', 'epoch': '0.09455'}
{'loss': '1.142', 'grad_norm': '0.5308', 'learning_rate': '6.504e-06', 'entropy': '1.274', 'num_tokens': '4.448e+05', 'mean_token_accuracy': '0.7065', 'epoch': '0.09456'}
{'loss': '0.7471', 'grad_norm': '0.404', 'learning_rate': '6.495e-06', 'entropy': '0.7399', 'num_tokens': '4.483e+05', 'mean_token_accuracy': '0.8041', 'epoch': '0.09457'}
{'loss': '1.102', 'grad_norm': '0.4782', 'learning_rate': '6.485e-06', 'entropy': '1.174', 'num_tokens': '4.534e+05', 'mean_token_accuracy': '0.726', 'epoch': '0.09458'}
{'loss': '1.052', 'grad_norm': '0.5666', 'learning_rate': '6.475e-06', 'entropy': '1.022', 'num_tokens': '4.572e+05', 'mean_token_accuracy': '0.7628', 'epoch': '0.09459'}
{'loss': '1.125', 'grad_norm': '0.5957', 'learning_rate': '6.465e-06', 'entropy': '1.073', 'num_tokens': '4.599e+05', 'mean_token_accuracy': '0.7446', 'epoch': '0.0946'}
{'loss': '0.6001', 'grad_norm': '0.3391', 'learning_rate': '6.456e-06', 'entropy': '0.5881', 'num_tokens': '4.66e+05', 'mean_token_accuracy': '0.85', 'epoch': '0.0946'}
{'loss': '0.6334', 'grad_norm': '0.3544', 'learning_rate': '6.446e-06', 'entropy': '0.6348', 'num_tokens': '4.707e+05', 'mean_token_accuracy': '0.8347', 'epoch': '0.09461'}
{'loss': '0.6622', 'grad_norm': '0.3632', 'learning_rate': '6.436e-06', 'entropy': '0.603', 'num_tokens': '4.765e+05', 'mean_token_accuracy': '0.8562', 'epoch': '0.09462'}
{'loss': '0.7155', 'grad_norm': '0.4269', 'learning_rate': '6.426e-06', 'entropy': '0.7218', 'num_tokens': '4.815e+05', 'mean_token_accuracy': '0.8231', 'epoch': '0.09463'}
{'loss': '0.7391', 'grad_norm': '0.358', 'learning_rate': '6.417e-06', 'entropy': '0.7071', 'num_tokens': '4.861e+05', 'mean_token_accuracy': '0.8135', 'epoch': '0.09464'}

 89%|████████▉ | 10695/12000 [03:17<36:56,  1.70s/it]
 89%|████████▉ | 10696/12000 [03:19<36:21,  1.67s/it]
                                                     

 89%|████████▉ | 10696/12000 [03:19<36:21,  1.67s/it]
 89%|████████▉ | 10697/12000 [03:20<36:05,  1.66s/it]
                                                     

 89%|████████▉ | 10697/12000 [03:20<36:05,  1.66s/it]
 89%|████████▉ | 10698/12000 [03:22<37:31,  1.73s/it]
                                                     

 89%|████████▉ | 10698/12000 [03:22<37:31,  1.73s/it]
 89%|████████▉ | 10699/12000 [03:24<36:47,  1.70s/it]
                                                     

 89%|████████▉ | 10699/12000 [03:24<36:47,  1.70s/it]
 89%|████████▉ | 10700/12000 [03:25<36:14,  1.67s/it]
                                                     

 89%|████████▉ | 10700/12000 [03:25<36:14,  1.67s/it]
 89%|████████▉ | 10701/12000 [03:29<45:28,  2.10s/it]
                                                     

 89%|████████▉ | 10701/12000 [03:29<45:28,  2.10s/it]
 89%|████████▉ | 10702/12000 [03:31<49:59,  2.31s/it]
                                                     

 89%|████████▉ | 10702/12000 [03:31<49:59,  2.31s/it]
 89%|████████▉ | 10703/12000 [03:33<43:54,  2.03s/it]
                                                     

 89%|████████▉ | 10703/12000 [03:33<43:54,  2.03s/it]
 89%|████████▉ | 10704/12000 [03:34<40:50,  1.89s/it]
                                                     

 89%|████████▉ | 10704/12000 [03:34<40:50,  1.89s/it]
 89%|████████▉ | 10705/12000 [03:36<42:27,  1.97s/it]
                                                     

 89%|████████▉ | 10705/12000 [03:36<42:27,  1.97s/it]
 89%|████████▉ | 10706/12000 [03:38<39:04,  1.81s/it]
                                                     

 89%|████████▉ | 10706/12000 [03:38<39:04,  1.81s/it]
 89%|████████▉ | 10707/12000 [03:40<39:46,  1.85s/it]
                                                     

 89%|████████▉ | 10707/12000 [03:40<39:46,  1.85s/it]
 89%|████████▉ | 10708/12000 [03:42<38:46,  1.80s/it]
                                                     

 89%|████████▉ | 10708/12000 [03:42<38:46,  1.80s/it]
 89%|████████▉ | 10709/12000 [03:43<36:27,  1.69s/it]
                                                     

 89%|████████▉ | 10709/12000 [03:43<36:27,  1.69s/it]
 89%|████████▉ | 10710/12000 [03:47<53:07,  2.47s/it]
                                                     

 89%|████████▉ | 10710/12000 [03:47<53:07,  2.47s/it]
 89%|████████▉ | 10711/12000 [03:49<46:56,  2.19s/it]
                                                     

 89%|████████▉ | 10711/12000 [03:49<46:56,  2.19s/it]
 89%|████████▉ | 10712/12000 [03:51<48:06,  2.24s/it]
                                                     

 89%|████████▉ | 10712/12000 [03:51<48:06,  2.24s/it]
 89%|████████▉ | 10713/12000 [03:53<42:28,  1.98s/it]
                                                     

 89%|████████▉ | 10713/12000 [03:53<42:28,  1.98s/it]
 89%|████████▉ | 10714/12000 [03:54<41:13,  1.92s/it]
                                                     

 89%|████████▉ | 10714/12000 [03:54<41:13,  1.92s/it]
 89%|████████▉ | 10715/12000 [03:56<38:44,  1.81s/it]
                                                     

 89%|████████▉ | 10715/12000 [03:56<38:44,  1.81s/it]
 89%|████████▉ | 10716/12000 [03:57<36:03,  1.68s/it]
                                                     

 89%|████████▉ | 10716/12000 [03:57<36:03,  1.68s/it]
 89%|████████▉ | 10717/12000 [03:59<39:11,  1.83s/it]
                                                     

 89%|████████▉ | 10717/12000 [03:59<39:11,  1.83s/it]
 89%|████████▉ | 10718/12000 [04:01<37:07,  1.74s/it]
                                                     

 89%|████████▉ | 10718/12000 [04:01<37:07,  1.74s/it]
 89%|████████▉ | 10719/12000 [04:02<35:10,  1.65s/it]
                                                     

 89%|████████▉ | 10719/12000 [04:02<35:10,  1.65s/it]
 89%|████████▉ | 10720/12000 [04:04<37:33,  1.76s/it]
                                                     

 89%|████████▉ | 10720/12000 [04:04<37:33,  1.76s/it]
 89%|████████▉ | 10721/12000 [04:07<40:37,  1.91s/it]
                                                     

 89%|████████▉ | 10721/12000 [04:07<40:37,  1.91s/it]
 89%|████████▉ | 10722/12000 [04:08<37:41,  1.77s/it]
                                                     

 89%|████████▉ | 10722/12000 [04:08<37:41,  1.77s/it]
 89%|████████▉ | 10723/12000 [04:10<35:47,  1.68s/it]
                                                     

 89%|████████▉ | 10723/12000 [04:10<35:47,  1.68s/it]
 89%|████████▉ | 10724/12000 [04:11<33:48,  1.59s/it]
                                                     

 89%|████████▉ | 10724/12000 [04:11<33:48,  1.59s/it]
 89%|████████▉ | 10725/12000 [04:13<35:12,  1.66s/it]
                                                     

 89%|████████▉ | 10725/12000 [04:13<35:12,  1.66s/it]
 89%|████████▉ | 10726/12000 [04:14<35:04,  1.65s/it]
                                                     

 89%|████████▉ | 10726/12000 [04:14<35:04,  1.65s/it]
 89%|████████▉ | 10727/12000 [04:17<38:57,  1.84s/it]
                                                     

 89%|████████▉ | 10727/12000 [04:17<38:57,  1.84s/it]
 89%|████████▉ | 10728/12000 [04:20<46:07,  2.18s/it]
                                                     

 89%|████████▉ | 10728/12000 [04:20<46:07,  2.18s/it]
 89%|████████▉ | 10729/12000 [04:21<42:30,  2.01s/it]
                                                     

 89%|████████▉ | 10729/12000 [04:21<42:30,  2.01s/it]
 89%|████████▉ | 10730/12000 [04:23<40:17,  1.90s/it]
                                                     

 89%|████████▉ | 10730/12000 [04:23<40:17,  1.90s/it]
 89%|████████▉ | 10731/12000 [04:26<47:02,  2.22s/it]
                                                     

 89%|████████▉ | 10731/12000 [04:26<47:02,  2.22s/it]
 89%|████████▉ | 10732/12000 [04:30<59:57,  2.84s/it]
                                                     

 89%|████████▉ | 10732/12000 [04:30<59:57,  2.84s/it]
 89%|████████▉ | 10733/12000 [04:33<58:46,  2.78s/it]
                                                     

 89%|████████▉ | 10733/12000 [04:33<58:46,  2.78s/it]
 89%|████████▉ | 10734/12000 [04:34<51:31,  2.44s/it]
                                                     

 89%|████████▉ | 10734/12000 [04:34<51:31,  2.44s/it]
 89%|████████▉ | 10735/12000 [04:37<53:24,  2.53s/it]
                                                     

 89%|████████▉ | 10735/12000 [04:37<53:24,  2.53s/it]
 89%|████████▉ | 10736/12000 [04:39<48:11,  2.29s/it]
                                                     

 89%|████████▉ | 10736/12000 [04:39<48:11,  2.29s/it]
 89%|████████▉ | 10737/12000 [04:41<43:51,  2.08s/it]
                                                     

 89%|████████▉ | 10737/12000 [04:41<43:51,  2.08s/it]
 89%|████████▉ | 10738/12000 [04:42<41:12,  1.96s/it]
                                                     

 89%|████████▉ | 10738/12000 [04:42<41:12,  1.96s/it]
 89%|████████▉ | 10739/12000 [04:45<47:00,  2.24s/it]
                                                     

 89%|████████▉ | 10739/12000 [04:45<47:00,  2.24s/it]
 90%|████████▉ | 10740/12000 [04:48<51:54,  2.47s/it]
                                                     

 90%|████████▉ | 10740/12000 [04:48<51:54,  2.47s/it]
 90%|████████▉ | 10741/12000 [04:51<53:31,  2.55s/it]
                                                     

 90%|████████▉ | 10741/12000 [04:51<53:31,  2.55s/it]
 90%|████████▉ | 10742/12000 [04:52<47:10,  2.25s/it]
                                                     
{'loss': '0.7454', 'grad_norm': '0.4515', 'learning_rate': '6.407e-06', 'entropy': '0.7684', 'num_tokens': '4.9e+05', 'mean_token_accuracy': '0.8012', 'epoch': '0.09465'}
{'loss': '0.6525', 'grad_norm': '0.4506', 'learning_rate': '6.397e-06', 'entropy': '0.6839', 'num_tokens': '4.943e+05', 'mean_token_accuracy': '0.8167', 'epoch': '0.09466'}
{'loss': '0.7748', 'grad_norm': '0.3847', 'learning_rate': '6.388e-06', 'entropy': '0.7874', 'num_tokens': '4.988e+05', 'mean_token_accuracy': '0.7989', 'epoch': '0.09467'}
{'loss': '0.9922', 'grad_norm': '0.3856', 'learning_rate': '6.378e-06', 'entropy': '1.03', 'num_tokens': '5.046e+05', 'mean_token_accuracy': '0.7542', 'epoch': '0.09468'}
{'loss': '0.9883', 'grad_norm': '0.429', 'learning_rate': '6.368e-06', 'entropy': '1.039', 'num_tokens': '5.094e+05', 'mean_token_accuracy': '0.731', 'epoch': '0.09468'}
{'loss': '0.7423', 'grad_norm': '0.4842', 'learning_rate': '6.359e-06', 'entropy': '0.7822', 'num_tokens': '5.142e+05', 'mean_token_accuracy': '0.8', 'epoch': '0.09469'}
{'loss': '1.011', 'grad_norm': '0.4128', 'learning_rate': '6.349e-06', 'entropy': '1.034', 'num_tokens': '5.192e+05', 'mean_token_accuracy': '0.759', 'epoch': '0.0947'}
{'loss': '0.6857', 'grad_norm': '0.3417', 'learning_rate': '6.339e-06', 'entropy': '0.7954', 'num_tokens': '5.265e+05', 'mean_token_accuracy': '0.7915', 'epoch': '0.09471'}
{'loss': '0.7208', 'grad_norm': '0.4475', 'learning_rate': '6.33e-06', 'entropy': '0.7051', 'num_tokens': '5.301e+05', 'mean_token_accuracy': '0.816', 'epoch': '0.09472'}
{'loss': '1.348', 'grad_norm': '0.4494', 'learning_rate': '6.32e-06', 'entropy': '1.316', 'num_tokens': '5.343e+05', 'mean_token_accuracy': '0.6875', 'epoch': '0.09473'}
{'loss': '1.073', 'grad_norm': '0.5149', 'learning_rate': '6.31e-06', 'entropy': '0.9934', 'num_tokens': '5.397e+05', 'mean_token_accuracy': '0.7539', 'epoch': '0.09474'}
{'loss': '0.7753', 'grad_norm': '0.3986', 'learning_rate': '6.301e-06', 'entropy': '0.7984', 'num_tokens': '5.435e+05', 'mean_token_accuracy': '0.7975', 'epoch': '0.09475'}
{'loss': '0.744', 'grad_norm': '0.4026', 'learning_rate': '6.291e-06', 'entropy': '0.7167', 'num_tokens': '5.493e+05', 'mean_token_accuracy': '0.8233', 'epoch': '0.09476'}
{'loss': '0.9259', 'grad_norm': '0.4165', 'learning_rate': '6.281e-06', 'entropy': '0.9797', 'num_tokens': '5.547e+05', 'mean_token_accuracy': '0.7659', 'epoch': '0.09476'}
{'loss': '0.5508', 'grad_norm': '0.3391', 'learning_rate': '6.272e-06', 'entropy': '0.5811', 'num_tokens': '5.589e+05', 'mean_token_accuracy': '0.8434', 'epoch': '0.09477'}
{'loss': '1.102', 'grad_norm': '0.4579', 'learning_rate': '6.262e-06', 'entropy': '0.858', 'num_tokens': '5.665e+05', 'mean_token_accuracy': '0.7883', 'epoch': '0.09478'}
{'loss': '0.6467', 'grad_norm': '0.3527', 'learning_rate': '6.253e-06', 'entropy': '0.6677', 'num_tokens': '5.705e+05', 'mean_token_accuracy': '0.8303', 'epoch': '0.09479'}
{'loss': '1.143', 'grad_norm': '0.4152', 'learning_rate': '6.243e-06', 'entropy': '1.046', 'num_tokens': '5.769e+05', 'mean_token_accuracy': '0.7463', 'epoch': '0.0948'}
{'loss': '0.5798', 'grad_norm': '0.3474', 'learning_rate': '6.233e-06', 'entropy': '0.6315', 'num_tokens': '5.805e+05', 'mean_token_accuracy': '0.8363', 'epoch': '0.09481'}
{'loss': '1.102', 'grad_norm': '0.4527', 'learning_rate': '6.224e-06', 'entropy': '1.099', 'num_tokens': '5.849e+05', 'mean_token_accuracy': '0.7236', 'epoch': '0.09482'}
{'loss': '0.4385', 'grad_norm': '0.3259', 'learning_rate': '6.214e-06', 'entropy': '0.4084', 'num_tokens': '5.901e+05', 'mean_token_accuracy': '0.8879', 'epoch': '0.09483'}
{'loss': '0.6942', 'grad_norm': '0.3947', 'learning_rate': '6.205e-06', 'entropy': '0.6646', 'num_tokens': '5.94e+05', 'mean_token_accuracy': '0.8297', 'epoch': '0.09483'}
{'loss': '0.9822', 'grad_norm': '0.5823', 'learning_rate': '6.195e-06', 'entropy': '0.8501', 'num_tokens': '5.984e+05', 'mean_token_accuracy': '0.7875', 'epoch': '0.09484'}
{'loss': '0.5808', 'grad_norm': '0.3943', 'learning_rate': '6.186e-06', 'entropy': '0.594', 'num_tokens': '6.024e+05', 'mean_token_accuracy': '0.8372', 'epoch': '0.09485'}
{'loss': '0.7431', 'grad_norm': '0.425', 'learning_rate': '6.176e-06', 'entropy': '0.8186', 'num_tokens': '6.059e+05', 'mean_token_accuracy': '0.7947', 'epoch': '0.09486'}
{'loss': '0.7178', 'grad_norm': '0.3458', 'learning_rate': '6.167e-06', 'entropy': '0.7244', 'num_tokens': '6.115e+05', 'mean_token_accuracy': '0.82', 'epoch': '0.09487'}
{'loss': '0.829', 'grad_norm': '0.4247', 'learning_rate': '6.157e-06', 'entropy': '0.7377', 'num_tokens': '6.181e+05', 'mean_token_accuracy': '0.8132', 'epoch': '0.09488'}
{'loss': '0.9177', 'grad_norm': '0.4625', 'learning_rate': '6.148e-06', 'entropy': '0.868', 'num_tokens': '6.218e+05', 'mean_token_accuracy': '0.7762', 'epoch': '0.09489'}
{'loss': '0.5843', 'grad_norm': '0.3318', 'learning_rate': '6.138e-06', 'entropy': '0.6836', 'num_tokens': '6.259e+05', 'mean_token_accuracy': '0.8311', 'epoch': '0.0949'}
{'loss': '1.04', 'grad_norm': '0.58', 'learning_rate': '6.129e-06', 'entropy': '1.05', 'num_tokens': '6.287e+05', 'mean_token_accuracy': '0.7445', 'epoch': '0.09491'}
{'loss': '0.971', 'grad_norm': '0.4207', 'learning_rate': '6.119e-06', 'entropy': '0.9214', 'num_tokens': '6.35e+05', 'mean_token_accuracy': '0.7553', 'epoch': '0.09491'}
{'loss': '0.8661', 'grad_norm': '0.4449', 'learning_rate': '6.11e-06', 'entropy': '0.9135', 'num_tokens': '6.398e+05', 'mean_token_accuracy': '0.7809', 'epoch': '0.09492'}
{'loss': '1.06', 'grad_norm': '0.52', 'learning_rate': '6.1e-06', 'entropy': '1.051', 'num_tokens': '6.453e+05', 'mean_token_accuracy': '0.7221', 'epoch': '0.09493'}
{'loss': '0.7301', 'grad_norm': '0.3725', 'learning_rate': '6.091e-06', 'entropy': '0.7909', 'num_tokens': '6.53e+05', 'mean_token_accuracy': '0.8055', 'epoch': '0.09494'}
{'loss': '0.7938', 'grad_norm': '0.3678', 'learning_rate': '6.081e-06', 'entropy': '0.8236', 'num_tokens': '6.576e+05', 'mean_token_accuracy': '0.7921', 'epoch': '0.09495'}
{'loss': '0.8751', 'grad_norm': '0.4437', 'learning_rate': '6.072e-06', 'entropy': '0.8454', 'num_tokens': '6.625e+05', 'mean_token_accuracy': '0.7761', 'epoch': '0.09496'}
{'loss': '1.128', 'grad_norm': '0.4102', 'learning_rate': '6.062e-06', 'entropy': '0.9993', 'num_tokens': '6.698e+05', 'mean_token_accuracy': '0.7573', 'epoch': '0.09497'}
{'loss': '0.8962', 'grad_norm': '0.4036', 'learning_rate': '6.053e-06', 'entropy': '0.9416', 'num_tokens': '6.756e+05', 'mean_token_accuracy': '0.77', 'epoch': '0.09498'}
{'loss': '0.662', 'grad_norm': '0.3663', 'learning_rate': '6.043e-06', 'entropy': '0.647', 'num_tokens': '6.816e+05', 'mean_token_accuracy': '0.8377', 'epoch': '0.09499'}
{'loss': '0.8358', 'grad_norm': '0.442', 'learning_rate': '6.034e-06', 'entropy': '0.8946', 'num_tokens': '6.859e+05', 'mean_token_accuracy': '0.78', 'epoch': '0.09499'}
{'loss': '0.4702', 'grad_norm': '0.2036', 'learning_rate': '6.024e-06', 'entropy': '0.5214', 'num_tokens': '6.926e+05', 'mean_token_accuracy': '0.8605', 'epoch': '0.095'}
{'loss': '0.6231', 'grad_norm': '0.3496', 'learning_rate': '6.015e-06', 'entropy': '0.6163', 'num_tokens': '6.982e+05', 'mean_token_accuracy': '0.8327', 'epoch': '0.09501'}
{'loss': '0.6845', 'grad_norm': '0.4275', 'learning_rate': '6.006e-06', 'entropy': '0.6923', 'num_tokens': '7.03e+05', 'mean_token_accuracy': '0.8141', 'epoch': '0.09502'}
{'loss': '0.948', 'grad_norm': '0.4375', 'learning_rate': '5.996e-06', 'entropy': '0.8773', 'num_tokens': '7.082e+05', 'mean_token_accuracy': '0.7811', 'epoch': '0.09503'}
{'loss': '0.7324', 'grad_norm': '0.354', 'learning_rate': '5.987e-06', 'entropy': '0.8278', 'num_tokens': '7.14e+05', 'mean_token_accuracy': '0.7882', 'epoch': '0.09504'}
{'loss': '1.02', 'grad_norm': '0.3961', 'learning_rate': '5.977e-06', 'entropy': '0.9574', 'num_tokens': '7.23e+05', 'mean_token_accuracy': '0.7615', 'epoch': '0.09505'}
{'loss': '0.6324', 'grad_norm': '0.3526', 'learning_rate': '5.968e-06', 'entropy': '0.6698', 'num_tokens': '7.307e+05', 'mean_token_accuracy': '0.8297', 'epoch': '0.09506'}

 90%|████████▉ | 10742/12000 [04:52<47:10,  2.25s/it]
 90%|████████▉ | 10743/12000 [04:54<44:28,  2.12s/it]
                                                     

 90%|████████▉ | 10743/12000 [04:54<44:28,  2.12s/it]
 90%|████████▉ | 10744/12000 [04:56<40:30,  1.93s/it]
                                                     

 90%|████████▉ | 10744/12000 [04:56<40:30,  1.93s/it]
 90%|████████▉ | 10745/12000 [04:57<37:49,  1.81s/it]
                                                     

 90%|████████▉ | 10745/12000 [04:57<37:49,  1.81s/it]
 90%|████████▉ | 10746/12000 [05:00<44:26,  2.13s/it]
                                                     

 90%|████████▉ | 10746/12000 [05:00<44:26,  2.13s/it]
 90%|████████▉ | 10747/12000 [05:04<52:34,  2.52s/it]
                                                     

 90%|████████▉ | 10747/12000 [05:04<52:34,  2.52s/it]
 90%|████████▉ | 10748/12000 [05:05<48:02,  2.30s/it]
                                                     

 90%|████████▉ | 10748/12000 [05:05<48:02,  2.30s/it]
 90%|████████▉ | 10749/12000 [05:07<43:48,  2.10s/it]
                                                     

 90%|████████▉ | 10749/12000 [05:07<43:48,  2.10s/it]
 90%|████████▉ | 10750/12000 [05:09<40:37,  1.95s/it]
                                                     

 90%|████████▉ | 10750/12000 [05:09<40:37,  1.95s/it]
 90%|████████▉ | 10751/12000 [05:11<46:42,  2.24s/it]
                                                     

 90%|████████▉ | 10751/12000 [05:11<46:42,  2.24s/it]
 90%|████████▉ | 10752/12000 [05:13<42:23,  2.04s/it]
                                                     

 90%|████████▉ | 10752/12000 [05:13<42:23,  2.04s/it]
 90%|████████▉ | 10753/12000 [05:16<45:39,  2.20s/it]
                                                     

 90%|████████▉ | 10753/12000 [05:16<45:39,  2.20s/it]
 90%|████████▉ | 10754/12000 [05:19<55:10,  2.66s/it]
                                                     

 90%|████████▉ | 10754/12000 [05:19<55:10,  2.66s/it]
 90%|████████▉ | 10755/12000 [05:22<53:45,  2.59s/it]
                                                     

 90%|████████▉ | 10755/12000 [05:22<53:45,  2.59s/it]
 90%|████████▉ | 10756/12000 [05:23<46:10,  2.23s/it]
                                                     

 90%|████████▉ | 10756/12000 [05:23<46:10,  2.23s/it]
 90%|████████▉ | 10757/12000 [05:25<42:53,  2.07s/it]
                                                     

 90%|████████▉ | 10757/12000 [05:25<42:53,  2.07s/it]
 90%|████████▉ | 10758/12000 [05:27<43:15,  2.09s/it]
                                                     

 90%|████████▉ | 10758/12000 [05:27<43:15,  2.09s/it]
 90%|████████▉ | 10759/12000 [05:29<39:41,  1.92s/it]
                                                     

 90%|████████▉ | 10759/12000 [05:29<39:41,  1.92s/it]
 90%|████████▉ | 10760/12000 [05:31<41:16,  2.00s/it]
                                                     

 90%|████████▉ | 10760/12000 [05:31<41:16,  2.00s/it]
 90%|████████▉ | 10761/12000 [05:32<38:59,  1.89s/it]
                                                     

 90%|████████▉ | 10761/12000 [05:32<38:59,  1.89s/it]
 90%|████████▉ | 10762/12000 [05:35<46:48,  2.27s/it]
                                                     

 90%|████████▉ | 10762/12000 [05:35<46:48,  2.27s/it]
 90%|████████▉ | 10763/12000 [05:37<42:30,  2.06s/it]
                                                     

 90%|████████▉ | 10763/12000 [05:37<42:30,  2.06s/it]
 90%|████████▉ | 10764/12000 [05:39<39:38,  1.92s/it]
                                                     

 90%|████████▉ | 10764/12000 [05:39<39:38,  1.92s/it]
 90%|████████▉ | 10765/12000 [05:40<37:06,  1.80s/it]
                                                     

 90%|████████▉ | 10765/12000 [05:40<37:06,  1.80s/it]
 90%|████████▉ | 10766/12000 [05:42<35:59,  1.75s/it]
                                                     

 90%|████████▉ | 10766/12000 [05:42<35:59,  1.75s/it]
 90%|████████▉ | 10767/12000 [05:43<35:08,  1.71s/it]
                                                     

 90%|████████▉ | 10767/12000 [05:43<35:08,  1.71s/it]
 90%|████████▉ | 10768/12000 [05:45<33:10,  1.62s/it]
                                                     

 90%|████████▉ | 10768/12000 [05:45<33:10,  1.62s/it]
 90%|████████▉ | 10769/12000 [05:46<32:35,  1.59s/it]
                                                     

 90%|████████▉ | 10769/12000 [05:46<32:35,  1.59s/it]
 90%|████████▉ | 10770/12000 [05:48<31:50,  1.55s/it]
                                                     

 90%|████████▉ | 10770/12000 [05:48<31:50,  1.55s/it]
 90%|████████▉ | 10771/12000 [05:50<37:48,  1.85s/it]
                                                     

 90%|████████▉ | 10771/12000 [05:50<37:48,  1.85s/it]
 90%|████████▉ | 10772/12000 [05:52<36:48,  1.80s/it]
                                                     

 90%|████████▉ | 10772/12000 [05:52<36:48,  1.80s/it]
 90%|████████▉ | 10773/12000 [05:54<36:54,  1.80s/it]
                                                     

 90%|████████▉ | 10773/12000 [05:54<36:54,  1.80s/it]
 90%|████████▉ | 10774/12000 [05:56<35:55,  1.76s/it]
                                                     

 90%|████████▉ | 10774/12000 [05:56<35:55,  1.76s/it]
 90%|████████▉ | 10775/12000 [05:57<33:27,  1.64s/it]
                                                     

 90%|████████▉ | 10775/12000 [05:57<33:27,  1.64s/it]
 90%|████████▉ | 10776/12000 [06:00<45:34,  2.23s/it]
                                                     

 90%|████████▉ | 10776/12000 [06:00<45:34,  2.23s/it]
 90%|████████▉ | 10777/12000 [06:02<41:04,  2.02s/it]
                                                     

 90%|████████▉ | 10777/12000 [06:02<41:04,  2.02s/it]
 90%|████████▉ | 10778/12000 [06:03<37:14,  1.83s/it]
                                                     

 90%|████████▉ | 10778/12000 [06:03<37:14,  1.83s/it]
 90%|████████▉ | 10779/12000 [06:05<34:55,  1.72s/it]
                                                     

 90%|████████▉ | 10779/12000 [06:05<34:55,  1.72s/it]
 90%|████████▉ | 10780/12000 [06:06<34:19,  1.69s/it]
                                                     

 90%|████████▉ | 10780/12000 [06:06<34:19,  1.69s/it]
 90%|████████▉ | 10781/12000 [06:08<34:55,  1.72s/it]
                                                     

 90%|████████▉ | 10781/12000 [06:08<34:55,  1.72s/it]
 90%|████████▉ | 10782/12000 [06:10<36:46,  1.81s/it]
                                                     

 90%|████████▉ | 10782/12000 [06:10<36:46,  1.81s/it]
 90%|████████▉ | 10783/12000 [06:12<36:17,  1.79s/it]
                                                     

 90%|████████▉ | 10783/12000 [06:12<36:17,  1.79s/it]
 90%|████████▉ | 10784/12000 [06:14<35:33,  1.75s/it]
                                                     

 90%|████████▉ | 10784/12000 [06:14<35:33,  1.75s/it]
 90%|████████▉ | 10785/12000 [06:16<36:07,  1.78s/it]
                                                     

 90%|████████▉ | 10785/12000 [06:16<36:07,  1.78s/it]
 90%|████████▉ | 10786/12000 [06:17<35:22,  1.75s/it]
                                                     

 90%|████████▉ | 10786/12000 [06:17<35:22,  1.75s/it]
 90%|████████▉ | 10787/12000 [06:20<38:37,  1.91s/it]
                                                     

 90%|████████▉ | 10787/12000 [06:20<38:37,  1.91s/it]
 90%|████████▉ | 10788/12000 [06:21<37:37,  1.86s/it]
                                                     

 90%|████████▉ | 10788/12000 [06:21<37:37,  1.86s/it]
 90%|████████▉ | 10789/12000 [06:24<40:01,  1.98s/it]
                                                     
{'loss': '0.6407', 'grad_norm': '0.3425', 'learning_rate': '5.959e-06', 'entropy': '0.6367', 'num_tokens': '7.354e+05', 'mean_token_accuracy': '0.8317', 'epoch': '0.09506'}
{'loss': '1.117', 'grad_norm': '0.4072', 'learning_rate': '5.949e-06', 'entropy': '1.193', 'num_tokens': '7.411e+05', 'mean_token_accuracy': '0.7231', 'epoch': '0.09507'}
{'loss': '0.5023', 'grad_norm': '0.3904', 'learning_rate': '5.94e-06', 'entropy': '0.5566', 'num_tokens': '7.461e+05', 'mean_token_accuracy': '0.8512', 'epoch': '0.09508'}
{'loss': '0.9419', 'grad_norm': '0.4032', 'learning_rate': '5.931e-06', 'entropy': '0.9066', 'num_tokens': '7.503e+05', 'mean_token_accuracy': '0.7641', 'epoch': '0.09509'}
{'loss': '0.7012', 'grad_norm': '0.3838', 'learning_rate': '5.921e-06', 'entropy': '0.7991', 'num_tokens': '7.569e+05', 'mean_token_accuracy': '0.7893', 'epoch': '0.0951'}
{'loss': '0.9754', 'grad_norm': '0.3525', 'learning_rate': '5.912e-06', 'entropy': '1.071', 'num_tokens': '7.656e+05', 'mean_token_accuracy': '0.7337', 'epoch': '0.09511'}
{'loss': '1.158', 'grad_norm': '0.5408', 'learning_rate': '5.903e-06', 'entropy': '1.109', 'num_tokens': '7.702e+05', 'mean_token_accuracy': '0.7287', 'epoch': '0.09512'}
{'loss': '0.6373', 'grad_norm': '0.3415', 'learning_rate': '5.893e-06', 'entropy': '0.6903', 'num_tokens': '7.748e+05', 'mean_token_accuracy': '0.8071', 'epoch': '0.09513'}
{'loss': '0.8133', 'grad_norm': '0.4235', 'learning_rate': '5.884e-06', 'entropy': '0.97', 'num_tokens': '7.787e+05', 'mean_token_accuracy': '0.7673', 'epoch': '0.09514'}
{'loss': '0.6435', 'grad_norm': '0.3741', 'learning_rate': '5.875e-06', 'entropy': '0.6936', 'num_tokens': '7.828e+05', 'mean_token_accuracy': '0.8138', 'epoch': '0.09514'}
{'loss': '0.668', 'grad_norm': '0.4823', 'learning_rate': '5.865e-06', 'entropy': '0.7108', 'num_tokens': '7.868e+05', 'mean_token_accuracy': '0.8081', 'epoch': '0.09515'}
{'loss': '0.752', 'grad_norm': '0.4174', 'learning_rate': '5.856e-06', 'entropy': '0.7915', 'num_tokens': '7.928e+05', 'mean_token_accuracy': '0.7994', 'epoch': '0.09516'}
{'loss': '0.9286', 'grad_norm': '0.4051', 'learning_rate': '5.847e-06', 'entropy': '0.8794', 'num_tokens': '7.973e+05', 'mean_token_accuracy': '0.7717', 'epoch': '0.09517'}
{'loss': '0.7883', 'grad_norm': '0.393', 'learning_rate': '5.837e-06', 'entropy': '0.7784', 'num_tokens': '8.036e+05', 'mean_token_accuracy': '0.8037', 'epoch': '0.09518'}
{'loss': '1.158', 'grad_norm': '0.4777', 'learning_rate': '5.828e-06', 'entropy': '1.111', 'num_tokens': '8.07e+05', 'mean_token_accuracy': '0.7214', 'epoch': '0.09519'}
{'loss': '1.143', 'grad_norm': '0.4727', 'learning_rate': '5.819e-06', 'entropy': '1.173', 'num_tokens': '8.125e+05', 'mean_token_accuracy': '0.7205', 'epoch': '0.0952'}
{'loss': '0.4696', 'grad_norm': '0.3651', 'learning_rate': '5.81e-06', 'entropy': '0.5175', 'num_tokens': '8.182e+05', 'mean_token_accuracy': '0.8602', 'epoch': '0.09521'}
{'loss': '0.7589', 'grad_norm': '0.3772', 'learning_rate': '5.8e-06', 'entropy': '0.7597', 'num_tokens': '8.222e+05', 'mean_token_accuracy': '0.8015', 'epoch': '0.09522'}
{'loss': '0.8842', 'grad_norm': '0.4372', 'learning_rate': '5.791e-06', 'entropy': '0.8266', 'num_tokens': '8.294e+05', 'mean_token_accuracy': '0.7836', 'epoch': '0.09522'}
{'loss': '0.6394', 'grad_norm': '0.3743', 'learning_rate': '5.782e-06', 'entropy': '0.6513', 'num_tokens': '8.348e+05', 'mean_token_accuracy': '0.8305', 'epoch': '0.09523'}
{'loss': '0.6661', 'grad_norm': '0.3763', 'learning_rate': '5.773e-06', 'entropy': '0.6491', 'num_tokens': '8.441e+05', 'mean_token_accuracy': '0.8343', 'epoch': '0.09524'}
{'loss': '0.6147', 'grad_norm': '0.3656', 'learning_rate': '5.763e-06', 'entropy': '0.6415', 'num_tokens': '8.491e+05', 'mean_token_accuracy': '0.8431', 'epoch': '0.09525'}
{'loss': '0.8254', 'grad_norm': '0.4189', 'learning_rate': '5.754e-06', 'entropy': '0.878', 'num_tokens': '8.537e+05', 'mean_token_accuracy': '0.7778', 'epoch': '0.09526'}
{'loss': '0.8967', 'grad_norm': '0.3663', 'learning_rate': '5.745e-06', 'entropy': '0.939', 'num_tokens': '8.586e+05', 'mean_token_accuracy': '0.7718', 'epoch': '0.09527'}
{'loss': '1.013', 'grad_norm': '0.4518', 'learning_rate': '5.736e-06', 'entropy': '1.019', 'num_tokens': '8.631e+05', 'mean_token_accuracy': '0.7532', 'epoch': '0.09528'}
{'loss': '0.6047', 'grad_norm': '0.3513', 'learning_rate': '5.727e-06', 'entropy': '0.6119', 'num_tokens': '8.681e+05', 'mean_token_accuracy': '0.8353', 'epoch': '0.09529'}
{'loss': '0.7511', 'grad_norm': '0.357', 'learning_rate': '5.717e-06', 'entropy': '0.7501', 'num_tokens': '8.72e+05', 'mean_token_accuracy': '0.8039', 'epoch': '0.09529'}
{'loss': '0.7343', 'grad_norm': '0.5485', 'learning_rate': '5.708e-06', 'entropy': '0.7355', 'num_tokens': '8.757e+05', 'mean_token_accuracy': '0.8025', 'epoch': '0.0953'}
{'loss': '0.7129', 'grad_norm': '0.3097', 'learning_rate': '5.699e-06', 'entropy': '0.6785', 'num_tokens': '8.803e+05', 'mean_token_accuracy': '0.8211', 'epoch': '0.09531'}
{'loss': '1.076', 'grad_norm': '0.3556', 'learning_rate': '5.69e-06', 'entropy': '0.9933', 'num_tokens': '8.867e+05', 'mean_token_accuracy': '0.74', 'epoch': '0.09532'}
{'loss': '0.6432', 'grad_norm': '0.3396', 'learning_rate': '5.681e-06', 'entropy': '0.6845', 'num_tokens': '8.92e+05', 'mean_token_accuracy': '0.8191', 'epoch': '0.09533'}
{'loss': '0.9329', 'grad_norm': '0.5009', 'learning_rate': '5.672e-06', 'entropy': '0.8921', 'num_tokens': '8.966e+05', 'mean_token_accuracy': '0.7682', 'epoch': '0.09534'}
{'loss': '0.6167', 'grad_norm': '0.3013', 'learning_rate': '5.662e-06', 'entropy': '0.6325', 'num_tokens': '9.021e+05', 'mean_token_accuracy': '0.8442', 'epoch': '0.09535'}
{'loss': '0.8905', 'grad_norm': '0.4719', 'learning_rate': '5.653e-06', 'entropy': '0.8971', 'num_tokens': '9.063e+05', 'mean_token_accuracy': '0.7641', 'epoch': '0.09536'}
{'loss': '1.039', 'grad_norm': '0.4542', 'learning_rate': '5.644e-06', 'entropy': '1.039', 'num_tokens': '9.117e+05', 'mean_token_accuracy': '0.7494', 'epoch': '0.09537'}
{'loss': '0.9132', 'grad_norm': '0.3826', 'learning_rate': '5.635e-06', 'entropy': '0.9386', 'num_tokens': '9.164e+05', 'mean_token_accuracy': '0.7629', 'epoch': '0.09537'}
{'loss': '0.9343', 'grad_norm': '0.3857', 'learning_rate': '5.626e-06', 'entropy': '0.8352', 'num_tokens': '9.205e+05', 'mean_token_accuracy': '0.7981', 'epoch': '0.09538'}
{'loss': '0.5475', 'grad_norm': '0.2916', 'learning_rate': '5.617e-06', 'entropy': '0.5686', 'num_tokens': '9.248e+05', 'mean_token_accuracy': '0.8426', 'epoch': '0.09539'}
{'loss': '0.6736', 'grad_norm': '0.439', 'learning_rate': '5.608e-06', 'entropy': '0.7381', 'num_tokens': '9.294e+05', 'mean_token_accuracy': '0.806', 'epoch': '0.0954'}
{'loss': '1.041', 'grad_norm': '0.6094', 'learning_rate': '5.599e-06', 'entropy': '1.013', 'num_tokens': '9.354e+05', 'mean_token_accuracy': '0.744', 'epoch': '0.09541'}
{'loss': '1.158', 'grad_norm': '0.3898', 'learning_rate': '5.59e-06', 'entropy': '1.079', 'num_tokens': '9.414e+05', 'mean_token_accuracy': '0.7204', 'epoch': '0.09542'}
{'loss': '0.731', 'grad_norm': '0.3447', 'learning_rate': '5.581e-06', 'entropy': '0.7376', 'num_tokens': '9.463e+05', 'mean_token_accuracy': '0.8188', 'epoch': '0.09543'}
{'loss': '0.7511', 'grad_norm': '0.3876', 'learning_rate': '5.571e-06', 'entropy': '0.6767', 'num_tokens': '9.518e+05', 'mean_token_accuracy': '0.8194', 'epoch': '0.09544'}
{'loss': '0.8537', 'grad_norm': '0.42', 'learning_rate': '5.562e-06', 'entropy': '0.8588', 'num_tokens': '9.58e+05', 'mean_token_accuracy': '0.7716', 'epoch': '0.09545'}
{'loss': '0.6661', 'grad_norm': '0.448', 'learning_rate': '5.553e-06', 'entropy': '0.8293', 'num_tokens': '9.633e+05', 'mean_token_accuracy': '0.7859', 'epoch': '0.09545'}
{'loss': '0.7293', 'grad_norm': '0.3925', 'learning_rate': '5.544e-06', 'entropy': '0.788', 'num_tokens': '9.693e+05', 'mean_token_accuracy': '0.7905', 'epoch': '0.09546'}
{'loss': '0.9651', 'grad_norm': '0.4509', 'learning_rate': '5.535e-06', 'entropy': '1.053', 'num_tokens': '9.748e+05', 'mean_token_accuracy': '0.7466', 'epoch': '0.09547'}

 90%|████████▉ | 10789/12000 [06:24<40:01,  1.98s/it]
 90%|████████▉ | 10790/12000 [06:25<36:33,  1.81s/it]
                                                     

 90%|████████▉ | 10790/12000 [06:25<36:33,  1.81s/it]
 90%|████████▉ | 10791/12000 [06:27<36:10,  1.80s/it]
                                                     

 90%|████████▉ | 10791/12000 [06:27<36:10,  1.80s/it]
 90%|████████▉ | 10792/12000 [06:28<35:57,  1.79s/it]
                                                     

 90%|████████▉ | 10792/12000 [06:28<35:57,  1.79s/it]
 90%|████████▉ | 10793/12000 [06:31<38:15,  1.90s/it]
                                                     

 90%|████████▉ | 10793/12000 [06:31<38:15,  1.90s/it]
 90%|████████▉ | 10794/12000 [06:32<35:17,  1.76s/it]
                                                     

 90%|████████▉ | 10794/12000 [06:32<35:17,  1.76s/it]
 90%|████████▉ | 10795/12000 [06:34<38:56,  1.94s/it]
                                                     

 90%|████████▉ | 10795/12000 [06:34<38:56,  1.94s/it]
 90%|████████▉ | 10796/12000 [06:37<41:28,  2.07s/it]
                                                     

 90%|████████▉ | 10796/12000 [06:37<41:28,  2.07s/it]
 90%|████████▉ | 10797/12000 [06:38<38:52,  1.94s/it]
                                                     

 90%|████████▉ | 10797/12000 [06:38<38:52,  1.94s/it]
 90%|████████▉ | 10798/12000 [06:43<55:06,  2.75s/it]
                                                     

 90%|████████▉ | 10798/12000 [06:43<55:06,  2.75s/it]
 90%|████████▉ | 10799/12000 [06:44<47:07,  2.35s/it]
                                                     

 90%|████████▉ | 10799/12000 [06:44<47:07,  2.35s/it]
 90%|█████████ | 10800/12000 [06:46<43:52,  2.19s/it]
                                                     

 90%|█████████ | 10800/12000 [06:46<43:52,  2.19s/it]
 90%|█████████ | 10801/12000 [06:50<50:49,  2.54s/it]
                                                     

 90%|█████████ | 10801/12000 [06:50<50:49,  2.54s/it]
 90%|█████████ | 10802/12000 [06:52<47:56,  2.40s/it]
                                                     

 90%|█████████ | 10802/12000 [06:52<47:56,  2.40s/it]
 90%|█████████ | 10803/12000 [06:53<43:27,  2.18s/it]
                                                     

 90%|█████████ | 10803/12000 [06:53<43:27,  2.18s/it]
 90%|█████████ | 10804/12000 [06:55<38:45,  1.94s/it]
                                                     

 90%|█████████ | 10804/12000 [06:55<38:45,  1.94s/it]
 90%|█████████ | 10805/12000 [07:00<55:21,  2.78s/it]
                                                     

 90%|█████████ | 10805/12000 [07:00<55:21,  2.78s/it]
 90%|█████████ | 10806/12000 [07:03<58:22,  2.93s/it]
                                                     

 90%|█████████ | 10806/12000 [07:03<58:22,  2.93s/it]
 90%|█████████ | 10807/12000 [07:04<49:15,  2.48s/it]
                                                     

 90%|█████████ | 10807/12000 [07:04<49:15,  2.48s/it]
 90%|█████████ | 10808/12000 [07:06<45:05,  2.27s/it]
                                                     

 90%|█████████ | 10808/12000 [07:06<45:05,  2.27s/it]
 90%|█████████ | 10809/12000 [07:08<41:35,  2.10s/it]
                                                     

 90%|█████████ | 10809/12000 [07:08<41:35,  2.10s/it]
 90%|█████████ | 10810/12000 [07:09<38:44,  1.95s/it]
                                                     

 90%|█████████ | 10810/12000 [07:09<38:44,  1.95s/it]
 90%|█████████ | 10811/12000 [07:11<36:10,  1.83s/it]
                                                     

 90%|█████████ | 10811/12000 [07:11<36:10,  1.83s/it]
 90%|█████████ | 10812/12000 [07:12<33:30,  1.69s/it]
                                                     

 90%|█████████ | 10812/12000 [07:12<33:30,  1.69s/it]
 90%|█████████ | 10813/12000 [07:15<38:31,  1.95s/it]
                                                     

 90%|█████████ | 10813/12000 [07:15<38:31,  1.95s/it]
 90%|█████████ | 10814/12000 [07:17<38:05,  1.93s/it]
                                                     

 90%|█████████ | 10814/12000 [07:17<38:05,  1.93s/it]
 90%|█████████ | 10815/12000 [07:20<44:05,  2.23s/it]
                                                     

 90%|█████████ | 10815/12000 [07:20<44:05,  2.23s/it]
 90%|█████████ | 10816/12000 [07:21<40:00,  2.03s/it]
                                                     

 90%|█████████ | 10816/12000 [07:21<40:00,  2.03s/it]
 90%|█████████ | 10817/12000 [07:23<39:15,  1.99s/it]
                                                     

 90%|█████████ | 10817/12000 [07:23<39:15,  1.99s/it]
 90%|█████████ | 10818/12000 [07:26<42:20,  2.15s/it]
                                                     

 90%|█████████ | 10818/12000 [07:26<42:20,  2.15s/it]
 90%|█████████ | 10819/12000 [07:29<48:02,  2.44s/it]
                                                     

 90%|█████████ | 10819/12000 [07:29<48:02,  2.44s/it]
 90%|█████████ | 10820/12000 [07:32<55:09,  2.80s/it]
                                                     

 90%|█████████ | 10820/12000 [07:32<55:09,  2.80s/it]
 90%|█████████ | 10821/12000 [07:34<47:19,  2.41s/it]
                                                     

 90%|█████████ | 10821/12000 [07:34<47:19,  2.41s/it]
 90%|█████████ | 10822/12000 [07:35<41:25,  2.11s/it]
                                                     

 90%|█████████ | 10822/12000 [07:35<41:25,  2.11s/it]
 90%|█████████ | 10823/12000 [07:37<39:58,  2.04s/it]
                                                     

 90%|█████████ | 10823/12000 [07:37<39:58,  2.04s/it]
 90%|█████████ | 10824/12000 [07:38<36:06,  1.84s/it]
                                                     

 90%|█████████ | 10824/12000 [07:38<36:06,  1.84s/it]
 90%|█████████ | 10825/12000 [07:40<35:49,  1.83s/it]
                                                     

 90%|█████████ | 10825/12000 [07:40<35:49,  1.83s/it]
 90%|█████████ | 10826/12000 [07:43<42:43,  2.18s/it]
                                                     

 90%|█████████ | 10826/12000 [07:43<42:43,  2.18s/it]
 90%|█████████ | 10827/12000 [07:45<38:30,  1.97s/it]
                                                     

 90%|█████████ | 10827/12000 [07:45<38:30,  1.97s/it]
 90%|█████████ | 10828/12000 [07:46<35:58,  1.84s/it]
                                                     

 90%|█████████ | 10828/12000 [07:46<35:58,  1.84s/it]
 90%|█████████ | 10829/12000 [07:49<40:58,  2.10s/it]
                                                     

 90%|█████████ | 10829/12000 [07:49<40:58,  2.10s/it]
 90%|█████████ | 10830/12000 [07:51<39:35,  2.03s/it]
                                                     

 90%|█████████ | 10830/12000 [07:51<39:35,  2.03s/it]
 90%|█████████ | 10831/12000 [07:53<37:25,  1.92s/it]
                                                     

 90%|█████████ | 10831/12000 [07:53<37:25,  1.92s/it]
 90%|█████████ | 10832/12000 [07:54<35:29,  1.82s/it]
                                                     

 90%|█████████ | 10832/12000 [07:54<35:29,  1.82s/it]
 90%|█████████ | 10833/12000 [07:56<33:53,  1.74s/it]
                                                     

 90%|█████████ | 10833/12000 [07:56<33:53,  1.74s/it]
 90%|█████████ | 10834/12000 [07:57<31:59,  1.65s/it]
                                                     

 90%|█████████ | 10834/12000 [07:57<31:59,  1.65s/it]
 90%|█████████ | 10835/12000 [07:59<31:36,  1.63s/it]
                                                     

 90%|█████████ | 10835/12000 [07:59<31:36,  1.63s/it]
 90%|█████████ | 10836/12000 [08:03<46:07,  2.38s/it]
                                                     
{'loss': '0.4774', 'grad_norm': '0.2614', 'learning_rate': '5.526e-06', 'entropy': '0.502', 'num_tokens': '9.813e+05', 'mean_token_accuracy': '0.8668', 'epoch': '0.09548'}
{'loss': '0.7378', 'grad_norm': '0.4535', 'learning_rate': '5.517e-06', 'entropy': '0.7858', 'num_tokens': '9.853e+05', 'mean_token_accuracy': '0.8004', 'epoch': '0.09549'}
{'loss': '1.046', 'grad_norm': '0.4527', 'learning_rate': '5.508e-06', 'entropy': '0.9576', 'num_tokens': '9.902e+05', 'mean_token_accuracy': '0.759', 'epoch': '0.0955'}
{'loss': '0.7731', 'grad_norm': '0.3655', 'learning_rate': '5.499e-06', 'entropy': '0.7534', 'num_tokens': '9.95e+05', 'mean_token_accuracy': '0.8009', 'epoch': '0.09551'}
{'loss': '0.8612', 'grad_norm': '0.4819', 'learning_rate': '5.49e-06', 'entropy': '0.8944', 'num_tokens': '1e+06', 'mean_token_accuracy': '0.7791', 'epoch': '0.09552'}
{'loss': '1.093', 'grad_norm': '0.5109', 'learning_rate': '5.481e-06', 'entropy': '1.048', 'num_tokens': '1.004e+06', 'mean_token_accuracy': '0.755', 'epoch': '0.09553'}
{'loss': '0.4207', 'grad_norm': '0.3515', 'learning_rate': '5.472e-06', 'entropy': '0.5743', 'num_tokens': '1.01e+06', 'mean_token_accuracy': '0.8425', 'epoch': '0.09553'}
{'loss': '0.5286', 'grad_norm': '0.3763', 'learning_rate': '5.463e-06', 'entropy': '0.4198', 'num_tokens': '1.016e+06', 'mean_token_accuracy': '0.883', 'epoch': '0.09554'}
{'loss': '0.6603', 'grad_norm': '0.357', 'learning_rate': '5.454e-06', 'entropy': '0.7114', 'num_tokens': '1.022e+06', 'mean_token_accuracy': '0.8237', 'epoch': '0.09555'}
{'loss': '0.7095', 'grad_norm': '0.4347', 'learning_rate': '5.445e-06', 'entropy': '0.7041', 'num_tokens': '1.028e+06', 'mean_token_accuracy': '0.8319', 'epoch': '0.09556'}
{'loss': '0.7795', 'grad_norm': '0.3584', 'learning_rate': '5.436e-06', 'entropy': '0.8046', 'num_tokens': '1.032e+06', 'mean_token_accuracy': '0.7857', 'epoch': '0.09557'}
{'loss': '0.7698', 'grad_norm': '0.4506', 'learning_rate': '5.427e-06', 'entropy': '0.8005', 'num_tokens': '1.039e+06', 'mean_token_accuracy': '0.8005', 'epoch': '0.09558'}
{'loss': '0.792', 'grad_norm': '0.3428', 'learning_rate': '5.418e-06', 'entropy': '0.7527', 'num_tokens': '1.045e+06', 'mean_token_accuracy': '0.8086', 'epoch': '0.09559'}
{'loss': '0.9193', 'grad_norm': '0.4981', 'learning_rate': '5.409e-06', 'entropy': '0.9239', 'num_tokens': '1.05e+06', 'mean_token_accuracy': '0.7642', 'epoch': '0.0956'}
{'loss': '0.8397', 'grad_norm': '0.5517', 'learning_rate': '5.4e-06', 'entropy': '0.8279', 'num_tokens': '1.054e+06', 'mean_token_accuracy': '0.779', 'epoch': '0.0956'}
{'loss': '0.6206', 'grad_norm': '0.3847', 'learning_rate': '5.391e-06', 'entropy': '0.6012', 'num_tokens': '1.058e+06', 'mean_token_accuracy': '0.853', 'epoch': '0.09561'}
{'loss': '1.241', 'grad_norm': '0.4444', 'learning_rate': '5.383e-06', 'entropy': '1.067', 'num_tokens': '1.066e+06', 'mean_token_accuracy': '0.7483', 'epoch': '0.09562'}
{'loss': '1.098', 'grad_norm': '0.4514', 'learning_rate': '5.374e-06', 'entropy': '0.9775', 'num_tokens': '1.071e+06', 'mean_token_accuracy': '0.7501', 'epoch': '0.09563'}
{'loss': '0.7404', 'grad_norm': '0.3439', 'learning_rate': '5.365e-06', 'entropy': '0.7956', 'num_tokens': '1.076e+06', 'mean_token_accuracy': '0.809', 'epoch': '0.09564'}
{'loss': '1.099', 'grad_norm': '0.4316', 'learning_rate': '5.356e-06', 'entropy': '1.15', 'num_tokens': '1.082e+06', 'mean_token_accuracy': '0.7418', 'epoch': '0.09565'}
{'loss': '0.6633', 'grad_norm': '0.3816', 'learning_rate': '5.347e-06', 'entropy': '0.6994', 'num_tokens': '1.087e+06', 'mean_token_accuracy': '0.8142', 'epoch': '0.09566'}
{'loss': '0.816', 'grad_norm': '0.4769', 'learning_rate': '5.338e-06', 'entropy': '0.8425', 'num_tokens': '1.091e+06', 'mean_token_accuracy': '0.7938', 'epoch': '0.09567'}
{'loss': '0.8216', 'grad_norm': '0.4433', 'learning_rate': '5.329e-06', 'entropy': '0.8101', 'num_tokens': '1.096e+06', 'mean_token_accuracy': '0.7947', 'epoch': '0.09568'}
{'loss': '0.5936', 'grad_norm': '0.3816', 'learning_rate': '5.32e-06', 'entropy': '0.6391', 'num_tokens': '1.1e+06', 'mean_token_accuracy': '0.8326', 'epoch': '0.09568'}
{'loss': '0.7035', 'grad_norm': '0.3465', 'learning_rate': '5.311e-06', 'entropy': '0.6643', 'num_tokens': '1.107e+06', 'mean_token_accuracy': '0.8296', 'epoch': '0.09569'}
{'loss': '0.8285', 'grad_norm': '0.3387', 'learning_rate': '5.303e-06', 'entropy': '0.8542', 'num_tokens': '1.112e+06', 'mean_token_accuracy': '0.7996', 'epoch': '0.0957'}
{'loss': '0.8073', 'grad_norm': '0.4046', 'learning_rate': '5.294e-06', 'entropy': '0.8549', 'num_tokens': '1.118e+06', 'mean_token_accuracy': '0.7811', 'epoch': '0.09571'}
{'loss': '0.6815', 'grad_norm': '0.3852', 'learning_rate': '5.285e-06', 'entropy': '0.703', 'num_tokens': '1.123e+06', 'mean_token_accuracy': '0.8198', 'epoch': '0.09572'}
{'loss': '0.7335', 'grad_norm': '0.4145', 'learning_rate': '5.276e-06', 'entropy': '0.7654', 'num_tokens': '1.128e+06', 'mean_token_accuracy': '0.8103', 'epoch': '0.09573'}
{'loss': '0.5349', 'grad_norm': '0.28', 'learning_rate': '5.267e-06', 'entropy': '0.5477', 'num_tokens': '1.136e+06', 'mean_token_accuracy': '0.8572', 'epoch': '0.09574'}
{'loss': '0.6228', 'grad_norm': '0.2741', 'learning_rate': '5.258e-06', 'entropy': '0.5974', 'num_tokens': '1.143e+06', 'mean_token_accuracy': '0.844', 'epoch': '0.09575'}
{'loss': '0.7537', 'grad_norm': '0.4133', 'learning_rate': '5.25e-06', 'entropy': '0.809', 'num_tokens': '1.148e+06', 'mean_token_accuracy': '0.7916', 'epoch': '0.09576'}
{'loss': '0.572', 'grad_norm': '0.3973', 'learning_rate': '5.241e-06', 'entropy': '0.6618', 'num_tokens': '1.152e+06', 'mean_token_accuracy': '0.8291', 'epoch': '0.09576'}
{'loss': '0.7865', 'grad_norm': '0.5068', 'learning_rate': '5.232e-06', 'entropy': '0.7548', 'num_tokens': '1.157e+06', 'mean_token_accuracy': '0.8055', 'epoch': '0.09577'}
{'loss': '0.7212', 'grad_norm': '0.3881', 'learning_rate': '5.223e-06', 'entropy': '0.7476', 'num_tokens': '1.163e+06', 'mean_token_accuracy': '0.8126', 'epoch': '0.09578'}
{'loss': '0.7911', 'grad_norm': '0.4115', 'learning_rate': '5.214e-06', 'entropy': '0.8079', 'num_tokens': '1.167e+06', 'mean_token_accuracy': '0.7975', 'epoch': '0.09579'}
{'loss': '0.8457', 'grad_norm': '0.4172', 'learning_rate': '5.206e-06', 'entropy': '0.8712', 'num_tokens': '1.172e+06', 'mean_token_accuracy': '0.7967', 'epoch': '0.0958'}
{'loss': '0.5267', 'grad_norm': '0.327', 'learning_rate': '5.197e-06', 'entropy': '0.5916', 'num_tokens': '1.179e+06', 'mean_token_accuracy': '0.8395', 'epoch': '0.09581'}
{'loss': '0.6562', 'grad_norm': '0.404', 'learning_rate': '5.188e-06', 'entropy': '0.7042', 'num_tokens': '1.183e+06', 'mean_token_accuracy': '0.8252', 'epoch': '0.09582'}
{'loss': '0.6067', 'grad_norm': '0.3905', 'learning_rate': '5.179e-06', 'entropy': '0.6302', 'num_tokens': '1.187e+06', 'mean_token_accuracy': '0.8327', 'epoch': '0.09583'}
{'loss': '1.785', 'grad_norm': '0.4859', 'learning_rate': '5.171e-06', 'entropy': '1.497', 'num_tokens': '1.194e+06', 'mean_token_accuracy': '0.6822', 'epoch': '0.09583'}
{'loss': '0.7809', 'grad_norm': '0.4501', 'learning_rate': '5.162e-06', 'entropy': '0.7889', 'num_tokens': '1.199e+06', 'mean_token_accuracy': '0.7996', 'epoch': '0.09584'}
{'loss': '0.8864', 'grad_norm': '0.4221', 'learning_rate': '5.153e-06', 'entropy': '0.8799', 'num_tokens': '1.204e+06', 'mean_token_accuracy': '0.7754', 'epoch': '0.09585'}
{'loss': '0.8924', 'grad_norm': '0.421', 'learning_rate': '5.144e-06', 'entropy': '0.8453', 'num_tokens': '1.209e+06', 'mean_token_accuracy': '0.7959', 'epoch': '0.09586'}
{'loss': '0.7043', 'grad_norm': '0.3284', 'learning_rate': '5.136e-06', 'entropy': '0.7011', 'num_tokens': '1.214e+06', 'mean_token_accuracy': '0.8218', 'epoch': '0.09587'}
{'loss': '0.6832', 'grad_norm': '0.4899', 'learning_rate': '5.127e-06', 'entropy': '0.7764', 'num_tokens': '1.218e+06', 'mean_token_accuracy': '0.7844', 'epoch': '0.09588'}
{'loss': '0.574', 'grad_norm': '0.3627', 'learning_rate': '5.118e-06', 'entropy': '0.6016', 'num_tokens': '1.223e+06', 'mean_token_accuracy': '0.8413', 'epoch': '0.09589'}

 90%|█████████ | 10836/12000 [08:03<46:07,  2.38s/it]
 90%|█████████ | 10837/12000 [08:07<59:10,  3.05s/it]
                                                     

 90%|█████████ | 10837/12000 [08:07<59:10,  3.05s/it]
 90%|█████████ | 10838/12000 [08:10<54:48,  2.83s/it]
                                                     

 90%|█████████ | 10838/12000 [08:10<54:48,  2.83s/it]
 90%|█████████ | 10839/12000 [08:12<49:25,  2.55s/it]
                                                     

 90%|█████████ | 10839/12000 [08:12<49:25,  2.55s/it]
 90%|█████████ | 10840/12000 [08:15<53:44,  2.78s/it]
                                                     

 90%|█████████ | 10840/12000 [08:15<53:44,  2.78s/it]
 90%|█████████ | 10841/12000 [08:17<47:08,  2.44s/it]
                                                     

 90%|█████████ | 10841/12000 [08:17<47:08,  2.44s/it]
 90%|█████████ | 10842/12000 [08:20<53:59,  2.80s/it]
                                                     

 90%|█████████ | 10842/12000 [08:20<53:59,  2.80s/it]
 90%|█████████ | 10843/12000 [08:22<47:18,  2.45s/it]
                                                     

 90%|█████████ | 10843/12000 [08:22<47:18,  2.45s/it]
 90%|█████████ | 10844/12000 [08:24<42:43,  2.22s/it]
                                                     

 90%|█████████ | 10844/12000 [08:24<42:43,  2.22s/it]
 90%|█████████ | 10845/12000 [08:25<38:59,  2.03s/it]
                                                     

 90%|█████████ | 10845/12000 [08:25<38:59,  2.03s/it]
 90%|█████████ | 10846/12000 [08:26<34:42,  1.80s/it]
                                                     

 90%|█████████ | 10846/12000 [08:26<34:42,  1.80s/it]
 90%|█████████ | 10847/12000 [08:28<35:22,  1.84s/it]
                                                     

 90%|█████████ | 10847/12000 [08:28<35:22,  1.84s/it]
 90%|█████████ | 10848/12000 [08:30<33:58,  1.77s/it]
                                                     

 90%|█████████ | 10848/12000 [08:30<33:58,  1.77s/it]
 90%|█████████ | 10849/12000 [08:31<31:57,  1.67s/it]
                                                     

 90%|█████████ | 10849/12000 [08:31<31:57,  1.67s/it]
 90%|█████████ | 10850/12000 [08:33<31:23,  1.64s/it]
                                                     

 90%|█████████ | 10850/12000 [08:33<31:23,  1.64s/it]
 90%|█████████ | 10851/12000 [08:36<40:57,  2.14s/it]
                                                     

 90%|█████████ | 10851/12000 [08:36<40:57,  2.14s/it]
 90%|█████████ | 10852/12000 [08:40<49:02,  2.56s/it]
                                                     

 90%|█████████ | 10852/12000 [08:40<49:02,  2.56s/it]
 90%|█████████ | 10853/12000 [08:45<1:01:30,  3.22s/it]
                                                       

 90%|█████████ | 10853/12000 [08:45<1:01:30,  3.22s/it]
 90%|█████████ | 10854/12000 [08:46<52:00,  2.72s/it]  
                                                     

 90%|█████████ | 10854/12000 [08:46<52:00,  2.72s/it]
 90%|█████████ | 10855/12000 [08:48<47:07,  2.47s/it]
                                                     

 90%|█████████ | 10855/12000 [08:48<47:07,  2.47s/it]
 90%|█████████ | 10856/12000 [08:50<41:45,  2.19s/it]
                                                     

 90%|█████████ | 10856/12000 [08:50<41:45,  2.19s/it]
 90%|█████████ | 10857/12000 [08:51<37:30,  1.97s/it]
                                                     

 90%|█████████ | 10857/12000 [08:51<37:30,  1.97s/it]
 90%|█████████ | 10858/12000 [08:53<36:23,  1.91s/it]
                                                     

 90%|█████████ | 10858/12000 [08:53<36:23,  1.91s/it]
 90%|█████████ | 10859/12000 [08:54<33:21,  1.75s/it]
                                                     

 90%|█████████ | 10859/12000 [08:54<33:21,  1.75s/it]
 90%|█████████ | 10860/12000 [08:56<32:32,  1.71s/it]
                                                     

 90%|█████████ | 10860/12000 [08:56<32:32,  1.71s/it]
 91%|█████████ | 10861/12000 [08:58<32:46,  1.73s/it]
                                                     

 91%|█████████ | 10861/12000 [08:58<32:46,  1.73s/it]
 91%|█████████ | 10862/12000 [08:59<30:43,  1.62s/it]
                                                     

 91%|█████████ | 10862/12000 [08:59<30:43,  1.62s/it]
 91%|█████████ | 10863/12000 [09:02<36:33,  1.93s/it]
                                                     

 91%|█████████ | 10863/12000 [09:02<36:33,  1.93s/it]
 91%|█████████ | 10864/12000 [09:05<46:02,  2.43s/it]
                                                     

 91%|█████████ | 10864/12000 [09:05<46:02,  2.43s/it]
 91%|█████████ | 10865/12000 [09:07<44:32,  2.35s/it]
                                                     

 91%|█████████ | 10865/12000 [09:07<44:32,  2.35s/it]
 91%|█████████ | 10866/12000 [09:09<41:32,  2.20s/it]
                                                     

 91%|█████████ | 10866/12000 [09:09<41:32,  2.20s/it]
 91%|█████████ | 10867/12000 [09:14<53:45,  2.85s/it]
                                                     

 91%|█████████ | 10867/12000 [09:14<53:45,  2.85s/it]
 91%|█████████ | 10868/12000 [09:16<49:25,  2.62s/it]
                                                     

 91%|█████████ | 10868/12000 [09:16<49:25,  2.62s/it]
 91%|█████████ | 10869/12000 [09:17<43:12,  2.29s/it]
                                                     

 91%|█████████ | 10869/12000 [09:17<43:12,  2.29s/it]
 91%|█████████ | 10870/12000 [09:19<38:23,  2.04s/it]
                                                     

 91%|█████████ | 10870/12000 [09:19<38:23,  2.04s/it]
 91%|█████████ | 10871/12000 [09:20<36:02,  1.92s/it]
                                                     

 91%|█████████ | 10871/12000 [09:20<36:02,  1.92s/it]
 91%|█████████ | 10872/12000 [09:22<34:46,  1.85s/it]
                                                     

 91%|█████████ | 10872/12000 [09:22<34:46,  1.85s/it]
 91%|█████████ | 10873/12000 [09:24<37:29,  2.00s/it]
                                                     

 91%|█████████ | 10873/12000 [09:24<37:29,  2.00s/it]
 91%|█████████ | 10874/12000 [09:26<37:41,  2.01s/it]
                                                     

 91%|█████████ | 10874/12000 [09:26<37:41,  2.01s/it]
 91%|█████████ | 10875/12000 [09:28<37:55,  2.02s/it]
                                                     

 91%|█████████ | 10875/12000 [09:28<37:55,  2.02s/it]
 91%|█████████ | 10876/12000 [09:32<44:11,  2.36s/it]
                                                     

 91%|█████████ | 10876/12000 [09:32<44:11,  2.36s/it]
 91%|█████████ | 10877/12000 [09:33<41:20,  2.21s/it]
                                                     

 91%|█████████ | 10877/12000 [09:33<41:20,  2.21s/it]
 91%|█████████ | 10878/12000 [09:35<37:00,  1.98s/it]
                                                     

 91%|█████████ | 10878/12000 [09:35<37:00,  1.98s/it]
 91%|█████████ | 10879/12000 [09:38<43:41,  2.34s/it]
                                                     

 91%|█████████ | 10879/12000 [09:38<43:41,  2.34s/it]
 91%|█████████ | 10880/12000 [09:39<38:48,  2.08s/it]
                                                     

 91%|█████████ | 10880/12000 [09:39<38:48,  2.08s/it]
 91%|█████████ | 10881/12000 [09:43<45:04,  2.42s/it]
                                                     

 91%|█████████ | 10881/12000 [09:43<45:04,  2.42s/it]
 91%|█████████ | 10882/12000 [09:45<45:03,  2.42s/it]
                                                     

 91%|█████████ | 10882/12000 [09:45<45:03,  2.42s/it]
 91%|█████████ | 10883/12000 [09:47<41:20,  2.22s/it]
                                                     
{'loss': '0.8114', 'grad_norm': '0.3426', 'learning_rate': '5.109e-06', 'entropy': '0.7748', 'num_tokens': '1.231e+06', 'mean_token_accuracy': '0.8028', 'epoch': '0.0959'}
{'loss': '0.7247', 'grad_norm': '0.3', 'learning_rate': '5.101e-06', 'entropy': '0.9405', 'num_tokens': '1.239e+06', 'mean_token_accuracy': '0.7756', 'epoch': '0.09591'}
{'loss': '0.8909', 'grad_norm': '0.493', 'learning_rate': '5.092e-06', 'entropy': '0.793', 'num_tokens': '1.245e+06', 'mean_token_accuracy': '0.802', 'epoch': '0.09591'}
{'loss': '0.843', 'grad_norm': '0.3717', 'learning_rate': '5.083e-06', 'entropy': '0.842', 'num_tokens': '1.251e+06', 'mean_token_accuracy': '0.7906', 'epoch': '0.09592'}
{'loss': '0.6181', 'grad_norm': '0.3389', 'learning_rate': '5.075e-06', 'entropy': '0.6656', 'num_tokens': '1.257e+06', 'mean_token_accuracy': '0.8284', 'epoch': '0.09593'}
{'loss': '0.7546', 'grad_norm': '0.3841', 'learning_rate': '5.066e-06', 'entropy': '0.8198', 'num_tokens': '1.263e+06', 'mean_token_accuracy': '0.7882', 'epoch': '0.09594'}
{'loss': '0.8449', 'grad_norm': '0.4029', 'learning_rate': '5.057e-06', 'entropy': '0.7501', 'num_tokens': '1.268e+06', 'mean_token_accuracy': '0.8038', 'epoch': '0.09595'}
{'loss': '0.475', 'grad_norm': '0.3794', 'learning_rate': '5.049e-06', 'entropy': '0.5215', 'num_tokens': '1.272e+06', 'mean_token_accuracy': '0.8602', 'epoch': '0.09596'}
{'loss': '0.7936', 'grad_norm': '0.3991', 'learning_rate': '5.04e-06', 'entropy': '0.8247', 'num_tokens': '1.278e+06', 'mean_token_accuracy': '0.7956', 'epoch': '0.09597'}
{'loss': '1.252', 'grad_norm': '0.4205', 'learning_rate': '5.032e-06', 'entropy': '1.202', 'num_tokens': '1.283e+06', 'mean_token_accuracy': '0.7166', 'epoch': '0.09598'}
{'loss': '1.195', 'grad_norm': '0.5264', 'learning_rate': '5.023e-06', 'entropy': '1.356', 'num_tokens': '1.286e+06', 'mean_token_accuracy': '0.6812', 'epoch': '0.09599'}
{'loss': '1', 'grad_norm': '0.4818', 'learning_rate': '5.014e-06', 'entropy': '1.078', 'num_tokens': '1.291e+06', 'mean_token_accuracy': '0.7273', 'epoch': '0.09599'}
{'loss': '0.549', 'grad_norm': '0.3496', 'learning_rate': '5.006e-06', 'entropy': '0.5718', 'num_tokens': '1.296e+06', 'mean_token_accuracy': '0.8554', 'epoch': '0.096'}
{'loss': '0.7024', 'grad_norm': '0.4294', 'learning_rate': '4.997e-06', 'entropy': '0.6826', 'num_tokens': '1.299e+06', 'mean_token_accuracy': '0.8248', 'epoch': '0.09601'}
{'loss': '0.6395', 'grad_norm': '0.4498', 'learning_rate': '4.988e-06', 'entropy': '0.6527', 'num_tokens': '1.304e+06', 'mean_token_accuracy': '0.827', 'epoch': '0.09602'}
{'loss': '0.7619', 'grad_norm': '0.3695', 'learning_rate': '4.98e-06', 'entropy': '0.7611', 'num_tokens': '1.309e+06', 'mean_token_accuracy': '0.799', 'epoch': '0.09603'}
{'loss': '1.128', 'grad_norm': '0.3987', 'learning_rate': '4.971e-06', 'entropy': '1.07', 'num_tokens': '1.318e+06', 'mean_token_accuracy': '0.7468', 'epoch': '0.09604'}
{'loss': '0.8359', 'grad_norm': '0.3735', 'learning_rate': '4.963e-06', 'entropy': '0.7226', 'num_tokens': '1.326e+06', 'mean_token_accuracy': '0.8122', 'epoch': '0.09605'}
{'loss': '1.034', 'grad_norm': '0.3662', 'learning_rate': '4.954e-06', 'entropy': '0.9978', 'num_tokens': '1.332e+06', 'mean_token_accuracy': '0.7615', 'epoch': '0.09606'}
{'loss': '0.8066', 'grad_norm': '0.5022', 'learning_rate': '4.946e-06', 'entropy': '0.8006', 'num_tokens': '1.337e+06', 'mean_token_accuracy': '0.7961', 'epoch': '0.09606'}
{'loss': '0.6492', 'grad_norm': '0.4122', 'learning_rate': '4.937e-06', 'entropy': '0.7286', 'num_tokens': '1.341e+06', 'mean_token_accuracy': '0.8062', 'epoch': '0.09607'}
{'loss': '0.6589', 'grad_norm': '0.3998', 'learning_rate': '4.928e-06', 'entropy': '0.7341', 'num_tokens': '1.345e+06', 'mean_token_accuracy': '0.8112', 'epoch': '0.09608'}
{'loss': '0.6871', 'grad_norm': '0.348', 'learning_rate': '4.92e-06', 'entropy': '0.7585', 'num_tokens': '1.35e+06', 'mean_token_accuracy': '0.8109', 'epoch': '0.09609'}
{'loss': '0.7532', 'grad_norm': '0.3653', 'learning_rate': '4.911e-06', 'entropy': '0.7532', 'num_tokens': '1.354e+06', 'mean_token_accuracy': '0.8053', 'epoch': '0.0961'}
{'loss': '0.6963', 'grad_norm': '0.4252', 'learning_rate': '4.903e-06', 'entropy': '0.7312', 'num_tokens': '1.359e+06', 'mean_token_accuracy': '0.8096', 'epoch': '0.09611'}
{'loss': '0.4687', 'grad_norm': '0.3755', 'learning_rate': '4.894e-06', 'entropy': '0.4484', 'num_tokens': '1.364e+06', 'mean_token_accuracy': '0.8753', 'epoch': '0.09612'}
{'loss': '0.8556', 'grad_norm': '0.4117', 'learning_rate': '4.886e-06', 'entropy': '0.9398', 'num_tokens': '1.368e+06', 'mean_token_accuracy': '0.7728', 'epoch': '0.09613'}
{'loss': '0.7195', 'grad_norm': '0.3487', 'learning_rate': '4.877e-06', 'entropy': '0.7201', 'num_tokens': '1.375e+06', 'mean_token_accuracy': '0.8209', 'epoch': '0.09614'}
{'loss': '0.7384', 'grad_norm': '0.4166', 'learning_rate': '4.869e-06', 'entropy': '0.7829', 'num_tokens': '1.38e+06', 'mean_token_accuracy': '0.799', 'epoch': '0.09614'}
{'loss': '0.8351', 'grad_norm': '0.402', 'learning_rate': '4.86e-06', 'entropy': '0.847', 'num_tokens': '1.385e+06', 'mean_token_accuracy': '0.7718', 'epoch': '0.09615'}
{'loss': '0.6552', 'grad_norm': '0.4178', 'learning_rate': '4.852e-06', 'entropy': '0.6219', 'num_tokens': '1.391e+06', 'mean_token_accuracy': '0.8344', 'epoch': '0.09616'}
{'loss': '1.164', 'grad_norm': '0.4477', 'learning_rate': '4.843e-06', 'entropy': '1.132', 'num_tokens': '1.399e+06', 'mean_token_accuracy': '0.7305', 'epoch': '0.09617'}
{'loss': '1.42', 'grad_norm': '0.4621', 'learning_rate': '4.835e-06', 'entropy': '1.222', 'num_tokens': '1.405e+06', 'mean_token_accuracy': '0.7224', 'epoch': '0.09618'}
{'loss': '0.6902', 'grad_norm': '0.3911', 'learning_rate': '4.826e-06', 'entropy': '0.6571', 'num_tokens': '1.41e+06', 'mean_token_accuracy': '0.8215', 'epoch': '0.09619'}
{'loss': '0.7837', 'grad_norm': '0.4486', 'learning_rate': '4.818e-06', 'entropy': '0.8673', 'num_tokens': '1.413e+06', 'mean_token_accuracy': '0.7816', 'epoch': '0.0962'}
{'loss': '0.9635', 'grad_norm': '0.5297', 'learning_rate': '4.81e-06', 'entropy': '0.8931', 'num_tokens': '1.416e+06', 'mean_token_accuracy': '0.7776', 'epoch': '0.09621'}
{'loss': '0.4609', 'grad_norm': '0.3672', 'learning_rate': '4.801e-06', 'entropy': '0.4857', 'num_tokens': '1.421e+06', 'mean_token_accuracy': '0.8676', 'epoch': '0.09622'}
{'loss': '0.8221', 'grad_norm': '0.3536', 'learning_rate': '4.793e-06', 'entropy': '0.8206', 'num_tokens': '1.427e+06', 'mean_token_accuracy': '0.8008', 'epoch': '0.09622'}
{'loss': '0.9343', 'grad_norm': '0.4057', 'learning_rate': '4.784e-06', 'entropy': '0.9126', 'num_tokens': '1.433e+06', 'mean_token_accuracy': '0.764', 'epoch': '0.09623'}
{'loss': '0.9769', 'grad_norm': '0.5849', 'learning_rate': '4.776e-06', 'entropy': '0.9013', 'num_tokens': '1.438e+06', 'mean_token_accuracy': '0.7722', 'epoch': '0.09624'}
{'loss': '0.9013', 'grad_norm': '0.3961', 'learning_rate': '4.767e-06', 'entropy': '0.8848', 'num_tokens': '1.446e+06', 'mean_token_accuracy': '0.7725', 'epoch': '0.09625'}
{'loss': '0.8324', 'grad_norm': '0.3848', 'learning_rate': '4.759e-06', 'entropy': '0.8311', 'num_tokens': '1.451e+06', 'mean_token_accuracy': '0.7862', 'epoch': '0.09626'}
{'loss': '1.16', 'grad_norm': '0.5063', 'learning_rate': '4.751e-06', 'entropy': '1.114', 'num_tokens': '1.455e+06', 'mean_token_accuracy': '0.7247', 'epoch': '0.09627'}
{'loss': '0.7293', 'grad_norm': '0.3681', 'learning_rate': '4.742e-06', 'entropy': '0.6968', 'num_tokens': '1.464e+06', 'mean_token_accuracy': '0.8187', 'epoch': '0.09628'}
{'loss': '1.035', 'grad_norm': '0.4222', 'learning_rate': '4.734e-06', 'entropy': '0.9902', 'num_tokens': '1.468e+06', 'mean_token_accuracy': '0.7526', 'epoch': '0.09629'}
{'loss': '1.089', 'grad_norm': '0.3685', 'learning_rate': '4.725e-06', 'entropy': '0.9693', 'num_tokens': '1.475e+06', 'mean_token_accuracy': '0.7619', 'epoch': '0.0963'}
{'loss': '0.6841', 'grad_norm': '0.3429', 'learning_rate': '4.717e-06', 'entropy': '0.7582', 'num_tokens': '1.481e+06', 'mean_token_accuracy': '0.8048', 'epoch': '0.0963'}

 91%|█████████ | 10883/12000 [09:47<41:20,  2.22s/it]
 91%|█████████ | 10884/12000 [09:49<41:31,  2.23s/it]
                                                     

 91%|█████████ | 10884/12000 [09:49<41:31,  2.23s/it]
 91%|█████████ | 10885/12000 [09:51<37:09,  2.00s/it]
                                                     

 91%|█████████ | 10885/12000 [09:51<37:09,  2.00s/it]
 91%|█████████ | 10886/12000 [09:54<45:49,  2.47s/it]
                                                     

 91%|█████████ | 10886/12000 [09:54<45:49,  2.47s/it]
 91%|█████████ | 10887/12000 [09:56<40:52,  2.20s/it]
                                                     

 91%|█████████ | 10887/12000 [09:56<40:52,  2.20s/it]
 91%|█████████ | 10888/12000 [09:59<44:31,  2.40s/it]
                                                     

 91%|█████████ | 10888/12000 [09:59<44:31,  2.40s/it]
 91%|█████████ | 10889/12000 [10:02<52:41,  2.85s/it]
                                                     

 91%|█████████ | 10889/12000 [10:02<52:41,  2.85s/it]
 91%|█████████ | 10890/12000 [10:04<45:16,  2.45s/it]
                                                     

 91%|█████████ | 10890/12000 [10:04<45:16,  2.45s/it]
 91%|█████████ | 10891/12000 [10:06<40:56,  2.22s/it]
                                                     

 91%|█████████ | 10891/12000 [10:06<40:56,  2.22s/it]
 91%|█████████ | 10892/12000 [10:07<37:10,  2.01s/it]
                                                     

 91%|█████████ | 10892/12000 [10:07<37:10,  2.01s/it]
 91%|█████████ | 10893/12000 [10:09<35:09,  1.91s/it]
                                                     

 91%|█████████ | 10893/12000 [10:09<35:09,  1.91s/it]
 91%|█████████ | 10894/12000 [10:10<33:15,  1.80s/it]
                                                     

 91%|█████████ | 10894/12000 [10:10<33:15,  1.80s/it]
 91%|█████████ | 10895/12000 [10:12<32:58,  1.79s/it]
                                                     

 91%|█████████ | 10895/12000 [10:12<32:58,  1.79s/it]
 91%|█████████ | 10896/12000 [10:14<30:47,  1.67s/it]
                                                     

 91%|█████████ | 10896/12000 [10:14<30:47,  1.67s/it]
 91%|█████████ | 10897/12000 [10:18<47:05,  2.56s/it]
                                                     

 91%|█████████ | 10897/12000 [10:18<47:05,  2.56s/it]
 91%|█████████ | 10898/12000 [10:22<53:42,  2.92s/it]
                                                     

 91%|█████████ | 10898/12000 [10:22<53:42,  2.92s/it]
 91%|█████████ | 10899/12000 [10:24<48:40,  2.65s/it]
                                                     

 91%|█████████ | 10899/12000 [10:24<48:40,  2.65s/it]
 91%|█████████ | 10900/12000 [10:26<44:20,  2.42s/it]
                                                     

 91%|█████████ | 10900/12000 [10:26<44:20,  2.42s/it]
 91%|█████████ | 10901/12000 [10:29<48:18,  2.64s/it]
                                                     

 91%|█████████ | 10901/12000 [10:29<48:18,  2.64s/it]
 91%|█████████ | 10902/12000 [10:31<42:43,  2.33s/it]
                                                     

 91%|█████████ | 10902/12000 [10:31<42:43,  2.33s/it]
 91%|█████████ | 10903/12000 [10:32<37:45,  2.06s/it]
                                                     

 91%|█████████ | 10903/12000 [10:32<37:45,  2.06s/it]
 91%|█████████ | 10904/12000 [10:34<35:32,  1.95s/it]
                                                     

 91%|█████████ | 10904/12000 [10:34<35:32,  1.95s/it]
 91%|█████████ | 10905/12000 [10:36<36:59,  2.03s/it]
                                                     

 91%|█████████ | 10905/12000 [10:36<36:59,  2.03s/it]
 91%|█████████ | 10906/12000 [10:38<34:23,  1.89s/it]
                                                     

 91%|█████████ | 10906/12000 [10:38<34:23,  1.89s/it]
 91%|█████████ | 10907/12000 [10:39<31:29,  1.73s/it]
                                                     

 91%|█████████ | 10907/12000 [10:39<31:29,  1.73s/it]
 91%|█████████ | 10908/12000 [10:46<59:02,  3.24s/it]
                                                     

 91%|█████████ | 10908/12000 [10:46<59:02,  3.24s/it]
 91%|█████████ | 10909/12000 [10:49<59:26,  3.27s/it]
                                                     

 91%|█████████ | 10909/12000 [10:49<59:26,  3.27s/it]
 91%|█████████ | 10910/12000 [10:51<50:13,  2.76s/it]
                                                     

 91%|█████████ | 10910/12000 [10:51<50:13,  2.76s/it]
 91%|█████████ | 10911/12000 [10:52<44:33,  2.45s/it]
                                                     

 91%|█████████ | 10911/12000 [10:52<44:33,  2.45s/it]
 91%|█████████ | 10912/12000 [10:56<48:55,  2.70s/it]
                                                     

 91%|█████████ | 10912/12000 [10:56<48:55,  2.70s/it]
 91%|█████████ | 10913/12000 [10:57<41:30,  2.29s/it]
                                                     

 91%|█████████ | 10913/12000 [10:57<41:30,  2.29s/it]
 91%|█████████ | 10914/12000 [10:59<37:38,  2.08s/it]
                                                     

 91%|█████████ | 10914/12000 [10:59<37:38,  2.08s/it]
 91%|█████████ | 10915/12000 [11:01<40:41,  2.25s/it]
                                                     

 91%|█████████ | 10915/12000 [11:01<40:41,  2.25s/it]
 91%|█████████ | 10916/12000 [11:03<37:13,  2.06s/it]
                                                     

 91%|█████████ | 10916/12000 [11:03<37:13,  2.06s/it]
 91%|█████████ | 10917/12000 [11:05<37:50,  2.10s/it]
                                                     

 91%|█████████ | 10917/12000 [11:05<37:50,  2.10s/it]
 91%|█████████ | 10918/12000 [11:07<34:49,  1.93s/it]
                                                     

 91%|█████████ | 10918/12000 [11:07<34:49,  1.93s/it]
 91%|█████████ | 10919/12000 [11:08<32:47,  1.82s/it]
                                                     

 91%|█████████ | 10919/12000 [11:08<32:47,  1.82s/it]
 91%|█████████ | 10920/12000 [11:10<32:50,  1.82s/it]
                                                     

 91%|█████████ | 10920/12000 [11:10<32:50,  1.82s/it]
 91%|█████████ | 10921/12000 [11:12<36:02,  2.00s/it]
                                                     

 91%|█████████ | 10921/12000 [11:12<36:02,  2.00s/it]
 91%|█████████ | 10922/12000 [11:14<34:46,  1.94s/it]
                                                     

 91%|█████████ | 10922/12000 [11:14<34:46,  1.94s/it]
 91%|█████████ | 10923/12000 [11:16<32:59,  1.84s/it]
                                                     

 91%|█████████ | 10923/12000 [11:16<32:59,  1.84s/it]
 91%|█████████ | 10924/12000 [11:17<32:14,  1.80s/it]
                                                     

 91%|█████████ | 10924/12000 [11:17<32:14,  1.80s/it]
 91%|█████████ | 10925/12000 [11:19<33:38,  1.88s/it]
                                                     

 91%|█████████ | 10925/12000 [11:19<33:38,  1.88s/it]
 91%|█████████ | 10926/12000 [11:21<31:46,  1.78s/it]
                                                     

 91%|█████████ | 10926/12000 [11:21<31:46,  1.78s/it]
 91%|█████████ | 10927/12000 [11:22<30:01,  1.68s/it]
                                                     

 91%|█████████ | 10927/12000 [11:22<30:01,  1.68s/it]
 91%|█████████ | 10928/12000 [11:24<30:41,  1.72s/it]
                                                     

 91%|█████████ | 10928/12000 [11:24<30:41,  1.72s/it]
 91%|█████████ | 10929/12000 [11:27<36:27,  2.04s/it]
                                                     

 91%|█████████ | 10929/12000 [11:27<36:27,  2.04s/it]
 91%|█████████ | 10930/12000 [11:31<47:10,  2.65s/it]
                                                     
{'loss': '0.904', 'grad_norm': '0.3563', 'learning_rate': '4.709e-06', 'entropy': '0.8656', 'num_tokens': '1.486e+06', 'mean_token_accuracy': '0.7831', 'epoch': '0.09631'}
{'loss': '1.236', 'grad_norm': '0.5411', 'learning_rate': '4.7e-06', 'entropy': '1.1', 'num_tokens': '1.491e+06', 'mean_token_accuracy': '0.7285', 'epoch': '0.09632'}
{'loss': '0.7443', 'grad_norm': '0.4385', 'learning_rate': '4.692e-06', 'entropy': '0.7882', 'num_tokens': '1.496e+06', 'mean_token_accuracy': '0.8018', 'epoch': '0.09633'}
{'loss': '0.5036', 'grad_norm': '0.2966', 'learning_rate': '4.684e-06', 'entropy': '0.5219', 'num_tokens': '1.5e+06', 'mean_token_accuracy': '0.8666', 'epoch': '0.09634'}
{'loss': '0.5913', 'grad_norm': '0.4429', 'learning_rate': '4.675e-06', 'entropy': '0.6287', 'num_tokens': '1.504e+06', 'mean_token_accuracy': '0.845', 'epoch': '0.09635'}
{'loss': '0.7098', 'grad_norm': '0.3689', 'learning_rate': '4.667e-06', 'entropy': '0.8601', 'num_tokens': '1.51e+06', 'mean_token_accuracy': '0.7915', 'epoch': '0.09636'}
{'loss': '0.7604', 'grad_norm': '0.3273', 'learning_rate': '4.659e-06', 'entropy': '0.7092', 'num_tokens': '1.521e+06', 'mean_token_accuracy': '0.8229', 'epoch': '0.09637'}
{'loss': '0.6537', 'grad_norm': '0.4874', 'learning_rate': '4.65e-06', 'entropy': '0.6767', 'num_tokens': '1.525e+06', 'mean_token_accuracy': '0.8396', 'epoch': '0.09637'}
{'loss': '0.7893', 'grad_norm': '0.4373', 'learning_rate': '4.642e-06', 'entropy': '0.8399', 'num_tokens': '1.53e+06', 'mean_token_accuracy': '0.7809', 'epoch': '0.09638'}
{'loss': '0.7155', 'grad_norm': '0.4745', 'learning_rate': '4.634e-06', 'entropy': '0.7354', 'num_tokens': '1.534e+06', 'mean_token_accuracy': '0.8', 'epoch': '0.09639'}
{'loss': '0.621', 'grad_norm': '0.3222', 'learning_rate': '4.626e-06', 'entropy': '0.6471', 'num_tokens': '1.539e+06', 'mean_token_accuracy': '0.8359', 'epoch': '0.0964'}
{'loss': '0.5249', 'grad_norm': '0.2855', 'learning_rate': '4.617e-06', 'entropy': '0.5789', 'num_tokens': '1.545e+06', 'mean_token_accuracy': '0.8613', 'epoch': '0.09641'}
{'loss': '0.7594', 'grad_norm': '0.4378', 'learning_rate': '4.609e-06', 'entropy': '0.8369', 'num_tokens': '1.551e+06', 'mean_token_accuracy': '0.7808', 'epoch': '0.09642'}
{'loss': '0.7164', 'grad_norm': '0.3801', 'learning_rate': '4.601e-06', 'entropy': '0.7009', 'num_tokens': '1.554e+06', 'mean_token_accuracy': '0.8244', 'epoch': '0.09643'}
{'loss': '0.8375', 'grad_norm': '0.365', 'learning_rate': '4.592e-06', 'entropy': '0.7761', 'num_tokens': '1.562e+06', 'mean_token_accuracy': '0.8004', 'epoch': '0.09644'}
{'loss': '0.8822', 'grad_norm': '0.468', 'learning_rate': '4.584e-06', 'entropy': '0.7969', 'num_tokens': '1.571e+06', 'mean_token_accuracy': '0.7949', 'epoch': '0.09645'}
{'loss': '0.7783', 'grad_norm': '0.4037', 'learning_rate': '4.576e-06', 'entropy': '0.8274', 'num_tokens': '1.576e+06', 'mean_token_accuracy': '0.7939', 'epoch': '0.09645'}
{'loss': '1.283', 'grad_norm': '0.4091', 'learning_rate': '4.568e-06', 'entropy': '1.252', 'num_tokens': '1.582e+06', 'mean_token_accuracy': '0.7108', 'epoch': '0.09646'}
{'loss': '0.6881', 'grad_norm': '0.4849', 'learning_rate': '4.559e-06', 'entropy': '0.7017', 'num_tokens': '1.586e+06', 'mean_token_accuracy': '0.8156', 'epoch': '0.09647'}
{'loss': '0.588', 'grad_norm': '0.4437', 'learning_rate': '4.551e-06', 'entropy': '0.5846', 'num_tokens': '1.59e+06', 'mean_token_accuracy': '0.8488', 'epoch': '0.09648'}
{'loss': '0.8075', 'grad_norm': '0.3186', 'learning_rate': '4.543e-06', 'entropy': '0.874', 'num_tokens': '1.594e+06', 'mean_token_accuracy': '0.7758', 'epoch': '0.09649'}
{'loss': '0.9801', 'grad_norm': '0.5333', 'learning_rate': '4.535e-06', 'entropy': '0.9781', 'num_tokens': '1.598e+06', 'mean_token_accuracy': '0.7598', 'epoch': '0.0965'}
{'loss': '0.9345', 'grad_norm': '0.5218', 'learning_rate': '4.527e-06', 'entropy': '0.9825', 'num_tokens': '1.603e+06', 'mean_token_accuracy': '0.7673', 'epoch': '0.09651'}
{'loss': '0.5423', 'grad_norm': '0.3603', 'learning_rate': '4.518e-06', 'entropy': '0.5551', 'num_tokens': '1.608e+06', 'mean_token_accuracy': '0.8444', 'epoch': '0.09652'}
{'loss': '0.672', 'grad_norm': '0.3372', 'learning_rate': '4.51e-06', 'entropy': '0.8002', 'num_tokens': '1.612e+06', 'mean_token_accuracy': '0.7947', 'epoch': '0.09653'}
{'loss': '0.7049', 'grad_norm': '0.3016', 'learning_rate': '4.502e-06', 'entropy': '0.7896', 'num_tokens': '1.621e+06', 'mean_token_accuracy': '0.8065', 'epoch': '0.09653'}
{'loss': '1.085', 'grad_norm': '0.3836', 'learning_rate': '4.494e-06', 'entropy': '0.9227', 'num_tokens': '1.629e+06', 'mean_token_accuracy': '0.7816', 'epoch': '0.09654'}
{'loss': '0.7955', 'grad_norm': '0.3488', 'learning_rate': '4.486e-06', 'entropy': '0.8837', 'num_tokens': '1.634e+06', 'mean_token_accuracy': '0.7858', 'epoch': '0.09655'}
{'loss': '0.5965', 'grad_norm': '0.4101', 'learning_rate': '4.478e-06', 'entropy': '0.5965', 'num_tokens': '1.639e+06', 'mean_token_accuracy': '0.8417', 'epoch': '0.09656'}
{'loss': '0.8431', 'grad_norm': '0.4265', 'learning_rate': '4.469e-06', 'entropy': '0.7629', 'num_tokens': '1.646e+06', 'mean_token_accuracy': '0.8104', 'epoch': '0.09657'}
{'loss': '0.4861', 'grad_norm': '0.3481', 'learning_rate': '4.461e-06', 'entropy': '0.4827', 'num_tokens': '1.65e+06', 'mean_token_accuracy': '0.8648', 'epoch': '0.09658'}
{'loss': '1.205', 'grad_norm': '0.5271', 'learning_rate': '4.453e-06', 'entropy': '1.17', 'num_tokens': '1.654e+06', 'mean_token_accuracy': '0.697', 'epoch': '0.09659'}
{'loss': '0.5734', 'grad_norm': '0.3623', 'learning_rate': '4.445e-06', 'entropy': '0.6391', 'num_tokens': '1.66e+06', 'mean_token_accuracy': '0.8244', 'epoch': '0.0966'}
{'loss': '0.5563', 'grad_norm': '0.3765', 'learning_rate': '4.437e-06', 'entropy': '0.5497', 'num_tokens': '1.664e+06', 'mean_token_accuracy': '0.8571', 'epoch': '0.0966'}
{'loss': '0.8446', 'grad_norm': '0.423', 'learning_rate': '4.429e-06', 'entropy': '0.8832', 'num_tokens': '1.671e+06', 'mean_token_accuracy': '0.789', 'epoch': '0.09661'}
{'loss': '0.901', 'grad_norm': '0.4714', 'learning_rate': '4.421e-06', 'entropy': '0.9321', 'num_tokens': '1.675e+06', 'mean_token_accuracy': '0.7739', 'epoch': '0.09662'}
{'loss': '1.049', 'grad_norm': '0.3951', 'learning_rate': '4.413e-06', 'entropy': '0.962', 'num_tokens': '1.68e+06', 'mean_token_accuracy': '0.7629', 'epoch': '0.09663'}
{'loss': '0.632', 'grad_norm': '0.4488', 'learning_rate': '4.405e-06', 'entropy': '0.6261', 'num_tokens': '1.685e+06', 'mean_token_accuracy': '0.8331', 'epoch': '0.09664'}
{'loss': '1.029', 'grad_norm': '0.4105', 'learning_rate': '4.396e-06', 'entropy': '0.9578', 'num_tokens': '1.692e+06', 'mean_token_accuracy': '0.7556', 'epoch': '0.09665'}
{'loss': '0.957', 'grad_norm': '0.4617', 'learning_rate': '4.388e-06', 'entropy': '0.8926', 'num_tokens': '1.698e+06', 'mean_token_accuracy': '0.7772', 'epoch': '0.09666'}
{'loss': '0.8465', 'grad_norm': '0.3868', 'learning_rate': '4.38e-06', 'entropy': '0.8131', 'num_tokens': '1.703e+06', 'mean_token_accuracy': '0.7894', 'epoch': '0.09667'}
{'loss': '1.105', 'grad_norm': '0.4688', 'learning_rate': '4.372e-06', 'entropy': '1.045', 'num_tokens': '1.708e+06', 'mean_token_accuracy': '0.7452', 'epoch': '0.09668'}
{'loss': '0.7528', 'grad_norm': '0.4827', 'learning_rate': '4.364e-06', 'entropy': '0.7244', 'num_tokens': '1.714e+06', 'mean_token_accuracy': '0.8131', 'epoch': '0.09668'}
{'loss': '0.5098', 'grad_norm': '0.3555', 'learning_rate': '4.356e-06', 'entropy': '0.5354', 'num_tokens': '1.718e+06', 'mean_token_accuracy': '0.8571', 'epoch': '0.09669'}
{'loss': '0.8065', 'grad_norm': '0.4372', 'learning_rate': '4.348e-06', 'entropy': '0.7845', 'num_tokens': '1.722e+06', 'mean_token_accuracy': '0.8079', 'epoch': '0.0967'}
{'loss': '0.6461', 'grad_norm': '0.4499', 'learning_rate': '4.34e-06', 'entropy': '0.6559', 'num_tokens': '1.729e+06', 'mean_token_accuracy': '0.8344', 'epoch': '0.09671'}
{'loss': '0.7284', 'grad_norm': '0.3663', 'learning_rate': '4.332e-06', 'entropy': '0.789', 'num_tokens': '1.735e+06', 'mean_token_accuracy': '0.8073', 'epoch': '0.09672'}

 91%|█████████ | 10930/12000 [11:31<47:10,  2.65s/it]
 91%|█████████ | 10931/12000 [11:33<41:01,  2.30s/it]
                                                     

 91%|█████████ | 10931/12000 [11:33<41:01,  2.30s/it]
 91%|█████████ | 10932/12000 [11:34<36:34,  2.06s/it]
                                                     

 91%|█████████ | 10932/12000 [11:34<36:34,  2.06s/it]
 91%|█████████ | 10933/12000 [11:36<34:31,  1.94s/it]
                                                     

 91%|█████████ | 10933/12000 [11:36<34:31,  1.94s/it]
 91%|█████████ | 10934/12000 [11:37<32:52,  1.85s/it]
                                                     

 91%|█████████ | 10934/12000 [11:37<32:52,  1.85s/it]
 91%|█████████ | 10935/12000 [11:39<32:24,  1.83s/it]
                                                     

 91%|█████████ | 10935/12000 [11:39<32:24,  1.83s/it]
 91%|█████████ | 10936/12000 [11:41<31:15,  1.76s/it]
                                                     

 91%|█████████ | 10936/12000 [11:41<31:15,  1.76s/it]
 91%|█████████ | 10937/12000 [11:44<41:10,  2.32s/it]
                                                     

 91%|█████████ | 10937/12000 [11:44<41:10,  2.32s/it]
 91%|█████████ | 10938/12000 [11:46<36:22,  2.06s/it]
                                                     

 91%|█████████ | 10938/12000 [11:46<36:22,  2.06s/it]
 91%|█████████ | 10939/12000 [11:48<34:20,  1.94s/it]
                                                     

 91%|█████████ | 10939/12000 [11:48<34:20,  1.94s/it]
 91%|█████████ | 10940/12000 [11:50<37:48,  2.14s/it]
                                                     

 91%|█████████ | 10940/12000 [11:50<37:48,  2.14s/it]
 91%|█████████ | 10941/12000 [11:53<39:47,  2.25s/it]
                                                     

 91%|█████████ | 10941/12000 [11:53<39:47,  2.25s/it]
 91%|█████████ | 10942/12000 [11:56<42:53,  2.43s/it]
                                                     

 91%|█████████ | 10942/12000 [11:56<42:53,  2.43s/it]
 91%|█████████ | 10943/12000 [11:57<37:48,  2.15s/it]
                                                     

 91%|█████████ | 10943/12000 [11:57<37:48,  2.15s/it]
 91%|█████████ | 10944/12000 [12:00<40:46,  2.32s/it]
                                                     

 91%|█████████ | 10944/12000 [12:00<40:46,  2.32s/it]
 91%|█████████ | 10945/12000 [12:07<1:09:01,  3.93s/it]
                                                       

 91%|█████████ | 10945/12000 [12:07<1:09:01,  3.93s/it]
 91%|█████████ | 10946/12000 [12:09<57:07,  3.25s/it]  
                                                     

 91%|█████████ | 10946/12000 [12:09<57:07,  3.25s/it]
 91%|█████████ | 10947/12000 [12:11<48:15,  2.75s/it]
                                                     

 91%|█████████ | 10947/12000 [12:11<48:15,  2.75s/it]
 91%|█████████ | 10948/12000 [12:12<42:27,  2.42s/it]
                                                     

 91%|█████████ | 10948/12000 [12:12<42:27,  2.42s/it]
 91%|█████████ | 10949/12000 [12:14<40:21,  2.30s/it]
                                                     

 91%|█████████ | 10949/12000 [12:14<40:21,  2.30s/it]
 91%|█████████▏| 10950/12000 [12:17<39:49,  2.28s/it]
                                                     

 91%|█████████▏| 10950/12000 [12:17<39:49,  2.28s/it]
 91%|█████████▏| 10951/12000 [12:20<46:08,  2.64s/it]
                                                     

 91%|█████████▏| 10951/12000 [12:20<46:08,  2.64s/it]
 91%|█████████▏| 10952/12000 [12:24<53:06,  3.04s/it]
                                                     

 91%|█████████▏| 10952/12000 [12:24<53:06,  3.04s/it]
 91%|█████████▏| 10953/12000 [12:26<49:49,  2.86s/it]
                                                     

 91%|█████████▏| 10953/12000 [12:26<49:49,  2.86s/it]
 91%|█████████▏| 10954/12000 [12:29<46:52,  2.69s/it]
                                                     

 91%|█████████▏| 10954/12000 [12:29<46:52,  2.69s/it]
 91%|█████████▏| 10955/12000 [12:30<41:45,  2.40s/it]
                                                     

 91%|█████████▏| 10955/12000 [12:30<41:45,  2.40s/it]
 91%|█████████▏| 10956/12000 [12:32<37:41,  2.17s/it]
                                                     

 91%|█████████▏| 10956/12000 [12:32<37:41,  2.17s/it]
 91%|█████████▏| 10957/12000 [12:35<40:48,  2.35s/it]
                                                     

 91%|█████████▏| 10957/12000 [12:35<40:48,  2.35s/it]
 91%|█████████▏| 10958/12000 [12:36<36:25,  2.10s/it]
                                                     

 91%|█████████▏| 10958/12000 [12:36<36:25,  2.10s/it]
 91%|█████████▏| 10959/12000 [12:38<35:25,  2.04s/it]
                                                     

 91%|█████████▏| 10959/12000 [12:38<35:25,  2.04s/it]
 91%|█████████▏| 10960/12000 [12:40<35:58,  2.08s/it]
                                                     

 91%|█████████▏| 10960/12000 [12:40<35:58,  2.08s/it]
 91%|█████████▏| 10961/12000 [12:42<35:23,  2.04s/it]
                                                     

 91%|█████████▏| 10961/12000 [12:42<35:23,  2.04s/it]
 91%|█████████▏| 10962/12000 [12:44<35:31,  2.05s/it]
                                                     

 91%|█████████▏| 10962/12000 [12:44<35:31,  2.05s/it]
 91%|█████████▏| 10963/12000 [12:47<38:24,  2.22s/it]
                                                     

 91%|█████████▏| 10963/12000 [12:47<38:24,  2.22s/it]
 91%|█████████▏| 10964/12000 [12:50<39:34,  2.29s/it]
                                                     

 91%|█████████▏| 10964/12000 [12:50<39:34,  2.29s/it]
 91%|█████████▏| 10965/12000 [12:51<35:28,  2.06s/it]
                                                     

 91%|█████████▏| 10965/12000 [12:51<35:28,  2.06s/it]
 91%|█████████▏| 10966/12000 [12:52<31:51,  1.85s/it]
                                                     

 91%|█████████▏| 10966/12000 [12:52<31:51,  1.85s/it]
 91%|█████████▏| 10967/12000 [12:55<35:26,  2.06s/it]
                                                     

 91%|█████████▏| 10967/12000 [12:55<35:26,  2.06s/it]
 91%|█████████▏| 10968/12000 [12:57<34:26,  2.00s/it]
                                                     

 91%|█████████▏| 10968/12000 [12:57<34:26,  2.00s/it]
 91%|█████████▏| 10969/12000 [12:58<31:54,  1.86s/it]
                                                     

 91%|█████████▏| 10969/12000 [12:58<31:54,  1.86s/it]
 91%|█████████▏| 10970/12000 [13:00<30:49,  1.80s/it]
                                                     

 91%|█████████▏| 10970/12000 [13:00<30:49,  1.80s/it]
 91%|█████████▏| 10971/12000 [13:02<31:03,  1.81s/it]
                                                     

 91%|█████████▏| 10971/12000 [13:02<31:03,  1.81s/it]
 91%|█████████▏| 10972/12000 [13:03<29:55,  1.75s/it]
                                                     

 91%|█████████▏| 10972/12000 [13:03<29:55,  1.75s/it]
 91%|█████████▏| 10973/12000 [13:05<29:00,  1.70s/it]
                                                     

 91%|█████████▏| 10973/12000 [13:05<29:00,  1.70s/it]
 91%|█████████▏| 10974/12000 [13:09<40:29,  2.37s/it]
                                                     

 91%|█████████▏| 10974/12000 [13:09<40:29,  2.37s/it]
 91%|█████████▏| 10975/12000 [13:11<36:23,  2.13s/it]
                                                     

 91%|█████████▏| 10975/12000 [13:11<36:23,  2.13s/it]
 91%|█████████▏| 10976/12000 [13:15<50:12,  2.94s/it]
                                                     

 91%|█████████▏| 10976/12000 [13:15<50:12,  2.94s/it]
 91%|█████████▏| 10977/12000 [13:17<42:13,  2.48s/it]
                                                     
{'loss': '0.9322', 'grad_norm': '0.3592', 'learning_rate': '4.324e-06', 'entropy': '0.8311', 'num_tokens': '1.741e+06', 'mean_token_accuracy': '0.7997', 'epoch': '0.09673'}
{'loss': '1.115', 'grad_norm': '0.4745', 'learning_rate': '4.316e-06', 'entropy': '1.035', 'num_tokens': '1.746e+06', 'mean_token_accuracy': '0.7373', 'epoch': '0.09674'}
{'loss': '0.7461', 'grad_norm': '0.4031', 'learning_rate': '4.308e-06', 'entropy': '0.682', 'num_tokens': '1.75e+06', 'mean_token_accuracy': '0.8151', 'epoch': '0.09675'}
{'loss': '0.6165', 'grad_norm': '0.3569', 'learning_rate': '4.3e-06', 'entropy': '0.7982', 'num_tokens': '1.756e+06', 'mean_token_accuracy': '0.801', 'epoch': '0.09676'}
{'loss': '0.5483', 'grad_norm': '0.4245', 'learning_rate': '4.292e-06', 'entropy': '0.5455', 'num_tokens': '1.76e+06', 'mean_token_accuracy': '0.8409', 'epoch': '0.09676'}
{'loss': '0.6459', 'grad_norm': '0.3373', 'learning_rate': '4.284e-06', 'entropy': '0.669', 'num_tokens': '1.765e+06', 'mean_token_accuracy': '0.8454', 'epoch': '0.09677'}
{'loss': '0.5396', 'grad_norm': '0.4037', 'learning_rate': '4.276e-06', 'entropy': '0.5978', 'num_tokens': '1.769e+06', 'mean_token_accuracy': '0.8476', 'epoch': '0.09678'}
{'loss': '0.8866', 'grad_norm': '0.3642', 'learning_rate': '4.268e-06', 'entropy': '0.9961', 'num_tokens': '1.778e+06', 'mean_token_accuracy': '0.766', 'epoch': '0.09679'}
{'loss': '0.9286', 'grad_norm': '0.4126', 'learning_rate': '4.26e-06', 'entropy': '0.9947', 'num_tokens': '1.782e+06', 'mean_token_accuracy': '0.748', 'epoch': '0.0968'}
{'loss': '0.9136', 'grad_norm': '0.4187', 'learning_rate': '4.252e-06', 'entropy': '0.9196', 'num_tokens': '1.787e+06', 'mean_token_accuracy': '0.7754', 'epoch': '0.09681'}
{'loss': '1.677', 'grad_norm': '0.5079', 'learning_rate': '4.244e-06', 'entropy': '1.263', 'num_tokens': '1.792e+06', 'mean_token_accuracy': '0.7091', 'epoch': '0.09682'}
{'loss': '0.8626', 'grad_norm': '0.3668', 'learning_rate': '4.236e-06', 'entropy': '0.9903', 'num_tokens': '1.799e+06', 'mean_token_accuracy': '0.7612', 'epoch': '0.09683'}
{'loss': '0.5971', 'grad_norm': '0.3282', 'learning_rate': '4.228e-06', 'entropy': '0.5857', 'num_tokens': '1.807e+06', 'mean_token_accuracy': '0.849', 'epoch': '0.09683'}
{'loss': '0.7467', 'grad_norm': '0.359', 'learning_rate': '4.22e-06', 'entropy': '0.7667', 'num_tokens': '1.811e+06', 'mean_token_accuracy': '0.8081', 'epoch': '0.09684'}
{'loss': '0.9562', 'grad_norm': '0.5022', 'learning_rate': '4.212e-06', 'entropy': '0.8893', 'num_tokens': '1.817e+06', 'mean_token_accuracy': '0.7762', 'epoch': '0.09685'}
{'loss': '1.634', 'grad_norm': '1.115', 'learning_rate': '4.205e-06', 'entropy': '1.404', 'num_tokens': '1.828e+06', 'mean_token_accuracy': '0.6741', 'epoch': '0.09686'}
{'loss': '0.7746', 'grad_norm': '0.4487', 'learning_rate': '4.197e-06', 'entropy': '0.8066', 'num_tokens': '1.832e+06', 'mean_token_accuracy': '0.8021', 'epoch': '0.09687'}
{'loss': '0.6658', 'grad_norm': '0.3866', 'learning_rate': '4.189e-06', 'entropy': '0.7338', 'num_tokens': '1.836e+06', 'mean_token_accuracy': '0.823', 'epoch': '0.09688'}
{'loss': '0.5161', 'grad_norm': '0.391', 'learning_rate': '4.181e-06', 'entropy': '0.5217', 'num_tokens': '1.842e+06', 'mean_token_accuracy': '0.8597', 'epoch': '0.09689'}
{'loss': '0.8004', 'grad_norm': '0.4068', 'learning_rate': '4.173e-06', 'entropy': '0.7675', 'num_tokens': '1.848e+06', 'mean_token_accuracy': '0.8063', 'epoch': '0.0969'}
{'loss': '1.173', 'grad_norm': '0.4001', 'learning_rate': '4.165e-06', 'entropy': '1.201', 'num_tokens': '1.855e+06', 'mean_token_accuracy': '0.7297', 'epoch': '0.09691'}
{'loss': '1.013', 'grad_norm': '0.4105', 'learning_rate': '4.157e-06', 'entropy': '0.9819', 'num_tokens': '1.86e+06', 'mean_token_accuracy': '0.7473', 'epoch': '0.09691'}
{'loss': '0.7699', 'grad_norm': '0.356', 'learning_rate': '4.149e-06', 'entropy': '0.7382', 'num_tokens': '1.866e+06', 'mean_token_accuracy': '0.81', 'epoch': '0.09692'}
{'loss': '1.056', 'grad_norm': '0.4539', 'learning_rate': '4.142e-06', 'entropy': '1.109', 'num_tokens': '1.873e+06', 'mean_token_accuracy': '0.7371', 'epoch': '0.09693'}
{'loss': '0.6945', 'grad_norm': '0.3628', 'learning_rate': '4.134e-06', 'entropy': '0.6843', 'num_tokens': '1.88e+06', 'mean_token_accuracy': '0.8322', 'epoch': '0.09694'}
{'loss': '0.5222', 'grad_norm': '0.3769', 'learning_rate': '4.126e-06', 'entropy': '0.5293', 'num_tokens': '1.884e+06', 'mean_token_accuracy': '0.8606', 'epoch': '0.09695'}
{'loss': '0.5765', 'grad_norm': '0.4313', 'learning_rate': '4.118e-06', 'entropy': '0.6013', 'num_tokens': '1.888e+06', 'mean_token_accuracy': '0.8409', 'epoch': '0.09696'}
{'loss': '0.7311', 'grad_norm': '0.3319', 'learning_rate': '4.11e-06', 'entropy': '0.6909', 'num_tokens': '1.895e+06', 'mean_token_accuracy': '0.8234', 'epoch': '0.09697'}
{'loss': '0.9484', 'grad_norm': '0.4579', 'learning_rate': '4.102e-06', 'entropy': '0.9967', 'num_tokens': '1.9e+06', 'mean_token_accuracy': '0.7599', 'epoch': '0.09698'}
{'loss': '0.8673', 'grad_norm': '0.4345', 'learning_rate': '4.095e-06', 'entropy': '0.8256', 'num_tokens': '1.905e+06', 'mean_token_accuracy': '0.7948', 'epoch': '0.09699'}
{'loss': '0.6442', 'grad_norm': '0.3302', 'learning_rate': '4.087e-06', 'entropy': '0.6302', 'num_tokens': '1.911e+06', 'mean_token_accuracy': '0.8433', 'epoch': '0.09699'}
{'loss': '0.9993', 'grad_norm': '0.49', 'learning_rate': '4.079e-06', 'entropy': '1.056', 'num_tokens': '1.916e+06', 'mean_token_accuracy': '0.7494', 'epoch': '0.097'}
{'loss': '0.6116', 'grad_norm': '0.325', 'learning_rate': '4.071e-06', 'entropy': '0.6245', 'num_tokens': '1.922e+06', 'mean_token_accuracy': '0.8416', 'epoch': '0.09701'}
{'loss': '0.5857', 'grad_norm': '0.3514', 'learning_rate': '4.063e-06', 'entropy': '0.7686', 'num_tokens': '1.927e+06', 'mean_token_accuracy': '0.8067', 'epoch': '0.09702'}
{'loss': '0.7399', 'grad_norm': '0.3671', 'learning_rate': '4.056e-06', 'entropy': '0.7186', 'num_tokens': '1.932e+06', 'mean_token_accuracy': '0.8129', 'epoch': '0.09703'}
{'loss': '0.7579', 'grad_norm': '0.4549', 'learning_rate': '4.048e-06', 'entropy': '0.887', 'num_tokens': '1.936e+06', 'mean_token_accuracy': '0.7774', 'epoch': '0.09704'}
{'loss': '0.9331', 'grad_norm': '0.4411', 'learning_rate': '4.04e-06', 'entropy': '0.8879', 'num_tokens': '1.941e+06', 'mean_token_accuracy': '0.7709', 'epoch': '0.09705'}
{'loss': '0.5459', 'grad_norm': '0.3906', 'learning_rate': '4.032e-06', 'entropy': '0.6305', 'num_tokens': '1.946e+06', 'mean_token_accuracy': '0.828', 'epoch': '0.09706'}
{'loss': '0.9262', 'grad_norm': '0.3837', 'learning_rate': '4.025e-06', 'entropy': '0.9029', 'num_tokens': '1.952e+06', 'mean_token_accuracy': '0.7837', 'epoch': '0.09706'}
{'loss': '0.7139', 'grad_norm': '0.4278', 'learning_rate': '4.017e-06', 'entropy': '0.7062', 'num_tokens': '1.956e+06', 'mean_token_accuracy': '0.8165', 'epoch': '0.09707'}
{'loss': '0.5228', 'grad_norm': '0.3312', 'learning_rate': '4.009e-06', 'entropy': '0.5383', 'num_tokens': '1.963e+06', 'mean_token_accuracy': '0.8599', 'epoch': '0.09708'}
{'loss': '0.7264', 'grad_norm': '0.455', 'learning_rate': '4.001e-06', 'entropy': '0.6931', 'num_tokens': '1.967e+06', 'mean_token_accuracy': '0.8167', 'epoch': '0.09709'}
{'loss': '1.076', 'grad_norm': '0.4765', 'learning_rate': '3.994e-06', 'entropy': '1.021', 'num_tokens': '1.972e+06', 'mean_token_accuracy': '0.7514', 'epoch': '0.0971'}
{'loss': '0.8007', 'grad_norm': '0.4333', 'learning_rate': '3.986e-06', 'entropy': '0.8211', 'num_tokens': '1.976e+06', 'mean_token_accuracy': '0.7852', 'epoch': '0.09711'}
{'loss': '1.014', 'grad_norm': '0.5024', 'learning_rate': '3.978e-06', 'entropy': '0.966', 'num_tokens': '1.982e+06', 'mean_token_accuracy': '0.7519', 'epoch': '0.09712'}
{'loss': '0.8376', 'grad_norm': '0.377', 'learning_rate': '3.971e-06', 'entropy': '0.9349', 'num_tokens': '1.987e+06', 'mean_token_accuracy': '0.7703', 'epoch': '0.09713'}
{'loss': '0.4411', 'grad_norm': '0.247', 'learning_rate': '3.963e-06', 'entropy': '0.5323', 'num_tokens': '1.996e+06', 'mean_token_accuracy': '0.8586', 'epoch': '0.09714'}

 91%|█████████▏| 10977/12000 [13:17<42:13,  2.48s/it]
 91%|█████████▏| 10978/12000 [13:18<36:48,  2.16s/it]
                                                     

 91%|█████████▏| 10978/12000 [13:18<36:48,  2.16s/it]
 91%|█████████▏| 10979/12000 [13:20<34:47,  2.04s/it]
                                                     

 91%|█████████▏| 10979/12000 [13:20<34:47,  2.04s/it]
 92%|█████████▏| 10980/12000 [13:22<33:27,  1.97s/it]
                                                     

 92%|█████████▏| 10980/12000 [13:22<33:27,  1.97s/it]
 92%|█████████▏| 10981/12000 [13:23<31:40,  1.87s/it]
                                                     

 92%|█████████▏| 10981/12000 [13:23<31:40,  1.87s/it]
 92%|█████████▏| 10982/12000 [13:26<33:08,  1.95s/it]
                                                     

 92%|█████████▏| 10982/12000 [13:26<33:08,  1.95s/it]
 92%|█████████▏| 10983/12000 [13:30<46:49,  2.76s/it]
                                                     

 92%|█████████▏| 10983/12000 [13:30<46:49,  2.76s/it]
 92%|█████████▏| 10984/12000 [13:32<41:17,  2.44s/it]
                                                     

 92%|█████████▏| 10984/12000 [13:32<41:17,  2.44s/it]
 92%|█████████▏| 10985/12000 [13:33<36:29,  2.16s/it]
                                                     

 92%|█████████▏| 10985/12000 [13:33<36:29,  2.16s/it]
 92%|█████████▏| 10986/12000 [13:35<32:48,  1.94s/it]
                                                     

 92%|█████████▏| 10986/12000 [13:35<32:48,  1.94s/it]
 92%|█████████▏| 10987/12000 [13:38<37:51,  2.24s/it]
                                                     

 92%|█████████▏| 10987/12000 [13:38<37:51,  2.24s/it]
 92%|█████████▏| 10988/12000 [13:39<35:06,  2.08s/it]
                                                     

 92%|█████████▏| 10988/12000 [13:39<35:06,  2.08s/it]
 92%|█████████▏| 10989/12000 [13:43<41:21,  2.45s/it]
                                                     

 92%|█████████▏| 10989/12000 [13:43<41:21,  2.45s/it]
 92%|█████████▏| 10990/12000 [13:46<45:43,  2.72s/it]
                                                     

 92%|█████████▏| 10990/12000 [13:46<45:43,  2.72s/it]
 92%|█████████▏| 10991/12000 [13:48<41:21,  2.46s/it]
                                                     

 92%|█████████▏| 10991/12000 [13:48<41:21,  2.46s/it]
 92%|█████████▏| 10992/12000 [13:49<35:45,  2.13s/it]
                                                     

 92%|█████████▏| 10992/12000 [13:49<35:45,  2.13s/it]
 92%|█████████▏| 10993/12000 [13:53<44:39,  2.66s/it]
                                                     

 92%|█████████▏| 10993/12000 [13:53<44:39,  2.66s/it]
 92%|█████████▏| 10994/12000 [13:55<38:17,  2.28s/it]
                                                     

 92%|█████████▏| 10994/12000 [13:55<38:17,  2.28s/it]
 92%|█████████▏| 10995/12000 [13:56<36:03,  2.15s/it]
                                                     

 92%|█████████▏| 10995/12000 [13:56<36:03,  2.15s/it]
 92%|█████████▏| 10996/12000 [14:00<43:33,  2.60s/it]
                                                     

 92%|█████████▏| 10996/12000 [14:00<43:33,  2.60s/it]
 92%|█████████▏| 10997/12000 [14:03<44:26,  2.66s/it]
                                                     

 92%|█████████▏| 10997/12000 [14:03<44:26,  2.66s/it]
 92%|█████████▏| 10998/12000 [14:04<38:56,  2.33s/it]
                                                     

 92%|█████████▏| 10998/12000 [14:04<38:56,  2.33s/it]
 92%|█████████▏| 10999/12000 [14:07<39:21,  2.36s/it]
                                                     

 92%|█████████▏| 10999/12000 [14:07<39:21,  2.36s/it]
 92%|█████████▏| 11000/12000 [14:09<36:40,  2.20s/it]
                                                     

 92%|█████████▏| 11000/12000 [14:09<36:40,  2.20s/it]
 92%|█████████▏| 11001/12000 [14:13<45:56,  2.76s/it]
                                                     

 92%|█████████▏| 11001/12000 [14:13<45:56,  2.76s/it]
 92%|█████████▏| 11002/12000 [14:14<40:15,  2.42s/it]
                                                     

 92%|█████████▏| 11002/12000 [14:14<40:15,  2.42s/it]
 92%|█████████▏| 11003/12000 [14:16<37:26,  2.25s/it]
                                                     

 92%|█████████▏| 11003/12000 [14:16<37:26,  2.25s/it]
 92%|█████████▏| 11004/12000 [14:19<41:28,  2.50s/it]
                                                     

 92%|█████████▏| 11004/12000 [14:19<41:28,  2.50s/it]
 92%|█████████▏| 11005/12000 [14:21<35:51,  2.16s/it]
                                                     

 92%|█████████▏| 11005/12000 [14:21<35:51,  2.16s/it]
 92%|█████████▏| 11006/12000 [14:23<33:50,  2.04s/it]
                                                     

 92%|█████████▏| 11006/12000 [14:23<33:50,  2.04s/it]
 92%|█████████▏| 11007/12000 [14:25<34:16,  2.07s/it]
                                                     

 92%|█████████▏| 11007/12000 [14:25<34:16,  2.07s/it]
 92%|█████████▏| 11008/12000 [14:27<35:32,  2.15s/it]
                                                     

 92%|█████████▏| 11008/12000 [14:27<35:32,  2.15s/it]
 92%|█████████▏| 11009/12000 [14:29<33:21,  2.02s/it]
                                                     

 92%|█████████▏| 11009/12000 [14:29<33:21,  2.02s/it]
 92%|█████████▏| 11010/12000 [14:30<30:47,  1.87s/it]
                                                     

 92%|█████████▏| 11010/12000 [14:30<30:47,  1.87s/it]
 92%|█████████▏| 11011/12000 [14:32<29:38,  1.80s/it]
                                                     

 92%|█████████▏| 11011/12000 [14:32<29:38,  1.80s/it]
 92%|█████████▏| 11012/12000 [14:35<38:27,  2.34s/it]
                                                     

 92%|█████████▏| 11012/12000 [14:35<38:27,  2.34s/it]
 92%|█████████▏| 11013/12000 [14:37<34:21,  2.09s/it]
                                                     

 92%|█████████▏| 11013/12000 [14:37<34:21,  2.09s/it]
 92%|█████████▏| 11014/12000 [14:38<31:20,  1.91s/it]
                                                     

 92%|█████████▏| 11014/12000 [14:38<31:20,  1.91s/it]
 92%|█████████▏| 11015/12000 [14:40<31:39,  1.93s/it]
                                                     

 92%|█████████▏| 11015/12000 [14:40<31:39,  1.93s/it]
 92%|█████████▏| 11016/12000 [14:42<31:12,  1.90s/it]
                                                     

 92%|█████████▏| 11016/12000 [14:42<31:12,  1.90s/it]
 92%|█████████▏| 11017/12000 [14:44<32:37,  1.99s/it]
                                                     

 92%|█████████▏| 11017/12000 [14:44<32:37,  1.99s/it]
 92%|█████████▏| 11018/12000 [14:48<39:53,  2.44s/it]
                                                     

 92%|█████████▏| 11018/12000 [14:48<39:53,  2.44s/it]
 92%|█████████▏| 11019/12000 [14:50<36:02,  2.20s/it]
                                                     

 92%|█████████▏| 11019/12000 [14:50<36:02,  2.20s/it]
 92%|█████████▏| 11020/12000 [14:51<32:59,  2.02s/it]
                                                     

 92%|█████████▏| 11020/12000 [14:51<32:59,  2.02s/it]
 92%|█████████▏| 11021/12000 [14:54<36:28,  2.24s/it]
                                                     

 92%|█████████▏| 11021/12000 [14:54<36:28,  2.24s/it]
 92%|█████████▏| 11022/12000 [14:57<38:47,  2.38s/it]
                                                     

 92%|█████████▏| 11022/12000 [14:57<38:47,  2.38s/it]
 92%|█████████▏| 11023/12000 [14:58<34:47,  2.14s/it]
                                                     

 92%|█████████▏| 11023/12000 [14:58<34:47,  2.14s/it]
 92%|█████████▏| 11024/12000 [15:00<31:29,  1.94s/it]
                                                     
{'loss': '0.7035', 'grad_norm': '0.4166', 'learning_rate': '3.955e-06', 'entropy': '0.8038', 'num_tokens': '2e+06', 'mean_token_accuracy': '0.7968', 'epoch': '0.09714'}
{'loss': '0.9515', 'grad_norm': '0.4115', 'learning_rate': '3.948e-06', 'entropy': '0.9828', 'num_tokens': '2.003e+06', 'mean_token_accuracy': '0.7546', 'epoch': '0.09715'}
{'loss': '0.9312', 'grad_norm': '0.3956', 'learning_rate': '3.94e-06', 'entropy': '0.9378', 'num_tokens': '2.009e+06', 'mean_token_accuracy': '0.7745', 'epoch': '0.09716'}
{'loss': '0.6956', 'grad_norm': '0.3915', 'learning_rate': '3.932e-06', 'entropy': '0.6883', 'num_tokens': '2.014e+06', 'mean_token_accuracy': '0.8195', 'epoch': '0.09717'}
{'loss': '0.6537', 'grad_norm': '0.2924', 'learning_rate': '3.925e-06', 'entropy': '0.7677', 'num_tokens': '2.019e+06', 'mean_token_accuracy': '0.8037', 'epoch': '0.09718'}
{'loss': '0.8811', 'grad_norm': '0.3402', 'learning_rate': '3.917e-06', 'entropy': '0.9071', 'num_tokens': '2.024e+06', 'mean_token_accuracy': '0.7541', 'epoch': '0.09719'}
{'loss': '1.182', 'grad_norm': '0.399', 'learning_rate': '3.909e-06', 'entropy': '0.8523', 'num_tokens': '2.033e+06', 'mean_token_accuracy': '0.7984', 'epoch': '0.0972'}
{'loss': '0.5739', 'grad_norm': '0.3159', 'learning_rate': '3.902e-06', 'entropy': '0.614', 'num_tokens': '2.038e+06', 'mean_token_accuracy': '0.8423', 'epoch': '0.09721'}
{'loss': '0.7755', 'grad_norm': '0.4426', 'learning_rate': '3.894e-06', 'entropy': '0.8263', 'num_tokens': '2.042e+06', 'mean_token_accuracy': '0.7958', 'epoch': '0.09722'}
{'loss': '0.755', 'grad_norm': '0.372', 'learning_rate': '3.887e-06', 'entropy': '0.8013', 'num_tokens': '2.046e+06', 'mean_token_accuracy': '0.7955', 'epoch': '0.09722'}
{'loss': '0.9909', 'grad_norm': '0.367', 'learning_rate': '3.879e-06', 'entropy': '1.055', 'num_tokens': '2.054e+06', 'mean_token_accuracy': '0.7357', 'epoch': '0.09723'}
{'loss': '0.6601', 'grad_norm': '0.4179', 'learning_rate': '3.871e-06', 'entropy': '0.7395', 'num_tokens': '2.059e+06', 'mean_token_accuracy': '0.8303', 'epoch': '0.09724'}
{'loss': '1.388', 'grad_norm': '0.4588', 'learning_rate': '3.864e-06', 'entropy': '1.16', 'num_tokens': '2.067e+06', 'mean_token_accuracy': '0.7373', 'epoch': '0.09725'}
{'loss': '1.052', 'grad_norm': '0.3454', 'learning_rate': '3.856e-06', 'entropy': '0.996', 'num_tokens': '2.074e+06', 'mean_token_accuracy': '0.7653', 'epoch': '0.09726'}
{'loss': '0.9621', 'grad_norm': '0.4058', 'learning_rate': '3.849e-06', 'entropy': '0.9973', 'num_tokens': '2.081e+06', 'mean_token_accuracy': '0.7503', 'epoch': '0.09727'}
{'loss': '0.5792', 'grad_norm': '0.394', 'learning_rate': '3.841e-06', 'entropy': '0.6295', 'num_tokens': '2.084e+06', 'mean_token_accuracy': '0.8358', 'epoch': '0.09728'}
{'loss': '0.8972', 'grad_norm': '0.398', 'learning_rate': '3.833e-06', 'entropy': '0.8062', 'num_tokens': '2.094e+06', 'mean_token_accuracy': '0.7896', 'epoch': '0.09729'}
{'loss': '1.108', 'grad_norm': '0.4895', 'learning_rate': '3.826e-06', 'entropy': '1.106', 'num_tokens': '2.098e+06', 'mean_token_accuracy': '0.7444', 'epoch': '0.0973'}
{'loss': '0.9502', 'grad_norm': '0.4333', 'learning_rate': '3.818e-06', 'entropy': '0.8939', 'num_tokens': '2.104e+06', 'mean_token_accuracy': '0.7751', 'epoch': '0.0973'}
{'loss': '0.4542', 'grad_norm': '0.3379', 'learning_rate': '3.811e-06', 'entropy': '0.5137', 'num_tokens': '2.109e+06', 'mean_token_accuracy': '0.8634', 'epoch': '0.09731'}
{'loss': '0.7334', 'grad_norm': '0.3371', 'learning_rate': '3.803e-06', 'entropy': '0.7706', 'num_tokens': '2.116e+06', 'mean_token_accuracy': '0.8079', 'epoch': '0.09732'}
{'loss': '0.6715', 'grad_norm': '0.4693', 'learning_rate': '3.796e-06', 'entropy': '0.7164', 'num_tokens': '2.12e+06', 'mean_token_accuracy': '0.8222', 'epoch': '0.09733'}
{'loss': '0.8737', 'grad_norm': '0.4387', 'learning_rate': '3.788e-06', 'entropy': '0.8668', 'num_tokens': '2.127e+06', 'mean_token_accuracy': '0.7742', 'epoch': '0.09734'}
{'loss': '0.7996', 'grad_norm': '0.4426', 'learning_rate': '3.781e-06', 'entropy': '0.9851', 'num_tokens': '2.133e+06', 'mean_token_accuracy': '0.7659', 'epoch': '0.09735'}
{'loss': '0.8248', 'grad_norm': '0.381', 'learning_rate': '3.773e-06', 'entropy': '0.7794', 'num_tokens': '2.138e+06', 'mean_token_accuracy': '0.8219', 'epoch': '0.09736'}
{'loss': '0.9902', 'grad_norm': '0.4933', 'learning_rate': '3.766e-06', 'entropy': '1.007', 'num_tokens': '2.142e+06', 'mean_token_accuracy': '0.7405', 'epoch': '0.09737'}
{'loss': '0.7479', 'grad_norm': '0.4065', 'learning_rate': '3.758e-06', 'entropy': '0.8031', 'num_tokens': '2.147e+06', 'mean_token_accuracy': '0.797', 'epoch': '0.09737'}
{'loss': '0.8596', 'grad_norm': '0.4034', 'learning_rate': '3.751e-06', 'entropy': '0.9171', 'num_tokens': '2.155e+06', 'mean_token_accuracy': '0.7735', 'epoch': '0.09738'}
{'loss': '1.065', 'grad_norm': '0.533', 'learning_rate': '3.743e-06', 'entropy': '1.072', 'num_tokens': '2.159e+06', 'mean_token_accuracy': '0.7451', 'epoch': '0.09739'}
{'loss': '0.9071', 'grad_norm': '0.3977', 'learning_rate': '3.736e-06', 'entropy': '0.9135', 'num_tokens': '2.164e+06', 'mean_token_accuracy': '0.77', 'epoch': '0.0974'}
{'loss': '1.02', 'grad_norm': '0.528', 'learning_rate': '3.728e-06', 'entropy': '0.8556', 'num_tokens': '2.169e+06', 'mean_token_accuracy': '0.7855', 'epoch': '0.09741'}
{'loss': '0.6416', 'grad_norm': '0.3847', 'learning_rate': '3.721e-06', 'entropy': '0.5946', 'num_tokens': '2.175e+06', 'mean_token_accuracy': '0.8486', 'epoch': '0.09742'}
{'loss': '0.8405', 'grad_norm': '0.4072', 'learning_rate': '3.713e-06', 'entropy': '0.8838', 'num_tokens': '2.18e+06', 'mean_token_accuracy': '0.7681', 'epoch': '0.09743'}
{'loss': '0.8164', 'grad_norm': '0.4245', 'learning_rate': '3.706e-06', 'entropy': '0.7989', 'num_tokens': '2.184e+06', 'mean_token_accuracy': '0.7986', 'epoch': '0.09744'}
{'loss': '0.8681', 'grad_norm': '0.4489', 'learning_rate': '3.699e-06', 'entropy': '0.8515', 'num_tokens': '2.188e+06', 'mean_token_accuracy': '0.7927', 'epoch': '0.09745'}
{'loss': '0.9458', 'grad_norm': '0.3954', 'learning_rate': '3.691e-06', 'entropy': '0.9192', 'num_tokens': '2.196e+06', 'mean_token_accuracy': '0.7735', 'epoch': '0.09745'}
{'loss': '0.8686', 'grad_norm': '0.3741', 'learning_rate': '3.684e-06', 'entropy': '0.8528', 'num_tokens': '2.201e+06', 'mean_token_accuracy': '0.7968', 'epoch': '0.09746'}
{'loss': '0.5593', 'grad_norm': '0.335', 'learning_rate': '3.676e-06', 'entropy': '0.5581', 'num_tokens': '2.205e+06', 'mean_token_accuracy': '0.8567', 'epoch': '0.09747'}
{'loss': '0.6884', 'grad_norm': '0.4499', 'learning_rate': '3.669e-06', 'entropy': '0.6897', 'num_tokens': '2.21e+06', 'mean_token_accuracy': '0.8032', 'epoch': '0.09748'}
{'loss': '1.101', 'grad_norm': '0.4646', 'learning_rate': '3.662e-06', 'entropy': '1.065', 'num_tokens': '2.215e+06', 'mean_token_accuracy': '0.739', 'epoch': '0.09749'}
{'loss': '1.058', 'grad_norm': '0.5596', 'learning_rate': '3.654e-06', 'entropy': '0.8488', 'num_tokens': '2.221e+06', 'mean_token_accuracy': '0.7877', 'epoch': '0.0975'}
{'loss': '0.6433', 'grad_norm': '0.397', 'learning_rate': '3.647e-06', 'entropy': '0.6447', 'num_tokens': '2.225e+06', 'mean_token_accuracy': '0.8293', 'epoch': '0.09751'}
{'loss': '0.9096', 'grad_norm': '0.5452', 'learning_rate': '3.639e-06', 'entropy': '0.9077', 'num_tokens': '2.229e+06', 'mean_token_accuracy': '0.7662', 'epoch': '0.09752'}
{'loss': '1.197', 'grad_norm': '0.4423', 'learning_rate': '3.632e-06', 'entropy': '1.305', 'num_tokens': '2.234e+06', 'mean_token_accuracy': '0.6946', 'epoch': '0.09753'}
{'loss': '0.867', 'grad_norm': '0.3812', 'learning_rate': '3.625e-06', 'entropy': '0.9165', 'num_tokens': '2.24e+06', 'mean_token_accuracy': '0.7715', 'epoch': '0.09753'}
{'loss': '0.9905', 'grad_norm': '0.3583', 'learning_rate': '3.617e-06', 'entropy': '0.9111', 'num_tokens': '2.248e+06', 'mean_token_accuracy': '0.78', 'epoch': '0.09754'}
{'loss': '0.5744', 'grad_norm': '0.4', 'learning_rate': '3.61e-06', 'entropy': '0.5705', 'num_tokens': '2.253e+06', 'mean_token_accuracy': '0.849', 'epoch': '0.09755'}

 92%|█████████▏| 11024/12000 [15:00<31:29,  1.94s/it]
 92%|█████████▏| 11025/12000 [15:06<51:32,  3.17s/it]
                                                     

 92%|█████████▏| 11025/12000 [15:06<51:32,  3.17s/it]
 92%|█████████▏| 11026/12000 [15:08<44:51,  2.76s/it]
                                                     

 92%|█████████▏| 11026/12000 [15:08<44:51,  2.76s/it]
 92%|█████████▏| 11027/12000 [15:09<38:36,  2.38s/it]
                                                     

 92%|█████████▏| 11027/12000 [15:09<38:36,  2.38s/it]
 92%|█████████▏| 11028/12000 [15:12<41:47,  2.58s/it]
                                                     

 92%|█████████▏| 11028/12000 [15:12<41:47,  2.58s/it]
 92%|█████████▏| 11029/12000 [15:15<43:33,  2.69s/it]
                                                     

 92%|█████████▏| 11029/12000 [15:15<43:33,  2.69s/it]
 92%|█████████▏| 11030/12000 [15:16<37:28,  2.32s/it]
                                                     

 92%|█████████▏| 11030/12000 [15:16<37:28,  2.32s/it]
 92%|█████████▏| 11031/12000 [15:18<34:44,  2.15s/it]
                                                     

 92%|█████████▏| 11031/12000 [15:18<34:44,  2.15s/it]
 92%|█████████▏| 11032/12000 [15:21<36:20,  2.25s/it]
                                                     

 92%|█████████▏| 11032/12000 [15:21<36:20,  2.25s/it]
 92%|█████████▏| 11033/12000 [15:22<32:27,  2.01s/it]
                                                     

 92%|█████████▏| 11033/12000 [15:22<32:27,  2.01s/it]
 92%|█████████▏| 11034/12000 [15:24<30:49,  1.91s/it]
                                                     

 92%|█████████▏| 11034/12000 [15:24<30:49,  1.91s/it]
 92%|█████████▏| 11035/12000 [15:27<36:08,  2.25s/it]
                                                     

 92%|█████████▏| 11035/12000 [15:27<36:08,  2.25s/it]
 92%|█████████▏| 11036/12000 [15:29<33:27,  2.08s/it]
                                                     

 92%|█████████▏| 11036/12000 [15:29<33:27,  2.08s/it]
 92%|█████████▏| 11037/12000 [15:30<32:24,  2.02s/it]
                                                     

 92%|█████████▏| 11037/12000 [15:30<32:24,  2.02s/it]
 92%|█████████▏| 11038/12000 [15:32<29:12,  1.82s/it]
                                                     

 92%|█████████▏| 11038/12000 [15:32<29:12,  1.82s/it]
 92%|█████████▏| 11039/12000 [15:34<28:31,  1.78s/it]
                                                     

 92%|█████████▏| 11039/12000 [15:34<28:31,  1.78s/it]
 92%|█████████▏| 11040/12000 [15:37<39:07,  2.45s/it]
                                                     

 92%|█████████▏| 11040/12000 [15:38<39:07,  2.45s/it]
 92%|█████████▏| 11041/12000 [15:39<35:16,  2.21s/it]
                                                     

 92%|█████████▏| 11041/12000 [15:39<35:16,  2.21s/it]
 92%|█████████▏| 11042/12000 [15:41<31:50,  1.99s/it]
                                                     

 92%|█████████▏| 11042/12000 [15:41<31:50,  1.99s/it]
 92%|█████████▏| 11043/12000 [15:43<35:48,  2.25s/it]
                                                     

 92%|█████████▏| 11043/12000 [15:43<35:48,  2.25s/it]
 92%|█████████▏| 11044/12000 [15:45<33:07,  2.08s/it]
                                                     

 92%|█████████▏| 11044/12000 [15:45<33:07,  2.08s/it]
 92%|█████████▏| 11045/12000 [15:49<41:23,  2.60s/it]
                                                     

 92%|█████████▏| 11045/12000 [15:49<41:23,  2.60s/it]
 92%|█████████▏| 11046/12000 [15:51<36:53,  2.32s/it]
                                                     

 92%|█████████▏| 11046/12000 [15:51<36:53,  2.32s/it]
 92%|█████████▏| 11047/12000 [15:53<34:51,  2.19s/it]
                                                     

 92%|█████████▏| 11047/12000 [15:53<34:51,  2.19s/it]
 92%|█████████▏| 11048/12000 [15:55<35:33,  2.24s/it]
                                                     

 92%|█████████▏| 11048/12000 [15:55<35:33,  2.24s/it]
 92%|█████████▏| 11049/12000 [15:58<41:05,  2.59s/it]
                                                     

 92%|█████████▏| 11049/12000 [15:58<41:05,  2.59s/it]
 92%|█████████▏| 11050/12000 [16:00<37:05,  2.34s/it]
                                                     

 92%|█████████▏| 11050/12000 [16:00<37:05,  2.34s/it]
 92%|█████████▏| 11051/12000 [16:04<43:51,  2.77s/it]
                                                     

 92%|█████████▏| 11051/12000 [16:04<43:51,  2.77s/it]
 92%|█████████▏| 11052/12000 [16:06<39:49,  2.52s/it]
                                                     

 92%|█████████▏| 11052/12000 [16:06<39:49,  2.52s/it]
 92%|█████████▏| 11053/12000 [16:07<35:24,  2.24s/it]
                                                     

 92%|█████████▏| 11053/12000 [16:07<35:24,  2.24s/it]
 92%|█████████▏| 11054/12000 [16:09<31:32,  2.00s/it]
                                                     

 92%|█████████▏| 11054/12000 [16:09<31:32,  2.00s/it]
 92%|█████████▏| 11055/12000 [16:10<29:45,  1.89s/it]
                                                     

 92%|█████████▏| 11055/12000 [16:10<29:45,  1.89s/it]
 92%|█████████▏| 11056/12000 [16:12<27:02,  1.72s/it]
                                                     

 92%|█████████▏| 11056/12000 [16:12<27:02,  1.72s/it]
 92%|█████████▏| 11057/12000 [16:14<30:54,  1.97s/it]
                                                     

 92%|█████████▏| 11057/12000 [16:14<30:54,  1.97s/it]
 92%|█████████▏| 11058/12000 [16:16<28:46,  1.83s/it]
                                                     

 92%|█████████▏| 11058/12000 [16:16<28:46,  1.83s/it]
 92%|█████████▏| 11059/12000 [16:18<28:33,  1.82s/it]
                                                     

 92%|█████████▏| 11059/12000 [16:18<28:33,  1.82s/it]
 92%|█████████▏| 11060/12000 [16:20<29:43,  1.90s/it]
                                                     

 92%|█████████▏| 11060/12000 [16:20<29:43,  1.90s/it]
 92%|█████████▏| 11061/12000 [16:21<28:55,  1.85s/it]
                                                     

 92%|█████████▏| 11061/12000 [16:21<28:55,  1.85s/it]
 92%|█████████▏| 11062/12000 [16:25<37:12,  2.38s/it]
                                                     

 92%|█████████▏| 11062/12000 [16:25<37:12,  2.38s/it]
 92%|█████████▏| 11063/12000 [16:27<34:07,  2.18s/it]
                                                     

 92%|█████████▏| 11063/12000 [16:27<34:07,  2.18s/it]
 92%|█████████▏| 11064/12000 [16:28<30:01,  1.93s/it]
                                                     

 92%|█████████▏| 11064/12000 [16:28<30:01,  1.93s/it]
 92%|█████████▏| 11065/12000 [16:30<27:56,  1.79s/it]
                                                     

 92%|█████████▏| 11065/12000 [16:30<27:56,  1.79s/it]
 92%|█████████▏| 11066/12000 [16:31<27:19,  1.76s/it]
                                                     

 92%|█████████▏| 11066/12000 [16:31<27:19,  1.76s/it]
 92%|█████████▏| 11067/12000 [16:33<25:47,  1.66s/it]
                                                     

 92%|█████████▏| 11067/12000 [16:33<25:47,  1.66s/it]
 92%|█████████▏| 11068/12000 [16:34<25:28,  1.64s/it]
                                                     

 92%|█████████▏| 11068/12000 [16:34<25:28,  1.64s/it]
 92%|█████████▏| 11069/12000 [16:36<28:06,  1.81s/it]
                                                     

 92%|█████████▏| 11069/12000 [16:36<28:06,  1.81s/it]
 92%|█████████▏| 11070/12000 [16:38<26:51,  1.73s/it]
                                                     

 92%|█████████▏| 11070/12000 [16:38<26:51,  1.73s/it]
 92%|█████████▏| 11071/12000 [16:39<25:19,  1.64s/it]
                                                     
{'loss': '0.7591', 'grad_norm': '0.442', 'learning_rate': '3.603e-06', 'entropy': '0.7977', 'num_tokens': '2.257e+06', 'mean_token_accuracy': '0.794', 'epoch': '0.09756'}
{'loss': '0.9865', 'grad_norm': '0.3074', 'learning_rate': '3.595e-06', 'entropy': '0.83', 'num_tokens': '2.268e+06', 'mean_token_accuracy': '0.8093', 'epoch': '0.09757'}
{'loss': '0.7925', 'grad_norm': '0.4298', 'learning_rate': '3.588e-06', 'entropy': '0.7614', 'num_tokens': '2.274e+06', 'mean_token_accuracy': '0.7977', 'epoch': '0.09758'}
{'loss': '0.7746', 'grad_norm': '0.4651', 'learning_rate': '3.581e-06', 'entropy': '0.7728', 'num_tokens': '2.278e+06', 'mean_token_accuracy': '0.798', 'epoch': '0.09759'}
{'loss': '0.7984', 'grad_norm': '0.3734', 'learning_rate': '3.573e-06', 'entropy': '0.8868', 'num_tokens': '2.285e+06', 'mean_token_accuracy': '0.7884', 'epoch': '0.0976'}
{'loss': '0.9091', 'grad_norm': '0.3414', 'learning_rate': '3.566e-06', 'entropy': '0.7908', 'num_tokens': '2.293e+06', 'mean_token_accuracy': '0.8102', 'epoch': '0.0976'}
{'loss': '0.5542', 'grad_norm': '0.3997', 'learning_rate': '3.559e-06', 'entropy': '0.6136', 'num_tokens': '2.297e+06', 'mean_token_accuracy': '0.8435', 'epoch': '0.09761'}
{'loss': '0.8043', 'grad_norm': '0.4679', 'learning_rate': '3.552e-06', 'entropy': '0.7341', 'num_tokens': '2.303e+06', 'mean_token_accuracy': '0.8083', 'epoch': '0.09762'}
{'loss': '0.9507', 'grad_norm': '0.363', 'learning_rate': '3.544e-06', 'entropy': '1.007', 'num_tokens': '2.309e+06', 'mean_token_accuracy': '0.7629', 'epoch': '0.09763'}
{'loss': '0.7022', 'grad_norm': '0.3841', 'learning_rate': '3.537e-06', 'entropy': '0.7472', 'num_tokens': '2.313e+06', 'mean_token_accuracy': '0.8206', 'epoch': '0.09764'}
{'loss': '0.5913', 'grad_norm': '0.3573', 'learning_rate': '3.53e-06', 'entropy': '0.6296', 'num_tokens': '2.318e+06', 'mean_token_accuracy': '0.8353', 'epoch': '0.09765'}
{'loss': '1.282', 'grad_norm': '0.405', 'learning_rate': '3.522e-06', 'entropy': '1.211', 'num_tokens': '2.325e+06', 'mean_token_accuracy': '0.7102', 'epoch': '0.09766'}
{'loss': '0.8658', 'grad_norm': '0.4293', 'learning_rate': '3.515e-06', 'entropy': '0.9329', 'num_tokens': '2.33e+06', 'mean_token_accuracy': '0.7774', 'epoch': '0.09767'}
{'loss': '0.6509', 'grad_norm': '0.3418', 'learning_rate': '3.508e-06', 'entropy': '0.6447', 'num_tokens': '2.337e+06', 'mean_token_accuracy': '0.8329', 'epoch': '0.09768'}
{'loss': '0.7945', 'grad_norm': '0.4012', 'learning_rate': '3.501e-06', 'entropy': '0.8081', 'num_tokens': '2.341e+06', 'mean_token_accuracy': '0.8027', 'epoch': '0.09768'}
{'loss': '0.6566', 'grad_norm': '0.4156', 'learning_rate': '3.494e-06', 'entropy': '0.7255', 'num_tokens': '2.345e+06', 'mean_token_accuracy': '0.816', 'epoch': '0.09769'}
{'loss': '0.6699', 'grad_norm': '0.3794', 'learning_rate': '3.486e-06', 'entropy': '0.6655', 'num_tokens': '2.351e+06', 'mean_token_accuracy': '0.8275', 'epoch': '0.0977'}
{'loss': '0.6316', 'grad_norm': '0.3481', 'learning_rate': '3.479e-06', 'entropy': '0.6548', 'num_tokens': '2.356e+06', 'mean_token_accuracy': '0.8359', 'epoch': '0.09771'}
{'loss': '0.7339', 'grad_norm': '0.4031', 'learning_rate': '3.472e-06', 'entropy': '0.7414', 'num_tokens': '2.361e+06', 'mean_token_accuracy': '0.8015', 'epoch': '0.09772'}
{'loss': '0.8154', 'grad_norm': '0.4188', 'learning_rate': '3.465e-06', 'entropy': '0.7435', 'num_tokens': '2.367e+06', 'mean_token_accuracy': '0.8094', 'epoch': '0.09773'}
{'loss': '0.8618', 'grad_norm': '0.3419', 'learning_rate': '3.458e-06', 'entropy': '0.8626', 'num_tokens': '2.373e+06', 'mean_token_accuracy': '0.7931', 'epoch': '0.09774'}
{'loss': '1.872', 'grad_norm': '0.4048', 'learning_rate': '3.45e-06', 'entropy': '1.149', 'num_tokens': '2.381e+06', 'mean_token_accuracy': '0.7557', 'epoch': '0.09775'}
{'loss': '1.041', 'grad_norm': '0.4283', 'learning_rate': '3.443e-06', 'entropy': '1.103', 'num_tokens': '2.387e+06', 'mean_token_accuracy': '0.7374', 'epoch': '0.09776'}
{'loss': '0.8528', 'grad_norm': '0.5497', 'learning_rate': '3.436e-06', 'entropy': '0.9489', 'num_tokens': '2.392e+06', 'mean_token_accuracy': '0.7659', 'epoch': '0.09776'}
{'loss': '0.7368', 'grad_norm': '0.326', 'learning_rate': '3.429e-06', 'entropy': '0.6754', 'num_tokens': '2.398e+06', 'mean_token_accuracy': '0.8175', 'epoch': '0.09777'}
{'loss': '0.6793', 'grad_norm': '0.2933', 'learning_rate': '3.422e-06', 'entropy': '0.7085', 'num_tokens': '2.406e+06', 'mean_token_accuracy': '0.8182', 'epoch': '0.09778'}
{'loss': '0.7232', 'grad_norm': '0.421', 'learning_rate': '3.415e-06', 'entropy': '0.7767', 'num_tokens': '2.411e+06', 'mean_token_accuracy': '0.8', 'epoch': '0.09779'}
{'loss': '0.6328', 'grad_norm': '0.2884', 'learning_rate': '3.407e-06', 'entropy': '0.7519', 'num_tokens': '2.417e+06', 'mean_token_accuracy': '0.8077', 'epoch': '0.0978'}
{'loss': '1.045', 'grad_norm': '0.4256', 'learning_rate': '3.4e-06', 'entropy': '1.064', 'num_tokens': '2.423e+06', 'mean_token_accuracy': '0.7402', 'epoch': '0.09781'}
{'loss': '0.7685', 'grad_norm': '0.3728', 'learning_rate': '3.393e-06', 'entropy': '0.7896', 'num_tokens': '2.428e+06', 'mean_token_accuracy': '0.7947', 'epoch': '0.09782'}
{'loss': '1.036', 'grad_norm': '0.4076', 'learning_rate': '3.386e-06', 'entropy': '1.043', 'num_tokens': '2.432e+06', 'mean_token_accuracy': '0.7443', 'epoch': '0.09783'}
{'loss': '0.5978', 'grad_norm': '0.3591', 'learning_rate': '3.379e-06', 'entropy': '0.6207', 'num_tokens': '2.437e+06', 'mean_token_accuracy': '0.8363', 'epoch': '0.09783'}
{'loss': '1.002', 'grad_norm': '0.4924', 'learning_rate': '3.372e-06', 'entropy': '1.034', 'num_tokens': '2.441e+06', 'mean_token_accuracy': '0.7486', 'epoch': '0.09784'}
{'loss': '0.6101', 'grad_norm': '0.3474', 'learning_rate': '3.365e-06', 'entropy': '0.6205', 'num_tokens': '2.447e+06', 'mean_token_accuracy': '0.8449', 'epoch': '0.09785'}
{'loss': '1.338', 'grad_norm': '0.471', 'learning_rate': '3.358e-06', 'entropy': '1.194', 'num_tokens': '2.451e+06', 'mean_token_accuracy': '0.7219', 'epoch': '0.09786'}
{'loss': '0.5552', 'grad_norm': '0.3833', 'learning_rate': '3.351e-06', 'entropy': '0.57', 'num_tokens': '2.457e+06', 'mean_token_accuracy': '0.8421', 'epoch': '0.09787'}
{'loss': '0.8259', 'grad_norm': '0.4387', 'learning_rate': '3.344e-06', 'entropy': '0.9643', 'num_tokens': '2.462e+06', 'mean_token_accuracy': '0.7703', 'epoch': '0.09788'}
{'loss': '0.7567', 'grad_norm': '0.4486', 'learning_rate': '3.336e-06', 'entropy': '0.7623', 'num_tokens': '2.466e+06', 'mean_token_accuracy': '0.8122', 'epoch': '0.09789'}
{'loss': '0.7721', 'grad_norm': '0.409', 'learning_rate': '3.329e-06', 'entropy': '0.7815', 'num_tokens': '2.471e+06', 'mean_token_accuracy': '0.8016', 'epoch': '0.0979'}
{'loss': '0.654', 'grad_norm': '0.3803', 'learning_rate': '3.322e-06', 'entropy': '0.6478', 'num_tokens': '2.476e+06', 'mean_token_accuracy': '0.8322', 'epoch': '0.09791'}
{'loss': '0.6145', 'grad_norm': '0.4022', 'learning_rate': '3.315e-06', 'entropy': '0.6308', 'num_tokens': '2.48e+06', 'mean_token_accuracy': '0.8359', 'epoch': '0.09791'}
{'loss': '0.6897', 'grad_norm': '0.3706', 'learning_rate': '3.308e-06', 'entropy': '0.7457', 'num_tokens': '2.485e+06', 'mean_token_accuracy': '0.809', 'epoch': '0.09792'}
{'loss': '0.7678', 'grad_norm': '0.3815', 'learning_rate': '3.301e-06', 'entropy': '0.8157', 'num_tokens': '2.49e+06', 'mean_token_accuracy': '0.8008', 'epoch': '0.09793'}
{'loss': '0.9231', 'grad_norm': '0.4317', 'learning_rate': '3.294e-06', 'entropy': '0.9208', 'num_tokens': '2.495e+06', 'mean_token_accuracy': '0.7699', 'epoch': '0.09794'}
{'loss': '0.7497', 'grad_norm': '0.4351', 'learning_rate': '3.287e-06', 'entropy': '0.762', 'num_tokens': '2.499e+06', 'mean_token_accuracy': '0.7928', 'epoch': '0.09795'}
{'loss': '0.7971', 'grad_norm': '0.4218', 'learning_rate': '3.28e-06', 'entropy': '0.8022', 'num_tokens': '2.505e+06', 'mean_token_accuracy': '0.7994', 'epoch': '0.09796'}
{'loss': '0.5935', 'grad_norm': '0.3748', 'learning_rate': '3.273e-06', 'entropy': '0.6057', 'num_tokens': '2.51e+06', 'mean_token_accuracy': '0.8505', 'epoch': '0.09797'}

 92%|█████████▏| 11071/12000 [16:39<25:19,  1.64s/it]
 92%|█████████▏| 11072/12000 [16:42<29:42,  1.92s/it]
                                                     

 92%|█████████▏| 11072/12000 [16:42<29:42,  1.92s/it]
 92%|█████████▏| 11073/12000 [16:43<26:58,  1.75s/it]
                                                     

 92%|█████████▏| 11073/12000 [16:43<26:58,  1.75s/it]
 92%|█████████▏| 11074/12000 [16:45<25:41,  1.66s/it]
                                                     

 92%|█████████▏| 11074/12000 [16:45<25:41,  1.66s/it]
 92%|█████████▏| 11075/12000 [16:46<23:57,  1.55s/it]
                                                     

 92%|█████████▏| 11075/12000 [16:46<23:57,  1.55s/it]
 92%|█████████▏| 11076/12000 [16:48<24:33,  1.59s/it]
                                                     

 92%|█████████▏| 11076/12000 [16:48<24:33,  1.59s/it]
 92%|█████████▏| 11077/12000 [16:50<26:58,  1.75s/it]
                                                     

 92%|█████████▏| 11077/12000 [16:50<26:58,  1.75s/it]
 92%|█████████▏| 11078/12000 [16:52<27:48,  1.81s/it]
                                                     

 92%|█████████▏| 11078/12000 [16:52<27:48,  1.81s/it]
 92%|█████████▏| 11079/12000 [16:55<32:34,  2.12s/it]
                                                     

 92%|█████████▏| 11079/12000 [16:55<32:34,  2.12s/it]
 92%|█████████▏| 11080/12000 [16:57<31:53,  2.08s/it]
                                                     

 92%|█████████▏| 11080/12000 [16:57<31:53,  2.08s/it]
 92%|█████████▏| 11081/12000 [16:58<30:05,  1.96s/it]
                                                     

 92%|█████████▏| 11081/12000 [16:58<30:05,  1.96s/it]
 92%|█████████▏| 11082/12000 [17:01<31:26,  2.05s/it]
                                                     

 92%|█████████▏| 11082/12000 [17:01<31:26,  2.05s/it]
 92%|█████████▏| 11083/12000 [17:03<33:15,  2.18s/it]
                                                     

 92%|█████████▏| 11083/12000 [17:03<33:15,  2.18s/it]
 92%|█████████▏| 11084/12000 [17:07<39:39,  2.60s/it]
                                                     

 92%|█████████▏| 11084/12000 [17:07<39:39,  2.60s/it]
 92%|█████████▏| 11085/12000 [17:08<33:24,  2.19s/it]
                                                     

 92%|█████████▏| 11085/12000 [17:08<33:24,  2.19s/it]
 92%|█████████▏| 11086/12000 [17:10<30:58,  2.03s/it]
                                                     

 92%|█████████▏| 11086/12000 [17:10<30:58,  2.03s/it]
 92%|█████████▏| 11087/12000 [17:11<27:51,  1.83s/it]
                                                     

 92%|█████████▏| 11087/12000 [17:11<27:51,  1.83s/it]
 92%|█████████▏| 11088/12000 [17:13<29:24,  1.94s/it]
                                                     

 92%|█████████▏| 11088/12000 [17:13<29:24,  1.94s/it]
 92%|█████████▏| 11089/12000 [17:15<29:43,  1.96s/it]
                                                     

 92%|█████████▏| 11089/12000 [17:15<29:43,  1.96s/it]
 92%|█████████▏| 11090/12000 [17:17<27:47,  1.83s/it]
                                                     

 92%|█████████▏| 11090/12000 [17:17<27:47,  1.83s/it]
 92%|█████████▏| 11091/12000 [17:20<32:09,  2.12s/it]
                                                     

 92%|█████████▏| 11091/12000 [17:20<32:09,  2.12s/it]
 92%|█████████▏| 11092/12000 [17:22<34:42,  2.29s/it]
                                                     

 92%|█████████▏| 11092/12000 [17:22<34:42,  2.29s/it]
 92%|█████████▏| 11093/12000 [17:25<35:31,  2.35s/it]
                                                     

 92%|█████████▏| 11093/12000 [17:25<35:31,  2.35s/it]
 92%|█████████▏| 11094/12000 [17:26<32:09,  2.13s/it]
                                                     

 92%|█████████▏| 11094/12000 [17:26<32:09,  2.13s/it]
 92%|█████████▏| 11095/12000 [17:29<35:47,  2.37s/it]
                                                     

 92%|█████████▏| 11095/12000 [17:29<35:47,  2.37s/it]
 92%|█████████▏| 11096/12000 [17:32<35:30,  2.36s/it]
                                                     

 92%|█████████▏| 11096/12000 [17:32<35:30,  2.36s/it]
 92%|█████████▏| 11097/12000 [17:33<32:13,  2.14s/it]
                                                     

 92%|█████████▏| 11097/12000 [17:33<32:13,  2.14s/it]
 92%|█████████▏| 11098/12000 [17:35<29:08,  1.94s/it]
                                                     

 92%|█████████▏| 11098/12000 [17:35<29:08,  1.94s/it]
 92%|█████████▏| 11099/12000 [17:36<26:41,  1.78s/it]
                                                     

 92%|█████████▏| 11099/12000 [17:36<26:41,  1.78s/it]
 92%|█████████▎| 11100/12000 [17:38<29:03,  1.94s/it]
                                                     

 92%|█████████▎| 11100/12000 [17:38<29:03,  1.94s/it]
 93%|█████████▎| 11101/12000 [17:42<34:40,  2.31s/it]
                                                     

 93%|█████████▎| 11101/12000 [17:42<34:40,  2.31s/it]
 93%|█████████▎| 11102/12000 [17:44<37:08,  2.48s/it]
                                                     

 93%|█████████▎| 11102/12000 [17:44<37:08,  2.48s/it]
 93%|█████████▎| 11103/12000 [17:46<34:46,  2.33s/it]
                                                     

 93%|█████████▎| 11103/12000 [17:46<34:46,  2.33s/it]
 93%|█████████▎| 11104/12000 [17:48<30:32,  2.05s/it]
                                                     

 93%|█████████▎| 11104/12000 [17:48<30:32,  2.05s/it]
 93%|█████████▎| 11105/12000 [17:53<45:36,  3.06s/it]
                                                     

 93%|█████████▎| 11105/12000 [17:53<45:36,  3.06s/it]
 93%|█████████▎| 11106/12000 [17:55<38:11,  2.56s/it]
                                                     

 93%|█████████▎| 11106/12000 [17:55<38:11,  2.56s/it]
 93%|█████████▎| 11107/12000 [17:58<40:47,  2.74s/it]
                                                     

 93%|█████████▎| 11107/12000 [17:58<40:47,  2.74s/it]
 93%|█████████▎| 11108/12000 [17:59<36:03,  2.43s/it]
                                                     

 93%|█████████▎| 11108/12000 [17:59<36:03,  2.43s/it]
 93%|█████████▎| 11109/12000 [18:01<31:12,  2.10s/it]
                                                     

 93%|█████████▎| 11109/12000 [18:01<31:12,  2.10s/it]
 93%|█████████▎| 11110/12000 [18:02<28:52,  1.95s/it]
                                                     

 93%|█████████▎| 11110/12000 [18:02<28:52,  1.95s/it]
 93%|█████████▎| 11111/12000 [18:04<25:57,  1.75s/it]
                                                     

 93%|█████████▎| 11111/12000 [18:04<25:57,  1.75s/it]
 93%|█████████▎| 11112/12000 [18:05<25:26,  1.72s/it]
                                                     

 93%|█████████▎| 11112/12000 [18:05<25:26,  1.72s/it]
 93%|█████████▎| 11113/12000 [18:07<24:23,  1.65s/it]
                                                     

 93%|█████████▎| 11113/12000 [18:07<24:23,  1.65s/it]
 93%|█████████▎| 11114/12000 [18:08<23:28,  1.59s/it]
                                                     

 93%|█████████▎| 11114/12000 [18:08<23:28,  1.59s/it]
 93%|█████████▎| 11115/12000 [18:10<23:31,  1.59s/it]
                                                     

 93%|█████████▎| 11115/12000 [18:10<23:31,  1.59s/it]
 93%|█████████▎| 11116/12000 [18:12<23:45,  1.61s/it]
                                                     

 93%|█████████▎| 11116/12000 [18:12<23:45,  1.61s/it]
 93%|█████████▎| 11117/12000 [18:14<28:48,  1.96s/it]
                                                     

 93%|█████████▎| 11117/12000 [18:14<28:48,  1.96s/it]
 93%|█████████▎| 11118/12000 [18:16<27:50,  1.89s/it]
                                                     
{'loss': '0.7096', 'grad_norm': '0.3732', 'learning_rate': '3.266e-06', 'entropy': '0.7429', 'num_tokens': '2.514e+06', 'mean_token_accuracy': '0.8123', 'epoch': '0.09798'}
{'loss': '0.5913', 'grad_norm': '0.3744', 'learning_rate': '3.259e-06', 'entropy': '0.6716', 'num_tokens': '2.52e+06', 'mean_token_accuracy': '0.8205', 'epoch': '0.09799'}
{'loss': '0.6732', 'grad_norm': '0.3593', 'learning_rate': '3.252e-06', 'entropy': '0.6611', 'num_tokens': '2.524e+06', 'mean_token_accuracy': '0.8241', 'epoch': '0.09799'}
{'loss': '0.8434', 'grad_norm': '0.4201', 'learning_rate': '3.245e-06', 'entropy': '0.8517', 'num_tokens': '2.528e+06', 'mean_token_accuracy': '0.7929', 'epoch': '0.098'}
{'loss': '0.8195', 'grad_norm': '0.4341', 'learning_rate': '3.238e-06', 'entropy': '0.8363', 'num_tokens': '2.532e+06', 'mean_token_accuracy': '0.789', 'epoch': '0.09801'}
{'loss': '0.6277', 'grad_norm': '0.3521', 'learning_rate': '3.231e-06', 'entropy': '0.6746', 'num_tokens': '2.536e+06', 'mean_token_accuracy': '0.8252', 'epoch': '0.09802'}
{'loss': '0.7231', 'grad_norm': '0.3335', 'learning_rate': '3.224e-06', 'entropy': '0.7698', 'num_tokens': '2.542e+06', 'mean_token_accuracy': '0.8092', 'epoch': '0.09803'}
{'loss': '0.7227', 'grad_norm': '0.3791', 'learning_rate': '3.218e-06', 'entropy': '0.6803', 'num_tokens': '2.548e+06', 'mean_token_accuracy': '0.8304', 'epoch': '0.09804'}
{'loss': '1.163', 'grad_norm': '0.4156', 'learning_rate': '3.211e-06', 'entropy': '1.084', 'num_tokens': '2.555e+06', 'mean_token_accuracy': '0.7352', 'epoch': '0.09805'}
{'loss': '0.7539', 'grad_norm': '0.4006', 'learning_rate': '3.204e-06', 'entropy': '0.7556', 'num_tokens': '2.561e+06', 'mean_token_accuracy': '0.8099', 'epoch': '0.09806'}
{'loss': '0.8117', 'grad_norm': '0.4494', 'learning_rate': '3.197e-06', 'entropy': '0.8451', 'num_tokens': '2.567e+06', 'mean_token_accuracy': '0.7962', 'epoch': '0.09806'}
{'loss': '1.031', 'grad_norm': '0.5056', 'learning_rate': '3.19e-06', 'entropy': '1.024', 'num_tokens': '2.573e+06', 'mean_token_accuracy': '0.7392', 'epoch': '0.09807'}
{'loss': '0.6411', 'grad_norm': '0.345', 'learning_rate': '3.183e-06', 'entropy': '0.7544', 'num_tokens': '2.578e+06', 'mean_token_accuracy': '0.8066', 'epoch': '0.09808'}
{'loss': '0.8212', 'grad_norm': '0.4375', 'learning_rate': '3.176e-06', 'entropy': '0.8689', 'num_tokens': '2.584e+06', 'mean_token_accuracy': '0.7882', 'epoch': '0.09809'}
{'loss': '0.8016', 'grad_norm': '0.3396', 'learning_rate': '3.169e-06', 'entropy': '0.8539', 'num_tokens': '2.587e+06', 'mean_token_accuracy': '0.7621', 'epoch': '0.0981'}
{'loss': '0.6672', 'grad_norm': '0.3579', 'learning_rate': '3.162e-06', 'entropy': '0.6386', 'num_tokens': '2.592e+06', 'mean_token_accuracy': '0.826', 'epoch': '0.09811'}
{'loss': '0.7708', 'grad_norm': '0.3709', 'learning_rate': '3.155e-06', 'entropy': '0.858', 'num_tokens': '2.596e+06', 'mean_token_accuracy': '0.7755', 'epoch': '0.09812'}
{'loss': '0.7587', 'grad_norm': '0.4917', 'learning_rate': '3.149e-06', 'entropy': '0.7163', 'num_tokens': '2.603e+06', 'mean_token_accuracy': '0.8032', 'epoch': '0.09813'}
{'loss': '0.9594', 'grad_norm': '0.4239', 'learning_rate': '3.142e-06', 'entropy': '0.8599', 'num_tokens': '2.609e+06', 'mean_token_accuracy': '0.7865', 'epoch': '0.09814'}
{'loss': '0.6298', 'grad_norm': '0.4208', 'learning_rate': '3.135e-06', 'entropy': '0.6758', 'num_tokens': '2.614e+06', 'mean_token_accuracy': '0.8135', 'epoch': '0.09814'}
{'loss': '1.338', 'grad_norm': '0.5605', 'learning_rate': '3.128e-06', 'entropy': '1.236', 'num_tokens': '2.62e+06', 'mean_token_accuracy': '0.6974', 'epoch': '0.09815'}
{'loss': '0.6405', 'grad_norm': '0.3854', 'learning_rate': '3.121e-06', 'entropy': '0.6811', 'num_tokens': '2.627e+06', 'mean_token_accuracy': '0.8215', 'epoch': '0.09816'}
{'loss': '0.893', 'grad_norm': '0.4474', 'learning_rate': '3.114e-06', 'entropy': '1.014', 'num_tokens': '2.633e+06', 'mean_token_accuracy': '0.7639', 'epoch': '0.09817'}
{'loss': '0.5137', 'grad_norm': '0.2793', 'learning_rate': '3.108e-06', 'entropy': '0.6103', 'num_tokens': '2.638e+06', 'mean_token_accuracy': '0.8344', 'epoch': '0.09818'}
{'loss': '0.8633', 'grad_norm': '0.3438', 'learning_rate': '3.101e-06', 'entropy': '0.678', 'num_tokens': '2.645e+06', 'mean_token_accuracy': '0.8283', 'epoch': '0.09819'}
{'loss': '0.8329', 'grad_norm': '0.4373', 'learning_rate': '3.094e-06', 'entropy': '0.8262', 'num_tokens': '2.651e+06', 'mean_token_accuracy': '0.7955', 'epoch': '0.0982'}
{'loss': '0.5171', 'grad_norm': '0.4137', 'learning_rate': '3.087e-06', 'entropy': '0.5486', 'num_tokens': '2.655e+06', 'mean_token_accuracy': '0.8621', 'epoch': '0.09821'}
{'loss': '0.5432', 'grad_norm': '0.3979', 'learning_rate': '3.08e-06', 'entropy': '0.5499', 'num_tokens': '2.659e+06', 'mean_token_accuracy': '0.8589', 'epoch': '0.09822'}
{'loss': '0.8702', 'grad_norm': '0.3943', 'learning_rate': '3.074e-06', 'entropy': '0.8992', 'num_tokens': '2.663e+06', 'mean_token_accuracy': '0.7713', 'epoch': '0.09822'}
{'loss': '0.7636', 'grad_norm': '0.3566', 'learning_rate': '3.067e-06', 'entropy': '0.8515', 'num_tokens': '2.67e+06', 'mean_token_accuracy': '0.7846', 'epoch': '0.09823'}
{'loss': '1.125', 'grad_norm': '0.4505', 'learning_rate': '3.06e-06', 'entropy': '1.113', 'num_tokens': '2.675e+06', 'mean_token_accuracy': '0.7345', 'epoch': '0.09824'}
{'loss': '0.85', 'grad_norm': '0.3814', 'learning_rate': '3.053e-06', 'entropy': '0.7689', 'num_tokens': '2.682e+06', 'mean_token_accuracy': '0.7961', 'epoch': '0.09825'}
{'loss': '1.064', 'grad_norm': '0.4838', 'learning_rate': '3.046e-06', 'entropy': '1.056', 'num_tokens': '2.688e+06', 'mean_token_accuracy': '0.7449', 'epoch': '0.09826'}
{'loss': '0.9028', 'grad_norm': '0.4442', 'learning_rate': '3.04e-06', 'entropy': '0.8939', 'num_tokens': '2.693e+06', 'mean_token_accuracy': '0.7772', 'epoch': '0.09827'}
{'loss': '0.3918', 'grad_norm': '0.2319', 'learning_rate': '3.033e-06', 'entropy': '0.4355', 'num_tokens': '2.701e+06', 'mean_token_accuracy': '0.8851', 'epoch': '0.09828'}
{'loss': '0.6767', 'grad_norm': '0.3683', 'learning_rate': '3.026e-06', 'entropy': '0.6942', 'num_tokens': '2.705e+06', 'mean_token_accuracy': '0.8148', 'epoch': '0.09829'}
{'loss': '0.6681', 'grad_norm': '0.3552', 'learning_rate': '3.02e-06', 'entropy': '0.7221', 'num_tokens': '2.713e+06', 'mean_token_accuracy': '0.8177', 'epoch': '0.0983'}
{'loss': '0.7769', 'grad_norm': '0.4769', 'learning_rate': '3.013e-06', 'entropy': '0.8327', 'num_tokens': '2.719e+06', 'mean_token_accuracy': '0.7864', 'epoch': '0.0983'}
{'loss': '0.6519', 'grad_norm': '0.3495', 'learning_rate': '3.006e-06', 'entropy': '0.6531', 'num_tokens': '2.723e+06', 'mean_token_accuracy': '0.8347', 'epoch': '0.09831'}
{'loss': '0.5627', 'grad_norm': '0.3417', 'learning_rate': '2.999e-06', 'entropy': '0.5781', 'num_tokens': '2.729e+06', 'mean_token_accuracy': '0.8529', 'epoch': '0.09832'}
{'loss': '0.6482', 'grad_norm': '0.3457', 'learning_rate': '2.993e-06', 'entropy': '0.6786', 'num_tokens': '2.733e+06', 'mean_token_accuracy': '0.814', 'epoch': '0.09833'}
{'loss': '0.4667', 'grad_norm': '0.3224', 'learning_rate': '2.986e-06', 'entropy': '0.5296', 'num_tokens': '2.738e+06', 'mean_token_accuracy': '0.8614', 'epoch': '0.09834'}
{'loss': '0.8545', 'grad_norm': '0.5253', 'learning_rate': '2.979e-06', 'entropy': '0.8587', 'num_tokens': '2.742e+06', 'mean_token_accuracy': '0.7762', 'epoch': '0.09835'}
{'loss': '0.7691', 'grad_norm': '0.4102', 'learning_rate': '2.973e-06', 'entropy': '0.8083', 'num_tokens': '2.746e+06', 'mean_token_accuracy': '0.7964', 'epoch': '0.09836'}
{'loss': '1.089', 'grad_norm': '0.4479', 'learning_rate': '2.966e-06', 'entropy': '1.074', 'num_tokens': '2.75e+06', 'mean_token_accuracy': '0.7512', 'epoch': '0.09837'}
{'loss': '0.5737', 'grad_norm': '0.4038', 'learning_rate': '2.959e-06', 'entropy': '0.5769', 'num_tokens': '2.755e+06', 'mean_token_accuracy': '0.8454', 'epoch': '0.09837'}
{'loss': '0.651', 'grad_norm': '0.3248', 'learning_rate': '2.953e-06', 'entropy': '0.6087', 'num_tokens': '2.763e+06', 'mean_token_accuracy': '0.8458', 'epoch': '0.09838'}

 93%|█████████▎| 11118/12000 [18:16<27:50,  1.89s/it]
 93%|█████████▎| 11119/12000 [18:18<26:49,  1.83s/it]
                                                     

 93%|█████████▎| 11119/12000 [18:18<26:49,  1.83s/it]
 93%|█████████▎| 11120/12000 [18:20<27:48,  1.90s/it]
                                                     

 93%|█████████▎| 11120/12000 [18:20<27:48,  1.90s/it]
 93%|█████████▎| 11121/12000 [18:21<26:49,  1.83s/it]
                                                     

 93%|█████████▎| 11121/12000 [18:21<26:49,  1.83s/it]
 93%|█████████▎| 11122/12000 [18:23<25:32,  1.75s/it]
                                                     

 93%|█████████▎| 11122/12000 [18:23<25:32,  1.75s/it]
 93%|█████████▎| 11123/12000 [18:25<26:21,  1.80s/it]
                                                     

 93%|█████████▎| 11123/12000 [18:25<26:21,  1.80s/it]
 93%|█████████▎| 11124/12000 [18:27<28:46,  1.97s/it]
                                                     

 93%|█████████▎| 11124/12000 [18:27<28:46,  1.97s/it]
 93%|█████████▎| 11125/12000 [18:29<27:31,  1.89s/it]
                                                     

 93%|█████████▎| 11125/12000 [18:29<27:31,  1.89s/it]
 93%|█████████▎| 11126/12000 [18:32<31:15,  2.15s/it]
                                                     

 93%|█████████▎| 11126/12000 [18:32<31:15,  2.15s/it]
 93%|█████████▎| 11127/12000 [18:36<40:20,  2.77s/it]
                                                     

 93%|█████████▎| 11127/12000 [18:36<40:20,  2.77s/it]
 93%|█████████▎| 11128/12000 [18:37<34:32,  2.38s/it]
                                                     

 93%|█████████▎| 11128/12000 [18:37<34:32,  2.38s/it]
 93%|█████████▎| 11129/12000 [18:39<30:27,  2.10s/it]
                                                     

 93%|█████████▎| 11129/12000 [18:39<30:27,  2.10s/it]
 93%|█████████▎| 11130/12000 [18:41<28:59,  2.00s/it]
                                                     

 93%|█████████▎| 11130/12000 [18:41<28:59,  2.00s/it]
 93%|█████████▎| 11131/12000 [18:42<26:55,  1.86s/it]
                                                     

 93%|█████████▎| 11131/12000 [18:42<26:55,  1.86s/it]
 93%|█████████▎| 11132/12000 [18:47<39:29,  2.73s/it]
                                                     

 93%|█████████▎| 11132/12000 [18:47<39:29,  2.73s/it]
 93%|█████████▎| 11133/12000 [18:48<33:40,  2.33s/it]
                                                     

 93%|█████████▎| 11133/12000 [18:48<33:40,  2.33s/it]
 93%|█████████▎| 11134/12000 [18:50<31:01,  2.15s/it]
                                                     

 93%|█████████▎| 11134/12000 [18:50<31:01,  2.15s/it]
 93%|█████████▎| 11135/12000 [18:54<39:39,  2.75s/it]
                                                     

 93%|█████████▎| 11135/12000 [18:54<39:39,  2.75s/it]
 93%|█████████▎| 11136/12000 [18:56<35:12,  2.44s/it]
                                                     

 93%|█████████▎| 11136/12000 [18:56<35:12,  2.44s/it]
 93%|█████████▎| 11137/12000 [18:58<31:14,  2.17s/it]
                                                     

 93%|█████████▎| 11137/12000 [18:58<31:14,  2.17s/it]
 93%|█████████▎| 11138/12000 [19:00<30:27,  2.12s/it]
                                                     

 93%|█████████▎| 11138/12000 [19:00<30:27,  2.12s/it]
 93%|█████████▎| 11139/12000 [19:02<30:49,  2.15s/it]
                                                     

 93%|█████████▎| 11139/12000 [19:02<30:49,  2.15s/it]
 93%|█████████▎| 11140/12000 [19:05<35:38,  2.49s/it]
                                                     

 93%|█████████▎| 11140/12000 [19:05<35:38,  2.49s/it]
 93%|█████████▎| 11141/12000 [19:07<32:41,  2.28s/it]
                                                     

 93%|█████████▎| 11141/12000 [19:07<32:41,  2.28s/it]
 93%|█████████▎| 11142/12000 [19:08<29:09,  2.04s/it]
                                                     

 93%|█████████▎| 11142/12000 [19:08<29:09,  2.04s/it]
 93%|█████████▎| 11143/12000 [19:10<27:49,  1.95s/it]
                                                     

 93%|█████████▎| 11143/12000 [19:10<27:49,  1.95s/it]
 93%|█████████▎| 11144/12000 [19:12<26:39,  1.87s/it]
                                                     

 93%|█████████▎| 11144/12000 [19:12<26:39,  1.87s/it]
 93%|█████████▎| 11145/12000 [19:13<24:19,  1.71s/it]
                                                     

 93%|█████████▎| 11145/12000 [19:13<24:19,  1.71s/it]
 93%|█████████▎| 11146/12000 [19:15<23:29,  1.65s/it]
                                                     

 93%|█████████▎| 11146/12000 [19:15<23:29,  1.65s/it]
 93%|█████████▎| 11147/12000 [19:17<28:31,  2.01s/it]
                                                     

 93%|█████████▎| 11147/12000 [19:17<28:31,  2.01s/it]
 93%|█████████▎| 11148/12000 [19:20<29:15,  2.06s/it]
                                                     

 93%|█████████▎| 11148/12000 [19:20<29:15,  2.06s/it]
 93%|█████████▎| 11149/12000 [19:24<39:20,  2.77s/it]
                                                     

 93%|█████████▎| 11149/12000 [19:24<39:20,  2.77s/it]
 93%|█████████▎| 11150/12000 [19:26<34:33,  2.44s/it]
                                                     

 93%|█████████▎| 11150/12000 [19:26<34:33,  2.44s/it]
 93%|█████████▎| 11151/12000 [19:29<39:10,  2.77s/it]
                                                     

 93%|█████████▎| 11151/12000 [19:29<39:10,  2.77s/it]
 93%|█████████▎| 11152/12000 [19:32<40:42,  2.88s/it]
                                                     

 93%|█████████▎| 11152/12000 [19:32<40:42,  2.88s/it]
 93%|█████████▎| 11153/12000 [19:35<37:43,  2.67s/it]
                                                     

 93%|█████████▎| 11153/12000 [19:35<37:43,  2.67s/it]
 93%|█████████▎| 11154/12000 [19:36<32:42,  2.32s/it]
                                                     

 93%|█████████▎| 11154/12000 [19:36<32:42,  2.32s/it]
 93%|█████████▎| 11155/12000 [19:38<29:41,  2.11s/it]
                                                     

 93%|█████████▎| 11155/12000 [19:38<29:41,  2.11s/it]
 93%|█████████▎| 11156/12000 [19:39<26:50,  1.91s/it]
                                                     

 93%|█████████▎| 11156/12000 [19:39<26:50,  1.91s/it]
 93%|█████████▎| 11157/12000 [19:41<24:50,  1.77s/it]
                                                     

 93%|█████████▎| 11157/12000 [19:41<24:50,  1.77s/it]
 93%|█████████▎| 11158/12000 [19:42<24:13,  1.73s/it]
                                                     

 93%|█████████▎| 11158/12000 [19:42<24:13,  1.73s/it]
 93%|█████████▎| 11159/12000 [19:44<25:02,  1.79s/it]
                                                     

 93%|█████████▎| 11159/12000 [19:44<25:02,  1.79s/it]
 93%|█████████▎| 11160/12000 [19:46<25:01,  1.79s/it]
                                                     

 93%|█████████▎| 11160/12000 [19:46<25:01,  1.79s/it]
 93%|█████████▎| 11161/12000 [19:49<30:35,  2.19s/it]
                                                     

 93%|█████████▎| 11161/12000 [19:49<30:35,  2.19s/it]
 93%|█████████▎| 11162/12000 [19:50<27:34,  1.97s/it]
                                                     

 93%|█████████▎| 11162/12000 [19:50<27:34,  1.97s/it]
 93%|█████████▎| 11163/12000 [19:53<31:41,  2.27s/it]
                                                     

 93%|█████████▎| 11163/12000 [19:53<31:41,  2.27s/it]
 93%|█████████▎| 11164/12000 [19:55<28:20,  2.03s/it]
                                                     

 93%|█████████▎| 11164/12000 [19:55<28:20,  2.03s/it]
 93%|█████████▎| 11165/12000 [19:56<25:20,  1.82s/it]
                                                     
{'loss': '0.7056', 'grad_norm': '0.4535', 'learning_rate': '2.946e-06', 'entropy': '0.7228', 'num_tokens': '2.768e+06', 'mean_token_accuracy': '0.8051', 'epoch': '0.09839'}
{'loss': '0.6602', 'grad_norm': '0.4523', 'learning_rate': '2.939e-06', 'entropy': '0.6737', 'num_tokens': '2.773e+06', 'mean_token_accuracy': '0.8136', 'epoch': '0.0984'}
{'loss': '1.065', 'grad_norm': '0.6641', 'learning_rate': '2.933e-06', 'entropy': '0.9691', 'num_tokens': '2.777e+06', 'mean_token_accuracy': '0.7539', 'epoch': '0.09841'}
{'loss': '1.114', 'grad_norm': '0.5484', 'learning_rate': '2.926e-06', 'entropy': '1.085', 'num_tokens': '2.781e+06', 'mean_token_accuracy': '0.7551', 'epoch': '0.09842'}
{'loss': '0.8771', 'grad_norm': '0.3508', 'learning_rate': '2.92e-06', 'entropy': '0.831', 'num_tokens': '2.786e+06', 'mean_token_accuracy': '0.797', 'epoch': '0.09843'}
{'loss': '0.8027', 'grad_norm': '0.4373', 'learning_rate': '2.913e-06', 'entropy': '0.7957', 'num_tokens': '2.792e+06', 'mean_token_accuracy': '0.7938', 'epoch': '0.09844'}
{'loss': '0.8918', 'grad_norm': '0.4267', 'learning_rate': '2.906e-06', 'entropy': '0.9278', 'num_tokens': '2.797e+06', 'mean_token_accuracy': '0.7665', 'epoch': '0.09845'}
{'loss': '0.7657', 'grad_norm': '0.4213', 'learning_rate': '2.9e-06', 'entropy': '0.7344', 'num_tokens': '2.802e+06', 'mean_token_accuracy': '0.8102', 'epoch': '0.09845'}
{'loss': '0.596', 'grad_norm': '0.3799', 'learning_rate': '2.893e-06', 'entropy': '0.6227', 'num_tokens': '2.809e+06', 'mean_token_accuracy': '0.8407', 'epoch': '0.09846'}
{'loss': '0.5729', 'grad_norm': '0.3563', 'learning_rate': '2.887e-06', 'entropy': '0.6073', 'num_tokens': '2.816e+06', 'mean_token_accuracy': '0.8504', 'epoch': '0.09847'}
{'loss': '0.8748', 'grad_norm': '0.3965', 'learning_rate': '2.88e-06', 'entropy': '0.8457', 'num_tokens': '2.819e+06', 'mean_token_accuracy': '0.7828', 'epoch': '0.09848'}
{'loss': '0.7016', 'grad_norm': '0.4481', 'learning_rate': '2.873e-06', 'entropy': '0.7284', 'num_tokens': '2.824e+06', 'mean_token_accuracy': '0.8106', 'epoch': '0.09849'}
{'loss': '0.7221', 'grad_norm': '0.4698', 'learning_rate': '2.867e-06', 'entropy': '0.7868', 'num_tokens': '2.828e+06', 'mean_token_accuracy': '0.8038', 'epoch': '0.0985'}
{'loss': '0.672', 'grad_norm': '0.3608', 'learning_rate': '2.86e-06', 'entropy': '0.6449', 'num_tokens': '2.833e+06', 'mean_token_accuracy': '0.8326', 'epoch': '0.09851'}
{'loss': '1.097', 'grad_norm': '0.4117', 'learning_rate': '2.854e-06', 'entropy': '1.025', 'num_tokens': '2.842e+06', 'mean_token_accuracy': '0.7499', 'epoch': '0.09852'}
{'loss': '0.9146', 'grad_norm': '0.3892', 'learning_rate': '2.847e-06', 'entropy': '0.9009', 'num_tokens': '2.847e+06', 'mean_token_accuracy': '0.7709', 'epoch': '0.09853'}
{'loss': '0.6705', 'grad_norm': '0.3931', 'learning_rate': '2.841e-06', 'entropy': '0.7679', 'num_tokens': '2.852e+06', 'mean_token_accuracy': '0.8088', 'epoch': '0.09853'}
{'loss': '0.9737', 'grad_norm': '0.3324', 'learning_rate': '2.834e-06', 'entropy': '1.123', 'num_tokens': '2.86e+06', 'mean_token_accuracy': '0.7251', 'epoch': '0.09854'}
{'loss': '0.7412', 'grad_norm': '0.3659', 'learning_rate': '2.828e-06', 'entropy': '0.7545', 'num_tokens': '2.865e+06', 'mean_token_accuracy': '0.803', 'epoch': '0.09855'}
{'loss': '0.4747', 'grad_norm': '0.3059', 'learning_rate': '2.821e-06', 'entropy': '0.4913', 'num_tokens': '2.87e+06', 'mean_token_accuracy': '0.8714', 'epoch': '0.09856'}
{'loss': '0.8337', 'grad_norm': '0.3744', 'learning_rate': '2.815e-06', 'entropy': '0.8283', 'num_tokens': '2.876e+06', 'mean_token_accuracy': '0.8054', 'epoch': '0.09857'}
{'loss': '1.006', 'grad_norm': '0.3567', 'learning_rate': '2.808e-06', 'entropy': '0.9654', 'num_tokens': '2.883e+06', 'mean_token_accuracy': '0.7636', 'epoch': '0.09858'}
{'loss': '0.6168', 'grad_norm': '0.2869', 'learning_rate': '2.802e-06', 'entropy': '0.6202', 'num_tokens': '2.89e+06', 'mean_token_accuracy': '0.8304', 'epoch': '0.09859'}
{'loss': '0.6619', 'grad_norm': '0.4667', 'learning_rate': '2.795e-06', 'entropy': '0.7662', 'num_tokens': '2.895e+06', 'mean_token_accuracy': '0.8067', 'epoch': '0.0986'}
{'loss': '0.622', 'grad_norm': '0.3616', 'learning_rate': '2.789e-06', 'entropy': '0.6346', 'num_tokens': '2.9e+06', 'mean_token_accuracy': '0.836', 'epoch': '0.0986'}
{'loss': '1.249', 'grad_norm': '0.4004', 'learning_rate': '2.782e-06', 'entropy': '1.016', 'num_tokens': '2.904e+06', 'mean_token_accuracy': '0.748', 'epoch': '0.09861'}
{'loss': '0.6223', 'grad_norm': '0.3863', 'learning_rate': '2.776e-06', 'entropy': '0.614', 'num_tokens': '2.91e+06', 'mean_token_accuracy': '0.8287', 'epoch': '0.09862'}
{'loss': '0.9261', 'grad_norm': '0.4956', 'learning_rate': '2.769e-06', 'entropy': '0.9618', 'num_tokens': '2.913e+06', 'mean_token_accuracy': '0.7743', 'epoch': '0.09863'}
{'loss': '1.011', 'grad_norm': '0.5002', 'learning_rate': '2.763e-06', 'entropy': '1.068', 'num_tokens': '2.917e+06', 'mean_token_accuracy': '0.7524', 'epoch': '0.09864'}
{'loss': '0.6923', 'grad_norm': '0.4317', 'learning_rate': '2.757e-06', 'entropy': '0.6814', 'num_tokens': '2.924e+06', 'mean_token_accuracy': '0.8186', 'epoch': '0.09865'}
{'loss': '1.025', 'grad_norm': '0.3549', 'learning_rate': '2.75e-06', 'entropy': '1.071', 'num_tokens': '2.93e+06', 'mean_token_accuracy': '0.7253', 'epoch': '0.09866'}
{'loss': '1.164', 'grad_norm': '0.3812', 'learning_rate': '2.744e-06', 'entropy': '0.9728', 'num_tokens': '2.938e+06', 'mean_token_accuracy': '0.7646', 'epoch': '0.09867'}
{'loss': '0.947', 'grad_norm': '0.4501', 'learning_rate': '2.737e-06', 'entropy': '0.9185', 'num_tokens': '2.943e+06', 'mean_token_accuracy': '0.7875', 'epoch': '0.09868'}
{'loss': '0.8557', 'grad_norm': '0.4003', 'learning_rate': '2.731e-06', 'entropy': '0.8608', 'num_tokens': '2.949e+06', 'mean_token_accuracy': '0.7836', 'epoch': '0.09868'}
{'loss': '0.9272', 'grad_norm': '0.4133', 'learning_rate': '2.725e-06', 'entropy': '1.052', 'num_tokens': '2.956e+06', 'mean_token_accuracy': '0.7443', 'epoch': '0.09869'}
{'loss': '0.9323', 'grad_norm': '0.4613', 'learning_rate': '2.718e-06', 'entropy': '0.8549', 'num_tokens': '2.963e+06', 'mean_token_accuracy': '0.7953', 'epoch': '0.0987'}
{'loss': '0.6768', 'grad_norm': '0.4043', 'learning_rate': '2.712e-06', 'entropy': '0.6977', 'num_tokens': '2.968e+06', 'mean_token_accuracy': '0.8241', 'epoch': '0.09871'}
{'loss': '1.122', 'grad_norm': '0.5042', 'learning_rate': '2.705e-06', 'entropy': '1.138', 'num_tokens': '2.973e+06', 'mean_token_accuracy': '0.7382', 'epoch': '0.09872'}
{'loss': '0.618', 'grad_norm': '0.3575', 'learning_rate': '2.699e-06', 'entropy': '0.6552', 'num_tokens': '2.977e+06', 'mean_token_accuracy': '0.8285', 'epoch': '0.09873'}
{'loss': '0.71', 'grad_norm': '0.5042', 'learning_rate': '2.693e-06', 'entropy': '0.7353', 'num_tokens': '2.98e+06', 'mean_token_accuracy': '0.8124', 'epoch': '0.09874'}
{'loss': '0.9064', 'grad_norm': '0.377', 'learning_rate': '2.686e-06', 'entropy': '0.9742', 'num_tokens': '2.986e+06', 'mean_token_accuracy': '0.7759', 'epoch': '0.09875'}
{'loss': '0.8519', 'grad_norm': '0.3482', 'learning_rate': '2.68e-06', 'entropy': '0.8324', 'num_tokens': '2.991e+06', 'mean_token_accuracy': '0.7817', 'epoch': '0.09876'}
{'loss': '1.067', 'grad_norm': '0.4086', 'learning_rate': '2.674e-06', 'entropy': '1.119', 'num_tokens': '2.998e+06', 'mean_token_accuracy': '0.7424', 'epoch': '0.09876'}
{'loss': '1.029', 'grad_norm': '0.4385', 'learning_rate': '2.667e-06', 'entropy': '0.9439', 'num_tokens': '3.004e+06', 'mean_token_accuracy': '0.7611', 'epoch': '0.09877'}
{'loss': '0.4899', 'grad_norm': '0.35', 'learning_rate': '2.661e-06', 'entropy': '0.5042', 'num_tokens': '3.009e+06', 'mean_token_accuracy': '0.8639', 'epoch': '0.09878'}
{'loss': '0.803', 'grad_norm': '0.3212', 'learning_rate': '2.655e-06', 'entropy': '0.8799', 'num_tokens': '3.016e+06', 'mean_token_accuracy': '0.7889', 'epoch': '0.09879'}
{'loss': '0.7026', 'grad_norm': '0.555', 'learning_rate': '2.648e-06', 'entropy': '0.7438', 'num_tokens': '3.02e+06', 'mean_token_accuracy': '0.8015', 'epoch': '0.0988'}

 93%|█████████▎| 11165/12000 [19:56<25:20,  1.82s/it]
 93%|█████████▎| 11166/12000 [19:58<24:30,  1.76s/it]
                                                     

 93%|█████████▎| 11166/12000 [19:58<24:30,  1.76s/it]
 93%|█████████▎| 11167/12000 [20:00<27:28,  1.98s/it]
                                                     

 93%|█████████▎| 11167/12000 [20:00<27:28,  1.98s/it]
 93%|█████████▎| 11168/12000 [20:02<27:08,  1.96s/it]
                                                     

 93%|█████████▎| 11168/12000 [20:02<27:08,  1.96s/it]
 93%|█████████▎| 11169/12000 [20:04<25:08,  1.82s/it]
                                                     

 93%|█████████▎| 11169/12000 [20:04<25:08,  1.82s/it]
 93%|█████████▎| 11170/12000 [20:05<23:39,  1.71s/it]
                                                     

 93%|█████████▎| 11170/12000 [20:05<23:39,  1.71s/it]
 93%|█████████▎| 11171/12000 [20:09<31:30,  2.28s/it]
                                                     

 93%|█████████▎| 11171/12000 [20:09<31:30,  2.28s/it]
 93%|█████████▎| 11172/12000 [20:10<28:12,  2.04s/it]
                                                     

 93%|█████████▎| 11172/12000 [20:10<28:12,  2.04s/it]
 93%|█████████▎| 11173/12000 [20:12<26:58,  1.96s/it]
                                                     

 93%|█████████▎| 11173/12000 [20:12<26:58,  1.96s/it]
 93%|█████████▎| 11174/12000 [20:14<26:19,  1.91s/it]
                                                     

 93%|█████████▎| 11174/12000 [20:14<26:19,  1.91s/it]
 93%|█████████▎| 11175/12000 [20:16<26:13,  1.91s/it]
                                                     

 93%|█████████▎| 11175/12000 [20:16<26:13,  1.91s/it]
 93%|█████████▎| 11176/12000 [20:19<30:59,  2.26s/it]
                                                     

 93%|█████████▎| 11176/12000 [20:19<30:59,  2.26s/it]
 93%|█████████▎| 11177/12000 [20:21<31:17,  2.28s/it]
                                                     

 93%|█████████▎| 11177/12000 [20:21<31:17,  2.28s/it]
 93%|█████████▎| 11178/12000 [20:26<43:24,  3.17s/it]
                                                     

 93%|█████████▎| 11178/12000 [20:26<43:24,  3.17s/it]
 93%|█████████▎| 11179/12000 [20:28<36:31,  2.67s/it]
                                                     

 93%|█████████▎| 11179/12000 [20:28<36:31,  2.67s/it]
 93%|█████████▎| 11180/12000 [20:30<32:08,  2.35s/it]
                                                     

 93%|█████████▎| 11180/12000 [20:30<32:08,  2.35s/it]
 93%|█████████▎| 11181/12000 [20:31<30:15,  2.22s/it]
                                                     

 93%|█████████▎| 11181/12000 [20:31<30:15,  2.22s/it]
 93%|█████████▎| 11182/12000 [20:34<30:15,  2.22s/it]
                                                     

 93%|█████████▎| 11182/12000 [20:34<30:15,  2.22s/it]
 93%|█████████▎| 11183/12000 [20:36<30:51,  2.27s/it]
                                                     

 93%|█████████▎| 11183/12000 [20:36<30:51,  2.27s/it]
 93%|█████████▎| 11184/12000 [20:37<27:22,  2.01s/it]
                                                     

 93%|█████████▎| 11184/12000 [20:37<27:22,  2.01s/it]
 93%|█████████▎| 11185/12000 [20:39<26:01,  1.92s/it]
                                                     

 93%|█████████▎| 11185/12000 [20:39<26:01,  1.92s/it]
 93%|█████████▎| 11186/12000 [20:41<25:26,  1.88s/it]
                                                     

 93%|█████████▎| 11186/12000 [20:41<25:26,  1.88s/it]
 93%|█████████▎| 11187/12000 [20:43<24:53,  1.84s/it]
                                                     

 93%|█████████▎| 11187/12000 [20:43<24:53,  1.84s/it]
 93%|█████████▎| 11188/12000 [20:45<25:43,  1.90s/it]
                                                     

 93%|█████████▎| 11188/12000 [20:45<25:43,  1.90s/it]
 93%|█████████▎| 11189/12000 [20:47<25:33,  1.89s/it]
                                                     

 93%|█████████▎| 11189/12000 [20:47<25:33,  1.89s/it]
 93%|█████████▎| 11190/12000 [20:48<24:20,  1.80s/it]
                                                     

 93%|█████████▎| 11190/12000 [20:48<24:20,  1.80s/it]
 93%|█████████▎| 11191/12000 [20:50<23:45,  1.76s/it]
                                                     

 93%|█████████▎| 11191/12000 [20:50<23:45,  1.76s/it]
 93%|█████████▎| 11192/12000 [20:51<22:56,  1.70s/it]
                                                     

 93%|█████████▎| 11192/12000 [20:51<22:56,  1.70s/it]
 93%|█████████▎| 11193/12000 [20:56<33:05,  2.46s/it]
                                                     

 93%|█████████▎| 11193/12000 [20:56<33:05,  2.46s/it]
 93%|█████████▎| 11194/12000 [20:58<33:11,  2.47s/it]
                                                     

 93%|█████████▎| 11194/12000 [20:58<33:11,  2.47s/it]
 93%|█████████▎| 11195/12000 [21:02<37:50,  2.82s/it]
                                                     

 93%|█████████▎| 11195/12000 [21:02<37:50,  2.82s/it]
 93%|█████████▎| 11196/12000 [21:04<35:16,  2.63s/it]
                                                     

 93%|█████████▎| 11196/12000 [21:04<35:16,  2.63s/it]
 93%|█████████▎| 11197/12000 [21:06<31:04,  2.32s/it]
                                                     

 93%|█████████▎| 11197/12000 [21:06<31:04,  2.32s/it]
 93%|█████████▎| 11198/12000 [21:08<32:29,  2.43s/it]
                                                     

 93%|█████████▎| 11198/12000 [21:08<32:29,  2.43s/it]
 93%|█████████▎| 11199/12000 [21:10<31:09,  2.33s/it]
                                                     

 93%|█████████▎| 11199/12000 [21:10<31:09,  2.33s/it]
 93%|█████████▎| 11200/12000 [21:12<29:42,  2.23s/it]
                                                     

 93%|█████████▎| 11200/12000 [21:12<29:42,  2.23s/it]
 93%|█████████▎| 11201/12000 [21:16<34:47,  2.61s/it]
                                                     

 93%|█████████▎| 11201/12000 [21:16<34:47,  2.61s/it]
 93%|█████████▎| 11202/12000 [21:19<37:10,  2.80s/it]
                                                     

 93%|█████████▎| 11202/12000 [21:19<37:10,  2.80s/it]
 93%|█████████▎| 11203/12000 [21:21<35:28,  2.67s/it]
                                                     

 93%|█████████▎| 11203/12000 [21:21<35:28,  2.67s/it]
 93%|█████████▎| 11204/12000 [21:24<36:41,  2.77s/it]
                                                     

 93%|█████████▎| 11204/12000 [21:24<36:41,  2.77s/it]
 93%|█████████▎| 11205/12000 [21:26<32:50,  2.48s/it]
                                                     

 93%|█████████▎| 11205/12000 [21:26<32:50,  2.48s/it]
 93%|█████████▎| 11206/12000 [21:28<29:15,  2.21s/it]
                                                     

 93%|█████████▎| 11206/12000 [21:28<29:15,  2.21s/it]
 93%|█████████▎| 11207/12000 [21:29<26:18,  1.99s/it]
                                                     

 93%|█████████▎| 11207/12000 [21:29<26:18,  1.99s/it]
 93%|█████████▎| 11208/12000 [21:31<25:59,  1.97s/it]
                                                     

 93%|█████████▎| 11208/12000 [21:31<25:59,  1.97s/it]
 93%|█████████▎| 11209/12000 [21:34<28:59,  2.20s/it]
                                                     

 93%|█████████▎| 11209/12000 [21:34<28:59,  2.20s/it]
 93%|█████████▎| 11210/12000 [21:36<29:26,  2.24s/it]
                                                     

 93%|█████████▎| 11210/12000 [21:36<29:26,  2.24s/it]
 93%|█████████▎| 11211/12000 [21:38<26:56,  2.05s/it]
                                                     

 93%|█████████▎| 11211/12000 [21:38<26:56,  2.05s/it]
 93%|█████████▎| 11212/12000 [21:40<25:15,  1.92s/it]
                                                     
{'loss': '0.7924', 'grad_norm': '0.4468', 'learning_rate': '2.642e-06', 'entropy': '0.784', 'num_tokens': '3.023e+06', 'mean_token_accuracy': '0.7964', 'epoch': '0.09881'}
{'loss': '0.6087', 'grad_norm': '0.357', 'learning_rate': '2.636e-06', 'entropy': '0.6332', 'num_tokens': '3.028e+06', 'mean_token_accuracy': '0.8349', 'epoch': '0.09882'}
{'loss': '0.7586', 'grad_norm': '0.3389', 'learning_rate': '2.63e-06', 'entropy': '0.7702', 'num_tokens': '3.034e+06', 'mean_token_accuracy': '0.7977', 'epoch': '0.09883'}
{'loss': '0.5792', 'grad_norm': '0.356', 'learning_rate': '2.623e-06', 'entropy': '0.5795', 'num_tokens': '3.039e+06', 'mean_token_accuracy': '0.8559', 'epoch': '0.09883'}
{'loss': '0.7067', 'grad_norm': '0.3639', 'learning_rate': '2.617e-06', 'entropy': '0.7066', 'num_tokens': '3.044e+06', 'mean_token_accuracy': '0.8153', 'epoch': '0.09884'}
{'loss': '0.76', 'grad_norm': '0.4095', 'learning_rate': '2.611e-06', 'entropy': '0.8206', 'num_tokens': '3.048e+06', 'mean_token_accuracy': '0.786', 'epoch': '0.09885'}
{'loss': '0.6135', 'grad_norm': '0.3381', 'learning_rate': '2.604e-06', 'entropy': '0.6332', 'num_tokens': '3.055e+06', 'mean_token_accuracy': '0.8309', 'epoch': '0.09886'}
{'loss': '0.6459', 'grad_norm': '0.4018', 'learning_rate': '2.598e-06', 'entropy': '0.6762', 'num_tokens': '3.06e+06', 'mean_token_accuracy': '0.8214', 'epoch': '0.09887'}
{'loss': '1.289', 'grad_norm': '0.5828', 'learning_rate': '2.592e-06', 'entropy': '1.094', 'num_tokens': '3.064e+06', 'mean_token_accuracy': '0.7393', 'epoch': '0.09888'}
{'loss': '0.9321', 'grad_norm': '0.5271', 'learning_rate': '2.586e-06', 'entropy': '0.9752', 'num_tokens': '3.068e+06', 'mean_token_accuracy': '0.7578', 'epoch': '0.09889'}
{'loss': '1.106', 'grad_norm': '0.4248', 'learning_rate': '2.58e-06', 'entropy': '0.9672', 'num_tokens': '3.074e+06', 'mean_token_accuracy': '0.7758', 'epoch': '0.0989'}
{'loss': '0.8425', 'grad_norm': '0.451', 'learning_rate': '2.573e-06', 'entropy': '0.7962', 'num_tokens': '3.081e+06', 'mean_token_accuracy': '0.7988', 'epoch': '0.09891'}
{'loss': '0.8093', 'grad_norm': '0.4329', 'learning_rate': '2.567e-06', 'entropy': '0.9011', 'num_tokens': '3.087e+06', 'mean_token_accuracy': '0.783', 'epoch': '0.09891'}
{'loss': '1.422', 'grad_norm': '0.3707', 'learning_rate': '2.561e-06', 'entropy': '1.126', 'num_tokens': '3.095e+06', 'mean_token_accuracy': '0.7446', 'epoch': '0.09892'}
{'loss': '0.8167', 'grad_norm': '0.3551', 'learning_rate': '2.555e-06', 'entropy': '0.783', 'num_tokens': '3.1e+06', 'mean_token_accuracy': '0.7913', 'epoch': '0.09893'}
{'loss': '0.6815', 'grad_norm': '0.408', 'learning_rate': '2.549e-06', 'entropy': '0.712', 'num_tokens': '3.104e+06', 'mean_token_accuracy': '0.8241', 'epoch': '0.09894'}
{'loss': '0.9655', 'grad_norm': '0.4256', 'learning_rate': '2.542e-06', 'entropy': '0.9518', 'num_tokens': '3.11e+06', 'mean_token_accuracy': '0.7684', 'epoch': '0.09895'}
{'loss': '1.181', 'grad_norm': '0.3891', 'learning_rate': '2.536e-06', 'entropy': '1.122', 'num_tokens': '3.116e+06', 'mean_token_accuracy': '0.7347', 'epoch': '0.09896'}
{'loss': '1.015', 'grad_norm': '0.552', 'learning_rate': '2.53e-06', 'entropy': '0.8559', 'num_tokens': '3.121e+06', 'mean_token_accuracy': '0.7815', 'epoch': '0.09897'}
{'loss': '0.8412', 'grad_norm': '0.3807', 'learning_rate': '2.524e-06', 'entropy': '0.8718', 'num_tokens': '3.125e+06', 'mean_token_accuracy': '0.7746', 'epoch': '0.09898'}
{'loss': '0.8348', 'grad_norm': '0.4064', 'learning_rate': '2.518e-06', 'entropy': '0.8186', 'num_tokens': '3.13e+06', 'mean_token_accuracy': '0.7949', 'epoch': '0.09899'}
{'loss': '0.6088', 'grad_norm': '0.2941', 'learning_rate': '2.512e-06', 'entropy': '0.6269', 'num_tokens': '3.135e+06', 'mean_token_accuracy': '0.8496', 'epoch': '0.09899'}
{'loss': '0.9668', 'grad_norm': '0.554', 'learning_rate': '2.505e-06', 'entropy': '0.9657', 'num_tokens': '3.139e+06', 'mean_token_accuracy': '0.7529', 'epoch': '0.099'}
{'loss': '0.9646', 'grad_norm': '0.4159', 'learning_rate': '2.499e-06', 'entropy': '0.8977', 'num_tokens': '3.145e+06', 'mean_token_accuracy': '0.7878', 'epoch': '0.09901'}
{'loss': '0.8015', 'grad_norm': '0.4515', 'learning_rate': '2.493e-06', 'entropy': '0.7458', 'num_tokens': '3.151e+06', 'mean_token_accuracy': '0.8174', 'epoch': '0.09902'}
{'loss': '0.7459', 'grad_norm': '0.3798', 'learning_rate': '2.487e-06', 'entropy': '0.7164', 'num_tokens': '3.156e+06', 'mean_token_accuracy': '0.8248', 'epoch': '0.09903'}
{'loss': '0.7688', 'grad_norm': '0.4011', 'learning_rate': '2.481e-06', 'entropy': '0.7888', 'num_tokens': '3.161e+06', 'mean_token_accuracy': '0.7926', 'epoch': '0.09904'}
{'loss': '0.7268', 'grad_norm': '0.3762', 'learning_rate': '2.475e-06', 'entropy': '0.7951', 'num_tokens': '3.166e+06', 'mean_token_accuracy': '0.8006', 'epoch': '0.09905'}
{'loss': '0.8967', 'grad_norm': '0.4573', 'learning_rate': '2.469e-06', 'entropy': '0.7509', 'num_tokens': '3.171e+06', 'mean_token_accuracy': '0.8108', 'epoch': '0.09906'}
{'loss': '0.6187', 'grad_norm': '0.3732', 'learning_rate': '2.463e-06', 'entropy': '0.6522', 'num_tokens': '3.177e+06', 'mean_token_accuracy': '0.8233', 'epoch': '0.09907'}
{'loss': '0.7086', 'grad_norm': '0.3359', 'learning_rate': '2.457e-06', 'entropy': '0.8168', 'num_tokens': '3.185e+06', 'mean_token_accuracy': '0.7892', 'epoch': '0.09907'}
{'loss': '1.094', 'grad_norm': '0.3779', 'learning_rate': '2.451e-06', 'entropy': '0.9698', 'num_tokens': '3.191e+06', 'mean_token_accuracy': '0.7524', 'epoch': '0.09908'}
{'loss': '0.7372', 'grad_norm': '0.4105', 'learning_rate': '2.445e-06', 'entropy': '0.7812', 'num_tokens': '3.195e+06', 'mean_token_accuracy': '0.7906', 'epoch': '0.09909'}
{'loss': '0.6223', 'grad_norm': '0.4579', 'learning_rate': '2.438e-06', 'entropy': '0.6837', 'num_tokens': '3.2e+06', 'mean_token_accuracy': '0.8238', 'epoch': '0.0991'}
{'loss': '0.7271', 'grad_norm': '0.5046', 'learning_rate': '2.432e-06', 'entropy': '0.6779', 'num_tokens': '3.206e+06', 'mean_token_accuracy': '0.8173', 'epoch': '0.09911'}
{'loss': '0.76', 'grad_norm': '0.3532', 'learning_rate': '2.426e-06', 'entropy': '0.7588', 'num_tokens': '3.212e+06', 'mean_token_accuracy': '0.8126', 'epoch': '0.09912'}
{'loss': '0.643', 'grad_norm': '0.3559', 'learning_rate': '2.42e-06', 'entropy': '0.6511', 'num_tokens': '3.218e+06', 'mean_token_accuracy': '0.8283', 'epoch': '0.09913'}
{'loss': '0.7157', 'grad_norm': '0.4204', 'learning_rate': '2.414e-06', 'entropy': '0.7107', 'num_tokens': '3.225e+06', 'mean_token_accuracy': '0.8239', 'epoch': '0.09914'}
{'loss': '0.4693', 'grad_norm': '0.2727', 'learning_rate': '2.408e-06', 'entropy': '0.48', 'num_tokens': '3.231e+06', 'mean_token_accuracy': '0.8704', 'epoch': '0.09914'}
{'loss': '0.8181', 'grad_norm': '0.4424', 'learning_rate': '2.402e-06', 'entropy': '0.858', 'num_tokens': '3.239e+06', 'mean_token_accuracy': '0.7914', 'epoch': '0.09915'}
{'loss': '0.6784', 'grad_norm': '0.4533', 'learning_rate': '2.396e-06', 'entropy': '0.6725', 'num_tokens': '3.244e+06', 'mean_token_accuracy': '0.8193', 'epoch': '0.09916'}
{'loss': '0.5877', 'grad_norm': '0.3551', 'learning_rate': '2.39e-06', 'entropy': '0.6425', 'num_tokens': '3.25e+06', 'mean_token_accuracy': '0.83', 'epoch': '0.09917'}
{'loss': '0.8745', 'grad_norm': '0.4931', 'learning_rate': '2.384e-06', 'entropy': '0.913', 'num_tokens': '3.253e+06', 'mean_token_accuracy': '0.7705', 'epoch': '0.09918'}
{'loss': '0.5494', 'grad_norm': '0.4043', 'learning_rate': '2.378e-06', 'entropy': '0.5477', 'num_tokens': '3.259e+06', 'mean_token_accuracy': '0.8593', 'epoch': '0.09919'}
{'loss': '0.7761', 'grad_norm': '0.3553', 'learning_rate': '2.372e-06', 'entropy': '0.8062', 'num_tokens': '3.266e+06', 'mean_token_accuracy': '0.7888', 'epoch': '0.0992'}
{'loss': '0.9031', 'grad_norm': '0.4106', 'learning_rate': '2.366e-06', 'entropy': '0.887', 'num_tokens': '3.273e+06', 'mean_token_accuracy': '0.7926', 'epoch': '0.09921'}
{'loss': '0.8688', 'grad_norm': '0.4187', 'learning_rate': '2.36e-06', 'entropy': '0.8178', 'num_tokens': '3.277e+06', 'mean_token_accuracy': '0.7882', 'epoch': '0.09922'}

 93%|█████████▎| 11212/12000 [21:40<25:15,  1.92s/it]
 93%|█████████▎| 11213/12000 [21:41<23:49,  1.82s/it]
                                                     

 93%|█████████▎| 11213/12000 [21:41<23:49,  1.82s/it]
 93%|█████████▎| 11214/12000 [21:43<23:12,  1.77s/it]
                                                     

 93%|█████████▎| 11214/12000 [21:43<23:12,  1.77s/it]
 93%|█████████▎| 11215/12000 [21:47<32:20,  2.47s/it]
                                                     

 93%|█████████▎| 11215/12000 [21:47<32:20,  2.47s/it]
 93%|█████████▎| 11216/12000 [21:48<28:51,  2.21s/it]
                                                     

 93%|█████████▎| 11216/12000 [21:48<28:51,  2.21s/it]
 93%|█████████▎| 11217/12000 [21:50<26:09,  2.00s/it]
                                                     

 93%|█████████▎| 11217/12000 [21:50<26:09,  2.00s/it]
 93%|█████████▎| 11218/12000 [21:52<24:59,  1.92s/it]
                                                     

 93%|█████████▎| 11218/12000 [21:52<24:59,  1.92s/it]
 93%|█████████▎| 11219/12000 [21:53<23:58,  1.84s/it]
                                                     

 93%|█████████▎| 11219/12000 [21:53<23:58,  1.84s/it]
 94%|█████████▎| 11220/12000 [21:55<22:43,  1.75s/it]
                                                     

 94%|█████████▎| 11220/12000 [21:55<22:43,  1.75s/it]
 94%|█████████▎| 11221/12000 [21:57<23:26,  1.81s/it]
                                                     

 94%|█████████▎| 11221/12000 [21:57<23:26,  1.81s/it]
 94%|█████████▎| 11222/12000 [21:59<23:09,  1.79s/it]
                                                     

 94%|█████████▎| 11222/12000 [21:59<23:09,  1.79s/it]
 94%|█████████▎| 11223/12000 [22:00<22:54,  1.77s/it]
                                                     

 94%|█████████▎| 11223/12000 [22:00<22:54,  1.77s/it]
 94%|█████████▎| 11224/12000 [22:02<22:47,  1.76s/it]
                                                     

 94%|█████████▎| 11224/12000 [22:02<22:47,  1.76s/it]
 94%|█████████▎| 11225/12000 [22:04<21:56,  1.70s/it]
                                                     

 94%|█████████▎| 11225/12000 [22:04<21:56,  1.70s/it]
 94%|█████████▎| 11226/12000 [22:05<21:41,  1.68s/it]
                                                     

 94%|█████████▎| 11226/12000 [22:05<21:41,  1.68s/it]
 94%|█████████▎| 11227/12000 [22:07<21:42,  1.69s/it]
                                                     

 94%|█████████▎| 11227/12000 [22:07<21:42,  1.69s/it]
 94%|█████████▎| 11228/12000 [22:08<20:44,  1.61s/it]
                                                     

 94%|█████████▎| 11228/12000 [22:08<20:44,  1.61s/it]
 94%|█████████▎| 11229/12000 [22:10<21:13,  1.65s/it]
                                                     

 94%|█████████▎| 11229/12000 [22:10<21:13,  1.65s/it]
 94%|█████████▎| 11230/12000 [22:12<20:50,  1.62s/it]
                                                     

 94%|█████████▎| 11230/12000 [22:12<20:50,  1.62s/it]
 94%|█████████▎| 11231/12000 [22:13<21:29,  1.68s/it]
                                                     

 94%|█████████▎| 11231/12000 [22:13<21:29,  1.68s/it]
 94%|█████████▎| 11232/12000 [22:15<20:07,  1.57s/it]
                                                     

 94%|█████████▎| 11232/12000 [22:15<20:07,  1.57s/it]
 94%|█████████▎| 11233/12000 [22:17<21:54,  1.71s/it]
                                                     

 94%|█████████▎| 11233/12000 [22:17<21:54,  1.71s/it]
 94%|█████████▎| 11234/12000 [22:19<21:48,  1.71s/it]
                                                     

 94%|█████████▎| 11234/12000 [22:19<21:48,  1.71s/it]
 94%|█████████▎| 11235/12000 [22:20<22:08,  1.74s/it]
                                                     

 94%|█████████▎| 11235/12000 [22:20<22:08,  1.74s/it]
 94%|█████████▎| 11236/12000 [22:22<22:14,  1.75s/it]
                                                     

 94%|█████████▎| 11236/12000 [22:22<22:14,  1.75s/it]
 94%|█████████▎| 11237/12000 [22:26<29:20,  2.31s/it]
                                                     

 94%|█████████▎| 11237/12000 [22:26<29:20,  2.31s/it]
 94%|█████████▎| 11238/12000 [22:27<25:33,  2.01s/it]
                                                     

 94%|█████████▎| 11238/12000 [22:27<25:33,  2.01s/it]
 94%|█████████▎| 11239/12000 [22:29<23:31,  1.85s/it]
                                                     

 94%|█████████▎| 11239/12000 [22:29<23:31,  1.85s/it]
 94%|█████████▎| 11240/12000 [22:30<23:05,  1.82s/it]
                                                     

 94%|█████████▎| 11240/12000 [22:30<23:05,  1.82s/it]
 94%|█████████▎| 11241/12000 [22:32<22:43,  1.80s/it]
                                                     

 94%|█████████▎| 11241/12000 [22:32<22:43,  1.80s/it]
 94%|█████████▎| 11242/12000 [22:33<21:22,  1.69s/it]
                                                     

 94%|█████████▎| 11242/12000 [22:33<21:22,  1.69s/it]
 94%|█████████▎| 11243/12000 [22:36<24:05,  1.91s/it]
                                                     

 94%|█████████▎| 11243/12000 [22:36<24:05,  1.91s/it]
 94%|█████████▎| 11244/12000 [22:38<24:52,  1.97s/it]
                                                     

 94%|█████████▎| 11244/12000 [22:38<24:52,  1.97s/it]
 94%|█████████▎| 11245/12000 [22:40<23:14,  1.85s/it]
                                                     

 94%|█████████▎| 11245/12000 [22:40<23:14,  1.85s/it]
 94%|█████████▎| 11246/12000 [22:42<24:41,  1.97s/it]
                                                     

 94%|█████████▎| 11246/12000 [22:42<24:41,  1.97s/it]
 94%|█████████▎| 11247/12000 [22:43<22:40,  1.81s/it]
                                                     

 94%|█████████▎| 11247/12000 [22:43<22:40,  1.81s/it]
 94%|█████████▎| 11248/12000 [22:47<30:17,  2.42s/it]
                                                     

 94%|█████████▎| 11248/12000 [22:47<30:17,  2.42s/it]
 94%|█████████▎| 11249/12000 [22:49<27:04,  2.16s/it]
                                                     

 94%|█████████▎| 11249/12000 [22:49<27:04,  2.16s/it]
 94%|█████████▍| 11250/12000 [22:50<24:27,  1.96s/it]
                                                     

 94%|█████████▍| 11250/12000 [22:50<24:27,  1.96s/it]
 94%|█████████▍| 11251/12000 [22:54<32:44,  2.62s/it]
                                                     

 94%|█████████▍| 11251/12000 [22:54<32:44,  2.62s/it]
 94%|█████████▍| 11252/12000 [22:56<28:26,  2.28s/it]
                                                     

 94%|█████████▍| 11252/12000 [22:56<28:26,  2.28s/it]
 94%|█████████▍| 11253/12000 [22:57<25:29,  2.05s/it]
                                                     

 94%|█████████▍| 11253/12000 [22:57<25:29,  2.05s/it]
 94%|█████████▍| 11254/12000 [22:59<25:41,  2.07s/it]
                                                     

 94%|█████████▍| 11254/12000 [22:59<25:41,  2.07s/it]
 94%|█████████▍| 11255/12000 [23:02<26:01,  2.10s/it]
                                                     

 94%|█████████▍| 11255/12000 [23:02<26:01,  2.10s/it]
 94%|█████████▍| 11256/12000 [23:03<23:27,  1.89s/it]
                                                     

 94%|█████████▍| 11256/12000 [23:03<23:27,  1.89s/it]
 94%|█████████▍| 11257/12000 [23:05<23:02,  1.86s/it]
                                                     

 94%|█████████▍| 11257/12000 [23:05<23:02,  1.86s/it]
 94%|█████████▍| 11258/12000 [23:08<26:24,  2.14s/it]
                                                     

 94%|█████████▍| 11258/12000 [23:08<26:24,  2.14s/it]
 94%|█████████▍| 11259/12000 [23:11<32:39,  2.64s/it]
                                                     
{'loss': '0.9556', 'grad_norm': '0.535', 'learning_rate': '2.355e-06', 'entropy': '0.9154', 'num_tokens': '3.282e+06', 'mean_token_accuracy': '0.7717', 'epoch': '0.09922'}
{'loss': '0.577', 'grad_norm': '0.3418', 'learning_rate': '2.349e-06', 'entropy': '0.6213', 'num_tokens': '3.286e+06', 'mean_token_accuracy': '0.8323', 'epoch': '0.09923'}
{'loss': '0.9115', 'grad_norm': '0.4347', 'learning_rate': '2.343e-06', 'entropy': '0.8945', 'num_tokens': '3.291e+06', 'mean_token_accuracy': '0.7802', 'epoch': '0.09924'}
{'loss': '0.7123', 'grad_norm': '0.3839', 'learning_rate': '2.337e-06', 'entropy': '0.7534', 'num_tokens': '3.297e+06', 'mean_token_accuracy': '0.8195', 'epoch': '0.09925'}
{'loss': '0.886', 'grad_norm': '0.517', 'learning_rate': '2.331e-06', 'entropy': '1.111', 'num_tokens': '3.301e+06', 'mean_token_accuracy': '0.7377', 'epoch': '0.09926'}
{'loss': '0.7263', 'grad_norm': '0.4448', 'learning_rate': '2.325e-06', 'entropy': '0.7431', 'num_tokens': '3.305e+06', 'mean_token_accuracy': '0.8068', 'epoch': '0.09927'}
{'loss': '0.9815', 'grad_norm': '0.4798', 'learning_rate': '2.319e-06', 'entropy': '0.9672', 'num_tokens': '3.309e+06', 'mean_token_accuracy': '0.736', 'epoch': '0.09928'}
{'loss': '0.8502', 'grad_norm': '0.3511', 'learning_rate': '2.313e-06', 'entropy': '0.8435', 'num_tokens': '3.315e+06', 'mean_token_accuracy': '0.7865', 'epoch': '0.09929'}
{'loss': '0.5784', 'grad_norm': '0.3599', 'learning_rate': '2.307e-06', 'entropy': '0.6757', 'num_tokens': '3.319e+06', 'mean_token_accuracy': '0.8177', 'epoch': '0.0993'}
{'loss': '0.5485', 'grad_norm': '0.3783', 'learning_rate': '2.301e-06', 'entropy': '0.5562', 'num_tokens': '3.324e+06', 'mean_token_accuracy': '0.8664', 'epoch': '0.0993'}
{'loss': '0.9324', 'grad_norm': '0.5046', 'learning_rate': '2.295e-06', 'entropy': '0.9031', 'num_tokens': '3.329e+06', 'mean_token_accuracy': '0.771', 'epoch': '0.09931'}
{'loss': '0.7375', 'grad_norm': '0.4587', 'learning_rate': '2.29e-06', 'entropy': '0.7222', 'num_tokens': '3.333e+06', 'mean_token_accuracy': '0.8198', 'epoch': '0.09932'}
{'loss': '0.8271', 'grad_norm': '0.365', 'learning_rate': '2.284e-06', 'entropy': '0.8696', 'num_tokens': '3.339e+06', 'mean_token_accuracy': '0.7837', 'epoch': '0.09933'}
{'loss': '0.7833', 'grad_norm': '0.4305', 'learning_rate': '2.278e-06', 'entropy': '0.7552', 'num_tokens': '3.343e+06', 'mean_token_accuracy': '0.8106', 'epoch': '0.09934'}
{'loss': '0.6281', 'grad_norm': '0.3541', 'learning_rate': '2.272e-06', 'entropy': '0.6354', 'num_tokens': '3.347e+06', 'mean_token_accuracy': '0.837', 'epoch': '0.09935'}
{'loss': '0.6098', 'grad_norm': '0.3779', 'learning_rate': '2.266e-06', 'entropy': '0.6321', 'num_tokens': '3.352e+06', 'mean_token_accuracy': '0.8385', 'epoch': '0.09936'}
{'loss': '0.9016', 'grad_norm': '0.5019', 'learning_rate': '2.26e-06', 'entropy': '0.9299', 'num_tokens': '3.355e+06', 'mean_token_accuracy': '0.7833', 'epoch': '0.09937'}
{'loss': '0.7609', 'grad_norm': '0.4106', 'learning_rate': '2.255e-06', 'entropy': '0.753', 'num_tokens': '3.36e+06', 'mean_token_accuracy': '0.8121', 'epoch': '0.09937'}
{'loss': '0.9394', 'grad_norm': '0.4205', 'learning_rate': '2.249e-06', 'entropy': '0.8803', 'num_tokens': '3.364e+06', 'mean_token_accuracy': '0.8024', 'epoch': '0.09938'}
{'loss': '0.6835', 'grad_norm': '0.3866', 'learning_rate': '2.243e-06', 'entropy': '0.6977', 'num_tokens': '3.37e+06', 'mean_token_accuracy': '0.8167', 'epoch': '0.09939'}
{'loss': '0.8966', 'grad_norm': '0.5168', 'learning_rate': '2.237e-06', 'entropy': '0.9513', 'num_tokens': '3.373e+06', 'mean_token_accuracy': '0.7575', 'epoch': '0.0994'}
{'loss': '1.075', 'grad_norm': '0.4301', 'learning_rate': '2.231e-06', 'entropy': '1.062', 'num_tokens': '3.379e+06', 'mean_token_accuracy': '0.7498', 'epoch': '0.09941'}
{'loss': '0.8961', 'grad_norm': '0.4083', 'learning_rate': '2.226e-06', 'entropy': '0.9355', 'num_tokens': '3.384e+06', 'mean_token_accuracy': '0.7689', 'epoch': '0.09942'}
{'loss': '1.179', 'grad_norm': '0.4936', 'learning_rate': '2.22e-06', 'entropy': '1.14', 'num_tokens': '3.388e+06', 'mean_token_accuracy': '0.7251', 'epoch': '0.09943'}
{'loss': '0.5443', 'grad_norm': '0.4963', 'learning_rate': '2.214e-06', 'entropy': '0.6177', 'num_tokens': '3.393e+06', 'mean_token_accuracy': '0.8385', 'epoch': '0.09944'}
{'loss': '0.7507', 'grad_norm': '0.3696', 'learning_rate': '2.208e-06', 'entropy': '0.8626', 'num_tokens': '3.398e+06', 'mean_token_accuracy': '0.7912', 'epoch': '0.09945'}
{'loss': '0.8009', 'grad_norm': '0.4266', 'learning_rate': '2.202e-06', 'entropy': '0.7842', 'num_tokens': '3.402e+06', 'mean_token_accuracy': '0.7903', 'epoch': '0.09945'}
{'loss': '0.6318', 'grad_norm': '0.343', 'learning_rate': '2.197e-06', 'entropy': '0.6296', 'num_tokens': '3.406e+06', 'mean_token_accuracy': '0.8465', 'epoch': '0.09946'}
{'loss': '1.179', 'grad_norm': '0.5104', 'learning_rate': '2.191e-06', 'entropy': '1.071', 'num_tokens': '3.411e+06', 'mean_token_accuracy': '0.7286', 'epoch': '0.09947'}
{'loss': '0.6257', 'grad_norm': '0.4825', 'learning_rate': '2.185e-06', 'entropy': '0.6498', 'num_tokens': '3.416e+06', 'mean_token_accuracy': '0.8256', 'epoch': '0.09948'}
{'loss': '0.5826', 'grad_norm': '0.3811', 'learning_rate': '2.18e-06', 'entropy': '0.6152', 'num_tokens': '3.42e+06', 'mean_token_accuracy': '0.8479', 'epoch': '0.09949'}
{'loss': '0.7878', 'grad_norm': '0.3193', 'learning_rate': '2.174e-06', 'entropy': '0.7908', 'num_tokens': '3.427e+06', 'mean_token_accuracy': '0.7976', 'epoch': '0.0995'}
{'loss': '0.8781', 'grad_norm': '0.4561', 'learning_rate': '2.168e-06', 'entropy': '0.8847', 'num_tokens': '3.433e+06', 'mean_token_accuracy': '0.7868', 'epoch': '0.09951'}
{'loss': '0.8852', 'grad_norm': '0.4774', 'learning_rate': '2.162e-06', 'entropy': '0.8629', 'num_tokens': '3.438e+06', 'mean_token_accuracy': '0.7798', 'epoch': '0.09952'}
{'loss': '0.8344', 'grad_norm': '0.4522', 'learning_rate': '2.157e-06', 'entropy': '0.7708', 'num_tokens': '3.445e+06', 'mean_token_accuracy': '0.8031', 'epoch': '0.09953'}
{'loss': '0.5834', 'grad_norm': '0.3415', 'learning_rate': '2.151e-06', 'entropy': '0.6325', 'num_tokens': '3.449e+06', 'mean_token_accuracy': '0.8412', 'epoch': '0.09953'}
{'loss': '0.5574', 'grad_norm': '0.3205', 'learning_rate': '2.145e-06', 'entropy': '0.8232', 'num_tokens': '3.455e+06', 'mean_token_accuracy': '0.7977', 'epoch': '0.09954'}
{'loss': '0.7691', 'grad_norm': '0.4321', 'learning_rate': '2.14e-06', 'entropy': '0.7677', 'num_tokens': '3.46e+06', 'mean_token_accuracy': '0.7997', 'epoch': '0.09955'}
{'loss': '1.018', 'grad_norm': '0.4661', 'learning_rate': '2.134e-06', 'entropy': '0.8896', 'num_tokens': '3.464e+06', 'mean_token_accuracy': '0.778', 'epoch': '0.09956'}
{'loss': '0.5313', 'grad_norm': '0.3963', 'learning_rate': '2.128e-06', 'entropy': '0.5767', 'num_tokens': '3.471e+06', 'mean_token_accuracy': '0.8446', 'epoch': '0.09957'}
{'loss': '0.6936', 'grad_norm': '0.4329', 'learning_rate': '2.123e-06', 'entropy': '0.6895', 'num_tokens': '3.474e+06', 'mean_token_accuracy': '0.8225', 'epoch': '0.09958'}
{'loss': '0.6031', 'grad_norm': '0.4184', 'learning_rate': '2.117e-06', 'entropy': '0.6832', 'num_tokens': '3.479e+06', 'mean_token_accuracy': '0.8166', 'epoch': '0.09959'}
{'loss': '0.5625', 'grad_norm': '0.2939', 'learning_rate': '2.111e-06', 'entropy': '0.645', 'num_tokens': '3.485e+06', 'mean_token_accuracy': '0.8351', 'epoch': '0.0996'}
{'loss': '0.9231', 'grad_norm': '0.4246', 'learning_rate': '2.106e-06', 'entropy': '0.8339', 'num_tokens': '3.492e+06', 'mean_token_accuracy': '0.7977', 'epoch': '0.0996'}
{'loss': '0.8117', 'grad_norm': '0.3796', 'learning_rate': '2.1e-06', 'entropy': '0.8156', 'num_tokens': '3.496e+06', 'mean_token_accuracy': '0.7806', 'epoch': '0.09961'}
{'loss': '0.9704', 'grad_norm': '0.4895', 'learning_rate': '2.095e-06', 'entropy': '1.119', 'num_tokens': '3.501e+06', 'mean_token_accuracy': '0.7309', 'epoch': '0.09962'}
{'loss': '0.691', 'grad_norm': '0.3288', 'learning_rate': '2.089e-06', 'entropy': '0.6969', 'num_tokens': '3.507e+06', 'mean_token_accuracy': '0.814', 'epoch': '0.09963'}

 94%|█████████▍| 11259/12000 [23:11<32:39,  2.64s/it]
 94%|█████████▍| 11260/12000 [23:14<32:10,  2.61s/it]
                                                     

 94%|█████████▍| 11260/12000 [23:14<32:10,  2.61s/it]
 94%|█████████▍| 11261/12000 [23:15<27:54,  2.27s/it]
                                                     

 94%|█████████▍| 11261/12000 [23:15<27:54,  2.27s/it]
 94%|█████████▍| 11262/12000 [23:17<26:39,  2.17s/it]
                                                     

 94%|█████████▍| 11262/12000 [23:17<26:39,  2.17s/it]
 94%|█████████▍| 11263/12000 [23:19<26:10,  2.13s/it]
                                                     

 94%|█████████▍| 11263/12000 [23:19<26:10,  2.13s/it]
 94%|█████████▍| 11264/12000 [23:21<23:28,  1.91s/it]
                                                     

 94%|█████████▍| 11264/12000 [23:21<23:28,  1.91s/it]
 94%|█████████▍| 11265/12000 [23:22<22:09,  1.81s/it]
                                                     

 94%|█████████▍| 11265/12000 [23:22<22:09,  1.81s/it]
 94%|█████████▍| 11266/12000 [23:24<21:05,  1.72s/it]
                                                     

 94%|█████████▍| 11266/12000 [23:24<21:05,  1.72s/it]
 94%|█████████▍| 11267/12000 [23:26<23:34,  1.93s/it]
                                                     

 94%|█████████▍| 11267/12000 [23:26<23:34,  1.93s/it]
 94%|█████████▍| 11268/12000 [23:28<22:06,  1.81s/it]
                                                     

 94%|█████████▍| 11268/12000 [23:28<22:06,  1.81s/it]
 94%|█████████▍| 11269/12000 [23:30<21:38,  1.78s/it]
                                                     

 94%|█████████▍| 11269/12000 [23:30<21:38,  1.78s/it]
 94%|█████████▍| 11270/12000 [23:31<21:59,  1.81s/it]
                                                     

 94%|█████████▍| 11270/12000 [23:31<21:59,  1.81s/it]
 94%|█████████▍| 11271/12000 [23:37<36:00,  2.96s/it]
                                                     

 94%|█████████▍| 11271/12000 [23:37<36:00,  2.96s/it]
 94%|█████████▍| 11272/12000 [23:40<34:41,  2.86s/it]
                                                     

 94%|█████████▍| 11272/12000 [23:40<34:41,  2.86s/it]
 94%|█████████▍| 11273/12000 [23:42<31:03,  2.56s/it]
                                                     

 94%|█████████▍| 11273/12000 [23:42<31:03,  2.56s/it]
 94%|█████████▍| 11274/12000 [23:43<28:12,  2.33s/it]
                                                     

 94%|█████████▍| 11274/12000 [23:43<28:12,  2.33s/it]
 94%|█████████▍| 11275/12000 [23:45<25:42,  2.13s/it]
                                                     

 94%|█████████▍| 11275/12000 [23:45<25:42,  2.13s/it]
 94%|█████████▍| 11276/12000 [23:47<24:08,  2.00s/it]
                                                     

 94%|█████████▍| 11276/12000 [23:47<24:08,  2.00s/it]
 94%|█████████▍| 11277/12000 [23:49<26:56,  2.24s/it]
                                                     

 94%|█████████▍| 11277/12000 [23:49<26:56,  2.24s/it]
 94%|█████████▍| 11278/12000 [23:53<30:29,  2.53s/it]
                                                     

 94%|█████████▍| 11278/12000 [23:53<30:29,  2.53s/it]
 94%|█████████▍| 11279/12000 [23:54<26:49,  2.23s/it]
                                                     

 94%|█████████▍| 11279/12000 [23:54<26:49,  2.23s/it]
 94%|█████████▍| 11280/12000 [23:56<26:01,  2.17s/it]
                                                     

 94%|█████████▍| 11280/12000 [23:56<26:01,  2.17s/it]
 94%|█████████▍| 11281/12000 [24:00<31:55,  2.66s/it]
                                                     

 94%|█████████▍| 11281/12000 [24:00<31:55,  2.66s/it]
 94%|█████████▍| 11282/12000 [24:02<29:55,  2.50s/it]
                                                     

 94%|█████████▍| 11282/12000 [24:02<29:55,  2.50s/it]
 94%|█████████▍| 11283/12000 [24:04<26:19,  2.20s/it]
                                                     

 94%|█████████▍| 11283/12000 [24:04<26:19,  2.20s/it]
 94%|█████████▍| 11284/12000 [24:05<23:45,  1.99s/it]
                                                     

 94%|█████████▍| 11284/12000 [24:05<23:45,  1.99s/it]
 94%|█████████▍| 11285/12000 [24:07<21:51,  1.83s/it]
                                                     

 94%|█████████▍| 11285/12000 [24:07<21:51,  1.83s/it]
 94%|█████████▍| 11286/12000 [24:08<21:32,  1.81s/it]
                                                     

 94%|█████████▍| 11286/12000 [24:08<21:32,  1.81s/it]
 94%|█████████▍| 11287/12000 [24:10<21:04,  1.77s/it]
                                                     

 94%|█████████▍| 11287/12000 [24:10<21:04,  1.77s/it]
 94%|█████████▍| 11288/12000 [24:13<24:54,  2.10s/it]
                                                     

 94%|█████████▍| 11288/12000 [24:13<24:54,  2.10s/it]
 94%|█████████▍| 11289/12000 [24:17<32:26,  2.74s/it]
                                                     

 94%|█████████▍| 11289/12000 [24:17<32:26,  2.74s/it]
 94%|█████████▍| 11290/12000 [24:19<28:43,  2.43s/it]
                                                     

 94%|█████████▍| 11290/12000 [24:19<28:43,  2.43s/it]
 94%|█████████▍| 11291/12000 [24:21<28:21,  2.40s/it]
                                                     

 94%|█████████▍| 11291/12000 [24:21<28:21,  2.40s/it]
 94%|█████████▍| 11292/12000 [24:24<29:06,  2.47s/it]
                                                     

 94%|█████████▍| 11292/12000 [24:24<29:06,  2.47s/it]
 94%|█████████▍| 11293/12000 [24:26<27:47,  2.36s/it]
                                                     

 94%|█████████▍| 11293/12000 [24:26<27:47,  2.36s/it]
 94%|█████████▍| 11294/12000 [24:28<25:17,  2.15s/it]
                                                     

 94%|█████████▍| 11294/12000 [24:28<25:17,  2.15s/it]
 94%|█████████▍| 11295/12000 [24:30<27:09,  2.31s/it]
                                                     

 94%|█████████▍| 11295/12000 [24:30<27:09,  2.31s/it]
 94%|█████████▍| 11296/12000 [24:32<24:19,  2.07s/it]
                                                     

 94%|█████████▍| 11296/12000 [24:32<24:19,  2.07s/it]
 94%|█████████▍| 11297/12000 [24:34<23:07,  1.97s/it]
                                                     

 94%|█████████▍| 11297/12000 [24:34<23:07,  1.97s/it]
 94%|█████████▍| 11298/12000 [24:35<21:38,  1.85s/it]
                                                     

 94%|█████████▍| 11298/12000 [24:35<21:38,  1.85s/it]
 94%|█████████▍| 11299/12000 [24:37<20:20,  1.74s/it]
                                                     

 94%|█████████▍| 11299/12000 [24:37<20:20,  1.74s/it]
 94%|█████████▍| 11300/12000 [24:39<23:48,  2.04s/it]
                                                     

 94%|█████████▍| 11300/12000 [24:39<23:48,  2.04s/it]
 94%|█████████▍| 11301/12000 [24:42<26:22,  2.26s/it]
                                                     

 94%|█████████▍| 11301/12000 [24:42<26:22,  2.26s/it]
 94%|█████████▍| 11302/12000 [24:44<23:37,  2.03s/it]
                                                     

 94%|█████████▍| 11302/12000 [24:44<23:37,  2.03s/it]
 94%|█████████▍| 11303/12000 [24:47<29:30,  2.54s/it]
                                                     

 94%|█████████▍| 11303/12000 [24:47<29:30,  2.54s/it]
 94%|█████████▍| 11304/12000 [24:52<37:04,  3.20s/it]
                                                     

 94%|█████████▍| 11304/12000 [24:52<37:04,  3.20s/it]
 94%|█████████▍| 11305/12000 [24:55<35:42,  3.08s/it]
                                                     

 94%|█████████▍| 11305/12000 [24:55<35:42,  3.08s/it]
 94%|█████████▍| 11306/12000 [24:56<30:24,  2.63s/it]
                                                     
{'loss': '0.7928', 'grad_norm': '0.431', 'learning_rate': '2.083e-06', 'entropy': '0.7319', 'num_tokens': '3.513e+06', 'mean_token_accuracy': '0.8224', 'epoch': '0.09964'}
{'loss': '0.8165', 'grad_norm': '0.3743', 'learning_rate': '2.078e-06', 'entropy': '0.8296', 'num_tokens': '3.52e+06', 'mean_token_accuracy': '0.7843', 'epoch': '0.09965'}
{'loss': '1.127', 'grad_norm': '0.4682', 'learning_rate': '2.072e-06', 'entropy': '1.107', 'num_tokens': '3.524e+06', 'mean_token_accuracy': '0.7295', 'epoch': '0.09966'}
{'loss': '0.7806', 'grad_norm': '0.3467', 'learning_rate': '2.067e-06', 'entropy': '0.7282', 'num_tokens': '3.53e+06', 'mean_token_accuracy': '0.8141', 'epoch': '0.09967'}
{'loss': '0.9444', 'grad_norm': '0.412', 'learning_rate': '2.061e-06', 'entropy': '0.9114', 'num_tokens': '3.537e+06', 'mean_token_accuracy': '0.7728', 'epoch': '0.09968'}
{'loss': '0.6884', 'grad_norm': '0.3448', 'learning_rate': '2.055e-06', 'entropy': '0.7255', 'num_tokens': '3.541e+06', 'mean_token_accuracy': '0.8233', 'epoch': '0.09968'}
{'loss': '0.779', 'grad_norm': '0.3372', 'learning_rate': '2.05e-06', 'entropy': '0.8123', 'num_tokens': '3.546e+06', 'mean_token_accuracy': '0.8136', 'epoch': '0.09969'}
{'loss': '0.724', 'grad_norm': '0.3132', 'learning_rate': '2.044e-06', 'entropy': '0.7311', 'num_tokens': '3.551e+06', 'mean_token_accuracy': '0.8029', 'epoch': '0.0997'}
{'loss': '0.7754', 'grad_norm': '0.4628', 'learning_rate': '2.039e-06', 'entropy': '0.7889', 'num_tokens': '3.557e+06', 'mean_token_accuracy': '0.786', 'epoch': '0.09971'}
{'loss': '0.7792', 'grad_norm': '0.4456', 'learning_rate': '2.033e-06', 'entropy': '0.7808', 'num_tokens': '3.561e+06', 'mean_token_accuracy': '0.8092', 'epoch': '0.09972'}
{'loss': '0.5777', 'grad_norm': '0.3955', 'learning_rate': '2.028e-06', 'entropy': '0.6435', 'num_tokens': '3.566e+06', 'mean_token_accuracy': '0.8417', 'epoch': '0.09973'}
{'loss': '1.039', 'grad_norm': '0.3758', 'learning_rate': '2.022e-06', 'entropy': '1.003', 'num_tokens': '3.572e+06', 'mean_token_accuracy': '0.751', 'epoch': '0.09974'}
{'loss': '1.014', 'grad_norm': '0.2781', 'learning_rate': '2.017e-06', 'entropy': '1.089', 'num_tokens': '3.584e+06', 'mean_token_accuracy': '0.7398', 'epoch': '0.09975'}
{'loss': '0.7449', 'grad_norm': '0.345', 'learning_rate': '2.011e-06', 'entropy': '0.8186', 'num_tokens': '3.59e+06', 'mean_token_accuracy': '0.8021', 'epoch': '0.09976'}
{'loss': '0.6536', 'grad_norm': '0.331', 'learning_rate': '2.006e-06', 'entropy': '0.6168', 'num_tokens': '3.595e+06', 'mean_token_accuracy': '0.8518', 'epoch': '0.09976'}
{'loss': '0.5035', 'grad_norm': '0.3104', 'learning_rate': '2e-06', 'entropy': '0.5037', 'num_tokens': '3.601e+06', 'mean_token_accuracy': '0.871', 'epoch': '0.09977'}
{'loss': '0.8078', 'grad_norm': '0.3735', 'learning_rate': '1.995e-06', 'entropy': '0.7488', 'num_tokens': '3.607e+06', 'mean_token_accuracy': '0.8072', 'epoch': '0.09978'}
{'loss': '0.5134', 'grad_norm': '0.3101', 'learning_rate': '1.989e-06', 'entropy': '0.5054', 'num_tokens': '3.612e+06', 'mean_token_accuracy': '0.8815', 'epoch': '0.09979'}
{'loss': '1.017', 'grad_norm': '0.4168', 'learning_rate': '1.984e-06', 'entropy': '0.9911', 'num_tokens': '3.619e+06', 'mean_token_accuracy': '0.7646', 'epoch': '0.0998'}
{'loss': '0.7904', 'grad_norm': '0.4827', 'learning_rate': '1.978e-06', 'entropy': '0.7161', 'num_tokens': '3.627e+06', 'mean_token_accuracy': '0.8212', 'epoch': '0.09981'}
{'loss': '0.8109', 'grad_norm': '0.5262', 'learning_rate': '1.973e-06', 'entropy': '0.8237', 'num_tokens': '3.63e+06', 'mean_token_accuracy': '0.7903', 'epoch': '0.09982'}
{'loss': '1.255', 'grad_norm': '0.4685', 'learning_rate': '1.967e-06', 'entropy': '1.225', 'num_tokens': '3.636e+06', 'mean_token_accuracy': '0.7007', 'epoch': '0.09983'}
{'loss': '1.472', 'grad_norm': '0.4884', 'learning_rate': '1.962e-06', 'entropy': '1.295', 'num_tokens': '3.64e+06', 'mean_token_accuracy': '0.7174', 'epoch': '0.09983'}
{'loss': '0.6329', 'grad_norm': '0.3145', 'learning_rate': '1.957e-06', 'entropy': '0.6143', 'num_tokens': '3.646e+06', 'mean_token_accuracy': '0.8453', 'epoch': '0.09984'}
{'loss': '0.5217', 'grad_norm': '0.3655', 'learning_rate': '1.951e-06', 'entropy': '0.5134', 'num_tokens': '3.65e+06', 'mean_token_accuracy': '0.8616', 'epoch': '0.09985'}
{'loss': '1.028', 'grad_norm': '0.5306', 'learning_rate': '1.946e-06', 'entropy': '0.9832', 'num_tokens': '3.654e+06', 'mean_token_accuracy': '0.7553', 'epoch': '0.09986'}
{'loss': '0.6753', 'grad_norm': '0.3277', 'learning_rate': '1.94e-06', 'entropy': '0.7099', 'num_tokens': '3.658e+06', 'mean_token_accuracy': '0.8002', 'epoch': '0.09987'}
{'loss': '0.566', 'grad_norm': '0.3085', 'learning_rate': '1.935e-06', 'entropy': '0.5406', 'num_tokens': '3.663e+06', 'mean_token_accuracy': '0.8586', 'epoch': '0.09988'}
{'loss': '0.7698', 'grad_norm': '0.38', 'learning_rate': '1.93e-06', 'entropy': '0.803', 'num_tokens': '3.667e+06', 'mean_token_accuracy': '0.795', 'epoch': '0.09989'}
{'loss': '0.9054', 'grad_norm': '0.4158', 'learning_rate': '1.924e-06', 'entropy': '0.9395', 'num_tokens': '3.674e+06', 'mean_token_accuracy': '0.7734', 'epoch': '0.0999'}
{'loss': '0.8615', 'grad_norm': '0.3171', 'learning_rate': '1.919e-06', 'entropy': '0.7203', 'num_tokens': '3.682e+06', 'mean_token_accuracy': '0.8126', 'epoch': '0.09991'}
{'loss': '0.7719', 'grad_norm': '0.4448', 'learning_rate': '1.913e-06', 'entropy': '0.7954', 'num_tokens': '3.687e+06', 'mean_token_accuracy': '0.7967', 'epoch': '0.09991'}
{'loss': '0.5532', 'grad_norm': '0.3246', 'learning_rate': '1.908e-06', 'entropy': '0.5447', 'num_tokens': '3.694e+06', 'mean_token_accuracy': '0.846', 'epoch': '0.09992'}
{'loss': '0.708', 'grad_norm': '0.3422', 'learning_rate': '1.903e-06', 'entropy': '0.6593', 'num_tokens': '3.701e+06', 'mean_token_accuracy': '0.8353', 'epoch': '0.09993'}
{'loss': '0.9965', 'grad_norm': '0.3749', 'learning_rate': '1.897e-06', 'entropy': '0.9999', 'num_tokens': '3.708e+06', 'mean_token_accuracy': '0.7611', 'epoch': '0.09994'}
{'loss': '0.9204', 'grad_norm': '0.5115', 'learning_rate': '1.892e-06', 'entropy': '0.947', 'num_tokens': '3.712e+06', 'mean_token_accuracy': '0.7698', 'epoch': '0.09995'}
{'loss': '1.009', 'grad_norm': '0.468', 'learning_rate': '1.887e-06', 'entropy': '0.9699', 'num_tokens': '3.719e+06', 'mean_token_accuracy': '0.7648', 'epoch': '0.09996'}
{'loss': '0.7271', 'grad_norm': '0.3827', 'learning_rate': '1.881e-06', 'entropy': '0.8025', 'num_tokens': '3.723e+06', 'mean_token_accuracy': '0.8024', 'epoch': '0.09997'}
{'loss': '0.5241', 'grad_norm': '0.3984', 'learning_rate': '1.876e-06', 'entropy': '0.5413', 'num_tokens': '3.729e+06', 'mean_token_accuracy': '0.8571', 'epoch': '0.09998'}
{'loss': '0.9347', 'grad_norm': '0.492', 'learning_rate': '1.871e-06', 'entropy': '0.94', 'num_tokens': '3.733e+06', 'mean_token_accuracy': '0.7736', 'epoch': '0.09999'}
{'loss': '0.7541', 'grad_norm': '0.4397', 'learning_rate': '1.865e-06', 'entropy': '0.8799', 'num_tokens': '3.737e+06', 'mean_token_accuracy': '0.7708', 'epoch': '0.09999'}
{'loss': '0.9451', 'grad_norm': '0.349', 'learning_rate': '1.86e-06', 'entropy': '0.9127', 'num_tokens': '3.744e+06', 'mean_token_accuracy': '0.7896', 'epoch': '0.1'}
{'loss': '0.6976', 'grad_norm': '0.447', 'learning_rate': '1.855e-06', 'entropy': '0.7538', 'num_tokens': '3.748e+06', 'mean_token_accuracy': '0.8149', 'epoch': '0.1'}
{'loss': '0.8284', 'grad_norm': '0.4259', 'learning_rate': '1.85e-06', 'entropy': '0.8339', 'num_tokens': '3.753e+06', 'mean_token_accuracy': '0.7923', 'epoch': '0.1'}
{'loss': '0.9643', 'grad_norm': '0.3942', 'learning_rate': '1.844e-06', 'entropy': '0.9228', 'num_tokens': '3.758e+06', 'mean_token_accuracy': '0.7785', 'epoch': '0.1'}
{'loss': '1.088', 'grad_norm': '0.4148', 'learning_rate': '1.839e-06', 'entropy': '0.8385', 'num_tokens': '3.766e+06', 'mean_token_accuracy': '0.7947', 'epoch': '0.1'}
{'loss': '0.5225', 'grad_norm': '0.302', 'learning_rate': '1.834e-06', 'entropy': '0.5595', 'num_tokens': '3.773e+06', 'mean_token_accuracy': '0.8466', 'epoch': '0.1'}

 94%|█████████▍| 11306/12000 [24:56<30:24,  2.63s/it]
 94%|█████████▍| 11307/12000 [24:59<29:35,  2.56s/it]
                                                     

 94%|█████████▍| 11307/12000 [24:59<29:35,  2.56s/it]
 94%|█████████▍| 11308/12000 [25:01<27:47,  2.41s/it]
                                                     

 94%|█████████▍| 11308/12000 [25:01<27:47,  2.41s/it]
 94%|█████████▍| 11309/12000 [25:03<25:27,  2.21s/it]
                                                     

 94%|█████████▍| 11309/12000 [25:03<25:27,  2.21s/it]
 94%|█████████▍| 11310/12000 [25:05<24:54,  2.17s/it]
                                                     

 94%|█████████▍| 11310/12000 [25:05<24:54,  2.17s/it]
 94%|█████████▍| 11311/12000 [25:07<25:40,  2.24s/it]
                                                     

 94%|█████████▍| 11311/12000 [25:07<25:40,  2.24s/it]
 94%|█████████▍| 11312/12000 [25:09<23:34,  2.06s/it]
                                                     

 94%|█████████▍| 11312/12000 [25:09<23:34,  2.06s/it]
 94%|█████████▍| 11313/12000 [25:12<26:05,  2.28s/it]
                                                     

 94%|█████████▍| 11313/12000 [25:12<26:05,  2.28s/it]
 94%|█████████▍| 11314/12000 [25:17<36:33,  3.20s/it]
                                                     

 94%|█████████▍| 11314/12000 [25:17<36:33,  3.20s/it]
 94%|█████████▍| 11315/12000 [25:22<43:47,  3.84s/it]
                                                     

 94%|█████████▍| 11315/12000 [25:22<43:47,  3.84s/it]
 94%|█████████▍| 11316/12000 [25:24<36:07,  3.17s/it]
                                                     

 94%|█████████▍| 11316/12000 [25:24<36:07,  3.17s/it]
 94%|█████████▍| 11317/12000 [25:26<32:33,  2.86s/it]
                                                     

 94%|█████████▍| 11317/12000 [25:26<32:33,  2.86s/it]
 94%|█████████▍| 11318/12000 [25:28<28:51,  2.54s/it]
                                                     

 94%|█████████▍| 11318/12000 [25:28<28:51,  2.54s/it]
 94%|█████████▍| 11319/12000 [25:30<26:46,  2.36s/it]
                                                     

 94%|█████████▍| 11319/12000 [25:30<26:46,  2.36s/it]
 94%|█████████▍| 11320/12000 [25:31<24:39,  2.18s/it]
                                                     

 94%|█████████▍| 11320/12000 [25:31<24:39,  2.18s/it]
 94%|█████████▍| 11321/12000 [25:33<23:31,  2.08s/it]
                                                     

 94%|█████████▍| 11321/12000 [25:33<23:31,  2.08s/it]
 94%|█████████▍| 11322/12000 [25:35<22:21,  1.98s/it]
                                                     

 94%|█████████▍| 11322/12000 [25:35<22:21,  1.98s/it]
 94%|█████████▍| 11323/12000 [25:37<21:35,  1.91s/it]
                                                     

 94%|█████████▍| 11323/12000 [25:37<21:35,  1.91s/it]
 94%|█████████▍| 11324/12000 [25:40<24:24,  2.17s/it]
                                                     

 94%|█████████▍| 11324/12000 [25:40<24:24,  2.17s/it]
 94%|█████████▍| 11325/12000 [25:44<31:39,  2.81s/it]
                                                     

 94%|█████████▍| 11325/12000 [25:44<31:39,  2.81s/it]
 94%|█████████▍| 11326/12000 [25:46<28:11,  2.51s/it]
                                                     

 94%|█████████▍| 11326/12000 [25:46<28:11,  2.51s/it]
 94%|█████████▍| 11327/12000 [25:47<24:57,  2.23s/it]
                                                     

 94%|█████████▍| 11327/12000 [25:47<24:57,  2.23s/it]
 94%|█████████▍| 11328/12000 [25:51<28:49,  2.57s/it]
                                                     

 94%|█████████▍| 11328/12000 [25:51<28:49,  2.57s/it]
 94%|█████████▍| 11329/12000 [25:54<31:06,  2.78s/it]
                                                     

 94%|█████████▍| 11329/12000 [25:54<31:06,  2.78s/it]
 94%|█████████▍| 11330/12000 [25:57<32:38,  2.92s/it]
                                                     

 94%|█████████▍| 11330/12000 [25:57<32:38,  2.92s/it]
 94%|█████████▍| 11331/12000 [25:59<28:16,  2.54s/it]
                                                     

 94%|█████████▍| 11331/12000 [25:59<28:16,  2.54s/it]
 94%|█████████▍| 11332/12000 [26:01<28:30,  2.56s/it]
                                                     

 94%|█████████▍| 11332/12000 [26:01<28:30,  2.56s/it]
 94%|█████████▍| 11333/12000 [26:03<25:09,  2.26s/it]
                                                     

 94%|█████████▍| 11333/12000 [26:03<25:09,  2.26s/it]
 94%|█████████▍| 11334/12000 [26:05<23:16,  2.10s/it]
                                                     

 94%|█████████▍| 11334/12000 [26:05<23:16,  2.10s/it]
 94%|█████████▍| 11335/12000 [26:06<21:44,  1.96s/it]
                                                     

 94%|█████████▍| 11335/12000 [26:06<21:44,  1.96s/it]
 94%|█████████▍| 11336/12000 [26:08<22:01,  1.99s/it]
                                                     

 94%|█████████▍| 11336/12000 [26:08<22:01,  1.99s/it]
 94%|█████████▍| 11337/12000 [26:10<20:43,  1.88s/it]
                                                     

 94%|█████████▍| 11337/12000 [26:10<20:43,  1.88s/it]
 94%|█████████▍| 11338/12000 [26:12<21:30,  1.95s/it]
                                                     

 94%|█████████▍| 11338/12000 [26:12<21:30,  1.95s/it]
 94%|█████████▍| 11339/12000 [26:14<20:02,  1.82s/it]
                                                     

 94%|█████████▍| 11339/12000 [26:14<20:02,  1.82s/it]
 94%|█████████▍| 11340/12000 [26:16<21:52,  1.99s/it]
                                                     

 94%|█████████▍| 11340/12000 [26:16<21:52,  1.99s/it]
 95%|█████████▍| 11341/12000 [26:17<20:01,  1.82s/it]
                                                     

 95%|█████████▍| 11341/12000 [26:17<20:01,  1.82s/it]
 95%|█████████▍| 11342/12000 [26:19<19:29,  1.78s/it]
                                                     

 95%|█████████▍| 11342/12000 [26:19<19:29,  1.78s/it]
 95%|█████████▍| 11343/12000 [26:21<18:26,  1.68s/it]
                                                     

 95%|█████████▍| 11343/12000 [26:21<18:26,  1.68s/it]
 95%|█████████▍| 11344/12000 [26:22<18:10,  1.66s/it]
                                                     

 95%|█████████▍| 11344/12000 [26:22<18:10,  1.66s/it]
 95%|█████████▍| 11345/12000 [26:24<17:55,  1.64s/it]
                                                     

 95%|█████████▍| 11345/12000 [26:24<17:55,  1.64s/it]
 95%|█████████▍| 11346/12000 [26:26<19:13,  1.76s/it]
                                                     

 95%|█████████▍| 11346/12000 [26:26<19:13,  1.76s/it]
 95%|█████████▍| 11347/12000 [26:29<24:58,  2.30s/it]
                                                     

 95%|█████████▍| 11347/12000 [26:29<24:58,  2.30s/it]
 95%|█████████▍| 11348/12000 [26:31<21:52,  2.01s/it]
                                                     

 95%|█████████▍| 11348/12000 [26:31<21:52,  2.01s/it]
 95%|█████████▍| 11349/12000 [26:33<23:37,  2.18s/it]
                                                     

 95%|█████████▍| 11349/12000 [26:33<23:37,  2.18s/it]
 95%|█████████▍| 11350/12000 [26:35<21:10,  1.96s/it]
                                                     

 95%|█████████▍| 11350/12000 [26:35<21:10,  1.96s/it]
 95%|█████████▍| 11351/12000 [26:38<23:57,  2.21s/it]
                                                     

 95%|█████████▍| 11351/12000 [26:38<23:57,  2.21s/it]
 95%|█████████▍| 11352/12000 [26:39<21:41,  2.01s/it]
                                                     

 95%|█████████▍| 11352/12000 [26:39<21:41,  2.01s/it]
 95%|█████████▍| 11353/12000 [26:42<25:50,  2.40s/it]
                                                     
{'loss': '0.6229', 'grad_norm': '0.3636', 'learning_rate': '1.829e-06', 'entropy': '0.7167', 'num_tokens': '3.777e+06', 'mean_token_accuracy': '0.8271', 'epoch': '0.1001'}
{'loss': '0.6281', 'grad_norm': '0.4254', 'learning_rate': '1.823e-06', 'entropy': '0.6601', 'num_tokens': '3.783e+06', 'mean_token_accuracy': '0.828', 'epoch': '0.1001'}
{'loss': '1.207', 'grad_norm': '0.4516', 'learning_rate': '1.818e-06', 'entropy': '1.046', 'num_tokens': '3.79e+06', 'mean_token_accuracy': '0.7466', 'epoch': '0.1001'}
{'loss': '0.9701', 'grad_norm': '0.5691', 'learning_rate': '1.813e-06', 'entropy': '0.9827', 'num_tokens': '3.794e+06', 'mean_token_accuracy': '0.7538', 'epoch': '0.1001'}
{'loss': '0.8455', 'grad_norm': '0.4172', 'learning_rate': '1.808e-06', 'entropy': '0.8384', 'num_tokens': '3.8e+06', 'mean_token_accuracy': '0.7916', 'epoch': '0.1001'}
{'loss': '0.6045', 'grad_norm': '0.3328', 'learning_rate': '1.802e-06', 'entropy': '0.6014', 'num_tokens': '3.806e+06', 'mean_token_accuracy': '0.8595', 'epoch': '0.1001'}
{'loss': '0.7864', 'grad_norm': '0.3725', 'learning_rate': '1.797e-06', 'entropy': '0.6981', 'num_tokens': '3.812e+06', 'mean_token_accuracy': '0.8233', 'epoch': '0.1001'}
{'loss': '0.5538', 'grad_norm': '0.3035', 'learning_rate': '1.792e-06', 'entropy': '0.6209', 'num_tokens': '3.818e+06', 'mean_token_accuracy': '0.8344', 'epoch': '0.1001'}
{'loss': '1.18', 'grad_norm': '0.3674', 'learning_rate': '1.787e-06', 'entropy': '1.043', 'num_tokens': '3.829e+06', 'mean_token_accuracy': '0.7667', 'epoch': '0.1001'}
{'loss': '1.246', 'grad_norm': '0.3519', 'learning_rate': '1.782e-06', 'entropy': '0.9478', 'num_tokens': '3.839e+06', 'mean_token_accuracy': '0.7741', 'epoch': '0.1001'}
{'loss': '0.6939', 'grad_norm': '0.4699', 'learning_rate': '1.776e-06', 'entropy': '0.7323', 'num_tokens': '3.843e+06', 'mean_token_accuracy': '0.8106', 'epoch': '0.1001'}
{'loss': '0.7061', 'grad_norm': '0.4528', 'learning_rate': '1.771e-06', 'entropy': '0.6657', 'num_tokens': '3.85e+06', 'mean_token_accuracy': '0.8305', 'epoch': '0.1002'}
{'loss': '0.573', 'grad_norm': '0.3744', 'learning_rate': '1.766e-06', 'entropy': '0.5894', 'num_tokens': '3.855e+06', 'mean_token_accuracy': '0.8449', 'epoch': '0.1002'}
{'loss': '0.6543', 'grad_norm': '0.3267', 'learning_rate': '1.761e-06', 'entropy': '0.7027', 'num_tokens': '3.861e+06', 'mean_token_accuracy': '0.8243', 'epoch': '0.1002'}
{'loss': '0.649', 'grad_norm': '0.4362', 'learning_rate': '1.756e-06', 'entropy': '0.6502', 'num_tokens': '3.866e+06', 'mean_token_accuracy': '0.8234', 'epoch': '0.1002'}
{'loss': '0.877', 'grad_norm': '0.3964', 'learning_rate': '1.751e-06', 'entropy': '0.9054', 'num_tokens': '3.872e+06', 'mean_token_accuracy': '0.7717', 'epoch': '0.1002'}
{'loss': '0.9044', 'grad_norm': '0.4639', 'learning_rate': '1.746e-06', 'entropy': '0.9335', 'num_tokens': '3.876e+06', 'mean_token_accuracy': '0.7731', 'epoch': '0.1002'}
{'loss': '0.645', 'grad_norm': '0.5371', 'learning_rate': '1.74e-06', 'entropy': '0.6589', 'num_tokens': '3.881e+06', 'mean_token_accuracy': '0.828', 'epoch': '0.1002'}
{'loss': '1.03', 'grad_norm': '0.3566', 'learning_rate': '1.735e-06', 'entropy': '0.9795', 'num_tokens': '3.888e+06', 'mean_token_accuracy': '0.7641', 'epoch': '0.1002'}
{'loss': '0.6972', 'grad_norm': '0.3866', 'learning_rate': '1.73e-06', 'entropy': '0.7013', 'num_tokens': '3.894e+06', 'mean_token_accuracy': '0.8057', 'epoch': '0.1002'}
{'loss': '1.257', 'grad_norm': '0.4519', 'learning_rate': '1.725e-06', 'entropy': '1.201', 'num_tokens': '3.899e+06', 'mean_token_accuracy': '0.7224', 'epoch': '0.1002'}
{'loss': '0.7321', 'grad_norm': '0.4099', 'learning_rate': '1.72e-06', 'entropy': '0.7574', 'num_tokens': '3.903e+06', 'mean_token_accuracy': '0.8032', 'epoch': '0.1002'}
{'loss': '0.7116', 'grad_norm': '0.3385', 'learning_rate': '1.715e-06', 'entropy': '0.8389', 'num_tokens': '3.912e+06', 'mean_token_accuracy': '0.7921', 'epoch': '0.1003'}
{'loss': '0.6788', 'grad_norm': '0.3285', 'learning_rate': '1.71e-06', 'entropy': '0.6372', 'num_tokens': '3.919e+06', 'mean_token_accuracy': '0.836', 'epoch': '0.1003'}
{'loss': '0.7449', 'grad_norm': '0.3361', 'learning_rate': '1.705e-06', 'entropy': '0.7565', 'num_tokens': '3.927e+06', 'mean_token_accuracy': '0.8088', 'epoch': '0.1003'}
{'loss': '0.852', 'grad_norm': '0.4142', 'learning_rate': '1.7e-06', 'entropy': '0.7598', 'num_tokens': '3.932e+06', 'mean_token_accuracy': '0.8117', 'epoch': '0.1003'}
{'loss': '0.8648', 'grad_norm': '0.3356', 'learning_rate': '1.695e-06', 'entropy': '0.9372', 'num_tokens': '3.939e+06', 'mean_token_accuracy': '0.7639', 'epoch': '0.1003'}
{'loss': '0.9298', 'grad_norm': '0.3901', 'learning_rate': '1.69e-06', 'entropy': '0.9007', 'num_tokens': '3.943e+06', 'mean_token_accuracy': '0.7759', 'epoch': '0.1003'}
{'loss': '0.6376', 'grad_norm': '0.427', 'learning_rate': '1.685e-06', 'entropy': '0.6423', 'num_tokens': '3.948e+06', 'mean_token_accuracy': '0.8269', 'epoch': '0.1003'}
{'loss': '0.7949', 'grad_norm': '0.4291', 'learning_rate': '1.68e-06', 'entropy': '0.8815', 'num_tokens': '3.954e+06', 'mean_token_accuracy': '0.7813', 'epoch': '0.1003'}
{'loss': '1.266', 'grad_norm': '0.5291', 'learning_rate': '1.675e-06', 'entropy': '1.124', 'num_tokens': '3.959e+06', 'mean_token_accuracy': '0.7256', 'epoch': '0.1003'}
{'loss': '0.6618', 'grad_norm': '0.3712', 'learning_rate': '1.669e-06', 'entropy': '0.7076', 'num_tokens': '3.963e+06', 'mean_token_accuracy': '0.8203', 'epoch': '0.1003'}
{'loss': '0.9102', 'grad_norm': '0.3704', 'learning_rate': '1.664e-06', 'entropy': '0.8566', 'num_tokens': '3.969e+06', 'mean_token_accuracy': '0.7885', 'epoch': '0.1003'}
{'loss': '0.5672', 'grad_norm': '0.3933', 'learning_rate': '1.659e-06', 'entropy': '0.6373', 'num_tokens': '3.973e+06', 'mean_token_accuracy': '0.8386', 'epoch': '0.1003'}
{'loss': '0.5599', 'grad_norm': '0.3309', 'learning_rate': '1.654e-06', 'entropy': '0.6244', 'num_tokens': '3.98e+06', 'mean_token_accuracy': '0.8325', 'epoch': '0.1004'}
{'loss': '0.8729', 'grad_norm': '0.4159', 'learning_rate': '1.649e-06', 'entropy': '0.9272', 'num_tokens': '3.984e+06', 'mean_token_accuracy': '0.7664', 'epoch': '0.1004'}
{'loss': '0.7713', 'grad_norm': '0.4417', 'learning_rate': '1.645e-06', 'entropy': '0.8002', 'num_tokens': '3.989e+06', 'mean_token_accuracy': '0.7983', 'epoch': '0.1004'}
{'loss': '0.7263', 'grad_norm': '0.3936', 'learning_rate': '1.64e-06', 'entropy': '0.7571', 'num_tokens': '3.994e+06', 'mean_token_accuracy': '0.8117', 'epoch': '0.1004'}
{'loss': '0.6085', 'grad_norm': '0.3365', 'learning_rate': '1.635e-06', 'entropy': '0.6094', 'num_tokens': '4e+06', 'mean_token_accuracy': '0.8342', 'epoch': '0.1004'}
{'loss': '0.9157', 'grad_norm': '0.3882', 'learning_rate': '1.63e-06', 'entropy': '0.8862', 'num_tokens': '4.005e+06', 'mean_token_accuracy': '0.7753', 'epoch': '0.1004'}
{'loss': '0.7966', 'grad_norm': '0.468', 'learning_rate': '1.625e-06', 'entropy': '0.6921', 'num_tokens': '4.01e+06', 'mean_token_accuracy': '0.8166', 'epoch': '0.1004'}
{'loss': '0.9032', 'grad_norm': '0.3872', 'learning_rate': '1.62e-06', 'entropy': '0.9367', 'num_tokens': '4.015e+06', 'mean_token_accuracy': '0.7606', 'epoch': '0.1004'}
{'loss': '0.8568', 'grad_norm': '0.4188', 'learning_rate': '1.615e-06', 'entropy': '0.807', 'num_tokens': '4.018e+06', 'mean_token_accuracy': '0.7899', 'epoch': '0.1004'}
{'loss': '1.219', 'grad_norm': '0.4396', 'learning_rate': '1.61e-06', 'entropy': '1.221', 'num_tokens': '4.025e+06', 'mean_token_accuracy': '0.7241', 'epoch': '0.1004'}
{'loss': '0.4821', 'grad_norm': '0.3697', 'learning_rate': '1.605e-06', 'entropy': '0.5539', 'num_tokens': '4.028e+06', 'mean_token_accuracy': '0.8615', 'epoch': '0.1004'}
{'loss': '0.6459', 'grad_norm': '0.3834', 'learning_rate': '1.6e-06', 'entropy': '0.7177', 'num_tokens': '4.032e+06', 'mean_token_accuracy': '0.8254', 'epoch': '0.1005'}
{'loss': '1.102', 'grad_norm': '0.4131', 'learning_rate': '1.595e-06', 'entropy': '1.047', 'num_tokens': '4.036e+06', 'mean_token_accuracy': '0.7406', 'epoch': '0.1005'}

 95%|█████████▍| 11353/12000 [26:42<25:50,  2.40s/it]
 95%|█████████▍| 11354/12000 [26:45<27:59,  2.60s/it]
                                                     

 95%|█████████▍| 11354/12000 [26:45<27:59,  2.60s/it]
 95%|█████████▍| 11355/12000 [26:47<24:42,  2.30s/it]
                                                     

 95%|█████████▍| 11355/12000 [26:47<24:42,  2.30s/it]
 95%|█████████▍| 11356/12000 [26:48<21:47,  2.03s/it]
                                                     

 95%|█████████▍| 11356/12000 [26:48<21:47,  2.03s/it]
 95%|█████████▍| 11357/12000 [26:50<19:57,  1.86s/it]
                                                     

 95%|█████████▍| 11357/12000 [26:50<19:57,  1.86s/it]
 95%|█████████▍| 11358/12000 [26:52<18:56,  1.77s/it]
                                                     

 95%|█████████▍| 11358/12000 [26:52<18:56,  1.77s/it]
 95%|█████████▍| 11359/12000 [26:55<24:29,  2.29s/it]
                                                     

 95%|█████████▍| 11359/12000 [26:55<24:29,  2.29s/it]
 95%|█████████▍| 11360/12000 [26:57<22:46,  2.13s/it]
                                                     

 95%|█████████▍| 11360/12000 [26:57<22:46,  2.13s/it]
 95%|█████████▍| 11361/12000 [26:58<21:00,  1.97s/it]
                                                     

 95%|█████████▍| 11361/12000 [26:58<21:00,  1.97s/it]
 95%|█████████▍| 11362/12000 [27:00<19:51,  1.87s/it]
                                                     

 95%|█████████▍| 11362/12000 [27:00<19:51,  1.87s/it]
 95%|█████████▍| 11363/12000 [27:02<18:47,  1.77s/it]
                                                     

 95%|█████████▍| 11363/12000 [27:02<18:47,  1.77s/it]
 95%|█████████▍| 11364/12000 [27:03<18:06,  1.71s/it]
                                                     

 95%|█████████▍| 11364/12000 [27:03<18:06,  1.71s/it]
 95%|█████████▍| 11365/12000 [27:05<17:12,  1.63s/it]
                                                     

 95%|█████████▍| 11365/12000 [27:05<17:12,  1.63s/it]
 95%|█████████▍| 11366/12000 [27:06<17:04,  1.62s/it]
                                                     

 95%|█████████▍| 11366/12000 [27:06<17:04,  1.62s/it]
 95%|█████████▍| 11367/12000 [27:08<17:45,  1.68s/it]
                                                     

 95%|█████████▍| 11367/12000 [27:08<17:45,  1.68s/it]
 95%|█████████▍| 11368/12000 [27:10<17:15,  1.64s/it]
                                                     

 95%|█████████▍| 11368/12000 [27:10<17:15,  1.64s/it]
 95%|█████████▍| 11369/12000 [27:14<25:54,  2.46s/it]
                                                     

 95%|█████████▍| 11369/12000 [27:14<25:54,  2.46s/it]
 95%|█████████▍| 11370/12000 [27:16<24:02,  2.29s/it]
                                                     

 95%|█████████▍| 11370/12000 [27:16<24:02,  2.29s/it]
 95%|█████████▍| 11371/12000 [27:20<29:19,  2.80s/it]
                                                     

 95%|█████████▍| 11371/12000 [27:20<29:19,  2.80s/it]
 95%|█████████▍| 11372/12000 [27:22<28:17,  2.70s/it]
                                                     

 95%|█████████▍| 11372/12000 [27:22<28:17,  2.70s/it]
 95%|█████████▍| 11373/12000 [27:24<26:04,  2.50s/it]
                                                     

 95%|█████████▍| 11373/12000 [27:24<26:04,  2.50s/it]
 95%|█████████▍| 11374/12000 [27:27<26:02,  2.50s/it]
                                                     

 95%|█████████▍| 11374/12000 [27:27<26:02,  2.50s/it]
 95%|█████████▍| 11375/12000 [27:29<25:14,  2.42s/it]
                                                     

 95%|█████████▍| 11375/12000 [27:29<25:14,  2.42s/it]
 95%|█████████▍| 11376/12000 [27:31<24:33,  2.36s/it]
                                                     

 95%|█████████▍| 11376/12000 [27:31<24:33,  2.36s/it]
 95%|█████████▍| 11377/12000 [27:33<21:46,  2.10s/it]
                                                     

 95%|█████████▍| 11377/12000 [27:33<21:46,  2.10s/it]
 95%|█████████▍| 11378/12000 [27:34<20:26,  1.97s/it]
                                                     

 95%|█████████▍| 11378/12000 [27:34<20:26,  1.97s/it]
 95%|█████████▍| 11379/12000 [27:36<20:08,  1.95s/it]
                                                     

 95%|█████████▍| 11379/12000 [27:36<20:08,  1.95s/it]
 95%|█████████▍| 11380/12000 [27:38<19:06,  1.85s/it]
                                                     

 95%|█████████▍| 11380/12000 [27:38<19:06,  1.85s/it]
 95%|█████████▍| 11381/12000 [27:42<25:14,  2.45s/it]
                                                     

 95%|█████████▍| 11381/12000 [27:42<25:14,  2.45s/it]
 95%|█████████▍| 11382/12000 [27:43<22:45,  2.21s/it]
                                                     

 95%|█████████▍| 11382/12000 [27:43<22:45,  2.21s/it]
 95%|█████████▍| 11383/12000 [27:45<20:38,  2.01s/it]
                                                     

 95%|█████████▍| 11383/12000 [27:45<20:38,  2.01s/it]
 95%|█████████▍| 11384/12000 [27:49<25:29,  2.48s/it]
                                                     

 95%|█████████▍| 11384/12000 [27:49<25:29,  2.48s/it]
 95%|█████████▍| 11385/12000 [27:50<23:32,  2.30s/it]
                                                     

 95%|█████████▍| 11385/12000 [27:50<23:32,  2.30s/it]
 95%|█████████▍| 11386/12000 [27:52<21:28,  2.10s/it]
                                                     

 95%|█████████▍| 11386/12000 [27:52<21:28,  2.10s/it]
 95%|█████████▍| 11387/12000 [27:54<20:08,  1.97s/it]
                                                     

 95%|█████████▍| 11387/12000 [27:54<20:08,  1.97s/it]
 95%|█████████▍| 11388/12000 [27:55<19:05,  1.87s/it]
                                                     

 95%|█████████▍| 11388/12000 [27:55<19:05,  1.87s/it]
 95%|█████████▍| 11389/12000 [27:58<20:06,  1.97s/it]
                                                     

 95%|█████████▍| 11389/12000 [27:58<20:06,  1.97s/it]
 95%|█████████▍| 11390/12000 [28:02<28:00,  2.75s/it]
                                                     

 95%|█████████▍| 11390/12000 [28:02<28:00,  2.75s/it]
 95%|█████████▍| 11391/12000 [28:05<27:14,  2.68s/it]
                                                     

 95%|█████████▍| 11391/12000 [28:05<27:14,  2.68s/it]
 95%|█████████▍| 11392/12000 [28:07<26:33,  2.62s/it]
                                                     

 95%|█████████▍| 11392/12000 [28:07<26:33,  2.62s/it]
 95%|█████████▍| 11393/12000 [28:09<25:18,  2.50s/it]
                                                     

 95%|█████████▍| 11393/12000 [28:09<25:18,  2.50s/it]
 95%|█████████▍| 11394/12000 [28:11<22:34,  2.24s/it]
                                                     

 95%|█████████▍| 11394/12000 [28:11<22:34,  2.24s/it]
 95%|█████████▍| 11395/12000 [28:13<22:15,  2.21s/it]
                                                     

 95%|█████████▍| 11395/12000 [28:13<22:15,  2.21s/it]
 95%|█████████▍| 11396/12000 [28:16<22:55,  2.28s/it]
                                                     

 95%|█████████▍| 11396/12000 [28:16<22:55,  2.28s/it]
 95%|█████████▍| 11397/12000 [28:17<20:19,  2.02s/it]
                                                     

 95%|█████████▍| 11397/12000 [28:17<20:19,  2.02s/it]
 95%|█████████▍| 11398/12000 [28:19<19:45,  1.97s/it]
                                                     

 95%|█████████▍| 11398/12000 [28:19<19:45,  1.97s/it]
 95%|█████████▍| 11399/12000 [28:26<35:56,  3.59s/it]
                                                     

 95%|█████████▍| 11399/12000 [28:26<35:56,  3.59s/it]
 95%|█████████▌| 11400/12000 [28:29<33:05,  3.31s/it]
                                                     
{'loss': '0.6978', 'grad_norm': '0.3373', 'learning_rate': '1.59e-06', 'entropy': '0.782', 'num_tokens': '4.044e+06', 'mean_token_accuracy': '0.7995', 'epoch': '0.1005'}
{'loss': '0.8964', 'grad_norm': '0.3436', 'learning_rate': '1.585e-06', 'entropy': '0.8945', 'num_tokens': '4.051e+06', 'mean_token_accuracy': '0.789', 'epoch': '0.1005'}
{'loss': '0.7605', 'grad_norm': '0.4719', 'learning_rate': '1.58e-06', 'entropy': '0.9061', 'num_tokens': '4.055e+06', 'mean_token_accuracy': '0.7852', 'epoch': '0.1005'}
{'loss': '0.8058', 'grad_norm': '0.4155', 'learning_rate': '1.576e-06', 'entropy': '0.8625', 'num_tokens': '4.059e+06', 'mean_token_accuracy': '0.783', 'epoch': '0.1005'}
{'loss': '1.027', 'grad_norm': '0.4156', 'learning_rate': '1.571e-06', 'entropy': '1.14', 'num_tokens': '4.063e+06', 'mean_token_accuracy': '0.7269', 'epoch': '0.1005'}
{'loss': '0.9155', 'grad_norm': '0.3806', 'learning_rate': '1.566e-06', 'entropy': '0.876', 'num_tokens': '4.068e+06', 'mean_token_accuracy': '0.7952', 'epoch': '0.1005'}
{'loss': '1.124', 'grad_norm': '0.5333', 'learning_rate': '1.561e-06', 'entropy': '1.059', 'num_tokens': '4.074e+06', 'mean_token_accuracy': '0.7461', 'epoch': '0.1005'}
{'loss': '0.7268', 'grad_norm': '0.409', 'learning_rate': '1.556e-06', 'entropy': '0.7144', 'num_tokens': '4.08e+06', 'mean_token_accuracy': '0.8116', 'epoch': '0.1005'}
{'loss': '0.649', 'grad_norm': '0.4325', 'learning_rate': '1.551e-06', 'entropy': '0.654', 'num_tokens': '4.084e+06', 'mean_token_accuracy': '0.8361', 'epoch': '0.1005'}
{'loss': '0.7906', 'grad_norm': '0.4966', 'learning_rate': '1.546e-06', 'entropy': '0.7999', 'num_tokens': '4.089e+06', 'mean_token_accuracy': '0.8055', 'epoch': '0.1006'}
{'loss': '0.8645', 'grad_norm': '0.3931', 'learning_rate': '1.542e-06', 'entropy': '0.9285', 'num_tokens': '4.093e+06', 'mean_token_accuracy': '0.7701', 'epoch': '0.1006'}
{'loss': '0.674', 'grad_norm': '0.3535', 'learning_rate': '1.537e-06', 'entropy': '0.6978', 'num_tokens': '4.099e+06', 'mean_token_accuracy': '0.8219', 'epoch': '0.1006'}
{'loss': '1.179', 'grad_norm': '0.514', 'learning_rate': '1.532e-06', 'entropy': '1.091', 'num_tokens': '4.103e+06', 'mean_token_accuracy': '0.7332', 'epoch': '0.1006'}
{'loss': '0.6334', 'grad_norm': '0.4781', 'learning_rate': '1.527e-06', 'entropy': '0.6415', 'num_tokens': '4.108e+06', 'mean_token_accuracy': '0.8307', 'epoch': '0.1006'}
{'loss': '0.7439', 'grad_norm': '0.4677', 'learning_rate': '1.522e-06', 'entropy': '0.7871', 'num_tokens': '4.113e+06', 'mean_token_accuracy': '0.7886', 'epoch': '0.1006'}
{'loss': '1.214', 'grad_norm': '0.4449', 'learning_rate': '1.518e-06', 'entropy': '1.181', 'num_tokens': '4.117e+06', 'mean_token_accuracy': '0.7265', 'epoch': '0.1006'}
{'loss': '0.9158', 'grad_norm': '0.4391', 'learning_rate': '1.513e-06', 'entropy': '0.8343', 'num_tokens': '4.124e+06', 'mean_token_accuracy': '0.7902', 'epoch': '0.1006'}
{'loss': '1.162', 'grad_norm': '0.4393', 'learning_rate': '1.508e-06', 'entropy': '1.034', 'num_tokens': '4.129e+06', 'mean_token_accuracy': '0.7486', 'epoch': '0.1006'}
{'loss': '1.493', 'grad_norm': '0.3857', 'learning_rate': '1.503e-06', 'entropy': '1.24', 'num_tokens': '4.136e+06', 'mean_token_accuracy': '0.7072', 'epoch': '0.1006'}
{'loss': '0.7069', 'grad_norm': '0.3069', 'learning_rate': '1.499e-06', 'entropy': '0.7218', 'num_tokens': '4.143e+06', 'mean_token_accuracy': '0.818', 'epoch': '0.1006'}
{'loss': '0.5003', 'grad_norm': '0.354', 'learning_rate': '1.494e-06', 'entropy': '0.5404', 'num_tokens': '4.15e+06', 'mean_token_accuracy': '0.8471', 'epoch': '0.1006'}
{'loss': '1.189', 'grad_norm': '0.4984', 'learning_rate': '1.489e-06', 'entropy': '1.131', 'num_tokens': '4.157e+06', 'mean_token_accuracy': '0.7205', 'epoch': '0.1007'}
{'loss': '0.8014', 'grad_norm': '0.3593', 'learning_rate': '1.484e-06', 'entropy': '0.7921', 'num_tokens': '4.163e+06', 'mean_token_accuracy': '0.8043', 'epoch': '0.1007'}
{'loss': '0.8401', 'grad_norm': '0.4368', 'learning_rate': '1.48e-06', 'entropy': '0.8112', 'num_tokens': '4.168e+06', 'mean_token_accuracy': '0.7979', 'epoch': '0.1007'}
{'loss': '0.4338', 'grad_norm': '0.3014', 'learning_rate': '1.475e-06', 'entropy': '0.4196', 'num_tokens': '4.173e+06', 'mean_token_accuracy': '0.8861', 'epoch': '0.1007'}
{'loss': '0.8386', 'grad_norm': '0.3849', 'learning_rate': '1.47e-06', 'entropy': '0.8548', 'num_tokens': '4.177e+06', 'mean_token_accuracy': '0.7849', 'epoch': '0.1007'}
{'loss': '0.9751', 'grad_norm': '0.4107', 'learning_rate': '1.465e-06', 'entropy': '0.981', 'num_tokens': '4.183e+06', 'mean_token_accuracy': '0.7575', 'epoch': '0.1007'}
{'loss': '0.8443', 'grad_norm': '0.4357', 'learning_rate': '1.461e-06', 'entropy': '0.8509', 'num_tokens': '4.188e+06', 'mean_token_accuracy': '0.7858', 'epoch': '0.1007'}
{'loss': '0.8462', 'grad_norm': '0.3618', 'learning_rate': '1.456e-06', 'entropy': '0.8725', 'num_tokens': '4.196e+06', 'mean_token_accuracy': '0.787', 'epoch': '0.1007'}
{'loss': '0.8187', 'grad_norm': '0.38', 'learning_rate': '1.451e-06', 'entropy': '0.8395', 'num_tokens': '4.202e+06', 'mean_token_accuracy': '0.792', 'epoch': '0.1007'}
{'loss': '0.606', 'grad_norm': '0.4083', 'learning_rate': '1.447e-06', 'entropy': '0.6511', 'num_tokens': '4.206e+06', 'mean_token_accuracy': '0.8337', 'epoch': '0.1007'}
{'loss': '0.7396', 'grad_norm': '0.346', 'learning_rate': '1.442e-06', 'entropy': '0.7976', 'num_tokens': '4.215e+06', 'mean_token_accuracy': '0.8052', 'epoch': '0.1007'}
{'loss': '0.972', 'grad_norm': '0.3263', 'learning_rate': '1.437e-06', 'entropy': '0.8931', 'num_tokens': '4.221e+06', 'mean_token_accuracy': '0.7799', 'epoch': '0.1008'}
{'loss': '0.6383', 'grad_norm': '0.4742', 'learning_rate': '1.433e-06', 'entropy': '0.6817', 'num_tokens': '4.225e+06', 'mean_token_accuracy': '0.8042', 'epoch': '0.1008'}
{'loss': '0.949', 'grad_norm': '0.4848', 'learning_rate': '1.428e-06', 'entropy': '0.9863', 'num_tokens': '4.23e+06', 'mean_token_accuracy': '0.7626', 'epoch': '0.1008'}
{'loss': '0.9891', 'grad_norm': '0.4105', 'learning_rate': '1.423e-06', 'entropy': '0.9854', 'num_tokens': '4.236e+06', 'mean_token_accuracy': '0.7551', 'epoch': '0.1008'}
{'loss': '0.6167', 'grad_norm': '0.3774', 'learning_rate': '1.419e-06', 'entropy': '0.7084', 'num_tokens': '4.241e+06', 'mean_token_accuracy': '0.8177', 'epoch': '0.1008'}
{'loss': '0.5259', 'grad_norm': '0.3736', 'learning_rate': '1.414e-06', 'entropy': '0.5596', 'num_tokens': '4.246e+06', 'mean_token_accuracy': '0.85', 'epoch': '0.1008'}
{'loss': '1.016', 'grad_norm': '0.4671', 'learning_rate': '1.41e-06', 'entropy': '0.9264', 'num_tokens': '4.252e+06', 'mean_token_accuracy': '0.773', 'epoch': '0.1008'}
{'loss': '0.8366', 'grad_norm': '0.4721', 'learning_rate': '1.405e-06', 'entropy': '0.7598', 'num_tokens': '4.258e+06', 'mean_token_accuracy': '0.8108', 'epoch': '0.1008'}
{'loss': '0.5456', 'grad_norm': '0.3452', 'learning_rate': '1.4e-06', 'entropy': '0.5594', 'num_tokens': '4.264e+06', 'mean_token_accuracy': '0.8495', 'epoch': '0.1008'}
{'loss': '0.5907', 'grad_norm': '0.3854', 'learning_rate': '1.396e-06', 'entropy': '0.7066', 'num_tokens': '4.268e+06', 'mean_token_accuracy': '0.8316', 'epoch': '0.1008'}
{'loss': '0.832', 'grad_norm': '0.4182', 'learning_rate': '1.391e-06', 'entropy': '0.8928', 'num_tokens': '4.273e+06', 'mean_token_accuracy': '0.7827', 'epoch': '0.1008'}
{'loss': '0.8963', 'grad_norm': '0.3991', 'learning_rate': '1.387e-06', 'entropy': '1.013', 'num_tokens': '4.28e+06', 'mean_token_accuracy': '0.7546', 'epoch': '0.1009'}
{'loss': '0.5324', 'grad_norm': '0.2849', 'learning_rate': '1.382e-06', 'entropy': '0.5902', 'num_tokens': '4.284e+06', 'mean_token_accuracy': '0.8417', 'epoch': '0.1009'}
{'loss': '1.439', 'grad_norm': '0.471', 'learning_rate': '1.378e-06', 'entropy': '1.336', 'num_tokens': '4.29e+06', 'mean_token_accuracy': '0.6939', 'epoch': '0.1009'}
{'loss': '0.7148', 'grad_norm': '0.2784', 'learning_rate': '1.373e-06', 'entropy': '0.6074', 'num_tokens': '4.301e+06', 'mean_token_accuracy': '0.8462', 'epoch': '0.1009'}

 95%|█████████▌| 11400/12000 [28:29<33:05,  3.31s/it]
 95%|█████████▌| 11401/12000 [28:33<36:31,  3.66s/it]
                                                     

 95%|█████████▌| 11401/12000 [28:33<36:31,  3.66s/it]
 95%|█████████▌| 11402/12000 [28:35<30:42,  3.08s/it]
                                                     

 95%|█████████▌| 11402/12000 [28:35<30:42,  3.08s/it]
 95%|█████████▌| 11403/12000 [28:37<28:24,  2.85s/it]
                                                     

 95%|█████████▌| 11403/12000 [28:37<28:24,  2.85s/it]
 95%|█████████▌| 11404/12000 [28:39<25:13,  2.54s/it]
                                                     

 95%|█████████▌| 11404/12000 [28:39<25:13,  2.54s/it]
 95%|█████████▌| 11405/12000 [28:41<22:24,  2.26s/it]
                                                     

 95%|█████████▌| 11405/12000 [28:41<22:24,  2.26s/it]
 95%|█████████▌| 11406/12000 [28:42<20:00,  2.02s/it]
                                                     

 95%|█████████▌| 11406/12000 [28:42<20:00,  2.02s/it]
 95%|█████████▌| 11407/12000 [28:44<18:12,  1.84s/it]
                                                     

 95%|█████████▌| 11407/12000 [28:44<18:12,  1.84s/it]
 95%|█████████▌| 11408/12000 [28:45<16:28,  1.67s/it]
                                                     

 95%|█████████▌| 11408/12000 [28:45<16:28,  1.67s/it]
 95%|█████████▌| 11409/12000 [28:47<18:51,  1.91s/it]
                                                     

 95%|█████████▌| 11409/12000 [28:47<18:51,  1.91s/it]
 95%|█████████▌| 11410/12000 [28:50<21:28,  2.18s/it]
                                                     

 95%|█████████▌| 11410/12000 [28:50<21:28,  2.18s/it]
 95%|█████████▌| 11411/12000 [28:52<20:26,  2.08s/it]
                                                     

 95%|█████████▌| 11411/12000 [28:52<20:26,  2.08s/it]
 95%|█████████▌| 11412/12000 [28:56<24:43,  2.52s/it]
                                                     

 95%|█████████▌| 11412/12000 [28:56<24:43,  2.52s/it]
 95%|█████████▌| 11413/12000 [28:57<22:15,  2.28s/it]
                                                     

 95%|█████████▌| 11413/12000 [28:57<22:15,  2.28s/it]
 95%|█████████▌| 11414/12000 [29:03<31:42,  3.25s/it]
                                                     

 95%|█████████▌| 11414/12000 [29:03<31:42,  3.25s/it]
 95%|█████████▌| 11415/12000 [29:04<26:50,  2.75s/it]
                                                     

 95%|█████████▌| 11415/12000 [29:04<26:50,  2.75s/it]
 95%|█████████▌| 11416/12000 [29:06<23:35,  2.42s/it]
                                                     

 95%|█████████▌| 11416/12000 [29:06<23:35,  2.42s/it]
 95%|█████████▌| 11417/12000 [29:08<20:37,  2.12s/it]
                                                     

 95%|█████████▌| 11417/12000 [29:08<20:37,  2.12s/it]
 95%|█████████▌| 11418/12000 [29:10<21:41,  2.24s/it]
                                                     

 95%|█████████▌| 11418/12000 [29:10<21:41,  2.24s/it]
 95%|█████████▌| 11419/12000 [29:12<19:45,  2.04s/it]
                                                     

 95%|█████████▌| 11419/12000 [29:12<19:45,  2.04s/it]
 95%|█████████▌| 11420/12000 [29:14<20:11,  2.09s/it]
                                                     

 95%|█████████▌| 11420/12000 [29:14<20:11,  2.09s/it]
 95%|█████████▌| 11421/12000 [29:16<20:39,  2.14s/it]
                                                     

 95%|█████████▌| 11421/12000 [29:16<20:39,  2.14s/it]
 95%|█████████▌| 11422/12000 [29:17<18:27,  1.92s/it]
                                                     

 95%|█████████▌| 11422/12000 [29:17<18:27,  1.92s/it]
 95%|█████████▌| 11423/12000 [29:19<16:42,  1.74s/it]
                                                     

 95%|█████████▌| 11423/12000 [29:19<16:42,  1.74s/it]
 95%|█████████▌| 11424/12000 [29:21<16:57,  1.77s/it]
                                                     

 95%|█████████▌| 11424/12000 [29:21<16:57,  1.77s/it]
 95%|█████████▌| 11425/12000 [29:23<18:12,  1.90s/it]
                                                     

 95%|█████████▌| 11425/12000 [29:23<18:12,  1.90s/it]
 95%|█████████▌| 11426/12000 [29:25<19:12,  2.01s/it]
                                                     

 95%|█████████▌| 11426/12000 [29:25<19:12,  2.01s/it]
 95%|█████████▌| 11427/12000 [29:27<18:35,  1.95s/it]
                                                     

 95%|█████████▌| 11427/12000 [29:27<18:35,  1.95s/it]
 95%|█████████▌| 11428/12000 [29:28<17:33,  1.84s/it]
                                                     

 95%|█████████▌| 11428/12000 [29:28<17:33,  1.84s/it]
 95%|█████████▌| 11429/12000 [29:30<16:46,  1.76s/it]
                                                     

 95%|█████████▌| 11429/12000 [29:30<16:46,  1.76s/it]
 95%|█████████▌| 11430/12000 [29:32<16:06,  1.70s/it]
                                                     

 95%|█████████▌| 11430/12000 [29:32<16:06,  1.70s/it]
 95%|█████████▌| 11431/12000 [29:33<15:59,  1.69s/it]
                                                     

 95%|█████████▌| 11431/12000 [29:33<15:59,  1.69s/it]
 95%|█████████▌| 11432/12000 [29:35<15:56,  1.68s/it]
                                                     

 95%|█████████▌| 11432/12000 [29:35<15:56,  1.68s/it]
 95%|█████████▌| 11433/12000 [29:37<17:07,  1.81s/it]
                                                     

 95%|█████████▌| 11433/12000 [29:37<17:07,  1.81s/it]
 95%|█████████▌| 11434/12000 [29:41<22:16,  2.36s/it]
                                                     

 95%|█████████▌| 11434/12000 [29:41<22:16,  2.36s/it]
 95%|█████████▌| 11435/12000 [29:42<20:20,  2.16s/it]
                                                     

 95%|█████████▌| 11435/12000 [29:42<20:20,  2.16s/it]
 95%|█████████▌| 11436/12000 [29:44<19:30,  2.08s/it]
                                                     

 95%|█████████▌| 11436/12000 [29:44<19:30,  2.08s/it]
 95%|█████████▌| 11437/12000 [29:46<18:14,  1.94s/it]
                                                     

 95%|█████████▌| 11437/12000 [29:46<18:14,  1.94s/it]
 95%|█████████▌| 11438/12000 [29:48<17:48,  1.90s/it]
                                                     

 95%|█████████▌| 11438/12000 [29:48<17:48,  1.90s/it]
 95%|█████████▌| 11439/12000 [29:49<16:50,  1.80s/it]
                                                     

 95%|█████████▌| 11439/12000 [29:49<16:50,  1.80s/it]
 95%|█████████▌| 11440/12000 [29:52<19:55,  2.13s/it]
                                                     

 95%|█████████▌| 11440/12000 [29:52<19:55,  2.13s/it]
 95%|█████████▌| 11441/12000 [29:54<20:16,  2.18s/it]
                                                     

 95%|█████████▌| 11441/12000 [29:54<20:16,  2.18s/it]
 95%|█████████▌| 11442/12000 [29:57<22:26,  2.41s/it]
                                                     

 95%|█████████▌| 11442/12000 [29:57<22:26,  2.41s/it]
 95%|█████████▌| 11443/12000 [29:59<20:54,  2.25s/it]
                                                     

 95%|█████████▌| 11443/12000 [29:59<20:54,  2.25s/it]
 95%|█████████▌| 11444/12000 [30:02<20:53,  2.25s/it]
                                                     

 95%|█████████▌| 11444/12000 [30:02<20:53,  2.25s/it]
 95%|█████████▌| 11445/12000 [30:03<18:33,  2.01s/it]
                                                     

 95%|█████████▌| 11445/12000 [30:03<18:33,  2.01s/it]
 95%|█████████▌| 11446/12000 [30:09<29:25,  3.19s/it]
                                                     

 95%|█████████▌| 11446/12000 [30:09<29:25,  3.19s/it]
 95%|█████████▌| 11447/12000 [30:11<27:31,  2.99s/it]
                                                     
{'loss': '0.6504', 'grad_norm': '0.3791', 'learning_rate': '1.368e-06', 'entropy': '0.7872', 'num_tokens': '4.308e+06', 'mean_token_accuracy': '0.8068', 'epoch': '0.1009'}
{'loss': '0.8236', 'grad_norm': '0.424', 'learning_rate': '1.364e-06', 'entropy': '0.7626', 'num_tokens': '4.314e+06', 'mean_token_accuracy': '0.7981', 'epoch': '0.1009'}
{'loss': '0.5951', 'grad_norm': '0.3725', 'learning_rate': '1.359e-06', 'entropy': '0.5874', 'num_tokens': '4.319e+06', 'mean_token_accuracy': '0.8484', 'epoch': '0.1009'}
{'loss': '0.855', 'grad_norm': '0.4028', 'learning_rate': '1.355e-06', 'entropy': '0.9419', 'num_tokens': '4.325e+06', 'mean_token_accuracy': '0.7692', 'epoch': '0.1009'}
{'loss': '0.782', 'grad_norm': '0.487', 'learning_rate': '1.35e-06', 'entropy': '0.7926', 'num_tokens': '4.331e+06', 'mean_token_accuracy': '0.7941', 'epoch': '0.1009'}
{'loss': '0.6096', 'grad_norm': '0.3754', 'learning_rate': '1.346e-06', 'entropy': '0.6792', 'num_tokens': '4.336e+06', 'mean_token_accuracy': '0.8301', 'epoch': '0.1009'}
{'loss': '0.7544', 'grad_norm': '0.3827', 'learning_rate': '1.341e-06', 'entropy': '0.8063', 'num_tokens': '4.339e+06', 'mean_token_accuracy': '0.7909', 'epoch': '0.1009'}
{'loss': '0.6211', 'grad_norm': '0.392', 'learning_rate': '1.337e-06', 'entropy': '0.6157', 'num_tokens': '4.343e+06', 'mean_token_accuracy': '0.8342', 'epoch': '0.101'}
{'loss': '0.8139', 'grad_norm': '0.4541', 'learning_rate': '1.332e-06', 'entropy': '0.8817', 'num_tokens': '4.346e+06', 'mean_token_accuracy': '0.7735', 'epoch': '0.101'}
{'loss': '0.8346', 'grad_norm': '0.422', 'learning_rate': '1.328e-06', 'entropy': '1.006', 'num_tokens': '4.351e+06', 'mean_token_accuracy': '0.7646', 'epoch': '0.101'}
{'loss': '0.5581', 'grad_norm': '0.2979', 'learning_rate': '1.323e-06', 'entropy': '0.5116', 'num_tokens': '4.358e+06', 'mean_token_accuracy': '0.8605', 'epoch': '0.101'}
{'loss': '0.8102', 'grad_norm': '0.4611', 'learning_rate': '1.319e-06', 'entropy': '0.816', 'num_tokens': '4.363e+06', 'mean_token_accuracy': '0.7982', 'epoch': '0.101'}
{'loss': '0.8779', 'grad_norm': '0.4724', 'learning_rate': '1.314e-06', 'entropy': '0.8385', 'num_tokens': '4.367e+06', 'mean_token_accuracy': '0.7977', 'epoch': '0.101'}
{'loss': '0.9971', 'grad_norm': '0.3869', 'learning_rate': '1.31e-06', 'entropy': '0.9886', 'num_tokens': '4.373e+06', 'mean_token_accuracy': '0.7656', 'epoch': '0.101'}
{'loss': '0.9846', 'grad_norm': '0.352', 'learning_rate': '1.306e-06', 'entropy': '0.734', 'num_tokens': '4.383e+06', 'mean_token_accuracy': '0.8383', 'epoch': '0.101'}
{'loss': '0.5535', 'grad_norm': '0.4186', 'learning_rate': '1.301e-06', 'entropy': '0.5613', 'num_tokens': '4.387e+06', 'mean_token_accuracy': '0.838', 'epoch': '0.101'}
{'loss': '0.6952', 'grad_norm': '0.4271', 'learning_rate': '1.297e-06', 'entropy': '0.676', 'num_tokens': '4.392e+06', 'mean_token_accuracy': '0.8183', 'epoch': '0.101'}
{'loss': '0.612', 'grad_norm': '0.3541', 'learning_rate': '1.292e-06', 'entropy': '0.6132', 'num_tokens': '4.396e+06', 'mean_token_accuracy': '0.8382', 'epoch': '0.101'}
{'loss': '0.6187', 'grad_norm': '0.3894', 'learning_rate': '1.288e-06', 'entropy': '0.6383', 'num_tokens': '4.402e+06', 'mean_token_accuracy': '0.8238', 'epoch': '0.101'}
{'loss': '0.778', 'grad_norm': '0.4367', 'learning_rate': '1.283e-06', 'entropy': '0.8304', 'num_tokens': '4.406e+06', 'mean_token_accuracy': '0.7942', 'epoch': '0.1011'}
{'loss': '0.644', 'grad_norm': '0.3192', 'learning_rate': '1.279e-06', 'entropy': '0.6057', 'num_tokens': '4.413e+06', 'mean_token_accuracy': '0.8498', 'epoch': '0.1011'}
{'loss': '0.6757', 'grad_norm': '0.3486', 'learning_rate': '1.275e-06', 'entropy': '0.7371', 'num_tokens': '4.418e+06', 'mean_token_accuracy': '0.8236', 'epoch': '0.1011'}
{'loss': '0.7395', 'grad_norm': '0.4878', 'learning_rate': '1.27e-06', 'entropy': '0.7979', 'num_tokens': '4.422e+06', 'mean_token_accuracy': '0.7902', 'epoch': '0.1011'}
{'loss': '0.8509', 'grad_norm': '0.4082', 'learning_rate': '1.266e-06', 'entropy': '0.8339', 'num_tokens': '4.425e+06', 'mean_token_accuracy': '0.786', 'epoch': '0.1011'}
{'loss': '0.7542', 'grad_norm': '0.3664', 'learning_rate': '1.262e-06', 'entropy': '0.7759', 'num_tokens': '4.431e+06', 'mean_token_accuracy': '0.8068', 'epoch': '0.1011'}
{'loss': '0.7378', 'grad_norm': '0.3915', 'learning_rate': '1.257e-06', 'entropy': '0.7435', 'num_tokens': '4.437e+06', 'mean_token_accuracy': '0.8212', 'epoch': '0.1011'}
{'loss': '0.5271', 'grad_norm': '0.3839', 'learning_rate': '1.253e-06', 'entropy': '0.6883', 'num_tokens': '4.442e+06', 'mean_token_accuracy': '0.8357', 'epoch': '0.1011'}
{'loss': '0.724', 'grad_norm': '0.3678', 'learning_rate': '1.248e-06', 'entropy': '0.7014', 'num_tokens': '4.447e+06', 'mean_token_accuracy': '0.8182', 'epoch': '0.1011'}
{'loss': '0.5695', 'grad_norm': '0.4071', 'learning_rate': '1.244e-06', 'entropy': '0.6087', 'num_tokens': '4.452e+06', 'mean_token_accuracy': '0.8466', 'epoch': '0.1011'}
{'loss': '0.8604', 'grad_norm': '0.3926', 'learning_rate': '1.24e-06', 'entropy': '0.7994', 'num_tokens': '4.457e+06', 'mean_token_accuracy': '0.8082', 'epoch': '0.1011'}
{'loss': '0.5398', 'grad_norm': '0.3384', 'learning_rate': '1.235e-06', 'entropy': '0.6157', 'num_tokens': '4.462e+06', 'mean_token_accuracy': '0.8357', 'epoch': '0.1012'}
{'loss': '0.695', 'grad_norm': '0.3571', 'learning_rate': '1.231e-06', 'entropy': '0.7009', 'num_tokens': '4.468e+06', 'mean_token_accuracy': '0.8118', 'epoch': '0.1012'}
{'loss': '0.8604', 'grad_norm': '0.4478', 'learning_rate': '1.227e-06', 'entropy': '0.805', 'num_tokens': '4.473e+06', 'mean_token_accuracy': '0.7831', 'epoch': '0.1012'}
{'loss': '0.6628', 'grad_norm': '0.3501', 'learning_rate': '1.223e-06', 'entropy': '0.6572', 'num_tokens': '4.479e+06', 'mean_token_accuracy': '0.8307', 'epoch': '0.1012'}
{'loss': '0.956', 'grad_norm': '0.4546', 'learning_rate': '1.218e-06', 'entropy': '0.9036', 'num_tokens': '4.483e+06', 'mean_token_accuracy': '0.7705', 'epoch': '0.1012'}
{'loss': '0.6503', 'grad_norm': '0.4425', 'learning_rate': '1.214e-06', 'entropy': '0.6334', 'num_tokens': '4.488e+06', 'mean_token_accuracy': '0.8275', 'epoch': '0.1012'}
{'loss': '1.099', 'grad_norm': '0.3905', 'learning_rate': '1.21e-06', 'entropy': '1.135', 'num_tokens': '4.494e+06', 'mean_token_accuracy': '0.7312', 'epoch': '0.1012'}
{'loss': '0.7938', 'grad_norm': '0.435', 'learning_rate': '1.205e-06', 'entropy': '0.7952', 'num_tokens': '4.499e+06', 'mean_token_accuracy': '0.8015', 'epoch': '0.1012'}
{'loss': '0.7016', 'grad_norm': '0.3612', 'learning_rate': '1.201e-06', 'entropy': '0.7075', 'num_tokens': '4.504e+06', 'mean_token_accuracy': '0.8183', 'epoch': '0.1012'}
{'loss': '0.6934', 'grad_norm': '0.3274', 'learning_rate': '1.197e-06', 'entropy': '0.7386', 'num_tokens': '4.51e+06', 'mean_token_accuracy': '0.8069', 'epoch': '0.1012'}
{'loss': '0.8595', 'grad_norm': '0.3358', 'learning_rate': '1.193e-06', 'entropy': '0.8262', 'num_tokens': '4.516e+06', 'mean_token_accuracy': '0.7914', 'epoch': '0.1012'}
{'loss': '0.8694', 'grad_norm': '0.4136', 'learning_rate': '1.188e-06', 'entropy': '0.8238', 'num_tokens': '4.523e+06', 'mean_token_accuracy': '0.7919', 'epoch': '0.1013'}
{'loss': '0.7903', 'grad_norm': '0.3622', 'learning_rate': '1.184e-06', 'entropy': '0.8195', 'num_tokens': '4.529e+06', 'mean_token_accuracy': '0.7922', 'epoch': '0.1013'}
{'loss': '0.8511', 'grad_norm': '0.4588', 'learning_rate': '1.18e-06', 'entropy': '0.8311', 'num_tokens': '4.533e+06', 'mean_token_accuracy': '0.7918', 'epoch': '0.1013'}
{'loss': '0.5613', 'grad_norm': '0.32', 'learning_rate': '1.176e-06', 'entropy': '0.5742', 'num_tokens': '4.538e+06', 'mean_token_accuracy': '0.8472', 'epoch': '0.1013'}
{'loss': '1.084', 'grad_norm': '0.4219', 'learning_rate': '1.172e-06', 'entropy': '1.027', 'num_tokens': '4.542e+06', 'mean_token_accuracy': '0.7405', 'epoch': '0.1013'}
{'loss': '1.305', 'grad_norm': '0.3704', 'learning_rate': '1.167e-06', 'entropy': '1.134', 'num_tokens': '4.555e+06', 'mean_token_accuracy': '0.7326', 'epoch': '0.1013'}

 95%|█████████▌| 11447/12000 [30:11<27:31,  2.99s/it]
 95%|█████████▌| 11448/12000 [30:13<23:50,  2.59s/it]
                                                     

 95%|█████████▌| 11448/12000 [30:13<23:50,  2.59s/it]
 95%|█████████▌| 11449/12000 [30:16<25:21,  2.76s/it]
                                                     

 95%|█████████▌| 11449/12000 [30:16<25:21,  2.76s/it]
 95%|█████████▌| 11450/12000 [30:18<22:48,  2.49s/it]
                                                     

 95%|█████████▌| 11450/12000 [30:18<22:48,  2.49s/it]
 95%|█████████▌| 11451/12000 [30:21<23:50,  2.61s/it]
                                                     

 95%|█████████▌| 11451/12000 [30:21<23:50,  2.61s/it]
 95%|█████████▌| 11452/12000 [30:23<22:45,  2.49s/it]
                                                     

 95%|█████████▌| 11452/12000 [30:23<22:45,  2.49s/it]
 95%|█████████▌| 11453/12000 [30:25<20:26,  2.24s/it]
                                                     

 95%|█████████▌| 11453/12000 [30:25<20:26,  2.24s/it]
 95%|█████████▌| 11454/12000 [30:26<18:36,  2.04s/it]
                                                     

 95%|█████████▌| 11454/12000 [30:26<18:36,  2.04s/it]
 95%|█████████▌| 11455/12000 [30:28<16:52,  1.86s/it]
                                                     

 95%|█████████▌| 11455/12000 [30:28<16:52,  1.86s/it]
 95%|█████████▌| 11456/12000 [30:32<23:55,  2.64s/it]
                                                     

 95%|█████████▌| 11456/12000 [30:32<23:55,  2.64s/it]
 95%|█████████▌| 11457/12000 [30:35<23:52,  2.64s/it]
                                                     

 95%|█████████▌| 11457/12000 [30:35<23:52,  2.64s/it]
 95%|█████████▌| 11458/12000 [30:36<20:37,  2.28s/it]
                                                     

 95%|█████████▌| 11458/12000 [30:36<20:37,  2.28s/it]
 95%|█████████▌| 11459/12000 [30:39<20:24,  2.26s/it]
                                                     

 95%|█████████▌| 11459/12000 [30:39<20:24,  2.26s/it]
 96%|█████████▌| 11460/12000 [30:42<22:24,  2.49s/it]
                                                     

 96%|█████████▌| 11460/12000 [30:42<22:24,  2.49s/it]
 96%|█████████▌| 11461/12000 [30:43<20:16,  2.26s/it]
                                                     

 96%|█████████▌| 11461/12000 [30:43<20:16,  2.26s/it]
 96%|█████████▌| 11462/12000 [30:46<21:52,  2.44s/it]
                                                     

 96%|█████████▌| 11462/12000 [30:46<21:52,  2.44s/it]
 96%|█████████▌| 11463/12000 [30:48<19:24,  2.17s/it]
                                                     

 96%|█████████▌| 11463/12000 [30:48<19:24,  2.17s/it]
 96%|█████████▌| 11464/12000 [30:49<17:44,  1.99s/it]
                                                     

 96%|█████████▌| 11464/12000 [30:49<17:44,  1.99s/it]
 96%|█████████▌| 11465/12000 [30:51<16:29,  1.85s/it]
                                                     

 96%|█████████▌| 11465/12000 [30:51<16:29,  1.85s/it]
 96%|█████████▌| 11466/12000 [30:53<16:58,  1.91s/it]
                                                     

 96%|█████████▌| 11466/12000 [30:53<16:58,  1.91s/it]
 96%|█████████▌| 11467/12000 [30:56<19:53,  2.24s/it]
                                                     

 96%|█████████▌| 11467/12000 [30:56<19:53,  2.24s/it]
 96%|█████████▌| 11468/12000 [30:59<20:46,  2.34s/it]
                                                     

 96%|█████████▌| 11468/12000 [30:59<20:46,  2.34s/it]
 96%|█████████▌| 11469/12000 [31:00<19:16,  2.18s/it]
                                                     

 96%|█████████▌| 11469/12000 [31:00<19:16,  2.18s/it]
 96%|█████████▌| 11470/12000 [31:02<17:49,  2.02s/it]
                                                     

 96%|█████████▌| 11470/12000 [31:02<17:49,  2.02s/it]
 96%|█████████▌| 11471/12000 [31:04<16:38,  1.89s/it]
                                                     

 96%|█████████▌| 11471/12000 [31:04<16:38,  1.89s/it]
 96%|█████████▌| 11472/12000 [31:06<17:43,  2.01s/it]
                                                     

 96%|█████████▌| 11472/12000 [31:06<17:43,  2.01s/it]
 96%|█████████▌| 11473/12000 [31:09<19:56,  2.27s/it]
                                                     

 96%|█████████▌| 11473/12000 [31:09<19:56,  2.27s/it]
 96%|█████████▌| 11474/12000 [31:10<17:50,  2.04s/it]
                                                     

 96%|█████████▌| 11474/12000 [31:10<17:50,  2.04s/it]
 96%|█████████▌| 11475/12000 [31:12<16:38,  1.90s/it]
                                                     

 96%|█████████▌| 11475/12000 [31:12<16:38,  1.90s/it]
 96%|█████████▌| 11476/12000 [31:13<15:19,  1.75s/it]
                                                     

 96%|█████████▌| 11476/12000 [31:13<15:19,  1.75s/it]
 96%|█████████▌| 11477/12000 [31:15<15:19,  1.76s/it]
                                                     

 96%|█████████▌| 11477/12000 [31:15<15:19,  1.76s/it]
 96%|█████████▌| 11478/12000 [31:19<20:00,  2.30s/it]
                                                     

 96%|█████████▌| 11478/12000 [31:19<20:00,  2.30s/it]
 96%|█████████▌| 11479/12000 [31:20<18:52,  2.17s/it]
                                                     

 96%|█████████▌| 11479/12000 [31:20<18:52,  2.17s/it]
 96%|█████████▌| 11480/12000 [31:23<20:39,  2.38s/it]
                                                     

 96%|█████████▌| 11480/12000 [31:23<20:39,  2.38s/it]
 96%|█████████▌| 11481/12000 [31:27<23:58,  2.77s/it]
                                                     

 96%|█████████▌| 11481/12000 [31:27<23:58,  2.77s/it]
 96%|█████████▌| 11482/12000 [31:30<23:26,  2.72s/it]
                                                     

 96%|█████████▌| 11482/12000 [31:30<23:26,  2.72s/it]
 96%|█████████▌| 11483/12000 [31:31<20:05,  2.33s/it]
                                                     

 96%|█████████▌| 11483/12000 [31:31<20:05,  2.33s/it]
 96%|█████████▌| 11484/12000 [31:33<18:05,  2.10s/it]
                                                     

 96%|█████████▌| 11484/12000 [31:33<18:05,  2.10s/it]
 96%|█████████▌| 11485/12000 [31:34<16:35,  1.93s/it]
                                                     

 96%|█████████▌| 11485/12000 [31:34<16:35,  1.93s/it]
 96%|█████████▌| 11486/12000 [31:36<15:28,  1.81s/it]
                                                     

 96%|█████████▌| 11486/12000 [31:36<15:28,  1.81s/it]
 96%|█████████▌| 11487/12000 [31:38<16:18,  1.91s/it]
                                                     

 96%|█████████▌| 11487/12000 [31:38<16:18,  1.91s/it]
 96%|█████████▌| 11488/12000 [31:39<15:37,  1.83s/it]
                                                     

 96%|█████████▌| 11488/12000 [31:39<15:37,  1.83s/it]
 96%|█████████▌| 11489/12000 [31:42<16:34,  1.95s/it]
                                                     

 96%|█████████▌| 11489/12000 [31:42<16:34,  1.95s/it]
 96%|█████████▌| 11490/12000 [31:44<18:30,  2.18s/it]
                                                     

 96%|█████████▌| 11490/12000 [31:44<18:30,  2.18s/it]
 96%|█████████▌| 11491/12000 [31:46<17:38,  2.08s/it]
                                                     

 96%|█████████▌| 11491/12000 [31:46<17:38,  2.08s/it]
 96%|█████████▌| 11492/12000 [31:51<24:56,  2.95s/it]
                                                     

 96%|█████████▌| 11492/12000 [31:51<24:56,  2.95s/it]
 96%|█████████▌| 11493/12000 [31:53<22:05,  2.61s/it]
                                                     

 96%|█████████▌| 11493/12000 [31:53<22:05,  2.61s/it]
 96%|█████████▌| 11494/12000 [31:55<20:43,  2.46s/it]
                                                     
{'loss': '1.099', 'grad_norm': '0.4115', 'learning_rate': '1.163e-06', 'entropy': '1.138', 'num_tokens': '4.562e+06', 'mean_token_accuracy': '0.7254', 'epoch': '0.1013'}
{'loss': '0.6904', 'grad_norm': '0.44', 'learning_rate': '1.159e-06', 'entropy': '0.7262', 'num_tokens': '4.566e+06', 'mean_token_accuracy': '0.8165', 'epoch': '0.1013'}
{'loss': '0.4929', 'grad_norm': '0.2714', 'learning_rate': '1.155e-06', 'entropy': '0.6542', 'num_tokens': '4.573e+06', 'mean_token_accuracy': '0.8266', 'epoch': '0.1013'}
{'loss': '1.074', 'grad_norm': '0.4545', 'learning_rate': '1.151e-06', 'entropy': '0.9482', 'num_tokens': '4.578e+06', 'mean_token_accuracy': '0.7789', 'epoch': '0.1013'}
{'loss': '0.7502', 'grad_norm': '0.4508', 'learning_rate': '1.146e-06', 'entropy': '0.7941', 'num_tokens': '4.583e+06', 'mean_token_accuracy': '0.795', 'epoch': '0.1013'}
{'loss': '0.802', 'grad_norm': '0.3266', 'learning_rate': '1.142e-06', 'entropy': '0.8203', 'num_tokens': '4.59e+06', 'mean_token_accuracy': '0.7969', 'epoch': '0.1013'}
{'loss': '0.9732', 'grad_norm': '0.4906', 'learning_rate': '1.138e-06', 'entropy': '0.9281', 'num_tokens': '4.593e+06', 'mean_token_accuracy': '0.7557', 'epoch': '0.1014'}
{'loss': '0.7043', 'grad_norm': '0.3676', 'learning_rate': '1.134e-06', 'entropy': '0.6854', 'num_tokens': '4.598e+06', 'mean_token_accuracy': '0.8257', 'epoch': '0.1014'}
{'loss': '0.8', 'grad_norm': '0.425', 'learning_rate': '1.13e-06', 'entropy': '0.8182', 'num_tokens': '4.602e+06', 'mean_token_accuracy': '0.7976', 'epoch': '0.1014'}
{'loss': '0.7252', 'grad_norm': '0.4618', 'learning_rate': '1.126e-06', 'entropy': '0.8281', 'num_tokens': '4.607e+06', 'mean_token_accuracy': '0.7795', 'epoch': '0.1014'}
{'loss': '0.8538', 'grad_norm': '0.3862', 'learning_rate': '1.122e-06', 'entropy': '0.9511', 'num_tokens': '4.615e+06', 'mean_token_accuracy': '0.7713', 'epoch': '0.1014'}
{'loss': '0.5459', 'grad_norm': '0.3906', 'learning_rate': '1.118e-06', 'entropy': '0.5421', 'num_tokens': '4.619e+06', 'mean_token_accuracy': '0.8607', 'epoch': '0.1014'}
{'loss': '0.6656', 'grad_norm': '0.3246', 'learning_rate': '1.113e-06', 'entropy': '0.7487', 'num_tokens': '4.626e+06', 'mean_token_accuracy': '0.8165', 'epoch': '0.1014'}
{'loss': '1.088', 'grad_norm': '0.3809', 'learning_rate': '1.109e-06', 'entropy': '1.074', 'num_tokens': '4.633e+06', 'mean_token_accuracy': '0.7426', 'epoch': '0.1014'}
{'loss': '0.6876', 'grad_norm': '0.3335', 'learning_rate': '1.105e-06', 'entropy': '0.6344', 'num_tokens': '4.637e+06', 'mean_token_accuracy': '0.84', 'epoch': '0.1014'}
{'loss': '0.4446', 'grad_norm': '0.2786', 'learning_rate': '1.101e-06', 'entropy': '0.5394', 'num_tokens': '4.645e+06', 'mean_token_accuracy': '0.8607', 'epoch': '0.1014'}
{'loss': '0.686', 'grad_norm': '0.4325', 'learning_rate': '1.097e-06', 'entropy': '0.7132', 'num_tokens': '4.649e+06', 'mean_token_accuracy': '0.8187', 'epoch': '0.1014'}
{'loss': '0.6065', 'grad_norm': '0.4114', 'learning_rate': '1.093e-06', 'entropy': '0.6494', 'num_tokens': '4.653e+06', 'mean_token_accuracy': '0.8346', 'epoch': '0.1015'}
{'loss': '0.4956', 'grad_norm': '0.4092', 'learning_rate': '1.089e-06', 'entropy': '0.5522', 'num_tokens': '4.657e+06', 'mean_token_accuracy': '0.8499', 'epoch': '0.1015'}
{'loss': '0.9004', 'grad_norm': '0.4794', 'learning_rate': '1.085e-06', 'entropy': '0.8586', 'num_tokens': '4.663e+06', 'mean_token_accuracy': '0.7809', 'epoch': '0.1015'}
{'loss': '0.7115', 'grad_norm': '0.3997', 'learning_rate': '1.081e-06', 'entropy': '0.7422', 'num_tokens': '4.671e+06', 'mean_token_accuracy': '0.8155', 'epoch': '0.1015'}
{'loss': '0.8086', 'grad_norm': '0.3579', 'learning_rate': '1.077e-06', 'entropy': '0.7967', 'num_tokens': '4.678e+06', 'mean_token_accuracy': '0.7916', 'epoch': '0.1015'}
{'loss': '0.8094', 'grad_norm': '0.4221', 'learning_rate': '1.073e-06', 'entropy': '0.768', 'num_tokens': '4.683e+06', 'mean_token_accuracy': '0.798', 'epoch': '0.1015'}
{'loss': '0.76', 'grad_norm': '0.4243', 'learning_rate': '1.069e-06', 'entropy': '0.8187', 'num_tokens': '4.687e+06', 'mean_token_accuracy': '0.7901', 'epoch': '0.1015'}
{'loss': '0.6518', 'grad_norm': '0.3278', 'learning_rate': '1.065e-06', 'entropy': '0.722', 'num_tokens': '4.692e+06', 'mean_token_accuracy': '0.8173', 'epoch': '0.1015'}
{'loss': '1.382', 'grad_norm': '0.5123', 'learning_rate': '1.061e-06', 'entropy': '1.265', 'num_tokens': '4.698e+06', 'mean_token_accuracy': '0.6973', 'epoch': '0.1015'}
{'loss': '0.9593', 'grad_norm': '0.4186', 'learning_rate': '1.057e-06', 'entropy': '0.9727', 'num_tokens': '4.705e+06', 'mean_token_accuracy': '0.7627', 'epoch': '0.1015'}
{'loss': '0.7459', 'grad_norm': '0.3871', 'learning_rate': '1.053e-06', 'entropy': '0.7389', 'num_tokens': '4.709e+06', 'mean_token_accuracy': '0.8092', 'epoch': '0.1015'}
{'loss': '0.9655', 'grad_norm': '0.4114', 'learning_rate': '1.049e-06', 'entropy': '0.9177', 'num_tokens': '4.714e+06', 'mean_token_accuracy': '0.7785', 'epoch': '0.1016'}
{'loss': '0.6491', 'grad_norm': '0.3289', 'learning_rate': '1.045e-06', 'entropy': '0.7282', 'num_tokens': '4.719e+06', 'mean_token_accuracy': '0.8183', 'epoch': '0.1016'}
{'loss': '0.5436', 'grad_norm': '0.3814', 'learning_rate': '1.041e-06', 'entropy': '0.5752', 'num_tokens': '4.723e+06', 'mean_token_accuracy': '0.8579', 'epoch': '0.1016'}
{'loss': '0.8726', 'grad_norm': '0.4085', 'learning_rate': '1.037e-06', 'entropy': '0.8361', 'num_tokens': '4.728e+06', 'mean_token_accuracy': '0.7919', 'epoch': '0.1016'}
{'loss': '1.159', 'grad_norm': '0.5223', 'learning_rate': '1.033e-06', 'entropy': '1.129', 'num_tokens': '4.734e+06', 'mean_token_accuracy': '0.7231', 'epoch': '0.1016'}
{'loss': '0.72', 'grad_norm': '0.3883', 'learning_rate': '1.029e-06', 'entropy': '0.7234', 'num_tokens': '4.74e+06', 'mean_token_accuracy': '0.8117', 'epoch': '0.1016'}
{'loss': '0.844', 'grad_norm': '0.3955', 'learning_rate': '1.025e-06', 'entropy': '0.8049', 'num_tokens': '4.748e+06', 'mean_token_accuracy': '0.7959', 'epoch': '0.1016'}
{'loss': '0.8645', 'grad_norm': '0.3962', 'learning_rate': '1.021e-06', 'entropy': '0.8291', 'num_tokens': '4.754e+06', 'mean_token_accuracy': '0.7957', 'epoch': '0.1016'}
{'loss': '0.6922', 'grad_norm': '0.367', 'learning_rate': '1.017e-06', 'entropy': '0.7182', 'num_tokens': '4.758e+06', 'mean_token_accuracy': '0.8056', 'epoch': '0.1016'}
{'loss': '0.6107', 'grad_norm': '0.359', 'learning_rate': '1.013e-06', 'entropy': '0.6214', 'num_tokens': '4.764e+06', 'mean_token_accuracy': '0.8442', 'epoch': '0.1016'}
{'loss': '0.6619', 'grad_norm': '0.3806', 'learning_rate': '1.009e-06', 'entropy': '0.6885', 'num_tokens': '4.768e+06', 'mean_token_accuracy': '0.8193', 'epoch': '0.1016'}
{'loss': '0.6995', 'grad_norm': '0.3761', 'learning_rate': '1.005e-06', 'entropy': '0.6912', 'num_tokens': '4.772e+06', 'mean_token_accuracy': '0.8151', 'epoch': '0.1016'}
{'loss': '1.053', 'grad_norm': '0.3856', 'learning_rate': '1.002e-06', 'entropy': '1.249', 'num_tokens': '4.777e+06', 'mean_token_accuracy': '0.7147', 'epoch': '0.1017'}
{'loss': '0.8551', 'grad_norm': '0.341', 'learning_rate': '9.976e-07', 'entropy': '0.8634', 'num_tokens': '4.783e+06', 'mean_token_accuracy': '0.7792', 'epoch': '0.1017'}
{'loss': '0.8677', 'grad_norm': '0.4778', 'learning_rate': '9.938e-07', 'entropy': '0.8653', 'num_tokens': '4.789e+06', 'mean_token_accuracy': '0.782', 'epoch': '0.1017'}
{'loss': '0.6612', 'grad_norm': '0.4213', 'learning_rate': '9.899e-07', 'entropy': '0.7736', 'num_tokens': '4.795e+06', 'mean_token_accuracy': '0.7966', 'epoch': '0.1017'}
{'loss': '0.6757', 'grad_norm': '0.3636', 'learning_rate': '9.86e-07', 'entropy': '0.6816', 'num_tokens': '4.801e+06', 'mean_token_accuracy': '0.8282', 'epoch': '0.1017'}
{'loss': '1.316', 'grad_norm': '0.365', 'learning_rate': '9.822e-07', 'entropy': '1.065', 'num_tokens': '4.81e+06', 'mean_token_accuracy': '0.7418', 'epoch': '0.1017'}
{'loss': '1.016', 'grad_norm': '0.4002', 'learning_rate': '9.783e-07', 'entropy': '0.9728', 'num_tokens': '4.814e+06', 'mean_token_accuracy': '0.7608', 'epoch': '0.1017'}

 96%|█████████▌| 11494/12000 [31:55<20:43,  2.46s/it]
 96%|█████████▌| 11495/12000 [31:57<18:31,  2.20s/it]
                                                     

 96%|█████████▌| 11495/12000 [31:57<18:31,  2.20s/it]
 96%|█████████▌| 11496/12000 [32:00<21:04,  2.51s/it]
                                                     

 96%|█████████▌| 11496/12000 [32:00<21:04,  2.51s/it]
 96%|█████████▌| 11497/12000 [32:04<24:02,  2.87s/it]
                                                     

 96%|█████████▌| 11497/12000 [32:04<24:02,  2.87s/it]
 96%|█████████▌| 11498/12000 [32:05<20:33,  2.46s/it]
                                                     

 96%|█████████▌| 11498/12000 [32:05<20:33,  2.46s/it]
 96%|█████████▌| 11499/12000 [32:07<17:57,  2.15s/it]
                                                     

 96%|█████████▌| 11499/12000 [32:07<17:57,  2.15s/it]
 96%|█████████▌| 11500/12000 [32:10<21:20,  2.56s/it]
                                                     

 96%|█████████▌| 11500/12000 [32:10<21:20,  2.56s/it]
 96%|█████████▌| 11501/12000 [32:13<22:18,  2.68s/it]
                                                     

 96%|█████████▌| 11501/12000 [32:13<22:18,  2.68s/it]
 96%|█████████▌| 11502/12000 [32:15<20:01,  2.41s/it]
                                                     

 96%|█████████▌| 11502/12000 [32:15<20:01,  2.41s/it]
 96%|█████████▌| 11503/12000 [32:18<21:01,  2.54s/it]
                                                     

 96%|█████████▌| 11503/12000 [32:18<21:01,  2.54s/it]
 96%|█████████▌| 11504/12000 [32:20<19:35,  2.37s/it]
                                                     

 96%|█████████▌| 11504/12000 [32:20<19:35,  2.37s/it]
 96%|█████████▌| 11505/12000 [32:21<17:42,  2.15s/it]
                                                     

 96%|█████████▌| 11505/12000 [32:21<17:42,  2.15s/it]
 96%|█████████▌| 11506/12000 [32:23<15:52,  1.93s/it]
                                                     

 96%|█████████▌| 11506/12000 [32:23<15:52,  1.93s/it]
 96%|█████████▌| 11507/12000 [32:24<15:04,  1.84s/it]
                                                     

 96%|█████████▌| 11507/12000 [32:24<15:04,  1.84s/it]
 96%|█████████▌| 11508/12000 [32:26<14:52,  1.81s/it]
                                                     

 96%|█████████▌| 11508/12000 [32:26<14:52,  1.81s/it]
 96%|█████████▌| 11509/12000 [32:30<19:11,  2.34s/it]
                                                     

 96%|█████████▌| 11509/12000 [32:30<19:11,  2.34s/it]
 96%|█████████▌| 11510/12000 [32:31<17:27,  2.14s/it]
                                                     

 96%|█████████▌| 11510/12000 [32:31<17:27,  2.14s/it]
 96%|█████████▌| 11511/12000 [32:33<16:08,  1.98s/it]
                                                     

 96%|█████████▌| 11511/12000 [32:33<16:08,  1.98s/it]
 96%|█████████▌| 11512/12000 [32:35<16:20,  2.01s/it]
                                                     

 96%|█████████▌| 11512/12000 [32:35<16:20,  2.01s/it]
 96%|█████████▌| 11513/12000 [32:38<18:31,  2.28s/it]
                                                     

 96%|█████████▌| 11513/12000 [32:38<18:31,  2.28s/it]
 96%|█████████▌| 11514/12000 [32:40<17:11,  2.12s/it]
                                                     

 96%|█████████▌| 11514/12000 [32:40<17:11,  2.12s/it]
 96%|█████████▌| 11515/12000 [32:41<15:56,  1.97s/it]
                                                     

 96%|█████████▌| 11515/12000 [32:41<15:56,  1.97s/it]
 96%|█████████▌| 11516/12000 [32:43<15:34,  1.93s/it]
                                                     

 96%|█████████▌| 11516/12000 [32:43<15:34,  1.93s/it]
 96%|█████████▌| 11517/12000 [32:45<14:57,  1.86s/it]
                                                     

 96%|█████████▌| 11517/12000 [32:45<14:57,  1.86s/it]
 96%|█████████▌| 11518/12000 [32:47<14:49,  1.85s/it]
                                                     

 96%|█████████▌| 11518/12000 [32:47<14:49,  1.85s/it]
 96%|█████████▌| 11519/12000 [32:49<15:12,  1.90s/it]
                                                     

 96%|█████████▌| 11519/12000 [32:49<15:12,  1.90s/it]
 96%|█████████▌| 11520/12000 [32:51<15:09,  1.90s/it]
                                                     

 96%|█████████▌| 11520/12000 [32:51<15:09,  1.90s/it]
 96%|█████████▌| 11521/12000 [32:53<17:16,  2.16s/it]
                                                     

 96%|█████████▌| 11521/12000 [32:53<17:16,  2.16s/it]
 96%|█████████▌| 11522/12000 [32:57<21:36,  2.71s/it]
                                                     

 96%|█████████▌| 11522/12000 [32:57<21:36,  2.71s/it]
 96%|█████████▌| 11523/12000 [32:59<18:46,  2.36s/it]
                                                     

 96%|█████████▌| 11523/12000 [32:59<18:46,  2.36s/it]
 96%|█████████▌| 11524/12000 [33:01<17:41,  2.23s/it]
                                                     

 96%|█████████▌| 11524/12000 [33:01<17:41,  2.23s/it]
 96%|█████████▌| 11525/12000 [33:02<16:21,  2.07s/it]
                                                     

 96%|█████████▌| 11525/12000 [33:02<16:21,  2.07s/it]
 96%|█████████▌| 11526/12000 [33:04<15:28,  1.96s/it]
                                                     

 96%|█████████▌| 11526/12000 [33:04<15:28,  1.96s/it]
 96%|█████████▌| 11527/12000 [33:06<15:08,  1.92s/it]
                                                     

 96%|█████████▌| 11527/12000 [33:06<15:08,  1.92s/it]
 96%|█████████▌| 11528/12000 [33:08<14:30,  1.84s/it]
                                                     

 96%|█████████▌| 11528/12000 [33:08<14:30,  1.84s/it]
 96%|█████████▌| 11529/12000 [33:10<14:54,  1.90s/it]
                                                     

 96%|█████████▌| 11529/12000 [33:10<14:54,  1.90s/it]
 96%|█████████▌| 11530/12000 [33:11<14:02,  1.79s/it]
                                                     

 96%|█████████▌| 11530/12000 [33:11<14:02,  1.79s/it]
 96%|█████████▌| 11531/12000 [33:14<15:20,  1.96s/it]
                                                     

 96%|█████████▌| 11531/12000 [33:14<15:20,  1.96s/it]
 96%|█████████▌| 11532/12000 [33:15<14:27,  1.85s/it]
                                                     

 96%|█████████▌| 11532/12000 [33:15<14:27,  1.85s/it]
 96%|█████████▌| 11533/12000 [33:17<14:48,  1.90s/it]
                                                     

 96%|█████████▌| 11533/12000 [33:17<14:48,  1.90s/it]
 96%|█████████▌| 11534/12000 [33:19<14:10,  1.83s/it]
                                                     

 96%|█████████▌| 11534/12000 [33:19<14:10,  1.83s/it]
 96%|█████████▌| 11535/12000 [33:21<14:25,  1.86s/it]
                                                     

 96%|█████████▌| 11535/12000 [33:21<14:25,  1.86s/it]
 96%|█████████▌| 11536/12000 [33:22<13:36,  1.76s/it]
                                                     

 96%|█████████▌| 11536/12000 [33:22<13:36,  1.76s/it]
 96%|█████████▌| 11537/12000 [33:25<14:45,  1.91s/it]
                                                     

 96%|█████████▌| 11537/12000 [33:25<14:45,  1.91s/it]
 96%|█████████▌| 11538/12000 [33:27<15:20,  1.99s/it]
                                                     

 96%|█████████▌| 11538/12000 [33:27<15:20,  1.99s/it]
 96%|█████████▌| 11539/12000 [33:29<16:54,  2.20s/it]
                                                     

 96%|█████████▌| 11539/12000 [33:29<16:54,  2.20s/it]
 96%|█████████▌| 11540/12000 [33:31<15:58,  2.08s/it]
                                                     

 96%|█████████▌| 11540/12000 [33:31<15:58,  2.08s/it]
 96%|█████████▌| 11541/12000 [33:33<15:32,  2.03s/it]
                                                     
{'loss': '1.01', 'grad_norm': '0.3524', 'learning_rate': '9.745e-07', 'entropy': '0.9079', 'num_tokens': '4.821e+06', 'mean_token_accuracy': '0.7748', 'epoch': '0.1017'}
{'loss': '0.8652', 'grad_norm': '0.3857', 'learning_rate': '9.706e-07', 'entropy': '0.8532', 'num_tokens': '4.826e+06', 'mean_token_accuracy': '0.787', 'epoch': '0.1017'}
{'loss': '0.8234', 'grad_norm': '0.4344', 'learning_rate': '9.668e-07', 'entropy': '0.8144', 'num_tokens': '4.834e+06', 'mean_token_accuracy': '0.7973', 'epoch': '0.1017'}
{'loss': '0.8212', 'grad_norm': '0.4423', 'learning_rate': '9.63e-07', 'entropy': '0.8307', 'num_tokens': '4.842e+06', 'mean_token_accuracy': '0.7887', 'epoch': '0.1017'}
{'loss': '0.8529', 'grad_norm': '0.4745', 'learning_rate': '9.592e-07', 'entropy': '0.9524', 'num_tokens': '4.846e+06', 'mean_token_accuracy': '0.7706', 'epoch': '0.1018'}
{'loss': '0.6304', 'grad_norm': '0.3968', 'learning_rate': '9.554e-07', 'entropy': '0.6113', 'num_tokens': '4.849e+06', 'mean_token_accuracy': '0.8351', 'epoch': '0.1018'}
{'loss': '0.7806', 'grad_norm': '0.382', 'learning_rate': '9.516e-07', 'entropy': '0.8055', 'num_tokens': '4.854e+06', 'mean_token_accuracy': '0.7939', 'epoch': '0.1018'}
{'loss': '0.8024', 'grad_norm': '0.4108', 'learning_rate': '9.478e-07', 'entropy': '0.9083', 'num_tokens': '4.858e+06', 'mean_token_accuracy': '0.7802', 'epoch': '0.1018'}
{'loss': '0.7723', 'grad_norm': '0.3486', 'learning_rate': '9.44e-07', 'entropy': '0.7769', 'num_tokens': '4.864e+06', 'mean_token_accuracy': '0.8142', 'epoch': '0.1018'}
{'loss': '0.8654', 'grad_norm': '0.3464', 'learning_rate': '9.402e-07', 'entropy': '0.7917', 'num_tokens': '4.872e+06', 'mean_token_accuracy': '0.7889', 'epoch': '0.1018'}
{'loss': '0.8675', 'grad_norm': '0.3201', 'learning_rate': '9.365e-07', 'entropy': '0.9312', 'num_tokens': '4.878e+06', 'mean_token_accuracy': '0.7675', 'epoch': '0.1018'}
{'loss': '0.8524', 'grad_norm': '0.4616', 'learning_rate': '9.327e-07', 'entropy': '0.8375', 'num_tokens': '4.883e+06', 'mean_token_accuracy': '0.7884', 'epoch': '0.1018'}
{'loss': '0.6253', 'grad_norm': '0.384', 'learning_rate': '9.29e-07', 'entropy': '0.6592', 'num_tokens': '4.887e+06', 'mean_token_accuracy': '0.8343', 'epoch': '0.1018'}
{'loss': '0.6071', 'grad_norm': '0.4032', 'learning_rate': '9.252e-07', 'entropy': '0.6535', 'num_tokens': '4.891e+06', 'mean_token_accuracy': '0.8262', 'epoch': '0.1018'}
{'loss': '0.737', 'grad_norm': '0.4', 'learning_rate': '9.215e-07', 'entropy': '0.7551', 'num_tokens': '4.897e+06', 'mean_token_accuracy': '0.8014', 'epoch': '0.1018'}
{'loss': '1.092', 'grad_norm': '0.3617', 'learning_rate': '9.178e-07', 'entropy': '0.9386', 'num_tokens': '4.904e+06', 'mean_token_accuracy': '0.7578', 'epoch': '0.1019'}
{'loss': '1.064', 'grad_norm': '0.4331', 'learning_rate': '9.14e-07', 'entropy': '0.9632', 'num_tokens': '4.909e+06', 'mean_token_accuracy': '0.7708', 'epoch': '0.1019'}
{'loss': '0.7937', 'grad_norm': '0.395', 'learning_rate': '9.103e-07', 'entropy': '0.8202', 'num_tokens': '4.914e+06', 'mean_token_accuracy': '0.7945', 'epoch': '0.1019'}
{'loss': '0.7889', 'grad_norm': '0.3946', 'learning_rate': '9.066e-07', 'entropy': '0.7865', 'num_tokens': '4.921e+06', 'mean_token_accuracy': '0.8069', 'epoch': '0.1019'}
{'loss': '0.7902', 'grad_norm': '0.3576', 'learning_rate': '9.029e-07', 'entropy': '0.8352', 'num_tokens': '4.927e+06', 'mean_token_accuracy': '0.7866', 'epoch': '0.1019'}
{'loss': '0.9138', 'grad_norm': '0.4835', 'learning_rate': '8.992e-07', 'entropy': '0.8637', 'num_tokens': '4.932e+06', 'mean_token_accuracy': '0.7954', 'epoch': '0.1019'}
{'loss': '0.7554', 'grad_norm': '0.3254', 'learning_rate': '8.955e-07', 'entropy': '0.8434', 'num_tokens': '4.937e+06', 'mean_token_accuracy': '0.7925', 'epoch': '0.1019'}
{'loss': '0.9603', 'grad_norm': '0.4197', 'learning_rate': '8.919e-07', 'entropy': '1.044', 'num_tokens': '4.942e+06', 'mean_token_accuracy': '0.7499', 'epoch': '0.1019'}
{'loss': '0.7035', 'grad_norm': '0.3392', 'learning_rate': '8.882e-07', 'entropy': '0.6506', 'num_tokens': '4.947e+06', 'mean_token_accuracy': '0.8284', 'epoch': '0.1019'}
{'loss': '0.8251', 'grad_norm': '0.4254', 'learning_rate': '8.845e-07', 'entropy': '0.8458', 'num_tokens': '4.952e+06', 'mean_token_accuracy': '0.7961', 'epoch': '0.1019'}
{'loss': '0.8956', 'grad_norm': '0.4024', 'learning_rate': '8.809e-07', 'entropy': '0.832', 'num_tokens': '4.958e+06', 'mean_token_accuracy': '0.8016', 'epoch': '0.1019'}
{'loss': '1.385', 'grad_norm': '0.4773', 'learning_rate': '8.772e-07', 'entropy': '1.279', 'num_tokens': '4.964e+06', 'mean_token_accuracy': '0.6973', 'epoch': '0.102'}
{'loss': '0.7158', 'grad_norm': '0.3922', 'learning_rate': '8.736e-07', 'entropy': '0.6641', 'num_tokens': '4.971e+06', 'mean_token_accuracy': '0.8199', 'epoch': '0.102'}
{'loss': '0.7372', 'grad_norm': '0.379', 'learning_rate': '8.7e-07', 'entropy': '0.732', 'num_tokens': '4.976e+06', 'mean_token_accuracy': '0.804', 'epoch': '0.102'}
{'loss': '0.472', 'grad_norm': '0.361', 'learning_rate': '8.663e-07', 'entropy': '0.4871', 'num_tokens': '4.98e+06', 'mean_token_accuracy': '0.8767', 'epoch': '0.102'}
{'loss': '0.7745', 'grad_norm': '0.4096', 'learning_rate': '8.627e-07', 'entropy': '0.7739', 'num_tokens': '4.986e+06', 'mean_token_accuracy': '0.7942', 'epoch': '0.102'}
{'loss': '0.6992', 'grad_norm': '0.3989', 'learning_rate': '8.591e-07', 'entropy': '0.7843', 'num_tokens': '4.991e+06', 'mean_token_accuracy': '0.7886', 'epoch': '0.102'}
{'loss': '0.7109', 'grad_norm': '0.4348', 'learning_rate': '8.555e-07', 'entropy': '0.7619', 'num_tokens': '4.995e+06', 'mean_token_accuracy': '0.7987', 'epoch': '0.102'}
{'loss': '1.084', 'grad_norm': '0.455', 'learning_rate': '8.519e-07', 'entropy': '0.9641', 'num_tokens': '5e+06', 'mean_token_accuracy': '0.7737', 'epoch': '0.102'}
{'loss': '0.671', 'grad_norm': '0.4012', 'learning_rate': '8.483e-07', 'entropy': '0.6874', 'num_tokens': '5.005e+06', 'mean_token_accuracy': '0.8268', 'epoch': '0.102'}
{'loss': '0.5114', 'grad_norm': '0.3623', 'learning_rate': '8.448e-07', 'entropy': '0.5878', 'num_tokens': '5.01e+06', 'mean_token_accuracy': '0.8384', 'epoch': '0.102'}
{'loss': '0.6483', 'grad_norm': '0.3634', 'learning_rate': '8.412e-07', 'entropy': '0.6874', 'num_tokens': '5.014e+06', 'mean_token_accuracy': '0.8202', 'epoch': '0.102'}
{'loss': '0.8459', 'grad_norm': '0.4175', 'learning_rate': '8.376e-07', 'entropy': '0.865', 'num_tokens': '5.02e+06', 'mean_token_accuracy': '0.7963', 'epoch': '0.102'}
{'loss': '0.8336', 'grad_norm': '0.559', 'learning_rate': '8.341e-07', 'entropy': '0.9204', 'num_tokens': '5.024e+06', 'mean_token_accuracy': '0.7701', 'epoch': '0.1021'}
{'loss': '1.17', 'grad_norm': '0.6316', 'learning_rate': '8.305e-07', 'entropy': '1.038', 'num_tokens': '5.028e+06', 'mean_token_accuracy': '0.7476', 'epoch': '0.1021'}
{'loss': '0.8706', 'grad_norm': '0.5019', 'learning_rate': '8.27e-07', 'entropy': '0.8618', 'num_tokens': '5.032e+06', 'mean_token_accuracy': '0.7902', 'epoch': '0.1021'}
{'loss': '0.7852', 'grad_norm': '0.4758', 'learning_rate': '8.234e-07', 'entropy': '0.813', 'num_tokens': '5.037e+06', 'mean_token_accuracy': '0.799', 'epoch': '0.1021'}
{'loss': '0.7587', 'grad_norm': '0.445', 'learning_rate': '8.199e-07', 'entropy': '0.781', 'num_tokens': '5.041e+06', 'mean_token_accuracy': '0.8196', 'epoch': '0.1021'}
{'loss': '0.9348', 'grad_norm': '0.3558', 'learning_rate': '8.164e-07', 'entropy': '0.946', 'num_tokens': '5.048e+06', 'mean_token_accuracy': '0.7677', 'epoch': '0.1021'}
{'loss': '0.7472', 'grad_norm': '0.3698', 'learning_rate': '8.129e-07', 'entropy': '0.7536', 'num_tokens': '5.055e+06', 'mean_token_accuracy': '0.8152', 'epoch': '0.1021'}
{'loss': '0.7458', 'grad_norm': '0.3022', 'learning_rate': '8.094e-07', 'entropy': '0.8104', 'num_tokens': '5.062e+06', 'mean_token_accuracy': '0.7996', 'epoch': '0.1021'}
{'loss': '0.7031', 'grad_norm': '0.361', 'learning_rate': '8.059e-07', 'entropy': '0.6873', 'num_tokens': '5.067e+06', 'mean_token_accuracy': '0.8299', 'epoch': '0.1021'}

 96%|█████████▌| 11541/12000 [33:33<15:32,  2.03s/it]
 96%|█████████▌| 11542/12000 [33:36<16:39,  2.18s/it]
                                                     

 96%|█████████▌| 11542/12000 [33:36<16:39,  2.18s/it]
 96%|█████████▌| 11543/12000 [33:37<15:14,  2.00s/it]
                                                     

 96%|█████████▌| 11543/12000 [33:37<15:14,  2.00s/it]
 96%|█████████▌| 11544/12000 [33:41<18:41,  2.46s/it]
                                                     

 96%|█████████▌| 11544/12000 [33:41<18:41,  2.46s/it]
 96%|█████████▌| 11545/12000 [33:44<19:12,  2.53s/it]
                                                     

 96%|█████████▌| 11545/12000 [33:44<19:12,  2.53s/it]
 96%|█████████▌| 11546/12000 [33:45<16:41,  2.21s/it]
                                                     

 96%|█████████▌| 11546/12000 [33:45<16:41,  2.21s/it]
 96%|█████████▌| 11547/12000 [33:47<15:51,  2.10s/it]
                                                     

 96%|█████████▌| 11547/12000 [33:47<15:51,  2.10s/it]
 96%|█████████▌| 11548/12000 [33:49<15:24,  2.05s/it]
                                                     

 96%|█████████▌| 11548/12000 [33:49<15:24,  2.05s/it]
 96%|█████████▌| 11549/12000 [33:51<14:47,  1.97s/it]
                                                     

 96%|█████████▌| 11549/12000 [33:51<14:47,  1.97s/it]
 96%|█████████▋| 11550/12000 [33:52<14:16,  1.90s/it]
                                                     

 96%|█████████▋| 11550/12000 [33:52<14:16,  1.90s/it]
 96%|█████████▋| 11551/12000 [33:57<20:03,  2.68s/it]
                                                     

 96%|█████████▋| 11551/12000 [33:57<20:03,  2.68s/it]
 96%|█████████▋| 11552/12000 [33:59<18:09,  2.43s/it]
                                                     

 96%|█████████▋| 11552/12000 [33:59<18:09,  2.43s/it]
 96%|█████████▋| 11553/12000 [34:00<16:45,  2.25s/it]
                                                     

 96%|█████████▋| 11553/12000 [34:00<16:45,  2.25s/it]
 96%|█████████▋| 11554/12000 [34:04<20:09,  2.71s/it]
                                                     

 96%|█████████▋| 11554/12000 [34:04<20:09,  2.71s/it]
 96%|█████████▋| 11555/12000 [34:06<18:36,  2.51s/it]
                                                     

 96%|█████████▋| 11555/12000 [34:06<18:36,  2.51s/it]
 96%|█████████▋| 11556/12000 [34:08<16:50,  2.27s/it]
                                                     

 96%|█████████▋| 11556/12000 [34:08<16:50,  2.27s/it]
 96%|█████████▋| 11557/12000 [34:10<16:23,  2.22s/it]
                                                     

 96%|█████████▋| 11557/12000 [34:10<16:23,  2.22s/it]
 96%|█████████▋| 11558/12000 [34:12<14:55,  2.03s/it]
                                                     

 96%|█████████▋| 11558/12000 [34:12<14:55,  2.03s/it]
 96%|█████████▋| 11559/12000 [34:14<15:08,  2.06s/it]
                                                     

 96%|█████████▋| 11559/12000 [34:14<15:08,  2.06s/it]
 96%|█████████▋| 11560/12000 [34:16<15:16,  2.08s/it]
                                                     

 96%|█████████▋| 11560/12000 [34:16<15:16,  2.08s/it]
 96%|█████████▋| 11561/12000 [34:18<14:33,  1.99s/it]
                                                     

 96%|█████████▋| 11561/12000 [34:18<14:33,  1.99s/it]
 96%|█████████▋| 11562/12000 [34:19<13:56,  1.91s/it]
                                                     

 96%|█████████▋| 11562/12000 [34:19<13:56,  1.91s/it]
 96%|█████████▋| 11563/12000 [34:22<15:59,  2.20s/it]
                                                     

 96%|█████████▋| 11563/12000 [34:22<15:59,  2.20s/it]
 96%|█████████▋| 11564/12000 [34:24<14:26,  1.99s/it]
                                                     

 96%|█████████▋| 11564/12000 [34:24<14:26,  1.99s/it]
 96%|█████████▋| 11565/12000 [34:25<13:35,  1.87s/it]
                                                     

 96%|█████████▋| 11565/12000 [34:25<13:35,  1.87s/it]
 96%|█████████▋| 11566/12000 [34:30<18:53,  2.61s/it]
                                                     

 96%|█████████▋| 11566/12000 [34:30<18:53,  2.61s/it]
 96%|█████████▋| 11567/12000 [34:34<21:23,  2.96s/it]
                                                     

 96%|█████████▋| 11567/12000 [34:34<21:23,  2.96s/it]
 96%|█████████▋| 11568/12000 [34:37<22:08,  3.08s/it]
                                                     

 96%|█████████▋| 11568/12000 [34:37<22:08,  3.08s/it]
 96%|█████████▋| 11569/12000 [34:39<19:13,  2.68s/it]
                                                     

 96%|█████████▋| 11569/12000 [34:39<19:13,  2.68s/it]
 96%|█████████▋| 11570/12000 [34:41<18:12,  2.54s/it]
                                                     

 96%|█████████▋| 11570/12000 [34:41<18:12,  2.54s/it]
 96%|█████████▋| 11571/12000 [34:44<19:50,  2.77s/it]
                                                     

 96%|█████████▋| 11571/12000 [34:44<19:50,  2.77s/it]
 96%|█████████▋| 11572/12000 [34:46<17:11,  2.41s/it]
                                                     

 96%|█████████▋| 11572/12000 [34:46<17:11,  2.41s/it]
 96%|█████████▋| 11573/12000 [34:48<16:15,  2.29s/it]
                                                     

 96%|█████████▋| 11573/12000 [34:48<16:15,  2.29s/it]
 96%|█████████▋| 11574/12000 [34:50<16:53,  2.38s/it]
                                                     

 96%|█████████▋| 11574/12000 [34:50<16:53,  2.38s/it]
 96%|█████████▋| 11575/12000 [34:53<17:08,  2.42s/it]
                                                     

 96%|█████████▋| 11575/12000 [34:53<17:08,  2.42s/it]
 96%|█████████▋| 11576/12000 [34:55<16:30,  2.34s/it]
                                                     

 96%|█████████▋| 11576/12000 [34:55<16:30,  2.34s/it]
 96%|█████████▋| 11577/12000 [34:58<18:31,  2.63s/it]
                                                     

 96%|█████████▋| 11577/12000 [34:58<18:31,  2.63s/it]
 96%|█████████▋| 11578/12000 [35:00<15:55,  2.26s/it]
                                                     

 96%|█████████▋| 11578/12000 [35:00<15:55,  2.26s/it]
 96%|█████████▋| 11579/12000 [35:02<15:34,  2.22s/it]
                                                     

 96%|█████████▋| 11579/12000 [35:02<15:34,  2.22s/it]
 96%|█████████▋| 11580/12000 [35:04<15:06,  2.16s/it]
                                                     

 96%|█████████▋| 11580/12000 [35:04<15:06,  2.16s/it]
 97%|█████████▋| 11581/12000 [35:05<13:24,  1.92s/it]
                                                     

 97%|█████████▋| 11581/12000 [35:05<13:24,  1.92s/it]
 97%|█████████▋| 11582/12000 [35:08<15:01,  2.16s/it]
                                                     

 97%|█████████▋| 11582/12000 [35:08<15:01,  2.16s/it]
 97%|█████████▋| 11583/12000 [35:09<13:34,  1.95s/it]
                                                     

 97%|█████████▋| 11583/12000 [35:09<13:34,  1.95s/it]
 97%|█████████▋| 11584/12000 [35:12<14:46,  2.13s/it]
                                                     

 97%|█████████▋| 11584/12000 [35:12<14:46,  2.13s/it]
 97%|█████████▋| 11585/12000 [35:14<13:44,  1.99s/it]
                                                     

 97%|█████████▋| 11585/12000 [35:14<13:44,  1.99s/it]
 97%|█████████▋| 11586/12000 [35:15<12:46,  1.85s/it]
                                                     

 97%|█████████▋| 11586/12000 [35:15<12:46,  1.85s/it]
 97%|█████████▋| 11587/12000 [35:17<12:01,  1.75s/it]
                                                     

 97%|█████████▋| 11587/12000 [35:17<12:01,  1.75s/it]
 97%|█████████▋| 11588/12000 [35:20<15:36,  2.27s/it]
                                                     
{'loss': '0.9361', 'grad_norm': '0.4183', 'learning_rate': '8.024e-07', 'entropy': '0.8466', 'num_tokens': '5.074e+06', 'mean_token_accuracy': '0.7897', 'epoch': '0.1021'}
{'loss': '0.6427', 'grad_norm': '0.2985', 'learning_rate': '7.989e-07', 'entropy': '0.7239', 'num_tokens': '5.08e+06', 'mean_token_accuracy': '0.8148', 'epoch': '0.1021'}
{'loss': '0.5846', 'grad_norm': '0.3835', 'learning_rate': '7.955e-07', 'entropy': '0.5976', 'num_tokens': '5.084e+06', 'mean_token_accuracy': '0.8377', 'epoch': '0.1022'}
{'loss': '0.6508', 'grad_norm': '0.3104', 'learning_rate': '7.92e-07', 'entropy': '0.6429', 'num_tokens': '5.088e+06', 'mean_token_accuracy': '0.8277', 'epoch': '0.1022'}
{'loss': '0.569', 'grad_norm': '0.2697', 'learning_rate': '7.885e-07', 'entropy': '0.6946', 'num_tokens': '5.094e+06', 'mean_token_accuracy': '0.8078', 'epoch': '0.1022'}
{'loss': '1.023', 'grad_norm': '0.472', 'learning_rate': '7.851e-07', 'entropy': '0.9777', 'num_tokens': '5.098e+06', 'mean_token_accuracy': '0.767', 'epoch': '0.1022'}
{'loss': '0.6129', 'grad_norm': '0.3993', 'learning_rate': '7.816e-07', 'entropy': '0.5864', 'num_tokens': '5.103e+06', 'mean_token_accuracy': '0.8479', 'epoch': '0.1022'}
{'loss': '1.35', 'grad_norm': '0.5158', 'learning_rate': '7.782e-07', 'entropy': '1.161', 'num_tokens': '5.108e+06', 'mean_token_accuracy': '0.7246', 'epoch': '0.1022'}
{'loss': '0.7117', 'grad_norm': '0.4028', 'learning_rate': '7.748e-07', 'entropy': '0.7393', 'num_tokens': '5.113e+06', 'mean_token_accuracy': '0.8048', 'epoch': '0.1022'}
{'loss': '1.025', 'grad_norm': '0.3928', 'learning_rate': '7.714e-07', 'entropy': '0.971', 'num_tokens': '5.118e+06', 'mean_token_accuracy': '0.766', 'epoch': '0.1022'}
{'loss': '1.241', 'grad_norm': '0.4298', 'learning_rate': '7.679e-07', 'entropy': '1.102', 'num_tokens': '5.124e+06', 'mean_token_accuracy': '0.735', 'epoch': '0.1022'}
{'loss': '0.7025', 'grad_norm': '0.3258', 'learning_rate': '7.645e-07', 'entropy': '0.6739', 'num_tokens': '5.129e+06', 'mean_token_accuracy': '0.8266', 'epoch': '0.1022'}
{'loss': '1.001', 'grad_norm': '0.4526', 'learning_rate': '7.611e-07', 'entropy': '0.9658', 'num_tokens': '5.134e+06', 'mean_token_accuracy': '0.7694', 'epoch': '0.1022'}
{'loss': '1.146', 'grad_norm': '0.3525', 'learning_rate': '7.577e-07', 'entropy': '0.9402', 'num_tokens': '5.142e+06', 'mean_token_accuracy': '0.7613', 'epoch': '0.1023'}
{'loss': '0.9337', 'grad_norm': '0.392', 'learning_rate': '7.544e-07', 'entropy': '0.8143', 'num_tokens': '5.147e+06', 'mean_token_accuracy': '0.8042', 'epoch': '0.1023'}
{'loss': '1.065', 'grad_norm': '0.4126', 'learning_rate': '7.51e-07', 'entropy': '1', 'num_tokens': '5.152e+06', 'mean_token_accuracy': '0.7521', 'epoch': '0.1023'}
{'loss': '1.16', 'grad_norm': '0.4173', 'learning_rate': '7.476e-07', 'entropy': '1.022', 'num_tokens': '5.157e+06', 'mean_token_accuracy': '0.753', 'epoch': '0.1023'}
{'loss': '0.6083', 'grad_norm': '0.3268', 'learning_rate': '7.443e-07', 'entropy': '0.6017', 'num_tokens': '5.161e+06', 'mean_token_accuracy': '0.846', 'epoch': '0.1023'}
{'loss': '0.7218', 'grad_norm': '0.3765', 'learning_rate': '7.409e-07', 'entropy': '0.7582', 'num_tokens': '5.167e+06', 'mean_token_accuracy': '0.8102', 'epoch': '0.1023'}
{'loss': '1.073', 'grad_norm': '0.409', 'learning_rate': '7.376e-07', 'entropy': '1.033', 'num_tokens': '5.173e+06', 'mean_token_accuracy': '0.741', 'epoch': '0.1023'}
{'loss': '0.8301', 'grad_norm': '0.3414', 'learning_rate': '7.342e-07', 'entropy': '0.8286', 'num_tokens': '5.179e+06', 'mean_token_accuracy': '0.7975', 'epoch': '0.1023'}
{'loss': '0.6176', 'grad_norm': '0.5159', 'learning_rate': '7.309e-07', 'entropy': '0.6395', 'num_tokens': '5.183e+06', 'mean_token_accuracy': '0.824', 'epoch': '0.1023'}
{'loss': '0.783', 'grad_norm': '0.3495', 'learning_rate': '7.276e-07', 'entropy': '0.7682', 'num_tokens': '5.19e+06', 'mean_token_accuracy': '0.8037', 'epoch': '0.1023'}
{'loss': '0.9771', 'grad_norm': '0.415', 'learning_rate': '7.243e-07', 'entropy': '0.9634', 'num_tokens': '5.195e+06', 'mean_token_accuracy': '0.7719', 'epoch': '0.1023'}
{'loss': '0.7998', 'grad_norm': '0.433', 'learning_rate': '7.21e-07', 'entropy': '0.7886', 'num_tokens': '5.199e+06', 'mean_token_accuracy': '0.8069', 'epoch': '0.1023'}
{'loss': '0.6732', 'grad_norm': '0.3709', 'learning_rate': '7.177e-07', 'entropy': '0.6676', 'num_tokens': '5.205e+06', 'mean_token_accuracy': '0.814', 'epoch': '0.1024'}
{'loss': '1.266', 'grad_norm': '0.3955', 'learning_rate': '7.144e-07', 'entropy': '0.8953', 'num_tokens': '5.213e+06', 'mean_token_accuracy': '0.7847', 'epoch': '0.1024'}
{'loss': '0.6065', 'grad_norm': '0.3058', 'learning_rate': '7.111e-07', 'entropy': '0.5282', 'num_tokens': '5.22e+06', 'mean_token_accuracy': '0.8626', 'epoch': '0.1024'}
{'loss': '1.057', 'grad_norm': '0.4136', 'learning_rate': '7.078e-07', 'entropy': '1.084', 'num_tokens': '5.226e+06', 'mean_token_accuracy': '0.7387', 'epoch': '0.1024'}
{'loss': '0.5522', 'grad_norm': '0.3851', 'learning_rate': '7.045e-07', 'entropy': '0.6462', 'num_tokens': '5.231e+06', 'mean_token_accuracy': '0.835', 'epoch': '0.1024'}
{'loss': '0.9753', 'grad_norm': '0.44', 'learning_rate': '7.013e-07', 'entropy': '0.9368', 'num_tokens': '5.239e+06', 'mean_token_accuracy': '0.7593', 'epoch': '0.1024'}
{'loss': '0.956', 'grad_norm': '0.457', 'learning_rate': '6.98e-07', 'entropy': '0.884', 'num_tokens': '5.243e+06', 'mean_token_accuracy': '0.7812', 'epoch': '0.1024'}
{'loss': '0.7751', 'grad_norm': '0.3525', 'learning_rate': '6.948e-07', 'entropy': '0.8042', 'num_tokens': '5.249e+06', 'mean_token_accuracy': '0.7959', 'epoch': '0.1024'}
{'loss': '0.8284', 'grad_norm': '0.3605', 'learning_rate': '6.915e-07', 'entropy': '0.7909', 'num_tokens': '5.255e+06', 'mean_token_accuracy': '0.7965', 'epoch': '0.1024'}
{'loss': '0.7859', 'grad_norm': '0.4158', 'learning_rate': '6.883e-07', 'entropy': '0.6954', 'num_tokens': '5.261e+06', 'mean_token_accuracy': '0.8125', 'epoch': '0.1024'}
{'loss': '1.031', 'grad_norm': '0.3716', 'learning_rate': '6.851e-07', 'entropy': '0.9653', 'num_tokens': '5.268e+06', 'mean_token_accuracy': '0.767', 'epoch': '0.1024'}
{'loss': '0.8576', 'grad_norm': '0.4877', 'learning_rate': '6.819e-07', 'entropy': '0.9087', 'num_tokens': '5.275e+06', 'mean_token_accuracy': '0.786', 'epoch': '0.1025'}
{'loss': '0.8277', 'grad_norm': '0.4096', 'learning_rate': '6.787e-07', 'entropy': '0.8344', 'num_tokens': '5.279e+06', 'mean_token_accuracy': '0.7848', 'epoch': '0.1025'}
{'loss': '0.8967', 'grad_norm': '0.3509', 'learning_rate': '6.755e-07', 'entropy': '0.882', 'num_tokens': '5.286e+06', 'mean_token_accuracy': '0.7775', 'epoch': '0.1025'}
{'loss': '0.6087', 'grad_norm': '0.3704', 'learning_rate': '6.723e-07', 'entropy': '0.6417', 'num_tokens': '5.293e+06', 'mean_token_accuracy': '0.8315', 'epoch': '0.1025'}
{'loss': '0.5799', 'grad_norm': '0.3757', 'learning_rate': '6.691e-07', 'entropy': '0.642', 'num_tokens': '5.298e+06', 'mean_token_accuracy': '0.832', 'epoch': '0.1025'}
{'loss': '0.8873', 'grad_norm': '0.4142', 'learning_rate': '6.659e-07', 'entropy': '0.8401', 'num_tokens': '5.304e+06', 'mean_token_accuracy': '0.7956', 'epoch': '0.1025'}
{'loss': '0.5002', 'grad_norm': '0.3409', 'learning_rate': '6.627e-07', 'entropy': '0.5413', 'num_tokens': '5.308e+06', 'mean_token_accuracy': '0.8616', 'epoch': '0.1025'}
{'loss': '0.7543', 'grad_norm': '0.3915', 'learning_rate': '6.596e-07', 'entropy': '0.8105', 'num_tokens': '5.314e+06', 'mean_token_accuracy': '0.8021', 'epoch': '0.1025'}
{'loss': '0.7251', 'grad_norm': '0.3701', 'learning_rate': '6.564e-07', 'entropy': '0.7542', 'num_tokens': '5.319e+06', 'mean_token_accuracy': '0.8064', 'epoch': '0.1025'}
{'loss': '0.7003', 'grad_norm': '0.3899', 'learning_rate': '6.533e-07', 'entropy': '0.7672', 'num_tokens': '5.324e+06', 'mean_token_accuracy': '0.7937', 'epoch': '0.1025'}
{'loss': '0.9672', 'grad_norm': '0.4743', 'learning_rate': '6.501e-07', 'entropy': '1.005', 'num_tokens': '5.328e+06', 'mean_token_accuracy': '0.7504', 'epoch': '0.1025'}

 97%|█████████▋| 11588/12000 [35:20<15:36,  2.27s/it]
 97%|█████████▋| 11589/12000 [35:24<18:00,  2.63s/it]
                                                     

 97%|█████████▋| 11589/12000 [35:24<18:00,  2.63s/it]
 97%|█████████▋| 11590/12000 [35:26<17:21,  2.54s/it]
                                                     

 97%|█████████▋| 11590/12000 [35:26<17:21,  2.54s/it]
 97%|█████████▋| 11591/12000 [35:28<16:28,  2.42s/it]
                                                     

 97%|█████████▋| 11591/12000 [35:28<16:28,  2.42s/it]
 97%|█████████▋| 11592/12000 [35:29<14:20,  2.11s/it]
                                                     

 97%|█████████▋| 11592/12000 [35:29<14:20,  2.11s/it]
 97%|█████████▋| 11593/12000 [35:33<16:42,  2.46s/it]
                                                     

 97%|█████████▋| 11593/12000 [35:33<16:42,  2.46s/it]
 97%|█████████▋| 11594/12000 [35:34<14:54,  2.20s/it]
                                                     

 97%|█████████▋| 11594/12000 [35:34<14:54,  2.20s/it]
 97%|█████████▋| 11595/12000 [35:37<15:46,  2.34s/it]
                                                     

 97%|█████████▋| 11595/12000 [35:37<15:46,  2.34s/it]
 97%|█████████▋| 11596/12000 [35:38<13:55,  2.07s/it]
                                                     

 97%|█████████▋| 11596/12000 [35:38<13:55,  2.07s/it]
 97%|█████████▋| 11597/12000 [35:40<12:22,  1.84s/it]
                                                     

 97%|█████████▋| 11597/12000 [35:40<12:22,  1.84s/it]
 97%|█████████▋| 11598/12000 [35:41<11:16,  1.68s/it]
                                                     

 97%|█████████▋| 11598/12000 [35:41<11:16,  1.68s/it]
 97%|█████████▋| 11599/12000 [35:42<10:43,  1.60s/it]
                                                     

 97%|█████████▋| 11599/12000 [35:42<10:43,  1.60s/it]
 97%|█████████▋| 11600/12000 [35:44<10:20,  1.55s/it]
                                                     

 97%|█████████▋| 11600/12000 [35:44<10:20,  1.55s/it]
 97%|█████████▋| 11601/12000 [35:47<13:35,  2.04s/it]
                                                     

 97%|█████████▋| 11601/12000 [35:47<13:35,  2.04s/it]
 97%|█████████▋| 11602/12000 [35:49<12:47,  1.93s/it]
                                                     

 97%|█████████▋| 11602/12000 [35:49<12:47,  1.93s/it]
 97%|█████████▋| 11603/12000 [35:51<13:58,  2.11s/it]
                                                     

 97%|█████████▋| 11603/12000 [35:51<13:58,  2.11s/it]
 97%|█████████▋| 11604/12000 [35:53<12:52,  1.95s/it]
                                                     

 97%|█████████▋| 11604/12000 [35:53<12:52,  1.95s/it]
 97%|█████████▋| 11605/12000 [35:58<18:14,  2.77s/it]
                                                     

 97%|█████████▋| 11605/12000 [35:58<18:14,  2.77s/it]
 97%|█████████▋| 11606/12000 [36:01<19:07,  2.91s/it]
                                                     

 97%|█████████▋| 11606/12000 [36:01<19:07,  2.91s/it]
 97%|█████████▋| 11607/12000 [36:03<17:01,  2.60s/it]
                                                     

 97%|█████████▋| 11607/12000 [36:03<17:01,  2.60s/it]
 97%|█████████▋| 11608/12000 [36:04<14:47,  2.26s/it]
                                                     

 97%|█████████▋| 11608/12000 [36:04<14:47,  2.26s/it]
 97%|█████████▋| 11609/12000 [36:07<15:04,  2.31s/it]
                                                     

 97%|█████████▋| 11609/12000 [36:07<15:04,  2.31s/it]
 97%|█████████▋| 11610/12000 [36:11<19:18,  2.97s/it]
                                                     

 97%|█████████▋| 11610/12000 [36:11<19:18,  2.97s/it]
 97%|█████████▋| 11611/12000 [36:13<16:39,  2.57s/it]
                                                     

 97%|█████████▋| 11611/12000 [36:13<16:39,  2.57s/it]
 97%|█████████▋| 11612/12000 [36:14<14:42,  2.28s/it]
                                                     

 97%|█████████▋| 11612/12000 [36:14<14:42,  2.28s/it]
 97%|█████████▋| 11613/12000 [36:17<14:51,  2.30s/it]
                                                     

 97%|█████████▋| 11613/12000 [36:17<14:51,  2.30s/it]
 97%|█████████▋| 11614/12000 [36:19<14:52,  2.31s/it]
                                                     

 97%|█████████▋| 11614/12000 [36:19<14:52,  2.31s/it]
 97%|█████████▋| 11615/12000 [36:21<13:26,  2.09s/it]
                                                     

 97%|█████████▋| 11615/12000 [36:21<13:26,  2.09s/it]
 97%|█████████▋| 11616/12000 [36:22<12:03,  1.88s/it]
                                                     

 97%|█████████▋| 11616/12000 [36:22<12:03,  1.88s/it]
 97%|█████████▋| 11617/12000 [36:23<11:09,  1.75s/it]
                                                     

 97%|█████████▋| 11617/12000 [36:23<11:09,  1.75s/it]
 97%|█████████▋| 11618/12000 [36:25<11:17,  1.77s/it]
                                                     

 97%|█████████▋| 11618/12000 [36:25<11:17,  1.77s/it]
 97%|█████████▋| 11619/12000 [36:28<13:01,  2.05s/it]
                                                     

 97%|█████████▋| 11619/12000 [36:28<13:01,  2.05s/it]
 97%|█████████▋| 11620/12000 [36:30<12:18,  1.94s/it]
                                                     

 97%|█████████▋| 11620/12000 [36:30<12:18,  1.94s/it]
 97%|█████████▋| 11621/12000 [36:31<11:43,  1.86s/it]
                                                     

 97%|█████████▋| 11621/12000 [36:31<11:43,  1.86s/it]
 97%|█████████▋| 11622/12000 [36:37<19:27,  3.09s/it]
                                                     

 97%|█████████▋| 11622/12000 [36:37<19:27,  3.09s/it]
 97%|█████████▋| 11623/12000 [36:41<20:25,  3.25s/it]
                                                     

 97%|█████████▋| 11623/12000 [36:41<20:25,  3.25s/it]
 97%|█████████▋| 11624/12000 [36:43<17:42,  2.83s/it]
                                                     

 97%|█████████▋| 11624/12000 [36:43<17:42,  2.83s/it]
 97%|█████████▋| 11625/12000 [36:45<16:43,  2.68s/it]
                                                     

 97%|█████████▋| 11625/12000 [36:45<16:43,  2.68s/it]
 97%|█████████▋| 11626/12000 [36:47<14:30,  2.33s/it]
                                                     

 97%|█████████▋| 11626/12000 [36:47<14:30,  2.33s/it]
 97%|█████████▋| 11627/12000 [36:48<13:04,  2.10s/it]
                                                     

 97%|█████████▋| 11627/12000 [36:48<13:04,  2.10s/it]
 97%|█████████▋| 11628/12000 [36:51<14:06,  2.27s/it]
                                                     

 97%|█████████▋| 11628/12000 [36:51<14:06,  2.27s/it]
 97%|█████████▋| 11629/12000 [36:52<12:40,  2.05s/it]
                                                     

 97%|█████████▋| 11629/12000 [36:52<12:40,  2.05s/it]
 97%|█████████▋| 11630/12000 [36:54<11:28,  1.86s/it]
                                                     

 97%|█████████▋| 11630/12000 [36:54<11:28,  1.86s/it]
 97%|█████████▋| 11631/12000 [36:56<11:29,  1.87s/it]
                                                     

 97%|█████████▋| 11631/12000 [36:56<11:29,  1.87s/it]
 97%|█████████▋| 11632/12000 [36:59<14:23,  2.35s/it]
                                                     

 97%|█████████▋| 11632/12000 [36:59<14:23,  2.35s/it]
 97%|█████████▋| 11633/12000 [37:01<13:14,  2.17s/it]
                                                     

 97%|█████████▋| 11633/12000 [37:01<13:14,  2.17s/it]
 97%|█████████▋| 11634/12000 [37:04<14:11,  2.33s/it]
                                                     

 97%|█████████▋| 11634/12000 [37:04<14:11,  2.33s/it]
 97%|█████████▋| 11635/12000 [37:05<12:41,  2.09s/it]
                                                     
{'loss': '0.9144', 'grad_norm': '0.4692', 'learning_rate': '6.47e-07', 'entropy': '1.015', 'num_tokens': '5.332e+06', 'mean_token_accuracy': '0.7515', 'epoch': '0.1026'}
{'loss': '0.6877', 'grad_norm': '0.3561', 'learning_rate': '6.439e-07', 'entropy': '0.6987', 'num_tokens': '5.34e+06', 'mean_token_accuracy': '0.8156', 'epoch': '0.1026'}
{'loss': '1.149', 'grad_norm': '0.5788', 'learning_rate': '6.407e-07', 'entropy': '1.121', 'num_tokens': '5.346e+06', 'mean_token_accuracy': '0.7267', 'epoch': '0.1026'}
{'loss': '0.8169', 'grad_norm': '0.4587', 'learning_rate': '6.376e-07', 'entropy': '0.7728', 'num_tokens': '5.352e+06', 'mean_token_accuracy': '0.7954', 'epoch': '0.1026'}
{'loss': '0.9229', 'grad_norm': '0.5397', 'learning_rate': '6.345e-07', 'entropy': '0.9404', 'num_tokens': '5.355e+06', 'mean_token_accuracy': '0.7716', 'epoch': '0.1026'}
{'loss': '0.6601', 'grad_norm': '0.3229', 'learning_rate': '6.314e-07', 'entropy': '0.7382', 'num_tokens': '5.363e+06', 'mean_token_accuracy': '0.8043', 'epoch': '0.1026'}
{'loss': '0.8091', 'grad_norm': '0.3634', 'learning_rate': '6.283e-07', 'entropy': '0.8281', 'num_tokens': '5.368e+06', 'mean_token_accuracy': '0.7976', 'epoch': '0.1026'}
{'loss': '0.841', 'grad_norm': '0.3666', 'learning_rate': '6.253e-07', 'entropy': '0.8844', 'num_tokens': '5.375e+06', 'mean_token_accuracy': '0.7804', 'epoch': '0.1026'}
{'loss': '0.8879', 'grad_norm': '0.4955', 'learning_rate': '6.222e-07', 'entropy': '0.8748', 'num_tokens': '5.38e+06', 'mean_token_accuracy': '0.7674', 'epoch': '0.1026'}
{'loss': '0.7854', 'grad_norm': '0.3478', 'learning_rate': '6.191e-07', 'entropy': '0.799', 'num_tokens': '5.383e+06', 'mean_token_accuracy': '0.7868', 'epoch': '0.1026'}
{'loss': '0.8915', 'grad_norm': '0.4785', 'learning_rate': '6.161e-07', 'entropy': '0.9402', 'num_tokens': '5.387e+06', 'mean_token_accuracy': '0.7601', 'epoch': '0.1026'}
{'loss': '1.058', 'grad_norm': '0.4812', 'learning_rate': '6.13e-07', 'entropy': '1.033', 'num_tokens': '5.39e+06', 'mean_token_accuracy': '0.7523', 'epoch': '0.1026'}
{'loss': '0.8597', 'grad_norm': '0.44', 'learning_rate': '6.1e-07', 'entropy': '0.855', 'num_tokens': '5.394e+06', 'mean_token_accuracy': '0.7801', 'epoch': '0.1027'}
{'loss': '0.8244', 'grad_norm': '0.4119', 'learning_rate': '6.069e-07', 'entropy': '0.7956', 'num_tokens': '5.399e+06', 'mean_token_accuracy': '0.7964', 'epoch': '0.1027'}
{'loss': '0.8356', 'grad_norm': '0.5166', 'learning_rate': '6.039e-07', 'entropy': '0.912', 'num_tokens': '5.404e+06', 'mean_token_accuracy': '0.7837', 'epoch': '0.1027'}
{'loss': '0.8538', 'grad_norm': '0.5664', 'learning_rate': '6.009e-07', 'entropy': '0.8031', 'num_tokens': '5.409e+06', 'mean_token_accuracy': '0.7947', 'epoch': '0.1027'}
{'loss': '0.6793', 'grad_norm': '0.3544', 'learning_rate': '5.979e-07', 'entropy': '0.665', 'num_tokens': '5.414e+06', 'mean_token_accuracy': '0.8342', 'epoch': '0.1027'}
{'loss': '1.678', 'grad_norm': '0.4047', 'learning_rate': '5.949e-07', 'entropy': '1.234', 'num_tokens': '5.423e+06', 'mean_token_accuracy': '0.7377', 'epoch': '0.1027'}
{'loss': '0.928', 'grad_norm': '0.4518', 'learning_rate': '5.919e-07', 'entropy': '0.8646', 'num_tokens': '5.43e+06', 'mean_token_accuracy': '0.7857', 'epoch': '0.1027'}
{'loss': '0.9253', 'grad_norm': '0.4591', 'learning_rate': '5.889e-07', 'entropy': '0.8483', 'num_tokens': '5.436e+06', 'mean_token_accuracy': '0.7916', 'epoch': '0.1027'}
{'loss': '0.6291', 'grad_norm': '0.3211', 'learning_rate': '5.859e-07', 'entropy': '0.6177', 'num_tokens': '5.44e+06', 'mean_token_accuracy': '0.8294', 'epoch': '0.1027'}
{'loss': '0.9644', 'grad_norm': '0.4054', 'learning_rate': '5.829e-07', 'entropy': '0.8516', 'num_tokens': '5.447e+06', 'mean_token_accuracy': '0.7891', 'epoch': '0.1027'}
{'loss': '0.6594', 'grad_norm': '0.3298', 'learning_rate': '5.8e-07', 'entropy': '0.6734', 'num_tokens': '5.455e+06', 'mean_token_accuracy': '0.823', 'epoch': '0.1027'}
{'loss': '0.5799', 'grad_norm': '0.5527', 'learning_rate': '5.77e-07', 'entropy': '0.5958', 'num_tokens': '5.459e+06', 'mean_token_accuracy': '0.8406', 'epoch': '0.1028'}
{'loss': '0.6402', 'grad_norm': '0.3791', 'learning_rate': '5.74e-07', 'entropy': '0.6818', 'num_tokens': '5.464e+06', 'mean_token_accuracy': '0.8199', 'epoch': '0.1028'}
{'loss': '0.9257', 'grad_norm': '0.3895', 'learning_rate': '5.711e-07', 'entropy': '0.9571', 'num_tokens': '5.471e+06', 'mean_token_accuracy': '0.7619', 'epoch': '0.1028'}
{'loss': '0.9648', 'grad_norm': '0.4452', 'learning_rate': '5.682e-07', 'entropy': '0.9871', 'num_tokens': '5.477e+06', 'mean_token_accuracy': '0.7519', 'epoch': '0.1028'}
{'loss': '1.054', 'grad_norm': '0.4513', 'learning_rate': '5.652e-07', 'entropy': '1.015', 'num_tokens': '5.482e+06', 'mean_token_accuracy': '0.7501', 'epoch': '0.1028'}
{'loss': '0.8071', 'grad_norm': '0.4447', 'learning_rate': '5.623e-07', 'entropy': '0.7774', 'num_tokens': '5.485e+06', 'mean_token_accuracy': '0.8009', 'epoch': '0.1028'}
{'loss': '0.5499', 'grad_norm': '0.2668', 'learning_rate': '5.594e-07', 'entropy': '0.5534', 'num_tokens': '5.489e+06', 'mean_token_accuracy': '0.8522', 'epoch': '0.1028'}
{'loss': '0.8585', 'grad_norm': '0.3905', 'learning_rate': '5.565e-07', 'entropy': '0.8987', 'num_tokens': '5.495e+06', 'mean_token_accuracy': '0.7691', 'epoch': '0.1028'}
{'loss': '0.6598', 'grad_norm': '0.3685', 'learning_rate': '5.536e-07', 'entropy': '0.6913', 'num_tokens': '5.502e+06', 'mean_token_accuracy': '0.8269', 'epoch': '0.1028'}
{'loss': '0.7032', 'grad_norm': '0.3874', 'learning_rate': '5.507e-07', 'entropy': '0.7551', 'num_tokens': '5.506e+06', 'mean_token_accuracy': '0.8122', 'epoch': '0.1028'}
{'loss': '0.9286', 'grad_norm': '0.4561', 'learning_rate': '5.478e-07', 'entropy': '0.9008', 'num_tokens': '5.511e+06', 'mean_token_accuracy': '0.7705', 'epoch': '0.1028'}
{'loss': '0.406', 'grad_norm': '0.2224', 'learning_rate': '5.449e-07', 'entropy': '0.4798', 'num_tokens': '5.522e+06', 'mean_token_accuracy': '0.8758', 'epoch': '0.1029'}
{'loss': '0.9024', 'grad_norm': '0.359', 'learning_rate': '5.421e-07', 'entropy': '0.9381', 'num_tokens': '5.529e+06', 'mean_token_accuracy': '0.7756', 'epoch': '0.1029'}
{'loss': '0.7749', 'grad_norm': '0.4479', 'learning_rate': '5.392e-07', 'entropy': '0.7466', 'num_tokens': '5.535e+06', 'mean_token_accuracy': '0.7999', 'epoch': '0.1029'}
{'loss': '1.199', 'grad_norm': '0.5321', 'learning_rate': '5.363e-07', 'entropy': '1.232', 'num_tokens': '5.541e+06', 'mean_token_accuracy': '0.693', 'epoch': '0.1029'}
{'loss': '0.6508', 'grad_norm': '0.3787', 'learning_rate': '5.335e-07', 'entropy': '0.7225', 'num_tokens': '5.546e+06', 'mean_token_accuracy': '0.8081', 'epoch': '0.1029'}
{'loss': '0.8852', 'grad_norm': '0.4819', 'learning_rate': '5.307e-07', 'entropy': '0.8361', 'num_tokens': '5.55e+06', 'mean_token_accuracy': '0.7775', 'epoch': '0.1029'}
{'loss': '0.6377', 'grad_norm': '0.3797', 'learning_rate': '5.278e-07', 'entropy': '0.7249', 'num_tokens': '5.556e+06', 'mean_token_accuracy': '0.8075', 'epoch': '0.1029'}
{'loss': '0.7184', 'grad_norm': '0.4249', 'learning_rate': '5.25e-07', 'entropy': '0.7261', 'num_tokens': '5.56e+06', 'mean_token_accuracy': '0.8185', 'epoch': '0.1029'}
{'loss': '0.6144', 'grad_norm': '0.2854', 'learning_rate': '5.222e-07', 'entropy': '0.6263', 'num_tokens': '5.565e+06', 'mean_token_accuracy': '0.8392', 'epoch': '0.1029'}
{'loss': '0.9075', 'grad_norm': '0.4024', 'learning_rate': '5.194e-07', 'entropy': '0.9265', 'num_tokens': '5.571e+06', 'mean_token_accuracy': '0.7616', 'epoch': '0.1029'}
{'loss': '0.5338', 'grad_norm': '0.2862', 'learning_rate': '5.166e-07', 'entropy': '0.5245', 'num_tokens': '5.576e+06', 'mean_token_accuracy': '0.8631', 'epoch': '0.1029'}
{'loss': '0.6281', 'grad_norm': '0.4093', 'learning_rate': '5.138e-07', 'entropy': '0.633', 'num_tokens': '5.581e+06', 'mean_token_accuracy': '0.8329', 'epoch': '0.103'}
{'loss': '0.596', 'grad_norm': '0.415', 'learning_rate': '5.11e-07', 'entropy': '0.6893', 'num_tokens': '5.586e+06', 'mean_token_accuracy': '0.8195', 'epoch': '0.103'}

 97%|█████████▋| 11635/12000 [37:05<12:41,  2.09s/it]
 97%|█████████▋| 11636/12000 [37:08<13:58,  2.30s/it]
                                                     

 97%|█████████▋| 11636/12000 [37:08<13:58,  2.30s/it]
 97%|█████████▋| 11637/12000 [37:10<12:45,  2.11s/it]
                                                     

 97%|█████████▋| 11637/12000 [37:10<12:45,  2.11s/it]
 97%|█████████▋| 11638/12000 [37:11<11:29,  1.91s/it]
                                                     

 97%|█████████▋| 11638/12000 [37:11<11:29,  1.91s/it]
 97%|█████████▋| 11639/12000 [37:13<11:48,  1.96s/it]
                                                     

 97%|█████████▋| 11639/12000 [37:13<11:48,  1.96s/it]
 97%|█████████▋| 11640/12000 [37:15<11:21,  1.89s/it]
                                                     

 97%|█████████▋| 11640/12000 [37:15<11:21,  1.89s/it]
 97%|█████████▋| 11641/12000 [37:17<11:23,  1.90s/it]
                                                     

 97%|█████████▋| 11641/12000 [37:17<11:23,  1.90s/it]
 97%|█████████▋| 11642/12000 [37:19<12:17,  2.06s/it]
                                                     

 97%|█████████▋| 11642/12000 [37:19<12:17,  2.06s/it]
 97%|█████████▋| 11643/12000 [37:21<12:45,  2.15s/it]
                                                     

 97%|█████████▋| 11643/12000 [37:21<12:45,  2.15s/it]
 97%|█████████▋| 11644/12000 [37:24<13:26,  2.26s/it]
                                                     

 97%|█████████▋| 11644/12000 [37:24<13:26,  2.26s/it]
 97%|█████████▋| 11645/12000 [37:26<12:34,  2.13s/it]
                                                     

 97%|█████████▋| 11645/12000 [37:26<12:34,  2.13s/it]
 97%|█████████▋| 11646/12000 [37:27<11:27,  1.94s/it]
                                                     

 97%|█████████▋| 11646/12000 [37:27<11:27,  1.94s/it]
 97%|█████████▋| 11647/12000 [37:29<11:14,  1.91s/it]
                                                     

 97%|█████████▋| 11647/12000 [37:29<11:14,  1.91s/it]
 97%|█████████▋| 11648/12000 [37:31<11:43,  2.00s/it]
                                                     

 97%|█████████▋| 11648/12000 [37:31<11:43,  2.00s/it]
 97%|█████████▋| 11649/12000 [37:34<11:58,  2.05s/it]
                                                     

 97%|█████████▋| 11649/12000 [37:34<11:58,  2.05s/it]
 97%|█████████▋| 11650/12000 [37:36<12:44,  2.18s/it]
                                                     

 97%|█████████▋| 11650/12000 [37:36<12:44,  2.18s/it]
 97%|█████████▋| 11651/12000 [37:39<14:45,  2.54s/it]
                                                     

 97%|█████████▋| 11651/12000 [37:39<14:45,  2.54s/it]
 97%|█████████▋| 11652/12000 [37:42<14:59,  2.59s/it]
                                                     

 97%|█████████▋| 11652/12000 [37:42<14:59,  2.59s/it]
 97%|█████████▋| 11653/12000 [37:44<12:59,  2.25s/it]
                                                     

 97%|█████████▋| 11653/12000 [37:44<12:59,  2.25s/it]
 97%|█████████▋| 11654/12000 [37:47<15:26,  2.68s/it]
                                                     

 97%|█████████▋| 11654/12000 [37:47<15:26,  2.68s/it]
 97%|█████████▋| 11655/12000 [37:50<15:19,  2.67s/it]
                                                     

 97%|█████████▋| 11655/12000 [37:50<15:19,  2.67s/it]
 97%|█████████▋| 11656/12000 [37:51<13:14,  2.31s/it]
                                                     

 97%|█████████▋| 11656/12000 [37:51<13:14,  2.31s/it]
 97%|█████████▋| 11657/12000 [37:54<13:43,  2.40s/it]
                                                     

 97%|█████████▋| 11657/12000 [37:54<13:43,  2.40s/it]
 97%|█████████▋| 11658/12000 [37:56<12:37,  2.22s/it]
                                                     

 97%|█████████▋| 11658/12000 [37:56<12:37,  2.22s/it]
 97%|█████████▋| 11659/12000 [37:57<11:41,  2.06s/it]
                                                     

 97%|█████████▋| 11659/12000 [37:57<11:41,  2.06s/it]
 97%|█████████▋| 11660/12000 [37:59<10:36,  1.87s/it]
                                                     

 97%|█████████▋| 11660/12000 [37:59<10:36,  1.87s/it]
 97%|█████████▋| 11661/12000 [38:02<12:54,  2.29s/it]
                                                     

 97%|█████████▋| 11661/12000 [38:02<12:54,  2.29s/it]
 97%|█████████▋| 11662/12000 [38:07<17:22,  3.09s/it]
                                                     

 97%|█████████▋| 11662/12000 [38:07<17:22,  3.09s/it]
 97%|█████████▋| 11663/12000 [38:09<14:35,  2.60s/it]
                                                     

 97%|█████████▋| 11663/12000 [38:09<14:35,  2.60s/it]
 97%|█████████▋| 11664/12000 [38:10<12:44,  2.28s/it]
                                                     

 97%|█████████▋| 11664/12000 [38:10<12:44,  2.28s/it]
 97%|█████████▋| 11665/12000 [38:13<13:01,  2.33s/it]
                                                     

 97%|█████████▋| 11665/12000 [38:13<13:01,  2.33s/it]
 97%|█████████▋| 11666/12000 [38:14<11:25,  2.05s/it]
                                                     

 97%|█████████▋| 11666/12000 [38:14<11:25,  2.05s/it]
 97%|█████████▋| 11667/12000 [38:15<10:34,  1.90s/it]
                                                     

 97%|█████████▋| 11667/12000 [38:15<10:34,  1.90s/it]
 97%|█████████▋| 11668/12000 [38:17<10:17,  1.86s/it]
                                                     

 97%|█████████▋| 11668/12000 [38:17<10:17,  1.86s/it]
 97%|█████████▋| 11669/12000 [38:20<10:55,  1.98s/it]
                                                     

 97%|█████████▋| 11669/12000 [38:20<10:55,  1.98s/it]
 97%|█████████▋| 11670/12000 [38:22<11:58,  2.18s/it]
                                                     

 97%|█████████▋| 11670/12000 [38:22<11:58,  2.18s/it]
 97%|█████████▋| 11671/12000 [38:24<11:24,  2.08s/it]
                                                     

 97%|█████████▋| 11671/12000 [38:24<11:24,  2.08s/it]
 97%|█████████▋| 11672/12000 [38:26<11:28,  2.10s/it]
                                                     

 97%|█████████▋| 11672/12000 [38:26<11:28,  2.10s/it]
 97%|█████████▋| 11673/12000 [38:28<10:42,  1.96s/it]
                                                     

 97%|█████████▋| 11673/12000 [38:28<10:42,  1.96s/it]
 97%|█████████▋| 11674/12000 [38:29<10:09,  1.87s/it]
                                                     

 97%|█████████▋| 11674/12000 [38:29<10:09,  1.87s/it]
 97%|█████████▋| 11675/12000 [38:31<09:35,  1.77s/it]
                                                     

 97%|█████████▋| 11675/12000 [38:31<09:35,  1.77s/it]
 97%|█████████▋| 11676/12000 [38:34<12:22,  2.29s/it]
                                                     

 97%|█████████▋| 11676/12000 [38:34<12:22,  2.29s/it]
 97%|█████████▋| 11677/12000 [38:37<13:17,  2.47s/it]
                                                     

 97%|█████████▋| 11677/12000 [38:37<13:17,  2.47s/it]
 97%|█████████▋| 11678/12000 [38:40<13:34,  2.53s/it]
                                                     

 97%|█████████▋| 11678/12000 [38:40<13:34,  2.53s/it]
 97%|█████████▋| 11679/12000 [38:42<12:05,  2.26s/it]
                                                     

 97%|█████████▋| 11679/12000 [38:42<12:05,  2.26s/it]
 97%|█████████▋| 11680/12000 [38:43<10:49,  2.03s/it]
                                                     

 97%|█████████▋| 11680/12000 [38:43<10:49,  2.03s/it]
 97%|█████████▋| 11681/12000 [38:45<10:51,  2.04s/it]
                                                     

 97%|█████████▋| 11681/12000 [38:45<10:51,  2.04s/it]
 97%|█████████▋| 11682/12000 [38:47<10:02,  1.90s/it]
                                                     
{'loss': '1.252', 'grad_norm': '0.5475', 'learning_rate': '5.082e-07', 'entropy': '1.178', 'num_tokens': '5.59e+06', 'mean_token_accuracy': '0.7298', 'epoch': '0.103'}
{'loss': '0.7476', 'grad_norm': '0.4074', 'learning_rate': '5.055e-07', 'entropy': '0.7573', 'num_tokens': '5.596e+06', 'mean_token_accuracy': '0.8014', 'epoch': '0.103'}
{'loss': '0.6816', 'grad_norm': '0.3632', 'learning_rate': '5.027e-07', 'entropy': '0.6979', 'num_tokens': '5.601e+06', 'mean_token_accuracy': '0.8234', 'epoch': '0.103'}
{'loss': '0.767', 'grad_norm': '0.4721', 'learning_rate': '4.999e-07', 'entropy': '0.8217', 'num_tokens': '5.605e+06', 'mean_token_accuracy': '0.7958', 'epoch': '0.103'}
{'loss': '1.177', 'grad_norm': '0.4183', 'learning_rate': '4.972e-07', 'entropy': '1.039', 'num_tokens': '5.611e+06', 'mean_token_accuracy': '0.7383', 'epoch': '0.103'}
{'loss': '0.7733', 'grad_norm': '0.4969', 'learning_rate': '4.944e-07', 'entropy': '0.8086', 'num_tokens': '5.616e+06', 'mean_token_accuracy': '0.7765', 'epoch': '0.103'}
{'loss': '1.076', 'grad_norm': '0.3969', 'learning_rate': '4.917e-07', 'entropy': '1.035', 'num_tokens': '5.622e+06', 'mean_token_accuracy': '0.7475', 'epoch': '0.103'}
{'loss': '0.9432', 'grad_norm': '0.4029', 'learning_rate': '4.89e-07', 'entropy': '0.8675', 'num_tokens': '5.628e+06', 'mean_token_accuracy': '0.7803', 'epoch': '0.103'}
{'loss': '0.5786', 'grad_norm': '0.2548', 'learning_rate': '4.863e-07', 'entropy': '0.6108', 'num_tokens': '5.636e+06', 'mean_token_accuracy': '0.837', 'epoch': '0.103'}
{'loss': '1.016', 'grad_norm': '0.3376', 'learning_rate': '4.836e-07', 'entropy': '0.8412', 'num_tokens': '5.644e+06', 'mean_token_accuracy': '0.7883', 'epoch': '0.103'}
{'loss': '1.117', 'grad_norm': '0.4195', 'learning_rate': '4.809e-07', 'entropy': '0.9875', 'num_tokens': '5.648e+06', 'mean_token_accuracy': '0.7557', 'epoch': '0.1031'}
{'loss': '0.7169', 'grad_norm': '0.4408', 'learning_rate': '4.782e-07', 'entropy': '0.7417', 'num_tokens': '5.652e+06', 'mean_token_accuracy': '0.816', 'epoch': '0.1031'}
{'loss': '0.7387', 'grad_norm': '0.3747', 'learning_rate': '4.755e-07', 'entropy': '0.8643', 'num_tokens': '5.658e+06', 'mean_token_accuracy': '0.7888', 'epoch': '0.1031'}
{'loss': '0.8123', 'grad_norm': '0.4931', 'learning_rate': '4.728e-07', 'entropy': '0.7945', 'num_tokens': '5.663e+06', 'mean_token_accuracy': '0.8032', 'epoch': '0.1031'}
{'loss': '0.9263', 'grad_norm': '0.3822', 'learning_rate': '4.701e-07', 'entropy': '0.9207', 'num_tokens': '5.67e+06', 'mean_token_accuracy': '0.7835', 'epoch': '0.1031'}
{'loss': '1.053', 'grad_norm': '0.3979', 'learning_rate': '4.675e-07', 'entropy': '1.011', 'num_tokens': '5.675e+06', 'mean_token_accuracy': '0.753', 'epoch': '0.1031'}
{'loss': '0.7919', 'grad_norm': '0.4859', 'learning_rate': '4.648e-07', 'entropy': '0.7675', 'num_tokens': '5.68e+06', 'mean_token_accuracy': '0.7973', 'epoch': '0.1031'}
{'loss': '0.7647', 'grad_norm': '0.3179', 'learning_rate': '4.621e-07', 'entropy': '0.6953', 'num_tokens': '5.687e+06', 'mean_token_accuracy': '0.8227', 'epoch': '0.1031'}
{'loss': '0.5898', 'grad_norm': '0.4135', 'learning_rate': '4.595e-07', 'entropy': '0.6122', 'num_tokens': '5.691e+06', 'mean_token_accuracy': '0.8347', 'epoch': '0.1031'}
{'loss': '0.5298', 'grad_norm': '0.3172', 'learning_rate': '4.569e-07', 'entropy': '0.5428', 'num_tokens': '5.695e+06', 'mean_token_accuracy': '0.8666', 'epoch': '0.1031'}
{'loss': '0.5939', 'grad_norm': '0.3483', 'learning_rate': '4.542e-07', 'entropy': '0.657', 'num_tokens': '5.702e+06', 'mean_token_accuracy': '0.8342', 'epoch': '0.1031'}
{'loss': '0.9778', 'grad_norm': '0.426', 'learning_rate': '4.516e-07', 'entropy': '0.9361', 'num_tokens': '5.707e+06', 'mean_token_accuracy': '0.7688', 'epoch': '0.1032'}
{'loss': '0.833', 'grad_norm': '0.3947', 'learning_rate': '4.49e-07', 'entropy': '0.8748', 'num_tokens': '5.713e+06', 'mean_token_accuracy': '0.7903', 'epoch': '0.1032'}
{'loss': '0.9225', 'grad_norm': '0.5059', 'learning_rate': '4.464e-07', 'entropy': '0.9177', 'num_tokens': '5.718e+06', 'mean_token_accuracy': '0.7723', 'epoch': '0.1032'}
{'loss': '0.6762', 'grad_norm': '0.4432', 'learning_rate': '4.438e-07', 'entropy': '0.7328', 'num_tokens': '5.723e+06', 'mean_token_accuracy': '0.8155', 'epoch': '0.1032'}
{'loss': '0.714', 'grad_norm': '0.3878', 'learning_rate': '4.412e-07', 'entropy': '0.6725', 'num_tokens': '5.727e+06', 'mean_token_accuracy': '0.8226', 'epoch': '0.1032'}
{'loss': '0.8502', 'grad_norm': '0.4326', 'learning_rate': '4.386e-07', 'entropy': '0.8049', 'num_tokens': '5.734e+06', 'mean_token_accuracy': '0.797', 'epoch': '0.1032'}
{'loss': '0.4753', 'grad_norm': '0.2611', 'learning_rate': '4.361e-07', 'entropy': '0.4881', 'num_tokens': '5.743e+06', 'mean_token_accuracy': '0.8725', 'epoch': '0.1032'}
{'loss': '0.7427', 'grad_norm': '0.4657', 'learning_rate': '4.335e-07', 'entropy': '0.8707', 'num_tokens': '5.747e+06', 'mean_token_accuracy': '0.7807', 'epoch': '0.1032'}
{'loss': '0.7694', 'grad_norm': '0.5242', 'learning_rate': '4.309e-07', 'entropy': '0.8222', 'num_tokens': '5.751e+06', 'mean_token_accuracy': '0.7739', 'epoch': '0.1032'}
{'loss': '0.9784', 'grad_norm': '0.3657', 'learning_rate': '4.284e-07', 'entropy': '0.9512', 'num_tokens': '5.757e+06', 'mean_token_accuracy': '0.7582', 'epoch': '0.1032'}
{'loss': '0.8669', 'grad_norm': '0.4276', 'learning_rate': '4.258e-07', 'entropy': '0.8622', 'num_tokens': '5.761e+06', 'mean_token_accuracy': '0.7891', 'epoch': '0.1032'}
{'loss': '0.765', 'grad_norm': '0.4282', 'learning_rate': '4.233e-07', 'entropy': '0.7753', 'num_tokens': '5.765e+06', 'mean_token_accuracy': '0.8016', 'epoch': '0.1033'}
{'loss': '0.65', 'grad_norm': '0.3981', 'learning_rate': '4.208e-07', 'entropy': '0.6615', 'num_tokens': '5.769e+06', 'mean_token_accuracy': '0.834', 'epoch': '0.1033'}
{'loss': '0.4843', 'grad_norm': '0.3942', 'learning_rate': '4.182e-07', 'entropy': '0.514', 'num_tokens': '5.775e+06', 'mean_token_accuracy': '0.8618', 'epoch': '0.1033'}
{'loss': '0.7239', 'grad_norm': '0.3433', 'learning_rate': '4.157e-07', 'entropy': '0.7112', 'num_tokens': '5.782e+06', 'mean_token_accuracy': '0.8057', 'epoch': '0.1033'}
{'loss': '0.5532', 'grad_norm': '0.3945', 'learning_rate': '4.132e-07', 'entropy': '0.5828', 'num_tokens': '5.788e+06', 'mean_token_accuracy': '0.8392', 'epoch': '0.1033'}
{'loss': '0.9885', 'grad_norm': '0.4551', 'learning_rate': '4.107e-07', 'entropy': '1.017', 'num_tokens': '5.794e+06', 'mean_token_accuracy': '0.755', 'epoch': '0.1033'}
{'loss': '0.7926', 'grad_norm': '0.4434', 'learning_rate': '4.082e-07', 'entropy': '0.7961', 'num_tokens': '5.799e+06', 'mean_token_accuracy': '0.8052', 'epoch': '0.1033'}
{'loss': '0.5728', 'grad_norm': '0.4103', 'learning_rate': '4.058e-07', 'entropy': '0.5725', 'num_tokens': '5.804e+06', 'mean_token_accuracy': '0.8503', 'epoch': '0.1033'}
{'loss': '0.6168', 'grad_norm': '0.3567', 'learning_rate': '4.033e-07', 'entropy': '0.7257', 'num_tokens': '5.808e+06', 'mean_token_accuracy': '0.8142', 'epoch': '0.1033'}
{'loss': '0.4737', 'grad_norm': '0.3322', 'learning_rate': '4.008e-07', 'entropy': '0.5503', 'num_tokens': '5.812e+06', 'mean_token_accuracy': '0.8587', 'epoch': '0.1033'}
{'loss': '1.009', 'grad_norm': '0.3753', 'learning_rate': '3.983e-07', 'entropy': '1.022', 'num_tokens': '5.819e+06', 'mean_token_accuracy': '0.757', 'epoch': '0.1033'}
{'loss': '0.6287', 'grad_norm': '0.3609', 'learning_rate': '3.959e-07', 'entropy': '0.627', 'num_tokens': '5.826e+06', 'mean_token_accuracy': '0.8362', 'epoch': '0.1033'}
{'loss': '0.7752', 'grad_norm': '0.4559', 'learning_rate': '3.934e-07', 'entropy': '0.7503', 'num_tokens': '5.831e+06', 'mean_token_accuracy': '0.8031', 'epoch': '0.1034'}
{'loss': '0.897', 'grad_norm': '0.508', 'learning_rate': '3.91e-07', 'entropy': '0.9245', 'num_tokens': '5.835e+06', 'mean_token_accuracy': '0.7691', 'epoch': '0.1034'}
{'loss': '0.8169', 'grad_norm': '0.424', 'learning_rate': '3.886e-07', 'entropy': '0.8388', 'num_tokens': '5.841e+06', 'mean_token_accuracy': '0.7874', 'epoch': '0.1034'}

 97%|█████████▋| 11682/12000 [38:47<10:02,  1.90s/it]
 97%|█████████▋| 11683/12000 [38:49<10:58,  2.08s/it]
                                                     

 97%|█████████▋| 11683/12000 [38:49<10:58,  2.08s/it]
 97%|█████████▋| 11684/12000 [38:51<10:05,  1.92s/it]
                                                     

 97%|█████████▋| 11684/12000 [38:51<10:05,  1.92s/it]
 97%|█████████▋| 11685/12000 [38:55<13:16,  2.53s/it]
                                                     

 97%|█████████▋| 11685/12000 [38:55<13:16,  2.53s/it]
 97%|█████████▋| 11686/12000 [38:57<12:02,  2.30s/it]
                                                     

 97%|█████████▋| 11686/12000 [38:57<12:02,  2.30s/it]
 97%|█████████▋| 11687/12000 [38:59<11:46,  2.26s/it]
                                                     

 97%|█████████▋| 11687/12000 [38:59<11:46,  2.26s/it]
 97%|█████████▋| 11688/12000 [39:00<10:54,  2.10s/it]
                                                     

 97%|█████████▋| 11688/12000 [39:00<10:54,  2.10s/it]
 97%|█████████▋| 11689/12000 [39:05<15:12,  2.94s/it]
                                                     

 97%|█████████▋| 11689/12000 [39:05<15:12,  2.94s/it]
 97%|█████████▋| 11690/12000 [39:08<14:21,  2.78s/it]
                                                     

 97%|█████████▋| 11690/12000 [39:08<14:21,  2.78s/it]
 97%|█████████▋| 11691/12000 [39:09<12:12,  2.37s/it]
                                                     

 97%|█████████▋| 11691/12000 [39:09<12:12,  2.37s/it]
 97%|█████████▋| 11692/12000 [39:11<10:52,  2.12s/it]
                                                     

 97%|█████████▋| 11692/12000 [39:11<10:52,  2.12s/it]
 97%|█████████▋| 11693/12000 [39:12<10:02,  1.96s/it]
                                                     

 97%|█████████▋| 11693/12000 [39:12<10:02,  1.96s/it]
 97%|█████████▋| 11694/12000 [39:14<09:20,  1.83s/it]
                                                     

 97%|█████████▋| 11694/12000 [39:14<09:20,  1.83s/it]
 97%|█████████▋| 11695/12000 [39:15<08:41,  1.71s/it]
                                                     

 97%|█████████▋| 11695/12000 [39:15<08:41,  1.71s/it]
 97%|█████████▋| 11696/12000 [39:17<08:24,  1.66s/it]
                                                     

 97%|█████████▋| 11696/12000 [39:17<08:24,  1.66s/it]
 97%|█████████▋| 11697/12000 [39:21<11:39,  2.31s/it]
                                                     

 97%|█████████▋| 11697/12000 [39:21<11:39,  2.31s/it]
 97%|█████████▋| 11698/12000 [39:23<12:24,  2.47s/it]
                                                     

 97%|█████████▋| 11698/12000 [39:23<12:24,  2.47s/it]
 97%|█████████▋| 11699/12000 [39:26<11:45,  2.34s/it]
                                                     

 97%|█████████▋| 11699/12000 [39:26<11:45,  2.34s/it]
 98%|█████████▊| 11700/12000 [39:27<10:27,  2.09s/it]
                                                     

 98%|█████████▊| 11700/12000 [39:27<10:27,  2.09s/it]
 98%|█████████▊| 11701/12000 [39:30<11:55,  2.39s/it]
                                                     

 98%|█████████▊| 11701/12000 [39:30<11:55,  2.39s/it]
 98%|█████████▊| 11702/12000 [39:32<10:54,  2.20s/it]
                                                     

 98%|█████████▊| 11702/12000 [39:32<10:54,  2.20s/it]
 98%|█████████▊| 11703/12000 [39:34<10:32,  2.13s/it]
                                                     

 98%|█████████▊| 11703/12000 [39:34<10:32,  2.13s/it]
 98%|█████████▊| 11704/12000 [39:36<09:54,  2.01s/it]
                                                     

 98%|█████████▊| 11704/12000 [39:36<09:54,  2.01s/it]
 98%|█████████▊| 11705/12000 [39:37<09:08,  1.86s/it]
                                                     

 98%|█████████▊| 11705/12000 [39:37<09:08,  1.86s/it]
 98%|█████████▊| 11706/12000 [39:40<10:29,  2.14s/it]
                                                     

 98%|█████████▊| 11706/12000 [39:40<10:29,  2.14s/it]
 98%|█████████▊| 11707/12000 [39:41<09:22,  1.92s/it]
                                                     

 98%|█████████▊| 11707/12000 [39:41<09:22,  1.92s/it]
 98%|█████████▊| 11708/12000 [39:44<10:28,  2.15s/it]
                                                     

 98%|█████████▊| 11708/12000 [39:44<10:28,  2.15s/it]
 98%|█████████▊| 11709/12000 [39:46<09:57,  2.05s/it]
                                                     

 98%|█████████▊| 11709/12000 [39:46<09:57,  2.05s/it]
 98%|█████████▊| 11710/12000 [39:48<09:55,  2.05s/it]
                                                     

 98%|█████████▊| 11710/12000 [39:48<09:55,  2.05s/it]
 98%|█████████▊| 11711/12000 [39:50<09:24,  1.95s/it]
                                                     

 98%|█████████▊| 11711/12000 [39:50<09:24,  1.95s/it]
 98%|█████████▊| 11712/12000 [39:51<09:04,  1.89s/it]
                                                     

 98%|█████████▊| 11712/12000 [39:51<09:04,  1.89s/it]
 98%|█████████▊| 11713/12000 [39:53<08:39,  1.81s/it]
                                                     

 98%|█████████▊| 11713/12000 [39:53<08:39,  1.81s/it]
 98%|█████████▊| 11714/12000 [39:55<08:37,  1.81s/it]
                                                     

 98%|█████████▊| 11714/12000 [39:55<08:37,  1.81s/it]
 98%|█████████▊| 11715/12000 [39:56<08:18,  1.75s/it]
                                                     

 98%|█████████▊| 11715/12000 [39:56<08:18,  1.75s/it]
 98%|█████████▊| 11716/12000 [39:58<08:08,  1.72s/it]
                                                     

 98%|█████████▊| 11716/12000 [39:58<08:08,  1.72s/it]
 98%|█████████▊| 11717/12000 [40:00<07:49,  1.66s/it]
                                                     

 98%|█████████▊| 11717/12000 [40:00<07:49,  1.66s/it]
 98%|█████████▊| 11718/12000 [40:01<07:35,  1.61s/it]
                                                     

 98%|█████████▊| 11718/12000 [40:01<07:35,  1.61s/it]
 98%|█████████▊| 11719/12000 [40:05<10:39,  2.28s/it]
                                                     

 98%|█████████▊| 11719/12000 [40:05<10:39,  2.28s/it]
 98%|█████████▊| 11720/12000 [40:07<10:16,  2.20s/it]
                                                     

 98%|█████████▊| 11720/12000 [40:07<10:16,  2.20s/it]
 98%|█████████▊| 11721/12000 [40:08<09:24,  2.02s/it]
                                                     

 98%|█████████▊| 11721/12000 [40:08<09:24,  2.02s/it]
 98%|█████████▊| 11722/12000 [40:10<08:48,  1.90s/it]
                                                     

 98%|█████████▊| 11722/12000 [40:10<08:48,  1.90s/it]
 98%|█████████▊| 11723/12000 [40:12<08:18,  1.80s/it]
                                                     

 98%|█████████▊| 11723/12000 [40:12<08:18,  1.80s/it]
 98%|█████████▊| 11724/12000 [40:13<07:53,  1.72s/it]
                                                     

 98%|█████████▊| 11724/12000 [40:13<07:53,  1.72s/it]
 98%|█████████▊| 11725/12000 [40:15<07:36,  1.66s/it]
                                                     

 98%|█████████▊| 11725/12000 [40:15<07:36,  1.66s/it]
 98%|█████████▊| 11726/12000 [40:16<07:27,  1.63s/it]
                                                     

 98%|█████████▊| 11726/12000 [40:16<07:27,  1.63s/it]
 98%|█████████▊| 11727/12000 [40:20<09:56,  2.18s/it]
                                                     

 98%|█████████▊| 11727/12000 [40:20<09:56,  2.18s/it]
 98%|█████████▊| 11728/12000 [40:22<10:27,  2.31s/it]
                                                     

 98%|█████████▊| 11728/12000 [40:22<10:27,  2.31s/it]
 98%|█████████▊| 11729/12000 [40:26<11:36,  2.57s/it]
                                                     
{'loss': '0.933', 'grad_norm': '0.5206', 'learning_rate': '3.862e-07', 'entropy': '0.9718', 'num_tokens': '5.845e+06', 'mean_token_accuracy': '0.7645', 'epoch': '0.1034'}
{'loss': '0.8539', 'grad_norm': '0.3806', 'learning_rate': '3.837e-07', 'entropy': '0.8736', 'num_tokens': '5.852e+06', 'mean_token_accuracy': '0.7814', 'epoch': '0.1034'}
{'loss': '0.5932', 'grad_norm': '0.2577', 'learning_rate': '3.813e-07', 'entropy': '0.6048', 'num_tokens': '5.856e+06', 'mean_token_accuracy': '0.8347', 'epoch': '0.1034'}
{'loss': '0.6949', 'grad_norm': '0.3168', 'learning_rate': '3.789e-07', 'entropy': '0.7199', 'num_tokens': '5.867e+06', 'mean_token_accuracy': '0.8033', 'epoch': '0.1034'}
{'loss': '1.169', 'grad_norm': '0.4246', 'learning_rate': '3.765e-07', 'entropy': '1.219', 'num_tokens': '5.872e+06', 'mean_token_accuracy': '0.7009', 'epoch': '0.1034'}
{'loss': '0.9976', 'grad_norm': '0.5594', 'learning_rate': '3.742e-07', 'entropy': '0.9318', 'num_tokens': '5.877e+06', 'mean_token_accuracy': '0.7668', 'epoch': '0.1034'}
{'loss': '0.7161', 'grad_norm': '0.522', 'learning_rate': '3.718e-07', 'entropy': '0.8422', 'num_tokens': '5.882e+06', 'mean_token_accuracy': '0.7886', 'epoch': '0.1034'}
{'loss': '1.126', 'grad_norm': '0.2883', 'learning_rate': '3.694e-07', 'entropy': '0.9752', 'num_tokens': '5.891e+06', 'mean_token_accuracy': '0.7379', 'epoch': '0.1034'}
{'loss': '0.5901', 'grad_norm': '0.3772', 'learning_rate': '3.67e-07', 'entropy': '0.6133', 'num_tokens': '5.897e+06', 'mean_token_accuracy': '0.8366', 'epoch': '0.1035'}
{'loss': '0.8209', 'grad_norm': '0.3908', 'learning_rate': '3.647e-07', 'entropy': '0.847', 'num_tokens': '5.901e+06', 'mean_token_accuracy': '0.7821', 'epoch': '0.1035'}
{'loss': '1.267', 'grad_norm': '0.443', 'learning_rate': '3.623e-07', 'entropy': '1.289', 'num_tokens': '5.906e+06', 'mean_token_accuracy': '0.6999', 'epoch': '0.1035'}
{'loss': '0.7823', 'grad_norm': '0.3992', 'learning_rate': '3.6e-07', 'entropy': '0.9215', 'num_tokens': '5.91e+06', 'mean_token_accuracy': '0.7755', 'epoch': '0.1035'}
{'loss': '0.7565', 'grad_norm': '0.3545', 'learning_rate': '3.577e-07', 'entropy': '0.7973', 'num_tokens': '5.915e+06', 'mean_token_accuracy': '0.8022', 'epoch': '0.1035'}
{'loss': '0.848', 'grad_norm': '0.4577', 'learning_rate': '3.553e-07', 'entropy': '0.8714', 'num_tokens': '5.919e+06', 'mean_token_accuracy': '0.7875', 'epoch': '0.1035'}
{'loss': '0.8694', 'grad_norm': '0.4645', 'learning_rate': '3.53e-07', 'entropy': '0.9021', 'num_tokens': '5.923e+06', 'mean_token_accuracy': '0.7782', 'epoch': '0.1035'}
{'loss': '0.4946', 'grad_norm': '0.3378', 'learning_rate': '3.507e-07', 'entropy': '0.5096', 'num_tokens': '5.929e+06', 'mean_token_accuracy': '0.8718', 'epoch': '0.1035'}
{'loss': '0.9017', 'grad_norm': '0.3505', 'learning_rate': '3.484e-07', 'entropy': '0.9352', 'num_tokens': '5.938e+06', 'mean_token_accuracy': '0.7639', 'epoch': '0.1035'}
{'loss': '0.987', 'grad_norm': '0.3889', 'learning_rate': '3.461e-07', 'entropy': '1.09', 'num_tokens': '5.943e+06', 'mean_token_accuracy': '0.7457', 'epoch': '0.1035'}
{'loss': '0.723', 'grad_norm': '0.4478', 'learning_rate': '3.438e-07', 'entropy': '0.794', 'num_tokens': '5.947e+06', 'mean_token_accuracy': '0.8005', 'epoch': '0.1035'}
{'loss': '0.9068', 'grad_norm': '0.456', 'learning_rate': '3.416e-07', 'entropy': '0.969', 'num_tokens': '5.952e+06', 'mean_token_accuracy': '0.7582', 'epoch': '0.1036'}
{'loss': '0.8875', 'grad_norm': '0.356', 'learning_rate': '3.393e-07', 'entropy': '0.9041', 'num_tokens': '5.958e+06', 'mean_token_accuracy': '0.7755', 'epoch': '0.1036'}
{'loss': '0.6373', 'grad_norm': '0.3753', 'learning_rate': '3.37e-07', 'entropy': '0.6279', 'num_tokens': '5.963e+06', 'mean_token_accuracy': '0.8336', 'epoch': '0.1036'}
{'loss': '0.911', 'grad_norm': '0.4316', 'learning_rate': '3.348e-07', 'entropy': '0.8353', 'num_tokens': '5.968e+06', 'mean_token_accuracy': '0.7997', 'epoch': '0.1036'}
{'loss': '0.9498', 'grad_norm': '0.4803', 'learning_rate': '3.325e-07', 'entropy': '1.098', 'num_tokens': '5.972e+06', 'mean_token_accuracy': '0.7396', 'epoch': '0.1036'}
{'loss': '0.7036', 'grad_norm': '0.3297', 'learning_rate': '3.303e-07', 'entropy': '0.7596', 'num_tokens': '5.98e+06', 'mean_token_accuracy': '0.8106', 'epoch': '0.1036'}
{'loss': '0.8901', 'grad_norm': '0.5031', 'learning_rate': '3.28e-07', 'entropy': '0.8376', 'num_tokens': '5.983e+06', 'mean_token_accuracy': '0.7915', 'epoch': '0.1036'}
{'loss': '0.7937', 'grad_norm': '0.3995', 'learning_rate': '3.258e-07', 'entropy': '0.7775', 'num_tokens': '5.989e+06', 'mean_token_accuracy': '0.803', 'epoch': '0.1036'}
{'loss': '0.7632', 'grad_norm': '0.2644', 'learning_rate': '3.236e-07', 'entropy': '0.7392', 'num_tokens': '5.995e+06', 'mean_token_accuracy': '0.8026', 'epoch': '0.1036'}
{'loss': '1.043', 'grad_norm': '0.4933', 'learning_rate': '3.214e-07', 'entropy': '1.076', 'num_tokens': '6.001e+06', 'mean_token_accuracy': '0.7353', 'epoch': '0.1036'}
{'loss': '0.7364', 'grad_norm': '0.4272', 'learning_rate': '3.192e-07', 'entropy': '0.7622', 'num_tokens': '6.005e+06', 'mean_token_accuracy': '0.7967', 'epoch': '0.1036'}
{'loss': '0.8912', 'grad_norm': '0.3967', 'learning_rate': '3.17e-07', 'entropy': '0.8424', 'num_tokens': '6.01e+06', 'mean_token_accuracy': '0.7779', 'epoch': '0.1036'}
{'loss': '0.8583', 'grad_norm': '0.4147', 'learning_rate': '3.148e-07', 'entropy': '0.832', 'num_tokens': '6.014e+06', 'mean_token_accuracy': '0.7806', 'epoch': '0.1037'}
{'loss': '0.7189', 'grad_norm': '0.3319', 'learning_rate': '3.126e-07', 'entropy': '0.7054', 'num_tokens': '6.02e+06', 'mean_token_accuracy': '0.817', 'epoch': '0.1037'}
{'loss': '0.7188', 'grad_norm': '0.3672', 'learning_rate': '3.104e-07', 'entropy': '0.7155', 'num_tokens': '6.026e+06', 'mean_token_accuracy': '0.8077', 'epoch': '0.1037'}
{'loss': '0.9514', 'grad_norm': '0.4438', 'learning_rate': '3.083e-07', 'entropy': '0.959', 'num_tokens': '6.031e+06', 'mean_token_accuracy': '0.7641', 'epoch': '0.1037'}
{'loss': '0.7886', 'grad_norm': '0.3053', 'learning_rate': '3.061e-07', 'entropy': '0.7815', 'num_tokens': '6.035e+06', 'mean_token_accuracy': '0.7895', 'epoch': '0.1037'}
{'loss': '0.8714', 'grad_norm': '0.4341', 'learning_rate': '3.04e-07', 'entropy': '0.8819', 'num_tokens': '6.039e+06', 'mean_token_accuracy': '0.7805', 'epoch': '0.1037'}
{'loss': '0.9936', 'grad_norm': '0.4123', 'learning_rate': '3.018e-07', 'entropy': '0.9644', 'num_tokens': '6.044e+06', 'mean_token_accuracy': '0.767', 'epoch': '0.1037'}
{'loss': '1.014', 'grad_norm': '0.7399', 'learning_rate': '2.997e-07', 'entropy': '0.9035', 'num_tokens': '6.05e+06', 'mean_token_accuracy': '0.7811', 'epoch': '0.1037'}
{'loss': '0.7115', 'grad_norm': '0.3668', 'learning_rate': '2.976e-07', 'entropy': '0.726', 'num_tokens': '6.055e+06', 'mean_token_accuracy': '0.8076', 'epoch': '0.1037'}
{'loss': '0.601', 'grad_norm': '0.4326', 'learning_rate': '2.954e-07', 'entropy': '0.7519', 'num_tokens': '6.059e+06', 'mean_token_accuracy': '0.8045', 'epoch': '0.1037'}
{'loss': '0.955', 'grad_norm': '0.4627', 'learning_rate': '2.933e-07', 'entropy': '0.8648', 'num_tokens': '6.064e+06', 'mean_token_accuracy': '0.7751', 'epoch': '0.1037'}
{'loss': '0.7006', 'grad_norm': '0.3462', 'learning_rate': '2.912e-07', 'entropy': '0.771', 'num_tokens': '6.069e+06', 'mean_token_accuracy': '0.8033', 'epoch': '0.1038'}
{'loss': '0.7365', 'grad_norm': '0.3519', 'learning_rate': '2.891e-07', 'entropy': '0.7256', 'num_tokens': '6.073e+06', 'mean_token_accuracy': '0.8119', 'epoch': '0.1038'}
{'loss': '0.7956', 'grad_norm': '0.3901', 'learning_rate': '2.87e-07', 'entropy': '0.8401', 'num_tokens': '6.079e+06', 'mean_token_accuracy': '0.7831', 'epoch': '0.1038'}
{'loss': '1.018', 'grad_norm': '0.3462', 'learning_rate': '2.849e-07', 'entropy': '0.912', 'num_tokens': '6.088e+06', 'mean_token_accuracy': '0.7772', 'epoch': '0.1038'}
{'loss': '0.5658', 'grad_norm': '0.286', 'learning_rate': '2.829e-07', 'entropy': '0.663', 'num_tokens': '6.093e+06', 'mean_token_accuracy': '0.8226', 'epoch': '0.1038'}

 98%|█████████▊| 11729/12000 [40:26<11:36,  2.57s/it]
 98%|█████████▊| 11730/12000 [40:27<10:25,  2.32s/it]
                                                     

 98%|█████████▊| 11730/12000 [40:27<10:25,  2.32s/it]
 98%|█████████▊| 11731/12000 [40:30<10:35,  2.36s/it]
                                                     

 98%|█████████▊| 11731/12000 [40:30<10:35,  2.36s/it]
 98%|█████████▊| 11732/12000 [40:32<09:58,  2.23s/it]
                                                     

 98%|█████████▊| 11732/12000 [40:32<09:58,  2.23s/it]
 98%|█████████▊| 11733/12000 [40:33<09:08,  2.05s/it]
                                                     

 98%|█████████▊| 11733/12000 [40:33<09:08,  2.05s/it]
 98%|█████████▊| 11734/12000 [40:35<08:19,  1.88s/it]
                                                     

 98%|█████████▊| 11734/12000 [40:35<08:19,  1.88s/it]
 98%|█████████▊| 11735/12000 [40:37<08:54,  2.02s/it]
                                                     

 98%|█████████▊| 11735/12000 [40:37<08:54,  2.02s/it]
 98%|█████████▊| 11736/12000 [40:39<08:10,  1.86s/it]
                                                     

 98%|█████████▊| 11736/12000 [40:39<08:10,  1.86s/it]
 98%|█████████▊| 11737/12000 [40:40<08:01,  1.83s/it]
                                                     

 98%|█████████▊| 11737/12000 [40:40<08:01,  1.83s/it]
 98%|█████████▊| 11738/12000 [40:44<10:38,  2.44s/it]
                                                     

 98%|█████████▊| 11738/12000 [40:44<10:38,  2.44s/it]
 98%|█████████▊| 11739/12000 [40:46<09:39,  2.22s/it]
                                                     

 98%|█████████▊| 11739/12000 [40:46<09:39,  2.22s/it]
 98%|█████████▊| 11740/12000 [40:48<08:55,  2.06s/it]
                                                     

 98%|█████████▊| 11740/12000 [40:48<08:55,  2.06s/it]
 98%|█████████▊| 11741/12000 [40:51<10:36,  2.46s/it]
                                                     

 98%|█████████▊| 11741/12000 [40:51<10:36,  2.46s/it]
 98%|█████████▊| 11742/12000 [40:53<09:27,  2.20s/it]
                                                     

 98%|█████████▊| 11742/12000 [40:53<09:27,  2.20s/it]
 98%|█████████▊| 11743/12000 [40:54<08:55,  2.08s/it]
                                                     

 98%|█████████▊| 11743/12000 [40:54<08:55,  2.08s/it]
 98%|█████████▊| 11744/12000 [40:56<07:59,  1.87s/it]
                                                     

 98%|█████████▊| 11744/12000 [40:56<07:59,  1.87s/it]
 98%|█████████▊| 11745/12000 [40:58<08:08,  1.91s/it]
                                                     

 98%|█████████▊| 11745/12000 [40:58<08:08,  1.91s/it]
 98%|█████████▊| 11746/12000 [41:03<12:29,  2.95s/it]
                                                     

 98%|█████████▊| 11746/12000 [41:03<12:29,  2.95s/it]
 98%|█████████▊| 11747/12000 [41:05<10:54,  2.59s/it]
                                                     

 98%|█████████▊| 11747/12000 [41:05<10:54,  2.59s/it]
 98%|█████████▊| 11748/12000 [41:07<09:41,  2.31s/it]
                                                     

 98%|█████████▊| 11748/12000 [41:07<09:41,  2.31s/it]
 98%|█████████▊| 11749/12000 [41:08<08:59,  2.15s/it]
                                                     

 98%|█████████▊| 11749/12000 [41:08<08:59,  2.15s/it]
 98%|█████████▊| 11750/12000 [41:13<12:12,  2.93s/it]
                                                     

 98%|█████████▊| 11750/12000 [41:13<12:12,  2.93s/it]
 98%|█████████▊| 11751/12000 [41:17<12:53,  3.11s/it]
                                                     

 98%|█████████▊| 11751/12000 [41:17<12:53,  3.11s/it]
 98%|█████████▊| 11752/12000 [41:19<11:39,  2.82s/it]
                                                     

 98%|█████████▊| 11752/12000 [41:19<11:39,  2.82s/it]
 98%|█████████▊| 11753/12000 [41:22<11:52,  2.89s/it]
                                                     

 98%|█████████▊| 11753/12000 [41:22<11:52,  2.89s/it]
 98%|█████████▊| 11754/12000 [41:24<11:10,  2.73s/it]
                                                     

 98%|█████████▊| 11754/12000 [41:24<11:10,  2.73s/it]
 98%|█████████▊| 11755/12000 [41:26<09:58,  2.44s/it]
                                                     

 98%|█████████▊| 11755/12000 [41:26<09:58,  2.44s/it]
 98%|█████████▊| 11756/12000 [41:28<09:13,  2.27s/it]
                                                     

 98%|█████████▊| 11756/12000 [41:28<09:13,  2.27s/it]
 98%|█████████▊| 11757/12000 [41:29<08:18,  2.05s/it]
                                                     

 98%|█████████▊| 11757/12000 [41:29<08:18,  2.05s/it]
 98%|█████████▊| 11758/12000 [41:31<07:37,  1.89s/it]
                                                     

 98%|█████████▊| 11758/12000 [41:31<07:37,  1.89s/it]
 98%|█████████▊| 11759/12000 [41:33<07:24,  1.84s/it]
                                                     

 98%|█████████▊| 11759/12000 [41:33<07:24,  1.84s/it]
 98%|█████████▊| 11760/12000 [41:34<07:04,  1.77s/it]
                                                     

 98%|█████████▊| 11760/12000 [41:34<07:04,  1.77s/it]
 98%|█████████▊| 11761/12000 [41:38<08:58,  2.25s/it]
                                                     

 98%|█████████▊| 11761/12000 [41:38<08:58,  2.25s/it]
 98%|█████████▊| 11762/12000 [41:40<09:38,  2.43s/it]
                                                     

 98%|█████████▊| 11762/12000 [41:40<09:38,  2.43s/it]
 98%|█████████▊| 11763/12000 [41:44<11:01,  2.79s/it]
                                                     

 98%|█████████▊| 11763/12000 [41:44<11:01,  2.79s/it]
 98%|█████████▊| 11764/12000 [41:46<09:34,  2.43s/it]
                                                     

 98%|█████████▊| 11764/12000 [41:46<09:34,  2.43s/it]
 98%|█████████▊| 11765/12000 [41:47<08:29,  2.17s/it]
                                                     

 98%|█████████▊| 11765/12000 [41:47<08:29,  2.17s/it]
 98%|█████████▊| 11766/12000 [41:49<08:10,  2.10s/it]
                                                     

 98%|█████████▊| 11766/12000 [41:49<08:10,  2.10s/it]
 98%|█████████▊| 11767/12000 [41:51<07:57,  2.05s/it]
                                                     

 98%|█████████▊| 11767/12000 [41:51<07:57,  2.05s/it]
 98%|█████████▊| 11768/12000 [41:54<08:49,  2.28s/it]
                                                     

 98%|█████████▊| 11768/12000 [41:54<08:49,  2.28s/it]
 98%|█████████▊| 11769/12000 [41:56<08:11,  2.13s/it]
                                                     

 98%|█████████▊| 11769/12000 [41:56<08:11,  2.13s/it]
 98%|█████████▊| 11770/12000 [41:57<07:41,  2.01s/it]
                                                     

 98%|█████████▊| 11770/12000 [41:57<07:41,  2.01s/it]
 98%|█████████▊| 11771/12000 [41:59<07:21,  1.93s/it]
                                                     

 98%|█████████▊| 11771/12000 [41:59<07:21,  1.93s/it]
 98%|█████████▊| 11772/12000 [42:01<07:09,  1.88s/it]
                                                     

 98%|█████████▊| 11772/12000 [42:01<07:09,  1.88s/it]
 98%|█████████▊| 11773/12000 [42:03<07:25,  1.96s/it]
                                                     

 98%|█████████▊| 11773/12000 [42:03<07:25,  1.96s/it]
 98%|█████████▊| 11774/12000 [42:07<09:12,  2.44s/it]
                                                     

 98%|█████████▊| 11774/12000 [42:07<09:12,  2.44s/it]
 98%|█████████▊| 11775/12000 [42:08<08:14,  2.20s/it]
                                                     

 98%|█████████▊| 11775/12000 [42:08<08:14,  2.20s/it]
 98%|█████████▊| 11776/12000 [42:10<07:30,  2.01s/it]
                                                     
{'loss': '0.8046', 'grad_norm': '0.3831', 'learning_rate': '2.808e-07', 'entropy': '0.9246', 'num_tokens': '6.1e+06', 'mean_token_accuracy': '0.7731', 'epoch': '0.1038'}
{'loss': '0.731', 'grad_norm': '0.3784', 'learning_rate': '2.787e-07', 'entropy': '0.7516', 'num_tokens': '6.106e+06', 'mean_token_accuracy': '0.7968', 'epoch': '0.1038'}
{'loss': '0.8308', 'grad_norm': '0.3469', 'learning_rate': '2.767e-07', 'entropy': '0.7455', 'num_tokens': '6.112e+06', 'mean_token_accuracy': '0.8074', 'epoch': '0.1038'}
{'loss': '0.7075', 'grad_norm': '0.4783', 'learning_rate': '2.746e-07', 'entropy': '0.7088', 'num_tokens': '6.117e+06', 'mean_token_accuracy': '0.817', 'epoch': '0.1038'}
{'loss': '0.5845', 'grad_norm': '0.3673', 'learning_rate': '2.726e-07', 'entropy': '0.6138', 'num_tokens': '6.121e+06', 'mean_token_accuracy': '0.8483', 'epoch': '0.1038'}
{'loss': '0.716', 'grad_norm': '0.353', 'learning_rate': '2.706e-07', 'entropy': '0.738', 'num_tokens': '6.124e+06', 'mean_token_accuracy': '0.7928', 'epoch': '0.1038'}
{'loss': '0.8915', 'grad_norm': '0.4334', 'learning_rate': '2.686e-07', 'entropy': '0.7747', 'num_tokens': '6.131e+06', 'mean_token_accuracy': '0.8098', 'epoch': '0.1039'}
{'loss': '1.041', 'grad_norm': '0.4837', 'learning_rate': '2.665e-07', 'entropy': '1.101', 'num_tokens': '6.135e+06', 'mean_token_accuracy': '0.7335', 'epoch': '0.1039'}
{'loss': '0.7744', 'grad_norm': '0.434', 'learning_rate': '2.645e-07', 'entropy': '0.7337', 'num_tokens': '6.14e+06', 'mean_token_accuracy': '0.8103', 'epoch': '0.1039'}
{'loss': '0.8562', 'grad_norm': '0.2754', 'learning_rate': '2.625e-07', 'entropy': '0.8594', 'num_tokens': '6.15e+06', 'mean_token_accuracy': '0.7947', 'epoch': '0.1039'}
{'loss': '0.6371', 'grad_norm': '0.3838', 'learning_rate': '2.605e-07', 'entropy': '0.6288', 'num_tokens': '6.156e+06', 'mean_token_accuracy': '0.829', 'epoch': '0.1039'}
{'loss': '0.8036', 'grad_norm': '0.3712', 'learning_rate': '2.586e-07', 'entropy': '0.8316', 'num_tokens': '6.161e+06', 'mean_token_accuracy': '0.7924', 'epoch': '0.1039'}
{'loss': '0.5596', 'grad_norm': '0.4403', 'learning_rate': '2.566e-07', 'entropy': '0.6747', 'num_tokens': '6.165e+06', 'mean_token_accuracy': '0.832', 'epoch': '0.1039'}
{'loss': '0.6334', 'grad_norm': '0.4583', 'learning_rate': '2.546e-07', 'entropy': '0.605', 'num_tokens': '6.17e+06', 'mean_token_accuracy': '0.8354', 'epoch': '0.1039'}
{'loss': '0.8262', 'grad_norm': '0.4216', 'learning_rate': '2.526e-07', 'entropy': '0.8118', 'num_tokens': '6.174e+06', 'mean_token_accuracy': '0.7853', 'epoch': '0.1039'}
{'loss': '0.6192', 'grad_norm': '0.3801', 'learning_rate': '2.507e-07', 'entropy': '0.6062', 'num_tokens': '6.177e+06', 'mean_token_accuracy': '0.839', 'epoch': '0.1039'}
{'loss': '0.9081', 'grad_norm': '0.4524', 'learning_rate': '2.487e-07', 'entropy': '0.9313', 'num_tokens': '6.183e+06', 'mean_token_accuracy': '0.7805', 'epoch': '0.1039'}
{'loss': '0.7456', 'grad_norm': '0.3378', 'learning_rate': '2.468e-07', 'entropy': '0.6633', 'num_tokens': '6.193e+06', 'mean_token_accuracy': '0.8313', 'epoch': '0.104'}
{'loss': '0.6824', 'grad_norm': '0.4197', 'learning_rate': '2.449e-07', 'entropy': '0.719', 'num_tokens': '6.198e+06', 'mean_token_accuracy': '0.8096', 'epoch': '0.104'}
{'loss': '0.9629', 'grad_norm': '0.4092', 'learning_rate': '2.43e-07', 'entropy': '0.991', 'num_tokens': '6.203e+06', 'mean_token_accuracy': '0.7603', 'epoch': '0.104'}
{'loss': '0.7294', 'grad_norm': '0.4885', 'learning_rate': '2.41e-07', 'entropy': '0.7719', 'num_tokens': '6.207e+06', 'mean_token_accuracy': '0.7998', 'epoch': '0.104'}
{'loss': '0.8063', 'grad_norm': '0.3792', 'learning_rate': '2.391e-07', 'entropy': '0.6979', 'num_tokens': '6.216e+06', 'mean_token_accuracy': '0.8258', 'epoch': '0.104'}
{'loss': '0.6478', 'grad_norm': '0.3864', 'learning_rate': '2.372e-07', 'entropy': '0.6766', 'num_tokens': '6.221e+06', 'mean_token_accuracy': '0.8369', 'epoch': '0.104'}
{'loss': '0.7246', 'grad_norm': '0.3491', 'learning_rate': '2.353e-07', 'entropy': '0.7431', 'num_tokens': '6.228e+06', 'mean_token_accuracy': '0.816', 'epoch': '0.104'}
{'loss': '0.7109', 'grad_norm': '0.294', 'learning_rate': '2.335e-07', 'entropy': '0.8304', 'num_tokens': '6.235e+06', 'mean_token_accuracy': '0.7898', 'epoch': '0.104'}
{'loss': '0.7595', 'grad_norm': '0.3528', 'learning_rate': '2.316e-07', 'entropy': '0.7535', 'num_tokens': '6.241e+06', 'mean_token_accuracy': '0.8007', 'epoch': '0.104'}
{'loss': '0.9182', 'grad_norm': '0.331', 'learning_rate': '2.297e-07', 'entropy': '0.9822', 'num_tokens': '6.247e+06', 'mean_token_accuracy': '0.7559', 'epoch': '0.104'}
{'loss': '0.6997', 'grad_norm': '0.3367', 'learning_rate': '2.278e-07', 'entropy': '0.7153', 'num_tokens': '6.251e+06', 'mean_token_accuracy': '0.8151', 'epoch': '0.104'}
{'loss': '0.7501', 'grad_norm': '0.4006', 'learning_rate': '2.26e-07', 'entropy': '0.8234', 'num_tokens': '6.254e+06', 'mean_token_accuracy': '0.7887', 'epoch': '0.104'}
{'loss': '0.7084', 'grad_norm': '0.4886', 'learning_rate': '2.241e-07', 'entropy': '0.7477', 'num_tokens': '6.259e+06', 'mean_token_accuracy': '0.8008', 'epoch': '0.1041'}
{'loss': '0.6974', 'grad_norm': '0.405', 'learning_rate': '2.223e-07', 'entropy': '0.7147', 'num_tokens': '6.265e+06', 'mean_token_accuracy': '0.8102', 'epoch': '0.1041'}
{'loss': '1', 'grad_norm': '0.4637', 'learning_rate': '2.205e-07', 'entropy': '0.9684', 'num_tokens': '6.269e+06', 'mean_token_accuracy': '0.7606', 'epoch': '0.1041'}
{'loss': '0.8398', 'grad_norm': '0.3815', 'learning_rate': '2.186e-07', 'entropy': '0.7901', 'num_tokens': '6.277e+06', 'mean_token_accuracy': '0.8014', 'epoch': '0.1041'}
{'loss': '0.972', 'grad_norm': '0.3702', 'learning_rate': '2.168e-07', 'entropy': '0.9626', 'num_tokens': '6.286e+06', 'mean_token_accuracy': '0.7537', 'epoch': '0.1041'}
{'loss': '0.6338', 'grad_norm': '0.3371', 'learning_rate': '2.15e-07', 'entropy': '0.6424', 'num_tokens': '6.29e+06', 'mean_token_accuracy': '0.8196', 'epoch': '0.1041'}
{'loss': '0.6529', 'grad_norm': '0.3623', 'learning_rate': '2.132e-07', 'entropy': '0.6392', 'num_tokens': '6.294e+06', 'mean_token_accuracy': '0.8207', 'epoch': '0.1041'}
{'loss': '0.6426', 'grad_norm': '0.3669', 'learning_rate': '2.114e-07', 'entropy': '0.6499', 'num_tokens': '6.299e+06', 'mean_token_accuracy': '0.8282', 'epoch': '0.1041'}
{'loss': '1.013', 'grad_norm': '0.4246', 'learning_rate': '2.096e-07', 'entropy': '1.028', 'num_tokens': '6.304e+06', 'mean_token_accuracy': '0.7442', 'epoch': '0.1041'}
{'loss': '0.9717', 'grad_norm': '0.5724', 'learning_rate': '2.078e-07', 'entropy': '0.9823', 'num_tokens': '6.309e+06', 'mean_token_accuracy': '0.7684', 'epoch': '0.1041'}
{'loss': '0.4706', 'grad_norm': '0.3012', 'learning_rate': '2.061e-07', 'entropy': '0.582', 'num_tokens': '6.315e+06', 'mean_token_accuracy': '0.8527', 'epoch': '0.1041'}
{'loss': '0.7389', 'grad_norm': '0.5532', 'learning_rate': '2.043e-07', 'entropy': '0.8013', 'num_tokens': '6.32e+06', 'mean_token_accuracy': '0.7956', 'epoch': '0.1042'}
{'loss': '0.4839', 'grad_norm': '0.3997', 'learning_rate': '2.026e-07', 'entropy': '0.5121', 'num_tokens': '6.324e+06', 'mean_token_accuracy': '0.857', 'epoch': '0.1042'}
{'loss': '1.025', 'grad_norm': '0.4155', 'learning_rate': '2.008e-07', 'entropy': '1.068', 'num_tokens': '6.329e+06', 'mean_token_accuracy': '0.7453', 'epoch': '0.1042'}
{'loss': '0.6236', 'grad_norm': '0.4087', 'learning_rate': '1.991e-07', 'entropy': '0.6207', 'num_tokens': '6.335e+06', 'mean_token_accuracy': '0.8469', 'epoch': '0.1042'}
{'loss': '1.077', 'grad_norm': '0.393', 'learning_rate': '1.973e-07', 'entropy': '0.9679', 'num_tokens': '6.341e+06', 'mean_token_accuracy': '0.7647', 'epoch': '0.1042'}
{'loss': '0.7468', 'grad_norm': '0.4482', 'learning_rate': '1.956e-07', 'entropy': '0.8648', 'num_tokens': '6.348e+06', 'mean_token_accuracy': '0.7932', 'epoch': '0.1042'}
{'loss': '0.7787', 'grad_norm': '0.3937', 'learning_rate': '1.939e-07', 'entropy': '0.7642', 'num_tokens': '6.352e+06', 'mean_token_accuracy': '0.808', 'epoch': '0.1042'}

 98%|█████████▊| 11776/12000 [42:10<07:30,  2.01s/it]
 98%|█████████▊| 11777/12000 [42:12<07:34,  2.04s/it]
                                                     

 98%|█████████▊| 11777/12000 [42:12<07:34,  2.04s/it]
 98%|█████████▊| 11778/12000 [42:13<06:57,  1.88s/it]
                                                     

 98%|█████████▊| 11778/12000 [42:13<06:57,  1.88s/it]
 98%|█████████▊| 11779/12000 [42:15<06:27,  1.75s/it]
                                                     

 98%|█████████▊| 11779/12000 [42:15<06:27,  1.75s/it]
 98%|█████████▊| 11780/12000 [42:17<06:17,  1.71s/it]
                                                     

 98%|█████████▊| 11780/12000 [42:17<06:17,  1.71s/it]
 98%|█████████▊| 11781/12000 [42:18<06:31,  1.79s/it]
                                                     

 98%|█████████▊| 11781/12000 [42:18<06:31,  1.79s/it]
 98%|█████████▊| 11782/12000 [42:20<06:17,  1.73s/it]
                                                     

 98%|█████████▊| 11782/12000 [42:20<06:17,  1.73s/it]
 98%|█████████▊| 11783/12000 [42:22<06:55,  1.92s/it]
                                                     

 98%|█████████▊| 11783/12000 [42:22<06:55,  1.92s/it]
 98%|█████████▊| 11784/12000 [42:25<07:29,  2.08s/it]
                                                     

 98%|█████████▊| 11784/12000 [42:25<07:29,  2.08s/it]
 98%|█████████▊| 11785/12000 [42:29<09:13,  2.58s/it]
                                                     

 98%|█████████▊| 11785/12000 [42:29<09:13,  2.58s/it]
 98%|█████████▊| 11786/12000 [42:30<07:55,  2.22s/it]
                                                     

 98%|█████████▊| 11786/12000 [42:30<07:55,  2.22s/it]
 98%|█████████▊| 11787/12000 [42:31<07:03,  1.99s/it]
                                                     

 98%|█████████▊| 11787/12000 [42:31<07:03,  1.99s/it]
 98%|█████████▊| 11788/12000 [42:33<06:26,  1.82s/it]
                                                     

 98%|█████████▊| 11788/12000 [42:33<06:26,  1.82s/it]
 98%|█████████▊| 11789/12000 [42:35<06:36,  1.88s/it]
                                                     

 98%|█████████▊| 11789/12000 [42:35<06:36,  1.88s/it]
 98%|█████████▊| 11790/12000 [42:36<06:16,  1.79s/it]
                                                     

 98%|█████████▊| 11790/12000 [42:36<06:16,  1.79s/it]
 98%|█████████▊| 11791/12000 [42:39<06:54,  1.98s/it]
                                                     

 98%|█████████▊| 11791/12000 [42:39<06:54,  1.98s/it]
 98%|█████████▊| 11792/12000 [42:42<07:54,  2.28s/it]
                                                     

 98%|█████████▊| 11792/12000 [42:42<07:54,  2.28s/it]
 98%|█████████▊| 11793/12000 [42:44<07:17,  2.11s/it]
                                                     

 98%|█████████▊| 11793/12000 [42:44<07:17,  2.11s/it]
 98%|█████████▊| 11794/12000 [42:45<07:00,  2.04s/it]
                                                     

 98%|█████████▊| 11794/12000 [42:45<07:00,  2.04s/it]
 98%|█████████▊| 11795/12000 [42:48<06:59,  2.05s/it]
                                                     

 98%|█████████▊| 11795/12000 [42:48<06:59,  2.05s/it]
 98%|█████████▊| 11796/12000 [42:49<06:39,  1.96s/it]
                                                     

 98%|█████████▊| 11796/12000 [42:49<06:39,  1.96s/it]
 98%|█████████▊| 11797/12000 [42:53<07:59,  2.36s/it]
                                                     

 98%|█████████▊| 11797/12000 [42:53<07:59,  2.36s/it]
 98%|█████████▊| 11798/12000 [42:54<07:07,  2.11s/it]
                                                     

 98%|█████████▊| 11798/12000 [42:54<07:07,  2.11s/it]
 98%|█████████▊| 11799/12000 [42:56<06:54,  2.06s/it]
                                                     

 98%|█████████▊| 11799/12000 [42:56<06:54,  2.06s/it]
 98%|█████████▊| 11800/12000 [42:58<06:36,  1.98s/it]
                                                     

 98%|█████████▊| 11800/12000 [42:58<06:36,  1.98s/it]
 98%|█████████▊| 11801/12000 [43:01<07:44,  2.34s/it]
                                                     

 98%|█████████▊| 11801/12000 [43:01<07:44,  2.34s/it]
 98%|█████████▊| 11802/12000 [43:04<08:45,  2.65s/it]
                                                     

 98%|█████████▊| 11802/12000 [43:04<08:45,  2.65s/it]
 98%|█████████▊| 11803/12000 [43:06<07:35,  2.31s/it]
                                                     

 98%|█████████▊| 11803/12000 [43:06<07:35,  2.31s/it]
 98%|█████████▊| 11804/12000 [43:08<06:51,  2.10s/it]
                                                     

 98%|█████████▊| 11804/12000 [43:08<06:51,  2.10s/it]
 98%|█████████▊| 11805/12000 [43:12<08:47,  2.70s/it]
                                                     

 98%|█████████▊| 11805/12000 [43:12<08:47,  2.70s/it]
 98%|█████████▊| 11806/12000 [43:14<07:54,  2.45s/it]
                                                     

 98%|█████████▊| 11806/12000 [43:14<07:54,  2.45s/it]
 98%|█████████▊| 11807/12000 [43:18<09:33,  2.97s/it]
                                                     

 98%|█████████▊| 11807/12000 [43:18<09:33,  2.97s/it]
 98%|█████████▊| 11808/12000 [43:19<08:12,  2.57s/it]
                                                     

 98%|█████████▊| 11808/12000 [43:19<08:12,  2.57s/it]
 98%|█████████▊| 11809/12000 [43:21<07:31,  2.37s/it]
                                                     

 98%|█████████▊| 11809/12000 [43:21<07:31,  2.37s/it]
 98%|█████████▊| 11810/12000 [43:25<08:22,  2.64s/it]
                                                     

 98%|█████████▊| 11810/12000 [43:25<08:22,  2.64s/it]
 98%|█████████▊| 11811/12000 [43:29<09:57,  3.16s/it]
                                                     

 98%|█████████▊| 11811/12000 [43:29<09:57,  3.16s/it]
 98%|█████████▊| 11812/12000 [43:31<08:35,  2.74s/it]
                                                     

 98%|█████████▊| 11812/12000 [43:31<08:35,  2.74s/it]
 98%|█████████▊| 11813/12000 [43:33<07:53,  2.53s/it]
                                                     

 98%|█████████▊| 11813/12000 [43:33<07:53,  2.53s/it]
 98%|█████████▊| 11814/12000 [43:34<06:57,  2.25s/it]
                                                     

 98%|█████████▊| 11814/12000 [43:34<06:57,  2.25s/it]
 98%|█████████▊| 11815/12000 [43:37<07:02,  2.28s/it]
                                                     

 98%|█████████▊| 11815/12000 [43:37<07:02,  2.28s/it]
 98%|█████████▊| 11816/12000 [43:39<07:08,  2.33s/it]
                                                     

 98%|█████████▊| 11816/12000 [43:39<07:08,  2.33s/it]
 98%|█████████▊| 11817/12000 [43:41<06:19,  2.07s/it]
                                                     

 98%|█████████▊| 11817/12000 [43:41<06:19,  2.07s/it]
 98%|█████████▊| 11818/12000 [43:42<05:52,  1.94s/it]
                                                     

 98%|█████████▊| 11818/12000 [43:42<05:52,  1.94s/it]
 98%|█████████▊| 11819/12000 [43:47<08:00,  2.66s/it]
                                                     

 98%|█████████▊| 11819/12000 [43:47<08:00,  2.66s/it]
 98%|█████████▊| 11820/12000 [43:49<07:37,  2.54s/it]
                                                     

 98%|█████████▊| 11820/12000 [43:49<07:37,  2.54s/it]
 99%|█████████▊| 11821/12000 [43:50<06:41,  2.24s/it]
                                                     

 99%|█████████▊| 11821/12000 [43:50<06:41,  2.24s/it]
 99%|█████████▊| 11822/12000 [43:53<07:05,  2.39s/it]
                                                     

 99%|█████████▊| 11822/12000 [43:53<07:05,  2.39s/it]
 99%|█████████▊| 11823/12000 [43:56<07:23,  2.51s/it]
                                                     
{'loss': '0.7181', 'grad_norm': '0.415', 'learning_rate': '1.922e-07', 'entropy': '0.7399', 'num_tokens': '6.357e+06', 'mean_token_accuracy': '0.7986', 'epoch': '0.1042'}
{'loss': '1.012', 'grad_norm': '0.4015', 'learning_rate': '1.905e-07', 'entropy': '1.087', 'num_tokens': '6.363e+06', 'mean_token_accuracy': '0.7433', 'epoch': '0.1042'}
{'loss': '0.7213', 'grad_norm': '0.4044', 'learning_rate': '1.888e-07', 'entropy': '0.7122', 'num_tokens': '6.367e+06', 'mean_token_accuracy': '0.8227', 'epoch': '0.1042'}
{'loss': '0.5743', 'grad_norm': '0.3672', 'learning_rate': '1.871e-07', 'entropy': '0.586', 'num_tokens': '6.371e+06', 'mean_token_accuracy': '0.8432', 'epoch': '0.1042'}
{'loss': '0.722', 'grad_norm': '0.4058', 'learning_rate': '1.854e-07', 'entropy': '0.7262', 'num_tokens': '6.376e+06', 'mean_token_accuracy': '0.8018', 'epoch': '0.1043'}
{'loss': '1.259', 'grad_norm': '0.455', 'learning_rate': '1.837e-07', 'entropy': '1.138', 'num_tokens': '6.381e+06', 'mean_token_accuracy': '0.7274', 'epoch': '0.1043'}
{'loss': '0.7506', 'grad_norm': '0.4294', 'learning_rate': '1.821e-07', 'entropy': '0.6903', 'num_tokens': '6.385e+06', 'mean_token_accuracy': '0.8155', 'epoch': '0.1043'}
{'loss': '0.9214', 'grad_norm': '0.5379', 'learning_rate': '1.804e-07', 'entropy': '0.8481', 'num_tokens': '6.391e+06', 'mean_token_accuracy': '0.7821', 'epoch': '0.1043'}
{'loss': '0.6554', 'grad_norm': '0.3204', 'learning_rate': '1.788e-07', 'entropy': '0.8412', 'num_tokens': '6.398e+06', 'mean_token_accuracy': '0.7836', 'epoch': '0.1043'}
{'loss': '0.6162', 'grad_norm': '0.3161', 'learning_rate': '1.771e-07', 'entropy': '0.6036', 'num_tokens': '6.403e+06', 'mean_token_accuracy': '0.8454', 'epoch': '0.1043'}
{'loss': '0.7244', 'grad_norm': '0.3728', 'learning_rate': '1.755e-07', 'entropy': '0.73', 'num_tokens': '6.407e+06', 'mean_token_accuracy': '0.812', 'epoch': '0.1043'}
{'loss': '0.8617', 'grad_norm': '0.4682', 'learning_rate': '1.738e-07', 'entropy': '0.8894', 'num_tokens': '6.411e+06', 'mean_token_accuracy': '0.7797', 'epoch': '0.1043'}
{'loss': '0.863', 'grad_norm': '0.5364', 'learning_rate': '1.722e-07', 'entropy': '0.8525', 'num_tokens': '6.414e+06', 'mean_token_accuracy': '0.7733', 'epoch': '0.1043'}
{'loss': '1.031', 'grad_norm': '0.593', 'learning_rate': '1.706e-07', 'entropy': '0.9814', 'num_tokens': '6.419e+06', 'mean_token_accuracy': '0.7612', 'epoch': '0.1043'}
{'loss': '1.133', 'grad_norm': '0.3986', 'learning_rate': '1.69e-07', 'entropy': '1.12', 'num_tokens': '6.424e+06', 'mean_token_accuracy': '0.7328', 'epoch': '0.1043'}
{'loss': '0.8129', 'grad_norm': '0.3894', 'learning_rate': '1.674e-07', 'entropy': '0.8379', 'num_tokens': '6.43e+06', 'mean_token_accuracy': '0.7901', 'epoch': '0.1043'}
{'loss': '0.6983', 'grad_norm': '0.3458', 'learning_rate': '1.658e-07', 'entropy': '0.6745', 'num_tokens': '6.437e+06', 'mean_token_accuracy': '0.8174', 'epoch': '0.1044'}
{'loss': '0.6675', 'grad_norm': '0.453', 'learning_rate': '1.642e-07', 'entropy': '0.7369', 'num_tokens': '6.441e+06', 'mean_token_accuracy': '0.8143', 'epoch': '0.1044'}
{'loss': '0.7223', 'grad_norm': '0.399', 'learning_rate': '1.627e-07', 'entropy': '0.6949', 'num_tokens': '6.447e+06', 'mean_token_accuracy': '0.8245', 'epoch': '0.1044'}
{'loss': '0.8972', 'grad_norm': '0.4715', 'learning_rate': '1.611e-07', 'entropy': '0.8304', 'num_tokens': '6.453e+06', 'mean_token_accuracy': '0.7935', 'epoch': '0.1044'}
{'loss': '0.922', 'grad_norm': '0.4463', 'learning_rate': '1.595e-07', 'entropy': '0.9288', 'num_tokens': '6.459e+06', 'mean_token_accuracy': '0.768', 'epoch': '0.1044'}
{'loss': '0.8038', 'grad_norm': '0.4426', 'learning_rate': '1.58e-07', 'entropy': '0.9064', 'num_tokens': '6.466e+06', 'mean_token_accuracy': '0.7767', 'epoch': '0.1044'}
{'loss': '0.732', 'grad_norm': '0.3723', 'learning_rate': '1.564e-07', 'entropy': '0.7939', 'num_tokens': '6.471e+06', 'mean_token_accuracy': '0.7891', 'epoch': '0.1044'}
{'loss': '1.372', 'grad_norm': '0.4342', 'learning_rate': '1.549e-07', 'entropy': '1.235', 'num_tokens': '6.475e+06', 'mean_token_accuracy': '0.678', 'epoch': '0.1044'}
{'loss': '0.7224', 'grad_norm': '0.379', 'learning_rate': '1.534e-07', 'entropy': '0.7169', 'num_tokens': '6.481e+06', 'mean_token_accuracy': '0.816', 'epoch': '0.1044'}
{'loss': '0.6573', 'grad_norm': '0.413', 'learning_rate': '1.518e-07', 'entropy': '0.6825', 'num_tokens': '6.486e+06', 'mean_token_accuracy': '0.8189', 'epoch': '0.1044'}
{'loss': '0.5857', 'grad_norm': '0.3569', 'learning_rate': '1.503e-07', 'entropy': '0.7002', 'num_tokens': '6.496e+06', 'mean_token_accuracy': '0.8239', 'epoch': '0.1044'}
{'loss': '1.089', 'grad_norm': '0.3979', 'learning_rate': '1.488e-07', 'entropy': '0.9284', 'num_tokens': '6.5e+06', 'mean_token_accuracy': '0.7679', 'epoch': '0.1045'}
{'loss': '0.7874', 'grad_norm': '0.4102', 'learning_rate': '1.473e-07', 'entropy': '0.8518', 'num_tokens': '6.506e+06', 'mean_token_accuracy': '0.7874', 'epoch': '0.1045'}
{'loss': '1.491', 'grad_norm': '0.3817', 'learning_rate': '1.458e-07', 'entropy': '1.211', 'num_tokens': '6.513e+06', 'mean_token_accuracy': '0.7192', 'epoch': '0.1045'}
{'loss': '0.7997', 'grad_norm': '0.3603', 'learning_rate': '1.444e-07', 'entropy': '0.7808', 'num_tokens': '6.52e+06', 'mean_token_accuracy': '0.792', 'epoch': '0.1045'}
{'loss': '0.7491', 'grad_norm': '0.4073', 'learning_rate': '1.429e-07', 'entropy': '0.7485', 'num_tokens': '6.525e+06', 'mean_token_accuracy': '0.8108', 'epoch': '0.1045'}
{'loss': '1.013', 'grad_norm': '0.5041', 'learning_rate': '1.414e-07', 'entropy': '0.9719', 'num_tokens': '6.529e+06', 'mean_token_accuracy': '0.749', 'epoch': '0.1045'}
{'loss': '1.025', 'grad_norm': '0.4975', 'learning_rate': '1.399e-07', 'entropy': '1.087', 'num_tokens': '6.533e+06', 'mean_token_accuracy': '0.7402', 'epoch': '0.1045'}
{'loss': '0.8425', 'grad_norm': '0.3311', 'learning_rate': '1.385e-07', 'entropy': '0.8203', 'num_tokens': '6.541e+06', 'mean_token_accuracy': '0.7981', 'epoch': '0.1045'}
{'loss': '1.046', 'grad_norm': '0.3741', 'learning_rate': '1.37e-07', 'entropy': '1.046', 'num_tokens': '6.552e+06', 'mean_token_accuracy': '0.7485', 'epoch': '0.1045'}
{'loss': '0.8174', 'grad_norm': '0.4079', 'learning_rate': '1.356e-07', 'entropy': '0.8989', 'num_tokens': '6.558e+06', 'mean_token_accuracy': '0.7732', 'epoch': '0.1045'}
{'loss': '0.8695', 'grad_norm': '0.4944', 'learning_rate': '1.342e-07', 'entropy': '0.8504', 'num_tokens': '6.563e+06', 'mean_token_accuracy': '0.7691', 'epoch': '0.1045'}
{'loss': '0.5871', 'grad_norm': '0.3773', 'learning_rate': '1.328e-07', 'entropy': '0.5868', 'num_tokens': '6.567e+06', 'mean_token_accuracy': '0.844', 'epoch': '0.1046'}
{'loss': '1.539', 'grad_norm': '0.4649', 'learning_rate': '1.313e-07', 'entropy': '1.322', 'num_tokens': '6.573e+06', 'mean_token_accuracy': '0.6849', 'epoch': '0.1046'}
{'loss': '0.9764', 'grad_norm': '0.4034', 'learning_rate': '1.299e-07', 'entropy': '1.021', 'num_tokens': '6.579e+06', 'mean_token_accuracy': '0.7529', 'epoch': '0.1046'}
{'loss': '0.9961', 'grad_norm': '0.3916', 'learning_rate': '1.285e-07', 'entropy': '1.112', 'num_tokens': '6.583e+06', 'mean_token_accuracy': '0.7322', 'epoch': '0.1046'}
{'loss': '0.6594', 'grad_norm': '0.3875', 'learning_rate': '1.271e-07', 'entropy': '0.6515', 'num_tokens': '6.588e+06', 'mean_token_accuracy': '0.8311', 'epoch': '0.1046'}
{'loss': '0.5557', 'grad_norm': '0.2983', 'learning_rate': '1.258e-07', 'entropy': '0.5636', 'num_tokens': '6.597e+06', 'mean_token_accuracy': '0.861', 'epoch': '0.1046'}
{'loss': '1.156', 'grad_norm': '0.357', 'learning_rate': '1.244e-07', 'entropy': '1.188', 'num_tokens': '6.603e+06', 'mean_token_accuracy': '0.7296', 'epoch': '0.1046'}
{'loss': '0.9371', 'grad_norm': '0.4222', 'learning_rate': '1.23e-07', 'entropy': '0.9424', 'num_tokens': '6.607e+06', 'mean_token_accuracy': '0.7584', 'epoch': '0.1046'}
{'loss': '0.9379', 'grad_norm': '0.3768', 'learning_rate': '1.216e-07', 'entropy': '0.8905', 'num_tokens': '6.614e+06', 'mean_token_accuracy': '0.7833', 'epoch': '0.1046'}

 99%|█████████▊| 11823/12000 [43:56<07:23,  2.51s/it]
 99%|█████████▊| 11824/12000 [43:57<06:31,  2.22s/it]
                                                     

 99%|█████████▊| 11824/12000 [43:57<06:31,  2.22s/it]
 99%|█████████▊| 11825/12000 [44:00<06:40,  2.29s/it]
                                                     

 99%|█████████▊| 11825/12000 [44:00<06:40,  2.29s/it]
 99%|█████████▊| 11826/12000 [44:01<05:52,  2.03s/it]
                                                     

 99%|█████████▊| 11826/12000 [44:01<05:52,  2.03s/it]
 99%|█████████▊| 11827/12000 [44:03<05:46,  2.00s/it]
                                                     

 99%|█████████▊| 11827/12000 [44:03<05:46,  2.00s/it]
 99%|█████████▊| 11828/12000 [44:05<05:21,  1.87s/it]
                                                     

 99%|█████████▊| 11828/12000 [44:05<05:21,  1.87s/it]
 99%|█████████▊| 11829/12000 [44:09<07:14,  2.54s/it]
                                                     

 99%|█████████▊| 11829/12000 [44:09<07:14,  2.54s/it]
 99%|█████████▊| 11830/12000 [44:11<06:26,  2.27s/it]
                                                     

 99%|█████████▊| 11830/12000 [44:11<06:26,  2.27s/it]
 99%|█████████▊| 11831/12000 [44:12<06:06,  2.17s/it]
                                                     

 99%|█████████▊| 11831/12000 [44:12<06:06,  2.17s/it]
 99%|█████████▊| 11832/12000 [44:14<05:23,  1.92s/it]
                                                     

 99%|█████████▊| 11832/12000 [44:14<05:23,  1.92s/it]
 99%|█████████▊| 11833/12000 [44:19<08:19,  2.99s/it]
                                                     

 99%|█████████▊| 11833/12000 [44:19<08:19,  2.99s/it]
 99%|█████████▊| 11834/12000 [44:21<06:59,  2.53s/it]
                                                     

 99%|█████████▊| 11834/12000 [44:21<06:59,  2.53s/it]
 99%|█████████▊| 11835/12000 [44:23<06:43,  2.45s/it]
                                                     

 99%|█████████▊| 11835/12000 [44:23<06:43,  2.45s/it]
 99%|█████████▊| 11836/12000 [44:25<06:06,  2.23s/it]
                                                     

 99%|█████████▊| 11836/12000 [44:25<06:06,  2.23s/it]
 99%|█████████▊| 11837/12000 [44:27<05:56,  2.19s/it]
                                                     

 99%|█████████▊| 11837/12000 [44:27<05:56,  2.19s/it]
 99%|█████████▊| 11838/12000 [44:28<05:19,  1.97s/it]
                                                     

 99%|█████████▊| 11838/12000 [44:28<05:19,  1.97s/it]
 99%|█████████▊| 11839/12000 [44:31<05:38,  2.11s/it]
                                                     

 99%|█████████▊| 11839/12000 [44:31<05:38,  2.11s/it]
 99%|█████████▊| 11840/12000 [44:32<05:22,  2.02s/it]
                                                     

 99%|█████████▊| 11840/12000 [44:32<05:22,  2.02s/it]
 99%|█████████▊| 11841/12000 [44:34<04:54,  1.85s/it]
                                                     

 99%|█████████▊| 11841/12000 [44:34<04:54,  1.85s/it]
 99%|█████████▊| 11842/12000 [44:36<04:43,  1.80s/it]
                                                     

 99%|█████████▊| 11842/12000 [44:36<04:43,  1.80s/it]
 99%|█████████▊| 11843/12000 [44:38<04:54,  1.88s/it]
                                                     

 99%|█████████▊| 11843/12000 [44:38<04:54,  1.88s/it]
 99%|█████████▊| 11844/12000 [44:41<06:01,  2.32s/it]
                                                     

 99%|█████████▊| 11844/12000 [44:41<06:01,  2.32s/it]
 99%|█████████▊| 11845/12000 [44:43<05:20,  2.07s/it]
                                                     

 99%|█████████▊| 11845/12000 [44:43<05:20,  2.07s/it]
 99%|█████████▊| 11846/12000 [44:44<05:10,  2.02s/it]
                                                     

 99%|█████████▊| 11846/12000 [44:44<05:10,  2.02s/it]
 99%|█████████▊| 11847/12000 [44:46<04:45,  1.86s/it]
                                                     

 99%|█████████▊| 11847/12000 [44:46<04:45,  1.86s/it]
 99%|█████████▊| 11848/12000 [44:47<04:28,  1.77s/it]
                                                     

 99%|█████████▊| 11848/12000 [44:47<04:28,  1.77s/it]
 99%|█████████▊| 11849/12000 [44:49<04:30,  1.79s/it]
                                                     

 99%|█████████▊| 11849/12000 [44:49<04:30,  1.79s/it]
 99%|█████████▉| 11850/12000 [44:51<04:31,  1.81s/it]
                                                     

 99%|█████████▉| 11850/12000 [44:51<04:31,  1.81s/it]
 99%|█████████▉| 11851/12000 [44:57<07:44,  3.12s/it]
                                                     

 99%|█████████▉| 11851/12000 [44:57<07:44,  3.12s/it]
 99%|█████████▉| 11852/12000 [44:59<06:29,  2.63s/it]
                                                     

 99%|█████████▉| 11852/12000 [44:59<06:29,  2.63s/it]
 99%|█████████▉| 11853/12000 [45:01<06:01,  2.46s/it]
                                                     

 99%|█████████▉| 11853/12000 [45:01<06:01,  2.46s/it]
 99%|█████████▉| 11854/12000 [45:02<05:18,  2.18s/it]
                                                     

 99%|█████████▉| 11854/12000 [45:02<05:18,  2.18s/it]
 99%|█████████▉| 11855/12000 [45:04<04:48,  1.99s/it]
                                                     

 99%|█████████▉| 11855/12000 [45:04<04:48,  1.99s/it]
 99%|█████████▉| 11856/12000 [45:06<05:03,  2.11s/it]
                                                     

 99%|█████████▉| 11856/12000 [45:06<05:03,  2.11s/it]
 99%|█████████▉| 11857/12000 [45:08<04:55,  2.07s/it]
                                                     

 99%|█████████▉| 11857/12000 [45:08<04:55,  2.07s/it]
 99%|█████████▉| 11858/12000 [45:10<04:44,  2.01s/it]
                                                     

 99%|█████████▉| 11858/12000 [45:10<04:44,  2.01s/it]
 99%|█████████▉| 11859/12000 [45:12<04:19,  1.84s/it]
                                                     

 99%|█████████▉| 11859/12000 [45:12<04:19,  1.84s/it]
 99%|█████████▉| 11860/12000 [45:13<04:13,  1.81s/it]
                                                     

 99%|█████████▉| 11860/12000 [45:13<04:13,  1.81s/it]
 99%|█████████▉| 11861/12000 [45:16<04:50,  2.09s/it]
                                                     

 99%|█████████▉| 11861/12000 [45:16<04:50,  2.09s/it]
 99%|█████████▉| 11862/12000 [45:19<05:07,  2.23s/it]
                                                     

 99%|█████████▉| 11862/12000 [45:19<05:07,  2.23s/it]
 99%|█████████▉| 11863/12000 [45:21<05:00,  2.19s/it]
                                                     

 99%|█████████▉| 11863/12000 [45:21<05:00,  2.19s/it]
 99%|█████████▉| 11864/12000 [45:23<04:41,  2.07s/it]
                                                     

 99%|█████████▉| 11864/12000 [45:23<04:41,  2.07s/it]
 99%|█████████▉| 11865/12000 [45:24<04:14,  1.89s/it]
                                                     

 99%|█████████▉| 11865/12000 [45:24<04:14,  1.89s/it]
 99%|█████████▉| 11866/12000 [45:29<05:58,  2.68s/it]
                                                     

 99%|█████████▉| 11866/12000 [45:29<05:58,  2.68s/it]
 99%|█████████▉| 11867/12000 [45:30<05:18,  2.40s/it]
                                                     

 99%|█████████▉| 11867/12000 [45:30<05:18,  2.40s/it]
 99%|█████████▉| 11868/12000 [45:33<05:09,  2.34s/it]
                                                     

 99%|█████████▉| 11868/12000 [45:33<05:09,  2.34s/it]
 99%|█████████▉| 11869/12000 [45:34<04:31,  2.08s/it]
                                                     

 99%|█████████▉| 11869/12000 [45:34<04:31,  2.08s/it]
 99%|█████████▉| 11870/12000 [45:35<04:08,  1.91s/it]
                                                     
{'loss': '0.803', 'grad_norm': '0.4045', 'learning_rate': '1.203e-07', 'entropy': '0.7428', 'num_tokens': '6.62e+06', 'mean_token_accuracy': '0.8141', 'epoch': '0.1046'}
{'loss': '0.8378', 'grad_norm': '0.4103', 'learning_rate': '1.189e-07', 'entropy': '0.785', 'num_tokens': '6.625e+06', 'mean_token_accuracy': '0.8065', 'epoch': '0.1046'}
{'loss': '0.7047', 'grad_norm': '0.3446', 'learning_rate': '1.176e-07', 'entropy': '0.6909', 'num_tokens': '6.631e+06', 'mean_token_accuracy': '0.8166', 'epoch': '0.1046'}
{'loss': '1.049', 'grad_norm': '0.4391', 'learning_rate': '1.163e-07', 'entropy': '1.06', 'num_tokens': '6.635e+06', 'mean_token_accuracy': '0.7247', 'epoch': '0.1047'}
{'loss': '0.915', 'grad_norm': '0.3416', 'learning_rate': '1.149e-07', 'entropy': '0.9143', 'num_tokens': '6.642e+06', 'mean_token_accuracy': '0.7719', 'epoch': '0.1047'}
{'loss': '0.5741', 'grad_norm': '0.4682', 'learning_rate': '1.136e-07', 'entropy': '0.602', 'num_tokens': '6.647e+06', 'mean_token_accuracy': '0.8419', 'epoch': '0.1047'}
{'loss': '1.107', 'grad_norm': '0.4042', 'learning_rate': '1.123e-07', 'entropy': '1.083', 'num_tokens': '6.653e+06', 'mean_token_accuracy': '0.7435', 'epoch': '0.1047'}
{'loss': '0.6379', 'grad_norm': '0.3322', 'learning_rate': '1.11e-07', 'entropy': '0.6625', 'num_tokens': '6.658e+06', 'mean_token_accuracy': '0.8352', 'epoch': '0.1047'}
{'loss': '0.5732', 'grad_norm': '0.4047', 'learning_rate': '1.097e-07', 'entropy': '0.6065', 'num_tokens': '6.663e+06', 'mean_token_accuracy': '0.844', 'epoch': '0.1047'}
{'loss': '1.037', 'grad_norm': '0.5427', 'learning_rate': '1.084e-07', 'entropy': '1.014', 'num_tokens': '6.666e+06', 'mean_token_accuracy': '0.7508', 'epoch': '0.1047'}
{'loss': '0.6322', 'grad_norm': '0.3123', 'learning_rate': '1.072e-07', 'entropy': '0.8407', 'num_tokens': '6.676e+06', 'mean_token_accuracy': '0.7859', 'epoch': '0.1047'}
{'loss': '1.049', 'grad_norm': '0.4793', 'learning_rate': '1.059e-07', 'entropy': '1.079', 'num_tokens': '6.68e+06', 'mean_token_accuracy': '0.731', 'epoch': '0.1047'}
{'loss': '0.9734', 'grad_norm': '0.4161', 'learning_rate': '1.046e-07', 'entropy': '0.9921', 'num_tokens': '6.685e+06', 'mean_token_accuracy': '0.7514', 'epoch': '0.1047'}
{'loss': '1.348', 'grad_norm': '0.4391', 'learning_rate': '1.034e-07', 'entropy': '1.449', 'num_tokens': '6.69e+06', 'mean_token_accuracy': '0.6609', 'epoch': '0.1047'}
{'loss': '0.5977', 'grad_norm': '0.3585', 'learning_rate': '1.021e-07', 'entropy': '0.6346', 'num_tokens': '6.695e+06', 'mean_token_accuracy': '0.8271', 'epoch': '0.1048'}
{'loss': '0.6395', 'grad_norm': '0.39', 'learning_rate': '1.009e-07', 'entropy': '0.6767', 'num_tokens': '6.7e+06', 'mean_token_accuracy': '0.8329', 'epoch': '0.1048'}
{'loss': '0.6693', 'grad_norm': '0.4106', 'learning_rate': '9.964e-08', 'entropy': '0.6543', 'num_tokens': '6.707e+06', 'mean_token_accuracy': '0.8229', 'epoch': '0.1048'}
{'loss': '1.081', 'grad_norm': '0.5242', 'learning_rate': '9.841e-08', 'entropy': '0.9506', 'num_tokens': '6.711e+06', 'mean_token_accuracy': '0.7633', 'epoch': '0.1048'}
{'loss': '1.046', 'grad_norm': '0.5371', 'learning_rate': '9.719e-08', 'entropy': '1.089', 'num_tokens': '6.715e+06', 'mean_token_accuracy': '0.7368', 'epoch': '0.1048'}
{'loss': '0.5621', 'grad_norm': '0.3777', 'learning_rate': '9.598e-08', 'entropy': '0.5532', 'num_tokens': '6.72e+06', 'mean_token_accuracy': '0.8466', 'epoch': '0.1048'}
{'loss': '0.8105', 'grad_norm': '0.3515', 'learning_rate': '9.478e-08', 'entropy': '0.7584', 'num_tokens': '6.726e+06', 'mean_token_accuracy': '0.8189', 'epoch': '0.1048'}
{'loss': '0.5927', 'grad_norm': '0.3871', 'learning_rate': '9.358e-08', 'entropy': '0.6468', 'num_tokens': '6.733e+06', 'mean_token_accuracy': '0.8319', 'epoch': '0.1048'}
{'loss': '0.7705', 'grad_norm': '0.3271', 'learning_rate': '9.239e-08', 'entropy': '0.8809', 'num_tokens': '6.737e+06', 'mean_token_accuracy': '0.78', 'epoch': '0.1048'}
{'loss': '0.8649', 'grad_norm': '0.4049', 'learning_rate': '9.121e-08', 'entropy': '0.9154', 'num_tokens': '6.743e+06', 'mean_token_accuracy': '0.7787', 'epoch': '0.1048'}
{'loss': '0.7769', 'grad_norm': '0.4511', 'learning_rate': '9.004e-08', 'entropy': '0.7835', 'num_tokens': '6.747e+06', 'mean_token_accuracy': '0.792', 'epoch': '0.1048'}
{'loss': '0.7983', 'grad_norm': '0.4505', 'learning_rate': '8.887e-08', 'entropy': '0.824', 'num_tokens': '6.752e+06', 'mean_token_accuracy': '0.7927', 'epoch': '0.1049'}
{'loss': '0.942', 'grad_norm': '0.4421', 'learning_rate': '8.772e-08', 'entropy': '0.9179', 'num_tokens': '6.757e+06', 'mean_token_accuracy': '0.7747', 'epoch': '0.1049'}
{'loss': '0.6831', 'grad_norm': '0.4069', 'learning_rate': '8.657e-08', 'entropy': '0.7186', 'num_tokens': '6.762e+06', 'mean_token_accuracy': '0.8166', 'epoch': '0.1049'}
{'loss': '0.7378', 'grad_norm': '0.351', 'learning_rate': '8.542e-08', 'entropy': '0.6858', 'num_tokens': '6.769e+06', 'mean_token_accuracy': '0.8272', 'epoch': '0.1049'}
{'loss': '0.7756', 'grad_norm': '0.4231', 'learning_rate': '8.429e-08', 'entropy': '0.8189', 'num_tokens': '6.773e+06', 'mean_token_accuracy': '0.7986', 'epoch': '0.1049'}
{'loss': '0.8095', 'grad_norm': '0.3822', 'learning_rate': '8.316e-08', 'entropy': '0.8132', 'num_tokens': '6.779e+06', 'mean_token_accuracy': '0.7908', 'epoch': '0.1049'}
{'loss': '0.6412', 'grad_norm': '0.448', 'learning_rate': '8.204e-08', 'entropy': '0.7171', 'num_tokens': '6.783e+06', 'mean_token_accuracy': '0.8088', 'epoch': '0.1049'}
{'loss': '0.7169', 'grad_norm': '0.4942', 'learning_rate': '8.093e-08', 'entropy': '0.7228', 'num_tokens': '6.787e+06', 'mean_token_accuracy': '0.8172', 'epoch': '0.1049'}
{'loss': '0.9109', 'grad_norm': '0.4437', 'learning_rate': '7.982e-08', 'entropy': '0.8463', 'num_tokens': '6.793e+06', 'mean_token_accuracy': '0.7884', 'epoch': '0.1049'}
{'loss': '0.7123', 'grad_norm': '0.3568', 'learning_rate': '7.873e-08', 'entropy': '0.6979', 'num_tokens': '6.799e+06', 'mean_token_accuracy': '0.8172', 'epoch': '0.1049'}
{'loss': '0.7133', 'grad_norm': '0.4127', 'learning_rate': '7.764e-08', 'entropy': '0.7249', 'num_tokens': '6.805e+06', 'mean_token_accuracy': '0.8057', 'epoch': '0.1049'}
{'loss': '0.8586', 'grad_norm': '0.4644', 'learning_rate': '7.656e-08', 'entropy': '0.9039', 'num_tokens': '6.808e+06', 'mean_token_accuracy': '0.7699', 'epoch': '0.105'}
{'loss': '0.7174', 'grad_norm': '0.4315', 'learning_rate': '7.548e-08', 'entropy': '0.7762', 'num_tokens': '6.813e+06', 'mean_token_accuracy': '0.8046', 'epoch': '0.105'}
{'loss': '0.8793', 'grad_norm': '0.405', 'learning_rate': '7.442e-08', 'entropy': '0.9201', 'num_tokens': '6.819e+06', 'mean_token_accuracy': '0.759', 'epoch': '0.105'}
{'loss': '0.8598', 'grad_norm': '0.3377', 'learning_rate': '7.336e-08', 'entropy': '0.805', 'num_tokens': '6.827e+06', 'mean_token_accuracy': '0.7964', 'epoch': '0.105'}
{'loss': '0.8667', 'grad_norm': '0.3754', 'learning_rate': '7.23e-08', 'entropy': '0.7137', 'num_tokens': '6.833e+06', 'mean_token_accuracy': '0.8206', 'epoch': '0.105'}
{'loss': '0.9414', 'grad_norm': '0.4017', 'learning_rate': '7.126e-08', 'entropy': '0.9328', 'num_tokens': '6.838e+06', 'mean_token_accuracy': '0.7805', 'epoch': '0.105'}
{'loss': '0.9844', 'grad_norm': '0.4864', 'learning_rate': '7.022e-08', 'entropy': '1.104', 'num_tokens': '6.841e+06', 'mean_token_accuracy': '0.7336', 'epoch': '0.105'}
{'loss': '0.6042', 'grad_norm': '0.2883', 'learning_rate': '6.92e-08', 'entropy': '0.8204', 'num_tokens': '6.848e+06', 'mean_token_accuracy': '0.8046', 'epoch': '0.105'}
{'loss': '0.5668', 'grad_norm': '0.3169', 'learning_rate': '6.817e-08', 'entropy': '0.653', 'num_tokens': '6.854e+06', 'mean_token_accuracy': '0.8369', 'epoch': '0.105'}
{'loss': '0.8763', 'grad_norm': '0.3767', 'learning_rate': '6.716e-08', 'entropy': '0.8824', 'num_tokens': '6.86e+06', 'mean_token_accuracy': '0.7723', 'epoch': '0.105'}
{'loss': '0.7719', 'grad_norm': '0.4121', 'learning_rate': '6.615e-08', 'entropy': '0.7919', 'num_tokens': '6.864e+06', 'mean_token_accuracy': '0.7892', 'epoch': '0.105'}

 99%|█████████▉| 11870/12000 [45:35<04:08,  1.91s/it]
 99%|█████████▉| 11871/12000 [45:37<03:48,  1.77s/it]
                                                     

 99%|█████████▉| 11871/12000 [45:37<03:48,  1.77s/it]
 99%|█████████▉| 11872/12000 [45:38<03:37,  1.70s/it]
                                                     

 99%|█████████▉| 11872/12000 [45:38<03:37,  1.70s/it]
 99%|█████████▉| 11873/12000 [45:42<04:44,  2.24s/it]
                                                     

 99%|█████████▉| 11873/12000 [45:42<04:44,  2.24s/it]
 99%|█████████▉| 11874/12000 [45:44<04:43,  2.25s/it]
                                                     

 99%|█████████▉| 11874/12000 [45:44<04:43,  2.25s/it]
 99%|█████████▉| 11875/12000 [45:46<04:16,  2.05s/it]
                                                     

 99%|█████████▉| 11875/12000 [45:46<04:16,  2.05s/it]
 99%|█████████▉| 11876/12000 [45:47<03:55,  1.90s/it]
                                                     

 99%|█████████▉| 11876/12000 [45:47<03:55,  1.90s/it]
 99%|█████████▉| 11877/12000 [45:49<03:45,  1.83s/it]
                                                     

 99%|█████████▉| 11877/12000 [45:49<03:45,  1.83s/it]
 99%|█████████▉| 11878/12000 [45:51<03:34,  1.76s/it]
                                                     

 99%|█████████▉| 11878/12000 [45:51<03:34,  1.76s/it]
 99%|█████████▉| 11879/12000 [45:52<03:26,  1.70s/it]
                                                     

 99%|█████████▉| 11879/12000 [45:52<03:26,  1.70s/it]
 99%|█████████▉| 11880/12000 [45:54<03:36,  1.81s/it]
                                                     

 99%|█████████▉| 11880/12000 [45:54<03:36,  1.81s/it]
 99%|█████████▉| 11881/12000 [45:56<03:32,  1.78s/it]
                                                     

 99%|█████████▉| 11881/12000 [45:56<03:32,  1.78s/it]
 99%|█████████▉| 11882/12000 [45:58<03:43,  1.90s/it]
                                                     

 99%|█████████▉| 11882/12000 [45:58<03:43,  1.90s/it]
 99%|█████████▉| 11883/12000 [46:00<03:56,  2.02s/it]
                                                     

 99%|█████████▉| 11883/12000 [46:00<03:56,  2.02s/it]
 99%|█████████▉| 11884/12000 [46:02<03:34,  1.85s/it]
                                                     

 99%|█████████▉| 11884/12000 [46:02<03:34,  1.85s/it]
 99%|█████████▉| 11885/12000 [46:04<03:30,  1.83s/it]
                                                     

 99%|█████████▉| 11885/12000 [46:04<03:30,  1.83s/it]
 99%|█████████▉| 11886/12000 [46:06<03:39,  1.92s/it]
                                                     

 99%|█████████▉| 11886/12000 [46:06<03:39,  1.92s/it]
 99%|█████████▉| 11887/12000 [46:07<03:27,  1.84s/it]
                                                     

 99%|█████████▉| 11887/12000 [46:07<03:27,  1.84s/it]
 99%|█████████▉| 11888/12000 [46:09<03:21,  1.80s/it]
                                                     

 99%|█████████▉| 11888/12000 [46:09<03:21,  1.80s/it]
 99%|█████████▉| 11889/12000 [46:12<03:57,  2.14s/it]
                                                     

 99%|█████████▉| 11889/12000 [46:12<03:57,  2.14s/it]
 99%|█████████▉| 11890/12000 [46:15<04:08,  2.26s/it]
                                                     

 99%|█████████▉| 11890/12000 [46:15<04:08,  2.26s/it]
 99%|█████████▉| 11891/12000 [46:17<04:09,  2.29s/it]
                                                     

 99%|█████████▉| 11891/12000 [46:17<04:09,  2.29s/it]
 99%|█████████▉| 11892/12000 [46:20<04:15,  2.37s/it]
                                                     

 99%|█████████▉| 11892/12000 [46:20<04:15,  2.37s/it]
 99%|█████████▉| 11893/12000 [46:21<03:43,  2.09s/it]
                                                     

 99%|█████████▉| 11893/12000 [46:21<03:43,  2.09s/it]
 99%|█████████▉| 11894/12000 [46:23<03:33,  2.02s/it]
                                                     

 99%|█████████▉| 11894/12000 [46:23<03:33,  2.02s/it]
 99%|█████████▉| 11895/12000 [46:26<04:16,  2.44s/it]
                                                     

 99%|█████████▉| 11895/12000 [46:26<04:16,  2.44s/it]
 99%|█████████▉| 11896/12000 [46:28<03:45,  2.16s/it]
                                                     

 99%|█████████▉| 11896/12000 [46:28<03:45,  2.16s/it]
 99%|█████████▉| 11897/12000 [46:29<03:23,  1.97s/it]
                                                     

 99%|█████████▉| 11897/12000 [46:29<03:23,  1.97s/it]
 99%|█████████▉| 11898/12000 [46:31<03:08,  1.84s/it]
                                                     

 99%|█████████▉| 11898/12000 [46:31<03:08,  1.84s/it]
 99%|█████████▉| 11899/12000 [46:33<03:16,  1.94s/it]
                                                     

 99%|█████████▉| 11899/12000 [46:33<03:16,  1.94s/it]
 99%|█████████▉| 11900/12000 [46:35<03:09,  1.89s/it]
                                                     

 99%|█████████▉| 11900/12000 [46:35<03:09,  1.89s/it]
 99%|█████████▉| 11901/12000 [46:39<04:05,  2.48s/it]
                                                     

 99%|█████████▉| 11901/12000 [46:39<04:05,  2.48s/it]
 99%|█████████▉| 11902/12000 [46:40<03:30,  2.15s/it]
                                                     

 99%|█████████▉| 11902/12000 [46:40<03:30,  2.15s/it]
 99%|█████████▉| 11903/12000 [46:42<03:31,  2.18s/it]
                                                     

 99%|█████████▉| 11903/12000 [46:42<03:31,  2.18s/it]
 99%|█████████▉| 11904/12000 [46:44<03:13,  2.02s/it]
                                                     

 99%|█████████▉| 11904/12000 [46:44<03:13,  2.02s/it]
 99%|█████████▉| 11905/12000 [46:46<03:02,  1.92s/it]
                                                     

 99%|█████████▉| 11905/12000 [46:46<03:02,  1.92s/it]
 99%|█████████▉| 11906/12000 [46:47<02:45,  1.76s/it]
                                                     

 99%|█████████▉| 11906/12000 [46:47<02:45,  1.76s/it]
 99%|█████████▉| 11907/12000 [46:51<03:43,  2.40s/it]
                                                     

 99%|█████████▉| 11907/12000 [46:51<03:43,  2.40s/it]
 99%|█████████▉| 11908/12000 [46:53<03:25,  2.23s/it]
                                                     

 99%|█████████▉| 11908/12000 [46:53<03:25,  2.23s/it]
 99%|█████████▉| 11909/12000 [46:54<03:00,  1.99s/it]
                                                     

 99%|█████████▉| 11909/12000 [46:54<03:00,  1.99s/it]
 99%|█████████▉| 11910/12000 [46:56<02:45,  1.84s/it]
                                                     

 99%|█████████▉| 11910/12000 [46:56<02:45,  1.84s/it]
 99%|█████████▉| 11911/12000 [46:57<02:29,  1.69s/it]
                                                     

 99%|█████████▉| 11911/12000 [46:57<02:29,  1.69s/it]
 99%|█████████▉| 11912/12000 [46:59<02:23,  1.63s/it]
                                                     

 99%|█████████▉| 11912/12000 [46:59<02:23,  1.63s/it]
 99%|█████████▉| 11913/12000 [47:00<02:17,  1.58s/it]
                                                     

 99%|█████████▉| 11913/12000 [47:00<02:17,  1.58s/it]
 99%|█████████▉| 11914/12000 [47:02<02:30,  1.75s/it]
                                                     

 99%|█████████▉| 11914/12000 [47:02<02:30,  1.75s/it]
 99%|█████████▉| 11915/12000 [47:04<02:32,  1.80s/it]
                                                     

 99%|█████████▉| 11915/12000 [47:04<02:32,  1.80s/it]
 99%|█████████▉| 11916/12000 [47:06<02:41,  1.92s/it]
                                                     

 99%|█████████▉| 11916/12000 [47:06<02:41,  1.92s/it]
 99%|█████████▉| 11917/12000 [47:10<03:26,  2.49s/it]
                                                     
{'loss': '0.6225', 'grad_norm': '0.3839', 'learning_rate': '6.516e-08', 'entropy': '0.6363', 'num_tokens': '6.868e+06', 'mean_token_accuracy': '0.8325', 'epoch': '0.105'}
{'loss': '0.6682', 'grad_norm': '0.4255', 'learning_rate': '6.417e-08', 'entropy': '0.6951', 'num_tokens': '6.871e+06', 'mean_token_accuracy': '0.8205', 'epoch': '0.1051'}
{'loss': '0.7568', 'grad_norm': '0.4044', 'learning_rate': '6.318e-08', 'entropy': '0.8158', 'num_tokens': '6.876e+06', 'mean_token_accuracy': '0.7971', 'epoch': '0.1051'}
{'loss': '0.6668', 'grad_norm': '0.3836', 'learning_rate': '6.221e-08', 'entropy': '0.744', 'num_tokens': '6.88e+06', 'mean_token_accuracy': '0.8078', 'epoch': '0.1051'}
{'loss': '1.026', 'grad_norm': '0.4431', 'learning_rate': '6.124e-08', 'entropy': '0.9369', 'num_tokens': '6.886e+06', 'mean_token_accuracy': '0.7695', 'epoch': '0.1051'}
{'loss': '0.7691', 'grad_norm': '0.3825', 'learning_rate': '6.028e-08', 'entropy': '0.7851', 'num_tokens': '6.891e+06', 'mean_token_accuracy': '0.7886', 'epoch': '0.1051'}
{'loss': '0.748', 'grad_norm': '0.4778', 'learning_rate': '5.932e-08', 'entropy': '0.8128', 'num_tokens': '6.895e+06', 'mean_token_accuracy': '0.7946', 'epoch': '0.1051'}
{'loss': '0.7687', 'grad_norm': '0.4331', 'learning_rate': '5.838e-08', 'entropy': '0.8115', 'num_tokens': '6.899e+06', 'mean_token_accuracy': '0.7893', 'epoch': '0.1051'}
{'loss': '0.7497', 'grad_norm': '0.4205', 'learning_rate': '5.744e-08', 'entropy': '0.837', 'num_tokens': '6.904e+06', 'mean_token_accuracy': '0.7799', 'epoch': '0.1051'}
{'loss': '0.807', 'grad_norm': '0.4423', 'learning_rate': '5.651e-08', 'entropy': '0.9056', 'num_tokens': '6.909e+06', 'mean_token_accuracy': '0.7824', 'epoch': '0.1051'}
{'loss': '0.678', 'grad_norm': '0.4277', 'learning_rate': '5.559e-08', 'entropy': '0.7281', 'num_tokens': '6.915e+06', 'mean_token_accuracy': '0.8129', 'epoch': '0.1051'}
{'loss': '0.6502', 'grad_norm': '0.4337', 'learning_rate': '5.467e-08', 'entropy': '0.6896', 'num_tokens': '6.919e+06', 'mean_token_accuracy': '0.8122', 'epoch': '0.1051'}
{'loss': '0.8572', 'grad_norm': '0.4441', 'learning_rate': '5.377e-08', 'entropy': '0.7824', 'num_tokens': '6.925e+06', 'mean_token_accuracy': '0.8035', 'epoch': '0.1052'}
{'loss': '0.5009', 'grad_norm': '0.2807', 'learning_rate': '5.287e-08', 'entropy': '0.5364', 'num_tokens': '6.932e+06', 'mean_token_accuracy': '0.8564', 'epoch': '0.1052'}
{'loss': '0.9168', 'grad_norm': '0.4162', 'learning_rate': '5.197e-08', 'entropy': '0.8921', 'num_tokens': '6.936e+06', 'mean_token_accuracy': '0.7728', 'epoch': '0.1052'}
{'loss': '0.897', 'grad_norm': '0.3736', 'learning_rate': '5.109e-08', 'entropy': '0.8777', 'num_tokens': '6.941e+06', 'mean_token_accuracy': '0.763', 'epoch': '0.1052'}
{'loss': '1.061', 'grad_norm': '0.4785', 'learning_rate': '5.021e-08', 'entropy': '1.027', 'num_tokens': '6.946e+06', 'mean_token_accuracy': '0.7426', 'epoch': '0.1052'}
{'loss': '0.7532', 'grad_norm': '0.5485', 'learning_rate': '4.934e-08', 'entropy': '0.7702', 'num_tokens': '6.949e+06', 'mean_token_accuracy': '0.8024', 'epoch': '0.1052'}
{'loss': '0.7835', 'grad_norm': '0.354', 'learning_rate': '4.848e-08', 'entropy': '0.7864', 'num_tokens': '6.955e+06', 'mean_token_accuracy': '0.8079', 'epoch': '0.1052'}
{'loss': '0.7148', 'grad_norm': '0.3523', 'learning_rate': '4.763e-08', 'entropy': '0.7151', 'num_tokens': '6.963e+06', 'mean_token_accuracy': '0.8095', 'epoch': '0.1052'}
{'loss': '0.84', 'grad_norm': '0.3603', 'learning_rate': '4.678e-08', 'entropy': '0.8641', 'num_tokens': '6.969e+06', 'mean_token_accuracy': '0.7846', 'epoch': '0.1052'}
{'loss': '0.7453', 'grad_norm': '0.4122', 'learning_rate': '4.594e-08', 'entropy': '0.7087', 'num_tokens': '6.975e+06', 'mean_token_accuracy': '0.8027', 'epoch': '0.1052'}
{'loss': '1.208', 'grad_norm': '0.4956', 'learning_rate': '4.511e-08', 'entropy': '1.085', 'num_tokens': '6.981e+06', 'mean_token_accuracy': '0.7374', 'epoch': '0.1052'}
{'loss': '0.7613', 'grad_norm': '0.4445', 'learning_rate': '4.429e-08', 'entropy': '0.8099', 'num_tokens': '6.986e+06', 'mean_token_accuracy': '0.7948', 'epoch': '0.1053'}
{'loss': '0.7149', 'grad_norm': '0.4015', 'learning_rate': '4.347e-08', 'entropy': '0.767', 'num_tokens': '6.992e+06', 'mean_token_accuracy': '0.8057', 'epoch': '0.1053'}
{'loss': '1.031', 'grad_norm': '0.4358', 'learning_rate': '4.266e-08', 'entropy': '1.037', 'num_tokens': '6.996e+06', 'mean_token_accuracy': '0.7525', 'epoch': '0.1053'}
{'loss': '0.5807', 'grad_norm': '0.3225', 'learning_rate': '4.186e-08', 'entropy': '0.6666', 'num_tokens': '7.001e+06', 'mean_token_accuracy': '0.8259', 'epoch': '0.1053'}
{'loss': '0.9257', 'grad_norm': '0.5835', 'learning_rate': '4.107e-08', 'entropy': '0.8627', 'num_tokens': '7.004e+06', 'mean_token_accuracy': '0.776', 'epoch': '0.1053'}
{'loss': '0.8259', 'grad_norm': '0.4757', 'learning_rate': '4.028e-08', 'entropy': '0.9445', 'num_tokens': '7.009e+06', 'mean_token_accuracy': '0.775', 'epoch': '0.1053'}
{'loss': '0.828', 'grad_norm': '0.4263', 'learning_rate': '3.95e-08', 'entropy': '0.8208', 'num_tokens': '7.015e+06', 'mean_token_accuracy': '0.7909', 'epoch': '0.1053'}
{'loss': '0.6388', 'grad_norm': '0.3724', 'learning_rate': '3.873e-08', 'entropy': '0.5978', 'num_tokens': '7.02e+06', 'mean_token_accuracy': '0.8428', 'epoch': '0.1053'}
{'loss': '1.164', 'grad_norm': '0.4768', 'learning_rate': '3.797e-08', 'entropy': '1.122', 'num_tokens': '7.026e+06', 'mean_token_accuracy': '0.7328', 'epoch': '0.1053'}
{'loss': '0.5287', 'grad_norm': '0.2852', 'learning_rate': '3.721e-08', 'entropy': '0.5529', 'num_tokens': '7.03e+06', 'mean_token_accuracy': '0.8544', 'epoch': '0.1053'}
{'loss': '0.5822', 'grad_norm': '0.3741', 'learning_rate': '3.647e-08', 'entropy': '0.6045', 'num_tokens': '7.036e+06', 'mean_token_accuracy': '0.8337', 'epoch': '0.1053'}
{'loss': '0.6049', 'grad_norm': '0.3149', 'learning_rate': '3.573e-08', 'entropy': '0.6245', 'num_tokens': '7.041e+06', 'mean_token_accuracy': '0.8309', 'epoch': '0.1053'}
{'loss': '1.048', 'grad_norm': '0.4537', 'learning_rate': '3.499e-08', 'entropy': '1.066', 'num_tokens': '7.045e+06', 'mean_token_accuracy': '0.7557', 'epoch': '0.1054'}
{'loss': '0.7108', 'grad_norm': '0.4445', 'learning_rate': '3.427e-08', 'entropy': '0.7297', 'num_tokens': '7.048e+06', 'mean_token_accuracy': '0.8097', 'epoch': '0.1054'}
{'loss': '1.165', 'grad_norm': '0.5007', 'learning_rate': '3.355e-08', 'entropy': '1.098', 'num_tokens': '7.057e+06', 'mean_token_accuracy': '0.7299', 'epoch': '0.1054'}
{'loss': '0.605', 'grad_norm': '0.3462', 'learning_rate': '3.284e-08', 'entropy': '0.5914', 'num_tokens': '7.062e+06', 'mean_token_accuracy': '0.8356', 'epoch': '0.1054'}
{'loss': '0.8197', 'grad_norm': '0.4378', 'learning_rate': '3.214e-08', 'entropy': '0.8248', 'num_tokens': '7.066e+06', 'mean_token_accuracy': '0.7852', 'epoch': '0.1054'}
{'loss': '1.026', 'grad_norm': '0.4564', 'learning_rate': '3.144e-08', 'entropy': '0.9667', 'num_tokens': '7.069e+06', 'mean_token_accuracy': '0.7666', 'epoch': '0.1054'}
{'loss': '0.9308', 'grad_norm': '0.4198', 'learning_rate': '3.076e-08', 'entropy': '0.9309', 'num_tokens': '7.073e+06', 'mean_token_accuracy': '0.7548', 'epoch': '0.1054'}
{'loss': '0.7323', 'grad_norm': '0.488', 'learning_rate': '3.008e-08', 'entropy': '0.8457', 'num_tokens': '7.077e+06', 'mean_token_accuracy': '0.7719', 'epoch': '0.1054'}
{'loss': '0.5984', 'grad_norm': '0.4034', 'learning_rate': '2.94e-08', 'entropy': '0.5931', 'num_tokens': '7.08e+06', 'mean_token_accuracy': '0.8483', 'epoch': '0.1054'}
{'loss': '1.198', 'grad_norm': '0.4797', 'learning_rate': '2.874e-08', 'entropy': '1.11', 'num_tokens': '7.087e+06', 'mean_token_accuracy': '0.7426', 'epoch': '0.1054'}
{'loss': '0.934', 'grad_norm': '0.4964', 'learning_rate': '2.808e-08', 'entropy': '0.9634', 'num_tokens': '7.092e+06', 'mean_token_accuracy': '0.7601', 'epoch': '0.1054'}
{'loss': '0.9592', 'grad_norm': '0.4028', 'learning_rate': '2.743e-08', 'entropy': '0.9927', 'num_tokens': '7.099e+06', 'mean_token_accuracy': '0.7515', 'epoch': '0.1055'}

 99%|█████████▉| 11917/12000 [47:10<03:26,  2.49s/it]
 99%|█████████▉| 11918/12000 [47:12<03:09,  2.31s/it]
                                                     

 99%|█████████▉| 11918/12000 [47:12<03:09,  2.31s/it]
 99%|█████████▉| 11919/12000 [47:14<03:11,  2.37s/it]
                                                     

 99%|█████████▉| 11919/12000 [47:14<03:11,  2.37s/it]
 99%|█████████▉| 11920/12000 [47:16<02:48,  2.11s/it]
                                                     

 99%|█████████▉| 11920/12000 [47:16<02:48,  2.11s/it]
 99%|█████████▉| 11921/12000 [47:18<02:35,  1.97s/it]
                                                     

 99%|█████████▉| 11921/12000 [47:18<02:35,  1.97s/it]
 99%|█████████▉| 11922/12000 [47:19<02:27,  1.89s/it]
                                                     

 99%|█████████▉| 11922/12000 [47:19<02:27,  1.89s/it]
 99%|█████████▉| 11923/12000 [47:21<02:28,  1.93s/it]
                                                     

 99%|█████████▉| 11923/12000 [47:21<02:28,  1.93s/it]
 99%|█████████▉| 11924/12000 [47:23<02:26,  1.92s/it]
                                                     

 99%|█████████▉| 11924/12000 [47:23<02:26,  1.92s/it]
 99%|█████████▉| 11925/12000 [47:25<02:20,  1.87s/it]
                                                     

 99%|█████████▉| 11925/12000 [47:25<02:20,  1.87s/it]
 99%|█████████▉| 11926/12000 [47:27<02:24,  1.95s/it]
                                                     

 99%|█████████▉| 11926/12000 [47:27<02:24,  1.95s/it]
 99%|█████████▉| 11927/12000 [47:29<02:21,  1.94s/it]
                                                     

 99%|█████████▉| 11927/12000 [47:29<02:21,  1.94s/it]
 99%|█████████▉| 11928/12000 [47:31<02:13,  1.85s/it]
                                                     

 99%|█████████▉| 11928/12000 [47:31<02:13,  1.85s/it]
 99%|█████████▉| 11929/12000 [47:33<02:13,  1.88s/it]
                                                     

 99%|█████████▉| 11929/12000 [47:33<02:13,  1.88s/it]
 99%|█████████▉| 11930/12000 [47:35<02:23,  2.05s/it]
                                                     

 99%|█████████▉| 11930/12000 [47:35<02:23,  2.05s/it]
 99%|█████████▉| 11931/12000 [47:37<02:09,  1.87s/it]
                                                     

 99%|█████████▉| 11931/12000 [47:37<02:09,  1.87s/it]
 99%|█████████▉| 11932/12000 [47:39<02:21,  2.07s/it]
                                                     

 99%|█████████▉| 11932/12000 [47:39<02:21,  2.07s/it]
 99%|█████████▉| 11933/12000 [47:41<02:12,  1.98s/it]
                                                     

 99%|█████████▉| 11933/12000 [47:41<02:12,  1.98s/it]
 99%|█████████▉| 11934/12000 [47:45<02:53,  2.64s/it]
                                                     

 99%|█████████▉| 11934/12000 [47:45<02:53,  2.64s/it]
 99%|█████████▉| 11935/12000 [47:47<02:31,  2.32s/it]
                                                     

 99%|█████████▉| 11935/12000 [47:47<02:31,  2.32s/it]
 99%|█████████▉| 11936/12000 [47:48<02:19,  2.17s/it]
                                                     

 99%|█████████▉| 11936/12000 [47:48<02:19,  2.17s/it]
 99%|█████████▉| 11937/12000 [47:50<02:09,  2.05s/it]
                                                     

 99%|█████████▉| 11937/12000 [47:50<02:09,  2.05s/it]
 99%|█████████▉| 11938/12000 [47:52<01:57,  1.90s/it]
                                                     

 99%|█████████▉| 11938/12000 [47:52<01:57,  1.90s/it]
 99%|█████████▉| 11939/12000 [47:56<02:44,  2.70s/it]
                                                     

 99%|█████████▉| 11939/12000 [47:56<02:44,  2.70s/it]
100%|█████████▉| 11940/12000 [47:58<02:25,  2.42s/it]
                                                     

100%|█████████▉| 11940/12000 [47:58<02:25,  2.42s/it]
100%|█████████▉| 11941/12000 [48:00<02:09,  2.20s/it]
                                                     

100%|█████████▉| 11941/12000 [48:00<02:09,  2.20s/it]
100%|█████████▉| 11942/12000 [48:01<01:55,  1.99s/it]
                                                     

100%|█████████▉| 11942/12000 [48:01<01:55,  1.99s/it]
100%|█████████▉| 11943/12000 [48:03<01:45,  1.86s/it]
                                                     

100%|█████████▉| 11943/12000 [48:03<01:45,  1.86s/it]
100%|█████████▉| 11944/12000 [48:04<01:38,  1.75s/it]
                                                     

100%|█████████▉| 11944/12000 [48:04<01:38,  1.75s/it]
100%|█████████▉| 11945/12000 [48:07<01:53,  2.06s/it]
                                                     

100%|█████████▉| 11945/12000 [48:07<01:53,  2.06s/it]
100%|█████████▉| 11946/12000 [48:09<01:55,  2.14s/it]
                                                     

100%|█████████▉| 11946/12000 [48:09<01:55,  2.14s/it]
100%|█████████▉| 11947/12000 [48:11<01:47,  2.03s/it]
                                                     

100%|█████████▉| 11947/12000 [48:11<01:47,  2.03s/it]
100%|█████████▉| 11948/12000 [48:13<01:41,  1.95s/it]
                                                     

100%|█████████▉| 11948/12000 [48:13<01:41,  1.95s/it]
100%|█████████▉| 11949/12000 [48:15<01:35,  1.86s/it]
                                                     

100%|█████████▉| 11949/12000 [48:15<01:35,  1.86s/it]
100%|█████████▉| 11950/12000 [48:16<01:28,  1.77s/it]
                                                     

100%|█████████▉| 11950/12000 [48:16<01:28,  1.77s/it]
100%|█████████▉| 11951/12000 [48:20<01:55,  2.36s/it]
                                                     

100%|█████████▉| 11951/12000 [48:20<01:55,  2.36s/it]
100%|█████████▉| 11952/12000 [48:22<01:45,  2.19s/it]
                                                     

100%|█████████▉| 11952/12000 [48:22<01:45,  2.19s/it]
100%|█████████▉| 11953/12000 [48:24<01:48,  2.30s/it]
                                                     

100%|█████████▉| 11953/12000 [48:24<01:48,  2.30s/it]
100%|█████████▉| 11954/12000 [48:26<01:38,  2.13s/it]
                                                     

100%|█████████▉| 11954/12000 [48:26<01:38,  2.13s/it]
100%|█████████▉| 11955/12000 [48:28<01:32,  2.05s/it]
                                                     

100%|█████████▉| 11955/12000 [48:28<01:32,  2.05s/it]
100%|█████████▉| 11956/12000 [48:29<01:22,  1.87s/it]
                                                     

100%|█████████▉| 11956/12000 [48:29<01:22,  1.87s/it]
100%|█████████▉| 11957/12000 [48:32<01:35,  2.22s/it]
                                                     

100%|█████████▉| 11957/12000 [48:32<01:35,  2.22s/it]
100%|█████████▉| 11958/12000 [48:35<01:34,  2.25s/it]
                                                     

100%|█████████▉| 11958/12000 [48:35<01:34,  2.25s/it]
100%|█████████▉| 11959/12000 [48:36<01:23,  2.04s/it]
                                                     

100%|█████████▉| 11959/12000 [48:36<01:23,  2.04s/it]
100%|█████████▉| 11960/12000 [48:38<01:18,  1.95s/it]
                                                     

100%|█████████▉| 11960/12000 [48:38<01:18,  1.95s/it]
100%|█████████▉| 11961/12000 [48:42<01:36,  2.47s/it]
                                                     

100%|█████████▉| 11961/12000 [48:42<01:36,  2.47s/it]
100%|█████████▉| 11962/12000 [48:44<01:30,  2.39s/it]
                                                     

100%|█████████▉| 11962/12000 [48:44<01:30,  2.39s/it]
100%|█████████▉| 11963/12000 [48:49<01:56,  3.16s/it]
                                                     

100%|█████████▉| 11963/12000 [48:49<01:56,  3.16s/it]
100%|█████████▉| 11964/12000 [48:51<01:39,  2.76s/it]
                                                     
{'loss': '0.7196', 'grad_norm': '0.423', 'learning_rate': '2.679e-08', 'entropy': '0.7248', 'num_tokens': '7.104e+06', 'mean_token_accuracy': '0.8111', 'epoch': '0.1055'}
{'loss': '1.202', 'grad_norm': '0.4831', 'learning_rate': '2.616e-08', 'entropy': '1.21', 'num_tokens': '7.109e+06', 'mean_token_accuracy': '0.7143', 'epoch': '0.1055'}
{'loss': '0.7156', 'grad_norm': '0.389', 'learning_rate': '2.553e-08', 'entropy': '0.8115', 'num_tokens': '7.114e+06', 'mean_token_accuracy': '0.7902', 'epoch': '0.1055'}
{'loss': '0.7143', 'grad_norm': '0.3939', 'learning_rate': '2.491e-08', 'entropy': '0.7189', 'num_tokens': '7.119e+06', 'mean_token_accuracy': '0.8128', 'epoch': '0.1055'}
{'loss': '0.6957', 'grad_norm': '0.3914', 'learning_rate': '2.43e-08', 'entropy': '0.764', 'num_tokens': '7.124e+06', 'mean_token_accuracy': '0.8064', 'epoch': '0.1055'}
{'loss': '0.6602', 'grad_norm': '0.3089', 'learning_rate': '2.37e-08', 'entropy': '0.6974', 'num_tokens': '7.129e+06', 'mean_token_accuracy': '0.8224', 'epoch': '0.1055'}
{'loss': '0.7092', 'grad_norm': '0.414', 'learning_rate': '2.31e-08', 'entropy': '0.7647', 'num_tokens': '7.135e+06', 'mean_token_accuracy': '0.8058', 'epoch': '0.1055'}
{'loss': '1.186', 'grad_norm': '0.4254', 'learning_rate': '2.251e-08', 'entropy': '1.049', 'num_tokens': '7.14e+06', 'mean_token_accuracy': '0.7365', 'epoch': '0.1055'}
{'loss': '0.62', 'grad_norm': '0.4499', 'learning_rate': '2.193e-08', 'entropy': '0.6087', 'num_tokens': '7.144e+06', 'mean_token_accuracy': '0.8314', 'epoch': '0.1055'}
{'loss': '1.18', 'grad_norm': '0.4757', 'learning_rate': '2.136e-08', 'entropy': '1.17', 'num_tokens': '7.15e+06', 'mean_token_accuracy': '0.698', 'epoch': '0.1055'}
{'loss': '0.624', 'grad_norm': '0.4282', 'learning_rate': '2.079e-08', 'entropy': '0.6399', 'num_tokens': '7.155e+06', 'mean_token_accuracy': '0.8263', 'epoch': '0.1056'}
{'loss': '1.072', 'grad_norm': '0.4219', 'learning_rate': '2.023e-08', 'entropy': '1.151', 'num_tokens': '7.159e+06', 'mean_token_accuracy': '0.7263', 'epoch': '0.1056'}
{'loss': '0.7157', 'grad_norm': '0.3482', 'learning_rate': '1.968e-08', 'entropy': '0.7064', 'num_tokens': '7.165e+06', 'mean_token_accuracy': '0.819', 'epoch': '0.1056'}
{'loss': '0.7774', 'grad_norm': '0.3484', 'learning_rate': '1.914e-08', 'entropy': '0.7307', 'num_tokens': '7.17e+06', 'mean_token_accuracy': '0.8212', 'epoch': '0.1056'}
{'loss': '0.8516', 'grad_norm': '0.4529', 'learning_rate': '1.861e-08', 'entropy': '0.834', 'num_tokens': '7.174e+06', 'mean_token_accuracy': '0.781', 'epoch': '0.1056'}
{'loss': '0.8116', 'grad_norm': '0.4272', 'learning_rate': '1.808e-08', 'entropy': '0.7837', 'num_tokens': '7.181e+06', 'mean_token_accuracy': '0.7867', 'epoch': '0.1056'}
{'loss': '0.6141', 'grad_norm': '0.4087', 'learning_rate': '1.756e-08', 'entropy': '0.6026', 'num_tokens': '7.186e+06', 'mean_token_accuracy': '0.845', 'epoch': '0.1056'}
{'loss': '0.6998', 'grad_norm': '0.2507', 'learning_rate': '1.705e-08', 'entropy': '0.7185', 'num_tokens': '7.198e+06', 'mean_token_accuracy': '0.8116', 'epoch': '0.1056'}
{'loss': '0.7704', 'grad_norm': '0.4724', 'learning_rate': '1.654e-08', 'entropy': '0.779', 'num_tokens': '7.202e+06', 'mean_token_accuracy': '0.8031', 'epoch': '0.1056'}
{'loss': '0.7218', 'grad_norm': '0.3864', 'learning_rate': '1.604e-08', 'entropy': '0.7933', 'num_tokens': '7.206e+06', 'mean_token_accuracy': '0.7915', 'epoch': '0.1056'}
{'loss': '0.7073', 'grad_norm': '0.3784', 'learning_rate': '1.555e-08', 'entropy': '0.7486', 'num_tokens': '7.212e+06', 'mean_token_accuracy': '0.8127', 'epoch': '0.1056'}
{'loss': '0.7384', 'grad_norm': '0.3462', 'learning_rate': '1.507e-08', 'entropy': '0.7244', 'num_tokens': '7.216e+06', 'mean_token_accuracy': '0.8027', 'epoch': '0.1056'}
{'loss': '0.551', 'grad_norm': '0.3762', 'learning_rate': '1.46e-08', 'entropy': '0.5961', 'num_tokens': '7.223e+06', 'mean_token_accuracy': '0.8457', 'epoch': '0.1057'}
{'loss': '0.6655', 'grad_norm': '0.3491', 'learning_rate': '1.413e-08', 'entropy': '0.6675', 'num_tokens': '7.229e+06', 'mean_token_accuracy': '0.8196', 'epoch': '0.1057'}
{'loss': '0.9899', 'grad_norm': '0.4817', 'learning_rate': '1.367e-08', 'entropy': '0.933', 'num_tokens': '7.234e+06', 'mean_token_accuracy': '0.7632', 'epoch': '0.1057'}
{'loss': '1.004', 'grad_norm': '0.4065', 'learning_rate': '1.322e-08', 'entropy': '1.04', 'num_tokens': '7.239e+06', 'mean_token_accuracy': '0.7533', 'epoch': '0.1057'}
{'loss': '0.7233', 'grad_norm': '0.3977', 'learning_rate': '1.277e-08', 'entropy': '0.811', 'num_tokens': '7.244e+06', 'mean_token_accuracy': '0.7984', 'epoch': '0.1057'}
{'loss': '1.091', 'grad_norm': '0.5175', 'learning_rate': '1.234e-08', 'entropy': '1.028', 'num_tokens': '7.248e+06', 'mean_token_accuracy': '0.7558', 'epoch': '0.1057'}
{'loss': '0.7495', 'grad_norm': '0.3813', 'learning_rate': '1.191e-08', 'entropy': '0.7827', 'num_tokens': '7.255e+06', 'mean_token_accuracy': '0.8016', 'epoch': '0.1057'}
{'loss': '0.8957', 'grad_norm': '1.521', 'learning_rate': '1.149e-08', 'entropy': '0.9683', 'num_tokens': '7.26e+06', 'mean_token_accuracy': '0.7665', 'epoch': '0.1057'}
{'loss': '0.8261', 'grad_norm': '0.4259', 'learning_rate': '1.107e-08', 'entropy': '0.8219', 'num_tokens': '7.265e+06', 'mean_token_accuracy': '0.7845', 'epoch': '0.1057'}
{'loss': '1.253', 'grad_norm': '0.4754', 'learning_rate': '1.067e-08', 'entropy': '1.056', 'num_tokens': '7.27e+06', 'mean_token_accuracy': '0.7342', 'epoch': '0.1057'}
{'loss': '0.9779', 'grad_norm': '0.4324', 'learning_rate': '1.027e-08', 'entropy': '1.013', 'num_tokens': '7.275e+06', 'mean_token_accuracy': '0.7595', 'epoch': '0.1057'}
{'loss': '0.66', 'grad_norm': '0.4175', 'learning_rate': '9.876e-09', 'entropy': '0.6383', 'num_tokens': '7.28e+06', 'mean_token_accuracy': '0.8281', 'epoch': '0.1058'}
{'loss': '0.7299', 'grad_norm': '0.3854', 'learning_rate': '9.493e-09', 'entropy': '0.7402', 'num_tokens': '7.286e+06', 'mean_token_accuracy': '0.809', 'epoch': '0.1058'}
{'loss': '0.7503', 'grad_norm': '0.437', 'learning_rate': '9.117e-09', 'entropy': '0.7697', 'num_tokens': '7.29e+06', 'mean_token_accuracy': '0.807', 'epoch': '0.1058'}
{'loss': '0.9893', 'grad_norm': '0.3571', 'learning_rate': '8.749e-09', 'entropy': '0.9043', 'num_tokens': '7.298e+06', 'mean_token_accuracy': '0.7763', 'epoch': '0.1058'}
{'loss': '0.8648', 'grad_norm': '0.4143', 'learning_rate': '8.388e-09', 'entropy': '0.7823', 'num_tokens': '7.303e+06', 'mean_token_accuracy': '0.8053', 'epoch': '0.1058'}
{'loss': '0.6231', 'grad_norm': '0.4256', 'learning_rate': '8.035e-09', 'entropy': '0.6144', 'num_tokens': '7.309e+06', 'mean_token_accuracy': '0.8295', 'epoch': '0.1058'}
{'loss': '0.8132', 'grad_norm': '0.4282', 'learning_rate': '7.689e-09', 'entropy': '0.9207', 'num_tokens': '7.313e+06', 'mean_token_accuracy': '0.771', 'epoch': '0.1058'}
{'loss': '0.8245', 'grad_norm': '0.397', 'learning_rate': '7.351e-09', 'entropy': '0.7845', 'num_tokens': '7.321e+06', 'mean_token_accuracy': '0.8083', 'epoch': '0.1058'}
{'loss': '0.9318', 'grad_norm': '0.4228', 'learning_rate': '7.021e-09', 'entropy': '1.062', 'num_tokens': '7.327e+06', 'mean_token_accuracy': '0.7532', 'epoch': '0.1058'}
{'loss': '0.9092', 'grad_norm': '0.4558', 'learning_rate': '6.698e-09', 'entropy': '0.987', 'num_tokens': '7.33e+06', 'mean_token_accuracy': '0.766', 'epoch': '0.1058'}
{'loss': '0.6723', 'grad_norm': '0.3998', 'learning_rate': '6.383e-09', 'entropy': '0.662', 'num_tokens': '7.336e+06', 'mean_token_accuracy': '0.8263', 'epoch': '0.1058'}
{'loss': '0.7074', 'grad_norm': '0.3788', 'learning_rate': '6.075e-09', 'entropy': '0.7164', 'num_tokens': '7.341e+06', 'mean_token_accuracy': '0.8174', 'epoch': '0.1059'}
{'loss': '0.9177', 'grad_norm': '0.399', 'learning_rate': '5.775e-09', 'entropy': '0.9322', 'num_tokens': '7.347e+06', 'mean_token_accuracy': '0.7715', 'epoch': '0.1059'}
{'loss': '0.8436', 'grad_norm': '0.3668', 'learning_rate': '5.483e-09', 'entropy': '0.8416', 'num_tokens': '7.357e+06', 'mean_token_accuracy': '0.7968', 'epoch': '0.1059'}

100%|█████████▉| 11964/12000 [48:51<01:39,  2.76s/it]
100%|█████████▉| 11965/12000 [48:53<01:27,  2.50s/it]
                                                     

100%|█████████▉| 11965/12000 [48:53<01:27,  2.50s/it]
100%|█████████▉| 11966/12000 [48:54<01:16,  2.24s/it]
                                                     

100%|█████████▉| 11966/12000 [48:54<01:16,  2.24s/it]
100%|█████████▉| 11967/12000 [48:56<01:05,  1.99s/it]
                                                     

100%|█████████▉| 11967/12000 [48:56<01:05,  1.99s/it]
100%|█████████▉| 11968/12000 [48:57<00:58,  1.83s/it]
                                                     

100%|█████████▉| 11968/12000 [48:57<00:58,  1.83s/it]
100%|█████████▉| 11969/12000 [48:59<00:58,  1.87s/it]
                                                     

100%|█████████▉| 11969/12000 [48:59<00:58,  1.87s/it]
100%|█████████▉| 11970/12000 [49:01<00:54,  1.82s/it]
                                                     

100%|█████████▉| 11970/12000 [49:01<00:54,  1.82s/it]
100%|█████████▉| 11971/12000 [49:02<00:51,  1.76s/it]
                                                     

100%|█████████▉| 11971/12000 [49:02<00:51,  1.76s/it]
100%|█████████▉| 11972/12000 [49:04<00:45,  1.64s/it]
                                                     

100%|█████████▉| 11972/12000 [49:04<00:45,  1.64s/it]
100%|█████████▉| 11973/12000 [49:06<00:46,  1.71s/it]
                                                     

100%|█████████▉| 11973/12000 [49:06<00:46,  1.71s/it]
100%|█████████▉| 11974/12000 [49:07<00:42,  1.64s/it]
                                                     

100%|█████████▉| 11974/12000 [49:07<00:42,  1.64s/it]
100%|█████████▉| 11975/12000 [49:09<00:39,  1.60s/it]
                                                     

100%|█████████▉| 11975/12000 [49:09<00:39,  1.60s/it]
100%|█████████▉| 11976/12000 [49:11<00:41,  1.74s/it]
                                                     

100%|█████████▉| 11976/12000 [49:11<00:41,  1.74s/it]
100%|█████████▉| 11977/12000 [49:12<00:38,  1.68s/it]
                                                     

100%|█████████▉| 11977/12000 [49:12<00:38,  1.68s/it]
100%|█████████▉| 11978/12000 [49:14<00:35,  1.60s/it]
                                                     

100%|█████████▉| 11978/12000 [49:14<00:35,  1.60s/it]
100%|█████████▉| 11979/12000 [49:15<00:34,  1.65s/it]
                                                     

100%|█████████▉| 11979/12000 [49:15<00:34,  1.65s/it]
100%|█████████▉| 11980/12000 [49:18<00:37,  1.89s/it]
                                                     

100%|█████████▉| 11980/12000 [49:18<00:37,  1.89s/it]
100%|█████████▉| 11981/12000 [49:19<00:33,  1.78s/it]
                                                     

100%|█████████▉| 11981/12000 [49:19<00:33,  1.78s/it]
100%|█████████▉| 11982/12000 [49:21<00:31,  1.77s/it]
                                                     

100%|█████████▉| 11982/12000 [49:21<00:31,  1.77s/it]
100%|█████████▉| 11983/12000 [49:25<00:39,  2.30s/it]
                                                     

100%|█████████▉| 11983/12000 [49:25<00:39,  2.30s/it]
100%|█████████▉| 11984/12000 [49:26<00:32,  2.03s/it]
                                                     

100%|█████████▉| 11984/12000 [49:26<00:32,  2.03s/it]
100%|█████████▉| 11985/12000 [49:28<00:28,  1.93s/it]
                                                     

100%|█████████▉| 11985/12000 [49:28<00:28,  1.93s/it]
100%|█████████▉| 11986/12000 [49:29<00:24,  1.77s/it]
                                                     

100%|█████████▉| 11986/12000 [49:29<00:24,  1.77s/it]
100%|█████████▉| 11987/12000 [49:32<00:26,  2.08s/it]
                                                     

100%|█████████▉| 11987/12000 [49:32<00:26,  2.08s/it]
100%|█████████▉| 11988/12000 [49:33<00:23,  1.92s/it]
                                                     

100%|█████████▉| 11988/12000 [49:33<00:23,  1.92s/it]
100%|█████████▉| 11989/12000 [49:35<00:19,  1.78s/it]
                                                     

100%|█████████▉| 11989/12000 [49:35<00:19,  1.78s/it]
100%|█████████▉| 11990/12000 [49:37<00:17,  1.79s/it]
                                                     

100%|█████████▉| 11990/12000 [49:37<00:17,  1.79s/it]
100%|█████████▉| 11991/12000 [49:38<00:15,  1.71s/it]
                                                     

100%|█████████▉| 11991/12000 [49:38<00:15,  1.71s/it]
100%|█████████▉| 11992/12000 [49:41<00:17,  2.14s/it]
                                                     

100%|█████████▉| 11992/12000 [49:41<00:17,  2.14s/it]
100%|█████████▉| 11993/12000 [49:43<00:14,  2.01s/it]
                                                     

100%|█████████▉| 11993/12000 [49:43<00:14,  2.01s/it]
100%|█████████▉| 11994/12000 [49:45<00:11,  1.92s/it]
                                                     

100%|█████████▉| 11994/12000 [49:45<00:11,  1.92s/it]
100%|█████████▉| 11995/12000 [49:46<00:09,  1.81s/it]
                                                     

100%|█████████▉| 11995/12000 [49:46<00:09,  1.81s/it]
100%|█████████▉| 11996/12000 [49:48<00:07,  1.75s/it]
                                                     

100%|█████████▉| 11996/12000 [49:48<00:07,  1.75s/it]
100%|█████████▉| 11997/12000 [49:49<00:04,  1.64s/it]
                                                     

100%|█████████▉| 11997/12000 [49:49<00:04,  1.64s/it]
100%|█████████▉| 11998/12000 [49:51<00:03,  1.70s/it]
                                                     

100%|█████████▉| 11998/12000 [49:51<00:03,  1.70s/it]
100%|█████████▉| 11999/12000 [49:56<00:02,  2.57s/it]
                                                     

100%|█████████▉| 11999/12000 [49:56<00:02,  2.57s/it]
100%|██████████| 12000/12000 [49:57<00:00,  2.28s/it]
                                                     

100%|██████████| 12000/12000 [49:57<00:00,  2.28s/it]
                                                     

100%|██████████| 12000/12000 [49:59<00:00,  2.28s/it]
100%|██████████| 12000/12000 [49:59<00:00,  4.00it/s]
{'loss': '1.122', 'grad_norm': '0.4178', 'learning_rate': '5.198e-09', 'entropy': '1.078', 'num_tokens': '7.362e+06', 'mean_token_accuracy': '0.7367', 'epoch': '0.1059'}
{'loss': '0.7238', 'grad_norm': '0.4233', 'learning_rate': '4.921e-09', 'entropy': '0.7284', 'num_tokens': '7.368e+06', 'mean_token_accuracy': '0.8022', 'epoch': '0.1059'}
{'loss': '1.04', 'grad_norm': '0.5073', 'learning_rate': '4.651e-09', 'entropy': '1.081', 'num_tokens': '7.373e+06', 'mean_token_accuracy': '0.745', 'epoch': '0.1059'}
{'loss': '0.6389', 'grad_norm': '0.3722', 'learning_rate': '4.389e-09', 'entropy': '0.6638', 'num_tokens': '7.378e+06', 'mean_token_accuracy': '0.8243', 'epoch': '0.1059'}
{'loss': '0.8131', 'grad_norm': '0.3757', 'learning_rate': '4.135e-09', 'entropy': '0.8447', 'num_tokens': '7.381e+06', 'mean_token_accuracy': '0.7889', 'epoch': '0.1059'}
{'loss': '1.417', 'grad_norm': '0.4644', 'learning_rate': '3.888e-09', 'entropy': '1.23', 'num_tokens': '7.386e+06', 'mean_token_accuracy': '0.7048', 'epoch': '0.1059'}
{'loss': '0.7922', 'grad_norm': '0.3269', 'learning_rate': '3.649e-09', 'entropy': '0.7186', 'num_tokens': '7.392e+06', 'mean_token_accuracy': '0.8212', 'epoch': '0.1059'}
{'loss': '0.6766', 'grad_norm': '0.416', 'learning_rate': '3.417e-09', 'entropy': '0.7292', 'num_tokens': '7.396e+06', 'mean_token_accuracy': '0.8156', 'epoch': '0.1059'}
{'loss': '0.8696', 'grad_norm': '0.4374', 'learning_rate': '3.193e-09', 'entropy': '0.8664', 'num_tokens': '7.4e+06', 'mean_token_accuracy': '0.7873', 'epoch': '0.106'}
{'loss': '0.8788', 'grad_norm': '0.4797', 'learning_rate': '2.977e-09', 'entropy': '0.8526', 'num_tokens': '7.406e+06', 'mean_token_accuracy': '0.8009', 'epoch': '0.106'}
{'loss': '0.563', 'grad_norm': '0.3232', 'learning_rate': '2.768e-09', 'entropy': '0.5557', 'num_tokens': '7.41e+06', 'mean_token_accuracy': '0.8594', 'epoch': '0.106'}
{'loss': '0.7866', 'grad_norm': '0.4075', 'learning_rate': '2.567e-09', 'entropy': '0.8105', 'num_tokens': '7.415e+06', 'mean_token_accuracy': '0.7977', 'epoch': '0.106'}
{'loss': '0.8776', 'grad_norm': '0.4069', 'learning_rate': '2.373e-09', 'entropy': '0.8964', 'num_tokens': '7.42e+06', 'mean_token_accuracy': '0.7754', 'epoch': '0.106'}
{'loss': '0.5777', 'grad_norm': '0.2903', 'learning_rate': '2.187e-09', 'entropy': '0.6269', 'num_tokens': '7.425e+06', 'mean_token_accuracy': '0.836', 'epoch': '0.106'}
{'loss': '0.8897', 'grad_norm': '0.4752', 'learning_rate': '2.009e-09', 'entropy': '0.8495', 'num_tokens': '7.429e+06', 'mean_token_accuracy': '0.7771', 'epoch': '0.106'}
{'loss': '0.8022', 'grad_norm': '0.4063', 'learning_rate': '1.838e-09', 'entropy': '0.7718', 'num_tokens': '7.433e+06', 'mean_token_accuracy': '0.8122', 'epoch': '0.106'}
{'loss': '1.223', 'grad_norm': '0.5092', 'learning_rate': '1.675e-09', 'entropy': '1.044', 'num_tokens': '7.439e+06', 'mean_token_accuracy': '0.7485', 'epoch': '0.106'}
{'loss': '0.5436', 'grad_norm': '0.3479', 'learning_rate': '1.519e-09', 'entropy': '0.546', 'num_tokens': '7.443e+06', 'mean_token_accuracy': '0.8575', 'epoch': '0.106'}
{'loss': '0.677', 'grad_norm': '0.4141', 'learning_rate': '1.371e-09', 'entropy': '0.6903', 'num_tokens': '7.447e+06', 'mean_token_accuracy': '0.8201', 'epoch': '0.106'}
{'loss': '0.9967', 'grad_norm': '0.4523', 'learning_rate': '1.23e-09', 'entropy': '1.194', 'num_tokens': '7.453e+06', 'mean_token_accuracy': '0.7079', 'epoch': '0.106'}
{'loss': '0.7037', 'grad_norm': '0.5449', 'learning_rate': '1.097e-09', 'entropy': '0.7757', 'num_tokens': '7.456e+06', 'mean_token_accuracy': '0.8019', 'epoch': '0.1061'}
{'loss': '1.438', 'grad_norm': '0.4643', 'learning_rate': '9.721e-10', 'entropy': '1.28', 'num_tokens': '7.462e+06', 'mean_token_accuracy': '0.6923', 'epoch': '0.1061'}
{'loss': '0.9223', 'grad_norm': '0.4721', 'learning_rate': '8.544e-10', 'entropy': '0.9416', 'num_tokens': '7.465e+06', 'mean_token_accuracy': '0.7751', 'epoch': '0.1061'}
{'loss': '0.9012', 'grad_norm': '0.3786', 'learning_rate': '7.442e-10', 'entropy': '0.9322', 'num_tokens': '7.472e+06', 'mean_token_accuracy': '0.7691', 'epoch': '0.1061'}
{'loss': '0.6487', 'grad_norm': '0.3917', 'learning_rate': '6.417e-10', 'entropy': '0.6786', 'num_tokens': '7.476e+06', 'mean_token_accuracy': '0.8234', 'epoch': '0.1061'}
{'loss': '0.4995', 'grad_norm': '0.3615', 'learning_rate': '5.468e-10', 'entropy': '0.6527', 'num_tokens': '7.48e+06', 'mean_token_accuracy': '0.8417', 'epoch': '0.1061'}
{'loss': '0.7336', 'grad_norm': '0.4747', 'learning_rate': '4.595e-10', 'entropy': '0.7343', 'num_tokens': '7.485e+06', 'mean_token_accuracy': '0.8175', 'epoch': '0.1061'}
{'loss': '0.8756', 'grad_norm': '0.4557', 'learning_rate': '3.797e-10', 'entropy': '0.844', 'num_tokens': '7.49e+06', 'mean_token_accuracy': '0.7916', 'epoch': '0.1061'}
{'loss': '0.6387', 'grad_norm': '0.345', 'learning_rate': '3.076e-10', 'entropy': '0.6244', 'num_tokens': '7.497e+06', 'mean_token_accuracy': '0.8335', 'epoch': '0.1061'}
{'loss': '0.9184', 'grad_norm': '0.4162', 'learning_rate': '2.43e-10', 'entropy': '0.9056', 'num_tokens': '7.502e+06', 'mean_token_accuracy': '0.7682', 'epoch': '0.1061'}
{'loss': '1.021', 'grad_norm': '0.4554', 'learning_rate': '1.861e-10', 'entropy': '0.8946', 'num_tokens': '7.507e+06', 'mean_token_accuracy': '0.7626', 'epoch': '0.1061'}
{'loss': '0.5512', 'grad_norm': '0.3607', 'learning_rate': '1.367e-10', 'entropy': '0.6017', 'num_tokens': '7.511e+06', 'mean_token_accuracy': '0.8505', 'epoch': '0.1062'}
{'loss': '0.9265', 'grad_norm': '0.4942', 'learning_rate': '9.493e-11', 'entropy': '0.8427', 'num_tokens': '7.515e+06', 'mean_token_accuracy': '0.791', 'epoch': '0.1062'}
{'loss': '1.217', 'grad_norm': '0.5912', 'learning_rate': '6.075e-11', 'entropy': '1.211', 'num_tokens': '7.519e+06', 'mean_token_accuracy': '0.7102', 'epoch': '0.1062'}
{'loss': '0.6956', 'grad_norm': '0.2899', 'learning_rate': '3.417e-11', 'entropy': '0.709', 'num_tokens': '7.524e+06', 'mean_token_accuracy': '0.8087', 'epoch': '0.1062'}
{'loss': '0.9851', 'grad_norm': '0.3939', 'learning_rate': '1.519e-11', 'entropy': '0.9089', 'num_tokens': '7.532e+06', 'mean_token_accuracy': '0.767', 'epoch': '0.1062'}
{'loss': '0.7145', 'grad_norm': '0.349', 'learning_rate': '3.797e-12', 'entropy': '0.7247', 'num_tokens': '7.536e+06', 'mean_token_accuracy': '0.8166', 'epoch': '0.1062'}
{'train_runtime': '2999', 'train_samples_per_second': '64.01', 'train_steps_per_second': '4.001', 'train_loss': '0.09485', 'epoch': '0.1062'}
TrainOutput(global_step=12000, training_loss=0.09484911877910296, metrics={'train_runtime': 2999.3777, 'train_samples_per_second': 64.013, 'train_steps_per_second': 4.001, 'total_flos': 2.8290843996951675e+18, 'train_loss': 0.09484911877910296, 'epoch': 0.10619797958343842})
Saving LoRA to /workspace/finetuned/cybersec-4b/lora
Merging to BF16: /workspace/finetuned/cybersec-4b/merged

Loading weights:   0%|          | 0/398 [00:00<?, ?it/s]
Loading weights:   0%|          | 1/398 [00:00<01:38,  4.03it/s]
Loading weights:   4%|▍         | 15/398 [00:00<00:07, 51.77it/s]
Loading weights:   8%|▊         | 33/398 [00:00<00:03, 92.45it/s]
Loading weights:  12%|█▏        | 48/398 [00:00<00:03, 108.00it/s]
Loading weights:  17%|█▋        | 66/398 [00:00<00:02, 128.01it/s]
Loading weights:  20%|██        | 81/398 [00:00<00:02, 132.03it/s]
Loading weights:  25%|██▍       | 98/398 [00:00<00:02, 136.50it/s]
Loading weights:  29%|██▉       | 115/398 [00:01<00:02, 139.94it/s]
Loading weights:  33%|███▎      | 133/398 [00:01<00:01, 150.94it/s]
Loading weights:  37%|███▋      | 149/398 [00:01<00:01, 135.72it/s]
Loading weights:  42%|████▏     | 168/398 [00:01<00:01, 146.05it/s]
Loading weights:  46%|████▌     | 184/398 [00:01<00:01, 143.76it/s]
Loading weights:  50%|█████     | 199/398 [00:01<00:01, 122.20it/s]
Loading weights:  57%|█████▋    | 225/398 [00:01<00:01, 149.82it/s]
Loading weights:  62%|██████▏   | 246/398 [00:01<00:00, 158.06it/s]
Loading weights:  66%|██████▋   | 264/398 [00:02<00:00, 161.13it/s]
Loading weights:  71%|███████   | 281/398 [00:02<00:00, 149.65it/s]
Loading weights:  76%|███████▌  | 301/398 [00:02<00:00, 156.40it/s]
Loading weights:  80%|███████▉  | 317/398 [00:02<00:00, 154.52it/s]
Loading weights:  84%|████████▍ | 334/398 [00:02<00:00, 155.98it/s]
Loading weights:  88%|████████▊ | 351/398 [00:02<00:00, 157.25it/s]
Loading weights:  92%|█████████▏| 367/398 [00:02<00:00, 154.40it/s]
Loading weights:  97%|█████████▋| 385/398 [00:02<00:00, 158.95it/s]
Loading weights: 100%|██████████| 398/398 [00:02<00:00, 139.22it/s]

Writing model shards:   0%|          | 0/3 [00:00<?, ?it/s]
Writing model shards:  33%|███▎      | 1/3 [00:04<00:09,  4.65s/it]
Writing model shards:  67%|██████▋   | 2/3 [00:08<00:04,  4.47s/it]
Writing model shards: 100%|██████████| 3/3 [00:09<00:00,  3.01s/it]
Merged model saved.