Instructions to use fpadovani/ind-latn-10mb-ppt-Dp-10mb_seed10 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use fpadovani/ind-latn-10mb-ppt-Dp-10mb_seed10 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="fpadovani/ind-latn-10mb-ppt-Dp-10mb_seed10")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("fpadovani/ind-latn-10mb-ppt-Dp-10mb_seed10")
model = AutoModelForCausalLM.from_pretrained("fpadovani/ind-latn-10mb-ppt-Dp-10mb_seed10")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use fpadovani/ind-latn-10mb-ppt-Dp-10mb_seed10 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "fpadovani/ind-latn-10mb-ppt-Dp-10mb_seed10"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fpadovani/ind-latn-10mb-ppt-Dp-10mb_seed10",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/fpadovani/ind-latn-10mb-ppt-Dp-10mb_seed10

SGLang

How to use fpadovani/ind-latn-10mb-ppt-Dp-10mb_seed10 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "fpadovani/ind-latn-10mb-ppt-Dp-10mb_seed10" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fpadovani/ind-latn-10mb-ppt-Dp-10mb_seed10",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "fpadovani/ind-latn-10mb-ppt-Dp-10mb_seed10" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "fpadovani/ind-latn-10mb-ppt-Dp-10mb_seed10",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use fpadovani/ind-latn-10mb-ppt-Dp-10mb_seed10 with Docker Model Runner:
```
docker model run hf.co/fpadovani/ind-latn-10mb-ppt-Dp-10mb_seed10
```

fpadovani commited on 11 days ago

Commit

db21b7e

verified ·

1 Parent(s): 06c0cf2

Training checkpoint: step 1500

Browse files

Files changed (10) hide show

checkpoint-1500/added_tokens.json +1202 -0
checkpoint-1500/config.json +34 -0
checkpoint-1500/generation_config.json +9 -0
checkpoint-1500/model.safetensors +3 -0
checkpoint-1500/rng_state.pth +3 -0
checkpoint-1500/special_tokens_map.json +1249 -0
checkpoint-1500/spiece.model +3 -0
checkpoint-1500/tokenizer_config.json +0 -0
checkpoint-1500/trainer_state.json +3034 -0
checkpoint-1500/training_args.bin +3 -0

checkpoint-1500/added_tokens.json ADDED Viewed

	@@ -0,0 +1,1202 @@

+{
+  "<pad>": 50002,
+  "[CLS]": 50000,
+  "[MASK]": 50003,
+  "[SEP]": 50001,
+  "[XXXXX0]": 50771,
+  "[XXXXX1000]": 50549,
+  "[XXXXX1001]": 50657,
+  "[XXXXX1002]": 51158,
+  "[XXXXX1003]": 51044,
+  "[XXXXX1004]": 50688,
+  "[XXXXX1005]": 50689,
+  "[XXXXX1006]": 50410,
+  "[XXXXX1007]": 50724,
+  "[XXXXX1008]": 50171,
+  "[XXXXX1009]": 51077,
+  "[XXXXX100]": 50091,
+  "[XXXXX1010]": 50639,
+  "[XXXXX1011]": 51094,
+  "[XXXXX1012]": 50392,
+  "[XXXXX1013]": 50626,
+  "[XXXXX1014]": 50991,
+  "[XXXXX1015]": 50529,
+  "[XXXXX1016]": 51193,
+  "[XXXXX1017]": 51073,
+  "[XXXXX1018]": 51104,
+  "[XXXXX1019]": 50775,
+  "[XXXXX101]": 50139,
+  "[XXXXX1020]": 50157,
+  "[XXXXX1021]": 50934,
+  "[XXXXX1022]": 51159,
+  "[XXXXX1023]": 50048,
+  "[XXXXX1024]": 51123,
+  "[XXXXX1025]": 50084,
+  "[XXXXX1026]": 50813,
+  "[XXXXX1027]": 50725,
+  "[XXXXX1028]": 50931,
+  "[XXXXX1029]": 50233,
+  "[XXXXX102]": 50213,
+  "[XXXXX1030]": 51023,
+  "[XXXXX1031]": 50098,
+  "[XXXXX1032]": 50802,
+  "[XXXXX1033]": 50061,
+  "[XXXXX1034]": 51139,
+  "[XXXXX1035]": 51110,
+  "[XXXXX1036]": 50204,
+  "[XXXXX1037]": 50023,
+  "[XXXXX1038]": 50142,
+  "[XXXXX1039]": 50328,
+  "[XXXXX103]": 50087,
+  "[XXXXX1040]": 50435,
+  "[XXXXX1041]": 50489,
+  "[XXXXX1042]": 50216,
+  "[XXXXX1043]": 50144,
+  "[XXXXX1044]": 50832,
+  "[XXXXX1045]": 50566,
+  "[XXXXX1046]": 50922,
+  "[XXXXX1047]": 50796,
+  "[XXXXX1048]": 50533,
+  "[XXXXX1049]": 50433,
+  "[XXXXX104]": 50478,
+  "[XXXXX1050]": 50953,
+  "[XXXXX1051]": 50497,
+  "[XXXXX1052]": 50655,
+  "[XXXXX1053]": 50881,
+  "[XXXXX1054]": 50461,
+  "[XXXXX1055]": 50052,
+  "[XXXXX1056]": 50219,
+  "[XXXXX1057]": 50789,
+  "[XXXXX1058]": 50121,
+  "[XXXXX1059]": 51002,
+  "[XXXXX105]": 51192,
+  "[XXXXX1060]": 50158,
+  "[XXXXX1061]": 50352,
+  "[XXXXX1062]": 50637,
+  "[XXXXX1063]": 50440,
+  "[XXXXX1064]": 50590,
+  "[XXXXX1065]": 51107,
+  "[XXXXX1066]": 50758,
+  "[XXXXX1067]": 51011,
+  "[XXXXX1068]": 50985,
+  "[XXXXX1069]": 51175,
+  "[XXXXX106]": 50532,
+  "[XXXXX1070]": 50790,
+  "[XXXXX1071]": 50394,
+  "[XXXXX1072]": 50852,
+  "[XXXXX1073]": 50869,
+  "[XXXXX1074]": 50126,
+  "[XXXXX1075]": 50783,
+  "[XXXXX1076]": 50806,
+  "[XXXXX1077]": 50882,
+  "[XXXXX1078]": 50726,
+  "[XXXXX1079]": 51010,
+  "[XXXXX107]": 50714,
+  "[XXXXX1080]": 50168,
+  "[XXXXX1081]": 50425,
+  "[XXXXX1082]": 50848,
+  "[XXXXX1083]": 50014,
+  "[XXXXX1084]": 50341,
+  "[XXXXX1085]": 50734,
+  "[XXXXX1086]": 50664,
+  "[XXXXX1087]": 51007,
+  "[XXXXX1088]": 51151,
+  "[XXXXX1089]": 50269,
+  "[XXXXX108]": 50996,
+  "[XXXXX1090]": 50826,
+  "[XXXXX1091]": 51105,
+  "[XXXXX1092]": 50731,
+  "[XXXXX1093]": 50856,
+  "[XXXXX1094]": 50369,
+  "[XXXXX1095]": 50065,
+  "[XXXXX1096]": 50309,
+  "[XXXXX1097]": 50838,
+  "[XXXXX1098]": 50451,
+  "[XXXXX1099]": 50384,
+  "[XXXXX109]": 50543,
+  "[XXXXX10]": 50325,
+  "[XXXXX1100]": 50318,
+  "[XXXXX1101]": 50764,
+  "[XXXXX1102]": 51025,
+  "[XXXXX1103]": 50642,
+  "[XXXXX1104]": 50665,
+  "[XXXXX1105]": 50855,
+  "[XXXXX1106]": 51063,
+  "[XXXXX1107]": 50025,
+  "[XXXXX1108]": 50122,
+  "[XXXXX1109]": 50470,
+  "[XXXXX110]": 50062,
+  "[XXXXX1110]": 51019,
+  "[XXXXX1111]": 51124,
+  "[XXXXX1112]": 50146,
+  "[XXXXX1113]": 51125,
+  "[XXXXX1114]": 50434,
+  "[XXXXX1115]": 50818,
+  "[XXXXX1116]": 50960,
+  "[XXXXX1117]": 50258,
+  "[XXXXX1118]": 50596,
+  "[XXXXX1119]": 50083,
+  "[XXXXX111]": 51064,
+  "[XXXXX1120]": 50915,
+  "[XXXXX1121]": 50835,
+  "[XXXXX1122]": 51056,
+  "[XXXXX1123]": 50172,
+  "[XXXXX1124]": 50972,
+  "[XXXXX1125]": 50275,
+  "[XXXXX1126]": 50430,
+  "[XXXXX1127]": 50743,
+  "[XXXXX1128]": 50551,
+  "[XXXXX1129]": 50439,
+  "[XXXXX112]": 50819,
+  "[XXXXX1130]": 50536,
+  "[XXXXX1131]": 50622,
+  "[XXXXX1132]": 50623,
+  "[XXXXX1133]": 50117,
+  "[XXXXX1134]": 50251,
+  "[XXXXX1135]": 50678,
+  "[XXXXX1136]": 50316,
+  "[XXXXX1137]": 50905,
+  "[XXXXX1138]": 50992,
+  "[XXXXX1139]": 50851,
+  "[XXXXX113]": 50443,
+  "[XXXXX1140]": 50499,
+  "[XXXXX1141]": 50629,
+  "[XXXXX1142]": 50944,
+  "[XXXXX1143]": 50373,
+  "[XXXXX1144]": 50123,
+  "[XXXXX1145]": 50173,
+  "[XXXXX1146]": 50891,
+  "[XXXXX1147]": 50471,
+  "[XXXXX1148]": 50560,
+  "[XXXXX1149]": 50541,
+  "[XXXXX114]": 50631,
+  "[XXXXX1150]": 50033,
+  "[XXXXX1151]": 50063,
+  "[XXXXX1152]": 50767,
+  "[XXXXX1153]": 50897,
+  "[XXXXX1154]": 50568,
+  "[XXXXX1155]": 51131,
+  "[XXXXX1156]": 50009,
+  "[XXXXX1157]": 50417,
+  "[XXXXX1158]": 50477,
+  "[XXXXX1159]": 50941,
+  "[XXXXX115]": 50294,
+  "[XXXXX1160]": 50679,
+  "[XXXXX1161]": 51108,
+  "[XXXXX1162]": 50332,
+  "[XXXXX1163]": 50424,
+  "[XXXXX1164]": 50473,
+  "[XXXXX1165]": 50793,
+  "[XXXXX1166]": 50453,
+  "[XXXXX1167]": 50807,
+  "[XXXXX1168]": 50128,
+  "[XXXXX1169]": 50039,
+  "[XXXXX116]": 50759,
+  "[XXXXX1170]": 50735,
+  "[XXXXX1171]": 50366,
+  "[XXXXX1172]": 50808,
+  "[XXXXX1173]": 50426,
+  "[XXXXX1174]": 50076,
+  "[XXXXX1175]": 50943,
+  "[XXXXX1176]": 50252,
+  "[XXXXX1177]": 50999,
+  "[XXXXX1178]": 50264,
+  "[XXXXX1179]": 50607,
+  "[XXXXX117]": 51036,
+  "[XXXXX1180]": 50986,
+  "[XXXXX1181]": 50744,
+  "[XXXXX1182]": 50186,
+  "[XXXXX1183]": 50555,
+  "[XXXXX1184]": 51076,
+  "[XXXXX1185]": 50849,
+  "[XXXXX1186]": 51128,
+  "[XXXXX1187]": 50858,
+  "[XXXXX1188]": 50656,
+  "[XXXXX1189]": 50064,
+  "[XXXXX118]": 50081,
+  "[XXXXX1190]": 51197,
+  "[XXXXX1191]": 50104,
+  "[XXXXX1192]": 50045,
+  "[XXXXX1193]": 50404,
+  "[XXXXX1194]": 50690,
+  "[XXXXX1195]": 50495,
+  "[XXXXX119]": 50108,
+  "[XXXXX11]": 50976,
+  "[XXXXX120]": 51081,
+  "[XXXXX121]": 50464,
+  "[XXXXX122]": 50829,
+  "[XXXXX123]": 50766,
+  "[XXXXX124]": 50732,
+  "[XXXXX125]": 50221,
+  "[XXXXX126]": 50810,
+  "[XXXXX127]": 50130,
+  "[XXXXX128]": 50751,
+  "[XXXXX129]": 50662,
+  "[XXXXX12]": 50602,
+  "[XXXXX130]": 50035,
+  "[XXXXX131]": 50715,
+  "[XXXXX132]": 50340,
+  "[XXXXX133]": 50563,
+  "[XXXXX134]": 50993,
+  "[XXXXX135]": 50304,
+  "[XXXXX136]": 50131,
+  "[XXXXX137]": 50578,
+  "[XXXXX138]": 50757,
+  "[XXXXX139]": 50582,
+  "[XXXXX13]": 50105,
+  "[XXXXX140]": 50085,
+  "[XXXXX141]": 50553,
+  "[XXXXX142]": 50749,
+  "[XXXXX143]": 50351,
+  "[XXXXX144]": 51024,
+  "[XXXXX145]": 51144,
+  "[XXXXX146]": 50042,
+  "[XXXXX147]": 50145,
+  "[XXXXX148]": 50066,
+  "[XXXXX149]": 50876,
+  "[XXXXX14]": 51006,
+  "[XXXXX150]": 50280,
+  "[XXXXX151]": 50093,
+  "[XXXXX152]": 50638,
+  "[XXXXX153]": 50684,
+  "[XXXXX154]": 50820,
+  "[XXXXX155]": 51196,
+  "[XXXXX156]": 50317,
+  "[XXXXX157]": 50284,
+  "[XXXXX158]": 50288,
+  "[XXXXX159]": 50814,
+  "[XXXXX15]": 50608,
+  "[XXXXX160]": 50004,
+  "[XXXXX161]": 50008,
+  "[XXXXX162]": 50377,
+  "[XXXXX163]": 50214,
+  "[XXXXX164]": 50708,
+  "[XXXXX165]": 50390,
+  "[XXXXX166]": 50741,
+  "[XXXXX167]": 50445,
+  "[XXXXX168]": 50936,
+  "[XXXXX169]": 50896,
+  "[XXXXX16]": 50442,
+  "[XXXXX170]": 50109,
+  "[XXXXX171]": 50140,
+  "[XXXXX172]": 50427,
+  "[XXXXX173]": 51074,
+  "[XXXXX174]": 50162,
+  "[XXXXX175]": 51015,
+  "[XXXXX176]": 50524,
+  "[XXXXX177]": 50673,
+  "[XXXXX178]": 50054,
+  "[XXXXX179]": 50805,
+  "[XXXXX17]": 50482,
+  "[XXXXX180]": 50253,
+  "[XXXXX181]": 50493,
+  "[XXXXX182]": 50421,
+  "[XXXXX183]": 51050,
+  "[XXXXX184]": 50521,
+  "[XXXXX185]": 50188,
+  "[XXXXX186]": 50147,
+  "[XXXXX187]": 50201,
+  "[XXXXX188]": 50973,
+  "[XXXXX189]": 50409,
+  "[XXXXX18]": 50115,
+  "[XXXXX190]": 50051,
+  "[XXXXX191]": 51020,
+  "[XXXXX192]": 51047,
+  "[XXXXX193]": 50669,
+  "[XXXXX194]": 50643,
+  "[XXXXX195]": 50159,
+  "[XXXXX196]": 50649,
+  "[XXXXX197]": 51178,
+  "[XXXXX198]": 50043,
+  "[XXXXX199]": 51147,
+  "[XXXXX19]": 51127,
+  "[XXXXX1]": 51114,
+  "[XXXXX200]": 50840,
+  "[XXXXX201]": 50365,
+  "[XXXXX202]": 50232,
+  "[XXXXX203]": 50241,
+  "[XXXXX204]": 50548,
+  "[XXXXX205]": 50781,
+  "[XXXXX206]": 50923,
+  "[XXXXX207]": 51115,
+  "[XXXXX208]": 50344,
+  "[XXXXX209]": 50299,
+  "[XXXXX20]": 51031,
+  "[XXXXX210]": 51179,
+  "[XXXXX211]": 51163,
+  "[XXXXX212]": 50562,
+  "[XXXXX213]": 50094,
+  "[XXXXX214]": 50257,
+  "[XXXXX215]": 50967,
+  "[XXXXX216]": 50523,
+  "[XXXXX217]": 51132,
+  "[XXXXX218]": 51146,
+  "[XXXXX219]": 50411,
+  "[XXXXX21]": 50354,
+  "[XXXXX220]": 50853,
+  "[XXXXX221]": 50886,
+  "[XXXXX222]": 50600,
+  "[XXXXX223]": 50873,
+  "[XXXXX224]": 50416,
+  "[XXXXX225]": 50675,
+  "[XXXXX226]": 50583,
+  "[XXXXX227]": 50151,
+  "[XXXXX228]": 50927,
+  "[XXXXX229]": 50609,
+  "[XXXXX22]": 50589,
+  "[XXXXX230]": 50238,
+  "[XXXXX231]": 50242,
+  "[XXXXX232]": 51016,
+  "[XXXXX233]": 50406,
+  "[XXXXX234]": 50908,
+  "[XXXXX235]": 50024,
+  "[XXXXX236]": 50270,
+  "[XXXXX237]": 50132,
+  "[XXXXX238]": 50800,
+  "[XXXXX239]": 50236,
+  "[XXXXX23]": 50007,
+  "[XXXXX240]": 50005,
+  "[XXXXX241]": 50894,
+  "[XXXXX242]": 50694,
+  "[XXXXX243]": 51071,
+  "[XXXXX244]": 51017,
+  "[XXXXX245]": 50068,
+  "[XXXXX246]": 50682,
+  "[XXXXX247]": 50913,
+  "[XXXXX248]": 51028,
+  "[XXXXX249]": 50037,
+  "[XXXXX24]": 51169,
+  "[XXXXX250]": 50747,
+  "[XXXXX251]": 50947,
+  "[XXXXX252]": 50329,
+  "[XXXXX253]": 50612,
+  "[XXXXX254]": 50110,
+  "[XXXXX255]": 50824,
+  "[XXXXX256]": 50558,
+  "[XXXXX257]": 50969,
+  "[XXXXX258]": 50729,
+  "[XXXXX259]": 50125,
+  "[XXXXX25]": 51160,
+  "[XXXXX260]": 50040,
+  "[XXXXX261]": 51038,
+  "[XXXXX262]": 50347,
+  "[XXXXX263]": 50740,
+  "[XXXXX264]": 51195,
+  "[XXXXX265]": 50879,
+  "[XXXXX266]": 50770,
+  "[XXXXX267]": 50178,
+  "[XXXXX268]": 50804,
+  "[XXXXX269]": 50906,
+  "[XXXXX26]": 50150,
+  "[XXXXX270]": 51191,
+  "[XXXXX271]": 50047,
+  "[XXXXX272]": 50195,
+  "[XXXXX273]": 50554,
+  "[XXXXX274]": 50833,
+  "[XXXXX275]": 50346,
+  "[XXXXX276]": 50950,
+  "[XXXXX277]": 50864,
+  "[XXXXX278]": 50259,
+  "[XXXXX279]": 50671,
+  "[XXXXX27]": 50468,
+  "[XXXXX280]": 50082,
+  "[XXXXX281]": 50565,
+  "[XXXXX282]": 50984,
+  "[XXXXX283]": 50650,
+  "[XXXXX284]": 50975,
+  "[XXXXX285]": 50603,
+  "[XXXXX286]": 50027,
+  "[XXXXX287]": 50181,
+  "[XXXXX288]": 50981,
+  "[XXXXX289]": 50885,
+  "[XXXXX28]": 50249,
+  "[XXXXX290]": 50522,
+  "[XXXXX291]": 50050,
+  "[XXXXX292]": 50313,
+  "[XXXXX293]": 50503,
+  "[XXXXX294]": 51176,
+  "[XXXXX295]": 51177,
+  "[XXXXX296]": 50379,
+  "[XXXXX297]": 50635,
+  "[XXXXX298]": 50723,
+  "[XXXXX299]": 50200,
+  "[XXXXX29]": 50892,
+  "[XXXXX2]": 51119,
+  "[XXXXX300]": 51026,
+  "[XXXXX301]": 50528,
+  "[XXXXX302]": 51149,
+  "[XXXXX303]": 50044,
+  "[XXXXX304]": 50209,
+  "[XXXXX305]": 50988,
+  "[XXXXX306]": 50518,
+  "[XXXXX307]": 50088,
+  "[XXXXX308]": 50733,
+  "[XXXXX309]": 51066,
+  "[XXXXX30]": 50506,
+  "[XXXXX310]": 50285,
+  "[XXXXX311]": 50859,
+  "[XXXXX312]": 50263,
+  "[XXXXX313]": 50594,
+  "[XXXXX314]": 51103,
+  "[XXXXX315]": 50871,
+  "[XXXXX316]": 50300,
+  "[XXXXX317]": 50760,
+  "[XXXXX318]": 50055,
+  "[XXXXX319]": 50389,
+  "[XXXXX31]": 50212,
+  "[XXXXX320]": 50444,
+  "[XXXXX321]": 50994,
+  "[XXXXX322]": 50281,
+  "[XXXXX323]": 50179,
+  "[XXXXX324]": 50455,
+  "[XXXXX325]": 50970,
+  "[XXXXX326]": 50666,
+  "[XXXXX327]": 50604,
+  "[XXXXX328]": 50627,
+  "[XXXXX329]": 50475,
+  "[XXXXX32]": 50277,
+  "[XXXXX330]": 50667,
+  "[XXXXX331]": 50348,
+  "[XXXXX332]": 50772,
+  "[XXXXX333]": 50163,
+  "[XXXXX334]": 50569,
+  "[XXXXX335]": 50018,
+  "[XXXXX336]": 50883,
+  "[XXXXX337]": 50067,
+  "[XXXXX338]": 51184,
+  "[XXXXX339]": 50997,
+  "[XXXXX33]": 50403,
+  "[XXXXX340]": 51013,
+  "[XXXXX341]": 50957,
+  "[XXXXX342]": 51106,
+  "[XXXXX343]": 51174,
+  "[XXXXX344]": 50391,
+  "[XXXXX345]": 50948,
+  "[XXXXX346]": 50180,
+  "[XXXXX347]": 50811,
+  "[XXXXX348]": 50401,
+  "[XXXXX349]": 50911,
+  "[XXXXX34]": 50223,
+  "[XXXXX350]": 51089,
+  "[XXXXX351]": 50777,
+  "[XXXXX352]": 50189,
+  "[XXXXX353]": 50492,
+  "[XXXXX354]": 50205,
+  "[XXXXX355]": 50374,
+  "[XXXXX356]": 51152,
+  "[XXXXX357]": 50695,
+  "[XXXXX358]": 51117,
+  "[XXXXX359]": 50573,
+  "[XXXXX35]": 50752,
+  "[XXXXX360]": 50955,
+  "[XXXXX361]": 50349,
+  "[XXXXX362]": 51137,
+  "[XXXXX363]": 50641,
+  "[XXXXX364]": 51148,
+  "[XXXXX365]": 50255,
+  "[XXXXX366]": 50310,
+  "[XXXXX367]": 50314,
+  "[XXXXX368]": 50971,
+  "[XXXXX369]": 50916,
+  "[XXXXX36]": 50437,
+  "[XXXXX370]": 51096,
+  "[XXXXX371]": 50557,
+  "[XXXXX372]": 51095,
+  "[XXXXX373]": 50382,
+  "[XXXXX374]": 50119,
+  "[XXXXX375]": 50958,
+  "[XXXXX376]": 50624,
+  "[XXXXX377]": 50483,
+  "[XXXXX378]": 50605,
+  "[XXXXX379]": 50017,
+  "[XXXXX37]": 50706,
+  "[XXXXX380]": 50246,
+  "[XXXXX381]": 50588,
+  "[XXXXX382]": 51008,
+  "[XXXXX383]": 50801,
+  "[XXXXX384]": 50290,
+  "[XXXXX385]": 50899,
+  "[XXXXX386]": 50206,
+  "[XXXXX387]": 50716,
+  "[XXXXX388]": 51082,
+  "[XXXXX389]": 50177,
+  "[XXXXX38]": 50736,
+  "[XXXXX390]": 50282,
+  "[XXXXX391]": 51183,
+  "[XXXXX392]": 50286,
+  "[XXXXX393]": 50659,
+  "[XXXXX394]": 50378,
+  "[XXXXX395]": 50709,
+  "[XXXXX396]": 50028,
+  "[XXXXX397]": 50235,
+  "[XXXXX398]": 51014,
+  "[XXXXX399]": 50880,
+  "[XXXXX39]": 50305,
+  "[XXXXX3]": 50077,
+  "[XXXXX400]": 50155,
+  "[XXXXX401]": 50710,
+  "[XXXXX402]": 50507,
+  "[XXXXX403]": 50860,
+  "[XXXXX404]": 50661,
+  "[XXXXX405]": 51083,
+  "[XXXXX406]": 50888,
+  "[XXXXX407]": 50069,
+  "[XXXXX408]": 50691,
+  "[XXXXX409]": 51055,
+  "[XXXXX40]": 50337,
+  "[XXXXX410]": 50137,
+  "[XXXXX411]": 50791,
+  "[XXXXX412]": 50136,
+  "[XXXXX413]": 50730,
+  "[XXXXX414]": 50267,
+  "[XXXXX415]": 50012,
+  "[XXXXX416]": 50681,
+  "[XXXXX417]": 50942,
+  "[XXXXX418]": 50720,
+  "[XXXXX419]": 50844,
+  "[XXXXX41]": 50387,
+  "[XXXXX420]": 50111,
+  "[XXXXX421]": 51003,
+  "[XXXXX422]": 50785,
+  "[XXXXX423]": 50149,
+  "[XXXXX424]": 50422,
+  "[XXXXX425]": 51086,
+  "[XXXXX426]": 50862,
+  "[XXXXX427]": 50576,
+  "[XXXXX428]": 51049,
+  "[XXXXX429]": 50630,
+  "[XXXXX42]": 50799,
+  "[XXXXX430]": 50289,
+  "[XXXXX431]": 50184,
+  "[XXXXX432]": 50319,
+  "[XXXXX433]": 51116,
+  "[XXXXX434]": 50519,
+  "[XXXXX435]": 50496,
+  "[XXXXX436]": 50517,
+  "[XXXXX437]": 50865,
+  "[XXXXX438]": 50737,
+  "[XXXXX439]": 50356,
+  "[XXXXX43]": 50359,
+  "[XXXXX440]": 50898,
+  "[XXXXX441]": 50113,
+  "[XXXXX442]": 50527,
+  "[XXXXX443]": 50762,
+  "[XXXXX444]": 50540,
+  "[XXXXX445]": 50925,
+  "[XXXXX446]": 50711,
+  "[XXXXX447]": 50143,
+  "[XXXXX448]": 51170,
+  "[XXXXX449]": 50413,
+  "[XXXXX44]": 50097,
+  "[XXXXX450]": 50375,
+  "[XXXXX451]": 50846,
+  "[XXXXX452]": 50217,
+  "[XXXXX453]": 50699,
+  "[XXXXX454]": 50412,
+  "[XXXXX455]": 50418,
+  "[XXXXX456]": 51143,
+  "[XXXXX457]": 51141,
+  "[XXXXX458]": 51157,
+  "[XXXXX459]": 50825,
+  "[XXXXX45]": 50203,
+  "[XXXXX460]": 50194,
+  "[XXXXX461]": 50148,
+  "[XXXXX462]": 50276,
+  "[XXXXX463]": 50271,
+  "[XXXXX464]": 50836,
+  "[XXXXX465]": 50854,
+  "[XXXXX466]": 50164,
+  "[XXXXX467]": 50484,
+  "[XXXXX468]": 51097,
+  "[XXXXX469]": 50182,
+  "[XXXXX46]": 50505,
+  "[XXXXX470]": 50670,
+  "[XXXXX471]": 50595,
+  "[XXXXX472]": 50237,
+  "[XXXXX473]": 50057,
+  "[XXXXX474]": 50431,
+  "[XXXXX475]": 50738,
+  "[XXXXX476]": 51145,
+  "[XXXXX477]": 50247,
+  "[XXXXX478]": 50029,
+  "[XXXXX479]": 50917,
+  "[XXXXX47]": 50579,
+  "[XXXXX480]": 50702,
+  "[XXXXX481]": 50831,
+  "[XXXXX482]": 50753,
+  "[XXXXX483]": 50598,
+  "[XXXXX484]": 51136,
+  "[XXXXX485]": 50092,
+  "[XXXXX486]": 50202,
+  "[XXXXX487]": 50106,
+  "[XXXXX488]": 50924,
+  "[XXXXX489]": 51113,
+  "[XXXXX48]": 50842,
+  "[XXXXX490]": 50761,
+  "[XXXXX491]": 50577,
+  "[XXXXX492]": 50692,
+  "[XXXXX493]": 51130,
+  "[XXXXX494]": 50932,
+  "[XXXXX495]": 50837,
+  "[XXXXX496]": 50893,
+  "[XXXXX497]": 50773,
+  "[XXXXX498]": 50360,
+  "[XXXXX499]": 50900,
+  "[XXXXX49]": 51062,
+  "[XXXXX4]": 50396,
+  "[XXXXX500]": 51034,
+  "[XXXXX501]": 50472,
+  "[XXXXX502]": 50660,
+  "[XXXXX503]": 50815,
+  "[XXXXX504]": 50939,
+  "[XXXXX505]": 50419,
+  "[XXXXX506]": 50324,
+  "[XXXXX507]": 50428,
+  "[XXXXX508]": 50274,
+  "[XXXXX509]": 50311,
+  "[XXXXX50]": 50459,
+  "[XXXXX510]": 50674,
+  "[XXXXX511]": 50580,
+  "[XXXXX512]": 50956,
+  "[XXXXX513]": 50165,
+  "[XXXXX514]": 50320,
+  "[XXXXX515]": 50648,
+  "[XXXXX516]": 50386,
+  "[XXXXX517]": 50058,
+  "[XXXXX518]": 51045,
+  "[XXXXX519]": 50103,
+  "[XXXXX51]": 50979,
+  "[XXXXX520]": 50530,
+  "[XXXXX521]": 50907,
+  "[XXXXX522]": 50260,
+  "[XXXXX523]": 50632,
+  "[XXXXX524]": 50926,
+  "[XXXXX525]": 50687,
+  "[XXXXX526]": 50160,
+  "[XXXXX527]": 51053,
+  "[XXXXX528]": 50652,
+  "[XXXXX529]": 51041,
+  "[XXXXX52]": 50272,
+  "[XXXXX530]": 51111,
+  "[XXXXX531]": 51009,
+  "[XXXXX532]": 51079,
+  "[XXXXX533]": 50487,
+  "[XXXXX534]": 51059,
+  "[XXXXX535]": 50335,
+  "[XXXXX536]": 50380,
+  "[XXXXX537]": 50301,
+  "[XXXXX538]": 51067,
+  "[XXXXX539]": 51052,
+  "[XXXXX53]": 51048,
+  "[XXXXX540]": 50822,
+  "[XXXXX541]": 51040,
+  "[XXXXX542]": 50490,
+  "[XXXXX543]": 50448,
+  "[XXXXX544]": 50812,
+  "[XXXXX545]": 50099,
+  "[XXXXX546]": 51199,
+  "[XXXXX547]": 50226,
+  "[XXXXX548]": 50538,
+  "[XXXXX549]": 50342,
+  "[XXXXX54]": 50701,
+  "[XXXXX550]": 50544,
+  "[XXXXX551]": 50398,
+  "[XXXXX552]": 51102,
+  "[XXXXX553]": 50919,
+  "[XXXXX554]": 50615,
+  "[XXXXX555]": 50769,
+  "[XXXXX556]": 50940,
+  "[XXXXX557]": 50070,
+  "[XXXXX558]": 50680,
+  "[XXXXX559]": 50611,
+  "[XXXXX55]": 50574,
+  "[XXXXX560]": 50381,
+  "[XXXXX561]": 50350,
+  "[XXXXX562]": 50175,
+  "[XXXXX563]": 50584,
+  "[XXXXX564]": 50703,
+  "[XXXXX565]": 50989,
+  "[XXXXX566]": 50877,
+  "[XXXXX567]": 51039,
+  "[XXXXX568]": 51018,
+  "[XXXXX569]": 50452,
+  "[XXXXX56]": 51099,
+  "[XXXXX570]": 50161,
+  "[XXXXX571]": 50921,
+  "[XXXXX572]": 50676,
+  "[XXXXX573]": 50326,
+  "[XXXXX574]": 50987,
+  "[XXXXX575]": 50361,
+  "[XXXXX576]": 50071,
+  "[XXXXX577]": 50803,
+  "[XXXXX578]": 50400,
+  "[XXXXX579]": 51091,
+  "[XXXXX57]": 50501,
+  "[XXXXX580]": 50887,
+  "[XXXXX581]": 50839,
+  "[XXXXX582]": 51138,
+  "[XXXXX583]": 50816,
+  "[XXXXX584]": 50339,
+  "[XXXXX585]": 50463,
+  "[XXXXX586]": 50399,
+  "[XXXXX587]": 51070,
+  "[XXXXX588]": 50457,
+  "[XXXXX589]": 50828,
+  "[XXXXX58]": 50183,
+  "[XXXXX590]": 50080,
+  "[XXXXX591]": 50550,
+  "[XXXXX592]": 50129,
+  "[XXXXX593]": 50273,
+  "[XXXXX594]": 50388,
+  "[XXXXX595]": 50491,
+  "[XXXXX596]": 50278,
+  "[XXXXX597]": 50011,
+  "[XXXXX598]": 50254,
+  "[XXXXX599]": 50302,
+  "[XXXXX59]": 50353,
+  "[XXXXX5]": 50010,
+  "[XXXXX600]": 50704,
+  "[XXXXX601]": 51118,
+  "[XXXXX602]": 50049,
+  "[XXXXX603]": 51190,
+  "[XXXXX604]": 51142,
+  "[XXXXX605]": 50707,
+  "[XXXXX606]": 50937,
+  "[XXXXX607]": 50875,
+  "[XXXXX608]": 50564,
+  "[XXXXX609]": 50345,
+  "[XXXXX60]": 50713,
+  "[XXXXX610]": 50949,
+  "[XXXXX611]": 50978,
+  "[XXXXX612]": 50330,
+  "[XXXXX613]": 51035,
+  "[XXXXX614]": 50402,
+  "[XXXXX615]": 50176,
+  "[XXXXX616]": 51133,
+  "[XXXXX617]": 50222,
+  "[XXXXX618]": 50118,
+  "[XXXXX619]": 50196,
+  "[XXXXX61]": 50298,
+  "[XXXXX620]": 50668,
+  "[XXXXX621]": 50462,
+  "[XXXXX622]": 50218,
+  "[XXXXX623]": 51101,
+  "[XXXXX624]": 50362,
+  "[XXXXX625]": 50779,
+  "[XXXXX626]": 50513,
+  "[XXXXX627]": 50746,
+  "[XXXXX628]": 50141,
+  "[XXXXX629]": 50479,
+  "[XXXXX62]": 50056,
+  "[XXXXX630]": 51180,
+  "[XXXXX631]": 50872,
+  "[XXXXX632]": 50315,
+  "[XXXXX633]": 50397,
+  "[XXXXX634]": 51165,
+  "[XXXXX635]": 50046,
+  "[XXXXX636]": 50795,
+  "[XXXXX637]": 50959,
+  "[XXXXX638]": 50786,
+  "[XXXXX639]": 50240,
+  "[XXXXX63]": 51061,
+  "[XXXXX640]": 50601,
+  "[XXXXX641]": 50904,
+  "[XXXXX642]": 50376,
+  "[XXXXX643]": 50663,
+  "[XXXXX644]": 50613,
+  "[XXXXX645]": 50467,
+  "[XXXXX646]": 50546,
+  "[XXXXX647]": 50465,
+  "[XXXXX648]": 50395,
+  "[XXXXX649]": 50287,
+  "[XXXXX64]": 51162,
+  "[XXXXX650]": 50474,
+  "[XXXXX651]": 50727,
+  "[XXXXX652]": 50243,
+  "[XXXXX653]": 51100,
+  "[XXXXX654]": 50705,
+  "[XXXXX655]": 51012,
+  "[XXXXX656]": 50447,
+  "[XXXXX657]": 50719,
+  "[XXXXX658]": 50712,
+  "[XXXXX659]": 50476,
+  "[XXXXX65]": 50587,
+  "[XXXXX660]": 51054,
+  "[XXXXX661]": 51166,
+  "[XXXXX662]": 50494,
+  "[XXXXX663]": 50230,
+  "[XXXXX664]": 50621,
+  "[XXXXX665]": 50809,
+  "[XXXXX666]": 50265,
+  "[XXXXX667]": 50645,
+  "[XXXXX668]": 50817,
+  "[XXXXX669]": 50030,
+  "[XXXXX66]": 51029,
+  "[XXXXX670]": 51004,
+  "[XXXXX671]": 50685,
+  "[XXXXX672]": 50625,
+  "[XXXXX673]": 50890,
+  "[XXXXX674]": 50383,
+  "[XXXXX675]": 50929,
+  "[XXXXX676]": 50116,
+  "[XXXXX677]": 51072,
+  "[XXXXX678]": 51126,
+  "[XXXXX679]": 50755,
+  "[XXXXX67]": 50717,
+  "[XXXXX680]": 50303,
+  "[XXXXX681]": 50504,
+  "[XXXXX682]": 51075,
+  "[XXXXX683]": 50261,
+  "[XXXXX684]": 50861,
+  "[XXXXX685]": 50306,
+  "[XXXXX686]": 50016,
+  "[XXXXX687]": 50101,
+  "[XXXXX688]": 51098,
+  "[XXXXX689]": 50827,
+  "[XXXXX68]": 50863,
+  "[XXXXX690]": 50640,
+  "[XXXXX691]": 50031,
+  "[XXXXX692]": 50768,
+  "[XXXXX693]": 50013,
+  "[XXXXX694]": 51032,
+  "[XXXXX695]": 50166,
+  "[XXXXX696]": 50127,
+  "[XXXXX697]": 50414,
+  "[XXXXX698]": 50742,
+  "[XXXXX699]": 51042,
+  "[XXXXX69]": 51112,
+  "[XXXXX6]": 50593,
+  "[XXXXX700]": 50292,
+  "[XXXXX701]": 50174,
+  "[XXXXX702]": 50547,
+  "[XXXXX703]": 50498,
+  "[XXXXX704]": 51078,
+  "[XXXXX705]": 50133,
+  "[XXXXX706]": 50041,
+  "[XXXXX707]": 51181,
+  "[XXXXX708]": 50585,
+  "[XXXXX709]": 50776,
+  "[XXXXX70]": 50053,
+  "[XXXXX710]": 50336,
+  "[XXXXX711]": 50765,
+  "[XXXXX712]": 50224,
+  "[XXXXX713]": 50599,
+  "[XXXXX714]": 50207,
+  "[XXXXX715]": 51043,
+  "[XXXXX716]": 51090,
+  "[XXXXX717]": 50006,
+  "[XXXXX718]": 50901,
+  "[XXXXX719]": 51001,
+  "[XXXXX71]": 50954,
+  "[XXXXX720]": 50646,
+  "[XXXXX721]": 50075,
+  "[XXXXX722]": 51120,
+  "[XXXXX723]": 50215,
+  "[XXXXX724]": 50415,
+  "[XXXXX725]": 50154,
+  "[XXXXX726]": 50935,
+  "[XXXXX727]": 50545,
+  "[XXXXX728]": 51085,
+  "[XXXXX729]": 50256,
+  "[XXXXX72]": 50363,
+  "[XXXXX730]": 50700,
+  "[XXXXX731]": 50262,
+  "[XXXXX732]": 50930,
+  "[XXXXX733]": 50333,
+  "[XXXXX734]": 50884,
+  "[XXXXX735]": 51030,
+  "[XXXXX736]": 50231,
+  "[XXXXX737]": 50405,
+  "[XXXXX738]": 50962,
+  "[XXXXX739]": 50458,
+  "[XXXXX73]": 50019,
+  "[XXXXX740]": 50878,
+  "[XXXXX741]": 50910,
+  "[XXXXX742]": 50170,
+  "[XXXXX743]": 50718,
+  "[XXXXX744]": 50616,
+  "[XXXXX745]": 50358,
+  "[XXXXX746]": 50152,
+  "[XXXXX747]": 50525,
+  "[XXXXX748]": 51167,
+  "[XXXXX749]": 50653,
+  "[XXXXX74]": 50952,
+  "[XXXXX750]": 50500,
+  "[XXXXX751]": 50446,
+  "[XXXXX752]": 50995,
+  "[XXXXX753]": 50945,
+  "[XXXXX754]": 51173,
+  "[XXXXX755]": 50823,
+  "[XXXXX756]": 51060,
+  "[XXXXX757]": 50748,
+  "[XXXXX758]": 50020,
+  "[XXXXX759]": 50197,
+  "[XXXXX75]": 50502,
+  "[XXXXX760]": 50291,
+  "[XXXXX761]": 50821,
+  "[XXXXX762]": 50190,
+  "[XXXXX763]": 50745,
+  "[XXXXX764]": 50778,
+  "[XXXXX765]": 50556,
+  "[XXXXX766]": 51161,
+  "[XXXXX767]": 50466,
+  "[XXXXX768]": 50792,
+  "[XXXXX769]": 50208,
+  "[XXXXX76]": 50169,
+  "[XXXXX770]": 50647,
+  "[XXXXX771]": 50610,
+  "[XXXXX772]": 51140,
+  "[XXXXX773]": 50441,
+  "[XXXXX774]": 50870,
+  "[XXXXX775]": 50368,
+  "[XXXXX776]": 50026,
+  "[XXXXX777]": 50677,
+  "[XXXXX778]": 51168,
+  "[XXXXX779]": 50481,
+  "[XXXXX77]": 50515,
+  "[XXXXX780]": 50225,
+  "[XXXXX781]": 50408,
+  "[XXXXX782]": 50090,
+  "[XXXXX783]": 50951,
+  "[XXXXX784]": 50015,
+  "[XXXXX785]": 50658,
+  "[XXXXX786]": 50617,
+  "[XXXXX787]": 50721,
+  "[XXXXX788]": 51084,
+  "[XXXXX789]": 50830,
+  "[XXXXX78]": 50100,
+  "[XXXXX790]": 50606,
+  "[XXXXX791]": 50124,
+  "[XXXXX792]": 50078,
+  "[XXXXX793]": 51188,
+  "[XXXXX794]": 50750,
+  "[XXXXX795]": 50138,
+  "[XXXXX796]": 50239,
+  "[XXXXX797]": 50385,
+  "[XXXXX798]": 50780,
+  "[XXXXX799]": 50683,
+  "[XXXXX79]": 50436,
+  "[XXXXX7]": 50211,
+  "[XXXXX800]": 51185,
+  "[XXXXX801]": 50250,
+  "[XXXXX802]": 50456,
+  "[XXXXX803]": 50508,
+  "[XXXXX804]": 51194,
+  "[XXXXX805]": 50021,
+  "[XXXXX806]": 50307,
+  "[XXXXX807]": 50199,
+  "[XXXXX808]": 51129,
+  "[XXXXX809]": 50686,
+  "[XXXXX80]": 50370,
+  "[XXXXX810]": 50343,
+  "[XXXXX811]": 50614,
+  "[XXXXX812]": 50636,
+  "[XXXXX813]": 51198,
+  "[XXXXX814]": 50788,
+  "[XXXXX815]": 50120,
+  "[XXXXX816]": 50135,
+  "[XXXXX817]": 50293,
+  "[XXXXX818]": 50153,
+  "[XXXXX819]": 50516,
+  "[XXXXX81]": 50581,
+  "[XXXXX820]": 50586,
+  "[XXXXX821]": 50509,
+  "[XXXXX822]": 50220,
+  "[XXXXX823]": 51068,
+  "[XXXXX824]": 50946,
+  "[XXXXX825]": 50965,
+  "[XXXXX826]": 50032,
+  "[XXXXX827]": 50156,
+  "[XXXXX828]": 50449,
+  "[XXXXX829]": 50531,
+  "[XXXXX82]": 51164,
+  "[XXXXX830]": 50486,
+  "[XXXXX831]": 50357,
+  "[XXXXX832]": 51022,
+  "[XXXXX833]": 50633,
+  "[XXXXX834]": 50754,
+  "[XXXXX835]": 51134,
+  "[XXXXX836]": 50193,
+  "[XXXXX837]": 51088,
+  "[XXXXX838]": 50722,
+  "[XXXXX839]": 50756,
+  "[XXXXX83]": 50977,
+  "[XXXXX840]": 50022,
+  "[XXXXX841]": 51186,
+  "[XXXXX842]": 50575,
+  "[XXXXX843]": 50079,
+  "[XXXXX844]": 50644,
+  "[XXXXX845]": 51033,
+  "[XXXXX846]": 50454,
+  "[XXXXX847]": 50095,
+  "[XXXXX848]": 50902,
+  "[XXXXX849]": 50597,
+  "[XXXXX84]": 50167,
+  "[XXXXX850]": 51051,
+  "[XXXXX851]": 50696,
+  "[XXXXX852]": 50966,
+  "[XXXXX853]": 50034,
+  "[XXXXX854]": 50889,
+  "[XXXXX855]": 50843,
+  "[XXXXX856]": 50963,
+  "[XXXXX857]": 50227,
+  "[XXXXX858]": 51058,
+  "[XXXXX859]": 50850,
+  "[XXXXX85]": 50192,
+  "[XXXXX860]": 50228,
+  "[XXXXX861]": 50570,
+  "[XXXXX862]": 50488,
+  "[XXXXX863]": 50539,
+  "[XXXXX864]": 51182,
+  "[XXXXX865]": 50990,
+  "[XXXXX866]": 50542,
+  "[XXXXX867]": 51069,
+  "[XXXXX868]": 50480,
+  "[XXXXX869]": 50847,
+  "[XXXXX86]": 50874,
+  "[XXXXX870]": 50308,
+  "[XXXXX871]": 50510,
+  "[XXXXX872]": 50423,
+  "[XXXXX873]": 50086,
+  "[XXXXX874]": 50591,
+  "[XXXXX875]": 50728,
+  "[XXXXX876]": 50372,
+  "[XXXXX877]": 50903,
+  "[XXXXX878]": 50868,
+  "[XXXXX879]": 50763,
+  "[XXXXX87]": 50571,
+  "[XXXXX880]": 50096,
+  "[XXXXX881]": 51150,
+  "[XXXXX882]": 50460,
+  "[XXXXX883]": 51087,
+  "[XXXXX884]": 51093,
+  "[XXXXX885]": 50295,
+  "[XXXXX886]": 51037,
+  "[XXXXX887]": 51171,
+  "[XXXXX888]": 50794,
+  "[XXXXX889]": 50628,
+  "[XXXXX88]": 50520,
+  "[XXXXX890]": 51172,
+  "[XXXXX891]": 51046,
+  "[XXXXX892]": 50918,
+  "[XXXXX893]": 50998,
+  "[XXXXX894]": 50371,
+  "[XXXXX895]": 51155,
+  "[XXXXX896]": 50866,
+  "[XXXXX897]": 50229,
+  "[XXXXX898]": 51057,
+  "[XXXXX899]": 50867,
+  "[XXXXX89]": 50537,
+  "[XXXXX8]": 50784,
+  "[XXXXX900]": 50739,
+  "[XXXXX901]": 51080,
+  "[XXXXX902]": 50038,
+  "[XXXXX903]": 50321,
+  "[XXXXX904]": 50774,
+  "[XXXXX905]": 50974,
+  "[XXXXX906]": 50920,
+  "[XXXXX907]": 50469,
+  "[XXXXX908]": 51122,
+  "[XXXXX909]": 51092,
+  "[XXXXX90]": 50514,
+  "[XXXXX910]": 50798,
+  "[XXXXX911]": 50245,
+  "[XXXXX912]": 50367,
+  "[XXXXX913]": 50618,
+  "[XXXXX914]": 50072,
+  "[XXXXX915]": 50450,
+  "[XXXXX916]": 51189,
+  "[XXXXX917]": 50485,
+  "[XXXXX918]": 50283,
+  "[XXXXX919]": 50845,
+  "[XXXXX91]": 50968,
+  "[XXXXX920]": 50429,
+  "[XXXXX921]": 50364,
+  "[XXXXX922]": 50782,
+  "[XXXXX923]": 50534,
+  "[XXXXX924]": 50059,
+  "[XXXXX925]": 50102,
+  "[XXXXX926]": 50185,
+  "[XXXXX927]": 50634,
+  "[XXXXX928]": 50567,
+  "[XXXXX929]": 50895,
+  "[XXXXX92]": 50210,
+  "[XXXXX930]": 50561,
+  "[XXXXX931]": 51021,
+  "[XXXXX932]": 50107,
+  "[XXXXX933]": 50407,
+  "[XXXXX934]": 50654,
+  "[XXXXX935]": 50268,
+  "[XXXXX936]": 51153,
+  "[XXXXX937]": 50552,
+  "[XXXXX938]": 50526,
+  "[XXXXX939]": 50322,
+  "[XXXXX93]": 50912,
+  "[XXXXX940]": 50296,
+  "[XXXXX941]": 50572,
+  "[XXXXX942]": 50393,
+  "[XXXXX943]": 50312,
+  "[XXXXX944]": 50592,
+  "[XXXXX945]": 50982,
+  "[XXXXX946]": 50089,
+  "[XXXXX947]": 50331,
+  "[XXXXX948]": 50279,
+  "[XXXXX949]": 51005,
+  "[XXXXX94]": 50198,
+  "[XXXXX950]": 50114,
+  "[XXXXX951]": 50074,
+  "[XXXXX952]": 50248,
+  "[XXXXX953]": 50857,
+  "[XXXXX954]": 50841,
+  "[XXXXX955]": 51065,
+  "[XXXXX956]": 50234,
+  "[XXXXX957]": 50834,
+  "[XXXXX958]": 50535,
+  "[XXXXX959]": 51109,
+  "[XXXXX95]": 50651,
+  "[XXXXX960]": 50797,
+  "[XXXXX961]": 50914,
+  "[XXXXX962]": 50112,
+  "[XXXXX963]": 50073,
+  "[XXXXX964]": 50559,
+  "[XXXXX965]": 50933,
+  "[XXXXX966]": 50619,
+  "[XXXXX967]": 51187,
+  "[XXXXX968]": 50327,
+  "[XXXXX969]": 50698,
+  "[XXXXX96]": 50420,
+  "[XXXXX970]": 50060,
+  "[XXXXX971]": 50432,
+  "[XXXXX972]": 50620,
+  "[XXXXX973]": 50693,
+  "[XXXXX974]": 50938,
+  "[XXXXX975]": 50983,
+  "[XXXXX976]": 50334,
+  "[XXXXX977]": 50438,
+  "[XXXXX978]": 51154,
+  "[XXXXX979]": 50964,
+  "[XXXXX97]": 50672,
+  "[XXXXX980]": 50511,
+  "[XXXXX981]": 50244,
+  "[XXXXX982]": 50266,
+  "[XXXXX983]": 50134,
+  "[XXXXX984]": 50338,
+  "[XXXXX985]": 51121,
+  "[XXXXX986]": 50323,
+  "[XXXXX987]": 50909,
+  "[XXXXX988]": 51000,
+  "[XXXXX989]": 50036,
+  "[XXXXX98]": 50787,
+  "[XXXXX990]": 50512,
+  "[XXXXX991]": 50191,
+  "[XXXXX992]": 50187,
+  "[XXXXX993]": 51156,
+  "[XXXXX994]": 51135,
+  "[XXXXX995]": 50355,
+  "[XXXXX996]": 51027,
+  "[XXXXX997]": 50297,
+  "[XXXXX998]": 50928,
+  "[XXXXX999]": 50697,
+  "[XXXXX99]": 50980,
+  "[XXXXX9]": 50961
+}

checkpoint-1500/config.json ADDED Viewed

	@@ -0,0 +1,34 @@

+{
+  "activation_function": "gelu",
+  "architectures": [
+    "GPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.1,
+  "bos_token_id": 50000,
+  "dtype": "bfloat16",
+  "embd_pdrop": 0.1,
+  "eos_token_id": 50001,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "gpt2",
+  "n_ctx": 512,
+  "n_embd": 512,
+  "n_head": 8,
+  "n_inner": 2048,
+  "n_layer": 4,
+  "n_positions": 512,
+  "pad_token_id": 50002,
+  "prefix": "[CLS]",
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.1,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "summary_activation": null,
+  "summary_first_dropout": 0.1,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "transformers_version": "4.56.2",
+  "use_cache": true,
+  "vocab_size": 51200
+}

checkpoint-1500/generation_config.json ADDED Viewed

	@@ -0,0 +1,9 @@

+{
+  "_from_model_config": true,
+  "bos_token_id": 50000,
+  "eos_token_id": [
+    50001
+  ],
+  "pad_token_id": 50002,
+  "transformers_version": "4.56.2"
+}

checkpoint-1500/model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:48abafdb399547f8362f07396bb9f33d74834b7d9dc787567487a0abf11891ff
+size 78179408

checkpoint-1500/rng_state.pth ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:367be88203ea56b50c3bf24db2568738de24c258b1f906b16ebbd965ba9c757d
+size 14645

checkpoint-1500/special_tokens_map.json ADDED Viewed

	@@ -0,0 +1,1249 @@

+{
+  "additional_special_tokens": [
+    "[XXXXX160]",
+    "[XXXXX240]",
+    "[XXXXX717]",
+    "[XXXXX23]",
+    "[XXXXX161]",
+    "[XXXXX1156]",
+    "[XXXXX5]",
+    "[XXXXX597]",
+    "[XXXXX415]",
+    "[XXXXX693]",
+    "[XXXXX1083]",
+    "[XXXXX784]",
+    "[XXXXX686]",
+    "[XXXXX379]",
+    "[XXXXX335]",
+    "[XXXXX73]",
+    "[XXXXX758]",
+    "[XXXXX805]",
+    "[XXXXX840]",
+    "[XXXXX1037]",
+    "[XXXXX235]",
+    "[XXXXX1107]",
+    "[XXXXX776]",
+    "[XXXXX286]",
+    "[XXXXX396]",
+    "[XXXXX478]",
+    "[XXXXX669]",
+    "[XXXXX691]",
+    "[XXXXX826]",
+    "[XXXXX1150]",
+    "[XXXXX853]",
+    "[XXXXX130]",
+    "[XXXXX989]",
+    "[XXXXX249]",
+    "[XXXXX902]",
+    "[XXXXX1169]",
+    "[XXXXX260]",
+    "[XXXXX706]",
+    "[XXXXX146]",
+    "[XXXXX198]",
+    "[XXXXX303]",
+    "[XXXXX1192]",
+    "[XXXXX635]",
+    "[XXXXX271]",
+    "[XXXXX1023]",
+    "[XXXXX602]",
+    "[XXXXX291]",
+    "[XXXXX190]",
+    "[XXXXX1055]",
+    "[XXXXX70]",
+    "[XXXXX178]",
+    "[XXXXX318]",
+    "[XXXXX62]",
+    "[XXXXX473]",
+    "[XXXXX517]",
+    "[XXXXX924]",
+    "[XXXXX970]",
+    "[XXXXX1033]",
+    "[XXXXX110]",
+    "[XXXXX1151]",
+    "[XXXXX1189]",
+    "[XXXXX1095]",
+    "[XXXXX148]",
+    "[XXXXX337]",
+    "[XXXXX245]",
+    "[XXXXX407]",
+    "[XXXXX557]",
+    "[XXXXX576]",
+    "[XXXXX914]",
+    "[XXXXX963]",
+    "[XXXXX951]",
+    "[XXXXX721]",
+    "[XXXXX1174]",
+    "[XXXXX3]",
+    "[XXXXX792]",
+    "[XXXXX843]",
+    "[XXXXX590]",
+    "[XXXXX118]",
+    "[XXXXX280]",
+    "[XXXXX1119]",
+    "[XXXXX1025]",
+    "[XXXXX140]",
+    "[XXXXX873]",
+    "[XXXXX103]",
+    "[XXXXX307]",
+    "[XXXXX946]",
+    "[XXXXX782]",
+    "[XXXXX100]",
+    "[XXXXX485]",
+    "[XXXXX151]",
+    "[XXXXX213]",
+    "[XXXXX847]",
+    "[XXXXX880]",
+    "[XXXXX44]",
+    "[XXXXX1031]",
+    "[XXXXX545]",
+    "[XXXXX78]",
+    "[XXXXX687]",
+    "[XXXXX925]",
+    "[XXXXX519]",
+    "[XXXXX1191]",
+    "[XXXXX13]",
+    "[XXXXX487]",
+    "[XXXXX932]",
+    "[XXXXX119]",
+    "[XXXXX170]",
+    "[XXXXX254]",
+    "[XXXXX420]",
+    "[XXXXX962]",
+    "[XXXXX441]",
+    "[XXXXX950]",
+    "[XXXXX18]",
+    "[XXXXX676]",
+    "[XXXXX1133]",
+    "[XXXXX618]",
+    "[XXXXX374]",
+    "[XXXXX815]",
+    "[XXXXX1058]",
+    "[XXXXX1108]",
+    "[XXXXX1144]",
+    "[XXXXX791]",
+    "[XXXXX259]",
+    "[XXXXX1074]",
+    "[XXXXX696]",
+    "[XXXXX1168]",
+    "[XXXXX592]",
+    "[XXXXX127]",
+    "[XXXXX136]",
+    "[XXXXX237]",
+    "[XXXXX705]",
+    "[XXXXX983]",
+    "[XXXXX816]",
+    "[XXXXX412]",
+    "[XXXXX410]",
+    "[XXXXX795]",
+    "[XXXXX101]",
+    "[XXXXX171]",
+    "[XXXXX628]",
+    "[XXXXX1038]",
+    "[XXXXX447]",
+    "[XXXXX1043]",
+    "[XXXXX147]",
+    "[XXXXX1112]",
+    "[XXXXX186]",
+    "[XXXXX461]",
+    "[XXXXX423]",
+    "[XXXXX26]",
+    "[XXXXX227]",
+    "[XXXXX746]",
+    "[XXXXX818]",
+    "[XXXXX725]",
+    "[XXXXX400]",
+    "[XXXXX827]",
+    "[XXXXX1020]",
+    "[XXXXX1060]",
+    "[XXXXX195]",
+    "[XXXXX526]",
+    "[XXXXX570]",
+    "[XXXXX174]",
+    "[XXXXX333]",
+    "[XXXXX466]",
+    "[XXXXX513]",
+    "[XXXXX695]",
+    "[XXXXX84]",
+    "[XXXXX1080]",
+    "[XXXXX76]",
+    "[XXXXX742]",
+    "[XXXXX1008]",
+    "[XXXXX1123]",
+    "[XXXXX1145]",
+    "[XXXXX701]",
+    "[XXXXX562]",
+    "[XXXXX615]",
+    "[XXXXX389]",
+    "[XXXXX267]",
+    "[XXXXX323]",
+    "[XXXXX346]",
+    "[XXXXX287]",
+    "[XXXXX469]",
+    "[XXXXX58]",
+    "[XXXXX431]",
+    "[XXXXX926]",
+    "[XXXXX1182]",
+    "[XXXXX992]",
+    "[XXXXX185]",
+    "[XXXXX352]",
+    "[XXXXX762]",
+    "[XXXXX991]",
+    "[XXXXX85]",
+    "[XXXXX836]",
+    "[XXXXX460]",
+    "[XXXXX272]",
+    "[XXXXX619]",
+    "[XXXXX759]",
+    "[XXXXX94]",
+    "[XXXXX807]",
+    "[XXXXX299]",
+    "[XXXXX187]",
+    "[XXXXX486]",
+    "[XXXXX45]",
+    "[XXXXX1036]",
+    "[XXXXX354]",
+    "[XXXXX386]",
+    "[XXXXX714]",
+    "[XXXXX769]",
+    "[XXXXX304]",
+    "[XXXXX92]",
+    "[XXXXX7]",
+    "[XXXXX31]",
+    "[XXXXX102]",
+    "[XXXXX163]",
+    "[XXXXX723]",
+    "[XXXXX1042]",
+    "[XXXXX452]",
+    "[XXXXX622]",
+    "[XXXXX1056]",
+    "[XXXXX822]",
+    "[XXXXX125]",
+    "[XXXXX617]",
+    "[XXXXX34]",
+    "[XXXXX712]",
+    "[XXXXX780]",
+    "[XXXXX547]",
+    "[XXXXX857]",
+    "[XXXXX860]",
+    "[XXXXX897]",
+    "[XXXXX663]",
+    "[XXXXX736]",
+    "[XXXXX202]",
+    "[XXXXX1029]",
+    "[XXXXX956]",
+    "[XXXXX397]",
+    "[XXXXX239]",
+    "[XXXXX472]",
+    "[XXXXX230]",
+    "[XXXXX796]",
+    "[XXXXX639]",
+    "[XXXXX203]",
+    "[XXXXX231]",
+    "[XXXXX652]",
+    "[XXXXX981]",
+    "[XXXXX911]",
+    "[XXXXX380]",
+    "[XXXXX477]",
+    "[XXXXX952]",
+    "[XXXXX28]",
+    "[XXXXX801]",
+    "[XXXXX1134]",
+    "[XXXXX1176]",
+    "[XXXXX180]",
+    "[XXXXX598]",
+    "[XXXXX365]",
+    "[XXXXX729]",
+    "[XXXXX214]",
+    "[XXXXX1117]",
+    "[XXXXX278]",
+    "[XXXXX522]",
+    "[XXXXX683]",
+    "[XXXXX731]",
+    "[XXXXX312]",
+    "[XXXXX1178]",
+    "[XXXXX666]",
+    "[XXXXX982]",
+    "[XXXXX414]",
+    "[XXXXX935]",
+    "[XXXXX1089]",
+    "[XXXXX236]",
+    "[XXXXX463]",
+    "[XXXXX52]",
+    "[XXXXX593]",
+    "[XXXXX508]",
+    "[XXXXX1125]",
+    "[XXXXX462]",
+    "[XXXXX32]",
+    "[XXXXX596]",
+    "[XXXXX948]",
+    "[XXXXX150]",
+    "[XXXXX322]",
+    "[XXXXX390]",
+    "[XXXXX918]",
+    "[XXXXX157]",
+    "[XXXXX310]",
+    "[XXXXX392]",
+    "[XXXXX649]",
+    "[XXXXX158]",
+    "[XXXXX430]",
+    "[XXXXX384]",
+    "[XXXXX760]",
+    "[XXXXX700]",
+    "[XXXXX817]",
+    "[XXXXX115]",
+    "[XXXXX885]",
+    "[XXXXX940]",
+    "[XXXXX997]",
+    "[XXXXX61]",
+    "[XXXXX209]",
+    "[XXXXX316]",
+    "[XXXXX537]",
+    "[XXXXX599]",
+    "[XXXXX680]",
+    "[XXXXX135]",
+    "[XXXXX39]",
+    "[XXXXX685]",
+    "[XXXXX806]",
+    "[XXXXX870]",
+    "[XXXXX1096]",
+    "[XXXXX366]",
+    "[XXXXX509]",
+    "[XXXXX943]",
+    "[XXXXX292]",
+    "[XXXXX367]",
+    "[XXXXX632]",
+    "[XXXXX1136]",
+    "[XXXXX156]",
+    "[XXXXX1100]",
+    "[XXXXX432]",
+    "[XXXXX514]",
+    "[XXXXX903]",
+    "[XXXXX939]",
+    "[XXXXX986]",
+    "[XXXXX506]",
+    "[XXXXX10]",
+    "[XXXXX573]",
+    "[XXXXX968]",
+    "[XXXXX1039]",
+    "[XXXXX252]",
+    "[XXXXX612]",
+    "[XXXXX947]",
+    "[XXXXX1162]",
+    "[XXXXX733]",
+    "[XXXXX976]",
+    "[XXXXX535]",
+    "[XXXXX710]",
+    "[XXXXX40]",
+    "[XXXXX984]",
+    "[XXXXX584]",
+    "[XXXXX132]",
+    "[XXXXX1084]",
+    "[XXXXX549]",
+    "[XXXXX810]",
+    "[XXXXX208]",
+    "[XXXXX609]",
+    "[XXXXX275]",
+    "[XXXXX262]",
+    "[XXXXX331]",
+    "[XXXXX361]",
+    "[XXXXX561]",
+    "[XXXXX143]",
+    "[XXXXX1061]",
+    "[XXXXX59]",
+    "[XXXXX21]",
+    "[XXXXX995]",
+    "[XXXXX439]",
+    "[XXXXX831]",
+    "[XXXXX745]",
+    "[XXXXX43]",
+    "[XXXXX498]",
+    "[XXXXX575]",
+    "[XXXXX624]",
+    "[XXXXX72]",
+    "[XXXXX921]",
+    "[XXXXX201]",
+    "[XXXXX1171]",
+    "[XXXXX912]",
+    "[XXXXX775]",
+    "[XXXXX1094]",
+    "[XXXXX80]",
+    "[XXXXX894]",
+    "[XXXXX876]",
+    "[XXXXX1143]",
+    "[XXXXX355]",
+    "[XXXXX450]",
+    "[XXXXX642]",
+    "[XXXXX162]",
+    "[XXXXX394]",
+    "[XXXXX296]",
+    "[XXXXX536]",
+    "[XXXXX560]",
+    "[XXXXX373]",
+    "[XXXXX674]",
+    "[XXXXX1099]",
+    "[XXXXX797]",
+    "[XXXXX516]",
+    "[XXXXX41]",
+    "[XXXXX594]",
+    "[XXXXX319]",
+    "[XXXXX165]",
+    "[XXXXX344]",
+    "[XXXXX1012]",
+    "[XXXXX942]",
+    "[XXXXX1071]",
+    "[XXXXX648]",
+    "[XXXXX4]",
+    "[XXXXX633]",
+    "[XXXXX551]",
+    "[XXXXX586]",
+    "[XXXXX578]",
+    "[XXXXX348]",
+    "[XXXXX614]",
+    "[XXXXX33]",
+    "[XXXXX1193]",
+    "[XXXXX737]",
+    "[XXXXX233]",
+    "[XXXXX933]",
+    "[XXXXX781]",
+    "[XXXXX189]",
+    "[XXXXX1006]",
+    "[XXXXX219]",
+    "[XXXXX454]",
+    "[XXXXX449]",
+    "[XXXXX697]",
+    "[XXXXX724]",
+    "[XXXXX224]",
+    "[XXXXX1157]",
+    "[XXXXX455]",
+    "[XXXXX505]",
+    "[XXXXX96]",
+    "[XXXXX182]",
+    "[XXXXX424]",
+    "[XXXXX872]",
+    "[XXXXX1163]",
+    "[XXXXX1081]",
+    "[XXXXX1173]",
+    "[XXXXX172]",
+    "[XXXXX507]",
+    "[XXXXX920]",
+    "[XXXXX1126]",
+    "[XXXXX474]",
+    "[XXXXX971]",
+    "[XXXXX1049]",
+    "[XXXXX1114]",
+    "[XXXXX1040]",
+    "[XXXXX79]",
+    "[XXXXX36]",
+    "[XXXXX977]",
+    "[XXXXX1129]",
+    "[XXXXX1063]",
+    "[XXXXX773]",
+    "[XXXXX16]",
+    "[XXXXX113]",
+    "[XXXXX320]",
+    "[XXXXX167]",
+    "[XXXXX751]",
+    "[XXXXX656]",
+    "[XXXXX543]",
+    "[XXXXX828]",
+    "[XXXXX915]",
+    "[XXXXX1098]",
+    "[XXXXX569]",
+    "[XXXXX1166]",
+    "[XXXXX846]",
+    "[XXXXX324]",
+    "[XXXXX802]",
+    "[XXXXX588]",
+    "[XXXXX739]",
+    "[XXXXX50]",
+    "[XXXXX882]",
+    "[XXXXX1054]",
+    "[XXXXX621]",
+    "[XXXXX585]",
+    "[XXXXX121]",
+    "[XXXXX647]",
+    "[XXXXX767]",
+    "[XXXXX645]",
+    "[XXXXX27]",
+    "[XXXXX907]",
+    "[XXXXX1109]",
+    "[XXXXX1147]",
+    "[XXXXX501]",
+    "[XXXXX1164]",
+    "[XXXXX650]",
+    "[XXXXX329]",
+    "[XXXXX659]",
+    "[XXXXX1158]",
+    "[XXXXX104]",
+    "[XXXXX629]",
+    "[XXXXX868]",
+    "[XXXXX779]",
+    "[XXXXX17]",
+    "[XXXXX377]",
+    "[XXXXX467]",
+    "[XXXXX917]",
+    "[XXXXX830]",
+    "[XXXXX533]",
+    "[XXXXX862]",
+    "[XXXXX1041]",
+    "[XXXXX542]",
+    "[XXXXX595]",
+    "[XXXXX353]",
+    "[XXXXX181]",
+    "[XXXXX662]",
+    "[XXXXX1195]",
+    "[XXXXX435]",
+    "[XXXXX1051]",
+    "[XXXXX703]",
+    "[XXXXX1140]",
+    "[XXXXX750]",
+    "[XXXXX57]",
+    "[XXXXX75]",
+    "[XXXXX293]",
+    "[XXXXX681]",
+    "[XXXXX46]",
+    "[XXXXX30]",
+    "[XXXXX402]",
+    "[XXXXX803]",
+    "[XXXXX821]",
+    "[XXXXX871]",
+    "[XXXXX980]",
+    "[XXXXX990]",
+    "[XXXXX626]",
+    "[XXXXX90]",
+    "[XXXXX77]",
+    "[XXXXX819]",
+    "[XXXXX436]",
+    "[XXXXX306]",
+    "[XXXXX434]",
+    "[XXXXX88]",
+    "[XXXXX184]",
+    "[XXXXX290]",
+    "[XXXXX216]",
+    "[XXXXX176]",
+    "[XXXXX747]",
+    "[XXXXX938]",
+    "[XXXXX442]",
+    "[XXXXX301]",
+    "[XXXXX1015]",
+    "[XXXXX520]",
+    "[XXXXX829]",
+    "[XXXXX106]",
+    "[XXXXX1048]",
+    "[XXXXX923]",
+    "[XXXXX958]",
+    "[XXXXX1130]",
+    "[XXXXX89]",
+    "[XXXXX548]",
+    "[XXXXX863]",
+    "[XXXXX444]",
+    "[XXXXX1149]",
+    "[XXXXX866]",
+    "[XXXXX109]",
+    "[XXXXX550]",
+    "[XXXXX727]",
+    "[XXXXX646]",
+    "[XXXXX702]",
+    "[XXXXX204]",
+    "[XXXXX1000]",
+    "[XXXXX591]",
+    "[XXXXX1128]",
+    "[XXXXX937]",
+    "[XXXXX141]",
+    "[XXXXX273]",
+    "[XXXXX1183]",
+    "[XXXXX765]",
+    "[XXXXX371]",
+    "[XXXXX256]",
+    "[XXXXX964]",
+    "[XXXXX1148]",
+    "[XXXXX930]",
+    "[XXXXX212]",
+    "[XXXXX133]",
+    "[XXXXX608]",
+    "[XXXXX281]",
+    "[XXXXX1045]",
+    "[XXXXX928]",
+    "[XXXXX1154]",
+    "[XXXXX334]",
+    "[XXXXX861]",
+    "[XXXXX87]",
+    "[XXXXX941]",
+    "[XXXXX359]",
+    "[XXXXX55]",
+    "[XXXXX842]",
+    "[XXXXX427]",
+    "[XXXXX491]",
+    "[XXXXX137]",
+    "[XXXXX47]",
+    "[XXXXX511]",
+    "[XXXXX81]",
+    "[XXXXX139]",
+    "[XXXXX226]",
+    "[XXXXX563]",
+    "[XXXXX708]",
+    "[XXXXX820]",
+    "[XXXXX65]",
+    "[XXXXX381]",
+    "[XXXXX22]",
+    "[XXXXX1064]",
+    "[XXXXX874]",
+    "[XXXXX944]",
+    "[XXXXX6]",
+    "[XXXXX313]",
+    "[XXXXX471]",
+    "[XXXXX1118]",
+    "[XXXXX849]",
+    "[XXXXX483]",
+    "[XXXXX713]",
+    "[XXXXX222]",
+    "[XXXXX640]",
+    "[XXXXX12]",
+    "[XXXXX285]",
+    "[XXXXX327]",
+    "[XXXXX378]",
+    "[XXXXX790]",
+    "[XXXXX1179]",
+    "[XXXXX15]",
+    "[XXXXX229]",
+    "[XXXXX771]",
+    "[XXXXX559]",
+    "[XXXXX253]",
+    "[XXXXX644]",
+    "[XXXXX811]",
+    "[XXXXX554]",
+    "[XXXXX744]",
+    "[XXXXX786]",
+    "[XXXXX913]",
+    "[XXXXX966]",
+    "[XXXXX972]",
+    "[XXXXX664]",
+    "[XXXXX1131]",
+    "[XXXXX1132]",
+    "[XXXXX376]",
+    "[XXXXX672]",
+    "[XXXXX1013]",
+    "[XXXXX328]",
+    "[XXXXX889]",
+    "[XXXXX1141]",
+    "[XXXXX429]",
+    "[XXXXX114]",
+    "[XXXXX523]",
+    "[XXXXX833]",
+    "[XXXXX927]",
+    "[XXXXX297]",
+    "[XXXXX812]",
+    "[XXXXX1062]",
+    "[XXXXX152]",
+    "[XXXXX1010]",
+    "[XXXXX690]",
+    "[XXXXX363]",
+    "[XXXXX1103]",
+    "[XXXXX194]",
+    "[XXXXX844]",
+    "[XXXXX667]",
+    "[XXXXX720]",
+    "[XXXXX770]",
+    "[XXXXX515]",
+    "[XXXXX196]",
+    "[XXXXX283]",
+    "[XXXXX95]",
+    "[XXXXX528]",
+    "[XXXXX749]",
+    "[XXXXX934]",
+    "[XXXXX1052]",
+    "[XXXXX1188]",
+    "[XXXXX1001]",
+    "[XXXXX785]",
+    "[XXXXX393]",
+    "[XXXXX502]",
+    "[XXXXX404]",
+    "[XXXXX129]",
+    "[XXXXX643]",
+    "[XXXXX1086]",
+    "[XXXXX1104]",
+    "[XXXXX326]",
+    "[XXXXX330]",
+    "[XXXXX620]",
+    "[XXXXX193]",
+    "[XXXXX470]",
+    "[XXXXX279]",
+    "[XXXXX97]",
+    "[XXXXX177]",
+    "[XXXXX510]",
+    "[XXXXX225]",
+    "[XXXXX572]",
+    "[XXXXX777]",
+    "[XXXXX1135]",
+    "[XXXXX1160]",
+    "[XXXXX558]",
+    "[XXXXX416]",
+    "[XXXXX246]",
+    "[XXXXX799]",
+    "[XXXXX153]",
+    "[XXXXX671]",
+    "[XXXXX809]",
+    "[XXXXX525]",
+    "[XXXXX1004]",
+    "[XXXXX1005]",
+    "[XXXXX1194]",
+    "[XXXXX408]",
+    "[XXXXX492]",
+    "[XXXXX973]",
+    "[XXXXX242]",
+    "[XXXXX357]",
+    "[XXXXX851]",
+    "[XXXXX999]",
+    "[XXXXX969]",
+    "[XXXXX453]",
+    "[XXXXX730]",
+    "[XXXXX54]",
+    "[XXXXX480]",
+    "[XXXXX564]",
+    "[XXXXX600]",
+    "[XXXXX654]",
+    "[XXXXX37]",
+    "[XXXXX605]",
+    "[XXXXX164]",
+    "[XXXXX395]",
+    "[XXXXX401]",
+    "[XXXXX446]",
+    "[XXXXX658]",
+    "[XXXXX60]",
+    "[XXXXX107]",
+    "[XXXXX131]",
+    "[XXXXX387]",
+    "[XXXXX67]",
+    "[XXXXX743]",
+    "[XXXXX657]",
+    "[XXXXX418]",
+    "[XXXXX787]",
+    "[XXXXX838]",
+    "[XXXXX298]",
+    "[XXXXX1007]",
+    "[XXXXX1027]",
+    "[XXXXX1078]",
+    "[XXXXX651]",
+    "[XXXXX875]",
+    "[XXXXX258]",
+    "[XXXXX413]",
+    "[XXXXX1092]",
+    "[XXXXX124]",
+    "[XXXXX308]",
+    "[XXXXX1085]",
+    "[XXXXX1170]",
+    "[XXXXX38]",
+    "[XXXXX438]",
+    "[XXXXX475]",
+    "[XXXXX900]",
+    "[XXXXX263]",
+    "[XXXXX166]",
+    "[XXXXX698]",
+    "[XXXXX1127]",
+    "[XXXXX1181]",
+    "[XXXXX763]",
+    "[XXXXX627]",
+    "[XXXXX250]",
+    "[XXXXX757]",
+    "[XXXXX142]",
+    "[XXXXX794]",
+    "[XXXXX128]",
+    "[XXXXX35]",
+    "[XXXXX482]",
+    "[XXXXX834]",
+    "[XXXXX679]",
+    "[XXXXX839]",
+    "[XXXXX138]",
+    "[XXXXX1066]",
+    "[XXXXX116]",
+    "[XXXXX317]",
+    "[XXXXX490]",
+    "[XXXXX443]",
+    "[XXXXX879]",
+    "[XXXXX1101]",
+    "[XXXXX711]",
+    "[XXXXX123]",
+    "[XXXXX1152]",
+    "[XXXXX692]",
+    "[XXXXX555]",
+    "[XXXXX266]",
+    "[XXXXX0]",
+    "[XXXXX332]",
+    "[XXXXX497]",
+    "[XXXXX904]",
+    "[XXXXX1019]",
+    "[XXXXX709]",
+    "[XXXXX351]",
+    "[XXXXX764]",
+    "[XXXXX625]",
+    "[XXXXX798]",
+    "[XXXXX205]",
+    "[XXXXX922]",
+    "[XXXXX1075]",
+    "[XXXXX8]",
+    "[XXXXX422]",
+    "[XXXXX638]",
+    "[XXXXX98]",
+    "[XXXXX814]",
+    "[XXXXX1057]",
+    "[XXXXX1070]",
+    "[XXXXX411]",
+    "[XXXXX768]",
+    "[XXXXX1165]",
+    "[XXXXX888]",
+    "[XXXXX636]",
+    "[XXXXX1047]",
+    "[XXXXX960]",
+    "[XXXXX910]",
+    "[XXXXX42]",
+    "[XXXXX238]",
+    "[XXXXX383]",
+    "[XXXXX1032]",
+    "[XXXXX577]",
+    "[XXXXX268]",
+    "[XXXXX179]",
+    "[XXXXX1076]",
+    "[XXXXX1167]",
+    "[XXXXX1172]",
+    "[XXXXX665]",
+    "[XXXXX126]",
+    "[XXXXX347]",
+    "[XXXXX544]",
+    "[XXXXX1026]",
+    "[XXXXX159]",
+    "[XXXXX503]",
+    "[XXXXX583]",
+    "[XXXXX668]",
+    "[XXXXX1115]",
+    "[XXXXX112]",
+    "[XXXXX154]",
+    "[XXXXX761]",
+    "[XXXXX540]",
+    "[XXXXX755]",
+    "[XXXXX255]",
+    "[XXXXX459]",
+    "[XXXXX1090]",
+    "[XXXXX689]",
+    "[XXXXX589]",
+    "[XXXXX122]",
+    "[XXXXX789]",
+    "[XXXXX481]",
+    "[XXXXX1044]",
+    "[XXXXX274]",
+    "[XXXXX957]",
+    "[XXXXX1121]",
+    "[XXXXX464]",
+    "[XXXXX495]",
+    "[XXXXX1097]",
+    "[XXXXX581]",
+    "[XXXXX200]",
+    "[XXXXX954]",
+    "[XXXXX48]",
+    "[XXXXX855]",
+    "[XXXXX419]",
+    "[XXXXX919]",
+    "[XXXXX451]",
+    "[XXXXX869]",
+    "[XXXXX1082]",
+    "[XXXXX1185]",
+    "[XXXXX859]",
+    "[XXXXX1139]",
+    "[XXXXX1072]",
+    "[XXXXX220]",
+    "[XXXXX465]",
+    "[XXXXX1105]",
+    "[XXXXX1093]",
+    "[XXXXX953]",
+    "[XXXXX1187]",
+    "[XXXXX311]",
+    "[XXXXX403]",
+    "[XXXXX684]",
+    "[XXXXX426]",
+    "[XXXXX68]",
+    "[XXXXX277]",
+    "[XXXXX437]",
+    "[XXXXX896]",
+    "[XXXXX899]",
+    "[XXXXX878]",
+    "[XXXXX1073]",
+    "[XXXXX774]",
+    "[XXXXX315]",
+    "[XXXXX631]",
+    "[XXXXX223]",
+    "[XXXXX86]",
+    "[XXXXX607]",
+    "[XXXXX149]",
+    "[XXXXX566]",
+    "[XXXXX740]",
+    "[XXXXX265]",
+    "[XXXXX399]",
+    "[XXXXX1053]",
+    "[XXXXX1077]",
+    "[XXXXX336]",
+    "[XXXXX734]",
+    "[XXXXX289]",
+    "[XXXXX221]",
+    "[XXXXX580]",
+    "[XXXXX406]",
+    "[XXXXX854]",
+    "[XXXXX673]",
+    "[XXXXX1146]",
+    "[XXXXX29]",
+    "[XXXXX496]",
+    "[XXXXX241]",
+    "[XXXXX929]",
+    "[XXXXX169]",
+    "[XXXXX1153]",
+    "[XXXXX440]",
+    "[XXXXX385]",
+    "[XXXXX499]",
+    "[XXXXX718]",
+    "[XXXXX848]",
+    "[XXXXX877]",
+    "[XXXXX641]",
+    "[XXXXX1137]",
+    "[XXXXX269]",
+    "[XXXXX521]",
+    "[XXXXX234]",
+    "[XXXXX987]",
+    "[XXXXX741]",
+    "[XXXXX349]",
+    "[XXXXX93]",
+    "[XXXXX247]",
+    "[XXXXX961]",
+    "[XXXXX1120]",
+    "[XXXXX369]",
+    "[XXXXX479]",
+    "[XXXXX892]",
+    "[XXXXX553]",
+    "[XXXXX906]",
+    "[XXXXX571]",
+    "[XXXXX1046]",
+    "[XXXXX206]",
+    "[XXXXX488]",
+    "[XXXXX445]",
+    "[XXXXX524]",
+    "[XXXXX228]",
+    "[XXXXX998]",
+    "[XXXXX675]",
+    "[XXXXX732]",
+    "[XXXXX1028]",
+    "[XXXXX494]",
+    "[XXXXX965]",
+    "[XXXXX1021]",
+    "[XXXXX726]",
+    "[XXXXX168]",
+    "[XXXXX606]",
+    "[XXXXX974]",
+    "[XXXXX504]",
+    "[XXXXX556]",
+    "[XXXXX1159]",
+    "[XXXXX417]",
+    "[XXXXX1175]",
+    "[XXXXX1142]",
+    "[XXXXX753]",
+    "[XXXXX824]",
+    "[XXXXX251]",
+    "[XXXXX345]",
+    "[XXXXX610]",
+    "[XXXXX276]",
+    "[XXXXX783]",
+    "[XXXXX74]",
+    "[XXXXX1050]",
+    "[XXXXX71]",
+    "[XXXXX360]",
+    "[XXXXX512]",
+    "[XXXXX341]",
+    "[XXXXX375]",
+    "[XXXXX637]",
+    "[XXXXX1116]",
+    "[XXXXX9]",
+    "[XXXXX738]",
+    "[XXXXX856]",
+    "[XXXXX979]",
+    "[XXXXX825]",
+    "[XXXXX852]",
+    "[XXXXX215]",
+    "[XXXXX91]",
+    "[XXXXX257]",
+    "[XXXXX325]",
+    "[XXXXX368]",
+    "[XXXXX1124]",
+    "[XXXXX188]",
+    "[XXXXX905]",
+    "[XXXXX284]",
+    "[XXXXX11]",
+    "[XXXXX83]",
+    "[XXXXX611]",
+    "[XXXXX51]",
+    "[XXXXX99]",
+    "[XXXXX288]",
+    "[XXXXX945]",
+    "[XXXXX975]",
+    "[XXXXX282]",
+    "[XXXXX1068]",
+    "[XXXXX1180]",
+    "[XXXXX574]",
+    "[XXXXX305]",
+    "[XXXXX565]",
+    "[XXXXX865]",
+    "[XXXXX1014]",
+    "[XXXXX1138]",
+    "[XXXXX134]",
+    "[XXXXX321]",
+    "[XXXXX752]",
+    "[XXXXX108]",
+    "[XXXXX339]",
+    "[XXXXX893]",
+    "[XXXXX1177]",
+    "[XXXXX988]",
+    "[XXXXX719]",
+    "[XXXXX1059]",
+    "[XXXXX421]",
+    "[XXXXX670]",
+    "[XXXXX949]",
+    "[XXXXX14]",
+    "[XXXXX1087]",
+    "[XXXXX382]",
+    "[XXXXX531]",
+    "[XXXXX1079]",
+    "[XXXXX1067]",
+    "[XXXXX655]",
+    "[XXXXX340]",
+    "[XXXXX398]",
+    "[XXXXX175]",
+    "[XXXXX232]",
+    "[XXXXX244]",
+    "[XXXXX568]",
+    "[XXXXX1110]",
+    "[XXXXX191]",
+    "[XXXXX931]",
+    "[XXXXX832]",
+    "[XXXXX1030]",
+    "[XXXXX144]",
+    "[XXXXX1102]",
+    "[XXXXX300]",
+    "[XXXXX996]",
+    "[XXXXX248]",
+    "[XXXXX66]",
+    "[XXXXX735]",
+    "[XXXXX20]",
+    "[XXXXX694]",
+    "[XXXXX845]",
+    "[XXXXX500]",
+    "[XXXXX613]",
+    "[XXXXX117]",
+    "[XXXXX886]",
+    "[XXXXX261]",
+    "[XXXXX567]",
+    "[XXXXX541]",
+    "[XXXXX529]",
+    "[XXXXX699]",
+    "[XXXXX715]",
+    "[XXXXX1003]",
+    "[XXXXX518]",
+    "[XXXXX891]",
+    "[XXXXX192]",
+    "[XXXXX53]",
+    "[XXXXX428]",
+    "[XXXXX183]",
+    "[XXXXX850]",
+    "[XXXXX539]",
+    "[XXXXX527]",
+    "[XXXXX660]",
+    "[XXXXX409]",
+    "[XXXXX1122]",
+    "[XXXXX898]",
+    "[XXXXX858]",
+    "[XXXXX534]",
+    "[XXXXX756]",
+    "[XXXXX63]",
+    "[XXXXX49]",
+    "[XXXXX1106]",
+    "[XXXXX111]",
+    "[XXXXX955]",
+    "[XXXXX309]",
+    "[XXXXX538]",
+    "[XXXXX823]",
+    "[XXXXX867]",
+    "[XXXXX587]",
+    "[XXXXX243]",
+    "[XXXXX677]",
+    "[XXXXX1017]",
+    "[XXXXX173]",
+    "[XXXXX682]",
+    "[XXXXX1184]",
+    "[XXXXX1009]",
+    "[XXXXX704]",
+    "[XXXXX532]",
+    "[XXXXX901]",
+    "[XXXXX120]",
+    "[XXXXX388]",
+    "[XXXXX405]",
+    "[XXXXX788]",
+    "[XXXXX728]",
+    "[XXXXX425]",
+    "[XXXXX883]",
+    "[XXXXX837]",
+    "[XXXXX350]",
+    "[XXXXX716]",
+    "[XXXXX579]",
+    "[XXXXX909]",
+    "[XXXXX884]",
+    "[XXXXX1011]",
+    "[XXXXX372]",
+    "[XXXXX370]",
+    "[XXXXX468]",
+    "[XXXXX688]",
+    "[XXXXX56]",
+    "[XXXXX653]",
+    "[XXXXX623]",
+    "[XXXXX552]",
+    "[XXXXX314]",
+    "[XXXXX1018]",
+    "[XXXXX1091]",
+    "[XXXXX342]",
+    "[XXXXX1065]",
+    "[XXXXX1161]",
+    "[XXXXX959]",
+    "[XXXXX1035]",
+    "[XXXXX530]",
+    "[XXXXX69]",
+    "[XXXXX489]",
+    "[XXXXX1]",
+    "[XXXXX207]",
+    "[XXXXX433]",
+    "[XXXXX358]",
+    "[XXXXX601]",
+    "[XXXXX2]",
+    "[XXXXX722]",
+    "[XXXXX985]",
+    "[XXXXX908]",
+    "[XXXXX1024]",
+    "[XXXXX1111]",
+    "[XXXXX1113]",
+    "[XXXXX678]",
+    "[XXXXX19]",
+    "[XXXXX1186]",
+    "[XXXXX808]",
+    "[XXXXX493]",
+    "[XXXXX1155]",
+    "[XXXXX217]",
+    "[XXXXX616]",
+    "[XXXXX835]",
+    "[XXXXX994]",
+    "[XXXXX484]",
+    "[XXXXX362]",
+    "[XXXXX582]",
+    "[XXXXX1034]",
+    "[XXXXX772]",
+    "[XXXXX457]",
+    "[XXXXX604]",
+    "[XXXXX456]",
+    "[XXXXX145]",
+    "[XXXXX476]",
+    "[XXXXX218]",
+    "[XXXXX199]",
+    "[XXXXX364]",
+    "[XXXXX302]",
+    "[XXXXX881]",
+    "[XXXXX1088]",
+    "[XXXXX356]",
+    "[XXXXX936]",
+    "[XXXXX978]",
+    "[XXXXX895]",
+    "[XXXXX993]",
+    "[XXXXX458]",
+    "[XXXXX1002]",
+    "[XXXXX1022]",
+    "[XXXXX25]",
+    "[XXXXX766]",
+    "[XXXXX64]",
+    "[XXXXX211]",
+    "[XXXXX82]",
+    "[XXXXX634]",
+    "[XXXXX661]",
+    "[XXXXX748]",
+    "[XXXXX778]",
+    "[XXXXX24]",
+    "[XXXXX448]",
+    "[XXXXX887]",
+    "[XXXXX890]",
+    "[XXXXX754]",
+    "[XXXXX343]",
+    "[XXXXX1069]",
+    "[XXXXX294]",
+    "[XXXXX295]",
+    "[XXXXX197]",
+    "[XXXXX210]",
+    "[XXXXX630]",
+    "[XXXXX707]",
+    "[XXXXX864]",
+    "[XXXXX391]",
+    "[XXXXX338]",
+    "[XXXXX800]",
+    "[XXXXX841]",
+    "[XXXXX967]",
+    "[XXXXX793]",
+    "[XXXXX916]",
+    "[XXXXX603]",
+    "[XXXXX270]",
+    "[XXXXX105]",
+    "[XXXXX1016]",
+    "[XXXXX804]",
+    "[XXXXX264]",
+    "[XXXXX155]",
+    "[XXXXX1190]",
+    "[XXXXX813]",
+    "[XXXXX546]"
+  ],
+  "bos_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-1500/spiece.model ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:521f1d3fcae0031363abcb3468b2ba4b5914798c61e596af42b34c9689bd3ed0
+size 1096235

checkpoint-1500/tokenizer_config.json ADDED Viewed

The diff for this file is too large to render. See raw diff

checkpoint-1500/trainer_state.json ADDED Viewed

	@@ -0,0 +1,3034 @@

+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.07715454054471106,
+  "eval_steps": 500,
+  "global_step": 1500,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 10.742102813720702,
+      "epoch": 0.0002571818018157035,
+      "grad_norm": 5.21875,
+      "learning_rate": 2e-06,
+      "loss": 10.7614,
+      "mean_token_accuracy": 8.620689623057842e-05,
+      "num_tokens": 9573.0,
+      "step": 5
+    },
+    {
+      "entropy": 10.742151069641114,
+      "epoch": 0.000514363603631407,
+      "grad_norm": 4.84375,
+      "learning_rate": 4.5e-06,
+      "loss": 10.7583,
+      "mean_token_accuracy": 0.0,
+      "num_tokens": 18933.0,
+      "step": 10
+    },
+    {
+      "entropy": 10.742130088806153,
+      "epoch": 0.0007715454054471106,
+      "grad_norm": 5.21875,
+      "learning_rate": 7e-06,
+      "loss": 10.7448,
+      "mean_token_accuracy": 0.0,
+      "num_tokens": 29028.0,
+      "step": 15
+    },
+    {
+      "entropy": 10.742134284973144,
+      "epoch": 0.001028727207262814,
+      "grad_norm": 4.9375,
+      "learning_rate": 9.5e-06,
+      "loss": 10.6881,
+      "mean_token_accuracy": 0.00012970168609172106,
+      "num_tokens": 37987.0,
+      "step": 20
+    },
+    {
+      "entropy": 10.742049884796142,
+      "epoch": 0.0012859090090785177,
+      "grad_norm": 4.4375,
+      "learning_rate": 1.2e-05,
+      "loss": 10.5852,
+      "mean_token_accuracy": 0.0029032987426035107,
+      "num_tokens": 47163.0,
+      "step": 25
+    },
+    {
+      "entropy": 10.741710948944093,
+      "epoch": 0.0015430908108942211,
+      "grad_norm": 4.25,
+      "learning_rate": 1.4500000000000002e-05,
+      "loss": 10.4809,
+      "mean_token_accuracy": 0.02479298785328865,
+      "num_tokens": 56096.0,
+      "step": 30
+    },
+    {
+      "entropy": 10.740671920776368,
+      "epoch": 0.0018002726127099246,
+      "grad_norm": 3.296875,
+      "learning_rate": 1.7000000000000003e-05,
+      "loss": 10.3641,
+      "mean_token_accuracy": 0.03960379287600517,
+      "num_tokens": 65823.0,
+      "step": 35
+    },
+    {
+      "entropy": 10.737626838684083,
+      "epoch": 0.002057454414525628,
+      "grad_norm": 2.578125,
+      "learning_rate": 1.95e-05,
+      "loss": 10.2147,
+      "mean_token_accuracy": 0.04078627172857523,
+      "num_tokens": 75005.0,
+      "step": 40
+    },
+    {
+      "entropy": 10.73199462890625,
+      "epoch": 0.0023146362163413317,
+      "grad_norm": 2.53125,
+      "learning_rate": 2.2e-05,
+      "loss": 10.1015,
+      "mean_token_accuracy": 0.04431910365819931,
+      "num_tokens": 83340.0,
+      "step": 45
+    },
+    {
+      "entropy": 10.72521743774414,
+      "epoch": 0.0025718180181570354,
+      "grad_norm": 2.140625,
+      "learning_rate": 2.4500000000000003e-05,
+      "loss": 10.0212,
+      "mean_token_accuracy": 0.049253365769982335,
+      "num_tokens": 92004.0,
+      "step": 50
+    },
+    {
+      "entropy": 10.719530010223389,
+      "epoch": 0.0028289998199727386,
+      "grad_norm": 2.078125,
+      "learning_rate": 2.7e-05,
+      "loss": 9.9673,
+      "mean_token_accuracy": 0.04715999849140644,
+      "num_tokens": 101196.0,
+      "step": 55
+    },
+    {
+      "entropy": 10.713758850097657,
+      "epoch": 0.0030861816217884423,
+      "grad_norm": 2.078125,
+      "learning_rate": 2.95e-05,
+      "loss": 9.8887,
+      "mean_token_accuracy": 0.04930844120681286,
+      "num_tokens": 109931.0,
+      "step": 60
+    },
+    {
+      "entropy": 10.709882545471192,
+      "epoch": 0.003343363423604146,
+      "grad_norm": 2.0625,
+      "learning_rate": 3.2e-05,
+      "loss": 9.8563,
+      "mean_token_accuracy": 0.046213630586862564,
+      "num_tokens": 118450.0,
+      "step": 65
+    },
+    {
+      "entropy": 10.70816535949707,
+      "epoch": 0.003600545225419849,
+      "grad_norm": 1.96875,
+      "learning_rate": 3.4500000000000005e-05,
+      "loss": 9.7843,
+      "mean_token_accuracy": 0.04638329595327377,
+      "num_tokens": 127311.0,
+      "step": 70
+    },
+    {
+      "entropy": 10.705178928375243,
+      "epoch": 0.003857727027235553,
+      "grad_norm": 1.875,
+      "learning_rate": 3.7e-05,
+      "loss": 9.7153,
+      "mean_token_accuracy": 0.05029761902987957,
+      "num_tokens": 136529.0,
+      "step": 75
+    },
+    {
+      "entropy": 10.696865940093994,
+      "epoch": 0.004114908829051256,
+      "grad_norm": 1.828125,
+      "learning_rate": 3.95e-05,
+      "loss": 9.6593,
+      "mean_token_accuracy": 0.050337107852101326,
+      "num_tokens": 145754.0,
+      "step": 80
+    },
+    {
+      "entropy": 10.685219669342041,
+      "epoch": 0.00437209063086696,
+      "grad_norm": 1.8046875,
+      "learning_rate": 4.2000000000000004e-05,
+      "loss": 9.6217,
+      "mean_token_accuracy": 0.04981501027941704,
+      "num_tokens": 155229.0,
+      "step": 85
+    },
+    {
+      "entropy": 10.669653797149659,
+      "epoch": 0.004629272432682663,
+      "grad_norm": 1.8984375,
+      "learning_rate": 4.45e-05,
+      "loss": 9.4894,
+      "mean_token_accuracy": 0.06370054408907891,
+      "num_tokens": 163766.0,
+      "step": 90
+    },
+    {
+      "entropy": 10.649745845794678,
+      "epoch": 0.004886454234498367,
+      "grad_norm": 1.7265625,
+      "learning_rate": 4.7000000000000004e-05,
+      "loss": 9.4341,
+      "mean_token_accuracy": 0.05732584930956364,
+      "num_tokens": 172309.0,
+      "step": 95
+    },
+    {
+      "entropy": 10.618613624572754,
+      "epoch": 0.005143636036314071,
+      "grad_norm": 1.7421875,
+      "learning_rate": 4.9500000000000004e-05,
+      "loss": 9.2855,
+      "mean_token_accuracy": 0.06327233165502548,
+      "num_tokens": 180242.0,
+      "step": 100
+    },
+    {
+      "entropy": 10.569914817810059,
+      "epoch": 0.0054008178381297735,
+      "grad_norm": 1.75,
+      "learning_rate": 5.2e-05,
+      "loss": 9.2324,
+      "mean_token_accuracy": 0.060527439787983896,
+      "num_tokens": 189044.0,
+      "step": 105
+    },
+    {
+      "entropy": 10.520080757141113,
+      "epoch": 0.005657999639945477,
+      "grad_norm": 1.5390625,
+      "learning_rate": 5.45e-05,
+      "loss": 9.1122,
+      "mean_token_accuracy": 0.06187737137079239,
+      "num_tokens": 197779.0,
+      "step": 110
+    },
+    {
+      "entropy": 10.468755054473878,
+      "epoch": 0.005915181441761181,
+      "grad_norm": 1.5859375,
+      "learning_rate": 5.7e-05,
+      "loss": 8.9561,
+      "mean_token_accuracy": 0.06379008032381535,
+      "num_tokens": 206025.0,
+      "step": 115
+    },
+    {
+      "entropy": 10.407035160064698,
+      "epoch": 0.0061723632435768845,
+      "grad_norm": 1.453125,
+      "learning_rate": 5.9499999999999996e-05,
+      "loss": 8.9663,
+      "mean_token_accuracy": 0.05617102049291134,
+      "num_tokens": 216620.0,
+      "step": 120
+    },
+    {
+      "entropy": 10.3653733253479,
+      "epoch": 0.006429545045392588,
+      "grad_norm": 1.375,
+      "learning_rate": 6.2e-05,
+      "loss": 8.7968,
+      "mean_token_accuracy": 0.059108608216047284,
+      "num_tokens": 225654.0,
+      "step": 125
+    },
+    {
+      "entropy": 10.301421070098877,
+      "epoch": 0.006686726847208292,
+      "grad_norm": 1.4609375,
+      "learning_rate": 6.450000000000001e-05,
+      "loss": 8.6036,
+      "mean_token_accuracy": 0.06164416745305061,
+      "num_tokens": 233967.0,
+      "step": 130
+    },
+    {
+      "entropy": 10.185540676116943,
+      "epoch": 0.006943908649023995,
+      "grad_norm": 1.4140625,
+      "learning_rate": 6.7e-05,
+      "loss": 8.5469,
+      "mean_token_accuracy": 0.06072752773761749,
+      "num_tokens": 241911.0,
+      "step": 135
+    },
+    {
+      "entropy": 10.0550705909729,
+      "epoch": 0.007201090450839698,
+      "grad_norm": 1.25,
+      "learning_rate": 6.950000000000001e-05,
+      "loss": 8.4323,
+      "mean_token_accuracy": 0.06439580023288727,
+      "num_tokens": 250771.0,
+      "step": 140
+    },
+    {
+      "entropy": 9.967238235473634,
+      "epoch": 0.007458272252655402,
+      "grad_norm": 1.1171875,
+      "learning_rate": 7.2e-05,
+      "loss": 8.3391,
+      "mean_token_accuracy": 0.06209088861942291,
+      "num_tokens": 260036.0,
+      "step": 145
+    },
+    {
+      "entropy": 9.832792282104492,
+      "epoch": 0.007715454054471106,
+      "grad_norm": 1.1875,
+      "learning_rate": 7.45e-05,
+      "loss": 8.2272,
+      "mean_token_accuracy": 0.06231529638171196,
+      "num_tokens": 268614.0,
+      "step": 150
+    },
+    {
+      "entropy": 9.642397022247314,
+      "epoch": 0.00797263585628681,
+      "grad_norm": 1.0546875,
+      "learning_rate": 7.7e-05,
+      "loss": 8.085,
+      "mean_token_accuracy": 0.06268828995525837,
+      "num_tokens": 278205.0,
+      "step": 155
+    },
+    {
+      "entropy": 9.471720123291016,
+      "epoch": 0.008229817658102512,
+      "grad_norm": 1.109375,
+      "learning_rate": 7.950000000000001e-05,
+      "loss": 8.0285,
+      "mean_token_accuracy": 0.054571619257330894,
+      "num_tokens": 287369.0,
+      "step": 160
+    },
+    {
+      "entropy": 9.277729606628418,
+      "epoch": 0.008486999459918217,
+      "grad_norm": 0.9765625,
+      "learning_rate": 8.2e-05,
+      "loss": 7.8389,
+      "mean_token_accuracy": 0.0658249743282795,
+      "num_tokens": 295469.0,
+      "step": 165
+    },
+    {
+      "entropy": 9.065653324127197,
+      "epoch": 0.00874418126173392,
+      "grad_norm": 0.875,
+      "learning_rate": 8.450000000000001e-05,
+      "loss": 7.8802,
+      "mean_token_accuracy": 0.05946259275078773,
+      "num_tokens": 304108.0,
+      "step": 170
+    },
+    {
+      "entropy": 8.9242995262146,
+      "epoch": 0.009001363063549624,
+      "grad_norm": 0.83203125,
+      "learning_rate": 8.7e-05,
+      "loss": 7.7549,
+      "mean_token_accuracy": 0.06102342195808887,
+      "num_tokens": 312797.0,
+      "step": 175
+    },
+    {
+      "entropy": 8.724368000030518,
+      "epoch": 0.009258544865365327,
+      "grad_norm": 0.89453125,
+      "learning_rate": 8.95e-05,
+      "loss": 7.726,
+      "mean_token_accuracy": 0.05993276461958885,
+      "num_tokens": 321947.0,
+      "step": 180
+    },
+    {
+      "entropy": 8.550490474700927,
+      "epoch": 0.00951572666718103,
+      "grad_norm": 0.859375,
+      "learning_rate": 9.2e-05,
+      "loss": 7.7268,
+      "mean_token_accuracy": 0.06107203662395477,
+      "num_tokens": 330881.0,
+      "step": 185
+    },
+    {
+      "entropy": 8.48734483718872,
+      "epoch": 0.009772908468996734,
+      "grad_norm": 0.8125,
+      "learning_rate": 9.45e-05,
+      "loss": 7.6738,
+      "mean_token_accuracy": 0.06392239518463612,
+      "num_tokens": 339615.0,
+      "step": 190
+    },
+    {
+      "entropy": 8.382297706604003,
+      "epoch": 0.010030090270812437,
+      "grad_norm": 0.9765625,
+      "learning_rate": 9.7e-05,
+      "loss": 7.5319,
+      "mean_token_accuracy": 0.0668424092233181,
+      "num_tokens": 347445.0,
+      "step": 195
+    },
+    {
+      "entropy": 8.23064022064209,
+      "epoch": 0.010287272072628141,
+      "grad_norm": 0.81640625,
+      "learning_rate": 9.95e-05,
+      "loss": 7.6527,
+      "mean_token_accuracy": 0.0642503272742033,
+      "num_tokens": 356466.0,
+      "step": 200
+    },
+    {
+      "entropy": 8.293260860443116,
+      "epoch": 0.010544453874443844,
+      "grad_norm": 0.890625,
+      "learning_rate": 0.000102,
+      "loss": 7.6586,
+      "mean_token_accuracy": 0.06924829706549644,
+      "num_tokens": 364582.0,
+      "step": 205
+    },
+    {
+      "entropy": 8.233483600616456,
+      "epoch": 0.010801635676259547,
+      "grad_norm": 0.87890625,
+      "learning_rate": 0.00010449999999999999,
+      "loss": 7.5999,
+      "mean_token_accuracy": 0.06699038669466972,
+      "num_tokens": 373816.0,
+      "step": 210
+    },
+    {
+      "entropy": 8.192768096923828,
+      "epoch": 0.011058817478075252,
+      "grad_norm": 0.96484375,
+      "learning_rate": 0.000107,
+      "loss": 7.6079,
+      "mean_token_accuracy": 0.06721281111240388,
+      "num_tokens": 382663.0,
+      "step": 215
+    },
+    {
+      "entropy": 8.15766954421997,
+      "epoch": 0.011315999279890954,
+      "grad_norm": 0.875,
+      "learning_rate": 0.0001095,
+      "loss": 7.5084,
+      "mean_token_accuracy": 0.06859233565628528,
+      "num_tokens": 391146.0,
+      "step": 220
+    },
+    {
+      "entropy": 8.103716373443604,
+      "epoch": 0.011573181081706659,
+      "grad_norm": 0.8125,
+      "learning_rate": 0.000112,
+      "loss": 7.5845,
+      "mean_token_accuracy": 0.06810487173497677,
+      "num_tokens": 400693.0,
+      "step": 225
+    },
+    {
+      "entropy": 8.105987453460694,
+      "epoch": 0.011830362883522362,
+      "grad_norm": 1.1171875,
+      "learning_rate": 0.0001145,
+      "loss": 7.539,
+      "mean_token_accuracy": 0.06601319462060928,
+      "num_tokens": 409706.0,
+      "step": 230
+    },
+    {
+      "entropy": 8.056668090820313,
+      "epoch": 0.012087544685338066,
+      "grad_norm": 0.9140625,
+      "learning_rate": 0.00011700000000000001,
+      "loss": 7.4696,
+      "mean_token_accuracy": 0.06857900470495223,
+      "num_tokens": 418717.0,
+      "step": 235
+    },
+    {
+      "entropy": 8.077108192443848,
+      "epoch": 0.012344726487153769,
+      "grad_norm": 1.046875,
+      "learning_rate": 0.00011949999999999999,
+      "loss": 7.5335,
+      "mean_token_accuracy": 0.06780378967523575,
+      "num_tokens": 428630.0,
+      "step": 240
+    },
+    {
+      "entropy": 8.065593910217284,
+      "epoch": 0.012601908288969472,
+      "grad_norm": 0.9609375,
+      "learning_rate": 0.000122,
+      "loss": 7.559,
+      "mean_token_accuracy": 0.06397245228290557,
+      "num_tokens": 438762.0,
+      "step": 245
+    },
+    {
+      "entropy": 8.016668844223023,
+      "epoch": 0.012859090090785176,
+      "grad_norm": 1.0546875,
+      "learning_rate": 0.0001245,
+      "loss": 7.4803,
+      "mean_token_accuracy": 0.06500645838677883,
+      "num_tokens": 447419.0,
+      "step": 250
+    },
+    {
+      "entropy": 8.066792869567871,
+      "epoch": 0.01311627189260088,
+      "grad_norm": 0.97265625,
+      "learning_rate": 0.000127,
+      "loss": 7.4872,
+      "mean_token_accuracy": 0.07560092583298683,
+      "num_tokens": 455818.0,
+      "step": 255
+    },
+    {
+      "entropy": 8.00608172416687,
+      "epoch": 0.013373453694416584,
+      "grad_norm": 1.0625,
+      "learning_rate": 0.0001295,
+      "loss": 7.4407,
+      "mean_token_accuracy": 0.0703579068183899,
+      "num_tokens": 464411.0,
+      "step": 260
+    },
+    {
+      "entropy": 8.006528282165528,
+      "epoch": 0.013630635496232286,
+      "grad_norm": 0.95703125,
+      "learning_rate": 0.000132,
+      "loss": 7.4791,
+      "mean_token_accuracy": 0.07310711704194546,
+      "num_tokens": 473181.0,
+      "step": 265
+    },
+    {
+      "entropy": 7.954403877258301,
+      "epoch": 0.01388781729804799,
+      "grad_norm": 0.890625,
+      "learning_rate": 0.00013450000000000002,
+      "loss": 7.3802,
+      "mean_token_accuracy": 0.07561531476676464,
+      "num_tokens": 481776.0,
+      "step": 270
+    },
+    {
+      "entropy": 7.960335302352905,
+      "epoch": 0.014144999099863694,
+      "grad_norm": 0.91796875,
+      "learning_rate": 0.00013700000000000002,
+      "loss": 7.3739,
+      "mean_token_accuracy": 0.07225163355469703,
+      "num_tokens": 490482.0,
+      "step": 275
+    },
+    {
+      "entropy": 7.908543682098388,
+      "epoch": 0.014402180901679397,
+      "grad_norm": 1.046875,
+      "learning_rate": 0.0001395,
+      "loss": 7.3373,
+      "mean_token_accuracy": 0.08000247925519943,
+      "num_tokens": 498394.0,
+      "step": 280
+    },
+    {
+      "entropy": 7.912304162979126,
+      "epoch": 0.014659362703495101,
+      "grad_norm": 1.3671875,
+      "learning_rate": 0.00014199999999999998,
+      "loss": 7.3847,
+      "mean_token_accuracy": 0.07234559133648873,
+      "num_tokens": 507194.0,
+      "step": 285
+    },
+    {
+      "entropy": 7.905929660797119,
+      "epoch": 0.014916544505310804,
+      "grad_norm": 1.0390625,
+      "learning_rate": 0.0001445,
+      "loss": 7.3824,
+      "mean_token_accuracy": 0.07954801470041276,
+      "num_tokens": 515538.0,
+      "step": 290
+    },
+    {
+      "entropy": 7.906702852249145,
+      "epoch": 0.015173726307126508,
+      "grad_norm": 0.98828125,
+      "learning_rate": 0.000147,
+      "loss": 7.4258,
+      "mean_token_accuracy": 0.0705490980297327,
+      "num_tokens": 524544.0,
+      "step": 295
+    },
+    {
+      "entropy": 7.956319189071655,
+      "epoch": 0.015430908108942211,
+      "grad_norm": 0.95703125,
+      "learning_rate": 0.0001495,
+      "loss": 7.3581,
+      "mean_token_accuracy": 0.0792141430079937,
+      "num_tokens": 533096.0,
+      "step": 300
+    },
+    {
+      "entropy": 7.908858060836792,
+      "epoch": 0.015688089910757916,
+      "grad_norm": 0.95703125,
+      "learning_rate": 0.000152,
+      "loss": 7.2829,
+      "mean_token_accuracy": 0.07599962502717972,
+      "num_tokens": 542254.0,
+      "step": 305
+    },
+    {
+      "entropy": 7.929060173034668,
+      "epoch": 0.01594527171257362,
+      "grad_norm": 0.97265625,
+      "learning_rate": 0.00015450000000000001,
+      "loss": 7.3804,
+      "mean_token_accuracy": 0.07816362082958221,
+      "num_tokens": 550985.0,
+      "step": 310
+    },
+    {
+      "entropy": 7.865027332305909,
+      "epoch": 0.01620245351438932,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.000157,
+      "loss": 7.3625,
+      "mean_token_accuracy": 0.07863678447902203,
+      "num_tokens": 560224.0,
+      "step": 315
+    },
+    {
+      "entropy": 7.856458759307861,
+      "epoch": 0.016459635316205024,
+      "grad_norm": 0.98828125,
+      "learning_rate": 0.0001595,
+      "loss": 7.2099,
+      "mean_token_accuracy": 0.0831373617053032,
+      "num_tokens": 569268.0,
+      "step": 320
+    },
+    {
+      "entropy": 7.8261981964111325,
+      "epoch": 0.01671681711802073,
+      "grad_norm": 1.03125,
+      "learning_rate": 0.000162,
+      "loss": 7.305,
+      "mean_token_accuracy": 0.07629284039139747,
+      "num_tokens": 577876.0,
+      "step": 325
+    },
+    {
+      "entropy": 7.825508499145508,
+      "epoch": 0.016973998919836433,
+      "grad_norm": 0.94921875,
+      "learning_rate": 0.00016450000000000001,
+      "loss": 7.1964,
+      "mean_token_accuracy": 0.08476228266954422,
+      "num_tokens": 586622.0,
+      "step": 330
+    },
+    {
+      "entropy": 7.87431845664978,
+      "epoch": 0.017231180721652136,
+      "grad_norm": 0.9609375,
+      "learning_rate": 0.00016700000000000002,
+      "loss": 7.3968,
+      "mean_token_accuracy": 0.07733883932232857,
+      "num_tokens": 596544.0,
+      "step": 335
+    },
+    {
+      "entropy": 7.786734390258789,
+      "epoch": 0.01748836252346784,
+      "grad_norm": 1.140625,
+      "learning_rate": 0.00016950000000000003,
+      "loss": 7.2342,
+      "mean_token_accuracy": 0.07911348566412926,
+      "num_tokens": 605349.0,
+      "step": 340
+    },
+    {
+      "entropy": 7.809148788452148,
+      "epoch": 0.01774554432528354,
+      "grad_norm": 0.98828125,
+      "learning_rate": 0.00017199999999999998,
+      "loss": 7.2457,
+      "mean_token_accuracy": 0.07915201708674431,
+      "num_tokens": 614266.0,
+      "step": 345
+    },
+    {
+      "entropy": 7.771593856811523,
+      "epoch": 0.018002726127099248,
+      "grad_norm": 1.0546875,
+      "learning_rate": 0.00017449999999999999,
+      "loss": 7.2829,
+      "mean_token_accuracy": 0.07773459404706955,
+      "num_tokens": 622534.0,
+      "step": 350
+    },
+    {
+      "entropy": 7.822271633148193,
+      "epoch": 0.01825990792891495,
+      "grad_norm": 1.0546875,
+      "learning_rate": 0.000177,
+      "loss": 7.2365,
+      "mean_token_accuracy": 0.08085938617587089,
+      "num_tokens": 631838.0,
+      "step": 355
+    },
+    {
+      "entropy": 7.698997259140015,
+      "epoch": 0.018517089730730654,
+      "grad_norm": 1.0859375,
+      "learning_rate": 0.0001795,
+      "loss": 7.2539,
+      "mean_token_accuracy": 0.08351444229483604,
+      "num_tokens": 641386.0,
+      "step": 360
+    },
+    {
+      "entropy": 7.703382873535157,
+      "epoch": 0.018774271532546356,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.000182,
+      "loss": 7.2384,
+      "mean_token_accuracy": 0.0824521966278553,
+      "num_tokens": 650465.0,
+      "step": 365
+    },
+    {
+      "entropy": 7.835447978973389,
+      "epoch": 0.01903145333436206,
+      "grad_norm": 1.0546875,
+      "learning_rate": 0.0001845,
+      "loss": 7.1401,
+      "mean_token_accuracy": 0.08636756986379623,
+      "num_tokens": 659509.0,
+      "step": 370
+    },
+    {
+      "entropy": 7.586853075027466,
+      "epoch": 0.019288635136177765,
+      "grad_norm": 1.109375,
+      "learning_rate": 0.000187,
+      "loss": 7.1678,
+      "mean_token_accuracy": 0.08730523958802223,
+      "num_tokens": 668137.0,
+      "step": 375
+    },
+    {
+      "entropy": 7.862004041671753,
+      "epoch": 0.019545816937993468,
+      "grad_norm": 1.078125,
+      "learning_rate": 0.0001895,
+      "loss": 7.2499,
+      "mean_token_accuracy": 0.0831152357161045,
+      "num_tokens": 677784.0,
+      "step": 380
+    },
+    {
+      "entropy": 7.637575674057007,
+      "epoch": 0.01980299873980917,
+      "grad_norm": 1.1953125,
+      "learning_rate": 0.000192,
+      "loss": 7.1701,
+      "mean_token_accuracy": 0.08491914793848991,
+      "num_tokens": 686088.0,
+      "step": 385
+    },
+    {
+      "entropy": 7.752926158905029,
+      "epoch": 0.020060180541624874,
+      "grad_norm": 1.0234375,
+      "learning_rate": 0.0001945,
+      "loss": 7.3326,
+      "mean_token_accuracy": 0.08027631081640721,
+      "num_tokens": 695131.0,
+      "step": 390
+    },
+    {
+      "entropy": 7.793869876861573,
+      "epoch": 0.020317362343440577,
+      "grad_norm": 1.0390625,
+      "learning_rate": 0.00019700000000000002,
+      "loss": 7.1152,
+      "mean_token_accuracy": 0.08675874844193458,
+      "num_tokens": 703976.0,
+      "step": 395
+    },
+    {
+      "entropy": 7.674239635467529,
+      "epoch": 0.020574544145256283,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.00019950000000000002,
+      "loss": 7.1607,
+      "mean_token_accuracy": 0.08738962039351464,
+      "num_tokens": 712080.0,
+      "step": 400
+    },
+    {
+      "entropy": 7.662798118591309,
+      "epoch": 0.020831725947071986,
+      "grad_norm": 1.109375,
+      "learning_rate": 0.000202,
+      "loss": 7.2073,
+      "mean_token_accuracy": 0.08529370948672295,
+      "num_tokens": 721151.0,
+      "step": 405
+    },
+    {
+      "entropy": 7.703192710876465,
+      "epoch": 0.02108890774888769,
+      "grad_norm": 1.03125,
+      "learning_rate": 0.00020449999999999998,
+      "loss": 7.2059,
+      "mean_token_accuracy": 0.08231882154941558,
+      "num_tokens": 730957.0,
+      "step": 410
+    },
+    {
+      "entropy": 7.6682192325592045,
+      "epoch": 0.02134608955070339,
+      "grad_norm": 1.0625,
+      "learning_rate": 0.000207,
+      "loss": 7.115,
+      "mean_token_accuracy": 0.09367254376411438,
+      "num_tokens": 740054.0,
+      "step": 415
+    },
+    {
+      "entropy": 7.710998058319092,
+      "epoch": 0.021603271352519094,
+      "grad_norm": 1.0390625,
+      "learning_rate": 0.0002095,
+      "loss": 7.1138,
+      "mean_token_accuracy": 0.0901214174926281,
+      "num_tokens": 748576.0,
+      "step": 420
+    },
+    {
+      "entropy": 7.615434312820435,
+      "epoch": 0.0218604531543348,
+      "grad_norm": 0.91015625,
+      "learning_rate": 0.000212,
+      "loss": 7.1257,
+      "mean_token_accuracy": 0.08843043595552444,
+      "num_tokens": 758078.0,
+      "step": 425
+    },
+    {
+      "entropy": 7.6622356414794925,
+      "epoch": 0.022117634956150503,
+      "grad_norm": 1.0390625,
+      "learning_rate": 0.0002145,
+      "loss": 7.1976,
+      "mean_token_accuracy": 0.08554179668426513,
+      "num_tokens": 767041.0,
+      "step": 430
+    },
+    {
+      "entropy": 7.6705575466156,
+      "epoch": 0.022374816757966206,
+      "grad_norm": 1.1328125,
+      "learning_rate": 0.00021700000000000002,
+      "loss": 7.192,
+      "mean_token_accuracy": 0.08901726603507995,
+      "num_tokens": 775675.0,
+      "step": 435
+    },
+    {
+      "entropy": 7.627669334411621,
+      "epoch": 0.02263199855978191,
+      "grad_norm": 1.1796875,
+      "learning_rate": 0.0002195,
+      "loss": 7.0133,
+      "mean_token_accuracy": 0.09209411665797233,
+      "num_tokens": 784105.0,
+      "step": 440
+    },
+    {
+      "entropy": 7.484321117401123,
+      "epoch": 0.022889180361597615,
+      "grad_norm": 1.03125,
+      "learning_rate": 0.000222,
+      "loss": 6.9712,
+      "mean_token_accuracy": 0.09117011949419976,
+      "num_tokens": 792861.0,
+      "step": 445
+    },
+    {
+      "entropy": 7.594349336624146,
+      "epoch": 0.023146362163413318,
+      "grad_norm": 1.0625,
+      "learning_rate": 0.0002245,
+      "loss": 7.0314,
+      "mean_token_accuracy": 0.08824475556612014,
+      "num_tokens": 801642.0,
+      "step": 450
+    },
+    {
+      "entropy": 7.497442388534546,
+      "epoch": 0.02340354396522902,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.00022700000000000002,
+      "loss": 6.9939,
+      "mean_token_accuracy": 0.08806813433766365,
+      "num_tokens": 811105.0,
+      "step": 455
+    },
+    {
+      "entropy": 7.556538200378418,
+      "epoch": 0.023660725767044723,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.00022950000000000002,
+      "loss": 7.0195,
+      "mean_token_accuracy": 0.09214109480381012,
+      "num_tokens": 819989.0,
+      "step": 460
+    },
+    {
+      "entropy": 7.516587781906128,
+      "epoch": 0.023917907568860426,
+      "grad_norm": 1.1953125,
+      "learning_rate": 0.00023200000000000003,
+      "loss": 7.0303,
+      "mean_token_accuracy": 0.0912808708846569,
+      "num_tokens": 829532.0,
+      "step": 465
+    },
+    {
+      "entropy": 7.524700498580932,
+      "epoch": 0.024175089370676132,
+      "grad_norm": 1.3046875,
+      "learning_rate": 0.00023449999999999998,
+      "loss": 6.9313,
+      "mean_token_accuracy": 0.09493043571710587,
+      "num_tokens": 838071.0,
+      "step": 470
+    },
+    {
+      "entropy": 7.6139366149902346,
+      "epoch": 0.024432271172491835,
+      "grad_norm": 1.03125,
+      "learning_rate": 0.000237,
+      "loss": 7.1399,
+      "mean_token_accuracy": 0.0876034751534462,
+      "num_tokens": 848094.0,
+      "step": 475
+    },
+    {
+      "entropy": 7.520268821716309,
+      "epoch": 0.024689452974307538,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.0002395,
+      "loss": 7.0461,
+      "mean_token_accuracy": 0.09003157168626785,
+      "num_tokens": 856585.0,
+      "step": 480
+    },
+    {
+      "entropy": 7.4084981918334964,
+      "epoch": 0.02494663477612324,
+      "grad_norm": 1.0,
+      "learning_rate": 0.000242,
+      "loss": 7.0064,
+      "mean_token_accuracy": 0.09524489566683769,
+      "num_tokens": 865866.0,
+      "step": 485
+    },
+    {
+      "entropy": 7.565576791763306,
+      "epoch": 0.025203816577938944,
+      "grad_norm": 1.390625,
+      "learning_rate": 0.0002445,
+      "loss": 7.0857,
+      "mean_token_accuracy": 0.09243087470531464,
+      "num_tokens": 874963.0,
+      "step": 490
+    },
+    {
+      "entropy": 7.522153425216675,
+      "epoch": 0.02546099837975465,
+      "grad_norm": 1.0859375,
+      "learning_rate": 0.000247,
+      "loss": 7.0253,
+      "mean_token_accuracy": 0.09747636616230011,
+      "num_tokens": 883670.0,
+      "step": 495
+    },
+    {
+      "entropy": 7.411280775070191,
+      "epoch": 0.025718180181570353,
+      "grad_norm": 1.234375,
+      "learning_rate": 0.0002495,
+      "loss": 6.9456,
+      "mean_token_accuracy": 0.09364836364984512,
+      "num_tokens": 892322.0,
+      "step": 500
+    },
+    {
+      "entropy": 7.502190637588501,
+      "epoch": 0.025975361983386056,
+      "grad_norm": 0.98828125,
+      "learning_rate": 0.000252,
+      "loss": 7.0,
+      "mean_token_accuracy": 0.09010454788804054,
+      "num_tokens": 901450.0,
+      "step": 505
+    },
+    {
+      "entropy": 7.450657367706299,
+      "epoch": 0.02623254378520176,
+      "grad_norm": 1.0703125,
+      "learning_rate": 0.0002545,
+      "loss": 6.9258,
+      "mean_token_accuracy": 0.0917640596628189,
+      "num_tokens": 910499.0,
+      "step": 510
+    },
+    {
+      "entropy": 7.4553405284881595,
+      "epoch": 0.02648972558701746,
+      "grad_norm": 0.9921875,
+      "learning_rate": 0.000257,
+      "loss": 7.042,
+      "mean_token_accuracy": 0.08597794771194459,
+      "num_tokens": 920223.0,
+      "step": 515
+    },
+    {
+      "entropy": 7.50772066116333,
+      "epoch": 0.026746907388833167,
+      "grad_norm": 1.2265625,
+      "learning_rate": 0.0002595,
+      "loss": 7.014,
+      "mean_token_accuracy": 0.08768303692340851,
+      "num_tokens": 929414.0,
+      "step": 520
+    },
+    {
+      "entropy": 7.4105640888214115,
+      "epoch": 0.02700408919064887,
+      "grad_norm": 1.28125,
+      "learning_rate": 0.000262,
+      "loss": 6.98,
+      "mean_token_accuracy": 0.08690723404288292,
+      "num_tokens": 938059.0,
+      "step": 525
+    },
+    {
+      "entropy": 7.362936401367188,
+      "epoch": 0.027261270992464573,
+      "grad_norm": 1.1875,
+      "learning_rate": 0.00026450000000000003,
+      "loss": 7.0251,
+      "mean_token_accuracy": 0.08964922651648521,
+      "num_tokens": 946868.0,
+      "step": 530
+    },
+    {
+      "entropy": 7.468600177764893,
+      "epoch": 0.027518452794280276,
+      "grad_norm": 1.0703125,
+      "learning_rate": 0.00026700000000000004,
+      "loss": 6.867,
+      "mean_token_accuracy": 0.09712589457631111,
+      "num_tokens": 954997.0,
+      "step": 535
+    },
+    {
+      "entropy": 7.428448724746704,
+      "epoch": 0.02777563459609598,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.00026950000000000005,
+      "loss": 6.9816,
+      "mean_token_accuracy": 0.09053401798009872,
+      "num_tokens": 964488.0,
+      "step": 540
+    },
+    {
+      "entropy": 7.317347574234009,
+      "epoch": 0.028032816397911685,
+      "grad_norm": 1.1015625,
+      "learning_rate": 0.00027200000000000005,
+      "loss": 6.8074,
+      "mean_token_accuracy": 0.10062685832381249,
+      "num_tokens": 972901.0,
+      "step": 545
+    },
+    {
+      "entropy": 7.45094141960144,
+      "epoch": 0.028289998199727388,
+      "grad_norm": 1.0,
+      "learning_rate": 0.0002745,
+      "loss": 7.0093,
+      "mean_token_accuracy": 0.09124675542116165,
+      "num_tokens": 981922.0,
+      "step": 550
+    },
+    {
+      "entropy": 7.441890525817871,
+      "epoch": 0.02854718000154309,
+      "grad_norm": 1.1953125,
+      "learning_rate": 0.000277,
+      "loss": 6.9774,
+      "mean_token_accuracy": 0.09369566291570663,
+      "num_tokens": 990419.0,
+      "step": 555
+    },
+    {
+      "entropy": 7.304813051223755,
+      "epoch": 0.028804361803358793,
+      "grad_norm": 1.125,
+      "learning_rate": 0.0002795,
+      "loss": 6.8424,
+      "mean_token_accuracy": 0.09960516914725304,
+      "num_tokens": 998236.0,
+      "step": 560
+    },
+    {
+      "entropy": 7.3862227439880375,
+      "epoch": 0.0290615436051745,
+      "grad_norm": 1.015625,
+      "learning_rate": 0.00028199999999999997,
+      "loss": 6.8905,
+      "mean_token_accuracy": 0.09915700182318687,
+      "num_tokens": 1008258.0,
+      "step": 565
+    },
+    {
+      "entropy": 7.355383062362671,
+      "epoch": 0.029318725406990202,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.0002845,
+      "loss": 6.8729,
+      "mean_token_accuracy": 0.09238902553915977,
+      "num_tokens": 1017029.0,
+      "step": 570
+    },
+    {
+      "entropy": 7.315412712097168,
+      "epoch": 0.029575907208805905,
+      "grad_norm": 1.1640625,
+      "learning_rate": 0.000287,
+      "loss": 6.9067,
+      "mean_token_accuracy": 0.09177370145916938,
+      "num_tokens": 1026497.0,
+      "step": 575
+    },
+    {
+      "entropy": 7.337841367721557,
+      "epoch": 0.029833089010621608,
+      "grad_norm": 1.234375,
+      "learning_rate": 0.0002895,
+      "loss": 6.8092,
+      "mean_token_accuracy": 0.10336045101284981,
+      "num_tokens": 1034728.0,
+      "step": 580
+    },
+    {
+      "entropy": 7.256781911849975,
+      "epoch": 0.03009027081243731,
+      "grad_norm": 1.1953125,
+      "learning_rate": 0.000292,
+      "loss": 6.8645,
+      "mean_token_accuracy": 0.09807764887809753,
+      "num_tokens": 1043556.0,
+      "step": 585
+    },
+    {
+      "entropy": 7.315465259552002,
+      "epoch": 0.030347452614253017,
+      "grad_norm": 1.25,
+      "learning_rate": 0.0002945,
+      "loss": 6.7914,
+      "mean_token_accuracy": 0.10462642684578896,
+      "num_tokens": 1053449.0,
+      "step": 590
+    },
+    {
+      "entropy": 7.294723844528198,
+      "epoch": 0.03060463441606872,
+      "grad_norm": 1.25,
+      "learning_rate": 0.000297,
+      "loss": 6.8701,
+      "mean_token_accuracy": 0.09261535704135895,
+      "num_tokens": 1062691.0,
+      "step": 595
+    },
+    {
+      "entropy": 7.328425025939941,
+      "epoch": 0.030861816217884423,
+      "grad_norm": 1.0703125,
+      "learning_rate": 0.0002995,
+      "loss": 6.8732,
+      "mean_token_accuracy": 0.09672143906354905,
+      "num_tokens": 1072194.0,
+      "step": 600
+    },
+    {
+      "entropy": 7.280484533309936,
+      "epoch": 0.031118998019700125,
+      "grad_norm": 1.109375,
+      "learning_rate": 0.000302,
+      "loss": 6.7832,
+      "mean_token_accuracy": 0.09872276112437248,
+      "num_tokens": 1080781.0,
+      "step": 605
+    },
+    {
+      "entropy": 7.2788474559783936,
+      "epoch": 0.03137617982151583,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.0003045,
+      "loss": 6.9191,
+      "mean_token_accuracy": 0.09485599771142006,
+      "num_tokens": 1090048.0,
+      "step": 610
+    },
+    {
+      "entropy": 7.34011402130127,
+      "epoch": 0.03163336162333153,
+      "grad_norm": 1.03125,
+      "learning_rate": 0.000307,
+      "loss": 6.9368,
+      "mean_token_accuracy": 0.09523111060261727,
+      "num_tokens": 1100121.0,
+      "step": 615
+    },
+    {
+      "entropy": 7.326197099685669,
+      "epoch": 0.03189054342514724,
+      "grad_norm": 1.1171875,
+      "learning_rate": 0.0003095,
+      "loss": 6.8036,
+      "mean_token_accuracy": 0.09568134695291519,
+      "num_tokens": 1108569.0,
+      "step": 620
+    },
+    {
+      "entropy": 7.213407182693482,
+      "epoch": 0.032147725226962943,
+      "grad_norm": 1.171875,
+      "learning_rate": 0.000312,
+      "loss": 6.776,
+      "mean_token_accuracy": 0.09914244189858437,
+      "num_tokens": 1118247.0,
+      "step": 625
+    },
+    {
+      "entropy": 7.210325479507446,
+      "epoch": 0.03240490702877864,
+      "grad_norm": 1.1875,
+      "learning_rate": 0.0003145,
+      "loss": 6.7999,
+      "mean_token_accuracy": 0.09887302219867707,
+      "num_tokens": 1126951.0,
+      "step": 630
+    },
+    {
+      "entropy": 7.209123611450195,
+      "epoch": 0.03266208883059435,
+      "grad_norm": 1.1953125,
+      "learning_rate": 0.000317,
+      "loss": 6.7651,
+      "mean_token_accuracy": 0.10773184821009636,
+      "num_tokens": 1135390.0,
+      "step": 635
+    },
+    {
+      "entropy": 7.2418413162231445,
+      "epoch": 0.03291927063241005,
+      "grad_norm": 1.171875,
+      "learning_rate": 0.0003195,
+      "loss": 6.8057,
+      "mean_token_accuracy": 0.09772193506360054,
+      "num_tokens": 1143911.0,
+      "step": 640
+    },
+    {
+      "entropy": 7.288219833374024,
+      "epoch": 0.033176452434225755,
+      "grad_norm": 1.2265625,
+      "learning_rate": 0.000322,
+      "loss": 6.7833,
+      "mean_token_accuracy": 0.10312128961086273,
+      "num_tokens": 1152957.0,
+      "step": 645
+    },
+    {
+      "entropy": 7.144729995727539,
+      "epoch": 0.03343363423604146,
+      "grad_norm": 0.96484375,
+      "learning_rate": 0.00032450000000000003,
+      "loss": 6.7073,
+      "mean_token_accuracy": 0.10490344762802124,
+      "num_tokens": 1161911.0,
+      "step": 650
+    },
+    {
+      "entropy": 7.214617156982422,
+      "epoch": 0.03369081603785716,
+      "grad_norm": 1.203125,
+      "learning_rate": 0.00032700000000000003,
+      "loss": 6.7621,
+      "mean_token_accuracy": 0.10395861864089966,
+      "num_tokens": 1170932.0,
+      "step": 655
+    },
+    {
+      "entropy": 7.226040458679199,
+      "epoch": 0.03394799783967287,
+      "grad_norm": 1.015625,
+      "learning_rate": 0.00032950000000000004,
+      "loss": 6.8979,
+      "mean_token_accuracy": 0.09301602616906166,
+      "num_tokens": 1180417.0,
+      "step": 660
+    },
+    {
+      "entropy": 7.148654270172119,
+      "epoch": 0.034205179641488566,
+      "grad_norm": 1.1328125,
+      "learning_rate": 0.00033200000000000005,
+      "loss": 6.7629,
+      "mean_token_accuracy": 0.09860587194561958,
+      "num_tokens": 1189153.0,
+      "step": 665
+    },
+    {
+      "entropy": 7.214554643630981,
+      "epoch": 0.03446236144330427,
+      "grad_norm": 1.2109375,
+      "learning_rate": 0.00033450000000000005,
+      "loss": 6.7476,
+      "mean_token_accuracy": 0.1043582171201706,
+      "num_tokens": 1197903.0,
+      "step": 670
+    },
+    {
+      "entropy": 7.1135680198669435,
+      "epoch": 0.03471954324511998,
+      "grad_norm": 1.046875,
+      "learning_rate": 0.000337,
+      "loss": 6.7435,
+      "mean_token_accuracy": 0.10275317579507828,
+      "num_tokens": 1207310.0,
+      "step": 675
+    },
+    {
+      "entropy": 7.197217321395874,
+      "epoch": 0.03497672504693568,
+      "grad_norm": 1.1953125,
+      "learning_rate": 0.0003395,
+      "loss": 6.7117,
+      "mean_token_accuracy": 0.10364129021763802,
+      "num_tokens": 1215577.0,
+      "step": 680
+    },
+    {
+      "entropy": 7.160997724533081,
+      "epoch": 0.035233906848751384,
+      "grad_norm": 1.2578125,
+      "learning_rate": 0.000342,
+      "loss": 6.801,
+      "mean_token_accuracy": 0.10323198661208152,
+      "num_tokens": 1223906.0,
+      "step": 685
+    },
+    {
+      "entropy": 7.1171308040618895,
+      "epoch": 0.03549108865056708,
+      "grad_norm": 1.1015625,
+      "learning_rate": 0.00034449999999999997,
+      "loss": 6.7434,
+      "mean_token_accuracy": 0.10217863768339157,
+      "num_tokens": 1232689.0,
+      "step": 690
+    },
+    {
+      "entropy": 7.243052577972412,
+      "epoch": 0.03574827045238279,
+      "grad_norm": 1.1953125,
+      "learning_rate": 0.000347,
+      "loss": 6.777,
+      "mean_token_accuracy": 0.10172305777668952,
+      "num_tokens": 1241079.0,
+      "step": 695
+    },
+    {
+      "entropy": 7.0958874225616455,
+      "epoch": 0.036005452254198496,
+      "grad_norm": 1.125,
+      "learning_rate": 0.0003495,
+      "loss": 6.6994,
+      "mean_token_accuracy": 0.10428043976426124,
+      "num_tokens": 1249540.0,
+      "step": 700
+    },
+    {
+      "entropy": 7.236631059646607,
+      "epoch": 0.036262634056014195,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.000352,
+      "loss": 6.767,
+      "mean_token_accuracy": 0.10423633679747582,
+      "num_tokens": 1257823.0,
+      "step": 705
+    },
+    {
+      "entropy": 7.162586879730225,
+      "epoch": 0.0365198158578299,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.0003545,
+      "loss": 6.7752,
+      "mean_token_accuracy": 0.10328136906027793,
+      "num_tokens": 1267242.0,
+      "step": 710
+    },
+    {
+      "entropy": 7.095451688766479,
+      "epoch": 0.0367769976596456,
+      "grad_norm": 1.1015625,
+      "learning_rate": 0.000357,
+      "loss": 6.6616,
+      "mean_token_accuracy": 0.10539852529764175,
+      "num_tokens": 1275942.0,
+      "step": 715
+    },
+    {
+      "entropy": 7.087832736968994,
+      "epoch": 0.03703417946146131,
+      "grad_norm": 1.1015625,
+      "learning_rate": 0.0003595,
+      "loss": 6.716,
+      "mean_token_accuracy": 0.10562722086906433,
+      "num_tokens": 1284840.0,
+      "step": 720
+    },
+    {
+      "entropy": 7.123103380203247,
+      "epoch": 0.03729136126327701,
+      "grad_norm": 1.1953125,
+      "learning_rate": 0.000362,
+      "loss": 6.6864,
+      "mean_token_accuracy": 0.10260284543037415,
+      "num_tokens": 1293592.0,
+      "step": 725
+    },
+    {
+      "entropy": 7.171104860305786,
+      "epoch": 0.03754854306509271,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.0003645,
+      "loss": 6.8006,
+      "mean_token_accuracy": 0.10065393671393394,
+      "num_tokens": 1301903.0,
+      "step": 730
+    },
+    {
+      "entropy": 7.110880374908447,
+      "epoch": 0.03780572486690842,
+      "grad_norm": 1.265625,
+      "learning_rate": 0.000367,
+      "loss": 6.6262,
+      "mean_token_accuracy": 0.105719206482172,
+      "num_tokens": 1310230.0,
+      "step": 735
+    },
+    {
+      "entropy": 7.178726625442505,
+      "epoch": 0.03806290666872412,
+      "grad_norm": 1.140625,
+      "learning_rate": 0.0003695,
+      "loss": 6.7676,
+      "mean_token_accuracy": 0.10490798503160477,
+      "num_tokens": 1320029.0,
+      "step": 740
+    },
+    {
+      "entropy": 7.001174449920654,
+      "epoch": 0.038320088470539825,
+      "grad_norm": 1.0703125,
+      "learning_rate": 0.000372,
+      "loss": 6.6133,
+      "mean_token_accuracy": 0.10703234896063804,
+      "num_tokens": 1329155.0,
+      "step": 745
+    },
+    {
+      "entropy": 7.125448083877563,
+      "epoch": 0.03857727027235553,
+      "grad_norm": 1.1484375,
+      "learning_rate": 0.0003745,
+      "loss": 6.8029,
+      "mean_token_accuracy": 0.1019430361688137,
+      "num_tokens": 1337958.0,
+      "step": 750
+    },
+    {
+      "entropy": 7.109465599060059,
+      "epoch": 0.03883445207417123,
+      "grad_norm": 1.3125,
+      "learning_rate": 0.000377,
+      "loss": 6.7375,
+      "mean_token_accuracy": 0.09966553151607513,
+      "num_tokens": 1347105.0,
+      "step": 755
+    },
+    {
+      "entropy": 7.161317777633667,
+      "epoch": 0.039091633875986936,
+      "grad_norm": 1.234375,
+      "learning_rate": 0.0003795,
+      "loss": 6.6791,
+      "mean_token_accuracy": 0.10060147866606713,
+      "num_tokens": 1356161.0,
+      "step": 760
+    },
+    {
+      "entropy": 6.9970433712005615,
+      "epoch": 0.039348815677802636,
+      "grad_norm": 1.140625,
+      "learning_rate": 0.000382,
+      "loss": 6.6891,
+      "mean_token_accuracy": 0.10932381451129913,
+      "num_tokens": 1365291.0,
+      "step": 765
+    },
+    {
+      "entropy": 7.0630707263946535,
+      "epoch": 0.03960599747961834,
+      "grad_norm": 1.2578125,
+      "learning_rate": 0.0003845,
+      "loss": 6.6481,
+      "mean_token_accuracy": 0.1059026412665844,
+      "num_tokens": 1374057.0,
+      "step": 770
+    },
+    {
+      "entropy": 7.118219041824341,
+      "epoch": 0.03986317928143405,
+      "grad_norm": 1.296875,
+      "learning_rate": 0.00038700000000000003,
+      "loss": 6.6895,
+      "mean_token_accuracy": 0.09991606175899506,
+      "num_tokens": 1383204.0,
+      "step": 775
+    },
+    {
+      "entropy": 7.0793726444244385,
+      "epoch": 0.04012036108324975,
+      "grad_norm": 1.2421875,
+      "learning_rate": 0.00038950000000000003,
+      "loss": 6.7114,
+      "mean_token_accuracy": 0.1018392451107502,
+      "num_tokens": 1392303.0,
+      "step": 780
+    },
+    {
+      "entropy": 7.041213703155518,
+      "epoch": 0.040377542885065454,
+      "grad_norm": 1.0546875,
+      "learning_rate": 0.00039200000000000004,
+      "loss": 6.6789,
+      "mean_token_accuracy": 0.10870900377631187,
+      "num_tokens": 1401955.0,
+      "step": 785
+    },
+    {
+      "entropy": 7.127471685409546,
+      "epoch": 0.04063472468688115,
+      "grad_norm": 1.1796875,
+      "learning_rate": 0.00039450000000000005,
+      "loss": 6.6934,
+      "mean_token_accuracy": 0.10036104544997215,
+      "num_tokens": 1410467.0,
+      "step": 790
+    },
+    {
+      "entropy": 6.975720405578613,
+      "epoch": 0.04089190648869686,
+      "grad_norm": 1.0625,
+      "learning_rate": 0.00039700000000000005,
+      "loss": 6.6748,
+      "mean_token_accuracy": 0.09731148779392243,
+      "num_tokens": 1420337.0,
+      "step": 795
+    },
+    {
+      "entropy": 7.059706926345825,
+      "epoch": 0.041149088290512566,
+      "grad_norm": 1.078125,
+      "learning_rate": 0.0003995,
+      "loss": 6.6711,
+      "mean_token_accuracy": 0.10144198536872864,
+      "num_tokens": 1429453.0,
+      "step": 800
+    },
+    {
+      "entropy": 7.094644546508789,
+      "epoch": 0.041406270092328265,
+      "grad_norm": 1.1171875,
+      "learning_rate": 0.000402,
+      "loss": 6.6208,
+      "mean_token_accuracy": 0.10604123547673225,
+      "num_tokens": 1438417.0,
+      "step": 805
+    },
+    {
+      "entropy": 6.986465740203857,
+      "epoch": 0.04166345189414397,
+      "grad_norm": 1.40625,
+      "learning_rate": 0.0004045,
+      "loss": 6.6667,
+      "mean_token_accuracy": 0.10473878160119057,
+      "num_tokens": 1446480.0,
+      "step": 810
+    },
+    {
+      "entropy": 6.946633005142212,
+      "epoch": 0.04192063369595967,
+      "grad_norm": 1.1171875,
+      "learning_rate": 0.00040699999999999997,
+      "loss": 6.6306,
+      "mean_token_accuracy": 0.10496868267655372,
+      "num_tokens": 1455102.0,
+      "step": 815
+    },
+    {
+      "entropy": 7.14404935836792,
+      "epoch": 0.04217781549777538,
+      "grad_norm": 1.1015625,
+      "learning_rate": 0.0004095,
+      "loss": 6.6538,
+      "mean_token_accuracy": 0.10508396551012993,
+      "num_tokens": 1463839.0,
+      "step": 820
+    },
+    {
+      "entropy": 7.024435377120971,
+      "epoch": 0.04243499729959108,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.000412,
+      "loss": 6.6836,
+      "mean_token_accuracy": 0.10788170769810676,
+      "num_tokens": 1472806.0,
+      "step": 825
+    },
+    {
+      "entropy": 6.979895305633545,
+      "epoch": 0.04269217910140678,
+      "grad_norm": 1.1015625,
+      "learning_rate": 0.0004145,
+      "loss": 6.6016,
+      "mean_token_accuracy": 0.10319365486502648,
+      "num_tokens": 1481707.0,
+      "step": 830
+    },
+    {
+      "entropy": 6.948284530639649,
+      "epoch": 0.04294936090322249,
+      "grad_norm": 1.1796875,
+      "learning_rate": 0.000417,
+      "loss": 6.6068,
+      "mean_token_accuracy": 0.10952205657958984,
+      "num_tokens": 1491057.0,
+      "step": 835
+    },
+    {
+      "entropy": 6.963069152832031,
+      "epoch": 0.04320654270503819,
+      "grad_norm": 1.0234375,
+      "learning_rate": 0.0004195,
+      "loss": 6.5666,
+      "mean_token_accuracy": 0.10177317038178443,
+      "num_tokens": 1500331.0,
+      "step": 840
+    },
+    {
+      "entropy": 6.964898347854614,
+      "epoch": 0.043463724506853894,
+      "grad_norm": 1.0859375,
+      "learning_rate": 0.000422,
+      "loss": 6.6376,
+      "mean_token_accuracy": 0.10519776120781898,
+      "num_tokens": 1509051.0,
+      "step": 845
+    },
+    {
+      "entropy": 7.032815408706665,
+      "epoch": 0.0437209063086696,
+      "grad_norm": 1.2421875,
+      "learning_rate": 0.0004245,
+      "loss": 6.5931,
+      "mean_token_accuracy": 0.10714114159345627,
+      "num_tokens": 1517082.0,
+      "step": 850
+    },
+    {
+      "entropy": 6.869009304046631,
+      "epoch": 0.0439780881104853,
+      "grad_norm": 1.203125,
+      "learning_rate": 0.000427,
+      "loss": 6.5338,
+      "mean_token_accuracy": 0.11201286613941193,
+      "num_tokens": 1525498.0,
+      "step": 855
+    },
+    {
+      "entropy": 6.827907609939575,
+      "epoch": 0.044235269912301006,
+      "grad_norm": 1.140625,
+      "learning_rate": 0.0004295,
+      "loss": 6.4978,
+      "mean_token_accuracy": 0.11273056790232658,
+      "num_tokens": 1534476.0,
+      "step": 860
+    },
+    {
+      "entropy": 6.958985805511475,
+      "epoch": 0.04449245171411671,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.000432,
+      "loss": 6.5896,
+      "mean_token_accuracy": 0.10771357268095016,
+      "num_tokens": 1542905.0,
+      "step": 865
+    },
+    {
+      "entropy": 6.942825794219971,
+      "epoch": 0.04474963351593241,
+      "grad_norm": 1.109375,
+      "learning_rate": 0.0004345,
+      "loss": 6.583,
+      "mean_token_accuracy": 0.10546278730034828,
+      "num_tokens": 1551763.0,
+      "step": 870
+    },
+    {
+      "entropy": 6.906496191024781,
+      "epoch": 0.04500681531774812,
+      "grad_norm": 1.109375,
+      "learning_rate": 0.000437,
+      "loss": 6.5085,
+      "mean_token_accuracy": 0.11341241300106049,
+      "num_tokens": 1560520.0,
+      "step": 875
+    },
+    {
+      "entropy": 6.904255485534668,
+      "epoch": 0.04526399711956382,
+      "grad_norm": 1.0625,
+      "learning_rate": 0.0004395,
+      "loss": 6.4871,
+      "mean_token_accuracy": 0.11152468100190163,
+      "num_tokens": 1569401.0,
+      "step": 880
+    },
+    {
+      "entropy": 6.922533941268921,
+      "epoch": 0.045521178921379524,
+      "grad_norm": 1.046875,
+      "learning_rate": 0.000442,
+      "loss": 6.5564,
+      "mean_token_accuracy": 0.10919757708907127,
+      "num_tokens": 1578440.0,
+      "step": 885
+    },
+    {
+      "entropy": 6.789845991134643,
+      "epoch": 0.04577836072319523,
+      "grad_norm": 1.234375,
+      "learning_rate": 0.0004445,
+      "loss": 6.4462,
+      "mean_token_accuracy": 0.11144870221614837,
+      "num_tokens": 1587551.0,
+      "step": 890
+    },
+    {
+      "entropy": 6.944276809692383,
+      "epoch": 0.04603554252501093,
+      "grad_norm": 1.1875,
+      "learning_rate": 0.000447,
+      "loss": 6.6154,
+      "mean_token_accuracy": 0.10818377807736397,
+      "num_tokens": 1596314.0,
+      "step": 895
+    },
+    {
+      "entropy": 6.900253582000732,
+      "epoch": 0.046292724326826636,
+      "grad_norm": 1.1953125,
+      "learning_rate": 0.00044950000000000003,
+      "loss": 6.4889,
+      "mean_token_accuracy": 0.1112966388463974,
+      "num_tokens": 1605025.0,
+      "step": 900
+    },
+    {
+      "entropy": 6.905750894546509,
+      "epoch": 0.046549906128642335,
+      "grad_norm": 1.2265625,
+      "learning_rate": 0.00045200000000000004,
+      "loss": 6.5348,
+      "mean_token_accuracy": 0.10826415568590164,
+      "num_tokens": 1613295.0,
+      "step": 905
+    },
+    {
+      "entropy": 6.862349224090576,
+      "epoch": 0.04680708793045804,
+      "grad_norm": 1.328125,
+      "learning_rate": 0.00045450000000000004,
+      "loss": 6.5131,
+      "mean_token_accuracy": 0.1080511450767517,
+      "num_tokens": 1621865.0,
+      "step": 910
+    },
+    {
+      "entropy": 6.87268099784851,
+      "epoch": 0.04706426973227375,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.00045700000000000005,
+      "loss": 6.517,
+      "mean_token_accuracy": 0.11334388554096222,
+      "num_tokens": 1631036.0,
+      "step": 915
+    },
+    {
+      "entropy": 6.960796642303467,
+      "epoch": 0.04732145153408945,
+      "grad_norm": 1.1640625,
+      "learning_rate": 0.00045950000000000006,
+      "loss": 6.6083,
+      "mean_token_accuracy": 0.10900305211544037,
+      "num_tokens": 1640311.0,
+      "step": 920
+    },
+    {
+      "entropy": 6.877505731582642,
+      "epoch": 0.04757863333590515,
+      "grad_norm": 1.2578125,
+      "learning_rate": 0.000462,
+      "loss": 6.5707,
+      "mean_token_accuracy": 0.11122839301824569,
+      "num_tokens": 1648651.0,
+      "step": 925
+    },
+    {
+      "entropy": 6.814219570159912,
+      "epoch": 0.04783581513772085,
+      "grad_norm": 1.2109375,
+      "learning_rate": 0.0004645,
+      "loss": 6.5091,
+      "mean_token_accuracy": 0.11311314180493355,
+      "num_tokens": 1657493.0,
+      "step": 930
+    },
+    {
+      "entropy": 6.919810724258423,
+      "epoch": 0.04809299693953656,
+      "grad_norm": 1.203125,
+      "learning_rate": 0.000467,
+      "loss": 6.5672,
+      "mean_token_accuracy": 0.10320627093315124,
+      "num_tokens": 1666909.0,
+      "step": 935
+    },
+    {
+      "entropy": 6.835229587554932,
+      "epoch": 0.048350178741352265,
+      "grad_norm": 1.109375,
+      "learning_rate": 0.0004695,
+      "loss": 6.5857,
+      "mean_token_accuracy": 0.10411025136709214,
+      "num_tokens": 1676071.0,
+      "step": 940
+    },
+    {
+      "entropy": 6.990052223205566,
+      "epoch": 0.048607360543167964,
+      "grad_norm": 1.1484375,
+      "learning_rate": 0.000472,
+      "loss": 6.6305,
+      "mean_token_accuracy": 0.10452140420675278,
+      "num_tokens": 1685240.0,
+      "step": 945
+    },
+    {
+      "entropy": 6.916949987411499,
+      "epoch": 0.04886454234498367,
+      "grad_norm": 1.078125,
+      "learning_rate": 0.0004745,
+      "loss": 6.5346,
+      "mean_token_accuracy": 0.117493636906147,
+      "num_tokens": 1694441.0,
+      "step": 950
+    },
+    {
+      "entropy": 6.721475076675415,
+      "epoch": 0.04912172414679937,
+      "grad_norm": 1.171875,
+      "learning_rate": 0.000477,
+      "loss": 6.4578,
+      "mean_token_accuracy": 0.1129125975072384,
+      "num_tokens": 1702925.0,
+      "step": 955
+    },
+    {
+      "entropy": 6.895248603820801,
+      "epoch": 0.049378905948615076,
+      "grad_norm": 1.125,
+      "learning_rate": 0.0004795,
+      "loss": 6.4835,
+      "mean_token_accuracy": 0.10699952915310859,
+      "num_tokens": 1711708.0,
+      "step": 960
+    },
+    {
+      "entropy": 6.799575614929199,
+      "epoch": 0.04963608775043078,
+      "grad_norm": 1.21875,
+      "learning_rate": 0.000482,
+      "loss": 6.4901,
+      "mean_token_accuracy": 0.10597067773342132,
+      "num_tokens": 1720543.0,
+      "step": 965
+    },
+    {
+      "entropy": 6.862289619445801,
+      "epoch": 0.04989326955224648,
+      "grad_norm": 1.171875,
+      "learning_rate": 0.0004845,
+      "loss": 6.4988,
+      "mean_token_accuracy": 0.11304299309849739,
+      "num_tokens": 1729776.0,
+      "step": 970
+    },
+    {
+      "entropy": 6.8804943561553955,
+      "epoch": 0.05015045135406219,
+      "grad_norm": 1.171875,
+      "learning_rate": 0.000487,
+      "loss": 6.4822,
+      "mean_token_accuracy": 0.11637242883443832,
+      "num_tokens": 1738249.0,
+      "step": 975
+    },
+    {
+      "entropy": 6.828695917129517,
+      "epoch": 0.05040763315587789,
+      "grad_norm": 1.171875,
+      "learning_rate": 0.0004895,
+      "loss": 6.5063,
+      "mean_token_accuracy": 0.11184739843010902,
+      "num_tokens": 1747262.0,
+      "step": 980
+    },
+    {
+      "entropy": 6.891150426864624,
+      "epoch": 0.050664814957693594,
+      "grad_norm": 1.2890625,
+      "learning_rate": 0.000492,
+      "loss": 6.5072,
+      "mean_token_accuracy": 0.11530979126691818,
+      "num_tokens": 1755320.0,
+      "step": 985
+    },
+    {
+      "entropy": 6.7496922492980955,
+      "epoch": 0.0509219967595093,
+      "grad_norm": 1.078125,
+      "learning_rate": 0.0004945,
+      "loss": 6.5038,
+      "mean_token_accuracy": 0.11211839094758033,
+      "num_tokens": 1763798.0,
+      "step": 990
+    },
+    {
+      "entropy": 6.788990116119384,
+      "epoch": 0.051179178561325,
+      "grad_norm": 1.1484375,
+      "learning_rate": 0.000497,
+      "loss": 6.4671,
+      "mean_token_accuracy": 0.11327328234910965,
+      "num_tokens": 1773862.0,
+      "step": 995
+    },
+    {
+      "entropy": 6.837374210357666,
+      "epoch": 0.051436360363140705,
+      "grad_norm": 1.0078125,
+      "learning_rate": 0.0004995,
+      "loss": 6.5248,
+      "mean_token_accuracy": 0.11146914437413216,
+      "num_tokens": 1783552.0,
+      "step": 1000
+    },
+    {
+      "entropy": 6.746343898773193,
+      "epoch": 0.051693542164956405,
+      "grad_norm": 1.03125,
+      "learning_rate": 0.000499998026082006,
+      "loss": 6.426,
+      "mean_token_accuracy": 0.11493942067027092,
+      "num_tokens": 1792245.0,
+      "step": 1005
+    },
+    {
+      "entropy": 6.774056434631348,
+      "epoch": 0.05195072396677211,
+      "grad_norm": 1.140625,
+      "learning_rate": 0.0004999900070995136,
+      "loss": 6.5069,
+      "mean_token_accuracy": 0.11212562620639802,
+      "num_tokens": 1800813.0,
+      "step": 1010
+    },
+    {
+      "entropy": 6.852898216247558,
+      "epoch": 0.05220790576858782,
+      "grad_norm": 1.0625,
+      "learning_rate": 0.0004999758199023239,
+      "loss": 6.5207,
+      "mean_token_accuracy": 0.11174971088767052,
+      "num_tokens": 1809671.0,
+      "step": 1015
+    },
+    {
+      "entropy": 6.741954612731933,
+      "epoch": 0.05246508757040352,
+      "grad_norm": 1.1015625,
+      "learning_rate": 0.0004999554648793858,
+      "loss": 6.4463,
+      "mean_token_accuracy": 0.11097860932350159,
+      "num_tokens": 1817968.0,
+      "step": 1020
+    },
+    {
+      "entropy": 6.815918350219727,
+      "epoch": 0.05272226937221922,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.0004999289425887425,
+      "loss": 6.529,
+      "mean_token_accuracy": 0.10783923268318177,
+      "num_tokens": 1826302.0,
+      "step": 1025
+    },
+    {
+      "entropy": 6.708252668380737,
+      "epoch": 0.05297945117403492,
+      "grad_norm": 1.109375,
+      "learning_rate": 0.0004998962537575161,
+      "loss": 6.4088,
+      "mean_token_accuracy": 0.11643101274967194,
+      "num_tokens": 1836008.0,
+      "step": 1030
+    },
+    {
+      "entropy": 6.737126874923706,
+      "epoch": 0.05323663297585063,
+      "grad_norm": 1.1484375,
+      "learning_rate": 0.0004998573992818874,
+      "loss": 6.4282,
+      "mean_token_accuracy": 0.11566728353500366,
+      "num_tokens": 1844730.0,
+      "step": 1035
+    },
+    {
+      "entropy": 6.724181604385376,
+      "epoch": 0.053493814777666335,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.0004998123802270715,
+      "loss": 6.402,
+      "mean_token_accuracy": 0.11093823388218879,
+      "num_tokens": 1853186.0,
+      "step": 1040
+    },
+    {
+      "entropy": 6.778470802307129,
+      "epoch": 0.053750996579482034,
+      "grad_norm": 1.21875,
+      "learning_rate": 0.0004997611978272886,
+      "loss": 6.376,
+      "mean_token_accuracy": 0.1124820165336132,
+      "num_tokens": 1861579.0,
+      "step": 1045
+    },
+    {
+      "entropy": 6.778669786453247,
+      "epoch": 0.05400817838129774,
+      "grad_norm": 1.1328125,
+      "learning_rate": 0.0004997038534857298,
+      "loss": 6.5295,
+      "mean_token_accuracy": 0.1051288291811943,
+      "num_tokens": 1871948.0,
+      "step": 1050
+    },
+    {
+      "entropy": 6.779291105270386,
+      "epoch": 0.05426536018311344,
+      "grad_norm": 1.140625,
+      "learning_rate": 0.0004996403487745194,
+      "loss": 6.4291,
+      "mean_token_accuracy": 0.11674032881855964,
+      "num_tokens": 1880142.0,
+      "step": 1055
+    },
+    {
+      "entropy": 6.749887180328369,
+      "epoch": 0.054522541984929146,
+      "grad_norm": 1.078125,
+      "learning_rate": 0.000499570685434671,
+      "loss": 6.4415,
+      "mean_token_accuracy": 0.10835661590099335,
+      "num_tokens": 1888929.0,
+      "step": 1060
+    },
+    {
+      "entropy": 6.763294887542725,
+      "epoch": 0.05477972378674485,
+      "grad_norm": 1.1796875,
+      "learning_rate": 0.0004994948653760405,
+      "loss": 6.4847,
+      "mean_token_accuracy": 0.10644253864884376,
+      "num_tokens": 1897982.0,
+      "step": 1065
+    },
+    {
+      "entropy": 6.66923336982727,
+      "epoch": 0.05503690558856055,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.0004994128906772729,
+      "loss": 6.3776,
+      "mean_token_accuracy": 0.11694736555218696,
+      "num_tokens": 1906583.0,
+      "step": 1070
+    },
+    {
+      "entropy": 6.752908086776733,
+      "epoch": 0.05529408739037626,
+      "grad_norm": 1.1328125,
+      "learning_rate": 0.000499324763585746,
+      "loss": 6.4433,
+      "mean_token_accuracy": 0.11435103714466095,
+      "num_tokens": 1915457.0,
+      "step": 1075
+    },
+    {
+      "entropy": 6.66996021270752,
+      "epoch": 0.05555126919219196,
+      "grad_norm": 1.0546875,
+      "learning_rate": 0.0004992304865175085,
+      "loss": 6.3785,
+      "mean_token_accuracy": 0.12019199803471566,
+      "num_tokens": 1923802.0,
+      "step": 1080
+    },
+    {
+      "entropy": 6.735610485076904,
+      "epoch": 0.05580845099400766,
+      "grad_norm": 1.0234375,
+      "learning_rate": 0.0004991300620572138,
+      "loss": 6.3917,
+      "mean_token_accuracy": 0.11757480353116989,
+      "num_tokens": 1933097.0,
+      "step": 1085
+    },
+    {
+      "entropy": 6.72240138053894,
+      "epoch": 0.05606563279582337,
+      "grad_norm": 1.1015625,
+      "learning_rate": 0.0004990234929580494,
+      "loss": 6.3646,
+      "mean_token_accuracy": 0.11228772699832916,
+      "num_tokens": 1942240.0,
+      "step": 1090
+    },
+    {
+      "entropy": 6.5773022174835205,
+      "epoch": 0.05632281459763907,
+      "grad_norm": 1.1171875,
+      "learning_rate": 0.0004989107821416609,
+      "loss": 6.3542,
+      "mean_token_accuracy": 0.11520460173487664,
+      "num_tokens": 1950501.0,
+      "step": 1095
+    },
+    {
+      "entropy": 6.835616827011108,
+      "epoch": 0.056579996399454775,
+      "grad_norm": 1.2421875,
+      "learning_rate": 0.0004987919326980723,
+      "loss": 6.4121,
+      "mean_token_accuracy": 0.11566629931330681,
+      "num_tokens": 1958621.0,
+      "step": 1100
+    },
+    {
+      "entropy": 6.723128461837769,
+      "epoch": 0.05683717820127048,
+      "grad_norm": 1.0625,
+      "learning_rate": 0.0004986669478856011,
+      "loss": 6.5071,
+      "mean_token_accuracy": 0.11123462915420532,
+      "num_tokens": 1966836.0,
+      "step": 1105
+    },
+    {
+      "entropy": 6.7716064453125,
+      "epoch": 0.05709436000308618,
+      "grad_norm": 1.0859375,
+      "learning_rate": 0.0004985358311307688,
+      "loss": 6.4827,
+      "mean_token_accuracy": 0.11127110049128533,
+      "num_tokens": 1976046.0,
+      "step": 1110
+    },
+    {
+      "entropy": 6.752973651885986,
+      "epoch": 0.05735154180490189,
+      "grad_norm": 1.1171875,
+      "learning_rate": 0.0004983985860282081,
+      "loss": 6.4645,
+      "mean_token_accuracy": 0.11258459836244583,
+      "num_tokens": 1984074.0,
+      "step": 1115
+    },
+    {
+      "entropy": 6.730236387252807,
+      "epoch": 0.057608723606717586,
+      "grad_norm": 1.0703125,
+      "learning_rate": 0.0004982552163405623,
+      "loss": 6.539,
+      "mean_token_accuracy": 0.11384548544883728,
+      "num_tokens": 1992898.0,
+      "step": 1120
+    },
+    {
+      "entropy": 6.832723760604859,
+      "epoch": 0.05786590540853329,
+      "grad_norm": 1.1328125,
+      "learning_rate": 0.0004981057259983839,
+      "loss": 6.4669,
+      "mean_token_accuracy": 0.11225469931960105,
+      "num_tokens": 2002390.0,
+      "step": 1125
+    },
+    {
+      "entropy": 6.657161140441895,
+      "epoch": 0.058123087210349,
+      "grad_norm": 1.1328125,
+      "learning_rate": 0.0004979501191000262,
+      "loss": 6.3538,
+      "mean_token_accuracy": 0.1199690505862236,
+      "num_tokens": 2011499.0,
+      "step": 1130
+    },
+    {
+      "entropy": 6.6502784252166744,
+      "epoch": 0.0583802690121647,
+      "grad_norm": 1.0546875,
+      "learning_rate": 0.0004977883999115311,
+      "loss": 6.4782,
+      "mean_token_accuracy": 0.11357165277004241,
+      "num_tokens": 2021336.0,
+      "step": 1135
+    },
+    {
+      "entropy": 6.731290435791015,
+      "epoch": 0.058637450813980405,
+      "grad_norm": 1.140625,
+      "learning_rate": 0.0004976205728665113,
+      "loss": 6.3638,
+      "mean_token_accuracy": 0.11489428207278252,
+      "num_tokens": 2029706.0,
+      "step": 1140
+    },
+    {
+      "entropy": 6.830005741119384,
+      "epoch": 0.058894632615796104,
+      "grad_norm": 1.171875,
+      "learning_rate": 0.0004974466425660307,
+      "loss": 6.5844,
+      "mean_token_accuracy": 0.10661658570170403,
+      "num_tokens": 2038915.0,
+      "step": 1145
+    },
+    {
+      "entropy": 6.701237726211548,
+      "epoch": 0.05915181441761181,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.0004972666137784759,
+      "loss": 6.418,
+      "mean_token_accuracy": 0.11196302473545075,
+      "num_tokens": 2047661.0,
+      "step": 1150
+    },
+    {
+      "entropy": 6.637448024749756,
+      "epoch": 0.059408996219427516,
+      "grad_norm": 1.1171875,
+      "learning_rate": 0.0004970804914394271,
+      "loss": 6.4053,
+      "mean_token_accuracy": 0.11128832325339318,
+      "num_tokens": 2056768.0,
+      "step": 1155
+    },
+    {
+      "entropy": 6.7576159000396725,
+      "epoch": 0.059666178021243216,
+      "grad_norm": 1.0625,
+      "learning_rate": 0.0004968882806515225,
+      "loss": 6.4509,
+      "mean_token_accuracy": 0.11101424172520638,
+      "num_tokens": 2066862.0,
+      "step": 1160
+    },
+    {
+      "entropy": 6.65157413482666,
+      "epoch": 0.05992335982305892,
+      "grad_norm": 1.1328125,
+      "learning_rate": 0.0004966899866843177,
+      "loss": 6.2771,
+      "mean_token_accuracy": 0.11530844420194626,
+      "num_tokens": 2075794.0,
+      "step": 1165
+    },
+    {
+      "entropy": 6.693396663665771,
+      "epoch": 0.06018054162487462,
+      "grad_norm": 1.1171875,
+      "learning_rate": 0.000496485614974142,
+      "loss": 6.4464,
+      "mean_token_accuracy": 0.11457487121224404,
+      "num_tokens": 2084806.0,
+      "step": 1170
+    },
+    {
+      "entropy": 6.578125905990601,
+      "epoch": 0.06043772342669033,
+      "grad_norm": 1.0,
+      "learning_rate": 0.0004962751711239492,
+      "loss": 6.3015,
+      "mean_token_accuracy": 0.11791201233863831,
+      "num_tokens": 2093884.0,
+      "step": 1175
+    },
+    {
+      "entropy": 6.601912546157837,
+      "epoch": 0.060694905228506034,
+      "grad_norm": 1.1015625,
+      "learning_rate": 0.0004960586609031636,
+      "loss": 6.2546,
+      "mean_token_accuracy": 0.12269692420959473,
+      "num_tokens": 2103716.0,
+      "step": 1180
+    },
+    {
+      "entropy": 6.633868169784546,
+      "epoch": 0.06095208703032173,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.0004958360902475224,
+      "loss": 6.3092,
+      "mean_token_accuracy": 0.11640048250555993,
+      "num_tokens": 2113627.0,
+      "step": 1185
+    },
+    {
+      "entropy": 6.588856792449951,
+      "epoch": 0.06120926883213744,
+      "grad_norm": 1.2109375,
+      "learning_rate": 0.0004956074652589125,
+      "loss": 6.2842,
+      "mean_token_accuracy": 0.1199034109711647,
+      "num_tokens": 2121455.0,
+      "step": 1190
+    },
+    {
+      "entropy": 6.636424875259399,
+      "epoch": 0.06146645063395314,
+      "grad_norm": 1.109375,
+      "learning_rate": 0.0004953727922052035,
+      "loss": 6.2933,
+      "mean_token_accuracy": 0.11959701478481292,
+      "num_tokens": 2130364.0,
+      "step": 1195
+    },
+    {
+      "entropy": 6.704743337631226,
+      "epoch": 0.061723632435768845,
+      "grad_norm": 1.203125,
+      "learning_rate": 0.0004951320775200756,
+      "loss": 6.4765,
+      "mean_token_accuracy": 0.11397643610835076,
+      "num_tokens": 2138893.0,
+      "step": 1200
+    },
+    {
+      "entropy": 6.7864457130432125,
+      "epoch": 0.06198081423758455,
+      "grad_norm": 1.1015625,
+      "learning_rate": 0.0004948853278028436,
+      "loss": 6.4775,
+      "mean_token_accuracy": 0.11080277115106582,
+      "num_tokens": 2148441.0,
+      "step": 1205
+    },
+    {
+      "entropy": 6.467393255233764,
+      "epoch": 0.06223799603940025,
+      "grad_norm": 1.125,
+      "learning_rate": 0.0004946325498182755,
+      "loss": 6.3488,
+      "mean_token_accuracy": 0.11968965232372283,
+      "num_tokens": 2157566.0,
+      "step": 1210
+    },
+    {
+      "entropy": 6.594685506820679,
+      "epoch": 0.06249517784121596,
+      "grad_norm": 1.0703125,
+      "learning_rate": 0.0004943737504964076,
+      "loss": 6.2783,
+      "mean_token_accuracy": 0.12351583763957023,
+      "num_tokens": 2166652.0,
+      "step": 1215
+    },
+    {
+      "entropy": 6.595351934432983,
+      "epoch": 0.06275235964303166,
+      "grad_norm": 1.078125,
+      "learning_rate": 0.000494108936932354,
+      "loss": 6.3271,
+      "mean_token_accuracy": 0.11962940394878388,
+      "num_tokens": 2175786.0,
+      "step": 1220
+    },
+    {
+      "entropy": 6.599175357818604,
+      "epoch": 0.06300954144484737,
+      "grad_norm": 1.0859375,
+      "learning_rate": 0.0004938381163861124,
+      "loss": 6.3318,
+      "mean_token_accuracy": 0.11382425799965859,
+      "num_tokens": 2184282.0,
+      "step": 1225
+    },
+    {
+      "entropy": 6.65804877281189,
+      "epoch": 0.06326672324666306,
+      "grad_norm": 1.1171875,
+      "learning_rate": 0.0004935612962823645,
+      "loss": 6.2634,
+      "mean_token_accuracy": 0.1293702930212021,
+      "num_tokens": 2192588.0,
+      "step": 1230
+    },
+    {
+      "entropy": 6.570264053344727,
+      "epoch": 0.06352390504847877,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.0004932784842102739,
+      "loss": 6.224,
+      "mean_token_accuracy": 0.1252226300537586,
+      "num_tokens": 2201247.0,
+      "step": 1235
+    },
+    {
+      "entropy": 6.581180286407471,
+      "epoch": 0.06378108685029447,
+      "grad_norm": 1.0546875,
+      "learning_rate": 0.0004929896879232758,
+      "loss": 6.4,
+      "mean_token_accuracy": 0.11070948839187622,
+      "num_tokens": 2210096.0,
+      "step": 1240
+    },
+    {
+      "entropy": 6.548767185211181,
+      "epoch": 0.06403826865211018,
+      "grad_norm": 1.0625,
+      "learning_rate": 0.0004926949153388668,
+      "loss": 6.2372,
+      "mean_token_accuracy": 0.12190242633223533,
+      "num_tokens": 2219221.0,
+      "step": 1245
+    },
+    {
+      "entropy": 6.613304138183594,
+      "epoch": 0.06429545045392589,
+      "grad_norm": 1.0234375,
+      "learning_rate": 0.0004923941745383859,
+      "loss": 6.3914,
+      "mean_token_accuracy": 0.11916265785694122,
+      "num_tokens": 2229094.0,
+      "step": 1250
+    },
+    {
+      "entropy": 6.622950220108033,
+      "epoch": 0.06455263225574158,
+      "grad_norm": 1.1328125,
+      "learning_rate": 0.000492087473766794,
+      "loss": 6.2814,
+      "mean_token_accuracy": 0.12494359165430069,
+      "num_tokens": 2236964.0,
+      "step": 1255
+    },
+    {
+      "entropy": 6.639262056350708,
+      "epoch": 0.06480981405755729,
+      "grad_norm": 1.0703125,
+      "learning_rate": 0.000491774821432448,
+      "loss": 6.2964,
+      "mean_token_accuracy": 0.11379757001996041,
+      "num_tokens": 2247044.0,
+      "step": 1260
+    },
+    {
+      "entropy": 6.578648328781128,
+      "epoch": 0.06506699585937299,
+      "grad_norm": 1.0859375,
+      "learning_rate": 0.0004914562261068693,
+      "loss": 6.3464,
+      "mean_token_accuracy": 0.11853989958763123,
+      "num_tokens": 2256106.0,
+      "step": 1265
+    },
+    {
+      "entropy": 6.5819896221160885,
+      "epoch": 0.0653241776611887,
+      "grad_norm": 1.015625,
+      "learning_rate": 0.0004911316965245098,
+      "loss": 6.333,
+      "mean_token_accuracy": 0.1119878426194191,
+      "num_tokens": 2265432.0,
+      "step": 1270
+    },
+    {
+      "entropy": 6.685585260391235,
+      "epoch": 0.0655813594630044,
+      "grad_norm": 1.125,
+      "learning_rate": 0.000490801241582512,
+      "loss": 6.3091,
+      "mean_token_accuracy": 0.11969617679715157,
+      "num_tokens": 2273882.0,
+      "step": 1275
+    },
+    {
+      "entropy": 6.507035255432129,
+      "epoch": 0.0658385412648201,
+      "grad_norm": 1.0546875,
+      "learning_rate": 0.000490464870340465,
+      "loss": 6.2311,
+      "mean_token_accuracy": 0.12235765084624291,
+      "num_tokens": 2282754.0,
+      "step": 1280
+    },
+    {
+      "entropy": 6.6120068550109865,
+      "epoch": 0.0660957230666358,
+      "grad_norm": 1.046875,
+      "learning_rate": 0.0004901225920201563,
+      "loss": 6.2385,
+      "mean_token_accuracy": 0.11968952193856239,
+      "num_tokens": 2291637.0,
+      "step": 1285
+    },
+    {
+      "entropy": 6.5052814960479735,
+      "epoch": 0.06635290486845151,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.000489774416005319,
+      "loss": 6.2752,
+      "mean_token_accuracy": 0.1284623309969902,
+      "num_tokens": 2300279.0,
+      "step": 1290
+    },
+    {
+      "entropy": 6.605392360687256,
+      "epoch": 0.06661008667026722,
+      "grad_norm": 1.203125,
+      "learning_rate": 0.0004894203518413742,
+      "loss": 6.3063,
+      "mean_token_accuracy": 0.11685983091592789,
+      "num_tokens": 2309413.0,
+      "step": 1295
+    },
+    {
+      "entropy": 6.453005313873291,
+      "epoch": 0.06686726847208292,
+      "grad_norm": 1.046875,
+      "learning_rate": 0.0004890604092351701,
+      "loss": 6.2339,
+      "mean_token_accuracy": 0.12537124752998352,
+      "num_tokens": 2318284.0,
+      "step": 1300
+    },
+    {
+      "entropy": 6.663999080657959,
+      "epoch": 0.06712445027389861,
+      "grad_norm": 1.140625,
+      "learning_rate": 0.000488694598054715,
+      "loss": 6.4158,
+      "mean_token_accuracy": 0.11596757918596268,
+      "num_tokens": 2327483.0,
+      "step": 1305
+    },
+    {
+      "entropy": 6.604641580581665,
+      "epoch": 0.06738163207571432,
+      "grad_norm": 1.046875,
+      "learning_rate": 0.0004883229283289071,
+      "loss": 6.3298,
+      "mean_token_accuracy": 0.11663304418325424,
+      "num_tokens": 2336096.0,
+      "step": 1310
+    },
+    {
+      "entropy": 6.647005414962768,
+      "epoch": 0.06763881387753003,
+      "grad_norm": 1.0,
+      "learning_rate": 0.00048794541024725993,
+      "loss": 6.2797,
+      "mean_token_accuracy": 0.11775869503617287,
+      "num_tokens": 2345393.0,
+      "step": 1315
+    },
+    {
+      "entropy": 6.554132890701294,
+      "epoch": 0.06789599567934573,
+      "grad_norm": 1.15625,
+      "learning_rate": 0.0004875620541596221,
+      "loss": 6.2501,
+      "mean_token_accuracy": 0.12467899695038795,
+      "num_tokens": 2353824.0,
+      "step": 1320
+    },
+    {
+      "entropy": 6.465411710739136,
+      "epoch": 0.06815317748116144,
+      "grad_norm": 1.140625,
+      "learning_rate": 0.00048717287057589454,
+      "loss": 6.2802,
+      "mean_token_accuracy": 0.12414587661623955,
+      "num_tokens": 2361992.0,
+      "step": 1325
+    },
+    {
+      "entropy": 6.607723951339722,
+      "epoch": 0.06841035928297713,
+      "grad_norm": 1.0390625,
+      "learning_rate": 0.0004867778701657417,
+      "loss": 6.1856,
+      "mean_token_accuracy": 0.12355614453554153,
+      "num_tokens": 2370849.0,
+      "step": 1330
+    },
+    {
+      "entropy": 6.542122459411621,
+      "epoch": 0.06866754108479284,
+      "grad_norm": 1.1953125,
+      "learning_rate": 0.00048637706375829955,
+      "loss": 6.2951,
+      "mean_token_accuracy": 0.125213223695755,
+      "num_tokens": 2379460.0,
+      "step": 1335
+    },
+    {
+      "entropy": 6.524771022796631,
+      "epoch": 0.06892472288660854,
+      "grad_norm": 1.046875,
+      "learning_rate": 0.000485970462341878,
+      "loss": 6.1841,
+      "mean_token_accuracy": 0.12624969705939293,
+      "num_tokens": 2388236.0,
+      "step": 1340
+    },
+    {
+      "entropy": 6.408080244064331,
+      "epoch": 0.06918190468842425,
+      "grad_norm": 1.1796875,
+      "learning_rate": 0.00048555807706366044,
+      "loss": 6.2347,
+      "mean_token_accuracy": 0.12203069031238556,
+      "num_tokens": 2397598.0,
+      "step": 1345
+    },
+    {
+      "entropy": 6.627772331237793,
+      "epoch": 0.06943908649023996,
+      "grad_norm": 1.125,
+      "learning_rate": 0.00048513991922939756,
+      "loss": 6.2066,
+      "mean_token_accuracy": 0.12295120730996131,
+      "num_tokens": 2406097.0,
+      "step": 1350
+    },
+    {
+      "entropy": 6.502293634414673,
+      "epoch": 0.06969626829205565,
+      "grad_norm": 1.0625,
+      "learning_rate": 0.00048471600030309744,
+      "loss": 6.3098,
+      "mean_token_accuracy": 0.11426047384738922,
+      "num_tokens": 2414590.0,
+      "step": 1355
+    },
+    {
+      "entropy": 6.440834903717041,
+      "epoch": 0.06995345009387136,
+      "grad_norm": 1.03125,
+      "learning_rate": 0.00048428633190671186,
+      "loss": 6.1354,
+      "mean_token_accuracy": 0.1235704205930233,
+      "num_tokens": 2424100.0,
+      "step": 1360
+    },
+    {
+      "entropy": 6.563741016387939,
+      "epoch": 0.07021063189568706,
+      "grad_norm": 1.1328125,
+      "learning_rate": 0.0004838509258198167,
+      "loss": 6.2091,
+      "mean_token_accuracy": 0.121873689442873,
+      "num_tokens": 2432751.0,
+      "step": 1365
+    },
+    {
+      "entropy": 6.543551826477051,
+      "epoch": 0.07046781369750277,
+      "grad_norm": 1.1015625,
+      "learning_rate": 0.00048340979397929,
+      "loss": 6.1861,
+      "mean_token_accuracy": 0.12034041285514832,
+      "num_tokens": 2442054.0,
+      "step": 1370
+    },
+    {
+      "entropy": 6.375408792495728,
+      "epoch": 0.07072499549931847,
+      "grad_norm": 1.109375,
+      "learning_rate": 0.00048296294847898386,
+      "loss": 6.1099,
+      "mean_token_accuracy": 0.12799823582172393,
+      "num_tokens": 2450711.0,
+      "step": 1375
+    },
+    {
+      "entropy": 6.405136442184448,
+      "epoch": 0.07098217730113417,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.0004825104015693934,
+      "loss": 6.1895,
+      "mean_token_accuracy": 0.12983948215842248,
+      "num_tokens": 2460203.0,
+      "step": 1380
+    },
+    {
+      "entropy": 6.558651971817016,
+      "epoch": 0.07123935910294987,
+      "grad_norm": 1.1015625,
+      "learning_rate": 0.0004820521656573208,
+      "loss": 6.2482,
+      "mean_token_accuracy": 0.12185781002044678,
+      "num_tokens": 2469656.0,
+      "step": 1385
+    },
+    {
+      "entropy": 6.4783261775970455,
+      "epoch": 0.07149654090476558,
+      "grad_norm": 1.1640625,
+      "learning_rate": 0.00048158825330553505,
+      "loss": 6.1986,
+      "mean_token_accuracy": 0.12839986979961396,
+      "num_tokens": 2478359.0,
+      "step": 1390
+    },
+    {
+      "entropy": 6.440516662597656,
+      "epoch": 0.07175372270658129,
+      "grad_norm": 1.015625,
+      "learning_rate": 0.00048111867723242763,
+      "loss": 6.1319,
+      "mean_token_accuracy": 0.12275411933660507,
+      "num_tokens": 2487330.0,
+      "step": 1395
+    },
+    {
+      "entropy": 6.464969682693481,
+      "epoch": 0.07201090450839699,
+      "grad_norm": 1.1328125,
+      "learning_rate": 0.0004806434503116637,
+      "loss": 6.1196,
+      "mean_token_accuracy": 0.12776034623384475,
+      "num_tokens": 2495445.0,
+      "step": 1400
+    },
+    {
+      "entropy": 6.352128219604492,
+      "epoch": 0.07226808631021268,
+      "grad_norm": 1.03125,
+      "learning_rate": 0.0004801625855718296,
+      "loss": 6.1221,
+      "mean_token_accuracy": 0.129915539175272,
+      "num_tokens": 2504061.0,
+      "step": 1405
+    },
+    {
+      "entropy": 6.479479789733887,
+      "epoch": 0.07252526811202839,
+      "grad_norm": 1.1640625,
+      "learning_rate": 0.00047967609619607477,
+      "loss": 6.1634,
+      "mean_token_accuracy": 0.12523439154028893,
+      "num_tokens": 2513044.0,
+      "step": 1410
+    },
+    {
+      "entropy": 6.456380176544189,
+      "epoch": 0.0727824499138441,
+      "grad_norm": 1.0625,
+      "learning_rate": 0.0004791839955217513,
+      "loss": 6.2452,
+      "mean_token_accuracy": 0.12148747369647026,
+      "num_tokens": 2522027.0,
+      "step": 1415
+    },
+    {
+      "entropy": 6.455992269515991,
+      "epoch": 0.0730396317156598,
+      "grad_norm": 1.03125,
+      "learning_rate": 0.00047868629704004786,
+      "loss": 6.1438,
+      "mean_token_accuracy": 0.12664651721715928,
+      "num_tokens": 2530888.0,
+      "step": 1420
+    },
+    {
+      "entropy": 6.309210348129272,
+      "epoch": 0.07329681351747551,
+      "grad_norm": 1.1328125,
+      "learning_rate": 0.00047818301439561965,
+      "loss": 6.1205,
+      "mean_token_accuracy": 0.13230895176529883,
+      "num_tokens": 2540210.0,
+      "step": 1425
+    },
+    {
+      "entropy": 6.500010251998901,
+      "epoch": 0.0735539953192912,
+      "grad_norm": 1.1171875,
+      "learning_rate": 0.00047767416138621454,
+      "loss": 6.1576,
+      "mean_token_accuracy": 0.13035310953855514,
+      "num_tokens": 2547943.0,
+      "step": 1430
+    },
+    {
+      "entropy": 6.4128905773162845,
+      "epoch": 0.07381117712110691,
+      "grad_norm": 1.09375,
+      "learning_rate": 0.000477159751962295,
+      "loss": 6.1758,
+      "mean_token_accuracy": 0.12096773236989974,
+      "num_tokens": 2557174.0,
+      "step": 1435
+    },
+    {
+      "entropy": 6.467635154724121,
+      "epoch": 0.07406835892292261,
+      "grad_norm": 1.109375,
+      "learning_rate": 0.00047663980022665507,
+      "loss": 6.1847,
+      "mean_token_accuracy": 0.12423764616250992,
+      "num_tokens": 2566139.0,
+      "step": 1440
+    },
+    {
+      "entropy": 6.443244886398316,
+      "epoch": 0.07432554072473832,
+      "grad_norm": 1.1015625,
+      "learning_rate": 0.00047611432043403437,
+      "loss": 6.1833,
+      "mean_token_accuracy": 0.12400569245219231,
+      "num_tokens": 2574701.0,
+      "step": 1445
+    },
+    {
+      "entropy": 6.48380708694458,
+      "epoch": 0.07458272252655403,
+      "grad_norm": 1.109375,
+      "learning_rate": 0.0004755833269907267,
+      "loss": 6.1683,
+      "mean_token_accuracy": 0.13298066854476928,
+      "num_tokens": 2583065.0,
+      "step": 1450
+    },
+    {
+      "entropy": 6.337066030502319,
+      "epoch": 0.07483990432836972,
+      "grad_norm": 1.0625,
+      "learning_rate": 0.0004750468344541857,
+      "loss": 6.0415,
+      "mean_token_accuracy": 0.13096242472529412,
+      "num_tokens": 2592619.0,
+      "step": 1455
+    },
+    {
+      "entropy": 6.387078046798706,
+      "epoch": 0.07509708613018543,
+      "grad_norm": 1.125,
+      "learning_rate": 0.00047450485753262525,
+      "loss": 6.1234,
+      "mean_token_accuracy": 0.12428422570228577,
+      "num_tokens": 2601434.0,
+      "step": 1460
+    },
+    {
+      "entropy": 6.404777574539184,
+      "epoch": 0.07535426793200113,
+      "grad_norm": 1.015625,
+      "learning_rate": 0.00047395741108461633,
+      "loss": 6.247,
+      "mean_token_accuracy": 0.11993053182959557,
+      "num_tokens": 2610925.0,
+      "step": 1465
+    },
+    {
+      "entropy": 6.445322704315186,
+      "epoch": 0.07561144973381684,
+      "grad_norm": 1.015625,
+      "learning_rate": 0.00047340451011867985,
+      "loss": 6.2036,
+      "mean_token_accuracy": 0.1239316701889038,
+      "num_tokens": 2620275.0,
+      "step": 1470
+    },
+    {
+      "entropy": 6.519756078720093,
+      "epoch": 0.07586863153563254,
+      "grad_norm": 1.0,
+      "learning_rate": 0.00047284616979287515,
+      "loss": 6.2099,
+      "mean_token_accuracy": 0.1233117938041687,
+      "num_tokens": 2629826.0,
+      "step": 1475
+    },
+    {
+      "entropy": 6.344198274612427,
+      "epoch": 0.07612581333744824,
+      "grad_norm": 1.0625,
+      "learning_rate": 0.00047228240541438433,
+      "loss": 5.9902,
+      "mean_token_accuracy": 0.13270971104502677,
+      "num_tokens": 2638680.0,
+      "step": 1480
+    },
+    {
+      "entropy": 6.357509803771973,
+      "epoch": 0.07638299513926394,
+      "grad_norm": 1.078125,
+      "learning_rate": 0.00047171323243909257,
+      "loss": 6.1679,
+      "mean_token_accuracy": 0.12576281726360322,
+      "num_tokens": 2647739.0,
+      "step": 1485
+    },
+    {
+      "entropy": 6.3423261642456055,
+      "epoch": 0.07664017694107965,
+      "grad_norm": 1.125,
+      "learning_rate": 0.00047113866647116457,
+      "loss": 6.0689,
+      "mean_token_accuracy": 0.12693566605448722,
+      "num_tokens": 2656137.0,
+      "step": 1490
+    },
+    {
+      "entropy": 6.399060344696045,
+      "epoch": 0.07689735874289536,
+      "grad_norm": 1.046875,
+      "learning_rate": 0.0004705587232626164,
+      "loss": 6.1868,
+      "mean_token_accuracy": 0.1253664128482342,
+      "num_tokens": 2665173.0,
+      "step": 1495
+    },
+    {
+      "entropy": 6.466083621978759,
+      "epoch": 0.07715454054471106,
+      "grad_norm": 1.1953125,
+      "learning_rate": 0.00046997341871288424,
+      "loss": 6.1583,
+      "mean_token_accuracy": 0.13294297680258751,
+      "num_tokens": 2673574.0,
+      "step": 1500
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 4000,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 529781529968640.0,
+  "train_batch_size": 16,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-1500/training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:288e11bbb43604afa3f89d608820cede35fd8bec0d9199f7dd4fdc6e5b315d41
+size 6289