Instructions to use barha/granite-genai-attack-vector-350m-alora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use barha/granite-genai-attack-vector-350m-alora with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("ibm-granite/granite-4.0-350m")
model = PeftModel.from_pretrained(base_model, "barha/granite-genai-attack-vector-350m-alora")

Transformers

How to use barha/granite-genai-attack-vector-350m-alora with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="barha/granite-genai-attack-vector-350m-alora")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("barha/granite-genai-attack-vector-350m-alora", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use barha/granite-genai-attack-vector-350m-alora with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "barha/granite-genai-attack-vector-350m-alora"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "barha/granite-genai-attack-vector-350m-alora",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/barha/granite-genai-attack-vector-350m-alora

SGLang

How to use barha/granite-genai-attack-vector-350m-alora with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "barha/granite-genai-attack-vector-350m-alora" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "barha/granite-genai-attack-vector-350m-alora",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "barha/granite-genai-attack-vector-350m-alora" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "barha/granite-genai-attack-vector-350m-alora",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use barha/granite-genai-attack-vector-350m-alora with Docker Model Runner:
```
docker model run hf.co/barha/granite-genai-attack-vector-350m-alora
```

barha commited on Jun 16

Commit

a756c33

verified ·

1 Parent(s): 85ba633

aLoRA retrain of genai-attack-vector on granite-4.0-350m (gated on assistant invocation tokens)

Browse files

Files changed (22) hide show

adapter_config.json +9 -7
adapter_model.safetensors +2 -2
checkpoint-40/adapter_config.json +9 -7
checkpoint-40/adapter_model.safetensors +2 -2
checkpoint-40/optimizer.pt +2 -2
checkpoint-40/rng_state_0.pth +1 -1
checkpoint-40/rng_state_1.pth +1 -1
checkpoint-40/rng_state_2.pth +1 -1
checkpoint-40/rng_state_3.pth +1 -1
checkpoint-40/trainer_state.json +158 -158
checkpoint-40/training_args.bin +1 -1
checkpoint-80/adapter_config.json +9 -7
checkpoint-80/adapter_model.safetensors +2 -2
checkpoint-80/optimizer.pt +2 -2
checkpoint-80/rng_state_0.pth +1 -1
checkpoint-80/rng_state_1.pth +1 -1
checkpoint-80/rng_state_2.pth +1 -1
checkpoint-80/rng_state_3.pth +1 -1
checkpoint-80/trainer_state.json +318 -318
checkpoint-80/training_args.bin +1 -1
predictions.jsonl +0 -0
training_args.bin +1 -1

adapter_config.json CHANGED Viewed

@@ -32,21 +32,23 @@
   "qalora_group_size": 16,
   "r": 32,
   "rank_pattern": {
-    "input_linear": 32,
     "k_proj": 32,
     "o_proj": 32,
-    "output_linear": 32,
     "q_proj": 32,
     "v_proj": 32
   },
   "revision": null,
   "target_modules": [
-    "k_proj",
-    "v_proj",
-    "output_linear",
-    "input_linear",
     "o_proj",
-    "q_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

   "qalora_group_size": 16,
   "r": 32,
   "rank_pattern": {
+    "down_proj": 32,
+    "gate_proj": 32,
     "k_proj": 32,
     "o_proj": 32,
     "q_proj": 32,
+    "up_proj": 32,
     "v_proj": 32
   },
   "revision": null,
   "target_modules": [
+    "up_proj",
     "o_proj",
+    "q_proj",
+    "gate_proj",
+    "down_proj",
+    "v_proj",
+    "k_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d64e4ba747c90d89c6855d7abd07cdcaf64b5178f7e1843492ad55fcb5e75bd4
-size 23885040

 version https://git-lfs.github.com/spec/v1
+oid sha256:ecd2ea37998d533a6a40fa093b8f1945d89505a7c42b4d282dfbfd7a4f0a27b4
+size 56936568

checkpoint-40/adapter_config.json CHANGED Viewed

@@ -32,21 +32,23 @@
   "qalora_group_size": 16,
   "r": 32,
   "rank_pattern": {
-    "input_linear": 32,
     "k_proj": 32,
     "o_proj": 32,
-    "output_linear": 32,
     "q_proj": 32,
     "v_proj": 32
   },
   "revision": null,
   "target_modules": [
-    "k_proj",
-    "v_proj",
-    "output_linear",
-    "input_linear",
     "o_proj",
-    "q_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

   "qalora_group_size": 16,
   "r": 32,
   "rank_pattern": {
+    "down_proj": 32,
+    "gate_proj": 32,
     "k_proj": 32,
     "o_proj": 32,
     "q_proj": 32,
+    "up_proj": 32,
     "v_proj": 32
   },
   "revision": null,
   "target_modules": [
+    "up_proj",
     "o_proj",
+    "q_proj",
+    "gate_proj",
+    "down_proj",
+    "v_proj",
+    "k_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

checkpoint-40/adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bbc48dbf31363eb1c0c7e33c0f0b268f31127fb96bf8a740d5896f89aca367b0
-size 23885040

 version https://git-lfs.github.com/spec/v1
+oid sha256:e0d28e7fda8b0f8b4f0aff9d45a0b44dfe6e663b1d7fb6897889d31cce7676b9
+size 56936568

checkpoint-40/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:035f2acdb28576d4d03d0de4a102328b3ef8e0d6da86a7b2eca7efdf604d0c2b
-size 47898234

 version https://git-lfs.github.com/spec/v1
+oid sha256:ee4ebdf4710b6ed7f48615c9d51e7e8b208efac0f2a01d9f07f273f8304775eb
+size 114100074

checkpoint-40/rng_state_0.pth CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:79517f24567abca24fcafe64e66c3d26fe6420354bb7376c3e5640018c1749da
 size 15024

 version https://git-lfs.github.com/spec/v1
+oid sha256:156a2c3d2e72cbd3a562437f1517820a9bd54ca6f55636975080785c11eb3269
 size 15024

checkpoint-40/rng_state_1.pth CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9f342ed60a7a965df784d68b512add7705a5be02c6147974eab3e29543fa2faf
 size 15024

 version https://git-lfs.github.com/spec/v1
+oid sha256:2f1b464d4137d3e65c50bc15cfdc2f7f74c37fdc4b19893c31c0bd4e2dd263aa
 size 15024

checkpoint-40/rng_state_2.pth CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:cf58302af06626e217f4f04548b41c8d0ec419c1358ecd59b2abe357cdf96979
 size 15024

 version https://git-lfs.github.com/spec/v1
+oid sha256:e1e4a8a3ed7a4c0eaf78504275dc1aed8730dc507d29311a7b57eb9a6fb589d8
 size 15024

checkpoint-40/rng_state_3.pth CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6bbb62acf4c0fb1f059744a428f56536668309d1bf1bd95817cfa9cf7b097360
 size 15024

 version https://git-lfs.github.com/spec/v1
+oid sha256:43e5b5f568c5f46f46d04020b4f12412fb64033a3ec02aeb47b62b24fbc66f65
 size 15024

checkpoint-40/trainer_state.json CHANGED Viewed

@@ -12,7 +12,7 @@
     {
       "entropy": 0.8529030084609985,
       "epoch": 0.02531645569620253,
-      "grad_norm": 78.52010345458984,
       "learning_rate": 0.0,
       "loss": 15.181689262390137,
       "mean_token_accuracy": 0.28961846977472305,
@@ -22,7 +22,7 @@
     {
       "entropy": 0.8989771902561188,
       "epoch": 0.05063291139240506,
-      "grad_norm": 75.26119232177734,
       "learning_rate": 5e-05,
       "loss": 14.95038890838623,
       "mean_token_accuracy": 0.27897340804338455,
@@ -30,382 +30,382 @@
       "step": 2
     },
     {
-      "entropy": 0.9821003973484039,
       "epoch": 0.0759493670886076,
-      "grad_norm": 74.54902648925781,
       "learning_rate": 0.0001,
-      "loss": 13.742369651794434,
-      "mean_token_accuracy": 0.2827275022864342,
       "num_tokens": 21366.0,
       "step": 3
     },
     {
-      "entropy": 1.0873801112174988,
       "epoch": 0.10126582278481013,
-      "grad_norm": 34.19524383544922,
       "learning_rate": 0.00015000000000000001,
-      "loss": 11.25832462310791,
-      "mean_token_accuracy": 0.44820327311754227,
       "num_tokens": 28250.0,
       "step": 4
     },
     {
-      "entropy": 1.0790044069290161,
       "epoch": 0.12658227848101267,
-      "grad_norm": 28.061630249023438,
       "learning_rate": 0.0002,
-      "loss": 9.99978256225586,
-      "mean_token_accuracy": 0.5489031970500946,
       "num_tokens": 34914.0,
       "step": 5
     },
     {
-      "entropy": 1.1546767055988312,
       "epoch": 0.1518987341772152,
-      "grad_norm": 24.789119720458984,
       "learning_rate": 0.0001999145758387301,
-      "loss": 8.632344245910645,
-      "mean_token_accuracy": 0.554060235619545,
       "num_tokens": 42532.0,
       "step": 6
     },
     {
-      "entropy": 1.0596064329147339,
       "epoch": 0.17721518987341772,
-      "grad_norm": 12.310446739196777,
       "learning_rate": 0.000199658449300667,
-      "loss": 7.437180995941162,
-      "mean_token_accuracy": 0.5873144865036011,
       "num_tokens": 49872.0,
       "step": 7
     },
     {
-      "entropy": 0.7889958769083023,
       "epoch": 0.20253164556962025,
-      "grad_norm": 10.70737361907959,
       "learning_rate": 0.0001992320579737045,
-      "loss": 7.076646327972412,
-      "mean_token_accuracy": 0.5939096212387085,
       "num_tokens": 55639.0,
       "step": 8
     },
     {
-      "entropy": 0.9904351979494095,
       "epoch": 0.22784810126582278,
-      "grad_norm": 39.2529296875,
       "learning_rate": 0.00019863613034027224,
-      "loss": 7.060567855834961,
-      "mean_token_accuracy": 0.6403393149375916,
       "num_tokens": 61649.0,
       "step": 9
     },
     {
-      "entropy": 1.3839358687400818,
       "epoch": 0.25316455696202533,
-      "grad_norm": 15.882004737854004,
       "learning_rate": 0.00019787168453273544,
-      "loss": 6.710319519042969,
-      "mean_token_accuracy": 0.603007972240448,
       "num_tokens": 69689.0,
       "step": 10
     },
     {
-      "entropy": 1.1738347113132477,
       "epoch": 0.27848101265822783,
-      "grad_norm": 8.31102180480957,
       "learning_rate": 0.00019694002659393305,
-      "loss": 6.315446853637695,
-      "mean_token_accuracy": 0.627836138010025,
       "num_tokens": 77528.0,
       "step": 11
     },
     {
-      "entropy": 1.2701810002326965,
       "epoch": 0.3037974683544304,
-      "grad_norm": 6.591884136199951,
       "learning_rate": 0.0001958427482458253,
-      "loss": 6.042618751525879,
-      "mean_token_accuracy": 0.6690841615200043,
       "num_tokens": 85064.0,
       "step": 12
     },
     {
-      "entropy": 1.2295758128166199,
       "epoch": 0.3291139240506329,
-      "grad_norm": 15.478667259216309,
       "learning_rate": 0.00019458172417006347,
-      "loss": 6.213565826416016,
-      "mean_token_accuracy": 0.6405076384544373,
       "num_tokens": 91009.0,
       "step": 13
     },
     {
-      "entropy": 1.268456757068634,
       "epoch": 0.35443037974683544,
-      "grad_norm": 7.625810146331787,
       "learning_rate": 0.0001931591088051279,
-      "loss": 6.13183069229126,
-      "mean_token_accuracy": 0.6365684270858765,
       "num_tokens": 97909.0,
       "step": 14
     },
     {
-      "entropy": 1.2001140415668488,
       "epoch": 0.379746835443038,
-      "grad_norm": 7.8025221824646,
       "learning_rate": 0.00019157733266550575,
-      "loss": 6.064075469970703,
-      "mean_token_accuracy": 0.6330698281526566,
       "num_tokens": 104187.0,
       "step": 15
     },
     {
-      "entropy": 1.066677764058113,
       "epoch": 0.4050632911392405,
-      "grad_norm": 8.150992393493652,
       "learning_rate": 0.0001898390981891979,
-      "loss": 5.798349380493164,
-      "mean_token_accuracy": 0.646238312125206,
       "num_tokens": 110300.0,
       "step": 16
     },
     {
-      "entropy": 1.0704185664653778,
       "epoch": 0.43037974683544306,
-      "grad_norm": 3.722452402114868,
       "learning_rate": 0.0001879473751206489,
-      "loss": 5.87784481048584,
-      "mean_token_accuracy": 0.6626823246479034,
       "num_tokens": 117157.0,
       "step": 17
     },
     {
-      "entropy": 1.1619559824466705,
       "epoch": 0.45569620253164556,
-      "grad_norm": 4.019715785980225,
       "learning_rate": 0.00018590539543698854,
-      "loss": 5.398146152496338,
-      "mean_token_accuracy": 0.714724525809288,
       "num_tokens": 123378.0,
       "step": 18
     },
     {
-      "entropy": 1.072197139263153,
       "epoch": 0.4810126582278481,
-      "grad_norm": 3.3014461994171143,
       "learning_rate": 0.00018371664782625287,
-      "loss": 5.488443851470947,
-      "mean_token_accuracy": 0.8233065903186798,
       "num_tokens": 129042.0,
       "step": 19
     },
     {
-      "entropy": 0.9806628823280334,
       "epoch": 0.5063291139240507,
-      "grad_norm": 2.70117449760437,
       "learning_rate": 0.0001813848717270195,
-      "loss": 5.3289875984191895,
-      "mean_token_accuracy": 0.8143462389707565,
       "num_tokens": 135747.0,
       "step": 20
     },
     {
-      "entropy": 0.8040178865194321,
       "epoch": 0.5316455696202531,
-      "grad_norm": 3.4564270973205566,
       "learning_rate": 0.00017891405093963938,
-      "loss": 5.2513909339904785,
-      "mean_token_accuracy": 0.8225801885128021,
       "num_tokens": 141848.0,
       "step": 21
     },
     {
-      "entropy": 0.7444496601819992,
       "epoch": 0.5569620253164557,
-      "grad_norm": 2.3326470851898193,
       "learning_rate": 0.00017630840681998066,
-      "loss": 5.4000067710876465,
-      "mean_token_accuracy": 0.8329752385616302,
       "num_tokens": 147566.0,
       "step": 22
     },
     {
-      "entropy": 0.6936303377151489,
       "epoch": 0.5822784810126582,
-      "grad_norm": 2.478424072265625,
       "learning_rate": 0.00017357239106731317,
-      "loss": 5.346803665161133,
-      "mean_token_accuracy": 0.8282431960105896,
       "num_tokens": 154620.0,
       "step": 23
     },
     {
-      "entropy": 0.6478566229343414,
       "epoch": 0.6075949367088608,
-      "grad_norm": 2.6104896068573,
       "learning_rate": 0.00017071067811865476,
-      "loss": 5.191404342651367,
-      "mean_token_accuracy": 0.8425344824790955,
       "num_tokens": 161701.0,
       "step": 24
     },
     {
-      "entropy": 0.6078712940216064,
       "epoch": 0.6329113924050633,
-      "grad_norm": 4.111148357391357,
       "learning_rate": 0.00016772815716257412,
-      "loss": 5.243197917938232,
-      "mean_token_accuracy": 0.8070437759160995,
       "num_tokens": 168435.0,
       "step": 25
     },
     {
-      "entropy": 0.5901128649711609,
       "epoch": 0.6582278481012658,
-      "grad_norm": 2.244276285171509,
       "learning_rate": 0.00016462992378609407,
-      "loss": 5.272450923919678,
-      "mean_token_accuracy": 0.8324518501758575,
       "num_tokens": 175650.0,
       "step": 26
     },
     {
-      "entropy": 0.5017133206129074,
       "epoch": 0.6835443037974683,
-      "grad_norm": 2.1174073219299316,
       "learning_rate": 0.0001614212712689668,
-      "loss": 5.215684413909912,
-      "mean_token_accuracy": 0.8321826756000519,
       "num_tokens": 181490.0,
       "step": 27
     },
     {
-      "entropy": 0.4748859405517578,
       "epoch": 0.7088607594936709,
-      "grad_norm": 2.087001323699951,
       "learning_rate": 0.00015810768154019385,
-      "loss": 5.15059757232666,
-      "mean_token_accuracy": 0.8388571739196777,
       "num_tokens": 188582.0,
       "step": 28
     },
     {
-      "entropy": 0.3888755962252617,
       "epoch": 0.7341772151898734,
-      "grad_norm": 1.487061858177185,
       "learning_rate": 0.00015469481581224272,
-      "loss": 5.174810409545898,
-      "mean_token_accuracy": 0.8587576746940613,
       "num_tokens": 193783.0,
       "step": 29
     },
     {
-      "entropy": 0.4288812130689621,
       "epoch": 0.759493670886076,
-      "grad_norm": 2.868377685546875,
       "learning_rate": 0.00015118850490896012,
-      "loss": 5.343541145324707,
-      "mean_token_accuracy": 0.8163273632526398,
       "num_tokens": 200686.0,
       "step": 30
     },
     {
-      "entropy": 0.39349576085805893,
       "epoch": 0.7848101265822784,
-      "grad_norm": 1.7554408311843872,
       "learning_rate": 0.00014759473930370736,
-      "loss": 5.089621543884277,
-      "mean_token_accuracy": 0.8510452210903168,
       "num_tokens": 208440.0,
       "step": 31
     },
     {
-      "entropy": 0.37267740815877914,
       "epoch": 0.810126582278481,
-      "grad_norm": 1.210075855255127,
       "learning_rate": 0.00014391965888473703,
-      "loss": 5.0665974617004395,
-      "mean_token_accuracy": 0.8576832711696625,
       "num_tokens": 215131.0,
       "step": 32
     },
     {
-      "entropy": 0.3740547075867653,
       "epoch": 0.8354430379746836,
-      "grad_norm": 2.1259520053863525,
       "learning_rate": 0.00014016954246529696,
-      "loss": 5.144742012023926,
-      "mean_token_accuracy": 0.8442028760910034,
       "num_tokens": 222195.0,
       "step": 33
     },
     {
-      "entropy": 0.33895356953144073,
       "epoch": 0.8607594936708861,
-      "grad_norm": 2.1364142894744873,
       "learning_rate": 0.00013635079705638298,
-      "loss": 5.227801322937012,
-      "mean_token_accuracy": 0.8455548137426376,
       "num_tokens": 229926.0,
       "step": 34
     },
     {
-      "entropy": 0.3552927002310753,
       "epoch": 0.8860759493670886,
-      "grad_norm": 2.4019367694854736,
       "learning_rate": 0.00013246994692046836,
-      "loss": 5.275204181671143,
-      "mean_token_accuracy": 0.8322960436344147,
       "num_tokens": 235766.0,
       "step": 35
     },
     {
-      "entropy": 0.3192586973309517,
       "epoch": 0.9113924050632911,
-      "grad_norm": 1.8943588733673096,
       "learning_rate": 0.00012853362242491053,
-      "loss": 5.1949639320373535,
-      "mean_token_accuracy": 0.8491553515195847,
       "num_tokens": 242731.0,
       "step": 36
     },
     {
-      "entropy": 0.31374089419841766,
       "epoch": 0.9367088607594937,
-      "grad_norm": 1.5797899961471558,
       "learning_rate": 0.00012454854871407994,
-      "loss": 5.137860298156738,
-      "mean_token_accuracy": 0.8447228074073792,
       "num_tokens": 249818.0,
       "step": 37
     },
     {
-      "entropy": 0.3000646084547043,
       "epoch": 0.9620253164556962,
-      "grad_norm": 1.9470438957214355,
       "learning_rate": 0.00012052153421956342,
-      "loss": 5.12597131729126,
-      "mean_token_accuracy": 0.8450468629598618,
       "num_tokens": 256043.0,
       "step": 38
     },
     {
-      "entropy": 0.2730342224240303,
       "epoch": 0.9873417721518988,
-      "grad_norm": 1.3998013734817505,
       "learning_rate": 0.00011645945902807341,
-      "loss": 5.182764530181885,
-      "mean_token_accuracy": 0.8505663871765137,
       "num_tokens": 263628.0,
       "step": 39
     },
     {
-      "entropy": 0.31550464034080505,
       "epoch": 1.0,
-      "grad_norm": 1.9481233358383179,
       "learning_rate": 0.00011236926312693479,
-      "loss": 5.046781063079834,
-      "mean_token_accuracy": 0.8476890921592712,
       "num_tokens": 266935.0,
       "step": 40
     },
@@ -415,9 +415,9 @@
       "eval_loss": NaN,
       "eval_mean_token_accuracy": 0.0,
       "eval_num_tokens": 266935.0,
-      "eval_runtime": 9.0564,
-      "eval_samples_per_second": 24.513,
-      "eval_steps_per_second": 6.184,
       "step": 40
     }
   ],
@@ -447,7 +447,7 @@
       "attributes": {}
     }
   },
-  "total_flos": 542852652204032.0,
   "train_batch_size": 2,
   "trial_name": null,
   "trial_params": null

     {
       "entropy": 0.8529030084609985,
       "epoch": 0.02531645569620253,
+      "grad_norm": 99.2493667602539,
       "learning_rate": 0.0,
       "loss": 15.181689262390137,
       "mean_token_accuracy": 0.28961846977472305,
     {
       "entropy": 0.8989771902561188,
       "epoch": 0.05063291139240506,
+      "grad_norm": 94.88607025146484,
       "learning_rate": 5e-05,
       "loss": 14.95038890838623,
       "mean_token_accuracy": 0.27897340804338455,
       "step": 2
     },
     {
+      "entropy": 1.1254765689373016,
       "epoch": 0.0759493670886076,
+      "grad_norm": 64.76227569580078,
       "learning_rate": 0.0001,
+      "loss": 12.595176696777344,
+      "mean_token_accuracy": 0.35700511932373047,
       "num_tokens": 21366.0,
       "step": 3
     },
     {
+      "entropy": 1.0708639919757843,
       "epoch": 0.10126582278481013,
+      "grad_norm": 36.60965347290039,
       "learning_rate": 0.00015000000000000001,
+      "loss": 9.597381591796875,
+      "mean_token_accuracy": 0.5474753379821777,
       "num_tokens": 28250.0,
       "step": 4
     },
     {
+      "entropy": 1.184021770954132,
       "epoch": 0.12658227848101267,
+      "grad_norm": 29.169525146484375,
       "learning_rate": 0.0002,
+      "loss": 8.115850448608398,
+      "mean_token_accuracy": 0.567672997713089,
       "num_tokens": 34914.0,
       "step": 5
     },
     {
+      "entropy": 0.7149983793497086,
       "epoch": 0.1518987341772152,
+      "grad_norm": 13.915483474731445,
       "learning_rate": 0.0001999145758387301,
+      "loss": 7.248701095581055,
+      "mean_token_accuracy": 0.5984602570533752,
       "num_tokens": 42532.0,
       "step": 6
     },
     {
+      "entropy": 1.2620182633399963,
       "epoch": 0.17721518987341772,
+      "grad_norm": 44.453094482421875,
       "learning_rate": 0.000199658449300667,
+      "loss": 6.674359321594238,
+      "mean_token_accuracy": 0.6354625821113586,
       "num_tokens": 49872.0,
       "step": 7
     },
     {
+      "entropy": 1.2574806809425354,
       "epoch": 0.20253164556962025,
+      "grad_norm": 17.659940719604492,
       "learning_rate": 0.0001992320579737045,
+      "loss": 6.544322967529297,
+      "mean_token_accuracy": 0.626764640212059,
       "num_tokens": 55639.0,
       "step": 8
     },
     {
+      "entropy": 1.1509844064712524,
       "epoch": 0.22784810126582278,
+      "grad_norm": 8.395023345947266,
       "learning_rate": 0.00019863613034027224,
+      "loss": 6.157719135284424,
+      "mean_token_accuracy": 0.669881597161293,
       "num_tokens": 61649.0,
       "step": 9
     },
     {
+      "entropy": 1.0907686650753021,
       "epoch": 0.25316455696202533,
+      "grad_norm": 4.606855392456055,
       "learning_rate": 0.00019787168453273544,
+      "loss": 5.892803192138672,
+      "mean_token_accuracy": 0.6802202314138412,
       "num_tokens": 69689.0,
       "step": 10
     },
     {
+      "entropy": 1.1551010608673096,
       "epoch": 0.27848101265822783,
+      "grad_norm": 14.0298490524292,
       "learning_rate": 0.00019694002659393305,
+      "loss": 5.6713948249816895,
+      "mean_token_accuracy": 0.6972689032554626,
       "num_tokens": 77528.0,
       "step": 11
     },
     {
+      "entropy": 0.8588106781244278,
       "epoch": 0.3037974683544304,
+      "grad_norm": 4.56960916519165,
       "learning_rate": 0.0001958427482458253,
+      "loss": 5.428313255310059,
+      "mean_token_accuracy": 0.7234913557767868,
       "num_tokens": 85064.0,
       "step": 12
     },
     {
+      "entropy": 0.705646425485611,
       "epoch": 0.3291139240506329,
+      "grad_norm": 3.344228744506836,
       "learning_rate": 0.00019458172417006347,
+      "loss": 5.444645404815674,
+      "mean_token_accuracy": 0.8130310028791428,
       "num_tokens": 91009.0,
       "step": 13
     },
     {
+      "entropy": 0.7050664573907852,
       "epoch": 0.35443037974683544,
+      "grad_norm": 3.312098979949951,
       "learning_rate": 0.0001931591088051279,
+      "loss": 5.357623100280762,
+      "mean_token_accuracy": 0.8219081908464432,
       "num_tokens": 97909.0,
       "step": 14
     },
     {
+      "entropy": 0.6056851297616959,
       "epoch": 0.379746835443038,
+      "grad_norm": 4.054323196411133,
       "learning_rate": 0.00019157733266550575,
+      "loss": 5.329057216644287,
+      "mean_token_accuracy": 0.8183465003967285,
       "num_tokens": 104187.0,
       "step": 15
     },
     {
+      "entropy": 0.46965425461530685,
       "epoch": 0.4050632911392405,
+      "grad_norm": 3.1617722511291504,
       "learning_rate": 0.0001898390981891979,
+      "loss": 5.159331321716309,
+      "mean_token_accuracy": 0.8249307423830032,
       "num_tokens": 110300.0,
       "step": 16
     },
     {
+      "entropy": 0.4227886497974396,
       "epoch": 0.43037974683544306,
+      "grad_norm": 6.028562545776367,
       "learning_rate": 0.0001879473751206489,
+      "loss": 5.351903915405273,
+      "mean_token_accuracy": 0.8241183459758759,
       "num_tokens": 117157.0,
       "step": 17
     },
     {
+      "entropy": 0.3527509272098541,
       "epoch": 0.45569620253164556,
+      "grad_norm": 3.293024778366089,
       "learning_rate": 0.00018590539543698854,
+      "loss": 4.932947158813477,
+      "mean_token_accuracy": 0.8373657613992691,
       "num_tokens": 123378.0,
       "step": 18
     },
     {
+      "entropy": 0.33450157940387726,
       "epoch": 0.4810126582278481,
+      "grad_norm": 1.6703517436981201,
       "learning_rate": 0.00018371664782625287,
+      "loss": 5.134032249450684,
+      "mean_token_accuracy": 0.8269297778606415,
       "num_tokens": 129042.0,
       "step": 19
     },
     {
+      "entropy": 0.3399146646261215,
       "epoch": 0.5063291139240507,
+      "grad_norm": 2.0219178199768066,
       "learning_rate": 0.0001813848717270195,
+      "loss": 5.027883052825928,
+      "mean_token_accuracy": 0.8322668075561523,
       "num_tokens": 135747.0,
       "step": 20
     },
     {
+      "entropy": 0.29102808609604836,
       "epoch": 0.5316455696202531,
+      "grad_norm": 1.350559949874878,
       "learning_rate": 0.00017891405093963938,
+      "loss": 4.954178810119629,
+      "mean_token_accuracy": 0.8439806550741196,
       "num_tokens": 141848.0,
       "step": 21
     },
     {
+      "entropy": 0.3136157840490341,
       "epoch": 0.5569620253164557,
+      "grad_norm": 1.3057941198349,
       "learning_rate": 0.00017630840681998066,
+      "loss": 5.179681301116943,
+      "mean_token_accuracy": 0.8438494354486465,
       "num_tokens": 147566.0,
       "step": 22
     },
     {
+      "entropy": 0.2950733080506325,
       "epoch": 0.5822784810126582,
+      "grad_norm": 1.1281788349151611,
       "learning_rate": 0.00017357239106731317,
+      "loss": 5.171978950500488,
+      "mean_token_accuracy": 0.8462631553411484,
       "num_tokens": 154620.0,
       "step": 23
     },
     {
+      "entropy": 0.27676206082105637,
       "epoch": 0.6075949367088608,
+      "grad_norm": 1.311663269996643,
       "learning_rate": 0.00017071067811865476,
+      "loss": 5.077099800109863,
+      "mean_token_accuracy": 0.8611445128917694,
       "num_tokens": 161701.0,
       "step": 24
     },
     {
+      "entropy": 0.30840296298265457,
       "epoch": 0.6329113924050633,
+      "grad_norm": 2.281773090362549,
       "learning_rate": 0.00016772815716257412,
+      "loss": 5.118645668029785,
+      "mean_token_accuracy": 0.8323634117841721,
       "num_tokens": 168435.0,
       "step": 25
     },
     {
+      "entropy": 0.29603931307792664,
       "epoch": 0.6582278481012658,
+      "grad_norm": 1.230368971824646,
       "learning_rate": 0.00016462992378609407,
+      "loss": 5.176095962524414,
+      "mean_token_accuracy": 0.839859664440155,
       "num_tokens": 175650.0,
       "step": 26
     },
     {
+      "entropy": 0.24447191506624222,
       "epoch": 0.6835443037974683,
+      "grad_norm": 1.7844265699386597,
       "learning_rate": 0.0001614212712689668,
+      "loss": 5.146295547485352,
+      "mean_token_accuracy": 0.8507861793041229,
       "num_tokens": 181490.0,
       "step": 27
     },
     {
+      "entropy": 0.23978285491466522,
       "epoch": 0.7088607594936709,
+      "grad_norm": 1.4895042181015015,
       "learning_rate": 0.00015810768154019385,
+      "loss": 5.0867919921875,
+      "mean_token_accuracy": 0.8498348891735077,
       "num_tokens": 188582.0,
       "step": 28
     },
     {
+      "entropy": 0.1993725746870041,
       "epoch": 0.7341772151898734,
+      "grad_norm": 1.2189953327178955,
       "learning_rate": 0.00015469481581224272,
+      "loss": 5.131974697113037,
+      "mean_token_accuracy": 0.8698419630527496,
       "num_tokens": 193783.0,
       "step": 29
     },
     {
+      "entropy": 0.24254225194454193,
       "epoch": 0.759493670886076,
+      "grad_norm": 3.0398507118225098,
       "learning_rate": 0.00015118850490896012,
+      "loss": 5.307463645935059,
+      "mean_token_accuracy": 0.8126509040594101,
       "num_tokens": 200686.0,
       "step": 30
     },
     {
+      "entropy": 0.18977511674165726,
       "epoch": 0.7848101265822784,
+      "grad_norm": 1.7506808042526245,
       "learning_rate": 0.00014759473930370736,
+      "loss": 5.047242164611816,
+      "mean_token_accuracy": 0.8546684086322784,
       "num_tokens": 208440.0,
       "step": 31
     },
     {
+      "entropy": 0.2233816385269165,
       "epoch": 0.810126582278481,
+      "grad_norm": 1.0086487531661987,
       "learning_rate": 0.00014391965888473703,
+      "loss": 5.027800559997559,
+      "mean_token_accuracy": 0.8576832860708237,
       "num_tokens": 215131.0,
       "step": 32
     },
     {
+      "entropy": 0.19872410595417023,
       "epoch": 0.8354430379746836,
+      "grad_norm": 1.6391977071762085,
       "learning_rate": 0.00014016954246529696,
+      "loss": 5.0654706954956055,
+      "mean_token_accuracy": 0.8516139537096024,
       "num_tokens": 222195.0,
       "step": 33
     },
     {
+      "entropy": 0.20351751521229744,
       "epoch": 0.8607594936708861,
+      "grad_norm": 1.5714291334152222,
       "learning_rate": 0.00013635079705638298,
+      "loss": 5.170158386230469,
+      "mean_token_accuracy": 0.8566406965255737,
       "num_tokens": 229926.0,
       "step": 34
     },
     {
+      "entropy": 0.21749146282672882,
       "epoch": 0.8860759493670886,
+      "grad_norm": 1.6517112255096436,
       "learning_rate": 0.00013246994692046836,
+      "loss": 5.215228080749512,
+      "mean_token_accuracy": 0.8288360238075256,
       "num_tokens": 235766.0,
       "step": 35
     },
     {
+      "entropy": 0.2011272795498371,
       "epoch": 0.9113924050632911,
+      "grad_norm": 1.4017918109893799,
       "learning_rate": 0.00012853362242491053,
+      "loss": 5.1320648193359375,
+      "mean_token_accuracy": 0.8528883457183838,
       "num_tokens": 242731.0,
       "step": 36
     },
     {
+      "entropy": 0.21461713686585426,
       "epoch": 0.9367088607594937,
+      "grad_norm": 1.77144193649292,
       "learning_rate": 0.00012454854871407994,
+      "loss": 5.11080265045166,
+      "mean_token_accuracy": 0.8521855026483536,
       "num_tokens": 249818.0,
       "step": 37
     },
     {
+      "entropy": 0.19789887592196465,
       "epoch": 0.9620253164556962,
+      "grad_norm": 3.2191762924194336,
       "learning_rate": 0.00012052153421956342,
+      "loss": 5.070086479187012,
+      "mean_token_accuracy": 0.848893016576767,
       "num_tokens": 256043.0,
       "step": 38
     },
     {
+      "entropy": 0.1783522628247738,
       "epoch": 0.9873417721518988,
+      "grad_norm": 1.3007391691207886,
       "learning_rate": 0.00011645945902807341,
+      "loss": 5.1378865242004395,
+      "mean_token_accuracy": 0.8583788871765137,
       "num_tokens": 263628.0,
       "step": 39
     },
     {
+      "entropy": 0.21517179906368256,
       "epoch": 1.0,
+      "grad_norm": 2.357231378555298,
       "learning_rate": 0.00011236926312693479,
+      "loss": 5.041143894195557,
+      "mean_token_accuracy": 0.8405462205410004,
       "num_tokens": 266935.0,
       "step": 40
     },
       "eval_loss": NaN,
       "eval_mean_token_accuracy": 0.0,
       "eval_num_tokens": 266935.0,
+      "eval_runtime": 5.9781,
+      "eval_samples_per_second": 37.136,
+      "eval_steps_per_second": 9.368,
       "step": 40
     }
   ],
       "attributes": {}
     }
   },
+  "total_flos": 560391459438592.0,
   "train_batch_size": 2,
   "trial_name": null,
   "trial_params": null

checkpoint-40/training_args.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:511c0cc2bbf14fcbcb4a1223f6f454f90d8d9544a8bb1a60eb8f5e59686fc6ff
 size 5496

 version https://git-lfs.github.com/spec/v1
+oid sha256:da3e7916a9c3a9bdc1d65a3a86e68c6c06ab04fd280e02675e7357542392c8fa
 size 5496

checkpoint-80/adapter_config.json CHANGED Viewed

@@ -32,21 +32,23 @@
   "qalora_group_size": 16,
   "r": 32,
   "rank_pattern": {
-    "input_linear": 32,
     "k_proj": 32,
     "o_proj": 32,
-    "output_linear": 32,
     "q_proj": 32,
     "v_proj": 32
   },
   "revision": null,
   "target_modules": [
-    "k_proj",
-    "v_proj",
-    "output_linear",
-    "input_linear",
     "o_proj",
-    "q_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

   "qalora_group_size": 16,
   "r": 32,
   "rank_pattern": {
+    "down_proj": 32,
+    "gate_proj": 32,
     "k_proj": 32,
     "o_proj": 32,
     "q_proj": 32,
+    "up_proj": 32,
     "v_proj": 32
   },
   "revision": null,
   "target_modules": [
+    "up_proj",
     "o_proj",
+    "q_proj",
+    "gate_proj",
+    "down_proj",
+    "v_proj",
+    "k_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

checkpoint-80/adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d64e4ba747c90d89c6855d7abd07cdcaf64b5178f7e1843492ad55fcb5e75bd4
-size 23885040

 version https://git-lfs.github.com/spec/v1
+oid sha256:ecd2ea37998d533a6a40fa093b8f1945d89505a7c42b4d282dfbfd7a4f0a27b4
+size 56936568

checkpoint-80/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:493b745b14fad337f163b7a46b108bba6556cf4b3d87d820bb0127ae8be839cf
-size 47898234

 version https://git-lfs.github.com/spec/v1
+oid sha256:c9fa050e0dec60e638f2efb84ba7b8faab798c03bf0ba5d5a65f12f112ecd360
+size 114100074

checkpoint-80/rng_state_0.pth CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b8b2455f9f522181173824e02d9df3d20d10252e0f5891c23c91f3ff3987c26f
 size 15024

 version https://git-lfs.github.com/spec/v1
+oid sha256:5dc2474f23a5a53cdabfdf34d0e28011700982aeb8610ac607384a176b69a792
 size 15024

checkpoint-80/rng_state_1.pth CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6114a4fea51222196849cf975221a083e711a7d2a7cd0063fa3775d01926ecef
 size 15024

 version https://git-lfs.github.com/spec/v1
+oid sha256:dac1653ac2bd7627a938bbf898819943e746d01bb0b238c150ec86728ce34b35
 size 15024

checkpoint-80/rng_state_2.pth CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0dd307a1e38871568f9d304d068150a5eb9dfb9ba674dd10e91d50ee44aba62f
 size 15024

 version https://git-lfs.github.com/spec/v1
+oid sha256:73080767711217104a0efa90f813b1457b44079ab028f84c8b79268634e6d313
 size 15024

checkpoint-80/rng_state_3.pth CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bc86639b4ecc3dd246ec9caffbc4e9262b4851d6a71cc486b1d159aa78aa8d86
 size 15024

 version https://git-lfs.github.com/spec/v1
+oid sha256:f90541ce67a17c479ed8c2931a361d029cd473f76ad7e415bdfc30d832f320bd
 size 15024

checkpoint-80/trainer_state.json CHANGED Viewed

@@ -12,7 +12,7 @@
     {
       "entropy": 0.8529030084609985,
       "epoch": 0.02531645569620253,
-      "grad_norm": 78.52010345458984,
       "learning_rate": 0.0,
       "loss": 15.181689262390137,
       "mean_token_accuracy": 0.28961846977472305,
@@ -22,7 +22,7 @@
     {
       "entropy": 0.8989771902561188,
       "epoch": 0.05063291139240506,
-      "grad_norm": 75.26119232177734,
       "learning_rate": 5e-05,
       "loss": 14.95038890838623,
       "mean_token_accuracy": 0.27897340804338455,
@@ -30,382 +30,382 @@
       "step": 2
     },
     {
-      "entropy": 0.9821003973484039,
       "epoch": 0.0759493670886076,
-      "grad_norm": 74.54902648925781,
       "learning_rate": 0.0001,
-      "loss": 13.742369651794434,
-      "mean_token_accuracy": 0.2827275022864342,
       "num_tokens": 21366.0,
       "step": 3
     },
     {
-      "entropy": 1.0873801112174988,
       "epoch": 0.10126582278481013,
-      "grad_norm": 34.19524383544922,
       "learning_rate": 0.00015000000000000001,
-      "loss": 11.25832462310791,
-      "mean_token_accuracy": 0.44820327311754227,
       "num_tokens": 28250.0,
       "step": 4
     },
     {
-      "entropy": 1.0790044069290161,
       "epoch": 0.12658227848101267,
-      "grad_norm": 28.061630249023438,
       "learning_rate": 0.0002,
-      "loss": 9.99978256225586,
-      "mean_token_accuracy": 0.5489031970500946,
       "num_tokens": 34914.0,
       "step": 5
     },
     {
-      "entropy": 1.1546767055988312,
       "epoch": 0.1518987341772152,
-      "grad_norm": 24.789119720458984,
       "learning_rate": 0.0001999145758387301,
-      "loss": 8.632344245910645,
-      "mean_token_accuracy": 0.554060235619545,
       "num_tokens": 42532.0,
       "step": 6
     },
     {
-      "entropy": 1.0596064329147339,
       "epoch": 0.17721518987341772,
-      "grad_norm": 12.310446739196777,
       "learning_rate": 0.000199658449300667,
-      "loss": 7.437180995941162,
-      "mean_token_accuracy": 0.5873144865036011,
       "num_tokens": 49872.0,
       "step": 7
     },
     {
-      "entropy": 0.7889958769083023,
       "epoch": 0.20253164556962025,
-      "grad_norm": 10.70737361907959,
       "learning_rate": 0.0001992320579737045,
-      "loss": 7.076646327972412,
-      "mean_token_accuracy": 0.5939096212387085,
       "num_tokens": 55639.0,
       "step": 8
     },
     {
-      "entropy": 0.9904351979494095,
       "epoch": 0.22784810126582278,
-      "grad_norm": 39.2529296875,
       "learning_rate": 0.00019863613034027224,
-      "loss": 7.060567855834961,
-      "mean_token_accuracy": 0.6403393149375916,
       "num_tokens": 61649.0,
       "step": 9
     },
     {
-      "entropy": 1.3839358687400818,
       "epoch": 0.25316455696202533,
-      "grad_norm": 15.882004737854004,
       "learning_rate": 0.00019787168453273544,
-      "loss": 6.710319519042969,
-      "mean_token_accuracy": 0.603007972240448,
       "num_tokens": 69689.0,
       "step": 10
     },
     {
-      "entropy": 1.1738347113132477,
       "epoch": 0.27848101265822783,
-      "grad_norm": 8.31102180480957,
       "learning_rate": 0.00019694002659393305,
-      "loss": 6.315446853637695,
-      "mean_token_accuracy": 0.627836138010025,
       "num_tokens": 77528.0,
       "step": 11
     },
     {
-      "entropy": 1.2701810002326965,
       "epoch": 0.3037974683544304,
-      "grad_norm": 6.591884136199951,
       "learning_rate": 0.0001958427482458253,
-      "loss": 6.042618751525879,
-      "mean_token_accuracy": 0.6690841615200043,
       "num_tokens": 85064.0,
       "step": 12
     },
     {
-      "entropy": 1.2295758128166199,
       "epoch": 0.3291139240506329,
-      "grad_norm": 15.478667259216309,
       "learning_rate": 0.00019458172417006347,
-      "loss": 6.213565826416016,
-      "mean_token_accuracy": 0.6405076384544373,
       "num_tokens": 91009.0,
       "step": 13
     },
     {
-      "entropy": 1.268456757068634,
       "epoch": 0.35443037974683544,
-      "grad_norm": 7.625810146331787,
       "learning_rate": 0.0001931591088051279,
-      "loss": 6.13183069229126,
-      "mean_token_accuracy": 0.6365684270858765,
       "num_tokens": 97909.0,
       "step": 14
     },
     {
-      "entropy": 1.2001140415668488,
       "epoch": 0.379746835443038,
-      "grad_norm": 7.8025221824646,
       "learning_rate": 0.00019157733266550575,
-      "loss": 6.064075469970703,
-      "mean_token_accuracy": 0.6330698281526566,
       "num_tokens": 104187.0,
       "step": 15
     },
     {
-      "entropy": 1.066677764058113,
       "epoch": 0.4050632911392405,
-      "grad_norm": 8.150992393493652,
       "learning_rate": 0.0001898390981891979,
-      "loss": 5.798349380493164,
-      "mean_token_accuracy": 0.646238312125206,
       "num_tokens": 110300.0,
       "step": 16
     },
     {
-      "entropy": 1.0704185664653778,
       "epoch": 0.43037974683544306,
-      "grad_norm": 3.722452402114868,
       "learning_rate": 0.0001879473751206489,
-      "loss": 5.87784481048584,
-      "mean_token_accuracy": 0.6626823246479034,
       "num_tokens": 117157.0,
       "step": 17
     },
     {
-      "entropy": 1.1619559824466705,
       "epoch": 0.45569620253164556,
-      "grad_norm": 4.019715785980225,
       "learning_rate": 0.00018590539543698854,
-      "loss": 5.398146152496338,
-      "mean_token_accuracy": 0.714724525809288,
       "num_tokens": 123378.0,
       "step": 18
     },
     {
-      "entropy": 1.072197139263153,
       "epoch": 0.4810126582278481,
-      "grad_norm": 3.3014461994171143,
       "learning_rate": 0.00018371664782625287,
-      "loss": 5.488443851470947,
-      "mean_token_accuracy": 0.8233065903186798,
       "num_tokens": 129042.0,
       "step": 19
     },
     {
-      "entropy": 0.9806628823280334,
       "epoch": 0.5063291139240507,
-      "grad_norm": 2.70117449760437,
       "learning_rate": 0.0001813848717270195,
-      "loss": 5.3289875984191895,
-      "mean_token_accuracy": 0.8143462389707565,
       "num_tokens": 135747.0,
       "step": 20
     },
     {
-      "entropy": 0.8040178865194321,
       "epoch": 0.5316455696202531,
-      "grad_norm": 3.4564270973205566,
       "learning_rate": 0.00017891405093963938,
-      "loss": 5.2513909339904785,
-      "mean_token_accuracy": 0.8225801885128021,
       "num_tokens": 141848.0,
       "step": 21
     },
     {
-      "entropy": 0.7444496601819992,
       "epoch": 0.5569620253164557,
-      "grad_norm": 2.3326470851898193,
       "learning_rate": 0.00017630840681998066,
-      "loss": 5.4000067710876465,
-      "mean_token_accuracy": 0.8329752385616302,
       "num_tokens": 147566.0,
       "step": 22
     },
     {
-      "entropy": 0.6936303377151489,
       "epoch": 0.5822784810126582,
-      "grad_norm": 2.478424072265625,
       "learning_rate": 0.00017357239106731317,
-      "loss": 5.346803665161133,
-      "mean_token_accuracy": 0.8282431960105896,
       "num_tokens": 154620.0,
       "step": 23
     },
     {
-      "entropy": 0.6478566229343414,
       "epoch": 0.6075949367088608,
-      "grad_norm": 2.6104896068573,
       "learning_rate": 0.00017071067811865476,
-      "loss": 5.191404342651367,
-      "mean_token_accuracy": 0.8425344824790955,
       "num_tokens": 161701.0,
       "step": 24
     },
     {
-      "entropy": 0.6078712940216064,
       "epoch": 0.6329113924050633,
-      "grad_norm": 4.111148357391357,
       "learning_rate": 0.00016772815716257412,
-      "loss": 5.243197917938232,
-      "mean_token_accuracy": 0.8070437759160995,
       "num_tokens": 168435.0,
       "step": 25
     },
     {
-      "entropy": 0.5901128649711609,
       "epoch": 0.6582278481012658,
-      "grad_norm": 2.244276285171509,
       "learning_rate": 0.00016462992378609407,
-      "loss": 5.272450923919678,
-      "mean_token_accuracy": 0.8324518501758575,
       "num_tokens": 175650.0,
       "step": 26
     },
     {
-      "entropy": 0.5017133206129074,
       "epoch": 0.6835443037974683,
-      "grad_norm": 2.1174073219299316,
       "learning_rate": 0.0001614212712689668,
-      "loss": 5.215684413909912,
-      "mean_token_accuracy": 0.8321826756000519,
       "num_tokens": 181490.0,
       "step": 27
     },
     {
-      "entropy": 0.4748859405517578,
       "epoch": 0.7088607594936709,
-      "grad_norm": 2.087001323699951,
       "learning_rate": 0.00015810768154019385,
-      "loss": 5.15059757232666,
-      "mean_token_accuracy": 0.8388571739196777,
       "num_tokens": 188582.0,
       "step": 28
     },
     {
-      "entropy": 0.3888755962252617,
       "epoch": 0.7341772151898734,
-      "grad_norm": 1.487061858177185,
       "learning_rate": 0.00015469481581224272,
-      "loss": 5.174810409545898,
-      "mean_token_accuracy": 0.8587576746940613,
       "num_tokens": 193783.0,
       "step": 29
     },
     {
-      "entropy": 0.4288812130689621,
       "epoch": 0.759493670886076,
-      "grad_norm": 2.868377685546875,
       "learning_rate": 0.00015118850490896012,
-      "loss": 5.343541145324707,
-      "mean_token_accuracy": 0.8163273632526398,
       "num_tokens": 200686.0,
       "step": 30
     },
     {
-      "entropy": 0.39349576085805893,
       "epoch": 0.7848101265822784,
-      "grad_norm": 1.7554408311843872,
       "learning_rate": 0.00014759473930370736,
-      "loss": 5.089621543884277,
-      "mean_token_accuracy": 0.8510452210903168,
       "num_tokens": 208440.0,
       "step": 31
     },
     {
-      "entropy": 0.37267740815877914,
       "epoch": 0.810126582278481,
-      "grad_norm": 1.210075855255127,
       "learning_rate": 0.00014391965888473703,
-      "loss": 5.0665974617004395,
-      "mean_token_accuracy": 0.8576832711696625,
       "num_tokens": 215131.0,
       "step": 32
     },
     {
-      "entropy": 0.3740547075867653,
       "epoch": 0.8354430379746836,
-      "grad_norm": 2.1259520053863525,
       "learning_rate": 0.00014016954246529696,
-      "loss": 5.144742012023926,
-      "mean_token_accuracy": 0.8442028760910034,
       "num_tokens": 222195.0,
       "step": 33
     },
     {
-      "entropy": 0.33895356953144073,
       "epoch": 0.8607594936708861,
-      "grad_norm": 2.1364142894744873,
       "learning_rate": 0.00013635079705638298,
-      "loss": 5.227801322937012,
-      "mean_token_accuracy": 0.8455548137426376,
       "num_tokens": 229926.0,
       "step": 34
     },
     {
-      "entropy": 0.3552927002310753,
       "epoch": 0.8860759493670886,
-      "grad_norm": 2.4019367694854736,
       "learning_rate": 0.00013246994692046836,
-      "loss": 5.275204181671143,
-      "mean_token_accuracy": 0.8322960436344147,
       "num_tokens": 235766.0,
       "step": 35
     },
     {
-      "entropy": 0.3192586973309517,
       "epoch": 0.9113924050632911,
-      "grad_norm": 1.8943588733673096,
       "learning_rate": 0.00012853362242491053,
-      "loss": 5.1949639320373535,
-      "mean_token_accuracy": 0.8491553515195847,
       "num_tokens": 242731.0,
       "step": 36
     },
     {
-      "entropy": 0.31374089419841766,
       "epoch": 0.9367088607594937,
-      "grad_norm": 1.5797899961471558,
       "learning_rate": 0.00012454854871407994,
-      "loss": 5.137860298156738,
-      "mean_token_accuracy": 0.8447228074073792,
       "num_tokens": 249818.0,
       "step": 37
     },
     {
-      "entropy": 0.3000646084547043,
       "epoch": 0.9620253164556962,
-      "grad_norm": 1.9470438957214355,
       "learning_rate": 0.00012052153421956342,
-      "loss": 5.12597131729126,
-      "mean_token_accuracy": 0.8450468629598618,
       "num_tokens": 256043.0,
       "step": 38
     },
     {
-      "entropy": 0.2730342224240303,
       "epoch": 0.9873417721518988,
-      "grad_norm": 1.3998013734817505,
       "learning_rate": 0.00011645945902807341,
-      "loss": 5.182764530181885,
-      "mean_token_accuracy": 0.8505663871765137,
       "num_tokens": 263628.0,
       "step": 39
     },
     {
-      "entropy": 0.31550464034080505,
       "epoch": 1.0,
-      "grad_norm": 1.9481233358383179,
       "learning_rate": 0.00011236926312693479,
-      "loss": 5.046781063079834,
-      "mean_token_accuracy": 0.8476890921592712,
       "num_tokens": 266935.0,
       "step": 40
     },
@@ -415,408 +415,408 @@
       "eval_loss": NaN,
       "eval_mean_token_accuracy": 0.0,
       "eval_num_tokens": 266935.0,
-      "eval_runtime": 9.0564,
-      "eval_samples_per_second": 24.513,
-      "eval_steps_per_second": 6.184,
       "step": 40
     },
     {
-      "entropy": 0.26519979164004326,
       "epoch": 1.0253164556962024,
-      "grad_norm": 1.8145737648010254,
       "learning_rate": 0.00010825793454723325,
-      "loss": 5.0367913246154785,
-      "mean_token_accuracy": 0.8414442837238312,
       "num_tokens": 272689.0,
       "step": 41
     },
     {
-      "entropy": 0.2995038107037544,
       "epoch": 1.0506329113924051,
-      "grad_norm": 1.809834361076355,
       "learning_rate": 0.00010413249742488131,
-      "loss": 5.1586761474609375,
-      "mean_token_accuracy": 0.8278436958789825,
       "num_tokens": 279320.0,
       "step": 42
     },
     {
-      "entropy": 0.2940419465303421,
       "epoch": 1.0759493670886076,
-      "grad_norm": 1.856474757194519,
       "learning_rate": 0.0001,
-      "loss": 4.957198619842529,
-      "mean_token_accuracy": 0.8536823838949203,
       "num_tokens": 285858.0,
       "step": 43
     },
     {
-      "entropy": 0.29139814525842667,
       "epoch": 1.1012658227848102,
-      "grad_norm": 1.7269527912139893,
       "learning_rate": 9.586750257511867e-05,
-      "loss": 5.181451320648193,
-      "mean_token_accuracy": 0.8414545953273773,
       "num_tokens": 293542.0,
       "step": 44
     },
     {
-      "entropy": 0.27789001166820526,
       "epoch": 1.1265822784810127,
-      "grad_norm": 1.2102724313735962,
       "learning_rate": 9.174206545276677e-05,
-      "loss": 5.100370407104492,
-      "mean_token_accuracy": 0.8416209667921066,
       "num_tokens": 300567.0,
       "step": 45
     },
     {
-      "entropy": 0.297809861600399,
       "epoch": 1.1518987341772151,
-      "grad_norm": 1.4553245306015015,
       "learning_rate": 8.763073687306524e-05,
-      "loss": 5.187261581420898,
-      "mean_token_accuracy": 0.8298662006855011,
       "num_tokens": 308202.0,
       "step": 46
     },
     {
-      "entropy": 0.29849978536367416,
       "epoch": 1.1772151898734178,
-      "grad_norm": 1.2045928239822388,
       "learning_rate": 8.35405409719266e-05,
-      "loss": 5.215890407562256,
-      "mean_token_accuracy": 0.8469023555517197,
       "num_tokens": 314689.0,
       "step": 47
     },
     {
-      "entropy": 0.3055413067340851,
       "epoch": 1.2025316455696202,
-      "grad_norm": 1.1849567890167236,
       "learning_rate": 7.947846578043659e-05,
-      "loss": 5.074217796325684,
-      "mean_token_accuracy": 0.8524564653635025,
       "num_tokens": 320323.0,
       "step": 48
     },
     {
-      "entropy": 0.2914985194802284,
       "epoch": 1.2278481012658227,
-      "grad_norm": 1.674543857574463,
       "learning_rate": 7.54514512859201e-05,
-      "loss": 5.055902481079102,
-      "mean_token_accuracy": 0.8409789949655533,
       "num_tokens": 327016.0,
       "step": 49
     },
     {
-      "entropy": 0.3027197793126106,
       "epoch": 1.2531645569620253,
-      "grad_norm": 1.4757080078125,
       "learning_rate": 7.146637757508949e-05,
-      "loss": 5.192047595977783,
-      "mean_token_accuracy": 0.8470398932695389,
       "num_tokens": 333272.0,
       "step": 50
     },
     {
-      "entropy": 0.28784017637372017,
       "epoch": 1.2784810126582278,
-      "grad_norm": 0.8825265169143677,
       "learning_rate": 6.753005307953167e-05,
-      "loss": 5.079134941101074,
-      "mean_token_accuracy": 0.8682127892971039,
       "num_tokens": 340919.0,
       "step": 51
     },
     {
-      "entropy": 0.3075410723686218,
       "epoch": 1.3037974683544304,
-      "grad_norm": 1.0990455150604248,
       "learning_rate": 6.3649202943617e-05,
-      "loss": 5.145890235900879,
-      "mean_token_accuracy": 0.8487024903297424,
       "num_tokens": 347473.0,
       "step": 52
     },
     {
-      "entropy": 0.29914823919534683,
       "epoch": 1.3291139240506329,
-      "grad_norm": 1.4213709831237793,
       "learning_rate": 5.983045753470308e-05,
-      "loss": 5.084028720855713,
-      "mean_token_accuracy": 0.8422594368457794,
       "num_tokens": 353979.0,
       "step": 53
     },
     {
-      "entropy": 0.2886178195476532,
       "epoch": 1.3544303797468356,
-      "grad_norm": 1.3735694885253906,
       "learning_rate": 5.608034111526298e-05,
-      "loss": 5.254447937011719,
-      "mean_token_accuracy": 0.8498187363147736,
       "num_tokens": 360939.0,
       "step": 54
     },
     {
-      "entropy": 0.31683316081762314,
       "epoch": 1.379746835443038,
-      "grad_norm": 1.1048169136047363,
       "learning_rate": 5.240526069629265e-05,
-      "loss": 5.1856770515441895,
-      "mean_token_accuracy": 0.8427773416042328,
       "num_tokens": 367684.0,
       "step": 55
     },
     {
-      "entropy": 0.24216660484671593,
       "epoch": 1.4050632911392404,
-      "grad_norm": 1.0284242630004883,
       "learning_rate": 4.8811495091039926e-05,
-      "loss": 5.0062761306762695,
       "mean_token_accuracy": 0.8663530200719833,
       "num_tokens": 374983.0,
       "step": 56
     },
     {
-      "entropy": 0.2965565323829651,
       "epoch": 1.4303797468354431,
-      "grad_norm": 1.2940185070037842,
       "learning_rate": 4.530518418775733e-05,
-      "loss": 5.221802711486816,
-      "mean_token_accuracy": 0.8448100984096527,
       "num_tokens": 380947.0,
       "step": 57
     },
     {
-      "entropy": 0.2857903391122818,
       "epoch": 1.4556962025316456,
-      "grad_norm": 1.4262173175811768,
       "learning_rate": 4.189231845980618e-05,
-      "loss": 5.07797908782959,
-      "mean_token_accuracy": 0.852640375494957,
       "num_tokens": 388400.0,
       "step": 58
     },
     {
-      "entropy": 0.2566057816147804,
       "epoch": 1.481012658227848,
-      "grad_norm": 0.816656231880188,
       "learning_rate": 3.857872873103322e-05,
-      "loss": 4.954668998718262,
       "mean_token_accuracy": 0.8559762090444565,
       "num_tokens": 394297.0,
       "step": 59
     },
     {
-      "entropy": 0.25938503444194794,
       "epoch": 1.5063291139240507,
-      "grad_norm": 1.187715768814087,
       "learning_rate": 3.53700762139059e-05,
-      "loss": 5.22895622253418,
-      "mean_token_accuracy": 0.847022071480751,
       "num_tokens": 401531.0,
       "step": 60
     },
     {
-      "entropy": 0.26262032613158226,
       "epoch": 1.5316455696202531,
-      "grad_norm": 1.491829752922058,
       "learning_rate": 3.227184283742591e-05,
-      "loss": 5.100276947021484,
-      "mean_token_accuracy": 0.8595085442066193,
       "num_tokens": 408001.0,
       "step": 61
     },
     {
-      "entropy": 0.2463800087571144,
       "epoch": 1.5569620253164556,
-      "grad_norm": 1.0418689250946045,
       "learning_rate": 2.9289321881345254e-05,
-      "loss": 5.202584743499756,
       "mean_token_accuracy": 0.8652869164943695,
       "num_tokens": 414848.0,
       "step": 62
     },
     {
-      "entropy": 0.2691369950771332,
       "epoch": 1.5822784810126582,
-      "grad_norm": 1.060405969619751,
       "learning_rate": 2.6427608932686843e-05,
-      "loss": 5.101626396179199,
-      "mean_token_accuracy": 0.8490467667579651,
       "num_tokens": 422561.0,
       "step": 63
     },
     {
-      "entropy": 0.2806598171591759,
       "epoch": 1.6075949367088609,
-      "grad_norm": 1.1078499555587769,
       "learning_rate": 2.3691593180019366e-05,
-      "loss": 5.148905277252197,
-      "mean_token_accuracy": 0.8523232638835907,
       "num_tokens": 428474.0,
       "step": 64
     },
     {
-      "entropy": 0.26543619111180305,
       "epoch": 1.6329113924050633,
-      "grad_norm": 1.503775954246521,
       "learning_rate": 2.1085949060360654e-05,
-      "loss": 5.094422817230225,
-      "mean_token_accuracy": 0.846227616071701,
       "num_tokens": 433867.0,
       "step": 65
     },
     {
-      "entropy": 0.25190359354019165,
       "epoch": 1.6582278481012658,
-      "grad_norm": 1.311462640762329,
       "learning_rate": 1.861512827298051e-05,
-      "loss": 5.036802768707275,
-      "mean_token_accuracy": 0.8547705113887787,
       "num_tokens": 440685.0,
       "step": 66
     },
     {
-      "entropy": 0.25101326778531075,
       "epoch": 1.6835443037974684,
-      "grad_norm": 0.8309802412986755,
       "learning_rate": 1.6283352173747145e-05,
-      "loss": 5.151118278503418,
-      "mean_token_accuracy": 0.8581923246383667,
       "num_tokens": 447730.0,
       "step": 67
     },
     {
-      "entropy": 0.24114958941936493,
       "epoch": 1.7088607594936709,
-      "grad_norm": 0.8836026191711426,
       "learning_rate": 1.4094604563011472e-05,
-      "loss": 5.134224891662598,
-      "mean_token_accuracy": 0.8657366037368774,
       "num_tokens": 454057.0,
       "step": 68
     },
     {
-      "entropy": 0.23641362413764,
       "epoch": 1.7341772151898733,
-      "grad_norm": 1.5712931156158447,
       "learning_rate": 1.2052624879351104e-05,
-      "loss": 5.161583423614502,
-      "mean_token_accuracy": 0.842081293463707,
       "num_tokens": 460989.0,
       "step": 69
     },
     {
-      "entropy": 0.2792535126209259,
       "epoch": 1.759493670886076,
-      "grad_norm": 1.1477248668670654,
       "learning_rate": 1.0160901810802115e-05,
-      "loss": 4.981236934661865,
-      "mean_token_accuracy": 0.8453088253736496,
       "num_tokens": 468216.0,
       "step": 70
     },
     {
-      "entropy": 0.24980313330888748,
       "epoch": 1.7848101265822784,
-      "grad_norm": 1.2068700790405273,
       "learning_rate": 8.422667334494249e-06,
-      "loss": 5.097178936004639,
-      "mean_token_accuracy": 0.8525224328041077,
       "num_tokens": 474276.0,
       "step": 71
     },
     {
-      "entropy": 0.2377922274172306,
       "epoch": 1.810126582278481,
-      "grad_norm": 1.532500982284546,
       "learning_rate": 6.840891194872112e-06,
-      "loss": 5.136972904205322,
-      "mean_token_accuracy": 0.8548215478658676,
       "num_tokens": 481319.0,
       "step": 72
     },
     {
-      "entropy": 0.2445918545126915,
       "epoch": 1.8354430379746836,
-      "grad_norm": 1.1625312566757202,
       "learning_rate": 5.418275829936537e-06,
-      "loss": 5.1382622718811035,
-      "mean_token_accuracy": 0.8484606295824051,
       "num_tokens": 488560.0,
       "step": 73
     },
     {
-      "entropy": 0.26759493350982666,
       "epoch": 1.8607594936708862,
-      "grad_norm": 1.0655369758605957,
       "learning_rate": 4.1572517541747294e-06,
-      "loss": 5.19594144821167,
-      "mean_token_accuracy": 0.8615467995405197,
       "num_tokens": 494057.0,
       "step": 74
     },
     {
-      "entropy": 0.23994890972971916,
       "epoch": 1.8860759493670884,
-      "grad_norm": 1.203637719154358,
       "learning_rate": 3.059973406066963e-06,
-      "loss": 4.939969539642334,
-      "mean_token_accuracy": 0.8601019978523254,
       "num_tokens": 501562.0,
       "step": 75
     },
     {
-      "entropy": 0.25689177215099335,
       "epoch": 1.9113924050632911,
-      "grad_norm": 1.2515466213226318,
       "learning_rate": 2.128315467264552e-06,
-      "loss": 5.10642671585083,
-      "mean_token_accuracy": 0.849346712231636,
       "num_tokens": 509439.0,
       "step": 76
     },
     {
-      "entropy": 0.2606606297194958,
       "epoch": 1.9367088607594938,
-      "grad_norm": 1.1933469772338867,
       "learning_rate": 1.3638696597277679e-06,
-      "loss": 5.1182708740234375,
-      "mean_token_accuracy": 0.8414004147052765,
       "num_tokens": 516267.0,
       "step": 77
     },
     {
-      "entropy": 0.2337033823132515,
       "epoch": 1.9620253164556962,
-      "grad_norm": 1.0293045043945312,
       "learning_rate": 7.679420262954984e-07,
-      "loss": 5.0495500564575195,
-      "mean_token_accuracy": 0.8529785424470901,
       "num_tokens": 523626.0,
       "step": 78
     },
     {
-      "entropy": 0.2539205774664879,
       "epoch": 1.9873417721518987,
-      "grad_norm": 1.125633716583252,
       "learning_rate": 3.415506993330153e-07,
-      "loss": 5.054381370544434,
-      "mean_token_accuracy": 0.850442111492157,
       "num_tokens": 530022.0,
       "step": 79
     },
     {
-      "entropy": 0.27268750965595245,
       "epoch": 2.0,
-      "grad_norm": 2.5797595977783203,
       "learning_rate": 8.542416126989805e-08,
-      "loss": 5.11109733581543,
-      "mean_token_accuracy": 0.8540067970752716,
       "num_tokens": 533275.0,
       "step": 80
     },
@@ -826,9 +826,9 @@
       "eval_loss": NaN,
       "eval_mean_token_accuracy": 0.0,
       "eval_num_tokens": 533275.0,
-      "eval_runtime": 8.6139,
-      "eval_samples_per_second": 25.772,
-      "eval_steps_per_second": 6.501,
       "step": 80
     }
   ],
@@ -858,7 +858,7 @@
       "attributes": {}
     }
   },
-  "total_flos": 1084659452608512.0,
   "train_batch_size": 2,
   "trial_name": null,
   "trial_params": null

     {
       "entropy": 0.8529030084609985,
       "epoch": 0.02531645569620253,
+      "grad_norm": 99.2493667602539,
       "learning_rate": 0.0,
       "loss": 15.181689262390137,
       "mean_token_accuracy": 0.28961846977472305,
     {
       "entropy": 0.8989771902561188,
       "epoch": 0.05063291139240506,
+      "grad_norm": 94.88607025146484,
       "learning_rate": 5e-05,
       "loss": 14.95038890838623,
       "mean_token_accuracy": 0.27897340804338455,
       "step": 2
     },
     {
+      "entropy": 1.1254765689373016,
       "epoch": 0.0759493670886076,
+      "grad_norm": 64.76227569580078,
       "learning_rate": 0.0001,
+      "loss": 12.595176696777344,
+      "mean_token_accuracy": 0.35700511932373047,
       "num_tokens": 21366.0,
       "step": 3
     },
     {
+      "entropy": 1.0708639919757843,
       "epoch": 0.10126582278481013,
+      "grad_norm": 36.60965347290039,
       "learning_rate": 0.00015000000000000001,
+      "loss": 9.597381591796875,
+      "mean_token_accuracy": 0.5474753379821777,
       "num_tokens": 28250.0,
       "step": 4
     },
     {
+      "entropy": 1.184021770954132,
       "epoch": 0.12658227848101267,
+      "grad_norm": 29.169525146484375,
       "learning_rate": 0.0002,
+      "loss": 8.115850448608398,
+      "mean_token_accuracy": 0.567672997713089,
       "num_tokens": 34914.0,
       "step": 5
     },
     {
+      "entropy": 0.7149983793497086,
       "epoch": 0.1518987341772152,
+      "grad_norm": 13.915483474731445,
       "learning_rate": 0.0001999145758387301,
+      "loss": 7.248701095581055,
+      "mean_token_accuracy": 0.5984602570533752,
       "num_tokens": 42532.0,
       "step": 6
     },
     {
+      "entropy": 1.2620182633399963,
       "epoch": 0.17721518987341772,
+      "grad_norm": 44.453094482421875,
       "learning_rate": 0.000199658449300667,
+      "loss": 6.674359321594238,
+      "mean_token_accuracy": 0.6354625821113586,
       "num_tokens": 49872.0,
       "step": 7
     },
     {
+      "entropy": 1.2574806809425354,
       "epoch": 0.20253164556962025,
+      "grad_norm": 17.659940719604492,
       "learning_rate": 0.0001992320579737045,
+      "loss": 6.544322967529297,
+      "mean_token_accuracy": 0.626764640212059,
       "num_tokens": 55639.0,
       "step": 8
     },
     {
+      "entropy": 1.1509844064712524,
       "epoch": 0.22784810126582278,
+      "grad_norm": 8.395023345947266,
       "learning_rate": 0.00019863613034027224,
+      "loss": 6.157719135284424,
+      "mean_token_accuracy": 0.669881597161293,
       "num_tokens": 61649.0,
       "step": 9
     },
     {
+      "entropy": 1.0907686650753021,
       "epoch": 0.25316455696202533,
+      "grad_norm": 4.606855392456055,
       "learning_rate": 0.00019787168453273544,
+      "loss": 5.892803192138672,
+      "mean_token_accuracy": 0.6802202314138412,
       "num_tokens": 69689.0,
       "step": 10
     },
     {
+      "entropy": 1.1551010608673096,
       "epoch": 0.27848101265822783,
+      "grad_norm": 14.0298490524292,
       "learning_rate": 0.00019694002659393305,
+      "loss": 5.6713948249816895,
+      "mean_token_accuracy": 0.6972689032554626,
       "num_tokens": 77528.0,
       "step": 11
     },
     {
+      "entropy": 0.8588106781244278,
       "epoch": 0.3037974683544304,
+      "grad_norm": 4.56960916519165,
       "learning_rate": 0.0001958427482458253,
+      "loss": 5.428313255310059,
+      "mean_token_accuracy": 0.7234913557767868,
       "num_tokens": 85064.0,
       "step": 12
     },
     {
+      "entropy": 0.705646425485611,
       "epoch": 0.3291139240506329,
+      "grad_norm": 3.344228744506836,
       "learning_rate": 0.00019458172417006347,
+      "loss": 5.444645404815674,
+      "mean_token_accuracy": 0.8130310028791428,
       "num_tokens": 91009.0,
       "step": 13
     },
     {
+      "entropy": 0.7050664573907852,
       "epoch": 0.35443037974683544,
+      "grad_norm": 3.312098979949951,
       "learning_rate": 0.0001931591088051279,
+      "loss": 5.357623100280762,
+      "mean_token_accuracy": 0.8219081908464432,
       "num_tokens": 97909.0,
       "step": 14
     },
     {
+      "entropy": 0.6056851297616959,
       "epoch": 0.379746835443038,
+      "grad_norm": 4.054323196411133,
       "learning_rate": 0.00019157733266550575,
+      "loss": 5.329057216644287,
+      "mean_token_accuracy": 0.8183465003967285,
       "num_tokens": 104187.0,
       "step": 15
     },
     {
+      "entropy": 0.46965425461530685,
       "epoch": 0.4050632911392405,
+      "grad_norm": 3.1617722511291504,
       "learning_rate": 0.0001898390981891979,
+      "loss": 5.159331321716309,
+      "mean_token_accuracy": 0.8249307423830032,
       "num_tokens": 110300.0,
       "step": 16
     },
     {
+      "entropy": 0.4227886497974396,
       "epoch": 0.43037974683544306,
+      "grad_norm": 6.028562545776367,
       "learning_rate": 0.0001879473751206489,
+      "loss": 5.351903915405273,
+      "mean_token_accuracy": 0.8241183459758759,
       "num_tokens": 117157.0,
       "step": 17
     },
     {
+      "entropy": 0.3527509272098541,
       "epoch": 0.45569620253164556,
+      "grad_norm": 3.293024778366089,
       "learning_rate": 0.00018590539543698854,
+      "loss": 4.932947158813477,
+      "mean_token_accuracy": 0.8373657613992691,
       "num_tokens": 123378.0,
       "step": 18
     },
     {
+      "entropy": 0.33450157940387726,
       "epoch": 0.4810126582278481,
+      "grad_norm": 1.6703517436981201,
       "learning_rate": 0.00018371664782625287,
+      "loss": 5.134032249450684,
+      "mean_token_accuracy": 0.8269297778606415,
       "num_tokens": 129042.0,
       "step": 19
     },
     {
+      "entropy": 0.3399146646261215,
       "epoch": 0.5063291139240507,
+      "grad_norm": 2.0219178199768066,
       "learning_rate": 0.0001813848717270195,
+      "loss": 5.027883052825928,
+      "mean_token_accuracy": 0.8322668075561523,
       "num_tokens": 135747.0,
       "step": 20
     },
     {
+      "entropy": 0.29102808609604836,
       "epoch": 0.5316455696202531,
+      "grad_norm": 1.350559949874878,
       "learning_rate": 0.00017891405093963938,
+      "loss": 4.954178810119629,
+      "mean_token_accuracy": 0.8439806550741196,
       "num_tokens": 141848.0,
       "step": 21
     },
     {
+      "entropy": 0.3136157840490341,
       "epoch": 0.5569620253164557,
+      "grad_norm": 1.3057941198349,
       "learning_rate": 0.00017630840681998066,
+      "loss": 5.179681301116943,
+      "mean_token_accuracy": 0.8438494354486465,
       "num_tokens": 147566.0,
       "step": 22
     },
     {
+      "entropy": 0.2950733080506325,
       "epoch": 0.5822784810126582,
+      "grad_norm": 1.1281788349151611,
       "learning_rate": 0.00017357239106731317,
+      "loss": 5.171978950500488,
+      "mean_token_accuracy": 0.8462631553411484,
       "num_tokens": 154620.0,
       "step": 23
     },
     {
+      "entropy": 0.27676206082105637,
       "epoch": 0.6075949367088608,
+      "grad_norm": 1.311663269996643,
       "learning_rate": 0.00017071067811865476,
+      "loss": 5.077099800109863,
+      "mean_token_accuracy": 0.8611445128917694,
       "num_tokens": 161701.0,
       "step": 24
     },
     {
+      "entropy": 0.30840296298265457,
       "epoch": 0.6329113924050633,
+      "grad_norm": 2.281773090362549,
       "learning_rate": 0.00016772815716257412,
+      "loss": 5.118645668029785,
+      "mean_token_accuracy": 0.8323634117841721,
       "num_tokens": 168435.0,
       "step": 25
     },
     {
+      "entropy": 0.29603931307792664,
       "epoch": 0.6582278481012658,
+      "grad_norm": 1.230368971824646,
       "learning_rate": 0.00016462992378609407,
+      "loss": 5.176095962524414,
+      "mean_token_accuracy": 0.839859664440155,
       "num_tokens": 175650.0,
       "step": 26
     },
     {
+      "entropy": 0.24447191506624222,
       "epoch": 0.6835443037974683,
+      "grad_norm": 1.7844265699386597,
       "learning_rate": 0.0001614212712689668,
+      "loss": 5.146295547485352,
+      "mean_token_accuracy": 0.8507861793041229,
       "num_tokens": 181490.0,
       "step": 27
     },
     {
+      "entropy": 0.23978285491466522,
       "epoch": 0.7088607594936709,
+      "grad_norm": 1.4895042181015015,
       "learning_rate": 0.00015810768154019385,
+      "loss": 5.0867919921875,
+      "mean_token_accuracy": 0.8498348891735077,
       "num_tokens": 188582.0,
       "step": 28
     },
     {
+      "entropy": 0.1993725746870041,
       "epoch": 0.7341772151898734,
+      "grad_norm": 1.2189953327178955,
       "learning_rate": 0.00015469481581224272,
+      "loss": 5.131974697113037,
+      "mean_token_accuracy": 0.8698419630527496,
       "num_tokens": 193783.0,
       "step": 29
     },
     {
+      "entropy": 0.24254225194454193,
       "epoch": 0.759493670886076,
+      "grad_norm": 3.0398507118225098,
       "learning_rate": 0.00015118850490896012,
+      "loss": 5.307463645935059,
+      "mean_token_accuracy": 0.8126509040594101,
       "num_tokens": 200686.0,
       "step": 30
     },
     {
+      "entropy": 0.18977511674165726,
       "epoch": 0.7848101265822784,
+      "grad_norm": 1.7506808042526245,
       "learning_rate": 0.00014759473930370736,
+      "loss": 5.047242164611816,
+      "mean_token_accuracy": 0.8546684086322784,
       "num_tokens": 208440.0,
       "step": 31
     },
     {
+      "entropy": 0.2233816385269165,
       "epoch": 0.810126582278481,
+      "grad_norm": 1.0086487531661987,
       "learning_rate": 0.00014391965888473703,
+      "loss": 5.027800559997559,
+      "mean_token_accuracy": 0.8576832860708237,
       "num_tokens": 215131.0,
       "step": 32
     },
     {
+      "entropy": 0.19872410595417023,
       "epoch": 0.8354430379746836,
+      "grad_norm": 1.6391977071762085,
       "learning_rate": 0.00014016954246529696,
+      "loss": 5.0654706954956055,
+      "mean_token_accuracy": 0.8516139537096024,
       "num_tokens": 222195.0,
       "step": 33
     },
     {
+      "entropy": 0.20351751521229744,
       "epoch": 0.8607594936708861,
+      "grad_norm": 1.5714291334152222,
       "learning_rate": 0.00013635079705638298,
+      "loss": 5.170158386230469,
+      "mean_token_accuracy": 0.8566406965255737,
       "num_tokens": 229926.0,
       "step": 34
     },
     {
+      "entropy": 0.21749146282672882,
       "epoch": 0.8860759493670886,
+      "grad_norm": 1.6517112255096436,
       "learning_rate": 0.00013246994692046836,
+      "loss": 5.215228080749512,
+      "mean_token_accuracy": 0.8288360238075256,
       "num_tokens": 235766.0,
       "step": 35
     },
     {
+      "entropy": 0.2011272795498371,
       "epoch": 0.9113924050632911,
+      "grad_norm": 1.4017918109893799,
       "learning_rate": 0.00012853362242491053,
+      "loss": 5.1320648193359375,
+      "mean_token_accuracy": 0.8528883457183838,
       "num_tokens": 242731.0,
       "step": 36
     },
     {
+      "entropy": 0.21461713686585426,
       "epoch": 0.9367088607594937,
+      "grad_norm": 1.77144193649292,
       "learning_rate": 0.00012454854871407994,
+      "loss": 5.11080265045166,
+      "mean_token_accuracy": 0.8521855026483536,
       "num_tokens": 249818.0,
       "step": 37
     },
     {
+      "entropy": 0.19789887592196465,
       "epoch": 0.9620253164556962,
+      "grad_norm": 3.2191762924194336,
       "learning_rate": 0.00012052153421956342,
+      "loss": 5.070086479187012,
+      "mean_token_accuracy": 0.848893016576767,
       "num_tokens": 256043.0,
       "step": 38
     },
     {
+      "entropy": 0.1783522628247738,
       "epoch": 0.9873417721518988,
+      "grad_norm": 1.3007391691207886,
       "learning_rate": 0.00011645945902807341,
+      "loss": 5.1378865242004395,
+      "mean_token_accuracy": 0.8583788871765137,
       "num_tokens": 263628.0,
       "step": 39
     },
     {
+      "entropy": 0.21517179906368256,
       "epoch": 1.0,
+      "grad_norm": 2.357231378555298,
       "learning_rate": 0.00011236926312693479,
+      "loss": 5.041143894195557,
+      "mean_token_accuracy": 0.8405462205410004,
       "num_tokens": 266935.0,
       "step": 40
     },
       "eval_loss": NaN,
       "eval_mean_token_accuracy": 0.0,
       "eval_num_tokens": 266935.0,
+      "eval_runtime": 5.9781,
+      "eval_samples_per_second": 37.136,
+      "eval_steps_per_second": 9.368,
       "step": 40
     },
     {
+      "entropy": 0.1771511249244213,
       "epoch": 1.0253164556962024,
+      "grad_norm": 1.8927557468414307,
       "learning_rate": 0.00010825793454723325,
+      "loss": 5.032665252685547,
+      "mean_token_accuracy": 0.8413392454385757,
       "num_tokens": 272689.0,
       "step": 41
     },
     {
+      "entropy": 0.19083253294229507,
       "epoch": 1.0506329113924051,
+      "grad_norm": 1.5800896883010864,
       "learning_rate": 0.00010413249742488131,
+      "loss": 5.118054389953613,
+      "mean_token_accuracy": 0.8499636799097061,
       "num_tokens": 279320.0,
       "step": 42
     },
     {
+      "entropy": 0.1893175132572651,
       "epoch": 1.0759493670886076,
+      "grad_norm": 1.6944535970687866,
       "learning_rate": 0.0001,
+      "loss": 4.96186637878418,
+      "mean_token_accuracy": 0.846690446138382,
       "num_tokens": 285858.0,
       "step": 43
     },
     {
+      "entropy": 0.19514397159218788,
       "epoch": 1.1012658227848102,
+      "grad_norm": 1.0285674333572388,
       "learning_rate": 9.586750257511867e-05,
+      "loss": 5.133388519287109,
+      "mean_token_accuracy": 0.8598480075597763,
       "num_tokens": 293542.0,
       "step": 44
     },
     {
+      "entropy": 0.1813015267252922,
       "epoch": 1.1265822784810127,
+      "grad_norm": 0.7471196055412292,
       "learning_rate": 9.174206545276677e-05,
+      "loss": 5.058955192565918,
+      "mean_token_accuracy": 0.8599645048379898,
       "num_tokens": 300567.0,
       "step": 45
     },
     {
+      "entropy": 0.19578944519162178,
       "epoch": 1.1518987341772151,
+      "grad_norm": 1.092103123664856,
       "learning_rate": 8.763073687306524e-05,
+      "loss": 5.134239196777344,
+      "mean_token_accuracy": 0.8482615053653717,
       "num_tokens": 308202.0,
       "step": 46
     },
     {
+      "entropy": 0.20517433807253838,
       "epoch": 1.1772151898734178,
+      "grad_norm": 1.2428195476531982,
       "learning_rate": 8.35405409719266e-05,
+      "loss": 5.1778974533081055,
+      "mean_token_accuracy": 0.8619973361492157,
       "num_tokens": 314689.0,
       "step": 47
     },
     {
+      "entropy": 0.19207627698779106,
       "epoch": 1.2025316455696202,
+      "grad_norm": 1.1919926404953003,
       "learning_rate": 7.947846578043659e-05,
+      "loss": 5.037685394287109,
+      "mean_token_accuracy": 0.8561878055334091,
       "num_tokens": 320323.0,
       "step": 48
     },
     {
+      "entropy": 0.1860572211444378,
       "epoch": 1.2278481012658227,
+      "grad_norm": 2.1096537113189697,
       "learning_rate": 7.54514512859201e-05,
+      "loss": 5.031648635864258,
+      "mean_token_accuracy": 0.8480775505304337,
       "num_tokens": 327016.0,
       "step": 49
     },
     {
+      "entropy": 0.1999019868671894,
       "epoch": 1.2531645569620253,
+      "grad_norm": 1.2548681497573853,
       "learning_rate": 7.146637757508949e-05,
+      "loss": 5.141640663146973,
+      "mean_token_accuracy": 0.865373820066452,
       "num_tokens": 333272.0,
       "step": 50
     },
     {
+      "entropy": 0.20417018979787827,
       "epoch": 1.2784810126582278,
+      "grad_norm": 0.847514808177948,
       "learning_rate": 6.753005307953167e-05,
+      "loss": 5.055359840393066,
+      "mean_token_accuracy": 0.8717339187860489,
       "num_tokens": 340919.0,
       "step": 51
     },
     {
+      "entropy": 0.20846642926335335,
       "epoch": 1.3037974683544304,
+      "grad_norm": 1.305163025856018,
       "learning_rate": 6.3649202943617e-05,
+      "loss": 5.115406513214111,
+      "mean_token_accuracy": 0.8486459404230118,
       "num_tokens": 347473.0,
       "step": 52
     },
     {
+      "entropy": 0.2167152315378189,
       "epoch": 1.3291139240506329,
+      "grad_norm": 1.275968074798584,
       "learning_rate": 5.983045753470308e-05,
+      "loss": 5.049930095672607,
+      "mean_token_accuracy": 0.8500020503997803,
       "num_tokens": 353979.0,
       "step": 53
     },
     {
+      "entropy": 0.1877211444079876,
       "epoch": 1.3544303797468356,
+      "grad_norm": 1.289473533630371,
       "learning_rate": 5.608034111526298e-05,
+      "loss": 5.212080001831055,
+      "mean_token_accuracy": 0.8610779196023941,
       "num_tokens": 360939.0,
       "step": 54
     },
     {
+      "entropy": 0.21858176216483116,
       "epoch": 1.379746835443038,
+      "grad_norm": 1.2420766353607178,
       "learning_rate": 5.240526069629265e-05,
+      "loss": 5.145570278167725,
+      "mean_token_accuracy": 0.8502416759729385,
       "num_tokens": 367684.0,
       "step": 55
     },
     {
+      "entropy": 0.16584616154432297,
       "epoch": 1.4050632911392404,
+      "grad_norm": 0.8850894570350647,
       "learning_rate": 4.8811495091039926e-05,
+      "loss": 4.982363224029541,
       "mean_token_accuracy": 0.8663530200719833,
       "num_tokens": 374983.0,
       "step": 56
     },
     {
+      "entropy": 0.20285014435648918,
       "epoch": 1.4303797468354431,
+      "grad_norm": 0.9019045233726501,
       "learning_rate": 4.530518418775733e-05,
+      "loss": 5.186882972717285,
+      "mean_token_accuracy": 0.8562371581792831,
       "num_tokens": 380947.0,
       "step": 57
     },
     {
+      "entropy": 0.19403913989663124,
       "epoch": 1.4556962025316456,
+      "grad_norm": 1.3603911399841309,
       "learning_rate": 4.189231845980618e-05,
+      "loss": 5.046579837799072,
+      "mean_token_accuracy": 0.856428250670433,
       "num_tokens": 388400.0,
       "step": 58
     },
     {
+      "entropy": 0.17723571881651878,
       "epoch": 1.481012658227848,
+      "grad_norm": 0.8954723477363586,
       "learning_rate": 3.857872873103322e-05,
+      "loss": 4.9259443283081055,
       "mean_token_accuracy": 0.8559762090444565,
       "num_tokens": 394297.0,
       "step": 59
     },
     {
+      "entropy": 0.1926303431391716,
       "epoch": 1.5063291139240507,
+      "grad_norm": 1.165939211845398,
       "learning_rate": 3.53700762139059e-05,
+      "loss": 5.209965705871582,
+      "mean_token_accuracy": 0.8545413017272949,
       "num_tokens": 401531.0,
       "step": 60
     },
     {
+      "entropy": 0.17567158862948418,
       "epoch": 1.5316455696202531,
+      "grad_norm": 1.256475806236267,
       "learning_rate": 3.227184283742591e-05,
+      "loss": 5.071780681610107,
+      "mean_token_accuracy": 0.8629807680845261,
       "num_tokens": 408001.0,
       "step": 61
     },
     {
+      "entropy": 0.15778229013085365,
       "epoch": 1.5569620253164556,
+      "grad_norm": 0.9394585490226746,
       "learning_rate": 2.9289321881345254e-05,
+      "loss": 5.174696445465088,
       "mean_token_accuracy": 0.8652869164943695,
       "num_tokens": 414848.0,
       "step": 62
     },
     {
+      "entropy": 0.2071729153394699,
       "epoch": 1.5822784810126582,
+      "grad_norm": 0.9868037700653076,
       "learning_rate": 2.6427608932686843e-05,
+      "loss": 5.067627906799316,
+      "mean_token_accuracy": 0.8562931418418884,
       "num_tokens": 422561.0,
       "step": 63
     },
     {
+      "entropy": 0.1966983526945114,
       "epoch": 1.6075949367088609,
+      "grad_norm": 0.7985685467720032,
       "learning_rate": 2.3691593180019366e-05,
+      "loss": 5.114544868469238,
+      "mean_token_accuracy": 0.8560546040534973,
       "num_tokens": 428474.0,
       "step": 64
     },
     {
+      "entropy": 0.1941680982708931,
       "epoch": 1.6329113924050633,
+      "grad_norm": 1.1260875463485718,
       "learning_rate": 2.1085949060360654e-05,
+      "loss": 5.048023223876953,
+      "mean_token_accuracy": 0.8535272777080536,
       "num_tokens": 433867.0,
       "step": 65
     },
     {
+      "entropy": 0.18942571431398392,
       "epoch": 1.6582278481012658,
+      "grad_norm": 0.8621047139167786,
       "learning_rate": 1.861512827298051e-05,
+      "loss": 4.9987688064575195,
+      "mean_token_accuracy": 0.8621815890073776,
       "num_tokens": 440685.0,
       "step": 66
     },
     {
+      "entropy": 0.18620551750063896,
       "epoch": 1.6835443037974684,
+      "grad_norm": 0.8113185167312622,
       "learning_rate": 1.6283352173747145e-05,
+      "loss": 5.130119323730469,
+      "mean_token_accuracy": 0.8656566739082336,
       "num_tokens": 447730.0,
       "step": 67
     },
     {
+      "entropy": 0.1812417395412922,
       "epoch": 1.7088607594936709,
+      "grad_norm": 0.657331645488739,
       "learning_rate": 1.4094604563011472e-05,
+      "loss": 5.11595344543457,
+      "mean_token_accuracy": 0.8689732104539871,
       "num_tokens": 454057.0,
       "step": 68
     },
     {
+      "entropy": 0.17315322160720825,
       "epoch": 1.7341772151898733,
+      "grad_norm": 1.2474181652069092,
       "learning_rate": 1.2052624879351104e-05,
+      "loss": 5.142193794250488,
+      "mean_token_accuracy": 0.8530041426420212,
       "num_tokens": 460989.0,
       "step": 69
     },
     {
+      "entropy": 0.20847681164741516,
       "epoch": 1.759493670886076,
+      "grad_norm": 0.9265520572662354,
       "learning_rate": 1.0160901810802115e-05,
+      "loss": 4.949027061462402,
+      "mean_token_accuracy": 0.8633775115013123,
       "num_tokens": 468216.0,
       "step": 70
     },
     {
+      "entropy": 0.1857427917420864,
       "epoch": 1.7848101265822784,
+      "grad_norm": 0.8429440259933472,
       "learning_rate": 8.422667334494249e-06,
+      "loss": 5.067360877990723,
+      "mean_token_accuracy": 0.8449466675519943,
       "num_tokens": 474276.0,
       "step": 71
     },
     {
+      "entropy": 0.17809578776359558,
       "epoch": 1.810126582278481,
+      "grad_norm": 1.0042821168899536,
       "learning_rate": 6.840891194872112e-06,
+      "loss": 5.115839958190918,
+      "mean_token_accuracy": 0.8699368089437485,
       "num_tokens": 481319.0,
       "step": 72
     },
     {
+      "entropy": 0.18285344913601875,
       "epoch": 1.8354430379746836,
+      "grad_norm": 0.9699199199676514,
       "learning_rate": 5.418275829936537e-06,
+      "loss": 5.113241672515869,
+      "mean_token_accuracy": 0.8597128540277481,
       "num_tokens": 488560.0,
       "step": 73
     },
     {
+      "entropy": 0.19144808873534203,
       "epoch": 1.8607594936708862,
+      "grad_norm": 0.7952200174331665,
       "learning_rate": 4.1572517541747294e-06,
+      "loss": 5.170323371887207,
+      "mean_token_accuracy": 0.86890609562397,
       "num_tokens": 494057.0,
       "step": 74
     },
     {
+      "entropy": 0.18293721601366997,
       "epoch": 1.8860759493670884,
+      "grad_norm": 0.8813140392303467,
       "learning_rate": 3.059973406066963e-06,
+      "loss": 4.910680770874023,
+      "mean_token_accuracy": 0.867401659488678,
       "num_tokens": 501562.0,
       "step": 75
     },
     {
+      "entropy": 0.1984117217361927,
       "epoch": 1.9113924050632911,
+      "grad_norm": 1.1274702548980713,
       "learning_rate": 2.128315467264552e-06,
+      "loss": 5.088738441467285,
+      "mean_token_accuracy": 0.8530780524015427,
       "num_tokens": 509439.0,
       "step": 76
     },
     {
+      "entropy": 0.18629329279065132,
       "epoch": 1.9367088607594938,
+      "grad_norm": 0.9891031980514526,
       "learning_rate": 1.3638696597277679e-06,
+      "loss": 5.085946083068848,
+      "mean_token_accuracy": 0.8562160283327103,
       "num_tokens": 516267.0,
       "step": 77
     },
     {
+      "entropy": 0.16653455793857574,
       "epoch": 1.9620253164556962,
+      "grad_norm": 0.746681809425354,
       "learning_rate": 7.679420262954984e-07,
+      "loss": 5.034550189971924,
+      "mean_token_accuracy": 0.8639013916254044,
       "num_tokens": 523626.0,
       "step": 78
     },
     {
+      "entropy": 0.1866554580628872,
       "epoch": 1.9873417721518987,
+      "grad_norm": 0.9339231848716736,
       "learning_rate": 3.415506993330153e-07,
+      "loss": 5.026074409484863,
+      "mean_token_accuracy": 0.8578499257564545,
       "num_tokens": 530022.0,
       "step": 79
     },
     {
+      "entropy": 0.19535863399505615,
       "epoch": 2.0,
+      "grad_norm": 1.5845301151275635,
       "learning_rate": 8.542416126989805e-08,
+      "loss": 5.070830821990967,
+      "mean_token_accuracy": 0.8686061203479767,
       "num_tokens": 533275.0,
       "step": 80
     },
       "eval_loss": NaN,
       "eval_mean_token_accuracy": 0.0,
       "eval_num_tokens": 533275.0,
+      "eval_runtime": 5.983,
+      "eval_samples_per_second": 37.105,
+      "eval_steps_per_second": 9.36,
       "step": 80
     }
   ],
       "attributes": {}
     }
   },
+  "total_flos": 1119703283007488.0,
   "train_batch_size": 2,
   "trial_name": null,
   "trial_params": null

checkpoint-80/training_args.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:511c0cc2bbf14fcbcb4a1223f6f454f90d8d9544a8bb1a60eb8f5e59686fc6ff
 size 5496

 version https://git-lfs.github.com/spec/v1
+oid sha256:da3e7916a9c3a9bdc1d65a3a86e68c6c06ab04fd280e02675e7357542392c8fa
 size 5496

predictions.jsonl CHANGED Viewed

The diff for this file is too large to render. See raw diff

training_args.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:511c0cc2bbf14fcbcb4a1223f6f454f90d8d9544a8bb1a60eb8f5e59686fc6ff
 size 5496

 version https://git-lfs.github.com/spec/v1
+oid sha256:da3e7916a9c3a9bdc1d65a3a86e68c6c06ab04fd280e02675e7357542392c8fa
 size 5496