Instructions to use mirsazzathossain/Llama-3.1-Tulu-3-8B-SFT-LoRA-capybara with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mirsazzathossain/Llama-3.1-Tulu-3-8B-SFT-LoRA-capybara with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("allenai/Llama-3.1-Tulu-3-8B")
model = PeftModel.from_pretrained(base_model, "mirsazzathossain/Llama-3.1-Tulu-3-8B-SFT-LoRA-capybara")

Transformers

How to use mirsazzathossain/Llama-3.1-Tulu-3-8B-SFT-LoRA-capybara with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="mirsazzathossain/Llama-3.1-Tulu-3-8B-SFT-LoRA-capybara")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("mirsazzathossain/Llama-3.1-Tulu-3-8B-SFT-LoRA-capybara", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use mirsazzathossain/Llama-3.1-Tulu-3-8B-SFT-LoRA-capybara with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mirsazzathossain/Llama-3.1-Tulu-3-8B-SFT-LoRA-capybara"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mirsazzathossain/Llama-3.1-Tulu-3-8B-SFT-LoRA-capybara",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/mirsazzathossain/Llama-3.1-Tulu-3-8B-SFT-LoRA-capybara

SGLang

How to use mirsazzathossain/Llama-3.1-Tulu-3-8B-SFT-LoRA-capybara with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "mirsazzathossain/Llama-3.1-Tulu-3-8B-SFT-LoRA-capybara" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mirsazzathossain/Llama-3.1-Tulu-3-8B-SFT-LoRA-capybara",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "mirsazzathossain/Llama-3.1-Tulu-3-8B-SFT-LoRA-capybara" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mirsazzathossain/Llama-3.1-Tulu-3-8B-SFT-LoRA-capybara",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use mirsazzathossain/Llama-3.1-Tulu-3-8B-SFT-LoRA-capybara with Docker Model Runner:
```
docker model run hf.co/mirsazzathossain/Llama-3.1-Tulu-3-8B-SFT-LoRA-capybara
```

mirsazzathossain commited on May 16

Commit

64a3efb

verified ·

1 Parent(s): 4ed6175

End of training

Browse files

Files changed (7) hide show

README.md +2 -2
all_results.json +10 -10
eval_results.json +5 -5
train_results.json +6 -6
trainer_state.json +546 -47
training_eval_loss.png +0 -0
training_loss.png +0 -0

README.md CHANGED Viewed

@@ -18,9 +18,9 @@ should probably proofread and complete it, then remove this comment. -->
 # Llama-3.1-Tulu-3-8B-SFT-LoRA-capybara
-This model is a fine-tuned version of [allenai/Llama-3.1-Tulu-3-8B](https://huggingface.co/allenai/Llama-3.1-Tulu-3-8B) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.9701
 ## Model description

 # Llama-3.1-Tulu-3-8B-SFT-LoRA-capybara
+This model is a fine-tuned version of [allenai/Llama-3.1-Tulu-3-8B](https://huggingface.co/allenai/Llama-3.1-Tulu-3-8B) on the capybara_sft_train dataset.
 It achieves the following results on the evaluation set:
+- Loss: 0.9697
 ## Model description

all_results.json CHANGED Viewed

@@ -1,12 +1,12 @@
 {
-    "epoch": 0.3333333333333333,
-    "eval_loss": 1.5940594673156738,
-    "eval_runtime": 1.5603,
-    "eval_samples_per_second": 3.845,
-    "eval_steps_per_second": 3.845,
-    "total_flos": 463681565491200.0,
-    "train_loss": 1.1900920867919922,
-    "train_runtime": 20.5212,
-    "train_samples_per_second": 0.975,
-    "train_steps_per_second": 0.244
 }

 {
+    "epoch": 2.0,
+    "eval_loss": 0.9696994423866272,
+    "eval_runtime": 362.185,
+    "eval_samples_per_second": 1.82,
+    "eval_steps_per_second": 1.82,
+    "total_flos": 6.06897586520064e+17,
+    "train_loss": 1.01524107314827,
+    "train_runtime": 23674.123,
+    "train_samples_per_second": 0.501,
+    "train_steps_per_second": 0.031
 }

eval_results.json CHANGED Viewed

@@ -1,7 +1,7 @@
 {
-    "epoch": 0.3333333333333333,
-    "eval_loss": 1.5940594673156738,
-    "eval_runtime": 1.5603,
-    "eval_samples_per_second": 3.845,
-    "eval_steps_per_second": 3.845
 }

 {
+    "epoch": 2.0,
+    "eval_loss": 0.9696994423866272,
+    "eval_runtime": 362.185,
+    "eval_samples_per_second": 1.82,
+    "eval_steps_per_second": 1.82
 }

train_results.json CHANGED Viewed

@@ -1,8 +1,8 @@
 {
-    "epoch": 0.3333333333333333,
-    "total_flos": 463681565491200.0,
-    "train_loss": 1.1900920867919922,
-    "train_runtime": 20.5212,
-    "train_samples_per_second": 0.975,
-    "train_steps_per_second": 0.244
 }

 {
+    "epoch": 2.0,
+    "total_flos": 6.06897586520064e+17,
+    "train_loss": 1.01524107314827,
+    "train_runtime": 23674.123,
+    "train_samples_per_second": 0.501,
+    "train_steps_per_second": 0.031
 }

trainer_state.json CHANGED Viewed

@@ -2,71 +2,570 @@
   "best_global_step": null,
   "best_metric": null,
   "best_model_checkpoint": null,
-  "epoch": 0.3333333333333333,
-  "eval_steps": 3,
-  "global_step": 5,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
   "log_history": [
     {
-      "epoch": 0.06666666666666667,
-      "grad_norm": 1.1794017553329468,
-      "learning_rate": 1e-05,
-      "loss": 1.0468,
-      "step": 1
     },
     {
-      "epoch": 0.13333333333333333,
-      "grad_norm": 1.246726155281067,
-      "learning_rate": 9.045084971874738e-06,
-      "loss": 1.2545,
-      "step": 2
     },
     {
-      "epoch": 0.2,
-      "grad_norm": 1.552769660949707,
-      "learning_rate": 6.545084971874738e-06,
-      "loss": 1.131,
-      "step": 3
     },
     {
-      "epoch": 0.2,
-      "eval_loss": 1.5975946187973022,
-      "eval_runtime": 1.56,
-      "eval_samples_per_second": 3.846,
-      "eval_steps_per_second": 3.846,
-      "step": 3
     },
     {
-      "epoch": 0.26666666666666666,
-      "grad_norm": 1.32817542552948,
-      "learning_rate": 3.4549150281252635e-06,
-      "loss": 1.2633,
-      "step": 4
     },
     {
-      "epoch": 0.3333333333333333,
-      "grad_norm": 1.3675721883773804,
-      "learning_rate": 9.549150281252633e-07,
-      "loss": 1.2548,
-      "step": 5
     },
     {
-      "epoch": 0.3333333333333333,
-      "step": 5,
-      "total_flos": 463681565491200.0,
-      "train_loss": 1.1900920867919922,
-      "train_runtime": 20.5212,
-      "train_samples_per_second": 0.975,
-      "train_steps_per_second": 0.244
     }
   ],
-  "logging_steps": 1,
-  "max_steps": 5,
   "num_input_tokens_seen": 0,
-  "num_train_epochs": 1,
-  "save_steps": 5,
   "stateful_callbacks": {
     "TrainerControl": {
       "args": {
@@ -79,8 +578,8 @@
       "attributes": {}
     }
   },
-  "total_flos": 463681565491200.0,
-  "train_batch_size": 4,
   "trial_name": null,
   "trial_params": null
 }

   "best_global_step": null,
   "best_metric": null,
   "best_model_checkpoint": null,
+  "epoch": 2.0,
+  "eval_steps": 200,
+  "global_step": 742,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
   "log_history": [
     {
+      "epoch": 0.02695871946082561,
+      "grad_norm": 0.8008378744125366,
+      "learning_rate": 1.2000000000000002e-06,
+      "loss": 1.2496,
+      "step": 10
     },
     {
+      "epoch": 0.05391743892165122,
+      "grad_norm": 0.7582265138626099,
+      "learning_rate": 2.5333333333333338e-06,
+      "loss": 1.2396,
+      "step": 20
     },
     {
+      "epoch": 0.08087615838247683,
+      "grad_norm": 0.7974638938903809,
+      "learning_rate": 3.866666666666667e-06,
+      "loss": 1.2612,
+      "step": 30
     },
     {
+      "epoch": 0.10783487784330244,
+      "grad_norm": 0.7764624953269958,
+      "learning_rate": 5.2e-06,
+      "loss": 1.2333,
+      "step": 40
     },
     {
+      "epoch": 0.13479359730412804,
+      "grad_norm": 0.7129917740821838,
+      "learning_rate": 6.533333333333334e-06,
+      "loss": 1.1534,
+      "step": 50
     },
     {
+      "epoch": 0.16175231676495366,
+      "grad_norm": 0.6209232211112976,
+      "learning_rate": 7.866666666666667e-06,
+      "loss": 1.0875,
+      "step": 60
     },
     {
+      "epoch": 0.18871103622577928,
+      "grad_norm": 0.44803526997566223,
+      "learning_rate": 9.200000000000002e-06,
+      "loss": 1.1057,
+      "step": 70
+    },
+    {
+      "epoch": 0.21566975568660487,
+      "grad_norm": 0.36510151624679565,
+      "learning_rate": 9.999112649450154e-06,
+      "loss": 1.1344,
+      "step": 80
+    },
+    {
+      "epoch": 0.2426284751474305,
+      "grad_norm": 0.2959080636501312,
+      "learning_rate": 9.989133572488716e-06,
+      "loss": 1.0469,
+      "step": 90
+    },
+    {
+      "epoch": 0.2695871946082561,
+      "grad_norm": 0.24933554232120514,
+      "learning_rate": 9.968088438109002e-06,
+      "loss": 1.1375,
+      "step": 100
+    },
+    {
+      "epoch": 0.2965459140690817,
+      "grad_norm": 0.2685732841491699,
+      "learning_rate": 9.93602392509041e-06,
+      "loss": 1.004,
+      "step": 110
+    },
+    {
+      "epoch": 0.3235046335299073,
+      "grad_norm": 0.22426341474056244,
+      "learning_rate": 9.893011153545679e-06,
+      "loss": 1.0332,
+      "step": 120
+    },
+    {
+      "epoch": 0.35046335299073295,
+      "grad_norm": 0.21612893044948578,
+      "learning_rate": 9.839145527174216e-06,
+      "loss": 1.044,
+      "step": 130
+    },
+    {
+      "epoch": 0.37742207245155857,
+      "grad_norm": 0.24937374889850616,
+      "learning_rate": 9.774546521653633e-06,
+      "loss": 1.0035,
+      "step": 140
+    },
+    {
+      "epoch": 0.4043807919123842,
+      "grad_norm": 0.23830914497375488,
+      "learning_rate": 9.699357419638904e-06,
+      "loss": 1.0611,
+      "step": 150
+    },
+    {
+      "epoch": 0.43133951137320975,
+      "grad_norm": 0.240906223654747,
+      "learning_rate": 9.613744992956844e-06,
+      "loss": 0.9973,
+      "step": 160
+    },
+    {
+      "epoch": 0.45829823083403537,
+      "grad_norm": 0.24612027406692505,
+      "learning_rate": 9.517899132700889e-06,
+      "loss": 1.0654,
+      "step": 170
+    },
+    {
+      "epoch": 0.485256950294861,
+      "grad_norm": 0.25542137026786804,
+      "learning_rate": 9.412032428046594e-06,
+      "loss": 1.0056,
+      "step": 180
+    },
+    {
+      "epoch": 0.5122156697556866,
+      "grad_norm": 0.2691582441329956,
+      "learning_rate": 9.296379694722051e-06,
+      "loss": 0.9783,
+      "step": 190
+    },
+    {
+      "epoch": 0.5391743892165122,
+      "grad_norm": 0.23645047843456268,
+      "learning_rate": 9.171197454179124e-06,
+      "loss": 0.9903,
+      "step": 200
+    },
+    {
+      "epoch": 0.5391743892165122,
+      "eval_loss": 0.9858898520469666,
+      "eval_runtime": 361.9638,
+      "eval_samples_per_second": 1.821,
+      "eval_steps_per_second": 1.821,
+      "step": 200
+    },
+    {
+      "epoch": 0.5661331086773378,
+      "grad_norm": 0.34743762016296387,
+      "learning_rate": 9.03676336462068e-06,
+      "loss": 0.9888,
+      "step": 210
+    },
+    {
+      "epoch": 0.5930918281381634,
+      "grad_norm": 0.22745391726493835,
+      "learning_rate": 8.893375605145837e-06,
+      "loss": 0.9646,
+      "step": 220
+    },
+    {
+      "epoch": 0.620050547598989,
+      "grad_norm": 0.2186407446861267,
+      "learning_rate": 8.74135221437921e-06,
+      "loss": 1.0027,
+      "step": 230
+    },
+    {
+      "epoch": 0.6470092670598147,
+      "grad_norm": 0.2490515559911728,
+      "learning_rate": 8.581030385051105e-06,
+      "loss": 0.9469,
+      "step": 240
+    },
+    {
+      "epoch": 0.6739679865206403,
+      "grad_norm": 0.24692057073116302,
+      "learning_rate": 8.412765716093273e-06,
+      "loss": 1.0131,
+      "step": 250
+    },
+    {
+      "epoch": 0.7009267059814659,
+      "grad_norm": 0.2852599322795868,
+      "learning_rate": 8.23693142390914e-06,
+      "loss": 0.9614,
+      "step": 260
+    },
+    {
+      "epoch": 0.7278854254422915,
+      "grad_norm": 0.3111582100391388,
+      "learning_rate": 8.053917514567927e-06,
+      "loss": 1.0357,
+      "step": 270
+    },
+    {
+      "epoch": 0.7548441449031171,
+      "grad_norm": 0.25308677554130554,
+      "learning_rate": 7.864129918758738e-06,
+      "loss": 1.0057,
+      "step": 280
+    },
+    {
+      "epoch": 0.7818028643639428,
+      "grad_norm": 0.2518717050552368,
+      "learning_rate": 7.667989591423349e-06,
+      "loss": 1.077,
+      "step": 290
+    },
+    {
+      "epoch": 0.8087615838247684,
+      "grad_norm": 0.2539553940296173,
+      "learning_rate": 7.465931578064703e-06,
+      "loss": 1.0043,
+      "step": 300
+    },
+    {
+      "epoch": 0.8357203032855939,
+      "grad_norm": 0.22149862349033356,
+      "learning_rate": 7.258404049802135e-06,
+      "loss": 0.9659,
+      "step": 310
+    },
+    {
+      "epoch": 0.8626790227464195,
+      "grad_norm": 0.31281328201293945,
+      "learning_rate": 7.045867309313499e-06,
+      "loss": 0.9659,
+      "step": 320
+    },
+    {
+      "epoch": 0.8896377422072451,
+      "grad_norm": 0.2311943918466568,
+      "learning_rate": 6.8287927698691745e-06,
+      "loss": 0.9671,
+      "step": 330
+    },
+    {
+      "epoch": 0.9165964616680707,
+      "grad_norm": 0.29115474224090576,
+      "learning_rate": 6.6076619097223735e-06,
+      "loss": 0.9943,
+      "step": 340
+    },
+    {
+      "epoch": 0.9435551811288964,
+      "grad_norm": 0.2641865015029907,
+      "learning_rate": 6.382965204175027e-06,
+      "loss": 0.968,
+      "step": 350
+    },
+    {
+      "epoch": 0.970513900589722,
+      "grad_norm": 0.2566356062889099,
+      "learning_rate": 6.155201037687917e-06,
+      "loss": 0.9882,
+      "step": 360
+    },
+    {
+      "epoch": 0.9974726200505476,
+      "grad_norm": 0.26403674483299255,
+      "learning_rate": 5.924874598448038e-06,
+      "loss": 0.9577,
+      "step": 370
+    },
+    {
+      "epoch": 1.024262847514743,
+      "grad_norm": 0.2462741881608963,
+      "learning_rate": 5.692496757845092e-06,
+      "loss": 0.9367,
+      "step": 380
+    },
+    {
+      "epoch": 1.0512215669755687,
+      "grad_norm": 0.2530335783958435,
+      "learning_rate": 5.45858293734244e-06,
+      "loss": 0.9893,
+      "step": 390
+    },
+    {
+      "epoch": 1.0781802864363943,
+      "grad_norm": 0.23828567564487457,
+      "learning_rate": 5.223651965255864e-06,
+      "loss": 0.9931,
+      "step": 400
+    },
+    {
+      "epoch": 1.0781802864363943,
+      "eval_loss": 0.974194347858429,
+      "eval_runtime": 361.5075,
+      "eval_samples_per_second": 1.823,
+      "eval_steps_per_second": 1.823,
+      "step": 400
+    },
+    {
+      "epoch": 1.10513900589722,
+      "grad_norm": 0.25133016705513,
+      "learning_rate": 4.988224925975799e-06,
+      "loss": 1.0022,
+      "step": 410
+    },
+    {
+      "epoch": 1.1320977253580455,
+      "grad_norm": 0.24585603177547455,
+      "learning_rate": 4.752824004185548e-06,
+      "loss": 0.9993,
+      "step": 420
+    },
+    {
+      "epoch": 1.1590564448188712,
+      "grad_norm": 0.27999183535575867,
+      "learning_rate": 4.5179713266389866e-06,
+      "loss": 0.9835,
+      "step": 430
+    },
+    {
+      "epoch": 1.1860151642796968,
+      "grad_norm": 0.3136424720287323,
+      "learning_rate": 4.284187804066764e-06,
+      "loss": 0.9242,
+      "step": 440
+    },
+    {
+      "epoch": 1.2129738837405224,
+      "grad_norm": 0.24964718520641327,
+      "learning_rate": 4.051991975779691e-06,
+      "loss": 0.9784,
+      "step": 450
+    },
+    {
+      "epoch": 1.239932603201348,
+      "grad_norm": 0.29519638419151306,
+      "learning_rate": 3.821898859532013e-06,
+      "loss": 1.0019,
+      "step": 460
+    },
+    {
+      "epoch": 1.2668913226621736,
+      "grad_norm": 0.28124356269836426,
+      "learning_rate": 3.5944188091955843e-06,
+      "loss": 1.0048,
+      "step": 470
+    },
+    {
+      "epoch": 1.2938500421229993,
+      "grad_norm": 0.26205387711524963,
+      "learning_rate": 3.3700563827787224e-06,
+      "loss": 0.9982,
+      "step": 480
+    },
+    {
+      "epoch": 1.3208087615838249,
+      "grad_norm": 0.4750404953956604,
+      "learning_rate": 3.149309223300428e-06,
+      "loss": 1.0254,
+      "step": 490
+    },
+    {
+      "epoch": 1.3477674810446505,
+      "grad_norm": 0.2478070706129074,
+      "learning_rate": 2.9326669550023124e-06,
+      "loss": 0.9551,
+      "step": 500
+    },
+    {
+      "epoch": 1.3747262005054761,
+      "grad_norm": 0.2824980616569519,
+      "learning_rate": 2.7206100973463958e-06,
+      "loss": 0.9784,
+      "step": 510
+    },
+    {
+      "epoch": 1.4016849199663017,
+      "grad_norm": 0.41172295808792114,
+      "learning_rate": 2.513608999207622e-06,
+      "loss": 1.0383,
+      "step": 520
+    },
+    {
+      "epoch": 1.4286436394271271,
+      "grad_norm": 0.25096771121025085,
+      "learning_rate": 2.3121227956250435e-06,
+      "loss": 0.9803,
+      "step": 530
+    },
+    {
+      "epoch": 1.4556023588879528,
+      "grad_norm": 0.268365740776062,
+      "learning_rate": 2.1165983894256647e-06,
+      "loss": 0.981,
+      "step": 540
+    },
+    {
+      "epoch": 1.4825610783487784,
+      "grad_norm": 0.2462984025478363,
+      "learning_rate": 1.9274694599797067e-06,
+      "loss": 0.9297,
+      "step": 550
+    },
+    {
+      "epoch": 1.509519797809604,
+      "grad_norm": 0.25125372409820557,
+      "learning_rate": 1.745155501285939e-06,
+      "loss": 1.0078,
+      "step": 560
+    },
+    {
+      "epoch": 1.5364785172704296,
+      "grad_norm": 0.3042011559009552,
+      "learning_rate": 1.5700608915205978e-06,
+      "loss": 1.0059,
+      "step": 570
+    },
+    {
+      "epoch": 1.5634372367312552,
+      "grad_norm": 0.25765061378479004,
+      "learning_rate": 1.4025739961137043e-06,
+      "loss": 1.0604,
+      "step": 580
+    },
+    {
+      "epoch": 1.5903959561920809,
+      "grad_norm": 0.28569814562797546,
+      "learning_rate": 1.2430663063421388e-06,
+      "loss": 0.9574,
+      "step": 590
+    },
+    {
+      "epoch": 1.6173546756529065,
+      "grad_norm": 0.27754876017570496,
+      "learning_rate": 1.091891615350147e-06,
+      "loss": 0.9973,
+      "step": 600
+    },
+    {
+      "epoch": 1.6173546756529065,
+      "eval_loss": 0.9700986742973328,
+      "eval_runtime": 361.0549,
+      "eval_samples_per_second": 1.825,
+      "eval_steps_per_second": 1.825,
+      "step": 600
+    },
+    {
+      "epoch": 1.644313395113732,
+      "grad_norm": 0.3200387954711914,
+      "learning_rate": 9.49385233424856e-07,
+      "loss": 0.9581,
+      "step": 610
+    },
+    {
+      "epoch": 1.6712721145745577,
+      "grad_norm": 0.3195631504058838,
+      "learning_rate": 8.158632442673603e-07,
+      "loss": 0.9446,
+      "step": 620
+    },
+    {
+      "epoch": 1.6982308340353833,
+      "grad_norm": 0.26112768054008484,
+      "learning_rate": 6.916218039089961e-07,
+      "loss": 0.9549,
+      "step": 630
+    },
+    {
+      "epoch": 1.725189553496209,
+      "grad_norm": 0.2657977044582367,
+      "learning_rate": 5.769364838278063e-07,
+      "loss": 0.995,
+      "step": 640
+    },
+    {
+      "epoch": 1.7521482729570346,
+      "grad_norm": 0.3035813868045807,
+      "learning_rate": 4.720616597222205e-07,
+      "loss": 0.9935,
+      "step": 650
+    },
+    {
+      "epoch": 1.7791069924178602,
+      "grad_norm": 0.29046186804771423,
+      "learning_rate": 3.7722994729763427e-07,
+      "loss": 1.0067,
+      "step": 660
+    },
+    {
+      "epoch": 1.8060657118786858,
+      "grad_norm": 0.27917370200157166,
+      "learning_rate": 2.9265168631736005e-07,
+      "loss": 1.0086,
+      "step": 670
+    },
+    {
+      "epoch": 1.8330244313395114,
+      "grad_norm": 0.26591578125953674,
+      "learning_rate": 2.1851447406231573e-07,
+      "loss": 0.9478,
+      "step": 680
+    },
+    {
+      "epoch": 1.8599831508003368,
+      "grad_norm": 0.30971062183380127,
+      "learning_rate": 1.5498274923427925e-07,
+      "loss": 0.9364,
+      "step": 690
+    },
+    {
+      "epoch": 1.8869418702611624,
+      "grad_norm": 0.24184127151966095,
+      "learning_rate": 1.0219742722559433e-07,
+      "loss": 1.0085,
+      "step": 700
+    },
+    {
+      "epoch": 1.913900589721988,
+      "grad_norm": 0.32377859950065613,
+      "learning_rate": 6.027558756434015e-08,
+      "loss": 1.0718,
+      "step": 710
+    },
+    {
+      "epoch": 1.9408593091828137,
+      "grad_norm": 0.24339060485363007,
+      "learning_rate": 2.9310214228202016e-08,
+      "loss": 0.9437,
+      "step": 720
+    },
+    {
+      "epoch": 1.9678180286436393,
+      "grad_norm": 0.2631000876426697,
+      "learning_rate": 9.369989403041347e-09,
+      "loss": 0.9958,
+      "step": 730
+    },
+    {
+      "epoch": 1.994776748104465,
+      "grad_norm": 0.24943451583385468,
+      "learning_rate": 4.991411436189308e-10,
+      "loss": 0.9993,
+      "step": 740
+    },
+    {
+      "epoch": 2.0,
+      "step": 742,
+      "total_flos": 6.06897586520064e+17,
+      "train_loss": 1.01524107314827,
+      "train_runtime": 23674.123,
+      "train_samples_per_second": 0.501,
+      "train_steps_per_second": 0.031
     }
   ],
+  "logging_steps": 10,
+  "max_steps": 742,
   "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 200,
   "stateful_callbacks": {
     "TrainerControl": {
       "args": {
       "attributes": {}
     }
   },
+  "total_flos": 6.06897586520064e+17,
+  "train_batch_size": 1,
   "trial_name": null,
   "trial_params": null
 }

training_eval_loss.png ADDED Viewed

training_loss.png ADDED Viewed