Instructions to use xummer/gemma2-9b-nli-P2-multi-n500-seed42-lora-all with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use xummer/gemma2-9b-nli-P2-multi-n500-seed42-lora-all with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("google/gemma-2-9b-it")
model = PeftModel.from_pretrained(base_model, "xummer/gemma2-9b-nli-P2-multi-n500-seed42-lora-all")

Transformers

How to use xummer/gemma2-9b-nli-P2-multi-n500-seed42-lora-all with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="xummer/gemma2-9b-nli-P2-multi-n500-seed42-lora-all")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("xummer/gemma2-9b-nli-P2-multi-n500-seed42-lora-all", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use xummer/gemma2-9b-nli-P2-multi-n500-seed42-lora-all with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "xummer/gemma2-9b-nli-P2-multi-n500-seed42-lora-all"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "xummer/gemma2-9b-nli-P2-multi-n500-seed42-lora-all",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/xummer/gemma2-9b-nli-P2-multi-n500-seed42-lora-all

SGLang

How to use xummer/gemma2-9b-nli-P2-multi-n500-seed42-lora-all with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "xummer/gemma2-9b-nli-P2-multi-n500-seed42-lora-all" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "xummer/gemma2-9b-nli-P2-multi-n500-seed42-lora-all",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "xummer/gemma2-9b-nli-P2-multi-n500-seed42-lora-all" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "xummer/gemma2-9b-nli-P2-multi-n500-seed42-lora-all",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use xummer/gemma2-9b-nli-P2-multi-n500-seed42-lora-all with Docker Model Runner:
```
docker model run hf.co/xummer/gemma2-9b-nli-P2-multi-n500-seed42-lora-all
```

xummer commited on May 25

Commit

04824d7

verified ·

1 Parent(s): 6b2e4bf

Upload folder using huggingface_hub

Browse files

Files changed (32) hide show

.gitattributes +3 -0
README.md +70 -0
adapter_config.json +46 -0
adapter_model.safetensors +3 -0
all_results.json +14 -0
chat_template.jinja +4 -0
checkpoint-500/README.md +208 -0
checkpoint-500/adapter_config.json +46 -0
checkpoint-500/adapter_model.safetensors +3 -0
checkpoint-500/chat_template.jinja +4 -0
checkpoint-500/tokenizer.json +3 -0
checkpoint-500/tokenizer_config.json +21 -0
checkpoint-500/trainer_state.json +394 -0
checkpoint-500/training_args.bin +3 -0
checkpoint-846/README.md +208 -0
checkpoint-846/adapter_config.json +46 -0
checkpoint-846/adapter_model.safetensors +3 -0
checkpoint-846/chat_template.jinja +4 -0
checkpoint-846/tokenizer.json +3 -0
checkpoint-846/tokenizer_config.json +21 -0
checkpoint-846/trainer_state.json +632 -0
checkpoint-846/training_args.bin +3 -0
eval_results.json +9 -0
tokenizer.json +3 -0
tokenizer_config.json +21 -0
train_results.json +8 -0
trainer_log.jsonl +86 -0
trainer_state.json +641 -0
training_args.bin +3 -0
training_eval_accuracy.png +0 -0
training_eval_loss.png +0 -0
training_loss.png +0 -0

.gitattributes CHANGED Viewed

@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+checkpoint-500/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-846/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md ADDED Viewed

	@@ -0,0 +1,70 @@

+---
+library_name: peft
+license: other
+base_model: google/gemma-2-9b-it
+tags:
+- base_model:adapter:google/gemma-2-9b-it
+- llama-factory
+- lora
+- transformers
+metrics:
+- accuracy
+pipeline_tag: text-generation
+model-index:
+- name: nli_P2_multi_n500_seed42
+  results: []
+---
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+# nli_P2_multi_n500_seed42
+This model is a fine-tuned version of [google/gemma-2-9b-it](https://huggingface.co/google/gemma-2-9b-it) on the nli_multi_n500_train dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.2139
+- Accuracy: 0.9458
+- Mcq Accuracy: 0.7311
+## Model description
+More information needed
+## Intended uses & limitations
+More information needed
+## Training and evaluation data
+More information needed
+## Training procedure
+### Training hyperparameters
+The following hyperparameters were used during training:
+- learning_rate: 0.0002
+- train_batch_size: 4
+- eval_batch_size: 4
+- seed: 42
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 16
+- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_steps: 0.1
+- num_epochs: 3.0
+### Training results
+| Training Loss | Epoch  | Step | Validation Loss | Accuracy | Mcq Accuracy |
+|:-------------:|:------:|:----:|:---------------:|:--------:|:------------:|
+| 0.0458        | 1.7751 | 500  | 0.1556          | 0.9427   | 0.7222       |
+### Framework versions
+- PEFT 0.18.1
+- Transformers 5.2.0
+- Pytorch 2.10.0+cu128
+- Datasets 4.0.0
+- Tokenizers 0.22.2

adapter_config.json ADDED Viewed

	@@ -0,0 +1,46 @@

+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "google/gemma-2-9b-it",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "down_proj",
+    "o_proj",
+    "k_proj",
+    "up_proj",
+    "v_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:25d2961ba873b47b3a0145266448f754de82479dc07e1e2a42d8e25899ec8820
+size 216151256

all_results.json ADDED Viewed

	@@ -0,0 +1,14 @@

+{
+    "epoch": 3.0,
+    "eval_accuracy": 0.9458148148148149,
+    "eval_loss": 0.21385908126831055,
+    "eval_mcq_accuracy": 0.7311111111111112,
+    "eval_runtime": 26.4009,
+    "eval_samples_per_second": 17.045,
+    "eval_steps_per_second": 4.28,
+    "total_flos": 1.0958550940031386e+17,
+    "train_loss": 0.06952556652285546,
+    "train_runtime": 2162.4382,
+    "train_samples_per_second": 6.243,
+    "train_steps_per_second": 0.391
+}

chat_template.jinja ADDED Viewed

	@@ -0,0 +1,4 @@

+{{ bos_token }}{% if messages[0]['role'] == 'system' %}{{ raise_exception('System role not supported') }}{% endif %}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if (message['role'] == 'assistant') %}{% set role = 'model' %}{% else %}{% set role = message['role'] %}{% endif %}{{ '<start_of_turn>' + role + '
+' + message['content'] | trim + '<end_of_turn>
+' }}{% endfor %}{% if add_generation_prompt %}{{'<start_of_turn>model
+'}}{% endif %}

checkpoint-500/README.md ADDED Viewed

	@@ -0,0 +1,208 @@

+---
+base_model: google/gemma-2-9b-it
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:google/gemma-2-9b-it
+- llama-factory
+- lora
+- transformers
+---
+# Model Card for Model ID
+<!-- Provide a quick summary of what the model is/does. -->
+## Model Details
+### Model Description
+<!-- Provide a longer summary of what this model is. -->
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+### Model Sources [optional]
+<!-- Provide the basic links for the model. -->
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+## Uses
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+### Direct Use
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+[More Information Needed]
+### Downstream Use [optional]
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+[More Information Needed]
+### Out-of-Scope Use
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+[More Information Needed]
+## Bias, Risks, and Limitations
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+[More Information Needed]
+### Recommendations
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+## How to Get Started with the Model
+Use the code below to get started with the model.
+[More Information Needed]
+## Training Details
+### Training Data
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+[More Information Needed]
+### Training Procedure
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+#### Preprocessing [optional]
+[More Information Needed]
+#### Training Hyperparameters
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+#### Speeds, Sizes, Times [optional]
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+[More Information Needed]
+## Evaluation
+<!-- This section describes the evaluation protocols and provides the results. -->
+### Testing Data, Factors & Metrics
+#### Testing Data
+<!-- This should link to a Dataset Card if possible. -->
+[More Information Needed]
+#### Factors
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+[More Information Needed]
+#### Metrics
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+[More Information Needed]
+### Results
+[More Information Needed]
+#### Summary
+## Model Examination [optional]
+<!-- Relevant interpretability work for the model goes here -->
+[More Information Needed]
+## Environmental Impact
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+## Technical Specifications [optional]
+### Model Architecture and Objective
+[More Information Needed]
+### Compute Infrastructure
+[More Information Needed]
+#### Hardware
+[More Information Needed]
+#### Software
+[More Information Needed]
+## Citation [optional]
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+**BibTeX:**
+[More Information Needed]
+**APA:**
+[More Information Needed]
+## Glossary [optional]
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+[More Information Needed]
+## More Information [optional]
+[More Information Needed]
+## Model Card Authors [optional]
+[More Information Needed]
+## Model Card Contact
+[More Information Needed]
+### Framework versions
+- PEFT 0.18.1

checkpoint-500/adapter_config.json ADDED Viewed

	@@ -0,0 +1,46 @@

+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "google/gemma-2-9b-it",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "down_proj",
+    "o_proj",
+    "k_proj",
+    "up_proj",
+    "v_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-500/adapter_model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a8015f958b88d941dfb41964df7e79a500ad58df6a4c8eb2df646d70ca1ac1ac
+size 216151256

checkpoint-500/chat_template.jinja ADDED Viewed

	@@ -0,0 +1,4 @@

+{{ bos_token }}{% if messages[0]['role'] == 'system' %}{{ raise_exception('System role not supported') }}{% endif %}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if (message['role'] == 'assistant') %}{% set role = 'model' %}{% else %}{% set role = message['role'] %}{% endif %}{{ '<start_of_turn>' + role + '
+' + message['content'] | trim + '<end_of_turn>
+' }}{% endfor %}{% if add_generation_prompt %}{{'<start_of_turn>model
+'}}{% endif %}

checkpoint-500/tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:394ace002a144ac6ad5486387502f2d36f70c087310c3d907857240c76fcb36e
+size 34362748

checkpoint-500/tokenizer_config.json ADDED Viewed

	@@ -0,0 +1,21 @@

+{
+  "backend": "tokenizers",
+  "bos_token": "<bos>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<eos>",
+  "extra_special_tokens": [
+    "<eos>",
+    "<end_of_turn>"
+  ],
+  "is_local": false,
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "padding_side": "right",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "split_special_tokens": false,
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

checkpoint-500/trainer_state.json ADDED Viewed

	@@ -0,0 +1,394 @@

+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.775111111111111,
+  "eval_steps": 500,
+  "global_step": 500,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.035555555555555556,
+      "grad_norm": 0.7644901871681213,
+      "learning_rate": 2.1176470588235296e-05,
+      "loss": 0.30520846843719485,
+      "step": 10
+    },
+    {
+      "epoch": 0.07111111111111111,
+      "grad_norm": 0.43560856580734253,
+      "learning_rate": 4.470588235294118e-05,
+      "loss": 0.26438264846801757,
+      "step": 20
+    },
+    {
+      "epoch": 0.10666666666666667,
+      "grad_norm": 0.2563800811767578,
+      "learning_rate": 6.823529411764707e-05,
+      "loss": 0.19181081056594848,
+      "step": 30
+    },
+    {
+      "epoch": 0.14222222222222222,
+      "grad_norm": 0.10259224474430084,
+      "learning_rate": 9.176470588235295e-05,
+      "loss": 0.15193926095962523,
+      "step": 40
+    },
+    {
+      "epoch": 0.17777777777777778,
+      "grad_norm": 0.1993834525346756,
+      "learning_rate": 0.00011529411764705881,
+      "loss": 0.13089948892593384,
+      "step": 50
+    },
+    {
+      "epoch": 0.21333333333333335,
+      "grad_norm": 0.19039888679981232,
+      "learning_rate": 0.00013882352941176472,
+      "loss": 0.1438336730003357,
+      "step": 60
+    },
+    {
+      "epoch": 0.24888888888888888,
+      "grad_norm": 0.153082013130188,
+      "learning_rate": 0.0001623529411764706,
+      "loss": 0.13305349349975587,
+      "step": 70
+    },
+    {
+      "epoch": 0.28444444444444444,
+      "grad_norm": 0.08913639187812805,
+      "learning_rate": 0.00018588235294117648,
+      "loss": 0.16499919891357423,
+      "step": 80
+    },
+    {
+      "epoch": 0.32,
+      "grad_norm": 0.18134109675884247,
+      "learning_rate": 0.00019998636639992777,
+      "loss": 0.13644593954086304,
+      "step": 90
+    },
+    {
+      "epoch": 0.35555555555555557,
+      "grad_norm": 0.09949612617492676,
+      "learning_rate": 0.00019983303108908946,
+      "loss": 0.13039498329162597,
+      "step": 100
+    },
+    {
+      "epoch": 0.39111111111111113,
+      "grad_norm": 0.12579314410686493,
+      "learning_rate": 0.00019950958062149127,
+      "loss": 0.1297929048538208,
+      "step": 110
+    },
+    {
+      "epoch": 0.4266666666666667,
+      "grad_norm": 0.1197328120470047,
+      "learning_rate": 0.00019901656615566656,
+      "loss": 0.1509210705757141,
+      "step": 120
+    },
+    {
+      "epoch": 0.4622222222222222,
+      "grad_norm": 0.15709449350833893,
+      "learning_rate": 0.00019835482778664425,
+      "loss": 0.14251822233200073,
+      "step": 130
+    },
+    {
+      "epoch": 0.49777777777777776,
+      "grad_norm": 0.2203192412853241,
+      "learning_rate": 0.0001975254931144296,
+      "loss": 0.1412465214729309,
+      "step": 140
+    },
+    {
+      "epoch": 0.5333333333333333,
+      "grad_norm": 0.32334810495376587,
+      "learning_rate": 0.0001965299753225775,
+      "loss": 0.1519417643547058,
+      "step": 150
+    },
+    {
+      "epoch": 0.5688888888888889,
+      "grad_norm": 0.1700880080461502,
+      "learning_rate": 0.00019536997077013236,
+      "loss": 0.13457289934158326,
+      "step": 160
+    },
+    {
+      "epoch": 0.6044444444444445,
+      "grad_norm": 0.17782148718833923,
+      "learning_rate": 0.00019404745610103786,
+      "loss": 0.14420714378356933,
+      "step": 170
+    },
+    {
+      "epoch": 0.64,
+      "grad_norm": 0.19534672796726227,
+      "learning_rate": 0.00019256468487594214,
+      "loss": 0.1206929087638855,
+      "step": 180
+    },
+    {
+      "epoch": 0.6755555555555556,
+      "grad_norm": 0.12961238622665405,
+      "learning_rate": 0.00019092418373213796,
+      "loss": 0.13416168689727784,
+      "step": 190
+    },
+    {
+      "epoch": 0.7111111111111111,
+      "grad_norm": 0.19645075500011444,
+      "learning_rate": 0.000189128748078181,
+      "loss": 0.13229730129241943,
+      "step": 200
+    },
+    {
+      "epoch": 0.7466666666666667,
+      "grad_norm": 0.17324262857437134,
+      "learning_rate": 0.00018718143733052278,
+      "loss": 0.12280937433242797,
+      "step": 210
+    },
+    {
+      "epoch": 0.7822222222222223,
+      "grad_norm": 0.10764078050851822,
+      "learning_rate": 0.0001850855697002753,
+      "loss": 0.12583138942718505,
+      "step": 220
+    },
+    {
+      "epoch": 0.8177777777777778,
+      "grad_norm": 0.13804763555526733,
+      "learning_rate": 0.00018284471653898994,
+      "loss": 0.13474913835525512,
+      "step": 230
+    },
+    {
+      "epoch": 0.8533333333333334,
+      "grad_norm": 0.11816427856683731,
+      "learning_rate": 0.00018046269625308648,
+      "loss": 0.13484352827072144,
+      "step": 240
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "grad_norm": 0.09677577018737793,
+      "learning_rate": 0.00017794356779730084,
+      "loss": 0.14849199056625367,
+      "step": 250
+    },
+    {
+      "epoch": 0.9244444444444444,
+      "grad_norm": 0.10201010853052139,
+      "learning_rate": 0.00017529162375823958,
+      "loss": 0.12720253467559814,
+      "step": 260
+    },
+    {
+      "epoch": 0.96,
+      "grad_norm": 0.23005172610282898,
+      "learning_rate": 0.00017251138303982675,
+      "loss": 0.11229866743087769,
+      "step": 270
+    },
+    {
+      "epoch": 0.9955555555555555,
+      "grad_norm": 0.15584969520568848,
+      "learning_rate": 0.00016960758316310597,
+      "loss": 0.12440342903137207,
+      "step": 280
+    },
+    {
+      "epoch": 1.0284444444444445,
+      "grad_norm": 0.08643736690282822,
+      "learning_rate": 0.0001665851721935205,
+      "loss": 0.0838462769985199,
+      "step": 290
+    },
+    {
+      "epoch": 1.064,
+      "grad_norm": 0.061173878610134125,
+      "learning_rate": 0.0001634493003094259,
+      "loss": 0.05495935678482056,
+      "step": 300
+    },
+    {
+      "epoch": 1.0995555555555556,
+      "grad_norm": 0.3276287019252777,
+      "learning_rate": 0.00016020531102620304,
+      "loss": 0.055119764804840085,
+      "step": 310
+    },
+    {
+      "epoch": 1.1351111111111112,
+      "grad_norm": 0.1756429374217987,
+      "learning_rate": 0.0001568587320909255,
+      "loss": 0.0650447130203247,
+      "step": 320
+    },
+    {
+      "epoch": 1.1706666666666667,
+      "grad_norm": 0.1355263888835907,
+      "learning_rate": 0.00015341526606309645,
+      "loss": 0.06544734239578247,
+      "step": 330
+    },
+    {
+      "epoch": 1.2062222222222223,
+      "grad_norm": 0.20916008949279785,
+      "learning_rate": 0.00014988078059750652,
+      "loss": 0.06727538704872131,
+      "step": 340
+    },
+    {
+      "epoch": 1.2417777777777779,
+      "grad_norm": 0.19566652178764343,
+      "learning_rate": 0.00014626129844576893,
+      "loss": 0.06408223509788513,
+      "step": 350
+    },
+    {
+      "epoch": 1.2773333333333334,
+      "grad_norm": 0.05987081304192543,
+      "learning_rate": 0.00014256298719357062,
+      "loss": 0.04535002112388611,
+      "step": 360
+    },
+    {
+      "epoch": 1.3128888888888888,
+      "grad_norm": 0.1902439296245575,
+      "learning_rate": 0.00013879214875112665,
+      "loss": 0.04922315180301666,
+      "step": 370
+    },
+    {
+      "epoch": 1.3484444444444446,
+      "grad_norm": 0.25207415223121643,
+      "learning_rate": 0.00013495520861474565,
+      "loss": 0.060137057304382326,
+      "step": 380
+    },
+    {
+      "epoch": 1.384,
+      "grad_norm": 0.34174537658691406,
+      "learning_rate": 0.00013105870491780558,
+      "loss": 0.05463656783103943,
+      "step": 390
+    },
+    {
+      "epoch": 1.4195555555555557,
+      "grad_norm": 0.19624291360378265,
+      "learning_rate": 0.00012710927728979568,
+      "loss": 0.06046912670135498,
+      "step": 400
+    },
+    {
+      "epoch": 1.455111111111111,
+      "grad_norm": 0.22849531471729279,
+      "learning_rate": 0.00012311365554240971,
+      "loss": 0.039173880219459535,
+      "step": 410
+    },
+    {
+      "epoch": 1.4906666666666666,
+      "grad_norm": 0.22941145300865173,
+      "learning_rate": 0.0001190786482019691,
+      "loss": 0.04884783029556274,
+      "step": 420
+    },
+    {
+      "epoch": 1.5262222222222221,
+      "grad_norm": 0.2867370843887329,
+      "learning_rate": 0.00011501113090771619,
+      "loss": 0.05086652636528015,
+      "step": 430
+    },
+    {
+      "epoch": 1.561777777777778,
+      "grad_norm": 0.27643826603889465,
+      "learning_rate": 0.00011091803469574789,
+      "loss": 0.04540249407291412,
+      "step": 440
+    },
+    {
+      "epoch": 1.5973333333333333,
+      "grad_norm": 0.1365540772676468,
+      "learning_rate": 0.00010680633418855267,
+      "loss": 0.0644813358783722,
+      "step": 450
+    },
+    {
+      "epoch": 1.6328888888888888,
+      "grad_norm": 0.21614767611026764,
+      "learning_rate": 0.00010268303571027696,
+      "loss": 0.07455622553825378,
+      "step": 460
+    },
+    {
+      "epoch": 1.6684444444444444,
+      "grad_norm": 0.08498027920722961,
+      "learning_rate": 9.855516534797187e-05,
+      "loss": 0.04775569438934326,
+      "step": 470
+    },
+    {
+      "epoch": 1.704,
+      "grad_norm": 0.2029683142900467,
+      "learning_rate": 9.442975697916372e-05,
+      "loss": 0.04402676820755005,
+      "step": 480
+    },
+    {
+      "epoch": 1.7395555555555555,
+      "grad_norm": 0.34462177753448486,
+      "learning_rate": 9.031384028615004e-05,
+      "loss": 0.05336908102035522,
+      "step": 490
+    },
+    {
+      "epoch": 1.775111111111111,
+      "grad_norm": 0.05993572995066643,
+      "learning_rate": 8.621442877744409e-05,
+      "loss": 0.04580667018890381,
+      "step": 500
+    },
+    {
+      "epoch": 1.775111111111111,
+      "eval_accuracy": 0.9426666666666668,
+      "eval_loss": 0.15560266375541687,
+      "eval_mcq_accuracy": 0.7222222222222222,
+      "eval_runtime": 26.3716,
+      "eval_samples_per_second": 17.064,
+      "eval_steps_per_second": 4.285,
+      "step": 500
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 846,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 6.465519316726579e+16,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-500/training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:06e72e4c171697e89ed577da3f438cb2d6210bd6fe361313102cda2f6b7351b8
+size 5649

checkpoint-846/README.md ADDED Viewed

	@@ -0,0 +1,208 @@

+---
+base_model: google/gemma-2-9b-it
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:google/gemma-2-9b-it
+- llama-factory
+- lora
+- transformers
+---
+# Model Card for Model ID
+<!-- Provide a quick summary of what the model is/does. -->
+## Model Details
+### Model Description
+<!-- Provide a longer summary of what this model is. -->
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+### Model Sources [optional]
+<!-- Provide the basic links for the model. -->
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+## Uses
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+### Direct Use
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+[More Information Needed]
+### Downstream Use [optional]
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+[More Information Needed]
+### Out-of-Scope Use
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+[More Information Needed]
+## Bias, Risks, and Limitations
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+[More Information Needed]
+### Recommendations
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+## How to Get Started with the Model
+Use the code below to get started with the model.
+[More Information Needed]
+## Training Details
+### Training Data
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+[More Information Needed]
+### Training Procedure
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+#### Preprocessing [optional]
+[More Information Needed]
+#### Training Hyperparameters
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+#### Speeds, Sizes, Times [optional]
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+[More Information Needed]
+## Evaluation
+<!-- This section describes the evaluation protocols and provides the results. -->
+### Testing Data, Factors & Metrics
+#### Testing Data
+<!-- This should link to a Dataset Card if possible. -->
+[More Information Needed]
+#### Factors
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+[More Information Needed]
+#### Metrics
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+[More Information Needed]
+### Results
+[More Information Needed]
+#### Summary
+## Model Examination [optional]
+<!-- Relevant interpretability work for the model goes here -->
+[More Information Needed]
+## Environmental Impact
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+## Technical Specifications [optional]
+### Model Architecture and Objective
+[More Information Needed]
+### Compute Infrastructure
+[More Information Needed]
+#### Hardware
+[More Information Needed]
+#### Software
+[More Information Needed]
+## Citation [optional]
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+**BibTeX:**
+[More Information Needed]
+**APA:**
+[More Information Needed]
+## Glossary [optional]
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+[More Information Needed]
+## More Information [optional]
+[More Information Needed]
+## Model Card Authors [optional]
+[More Information Needed]
+## Model Card Contact
+[More Information Needed]
+### Framework versions
+- PEFT 0.18.1

checkpoint-846/adapter_config.json ADDED Viewed

	@@ -0,0 +1,46 @@

+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "google/gemma-2-9b-it",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "down_proj",
+    "o_proj",
+    "k_proj",
+    "up_proj",
+    "v_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-846/adapter_model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:25d2961ba873b47b3a0145266448f754de82479dc07e1e2a42d8e25899ec8820
+size 216151256

checkpoint-846/chat_template.jinja ADDED Viewed

	@@ -0,0 +1,4 @@

+{{ bos_token }}{% if messages[0]['role'] == 'system' %}{{ raise_exception('System role not supported') }}{% endif %}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if (message['role'] == 'assistant') %}{% set role = 'model' %}{% else %}{% set role = message['role'] %}{% endif %}{{ '<start_of_turn>' + role + '
+' + message['content'] | trim + '<end_of_turn>
+' }}{% endfor %}{% if add_generation_prompt %}{{'<start_of_turn>model
+'}}{% endif %}

checkpoint-846/tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:394ace002a144ac6ad5486387502f2d36f70c087310c3d907857240c76fcb36e
+size 34362748

checkpoint-846/tokenizer_config.json ADDED Viewed

	@@ -0,0 +1,21 @@

+{
+  "backend": "tokenizers",
+  "bos_token": "<bos>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<eos>",
+  "extra_special_tokens": [
+    "<eos>",
+    "<end_of_turn>"
+  ],
+  "is_local": false,
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "padding_side": "right",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "split_special_tokens": false,
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

checkpoint-846/trainer_state.json ADDED Viewed

	@@ -0,0 +1,632 @@

+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 3.0,
+  "eval_steps": 500,
+  "global_step": 846,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.035555555555555556,
+      "grad_norm": 0.7644901871681213,
+      "learning_rate": 2.1176470588235296e-05,
+      "loss": 0.30520846843719485,
+      "step": 10
+    },
+    {
+      "epoch": 0.07111111111111111,
+      "grad_norm": 0.43560856580734253,
+      "learning_rate": 4.470588235294118e-05,
+      "loss": 0.26438264846801757,
+      "step": 20
+    },
+    {
+      "epoch": 0.10666666666666667,
+      "grad_norm": 0.2563800811767578,
+      "learning_rate": 6.823529411764707e-05,
+      "loss": 0.19181081056594848,
+      "step": 30
+    },
+    {
+      "epoch": 0.14222222222222222,
+      "grad_norm": 0.10259224474430084,
+      "learning_rate": 9.176470588235295e-05,
+      "loss": 0.15193926095962523,
+      "step": 40
+    },
+    {
+      "epoch": 0.17777777777777778,
+      "grad_norm": 0.1993834525346756,
+      "learning_rate": 0.00011529411764705881,
+      "loss": 0.13089948892593384,
+      "step": 50
+    },
+    {
+      "epoch": 0.21333333333333335,
+      "grad_norm": 0.19039888679981232,
+      "learning_rate": 0.00013882352941176472,
+      "loss": 0.1438336730003357,
+      "step": 60
+    },
+    {
+      "epoch": 0.24888888888888888,
+      "grad_norm": 0.153082013130188,
+      "learning_rate": 0.0001623529411764706,
+      "loss": 0.13305349349975587,
+      "step": 70
+    },
+    {
+      "epoch": 0.28444444444444444,
+      "grad_norm": 0.08913639187812805,
+      "learning_rate": 0.00018588235294117648,
+      "loss": 0.16499919891357423,
+      "step": 80
+    },
+    {
+      "epoch": 0.32,
+      "grad_norm": 0.18134109675884247,
+      "learning_rate": 0.00019998636639992777,
+      "loss": 0.13644593954086304,
+      "step": 90
+    },
+    {
+      "epoch": 0.35555555555555557,
+      "grad_norm": 0.09949612617492676,
+      "learning_rate": 0.00019983303108908946,
+      "loss": 0.13039498329162597,
+      "step": 100
+    },
+    {
+      "epoch": 0.39111111111111113,
+      "grad_norm": 0.12579314410686493,
+      "learning_rate": 0.00019950958062149127,
+      "loss": 0.1297929048538208,
+      "step": 110
+    },
+    {
+      "epoch": 0.4266666666666667,
+      "grad_norm": 0.1197328120470047,
+      "learning_rate": 0.00019901656615566656,
+      "loss": 0.1509210705757141,
+      "step": 120
+    },
+    {
+      "epoch": 0.4622222222222222,
+      "grad_norm": 0.15709449350833893,
+      "learning_rate": 0.00019835482778664425,
+      "loss": 0.14251822233200073,
+      "step": 130
+    },
+    {
+      "epoch": 0.49777777777777776,
+      "grad_norm": 0.2203192412853241,
+      "learning_rate": 0.0001975254931144296,
+      "loss": 0.1412465214729309,
+      "step": 140
+    },
+    {
+      "epoch": 0.5333333333333333,
+      "grad_norm": 0.32334810495376587,
+      "learning_rate": 0.0001965299753225775,
+      "loss": 0.1519417643547058,
+      "step": 150
+    },
+    {
+      "epoch": 0.5688888888888889,
+      "grad_norm": 0.1700880080461502,
+      "learning_rate": 0.00019536997077013236,
+      "loss": 0.13457289934158326,
+      "step": 160
+    },
+    {
+      "epoch": 0.6044444444444445,
+      "grad_norm": 0.17782148718833923,
+      "learning_rate": 0.00019404745610103786,
+      "loss": 0.14420714378356933,
+      "step": 170
+    },
+    {
+      "epoch": 0.64,
+      "grad_norm": 0.19534672796726227,
+      "learning_rate": 0.00019256468487594214,
+      "loss": 0.1206929087638855,
+      "step": 180
+    },
+    {
+      "epoch": 0.6755555555555556,
+      "grad_norm": 0.12961238622665405,
+      "learning_rate": 0.00019092418373213796,
+      "loss": 0.13416168689727784,
+      "step": 190
+    },
+    {
+      "epoch": 0.7111111111111111,
+      "grad_norm": 0.19645075500011444,
+      "learning_rate": 0.000189128748078181,
+      "loss": 0.13229730129241943,
+      "step": 200
+    },
+    {
+      "epoch": 0.7466666666666667,
+      "grad_norm": 0.17324262857437134,
+      "learning_rate": 0.00018718143733052278,
+      "loss": 0.12280937433242797,
+      "step": 210
+    },
+    {
+      "epoch": 0.7822222222222223,
+      "grad_norm": 0.10764078050851822,
+      "learning_rate": 0.0001850855697002753,
+      "loss": 0.12583138942718505,
+      "step": 220
+    },
+    {
+      "epoch": 0.8177777777777778,
+      "grad_norm": 0.13804763555526733,
+      "learning_rate": 0.00018284471653898994,
+      "loss": 0.13474913835525512,
+      "step": 230
+    },
+    {
+      "epoch": 0.8533333333333334,
+      "grad_norm": 0.11816427856683731,
+      "learning_rate": 0.00018046269625308648,
+      "loss": 0.13484352827072144,
+      "step": 240
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "grad_norm": 0.09677577018737793,
+      "learning_rate": 0.00017794356779730084,
+      "loss": 0.14849199056625367,
+      "step": 250
+    },
+    {
+      "epoch": 0.9244444444444444,
+      "grad_norm": 0.10201010853052139,
+      "learning_rate": 0.00017529162375823958,
+      "loss": 0.12720253467559814,
+      "step": 260
+    },
+    {
+      "epoch": 0.96,
+      "grad_norm": 0.23005172610282898,
+      "learning_rate": 0.00017251138303982675,
+      "loss": 0.11229866743087769,
+      "step": 270
+    },
+    {
+      "epoch": 0.9955555555555555,
+      "grad_norm": 0.15584969520568848,
+      "learning_rate": 0.00016960758316310597,
+      "loss": 0.12440342903137207,
+      "step": 280
+    },
+    {
+      "epoch": 1.0284444444444445,
+      "grad_norm": 0.08643736690282822,
+      "learning_rate": 0.0001665851721935205,
+      "loss": 0.0838462769985199,
+      "step": 290
+    },
+    {
+      "epoch": 1.064,
+      "grad_norm": 0.061173878610134125,
+      "learning_rate": 0.0001634493003094259,
+      "loss": 0.05495935678482056,
+      "step": 300
+    },
+    {
+      "epoch": 1.0995555555555556,
+      "grad_norm": 0.3276287019252777,
+      "learning_rate": 0.00016020531102620304,
+      "loss": 0.055119764804840085,
+      "step": 310
+    },
+    {
+      "epoch": 1.1351111111111112,
+      "grad_norm": 0.1756429374217987,
+      "learning_rate": 0.0001568587320909255,
+      "loss": 0.0650447130203247,
+      "step": 320
+    },
+    {
+      "epoch": 1.1706666666666667,
+      "grad_norm": 0.1355263888835907,
+      "learning_rate": 0.00015341526606309645,
+      "loss": 0.06544734239578247,
+      "step": 330
+    },
+    {
+      "epoch": 1.2062222222222223,
+      "grad_norm": 0.20916008949279785,
+      "learning_rate": 0.00014988078059750652,
+      "loss": 0.06727538704872131,
+      "step": 340
+    },
+    {
+      "epoch": 1.2417777777777779,
+      "grad_norm": 0.19566652178764343,
+      "learning_rate": 0.00014626129844576893,
+      "loss": 0.06408223509788513,
+      "step": 350
+    },
+    {
+      "epoch": 1.2773333333333334,
+      "grad_norm": 0.05987081304192543,
+      "learning_rate": 0.00014256298719357062,
+      "loss": 0.04535002112388611,
+      "step": 360
+    },
+    {
+      "epoch": 1.3128888888888888,
+      "grad_norm": 0.1902439296245575,
+      "learning_rate": 0.00013879214875112665,
+      "loss": 0.04922315180301666,
+      "step": 370
+    },
+    {
+      "epoch": 1.3484444444444446,
+      "grad_norm": 0.25207415223121643,
+      "learning_rate": 0.00013495520861474565,
+      "loss": 0.060137057304382326,
+      "step": 380
+    },
+    {
+      "epoch": 1.384,
+      "grad_norm": 0.34174537658691406,
+      "learning_rate": 0.00013105870491780558,
+      "loss": 0.05463656783103943,
+      "step": 390
+    },
+    {
+      "epoch": 1.4195555555555557,
+      "grad_norm": 0.19624291360378265,
+      "learning_rate": 0.00012710927728979568,
+      "loss": 0.06046912670135498,
+      "step": 400
+    },
+    {
+      "epoch": 1.455111111111111,
+      "grad_norm": 0.22849531471729279,
+      "learning_rate": 0.00012311365554240971,
+      "loss": 0.039173880219459535,
+      "step": 410
+    },
+    {
+      "epoch": 1.4906666666666666,
+      "grad_norm": 0.22941145300865173,
+      "learning_rate": 0.0001190786482019691,
+      "loss": 0.04884783029556274,
+      "step": 420
+    },
+    {
+      "epoch": 1.5262222222222221,
+      "grad_norm": 0.2867370843887329,
+      "learning_rate": 0.00011501113090771619,
+      "loss": 0.05086652636528015,
+      "step": 430
+    },
+    {
+      "epoch": 1.561777777777778,
+      "grad_norm": 0.27643826603889465,
+      "learning_rate": 0.00011091803469574789,
+      "loss": 0.04540249407291412,
+      "step": 440
+    },
+    {
+      "epoch": 1.5973333333333333,
+      "grad_norm": 0.1365540772676468,
+      "learning_rate": 0.00010680633418855267,
+      "loss": 0.0644813358783722,
+      "step": 450
+    },
+    {
+      "epoch": 1.6328888888888888,
+      "grad_norm": 0.21614767611026764,
+      "learning_rate": 0.00010268303571027696,
+      "loss": 0.07455622553825378,
+      "step": 460
+    },
+    {
+      "epoch": 1.6684444444444444,
+      "grad_norm": 0.08498027920722961,
+      "learning_rate": 9.855516534797187e-05,
+      "loss": 0.04775569438934326,
+      "step": 470
+    },
+    {
+      "epoch": 1.704,
+      "grad_norm": 0.2029683142900467,
+      "learning_rate": 9.442975697916372e-05,
+      "loss": 0.04402676820755005,
+      "step": 480
+    },
+    {
+      "epoch": 1.7395555555555555,
+      "grad_norm": 0.34462177753448486,
+      "learning_rate": 9.031384028615004e-05,
+      "loss": 0.05336908102035522,
+      "step": 490
+    },
+    {
+      "epoch": 1.775111111111111,
+      "grad_norm": 0.05993572995066643,
+      "learning_rate": 8.621442877744409e-05,
+      "loss": 0.04580667018890381,
+      "step": 500
+    },
+    {
+      "epoch": 1.775111111111111,
+      "eval_accuracy": 0.9426666666666668,
+      "eval_loss": 0.15560266375541687,
+      "eval_mcq_accuracy": 0.7222222222222222,
+      "eval_runtime": 26.3716,
+      "eval_samples_per_second": 17.064,
+      "eval_steps_per_second": 4.285,
+      "step": 500
+    },
+    {
+      "epoch": 1.8106666666666666,
+      "grad_norm": 0.3891923427581787,
+      "learning_rate": 8.213850783677925e-05,
+      "loss": 0.03394646048545837,
+      "step": 510
+    },
+    {
+      "epoch": 1.8462222222222222,
+      "grad_norm": 0.1975124329328537,
+      "learning_rate": 7.809302282003823e-05,
+      "loss": 0.054677408933639524,
+      "step": 520
+    },
+    {
+      "epoch": 1.8817777777777778,
+      "grad_norm": 0.1962609589099884,
+      "learning_rate": 7.408486722038943e-05,
+      "loss": 0.06665679812431335,
+      "step": 530
+    },
+    {
+      "epoch": 1.9173333333333333,
+      "grad_norm": 0.2969353199005127,
+      "learning_rate": 7.012087092179724e-05,
+      "loss": 0.048915204405784604,
+      "step": 540
+    },
+    {
+      "epoch": 1.952888888888889,
+      "grad_norm": 0.23223088681697845,
+      "learning_rate": 6.620778856092227e-05,
+      "loss": 0.04671376347541809,
+      "step": 550
+    },
+    {
+      "epoch": 1.9884444444444445,
+      "grad_norm": 0.4542539417743683,
+      "learning_rate": 6.235228801724253e-05,
+      "loss": 0.05778223276138306,
+      "step": 560
+    },
+    {
+      "epoch": 2.021333333333333,
+      "grad_norm": 0.09194125235080719,
+      "learning_rate": 5.856093905100899e-05,
+      "loss": 0.022845838963985444,
+      "step": 570
+    },
+    {
+      "epoch": 2.056888888888889,
+      "grad_norm": 0.10670984536409378,
+      "learning_rate": 5.4840202108395466e-05,
+      "loss": 0.00663934126496315,
+      "step": 580
+    },
+    {
+      "epoch": 2.0924444444444443,
+      "grad_norm": 0.015582868829369545,
+      "learning_rate": 5.119641731291971e-05,
+      "loss": 0.007281148433685302,
+      "step": 590
+    },
+    {
+      "epoch": 2.128,
+      "grad_norm": 0.17645655572414398,
+      "learning_rate": 4.7635793661893666e-05,
+      "loss": 0.00572865828871727,
+      "step": 600
+    },
+    {
+      "epoch": 2.1635555555555555,
+      "grad_norm": 0.46430906653404236,
+      "learning_rate": 4.416439844631271e-05,
+      "loss": 0.008638855814933778,
+      "step": 610
+    },
+    {
+      "epoch": 2.1991111111111112,
+      "grad_norm": 0.011537984013557434,
+      "learning_rate": 4.078814691221139e-05,
+      "loss": 0.004736468568444252,
+      "step": 620
+    },
+    {
+      "epoch": 2.2346666666666666,
+      "grad_norm": 0.00896433461457491,
+      "learning_rate": 3.751279218110387e-05,
+      "loss": 0.007740923762321472,
+      "step": 630
+    },
+    {
+      "epoch": 2.2702222222222224,
+      "grad_norm": 0.002848990960046649,
+      "learning_rate": 3.434391544668383e-05,
+      "loss": 0.007571302354335785,
+      "step": 640
+    },
+    {
+      "epoch": 2.3057777777777777,
+      "grad_norm": 0.01570839062333107,
+      "learning_rate": 3.1286916464488505e-05,
+      "loss": 0.006724107265472412,
+      "step": 650
+    },
+    {
+      "epoch": 2.3413333333333335,
+      "grad_norm": 0.003965158946812153,
+      "learning_rate": 2.8347004350733185e-05,
+      "loss": 0.008114227652549743,
+      "step": 660
+    },
+    {
+      "epoch": 2.376888888888889,
+      "grad_norm": 0.10036994516849518,
+      "learning_rate": 2.55291887059944e-05,
+      "loss": 0.0027894463390111925,
+      "step": 670
+    },
+    {
+      "epoch": 2.4124444444444446,
+      "grad_norm": 0.1702636480331421,
+      "learning_rate": 2.2838271078866714e-05,
+      "loss": 0.0071901664137840274,
+      "step": 680
+    },
+    {
+      "epoch": 2.448,
+      "grad_norm": 0.003337263595312834,
+      "learning_rate": 2.0278836784140044e-05,
+      "loss": 0.0033892091363668443,
+      "step": 690
+    },
+    {
+      "epoch": 2.4835555555555557,
+      "grad_norm": 0.0054590729996562,
+      "learning_rate": 1.785524708943802e-05,
+      "loss": 0.002477823756635189,
+      "step": 700
+    },
+    {
+      "epoch": 2.519111111111111,
+      "grad_norm": 0.02814805507659912,
+      "learning_rate": 1.557163178363251e-05,
+      "loss": 0.0032980531454086305,
+      "step": 710
+    },
+    {
+      "epoch": 2.554666666666667,
+      "grad_norm": 0.01721755601465702,
+      "learning_rate": 1.3431882139696916e-05,
+      "loss": 0.00092921182513237,
+      "step": 720
+    },
+    {
+      "epoch": 2.590222222222222,
+      "grad_norm": 0.004700134973973036,
+      "learning_rate": 1.1439644283989747e-05,
+      "loss": 0.008682060241699218,
+      "step": 730
+    },
+    {
+      "epoch": 2.6257777777777775,
+      "grad_norm": 0.02318074181675911,
+      "learning_rate": 9.59831298326731e-06,
+      "loss": 0.0035014577209949494,
+      "step": 740
+    },
+    {
+      "epoch": 2.6613333333333333,
+      "grad_norm": 0.10873377323150635,
+      "learning_rate": 7.911025860012444e-06,
+      "loss": 0.007874777913093567,
+      "step": 750
+    },
+    {
+      "epoch": 2.696888888888889,
+      "grad_norm": 0.007307600695639849,
+      "learning_rate": 6.38065804593595e-06,
+      "loss": 0.003820793330669403,
+      "step": 760
+    },
+    {
+      "epoch": 2.7324444444444445,
+      "grad_norm": 0.002649878617376089,
+      "learning_rate": 5.009817282761675e-06,
+      "loss": 0.0014227939769625663,
+      "step": 770
+    },
+    {
+      "epoch": 2.768,
+      "grad_norm": 0.02097943052649498,
+      "learning_rate": 3.800839478643259e-06,
+      "loss": 0.007305952161550522,
+      "step": 780
+    },
+    {
+      "epoch": 2.8035555555555556,
+      "grad_norm": 0.11966679990291595,
+      "learning_rate": 2.7557847277841942e-06,
+      "loss": 0.004338302463293075,
+      "step": 790
+    },
+    {
+      "epoch": 2.8391111111111114,
+      "grad_norm": 0.07167425006628036,
+      "learning_rate": 1.8764338000442083e-06,
+      "loss": 0.002496876008808613,
+      "step": 800
+    },
+    {
+      "epoch": 2.8746666666666667,
+      "grad_norm": 0.010830031707882881,
+      "learning_rate": 1.1642851065131633e-06,
+      "loss": 0.0015433511696755886,
+      "step": 810
+    },
+    {
+      "epoch": 2.910222222222222,
+      "grad_norm": 0.195694237947464,
+      "learning_rate": 6.205521462235186e-07,
+      "loss": 0.001970846764743328,
+      "step": 820
+    },
+    {
+      "epoch": 2.945777777777778,
+      "grad_norm": 0.00489334249868989,
+      "learning_rate": 2.4616143835202166e-07,
+      "loss": 0.010426017642021179,
+      "step": 830
+    },
+    {
+      "epoch": 2.981333333333333,
+      "grad_norm": 0.009853039868175983,
+      "learning_rate": 4.1750943434026855e-08,
+      "loss": 0.0070929393172264096,
+      "step": 840
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 846,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.0958550940031386e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-846/training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:06e72e4c171697e89ed577da3f438cb2d6210bd6fe361313102cda2f6b7351b8
+size 5649

eval_results.json ADDED Viewed

	@@ -0,0 +1,9 @@

+{
+    "epoch": 3.0,
+    "eval_accuracy": 0.9458148148148149,
+    "eval_loss": 0.21385908126831055,
+    "eval_mcq_accuracy": 0.7311111111111112,
+    "eval_runtime": 26.4009,
+    "eval_samples_per_second": 17.045,
+    "eval_steps_per_second": 4.28
+}

tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:394ace002a144ac6ad5486387502f2d36f70c087310c3d907857240c76fcb36e
+size 34362748

tokenizer_config.json ADDED Viewed

	@@ -0,0 +1,21 @@

+{
+  "backend": "tokenizers",
+  "bos_token": "<bos>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<eos>",
+  "extra_special_tokens": [
+    "<eos>",
+    "<end_of_turn>"
+  ],
+  "is_local": false,
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "padding_side": "right",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "split_special_tokens": false,
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

train_results.json ADDED Viewed

	@@ -0,0 +1,8 @@

+{
+    "epoch": 3.0,
+    "total_flos": 1.0958550940031386e+17,
+    "train_loss": 0.06952556652285546,
+    "train_runtime": 2162.4382,
+    "train_samples_per_second": 6.243,
+    "train_steps_per_second": 0.391
+}

trainer_log.jsonl ADDED Viewed

	@@ -0,0 +1,86 @@

+{"current_steps": 10, "total_steps": 846, "loss": 0.30520846843719485, "lr": 2.1176470588235296e-05, "epoch": 0.035555555555555556, "percentage": 1.18, "elapsed_time": "0:00:25", "remaining_time": "0:36:12"}
+{"current_steps": 20, "total_steps": 846, "loss": 0.26438264846801757, "lr": 4.470588235294118e-05, "epoch": 0.07111111111111111, "percentage": 2.36, "elapsed_time": "0:00:49", "remaining_time": "0:34:05"}
+{"current_steps": 30, "total_steps": 846, "loss": 0.19181081056594848, "lr": 6.823529411764707e-05, "epoch": 0.10666666666666667, "percentage": 3.55, "elapsed_time": "0:01:14", "remaining_time": "0:33:51"}
+{"current_steps": 40, "total_steps": 846, "loss": 0.15193926095962523, "lr": 9.176470588235295e-05, "epoch": 0.14222222222222222, "percentage": 4.73, "elapsed_time": "0:01:42", "remaining_time": "0:34:33"}
+{"current_steps": 50, "total_steps": 846, "loss": 0.13089948892593384, "lr": 0.00011529411764705881, "epoch": 0.17777777777777778, "percentage": 5.91, "elapsed_time": "0:02:08", "remaining_time": "0:34:01"}
+{"current_steps": 60, "total_steps": 846, "loss": 0.1438336730003357, "lr": 0.00013882352941176472, "epoch": 0.21333333333333335, "percentage": 7.09, "elapsed_time": "0:02:33", "remaining_time": "0:33:30"}
+{"current_steps": 70, "total_steps": 846, "loss": 0.13305349349975587, "lr": 0.0001623529411764706, "epoch": 0.24888888888888888, "percentage": 8.27, "elapsed_time": "0:02:57", "remaining_time": "0:32:46"}
+{"current_steps": 80, "total_steps": 846, "loss": 0.16499919891357423, "lr": 0.00018588235294117648, "epoch": 0.28444444444444444, "percentage": 9.46, "elapsed_time": "0:03:22", "remaining_time": "0:32:16"}
+{"current_steps": 90, "total_steps": 846, "loss": 0.13644593954086304, "lr": 0.00019998636639992777, "epoch": 0.32, "percentage": 10.64, "elapsed_time": "0:03:46", "remaining_time": "0:31:39"}
+{"current_steps": 100, "total_steps": 846, "loss": 0.13039498329162597, "lr": 0.00019983303108908946, "epoch": 0.35555555555555557, "percentage": 11.82, "elapsed_time": "0:04:10", "remaining_time": "0:31:10"}
+{"current_steps": 110, "total_steps": 846, "loss": 0.1297929048538208, "lr": 0.00019950958062149127, "epoch": 0.39111111111111113, "percentage": 13.0, "elapsed_time": "0:04:34", "remaining_time": "0:30:37"}
+{"current_steps": 120, "total_steps": 846, "loss": 0.1509210705757141, "lr": 0.00019901656615566656, "epoch": 0.4266666666666667, "percentage": 14.18, "elapsed_time": "0:04:59", "remaining_time": "0:30:10"}
+{"current_steps": 130, "total_steps": 846, "loss": 0.14251822233200073, "lr": 0.00019835482778664425, "epoch": 0.4622222222222222, "percentage": 15.37, "elapsed_time": "0:05:25", "remaining_time": "0:29:55"}
+{"current_steps": 140, "total_steps": 846, "loss": 0.1412465214729309, "lr": 0.0001975254931144296, "epoch": 0.49777777777777776, "percentage": 16.55, "elapsed_time": "0:05:52", "remaining_time": "0:29:37"}
+{"current_steps": 150, "total_steps": 846, "loss": 0.1519417643547058, "lr": 0.0001965299753225775, "epoch": 0.5333333333333333, "percentage": 17.73, "elapsed_time": "0:06:15", "remaining_time": "0:29:04"}
+{"current_steps": 160, "total_steps": 846, "loss": 0.13457289934158326, "lr": 0.00019536997077013236, "epoch": 0.5688888888888889, "percentage": 18.91, "elapsed_time": "0:06:41", "remaining_time": "0:28:41"}
+{"current_steps": 170, "total_steps": 846, "loss": 0.14420714378356933, "lr": 0.00019404745610103786, "epoch": 0.6044444444444445, "percentage": 20.09, "elapsed_time": "0:07:09", "remaining_time": "0:28:26"}
+{"current_steps": 180, "total_steps": 846, "loss": 0.1206929087638855, "lr": 0.00019256468487594214, "epoch": 0.64, "percentage": 21.28, "elapsed_time": "0:07:35", "remaining_time": "0:28:04"}
+{"current_steps": 190, "total_steps": 846, "loss": 0.13416168689727784, "lr": 0.00019092418373213796, "epoch": 0.6755555555555556, "percentage": 22.46, "elapsed_time": "0:07:59", "remaining_time": "0:27:34"}
+{"current_steps": 200, "total_steps": 846, "loss": 0.13229730129241943, "lr": 0.000189128748078181, "epoch": 0.7111111111111111, "percentage": 23.64, "elapsed_time": "0:08:24", "remaining_time": "0:27:10"}
+{"current_steps": 210, "total_steps": 846, "loss": 0.12280937433242797, "lr": 0.00018718143733052278, "epoch": 0.7466666666666667, "percentage": 24.82, "elapsed_time": "0:08:52", "remaining_time": "0:26:53"}
+{"current_steps": 220, "total_steps": 846, "loss": 0.12583138942718505, "lr": 0.0001850855697002753, "epoch": 0.7822222222222223, "percentage": 26.0, "elapsed_time": "0:09:17", "remaining_time": "0:26:26"}
+{"current_steps": 230, "total_steps": 846, "loss": 0.13474913835525512, "lr": 0.00018284471653898994, "epoch": 0.8177777777777778, "percentage": 27.19, "elapsed_time": "0:09:42", "remaining_time": "0:25:59"}
+{"current_steps": 240, "total_steps": 846, "loss": 0.13484352827072144, "lr": 0.00018046269625308648, "epoch": 0.8533333333333334, "percentage": 28.37, "elapsed_time": "0:10:07", "remaining_time": "0:25:32"}
+{"current_steps": 250, "total_steps": 846, "loss": 0.14849199056625367, "lr": 0.00017794356779730084, "epoch": 0.8888888888888888, "percentage": 29.55, "elapsed_time": "0:10:31", "remaining_time": "0:25:05"}
+{"current_steps": 260, "total_steps": 846, "loss": 0.12720253467559814, "lr": 0.00017529162375823958, "epoch": 0.9244444444444444, "percentage": 30.73, "elapsed_time": "0:10:56", "remaining_time": "0:24:39"}
+{"current_steps": 270, "total_steps": 846, "loss": 0.11229866743087769, "lr": 0.00017251138303982675, "epoch": 0.96, "percentage": 31.91, "elapsed_time": "0:11:20", "remaining_time": "0:24:12"}
+{"current_steps": 280, "total_steps": 846, "loss": 0.12440342903137207, "lr": 0.00016960758316310597, "epoch": 0.9955555555555555, "percentage": 33.1, "elapsed_time": "0:11:45", "remaining_time": "0:23:47"}
+{"current_steps": 290, "total_steps": 846, "loss": 0.0838462769985199, "lr": 0.0001665851721935205, "epoch": 1.0284444444444445, "percentage": 34.28, "elapsed_time": "0:12:10", "remaining_time": "0:23:20"}
+{"current_steps": 300, "total_steps": 846, "loss": 0.05495935678482056, "lr": 0.0001634493003094259, "epoch": 1.064, "percentage": 35.46, "elapsed_time": "0:12:34", "remaining_time": "0:22:52"}
+{"current_steps": 310, "total_steps": 846, "loss": 0.055119764804840085, "lr": 0.00016020531102620304, "epoch": 1.0995555555555556, "percentage": 36.64, "elapsed_time": "0:12:58", "remaining_time": "0:22:25"}
+{"current_steps": 320, "total_steps": 846, "loss": 0.0650447130203247, "lr": 0.0001568587320909255, "epoch": 1.1351111111111112, "percentage": 37.83, "elapsed_time": "0:13:23", "remaining_time": "0:22:01"}
+{"current_steps": 330, "total_steps": 846, "loss": 0.06544734239578247, "lr": 0.00015341526606309645, "epoch": 1.1706666666666667, "percentage": 39.01, "elapsed_time": "0:13:50", "remaining_time": "0:21:38"}
+{"current_steps": 340, "total_steps": 846, "loss": 0.06727538704872131, "lr": 0.00014988078059750652, "epoch": 1.2062222222222223, "percentage": 40.19, "elapsed_time": "0:14:14", "remaining_time": "0:21:12"}
+{"current_steps": 350, "total_steps": 846, "loss": 0.06408223509788513, "lr": 0.00014626129844576893, "epoch": 1.2417777777777779, "percentage": 41.37, "elapsed_time": "0:14:39", "remaining_time": "0:20:46"}
+{"current_steps": 360, "total_steps": 846, "loss": 0.04535002112388611, "lr": 0.00014256298719357062, "epoch": 1.2773333333333334, "percentage": 42.55, "elapsed_time": "0:15:05", "remaining_time": "0:20:22"}
+{"current_steps": 370, "total_steps": 846, "loss": 0.04922315180301666, "lr": 0.00013879214875112665, "epoch": 1.3128888888888888, "percentage": 43.74, "elapsed_time": "0:15:31", "remaining_time": "0:19:57"}
+{"current_steps": 380, "total_steps": 846, "loss": 0.060137057304382326, "lr": 0.00013495520861474565, "epoch": 1.3484444444444446, "percentage": 44.92, "elapsed_time": "0:15:57", "remaining_time": "0:19:33"}
+{"current_steps": 390, "total_steps": 846, "loss": 0.05463656783103943, "lr": 0.00013105870491780558, "epoch": 1.384, "percentage": 46.1, "elapsed_time": "0:16:21", "remaining_time": "0:19:07"}
+{"current_steps": 400, "total_steps": 846, "loss": 0.06046912670135498, "lr": 0.00012710927728979568, "epoch": 1.4195555555555557, "percentage": 47.28, "elapsed_time": "0:16:46", "remaining_time": "0:18:42"}
+{"current_steps": 410, "total_steps": 846, "loss": 0.039173880219459535, "lr": 0.00012311365554240971, "epoch": 1.455111111111111, "percentage": 48.46, "elapsed_time": "0:17:11", "remaining_time": "0:18:16"}
+{"current_steps": 420, "total_steps": 846, "loss": 0.04884783029556274, "lr": 0.0001190786482019691, "epoch": 1.4906666666666666, "percentage": 49.65, "elapsed_time": "0:17:36", "remaining_time": "0:17:51"}
+{"current_steps": 430, "total_steps": 846, "loss": 0.05086652636528015, "lr": 0.00011501113090771619, "epoch": 1.5262222222222221, "percentage": 50.83, "elapsed_time": "0:18:05", "remaining_time": "0:17:29"}
+{"current_steps": 440, "total_steps": 846, "loss": 0.04540249407291412, "lr": 0.00011091803469574789, "epoch": 1.561777777777778, "percentage": 52.01, "elapsed_time": "0:18:30", "remaining_time": "0:17:04"}
+{"current_steps": 450, "total_steps": 846, "loss": 0.0644813358783722, "lr": 0.00010680633418855267, "epoch": 1.5973333333333333, "percentage": 53.19, "elapsed_time": "0:18:52", "remaining_time": "0:16:37"}
+{"current_steps": 460, "total_steps": 846, "loss": 0.07455622553825378, "lr": 0.00010268303571027696, "epoch": 1.6328888888888888, "percentage": 54.37, "elapsed_time": "0:19:18", "remaining_time": "0:16:12"}
+{"current_steps": 470, "total_steps": 846, "loss": 0.04775569438934326, "lr": 9.855516534797187e-05, "epoch": 1.6684444444444444, "percentage": 55.56, "elapsed_time": "0:19:43", "remaining_time": "0:15:47"}
+{"current_steps": 480, "total_steps": 846, "loss": 0.04402676820755005, "lr": 9.442975697916372e-05, "epoch": 1.704, "percentage": 56.74, "elapsed_time": "0:20:10", "remaining_time": "0:15:22"}
+{"current_steps": 490, "total_steps": 846, "loss": 0.05336908102035522, "lr": 9.031384028615004e-05, "epoch": 1.7395555555555555, "percentage": 57.92, "elapsed_time": "0:20:34", "remaining_time": "0:14:57"}
+{"current_steps": 500, "total_steps": 846, "loss": 0.04580667018890381, "lr": 8.621442877744409e-05, "epoch": 1.775111111111111, "percentage": 59.1, "elapsed_time": "0:20:58", "remaining_time": "0:14:31"}
+{"current_steps": 500, "total_steps": 846, "eval_loss": 0.15560266375541687, "epoch": 1.775111111111111, "percentage": 59.1, "elapsed_time": "0:21:25", "remaining_time": "0:14:49"}
+{"current_steps": 510, "total_steps": 846, "loss": 0.03394646048545837, "lr": 8.213850783677925e-05, "epoch": 1.8106666666666666, "percentage": 60.28, "elapsed_time": "0:21:52", "remaining_time": "0:14:24"}
+{"current_steps": 520, "total_steps": 846, "loss": 0.054677408933639524, "lr": 7.809302282003823e-05, "epoch": 1.8462222222222222, "percentage": 61.47, "elapsed_time": "0:22:16", "remaining_time": "0:13:57"}
+{"current_steps": 530, "total_steps": 846, "loss": 0.06665679812431335, "lr": 7.408486722038943e-05, "epoch": 1.8817777777777778, "percentage": 62.65, "elapsed_time": "0:22:41", "remaining_time": "0:13:31"}
+{"current_steps": 540, "total_steps": 846, "loss": 0.048915204405784604, "lr": 7.012087092179724e-05, "epoch": 1.9173333333333333, "percentage": 63.83, "elapsed_time": "0:23:07", "remaining_time": "0:13:06"}
+{"current_steps": 550, "total_steps": 846, "loss": 0.04671376347541809, "lr": 6.620778856092227e-05, "epoch": 1.952888888888889, "percentage": 65.01, "elapsed_time": "0:23:32", "remaining_time": "0:12:40"}
+{"current_steps": 560, "total_steps": 846, "loss": 0.05778223276138306, "lr": 6.235228801724253e-05, "epoch": 1.9884444444444445, "percentage": 66.19, "elapsed_time": "0:23:59", "remaining_time": "0:12:15"}
+{"current_steps": 570, "total_steps": 846, "loss": 0.022845838963985444, "lr": 5.856093905100899e-05, "epoch": 2.021333333333333, "percentage": 67.38, "elapsed_time": "0:24:24", "remaining_time": "0:11:49"}
+{"current_steps": 580, "total_steps": 846, "loss": 0.00663934126496315, "lr": 5.4840202108395466e-05, "epoch": 2.056888888888889, "percentage": 68.56, "elapsed_time": "0:24:49", "remaining_time": "0:11:23"}
+{"current_steps": 590, "total_steps": 846, "loss": 0.007281148433685302, "lr": 5.119641731291971e-05, "epoch": 2.0924444444444443, "percentage": 69.74, "elapsed_time": "0:25:14", "remaining_time": "0:10:57"}
+{"current_steps": 600, "total_steps": 846, "loss": 0.00572865828871727, "lr": 4.7635793661893666e-05, "epoch": 2.128, "percentage": 70.92, "elapsed_time": "0:25:40", "remaining_time": "0:10:31"}
+{"current_steps": 610, "total_steps": 846, "loss": 0.008638855814933778, "lr": 4.416439844631271e-05, "epoch": 2.1635555555555555, "percentage": 72.1, "elapsed_time": "0:26:05", "remaining_time": "0:10:05"}
+{"current_steps": 620, "total_steps": 846, "loss": 0.004736468568444252, "lr": 4.078814691221139e-05, "epoch": 2.1991111111111112, "percentage": 73.29, "elapsed_time": "0:26:30", "remaining_time": "0:09:39"}
+{"current_steps": 630, "total_steps": 846, "loss": 0.007740923762321472, "lr": 3.751279218110387e-05, "epoch": 2.2346666666666666, "percentage": 74.47, "elapsed_time": "0:26:55", "remaining_time": "0:09:14"}
+{"current_steps": 640, "total_steps": 846, "loss": 0.007571302354335785, "lr": 3.434391544668383e-05, "epoch": 2.2702222222222224, "percentage": 75.65, "elapsed_time": "0:27:19", "remaining_time": "0:08:47"}
+{"current_steps": 650, "total_steps": 846, "loss": 0.006724107265472412, "lr": 3.1286916464488505e-05, "epoch": 2.3057777777777777, "percentage": 76.83, "elapsed_time": "0:27:43", "remaining_time": "0:08:21"}
+{"current_steps": 660, "total_steps": 846, "loss": 0.008114227652549743, "lr": 2.8347004350733185e-05, "epoch": 2.3413333333333335, "percentage": 78.01, "elapsed_time": "0:28:12", "remaining_time": "0:07:57"}
+{"current_steps": 670, "total_steps": 846, "loss": 0.0027894463390111925, "lr": 2.55291887059944e-05, "epoch": 2.376888888888889, "percentage": 79.2, "elapsed_time": "0:28:40", "remaining_time": "0:07:31"}
+{"current_steps": 680, "total_steps": 846, "loss": 0.0071901664137840274, "lr": 2.2838271078866714e-05, "epoch": 2.4124444444444446, "percentage": 80.38, "elapsed_time": "0:29:05", "remaining_time": "0:07:06"}
+{"current_steps": 690, "total_steps": 846, "loss": 0.0033892091363668443, "lr": 2.0278836784140044e-05, "epoch": 2.448, "percentage": 81.56, "elapsed_time": "0:29:29", "remaining_time": "0:06:40"}
+{"current_steps": 700, "total_steps": 846, "loss": 0.002477823756635189, "lr": 1.785524708943802e-05, "epoch": 2.4835555555555557, "percentage": 82.74, "elapsed_time": "0:29:55", "remaining_time": "0:06:14"}
+{"current_steps": 710, "total_steps": 846, "loss": 0.0032980531454086305, "lr": 1.557163178363251e-05, "epoch": 2.519111111111111, "percentage": 83.92, "elapsed_time": "0:30:19", "remaining_time": "0:05:48"}
+{"current_steps": 720, "total_steps": 846, "loss": 0.00092921182513237, "lr": 1.3431882139696916e-05, "epoch": 2.554666666666667, "percentage": 85.11, "elapsed_time": "0:30:43", "remaining_time": "0:05:22"}
+{"current_steps": 730, "total_steps": 846, "loss": 0.008682060241699218, "lr": 1.1439644283989747e-05, "epoch": 2.590222222222222, "percentage": 86.29, "elapsed_time": "0:31:10", "remaining_time": "0:04:57"}
+{"current_steps": 740, "total_steps": 846, "loss": 0.0035014577209949494, "lr": 9.59831298326731e-06, "epoch": 2.6257777777777775, "percentage": 87.47, "elapsed_time": "0:31:34", "remaining_time": "0:04:31"}
+{"current_steps": 750, "total_steps": 846, "loss": 0.007874777913093567, "lr": 7.911025860012444e-06, "epoch": 2.6613333333333333, "percentage": 88.65, "elapsed_time": "0:32:00", "remaining_time": "0:04:05"}
+{"current_steps": 760, "total_steps": 846, "loss": 0.003820793330669403, "lr": 6.38065804593595e-06, "epoch": 2.696888888888889, "percentage": 89.83, "elapsed_time": "0:32:24", "remaining_time": "0:03:40"}
+{"current_steps": 770, "total_steps": 846, "loss": 0.0014227939769625663, "lr": 5.009817282761675e-06, "epoch": 2.7324444444444445, "percentage": 91.02, "elapsed_time": "0:32:48", "remaining_time": "0:03:14"}
+{"current_steps": 780, "total_steps": 846, "loss": 0.007305952161550522, "lr": 3.800839478643259e-06, "epoch": 2.768, "percentage": 92.2, "elapsed_time": "0:33:14", "remaining_time": "0:02:48"}
+{"current_steps": 790, "total_steps": 846, "loss": 0.004338302463293075, "lr": 2.7557847277841942e-06, "epoch": 2.8035555555555556, "percentage": 93.38, "elapsed_time": "0:33:39", "remaining_time": "0:02:23"}
+{"current_steps": 800, "total_steps": 846, "loss": 0.002496876008808613, "lr": 1.8764338000442083e-06, "epoch": 2.8391111111111114, "percentage": 94.56, "elapsed_time": "0:34:07", "remaining_time": "0:01:57"}
+{"current_steps": 810, "total_steps": 846, "loss": 0.0015433511696755886, "lr": 1.1642851065131633e-06, "epoch": 2.8746666666666667, "percentage": 95.74, "elapsed_time": "0:34:32", "remaining_time": "0:01:32"}
+{"current_steps": 820, "total_steps": 846, "loss": 0.001970846764743328, "lr": 6.205521462235186e-07, "epoch": 2.910222222222222, "percentage": 96.93, "elapsed_time": "0:34:57", "remaining_time": "0:01:06"}
+{"current_steps": 830, "total_steps": 846, "loss": 0.010426017642021179, "lr": 2.4616143835202166e-07, "epoch": 2.945777777777778, "percentage": 98.11, "elapsed_time": "0:35:21", "remaining_time": "0:00:40"}
+{"current_steps": 840, "total_steps": 846, "loss": 0.0070929393172264096, "lr": 4.1750943434026855e-08, "epoch": 2.981333333333333, "percentage": 99.29, "elapsed_time": "0:35:46", "remaining_time": "0:00:15"}
+{"current_steps": 846, "total_steps": 846, "epoch": 3.0, "percentage": 100.0, "elapsed_time": "0:36:00", "remaining_time": "0:00:00"}

trainer_state.json ADDED Viewed

	@@ -0,0 +1,641 @@

+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 3.0,
+  "eval_steps": 500,
+  "global_step": 846,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.035555555555555556,
+      "grad_norm": 0.7644901871681213,
+      "learning_rate": 2.1176470588235296e-05,
+      "loss": 0.30520846843719485,
+      "step": 10
+    },
+    {
+      "epoch": 0.07111111111111111,
+      "grad_norm": 0.43560856580734253,
+      "learning_rate": 4.470588235294118e-05,
+      "loss": 0.26438264846801757,
+      "step": 20
+    },
+    {
+      "epoch": 0.10666666666666667,
+      "grad_norm": 0.2563800811767578,
+      "learning_rate": 6.823529411764707e-05,
+      "loss": 0.19181081056594848,
+      "step": 30
+    },
+    {
+      "epoch": 0.14222222222222222,
+      "grad_norm": 0.10259224474430084,
+      "learning_rate": 9.176470588235295e-05,
+      "loss": 0.15193926095962523,
+      "step": 40
+    },
+    {
+      "epoch": 0.17777777777777778,
+      "grad_norm": 0.1993834525346756,
+      "learning_rate": 0.00011529411764705881,
+      "loss": 0.13089948892593384,
+      "step": 50
+    },
+    {
+      "epoch": 0.21333333333333335,
+      "grad_norm": 0.19039888679981232,
+      "learning_rate": 0.00013882352941176472,
+      "loss": 0.1438336730003357,
+      "step": 60
+    },
+    {
+      "epoch": 0.24888888888888888,
+      "grad_norm": 0.153082013130188,
+      "learning_rate": 0.0001623529411764706,
+      "loss": 0.13305349349975587,
+      "step": 70
+    },
+    {
+      "epoch": 0.28444444444444444,
+      "grad_norm": 0.08913639187812805,
+      "learning_rate": 0.00018588235294117648,
+      "loss": 0.16499919891357423,
+      "step": 80
+    },
+    {
+      "epoch": 0.32,
+      "grad_norm": 0.18134109675884247,
+      "learning_rate": 0.00019998636639992777,
+      "loss": 0.13644593954086304,
+      "step": 90
+    },
+    {
+      "epoch": 0.35555555555555557,
+      "grad_norm": 0.09949612617492676,
+      "learning_rate": 0.00019983303108908946,
+      "loss": 0.13039498329162597,
+      "step": 100
+    },
+    {
+      "epoch": 0.39111111111111113,
+      "grad_norm": 0.12579314410686493,
+      "learning_rate": 0.00019950958062149127,
+      "loss": 0.1297929048538208,
+      "step": 110
+    },
+    {
+      "epoch": 0.4266666666666667,
+      "grad_norm": 0.1197328120470047,
+      "learning_rate": 0.00019901656615566656,
+      "loss": 0.1509210705757141,
+      "step": 120
+    },
+    {
+      "epoch": 0.4622222222222222,
+      "grad_norm": 0.15709449350833893,
+      "learning_rate": 0.00019835482778664425,
+      "loss": 0.14251822233200073,
+      "step": 130
+    },
+    {
+      "epoch": 0.49777777777777776,
+      "grad_norm": 0.2203192412853241,
+      "learning_rate": 0.0001975254931144296,
+      "loss": 0.1412465214729309,
+      "step": 140
+    },
+    {
+      "epoch": 0.5333333333333333,
+      "grad_norm": 0.32334810495376587,
+      "learning_rate": 0.0001965299753225775,
+      "loss": 0.1519417643547058,
+      "step": 150
+    },
+    {
+      "epoch": 0.5688888888888889,
+      "grad_norm": 0.1700880080461502,
+      "learning_rate": 0.00019536997077013236,
+      "loss": 0.13457289934158326,
+      "step": 160
+    },
+    {
+      "epoch": 0.6044444444444445,
+      "grad_norm": 0.17782148718833923,
+      "learning_rate": 0.00019404745610103786,
+      "loss": 0.14420714378356933,
+      "step": 170
+    },
+    {
+      "epoch": 0.64,
+      "grad_norm": 0.19534672796726227,
+      "learning_rate": 0.00019256468487594214,
+      "loss": 0.1206929087638855,
+      "step": 180
+    },
+    {
+      "epoch": 0.6755555555555556,
+      "grad_norm": 0.12961238622665405,
+      "learning_rate": 0.00019092418373213796,
+      "loss": 0.13416168689727784,
+      "step": 190
+    },
+    {
+      "epoch": 0.7111111111111111,
+      "grad_norm": 0.19645075500011444,
+      "learning_rate": 0.000189128748078181,
+      "loss": 0.13229730129241943,
+      "step": 200
+    },
+    {
+      "epoch": 0.7466666666666667,
+      "grad_norm": 0.17324262857437134,
+      "learning_rate": 0.00018718143733052278,
+      "loss": 0.12280937433242797,
+      "step": 210
+    },
+    {
+      "epoch": 0.7822222222222223,
+      "grad_norm": 0.10764078050851822,
+      "learning_rate": 0.0001850855697002753,
+      "loss": 0.12583138942718505,
+      "step": 220
+    },
+    {
+      "epoch": 0.8177777777777778,
+      "grad_norm": 0.13804763555526733,
+      "learning_rate": 0.00018284471653898994,
+      "loss": 0.13474913835525512,
+      "step": 230
+    },
+    {
+      "epoch": 0.8533333333333334,
+      "grad_norm": 0.11816427856683731,
+      "learning_rate": 0.00018046269625308648,
+      "loss": 0.13484352827072144,
+      "step": 240
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "grad_norm": 0.09677577018737793,
+      "learning_rate": 0.00017794356779730084,
+      "loss": 0.14849199056625367,
+      "step": 250
+    },
+    {
+      "epoch": 0.9244444444444444,
+      "grad_norm": 0.10201010853052139,
+      "learning_rate": 0.00017529162375823958,
+      "loss": 0.12720253467559814,
+      "step": 260
+    },
+    {
+      "epoch": 0.96,
+      "grad_norm": 0.23005172610282898,
+      "learning_rate": 0.00017251138303982675,
+      "loss": 0.11229866743087769,
+      "step": 270
+    },
+    {
+      "epoch": 0.9955555555555555,
+      "grad_norm": 0.15584969520568848,
+      "learning_rate": 0.00016960758316310597,
+      "loss": 0.12440342903137207,
+      "step": 280
+    },
+    {
+      "epoch": 1.0284444444444445,
+      "grad_norm": 0.08643736690282822,
+      "learning_rate": 0.0001665851721935205,
+      "loss": 0.0838462769985199,
+      "step": 290
+    },
+    {
+      "epoch": 1.064,
+      "grad_norm": 0.061173878610134125,
+      "learning_rate": 0.0001634493003094259,
+      "loss": 0.05495935678482056,
+      "step": 300
+    },
+    {
+      "epoch": 1.0995555555555556,
+      "grad_norm": 0.3276287019252777,
+      "learning_rate": 0.00016020531102620304,
+      "loss": 0.055119764804840085,
+      "step": 310
+    },
+    {
+      "epoch": 1.1351111111111112,
+      "grad_norm": 0.1756429374217987,
+      "learning_rate": 0.0001568587320909255,
+      "loss": 0.0650447130203247,
+      "step": 320
+    },
+    {
+      "epoch": 1.1706666666666667,
+      "grad_norm": 0.1355263888835907,
+      "learning_rate": 0.00015341526606309645,
+      "loss": 0.06544734239578247,
+      "step": 330
+    },
+    {
+      "epoch": 1.2062222222222223,
+      "grad_norm": 0.20916008949279785,
+      "learning_rate": 0.00014988078059750652,
+      "loss": 0.06727538704872131,
+      "step": 340
+    },
+    {
+      "epoch": 1.2417777777777779,
+      "grad_norm": 0.19566652178764343,
+      "learning_rate": 0.00014626129844576893,
+      "loss": 0.06408223509788513,
+      "step": 350
+    },
+    {
+      "epoch": 1.2773333333333334,
+      "grad_norm": 0.05987081304192543,
+      "learning_rate": 0.00014256298719357062,
+      "loss": 0.04535002112388611,
+      "step": 360
+    },
+    {
+      "epoch": 1.3128888888888888,
+      "grad_norm": 0.1902439296245575,
+      "learning_rate": 0.00013879214875112665,
+      "loss": 0.04922315180301666,
+      "step": 370
+    },
+    {
+      "epoch": 1.3484444444444446,
+      "grad_norm": 0.25207415223121643,
+      "learning_rate": 0.00013495520861474565,
+      "loss": 0.060137057304382326,
+      "step": 380
+    },
+    {
+      "epoch": 1.384,
+      "grad_norm": 0.34174537658691406,
+      "learning_rate": 0.00013105870491780558,
+      "loss": 0.05463656783103943,
+      "step": 390
+    },
+    {
+      "epoch": 1.4195555555555557,
+      "grad_norm": 0.19624291360378265,
+      "learning_rate": 0.00012710927728979568,
+      "loss": 0.06046912670135498,
+      "step": 400
+    },
+    {
+      "epoch": 1.455111111111111,
+      "grad_norm": 0.22849531471729279,
+      "learning_rate": 0.00012311365554240971,
+      "loss": 0.039173880219459535,
+      "step": 410
+    },
+    {
+      "epoch": 1.4906666666666666,
+      "grad_norm": 0.22941145300865173,
+      "learning_rate": 0.0001190786482019691,
+      "loss": 0.04884783029556274,
+      "step": 420
+    },
+    {
+      "epoch": 1.5262222222222221,
+      "grad_norm": 0.2867370843887329,
+      "learning_rate": 0.00011501113090771619,
+      "loss": 0.05086652636528015,
+      "step": 430
+    },
+    {
+      "epoch": 1.561777777777778,
+      "grad_norm": 0.27643826603889465,
+      "learning_rate": 0.00011091803469574789,
+      "loss": 0.04540249407291412,
+      "step": 440
+    },
+    {
+      "epoch": 1.5973333333333333,
+      "grad_norm": 0.1365540772676468,
+      "learning_rate": 0.00010680633418855267,
+      "loss": 0.0644813358783722,
+      "step": 450
+    },
+    {
+      "epoch": 1.6328888888888888,
+      "grad_norm": 0.21614767611026764,
+      "learning_rate": 0.00010268303571027696,
+      "loss": 0.07455622553825378,
+      "step": 460
+    },
+    {
+      "epoch": 1.6684444444444444,
+      "grad_norm": 0.08498027920722961,
+      "learning_rate": 9.855516534797187e-05,
+      "loss": 0.04775569438934326,
+      "step": 470
+    },
+    {
+      "epoch": 1.704,
+      "grad_norm": 0.2029683142900467,
+      "learning_rate": 9.442975697916372e-05,
+      "loss": 0.04402676820755005,
+      "step": 480
+    },
+    {
+      "epoch": 1.7395555555555555,
+      "grad_norm": 0.34462177753448486,
+      "learning_rate": 9.031384028615004e-05,
+      "loss": 0.05336908102035522,
+      "step": 490
+    },
+    {
+      "epoch": 1.775111111111111,
+      "grad_norm": 0.05993572995066643,
+      "learning_rate": 8.621442877744409e-05,
+      "loss": 0.04580667018890381,
+      "step": 500
+    },
+    {
+      "epoch": 1.775111111111111,
+      "eval_accuracy": 0.9426666666666668,
+      "eval_loss": 0.15560266375541687,
+      "eval_mcq_accuracy": 0.7222222222222222,
+      "eval_runtime": 26.3716,
+      "eval_samples_per_second": 17.064,
+      "eval_steps_per_second": 4.285,
+      "step": 500
+    },
+    {
+      "epoch": 1.8106666666666666,
+      "grad_norm": 0.3891923427581787,
+      "learning_rate": 8.213850783677925e-05,
+      "loss": 0.03394646048545837,
+      "step": 510
+    },
+    {
+      "epoch": 1.8462222222222222,
+      "grad_norm": 0.1975124329328537,
+      "learning_rate": 7.809302282003823e-05,
+      "loss": 0.054677408933639524,
+      "step": 520
+    },
+    {
+      "epoch": 1.8817777777777778,
+      "grad_norm": 0.1962609589099884,
+      "learning_rate": 7.408486722038943e-05,
+      "loss": 0.06665679812431335,
+      "step": 530
+    },
+    {
+      "epoch": 1.9173333333333333,
+      "grad_norm": 0.2969353199005127,
+      "learning_rate": 7.012087092179724e-05,
+      "loss": 0.048915204405784604,
+      "step": 540
+    },
+    {
+      "epoch": 1.952888888888889,
+      "grad_norm": 0.23223088681697845,
+      "learning_rate": 6.620778856092227e-05,
+      "loss": 0.04671376347541809,
+      "step": 550
+    },
+    {
+      "epoch": 1.9884444444444445,
+      "grad_norm": 0.4542539417743683,
+      "learning_rate": 6.235228801724253e-05,
+      "loss": 0.05778223276138306,
+      "step": 560
+    },
+    {
+      "epoch": 2.021333333333333,
+      "grad_norm": 0.09194125235080719,
+      "learning_rate": 5.856093905100899e-05,
+      "loss": 0.022845838963985444,
+      "step": 570
+    },
+    {
+      "epoch": 2.056888888888889,
+      "grad_norm": 0.10670984536409378,
+      "learning_rate": 5.4840202108395466e-05,
+      "loss": 0.00663934126496315,
+      "step": 580
+    },
+    {
+      "epoch": 2.0924444444444443,
+      "grad_norm": 0.015582868829369545,
+      "learning_rate": 5.119641731291971e-05,
+      "loss": 0.007281148433685302,
+      "step": 590
+    },
+    {
+      "epoch": 2.128,
+      "grad_norm": 0.17645655572414398,
+      "learning_rate": 4.7635793661893666e-05,
+      "loss": 0.00572865828871727,
+      "step": 600
+    },
+    {
+      "epoch": 2.1635555555555555,
+      "grad_norm": 0.46430906653404236,
+      "learning_rate": 4.416439844631271e-05,
+      "loss": 0.008638855814933778,
+      "step": 610
+    },
+    {
+      "epoch": 2.1991111111111112,
+      "grad_norm": 0.011537984013557434,
+      "learning_rate": 4.078814691221139e-05,
+      "loss": 0.004736468568444252,
+      "step": 620
+    },
+    {
+      "epoch": 2.2346666666666666,
+      "grad_norm": 0.00896433461457491,
+      "learning_rate": 3.751279218110387e-05,
+      "loss": 0.007740923762321472,
+      "step": 630
+    },
+    {
+      "epoch": 2.2702222222222224,
+      "grad_norm": 0.002848990960046649,
+      "learning_rate": 3.434391544668383e-05,
+      "loss": 0.007571302354335785,
+      "step": 640
+    },
+    {
+      "epoch": 2.3057777777777777,
+      "grad_norm": 0.01570839062333107,
+      "learning_rate": 3.1286916464488505e-05,
+      "loss": 0.006724107265472412,
+      "step": 650
+    },
+    {
+      "epoch": 2.3413333333333335,
+      "grad_norm": 0.003965158946812153,
+      "learning_rate": 2.8347004350733185e-05,
+      "loss": 0.008114227652549743,
+      "step": 660
+    },
+    {
+      "epoch": 2.376888888888889,
+      "grad_norm": 0.10036994516849518,
+      "learning_rate": 2.55291887059944e-05,
+      "loss": 0.0027894463390111925,
+      "step": 670
+    },
+    {
+      "epoch": 2.4124444444444446,
+      "grad_norm": 0.1702636480331421,
+      "learning_rate": 2.2838271078866714e-05,
+      "loss": 0.0071901664137840274,
+      "step": 680
+    },
+    {
+      "epoch": 2.448,
+      "grad_norm": 0.003337263595312834,
+      "learning_rate": 2.0278836784140044e-05,
+      "loss": 0.0033892091363668443,
+      "step": 690
+    },
+    {
+      "epoch": 2.4835555555555557,
+      "grad_norm": 0.0054590729996562,
+      "learning_rate": 1.785524708943802e-05,
+      "loss": 0.002477823756635189,
+      "step": 700
+    },
+    {
+      "epoch": 2.519111111111111,
+      "grad_norm": 0.02814805507659912,
+      "learning_rate": 1.557163178363251e-05,
+      "loss": 0.0032980531454086305,
+      "step": 710
+    },
+    {
+      "epoch": 2.554666666666667,
+      "grad_norm": 0.01721755601465702,
+      "learning_rate": 1.3431882139696916e-05,
+      "loss": 0.00092921182513237,
+      "step": 720
+    },
+    {
+      "epoch": 2.590222222222222,
+      "grad_norm": 0.004700134973973036,
+      "learning_rate": 1.1439644283989747e-05,
+      "loss": 0.008682060241699218,
+      "step": 730
+    },
+    {
+      "epoch": 2.6257777777777775,
+      "grad_norm": 0.02318074181675911,
+      "learning_rate": 9.59831298326731e-06,
+      "loss": 0.0035014577209949494,
+      "step": 740
+    },
+    {
+      "epoch": 2.6613333333333333,
+      "grad_norm": 0.10873377323150635,
+      "learning_rate": 7.911025860012444e-06,
+      "loss": 0.007874777913093567,
+      "step": 750
+    },
+    {
+      "epoch": 2.696888888888889,
+      "grad_norm": 0.007307600695639849,
+      "learning_rate": 6.38065804593595e-06,
+      "loss": 0.003820793330669403,
+      "step": 760
+    },
+    {
+      "epoch": 2.7324444444444445,
+      "grad_norm": 0.002649878617376089,
+      "learning_rate": 5.009817282761675e-06,
+      "loss": 0.0014227939769625663,
+      "step": 770
+    },
+    {
+      "epoch": 2.768,
+      "grad_norm": 0.02097943052649498,
+      "learning_rate": 3.800839478643259e-06,
+      "loss": 0.007305952161550522,
+      "step": 780
+    },
+    {
+      "epoch": 2.8035555555555556,
+      "grad_norm": 0.11966679990291595,
+      "learning_rate": 2.7557847277841942e-06,
+      "loss": 0.004338302463293075,
+      "step": 790
+    },
+    {
+      "epoch": 2.8391111111111114,
+      "grad_norm": 0.07167425006628036,
+      "learning_rate": 1.8764338000442083e-06,
+      "loss": 0.002496876008808613,
+      "step": 800
+    },
+    {
+      "epoch": 2.8746666666666667,
+      "grad_norm": 0.010830031707882881,
+      "learning_rate": 1.1642851065131633e-06,
+      "loss": 0.0015433511696755886,
+      "step": 810
+    },
+    {
+      "epoch": 2.910222222222222,
+      "grad_norm": 0.195694237947464,
+      "learning_rate": 6.205521462235186e-07,
+      "loss": 0.001970846764743328,
+      "step": 820
+    },
+    {
+      "epoch": 2.945777777777778,
+      "grad_norm": 0.00489334249868989,
+      "learning_rate": 2.4616143835202166e-07,
+      "loss": 0.010426017642021179,
+      "step": 830
+    },
+    {
+      "epoch": 2.981333333333333,
+      "grad_norm": 0.009853039868175983,
+      "learning_rate": 4.1750943434026855e-08,
+      "loss": 0.0070929393172264096,
+      "step": 840
+    },
+    {
+      "epoch": 3.0,
+      "step": 846,
+      "total_flos": 1.0958550940031386e+17,
+      "train_loss": 0.06952556652285546,
+      "train_runtime": 2162.4382,
+      "train_samples_per_second": 6.243,
+      "train_steps_per_second": 0.391
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 846,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.0958550940031386e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:06e72e4c171697e89ed577da3f438cb2d6210bd6fe361313102cda2f6b7351b8
+size 5649

training_eval_accuracy.png ADDED Viewed

training_eval_loss.png ADDED Viewed

training_loss.png ADDED Viewed