Hugging Face's logo

chauhoang
/
4ccc321d-991e-4c63-b481-6e243305b407

PEFT
Safetensors
llama
axolotl
Generated from Trainer
Model card Files
xet
 Community
chauhoang commited on
Commit
a327c77
·
verified ·
1 Parent(s): c8c60e8

End of training

Browse files
Files changed (2) hide show
  1. README.md +11 -4
  2. adapter_model.bin +1 -1
README.md CHANGED
@@ -66,7 +66,7 @@ lora_model_dir: null
66
  lora_r: 8
67
  lora_target_linear: true
68
  lr_scheduler: cosine
69
- max_steps: 1
70
  micro_batch_size: 2
71
  mlflow_experiment_name: /tmp/e1129b87dc298242_train_data.json
72
  model_type: AutoModelForCausalLM
@@ -93,7 +93,7 @@ wandb_name: b9777803-96c3-4d3c-9b73-a5f9e48688f6
93
  wandb_project: Gradients-On-Demand
94
  wandb_run: your_name
95
  wandb_runid: b9777803-96c3-4d3c-9b73-a5f9e48688f6
96
- warmup_steps: 1
97
  weight_decay: 0.0
98
  xformers_attention: null
99
 
@@ -104,6 +104,8 @@ xformers_attention: null
104
  # 4ccc321d-991e-4c63-b481-6e243305b407
105
 
106
  This model is a fine-tuned version of [Maykeye/TinyLLama-v0](https://huggingface.co/Maykeye/TinyLLama-v0) on the None dataset.
 
 
107
 
108
  ## Model description
109
 
@@ -130,14 +132,19 @@ The following hyperparameters were used during training:
130
  - total_train_batch_size: 8
131
  - optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
132
  - lr_scheduler_type: cosine
133
- - lr_scheduler_warmup_steps: 2
134
- - training_steps: 1
135
 
136
  ### Training results
137
 
138
  | Training Loss | Epoch | Step | Validation Loss |
139
  |:-------------:|:------:|:----:|:---------------:|
140
  | No log | 0.0001 | 1 | 6.6634 |
 
 
 
 
 
141
 
142
 
143
  ### Framework versions
 
66
  lora_r: 8
67
  lora_target_linear: true
68
  lr_scheduler: cosine
69
+ max_steps: 50
70
  micro_batch_size: 2
71
  mlflow_experiment_name: /tmp/e1129b87dc298242_train_data.json
72
  model_type: AutoModelForCausalLM
 
93
  wandb_project: Gradients-On-Demand
94
  wandb_run: your_name
95
  wandb_runid: b9777803-96c3-4d3c-9b73-a5f9e48688f6
96
+ warmup_steps: 10
97
  weight_decay: 0.0
98
  xformers_attention: null
99
 
 
104
  # 4ccc321d-991e-4c63-b481-6e243305b407
105
 
106
  This model is a fine-tuned version of [Maykeye/TinyLLama-v0](https://huggingface.co/Maykeye/TinyLLama-v0) on the None dataset.
107
+ It achieves the following results on the evaluation set:
108
+ - Loss: 6.4528
109
 
110
  ## Model description
111
 
 
132
  - total_train_batch_size: 8
133
  - optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
134
  - lr_scheduler_type: cosine
135
+ - lr_scheduler_warmup_steps: 10
136
+ - training_steps: 50
137
 
138
  ### Training results
139
 
140
  | Training Loss | Epoch | Step | Validation Loss |
141
  |:-------------:|:------:|:----:|:---------------:|
142
  | No log | 0.0001 | 1 | 6.6634 |
143
+ | 6.6596 | 0.0007 | 10 | 6.6318 |
144
+ | 6.4754 | 0.0013 | 20 | 6.5412 |
145
+ | 6.3179 | 0.0020 | 30 | 6.4838 |
146
+ | 6.3783 | 0.0026 | 40 | 6.4569 |
147
+ | 6.6516 | 0.0033 | 50 | 6.4528 |
148
 
149
 
150
  ### Framework versions
adapter_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0c345d3accf87f3b88a05dccb4eac54e401b25c5af2d8d4fe361db887d59de62
3
  size 416906
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d993c033d6027e62e8d71d57a9f7c92ae3875b407dbf70e7022cba6721a9f02b
3
  size 416906