Hugging Face's logo

chauhoang
/
f4ceaa19-6420-4b44-a08d-f9e4b1bc8dc3

PEFT
Safetensors
mistral
axolotl
Generated from Trainer
Model card Files
xet
 Community
chauhoang commited on
Commit
39bef1a
·
verified ·
1 Parent(s): f8ff3a6

End of training

Browse files
Files changed (2) hide show
  1. README.md +4 -11
  2. adapter_model.bin +1 -1
README.md CHANGED
@@ -66,7 +66,7 @@ lora_model_dir: null
66
  lora_r: 8
67
  lora_target_linear: true
68
  lr_scheduler: cosine
69
- max_steps: 50
70
  micro_batch_size: 2
71
  mlflow_experiment_name: /tmp/f4a6d2ad1888e73f_train_data.json
72
  model_type: AutoModelForCausalLM
@@ -91,7 +91,7 @@ wandb_name: f4ceaa19-6420-4b44-a08d-f9e4b1bc8dc3
91
  wandb_project: Gradients-On-Demand
92
  wandb_run: your_name
93
  wandb_runid: f4ceaa19-6420-4b44-a08d-f9e4b1bc8dc3
94
- warmup_steps: 10
95
  weight_decay: 0.0
96
  xformers_attention: null
97
 
@@ -102,8 +102,6 @@ xformers_attention: null
102
  # f4ceaa19-6420-4b44-a08d-f9e4b1bc8dc3
103
 
104
  This model is a fine-tuned version of [NousResearch/Hermes-2-Pro-Mistral-7B](https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B) on the None dataset.
105
- It achieves the following results on the evaluation set:
106
- - Loss: 1.2440
107
 
108
  ## Model description
109
 
@@ -130,19 +128,14 @@ The following hyperparameters were used during training:
130
  - total_train_batch_size: 8
131
  - optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
132
  - lr_scheduler_type: cosine
133
- - lr_scheduler_warmup_steps: 10
134
- - training_steps: 50
135
 
136
  ### Training results
137
 
138
  | Training Loss | Epoch | Step | Validation Loss |
139
  |:-------------:|:------:|:----:|:---------------:|
140
  | No log | 0.0000 | 1 | 3.2407 |
141
- | 8.5399 | 0.0002 | 10 | 1.7466 |
142
- | 6.5063 | 0.0003 | 20 | 1.3497 |
143
- | 5.1197 | 0.0005 | 30 | 1.2679 |
144
- | 5.4107 | 0.0007 | 40 | 1.2511 |
145
- | 4.3659 | 0.0009 | 50 | 1.2440 |
146
 
147
 
148
  ### Framework versions
 
66
  lora_r: 8
67
  lora_target_linear: true
68
  lr_scheduler: cosine
69
+ max_steps: 1
70
  micro_batch_size: 2
71
  mlflow_experiment_name: /tmp/f4a6d2ad1888e73f_train_data.json
72
  model_type: AutoModelForCausalLM
 
91
  wandb_project: Gradients-On-Demand
92
  wandb_run: your_name
93
  wandb_runid: f4ceaa19-6420-4b44-a08d-f9e4b1bc8dc3
94
+ warmup_steps: 1
95
  weight_decay: 0.0
96
  xformers_attention: null
97
 
 
102
  # f4ceaa19-6420-4b44-a08d-f9e4b1bc8dc3
103
 
104
  This model is a fine-tuned version of [NousResearch/Hermes-2-Pro-Mistral-7B](https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B) on the None dataset.
 
 
105
 
106
  ## Model description
107
 
 
128
  - total_train_batch_size: 8
129
  - optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
130
  - lr_scheduler_type: cosine
131
+ - lr_scheduler_warmup_steps: 2
132
+ - training_steps: 1
133
 
134
  ### Training results
135
 
136
  | Training Loss | Epoch | Step | Validation Loss |
137
  |:-------------:|:------:|:----:|:---------------:|
138
  | No log | 0.0000 | 1 | 3.2407 |
 
 
 
 
 
139
 
140
 
141
  ### Framework versions
adapter_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6e26d4d5e84423ea6cc40293763b164554df6e071cf2e53c7762a6bf874c3789
3
  size 84047370
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5847f6bb1af3f6d66fd9afc2e3c61b6e909091fde2d38d729226830e75ebde9f
3
  size 84047370