Hugging Face's logo

laquythang
/
e4c65784-ac5d-438c-9e1a-2ff5b592fad3

PEFT
Safetensors
llama
axolotl
Generated from Trainer
8-bit precision
bitsandbytes
Model card Files
xet
 Community
laquythang commited on
Commit
8644fbe
·
verified ·
1 Parent(s): 40c4e14

End of training

Browse files
Files changed (5) hide show
  1. README.md +4 -4
  2. adapter_config.json +5 -5
  3. adapter_model.bin +1 -1
  4. adapter_model.safetensors +1 -1
  5. training_args.bin +1 -1
README.md CHANGED
@@ -105,7 +105,7 @@ xformers_attention: null
105
 
106
  This model is a fine-tuned version of [Maykeye/TinyLLama-v0](https://huggingface.co/Maykeye/TinyLLama-v0) on the None dataset.
107
  It achieves the following results on the evaluation set:
108
- - Loss: 8.3080
109
 
110
  ## Model description
111
 
@@ -140,9 +140,9 @@ The following hyperparameters were used during training:
140
  | Training Loss | Epoch | Step | Validation Loss |
141
  |:-------------:|:------:|:----:|:---------------:|
142
  | 8.9009 | 0.0004 | 1 | 8.5327 |
143
- | 8.9824 | 0.0013 | 3 | 8.5247 |
144
- | 8.7049 | 0.0026 | 6 | 8.4378 |
145
- | 8.0412 | 0.0038 | 9 | 8.3080 |
146
 
147
 
148
  ### Framework versions
 
105
 
106
  This model is a fine-tuned version of [Maykeye/TinyLLama-v0](https://huggingface.co/Maykeye/TinyLLama-v0) on the None dataset.
107
  It achieves the following results on the evaluation set:
108
+ - Loss: 8.3100
109
 
110
  ## Model description
111
 
 
140
  | Training Loss | Epoch | Step | Validation Loss |
141
  |:-------------:|:------:|:----:|:---------------:|
142
  | 8.9009 | 0.0004 | 1 | 8.5327 |
143
+ | 8.9808 | 0.0013 | 3 | 8.5248 |
144
+ | 8.7058 | 0.0026 | 6 | 8.4387 |
145
+ | 8.0435 | 0.0038 | 9 | 8.3100 |
146
 
147
 
148
  ### Framework versions
adapter_config.json CHANGED
@@ -20,13 +20,13 @@
20
  "rank_pattern": {},
21
  "revision": null,
22
  "target_modules": [
23
- "gate_proj",
24
- "down_proj",
25
- "up_proj",
26
- "q_proj",
27
  "o_proj",
28
  "k_proj",
29
- "v_proj"
 
 
 
30
  ],
31
  "task_type": "CAUSAL_LM",
32
  "use_dora": false,
 
20
  "rank_pattern": {},
21
  "revision": null,
22
  "target_modules": [
23
+ "v_proj",
 
 
 
24
  "o_proj",
25
  "k_proj",
26
+ "up_proj",
27
+ "down_proj",
28
+ "gate_proj",
29
+ "q_proj"
30
  ],
31
  "task_type": "CAUSAL_LM",
32
  "use_dora": false,
adapter_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e71419d945419a56c7d1cb48ba59594336ef9619bed1be96dcb7083d5821402c
3
  size 793738
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:48b915b3731e3b225fc4000896de6aa63bf6aba7d6baa5f2021fac2ae7c5aea3
3
  size 793738
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e3bc9666e2801002e670241ff56b3338f0eef52f89fc962c656969bde6a48623
3
  size 767856
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4ec1241a27264a4a01bbb34044633f460a05839fdbbc3803c3ff89a4abcf4f49
3
  size 767856
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8ff47a96f9978902bfad1f0a052b77a361dde0b114eea63e5778e39b40eb03f3
3
  size 6776
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ca57a7f19ce3bff51206cc835b0ca5be2e6264eefdd0b0a672534122784ff74e
3
  size 6776