Hugging Face's logo

liu-nlp
/
aise-llama3-8b-smol-smoltalk-sv

Text Generation
Transformers
Safetensors
llama
Generated from Trainer
sft
trl
conversational
text-generation-inference
Model card Files
xet
 Community
jekunz commited on
Commit
1c0414e
·
verified ·
1 Parent(s): cdb6104

Training in progress, step 50000

Browse files
Files changed (9) hide show
  1. README.md +37 -12
  2. config.json +1 -1
  3. model-00001-of-00004.safetensors +1 -1
  4. model-00002-of-00004.safetensors +1 -1
  5. model-00003-of-00004.safetensors +1 -1
  6. model-00004-of-00004.safetensors +1 -1
  7. model.safetensors.index.json +1 -0
  8. tokenizer_config.json +1 -1
  9. training_args.bin +2 -2
README.md CHANGED
@@ -1,17 +1,18 @@
1
  ---
2
- license: apache-2.0
3
- datasets:
4
- - HuggingFaceTB/smol-smoltalk
5
- language:
6
- - sv
7
- base_model:
8
- - AI-Sweden-Models/Llama-3-8B
9
- pipeline_tag: text-generation
10
  ---
11
- # Model Card for salamandra-7b-smol-smoltalk-sv
 
12
 
13
  This model is a fine-tuned version of [AI-Sweden-Models/Llama-3-8B](https://huggingface.co/AI-Sweden-Models/Llama-3-8B).
14
- It has been trained on a Swedish version of [smol-smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smol-smoltalk) that was translated with [Gemma3-27B](google/gemma-3-27b-it).
15
 
16
  ## Quick start
17
 
@@ -19,15 +20,39 @@ It has been trained on a Swedish version of [smol-smoltalk](https://huggingface.
19
  from transformers import pipeline
20
 
21
  question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
22
- generator = pipeline("text-generation", model="lliu-nlp/aise-llama3-8b-smol-smoltalk-sv", device="cuda")
23
  output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
24
  print(output["generated_text"])
25
  ```
26
 
 
 
 
 
 
 
 
27
  ### Framework versions
28
 
29
  - TRL: 0.21.0
30
  - Transformers: 4.55.1
31
  - Pytorch: 2.8.0
32
  - Datasets: 4.0.0
33
- - Tokenizers: 0.21.4
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ base_model: AI-Sweden-Models/Llama-3-8B
3
+ library_name: transformers
4
+ model_name: aise-llama3-8b-smol-smoltalk-sv
5
+ tags:
6
+ - generated_from_trainer
7
+ - sft
8
+ - trl
9
+ licence: license
10
  ---
11
+
12
+ # Model Card for aise-llama3-8b-smol-smoltalk-sv
13
 
14
  This model is a fine-tuned version of [AI-Sweden-Models/Llama-3-8B](https://huggingface.co/AI-Sweden-Models/Llama-3-8B).
15
+ It has been trained using [TRL](https://github.com/huggingface/trl).
16
 
17
  ## Quick start
18
 
 
20
  from transformers import pipeline
21
 
22
  question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
23
+ generator = pipeline("text-generation", model="liu-nlp/aise-llama3-8b-smol-smoltalk-sv", device="cuda")
24
  output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
25
  print(output["generated_text"])
26
  ```
27
 
28
+ ## Training procedure
29
+
30
+ [<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/jenny-kunz-liu/huggingface/runs/qkyeivpx)
31
+
32
+
33
+ This model was trained with SFT.
34
+
35
  ### Framework versions
36
 
37
  - TRL: 0.21.0
38
  - Transformers: 4.55.1
39
  - Pytorch: 2.8.0
40
  - Datasets: 4.0.0
41
+ - Tokenizers: 0.21.4
42
+
43
+ ## Citations
44
+
45
+
46
+
47
+ Cite TRL as:
48
+
49
+ ```bibtex
50
+ @misc{vonwerra2022trl,
51
+ title = {{TRL: Transformer Reinforcement Learning}},
52
+ author = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
53
+ year = 2020,
54
+ journal = {GitHub repository},
55
+ publisher = {GitHub},
56
+ howpublished = {\url{https://github.com/huggingface/trl}}
57
+ }
58
+ ```
config.json CHANGED
@@ -24,7 +24,7 @@
24
  "rope_theta": 500000.0,
25
  "tie_word_embeddings": false,
26
  "torch_dtype": "bfloat16",
27
- "transformers_version": "4.52.4",
28
  "use_cache": false,
29
  "vocab_size": 128258
30
  }
 
24
  "rope_theta": 500000.0,
25
  "tie_word_embeddings": false,
26
  "torch_dtype": "bfloat16",
27
+ "transformers_version": "4.55.1",
28
  "use_cache": false,
29
  "vocab_size": 128258
30
  }
model-00001-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5b63e70c21dab386f74d04103dce8315cf869e13a94c43731ff53a0f7a2955cf
3
  size 4976715056
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f52afbb2ab8ee238f6bb5de18daf41ba8983858083de437b15bc0006148af90f
3
  size 4976715056
model-00002-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:4ca92112aa1342c74698b9aa223514f08b68a7512a0c0fdeb7e5a27632044814
3
  size 4999802720
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2f2d10194a8c25416ce31b03011ac35c3d8d8836a50af9a80046b524a9b69b54
3
  size 4999802720
model-00003-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:39250b7b32759c08e212ad538c3e27ee062a7666db5f5a8c0156f7bc0b98f14f
3
  size 4915916176
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c4d9b6ee7b95e721e44cb9f557ecdb223cb0438a545f9bb20397b101fe44b199
3
  size 4915916176
model-00004-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:cd2c7bf99177f58a25070833947e529d750b76954e36b5819794576c4ac4132c
3
  size 1168155192
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dbbd6ba3fe01e6633bb32d0a872a0ab7275efaf1d03259d59fb7b5613d08e4cf
3
  size 1168155192
model.safetensors.index.json CHANGED
@@ -1,5 +1,6 @@
1
  {
2
  "metadata": {
 
3
  "total_size": 16060555264
4
  },
5
  "weight_map": {
 
1
  {
2
  "metadata": {
3
+ "total_parameters": 8030277632,
4
  "total_size": 16060555264
5
  },
6
  "weight_map": {
tokenizer_config.json CHANGED
@@ -2079,5 +2079,5 @@
2079
  ],
2080
  "model_max_length": 1000000000000000019884624838656,
2081
  "pad_token": "<|im_end|>",
2082
- "tokenizer_class": "PreTrainedTokenizer"
2083
  }
 
2079
  ],
2080
  "model_max_length": 1000000000000000019884624838656,
2081
  "pad_token": "<|im_end|>",
2082
+ "tokenizer_class": "PreTrainedTokenizerFast"
2083
  }
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e0b73574b0e864d75e0057f45fd13bc7f28d080ee668cd33637ef70ca969d3c9
3
- size 5688
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8ee77f2a6c2d68b6c7b95197d8c1881d7be353d22fd2f22be127a34402c5ba0d
3
+ size 6161