desh2806 commited on
Commit
50f1836
·
verified ·
1 Parent(s): 2b2f0da

Upload mixture final checkpoint (step 1421)

Browse files
Files changed (2) hide show
  1. README.md +5 -3
  2. model.safetensors +1 -1
README.md CHANGED
@@ -33,7 +33,7 @@ the **final-step checkpoint** (end of the single training epoch).
33
  | seed | 42 |
34
 
35
  Validation loss at the final checkpoint (mean cross-entropy / scored token):
36
- - `val_mix`: 1.7961
37
 
38
  ## Usage
39
 
@@ -51,5 +51,7 @@ out = model.generate(seed, max_new_tokens=150, do_sample=True, temperature=1.0,
51
  print(tokenizer.decode(out[0][1:], skip_special_tokens=True))
52
  ```
53
 
54
- Tokenization convention used in training: `add_special_tokens=False`, EOS (id=1)
55
- appended to every story, truncated to 512 tokens.
 
 
 
33
  | seed | 42 |
34
 
35
  Validation loss at the final checkpoint (mean cross-entropy / scored token):
36
+ - `val_mix`: 1.7983
37
 
38
  ## Usage
39
 
 
51
  print(tokenizer.decode(out[0][1:], skip_special_tokens=True))
52
  ```
53
 
54
+ Tokenization convention used in training: `add_special_tokens=False`, every story
55
+ wrapped in EOS (id=1) on both sides — `[EOS, tokens…, EOS]` — truncated to 512
56
+ tokens. The leading EOS conditions the opening token and matches the generation seed
57
+ above; the trailing EOS teaches termination.
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8706b53e9dabb4ae241effa2fb212794a6a7c1c2af41058a9e469426df4b07ca
3
  size 21424032
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fd3a828a4d863c4be9c3c82a800cd08acc4dc2c82d5f8561abcf7c4929ea88cd
3
  size 21424032