Upload mixture final checkpoint (step 1421)
Browse files- README.md +5 -3
- model.safetensors +1 -1
README.md
CHANGED
|
@@ -33,7 +33,7 @@ the **final-step checkpoint** (end of the single training epoch).
|
|
| 33 |
| seed | 42 |
|
| 34 |
|
| 35 |
Validation loss at the final checkpoint (mean cross-entropy / scored token):
|
| 36 |
-
- `val_mix`: 1.
|
| 37 |
|
| 38 |
## Usage
|
| 39 |
|
|
@@ -51,5 +51,7 @@ out = model.generate(seed, max_new_tokens=150, do_sample=True, temperature=1.0,
|
|
| 51 |
print(tokenizer.decode(out[0][1:], skip_special_tokens=True))
|
| 52 |
```
|
| 53 |
|
| 54 |
-
Tokenization convention used in training: `add_special_tokens=False`,
|
| 55 |
-
|
|
|
|
|
|
|
|
|
| 33 |
| seed | 42 |
|
| 34 |
|
| 35 |
Validation loss at the final checkpoint (mean cross-entropy / scored token):
|
| 36 |
+
- `val_mix`: 1.7983
|
| 37 |
|
| 38 |
## Usage
|
| 39 |
|
|
|
|
| 51 |
print(tokenizer.decode(out[0][1:], skip_special_tokens=True))
|
| 52 |
```
|
| 53 |
|
| 54 |
+
Tokenization convention used in training: `add_special_tokens=False`, every story
|
| 55 |
+
wrapped in EOS (id=1) on both sides — `[EOS, tokens…, EOS]` — truncated to 512
|
| 56 |
+
tokens. The leading EOS conditions the opening token and matches the generation seed
|
| 57 |
+
above; the trailing EOS teaches termination.
|
model.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 21424032
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fd3a828a4d863c4be9c3c82a800cd08acc4dc2c82d5f8561abcf7c4929ea88cd
|
| 3 |
size 21424032
|