---
license: apache-2.0
base_model: EleutherAI/pythia-70m-deduped
tags:
- generated_from_trainer
model-index:
- name: chessdevilai
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# chessdevilai

This model is a fine-tuned version of [EleutherAI/pythia-70m-deduped](https://huggingface.co/EleutherAI/pythia-70m-deduped) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.9229

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- num_epochs: 1

### Training results

| Training Loss | Epoch  | Step | Validation Loss |
|:-------------:|:------:|:----:|:---------------:|
| 1.4125        | 0.0101 | 19   | 1.4113          |
| 1.2711        | 0.0203 | 38   | 1.2734          |
| 1.275         | 0.0304 | 57   | 1.2185          |
| 1.2618        | 0.0405 | 76   | 1.1850          |
| 1.2098        | 0.0507 | 95   | 1.1672          |
| 1.1458        | 0.0608 | 114  | 1.1452          |
| 1.1457        | 0.0709 | 133  | 1.1275          |
| 1.1003        | 0.0811 | 152  | 1.1198          |
| 1.1089        | 0.0912 | 171  | 1.1120          |
| 1.1051        | 0.1013 | 190  | 1.0940          |
| 1.06          | 0.1115 | 209  | 1.0922          |
| 1.0822        | 0.1216 | 228  | 1.0840          |
| 1.0617        | 0.1317 | 247  | 1.0745          |
| 1.0888        | 0.1419 | 266  | 1.0689          |
| 1.0703        | 0.152  | 285  | 1.0629          |
| 1.0849        | 0.1621 | 304  | 1.0559          |
| 1.0098        | 0.1723 | 323  | 1.0513          |
| 1.0548        | 0.1824 | 342  | 1.0480          |
| 1.0344        | 0.1925 | 361  | 1.0453          |
| 1.1181        | 0.2027 | 380  | 1.0379          |
| 1.0027        | 0.2128 | 399  | 1.0370          |
| 0.9994        | 0.2229 | 418  | 1.0329          |
| 1.0089        | 0.2331 | 437  | 1.0315          |
| 1.1145        | 0.2432 | 456  | 1.0243          |
| 1.024         | 0.2533 | 475  | 1.0247          |
| 1.0804        | 0.2635 | 494  | 1.0207          |
| 1.0525        | 0.2736 | 513  | 1.0189          |
| 1.0473        | 0.2837 | 532  | 1.0177          |
| 1.0761        | 0.2939 | 551  | 1.0098          |
| 0.9665        | 0.304  | 570  | 1.0065          |
| 0.9576        | 0.3141 | 589  | 1.0043          |
| 1.0517        | 0.3243 | 608  | 1.0063          |
| 1.0393        | 0.3344 | 627  | 1.0029          |
| 0.9573        | 0.3445 | 646  | 1.0048          |
| 1.0322        | 0.3547 | 665  | 1.0001          |
| 1.0246        | 0.3648 | 684  | 0.9922          |
| 0.9412        | 0.3749 | 703  | 0.9923          |
| 1.0155        | 0.3851 | 722  | 0.9910          |
| 1.0375        | 0.3952 | 741  | 0.9849          |
| 0.9608        | 0.4053 | 760  | 0.9859          |
| 1.0077        | 0.4155 | 779  | 0.9820          |
| 0.9509        | 0.4256 | 798  | 0.9820          |
| 0.9974        | 0.4357 | 817  | 0.9802          |
| 0.953         | 0.4459 | 836  | 0.9789          |
| 0.9422        | 0.456  | 855  | 0.9756          |
| 0.9781        | 0.4661 | 874  | 0.9757          |
| 1.008         | 0.4763 | 893  | 0.9729          |
| 0.9776        | 0.4864 | 912  | 0.9736          |
| 0.996         | 0.4965 | 931  | 0.9707          |
| 0.9971        | 0.5067 | 950  | 0.9676          |
| 0.9908        | 0.5168 | 969  | 0.9668          |
| 0.984         | 0.5269 | 988  | 0.9627          |
| 0.9498        | 0.5371 | 1007 | 0.9627          |
| 0.9978        | 0.5472 | 1026 | 0.9590          |
| 0.9228        | 0.5573 | 1045 | 0.9577          |
| 0.9223        | 0.5675 | 1064 | 0.9579          |
| 0.984         | 0.5776 | 1083 | 0.9564          |
| 0.955         | 0.5877 | 1102 | 0.9527          |
| 0.9582        | 0.5979 | 1121 | 0.9519          |
| 0.9493        | 0.608  | 1140 | 0.9508          |
| 0.9451        | 0.6181 | 1159 | 0.9491          |
| 0.9552        | 0.6283 | 1178 | 0.9486          |
| 0.9433        | 0.6384 | 1197 | 0.9456          |
| 0.967         | 0.6485 | 1216 | 0.9451          |
| 0.9235        | 0.6587 | 1235 | 0.9424          |
| 0.9173        | 0.6688 | 1254 | 0.9433          |
| 0.9439        | 0.6789 | 1273 | 0.9408          |
| 0.97          | 0.6891 | 1292 | 0.9403          |
| 0.9765        | 0.6992 | 1311 | 0.9387          |
| 0.9627        | 0.7093 | 1330 | 0.9378          |
| 0.8907        | 0.7195 | 1349 | 0.9379          |
| 0.9282        | 0.7296 | 1368 | 0.9356          |
| 0.9235        | 0.7397 | 1387 | 0.9348          |
| 0.9221        | 0.7499 | 1406 | 0.9329          |
| 0.9527        | 0.76   | 1425 | 0.9322          |
| 0.9015        | 0.7701 | 1444 | 0.9321          |
| 0.9371        | 0.7803 | 1463 | 0.9309          |
| 0.9379        | 0.7904 | 1482 | 0.9301          |
| 0.9294        | 0.8005 | 1501 | 0.9299          |
| 0.8619        | 0.8107 | 1520 | 0.9289          |
| 0.9352        | 0.8208 | 1539 | 0.9283          |
| 0.9187        | 0.8309 | 1558 | 0.9273          |
| 0.9197        | 0.8411 | 1577 | 0.9274          |
| 0.9627        | 0.8512 | 1596 | 0.9268          |
| 0.9634        | 0.8613 | 1615 | 0.9259          |
| 0.9242        | 0.8715 | 1634 | 0.9253          |
| 0.937         | 0.8816 | 1653 | 0.9249          |
| 0.9317        | 0.8917 | 1672 | 0.9246          |
| 0.9478        | 0.9019 | 1691 | 0.9245          |
| 0.9598        | 0.912  | 1710 | 0.9241          |
| 0.9347        | 0.9221 | 1729 | 0.9239          |
| 0.9048        | 0.9323 | 1748 | 0.9234          |
| 0.8737        | 0.9424 | 1767 | 0.9232          |
| 0.9272        | 0.9525 | 1786 | 0.9231          |
| 0.9176        | 0.9627 | 1805 | 0.9230          |
| 0.946         | 0.9728 | 1824 | 0.9229          |
| 0.9279        | 0.9829 | 1843 | 0.9229          |
| 0.907         | 0.9931 | 1862 | 0.9229          |


### Framework versions

- Transformers 4.41.2
- Pytorch 2.3.0+cu121
- Datasets 2.19.2
- Tokenizers 0.19.1