Upload folder using huggingface_hub

README.md

@@ -0,0 +1,202 @@
+---
+base_model: microsoft/mdeberta-v3-base
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2

adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "microsoft/mdeberta-v3-base",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "classifier",
+    "score"
+  ],
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "query_proj",
+    "value_proj"
+  ],
+  "task_type": "SEQ_CLS",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:29f22d62d2012ec31be0e1ca8ddece456420260ec54c4ad0aa0cd93a7da714cd
+size 1189964

added_tokens.json

@@ -0,0 +1,3 @@
+{
+  "[MASK]": 250101
+}

checkpoint-5128/README.md

@@ -0,0 +1,202 @@
+---
+base_model: microsoft/mdeberta-v3-base
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2

checkpoint-5128/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "microsoft/mdeberta-v3-base",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "classifier",
+    "score"
+  ],
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "query_proj",
+    "value_proj"
+  ],
+  "task_type": "SEQ_CLS",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-5128/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:106e108f431c8d12ce780ffb65330a7d748b56392a6112a2dbdfa07958072db8
+size 1189964

checkpoint-5128/added_tokens.json

@@ -0,0 +1,3 @@
+{
+  "[MASK]": 250101
+}

checkpoint-5128/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d22ed2ca6fb996d285b5054f4daa0d5518e3ac38b34dba50884643de09fced8d
+size 2407802

checkpoint-5128/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e85a270f419874122744c61d3c2defe237b2f785e30063fd86401827e89633c7
+size 14244

checkpoint-5128/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bfa8c33f8fab57aa2566adb9727bffb2bd6c2fba2ec85f7ff8895e36c2ec54a1
+size 1064

checkpoint-5128/special_tokens_map.json

@@ -0,0 +1,15 @@
+{
+  "bos_token": "[CLS]",
+  "cls_token": "[CLS]",
+  "eos_token": "[SEP]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-5128/spm.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:13c8d666d62a7bc4ac8f040aab68e942c861f93303156cc28f5c7e885d86d6e3
+size 4305025

checkpoint-5128/tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "250101": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "[CLS]",
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "[CLS]",
+  "do_lower_case": false,
+  "eos_token": "[SEP]",
+  "mask_token": "[MASK]",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "sp_model_kwargs": {},
+  "split_by_punct": false,
+  "tokenizer_class": "DebertaV2Tokenizer",
+  "unk_token": "[UNK]",
+  "vocab_type": "spm"
+}

checkpoint-5128/trainer_state.json

@@ -0,0 +1,765 @@
+{
+  "best_metric": 1.3862944841384888,
+  "best_model_checkpoint": "./mdeberta-mcqa-lora\\checkpoint-5128",
+  "epoch": 2.0,
+  "eval_steps": 500,
+  "global_step": 5128,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.01950078003120125,
+      "grad_norm": 1.293081283569336,
+      "learning_rate": 0.00019869994799791993,
+      "loss": 1.3873,
+      "step": 50
+    },
+    {
+      "epoch": 0.0390015600624025,
+      "grad_norm": 1.2752628326416016,
+      "learning_rate": 0.00019739989599583984,
+      "loss": 1.3847,
+      "step": 100
+    },
+    {
+      "epoch": 0.05850234009360374,
+      "grad_norm": 1.2131123542785645,
+      "learning_rate": 0.00019609984399375976,
+      "loss": 1.3899,
+      "step": 150
+    },
+    {
+      "epoch": 0.078003120124805,
+      "grad_norm": 1.3712372779846191,
+      "learning_rate": 0.00019479979199167968,
+      "loss": 1.3931,
+      "step": 200
+    },
+    {
+      "epoch": 0.09750390015600624,
+      "grad_norm": 1.2774697542190552,
+      "learning_rate": 0.00019349973998959957,
+      "loss": 1.3901,
+      "step": 250
+    },
+    {
+      "epoch": 0.11700468018720749,
+      "grad_norm": 1.2445868253707886,
+      "learning_rate": 0.00019219968798751951,
+      "loss": 1.3884,
+      "step": 300
+    },
+    {
+      "epoch": 0.13650546021840873,
+      "grad_norm": 1.5778692960739136,
+      "learning_rate": 0.00019089963598543943,
+      "loss": 1.3856,
+      "step": 350
+    },
+    {
+      "epoch": 0.15600624024961,
+      "grad_norm": 1.329308032989502,
+      "learning_rate": 0.00018959958398335935,
+      "loss": 1.3868,
+      "step": 400
+    },
+    {
+      "epoch": 0.17550702028081122,
+      "grad_norm": 1.289580225944519,
+      "learning_rate": 0.00018829953198127927,
+      "loss": 1.3831,
+      "step": 450
+    },
+    {
+      "epoch": 0.19500780031201248,
+      "grad_norm": 1.373437523841858,
+      "learning_rate": 0.00018699947997919918,
+      "loss": 1.3821,
+      "step": 500
+    },
+    {
+      "epoch": 0.21450858034321374,
+      "grad_norm": 1.332190752029419,
+      "learning_rate": 0.0001856994279771191,
+      "loss": 1.3942,
+      "step": 550
+    },
+    {
+      "epoch": 0.23400936037441497,
+      "grad_norm": 1.303031086921692,
+      "learning_rate": 0.000184399375975039,
+      "loss": 1.3914,
+      "step": 600
+    },
+    {
+      "epoch": 0.2535101404056162,
+      "grad_norm": 1.450868010520935,
+      "learning_rate": 0.00018309932397295894,
+      "loss": 1.3958,
+      "step": 650
+    },
+    {
+      "epoch": 0.27301092043681746,
+      "grad_norm": 1.4821816682815552,
+      "learning_rate": 0.00018179927197087885,
+      "loss": 1.39,
+      "step": 700
+    },
+    {
+      "epoch": 0.2925117004680187,
+      "grad_norm": 1.4727306365966797,
+      "learning_rate": 0.00018049921996879874,
+      "loss": 1.3933,
+      "step": 750
+    },
+    {
+      "epoch": 0.31201248049922,
+      "grad_norm": 1.2481704950332642,
+      "learning_rate": 0.00017919916796671866,
+      "loss": 1.3848,
+      "step": 800
+    },
+    {
+      "epoch": 0.33151326053042124,
+      "grad_norm": 1.5019850730895996,
+      "learning_rate": 0.0001778991159646386,
+      "loss": 1.388,
+      "step": 850
+    },
+    {
+      "epoch": 0.35101404056162244,
+      "grad_norm": 1.3102284669876099,
+      "learning_rate": 0.00017659906396255852,
+      "loss": 1.3879,
+      "step": 900
+    },
+    {
+      "epoch": 0.3705148205928237,
+      "grad_norm": 1.462587594985962,
+      "learning_rate": 0.00017529901196047841,
+      "loss": 1.3816,
+      "step": 950
+    },
+    {
+      "epoch": 0.39001560062402496,
+      "grad_norm": 1.5721924304962158,
+      "learning_rate": 0.00017399895995839833,
+      "loss": 1.3829,
+      "step": 1000
+    },
+    {
+      "epoch": 0.4095163806552262,
+      "grad_norm": 1.3377692699432373,
+      "learning_rate": 0.00017269890795631828,
+      "loss": 1.3847,
+      "step": 1050
+    },
+    {
+      "epoch": 0.4290171606864275,
+      "grad_norm": 1.7942304611206055,
+      "learning_rate": 0.00017139885595423817,
+      "loss": 1.3894,
+      "step": 1100
+    },
+    {
+      "epoch": 0.4485179407176287,
+      "grad_norm": 1.4108786582946777,
+      "learning_rate": 0.00017009880395215808,
+      "loss": 1.3839,
+      "step": 1150
+    },
+    {
+      "epoch": 0.46801872074882994,
+      "grad_norm": 1.334272027015686,
+      "learning_rate": 0.00016879875195007803,
+      "loss": 1.3892,
+      "step": 1200
+    },
+    {
+      "epoch": 0.4875195007800312,
+      "grad_norm": 1.4627348184585571,
+      "learning_rate": 0.00016749869994799792,
+      "loss": 1.3901,
+      "step": 1250
+    },
+    {
+      "epoch": 0.5070202808112324,
+      "grad_norm": 1.3052252531051636,
+      "learning_rate": 0.00016619864794591784,
+      "loss": 1.3832,
+      "step": 1300
+    },
+    {
+      "epoch": 0.5265210608424337,
+      "grad_norm": 1.37651526927948,
+      "learning_rate": 0.00016489859594383775,
+      "loss": 1.3815,
+      "step": 1350
+    },
+    {
+      "epoch": 0.5460218408736349,
+      "grad_norm": 1.187026858329773,
+      "learning_rate": 0.0001635985439417577,
+      "loss": 1.3827,
+      "step": 1400
+    },
+    {
+      "epoch": 0.5655226209048362,
+      "grad_norm": 1.6805840730667114,
+      "learning_rate": 0.0001622984919396776,
+      "loss": 1.3895,
+      "step": 1450
+    },
+    {
+      "epoch": 0.5850234009360374,
+      "grad_norm": 1.302393913269043,
+      "learning_rate": 0.0001609984399375975,
+      "loss": 1.3942,
+      "step": 1500
+    },
+    {
+      "epoch": 0.6045241809672387,
+      "grad_norm": 1.3884400129318237,
+      "learning_rate": 0.00015969838793551742,
+      "loss": 1.3951,
+      "step": 1550
+    },
+    {
+      "epoch": 0.62402496099844,
+      "grad_norm": 1.5166919231414795,
+      "learning_rate": 0.00015839833593343734,
+      "loss": 1.3961,
+      "step": 1600
+    },
+    {
+      "epoch": 0.6435257410296412,
+      "grad_norm": 1.1475179195404053,
+      "learning_rate": 0.00015709828393135726,
+      "loss": 1.3851,
+      "step": 1650
+    },
+    {
+      "epoch": 0.6630265210608425,
+      "grad_norm": 1.2139875888824463,
+      "learning_rate": 0.00015579823192927718,
+      "loss": 1.3895,
+      "step": 1700
+    },
+    {
+      "epoch": 0.6825273010920437,
+      "grad_norm": 1.2412734031677246,
+      "learning_rate": 0.0001544981799271971,
+      "loss": 1.3853,
+      "step": 1750
+    },
+    {
+      "epoch": 0.7020280811232449,
+      "grad_norm": 1.587342619895935,
+      "learning_rate": 0.000153198127925117,
+      "loss": 1.391,
+      "step": 1800
+    },
+    {
+      "epoch": 0.7215288611544461,
+      "grad_norm": 1.2071484327316284,
+      "learning_rate": 0.00015189807592303693,
+      "loss": 1.3917,
+      "step": 1850
+    },
+    {
+      "epoch": 0.7410296411856474,
+      "grad_norm": 1.3918330669403076,
+      "learning_rate": 0.00015059802392095685,
+      "loss": 1.3912,
+      "step": 1900
+    },
+    {
+      "epoch": 0.7605304212168487,
+      "grad_norm": 1.3605890274047852,
+      "learning_rate": 0.00014929797191887676,
+      "loss": 1.387,
+      "step": 1950
+    },
+    {
+      "epoch": 0.7800312012480499,
+      "grad_norm": 1.460693597793579,
+      "learning_rate": 0.00014799791991679668,
+      "loss": 1.3871,
+      "step": 2000
+    },
+    {
+      "epoch": 0.7995319812792512,
+      "grad_norm": 1.3369407653808594,
+      "learning_rate": 0.00014672386895475822,
+      "loss": 1.3895,
+      "step": 2050
+    },
+    {
+      "epoch": 0.8190327613104524,
+      "grad_norm": 1.328797459602356,
+      "learning_rate": 0.0001454238169526781,
+      "loss": 1.3825,
+      "step": 2100
+    },
+    {
+      "epoch": 0.8385335413416537,
+      "grad_norm": 1.2546560764312744,
+      "learning_rate": 0.00014412376495059802,
+      "loss": 1.3891,
+      "step": 2150
+    },
+    {
+      "epoch": 0.858034321372855,
+      "grad_norm": 1.578344702720642,
+      "learning_rate": 0.00014282371294851794,
+      "loss": 1.3848,
+      "step": 2200
+    },
+    {
+      "epoch": 0.8775351014040562,
+      "grad_norm": 1.6272181272506714,
+      "learning_rate": 0.0001415236609464379,
+      "loss": 1.3902,
+      "step": 2250
+    },
+    {
+      "epoch": 0.8970358814352574,
+      "grad_norm": 1.1834017038345337,
+      "learning_rate": 0.00014022360894435778,
+      "loss": 1.3948,
+      "step": 2300
+    },
+    {
+      "epoch": 0.9165366614664586,
+      "grad_norm": 1.5742530822753906,
+      "learning_rate": 0.0001389235569422777,
+      "loss": 1.3907,
+      "step": 2350
+    },
+    {
+      "epoch": 0.9360374414976599,
+      "grad_norm": 1.8273956775665283,
+      "learning_rate": 0.0001376235049401976,
+      "loss": 1.3859,
+      "step": 2400
+    },
+    {
+      "epoch": 0.9555382215288611,
+      "grad_norm": 1.108070969581604,
+      "learning_rate": 0.00013632345293811753,
+      "loss": 1.3861,
+      "step": 2450
+    },
+    {
+      "epoch": 0.9750390015600624,
+      "grad_norm": 1.1575729846954346,
+      "learning_rate": 0.00013502340093603745,
+      "loss": 1.3874,
+      "step": 2500
+    },
+    {
+      "epoch": 0.9945397815912637,
+      "grad_norm": 1.2663004398345947,
+      "learning_rate": 0.00013372334893395736,
+      "loss": 1.3919,
+      "step": 2550
+    },
+    {
+      "epoch": 1.0,
+      "eval_accuracy": 0.26053042121684866,
+      "eval_loss": 1.386296033859253,
+      "eval_runtime": 142.8097,
+      "eval_samples_per_second": 8.977,
+      "eval_steps_per_second": 2.248,
+      "step": 2564
+    },
+    {
+      "epoch": 1.0140405616224648,
+      "grad_norm": 1.3720601797103882,
+      "learning_rate": 0.00013242329693187728,
+      "loss": 1.3917,
+      "step": 2600
+    },
+    {
+      "epoch": 1.033541341653666,
+      "grad_norm": 1.375525951385498,
+      "learning_rate": 0.0001311232449297972,
+      "loss": 1.391,
+      "step": 2650
+    },
+    {
+      "epoch": 1.0530421216848673,
+      "grad_norm": 1.3909684419631958,
+      "learning_rate": 0.00012982319292771712,
+      "loss": 1.3894,
+      "step": 2700
+    },
+    {
+      "epoch": 1.0725429017160686,
+      "grad_norm": 1.4531859159469604,
+      "learning_rate": 0.00012852314092563703,
+      "loss": 1.3853,
+      "step": 2750
+    },
+    {
+      "epoch": 1.0920436817472698,
+      "grad_norm": 1.406068205833435,
+      "learning_rate": 0.00012722308892355695,
+      "loss": 1.382,
+      "step": 2800
+    },
+    {
+      "epoch": 1.111544461778471,
+      "grad_norm": 1.7967036962509155,
+      "learning_rate": 0.00012592303692147687,
+      "loss": 1.3943,
+      "step": 2850
+    },
+    {
+      "epoch": 1.1310452418096724,
+      "grad_norm": 1.228324294090271,
+      "learning_rate": 0.0001246229849193968,
+      "loss": 1.3831,
+      "step": 2900
+    },
+    {
+      "epoch": 1.1505460218408736,
+      "grad_norm": 1.0168524980545044,
+      "learning_rate": 0.0001233229329173167,
+      "loss": 1.3875,
+      "step": 2950
+    },
+    {
+      "epoch": 1.1700468018720749,
+      "grad_norm": 1.3154783248901367,
+      "learning_rate": 0.00012202288091523661,
+      "loss": 1.3853,
+      "step": 3000
+    },
+    {
+      "epoch": 1.1895475819032761,
+      "grad_norm": 1.4282019138336182,
+      "learning_rate": 0.00012072282891315654,
+      "loss": 1.3791,
+      "step": 3050
+    },
+    {
+      "epoch": 1.2090483619344774,
+      "grad_norm": 1.3305896520614624,
+      "learning_rate": 0.00011942277691107644,
+      "loss": 1.3806,
+      "step": 3100
+    },
+    {
+      "epoch": 1.2285491419656787,
+      "grad_norm": 1.8123506307601929,
+      "learning_rate": 0.00011812272490899636,
+      "loss": 1.3849,
+      "step": 3150
+    },
+    {
+      "epoch": 1.24804992199688,
+      "grad_norm": 1.4277228116989136,
+      "learning_rate": 0.00011682267290691629,
+      "loss": 1.3847,
+      "step": 3200
+    },
+    {
+      "epoch": 1.2675507020280812,
+      "grad_norm": 1.2390486001968384,
+      "learning_rate": 0.00011552262090483621,
+      "loss": 1.3908,
+      "step": 3250
+    },
+    {
+      "epoch": 1.2870514820592824,
+      "grad_norm": 1.3256953954696655,
+      "learning_rate": 0.00011422256890275611,
+      "loss": 1.3885,
+      "step": 3300
+    },
+    {
+      "epoch": 1.3065522620904837,
+      "grad_norm": 1.5108897686004639,
+      "learning_rate": 0.00011292251690067603,
+      "loss": 1.3841,
+      "step": 3350
+    },
+    {
+      "epoch": 1.3260530421216847,
+      "grad_norm": 1.3400274515151978,
+      "learning_rate": 0.00011162246489859596,
+      "loss": 1.3862,
+      "step": 3400
+    },
+    {
+      "epoch": 1.345553822152886,
+      "grad_norm": 1.4532562494277954,
+      "learning_rate": 0.00011032241289651587,
+      "loss": 1.4003,
+      "step": 3450
+    },
+    {
+      "epoch": 1.3650546021840873,
+      "grad_norm": 1.1774451732635498,
+      "learning_rate": 0.00010902236089443578,
+      "loss": 1.3883,
+      "step": 3500
+    },
+    {
+      "epoch": 1.3845553822152885,
+      "grad_norm": 1.5734331607818604,
+      "learning_rate": 0.0001077223088923557,
+      "loss": 1.3842,
+      "step": 3550
+    },
+    {
+      "epoch": 1.4040561622464898,
+      "grad_norm": 1.347231149673462,
+      "learning_rate": 0.0001064222568902756,
+      "loss": 1.3931,
+      "step": 3600
+    },
+    {
+      "epoch": 1.423556942277691,
+      "grad_norm": 1.2658965587615967,
+      "learning_rate": 0.00010512220488819554,
+      "loss": 1.3889,
+      "step": 3650
+    },
+    {
+      "epoch": 1.4430577223088923,
+      "grad_norm": 1.4009519815444946,
+      "learning_rate": 0.00010382215288611545,
+      "loss": 1.395,
+      "step": 3700
+    },
+    {
+      "epoch": 1.4625585023400935,
+      "grad_norm": 1.3800654411315918,
+      "learning_rate": 0.00010252210088403537,
+      "loss": 1.3909,
+      "step": 3750
+    },
+    {
+      "epoch": 1.4820592823712948,
+      "grad_norm": 1.6213831901550293,
+      "learning_rate": 0.00010122204888195527,
+      "loss": 1.393,
+      "step": 3800
+    },
+    {
+      "epoch": 1.501560062402496,
+      "grad_norm": 1.7539178133010864,
+      "learning_rate": 9.99219968798752e-05,
+      "loss": 1.3896,
+      "step": 3850
+    },
+    {
+      "epoch": 1.5210608424336973,
+      "grad_norm": 1.4965327978134155,
+      "learning_rate": 9.862194487779511e-05,
+      "loss": 1.385,
+      "step": 3900
+    },
+    {
+      "epoch": 1.5405616224648986,
+      "grad_norm": 1.2038394212722778,
+      "learning_rate": 9.732189287571504e-05,
+      "loss": 1.3892,
+      "step": 3950
+    },
+    {
+      "epoch": 1.5600624024960998,
+      "grad_norm": 1.5250635147094727,
+      "learning_rate": 9.602184087363494e-05,
+      "loss": 1.3817,
+      "step": 4000
+    },
+    {
+      "epoch": 1.579563182527301,
+      "grad_norm": 1.1984943151474,
+      "learning_rate": 9.472178887155486e-05,
+      "loss": 1.3865,
+      "step": 4050
+    },
+    {
+      "epoch": 1.5990639625585024,
+      "grad_norm": 1.605357050895691,
+      "learning_rate": 9.344773790951638e-05,
+      "loss": 1.3855,
+      "step": 4100
+    },
+    {
+      "epoch": 1.6185647425897036,
+      "grad_norm": 1.4252681732177734,
+      "learning_rate": 9.21476859074363e-05,
+      "loss": 1.3841,
+      "step": 4150
+    },
+    {
+      "epoch": 1.6380655226209049,
+      "grad_norm": 1.3211374282836914,
+      "learning_rate": 9.084763390535622e-05,
+      "loss": 1.3782,
+      "step": 4200
+    },
+    {
+      "epoch": 1.6575663026521061,
+      "grad_norm": 1.4356790781021118,
+      "learning_rate": 8.954758190327614e-05,
+      "loss": 1.3862,
+      "step": 4250
+    },
+    {
+      "epoch": 1.6770670826833074,
+      "grad_norm": 1.3711603879928589,
+      "learning_rate": 8.824752990119605e-05,
+      "loss": 1.3906,
+      "step": 4300
+    },
+    {
+      "epoch": 1.6965678627145087,
+      "grad_norm": 1.481302261352539,
+      "learning_rate": 8.694747789911597e-05,
+      "loss": 1.3863,
+      "step": 4350
+    },
+    {
+      "epoch": 1.71606864274571,
+      "grad_norm": 1.3888981342315674,
+      "learning_rate": 8.564742589703589e-05,
+      "loss": 1.3818,
+      "step": 4400
+    },
+    {
+      "epoch": 1.7355694227769112,
+      "grad_norm": 1.4192887544631958,
+      "learning_rate": 8.43473738949558e-05,
+      "loss": 1.3835,
+      "step": 4450
+    },
+    {
+      "epoch": 1.7550702028081124,
+      "grad_norm": 1.0555356740951538,
+      "learning_rate": 8.304732189287572e-05,
+      "loss": 1.3967,
+      "step": 4500
+    },
+    {
+      "epoch": 1.7745709828393137,
+      "grad_norm": 1.3277314901351929,
+      "learning_rate": 8.174726989079563e-05,
+      "loss": 1.3863,
+      "step": 4550
+    },
+    {
+      "epoch": 1.794071762870515,
+      "grad_norm": 1.600809097290039,
+      "learning_rate": 8.044721788871556e-05,
+      "loss": 1.3833,
+      "step": 4600
+    },
+    {
+      "epoch": 1.8135725429017162,
+      "grad_norm": 1.5007127523422241,
+      "learning_rate": 7.914716588663546e-05,
+      "loss": 1.3892,
+      "step": 4650
+    },
+    {
+      "epoch": 1.8330733229329175,
+      "grad_norm": 1.4107766151428223,
+      "learning_rate": 7.784711388455539e-05,
+      "loss": 1.3925,
+      "step": 4700
+    },
+    {
+      "epoch": 1.8525741029641187,
+      "grad_norm": 1.4171024560928345,
+      "learning_rate": 7.65470618824753e-05,
+      "loss": 1.3899,
+      "step": 4750
+    },
+    {
+      "epoch": 1.8720748829953198,
+      "grad_norm": 1.453108787536621,
+      "learning_rate": 7.524700988039521e-05,
+      "loss": 1.3866,
+      "step": 4800
+    },
+    {
+      "epoch": 1.891575663026521,
+      "grad_norm": 1.537400722503662,
+      "learning_rate": 7.394695787831513e-05,
+      "loss": 1.3886,
+      "step": 4850
+    },
+    {
+      "epoch": 1.9110764430577223,
+      "grad_norm": 1.4526410102844238,
+      "learning_rate": 7.264690587623505e-05,
+      "loss": 1.3791,
+      "step": 4900
+    },
+    {
+      "epoch": 1.9305772230889235,
+      "grad_norm": 1.2780728340148926,
+      "learning_rate": 7.134685387415497e-05,
+      "loss": 1.3832,
+      "step": 4950
+    },
+    {
+      "epoch": 1.9500780031201248,
+      "grad_norm": 1.2071354389190674,
+      "learning_rate": 7.004680187207488e-05,
+      "loss": 1.3917,
+      "step": 5000
+    },
+    {
+      "epoch": 1.969578783151326,
+      "grad_norm": 1.2375810146331787,
+      "learning_rate": 6.87467498699948e-05,
+      "loss": 1.3917,
+      "step": 5050
+    },
+    {
+      "epoch": 1.9890795631825273,
+      "grad_norm": 1.3808921575546265,
+      "learning_rate": 6.744669786791472e-05,
+      "loss": 1.391,
+      "step": 5100
+    },
+    {
+      "epoch": 2.0,
+      "eval_accuracy": 0.24960998439937598,
+      "eval_loss": 1.3862944841384888,
+      "eval_runtime": 142.4422,
+      "eval_samples_per_second": 9.0,
+      "eval_steps_per_second": 2.254,
+      "step": 5128
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 7692,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.166034384625664e+16,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-5128/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:60592bff6cbc053d3a93f6d39ee62300fafc1a17252704e129e528cb1b434102
+size 5304

checkpoint-7692/README.md

@@ -0,0 +1,202 @@
+---
+base_model: microsoft/mdeberta-v3-base
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2

checkpoint-7692/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "microsoft/mdeberta-v3-base",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "classifier",
+    "score"
+  ],
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "query_proj",
+    "value_proj"
+  ],
+  "task_type": "SEQ_CLS",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-7692/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:29f22d62d2012ec31be0e1ca8ddece456420260ec54c4ad0aa0cd93a7da714cd
+size 1189964

checkpoint-7692/added_tokens.json

@@ -0,0 +1,3 @@
+{
+  "[MASK]": 250101
+}

checkpoint-7692/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:05e1e100baad558cee11734667fdf308f08c71c86162eceb6264715b3ac740a7
+size 2407802

checkpoint-7692/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da8a9225696814ebd96c0b35b8004fe8de0358bd224dd5d2f426b1b6bd06dd07
+size 14244

checkpoint-7692/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d0767faac71e2e7e1bfc360c83d9ed78d74c8c8b707490b7514ab91c1f5c23f
+size 1064

checkpoint-7692/special_tokens_map.json

@@ -0,0 +1,15 @@
+{
+  "bos_token": "[CLS]",
+  "cls_token": "[CLS]",
+  "eos_token": "[SEP]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-7692/spm.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:13c8d666d62a7bc4ac8f040aab68e942c861f93303156cc28f5c7e885d86d6e3
+size 4305025

checkpoint-7692/tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "250101": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "[CLS]",
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "[CLS]",
+  "do_lower_case": false,
+  "eos_token": "[SEP]",
+  "mask_token": "[MASK]",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "sp_model_kwargs": {},
+  "split_by_punct": false,
+  "tokenizer_class": "DebertaV2Tokenizer",
+  "unk_token": "[UNK]",
+  "vocab_type": "spm"
+}

checkpoint-7692/trainer_state.json

@@ -0,0 +1,1131 @@
+{
+  "best_metric": 1.3862601518630981,
+  "best_model_checkpoint": "./mdeberta-mcqa-lora\\checkpoint-7692",
+  "epoch": 3.0,
+  "eval_steps": 500,
+  "global_step": 7692,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.01950078003120125,
+      "grad_norm": 1.293081283569336,
+      "learning_rate": 0.00019869994799791993,
+      "loss": 1.3873,
+      "step": 50
+    },
+    {
+      "epoch": 0.0390015600624025,
+      "grad_norm": 1.2752628326416016,
+      "learning_rate": 0.00019739989599583984,
+      "loss": 1.3847,
+      "step": 100
+    },
+    {
+      "epoch": 0.05850234009360374,
+      "grad_norm": 1.2131123542785645,
+      "learning_rate": 0.00019609984399375976,
+      "loss": 1.3899,
+      "step": 150
+    },
+    {
+      "epoch": 0.078003120124805,
+      "grad_norm": 1.3712372779846191,
+      "learning_rate": 0.00019479979199167968,
+      "loss": 1.3931,
+      "step": 200
+    },
+    {
+      "epoch": 0.09750390015600624,
+      "grad_norm": 1.2774697542190552,
+      "learning_rate": 0.00019349973998959957,
+      "loss": 1.3901,
+      "step": 250
+    },
+    {
+      "epoch": 0.11700468018720749,
+      "grad_norm": 1.2445868253707886,
+      "learning_rate": 0.00019219968798751951,
+      "loss": 1.3884,
+      "step": 300
+    },
+    {
+      "epoch": 0.13650546021840873,
+      "grad_norm": 1.5778692960739136,
+      "learning_rate": 0.00019089963598543943,
+      "loss": 1.3856,
+      "step": 350
+    },
+    {
+      "epoch": 0.15600624024961,
+      "grad_norm": 1.329308032989502,
+      "learning_rate": 0.00018959958398335935,
+      "loss": 1.3868,
+      "step": 400
+    },
+    {
+      "epoch": 0.17550702028081122,
+      "grad_norm": 1.289580225944519,
+      "learning_rate": 0.00018829953198127927,
+      "loss": 1.3831,
+      "step": 450
+    },
+    {
+      "epoch": 0.19500780031201248,
+      "grad_norm": 1.373437523841858,
+      "learning_rate": 0.00018699947997919918,
+      "loss": 1.3821,
+      "step": 500
+    },
+    {
+      "epoch": 0.21450858034321374,
+      "grad_norm": 1.332190752029419,
+      "learning_rate": 0.0001856994279771191,
+      "loss": 1.3942,
+      "step": 550
+    },
+    {
+      "epoch": 0.23400936037441497,
+      "grad_norm": 1.303031086921692,
+      "learning_rate": 0.000184399375975039,
+      "loss": 1.3914,
+      "step": 600
+    },
+    {
+      "epoch": 0.2535101404056162,
+      "grad_norm": 1.450868010520935,
+      "learning_rate": 0.00018309932397295894,
+      "loss": 1.3958,
+      "step": 650
+    },
+    {
+      "epoch": 0.27301092043681746,
+      "grad_norm": 1.4821816682815552,
+      "learning_rate": 0.00018179927197087885,
+      "loss": 1.39,
+      "step": 700
+    },
+    {
+      "epoch": 0.2925117004680187,
+      "grad_norm": 1.4727306365966797,
+      "learning_rate": 0.00018049921996879874,
+      "loss": 1.3933,
+      "step": 750
+    },
+    {
+      "epoch": 0.31201248049922,
+      "grad_norm": 1.2481704950332642,
+      "learning_rate": 0.00017919916796671866,
+      "loss": 1.3848,
+      "step": 800
+    },
+    {
+      "epoch": 0.33151326053042124,
+      "grad_norm": 1.5019850730895996,
+      "learning_rate": 0.0001778991159646386,
+      "loss": 1.388,
+      "step": 850
+    },
+    {
+      "epoch": 0.35101404056162244,
+      "grad_norm": 1.3102284669876099,
+      "learning_rate": 0.00017659906396255852,
+      "loss": 1.3879,
+      "step": 900
+    },
+    {
+      "epoch": 0.3705148205928237,
+      "grad_norm": 1.462587594985962,
+      "learning_rate": 0.00017529901196047841,
+      "loss": 1.3816,
+      "step": 950
+    },
+    {
+      "epoch": 0.39001560062402496,
+      "grad_norm": 1.5721924304962158,
+      "learning_rate": 0.00017399895995839833,
+      "loss": 1.3829,
+      "step": 1000
+    },
+    {
+      "epoch": 0.4095163806552262,
+      "grad_norm": 1.3377692699432373,
+      "learning_rate": 0.00017269890795631828,
+      "loss": 1.3847,
+      "step": 1050
+    },
+    {
+      "epoch": 0.4290171606864275,
+      "grad_norm": 1.7942304611206055,
+      "learning_rate": 0.00017139885595423817,
+      "loss": 1.3894,
+      "step": 1100
+    },
+    {
+      "epoch": 0.4485179407176287,
+      "grad_norm": 1.4108786582946777,
+      "learning_rate": 0.00017009880395215808,
+      "loss": 1.3839,
+      "step": 1150
+    },
+    {
+      "epoch": 0.46801872074882994,
+      "grad_norm": 1.334272027015686,
+      "learning_rate": 0.00016879875195007803,
+      "loss": 1.3892,
+      "step": 1200
+    },
+    {
+      "epoch": 0.4875195007800312,
+      "grad_norm": 1.4627348184585571,
+      "learning_rate": 0.00016749869994799792,
+      "loss": 1.3901,
+      "step": 1250
+    },
+    {
+      "epoch": 0.5070202808112324,
+      "grad_norm": 1.3052252531051636,
+      "learning_rate": 0.00016619864794591784,
+      "loss": 1.3832,
+      "step": 1300
+    },
+    {
+      "epoch": 0.5265210608424337,
+      "grad_norm": 1.37651526927948,
+      "learning_rate": 0.00016489859594383775,
+      "loss": 1.3815,
+      "step": 1350
+    },
+    {
+      "epoch": 0.5460218408736349,
+      "grad_norm": 1.187026858329773,
+      "learning_rate": 0.0001635985439417577,
+      "loss": 1.3827,
+      "step": 1400
+    },
+    {
+      "epoch": 0.5655226209048362,
+      "grad_norm": 1.6805840730667114,
+      "learning_rate": 0.0001622984919396776,
+      "loss": 1.3895,
+      "step": 1450
+    },
+    {
+      "epoch": 0.5850234009360374,
+      "grad_norm": 1.302393913269043,
+      "learning_rate": 0.0001609984399375975,
+      "loss": 1.3942,
+      "step": 1500
+    },
+    {
+      "epoch": 0.6045241809672387,
+      "grad_norm": 1.3884400129318237,
+      "learning_rate": 0.00015969838793551742,
+      "loss": 1.3951,
+      "step": 1550
+    },
+    {
+      "epoch": 0.62402496099844,
+      "grad_norm": 1.5166919231414795,
+      "learning_rate": 0.00015839833593343734,
+      "loss": 1.3961,
+      "step": 1600
+    },
+    {
+      "epoch": 0.6435257410296412,
+      "grad_norm": 1.1475179195404053,
+      "learning_rate": 0.00015709828393135726,
+      "loss": 1.3851,
+      "step": 1650
+    },
+    {
+      "epoch": 0.6630265210608425,
+      "grad_norm": 1.2139875888824463,
+      "learning_rate": 0.00015579823192927718,
+      "loss": 1.3895,
+      "step": 1700
+    },
+    {
+      "epoch": 0.6825273010920437,
+      "grad_norm": 1.2412734031677246,
+      "learning_rate": 0.0001544981799271971,
+      "loss": 1.3853,
+      "step": 1750
+    },
+    {
+      "epoch": 0.7020280811232449,
+      "grad_norm": 1.587342619895935,
+      "learning_rate": 0.000153198127925117,
+      "loss": 1.391,
+      "step": 1800
+    },
+    {
+      "epoch": 0.7215288611544461,
+      "grad_norm": 1.2071484327316284,
+      "learning_rate": 0.00015189807592303693,
+      "loss": 1.3917,
+      "step": 1850
+    },
+    {
+      "epoch": 0.7410296411856474,
+      "grad_norm": 1.3918330669403076,
+      "learning_rate": 0.00015059802392095685,
+      "loss": 1.3912,
+      "step": 1900
+    },
+    {
+      "epoch": 0.7605304212168487,
+      "grad_norm": 1.3605890274047852,
+      "learning_rate": 0.00014929797191887676,
+      "loss": 1.387,
+      "step": 1950
+    },
+    {
+      "epoch": 0.7800312012480499,
+      "grad_norm": 1.460693597793579,
+      "learning_rate": 0.00014799791991679668,
+      "loss": 1.3871,
+      "step": 2000
+    },
+    {
+      "epoch": 0.7995319812792512,
+      "grad_norm": 1.3369407653808594,
+      "learning_rate": 0.00014672386895475822,
+      "loss": 1.3895,
+      "step": 2050
+    },
+    {
+      "epoch": 0.8190327613104524,
+      "grad_norm": 1.328797459602356,
+      "learning_rate": 0.0001454238169526781,
+      "loss": 1.3825,
+      "step": 2100
+    },
+    {
+      "epoch": 0.8385335413416537,
+      "grad_norm": 1.2546560764312744,
+      "learning_rate": 0.00014412376495059802,
+      "loss": 1.3891,
+      "step": 2150
+    },
+    {
+      "epoch": 0.858034321372855,
+      "grad_norm": 1.578344702720642,
+      "learning_rate": 0.00014282371294851794,
+      "loss": 1.3848,
+      "step": 2200
+    },
+    {
+      "epoch": 0.8775351014040562,
+      "grad_norm": 1.6272181272506714,
+      "learning_rate": 0.0001415236609464379,
+      "loss": 1.3902,
+      "step": 2250
+    },
+    {
+      "epoch": 0.8970358814352574,
+      "grad_norm": 1.1834017038345337,
+      "learning_rate": 0.00014022360894435778,
+      "loss": 1.3948,
+      "step": 2300
+    },
+    {
+      "epoch": 0.9165366614664586,
+      "grad_norm": 1.5742530822753906,
+      "learning_rate": 0.0001389235569422777,
+      "loss": 1.3907,
+      "step": 2350
+    },
+    {
+      "epoch": 0.9360374414976599,
+      "grad_norm": 1.8273956775665283,
+      "learning_rate": 0.0001376235049401976,
+      "loss": 1.3859,
+      "step": 2400
+    },
+    {
+      "epoch": 0.9555382215288611,
+      "grad_norm": 1.108070969581604,
+      "learning_rate": 0.00013632345293811753,
+      "loss": 1.3861,
+      "step": 2450
+    },
+    {
+      "epoch": 0.9750390015600624,
+      "grad_norm": 1.1575729846954346,
+      "learning_rate": 0.00013502340093603745,
+      "loss": 1.3874,
+      "step": 2500
+    },
+    {
+      "epoch": 0.9945397815912637,
+      "grad_norm": 1.2663004398345947,
+      "learning_rate": 0.00013372334893395736,
+      "loss": 1.3919,
+      "step": 2550
+    },
+    {
+      "epoch": 1.0,
+      "eval_accuracy": 0.26053042121684866,
+      "eval_loss": 1.386296033859253,
+      "eval_runtime": 142.8097,
+      "eval_samples_per_second": 8.977,
+      "eval_steps_per_second": 2.248,
+      "step": 2564
+    },
+    {
+      "epoch": 1.0140405616224648,
+      "grad_norm": 1.3720601797103882,
+      "learning_rate": 0.00013242329693187728,
+      "loss": 1.3917,
+      "step": 2600
+    },
+    {
+      "epoch": 1.033541341653666,
+      "grad_norm": 1.375525951385498,
+      "learning_rate": 0.0001311232449297972,
+      "loss": 1.391,
+      "step": 2650
+    },
+    {
+      "epoch": 1.0530421216848673,
+      "grad_norm": 1.3909684419631958,
+      "learning_rate": 0.00012982319292771712,
+      "loss": 1.3894,
+      "step": 2700
+    },
+    {
+      "epoch": 1.0725429017160686,
+      "grad_norm": 1.4531859159469604,
+      "learning_rate": 0.00012852314092563703,
+      "loss": 1.3853,
+      "step": 2750
+    },
+    {
+      "epoch": 1.0920436817472698,
+      "grad_norm": 1.406068205833435,
+      "learning_rate": 0.00012722308892355695,
+      "loss": 1.382,
+      "step": 2800
+    },
+    {
+      "epoch": 1.111544461778471,
+      "grad_norm": 1.7967036962509155,
+      "learning_rate": 0.00012592303692147687,
+      "loss": 1.3943,
+      "step": 2850
+    },
+    {
+      "epoch": 1.1310452418096724,
+      "grad_norm": 1.228324294090271,
+      "learning_rate": 0.0001246229849193968,
+      "loss": 1.3831,
+      "step": 2900
+    },
+    {
+      "epoch": 1.1505460218408736,
+      "grad_norm": 1.0168524980545044,
+      "learning_rate": 0.0001233229329173167,
+      "loss": 1.3875,
+      "step": 2950
+    },
+    {
+      "epoch": 1.1700468018720749,
+      "grad_norm": 1.3154783248901367,
+      "learning_rate": 0.00012202288091523661,
+      "loss": 1.3853,
+      "step": 3000
+    },
+    {
+      "epoch": 1.1895475819032761,
+      "grad_norm": 1.4282019138336182,
+      "learning_rate": 0.00012072282891315654,
+      "loss": 1.3791,
+      "step": 3050
+    },
+    {
+      "epoch": 1.2090483619344774,
+      "grad_norm": 1.3305896520614624,
+      "learning_rate": 0.00011942277691107644,
+      "loss": 1.3806,
+      "step": 3100
+    },
+    {
+      "epoch": 1.2285491419656787,
+      "grad_norm": 1.8123506307601929,
+      "learning_rate": 0.00011812272490899636,
+      "loss": 1.3849,
+      "step": 3150
+    },
+    {
+      "epoch": 1.24804992199688,
+      "grad_norm": 1.4277228116989136,
+      "learning_rate": 0.00011682267290691629,
+      "loss": 1.3847,
+      "step": 3200
+    },
+    {
+      "epoch": 1.2675507020280812,
+      "grad_norm": 1.2390486001968384,
+      "learning_rate": 0.00011552262090483621,
+      "loss": 1.3908,
+      "step": 3250
+    },
+    {
+      "epoch": 1.2870514820592824,
+      "grad_norm": 1.3256953954696655,
+      "learning_rate": 0.00011422256890275611,
+      "loss": 1.3885,
+      "step": 3300
+    },
+    {
+      "epoch": 1.3065522620904837,
+      "grad_norm": 1.5108897686004639,
+      "learning_rate": 0.00011292251690067603,
+      "loss": 1.3841,
+      "step": 3350
+    },
+    {
+      "epoch": 1.3260530421216847,
+      "grad_norm": 1.3400274515151978,
+      "learning_rate": 0.00011162246489859596,
+      "loss": 1.3862,
+      "step": 3400
+    },
+    {
+      "epoch": 1.345553822152886,
+      "grad_norm": 1.4532562494277954,
+      "learning_rate": 0.00011032241289651587,
+      "loss": 1.4003,
+      "step": 3450
+    },
+    {
+      "epoch": 1.3650546021840873,
+      "grad_norm": 1.1774451732635498,
+      "learning_rate": 0.00010902236089443578,
+      "loss": 1.3883,
+      "step": 3500
+    },
+    {
+      "epoch": 1.3845553822152885,
+      "grad_norm": 1.5734331607818604,
+      "learning_rate": 0.0001077223088923557,
+      "loss": 1.3842,
+      "step": 3550
+    },
+    {
+      "epoch": 1.4040561622464898,
+      "grad_norm": 1.347231149673462,
+      "learning_rate": 0.0001064222568902756,
+      "loss": 1.3931,
+      "step": 3600
+    },
+    {
+      "epoch": 1.423556942277691,
+      "grad_norm": 1.2658965587615967,
+      "learning_rate": 0.00010512220488819554,
+      "loss": 1.3889,
+      "step": 3650
+    },
+    {
+      "epoch": 1.4430577223088923,
+      "grad_norm": 1.4009519815444946,
+      "learning_rate": 0.00010382215288611545,
+      "loss": 1.395,
+      "step": 3700
+    },
+    {
+      "epoch": 1.4625585023400935,
+      "grad_norm": 1.3800654411315918,
+      "learning_rate": 0.00010252210088403537,
+      "loss": 1.3909,
+      "step": 3750
+    },
+    {
+      "epoch": 1.4820592823712948,
+      "grad_norm": 1.6213831901550293,
+      "learning_rate": 0.00010122204888195527,
+      "loss": 1.393,
+      "step": 3800
+    },
+    {
+      "epoch": 1.501560062402496,
+      "grad_norm": 1.7539178133010864,
+      "learning_rate": 9.99219968798752e-05,
+      "loss": 1.3896,
+      "step": 3850
+    },
+    {
+      "epoch": 1.5210608424336973,
+      "grad_norm": 1.4965327978134155,
+      "learning_rate": 9.862194487779511e-05,
+      "loss": 1.385,
+      "step": 3900
+    },
+    {
+      "epoch": 1.5405616224648986,
+      "grad_norm": 1.2038394212722778,
+      "learning_rate": 9.732189287571504e-05,
+      "loss": 1.3892,
+      "step": 3950
+    },
+    {
+      "epoch": 1.5600624024960998,
+      "grad_norm": 1.5250635147094727,
+      "learning_rate": 9.602184087363494e-05,
+      "loss": 1.3817,
+      "step": 4000
+    },
+    {
+      "epoch": 1.579563182527301,
+      "grad_norm": 1.1984943151474,
+      "learning_rate": 9.472178887155486e-05,
+      "loss": 1.3865,
+      "step": 4050
+    },
+    {
+      "epoch": 1.5990639625585024,
+      "grad_norm": 1.605357050895691,
+      "learning_rate": 9.344773790951638e-05,
+      "loss": 1.3855,
+      "step": 4100
+    },
+    {
+      "epoch": 1.6185647425897036,
+      "grad_norm": 1.4252681732177734,
+      "learning_rate": 9.21476859074363e-05,
+      "loss": 1.3841,
+      "step": 4150
+    },
+    {
+      "epoch": 1.6380655226209049,
+      "grad_norm": 1.3211374282836914,
+      "learning_rate": 9.084763390535622e-05,
+      "loss": 1.3782,
+      "step": 4200
+    },
+    {
+      "epoch": 1.6575663026521061,
+      "grad_norm": 1.4356790781021118,
+      "learning_rate": 8.954758190327614e-05,
+      "loss": 1.3862,
+      "step": 4250
+    },
+    {
+      "epoch": 1.6770670826833074,
+      "grad_norm": 1.3711603879928589,
+      "learning_rate": 8.824752990119605e-05,
+      "loss": 1.3906,
+      "step": 4300
+    },
+    {
+      "epoch": 1.6965678627145087,
+      "grad_norm": 1.481302261352539,
+      "learning_rate": 8.694747789911597e-05,
+      "loss": 1.3863,
+      "step": 4350
+    },
+    {
+      "epoch": 1.71606864274571,
+      "grad_norm": 1.3888981342315674,
+      "learning_rate": 8.564742589703589e-05,
+      "loss": 1.3818,
+      "step": 4400
+    },
+    {
+      "epoch": 1.7355694227769112,
+      "grad_norm": 1.4192887544631958,
+      "learning_rate": 8.43473738949558e-05,
+      "loss": 1.3835,
+      "step": 4450
+    },
+    {
+      "epoch": 1.7550702028081124,
+      "grad_norm": 1.0555356740951538,
+      "learning_rate": 8.304732189287572e-05,
+      "loss": 1.3967,
+      "step": 4500
+    },
+    {
+      "epoch": 1.7745709828393137,
+      "grad_norm": 1.3277314901351929,
+      "learning_rate": 8.174726989079563e-05,
+      "loss": 1.3863,
+      "step": 4550
+    },
+    {
+      "epoch": 1.794071762870515,
+      "grad_norm": 1.600809097290039,
+      "learning_rate": 8.044721788871556e-05,
+      "loss": 1.3833,
+      "step": 4600
+    },
+    {
+      "epoch": 1.8135725429017162,
+      "grad_norm": 1.5007127523422241,
+      "learning_rate": 7.914716588663546e-05,
+      "loss": 1.3892,
+      "step": 4650
+    },
+    {
+      "epoch": 1.8330733229329175,
+      "grad_norm": 1.4107766151428223,
+      "learning_rate": 7.784711388455539e-05,
+      "loss": 1.3925,
+      "step": 4700
+    },
+    {
+      "epoch": 1.8525741029641187,
+      "grad_norm": 1.4171024560928345,
+      "learning_rate": 7.65470618824753e-05,
+      "loss": 1.3899,
+      "step": 4750
+    },
+    {
+      "epoch": 1.8720748829953198,
+      "grad_norm": 1.453108787536621,
+      "learning_rate": 7.524700988039521e-05,
+      "loss": 1.3866,
+      "step": 4800
+    },
+    {
+      "epoch": 1.891575663026521,
+      "grad_norm": 1.537400722503662,
+      "learning_rate": 7.394695787831513e-05,
+      "loss": 1.3886,
+      "step": 4850
+    },
+    {
+      "epoch": 1.9110764430577223,
+      "grad_norm": 1.4526410102844238,
+      "learning_rate": 7.264690587623505e-05,
+      "loss": 1.3791,
+      "step": 4900
+    },
+    {
+      "epoch": 1.9305772230889235,
+      "grad_norm": 1.2780728340148926,
+      "learning_rate": 7.134685387415497e-05,
+      "loss": 1.3832,
+      "step": 4950
+    },
+    {
+      "epoch": 1.9500780031201248,
+      "grad_norm": 1.2071354389190674,
+      "learning_rate": 7.004680187207488e-05,
+      "loss": 1.3917,
+      "step": 5000
+    },
+    {
+      "epoch": 1.969578783151326,
+      "grad_norm": 1.2375810146331787,
+      "learning_rate": 6.87467498699948e-05,
+      "loss": 1.3917,
+      "step": 5050
+    },
+    {
+      "epoch": 1.9890795631825273,
+      "grad_norm": 1.3808921575546265,
+      "learning_rate": 6.744669786791472e-05,
+      "loss": 1.391,
+      "step": 5100
+    },
+    {
+      "epoch": 2.0,
+      "eval_accuracy": 0.24960998439937598,
+      "eval_loss": 1.3862944841384888,
+      "eval_runtime": 142.4422,
+      "eval_samples_per_second": 9.0,
+      "eval_steps_per_second": 2.254,
+      "step": 5128
+    },
+    {
+      "epoch": 2.0085803432137284,
+      "grad_norm": 1.2927422523498535,
+      "learning_rate": 6.614664586583464e-05,
+      "loss": 1.3859,
+      "step": 5150
+    },
+    {
+      "epoch": 2.0280811232449296,
+      "grad_norm": 1.351449966430664,
+      "learning_rate": 6.484659386375455e-05,
+      "loss": 1.3904,
+      "step": 5200
+    },
+    {
+      "epoch": 2.047581903276131,
+      "grad_norm": 1.3569812774658203,
+      "learning_rate": 6.354654186167447e-05,
+      "loss": 1.3879,
+      "step": 5250
+    },
+    {
+      "epoch": 2.067082683307332,
+      "grad_norm": 1.5670419931411743,
+      "learning_rate": 6.224648985959438e-05,
+      "loss": 1.3898,
+      "step": 5300
+    },
+    {
+      "epoch": 2.0865834633385334,
+      "grad_norm": 1.4736590385437012,
+      "learning_rate": 6.094643785751431e-05,
+      "loss": 1.3779,
+      "step": 5350
+    },
+    {
+      "epoch": 2.1060842433697347,
+      "grad_norm": 1.5393387079238892,
+      "learning_rate": 5.964638585543422e-05,
+      "loss": 1.3848,
+      "step": 5400
+    },
+    {
+      "epoch": 2.125585023400936,
+      "grad_norm": 1.4814316034317017,
+      "learning_rate": 5.834633385335414e-05,
+      "loss": 1.3917,
+      "step": 5450
+    },
+    {
+      "epoch": 2.145085803432137,
+      "grad_norm": 1.4354616403579712,
+      "learning_rate": 5.704628185127405e-05,
+      "loss": 1.3883,
+      "step": 5500
+    },
+    {
+      "epoch": 2.1645865834633384,
+      "grad_norm": 1.2791849374771118,
+      "learning_rate": 5.574622984919396e-05,
+      "loss": 1.3829,
+      "step": 5550
+    },
+    {
+      "epoch": 2.1840873634945397,
+      "grad_norm": 1.5058386325836182,
+      "learning_rate": 5.444617784711389e-05,
+      "loss": 1.3897,
+      "step": 5600
+    },
+    {
+      "epoch": 2.203588143525741,
+      "grad_norm": 1.3470947742462158,
+      "learning_rate": 5.31461258450338e-05,
+      "loss": 1.3892,
+      "step": 5650
+    },
+    {
+      "epoch": 2.223088923556942,
+      "grad_norm": 1.2379658222198486,
+      "learning_rate": 5.184607384295372e-05,
+      "loss": 1.3866,
+      "step": 5700
+    },
+    {
+      "epoch": 2.2425897035881435,
+      "grad_norm": 1.3489643335342407,
+      "learning_rate": 5.054602184087363e-05,
+      "loss": 1.3921,
+      "step": 5750
+    },
+    {
+      "epoch": 2.2620904836193447,
+      "grad_norm": 1.891623854637146,
+      "learning_rate": 4.924596983879356e-05,
+      "loss": 1.3933,
+      "step": 5800
+    },
+    {
+      "epoch": 2.281591263650546,
+      "grad_norm": 1.2890557050704956,
+      "learning_rate": 4.7945917836713475e-05,
+      "loss": 1.3881,
+      "step": 5850
+    },
+    {
+      "epoch": 2.3010920436817472,
+      "grad_norm": 1.4014248847961426,
+      "learning_rate": 4.664586583463339e-05,
+      "loss": 1.3833,
+      "step": 5900
+    },
+    {
+      "epoch": 2.3205928237129485,
+      "grad_norm": 1.367763876914978,
+      "learning_rate": 4.53458138325533e-05,
+      "loss": 1.393,
+      "step": 5950
+    },
+    {
+      "epoch": 2.3400936037441498,
+      "grad_norm": 1.4445606470108032,
+      "learning_rate": 4.404576183047322e-05,
+      "loss": 1.3844,
+      "step": 6000
+    },
+    {
+      "epoch": 2.359594383775351,
+      "grad_norm": 1.5409669876098633,
+      "learning_rate": 4.274570982839314e-05,
+      "loss": 1.3882,
+      "step": 6050
+    },
+    {
+      "epoch": 2.3790951638065523,
+      "grad_norm": 1.3444645404815674,
+      "learning_rate": 4.147165886635466e-05,
+      "loss": 1.3862,
+      "step": 6100
+    },
+    {
+      "epoch": 2.3985959438377535,
+      "grad_norm": 1.3359966278076172,
+      "learning_rate": 4.017160686427458e-05,
+      "loss": 1.3867,
+      "step": 6150
+    },
+    {
+      "epoch": 2.418096723868955,
+      "grad_norm": 1.2808890342712402,
+      "learning_rate": 3.887155486219449e-05,
+      "loss": 1.3858,
+      "step": 6200
+    },
+    {
+      "epoch": 2.437597503900156,
+      "grad_norm": 1.3361515998840332,
+      "learning_rate": 3.7571502860114405e-05,
+      "loss": 1.3914,
+      "step": 6250
+    },
+    {
+      "epoch": 2.4570982839313573,
+      "grad_norm": 1.3615878820419312,
+      "learning_rate": 3.627145085803432e-05,
+      "loss": 1.3878,
+      "step": 6300
+    },
+    {
+      "epoch": 2.4765990639625586,
+      "grad_norm": 1.1526952981948853,
+      "learning_rate": 3.497139885595424e-05,
+      "loss": 1.3856,
+      "step": 6350
+    },
+    {
+      "epoch": 2.49609984399376,
+      "grad_norm": 1.3968396186828613,
+      "learning_rate": 3.367134685387416e-05,
+      "loss": 1.3897,
+      "step": 6400
+    },
+    {
+      "epoch": 2.515600624024961,
+      "grad_norm": 1.4438717365264893,
+      "learning_rate": 3.237129485179407e-05,
+      "loss": 1.3888,
+      "step": 6450
+    },
+    {
+      "epoch": 2.5351014040561624,
+      "grad_norm": 1.36077880859375,
+      "learning_rate": 3.1071242849713986e-05,
+      "loss": 1.3842,
+      "step": 6500
+    },
+    {
+      "epoch": 2.5546021840873636,
+      "grad_norm": 1.4155772924423218,
+      "learning_rate": 2.9771190847633907e-05,
+      "loss": 1.3924,
+      "step": 6550
+    },
+    {
+      "epoch": 2.574102964118565,
+      "grad_norm": 1.7112796306610107,
+      "learning_rate": 2.8471138845553824e-05,
+      "loss": 1.3886,
+      "step": 6600
+    },
+    {
+      "epoch": 2.593603744149766,
+      "grad_norm": 1.2608891725540161,
+      "learning_rate": 2.7171086843473742e-05,
+      "loss": 1.3854,
+      "step": 6650
+    },
+    {
+      "epoch": 2.6131045241809674,
+      "grad_norm": 1.7250005006790161,
+      "learning_rate": 2.5871034841393656e-05,
+      "loss": 1.3917,
+      "step": 6700
+    },
+    {
+      "epoch": 2.6326053042121687,
+      "grad_norm": 1.2738689184188843,
+      "learning_rate": 2.4570982839313573e-05,
+      "loss": 1.3871,
+      "step": 6750
+    },
+    {
+      "epoch": 2.6521060842433695,
+      "grad_norm": 1.44166100025177,
+      "learning_rate": 2.327093083723349e-05,
+      "loss": 1.3884,
+      "step": 6800
+    },
+    {
+      "epoch": 2.6716068642745707,
+      "grad_norm": 1.2396310567855835,
+      "learning_rate": 2.1970878835153408e-05,
+      "loss": 1.3872,
+      "step": 6850
+    },
+    {
+      "epoch": 2.691107644305772,
+      "grad_norm": 1.3184744119644165,
+      "learning_rate": 2.0670826833073322e-05,
+      "loss": 1.3828,
+      "step": 6900
+    },
+    {
+      "epoch": 2.7106084243369732,
+      "grad_norm": 1.4840060472488403,
+      "learning_rate": 1.937077483099324e-05,
+      "loss": 1.3882,
+      "step": 6950
+    },
+    {
+      "epoch": 2.7301092043681745,
+      "grad_norm": 1.2850672006607056,
+      "learning_rate": 1.8070722828913157e-05,
+      "loss": 1.3857,
+      "step": 7000
+    },
+    {
+      "epoch": 2.7496099843993758,
+      "grad_norm": 1.3318556547164917,
+      "learning_rate": 1.6770670826833075e-05,
+      "loss": 1.3842,
+      "step": 7050
+    },
+    {
+      "epoch": 2.769110764430577,
+      "grad_norm": 1.5529357194900513,
+      "learning_rate": 1.5470618824752992e-05,
+      "loss": 1.3901,
+      "step": 7100
+    },
+    {
+      "epoch": 2.7886115444617783,
+      "grad_norm": 1.496201992034912,
+      "learning_rate": 1.4170566822672906e-05,
+      "loss": 1.3822,
+      "step": 7150
+    },
+    {
+      "epoch": 2.8081123244929795,
+      "grad_norm": 1.2309885025024414,
+      "learning_rate": 1.2870514820592824e-05,
+      "loss": 1.387,
+      "step": 7200
+    },
+    {
+      "epoch": 2.827613104524181,
+      "grad_norm": 1.1437956094741821,
+      "learning_rate": 1.1570462818512741e-05,
+      "loss": 1.3871,
+      "step": 7250
+    },
+    {
+      "epoch": 2.847113884555382,
+      "grad_norm": 1.314348816871643,
+      "learning_rate": 1.0270410816432657e-05,
+      "loss": 1.3841,
+      "step": 7300
+    },
+    {
+      "epoch": 2.8666146645865833,
+      "grad_norm": 1.5572781562805176,
+      "learning_rate": 8.970358814352575e-06,
+      "loss": 1.386,
+      "step": 7350
+    },
+    {
+      "epoch": 2.8861154446177846,
+      "grad_norm": 1.4093748331069946,
+      "learning_rate": 7.670306812272492e-06,
+      "loss": 1.3895,
+      "step": 7400
+    },
+    {
+      "epoch": 2.905616224648986,
+      "grad_norm": 1.3720757961273193,
+      "learning_rate": 6.370254810192408e-06,
+      "loss": 1.3884,
+      "step": 7450
+    },
+    {
+      "epoch": 2.925117004680187,
+      "grad_norm": 1.285584807395935,
+      "learning_rate": 5.070202808112324e-06,
+      "loss": 1.3937,
+      "step": 7500
+    },
+    {
+      "epoch": 2.9446177847113884,
+      "grad_norm": 1.2192853689193726,
+      "learning_rate": 3.770150806032242e-06,
+      "loss": 1.3859,
+      "step": 7550
+    },
+    {
+      "epoch": 2.9641185647425896,
+      "grad_norm": 1.4732896089553833,
+      "learning_rate": 2.470098803952158e-06,
+      "loss": 1.3872,
+      "step": 7600
+    },
+    {
+      "epoch": 2.983619344773791,
+      "grad_norm": 1.5972859859466553,
+      "learning_rate": 1.170046801872075e-06,
+      "loss": 1.3902,
+      "step": 7650
+    },
+    {
+      "epoch": 3.0,
+      "eval_accuracy": 0.24414976599063962,
+      "eval_loss": 1.3862601518630981,
+      "eval_runtime": 142.7591,
+      "eval_samples_per_second": 8.98,
+      "eval_steps_per_second": 2.249,
+      "step": 7692
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 7692,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 3.249051576938496e+16,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-7692/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:60592bff6cbc053d3a93f6d39ee62300fafc1a17252704e129e528cb1b434102
+size 5304

special_tokens_map.json

@@ -0,0 +1,15 @@
+{
+  "bos_token": "[CLS]",
+  "cls_token": "[CLS]",
+  "eos_token": "[SEP]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

spm.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:13c8d666d62a7bc4ac8f040aab68e942c861f93303156cc28f5c7e885d86d6e3
+size 4305025

tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "250101": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "[CLS]",
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "[CLS]",
+  "do_lower_case": false,
+  "eos_token": "[SEP]",
+  "mask_token": "[MASK]",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "sp_model_kwargs": {},
+  "split_by_punct": false,
+  "tokenizer_class": "DebertaV2Tokenizer",
+  "unk_token": "[UNK]",
+  "vocab_type": "spm"
+}