|                 Tasks                 |Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|---------------------------------------|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|mmlu                                   |      2|none            |      |acc        |   |0.8565|±  |0.0028|
| - humanities                          |      2|none            |     0|acc        |↑  |0.7896|±  |0.0057|
|  - formal_logic                       |      1|none            |     0|acc        |↑  |0.7460|±  |0.0389|
|  - high_school_european_history       |      1|none            |     0|acc        |↑  |0.9152|±  |0.0218|
|  - high_school_us_history             |      1|none            |     0|acc        |↑  |0.9461|±  |0.0159|
|  - high_school_world_history          |      1|none            |     0|acc        |↑  |0.9662|±  |0.0118|
|  - international_law                  |      1|none            |     0|acc        |↑  |0.9587|±  |0.0182|
|  - jurisprudence                      |      1|none            |     0|acc        |↑  |0.8981|±  |0.0292|
|  - logical_fallacies                  |      1|none            |     0|acc        |↑  |0.9080|±  |0.0227|
|  - moral_disputes                     |      1|none            |     0|acc        |↑  |0.8671|±  |0.0183|
|  - moral_scenarios                    |      1|none            |     0|acc        |↑  |0.6525|±  |0.0159|
|  - philosophy                         |      1|none            |     0|acc        |↑  |0.9003|±  |0.0170|
|  - prehistory                         |      1|none            |     0|acc        |↑  |0.9352|±  |0.0137|
|  - professional_law                   |      1|none            |     0|acc        |↑  |0.6930|±  |0.0118|
|  - world_religions                    |      1|none            |     0|acc        |↑  |0.9181|±  |0.0210|
| - other                               |      2|none            |     0|acc        |↑  |0.8857|±  |0.0055|
|  - business_ethics                    |      1|none            |     0|acc        |↑  |0.8900|±  |0.0314|
|  - clinical_knowledge                 |      1|none            |     0|acc        |↑  |0.9321|±  |0.0155|
|  - college_medicine                   |      1|none            |     0|acc        |↑  |0.8382|±  |0.0281|
|  - global_facts                       |      1|none            |     0|acc        |↑  |0.7100|±  |0.0456|
|  - human_aging                        |      1|none            |     0|acc        |↑  |0.8206|±  |0.0257|
|  - management                         |      1|none            |     0|acc        |↑  |0.9126|±  |0.0280|
|  - marketing                          |      1|none            |     0|acc        |↑  |0.9615|±  |0.0126|
|  - medical_genetics                   |      1|none            |     0|acc        |↑  |0.9700|±  |0.0171|
|  - miscellaneous                      |      1|none            |     0|acc        |↑  |0.9515|±  |0.0077|
|  - nutrition                          |      1|none            |     0|acc        |↑  |0.8889|±  |0.0180|
|  - professional_accounting            |      1|none            |     0|acc        |↑  |0.8191|±  |0.0230|
|  - professional_medicine              |      1|none            |     0|acc        |↑  |0.9375|±  |0.0147|
|  - virology                           |      1|none            |     0|acc        |↑  |0.5904|±  |0.0383|
| - social sciences                     |      2|none            |     0|acc        |↑  |0.9214|±  |0.0048|
|  - econometrics                       |      1|none            |     0|acc        |↑  |0.7982|±  |0.0378|
|  - high_school_geography              |      1|none            |     0|acc        |↑  |0.9293|±  |0.0183|
|  - high_school_government_and_politics|      1|none            |     0|acc        |↑  |0.9845|±  |0.0089|
|  - high_school_macroeconomics         |      1|none            |     0|acc        |↑  |0.9205|±  |0.0137|
|  - high_school_microeconomics         |      1|none            |     0|acc        |↑  |0.9664|±  |0.0117|
|  - high_school_psychology             |      1|none            |     0|acc        |↑  |0.9596|±  |0.0084|
|  - human_sexuality                    |      1|none            |     0|acc        |↑  |0.8855|±  |0.0279|
|  - professional_psychology            |      1|none            |     0|acc        |↑  |0.9232|±  |0.0108|
|  - public_relations                   |      1|none            |     0|acc        |↑  |0.8091|±  |0.0376|
|  - security_studies                   |      1|none            |     0|acc        |↑  |0.8571|±  |0.0224|
|  - sociology                          |      1|none            |     0|acc        |↑  |0.9055|±  |0.0207|
|  - us_foreign_policy                  |      1|none            |     0|acc        |↑  |0.9600|±  |0.0197|
| - stem                                |      2|none            |     0|acc        |↑  |0.8643|±  |0.0059|
|  - abstract_algebra                   |      1|none            |     0|acc        |↑  |0.7200|±  |0.0451|
|  - anatomy                            |      1|none            |     0|acc        |↑  |0.8815|±  |0.0279|
|  - astronomy                          |      1|none            |     0|acc        |↑  |0.9079|±  |0.0235|
|  - college_biology                    |      1|none            |     0|acc        |↑  |0.9306|±  |0.0213|
|  - college_chemistry                  |      1|none            |     0|acc        |↑  |0.6800|±  |0.0469|
|  - college_computer_science           |      1|none            |     0|acc        |↑  |0.8000|±  |0.0402|
|  - college_mathematics                |      1|none            |     0|acc        |↑  |0.8000|±  |0.0402|
|  - college_physics                    |      1|none            |     0|acc        |↑  |0.8627|±  |0.0342|
|  - computer_security                  |      1|none            |     0|acc        |↑  |0.8600|±  |0.0349|
|  - conceptual_physics                 |      1|none            |     0|acc        |↑  |0.9277|±  |0.0169|
|  - electrical_engineering             |      1|none            |     0|acc        |↑  |0.8897|±  |0.0261|
|  - elementary_mathematics             |      1|none            |     0|acc        |↑  |0.9312|±  |0.0130|
|  - high_school_biology                |      1|none            |     0|acc        |↑  |0.9484|±  |0.0126|
|  - high_school_chemistry              |      1|none            |     0|acc        |↑  |0.8670|±  |0.0239|
|  - high_school_computer_science       |      1|none            |     0|acc        |↑  |0.9700|±  |0.0171|
|  - high_school_mathematics            |      1|none            |     0|acc        |↑  |0.6926|±  |0.0281|
|  - high_school_physics                |      1|none            |     0|acc        |↑  |0.8411|±  |0.0299|
|  - high_school_statistics             |      1|none            |     0|acc        |↑  |0.8750|±  |0.0226|
|  - machine_learning                   |      1|none            |     0|acc        |↑  |0.8125|±  |0.0370|
|gsm8k                                  |      3|flexible-extract|     5|exact_match|↑  |0.9454|±  |0.0063|
|                                       |       |strict-match    |     5|exact_match|↑  |0.9462|±  |0.0062|

|      Groups      |Version|Filter|n-shot|Metric|   |Value |   |Stderr|
|------------------|------:|------|-----:|------|---|-----:|---|-----:|
|mmlu              |      2|none  |      |acc   |   |0.8565|±  |0.0028|
| - humanities     |      2|none  |     0|acc   |↑  |0.7896|±  |0.0057|
| - other          |      2|none  |     0|acc   |↑  |0.8857|±  |0.0055|
| - social sciences|      2|none  |     0|acc   |↑  |0.9214|±  |0.0048|
| - stem           |      2|none  |     0|acc   |↑  |0.8643|±  |0.0059|