| --- |
| license: apache-2.0 |
| language: |
| - en |
| tags: |
| - math-code |
| - mathematics |
| - verification |
| - closed-label |
| - autoscientist |
| base_model: mistralai/Mixtral-8x7B-Instruct-v0.1 |
| --- |
| |
| # Math Solution Verification Classifier |
|
|
| **Author:** Hussein Adeiza (mabera) |
| **Role:** Licensed Environmental Health Officer, Abuja Nigeria |
| **Base Model:** Mixtral 8x7B |
| **Fine-tuned with:** AutoScientist by Adaption Labs |
|
|
| ## Model Description |
| A LoRA adapter fine-tuned to verify whether a proposed answer to a real |
| competition math problem is correct, classifying it as Correct or |
| Incorrect. This is a genuinely global-scope submission (not Nigeria- |
| specific), addressing a universal AI capability question: can a model |
| reliably grade mathematical correctness? |
|
|
| ## Training Data |
| - Source: MATH dataset (Hendrycks et al., NeurIPS 2021), accessed via |
| a properly-cited GitHub derivative (rasbt/math_full_minus_math500), |
| downloaded directly, 12,000 real competition problems |
| - Dataset: 20 rows (10 real problems x 2 answer variants each: one |
| correct, one deliberately perturbed incorrect answer, disclosed as |
| a constructed perturbation, not a real student error) |
| - Kaggle: https://www.kaggle.com/datasets/yunusahusseinadeiza/math-solution-verification-classifier |
| |
| ## Important Note: Column Selection Correction |
| During training setup, the platform defaulted to training on |
| "Enhanced completion" text, which had drifted away from the closed- |
| label Correct/Incorrect structure into generic step-by-step tutoring |
| language, losing the classification task entirely. This was caught |
| and manually corrected by selecting "Original completion" instead |
| before training. Worth flagging for other builders working on |
| closed-label tasks: check which completion column is actually |
| selected before training, since the platform default may not be |
| the one you expect. |
| |
| ## Training Metrics |
| - **Win rate (on dataset): 76% adapted vs 24% base model** |
| - Base model: mistralai/Mixtral-8x7B-Instruct-v0.1 |
| - Method: LoRA (confirmed via training config), no recipe modifications |
| - Dataset quality: 7.0 → 9.0 (+28.6% relative improvement, **Grade A**) |
| - Percentile: 33.0 |
| - Domain classification: Math (100%), a clean, accurate match |
| |
| ## Verification |
| All 20 rows independently, programmatically verified before training: |
| every Correct/Incorrect classification checked against the real |
| ground-truth answer looked up directly in the raw MATH dataset source |
| file. 20/20 pass rate, demonstrated live in the accompanying Kaggle |
| notebook. |
| |
| ## Why This Result Matters |
| This is the highest quality grade (A) and highest quality score |
| improvement (+28.6%) in this author's 18-submission AutoScientist |
| portfolio, achieved using the same disciplined closed-label methodology |
| (no recipe modifications, deterministic ground truth, independent |
| verification) established across the AMR and Loan Classifier |
| submissions, applied here to a genuinely global rather than |
| Nigeria-specific problem. |
| |
| ## Credits |
| Powered by Adaptive Data — Adaption Labs |
| AutoScientist Challenge 2026, Part 2 — Math & Code Category |
