Upload final SFT boxed-only general knowledge model

README.md

@@ -1,40 +1,171 @@
 ---
 license: apache-2.0
+base_model: Qwen/Qwen3-1.7B
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- qwen3
+- multiple-choice
+- general-knowledge
+- lora
+- sft
+- boxed-answer
 ---
 
-## v3 GRPO general knowledge model
+# General Knowledge Model
 
-Updated: 2026-06-04 11:22 UTC
+This is the final General Knowledge individual model for the CS-552 Modern NLP Spring 2026 standardized project.
 
-This repository stores the final v3 GRPO general knowledge model for the CS-552 2026 Databand project.
+The submitted model is the SFT-only merged model. A later DPO experiment was run on ARC/CommonsenseQA mistakes, but it reduced benchmark accuracy, so it was not selected as the final model.
 
-Model source on the training cluster:
+## Model behavior
 
-/scratch/general_knowledge_sft_v3_lora_grpo/outputs/grpo_v3_maxredux_4000/final
+The model is specialized for multiple-choice general knowledge questions. It is prompted to output exactly one final boxed answer, for example:
 
-The model was trained from the v3 LoRA SFT model using GRPO on the MMLU-Pro / MMLU-Redux general-knowledge data split.
+\boxed{A}
 
-The final model files were verified locally before upload, including:
+The chat template enforces concise answer-only behavior and supports choices labeled from A through T.
 
-- config.json
-- generation_config.json
-- model.safetensors
-- tokenizer.json
-- tokenizer_config.json
-- chat_template.jinja
+## Training setup
 
-Important generation/config fields:
+Starting point:
 
-- bos_token_id = 151643
-- eos_token_id = 151645
-- pad_token_id = 151643
-- use_cache = True
-- generation eos_token_id = [151645, 151643]
-- temperature = 0.1
-- top_k = 20
-- top_p = 0.8
+- Baseline working model folder with the project chat template and generation config
+- LoRA SFT on top of the baseline model
+- Final model produced by merging the LoRA adapter into the baseline model
 
-Expected output format:
+Training method:
 
-\boxed{LETTER}
+- LoRA supervised fine-tuning
+- Loss masked so that only the final assistant boxed answer contributes to training
+- Prompt, system message, question text, choices, chat markers, and template tokens are masked with -100
+- Assistant target format: \boxed{LETTER}
 
+LoRA configuration:
+
+- r = 16
+- lora_alpha = 32
+- lora_dropout = 0.05
+- Target modules:
+  - q_proj
+  - k_proj
+  - v_proj
+  - o_proj
+  - gate_proj
+  - up_proj
+  - down_proj
+
+Main training hyperparameters:
+
+- Learning rate: 8e-5
+- Epochs: 1
+- Batch size per device: 1
+- Gradient accumulation steps: 8
+- Max sequence length: 8192
+- Precision: bf16
+- Scheduler: cosine
+- Warmup steps: 20
+
+## SFT datasets
+
+The SFT training data was built from:
+
+1. Kaggle LLM Science
+2. EduQG
+3. EduAdapt, MCQ-only questions
+4. NCERT_MCQs
+5. SciQ train
+6. OpenBookQA train
+
+The final SFT dataset was capped below 30,000 training rows.
+
+Final SFT data sizes:
+
+- Train: 26,120
+- Validation: 2,000
+
+The answer labels were balanced uniformly across A through T separately for train and validation.
+
+Train answer distribution:
+
+- A through T: 1,306 examples each
+
+Validation answer distribution:
+
+- A through T: 100 examples each
+
+## Evaluation
+
+The final selected model is the SFT-only merged model.
+
+Evaluation sets:
+
+- Validation set: 2,000
+- MMLU Pro: 2,000, uniformly sampled across categories
+- MMLU Redux: 2,000, uniformly sampled across subjects
+- SuperGPQA: 2,000, uniformly sampled across disciplines
+
+SFT-only results:
+
+| Benchmark | Boxed extraction | Accuracy |
+|---|---:|---:|
+| MMLU Pro 2k | 100.00% | 37.85% |
+| MMLU Redux 2k | 100.00% | 56.25% |
+| SuperGPQA 2k | 99.95% | 27.55% |
+
+Baseline comparison:
+
+| Benchmark | Baseline accuracy | SFT-only accuracy |
+|---|---:|---:|
+| Validation 2k | 16.00% | not logged in the final grep output |
+| MMLU Pro 2k | 18.05% | 37.85% |
+| MMLU Redux 2k | 11.40% | 56.25% |
+| SuperGPQA 2k | 15.85% | 27.55% |
+
+DPO experiment, not selected:
+
+| Benchmark | SFT + DPO accuracy |
+|---|---:|
+| Validation 2k | 79.75% |
+| MMLU Pro 2k | 35.25% |
+| MMLU Redux 2k | 50.90% |
+| SuperGPQA 2k | 23.45% |
+
+DPO improved the internal validation metric but reduced the external benchmark scores, so the SFT-only model was selected.
+
+## Expected input format
+
+The model expects a multiple-choice question formatted like:
+
+Question text here?
+
+Choices:
+A. first option
+B. second option
+C. third option
+D. fourth option
+
+It should answer with only:
+
+\boxed{A}
+
+## Reproducibility notes
+
+Important files from the training folder:
+
+- SFT trainer: scripts/train_v3_lora_sft_masked.py
+- SFT data builder: scripts/build_my_sft_data_balanced.py
+- Merge script: scripts/merge_v3_lora_adapter.py
+- Evaluation script: scripts/evaluate_mcq_accuracy.py
+
+Final selected model folder before upload:
+
+outputs/lora_sft_v3_boxed_only/merged_full_model
+
+SFT LoRA adapter:
+
+outputs/lora_sft_v3_boxed_only/final_adapter
+
+DPO adapter, experimental and not selected:
+
+outputs/lora_dpo_arc_csqa_on_sft/final_adapter

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:813eaad9f372af34c6dbe827ef83b2f6fa4242f384b4b4513f0aa5030b9e5e20
+oid sha256:bbac80fd49d664b0096daf5c93346809d1275bde1375705d0bda731204b5ab90
 size 3441185608