asinha08/slm-125m-sft-pilot-2k

SLM 125M SFT Pilot 2K

This is a supervised fine-tuned 125M-parameter causal language model based on thesreedath/slm-125m-base.

Training Summary

Field	Value
SFT dataset run	pilot-2k-v2
Train examples	1800
Validation examples	100
Epochs	3
Optimizer steps	339
Train tokens per epoch	1262981
Assistant-label tokens per epoch	62190
Total train-token exposures	3788943
Total assistant-label token exposures	186570
Initial validation loss	3.6077
Initial validation perplexity	36.88
Final validation loss	1.5028
Final validation perplexity	4.49
Training elapsed seconds	160.1
Estimated GPU-only training cost	$0.0355

Dataset

The SFT dataset contains 2,000 generated and judged instruction examples:

Split	Examples
Train	1,800
Validation	100
Test	100

Source mix:

Source	Examples
Case law	703
SEC filings	890
FineWeb-Edu	407

Task types include grounded question answering, unanswerable/refusal cases, multi-step QA, JSON extraction, summarization, plain-English rewrites, and comparative QA.

Intended Use

This is a small research/experimentation model for legal and financial instruction-following behavior. It is not a substitute for professional legal, financial, or compliance advice.

Prompt Format

The model was trained with the tokenizer/chat markers from the base model:

<|system|>
You are a careful legal and financial assistant...
<|user|>
Context:
...

Question:
...
<|assistant|>
...
<|eos|>

Limitations

• The SFT dataset is small, so behavior should be evaluated carefully.
• Answers should be grounded in supplied context.
• The model may hallucinate if used without retrieval/context.
• Perplexity is measured on the generated validation split, not a broad external benchmark.