File size: 2,466 Bytes
3c6989d 45dc42b 3c6989d 45dc42b 3c6989d 45dc42b 3c6989d 45dc42b 3c6989d 45dc42b 3c6989d 45dc42b 3c6989d 45dc42b 3c6989d 45dc42b 3c6989d 45dc42b 3c6989d 45dc42b 3c6989d 45dc42b 3c6989d 45dc42b 3c6989d 45dc42b 3c6989d 45dc42b 3c6989d 45dc42b 3c6989d 45dc42b 3c6989d 45dc42b 3c6989d 45dc42b 3c6989d 45dc42b 3c6989d 45dc42b 3c6989d 45dc42b 3c6989d 45dc42b 3c6989d 45dc42b 3c6989d 45dc42b 3c6989d 45dc42b 3c6989d 45dc42b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 | ---
base_model: unsloth/Qwen3-0.6B-unsloth-bnb-4bit
library_name: peft
pipeline_tag: text-generation
tags:
- base_model:adapter:unsloth/Qwen3-0.6B-unsloth-bnb-4bit
- lora
- sft
- transformers
- trl
- unsloth
---
# Qwen3 Decodable Story SFT
## Overview
Qwen3 Decodable Story SFT is a QLoRA fine-tuned version of Qwen3-0.6B developed to generate decodable reading stories for beginning readers.
Given one or more target phonics patterns, the model generates a short story intended to emphasize those patterns while maintaining a coherent beginning, middle, and ending.
This project explores whether supervised fine-tuning on a relatively small, curated dataset can teach a reliable educational behavior to a small language model.
## Model Details
- **Base model:** `unsloth/Qwen3-0.6B-unsloth-bnb-4bit`
- **Fine-tuning method:** QLoRA (Unsloth)
- **Training framework:** TRL SFTTrainer
- **Task:** Decodable story generation
## Supported Phonics Patterns
- Short-vowel words
- Consonant blends
- Consonant digraphs
- Final-e words
- Vowel teams
- R-controlled vowels
- Diphthongs
- Multisyllabic words
## Evaluation
The model was evaluated against the base model using held-out prompts.
Evaluation consisted of:
- Objective phonics-pattern compliance metrics
- Blind LLM-as-a-judge scoring
- Qualitative error analysis
The fine-tuned model demonstrated improved adherence to the requested phonics behavior compared with the base model while continuing to exhibit limitations on more complex phonics combinations.
## Intended Use
This model is intended for:
- Educational NLP experiments
- Decodable story generation
- Small language model fine-tuning demonstrations
- Research on behavior-specific supervised fine-tuning
## Limitations
This is a research prototype and is not intended for instructional or production use.
Performance is stronger on simpler phonics patterns than on prompts requiring multiple or more complex phonics constraints. Additional curated training data would likely improve consistency.
## Example Prompt
```
Write a decodable story.
Emphasize these phonics patterns:
- final-e words
- consonant blends
Example words containing these patterns:
- final-e words: cake, game, bike, time, home
- consonant blends: stop, frog, black, plant, grin
Write a coherent narrative with a clear beginning, middle, and ending.
Include a natural story title.
Do not mention phonics terms or pattern names in the title or story.
``` |