deadbear34/qwen35-4b-plantdisease-sft-lora

Qwen3.5-4B Plant Disease — LoRA SFT Adapter (v2)

LoRA adapter for thinking-style reasoning + plant disease domain expertise.

Base Model

• deadbear34/qwen35-4b-plantdisease-cpt (CPT result on plant disease corpus)

Training Data

• plant-disease-qa-bilingual: ~13k samples (in-domain)
• claude-sonnet-4.6-100000X-filtered: ~35k samples (general thinking)
• claude-sonnet-4.6-120000x: ~35k samples (diverse difficulties)
• thinking-dataset-en-v2: ~75k samples (reasoning patterns)

LoRA Configuration

• Rank: 256
• Alpha: 256
• Target modules: ['q_proj', 'k_proj', 'v_proj', 'o_proj', 'gate_proj', 'up_proj', 'down_proj']
• Dropout: 0.05

Training Configuration

• Optimizer: 8-bit AdamW (lr=0.0001)
• Schedule: cosine warmup (3.0%)
• Effective batch: 4
• Sequence length: 2048

Evaluation

• Final val PPL: {FINAL_VAL_PPL}
• Best val PPL: {BEST_VAL_PPL} at step {BEST_STEP}

Format

<|user|>
{user_query}
<|assistant|>
<thinking>
{reasoning}
</thinking>
{final_answer}

Usage

from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoTokenizer

base = AutoModelForImageTextToText.from_pretrained('deadbear34/qwen35-4b-plantdisease-cpt', torch_dtype='bfloat16')
model = PeftModel.from_pretrained(base, 'deadbear34/qwen35-4b-plantdisease-sft-lora')
tokenizer = AutoTokenizer.from_pretrained('deadbear34/qwen35-4b-plantdisease-sft-lora')

prompt = '<|user|>
What causes late blight?
<|assistant|>
'
inputs = tokenizer(prompt, return_tensors='pt').to('cuda')
out = model.generate(**inputs, max_new_tokens=300)
print(tokenizer.decode(out[0]))

---

Trained: 2026-05-10