Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,78 @@
+---
+base_model: TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T
+tags:
+- instruction-tuning
+- pharma
+- causal-lm
+- merged
+datasets:
+- ThakrePranjal/pharma-instruction-dataset
+---
+
+# Pharma TinyLlama — Instruction Merged Model (Stage 2)
+
+This is the **fully merged standalone instruction-tuned model**:
+`TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T` with both:
+
+1. Stage 1 domain LoRA merged in (pharma-domain pretraining)
+2. Stage 2 instruction LoRA merged in (pharma instruction tuning)
+
+No PEFT/LoRA library needed at inference time — load directly with 🤗 Transformers.
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+model     = AutoModelForCausalLM.from_pretrained("ThakrePranjal/pharma-tinyllama-instruct-merged")
+tokenizer = AutoTokenizer.from_pretrained("ThakrePranjal/pharma-tinyllama-instruct-merged")
+model.eval()
+
+def generate(instruction, input_text="", max_new_tokens=150):
+    if input_text.strip():
+        prompt = (
+            f"### Instruction:\n{instruction}\n\n"
+            f"### Input:\n{input_text}\n\n"
+            f"### Response:\n"
+        )
+    else:
+        prompt = (
+            f"### Instruction:\n{instruction}\n\n"
+            f"### Response:\n"
+        )
+    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+    with torch.no_grad():
+        out = model.generate(
+            **inputs,
+            max_new_tokens=max_new_tokens,
+            do_sample=True,
+            temperature=0.7,
+            top_p=0.9,
+            repetition_penalty=1.1,
+            pad_token_id=tokenizer.eos_token_id,
+        )
+    return tokenizer.decode(out[0], skip_special_tokens=True)
+
+print(generate("Explain the primary mechanism of action of metformin."))
+```
+
+## Full training pipeline
+
+```
+TinyLlama (base)
+    → Stage 1: Domain Pretraining LoRA    [ThakrePranjal/pharma-tinyllama-domain-lora]
+    → Merge  → Stage 1 Merged Model       [ThakrePranjal/pharma-tinyllama-domain-lora]
+    → Stage 2: Instruction LoRA           [ThakrePranjal/pharma-tinyllama-instruct-lora]
+    → Merge  → THIS MODEL (Stage 2 Merged)
+    → Stage 3: Preference Tuning (DPO)    (upcoming)
+```
+
+## Dataset
+
+[ThakrePranjal/pharma-instruction-dataset](https://huggingface.co/datasets/ThakrePranjal/pharma-instruction-dataset)
+
+## Limitations
+
+Trained on a small pharma corpus. Not validated for clinical or production use.
+Intended for educational/research purposes only.