Instructions to use SivaSai8143/pharma-tinyllama-dpo-merged with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use SivaSai8143/pharma-tinyllama-dpo-merged with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="SivaSai8143/pharma-tinyllama-dpo-merged")

# Load model directly
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("SivaSai8143/pharma-tinyllama-dpo-merged")
model = AutoModel.from_pretrained("SivaSai8143/pharma-tinyllama-dpo-merged", device_map="auto")

PEFT
How to use SivaSai8143/pharma-tinyllama-dpo-merged with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use SivaSai8143/pharma-tinyllama-dpo-merged with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SivaSai8143/pharma-tinyllama-dpo-merged"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SivaSai8143/pharma-tinyllama-dpo-merged",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/SivaSai8143/pharma-tinyllama-dpo-merged

SGLang

How to use SivaSai8143/pharma-tinyllama-dpo-merged with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "SivaSai8143/pharma-tinyllama-dpo-merged" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SivaSai8143/pharma-tinyllama-dpo-merged",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "SivaSai8143/pharma-tinyllama-dpo-merged" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SivaSai8143/pharma-tinyllama-dpo-merged",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use SivaSai8143/pharma-tinyllama-dpo-merged with Docker Model Runner:
```
docker model run hf.co/SivaSai8143/pharma-tinyllama-dpo-merged
```

pharma-tinyllama-dpo-merged

Model Summary

This is the Stage 3 merged model (final model) from the llm-finetuning-playbook pipeline.

It is produced by preference tuning (DPO) of the Stage-2 instruction-tuned model (SivaSai8143/pharma-tinyllama-instruction-merged) on prompt/chosen/rejected triples, using QLoRA (4-bit, nf4), then merging the LoRA adapter back into the base weights.

This is the final aligned model in the pipeline.

Pipeline Position

TinyLlama-1.1B (base)
        ↓  Stage 1: Non-Instruction FT
pharma-tinyllama-non-instruction-merged
        ↓  Stage 2: Instruction FT / SFT
pharma-tinyllama-instruction-merged
        ↓  Stage 3: Preference Tuning / DPO (this model)
pharma-tinyllama-dpo-merged  ← you are here

Training Details

Parameter	Value
Base model	SivaSai8143/pharma-tinyllama-instruction-merged
Method	QLoRA (4-bit nf4, double quant) + DPO
LoRA rank	16
LoRA alpha	32
LoRA dropout	0.05
Target modules	q/k/v/o_proj, gate/up/down_proj
beta	0.1
Max length	512 tokens
Max prompt length	256 tokens
Epochs	3
Max steps	5
Batch size	1 (grad accum 8, effective = 8)
Learning rate	5e-5
Warmup steps	2
Weight decay	0.01
ref_model	None (TRL handles reference policy internally)
Environment	Google Colab T4 GPU

Training Data

Trained on SivaSai8143/pharma-finetuning-data (config: preference).

48 prompt/chosen/rejected triples built on the same instruction prompts as Stage 2. chosen responses are accurate domain answers; rejected responses are plausible-sounding but factually wrong or off-target answers.

{
  "prompt": "### Instruction:\\nExplain the primary mechanism of action of metformin.\\n\\n### Response:\\n",
  "chosen": "Metformin primarily acts by activating AMP-activated protein kinase (AMPK)...",
  "rejected": "Metformin mainly works by increasing insulin secretion from the pancreas..."
}

Covering:

Metformin pharmacology, pharmacokinetics, safety & clinical use
Lipid-lowering therapy (Atorvastatin + Ezetimibe), familial hypercholesterolemia
mRNA vaccine platforms and immune response
AI in drug discovery, lead optimization, ADME/toxicology
Clinical trial terminology and pharmacovigilance

Related Artifacts

Artifact	Link
Stage 3 merged model	pharma-tinyllama-dpo-merged
Stage 2 merged model	pharma-tinyllama-instruction-merged
Stage 1 merged model	pharma-tinyllama-non-instruction-merged
Training notebook	llm-finetuning-playbook
Dataset	pharma-finetuning-data

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "SivaSai8143/pharma-tinyllama-dpo-merged"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

prompt = \"\"\"### Instruction:
Explain the primary mechanism of action of metformin.

### Response:
\"\"\"

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=150, do_sample=True, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Disclaimer

Educational fine-tuning project for demonstrating LLM training pipelines. The pharma content is for technical demonstration only and is not medical advice. """

Downloads last month: 41

Safetensors

Model size

1B params

Tensor type

F16

Model tree for SivaSai8143/pharma-tinyllama-dpo-merged

Base model

TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T

Adapter

SivaSai8143/pharma-tinyllama-non-instruction-merged

Adapter

SivaSai8143/pharma-tinyllama-instruction-merged

Adapter

(1)

this model