Instructions to use pjspjs0987/Qwen2.5-7B-GrapheneContact-QDoRA-CoT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use pjspjs0987/Qwen2.5-7B-GrapheneContact-QDoRA-CoT with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
model = PeftModel.from_pretrained(base_model, "pjspjs0987/Qwen2.5-7B-GrapheneContact-QDoRA-CoT")

Transformers

How to use pjspjs0987/Qwen2.5-7B-GrapheneContact-QDoRA-CoT with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="pjspjs0987/Qwen2.5-7B-GrapheneContact-QDoRA-CoT")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("pjspjs0987/Qwen2.5-7B-GrapheneContact-QDoRA-CoT", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use pjspjs0987/Qwen2.5-7B-GrapheneContact-QDoRA-CoT with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "pjspjs0987/Qwen2.5-7B-GrapheneContact-QDoRA-CoT"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "pjspjs0987/Qwen2.5-7B-GrapheneContact-QDoRA-CoT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/pjspjs0987/Qwen2.5-7B-GrapheneContact-QDoRA-CoT

SGLang

How to use pjspjs0987/Qwen2.5-7B-GrapheneContact-QDoRA-CoT with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "pjspjs0987/Qwen2.5-7B-GrapheneContact-QDoRA-CoT" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "pjspjs0987/Qwen2.5-7B-GrapheneContact-QDoRA-CoT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "pjspjs0987/Qwen2.5-7B-GrapheneContact-QDoRA-CoT" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "pjspjs0987/Qwen2.5-7B-GrapheneContact-QDoRA-CoT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use pjspjs0987/Qwen2.5-7B-GrapheneContact-QDoRA-CoT with Docker Model Runner:
```
docker model run hf.co/pjspjs0987/Qwen2.5-7B-GrapheneContact-QDoRA-CoT
```

Qwen2.5-7B-GrapheneContact-QDoRA-CoT

A domain-specific language model fine-tuned for condensed matter physics, specifically graphene–metal contact resistance research. This 7B model, trained on only 64 question–answer pairs, outperforms a 235B-parameter model with open-book access under closed-book evaluation conditions.

Key Result

Model	Parameters	Condition	Correct (✓)	Partial (△)	Wrong (✗)	Accuracy
Base 7B	7B	Open-book	9	17	4	30%
FT 7B (this model)	7B	Closed-book	22	5	3	73%
Qwen3-235B	235B	Open-book	18	9	3	60%

A fine-tuned 7B model with no access to the source paper outperforms a 33× larger model that receives the paper's content as a prompt.

Model Details

Model Description

Developed by: Junsu Park
Affiliation: Department of Physics and Photon Science, Gwangju Institute of Science and Technology (GIST), Korea
Base model: Qwen/Qwen2.5-7B-Instruct
Fine-tuning method: QDoRA (Quantized Weight-Decomposed Low-Rank Adaptation) + Chain-of-Thought reasoning
Language: English
License: Apache 2.0
Domain: Condensed matter physics — graphene–metal contact resistance (TLCD/TECD devices)

Model Sources

Repository: GitHub (coming soon)
Research Article: "Small Data, Small Model, Superior Domain Accuracy: Fine-Tuning a 7B Language Model as a Condensed Matter Physics Expert with 64 Examples"(https://www.preprints.org/manuscript/202603.1691)

Uses

Direct Use

This model is designed to answer domain-specific questions about graphene–metal contact resistance, including device structure (TLCD/TECD), fabrication processes, theoretical models (Landauer, BTH), and experimental results. It serves as a personalized research assistant for this specific domain.

Out-of-Scope Use

This model is trained on a single research paper and should not be used for general scientific reasoning, other physics domains, or any task outside graphene–metal contact resistance research.

Training Details

Training Data

64 expert-crafted question–answer pairs from a single condensed matter physics paper, organized into five categories:

Category	Description
Structure / Geometry	TLCD vs TECD device structure, contact dimensions
Theory / Mechanism	Landauer model, BTH model, current crowding
Numerical / Experimental	Contact resistance values, TLM measurements
Fabrication	Metal-on-bottom process, Cr/Au deposition
Novelty / Contribution	First experimental verification of area-dependent Rc

Each answer includes Chain-of-Thought (CoT) reasoning with <think> and <final> tags.

Training Hyperparameters

Parameter	Value
Adapter	QDoRA (`use_dora=True`)
Rank (r)	64
Alpha (α)	128
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Quantization	4-bit NF4, double quantization, bfloat16 compute
Max steps	80
Learning rate	5×10⁻⁴
Batch size	1 (gradient accumulation = 8, effective batch = 8)
Max sequence length	2048
Optimizer	paged_adamw_8bit
Training time	~20 minutes
Training regime	bf16 mixed precision

Hardware

Component	Specification
GPU	NVIDIA RTX 5090 (32 GB VRAM)
CPU	AMD Threadripper PRO 5975WX (32 cores)
RAM	256 GB DDR4-3200 ECC
OS	Ubuntu

Evaluation

30 expert-designed questions (distinct from training set) evaluated by the domain expert (first author of the source paper) using a three-tier rubric: ✓ (correct), △ (partial), ✗ (wrong).

Results by Category

Category	Questions	Base 7B	FT 7B	235B
Structure / Geometry	4	1	3	3
Theory / Mechanism	10	4	8	7
Numerical / Experimental	6	2	5	4
Fabrication	6	1	4	2
Novelty / Contribution	4	1	2	2

Error Pattern Analysis

Each model exhibits qualitatively distinct failure modes:

Base 7B: Dominated by partial coverage (PC) — generates relevant but insufficiently specific responses
235B: Dominated by over-extrapolation (OE) — fills knowledge gaps with plausible but incorrect scientific details
FT 7B (this model): Occasional directional reversal (DR) — correctly identifies relevant concepts but sometimes inverts their relationships

How to Get Started with the Model

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-7B-Instruct",
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(
    "pjspjs0987/Qwen2.5-7B-GrapheneContact-QDoRA-CoT"
)

# Load fine-tuned adapter
model = PeftModel.from_pretrained(
    base_model,
    "pjspjs0987/Qwen2.5-7B-GrapheneContact-QDoRA-CoT"
)

# Inference
messages = [
    {"role": "user", "content": "Why does increasing contact area reduce contact resistance in graphene-metal interfaces?"}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, top_p=0.9)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Citation

@article{park2026small,
  title={Small Data, Small Model, Superior Domain Accuracy: Fine-Tuning a 7B Language Model as a Condensed Matter Physics Expert with 64 Examples},
  author={Park, Junsu},
  year={2026},
  institution={Gwangju Institute of Science and Technology (GIST)}
}

Contact

Author: Junsu Park
Affiliation: GIST, Department of Physics and Photon Science
Hugging Face: pjspjs0987

Framework versions

PEFT 0.18.0

Downloads last month: 11

Model tree for pjspjs0987/Qwen2.5-7B-GrapheneContact-QDoRA-CoT

Base model

Qwen/Qwen2.5-7B

Finetuned

Qwen/Qwen2.5-7B-Instruct

Adapter

(2776)

this model