Instructions to use shounakpatra/Llama-3.1-8B-Refined-Codex-5.5 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use shounakpatra/Llama-3.1-8B-Refined-Codex-5.5 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="shounakpatra/Llama-3.1-8B-Refined-Codex-5.5")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("shounakpatra/Llama-3.1-8B-Refined-Codex-5.5")
model = AutoModelForCausalLM.from_pretrained("shounakpatra/Llama-3.1-8B-Refined-Codex-5.5", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

PEFT
How to use shounakpatra/Llama-3.1-8B-Refined-Codex-5.5 with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use shounakpatra/Llama-3.1-8B-Refined-Codex-5.5 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf shounakpatra/Llama-3.1-8B-Refined-Codex-5.5:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf shounakpatra/Llama-3.1-8B-Refined-Codex-5.5:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf shounakpatra/Llama-3.1-8B-Refined-Codex-5.5:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf shounakpatra/Llama-3.1-8B-Refined-Codex-5.5:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf shounakpatra/Llama-3.1-8B-Refined-Codex-5.5:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf shounakpatra/Llama-3.1-8B-Refined-Codex-5.5:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf shounakpatra/Llama-3.1-8B-Refined-Codex-5.5:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf shounakpatra/Llama-3.1-8B-Refined-Codex-5.5:Q4_K_M

Use Docker

docker model run hf.co/shounakpatra/Llama-3.1-8B-Refined-Codex-5.5:Q4_K_M

LM Studio
Jan

vLLM

How to use shounakpatra/Llama-3.1-8B-Refined-Codex-5.5 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "shounakpatra/Llama-3.1-8B-Refined-Codex-5.5"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "shounakpatra/Llama-3.1-8B-Refined-Codex-5.5",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/shounakpatra/Llama-3.1-8B-Refined-Codex-5.5:Q4_K_M

SGLang

How to use shounakpatra/Llama-3.1-8B-Refined-Codex-5.5 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "shounakpatra/Llama-3.1-8B-Refined-Codex-5.5" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "shounakpatra/Llama-3.1-8B-Refined-Codex-5.5",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "shounakpatra/Llama-3.1-8B-Refined-Codex-5.5" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "shounakpatra/Llama-3.1-8B-Refined-Codex-5.5",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use shounakpatra/Llama-3.1-8B-Refined-Codex-5.5 with Ollama:
```
ollama run hf.co/shounakpatra/Llama-3.1-8B-Refined-Codex-5.5:Q4_K_M
```
Unsloth Desktop

How to use shounakpatra/Llama-3.1-8B-Refined-Codex-5.5 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf shounakpatra/Llama-3.1-8B-Refined-Codex-5.5:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "shounakpatra/Llama-3.1-8B-Refined-Codex-5.5:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use shounakpatra/Llama-3.1-8B-Refined-Codex-5.5 with Docker Model Runner:
```
docker model run hf.co/shounakpatra/Llama-3.1-8B-Refined-Codex-5.5:Q4_K_M
```

Lemonade

How to use shounakpatra/Llama-3.1-8B-Refined-Codex-5.5 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull shounakpatra/Llama-3.1-8B-Refined-Codex-5.5:Q4_K_M

Run and chat with the model

lemonade run user.Llama-3.1-8B-Refined-Codex-5.5-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use shounakpatra/Llama-3.1-8B-Refined-Codex-5.5 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf shounakpatra/Llama-3.1-8B-Refined-Codex-5.5:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default shounakpatra/Llama-3.1-8B-Refined-Codex-5.5:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use shounakpatra/Llama-3.1-8B-Refined-Codex-5.5 with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf shounakpatra/Llama-3.1-8B-Refined-Codex-5.5:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "shounakpatra/Llama-3.1-8B-Refined-Codex-5.5:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Llama-3.1-8B-Refined-Codex-5.5

About

Llama-3.1-8B-Refined-Codex-5.5 is an experimental assistant-focused fine-tune built using Unsloth, QLoRA/LoRA, PEFT, Transformers, and TRL.

This is a LoRA/QLoRA fine-tuned derivative of Llama 3.1 8B Instruct, optimized for general assistant behavior, instruction following, explanations, coding help, study support, and RAG-based knowledge augmentation.

This is not an official Meta, OpenAI, or Codex model. It is an independent fine-tuned derivative prepared for educational use, prototyping, and deployment experiments.

Creator

Developed by: Shounak Patra
GitHub: ShounakPatra
Hugging Face: shounakpatra

Model Details

Field	Value
Model name	Llama-3.1-8B-Refined-Codex-5.5
Base model	meta-llama/Llama-3.1-8B-Instruct
Training base	unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit
Fine-tuning method	LoRA / QLoRA
Training framework	Unsloth, PEFT, Transformers, TRL
Training hardware	Google Colab T4 GPU
Model type	Causal language model
Primary use	Assistant-style text generation
Version	5.5

Repository Layout

repo root/
  config.json
  generation_config.json
  model-00001-of-00004.safetensors
  model-00002-of-00004.safetensors
  model-00003-of-00004.safetensors
  model-00004-of-00004.safetensors
  model.safetensors.index.json
  tokenizer.json
  tokenizer_config.json
  README.md

  adapter/
    adapter_config.json
    adapter_model.safetensors

  gguf/
    Llama-3.1-8B-Refined-Codex-5.5-Q4_K_M.gguf

Root files contain the full merged Transformers model.
adapter/ contains the LoRA adapter.
gguf/ contains the downloadable local CPU/GPU version for llama.cpp, LM Studio, Jan, Ollama, and other GGUF-compatible apps.

Intended Use

This model is intended for:

General assistant-style conversation
Instruction following
Explanations and tutoring-style responses
Coding help and debugging support
Study support and summarization
RAG-based question answering
Chatbot prototyping with Gradio or Hugging Face Spaces
Local CPU/GPU inference experiments

Downloadable GGUF Version

This repository includes a Q4_K_M GGUF build for local inference.

GGUF file:

gguf/Llama-3.1-8B-Refined-Codex-5.5-Q4_K_M.gguf

Approximate size:

4.92 GB

Recommended for:

Local CPU inference
Low-VRAM GPU inference
Ollama
llama.cpp
LM Studio
Jan
KoboldCpp
Other GGUF-compatible runtimes

Run Locally With Ollama

Ollama can import GGUF models using a Modelfile.

Official Ollama docs:

Step 1: Install Ollama

Download and install Ollama from:

https://ollama.com/download

After installing, open a terminal and check:

ollama --version

Step 2: Download The GGUF File

Go to this repository:

https://huggingface.co/shounakpatra/Llama-3.1-8B-Refined-Codex-5.5

Then open:

Files and versions -> gguf

Download:

Llama-3.1-8B-Refined-Codex-5.5-Q4_K_M.gguf

If this repository is gated or private, you must be logged in to Hugging Face and have access before downloading.

Step 3: Create A Folder For The Model

Example on Windows:

C:\Users\YourName\llama-refined-codex

Example on macOS/Linux:

~/llama-refined-codex

Put this file inside that folder:

Llama-3.1-8B-Refined-Codex-5.5-Q4_K_M.gguf

Step 4: Create A Modelfile

Inside the same folder, create a file named exactly:

Modelfile

Important: the file should be named Modelfile, not Modelfile.txt.

Paste this inside:

FROM ./Llama-3.1-8B-Refined-Codex-5.5-Q4_K_M.gguf

SYSTEM """You are Llama-3.1-8B-Refined-Codex-5.5, a helpful, clear, honest AI assistant. You help with explanations, coding, study support, writing, brainstorming, and general problem solving. If you are unsure, say so clearly. You do not claim live internet access unless a web-search tool is connected."""

PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER num_ctx 2048

Step 5: Create The Ollama Model

Open a terminal in the folder containing the GGUF file and Modelfile.

Windows example:

cd C:\Users\YourName\llama-refined-codex

macOS/Linux example:

cd ~/llama-refined-codex

Create the Ollama model:

ollama create llama-refined-codex-5.5 -f Modelfile

Step 6: Run The Model

ollama run llama-refined-codex-5.5

Try:

Hi, who are you?

Or:

Explain black holes simply.

Step 7: Stop Chatting

Type:

/bye

Run With llama.cpp

If you have llama.cpp installed:

llama-cli -hf shounakpatra/Llama-3.1-8B-Refined-Codex-5.5:gguf/Llama-3.1-8B-Refined-Codex-5.5-Q4_K_M.gguf -p "Hello, who are you?"

Run With LM Studio

Open LM Studio.
Search for:

shounakpatra/Llama-3.1-8B-Refined-Codex-5.5

Download the Q4_K_M GGUF file.
Load the model.
Start chatting.

Run Full Merged Model With Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo_id = "shounakpatra/Llama-3.1-8B-Refined-Codex-5.5"

tokenizer = AutoTokenizer.from_pretrained(repo_id)

model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    device_map="auto",
    torch_dtype="auto",
)

prompt = "Explain black holes in simple terms."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=300,
    temperature=0.7,
    do_sample=True,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Run LoRA Adapter

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = "meta-llama/Llama-3.1-8B-Instruct"
repo_id = "shounakpatra/Llama-3.1-8B-Refined-Codex-5.5"

tokenizer = AutoTokenizer.from_pretrained(repo_id, subfolder="adapter")

model = AutoModelForCausalLM.from_pretrained(
    base_model,
    device_map="auto",
    torch_dtype="auto",
)

model = PeftModel.from_pretrained(
    model,
    repo_id,
    subfolder="adapter",
)

Recommended Architecture

Fine-tuned LoRA / merged model
+ FAISS RAG corpus
+ optional web-search API
+ Gradio or Hugging Face Space frontend

RAG and web search should be added at the application layer. The model itself does not permanently store external documents or browse the internet.

RAG Recommendation

For factual or private knowledge, use retrieval-augmented generation:

User question
-> search FAISS/vector database
-> retrieve relevant chunks
-> insert chunks into prompt
-> model answers using provided context

This is recommended for private TXT, PDF, website, or documentation knowledge bases.

Web Browsing Note

This model does not have built-in live internet access.

To answer current questions, connect it to a web-search API such as:

Tavily
Brave Search API
SerpAPI
Bing Web Search
Google Custom Search

Web-search RAG should be handled by the app, API server, or Space frontend.

Example Prompts

Hi, who are you?
Explain black holes in simple terms.
Write a Python function to clean a CSV file.
Summarize this paragraph for a class 10 student.
Help me debug this error message.
Create a study plan for learning machine learning.
Answer using only the retrieved RAG context: ...

Limitations

May hallucinate.
Not always factually correct.
No built-in real-time internet access.
Knowledge depends on the base model and any provided RAG corpus.
Quality depends on prompt formatting and retrieval quality.
Local CPU inference may be slow, especially on older processors.
Not suitable for medical, legal, or financial decisions without expert review.
May reflect limitations or biases from the base model or training data.

Safety Note

Use this model responsibly. Validate important outputs before relying on them. For high-stakes domains, use expert review and additional safety layers.

License / Attribution

Usage must follow the license terms of the base Llama 3.1 model and any datasets used during fine-tuning.

This model is not affiliated with or endorsed by Meta, OpenAI, or Codex.

Version Information

Version: 5.5
Release name: Llama-3.1-8B-Refined-Codex-5.5
Fine-tuning stack: Unsloth + QLoRA/LoRA + PEFT + Transformers + TRL
Training environment: Google Colab T4 GPU
Local runtime format: GGUF Q4_K_M

Disclaimer

This repository is provided for educational and experimental use. It does not claim parity with commercial systems such as ChatGPT, Gemini, or Claude, and it should not be treated as a source of guaranteed factual truth.

Downloads last month: 21

Safetensors

Model size

8B params

Tensor type

BF16

Model tree for shounakpatra/Llama-3.1-8B-Refined-Codex-5.5

Base model

meta-llama/Llama-3.1-8B

Finetuned

meta-llama/Llama-3.1-8B-Instruct

Adapter

(2901)

this model

shounakpatra
/

Llama-3.1-8B-Refined-Codex-5.5

You need to agree to share your contact information to access this model

Llama-3.1-8B-Refined-Codex-5.5

About

Creator

Model Details

Repository Layout

Intended Use

Downloadable GGUF Version

Run Locally With Ollama

Step 1: Install Ollama

Step 2: Download The GGUF File

Step 3: Create A Folder For The Model

Step 4: Create A Modelfile

Step 5: Create The Ollama Model

Step 6: Run The Model

Step 7: Stop Chatting

Run With llama.cpp

Run With LM Studio

Run Full Merged Model With Transformers

Run LoRA Adapter

Recommended Architecture

RAG Recommendation

Web Browsing Note

Example Prompts

Limitations

Safety Note

License / Attribution

Version Information

Disclaimer

Model tree for shounakpatra/Llama-3.1-8B-Refined-Codex-5.5

Spaces using shounakpatra/Llama-3.1-8B-Refined-Codex-5.5 2