Instructions to use pearl-ai/Llama-3.3-70B-Instruct-pearl with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use pearl-ai/Llama-3.3-70B-Instruct-pearl with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="pearl-ai/Llama-3.3-70B-Instruct-pearl")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("pearl-ai/Llama-3.3-70B-Instruct-pearl")
model = AutoModelForCausalLM.from_pretrained("pearl-ai/Llama-3.3-70B-Instruct-pearl", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use pearl-ai/Llama-3.3-70B-Instruct-pearl with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "pearl-ai/Llama-3.3-70B-Instruct-pearl"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "pearl-ai/Llama-3.3-70B-Instruct-pearl",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/pearl-ai/Llama-3.3-70B-Instruct-pearl

SGLang

How to use pearl-ai/Llama-3.3-70B-Instruct-pearl with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "pearl-ai/Llama-3.3-70B-Instruct-pearl" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "pearl-ai/Llama-3.3-70B-Instruct-pearl",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "pearl-ai/Llama-3.3-70B-Instruct-pearl" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "pearl-ai/Llama-3.3-70B-Instruct-pearl",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use pearl-ai/Llama-3.3-70B-Instruct-pearl with Docker Model Runner:
```
docker model run hf.co/pearl-ai/Llama-3.3-70B-Instruct-pearl
```

Llama-3.3-70B-Instruct-pearl / README.md

netanell-pearl

Add public-safe model card with Pearl vLLM usage

6cc401c verified 4 months ago

preview code

Raw

History Blame Contribute Delete

3.53 kB

metadata

language:
  - en
license: llama3.3
library_name: transformers
pipeline_tag: text-generation
tags:
  - pearl
  - llama
  - llama-3.3
  - instruct
  - large-language-model
  - quantization
  - vllm
  - mining
base_model: meta-llama/Llama-3.3-70B-Instruct

pearl-ai/Llama-3.3-70B-Instruct-pearl

Pearl-certified variant of Llama-3.3-70B-Instruct, intended to run with the Pearl vLLM mining plugin.

Project website: https://pearlresearch.ai
Pearl repository: https://github.com/pearl-research-labs/pearl
Miner docs: https://github.com/pearl-research-labs/pearl/tree/master/miner

Launch Benchmark

Original (Meta's) llama-3.3-70B-Instruct vs. our "two-for-one" Pearl-certified variant. Both executions were done with 4xH200 GPUs. We explore several parallelism techniques. TMADs, i.e., Tera MADs, is a metric counting number of Multiply-Add (MAD) operations. Useful MADs is the total number of MAD operations done anyway that are used for mining.

Model	Parallelism	Score (MMLU)	Throughput (tok/sec)	Time (sec)	Useful MADs (TMADs/sec)
Meta's LLaMA 70B	PP=4	0.8198	15,269.81	441.100	-
Meta's LLaMA 70B	TP=4	0.8193	13,218	510	-
Meta's LLaMA 70B	DP=2, TP=2	0.8197	13,162	512	-
Meta's LLaMA 70B (DP=4): OOM - bf16 model (~140 GB) exceeds single GPU VRAM
Pearl-certified	PP=4	0.8190	17,206.26	391.457	806
Pearl-certified	TP=4	0.8180	13,264.38	507.789	620
Pearl-certified	DP=4	0.8198	18,291.66	368.229	981

How To Use (Pearl vLLM Plugin)

This model is intended to be served through the Pearl miner stack, where vLLM inference is integrated with Pearl mining workflows.

Typical flow:

Run pearld with RPC enabled.
Start the Pearl miner/vLLM stack.
Serve this model through vLLM while Pearl gateway/miner components handle mining-side integration.

High-level prerequisites:

Python 3.12
uv
CUDA + NVIDIA GPU (sm90 class, e.g. H100/H200, per project docs)
Rust toolchain
Running pearld node with RPC credentials

Docker Example

From the Pearl repository root:

docker buildx build -t vllm_miner . -f miner/vllm-miner/Dockerfile

docker run --rm -it --gpus all \
  -p 8000:8000 -p 8337:8337 -p 8339:8339 \
  -e PEARLD_RPC_URL=<PEARLD_URL> \
  -e PEARLD_RPC_USER=<RPC_USER> \
  -e PEARLD_RPC_PASSWORD=<RPC_PASSWORD> \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  --shm-size 8g \
  vllm_miner:latest \
  pearl-ai/Llama-3.3-70B-Instruct-pearl \
  --host 0.0.0.0 --port 8000 \
  --max-model-len 8192 \
  --gpu-memory-utilization 0.9 \
  --enforce-eager