Instructions to use SC117/Ornith-1.0-35B-MTP-APEX-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use SC117/Ornith-1.0-35B-MTP-APEX-GGUF with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="SC117/Ornith-1.0-35B-MTP-APEX-GGUF")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("SC117/Ornith-1.0-35B-MTP-APEX-GGUF", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use SC117/Ornith-1.0-35B-MTP-APEX-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SC117/Ornith-1.0-35B-MTP-APEX-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SC117/Ornith-1.0-35B-MTP-APEX-GGUF",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/SC117/Ornith-1.0-35B-MTP-APEX-GGUF

SGLang

How to use SC117/Ornith-1.0-35B-MTP-APEX-GGUF with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "SC117/Ornith-1.0-35B-MTP-APEX-GGUF" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SC117/Ornith-1.0-35B-MTP-APEX-GGUF",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "SC117/Ornith-1.0-35B-MTP-APEX-GGUF" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SC117/Ornith-1.0-35B-MTP-APEX-GGUF",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use SC117/Ornith-1.0-35B-MTP-APEX-GGUF with Docker Model Runner:
```
docker model run hf.co/SC117/Ornith-1.0-35B-MTP-APEX-GGUF
```

SC117 commited on 5 days ago

Commit

aa79a3f

verified ·

1 Parent(s): fba90de

Upload 2 files

Browse files

Files changed (2) hide show

README.md +2 -2
README_zh.md +2 -2

README.md CHANGED Viewed

@@ -28,7 +28,7 @@ base_model:
 <span style="background: #ff9500; color: white; font-size: 11px; font-weight: 600; padding: 5px 14px; border-radius: 20px;">MIT</span>
 </div>
 <h1 style="margin: 0 0 8px 0; font-size: 32px; font-weight: 700; color: #1d1d1f; letter-spacing: -0.5px; border: none; position: relative; z-index: 1;">Ornith-1.0-35B-MTP</h1>
-<p style="margin: 8px 0 0 0; font-size: 14px; position: relative; z-index: 1;"><span style="color: #86868b;">English</span> | <a href="https://huggingface.co/SC117/Ornith-1.0-35B-MTP-GGUF/blob/main/README_zh.md" style="color: #007aff; text-decoration: none;">📖 中文文档</a></p>
 <p style="margin: 0; font-size: 15px; color: #86868b; position: relative; z-index: 1;">Self-improving agentic coding model · APEX quantized GGUFs + BF16 + mmproj</p>
 </div>
 </div>
@@ -74,7 +74,7 @@ base_model:
 <div style="background: linear-gradient(135deg, #ff6b35 0%, #f7931e 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>🚀</span> Usage</div>
 <div style="padding: 16px; font-size: 13px; color: #334155; line-height: 1.7;">
 <p style="margin: 0 0 8px 0; font-weight: bold; color: #1e293b;">llama.cpp (text only)</p>
-<p style="margin: 0; font-family: monospace; background: #f8fafc; padding: 10px 14px; border-radius: 6px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b; white-space: pre-wrap;">hf download SC117/Ornith-1.0-35B-MTP-GGUF --include "*.gguf" --local-dir ./models
 ./llama-server -m ./models/Ornith-1.0-35B-MTP-APEX-I-Compact.gguf -ngl 99 -c 131072</p>
 <p style="margin: 12px 0 8px 0; font-weight: bold; color: #1e293b;">llama.cpp (vision + text)</p>
 <p style="margin: 0; font-family: monospace; background: #f8fafc; padding: 10px 14px; border-radius: 6px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b; white-space: pre-wrap;">./llama-server -m ./models/Ornith-1.0-35B-MTP-APEX-I-Compact.gguf --mmproj ./models/mmproj-F16.gguf -ngl 99 -c 131072</p>

 <span style="background: #ff9500; color: white; font-size: 11px; font-weight: 600; padding: 5px 14px; border-radius: 20px;">MIT</span>
 </div>
 <h1 style="margin: 0 0 8px 0; font-size: 32px; font-weight: 700; color: #1d1d1f; letter-spacing: -0.5px; border: none; position: relative; z-index: 1;">Ornith-1.0-35B-MTP</h1>
+<p style="margin: 8px 0 0 0; font-size: 14px; position: relative; z-index: 1;"><span style="color: #86868b;">English</span> | <a href="https://huggingface.co/SC117/Ornith-1.0-35B-MTP-APEX-GGUF/blob/main/README_zh.md" style="color: #007aff; text-decoration: none;">📖 中文文档</a></p>
 <p style="margin: 0; font-size: 15px; color: #86868b; position: relative; z-index: 1;">Self-improving agentic coding model · APEX quantized GGUFs + BF16 + mmproj</p>
 </div>
 </div>
 <div style="background: linear-gradient(135deg, #ff6b35 0%, #f7931e 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>🚀</span> Usage</div>
 <div style="padding: 16px; font-size: 13px; color: #334155; line-height: 1.7;">
 <p style="margin: 0 0 8px 0; font-weight: bold; color: #1e293b;">llama.cpp (text only)</p>
+<p style="margin: 0; font-family: monospace; background: #f8fafc; padding: 10px 14px; border-radius: 6px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b; white-space: pre-wrap;">hf download SC117/Ornith-1.0-35B-MTP-APEX-GGUF --include "*.gguf" --local-dir ./models
 ./llama-server -m ./models/Ornith-1.0-35B-MTP-APEX-I-Compact.gguf -ngl 99 -c 131072</p>
 <p style="margin: 12px 0 8px 0; font-weight: bold; color: #1e293b;">llama.cpp (vision + text)</p>
 <p style="margin: 0; font-family: monospace; background: #f8fafc; padding: 10px 14px; border-radius: 6px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b; white-space: pre-wrap;">./llama-server -m ./models/Ornith-1.0-35B-MTP-APEX-I-Compact.gguf --mmproj ./models/mmproj-F16.gguf -ngl 99 -c 131072</p>

README_zh.md CHANGED Viewed

@@ -28,7 +28,7 @@ base_model:
 <span style="background: #ff9500; color: white; font-size: 11px; font-weight: 600; padding: 5px 14px; border-radius: 20px;">MIT</span>
 </div>
 <h1 style="margin: 0 0 8px 0; font-size: 32px; font-weight: 700; color: #1d1d1f; letter-spacing: -0.5px; border: none; position: relative; z-index: 1;">Ornith-1.0-35B-MTP</h1>
-<p style="margin: 8px 0 0 0; font-size: 14px; position: relative; z-index: 1;"><a href="https://huggingface.co/SC117/Ornith-1.0-35B-MTP-GGUF/blob/main/README.md" style="color: #007aff; text-decoration: none;">📖 English</a> | <span style="color: #86868b;">中文文档</span></p>
 <p style="margin: 0; font-size: 15px; color: #86868b; position: relative; z-index: 1;">自改进 agentic coding 推理模型 · APEX 量化 GGUF + BF16 + mmproj</p>
 </div>
 </div>
@@ -74,7 +74,7 @@ base_model:
 <div style="background: linear-gradient(135deg, #ff6b35 0%, #f7931e 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>🚀</span> 使用方法</div>
 <div style="padding: 16px; font-size: 13px; color: #334155; line-height: 1.7;">
 <p style="margin: 0 0 8px 0; font-weight: bold; color: #1e293b;">llama.cpp（纯文本）</p>
-<p style="margin: 0; font-family: monospace; background: #f8fafc; padding: 10px 14px; border-radius: 6px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b; white-space: pre-wrap;">hf download SC117/Ornith-1.0-35B-MTP-GGUF --include "*.gguf" --local-dir ./models
 ./llama-server -m ./models/Ornith-1.0-35B-MTP-APEX-I-Compact.gguf -ngl 99 -c 131072</p>
 <p style="margin: 12px 0 8px 0; font-weight: bold; color: #1e293b;">llama.cpp（视觉 + 文本）</p>
 <p style="margin: 0; font-family: monospace; background: #f8fafc; padding: 10px 14px; border-radius: 6px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b; white-space: pre-wrap;">./llama-server -m ./models/Ornith-1.0-35B-MTP-APEX-I-Compact.gguf --mmproj ./models/mmproj-F16.gguf -ngl 99 -c 131072</p>

 <span style="background: #ff9500; color: white; font-size: 11px; font-weight: 600; padding: 5px 14px; border-radius: 20px;">MIT</span>
 </div>
 <h1 style="margin: 0 0 8px 0; font-size: 32px; font-weight: 700; color: #1d1d1f; letter-spacing: -0.5px; border: none; position: relative; z-index: 1;">Ornith-1.0-35B-MTP</h1>
+<p style="margin: 8px 0 0 0; font-size: 14px; position: relative; z-index: 1;"><a href="https://huggingface.co/SC117/Ornith-1.0-35B-MTP-APEX-GGUF/blob/main/README.md" style="color: #007aff; text-decoration: none;">📖 English</a> | <span style="color: #86868b;">中文文档</span></p>
 <p style="margin: 0; font-size: 15px; color: #86868b; position: relative; z-index: 1;">自改进 agentic coding 推理模型 · APEX 量化 GGUF + BF16 + mmproj</p>
 </div>
 </div>
 <div style="background: linear-gradient(135deg, #ff6b35 0%, #f7931e 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>🚀</span> 使用方法</div>
 <div style="padding: 16px; font-size: 13px; color: #334155; line-height: 1.7;">
 <p style="margin: 0 0 8px 0; font-weight: bold; color: #1e293b;">llama.cpp（纯文本）</p>
+<p style="margin: 0; font-family: monospace; background: #f8fafc; padding: 10px 14px; border-radius: 6px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b; white-space: pre-wrap;">hf download SC117/Ornith-1.0-35B-MTP-APEX-GGUF --include "*.gguf" --local-dir ./models
 ./llama-server -m ./models/Ornith-1.0-35B-MTP-APEX-I-Compact.gguf -ngl 99 -c 131072</p>
 <p style="margin: 12px 0 8px 0; font-weight: bold; color: #1e293b;">llama.cpp（视觉 + 文本）</p>
 <p style="margin: 0; font-family: monospace; background: #f8fafc; padding: 10px 14px; border-radius: 6px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b; white-space: pre-wrap;">./llama-server -m ./models/Ornith-1.0-35B-MTP-APEX-I-Compact.gguf --mmproj ./models/mmproj-F16.gguf -ngl 99 -c 131072</p>