Instructions to use AbteeXAILab/lumynax-chat-yi-15-34b-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use AbteeXAILab/lumynax-chat-yi-15-34b-gguf with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AbteeXAILab/lumynax-chat-yi-15-34b-gguf:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf AbteeXAILab/lumynax-chat-yi-15-34b-gguf:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AbteeXAILab/lumynax-chat-yi-15-34b-gguf:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf AbteeXAILab/lumynax-chat-yi-15-34b-gguf:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf AbteeXAILab/lumynax-chat-yi-15-34b-gguf:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf AbteeXAILab/lumynax-chat-yi-15-34b-gguf:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf AbteeXAILab/lumynax-chat-yi-15-34b-gguf:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf AbteeXAILab/lumynax-chat-yi-15-34b-gguf:Q4_K_M

Use Docker

docker model run hf.co/AbteeXAILab/lumynax-chat-yi-15-34b-gguf:Q4_K_M

LM Studio
Jan

vLLM

How to use AbteeXAILab/lumynax-chat-yi-15-34b-gguf with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AbteeXAILab/lumynax-chat-yi-15-34b-gguf"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AbteeXAILab/lumynax-chat-yi-15-34b-gguf",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/AbteeXAILab/lumynax-chat-yi-15-34b-gguf:Q4_K_M

Ollama
How to use AbteeXAILab/lumynax-chat-yi-15-34b-gguf with Ollama:
```
ollama run hf.co/AbteeXAILab/lumynax-chat-yi-15-34b-gguf:Q4_K_M
```
Unsloth Desktop
Docker Model Runner
How to use AbteeXAILab/lumynax-chat-yi-15-34b-gguf with Docker Model Runner:
```
docker model run hf.co/AbteeXAILab/lumynax-chat-yi-15-34b-gguf:Q4_K_M
```

Lemonade

How to use AbteeXAILab/lumynax-chat-yi-15-34b-gguf with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull AbteeXAILab/lumynax-chat-yi-15-34b-gguf:Q4_K_M

Run and chat with the model

lemonade run user.lumynax-chat-yi-15-34b-gguf-Q4_K_M

List all available models

lemonade list

Atomic Chat

AbteeXAILabs commited on 4 days ago

Commit

c03ccfa

verified ·

1 Parent(s): 9f64059

chore: standardise legacy release metadata

Browse files

Add canonical lifecycle and infusion metadata, correct Hugging Face base-model and pipeline metadata, and refresh integrity records.

Files changed (3) hide show

README.md +1 -0
checksums.sha256 +2 -2
release_export_manifest.json +54 -35

README.md CHANGED Viewed

@@ -29,6 +29,7 @@ tags:
 - nem-convert-required
 - legacy
 - outdated
 ---
 # LumynaX Chat Yi-1.5 34B GGUF (multilingual)

 - nem-convert-required
 - legacy
 - outdated
+base_model: 01-ai/Yi-1.5-34B-Chat
 ---
 # LumynaX Chat Yi-1.5 34B GGUF (multilingual)

checksums.sha256 CHANGED Viewed

@@ -2,6 +2,6 @@
 d0b4f9120ba026c00fa23cb84b4e1620a2e6436592e58155a5151653179572c0  VERSION.txt
 7f1d1228bb0006621ececcea13102f7c9595d17102d2b8824c51d474d7dfd5a5  ollama/Modelfile
 26b2fb6f2558c3e872f791f80950343efad42948cefffd1597d54a23fbe65e5d  quickstart.py
-56383a64fceced83566516469f8f81f4e0bd1668b76c3ca89e3f21274f430321  release_export_manifest.json
 44fc2c3474d1cc9ef66f42dfc3fcc4aa571eae4bf7720fae3569a7451f7e4a3f  requirements.txt
-0a35cbfbb8fafe615791b0b9ae6bcc233b55e5df3a447f1f3cbabe334aa83ddb  README.md

 d0b4f9120ba026c00fa23cb84b4e1620a2e6436592e58155a5151653179572c0  VERSION.txt
 7f1d1228bb0006621ececcea13102f7c9595d17102d2b8824c51d474d7dfd5a5  ollama/Modelfile
 26b2fb6f2558c3e872f791f80950343efad42948cefffd1597d54a23fbe65e5d  quickstart.py
+99e80e7b06432161601e3fce0e0cec959b412228ee71d710eb4d0d88e75f2509  release_export_manifest.json
 44fc2c3474d1cc9ef66f42dfc3fcc4aa571eae4bf7720fae3569a7451f7e4a3f  requirements.txt
+feea4c6a1d5ef696f07f5f65369698638f01eec689df109a249fab4330b4c000  README.md

release_export_manifest.json CHANGED Viewed

@@ -1,35 +1,54 @@
-{
-  "release_id": "lumynax-chat-yi-15-34b-gguf",
-  "title": "LumynaX Chat Yi-1.5 34B GGUF (multilingual)",
-  "repo_id": "AbteeXAILab/lumynax-chat-yi-15-34b-gguf",
-  "version": "v0.1.0",
-  "publisher": "AbteeX AI Labs",
-  "publisher_org": "AbteeXAILab",
-  "jurisdiction": "NZ",
-  "residency": [
-    "NZ"
-  ],
-  "upstream": {
-    "repo_id": "01-ai/Yi-1.5-34B-Chat",
-    "license": "apache-2.0",
-    "prompt_format": "chatml",
-    "source_note": "01-ai Yi"
-  },
-  "gguf_mirror": "bartowski/Yi-1.5-34B-Chat-GGUF",
-  "runtime": "llama_cpp",
-  "modalities": [
-    "text"
-  ],
-  "context_tokens": 32768,
-  "total_params_b": 34,
-  "active_params_b": null,
-  "quantization": "Q4_K_M GGUF",
-  "primary_artifact": "Yi-1.5-34B-Chat-Q4_K_M.gguf",
-  "package_state": "weights_mirrored",
-  "validation_status": "scaffold_verified",
-  "sovereignty_tier": 3,
-  "supports_tools": true,
-  "supports_json": true,
-  "audit_hash_chain": "SHA-256 over canonical request + response JSON",
-  "build_date_utc": "2026-05-17T02:13:28Z"
-}

+{
+  "release_id": "lumynax-chat-yi-15-34b-gguf",
+  "title": "LumynaX Chat Yi-1.5 34B GGUF (multilingual)",
+  "repo_id": "AbteeXAILab/lumynax-chat-yi-15-34b-gguf",
+  "version": "v0.1.0",
+  "publisher": "AbteeX AI Labs",
+  "publisher_org": "AbteeXAILab",
+  "jurisdiction": "NZ",
+  "residency": [
+    "NZ"
+  ],
+  "upstream": {
+    "repo_id": "01-ai/Yi-1.5-34B-Chat",
+    "license": "apache-2.0",
+    "prompt_format": "chatml",
+    "source_note": "01-ai Yi"
+  },
+  "gguf_mirror": "bartowski/Yi-1.5-34B-Chat-GGUF",
+  "runtime": "llama_cpp",
+  "modalities": [
+    "text"
+  ],
+  "context_tokens": 32768,
+  "total_params_b": 34,
+  "active_params_b": null,
+  "quantization": "Q4_K_M GGUF",
+  "primary_artifact": "Yi-1.5-34B-Chat-Q4_K_M.gguf",
+  "package_state": "weights_mirrored",
+  "validation_status": "scaffold_verified",
+  "sovereignty_tier": 3,
+  "supports_tools": true,
+  "supports_json": true,
+  "audit_hash_chain": "SHA-256 over canonical request + response JSON",
+  "build_date_utc": "2026-05-17T02:13:28Z",
+  "lumynax_manifest_version": 3,
+  "lifecycle": {
+    "status": "legacy",
+    "maintenance": "unmaintained",
+    "production_recommended": false
+  },
+  "lumynax_infusion": {
+    "core_role": "primary_intelligence_model",
+    "method": "routed",
+    "infused_model": "01-ai/Yi-1.5-34B-Chat",
+    "weight_composition_applied": false,
+    "moe_composition": "not_applied"
+  },
+  "licensing": {
+    "id": "apache-2.0",
+    "license_file": "LICENSE.txt",
+    "source_model_repo": "01-ai/Yi-1.5-34B-Chat",
+    "weights_subject_to_source_license": true
+  }
+}