Instructions to use mradermacher/Jade1.7b-i1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mradermacher/Jade1.7b-i1-GGUF with Transformers:

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("mradermacher/Jade1.7b-i1-GGUF", device_map="auto")

llama-cpp-python

How to use mradermacher/Jade1.7b-i1-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="mradermacher/Jade1.7b-i1-GGUF",
	filename="Jade1.7b.i1-IQ1_M.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use mradermacher/Jade1.7b-i1-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mradermacher/Jade1.7b-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mradermacher/Jade1.7b-i1-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mradermacher/Jade1.7b-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mradermacher/Jade1.7b-i1-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf mradermacher/Jade1.7b-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf mradermacher/Jade1.7b-i1-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf mradermacher/Jade1.7b-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf mradermacher/Jade1.7b-i1-GGUF:Q4_K_M

Use Docker

docker model run hf.co/mradermacher/Jade1.7b-i1-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use mradermacher/Jade1.7b-i1-GGUF with Ollama:
```
ollama run hf.co/mradermacher/Jade1.7b-i1-GGUF:Q4_K_M
```

Unsloth Studio

How to use mradermacher/Jade1.7b-i1-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for mradermacher/Jade1.7b-i1-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for mradermacher/Jade1.7b-i1-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for mradermacher/Jade1.7b-i1-GGUF to start chatting

Atomic Chat new
Docker Model Runner
How to use mradermacher/Jade1.7b-i1-GGUF with Docker Model Runner:
```
docker model run hf.co/mradermacher/Jade1.7b-i1-GGUF:Q4_K_M
```

Lemonade

How to use mradermacher/Jade1.7b-i1-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull mradermacher/Jade1.7b-i1-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Jade1.7b-i1-GGUF-Q4_K_M

List all available models

lemonade list

mradermacher commited on Mar 26

Commit

3d17247

verified ·

1 Parent(s): a9f2e06

uploaded from leia

Browse files

Files changed (25) hide show

.gitattributes +24 -0
Jade1.7b.i1-IQ1_M.gguf +3 -0
Jade1.7b.i1-IQ1_S.gguf +3 -0
Jade1.7b.i1-IQ2_M.gguf +3 -0
Jade1.7b.i1-IQ2_S.gguf +3 -0
Jade1.7b.i1-IQ2_XS.gguf +3 -0
Jade1.7b.i1-IQ2_XXS.gguf +3 -0
Jade1.7b.i1-IQ3_M.gguf +3 -0
Jade1.7b.i1-IQ3_S.gguf +3 -0
Jade1.7b.i1-IQ3_XS.gguf +3 -0
Jade1.7b.i1-IQ3_XXS.gguf +3 -0
Jade1.7b.i1-IQ4_NL.gguf +3 -0
Jade1.7b.i1-IQ4_XS.gguf +3 -0
Jade1.7b.i1-Q2_K.gguf +3 -0
Jade1.7b.i1-Q2_K_S.gguf +3 -0
Jade1.7b.i1-Q3_K_L.gguf +3 -0
Jade1.7b.i1-Q3_K_M.gguf +3 -0
Jade1.7b.i1-Q3_K_S.gguf +3 -0
Jade1.7b.i1-Q4_0.gguf +3 -0
Jade1.7b.i1-Q4_1.gguf +3 -0
Jade1.7b.i1-Q4_K_M.gguf +3 -0
Jade1.7b.i1-Q4_K_S.gguf +3 -0
Jade1.7b.i1-Q5_K_M.gguf +3 -0
Jade1.7b.i1-Q5_K_S.gguf +3 -0
Jade1.7b.i1-Q6_K.gguf +3 -0

.gitattributes CHANGED Viewed

@@ -34,3 +34,27 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 Jade1.7b.imatrix.gguf filter=lfs diff=lfs merge=lfs -text

 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 Jade1.7b.imatrix.gguf filter=lfs diff=lfs merge=lfs -text
+Jade1.7b.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
+Jade1.7b.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
+Jade1.7b.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
+Jade1.7b.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
+Jade1.7b.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
+Jade1.7b.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
+Jade1.7b.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
+Jade1.7b.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
+Jade1.7b.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
+Jade1.7b.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
+Jade1.7b.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
+Jade1.7b.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
+Jade1.7b.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+Jade1.7b.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Jade1.7b.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+Jade1.7b.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Jade1.7b.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Jade1.7b.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+Jade1.7b.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
+Jade1.7b.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Jade1.7b.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Jade1.7b.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Jade1.7b.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Jade1.7b.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text

Jade1.7b.i1-IQ1_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c5cd2d032d30a54708cdb5dc85fa52ef6b7c4c838d6ed4c8007351a3d11dd7ae
+size 543790272

Jade1.7b.i1-IQ1_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:13d42b42f91f3787f369e06e09adfe1ea4f9165974253ac7283c0890218108a0
+size 515773632

Jade1.7b.i1-IQ2_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:eff91fb6b52ea23582d02b96afc526c55b22c5cfe9d356ea65d556fe3afcf580
+size 695178432

Jade1.7b.i1-IQ2_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c465e9b9c634c77111e746ea102a6bd0fd894208219708ab16c5f20f7cf9f7a6
+size 657822912

Jade1.7b.i1-IQ2_XS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4153cfac57bded947e84e07bf140b36f5fc7fd53555c2c1f56bca81c7bf96d00
+size 631510208

Jade1.7b.i1-IQ2_XXS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:913a98919e144754d461083a5869c830eadbb072a21cc74b1638c869b3f98b7a
+size 590484672

Jade1.7b.i1-IQ3_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6b60669802215db6f6077ae50f55c65cc7391735699a396f90662eaa2539a73e
+size 895659200

Jade1.7b.i1-IQ3_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e16a09402f79688369ff317698cf9e7f23a1165e96ee24ef38e2bd2437fd0fc2
+size 867249344

Jade1.7b.i1-IQ3_XS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:69c44979bd4fef2f29ffe2cac23dade8e48e5b8ee44efb50c42f8891ee61b8bd
+size 834219200

Jade1.7b.i1-IQ3_XXS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:26b50c4b2526e535fe71f610318c8d5e807c3f8fe00d04192a4e000dbdb91c9f
+size 754357440

Jade1.7b.i1-IQ4_NL.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2f65bd5d1019f8f17b867712a3f4cbc7eeac18a160d7eae9684c12da7d814199
+size 1054420160

Jade1.7b.i1-IQ4_XS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c1f55367e1c92cd6eab094839466fe5da30499fd38002e97fd0a0c871bc72e72
+size 1010379968

Jade1.7b.i1-Q2_K.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:eb87742ca4a1797af7633bf29a3da23fbe109f66fde7746423ceb2cc3973df7f
+size 777792704

Jade1.7b.i1-Q2_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9a8e94adfdf1546953fcfa518eb734ff9d649fe6b3c738be30fd5bee56d157c3
+size 732966080

Jade1.7b.i1-Q3_K_L.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f8b1510c7ebf411fcde95b531181fcb25d0183bd36d8e491f0fe58eed9fe9ba3
+size 1003498688

Jade1.7b.i1-Q3_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f09258116f8d33505ae387d19629014e491a3ccf64fa76126ffb979b813966f7
+size 939535552

Jade1.7b.i1-Q3_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ad3f736e9f32b8ec38285a9da22a6ec169d718a42dbe3db0323d52c0d068dd21
+size 867249344

Jade1.7b.i1-Q4_0.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cf84deb21169e190c9b7ac30699e9154551bb77a393ceff75171f21b66b7833d
+size 1056779456

Jade1.7b.i1-Q4_1.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:092d60eb3bcc75966ec70c7f96975b2500c83d9d56f22e59a0e1565cb5655980
+size 1142500544

Jade1.7b.i1-Q4_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8294332babd62cd6ca8409d67784c40d1a350624339bf88f8901aafb7561ea35
+size 1107406016

Jade1.7b.i1-Q4_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:96c93062129f5ac0c2fdf82260ec583e3983dab14200b57e3a3b7f203fca3606
+size 1060187328

Jade1.7b.i1-Q5_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f40c53d908889e86fed915f3401a939344e6cd03500468066c009c262e1cf1e5
+size 1257876672

Jade1.7b.i1-Q5_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e3502caf0466c29682b1f1b35615fd362aaac17424abc71e1423edd24d7745e8
+size 1230580928

Jade1.7b.i1-Q6_K.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:97ba10a2eb45b5ca82daebebc6acac58683ac302f5a0867a185c97572e3d77ec
+size 1417751744