Instructions to use mradermacher/KCOMP-MASH-QA-i1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use mradermacher/KCOMP-MASH-QA-i1-GGUF with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("mradermacher/KCOMP-MASH-QA-i1-GGUF", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use mradermacher/KCOMP-MASH-QA-i1-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf mradermacher/KCOMP-MASH-QA-i1-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf mradermacher/KCOMP-MASH-QA-i1-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf mradermacher/KCOMP-MASH-QA-i1-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf mradermacher/KCOMP-MASH-QA-i1-GGUF:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf mradermacher/KCOMP-MASH-QA-i1-GGUF:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf mradermacher/KCOMP-MASH-QA-i1-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf mradermacher/KCOMP-MASH-QA-i1-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf mradermacher/KCOMP-MASH-QA-i1-GGUF:Q4_K_M
Use Docker
docker model run hf.co/mradermacher/KCOMP-MASH-QA-i1-GGUF:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use mradermacher/KCOMP-MASH-QA-i1-GGUF with Ollama:
ollama run hf.co/mradermacher/KCOMP-MASH-QA-i1-GGUF:Q4_K_M
- Unsloth Studio
How to use mradermacher/KCOMP-MASH-QA-i1-GGUF with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for mradermacher/KCOMP-MASH-QA-i1-GGUF to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for mradermacher/KCOMP-MASH-QA-i1-GGUF to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for mradermacher/KCOMP-MASH-QA-i1-GGUF to start chatting
- Docker Model Runner
How to use mradermacher/KCOMP-MASH-QA-i1-GGUF with Docker Model Runner:
docker model run hf.co/mradermacher/KCOMP-MASH-QA-i1-GGUF:Q4_K_M
- Lemonade
How to use mradermacher/KCOMP-MASH-QA-i1-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull mradermacher/KCOMP-MASH-QA-i1-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.KCOMP-MASH-QA-i1-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
uploaded from kaos
Browse files- .gitattributes +24 -0
- KCOMP-MASH-QA.i1-IQ1_M.gguf +3 -0
- KCOMP-MASH-QA.i1-IQ1_S.gguf +3 -0
- KCOMP-MASH-QA.i1-IQ2_M.gguf +3 -0
- KCOMP-MASH-QA.i1-IQ2_S.gguf +3 -0
- KCOMP-MASH-QA.i1-IQ2_XS.gguf +3 -0
- KCOMP-MASH-QA.i1-IQ2_XXS.gguf +3 -0
- KCOMP-MASH-QA.i1-IQ3_M.gguf +3 -0
- KCOMP-MASH-QA.i1-IQ3_S.gguf +3 -0
- KCOMP-MASH-QA.i1-IQ3_XS.gguf +3 -0
- KCOMP-MASH-QA.i1-IQ3_XXS.gguf +3 -0
- KCOMP-MASH-QA.i1-IQ4_NL.gguf +3 -0
- KCOMP-MASH-QA.i1-IQ4_XS.gguf +3 -0
- KCOMP-MASH-QA.i1-Q2_K.gguf +3 -0
- KCOMP-MASH-QA.i1-Q2_K_S.gguf +3 -0
- KCOMP-MASH-QA.i1-Q3_K_L.gguf +3 -0
- KCOMP-MASH-QA.i1-Q3_K_M.gguf +3 -0
- KCOMP-MASH-QA.i1-Q3_K_S.gguf +3 -0
- KCOMP-MASH-QA.i1-Q4_0.gguf +3 -0
- KCOMP-MASH-QA.i1-Q4_1.gguf +3 -0
- KCOMP-MASH-QA.i1-Q4_K_M.gguf +3 -0
- KCOMP-MASH-QA.i1-Q4_K_S.gguf +3 -0
- KCOMP-MASH-QA.i1-Q5_K_M.gguf +3 -0
- KCOMP-MASH-QA.i1-Q5_K_S.gguf +3 -0
- KCOMP-MASH-QA.i1-Q6_K.gguf +3 -0
.gitattributes
CHANGED
|
@@ -34,3 +34,27 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
imatrix.dat filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
imatrix.dat filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
KCOMP-MASH-QA.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
KCOMP-MASH-QA.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
KCOMP-MASH-QA.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
KCOMP-MASH-QA.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
|
| 41 |
+
KCOMP-MASH-QA.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
KCOMP-MASH-QA.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
| 43 |
+
KCOMP-MASH-QA.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
|
| 44 |
+
KCOMP-MASH-QA.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
|
| 45 |
+
KCOMP-MASH-QA.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
| 46 |
+
KCOMP-MASH-QA.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
| 47 |
+
KCOMP-MASH-QA.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
|
| 48 |
+
KCOMP-MASH-QA.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
| 49 |
+
KCOMP-MASH-QA.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
|
| 50 |
+
KCOMP-MASH-QA.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
| 51 |
+
KCOMP-MASH-QA.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
|
| 52 |
+
KCOMP-MASH-QA.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
| 53 |
+
KCOMP-MASH-QA.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
| 54 |
+
KCOMP-MASH-QA.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
| 55 |
+
KCOMP-MASH-QA.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
|
| 56 |
+
KCOMP-MASH-QA.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
| 57 |
+
KCOMP-MASH-QA.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
| 58 |
+
KCOMP-MASH-QA.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
| 59 |
+
KCOMP-MASH-QA.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
| 60 |
+
KCOMP-MASH-QA.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
KCOMP-MASH-QA.i1-IQ1_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7c3fe32382020474fe1d978588c1a1dd623dcdbbdabbfd84ce48912b2a8835de
|
| 3 |
+
size 813847264
|
KCOMP-MASH-QA.i1-IQ1_S.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8dd69c7a9b9c8b14f3bf1bcba35ef5b1f81f77fc8a9961887c9c96f78fed90f8
|
| 3 |
+
size 770962144
|
KCOMP-MASH-QA.i1-IQ2_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:467c7602f859e245e628488c2902223b17ed9665ae49f877222adf00d31405c5
|
| 3 |
+
size 1019474656
|
KCOMP-MASH-QA.i1-IQ2_S.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7f640b86d91a075b9a65a8a2fd6eaba3bdd6d1936cb69d0fd661124cf09fcdd8
|
| 3 |
+
size 962294496
|
KCOMP-MASH-QA.i1-IQ2_XS.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:be46c30fe132d818b8256eb5cfb7e8e803223097c9c812e51f239f06bd71e74e
|
| 3 |
+
size 944861920
|
KCOMP-MASH-QA.i1-IQ2_XXS.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6bde8d822863c2b767568e2bd0d7eba3be7da91cd65a06715be9fb457a2ea4ec
|
| 3 |
+
size 885322464
|
KCOMP-MASH-QA.i1-IQ3_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:557f0a25b7a72971200bc2b5db93fdaba790216a5973632b881c34dde5228847
|
| 3 |
+
size 1308177664
|
KCOMP-MASH-QA.i1-IQ3_S.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1b65910283769ff113ad6c675328a87e5b8a74fec1d66c4ca2868efab742a98e
|
| 3 |
+
size 1289237760
|
KCOMP-MASH-QA.i1-IQ3_XS.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:017f474c3a9947f4465efec4fe5f5fe92f189a59815f7c420ff81524d1807e1e
|
| 3 |
+
size 1244361984
|
KCOMP-MASH-QA.i1-IQ3_XXS.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e912e237e9a685f01a49f620d0756aea20028affffa987d95252536df875e105
|
| 3 |
+
size 1125380832
|
KCOMP-MASH-QA.i1-IQ4_NL.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d4bc65d8d09e0c96891d94bed9ebfbb4e5a5fdedb0c445513121a9f12918fc4e
|
| 3 |
+
size 1552372992
|
KCOMP-MASH-QA.i1-IQ4_XS.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:32724127ef1d8af4545b7f3873f8dd8a1cc0bb0f55507cd969db7c6ee723db8d
|
| 3 |
+
size 1490736384
|
KCOMP-MASH-QA.i1-Q2_K.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d118a658f8e6aa673bd43bda9ae2f3509cd223faaeb61bc734a05f1d91abe234
|
| 3 |
+
size 1157928192
|
KCOMP-MASH-QA.i1-Q2_K_S.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f496d53cb140d27ca4cbe50450e88bd5673ab395e4e51339f5c568480d3d9465
|
| 3 |
+
size 1104647424
|
KCOMP-MASH-QA.i1-Q3_K_L.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a93e7f0b480bbb85b114303b686c76783f2eb99037c79868d421c3ed8eecb66a
|
| 3 |
+
size 1465595136
|
KCOMP-MASH-QA.i1-Q3_K_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:697ea1a9c8f448d96c87e31a9e9a04ba210d673b3a091f16ba09132832b93dcf
|
| 3 |
+
size 1383806208
|
KCOMP-MASH-QA.i1-Q3_K_S.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:35c7dea3f3a9a58c7829a0c464e0771f6a1e563b6c689c4ea5dc1a2d0fb3a6a1
|
| 3 |
+
size 1287984384
|
KCOMP-MASH-QA.i1-Q4_0.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4f5efee349bf263af6108bf873865902ce099d4acf750dddcc023b481b4da455
|
| 3 |
+
size 1555387648
|
KCOMP-MASH-QA.i1-Q4_1.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:dbad05d8d8103667e4a9759e8c2ed90e70b621c256d24428f761895d376237b3
|
| 3 |
+
size 1675056384
|
KCOMP-MASH-QA.i1-Q4_K_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fbe881cbefaf355ad5f3a80bf058b59584524f45c01ee95bd7566557a1856c0a
|
| 3 |
+
size 1630266624
|
KCOMP-MASH-QA.i1-Q4_K_S.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:568e48c09ef29a52927c72adc38b46a9903706e06f1da8fc6f0494afe53a3568
|
| 3 |
+
size 1559844096
|
KCOMP-MASH-QA.i1-Q5_K_M.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1b1854db4b60da2bd2ab4bdfcb2cd125141e191b249bbf88026f95c39aacda91
|
| 3 |
+
size 1839654144
|
KCOMP-MASH-QA.i1-Q5_K_S.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:323382e682c80e5d16c7374a7292e8f99300eda60df6b24c7372ee782ac69947
|
| 3 |
+
size 1798919424
|
KCOMP-MASH-QA.i1-Q6_K.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:db350274514526b4969b2dff99a1aed9b7a1b391b19fe10db6675ec5d8952977
|
| 3 |
+
size 2062128384
|