Instructions to use nomic-ai/nomic-embed-text-v2-moe-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use nomic-ai/nomic-embed-text-v2-moe-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf nomic-ai/nomic-embed-text-v2-moe-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf nomic-ai/nomic-embed-text-v2-moe-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf nomic-ai/nomic-embed-text-v2-moe-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf nomic-ai/nomic-embed-text-v2-moe-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf nomic-ai/nomic-embed-text-v2-moe-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf nomic-ai/nomic-embed-text-v2-moe-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf nomic-ai/nomic-embed-text-v2-moe-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf nomic-ai/nomic-embed-text-v2-moe-GGUF:Q4_K_M

Use Docker

docker model run hf.co/nomic-ai/nomic-embed-text-v2-moe-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use nomic-ai/nomic-embed-text-v2-moe-GGUF with Ollama:
```
ollama run hf.co/nomic-ai/nomic-embed-text-v2-moe-GGUF:Q4_K_M
```

Unsloth Studio

How to use nomic-ai/nomic-embed-text-v2-moe-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for nomic-ai/nomic-embed-text-v2-moe-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for nomic-ai/nomic-embed-text-v2-moe-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for nomic-ai/nomic-embed-text-v2-moe-GGUF to start chatting

Docker Model Runner
How to use nomic-ai/nomic-embed-text-v2-moe-GGUF with Docker Model Runner:
```
docker model run hf.co/nomic-ai/nomic-embed-text-v2-moe-GGUF:Q4_K_M
```

Lemonade

How to use nomic-ai/nomic-embed-text-v2-moe-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull nomic-ai/nomic-embed-text-v2-moe-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.nomic-embed-text-v2-moe-GGUF-Q4_K_M

List all available models

lemonade list

Atomic Chat

Cebtenzzre commited on May 15, 2025

Commit

ffbcf4c

1 Parent(s): 8a7e7cb

reconvert with latest llama.cpp

Browse files

Files changed (16) hide show

README.md +1 -1
nomic-embed-text-v2-moe.Q2_K.gguf +1 -1
nomic-embed-text-v2-moe.Q3_K_L.gguf +1 -1
nomic-embed-text-v2-moe.Q3_K_M.gguf +1 -1
nomic-embed-text-v2-moe.Q3_K_S.gguf +1 -1
nomic-embed-text-v2-moe.Q4_0.gguf +1 -1
nomic-embed-text-v2-moe.Q4_1.gguf +1 -1
nomic-embed-text-v2-moe.Q4_K_M.gguf +1 -1
nomic-embed-text-v2-moe.Q4_K_S.gguf +1 -1
nomic-embed-text-v2-moe.Q5_K_M.gguf +1 -1
nomic-embed-text-v2-moe.Q5_K_S.gguf +1 -1
nomic-embed-text-v2-moe.Q6_K.gguf +1 -1
nomic-embed-text-v2-moe.Q8_0.gguf +1 -1
nomic-embed-text-v2-moe.bf16.gguf +1 -1
nomic-embed-text-v2-moe.f16.gguf +1 -1
nomic-embed-text-v2-moe.f32.gguf +2 -2

README.md CHANGED Viewed

@@ -117,7 +117,7 @@ language:
 This model was presented in the paper [Training Sparse Mixture Of Experts Text Embedding Models](https://huggingface.co/papers/2502.07972).
-Using <a href="https://github.com/ggml-org/llama.cpp">llama.cpp</a> commit <a href="https://github.com/ggml-org/llama.cpp/tree/a96786c0b01bc913343c861af8c460a715e68c20">a96786c0b</a> for quantization.
 Original model: [nomic-embed-text-v2-moe](https://huggingface.co/nomic-ai/nomic-embed-text-v2-moe)

 This model was presented in the paper [Training Sparse Mixture Of Experts Text Embedding Models](https://huggingface.co/papers/2502.07972).
+Using <a href="https://github.com/ggml-org/llama.cpp">llama.cpp</a> commit <a href="https://github.com/ggml-org/llama.cpp/tree/e3a9421b78da5c810d812e6348a405f5acc39f34">e3a9421b7</a> for quantization.
 Original model: [nomic-embed-text-v2-moe](https://huggingface.co/nomic-ai/nomic-embed-text-v2-moe)

nomic-embed-text-v2-moe.Q2_K.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c3d4791d779921323ca7700880417ca4472c3b520311c25a28c092f47a77ab55
 size 273286112

 version https://git-lfs.github.com/spec/v1
+oid sha256:11843331c8f0d14dca2be4809e1146a3a0411892e33b78ae6971b3f619f8e78b
 size 273286112

nomic-embed-text-v2-moe.Q3_K_L.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e17509c5ed7cbefcf70f6e2a1a5c8ade7c6f124834fae2c82c95ea575ab3bffc
 size 322223072

 version https://git-lfs.github.com/spec/v1
+oid sha256:0be98ccfafd74dd15d4c737f76744f06ac045eddf021fddbc256b60856ddfeef
 size 322223072

nomic-embed-text-v2-moe.Q3_K_M.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:49274ff1ef657b839efc3dd484b876523e6cd8f5b88c463636b302cab5893e25
 size 308122592

 version https://git-lfs.github.com/spec/v1
+oid sha256:3444c50a7044f0e7b898896e745eab21a0e313a45a71d8aee6df3fde1d0c3d25
 size 308122592

nomic-embed-text-v2-moe.Q3_K_S.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b750b53963ae4b3ea1ca3901602d5c9a3bad787c8076d48d5a7183816c1132c5
 size 288381920

 version https://git-lfs.github.com/spec/v1
+oid sha256:e6ae830aa61bd0684c30f1b57aba5559e48b1d40dd7266a93717ff1deda4861a
 size 288381920

nomic-embed-text-v2-moe.Q4_0.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ff6b50ae777fcef5150b153720a28ea65c43c9143b2fcc2f9b01a0f04f3eb928
 size 324158432

 version https://git-lfs.github.com/spec/v1
+oid sha256:c15fd36aa079618c10037d9ea3b37237d8a4147b9fc4f8325f32539437d38ae7
 size 324158432

nomic-embed-text-v2-moe.Q4_1.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9df9d1930732c1833e1bf152816ad5bef4f470acc152d82013bb83158171f398
 size 341853152

 version https://git-lfs.github.com/spec/v1
+oid sha256:971f584a5739a1c4f0ab6e4395d72059d5c6c8b0f4815c4b56e49136ef340db4
 size 341853152

nomic-embed-text-v2-moe.Q4_K_M.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8272aa8a695585cde44ddee36a7f8e3aa9978547a45d05a29198a50975879288
 size 344120288

 version https://git-lfs.github.com/spec/v1
+oid sha256:b5fb2811647b8ef461519a68a3bf67014a84a66a130c8a2af9413ff9f06d3f22
 size 344120288

nomic-embed-text-v2-moe.Q4_K_S.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a9773393c85167567cf702e867b7189864bb2ca0b755bf464f4231c1469d309a
 size 325338080

 version https://git-lfs.github.com/spec/v1
+oid sha256:db0608a87a2daf4a52b74912dd678ca6122db26d971bdfcf16d3b11b77047663
 size 325338080

nomic-embed-text-v2-moe.Q5_K_M.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3400b01b9d3e5818bcf58242544a7c05c4290b939c4eb4d00a7c78ea129a67df
 size 370828256

 version https://git-lfs.github.com/spec/v1
+oid sha256:a6546bdf4e46288c93d79e6292bbdd8efa7ca87c242a8e60afa9cbacdc641471
 size 370828256

nomic-embed-text-v2-moe.Q5_K_S.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d818990d39f20d06aeb091dc9af1d3c2f5020e0263e1f9a056073b210212787f
 size 359547872

 version https://git-lfs.github.com/spec/v1
+oid sha256:540055ed857b6f23521d2fca1a17d31a0e0d0f21aa8e628d48c3e56ef2c20344
 size 359547872

nomic-embed-text-v2-moe.Q6_K.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e2f15429bd8dd6e9b40d2a4209531e49233475a93a3c3183f2b95929e6ada7f1
 size 397149152

 version https://git-lfs.github.com/spec/v1
+oid sha256:e74dafe6932fc7ae53f98061401f5cae136203a6b09fa940a2f1eb0ab5804ed8
 size 397149152

nomic-embed-text-v2-moe.Q8_0.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1e326f7e21ab58b1730c1d84e72667c252b9e2cc57ca45a6e8bb0aa6c2bf319a
 size 512225120

 version https://git-lfs.github.com/spec/v1
+oid sha256:06e7a7e594a26985523c18383aba4aad39fe6e14f08ffc6ab5b554e1ccdc3cff
 size 512225120

nomic-embed-text-v2-moe.bf16.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bae48008d884de3831ac282e80c41d0103845160a051567e7360d8115ef9b160
 size 957680480

 version https://git-lfs.github.com/spec/v1
+oid sha256:94c6c2cc3b9212cb9f11067c4d7fa17319dc54c47a488384ddcee4e3163ca807
 size 957680480

nomic-embed-text-v2-moe.f16.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5687a87d4ca943a6077b9d06f605d25d19da4430b32a44e9929831c0aa2c4bfc
 size 957680480

 version https://git-lfs.github.com/spec/v1
+oid sha256:a5db3381f2e514d3490a3a31fe70eb1a65e95016c85c6c2c23223b810806594f
 size 957680480

nomic-embed-text-v2-moe.f32.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6885d296889479db165c912cc87b9657e181fab520a9fad72559d2aeb43549c9
-size 1907985248

 version https://git-lfs.github.com/spec/v1
+oid sha256:7c9a85afd70f95e953f36d91c3b7f3d7dda72e3840f63b6ab66fc1856b47506f
+size 1907985280