Instructions to use sakamakismile/Nex-N2.5-Max-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use sakamakismile/Nex-N2.5-Max-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf sakamakismile/Nex-N2.5-Max-GGUF:Q3_K_M
# Run inference directly in the terminal:
llama cli -hf sakamakismile/Nex-N2.5-Max-GGUF:Q3_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf sakamakismile/Nex-N2.5-Max-GGUF:Q3_K_M
# Run inference directly in the terminal:
llama cli -hf sakamakismile/Nex-N2.5-Max-GGUF:Q3_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf sakamakismile/Nex-N2.5-Max-GGUF:Q3_K_M
# Run inference directly in the terminal:
./llama-cli -hf sakamakismile/Nex-N2.5-Max-GGUF:Q3_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf sakamakismile/Nex-N2.5-Max-GGUF:Q3_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf sakamakismile/Nex-N2.5-Max-GGUF:Q3_K_M

Use Docker

docker model run hf.co/sakamakismile/Nex-N2.5-Max-GGUF:Q3_K_M

LM Studio
Jan

vLLM

How to use sakamakismile/Nex-N2.5-Max-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "sakamakismile/Nex-N2.5-Max-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sakamakismile/Nex-N2.5-Max-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/sakamakismile/Nex-N2.5-Max-GGUF:Q3_K_M

Ollama
How to use sakamakismile/Nex-N2.5-Max-GGUF with Ollama:
```
ollama run hf.co/sakamakismile/Nex-N2.5-Max-GGUF:Q3_K_M
```
Unsloth Desktop

How to use sakamakismile/Nex-N2.5-Max-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf sakamakismile/Nex-N2.5-Max-GGUF:Q3_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "sakamakismile/Nex-N2.5-Max-GGUF:Q3_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use sakamakismile/Nex-N2.5-Max-GGUF with Docker Model Runner:
```
docker model run hf.co/sakamakismile/Nex-N2.5-Max-GGUF:Q3_K_M
```

Lemonade

How to use sakamakismile/Nex-N2.5-Max-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull sakamakismile/Nex-N2.5-Max-GGUF:Q3_K_M

Run and chat with the model

lemonade run user.Nex-N2.5-Max-GGUF-Q3_K_M

List all available models

lemonade list

Hermes Agent

How to use sakamakismile/Nex-N2.5-Max-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf sakamakismile/Nex-N2.5-Max-GGUF:Q3_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default sakamakismile/Nex-N2.5-Max-GGUF:Q3_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use sakamakismile/Nex-N2.5-Max-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf sakamakismile/Nex-N2.5-Max-GGUF:Q3_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "sakamakismile/Nex-N2.5-Max-GGUF:Q3_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

sakamakismile commited on 26 days ago

Commit

1a09ed0

verified ·

1 Parent(s): 1aefbcd

Add files using upload-large-folder tool

Browse files

Files changed (42) hide show

.gitattributes +39 -0
Q3_K_M/Nex-N2.5-Max-Q3_K_M-00001-of-00018.gguf +3 -0
Q3_K_M/Nex-N2.5-Max-Q3_K_M-00002-of-00018.gguf +3 -0
Q3_K_M/Nex-N2.5-Max-Q3_K_M-00003-of-00018.gguf +3 -0
Q3_K_M/Nex-N2.5-Max-Q3_K_M-00004-of-00018.gguf +3 -0
Q3_K_M/Nex-N2.5-Max-Q3_K_M-00005-of-00018.gguf +3 -0
Q3_K_M/Nex-N2.5-Max-Q3_K_M-00006-of-00018.gguf +3 -0
Q3_K_M/Nex-N2.5-Max-Q3_K_M-00007-of-00018.gguf +3 -0
Q3_K_M/Nex-N2.5-Max-Q3_K_M-00008-of-00018.gguf +3 -0
Q3_K_M/Nex-N2.5-Max-Q3_K_M-00009-of-00018.gguf +3 -0
Q3_K_M/Nex-N2.5-Max-Q3_K_M-00010-of-00018.gguf +3 -0
Q3_K_M/Nex-N2.5-Max-Q3_K_M-00011-of-00018.gguf +3 -0
Q3_K_M/Nex-N2.5-Max-Q3_K_M-00012-of-00018.gguf +3 -0
Q3_K_M/Nex-N2.5-Max-Q3_K_M-00013-of-00018.gguf +3 -0
Q3_K_M/Nex-N2.5-Max-Q3_K_M-00014-of-00018.gguf +3 -0
Q3_K_M/Nex-N2.5-Max-Q3_K_M-00015-of-00018.gguf +3 -0
Q3_K_M/Nex-N2.5-Max-Q3_K_M-00016-of-00018.gguf +3 -0
Q3_K_M/Nex-N2.5-Max-Q3_K_M-00017-of-00018.gguf +3 -0
Q3_K_M/Nex-N2.5-Max-Q3_K_M-00018-of-00018.gguf +3 -0
Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00001-of-00021.gguf +3 -0
Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00002-of-00021.gguf +3 -0
Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00003-of-00021.gguf +3 -0
Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00004-of-00021.gguf +3 -0
Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00005-of-00021.gguf +3 -0
Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00006-of-00021.gguf +3 -0
Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00007-of-00021.gguf +3 -0
Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00008-of-00021.gguf +3 -0
Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00009-of-00021.gguf +3 -0
Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00010-of-00021.gguf +3 -0
Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00011-of-00021.gguf +3 -0
Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00012-of-00021.gguf +3 -0
Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00013-of-00021.gguf +3 -0
Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00014-of-00021.gguf +3 -0
Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00015-of-00021.gguf +3 -0
Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00016-of-00021.gguf +3 -0
Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00017-of-00021.gguf +3 -0
Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00018-of-00021.gguf +3 -0
Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00019-of-00021.gguf +3 -0
Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00020-of-00021.gguf +3 -0
Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00021-of-00021.gguf +3 -0
README.md +103 -0
llama.cpp-convert-fp8-experts.patch +48 -0

.gitattributes CHANGED Viewed

@@ -33,3 +33,42 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00003-of-00021.gguf filter=lfs diff=lfs merge=lfs -text
+Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00008-of-00021.gguf filter=lfs diff=lfs merge=lfs -text
+Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00005-of-00021.gguf filter=lfs diff=lfs merge=lfs -text
+Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00007-of-00021.gguf filter=lfs diff=lfs merge=lfs -text
+Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00001-of-00021.gguf filter=lfs diff=lfs merge=lfs -text
+Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00011-of-00021.gguf filter=lfs diff=lfs merge=lfs -text
+Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00004-of-00021.gguf filter=lfs diff=lfs merge=lfs -text
+Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00009-of-00021.gguf filter=lfs diff=lfs merge=lfs -text
+Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00002-of-00021.gguf filter=lfs diff=lfs merge=lfs -text
+Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00010-of-00021.gguf filter=lfs diff=lfs merge=lfs -text
+Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00006-of-00021.gguf filter=lfs diff=lfs merge=lfs -text
+Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00012-of-00021.gguf filter=lfs diff=lfs merge=lfs -text
+Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00021-of-00021.gguf filter=lfs diff=lfs merge=lfs -text
+Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00014-of-00021.gguf filter=lfs diff=lfs merge=lfs -text
+Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00015-of-00021.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_M/Nex-N2.5-Max-Q3_K_M-00001-of-00018.gguf filter=lfs diff=lfs merge=lfs -text
+Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00017-of-00021.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_M/Nex-N2.5-Max-Q3_K_M-00003-of-00018.gguf filter=lfs diff=lfs merge=lfs -text
+Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00019-of-00021.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_M/Nex-N2.5-Max-Q3_K_M-00002-of-00018.gguf filter=lfs diff=lfs merge=lfs -text
+Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00013-of-00021.gguf filter=lfs diff=lfs merge=lfs -text
+Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00016-of-00021.gguf filter=lfs diff=lfs merge=lfs -text
+Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00020-of-00021.gguf filter=lfs diff=lfs merge=lfs -text
+Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00018-of-00021.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_M/Nex-N2.5-Max-Q3_K_M-00004-of-00018.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_M/Nex-N2.5-Max-Q3_K_M-00005-of-00018.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_M/Nex-N2.5-Max-Q3_K_M-00009-of-00018.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_M/Nex-N2.5-Max-Q3_K_M-00011-of-00018.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_M/Nex-N2.5-Max-Q3_K_M-00015-of-00018.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_M/Nex-N2.5-Max-Q3_K_M-00007-of-00018.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_M/Nex-N2.5-Max-Q3_K_M-00006-of-00018.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_M/Nex-N2.5-Max-Q3_K_M-00013-of-00018.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_M/Nex-N2.5-Max-Q3_K_M-00012-of-00018.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_M/Nex-N2.5-Max-Q3_K_M-00014-of-00018.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_M/Nex-N2.5-Max-Q3_K_M-00008-of-00018.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_M/Nex-N2.5-Max-Q3_K_M-00010-of-00018.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_M/Nex-N2.5-Max-Q3_K_M-00016-of-00018.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_M/Nex-N2.5-Max-Q3_K_M-00017-of-00018.gguf filter=lfs diff=lfs merge=lfs -text
+Q3_K_M/Nex-N2.5-Max-Q3_K_M-00018-of-00018.gguf filter=lfs diff=lfs merge=lfs -text

Q3_K_M/Nex-N2.5-Max-Q3_K_M-00001-of-00018.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:73dfec7d043e4914cc2a7c60efe402b05a19d832a2ca2da98a8655ad5fab9fb5
+size 41072005920

Q3_K_M/Nex-N2.5-Max-Q3_K_M-00002-of-00018.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:80d517abaf8a2f911aa9941f3aa6ca1d1acf89a1cc514c3713ef078f617160b8
+size 44998463072

Q3_K_M/Nex-N2.5-Max-Q3_K_M-00003-of-00018.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bd0601e4bf8e997fdbf3fbbb1bd2c931f049e07e19580721f07bf217f3f96409
+size 40383041440

Q3_K_M/Nex-N2.5-Max-Q3_K_M-00004-of-00018.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b8136fb61ff05d5086c71202211466b51383578be9cd5656b0b1abc50c1bd14f
+size 44997012768

Q3_K_M/Nex-N2.5-Max-Q3_K_M-00005-of-00018.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1a334fbccb5977ea347d38059f0df530c7c0ade20ab2cef7a2492b06c2cfbe8b
+size 40388546624

Q3_K_M/Nex-N2.5-Max-Q3_K_M-00006-of-00018.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4b755ec0a9d11c1e23b4bb828b8fb564f436443e794217694afd9c27f5dfebdd
+size 44998463168

Q3_K_M/Nex-N2.5-Max-Q3_K_M-00007-of-00018.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:891381ef4387c3a3afb91dda1083303349abc219ec6d062dcc04f42a7b8f581e
+size 40383041536

Q3_K_M/Nex-N2.5-Max-Q3_K_M-00008-of-00018.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:677a570012eceb96e2c2ff1602f76306e81cee48bb119d0175ecaa3b730b9fcf
+size 44997012768

Q3_K_M/Nex-N2.5-Max-Q3_K_M-00009-of-00018.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ee9fd5fe1ee01cc41a7208f3408651369f9556d042ae15bd853eb2ebbc62e948
+size 40388546624

Q3_K_M/Nex-N2.5-Max-Q3_K_M-00010-of-00018.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3a155d9ebca87bbcae1175d83bedc423faf05b3fb941684ac7c33bf5a0a3211d
+size 44998463168

Q3_K_M/Nex-N2.5-Max-Q3_K_M-00011-of-00018.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f1698dbdf3316dfc25dbb4e1ec6801be8faf92ddbcaaf31c79bff4238cdcf5e7
+size 40383041536

Q3_K_M/Nex-N2.5-Max-Q3_K_M-00012-of-00018.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8075b305c747876c21f8a50cf15977d0f7622efe3b302c02ae2c6d1611d7bf6b
+size 44997012768

Q3_K_M/Nex-N2.5-Max-Q3_K_M-00013-of-00018.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c61cf7d9d8177212b3de26bb8b6c1df2ab342c1fc9f5c013744e9931c31ebbcf
+size 40388546624

Q3_K_M/Nex-N2.5-Max-Q3_K_M-00014-of-00018.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:00619b2c76775e29e6b8063451fed74ca774be066a78cc3943619fe6e26700c3
+size 44998463168

Q3_K_M/Nex-N2.5-Max-Q3_K_M-00015-of-00018.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1e3effe7c44a21f584b7170fa34dfb4283e2b715c6549583103391c155017e19
+size 40383041536

Q3_K_M/Nex-N2.5-Max-Q3_K_M-00016-of-00018.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:df0715460f20b74bebc52ba2a77859fcf3cbc6d1ccfb634f10f4b0d669932564
+size 44997012768

Q3_K_M/Nex-N2.5-Max-Q3_K_M-00017-of-00018.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:47d77409a0ddf57b7aa3041d0fd5d2e0b39d46854c013794e1753d1e6104c576
+size 40388546624

Q3_K_M/Nex-N2.5-Max-Q3_K_M-00018-of-00018.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:998794ed27ba16a5352d57b038544a8da5e18b033d1c960045e986292fd746cf
+size 24263210848

Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00001-of-00021.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0ce68f72d3faec989664391314ae4944a2ffdba86f50d645fbd04ae7378b23e3
+size 41227951872

Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00002-of-00021.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:039fcdbc57001274d4b528cde898fbd4cba3f5a47910e1f62fbea6ae92cfdbe7
+size 44060481792

Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00003-of-00021.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e198416327d6ba4fa34643667da1e58d43247baef8229add060076732a41d3ec
+size 44050457120

Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00004-of-00021.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3fcd31dfdfaedb2916013e162cda5b6d04fc47852511f22ee74aed6686724fcf
+size 44060481856

Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00005-of-00021.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3ae744d1d865aa7cd8d7e95cefc44ae4c49660146b5d79a46ae3ca0b09923675
+size 44050457184

Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00006-of-00021.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bfc1708429a713065bddc822d0e1c66a1ecfdef56075fdfb3d0bc2e8668c8641
+size 44060481888

Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00007-of-00021.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1f339732822e80a6ae9bd9fc6b567ea52bee1f519ba10d48df749454894eb1c3
+size 44050457184

Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00008-of-00021.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:067e7e03262fee3b8b28f35cdf4356bbb6cf2ddfdc79311c0a48b89e297849e8
+size 44060481888

Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00009-of-00021.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:31919c1cbb3259446c6c6ff0eabca3c3d4d82feb2dfc3aaf8883969b08c42032
+size 44050457184

Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00010-of-00021.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ca76e2535023150b76c9d43fc3233b8c3dcba3b5da34542382d963f5cd77b473
+size 44060481888

Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00011-of-00021.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8d0da0db197b8f73798f7f07b286a3e207e6e9c5c8def208013478730ed47fef
+size 44050457184

Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00012-of-00021.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:35c3d8f707a652a078f5ae5d1565324568503b8b6bbff2b939fac95b8affbbf2
+size 44060481888

Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00013-of-00021.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1e1b0f49905038f7b98eeb01a4e02d704c0394e11ab1fba7206658c3c59ffa6a
+size 44050457184

Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00014-of-00021.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a183f4f9c4f1acb6269831c09605667cd239b95b7acddea6e6a75a07f7a4825c
+size 44060481888

Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00015-of-00021.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7fac5745cfa27f6d9f3f74fdf828a5610dc349c11813e4d0b0fcb971a933f2bb
+size 44050457184

Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00016-of-00021.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4ed2a7d6700ef5e864c10e4c48ed63dfdf9dce2a5ee5ecc505a93ff2f4605429
+size 44060481888

Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00017-of-00021.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e5c391b2b45d54e7d343f896d5a19dcf13aeb1f9c9a9b5a4a6d9a47dc595ff92
+size 44050457184

Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00018-of-00021.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:49c07afe26bfa8964de2911a9fb91b87aaec18379df9f6510ff49a1d3c376ae2
+size 44060481888

Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00019-of-00021.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9056df033852fb7f65426383e10a2b06194d86f22ed8080f544f0099ba707f50
+size 44050457184

Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00020-of-00021.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:302945b31257a3e070bc3a1fd07d5c0536a8c19c117475d0abcbb5b8dd197349
+size 44060481888

Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00021-of-00021.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a4ee87c3e80d76923789b36508b7aee245f60ad9ae4a09e811c4b7b782f1eab6
+size 19478758496

README.md ADDED Viewed

	@@ -0,0 +1,103 @@

+---
+license: apache-2.0
+base_model: nex-agi/Nex-N2.5-Max
+base_model_relation: quantized
+pipeline_tag: text-generation
+library_name: llama.cpp
+tags:
+- gguf
+- deepseek_v4
+- moe
+- nex-n2.5
+- lna-lab
+language:
+- en
+- ja
+- zh
+---
+# Nex-N2.5-Max GGUF (llama.cpp)
+GGUF quantizations of [nex-agi/Nex-N2.5-Max](https://huggingface.co/nex-agi/Nex-N2.5-Max)
+(1.6T-parameter text-only MoE, 384 routed experts / top-6, 61 layers, 1M context, MTP head),
+made at Lna-Lab on 2026-09-09 for **pure-CPU inference** on a 1 TiB DDR5 workstation.
+Nex-N2.5-Max uses the **DeepSeek-V4-Pro architecture unchanged** (`DeepseekV4ForCausalLM`,
+hidden 7168, MoE inter 3072, indexer 64 heads / top-1024, hyper-connections x4, 3 hash-routed layers).
+Any llama.cpp build that runs DeepSeek-V4 runs these files.
+## Files
+| folder | recipe | size | BPW | notes |
+|---|---|---|---|---|
+| `Q4K-exps-Q8dense/` | routed experts **Q4_K** (pure), everything else (attention, shared expert, indexer, router, embeddings, output) **Q8_0** | 898 GB (836 GiB) | 4.57 | fastest on CPU; needs ~880 GiB of free RAM |
+| `Q3_K_M/` | stock llama.cpp `Q3_K_M` mixture | 748 GB (697 GiB) | 3.81 | fits alongside other residents; slower on CPU (no repacked kernel for Q3_K) |
+Each folder is a standard llama.cpp split (`-00001-of-000NN.gguf`); point `-m` at the first shard.
+## Measured (not estimated)
+Threadripper PRO 9985WX (64 cores), 1 TiB DDR5-8ch, **no GPU** (`-ngl 0 -t 64`), llama.cpp master `465e49b`, warm page cache, single stream:
+| quant | prompt tok/s | generation tok/s |
+|---|---|---|
+| Q4K-exps-Q8dense | 27.8 | **5.35** |
+| Q3_K_M | 25.6 | 3.04 |
+Q3_K_M is *slower* although smaller: llama.cpp has an AVX-512 repacked kernel for Q4_K but not for Q3_K.
+The repack also copies the Q4_K experts into anonymous memory (RssAnon ≈ 811 GiB for the Q4 file),
+so `mmap` does not save you RAM with that file. Cold first run (page faults) is ~0.5 tok/s; warm the file first
+(`cat file > /dev/null` in parallel chunks, ~50 s on a fast RAID) or just wait for the first generation.
+Quality spot check (Japanese haiku with one-line commentary, thinking off): both quants produce clean Japanese
+with sensible kigo; we could not tell them apart on that task. The model **thinks in English** when
+`enable_thinking` is on (gpt-oss style), and answers in the user's language.
+## Running
+```bash
+llama-server -m Q4K-exps-Q8dense/Nex-N2.5-Max-Q4K-exps-Q8dense-00001-of-00021.gguf \
+  -ngl 0 -t 64 -c 16384 -fa on --jinja --port 8030
+```
+The chat template is embedded (from the original repo) and supports `chat_template_kwargs`:
+`enable_thinking` (default true) and `thinking_mode` (`interleaved` | `full` | `drop`).
+Known issue (llama.cpp `465e49b`): `/v1/chat/completions` can return HTTP 500
+*"The model produced output that does not match the expected peg-native format"* after a complete answer.
+The model is fine; the server-side output parser is not. Work-around: render with `POST /apply-template`
+and generate with `POST /completion`.
+## How these were made
+The original checkpoint stores routed experts as **FP8 e4m3 with 128x128 block scales stored as F32**
+(`expert_dtype: fp8`, `scale_fmt: ue8m0`), whereas the official DeepSeek-V4 release packs experts as MXFP4.
+llama.cpp's `convert_hf_to_gguf.py` (DeepSeek-V4 path) dequantizes the FP8 tensors, drops the scales, and then
+tries to repack experts as MXFP4 -> `KeyError: Missing routed expert tensors`. It also reads F32 scales as e8m0 bytes.
+`llama.cpp-convert-fp8-experts.patch` (against `conversion/deepseek.py`, master `465e49b`) fixes both:
+1. float-typed block scales are used as values;
+2. FP8-dequantized experts are stacked (lazily, 384 per projection) and emitted as ordinary expert tensors
+   (they land as Q8_0 in the intermediate, per the converter's existing FP8 policy).
+Pipeline: `convert_hf_to_gguf.py --outtype bf16` (intermediate: experts Q8_0 + rest BF16, 1.70 TB, 2h18m) ->
+`llama-quantize --allow-requantize`:
+```bash
+# Q4K-exps-Q8dense
+llama-quantize --allow-requantize --pure --output-tensor-type q8_0 --token-embedding-type q8_0 \
+  --tensor-type 'attn_.*=q8_0' --tensor-type 'ffn_.*_shexp=q8_0' --tensor-type 'indexer.*=q8_0' --tensor-type 'ffn_gate_inp=q8_0' \
+  intermediate.gguf out.gguf Q4_K_M 48
+# Q3_K_M
+llama-quantize --allow-requantize intermediate.gguf out.gguf Q3_K_M 48
+```
+No importance matrix was used. A stock `Q4_K_M` was also baked (951 GB, 4.84 BPW) but is not published:
+it does not fit in 1 TiB RAM next to anything else, and its attention weights are 4-bit anyway.
+MTP layer is not included (the converter's main-model path skips `mtp.*`; those tensors are BF16 in the source).
+## Credits
+- Model: [Nex-AGI](https://nex-agi.com/), Apache-2.0. Architecture: DeepSeek.
+- Quantization and converter patch: YUKI (Claude Fable 5.1), researching together with Ken at Lna-Lab — [@Tono_Ken3](https://x.com/Tono_Ken3)

llama.cpp-convert-fp8-experts.patch ADDED Viewed

	@@ -0,0 +1,48 @@

+diff --git a/conversion/deepseek.py b/conversion/deepseek.py
+index 817eb76..a49f27a 100644
+--- a/conversion/deepseek.py
++++ b/conversion/deepseek.py
+@@ -694,7 +694,11 @@ class DeepseekV4Model(TextModel):
+         def dequant_fp8_weight(weight: Tensor, scale: Tensor) -> Tensor:
+             out_features, in_features = weight.shape
+-            scale_f = self._e8m0_to_float(scale)
++            # Lna-Lab 2026-09-09: Nex-N2.5-Max stores block scales as F32 values (ue8m0 semantics), not e8m0 bytes
++            if scale.dtype in (torch.float32, torch.float16, torch.bfloat16):
++                scale_f = scale.float()
++            else:
++                scale_f = self._e8m0_to_float(scale)
+             scale_f = scale_f.repeat_interleave(128, 0)[:out_features]
+             scale_f = scale_f.repeat_interleave(128, 1)[:, :in_features]
+             return weight.float() * scale_f
+@@ -780,6 +784,20 @@ class DeepseekV4Model(TextModel):
+         for bid in range(self.block_count):
+             if self.mtp_only and bid < main_layers:
+                 continue
++            if f"layers.{bid}.ffn.experts.0.w1.weight" in self._dsv4_fp8_dequantized:
++                # Lna-Lab 2026-09-09: FP8 routed experts (Nex-N2.5-Max) -> stack per projection, no MXFP4 repack
++                n_experts = self.hparams["n_routed_experts"]
++                for proj in ("w1", "w2", "w3"):
++                    parts = []
++                    for eid in range(n_experts):
++                        wname = f"layers.{bid}.ffn.experts.{eid}.{proj}.weight"
++                        parts.append(self.model_tensors[wname]().to(torch.bfloat16))  # lazy: evaluated at write time
++                        consumed.append(wname)
++                    stacked_name = f"layers.{bid}.ffn.experts.{proj}.weight"
++                    self._dsv4_fp8_dequantized.add(stacked_name)
++                    logger.info(f"{stacked_name}: stacked {n_experts} FP8-dequantized experts")
++                    yield (stacked_name, torch.stack(parts, dim=0))
++                continue
+             consumed.extend(self._write_mxfp4_expert_tensor(bid, "w1", gguf.MODEL_TENSOR.FFN_GATE_EXP))
+             consumed.extend(self._write_mxfp4_expert_tensor(bid, "w2", gguf.MODEL_TENSOR.FFN_DOWN_EXP))
+             consumed.extend(self._write_mxfp4_expert_tensor(bid, "w3", gguf.MODEL_TENSOR.FFN_UP_EXP))
+@@ -857,6 +875,9 @@ class DeepseekV4Model(TextModel):
+             "ffn.gate.bias": (gguf.MODEL_TENSOR.FFN_EXP_PROBS_B, ".bias"),
+             "ffn.gate.bias_vl": (gguf.MODEL_TENSOR.FFN_EXP_PROBS_B_VL, ".bias"),
+             "ffn.gate.tid2eid": (gguf.MODEL_TENSOR.FFN_GATE_TID2EID, ".weight"),
++            "ffn.experts.w1.weight": (gguf.MODEL_TENSOR.FFN_GATE_EXP, ".weight"),
++            "ffn.experts.w2.weight": (gguf.MODEL_TENSOR.FFN_DOWN_EXP, ".weight"),
++            "ffn.experts.w3.weight": (gguf.MODEL_TENSOR.FFN_UP_EXP, ".weight"),
+             "ffn.shared_experts.w1.weight": (gguf.MODEL_TENSOR.FFN_GATE_SHEXP, ".weight"),
+             "ffn.shared_experts.w2.weight": (gguf.MODEL_TENSOR.FFN_DOWN_SHEXP, ".weight"),
+             "ffn.shared_experts.w3.weight": (gguf.MODEL_TENSOR.FFN_UP_SHEXP, ".weight"),