Instructions to use eaddario/Qwen3-30B-A3B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use eaddario/Qwen3-30B-A3B-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="eaddario/Qwen3-30B-A3B-GGUF",
	filename="Qwen3-30B-A3B-F16-00001-of-00003.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use eaddario/Qwen3-30B-A3B-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use eaddario/Qwen3-30B-A3B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "eaddario/Qwen3-30B-A3B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "eaddario/Qwen3-30B-A3B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Ollama
How to use eaddario/Qwen3-30B-A3B-GGUF with Ollama:
```
ollama run hf.co/eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M
```

Unsloth Studio

How to use eaddario/Qwen3-30B-A3B-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for eaddario/Qwen3-30B-A3B-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for eaddario/Qwen3-30B-A3B-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for eaddario/Qwen3-30B-A3B-GGUF to start chatting

How to use eaddario/Qwen3-30B-A3B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use eaddario/Qwen3-30B-A3B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use eaddario/Qwen3-30B-A3B-GGUF with Docker Model Runner:
```
docker model run hf.co/eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M
```

Lemonade

How to use eaddario/Qwen3-30B-A3B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Qwen3-30B-A3B-GGUF-Q4_K_M

List all available models

lemonade list

Qwen3-30B-A3B-GGUF / scores /Qwen3-30B-A3B-Q3_K_M.md

eaddario

Add GGUF internal file structure

a630e18 verified about 1 year ago

preview code

raw

history blame

187 kB

	# Qwen3-30B-A3B-Q3_K_M.gguf - GGUF Internal File Dump

	- Endian: LITTLE endian

	## Key Value Metadata Store

	There are 44 key-value pairs in this file

	\| POS \| TYPE \| Count \| Key \| Value \|
	\|----:\|:---------\|-------:\|:------------------------------------------\|:--------------------------------------------------------------------\|
	\| 1 \| UINT32 \| 1 \| GGUF.version \| 3 \|
	\| 2 \| UINT64 \| 1 \| GGUF.tensor_count \| 579 \|
	\| 3 \| UINT64 \| 1 \| GGUF.kv_count \| 41 \|
	\| 4 \| STRING \| 1 \| general.architecture \| `qwen3moe` \|
	\| 5 \| STRING \| 1 \| general.type \| `model` \|
	\| 6 \| STRING \| 1 \| general.name \| `Qwen3 30B A3B` \|
	\| 7 \| STRING \| 1 \| general.basename \| `Qwen3` \|
	\| 8 \| STRING \| 1 \| general.size_label \| `30B-A3B` \|
	\| 9 \| STRING \| 1 \| general.license \| `apache-2.0` \|
	\| 10 \| STRING \| 1 \| general.license.link \| `https://huggingface.co/Qwen/Qwen3-30B-A3B/blob/main/LICENSE` \|
	\| 11 \| UINT32 \| 1 \| general.base_model.count \| 1 \|
	\| 12 \| STRING \| 1 \| general.base_model.0.name \| `Qwen3 30B A3B Base` \|
	\| 13 \| STRING \| 1 \| general.base_model.0.organization \| `Qwen` \|
	\| 14 \| STRING \| 1 \| general.base_model.0.repo_url \| `https://huggingface.co/Qwen/Qwen3-30B-A3B-Base` \|
	\| 15 \| [STRING] \| 1 \| general.tags \| [ `text-generation` ] \|
	\| 16 \| UINT32 \| 1 \| qwen3moe.block_count \| 48 \|
	\| 17 \| UINT32 \| 1 \| qwen3moe.context_length \| 40960 \|
	\| 18 \| UINT32 \| 1 \| qwen3moe.embedding_length \| 2048 \|
	\| 19 \| UINT32 \| 1 \| qwen3moe.feed_forward_length \| 6144 \|
	\| 20 \| UINT32 \| 1 \| qwen3moe.attention.head_count \| 32 \|
	\| 21 \| UINT32 \| 1 \| qwen3moe.attention.head_count_kv \| 4 \|
	\| 22 \| FLOAT32 \| 1 \| qwen3moe.rope.freq_base \| 1000000.0 \|
	\| 23 \| FLOAT32 \| 1 \| qwen3moe.attention.layer_norm_rms_epsilon \| 1e-06 \|
	\| 24 \| UINT32 \| 1 \| qwen3moe.expert_used_count \| 8 \|
	\| 25 \| UINT32 \| 1 \| qwen3moe.attention.key_length \| 128 \|
	\| 26 \| UINT32 \| 1 \| qwen3moe.attention.value_length \| 128 \|
	\| 27 \| UINT32 \| 1 \| qwen3moe.expert_count \| 128 \|
	\| 28 \| UINT32 \| 1 \| qwen3moe.expert_feed_forward_length \| 768 \|
	\| 29 \| STRING \| 1 \| tokenizer.ggml.model \| `gpt2` \|
	\| 30 \| STRING \| 1 \| tokenizer.ggml.pre \| `qwen2` \|
	\| 31 \| [STRING] \| 151936 \| tokenizer.ggml.tokens \| [ `!`, `"`, `#`, `$`, `%`, ... ] \|
	\| 32 \| [INT32] \| 151936 \| tokenizer.ggml.token_type \| [ 1, 1, 1, 1, 1, 1, 1, ... ] \|
	\| 33 \| [STRING] \| 151387 \| tokenizer.ggml.merges \| [ `Ġ Ġ`, `ĠĠ ĠĠ`, `i n`, `Ġ t`, `ĠĠĠĠ ĠĠĠĠ`, ... ] \|
	\| 34 \| UINT32 \| 1 \| tokenizer.ggml.eos_token_id \| 151645 \|
	\| 35 \| UINT32 \| 1 \| tokenizer.ggml.padding_token_id \| 151643 \|
	\| 36 \| UINT32 \| 1 \| tokenizer.ggml.bos_token_id \| 151643 \|
	\| 37 \| BOOL \| 1 \| tokenizer.ggml.add_bos_token \| False \|
	\| 38 \| STRING \| 1 \| tokenizer.chat_template \| `{%- if tools %}{{- '<\|im_`...`{%- endif %}{%- endif %}` \|
	\| 39 \| UINT32 \| 1 \| general.quantization_version \| 2 \|
	\| 40 \| UINT32 \| 1 \| general.file_type \| 12 \|
	\| 41 \| STRING \| 1 \| quantize.imatrix.file \| `./imatrix/imatrix-Qwen3-30B-A3B-large.dat` \|
	\| 42 \| STRING \| 1 \| quantize.imatrix.dataset \| `../../datasets/imatrix/calibration_all_large.txt` \|
	\| 43 \| INT32 \| 1 \| quantize.imatrix.entries_count \| 382 \|
	\| 44 \| INT32 \| 1 \| quantize.imatrix.chunks_count \| 4978 \|

	## Tensors Overview ~31B Elements

	Total number of elements in all tensors: 30532122624 Elements

	- [Qwen3-30B-A3B-Q3\_K\_M.gguf - GGUF Internal File Dump](#qwen3-30b-a3b-q3_k_mgguf---gguf-internal-file-dump)
	- [Key Value Metadata Store](#key-value-metadata-store)
	- [Tensors Overview ~31B Elements](#tensors-overview-31b-elements)
	- [Tensor Data Offset](#tensor-data-offset)
	- [Base Tensor Group : ~622M Elements](#base-tensor-group--622m-elements)
	- [Block 0 Tensor Group : ~623M Elements](#block-0-tensor-group--623m-elements)
	- [Block 1 Tensor Group : ~623M Elements](#block-1-tensor-group--623m-elements)
	- [Block 2 Tensor Group : ~623M Elements](#block-2-tensor-group--623m-elements)
	- [Block 3 Tensor Group : ~623M Elements](#block-3-tensor-group--623m-elements)
	- [Block 4 Tensor Group : ~623M Elements](#block-4-tensor-group--623m-elements)
	- [Block 5 Tensor Group : ~623M Elements](#block-5-tensor-group--623m-elements)
	- [Block 6 Tensor Group : ~623M Elements](#block-6-tensor-group--623m-elements)
	- [Block 7 Tensor Group : ~623M Elements](#block-7-tensor-group--623m-elements)
	- [Block 8 Tensor Group : ~623M Elements](#block-8-tensor-group--623m-elements)
	- [Block 9 Tensor Group : ~623M Elements](#block-9-tensor-group--623m-elements)
	- [Block 10 Tensor Group : ~623M Elements](#block-10-tensor-group--623m-elements)
	- [Block 11 Tensor Group : ~623M Elements](#block-11-tensor-group--623m-elements)
	- [Block 12 Tensor Group : ~623M Elements](#block-12-tensor-group--623m-elements)
	- [Block 13 Tensor Group : ~623M Elements](#block-13-tensor-group--623m-elements)
	- [Block 14 Tensor Group : ~623M Elements](#block-14-tensor-group--623m-elements)
	- [Block 15 Tensor Group : ~623M Elements](#block-15-tensor-group--623m-elements)
	- [Block 16 Tensor Group : ~623M Elements](#block-16-tensor-group--623m-elements)
	- [Block 17 Tensor Group : ~623M Elements](#block-17-tensor-group--623m-elements)
	- [Block 18 Tensor Group : ~623M Elements](#block-18-tensor-group--623m-elements)
	- [Block 19 Tensor Group : ~623M Elements](#block-19-tensor-group--623m-elements)
	- [Block 20 Tensor Group : ~623M Elements](#block-20-tensor-group--623m-elements)
	- [Block 21 Tensor Group : ~623M Elements](#block-21-tensor-group--623m-elements)
	- [Block 22 Tensor Group : ~623M Elements](#block-22-tensor-group--623m-elements)
	- [Block 23 Tensor Group : ~623M Elements](#block-23-tensor-group--623m-elements)
	- [Block 24 Tensor Group : ~623M Elements](#block-24-tensor-group--623m-elements)
	- [Block 25 Tensor Group : ~623M Elements](#block-25-tensor-group--623m-elements)
	- [Block 26 Tensor Group : ~623M Elements](#block-26-tensor-group--623m-elements)
	- [Block 27 Tensor Group : ~623M Elements](#block-27-tensor-group--623m-elements)
	- [Block 28 Tensor Group : ~623M Elements](#block-28-tensor-group--623m-elements)
	- [Block 29 Tensor Group : ~623M Elements](#block-29-tensor-group--623m-elements)
	- [Block 30 Tensor Group : ~623M Elements](#block-30-tensor-group--623m-elements)
	- [Block 31 Tensor Group : ~623M Elements](#block-31-tensor-group--623m-elements)
	- [Block 32 Tensor Group : ~623M Elements](#block-32-tensor-group--623m-elements)
	- [Block 33 Tensor Group : ~623M Elements](#block-33-tensor-group--623m-elements)
	- [Block 34 Tensor Group : ~623M Elements](#block-34-tensor-group--623m-elements)
	- [Block 35 Tensor Group : ~623M Elements](#block-35-tensor-group--623m-elements)
	- [Block 36 Tensor Group : ~623M Elements](#block-36-tensor-group--623m-elements)
	- [Block 37 Tensor Group : ~623M Elements](#block-37-tensor-group--623m-elements)
	- [Block 38 Tensor Group : ~623M Elements](#block-38-tensor-group--623m-elements)
	- [Block 39 Tensor Group : ~623M Elements](#block-39-tensor-group--623m-elements)
	- [Block 40 Tensor Group : ~623M Elements](#block-40-tensor-group--623m-elements)
	- [Block 41 Tensor Group : ~623M Elements](#block-41-tensor-group--623m-elements)
	- [Block 42 Tensor Group : ~623M Elements](#block-42-tensor-group--623m-elements)
	- [Block 43 Tensor Group : ~623M Elements](#block-43-tensor-group--623m-elements)
	- [Block 44 Tensor Group : ~623M Elements](#block-44-tensor-group--623m-elements)
	- [Block 45 Tensor Group : ~623M Elements](#block-45-tensor-group--623m-elements)
	- [Block 46 Tensor Group : ~623M Elements](#block-46-tensor-group--623m-elements)
	- [Block 47 Tensor Group : ~623M Elements](#block-47-tensor-group--623m-elements)

	### Tensor Data Offset

	This table contains the offset and data segment relative to start of file

	\| T_ID \| Tensor Layer Name \| Data Offset (B) \| Data Size (B) \|
	\|-----:\|:----------------------------\|-----------------:\|-----------------:\|
	\| 0 \| output.weight \| 0x5b18c0 \| 0x7f82800 \|
	\| 1 \| output_norm.weight \| 0x85340c0 \| 0x2000 \|
	\| 2 \| token_embd.weight \| 0x85360c0 \| 0x7f82800 \|
	\| 3 \| blk.0.attn_k.weight \| 0x104b88c0 \| 0x54000 \|
	\| 4 \| blk.0.attn_k_norm.weight \| 0x1050c8c0 \| 0x200 \|
	\| 5 \| blk.0.attn_norm.weight \| 0x1050cac0 \| 0x2000 \|
	\| 6 \| blk.0.attn_output.weight \| 0x1050eac0 \| 0x480000 \|
	\| 7 \| blk.0.attn_q.weight \| 0x1098eac0 \| 0x2a0000 \|
	\| 8 \| blk.0.attn_q_norm.weight \| 0x10c2eac0 \| 0x200 \|
	\| 9 \| blk.0.attn_v.weight \| 0x10c2ecc0 \| 0x6e000 \|
	\| 10 \| blk.0.ffn_down_exps.weight \| 0x10c9ccc0 \| 0x6c00000 \|
	\| 11 \| blk.0.ffn_gate_exps.weight \| 0x1789ccc0 \| 0x3f00000 \|
	\| 12 \| blk.0.ffn_gate_inp.weight \| 0x1b79ccc0 \| 0x100000 \|
	\| 13 \| blk.0.ffn_norm.weight \| 0x1b89ccc0 \| 0x2000 \|
	\| 14 \| blk.0.ffn_up_exps.weight \| 0x1b89ecc0 \| 0x3f00000 \|
	\| 15 \| blk.1.attn_k.weight \| 0x1f79ecc0 \| 0x54000 \|
	\| 16 \| blk.1.attn_k_norm.weight \| 0x1f7f2cc0 \| 0x200 \|
	\| 17 \| blk.1.attn_norm.weight \| 0x1f7f2ec0 \| 0x2000 \|
	\| 18 \| blk.1.attn_output.weight \| 0x1f7f4ec0 \| 0x480000 \|
	\| 19 \| blk.1.attn_q.weight \| 0x1fc74ec0 \| 0x2a0000 \|
	\| 20 \| blk.1.attn_q_norm.weight \| 0x1ff14ec0 \| 0x200 \|
	\| 21 \| blk.1.attn_v.weight \| 0x1ff150c0 \| 0x6e000 \|
	\| 22 \| blk.1.ffn_down_exps.weight \| 0x1ff830c0 \| 0x6c00000 \|
	\| 23 \| blk.1.ffn_gate_exps.weight \| 0x26b830c0 \| 0x3f00000 \|
	\| 24 \| blk.1.ffn_gate_inp.weight \| 0x2aa830c0 \| 0x100000 \|
	\| 25 \| blk.1.ffn_norm.weight \| 0x2ab830c0 \| 0x2000 \|
	\| 26 \| blk.1.ffn_up_exps.weight \| 0x2ab850c0 \| 0x3f00000 \|
	\| 27 \| blk.2.attn_k.weight \| 0x2ea850c0 \| 0x54000 \|
	\| 28 \| blk.2.attn_k_norm.weight \| 0x2ead90c0 \| 0x200 \|
	\| 29 \| blk.2.attn_norm.weight \| 0x2ead92c0 \| 0x2000 \|
	\| 30 \| blk.2.attn_output.weight \| 0x2eadb2c0 \| 0x480000 \|
	\| 31 \| blk.2.attn_q.weight \| 0x2ef5b2c0 \| 0x2a0000 \|
	\| 32 \| blk.2.attn_q_norm.weight \| 0x2f1fb2c0 \| 0x200 \|
	\| 33 \| blk.2.attn_v.weight \| 0x2f1fb4c0 \| 0x6e000 \|
	\| 34 \| blk.2.ffn_down_exps.weight \| 0x2f2694c0 \| 0x6c00000 \|
	\| 35 \| blk.2.ffn_gate_exps.weight \| 0x35e694c0 \| 0x3f00000 \|
	\| 36 \| blk.2.ffn_gate_inp.weight \| 0x39d694c0 \| 0x100000 \|
	\| 37 \| blk.2.ffn_norm.weight \| 0x39e694c0 \| 0x2000 \|
	\| 38 \| blk.2.ffn_up_exps.weight \| 0x39e6b4c0 \| 0x3f00000 \|
	\| 39 \| blk.3.attn_k.weight \| 0x3dd6b4c0 \| 0x54000 \|
	\| 40 \| blk.3.attn_k_norm.weight \| 0x3ddbf4c0 \| 0x200 \|
	\| 41 \| blk.3.attn_norm.weight \| 0x3ddbf6c0 \| 0x2000 \|
	\| 42 \| blk.3.attn_output.weight \| 0x3ddc16c0 \| 0x480000 \|
	\| 43 \| blk.3.attn_q.weight \| 0x3e2416c0 \| 0x2a0000 \|
	\| 44 \| blk.3.attn_q_norm.weight \| 0x3e4e16c0 \| 0x200 \|
	\| 45 \| blk.3.attn_v.weight \| 0x3e4e18c0 \| 0x6e000 \|
	\| 46 \| blk.3.ffn_down_exps.weight \| 0x3e54f8c0 \| 0x6c00000 \|
	\| 47 \| blk.3.ffn_gate_exps.weight \| 0x4514f8c0 \| 0x3f00000 \|
	\| 48 \| blk.3.ffn_gate_inp.weight \| 0x4904f8c0 \| 0x100000 \|
	\| 49 \| blk.3.ffn_norm.weight \| 0x4914f8c0 \| 0x2000 \|
	\| 50 \| blk.3.ffn_up_exps.weight \| 0x491518c0 \| 0x3f00000 \|
	\| 51 \| blk.4.attn_k.weight \| 0x4d0518c0 \| 0x54000 \|
	\| 52 \| blk.4.attn_k_norm.weight \| 0x4d0a58c0 \| 0x200 \|
	\| 53 \| blk.4.attn_norm.weight \| 0x4d0a5ac0 \| 0x2000 \|
	\| 54 \| blk.4.attn_output.weight \| 0x4d0a7ac0 \| 0x480000 \|
	\| 55 \| blk.4.attn_q.weight \| 0x4d527ac0 \| 0x2a0000 \|
	\| 56 \| blk.4.attn_q_norm.weight \| 0x4d7c7ac0 \| 0x200 \|
	\| 57 \| blk.4.attn_v.weight \| 0x4d7c7cc0 \| 0x6e000 \|
	\| 58 \| blk.4.ffn_down_exps.weight \| 0x4d835cc0 \| 0x6c00000 \|
	\| 59 \| blk.4.ffn_gate_exps.weight \| 0x54435cc0 \| 0x3f00000 \|
	\| 60 \| blk.4.ffn_gate_inp.weight \| 0x58335cc0 \| 0x100000 \|
	\| 61 \| blk.4.ffn_norm.weight \| 0x58435cc0 \| 0x2000 \|
	\| 62 \| blk.4.ffn_up_exps.weight \| 0x58437cc0 \| 0x3f00000 \|
	\| 63 \| blk.5.attn_k.weight \| 0x5c337cc0 \| 0x54000 \|
	\| 64 \| blk.5.attn_k_norm.weight \| 0x5c38bcc0 \| 0x200 \|
	\| 65 \| blk.5.attn_norm.weight \| 0x5c38bec0 \| 0x2000 \|
	\| 66 \| blk.5.attn_output.weight \| 0x5c38dec0 \| 0x480000 \|
	\| 67 \| blk.5.attn_q.weight \| 0x5c80dec0 \| 0x2a0000 \|
	\| 68 \| blk.5.attn_q_norm.weight \| 0x5caadec0 \| 0x200 \|
	\| 69 \| blk.5.attn_v.weight \| 0x5caae0c0 \| 0x6e000 \|
	\| 70 \| blk.5.ffn_down_exps.weight \| 0x5cb1c0c0 \| 0x6c00000 \|
	\| 71 \| blk.5.ffn_gate_exps.weight \| 0x6371c0c0 \| 0x3f00000 \|
	\| 72 \| blk.5.ffn_gate_inp.weight \| 0x6761c0c0 \| 0x100000 \|
	\| 73 \| blk.5.ffn_norm.weight \| 0x6771c0c0 \| 0x2000 \|
	\| 74 \| blk.5.ffn_up_exps.weight \| 0x6771e0c0 \| 0x3f00000 \|
	\| 75 \| blk.6.attn_k.weight \| 0x6b61e0c0 \| 0x54000 \|
	\| 76 \| blk.6.attn_k_norm.weight \| 0x6b6720c0 \| 0x200 \|
	\| 77 \| blk.6.attn_norm.weight \| 0x6b6722c0 \| 0x2000 \|
	\| 78 \| blk.6.attn_output.weight \| 0x6b6742c0 \| 0x480000 \|
	\| 79 \| blk.6.attn_q.weight \| 0x6baf42c0 \| 0x2a0000 \|
	\| 80 \| blk.6.attn_q_norm.weight \| 0x6bd942c0 \| 0x200 \|
	\| 81 \| blk.6.attn_v.weight \| 0x6bd944c0 \| 0x6e000 \|
	\| 82 \| blk.6.ffn_down_exps.weight \| 0x6be024c0 \| 0x6c00000 \|
	\| 83 \| blk.6.ffn_gate_exps.weight \| 0x72a024c0 \| 0x3f00000 \|
	\| 84 \| blk.6.ffn_gate_inp.weight \| 0x769024c0 \| 0x100000 \|
	\| 85 \| blk.6.ffn_norm.weight \| 0x76a024c0 \| 0x2000 \|
	\| 86 \| blk.6.ffn_up_exps.weight \| 0x76a044c0 \| 0x3f00000 \|
	\| 87 \| blk.7.attn_k.weight \| 0x7a9044c0 \| 0x54000 \|
	\| 88 \| blk.7.attn_k_norm.weight \| 0x7a9584c0 \| 0x200 \|
	\| 89 \| blk.7.attn_norm.weight \| 0x7a9586c0 \| 0x2000 \|
	\| 90 \| blk.7.attn_output.weight \| 0x7a95a6c0 \| 0x480000 \|
	\| 91 \| blk.7.attn_q.weight \| 0x7adda6c0 \| 0x2a0000 \|
	\| 92 \| blk.7.attn_q_norm.weight \| 0x7b07a6c0 \| 0x200 \|
	\| 93 \| blk.7.attn_v.weight \| 0x7b07a8c0 \| 0x6e000 \|
	\| 94 \| blk.7.ffn_down_exps.weight \| 0x7b0e88c0 \| 0x6c00000 \|
	\| 95 \| blk.7.ffn_gate_exps.weight \| 0x81ce88c0 \| 0x3f00000 \|
	\| 96 \| blk.7.ffn_gate_inp.weight \| 0x85be88c0 \| 0x100000 \|
	\| 97 \| blk.7.ffn_norm.weight \| 0x85ce88c0 \| 0x2000 \|
	\| 98 \| blk.7.ffn_up_exps.weight \| 0x85cea8c0 \| 0x3f00000 \|
	\| 99 \| blk.8.attn_k.weight \| 0x89bea8c0 \| 0x54000 \|
	\| 100 \| blk.8.attn_k_norm.weight \| 0x89c3e8c0 \| 0x200 \|
	\| 101 \| blk.8.attn_norm.weight \| 0x89c3eac0 \| 0x2000 \|
	\| 102 \| blk.8.attn_output.weight \| 0x89c40ac0 \| 0x480000 \|
	\| 103 \| blk.8.attn_q.weight \| 0x8a0c0ac0 \| 0x2a0000 \|
	\| 104 \| blk.8.attn_q_norm.weight \| 0x8a360ac0 \| 0x200 \|
	\| 105 \| blk.8.attn_v.weight \| 0x8a360cc0 \| 0x6e000 \|
	\| 106 \| blk.8.ffn_down_exps.weight \| 0x8a3cecc0 \| 0x6c00000 \|
	\| 107 \| blk.8.ffn_gate_exps.weight \| 0x90fcecc0 \| 0x3f00000 \|
	\| 108 \| blk.8.ffn_gate_inp.weight \| 0x94ececc0 \| 0x100000 \|
	\| 109 \| blk.8.ffn_norm.weight \| 0x94fcecc0 \| 0x2000 \|
	\| 110 \| blk.8.ffn_up_exps.weight \| 0x94fd0cc0 \| 0x3f00000 \|
	\| 111 \| blk.9.attn_k.weight \| 0x98ed0cc0 \| 0x54000 \|
	\| 112 \| blk.9.attn_k_norm.weight \| 0x98f24cc0 \| 0x200 \|
	\| 113 \| blk.9.attn_norm.weight \| 0x98f24ec0 \| 0x2000 \|
	\| 114 \| blk.9.attn_output.weight \| 0x98f26ec0 \| 0x480000 \|
	\| 115 \| blk.9.attn_q.weight \| 0x993a6ec0 \| 0x2a0000 \|
	\| 116 \| blk.9.attn_q_norm.weight \| 0x99646ec0 \| 0x200 \|
	\| 117 \| blk.9.attn_v.weight \| 0x996470c0 \| 0x6e000 \|
	\| 118 \| blk.9.ffn_down_exps.weight \| 0x996b50c0 \| 0x6c00000 \|
	\| 119 \| blk.9.ffn_gate_exps.weight \| 0xa02b50c0 \| 0x3f00000 \|
	\| 120 \| blk.9.ffn_gate_inp.weight \| 0xa41b50c0 \| 0x100000 \|
	\| 121 \| blk.9.ffn_norm.weight \| 0xa42b50c0 \| 0x2000 \|
	\| 122 \| blk.9.ffn_up_exps.weight \| 0xa42b70c0 \| 0x3f00000 \|
	\| 123 \| blk.10.attn_k.weight \| 0xa81b70c0 \| 0x54000 \|
	\| 124 \| blk.10.attn_k_norm.weight \| 0xa820b0c0 \| 0x200 \|
	\| 125 \| blk.10.attn_norm.weight \| 0xa820b2c0 \| 0x2000 \|
	\| 126 \| blk.10.attn_output.weight \| 0xa820d2c0 \| 0x480000 \|
	\| 127 \| blk.10.attn_q.weight \| 0xa868d2c0 \| 0x2a0000 \|
	\| 128 \| blk.10.attn_q_norm.weight \| 0xa892d2c0 \| 0x200 \|
	\| 129 \| blk.10.attn_v.weight \| 0xa892d4c0 \| 0x6e000 \|
	\| 130 \| blk.10.ffn_down_exps.weight \| 0xa899b4c0 \| 0x6c00000 \|
	\| 131 \| blk.10.ffn_gate_exps.weight \| 0xaf59b4c0 \| 0x3f00000 \|
	\| 132 \| blk.10.ffn_gate_inp.weight \| 0xb349b4c0 \| 0x100000 \|
	\| 133 \| blk.10.ffn_norm.weight \| 0xb359b4c0 \| 0x2000 \|
	\| 134 \| blk.10.ffn_up_exps.weight \| 0xb359d4c0 \| 0x3f00000 \|
	\| 135 \| blk.11.attn_k.weight \| 0xb749d4c0 \| 0x54000 \|
	\| 136 \| blk.11.attn_k_norm.weight \| 0xb74f14c0 \| 0x200 \|
	\| 137 \| blk.11.attn_norm.weight \| 0xb74f16c0 \| 0x2000 \|
	\| 138 \| blk.11.attn_output.weight \| 0xb74f36c0 \| 0x480000 \|
	\| 139 \| blk.11.attn_q.weight \| 0xb79736c0 \| 0x2a0000 \|
	\| 140 \| blk.11.attn_q_norm.weight \| 0xb7c136c0 \| 0x200 \|
	\| 141 \| blk.11.attn_v.weight \| 0xb7c138c0 \| 0x6e000 \|
	\| 142 \| blk.11.ffn_down_exps.weight \| 0xb7c818c0 \| 0x6c00000 \|
	\| 143 \| blk.11.ffn_gate_exps.weight \| 0xbe8818c0 \| 0x3f00000 \|
	\| 144 \| blk.11.ffn_gate_inp.weight \| 0xc27818c0 \| 0x100000 \|
	\| 145 \| blk.11.ffn_norm.weight \| 0xc28818c0 \| 0x2000 \|
	\| 146 \| blk.11.ffn_up_exps.weight \| 0xc28838c0 \| 0x3f00000 \|
	\| 147 \| blk.12.attn_k.weight \| 0xc67838c0 \| 0x54000 \|
	\| 148 \| blk.12.attn_k_norm.weight \| 0xc67d78c0 \| 0x200 \|
	\| 149 \| blk.12.attn_norm.weight \| 0xc67d7ac0 \| 0x2000 \|
	\| 150 \| blk.12.attn_output.weight \| 0xc67d9ac0 \| 0x480000 \|
	\| 151 \| blk.12.attn_q.weight \| 0xc6c59ac0 \| 0x2a0000 \|
	\| 152 \| blk.12.attn_q_norm.weight \| 0xc6ef9ac0 \| 0x200 \|
	\| 153 \| blk.12.attn_v.weight \| 0xc6ef9cc0 \| 0x6e000 \|
	\| 154 \| blk.12.ffn_down_exps.weight \| 0xc6f67cc0 \| 0x6c00000 \|
	\| 155 \| blk.12.ffn_gate_exps.weight \| 0xcdb67cc0 \| 0x3f00000 \|
	\| 156 \| blk.12.ffn_gate_inp.weight \| 0xd1a67cc0 \| 0x100000 \|
	\| 157 \| blk.12.ffn_norm.weight \| 0xd1b67cc0 \| 0x2000 \|
	\| 158 \| blk.12.ffn_up_exps.weight \| 0xd1b69cc0 \| 0x3f00000 \|
	\| 159 \| blk.13.attn_k.weight \| 0xd5a69cc0 \| 0x54000 \|
	\| 160 \| blk.13.attn_k_norm.weight \| 0xd5abdcc0 \| 0x200 \|
	\| 161 \| blk.13.attn_norm.weight \| 0xd5abdec0 \| 0x2000 \|
	\| 162 \| blk.13.attn_output.weight \| 0xd5abfec0 \| 0x480000 \|
	\| 163 \| blk.13.attn_q.weight \| 0xd5f3fec0 \| 0x2a0000 \|
	\| 164 \| blk.13.attn_q_norm.weight \| 0xd61dfec0 \| 0x200 \|
	\| 165 \| blk.13.attn_v.weight \| 0xd61e00c0 \| 0x6e000 \|
	\| 166 \| blk.13.ffn_down_exps.weight \| 0xd624e0c0 \| 0x6c00000 \|
	\| 167 \| blk.13.ffn_gate_exps.weight \| 0xdce4e0c0 \| 0x5280000 \|
	\| 168 \| blk.13.ffn_gate_inp.weight \| 0xe20ce0c0 \| 0x100000 \|
	\| 169 \| blk.13.ffn_norm.weight \| 0xe21ce0c0 \| 0x2000 \|
	\| 170 \| blk.13.ffn_up_exps.weight \| 0xe21d00c0 \| 0x5280000 \|
	\| 171 \| blk.14.attn_k.weight \| 0xe74500c0 \| 0x54000 \|
	\| 172 \| blk.14.attn_k_norm.weight \| 0xe74a40c0 \| 0x200 \|
	\| 173 \| blk.14.attn_norm.weight \| 0xe74a42c0 \| 0x2000 \|
	\| 174 \| blk.14.attn_output.weight \| 0xe74a62c0 \| 0x480000 \|
	\| 175 \| blk.14.attn_q.weight \| 0xe79262c0 \| 0x2a0000 \|
	\| 176 \| blk.14.attn_q_norm.weight \| 0xe7bc62c0 \| 0x200 \|
	\| 177 \| blk.14.attn_v.weight \| 0xe7bc64c0 \| 0x6e000 \|
	\| 178 \| blk.14.ffn_down_exps.weight \| 0xe7c344c0 \| 0x6c00000 \|
	\| 179 \| blk.14.ffn_gate_exps.weight \| 0xee8344c0 \| 0x3f00000 \|
	\| 180 \| blk.14.ffn_gate_inp.weight \| 0xf27344c0 \| 0x100000 \|
	\| 181 \| blk.14.ffn_norm.weight \| 0xf28344c0 \| 0x2000 \|
	\| 182 \| blk.14.ffn_up_exps.weight \| 0xf28364c0 \| 0x3f00000 \|
	\| 183 \| blk.15.attn_k.weight \| 0xf67364c0 \| 0x54000 \|
	\| 184 \| blk.15.attn_k_norm.weight \| 0xf678a4c0 \| 0x200 \|
	\| 185 \| blk.15.attn_norm.weight \| 0xf678a6c0 \| 0x2000 \|
	\| 186 \| blk.15.attn_output.weight \| 0xf678c6c0 \| 0x480000 \|
	\| 187 \| blk.15.attn_q.weight \| 0xf6c0c6c0 \| 0x2a0000 \|
	\| 188 \| blk.15.attn_q_norm.weight \| 0xf6eac6c0 \| 0x200 \|
	\| 189 \| blk.15.attn_v.weight \| 0xf6eac8c0 \| 0x6e000 \|
	\| 190 \| blk.15.ffn_down_exps.weight \| 0xf6f1a8c0 \| 0x6c00000 \|
	\| 191 \| blk.15.ffn_gate_exps.weight \| 0xfdb1a8c0 \| 0x5280000 \|
	\| 192 \| blk.15.ffn_gate_inp.weight \| 0x102d9a8c0 \| 0x100000 \|
	\| 193 \| blk.15.ffn_norm.weight \| 0x102e9a8c0 \| 0x2000 \|
	\| 194 \| blk.15.ffn_up_exps.weight \| 0x102e9c8c0 \| 0x5280000 \|
	\| 195 \| blk.16.attn_k.weight \| 0x10811c8c0 \| 0x54000 \|
	\| 196 \| blk.16.attn_k_norm.weight \| 0x1081708c0 \| 0x200 \|
	\| 197 \| blk.16.attn_norm.weight \| 0x108170ac0 \| 0x2000 \|
	\| 198 \| blk.16.attn_output.weight \| 0x108172ac0 \| 0x480000 \|
	\| 199 \| blk.16.attn_q.weight \| 0x1085f2ac0 \| 0x2a0000 \|
	\| 200 \| blk.16.attn_q_norm.weight \| 0x108892ac0 \| 0x200 \|
	\| 201 \| blk.16.attn_v.weight \| 0x108892cc0 \| 0x6e000 \|
	\| 202 \| blk.16.ffn_down_exps.weight \| 0x108900cc0 \| 0x6c00000 \|
	\| 203 \| blk.16.ffn_gate_exps.weight \| 0x10f500cc0 \| 0x3f00000 \|
	\| 204 \| blk.16.ffn_gate_inp.weight \| 0x113400cc0 \| 0x100000 \|
	\| 205 \| blk.16.ffn_norm.weight \| 0x113500cc0 \| 0x2000 \|
	\| 206 \| blk.16.ffn_up_exps.weight \| 0x113502cc0 \| 0x3f00000 \|
	\| 207 \| blk.17.attn_k.weight \| 0x117402cc0 \| 0x54000 \|
	\| 208 \| blk.17.attn_k_norm.weight \| 0x117456cc0 \| 0x200 \|
	\| 209 \| blk.17.attn_norm.weight \| 0x117456ec0 \| 0x2000 \|
	\| 210 \| blk.17.attn_output.weight \| 0x117458ec0 \| 0x480000 \|
	\| 211 \| blk.17.attn_q.weight \| 0x1178d8ec0 \| 0x2a0000 \|
	\| 212 \| blk.17.attn_q_norm.weight \| 0x117b78ec0 \| 0x200 \|
	\| 213 \| blk.17.attn_v.weight \| 0x117b790c0 \| 0x6e000 \|
	\| 214 \| blk.17.ffn_down_exps.weight \| 0x117be70c0 \| 0x6c00000 \|
	\| 215 \| blk.17.ffn_gate_exps.weight \| 0x11e7e70c0 \| 0x3f00000 \|
	\| 216 \| blk.17.ffn_gate_inp.weight \| 0x1226e70c0 \| 0x100000 \|
	\| 217 \| blk.17.ffn_norm.weight \| 0x1227e70c0 \| 0x2000 \|
	\| 218 \| blk.17.ffn_up_exps.weight \| 0x1227e90c0 \| 0x3f00000 \|
	\| 219 \| blk.18.attn_k.weight \| 0x1266e90c0 \| 0x54000 \|
	\| 220 \| blk.18.attn_k_norm.weight \| 0x12673d0c0 \| 0x200 \|
	\| 221 \| blk.18.attn_norm.weight \| 0x12673d2c0 \| 0x2000 \|
	\| 222 \| blk.18.attn_output.weight \| 0x12673f2c0 \| 0x480000 \|
	\| 223 \| blk.18.attn_q.weight \| 0x126bbf2c0 \| 0x2a0000 \|
	\| 224 \| blk.18.attn_q_norm.weight \| 0x126e5f2c0 \| 0x200 \|
	\| 225 \| blk.18.attn_v.weight \| 0x126e5f4c0 \| 0x6e000 \|
	\| 226 \| blk.18.ffn_down_exps.weight \| 0x126ecd4c0 \| 0x6c00000 \|
	\| 227 \| blk.18.ffn_gate_exps.weight \| 0x12dacd4c0 \| 0x3f00000 \|
	\| 228 \| blk.18.ffn_gate_inp.weight \| 0x1319cd4c0 \| 0x100000 \|
	\| 229 \| blk.18.ffn_norm.weight \| 0x131acd4c0 \| 0x2000 \|
	\| 230 \| blk.18.ffn_up_exps.weight \| 0x131acf4c0 \| 0x3f00000 \|
	\| 231 \| blk.19.attn_k.weight \| 0x1359cf4c0 \| 0x54000 \|
	\| 232 \| blk.19.attn_k_norm.weight \| 0x135a234c0 \| 0x200 \|
	\| 233 \| blk.19.attn_norm.weight \| 0x135a236c0 \| 0x2000 \|
	\| 234 \| blk.19.attn_output.weight \| 0x135a256c0 \| 0x480000 \|
	\| 235 \| blk.19.attn_q.weight \| 0x135ea56c0 \| 0x2a0000 \|
	\| 236 \| blk.19.attn_q_norm.weight \| 0x1361456c0 \| 0x200 \|
	\| 237 \| blk.19.attn_v.weight \| 0x1361458c0 \| 0x6e000 \|
	\| 238 \| blk.19.ffn_down_exps.weight \| 0x1361b38c0 \| 0x6c00000 \|
	\| 239 \| blk.19.ffn_gate_exps.weight \| 0x13cdb38c0 \| 0x3f00000 \|
	\| 240 \| blk.19.ffn_gate_inp.weight \| 0x140cb38c0 \| 0x100000 \|
	\| 241 \| blk.19.ffn_norm.weight \| 0x140db38c0 \| 0x2000 \|
	\| 242 \| blk.19.ffn_up_exps.weight \| 0x140db58c0 \| 0x3f00000 \|
	\| 243 \| blk.20.attn_k.weight \| 0x144cb58c0 \| 0x54000 \|
	\| 244 \| blk.20.attn_k_norm.weight \| 0x144d098c0 \| 0x200 \|
	\| 245 \| blk.20.attn_norm.weight \| 0x144d09ac0 \| 0x2000 \|
	\| 246 \| blk.20.attn_output.weight \| 0x144d0bac0 \| 0x480000 \|
	\| 247 \| blk.20.attn_q.weight \| 0x14518bac0 \| 0x2a0000 \|
	\| 248 \| blk.20.attn_q_norm.weight \| 0x14542bac0 \| 0x200 \|
	\| 249 \| blk.20.attn_v.weight \| 0x14542bcc0 \| 0x6e000 \|
	\| 250 \| blk.20.ffn_down_exps.weight \| 0x145499cc0 \| 0x6c00000 \|
	\| 251 \| blk.20.ffn_gate_exps.weight \| 0x14c099cc0 \| 0x3f00000 \|
	\| 252 \| blk.20.ffn_gate_inp.weight \| 0x14ff99cc0 \| 0x100000 \|
	\| 253 \| blk.20.ffn_norm.weight \| 0x150099cc0 \| 0x2000 \|
	\| 254 \| blk.20.ffn_up_exps.weight \| 0x15009bcc0 \| 0x3f00000 \|
	\| 255 \| blk.21.attn_k.weight \| 0x153f9bcc0 \| 0x54000 \|
	\| 256 \| blk.21.attn_k_norm.weight \| 0x153fefcc0 \| 0x200 \|
	\| 257 \| blk.21.attn_norm.weight \| 0x153fefec0 \| 0x2000 \|
	\| 258 \| blk.21.attn_output.weight \| 0x153ff1ec0 \| 0x480000 \|
	\| 259 \| blk.21.attn_q.weight \| 0x154471ec0 \| 0x2a0000 \|
	\| 260 \| blk.21.attn_q_norm.weight \| 0x154711ec0 \| 0x200 \|
	\| 261 \| blk.21.attn_v.weight \| 0x1547120c0 \| 0x6e000 \|
	\| 262 \| blk.21.ffn_down_exps.weight \| 0x1547800c0 \| 0x6c00000 \|
	\| 263 \| blk.21.ffn_gate_exps.weight \| 0x15b3800c0 \| 0x3f00000 \|
	\| 264 \| blk.21.ffn_gate_inp.weight \| 0x15f2800c0 \| 0x100000 \|
	\| 265 \| blk.21.ffn_norm.weight \| 0x15f3800c0 \| 0x2000 \|
	\| 266 \| blk.21.ffn_up_exps.weight \| 0x15f3820c0 \| 0x3f00000 \|
	\| 267 \| blk.22.attn_k.weight \| 0x1632820c0 \| 0x54000 \|
	\| 268 \| blk.22.attn_k_norm.weight \| 0x1632d60c0 \| 0x200 \|
	\| 269 \| blk.22.attn_norm.weight \| 0x1632d62c0 \| 0x2000 \|
	\| 270 \| blk.22.attn_output.weight \| 0x1632d82c0 \| 0x480000 \|
	\| 271 \| blk.22.attn_q.weight \| 0x1637582c0 \| 0x2a0000 \|
	\| 272 \| blk.22.attn_q_norm.weight \| 0x1639f82c0 \| 0x200 \|
	\| 273 \| blk.22.attn_v.weight \| 0x1639f84c0 \| 0x6e000 \|
	\| 274 \| blk.22.ffn_down_exps.weight \| 0x163a664c0 \| 0x6c00000 \|
	\| 275 \| blk.22.ffn_gate_exps.weight \| 0x16a6664c0 \| 0x3f00000 \|
	\| 276 \| blk.22.ffn_gate_inp.weight \| 0x16e5664c0 \| 0x100000 \|
	\| 277 \| blk.22.ffn_norm.weight \| 0x16e6664c0 \| 0x2000 \|
	\| 278 \| blk.22.ffn_up_exps.weight \| 0x16e6684c0 \| 0x3f00000 \|
	\| 279 \| blk.23.attn_k.weight \| 0x1725684c0 \| 0x54000 \|
	\| 280 \| blk.23.attn_k_norm.weight \| 0x1725bc4c0 \| 0x200 \|
	\| 281 \| blk.23.attn_norm.weight \| 0x1725bc6c0 \| 0x2000 \|
	\| 282 \| blk.23.attn_output.weight \| 0x1725be6c0 \| 0x480000 \|
	\| 283 \| blk.23.attn_q.weight \| 0x172a3e6c0 \| 0x2a0000 \|
	\| 284 \| blk.23.attn_q_norm.weight \| 0x172cde6c0 \| 0x200 \|
	\| 285 \| blk.23.attn_v.weight \| 0x172cde8c0 \| 0x6e000 \|
	\| 286 \| blk.23.ffn_down_exps.weight \| 0x172d4c8c0 \| 0x6c00000 \|
	\| 287 \| blk.23.ffn_gate_exps.weight \| 0x17994c8c0 \| 0x3f00000 \|
	\| 288 \| blk.23.ffn_gate_inp.weight \| 0x17d84c8c0 \| 0x100000 \|
	\| 289 \| blk.23.ffn_norm.weight \| 0x17d94c8c0 \| 0x2000 \|
	\| 290 \| blk.23.ffn_up_exps.weight \| 0x17d94e8c0 \| 0x3f00000 \|
	\| 291 \| blk.24.attn_k.weight \| 0x18184e8c0 \| 0x6e000 \|
	\| 292 \| blk.24.attn_k_norm.weight \| 0x1818bc8c0 \| 0x200 \|
	\| 293 \| blk.24.attn_norm.weight \| 0x1818bcac0 \| 0x2000 \|
	\| 294 \| blk.24.attn_output.weight \| 0x1818beac0 \| 0x480000 \|
	\| 295 \| blk.24.attn_q.weight \| 0x181d3eac0 \| 0x370000 \|
	\| 296 \| blk.24.attn_q_norm.weight \| 0x1820aeac0 \| 0x200 \|
	\| 297 \| blk.24.attn_v.weight \| 0x1820aecc0 \| 0x90000 \|
	\| 298 \| blk.24.ffn_down_exps.weight \| 0x18213ecc0 \| 0x6c00000 \|
	\| 299 \| blk.24.ffn_gate_exps.weight \| 0x188d3ecc0 \| 0x3f00000 \|
	\| 300 \| blk.24.ffn_gate_inp.weight \| 0x18cc3ecc0 \| 0x100000 \|
	\| 301 \| blk.24.ffn_norm.weight \| 0x18cd3ecc0 \| 0x2000 \|
	\| 302 \| blk.24.ffn_up_exps.weight \| 0x18cd40cc0 \| 0x3f00000 \|
	\| 303 \| blk.25.attn_k.weight \| 0x190c40cc0 \| 0x6e000 \|
	\| 304 \| blk.25.attn_k_norm.weight \| 0x190caecc0 \| 0x200 \|
	\| 305 \| blk.25.attn_norm.weight \| 0x190caeec0 \| 0x2000 \|
	\| 306 \| blk.25.attn_output.weight \| 0x190cb0ec0 \| 0x480000 \|
	\| 307 \| blk.25.attn_q.weight \| 0x191130ec0 \| 0x370000 \|
	\| 308 \| blk.25.attn_q_norm.weight \| 0x1914a0ec0 \| 0x200 \|
	\| 309 \| blk.25.attn_v.weight \| 0x1914a10c0 \| 0x90000 \|
	\| 310 \| blk.25.ffn_down_exps.weight \| 0x1915310c0 \| 0x6c00000 \|
	\| 311 \| blk.25.ffn_gate_exps.weight \| 0x1981310c0 \| 0x5280000 \|
	\| 312 \| blk.25.ffn_gate_inp.weight \| 0x19d3b10c0 \| 0x100000 \|
	\| 313 \| blk.25.ffn_norm.weight \| 0x19d4b10c0 \| 0x2000 \|
	\| 314 \| blk.25.ffn_up_exps.weight \| 0x19d4b30c0 \| 0x5280000 \|
	\| 315 \| blk.26.attn_k.weight \| 0x1a27330c0 \| 0x6e000 \|
	\| 316 \| blk.26.attn_k_norm.weight \| 0x1a27a10c0 \| 0x200 \|
	\| 317 \| blk.26.attn_norm.weight \| 0x1a27a12c0 \| 0x2000 \|
	\| 318 \| blk.26.attn_output.weight \| 0x1a27a32c0 \| 0x480000 \|
	\| 319 \| blk.26.attn_q.weight \| 0x1a2c232c0 \| 0x370000 \|
	\| 320 \| blk.26.attn_q_norm.weight \| 0x1a2f932c0 \| 0x200 \|
	\| 321 \| blk.26.attn_v.weight \| 0x1a2f934c0 \| 0x90000 \|
	\| 322 \| blk.26.ffn_down_exps.weight \| 0x1a30234c0 \| 0x6c00000 \|
	\| 323 \| blk.26.ffn_gate_exps.weight \| 0x1a9c234c0 \| 0x3f00000 \|
	\| 324 \| blk.26.ffn_gate_inp.weight \| 0x1adb234c0 \| 0x100000 \|
	\| 325 \| blk.26.ffn_norm.weight \| 0x1adc234c0 \| 0x2000 \|
	\| 326 \| blk.26.ffn_up_exps.weight \| 0x1adc254c0 \| 0x3f00000 \|
	\| 327 \| blk.27.attn_k.weight \| 0x1b1b254c0 \| 0x6e000 \|
	\| 328 \| blk.27.attn_k_norm.weight \| 0x1b1b934c0 \| 0x200 \|
	\| 329 \| blk.27.attn_norm.weight \| 0x1b1b936c0 \| 0x2000 \|
	\| 330 \| blk.27.attn_output.weight \| 0x1b1b956c0 \| 0x480000 \|
	\| 331 \| blk.27.attn_q.weight \| 0x1b20156c0 \| 0x370000 \|
	\| 332 \| blk.27.attn_q_norm.weight \| 0x1b23856c0 \| 0x200 \|
	\| 333 \| blk.27.attn_v.weight \| 0x1b23858c0 \| 0x90000 \|
	\| 334 \| blk.27.ffn_down_exps.weight \| 0x1b24158c0 \| 0x6c00000 \|
	\| 335 \| blk.27.ffn_gate_exps.weight \| 0x1b90158c0 \| 0x5280000 \|
	\| 336 \| blk.27.ffn_gate_inp.weight \| 0x1be2958c0 \| 0x100000 \|
	\| 337 \| blk.27.ffn_norm.weight \| 0x1be3958c0 \| 0x2000 \|
	\| 338 \| blk.27.ffn_up_exps.weight \| 0x1be3978c0 \| 0x5280000 \|
	\| 339 \| blk.28.attn_k.weight \| 0x1c36178c0 \| 0x6e000 \|
	\| 340 \| blk.28.attn_k_norm.weight \| 0x1c36858c0 \| 0x200 \|
	\| 341 \| blk.28.attn_norm.weight \| 0x1c3685ac0 \| 0x2000 \|
	\| 342 \| blk.28.attn_output.weight \| 0x1c3687ac0 \| 0x480000 \|
	\| 343 \| blk.28.attn_q.weight \| 0x1c3b07ac0 \| 0x370000 \|
	\| 344 \| blk.28.attn_q_norm.weight \| 0x1c3e77ac0 \| 0x200 \|
	\| 345 \| blk.28.attn_v.weight \| 0x1c3e77cc0 \| 0x90000 \|
	\| 346 \| blk.28.ffn_down_exps.weight \| 0x1c3f07cc0 \| 0x6c00000 \|
	\| 347 \| blk.28.ffn_gate_exps.weight \| 0x1cab07cc0 \| 0x5280000 \|
	\| 348 \| blk.28.ffn_gate_inp.weight \| 0x1cfd87cc0 \| 0x100000 \|
	\| 349 \| blk.28.ffn_norm.weight \| 0x1cfe87cc0 \| 0x2000 \|
	\| 350 \| blk.28.ffn_up_exps.weight \| 0x1cfe89cc0 \| 0x5280000 \|
	\| 351 \| blk.29.attn_k.weight \| 0x1d5109cc0 \| 0x6e000 \|
	\| 352 \| blk.29.attn_k_norm.weight \| 0x1d5177cc0 \| 0x200 \|
	\| 353 \| blk.29.attn_norm.weight \| 0x1d5177ec0 \| 0x2000 \|
	\| 354 \| blk.29.attn_output.weight \| 0x1d5179ec0 \| 0x480000 \|
	\| 355 \| blk.29.attn_q.weight \| 0x1d55f9ec0 \| 0x370000 \|
	\| 356 \| blk.29.attn_q_norm.weight \| 0x1d5969ec0 \| 0x200 \|
	\| 357 \| blk.29.attn_v.weight \| 0x1d596a0c0 \| 0x90000 \|
	\| 358 \| blk.29.ffn_down_exps.weight \| 0x1d59fa0c0 \| 0x6c00000 \|
	\| 359 \| blk.29.ffn_gate_exps.weight \| 0x1dc5fa0c0 \| 0x5280000 \|
	\| 360 \| blk.29.ffn_gate_inp.weight \| 0x1e187a0c0 \| 0x100000 \|
	\| 361 \| blk.29.ffn_norm.weight \| 0x1e197a0c0 \| 0x2000 \|
	\| 362 \| blk.29.ffn_up_exps.weight \| 0x1e197c0c0 \| 0x5280000 \|
	\| 363 \| blk.30.attn_k.weight \| 0x1e6bfc0c0 \| 0x6e000 \|
	\| 364 \| blk.30.attn_k_norm.weight \| 0x1e6c6a0c0 \| 0x200 \|
	\| 365 \| blk.30.attn_norm.weight \| 0x1e6c6a2c0 \| 0x2000 \|
	\| 366 \| blk.30.attn_output.weight \| 0x1e6c6c2c0 \| 0x480000 \|
	\| 367 \| blk.30.attn_q.weight \| 0x1e70ec2c0 \| 0x370000 \|
	\| 368 \| blk.30.attn_q_norm.weight \| 0x1e745c2c0 \| 0x200 \|
	\| 369 \| blk.30.attn_v.weight \| 0x1e745c4c0 \| 0x90000 \|
	\| 370 \| blk.30.ffn_down_exps.weight \| 0x1e74ec4c0 \| 0x6c00000 \|
	\| 371 \| blk.30.ffn_gate_exps.weight \| 0x1ee0ec4c0 \| 0x5280000 \|
	\| 372 \| blk.30.ffn_gate_inp.weight \| 0x1f336c4c0 \| 0x100000 \|
	\| 373 \| blk.30.ffn_norm.weight \| 0x1f346c4c0 \| 0x2000 \|
	\| 374 \| blk.30.ffn_up_exps.weight \| 0x1f346e4c0 \| 0x5280000 \|
	\| 375 \| blk.31.attn_k.weight \| 0x1f86ee4c0 \| 0x6e000 \|
	\| 376 \| blk.31.attn_k_norm.weight \| 0x1f875c4c0 \| 0x200 \|
	\| 377 \| blk.31.attn_norm.weight \| 0x1f875c6c0 \| 0x2000 \|
	\| 378 \| blk.31.attn_output.weight \| 0x1f875e6c0 \| 0x480000 \|
	\| 379 \| blk.31.attn_q.weight \| 0x1f8bde6c0 \| 0x370000 \|
	\| 380 \| blk.31.attn_q_norm.weight \| 0x1f8f4e6c0 \| 0x200 \|
	\| 381 \| blk.31.attn_v.weight \| 0x1f8f4e8c0 \| 0x90000 \|
	\| 382 \| blk.31.ffn_down_exps.weight \| 0x1f8fde8c0 \| 0x6c00000 \|
	\| 383 \| blk.31.ffn_gate_exps.weight \| 0x1ffbde8c0 \| 0x5280000 \|
	\| 384 \| blk.31.ffn_gate_inp.weight \| 0x204e5e8c0 \| 0x100000 \|
	\| 385 \| blk.31.ffn_norm.weight \| 0x204f5e8c0 \| 0x2000 \|
	\| 386 \| blk.31.ffn_up_exps.weight \| 0x204f608c0 \| 0x5280000 \|
	\| 387 \| blk.32.attn_k.weight \| 0x20a1e08c0 \| 0x6e000 \|
	\| 388 \| blk.32.attn_k_norm.weight \| 0x20a24e8c0 \| 0x200 \|
	\| 389 \| blk.32.attn_norm.weight \| 0x20a24eac0 \| 0x2000 \|
	\| 390 \| blk.32.attn_output.weight \| 0x20a250ac0 \| 0x480000 \|
	\| 391 \| blk.32.attn_q.weight \| 0x20a6d0ac0 \| 0x370000 \|
	\| 392 \| blk.32.attn_q_norm.weight \| 0x20aa40ac0 \| 0x200 \|
	\| 393 \| blk.32.attn_v.weight \| 0x20aa40cc0 \| 0x90000 \|
	\| 394 \| blk.32.ffn_down_exps.weight \| 0x20aad0cc0 \| 0x6c00000 \|
	\| 395 \| blk.32.ffn_gate_exps.weight \| 0x2116d0cc0 \| 0x5280000 \|
	\| 396 \| blk.32.ffn_gate_inp.weight \| 0x216950cc0 \| 0x100000 \|
	\| 397 \| blk.32.ffn_norm.weight \| 0x216a50cc0 \| 0x2000 \|
	\| 398 \| blk.32.ffn_up_exps.weight \| 0x216a52cc0 \| 0x5280000 \|
	\| 399 \| blk.33.attn_k.weight \| 0x21bcd2cc0 \| 0x6e000 \|
	\| 400 \| blk.33.attn_k_norm.weight \| 0x21bd40cc0 \| 0x200 \|
	\| 401 \| blk.33.attn_norm.weight \| 0x21bd40ec0 \| 0x2000 \|
	\| 402 \| blk.33.attn_output.weight \| 0x21bd42ec0 \| 0x480000 \|
	\| 403 \| blk.33.attn_q.weight \| 0x21c1c2ec0 \| 0x370000 \|
	\| 404 \| blk.33.attn_q_norm.weight \| 0x21c532ec0 \| 0x200 \|
	\| 405 \| blk.33.attn_v.weight \| 0x21c5330c0 \| 0x90000 \|
	\| 406 \| blk.33.ffn_down_exps.weight \| 0x21c5c30c0 \| 0x6c00000 \|
	\| 407 \| blk.33.ffn_gate_exps.weight \| 0x2231c30c0 \| 0x5280000 \|
	\| 408 \| blk.33.ffn_gate_inp.weight \| 0x2284430c0 \| 0x100000 \|
	\| 409 \| blk.33.ffn_norm.weight \| 0x2285430c0 \| 0x2000 \|
	\| 410 \| blk.33.ffn_up_exps.weight \| 0x2285450c0 \| 0x5280000 \|
	\| 411 \| blk.34.attn_k.weight \| 0x22d7c50c0 \| 0x6e000 \|
	\| 412 \| blk.34.attn_k_norm.weight \| 0x22d8330c0 \| 0x200 \|
	\| 413 \| blk.34.attn_norm.weight \| 0x22d8332c0 \| 0x2000 \|
	\| 414 \| blk.34.attn_output.weight \| 0x22d8352c0 \| 0x480000 \|
	\| 415 \| blk.34.attn_q.weight \| 0x22dcb52c0 \| 0x370000 \|
	\| 416 \| blk.34.attn_q_norm.weight \| 0x22e0252c0 \| 0x200 \|
	\| 417 \| blk.34.attn_v.weight \| 0x22e0254c0 \| 0x90000 \|
	\| 418 \| blk.34.ffn_down_exps.weight \| 0x22e0b54c0 \| 0x6c00000 \|
	\| 419 \| blk.34.ffn_gate_exps.weight \| 0x234cb54c0 \| 0x5280000 \|
	\| 420 \| blk.34.ffn_gate_inp.weight \| 0x239f354c0 \| 0x100000 \|
	\| 421 \| blk.34.ffn_norm.weight \| 0x23a0354c0 \| 0x2000 \|
	\| 422 \| blk.34.ffn_up_exps.weight \| 0x23a0374c0 \| 0x5280000 \|
	\| 423 \| blk.35.attn_k.weight \| 0x23f2b74c0 \| 0x6e000 \|
	\| 424 \| blk.35.attn_k_norm.weight \| 0x23f3254c0 \| 0x200 \|
	\| 425 \| blk.35.attn_norm.weight \| 0x23f3256c0 \| 0x2000 \|
	\| 426 \| blk.35.attn_output.weight \| 0x23f3276c0 \| 0x480000 \|
	\| 427 \| blk.35.attn_q.weight \| 0x23f7a76c0 \| 0x370000 \|
	\| 428 \| blk.35.attn_q_norm.weight \| 0x23fb176c0 \| 0x200 \|
	\| 429 \| blk.35.attn_v.weight \| 0x23fb178c0 \| 0x90000 \|
	\| 430 \| blk.35.ffn_down_exps.weight \| 0x23fba78c0 \| 0x6c00000 \|
	\| 431 \| blk.35.ffn_gate_exps.weight \| 0x2467a78c0 \| 0x5280000 \|
	\| 432 \| blk.35.ffn_gate_inp.weight \| 0x24ba278c0 \| 0x100000 \|
	\| 433 \| blk.35.ffn_norm.weight \| 0x24bb278c0 \| 0x2000 \|
	\| 434 \| blk.35.ffn_up_exps.weight \| 0x24bb298c0 \| 0x5280000 \|
	\| 435 \| blk.36.attn_k.weight \| 0x250da98c0 \| 0x6e000 \|
	\| 436 \| blk.36.attn_k_norm.weight \| 0x250e178c0 \| 0x200 \|
	\| 437 \| blk.36.attn_norm.weight \| 0x250e17ac0 \| 0x2000 \|
	\| 438 \| blk.36.attn_output.weight \| 0x250e19ac0 \| 0x480000 \|
	\| 439 \| blk.36.attn_q.weight \| 0x251299ac0 \| 0x370000 \|
	\| 440 \| blk.36.attn_q_norm.weight \| 0x251609ac0 \| 0x200 \|
	\| 441 \| blk.36.attn_v.weight \| 0x251609cc0 \| 0x90000 \|
	\| 442 \| blk.36.ffn_down_exps.weight \| 0x251699cc0 \| 0x6c00000 \|
	\| 443 \| blk.36.ffn_gate_exps.weight \| 0x258299cc0 \| 0x5280000 \|
	\| 444 \| blk.36.ffn_gate_inp.weight \| 0x25d519cc0 \| 0x100000 \|
	\| 445 \| blk.36.ffn_norm.weight \| 0x25d619cc0 \| 0x2000 \|
	\| 446 \| blk.36.ffn_up_exps.weight \| 0x25d61bcc0 \| 0x5280000 \|
	\| 447 \| blk.37.attn_k.weight \| 0x26289bcc0 \| 0x6e000 \|
	\| 448 \| blk.37.attn_k_norm.weight \| 0x262909cc0 \| 0x200 \|
	\| 449 \| blk.37.attn_norm.weight \| 0x262909ec0 \| 0x2000 \|
	\| 450 \| blk.37.attn_output.weight \| 0x26290bec0 \| 0x480000 \|
	\| 451 \| blk.37.attn_q.weight \| 0x262d8bec0 \| 0x370000 \|
	\| 452 \| blk.37.attn_q_norm.weight \| 0x2630fbec0 \| 0x200 \|
	\| 453 \| blk.37.attn_v.weight \| 0x2630fc0c0 \| 0x90000 \|
	\| 454 \| blk.37.ffn_down_exps.weight \| 0x26318c0c0 \| 0x6c00000 \|
	\| 455 \| blk.37.ffn_gate_exps.weight \| 0x269d8c0c0 \| 0x5280000 \|
	\| 456 \| blk.37.ffn_gate_inp.weight \| 0x26f00c0c0 \| 0x100000 \|
	\| 457 \| blk.37.ffn_norm.weight \| 0x26f10c0c0 \| 0x2000 \|
	\| 458 \| blk.37.ffn_up_exps.weight \| 0x26f10e0c0 \| 0x5280000 \|
	\| 459 \| blk.38.attn_k.weight \| 0x27438e0c0 \| 0x6e000 \|
	\| 460 \| blk.38.attn_k_norm.weight \| 0x2743fc0c0 \| 0x200 \|
	\| 461 \| blk.38.attn_norm.weight \| 0x2743fc2c0 \| 0x2000 \|
	\| 462 \| blk.38.attn_output.weight \| 0x2743fe2c0 \| 0x480000 \|
	\| 463 \| blk.38.attn_q.weight \| 0x27487e2c0 \| 0x370000 \|
	\| 464 \| blk.38.attn_q_norm.weight \| 0x274bee2c0 \| 0x200 \|
	\| 465 \| blk.38.attn_v.weight \| 0x274bee4c0 \| 0x90000 \|
	\| 466 \| blk.38.ffn_down_exps.weight \| 0x274c7e4c0 \| 0x6c00000 \|
	\| 467 \| blk.38.ffn_gate_exps.weight \| 0x27b87e4c0 \| 0x5280000 \|
	\| 468 \| blk.38.ffn_gate_inp.weight \| 0x280afe4c0 \| 0x100000 \|
	\| 469 \| blk.38.ffn_norm.weight \| 0x280bfe4c0 \| 0x2000 \|
	\| 470 \| blk.38.ffn_up_exps.weight \| 0x280c004c0 \| 0x5280000 \|
	\| 471 \| blk.39.attn_k.weight \| 0x285e804c0 \| 0x6e000 \|
	\| 472 \| blk.39.attn_k_norm.weight \| 0x285eee4c0 \| 0x200 \|
	\| 473 \| blk.39.attn_norm.weight \| 0x285eee6c0 \| 0x2000 \|
	\| 474 \| blk.39.attn_output.weight \| 0x285ef06c0 \| 0x480000 \|
	\| 475 \| blk.39.attn_q.weight \| 0x2863706c0 \| 0x370000 \|
	\| 476 \| blk.39.attn_q_norm.weight \| 0x2866e06c0 \| 0x200 \|
	\| 477 \| blk.39.attn_v.weight \| 0x2866e08c0 \| 0x90000 \|
	\| 478 \| blk.39.ffn_down_exps.weight \| 0x2867708c0 \| 0x6c00000 \|
	\| 479 \| blk.39.ffn_gate_exps.weight \| 0x28d3708c0 \| 0x5280000 \|
	\| 480 \| blk.39.ffn_gate_inp.weight \| 0x2925f08c0 \| 0x100000 \|
	\| 481 \| blk.39.ffn_norm.weight \| 0x2926f08c0 \| 0x2000 \|
	\| 482 \| blk.39.ffn_up_exps.weight \| 0x2926f28c0 \| 0x5280000 \|
	\| 483 \| blk.40.attn_k.weight \| 0x2979728c0 \| 0x6e000 \|
	\| 484 \| blk.40.attn_k_norm.weight \| 0x2979e08c0 \| 0x200 \|
	\| 485 \| blk.40.attn_norm.weight \| 0x2979e0ac0 \| 0x2000 \|
	\| 486 \| blk.40.attn_output.weight \| 0x2979e2ac0 \| 0x480000 \|
	\| 487 \| blk.40.attn_q.weight \| 0x297e62ac0 \| 0x370000 \|
	\| 488 \| blk.40.attn_q_norm.weight \| 0x2981d2ac0 \| 0x200 \|
	\| 489 \| blk.40.attn_v.weight \| 0x2981d2cc0 \| 0x90000 \|
	\| 490 \| blk.40.ffn_down_exps.weight \| 0x298262cc0 \| 0x6c00000 \|
	\| 491 \| blk.40.ffn_gate_exps.weight \| 0x29ee62cc0 \| 0x5280000 \|
	\| 492 \| blk.40.ffn_gate_inp.weight \| 0x2a40e2cc0 \| 0x100000 \|
	\| 493 \| blk.40.ffn_norm.weight \| 0x2a41e2cc0 \| 0x2000 \|
	\| 494 \| blk.40.ffn_up_exps.weight \| 0x2a41e4cc0 \| 0x5280000 \|
	\| 495 \| blk.41.attn_k.weight \| 0x2a9464cc0 \| 0x6e000 \|
	\| 496 \| blk.41.attn_k_norm.weight \| 0x2a94d2cc0 \| 0x200 \|
	\| 497 \| blk.41.attn_norm.weight \| 0x2a94d2ec0 \| 0x2000 \|
	\| 498 \| blk.41.attn_output.weight \| 0x2a94d4ec0 \| 0x480000 \|
	\| 499 \| blk.41.attn_q.weight \| 0x2a9954ec0 \| 0x370000 \|
	\| 500 \| blk.41.attn_q_norm.weight \| 0x2a9cc4ec0 \| 0x200 \|
	\| 501 \| blk.41.attn_v.weight \| 0x2a9cc50c0 \| 0x90000 \|
	\| 502 \| blk.41.ffn_down_exps.weight \| 0x2a9d550c0 \| 0x6c00000 \|
	\| 503 \| blk.41.ffn_gate_exps.weight \| 0x2b09550c0 \| 0x5280000 \|
	\| 504 \| blk.41.ffn_gate_inp.weight \| 0x2b5bd50c0 \| 0x100000 \|
	\| 505 \| blk.41.ffn_norm.weight \| 0x2b5cd50c0 \| 0x2000 \|
	\| 506 \| blk.41.ffn_up_exps.weight \| 0x2b5cd70c0 \| 0x5280000 \|
	\| 507 \| blk.42.attn_k.weight \| 0x2baf570c0 \| 0x6e000 \|
	\| 508 \| blk.42.attn_k_norm.weight \| 0x2bafc50c0 \| 0x200 \|
	\| 509 \| blk.42.attn_norm.weight \| 0x2bafc52c0 \| 0x2000 \|
	\| 510 \| blk.42.attn_output.weight \| 0x2bafc72c0 \| 0x480000 \|
	\| 511 \| blk.42.attn_q.weight \| 0x2bb4472c0 \| 0x370000 \|
	\| 512 \| blk.42.attn_q_norm.weight \| 0x2bb7b72c0 \| 0x200 \|
	\| 513 \| blk.42.attn_v.weight \| 0x2bb7b74c0 \| 0x90000 \|
	\| 514 \| blk.42.ffn_down_exps.weight \| 0x2bb8474c0 \| 0x6c00000 \|
	\| 515 \| blk.42.ffn_gate_exps.weight \| 0x2c24474c0 \| 0x5280000 \|
	\| 516 \| blk.42.ffn_gate_inp.weight \| 0x2c76c74c0 \| 0x100000 \|
	\| 517 \| blk.42.ffn_norm.weight \| 0x2c77c74c0 \| 0x2000 \|
	\| 518 \| blk.42.ffn_up_exps.weight \| 0x2c77c94c0 \| 0x5280000 \|
	\| 519 \| blk.43.attn_k.weight \| 0x2cca494c0 \| 0x6e000 \|
	\| 520 \| blk.43.attn_k_norm.weight \| 0x2ccab74c0 \| 0x200 \|
	\| 521 \| blk.43.attn_norm.weight \| 0x2ccab76c0 \| 0x2000 \|
	\| 522 \| blk.43.attn_output.weight \| 0x2ccab96c0 \| 0x480000 \|
	\| 523 \| blk.43.attn_q.weight \| 0x2ccf396c0 \| 0x370000 \|
	\| 524 \| blk.43.attn_q_norm.weight \| 0x2cd2a96c0 \| 0x200 \|
	\| 525 \| blk.43.attn_v.weight \| 0x2cd2a98c0 \| 0x90000 \|
	\| 526 \| blk.43.ffn_down_exps.weight \| 0x2cd3398c0 \| 0x6c00000 \|
	\| 527 \| blk.43.ffn_gate_exps.weight \| 0x2d3f398c0 \| 0x5280000 \|
	\| 528 \| blk.43.ffn_gate_inp.weight \| 0x2d91b98c0 \| 0x100000 \|
	\| 529 \| blk.43.ffn_norm.weight \| 0x2d92b98c0 \| 0x2000 \|
	\| 530 \| blk.43.ffn_up_exps.weight \| 0x2d92bb8c0 \| 0x5280000 \|
	\| 531 \| blk.44.attn_k.weight \| 0x2de53b8c0 \| 0x6e000 \|
	\| 532 \| blk.44.attn_k_norm.weight \| 0x2de5a98c0 \| 0x200 \|
	\| 533 \| blk.44.attn_norm.weight \| 0x2de5a9ac0 \| 0x2000 \|
	\| 534 \| blk.44.attn_output.weight \| 0x2de5abac0 \| 0x480000 \|
	\| 535 \| blk.44.attn_q.weight \| 0x2dea2bac0 \| 0x370000 \|
	\| 536 \| blk.44.attn_q_norm.weight \| 0x2ded9bac0 \| 0x200 \|
	\| 537 \| blk.44.attn_v.weight \| 0x2ded9bcc0 \| 0x90000 \|
	\| 538 \| blk.44.ffn_down_exps.weight \| 0x2dee2bcc0 \| 0x6c00000 \|
	\| 539 \| blk.44.ffn_gate_exps.weight \| 0x2e5a2bcc0 \| 0x5280000 \|
	\| 540 \| blk.44.ffn_gate_inp.weight \| 0x2eacabcc0 \| 0x100000 \|
	\| 541 \| blk.44.ffn_norm.weight \| 0x2eadabcc0 \| 0x2000 \|
	\| 542 \| blk.44.ffn_up_exps.weight \| 0x2eadadcc0 \| 0x5280000 \|
	\| 543 \| blk.45.attn_k.weight \| 0x2f002dcc0 \| 0x6e000 \|
	\| 544 \| blk.45.attn_k_norm.weight \| 0x2f009bcc0 \| 0x200 \|
	\| 545 \| blk.45.attn_norm.weight \| 0x2f009bec0 \| 0x2000 \|
	\| 546 \| blk.45.attn_output.weight \| 0x2f009dec0 \| 0x480000 \|
	\| 547 \| blk.45.attn_q.weight \| 0x2f051dec0 \| 0x370000 \|
	\| 548 \| blk.45.attn_q_norm.weight \| 0x2f088dec0 \| 0x200 \|
	\| 549 \| blk.45.attn_v.weight \| 0x2f088e0c0 \| 0x90000 \|
	\| 550 \| blk.45.ffn_down_exps.weight \| 0x2f091e0c0 \| 0x6c00000 \|
	\| 551 \| blk.45.ffn_gate_exps.weight \| 0x2f751e0c0 \| 0x5280000 \|
	\| 552 \| blk.45.ffn_gate_inp.weight \| 0x2fc79e0c0 \| 0x100000 \|
	\| 553 \| blk.45.ffn_norm.weight \| 0x2fc89e0c0 \| 0x2000 \|
	\| 554 \| blk.45.ffn_up_exps.weight \| 0x2fc8a00c0 \| 0x5280000 \|
	\| 555 \| blk.46.attn_k.weight \| 0x301b200c0 \| 0x6e000 \|
	\| 556 \| blk.46.attn_k_norm.weight \| 0x301b8e0c0 \| 0x200 \|
	\| 557 \| blk.46.attn_norm.weight \| 0x301b8e2c0 \| 0x2000 \|
	\| 558 \| blk.46.attn_output.weight \| 0x301b902c0 \| 0x480000 \|
	\| 559 \| blk.46.attn_q.weight \| 0x3020102c0 \| 0x370000 \|
	\| 560 \| blk.46.attn_q_norm.weight \| 0x3023802c0 \| 0x200 \|
	\| 561 \| blk.46.attn_v.weight \| 0x3023804c0 \| 0x90000 \|
	\| 562 \| blk.46.ffn_down_exps.weight \| 0x3024104c0 \| 0x6c00000 \|
	\| 563 \| blk.46.ffn_gate_exps.weight \| 0x3090104c0 \| 0x5280000 \|
	\| 564 \| blk.46.ffn_gate_inp.weight \| 0x30e2904c0 \| 0x100000 \|
	\| 565 \| blk.46.ffn_norm.weight \| 0x30e3904c0 \| 0x2000 \|
	\| 566 \| blk.46.ffn_up_exps.weight \| 0x30e3924c0 \| 0x5280000 \|
	\| 567 \| blk.47.attn_k.weight \| 0x3136124c0 \| 0x6e000 \|
	\| 568 \| blk.47.attn_k_norm.weight \| 0x3136804c0 \| 0x200 \|
	\| 569 \| blk.47.attn_norm.weight \| 0x3136806c0 \| 0x2000 \|
	\| 570 \| blk.47.attn_output.weight \| 0x3136826c0 \| 0x480000 \|
	\| 571 \| blk.47.attn_q.weight \| 0x313b026c0 \| 0x370000 \|
	\| 572 \| blk.47.attn_q_norm.weight \| 0x313e726c0 \| 0x200 \|
	\| 573 \| blk.47.attn_v.weight \| 0x313e728c0 \| 0x90000 \|
	\| 574 \| blk.47.ffn_down_exps.weight \| 0x313f028c0 \| 0x6c00000 \|
	\| 575 \| blk.47.ffn_gate_exps.weight \| 0x31ab028c0 \| 0x5280000 \|
	\| 576 \| blk.47.ffn_gate_inp.weight \| 0x31fd828c0 \| 0x100000 \|
	\| 577 \| blk.47.ffn_norm.weight \| 0x31fe828c0 \| 0x2000 \|
	\| 578 \| blk.47.ffn_up_exps.weight \| 0x31fe848c0 \| 0x5280000 \|

	### <a name="base">Base Tensor Group : ~622M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:-------------------\|:---------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 0 \| output.weight \| Output (W) \| (~311M) 311164928 \| 2048 x 151936 x 1 x 1 \| Q3_K \|
	\| 1 \| output_norm.weight \| Output Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 2 \| token_embd.weight \| Token Embedding (W) \| (~311M) 311164928 \| 2048 x 151936 x 1 x 1 \| Q3_K \|

	- Total elements in base: (~622M) 622331904
	- Percentage of total elements: 2.04%


	### <a name="blk_0">Block 0 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 3 \| blk.0.attn_k.weight \| Block 0 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q2_K \|
	\| 4 \| blk.0.attn_k_norm.weight \| Block 0 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 5 \| blk.0.attn_norm.weight \| Block 0 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 6 \| blk.0.attn_output.weight \| Block 0 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 7 \| blk.0.attn_q.weight \| Block 0 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q2_K \|
	\| 8 \| blk.0.attn_q_norm.weight \| Block 0 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 9 \| blk.0.attn_v.weight \| Block 0 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 10 \| blk.0.ffn_down_exps.weight \| Block 0 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 11 \| blk.0.ffn_gate_exps.weight \| Block 0 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|
	\| 12 \| blk.0.ffn_gate_inp.weight \| Block 0 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 13 \| blk.0.ffn_norm.weight \| Block 0 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 14 \| blk.0.ffn_up_exps.weight \| Block 0 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|

	- Total elements in blk.0: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_1">Block 1 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 15 \| blk.1.attn_k.weight \| Block 1 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q2_K \|
	\| 16 \| blk.1.attn_k_norm.weight \| Block 1 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 17 \| blk.1.attn_norm.weight \| Block 1 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 18 \| blk.1.attn_output.weight \| Block 1 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 19 \| blk.1.attn_q.weight \| Block 1 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q2_K \|
	\| 20 \| blk.1.attn_q_norm.weight \| Block 1 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 21 \| blk.1.attn_v.weight \| Block 1 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 22 \| blk.1.ffn_down_exps.weight \| Block 1 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 23 \| blk.1.ffn_gate_exps.weight \| Block 1 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|
	\| 24 \| blk.1.ffn_gate_inp.weight \| Block 1 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 25 \| blk.1.ffn_norm.weight \| Block 1 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 26 \| blk.1.ffn_up_exps.weight \| Block 1 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|

	- Total elements in blk.1: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_2">Block 2 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 27 \| blk.2.attn_k.weight \| Block 2 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q2_K \|
	\| 28 \| blk.2.attn_k_norm.weight \| Block 2 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 29 \| blk.2.attn_norm.weight \| Block 2 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 30 \| blk.2.attn_output.weight \| Block 2 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 31 \| blk.2.attn_q.weight \| Block 2 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q2_K \|
	\| 32 \| blk.2.attn_q_norm.weight \| Block 2 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 33 \| blk.2.attn_v.weight \| Block 2 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 34 \| blk.2.ffn_down_exps.weight \| Block 2 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 35 \| blk.2.ffn_gate_exps.weight \| Block 2 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|
	\| 36 \| blk.2.ffn_gate_inp.weight \| Block 2 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 37 \| blk.2.ffn_norm.weight \| Block 2 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 38 \| blk.2.ffn_up_exps.weight \| Block 2 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|

	- Total elements in blk.2: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_3">Block 3 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 39 \| blk.3.attn_k.weight \| Block 3 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q2_K \|
	\| 40 \| blk.3.attn_k_norm.weight \| Block 3 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 41 \| blk.3.attn_norm.weight \| Block 3 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 42 \| blk.3.attn_output.weight \| Block 3 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 43 \| blk.3.attn_q.weight \| Block 3 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q2_K \|
	\| 44 \| blk.3.attn_q_norm.weight \| Block 3 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 45 \| blk.3.attn_v.weight \| Block 3 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 46 \| blk.3.ffn_down_exps.weight \| Block 3 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 47 \| blk.3.ffn_gate_exps.weight \| Block 3 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|
	\| 48 \| blk.3.ffn_gate_inp.weight \| Block 3 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 49 \| blk.3.ffn_norm.weight \| Block 3 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 50 \| blk.3.ffn_up_exps.weight \| Block 3 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|

	- Total elements in blk.3: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_4">Block 4 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 51 \| blk.4.attn_k.weight \| Block 4 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q2_K \|
	\| 52 \| blk.4.attn_k_norm.weight \| Block 4 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 53 \| blk.4.attn_norm.weight \| Block 4 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 54 \| blk.4.attn_output.weight \| Block 4 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 55 \| blk.4.attn_q.weight \| Block 4 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q2_K \|
	\| 56 \| blk.4.attn_q_norm.weight \| Block 4 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 57 \| blk.4.attn_v.weight \| Block 4 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 58 \| blk.4.ffn_down_exps.weight \| Block 4 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 59 \| blk.4.ffn_gate_exps.weight \| Block 4 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|
	\| 60 \| blk.4.ffn_gate_inp.weight \| Block 4 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 61 \| blk.4.ffn_norm.weight \| Block 4 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 62 \| blk.4.ffn_up_exps.weight \| Block 4 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|

	- Total elements in blk.4: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_5">Block 5 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 63 \| blk.5.attn_k.weight \| Block 5 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q2_K \|
	\| 64 \| blk.5.attn_k_norm.weight \| Block 5 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 65 \| blk.5.attn_norm.weight \| Block 5 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 66 \| blk.5.attn_output.weight \| Block 5 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 67 \| blk.5.attn_q.weight \| Block 5 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q2_K \|
	\| 68 \| blk.5.attn_q_norm.weight \| Block 5 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 69 \| blk.5.attn_v.weight \| Block 5 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 70 \| blk.5.ffn_down_exps.weight \| Block 5 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 71 \| blk.5.ffn_gate_exps.weight \| Block 5 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|
	\| 72 \| blk.5.ffn_gate_inp.weight \| Block 5 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 73 \| blk.5.ffn_norm.weight \| Block 5 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 74 \| blk.5.ffn_up_exps.weight \| Block 5 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|

	- Total elements in blk.5: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_6">Block 6 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 75 \| blk.6.attn_k.weight \| Block 6 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q2_K \|
	\| 76 \| blk.6.attn_k_norm.weight \| Block 6 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 77 \| blk.6.attn_norm.weight \| Block 6 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 78 \| blk.6.attn_output.weight \| Block 6 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 79 \| blk.6.attn_q.weight \| Block 6 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q2_K \|
	\| 80 \| blk.6.attn_q_norm.weight \| Block 6 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 81 \| blk.6.attn_v.weight \| Block 6 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 82 \| blk.6.ffn_down_exps.weight \| Block 6 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 83 \| blk.6.ffn_gate_exps.weight \| Block 6 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|
	\| 84 \| blk.6.ffn_gate_inp.weight \| Block 6 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 85 \| blk.6.ffn_norm.weight \| Block 6 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 86 \| blk.6.ffn_up_exps.weight \| Block 6 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|

	- Total elements in blk.6: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_7">Block 7 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 87 \| blk.7.attn_k.weight \| Block 7 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q2_K \|
	\| 88 \| blk.7.attn_k_norm.weight \| Block 7 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 89 \| blk.7.attn_norm.weight \| Block 7 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 90 \| blk.7.attn_output.weight \| Block 7 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 91 \| blk.7.attn_q.weight \| Block 7 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q2_K \|
	\| 92 \| blk.7.attn_q_norm.weight \| Block 7 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 93 \| blk.7.attn_v.weight \| Block 7 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 94 \| blk.7.ffn_down_exps.weight \| Block 7 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 95 \| blk.7.ffn_gate_exps.weight \| Block 7 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|
	\| 96 \| blk.7.ffn_gate_inp.weight \| Block 7 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 97 \| blk.7.ffn_norm.weight \| Block 7 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 98 \| blk.7.ffn_up_exps.weight \| Block 7 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|

	- Total elements in blk.7: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_8">Block 8 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 99 \| blk.8.attn_k.weight \| Block 8 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q2_K \|
	\| 100 \| blk.8.attn_k_norm.weight \| Block 8 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 101 \| blk.8.attn_norm.weight \| Block 8 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 102 \| blk.8.attn_output.weight \| Block 8 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 103 \| blk.8.attn_q.weight \| Block 8 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q2_K \|
	\| 104 \| blk.8.attn_q_norm.weight \| Block 8 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 105 \| blk.8.attn_v.weight \| Block 8 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 106 \| blk.8.ffn_down_exps.weight \| Block 8 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 107 \| blk.8.ffn_gate_exps.weight \| Block 8 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|
	\| 108 \| blk.8.ffn_gate_inp.weight \| Block 8 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 109 \| blk.8.ffn_norm.weight \| Block 8 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 110 \| blk.8.ffn_up_exps.weight \| Block 8 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|

	- Total elements in blk.8: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_9">Block 9 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 111 \| blk.9.attn_k.weight \| Block 9 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q2_K \|
	\| 112 \| blk.9.attn_k_norm.weight \| Block 9 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 113 \| blk.9.attn_norm.weight \| Block 9 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 114 \| blk.9.attn_output.weight \| Block 9 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 115 \| blk.9.attn_q.weight \| Block 9 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q2_K \|
	\| 116 \| blk.9.attn_q_norm.weight \| Block 9 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 117 \| blk.9.attn_v.weight \| Block 9 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 118 \| blk.9.ffn_down_exps.weight \| Block 9 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 119 \| blk.9.ffn_gate_exps.weight \| Block 9 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|
	\| 120 \| blk.9.ffn_gate_inp.weight \| Block 9 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 121 \| blk.9.ffn_norm.weight \| Block 9 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 122 \| blk.9.ffn_up_exps.weight \| Block 9 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|

	- Total elements in blk.9: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_10">Block 10 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 123 \| blk.10.attn_k.weight \| Block 10 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q2_K \|
	\| 124 \| blk.10.attn_k_norm.weight \| Block 10 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 125 \| blk.10.attn_norm.weight \| Block 10 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 126 \| blk.10.attn_output.weight \| Block 10 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 127 \| blk.10.attn_q.weight \| Block 10 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q2_K \|
	\| 128 \| blk.10.attn_q_norm.weight \| Block 10 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 129 \| blk.10.attn_v.weight \| Block 10 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 130 \| blk.10.ffn_down_exps.weight \| Block 10 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 131 \| blk.10.ffn_gate_exps.weight \| Block 10 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|
	\| 132 \| blk.10.ffn_gate_inp.weight \| Block 10 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 133 \| blk.10.ffn_norm.weight \| Block 10 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 134 \| blk.10.ffn_up_exps.weight \| Block 10 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|

	- Total elements in blk.10: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_11">Block 11 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 135 \| blk.11.attn_k.weight \| Block 11 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q2_K \|
	\| 136 \| blk.11.attn_k_norm.weight \| Block 11 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 137 \| blk.11.attn_norm.weight \| Block 11 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 138 \| blk.11.attn_output.weight \| Block 11 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 139 \| blk.11.attn_q.weight \| Block 11 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q2_K \|
	\| 140 \| blk.11.attn_q_norm.weight \| Block 11 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 141 \| blk.11.attn_v.weight \| Block 11 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 142 \| blk.11.ffn_down_exps.weight \| Block 11 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 143 \| blk.11.ffn_gate_exps.weight \| Block 11 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|
	\| 144 \| blk.11.ffn_gate_inp.weight \| Block 11 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 145 \| blk.11.ffn_norm.weight \| Block 11 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 146 \| blk.11.ffn_up_exps.weight \| Block 11 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|

	- Total elements in blk.11: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_12">Block 12 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 147 \| blk.12.attn_k.weight \| Block 12 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q2_K \|
	\| 148 \| blk.12.attn_k_norm.weight \| Block 12 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 149 \| blk.12.attn_norm.weight \| Block 12 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 150 \| blk.12.attn_output.weight \| Block 12 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 151 \| blk.12.attn_q.weight \| Block 12 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q2_K \|
	\| 152 \| blk.12.attn_q_norm.weight \| Block 12 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 153 \| blk.12.attn_v.weight \| Block 12 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 154 \| blk.12.ffn_down_exps.weight \| Block 12 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 155 \| blk.12.ffn_gate_exps.weight \| Block 12 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|
	\| 156 \| blk.12.ffn_gate_inp.weight \| Block 12 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 157 \| blk.12.ffn_norm.weight \| Block 12 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 158 \| blk.12.ffn_up_exps.weight \| Block 12 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|

	- Total elements in blk.12: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_13">Block 13 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 159 \| blk.13.attn_k.weight \| Block 13 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q2_K \|
	\| 160 \| blk.13.attn_k_norm.weight \| Block 13 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 161 \| blk.13.attn_norm.weight \| Block 13 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 162 \| blk.13.attn_output.weight \| Block 13 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 163 \| blk.13.attn_q.weight \| Block 13 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q2_K \|
	\| 164 \| blk.13.attn_q_norm.weight \| Block 13 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 165 \| blk.13.attn_v.weight \| Block 13 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 166 \| blk.13.ffn_down_exps.weight \| Block 13 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 167 \| blk.13.ffn_gate_exps.weight \| Block 13 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|
	\| 168 \| blk.13.ffn_gate_inp.weight \| Block 13 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 169 \| blk.13.ffn_norm.weight \| Block 13 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 170 \| blk.13.ffn_up_exps.weight \| Block 13 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|

	- Total elements in blk.13: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_14">Block 14 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 171 \| blk.14.attn_k.weight \| Block 14 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q2_K \|
	\| 172 \| blk.14.attn_k_norm.weight \| Block 14 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 173 \| blk.14.attn_norm.weight \| Block 14 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 174 \| blk.14.attn_output.weight \| Block 14 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 175 \| blk.14.attn_q.weight \| Block 14 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q2_K \|
	\| 176 \| blk.14.attn_q_norm.weight \| Block 14 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 177 \| blk.14.attn_v.weight \| Block 14 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 178 \| blk.14.ffn_down_exps.weight \| Block 14 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 179 \| blk.14.ffn_gate_exps.weight \| Block 14 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|
	\| 180 \| blk.14.ffn_gate_inp.weight \| Block 14 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 181 \| blk.14.ffn_norm.weight \| Block 14 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 182 \| blk.14.ffn_up_exps.weight \| Block 14 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|

	- Total elements in blk.14: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_15">Block 15 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 183 \| blk.15.attn_k.weight \| Block 15 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q2_K \|
	\| 184 \| blk.15.attn_k_norm.weight \| Block 15 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 185 \| blk.15.attn_norm.weight \| Block 15 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 186 \| blk.15.attn_output.weight \| Block 15 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 187 \| blk.15.attn_q.weight \| Block 15 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q2_K \|
	\| 188 \| blk.15.attn_q_norm.weight \| Block 15 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 189 \| blk.15.attn_v.weight \| Block 15 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 190 \| blk.15.ffn_down_exps.weight \| Block 15 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 191 \| blk.15.ffn_gate_exps.weight \| Block 15 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|
	\| 192 \| blk.15.ffn_gate_inp.weight \| Block 15 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 193 \| blk.15.ffn_norm.weight \| Block 15 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 194 \| blk.15.ffn_up_exps.weight \| Block 15 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|

	- Total elements in blk.15: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_16">Block 16 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 195 \| blk.16.attn_k.weight \| Block 16 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q2_K \|
	\| 196 \| blk.16.attn_k_norm.weight \| Block 16 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 197 \| blk.16.attn_norm.weight \| Block 16 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 198 \| blk.16.attn_output.weight \| Block 16 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 199 \| blk.16.attn_q.weight \| Block 16 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q2_K \|
	\| 200 \| blk.16.attn_q_norm.weight \| Block 16 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 201 \| blk.16.attn_v.weight \| Block 16 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 202 \| blk.16.ffn_down_exps.weight \| Block 16 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 203 \| blk.16.ffn_gate_exps.weight \| Block 16 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|
	\| 204 \| blk.16.ffn_gate_inp.weight \| Block 16 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 205 \| blk.16.ffn_norm.weight \| Block 16 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 206 \| blk.16.ffn_up_exps.weight \| Block 16 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|

	- Total elements in blk.16: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_17">Block 17 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 207 \| blk.17.attn_k.weight \| Block 17 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q2_K \|
	\| 208 \| blk.17.attn_k_norm.weight \| Block 17 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 209 \| blk.17.attn_norm.weight \| Block 17 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 210 \| blk.17.attn_output.weight \| Block 17 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 211 \| blk.17.attn_q.weight \| Block 17 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q2_K \|
	\| 212 \| blk.17.attn_q_norm.weight \| Block 17 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 213 \| blk.17.attn_v.weight \| Block 17 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 214 \| blk.17.ffn_down_exps.weight \| Block 17 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 215 \| blk.17.ffn_gate_exps.weight \| Block 17 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|
	\| 216 \| blk.17.ffn_gate_inp.weight \| Block 17 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 217 \| blk.17.ffn_norm.weight \| Block 17 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 218 \| blk.17.ffn_up_exps.weight \| Block 17 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|

	- Total elements in blk.17: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_18">Block 18 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 219 \| blk.18.attn_k.weight \| Block 18 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q2_K \|
	\| 220 \| blk.18.attn_k_norm.weight \| Block 18 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 221 \| blk.18.attn_norm.weight \| Block 18 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 222 \| blk.18.attn_output.weight \| Block 18 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 223 \| blk.18.attn_q.weight \| Block 18 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q2_K \|
	\| 224 \| blk.18.attn_q_norm.weight \| Block 18 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 225 \| blk.18.attn_v.weight \| Block 18 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 226 \| blk.18.ffn_down_exps.weight \| Block 18 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 227 \| blk.18.ffn_gate_exps.weight \| Block 18 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|
	\| 228 \| blk.18.ffn_gate_inp.weight \| Block 18 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 229 \| blk.18.ffn_norm.weight \| Block 18 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 230 \| blk.18.ffn_up_exps.weight \| Block 18 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|

	- Total elements in blk.18: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_19">Block 19 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 231 \| blk.19.attn_k.weight \| Block 19 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q2_K \|
	\| 232 \| blk.19.attn_k_norm.weight \| Block 19 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 233 \| blk.19.attn_norm.weight \| Block 19 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 234 \| blk.19.attn_output.weight \| Block 19 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 235 \| blk.19.attn_q.weight \| Block 19 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q2_K \|
	\| 236 \| blk.19.attn_q_norm.weight \| Block 19 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 237 \| blk.19.attn_v.weight \| Block 19 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 238 \| blk.19.ffn_down_exps.weight \| Block 19 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 239 \| blk.19.ffn_gate_exps.weight \| Block 19 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|
	\| 240 \| blk.19.ffn_gate_inp.weight \| Block 19 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 241 \| blk.19.ffn_norm.weight \| Block 19 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 242 \| blk.19.ffn_up_exps.weight \| Block 19 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|

	- Total elements in blk.19: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_20">Block 20 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 243 \| blk.20.attn_k.weight \| Block 20 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q2_K \|
	\| 244 \| blk.20.attn_k_norm.weight \| Block 20 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 245 \| blk.20.attn_norm.weight \| Block 20 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 246 \| blk.20.attn_output.weight \| Block 20 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 247 \| blk.20.attn_q.weight \| Block 20 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q2_K \|
	\| 248 \| blk.20.attn_q_norm.weight \| Block 20 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 249 \| blk.20.attn_v.weight \| Block 20 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 250 \| blk.20.ffn_down_exps.weight \| Block 20 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 251 \| blk.20.ffn_gate_exps.weight \| Block 20 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|
	\| 252 \| blk.20.ffn_gate_inp.weight \| Block 20 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 253 \| blk.20.ffn_norm.weight \| Block 20 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 254 \| blk.20.ffn_up_exps.weight \| Block 20 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|

	- Total elements in blk.20: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_21">Block 21 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 255 \| blk.21.attn_k.weight \| Block 21 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q2_K \|
	\| 256 \| blk.21.attn_k_norm.weight \| Block 21 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 257 \| blk.21.attn_norm.weight \| Block 21 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 258 \| blk.21.attn_output.weight \| Block 21 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 259 \| blk.21.attn_q.weight \| Block 21 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q2_K \|
	\| 260 \| blk.21.attn_q_norm.weight \| Block 21 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 261 \| blk.21.attn_v.weight \| Block 21 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 262 \| blk.21.ffn_down_exps.weight \| Block 21 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 263 \| blk.21.ffn_gate_exps.weight \| Block 21 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|
	\| 264 \| blk.21.ffn_gate_inp.weight \| Block 21 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 265 \| blk.21.ffn_norm.weight \| Block 21 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 266 \| blk.21.ffn_up_exps.weight \| Block 21 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|

	- Total elements in blk.21: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_22">Block 22 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 267 \| blk.22.attn_k.weight \| Block 22 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q2_K \|
	\| 268 \| blk.22.attn_k_norm.weight \| Block 22 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 269 \| blk.22.attn_norm.weight \| Block 22 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 270 \| blk.22.attn_output.weight \| Block 22 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 271 \| blk.22.attn_q.weight \| Block 22 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q2_K \|
	\| 272 \| blk.22.attn_q_norm.weight \| Block 22 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 273 \| blk.22.attn_v.weight \| Block 22 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 274 \| blk.22.ffn_down_exps.weight \| Block 22 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 275 \| blk.22.ffn_gate_exps.weight \| Block 22 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|
	\| 276 \| blk.22.ffn_gate_inp.weight \| Block 22 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 277 \| blk.22.ffn_norm.weight \| Block 22 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 278 \| blk.22.ffn_up_exps.weight \| Block 22 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|

	- Total elements in blk.22: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_23">Block 23 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 279 \| blk.23.attn_k.weight \| Block 23 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q2_K \|
	\| 280 \| blk.23.attn_k_norm.weight \| Block 23 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 281 \| blk.23.attn_norm.weight \| Block 23 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 282 \| blk.23.attn_output.weight \| Block 23 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 283 \| blk.23.attn_q.weight \| Block 23 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q2_K \|
	\| 284 \| blk.23.attn_q_norm.weight \| Block 23 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 285 \| blk.23.attn_v.weight \| Block 23 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 286 \| blk.23.ffn_down_exps.weight \| Block 23 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 287 \| blk.23.ffn_gate_exps.weight \| Block 23 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|
	\| 288 \| blk.23.ffn_gate_inp.weight \| Block 23 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 289 \| blk.23.ffn_norm.weight \| Block 23 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 290 \| blk.23.ffn_up_exps.weight \| Block 23 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|

	- Total elements in blk.23: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_24">Block 24 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 291 \| blk.24.attn_k.weight \| Block 24 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 292 \| blk.24.attn_k_norm.weight \| Block 24 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 293 \| blk.24.attn_norm.weight \| Block 24 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 294 \| blk.24.attn_output.weight \| Block 24 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 295 \| blk.24.attn_q.weight \| Block 24 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q3_K \|
	\| 296 \| blk.24.attn_q_norm.weight \| Block 24 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 297 \| blk.24.attn_v.weight \| Block 24 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q4_K \|
	\| 298 \| blk.24.ffn_down_exps.weight \| Block 24 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 299 \| blk.24.ffn_gate_exps.weight \| Block 24 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|
	\| 300 \| blk.24.ffn_gate_inp.weight \| Block 24 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 301 \| blk.24.ffn_norm.weight \| Block 24 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 302 \| blk.24.ffn_up_exps.weight \| Block 24 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|

	- Total elements in blk.24: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_25">Block 25 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 303 \| blk.25.attn_k.weight \| Block 25 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 304 \| blk.25.attn_k_norm.weight \| Block 25 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 305 \| blk.25.attn_norm.weight \| Block 25 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 306 \| blk.25.attn_output.weight \| Block 25 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 307 \| blk.25.attn_q.weight \| Block 25 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q3_K \|
	\| 308 \| blk.25.attn_q_norm.weight \| Block 25 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 309 \| blk.25.attn_v.weight \| Block 25 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q4_K \|
	\| 310 \| blk.25.ffn_down_exps.weight \| Block 25 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 311 \| blk.25.ffn_gate_exps.weight \| Block 25 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|
	\| 312 \| blk.25.ffn_gate_inp.weight \| Block 25 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 313 \| blk.25.ffn_norm.weight \| Block 25 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 314 \| blk.25.ffn_up_exps.weight \| Block 25 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|

	- Total elements in blk.25: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_26">Block 26 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 315 \| blk.26.attn_k.weight \| Block 26 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 316 \| blk.26.attn_k_norm.weight \| Block 26 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 317 \| blk.26.attn_norm.weight \| Block 26 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 318 \| blk.26.attn_output.weight \| Block 26 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 319 \| blk.26.attn_q.weight \| Block 26 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q3_K \|
	\| 320 \| blk.26.attn_q_norm.weight \| Block 26 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 321 \| blk.26.attn_v.weight \| Block 26 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q4_K \|
	\| 322 \| blk.26.ffn_down_exps.weight \| Block 26 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 323 \| blk.26.ffn_gate_exps.weight \| Block 26 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|
	\| 324 \| blk.26.ffn_gate_inp.weight \| Block 26 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 325 \| blk.26.ffn_norm.weight \| Block 26 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 326 \| blk.26.ffn_up_exps.weight \| Block 26 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q2_K \|

	- Total elements in blk.26: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_27">Block 27 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 327 \| blk.27.attn_k.weight \| Block 27 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 328 \| blk.27.attn_k_norm.weight \| Block 27 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 329 \| blk.27.attn_norm.weight \| Block 27 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 330 \| blk.27.attn_output.weight \| Block 27 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 331 \| blk.27.attn_q.weight \| Block 27 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q3_K \|
	\| 332 \| blk.27.attn_q_norm.weight \| Block 27 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 333 \| blk.27.attn_v.weight \| Block 27 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q4_K \|
	\| 334 \| blk.27.ffn_down_exps.weight \| Block 27 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 335 \| blk.27.ffn_gate_exps.weight \| Block 27 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|
	\| 336 \| blk.27.ffn_gate_inp.weight \| Block 27 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 337 \| blk.27.ffn_norm.weight \| Block 27 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 338 \| blk.27.ffn_up_exps.weight \| Block 27 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|

	- Total elements in blk.27: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_28">Block 28 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 339 \| blk.28.attn_k.weight \| Block 28 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 340 \| blk.28.attn_k_norm.weight \| Block 28 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 341 \| blk.28.attn_norm.weight \| Block 28 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 342 \| blk.28.attn_output.weight \| Block 28 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 343 \| blk.28.attn_q.weight \| Block 28 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q3_K \|
	\| 344 \| blk.28.attn_q_norm.weight \| Block 28 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 345 \| blk.28.attn_v.weight \| Block 28 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q4_K \|
	\| 346 \| blk.28.ffn_down_exps.weight \| Block 28 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 347 \| blk.28.ffn_gate_exps.weight \| Block 28 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|
	\| 348 \| blk.28.ffn_gate_inp.weight \| Block 28 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 349 \| blk.28.ffn_norm.weight \| Block 28 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 350 \| blk.28.ffn_up_exps.weight \| Block 28 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|

	- Total elements in blk.28: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_29">Block 29 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 351 \| blk.29.attn_k.weight \| Block 29 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 352 \| blk.29.attn_k_norm.weight \| Block 29 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 353 \| blk.29.attn_norm.weight \| Block 29 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 354 \| blk.29.attn_output.weight \| Block 29 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 355 \| blk.29.attn_q.weight \| Block 29 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q3_K \|
	\| 356 \| blk.29.attn_q_norm.weight \| Block 29 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 357 \| blk.29.attn_v.weight \| Block 29 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q4_K \|
	\| 358 \| blk.29.ffn_down_exps.weight \| Block 29 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 359 \| blk.29.ffn_gate_exps.weight \| Block 29 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|
	\| 360 \| blk.29.ffn_gate_inp.weight \| Block 29 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 361 \| blk.29.ffn_norm.weight \| Block 29 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 362 \| blk.29.ffn_up_exps.weight \| Block 29 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|

	- Total elements in blk.29: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_30">Block 30 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 363 \| blk.30.attn_k.weight \| Block 30 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 364 \| blk.30.attn_k_norm.weight \| Block 30 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 365 \| blk.30.attn_norm.weight \| Block 30 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 366 \| blk.30.attn_output.weight \| Block 30 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 367 \| blk.30.attn_q.weight \| Block 30 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q3_K \|
	\| 368 \| blk.30.attn_q_norm.weight \| Block 30 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 369 \| blk.30.attn_v.weight \| Block 30 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q4_K \|
	\| 370 \| blk.30.ffn_down_exps.weight \| Block 30 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 371 \| blk.30.ffn_gate_exps.weight \| Block 30 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|
	\| 372 \| blk.30.ffn_gate_inp.weight \| Block 30 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 373 \| blk.30.ffn_norm.weight \| Block 30 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 374 \| blk.30.ffn_up_exps.weight \| Block 30 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|

	- Total elements in blk.30: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_31">Block 31 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 375 \| blk.31.attn_k.weight \| Block 31 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 376 \| blk.31.attn_k_norm.weight \| Block 31 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 377 \| blk.31.attn_norm.weight \| Block 31 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 378 \| blk.31.attn_output.weight \| Block 31 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 379 \| blk.31.attn_q.weight \| Block 31 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q3_K \|
	\| 380 \| blk.31.attn_q_norm.weight \| Block 31 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 381 \| blk.31.attn_v.weight \| Block 31 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q4_K \|
	\| 382 \| blk.31.ffn_down_exps.weight \| Block 31 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 383 \| blk.31.ffn_gate_exps.weight \| Block 31 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|
	\| 384 \| blk.31.ffn_gate_inp.weight \| Block 31 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 385 \| blk.31.ffn_norm.weight \| Block 31 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 386 \| blk.31.ffn_up_exps.weight \| Block 31 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|

	- Total elements in blk.31: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_32">Block 32 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 387 \| blk.32.attn_k.weight \| Block 32 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 388 \| blk.32.attn_k_norm.weight \| Block 32 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 389 \| blk.32.attn_norm.weight \| Block 32 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 390 \| blk.32.attn_output.weight \| Block 32 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 391 \| blk.32.attn_q.weight \| Block 32 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q3_K \|
	\| 392 \| blk.32.attn_q_norm.weight \| Block 32 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 393 \| blk.32.attn_v.weight \| Block 32 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q4_K \|
	\| 394 \| blk.32.ffn_down_exps.weight \| Block 32 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 395 \| blk.32.ffn_gate_exps.weight \| Block 32 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|
	\| 396 \| blk.32.ffn_gate_inp.weight \| Block 32 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 397 \| blk.32.ffn_norm.weight \| Block 32 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 398 \| blk.32.ffn_up_exps.weight \| Block 32 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|

	- Total elements in blk.32: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_33">Block 33 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 399 \| blk.33.attn_k.weight \| Block 33 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 400 \| blk.33.attn_k_norm.weight \| Block 33 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 401 \| blk.33.attn_norm.weight \| Block 33 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 402 \| blk.33.attn_output.weight \| Block 33 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 403 \| blk.33.attn_q.weight \| Block 33 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q3_K \|
	\| 404 \| blk.33.attn_q_norm.weight \| Block 33 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 405 \| blk.33.attn_v.weight \| Block 33 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q4_K \|
	\| 406 \| blk.33.ffn_down_exps.weight \| Block 33 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 407 \| blk.33.ffn_gate_exps.weight \| Block 33 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|
	\| 408 \| blk.33.ffn_gate_inp.weight \| Block 33 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 409 \| blk.33.ffn_norm.weight \| Block 33 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 410 \| blk.33.ffn_up_exps.weight \| Block 33 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|

	- Total elements in blk.33: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_34">Block 34 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 411 \| blk.34.attn_k.weight \| Block 34 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 412 \| blk.34.attn_k_norm.weight \| Block 34 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 413 \| blk.34.attn_norm.weight \| Block 34 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 414 \| blk.34.attn_output.weight \| Block 34 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 415 \| blk.34.attn_q.weight \| Block 34 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q3_K \|
	\| 416 \| blk.34.attn_q_norm.weight \| Block 34 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 417 \| blk.34.attn_v.weight \| Block 34 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q4_K \|
	\| 418 \| blk.34.ffn_down_exps.weight \| Block 34 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 419 \| blk.34.ffn_gate_exps.weight \| Block 34 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|
	\| 420 \| blk.34.ffn_gate_inp.weight \| Block 34 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 421 \| blk.34.ffn_norm.weight \| Block 34 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 422 \| blk.34.ffn_up_exps.weight \| Block 34 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|

	- Total elements in blk.34: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_35">Block 35 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 423 \| blk.35.attn_k.weight \| Block 35 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 424 \| blk.35.attn_k_norm.weight \| Block 35 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 425 \| blk.35.attn_norm.weight \| Block 35 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 426 \| blk.35.attn_output.weight \| Block 35 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 427 \| blk.35.attn_q.weight \| Block 35 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q3_K \|
	\| 428 \| blk.35.attn_q_norm.weight \| Block 35 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 429 \| blk.35.attn_v.weight \| Block 35 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q4_K \|
	\| 430 \| blk.35.ffn_down_exps.weight \| Block 35 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 431 \| blk.35.ffn_gate_exps.weight \| Block 35 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|
	\| 432 \| blk.35.ffn_gate_inp.weight \| Block 35 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 433 \| blk.35.ffn_norm.weight \| Block 35 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 434 \| blk.35.ffn_up_exps.weight \| Block 35 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|

	- Total elements in blk.35: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_36">Block 36 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 435 \| blk.36.attn_k.weight \| Block 36 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 436 \| blk.36.attn_k_norm.weight \| Block 36 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 437 \| blk.36.attn_norm.weight \| Block 36 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 438 \| blk.36.attn_output.weight \| Block 36 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 439 \| blk.36.attn_q.weight \| Block 36 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q3_K \|
	\| 440 \| blk.36.attn_q_norm.weight \| Block 36 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 441 \| blk.36.attn_v.weight \| Block 36 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q4_K \|
	\| 442 \| blk.36.ffn_down_exps.weight \| Block 36 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 443 \| blk.36.ffn_gate_exps.weight \| Block 36 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|
	\| 444 \| blk.36.ffn_gate_inp.weight \| Block 36 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 445 \| blk.36.ffn_norm.weight \| Block 36 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 446 \| blk.36.ffn_up_exps.weight \| Block 36 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|

	- Total elements in blk.36: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_37">Block 37 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 447 \| blk.37.attn_k.weight \| Block 37 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 448 \| blk.37.attn_k_norm.weight \| Block 37 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 449 \| blk.37.attn_norm.weight \| Block 37 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 450 \| blk.37.attn_output.weight \| Block 37 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 451 \| blk.37.attn_q.weight \| Block 37 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q3_K \|
	\| 452 \| blk.37.attn_q_norm.weight \| Block 37 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 453 \| blk.37.attn_v.weight \| Block 37 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q4_K \|
	\| 454 \| blk.37.ffn_down_exps.weight \| Block 37 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 455 \| blk.37.ffn_gate_exps.weight \| Block 37 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|
	\| 456 \| blk.37.ffn_gate_inp.weight \| Block 37 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 457 \| blk.37.ffn_norm.weight \| Block 37 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 458 \| blk.37.ffn_up_exps.weight \| Block 37 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|

	- Total elements in blk.37: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_38">Block 38 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 459 \| blk.38.attn_k.weight \| Block 38 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 460 \| blk.38.attn_k_norm.weight \| Block 38 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 461 \| blk.38.attn_norm.weight \| Block 38 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 462 \| blk.38.attn_output.weight \| Block 38 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 463 \| blk.38.attn_q.weight \| Block 38 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q3_K \|
	\| 464 \| blk.38.attn_q_norm.weight \| Block 38 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 465 \| blk.38.attn_v.weight \| Block 38 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q4_K \|
	\| 466 \| blk.38.ffn_down_exps.weight \| Block 38 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 467 \| blk.38.ffn_gate_exps.weight \| Block 38 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|
	\| 468 \| blk.38.ffn_gate_inp.weight \| Block 38 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 469 \| blk.38.ffn_norm.weight \| Block 38 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 470 \| blk.38.ffn_up_exps.weight \| Block 38 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|

	- Total elements in blk.38: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_39">Block 39 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 471 \| blk.39.attn_k.weight \| Block 39 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 472 \| blk.39.attn_k_norm.weight \| Block 39 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 473 \| blk.39.attn_norm.weight \| Block 39 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 474 \| blk.39.attn_output.weight \| Block 39 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 475 \| blk.39.attn_q.weight \| Block 39 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q3_K \|
	\| 476 \| blk.39.attn_q_norm.weight \| Block 39 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 477 \| blk.39.attn_v.weight \| Block 39 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q4_K \|
	\| 478 \| blk.39.ffn_down_exps.weight \| Block 39 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 479 \| blk.39.ffn_gate_exps.weight \| Block 39 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|
	\| 480 \| blk.39.ffn_gate_inp.weight \| Block 39 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 481 \| blk.39.ffn_norm.weight \| Block 39 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 482 \| blk.39.ffn_up_exps.weight \| Block 39 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|

	- Total elements in blk.39: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_40">Block 40 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 483 \| blk.40.attn_k.weight \| Block 40 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 484 \| blk.40.attn_k_norm.weight \| Block 40 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 485 \| blk.40.attn_norm.weight \| Block 40 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 486 \| blk.40.attn_output.weight \| Block 40 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 487 \| blk.40.attn_q.weight \| Block 40 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q3_K \|
	\| 488 \| blk.40.attn_q_norm.weight \| Block 40 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 489 \| blk.40.attn_v.weight \| Block 40 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q4_K \|
	\| 490 \| blk.40.ffn_down_exps.weight \| Block 40 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 491 \| blk.40.ffn_gate_exps.weight \| Block 40 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|
	\| 492 \| blk.40.ffn_gate_inp.weight \| Block 40 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 493 \| blk.40.ffn_norm.weight \| Block 40 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 494 \| blk.40.ffn_up_exps.weight \| Block 40 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|

	- Total elements in blk.40: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_41">Block 41 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 495 \| blk.41.attn_k.weight \| Block 41 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 496 \| blk.41.attn_k_norm.weight \| Block 41 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 497 \| blk.41.attn_norm.weight \| Block 41 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 498 \| blk.41.attn_output.weight \| Block 41 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 499 \| blk.41.attn_q.weight \| Block 41 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q3_K \|
	\| 500 \| blk.41.attn_q_norm.weight \| Block 41 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 501 \| blk.41.attn_v.weight \| Block 41 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q4_K \|
	\| 502 \| blk.41.ffn_down_exps.weight \| Block 41 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 503 \| blk.41.ffn_gate_exps.weight \| Block 41 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|
	\| 504 \| blk.41.ffn_gate_inp.weight \| Block 41 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 505 \| blk.41.ffn_norm.weight \| Block 41 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 506 \| blk.41.ffn_up_exps.weight \| Block 41 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|

	- Total elements in blk.41: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_42">Block 42 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 507 \| blk.42.attn_k.weight \| Block 42 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 508 \| blk.42.attn_k_norm.weight \| Block 42 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 509 \| blk.42.attn_norm.weight \| Block 42 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 510 \| blk.42.attn_output.weight \| Block 42 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 511 \| blk.42.attn_q.weight \| Block 42 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q3_K \|
	\| 512 \| blk.42.attn_q_norm.weight \| Block 42 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 513 \| blk.42.attn_v.weight \| Block 42 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q4_K \|
	\| 514 \| blk.42.ffn_down_exps.weight \| Block 42 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 515 \| blk.42.ffn_gate_exps.weight \| Block 42 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|
	\| 516 \| blk.42.ffn_gate_inp.weight \| Block 42 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 517 \| blk.42.ffn_norm.weight \| Block 42 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 518 \| blk.42.ffn_up_exps.weight \| Block 42 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|

	- Total elements in blk.42: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_43">Block 43 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 519 \| blk.43.attn_k.weight \| Block 43 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 520 \| blk.43.attn_k_norm.weight \| Block 43 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 521 \| blk.43.attn_norm.weight \| Block 43 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 522 \| blk.43.attn_output.weight \| Block 43 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 523 \| blk.43.attn_q.weight \| Block 43 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q3_K \|
	\| 524 \| blk.43.attn_q_norm.weight \| Block 43 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 525 \| blk.43.attn_v.weight \| Block 43 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q4_K \|
	\| 526 \| blk.43.ffn_down_exps.weight \| Block 43 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 527 \| blk.43.ffn_gate_exps.weight \| Block 43 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|
	\| 528 \| blk.43.ffn_gate_inp.weight \| Block 43 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 529 \| blk.43.ffn_norm.weight \| Block 43 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 530 \| blk.43.ffn_up_exps.weight \| Block 43 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|

	- Total elements in blk.43: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_44">Block 44 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 531 \| blk.44.attn_k.weight \| Block 44 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 532 \| blk.44.attn_k_norm.weight \| Block 44 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 533 \| blk.44.attn_norm.weight \| Block 44 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 534 \| blk.44.attn_output.weight \| Block 44 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 535 \| blk.44.attn_q.weight \| Block 44 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q3_K \|
	\| 536 \| blk.44.attn_q_norm.weight \| Block 44 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 537 \| blk.44.attn_v.weight \| Block 44 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q4_K \|
	\| 538 \| blk.44.ffn_down_exps.weight \| Block 44 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 539 \| blk.44.ffn_gate_exps.weight \| Block 44 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|
	\| 540 \| blk.44.ffn_gate_inp.weight \| Block 44 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 541 \| blk.44.ffn_norm.weight \| Block 44 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 542 \| blk.44.ffn_up_exps.weight \| Block 44 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|

	- Total elements in blk.44: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_45">Block 45 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 543 \| blk.45.attn_k.weight \| Block 45 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 544 \| blk.45.attn_k_norm.weight \| Block 45 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 545 \| blk.45.attn_norm.weight \| Block 45 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 546 \| blk.45.attn_output.weight \| Block 45 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 547 \| blk.45.attn_q.weight \| Block 45 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q3_K \|
	\| 548 \| blk.45.attn_q_norm.weight \| Block 45 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 549 \| blk.45.attn_v.weight \| Block 45 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q4_K \|
	\| 550 \| blk.45.ffn_down_exps.weight \| Block 45 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 551 \| blk.45.ffn_gate_exps.weight \| Block 45 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|
	\| 552 \| blk.45.ffn_gate_inp.weight \| Block 45 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 553 \| blk.45.ffn_norm.weight \| Block 45 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 554 \| blk.45.ffn_up_exps.weight \| Block 45 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|

	- Total elements in blk.45: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_46">Block 46 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 555 \| blk.46.attn_k.weight \| Block 46 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 556 \| blk.46.attn_k_norm.weight \| Block 46 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 557 \| blk.46.attn_norm.weight \| Block 46 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 558 \| blk.46.attn_output.weight \| Block 46 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 559 \| blk.46.attn_q.weight \| Block 46 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q3_K \|
	\| 560 \| blk.46.attn_q_norm.weight \| Block 46 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 561 \| blk.46.attn_v.weight \| Block 46 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q4_K \|
	\| 562 \| blk.46.ffn_down_exps.weight \| Block 46 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 563 \| blk.46.ffn_gate_exps.weight \| Block 46 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|
	\| 564 \| blk.46.ffn_gate_inp.weight \| Block 46 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 565 \| blk.46.ffn_norm.weight \| Block 46 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 566 \| blk.46.ffn_up_exps.weight \| Block 46 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|

	- Total elements in blk.46: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_47">Block 47 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-----\|
	\| 567 \| blk.47.attn_k.weight \| Block 47 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q3_K \|
	\| 568 \| blk.47.attn_k_norm.weight \| Block 47 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 569 \| blk.47.attn_norm.weight \| Block 47 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 570 \| blk.47.attn_output.weight \| Block 47 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| Q4_K \|
	\| 571 \| blk.47.attn_q.weight \| Block 47 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| Q3_K \|
	\| 572 \| blk.47.attn_q_norm.weight \| Block 47 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 573 \| blk.47.attn_v.weight \| Block 47 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| Q4_K \|
	\| 574 \| blk.47.ffn_down_exps.weight \| Block 47 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q4_K \|
	\| 575 \| blk.47.ffn_gate_exps.weight \| Block 47 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|
	\| 576 \| blk.47.ffn_gate_inp.weight \| Block 47 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 577 \| blk.47.ffn_norm.weight \| Block 47 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 578 \| blk.47.ffn_up_exps.weight \| Block 47 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| Q3_K \|

	- Total elements in blk.47: (~623M) 623120640
	- Percentage of total elements: 2.04%