Instructions to use eaddario/Qwen3-30B-A3B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use eaddario/Qwen3-30B-A3B-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="eaddario/Qwen3-30B-A3B-GGUF",
	filename="Qwen3-30B-A3B-F16-00001-of-00003.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use eaddario/Qwen3-30B-A3B-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use eaddario/Qwen3-30B-A3B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "eaddario/Qwen3-30B-A3B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "eaddario/Qwen3-30B-A3B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Ollama
How to use eaddario/Qwen3-30B-A3B-GGUF with Ollama:
```
ollama run hf.co/eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M
```

Unsloth Studio

How to use eaddario/Qwen3-30B-A3B-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for eaddario/Qwen3-30B-A3B-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for eaddario/Qwen3-30B-A3B-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for eaddario/Qwen3-30B-A3B-GGUF to start chatting

How to use eaddario/Qwen3-30B-A3B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use eaddario/Qwen3-30B-A3B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use eaddario/Qwen3-30B-A3B-GGUF with Docker Model Runner:
```
docker model run hf.co/eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M
```

Lemonade

How to use eaddario/Qwen3-30B-A3B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Qwen3-30B-A3B-GGUF-Q4_K_M

List all available models

lemonade list

Qwen3-30B-A3B-GGUF / scores /Qwen3-30B-A3B-IQ3_S.md

eaddario

Add GGUF internal file structure

a630e18 verified about 1 year ago

preview code

raw

history blame

188 kB

	# Qwen3-30B-A3B-IQ3_S.gguf - GGUF Internal File Dump

	- Endian: LITTLE endian

	## Key Value Metadata Store

	There are 44 key-value pairs in this file

	\| POS \| TYPE \| Count \| Key \| Value \|
	\|----:\|:---------\|-------:\|:------------------------------------------\|:--------------------------------------------------------------------\|
	\| 1 \| UINT32 \| 1 \| GGUF.version \| 3 \|
	\| 2 \| UINT64 \| 1 \| GGUF.tensor_count \| 579 \|
	\| 3 \| UINT64 \| 1 \| GGUF.kv_count \| 41 \|
	\| 4 \| STRING \| 1 \| general.architecture \| `qwen3moe` \|
	\| 5 \| STRING \| 1 \| general.type \| `model` \|
	\| 6 \| STRING \| 1 \| general.name \| `Qwen3 30B A3B` \|
	\| 7 \| STRING \| 1 \| general.basename \| `Qwen3` \|
	\| 8 \| STRING \| 1 \| general.size_label \| `30B-A3B` \|
	\| 9 \| STRING \| 1 \| general.license \| `apache-2.0` \|
	\| 10 \| STRING \| 1 \| general.license.link \| `https://huggingface.co/Qwen/Qwen3-30B-A3B/blob/main/LICENSE` \|
	\| 11 \| UINT32 \| 1 \| general.base_model.count \| 1 \|
	\| 12 \| STRING \| 1 \| general.base_model.0.name \| `Qwen3 30B A3B Base` \|
	\| 13 \| STRING \| 1 \| general.base_model.0.organization \| `Qwen` \|
	\| 14 \| STRING \| 1 \| general.base_model.0.repo_url \| `https://huggingface.co/Qwen/Qwen3-30B-A3B-Base` \|
	\| 15 \| [STRING] \| 1 \| general.tags \| [ `text-generation` ] \|
	\| 16 \| UINT32 \| 1 \| qwen3moe.block_count \| 48 \|
	\| 17 \| UINT32 \| 1 \| qwen3moe.context_length \| 40960 \|
	\| 18 \| UINT32 \| 1 \| qwen3moe.embedding_length \| 2048 \|
	\| 19 \| UINT32 \| 1 \| qwen3moe.feed_forward_length \| 6144 \|
	\| 20 \| UINT32 \| 1 \| qwen3moe.attention.head_count \| 32 \|
	\| 21 \| UINT32 \| 1 \| qwen3moe.attention.head_count_kv \| 4 \|
	\| 22 \| FLOAT32 \| 1 \| qwen3moe.rope.freq_base \| 1000000.0 \|
	\| 23 \| FLOAT32 \| 1 \| qwen3moe.attention.layer_norm_rms_epsilon \| 1e-06 \|
	\| 24 \| UINT32 \| 1 \| qwen3moe.expert_used_count \| 8 \|
	\| 25 \| UINT32 \| 1 \| qwen3moe.attention.key_length \| 128 \|
	\| 26 \| UINT32 \| 1 \| qwen3moe.attention.value_length \| 128 \|
	\| 27 \| UINT32 \| 1 \| qwen3moe.expert_count \| 128 \|
	\| 28 \| UINT32 \| 1 \| qwen3moe.expert_feed_forward_length \| 768 \|
	\| 29 \| STRING \| 1 \| tokenizer.ggml.model \| `gpt2` \|
	\| 30 \| STRING \| 1 \| tokenizer.ggml.pre \| `qwen2` \|
	\| 31 \| [STRING] \| 151936 \| tokenizer.ggml.tokens \| [ `!`, `"`, `#`, `$`, `%`, ... ] \|
	\| 32 \| [INT32] \| 151936 \| tokenizer.ggml.token_type \| [ 1, 1, 1, 1, 1, 1, 1, ... ] \|
	\| 33 \| [STRING] \| 151387 \| tokenizer.ggml.merges \| [ `Ġ Ġ`, `ĠĠ ĠĠ`, `i n`, `Ġ t`, `ĠĠĠĠ ĠĠĠĠ`, ... ] \|
	\| 34 \| UINT32 \| 1 \| tokenizer.ggml.eos_token_id \| 151645 \|
	\| 35 \| UINT32 \| 1 \| tokenizer.ggml.padding_token_id \| 151643 \|
	\| 36 \| UINT32 \| 1 \| tokenizer.ggml.bos_token_id \| 151643 \|
	\| 37 \| BOOL \| 1 \| tokenizer.ggml.add_bos_token \| False \|
	\| 38 \| STRING \| 1 \| tokenizer.chat_template \| `{%- if tools %}{{- '<\|im_`...`{%- endif %}{%- endif %}` \|
	\| 39 \| UINT32 \| 1 \| general.quantization_version \| 2 \|
	\| 40 \| UINT32 \| 1 \| general.file_type \| 26 \|
	\| 41 \| STRING \| 1 \| quantize.imatrix.file \| `./imatrix/imatrix-Qwen3-30B-A3B-large.dat` \|
	\| 42 \| STRING \| 1 \| quantize.imatrix.dataset \| `../../datasets/imatrix/calibration_all_large.txt` \|
	\| 43 \| INT32 \| 1 \| quantize.imatrix.entries_count \| 382 \|
	\| 44 \| INT32 \| 1 \| quantize.imatrix.chunks_count \| 4978 \|

	## Tensors Overview ~31B Elements

	Total number of elements in all tensors: 30532122624 Elements

	- [Qwen3-30B-A3B-IQ3\_S.gguf - GGUF Internal File Dump](#qwen3-30b-a3b-iq3_sgguf---gguf-internal-file-dump)
	- [Key Value Metadata Store](#key-value-metadata-store)
	- [Tensors Overview ~31B Elements](#tensors-overview-31b-elements)
	- [Tensor Data Offset](#tensor-data-offset)
	- [Base Tensor Group : ~622M Elements](#base-tensor-group--622m-elements)
	- [Block 0 Tensor Group : ~623M Elements](#block-0-tensor-group--623m-elements)
	- [Block 1 Tensor Group : ~623M Elements](#block-1-tensor-group--623m-elements)
	- [Block 2 Tensor Group : ~623M Elements](#block-2-tensor-group--623m-elements)
	- [Block 3 Tensor Group : ~623M Elements](#block-3-tensor-group--623m-elements)
	- [Block 4 Tensor Group : ~623M Elements](#block-4-tensor-group--623m-elements)
	- [Block 5 Tensor Group : ~623M Elements](#block-5-tensor-group--623m-elements)
	- [Block 6 Tensor Group : ~623M Elements](#block-6-tensor-group--623m-elements)
	- [Block 7 Tensor Group : ~623M Elements](#block-7-tensor-group--623m-elements)
	- [Block 8 Tensor Group : ~623M Elements](#block-8-tensor-group--623m-elements)
	- [Block 9 Tensor Group : ~623M Elements](#block-9-tensor-group--623m-elements)
	- [Block 10 Tensor Group : ~623M Elements](#block-10-tensor-group--623m-elements)
	- [Block 11 Tensor Group : ~623M Elements](#block-11-tensor-group--623m-elements)
	- [Block 12 Tensor Group : ~623M Elements](#block-12-tensor-group--623m-elements)
	- [Block 13 Tensor Group : ~623M Elements](#block-13-tensor-group--623m-elements)
	- [Block 14 Tensor Group : ~623M Elements](#block-14-tensor-group--623m-elements)
	- [Block 15 Tensor Group : ~623M Elements](#block-15-tensor-group--623m-elements)
	- [Block 16 Tensor Group : ~623M Elements](#block-16-tensor-group--623m-elements)
	- [Block 17 Tensor Group : ~623M Elements](#block-17-tensor-group--623m-elements)
	- [Block 18 Tensor Group : ~623M Elements](#block-18-tensor-group--623m-elements)
	- [Block 19 Tensor Group : ~623M Elements](#block-19-tensor-group--623m-elements)
	- [Block 20 Tensor Group : ~623M Elements](#block-20-tensor-group--623m-elements)
	- [Block 21 Tensor Group : ~623M Elements](#block-21-tensor-group--623m-elements)
	- [Block 22 Tensor Group : ~623M Elements](#block-22-tensor-group--623m-elements)
	- [Block 23 Tensor Group : ~623M Elements](#block-23-tensor-group--623m-elements)
	- [Block 24 Tensor Group : ~623M Elements](#block-24-tensor-group--623m-elements)
	- [Block 25 Tensor Group : ~623M Elements](#block-25-tensor-group--623m-elements)
	- [Block 26 Tensor Group : ~623M Elements](#block-26-tensor-group--623m-elements)
	- [Block 27 Tensor Group : ~623M Elements](#block-27-tensor-group--623m-elements)
	- [Block 28 Tensor Group : ~623M Elements](#block-28-tensor-group--623m-elements)
	- [Block 29 Tensor Group : ~623M Elements](#block-29-tensor-group--623m-elements)
	- [Block 30 Tensor Group : ~623M Elements](#block-30-tensor-group--623m-elements)
	- [Block 31 Tensor Group : ~623M Elements](#block-31-tensor-group--623m-elements)
	- [Block 32 Tensor Group : ~623M Elements](#block-32-tensor-group--623m-elements)
	- [Block 33 Tensor Group : ~623M Elements](#block-33-tensor-group--623m-elements)
	- [Block 34 Tensor Group : ~623M Elements](#block-34-tensor-group--623m-elements)
	- [Block 35 Tensor Group : ~623M Elements](#block-35-tensor-group--623m-elements)
	- [Block 36 Tensor Group : ~623M Elements](#block-36-tensor-group--623m-elements)
	- [Block 37 Tensor Group : ~623M Elements](#block-37-tensor-group--623m-elements)
	- [Block 38 Tensor Group : ~623M Elements](#block-38-tensor-group--623m-elements)
	- [Block 39 Tensor Group : ~623M Elements](#block-39-tensor-group--623m-elements)
	- [Block 40 Tensor Group : ~623M Elements](#block-40-tensor-group--623m-elements)
	- [Block 41 Tensor Group : ~623M Elements](#block-41-tensor-group--623m-elements)
	- [Block 42 Tensor Group : ~623M Elements](#block-42-tensor-group--623m-elements)
	- [Block 43 Tensor Group : ~623M Elements](#block-43-tensor-group--623m-elements)
	- [Block 44 Tensor Group : ~623M Elements](#block-44-tensor-group--623m-elements)
	- [Block 45 Tensor Group : ~623M Elements](#block-45-tensor-group--623m-elements)
	- [Block 46 Tensor Group : ~623M Elements](#block-46-tensor-group--623m-elements)
	- [Block 47 Tensor Group : ~623M Elements](#block-47-tensor-group--623m-elements)

	### Tensor Data Offset

	This table contains the offset and data segment relative to start of file

	\| T_ID \| Tensor Layer Name \| Data Offset (B) \| Data Size (B) \|
	\|-----:\|:----------------------------\|-----------------:\|-----------------:\|
	\| 0 \| output.weight \| 0x5b18c0 \| 0x7199800 \|
	\| 1 \| output_norm.weight \| 0x774b0c0 \| 0x2000 \|
	\| 2 \| token_embd.weight \| 0x774d0c0 \| 0x7199800 \|
	\| 3 \| blk.0.attn_k.weight \| 0xe8e68c0 \| 0x62000 \|
	\| 4 \| blk.0.attn_k_norm.weight \| 0xe9488c0 \| 0x200 \|
	\| 5 \| blk.0.attn_norm.weight \| 0xe948ac0 \| 0x2000 \|
	\| 6 \| blk.0.attn_output.weight \| 0xe94aac0 \| 0x370000 \|
	\| 7 \| blk.0.attn_q.weight \| 0xecbaac0 \| 0x310000 \|
	\| 8 \| blk.0.attn_q_norm.weight \| 0xefcaac0 \| 0x200 \|
	\| 9 \| blk.0.attn_v.weight \| 0xefcacc0 \| 0x6e000 \|
	\| 10 \| blk.0.ffn_down_exps.weight \| 0xf038cc0 \| 0x5280000 \|
	\| 11 \| blk.0.ffn_gate_exps.weight \| 0x142b8cc0 \| 0x4980000 \|
	\| 12 \| blk.0.ffn_gate_inp.weight \| 0x18c38cc0 \| 0x100000 \|
	\| 13 \| blk.0.ffn_norm.weight \| 0x18d38cc0 \| 0x2000 \|
	\| 14 \| blk.0.ffn_up_exps.weight \| 0x18d3acc0 \| 0x4980000 \|
	\| 15 \| blk.1.attn_k.weight \| 0x1d6bacc0 \| 0x62000 \|
	\| 16 \| blk.1.attn_k_norm.weight \| 0x1d71ccc0 \| 0x200 \|
	\| 17 \| blk.1.attn_norm.weight \| 0x1d71cec0 \| 0x2000 \|
	\| 18 \| blk.1.attn_output.weight \| 0x1d71eec0 \| 0x370000 \|
	\| 19 \| blk.1.attn_q.weight \| 0x1da8eec0 \| 0x310000 \|
	\| 20 \| blk.1.attn_q_norm.weight \| 0x1dd9eec0 \| 0x200 \|
	\| 21 \| blk.1.attn_v.weight \| 0x1dd9f0c0 \| 0x6e000 \|
	\| 22 \| blk.1.ffn_down_exps.weight \| 0x1de0d0c0 \| 0x5280000 \|
	\| 23 \| blk.1.ffn_gate_exps.weight \| 0x2308d0c0 \| 0x4980000 \|
	\| 24 \| blk.1.ffn_gate_inp.weight \| 0x27a0d0c0 \| 0x100000 \|
	\| 25 \| blk.1.ffn_norm.weight \| 0x27b0d0c0 \| 0x2000 \|
	\| 26 \| blk.1.ffn_up_exps.weight \| 0x27b0f0c0 \| 0x4980000 \|
	\| 27 \| blk.2.attn_k.weight \| 0x2c48f0c0 \| 0x62000 \|
	\| 28 \| blk.2.attn_k_norm.weight \| 0x2c4f10c0 \| 0x200 \|
	\| 29 \| blk.2.attn_norm.weight \| 0x2c4f12c0 \| 0x2000 \|
	\| 30 \| blk.2.attn_output.weight \| 0x2c4f32c0 \| 0x370000 \|
	\| 31 \| blk.2.attn_q.weight \| 0x2c8632c0 \| 0x310000 \|
	\| 32 \| blk.2.attn_q_norm.weight \| 0x2cb732c0 \| 0x200 \|
	\| 33 \| blk.2.attn_v.weight \| 0x2cb734c0 \| 0x6e000 \|
	\| 34 \| blk.2.ffn_down_exps.weight \| 0x2cbe14c0 \| 0x6c00000 \|
	\| 35 \| blk.2.ffn_gate_exps.weight \| 0x337e14c0 \| 0x4980000 \|
	\| 36 \| blk.2.ffn_gate_inp.weight \| 0x381614c0 \| 0x100000 \|
	\| 37 \| blk.2.ffn_norm.weight \| 0x382614c0 \| 0x2000 \|
	\| 38 \| blk.2.ffn_up_exps.weight \| 0x382634c0 \| 0x4980000 \|
	\| 39 \| blk.3.attn_k.weight \| 0x3cbe34c0 \| 0x62000 \|
	\| 40 \| blk.3.attn_k_norm.weight \| 0x3cc454c0 \| 0x200 \|
	\| 41 \| blk.3.attn_norm.weight \| 0x3cc456c0 \| 0x2000 \|
	\| 42 \| blk.3.attn_output.weight \| 0x3cc476c0 \| 0x370000 \|
	\| 43 \| blk.3.attn_q.weight \| 0x3cfb76c0 \| 0x310000 \|
	\| 44 \| blk.3.attn_q_norm.weight \| 0x3d2c76c0 \| 0x200 \|
	\| 45 \| blk.3.attn_v.weight \| 0x3d2c78c0 \| 0x6e000 \|
	\| 46 \| blk.3.ffn_down_exps.weight \| 0x3d3358c0 \| 0x5280000 \|
	\| 47 \| blk.3.ffn_gate_exps.weight \| 0x425b58c0 \| 0x4980000 \|
	\| 48 \| blk.3.ffn_gate_inp.weight \| 0x46f358c0 \| 0x100000 \|
	\| 49 \| blk.3.ffn_norm.weight \| 0x470358c0 \| 0x2000 \|
	\| 50 \| blk.3.ffn_up_exps.weight \| 0x470378c0 \| 0x4980000 \|
	\| 51 \| blk.4.attn_k.weight \| 0x4b9b78c0 \| 0x62000 \|
	\| 52 \| blk.4.attn_k_norm.weight \| 0x4ba198c0 \| 0x200 \|
	\| 53 \| blk.4.attn_norm.weight \| 0x4ba19ac0 \| 0x2000 \|
	\| 54 \| blk.4.attn_output.weight \| 0x4ba1bac0 \| 0x370000 \|
	\| 55 \| blk.4.attn_q.weight \| 0x4bd8bac0 \| 0x310000 \|
	\| 56 \| blk.4.attn_q_norm.weight \| 0x4c09bac0 \| 0x200 \|
	\| 57 \| blk.4.attn_v.weight \| 0x4c09bcc0 \| 0x6e000 \|
	\| 58 \| blk.4.ffn_down_exps.weight \| 0x4c109cc0 \| 0x5280000 \|
	\| 59 \| blk.4.ffn_gate_exps.weight \| 0x51389cc0 \| 0x4980000 \|
	\| 60 \| blk.4.ffn_gate_inp.weight \| 0x55d09cc0 \| 0x100000 \|
	\| 61 \| blk.4.ffn_norm.weight \| 0x55e09cc0 \| 0x2000 \|
	\| 62 \| blk.4.ffn_up_exps.weight \| 0x55e0bcc0 \| 0x4980000 \|
	\| 63 \| blk.5.attn_k.weight \| 0x5a78bcc0 \| 0x62000 \|
	\| 64 \| blk.5.attn_k_norm.weight \| 0x5a7edcc0 \| 0x200 \|
	\| 65 \| blk.5.attn_norm.weight \| 0x5a7edec0 \| 0x2000 \|
	\| 66 \| blk.5.attn_output.weight \| 0x5a7efec0 \| 0x370000 \|
	\| 67 \| blk.5.attn_q.weight \| 0x5ab5fec0 \| 0x310000 \|
	\| 68 \| blk.5.attn_q_norm.weight \| 0x5ae6fec0 \| 0x200 \|
	\| 69 \| blk.5.attn_v.weight \| 0x5ae700c0 \| 0x6e000 \|
	\| 70 \| blk.5.ffn_down_exps.weight \| 0x5aede0c0 \| 0x5280000 \|
	\| 71 \| blk.5.ffn_gate_exps.weight \| 0x6015e0c0 \| 0x4980000 \|
	\| 72 \| blk.5.ffn_gate_inp.weight \| 0x64ade0c0 \| 0x100000 \|
	\| 73 \| blk.5.ffn_norm.weight \| 0x64bde0c0 \| 0x2000 \|
	\| 74 \| blk.5.ffn_up_exps.weight \| 0x64be00c0 \| 0x4980000 \|
	\| 75 \| blk.6.attn_k.weight \| 0x695600c0 \| 0x62000 \|
	\| 76 \| blk.6.attn_k_norm.weight \| 0x695c20c0 \| 0x200 \|
	\| 77 \| blk.6.attn_norm.weight \| 0x695c22c0 \| 0x2000 \|
	\| 78 \| blk.6.attn_output.weight \| 0x695c42c0 \| 0x370000 \|
	\| 79 \| blk.6.attn_q.weight \| 0x699342c0 \| 0x310000 \|
	\| 80 \| blk.6.attn_q_norm.weight \| 0x69c442c0 \| 0x200 \|
	\| 81 \| blk.6.attn_v.weight \| 0x69c444c0 \| 0x6e000 \|
	\| 82 \| blk.6.ffn_down_exps.weight \| 0x69cb24c0 \| 0x5280000 \|
	\| 83 \| blk.6.ffn_gate_exps.weight \| 0x6ef324c0 \| 0x4980000 \|
	\| 84 \| blk.6.ffn_gate_inp.weight \| 0x738b24c0 \| 0x100000 \|
	\| 85 \| blk.6.ffn_norm.weight \| 0x739b24c0 \| 0x2000 \|
	\| 86 \| blk.6.ffn_up_exps.weight \| 0x739b44c0 \| 0x4980000 \|
	\| 87 \| blk.7.attn_k.weight \| 0x783344c0 \| 0x62000 \|
	\| 88 \| blk.7.attn_k_norm.weight \| 0x783964c0 \| 0x200 \|
	\| 89 \| blk.7.attn_norm.weight \| 0x783966c0 \| 0x2000 \|
	\| 90 \| blk.7.attn_output.weight \| 0x783986c0 \| 0x370000 \|
	\| 91 \| blk.7.attn_q.weight \| 0x787086c0 \| 0x310000 \|
	\| 92 \| blk.7.attn_q_norm.weight \| 0x78a186c0 \| 0x200 \|
	\| 93 \| blk.7.attn_v.weight \| 0x78a188c0 \| 0x6e000 \|
	\| 94 \| blk.7.ffn_down_exps.weight \| 0x78a868c0 \| 0x5280000 \|
	\| 95 \| blk.7.ffn_gate_exps.weight \| 0x7dd068c0 \| 0x4980000 \|
	\| 96 \| blk.7.ffn_gate_inp.weight \| 0x826868c0 \| 0x100000 \|
	\| 97 \| blk.7.ffn_norm.weight \| 0x827868c0 \| 0x2000 \|
	\| 98 \| blk.7.ffn_up_exps.weight \| 0x827888c0 \| 0x4980000 \|
	\| 99 \| blk.8.attn_k.weight \| 0x871088c0 \| 0x62000 \|
	\| 100 \| blk.8.attn_k_norm.weight \| 0x8716a8c0 \| 0x200 \|
	\| 101 \| blk.8.attn_norm.weight \| 0x8716aac0 \| 0x2000 \|
	\| 102 \| blk.8.attn_output.weight \| 0x8716cac0 \| 0x370000 \|
	\| 103 \| blk.8.attn_q.weight \| 0x874dcac0 \| 0x310000 \|
	\| 104 \| blk.8.attn_q_norm.weight \| 0x877ecac0 \| 0x200 \|
	\| 105 \| blk.8.attn_v.weight \| 0x877eccc0 \| 0x6e000 \|
	\| 106 \| blk.8.ffn_down_exps.weight \| 0x8785acc0 \| 0x5280000 \|
	\| 107 \| blk.8.ffn_gate_exps.weight \| 0x8cadacc0 \| 0x4980000 \|
	\| 108 \| blk.8.ffn_gate_inp.weight \| 0x9145acc0 \| 0x100000 \|
	\| 109 \| blk.8.ffn_norm.weight \| 0x9155acc0 \| 0x2000 \|
	\| 110 \| blk.8.ffn_up_exps.weight \| 0x9155ccc0 \| 0x4980000 \|
	\| 111 \| blk.9.attn_k.weight \| 0x95edccc0 \| 0x62000 \|
	\| 112 \| blk.9.attn_k_norm.weight \| 0x95f3ecc0 \| 0x200 \|
	\| 113 \| blk.9.attn_norm.weight \| 0x95f3eec0 \| 0x2000 \|
	\| 114 \| blk.9.attn_output.weight \| 0x95f40ec0 \| 0x370000 \|
	\| 115 \| blk.9.attn_q.weight \| 0x962b0ec0 \| 0x310000 \|
	\| 116 \| blk.9.attn_q_norm.weight \| 0x965c0ec0 \| 0x200 \|
	\| 117 \| blk.9.attn_v.weight \| 0x965c10c0 \| 0x6e000 \|
	\| 118 \| blk.9.ffn_down_exps.weight \| 0x9662f0c0 \| 0x5280000 \|
	\| 119 \| blk.9.ffn_gate_exps.weight \| 0x9b8af0c0 \| 0x4980000 \|
	\| 120 \| blk.9.ffn_gate_inp.weight \| 0xa022f0c0 \| 0x100000 \|
	\| 121 \| blk.9.ffn_norm.weight \| 0xa032f0c0 \| 0x2000 \|
	\| 122 \| blk.9.ffn_up_exps.weight \| 0xa03310c0 \| 0x4980000 \|
	\| 123 \| blk.10.attn_k.weight \| 0xa4cb10c0 \| 0x62000 \|
	\| 124 \| blk.10.attn_k_norm.weight \| 0xa4d130c0 \| 0x200 \|
	\| 125 \| blk.10.attn_norm.weight \| 0xa4d132c0 \| 0x2000 \|
	\| 126 \| blk.10.attn_output.weight \| 0xa4d152c0 \| 0x370000 \|
	\| 127 \| blk.10.attn_q.weight \| 0xa50852c0 \| 0x310000 \|
	\| 128 \| blk.10.attn_q_norm.weight \| 0xa53952c0 \| 0x200 \|
	\| 129 \| blk.10.attn_v.weight \| 0xa53954c0 \| 0x6e000 \|
	\| 130 \| blk.10.ffn_down_exps.weight \| 0xa54034c0 \| 0x5280000 \|
	\| 131 \| blk.10.ffn_gate_exps.weight \| 0xaa6834c0 \| 0x4980000 \|
	\| 132 \| blk.10.ffn_gate_inp.weight \| 0xaf0034c0 \| 0x100000 \|
	\| 133 \| blk.10.ffn_norm.weight \| 0xaf1034c0 \| 0x2000 \|
	\| 134 \| blk.10.ffn_up_exps.weight \| 0xaf1054c0 \| 0x4980000 \|
	\| 135 \| blk.11.attn_k.weight \| 0xb3a854c0 \| 0x62000 \|
	\| 136 \| blk.11.attn_k_norm.weight \| 0xb3ae74c0 \| 0x200 \|
	\| 137 \| blk.11.attn_norm.weight \| 0xb3ae76c0 \| 0x2000 \|
	\| 138 \| blk.11.attn_output.weight \| 0xb3ae96c0 \| 0x370000 \|
	\| 139 \| blk.11.attn_q.weight \| 0xb3e596c0 \| 0x310000 \|
	\| 140 \| blk.11.attn_q_norm.weight \| 0xb41696c0 \| 0x200 \|
	\| 141 \| blk.11.attn_v.weight \| 0xb41698c0 \| 0x6e000 \|
	\| 142 \| blk.11.ffn_down_exps.weight \| 0xb41d78c0 \| 0x5280000 \|
	\| 143 \| blk.11.ffn_gate_exps.weight \| 0xb94578c0 \| 0x4980000 \|
	\| 144 \| blk.11.ffn_gate_inp.weight \| 0xbddd78c0 \| 0x100000 \|
	\| 145 \| blk.11.ffn_norm.weight \| 0xbded78c0 \| 0x2000 \|
	\| 146 \| blk.11.ffn_up_exps.weight \| 0xbded98c0 \| 0x4980000 \|
	\| 147 \| blk.12.attn_k.weight \| 0xc28598c0 \| 0x62000 \|
	\| 148 \| blk.12.attn_k_norm.weight \| 0xc28bb8c0 \| 0x200 \|
	\| 149 \| blk.12.attn_norm.weight \| 0xc28bbac0 \| 0x2000 \|
	\| 150 \| blk.12.attn_output.weight \| 0xc28bdac0 \| 0x370000 \|
	\| 151 \| blk.12.attn_q.weight \| 0xc2c2dac0 \| 0x310000 \|
	\| 152 \| blk.12.attn_q_norm.weight \| 0xc2f3dac0 \| 0x200 \|
	\| 153 \| blk.12.attn_v.weight \| 0xc2f3dcc0 \| 0x6e000 \|
	\| 154 \| blk.12.ffn_down_exps.weight \| 0xc2fabcc0 \| 0x5280000 \|
	\| 155 \| blk.12.ffn_gate_exps.weight \| 0xc822bcc0 \| 0x4980000 \|
	\| 156 \| blk.12.ffn_gate_inp.weight \| 0xccbabcc0 \| 0x100000 \|
	\| 157 \| blk.12.ffn_norm.weight \| 0xcccabcc0 \| 0x2000 \|
	\| 158 \| blk.12.ffn_up_exps.weight \| 0xcccadcc0 \| 0x4980000 \|
	\| 159 \| blk.13.attn_k.weight \| 0xd162dcc0 \| 0x62000 \|
	\| 160 \| blk.13.attn_k_norm.weight \| 0xd168fcc0 \| 0x200 \|
	\| 161 \| blk.13.attn_norm.weight \| 0xd168fec0 \| 0x2000 \|
	\| 162 \| blk.13.attn_output.weight \| 0xd1691ec0 \| 0x370000 \|
	\| 163 \| blk.13.attn_q.weight \| 0xd1a01ec0 \| 0x310000 \|
	\| 164 \| blk.13.attn_q_norm.weight \| 0xd1d11ec0 \| 0x200 \|
	\| 165 \| blk.13.attn_v.weight \| 0xd1d120c0 \| 0x6e000 \|
	\| 166 \| blk.13.ffn_down_exps.weight \| 0xd1d800c0 \| 0x6c00000 \|
	\| 167 \| blk.13.ffn_gate_exps.weight \| 0xd89800c0 \| 0x5280000 \|
	\| 168 \| blk.13.ffn_gate_inp.weight \| 0xddc000c0 \| 0x100000 \|
	\| 169 \| blk.13.ffn_norm.weight \| 0xddd000c0 \| 0x2000 \|
	\| 170 \| blk.13.ffn_up_exps.weight \| 0xddd020c0 \| 0x5280000 \|
	\| 171 \| blk.14.attn_k.weight \| 0xe2f820c0 \| 0x62000 \|
	\| 172 \| blk.14.attn_k_norm.weight \| 0xe2fe40c0 \| 0x200 \|
	\| 173 \| blk.14.attn_norm.weight \| 0xe2fe42c0 \| 0x2000 \|
	\| 174 \| blk.14.attn_output.weight \| 0xe2fe62c0 \| 0x370000 \|
	\| 175 \| blk.14.attn_q.weight \| 0xe33562c0 \| 0x310000 \|
	\| 176 \| blk.14.attn_q_norm.weight \| 0xe36662c0 \| 0x200 \|
	\| 177 \| blk.14.attn_v.weight \| 0xe36664c0 \| 0x6e000 \|
	\| 178 \| blk.14.ffn_down_exps.weight \| 0xe36d44c0 \| 0x5280000 \|
	\| 179 \| blk.14.ffn_gate_exps.weight \| 0xe89544c0 \| 0x4980000 \|
	\| 180 \| blk.14.ffn_gate_inp.weight \| 0xed2d44c0 \| 0x100000 \|
	\| 181 \| blk.14.ffn_norm.weight \| 0xed3d44c0 \| 0x2000 \|
	\| 182 \| blk.14.ffn_up_exps.weight \| 0xed3d64c0 \| 0x4980000 \|
	\| 183 \| blk.15.attn_k.weight \| 0xf1d564c0 \| 0x62000 \|
	\| 184 \| blk.15.attn_k_norm.weight \| 0xf1db84c0 \| 0x200 \|
	\| 185 \| blk.15.attn_norm.weight \| 0xf1db86c0 \| 0x2000 \|
	\| 186 \| blk.15.attn_output.weight \| 0xf1dba6c0 \| 0x370000 \|
	\| 187 \| blk.15.attn_q.weight \| 0xf212a6c0 \| 0x310000 \|
	\| 188 \| blk.15.attn_q_norm.weight \| 0xf243a6c0 \| 0x200 \|
	\| 189 \| blk.15.attn_v.weight \| 0xf243a8c0 \| 0x6e000 \|
	\| 190 \| blk.15.ffn_down_exps.weight \| 0xf24a88c0 \| 0x6c00000 \|
	\| 191 \| blk.15.ffn_gate_exps.weight \| 0xf90a88c0 \| 0x5280000 \|
	\| 192 \| blk.15.ffn_gate_inp.weight \| 0xfe3288c0 \| 0x100000 \|
	\| 193 \| blk.15.ffn_norm.weight \| 0xfe4288c0 \| 0x2000 \|
	\| 194 \| blk.15.ffn_up_exps.weight \| 0xfe42a8c0 \| 0x5280000 \|
	\| 195 \| blk.16.attn_k.weight \| 0x1036aa8c0 \| 0x62000 \|
	\| 196 \| blk.16.attn_k_norm.weight \| 0x10370c8c0 \| 0x200 \|
	\| 197 \| blk.16.attn_norm.weight \| 0x10370cac0 \| 0x2000 \|
	\| 198 \| blk.16.attn_output.weight \| 0x10370eac0 \| 0x370000 \|
	\| 199 \| blk.16.attn_q.weight \| 0x103a7eac0 \| 0x310000 \|
	\| 200 \| blk.16.attn_q_norm.weight \| 0x103d8eac0 \| 0x200 \|
	\| 201 \| blk.16.attn_v.weight \| 0x103d8ecc0 \| 0x6e000 \|
	\| 202 \| blk.16.ffn_down_exps.weight \| 0x103dfccc0 \| 0x5280000 \|
	\| 203 \| blk.16.ffn_gate_exps.weight \| 0x10907ccc0 \| 0x4980000 \|
	\| 204 \| blk.16.ffn_gate_inp.weight \| 0x10d9fccc0 \| 0x100000 \|
	\| 205 \| blk.16.ffn_norm.weight \| 0x10dafccc0 \| 0x2000 \|
	\| 206 \| blk.16.ffn_up_exps.weight \| 0x10dafecc0 \| 0x4980000 \|
	\| 207 \| blk.17.attn_k.weight \| 0x11247ecc0 \| 0x62000 \|
	\| 208 \| blk.17.attn_k_norm.weight \| 0x1124e0cc0 \| 0x200 \|
	\| 209 \| blk.17.attn_norm.weight \| 0x1124e0ec0 \| 0x2000 \|
	\| 210 \| blk.17.attn_output.weight \| 0x1124e2ec0 \| 0x370000 \|
	\| 211 \| blk.17.attn_q.weight \| 0x112852ec0 \| 0x310000 \|
	\| 212 \| blk.17.attn_q_norm.weight \| 0x112b62ec0 \| 0x200 \|
	\| 213 \| blk.17.attn_v.weight \| 0x112b630c0 \| 0x6e000 \|
	\| 214 \| blk.17.ffn_down_exps.weight \| 0x112bd10c0 \| 0x5280000 \|
	\| 215 \| blk.17.ffn_gate_exps.weight \| 0x117e510c0 \| 0x4980000 \|
	\| 216 \| blk.17.ffn_gate_inp.weight \| 0x11c7d10c0 \| 0x100000 \|
	\| 217 \| blk.17.ffn_norm.weight \| 0x11c8d10c0 \| 0x2000 \|
	\| 218 \| blk.17.ffn_up_exps.weight \| 0x11c8d30c0 \| 0x4980000 \|
	\| 219 \| blk.18.attn_k.weight \| 0x1212530c0 \| 0x62000 \|
	\| 220 \| blk.18.attn_k_norm.weight \| 0x1212b50c0 \| 0x200 \|
	\| 221 \| blk.18.attn_norm.weight \| 0x1212b52c0 \| 0x2000 \|
	\| 222 \| blk.18.attn_output.weight \| 0x1212b72c0 \| 0x370000 \|
	\| 223 \| blk.18.attn_q.weight \| 0x1216272c0 \| 0x310000 \|
	\| 224 \| blk.18.attn_q_norm.weight \| 0x1219372c0 \| 0x200 \|
	\| 225 \| blk.18.attn_v.weight \| 0x1219374c0 \| 0x6e000 \|
	\| 226 \| blk.18.ffn_down_exps.weight \| 0x1219a54c0 \| 0x5280000 \|
	\| 227 \| blk.18.ffn_gate_exps.weight \| 0x126c254c0 \| 0x4980000 \|
	\| 228 \| blk.18.ffn_gate_inp.weight \| 0x12b5a54c0 \| 0x100000 \|
	\| 229 \| blk.18.ffn_norm.weight \| 0x12b6a54c0 \| 0x2000 \|
	\| 230 \| blk.18.ffn_up_exps.weight \| 0x12b6a74c0 \| 0x4980000 \|
	\| 231 \| blk.19.attn_k.weight \| 0x1300274c0 \| 0x62000 \|
	\| 232 \| blk.19.attn_k_norm.weight \| 0x1300894c0 \| 0x200 \|
	\| 233 \| blk.19.attn_norm.weight \| 0x1300896c0 \| 0x2000 \|
	\| 234 \| blk.19.attn_output.weight \| 0x13008b6c0 \| 0x370000 \|
	\| 235 \| blk.19.attn_q.weight \| 0x1303fb6c0 \| 0x310000 \|
	\| 236 \| blk.19.attn_q_norm.weight \| 0x13070b6c0 \| 0x200 \|
	\| 237 \| blk.19.attn_v.weight \| 0x13070b8c0 \| 0x6e000 \|
	\| 238 \| blk.19.ffn_down_exps.weight \| 0x1307798c0 \| 0x5280000 \|
	\| 239 \| blk.19.ffn_gate_exps.weight \| 0x1359f98c0 \| 0x4980000 \|
	\| 240 \| blk.19.ffn_gate_inp.weight \| 0x13a3798c0 \| 0x100000 \|
	\| 241 \| blk.19.ffn_norm.weight \| 0x13a4798c0 \| 0x2000 \|
	\| 242 \| blk.19.ffn_up_exps.weight \| 0x13a47b8c0 \| 0x4980000 \|
	\| 243 \| blk.20.attn_k.weight \| 0x13edfb8c0 \| 0x62000 \|
	\| 244 \| blk.20.attn_k_norm.weight \| 0x13ee5d8c0 \| 0x200 \|
	\| 245 \| blk.20.attn_norm.weight \| 0x13ee5dac0 \| 0x2000 \|
	\| 246 \| blk.20.attn_output.weight \| 0x13ee5fac0 \| 0x370000 \|
	\| 247 \| blk.20.attn_q.weight \| 0x13f1cfac0 \| 0x310000 \|
	\| 248 \| blk.20.attn_q_norm.weight \| 0x13f4dfac0 \| 0x200 \|
	\| 249 \| blk.20.attn_v.weight \| 0x13f4dfcc0 \| 0x6e000 \|
	\| 250 \| blk.20.ffn_down_exps.weight \| 0x13f54dcc0 \| 0x5280000 \|
	\| 251 \| blk.20.ffn_gate_exps.weight \| 0x1447cdcc0 \| 0x4980000 \|
	\| 252 \| blk.20.ffn_gate_inp.weight \| 0x14914dcc0 \| 0x100000 \|
	\| 253 \| blk.20.ffn_norm.weight \| 0x14924dcc0 \| 0x2000 \|
	\| 254 \| blk.20.ffn_up_exps.weight \| 0x14924fcc0 \| 0x4980000 \|
	\| 255 \| blk.21.attn_k.weight \| 0x14dbcfcc0 \| 0x62000 \|
	\| 256 \| blk.21.attn_k_norm.weight \| 0x14dc31cc0 \| 0x200 \|
	\| 257 \| blk.21.attn_norm.weight \| 0x14dc31ec0 \| 0x2000 \|
	\| 258 \| blk.21.attn_output.weight \| 0x14dc33ec0 \| 0x370000 \|
	\| 259 \| blk.21.attn_q.weight \| 0x14dfa3ec0 \| 0x310000 \|
	\| 260 \| blk.21.attn_q_norm.weight \| 0x14e2b3ec0 \| 0x200 \|
	\| 261 \| blk.21.attn_v.weight \| 0x14e2b40c0 \| 0x6e000 \|
	\| 262 \| blk.21.ffn_down_exps.weight \| 0x14e3220c0 \| 0x5280000 \|
	\| 263 \| blk.21.ffn_gate_exps.weight \| 0x1535a20c0 \| 0x4980000 \|
	\| 264 \| blk.21.ffn_gate_inp.weight \| 0x157f220c0 \| 0x100000 \|
	\| 265 \| blk.21.ffn_norm.weight \| 0x1580220c0 \| 0x2000 \|
	\| 266 \| blk.21.ffn_up_exps.weight \| 0x1580240c0 \| 0x4980000 \|
	\| 267 \| blk.22.attn_k.weight \| 0x15c9a40c0 \| 0x62000 \|
	\| 268 \| blk.22.attn_k_norm.weight \| 0x15ca060c0 \| 0x200 \|
	\| 269 \| blk.22.attn_norm.weight \| 0x15ca062c0 \| 0x2000 \|
	\| 270 \| blk.22.attn_output.weight \| 0x15ca082c0 \| 0x370000 \|
	\| 271 \| blk.22.attn_q.weight \| 0x15cd782c0 \| 0x310000 \|
	\| 272 \| blk.22.attn_q_norm.weight \| 0x15d0882c0 \| 0x200 \|
	\| 273 \| blk.22.attn_v.weight \| 0x15d0884c0 \| 0x6e000 \|
	\| 274 \| blk.22.ffn_down_exps.weight \| 0x15d0f64c0 \| 0x5280000 \|
	\| 275 \| blk.22.ffn_gate_exps.weight \| 0x1623764c0 \| 0x4980000 \|
	\| 276 \| blk.22.ffn_gate_inp.weight \| 0x166cf64c0 \| 0x100000 \|
	\| 277 \| blk.22.ffn_norm.weight \| 0x166df64c0 \| 0x2000 \|
	\| 278 \| blk.22.ffn_up_exps.weight \| 0x166df84c0 \| 0x4980000 \|
	\| 279 \| blk.23.attn_k.weight \| 0x16b7784c0 \| 0x62000 \|
	\| 280 \| blk.23.attn_k_norm.weight \| 0x16b7da4c0 \| 0x200 \|
	\| 281 \| blk.23.attn_norm.weight \| 0x16b7da6c0 \| 0x2000 \|
	\| 282 \| blk.23.attn_output.weight \| 0x16b7dc6c0 \| 0x370000 \|
	\| 283 \| blk.23.attn_q.weight \| 0x16bb4c6c0 \| 0x310000 \|
	\| 284 \| blk.23.attn_q_norm.weight \| 0x16be5c6c0 \| 0x200 \|
	\| 285 \| blk.23.attn_v.weight \| 0x16be5c8c0 \| 0x6e000 \|
	\| 286 \| blk.23.ffn_down_exps.weight \| 0x16beca8c0 \| 0x5280000 \|
	\| 287 \| blk.23.ffn_gate_exps.weight \| 0x17114a8c0 \| 0x4980000 \|
	\| 288 \| blk.23.ffn_gate_inp.weight \| 0x175aca8c0 \| 0x100000 \|
	\| 289 \| blk.23.ffn_norm.weight \| 0x175bca8c0 \| 0x2000 \|
	\| 290 \| blk.23.ffn_up_exps.weight \| 0x175bcc8c0 \| 0x4980000 \|
	\| 291 \| blk.24.attn_k.weight \| 0x17a54c8c0 \| 0x6e000 \|
	\| 292 \| blk.24.attn_k_norm.weight \| 0x17a5ba8c0 \| 0x200 \|
	\| 293 \| blk.24.attn_norm.weight \| 0x17a5baac0 \| 0x2000 \|
	\| 294 \| blk.24.attn_output.weight \| 0x17a5bcac0 \| 0x370000 \|
	\| 295 \| blk.24.attn_q.weight \| 0x17a92cac0 \| 0x370000 \|
	\| 296 \| blk.24.attn_q_norm.weight \| 0x17ac9cac0 \| 0x200 \|
	\| 297 \| blk.24.attn_v.weight \| 0x17ac9ccc0 \| 0x6e000 \|
	\| 298 \| blk.24.ffn_down_exps.weight \| 0x17ad0acc0 \| 0x5280000 \|
	\| 299 \| blk.24.ffn_gate_exps.weight \| 0x17ff8acc0 \| 0x4980000 \|
	\| 300 \| blk.24.ffn_gate_inp.weight \| 0x18490acc0 \| 0x100000 \|
	\| 301 \| blk.24.ffn_norm.weight \| 0x184a0acc0 \| 0x2000 \|
	\| 302 \| blk.24.ffn_up_exps.weight \| 0x184a0ccc0 \| 0x4980000 \|
	\| 303 \| blk.25.attn_k.weight \| 0x18938ccc0 \| 0x6e000 \|
	\| 304 \| blk.25.attn_k_norm.weight \| 0x1893facc0 \| 0x200 \|
	\| 305 \| blk.25.attn_norm.weight \| 0x1893faec0 \| 0x2000 \|
	\| 306 \| blk.25.attn_output.weight \| 0x1893fcec0 \| 0x370000 \|
	\| 307 \| blk.25.attn_q.weight \| 0x18976cec0 \| 0x370000 \|
	\| 308 \| blk.25.attn_q_norm.weight \| 0x189adcec0 \| 0x200 \|
	\| 309 \| blk.25.attn_v.weight \| 0x189add0c0 \| 0x6e000 \|
	\| 310 \| blk.25.ffn_down_exps.weight \| 0x189b4b0c0 \| 0x6c00000 \|
	\| 311 \| blk.25.ffn_gate_exps.weight \| 0x19074b0c0 \| 0x5280000 \|
	\| 312 \| blk.25.ffn_gate_inp.weight \| 0x1959cb0c0 \| 0x100000 \|
	\| 313 \| blk.25.ffn_norm.weight \| 0x195acb0c0 \| 0x2000 \|
	\| 314 \| blk.25.ffn_up_exps.weight \| 0x195acd0c0 \| 0x5280000 \|
	\| 315 \| blk.26.attn_k.weight \| 0x19ad4d0c0 \| 0x6e000 \|
	\| 316 \| blk.26.attn_k_norm.weight \| 0x19adbb0c0 \| 0x200 \|
	\| 317 \| blk.26.attn_norm.weight \| 0x19adbb2c0 \| 0x2000 \|
	\| 318 \| blk.26.attn_output.weight \| 0x19adbd2c0 \| 0x370000 \|
	\| 319 \| blk.26.attn_q.weight \| 0x19b12d2c0 \| 0x370000 \|
	\| 320 \| blk.26.attn_q_norm.weight \| 0x19b49d2c0 \| 0x200 \|
	\| 321 \| blk.26.attn_v.weight \| 0x19b49d4c0 \| 0x6e000 \|
	\| 322 \| blk.26.ffn_down_exps.weight \| 0x19b50b4c0 \| 0x5280000 \|
	\| 323 \| blk.26.ffn_gate_exps.weight \| 0x1a078b4c0 \| 0x4980000 \|
	\| 324 \| blk.26.ffn_gate_inp.weight \| 0x1a510b4c0 \| 0x100000 \|
	\| 325 \| blk.26.ffn_norm.weight \| 0x1a520b4c0 \| 0x2000 \|
	\| 326 \| blk.26.ffn_up_exps.weight \| 0x1a520d4c0 \| 0x4980000 \|
	\| 327 \| blk.27.attn_k.weight \| 0x1a9b8d4c0 \| 0x6e000 \|
	\| 328 \| blk.27.attn_k_norm.weight \| 0x1a9bfb4c0 \| 0x200 \|
	\| 329 \| blk.27.attn_norm.weight \| 0x1a9bfb6c0 \| 0x2000 \|
	\| 330 \| blk.27.attn_output.weight \| 0x1a9bfd6c0 \| 0x370000 \|
	\| 331 \| blk.27.attn_q.weight \| 0x1a9f6d6c0 \| 0x370000 \|
	\| 332 \| blk.27.attn_q_norm.weight \| 0x1aa2dd6c0 \| 0x200 \|
	\| 333 \| blk.27.attn_v.weight \| 0x1aa2dd8c0 \| 0x6e000 \|
	\| 334 \| blk.27.ffn_down_exps.weight \| 0x1aa34b8c0 \| 0x5280000 \|
	\| 335 \| blk.27.ffn_gate_exps.weight \| 0x1af5cb8c0 \| 0x5280000 \|
	\| 336 \| blk.27.ffn_gate_inp.weight \| 0x1b484b8c0 \| 0x100000 \|
	\| 337 \| blk.27.ffn_norm.weight \| 0x1b494b8c0 \| 0x2000 \|
	\| 338 \| blk.27.ffn_up_exps.weight \| 0x1b494d8c0 \| 0x5280000 \|
	\| 339 \| blk.28.attn_k.weight \| 0x1b9bcd8c0 \| 0x6e000 \|
	\| 340 \| blk.28.attn_k_norm.weight \| 0x1b9c3b8c0 \| 0x200 \|
	\| 341 \| blk.28.attn_norm.weight \| 0x1b9c3bac0 \| 0x2000 \|
	\| 342 \| blk.28.attn_output.weight \| 0x1b9c3dac0 \| 0x370000 \|
	\| 343 \| blk.28.attn_q.weight \| 0x1b9fadac0 \| 0x370000 \|
	\| 344 \| blk.28.attn_q_norm.weight \| 0x1ba31dac0 \| 0x200 \|
	\| 345 \| blk.28.attn_v.weight \| 0x1ba31dcc0 \| 0x6e000 \|
	\| 346 \| blk.28.ffn_down_exps.weight \| 0x1ba38bcc0 \| 0x6c00000 \|
	\| 347 \| blk.28.ffn_gate_exps.weight \| 0x1c0f8bcc0 \| 0x5280000 \|
	\| 348 \| blk.28.ffn_gate_inp.weight \| 0x1c620bcc0 \| 0x100000 \|
	\| 349 \| blk.28.ffn_norm.weight \| 0x1c630bcc0 \| 0x2000 \|
	\| 350 \| blk.28.ffn_up_exps.weight \| 0x1c630dcc0 \| 0x5280000 \|
	\| 351 \| blk.29.attn_k.weight \| 0x1cb58dcc0 \| 0x6e000 \|
	\| 352 \| blk.29.attn_k_norm.weight \| 0x1cb5fbcc0 \| 0x200 \|
	\| 353 \| blk.29.attn_norm.weight \| 0x1cb5fbec0 \| 0x2000 \|
	\| 354 \| blk.29.attn_output.weight \| 0x1cb5fdec0 \| 0x370000 \|
	\| 355 \| blk.29.attn_q.weight \| 0x1cb96dec0 \| 0x370000 \|
	\| 356 \| blk.29.attn_q_norm.weight \| 0x1cbcddec0 \| 0x200 \|
	\| 357 \| blk.29.attn_v.weight \| 0x1cbcde0c0 \| 0x6e000 \|
	\| 358 \| blk.29.ffn_down_exps.weight \| 0x1cbd4c0c0 \| 0x6c00000 \|
	\| 359 \| blk.29.ffn_gate_exps.weight \| 0x1d294c0c0 \| 0x5280000 \|
	\| 360 \| blk.29.ffn_gate_inp.weight \| 0x1d7bcc0c0 \| 0x100000 \|
	\| 361 \| blk.29.ffn_norm.weight \| 0x1d7ccc0c0 \| 0x2000 \|
	\| 362 \| blk.29.ffn_up_exps.weight \| 0x1d7cce0c0 \| 0x5280000 \|
	\| 363 \| blk.30.attn_k.weight \| 0x1dcf4e0c0 \| 0x6e000 \|
	\| 364 \| blk.30.attn_k_norm.weight \| 0x1dcfbc0c0 \| 0x200 \|
	\| 365 \| blk.30.attn_norm.weight \| 0x1dcfbc2c0 \| 0x2000 \|
	\| 366 \| blk.30.attn_output.weight \| 0x1dcfbe2c0 \| 0x370000 \|
	\| 367 \| blk.30.attn_q.weight \| 0x1dd32e2c0 \| 0x370000 \|
	\| 368 \| blk.30.attn_q_norm.weight \| 0x1dd69e2c0 \| 0x200 \|
	\| 369 \| blk.30.attn_v.weight \| 0x1dd69e4c0 \| 0x6e000 \|
	\| 370 \| blk.30.ffn_down_exps.weight \| 0x1dd70c4c0 \| 0x6c00000 \|
	\| 371 \| blk.30.ffn_gate_exps.weight \| 0x1e430c4c0 \| 0x5280000 \|
	\| 372 \| blk.30.ffn_gate_inp.weight \| 0x1e958c4c0 \| 0x100000 \|
	\| 373 \| blk.30.ffn_norm.weight \| 0x1e968c4c0 \| 0x2000 \|
	\| 374 \| blk.30.ffn_up_exps.weight \| 0x1e968e4c0 \| 0x5280000 \|
	\| 375 \| blk.31.attn_k.weight \| 0x1ee90e4c0 \| 0x6e000 \|
	\| 376 \| blk.31.attn_k_norm.weight \| 0x1ee97c4c0 \| 0x200 \|
	\| 377 \| blk.31.attn_norm.weight \| 0x1ee97c6c0 \| 0x2000 \|
	\| 378 \| blk.31.attn_output.weight \| 0x1ee97e6c0 \| 0x370000 \|
	\| 379 \| blk.31.attn_q.weight \| 0x1eecee6c0 \| 0x370000 \|
	\| 380 \| blk.31.attn_q_norm.weight \| 0x1ef05e6c0 \| 0x200 \|
	\| 381 \| blk.31.attn_v.weight \| 0x1ef05e8c0 \| 0x6e000 \|
	\| 382 \| blk.31.ffn_down_exps.weight \| 0x1ef0cc8c0 \| 0x6c00000 \|
	\| 383 \| blk.31.ffn_gate_exps.weight \| 0x1f5ccc8c0 \| 0x5280000 \|
	\| 384 \| blk.31.ffn_gate_inp.weight \| 0x1faf4c8c0 \| 0x100000 \|
	\| 385 \| blk.31.ffn_norm.weight \| 0x1fb04c8c0 \| 0x2000 \|
	\| 386 \| blk.31.ffn_up_exps.weight \| 0x1fb04e8c0 \| 0x5280000 \|
	\| 387 \| blk.32.attn_k.weight \| 0x2002ce8c0 \| 0x6e000 \|
	\| 388 \| blk.32.attn_k_norm.weight \| 0x20033c8c0 \| 0x200 \|
	\| 389 \| blk.32.attn_norm.weight \| 0x20033cac0 \| 0x2000 \|
	\| 390 \| blk.32.attn_output.weight \| 0x20033eac0 \| 0x370000 \|
	\| 391 \| blk.32.attn_q.weight \| 0x2006aeac0 \| 0x370000 \|
	\| 392 \| blk.32.attn_q_norm.weight \| 0x200a1eac0 \| 0x200 \|
	\| 393 \| blk.32.attn_v.weight \| 0x200a1ecc0 \| 0x6e000 \|
	\| 394 \| blk.32.ffn_down_exps.weight \| 0x200a8ccc0 \| 0x6c00000 \|
	\| 395 \| blk.32.ffn_gate_exps.weight \| 0x20768ccc0 \| 0x5280000 \|
	\| 396 \| blk.32.ffn_gate_inp.weight \| 0x20c90ccc0 \| 0x100000 \|
	\| 397 \| blk.32.ffn_norm.weight \| 0x20ca0ccc0 \| 0x2000 \|
	\| 398 \| blk.32.ffn_up_exps.weight \| 0x20ca0ecc0 \| 0x5280000 \|
	\| 399 \| blk.33.attn_k.weight \| 0x211c8ecc0 \| 0x6e000 \|
	\| 400 \| blk.33.attn_k_norm.weight \| 0x211cfccc0 \| 0x200 \|
	\| 401 \| blk.33.attn_norm.weight \| 0x211cfcec0 \| 0x2000 \|
	\| 402 \| blk.33.attn_output.weight \| 0x211cfeec0 \| 0x370000 \|
	\| 403 \| blk.33.attn_q.weight \| 0x21206eec0 \| 0x370000 \|
	\| 404 \| blk.33.attn_q_norm.weight \| 0x2123deec0 \| 0x200 \|
	\| 405 \| blk.33.attn_v.weight \| 0x2123df0c0 \| 0x6e000 \|
	\| 406 \| blk.33.ffn_down_exps.weight \| 0x21244d0c0 \| 0x6c00000 \|
	\| 407 \| blk.33.ffn_gate_exps.weight \| 0x21904d0c0 \| 0x5280000 \|
	\| 408 \| blk.33.ffn_gate_inp.weight \| 0x21e2cd0c0 \| 0x100000 \|
	\| 409 \| blk.33.ffn_norm.weight \| 0x21e3cd0c0 \| 0x2000 \|
	\| 410 \| blk.33.ffn_up_exps.weight \| 0x21e3cf0c0 \| 0x5280000 \|
	\| 411 \| blk.34.attn_k.weight \| 0x22364f0c0 \| 0x6e000 \|
	\| 412 \| blk.34.attn_k_norm.weight \| 0x2236bd0c0 \| 0x200 \|
	\| 413 \| blk.34.attn_norm.weight \| 0x2236bd2c0 \| 0x2000 \|
	\| 414 \| blk.34.attn_output.weight \| 0x2236bf2c0 \| 0x370000 \|
	\| 415 \| blk.34.attn_q.weight \| 0x223a2f2c0 \| 0x370000 \|
	\| 416 \| blk.34.attn_q_norm.weight \| 0x223d9f2c0 \| 0x200 \|
	\| 417 \| blk.34.attn_v.weight \| 0x223d9f4c0 \| 0x6e000 \|
	\| 418 \| blk.34.ffn_down_exps.weight \| 0x223e0d4c0 \| 0x6c00000 \|
	\| 419 \| blk.34.ffn_gate_exps.weight \| 0x22aa0d4c0 \| 0x5280000 \|
	\| 420 \| blk.34.ffn_gate_inp.weight \| 0x22fc8d4c0 \| 0x100000 \|
	\| 421 \| blk.34.ffn_norm.weight \| 0x22fd8d4c0 \| 0x2000 \|
	\| 422 \| blk.34.ffn_up_exps.weight \| 0x22fd8f4c0 \| 0x5280000 \|
	\| 423 \| blk.35.attn_k.weight \| 0x23500f4c0 \| 0x6e000 \|
	\| 424 \| blk.35.attn_k_norm.weight \| 0x23507d4c0 \| 0x200 \|
	\| 425 \| blk.35.attn_norm.weight \| 0x23507d6c0 \| 0x2000 \|
	\| 426 \| blk.35.attn_output.weight \| 0x23507f6c0 \| 0x370000 \|
	\| 427 \| blk.35.attn_q.weight \| 0x2353ef6c0 \| 0x370000 \|
	\| 428 \| blk.35.attn_q_norm.weight \| 0x23575f6c0 \| 0x200 \|
	\| 429 \| blk.35.attn_v.weight \| 0x23575f8c0 \| 0x6e000 \|
	\| 430 \| blk.35.ffn_down_exps.weight \| 0x2357cd8c0 \| 0x6c00000 \|
	\| 431 \| blk.35.ffn_gate_exps.weight \| 0x23c3cd8c0 \| 0x5280000 \|
	\| 432 \| blk.35.ffn_gate_inp.weight \| 0x24164d8c0 \| 0x100000 \|
	\| 433 \| blk.35.ffn_norm.weight \| 0x24174d8c0 \| 0x2000 \|
	\| 434 \| blk.35.ffn_up_exps.weight \| 0x24174f8c0 \| 0x5280000 \|
	\| 435 \| blk.36.attn_k.weight \| 0x2469cf8c0 \| 0x6e000 \|
	\| 436 \| blk.36.attn_k_norm.weight \| 0x246a3d8c0 \| 0x200 \|
	\| 437 \| blk.36.attn_norm.weight \| 0x246a3dac0 \| 0x2000 \|
	\| 438 \| blk.36.attn_output.weight \| 0x246a3fac0 \| 0x370000 \|
	\| 439 \| blk.36.attn_q.weight \| 0x246dafac0 \| 0x370000 \|
	\| 440 \| blk.36.attn_q_norm.weight \| 0x24711fac0 \| 0x200 \|
	\| 441 \| blk.36.attn_v.weight \| 0x24711fcc0 \| 0x6e000 \|
	\| 442 \| blk.36.ffn_down_exps.weight \| 0x24718dcc0 \| 0x6c00000 \|
	\| 443 \| blk.36.ffn_gate_exps.weight \| 0x24dd8dcc0 \| 0x5280000 \|
	\| 444 \| blk.36.ffn_gate_inp.weight \| 0x25300dcc0 \| 0x100000 \|
	\| 445 \| blk.36.ffn_norm.weight \| 0x25310dcc0 \| 0x2000 \|
	\| 446 \| blk.36.ffn_up_exps.weight \| 0x25310fcc0 \| 0x5280000 \|
	\| 447 \| blk.37.attn_k.weight \| 0x25838fcc0 \| 0x6e000 \|
	\| 448 \| blk.37.attn_k_norm.weight \| 0x2583fdcc0 \| 0x200 \|
	\| 449 \| blk.37.attn_norm.weight \| 0x2583fdec0 \| 0x2000 \|
	\| 450 \| blk.37.attn_output.weight \| 0x2583ffec0 \| 0x370000 \|
	\| 451 \| blk.37.attn_q.weight \| 0x25876fec0 \| 0x370000 \|
	\| 452 \| blk.37.attn_q_norm.weight \| 0x258adfec0 \| 0x200 \|
	\| 453 \| blk.37.attn_v.weight \| 0x258ae00c0 \| 0x6e000 \|
	\| 454 \| blk.37.ffn_down_exps.weight \| 0x258b4e0c0 \| 0x6c00000 \|
	\| 455 \| blk.37.ffn_gate_exps.weight \| 0x25f74e0c0 \| 0x5280000 \|
	\| 456 \| blk.37.ffn_gate_inp.weight \| 0x2649ce0c0 \| 0x100000 \|
	\| 457 \| blk.37.ffn_norm.weight \| 0x264ace0c0 \| 0x2000 \|
	\| 458 \| blk.37.ffn_up_exps.weight \| 0x264ad00c0 \| 0x5280000 \|
	\| 459 \| blk.38.attn_k.weight \| 0x269d500c0 \| 0x6e000 \|
	\| 460 \| blk.38.attn_k_norm.weight \| 0x269dbe0c0 \| 0x200 \|
	\| 461 \| blk.38.attn_norm.weight \| 0x269dbe2c0 \| 0x2000 \|
	\| 462 \| blk.38.attn_output.weight \| 0x269dc02c0 \| 0x370000 \|
	\| 463 \| blk.38.attn_q.weight \| 0x26a1302c0 \| 0x370000 \|
	\| 464 \| blk.38.attn_q_norm.weight \| 0x26a4a02c0 \| 0x200 \|
	\| 465 \| blk.38.attn_v.weight \| 0x26a4a04c0 \| 0x6e000 \|
	\| 466 \| blk.38.ffn_down_exps.weight \| 0x26a50e4c0 \| 0x6c00000 \|
	\| 467 \| blk.38.ffn_gate_exps.weight \| 0x27110e4c0 \| 0x5280000 \|
	\| 468 \| blk.38.ffn_gate_inp.weight \| 0x27638e4c0 \| 0x100000 \|
	\| 469 \| blk.38.ffn_norm.weight \| 0x27648e4c0 \| 0x2000 \|
	\| 470 \| blk.38.ffn_up_exps.weight \| 0x2764904c0 \| 0x5280000 \|
	\| 471 \| blk.39.attn_k.weight \| 0x27b7104c0 \| 0x6e000 \|
	\| 472 \| blk.39.attn_k_norm.weight \| 0x27b77e4c0 \| 0x200 \|
	\| 473 \| blk.39.attn_norm.weight \| 0x27b77e6c0 \| 0x2000 \|
	\| 474 \| blk.39.attn_output.weight \| 0x27b7806c0 \| 0x370000 \|
	\| 475 \| blk.39.attn_q.weight \| 0x27baf06c0 \| 0x370000 \|
	\| 476 \| blk.39.attn_q_norm.weight \| 0x27be606c0 \| 0x200 \|
	\| 477 \| blk.39.attn_v.weight \| 0x27be608c0 \| 0x6e000 \|
	\| 478 \| blk.39.ffn_down_exps.weight \| 0x27bece8c0 \| 0x6c00000 \|
	\| 479 \| blk.39.ffn_gate_exps.weight \| 0x282ace8c0 \| 0x5280000 \|
	\| 480 \| blk.39.ffn_gate_inp.weight \| 0x287d4e8c0 \| 0x100000 \|
	\| 481 \| blk.39.ffn_norm.weight \| 0x287e4e8c0 \| 0x2000 \|
	\| 482 \| blk.39.ffn_up_exps.weight \| 0x287e508c0 \| 0x5280000 \|
	\| 483 \| blk.40.attn_k.weight \| 0x28d0d08c0 \| 0x6e000 \|
	\| 484 \| blk.40.attn_k_norm.weight \| 0x28d13e8c0 \| 0x200 \|
	\| 485 \| blk.40.attn_norm.weight \| 0x28d13eac0 \| 0x2000 \|
	\| 486 \| blk.40.attn_output.weight \| 0x28d140ac0 \| 0x370000 \|
	\| 487 \| blk.40.attn_q.weight \| 0x28d4b0ac0 \| 0x370000 \|
	\| 488 \| blk.40.attn_q_norm.weight \| 0x28d820ac0 \| 0x200 \|
	\| 489 \| blk.40.attn_v.weight \| 0x28d820cc0 \| 0x6e000 \|
	\| 490 \| blk.40.ffn_down_exps.weight \| 0x28d88ecc0 \| 0x6c00000 \|
	\| 491 \| blk.40.ffn_gate_exps.weight \| 0x29448ecc0 \| 0x5280000 \|
	\| 492 \| blk.40.ffn_gate_inp.weight \| 0x29970ecc0 \| 0x100000 \|
	\| 493 \| blk.40.ffn_norm.weight \| 0x29980ecc0 \| 0x2000 \|
	\| 494 \| blk.40.ffn_up_exps.weight \| 0x299810cc0 \| 0x5280000 \|
	\| 495 \| blk.41.attn_k.weight \| 0x29ea90cc0 \| 0x6e000 \|
	\| 496 \| blk.41.attn_k_norm.weight \| 0x29eafecc0 \| 0x200 \|
	\| 497 \| blk.41.attn_norm.weight \| 0x29eafeec0 \| 0x2000 \|
	\| 498 \| blk.41.attn_output.weight \| 0x29eb00ec0 \| 0x370000 \|
	\| 499 \| blk.41.attn_q.weight \| 0x29ee70ec0 \| 0x370000 \|
	\| 500 \| blk.41.attn_q_norm.weight \| 0x29f1e0ec0 \| 0x200 \|
	\| 501 \| blk.41.attn_v.weight \| 0x29f1e10c0 \| 0x6e000 \|
	\| 502 \| blk.41.ffn_down_exps.weight \| 0x29f24f0c0 \| 0x6c00000 \|
	\| 503 \| blk.41.ffn_gate_exps.weight \| 0x2a5e4f0c0 \| 0x5280000 \|
	\| 504 \| blk.41.ffn_gate_inp.weight \| 0x2ab0cf0c0 \| 0x100000 \|
	\| 505 \| blk.41.ffn_norm.weight \| 0x2ab1cf0c0 \| 0x2000 \|
	\| 506 \| blk.41.ffn_up_exps.weight \| 0x2ab1d10c0 \| 0x5280000 \|
	\| 507 \| blk.42.attn_k.weight \| 0x2b04510c0 \| 0x6e000 \|
	\| 508 \| blk.42.attn_k_norm.weight \| 0x2b04bf0c0 \| 0x200 \|
	\| 509 \| blk.42.attn_norm.weight \| 0x2b04bf2c0 \| 0x2000 \|
	\| 510 \| blk.42.attn_output.weight \| 0x2b04c12c0 \| 0x370000 \|
	\| 511 \| blk.42.attn_q.weight \| 0x2b08312c0 \| 0x370000 \|
	\| 512 \| blk.42.attn_q_norm.weight \| 0x2b0ba12c0 \| 0x200 \|
	\| 513 \| blk.42.attn_v.weight \| 0x2b0ba14c0 \| 0x6e000 \|
	\| 514 \| blk.42.ffn_down_exps.weight \| 0x2b0c0f4c0 \| 0x6c00000 \|
	\| 515 \| blk.42.ffn_gate_exps.weight \| 0x2b780f4c0 \| 0x5280000 \|
	\| 516 \| blk.42.ffn_gate_inp.weight \| 0x2bca8f4c0 \| 0x100000 \|
	\| 517 \| blk.42.ffn_norm.weight \| 0x2bcb8f4c0 \| 0x2000 \|
	\| 518 \| blk.42.ffn_up_exps.weight \| 0x2bcb914c0 \| 0x5280000 \|
	\| 519 \| blk.43.attn_k.weight \| 0x2c1e114c0 \| 0x6e000 \|
	\| 520 \| blk.43.attn_k_norm.weight \| 0x2c1e7f4c0 \| 0x200 \|
	\| 521 \| blk.43.attn_norm.weight \| 0x2c1e7f6c0 \| 0x2000 \|
	\| 522 \| blk.43.attn_output.weight \| 0x2c1e816c0 \| 0x370000 \|
	\| 523 \| blk.43.attn_q.weight \| 0x2c21f16c0 \| 0x370000 \|
	\| 524 \| blk.43.attn_q_norm.weight \| 0x2c25616c0 \| 0x200 \|
	\| 525 \| blk.43.attn_v.weight \| 0x2c25618c0 \| 0x6e000 \|
	\| 526 \| blk.43.ffn_down_exps.weight \| 0x2c25cf8c0 \| 0x6c00000 \|
	\| 527 \| blk.43.ffn_gate_exps.weight \| 0x2c91cf8c0 \| 0x5280000 \|
	\| 528 \| blk.43.ffn_gate_inp.weight \| 0x2ce44f8c0 \| 0x100000 \|
	\| 529 \| blk.43.ffn_norm.weight \| 0x2ce54f8c0 \| 0x2000 \|
	\| 530 \| blk.43.ffn_up_exps.weight \| 0x2ce5518c0 \| 0x5280000 \|
	\| 531 \| blk.44.attn_k.weight \| 0x2d37d18c0 \| 0x6e000 \|
	\| 532 \| blk.44.attn_k_norm.weight \| 0x2d383f8c0 \| 0x200 \|
	\| 533 \| blk.44.attn_norm.weight \| 0x2d383fac0 \| 0x2000 \|
	\| 534 \| blk.44.attn_output.weight \| 0x2d3841ac0 \| 0x370000 \|
	\| 535 \| blk.44.attn_q.weight \| 0x2d3bb1ac0 \| 0x370000 \|
	\| 536 \| blk.44.attn_q_norm.weight \| 0x2d3f21ac0 \| 0x200 \|
	\| 537 \| blk.44.attn_v.weight \| 0x2d3f21cc0 \| 0x6e000 \|
	\| 538 \| blk.44.ffn_down_exps.weight \| 0x2d3f8fcc0 \| 0x6c00000 \|
	\| 539 \| blk.44.ffn_gate_exps.weight \| 0x2dab8fcc0 \| 0x5280000 \|
	\| 540 \| blk.44.ffn_gate_inp.weight \| 0x2dfe0fcc0 \| 0x100000 \|
	\| 541 \| blk.44.ffn_norm.weight \| 0x2dff0fcc0 \| 0x2000 \|
	\| 542 \| blk.44.ffn_up_exps.weight \| 0x2dff11cc0 \| 0x5280000 \|
	\| 543 \| blk.45.attn_k.weight \| 0x2e5191cc0 \| 0x6e000 \|
	\| 544 \| blk.45.attn_k_norm.weight \| 0x2e51ffcc0 \| 0x200 \|
	\| 545 \| blk.45.attn_norm.weight \| 0x2e51ffec0 \| 0x2000 \|
	\| 546 \| blk.45.attn_output.weight \| 0x2e5201ec0 \| 0x370000 \|
	\| 547 \| blk.45.attn_q.weight \| 0x2e5571ec0 \| 0x370000 \|
	\| 548 \| blk.45.attn_q_norm.weight \| 0x2e58e1ec0 \| 0x200 \|
	\| 549 \| blk.45.attn_v.weight \| 0x2e58e20c0 \| 0x6e000 \|
	\| 550 \| blk.45.ffn_down_exps.weight \| 0x2e59500c0 \| 0x6c00000 \|
	\| 551 \| blk.45.ffn_gate_exps.weight \| 0x2ec5500c0 \| 0x5280000 \|
	\| 552 \| blk.45.ffn_gate_inp.weight \| 0x2f17d00c0 \| 0x100000 \|
	\| 553 \| blk.45.ffn_norm.weight \| 0x2f18d00c0 \| 0x2000 \|
	\| 554 \| blk.45.ffn_up_exps.weight \| 0x2f18d20c0 \| 0x5280000 \|
	\| 555 \| blk.46.attn_k.weight \| 0x2f6b520c0 \| 0x6e000 \|
	\| 556 \| blk.46.attn_k_norm.weight \| 0x2f6bc00c0 \| 0x200 \|
	\| 557 \| blk.46.attn_norm.weight \| 0x2f6bc02c0 \| 0x2000 \|
	\| 558 \| blk.46.attn_output.weight \| 0x2f6bc22c0 \| 0x370000 \|
	\| 559 \| blk.46.attn_q.weight \| 0x2f6f322c0 \| 0x370000 \|
	\| 560 \| blk.46.attn_q_norm.weight \| 0x2f72a22c0 \| 0x200 \|
	\| 561 \| blk.46.attn_v.weight \| 0x2f72a24c0 \| 0x6e000 \|
	\| 562 \| blk.46.ffn_down_exps.weight \| 0x2f73104c0 \| 0x6c00000 \|
	\| 563 \| blk.46.ffn_gate_exps.weight \| 0x2fdf104c0 \| 0x5280000 \|
	\| 564 \| blk.46.ffn_gate_inp.weight \| 0x3031904c0 \| 0x100000 \|
	\| 565 \| blk.46.ffn_norm.weight \| 0x3032904c0 \| 0x2000 \|
	\| 566 \| blk.46.ffn_up_exps.weight \| 0x3032924c0 \| 0x5280000 \|
	\| 567 \| blk.47.attn_k.weight \| 0x3085124c0 \| 0x6e000 \|
	\| 568 \| blk.47.attn_k_norm.weight \| 0x3085804c0 \| 0x200 \|
	\| 569 \| blk.47.attn_norm.weight \| 0x3085806c0 \| 0x2000 \|
	\| 570 \| blk.47.attn_output.weight \| 0x3085826c0 \| 0x370000 \|
	\| 571 \| blk.47.attn_q.weight \| 0x3088f26c0 \| 0x370000 \|
	\| 572 \| blk.47.attn_q_norm.weight \| 0x308c626c0 \| 0x200 \|
	\| 573 \| blk.47.attn_v.weight \| 0x308c628c0 \| 0x6e000 \|
	\| 574 \| blk.47.ffn_down_exps.weight \| 0x308cd08c0 \| 0x6c00000 \|
	\| 575 \| blk.47.ffn_gate_exps.weight \| 0x30f8d08c0 \| 0x5280000 \|
	\| 576 \| blk.47.ffn_gate_inp.weight \| 0x314b508c0 \| 0x100000 \|
	\| 577 \| blk.47.ffn_norm.weight \| 0x314c508c0 \| 0x2000 \|
	\| 578 \| blk.47.ffn_up_exps.weight \| 0x314c528c0 \| 0x5280000 \|

	### <a name="base">Base Tensor Group : ~622M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:-------------------\|:---------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 0 \| output.weight \| Output (W) \| (~311M) 311164928 \| 2048 x 151936 x 1 x 1 \| IQ3_XXS \|
	\| 1 \| output_norm.weight \| Output Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 2 \| token_embd.weight \| Token Embedding (W) \| (~311M) 311164928 \| 2048 x 151936 x 1 x 1 \| IQ3_XXS \|

	- Total elements in base: (~622M) 622331904
	- Percentage of total elements: 2.04%


	### <a name="blk_0">Block 0 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 3 \| blk.0.attn_k.weight \| Block 0 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_XXS \|
	\| 4 \| blk.0.attn_k_norm.weight \| Block 0 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 5 \| blk.0.attn_norm.weight \| Block 0 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 6 \| blk.0.attn_output.weight \| Block 0 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 7 \| blk.0.attn_q.weight \| Block 0 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_XXS \|
	\| 8 \| blk.0.attn_q_norm.weight \| Block 0 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 9 \| blk.0.attn_v.weight \| Block 0 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 10 \| blk.0.ffn_down_exps.weight \| Block 0 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ3_S \|
	\| 11 \| blk.0.ffn_gate_exps.weight \| Block 0 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|
	\| 12 \| blk.0.ffn_gate_inp.weight \| Block 0 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 13 \| blk.0.ffn_norm.weight \| Block 0 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 14 \| blk.0.ffn_up_exps.weight \| Block 0 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|

	- Total elements in blk.0: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_1">Block 1 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 15 \| blk.1.attn_k.weight \| Block 1 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_XXS \|
	\| 16 \| blk.1.attn_k_norm.weight \| Block 1 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 17 \| blk.1.attn_norm.weight \| Block 1 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 18 \| blk.1.attn_output.weight \| Block 1 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 19 \| blk.1.attn_q.weight \| Block 1 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_XXS \|
	\| 20 \| blk.1.attn_q_norm.weight \| Block 1 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 21 \| blk.1.attn_v.weight \| Block 1 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 22 \| blk.1.ffn_down_exps.weight \| Block 1 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ3_S \|
	\| 23 \| blk.1.ffn_gate_exps.weight \| Block 1 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|
	\| 24 \| blk.1.ffn_gate_inp.weight \| Block 1 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 25 \| blk.1.ffn_norm.weight \| Block 1 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 26 \| blk.1.ffn_up_exps.weight \| Block 1 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|

	- Total elements in blk.1: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_2">Block 2 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 27 \| blk.2.attn_k.weight \| Block 2 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_XXS \|
	\| 28 \| blk.2.attn_k_norm.weight \| Block 2 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 29 \| blk.2.attn_norm.weight \| Block 2 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 30 \| blk.2.attn_output.weight \| Block 2 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 31 \| blk.2.attn_q.weight \| Block 2 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_XXS \|
	\| 32 \| blk.2.attn_q_norm.weight \| Block 2 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 33 \| blk.2.attn_v.weight \| Block 2 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 34 \| blk.2.ffn_down_exps.weight \| Block 2 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ4_NL \|
	\| 35 \| blk.2.ffn_gate_exps.weight \| Block 2 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|
	\| 36 \| blk.2.ffn_gate_inp.weight \| Block 2 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 37 \| blk.2.ffn_norm.weight \| Block 2 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 38 \| blk.2.ffn_up_exps.weight \| Block 2 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|

	- Total elements in blk.2: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_3">Block 3 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 39 \| blk.3.attn_k.weight \| Block 3 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_XXS \|
	\| 40 \| blk.3.attn_k_norm.weight \| Block 3 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 41 \| blk.3.attn_norm.weight \| Block 3 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 42 \| blk.3.attn_output.weight \| Block 3 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 43 \| blk.3.attn_q.weight \| Block 3 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_XXS \|
	\| 44 \| blk.3.attn_q_norm.weight \| Block 3 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 45 \| blk.3.attn_v.weight \| Block 3 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 46 \| blk.3.ffn_down_exps.weight \| Block 3 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ3_S \|
	\| 47 \| blk.3.ffn_gate_exps.weight \| Block 3 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|
	\| 48 \| blk.3.ffn_gate_inp.weight \| Block 3 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 49 \| blk.3.ffn_norm.weight \| Block 3 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 50 \| blk.3.ffn_up_exps.weight \| Block 3 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|

	- Total elements in blk.3: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_4">Block 4 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 51 \| blk.4.attn_k.weight \| Block 4 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_XXS \|
	\| 52 \| blk.4.attn_k_norm.weight \| Block 4 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 53 \| blk.4.attn_norm.weight \| Block 4 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 54 \| blk.4.attn_output.weight \| Block 4 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 55 \| blk.4.attn_q.weight \| Block 4 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_XXS \|
	\| 56 \| blk.4.attn_q_norm.weight \| Block 4 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 57 \| blk.4.attn_v.weight \| Block 4 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 58 \| blk.4.ffn_down_exps.weight \| Block 4 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ3_S \|
	\| 59 \| blk.4.ffn_gate_exps.weight \| Block 4 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|
	\| 60 \| blk.4.ffn_gate_inp.weight \| Block 4 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 61 \| blk.4.ffn_norm.weight \| Block 4 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 62 \| blk.4.ffn_up_exps.weight \| Block 4 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|

	- Total elements in blk.4: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_5">Block 5 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 63 \| blk.5.attn_k.weight \| Block 5 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_XXS \|
	\| 64 \| blk.5.attn_k_norm.weight \| Block 5 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 65 \| blk.5.attn_norm.weight \| Block 5 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 66 \| blk.5.attn_output.weight \| Block 5 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 67 \| blk.5.attn_q.weight \| Block 5 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_XXS \|
	\| 68 \| blk.5.attn_q_norm.weight \| Block 5 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 69 \| blk.5.attn_v.weight \| Block 5 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 70 \| blk.5.ffn_down_exps.weight \| Block 5 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ3_S \|
	\| 71 \| blk.5.ffn_gate_exps.weight \| Block 5 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|
	\| 72 \| blk.5.ffn_gate_inp.weight \| Block 5 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 73 \| blk.5.ffn_norm.weight \| Block 5 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 74 \| blk.5.ffn_up_exps.weight \| Block 5 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|

	- Total elements in blk.5: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_6">Block 6 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 75 \| blk.6.attn_k.weight \| Block 6 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_XXS \|
	\| 76 \| blk.6.attn_k_norm.weight \| Block 6 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 77 \| blk.6.attn_norm.weight \| Block 6 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 78 \| blk.6.attn_output.weight \| Block 6 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 79 \| blk.6.attn_q.weight \| Block 6 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_XXS \|
	\| 80 \| blk.6.attn_q_norm.weight \| Block 6 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 81 \| blk.6.attn_v.weight \| Block 6 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 82 \| blk.6.ffn_down_exps.weight \| Block 6 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ3_S \|
	\| 83 \| blk.6.ffn_gate_exps.weight \| Block 6 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|
	\| 84 \| blk.6.ffn_gate_inp.weight \| Block 6 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 85 \| blk.6.ffn_norm.weight \| Block 6 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 86 \| blk.6.ffn_up_exps.weight \| Block 6 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|

	- Total elements in blk.6: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_7">Block 7 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 87 \| blk.7.attn_k.weight \| Block 7 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_XXS \|
	\| 88 \| blk.7.attn_k_norm.weight \| Block 7 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 89 \| blk.7.attn_norm.weight \| Block 7 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 90 \| blk.7.attn_output.weight \| Block 7 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 91 \| blk.7.attn_q.weight \| Block 7 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_XXS \|
	\| 92 \| blk.7.attn_q_norm.weight \| Block 7 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 93 \| blk.7.attn_v.weight \| Block 7 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 94 \| blk.7.ffn_down_exps.weight \| Block 7 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ3_S \|
	\| 95 \| blk.7.ffn_gate_exps.weight \| Block 7 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|
	\| 96 \| blk.7.ffn_gate_inp.weight \| Block 7 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 97 \| blk.7.ffn_norm.weight \| Block 7 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 98 \| blk.7.ffn_up_exps.weight \| Block 7 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|

	- Total elements in blk.7: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_8">Block 8 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 99 \| blk.8.attn_k.weight \| Block 8 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_XXS \|
	\| 100 \| blk.8.attn_k_norm.weight \| Block 8 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 101 \| blk.8.attn_norm.weight \| Block 8 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 102 \| blk.8.attn_output.weight \| Block 8 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 103 \| blk.8.attn_q.weight \| Block 8 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_XXS \|
	\| 104 \| blk.8.attn_q_norm.weight \| Block 8 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 105 \| blk.8.attn_v.weight \| Block 8 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 106 \| blk.8.ffn_down_exps.weight \| Block 8 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ3_S \|
	\| 107 \| blk.8.ffn_gate_exps.weight \| Block 8 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|
	\| 108 \| blk.8.ffn_gate_inp.weight \| Block 8 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 109 \| blk.8.ffn_norm.weight \| Block 8 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 110 \| blk.8.ffn_up_exps.weight \| Block 8 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|

	- Total elements in blk.8: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_9">Block 9 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 111 \| blk.9.attn_k.weight \| Block 9 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_XXS \|
	\| 112 \| blk.9.attn_k_norm.weight \| Block 9 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 113 \| blk.9.attn_norm.weight \| Block 9 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 114 \| blk.9.attn_output.weight \| Block 9 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 115 \| blk.9.attn_q.weight \| Block 9 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_XXS \|
	\| 116 \| blk.9.attn_q_norm.weight \| Block 9 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 117 \| blk.9.attn_v.weight \| Block 9 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 118 \| blk.9.ffn_down_exps.weight \| Block 9 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ3_S \|
	\| 119 \| blk.9.ffn_gate_exps.weight \| Block 9 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|
	\| 120 \| blk.9.ffn_gate_inp.weight \| Block 9 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 121 \| blk.9.ffn_norm.weight \| Block 9 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 122 \| blk.9.ffn_up_exps.weight \| Block 9 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|

	- Total elements in blk.9: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_10">Block 10 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 123 \| blk.10.attn_k.weight \| Block 10 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_XXS \|
	\| 124 \| blk.10.attn_k_norm.weight \| Block 10 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 125 \| blk.10.attn_norm.weight \| Block 10 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 126 \| blk.10.attn_output.weight \| Block 10 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 127 \| blk.10.attn_q.weight \| Block 10 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_XXS \|
	\| 128 \| blk.10.attn_q_norm.weight \| Block 10 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 129 \| blk.10.attn_v.weight \| Block 10 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 130 \| blk.10.ffn_down_exps.weight \| Block 10 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ3_S \|
	\| 131 \| blk.10.ffn_gate_exps.weight \| Block 10 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|
	\| 132 \| blk.10.ffn_gate_inp.weight \| Block 10 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 133 \| blk.10.ffn_norm.weight \| Block 10 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 134 \| blk.10.ffn_up_exps.weight \| Block 10 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|

	- Total elements in blk.10: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_11">Block 11 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 135 \| blk.11.attn_k.weight \| Block 11 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_XXS \|
	\| 136 \| blk.11.attn_k_norm.weight \| Block 11 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 137 \| blk.11.attn_norm.weight \| Block 11 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 138 \| blk.11.attn_output.weight \| Block 11 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 139 \| blk.11.attn_q.weight \| Block 11 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_XXS \|
	\| 140 \| blk.11.attn_q_norm.weight \| Block 11 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 141 \| blk.11.attn_v.weight \| Block 11 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 142 \| blk.11.ffn_down_exps.weight \| Block 11 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ3_S \|
	\| 143 \| blk.11.ffn_gate_exps.weight \| Block 11 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|
	\| 144 \| blk.11.ffn_gate_inp.weight \| Block 11 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 145 \| blk.11.ffn_norm.weight \| Block 11 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 146 \| blk.11.ffn_up_exps.weight \| Block 11 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|

	- Total elements in blk.11: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_12">Block 12 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 147 \| blk.12.attn_k.weight \| Block 12 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_XXS \|
	\| 148 \| blk.12.attn_k_norm.weight \| Block 12 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 149 \| blk.12.attn_norm.weight \| Block 12 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 150 \| blk.12.attn_output.weight \| Block 12 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 151 \| blk.12.attn_q.weight \| Block 12 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_XXS \|
	\| 152 \| blk.12.attn_q_norm.weight \| Block 12 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 153 \| blk.12.attn_v.weight \| Block 12 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 154 \| blk.12.ffn_down_exps.weight \| Block 12 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ3_S \|
	\| 155 \| blk.12.ffn_gate_exps.weight \| Block 12 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|
	\| 156 \| blk.12.ffn_gate_inp.weight \| Block 12 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 157 \| blk.12.ffn_norm.weight \| Block 12 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 158 \| blk.12.ffn_up_exps.weight \| Block 12 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|

	- Total elements in blk.12: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_13">Block 13 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 159 \| blk.13.attn_k.weight \| Block 13 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_XXS \|
	\| 160 \| blk.13.attn_k_norm.weight \| Block 13 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 161 \| blk.13.attn_norm.weight \| Block 13 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 162 \| blk.13.attn_output.weight \| Block 13 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 163 \| blk.13.attn_q.weight \| Block 13 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_XXS \|
	\| 164 \| blk.13.attn_q_norm.weight \| Block 13 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 165 \| blk.13.attn_v.weight \| Block 13 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 166 \| blk.13.ffn_down_exps.weight \| Block 13 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ4_NL \|
	\| 167 \| blk.13.ffn_gate_exps.weight \| Block 13 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 168 \| blk.13.ffn_gate_inp.weight \| Block 13 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 169 \| blk.13.ffn_norm.weight \| Block 13 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 170 \| blk.13.ffn_up_exps.weight \| Block 13 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.13: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_14">Block 14 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 171 \| blk.14.attn_k.weight \| Block 14 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_XXS \|
	\| 172 \| blk.14.attn_k_norm.weight \| Block 14 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 173 \| blk.14.attn_norm.weight \| Block 14 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 174 \| blk.14.attn_output.weight \| Block 14 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 175 \| blk.14.attn_q.weight \| Block 14 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_XXS \|
	\| 176 \| blk.14.attn_q_norm.weight \| Block 14 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 177 \| blk.14.attn_v.weight \| Block 14 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 178 \| blk.14.ffn_down_exps.weight \| Block 14 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ3_S \|
	\| 179 \| blk.14.ffn_gate_exps.weight \| Block 14 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|
	\| 180 \| blk.14.ffn_gate_inp.weight \| Block 14 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 181 \| blk.14.ffn_norm.weight \| Block 14 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 182 \| blk.14.ffn_up_exps.weight \| Block 14 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|

	- Total elements in blk.14: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_15">Block 15 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 183 \| blk.15.attn_k.weight \| Block 15 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_XXS \|
	\| 184 \| blk.15.attn_k_norm.weight \| Block 15 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 185 \| blk.15.attn_norm.weight \| Block 15 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 186 \| blk.15.attn_output.weight \| Block 15 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 187 \| blk.15.attn_q.weight \| Block 15 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_XXS \|
	\| 188 \| blk.15.attn_q_norm.weight \| Block 15 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 189 \| blk.15.attn_v.weight \| Block 15 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 190 \| blk.15.ffn_down_exps.weight \| Block 15 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ4_NL \|
	\| 191 \| blk.15.ffn_gate_exps.weight \| Block 15 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 192 \| blk.15.ffn_gate_inp.weight \| Block 15 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 193 \| blk.15.ffn_norm.weight \| Block 15 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 194 \| blk.15.ffn_up_exps.weight \| Block 15 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.15: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_16">Block 16 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 195 \| blk.16.attn_k.weight \| Block 16 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_XXS \|
	\| 196 \| blk.16.attn_k_norm.weight \| Block 16 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 197 \| blk.16.attn_norm.weight \| Block 16 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 198 \| blk.16.attn_output.weight \| Block 16 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 199 \| blk.16.attn_q.weight \| Block 16 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_XXS \|
	\| 200 \| blk.16.attn_q_norm.weight \| Block 16 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 201 \| blk.16.attn_v.weight \| Block 16 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 202 \| blk.16.ffn_down_exps.weight \| Block 16 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ3_S \|
	\| 203 \| blk.16.ffn_gate_exps.weight \| Block 16 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|
	\| 204 \| blk.16.ffn_gate_inp.weight \| Block 16 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 205 \| blk.16.ffn_norm.weight \| Block 16 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 206 \| blk.16.ffn_up_exps.weight \| Block 16 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|

	- Total elements in blk.16: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_17">Block 17 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 207 \| blk.17.attn_k.weight \| Block 17 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_XXS \|
	\| 208 \| blk.17.attn_k_norm.weight \| Block 17 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 209 \| blk.17.attn_norm.weight \| Block 17 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 210 \| blk.17.attn_output.weight \| Block 17 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 211 \| blk.17.attn_q.weight \| Block 17 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_XXS \|
	\| 212 \| blk.17.attn_q_norm.weight \| Block 17 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 213 \| blk.17.attn_v.weight \| Block 17 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 214 \| blk.17.ffn_down_exps.weight \| Block 17 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ3_S \|
	\| 215 \| blk.17.ffn_gate_exps.weight \| Block 17 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|
	\| 216 \| blk.17.ffn_gate_inp.weight \| Block 17 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 217 \| blk.17.ffn_norm.weight \| Block 17 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 218 \| blk.17.ffn_up_exps.weight \| Block 17 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|

	- Total elements in blk.17: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_18">Block 18 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 219 \| blk.18.attn_k.weight \| Block 18 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_XXS \|
	\| 220 \| blk.18.attn_k_norm.weight \| Block 18 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 221 \| blk.18.attn_norm.weight \| Block 18 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 222 \| blk.18.attn_output.weight \| Block 18 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 223 \| blk.18.attn_q.weight \| Block 18 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_XXS \|
	\| 224 \| blk.18.attn_q_norm.weight \| Block 18 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 225 \| blk.18.attn_v.weight \| Block 18 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 226 \| blk.18.ffn_down_exps.weight \| Block 18 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ3_S \|
	\| 227 \| blk.18.ffn_gate_exps.weight \| Block 18 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|
	\| 228 \| blk.18.ffn_gate_inp.weight \| Block 18 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 229 \| blk.18.ffn_norm.weight \| Block 18 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 230 \| blk.18.ffn_up_exps.weight \| Block 18 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|

	- Total elements in blk.18: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_19">Block 19 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 231 \| blk.19.attn_k.weight \| Block 19 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_XXS \|
	\| 232 \| blk.19.attn_k_norm.weight \| Block 19 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 233 \| blk.19.attn_norm.weight \| Block 19 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 234 \| blk.19.attn_output.weight \| Block 19 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 235 \| blk.19.attn_q.weight \| Block 19 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_XXS \|
	\| 236 \| blk.19.attn_q_norm.weight \| Block 19 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 237 \| blk.19.attn_v.weight \| Block 19 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 238 \| blk.19.ffn_down_exps.weight \| Block 19 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ3_S \|
	\| 239 \| blk.19.ffn_gate_exps.weight \| Block 19 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|
	\| 240 \| blk.19.ffn_gate_inp.weight \| Block 19 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 241 \| blk.19.ffn_norm.weight \| Block 19 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 242 \| blk.19.ffn_up_exps.weight \| Block 19 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|

	- Total elements in blk.19: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_20">Block 20 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 243 \| blk.20.attn_k.weight \| Block 20 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_XXS \|
	\| 244 \| blk.20.attn_k_norm.weight \| Block 20 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 245 \| blk.20.attn_norm.weight \| Block 20 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 246 \| blk.20.attn_output.weight \| Block 20 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 247 \| blk.20.attn_q.weight \| Block 20 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_XXS \|
	\| 248 \| blk.20.attn_q_norm.weight \| Block 20 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 249 \| blk.20.attn_v.weight \| Block 20 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 250 \| blk.20.ffn_down_exps.weight \| Block 20 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ3_S \|
	\| 251 \| blk.20.ffn_gate_exps.weight \| Block 20 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|
	\| 252 \| blk.20.ffn_gate_inp.weight \| Block 20 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 253 \| blk.20.ffn_norm.weight \| Block 20 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 254 \| blk.20.ffn_up_exps.weight \| Block 20 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|

	- Total elements in blk.20: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_21">Block 21 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 255 \| blk.21.attn_k.weight \| Block 21 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_XXS \|
	\| 256 \| blk.21.attn_k_norm.weight \| Block 21 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 257 \| blk.21.attn_norm.weight \| Block 21 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 258 \| blk.21.attn_output.weight \| Block 21 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 259 \| blk.21.attn_q.weight \| Block 21 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_XXS \|
	\| 260 \| blk.21.attn_q_norm.weight \| Block 21 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 261 \| blk.21.attn_v.weight \| Block 21 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 262 \| blk.21.ffn_down_exps.weight \| Block 21 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ3_S \|
	\| 263 \| blk.21.ffn_gate_exps.weight \| Block 21 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|
	\| 264 \| blk.21.ffn_gate_inp.weight \| Block 21 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 265 \| blk.21.ffn_norm.weight \| Block 21 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 266 \| blk.21.ffn_up_exps.weight \| Block 21 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|

	- Total elements in blk.21: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_22">Block 22 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 267 \| blk.22.attn_k.weight \| Block 22 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_XXS \|
	\| 268 \| blk.22.attn_k_norm.weight \| Block 22 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 269 \| blk.22.attn_norm.weight \| Block 22 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 270 \| blk.22.attn_output.weight \| Block 22 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 271 \| blk.22.attn_q.weight \| Block 22 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_XXS \|
	\| 272 \| blk.22.attn_q_norm.weight \| Block 22 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 273 \| blk.22.attn_v.weight \| Block 22 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 274 \| blk.22.ffn_down_exps.weight \| Block 22 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ3_S \|
	\| 275 \| blk.22.ffn_gate_exps.weight \| Block 22 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|
	\| 276 \| blk.22.ffn_gate_inp.weight \| Block 22 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 277 \| blk.22.ffn_norm.weight \| Block 22 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 278 \| blk.22.ffn_up_exps.weight \| Block 22 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|

	- Total elements in blk.22: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_23">Block 23 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 279 \| blk.23.attn_k.weight \| Block 23 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_XXS \|
	\| 280 \| blk.23.attn_k_norm.weight \| Block 23 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 281 \| blk.23.attn_norm.weight \| Block 23 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 282 \| blk.23.attn_output.weight \| Block 23 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 283 \| blk.23.attn_q.weight \| Block 23 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_XXS \|
	\| 284 \| blk.23.attn_q_norm.weight \| Block 23 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 285 \| blk.23.attn_v.weight \| Block 23 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 286 \| blk.23.ffn_down_exps.weight \| Block 23 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ3_S \|
	\| 287 \| blk.23.ffn_gate_exps.weight \| Block 23 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|
	\| 288 \| blk.23.ffn_gate_inp.weight \| Block 23 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 289 \| blk.23.ffn_norm.weight \| Block 23 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 290 \| blk.23.ffn_up_exps.weight \| Block 23 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|

	- Total elements in blk.23: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_24">Block 24 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 291 \| blk.24.attn_k.weight \| Block 24 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 292 \| blk.24.attn_k_norm.weight \| Block 24 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 293 \| blk.24.attn_norm.weight \| Block 24 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 294 \| blk.24.attn_output.weight \| Block 24 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 295 \| blk.24.attn_q.weight \| Block 24 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 296 \| blk.24.attn_q_norm.weight \| Block 24 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 297 \| blk.24.attn_v.weight \| Block 24 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 298 \| blk.24.ffn_down_exps.weight \| Block 24 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ3_S \|
	\| 299 \| blk.24.ffn_gate_exps.weight \| Block 24 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|
	\| 300 \| blk.24.ffn_gate_inp.weight \| Block 24 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 301 \| blk.24.ffn_norm.weight \| Block 24 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 302 \| blk.24.ffn_up_exps.weight \| Block 24 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|

	- Total elements in blk.24: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_25">Block 25 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 303 \| blk.25.attn_k.weight \| Block 25 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 304 \| blk.25.attn_k_norm.weight \| Block 25 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 305 \| blk.25.attn_norm.weight \| Block 25 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 306 \| blk.25.attn_output.weight \| Block 25 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 307 \| blk.25.attn_q.weight \| Block 25 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 308 \| blk.25.attn_q_norm.weight \| Block 25 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 309 \| blk.25.attn_v.weight \| Block 25 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 310 \| blk.25.ffn_down_exps.weight \| Block 25 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ4_NL \|
	\| 311 \| blk.25.ffn_gate_exps.weight \| Block 25 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 312 \| blk.25.ffn_gate_inp.weight \| Block 25 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 313 \| blk.25.ffn_norm.weight \| Block 25 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 314 \| blk.25.ffn_up_exps.weight \| Block 25 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.25: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_26">Block 26 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:--------\|
	\| 315 \| blk.26.attn_k.weight \| Block 26 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 316 \| blk.26.attn_k_norm.weight \| Block 26 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 317 \| blk.26.attn_norm.weight \| Block 26 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 318 \| blk.26.attn_output.weight \| Block 26 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 319 \| blk.26.attn_q.weight \| Block 26 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 320 \| blk.26.attn_q_norm.weight \| Block 26 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 321 \| blk.26.attn_v.weight \| Block 26 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 322 \| blk.26.ffn_down_exps.weight \| Block 26 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ3_S \|
	\| 323 \| blk.26.ffn_gate_exps.weight \| Block 26 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|
	\| 324 \| blk.26.ffn_gate_inp.weight \| Block 26 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 325 \| blk.26.ffn_norm.weight \| Block 26 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 326 \| blk.26.ffn_up_exps.weight \| Block 26 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_XXS \|

	- Total elements in blk.26: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_27">Block 27 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:------\|
	\| 327 \| blk.27.attn_k.weight \| Block 27 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 328 \| blk.27.attn_k_norm.weight \| Block 27 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 329 \| blk.27.attn_norm.weight \| Block 27 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 330 \| blk.27.attn_output.weight \| Block 27 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 331 \| blk.27.attn_q.weight \| Block 27 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 332 \| blk.27.attn_q_norm.weight \| Block 27 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 333 \| blk.27.attn_v.weight \| Block 27 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 334 \| blk.27.ffn_down_exps.weight \| Block 27 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ3_S \|
	\| 335 \| blk.27.ffn_gate_exps.weight \| Block 27 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 336 \| blk.27.ffn_gate_inp.weight \| Block 27 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 337 \| blk.27.ffn_norm.weight \| Block 27 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 338 \| blk.27.ffn_up_exps.weight \| Block 27 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.27: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_28">Block 28 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 339 \| blk.28.attn_k.weight \| Block 28 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 340 \| blk.28.attn_k_norm.weight \| Block 28 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 341 \| blk.28.attn_norm.weight \| Block 28 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 342 \| blk.28.attn_output.weight \| Block 28 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 343 \| blk.28.attn_q.weight \| Block 28 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 344 \| blk.28.attn_q_norm.weight \| Block 28 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 345 \| blk.28.attn_v.weight \| Block 28 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 346 \| blk.28.ffn_down_exps.weight \| Block 28 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ4_NL \|
	\| 347 \| blk.28.ffn_gate_exps.weight \| Block 28 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 348 \| blk.28.ffn_gate_inp.weight \| Block 28 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 349 \| blk.28.ffn_norm.weight \| Block 28 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 350 \| blk.28.ffn_up_exps.weight \| Block 28 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.28: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_29">Block 29 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 351 \| blk.29.attn_k.weight \| Block 29 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 352 \| blk.29.attn_k_norm.weight \| Block 29 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 353 \| blk.29.attn_norm.weight \| Block 29 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 354 \| blk.29.attn_output.weight \| Block 29 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 355 \| blk.29.attn_q.weight \| Block 29 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 356 \| blk.29.attn_q_norm.weight \| Block 29 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 357 \| blk.29.attn_v.weight \| Block 29 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 358 \| blk.29.ffn_down_exps.weight \| Block 29 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ4_NL \|
	\| 359 \| blk.29.ffn_gate_exps.weight \| Block 29 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 360 \| blk.29.ffn_gate_inp.weight \| Block 29 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 361 \| blk.29.ffn_norm.weight \| Block 29 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 362 \| blk.29.ffn_up_exps.weight \| Block 29 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.29: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_30">Block 30 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 363 \| blk.30.attn_k.weight \| Block 30 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 364 \| blk.30.attn_k_norm.weight \| Block 30 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 365 \| blk.30.attn_norm.weight \| Block 30 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 366 \| blk.30.attn_output.weight \| Block 30 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 367 \| blk.30.attn_q.weight \| Block 30 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 368 \| blk.30.attn_q_norm.weight \| Block 30 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 369 \| blk.30.attn_v.weight \| Block 30 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 370 \| blk.30.ffn_down_exps.weight \| Block 30 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ4_NL \|
	\| 371 \| blk.30.ffn_gate_exps.weight \| Block 30 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 372 \| blk.30.ffn_gate_inp.weight \| Block 30 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 373 \| blk.30.ffn_norm.weight \| Block 30 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 374 \| blk.30.ffn_up_exps.weight \| Block 30 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.30: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_31">Block 31 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 375 \| blk.31.attn_k.weight \| Block 31 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 376 \| blk.31.attn_k_norm.weight \| Block 31 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 377 \| blk.31.attn_norm.weight \| Block 31 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 378 \| blk.31.attn_output.weight \| Block 31 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 379 \| blk.31.attn_q.weight \| Block 31 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 380 \| blk.31.attn_q_norm.weight \| Block 31 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 381 \| blk.31.attn_v.weight \| Block 31 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 382 \| blk.31.ffn_down_exps.weight \| Block 31 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ4_NL \|
	\| 383 \| blk.31.ffn_gate_exps.weight \| Block 31 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 384 \| blk.31.ffn_gate_inp.weight \| Block 31 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 385 \| blk.31.ffn_norm.weight \| Block 31 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 386 \| blk.31.ffn_up_exps.weight \| Block 31 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.31: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_32">Block 32 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 387 \| blk.32.attn_k.weight \| Block 32 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 388 \| blk.32.attn_k_norm.weight \| Block 32 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 389 \| blk.32.attn_norm.weight \| Block 32 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 390 \| blk.32.attn_output.weight \| Block 32 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 391 \| blk.32.attn_q.weight \| Block 32 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 392 \| blk.32.attn_q_norm.weight \| Block 32 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 393 \| blk.32.attn_v.weight \| Block 32 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 394 \| blk.32.ffn_down_exps.weight \| Block 32 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ4_NL \|
	\| 395 \| blk.32.ffn_gate_exps.weight \| Block 32 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 396 \| blk.32.ffn_gate_inp.weight \| Block 32 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 397 \| blk.32.ffn_norm.weight \| Block 32 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 398 \| blk.32.ffn_up_exps.weight \| Block 32 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.32: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_33">Block 33 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 399 \| blk.33.attn_k.weight \| Block 33 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 400 \| blk.33.attn_k_norm.weight \| Block 33 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 401 \| blk.33.attn_norm.weight \| Block 33 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 402 \| blk.33.attn_output.weight \| Block 33 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 403 \| blk.33.attn_q.weight \| Block 33 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 404 \| blk.33.attn_q_norm.weight \| Block 33 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 405 \| blk.33.attn_v.weight \| Block 33 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 406 \| blk.33.ffn_down_exps.weight \| Block 33 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ4_NL \|
	\| 407 \| blk.33.ffn_gate_exps.weight \| Block 33 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 408 \| blk.33.ffn_gate_inp.weight \| Block 33 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 409 \| blk.33.ffn_norm.weight \| Block 33 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 410 \| blk.33.ffn_up_exps.weight \| Block 33 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.33: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_34">Block 34 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 411 \| blk.34.attn_k.weight \| Block 34 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 412 \| blk.34.attn_k_norm.weight \| Block 34 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 413 \| blk.34.attn_norm.weight \| Block 34 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 414 \| blk.34.attn_output.weight \| Block 34 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 415 \| blk.34.attn_q.weight \| Block 34 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 416 \| blk.34.attn_q_norm.weight \| Block 34 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 417 \| blk.34.attn_v.weight \| Block 34 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 418 \| blk.34.ffn_down_exps.weight \| Block 34 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ4_NL \|
	\| 419 \| blk.34.ffn_gate_exps.weight \| Block 34 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 420 \| blk.34.ffn_gate_inp.weight \| Block 34 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 421 \| blk.34.ffn_norm.weight \| Block 34 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 422 \| blk.34.ffn_up_exps.weight \| Block 34 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.34: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_35">Block 35 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 423 \| blk.35.attn_k.weight \| Block 35 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 424 \| blk.35.attn_k_norm.weight \| Block 35 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 425 \| blk.35.attn_norm.weight \| Block 35 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 426 \| blk.35.attn_output.weight \| Block 35 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 427 \| blk.35.attn_q.weight \| Block 35 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 428 \| blk.35.attn_q_norm.weight \| Block 35 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 429 \| blk.35.attn_v.weight \| Block 35 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 430 \| blk.35.ffn_down_exps.weight \| Block 35 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ4_NL \|
	\| 431 \| blk.35.ffn_gate_exps.weight \| Block 35 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 432 \| blk.35.ffn_gate_inp.weight \| Block 35 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 433 \| blk.35.ffn_norm.weight \| Block 35 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 434 \| blk.35.ffn_up_exps.weight \| Block 35 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.35: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_36">Block 36 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 435 \| blk.36.attn_k.weight \| Block 36 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 436 \| blk.36.attn_k_norm.weight \| Block 36 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 437 \| blk.36.attn_norm.weight \| Block 36 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 438 \| blk.36.attn_output.weight \| Block 36 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 439 \| blk.36.attn_q.weight \| Block 36 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 440 \| blk.36.attn_q_norm.weight \| Block 36 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 441 \| blk.36.attn_v.weight \| Block 36 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 442 \| blk.36.ffn_down_exps.weight \| Block 36 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ4_NL \|
	\| 443 \| blk.36.ffn_gate_exps.weight \| Block 36 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 444 \| blk.36.ffn_gate_inp.weight \| Block 36 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 445 \| blk.36.ffn_norm.weight \| Block 36 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 446 \| blk.36.ffn_up_exps.weight \| Block 36 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.36: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_37">Block 37 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 447 \| blk.37.attn_k.weight \| Block 37 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 448 \| blk.37.attn_k_norm.weight \| Block 37 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 449 \| blk.37.attn_norm.weight \| Block 37 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 450 \| blk.37.attn_output.weight \| Block 37 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 451 \| blk.37.attn_q.weight \| Block 37 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 452 \| blk.37.attn_q_norm.weight \| Block 37 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 453 \| blk.37.attn_v.weight \| Block 37 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 454 \| blk.37.ffn_down_exps.weight \| Block 37 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ4_NL \|
	\| 455 \| blk.37.ffn_gate_exps.weight \| Block 37 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 456 \| blk.37.ffn_gate_inp.weight \| Block 37 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 457 \| blk.37.ffn_norm.weight \| Block 37 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 458 \| blk.37.ffn_up_exps.weight \| Block 37 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.37: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_38">Block 38 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 459 \| blk.38.attn_k.weight \| Block 38 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 460 \| blk.38.attn_k_norm.weight \| Block 38 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 461 \| blk.38.attn_norm.weight \| Block 38 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 462 \| blk.38.attn_output.weight \| Block 38 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 463 \| blk.38.attn_q.weight \| Block 38 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 464 \| blk.38.attn_q_norm.weight \| Block 38 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 465 \| blk.38.attn_v.weight \| Block 38 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 466 \| blk.38.ffn_down_exps.weight \| Block 38 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ4_NL \|
	\| 467 \| blk.38.ffn_gate_exps.weight \| Block 38 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 468 \| blk.38.ffn_gate_inp.weight \| Block 38 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 469 \| blk.38.ffn_norm.weight \| Block 38 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 470 \| blk.38.ffn_up_exps.weight \| Block 38 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.38: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_39">Block 39 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 471 \| blk.39.attn_k.weight \| Block 39 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 472 \| blk.39.attn_k_norm.weight \| Block 39 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 473 \| blk.39.attn_norm.weight \| Block 39 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 474 \| blk.39.attn_output.weight \| Block 39 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 475 \| blk.39.attn_q.weight \| Block 39 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 476 \| blk.39.attn_q_norm.weight \| Block 39 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 477 \| blk.39.attn_v.weight \| Block 39 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 478 \| blk.39.ffn_down_exps.weight \| Block 39 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ4_NL \|
	\| 479 \| blk.39.ffn_gate_exps.weight \| Block 39 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 480 \| blk.39.ffn_gate_inp.weight \| Block 39 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 481 \| blk.39.ffn_norm.weight \| Block 39 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 482 \| blk.39.ffn_up_exps.weight \| Block 39 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.39: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_40">Block 40 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 483 \| blk.40.attn_k.weight \| Block 40 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 484 \| blk.40.attn_k_norm.weight \| Block 40 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 485 \| blk.40.attn_norm.weight \| Block 40 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 486 \| blk.40.attn_output.weight \| Block 40 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 487 \| blk.40.attn_q.weight \| Block 40 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 488 \| blk.40.attn_q_norm.weight \| Block 40 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 489 \| blk.40.attn_v.weight \| Block 40 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 490 \| blk.40.ffn_down_exps.weight \| Block 40 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ4_NL \|
	\| 491 \| blk.40.ffn_gate_exps.weight \| Block 40 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 492 \| blk.40.ffn_gate_inp.weight \| Block 40 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 493 \| blk.40.ffn_norm.weight \| Block 40 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 494 \| blk.40.ffn_up_exps.weight \| Block 40 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.40: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_41">Block 41 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 495 \| blk.41.attn_k.weight \| Block 41 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 496 \| blk.41.attn_k_norm.weight \| Block 41 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 497 \| blk.41.attn_norm.weight \| Block 41 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 498 \| blk.41.attn_output.weight \| Block 41 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 499 \| blk.41.attn_q.weight \| Block 41 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 500 \| blk.41.attn_q_norm.weight \| Block 41 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 501 \| blk.41.attn_v.weight \| Block 41 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 502 \| blk.41.ffn_down_exps.weight \| Block 41 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ4_NL \|
	\| 503 \| blk.41.ffn_gate_exps.weight \| Block 41 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 504 \| blk.41.ffn_gate_inp.weight \| Block 41 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 505 \| blk.41.ffn_norm.weight \| Block 41 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 506 \| blk.41.ffn_up_exps.weight \| Block 41 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.41: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_42">Block 42 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 507 \| blk.42.attn_k.weight \| Block 42 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 508 \| blk.42.attn_k_norm.weight \| Block 42 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 509 \| blk.42.attn_norm.weight \| Block 42 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 510 \| blk.42.attn_output.weight \| Block 42 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 511 \| blk.42.attn_q.weight \| Block 42 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 512 \| blk.42.attn_q_norm.weight \| Block 42 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 513 \| blk.42.attn_v.weight \| Block 42 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 514 \| blk.42.ffn_down_exps.weight \| Block 42 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ4_NL \|
	\| 515 \| blk.42.ffn_gate_exps.weight \| Block 42 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 516 \| blk.42.ffn_gate_inp.weight \| Block 42 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 517 \| blk.42.ffn_norm.weight \| Block 42 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 518 \| blk.42.ffn_up_exps.weight \| Block 42 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.42: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_43">Block 43 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 519 \| blk.43.attn_k.weight \| Block 43 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 520 \| blk.43.attn_k_norm.weight \| Block 43 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 521 \| blk.43.attn_norm.weight \| Block 43 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 522 \| blk.43.attn_output.weight \| Block 43 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 523 \| blk.43.attn_q.weight \| Block 43 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 524 \| blk.43.attn_q_norm.weight \| Block 43 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 525 \| blk.43.attn_v.weight \| Block 43 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 526 \| blk.43.ffn_down_exps.weight \| Block 43 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ4_NL \|
	\| 527 \| blk.43.ffn_gate_exps.weight \| Block 43 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 528 \| blk.43.ffn_gate_inp.weight \| Block 43 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 529 \| blk.43.ffn_norm.weight \| Block 43 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 530 \| blk.43.ffn_up_exps.weight \| Block 43 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.43: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_44">Block 44 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 531 \| blk.44.attn_k.weight \| Block 44 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 532 \| blk.44.attn_k_norm.weight \| Block 44 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 533 \| blk.44.attn_norm.weight \| Block 44 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 534 \| blk.44.attn_output.weight \| Block 44 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 535 \| blk.44.attn_q.weight \| Block 44 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 536 \| blk.44.attn_q_norm.weight \| Block 44 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 537 \| blk.44.attn_v.weight \| Block 44 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 538 \| blk.44.ffn_down_exps.weight \| Block 44 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ4_NL \|
	\| 539 \| blk.44.ffn_gate_exps.weight \| Block 44 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 540 \| blk.44.ffn_gate_inp.weight \| Block 44 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 541 \| blk.44.ffn_norm.weight \| Block 44 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 542 \| blk.44.ffn_up_exps.weight \| Block 44 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.44: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_45">Block 45 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 543 \| blk.45.attn_k.weight \| Block 45 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 544 \| blk.45.attn_k_norm.weight \| Block 45 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 545 \| blk.45.attn_norm.weight \| Block 45 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 546 \| blk.45.attn_output.weight \| Block 45 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 547 \| blk.45.attn_q.weight \| Block 45 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 548 \| blk.45.attn_q_norm.weight \| Block 45 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 549 \| blk.45.attn_v.weight \| Block 45 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 550 \| blk.45.ffn_down_exps.weight \| Block 45 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ4_NL \|
	\| 551 \| blk.45.ffn_gate_exps.weight \| Block 45 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 552 \| blk.45.ffn_gate_inp.weight \| Block 45 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 553 \| blk.45.ffn_norm.weight \| Block 45 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 554 \| blk.45.ffn_up_exps.weight \| Block 45 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.45: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_46">Block 46 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 555 \| blk.46.attn_k.weight \| Block 46 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 556 \| blk.46.attn_k_norm.weight \| Block 46 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 557 \| blk.46.attn_norm.weight \| Block 46 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 558 \| blk.46.attn_output.weight \| Block 46 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 559 \| blk.46.attn_q.weight \| Block 46 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 560 \| blk.46.attn_q_norm.weight \| Block 46 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 561 \| blk.46.attn_v.weight \| Block 46 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 562 \| blk.46.ffn_down_exps.weight \| Block 46 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ4_NL \|
	\| 563 \| blk.46.ffn_gate_exps.weight \| Block 46 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 564 \| blk.46.ffn_gate_inp.weight \| Block 46 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 565 \| blk.46.ffn_norm.weight \| Block 46 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 566 \| blk.46.ffn_up_exps.weight \| Block 46 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.46: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_47">Block 47 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 567 \| blk.47.attn_k.weight \| Block 47 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 568 \| blk.47.attn_k_norm.weight \| Block 47 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 569 \| blk.47.attn_norm.weight \| Block 47 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 570 \| blk.47.attn_output.weight \| Block 47 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ3_S \|
	\| 571 \| blk.47.attn_q.weight \| Block 47 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 572 \| blk.47.attn_q_norm.weight \| Block 47 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 573 \| blk.47.attn_v.weight \| Block 47 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 574 \| blk.47.ffn_down_exps.weight \| Block 47 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| IQ4_NL \|
	\| 575 \| blk.47.ffn_gate_exps.weight \| Block 47 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 576 \| blk.47.ffn_gate_inp.weight \| Block 47 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 577 \| blk.47.ffn_norm.weight \| Block 47 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 578 \| blk.47.ffn_up_exps.weight \| Block 47 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.47: (~623M) 623120640
	- Percentage of total elements: 2.04%