Instructions to use eaddario/Qwen3-30B-A3B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use eaddario/Qwen3-30B-A3B-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="eaddario/Qwen3-30B-A3B-GGUF",
	filename="Qwen3-30B-A3B-F16-00001-of-00003.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use eaddario/Qwen3-30B-A3B-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use eaddario/Qwen3-30B-A3B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "eaddario/Qwen3-30B-A3B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "eaddario/Qwen3-30B-A3B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Ollama
How to use eaddario/Qwen3-30B-A3B-GGUF with Ollama:
```
ollama run hf.co/eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M
```

Unsloth Studio

How to use eaddario/Qwen3-30B-A3B-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for eaddario/Qwen3-30B-A3B-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for eaddario/Qwen3-30B-A3B-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for eaddario/Qwen3-30B-A3B-GGUF to start chatting

How to use eaddario/Qwen3-30B-A3B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use eaddario/Qwen3-30B-A3B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use eaddario/Qwen3-30B-A3B-GGUF with Docker Model Runner:
```
docker model run hf.co/eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M
```

Lemonade

How to use eaddario/Qwen3-30B-A3B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Qwen3-30B-A3B-GGUF-Q4_K_M

List all available models

lemonade list

Qwen3-30B-A3B-GGUF / scores /Qwen3-30B-A3B-IQ4_NL.md

eaddario

Add GGUF internal file structure

a630e18 verified about 1 year ago

preview code

raw

history blame

188 kB

	# Qwen3-30B-A3B-IQ4_NL.gguf - GGUF Internal File Dump

	- Endian: LITTLE endian

	## Key Value Metadata Store

	There are 44 key-value pairs in this file

	\| POS \| TYPE \| Count \| Key \| Value \|
	\|----:\|:---------\|-------:\|:------------------------------------------\|:--------------------------------------------------------------------\|
	\| 1 \| UINT32 \| 1 \| GGUF.version \| 3 \|
	\| 2 \| UINT64 \| 1 \| GGUF.tensor_count \| 579 \|
	\| 3 \| UINT64 \| 1 \| GGUF.kv_count \| 41 \|
	\| 4 \| STRING \| 1 \| general.architecture \| `qwen3moe` \|
	\| 5 \| STRING \| 1 \| general.type \| `model` \|
	\| 6 \| STRING \| 1 \| general.name \| `Qwen3 30B A3B` \|
	\| 7 \| STRING \| 1 \| general.basename \| `Qwen3` \|
	\| 8 \| STRING \| 1 \| general.size_label \| `30B-A3B` \|
	\| 9 \| STRING \| 1 \| general.license \| `apache-2.0` \|
	\| 10 \| STRING \| 1 \| general.license.link \| `https://huggingface.co/Qwen/Qwen3-30B-A3B/blob/main/LICENSE` \|
	\| 11 \| UINT32 \| 1 \| general.base_model.count \| 1 \|
	\| 12 \| STRING \| 1 \| general.base_model.0.name \| `Qwen3 30B A3B Base` \|
	\| 13 \| STRING \| 1 \| general.base_model.0.organization \| `Qwen` \|
	\| 14 \| STRING \| 1 \| general.base_model.0.repo_url \| `https://huggingface.co/Qwen/Qwen3-30B-A3B-Base` \|
	\| 15 \| [STRING] \| 1 \| general.tags \| [ `text-generation` ] \|
	\| 16 \| UINT32 \| 1 \| qwen3moe.block_count \| 48 \|
	\| 17 \| UINT32 \| 1 \| qwen3moe.context_length \| 40960 \|
	\| 18 \| UINT32 \| 1 \| qwen3moe.embedding_length \| 2048 \|
	\| 19 \| UINT32 \| 1 \| qwen3moe.feed_forward_length \| 6144 \|
	\| 20 \| UINT32 \| 1 \| qwen3moe.attention.head_count \| 32 \|
	\| 21 \| UINT32 \| 1 \| qwen3moe.attention.head_count_kv \| 4 \|
	\| 22 \| FLOAT32 \| 1 \| qwen3moe.rope.freq_base \| 1000000.0 \|
	\| 23 \| FLOAT32 \| 1 \| qwen3moe.attention.layer_norm_rms_epsilon \| 1e-06 \|
	\| 24 \| UINT32 \| 1 \| qwen3moe.expert_used_count \| 8 \|
	\| 25 \| UINT32 \| 1 \| qwen3moe.attention.key_length \| 128 \|
	\| 26 \| UINT32 \| 1 \| qwen3moe.attention.value_length \| 128 \|
	\| 27 \| UINT32 \| 1 \| qwen3moe.expert_count \| 128 \|
	\| 28 \| UINT32 \| 1 \| qwen3moe.expert_feed_forward_length \| 768 \|
	\| 29 \| STRING \| 1 \| tokenizer.ggml.model \| `gpt2` \|
	\| 30 \| STRING \| 1 \| tokenizer.ggml.pre \| `qwen2` \|
	\| 31 \| [STRING] \| 151936 \| tokenizer.ggml.tokens \| [ `!`, `"`, `#`, `$`, `%`, ... ] \|
	\| 32 \| [INT32] \| 151936 \| tokenizer.ggml.token_type \| [ 1, 1, 1, 1, 1, 1, 1, ... ] \|
	\| 33 \| [STRING] \| 151387 \| tokenizer.ggml.merges \| [ `Ġ Ġ`, `ĠĠ ĠĠ`, `i n`, `Ġ t`, `ĠĠĠĠ ĠĠĠĠ`, ... ] \|
	\| 34 \| UINT32 \| 1 \| tokenizer.ggml.eos_token_id \| 151645 \|
	\| 35 \| UINT32 \| 1 \| tokenizer.ggml.padding_token_id \| 151643 \|
	\| 36 \| UINT32 \| 1 \| tokenizer.ggml.bos_token_id \| 151643 \|
	\| 37 \| BOOL \| 1 \| tokenizer.ggml.add_bos_token \| False \|
	\| 38 \| STRING \| 1 \| tokenizer.chat_template \| `{%- if tools %}{{- '<\|im_`...`{%- endif %}{%- endif %}` \|
	\| 39 \| UINT32 \| 1 \| general.quantization_version \| 2 \|
	\| 40 \| UINT32 \| 1 \| general.file_type \| 25 \|
	\| 41 \| STRING \| 1 \| quantize.imatrix.file \| `./imatrix/imatrix-Qwen3-30B-A3B-large.dat` \|
	\| 42 \| STRING \| 1 \| quantize.imatrix.dataset \| `../../datasets/imatrix/calibration_all_large.txt` \|
	\| 43 \| INT32 \| 1 \| quantize.imatrix.entries_count \| 382 \|
	\| 44 \| INT32 \| 1 \| quantize.imatrix.chunks_count \| 4978 \|

	## Tensors Overview ~31B Elements

	Total number of elements in all tensors: 30532122624 Elements

	- [Qwen3-30B-A3B-IQ4\_NL.gguf - GGUF Internal File Dump](#qwen3-30b-a3b-iq4_nlgguf---gguf-internal-file-dump)
	- [Key Value Metadata Store](#key-value-metadata-store)
	- [Tensors Overview ~31B Elements](#tensors-overview-31b-elements)
	- [Tensor Data Offset](#tensor-data-offset)
	- [Base Tensor Group : ~622M Elements](#base-tensor-group--622m-elements)
	- [Block 0 Tensor Group : ~623M Elements](#block-0-tensor-group--623m-elements)
	- [Block 1 Tensor Group : ~623M Elements](#block-1-tensor-group--623m-elements)
	- [Block 2 Tensor Group : ~623M Elements](#block-2-tensor-group--623m-elements)
	- [Block 3 Tensor Group : ~623M Elements](#block-3-tensor-group--623m-elements)
	- [Block 4 Tensor Group : ~623M Elements](#block-4-tensor-group--623m-elements)
	- [Block 5 Tensor Group : ~623M Elements](#block-5-tensor-group--623m-elements)
	- [Block 6 Tensor Group : ~623M Elements](#block-6-tensor-group--623m-elements)
	- [Block 7 Tensor Group : ~623M Elements](#block-7-tensor-group--623m-elements)
	- [Block 8 Tensor Group : ~623M Elements](#block-8-tensor-group--623m-elements)
	- [Block 9 Tensor Group : ~623M Elements](#block-9-tensor-group--623m-elements)
	- [Block 10 Tensor Group : ~623M Elements](#block-10-tensor-group--623m-elements)
	- [Block 11 Tensor Group : ~623M Elements](#block-11-tensor-group--623m-elements)
	- [Block 12 Tensor Group : ~623M Elements](#block-12-tensor-group--623m-elements)
	- [Block 13 Tensor Group : ~623M Elements](#block-13-tensor-group--623m-elements)
	- [Block 14 Tensor Group : ~623M Elements](#block-14-tensor-group--623m-elements)
	- [Block 15 Tensor Group : ~623M Elements](#block-15-tensor-group--623m-elements)
	- [Block 16 Tensor Group : ~623M Elements](#block-16-tensor-group--623m-elements)
	- [Block 17 Tensor Group : ~623M Elements](#block-17-tensor-group--623m-elements)
	- [Block 18 Tensor Group : ~623M Elements](#block-18-tensor-group--623m-elements)
	- [Block 19 Tensor Group : ~623M Elements](#block-19-tensor-group--623m-elements)
	- [Block 20 Tensor Group : ~623M Elements](#block-20-tensor-group--623m-elements)
	- [Block 21 Tensor Group : ~623M Elements](#block-21-tensor-group--623m-elements)
	- [Block 22 Tensor Group : ~623M Elements](#block-22-tensor-group--623m-elements)
	- [Block 23 Tensor Group : ~623M Elements](#block-23-tensor-group--623m-elements)
	- [Block 24 Tensor Group : ~623M Elements](#block-24-tensor-group--623m-elements)
	- [Block 25 Tensor Group : ~623M Elements](#block-25-tensor-group--623m-elements)
	- [Block 26 Tensor Group : ~623M Elements](#block-26-tensor-group--623m-elements)
	- [Block 27 Tensor Group : ~623M Elements](#block-27-tensor-group--623m-elements)
	- [Block 28 Tensor Group : ~623M Elements](#block-28-tensor-group--623m-elements)
	- [Block 29 Tensor Group : ~623M Elements](#block-29-tensor-group--623m-elements)
	- [Block 30 Tensor Group : ~623M Elements](#block-30-tensor-group--623m-elements)
	- [Block 31 Tensor Group : ~623M Elements](#block-31-tensor-group--623m-elements)
	- [Block 32 Tensor Group : ~623M Elements](#block-32-tensor-group--623m-elements)
	- [Block 33 Tensor Group : ~623M Elements](#block-33-tensor-group--623m-elements)
	- [Block 34 Tensor Group : ~623M Elements](#block-34-tensor-group--623m-elements)
	- [Block 35 Tensor Group : ~623M Elements](#block-35-tensor-group--623m-elements)
	- [Block 36 Tensor Group : ~623M Elements](#block-36-tensor-group--623m-elements)
	- [Block 37 Tensor Group : ~623M Elements](#block-37-tensor-group--623m-elements)
	- [Block 38 Tensor Group : ~623M Elements](#block-38-tensor-group--623m-elements)
	- [Block 39 Tensor Group : ~623M Elements](#block-39-tensor-group--623m-elements)
	- [Block 40 Tensor Group : ~623M Elements](#block-40-tensor-group--623m-elements)
	- [Block 41 Tensor Group : ~623M Elements](#block-41-tensor-group--623m-elements)
	- [Block 42 Tensor Group : ~623M Elements](#block-42-tensor-group--623m-elements)
	- [Block 43 Tensor Group : ~623M Elements](#block-43-tensor-group--623m-elements)
	- [Block 44 Tensor Group : ~623M Elements](#block-44-tensor-group--623m-elements)
	- [Block 45 Tensor Group : ~623M Elements](#block-45-tensor-group--623m-elements)
	- [Block 46 Tensor Group : ~623M Elements](#block-46-tensor-group--623m-elements)
	- [Block 47 Tensor Group : ~623M Elements](#block-47-tensor-group--623m-elements)

	### Tensor Data Offset

	This table contains the offset and data segment relative to start of file

	\| T_ID \| Tensor Layer Name \| Data Offset (B) \| Data Size (B) \|
	\|-----:\|:----------------------------\|-----------------:\|-----------------:\|
	\| 0 \| output.weight \| 0x5b18c0 \| 0xa6ec000 \|
	\| 1 \| output_norm.weight \| 0xac9d8c0 \| 0x2000 \|
	\| 2 \| token_embd.weight \| 0xac9f8c0 \| 0x7f82800 \|
	\| 3 \| blk.0.attn_k.weight \| 0x12c220c0 \| 0x6e000 \|
	\| 4 \| blk.0.attn_k_norm.weight \| 0x12c900c0 \| 0x200 \|
	\| 5 \| blk.0.attn_norm.weight \| 0x12c902c0 \| 0x2000 \|
	\| 6 \| blk.0.attn_output.weight \| 0x12c922c0 \| 0x480000 \|
	\| 7 \| blk.0.attn_q.weight \| 0x131122c0 \| 0x370000 \|
	\| 8 \| blk.0.attn_q_norm.weight \| 0x134822c0 \| 0x200 \|
	\| 9 \| blk.0.attn_v.weight \| 0x134824c0 \| 0x88000 \|
	\| 10 \| blk.0.ffn_down_exps.weight \| 0x1350a4c0 \| 0x8400000 \|
	\| 11 \| blk.0.ffn_gate_exps.weight \| 0x1b90a4c0 \| 0x5280000 \|
	\| 12 \| blk.0.ffn_gate_inp.weight \| 0x20b8a4c0 \| 0x100000 \|
	\| 13 \| blk.0.ffn_norm.weight \| 0x20c8a4c0 \| 0x2000 \|
	\| 14 \| blk.0.ffn_up_exps.weight \| 0x20c8c4c0 \| 0x5280000 \|
	\| 15 \| blk.1.attn_k.weight \| 0x25f0c4c0 \| 0x6e000 \|
	\| 16 \| blk.1.attn_k_norm.weight \| 0x25f7a4c0 \| 0x200 \|
	\| 17 \| blk.1.attn_norm.weight \| 0x25f7a6c0 \| 0x2000 \|
	\| 18 \| blk.1.attn_output.weight \| 0x25f7c6c0 \| 0x480000 \|
	\| 19 \| blk.1.attn_q.weight \| 0x263fc6c0 \| 0x370000 \|
	\| 20 \| blk.1.attn_q_norm.weight \| 0x2676c6c0 \| 0x200 \|
	\| 21 \| blk.1.attn_v.weight \| 0x2676c8c0 \| 0x88000 \|
	\| 22 \| blk.1.ffn_down_exps.weight \| 0x267f48c0 \| 0x8400000 \|
	\| 23 \| blk.1.ffn_gate_exps.weight \| 0x2ebf48c0 \| 0x5280000 \|
	\| 24 \| blk.1.ffn_gate_inp.weight \| 0x33e748c0 \| 0x100000 \|
	\| 25 \| blk.1.ffn_norm.weight \| 0x33f748c0 \| 0x2000 \|
	\| 26 \| blk.1.ffn_up_exps.weight \| 0x33f768c0 \| 0x5280000 \|
	\| 27 \| blk.2.attn_k.weight \| 0x391f68c0 \| 0x6e000 \|
	\| 28 \| blk.2.attn_k_norm.weight \| 0x392648c0 \| 0x200 \|
	\| 29 \| blk.2.attn_norm.weight \| 0x39264ac0 \| 0x2000 \|
	\| 30 \| blk.2.attn_output.weight \| 0x39266ac0 \| 0x480000 \|
	\| 31 \| blk.2.attn_q.weight \| 0x396e6ac0 \| 0x370000 \|
	\| 32 \| blk.2.attn_q_norm.weight \| 0x39a56ac0 \| 0x200 \|
	\| 33 \| blk.2.attn_v.weight \| 0x39a56cc0 \| 0x88000 \|
	\| 34 \| blk.2.ffn_down_exps.weight \| 0x39adecc0 \| 0x8400000 \|
	\| 35 \| blk.2.ffn_gate_exps.weight \| 0x41edecc0 \| 0x5280000 \|
	\| 36 \| blk.2.ffn_gate_inp.weight \| 0x4715ecc0 \| 0x100000 \|
	\| 37 \| blk.2.ffn_norm.weight \| 0x4725ecc0 \| 0x2000 \|
	\| 38 \| blk.2.ffn_up_exps.weight \| 0x47260cc0 \| 0x5280000 \|
	\| 39 \| blk.3.attn_k.weight \| 0x4c4e0cc0 \| 0x6e000 \|
	\| 40 \| blk.3.attn_k_norm.weight \| 0x4c54ecc0 \| 0x200 \|
	\| 41 \| blk.3.attn_norm.weight \| 0x4c54eec0 \| 0x2000 \|
	\| 42 \| blk.3.attn_output.weight \| 0x4c550ec0 \| 0x480000 \|
	\| 43 \| blk.3.attn_q.weight \| 0x4c9d0ec0 \| 0x370000 \|
	\| 44 \| blk.3.attn_q_norm.weight \| 0x4cd40ec0 \| 0x200 \|
	\| 45 \| blk.3.attn_v.weight \| 0x4cd410c0 \| 0x88000 \|
	\| 46 \| blk.3.ffn_down_exps.weight \| 0x4cdc90c0 \| 0x8400000 \|
	\| 47 \| blk.3.ffn_gate_exps.weight \| 0x551c90c0 \| 0x5280000 \|
	\| 48 \| blk.3.ffn_gate_inp.weight \| 0x5a4490c0 \| 0x100000 \|
	\| 49 \| blk.3.ffn_norm.weight \| 0x5a5490c0 \| 0x2000 \|
	\| 50 \| blk.3.ffn_up_exps.weight \| 0x5a54b0c0 \| 0x5280000 \|
	\| 51 \| blk.4.attn_k.weight \| 0x5f7cb0c0 \| 0x6e000 \|
	\| 52 \| blk.4.attn_k_norm.weight \| 0x5f8390c0 \| 0x200 \|
	\| 53 \| blk.4.attn_norm.weight \| 0x5f8392c0 \| 0x2000 \|
	\| 54 \| blk.4.attn_output.weight \| 0x5f83b2c0 \| 0x480000 \|
	\| 55 \| blk.4.attn_q.weight \| 0x5fcbb2c0 \| 0x370000 \|
	\| 56 \| blk.4.attn_q_norm.weight \| 0x6002b2c0 \| 0x200 \|
	\| 57 \| blk.4.attn_v.weight \| 0x6002b4c0 \| 0x88000 \|
	\| 58 \| blk.4.ffn_down_exps.weight \| 0x600b34c0 \| 0x8400000 \|
	\| 59 \| blk.4.ffn_gate_exps.weight \| 0x684b34c0 \| 0x5280000 \|
	\| 60 \| blk.4.ffn_gate_inp.weight \| 0x6d7334c0 \| 0x100000 \|
	\| 61 \| blk.4.ffn_norm.weight \| 0x6d8334c0 \| 0x2000 \|
	\| 62 \| blk.4.ffn_up_exps.weight \| 0x6d8354c0 \| 0x5280000 \|
	\| 63 \| blk.5.attn_k.weight \| 0x72ab54c0 \| 0x6e000 \|
	\| 64 \| blk.5.attn_k_norm.weight \| 0x72b234c0 \| 0x200 \|
	\| 65 \| blk.5.attn_norm.weight \| 0x72b236c0 \| 0x2000 \|
	\| 66 \| blk.5.attn_output.weight \| 0x72b256c0 \| 0x480000 \|
	\| 67 \| blk.5.attn_q.weight \| 0x72fa56c0 \| 0x370000 \|
	\| 68 \| blk.5.attn_q_norm.weight \| 0x733156c0 \| 0x200 \|
	\| 69 \| blk.5.attn_v.weight \| 0x733158c0 \| 0x88000 \|
	\| 70 \| blk.5.ffn_down_exps.weight \| 0x7339d8c0 \| 0x8400000 \|
	\| 71 \| blk.5.ffn_gate_exps.weight \| 0x7b79d8c0 \| 0x5280000 \|
	\| 72 \| blk.5.ffn_gate_inp.weight \| 0x80a1d8c0 \| 0x100000 \|
	\| 73 \| blk.5.ffn_norm.weight \| 0x80b1d8c0 \| 0x2000 \|
	\| 74 \| blk.5.ffn_up_exps.weight \| 0x80b1f8c0 \| 0x5280000 \|
	\| 75 \| blk.6.attn_k.weight \| 0x85d9f8c0 \| 0x6e000 \|
	\| 76 \| blk.6.attn_k_norm.weight \| 0x85e0d8c0 \| 0x200 \|
	\| 77 \| blk.6.attn_norm.weight \| 0x85e0dac0 \| 0x2000 \|
	\| 78 \| blk.6.attn_output.weight \| 0x85e0fac0 \| 0x480000 \|
	\| 79 \| blk.6.attn_q.weight \| 0x8628fac0 \| 0x370000 \|
	\| 80 \| blk.6.attn_q_norm.weight \| 0x865ffac0 \| 0x200 \|
	\| 81 \| blk.6.attn_v.weight \| 0x865ffcc0 \| 0x88000 \|
	\| 82 \| blk.6.ffn_down_exps.weight \| 0x86687cc0 \| 0x8400000 \|
	\| 83 \| blk.6.ffn_gate_exps.weight \| 0x8ea87cc0 \| 0x5280000 \|
	\| 84 \| blk.6.ffn_gate_inp.weight \| 0x93d07cc0 \| 0x100000 \|
	\| 85 \| blk.6.ffn_norm.weight \| 0x93e07cc0 \| 0x2000 \|
	\| 86 \| blk.6.ffn_up_exps.weight \| 0x93e09cc0 \| 0x5280000 \|
	\| 87 \| blk.7.attn_k.weight \| 0x99089cc0 \| 0x6e000 \|
	\| 88 \| blk.7.attn_k_norm.weight \| 0x990f7cc0 \| 0x200 \|
	\| 89 \| blk.7.attn_norm.weight \| 0x990f7ec0 \| 0x2000 \|
	\| 90 \| blk.7.attn_output.weight \| 0x990f9ec0 \| 0x480000 \|
	\| 91 \| blk.7.attn_q.weight \| 0x99579ec0 \| 0x370000 \|
	\| 92 \| blk.7.attn_q_norm.weight \| 0x998e9ec0 \| 0x200 \|
	\| 93 \| blk.7.attn_v.weight \| 0x998ea0c0 \| 0x88000 \|
	\| 94 \| blk.7.ffn_down_exps.weight \| 0x999720c0 \| 0x8400000 \|
	\| 95 \| blk.7.ffn_gate_exps.weight \| 0xa1d720c0 \| 0x5280000 \|
	\| 96 \| blk.7.ffn_gate_inp.weight \| 0xa6ff20c0 \| 0x100000 \|
	\| 97 \| blk.7.ffn_norm.weight \| 0xa70f20c0 \| 0x2000 \|
	\| 98 \| blk.7.ffn_up_exps.weight \| 0xa70f40c0 \| 0x5280000 \|
	\| 99 \| blk.8.attn_k.weight \| 0xac3740c0 \| 0x6e000 \|
	\| 100 \| blk.8.attn_k_norm.weight \| 0xac3e20c0 \| 0x200 \|
	\| 101 \| blk.8.attn_norm.weight \| 0xac3e22c0 \| 0x2000 \|
	\| 102 \| blk.8.attn_output.weight \| 0xac3e42c0 \| 0x480000 \|
	\| 103 \| blk.8.attn_q.weight \| 0xac8642c0 \| 0x370000 \|
	\| 104 \| blk.8.attn_q_norm.weight \| 0xacbd42c0 \| 0x200 \|
	\| 105 \| blk.8.attn_v.weight \| 0xacbd44c0 \| 0x88000 \|
	\| 106 \| blk.8.ffn_down_exps.weight \| 0xacc5c4c0 \| 0x8400000 \|
	\| 107 \| blk.8.ffn_gate_exps.weight \| 0xb505c4c0 \| 0x5280000 \|
	\| 108 \| blk.8.ffn_gate_inp.weight \| 0xba2dc4c0 \| 0x100000 \|
	\| 109 \| blk.8.ffn_norm.weight \| 0xba3dc4c0 \| 0x2000 \|
	\| 110 \| blk.8.ffn_up_exps.weight \| 0xba3de4c0 \| 0x5280000 \|
	\| 111 \| blk.9.attn_k.weight \| 0xbf65e4c0 \| 0x6e000 \|
	\| 112 \| blk.9.attn_k_norm.weight \| 0xbf6cc4c0 \| 0x200 \|
	\| 113 \| blk.9.attn_norm.weight \| 0xbf6cc6c0 \| 0x2000 \|
	\| 114 \| blk.9.attn_output.weight \| 0xbf6ce6c0 \| 0x480000 \|
	\| 115 \| blk.9.attn_q.weight \| 0xbfb4e6c0 \| 0x370000 \|
	\| 116 \| blk.9.attn_q_norm.weight \| 0xbfebe6c0 \| 0x200 \|
	\| 117 \| blk.9.attn_v.weight \| 0xbfebe8c0 \| 0x88000 \|
	\| 118 \| blk.9.ffn_down_exps.weight \| 0xbff468c0 \| 0x8400000 \|
	\| 119 \| blk.9.ffn_gate_exps.weight \| 0xc83468c0 \| 0x5280000 \|
	\| 120 \| blk.9.ffn_gate_inp.weight \| 0xcd5c68c0 \| 0x100000 \|
	\| 121 \| blk.9.ffn_norm.weight \| 0xcd6c68c0 \| 0x2000 \|
	\| 122 \| blk.9.ffn_up_exps.weight \| 0xcd6c88c0 \| 0x5280000 \|
	\| 123 \| blk.10.attn_k.weight \| 0xd29488c0 \| 0x6e000 \|
	\| 124 \| blk.10.attn_k_norm.weight \| 0xd29b68c0 \| 0x200 \|
	\| 125 \| blk.10.attn_norm.weight \| 0xd29b6ac0 \| 0x2000 \|
	\| 126 \| blk.10.attn_output.weight \| 0xd29b8ac0 \| 0x480000 \|
	\| 127 \| blk.10.attn_q.weight \| 0xd2e38ac0 \| 0x370000 \|
	\| 128 \| blk.10.attn_q_norm.weight \| 0xd31a8ac0 \| 0x200 \|
	\| 129 \| blk.10.attn_v.weight \| 0xd31a8cc0 \| 0x88000 \|
	\| 130 \| blk.10.ffn_down_exps.weight \| 0xd3230cc0 \| 0x8400000 \|
	\| 131 \| blk.10.ffn_gate_exps.weight \| 0xdb630cc0 \| 0x5280000 \|
	\| 132 \| blk.10.ffn_gate_inp.weight \| 0xe08b0cc0 \| 0x100000 \|
	\| 133 \| blk.10.ffn_norm.weight \| 0xe09b0cc0 \| 0x2000 \|
	\| 134 \| blk.10.ffn_up_exps.weight \| 0xe09b2cc0 \| 0x5280000 \|
	\| 135 \| blk.11.attn_k.weight \| 0xe5c32cc0 \| 0x6e000 \|
	\| 136 \| blk.11.attn_k_norm.weight \| 0xe5ca0cc0 \| 0x200 \|
	\| 137 \| blk.11.attn_norm.weight \| 0xe5ca0ec0 \| 0x2000 \|
	\| 138 \| blk.11.attn_output.weight \| 0xe5ca2ec0 \| 0x480000 \|
	\| 139 \| blk.11.attn_q.weight \| 0xe6122ec0 \| 0x370000 \|
	\| 140 \| blk.11.attn_q_norm.weight \| 0xe6492ec0 \| 0x200 \|
	\| 141 \| blk.11.attn_v.weight \| 0xe64930c0 \| 0x88000 \|
	\| 142 \| blk.11.ffn_down_exps.weight \| 0xe651b0c0 \| 0x8400000 \|
	\| 143 \| blk.11.ffn_gate_exps.weight \| 0xee91b0c0 \| 0x5280000 \|
	\| 144 \| blk.11.ffn_gate_inp.weight \| 0xf3b9b0c0 \| 0x100000 \|
	\| 145 \| blk.11.ffn_norm.weight \| 0xf3c9b0c0 \| 0x2000 \|
	\| 146 \| blk.11.ffn_up_exps.weight \| 0xf3c9d0c0 \| 0x5280000 \|
	\| 147 \| blk.12.attn_k.weight \| 0xf8f1d0c0 \| 0x6e000 \|
	\| 148 \| blk.12.attn_k_norm.weight \| 0xf8f8b0c0 \| 0x200 \|
	\| 149 \| blk.12.attn_norm.weight \| 0xf8f8b2c0 \| 0x2000 \|
	\| 150 \| blk.12.attn_output.weight \| 0xf8f8d2c0 \| 0x480000 \|
	\| 151 \| blk.12.attn_q.weight \| 0xf940d2c0 \| 0x370000 \|
	\| 152 \| blk.12.attn_q_norm.weight \| 0xf977d2c0 \| 0x200 \|
	\| 153 \| blk.12.attn_v.weight \| 0xf977d4c0 \| 0x88000 \|
	\| 154 \| blk.12.ffn_down_exps.weight \| 0xf98054c0 \| 0x8400000 \|
	\| 155 \| blk.12.ffn_gate_exps.weight \| 0x101c054c0 \| 0x5280000 \|
	\| 156 \| blk.12.ffn_gate_inp.weight \| 0x106e854c0 \| 0x100000 \|
	\| 157 \| blk.12.ffn_norm.weight \| 0x106f854c0 \| 0x2000 \|
	\| 158 \| blk.12.ffn_up_exps.weight \| 0x106f874c0 \| 0x5280000 \|
	\| 159 \| blk.13.attn_k.weight \| 0x10c2074c0 \| 0x6e000 \|
	\| 160 \| blk.13.attn_k_norm.weight \| 0x10c2754c0 \| 0x200 \|
	\| 161 \| blk.13.attn_norm.weight \| 0x10c2756c0 \| 0x2000 \|
	\| 162 \| blk.13.attn_output.weight \| 0x10c2776c0 \| 0x480000 \|
	\| 163 \| blk.13.attn_q.weight \| 0x10c6f76c0 \| 0x370000 \|
	\| 164 \| blk.13.attn_q_norm.weight \| 0x10ca676c0 \| 0x200 \|
	\| 165 \| blk.13.attn_v.weight \| 0x10ca678c0 \| 0x88000 \|
	\| 166 \| blk.13.ffn_down_exps.weight \| 0x10caef8c0 \| 0x8400000 \|
	\| 167 \| blk.13.ffn_gate_exps.weight \| 0x114eef8c0 \| 0x6c00000 \|
	\| 168 \| blk.13.ffn_gate_inp.weight \| 0x11baef8c0 \| 0x100000 \|
	\| 169 \| blk.13.ffn_norm.weight \| 0x11bbef8c0 \| 0x2000 \|
	\| 170 \| blk.13.ffn_up_exps.weight \| 0x11bbf18c0 \| 0x6c00000 \|
	\| 171 \| blk.14.attn_k.weight \| 0x1227f18c0 \| 0x6e000 \|
	\| 172 \| blk.14.attn_k_norm.weight \| 0x12285f8c0 \| 0x200 \|
	\| 173 \| blk.14.attn_norm.weight \| 0x12285fac0 \| 0x2000 \|
	\| 174 \| blk.14.attn_output.weight \| 0x122861ac0 \| 0x480000 \|
	\| 175 \| blk.14.attn_q.weight \| 0x122ce1ac0 \| 0x370000 \|
	\| 176 \| blk.14.attn_q_norm.weight \| 0x123051ac0 \| 0x200 \|
	\| 177 \| blk.14.attn_v.weight \| 0x123051cc0 \| 0x88000 \|
	\| 178 \| blk.14.ffn_down_exps.weight \| 0x1230d9cc0 \| 0x8400000 \|
	\| 179 \| blk.14.ffn_gate_exps.weight \| 0x12b4d9cc0 \| 0x5280000 \|
	\| 180 \| blk.14.ffn_gate_inp.weight \| 0x130759cc0 \| 0x100000 \|
	\| 181 \| blk.14.ffn_norm.weight \| 0x130859cc0 \| 0x2000 \|
	\| 182 \| blk.14.ffn_up_exps.weight \| 0x13085bcc0 \| 0x5280000 \|
	\| 183 \| blk.15.attn_k.weight \| 0x135adbcc0 \| 0x6e000 \|
	\| 184 \| blk.15.attn_k_norm.weight \| 0x135b49cc0 \| 0x200 \|
	\| 185 \| blk.15.attn_norm.weight \| 0x135b49ec0 \| 0x2000 \|
	\| 186 \| blk.15.attn_output.weight \| 0x135b4bec0 \| 0x480000 \|
	\| 187 \| blk.15.attn_q.weight \| 0x135fcbec0 \| 0x370000 \|
	\| 188 \| blk.15.attn_q_norm.weight \| 0x13633bec0 \| 0x200 \|
	\| 189 \| blk.15.attn_v.weight \| 0x13633c0c0 \| 0x88000 \|
	\| 190 \| blk.15.ffn_down_exps.weight \| 0x1363c40c0 \| 0x8400000 \|
	\| 191 \| blk.15.ffn_gate_exps.weight \| 0x13e7c40c0 \| 0x6c00000 \|
	\| 192 \| blk.15.ffn_gate_inp.weight \| 0x1453c40c0 \| 0x100000 \|
	\| 193 \| blk.15.ffn_norm.weight \| 0x1454c40c0 \| 0x2000 \|
	\| 194 \| blk.15.ffn_up_exps.weight \| 0x1454c60c0 \| 0x6c00000 \|
	\| 195 \| blk.16.attn_k.weight \| 0x14c0c60c0 \| 0x6e000 \|
	\| 196 \| blk.16.attn_k_norm.weight \| 0x14c1340c0 \| 0x200 \|
	\| 197 \| blk.16.attn_norm.weight \| 0x14c1342c0 \| 0x2000 \|
	\| 198 \| blk.16.attn_output.weight \| 0x14c1362c0 \| 0x480000 \|
	\| 199 \| blk.16.attn_q.weight \| 0x14c5b62c0 \| 0x370000 \|
	\| 200 \| blk.16.attn_q_norm.weight \| 0x14c9262c0 \| 0x200 \|
	\| 201 \| blk.16.attn_v.weight \| 0x14c9264c0 \| 0x88000 \|
	\| 202 \| blk.16.ffn_down_exps.weight \| 0x14c9ae4c0 \| 0x8400000 \|
	\| 203 \| blk.16.ffn_gate_exps.weight \| 0x154dae4c0 \| 0x5280000 \|
	\| 204 \| blk.16.ffn_gate_inp.weight \| 0x15a02e4c0 \| 0x100000 \|
	\| 205 \| blk.16.ffn_norm.weight \| 0x15a12e4c0 \| 0x2000 \|
	\| 206 \| blk.16.ffn_up_exps.weight \| 0x15a1304c0 \| 0x5280000 \|
	\| 207 \| blk.17.attn_k.weight \| 0x15f3b04c0 \| 0x6e000 \|
	\| 208 \| blk.17.attn_k_norm.weight \| 0x15f41e4c0 \| 0x200 \|
	\| 209 \| blk.17.attn_norm.weight \| 0x15f41e6c0 \| 0x2000 \|
	\| 210 \| blk.17.attn_output.weight \| 0x15f4206c0 \| 0x480000 \|
	\| 211 \| blk.17.attn_q.weight \| 0x15f8a06c0 \| 0x370000 \|
	\| 212 \| blk.17.attn_q_norm.weight \| 0x15fc106c0 \| 0x200 \|
	\| 213 \| blk.17.attn_v.weight \| 0x15fc108c0 \| 0x88000 \|
	\| 214 \| blk.17.ffn_down_exps.weight \| 0x15fc988c0 \| 0x8400000 \|
	\| 215 \| blk.17.ffn_gate_exps.weight \| 0x1680988c0 \| 0x5280000 \|
	\| 216 \| blk.17.ffn_gate_inp.weight \| 0x16d3188c0 \| 0x100000 \|
	\| 217 \| blk.17.ffn_norm.weight \| 0x16d4188c0 \| 0x2000 \|
	\| 218 \| blk.17.ffn_up_exps.weight \| 0x16d41a8c0 \| 0x5280000 \|
	\| 219 \| blk.18.attn_k.weight \| 0x17269a8c0 \| 0x6e000 \|
	\| 220 \| blk.18.attn_k_norm.weight \| 0x1727088c0 \| 0x200 \|
	\| 221 \| blk.18.attn_norm.weight \| 0x172708ac0 \| 0x2000 \|
	\| 222 \| blk.18.attn_output.weight \| 0x17270aac0 \| 0x480000 \|
	\| 223 \| blk.18.attn_q.weight \| 0x172b8aac0 \| 0x370000 \|
	\| 224 \| blk.18.attn_q_norm.weight \| 0x172efaac0 \| 0x200 \|
	\| 225 \| blk.18.attn_v.weight \| 0x172efacc0 \| 0x88000 \|
	\| 226 \| blk.18.ffn_down_exps.weight \| 0x172f82cc0 \| 0x8400000 \|
	\| 227 \| blk.18.ffn_gate_exps.weight \| 0x17b382cc0 \| 0x5280000 \|
	\| 228 \| blk.18.ffn_gate_inp.weight \| 0x180602cc0 \| 0x100000 \|
	\| 229 \| blk.18.ffn_norm.weight \| 0x180702cc0 \| 0x2000 \|
	\| 230 \| blk.18.ffn_up_exps.weight \| 0x180704cc0 \| 0x5280000 \|
	\| 231 \| blk.19.attn_k.weight \| 0x185984cc0 \| 0x6e000 \|
	\| 232 \| blk.19.attn_k_norm.weight \| 0x1859f2cc0 \| 0x200 \|
	\| 233 \| blk.19.attn_norm.weight \| 0x1859f2ec0 \| 0x2000 \|
	\| 234 \| blk.19.attn_output.weight \| 0x1859f4ec0 \| 0x480000 \|
	\| 235 \| blk.19.attn_q.weight \| 0x185e74ec0 \| 0x370000 \|
	\| 236 \| blk.19.attn_q_norm.weight \| 0x1861e4ec0 \| 0x200 \|
	\| 237 \| blk.19.attn_v.weight \| 0x1861e50c0 \| 0x88000 \|
	\| 238 \| blk.19.ffn_down_exps.weight \| 0x18626d0c0 \| 0x8400000 \|
	\| 239 \| blk.19.ffn_gate_exps.weight \| 0x18e66d0c0 \| 0x5280000 \|
	\| 240 \| blk.19.ffn_gate_inp.weight \| 0x1938ed0c0 \| 0x100000 \|
	\| 241 \| blk.19.ffn_norm.weight \| 0x1939ed0c0 \| 0x2000 \|
	\| 242 \| blk.19.ffn_up_exps.weight \| 0x1939ef0c0 \| 0x5280000 \|
	\| 243 \| blk.20.attn_k.weight \| 0x198c6f0c0 \| 0x6e000 \|
	\| 244 \| blk.20.attn_k_norm.weight \| 0x198cdd0c0 \| 0x200 \|
	\| 245 \| blk.20.attn_norm.weight \| 0x198cdd2c0 \| 0x2000 \|
	\| 246 \| blk.20.attn_output.weight \| 0x198cdf2c0 \| 0x480000 \|
	\| 247 \| blk.20.attn_q.weight \| 0x19915f2c0 \| 0x370000 \|
	\| 248 \| blk.20.attn_q_norm.weight \| 0x1994cf2c0 \| 0x200 \|
	\| 249 \| blk.20.attn_v.weight \| 0x1994cf4c0 \| 0x88000 \|
	\| 250 \| blk.20.ffn_down_exps.weight \| 0x1995574c0 \| 0x8400000 \|
	\| 251 \| blk.20.ffn_gate_exps.weight \| 0x1a19574c0 \| 0x5280000 \|
	\| 252 \| blk.20.ffn_gate_inp.weight \| 0x1a6bd74c0 \| 0x100000 \|
	\| 253 \| blk.20.ffn_norm.weight \| 0x1a6cd74c0 \| 0x2000 \|
	\| 254 \| blk.20.ffn_up_exps.weight \| 0x1a6cd94c0 \| 0x5280000 \|
	\| 255 \| blk.21.attn_k.weight \| 0x1abf594c0 \| 0x6e000 \|
	\| 256 \| blk.21.attn_k_norm.weight \| 0x1abfc74c0 \| 0x200 \|
	\| 257 \| blk.21.attn_norm.weight \| 0x1abfc76c0 \| 0x2000 \|
	\| 258 \| blk.21.attn_output.weight \| 0x1abfc96c0 \| 0x480000 \|
	\| 259 \| blk.21.attn_q.weight \| 0x1ac4496c0 \| 0x370000 \|
	\| 260 \| blk.21.attn_q_norm.weight \| 0x1ac7b96c0 \| 0x200 \|
	\| 261 \| blk.21.attn_v.weight \| 0x1ac7b98c0 \| 0x88000 \|
	\| 262 \| blk.21.ffn_down_exps.weight \| 0x1ac8418c0 \| 0x8400000 \|
	\| 263 \| blk.21.ffn_gate_exps.weight \| 0x1b4c418c0 \| 0x5280000 \|
	\| 264 \| blk.21.ffn_gate_inp.weight \| 0x1b9ec18c0 \| 0x100000 \|
	\| 265 \| blk.21.ffn_norm.weight \| 0x1b9fc18c0 \| 0x2000 \|
	\| 266 \| blk.21.ffn_up_exps.weight \| 0x1b9fc38c0 \| 0x5280000 \|
	\| 267 \| blk.22.attn_k.weight \| 0x1bf2438c0 \| 0x6e000 \|
	\| 268 \| blk.22.attn_k_norm.weight \| 0x1bf2b18c0 \| 0x200 \|
	\| 269 \| blk.22.attn_norm.weight \| 0x1bf2b1ac0 \| 0x2000 \|
	\| 270 \| blk.22.attn_output.weight \| 0x1bf2b3ac0 \| 0x480000 \|
	\| 271 \| blk.22.attn_q.weight \| 0x1bf733ac0 \| 0x370000 \|
	\| 272 \| blk.22.attn_q_norm.weight \| 0x1bfaa3ac0 \| 0x200 \|
	\| 273 \| blk.22.attn_v.weight \| 0x1bfaa3cc0 \| 0x88000 \|
	\| 274 \| blk.22.ffn_down_exps.weight \| 0x1bfb2bcc0 \| 0x8400000 \|
	\| 275 \| blk.22.ffn_gate_exps.weight \| 0x1c7f2bcc0 \| 0x5280000 \|
	\| 276 \| blk.22.ffn_gate_inp.weight \| 0x1cd1abcc0 \| 0x100000 \|
	\| 277 \| blk.22.ffn_norm.weight \| 0x1cd2abcc0 \| 0x2000 \|
	\| 278 \| blk.22.ffn_up_exps.weight \| 0x1cd2adcc0 \| 0x5280000 \|
	\| 279 \| blk.23.attn_k.weight \| 0x1d252dcc0 \| 0x6e000 \|
	\| 280 \| blk.23.attn_k_norm.weight \| 0x1d259bcc0 \| 0x200 \|
	\| 281 \| blk.23.attn_norm.weight \| 0x1d259bec0 \| 0x2000 \|
	\| 282 \| blk.23.attn_output.weight \| 0x1d259dec0 \| 0x480000 \|
	\| 283 \| blk.23.attn_q.weight \| 0x1d2a1dec0 \| 0x370000 \|
	\| 284 \| blk.23.attn_q_norm.weight \| 0x1d2d8dec0 \| 0x200 \|
	\| 285 \| blk.23.attn_v.weight \| 0x1d2d8e0c0 \| 0x88000 \|
	\| 286 \| blk.23.ffn_down_exps.weight \| 0x1d2e160c0 \| 0x8400000 \|
	\| 287 \| blk.23.ffn_gate_exps.weight \| 0x1db2160c0 \| 0x5280000 \|
	\| 288 \| blk.23.ffn_gate_inp.weight \| 0x1e04960c0 \| 0x100000 \|
	\| 289 \| blk.23.ffn_norm.weight \| 0x1e05960c0 \| 0x2000 \|
	\| 290 \| blk.23.ffn_up_exps.weight \| 0x1e05980c0 \| 0x5280000 \|
	\| 291 \| blk.24.attn_k.weight \| 0x1e58180c0 \| 0x90000 \|
	\| 292 \| blk.24.attn_k_norm.weight \| 0x1e58a80c0 \| 0x200 \|
	\| 293 \| blk.24.attn_norm.weight \| 0x1e58a82c0 \| 0x2000 \|
	\| 294 \| blk.24.attn_output.weight \| 0x1e58aa2c0 \| 0x480000 \|
	\| 295 \| blk.24.attn_q.weight \| 0x1e5d2a2c0 \| 0x480000 \|
	\| 296 \| blk.24.attn_q_norm.weight \| 0x1e61aa2c0 \| 0x200 \|
	\| 297 \| blk.24.attn_v.weight \| 0x1e61aa4c0 \| 0x90000 \|
	\| 298 \| blk.24.ffn_down_exps.weight \| 0x1e623a4c0 \| 0x8400000 \|
	\| 299 \| blk.24.ffn_gate_exps.weight \| 0x1ee63a4c0 \| 0x5280000 \|
	\| 300 \| blk.24.ffn_gate_inp.weight \| 0x1f38ba4c0 \| 0x100000 \|
	\| 301 \| blk.24.ffn_norm.weight \| 0x1f39ba4c0 \| 0x2000 \|
	\| 302 \| blk.24.ffn_up_exps.weight \| 0x1f39bc4c0 \| 0x5280000 \|
	\| 303 \| blk.25.attn_k.weight \| 0x1f8c3c4c0 \| 0x90000 \|
	\| 304 \| blk.25.attn_k_norm.weight \| 0x1f8ccc4c0 \| 0x200 \|
	\| 305 \| blk.25.attn_norm.weight \| 0x1f8ccc6c0 \| 0x2000 \|
	\| 306 \| blk.25.attn_output.weight \| 0x1f8cce6c0 \| 0x480000 \|
	\| 307 \| blk.25.attn_q.weight \| 0x1f914e6c0 \| 0x480000 \|
	\| 308 \| blk.25.attn_q_norm.weight \| 0x1f95ce6c0 \| 0x200 \|
	\| 309 \| blk.25.attn_v.weight \| 0x1f95ce8c0 \| 0x90000 \|
	\| 310 \| blk.25.ffn_down_exps.weight \| 0x1f965e8c0 \| 0x8400000 \|
	\| 311 \| blk.25.ffn_gate_exps.weight \| 0x201a5e8c0 \| 0x6c00000 \|
	\| 312 \| blk.25.ffn_gate_inp.weight \| 0x20865e8c0 \| 0x100000 \|
	\| 313 \| blk.25.ffn_norm.weight \| 0x20875e8c0 \| 0x2000 \|
	\| 314 \| blk.25.ffn_up_exps.weight \| 0x2087608c0 \| 0x6c00000 \|
	\| 315 \| blk.26.attn_k.weight \| 0x20f3608c0 \| 0x90000 \|
	\| 316 \| blk.26.attn_k_norm.weight \| 0x20f3f08c0 \| 0x200 \|
	\| 317 \| blk.26.attn_norm.weight \| 0x20f3f0ac0 \| 0x2000 \|
	\| 318 \| blk.26.attn_output.weight \| 0x20f3f2ac0 \| 0x480000 \|
	\| 319 \| blk.26.attn_q.weight \| 0x20f872ac0 \| 0x480000 \|
	\| 320 \| blk.26.attn_q_norm.weight \| 0x20fcf2ac0 \| 0x200 \|
	\| 321 \| blk.26.attn_v.weight \| 0x20fcf2cc0 \| 0x90000 \|
	\| 322 \| blk.26.ffn_down_exps.weight \| 0x20fd82cc0 \| 0x8400000 \|
	\| 323 \| blk.26.ffn_gate_exps.weight \| 0x218182cc0 \| 0x5280000 \|
	\| 324 \| blk.26.ffn_gate_inp.weight \| 0x21d402cc0 \| 0x100000 \|
	\| 325 \| blk.26.ffn_norm.weight \| 0x21d502cc0 \| 0x2000 \|
	\| 326 \| blk.26.ffn_up_exps.weight \| 0x21d504cc0 \| 0x5280000 \|
	\| 327 \| blk.27.attn_k.weight \| 0x222784cc0 \| 0x90000 \|
	\| 328 \| blk.27.attn_k_norm.weight \| 0x222814cc0 \| 0x200 \|
	\| 329 \| blk.27.attn_norm.weight \| 0x222814ec0 \| 0x2000 \|
	\| 330 \| blk.27.attn_output.weight \| 0x222816ec0 \| 0x480000 \|
	\| 331 \| blk.27.attn_q.weight \| 0x222c96ec0 \| 0x480000 \|
	\| 332 \| blk.27.attn_q_norm.weight \| 0x223116ec0 \| 0x200 \|
	\| 333 \| blk.27.attn_v.weight \| 0x2231170c0 \| 0x90000 \|
	\| 334 \| blk.27.ffn_down_exps.weight \| 0x2231a70c0 \| 0x8400000 \|
	\| 335 \| blk.27.ffn_gate_exps.weight \| 0x22b5a70c0 \| 0x6c00000 \|
	\| 336 \| blk.27.ffn_gate_inp.weight \| 0x2321a70c0 \| 0x100000 \|
	\| 337 \| blk.27.ffn_norm.weight \| 0x2322a70c0 \| 0x2000 \|
	\| 338 \| blk.27.ffn_up_exps.weight \| 0x2322a90c0 \| 0x6c00000 \|
	\| 339 \| blk.28.attn_k.weight \| 0x238ea90c0 \| 0x90000 \|
	\| 340 \| blk.28.attn_k_norm.weight \| 0x238f390c0 \| 0x200 \|
	\| 341 \| blk.28.attn_norm.weight \| 0x238f392c0 \| 0x2000 \|
	\| 342 \| blk.28.attn_output.weight \| 0x238f3b2c0 \| 0x480000 \|
	\| 343 \| blk.28.attn_q.weight \| 0x2393bb2c0 \| 0x480000 \|
	\| 344 \| blk.28.attn_q_norm.weight \| 0x23983b2c0 \| 0x200 \|
	\| 345 \| blk.28.attn_v.weight \| 0x23983b4c0 \| 0x90000 \|
	\| 346 \| blk.28.ffn_down_exps.weight \| 0x2398cb4c0 \| 0x8400000 \|
	\| 347 \| blk.28.ffn_gate_exps.weight \| 0x241ccb4c0 \| 0x6c00000 \|
	\| 348 \| blk.28.ffn_gate_inp.weight \| 0x2488cb4c0 \| 0x100000 \|
	\| 349 \| blk.28.ffn_norm.weight \| 0x2489cb4c0 \| 0x2000 \|
	\| 350 \| blk.28.ffn_up_exps.weight \| 0x2489cd4c0 \| 0x6c00000 \|
	\| 351 \| blk.29.attn_k.weight \| 0x24f5cd4c0 \| 0x90000 \|
	\| 352 \| blk.29.attn_k_norm.weight \| 0x24f65d4c0 \| 0x200 \|
	\| 353 \| blk.29.attn_norm.weight \| 0x24f65d6c0 \| 0x2000 \|
	\| 354 \| blk.29.attn_output.weight \| 0x24f65f6c0 \| 0x480000 \|
	\| 355 \| blk.29.attn_q.weight \| 0x24fadf6c0 \| 0x480000 \|
	\| 356 \| blk.29.attn_q_norm.weight \| 0x24ff5f6c0 \| 0x200 \|
	\| 357 \| blk.29.attn_v.weight \| 0x24ff5f8c0 \| 0x90000 \|
	\| 358 \| blk.29.ffn_down_exps.weight \| 0x24ffef8c0 \| 0x8400000 \|
	\| 359 \| blk.29.ffn_gate_exps.weight \| 0x2583ef8c0 \| 0x6c00000 \|
	\| 360 \| blk.29.ffn_gate_inp.weight \| 0x25efef8c0 \| 0x100000 \|
	\| 361 \| blk.29.ffn_norm.weight \| 0x25f0ef8c0 \| 0x2000 \|
	\| 362 \| blk.29.ffn_up_exps.weight \| 0x25f0f18c0 \| 0x6c00000 \|
	\| 363 \| blk.30.attn_k.weight \| 0x265cf18c0 \| 0x90000 \|
	\| 364 \| blk.30.attn_k_norm.weight \| 0x265d818c0 \| 0x200 \|
	\| 365 \| blk.30.attn_norm.weight \| 0x265d81ac0 \| 0x2000 \|
	\| 366 \| blk.30.attn_output.weight \| 0x265d83ac0 \| 0x480000 \|
	\| 367 \| blk.30.attn_q.weight \| 0x266203ac0 \| 0x480000 \|
	\| 368 \| blk.30.attn_q_norm.weight \| 0x266683ac0 \| 0x200 \|
	\| 369 \| blk.30.attn_v.weight \| 0x266683cc0 \| 0x90000 \|
	\| 370 \| blk.30.ffn_down_exps.weight \| 0x266713cc0 \| 0x8400000 \|
	\| 371 \| blk.30.ffn_gate_exps.weight \| 0x26eb13cc0 \| 0x6c00000 \|
	\| 372 \| blk.30.ffn_gate_inp.weight \| 0x275713cc0 \| 0x100000 \|
	\| 373 \| blk.30.ffn_norm.weight \| 0x275813cc0 \| 0x2000 \|
	\| 374 \| blk.30.ffn_up_exps.weight \| 0x275815cc0 \| 0x6c00000 \|
	\| 375 \| blk.31.attn_k.weight \| 0x27c415cc0 \| 0x90000 \|
	\| 376 \| blk.31.attn_k_norm.weight \| 0x27c4a5cc0 \| 0x200 \|
	\| 377 \| blk.31.attn_norm.weight \| 0x27c4a5ec0 \| 0x2000 \|
	\| 378 \| blk.31.attn_output.weight \| 0x27c4a7ec0 \| 0x480000 \|
	\| 379 \| blk.31.attn_q.weight \| 0x27c927ec0 \| 0x480000 \|
	\| 380 \| blk.31.attn_q_norm.weight \| 0x27cda7ec0 \| 0x200 \|
	\| 381 \| blk.31.attn_v.weight \| 0x27cda80c0 \| 0x90000 \|
	\| 382 \| blk.31.ffn_down_exps.weight \| 0x27ce380c0 \| 0x8400000 \|
	\| 383 \| blk.31.ffn_gate_exps.weight \| 0x2852380c0 \| 0x6c00000 \|
	\| 384 \| blk.31.ffn_gate_inp.weight \| 0x28be380c0 \| 0x100000 \|
	\| 385 \| blk.31.ffn_norm.weight \| 0x28bf380c0 \| 0x2000 \|
	\| 386 \| blk.31.ffn_up_exps.weight \| 0x28bf3a0c0 \| 0x6c00000 \|
	\| 387 \| blk.32.attn_k.weight \| 0x292b3a0c0 \| 0x90000 \|
	\| 388 \| blk.32.attn_k_norm.weight \| 0x292bca0c0 \| 0x200 \|
	\| 389 \| blk.32.attn_norm.weight \| 0x292bca2c0 \| 0x2000 \|
	\| 390 \| blk.32.attn_output.weight \| 0x292bcc2c0 \| 0x480000 \|
	\| 391 \| blk.32.attn_q.weight \| 0x29304c2c0 \| 0x480000 \|
	\| 392 \| blk.32.attn_q_norm.weight \| 0x2934cc2c0 \| 0x200 \|
	\| 393 \| blk.32.attn_v.weight \| 0x2934cc4c0 \| 0x90000 \|
	\| 394 \| blk.32.ffn_down_exps.weight \| 0x29355c4c0 \| 0x8400000 \|
	\| 395 \| blk.32.ffn_gate_exps.weight \| 0x29b95c4c0 \| 0x6c00000 \|
	\| 396 \| blk.32.ffn_gate_inp.weight \| 0x2a255c4c0 \| 0x100000 \|
	\| 397 \| blk.32.ffn_norm.weight \| 0x2a265c4c0 \| 0x2000 \|
	\| 398 \| blk.32.ffn_up_exps.weight \| 0x2a265e4c0 \| 0x6c00000 \|
	\| 399 \| blk.33.attn_k.weight \| 0x2a925e4c0 \| 0x90000 \|
	\| 400 \| blk.33.attn_k_norm.weight \| 0x2a92ee4c0 \| 0x200 \|
	\| 401 \| blk.33.attn_norm.weight \| 0x2a92ee6c0 \| 0x2000 \|
	\| 402 \| blk.33.attn_output.weight \| 0x2a92f06c0 \| 0x480000 \|
	\| 403 \| blk.33.attn_q.weight \| 0x2a97706c0 \| 0x480000 \|
	\| 404 \| blk.33.attn_q_norm.weight \| 0x2a9bf06c0 \| 0x200 \|
	\| 405 \| blk.33.attn_v.weight \| 0x2a9bf08c0 \| 0x90000 \|
	\| 406 \| blk.33.ffn_down_exps.weight \| 0x2a9c808c0 \| 0x8400000 \|
	\| 407 \| blk.33.ffn_gate_exps.weight \| 0x2b20808c0 \| 0x6c00000 \|
	\| 408 \| blk.33.ffn_gate_inp.weight \| 0x2b8c808c0 \| 0x100000 \|
	\| 409 \| blk.33.ffn_norm.weight \| 0x2b8d808c0 \| 0x2000 \|
	\| 410 \| blk.33.ffn_up_exps.weight \| 0x2b8d828c0 \| 0x6c00000 \|
	\| 411 \| blk.34.attn_k.weight \| 0x2bf9828c0 \| 0x90000 \|
	\| 412 \| blk.34.attn_k_norm.weight \| 0x2bfa128c0 \| 0x200 \|
	\| 413 \| blk.34.attn_norm.weight \| 0x2bfa12ac0 \| 0x2000 \|
	\| 414 \| blk.34.attn_output.weight \| 0x2bfa14ac0 \| 0x480000 \|
	\| 415 \| blk.34.attn_q.weight \| 0x2bfe94ac0 \| 0x480000 \|
	\| 416 \| blk.34.attn_q_norm.weight \| 0x2c0314ac0 \| 0x200 \|
	\| 417 \| blk.34.attn_v.weight \| 0x2c0314cc0 \| 0x90000 \|
	\| 418 \| blk.34.ffn_down_exps.weight \| 0x2c03a4cc0 \| 0x8400000 \|
	\| 419 \| blk.34.ffn_gate_exps.weight \| 0x2c87a4cc0 \| 0x6c00000 \|
	\| 420 \| blk.34.ffn_gate_inp.weight \| 0x2cf3a4cc0 \| 0x100000 \|
	\| 421 \| blk.34.ffn_norm.weight \| 0x2cf4a4cc0 \| 0x2000 \|
	\| 422 \| blk.34.ffn_up_exps.weight \| 0x2cf4a6cc0 \| 0x6c00000 \|
	\| 423 \| blk.35.attn_k.weight \| 0x2d60a6cc0 \| 0x90000 \|
	\| 424 \| blk.35.attn_k_norm.weight \| 0x2d6136cc0 \| 0x200 \|
	\| 425 \| blk.35.attn_norm.weight \| 0x2d6136ec0 \| 0x2000 \|
	\| 426 \| blk.35.attn_output.weight \| 0x2d6138ec0 \| 0x480000 \|
	\| 427 \| blk.35.attn_q.weight \| 0x2d65b8ec0 \| 0x480000 \|
	\| 428 \| blk.35.attn_q_norm.weight \| 0x2d6a38ec0 \| 0x200 \|
	\| 429 \| blk.35.attn_v.weight \| 0x2d6a390c0 \| 0x90000 \|
	\| 430 \| blk.35.ffn_down_exps.weight \| 0x2d6ac90c0 \| 0x8400000 \|
	\| 431 \| blk.35.ffn_gate_exps.weight \| 0x2deec90c0 \| 0x6c00000 \|
	\| 432 \| blk.35.ffn_gate_inp.weight \| 0x2e5ac90c0 \| 0x100000 \|
	\| 433 \| blk.35.ffn_norm.weight \| 0x2e5bc90c0 \| 0x2000 \|
	\| 434 \| blk.35.ffn_up_exps.weight \| 0x2e5bcb0c0 \| 0x6c00000 \|
	\| 435 \| blk.36.attn_k.weight \| 0x2ec7cb0c0 \| 0x90000 \|
	\| 436 \| blk.36.attn_k_norm.weight \| 0x2ec85b0c0 \| 0x200 \|
	\| 437 \| blk.36.attn_norm.weight \| 0x2ec85b2c0 \| 0x2000 \|
	\| 438 \| blk.36.attn_output.weight \| 0x2ec85d2c0 \| 0x480000 \|
	\| 439 \| blk.36.attn_q.weight \| 0x2eccdd2c0 \| 0x480000 \|
	\| 440 \| blk.36.attn_q_norm.weight \| 0x2ed15d2c0 \| 0x200 \|
	\| 441 \| blk.36.attn_v.weight \| 0x2ed15d4c0 \| 0x90000 \|
	\| 442 \| blk.36.ffn_down_exps.weight \| 0x2ed1ed4c0 \| 0x8400000 \|
	\| 443 \| blk.36.ffn_gate_exps.weight \| 0x2f55ed4c0 \| 0x6c00000 \|
	\| 444 \| blk.36.ffn_gate_inp.weight \| 0x2fc1ed4c0 \| 0x100000 \|
	\| 445 \| blk.36.ffn_norm.weight \| 0x2fc2ed4c0 \| 0x2000 \|
	\| 446 \| blk.36.ffn_up_exps.weight \| 0x2fc2ef4c0 \| 0x6c00000 \|
	\| 447 \| blk.37.attn_k.weight \| 0x302eef4c0 \| 0x90000 \|
	\| 448 \| blk.37.attn_k_norm.weight \| 0x302f7f4c0 \| 0x200 \|
	\| 449 \| blk.37.attn_norm.weight \| 0x302f7f6c0 \| 0x2000 \|
	\| 450 \| blk.37.attn_output.weight \| 0x302f816c0 \| 0x480000 \|
	\| 451 \| blk.37.attn_q.weight \| 0x3034016c0 \| 0x480000 \|
	\| 452 \| blk.37.attn_q_norm.weight \| 0x3038816c0 \| 0x200 \|
	\| 453 \| blk.37.attn_v.weight \| 0x3038818c0 \| 0x90000 \|
	\| 454 \| blk.37.ffn_down_exps.weight \| 0x3039118c0 \| 0x8400000 \|
	\| 455 \| blk.37.ffn_gate_exps.weight \| 0x30bd118c0 \| 0x6c00000 \|
	\| 456 \| blk.37.ffn_gate_inp.weight \| 0x3129118c0 \| 0x100000 \|
	\| 457 \| blk.37.ffn_norm.weight \| 0x312a118c0 \| 0x2000 \|
	\| 458 \| blk.37.ffn_up_exps.weight \| 0x312a138c0 \| 0x6c00000 \|
	\| 459 \| blk.38.attn_k.weight \| 0x3196138c0 \| 0x90000 \|
	\| 460 \| blk.38.attn_k_norm.weight \| 0x3196a38c0 \| 0x200 \|
	\| 461 \| blk.38.attn_norm.weight \| 0x3196a3ac0 \| 0x2000 \|
	\| 462 \| blk.38.attn_output.weight \| 0x3196a5ac0 \| 0x480000 \|
	\| 463 \| blk.38.attn_q.weight \| 0x319b25ac0 \| 0x480000 \|
	\| 464 \| blk.38.attn_q_norm.weight \| 0x319fa5ac0 \| 0x200 \|
	\| 465 \| blk.38.attn_v.weight \| 0x319fa5cc0 \| 0x90000 \|
	\| 466 \| blk.38.ffn_down_exps.weight \| 0x31a035cc0 \| 0x8400000 \|
	\| 467 \| blk.38.ffn_gate_exps.weight \| 0x322435cc0 \| 0x6c00000 \|
	\| 468 \| blk.38.ffn_gate_inp.weight \| 0x329035cc0 \| 0x100000 \|
	\| 469 \| blk.38.ffn_norm.weight \| 0x329135cc0 \| 0x2000 \|
	\| 470 \| blk.38.ffn_up_exps.weight \| 0x329137cc0 \| 0x6c00000 \|
	\| 471 \| blk.39.attn_k.weight \| 0x32fd37cc0 \| 0x90000 \|
	\| 472 \| blk.39.attn_k_norm.weight \| 0x32fdc7cc0 \| 0x200 \|
	\| 473 \| blk.39.attn_norm.weight \| 0x32fdc7ec0 \| 0x2000 \|
	\| 474 \| blk.39.attn_output.weight \| 0x32fdc9ec0 \| 0x480000 \|
	\| 475 \| blk.39.attn_q.weight \| 0x330249ec0 \| 0x480000 \|
	\| 476 \| blk.39.attn_q_norm.weight \| 0x3306c9ec0 \| 0x200 \|
	\| 477 \| blk.39.attn_v.weight \| 0x3306ca0c0 \| 0x90000 \|
	\| 478 \| blk.39.ffn_down_exps.weight \| 0x33075a0c0 \| 0x8400000 \|
	\| 479 \| blk.39.ffn_gate_exps.weight \| 0x338b5a0c0 \| 0x6c00000 \|
	\| 480 \| blk.39.ffn_gate_inp.weight \| 0x33f75a0c0 \| 0x100000 \|
	\| 481 \| blk.39.ffn_norm.weight \| 0x33f85a0c0 \| 0x2000 \|
	\| 482 \| blk.39.ffn_up_exps.weight \| 0x33f85c0c0 \| 0x6c00000 \|
	\| 483 \| blk.40.attn_k.weight \| 0x34645c0c0 \| 0x90000 \|
	\| 484 \| blk.40.attn_k_norm.weight \| 0x3464ec0c0 \| 0x200 \|
	\| 485 \| blk.40.attn_norm.weight \| 0x3464ec2c0 \| 0x2000 \|
	\| 486 \| blk.40.attn_output.weight \| 0x3464ee2c0 \| 0x480000 \|
	\| 487 \| blk.40.attn_q.weight \| 0x34696e2c0 \| 0x480000 \|
	\| 488 \| blk.40.attn_q_norm.weight \| 0x346dee2c0 \| 0x200 \|
	\| 489 \| blk.40.attn_v.weight \| 0x346dee4c0 \| 0x90000 \|
	\| 490 \| blk.40.ffn_down_exps.weight \| 0x346e7e4c0 \| 0x8400000 \|
	\| 491 \| blk.40.ffn_gate_exps.weight \| 0x34f27e4c0 \| 0x6c00000 \|
	\| 492 \| blk.40.ffn_gate_inp.weight \| 0x355e7e4c0 \| 0x100000 \|
	\| 493 \| blk.40.ffn_norm.weight \| 0x355f7e4c0 \| 0x2000 \|
	\| 494 \| blk.40.ffn_up_exps.weight \| 0x355f804c0 \| 0x6c00000 \|
	\| 495 \| blk.41.attn_k.weight \| 0x35cb804c0 \| 0x90000 \|
	\| 496 \| blk.41.attn_k_norm.weight \| 0x35cc104c0 \| 0x200 \|
	\| 497 \| blk.41.attn_norm.weight \| 0x35cc106c0 \| 0x2000 \|
	\| 498 \| blk.41.attn_output.weight \| 0x35cc126c0 \| 0x480000 \|
	\| 499 \| blk.41.attn_q.weight \| 0x35d0926c0 \| 0x480000 \|
	\| 500 \| blk.41.attn_q_norm.weight \| 0x35d5126c0 \| 0x200 \|
	\| 501 \| blk.41.attn_v.weight \| 0x35d5128c0 \| 0x90000 \|
	\| 502 \| blk.41.ffn_down_exps.weight \| 0x35d5a28c0 \| 0x8400000 \|
	\| 503 \| blk.41.ffn_gate_exps.weight \| 0x3659a28c0 \| 0x6c00000 \|
	\| 504 \| blk.41.ffn_gate_inp.weight \| 0x36c5a28c0 \| 0x100000 \|
	\| 505 \| blk.41.ffn_norm.weight \| 0x36c6a28c0 \| 0x2000 \|
	\| 506 \| blk.41.ffn_up_exps.weight \| 0x36c6a48c0 \| 0x6c00000 \|
	\| 507 \| blk.42.attn_k.weight \| 0x3732a48c0 \| 0x90000 \|
	\| 508 \| blk.42.attn_k_norm.weight \| 0x3733348c0 \| 0x200 \|
	\| 509 \| blk.42.attn_norm.weight \| 0x373334ac0 \| 0x2000 \|
	\| 510 \| blk.42.attn_output.weight \| 0x373336ac0 \| 0x480000 \|
	\| 511 \| blk.42.attn_q.weight \| 0x3737b6ac0 \| 0x480000 \|
	\| 512 \| blk.42.attn_q_norm.weight \| 0x373c36ac0 \| 0x200 \|
	\| 513 \| blk.42.attn_v.weight \| 0x373c36cc0 \| 0x90000 \|
	\| 514 \| blk.42.ffn_down_exps.weight \| 0x373cc6cc0 \| 0x8400000 \|
	\| 515 \| blk.42.ffn_gate_exps.weight \| 0x37c0c6cc0 \| 0x6c00000 \|
	\| 516 \| blk.42.ffn_gate_inp.weight \| 0x382cc6cc0 \| 0x100000 \|
	\| 517 \| blk.42.ffn_norm.weight \| 0x382dc6cc0 \| 0x2000 \|
	\| 518 \| blk.42.ffn_up_exps.weight \| 0x382dc8cc0 \| 0x6c00000 \|
	\| 519 \| blk.43.attn_k.weight \| 0x3899c8cc0 \| 0x90000 \|
	\| 520 \| blk.43.attn_k_norm.weight \| 0x389a58cc0 \| 0x200 \|
	\| 521 \| blk.43.attn_norm.weight \| 0x389a58ec0 \| 0x2000 \|
	\| 522 \| blk.43.attn_output.weight \| 0x389a5aec0 \| 0x480000 \|
	\| 523 \| blk.43.attn_q.weight \| 0x389edaec0 \| 0x480000 \|
	\| 524 \| blk.43.attn_q_norm.weight \| 0x38a35aec0 \| 0x200 \|
	\| 525 \| blk.43.attn_v.weight \| 0x38a35b0c0 \| 0x90000 \|
	\| 526 \| blk.43.ffn_down_exps.weight \| 0x38a3eb0c0 \| 0x8400000 \|
	\| 527 \| blk.43.ffn_gate_exps.weight \| 0x3927eb0c0 \| 0x6c00000 \|
	\| 528 \| blk.43.ffn_gate_inp.weight \| 0x3993eb0c0 \| 0x100000 \|
	\| 529 \| blk.43.ffn_norm.weight \| 0x3994eb0c0 \| 0x2000 \|
	\| 530 \| blk.43.ffn_up_exps.weight \| 0x3994ed0c0 \| 0x6c00000 \|
	\| 531 \| blk.44.attn_k.weight \| 0x3a00ed0c0 \| 0x90000 \|
	\| 532 \| blk.44.attn_k_norm.weight \| 0x3a017d0c0 \| 0x200 \|
	\| 533 \| blk.44.attn_norm.weight \| 0x3a017d2c0 \| 0x2000 \|
	\| 534 \| blk.44.attn_output.weight \| 0x3a017f2c0 \| 0x480000 \|
	\| 535 \| blk.44.attn_q.weight \| 0x3a05ff2c0 \| 0x480000 \|
	\| 536 \| blk.44.attn_q_norm.weight \| 0x3a0a7f2c0 \| 0x200 \|
	\| 537 \| blk.44.attn_v.weight \| 0x3a0a7f4c0 \| 0x90000 \|
	\| 538 \| blk.44.ffn_down_exps.weight \| 0x3a0b0f4c0 \| 0x8400000 \|
	\| 539 \| blk.44.ffn_gate_exps.weight \| 0x3a8f0f4c0 \| 0x6c00000 \|
	\| 540 \| blk.44.ffn_gate_inp.weight \| 0x3afb0f4c0 \| 0x100000 \|
	\| 541 \| blk.44.ffn_norm.weight \| 0x3afc0f4c0 \| 0x2000 \|
	\| 542 \| blk.44.ffn_up_exps.weight \| 0x3afc114c0 \| 0x6c00000 \|
	\| 543 \| blk.45.attn_k.weight \| 0x3b68114c0 \| 0x90000 \|
	\| 544 \| blk.45.attn_k_norm.weight \| 0x3b68a14c0 \| 0x200 \|
	\| 545 \| blk.45.attn_norm.weight \| 0x3b68a16c0 \| 0x2000 \|
	\| 546 \| blk.45.attn_output.weight \| 0x3b68a36c0 \| 0x480000 \|
	\| 547 \| blk.45.attn_q.weight \| 0x3b6d236c0 \| 0x480000 \|
	\| 548 \| blk.45.attn_q_norm.weight \| 0x3b71a36c0 \| 0x200 \|
	\| 549 \| blk.45.attn_v.weight \| 0x3b71a38c0 \| 0x90000 \|
	\| 550 \| blk.45.ffn_down_exps.weight \| 0x3b72338c0 \| 0x8400000 \|
	\| 551 \| blk.45.ffn_gate_exps.weight \| 0x3bf6338c0 \| 0x6c00000 \|
	\| 552 \| blk.45.ffn_gate_inp.weight \| 0x3c62338c0 \| 0x100000 \|
	\| 553 \| blk.45.ffn_norm.weight \| 0x3c63338c0 \| 0x2000 \|
	\| 554 \| blk.45.ffn_up_exps.weight \| 0x3c63358c0 \| 0x6c00000 \|
	\| 555 \| blk.46.attn_k.weight \| 0x3ccf358c0 \| 0x90000 \|
	\| 556 \| blk.46.attn_k_norm.weight \| 0x3ccfc58c0 \| 0x200 \|
	\| 557 \| blk.46.attn_norm.weight \| 0x3ccfc5ac0 \| 0x2000 \|
	\| 558 \| blk.46.attn_output.weight \| 0x3ccfc7ac0 \| 0x480000 \|
	\| 559 \| blk.46.attn_q.weight \| 0x3cd447ac0 \| 0x480000 \|
	\| 560 \| blk.46.attn_q_norm.weight \| 0x3cd8c7ac0 \| 0x200 \|
	\| 561 \| blk.46.attn_v.weight \| 0x3cd8c7cc0 \| 0x90000 \|
	\| 562 \| blk.46.ffn_down_exps.weight \| 0x3cd957cc0 \| 0x8400000 \|
	\| 563 \| blk.46.ffn_gate_exps.weight \| 0x3d5d57cc0 \| 0x6c00000 \|
	\| 564 \| blk.46.ffn_gate_inp.weight \| 0x3dc957cc0 \| 0x100000 \|
	\| 565 \| blk.46.ffn_norm.weight \| 0x3dca57cc0 \| 0x2000 \|
	\| 566 \| blk.46.ffn_up_exps.weight \| 0x3dca59cc0 \| 0x6c00000 \|
	\| 567 \| blk.47.attn_k.weight \| 0x3e3659cc0 \| 0x90000 \|
	\| 568 \| blk.47.attn_k_norm.weight \| 0x3e36e9cc0 \| 0x200 \|
	\| 569 \| blk.47.attn_norm.weight \| 0x3e36e9ec0 \| 0x2000 \|
	\| 570 \| blk.47.attn_output.weight \| 0x3e36ebec0 \| 0x480000 \|
	\| 571 \| blk.47.attn_q.weight \| 0x3e3b6bec0 \| 0x480000 \|
	\| 572 \| blk.47.attn_q_norm.weight \| 0x3e3febec0 \| 0x200 \|
	\| 573 \| blk.47.attn_v.weight \| 0x3e3fec0c0 \| 0x90000 \|
	\| 574 \| blk.47.ffn_down_exps.weight \| 0x3e407c0c0 \| 0x8400000 \|
	\| 575 \| blk.47.ffn_gate_exps.weight \| 0x3ec47c0c0 \| 0x6c00000 \|
	\| 576 \| blk.47.ffn_gate_inp.weight \| 0x3f307c0c0 \| 0x100000 \|
	\| 577 \| blk.47.ffn_norm.weight \| 0x3f317c0c0 \| 0x2000 \|
	\| 578 \| blk.47.ffn_up_exps.weight \| 0x3f317e0c0 \| 0x6c00000 \|

	### <a name="base">Base Tensor Group : ~622M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:-------------------\|:---------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 0 \| output.weight \| Output (W) \| (~311M) 311164928 \| 2048 x 151936 x 1 x 1 \| IQ4_NL \|
	\| 1 \| output_norm.weight \| Output Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 2 \| token_embd.weight \| Token Embedding (W) \| (~311M) 311164928 \| 2048 x 151936 x 1 x 1 \| IQ3_S \|

	- Total elements in base: (~622M) 622331904
	- Percentage of total elements: 2.04%


	### <a name="blk_0">Block 0 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 3 \| blk.0.attn_k.weight \| Block 0 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 4 \| blk.0.attn_k_norm.weight \| Block 0 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 5 \| blk.0.attn_norm.weight \| Block 0 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 6 \| blk.0.attn_output.weight \| Block 0 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 7 \| blk.0.attn_q.weight \| Block 0 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 8 \| blk.0.attn_q_norm.weight \| Block 0 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 9 \| blk.0.attn_v.weight \| Block 0 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_XS \|
	\| 10 \| blk.0.ffn_down_exps.weight \| Block 0 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 11 \| blk.0.ffn_gate_exps.weight \| Block 0 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 12 \| blk.0.ffn_gate_inp.weight \| Block 0 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 13 \| blk.0.ffn_norm.weight \| Block 0 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 14 \| blk.0.ffn_up_exps.weight \| Block 0 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.0: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_1">Block 1 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 15 \| blk.1.attn_k.weight \| Block 1 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 16 \| blk.1.attn_k_norm.weight \| Block 1 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 17 \| blk.1.attn_norm.weight \| Block 1 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 18 \| blk.1.attn_output.weight \| Block 1 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 19 \| blk.1.attn_q.weight \| Block 1 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 20 \| blk.1.attn_q_norm.weight \| Block 1 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 21 \| blk.1.attn_v.weight \| Block 1 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_XS \|
	\| 22 \| blk.1.ffn_down_exps.weight \| Block 1 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 23 \| blk.1.ffn_gate_exps.weight \| Block 1 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 24 \| blk.1.ffn_gate_inp.weight \| Block 1 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 25 \| blk.1.ffn_norm.weight \| Block 1 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 26 \| blk.1.ffn_up_exps.weight \| Block 1 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.1: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_2">Block 2 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 27 \| blk.2.attn_k.weight \| Block 2 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 28 \| blk.2.attn_k_norm.weight \| Block 2 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 29 \| blk.2.attn_norm.weight \| Block 2 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 30 \| blk.2.attn_output.weight \| Block 2 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 31 \| blk.2.attn_q.weight \| Block 2 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 32 \| blk.2.attn_q_norm.weight \| Block 2 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 33 \| blk.2.attn_v.weight \| Block 2 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_XS \|
	\| 34 \| blk.2.ffn_down_exps.weight \| Block 2 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 35 \| blk.2.ffn_gate_exps.weight \| Block 2 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 36 \| blk.2.ffn_gate_inp.weight \| Block 2 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 37 \| blk.2.ffn_norm.weight \| Block 2 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 38 \| blk.2.ffn_up_exps.weight \| Block 2 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.2: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_3">Block 3 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 39 \| blk.3.attn_k.weight \| Block 3 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 40 \| blk.3.attn_k_norm.weight \| Block 3 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 41 \| blk.3.attn_norm.weight \| Block 3 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 42 \| blk.3.attn_output.weight \| Block 3 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 43 \| blk.3.attn_q.weight \| Block 3 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 44 \| blk.3.attn_q_norm.weight \| Block 3 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 45 \| blk.3.attn_v.weight \| Block 3 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_XS \|
	\| 46 \| blk.3.ffn_down_exps.weight \| Block 3 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 47 \| blk.3.ffn_gate_exps.weight \| Block 3 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 48 \| blk.3.ffn_gate_inp.weight \| Block 3 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 49 \| blk.3.ffn_norm.weight \| Block 3 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 50 \| blk.3.ffn_up_exps.weight \| Block 3 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.3: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_4">Block 4 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 51 \| blk.4.attn_k.weight \| Block 4 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 52 \| blk.4.attn_k_norm.weight \| Block 4 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 53 \| blk.4.attn_norm.weight \| Block 4 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 54 \| blk.4.attn_output.weight \| Block 4 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 55 \| blk.4.attn_q.weight \| Block 4 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 56 \| blk.4.attn_q_norm.weight \| Block 4 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 57 \| blk.4.attn_v.weight \| Block 4 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_XS \|
	\| 58 \| blk.4.ffn_down_exps.weight \| Block 4 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 59 \| blk.4.ffn_gate_exps.weight \| Block 4 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 60 \| blk.4.ffn_gate_inp.weight \| Block 4 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 61 \| blk.4.ffn_norm.weight \| Block 4 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 62 \| blk.4.ffn_up_exps.weight \| Block 4 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.4: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_5">Block 5 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 63 \| blk.5.attn_k.weight \| Block 5 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 64 \| blk.5.attn_k_norm.weight \| Block 5 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 65 \| blk.5.attn_norm.weight \| Block 5 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 66 \| blk.5.attn_output.weight \| Block 5 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 67 \| blk.5.attn_q.weight \| Block 5 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 68 \| blk.5.attn_q_norm.weight \| Block 5 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 69 \| blk.5.attn_v.weight \| Block 5 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_XS \|
	\| 70 \| blk.5.ffn_down_exps.weight \| Block 5 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 71 \| blk.5.ffn_gate_exps.weight \| Block 5 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 72 \| blk.5.ffn_gate_inp.weight \| Block 5 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 73 \| blk.5.ffn_norm.weight \| Block 5 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 74 \| blk.5.ffn_up_exps.weight \| Block 5 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.5: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_6">Block 6 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 75 \| blk.6.attn_k.weight \| Block 6 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 76 \| blk.6.attn_k_norm.weight \| Block 6 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 77 \| blk.6.attn_norm.weight \| Block 6 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 78 \| blk.6.attn_output.weight \| Block 6 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 79 \| blk.6.attn_q.weight \| Block 6 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 80 \| blk.6.attn_q_norm.weight \| Block 6 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 81 \| blk.6.attn_v.weight \| Block 6 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_XS \|
	\| 82 \| blk.6.ffn_down_exps.weight \| Block 6 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 83 \| blk.6.ffn_gate_exps.weight \| Block 6 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 84 \| blk.6.ffn_gate_inp.weight \| Block 6 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 85 \| blk.6.ffn_norm.weight \| Block 6 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 86 \| blk.6.ffn_up_exps.weight \| Block 6 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.6: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_7">Block 7 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 87 \| blk.7.attn_k.weight \| Block 7 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 88 \| blk.7.attn_k_norm.weight \| Block 7 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 89 \| blk.7.attn_norm.weight \| Block 7 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 90 \| blk.7.attn_output.weight \| Block 7 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 91 \| blk.7.attn_q.weight \| Block 7 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 92 \| blk.7.attn_q_norm.weight \| Block 7 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 93 \| blk.7.attn_v.weight \| Block 7 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_XS \|
	\| 94 \| blk.7.ffn_down_exps.weight \| Block 7 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 95 \| blk.7.ffn_gate_exps.weight \| Block 7 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 96 \| blk.7.ffn_gate_inp.weight \| Block 7 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 97 \| blk.7.ffn_norm.weight \| Block 7 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 98 \| blk.7.ffn_up_exps.weight \| Block 7 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.7: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_8">Block 8 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 99 \| blk.8.attn_k.weight \| Block 8 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 100 \| blk.8.attn_k_norm.weight \| Block 8 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 101 \| blk.8.attn_norm.weight \| Block 8 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 102 \| blk.8.attn_output.weight \| Block 8 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 103 \| blk.8.attn_q.weight \| Block 8 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 104 \| blk.8.attn_q_norm.weight \| Block 8 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 105 \| blk.8.attn_v.weight \| Block 8 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_XS \|
	\| 106 \| blk.8.ffn_down_exps.weight \| Block 8 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 107 \| blk.8.ffn_gate_exps.weight \| Block 8 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 108 \| blk.8.ffn_gate_inp.weight \| Block 8 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 109 \| blk.8.ffn_norm.weight \| Block 8 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 110 \| blk.8.ffn_up_exps.weight \| Block 8 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.8: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_9">Block 9 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:---------------------------\|:------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 111 \| blk.9.attn_k.weight \| Block 9 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 112 \| blk.9.attn_k_norm.weight \| Block 9 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 113 \| blk.9.attn_norm.weight \| Block 9 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 114 \| blk.9.attn_output.weight \| Block 9 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 115 \| blk.9.attn_q.weight \| Block 9 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 116 \| blk.9.attn_q_norm.weight \| Block 9 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 117 \| blk.9.attn_v.weight \| Block 9 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_XS \|
	\| 118 \| blk.9.ffn_down_exps.weight \| Block 9 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 119 \| blk.9.ffn_gate_exps.weight \| Block 9 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 120 \| blk.9.ffn_gate_inp.weight \| Block 9 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 121 \| blk.9.ffn_norm.weight \| Block 9 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 122 \| blk.9.ffn_up_exps.weight \| Block 9 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.9: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_10">Block 10 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 123 \| blk.10.attn_k.weight \| Block 10 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 124 \| blk.10.attn_k_norm.weight \| Block 10 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 125 \| blk.10.attn_norm.weight \| Block 10 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 126 \| blk.10.attn_output.weight \| Block 10 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 127 \| blk.10.attn_q.weight \| Block 10 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 128 \| blk.10.attn_q_norm.weight \| Block 10 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 129 \| blk.10.attn_v.weight \| Block 10 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_XS \|
	\| 130 \| blk.10.ffn_down_exps.weight \| Block 10 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 131 \| blk.10.ffn_gate_exps.weight \| Block 10 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 132 \| blk.10.ffn_gate_inp.weight \| Block 10 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 133 \| blk.10.ffn_norm.weight \| Block 10 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 134 \| blk.10.ffn_up_exps.weight \| Block 10 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.10: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_11">Block 11 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 135 \| blk.11.attn_k.weight \| Block 11 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 136 \| blk.11.attn_k_norm.weight \| Block 11 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 137 \| blk.11.attn_norm.weight \| Block 11 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 138 \| blk.11.attn_output.weight \| Block 11 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 139 \| blk.11.attn_q.weight \| Block 11 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 140 \| blk.11.attn_q_norm.weight \| Block 11 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 141 \| blk.11.attn_v.weight \| Block 11 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_XS \|
	\| 142 \| blk.11.ffn_down_exps.weight \| Block 11 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 143 \| blk.11.ffn_gate_exps.weight \| Block 11 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 144 \| blk.11.ffn_gate_inp.weight \| Block 11 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 145 \| blk.11.ffn_norm.weight \| Block 11 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 146 \| blk.11.ffn_up_exps.weight \| Block 11 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.11: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_12">Block 12 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 147 \| blk.12.attn_k.weight \| Block 12 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 148 \| blk.12.attn_k_norm.weight \| Block 12 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 149 \| blk.12.attn_norm.weight \| Block 12 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 150 \| blk.12.attn_output.weight \| Block 12 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 151 \| blk.12.attn_q.weight \| Block 12 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 152 \| blk.12.attn_q_norm.weight \| Block 12 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 153 \| blk.12.attn_v.weight \| Block 12 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_XS \|
	\| 154 \| blk.12.ffn_down_exps.weight \| Block 12 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 155 \| blk.12.ffn_gate_exps.weight \| Block 12 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 156 \| blk.12.ffn_gate_inp.weight \| Block 12 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 157 \| blk.12.ffn_norm.weight \| Block 12 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 158 \| blk.12.ffn_up_exps.weight \| Block 12 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.12: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_13">Block 13 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 159 \| blk.13.attn_k.weight \| Block 13 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 160 \| blk.13.attn_k_norm.weight \| Block 13 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 161 \| blk.13.attn_norm.weight \| Block 13 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 162 \| blk.13.attn_output.weight \| Block 13 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 163 \| blk.13.attn_q.weight \| Block 13 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 164 \| blk.13.attn_q_norm.weight \| Block 13 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 165 \| blk.13.attn_v.weight \| Block 13 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_XS \|
	\| 166 \| blk.13.ffn_down_exps.weight \| Block 13 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 167 \| blk.13.ffn_gate_exps.weight \| Block 13 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|
	\| 168 \| blk.13.ffn_gate_inp.weight \| Block 13 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 169 \| blk.13.ffn_norm.weight \| Block 13 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 170 \| blk.13.ffn_up_exps.weight \| Block 13 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|

	- Total elements in blk.13: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_14">Block 14 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 171 \| blk.14.attn_k.weight \| Block 14 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 172 \| blk.14.attn_k_norm.weight \| Block 14 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 173 \| blk.14.attn_norm.weight \| Block 14 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 174 \| blk.14.attn_output.weight \| Block 14 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 175 \| blk.14.attn_q.weight \| Block 14 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 176 \| blk.14.attn_q_norm.weight \| Block 14 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 177 \| blk.14.attn_v.weight \| Block 14 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_XS \|
	\| 178 \| blk.14.ffn_down_exps.weight \| Block 14 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 179 \| blk.14.ffn_gate_exps.weight \| Block 14 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 180 \| blk.14.ffn_gate_inp.weight \| Block 14 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 181 \| blk.14.ffn_norm.weight \| Block 14 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 182 \| blk.14.ffn_up_exps.weight \| Block 14 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.14: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_15">Block 15 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 183 \| blk.15.attn_k.weight \| Block 15 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 184 \| blk.15.attn_k_norm.weight \| Block 15 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 185 \| blk.15.attn_norm.weight \| Block 15 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 186 \| blk.15.attn_output.weight \| Block 15 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 187 \| blk.15.attn_q.weight \| Block 15 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 188 \| blk.15.attn_q_norm.weight \| Block 15 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 189 \| blk.15.attn_v.weight \| Block 15 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_XS \|
	\| 190 \| blk.15.ffn_down_exps.weight \| Block 15 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 191 \| blk.15.ffn_gate_exps.weight \| Block 15 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|
	\| 192 \| blk.15.ffn_gate_inp.weight \| Block 15 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 193 \| blk.15.ffn_norm.weight \| Block 15 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 194 \| blk.15.ffn_up_exps.weight \| Block 15 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|

	- Total elements in blk.15: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_16">Block 16 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 195 \| blk.16.attn_k.weight \| Block 16 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 196 \| blk.16.attn_k_norm.weight \| Block 16 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 197 \| blk.16.attn_norm.weight \| Block 16 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 198 \| blk.16.attn_output.weight \| Block 16 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 199 \| blk.16.attn_q.weight \| Block 16 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 200 \| blk.16.attn_q_norm.weight \| Block 16 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 201 \| blk.16.attn_v.weight \| Block 16 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_XS \|
	\| 202 \| blk.16.ffn_down_exps.weight \| Block 16 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 203 \| blk.16.ffn_gate_exps.weight \| Block 16 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 204 \| blk.16.ffn_gate_inp.weight \| Block 16 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 205 \| blk.16.ffn_norm.weight \| Block 16 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 206 \| blk.16.ffn_up_exps.weight \| Block 16 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.16: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_17">Block 17 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 207 \| blk.17.attn_k.weight \| Block 17 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 208 \| blk.17.attn_k_norm.weight \| Block 17 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 209 \| blk.17.attn_norm.weight \| Block 17 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 210 \| blk.17.attn_output.weight \| Block 17 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 211 \| blk.17.attn_q.weight \| Block 17 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 212 \| blk.17.attn_q_norm.weight \| Block 17 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 213 \| blk.17.attn_v.weight \| Block 17 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_XS \|
	\| 214 \| blk.17.ffn_down_exps.weight \| Block 17 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 215 \| blk.17.ffn_gate_exps.weight \| Block 17 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 216 \| blk.17.ffn_gate_inp.weight \| Block 17 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 217 \| blk.17.ffn_norm.weight \| Block 17 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 218 \| blk.17.ffn_up_exps.weight \| Block 17 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.17: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_18">Block 18 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 219 \| blk.18.attn_k.weight \| Block 18 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 220 \| blk.18.attn_k_norm.weight \| Block 18 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 221 \| blk.18.attn_norm.weight \| Block 18 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 222 \| blk.18.attn_output.weight \| Block 18 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 223 \| blk.18.attn_q.weight \| Block 18 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 224 \| blk.18.attn_q_norm.weight \| Block 18 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 225 \| blk.18.attn_v.weight \| Block 18 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_XS \|
	\| 226 \| blk.18.ffn_down_exps.weight \| Block 18 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 227 \| blk.18.ffn_gate_exps.weight \| Block 18 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 228 \| blk.18.ffn_gate_inp.weight \| Block 18 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 229 \| blk.18.ffn_norm.weight \| Block 18 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 230 \| blk.18.ffn_up_exps.weight \| Block 18 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.18: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_19">Block 19 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 231 \| blk.19.attn_k.weight \| Block 19 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 232 \| blk.19.attn_k_norm.weight \| Block 19 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 233 \| blk.19.attn_norm.weight \| Block 19 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 234 \| blk.19.attn_output.weight \| Block 19 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 235 \| blk.19.attn_q.weight \| Block 19 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 236 \| blk.19.attn_q_norm.weight \| Block 19 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 237 \| blk.19.attn_v.weight \| Block 19 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_XS \|
	\| 238 \| blk.19.ffn_down_exps.weight \| Block 19 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 239 \| blk.19.ffn_gate_exps.weight \| Block 19 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 240 \| blk.19.ffn_gate_inp.weight \| Block 19 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 241 \| blk.19.ffn_norm.weight \| Block 19 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 242 \| blk.19.ffn_up_exps.weight \| Block 19 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.19: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_20">Block 20 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 243 \| blk.20.attn_k.weight \| Block 20 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 244 \| blk.20.attn_k_norm.weight \| Block 20 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 245 \| blk.20.attn_norm.weight \| Block 20 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 246 \| blk.20.attn_output.weight \| Block 20 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 247 \| blk.20.attn_q.weight \| Block 20 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 248 \| blk.20.attn_q_norm.weight \| Block 20 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 249 \| blk.20.attn_v.weight \| Block 20 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_XS \|
	\| 250 \| blk.20.ffn_down_exps.weight \| Block 20 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 251 \| blk.20.ffn_gate_exps.weight \| Block 20 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 252 \| blk.20.ffn_gate_inp.weight \| Block 20 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 253 \| blk.20.ffn_norm.weight \| Block 20 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 254 \| blk.20.ffn_up_exps.weight \| Block 20 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.20: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_21">Block 21 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 255 \| blk.21.attn_k.weight \| Block 21 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 256 \| blk.21.attn_k_norm.weight \| Block 21 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 257 \| blk.21.attn_norm.weight \| Block 21 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 258 \| blk.21.attn_output.weight \| Block 21 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 259 \| blk.21.attn_q.weight \| Block 21 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 260 \| blk.21.attn_q_norm.weight \| Block 21 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 261 \| blk.21.attn_v.weight \| Block 21 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_XS \|
	\| 262 \| blk.21.ffn_down_exps.weight \| Block 21 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 263 \| blk.21.ffn_gate_exps.weight \| Block 21 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 264 \| blk.21.ffn_gate_inp.weight \| Block 21 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 265 \| blk.21.ffn_norm.weight \| Block 21 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 266 \| blk.21.ffn_up_exps.weight \| Block 21 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.21: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_22">Block 22 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 267 \| blk.22.attn_k.weight \| Block 22 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 268 \| blk.22.attn_k_norm.weight \| Block 22 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 269 \| blk.22.attn_norm.weight \| Block 22 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 270 \| blk.22.attn_output.weight \| Block 22 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 271 \| blk.22.attn_q.weight \| Block 22 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 272 \| blk.22.attn_q_norm.weight \| Block 22 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 273 \| blk.22.attn_v.weight \| Block 22 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_XS \|
	\| 274 \| blk.22.ffn_down_exps.weight \| Block 22 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 275 \| blk.22.ffn_gate_exps.weight \| Block 22 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 276 \| blk.22.ffn_gate_inp.weight \| Block 22 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 277 \| blk.22.ffn_norm.weight \| Block 22 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 278 \| blk.22.ffn_up_exps.weight \| Block 22 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.22: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_23">Block 23 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 279 \| blk.23.attn_k.weight \| Block 23 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ3_S \|
	\| 280 \| blk.23.attn_k_norm.weight \| Block 23 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 281 \| blk.23.attn_norm.weight \| Block 23 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 282 \| blk.23.attn_output.weight \| Block 23 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 283 \| blk.23.attn_q.weight \| Block 23 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ3_S \|
	\| 284 \| blk.23.attn_q_norm.weight \| Block 23 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 285 \| blk.23.attn_v.weight \| Block 23 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_XS \|
	\| 286 \| blk.23.ffn_down_exps.weight \| Block 23 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 287 \| blk.23.ffn_gate_exps.weight \| Block 23 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 288 \| blk.23.ffn_gate_inp.weight \| Block 23 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 289 \| blk.23.ffn_norm.weight \| Block 23 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 290 \| blk.23.ffn_up_exps.weight \| Block 23 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.23: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_24">Block 24 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 291 \| blk.24.attn_k.weight \| Block 24 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 292 \| blk.24.attn_k_norm.weight \| Block 24 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 293 \| blk.24.attn_norm.weight \| Block 24 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 294 \| blk.24.attn_output.weight \| Block 24 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 295 \| blk.24.attn_q.weight \| Block 24 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ4_NL \|
	\| 296 \| blk.24.attn_q_norm.weight \| Block 24 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 297 \| blk.24.attn_v.weight \| Block 24 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 298 \| blk.24.ffn_down_exps.weight \| Block 24 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 299 \| blk.24.ffn_gate_exps.weight \| Block 24 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 300 \| blk.24.ffn_gate_inp.weight \| Block 24 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 301 \| blk.24.ffn_norm.weight \| Block 24 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 302 \| blk.24.ffn_up_exps.weight \| Block 24 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.24: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_25">Block 25 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 303 \| blk.25.attn_k.weight \| Block 25 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 304 \| blk.25.attn_k_norm.weight \| Block 25 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 305 \| blk.25.attn_norm.weight \| Block 25 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 306 \| blk.25.attn_output.weight \| Block 25 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 307 \| blk.25.attn_q.weight \| Block 25 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ4_NL \|
	\| 308 \| blk.25.attn_q_norm.weight \| Block 25 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 309 \| blk.25.attn_v.weight \| Block 25 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 310 \| blk.25.ffn_down_exps.weight \| Block 25 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 311 \| blk.25.ffn_gate_exps.weight \| Block 25 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|
	\| 312 \| blk.25.ffn_gate_inp.weight \| Block 25 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 313 \| blk.25.ffn_norm.weight \| Block 25 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 314 \| blk.25.ffn_up_exps.weight \| Block 25 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|

	- Total elements in blk.25: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_26">Block 26 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 315 \| blk.26.attn_k.weight \| Block 26 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 316 \| blk.26.attn_k_norm.weight \| Block 26 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 317 \| blk.26.attn_norm.weight \| Block 26 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 318 \| blk.26.attn_output.weight \| Block 26 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 319 \| blk.26.attn_q.weight \| Block 26 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ4_NL \|
	\| 320 \| blk.26.attn_q_norm.weight \| Block 26 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 321 \| blk.26.attn_v.weight \| Block 26 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 322 \| blk.26.ffn_down_exps.weight \| Block 26 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 323 \| blk.26.ffn_gate_exps.weight \| Block 26 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|
	\| 324 \| blk.26.ffn_gate_inp.weight \| Block 26 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 325 \| blk.26.ffn_norm.weight \| Block 26 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 326 \| blk.26.ffn_up_exps.weight \| Block 26 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ3_S \|

	- Total elements in blk.26: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_27">Block 27 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 327 \| blk.27.attn_k.weight \| Block 27 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 328 \| blk.27.attn_k_norm.weight \| Block 27 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 329 \| blk.27.attn_norm.weight \| Block 27 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 330 \| blk.27.attn_output.weight \| Block 27 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 331 \| blk.27.attn_q.weight \| Block 27 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ4_NL \|
	\| 332 \| blk.27.attn_q_norm.weight \| Block 27 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 333 \| blk.27.attn_v.weight \| Block 27 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 334 \| blk.27.ffn_down_exps.weight \| Block 27 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 335 \| blk.27.ffn_gate_exps.weight \| Block 27 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|
	\| 336 \| blk.27.ffn_gate_inp.weight \| Block 27 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 337 \| blk.27.ffn_norm.weight \| Block 27 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 338 \| blk.27.ffn_up_exps.weight \| Block 27 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|

	- Total elements in blk.27: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_28">Block 28 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 339 \| blk.28.attn_k.weight \| Block 28 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 340 \| blk.28.attn_k_norm.weight \| Block 28 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 341 \| blk.28.attn_norm.weight \| Block 28 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 342 \| blk.28.attn_output.weight \| Block 28 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 343 \| blk.28.attn_q.weight \| Block 28 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ4_NL \|
	\| 344 \| blk.28.attn_q_norm.weight \| Block 28 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 345 \| blk.28.attn_v.weight \| Block 28 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 346 \| blk.28.ffn_down_exps.weight \| Block 28 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 347 \| blk.28.ffn_gate_exps.weight \| Block 28 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|
	\| 348 \| blk.28.ffn_gate_inp.weight \| Block 28 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 349 \| blk.28.ffn_norm.weight \| Block 28 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 350 \| blk.28.ffn_up_exps.weight \| Block 28 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|

	- Total elements in blk.28: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_29">Block 29 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 351 \| blk.29.attn_k.weight \| Block 29 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 352 \| blk.29.attn_k_norm.weight \| Block 29 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 353 \| blk.29.attn_norm.weight \| Block 29 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 354 \| blk.29.attn_output.weight \| Block 29 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 355 \| blk.29.attn_q.weight \| Block 29 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ4_NL \|
	\| 356 \| blk.29.attn_q_norm.weight \| Block 29 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 357 \| blk.29.attn_v.weight \| Block 29 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 358 \| blk.29.ffn_down_exps.weight \| Block 29 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 359 \| blk.29.ffn_gate_exps.weight \| Block 29 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|
	\| 360 \| blk.29.ffn_gate_inp.weight \| Block 29 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 361 \| blk.29.ffn_norm.weight \| Block 29 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 362 \| blk.29.ffn_up_exps.weight \| Block 29 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|

	- Total elements in blk.29: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_30">Block 30 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 363 \| blk.30.attn_k.weight \| Block 30 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 364 \| blk.30.attn_k_norm.weight \| Block 30 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 365 \| blk.30.attn_norm.weight \| Block 30 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 366 \| blk.30.attn_output.weight \| Block 30 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 367 \| blk.30.attn_q.weight \| Block 30 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ4_NL \|
	\| 368 \| blk.30.attn_q_norm.weight \| Block 30 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 369 \| blk.30.attn_v.weight \| Block 30 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 370 \| blk.30.ffn_down_exps.weight \| Block 30 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 371 \| blk.30.ffn_gate_exps.weight \| Block 30 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|
	\| 372 \| blk.30.ffn_gate_inp.weight \| Block 30 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 373 \| blk.30.ffn_norm.weight \| Block 30 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 374 \| blk.30.ffn_up_exps.weight \| Block 30 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|

	- Total elements in blk.30: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_31">Block 31 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 375 \| blk.31.attn_k.weight \| Block 31 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 376 \| blk.31.attn_k_norm.weight \| Block 31 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 377 \| blk.31.attn_norm.weight \| Block 31 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 378 \| blk.31.attn_output.weight \| Block 31 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 379 \| blk.31.attn_q.weight \| Block 31 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ4_NL \|
	\| 380 \| blk.31.attn_q_norm.weight \| Block 31 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 381 \| blk.31.attn_v.weight \| Block 31 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 382 \| blk.31.ffn_down_exps.weight \| Block 31 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 383 \| blk.31.ffn_gate_exps.weight \| Block 31 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|
	\| 384 \| blk.31.ffn_gate_inp.weight \| Block 31 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 385 \| blk.31.ffn_norm.weight \| Block 31 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 386 \| blk.31.ffn_up_exps.weight \| Block 31 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|

	- Total elements in blk.31: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_32">Block 32 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 387 \| blk.32.attn_k.weight \| Block 32 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 388 \| blk.32.attn_k_norm.weight \| Block 32 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 389 \| blk.32.attn_norm.weight \| Block 32 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 390 \| blk.32.attn_output.weight \| Block 32 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 391 \| blk.32.attn_q.weight \| Block 32 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ4_NL \|
	\| 392 \| blk.32.attn_q_norm.weight \| Block 32 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 393 \| blk.32.attn_v.weight \| Block 32 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 394 \| blk.32.ffn_down_exps.weight \| Block 32 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 395 \| blk.32.ffn_gate_exps.weight \| Block 32 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|
	\| 396 \| blk.32.ffn_gate_inp.weight \| Block 32 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 397 \| blk.32.ffn_norm.weight \| Block 32 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 398 \| blk.32.ffn_up_exps.weight \| Block 32 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|

	- Total elements in blk.32: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_33">Block 33 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 399 \| blk.33.attn_k.weight \| Block 33 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 400 \| blk.33.attn_k_norm.weight \| Block 33 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 401 \| blk.33.attn_norm.weight \| Block 33 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 402 \| blk.33.attn_output.weight \| Block 33 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 403 \| blk.33.attn_q.weight \| Block 33 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ4_NL \|
	\| 404 \| blk.33.attn_q_norm.weight \| Block 33 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 405 \| blk.33.attn_v.weight \| Block 33 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 406 \| blk.33.ffn_down_exps.weight \| Block 33 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 407 \| blk.33.ffn_gate_exps.weight \| Block 33 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|
	\| 408 \| blk.33.ffn_gate_inp.weight \| Block 33 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 409 \| blk.33.ffn_norm.weight \| Block 33 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 410 \| blk.33.ffn_up_exps.weight \| Block 33 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|

	- Total elements in blk.33: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_34">Block 34 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 411 \| blk.34.attn_k.weight \| Block 34 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 412 \| blk.34.attn_k_norm.weight \| Block 34 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 413 \| blk.34.attn_norm.weight \| Block 34 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 414 \| blk.34.attn_output.weight \| Block 34 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 415 \| blk.34.attn_q.weight \| Block 34 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ4_NL \|
	\| 416 \| blk.34.attn_q_norm.weight \| Block 34 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 417 \| blk.34.attn_v.weight \| Block 34 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 418 \| blk.34.ffn_down_exps.weight \| Block 34 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 419 \| blk.34.ffn_gate_exps.weight \| Block 34 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|
	\| 420 \| blk.34.ffn_gate_inp.weight \| Block 34 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 421 \| blk.34.ffn_norm.weight \| Block 34 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 422 \| blk.34.ffn_up_exps.weight \| Block 34 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|

	- Total elements in blk.34: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_35">Block 35 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 423 \| blk.35.attn_k.weight \| Block 35 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 424 \| blk.35.attn_k_norm.weight \| Block 35 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 425 \| blk.35.attn_norm.weight \| Block 35 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 426 \| blk.35.attn_output.weight \| Block 35 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 427 \| blk.35.attn_q.weight \| Block 35 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ4_NL \|
	\| 428 \| blk.35.attn_q_norm.weight \| Block 35 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 429 \| blk.35.attn_v.weight \| Block 35 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 430 \| blk.35.ffn_down_exps.weight \| Block 35 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 431 \| blk.35.ffn_gate_exps.weight \| Block 35 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|
	\| 432 \| blk.35.ffn_gate_inp.weight \| Block 35 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 433 \| blk.35.ffn_norm.weight \| Block 35 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 434 \| blk.35.ffn_up_exps.weight \| Block 35 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|

	- Total elements in blk.35: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_36">Block 36 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 435 \| blk.36.attn_k.weight \| Block 36 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 436 \| blk.36.attn_k_norm.weight \| Block 36 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 437 \| blk.36.attn_norm.weight \| Block 36 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 438 \| blk.36.attn_output.weight \| Block 36 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 439 \| blk.36.attn_q.weight \| Block 36 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ4_NL \|
	\| 440 \| blk.36.attn_q_norm.weight \| Block 36 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 441 \| blk.36.attn_v.weight \| Block 36 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 442 \| blk.36.ffn_down_exps.weight \| Block 36 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 443 \| blk.36.ffn_gate_exps.weight \| Block 36 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|
	\| 444 \| blk.36.ffn_gate_inp.weight \| Block 36 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 445 \| blk.36.ffn_norm.weight \| Block 36 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 446 \| blk.36.ffn_up_exps.weight \| Block 36 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|

	- Total elements in blk.36: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_37">Block 37 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 447 \| blk.37.attn_k.weight \| Block 37 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 448 \| blk.37.attn_k_norm.weight \| Block 37 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 449 \| blk.37.attn_norm.weight \| Block 37 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 450 \| blk.37.attn_output.weight \| Block 37 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 451 \| blk.37.attn_q.weight \| Block 37 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ4_NL \|
	\| 452 \| blk.37.attn_q_norm.weight \| Block 37 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 453 \| blk.37.attn_v.weight \| Block 37 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 454 \| blk.37.ffn_down_exps.weight \| Block 37 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 455 \| blk.37.ffn_gate_exps.weight \| Block 37 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|
	\| 456 \| blk.37.ffn_gate_inp.weight \| Block 37 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 457 \| blk.37.ffn_norm.weight \| Block 37 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 458 \| blk.37.ffn_up_exps.weight \| Block 37 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|

	- Total elements in blk.37: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_38">Block 38 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 459 \| blk.38.attn_k.weight \| Block 38 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 460 \| blk.38.attn_k_norm.weight \| Block 38 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 461 \| blk.38.attn_norm.weight \| Block 38 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 462 \| blk.38.attn_output.weight \| Block 38 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 463 \| blk.38.attn_q.weight \| Block 38 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ4_NL \|
	\| 464 \| blk.38.attn_q_norm.weight \| Block 38 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 465 \| blk.38.attn_v.weight \| Block 38 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 466 \| blk.38.ffn_down_exps.weight \| Block 38 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 467 \| blk.38.ffn_gate_exps.weight \| Block 38 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|
	\| 468 \| blk.38.ffn_gate_inp.weight \| Block 38 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 469 \| blk.38.ffn_norm.weight \| Block 38 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 470 \| blk.38.ffn_up_exps.weight \| Block 38 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|

	- Total elements in blk.38: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_39">Block 39 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 471 \| blk.39.attn_k.weight \| Block 39 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 472 \| blk.39.attn_k_norm.weight \| Block 39 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 473 \| blk.39.attn_norm.weight \| Block 39 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 474 \| blk.39.attn_output.weight \| Block 39 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 475 \| blk.39.attn_q.weight \| Block 39 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ4_NL \|
	\| 476 \| blk.39.attn_q_norm.weight \| Block 39 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 477 \| blk.39.attn_v.weight \| Block 39 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 478 \| blk.39.ffn_down_exps.weight \| Block 39 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 479 \| blk.39.ffn_gate_exps.weight \| Block 39 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|
	\| 480 \| blk.39.ffn_gate_inp.weight \| Block 39 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 481 \| blk.39.ffn_norm.weight \| Block 39 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 482 \| blk.39.ffn_up_exps.weight \| Block 39 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|

	- Total elements in blk.39: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_40">Block 40 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 483 \| blk.40.attn_k.weight \| Block 40 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 484 \| blk.40.attn_k_norm.weight \| Block 40 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 485 \| blk.40.attn_norm.weight \| Block 40 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 486 \| blk.40.attn_output.weight \| Block 40 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 487 \| blk.40.attn_q.weight \| Block 40 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ4_NL \|
	\| 488 \| blk.40.attn_q_norm.weight \| Block 40 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 489 \| blk.40.attn_v.weight \| Block 40 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 490 \| blk.40.ffn_down_exps.weight \| Block 40 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 491 \| blk.40.ffn_gate_exps.weight \| Block 40 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|
	\| 492 \| blk.40.ffn_gate_inp.weight \| Block 40 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 493 \| blk.40.ffn_norm.weight \| Block 40 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 494 \| blk.40.ffn_up_exps.weight \| Block 40 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|

	- Total elements in blk.40: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_41">Block 41 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 495 \| blk.41.attn_k.weight \| Block 41 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 496 \| blk.41.attn_k_norm.weight \| Block 41 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 497 \| blk.41.attn_norm.weight \| Block 41 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 498 \| blk.41.attn_output.weight \| Block 41 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 499 \| blk.41.attn_q.weight \| Block 41 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ4_NL \|
	\| 500 \| blk.41.attn_q_norm.weight \| Block 41 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 501 \| blk.41.attn_v.weight \| Block 41 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 502 \| blk.41.ffn_down_exps.weight \| Block 41 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 503 \| blk.41.ffn_gate_exps.weight \| Block 41 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|
	\| 504 \| blk.41.ffn_gate_inp.weight \| Block 41 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 505 \| blk.41.ffn_norm.weight \| Block 41 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 506 \| blk.41.ffn_up_exps.weight \| Block 41 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|

	- Total elements in blk.41: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_42">Block 42 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 507 \| blk.42.attn_k.weight \| Block 42 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 508 \| blk.42.attn_k_norm.weight \| Block 42 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 509 \| blk.42.attn_norm.weight \| Block 42 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 510 \| blk.42.attn_output.weight \| Block 42 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 511 \| blk.42.attn_q.weight \| Block 42 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ4_NL \|
	\| 512 \| blk.42.attn_q_norm.weight \| Block 42 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 513 \| blk.42.attn_v.weight \| Block 42 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 514 \| blk.42.ffn_down_exps.weight \| Block 42 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 515 \| blk.42.ffn_gate_exps.weight \| Block 42 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|
	\| 516 \| blk.42.ffn_gate_inp.weight \| Block 42 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 517 \| blk.42.ffn_norm.weight \| Block 42 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 518 \| blk.42.ffn_up_exps.weight \| Block 42 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|

	- Total elements in blk.42: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_43">Block 43 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 519 \| blk.43.attn_k.weight \| Block 43 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 520 \| blk.43.attn_k_norm.weight \| Block 43 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 521 \| blk.43.attn_norm.weight \| Block 43 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 522 \| blk.43.attn_output.weight \| Block 43 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 523 \| blk.43.attn_q.weight \| Block 43 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ4_NL \|
	\| 524 \| blk.43.attn_q_norm.weight \| Block 43 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 525 \| blk.43.attn_v.weight \| Block 43 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 526 \| blk.43.ffn_down_exps.weight \| Block 43 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 527 \| blk.43.ffn_gate_exps.weight \| Block 43 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|
	\| 528 \| blk.43.ffn_gate_inp.weight \| Block 43 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 529 \| blk.43.ffn_norm.weight \| Block 43 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 530 \| blk.43.ffn_up_exps.weight \| Block 43 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|

	- Total elements in blk.43: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_44">Block 44 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 531 \| blk.44.attn_k.weight \| Block 44 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 532 \| blk.44.attn_k_norm.weight \| Block 44 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 533 \| blk.44.attn_norm.weight \| Block 44 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 534 \| blk.44.attn_output.weight \| Block 44 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 535 \| blk.44.attn_q.weight \| Block 44 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ4_NL \|
	\| 536 \| blk.44.attn_q_norm.weight \| Block 44 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 537 \| blk.44.attn_v.weight \| Block 44 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 538 \| blk.44.ffn_down_exps.weight \| Block 44 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 539 \| blk.44.ffn_gate_exps.weight \| Block 44 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|
	\| 540 \| blk.44.ffn_gate_inp.weight \| Block 44 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 541 \| blk.44.ffn_norm.weight \| Block 44 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 542 \| blk.44.ffn_up_exps.weight \| Block 44 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|

	- Total elements in blk.44: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_45">Block 45 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 543 \| blk.45.attn_k.weight \| Block 45 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 544 \| blk.45.attn_k_norm.weight \| Block 45 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 545 \| blk.45.attn_norm.weight \| Block 45 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 546 \| blk.45.attn_output.weight \| Block 45 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 547 \| blk.45.attn_q.weight \| Block 45 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ4_NL \|
	\| 548 \| blk.45.attn_q_norm.weight \| Block 45 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 549 \| blk.45.attn_v.weight \| Block 45 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 550 \| blk.45.ffn_down_exps.weight \| Block 45 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 551 \| blk.45.ffn_gate_exps.weight \| Block 45 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|
	\| 552 \| blk.45.ffn_gate_inp.weight \| Block 45 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 553 \| blk.45.ffn_norm.weight \| Block 45 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 554 \| blk.45.ffn_up_exps.weight \| Block 45 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|

	- Total elements in blk.45: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_46">Block 46 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 555 \| blk.46.attn_k.weight \| Block 46 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 556 \| blk.46.attn_k_norm.weight \| Block 46 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 557 \| blk.46.attn_norm.weight \| Block 46 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 558 \| blk.46.attn_output.weight \| Block 46 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 559 \| blk.46.attn_q.weight \| Block 46 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ4_NL \|
	\| 560 \| blk.46.attn_q_norm.weight \| Block 46 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 561 \| blk.46.attn_v.weight \| Block 46 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 562 \| blk.46.ffn_down_exps.weight \| Block 46 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 563 \| blk.46.ffn_gate_exps.weight \| Block 46 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|
	\| 564 \| blk.46.ffn_gate_inp.weight \| Block 46 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 565 \| blk.46.ffn_norm.weight \| Block 46 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 566 \| blk.46.ffn_up_exps.weight \| Block 46 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|

	- Total elements in blk.46: (~623M) 623120640
	- Percentage of total elements: 2.04%


	### <a name="blk_47">Block 47 Tensor Group : ~623M Elements</a>

	\| T_ID \| Tensor Layer Name \| Human Friendly Tensor Layer Name \| Elements \| Shape \| Type \|
	\|-----:\|:----------------------------\|:-------------------------------------------------------------------------------------------\|:------------------\|:----------------------\|:-------\|
	\| 567 \| blk.47.attn_k.weight \| Block 47 Attention Key (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 568 \| blk.47.attn_k_norm.weight \| Block 47 Attn_K_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 569 \| blk.47.attn_norm.weight \| Block 47 Attention Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 570 \| blk.47.attn_output.weight \| Block 47 Attention Output (W) \| ( ~8M) 8388608 \| 4096 x 2048 x 1 x 1 \| IQ4_NL \|
	\| 571 \| blk.47.attn_q.weight \| Block 47 Attention Query (W) \| ( ~8M) 8388608 \| 2048 x 4096 x 1 x 1 \| IQ4_NL \|
	\| 572 \| blk.47.attn_q_norm.weight \| Block 47 Attn_Q_Norm (W) \| ( 128) 128 \| 128 x 1 x 1 x 1 \| F32 \|
	\| 573 \| blk.47.attn_v.weight \| Block 47 Attention Value (W) \| ( ~1M) 1048576 \| 2048 x 512 x 1 x 1 \| IQ4_NL \|
	\| 574 \| blk.47.ffn_down_exps.weight \| Block 47 Ffn_Down_Exps (W) \| (~201M) 201326592 \| 768 x 2048 x 128 x 1 \| Q5_K \|
	\| 575 \| blk.47.ffn_gate_exps.weight \| Block 47 Ffn_Gate_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|
	\| 576 \| blk.47.ffn_gate_inp.weight \| Block 47 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) \| (~262K) 262144 \| 2048 x 128 x 1 x 1 \| F32 \|
	\| 577 \| blk.47.ffn_norm.weight \| Block 47 Feed-Forward Network Normalization (W) \| ( ~2K) 2048 \| 2048 x 1 x 1 x 1 \| F32 \|
	\| 578 \| blk.47.ffn_up_exps.weight \| Block 47 Ffn_Up_Exps (W) \| (~201M) 201326592 \| 2048 x 768 x 128 x 1 \| IQ4_NL \|

	- Total elements in blk.47: (~623M) 623120640
	- Percentage of total elements: 2.04%