Instructions to use eaddario/Qwen3-30B-A3B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use eaddario/Qwen3-30B-A3B-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="eaddario/Qwen3-30B-A3B-GGUF",
	filename="Qwen3-30B-A3B-F16-00001-of-00003.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use eaddario/Qwen3-30B-A3B-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use eaddario/Qwen3-30B-A3B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "eaddario/Qwen3-30B-A3B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "eaddario/Qwen3-30B-A3B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Ollama
How to use eaddario/Qwen3-30B-A3B-GGUF with Ollama:
```
ollama run hf.co/eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M
```

Unsloth Studio

How to use eaddario/Qwen3-30B-A3B-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for eaddario/Qwen3-30B-A3B-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for eaddario/Qwen3-30B-A3B-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for eaddario/Qwen3-30B-A3B-GGUF to start chatting

How to use eaddario/Qwen3-30B-A3B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use eaddario/Qwen3-30B-A3B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use eaddario/Qwen3-30B-A3B-GGUF with Docker Model Runner:
```
docker model run hf.co/eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M
```

Lemonade

How to use eaddario/Qwen3-30B-A3B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull eaddario/Qwen3-30B-A3B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Qwen3-30B-A3B-GGUF-Q4_K_M

List all available models

lemonade list

Qwen3-30B-A3B-GGUF / scores /Qwen3-30B-A3B-IQ4_NL.md

eaddario

Add GGUF internal file structure

a630e18 verified about 1 year ago

preview code

raw

history blame

188 kB

Qwen3-30B-A3B-IQ4_NL.gguf - GGUF Internal File Dump

Endian: LITTLE endian

Key Value Metadata Store

There are 44 key-value pairs in this file

POS	TYPE	Count	Key	Value
1	UINT32	1	GGUF.version	3
2	UINT64	1	GGUF.tensor_count	579
3	UINT64	1	GGUF.kv_count	41
4	STRING	1	general.architecture	`qwen3moe`
5	STRING	1	general.type	`model`
6	STRING	1	general.name	`Qwen3 30B A3B`
7	STRING	1	general.basename	`Qwen3`
8	STRING	1	general.size_label	`30B-A3B`
9	STRING	1	general.license	`apache-2.0`
10	STRING	1	general.license.link	`https://huggingface.co/Qwen/Qwen3-30B-A3B/blob/main/LICENSE`
11	UINT32	1	general.base_model.count	1
12	STRING	1	general.base_model.0.name	`Qwen3 30B A3B Base`
13	STRING	1	general.base_model.0.organization	`Qwen`
14	STRING	1	general.base_model.0.repo_url	`https://huggingface.co/Qwen/Qwen3-30B-A3B-Base`
15	[STRING]	1	general.tags	[ `text-generation` ]
16	UINT32	1	qwen3moe.block_count	48
17	UINT32	1	qwen3moe.context_length	40960
18	UINT32	1	qwen3moe.embedding_length	2048
19	UINT32	1	qwen3moe.feed_forward_length	6144
20	UINT32	1	qwen3moe.attention.head_count	32
21	UINT32	1	qwen3moe.attention.head_count_kv	4
22	FLOAT32	1	qwen3moe.rope.freq_base	1000000.0
23	FLOAT32	1	qwen3moe.attention.layer_norm_rms_epsilon	1e-06
24	UINT32	1	qwen3moe.expert_used_count	8
25	UINT32	1	qwen3moe.attention.key_length	128
26	UINT32	1	qwen3moe.attention.value_length	128
27	UINT32	1	qwen3moe.expert_count	128
28	UINT32	1	qwen3moe.expert_feed_forward_length	768
29	STRING	1	tokenizer.ggml.model	`gpt2`
30	STRING	1	tokenizer.ggml.pre	`qwen2`
31	[STRING]	151936	tokenizer.ggml.tokens	[ `!`, `"`, `#`, `$`, `%`, ... ]
32	[INT32]	151936	tokenizer.ggml.token_type	[ 1, 1, 1, 1, 1, 1, 1, ... ]
33	[STRING]	151387	tokenizer.ggml.merges	[ `Ġ Ġ`, `ĠĠ ĠĠ`, `i n`, `Ġ t`, `ĠĠĠĠ ĠĠĠĠ`, ... ]
34	UINT32	1	tokenizer.ggml.eos_token_id	151645
35	UINT32	1	tokenizer.ggml.padding_token_id	151643
36	UINT32	1	tokenizer.ggml.bos_token_id	151643
37	BOOL	1	tokenizer.ggml.add_bos_token	False
38	STRING	1	tokenizer.chat_template	`{%- if tools %}{{- '<
39	UINT32	1	general.quantization_version	2
40	UINT32	1	general.file_type	25
41	STRING	1	quantize.imatrix.file	`./imatrix/imatrix-Qwen3-30B-A3B-large.dat`
42	STRING	1	quantize.imatrix.dataset	`../../datasets/imatrix/calibration_all_large.txt`
43	INT32	1	quantize.imatrix.entries_count	382
44	INT32	1	quantize.imatrix.chunks_count	4978

Tensors Overview ~31B Elements

Total number of elements in all tensors: 30532122624 Elements

Qwen3-30B-A3B-IQ4_NL.gguf - GGUF Internal File Dump
- Key Value Metadata Store
- Tensors Overview ~31B Elements

Tensor Data Offset

This table contains the offset and data segment relative to start of file

T_ID	Tensor Layer Name	Data Offset (B)	Data Size (B)
0	output.weight	0x5b18c0	0xa6ec000
1	output_norm.weight	0xac9d8c0	0x2000
2	token_embd.weight	0xac9f8c0	0x7f82800
3	blk.0.attn_k.weight	0x12c220c0	0x6e000
4	blk.0.attn_k_norm.weight	0x12c900c0	0x200
5	blk.0.attn_norm.weight	0x12c902c0	0x2000
6	blk.0.attn_output.weight	0x12c922c0	0x480000
7	blk.0.attn_q.weight	0x131122c0	0x370000
8	blk.0.attn_q_norm.weight	0x134822c0	0x200
9	blk.0.attn_v.weight	0x134824c0	0x88000
10	blk.0.ffn_down_exps.weight	0x1350a4c0	0x8400000
11	blk.0.ffn_gate_exps.weight	0x1b90a4c0	0x5280000
12	blk.0.ffn_gate_inp.weight	0x20b8a4c0	0x100000
13	blk.0.ffn_norm.weight	0x20c8a4c0	0x2000
14	blk.0.ffn_up_exps.weight	0x20c8c4c0	0x5280000
15	blk.1.attn_k.weight	0x25f0c4c0	0x6e000
16	blk.1.attn_k_norm.weight	0x25f7a4c0	0x200
17	blk.1.attn_norm.weight	0x25f7a6c0	0x2000
18	blk.1.attn_output.weight	0x25f7c6c0	0x480000
19	blk.1.attn_q.weight	0x263fc6c0	0x370000
20	blk.1.attn_q_norm.weight	0x2676c6c0	0x200
21	blk.1.attn_v.weight	0x2676c8c0	0x88000
22	blk.1.ffn_down_exps.weight	0x267f48c0	0x8400000
23	blk.1.ffn_gate_exps.weight	0x2ebf48c0	0x5280000
24	blk.1.ffn_gate_inp.weight	0x33e748c0	0x100000
25	blk.1.ffn_norm.weight	0x33f748c0	0x2000
26	blk.1.ffn_up_exps.weight	0x33f768c0	0x5280000
27	blk.2.attn_k.weight	0x391f68c0	0x6e000
28	blk.2.attn_k_norm.weight	0x392648c0	0x200
29	blk.2.attn_norm.weight	0x39264ac0	0x2000
30	blk.2.attn_output.weight	0x39266ac0	0x480000
31	blk.2.attn_q.weight	0x396e6ac0	0x370000
32	blk.2.attn_q_norm.weight	0x39a56ac0	0x200
33	blk.2.attn_v.weight	0x39a56cc0	0x88000
34	blk.2.ffn_down_exps.weight	0x39adecc0	0x8400000
35	blk.2.ffn_gate_exps.weight	0x41edecc0	0x5280000
36	blk.2.ffn_gate_inp.weight	0x4715ecc0	0x100000
37	blk.2.ffn_norm.weight	0x4725ecc0	0x2000
38	blk.2.ffn_up_exps.weight	0x47260cc0	0x5280000
39	blk.3.attn_k.weight	0x4c4e0cc0	0x6e000
40	blk.3.attn_k_norm.weight	0x4c54ecc0	0x200
41	blk.3.attn_norm.weight	0x4c54eec0	0x2000
42	blk.3.attn_output.weight	0x4c550ec0	0x480000
43	blk.3.attn_q.weight	0x4c9d0ec0	0x370000
44	blk.3.attn_q_norm.weight	0x4cd40ec0	0x200
45	blk.3.attn_v.weight	0x4cd410c0	0x88000
46	blk.3.ffn_down_exps.weight	0x4cdc90c0	0x8400000
47	blk.3.ffn_gate_exps.weight	0x551c90c0	0x5280000
48	blk.3.ffn_gate_inp.weight	0x5a4490c0	0x100000
49	blk.3.ffn_norm.weight	0x5a5490c0	0x2000
50	blk.3.ffn_up_exps.weight	0x5a54b0c0	0x5280000
51	blk.4.attn_k.weight	0x5f7cb0c0	0x6e000
52	blk.4.attn_k_norm.weight	0x5f8390c0	0x200
53	blk.4.attn_norm.weight	0x5f8392c0	0x2000
54	blk.4.attn_output.weight	0x5f83b2c0	0x480000
55	blk.4.attn_q.weight	0x5fcbb2c0	0x370000
56	blk.4.attn_q_norm.weight	0x6002b2c0	0x200
57	blk.4.attn_v.weight	0x6002b4c0	0x88000
58	blk.4.ffn_down_exps.weight	0x600b34c0	0x8400000
59	blk.4.ffn_gate_exps.weight	0x684b34c0	0x5280000
60	blk.4.ffn_gate_inp.weight	0x6d7334c0	0x100000
61	blk.4.ffn_norm.weight	0x6d8334c0	0x2000
62	blk.4.ffn_up_exps.weight	0x6d8354c0	0x5280000
63	blk.5.attn_k.weight	0x72ab54c0	0x6e000
64	blk.5.attn_k_norm.weight	0x72b234c0	0x200
65	blk.5.attn_norm.weight	0x72b236c0	0x2000
66	blk.5.attn_output.weight	0x72b256c0	0x480000
67	blk.5.attn_q.weight	0x72fa56c0	0x370000
68	blk.5.attn_q_norm.weight	0x733156c0	0x200
69	blk.5.attn_v.weight	0x733158c0	0x88000
70	blk.5.ffn_down_exps.weight	0x7339d8c0	0x8400000
71	blk.5.ffn_gate_exps.weight	0x7b79d8c0	0x5280000
72	blk.5.ffn_gate_inp.weight	0x80a1d8c0	0x100000
73	blk.5.ffn_norm.weight	0x80b1d8c0	0x2000
74	blk.5.ffn_up_exps.weight	0x80b1f8c0	0x5280000
75	blk.6.attn_k.weight	0x85d9f8c0	0x6e000
76	blk.6.attn_k_norm.weight	0x85e0d8c0	0x200
77	blk.6.attn_norm.weight	0x85e0dac0	0x2000
78	blk.6.attn_output.weight	0x85e0fac0	0x480000
79	blk.6.attn_q.weight	0x8628fac0	0x370000
80	blk.6.attn_q_norm.weight	0x865ffac0	0x200
81	blk.6.attn_v.weight	0x865ffcc0	0x88000
82	blk.6.ffn_down_exps.weight	0x86687cc0	0x8400000
83	blk.6.ffn_gate_exps.weight	0x8ea87cc0	0x5280000
84	blk.6.ffn_gate_inp.weight	0x93d07cc0	0x100000
85	blk.6.ffn_norm.weight	0x93e07cc0	0x2000
86	blk.6.ffn_up_exps.weight	0x93e09cc0	0x5280000
87	blk.7.attn_k.weight	0x99089cc0	0x6e000
88	blk.7.attn_k_norm.weight	0x990f7cc0	0x200
89	blk.7.attn_norm.weight	0x990f7ec0	0x2000
90	blk.7.attn_output.weight	0x990f9ec0	0x480000
91	blk.7.attn_q.weight	0x99579ec0	0x370000
92	blk.7.attn_q_norm.weight	0x998e9ec0	0x200
93	blk.7.attn_v.weight	0x998ea0c0	0x88000
94	blk.7.ffn_down_exps.weight	0x999720c0	0x8400000
95	blk.7.ffn_gate_exps.weight	0xa1d720c0	0x5280000
96	blk.7.ffn_gate_inp.weight	0xa6ff20c0	0x100000
97	blk.7.ffn_norm.weight	0xa70f20c0	0x2000
98	blk.7.ffn_up_exps.weight	0xa70f40c0	0x5280000
99	blk.8.attn_k.weight	0xac3740c0	0x6e000
100	blk.8.attn_k_norm.weight	0xac3e20c0	0x200
101	blk.8.attn_norm.weight	0xac3e22c0	0x2000
102	blk.8.attn_output.weight	0xac3e42c0	0x480000
103	blk.8.attn_q.weight	0xac8642c0	0x370000
104	blk.8.attn_q_norm.weight	0xacbd42c0	0x200
105	blk.8.attn_v.weight	0xacbd44c0	0x88000
106	blk.8.ffn_down_exps.weight	0xacc5c4c0	0x8400000
107	blk.8.ffn_gate_exps.weight	0xb505c4c0	0x5280000
108	blk.8.ffn_gate_inp.weight	0xba2dc4c0	0x100000
109	blk.8.ffn_norm.weight	0xba3dc4c0	0x2000
110	blk.8.ffn_up_exps.weight	0xba3de4c0	0x5280000
111	blk.9.attn_k.weight	0xbf65e4c0	0x6e000
112	blk.9.attn_k_norm.weight	0xbf6cc4c0	0x200
113	blk.9.attn_norm.weight	0xbf6cc6c0	0x2000
114	blk.9.attn_output.weight	0xbf6ce6c0	0x480000
115	blk.9.attn_q.weight	0xbfb4e6c0	0x370000
116	blk.9.attn_q_norm.weight	0xbfebe6c0	0x200
117	blk.9.attn_v.weight	0xbfebe8c0	0x88000
118	blk.9.ffn_down_exps.weight	0xbff468c0	0x8400000
119	blk.9.ffn_gate_exps.weight	0xc83468c0	0x5280000
120	blk.9.ffn_gate_inp.weight	0xcd5c68c0	0x100000
121	blk.9.ffn_norm.weight	0xcd6c68c0	0x2000
122	blk.9.ffn_up_exps.weight	0xcd6c88c0	0x5280000
123	blk.10.attn_k.weight	0xd29488c0	0x6e000
124	blk.10.attn_k_norm.weight	0xd29b68c0	0x200
125	blk.10.attn_norm.weight	0xd29b6ac0	0x2000
126	blk.10.attn_output.weight	0xd29b8ac0	0x480000
127	blk.10.attn_q.weight	0xd2e38ac0	0x370000
128	blk.10.attn_q_norm.weight	0xd31a8ac0	0x200
129	blk.10.attn_v.weight	0xd31a8cc0	0x88000
130	blk.10.ffn_down_exps.weight	0xd3230cc0	0x8400000
131	blk.10.ffn_gate_exps.weight	0xdb630cc0	0x5280000
132	blk.10.ffn_gate_inp.weight	0xe08b0cc0	0x100000
133	blk.10.ffn_norm.weight	0xe09b0cc0	0x2000
134	blk.10.ffn_up_exps.weight	0xe09b2cc0	0x5280000
135	blk.11.attn_k.weight	0xe5c32cc0	0x6e000
136	blk.11.attn_k_norm.weight	0xe5ca0cc0	0x200
137	blk.11.attn_norm.weight	0xe5ca0ec0	0x2000
138	blk.11.attn_output.weight	0xe5ca2ec0	0x480000
139	blk.11.attn_q.weight	0xe6122ec0	0x370000
140	blk.11.attn_q_norm.weight	0xe6492ec0	0x200
141	blk.11.attn_v.weight	0xe64930c0	0x88000
142	blk.11.ffn_down_exps.weight	0xe651b0c0	0x8400000
143	blk.11.ffn_gate_exps.weight	0xee91b0c0	0x5280000
144	blk.11.ffn_gate_inp.weight	0xf3b9b0c0	0x100000
145	blk.11.ffn_norm.weight	0xf3c9b0c0	0x2000
146	blk.11.ffn_up_exps.weight	0xf3c9d0c0	0x5280000
147	blk.12.attn_k.weight	0xf8f1d0c0	0x6e000
148	blk.12.attn_k_norm.weight	0xf8f8b0c0	0x200
149	blk.12.attn_norm.weight	0xf8f8b2c0	0x2000
150	blk.12.attn_output.weight	0xf8f8d2c0	0x480000
151	blk.12.attn_q.weight	0xf940d2c0	0x370000
152	blk.12.attn_q_norm.weight	0xf977d2c0	0x200
153	blk.12.attn_v.weight	0xf977d4c0	0x88000
154	blk.12.ffn_down_exps.weight	0xf98054c0	0x8400000
155	blk.12.ffn_gate_exps.weight	0x101c054c0	0x5280000
156	blk.12.ffn_gate_inp.weight	0x106e854c0	0x100000
157	blk.12.ffn_norm.weight	0x106f854c0	0x2000
158	blk.12.ffn_up_exps.weight	0x106f874c0	0x5280000
159	blk.13.attn_k.weight	0x10c2074c0	0x6e000
160	blk.13.attn_k_norm.weight	0x10c2754c0	0x200
161	blk.13.attn_norm.weight	0x10c2756c0	0x2000
162	blk.13.attn_output.weight	0x10c2776c0	0x480000
163	blk.13.attn_q.weight	0x10c6f76c0	0x370000
164	blk.13.attn_q_norm.weight	0x10ca676c0	0x200
165	blk.13.attn_v.weight	0x10ca678c0	0x88000
166	blk.13.ffn_down_exps.weight	0x10caef8c0	0x8400000
167	blk.13.ffn_gate_exps.weight	0x114eef8c0	0x6c00000
168	blk.13.ffn_gate_inp.weight	0x11baef8c0	0x100000
169	blk.13.ffn_norm.weight	0x11bbef8c0	0x2000
170	blk.13.ffn_up_exps.weight	0x11bbf18c0	0x6c00000
171	blk.14.attn_k.weight	0x1227f18c0	0x6e000
172	blk.14.attn_k_norm.weight	0x12285f8c0	0x200
173	blk.14.attn_norm.weight	0x12285fac0	0x2000
174	blk.14.attn_output.weight	0x122861ac0	0x480000
175	blk.14.attn_q.weight	0x122ce1ac0	0x370000
176	blk.14.attn_q_norm.weight	0x123051ac0	0x200
177	blk.14.attn_v.weight	0x123051cc0	0x88000
178	blk.14.ffn_down_exps.weight	0x1230d9cc0	0x8400000
179	blk.14.ffn_gate_exps.weight	0x12b4d9cc0	0x5280000
180	blk.14.ffn_gate_inp.weight	0x130759cc0	0x100000
181	blk.14.ffn_norm.weight	0x130859cc0	0x2000
182	blk.14.ffn_up_exps.weight	0x13085bcc0	0x5280000
183	blk.15.attn_k.weight	0x135adbcc0	0x6e000
184	blk.15.attn_k_norm.weight	0x135b49cc0	0x200
185	blk.15.attn_norm.weight	0x135b49ec0	0x2000
186	blk.15.attn_output.weight	0x135b4bec0	0x480000
187	blk.15.attn_q.weight	0x135fcbec0	0x370000
188	blk.15.attn_q_norm.weight	0x13633bec0	0x200
189	blk.15.attn_v.weight	0x13633c0c0	0x88000
190	blk.15.ffn_down_exps.weight	0x1363c40c0	0x8400000
191	blk.15.ffn_gate_exps.weight	0x13e7c40c0	0x6c00000
192	blk.15.ffn_gate_inp.weight	0x1453c40c0	0x100000
193	blk.15.ffn_norm.weight	0x1454c40c0	0x2000
194	blk.15.ffn_up_exps.weight	0x1454c60c0	0x6c00000
195	blk.16.attn_k.weight	0x14c0c60c0	0x6e000
196	blk.16.attn_k_norm.weight	0x14c1340c0	0x200
197	blk.16.attn_norm.weight	0x14c1342c0	0x2000
198	blk.16.attn_output.weight	0x14c1362c0	0x480000
199	blk.16.attn_q.weight	0x14c5b62c0	0x370000
200	blk.16.attn_q_norm.weight	0x14c9262c0	0x200
201	blk.16.attn_v.weight	0x14c9264c0	0x88000
202	blk.16.ffn_down_exps.weight	0x14c9ae4c0	0x8400000
203	blk.16.ffn_gate_exps.weight	0x154dae4c0	0x5280000
204	blk.16.ffn_gate_inp.weight	0x15a02e4c0	0x100000
205	blk.16.ffn_norm.weight	0x15a12e4c0	0x2000
206	blk.16.ffn_up_exps.weight	0x15a1304c0	0x5280000
207	blk.17.attn_k.weight	0x15f3b04c0	0x6e000
208	blk.17.attn_k_norm.weight	0x15f41e4c0	0x200
209	blk.17.attn_norm.weight	0x15f41e6c0	0x2000
210	blk.17.attn_output.weight	0x15f4206c0	0x480000
211	blk.17.attn_q.weight	0x15f8a06c0	0x370000
212	blk.17.attn_q_norm.weight	0x15fc106c0	0x200
213	blk.17.attn_v.weight	0x15fc108c0	0x88000
214	blk.17.ffn_down_exps.weight	0x15fc988c0	0x8400000
215	blk.17.ffn_gate_exps.weight	0x1680988c0	0x5280000
216	blk.17.ffn_gate_inp.weight	0x16d3188c0	0x100000
217	blk.17.ffn_norm.weight	0x16d4188c0	0x2000
218	blk.17.ffn_up_exps.weight	0x16d41a8c0	0x5280000
219	blk.18.attn_k.weight	0x17269a8c0	0x6e000
220	blk.18.attn_k_norm.weight	0x1727088c0	0x200
221	blk.18.attn_norm.weight	0x172708ac0	0x2000
222	blk.18.attn_output.weight	0x17270aac0	0x480000
223	blk.18.attn_q.weight	0x172b8aac0	0x370000
224	blk.18.attn_q_norm.weight	0x172efaac0	0x200
225	blk.18.attn_v.weight	0x172efacc0	0x88000
226	blk.18.ffn_down_exps.weight	0x172f82cc0	0x8400000
227	blk.18.ffn_gate_exps.weight	0x17b382cc0	0x5280000
228	blk.18.ffn_gate_inp.weight	0x180602cc0	0x100000
229	blk.18.ffn_norm.weight	0x180702cc0	0x2000
230	blk.18.ffn_up_exps.weight	0x180704cc0	0x5280000
231	blk.19.attn_k.weight	0x185984cc0	0x6e000
232	blk.19.attn_k_norm.weight	0x1859f2cc0	0x200
233	blk.19.attn_norm.weight	0x1859f2ec0	0x2000
234	blk.19.attn_output.weight	0x1859f4ec0	0x480000
235	blk.19.attn_q.weight	0x185e74ec0	0x370000
236	blk.19.attn_q_norm.weight	0x1861e4ec0	0x200
237	blk.19.attn_v.weight	0x1861e50c0	0x88000
238	blk.19.ffn_down_exps.weight	0x18626d0c0	0x8400000
239	blk.19.ffn_gate_exps.weight	0x18e66d0c0	0x5280000
240	blk.19.ffn_gate_inp.weight	0x1938ed0c0	0x100000
241	blk.19.ffn_norm.weight	0x1939ed0c0	0x2000
242	blk.19.ffn_up_exps.weight	0x1939ef0c0	0x5280000
243	blk.20.attn_k.weight	0x198c6f0c0	0x6e000
244	blk.20.attn_k_norm.weight	0x198cdd0c0	0x200
245	blk.20.attn_norm.weight	0x198cdd2c0	0x2000
246	blk.20.attn_output.weight	0x198cdf2c0	0x480000
247	blk.20.attn_q.weight	0x19915f2c0	0x370000
248	blk.20.attn_q_norm.weight	0x1994cf2c0	0x200
249	blk.20.attn_v.weight	0x1994cf4c0	0x88000
250	blk.20.ffn_down_exps.weight	0x1995574c0	0x8400000
251	blk.20.ffn_gate_exps.weight	0x1a19574c0	0x5280000
252	blk.20.ffn_gate_inp.weight	0x1a6bd74c0	0x100000
253	blk.20.ffn_norm.weight	0x1a6cd74c0	0x2000
254	blk.20.ffn_up_exps.weight	0x1a6cd94c0	0x5280000
255	blk.21.attn_k.weight	0x1abf594c0	0x6e000
256	blk.21.attn_k_norm.weight	0x1abfc74c0	0x200
257	blk.21.attn_norm.weight	0x1abfc76c0	0x2000
258	blk.21.attn_output.weight	0x1abfc96c0	0x480000
259	blk.21.attn_q.weight	0x1ac4496c0	0x370000
260	blk.21.attn_q_norm.weight	0x1ac7b96c0	0x200
261	blk.21.attn_v.weight	0x1ac7b98c0	0x88000
262	blk.21.ffn_down_exps.weight	0x1ac8418c0	0x8400000
263	blk.21.ffn_gate_exps.weight	0x1b4c418c0	0x5280000
264	blk.21.ffn_gate_inp.weight	0x1b9ec18c0	0x100000
265	blk.21.ffn_norm.weight	0x1b9fc18c0	0x2000
266	blk.21.ffn_up_exps.weight	0x1b9fc38c0	0x5280000
267	blk.22.attn_k.weight	0x1bf2438c0	0x6e000
268	blk.22.attn_k_norm.weight	0x1bf2b18c0	0x200
269	blk.22.attn_norm.weight	0x1bf2b1ac0	0x2000
270	blk.22.attn_output.weight	0x1bf2b3ac0	0x480000
271	blk.22.attn_q.weight	0x1bf733ac0	0x370000
272	blk.22.attn_q_norm.weight	0x1bfaa3ac0	0x200
273	blk.22.attn_v.weight	0x1bfaa3cc0	0x88000
274	blk.22.ffn_down_exps.weight	0x1bfb2bcc0	0x8400000
275	blk.22.ffn_gate_exps.weight	0x1c7f2bcc0	0x5280000
276	blk.22.ffn_gate_inp.weight	0x1cd1abcc0	0x100000
277	blk.22.ffn_norm.weight	0x1cd2abcc0	0x2000
278	blk.22.ffn_up_exps.weight	0x1cd2adcc0	0x5280000
279	blk.23.attn_k.weight	0x1d252dcc0	0x6e000
280	blk.23.attn_k_norm.weight	0x1d259bcc0	0x200
281	blk.23.attn_norm.weight	0x1d259bec0	0x2000
282	blk.23.attn_output.weight	0x1d259dec0	0x480000
283	blk.23.attn_q.weight	0x1d2a1dec0	0x370000
284	blk.23.attn_q_norm.weight	0x1d2d8dec0	0x200
285	blk.23.attn_v.weight	0x1d2d8e0c0	0x88000
286	blk.23.ffn_down_exps.weight	0x1d2e160c0	0x8400000
287	blk.23.ffn_gate_exps.weight	0x1db2160c0	0x5280000
288	blk.23.ffn_gate_inp.weight	0x1e04960c0	0x100000
289	blk.23.ffn_norm.weight	0x1e05960c0	0x2000
290	blk.23.ffn_up_exps.weight	0x1e05980c0	0x5280000
291	blk.24.attn_k.weight	0x1e58180c0	0x90000
292	blk.24.attn_k_norm.weight	0x1e58a80c0	0x200
293	blk.24.attn_norm.weight	0x1e58a82c0	0x2000
294	blk.24.attn_output.weight	0x1e58aa2c0	0x480000
295	blk.24.attn_q.weight	0x1e5d2a2c0	0x480000
296	blk.24.attn_q_norm.weight	0x1e61aa2c0	0x200
297	blk.24.attn_v.weight	0x1e61aa4c0	0x90000
298	blk.24.ffn_down_exps.weight	0x1e623a4c0	0x8400000
299	blk.24.ffn_gate_exps.weight	0x1ee63a4c0	0x5280000
300	blk.24.ffn_gate_inp.weight	0x1f38ba4c0	0x100000
301	blk.24.ffn_norm.weight	0x1f39ba4c0	0x2000
302	blk.24.ffn_up_exps.weight	0x1f39bc4c0	0x5280000
303	blk.25.attn_k.weight	0x1f8c3c4c0	0x90000
304	blk.25.attn_k_norm.weight	0x1f8ccc4c0	0x200
305	blk.25.attn_norm.weight	0x1f8ccc6c0	0x2000
306	blk.25.attn_output.weight	0x1f8cce6c0	0x480000
307	blk.25.attn_q.weight	0x1f914e6c0	0x480000
308	blk.25.attn_q_norm.weight	0x1f95ce6c0	0x200
309	blk.25.attn_v.weight	0x1f95ce8c0	0x90000
310	blk.25.ffn_down_exps.weight	0x1f965e8c0	0x8400000
311	blk.25.ffn_gate_exps.weight	0x201a5e8c0	0x6c00000
312	blk.25.ffn_gate_inp.weight	0x20865e8c0	0x100000
313	blk.25.ffn_norm.weight	0x20875e8c0	0x2000
314	blk.25.ffn_up_exps.weight	0x2087608c0	0x6c00000
315	blk.26.attn_k.weight	0x20f3608c0	0x90000
316	blk.26.attn_k_norm.weight	0x20f3f08c0	0x200
317	blk.26.attn_norm.weight	0x20f3f0ac0	0x2000
318	blk.26.attn_output.weight	0x20f3f2ac0	0x480000
319	blk.26.attn_q.weight	0x20f872ac0	0x480000
320	blk.26.attn_q_norm.weight	0x20fcf2ac0	0x200
321	blk.26.attn_v.weight	0x20fcf2cc0	0x90000
322	blk.26.ffn_down_exps.weight	0x20fd82cc0	0x8400000
323	blk.26.ffn_gate_exps.weight	0x218182cc0	0x5280000
324	blk.26.ffn_gate_inp.weight	0x21d402cc0	0x100000
325	blk.26.ffn_norm.weight	0x21d502cc0	0x2000
326	blk.26.ffn_up_exps.weight	0x21d504cc0	0x5280000
327	blk.27.attn_k.weight	0x222784cc0	0x90000
328	blk.27.attn_k_norm.weight	0x222814cc0	0x200
329	blk.27.attn_norm.weight	0x222814ec0	0x2000
330	blk.27.attn_output.weight	0x222816ec0	0x480000
331	blk.27.attn_q.weight	0x222c96ec0	0x480000
332	blk.27.attn_q_norm.weight	0x223116ec0	0x200
333	blk.27.attn_v.weight	0x2231170c0	0x90000
334	blk.27.ffn_down_exps.weight	0x2231a70c0	0x8400000
335	blk.27.ffn_gate_exps.weight	0x22b5a70c0	0x6c00000
336	blk.27.ffn_gate_inp.weight	0x2321a70c0	0x100000
337	blk.27.ffn_norm.weight	0x2322a70c0	0x2000
338	blk.27.ffn_up_exps.weight	0x2322a90c0	0x6c00000
339	blk.28.attn_k.weight	0x238ea90c0	0x90000
340	blk.28.attn_k_norm.weight	0x238f390c0	0x200
341	blk.28.attn_norm.weight	0x238f392c0	0x2000
342	blk.28.attn_output.weight	0x238f3b2c0	0x480000
343	blk.28.attn_q.weight	0x2393bb2c0	0x480000
344	blk.28.attn_q_norm.weight	0x23983b2c0	0x200
345	blk.28.attn_v.weight	0x23983b4c0	0x90000
346	blk.28.ffn_down_exps.weight	0x2398cb4c0	0x8400000
347	blk.28.ffn_gate_exps.weight	0x241ccb4c0	0x6c00000
348	blk.28.ffn_gate_inp.weight	0x2488cb4c0	0x100000
349	blk.28.ffn_norm.weight	0x2489cb4c0	0x2000
350	blk.28.ffn_up_exps.weight	0x2489cd4c0	0x6c00000
351	blk.29.attn_k.weight	0x24f5cd4c0	0x90000
352	blk.29.attn_k_norm.weight	0x24f65d4c0	0x200
353	blk.29.attn_norm.weight	0x24f65d6c0	0x2000
354	blk.29.attn_output.weight	0x24f65f6c0	0x480000
355	blk.29.attn_q.weight	0x24fadf6c0	0x480000
356	blk.29.attn_q_norm.weight	0x24ff5f6c0	0x200
357	blk.29.attn_v.weight	0x24ff5f8c0	0x90000
358	blk.29.ffn_down_exps.weight	0x24ffef8c0	0x8400000
359	blk.29.ffn_gate_exps.weight	0x2583ef8c0	0x6c00000
360	blk.29.ffn_gate_inp.weight	0x25efef8c0	0x100000
361	blk.29.ffn_norm.weight	0x25f0ef8c0	0x2000
362	blk.29.ffn_up_exps.weight	0x25f0f18c0	0x6c00000
363	blk.30.attn_k.weight	0x265cf18c0	0x90000
364	blk.30.attn_k_norm.weight	0x265d818c0	0x200
365	blk.30.attn_norm.weight	0x265d81ac0	0x2000
366	blk.30.attn_output.weight	0x265d83ac0	0x480000
367	blk.30.attn_q.weight	0x266203ac0	0x480000
368	blk.30.attn_q_norm.weight	0x266683ac0	0x200
369	blk.30.attn_v.weight	0x266683cc0	0x90000
370	blk.30.ffn_down_exps.weight	0x266713cc0	0x8400000
371	blk.30.ffn_gate_exps.weight	0x26eb13cc0	0x6c00000
372	blk.30.ffn_gate_inp.weight	0x275713cc0	0x100000
373	blk.30.ffn_norm.weight	0x275813cc0	0x2000
374	blk.30.ffn_up_exps.weight	0x275815cc0	0x6c00000
375	blk.31.attn_k.weight	0x27c415cc0	0x90000
376	blk.31.attn_k_norm.weight	0x27c4a5cc0	0x200
377	blk.31.attn_norm.weight	0x27c4a5ec0	0x2000
378	blk.31.attn_output.weight	0x27c4a7ec0	0x480000
379	blk.31.attn_q.weight	0x27c927ec0	0x480000
380	blk.31.attn_q_norm.weight	0x27cda7ec0	0x200
381	blk.31.attn_v.weight	0x27cda80c0	0x90000
382	blk.31.ffn_down_exps.weight	0x27ce380c0	0x8400000
383	blk.31.ffn_gate_exps.weight	0x2852380c0	0x6c00000
384	blk.31.ffn_gate_inp.weight	0x28be380c0	0x100000
385	blk.31.ffn_norm.weight	0x28bf380c0	0x2000
386	blk.31.ffn_up_exps.weight	0x28bf3a0c0	0x6c00000
387	blk.32.attn_k.weight	0x292b3a0c0	0x90000
388	blk.32.attn_k_norm.weight	0x292bca0c0	0x200
389	blk.32.attn_norm.weight	0x292bca2c0	0x2000
390	blk.32.attn_output.weight	0x292bcc2c0	0x480000
391	blk.32.attn_q.weight	0x29304c2c0	0x480000
392	blk.32.attn_q_norm.weight	0x2934cc2c0	0x200
393	blk.32.attn_v.weight	0x2934cc4c0	0x90000
394	blk.32.ffn_down_exps.weight	0x29355c4c0	0x8400000
395	blk.32.ffn_gate_exps.weight	0x29b95c4c0	0x6c00000
396	blk.32.ffn_gate_inp.weight	0x2a255c4c0	0x100000
397	blk.32.ffn_norm.weight	0x2a265c4c0	0x2000
398	blk.32.ffn_up_exps.weight	0x2a265e4c0	0x6c00000
399	blk.33.attn_k.weight	0x2a925e4c0	0x90000
400	blk.33.attn_k_norm.weight	0x2a92ee4c0	0x200
401	blk.33.attn_norm.weight	0x2a92ee6c0	0x2000
402	blk.33.attn_output.weight	0x2a92f06c0	0x480000
403	blk.33.attn_q.weight	0x2a97706c0	0x480000
404	blk.33.attn_q_norm.weight	0x2a9bf06c0	0x200
405	blk.33.attn_v.weight	0x2a9bf08c0	0x90000
406	blk.33.ffn_down_exps.weight	0x2a9c808c0	0x8400000
407	blk.33.ffn_gate_exps.weight	0x2b20808c0	0x6c00000
408	blk.33.ffn_gate_inp.weight	0x2b8c808c0	0x100000
409	blk.33.ffn_norm.weight	0x2b8d808c0	0x2000
410	blk.33.ffn_up_exps.weight	0x2b8d828c0	0x6c00000
411	blk.34.attn_k.weight	0x2bf9828c0	0x90000
412	blk.34.attn_k_norm.weight	0x2bfa128c0	0x200
413	blk.34.attn_norm.weight	0x2bfa12ac0	0x2000
414	blk.34.attn_output.weight	0x2bfa14ac0	0x480000
415	blk.34.attn_q.weight	0x2bfe94ac0	0x480000
416	blk.34.attn_q_norm.weight	0x2c0314ac0	0x200
417	blk.34.attn_v.weight	0x2c0314cc0	0x90000
418	blk.34.ffn_down_exps.weight	0x2c03a4cc0	0x8400000
419	blk.34.ffn_gate_exps.weight	0x2c87a4cc0	0x6c00000
420	blk.34.ffn_gate_inp.weight	0x2cf3a4cc0	0x100000
421	blk.34.ffn_norm.weight	0x2cf4a4cc0	0x2000
422	blk.34.ffn_up_exps.weight	0x2cf4a6cc0	0x6c00000
423	blk.35.attn_k.weight	0x2d60a6cc0	0x90000
424	blk.35.attn_k_norm.weight	0x2d6136cc0	0x200
425	blk.35.attn_norm.weight	0x2d6136ec0	0x2000
426	blk.35.attn_output.weight	0x2d6138ec0	0x480000
427	blk.35.attn_q.weight	0x2d65b8ec0	0x480000
428	blk.35.attn_q_norm.weight	0x2d6a38ec0	0x200
429	blk.35.attn_v.weight	0x2d6a390c0	0x90000
430	blk.35.ffn_down_exps.weight	0x2d6ac90c0	0x8400000
431	blk.35.ffn_gate_exps.weight	0x2deec90c0	0x6c00000
432	blk.35.ffn_gate_inp.weight	0x2e5ac90c0	0x100000
433	blk.35.ffn_norm.weight	0x2e5bc90c0	0x2000
434	blk.35.ffn_up_exps.weight	0x2e5bcb0c0	0x6c00000
435	blk.36.attn_k.weight	0x2ec7cb0c0	0x90000
436	blk.36.attn_k_norm.weight	0x2ec85b0c0	0x200
437	blk.36.attn_norm.weight	0x2ec85b2c0	0x2000
438	blk.36.attn_output.weight	0x2ec85d2c0	0x480000
439	blk.36.attn_q.weight	0x2eccdd2c0	0x480000
440	blk.36.attn_q_norm.weight	0x2ed15d2c0	0x200
441	blk.36.attn_v.weight	0x2ed15d4c0	0x90000
442	blk.36.ffn_down_exps.weight	0x2ed1ed4c0	0x8400000
443	blk.36.ffn_gate_exps.weight	0x2f55ed4c0	0x6c00000
444	blk.36.ffn_gate_inp.weight	0x2fc1ed4c0	0x100000
445	blk.36.ffn_norm.weight	0x2fc2ed4c0	0x2000
446	blk.36.ffn_up_exps.weight	0x2fc2ef4c0	0x6c00000
447	blk.37.attn_k.weight	0x302eef4c0	0x90000
448	blk.37.attn_k_norm.weight	0x302f7f4c0	0x200
449	blk.37.attn_norm.weight	0x302f7f6c0	0x2000
450	blk.37.attn_output.weight	0x302f816c0	0x480000
451	blk.37.attn_q.weight	0x3034016c0	0x480000
452	blk.37.attn_q_norm.weight	0x3038816c0	0x200
453	blk.37.attn_v.weight	0x3038818c0	0x90000
454	blk.37.ffn_down_exps.weight	0x3039118c0	0x8400000
455	blk.37.ffn_gate_exps.weight	0x30bd118c0	0x6c00000
456	blk.37.ffn_gate_inp.weight	0x3129118c0	0x100000
457	blk.37.ffn_norm.weight	0x312a118c0	0x2000
458	blk.37.ffn_up_exps.weight	0x312a138c0	0x6c00000
459	blk.38.attn_k.weight	0x3196138c0	0x90000
460	blk.38.attn_k_norm.weight	0x3196a38c0	0x200
461	blk.38.attn_norm.weight	0x3196a3ac0	0x2000
462	blk.38.attn_output.weight	0x3196a5ac0	0x480000
463	blk.38.attn_q.weight	0x319b25ac0	0x480000
464	blk.38.attn_q_norm.weight	0x319fa5ac0	0x200
465	blk.38.attn_v.weight	0x319fa5cc0	0x90000
466	blk.38.ffn_down_exps.weight	0x31a035cc0	0x8400000
467	blk.38.ffn_gate_exps.weight	0x322435cc0	0x6c00000
468	blk.38.ffn_gate_inp.weight	0x329035cc0	0x100000
469	blk.38.ffn_norm.weight	0x329135cc0	0x2000
470	blk.38.ffn_up_exps.weight	0x329137cc0	0x6c00000
471	blk.39.attn_k.weight	0x32fd37cc0	0x90000
472	blk.39.attn_k_norm.weight	0x32fdc7cc0	0x200
473	blk.39.attn_norm.weight	0x32fdc7ec0	0x2000
474	blk.39.attn_output.weight	0x32fdc9ec0	0x480000
475	blk.39.attn_q.weight	0x330249ec0	0x480000
476	blk.39.attn_q_norm.weight	0x3306c9ec0	0x200
477	blk.39.attn_v.weight	0x3306ca0c0	0x90000
478	blk.39.ffn_down_exps.weight	0x33075a0c0	0x8400000
479	blk.39.ffn_gate_exps.weight	0x338b5a0c0	0x6c00000
480	blk.39.ffn_gate_inp.weight	0x33f75a0c0	0x100000
481	blk.39.ffn_norm.weight	0x33f85a0c0	0x2000
482	blk.39.ffn_up_exps.weight	0x33f85c0c0	0x6c00000
483	blk.40.attn_k.weight	0x34645c0c0	0x90000
484	blk.40.attn_k_norm.weight	0x3464ec0c0	0x200
485	blk.40.attn_norm.weight	0x3464ec2c0	0x2000
486	blk.40.attn_output.weight	0x3464ee2c0	0x480000
487	blk.40.attn_q.weight	0x34696e2c0	0x480000
488	blk.40.attn_q_norm.weight	0x346dee2c0	0x200
489	blk.40.attn_v.weight	0x346dee4c0	0x90000
490	blk.40.ffn_down_exps.weight	0x346e7e4c0	0x8400000
491	blk.40.ffn_gate_exps.weight	0x34f27e4c0	0x6c00000
492	blk.40.ffn_gate_inp.weight	0x355e7e4c0	0x100000
493	blk.40.ffn_norm.weight	0x355f7e4c0	0x2000
494	blk.40.ffn_up_exps.weight	0x355f804c0	0x6c00000
495	blk.41.attn_k.weight	0x35cb804c0	0x90000
496	blk.41.attn_k_norm.weight	0x35cc104c0	0x200
497	blk.41.attn_norm.weight	0x35cc106c0	0x2000
498	blk.41.attn_output.weight	0x35cc126c0	0x480000
499	blk.41.attn_q.weight	0x35d0926c0	0x480000
500	blk.41.attn_q_norm.weight	0x35d5126c0	0x200
501	blk.41.attn_v.weight	0x35d5128c0	0x90000
502	blk.41.ffn_down_exps.weight	0x35d5a28c0	0x8400000
503	blk.41.ffn_gate_exps.weight	0x3659a28c0	0x6c00000
504	blk.41.ffn_gate_inp.weight	0x36c5a28c0	0x100000
505	blk.41.ffn_norm.weight	0x36c6a28c0	0x2000
506	blk.41.ffn_up_exps.weight	0x36c6a48c0	0x6c00000
507	blk.42.attn_k.weight	0x3732a48c0	0x90000
508	blk.42.attn_k_norm.weight	0x3733348c0	0x200
509	blk.42.attn_norm.weight	0x373334ac0	0x2000
510	blk.42.attn_output.weight	0x373336ac0	0x480000
511	blk.42.attn_q.weight	0x3737b6ac0	0x480000
512	blk.42.attn_q_norm.weight	0x373c36ac0	0x200
513	blk.42.attn_v.weight	0x373c36cc0	0x90000
514	blk.42.ffn_down_exps.weight	0x373cc6cc0	0x8400000
515	blk.42.ffn_gate_exps.weight	0x37c0c6cc0	0x6c00000
516	blk.42.ffn_gate_inp.weight	0x382cc6cc0	0x100000
517	blk.42.ffn_norm.weight	0x382dc6cc0	0x2000
518	blk.42.ffn_up_exps.weight	0x382dc8cc0	0x6c00000
519	blk.43.attn_k.weight	0x3899c8cc0	0x90000
520	blk.43.attn_k_norm.weight	0x389a58cc0	0x200
521	blk.43.attn_norm.weight	0x389a58ec0	0x2000
522	blk.43.attn_output.weight	0x389a5aec0	0x480000
523	blk.43.attn_q.weight	0x389edaec0	0x480000
524	blk.43.attn_q_norm.weight	0x38a35aec0	0x200
525	blk.43.attn_v.weight	0x38a35b0c0	0x90000
526	blk.43.ffn_down_exps.weight	0x38a3eb0c0	0x8400000
527	blk.43.ffn_gate_exps.weight	0x3927eb0c0	0x6c00000
528	blk.43.ffn_gate_inp.weight	0x3993eb0c0	0x100000
529	blk.43.ffn_norm.weight	0x3994eb0c0	0x2000
530	blk.43.ffn_up_exps.weight	0x3994ed0c0	0x6c00000
531	blk.44.attn_k.weight	0x3a00ed0c0	0x90000
532	blk.44.attn_k_norm.weight	0x3a017d0c0	0x200
533	blk.44.attn_norm.weight	0x3a017d2c0	0x2000
534	blk.44.attn_output.weight	0x3a017f2c0	0x480000
535	blk.44.attn_q.weight	0x3a05ff2c0	0x480000
536	blk.44.attn_q_norm.weight	0x3a0a7f2c0	0x200
537	blk.44.attn_v.weight	0x3a0a7f4c0	0x90000
538	blk.44.ffn_down_exps.weight	0x3a0b0f4c0	0x8400000
539	blk.44.ffn_gate_exps.weight	0x3a8f0f4c0	0x6c00000
540	blk.44.ffn_gate_inp.weight	0x3afb0f4c0	0x100000
541	blk.44.ffn_norm.weight	0x3afc0f4c0	0x2000
542	blk.44.ffn_up_exps.weight	0x3afc114c0	0x6c00000
543	blk.45.attn_k.weight	0x3b68114c0	0x90000
544	blk.45.attn_k_norm.weight	0x3b68a14c0	0x200
545	blk.45.attn_norm.weight	0x3b68a16c0	0x2000
546	blk.45.attn_output.weight	0x3b68a36c0	0x480000
547	blk.45.attn_q.weight	0x3b6d236c0	0x480000
548	blk.45.attn_q_norm.weight	0x3b71a36c0	0x200
549	blk.45.attn_v.weight	0x3b71a38c0	0x90000
550	blk.45.ffn_down_exps.weight	0x3b72338c0	0x8400000
551	blk.45.ffn_gate_exps.weight	0x3bf6338c0	0x6c00000
552	blk.45.ffn_gate_inp.weight	0x3c62338c0	0x100000
553	blk.45.ffn_norm.weight	0x3c63338c0	0x2000
554	blk.45.ffn_up_exps.weight	0x3c63358c0	0x6c00000
555	blk.46.attn_k.weight	0x3ccf358c0	0x90000
556	blk.46.attn_k_norm.weight	0x3ccfc58c0	0x200
557	blk.46.attn_norm.weight	0x3ccfc5ac0	0x2000
558	blk.46.attn_output.weight	0x3ccfc7ac0	0x480000
559	blk.46.attn_q.weight	0x3cd447ac0	0x480000
560	blk.46.attn_q_norm.weight	0x3cd8c7ac0	0x200
561	blk.46.attn_v.weight	0x3cd8c7cc0	0x90000
562	blk.46.ffn_down_exps.weight	0x3cd957cc0	0x8400000
563	blk.46.ffn_gate_exps.weight	0x3d5d57cc0	0x6c00000
564	blk.46.ffn_gate_inp.weight	0x3dc957cc0	0x100000
565	blk.46.ffn_norm.weight	0x3dca57cc0	0x2000
566	blk.46.ffn_up_exps.weight	0x3dca59cc0	0x6c00000
567	blk.47.attn_k.weight	0x3e3659cc0	0x90000
568	blk.47.attn_k_norm.weight	0x3e36e9cc0	0x200
569	blk.47.attn_norm.weight	0x3e36e9ec0	0x2000
570	blk.47.attn_output.weight	0x3e36ebec0	0x480000
571	blk.47.attn_q.weight	0x3e3b6bec0	0x480000
572	blk.47.attn_q_norm.weight	0x3e3febec0	0x200
573	blk.47.attn_v.weight	0x3e3fec0c0	0x90000
574	blk.47.ffn_down_exps.weight	0x3e407c0c0	0x8400000
575	blk.47.ffn_gate_exps.weight	0x3ec47c0c0	0x6c00000
576	blk.47.ffn_gate_inp.weight	0x3f307c0c0	0x100000
577	blk.47.ffn_norm.weight	0x3f317c0c0	0x2000
578	blk.47.ffn_up_exps.weight	0x3f317e0c0	0x6c00000

Base Tensor Group : ~622M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
0	output.weight	Output (W)	(~311M) 311164928	2048 x 151936 x 1 x 1	IQ4_NL
1	output_norm.weight	Output Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
2	token_embd.weight	Token Embedding (W)	(~311M) 311164928	2048 x 151936 x 1 x 1	IQ3_S

Total elements in base: (~622M) 622331904
Percentage of total elements: 2.04%

Block 0 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
3	blk.0.attn_k.weight	Block 0 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ3_S
4	blk.0.attn_k_norm.weight	Block 0 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
5	blk.0.attn_norm.weight	Block 0 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
6	blk.0.attn_output.weight	Block 0 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
7	blk.0.attn_q.weight	Block 0 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ3_S
8	blk.0.attn_q_norm.weight	Block 0 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
9	blk.0.attn_v.weight	Block 0 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_XS
10	blk.0.ffn_down_exps.weight	Block 0 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
11	blk.0.ffn_gate_exps.weight	Block 0 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S
12	blk.0.ffn_gate_inp.weight	Block 0 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
13	blk.0.ffn_norm.weight	Block 0 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
14	blk.0.ffn_up_exps.weight	Block 0 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S

Total elements in blk.0: (~623M) 623120640
Percentage of total elements: 2.04%

Block 1 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
15	blk.1.attn_k.weight	Block 1 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ3_S
16	blk.1.attn_k_norm.weight	Block 1 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
17	blk.1.attn_norm.weight	Block 1 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
18	blk.1.attn_output.weight	Block 1 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
19	blk.1.attn_q.weight	Block 1 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ3_S
20	blk.1.attn_q_norm.weight	Block 1 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
21	blk.1.attn_v.weight	Block 1 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_XS
22	blk.1.ffn_down_exps.weight	Block 1 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
23	blk.1.ffn_gate_exps.weight	Block 1 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S
24	blk.1.ffn_gate_inp.weight	Block 1 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
25	blk.1.ffn_norm.weight	Block 1 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
26	blk.1.ffn_up_exps.weight	Block 1 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S

Total elements in blk.1: (~623M) 623120640
Percentage of total elements: 2.04%

Block 2 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
27	blk.2.attn_k.weight	Block 2 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ3_S
28	blk.2.attn_k_norm.weight	Block 2 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
29	blk.2.attn_norm.weight	Block 2 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
30	blk.2.attn_output.weight	Block 2 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
31	blk.2.attn_q.weight	Block 2 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ3_S
32	blk.2.attn_q_norm.weight	Block 2 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
33	blk.2.attn_v.weight	Block 2 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_XS
34	blk.2.ffn_down_exps.weight	Block 2 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
35	blk.2.ffn_gate_exps.weight	Block 2 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S
36	blk.2.ffn_gate_inp.weight	Block 2 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
37	blk.2.ffn_norm.weight	Block 2 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
38	blk.2.ffn_up_exps.weight	Block 2 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S

Total elements in blk.2: (~623M) 623120640
Percentage of total elements: 2.04%

Block 3 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
39	blk.3.attn_k.weight	Block 3 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ3_S
40	blk.3.attn_k_norm.weight	Block 3 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
41	blk.3.attn_norm.weight	Block 3 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
42	blk.3.attn_output.weight	Block 3 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
43	blk.3.attn_q.weight	Block 3 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ3_S
44	blk.3.attn_q_norm.weight	Block 3 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
45	blk.3.attn_v.weight	Block 3 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_XS
46	blk.3.ffn_down_exps.weight	Block 3 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
47	blk.3.ffn_gate_exps.weight	Block 3 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S
48	blk.3.ffn_gate_inp.weight	Block 3 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
49	blk.3.ffn_norm.weight	Block 3 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
50	blk.3.ffn_up_exps.weight	Block 3 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S

Total elements in blk.3: (~623M) 623120640
Percentage of total elements: 2.04%

Block 4 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
51	blk.4.attn_k.weight	Block 4 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ3_S
52	blk.4.attn_k_norm.weight	Block 4 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
53	blk.4.attn_norm.weight	Block 4 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
54	blk.4.attn_output.weight	Block 4 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
55	blk.4.attn_q.weight	Block 4 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ3_S
56	blk.4.attn_q_norm.weight	Block 4 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
57	blk.4.attn_v.weight	Block 4 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_XS
58	blk.4.ffn_down_exps.weight	Block 4 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
59	blk.4.ffn_gate_exps.weight	Block 4 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S
60	blk.4.ffn_gate_inp.weight	Block 4 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
61	blk.4.ffn_norm.weight	Block 4 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
62	blk.4.ffn_up_exps.weight	Block 4 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S

Total elements in blk.4: (~623M) 623120640
Percentage of total elements: 2.04%

Block 5 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
63	blk.5.attn_k.weight	Block 5 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ3_S
64	blk.5.attn_k_norm.weight	Block 5 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
65	blk.5.attn_norm.weight	Block 5 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
66	blk.5.attn_output.weight	Block 5 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
67	blk.5.attn_q.weight	Block 5 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ3_S
68	blk.5.attn_q_norm.weight	Block 5 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
69	blk.5.attn_v.weight	Block 5 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_XS
70	blk.5.ffn_down_exps.weight	Block 5 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
71	blk.5.ffn_gate_exps.weight	Block 5 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S
72	blk.5.ffn_gate_inp.weight	Block 5 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
73	blk.5.ffn_norm.weight	Block 5 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
74	blk.5.ffn_up_exps.weight	Block 5 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S

Total elements in blk.5: (~623M) 623120640
Percentage of total elements: 2.04%

Block 6 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
75	blk.6.attn_k.weight	Block 6 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ3_S
76	blk.6.attn_k_norm.weight	Block 6 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
77	blk.6.attn_norm.weight	Block 6 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
78	blk.6.attn_output.weight	Block 6 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
79	blk.6.attn_q.weight	Block 6 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ3_S
80	blk.6.attn_q_norm.weight	Block 6 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
81	blk.6.attn_v.weight	Block 6 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_XS
82	blk.6.ffn_down_exps.weight	Block 6 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
83	blk.6.ffn_gate_exps.weight	Block 6 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S
84	blk.6.ffn_gate_inp.weight	Block 6 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
85	blk.6.ffn_norm.weight	Block 6 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
86	blk.6.ffn_up_exps.weight	Block 6 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S

Total elements in blk.6: (~623M) 623120640
Percentage of total elements: 2.04%

Block 7 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
87	blk.7.attn_k.weight	Block 7 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ3_S
88	blk.7.attn_k_norm.weight	Block 7 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
89	blk.7.attn_norm.weight	Block 7 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
90	blk.7.attn_output.weight	Block 7 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
91	blk.7.attn_q.weight	Block 7 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ3_S
92	blk.7.attn_q_norm.weight	Block 7 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
93	blk.7.attn_v.weight	Block 7 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_XS
94	blk.7.ffn_down_exps.weight	Block 7 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
95	blk.7.ffn_gate_exps.weight	Block 7 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S
96	blk.7.ffn_gate_inp.weight	Block 7 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
97	blk.7.ffn_norm.weight	Block 7 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
98	blk.7.ffn_up_exps.weight	Block 7 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S

Total elements in blk.7: (~623M) 623120640
Percentage of total elements: 2.04%

Block 8 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
99	blk.8.attn_k.weight	Block 8 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ3_S
100	blk.8.attn_k_norm.weight	Block 8 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
101	blk.8.attn_norm.weight	Block 8 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
102	blk.8.attn_output.weight	Block 8 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
103	blk.8.attn_q.weight	Block 8 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ3_S
104	blk.8.attn_q_norm.weight	Block 8 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
105	blk.8.attn_v.weight	Block 8 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_XS
106	blk.8.ffn_down_exps.weight	Block 8 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
107	blk.8.ffn_gate_exps.weight	Block 8 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S
108	blk.8.ffn_gate_inp.weight	Block 8 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
109	blk.8.ffn_norm.weight	Block 8 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
110	blk.8.ffn_up_exps.weight	Block 8 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S

Total elements in blk.8: (~623M) 623120640
Percentage of total elements: 2.04%

Block 9 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
111	blk.9.attn_k.weight	Block 9 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ3_S
112	blk.9.attn_k_norm.weight	Block 9 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
113	blk.9.attn_norm.weight	Block 9 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
114	blk.9.attn_output.weight	Block 9 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
115	blk.9.attn_q.weight	Block 9 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ3_S
116	blk.9.attn_q_norm.weight	Block 9 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
117	blk.9.attn_v.weight	Block 9 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_XS
118	blk.9.ffn_down_exps.weight	Block 9 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
119	blk.9.ffn_gate_exps.weight	Block 9 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S
120	blk.9.ffn_gate_inp.weight	Block 9 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
121	blk.9.ffn_norm.weight	Block 9 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
122	blk.9.ffn_up_exps.weight	Block 9 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S

Total elements in blk.9: (~623M) 623120640
Percentage of total elements: 2.04%

Block 10 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
123	blk.10.attn_k.weight	Block 10 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ3_S
124	blk.10.attn_k_norm.weight	Block 10 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
125	blk.10.attn_norm.weight	Block 10 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
126	blk.10.attn_output.weight	Block 10 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
127	blk.10.attn_q.weight	Block 10 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ3_S
128	blk.10.attn_q_norm.weight	Block 10 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
129	blk.10.attn_v.weight	Block 10 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_XS
130	blk.10.ffn_down_exps.weight	Block 10 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
131	blk.10.ffn_gate_exps.weight	Block 10 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S
132	blk.10.ffn_gate_inp.weight	Block 10 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
133	blk.10.ffn_norm.weight	Block 10 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
134	blk.10.ffn_up_exps.weight	Block 10 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S

Total elements in blk.10: (~623M) 623120640
Percentage of total elements: 2.04%

Block 11 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
135	blk.11.attn_k.weight	Block 11 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ3_S
136	blk.11.attn_k_norm.weight	Block 11 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
137	blk.11.attn_norm.weight	Block 11 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
138	blk.11.attn_output.weight	Block 11 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
139	blk.11.attn_q.weight	Block 11 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ3_S
140	blk.11.attn_q_norm.weight	Block 11 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
141	blk.11.attn_v.weight	Block 11 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_XS
142	blk.11.ffn_down_exps.weight	Block 11 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
143	blk.11.ffn_gate_exps.weight	Block 11 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S
144	blk.11.ffn_gate_inp.weight	Block 11 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
145	blk.11.ffn_norm.weight	Block 11 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
146	blk.11.ffn_up_exps.weight	Block 11 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S

Total elements in blk.11: (~623M) 623120640
Percentage of total elements: 2.04%

Block 12 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
147	blk.12.attn_k.weight	Block 12 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ3_S
148	blk.12.attn_k_norm.weight	Block 12 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
149	blk.12.attn_norm.weight	Block 12 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
150	blk.12.attn_output.weight	Block 12 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
151	blk.12.attn_q.weight	Block 12 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ3_S
152	blk.12.attn_q_norm.weight	Block 12 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
153	blk.12.attn_v.weight	Block 12 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_XS
154	blk.12.ffn_down_exps.weight	Block 12 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
155	blk.12.ffn_gate_exps.weight	Block 12 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S
156	blk.12.ffn_gate_inp.weight	Block 12 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
157	blk.12.ffn_norm.weight	Block 12 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
158	blk.12.ffn_up_exps.weight	Block 12 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S

Total elements in blk.12: (~623M) 623120640
Percentage of total elements: 2.04%

Block 13 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
159	blk.13.attn_k.weight	Block 13 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ3_S
160	blk.13.attn_k_norm.weight	Block 13 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
161	blk.13.attn_norm.weight	Block 13 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
162	blk.13.attn_output.weight	Block 13 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
163	blk.13.attn_q.weight	Block 13 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ3_S
164	blk.13.attn_q_norm.weight	Block 13 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
165	blk.13.attn_v.weight	Block 13 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_XS
166	blk.13.ffn_down_exps.weight	Block 13 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
167	blk.13.ffn_gate_exps.weight	Block 13 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL
168	blk.13.ffn_gate_inp.weight	Block 13 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
169	blk.13.ffn_norm.weight	Block 13 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
170	blk.13.ffn_up_exps.weight	Block 13 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL

Total elements in blk.13: (~623M) 623120640
Percentage of total elements: 2.04%

Block 14 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
171	blk.14.attn_k.weight	Block 14 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ3_S
172	blk.14.attn_k_norm.weight	Block 14 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
173	blk.14.attn_norm.weight	Block 14 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
174	blk.14.attn_output.weight	Block 14 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
175	blk.14.attn_q.weight	Block 14 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ3_S
176	blk.14.attn_q_norm.weight	Block 14 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
177	blk.14.attn_v.weight	Block 14 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_XS
178	blk.14.ffn_down_exps.weight	Block 14 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
179	blk.14.ffn_gate_exps.weight	Block 14 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S
180	blk.14.ffn_gate_inp.weight	Block 14 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
181	blk.14.ffn_norm.weight	Block 14 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
182	blk.14.ffn_up_exps.weight	Block 14 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S

Total elements in blk.14: (~623M) 623120640
Percentage of total elements: 2.04%

Block 15 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
183	blk.15.attn_k.weight	Block 15 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ3_S
184	blk.15.attn_k_norm.weight	Block 15 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
185	blk.15.attn_norm.weight	Block 15 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
186	blk.15.attn_output.weight	Block 15 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
187	blk.15.attn_q.weight	Block 15 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ3_S
188	blk.15.attn_q_norm.weight	Block 15 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
189	blk.15.attn_v.weight	Block 15 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_XS
190	blk.15.ffn_down_exps.weight	Block 15 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
191	blk.15.ffn_gate_exps.weight	Block 15 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL
192	blk.15.ffn_gate_inp.weight	Block 15 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
193	blk.15.ffn_norm.weight	Block 15 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
194	blk.15.ffn_up_exps.weight	Block 15 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL

Total elements in blk.15: (~623M) 623120640
Percentage of total elements: 2.04%

Block 16 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
195	blk.16.attn_k.weight	Block 16 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ3_S
196	blk.16.attn_k_norm.weight	Block 16 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
197	blk.16.attn_norm.weight	Block 16 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
198	blk.16.attn_output.weight	Block 16 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
199	blk.16.attn_q.weight	Block 16 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ3_S
200	blk.16.attn_q_norm.weight	Block 16 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
201	blk.16.attn_v.weight	Block 16 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_XS
202	blk.16.ffn_down_exps.weight	Block 16 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
203	blk.16.ffn_gate_exps.weight	Block 16 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S
204	blk.16.ffn_gate_inp.weight	Block 16 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
205	blk.16.ffn_norm.weight	Block 16 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
206	blk.16.ffn_up_exps.weight	Block 16 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S

Total elements in blk.16: (~623M) 623120640
Percentage of total elements: 2.04%

Block 17 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
207	blk.17.attn_k.weight	Block 17 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ3_S
208	blk.17.attn_k_norm.weight	Block 17 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
209	blk.17.attn_norm.weight	Block 17 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
210	blk.17.attn_output.weight	Block 17 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
211	blk.17.attn_q.weight	Block 17 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ3_S
212	blk.17.attn_q_norm.weight	Block 17 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
213	blk.17.attn_v.weight	Block 17 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_XS
214	blk.17.ffn_down_exps.weight	Block 17 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
215	blk.17.ffn_gate_exps.weight	Block 17 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S
216	blk.17.ffn_gate_inp.weight	Block 17 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
217	blk.17.ffn_norm.weight	Block 17 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
218	blk.17.ffn_up_exps.weight	Block 17 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S

Total elements in blk.17: (~623M) 623120640
Percentage of total elements: 2.04%

Block 18 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
219	blk.18.attn_k.weight	Block 18 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ3_S
220	blk.18.attn_k_norm.weight	Block 18 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
221	blk.18.attn_norm.weight	Block 18 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
222	blk.18.attn_output.weight	Block 18 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
223	blk.18.attn_q.weight	Block 18 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ3_S
224	blk.18.attn_q_norm.weight	Block 18 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
225	blk.18.attn_v.weight	Block 18 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_XS
226	blk.18.ffn_down_exps.weight	Block 18 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
227	blk.18.ffn_gate_exps.weight	Block 18 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S
228	blk.18.ffn_gate_inp.weight	Block 18 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
229	blk.18.ffn_norm.weight	Block 18 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
230	blk.18.ffn_up_exps.weight	Block 18 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S

Total elements in blk.18: (~623M) 623120640
Percentage of total elements: 2.04%

Block 19 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
231	blk.19.attn_k.weight	Block 19 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ3_S
232	blk.19.attn_k_norm.weight	Block 19 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
233	blk.19.attn_norm.weight	Block 19 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
234	blk.19.attn_output.weight	Block 19 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
235	blk.19.attn_q.weight	Block 19 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ3_S
236	blk.19.attn_q_norm.weight	Block 19 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
237	blk.19.attn_v.weight	Block 19 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_XS
238	blk.19.ffn_down_exps.weight	Block 19 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
239	blk.19.ffn_gate_exps.weight	Block 19 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S
240	blk.19.ffn_gate_inp.weight	Block 19 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
241	blk.19.ffn_norm.weight	Block 19 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
242	blk.19.ffn_up_exps.weight	Block 19 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S

Total elements in blk.19: (~623M) 623120640
Percentage of total elements: 2.04%

Block 20 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
243	blk.20.attn_k.weight	Block 20 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ3_S
244	blk.20.attn_k_norm.weight	Block 20 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
245	blk.20.attn_norm.weight	Block 20 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
246	blk.20.attn_output.weight	Block 20 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
247	blk.20.attn_q.weight	Block 20 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ3_S
248	blk.20.attn_q_norm.weight	Block 20 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
249	blk.20.attn_v.weight	Block 20 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_XS
250	blk.20.ffn_down_exps.weight	Block 20 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
251	blk.20.ffn_gate_exps.weight	Block 20 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S
252	blk.20.ffn_gate_inp.weight	Block 20 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
253	blk.20.ffn_norm.weight	Block 20 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
254	blk.20.ffn_up_exps.weight	Block 20 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S

Total elements in blk.20: (~623M) 623120640
Percentage of total elements: 2.04%

Block 21 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
255	blk.21.attn_k.weight	Block 21 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ3_S
256	blk.21.attn_k_norm.weight	Block 21 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
257	blk.21.attn_norm.weight	Block 21 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
258	blk.21.attn_output.weight	Block 21 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
259	blk.21.attn_q.weight	Block 21 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ3_S
260	blk.21.attn_q_norm.weight	Block 21 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
261	blk.21.attn_v.weight	Block 21 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_XS
262	blk.21.ffn_down_exps.weight	Block 21 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
263	blk.21.ffn_gate_exps.weight	Block 21 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S
264	blk.21.ffn_gate_inp.weight	Block 21 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
265	blk.21.ffn_norm.weight	Block 21 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
266	blk.21.ffn_up_exps.weight	Block 21 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S

Total elements in blk.21: (~623M) 623120640
Percentage of total elements: 2.04%

Block 22 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
267	blk.22.attn_k.weight	Block 22 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ3_S
268	blk.22.attn_k_norm.weight	Block 22 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
269	blk.22.attn_norm.weight	Block 22 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
270	blk.22.attn_output.weight	Block 22 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
271	blk.22.attn_q.weight	Block 22 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ3_S
272	blk.22.attn_q_norm.weight	Block 22 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
273	blk.22.attn_v.weight	Block 22 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_XS
274	blk.22.ffn_down_exps.weight	Block 22 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
275	blk.22.ffn_gate_exps.weight	Block 22 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S
276	blk.22.ffn_gate_inp.weight	Block 22 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
277	blk.22.ffn_norm.weight	Block 22 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
278	blk.22.ffn_up_exps.weight	Block 22 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S

Total elements in blk.22: (~623M) 623120640
Percentage of total elements: 2.04%

Block 23 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
279	blk.23.attn_k.weight	Block 23 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ3_S
280	blk.23.attn_k_norm.weight	Block 23 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
281	blk.23.attn_norm.weight	Block 23 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
282	blk.23.attn_output.weight	Block 23 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
283	blk.23.attn_q.weight	Block 23 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ3_S
284	blk.23.attn_q_norm.weight	Block 23 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
285	blk.23.attn_v.weight	Block 23 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_XS
286	blk.23.ffn_down_exps.weight	Block 23 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
287	blk.23.ffn_gate_exps.weight	Block 23 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S
288	blk.23.ffn_gate_inp.weight	Block 23 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
289	blk.23.ffn_norm.weight	Block 23 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
290	blk.23.ffn_up_exps.weight	Block 23 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S

Total elements in blk.23: (~623M) 623120640
Percentage of total elements: 2.04%

Block 24 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
291	blk.24.attn_k.weight	Block 24 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
292	blk.24.attn_k_norm.weight	Block 24 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
293	blk.24.attn_norm.weight	Block 24 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
294	blk.24.attn_output.weight	Block 24 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
295	blk.24.attn_q.weight	Block 24 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ4_NL
296	blk.24.attn_q_norm.weight	Block 24 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
297	blk.24.attn_v.weight	Block 24 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
298	blk.24.ffn_down_exps.weight	Block 24 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
299	blk.24.ffn_gate_exps.weight	Block 24 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S
300	blk.24.ffn_gate_inp.weight	Block 24 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
301	blk.24.ffn_norm.weight	Block 24 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
302	blk.24.ffn_up_exps.weight	Block 24 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S

Total elements in blk.24: (~623M) 623120640
Percentage of total elements: 2.04%

Block 25 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
303	blk.25.attn_k.weight	Block 25 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
304	blk.25.attn_k_norm.weight	Block 25 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
305	blk.25.attn_norm.weight	Block 25 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
306	blk.25.attn_output.weight	Block 25 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
307	blk.25.attn_q.weight	Block 25 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ4_NL
308	blk.25.attn_q_norm.weight	Block 25 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
309	blk.25.attn_v.weight	Block 25 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
310	blk.25.ffn_down_exps.weight	Block 25 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
311	blk.25.ffn_gate_exps.weight	Block 25 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL
312	blk.25.ffn_gate_inp.weight	Block 25 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
313	blk.25.ffn_norm.weight	Block 25 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
314	blk.25.ffn_up_exps.weight	Block 25 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL

Total elements in blk.25: (~623M) 623120640
Percentage of total elements: 2.04%

Block 26 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
315	blk.26.attn_k.weight	Block 26 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
316	blk.26.attn_k_norm.weight	Block 26 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
317	blk.26.attn_norm.weight	Block 26 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
318	blk.26.attn_output.weight	Block 26 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
319	blk.26.attn_q.weight	Block 26 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ4_NL
320	blk.26.attn_q_norm.weight	Block 26 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
321	blk.26.attn_v.weight	Block 26 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
322	blk.26.ffn_down_exps.weight	Block 26 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
323	blk.26.ffn_gate_exps.weight	Block 26 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S
324	blk.26.ffn_gate_inp.weight	Block 26 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
325	blk.26.ffn_norm.weight	Block 26 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
326	blk.26.ffn_up_exps.weight	Block 26 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ3_S

Total elements in blk.26: (~623M) 623120640
Percentage of total elements: 2.04%

Block 27 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
327	blk.27.attn_k.weight	Block 27 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
328	blk.27.attn_k_norm.weight	Block 27 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
329	blk.27.attn_norm.weight	Block 27 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
330	blk.27.attn_output.weight	Block 27 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
331	blk.27.attn_q.weight	Block 27 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ4_NL
332	blk.27.attn_q_norm.weight	Block 27 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
333	blk.27.attn_v.weight	Block 27 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
334	blk.27.ffn_down_exps.weight	Block 27 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
335	blk.27.ffn_gate_exps.weight	Block 27 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL
336	blk.27.ffn_gate_inp.weight	Block 27 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
337	blk.27.ffn_norm.weight	Block 27 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
338	blk.27.ffn_up_exps.weight	Block 27 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL

Total elements in blk.27: (~623M) 623120640
Percentage of total elements: 2.04%

Block 28 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
339	blk.28.attn_k.weight	Block 28 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
340	blk.28.attn_k_norm.weight	Block 28 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
341	blk.28.attn_norm.weight	Block 28 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
342	blk.28.attn_output.weight	Block 28 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
343	blk.28.attn_q.weight	Block 28 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ4_NL
344	blk.28.attn_q_norm.weight	Block 28 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
345	blk.28.attn_v.weight	Block 28 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
346	blk.28.ffn_down_exps.weight	Block 28 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
347	blk.28.ffn_gate_exps.weight	Block 28 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL
348	blk.28.ffn_gate_inp.weight	Block 28 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
349	blk.28.ffn_norm.weight	Block 28 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
350	blk.28.ffn_up_exps.weight	Block 28 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL

Total elements in blk.28: (~623M) 623120640
Percentage of total elements: 2.04%

Block 29 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
351	blk.29.attn_k.weight	Block 29 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
352	blk.29.attn_k_norm.weight	Block 29 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
353	blk.29.attn_norm.weight	Block 29 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
354	blk.29.attn_output.weight	Block 29 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
355	blk.29.attn_q.weight	Block 29 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ4_NL
356	blk.29.attn_q_norm.weight	Block 29 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
357	blk.29.attn_v.weight	Block 29 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
358	blk.29.ffn_down_exps.weight	Block 29 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
359	blk.29.ffn_gate_exps.weight	Block 29 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL
360	blk.29.ffn_gate_inp.weight	Block 29 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
361	blk.29.ffn_norm.weight	Block 29 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
362	blk.29.ffn_up_exps.weight	Block 29 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL

Total elements in blk.29: (~623M) 623120640
Percentage of total elements: 2.04%

Block 30 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
363	blk.30.attn_k.weight	Block 30 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
364	blk.30.attn_k_norm.weight	Block 30 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
365	blk.30.attn_norm.weight	Block 30 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
366	blk.30.attn_output.weight	Block 30 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
367	blk.30.attn_q.weight	Block 30 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ4_NL
368	blk.30.attn_q_norm.weight	Block 30 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
369	blk.30.attn_v.weight	Block 30 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
370	blk.30.ffn_down_exps.weight	Block 30 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
371	blk.30.ffn_gate_exps.weight	Block 30 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL
372	blk.30.ffn_gate_inp.weight	Block 30 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
373	blk.30.ffn_norm.weight	Block 30 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
374	blk.30.ffn_up_exps.weight	Block 30 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL

Total elements in blk.30: (~623M) 623120640
Percentage of total elements: 2.04%

Block 31 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
375	blk.31.attn_k.weight	Block 31 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
376	blk.31.attn_k_norm.weight	Block 31 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
377	blk.31.attn_norm.weight	Block 31 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
378	blk.31.attn_output.weight	Block 31 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
379	blk.31.attn_q.weight	Block 31 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ4_NL
380	blk.31.attn_q_norm.weight	Block 31 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
381	blk.31.attn_v.weight	Block 31 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
382	blk.31.ffn_down_exps.weight	Block 31 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
383	blk.31.ffn_gate_exps.weight	Block 31 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL
384	blk.31.ffn_gate_inp.weight	Block 31 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
385	blk.31.ffn_norm.weight	Block 31 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
386	blk.31.ffn_up_exps.weight	Block 31 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL

Total elements in blk.31: (~623M) 623120640
Percentage of total elements: 2.04%

Block 32 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
387	blk.32.attn_k.weight	Block 32 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
388	blk.32.attn_k_norm.weight	Block 32 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
389	blk.32.attn_norm.weight	Block 32 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
390	blk.32.attn_output.weight	Block 32 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
391	blk.32.attn_q.weight	Block 32 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ4_NL
392	blk.32.attn_q_norm.weight	Block 32 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
393	blk.32.attn_v.weight	Block 32 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
394	blk.32.ffn_down_exps.weight	Block 32 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
395	blk.32.ffn_gate_exps.weight	Block 32 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL
396	blk.32.ffn_gate_inp.weight	Block 32 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
397	blk.32.ffn_norm.weight	Block 32 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
398	blk.32.ffn_up_exps.weight	Block 32 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL

Total elements in blk.32: (~623M) 623120640
Percentage of total elements: 2.04%

Block 33 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
399	blk.33.attn_k.weight	Block 33 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
400	blk.33.attn_k_norm.weight	Block 33 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
401	blk.33.attn_norm.weight	Block 33 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
402	blk.33.attn_output.weight	Block 33 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
403	blk.33.attn_q.weight	Block 33 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ4_NL
404	blk.33.attn_q_norm.weight	Block 33 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
405	blk.33.attn_v.weight	Block 33 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
406	blk.33.ffn_down_exps.weight	Block 33 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
407	blk.33.ffn_gate_exps.weight	Block 33 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL
408	blk.33.ffn_gate_inp.weight	Block 33 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
409	blk.33.ffn_norm.weight	Block 33 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
410	blk.33.ffn_up_exps.weight	Block 33 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL

Total elements in blk.33: (~623M) 623120640
Percentage of total elements: 2.04%

Block 34 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
411	blk.34.attn_k.weight	Block 34 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
412	blk.34.attn_k_norm.weight	Block 34 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
413	blk.34.attn_norm.weight	Block 34 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
414	blk.34.attn_output.weight	Block 34 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
415	blk.34.attn_q.weight	Block 34 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ4_NL
416	blk.34.attn_q_norm.weight	Block 34 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
417	blk.34.attn_v.weight	Block 34 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
418	blk.34.ffn_down_exps.weight	Block 34 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
419	blk.34.ffn_gate_exps.weight	Block 34 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL
420	blk.34.ffn_gate_inp.weight	Block 34 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
421	blk.34.ffn_norm.weight	Block 34 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
422	blk.34.ffn_up_exps.weight	Block 34 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL

Total elements in blk.34: (~623M) 623120640
Percentage of total elements: 2.04%

Block 35 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
423	blk.35.attn_k.weight	Block 35 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
424	blk.35.attn_k_norm.weight	Block 35 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
425	blk.35.attn_norm.weight	Block 35 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
426	blk.35.attn_output.weight	Block 35 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
427	blk.35.attn_q.weight	Block 35 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ4_NL
428	blk.35.attn_q_norm.weight	Block 35 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
429	blk.35.attn_v.weight	Block 35 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
430	blk.35.ffn_down_exps.weight	Block 35 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
431	blk.35.ffn_gate_exps.weight	Block 35 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL
432	blk.35.ffn_gate_inp.weight	Block 35 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
433	blk.35.ffn_norm.weight	Block 35 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
434	blk.35.ffn_up_exps.weight	Block 35 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL

Total elements in blk.35: (~623M) 623120640
Percentage of total elements: 2.04%

Block 36 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
435	blk.36.attn_k.weight	Block 36 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
436	blk.36.attn_k_norm.weight	Block 36 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
437	blk.36.attn_norm.weight	Block 36 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
438	blk.36.attn_output.weight	Block 36 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
439	blk.36.attn_q.weight	Block 36 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ4_NL
440	blk.36.attn_q_norm.weight	Block 36 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
441	blk.36.attn_v.weight	Block 36 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
442	blk.36.ffn_down_exps.weight	Block 36 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
443	blk.36.ffn_gate_exps.weight	Block 36 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL
444	blk.36.ffn_gate_inp.weight	Block 36 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
445	blk.36.ffn_norm.weight	Block 36 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
446	blk.36.ffn_up_exps.weight	Block 36 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL

Total elements in blk.36: (~623M) 623120640
Percentage of total elements: 2.04%

Block 37 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
447	blk.37.attn_k.weight	Block 37 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
448	blk.37.attn_k_norm.weight	Block 37 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
449	blk.37.attn_norm.weight	Block 37 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
450	blk.37.attn_output.weight	Block 37 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
451	blk.37.attn_q.weight	Block 37 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ4_NL
452	blk.37.attn_q_norm.weight	Block 37 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
453	blk.37.attn_v.weight	Block 37 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
454	blk.37.ffn_down_exps.weight	Block 37 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
455	blk.37.ffn_gate_exps.weight	Block 37 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL
456	blk.37.ffn_gate_inp.weight	Block 37 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
457	blk.37.ffn_norm.weight	Block 37 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
458	blk.37.ffn_up_exps.weight	Block 37 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL

Total elements in blk.37: (~623M) 623120640
Percentage of total elements: 2.04%

Block 38 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
459	blk.38.attn_k.weight	Block 38 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
460	blk.38.attn_k_norm.weight	Block 38 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
461	blk.38.attn_norm.weight	Block 38 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
462	blk.38.attn_output.weight	Block 38 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
463	blk.38.attn_q.weight	Block 38 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ4_NL
464	blk.38.attn_q_norm.weight	Block 38 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
465	blk.38.attn_v.weight	Block 38 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
466	blk.38.ffn_down_exps.weight	Block 38 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
467	blk.38.ffn_gate_exps.weight	Block 38 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL
468	blk.38.ffn_gate_inp.weight	Block 38 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
469	blk.38.ffn_norm.weight	Block 38 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
470	blk.38.ffn_up_exps.weight	Block 38 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL

Total elements in blk.38: (~623M) 623120640
Percentage of total elements: 2.04%

Block 39 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
471	blk.39.attn_k.weight	Block 39 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
472	blk.39.attn_k_norm.weight	Block 39 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
473	blk.39.attn_norm.weight	Block 39 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
474	blk.39.attn_output.weight	Block 39 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
475	blk.39.attn_q.weight	Block 39 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ4_NL
476	blk.39.attn_q_norm.weight	Block 39 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
477	blk.39.attn_v.weight	Block 39 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
478	blk.39.ffn_down_exps.weight	Block 39 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
479	blk.39.ffn_gate_exps.weight	Block 39 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL
480	blk.39.ffn_gate_inp.weight	Block 39 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
481	blk.39.ffn_norm.weight	Block 39 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
482	blk.39.ffn_up_exps.weight	Block 39 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL

Total elements in blk.39: (~623M) 623120640
Percentage of total elements: 2.04%

Block 40 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
483	blk.40.attn_k.weight	Block 40 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
484	blk.40.attn_k_norm.weight	Block 40 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
485	blk.40.attn_norm.weight	Block 40 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
486	blk.40.attn_output.weight	Block 40 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
487	blk.40.attn_q.weight	Block 40 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ4_NL
488	blk.40.attn_q_norm.weight	Block 40 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
489	blk.40.attn_v.weight	Block 40 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
490	blk.40.ffn_down_exps.weight	Block 40 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
491	blk.40.ffn_gate_exps.weight	Block 40 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL
492	blk.40.ffn_gate_inp.weight	Block 40 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
493	blk.40.ffn_norm.weight	Block 40 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
494	blk.40.ffn_up_exps.weight	Block 40 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL

Total elements in blk.40: (~623M) 623120640
Percentage of total elements: 2.04%

Block 41 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
495	blk.41.attn_k.weight	Block 41 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
496	blk.41.attn_k_norm.weight	Block 41 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
497	blk.41.attn_norm.weight	Block 41 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
498	blk.41.attn_output.weight	Block 41 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
499	blk.41.attn_q.weight	Block 41 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ4_NL
500	blk.41.attn_q_norm.weight	Block 41 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
501	blk.41.attn_v.weight	Block 41 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
502	blk.41.ffn_down_exps.weight	Block 41 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
503	blk.41.ffn_gate_exps.weight	Block 41 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL
504	blk.41.ffn_gate_inp.weight	Block 41 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
505	blk.41.ffn_norm.weight	Block 41 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
506	blk.41.ffn_up_exps.weight	Block 41 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL

Total elements in blk.41: (~623M) 623120640
Percentage of total elements: 2.04%

Block 42 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
507	blk.42.attn_k.weight	Block 42 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
508	blk.42.attn_k_norm.weight	Block 42 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
509	blk.42.attn_norm.weight	Block 42 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
510	blk.42.attn_output.weight	Block 42 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
511	blk.42.attn_q.weight	Block 42 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ4_NL
512	blk.42.attn_q_norm.weight	Block 42 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
513	blk.42.attn_v.weight	Block 42 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
514	blk.42.ffn_down_exps.weight	Block 42 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
515	blk.42.ffn_gate_exps.weight	Block 42 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL
516	blk.42.ffn_gate_inp.weight	Block 42 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
517	blk.42.ffn_norm.weight	Block 42 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
518	blk.42.ffn_up_exps.weight	Block 42 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL

Total elements in blk.42: (~623M) 623120640
Percentage of total elements: 2.04%

Block 43 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
519	blk.43.attn_k.weight	Block 43 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
520	blk.43.attn_k_norm.weight	Block 43 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
521	blk.43.attn_norm.weight	Block 43 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
522	blk.43.attn_output.weight	Block 43 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
523	blk.43.attn_q.weight	Block 43 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ4_NL
524	blk.43.attn_q_norm.weight	Block 43 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
525	blk.43.attn_v.weight	Block 43 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
526	blk.43.ffn_down_exps.weight	Block 43 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
527	blk.43.ffn_gate_exps.weight	Block 43 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL
528	blk.43.ffn_gate_inp.weight	Block 43 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
529	blk.43.ffn_norm.weight	Block 43 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
530	blk.43.ffn_up_exps.weight	Block 43 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL

Total elements in blk.43: (~623M) 623120640
Percentage of total elements: 2.04%

Block 44 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
531	blk.44.attn_k.weight	Block 44 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
532	blk.44.attn_k_norm.weight	Block 44 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
533	blk.44.attn_norm.weight	Block 44 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
534	blk.44.attn_output.weight	Block 44 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
535	blk.44.attn_q.weight	Block 44 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ4_NL
536	blk.44.attn_q_norm.weight	Block 44 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
537	blk.44.attn_v.weight	Block 44 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
538	blk.44.ffn_down_exps.weight	Block 44 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
539	blk.44.ffn_gate_exps.weight	Block 44 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL
540	blk.44.ffn_gate_inp.weight	Block 44 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
541	blk.44.ffn_norm.weight	Block 44 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
542	blk.44.ffn_up_exps.weight	Block 44 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL

Total elements in blk.44: (~623M) 623120640
Percentage of total elements: 2.04%

Block 45 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
543	blk.45.attn_k.weight	Block 45 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
544	blk.45.attn_k_norm.weight	Block 45 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
545	blk.45.attn_norm.weight	Block 45 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
546	blk.45.attn_output.weight	Block 45 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
547	blk.45.attn_q.weight	Block 45 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ4_NL
548	blk.45.attn_q_norm.weight	Block 45 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
549	blk.45.attn_v.weight	Block 45 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
550	blk.45.ffn_down_exps.weight	Block 45 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
551	blk.45.ffn_gate_exps.weight	Block 45 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL
552	blk.45.ffn_gate_inp.weight	Block 45 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
553	blk.45.ffn_norm.weight	Block 45 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
554	blk.45.ffn_up_exps.weight	Block 45 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL

Total elements in blk.45: (~623M) 623120640
Percentage of total elements: 2.04%

Block 46 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
555	blk.46.attn_k.weight	Block 46 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
556	blk.46.attn_k_norm.weight	Block 46 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
557	blk.46.attn_norm.weight	Block 46 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
558	blk.46.attn_output.weight	Block 46 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
559	blk.46.attn_q.weight	Block 46 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ4_NL
560	blk.46.attn_q_norm.weight	Block 46 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
561	blk.46.attn_v.weight	Block 46 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
562	blk.46.ffn_down_exps.weight	Block 46 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
563	blk.46.ffn_gate_exps.weight	Block 46 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL
564	blk.46.ffn_gate_inp.weight	Block 46 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
565	blk.46.ffn_norm.weight	Block 46 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
566	blk.46.ffn_up_exps.weight	Block 46 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL

Total elements in blk.46: (~623M) 623120640
Percentage of total elements: 2.04%

Block 47 Tensor Group : ~623M Elements

T_ID	Tensor Layer Name	Human Friendly Tensor Layer Name	Elements	Shape	Type
567	blk.47.attn_k.weight	Block 47 Attention Key (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
568	blk.47.attn_k_norm.weight	Block 47 Attn_K_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
569	blk.47.attn_norm.weight	Block 47 Attention Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
570	blk.47.attn_output.weight	Block 47 Attention Output (W)	( ~8M) 8388608	4096 x 2048 x 1 x 1	IQ4_NL
571	blk.47.attn_q.weight	Block 47 Attention Query (W)	( ~8M) 8388608	2048 x 4096 x 1 x 1	IQ4_NL
572	blk.47.attn_q_norm.weight	Block 47 Attn_Q_Norm (W)	( 128) 128	128 x 1 x 1 x 1	F32
573	blk.47.attn_v.weight	Block 47 Attention Value (W)	( ~1M) 1048576	2048 x 512 x 1 x 1	IQ4_NL
574	blk.47.ffn_down_exps.weight	Block 47 Ffn_Down_Exps (W)	(~201M) 201326592	768 x 2048 x 128 x 1	Q5_K
575	blk.47.ffn_gate_exps.weight	Block 47 Ffn_Gate_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL
576	blk.47.ffn_gate_inp.weight	Block 47 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W)	(~262K) 262144	2048 x 128 x 1 x 1	F32
577	blk.47.ffn_norm.weight	Block 47 Feed-Forward Network Normalization (W)	( ~2K) 2048	2048 x 1 x 1 x 1	F32
578	blk.47.ffn_up_exps.weight	Block 47 Ffn_Up_Exps (W)	(~201M) 201326592	2048 x 768 x 128 x 1	IQ4_NL

Total elements in blk.47: (~623M) 623120640
Percentage of total elements: 2.04%