# Qwen3-30B-A3B-IQ4_NL.gguf - GGUF Internal File Dump - Endian: LITTLE endian ## Key Value Metadata Store There are 44 key-value pairs in this file | POS | TYPE | Count | Key | Value | |----:|:---------|-------:|:------------------------------------------|:--------------------------------------------------------------------| | 1 | UINT32 | 1 | GGUF.version | 3 | | 2 | UINT64 | 1 | GGUF.tensor_count | 579 | | 3 | UINT64 | 1 | GGUF.kv_count | 41 | | 4 | STRING | 1 | general.architecture | `qwen3moe` | | 5 | STRING | 1 | general.type | `model` | | 6 | STRING | 1 | general.name | `Qwen3 30B A3B` | | 7 | STRING | 1 | general.basename | `Qwen3` | | 8 | STRING | 1 | general.size_label | `30B-A3B` | | 9 | STRING | 1 | general.license | `apache-2.0` | | 10 | STRING | 1 | general.license.link | `https://huggingface.co/Qwen/Qwen3-30B-A3B/blob/main/LICENSE` | | 11 | UINT32 | 1 | general.base_model.count | 1 | | 12 | STRING | 1 | general.base_model.0.name | `Qwen3 30B A3B Base` | | 13 | STRING | 1 | general.base_model.0.organization | `Qwen` | | 14 | STRING | 1 | general.base_model.0.repo_url | `https://huggingface.co/Qwen/Qwen3-30B-A3B-Base` | | 15 | [STRING] | 1 | general.tags | [ `text-generation` ] | | 16 | UINT32 | 1 | qwen3moe.block_count | 48 | | 17 | UINT32 | 1 | qwen3moe.context_length | 40960 | | 18 | UINT32 | 1 | qwen3moe.embedding_length | 2048 | | 19 | UINT32 | 1 | qwen3moe.feed_forward_length | 6144 | | 20 | UINT32 | 1 | qwen3moe.attention.head_count | 32 | | 21 | UINT32 | 1 | qwen3moe.attention.head_count_kv | 4 | | 22 | FLOAT32 | 1 | qwen3moe.rope.freq_base | 1000000.0 | | 23 | FLOAT32 | 1 | qwen3moe.attention.layer_norm_rms_epsilon | 1e-06 | | 24 | UINT32 | 1 | qwen3moe.expert_used_count | 8 | | 25 | UINT32 | 1 | qwen3moe.attention.key_length | 128 | | 26 | UINT32 | 1 | qwen3moe.attention.value_length | 128 | | 27 | UINT32 | 1 | qwen3moe.expert_count | 128 | | 28 | UINT32 | 1 | qwen3moe.expert_feed_forward_length | 768 | | 29 | STRING | 1 | tokenizer.ggml.model | `gpt2` | | 30 | STRING | 1 | tokenizer.ggml.pre | `qwen2` | | 31 | [STRING] | 151936 | tokenizer.ggml.tokens | [ `!`, `"`, `#`, `$`, `%`, ... ] | | 32 | [INT32] | 151936 | tokenizer.ggml.token_type | [ 1, 1, 1, 1, 1, 1, 1, ... ] | | 33 | [STRING] | 151387 | tokenizer.ggml.merges | [ `Ġ Ġ`, `ĠĠ ĠĠ`, `i n`, `Ġ t`, `ĠĠĠĠ ĠĠĠĠ`, ... ] | | 34 | UINT32 | 1 | tokenizer.ggml.eos_token_id | 151645 | | 35 | UINT32 | 1 | tokenizer.ggml.padding_token_id | 151643 | | 36 | UINT32 | 1 | tokenizer.ggml.bos_token_id | 151643 | | 37 | BOOL | 1 | tokenizer.ggml.add_bos_token | False | | 38 | STRING | 1 | tokenizer.chat_template | `{%- if tools %}{{- '<|im_`...`{%- endif %}{%- endif %}` | | 39 | UINT32 | 1 | general.quantization_version | 2 | | 40 | UINT32 | 1 | general.file_type | 25 | | 41 | STRING | 1 | quantize.imatrix.file | `./imatrix/imatrix-Qwen3-30B-A3B-large.dat` | | 42 | STRING | 1 | quantize.imatrix.dataset | `../../datasets/imatrix/calibration_all_large.txt` | | 43 | INT32 | 1 | quantize.imatrix.entries_count | 382 | | 44 | INT32 | 1 | quantize.imatrix.chunks_count | 4978 | ## Tensors Overview ~31B Elements Total number of elements in all tensors: 30532122624 Elements - [Qwen3-30B-A3B-IQ4\_NL.gguf - GGUF Internal File Dump](#qwen3-30b-a3b-iq4_nlgguf---gguf-internal-file-dump) - [Key Value Metadata Store](#key-value-metadata-store) - [Tensors Overview ~31B Elements](#tensors-overview-31b-elements) - [Tensor Data Offset](#tensor-data-offset) - [Base Tensor Group : ~622M Elements](#base-tensor-group--622m-elements) - [Block 0 Tensor Group : ~623M Elements](#block-0-tensor-group--623m-elements) - [Block 1 Tensor Group : ~623M Elements](#block-1-tensor-group--623m-elements) - [Block 2 Tensor Group : ~623M Elements](#block-2-tensor-group--623m-elements) - [Block 3 Tensor Group : ~623M Elements](#block-3-tensor-group--623m-elements) - [Block 4 Tensor Group : ~623M Elements](#block-4-tensor-group--623m-elements) - [Block 5 Tensor Group : ~623M Elements](#block-5-tensor-group--623m-elements) - [Block 6 Tensor Group : ~623M Elements](#block-6-tensor-group--623m-elements) - [Block 7 Tensor Group : ~623M Elements](#block-7-tensor-group--623m-elements) - [Block 8 Tensor Group : ~623M Elements](#block-8-tensor-group--623m-elements) - [Block 9 Tensor Group : ~623M Elements](#block-9-tensor-group--623m-elements) - [Block 10 Tensor Group : ~623M Elements](#block-10-tensor-group--623m-elements) - [Block 11 Tensor Group : ~623M Elements](#block-11-tensor-group--623m-elements) - [Block 12 Tensor Group : ~623M Elements](#block-12-tensor-group--623m-elements) - [Block 13 Tensor Group : ~623M Elements](#block-13-tensor-group--623m-elements) - [Block 14 Tensor Group : ~623M Elements](#block-14-tensor-group--623m-elements) - [Block 15 Tensor Group : ~623M Elements](#block-15-tensor-group--623m-elements) - [Block 16 Tensor Group : ~623M Elements](#block-16-tensor-group--623m-elements) - [Block 17 Tensor Group : ~623M Elements](#block-17-tensor-group--623m-elements) - [Block 18 Tensor Group : ~623M Elements](#block-18-tensor-group--623m-elements) - [Block 19 Tensor Group : ~623M Elements](#block-19-tensor-group--623m-elements) - [Block 20 Tensor Group : ~623M Elements](#block-20-tensor-group--623m-elements) - [Block 21 Tensor Group : ~623M Elements](#block-21-tensor-group--623m-elements) - [Block 22 Tensor Group : ~623M Elements](#block-22-tensor-group--623m-elements) - [Block 23 Tensor Group : ~623M Elements](#block-23-tensor-group--623m-elements) - [Block 24 Tensor Group : ~623M Elements](#block-24-tensor-group--623m-elements) - [Block 25 Tensor Group : ~623M Elements](#block-25-tensor-group--623m-elements) - [Block 26 Tensor Group : ~623M Elements](#block-26-tensor-group--623m-elements) - [Block 27 Tensor Group : ~623M Elements](#block-27-tensor-group--623m-elements) - [Block 28 Tensor Group : ~623M Elements](#block-28-tensor-group--623m-elements) - [Block 29 Tensor Group : ~623M Elements](#block-29-tensor-group--623m-elements) - [Block 30 Tensor Group : ~623M Elements](#block-30-tensor-group--623m-elements) - [Block 31 Tensor Group : ~623M Elements](#block-31-tensor-group--623m-elements) - [Block 32 Tensor Group : ~623M Elements](#block-32-tensor-group--623m-elements) - [Block 33 Tensor Group : ~623M Elements](#block-33-tensor-group--623m-elements) - [Block 34 Tensor Group : ~623M Elements](#block-34-tensor-group--623m-elements) - [Block 35 Tensor Group : ~623M Elements](#block-35-tensor-group--623m-elements) - [Block 36 Tensor Group : ~623M Elements](#block-36-tensor-group--623m-elements) - [Block 37 Tensor Group : ~623M Elements](#block-37-tensor-group--623m-elements) - [Block 38 Tensor Group : ~623M Elements](#block-38-tensor-group--623m-elements) - [Block 39 Tensor Group : ~623M Elements](#block-39-tensor-group--623m-elements) - [Block 40 Tensor Group : ~623M Elements](#block-40-tensor-group--623m-elements) - [Block 41 Tensor Group : ~623M Elements](#block-41-tensor-group--623m-elements) - [Block 42 Tensor Group : ~623M Elements](#block-42-tensor-group--623m-elements) - [Block 43 Tensor Group : ~623M Elements](#block-43-tensor-group--623m-elements) - [Block 44 Tensor Group : ~623M Elements](#block-44-tensor-group--623m-elements) - [Block 45 Tensor Group : ~623M Elements](#block-45-tensor-group--623m-elements) - [Block 46 Tensor Group : ~623M Elements](#block-46-tensor-group--623m-elements) - [Block 47 Tensor Group : ~623M Elements](#block-47-tensor-group--623m-elements) ### Tensor Data Offset This table contains the offset and data segment relative to start of file | T_ID | Tensor Layer Name | Data Offset (B) | Data Size (B) | |-----:|:----------------------------|-----------------:|-----------------:| | 0 | output.weight | 0x5b18c0 | 0xa6ec000 | | 1 | output_norm.weight | 0xac9d8c0 | 0x2000 | | 2 | token_embd.weight | 0xac9f8c0 | 0x7f82800 | | 3 | blk.0.attn_k.weight | 0x12c220c0 | 0x6e000 | | 4 | blk.0.attn_k_norm.weight | 0x12c900c0 | 0x200 | | 5 | blk.0.attn_norm.weight | 0x12c902c0 | 0x2000 | | 6 | blk.0.attn_output.weight | 0x12c922c0 | 0x480000 | | 7 | blk.0.attn_q.weight | 0x131122c0 | 0x370000 | | 8 | blk.0.attn_q_norm.weight | 0x134822c0 | 0x200 | | 9 | blk.0.attn_v.weight | 0x134824c0 | 0x88000 | | 10 | blk.0.ffn_down_exps.weight | 0x1350a4c0 | 0x8400000 | | 11 | blk.0.ffn_gate_exps.weight | 0x1b90a4c0 | 0x5280000 | | 12 | blk.0.ffn_gate_inp.weight | 0x20b8a4c0 | 0x100000 | | 13 | blk.0.ffn_norm.weight | 0x20c8a4c0 | 0x2000 | | 14 | blk.0.ffn_up_exps.weight | 0x20c8c4c0 | 0x5280000 | | 15 | blk.1.attn_k.weight | 0x25f0c4c0 | 0x6e000 | | 16 | blk.1.attn_k_norm.weight | 0x25f7a4c0 | 0x200 | | 17 | blk.1.attn_norm.weight | 0x25f7a6c0 | 0x2000 | | 18 | blk.1.attn_output.weight | 0x25f7c6c0 | 0x480000 | | 19 | blk.1.attn_q.weight | 0x263fc6c0 | 0x370000 | | 20 | blk.1.attn_q_norm.weight | 0x2676c6c0 | 0x200 | | 21 | blk.1.attn_v.weight | 0x2676c8c0 | 0x88000 | | 22 | blk.1.ffn_down_exps.weight | 0x267f48c0 | 0x8400000 | | 23 | blk.1.ffn_gate_exps.weight | 0x2ebf48c0 | 0x5280000 | | 24 | blk.1.ffn_gate_inp.weight | 0x33e748c0 | 0x100000 | | 25 | blk.1.ffn_norm.weight | 0x33f748c0 | 0x2000 | | 26 | blk.1.ffn_up_exps.weight | 0x33f768c0 | 0x5280000 | | 27 | blk.2.attn_k.weight | 0x391f68c0 | 0x6e000 | | 28 | blk.2.attn_k_norm.weight | 0x392648c0 | 0x200 | | 29 | blk.2.attn_norm.weight | 0x39264ac0 | 0x2000 | | 30 | blk.2.attn_output.weight | 0x39266ac0 | 0x480000 | | 31 | blk.2.attn_q.weight | 0x396e6ac0 | 0x370000 | | 32 | blk.2.attn_q_norm.weight | 0x39a56ac0 | 0x200 | | 33 | blk.2.attn_v.weight | 0x39a56cc0 | 0x88000 | | 34 | blk.2.ffn_down_exps.weight | 0x39adecc0 | 0x8400000 | | 35 | blk.2.ffn_gate_exps.weight | 0x41edecc0 | 0x5280000 | | 36 | blk.2.ffn_gate_inp.weight | 0x4715ecc0 | 0x100000 | | 37 | blk.2.ffn_norm.weight | 0x4725ecc0 | 0x2000 | | 38 | blk.2.ffn_up_exps.weight | 0x47260cc0 | 0x5280000 | | 39 | blk.3.attn_k.weight | 0x4c4e0cc0 | 0x6e000 | | 40 | blk.3.attn_k_norm.weight | 0x4c54ecc0 | 0x200 | | 41 | blk.3.attn_norm.weight | 0x4c54eec0 | 0x2000 | | 42 | blk.3.attn_output.weight | 0x4c550ec0 | 0x480000 | | 43 | blk.3.attn_q.weight | 0x4c9d0ec0 | 0x370000 | | 44 | blk.3.attn_q_norm.weight | 0x4cd40ec0 | 0x200 | | 45 | blk.3.attn_v.weight | 0x4cd410c0 | 0x88000 | | 46 | blk.3.ffn_down_exps.weight | 0x4cdc90c0 | 0x8400000 | | 47 | blk.3.ffn_gate_exps.weight | 0x551c90c0 | 0x5280000 | | 48 | blk.3.ffn_gate_inp.weight | 0x5a4490c0 | 0x100000 | | 49 | blk.3.ffn_norm.weight | 0x5a5490c0 | 0x2000 | | 50 | blk.3.ffn_up_exps.weight | 0x5a54b0c0 | 0x5280000 | | 51 | blk.4.attn_k.weight | 0x5f7cb0c0 | 0x6e000 | | 52 | blk.4.attn_k_norm.weight | 0x5f8390c0 | 0x200 | | 53 | blk.4.attn_norm.weight | 0x5f8392c0 | 0x2000 | | 54 | blk.4.attn_output.weight | 0x5f83b2c0 | 0x480000 | | 55 | blk.4.attn_q.weight | 0x5fcbb2c0 | 0x370000 | | 56 | blk.4.attn_q_norm.weight | 0x6002b2c0 | 0x200 | | 57 | blk.4.attn_v.weight | 0x6002b4c0 | 0x88000 | | 58 | blk.4.ffn_down_exps.weight | 0x600b34c0 | 0x8400000 | | 59 | blk.4.ffn_gate_exps.weight | 0x684b34c0 | 0x5280000 | | 60 | blk.4.ffn_gate_inp.weight | 0x6d7334c0 | 0x100000 | | 61 | blk.4.ffn_norm.weight | 0x6d8334c0 | 0x2000 | | 62 | blk.4.ffn_up_exps.weight | 0x6d8354c0 | 0x5280000 | | 63 | blk.5.attn_k.weight | 0x72ab54c0 | 0x6e000 | | 64 | blk.5.attn_k_norm.weight | 0x72b234c0 | 0x200 | | 65 | blk.5.attn_norm.weight | 0x72b236c0 | 0x2000 | | 66 | blk.5.attn_output.weight | 0x72b256c0 | 0x480000 | | 67 | blk.5.attn_q.weight | 0x72fa56c0 | 0x370000 | | 68 | blk.5.attn_q_norm.weight | 0x733156c0 | 0x200 | | 69 | blk.5.attn_v.weight | 0x733158c0 | 0x88000 | | 70 | blk.5.ffn_down_exps.weight | 0x7339d8c0 | 0x8400000 | | 71 | blk.5.ffn_gate_exps.weight | 0x7b79d8c0 | 0x5280000 | | 72 | blk.5.ffn_gate_inp.weight | 0x80a1d8c0 | 0x100000 | | 73 | blk.5.ffn_norm.weight | 0x80b1d8c0 | 0x2000 | | 74 | blk.5.ffn_up_exps.weight | 0x80b1f8c0 | 0x5280000 | | 75 | blk.6.attn_k.weight | 0x85d9f8c0 | 0x6e000 | | 76 | blk.6.attn_k_norm.weight | 0x85e0d8c0 | 0x200 | | 77 | blk.6.attn_norm.weight | 0x85e0dac0 | 0x2000 | | 78 | blk.6.attn_output.weight | 0x85e0fac0 | 0x480000 | | 79 | blk.6.attn_q.weight | 0x8628fac0 | 0x370000 | | 80 | blk.6.attn_q_norm.weight | 0x865ffac0 | 0x200 | | 81 | blk.6.attn_v.weight | 0x865ffcc0 | 0x88000 | | 82 | blk.6.ffn_down_exps.weight | 0x86687cc0 | 0x8400000 | | 83 | blk.6.ffn_gate_exps.weight | 0x8ea87cc0 | 0x5280000 | | 84 | blk.6.ffn_gate_inp.weight | 0x93d07cc0 | 0x100000 | | 85 | blk.6.ffn_norm.weight | 0x93e07cc0 | 0x2000 | | 86 | blk.6.ffn_up_exps.weight | 0x93e09cc0 | 0x5280000 | | 87 | blk.7.attn_k.weight | 0x99089cc0 | 0x6e000 | | 88 | blk.7.attn_k_norm.weight | 0x990f7cc0 | 0x200 | | 89 | blk.7.attn_norm.weight | 0x990f7ec0 | 0x2000 | | 90 | blk.7.attn_output.weight | 0x990f9ec0 | 0x480000 | | 91 | blk.7.attn_q.weight | 0x99579ec0 | 0x370000 | | 92 | blk.7.attn_q_norm.weight | 0x998e9ec0 | 0x200 | | 93 | blk.7.attn_v.weight | 0x998ea0c0 | 0x88000 | | 94 | blk.7.ffn_down_exps.weight | 0x999720c0 | 0x8400000 | | 95 | blk.7.ffn_gate_exps.weight | 0xa1d720c0 | 0x5280000 | | 96 | blk.7.ffn_gate_inp.weight | 0xa6ff20c0 | 0x100000 | | 97 | blk.7.ffn_norm.weight | 0xa70f20c0 | 0x2000 | | 98 | blk.7.ffn_up_exps.weight | 0xa70f40c0 | 0x5280000 | | 99 | blk.8.attn_k.weight | 0xac3740c0 | 0x6e000 | | 100 | blk.8.attn_k_norm.weight | 0xac3e20c0 | 0x200 | | 101 | blk.8.attn_norm.weight | 0xac3e22c0 | 0x2000 | | 102 | blk.8.attn_output.weight | 0xac3e42c0 | 0x480000 | | 103 | blk.8.attn_q.weight | 0xac8642c0 | 0x370000 | | 104 | blk.8.attn_q_norm.weight | 0xacbd42c0 | 0x200 | | 105 | blk.8.attn_v.weight | 0xacbd44c0 | 0x88000 | | 106 | blk.8.ffn_down_exps.weight | 0xacc5c4c0 | 0x8400000 | | 107 | blk.8.ffn_gate_exps.weight | 0xb505c4c0 | 0x5280000 | | 108 | blk.8.ffn_gate_inp.weight | 0xba2dc4c0 | 0x100000 | | 109 | blk.8.ffn_norm.weight | 0xba3dc4c0 | 0x2000 | | 110 | blk.8.ffn_up_exps.weight | 0xba3de4c0 | 0x5280000 | | 111 | blk.9.attn_k.weight | 0xbf65e4c0 | 0x6e000 | | 112 | blk.9.attn_k_norm.weight | 0xbf6cc4c0 | 0x200 | | 113 | blk.9.attn_norm.weight | 0xbf6cc6c0 | 0x2000 | | 114 | blk.9.attn_output.weight | 0xbf6ce6c0 | 0x480000 | | 115 | blk.9.attn_q.weight | 0xbfb4e6c0 | 0x370000 | | 116 | blk.9.attn_q_norm.weight | 0xbfebe6c0 | 0x200 | | 117 | blk.9.attn_v.weight | 0xbfebe8c0 | 0x88000 | | 118 | blk.9.ffn_down_exps.weight | 0xbff468c0 | 0x8400000 | | 119 | blk.9.ffn_gate_exps.weight | 0xc83468c0 | 0x5280000 | | 120 | blk.9.ffn_gate_inp.weight | 0xcd5c68c0 | 0x100000 | | 121 | blk.9.ffn_norm.weight | 0xcd6c68c0 | 0x2000 | | 122 | blk.9.ffn_up_exps.weight | 0xcd6c88c0 | 0x5280000 | | 123 | blk.10.attn_k.weight | 0xd29488c0 | 0x6e000 | | 124 | blk.10.attn_k_norm.weight | 0xd29b68c0 | 0x200 | | 125 | blk.10.attn_norm.weight | 0xd29b6ac0 | 0x2000 | | 126 | blk.10.attn_output.weight | 0xd29b8ac0 | 0x480000 | | 127 | blk.10.attn_q.weight | 0xd2e38ac0 | 0x370000 | | 128 | blk.10.attn_q_norm.weight | 0xd31a8ac0 | 0x200 | | 129 | blk.10.attn_v.weight | 0xd31a8cc0 | 0x88000 | | 130 | blk.10.ffn_down_exps.weight | 0xd3230cc0 | 0x8400000 | | 131 | blk.10.ffn_gate_exps.weight | 0xdb630cc0 | 0x5280000 | | 132 | blk.10.ffn_gate_inp.weight | 0xe08b0cc0 | 0x100000 | | 133 | blk.10.ffn_norm.weight | 0xe09b0cc0 | 0x2000 | | 134 | blk.10.ffn_up_exps.weight | 0xe09b2cc0 | 0x5280000 | | 135 | blk.11.attn_k.weight | 0xe5c32cc0 | 0x6e000 | | 136 | blk.11.attn_k_norm.weight | 0xe5ca0cc0 | 0x200 | | 137 | blk.11.attn_norm.weight | 0xe5ca0ec0 | 0x2000 | | 138 | blk.11.attn_output.weight | 0xe5ca2ec0 | 0x480000 | | 139 | blk.11.attn_q.weight | 0xe6122ec0 | 0x370000 | | 140 | blk.11.attn_q_norm.weight | 0xe6492ec0 | 0x200 | | 141 | blk.11.attn_v.weight | 0xe64930c0 | 0x88000 | | 142 | blk.11.ffn_down_exps.weight | 0xe651b0c0 | 0x8400000 | | 143 | blk.11.ffn_gate_exps.weight | 0xee91b0c0 | 0x5280000 | | 144 | blk.11.ffn_gate_inp.weight | 0xf3b9b0c0 | 0x100000 | | 145 | blk.11.ffn_norm.weight | 0xf3c9b0c0 | 0x2000 | | 146 | blk.11.ffn_up_exps.weight | 0xf3c9d0c0 | 0x5280000 | | 147 | blk.12.attn_k.weight | 0xf8f1d0c0 | 0x6e000 | | 148 | blk.12.attn_k_norm.weight | 0xf8f8b0c0 | 0x200 | | 149 | blk.12.attn_norm.weight | 0xf8f8b2c0 | 0x2000 | | 150 | blk.12.attn_output.weight | 0xf8f8d2c0 | 0x480000 | | 151 | blk.12.attn_q.weight | 0xf940d2c0 | 0x370000 | | 152 | blk.12.attn_q_norm.weight | 0xf977d2c0 | 0x200 | | 153 | blk.12.attn_v.weight | 0xf977d4c0 | 0x88000 | | 154 | blk.12.ffn_down_exps.weight | 0xf98054c0 | 0x8400000 | | 155 | blk.12.ffn_gate_exps.weight | 0x101c054c0 | 0x5280000 | | 156 | blk.12.ffn_gate_inp.weight | 0x106e854c0 | 0x100000 | | 157 | blk.12.ffn_norm.weight | 0x106f854c0 | 0x2000 | | 158 | blk.12.ffn_up_exps.weight | 0x106f874c0 | 0x5280000 | | 159 | blk.13.attn_k.weight | 0x10c2074c0 | 0x6e000 | | 160 | blk.13.attn_k_norm.weight | 0x10c2754c0 | 0x200 | | 161 | blk.13.attn_norm.weight | 0x10c2756c0 | 0x2000 | | 162 | blk.13.attn_output.weight | 0x10c2776c0 | 0x480000 | | 163 | blk.13.attn_q.weight | 0x10c6f76c0 | 0x370000 | | 164 | blk.13.attn_q_norm.weight | 0x10ca676c0 | 0x200 | | 165 | blk.13.attn_v.weight | 0x10ca678c0 | 0x88000 | | 166 | blk.13.ffn_down_exps.weight | 0x10caef8c0 | 0x8400000 | | 167 | blk.13.ffn_gate_exps.weight | 0x114eef8c0 | 0x6c00000 | | 168 | blk.13.ffn_gate_inp.weight | 0x11baef8c0 | 0x100000 | | 169 | blk.13.ffn_norm.weight | 0x11bbef8c0 | 0x2000 | | 170 | blk.13.ffn_up_exps.weight | 0x11bbf18c0 | 0x6c00000 | | 171 | blk.14.attn_k.weight | 0x1227f18c0 | 0x6e000 | | 172 | blk.14.attn_k_norm.weight | 0x12285f8c0 | 0x200 | | 173 | blk.14.attn_norm.weight | 0x12285fac0 | 0x2000 | | 174 | blk.14.attn_output.weight | 0x122861ac0 | 0x480000 | | 175 | blk.14.attn_q.weight | 0x122ce1ac0 | 0x370000 | | 176 | blk.14.attn_q_norm.weight | 0x123051ac0 | 0x200 | | 177 | blk.14.attn_v.weight | 0x123051cc0 | 0x88000 | | 178 | blk.14.ffn_down_exps.weight | 0x1230d9cc0 | 0x8400000 | | 179 | blk.14.ffn_gate_exps.weight | 0x12b4d9cc0 | 0x5280000 | | 180 | blk.14.ffn_gate_inp.weight | 0x130759cc0 | 0x100000 | | 181 | blk.14.ffn_norm.weight | 0x130859cc0 | 0x2000 | | 182 | blk.14.ffn_up_exps.weight | 0x13085bcc0 | 0x5280000 | | 183 | blk.15.attn_k.weight | 0x135adbcc0 | 0x6e000 | | 184 | blk.15.attn_k_norm.weight | 0x135b49cc0 | 0x200 | | 185 | blk.15.attn_norm.weight | 0x135b49ec0 | 0x2000 | | 186 | blk.15.attn_output.weight | 0x135b4bec0 | 0x480000 | | 187 | blk.15.attn_q.weight | 0x135fcbec0 | 0x370000 | | 188 | blk.15.attn_q_norm.weight | 0x13633bec0 | 0x200 | | 189 | blk.15.attn_v.weight | 0x13633c0c0 | 0x88000 | | 190 | blk.15.ffn_down_exps.weight | 0x1363c40c0 | 0x8400000 | | 191 | blk.15.ffn_gate_exps.weight | 0x13e7c40c0 | 0x6c00000 | | 192 | blk.15.ffn_gate_inp.weight | 0x1453c40c0 | 0x100000 | | 193 | blk.15.ffn_norm.weight | 0x1454c40c0 | 0x2000 | | 194 | blk.15.ffn_up_exps.weight | 0x1454c60c0 | 0x6c00000 | | 195 | blk.16.attn_k.weight | 0x14c0c60c0 | 0x6e000 | | 196 | blk.16.attn_k_norm.weight | 0x14c1340c0 | 0x200 | | 197 | blk.16.attn_norm.weight | 0x14c1342c0 | 0x2000 | | 198 | blk.16.attn_output.weight | 0x14c1362c0 | 0x480000 | | 199 | blk.16.attn_q.weight | 0x14c5b62c0 | 0x370000 | | 200 | blk.16.attn_q_norm.weight | 0x14c9262c0 | 0x200 | | 201 | blk.16.attn_v.weight | 0x14c9264c0 | 0x88000 | | 202 | blk.16.ffn_down_exps.weight | 0x14c9ae4c0 | 0x8400000 | | 203 | blk.16.ffn_gate_exps.weight | 0x154dae4c0 | 0x5280000 | | 204 | blk.16.ffn_gate_inp.weight | 0x15a02e4c0 | 0x100000 | | 205 | blk.16.ffn_norm.weight | 0x15a12e4c0 | 0x2000 | | 206 | blk.16.ffn_up_exps.weight | 0x15a1304c0 | 0x5280000 | | 207 | blk.17.attn_k.weight | 0x15f3b04c0 | 0x6e000 | | 208 | blk.17.attn_k_norm.weight | 0x15f41e4c0 | 0x200 | | 209 | blk.17.attn_norm.weight | 0x15f41e6c0 | 0x2000 | | 210 | blk.17.attn_output.weight | 0x15f4206c0 | 0x480000 | | 211 | blk.17.attn_q.weight | 0x15f8a06c0 | 0x370000 | | 212 | blk.17.attn_q_norm.weight | 0x15fc106c0 | 0x200 | | 213 | blk.17.attn_v.weight | 0x15fc108c0 | 0x88000 | | 214 | blk.17.ffn_down_exps.weight | 0x15fc988c0 | 0x8400000 | | 215 | blk.17.ffn_gate_exps.weight | 0x1680988c0 | 0x5280000 | | 216 | blk.17.ffn_gate_inp.weight | 0x16d3188c0 | 0x100000 | | 217 | blk.17.ffn_norm.weight | 0x16d4188c0 | 0x2000 | | 218 | blk.17.ffn_up_exps.weight | 0x16d41a8c0 | 0x5280000 | | 219 | blk.18.attn_k.weight | 0x17269a8c0 | 0x6e000 | | 220 | blk.18.attn_k_norm.weight | 0x1727088c0 | 0x200 | | 221 | blk.18.attn_norm.weight | 0x172708ac0 | 0x2000 | | 222 | blk.18.attn_output.weight | 0x17270aac0 | 0x480000 | | 223 | blk.18.attn_q.weight | 0x172b8aac0 | 0x370000 | | 224 | blk.18.attn_q_norm.weight | 0x172efaac0 | 0x200 | | 225 | blk.18.attn_v.weight | 0x172efacc0 | 0x88000 | | 226 | blk.18.ffn_down_exps.weight | 0x172f82cc0 | 0x8400000 | | 227 | blk.18.ffn_gate_exps.weight | 0x17b382cc0 | 0x5280000 | | 228 | blk.18.ffn_gate_inp.weight | 0x180602cc0 | 0x100000 | | 229 | blk.18.ffn_norm.weight | 0x180702cc0 | 0x2000 | | 230 | blk.18.ffn_up_exps.weight | 0x180704cc0 | 0x5280000 | | 231 | blk.19.attn_k.weight | 0x185984cc0 | 0x6e000 | | 232 | blk.19.attn_k_norm.weight | 0x1859f2cc0 | 0x200 | | 233 | blk.19.attn_norm.weight | 0x1859f2ec0 | 0x2000 | | 234 | blk.19.attn_output.weight | 0x1859f4ec0 | 0x480000 | | 235 | blk.19.attn_q.weight | 0x185e74ec0 | 0x370000 | | 236 | blk.19.attn_q_norm.weight | 0x1861e4ec0 | 0x200 | | 237 | blk.19.attn_v.weight | 0x1861e50c0 | 0x88000 | | 238 | blk.19.ffn_down_exps.weight | 0x18626d0c0 | 0x8400000 | | 239 | blk.19.ffn_gate_exps.weight | 0x18e66d0c0 | 0x5280000 | | 240 | blk.19.ffn_gate_inp.weight | 0x1938ed0c0 | 0x100000 | | 241 | blk.19.ffn_norm.weight | 0x1939ed0c0 | 0x2000 | | 242 | blk.19.ffn_up_exps.weight | 0x1939ef0c0 | 0x5280000 | | 243 | blk.20.attn_k.weight | 0x198c6f0c0 | 0x6e000 | | 244 | blk.20.attn_k_norm.weight | 0x198cdd0c0 | 0x200 | | 245 | blk.20.attn_norm.weight | 0x198cdd2c0 | 0x2000 | | 246 | blk.20.attn_output.weight | 0x198cdf2c0 | 0x480000 | | 247 | blk.20.attn_q.weight | 0x19915f2c0 | 0x370000 | | 248 | blk.20.attn_q_norm.weight | 0x1994cf2c0 | 0x200 | | 249 | blk.20.attn_v.weight | 0x1994cf4c0 | 0x88000 | | 250 | blk.20.ffn_down_exps.weight | 0x1995574c0 | 0x8400000 | | 251 | blk.20.ffn_gate_exps.weight | 0x1a19574c0 | 0x5280000 | | 252 | blk.20.ffn_gate_inp.weight | 0x1a6bd74c0 | 0x100000 | | 253 | blk.20.ffn_norm.weight | 0x1a6cd74c0 | 0x2000 | | 254 | blk.20.ffn_up_exps.weight | 0x1a6cd94c0 | 0x5280000 | | 255 | blk.21.attn_k.weight | 0x1abf594c0 | 0x6e000 | | 256 | blk.21.attn_k_norm.weight | 0x1abfc74c0 | 0x200 | | 257 | blk.21.attn_norm.weight | 0x1abfc76c0 | 0x2000 | | 258 | blk.21.attn_output.weight | 0x1abfc96c0 | 0x480000 | | 259 | blk.21.attn_q.weight | 0x1ac4496c0 | 0x370000 | | 260 | blk.21.attn_q_norm.weight | 0x1ac7b96c0 | 0x200 | | 261 | blk.21.attn_v.weight | 0x1ac7b98c0 | 0x88000 | | 262 | blk.21.ffn_down_exps.weight | 0x1ac8418c0 | 0x8400000 | | 263 | blk.21.ffn_gate_exps.weight | 0x1b4c418c0 | 0x5280000 | | 264 | blk.21.ffn_gate_inp.weight | 0x1b9ec18c0 | 0x100000 | | 265 | blk.21.ffn_norm.weight | 0x1b9fc18c0 | 0x2000 | | 266 | blk.21.ffn_up_exps.weight | 0x1b9fc38c0 | 0x5280000 | | 267 | blk.22.attn_k.weight | 0x1bf2438c0 | 0x6e000 | | 268 | blk.22.attn_k_norm.weight | 0x1bf2b18c0 | 0x200 | | 269 | blk.22.attn_norm.weight | 0x1bf2b1ac0 | 0x2000 | | 270 | blk.22.attn_output.weight | 0x1bf2b3ac0 | 0x480000 | | 271 | blk.22.attn_q.weight | 0x1bf733ac0 | 0x370000 | | 272 | blk.22.attn_q_norm.weight | 0x1bfaa3ac0 | 0x200 | | 273 | blk.22.attn_v.weight | 0x1bfaa3cc0 | 0x88000 | | 274 | blk.22.ffn_down_exps.weight | 0x1bfb2bcc0 | 0x8400000 | | 275 | blk.22.ffn_gate_exps.weight | 0x1c7f2bcc0 | 0x5280000 | | 276 | blk.22.ffn_gate_inp.weight | 0x1cd1abcc0 | 0x100000 | | 277 | blk.22.ffn_norm.weight | 0x1cd2abcc0 | 0x2000 | | 278 | blk.22.ffn_up_exps.weight | 0x1cd2adcc0 | 0x5280000 | | 279 | blk.23.attn_k.weight | 0x1d252dcc0 | 0x6e000 | | 280 | blk.23.attn_k_norm.weight | 0x1d259bcc0 | 0x200 | | 281 | blk.23.attn_norm.weight | 0x1d259bec0 | 0x2000 | | 282 | blk.23.attn_output.weight | 0x1d259dec0 | 0x480000 | | 283 | blk.23.attn_q.weight | 0x1d2a1dec0 | 0x370000 | | 284 | blk.23.attn_q_norm.weight | 0x1d2d8dec0 | 0x200 | | 285 | blk.23.attn_v.weight | 0x1d2d8e0c0 | 0x88000 | | 286 | blk.23.ffn_down_exps.weight | 0x1d2e160c0 | 0x8400000 | | 287 | blk.23.ffn_gate_exps.weight | 0x1db2160c0 | 0x5280000 | | 288 | blk.23.ffn_gate_inp.weight | 0x1e04960c0 | 0x100000 | | 289 | blk.23.ffn_norm.weight | 0x1e05960c0 | 0x2000 | | 290 | blk.23.ffn_up_exps.weight | 0x1e05980c0 | 0x5280000 | | 291 | blk.24.attn_k.weight | 0x1e58180c0 | 0x90000 | | 292 | blk.24.attn_k_norm.weight | 0x1e58a80c0 | 0x200 | | 293 | blk.24.attn_norm.weight | 0x1e58a82c0 | 0x2000 | | 294 | blk.24.attn_output.weight | 0x1e58aa2c0 | 0x480000 | | 295 | blk.24.attn_q.weight | 0x1e5d2a2c0 | 0x480000 | | 296 | blk.24.attn_q_norm.weight | 0x1e61aa2c0 | 0x200 | | 297 | blk.24.attn_v.weight | 0x1e61aa4c0 | 0x90000 | | 298 | blk.24.ffn_down_exps.weight | 0x1e623a4c0 | 0x8400000 | | 299 | blk.24.ffn_gate_exps.weight | 0x1ee63a4c0 | 0x5280000 | | 300 | blk.24.ffn_gate_inp.weight | 0x1f38ba4c0 | 0x100000 | | 301 | blk.24.ffn_norm.weight | 0x1f39ba4c0 | 0x2000 | | 302 | blk.24.ffn_up_exps.weight | 0x1f39bc4c0 | 0x5280000 | | 303 | blk.25.attn_k.weight | 0x1f8c3c4c0 | 0x90000 | | 304 | blk.25.attn_k_norm.weight | 0x1f8ccc4c0 | 0x200 | | 305 | blk.25.attn_norm.weight | 0x1f8ccc6c0 | 0x2000 | | 306 | blk.25.attn_output.weight | 0x1f8cce6c0 | 0x480000 | | 307 | blk.25.attn_q.weight | 0x1f914e6c0 | 0x480000 | | 308 | blk.25.attn_q_norm.weight | 0x1f95ce6c0 | 0x200 | | 309 | blk.25.attn_v.weight | 0x1f95ce8c0 | 0x90000 | | 310 | blk.25.ffn_down_exps.weight | 0x1f965e8c0 | 0x8400000 | | 311 | blk.25.ffn_gate_exps.weight | 0x201a5e8c0 | 0x6c00000 | | 312 | blk.25.ffn_gate_inp.weight | 0x20865e8c0 | 0x100000 | | 313 | blk.25.ffn_norm.weight | 0x20875e8c0 | 0x2000 | | 314 | blk.25.ffn_up_exps.weight | 0x2087608c0 | 0x6c00000 | | 315 | blk.26.attn_k.weight | 0x20f3608c0 | 0x90000 | | 316 | blk.26.attn_k_norm.weight | 0x20f3f08c0 | 0x200 | | 317 | blk.26.attn_norm.weight | 0x20f3f0ac0 | 0x2000 | | 318 | blk.26.attn_output.weight | 0x20f3f2ac0 | 0x480000 | | 319 | blk.26.attn_q.weight | 0x20f872ac0 | 0x480000 | | 320 | blk.26.attn_q_norm.weight | 0x20fcf2ac0 | 0x200 | | 321 | blk.26.attn_v.weight | 0x20fcf2cc0 | 0x90000 | | 322 | blk.26.ffn_down_exps.weight | 0x20fd82cc0 | 0x8400000 | | 323 | blk.26.ffn_gate_exps.weight | 0x218182cc0 | 0x5280000 | | 324 | blk.26.ffn_gate_inp.weight | 0x21d402cc0 | 0x100000 | | 325 | blk.26.ffn_norm.weight | 0x21d502cc0 | 0x2000 | | 326 | blk.26.ffn_up_exps.weight | 0x21d504cc0 | 0x5280000 | | 327 | blk.27.attn_k.weight | 0x222784cc0 | 0x90000 | | 328 | blk.27.attn_k_norm.weight | 0x222814cc0 | 0x200 | | 329 | blk.27.attn_norm.weight | 0x222814ec0 | 0x2000 | | 330 | blk.27.attn_output.weight | 0x222816ec0 | 0x480000 | | 331 | blk.27.attn_q.weight | 0x222c96ec0 | 0x480000 | | 332 | blk.27.attn_q_norm.weight | 0x223116ec0 | 0x200 | | 333 | blk.27.attn_v.weight | 0x2231170c0 | 0x90000 | | 334 | blk.27.ffn_down_exps.weight | 0x2231a70c0 | 0x8400000 | | 335 | blk.27.ffn_gate_exps.weight | 0x22b5a70c0 | 0x6c00000 | | 336 | blk.27.ffn_gate_inp.weight | 0x2321a70c0 | 0x100000 | | 337 | blk.27.ffn_norm.weight | 0x2322a70c0 | 0x2000 | | 338 | blk.27.ffn_up_exps.weight | 0x2322a90c0 | 0x6c00000 | | 339 | blk.28.attn_k.weight | 0x238ea90c0 | 0x90000 | | 340 | blk.28.attn_k_norm.weight | 0x238f390c0 | 0x200 | | 341 | blk.28.attn_norm.weight | 0x238f392c0 | 0x2000 | | 342 | blk.28.attn_output.weight | 0x238f3b2c0 | 0x480000 | | 343 | blk.28.attn_q.weight | 0x2393bb2c0 | 0x480000 | | 344 | blk.28.attn_q_norm.weight | 0x23983b2c0 | 0x200 | | 345 | blk.28.attn_v.weight | 0x23983b4c0 | 0x90000 | | 346 | blk.28.ffn_down_exps.weight | 0x2398cb4c0 | 0x8400000 | | 347 | blk.28.ffn_gate_exps.weight | 0x241ccb4c0 | 0x6c00000 | | 348 | blk.28.ffn_gate_inp.weight | 0x2488cb4c0 | 0x100000 | | 349 | blk.28.ffn_norm.weight | 0x2489cb4c0 | 0x2000 | | 350 | blk.28.ffn_up_exps.weight | 0x2489cd4c0 | 0x6c00000 | | 351 | blk.29.attn_k.weight | 0x24f5cd4c0 | 0x90000 | | 352 | blk.29.attn_k_norm.weight | 0x24f65d4c0 | 0x200 | | 353 | blk.29.attn_norm.weight | 0x24f65d6c0 | 0x2000 | | 354 | blk.29.attn_output.weight | 0x24f65f6c0 | 0x480000 | | 355 | blk.29.attn_q.weight | 0x24fadf6c0 | 0x480000 | | 356 | blk.29.attn_q_norm.weight | 0x24ff5f6c0 | 0x200 | | 357 | blk.29.attn_v.weight | 0x24ff5f8c0 | 0x90000 | | 358 | blk.29.ffn_down_exps.weight | 0x24ffef8c0 | 0x8400000 | | 359 | blk.29.ffn_gate_exps.weight | 0x2583ef8c0 | 0x6c00000 | | 360 | blk.29.ffn_gate_inp.weight | 0x25efef8c0 | 0x100000 | | 361 | blk.29.ffn_norm.weight | 0x25f0ef8c0 | 0x2000 | | 362 | blk.29.ffn_up_exps.weight | 0x25f0f18c0 | 0x6c00000 | | 363 | blk.30.attn_k.weight | 0x265cf18c0 | 0x90000 | | 364 | blk.30.attn_k_norm.weight | 0x265d818c0 | 0x200 | | 365 | blk.30.attn_norm.weight | 0x265d81ac0 | 0x2000 | | 366 | blk.30.attn_output.weight | 0x265d83ac0 | 0x480000 | | 367 | blk.30.attn_q.weight | 0x266203ac0 | 0x480000 | | 368 | blk.30.attn_q_norm.weight | 0x266683ac0 | 0x200 | | 369 | blk.30.attn_v.weight | 0x266683cc0 | 0x90000 | | 370 | blk.30.ffn_down_exps.weight | 0x266713cc0 | 0x8400000 | | 371 | blk.30.ffn_gate_exps.weight | 0x26eb13cc0 | 0x6c00000 | | 372 | blk.30.ffn_gate_inp.weight | 0x275713cc0 | 0x100000 | | 373 | blk.30.ffn_norm.weight | 0x275813cc0 | 0x2000 | | 374 | blk.30.ffn_up_exps.weight | 0x275815cc0 | 0x6c00000 | | 375 | blk.31.attn_k.weight | 0x27c415cc0 | 0x90000 | | 376 | blk.31.attn_k_norm.weight | 0x27c4a5cc0 | 0x200 | | 377 | blk.31.attn_norm.weight | 0x27c4a5ec0 | 0x2000 | | 378 | blk.31.attn_output.weight | 0x27c4a7ec0 | 0x480000 | | 379 | blk.31.attn_q.weight | 0x27c927ec0 | 0x480000 | | 380 | blk.31.attn_q_norm.weight | 0x27cda7ec0 | 0x200 | | 381 | blk.31.attn_v.weight | 0x27cda80c0 | 0x90000 | | 382 | blk.31.ffn_down_exps.weight | 0x27ce380c0 | 0x8400000 | | 383 | blk.31.ffn_gate_exps.weight | 0x2852380c0 | 0x6c00000 | | 384 | blk.31.ffn_gate_inp.weight | 0x28be380c0 | 0x100000 | | 385 | blk.31.ffn_norm.weight | 0x28bf380c0 | 0x2000 | | 386 | blk.31.ffn_up_exps.weight | 0x28bf3a0c0 | 0x6c00000 | | 387 | blk.32.attn_k.weight | 0x292b3a0c0 | 0x90000 | | 388 | blk.32.attn_k_norm.weight | 0x292bca0c0 | 0x200 | | 389 | blk.32.attn_norm.weight | 0x292bca2c0 | 0x2000 | | 390 | blk.32.attn_output.weight | 0x292bcc2c0 | 0x480000 | | 391 | blk.32.attn_q.weight | 0x29304c2c0 | 0x480000 | | 392 | blk.32.attn_q_norm.weight | 0x2934cc2c0 | 0x200 | | 393 | blk.32.attn_v.weight | 0x2934cc4c0 | 0x90000 | | 394 | blk.32.ffn_down_exps.weight | 0x29355c4c0 | 0x8400000 | | 395 | blk.32.ffn_gate_exps.weight | 0x29b95c4c0 | 0x6c00000 | | 396 | blk.32.ffn_gate_inp.weight | 0x2a255c4c0 | 0x100000 | | 397 | blk.32.ffn_norm.weight | 0x2a265c4c0 | 0x2000 | | 398 | blk.32.ffn_up_exps.weight | 0x2a265e4c0 | 0x6c00000 | | 399 | blk.33.attn_k.weight | 0x2a925e4c0 | 0x90000 | | 400 | blk.33.attn_k_norm.weight | 0x2a92ee4c0 | 0x200 | | 401 | blk.33.attn_norm.weight | 0x2a92ee6c0 | 0x2000 | | 402 | blk.33.attn_output.weight | 0x2a92f06c0 | 0x480000 | | 403 | blk.33.attn_q.weight | 0x2a97706c0 | 0x480000 | | 404 | blk.33.attn_q_norm.weight | 0x2a9bf06c0 | 0x200 | | 405 | blk.33.attn_v.weight | 0x2a9bf08c0 | 0x90000 | | 406 | blk.33.ffn_down_exps.weight | 0x2a9c808c0 | 0x8400000 | | 407 | blk.33.ffn_gate_exps.weight | 0x2b20808c0 | 0x6c00000 | | 408 | blk.33.ffn_gate_inp.weight | 0x2b8c808c0 | 0x100000 | | 409 | blk.33.ffn_norm.weight | 0x2b8d808c0 | 0x2000 | | 410 | blk.33.ffn_up_exps.weight | 0x2b8d828c0 | 0x6c00000 | | 411 | blk.34.attn_k.weight | 0x2bf9828c0 | 0x90000 | | 412 | blk.34.attn_k_norm.weight | 0x2bfa128c0 | 0x200 | | 413 | blk.34.attn_norm.weight | 0x2bfa12ac0 | 0x2000 | | 414 | blk.34.attn_output.weight | 0x2bfa14ac0 | 0x480000 | | 415 | blk.34.attn_q.weight | 0x2bfe94ac0 | 0x480000 | | 416 | blk.34.attn_q_norm.weight | 0x2c0314ac0 | 0x200 | | 417 | blk.34.attn_v.weight | 0x2c0314cc0 | 0x90000 | | 418 | blk.34.ffn_down_exps.weight | 0x2c03a4cc0 | 0x8400000 | | 419 | blk.34.ffn_gate_exps.weight | 0x2c87a4cc0 | 0x6c00000 | | 420 | blk.34.ffn_gate_inp.weight | 0x2cf3a4cc0 | 0x100000 | | 421 | blk.34.ffn_norm.weight | 0x2cf4a4cc0 | 0x2000 | | 422 | blk.34.ffn_up_exps.weight | 0x2cf4a6cc0 | 0x6c00000 | | 423 | blk.35.attn_k.weight | 0x2d60a6cc0 | 0x90000 | | 424 | blk.35.attn_k_norm.weight | 0x2d6136cc0 | 0x200 | | 425 | blk.35.attn_norm.weight | 0x2d6136ec0 | 0x2000 | | 426 | blk.35.attn_output.weight | 0x2d6138ec0 | 0x480000 | | 427 | blk.35.attn_q.weight | 0x2d65b8ec0 | 0x480000 | | 428 | blk.35.attn_q_norm.weight | 0x2d6a38ec0 | 0x200 | | 429 | blk.35.attn_v.weight | 0x2d6a390c0 | 0x90000 | | 430 | blk.35.ffn_down_exps.weight | 0x2d6ac90c0 | 0x8400000 | | 431 | blk.35.ffn_gate_exps.weight | 0x2deec90c0 | 0x6c00000 | | 432 | blk.35.ffn_gate_inp.weight | 0x2e5ac90c0 | 0x100000 | | 433 | blk.35.ffn_norm.weight | 0x2e5bc90c0 | 0x2000 | | 434 | blk.35.ffn_up_exps.weight | 0x2e5bcb0c0 | 0x6c00000 | | 435 | blk.36.attn_k.weight | 0x2ec7cb0c0 | 0x90000 | | 436 | blk.36.attn_k_norm.weight | 0x2ec85b0c0 | 0x200 | | 437 | blk.36.attn_norm.weight | 0x2ec85b2c0 | 0x2000 | | 438 | blk.36.attn_output.weight | 0x2ec85d2c0 | 0x480000 | | 439 | blk.36.attn_q.weight | 0x2eccdd2c0 | 0x480000 | | 440 | blk.36.attn_q_norm.weight | 0x2ed15d2c0 | 0x200 | | 441 | blk.36.attn_v.weight | 0x2ed15d4c0 | 0x90000 | | 442 | blk.36.ffn_down_exps.weight | 0x2ed1ed4c0 | 0x8400000 | | 443 | blk.36.ffn_gate_exps.weight | 0x2f55ed4c0 | 0x6c00000 | | 444 | blk.36.ffn_gate_inp.weight | 0x2fc1ed4c0 | 0x100000 | | 445 | blk.36.ffn_norm.weight | 0x2fc2ed4c0 | 0x2000 | | 446 | blk.36.ffn_up_exps.weight | 0x2fc2ef4c0 | 0x6c00000 | | 447 | blk.37.attn_k.weight | 0x302eef4c0 | 0x90000 | | 448 | blk.37.attn_k_norm.weight | 0x302f7f4c0 | 0x200 | | 449 | blk.37.attn_norm.weight | 0x302f7f6c0 | 0x2000 | | 450 | blk.37.attn_output.weight | 0x302f816c0 | 0x480000 | | 451 | blk.37.attn_q.weight | 0x3034016c0 | 0x480000 | | 452 | blk.37.attn_q_norm.weight | 0x3038816c0 | 0x200 | | 453 | blk.37.attn_v.weight | 0x3038818c0 | 0x90000 | | 454 | blk.37.ffn_down_exps.weight | 0x3039118c0 | 0x8400000 | | 455 | blk.37.ffn_gate_exps.weight | 0x30bd118c0 | 0x6c00000 | | 456 | blk.37.ffn_gate_inp.weight | 0x3129118c0 | 0x100000 | | 457 | blk.37.ffn_norm.weight | 0x312a118c0 | 0x2000 | | 458 | blk.37.ffn_up_exps.weight | 0x312a138c0 | 0x6c00000 | | 459 | blk.38.attn_k.weight | 0x3196138c0 | 0x90000 | | 460 | blk.38.attn_k_norm.weight | 0x3196a38c0 | 0x200 | | 461 | blk.38.attn_norm.weight | 0x3196a3ac0 | 0x2000 | | 462 | blk.38.attn_output.weight | 0x3196a5ac0 | 0x480000 | | 463 | blk.38.attn_q.weight | 0x319b25ac0 | 0x480000 | | 464 | blk.38.attn_q_norm.weight | 0x319fa5ac0 | 0x200 | | 465 | blk.38.attn_v.weight | 0x319fa5cc0 | 0x90000 | | 466 | blk.38.ffn_down_exps.weight | 0x31a035cc0 | 0x8400000 | | 467 | blk.38.ffn_gate_exps.weight | 0x322435cc0 | 0x6c00000 | | 468 | blk.38.ffn_gate_inp.weight | 0x329035cc0 | 0x100000 | | 469 | blk.38.ffn_norm.weight | 0x329135cc0 | 0x2000 | | 470 | blk.38.ffn_up_exps.weight | 0x329137cc0 | 0x6c00000 | | 471 | blk.39.attn_k.weight | 0x32fd37cc0 | 0x90000 | | 472 | blk.39.attn_k_norm.weight | 0x32fdc7cc0 | 0x200 | | 473 | blk.39.attn_norm.weight | 0x32fdc7ec0 | 0x2000 | | 474 | blk.39.attn_output.weight | 0x32fdc9ec0 | 0x480000 | | 475 | blk.39.attn_q.weight | 0x330249ec0 | 0x480000 | | 476 | blk.39.attn_q_norm.weight | 0x3306c9ec0 | 0x200 | | 477 | blk.39.attn_v.weight | 0x3306ca0c0 | 0x90000 | | 478 | blk.39.ffn_down_exps.weight | 0x33075a0c0 | 0x8400000 | | 479 | blk.39.ffn_gate_exps.weight | 0x338b5a0c0 | 0x6c00000 | | 480 | blk.39.ffn_gate_inp.weight | 0x33f75a0c0 | 0x100000 | | 481 | blk.39.ffn_norm.weight | 0x33f85a0c0 | 0x2000 | | 482 | blk.39.ffn_up_exps.weight | 0x33f85c0c0 | 0x6c00000 | | 483 | blk.40.attn_k.weight | 0x34645c0c0 | 0x90000 | | 484 | blk.40.attn_k_norm.weight | 0x3464ec0c0 | 0x200 | | 485 | blk.40.attn_norm.weight | 0x3464ec2c0 | 0x2000 | | 486 | blk.40.attn_output.weight | 0x3464ee2c0 | 0x480000 | | 487 | blk.40.attn_q.weight | 0x34696e2c0 | 0x480000 | | 488 | blk.40.attn_q_norm.weight | 0x346dee2c0 | 0x200 | | 489 | blk.40.attn_v.weight | 0x346dee4c0 | 0x90000 | | 490 | blk.40.ffn_down_exps.weight | 0x346e7e4c0 | 0x8400000 | | 491 | blk.40.ffn_gate_exps.weight | 0x34f27e4c0 | 0x6c00000 | | 492 | blk.40.ffn_gate_inp.weight | 0x355e7e4c0 | 0x100000 | | 493 | blk.40.ffn_norm.weight | 0x355f7e4c0 | 0x2000 | | 494 | blk.40.ffn_up_exps.weight | 0x355f804c0 | 0x6c00000 | | 495 | blk.41.attn_k.weight | 0x35cb804c0 | 0x90000 | | 496 | blk.41.attn_k_norm.weight | 0x35cc104c0 | 0x200 | | 497 | blk.41.attn_norm.weight | 0x35cc106c0 | 0x2000 | | 498 | blk.41.attn_output.weight | 0x35cc126c0 | 0x480000 | | 499 | blk.41.attn_q.weight | 0x35d0926c0 | 0x480000 | | 500 | blk.41.attn_q_norm.weight | 0x35d5126c0 | 0x200 | | 501 | blk.41.attn_v.weight | 0x35d5128c0 | 0x90000 | | 502 | blk.41.ffn_down_exps.weight | 0x35d5a28c0 | 0x8400000 | | 503 | blk.41.ffn_gate_exps.weight | 0x3659a28c0 | 0x6c00000 | | 504 | blk.41.ffn_gate_inp.weight | 0x36c5a28c0 | 0x100000 | | 505 | blk.41.ffn_norm.weight | 0x36c6a28c0 | 0x2000 | | 506 | blk.41.ffn_up_exps.weight | 0x36c6a48c0 | 0x6c00000 | | 507 | blk.42.attn_k.weight | 0x3732a48c0 | 0x90000 | | 508 | blk.42.attn_k_norm.weight | 0x3733348c0 | 0x200 | | 509 | blk.42.attn_norm.weight | 0x373334ac0 | 0x2000 | | 510 | blk.42.attn_output.weight | 0x373336ac0 | 0x480000 | | 511 | blk.42.attn_q.weight | 0x3737b6ac0 | 0x480000 | | 512 | blk.42.attn_q_norm.weight | 0x373c36ac0 | 0x200 | | 513 | blk.42.attn_v.weight | 0x373c36cc0 | 0x90000 | | 514 | blk.42.ffn_down_exps.weight | 0x373cc6cc0 | 0x8400000 | | 515 | blk.42.ffn_gate_exps.weight | 0x37c0c6cc0 | 0x6c00000 | | 516 | blk.42.ffn_gate_inp.weight | 0x382cc6cc0 | 0x100000 | | 517 | blk.42.ffn_norm.weight | 0x382dc6cc0 | 0x2000 | | 518 | blk.42.ffn_up_exps.weight | 0x382dc8cc0 | 0x6c00000 | | 519 | blk.43.attn_k.weight | 0x3899c8cc0 | 0x90000 | | 520 | blk.43.attn_k_norm.weight | 0x389a58cc0 | 0x200 | | 521 | blk.43.attn_norm.weight | 0x389a58ec0 | 0x2000 | | 522 | blk.43.attn_output.weight | 0x389a5aec0 | 0x480000 | | 523 | blk.43.attn_q.weight | 0x389edaec0 | 0x480000 | | 524 | blk.43.attn_q_norm.weight | 0x38a35aec0 | 0x200 | | 525 | blk.43.attn_v.weight | 0x38a35b0c0 | 0x90000 | | 526 | blk.43.ffn_down_exps.weight | 0x38a3eb0c0 | 0x8400000 | | 527 | blk.43.ffn_gate_exps.weight | 0x3927eb0c0 | 0x6c00000 | | 528 | blk.43.ffn_gate_inp.weight | 0x3993eb0c0 | 0x100000 | | 529 | blk.43.ffn_norm.weight | 0x3994eb0c0 | 0x2000 | | 530 | blk.43.ffn_up_exps.weight | 0x3994ed0c0 | 0x6c00000 | | 531 | blk.44.attn_k.weight | 0x3a00ed0c0 | 0x90000 | | 532 | blk.44.attn_k_norm.weight | 0x3a017d0c0 | 0x200 | | 533 | blk.44.attn_norm.weight | 0x3a017d2c0 | 0x2000 | | 534 | blk.44.attn_output.weight | 0x3a017f2c0 | 0x480000 | | 535 | blk.44.attn_q.weight | 0x3a05ff2c0 | 0x480000 | | 536 | blk.44.attn_q_norm.weight | 0x3a0a7f2c0 | 0x200 | | 537 | blk.44.attn_v.weight | 0x3a0a7f4c0 | 0x90000 | | 538 | blk.44.ffn_down_exps.weight | 0x3a0b0f4c0 | 0x8400000 | | 539 | blk.44.ffn_gate_exps.weight | 0x3a8f0f4c0 | 0x6c00000 | | 540 | blk.44.ffn_gate_inp.weight | 0x3afb0f4c0 | 0x100000 | | 541 | blk.44.ffn_norm.weight | 0x3afc0f4c0 | 0x2000 | | 542 | blk.44.ffn_up_exps.weight | 0x3afc114c0 | 0x6c00000 | | 543 | blk.45.attn_k.weight | 0x3b68114c0 | 0x90000 | | 544 | blk.45.attn_k_norm.weight | 0x3b68a14c0 | 0x200 | | 545 | blk.45.attn_norm.weight | 0x3b68a16c0 | 0x2000 | | 546 | blk.45.attn_output.weight | 0x3b68a36c0 | 0x480000 | | 547 | blk.45.attn_q.weight | 0x3b6d236c0 | 0x480000 | | 548 | blk.45.attn_q_norm.weight | 0x3b71a36c0 | 0x200 | | 549 | blk.45.attn_v.weight | 0x3b71a38c0 | 0x90000 | | 550 | blk.45.ffn_down_exps.weight | 0x3b72338c0 | 0x8400000 | | 551 | blk.45.ffn_gate_exps.weight | 0x3bf6338c0 | 0x6c00000 | | 552 | blk.45.ffn_gate_inp.weight | 0x3c62338c0 | 0x100000 | | 553 | blk.45.ffn_norm.weight | 0x3c63338c0 | 0x2000 | | 554 | blk.45.ffn_up_exps.weight | 0x3c63358c0 | 0x6c00000 | | 555 | blk.46.attn_k.weight | 0x3ccf358c0 | 0x90000 | | 556 | blk.46.attn_k_norm.weight | 0x3ccfc58c0 | 0x200 | | 557 | blk.46.attn_norm.weight | 0x3ccfc5ac0 | 0x2000 | | 558 | blk.46.attn_output.weight | 0x3ccfc7ac0 | 0x480000 | | 559 | blk.46.attn_q.weight | 0x3cd447ac0 | 0x480000 | | 560 | blk.46.attn_q_norm.weight | 0x3cd8c7ac0 | 0x200 | | 561 | blk.46.attn_v.weight | 0x3cd8c7cc0 | 0x90000 | | 562 | blk.46.ffn_down_exps.weight | 0x3cd957cc0 | 0x8400000 | | 563 | blk.46.ffn_gate_exps.weight | 0x3d5d57cc0 | 0x6c00000 | | 564 | blk.46.ffn_gate_inp.weight | 0x3dc957cc0 | 0x100000 | | 565 | blk.46.ffn_norm.weight | 0x3dca57cc0 | 0x2000 | | 566 | blk.46.ffn_up_exps.weight | 0x3dca59cc0 | 0x6c00000 | | 567 | blk.47.attn_k.weight | 0x3e3659cc0 | 0x90000 | | 568 | blk.47.attn_k_norm.weight | 0x3e36e9cc0 | 0x200 | | 569 | blk.47.attn_norm.weight | 0x3e36e9ec0 | 0x2000 | | 570 | blk.47.attn_output.weight | 0x3e36ebec0 | 0x480000 | | 571 | blk.47.attn_q.weight | 0x3e3b6bec0 | 0x480000 | | 572 | blk.47.attn_q_norm.weight | 0x3e3febec0 | 0x200 | | 573 | blk.47.attn_v.weight | 0x3e3fec0c0 | 0x90000 | | 574 | blk.47.ffn_down_exps.weight | 0x3e407c0c0 | 0x8400000 | | 575 | blk.47.ffn_gate_exps.weight | 0x3ec47c0c0 | 0x6c00000 | | 576 | blk.47.ffn_gate_inp.weight | 0x3f307c0c0 | 0x100000 | | 577 | blk.47.ffn_norm.weight | 0x3f317c0c0 | 0x2000 | | 578 | blk.47.ffn_up_exps.weight | 0x3f317e0c0 | 0x6c00000 | ### Base Tensor Group : ~622M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:-------------------|:---------------------------------|:------------------|:----------------------|:-------| | 0 | output.weight | Output (W) | (~311M) 311164928 | 2048 x 151936 x 1 x 1 | IQ4_NL | | 1 | output_norm.weight | Output Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 2 | token_embd.weight | Token Embedding (W) | (~311M) 311164928 | 2048 x 151936 x 1 x 1 | IQ3_S | - Total elements in base: (~622M) 622331904 - Percentage of total elements: 2.04% ### Block 0 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:---------------------------|:------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 3 | blk.0.attn_k.weight | Block 0 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ3_S | | 4 | blk.0.attn_k_norm.weight | Block 0 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 5 | blk.0.attn_norm.weight | Block 0 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 6 | blk.0.attn_output.weight | Block 0 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 7 | blk.0.attn_q.weight | Block 0 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ3_S | | 8 | blk.0.attn_q_norm.weight | Block 0 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 9 | blk.0.attn_v.weight | Block 0 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_XS | | 10 | blk.0.ffn_down_exps.weight | Block 0 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 11 | blk.0.ffn_gate_exps.weight | Block 0 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | | 12 | blk.0.ffn_gate_inp.weight | Block 0 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 13 | blk.0.ffn_norm.weight | Block 0 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 14 | blk.0.ffn_up_exps.weight | Block 0 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | - Total elements in blk.0: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 1 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:---------------------------|:------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 15 | blk.1.attn_k.weight | Block 1 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ3_S | | 16 | blk.1.attn_k_norm.weight | Block 1 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 17 | blk.1.attn_norm.weight | Block 1 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 18 | blk.1.attn_output.weight | Block 1 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 19 | blk.1.attn_q.weight | Block 1 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ3_S | | 20 | blk.1.attn_q_norm.weight | Block 1 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 21 | blk.1.attn_v.weight | Block 1 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_XS | | 22 | blk.1.ffn_down_exps.weight | Block 1 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 23 | blk.1.ffn_gate_exps.weight | Block 1 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | | 24 | blk.1.ffn_gate_inp.weight | Block 1 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 25 | blk.1.ffn_norm.weight | Block 1 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 26 | blk.1.ffn_up_exps.weight | Block 1 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | - Total elements in blk.1: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 2 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:---------------------------|:------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 27 | blk.2.attn_k.weight | Block 2 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ3_S | | 28 | blk.2.attn_k_norm.weight | Block 2 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 29 | blk.2.attn_norm.weight | Block 2 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 30 | blk.2.attn_output.weight | Block 2 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 31 | blk.2.attn_q.weight | Block 2 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ3_S | | 32 | blk.2.attn_q_norm.weight | Block 2 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 33 | blk.2.attn_v.weight | Block 2 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_XS | | 34 | blk.2.ffn_down_exps.weight | Block 2 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 35 | blk.2.ffn_gate_exps.weight | Block 2 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | | 36 | blk.2.ffn_gate_inp.weight | Block 2 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 37 | blk.2.ffn_norm.weight | Block 2 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 38 | blk.2.ffn_up_exps.weight | Block 2 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | - Total elements in blk.2: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 3 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:---------------------------|:------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 39 | blk.3.attn_k.weight | Block 3 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ3_S | | 40 | blk.3.attn_k_norm.weight | Block 3 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 41 | blk.3.attn_norm.weight | Block 3 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 42 | blk.3.attn_output.weight | Block 3 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 43 | blk.3.attn_q.weight | Block 3 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ3_S | | 44 | blk.3.attn_q_norm.weight | Block 3 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 45 | blk.3.attn_v.weight | Block 3 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_XS | | 46 | blk.3.ffn_down_exps.weight | Block 3 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 47 | blk.3.ffn_gate_exps.weight | Block 3 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | | 48 | blk.3.ffn_gate_inp.weight | Block 3 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 49 | blk.3.ffn_norm.weight | Block 3 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 50 | blk.3.ffn_up_exps.weight | Block 3 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | - Total elements in blk.3: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 4 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:---------------------------|:------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 51 | blk.4.attn_k.weight | Block 4 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ3_S | | 52 | blk.4.attn_k_norm.weight | Block 4 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 53 | blk.4.attn_norm.weight | Block 4 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 54 | blk.4.attn_output.weight | Block 4 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 55 | blk.4.attn_q.weight | Block 4 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ3_S | | 56 | blk.4.attn_q_norm.weight | Block 4 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 57 | blk.4.attn_v.weight | Block 4 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_XS | | 58 | blk.4.ffn_down_exps.weight | Block 4 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 59 | blk.4.ffn_gate_exps.weight | Block 4 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | | 60 | blk.4.ffn_gate_inp.weight | Block 4 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 61 | blk.4.ffn_norm.weight | Block 4 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 62 | blk.4.ffn_up_exps.weight | Block 4 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | - Total elements in blk.4: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 5 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:---------------------------|:------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 63 | blk.5.attn_k.weight | Block 5 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ3_S | | 64 | blk.5.attn_k_norm.weight | Block 5 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 65 | blk.5.attn_norm.weight | Block 5 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 66 | blk.5.attn_output.weight | Block 5 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 67 | blk.5.attn_q.weight | Block 5 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ3_S | | 68 | blk.5.attn_q_norm.weight | Block 5 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 69 | blk.5.attn_v.weight | Block 5 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_XS | | 70 | blk.5.ffn_down_exps.weight | Block 5 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 71 | blk.5.ffn_gate_exps.weight | Block 5 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | | 72 | blk.5.ffn_gate_inp.weight | Block 5 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 73 | blk.5.ffn_norm.weight | Block 5 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 74 | blk.5.ffn_up_exps.weight | Block 5 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | - Total elements in blk.5: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 6 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:---------------------------|:------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 75 | blk.6.attn_k.weight | Block 6 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ3_S | | 76 | blk.6.attn_k_norm.weight | Block 6 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 77 | blk.6.attn_norm.weight | Block 6 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 78 | blk.6.attn_output.weight | Block 6 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 79 | blk.6.attn_q.weight | Block 6 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ3_S | | 80 | blk.6.attn_q_norm.weight | Block 6 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 81 | blk.6.attn_v.weight | Block 6 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_XS | | 82 | blk.6.ffn_down_exps.weight | Block 6 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 83 | blk.6.ffn_gate_exps.weight | Block 6 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | | 84 | blk.6.ffn_gate_inp.weight | Block 6 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 85 | blk.6.ffn_norm.weight | Block 6 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 86 | blk.6.ffn_up_exps.weight | Block 6 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | - Total elements in blk.6: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 7 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:---------------------------|:------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 87 | blk.7.attn_k.weight | Block 7 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ3_S | | 88 | blk.7.attn_k_norm.weight | Block 7 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 89 | blk.7.attn_norm.weight | Block 7 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 90 | blk.7.attn_output.weight | Block 7 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 91 | blk.7.attn_q.weight | Block 7 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ3_S | | 92 | blk.7.attn_q_norm.weight | Block 7 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 93 | blk.7.attn_v.weight | Block 7 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_XS | | 94 | blk.7.ffn_down_exps.weight | Block 7 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 95 | blk.7.ffn_gate_exps.weight | Block 7 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | | 96 | blk.7.ffn_gate_inp.weight | Block 7 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 97 | blk.7.ffn_norm.weight | Block 7 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 98 | blk.7.ffn_up_exps.weight | Block 7 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | - Total elements in blk.7: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 8 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:---------------------------|:------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 99 | blk.8.attn_k.weight | Block 8 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ3_S | | 100 | blk.8.attn_k_norm.weight | Block 8 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 101 | blk.8.attn_norm.weight | Block 8 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 102 | blk.8.attn_output.weight | Block 8 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 103 | blk.8.attn_q.weight | Block 8 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ3_S | | 104 | blk.8.attn_q_norm.weight | Block 8 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 105 | blk.8.attn_v.weight | Block 8 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_XS | | 106 | blk.8.ffn_down_exps.weight | Block 8 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 107 | blk.8.ffn_gate_exps.weight | Block 8 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | | 108 | blk.8.ffn_gate_inp.weight | Block 8 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 109 | blk.8.ffn_norm.weight | Block 8 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 110 | blk.8.ffn_up_exps.weight | Block 8 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | - Total elements in blk.8: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 9 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:---------------------------|:------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 111 | blk.9.attn_k.weight | Block 9 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ3_S | | 112 | blk.9.attn_k_norm.weight | Block 9 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 113 | blk.9.attn_norm.weight | Block 9 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 114 | blk.9.attn_output.weight | Block 9 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 115 | blk.9.attn_q.weight | Block 9 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ3_S | | 116 | blk.9.attn_q_norm.weight | Block 9 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 117 | blk.9.attn_v.weight | Block 9 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_XS | | 118 | blk.9.ffn_down_exps.weight | Block 9 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 119 | blk.9.ffn_gate_exps.weight | Block 9 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | | 120 | blk.9.ffn_gate_inp.weight | Block 9 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 121 | blk.9.ffn_norm.weight | Block 9 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 122 | blk.9.ffn_up_exps.weight | Block 9 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | - Total elements in blk.9: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 10 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 123 | blk.10.attn_k.weight | Block 10 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ3_S | | 124 | blk.10.attn_k_norm.weight | Block 10 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 125 | blk.10.attn_norm.weight | Block 10 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 126 | blk.10.attn_output.weight | Block 10 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 127 | blk.10.attn_q.weight | Block 10 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ3_S | | 128 | blk.10.attn_q_norm.weight | Block 10 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 129 | blk.10.attn_v.weight | Block 10 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_XS | | 130 | blk.10.ffn_down_exps.weight | Block 10 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 131 | blk.10.ffn_gate_exps.weight | Block 10 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | | 132 | blk.10.ffn_gate_inp.weight | Block 10 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 133 | blk.10.ffn_norm.weight | Block 10 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 134 | blk.10.ffn_up_exps.weight | Block 10 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | - Total elements in blk.10: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 11 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 135 | blk.11.attn_k.weight | Block 11 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ3_S | | 136 | blk.11.attn_k_norm.weight | Block 11 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 137 | blk.11.attn_norm.weight | Block 11 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 138 | blk.11.attn_output.weight | Block 11 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 139 | blk.11.attn_q.weight | Block 11 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ3_S | | 140 | blk.11.attn_q_norm.weight | Block 11 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 141 | blk.11.attn_v.weight | Block 11 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_XS | | 142 | blk.11.ffn_down_exps.weight | Block 11 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 143 | blk.11.ffn_gate_exps.weight | Block 11 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | | 144 | blk.11.ffn_gate_inp.weight | Block 11 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 145 | blk.11.ffn_norm.weight | Block 11 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 146 | blk.11.ffn_up_exps.weight | Block 11 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | - Total elements in blk.11: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 12 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 147 | blk.12.attn_k.weight | Block 12 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ3_S | | 148 | blk.12.attn_k_norm.weight | Block 12 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 149 | blk.12.attn_norm.weight | Block 12 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 150 | blk.12.attn_output.weight | Block 12 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 151 | blk.12.attn_q.weight | Block 12 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ3_S | | 152 | blk.12.attn_q_norm.weight | Block 12 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 153 | blk.12.attn_v.weight | Block 12 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_XS | | 154 | blk.12.ffn_down_exps.weight | Block 12 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 155 | blk.12.ffn_gate_exps.weight | Block 12 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | | 156 | blk.12.ffn_gate_inp.weight | Block 12 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 157 | blk.12.ffn_norm.weight | Block 12 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 158 | blk.12.ffn_up_exps.weight | Block 12 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | - Total elements in blk.12: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 13 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 159 | blk.13.attn_k.weight | Block 13 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ3_S | | 160 | blk.13.attn_k_norm.weight | Block 13 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 161 | blk.13.attn_norm.weight | Block 13 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 162 | blk.13.attn_output.weight | Block 13 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 163 | blk.13.attn_q.weight | Block 13 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ3_S | | 164 | blk.13.attn_q_norm.weight | Block 13 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 165 | blk.13.attn_v.weight | Block 13 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_XS | | 166 | blk.13.ffn_down_exps.weight | Block 13 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 167 | blk.13.ffn_gate_exps.weight | Block 13 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | | 168 | blk.13.ffn_gate_inp.weight | Block 13 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 169 | blk.13.ffn_norm.weight | Block 13 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 170 | blk.13.ffn_up_exps.weight | Block 13 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | - Total elements in blk.13: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 14 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 171 | blk.14.attn_k.weight | Block 14 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ3_S | | 172 | blk.14.attn_k_norm.weight | Block 14 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 173 | blk.14.attn_norm.weight | Block 14 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 174 | blk.14.attn_output.weight | Block 14 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 175 | blk.14.attn_q.weight | Block 14 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ3_S | | 176 | blk.14.attn_q_norm.weight | Block 14 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 177 | blk.14.attn_v.weight | Block 14 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_XS | | 178 | blk.14.ffn_down_exps.weight | Block 14 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 179 | blk.14.ffn_gate_exps.weight | Block 14 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | | 180 | blk.14.ffn_gate_inp.weight | Block 14 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 181 | blk.14.ffn_norm.weight | Block 14 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 182 | blk.14.ffn_up_exps.weight | Block 14 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | - Total elements in blk.14: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 15 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 183 | blk.15.attn_k.weight | Block 15 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ3_S | | 184 | blk.15.attn_k_norm.weight | Block 15 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 185 | blk.15.attn_norm.weight | Block 15 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 186 | blk.15.attn_output.weight | Block 15 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 187 | blk.15.attn_q.weight | Block 15 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ3_S | | 188 | blk.15.attn_q_norm.weight | Block 15 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 189 | blk.15.attn_v.weight | Block 15 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_XS | | 190 | blk.15.ffn_down_exps.weight | Block 15 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 191 | blk.15.ffn_gate_exps.weight | Block 15 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | | 192 | blk.15.ffn_gate_inp.weight | Block 15 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 193 | blk.15.ffn_norm.weight | Block 15 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 194 | blk.15.ffn_up_exps.weight | Block 15 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | - Total elements in blk.15: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 16 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 195 | blk.16.attn_k.weight | Block 16 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ3_S | | 196 | blk.16.attn_k_norm.weight | Block 16 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 197 | blk.16.attn_norm.weight | Block 16 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 198 | blk.16.attn_output.weight | Block 16 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 199 | blk.16.attn_q.weight | Block 16 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ3_S | | 200 | blk.16.attn_q_norm.weight | Block 16 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 201 | blk.16.attn_v.weight | Block 16 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_XS | | 202 | blk.16.ffn_down_exps.weight | Block 16 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 203 | blk.16.ffn_gate_exps.weight | Block 16 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | | 204 | blk.16.ffn_gate_inp.weight | Block 16 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 205 | blk.16.ffn_norm.weight | Block 16 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 206 | blk.16.ffn_up_exps.weight | Block 16 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | - Total elements in blk.16: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 17 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 207 | blk.17.attn_k.weight | Block 17 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ3_S | | 208 | blk.17.attn_k_norm.weight | Block 17 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 209 | blk.17.attn_norm.weight | Block 17 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 210 | blk.17.attn_output.weight | Block 17 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 211 | blk.17.attn_q.weight | Block 17 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ3_S | | 212 | blk.17.attn_q_norm.weight | Block 17 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 213 | blk.17.attn_v.weight | Block 17 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_XS | | 214 | blk.17.ffn_down_exps.weight | Block 17 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 215 | blk.17.ffn_gate_exps.weight | Block 17 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | | 216 | blk.17.ffn_gate_inp.weight | Block 17 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 217 | blk.17.ffn_norm.weight | Block 17 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 218 | blk.17.ffn_up_exps.weight | Block 17 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | - Total elements in blk.17: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 18 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 219 | blk.18.attn_k.weight | Block 18 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ3_S | | 220 | blk.18.attn_k_norm.weight | Block 18 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 221 | blk.18.attn_norm.weight | Block 18 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 222 | blk.18.attn_output.weight | Block 18 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 223 | blk.18.attn_q.weight | Block 18 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ3_S | | 224 | blk.18.attn_q_norm.weight | Block 18 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 225 | blk.18.attn_v.weight | Block 18 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_XS | | 226 | blk.18.ffn_down_exps.weight | Block 18 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 227 | blk.18.ffn_gate_exps.weight | Block 18 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | | 228 | blk.18.ffn_gate_inp.weight | Block 18 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 229 | blk.18.ffn_norm.weight | Block 18 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 230 | blk.18.ffn_up_exps.weight | Block 18 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | - Total elements in blk.18: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 19 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 231 | blk.19.attn_k.weight | Block 19 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ3_S | | 232 | blk.19.attn_k_norm.weight | Block 19 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 233 | blk.19.attn_norm.weight | Block 19 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 234 | blk.19.attn_output.weight | Block 19 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 235 | blk.19.attn_q.weight | Block 19 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ3_S | | 236 | blk.19.attn_q_norm.weight | Block 19 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 237 | blk.19.attn_v.weight | Block 19 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_XS | | 238 | blk.19.ffn_down_exps.weight | Block 19 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 239 | blk.19.ffn_gate_exps.weight | Block 19 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | | 240 | blk.19.ffn_gate_inp.weight | Block 19 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 241 | blk.19.ffn_norm.weight | Block 19 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 242 | blk.19.ffn_up_exps.weight | Block 19 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | - Total elements in blk.19: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 20 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 243 | blk.20.attn_k.weight | Block 20 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ3_S | | 244 | blk.20.attn_k_norm.weight | Block 20 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 245 | blk.20.attn_norm.weight | Block 20 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 246 | blk.20.attn_output.weight | Block 20 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 247 | blk.20.attn_q.weight | Block 20 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ3_S | | 248 | blk.20.attn_q_norm.weight | Block 20 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 249 | blk.20.attn_v.weight | Block 20 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_XS | | 250 | blk.20.ffn_down_exps.weight | Block 20 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 251 | blk.20.ffn_gate_exps.weight | Block 20 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | | 252 | blk.20.ffn_gate_inp.weight | Block 20 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 253 | blk.20.ffn_norm.weight | Block 20 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 254 | blk.20.ffn_up_exps.weight | Block 20 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | - Total elements in blk.20: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 21 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 255 | blk.21.attn_k.weight | Block 21 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ3_S | | 256 | blk.21.attn_k_norm.weight | Block 21 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 257 | blk.21.attn_norm.weight | Block 21 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 258 | blk.21.attn_output.weight | Block 21 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 259 | blk.21.attn_q.weight | Block 21 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ3_S | | 260 | blk.21.attn_q_norm.weight | Block 21 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 261 | blk.21.attn_v.weight | Block 21 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_XS | | 262 | blk.21.ffn_down_exps.weight | Block 21 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 263 | blk.21.ffn_gate_exps.weight | Block 21 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | | 264 | blk.21.ffn_gate_inp.weight | Block 21 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 265 | blk.21.ffn_norm.weight | Block 21 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 266 | blk.21.ffn_up_exps.weight | Block 21 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | - Total elements in blk.21: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 22 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 267 | blk.22.attn_k.weight | Block 22 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ3_S | | 268 | blk.22.attn_k_norm.weight | Block 22 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 269 | blk.22.attn_norm.weight | Block 22 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 270 | blk.22.attn_output.weight | Block 22 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 271 | blk.22.attn_q.weight | Block 22 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ3_S | | 272 | blk.22.attn_q_norm.weight | Block 22 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 273 | blk.22.attn_v.weight | Block 22 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_XS | | 274 | blk.22.ffn_down_exps.weight | Block 22 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 275 | blk.22.ffn_gate_exps.weight | Block 22 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | | 276 | blk.22.ffn_gate_inp.weight | Block 22 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 277 | blk.22.ffn_norm.weight | Block 22 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 278 | blk.22.ffn_up_exps.weight | Block 22 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | - Total elements in blk.22: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 23 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 279 | blk.23.attn_k.weight | Block 23 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ3_S | | 280 | blk.23.attn_k_norm.weight | Block 23 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 281 | blk.23.attn_norm.weight | Block 23 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 282 | blk.23.attn_output.weight | Block 23 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 283 | blk.23.attn_q.weight | Block 23 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ3_S | | 284 | blk.23.attn_q_norm.weight | Block 23 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 285 | blk.23.attn_v.weight | Block 23 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_XS | | 286 | blk.23.ffn_down_exps.weight | Block 23 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 287 | blk.23.ffn_gate_exps.weight | Block 23 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | | 288 | blk.23.ffn_gate_inp.weight | Block 23 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 289 | blk.23.ffn_norm.weight | Block 23 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 290 | blk.23.ffn_up_exps.weight | Block 23 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | - Total elements in blk.23: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 24 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 291 | blk.24.attn_k.weight | Block 24 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 292 | blk.24.attn_k_norm.weight | Block 24 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 293 | blk.24.attn_norm.weight | Block 24 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 294 | blk.24.attn_output.weight | Block 24 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 295 | blk.24.attn_q.weight | Block 24 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ4_NL | | 296 | blk.24.attn_q_norm.weight | Block 24 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 297 | blk.24.attn_v.weight | Block 24 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 298 | blk.24.ffn_down_exps.weight | Block 24 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 299 | blk.24.ffn_gate_exps.weight | Block 24 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | | 300 | blk.24.ffn_gate_inp.weight | Block 24 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 301 | blk.24.ffn_norm.weight | Block 24 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 302 | blk.24.ffn_up_exps.weight | Block 24 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | - Total elements in blk.24: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 25 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 303 | blk.25.attn_k.weight | Block 25 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 304 | blk.25.attn_k_norm.weight | Block 25 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 305 | blk.25.attn_norm.weight | Block 25 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 306 | blk.25.attn_output.weight | Block 25 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 307 | blk.25.attn_q.weight | Block 25 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ4_NL | | 308 | blk.25.attn_q_norm.weight | Block 25 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 309 | blk.25.attn_v.weight | Block 25 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 310 | blk.25.ffn_down_exps.weight | Block 25 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 311 | blk.25.ffn_gate_exps.weight | Block 25 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | | 312 | blk.25.ffn_gate_inp.weight | Block 25 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 313 | blk.25.ffn_norm.weight | Block 25 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 314 | blk.25.ffn_up_exps.weight | Block 25 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | - Total elements in blk.25: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 26 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 315 | blk.26.attn_k.weight | Block 26 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 316 | blk.26.attn_k_norm.weight | Block 26 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 317 | blk.26.attn_norm.weight | Block 26 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 318 | blk.26.attn_output.weight | Block 26 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 319 | blk.26.attn_q.weight | Block 26 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ4_NL | | 320 | blk.26.attn_q_norm.weight | Block 26 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 321 | blk.26.attn_v.weight | Block 26 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 322 | blk.26.ffn_down_exps.weight | Block 26 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 323 | blk.26.ffn_gate_exps.weight | Block 26 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | | 324 | blk.26.ffn_gate_inp.weight | Block 26 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 325 | blk.26.ffn_norm.weight | Block 26 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 326 | blk.26.ffn_up_exps.weight | Block 26 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ3_S | - Total elements in blk.26: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 27 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 327 | blk.27.attn_k.weight | Block 27 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 328 | blk.27.attn_k_norm.weight | Block 27 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 329 | blk.27.attn_norm.weight | Block 27 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 330 | blk.27.attn_output.weight | Block 27 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 331 | blk.27.attn_q.weight | Block 27 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ4_NL | | 332 | blk.27.attn_q_norm.weight | Block 27 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 333 | blk.27.attn_v.weight | Block 27 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 334 | blk.27.ffn_down_exps.weight | Block 27 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 335 | blk.27.ffn_gate_exps.weight | Block 27 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | | 336 | blk.27.ffn_gate_inp.weight | Block 27 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 337 | blk.27.ffn_norm.weight | Block 27 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 338 | blk.27.ffn_up_exps.weight | Block 27 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | - Total elements in blk.27: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 28 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 339 | blk.28.attn_k.weight | Block 28 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 340 | blk.28.attn_k_norm.weight | Block 28 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 341 | blk.28.attn_norm.weight | Block 28 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 342 | blk.28.attn_output.weight | Block 28 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 343 | blk.28.attn_q.weight | Block 28 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ4_NL | | 344 | blk.28.attn_q_norm.weight | Block 28 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 345 | blk.28.attn_v.weight | Block 28 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 346 | blk.28.ffn_down_exps.weight | Block 28 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 347 | blk.28.ffn_gate_exps.weight | Block 28 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | | 348 | blk.28.ffn_gate_inp.weight | Block 28 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 349 | blk.28.ffn_norm.weight | Block 28 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 350 | blk.28.ffn_up_exps.weight | Block 28 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | - Total elements in blk.28: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 29 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 351 | blk.29.attn_k.weight | Block 29 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 352 | blk.29.attn_k_norm.weight | Block 29 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 353 | blk.29.attn_norm.weight | Block 29 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 354 | blk.29.attn_output.weight | Block 29 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 355 | blk.29.attn_q.weight | Block 29 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ4_NL | | 356 | blk.29.attn_q_norm.weight | Block 29 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 357 | blk.29.attn_v.weight | Block 29 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 358 | blk.29.ffn_down_exps.weight | Block 29 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 359 | blk.29.ffn_gate_exps.weight | Block 29 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | | 360 | blk.29.ffn_gate_inp.weight | Block 29 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 361 | blk.29.ffn_norm.weight | Block 29 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 362 | blk.29.ffn_up_exps.weight | Block 29 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | - Total elements in blk.29: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 30 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 363 | blk.30.attn_k.weight | Block 30 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 364 | blk.30.attn_k_norm.weight | Block 30 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 365 | blk.30.attn_norm.weight | Block 30 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 366 | blk.30.attn_output.weight | Block 30 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 367 | blk.30.attn_q.weight | Block 30 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ4_NL | | 368 | blk.30.attn_q_norm.weight | Block 30 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 369 | blk.30.attn_v.weight | Block 30 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 370 | blk.30.ffn_down_exps.weight | Block 30 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 371 | blk.30.ffn_gate_exps.weight | Block 30 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | | 372 | blk.30.ffn_gate_inp.weight | Block 30 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 373 | blk.30.ffn_norm.weight | Block 30 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 374 | blk.30.ffn_up_exps.weight | Block 30 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | - Total elements in blk.30: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 31 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 375 | blk.31.attn_k.weight | Block 31 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 376 | blk.31.attn_k_norm.weight | Block 31 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 377 | blk.31.attn_norm.weight | Block 31 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 378 | blk.31.attn_output.weight | Block 31 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 379 | blk.31.attn_q.weight | Block 31 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ4_NL | | 380 | blk.31.attn_q_norm.weight | Block 31 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 381 | blk.31.attn_v.weight | Block 31 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 382 | blk.31.ffn_down_exps.weight | Block 31 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 383 | blk.31.ffn_gate_exps.weight | Block 31 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | | 384 | blk.31.ffn_gate_inp.weight | Block 31 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 385 | blk.31.ffn_norm.weight | Block 31 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 386 | blk.31.ffn_up_exps.weight | Block 31 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | - Total elements in blk.31: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 32 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 387 | blk.32.attn_k.weight | Block 32 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 388 | blk.32.attn_k_norm.weight | Block 32 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 389 | blk.32.attn_norm.weight | Block 32 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 390 | blk.32.attn_output.weight | Block 32 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 391 | blk.32.attn_q.weight | Block 32 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ4_NL | | 392 | blk.32.attn_q_norm.weight | Block 32 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 393 | blk.32.attn_v.weight | Block 32 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 394 | blk.32.ffn_down_exps.weight | Block 32 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 395 | blk.32.ffn_gate_exps.weight | Block 32 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | | 396 | blk.32.ffn_gate_inp.weight | Block 32 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 397 | blk.32.ffn_norm.weight | Block 32 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 398 | blk.32.ffn_up_exps.weight | Block 32 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | - Total elements in blk.32: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 33 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 399 | blk.33.attn_k.weight | Block 33 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 400 | blk.33.attn_k_norm.weight | Block 33 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 401 | blk.33.attn_norm.weight | Block 33 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 402 | blk.33.attn_output.weight | Block 33 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 403 | blk.33.attn_q.weight | Block 33 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ4_NL | | 404 | blk.33.attn_q_norm.weight | Block 33 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 405 | blk.33.attn_v.weight | Block 33 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 406 | blk.33.ffn_down_exps.weight | Block 33 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 407 | blk.33.ffn_gate_exps.weight | Block 33 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | | 408 | blk.33.ffn_gate_inp.weight | Block 33 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 409 | blk.33.ffn_norm.weight | Block 33 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 410 | blk.33.ffn_up_exps.weight | Block 33 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | - Total elements in blk.33: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 34 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 411 | blk.34.attn_k.weight | Block 34 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 412 | blk.34.attn_k_norm.weight | Block 34 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 413 | blk.34.attn_norm.weight | Block 34 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 414 | blk.34.attn_output.weight | Block 34 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 415 | blk.34.attn_q.weight | Block 34 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ4_NL | | 416 | blk.34.attn_q_norm.weight | Block 34 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 417 | blk.34.attn_v.weight | Block 34 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 418 | blk.34.ffn_down_exps.weight | Block 34 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 419 | blk.34.ffn_gate_exps.weight | Block 34 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | | 420 | blk.34.ffn_gate_inp.weight | Block 34 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 421 | blk.34.ffn_norm.weight | Block 34 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 422 | blk.34.ffn_up_exps.weight | Block 34 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | - Total elements in blk.34: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 35 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 423 | blk.35.attn_k.weight | Block 35 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 424 | blk.35.attn_k_norm.weight | Block 35 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 425 | blk.35.attn_norm.weight | Block 35 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 426 | blk.35.attn_output.weight | Block 35 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 427 | blk.35.attn_q.weight | Block 35 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ4_NL | | 428 | blk.35.attn_q_norm.weight | Block 35 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 429 | blk.35.attn_v.weight | Block 35 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 430 | blk.35.ffn_down_exps.weight | Block 35 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 431 | blk.35.ffn_gate_exps.weight | Block 35 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | | 432 | blk.35.ffn_gate_inp.weight | Block 35 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 433 | blk.35.ffn_norm.weight | Block 35 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 434 | blk.35.ffn_up_exps.weight | Block 35 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | - Total elements in blk.35: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 36 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 435 | blk.36.attn_k.weight | Block 36 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 436 | blk.36.attn_k_norm.weight | Block 36 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 437 | blk.36.attn_norm.weight | Block 36 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 438 | blk.36.attn_output.weight | Block 36 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 439 | blk.36.attn_q.weight | Block 36 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ4_NL | | 440 | blk.36.attn_q_norm.weight | Block 36 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 441 | blk.36.attn_v.weight | Block 36 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 442 | blk.36.ffn_down_exps.weight | Block 36 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 443 | blk.36.ffn_gate_exps.weight | Block 36 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | | 444 | blk.36.ffn_gate_inp.weight | Block 36 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 445 | blk.36.ffn_norm.weight | Block 36 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 446 | blk.36.ffn_up_exps.weight | Block 36 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | - Total elements in blk.36: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 37 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 447 | blk.37.attn_k.weight | Block 37 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 448 | blk.37.attn_k_norm.weight | Block 37 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 449 | blk.37.attn_norm.weight | Block 37 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 450 | blk.37.attn_output.weight | Block 37 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 451 | blk.37.attn_q.weight | Block 37 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ4_NL | | 452 | blk.37.attn_q_norm.weight | Block 37 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 453 | blk.37.attn_v.weight | Block 37 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 454 | blk.37.ffn_down_exps.weight | Block 37 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 455 | blk.37.ffn_gate_exps.weight | Block 37 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | | 456 | blk.37.ffn_gate_inp.weight | Block 37 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 457 | blk.37.ffn_norm.weight | Block 37 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 458 | blk.37.ffn_up_exps.weight | Block 37 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | - Total elements in blk.37: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 38 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 459 | blk.38.attn_k.weight | Block 38 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 460 | blk.38.attn_k_norm.weight | Block 38 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 461 | blk.38.attn_norm.weight | Block 38 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 462 | blk.38.attn_output.weight | Block 38 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 463 | blk.38.attn_q.weight | Block 38 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ4_NL | | 464 | blk.38.attn_q_norm.weight | Block 38 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 465 | blk.38.attn_v.weight | Block 38 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 466 | blk.38.ffn_down_exps.weight | Block 38 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 467 | blk.38.ffn_gate_exps.weight | Block 38 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | | 468 | blk.38.ffn_gate_inp.weight | Block 38 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 469 | blk.38.ffn_norm.weight | Block 38 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 470 | blk.38.ffn_up_exps.weight | Block 38 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | - Total elements in blk.38: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 39 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 471 | blk.39.attn_k.weight | Block 39 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 472 | blk.39.attn_k_norm.weight | Block 39 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 473 | blk.39.attn_norm.weight | Block 39 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 474 | blk.39.attn_output.weight | Block 39 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 475 | blk.39.attn_q.weight | Block 39 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ4_NL | | 476 | blk.39.attn_q_norm.weight | Block 39 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 477 | blk.39.attn_v.weight | Block 39 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 478 | blk.39.ffn_down_exps.weight | Block 39 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 479 | blk.39.ffn_gate_exps.weight | Block 39 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | | 480 | blk.39.ffn_gate_inp.weight | Block 39 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 481 | blk.39.ffn_norm.weight | Block 39 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 482 | blk.39.ffn_up_exps.weight | Block 39 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | - Total elements in blk.39: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 40 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 483 | blk.40.attn_k.weight | Block 40 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 484 | blk.40.attn_k_norm.weight | Block 40 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 485 | blk.40.attn_norm.weight | Block 40 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 486 | blk.40.attn_output.weight | Block 40 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 487 | blk.40.attn_q.weight | Block 40 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ4_NL | | 488 | blk.40.attn_q_norm.weight | Block 40 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 489 | blk.40.attn_v.weight | Block 40 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 490 | blk.40.ffn_down_exps.weight | Block 40 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 491 | blk.40.ffn_gate_exps.weight | Block 40 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | | 492 | blk.40.ffn_gate_inp.weight | Block 40 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 493 | blk.40.ffn_norm.weight | Block 40 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 494 | blk.40.ffn_up_exps.weight | Block 40 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | - Total elements in blk.40: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 41 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 495 | blk.41.attn_k.weight | Block 41 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 496 | blk.41.attn_k_norm.weight | Block 41 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 497 | blk.41.attn_norm.weight | Block 41 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 498 | blk.41.attn_output.weight | Block 41 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 499 | blk.41.attn_q.weight | Block 41 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ4_NL | | 500 | blk.41.attn_q_norm.weight | Block 41 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 501 | blk.41.attn_v.weight | Block 41 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 502 | blk.41.ffn_down_exps.weight | Block 41 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 503 | blk.41.ffn_gate_exps.weight | Block 41 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | | 504 | blk.41.ffn_gate_inp.weight | Block 41 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 505 | blk.41.ffn_norm.weight | Block 41 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 506 | blk.41.ffn_up_exps.weight | Block 41 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | - Total elements in blk.41: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 42 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 507 | blk.42.attn_k.weight | Block 42 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 508 | blk.42.attn_k_norm.weight | Block 42 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 509 | blk.42.attn_norm.weight | Block 42 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 510 | blk.42.attn_output.weight | Block 42 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 511 | blk.42.attn_q.weight | Block 42 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ4_NL | | 512 | blk.42.attn_q_norm.weight | Block 42 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 513 | blk.42.attn_v.weight | Block 42 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 514 | blk.42.ffn_down_exps.weight | Block 42 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 515 | blk.42.ffn_gate_exps.weight | Block 42 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | | 516 | blk.42.ffn_gate_inp.weight | Block 42 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 517 | blk.42.ffn_norm.weight | Block 42 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 518 | blk.42.ffn_up_exps.weight | Block 42 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | - Total elements in blk.42: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 43 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 519 | blk.43.attn_k.weight | Block 43 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 520 | blk.43.attn_k_norm.weight | Block 43 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 521 | blk.43.attn_norm.weight | Block 43 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 522 | blk.43.attn_output.weight | Block 43 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 523 | blk.43.attn_q.weight | Block 43 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ4_NL | | 524 | blk.43.attn_q_norm.weight | Block 43 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 525 | blk.43.attn_v.weight | Block 43 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 526 | blk.43.ffn_down_exps.weight | Block 43 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 527 | blk.43.ffn_gate_exps.weight | Block 43 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | | 528 | blk.43.ffn_gate_inp.weight | Block 43 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 529 | blk.43.ffn_norm.weight | Block 43 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 530 | blk.43.ffn_up_exps.weight | Block 43 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | - Total elements in blk.43: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 44 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 531 | blk.44.attn_k.weight | Block 44 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 532 | blk.44.attn_k_norm.weight | Block 44 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 533 | blk.44.attn_norm.weight | Block 44 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 534 | blk.44.attn_output.weight | Block 44 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 535 | blk.44.attn_q.weight | Block 44 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ4_NL | | 536 | blk.44.attn_q_norm.weight | Block 44 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 537 | blk.44.attn_v.weight | Block 44 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 538 | blk.44.ffn_down_exps.weight | Block 44 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 539 | blk.44.ffn_gate_exps.weight | Block 44 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | | 540 | blk.44.ffn_gate_inp.weight | Block 44 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 541 | blk.44.ffn_norm.weight | Block 44 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 542 | blk.44.ffn_up_exps.weight | Block 44 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | - Total elements in blk.44: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 45 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 543 | blk.45.attn_k.weight | Block 45 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 544 | blk.45.attn_k_norm.weight | Block 45 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 545 | blk.45.attn_norm.weight | Block 45 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 546 | blk.45.attn_output.weight | Block 45 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 547 | blk.45.attn_q.weight | Block 45 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ4_NL | | 548 | blk.45.attn_q_norm.weight | Block 45 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 549 | blk.45.attn_v.weight | Block 45 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 550 | blk.45.ffn_down_exps.weight | Block 45 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 551 | blk.45.ffn_gate_exps.weight | Block 45 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | | 552 | blk.45.ffn_gate_inp.weight | Block 45 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 553 | blk.45.ffn_norm.weight | Block 45 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 554 | blk.45.ffn_up_exps.weight | Block 45 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | - Total elements in blk.45: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 46 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 555 | blk.46.attn_k.weight | Block 46 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 556 | blk.46.attn_k_norm.weight | Block 46 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 557 | blk.46.attn_norm.weight | Block 46 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 558 | blk.46.attn_output.weight | Block 46 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 559 | blk.46.attn_q.weight | Block 46 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ4_NL | | 560 | blk.46.attn_q_norm.weight | Block 46 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 561 | blk.46.attn_v.weight | Block 46 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 562 | blk.46.ffn_down_exps.weight | Block 46 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 563 | blk.46.ffn_gate_exps.weight | Block 46 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | | 564 | blk.46.ffn_gate_inp.weight | Block 46 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 565 | blk.46.ffn_norm.weight | Block 46 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 566 | blk.46.ffn_up_exps.weight | Block 46 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | - Total elements in blk.46: (~623M) 623120640 - Percentage of total elements: 2.04% ### Block 47 Tensor Group : ~623M Elements | T_ID | Tensor Layer Name | Human Friendly Tensor Layer Name | Elements | Shape | Type | |-----:|:----------------------------|:-------------------------------------------------------------------------------------------|:------------------|:----------------------|:-------| | 567 | blk.47.attn_k.weight | Block 47 Attention Key (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 568 | blk.47.attn_k_norm.weight | Block 47 Attn_K_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 569 | blk.47.attn_norm.weight | Block 47 Attention Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 570 | blk.47.attn_output.weight | Block 47 Attention Output (W) | ( ~8M) 8388608 | 4096 x 2048 x 1 x 1 | IQ4_NL | | 571 | blk.47.attn_q.weight | Block 47 Attention Query (W) | ( ~8M) 8388608 | 2048 x 4096 x 1 x 1 | IQ4_NL | | 572 | blk.47.attn_q_norm.weight | Block 47 Attn_Q_Norm (W) | ( 128) 128 | 128 x 1 x 1 x 1 | F32 | | 573 | blk.47.attn_v.weight | Block 47 Attention Value (W) | ( ~1M) 1048576 | 2048 x 512 x 1 x 1 | IQ4_NL | | 574 | blk.47.ffn_down_exps.weight | Block 47 Ffn_Down_Exps (W) | (~201M) 201326592 | 768 x 2048 x 128 x 1 | Q5_K | | 575 | blk.47.ffn_gate_exps.weight | Block 47 Ffn_Gate_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | | 576 | blk.47.ffn_gate_inp.weight | Block 47 Expert-Routing Layer For The Feed-Forward Network In Mixture Of Expert Models (W) | (~262K) 262144 | 2048 x 128 x 1 x 1 | F32 | | 577 | blk.47.ffn_norm.weight | Block 47 Feed-Forward Network Normalization (W) | ( ~2K) 2048 | 2048 x 1 x 1 x 1 | F32 | | 578 | blk.47.ffn_up_exps.weight | Block 47 Ffn_Up_Exps (W) | (~201M) 201326592 | 2048 x 768 x 128 x 1 | IQ4_NL | - Total elements in blk.47: (~623M) 623120640 - Percentage of total elements: 2.04%