Expert-pruned 288->80, 62GB, with model card

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,83 @@
 ---
 license: apache-2.0
+base_model: orcarouter/GLM-5.3-Flash-Uncensored-NVFP4
+tags:
+- moe
+- nvfp4
+- compressed-tensors
+- expert-pruning
+- uncensored
+- glm5
+- code
+- reasoning
+language:
+- en
+- ru
 ---
+
+# GLM-5.3-Flash-Uncensored-NVFP4-Pruning-62GB
+
+Expert-pruned version of [orcarouter/GLM-5.3-Flash-Uncensored-NVFP4](https://huggingface.co/orcarouter/GLM-5.3-Flash-Uncensored-NVFP4),
+cut to fit a single 96GB GPU with generous headroom left for KV-cache and long context.
+
+## What was pruned
+
+- MoE experts per sparse layer: **288 -> 80** (42 sparse layers, top-8 routing kept)
+- Shared experts, dense layers (0-2), attention (MLA), vision tower: **untouched**
+- Router (`mlp.gate.weight`, `e_score_correction_bias`) sliced to the kept experts, indices remapped 0..79
+- NVFP4 `compressed-tensors` quantization preserved (packed weights + scales remapped, no requantization)
+
+## Pruning method
+
+Per-expert importance score per layer:
+
+```
+score = z(router_norm) + z(expert_magnitude) + 0.5 * z(calibration_affinity)
+```
+
+- `router_norm` ? L2 norm of the router row for the expert
+- `expert_magnitude` ? mean FP8 scale x global scale over gate/up/down projections
+- `calibration_affinity` ? dot product of the normalized router row with the centroid
+  of ~200 calibration tokens covering **coding, math, logic and dialogue (EN)**,
+  embedded with the model's own `embed_tokens`
+
+Top-80 experts per layer kept. Pruning target was coding / agentic / logic / math /
+conversational ability; rare-domain experts were cut first.
+
+## Stats
+
+| | Base | Pruned |
+|---|---|---|
+| Weights on disk | ~177 GiB (190 GB) | ~62 GiB (66 GB) |
+| Tensors | 110457 | 31833 |
+| Routed experts / MoE layer | 288 | 80 |
+| MoE layers | 42 | 42 |
+| VRAM on RTX PRO 6000 96GB | does not fit | fits, ~30+ GiB left for context |
+
+## Usage
+
+```python
+from transformers import AutoTokenizer, AutoModelForImageTextToText
+import torch
+
+model_id = "Asilarkness/GLM-5.3-Flash-Uncensored-NVFP4-Pruning-62GB"
+tok = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
+model = AutoModelForImageTextToText.from_pretrained(
+    model_id,
+    trust_remote_code=True,
+    device_map="auto",
+    torch_dtype="auto",
+)
+```
+
+Requirements: `transformers>=5.16`, `accelerate`, `compressed-tensors`.
+For low-VRAM edge cases add `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True`
+and `max_memory` with an offload folder.
+
+## Limitations
+
+- Base model is **uncensored/abliterated** ? outputs are unfiltered.
+- Expert pruning trades rare-domain knowledge for size; coding/reasoning/dialogue
+  were prioritized during expert selection.
+- Only smoke-tested after pruning (short greedy generations); run your own evals
+  (HumanEval, GSM8K, MMLU) before production use.

chat_template.jinja

@@ -0,0 +1,257 @@
+[gMASK]<sop>
+{%- set effective_reasoning_effort = reasoning_effort if reasoning_effort is defined and reasoning_effort in ['low', 'high'] else 'max' -%}
+{%- if effective_reasoning_effort is not none -%}<|system|>Reasoning Effort: {{ effective_reasoning_effort | capitalize }}{%- endif -%}
+{%- set clear_thinking = clear_thinking if clear_thinking is defined else false -%}
+{%- if tools -%}
+{%- macro tool_to_json(tool) -%}
+    {%- set ns_tool = namespace(first=true) -%}
+    {{ '{' -}}
+    {%- for k, v in tool.items() -%}
+        {%- if k != 'defer_loading' and k != 'strict' -%}
+            {%- if not ns_tool.first -%}{{- ', ' -}}{%- endif -%}
+            {%- set ns_tool.first = false -%}
+            "{{ k }}": {{ v | tojson(ensure_ascii=False) }}
+        {%- endif -%}
+    {%- endfor -%}
+    {{- '}' -}}
+{%- endmacro -%}
+{%- macro tool_references_to_response(refs) -%}
+    {{- '<tool_response><tools>\n' -}}
+    {%- for tr in refs -%}
+        {%- for tool in tools -%}
+            {%- if 'function' in tool -%}
+                {%- set tool = tool['function'] -%}
+            {%- endif -%}
+            {%- if tool.name == tr.name -%}
+                {{- tool_to_json(tool) + '\n' -}}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- endfor -%}
+    {{- '</tools></tool_response>' -}}
+{%- endmacro -%}
+<|system|>
+# Tools
+
+You may call one or more functions to assist with the user query.
+
+You are provided with function signatures within <tools></tools> XML tags:
+<tools>
+{% for tool in tools %}
+{%- if 'function' in tool -%}
+    {%- set tool = tool['function'] -%}
+{%- endif -%}
+{% if tool.defer_loading is not defined or not tool.defer_loading %}
+{{ tool_to_json(tool) }}
+{% endif %}
+{% endfor %}
+</tools>
+
+For each function call, output the function name and arguments within the following XML format:
+<tool_call>{function-name}<arg_key>{arg-key-1}</arg_key><arg_value>{arg-value-1}</arg_value><arg_key>{arg-key-2}</arg_key><arg_value>{arg-value-2}</arg_value>...</tool_call>{%- endif -%}
+{%- macro emit_image() -%}<|begin_of_image|><|image|><|end_of_image|>{%- endmacro -%}
+{%- macro emit_video() -%}<|begin_of_video|><|video|><|end_of_video|>{%- endmacro -%}
+{%- macro emit_audio() -%}<|begin_of_audio|><|end_of_audio|>{%- endmacro -%}
+{%- macro visible_text(content) -%}
+    {%- if content is string -%}
+        {{- content -}}
+    {%- elif content is iterable and content is not mapping -%}
+        {%- for item in content -%}
+            {%- if item is mapping and item.type == 'text' -%}
+                {{- item.text -}}
+            {%- elif item is string -%}
+                {{- item -}}
+            {%- elif item is mapping and item.type in ['image', 'image_url'] -%}
+                {{- emit_image() -}}
+            {%- elif item is mapping and item.type in ['video', 'video_url'] -%}
+                {{- emit_video() -}}
+            {%- elif item is mapping and item.type in ['audio', 'audio_url', 'input_audio'] -%}
+                {{- emit_audio() -}}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{- content }}
+    {%- endif -%}
+{%- endmacro -%}
+{%- macro tool_response(text) -%}
+{{- '<tool_response>' + text + '</tool_response>' -}}
+{%- endmacro -%}
+{%- macro render_tool_response(m) -%}
+{%- if m.content is string -%}
+    {{- tool_response(m.content) -}}
+{%- elif m.content and m.content is not mapping and m.content.0.type == "tool_reference" -%}
+    {{- tool_references_to_response(m.content) -}}
+{%- elif is_list_of_outputs(m) -%}
+    {%- for tr in m.content -%}
+        {%- if tr.output is iterable and tr.output is not string and tr.output is not mapping and tr.output and tr.output.0.type == "tool_reference" -%}
+            {{- tool_references_to_response(tr.output) -}}
+        {%- else -%}
+            {{- tool_response(visible_text(tr.output)) -}}
+        {%- endif -%}
+    {%- endfor -%}
+{%- else -%}
+    {{- tool_response(visible_text(m.content)) -}}
+{%- endif -%}
+{%- endmacro -%}
+{%- macro id_of(obj) -%}
+    {%- if obj.tool_call_id -%}
+        {{- obj.tool_call_id -}}
+    {%- elif obj.id -%}
+        {{- obj.id -}}
+    {%- endif -%}
+{%- endmacro -%}
+{%- macro is_list_of_outputs(m) -%}
+    {%- if m.content and m.content.0.output is defined -%}1{%- endif -%}
+{%- endmacro -%}
+{%- macro has_dup_tool_result_id(lo, hi, target) -%}
+    {%- set ns_cnt = namespace(n=0) -%}
+    {%- for k in range(lo, hi + 1) -%}
+        {%- set m = messages[k] -%}
+        {%- if is_list_of_outputs(m) -%}
+            {%- for entry in m.content -%}
+                {%- if id_of(entry) == target -%}
+                    {%- set ns_cnt.n = ns_cnt.n + 1 -%}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- elif id_of(m) == target -%}
+            {%- set ns_cnt.n = ns_cnt.n + 1 -%}
+        {%- endif -%}
+        {%- if ns_cnt.n > 1 -%}{%- break -%}{%- endif -%}
+    {%- endfor -%}
+    {%- if ns_cnt.n > 1 -%}1{%- endif -%}
+{%- endmacro -%}
+{%- macro tc_id_exists(tcs, target) -%}
+    {%- set ns_f = namespace(found=false) -%}
+    {%- for tc in tcs -%}
+        {%- if id_of(tc) == target -%}
+            {%- set ns_f.found = true -%}
+            {%- break -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {%- if ns_f.found -%}1{%- endif -%}
+{%- endmacro -%}
+{%- set ns = namespace(last_user_index=-1) -%}
+{%- for m in messages %}
+    {%- if m.role == 'user' %}
+        {%- set ns.last_user_index = loop.index0 -%}
+    {%- endif %}
+{%- endfor %}
+{%- for m in messages -%}
+{%- if m.role == 'user' -%}<|user|>{{ visible_text(m.content) }}
+{%- elif m.role == 'assistant' -%}
+<|assistant|>
+{%- set content = visible_text(m.content) %}
+{%- if m.reasoning_content is string %}
+    {%- set reasoning_content = m.reasoning_content %}
+{%- elif '</think>' in content %}
+    {%- set reasoning_content = content.split('</think>')[0].split('<think>')[-1] %}
+    {%- set content = content.split('</think>')[-1] %}
+{%- endif %}
+{%- if (not clear_thinking or loop.index0 > ns.last_user_index) and reasoning_content is defined -%}
+{{ '<think>' + reasoning_content +  '</think>'}}
+{%- else -%}
+{{ '<think></think>' }}
+{%- endif -%}
+{%- if content.strip() -%}
+{{ content.strip() }}
+{%- endif -%}
+{% if m.tool_calls %}
+{% for tc in m.tool_calls %}
+{%- if tc.function %}
+    {%- set tc = tc.function %}
+{%- endif %}
+{{- '<tool_call>' + tc.name -}}
+{% set _args = tc.arguments %}{% for k, v in _args.items() %}<arg_key>{{ k }}</arg_key><arg_value>{{ v | tojson(ensure_ascii=False) if v is not string else v }}</arg_value>{% endfor %}</tool_call>{% endfor %}
+{% endif %}
+{%- elif m.role == 'tool' -%}
+{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+    {{- '<|observation|>' -}}
+    {%- set block_start = loop.index0 -%}
+    {%- set ns_blk = namespace(end=block_start) -%}
+    {%- for j in range(block_start, messages|length) -%}
+        {%- if messages[j].role == 'tool' -%}
+            {%- set ns_blk.end = j -%}
+        {%- else -%}
+            {%- break -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {%- set ns_a = namespace(tool_calls=none) -%}
+    {%- if block_start > 0 and messages[block_start - 1].role == 'assistant' and messages[block_start - 1].tool_calls -%}
+        {%- set ns_a.tool_calls = messages[block_start - 1].tool_calls -%}
+    {%- endif -%}
+    {%- set ns_chk = namespace(can_sort=true) -%}
+    {%- if not ns_a.tool_calls -%}
+        {%- set ns_chk.can_sort = false -%}
+    {%- else -%}
+        {%- for k in range(block_start, ns_blk.end + 1) -%}
+            {%- set m = messages[k] -%}
+            {%- if is_list_of_outputs(m) -%}
+                {%- for entry in m.content -%}
+                    {%- set eid = id_of(entry) -%}
+                    {%- if not eid -%}
+                        {%- set ns_chk.can_sort = false -%}
+                    {%- elif has_dup_tool_result_id(block_start, ns_blk.end, eid) -%}
+                        {%- set ns_chk.can_sort = false -%}
+                    {%- elif not tc_id_exists(ns_a.tool_calls, eid) -%}
+                        {%- set ns_chk.can_sort = false -%}
+                    {%- endif -%}
+                {%- endfor -%}
+            {%- else -%}
+                {%- set tk_id = id_of(m) -%}
+                {%- if not tk_id -%}
+                    {%- set ns_chk.can_sort = false -%}
+                {%- elif has_dup_tool_result_id(block_start, ns_blk.end, tk_id) -%}
+                    {%- set ns_chk.can_sort = false -%}
+                {%- elif not tc_id_exists(ns_a.tool_calls, tk_id) -%}
+                    {%- set ns_chk.can_sort = false -%}
+                {%- endif -%}
+            {%- endif -%}
+        {%- endfor -%}
+        {%- for i in range(ns_a.tool_calls | length) -%}
+            {%- set tc_id = id_of(ns_a.tool_calls[i]) -%}
+            {%- if not tc_id -%}
+                {%- set ns_chk.can_sort = false -%}
+            {%- endif -%}
+            {%- for j in range(i + 1, ns_a.tool_calls | length) -%}
+                {%- if id_of(ns_a.tool_calls[j]) == tc_id -%}
+                    {%- set ns_chk.can_sort = false -%}
+                {%- endif -%}
+            {%- endfor -%}
+    {%- endfor -%}
+    {%- endif -%}
+    {%- if ns_chk.can_sort -%}
+        {%- for tc in ns_a.tool_calls -%}
+            {%- set tc_id = id_of(tc) -%}
+            {%- for k in range(block_start, ns_blk.end + 1) -%}
+                {%- set m = messages[k] -%}
+                {%- if is_list_of_outputs(m) -%}
+                    {%- for entry in m.content -%}
+                        {%- set eid = id_of(entry) -%}
+                        {%- if eid == tc_id -%}
+                            {%- if entry.output is iterable and entry.output is not string and entry.output is not mapping and entry.output and entry.output.0.type == "tool_reference" -%}
+                                {{- tool_references_to_response(entry.output) -}}
+                            {%- else -%}
+                                {{- tool_response(visible_text(entry.output)) -}}
+                            {%- endif -%}
+                        {%- endif -%}
+    {%- endfor -%}
+{%- else -%}
+                    {%- set tk_id = id_of(m) -%}
+                    {%- if tk_id == tc_id -%}
+                        {{- render_tool_response(m) -}}
+                    {%- endif -%}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- endfor -%}
+    {%- else -%}
+        {%- for k in range(block_start, ns_blk.end + 1) -%}
+            {{- render_tool_response(messages[k]) -}}
+        {%- endfor -%}
+    {%- endif -%}
+{% endif -%}
+{%- elif m.role == 'system' -%}
+<|system|>{{ visible_text(m.content) }}
+{%- endif -%}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    <|assistant|>{{- '<think>' -}}
+{%- endif -%}

config.json

@@ -0,0 +1,1129 @@
+{
+  "architectures": [
+    "Glm5NextForConditionalGeneration"
+  ],
+  "text_config": {
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "dtype": "bfloat16",
+    "eos_token_id": [
+      154820,
+      154827,
+      154829
+    ],
+    "first_k_dense_replace": 3,
+    "hc_eps": 1e-06,
+    "hc_mult": 4,
+    "hc_sinkhorn_iters": 20,
+    "head_dim": 0,
+    "hidden_act": "silu",
+    "hidden_size": 4096,
+    "index_head_dim": 128,
+    "index_kpool": 4,
+    "index_kpool_always_select_tail": true,
+    "index_kpool_compress": true,
+    "index_n_heads": 32,
+    "index_topk": 2048,
+    "index_share_for_mtp_iteration": true,
+    "indexer_rope_interleave": true,
+    "indexer_types": [
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full"
+    ],
+    "initializer_range": 0.02,
+    "intermediate_size": 12288,
+    "kv_lora_rank": 512,
+    "layer_types": [
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention"
+    ],
+    "linear_attn_config": {
+      "num_heads": 64,
+      "gate_lower_bound": -5.0,
+      "head_dim": 128,
+      "short_conv_kernel_size": 4,
+      "kda_layers": [
+        0,
+        1,
+        2,
+        4,
+        5,
+        6,
+        8,
+        9,
+        10,
+        12,
+        13,
+        14,
+        16,
+        17,
+        18,
+        20,
+        21,
+        22,
+        24,
+        25,
+        26,
+        28,
+        29,
+        30,
+        32,
+        33,
+        34,
+        36,
+        37,
+        38,
+        40,
+        41,
+        42,
+        44
+      ],
+      "full_attn_layers": [
+        3,
+        7,
+        11,
+        15,
+        19,
+        23,
+        27,
+        31,
+        35,
+        39,
+        43
+      ]
+    },
+    "max_position_embeddings": 1048576,
+    "mhc": true,
+    "mla_use_nope": true,
+    "mlp_layer_types": [
+      "dense",
+      "dense",
+      "dense",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse"
+    ],
+    "model_type": "glm5_next_text",
+    "moe_intermediate_size": 2048,
+    "moe_router_dtype": "float32",
+    "n_group": 1,
+    "n_routed_experts": 80,
+    "n_shared_experts": 1,
+    "norm_topk_prob": true,
+    "num_attention_heads": 64,
+    "num_experts_per_tok": 8,
+    "num_hidden_layers": 45,
+    "num_key_value_heads": 64,
+    "num_nextn_predict_layers": 1,
+    "output_router_logits": false,
+    "pad_token_id": 154820,
+    "q_lora_rank": 1536,
+    "qk_head_dim": 256,
+    "qk_nope_head_dim": 256,
+    "qk_rope_head_dim": 0,
+    "rms_norm_eps": 1e-05,
+    "routed_scaling_factor": 2.5,
+    "router_aux_loss_coef": 0.001,
+    "scoring_func": "sigmoid",
+    "swiglu_limit": 10.0,
+    "tie_word_embeddings": false,
+    "topk_group": 1,
+    "topk_method": "noaux_tc",
+    "use_cache": true,
+    "v_head_dim": 256,
+    "vocab_size": 154880
+  },
+  "vision_config": {
+    "attention_bias": true,
+    "attention_dropout": 0.0,
+    "depth": 24,
+    "hidden_act": "silu",
+    "hidden_size": 1024,
+    "image_size": 448,
+    "in_channels": 3,
+    "initializer_range": 0.02,
+    "intermediate_size": 4096,
+    "model_type": "glm5_next_vision",
+    "num_heads": 16,
+    "out_hidden_size": 4096,
+    "patch_size": 14,
+    "projection_intermediate_size": 10240,
+    "rms_norm_eps": 1e-05,
+    "spatial_merge_size": 2,
+    "swiglu_limit": 10.0,
+    "temporal_patch_size": 2
+  },
+  "image_token_id": 154854,
+  "video_token_id": 154855,
+  "image_start_token_id": 154830,
+  "image_end_token_id": 154831,
+  "video_start_token_id": 154832,
+  "video_end_token_id": 154833,
+  "tie_word_embeddings": false,
+  "model_type": "glm5_next",
+  "transformers_version": "5.16.0",
+  "quantization_config": {
+    "quant_method": "compressed-tensors",
+    "format": "nvfp4-pack-quantized",
+    "config_groups": {
+      "group_0": {
+        "targets": [
+          "re:.*\\.layers\\.(?:3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32|33|34|35|36|37|38|39|40|41|42|43|44)\\.mlp\\.experts\\.\\d+\\.(gate_proj|up_proj|down_proj)$"
+        ],
+        "weights": {
+          "num_bits": 4,
+          "type": "float",
+          "symmetric": true,
+          "group_size": 16,
+          "strategy": "tensor_group",
+          "dynamic": false,
+          "observer": "memoryless_minmax",
+          "observer_kwargs": {},
+          "actorder": null,
+          "block_structure": null
+        },
+        "input_activations": {
+          "num_bits": 4,
+          "type": "float",
+          "symmetric": true,
+          "group_size": 16,
+          "strategy": "tensor_group",
+          "dynamic": true,
+          "observer": null,
+          "observer_kwargs": {},
+          "actorder": null,
+          "block_structure": null
+        },
+        "output_activations": null,
+        "format": "nvfp4-pack-quantized"
+      }
+    },
+    "ignore": [
+      "lm_head",
+      "model.embed_tokens",
+      "model.layers.0.mlp.down_proj",
+      "model.layers.0.mlp.gate_proj",
+      "model.layers.0.mlp.up_proj",
+      "model.layers.0.self_attn.b_proj",
+      "model.layers.0.self_attn.f_a_proj",
+      "model.layers.0.self_attn.f_b_proj",
+      "model.layers.0.self_attn.g_a_proj",
+      "model.layers.0.self_attn.g_b_proj",
+      "model.layers.0.self_attn.k_proj",
+      "model.layers.0.self_attn.o_proj",
+      "model.layers.0.self_attn.q_proj",
+      "model.layers.0.self_attn.v_proj",
+      "model.layers.1.mlp.down_proj",
+      "model.layers.1.mlp.gate_proj",
+      "model.layers.1.mlp.up_proj",
+      "model.layers.1.self_attn.b_proj",
+      "model.layers.1.self_attn.f_a_proj",
+      "model.layers.1.self_attn.f_b_proj",
+      "model.layers.1.self_attn.g_a_proj",
+      "model.layers.1.self_attn.g_b_proj",
+      "model.layers.1.self_attn.k_proj",
+      "model.layers.1.self_attn.o_proj",
+      "model.layers.1.self_attn.q_proj",
+      "model.layers.1.self_attn.v_proj",
+      "model.layers.10.mlp.gate",
+      "model.layers.10.mlp.shared_experts.down_proj",
+      "model.layers.10.mlp.shared_experts.gate_proj",
+      "model.layers.10.mlp.shared_experts.up_proj",
+      "model.layers.10.self_attn.b_proj",
+      "model.layers.10.self_attn.f_a_proj",
+      "model.layers.10.self_attn.f_b_proj",
+      "model.layers.10.self_attn.g_a_proj",
+      "model.layers.10.self_attn.g_b_proj",
+      "model.layers.10.self_attn.k_proj",
+      "model.layers.10.self_attn.o_proj",
+      "model.layers.10.self_attn.q_proj",
+      "model.layers.10.self_attn.v_proj",
+      "model.layers.11.mlp.gate",
+      "model.layers.11.mlp.shared_experts.down_proj",
+      "model.layers.11.mlp.shared_experts.gate_proj",
+      "model.layers.11.mlp.shared_experts.up_proj",
+      "model.layers.11.self_attn.indexer.weights_proj",
+      "model.layers.11.self_attn.indexer.wk",
+      "model.layers.11.self_attn.indexer.wq_b",
+      "model.layers.11.self_attn.kv_a_proj_with_mqa",
+      "model.layers.11.self_attn.kv_b_proj",
+      "model.layers.11.self_attn.o_proj",
+      "model.layers.11.self_attn.q_a_proj",
+      "model.layers.11.self_attn.q_b_proj",
+      "model.layers.12.mlp.gate",
+      "model.layers.12.mlp.shared_experts.down_proj",
+      "model.layers.12.mlp.shared_experts.gate_proj",
+      "model.layers.12.mlp.shared_experts.up_proj",
+      "model.layers.12.self_attn.b_proj",
+      "model.layers.12.self_attn.f_a_proj",
+      "model.layers.12.self_attn.f_b_proj",
+      "model.layers.12.self_attn.g_a_proj",
+      "model.layers.12.self_attn.g_b_proj",
+      "model.layers.12.self_attn.k_proj",
+      "model.layers.12.self_attn.o_proj",
+      "model.layers.12.self_attn.q_proj",
+      "model.layers.12.self_attn.v_proj",
+      "model.layers.13.mlp.gate",
+      "model.layers.13.mlp.shared_experts.down_proj",
+      "model.layers.13.mlp.shared_experts.gate_proj",
+      "model.layers.13.mlp.shared_experts.up_proj",
+      "model.layers.13.self_attn.b_proj",
+      "model.layers.13.self_attn.f_a_proj",
+      "model.layers.13.self_attn.f_b_proj",
+      "model.layers.13.self_attn.g_a_proj",
+      "model.layers.13.self_attn.g_b_proj",
+      "model.layers.13.self_attn.k_proj",
+      "model.layers.13.self_attn.o_proj",
+      "model.layers.13.self_attn.q_proj",
+      "model.layers.13.self_attn.v_proj",
+      "model.layers.14.mlp.gate",
+      "model.layers.14.mlp.shared_experts.down_proj",
+      "model.layers.14.mlp.shared_experts.gate_proj",
+      "model.layers.14.mlp.shared_experts.up_proj",
+      "model.layers.14.self_attn.b_proj",
+      "model.layers.14.self_attn.f_a_proj",
+      "model.layers.14.self_attn.f_b_proj",
+      "model.layers.14.self_attn.g_a_proj",
+      "model.layers.14.self_attn.g_b_proj",
+      "model.layers.14.self_attn.k_proj",
+      "model.layers.14.self_attn.o_proj",
+      "model.layers.14.self_attn.q_proj",
+      "model.layers.14.self_attn.v_proj",
+      "model.layers.15.mlp.gate",
+      "model.layers.15.mlp.shared_experts.down_proj",
+      "model.layers.15.mlp.shared_experts.gate_proj",
+      "model.layers.15.mlp.shared_experts.up_proj",
+      "model.layers.15.self_attn.indexer.weights_proj",
+      "model.layers.15.self_attn.indexer.wk",
+      "model.layers.15.self_attn.indexer.wq_b",
+      "model.layers.15.self_attn.kv_a_proj_with_mqa",
+      "model.layers.15.self_attn.kv_b_proj",
+      "model.layers.15.self_attn.o_proj",
+      "model.layers.15.self_attn.q_a_proj",
+      "model.layers.15.self_attn.q_b_proj",
+      "model.layers.16.mlp.gate",
+      "model.layers.16.mlp.shared_experts.down_proj",
+      "model.layers.16.mlp.shared_experts.gate_proj",
+      "model.layers.16.mlp.shared_experts.up_proj",
+      "model.layers.16.self_attn.b_proj",
+      "model.layers.16.self_attn.f_a_proj",
+      "model.layers.16.self_attn.f_b_proj",
+      "model.layers.16.self_attn.g_a_proj",
+      "model.layers.16.self_attn.g_b_proj",
+      "model.layers.16.self_attn.k_proj",
+      "model.layers.16.self_attn.o_proj",
+      "model.layers.16.self_attn.q_proj",
+      "model.layers.16.self_attn.v_proj",
+      "model.layers.17.mlp.gate",
+      "model.layers.17.mlp.shared_experts.down_proj",
+      "model.layers.17.mlp.shared_experts.gate_proj",
+      "model.layers.17.mlp.shared_experts.up_proj",
+      "model.layers.17.self_attn.b_proj",
+      "model.layers.17.self_attn.f_a_proj",
+      "model.layers.17.self_attn.f_b_proj",
+      "model.layers.17.self_attn.g_a_proj",
+      "model.layers.17.self_attn.g_b_proj",
+      "model.layers.17.self_attn.k_proj",
+      "model.layers.17.self_attn.o_proj",
+      "model.layers.17.self_attn.q_proj",
+      "model.layers.17.self_attn.v_proj",
+      "model.layers.18.mlp.gate",
+      "model.layers.18.mlp.shared_experts.down_proj",
+      "model.layers.18.mlp.shared_experts.gate_proj",
+      "model.layers.18.mlp.shared_experts.up_proj",
+      "model.layers.18.self_attn.b_proj",
+      "model.layers.18.self_attn.f_a_proj",
+      "model.layers.18.self_attn.f_b_proj",
+      "model.layers.18.self_attn.g_a_proj",
+      "model.layers.18.self_attn.g_b_proj",
+      "model.layers.18.self_attn.k_proj",
+      "model.layers.18.self_attn.o_proj",
+      "model.layers.18.self_attn.q_proj",
+      "model.layers.18.self_attn.v_proj",
+      "model.layers.19.mlp.gate",
+      "model.layers.19.mlp.shared_experts.down_proj",
+      "model.layers.19.mlp.shared_experts.gate_proj",
+      "model.layers.19.mlp.shared_experts.up_proj",
+      "model.layers.19.self_attn.indexer.weights_proj",
+      "model.layers.19.self_attn.indexer.wk",
+      "model.layers.19.self_attn.indexer.wq_b",
+      "model.layers.19.self_attn.kv_a_proj_with_mqa",
+      "model.layers.19.self_attn.kv_b_proj",
+      "model.layers.19.self_attn.o_proj",
+      "model.layers.19.self_attn.q_a_proj",
+      "model.layers.19.self_attn.q_b_proj",
+      "model.layers.2.mlp.down_proj",
+      "model.layers.2.mlp.gate_proj",
+      "model.layers.2.mlp.up_proj",
+      "model.layers.2.self_attn.b_proj",
+      "model.layers.2.self_attn.f_a_proj",
+      "model.layers.2.self_attn.f_b_proj",
+      "model.layers.2.self_attn.g_a_proj",
+      "model.layers.2.self_attn.g_b_proj",
+      "model.layers.2.self_attn.k_proj",
+      "model.layers.2.self_attn.o_proj",
+      "model.layers.2.self_attn.q_proj",
+      "model.layers.2.self_attn.v_proj",
+      "model.layers.20.mlp.gate",
+      "model.layers.20.mlp.shared_experts.down_proj",
+      "model.layers.20.mlp.shared_experts.gate_proj",
+      "model.layers.20.mlp.shared_experts.up_proj",
+      "model.layers.20.self_attn.b_proj",
+      "model.layers.20.self_attn.f_a_proj",
+      "model.layers.20.self_attn.f_b_proj",
+      "model.layers.20.self_attn.g_a_proj",
+      "model.layers.20.self_attn.g_b_proj",
+      "model.layers.20.self_attn.k_proj",
+      "model.layers.20.self_attn.o_proj",
+      "model.layers.20.self_attn.q_proj",
+      "model.layers.20.self_attn.v_proj",
+      "model.layers.21.mlp.gate",
+      "model.layers.21.mlp.shared_experts.down_proj",
+      "model.layers.21.mlp.shared_experts.gate_proj",
+      "model.layers.21.mlp.shared_experts.up_proj",
+      "model.layers.21.self_attn.b_proj",
+      "model.layers.21.self_attn.f_a_proj",
+      "model.layers.21.self_attn.f_b_proj",
+      "model.layers.21.self_attn.g_a_proj",
+      "model.layers.21.self_attn.g_b_proj",
+      "model.layers.21.self_attn.k_proj",
+      "model.layers.21.self_attn.o_proj",
+      "model.layers.21.self_attn.q_proj",
+      "model.layers.21.self_attn.v_proj",
+      "model.layers.22.mlp.gate",
+      "model.layers.22.mlp.shared_experts.down_proj",
+      "model.layers.22.mlp.shared_experts.gate_proj",
+      "model.layers.22.mlp.shared_experts.up_proj",
+      "model.layers.22.self_attn.b_proj",
+      "model.layers.22.self_attn.f_a_proj",
+      "model.layers.22.self_attn.f_b_proj",
+      "model.layers.22.self_attn.g_a_proj",
+      "model.layers.22.self_attn.g_b_proj",
+      "model.layers.22.self_attn.k_proj",
+      "model.layers.22.self_attn.o_proj",
+      "model.layers.22.self_attn.q_proj",
+      "model.layers.22.self_attn.v_proj",
+      "model.layers.23.mlp.gate",
+      "model.layers.23.mlp.shared_experts.down_proj",
+      "model.layers.23.mlp.shared_experts.gate_proj",
+      "model.layers.23.mlp.shared_experts.up_proj",
+      "model.layers.23.self_attn.indexer.weights_proj",
+      "model.layers.23.self_attn.indexer.wk",
+      "model.layers.23.self_attn.indexer.wq_b",
+      "model.layers.23.self_attn.kv_a_proj_with_mqa",
+      "model.layers.23.self_attn.kv_b_proj",
+      "model.layers.23.self_attn.o_proj",
+      "model.layers.23.self_attn.q_a_proj",
+      "model.layers.23.self_attn.q_b_proj",
+      "model.layers.24.mlp.gate",
+      "model.layers.24.mlp.shared_experts.down_proj",
+      "model.layers.24.mlp.shared_experts.gate_proj",
+      "model.layers.24.mlp.shared_experts.up_proj",
+      "model.layers.24.self_attn.b_proj",
+      "model.layers.24.self_attn.f_a_proj",
+      "model.layers.24.self_attn.f_b_proj",
+      "model.layers.24.self_attn.g_a_proj",
+      "model.layers.24.self_attn.g_b_proj",
+      "model.layers.24.self_attn.k_proj",
+      "model.layers.24.self_attn.o_proj",
+      "model.layers.24.self_attn.q_proj",
+      "model.layers.24.self_attn.v_proj",
+      "model.layers.25.mlp.gate",
+      "model.layers.25.mlp.shared_experts.down_proj",
+      "model.layers.25.mlp.shared_experts.gate_proj",
+      "model.layers.25.mlp.shared_experts.up_proj",
+      "model.layers.25.self_attn.b_proj",
+      "model.layers.25.self_attn.f_a_proj",
+      "model.layers.25.self_attn.f_b_proj",
+      "model.layers.25.self_attn.g_a_proj",
+      "model.layers.25.self_attn.g_b_proj",
+      "model.layers.25.self_attn.k_proj",
+      "model.layers.25.self_attn.o_proj",
+      "model.layers.25.self_attn.q_proj",
+      "model.layers.25.self_attn.v_proj",
+      "model.layers.26.mlp.gate",
+      "model.layers.26.mlp.shared_experts.down_proj",
+      "model.layers.26.mlp.shared_experts.gate_proj",
+      "model.layers.26.mlp.shared_experts.up_proj",
+      "model.layers.26.self_attn.b_proj",
+      "model.layers.26.self_attn.f_a_proj",
+      "model.layers.26.self_attn.f_b_proj",
+      "model.layers.26.self_attn.g_a_proj",
+      "model.layers.26.self_attn.g_b_proj",
+      "model.layers.26.self_attn.k_proj",
+      "model.layers.26.self_attn.o_proj",
+      "model.layers.26.self_attn.q_proj",
+      "model.layers.26.self_attn.v_proj",
+      "model.layers.27.mlp.gate",
+      "model.layers.27.mlp.shared_experts.down_proj",
+      "model.layers.27.mlp.shared_experts.gate_proj",
+      "model.layers.27.mlp.shared_experts.up_proj",
+      "model.layers.27.self_attn.indexer.weights_proj",
+      "model.layers.27.self_attn.indexer.wk",
+      "model.layers.27.self_attn.indexer.wq_b",
+      "model.layers.27.self_attn.kv_a_proj_with_mqa",
+      "model.layers.27.self_attn.kv_b_proj",
+      "model.layers.27.self_attn.o_proj",
+      "model.layers.27.self_attn.q_a_proj",
+      "model.layers.27.self_attn.q_b_proj",
+      "model.layers.28.mlp.gate",
+      "model.layers.28.mlp.shared_experts.down_proj",
+      "model.layers.28.mlp.shared_experts.gate_proj",
+      "model.layers.28.mlp.shared_experts.up_proj",
+      "model.layers.28.self_attn.b_proj",
+      "model.layers.28.self_attn.f_a_proj",
+      "model.layers.28.self_attn.f_b_proj",
+      "model.layers.28.self_attn.g_a_proj",
+      "model.layers.28.self_attn.g_b_proj",
+      "model.layers.28.self_attn.k_proj",
+      "model.layers.28.self_attn.o_proj",
+      "model.layers.28.self_attn.q_proj",
+      "model.layers.28.self_attn.v_proj",
+      "model.layers.29.mlp.gate",
+      "model.layers.29.mlp.shared_experts.down_proj",
+      "model.layers.29.mlp.shared_experts.gate_proj",
+      "model.layers.29.mlp.shared_experts.up_proj",
+      "model.layers.29.self_attn.b_proj",
+      "model.layers.29.self_attn.f_a_proj",
+      "model.layers.29.self_attn.f_b_proj",
+      "model.layers.29.self_attn.g_a_proj",
+      "model.layers.29.self_attn.g_b_proj",
+      "model.layers.29.self_attn.k_proj",
+      "model.layers.29.self_attn.o_proj",
+      "model.layers.29.self_attn.q_proj",
+      "model.layers.29.self_attn.v_proj",
+      "model.layers.3.mlp.gate",
+      "model.layers.3.mlp.shared_experts.down_proj",
+      "model.layers.3.mlp.shared_experts.gate_proj",
+      "model.layers.3.mlp.shared_experts.up_proj",
+      "model.layers.3.self_attn.indexer.weights_proj",
+      "model.layers.3.self_attn.indexer.wk",
+      "model.layers.3.self_attn.indexer.wq_b",
+      "model.layers.3.self_attn.kv_a_proj_with_mqa",
+      "model.layers.3.self_attn.kv_b_proj",
+      "model.layers.3.self_attn.o_proj",
+      "model.layers.3.self_attn.q_a_proj",
+      "model.layers.3.self_attn.q_b_proj",
+      "model.layers.30.mlp.gate",
+      "model.layers.30.mlp.shared_experts.down_proj",
+      "model.layers.30.mlp.shared_experts.gate_proj",
+      "model.layers.30.mlp.shared_experts.up_proj",
+      "model.layers.30.self_attn.b_proj",
+      "model.layers.30.self_attn.f_a_proj",
+      "model.layers.30.self_attn.f_b_proj",
+      "model.layers.30.self_attn.g_a_proj",
+      "model.layers.30.self_attn.g_b_proj",
+      "model.layers.30.self_attn.k_proj",
+      "model.layers.30.self_attn.o_proj",
+      "model.layers.30.self_attn.q_proj",
+      "model.layers.30.self_attn.v_proj",
+      "model.layers.31.mlp.gate",
+      "model.layers.31.mlp.shared_experts.down_proj",
+      "model.layers.31.mlp.shared_experts.gate_proj",
+      "model.layers.31.mlp.shared_experts.up_proj",
+      "model.layers.31.self_attn.indexer.weights_proj",
+      "model.layers.31.self_attn.indexer.wk",
+      "model.layers.31.self_attn.indexer.wq_b",
+      "model.layers.31.self_attn.kv_a_proj_with_mqa",
+      "model.layers.31.self_attn.kv_b_proj",
+      "model.layers.31.self_attn.o_proj",
+      "model.layers.31.self_attn.q_a_proj",
+      "model.layers.31.self_attn.q_b_proj",
+      "model.layers.32.mlp.gate",
+      "model.layers.32.mlp.shared_experts.down_proj",
+      "model.layers.32.mlp.shared_experts.gate_proj",
+      "model.layers.32.mlp.shared_experts.up_proj",
+      "model.layers.32.self_attn.b_proj",
+      "model.layers.32.self_attn.f_a_proj",
+      "model.layers.32.self_attn.f_b_proj",
+      "model.layers.32.self_attn.g_a_proj",
+      "model.layers.32.self_attn.g_b_proj",
+      "model.layers.32.self_attn.k_proj",
+      "model.layers.32.self_attn.o_proj",
+      "model.layers.32.self_attn.q_proj",
+      "model.layers.32.self_attn.v_proj",
+      "model.layers.33.mlp.gate",
+      "model.layers.33.mlp.shared_experts.down_proj",
+      "model.layers.33.mlp.shared_experts.gate_proj",
+      "model.layers.33.mlp.shared_experts.up_proj",
+      "model.layers.33.self_attn.b_proj",
+      "model.layers.33.self_attn.f_a_proj",
+      "model.layers.33.self_attn.f_b_proj",
+      "model.layers.33.self_attn.g_a_proj",
+      "model.layers.33.self_attn.g_b_proj",
+      "model.layers.33.self_attn.k_proj",
+      "model.layers.33.self_attn.o_proj",
+      "model.layers.33.self_attn.q_proj",
+      "model.layers.33.self_attn.v_proj",
+      "model.layers.34.mlp.gate",
+      "model.layers.34.mlp.shared_experts.down_proj",
+      "model.layers.34.mlp.shared_experts.gate_proj",
+      "model.layers.34.mlp.shared_experts.up_proj",
+      "model.layers.34.self_attn.b_proj",
+      "model.layers.34.self_attn.f_a_proj",
+      "model.layers.34.self_attn.f_b_proj",
+      "model.layers.34.self_attn.g_a_proj",
+      "model.layers.34.self_attn.g_b_proj",
+      "model.layers.34.self_attn.k_proj",
+      "model.layers.34.self_attn.o_proj",
+      "model.layers.34.self_attn.q_proj",
+      "model.layers.34.self_attn.v_proj",
+      "model.layers.35.mlp.gate",
+      "model.layers.35.mlp.shared_experts.down_proj",
+      "model.layers.35.mlp.shared_experts.gate_proj",
+      "model.layers.35.mlp.shared_experts.up_proj",
+      "model.layers.35.self_attn.indexer.weights_proj",
+      "model.layers.35.self_attn.indexer.wk",
+      "model.layers.35.self_attn.indexer.wq_b",
+      "model.layers.35.self_attn.kv_a_proj_with_mqa",
+      "model.layers.35.self_attn.kv_b_proj",
+      "model.layers.35.self_attn.o_proj",
+      "model.layers.35.self_attn.q_a_proj",
+      "model.layers.35.self_attn.q_b_proj",
+      "model.layers.36.mlp.gate",
+      "model.layers.36.mlp.shared_experts.down_proj",
+      "model.layers.36.mlp.shared_experts.gate_proj",
+      "model.layers.36.mlp.shared_experts.up_proj",
+      "model.layers.36.self_attn.b_proj",
+      "model.layers.36.self_attn.f_a_proj",
+      "model.layers.36.self_attn.f_b_proj",
+      "model.layers.36.self_attn.g_a_proj",
+      "model.layers.36.self_attn.g_b_proj",
+      "model.layers.36.self_attn.k_proj",
+      "model.layers.36.self_attn.o_proj",
+      "model.layers.36.self_attn.q_proj",
+      "model.layers.36.self_attn.v_proj",
+      "model.layers.37.mlp.gate",
+      "model.layers.37.mlp.shared_experts.down_proj",
+      "model.layers.37.mlp.shared_experts.gate_proj",
+      "model.layers.37.mlp.shared_experts.up_proj",
+      "model.layers.37.self_attn.b_proj",
+      "model.layers.37.self_attn.f_a_proj",
+      "model.layers.37.self_attn.f_b_proj",
+      "model.layers.37.self_attn.g_a_proj",
+      "model.layers.37.self_attn.g_b_proj",
+      "model.layers.37.self_attn.k_proj",
+      "model.layers.37.self_attn.o_proj",
+      "model.layers.37.self_attn.q_proj",
+      "model.layers.37.self_attn.v_proj",
+      "model.layers.38.mlp.gate",
+      "model.layers.38.mlp.shared_experts.down_proj",
+      "model.layers.38.mlp.shared_experts.gate_proj",
+      "model.layers.38.mlp.shared_experts.up_proj",
+      "model.layers.38.self_attn.b_proj",
+      "model.layers.38.self_attn.f_a_proj",
+      "model.layers.38.self_attn.f_b_proj",
+      "model.layers.38.self_attn.g_a_proj",
+      "model.layers.38.self_attn.g_b_proj",
+      "model.layers.38.self_attn.k_proj",
+      "model.layers.38.self_attn.o_proj",
+      "model.layers.38.self_attn.q_proj",
+      "model.layers.38.self_attn.v_proj",
+      "model.layers.39.mlp.gate",
+      "model.layers.39.mlp.shared_experts.down_proj",
+      "model.layers.39.mlp.shared_experts.gate_proj",
+      "model.layers.39.mlp.shared_experts.up_proj",
+      "model.layers.39.self_attn.indexer.weights_proj",
+      "model.layers.39.self_attn.indexer.wk",
+      "model.layers.39.self_attn.indexer.wq_b",
+      "model.layers.39.self_attn.kv_a_proj_with_mqa",
+      "model.layers.39.self_attn.kv_b_proj",
+      "model.layers.39.self_attn.o_proj",
+      "model.layers.39.self_attn.q_a_proj",
+      "model.layers.39.self_attn.q_b_proj",
+      "model.layers.4.mlp.gate",
+      "model.layers.4.mlp.shared_experts.down_proj",
+      "model.layers.4.mlp.shared_experts.gate_proj",
+      "model.layers.4.mlp.shared_experts.up_proj",
+      "model.layers.4.self_attn.b_proj",
+      "model.layers.4.self_attn.f_a_proj",
+      "model.layers.4.self_attn.f_b_proj",
+      "model.layers.4.self_attn.g_a_proj",
+      "model.layers.4.self_attn.g_b_proj",
+      "model.layers.4.self_attn.k_proj",
+      "model.layers.4.self_attn.o_proj",
+      "model.layers.4.self_attn.q_proj",
+      "model.layers.4.self_attn.v_proj",
+      "model.layers.40.mlp.gate",
+      "model.layers.40.mlp.shared_experts.down_proj",
+      "model.layers.40.mlp.shared_experts.gate_proj",
+      "model.layers.40.mlp.shared_experts.up_proj",
+      "model.layers.40.self_attn.b_proj",
+      "model.layers.40.self_attn.f_a_proj",
+      "model.layers.40.self_attn.f_b_proj",
+      "model.layers.40.self_attn.g_a_proj",
+      "model.layers.40.self_attn.g_b_proj",
+      "model.layers.40.self_attn.k_proj",
+      "model.layers.40.self_attn.o_proj",
+      "model.layers.40.self_attn.q_proj",
+      "model.layers.40.self_attn.v_proj",
+      "model.layers.41.mlp.gate",
+      "model.layers.41.mlp.shared_experts.down_proj",
+      "model.layers.41.mlp.shared_experts.gate_proj",
+      "model.layers.41.mlp.shared_experts.up_proj",
+      "model.layers.41.self_attn.b_proj",
+      "model.layers.41.self_attn.f_a_proj",
+      "model.layers.41.self_attn.f_b_proj",
+      "model.layers.41.self_attn.g_a_proj",
+      "model.layers.41.self_attn.g_b_proj",
+      "model.layers.41.self_attn.k_proj",
+      "model.layers.41.self_attn.o_proj",
+      "model.layers.41.self_attn.q_proj",
+      "model.layers.41.self_attn.v_proj",
+      "model.layers.42.mlp.gate",
+      "model.layers.42.mlp.shared_experts.down_proj",
+      "model.layers.42.mlp.shared_experts.gate_proj",
+      "model.layers.42.mlp.shared_experts.up_proj",
+      "model.layers.42.self_attn.b_proj",
+      "model.layers.42.self_attn.f_a_proj",
+      "model.layers.42.self_attn.f_b_proj",
+      "model.layers.42.self_attn.g_a_proj",
+      "model.layers.42.self_attn.g_b_proj",
+      "model.layers.42.self_attn.k_proj",
+      "model.layers.42.self_attn.o_proj",
+      "model.layers.42.self_attn.q_proj",
+      "model.layers.42.self_attn.v_proj",
+      "model.layers.43.mlp.gate",
+      "model.layers.43.mlp.shared_experts.down_proj",
+      "model.layers.43.mlp.shared_experts.gate_proj",
+      "model.layers.43.mlp.shared_experts.up_proj",
+      "model.layers.43.self_attn.indexer.weights_proj",
+      "model.layers.43.self_attn.indexer.wk",
+      "model.layers.43.self_attn.indexer.wq_b",
+      "model.layers.43.self_attn.kv_a_proj_with_mqa",
+      "model.layers.43.self_attn.kv_b_proj",
+      "model.layers.43.self_attn.o_proj",
+      "model.layers.43.self_attn.q_a_proj",
+      "model.layers.43.self_attn.q_b_proj",
+      "model.layers.44.mlp.gate",
+      "model.layers.44.mlp.shared_experts.down_proj",
+      "model.layers.44.mlp.shared_experts.gate_proj",
+      "model.layers.44.mlp.shared_experts.up_proj",
+      "model.layers.44.self_attn.b_proj",
+      "model.layers.44.self_attn.f_a_proj",
+      "model.layers.44.self_attn.f_b_proj",
+      "model.layers.44.self_attn.g_a_proj",
+      "model.layers.44.self_attn.g_b_proj",
+      "model.layers.44.self_attn.k_proj",
+      "model.layers.44.self_attn.o_proj",
+      "model.layers.44.self_attn.q_proj",
+      "model.layers.44.self_attn.v_proj",
+      "model.layers.5.mlp.gate",
+      "model.layers.5.mlp.shared_experts.down_proj",
+      "model.layers.5.mlp.shared_experts.gate_proj",
+      "model.layers.5.mlp.shared_experts.up_proj",
+      "model.layers.5.self_attn.b_proj",
+      "model.layers.5.self_attn.f_a_proj",
+      "model.layers.5.self_attn.f_b_proj",
+      "model.layers.5.self_attn.g_a_proj",
+      "model.layers.5.self_attn.g_b_proj",
+      "model.layers.5.self_attn.k_proj",
+      "model.layers.5.self_attn.o_proj",
+      "model.layers.5.self_attn.q_proj",
+      "model.layers.5.self_attn.v_proj",
+      "model.layers.6.mlp.gate",
+      "model.layers.6.mlp.shared_experts.down_proj",
+      "model.layers.6.mlp.shared_experts.gate_proj",
+      "model.layers.6.mlp.shared_experts.up_proj",
+      "model.layers.6.self_attn.b_proj",
+      "model.layers.6.self_attn.f_a_proj",
+      "model.layers.6.self_attn.f_b_proj",
+      "model.layers.6.self_attn.g_a_proj",
+      "model.layers.6.self_attn.g_b_proj",
+      "model.layers.6.self_attn.k_proj",
+      "model.layers.6.self_attn.o_proj",
+      "model.layers.6.self_attn.q_proj",
+      "model.layers.6.self_attn.v_proj",
+      "model.layers.7.mlp.gate",
+      "model.layers.7.mlp.shared_experts.down_proj",
+      "model.layers.7.mlp.shared_experts.gate_proj",
+      "model.layers.7.mlp.shared_experts.up_proj",
+      "model.layers.7.self_attn.indexer.weights_proj",
+      "model.layers.7.self_attn.indexer.wk",
+      "model.layers.7.self_attn.indexer.wq_b",
+      "model.layers.7.self_attn.kv_a_proj_with_mqa",
+      "model.layers.7.self_attn.kv_b_proj",
+      "model.layers.7.self_attn.o_proj",
+      "model.layers.7.self_attn.q_a_proj",
+      "model.layers.7.self_attn.q_b_proj",
+      "model.layers.8.mlp.gate",
+      "model.layers.8.mlp.shared_experts.down_proj",
+      "model.layers.8.mlp.shared_experts.gate_proj",
+      "model.layers.8.mlp.shared_experts.up_proj",
+      "model.layers.8.self_attn.b_proj",
+      "model.layers.8.self_attn.f_a_proj",
+      "model.layers.8.self_attn.f_b_proj",
+      "model.layers.8.self_attn.g_a_proj",
+      "model.layers.8.self_attn.g_b_proj",
+      "model.layers.8.self_attn.k_proj",
+      "model.layers.8.self_attn.o_proj",
+      "model.layers.8.self_attn.q_proj",
+      "model.layers.8.self_attn.v_proj",
+      "model.layers.9.mlp.gate",
+      "model.layers.9.mlp.shared_experts.down_proj",
+      "model.layers.9.mlp.shared_experts.gate_proj",
+      "model.layers.9.mlp.shared_experts.up_proj",
+      "model.layers.9.self_attn.b_proj",
+      "model.layers.9.self_attn.f_a_proj",
+      "model.layers.9.self_attn.f_b_proj",
+      "model.layers.9.self_attn.g_a_proj",
+      "model.layers.9.self_attn.g_b_proj",
+      "model.layers.9.self_attn.k_proj",
+      "model.layers.9.self_attn.o_proj",
+      "model.layers.9.self_attn.q_proj",
+      "model.layers.9.self_attn.v_proj",
+      "model.visual.blocks.0.attn.k_norm",
+      "model.visual.blocks.0.attn.proj",
+      "model.visual.blocks.0.attn.q_norm",
+      "model.visual.blocks.0.attn.qkv",
+      "model.visual.blocks.0.mlp.down_proj",
+      "model.visual.blocks.0.mlp.gate_proj",
+      "model.visual.blocks.0.mlp.up_proj",
+      "model.visual.blocks.0.norm1",
+      "model.visual.blocks.0.norm2",
+      "model.visual.blocks.1.attn.k_norm",
+      "model.visual.blocks.1.attn.proj",
+      "model.visual.blocks.1.attn.q_norm",
+      "model.visual.blocks.1.attn.qkv",
+      "model.visual.blocks.1.mlp.down_proj",
+      "model.visual.blocks.1.mlp.gate_proj",
+      "model.visual.blocks.1.mlp.up_proj",
+      "model.visual.blocks.1.norm1",
+      "model.visual.blocks.1.norm2",
+      "model.visual.blocks.10.attn.k_norm",
+      "model.visual.blocks.10.attn.proj",
+      "model.visual.blocks.10.attn.q_norm",
+      "model.visual.blocks.10.attn.qkv",
+      "model.visual.blocks.10.mlp.down_proj",
+      "model.visual.blocks.10.mlp.gate_proj",
+      "model.visual.blocks.10.mlp.up_proj",
+      "model.visual.blocks.10.norm1",
+      "model.visual.blocks.10.norm2",
+      "model.visual.blocks.11.attn.k_norm",
+      "model.visual.blocks.11.attn.proj",
+      "model.visual.blocks.11.attn.q_norm",
+      "model.visual.blocks.11.attn.qkv",
+      "model.visual.blocks.11.mlp.down_proj",
+      "model.visual.blocks.11.mlp.gate_proj",
+      "model.visual.blocks.11.mlp.up_proj",
+      "model.visual.blocks.11.norm1",
+      "model.visual.blocks.11.norm2",
+      "model.visual.blocks.12.attn.k_norm",
+      "model.visual.blocks.12.attn.proj",
+      "model.visual.blocks.12.attn.q_norm",
+      "model.visual.blocks.12.attn.qkv",
+      "model.visual.blocks.12.mlp.down_proj",
+      "model.visual.blocks.12.mlp.gate_proj",
+      "model.visual.blocks.12.mlp.up_proj",
+      "model.visual.blocks.12.norm1",
+      "model.visual.blocks.12.norm2",
+      "model.visual.blocks.13.attn.k_norm",
+      "model.visual.blocks.13.attn.proj",
+      "model.visual.blocks.13.attn.q_norm",
+      "model.visual.blocks.13.attn.qkv",
+      "model.visual.blocks.13.mlp.down_proj",
+      "model.visual.blocks.13.mlp.gate_proj",
+      "model.visual.blocks.13.mlp.up_proj",
+      "model.visual.blocks.13.norm1",
+      "model.visual.blocks.13.norm2",
+      "model.visual.blocks.14.attn.k_norm",
+      "model.visual.blocks.14.attn.proj",
+      "model.visual.blocks.14.attn.q_norm",
+      "model.visual.blocks.14.attn.qkv",
+      "model.visual.blocks.14.mlp.down_proj",
+      "model.visual.blocks.14.mlp.gate_proj",
+      "model.visual.blocks.14.mlp.up_proj",
+      "model.visual.blocks.14.norm1",
+      "model.visual.blocks.14.norm2",
+      "model.visual.blocks.15.attn.k_norm",
+      "model.visual.blocks.15.attn.proj",
+      "model.visual.blocks.15.attn.q_norm",
+      "model.visual.blocks.15.attn.qkv",
+      "model.visual.blocks.15.mlp.down_proj",
+      "model.visual.blocks.15.mlp.gate_proj",
+      "model.visual.blocks.15.mlp.up_proj",
+      "model.visual.blocks.15.norm1",
+      "model.visual.blocks.15.norm2",
+      "model.visual.blocks.16.attn.k_norm",
+      "model.visual.blocks.16.attn.proj",
+      "model.visual.blocks.16.attn.q_norm",
+      "model.visual.blocks.16.attn.qkv",
+      "model.visual.blocks.16.mlp.down_proj",
+      "model.visual.blocks.16.mlp.gate_proj",
+      "model.visual.blocks.16.mlp.up_proj",
+      "model.visual.blocks.16.norm1",
+      "model.visual.blocks.16.norm2",
+      "model.visual.blocks.17.attn.k_norm",
+      "model.visual.blocks.17.attn.proj",
+      "model.visual.blocks.17.attn.q_norm",
+      "model.visual.blocks.17.attn.qkv",
+      "model.visual.blocks.17.mlp.down_proj",
+      "model.visual.blocks.17.mlp.gate_proj",
+      "model.visual.blocks.17.mlp.up_proj",
+      "model.visual.blocks.17.norm1",
+      "model.visual.blocks.17.norm2",
+      "model.visual.blocks.18.attn.k_norm",
+      "model.visual.blocks.18.attn.proj",
+      "model.visual.blocks.18.attn.q_norm",
+      "model.visual.blocks.18.attn.qkv",
+      "model.visual.blocks.18.mlp.down_proj",
+      "model.visual.blocks.18.mlp.gate_proj",
+      "model.visual.blocks.18.mlp.up_proj",
+      "model.visual.blocks.18.norm1",
+      "model.visual.blocks.18.norm2",
+      "model.visual.blocks.19.attn.k_norm",
+      "model.visual.blocks.19.attn.proj",
+      "model.visual.blocks.19.attn.q_norm",
+      "model.visual.blocks.19.attn.qkv",
+      "model.visual.blocks.19.mlp.down_proj",
+      "model.visual.blocks.19.mlp.gate_proj",
+      "model.visual.blocks.19.mlp.up_proj",
+      "model.visual.blocks.19.norm1",
+      "model.visual.blocks.19.norm2",
+      "model.visual.blocks.2.attn.k_norm",
+      "model.visual.blocks.2.attn.proj",
+      "model.visual.blocks.2.attn.q_norm",
+      "model.visual.blocks.2.attn.qkv",
+      "model.visual.blocks.2.mlp.down_proj",
+      "model.visual.blocks.2.mlp.gate_proj",
+      "model.visual.blocks.2.mlp.up_proj",
+      "model.visual.blocks.2.norm1",
+      "model.visual.blocks.2.norm2",
+      "model.visual.blocks.20.attn.k_norm",
+      "model.visual.blocks.20.attn.proj",
+      "model.visual.blocks.20.attn.q_norm",
+      "model.visual.blocks.20.attn.qkv",
+      "model.visual.blocks.20.mlp.down_proj",
+      "model.visual.blocks.20.mlp.gate_proj",
+      "model.visual.blocks.20.mlp.up_proj",
+      "model.visual.blocks.20.norm1",
+      "model.visual.blocks.20.norm2",
+      "model.visual.blocks.21.attn.k_norm",
+      "model.visual.blocks.21.attn.proj",
+      "model.visual.blocks.21.attn.q_norm",
+      "model.visual.blocks.21.attn.qkv",
+      "model.visual.blocks.21.mlp.down_proj",
+      "model.visual.blocks.21.mlp.gate_proj",
+      "model.visual.blocks.21.mlp.up_proj",
+      "model.visual.blocks.21.norm1",
+      "model.visual.blocks.21.norm2",
+      "model.visual.blocks.22.attn.k_norm",
+      "model.visual.blocks.22.attn.proj",
+      "model.visual.blocks.22.attn.q_norm",
+      "model.visual.blocks.22.attn.qkv",
+      "model.visual.blocks.22.mlp.down_proj",
+      "model.visual.blocks.22.mlp.gate_proj",
+      "model.visual.blocks.22.mlp.up_proj",
+      "model.visual.blocks.22.norm1",
+      "model.visual.blocks.22.norm2",
+      "model.visual.blocks.23.attn.k_norm",
+      "model.visual.blocks.23.attn.proj",
+      "model.visual.blocks.23.attn.q_norm",
+      "model.visual.blocks.23.attn.qkv",
+      "model.visual.blocks.23.mlp.down_proj",
+      "model.visual.blocks.23.mlp.gate_proj",
+      "model.visual.blocks.23.mlp.up_proj",
+      "model.visual.blocks.23.norm1",
+      "model.visual.blocks.23.norm2",
+      "model.visual.blocks.3.attn.k_norm",
+      "model.visual.blocks.3.attn.proj",
+      "model.visual.blocks.3.attn.q_norm",
+      "model.visual.blocks.3.attn.qkv",
+      "model.visual.blocks.3.mlp.down_proj",
+      "model.visual.blocks.3.mlp.gate_proj",
+      "model.visual.blocks.3.mlp.up_proj",
+      "model.visual.blocks.3.norm1",
+      "model.visual.blocks.3.norm2",
+      "model.visual.blocks.4.attn.k_norm",
+      "model.visual.blocks.4.attn.proj",
+      "model.visual.blocks.4.attn.q_norm",
+      "model.visual.blocks.4.attn.qkv",
+      "model.visual.blocks.4.mlp.down_proj",
+      "model.visual.blocks.4.mlp.gate_proj",
+      "model.visual.blocks.4.mlp.up_proj",
+      "model.visual.blocks.4.norm1",
+      "model.visual.blocks.4.norm2",
+      "model.visual.blocks.5.attn.k_norm",
+      "model.visual.blocks.5.attn.proj",
+      "model.visual.blocks.5.attn.q_norm",
+      "model.visual.blocks.5.attn.qkv",
+      "model.visual.blocks.5.mlp.down_proj",
+      "model.visual.blocks.5.mlp.gate_proj",
+      "model.visual.blocks.5.mlp.up_proj",
+      "model.visual.blocks.5.norm1",
+      "model.visual.blocks.5.norm2",
+      "model.visual.blocks.6.attn.k_norm",
+      "model.visual.blocks.6.attn.proj",
+      "model.visual.blocks.6.attn.q_norm",
+      "model.visual.blocks.6.attn.qkv",
+      "model.visual.blocks.6.mlp.down_proj",
+      "model.visual.blocks.6.mlp.gate_proj",
+      "model.visual.blocks.6.mlp.up_proj",
+      "model.visual.blocks.6.norm1",
+      "model.visual.blocks.6.norm2",
+      "model.visual.blocks.7.attn.k_norm",
+      "model.visual.blocks.7.attn.proj",
+      "model.visual.blocks.7.attn.q_norm",
+      "model.visual.blocks.7.attn.qkv",
+      "model.visual.blocks.7.mlp.down_proj",
+      "model.visual.blocks.7.mlp.gate_proj",
+      "model.visual.blocks.7.mlp.up_proj",
+      "model.visual.blocks.7.norm1",
+      "model.visual.blocks.7.norm2",
+      "model.visual.blocks.8.attn.k_norm",
+      "model.visual.blocks.8.attn.proj",
+      "model.visual.blocks.8.attn.q_norm",
+      "model.visual.blocks.8.attn.qkv",
+      "model.visual.blocks.8.mlp.down_proj",
+      "model.visual.blocks.8.mlp.gate_proj",
+      "model.visual.blocks.8.mlp.up_proj",
+      "model.visual.blocks.8.norm1",
+      "model.visual.blocks.8.norm2",
+      "model.visual.blocks.9.attn.k_norm",
+      "model.visual.blocks.9.attn.proj",
+      "model.visual.blocks.9.attn.q_norm",
+      "model.visual.blocks.9.attn.qkv",
+      "model.visual.blocks.9.mlp.down_proj",
+      "model.visual.blocks.9.mlp.gate_proj",
+      "model.visual.blocks.9.mlp.up_proj",
+      "model.visual.blocks.9.norm1",
+      "model.visual.blocks.9.norm2",
+      "model.visual.downsample",
+      "model.visual.merger.down_proj",
+      "model.visual.merger.gate_proj",
+      "model.visual.merger.post_projection_norm",
+      "model.visual.merger.proj",
+      "model.visual.merger.up_proj",
+      "model.visual.patch_embed.proj",
+      "model.visual.post_layernorm"
+    ],
+    "quantization_status": "compressed",
+    "kv_cache_scheme": null,
+    "version": "0.18.0"
+  }
+}

generation_config.json

@@ -0,0 +1,12 @@
+{
+  "_from_model_config": true,
+  "eos_token_id": [
+    154820,
+    154827,
+    154829
+  ],
+  "pad_token_id": 154820,
+  "temperature": 1.0,
+  "top_p": 0.95,
+  "transformers_version": "5.16.0"
+}

processor_config.json

@@ -0,0 +1,44 @@
+{
+  "image_processor": {
+    "do_rescale": true,
+    "patch_expand_factor": 1,
+    "merge_size": 2,
+    "image_mean": [
+      0.48145466,
+      0.4578275,
+      0.40821073
+    ],
+    "image_std": [
+      0.26862954,
+      0.26130258,
+      0.27577711
+    ],
+    "temporal_patch_size": 2,
+    "patch_size": 14,
+    "min_image_tokens": 16,
+    "max_image_tokens": 8000,
+    "image_processor_type": "Glm5NextImageProcessor"
+  },
+  "video_processor": {
+    "do_rescale": true,
+    "video_processor_type": "Glm5NextVideoProcessor",
+    "patch_expand_factor": 1,
+    "merge_size": 2,
+    "image_mean": [
+      0.48145466,
+      0.4578275,
+      0.40821073
+    ],
+    "image_std": [
+      0.26862954,
+      0.26130258,
+      0.27577711
+    ],
+    "temporal_patch_size": 2,
+    "patch_size": 14,
+    "min_image_tokens": 16,
+    "max_image_tokens": 240000,
+    "fps": 2
+  },
+  "processor_class": "Glm5NextProcessor"
+}

shard-00000.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0dbe913f8fc7a3ff879d6e4deeee20405b233c7c59cdba42757496a98263df6b
+size 2537554136

shard-00001.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f5aecf0ec1a7d4028a9c1624eb0ae358179938a2085fba8b3f37638c9446ae2
+size 956668904

shard-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d6805f2ae09e3bde357b386addedb7797e8db5872362debff2b50d018886aa3e
+size 953363820

shard-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:58428665928b911aa27524d3814ad7de62a41665072fb1d253344d9c9b8409b3
+size 1205798720

shard-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:802b97b1fe5c4985fe77420cfebb6e2d3521bbc736475f7d564d5326e6c363ad
+size 1024418492

shard-00005.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ab4a440000913d4b6729b1628b57b38f6fd3d16c1d3d2ad94fb996dc595d9a56
+size 835265316

shard-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc5bf8e6c4ea052e8d43e768e867583c5bf02b693fa96144c4e63150aed2de8d
+size 1205798872

shard-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:58a18da5f3b338b26bb995128fa0346c3956eb51920ae92750a72c81f44b388c
+size 979285608

shard-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:33bcd5fb94ad6258ba466bdff7d2937dbc4bdfa6fa4ffba7280d4b53d15012ab
+size 949177604

shard-00009.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:89cee9385c45a9696c929ee6cc7c53281c2ba3d9690b54f88b97141651d7674e
+size 1077729596

shard-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:20f89275650cd07cdf36b027698b93d28bbae9b41932fe19bb7873b60f43b509
+size 1165989132

shard-00011.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:59a11fe7cc414c1d7558ea8086d89eb27addb99dbfdbf4f0b7818897be05b336
+size 1177484352

shard-00012.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5150bc4589edf5cb69de253e55729547ce3010cf5b7919779d5eac40ac81d47f
+size 707851936

shard-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f2877dee40f5000aa17a3ed5bcfbd98a3ed4499f5a665c1cc18d343af3680100
+size 1191641796

shard-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b8b692d045c93dfece60bfe3b2016b4c6387c7339c2c53ee9a1ba15eda308aa
+size 1078385020

shard-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cf8c006a206645bddd423a2ef13923cb3ca77d2edfd29ff82b5f2887e9d9e12a
+size 835265332

shard-00016.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a129b015c873ce036c3fd1b1cd3cf3844d155153f6e5a336046d641bb7c96151
+size 1546854684

shard-00017.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4b43f8e6b2b721d333ac2e4174c0df04e4f2f7a5dcce1710fd00abd8ecc7a576
+size 1205798624

shard-00018.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:32891999c8b0a40f7fedf4b15fd4d3f4fde8332ff7c769058c18870597ea9665
+size 1149170136

shard-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf68546501900b4fb3d5315fbae66e47f5f3e6f5df0ecdc8c3b4fd3cfa502b06
+size 863579420

shard-00020.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a04f9353b81047678127f8287e29a97de369b30da12f089b03252d23008f69ef
+size 1135013548

shard-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:39a04bfff6b9ceeef8ccac47cf836951de5bd269dd0f5ccea5d614108ba66208
+size 1024418500

shard-00022.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c6734f9985cd13c566a6d1609c174d502936d7305af91cba643fade61445961
+size 906050536

shard-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc3e1ee32ea731dfcf922a7bdd8a22cee2013bf30fa6fe068e003d909b4a7c99
+size 1120856632

shard-00024.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3a91487b7e4b6ea8c11679bdbf7f8dfbb201c1fbe7528ae015ab8595e00a7704
+size 1149170472

shard-00025.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9be9cc451ab910f296cc3c9063de734f95f2b42ef563383939c929087f2f9a03
+size 1078384844

shard-00026.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:659fd210989bed9c151722db21728c6990d02030151e2307690eae5048574d42
+size 792794208

shard-00027.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:33dca7cbe8d02408a09791103a2ceab58f6f09399bb5a00c9dca86655957f47c
+size 1137675020

shard-00028.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ea03b6edf04d2b00d543d7b4cc35caa01638b8ecfdb0d74c20b41d0527521710
+size 1149170296

shard-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2341c4cce7dd66baeb37711109a1257c0f0e23c912d68a803e79f40bbc07cbdd
+size 806951236

shard-00030.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa4f75cd4354d757bbc0db2b1795cb00137b2d82da65991b5a0a6f3139fd8a57
+size 1050070860

shard-00031.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:05528d561c449d572c27dd0227457893ddefe5e304885f83add7f043b95e63df
+size 1236774072

shard-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5da0b87997d06d1ebdcfbb7fafed307fa39d1ebf943853c8c94b216b54083a00
+size 1050070740

shard-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b809dd7be96238140513b07a57077a1401642ae757d7740f90c6dd454a3728ed
+size 806951228

shard-00034.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:097ad67f6e7e1b904d97c193470d58be2315f25dd0a9af3dfa69c2f02eef7f52
+size 1137675028

shard-00035.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf7a737c4c1999edc3c06f1161fd8788fcfd1803182b7447c3492785b7585d90
+size 1078384924

shard-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:01414582ab03497bf21751e7f7af3840cbf38896b2453767a19952770d672ee3
+size 920207588

shard-00037.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c9fd118be4e4c5d3dde9471b7c280498d4b6aeeea511a3aecd97ac0d33d93d4c
+size 1120856648

shard-00038.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9292cb66c3879843a2eb129704b5f0592b5c029c1b9efa56d02ce8ea992a691c
+size 1106699252

shard-00039.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c98c854aee9a66a44ad71f2c4ff276e52b2bb662e930661eef07b148793cfdc0
+size 1066889608

shard-00040.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5d912377db95c392d3a4fbba2099148c084137159e0fbbef11b610a1a69dd615
+size 906050544

shard-00041.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c4ff88274b4260eef7df3892159b86702e6b021250c1cca2b3099ddbb2db834
+size 950971712

shard-00042.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:de997da1aa0f4d7168142ba9d40a3db0e312c27f9901d89dec20161c8d9b2e70
+size 1234112560